MX lookup fails with "DNSSEC validation failed" for signed zone (mail.de, RSA/SHA-512) on 0.16.21, regardless of DANE setting

Issues
1

Issue Description

Note: domain names are written with " DOT " because of the new-user link limit.

Outbound delivery to mail DOT de fails at the MX lookup stage with “DNSSEC validation failed”, although the zone validates cleanly with dig and delv on the same host. The failure occurs with dane set to disable, so the DANE strategy has no effect on it. Messages stay queued until they expire.

The resolver in use is a custom one (UDP and TCP entries pointing at two public validating resolvers), not the Docker internal resolver. Other signed zones using algorithm 8 and algorithm 13 validate correctly against the same resolver. The failing zone signs with algorithm 10 (RSA/SHA-512) and currently publishes 5 DNSKEY records.

Expected Behavior

The MX lookup succeeds and delivery proceeds, the same way it does for posteo DOT de (algorithm 8) and mailbox DOT org (algorithm 13).

Actual Behavior

The MX lookup fails immediately on every attempt, the message is rescheduled, and the queue entry eventually expires. The elapsed time of 2 ms suggests the failure happens without a network round trip.

Reproduction Steps

  1. Configure a custom DNS resolver with udp and tcp entries, not the system or Docker resolver.
  2. Set dane to disable on the outbound TLS strategy.
  3. Restart the server so the setting is picked up.
  4. Send a message to any recipient at mail DOT de.

Relevant Log Output

INFO MX record lookup failed (delivery.mx-lookup-failed) queueId = 328571896014597120, queueName = “remote”, domain = “mail DOT de”, causedBy = DNS error (mail-auth.dns-error) { details = “DNS error: DNSSEC validation failed” }, elapsed = 2ms
INFO Message rescheduled for delivery (queue.rescheduled) queueId = 328571896014597120, queueName = “remote”, nextRetry = 2026-09-09T13:00:38Z, expires = 2026-09-12T12:01:29Z

External validation from the same host succeeds:

delv TLSA _25 DOT _tcp DOT mx02 DOT mail DOT de
; fully validated
TLSA 3 1 1 9730904A64883FB665C97071DC77C86515F0DEF54DA120EE33B3796939CD171E
TLSA 3 1 1 AAFECCBE29EDB69BBB13D78011697844BFACA8D30108A892377B0A14059DBD01
RRSIG TLSA 10 5 600 20260920004933 20260906004933 31896

The DNSKEY response is 1326 bytes and arrives over UDP without the tc flag, so truncation is not involved:

dig +dnssec +notcp DNSKEY mail DOT de
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1
;; MSG SIZE rcvd: 1326

Stalwart Version

v0.16.x

Installation Method

Docker

Database Backend

SQLite

Blob Storage

RocksDB

Search Engine

Internal

Directory Backend

Internal

Additional Context

Stalwart version: 0.16.21
Installation method: Docker
DNS resolver: custom, udp and tcp entries for two public validating resolvers

Comparison across German providers, all queried against the same resolver:

posteo DOT de mx01 DOT posteo DOT de algorithm 8 DANE verification passed
gmx DOT net mx00 DOT emig DOT gmx DOT net algorithm 8 not tested
web DOT de mx-ha02 DOT web DOT de algorithm 8 not tested
freenet DOT de mx DOT freenet DOT de algorithm 13 not tested
mailbox DOT org mx1 DOT mailbox DOT org algorithm 13 not tested
mail DOT de mx01 DOT mail DOT de algorithm 10 MX lookup fails

The troubleshooting tool reports the MX lookup as successful for this domain and only shows the error at the TLSA step. That does not match what the delivery path does and made the problem harder to locate.

Possibly related: with dane set to optional, the IP address lookup for hosts under olc DOT protection DOT outlook DOT com also failed with “DNSSEC validation failed”, which blocked delivery to all Exchange Online recipients. That stopped after setting dane to disable and restarting. The TLSA SERVFAIL handling fixed in 0.16.20 works as intended here, the log shows “TLSA record not DNSSEC signed” followed by successful delivery for those hosts.

I have reviewed the documentation and FAQ and confirm that my issue is NOT addressed there.

on

I have searched this support forum (open and closed topics) and confirm this is not a duplicate.

on

I understand that topics in this category are triaged by a bot first but a human reply will follow up. If I’d prefer a human-only reply, I’ll add the no-ai tag to my topic.

on

2

This looks like a genuine bug in the DNSSEC validation path rather than a documented configuration behavior, and I can’t find anything in the current docs or changelog that addresses RSA/SHA-512 (algorithm 10) zones specifically failing MX lookups while the same resolver validates other algorithms correctly. The 0.16.20 fix you referenced only covers TLSA SERVFAIL handling, not this MX-lookup DNSSEC failure.

This isn’t covered in the current docs. A community member or staff member will follow up.

This is an automated reply from the Stalwart Help Bot. Other community members may follow up if this answer is incomplete or wrong.

3

Make sure your DNS server is not truncating the responses, that’s usually the reason why DNSSEC fails. Also make sure you’re on the latest version as there was a Hickory bug related to this not long ago.

4

Same error here, and I have the version bisect that might save you a step.

0.16.20 is healthy, 0.16.21 is not. My container started on 0.16.21 at 07:09 UTC on
7 September and the first DNSSEC validation failed appeared at 07:18. The string does
not occur anywhere in my logs before that. I rolled back to 0.16.20 two days later,
changed nothing else, forced the queue to retry, and everything went out. Zero
occurrences since. So updating to the latest version will not help; latest is where this
starts.

Stalwart’s own DNS code is byte-identical between the two tags. I diffed
dane/dnssec.rs, outbound/lookup.rs, outbound/delivery.rs and
config/smtp/resolver.rs and they are unchanged. What moved is hickory-proto and
hickory-resolver, 0.26.1 to 0.26.2.

Diffing those two crates, five files changed under src/dnssec: crypto.rs, ds.rs,
nsec.rs, nsec3.rs and trust_anchor.rs. crypto.rs only lets algorithm 7 use the
same RSA/SHA-1 path as algorithm 5, so it does not explain algorithm 10. ds.rs looks
more relevant: the key tag is no longer computed as key_tag(public_key.public_bytes())
but via dnskey.calculate_key_tag(). That is the step matching a DS record to the right
DNSKEY, and a mismatch would reject a fully signed zone locally, with no network round
trip, which fits your 2 ms failure and your five published DNSKEYs.

Your Exchange Online case under olc.protection.outlook.com is the shape I am hit by,
and it is a different one. There the MX host sits in an unsigned zone that is itself a
delegated cut below another unsigned zone, so no signed denial of DS can exist. The
nsec.rs and nsec3.rs changes are the obvious suspects for that half. Both are leads,
not conclusions; I have not run the code.

@mdecimus, is pinning hickory-resolver 0.26.1, or a patched commit, an option for the
next patch release?

5

This was a hickory bug, Stalwart v0.16.22 will use the latest hickory release which fixes the issue.

6

I had the same issue. When downgrading to 0.16.20 the problem was solved.

With Stalwart v0.16.21, delivery repeatedly failed during the DNS lookup of the MX hostname with:

DNSSEC validation failed

The message was therefore kept in the remote queue and rescheduled for retry.

Stalwart configuration

  • Stalwart version: 0.16.21
  • Resolver type: System Resolver
  • Max attempts: 2
  • Concurrent requests: 2
  • Timeout: 5s
  • EDNS: enabled
  • Preserve intermediates: enabled
  • Try TCP on error: enabled

The system resolver is:

  • 1.1.1.1
  • 1.0.0.1

resolvectl status reports:

DNSSEC=no/unsupported

Relevant Stalwart log
2026-09-11T11:42:25Z INFO New delivery attempt for domain (delivery.domain-delivery-start) queueId = 328720235383643648, queueName = “remote”, from = “[email protected]”, to = [“[email protected]”], size = 10729, total = 1, domain = “recepient.com”

2026-09-11T11:42:25Z INFO MTA-STS policy not found (mta-sts.policy-not-found) queueId = 328720235383643648, queueName = “remote”, from = “[email protected]”, to = [“[email protected]”], size = 10729, total = 1, domain = “recepient.com”, strict = false, elapsed = 54ms

2026-09-11T11:42:26Z INFO IP address lookup failed (delivery.ip-lookup-failed) queueId = 328720235383643648, queueName = “remote”, from = “[email protected]”, to = [“[email protected]”], size = 10729, total = 1, domain = “recepient.com”, hostname = “recepient-com.mail.protection.outlook.com”, details = “Temporary Failure for recepient-com.mail.protection.outlook.com: Connection failed: lookup error: DNS resolution error: DNS error: DNSSEC validation failed”, elapsed = 204ms

2026-09-11T11:42:26Z INFO Message rescheduled for delivery (queue.rescheduled) queueId = 328720235383643648, queueName = “remote”, from = “[email protected]”, to = [“[email protected]”], size = 10729, total = 1, nextRetry = 2026-09-14T11:42:26Z, nextDsn = 2026-09-14T07:40:22Z, expires = 2026-09-13T07:40:22Z
DNS verification
I verified the DNS records independently using both Cloudflare resolvers.
For example:
dig +dnssec MX recepient.com @1.0.0.11.0.0.1
returns NOERROR with the ad flag and a valid MX record.
The A lookup for the MX hostname also succeeds:
dig +dnssec A recepient-com.mail.protection.outlook.com @1.0.0.1
and returns multiple Microsoft 365 IP addresses.
I also verified the DNSSEC chain:
• DS: present
• DNSKEY: present
• RRSIG: present
• MX: present and signed
• NS: present and signed
The same DNS results were obtained successfully via 1.1.1.1.
Important version comparison
I downgraded Stalwart from 0.16.21 to 0.16.20.
After the downgrade, delivery to the same recipient/domain succeeded.
The successful delivery log contains:
2026-09-11T12:28:36Z INFO TLSA record not DNSSEC signed (dane.tlsa-record-not-dnssec-signed) … strict = false

2026-09-11T12:28:36Z INFO Connecting to remote server (delivery.connect) … hostname = “recepient-com.mail.protection.outlook.com”, remoteIp = 52.101.185.114, remotePort = 25

2026-09-11T12:28:36Z INFO SMTP STARTTLS command (delivery.start-tls) … version = “TLSv1_3”, details = “TLS13_AES_256_GCM_SHA384”

2026-09-11T12:28:37Z INFO Message delivered (delivery.delivered) … code = 250

2026-09-11T12:28:37Z INFO DSN success notification (delivery.dsn-success) … code = 250, details = “Recipient OK”

2026-09-11T12:28:37Z INFO Delivery completed (delivery.completed)
Therefore the same destination and DNS configuration work correctly with v0.16.20.

7

Partially fixed in 0.16.22.

Confirmed first that 0.16.22 ships hickory 0.26.3 from crates.io,
while 0.16.20 pinned 0.26.1 from git (rev e645086).

What works now on 0.16.22:

Exchange Online - TLSA lookup returns SERVFAIL, delivery proceeds, OK
posteo DOT de (algorithm 8) - TLSA record found, DANE verification passed

What does not work:

mail DOT de (algorithm 10) - TLSA lookup fails with
“DNS resolution error: DNS error: DNSSEC validation failed”,
and delivery then proceeds WITHOUT DANE.

On 0.16.20 the same trace showed “TLSA record found” followed by
“DANE verification passed”, so this is a regression that 0.26.3
does not cover.

Two points worth flagging:

  1. This is now a silent DANE bypass rather than a failed delivery.
    The message is delivered over unauthenticated STARTTLS and nothing
    in the log indicates that DANE was skipped. Under RFC 7672 a DNSSEC
    failure on the TLSA lookup should defer delivery, not fall through.

  2. The zone has not changed since the working 0.16.20 test: still
    algorithm 10, still a single RRSIG over the TLSA set (key tag 31896).
    So the multi-RRSIG backport in 0.26.3 does not cover this case.
    The remaining suspect is either RSA/SHA-512 itself or the key tag
    computation mentioned earlier in this thread.

Externally the zone validates cleanly:

delv TLSA _25 DOT _tcp DOT mx01 DOT mail DOT de
; fully validated

8

Correction to my previous post: on 0.16.22 the failure is not limited to
the TLSA step. 19 hours after the upgrade the MX lookup itself is failing
again with “DNSSEC validation failed”, exactly as on 0.16.21, and the
message is rescheduled instead of delivered.

Right after the upgrade the MX lookup worked and only DANE was skipped.
The difference appears to be cache state: once the DNSSEC cache expires
and the chain is validated again, the MX lookup fails.

So 0.16.20 remains the only version where this domain works.

9

This is another hickory bug, added a workaround to limit concurrency to 1 in v0.16.23 (yet to be released).

10

Resolved in 0.16.23.

Running 0.16.23 for two days with dane set to optional. No occurrence of
“DNSSEC validation failed” in that period, and mail DOT de now passes DANE
in production, including well after the DNSSEC cache expired - which is
where 0.16.22 regressed.

The fix that matters here is likely the one listed as “the DNSSEC resolver
queries a single nameserver at a time”, since my setup uses two resolvers
and the affected zone has a DNSKEY response above the 1232 byte EDNS limit.

One remaining oddity: in the troubleshoot tool, every trace to an Exchange
Online host now ends with “Delivery failed” at the TLSA step, because
Microsoft answers TLSA queries with SERVFAIL. Real delivery to the same
host works - the log shows “TLSA record not DNSSEC signed” and the message
is delivered. The trace does not apply the RFC 7672 check that the delivery path does.

11

Good to hear mail.de passes DANE now.

The troubleshoot result is a bug in the tool. It asks for TLSA on every MX host up front and fails the host on any TLSA error. This has now been fixed in v0.16.24 (yet to be released).

12

Still see “DNSSEC validation failed” on 0.16.24 with DANE enabled, but this is different case than mail.de. Domain akokan.pl is unsigned and has no NS at zone apex, hickory picks wrong zone cut and returns Bogus.

Reported in hickory: DNSSEC Bogus for unsigned zone with no NS at apex · Issue #4027 · hickory-dns/hickory-dns · GitHub

Team, please bump hickory when it is fixed.

13

akokan.pl is a different hickory bug from the mail.de one. With no NS record at the apex, hickory settles on the wrong zone cut and reports the unsigned zone as Bogus, and with DANE enabled that defers the message, which is intentional. You’ve already filed it as hickory-dns#4027 and the fix (#4028) hasn’t been merged, so there’s nothing to pull in yet; I’ll bump hickory once it lands.

You can exempt just that domain. Create an MtaTlsStrategy called, for example, no-dane with dane set to disable, and put a rule ahead of the existing ones in the tls expression on MtaOutboundStrategy:

{"if": "rcpt_domain == 'akokan.pl'", "then": "'no-dane'"}

Since 0.16.24 a strategy with DANE disabled also does the MX lookup without DNSSEC validation, so mail to that domain goes out and everything else keeps DANE. Match on the recipient domain rather than the MX host: the expression is evaluated before the MX records are known. Field reference: MtaTlsStrategy | Stalwart

14

Hi everyone,

This is an update on the DNSSEC and DANE delivery failures reported across several threads here. They come from bugs in hickory, the DNS library Stalwart uses to validate DNSSEC, and each one now has a fix submitted upstream:

  • Google Workspace MX hosts (aspmx.l.google.com and the altN hosts) failing with “exceeded max validation depth”: hickory#3974, fixed in #3976 and #4007. Both are merged but not yet in a hickory release.
  • Domains delegated two or more labels below an NSEC3 opt-out zone, such as port.venice.it and the comune.*.fr.it municipality domains: hickory#3985, fix in #4001.
  • Unsigned zones without an NS record at their apex, for example akokan.pl: hickory#4027, fix in #4028.
  • Microsoft 365 hosts (*.mail.protection.outlook.com) failing when the upstream resolver answers SERVFAIL for them, which Google Public DNS and many local resolvers do: hickory#4063, fix in #4064.
  • Truncated UDP responses never retried over TCP when several name servers are queried at the same time: hickory#3963, fix in #4062.
  • Valid answers rejected because the response also carries an unrelated NSEC record, plus some correctly signed wildcard answers rejected: hickory#4047, fix in #4055.
  • Validating queries sent without the CD bit, which makes a validating upstream resolver answer SERVFAIL: hickory#3966, fix in #4014 and #4036.

DANE delivery has been broken for some of you for weeks now, and the hickory maintainers seem to be busy at the moment, so most of these PRs are still waiting for review. We have forked hickory at stalwartlabs/hickory-dns with all of the fixes above applied, and Stalwart will build against that fork until hickory publishes a release that contains all of them. We ran the patched resolver against every domain reported in these threads, with Cloudflare, Google and Quad9 as upstream resolvers, and they all validate correctly now.

v0.16.26 ships with the patched resolver and will be out today or tomorrow. After upgrading you can remove any TLS strategy you added to turn DANE off for specific domains. If a domain still fails after the upgrade, post it in this thread together with the DNS resolver you use and we’ll look into it.