Bug: Outbound delivery fails to any MX host with an IDN (A-label) hostname

Issues
1

Issue Description

Stalwart cannot deliver to any domain whose MX exchange is an IDN hostname (xn--…). The MX hostname is converted from its A-label to its U-label before being used as the TLS server name, rustls rejects the non-ASCII name, and delivery aborts before the TLS ClientHello is sent.

The recipient domain does not matter — only the MX host. A plain-ASCII domain whose MX points at an IDN host fails identically.

Expected Behavior

The TLS server name is the A-label, the handshake completes, and the message is delivered.

Actual Behavior

The bounce reports a hostname that does not exist in DNS:

<[email protected]> (TLS error from 'mail.מייל.קום': Invalid hostname)
Final-Recipient: rfc822;[email protected]
Action: failed
Status: 5.0.0
Remote-MTA: dns;mail.מייל.קום

The MX record is mail.xn--eebajf.xn--9dbq2a and the certificate’s CN/SAN is mail.xn--eebajf.xn--9dbq2a. The U-label form mail.מייל.קום is never published anywhere.

On the receiving Postfix, the session ends immediately after the 220 response to STARTTLS, with no ClientHello:

connect from mail.<sender>[…]
SSL_accept error from mail.<sender>[…]: lost connection
NOQUEUE: lost connection after STARTTLS from mail.<sender>[…]
disconnect from mail.<sender>[…] ehlo=1 starttls=0/1 commands=1/2
starttls=0/1

= the handshake never begins, so the receiver never sees MAIL FROM or RCPT. Other senders (e.g. Gmail) deliver to the same host over TLS 1.3 without issue.

Reproduction Steps

Point a test domain’s MX at a hostname containing an IDN label, e.g.

example.test.  MX  10  mail.xn--eebajf.xn--9dbq2a.

Ensure that host serves STARTTLS with a certificate whose SAN is the A-label form.
Send a message to that domain from Stalwart.
I hit this sending to a Postfix server I run. Delivery fails every time with the bounce above.

Stalwart Version

v0.16.x

Installation Method

Docker

Database Backend

RocksDB

Blob Storage

RocksDB

Search Engine

Internal

Directory Backend

Internal

I have reviewed the documentation and FAQ and confirm that my issue is NOT addressed there.

on

I have searched this support forum (open and closed topics) and confirm this is not a duplicate.

on

I understand that topics in this category are triaged by a bot first but a human reply will follow up. If I’d prefer a human-only reply, I’ll add the no-ai tag to my topic.

on

2

Second, related issue: MTA-STS mode: testing is treated as strict

This determines whether the failure above is permanent or temporary, but is independent of it.

crates/smtp/src/outbound/delivery.rs:1109:


let is_strict_tls = tls_strategy.is_tls_required()

    || (message.message.flags & MAIL_REQUIRETLS) != 0

    || mta_sts_policy.is_some()

    || dane_policy.is_some();

The MTA-STS arm tests only for the presence of a policy, ignoring its mode. At

delivery.rs:1355 this decides the failure class:


last_status = if is_strict_tls {

    Status::from_tls_error(envelope.mx, error)

} else {

    Status::from_tls_error(envelope.mx, error).into_temporary()

};

So a domain publishing a policy with mode: testing gets a permanent 5.x.x bounce on any TLS error, while an otherwise identical domain with no policy gets a temporary failure and is retried.

RFC 8461 §5 specifies that in testing mode a failure to apply the policy must not affect delivery of the message; the intent is to report the failure via TLSRPT and deliver anyway.

The predicate already exists — Policy::enforce() (outbound/mta_sts/verify.rs:41, self.mode == Mode::Enforce) — and is used on the next line for is_mta_sts_enforced. It just isn’t applied to is_strict_tls.

3

Fixed in v0.16.18 (yet to be released).