DSN enqueue failure retires original queued message, causing silent mail loss

Issues
1

Issue Description

When a recipient returns a permanent delivery failure, Stalwart generates a delivery status notification (DSN) for the original sender.

If storing the DSN blob fails, Stalwart removes the original queued message even though the DSN was not durably queued. After storage recovers and Stalwart restarts, the queue remains empty.

The original SMTP submission was accepted, the message was not delivered, and the sender never receives a failure notification.

This was reproduced deterministically using an isolated Docker fixture. Healthy controls immediately before and after fault injection successfully generated and delivered DSNs with the same sender, recipients, routing, and rejection behavior.

Environment:

  • Stalwart v0.16.21
  • Source commit: e96200385781a6a9995a8b839ac27d6c75a983ee
  • Official Docker image:
    stalwartlabs/stalwart@sha256:93c574e52249c1ebf90061da2c4c0756a7b72abfcc1fec34506a03c2e38b5977
  • Linux ARM64
  • PostgreSQL 17 used as the data and blob store
  • Internal directory
  • No Enterprise license
  • Verified-TLS LMTP delivery route

Expected Behavior

Stalwart must retain durable ownership of either:

  1. the original queued message and its pending failure-notification work; or
  2. a successfully queued DSN.

If queuing the DSN fails, Stalwart should retain or retry the original queue work. It must not mark the DSN as completed or retire the original message before the DSN is durably accepted.

Recovery should also handle a crash between successfully queuing the DSN and retiring the original message without losing the notification or generating uncontrolled duplicate DSNs.

Actual Behavior

Stalwart logs a blob-write failure while generating the DSN, but then removes the original queued message.

QueuedMessage/get returns an empty queue. After removing the injected storage fault and restarting Stalwart, the queue remains empty and no DSN is delivered.

The accepted original message is therefore neither delivered nor followed by a durable failure notification.

Reproduction Steps

  1. Configure:

    • one registered envelope sender;
    • two registered recipients;
    • a verified-TLS LMTP delivery route;
    • PostgreSQL as both the data and blob store;
    • a one-second retry interval.

  2. Submit a message using a nonempty envelope sender.

  3. Have the controlled LMTP peer read DATA and disconnect before sending the final recipient responses.

  4. Confirm through QueuedMessage/get that the accepted original message remains queued.

  5. Install the following fault injector in the disposable Stalwart PostgreSQL database. It rejects new blob writes while leaving existing blob reads and queue-metadata operations available:

CREATE FUNCTION proof_reject_blob() RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN
RAISE EXCEPTION ‘proof: DSN blob write unavailable’;
END $$;

CREATE TRIGGER proof_reject_blob
BEFORE INSERT OR UPDATE ON t
FOR EACH ROW EXECUTE FUNCTION proof_reject_blob();

  1. Change the LMTP peer to return permanent 550 responses for the original recipients.

  2. Observe Stalwart logging Failed to write blob and the injected PostgreSQL exception.

  3. Query QueuedMessage/get. The original queued message and the expected null-envelope DSN are both absent.

  4. Remove the trigger and function:

DROP TRIGGER proof_reject_blob ON t;
DROP FUNCTION proof_reject_blob();

  1. Restart Stalwart.

  2. Query the queue repeatedly for five seconds, covering multiple configured retry intervals. The queue remains empty and no DSN is delivered.

  3. Repeat without fault injection. The healthy control replaces the original queued message with a visible null-envelope DSN and successfully delivers it.

Relevant Log Output

PASS: dsn-control-before: original replaced by visible queued DSN; DSN delivered
REPRODUCED: injected blob error observed; original and DSN absent from queue
PASS: dsn-control-after: original replaced by visible queued DSN; DSN delivered
PASS: storage/restart recovery delivers new DSNs; failed DSN remains absent
VALIDATED: pinned-release DSN loss under injected blob-write failure; release gate remains failed

Stalwart Version

v0.16.x

Installation Method

Docker

Database Backend

PostgreSQL

Blob Storage

PostgreSQL

Search Engine

Internal

Directory Backend

Internal

I have reviewed the documentation and FAQ and confirm that my issue is NOT addressed there.

on

I have searched this support forum (open and closed topics) and confirm this is not a duplicate.

on

I understand that topics in this category are triaged by a bot first but a human reply will follow up. If I’d prefer a human-only reply, I’ll add the no-ai tag to my topic.

on

2

Thanks for the detailed, reproducible report. I looked at the relevant code path and can confirm the sequence you describe is consistent with what the source does today:

  • SendDsn::send_dsn (crates/smtp/src/queue/dsn.rs) calls message.build_dsn(), which sets the RCPT_DSN_SENT flag on each recipient as part of building the DSN body, before the DSN message is actually queued.
  • It then calls dsn_message.queue(...).await to persist the DSN (which is where the blob write happens), but the boolean success/failure return value of queue() is discarded — there is no branch that reverts the RCPT_DSN_SENT flag or otherwise retries if the blob write fails.
  • Back in deliver_task (crates/smtp/src/outbound/delivery.rs), once all recipients are marked as handled (RCPT_DSN_SENT/no pending delivery), the code proceeds to message.remove(...), retiring the original queued message unconditionally, independent of whether the DSN blob write actually succeeded.

That matches your observation: a failed put_blob for the DSN doesn’t stop the recipient flags from being marked “handled,” and the original message then gets retired with no durably queued DSN and no way to recover on restart.

I don’t have the authority to commit to a fix or timeline here, so this needs a human engineer to confirm intended behavior and design the correct failure/retry handling (e.g. checking queue()'s return value before marking recipients as DSN-sent, or making DSN queuing transactional with retiring the original message). Flagging this for staff follow-up.

This is an automated reply from the Stalwart Help Bot. Other community members may follow up if this answer is incomplete or wrong.

3

Fixed in v0.16.23 (yet to be released).