Subaddressing information lost after delivery

Issue Description

If I send an email to foo+bar@baz with subaddressing enabled, the email gets correctly delivered to foo@baz. However the “bar” information is nowhere to be seen in the headers, which makes it impossible to write sieve scripts (e.g. file into “bar” folder).

Expected Behavior

The additional part of the subaddressing should be visible and exposed to sieve

Actual Behavior

The additional part is nowhere to be seen in email headers

Reproduction Steps

  1. Have a foo@baz email configured
  2. Enable subaddressing for “baz” domain
  3. Send an email to foo+bar@baz
  4. Log through IMAP as foo@baz, inspect email headers
  5. Search for “bar” and find no results

Stalwart Version

v0.16.x

Installation Method

Binary (Linux)

Database Backend

PostgreSQL

Blob Storage

PostgreSQL

Search Engine

Internal

Directory Backend

Internal

I have reviewed the documentation and FAQ and confirm that my issue is NOT addressed there.

on

I have searched this support forum (open and closed topics) and confirm this is not a duplicate.

on

I understand that topics in this category are triaged by a bot first but a human reply will follow up. If I’d prefer a human-only reply, I’ll add the no-ai tag to my topic.

on

See my post for details and a hacky solution.

In the end, did this in the user script until it’s fixed:

if header :regex "To" "(^|[, ]+)([^+]+)\+([^@]+)@([a-z\\.]+)" {
   set :upperfirst "folder" "${3}";
   fileinto :copy :create "Alias";
   fileinto :create "Alias/${folder}";
#   notify :message "Uncomment for debugging the variables/${2}/${1}/${0}/${3}" "mailto:[email protected]";
   stop;
}

This is a known behavior gap. When subaddressing is enabled, Stalwart strips the +detail part during the RCPT TO lookup to find the real mailbox, but the original full address from RCPT TO is available in envelope.to within a server-side Sieve script (not user scripts). The +detail part is not automatically added to any message header, so it is indeed invisible after delivery.

Here is how you can expose the detail for user-level filtering:

Option 1: Add a header using a server-side DATA-stage Sieve script

You can write a server-side Sieve script at the DATA stage that reads the original envelope.to (which still contains foo+bar@baz at that point), extracts the detail part, and adds a custom header such as X-Subaddress. User Sieve scripts can then test against that header.

For example:

require ["variables", "envelope", "editheader"];

if envelope :matches "to" "*+*@*" {
    set "detail" "${2}";
    addheader "X-Subaddress" "${detail}";
}

Assign this script to the script field of your DATA stage (via the WebUI under the inbound MTA > DATA stage settings).

Option 2: Use the Sieve subaddress extension in user scripts

Stalwart’s user Sieve interpreter supports the subaddress extension (RFC 5233), but the caveat documented in the variables reference is that user scripts only have access to the From and To envelope parts. If the envelope.to seen by the user script still carries the full foo+bar@baz address (i.e., the rewriting has not already stripped it), you can use:

require ["subaddress", "fileinto"];

if envelope :detail "to" "bar" {
    fileinto "bar";
}

Whether this works depends on what value envelope.to has at user-script execution time. Given the behavior you’ve observed, it may already be rewritten to foo@baz by then, which would make Option 1 (the header-injection approach) the more reliable path.

Docs referenced:

This is an automated reply from the Stalwart Help Bot. Other community members may follow up if this answer is incomplete or wrong. If you found it useful, mention @helpbot in any reply to ask follow-up questions and the bot will respond again.

None of the options you’ve written works (please refer to my thread with all the tests I’ve made to show the issues) and it’s a regression from 0.15.x where it worked flawlessly.

Adding an header in RCPT_TO stage is invisible in the user’s script (checked the message source in the client too).

Also, when changing the envelope.to in the RCPT_TO to subaddress, the initial To header is still checked for the account (not the modified one), so it’s not possible to rewrite the address here (so bad luck for free aliases or if you want to hide the real account).

envelope :detail does not work in user scripts, it never match (try with

if envelope :detail “to” “*” {
   notify :message "Subadressing ${0} / ${1} / ${2}" "mailto:youremail@domain";
}

And you will never receive the notified message. Also, you can’t do both regex and detail at the same time, so if you need to match an unknown value (like a free alias), you’re struck.

This is very informative. The code confirms what X-Ryl669 is reporting: when subaddressing resolves (line ~201-218), the original foo+bar@baz address is replaced with the rewritten foo@baz address in the session’s rcpt_to list. The original address is saved only in dsn_info (as rfc822;...), not preserved in the envelope for Sieve. Let me check if the DATA stage / user Sieve delivery has access to the original address at all.

This is very revealing. Looking at the source code directly:

When subaddressing resolves, rcpt_resolve() returns RcptResolution::Rewrite(address) with the stripped address (e.g. foo@baz). Then in rcpt.rs lines 200-217, the original address is popped and replaced with the new SessionAddress for foo@baz. The original foo+bar@baz address is lost from rcpt_to at that point.

When build_script_parameters() builds the Sieve envelope for the DATA stage (lines 106-111), it iterates over self.data.rcpt_to which by then only contains the rewritten address (foo@baz). So envelope.to in both server-side DATA scripts and user scripts sees only the stripped address – the +bar detail is gone by the time any Sieve script runs at DATA time.

This confirms that X-Ryl669’s findings are correct: this is a genuine code-level regression/bug. The sub-address detail is discarded from the in-session recipient list when RcptResolution::Rewrite fires, and there is no mechanism currently preserving it for later Sieve access.

The RCPT-stage server Sieve runs before rcpt_resolve() is called (see rcpt.rs lines ~108-135), so at that point envelope.to still has the original foo+bar@baz. However, adding a header at RCPT stage does not persist to the message (headers are message-level, added at DATA stage), so it is useless there.

The only currently working workaround is the regex approach on the To: message header that X-Ryl669 already shared – matching against the To header in the user Sieve script, since the To: header in the message body is not modified by Stalwart’s address resolution.

To be completely transparent: this appears to be a known bug/gap in current Stalwart. The sub-address detail (+bar) is not preserved after RCPT resolution rewrites the address, so it is unavailable to envelope :detail or envelope.to at delivery-time Sieve. The To: header workaround is the only viable path until this is addressed in the server code.

This is an automated reply from the Stalwart Help Bot. Other community members may follow up if this answer is incomplete or wrong. If you found it useful, mention @helpbot in any reply to ask follow-up questions and the bot will respond again.

v0.16.6 will support accessing the original rcpt before rewriting as follows:

envelope :detail "orcpt"

How is it different from checking the initial To header in User’s script ?

What about the issue that a rewrite in RCPT_TO script doesn’t work (the rewritten address isn’t used to check for the account existence, only the original rcpt address is)?

Tried with 0.16.7 and it doesn’t work:


# Either a catch all
if envelope :matches "orcpt" "*" {
   notify :message "XRCPT/${0}/${1}" "mailto:[email protected]";
   stop;
}


# Or a regex
if envelope :detail :regex "orcpt" "([^.]+)\+([^@]+)@(.*)$" {
   notify :message "XRCPTregex/${0}/${1}/${2}/${3}" "mailto:[email protected]";
   stop;
}



Do you have any example that’s working with subadressing to show us?

The orcpt support I mentioned did land in 0.16.6 and is in 0.16.7, so envelope :detail “orcpt” is the intended way to get at the detail part. One thing that may be tripping the match: the orcpt value is stored with an rfc822; prefix, so a bare :is or :matches against the whole value will not look like an address. If you use envelope :detail “orcpt” it should extract just the tag.

If even that returns nothing, then orcpt is not being populated for your delivery, which would be the real issue to chase. Can you run one delivery with tracing on and confirm whether the orcpt/dsn_info value is set at all on the delivered message? If you paste exactly the script you are testing (the if envelope :detail “orcpt” form, not a regex on To) and what it evaluates to, I can tell whether this is the rfc822; prefix getting in the way or orcpt genuinely being empty.

Wouldn’t the first test in my example code triggers unconditionally, whatever the format of orcpt?

Because it doesn’t, I’m not notified

I’ll try to enable the traces and let you know, thank you for your answer.

@mdecimus I’ve also noticed that according to the rfc 6009:

The envelope test’s ADDRESS-PART argument assumes the string being tested has the syntax of an email address. None of the new envelope parts defined here have address syntax; accordingly, it is an error to specify an ADDRESS-PART argument in conjunction with these new envelope parts.

For convenience, the “ADDRESS-PART” syntax element defined in
[RFC5228], Section 2.7.4, is augmented here as follows:

     ADDRESS-PART  =/  ":user" / ":detail"

So, I don’t think the envelope-dsn match is ideal, and doing a match like *+marketing@*" is hideous. It is very annoying that they’ve designed it like that.

The way I’ve done it is like this using editheader capability:

require ["variables", "fileinto", "envelope", "envelope-dsn", "subaddress", "mailbox", "editheader"];

deleteheader "X-Original-To";
if envelope :matches "orcpt" "rfc822;*" {
  addheader "X-Original-To" "${1}";
}

if address :localpart :is "X-Original-To" "postmaster" {
  fileinto :create "Reports";
  stop;
}

if address :matches :detail "X-Original-To" ["shrine", "marketing"] {
  set :lower :upperfirst "name" "${0}";
  fileinto :create "Inbox/${name}";
  stop;
}

It is possible to create a header and then refer to it in the same sieve script, though I’m not a fan.

There were really two problems tangled together in this thread: one that was already solved but badly documented, and one that was broken on our side.

The user script side has worked since v0.16.6. This does what you would expect, as long as the recipient was actually rewritten:

require ["envelope", "envelope-dsn", "subaddress", "fileinto", "variables"];

if envelope :matches "orcpt" "rfc822;*" {
    # ${1} is the original recipient, e.g. "[email protected]"
}

if envelope :matches :detail "orcpt" "*" {
    fileinto :create "Inbox/${1}";
}

Two things catch people out. First, :detail and :user need the subaddress extension and orcpt needs envelope-dsn; if either is missing from require, the script does not compile and nothing in it runs, which is exactly why an unconditional envelope :matches "orcpt" "*" looks like it never fires. That is my best guess at what you were hitting in post #10, so it is worth checking whether the script validates at all. Second, the value keeps the address type prefix, so it reads rfc822;[email protected] rather than the bare address. Match it with "rfc822;*", or use :detail, which skips past the prefix.

orcpt is only set when the address was rewritten during RCPT TO: sub-addressing, catch-all, alias resolution, mailing list expansion, or an ORCPT= parameter from the sending server. If the address was delivered unchanged there is nothing to recover, because the full address is already in envelope.to. The script kanashimia posted in #12 is a good template, and stamping the value into X-Original-To so the address test can be used on it is a sensible pattern.

The system script side is where the issue was. The advice in post #3, to pull the detail out in a DATA stage script and add a header, could never have worked: by the time DATA runs, recipients have already been rewritten to their final address, and the original was not handed to the script at all. That is fixed for the next release. envelope.orcpt is now available at the DATA stage as an array that lines up index for index with envelope.to, so this works:

require ["envelope", "envelope-dsn", "editheader", "variables"];

if envelope :matches :detail "orcpt" "*" {
    addheader "X-Original-Detail" "${1}";
}

If a transaction has more than one recipient, use the expression form to correlate the two lists, since the envelope test flattens arrays into a single value:

let "original" "envelope.orcpt[0]";
let "final" "envelope.to[0]";

The array is only present when at least one recipient in the transaction was rewritten, matching the behaviour of user scripts.

One clarification on post #4: headers added from a RCPT TO stage script are discarded because there is no message at that point in the transaction. addheader and deleteheader only take effect in DATA stage scripts, and setenvelope is the only modification that applies at the MAIL FROM and RCPT TO stages.

On the RFC 6009 point, kanashimia is right. The spec says it is an error to combine an ADDRESS-PART argument such as :detail with the DSN envelope parts, since they do not have address syntax. Stalwart accepts it anyway and applies the address parsing after the type prefix, because it is by far the most convenient way to reach the detail. That is a deliberate deviation, and it will be documented as one rather than removed, since scripts already depend on it.

A few related problems turned up while tracing this and are being handled separately: recipients rewritten through the RCPT TO rewrite expression, through setenvelope "to" in a system script, or through the queue expansion path do not record the original address, so orcpt stays empty in those setups even though a rewrite took place; the ORCPT value supplied by a remote server is xtext encoded and we do not decode it before handing it to Sieve, so a + in the original address shows up as +2B; and the Original-Recipient field in delivery status notifications repeats the rfc822; prefix. The Sieve variables documentation is also out of date, as it still says user scripts can only see From and To, which stopped being true in v0.16.6.