IMAP: Fetch does not list moved emails until I list the origin folder

Issues
1

Issue Description

Messages moved from another folder (Spam) to Inbox are not visible to IMAP connections until that IMAP connection opened the source folder (Spam). IMAP commands designed to list the latest messages are not returning the message. The same commands issued on the same folder later work, but only after the source folder was opened on the same IMAP connection.

Expected Behavior

  1. The email appears at the same time in the same folder in JMAP and in IMAP
  2. The email appears at the same time in the same folder in all IMAP connections
  3. If the client issues a command, it always gives the correct result with the latest messages, including messages moved into the folder.
  4. The command result for Inbox does not depend on having opened the source folder first.
  5. The server takes initiative and notifies the client of the new email, if the client is doing IDLE on the target folder (as mine was) (PUSH mail)

Actual Behavior

  • JMAP sees the message move immediately
  • IMAP does not get a notification during IDLE on the target folder
  • IMAP does not see the message after opening another folder that is not the source folder
  • IMAP sees the message only after having opened the source folder.

Reproduction Steps

  1. Have the same account set up 3 times: Once as JMAP and twice IMAP acccount, with the same account and credentials.
  2. Send yourself an email.
  3. Stalwart classifies it as spam and moves it into the Spam folder. (No idea why. In my case, the body is empty apart from the signature.)
  4. → The JMAP account sees the email immediately, in the spam folder.
  5. In the JMAP account, go to the spam folder, and move the message back to the Inbox
  6. → The JMAP account immediately sees the message in the inbox
  7. Open the inbox of the first IMAP account.
    The client does FETCH {uid: '13:*'} =
UID FETCH 13:* (UID ... MODSEQ)

and
FETCH {all: true} {changedSince: 47n} =

UID FETCH 1:* (UID ... MODSEQ) (CHANGEDSINCE 47)
  1. → The message is not returned by the server (bug)
  2. Click on Sent mail
  3. Click on Inbox again.
  4. → The message is not there (bug)
  5. Click on the Spam folder where the message was (but no longer is)
  6. → Older spam is shown, but not the moved message (correct)
  7. Click on Inbox again.
    The client makes the same (!) commands as in step 7.
  8. → The message is returned. (correct)
* 14 FETCH (UID 14 FLAGS ($Junk \Seen) ENVELOPE ... EMAILID (iaaaaap) THREADID (iaaaaap) MODSEQ (50))
  1. Repeat steps 7. to 15. with the second IMAP account. They same happens there. The problem seems to be per-connection.

Here’s a later client-server exchange (different modseq numbers from above):
Step 8:

Client: UID FETCH 1:* (UID ... MODSEQ) (CHANGEDSINCE 50)
Server: * OK [HIGHESTMODSEQ 60] Highest Modseq

Step 12: Clicking on Spam folder
Step 15: Same client IMAP command as above, but different response, this time with the message:

Client: UID FETCH 1:* (UID … MODSEQ) (CHANGEDSINCE 50)
Server: * 15 FETCH (UID 15 FLAGS ($Junk \Seen) ENVELOPE … EMAILID (iaaaaas) THREADID (iaaaaas) MODSEQ (59))
Server: * OK [HIGHESTMODSEQ 60] Highest Modseq



### Relevant Log Output
Stalwart 0.16.5
See IMAP logs in the reproduction

### Stalwart Version
v0.16.x

### Installation Method
Docker

### Database Backend
RocksDB

### Blob Storage
RocksDB

### Search Engine
Internal

### Directory Backend
Internal



### I have reviewed the documentation and FAQ and confirm that my issue is NOT addressed there.
on

### I have searched this support forum (open and closed topics) and confirm this is not a duplicate.
on
2

Fixed.

3

Thanks. Commit: Fix IMAP: `UID FETCH N:*` could miss messages moved into a SELECTed m… · stalwartlabs/stalwart@7704248 · GitHub