Email/query returns position: 0 for anchored queries → clients stuck on the first page (20 messages)

Issues
1

Issue Description

When a JMAP client pages through a mailbox using an anchored Email/query (the standard
anchor / anchorOffset mechanism), Stalwart returns the correct ids for the next page
but reports "position": 0 instead of the anchor’s real index in the result set.

Because the second page claims to start at position 0 while the client already holds positions
0–19, a spec-compliant client treats it as an inconsistent/overlapping page, discards it, and
re-fetches page 1 — looping forever. The practical result is that any folder with more than
one page is permanently stuck showing only the first 20 messages.

This is not a configuration problem: the ids returned are correct, only the position
field is wrong, and it is a direct deviation from RFC 8620 §5.5. It reproduces on a folder of
just 37 messages / 33 threads, so it is unrelated to mailbox size or data import.

Observed with the ltt.rs Android client (jmap-client/0.9.1), collapseThreads: true,
sorted by receivedAt descending, page size 20.

Expected Behavior

Per RFC 8620 §5.5 (/query):

  • position (response) is “the 0-based index of the first result in the ids array within
    the complete list of query results.”
  • When anchor is supplied, the server locates the anchor in the result list and returns
    results from anchorIndex + anchorOffset, with position set to that index. If the anchor
    is not found, the method must fail with anchorNotFound.

For an anchor that sits at index 19 with anchorOffset: 1, the response position must be
20.

Reference: RFC 8620: The JSON Meta Application Protocol (JMAP) | RFC Editor

Actual Behavior

The anchored Email/query returns the correct next-page ids, but position is 0 instead
of 20, and no anchorNotFound error is raised. The deviation is exactly the position
field: expected 20, actual 0.

A second manifestation: when the anchor is the last item (no further results), Stalwart returns
{"position": 0, "ids": []} instead of {"position": <count>, "ids": []}.

Reproduction Steps

Mailbox with 37 emails / 33 threads (Mailbox/get → totalEmails: 37, totalThreads: 33).

  1. Request the first page (no anchor):
["Email/query", {
  "accountId": "e",
  "filter": { "inMailbox": "bc" },
  "sort": [ { "property": "receivedAt", "isAscending": false } ],
  "collapseThreads": true,
  "limit": 20
}, "9"]

Response is correct: position: 0 with 20 ids. The 20th (last) id is oaaaaadq.

  1. Request the next page, anchored on that last id:
["Email/query", {
  "accountId": "e",
  "filter": { "inMailbox": "bc" },
  "sort": [ { "property": "receivedAt", "isAscending": false } ],
  "collapseThreads": true,
  "anchor": "oaaaaadq",
  "anchorOffset": 1,
  "limit": 20
}, "11"]
  1. Observe the response: the ids are the correct remaining 13 thread-heads (threads 21–33),
    but position is 0 instead of 20. See the response in Relevant Log Output below.

Relevant Log Output

Server response to the anchored request from step 2 (only position is wrong):

["Email/query", {
  "accountId": "e",
  "queryState": "srctao",
  "canCalculateChanges": true,
  "position": 0,
  "ids": ["vyaaaafo","nmaaaadl","wuaaaafv","e2aaaabh","saaaaaeq","naaaaadi",
          "nyaaaado","bqaaaaam","wmaaaaft","qeaaaaeb","xyaaaaf1","iaaaaac","iqaaaace"],
  "limit": 20
}, "11"]

Resulting client-side error (ltt.rs), which forces a cache reset and an endless reload of
page 1:

rs.ltt.jmap.mua.cache.exception.CorruptCacheException:
  Unexpected QueryPage. Cache ends with position 19. Page starts at position 0

Stalwart Version

v0.16.x

Installation Method

Docker

Database Backend

RocksDB

Blob Storage

RocksDB

Search Engine

Internal

Directory Backend

Internal

Additional Context

  • Client: ltt.rs (Android), jmap-client/0.9.1. Capabilities in use:
    urn:ietf:params:jmap:core, urn:ietf:params:jmap:mail.
  • Reverse proxy: nginx in front of the JMAP endpoint. It does not alter the JSON bodies; the
    request/response above are reproduced verbatim from the client-side trace.
  • Possibly related: Discussion #2873 reports a different wrong field (total returns the raw
    email count instead of the thread count with collapseThreads) and locates the cause in
    crates/jmap/src/email/query.rs, in the accounting around the thread-collapse loop — the
    same area where position is computed. The two may share a root cause in post-collapse
    index/count accounting.

Suggested fix: when anchor is supplied to Email/query (and, by the shared /query
contract, the other */query methods), set the response position to
resolvedAnchorIndex + anchorOffset (clamped to ≥ 0) instead of 0; if the anchor id is not
present in the result set, return the anchorNotFound error.

I have reviewed the documentation and FAQ and confirm that my issue is NOT addressed there.

on

I have searched this support forum (open and closed topics) and confirm this is not a duplicate.

on

I understand that topics in this category are triaged by a bot first but a human reply will follow up. If I’d prefer a human-only reply, I’ll add the no-ai tag to my topic.

on

2

Let me look at the relevant source code to verify this report.

This is a confirmed bug. I’ve traced the exact code path. Here’s what’s happening:

Root cause: In QueryResponseBuilder::build() (crates/jmap/src/api/query.rs), when an anchor is provided and found, the position field in the response is only updated from its initial value of 0 via the negative-anchor_offset branch. For the standard positive/zero anchorOffset case (the most common, as in your example), self.position is never updated from its initial 0 when the anchor path is taken – so response.position stays 0.

Specifically, looking at build():

if !self.has_anchor && self.requested_position >= 0 {
    self.response.position = if self.position == 0 {
        self.requested_position
    } else {
        0
    };
} else if self.position >= 0 {
    self.response.position = self.position; // <-- self.position is still 0 for anchor case!
} else {
    // negative anchor_offset branch -- this one correctly calculates position
}

When anchor is used with anchorOffset >= 0, self.position is never set to anchorIndex + anchorOffset – it stays 0 from initialization. RFC 8620 §5.5 requires it to be set to the actual index of the first returned result.

Compounding factor with collapseThreads: The results.len() passed to QueryResponseBuilder::new() is the raw document count (pre-collapse), so total_results and therefore limit accounting is also off when thread-collapsing is active – which is consistent with the related discussion you referenced (#2873).

Suggested fix direction (for the maintainers): In add_id(), when the anchor is found with anchor_offset >= 0, track the running index of items seen and store it as anchor_index + anchor_offset. Then in build(), set response.position to that computed value. For the collapseThreads case, total_results should be computed after the collapse loop, not from the raw search result count.

This is a genuine spec compliance bug affecting all anchored */query calls when anchorOffset >= 0.

This is an automated reply from the Stalwart Help Bot. Other community members may follow up if this answer is incomplete or wrong.

3

This has been fixed as part of the JMAP conformance test fixes which will be included in v0.16.10.