Initial CalDAV/CardDAV `sync-collection` is truncated at the max results limit, but the returned sync token says the sync is complete, so the remaining entries are never synced

Issues
1

Issue Description

I found this with my iPhone: my address book has more than 3000 contacts, but the iPhone only synced 1999 of them. 2000 is the default value of the maximum number of results per DAV response (WebDAV setting maxResults). Any CalDAV/CardDAV collection with more entries than that default is affected out of the box.

When a client does its first sync-collection REPORT (empty <D:sync-token/>) on a calendar or address book with more entries than the configured maximum results per DAV response, Stalwart:

  1. returns only the first N entries,
  2. adds a 507 Insufficient Storage / <D:number-of-matches-within-limits/> response saying the result was truncated,
  3. but returns the sync token for the current state of the collection.

The client then asks for the rest by sending that token, as RFC 6578 §3.6 describes. Stalwart answers that there are no changes. The entries after N are never delivered, and the client’s copy stays incomplete for good. It only fills in later when entries are changed, because those changes then show up in the changelog.

Paging does work for incremental syncs: when a client sends an existing token and there are more than N changes, Stalwart returns a paging token for the next batch. The initial sync has no such paging.

The only workaround is to set the WebDAV maxResults setting above the size of the largest calendar or address book.

Expected Behavior

All entries of the calendar or address book are synced to the client, not just the first 2000 or maxResults.

If the server has to split the initial sync-collection response because of maxResults, the returned sync token must let the client fetch the remaining entries (RFC 6578 §3.6). Repeating the request with each new token must eventually return every member of the collection, followed by a final token with no 507.

Actual Behavior

With the default WebDAV maxResults of 2000 and an address book with more than 3000 contacts:

  • Initial sync (sync-collection with an empty <D:sync-token/>): the response is cut off at the limit and contains:

    <D:response>
  <D:href>/dav/card/<account>/default/</D:href>
  <D:status>HTTP/1.1 507 Insufficient Storage</D:status>
  <D:error><D:number-of-matches-within-limits/></D:error>
  <D:responsedescription>The number of matches exceeds the limit of 2000</D:responsedescription>
</D:response>

    It also contains the sync token for the current state of the address book.

  • Follow-up (sync-collection with that token): no further entries, no 507, and the same token again.

The iPhone ends up with 1999 of the more than 3000 contacts and considers itself fully in sync. The remaining contacts are never delivered.

Reproduction Steps

To keep the test small, lower the limit to 10 instead of creating more than 2000 entries:

  1. Set the WebDAV maxResults setting to 10.
  2. Create 15 contacts in an address book (or 15 events in a calendar).
  3. Initial sync:
curl -s -u [email protected]:PASSWORD -X REPORT -H 'Depth: 1' \
  -H 'Content-Type: application/xml' \
  --data '<?xml version="1.0"?>
<D:sync-collection xmlns:D="DAV:">
  <D:sync-token/>
  <D:sync-level>1</D:sync-level>
  <D:prop><D:getetag/></D:prop>
</D:sync-collection>' \
  https://mail.example.com/dav/card/user%40example.com/default/

You get 10 contacts, the 507 number-of-matches-within-limits response, and a sync token.

  1. Repeat with the returned token in <D:sync-token>…</D:sync-token>. The response has no entries and the same token. The other 5 contacts are never returned.
  2. Raise maxResults above 15 and repeat step 3. All 15 contacts are returned.

Stalwart Version

v0.16.x

Installation Method

Docker

Database Backend

RocksDB

Blob Storage

RocksDB

Search Engine

Internal

Directory Backend

Internal

Additional Context

  • Related: GitHub discussion #2897, “Apple Calendar Silently Fails synchronisation on more than 2000 events (default WebDAV Value)”. It reports the same symptom for CalDAV, is still open, and has no maintainer reply. It assumes Apple Calendar doesn’t follow the returned sync token. The reproduction above shows that following the token doesn’t help either: the server returns no further entries. So this is a server-side problem that affects any client, CalDAV and CardDAV alike.

  • If needed, I can provide a Claude generated Python script to reproduce the issue. Does just execute automatically the above mentioned steps.

I have reviewed the documentation and FAQ and confirm that my issue is NOT addressed there.

on

I have searched this support forum (open and closed topics) and confirm this is not a duplicate.

on

I understand that topics in this category are triaged by a bot first but a human reply will follow up. If I’d prefer a human-only reply, I’ll add the no-ai tag to my topic.

on

2

Thanks for the detailed writeup and reproduction steps. I looked at the WebDAV propfind/sync-collection handling in the server code and can confirm the mechanics you describe: when the number of items returned by a sync-collection REPORT exceeds maxResults, the server sets a flag (is_sync_limited) and appends a 507 Insufficient Storage / number-of-matches-within-limits error to the multistatus response, but the sync-token returned in that same response is generated from the current overall collection state, not from a checkpoint corresponding to where enumeration was truncated. So a client that dutifully retries with that token gets no further changes, exactly as you found with both CalDAV and CardDAV collections.

This does look like a real gap versus RFC 6578 §3.6 (a truncated initial sync should hand back a token the client can use to fetch the rest), and increasing maxResults (documented at /docs/collaboration/webdav#request-and-result-limits, default 2000) is indeed the only current workaround.

Since this is filed under Issues, a human maintainer will need to review and confirm whether/how this will be addressed; I can’t commit to a fix or timeline myself.

This is an automated reply from the Stalwart Help Bot. Other community members may follow up if this answer is incomplete or wrong.

3

This has been fixed in the v1.0.0 branch but unfortunately the fix cannot be backported to v0.16. v1.0.0 will be released at the end of this month or early November.