perf: Email/query loads all message metadata into memory before sorting, causing OOM and high latency on large mailboxes

Issues
1

Issue Description

When a client calls Email/query, Stalwart loads all message metadata (sort fields such as receivedAt, size, subject, etc.) for the entire account into memory before applying sort and pagination. On large mailboxes (e.g. 100k+ messages), this results in significant memory pressure, high latency, and excessive internal data transfer — even when the client only needs the first page of 50 results.

Expected Behavior

For SQL-backed deployments, push the sort and pagination down to the database

Stalwart Version

v0.16.x

Installation Method

Docker

Database Backend

FoundationDB

Blob Storage

S3-compatible

Search Engine

Elasticsearch

Directory Backend

Internal

I have reviewed the documentation and FAQ and confirm that my issue is NOT addressed there.

on

I have searched this support forum (open and closed topics) and confirm this is not a duplicate.

on

I understand that topics in this category are triaged by a bot first but a human reply will follow up. If I’d prefer a human-only reply, I’ll add the no-ai tag to my topic.

on

2

Some light metadata (size, flags, etc) is always kept in the cache as it reduces database hits and increases performance. Sort is always done from the database unless when the query can’t be expressed in SQL. In any case, on of the items before 1.0 is released is to improve the search layer as JMAP queries are hybrid and can’t be delegated fully to SQL or ElasticSearch.