V0.16: FAST bulk export/import of individual mailboxes for backup and GDPR reasons?

Your question

Good day,

It seems to us that “stalwart-cli export account” has been retired in v0.16. We are aware of the new JMAP methods to export account data and mails but this seems to be a slow process (about 50msgs/s with RocksDB on plain NVMe, even much less in K8S Longhorn PVCs + PGSQL backend). We would like to know what is the fastest way for RAPID mailbox export and import ?

thank you in advance

I understand that topics in this category are triaged by a bot first but a human reply will follow up. If I’d prefer a human-only reply, I’ll add the no-ai tag to my topic.

on

Based on my research, here is what the current docs say:

There is no per-account mailbox (message) export/import tool in the current version. The stalwart-cli export account command from earlier versions has not been replaced with an equivalent per-account mail data export tool. The current options are:

  1. Full-database binary dump (stalwart --export): Exports the entire internal database as a binary dump. This is fast (direct store access, no protocol overhead) but is designed for whole-server backend migration, not per-user GDPR exports. The server must be stopped while running it. You can scope it to specific data families (e.g. EXPORT_TYPES="blob,data") but you cannot filter by account.

  2. JMAP API / stalwart-cli object snapshot: stalwart-cli snapshot can export configuration objects (accounts, domains, settings, etc.) as NDJSON plans, but this covers server configuration objects – not individual email message data.

  3. IMAP APPEND / standard mail tools: For per-user mail data, the most practical approach with the current architecture is to use standard IMAP synchronization tools (such as imapsync or mbsync/isync) to pull a user’s mailboxes via IMAP and store them as local Maildirs or mbox files. This can be parallelized across accounts and does not require stopping the server. The speed will depend on your infrastructure and IMAP concurrency settings, but this is the standard, interoperable approach for per-account portability.

The docs explicitly note that the binary export/import utility is not a backup solution and that users should rely on their database backend’s native tools for routine backup/recovery.

For GDPR data-subject requests specifically, the JMAP email export path (what you’re already using) or IMAP-based mirroring are the only per-account options currently documented. If 50 msg/s is too slow for your use case, parallelizing IMAP connections per-account across multiple threads/processes with a tool like imapsync --usecache would be the documented-adjacent approach – but there is no built-in bulk per-account mail export CLI in the current release.

Relevant docs:

This is an automated reply from the Stalwart Help Bot. Other community members may follow up if this answer is incomplete or wrong. If you found it useful, mention @helpbot in any reply to ask follow-up questions and the bot will respond again.

I believe this is a real gap, can we escalate this into an issue ? IMAP READ might be fast, but IMAP APPEND is still slow and can also cause inconsistencies when backups are restored into an existing live mailbox. Additionally, the mailbox passwords must be known for IMAP procedures, which is counter-productive for platform-level backups.

Your concern about IMAP APPEND speed and restore inconsistencies is valid and well-reasoned. Unfortunately, the current Stalwart documentation does not cover a dedicated per-account email message export/import mechanism beyond what was already described – there is no built-in fast path for this use case in the current release.

Regarding escalating to a GitHub issue: as per this forum’s guidelines, that’s something a human community member or staff member can assess and action. The bot cannot open issues or contact maintainers on your behalf.

What the docs do show that’s worth noting for your use case:

  • ArchivedItems (stalwart-cli create ArchivedItem/Email) provide a structured way to restore individual emails into an account via CLI using a blobId, which could theoretically be scripted – but this is designed for undelete/recovery scenarios, not bulk backup/restore.
  • The binary --export dump (with EXPORT_TYPES="blob,data") is the fastest path to capturing raw message data at the store level, but it’s not per-account and requires a server stop.

The gap you’ve identified – a fast, per-account, live mail export/import that avoids IMAP APPEND overhead and restore inconsistencies – is not covered in the current docs. A community member or staff member will follow up.

This is an automated reply from the Stalwart Help Bot. Other community members may follow up if this answer is incomplete or wrong. If you found it useful, mention @helpbot in any reply to ask follow-up questions and the bot will respond again.

A migration tool will be released in about 1 or 2 weeks.

@mdecimus not sure if that is addressing the issue. Will the migration tool support future-proof, FAST export/import of INDIVIDUAL mailboxes ?

Yes, it will store the imported data in a protocol agnostic format