When managing Stalwart server configuration declaratively via stalwart-cli apply (e.g., using an NDJSON plan generated via stalwart-cli snapshot Tracer or IaC/Ansible automation), Tracer objects are duplicated on every apply execution rather than updated in place.
Unlike entities such as Domain (which supports matchOn: ["name"]) or AllowedIp (which supports matchOn: ["address"]), Tracer is a tagged enum variant (Stdout, Journal, Log, OtelHttp, OtelGrpc) that does not contain a name property. When stalwart-cli snapshot Tracer exports the tracer configuration, it produces an upsert block without a matchOn specification. Consequently, each subsequent apply invocation fails to recognize existing tracers and creates an additional active tracer with a new random object ID. Over multiple deployments or plan reconciliations, duplicate active tracers accumulate, multiplying stdout/journal log entries and increasing system load.
Expected Behavior
Applying an NDJSON plan containing a Tracer declaration (e.g. Stdout tracer) via stalwart-cli apply should be idempotent. If an existing Stdout tracer is already configured on the server, apply should either update it in place or leave it unchanged without creating a new duplicate object.
Actual Behavior
Every execution of stalwart-cli apply creates a new, additional Tracer object with a new ID (e.g., jctphoqyabqa, jcgn1le7aaqa), leaving all previously created tracers active simultaneously.
Export the current tracer configuration using snapshot:
stalwart-cli snapshot Tracer > tracer-plan.ndjson
Generated plan content:
{“@type”:“upsert”,“object”:“Tracer”,“value”:{“tracer-jctntpqsaaqa”:{“@type”:“Stdout”,“enable”:true,“eventsPolicy”:“exclude”,“events”:{},
“multiline”:false,“lossy”:false,“level”:“info”,“ansi”:false,“buffered”:true}}}
Re-apply the unchanged snapshot to the server:
stalwart-cli apply --file tracer-plan.ndjson
Query tracers again:
stalwart-cli query Tracer
Observe that there are now 2 active Stdout tracer objects instead of 1.
Relevant Log Output
Query output after applying snapshot twice:
Id Method Logging level Enable this tracer
jctphoqyabqa Stdout Info Yes
jctntpqsaaqa Stdout Info Yes
Stalwart Version
v0.16.x
Installation Method
Docker
Database Backend
RocksDB
Blob Storage
RocksDB
Search Engine
Internal
Directory Backend
Internal
Additional Context
OS: Linux (Debian / Ubuntu, x86_64)
Container Engine: Podman (rootless Quadlet)
I have reviewed the documentation and FAQ and confirm that my issue is NOT addressed there.
on
I have searched this support forum (open and closed topics) and confirm this is not a duplicate.
on
I understand that topics in this category are triaged by a bot first but a human reply will follow up. If I’d prefer a human-only reply, I’ll add the no-ai tag to my topic.
Tracer objects (and other keyless, tagged-enum types) are matched by full-value comparison when matchOn is omitted from the plan, not left unmatched. This “value matching” behavior (equivalent to an implicit matchOn: "*") was made the default fallback for types with no label property, and several duplicate-creation bugs around it were fixed in stalwart-cli 1.0.10–1.0.12:
1.0.10: “upsert removed required matchOn field” (so it no longer errors on keyless types).
1.0.12: "matchOn": "*" was made an explicit opt-in for value matching, and the changelog specifically records fixes for apply creating a duplicate when the same match key is upserted twice, and value matching misreading server null fields as drift.
If you’re on an older stalwart-cli build than 1.0.12, upgrading should resolve this, since snapshot Tracer intentionally omits matchOn for Tracer and relies on the client falling back to value matching against all comparable scalar fields (including @type) to find the existing object instead of creating a new one. If you’re already on 1.0.12+ and still see duplicates, it likely means one of the round-tripped fields (e.g. events, eventsPolicy, or a default value) doesn’t compare byte-for-byte equal to what the server returns, causing the value match to miss; in that case pasting the exact CLI version and a --dry-run output showing the mismatch would help narrow it down further.
This is an automated reply from the Stalwart Help Bot. Other community members may follow up if this answer is incomplete or wrong.
Thanks for the detailed clarification! That explains the exact behavior we observed:
The server had an existing Stdout tracer that was temporarily set to level: "trace". When we deployed an updated IaC plan containing level: "info", the full-value comparison missed (because level differed), causing stalwart-cli apply to assume no matching object existed and creating a second Stdout tracer rather than updating the existing one in place.
So while full-value matching works when values are 100% identical, it prevents in-place updates: changing any field (like level from trace to info) breaks the match and spawns a duplicate active tracer.
Being able to specify a match key for the variant (e.g. matchOn: ["@type"], or matching on the Stdout/Journal method) would allow idempotent updates to existing tracers without creating duplicates.
snapshot leaves it out because Tracer declares no label property in the schema, and that is the only thing snapshot derives a matchOn from. It does warn about it, but on stderr, so if you are redirecting stdout to a file the warning scrolls past you.
Delete the tracers you have already accumulated before applying the keyed plan. With two Stdout tracers present the upsert refuses to guess and fails with ambiguous upsert; 2 existing objects match on @type. ["@type"] also assumes at most one tracer per method; if you run two Log tracers, key on ["@type","path"].
After cleaning up the accumulated duplicates beforehand, subsequent stalwart-cli apply executions now update the existing Stdout tracer in-place
idempotently without creating duplicate objects. Marking this as solved!