Store Maintenance Task Crashes When Trace/Metrics Retention is Set to More Than ~5 Years

Issue Description

When configuring holdTracesFor (or holdMetricsFor) to a value exceeding approximately 1800 days (~5 years from the system epoch), the scheduled store maintenance task crashes with an unexpected error.

Expected Behavior

The maintenance task should successfully purge old trace data without crashing, regardless of the configured retention period.

Actual Behavior

The maintenance task fails.

Reproduction Steps

  1. Have enterprise features enabled
  2. configure a tracing store
  3. configure the retention policy for tracing history to e.g. 100000 days
  4. wait for the Store Maintenance job to run
  5. The job fails

Relevant Log Output

Unexpected store error (store.unexpected-error): causedBy = crates/common/src/telemetry/tracers/store.rs:104, reason = Failed to generate reference span id.   

Stalwart Version

v0.16.x

Installation Method

Docker

Database Backend

RocksDB

Blob Storage

RocksDB

Search Engine

Internal

Directory Backend

Internal

Additional Context

My actual goal would be to disable the cleanup entirely. That’s why I tried to set it to a large value.

I have reviewed the documentation and FAQ and confirm that my issue is NOT addressed there.

on

I have searched this support forum (open and closed topics) and confirm this is not a duplicate.

on

I understand that topics in this category are triaged by a bot first but a human reply will follow up. If I’d prefer a human-only reply, I’ll add the no-ai tag to my topic.

on

This matches a real bug: the confirmed root cause is in the snowflake ID generator used to compute the deletion boundary. from_duration computes elapsed_since_internal_epoch - retention_period, and that internal epoch is fixed at 2021 (DEFAULT_EPOCH). If your retention period is larger than the time elapsed since that epoch (which happens once you exceed roughly 4-5 years), the subtraction underflows, returns None, and the maintenance task bails out with store.unexpected-error. So this is a genuine edge-case crash, not a config mistake on your part.

For your actual goal (disabling cleanup rather than setting a huge retention value), the docs show that holdTracesFor and holdMetricsFor are nullable Duration? fields, and setting them to null disables purge/storage entirely rather than trying to pick an arbitrarily long duration:

holdMtaReportsFor … “The duration for which MTA reports should be stored before being deleted, or None to disable storage”

The same nullable pattern applies to holdTracesFor/holdMetricsFor on the DataRetention object. Set the field to null in the DataRetention object (via WebUI or JMAP) instead of a very large duration, and the maintenance task will simply skip purging that data.

This is an automated reply from the Stalwart Help Bot. Other community members may follow up if this answer is incomplete or wrong.

None to disable storage means it doesn’t store traces at all, not what I want.

This has been fixed in v0.16.12 which will be released this week.