Safe Procedure to Change In-Memory Store

Your question

I currently have a 3 node cluster and I want to take it down to a single node without having to rebuild.

Step 1 would be to shut off nodes 2 and 3 so that the in-memory store is no longer required for cluster coordination, then disable the coordinator.

Step 2 would then be to change the in-memory store to use data store. I am guessing the data would simply be lost and started over, and restarting the server would be prudent, but since this is all transitory info, nothing of consequence should be lost?

From there is there anything else I need to clean up for the two missing nodes? (Strictly Stalwart, I realize there are load balancers, firewalls, etc.)

Thanks

I understand that topics in this category are triaged by a bot first but a human reply will follow up. If I’d prefer a human-only reply, I’ll add the no-ai tag to my topic.

on

Now let’s check the coordination index page for details on peer-to-peer vs external, and how to switch coordinator to none/disabled.

The plan looks correct, and the contents are disposable: rate-limit and fail2ban counters, distributed locks, OAuth authorization codes, ACME tokens, greylist and Sieve tracking, WebDAV locks. Two visible side effects of dropping them: active auto-ban counters reset, and anyone mid-way through an OAuth authorization exchange has to retry. No sessions are lost, since access and refresh tokens do not live there.

One thing to add to your list. With the data-store variant nothing expires keys natively, so you need dataCleanupSchedule set on the DataRetention object or expired keys will just accumulate in the data store. That is the one real difference from a dedicated in-memory backend.

Nothing to clean up for the two nodes you are removing: node ids are hostname-keyed leases, the ClusterNode view is read-only, entries go stale after an hour without renewal and the rows are cleared automatically after 24 hours. The coordinator and the in-memory store are independent singletons so the order between those two changes does not matter, and restarting after the change is still the clean way to do it. Also take a look on the way down for any ClusterRole objects or per-node environment settings that only existed for nodes 2 and 3.

Is this a normal procedure if you start a new install using the data store, or does Stalwart create a task for you in that case?

Cluster entries went Stale then to Inactive and don’t seem to want to go away on their own. They’ve been there >72 hours now.

Good advice, the only thing I had was per-node entries in the connection strategies so that the outbound hostname was lining up per node.

I wanted to make a correction to what I wrote before, dataCleanupSchedule is not something you need to add: it ships as a default on DataRetention, daily at 02:00, so a fresh data-store install is already purging expired keys and there is nothing to create.

And what I said “cleared automatically after 24 hours” was also wrong. That sweep runs only when a node acquires its node id at startup, never on a timer. Your surviving node restarted while those rows were still younger than 24 hours, skipped them, and has had no reason to look again since, which is why they have sat there for three days. Restart it now and the sweep will clear them, since they are well past the cutoff this time.

v0.16.20 will include a fix that deals with these node id leasing issues.

I did restart when upgrading from 0.16.18 to 0.16.19, nodes 2 and 3 are gone, only node 1 remains as “inactive”. I will check back after installing 0.16.20.

On v0.16.20 Node 1 has now updated the name correctly and is marked Active.