Accept loop spins all CPU cores with no back-off when the process hits EMFILE (too many open files). AcceptError only logged at debug

Issues
1

Issue Description

When Stalwart hits its open-file-descriptor limit (EMFILE), the accept loop in crates/common/src/network/listen.rs retries accept() immediately with no back-off, pinning every worker thread at ~100% CPU. All listeners (SMTP/IMAP/submission and the management port) go deaf while the process still looks “up”, and nothing is logged at normal levels. It only recovers on restart. Imprtant: its not config. the accept error emits a debug-only AcceptError event and loops with no sleep (verified on main/0.16.12), and the OS FD ceiling (1024, Docker default) is exhausted before Stalwart’s own maxConnections limiter (8192) so the graceful drop never runs and raising maxConnections can’t help.

Expected Behavior

On repeated accept() errors (especially EMFILE/ENFILE), back off with a short capped sleep instead of busy-looping, and log the failure at warn/error so it’s visible. Ideally document a recommended nofile ulimit for the Docker image

Actual Behavior

accept() fails every iteration, all workers busy-spin: load pins to the vCPU count (4.00 on 4 vCPU), all cores ~100%, every listener plus 127.0.0.1:8080 unresponsive, process still “up”, nothing at info/warn/error. Restart recovers in ~10s, then it re-wedges as FDs refill. Raising nofile to 65536 eliminated it.

Reproduction Steps

  1. Run Stalwart in a container with --ulimit nofile=1024:1024.
  2. Exhaust FDs: open connections faster than they close, or hold ~1024 open (in prod, a load balancer HTTPS health monitor on :443 leaked probe connections at ~6/min).
  3. All cores hit ~100%, every listener incl. management unresponsive, nothing logged.
  4. Restart recovers; it re-wedges at the limit.

Relevant Log Output

Nothing at default levels (that silence is part of the bug). At trace, roughly: NetworkEvent::AcceptError localPort = 443 reason = “Too many open files (os error 24)”

Stalwart Version

v0.16.x

Installation Method

Docker

Database Backend

PostgreSQL

Blob Storage

S3-compatible

Search Engine

Internal

Directory Backend

Internal

Additional Context

Still present on main/0.16.12 (nothing in the 0.16.9–0.16.12 changelog fixes it). Triggered by a load-balancer health monitor leaking FDs at Docker-default nofile=1024. Distinct from forum #336 (NLB TLS-state reuse) and GitHub #863 (v0.10.4, not FD-related). Workaround: nofile raised to 65536.

I have reviewed the documentation and FAQ and confirm that my issue is NOT addressed there.

on

I have searched this support forum (open and closed topics) and confirm this is not a duplicate.

on

I understand that topics in this category are triaged by a bot first but a human reply will follow up. If I’d prefer a human-only reply, I’ll add the no-ai tag to my topic.

on

2

This confirms the report: the code does indeed loop back to accept() immediately with no sleep on any error (including EMFILE), and logs it via trc::event!(Network(NetworkEvent::AcceptError), ...) with no special severity override, so it inherits whatever level the AcceptError event type defaults to (debug/trace per the report). No back-off or ulimit guidance exists in the docs or code that I can find.

This matches your analysis of a genuine bug in crates/common/src/network/listen.rs: on accept() failure (including EMFILE), the loop immediately retries with no delay, and the event is only surfaced via NetworkEvent::AcceptError at its default (non-warn) verbosity, with no distinction made for EMFILE/ENFILE. I don’t see any built-in back-off logic or documented recommended nofile ulimit for the Docker image in current docs or source.

This is an automated reply from the Stalwart Help Bot. Other community members may follow up if this answer is incomplete or wrong.

3

This has been fixed in v0.16.12 which will be released this week.