OIDC-only deployment (v0.16.8): first-boot directoryId churn, /account web-UI login rejected, and app passwords with a store-less Oidc directory

Questions
1

Your question

TL;DR

We run Stalwart with a single @type: Oidc directory as the only auth backend (IdP = Authentik), provisioned entirely by database via the JMAP management API. Webmail + IMAP OAUTHBEARER work.

Now facing three issues:

  1. First boot: x:Authentication.directoryId is silently cleared to null on restart for ~10–20 min after a cold boot, then sticks permanently. We can’t see why.
  2. Web-UI / /account login: the self-service portal’s OAuth token (client_id=stalwart-webui) is rejected by the directory, which is pinned to a different client (bulwark). Users can’t reach /account to mint app passwords.
  3. App passwords: the @type: Oidc directory has no store field (per your reference schema) — so it’s unclear whether app passwords can work for an OIDC-backed principal at all, or where the secret would live.

Understood the following from the docs

  • Apple Mail / Thunderbird / K-9 can’t do third-party OAUTHBEARER; app passwords are the documented workaround. That’s exactly what we’re trying to stand up.
  • requireAudience (default "stalwart") gates aud; claimUsername + usernameDomain (with email-claim fallback) handle our full-email usernames. Working.
  • OIDC accounts must be pre-created before first sign-in. Done.
  • Tracing: we will run a Tracer (@type: Stdout, level: trace, capturing auth.* + config.build-error/config.fetch-error) to capture the errors below.

Shared environment

  • Single node, Docker Compose: postgres + Authentik (server/worker) + Stalwart + webmail (Bulwark) + Caddy on one bridge network. Caddy terminates TLS and reverse-proxies the IdP and Stalwart JMAP/:8080 on :443.
  • Stalwart v0.16.8; the only on-disk file is config.json = {"@type":"RocksDb","path":"/var/lib/stalwart/"}. All provisioning via JMAP x:* methods on loopback :8080 (admin basic-auth); stalwart-cli is not in the image.
  • One x:Directory (@type: Oidc) is the default directory (x:Authentication.directoryId):
    • issuerUrl: https://<idp>/application/o/bulwark/ (Authentik per-provider issuer)
    • requireAudience: bulwark (the OAuth client_id our webmail uses)
    • requireScopes: ["openid","email"], claimUsername: preferred_username, claimName, claimGroups.
  • No internal directory alongside the OIDC one. Webmail JMAP + IMAP OAUTHBEARER with bulwark-issued tokens (iss=.../o/bulwark/, aud=bulwark) work perfectly.

Happy to share full x:Directory / x:Authentication JSON, the authorize request, and decoded (non-secret) token claims, or run any diagnostic.

Group 1 — x:Authentication.directoryId won’t persist on cold first boot

Symptom

On a cold first boot:

  1. We provision in order: x:Domain → x:SystemSettings (defaultHostname + defaultDomainId) → the OIDC x:Directory → x:Authentication/set { directoryId: "<oidc-dir-id>", defaultUserRoleIds: {…} }. The set succeeds; an immediate get shows directoryId correct.
  2. We restart Stalwart (a restart appears required for a new directoryId to take effect — setting it live leaves auth at 401).
  3. After the restart, directoryId reads back as null (→ per your reference, that falls back to the internal directory, which is empty → all valid OIDC bearer tokens rejected with 401 "You have to authenticate first.").
  4. This repeats on every restart for ~10–20 min after first boot (8 automated set+restart cycles over ~8.5 min — all reverted to null).
  5. After that window, the exact same set+restart persists, OIDC auth returns JMAP 200, and stays stable across all later restarts — it even survives the IdP being stopped entirely afterwards.
What we’ve ruled out (via live JMAP probing)
  • Not directory-id churn — the OIDC x:Directory id is stable across all restarts; only x:Authentication.directoryId clears.
  • Not the payload — directoryId alone, or with defaultUserRoleIds, both persist fine once settled.
  • Not “a failed build clears it” — on a settled node we stopped the IdP (discovery → 502), set directoryId, restarted → it stayed set. A directory that merely fails discovery does not lose the binding.
  • Not network reachability — during the failing window, .well-known/openid-configuration returns 200 both externally and from inside the Stalwart container (~0.5s, valid issuer + jwks_uri).
  • Only reproducible on a genuine cold first boot; not reproducible on a settled node by any restart / IdP-down combination.

Working hypothesis: the OIDC directory must complete a first successful build (discovery + JWKS) with the IdP fully ready, and until then the default-directory reference is dropped on each restart — but we can’t confirm without visibility into the build state.

Questions
  1. What clears x:Authentication.directoryId on restart? Is the binding dropped when the referenced OIDC directory hasn’t yet successfully built/validated against the IdP at startup?
  2. Is there a deterministic readiness signal — an API field on x:Directory, a healthz probe, or a specific log event — indicating the OIDC directory has successfully built (discovery + JWKS), so provisioning can wait before binding it?
  3. Do repeated restarts during IdP warm-up hurt? Does a restart abort an in-progress async directory build, so frequent restarts prevent the first successful build from completing?
  4. Is there negative caching of a failed discovery/JWKS fetch (and a TTL)? Any supported way to force a refresh without a restart?
  5. What is the supported way to provision OIDC as the default directory on first boot so the binding is reliable (ordering, readiness wait, or a mechanism other than setting x:Authentication.directoryId)?

Group 2 — web admin / /account self-service login against an external IdP

Goal: users open /account/ to mint app passwords for native clients (the documented workaround).

Symptom

Opening https://<mail-host>/account/ redirects to the IdP with client_id=stalwart-webui (PKCE/S256, no secret, redirect_uri=.../account/oauth/callback). We had to create an IdP client with exactly that client_id for the authorize step to succeed. The user authenticates (password + 2FA) and the IdP reports success — but Stalwart then returns HTTP 401 + WWW-Authenticate: Basic and establishes no session.

The stalwart-webui token doesn’t match what the single default directory requires:

Directory requires stalwart-webui token carries
issuer https://<idp>/application/o/bulwark/ https://<idp>/ (IdP global issuer)
audience bulwark stalwart-webui

i.e. the interactive web-UI login appears to be validated against the same single directory that validates IMAP/JMAP, which is pinned to bulwark.

Questions
  1. Is the web-UI OAuth client_id configurable, and if so which setting key? (We provision by DB via JMAP; config.json is only the RocksDb pointer — we can’t find where stalwart-webui is defined.) Or is stalwart-webui a fixed built-in?
  2. Which directory / issuer / audience validates the interactive web-UI login token — always x:Authentication.directoryId, or can the web-UI login be bound to a specific directory?
  3. For a setup where IMAP/JMAP validation is pinned to one audience (bulwark), what is the supported way to also accept the web-UI login — (A) make the web UI reuse the bulwark client, or (B) add a second @type: Oidc directory matching stalwart-webui (iss=.../o/stalwart-webui/, aud=stalwart-webui) for the web-UI login while bulwark keeps serving IMAP/JMAP? Is either supported/recommended?

Group 3 — do app passwords work when the only directory is @type: Oidc?

The AppPassword docs say the secret is stored “on the account as one of its secrets” ($app$name$password). But the Directory reference object shows the Oidc type has no store field — unlike Sql (store: “Storage backend where accounts and groups are stored”) and Ldap (attrSecret). So an OIDC directory appears to have no writable backing store for that secret. We also can’t yet mint one to test, because we can’t reach /account (Group 2), and the docs say admins can’t create app passwords for users — only the user via the portal.

What we observe today: OIDC principals materialise as x:Account with an empty credentials {} map; x:Authentication.maxAppPasswords: 5 is set; IMAP/SMTP advertise AUTH=PLAIN/AUTH PLAIN LOGIN; a PLAIN attempt for an OIDC principal is processed (wrong password → clean NO [AUTHENTICATIONFAILED], not “mechanism unsupported”).

Questions
  1. Do app passwords authenticate IMAP/SMTP AUTH PLAIN for a principal whose authoritative directory is @type: Oidc? If yes — where is the $app$ secret persisted (the OIDC directory has no store; does it land on an internal data-store principal instead?), and is any additional directory/config required (e.g. an internal fallback directory) for app-password validation to be consulted alongside OIDC?
  2. If app passwords don’t work with an OIDC-only directory, what is the supported way to give Apple Mail / Thunderbird / K-9 users a password credential while OIDC remains the IMAP/JMAP OAUTHBEARER authority?

Group 4 — minor

  1. login_hint: Stalwart sends the mailbox address as the OIDC login_hint (+ prompt=login). Our IdP keys users by a username that differs from the mailbox, so the hint pre-fills a non-matching identity. Can login_hint be omitted or customised?

I understand that topics in this category are triaged by a bot first but a human reply will follow up. If I’d prefer a human-only reply, I’ll add the no-ai tag to my topic.

on

2

Hi, I have similar issues. OIDC support in Stalwart is not as good as we’d like. It’s deffinitely changing, some parts are better but also some parts are worse than before.

In Stalwart 0.15 end users cannot login into management UI at all and admin has to set their app passwords (they can be whatever we want).

Stalwart 0.16 is a bit different:

  • end users can login into management UI (good)
    • but only if you webmail (Roundcube, Bulwark, …) is using stalwart-webui client id (bad, but manageable)
  • admin cannot set app paswords for users at all (bad)
  • impersonation feature does not work when using OIDC directory (really bad for managing service accounts)
  • app passwords are only server generated (really bad, migration forces you to change the passwords in all printers, routers, server managements, NASes, CCTV cameras, etc.)
3
  • App Passwords: The WebUI does not support setting app passwords for users but you can create them using the CLI, authenticating as an administrator with impersonate privileges. App Passwords are auto generated as they need to include some internal metadata, providing a custom password is not supported. I believe Gmail also auto-generates app passwords.
  • Impersonation: Since authentication is delegated to the OIDC, we cannot forward impersonation requests shaped as user%admin as these are rejected by the IdP. The only way to impersonate with OIDC is using the recovery admin.