Testing AI/LLM classifier functionality

Questions
1

Your question

I’ve setup a local Ollama instance with QWEN 2.5, 7B

How can I test the effectivenes/functionality from a stalwart server without breaking all active users?

2

Found it — there’s a ClassifySpam management action that lets you submit a raw message directly to the spam filter and see the resulting score/tags, without it ever touching real mail flow or affecting active users.

Now let’s check tasks-actions/actions.md for how to actually invoke this via WebUI or CLI.

I found the safe testing mechanism: Stalwart’s ClassifySpam action lets you submit a raw message through the whole spam filter (including the LLM classifier) and see the resulting score/tags without it ever touching live mail flow or user mailboxes.

How to test:

  1. Configure your Ollama model as an AiModel and enable it in SpamLlm ({"@type": "Enable", "modelId": "chat"}, pointing at your local QWEN endpoint).
  2. Rather than sending live mail, create a ClassifySpam Action via the WebUI (Management → Actions) or CLI (stalwart-cli create action/ClassifySpam), supplying:
    • message: a raw sample email (EML text) you want to test
    • envelope fields: remoteIp, ehloDomain, envFrom, envRcptTo, etc.
  3. Stalwart runs the message through the full spam filter pipeline, including the LLM classifier, and returns the score, contributing tags (e.g. LLM_UNSOLICITED_HIGH), and final result (spam/ham/reject/discard) — all without delivering anything or affecting real users.

You can retrieve results afterward with x:Action/get or stalwart-cli get action <id>. This lets you iterate on your prompt/model tuning purely against sample messages before enabling the LLM classifier for live traffic. Also check the X-Spam-LLM header format described in the LLM classifier docs to see exactly what your model returned.

Be aware of the performance caveat: CPU-only local models like a 7B Qwen via Ollama can add noticeable latency per message if you later enable this in production, so test throughput as well as accuracy.

Docs used: LLM classifier and Actions.

This is an automated reply from the Stalwart Help Bot. Other community members may follow up if this answer is incomplete or wrong.