Pyzor: messages without qualifying text lines are checked with the empty digest and tagged PYZOR

Issues
1

Issue Description

When none of a message’s text lines survives Pyzor normalisation (each line must keep at least 8 characters after tokens of 10 or more bytes, URLs and addresses are removed), the digest is the SHA-1 of empty input, da39a3ee5e6b4b0d3255bfef95601890afd80709. Stalwart still queries it. On public.pyzor.org this digest currently returns Count 30358699 and WL-Count 361258, so the message is tagged PYZOR.

Code (v0.16.24):

  • crates/spam-filter/src/modules/pyzor.rs L220-285 (pyzor_digest): lines shorter than MIN_LINE_LENGTH (8) are dropped. When none remains, the SHA-1 of empty input is produced.
  • crates/spam-filter/src/modules/pyzor.rs L30-41: the query is skipped only when the message has no text or HTML part.
  • crates/spam-filter/src/analysis/pyzor.rs L26-29: is_spam = count > min_count && (wl_count < min_wl_count || wl_count / count < ratio). With the defaults (5, 10, 0.2) the empty digest qualifies (361258 / 30358699 ≈ 0.012).

For comparison, SpamAssassin’s Pyzor plugin, whose defaults these thresholds match, ignores results above a ceiling for this reason (lib/Mail/SpamAssassin/Plugin/Pyzor.pm: “Empty body etc results in same hash, we should skip very large numbers”).

Expected Behavior

A digest computed from no message content carries no information about that content, so it does not produce a PYZOR result.

Actual Behavior

A message whose text lines all fall below the minimum length after normalisation, including one with an empty body, is tagged PYZOR (+3.5 with the default ruleset). This includes messages whose content is entirely in an attachment.

Reproduction Steps

  1. Pyzor enabled with its defaults (public.pyzor.org).
  2. Deliver a message with an empty text/plain part and a PDF attachment.
  3. The spam filter result contains PYZOR, and the Pyzor trace event shows Count ≈ 30 million and WL-Count ≈ 361 thousand.

Stalwart Version

v0.16.x

Installation Method

Binary (Linux)

Database Backend

RocksDB

Blob Storage

RocksDB

Search Engine

Internal

Directory Backend

Internal

Additional Context

A possible fix could be to skip the Pyzor lookup when no line qualifies for the digest, so that no PYZOR tag is added while the other spam checks still apply. Messages with content would keep the existing digest and thresholds, and, unlike a ceiling on the counts, genuine high-count matches would still be detected.

I have reviewed the documentation and FAQ and confirm that my issue is NOT addressed there.

on

I have searched this support forum (open and closed topics) and confirm this is not a duplicate.

on

I understand that topics in this category are triaged by a bot first but a human reply will follow up. If I’d prefer a human-only reply, I’ll add the no-ai tag to my topic.

on

2

Confirmed against the current source. In crates/spam-filter/src/modules/pyzor.rs, the Pyzor lookup is skipped only when the message has no text or HTML part at all (lines ~30-36), not when every line fails the MIN_LINE_LENGTH (8-byte) filter inside pyzor_digest. If all qualifying text is stripped out (e.g. an empty text/plain part with content only in an attachment), the resulting digest is the SHA-1 of empty input, and in crates/spam-filter/src/analysis/pyzor.rs the spam decision is purely count > min_count && (wl_count < min_wl_count || wl_count/count < ratio) with no special-casing of that empty digest, so a known high-traffic empty-digest response on public.pyzor.org will trigger the PYZOR tag with the defaults (min_count 5, min_wl_count 10, ratio 0.2).

Your reproduction and proposed fix (skip the lookup entirely when no line survives normalization, rather than imposing a count ceiling) match what the code shows. This is a legitimate bug report rather than a documented/configurable behavior, so a human maintainer will need to follow up on addressing it in the code.

This is an automated reply from the Stalwart Help Bot. Other community members may follow up if this answer is incomplete or wrong.

3

Fixed in v0.16.25 (will be released today).