On August 23rd a bot submitted gibberish - random consonant strings, no links, no email worth blocking - to a customer's form, and our spam engine scored it 0.0. Clean pass, straight to the customer's inbox. That incident and its fix are a decent tour of how form spam filtering actually works, including the part most services keep vague: what happens when the filter is wrong.
How gibberish scores zero
A spam score is a sum of signals, and every signal is a bet about what spam looks like. Ours at the time bet on links (spam wants you to click something), blocked keywords, disposable email domains, and form-fill speed. The probe had no links, no keywords on any list, a plausible-looking address, and arrived at human speed. Every signal individually said clean, so their sum said clean. The filter wasn't broken - it was measuring exactly what we told it to measure, and the bot happened to be shaped like nothing on the list.
This is the general failure mode of deterministic filters, and it is worth being honest about: a rule-based engine only catches what someone anticipated. The fixes are to anticipate more, to make the threshold less forgiving, or to add a judge that generalises. We did all three.
Anticipating more: signals that don't need an AI call
The embarrassing part of the incident was that the message was obviously junk to any human glance. Three signals now formalise that glance, all computable in microseconds: a link-flood check that counts URLs against message length, an HTML-markup check (real people do not type anchor tags into contact forms), and a script-mixing check for text that switches alphabets mid-word, which legitimate mail essentially never does but keyword-evading spam does constantly. Each was validated against every stored submission on the platform before shipping - the test suite pins down not just what they catch but what they must not catch, because a heuristic that flags real customers is worse than no heuristic.
The threshold: why 0.8 was wrong and 0.6 is not arbitrary
The quarantine threshold was 0.8, which sounds prudently conservative until you do the arithmetic: individual signals contribute 0.2 to 0.4 each, so 0.8 demanded roughly three independent signals agreeing before anything was held. A submission with two strong signals - say a link flood AND mixed scripts - scored around 0.75 and sailed through. Under 0.6, two agreeing signals quarantine; one weak signal still passes. Before changing the default we replayed it against all stored submissions: the only rows newly caught in the 0.6-0.8 band were four Russian SEO spam messages, already flagged by two signals each. Zero legitimate messages moved. The new default is pinned by a test so nobody can quietly raise it back in a refactor.
Why quarantine and not delete
Every choice above trades false negatives against false positives, and no threshold makes both zero. So the design question is not "how do we never be wrong" but "what does being wrong cost". If a wrong verdict deletes the submission, a false positive costs a customer a real lead, silently, with no recourse. If a wrong verdict quarantines it, the cost is a row in a spam tab that a human can overrule. That asymmetry is the whole argument: quarantined submissions are stored, readable, and restorable, and only the truly unambiguous cases - a filled honeypot, a sub-second form fill, a failed captcha - are rejected outright, because those are not probabilistic judgements.
With a relay-style service that forwards email and keeps nothing, a false positive is unrecoverable by construction. This is the quiet argument for stored submissions that has nothing to do with dashboards: storage is what makes a spam filter allowed to be aggressive.
The learning loop: expensive judgement, cached
Above the heuristics sits an optional AI classifier for the ambiguous middle band - the only layer that generalises to spam nobody anticipated. It is also the expensive layer: a network call with a timeout budget on every ambiguous submission. Paying that cost repeatedly for the same sender would be absurd, so verdicts feed back into sender reputation: a confident spam verdict counts against the sender's address and domain, a confident clean verdict counts for them, and the next submission from that sender is scored by the free deterministic engine alone, already nudged out of the ambiguous band. The LLM's judgement gets cached as reputation, and the system stops asking questions it has already paid to answer. When you mark a quarantined submission as not-spam, the same machinery runs in reverse.
Two things are deliberately never learned. Message content: auto-adding phrases to a blocklist on an LLM's say-so is how a filter poisons itself - one bad verdict involving the word "invoice" and every real invoice enquiry is quarantined forever after. And IP addresses: they are shared, reassigned and NAT-ed, so punishing an IP punishes strangers. Only identity a sender actually controls - their address, their domain - accumulates reputation. Uncertain verdicts near 0.5 are ignored entirely, because recording "I don't know" as evidence just drifts every reputation toward the mean while looking like learning.
What generalises
- A spam score is a sum of bets. Audit what the bets assume, because spam that matches no assumption scores zero.
- Set thresholds by replaying real data, not by picking a number that sounds strict.
- Choose failure costs, not failure rates: quarantine makes false positives cheap, which is what permits catching more.
- Cache expensive judgement as reputation, but only on identity the sender controls.
- Pin every tuned constant with a test, or a refactor will untune it.