Negative Keyword Filter¶

A retrieval filter — instantiates Search Space Pruning

Excludes documents or results that match an explicit blocklist of terms or metadata — a cheap, transparent way to carve out whole irrelevant regions, kept honest by ongoing list maintenance.

Negative Keyword Filter prunes a retrieval space by what candidates contain, not by what they lack. It maintains an explicit blocklist — terms, phrases, or metadata tags — and removes any document or result that matches, sweeping out an entire irrelevant region in one cheap pass. Its defining trait among the siblings is that the exclusion signal is a surface feature (a word, a source tag), so the rule is transparent and fast but coarse: it acts on the appearance of a term, not on any verified property of the candidate. Because a blocked term is only a proxy for irrelevance, the filter lives and dies by two things — how confidently a match implies "genuinely irrelevant," and how well the term list keeps pace with a shifting corpus and vocabulary.

Example¶

A legal team running e-discovery over 900,000 collected documents needs to cut the obviously-irrelevant before costly attorney review. A Negative Keyword Filter excludes documents matching a blocklist: automated calendar invites, the cafeteria menu distribution list, newsletter footers, a specific spam-domain sender. In one pass it removes ~240,000 documents, none of which any reviewer needs to open.

But the list is a proxy, and the team knows it. The term "conference" was almost added to the blocklist to kill meeting spam — until someone noted it would also bury the "press conference" documents central to the case. So the filter blocks "conference room booking" as a phrase, not "conference" alone, and every blocklist term is logged with the count it excluded so a spot-check can catch over-broad matches. As new spam patterns appear mid-review, the list is updated rather than frozen.

How it works¶

Build the blocklist — enumerate the terms, phrases, and metadata tags that reliably mark a region as irrelevant.
Match and exclude — remove any candidate matching the list, using phrase- and context-sensitivity to avoid blocking on a term's mere presence.
Set the match confidence — decide how strong a match must be (exact phrase vs. loose token, one hit vs. several) before exclusion fires.
Maintain the list — revise terms as the corpus, vocabulary, and adversaries shift; a stale blocklist silently mis-cuts.

The distinctive commitment is exclusion by explicit surface term, which buys transparency and speed at the cost of precision.

Tuning parameters¶

Match strictness / confidence threshold — exact phrase versus fuzzy token, and how many hits trigger exclusion. Loose matching prunes more but wrongly cuts documents that merely mention a blocked word in a relevant context.
Blocklist breadth — how many terms. A longer list removes more noise and raises the odds one term is over-broad and buries relevant material.
Scope of match — whether a term is checked in full text, titles, or metadata only; metadata-only matching is safer but misses in-body noise.
Update cadence — how often the list is revisited; frequent updates track a drifting corpus, infrequent ones let it rot.
Whitelist overrides — protected terms or documents that a block never removes, guarding known-relevant regions.

When it helps, and when it misleads¶

Its strength is cheap, transparent bulk exclusion: anyone can read the blocklist and see exactly why a region was cut, and it removes large volumes of clearly-irrelevant material for almost no compute. Where irrelevance has a reliable surface signature, it is the first, fastest cut.

It misleads because a keyword is a blunt proxy. An over-broad term buries relevant documents that happen to contain it — the precision-versus-recall tension in sharp form, where tightening the filter to catch more junk also discards more signal.^[1] A stale list keeps blocking yesterday's noise while missing today's, and adversaries (spammers, evasive drafters) route around fixed terms deliberately. The disciplines are to prefer precise phrase- and metadata-matches over loose single tokens, to spot-check what each term actually excluded, and to treat the blocklist as a maintained artifact with update triggers rather than a set-and-forget rule.

How it implements the components¶

pruning_rule — the blocklist is the elimination condition: match a listed term or tag and the candidate is removed, no deeper look.
confidence_threshold — the match-strictness setting governs how sure a term-match must make you before excluding, trading recall for precision.
criteria_update_trigger — the maintenance cadence defines when the term list is revisited as the corpus, vocabulary, or adversaries change.

It carries no feasibility or dominance logic and no ownership; systematically re-checking blocked material for wrongly-excluded relevant items is Sample Audit of Exclusions's role, and comparative ranking of survivors belongs to Dominated-Option Removal.

Instantiates: Search Space Pruning — Negative Keyword Filter is the surface-term retrieval cut.
Sibling mechanisms: Sample Audit of Exclusions · Red-Flag Screen · Branch and Bound · Beam Search · Constraint Filtering · Decision Tree Pruning · Dominated-Option Removal · Eligibility Screening · Safety or Compliance Exclusion · Shortlisting · Triage Filter

References¶

[1] Precision is the share of excluded items that were truly irrelevant; recall here is the share of irrelevant items caught. A keyword filter can raise one only by pressuring the other — catch more junk (higher recall) and you inevitably block some relevant documents (lower precision). ↩