Cue-Hijack Red-Team Review¶
Procedure — instantiates Supernormal Cue Guardrail Design
Tasks an adversarial reviewer with actively finding ways a design could capture attention, appetite, fear, or reward-seeking beyond user intent, focusing on vulnerable slices.
Most guardrails check a design against a known list. Cue-Hijack Red-Team Review does the opposite: it hands the design to someone whose explicit job is to exploit it — to think like an attention-harvesting adversary and discover, before release, how the design could be turned to capture a responder against their own interest. Its defining idea is adversarial, generative discovery of unlisted hijack paths, run by a party independent of the builders. Where an inventory catalogs cues you already know are strong, the red team invents the cheap, clever escalation nobody wrote down: the near-miss animation that could be added, the vulnerable-slice pressure that a growth experiment would eventually find. It is structured imagination of misuse, and it works precisely because it does not wait for a checklist to name the threat.
Example¶
A wellness app is about to ship a habit-streak feature meant to encourage daily meditation. Before launch, a red-team review convenes a small group deliberately outside the product team — including someone with clinical background in compulsive behavior. Their brief: assume you are trying to maximize checking and anxiety, and show how this design lets you. They quickly find several paths. The streak counter, framed as loss ("Don't break your 43-day streak!"), could be tuned to exploit loss aversion in exactly the fatigued, anxious users the app is meant to help. A "friends' streaks" leaderboard would convert a private practice into social-comparison pressure. A variable "bonus insight" reward on random days would turn calm practice into compulsive re-opening.
None of these are yet in the build; the red team surfaces them as available hijacks the roadmap would drift toward under engagement pressure. The team documents each, flags the vulnerable slice it would hit hardest, and hands the findings to the guardrail designers to pre-empt. The review's value is that it caught the exploits while they were still hypothetical.
How it works¶
- Assign the adversarial brief. Reviewers are explicitly told to maximize hijack — attention capture, appetite, fear, status-seeking, compulsive checking — rather than to evaluate the design charitably.
- Center the vulnerable slice. Attacks are aimed where they would bite hardest: children, fatigued, isolated, novice, or scarcity-pressured users, since a cue ordinary for one responder is supernormal for another.
- Keep the team independent. The reviewers sit outside the building team, so incentive to rationalize the design is removed and findings carry oversight weight.
- Produce exploit paths, not verdicts. Output is a ranked list of concrete hijack routes and the harm signals each would produce — handed to designers to close, not a pass/fail stamp.
Tuning parameters¶
- Adversary aggressiveness — how far reviewers are licensed to go. A more ruthless brief finds more exploits but can generate implausible edge cases; a tame one misses the clever ones.
- Slice coverage — how many vulnerability profiles the attack deliberately targets. Broad coverage catches more but costs time; narrow coverage risks missing the most exposed group.
- Independence depth — internal-but-separate team versus fully external reviewers. Greater independence reduces capture but costs access and context.
- Cadence — one-time pre-launch versus recurring review as the product evolves. Recurring review catches drift; one-time review is cheaper but goes stale.
When it helps, and when it misleads¶
Its strength is generativity: it finds hijack paths no checklist contains, because an adversary's imagination outruns any fixed inventory — the same logic that makes red teaming valuable in security, where defenders learn most from someone actively trying to break in.[n1] Centering vulnerable slices makes it especially good at catching harms that averaged metrics hide.
Its failure mode is that it depends heavily on reviewer skill and independence: a captured or incurious red team produces comfortable findings that miss the real exploits, and a purely internal one may pull punches. It can also over-generate — a flood of implausible attacks that buries the credible ones and invites the team to dismiss the whole exercise. And a review is a snapshot; the design it cleared can drift into hijack later under optimization pressure. The guarding discipline is to keep the team genuinely independent, prioritize findings by plausibility and vulnerable-slice impact, and re-run as the product changes rather than treating one clean review as permanent clearance.
How it implements the components¶
vulnerability_profile— it targets attacks at specific at-risk responder groups, making the profile the axis along which hijacks are hunted.harm_and_compulsion_signal— each exploit path is characterized by the harm it would produce (compulsion, regret, fear, crowd-out), so findings name the signal to watch.independent_oversight_channel— the review is run by a party outside the building team, giving its findings oversight standing rather than self-assessment.
It does NOT implement response_ceiling or deamplification_rule — setting and enforcing the limits that close the exploits it finds is Cue Intensity Cap Protocol's job; this procedure discovers hijack paths, it does not impose the caps that fix them.
Related¶
- Instantiates: Supernormal Cue Guardrail Design — supplies the adversarial-discovery step that finds hijack paths before release.
- Sibling mechanisms: Supernormal Cue Audit · Salience Normalization Test · Variable-Reward Schedule Limit · Cue Intensity Cap Protocol
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Cue-Hijack Red-Team Review operates as a bounded evaluation of existing evidence or work that produces a finding or disposition because it tasks an adversarial reviewer with actively finding ways a design could capture attention, appetite, fear, or reward-seeking beyond user intent, focusing on vulnerable slices.
Independent corroboration: The frozen evidence defines Cue-Hijack Red-Team Review as 'Tasks an adversarial reviewer with actively finding ways a design could capture attention, appetite, fear, or reward-seeking beyond user intent, focusing on vulnerable slices', so its operative form is Assessment, Review & Assurance.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Military & Strategic Studies
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Independent adversarial red teams originate in military planning and were extended through security testing. Applying that attack posture to attention and reward cues synthesizes HCI, behavioral, and technology-governance lineages.
Related originating lineages:
- Human-Computer Interaction — Dark-pattern and interface research supplies concrete attention- and choice-manipulation attacks.
- Psychology — Behavioral research explains salience, fear, reward, scarcity, and vulnerable-slice susceptibility.
- Security Studies & Intelligence Analysis — Security red teaming generalizes independent adversarial search for exploit paths and unanticipated vulnerabilities.
- Ethics of Technology & AI Governance — Technology-risk governance extends red-team findings into release controls and protection of vulnerable users.
Review resolution: Independent adversarial red teams originate in military planning and were extended through security testing. Applying that attack posture to attention and reward cues synthesizes HCI, behavioral, and technology-governance lineages.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
- U.S. Army: Red Team Handbook
- NIST: Artificial-intelligence red teaming
- U.S. FTC: Bringing Dark Patterns to Light
Notes¶
The red-team review pairs naturally with the Supernormal Cue Audit: the audit checks against known cue categories, the red team invents the ones not yet on the list. Running only the audit leaves you blind to novel exploits; running only the red team leaves the routine ones uncatalogued. They are complementary, not redundant.
[n1] "Red teaming" originates in military and security practice, where an independent group role-plays an adversary to probe defenses that the defenders themselves are too close to see. Applied to cue design, the adversary's goal is hijack rather than intrusion, but the epistemic move — structured attack reveals what charitable review misses — is the same. ↩