Automation Bias¶

The human-factors regularity in which operators over-weight an automated system's outputs and under-weight contradicting evidence, so the human-automation team can perform worse on rare anomalies than the unaided operator would — through commission errors (acting on a false recommendation) or omission errors (staying passive when the system is silent).

Version: v4 · 2026-09-28 · History
Domain-specific #: 69
Domain group: Social Sciences
Origin domain: Psychology & Behavioral Sciences
Subdomain: Human Machine Interaction and Decision Support → Psychology & Behavioral Sciences

Core Idea¶

Automation bias is the regularity that operators supervising automated decision-support over-weight the system's outputs and under-weight contradicting evidence, producing errors that can exceed unaided operation. It has two failure modes: commission errors (acting on a false recommendation despite contradicting evidence) and omission errors (staying passive because the automation did not flag a real problem). Highly reliable automation acquires "machine-says-so" authority, displacing the operator's independent evaluation.

How would you explain it like I'm…

The Machine-Says-So Trap

When a helper machine is almost always right, people start trusting it too much. If it says something wrong, they may just do it anyway, even when they could see it was wrong. And if something goes wrong that the machine didn't point out, they might not notice at all.

Trusting the Computer Too Much

Automation Bias is when people who work with computer helpers trust the computer too much. If the computer gives wrong advice, they may follow it even though other clues show it's wrong. And if the computer fails to warn them about a problem, they may not notice the problem at all, because they were waiting for the computer to tell them. This happens more when the computer is usually right, when people are very busy, and when the computer is the main thing they look at. Sometimes people using a computer helper this way make more mistakes than they would have with no helper at all.

Over-Reliance on Automated Aids

Automation bias is the tendency of people who supervise or use automated decision aids to give too much weight to what the system says and too little to conflicting evidence around them, sometimes making more errors than they would without the aid. It shows up as commission errors, following a wrong recommendation despite contrary evidence, and omission errors, missing a real problem because the system didn't flag it. It has been seen in aviation, medicine, industrial control, and supervising self-driving cars. It's strongest when the automation is highly reliable (so trust grows beyond what's justified), when workload is high, and when the system is the main, most noticeable source of information. It links to skill loss from rarely doing the task manually and to alarm fatigue from too many false alarms. Better design shows uncertainty, highlights conflicting evidence, keeps people practicing manual skills, and builds in cross-checks.

Automation bias is the empirical regularity that operators supervising or assisted by automated decision-support systems overweight the system's outputs and underweight contradicting environmental evidence, with error rates that can exceed unaided performance. It has two distinguishable failure modes: commission errors, acting on an incorrect automated recommendation despite contrary evidence, and omission errors, failing to detect a real problem the automation did not flag. It is documented in aviation, clinical decision support, process control, and autonomous-vehicle supervision, from Mosier and Skitka's 1990s cockpit cross-check studies to radiologists accepting incorrect AI nodule classifications. The consistent structure is that automation reliable in the common case acquires 'machine-says-so' authority, displacing independent evaluation and eroding vigilance for rare failures. Three conditions concentrate the effect: high reliability inflating trust, high workload or divided attention, and the automation being the salient, sole authoritative source. It interacts with out-of-the-loop skill atrophy and with alarm fatigue from high false-positive rates. The remedy is calibrated human–automation interaction: outputs with uncertainty estimates, foregrounded contradicting evidence, periodic manual practice, and team protocols with explicit cross-check and escalation paths.

Scope of Application¶

Automation bias lives across human-factors subfields sharing one substrate — a human supervising an authoritative automated source under divided attention, with a rare anomaly against a reliable common case.

Aviation human factors — instrument cross-check failure, autopilot-mode confusion, GPS over-reliance.
Clinical decision support — radiologists agreeing with wrong AI flags; missed unflagged interactions.
Process and supervisory control — dispatchers drifting into passive supervision, then failing at anomalies.
Autonomous-vehicle supervision — driver inattention in conditions the system was never rated for.
AI assistants for knowledge work — over-trust in LLM outputs in programming, law, and writing.

Clarity¶

Naming automation bias makes legible a failure that automation actively creates: a team can perform worse on anomalous cases than the unaided operator, even when the automation is individually more accurate. It reframes the unit of analysis from component to team, and warns that a more reliable system can deepen the bias by earning more unwarranted authority. It also separates commission from omission, and automation bias proper from its companions — skill atrophy and alarm fatigue.

Manages Complexity¶

Case by case, these failures look nothing alike — a pilot on a stale display, a radiologist accepting a wrong flag, a dispatcher passive through a fault. The construct reduces the standing question to four trackable quantities: common-case reliability, operator load, automation salience, and whether the anomaly arrives with the automation asserting-false or silent. That last classifier sorts the incident into commission or omission and, with it, the applicable intervention class — without re-modeling the operator's full attention-and-trust dynamics.

Abstract Reasoning¶

The construct licenses diagnostic reasoning — from a team underperforming its unaided baseline back to mis-set trust, then sorting by the assert-false-versus-silent signature and disentangling trust-miscalibration from skill atrophy from alarm fatigue. Its interventionist moves attach a predicted direction to each lever (uncertainty estimates, foregrounded contradicting channels, manual-mode practice), including the counterintuitive non-monotonic prediction that a more reliable component can worsen the team. It also supports boundary-drawing (requires a human in the loop with rare-event structure) and order-of-events forecasting.

Knowledge Transfer¶

Within human-factors research the construct transfers as mechanism, but its home is one substrate — a human supervising an authoritative automated source under divided attention against rare anomalies — deployed across aviation, clinical, process-control, vehicle, and now AI-assistant settings; the commission/omission taxonomy, four parameters, and non-monotonic prediction carry intact. Remove the human or the rare-event structure and there is no automation bias — the failure is plain component error. What little is portable belongs to the parent Authority Bias (an authoritative source earning more credence than warranted), plus thinner Bias and Attention; "automation bias" itself stays home-bound.

Relationships to Other Abstractions¶

Current abstraction Automation Bias Domain-specific

Parents (1) — outgoing typed relations

Automation Bias is a kind of Authority Bias Domain-specific

Automation bias is authority bias specialized to an automated source that has acquired machine-says-so authority for a human supervisor.

Hierarchy path (1) — routes to 1 parentless root

Automation Bias → is a kind of Authority Bias → is a kind of Bias

Neighborhood in Abstraction Space¶

Automation Bias sits in a sparse region of the domain-specific corpus (65^th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.

Family — Organizational & Operational Failure Modes (38 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-10-25