Skip to content

First-Attempt Discovery Test

Behavioral discovery test — instantiates Latent Affordance Surfacing

Puts a fresh user in front of the real interface with a goal and no hints, and measures whether they discover an already-available capability unaided — turning "is it findable?" into a repeatable number.

Inspection can predict that a capability is hard to find; only a person trying can prove it. First-Attempt Discovery Test recruits representative users who have never used the feature, gives each a realistic goal, and watches — silently, without prompting — whether they locate and recognize the capability on their first unaided attempt. Its defining move is that it measures perceived availability behaviorally: not whether an inspector judges the cue visible, but whether a real newcomer, on first encounter, forms the belief that the option exists and acts on it. The output is a discovery rate and the exact point where the search collapsed, refreshed each release so the number can be trended rather than argued.

Example

A music-streaming app has shipped a sleep timer — it exists, buried under a now-playing overflow menu. Support keeps hearing "I wish it would just stop after I fall asleep." Rather than debate whether the control is "obvious," the team runs a First-Attempt Discovery Test: eight first-time users are each handed a phone and told "set it so the music stops on its own after you doze off," then left alone. About three of eight find it within a minute; the rest tap into Settings, hunt the playback screen, and give up at the overflow "⋯" they never think to open. The test records first tap, path taken, time-to-find, and the give-up point.

That ≈3-of-8 discovery rate is the gap made into evidence — and re-running it after any redesign says, in one number, whether the fix actually moved discovery or just felt better in a review.

How it works

  • One unaided first encounter. Recruit users new to the capability, give a goal framed in their words (never the feature's name), and offer no hints — a second attempt is contaminated, so the first is the whole measurement.
  • Instrument the attempt. Record success/failure, first-click location, path, time-to-find, and where the user abandoned — the abandonment surface is often more useful than the win rate.
  • Report a rate, then re-run. Express discoverability as a rate with its task and sample, and repeat it on the same task after each change so the measurement becomes a loop, not a one-off verdict.

The distinguishing trait is behavioral first-contact: it can only speak about capabilities a user is given a reason to seek, but for those it reports what people actually do, not what a layout implies.[1]

Tuning parameters

  • Task framing — how much the prompt telegraphs the answer. A task that names or hints at the feature manufactures success; blind, goal-only phrasing is the single biggest validity dial.
  • Participant freshness — true novices versus returning users. Only genuine first-encounter users measure discovery; returners measure recall.
  • Success criterion — found versus found-and-completed. The stricter bar catches cues that attract a click but strand the user.
  • Sample size — more users narrow the confidence on the rate; a handful catches the obvious cliff cheaply but can't grade a near-miss.
  • Moderation — moderated (probe the give-up moment) versus unmoderated (cleaner, cheaper, scales). Moderation adds why at the cost of how many.

When it helps, and when it misleads

Its strength is settling "it's obviously there" disputes with evidence and pinning the exact surface where discovery fails — and, re-run, it is the honest scorecard for whether a surfacing change worked. It is the behavioral confirmation the inspection-based audit's gap list is waiting for.

Its failure modes are those of any usability test. A leading task prompt is the classic misuse — write the task so it names the feature and you "prove" discoverability you engineered. It measures only what users are prompted to seek, so it is blind to capabilities no one thinks to want; small samples over-read noise; and it scores finding, not whether the capability was worth finding. The discipline that guards against this is blind, goal-only task phrasing piloted before use, reporting the give-up paths alongside the win rate, and treating the number as a measurement to trend rather than a one-time pass.

How it implements the components

First-Attempt Discovery Test realizes the behavioral measurement side of the archetype — it confirms and tracks the gap rather than inventorying or closing it:

  • discovery_measurement_loop — its core: a repeatable instrument that yields a discovery rate per task and can be re-run each release to trend whether surfacing is improving.
  • effective_option_set_gap — the discovery rate is the gap measured behaviorally: the share of real users for whom an available capability is effectively absent on first encounter.

It does not build the capability inventory or the surface map, and it does not judge visibility by inspection — those are Affordance Visibility Audit; nor does it design or place any cue (that is Signage and Wayfinding Revision, Contextual Hint or Tooltip). It measures; others map and fix.

Notes

A discovery test is active and prompted: it can only measure capabilities users are given a goal to seek. Its natural complement is a passive method — Failed-Search and Helpdesk Query Analysis — which catches the capabilities users wanted so badly they searched or filed a ticket, including ones no test task would have thought to prompt. Run the test to confirm known gaps and the passive analysis to surface unknown ones.

References

[1] First-click testing — observing where a user first clicks (or taps) when given a task — is an established usability technique, on the well-supported premise that a right first move strongly predicts task success. It is used here only as a means of measuring unaided discovery; the numbers in the example are illustrative, not drawn from any study. ↩