Adoption Analytics and Field Review¶
Feedback method — instantiates Sociotechnical Integration
Combines usage data with field evidence about burden, workarounds, trust, outcomes, and unintended consequences.
Adoption Analytics and Field Review is the post-deployment feedback method that keeps a live system honest by refusing to trust the usage dashboard alone. Its defining move is triangulation: it pairs the quantitative signal — who uses what, how often, how completely — with field evidence gathered by watching and asking, precisely because adoption metrics can glow green while the real work is failing underneath. Logins and record counts say a system is used; only field review reveals whether it is working — whether people trust it, what it costs them per task, which workarounds they have built around it, and whether the outcomes actually improved. Crucially, it reads the gap between the two as diagnosis: a healthy metric over an unhealthy floor usually means the metric is being fed rather than the system being used, and the workaround that produces that gap points straight at a misaligned incentive or an unbudgeted burden.
Example¶
A software company rolls out a new CRM to its 300-person sales force, requiring every opportunity to be logged with a dozen fields. Six weeks in, the adoption dashboard is a triumph: 96% weekly active users, thousands of records created, activity trending up. Leadership is ready to call it a win. Adoption Analytics and Field Review looks harder — and the field tells a different story.
Ride-alongs and interviews find that reps batch-fill the CRM every Friday afternoon with thin, half-real entries, keep their actual pipeline in a private spreadsheet they trust, and lose about twenty minutes a day to required fields they consider pointless. Trust in the CRM's forecast is low, and — the outcome that matters — the forecasts it produces are worse than the old spreadsheet's. The method quantifies the burden, audits the workarounds, and traces them to their root: reps are rewarded for closed deals, not data hygiene, so honest logging is time stolen from selling. The green dashboard was measuring compliance theater. The review's output is not a grade but a feedback packet — burden numbers, the shadow-spreadsheet finding, the incentive diagnosis — routed back to the implementation team to cut fields and change what the metric rewards.
How it works¶
The method works by never letting one signal stand alone:
- Instrument usage, but distrust it. Collect the quantitative signal — frequency, depth, completeness — and treat a healthy number as a question, not an answer.
- Go to the field. Observe and interview real users to surface trust, workarounds, hidden labor, and the outcomes the dashboard cannot see.
- Read the gap. Where usage looks fine but the floor does not, find the workaround closing the gap — it is evidence of misfit, not of noncompliance.
- Diagnose to root and route back. Trace burden and workarounds to the incentive or design that causes them, and feed the finding back into redesign rather than filing it as a report.
Tuning parameters¶
- Quant/qual balance — how much weight rests on dashboards versus field study; leaning on analytics scales cheaply but misses lived reality, while field study is rich but costly and small-sample.
- Review cadence — how often the loop runs; frequent review catches drift early but burdens both reviewers and users with observation.
- Field depth — from a quick survey to embedded observation; deeper methods find the hidden workarounds surveys miss, at higher effort.
- Metric skepticism — how hard the method probes whether a good number is being gamed; more skepticism catches vanity signals but can read too much into honest success.
- Feedback latency — how fast findings reach the people who can act; low latency keeps the system adaptive but demands a standing channel into redesign.
When it helps, and when it misleads¶
Its strength is that it is the archetype's early-warning system: it catches metric-only "success" before it hardens into a shadow system, and by measuring burden and tracing workarounds it turns vague dissatisfaction into an actionable diagnosis. It is the mechanism that keeps rollout from being treated as the finish line.
Its failure mode is trusting the very metric it exists to interrogate — Goodhart's law warns that once a usage number becomes the target, people optimize the number rather than the work, and the dashboard measures compliance theater instead of adoption.[n1] The classic misuse is declaring victory from analytics alone and skipping the field entirely, so the private spreadsheets and the twenty-minute daily tax stay invisible until the forecast fails. A subtler misuse is field review that documents beautifully but connects to no redesign channel — evidence that changes nothing. The guarding discipline is to always pair the number with a look at the floor, to read workarounds as data about system fit rather than as user failing, and to wire every finding to an owner who can act.
How it implements the components¶
This method fills the post-deployment learning side of the archetype — the feedback loop, not the pre-launch argument or the redesign itself:
adoption_feedback_loop— its core function: tracking field use, nonuse, workarounds, trust, and outcomes after launch, keeping the system adaptive rather than treating rollout as the end.human_burden_budget— it measures in the field what the system actually costs people per task, checking the burden against what was budgeted at design time.incentive_alignment_check— it reads workarounds as evidence of where following the system is locally irrational, tracing the misfit back to the incentive that causes it.
It does NOT make a forward-looking argument that the system is acceptably safe (safety_or_risk_case — that is its nearest twin, Safety Case Review, which reasons before deployment rather than measuring after it), nor redesign the process the findings point to (joint_redesign_rule — Joint Process and System Redesign).
Related¶
- Instantiates: Sociotechnical Integration — the method keeps the integrated system adaptive by feeding field reality back into the design.
- Consumes: Workflow-Integrated Tooling supplies the deployed tool whose real-world use and burden this review measures.
- Sibling mechanisms: Sociotechnical Design Workshop · Joint Process and System Redesign · Workflow-Integrated Tooling · Training and Enablement Rollout · Human-in-the-Loop Operating Model · Cross-Functional Implementation Team · Safety Case Review
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: The mechanism combines usage data with field evidence about burden, workarounds, trust, outcomes, and unintended consequences, so its operative form is a bounded assessment of existing evidence or work.
Independent corroboration: The frozen evidence defines Adoption Analytics and Field Review as 'Combines usage data with field evidence about burden, workarounds, trust, outcomes, and unintended consequences', so its operative form is Assessment, Review & Assurance.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Organizational & Management Science
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Post-deployment review of whether a change is truly embedded in work belongs to organizational change and implementation management.
Related originating lineages:
- Data Science & Analytics — Usage, completion, relapse, and outcome measures supply the quantitative adoption evidence.
- Ethnography & Qualitative Methods — Observation and interviews reveal workarounds, trust, and burdens that telemetry cannot see.
- Human-Computer Interaction — Usability and actual-use studies contribute the sociotechnical interpretation of friction.
Review resolution: The reviewers exactly agree that this is an organizational feedback synthesis combining analytics, field inquiry, and interaction evidence. The reported ambiguity only warns that quantitative adoption and qualitative workarounds must remain jointly interpreted.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] Goodhart's law — "when a measure becomes a target, it ceases to be a good measure." Once a usage statistic is what people are held to, they optimize the statistic rather than the underlying work, so a rising adoption number can reflect gaming rather than genuine use — exactly the illusion field review exists to pierce. ↩