Blinded Frame Review¶
Review protocol — instantiates Framing Effect Audit
Has reviewers evaluate competing presentations without knowing which one the team prefers or what result each produces, so the frame is chosen on stated criteria rather than on the answer it yields.
Blinded Frame Review is a procedure that strips each candidate presentation of its provenance before anyone judges it: reviewers see the alternate frames with the team's preferred version, the sponsor's identity, and the result each frame produced all masked. Its defining move is to break the link between which frame you like and which answer it gives, so selection is forced onto criteria fixed in advance — clarity, fairness, fidelity to the data — rather than onto the conclusion the team was hoping for. Where the other audit mechanisms detect and size frame effects, this one guards the decision about the frame against the audit's most corrosive failure: choosing the framing that flatters the desired result and calling it validated.
Example¶
A private foundation is deciding how to present the results of a job-training program it funded, ahead of a board vote on renewal. The program team has drafted three summaries of the same evaluation data: one leading with the employment-rate gain, one leading with cost-per-placement, one leading with participant retention. Left to themselves, the team would naturally reach for whichever frame makes renewal likeliest. Blinded Frame Review intervenes: a review panel drawn from unrelated program areas receives the three summaries with all identifying and outcome-flattering cues removed — no note on which the team prefers, no "recommended" label — and scores each only against a rubric agreed before the data were seen (does it state the denominator, does it disclose the comparison group, does it avoid a leading headline). The panel's ranking, plus a short written rationale, becomes the selection of record. If the team's favored frame wins on the rubric, fine; if it loses, the foundation has a documented, answer-independent reason for the frame it publishes.
How it works¶
- Fix the selection criteria first. Agree the rubric for a good presentation before any results are visible, so the standard cannot be reverse-engineered from the answer.
- Mask provenance and outcome. Present the alternate frames stripped of author preference, sponsor identity, and the result each yields, so reviewers cannot infer the "wanted" frame.
- Score independently, then reconcile. Reviewers rate each frame against the fixed rubric separately; disagreements are surfaced rather than averaged away, and the winning frame plus its rationale is recorded.
The distinguishing discipline is that it certifies the choice of frame as answer-independent — it is a bias control on selection, not a detector of the effect or a size estimate of it.
Tuning parameters¶
- Blinding depth — how much is masked (author only, or author plus outcome plus sponsor). Deeper blinding better severs the like/answer link but is harder to arrange and can strip context reviewers legitimately need.
- Reviewer independence — how far reviewers sit from the result's stakes. More distance reduces motivated scoring but costs domain fluency.
- Rubric specificity — how concrete the pre-fixed criteria are. Sharp criteria resist post-hoc bending; over-specified ones can miss a frame's real virtues.
- Reconciliation rule — whether split scores are debated, escalated, or mechanically aggregated. Debate surfaces genuine disagreement; aggregation is faster but hides it.
When it helps, and when it misleads¶
Its strength is that it targets answer-seeking frame choice directly, at the one point most audits leave unguarded — the moment someone picks the winning presentation — and leaves an accountable record that the pick was criteria-driven.[n1]
It can mislead when blinding is theater: if reviewers can guess the preferred frame from surface tells, or if the rubric was quietly written to favor it, the protocol launders a foregone conclusion into a "reviewed" one. It also cannot fix a frame slate that is uniformly slanted — masking three flattering summaries just picks the least obvious. The discipline that keeps it honest is to fix criteria before results, audit the blinding for leakage, and require that at least one genuinely unfavorable framing sit in the set being judged.
How it implements the components¶
Blinded Frame Review realizes the bias-controlled selection side of the archetype:
response_comparison_design— it structures how reviewer judgments are collected across frames, with masking as the design's core feature.frame_disclosure_or_selection_rule— its output is the rule by which the frame is chosen or disclosed: a pre-fixed rubric and a recorded, answer-independent decision.frame_neutrality_claim— the documented outcome supports an accountable claim that the selection was made on stated criteria rather than on the result.
It does not set the materiality bar for how large a shift matters (frame_effect_threshold) or map the audit to the downstream decision the frame feeds (downstream_use_context) — recording tested shifts against a threshold and their downstream use is Presentation Sensitivity Table. The separation is that a blinded review governs *who decides and how the frame is chosen, while the sensitivity table is the ledger of what was found and what it drives.*
Related¶
- Instantiates: Framing Effect Audit — Blinded Frame Review is the safeguard that keeps the audit's frame-selection step from becoming answer-seeking.
- Consumes: Presentation Sensitivity Table or a bare alternate-frame set supplies the candidate presentations the panel scores.
- Sibling mechanisms: Baseline Reference Swap · Dashboard Framing Review · Gain/Loss Frame Comparison · Order-Effect Check · Presentation Sensitivity Table · Survey Frame Split Sample · Visual Framing Audit · Wording Test
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Has reviewers evaluate competing presentations without knowing which one the team prefers or what result each produces, so the frame is chosen on stated criteria rather than on the answer it yields, making its operative form a bounded evaluation of existing evidence or work that produces a finding or disposition.
Independent corroboration: The frozen evidence defines Blinded Frame Review as 'Has reviewers evaluate competing presentations without knowing which one the team prefers or what result each produces, so the frame is chosen on stated criteria rather than on the answer it yields', so its operative form is Assessment, Review & Assurance.
Review outcome: Independent reviewer agreement; medium confidence.
Origin Attribution¶
Primary origin: Psychology
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Masking preferred outcomes while comparing presentations directly addresses framing and motivated-evaluation effects from judgment research.
Related originating lineages:
- Communication & Media Studies — Message framing and editorial presentation supply the object being compared.
- Statistics & Experimental Design — Blinding and predeclared criteria make the review resistant to outcome-based selection.
Review resolution: Psychology is the agreed primary lineage through framing and motivated-evaluation effects. Communication studies explains presentation frames and statistics supplies masking discipline; the comparative review is a cross-disciplinary Encyclopedia synthesis.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] Blind analysis (or masking) — a long-standing methodological safeguard in which those making a judgment are kept unaware of which choice would produce the hoped-for result, so that analytic and presentational decisions cannot be steered, consciously or not, toward it. ↩