Intersubjective Replication Check¶
Cross-observer replication check (review) — instantiates Sense-Experience Reduction Protocol
Tests whether independent observers, following the same access, arrive at the same experience—turning a private observation into a shared one.
Intersubjective Replication Check asks the question that separates an observation from an impression: would someone else, gaining the same access independently, encounter the same thing? Its defining move is independent repetition across observers. A single person's report might be a genuine encounter or a private artefact — an expectation, a mis-seeing, a one-off. This mechanism sends the same observation through two or more independent observers or sites and measures whether their experiences converge, then records that agreement as a reliability figure. It is how the archetype earns the word "objective": not by appeal to a thing-in-itself, but by showing that the experience is reproducible across the people who look.
Example¶
A physics group reports a value for a material's magnetic transition temperature. On its own that is one lab's reading. An Intersubjective Replication Check has the measurement repeated independently: samples from the same batch are sent to two other labs, each running the measurement blind to the others' results, each with its own apparatus and operator. The three come back at ≈211 K, 213 K, and 210 K — close, and within stated error.
The check's product is not the number itself but the agreement: three independent observers, accessing the same property separately, converged, and the spread is logged as a reliability record. Had one lab returned 250 K, the check would have surfaced a discrepancy demanding explanation — a calibration fault, a contaminated sample, a procedural difference — rather than letting a lone reading stand as fact. The observation graduates from "we measured" to "it reproduces."
How it works¶
- Fix a shared access, then split the observers. Hold the observation procedure constant while making the observers genuinely independent — different people, ideally different instruments and sites, blind to each other.
- Compare encounters, not conclusions. Check whether the observations converge, before anyone reconciles interpretations that could paper over a real disagreement.
- Quantify the agreement. Express convergence as a reliability figure (spread, concordance, an inter-rater statistic) rather than a yes/no.
- Treat divergence as signal. A discrepancy is a finding: it points to a hidden condition, a calibration gap, or an observer effect that a single run would have concealed.
Tuning parameters¶
- Independence strength — how thoroughly observers are insulated from each other (blinding, separate sites, different instruments). More independence makes agreement mean more but costs coordination.
- Number of replicates — how many observers or sites. More sharpen the reliability estimate and expose outliers but multiply cost.
- Agreement threshold — how close counts as "the same." Strict thresholds catch subtle divergence but may reject genuine, noisy convergence.
- Reliability metric — raw spread, concordance, or a chance-corrected statistic. Chance-correction is fairer for categorical calls but harder to communicate.
- Discrepancy protocol — whether a disagreement triggers adjudication, re-measurement, or investigation. Aggressive follow-up finds root causes but slows closure.
When it helps, and when it misleads¶
Its strength is converting private observation into public knowledge: it is the standing guard against the one-off reading, the wishful measurement, and the observer who sees what they expect, and it is the practical form of reproducibility as a warrant for an empirical claim. Reliability is made explicit through an inter-rater or inter-lab statistic rather than assumed.[1]
Its blind spot is that agreement is not truth: observers who share a bias, a mis-calibrated standard, or a common training will converge confidently on the same error, and the check will certify it as reliable. High agreement across observers who are not truly independent is the classic false positive — correlated errors masquerading as objectivity. It also measures reliability, not validity: reproducibly measuring the wrong thing still reproduces. The discipline is to make the observers genuinely independent (different instruments, blinding), to pair the reliability check with a validity check where the target allows it, and to read suspiciously perfect agreement as a warning rather than a triumph.
How it implements the components¶
intersubjective_alignment_check— its core operation: comparing independent observers' encounters to see whether they converge.calibration_and_reliability_record— logs the resulting agreement as a dated reliability figure, including the spread and any discrepancies found.
It does not test whether a proxy validly tracks its target against a criterion (Proxy Validity Audit), elicit a single participant's first-person report (Phenomenological Interview), or design the shared procedure the observers follow (Observation Protocol Design).
Related¶
- Instantiates: Sense-Experience Reduction Protocol — earns the archetype's claim to objectivity by showing an observation reproduces across independent observers.
- Consumes: Observation Protocol Design — supplies the shared procedure the independent observers each follow.
- Sibling mechanisms: Proxy Validity Audit · Phenomenological Interview · Observation Protocol Design · Instrument Readout Mapping · Disconfirming Condition Probe · Experience Prediction Matrix · Counterfactual Conditional Translation · Sensory Walkthrough · Operational Definition Table · Access Boundary Annotation · Ethical Non-Production Rule
Editorial Notes¶
Form Classification¶
Form family: Experiment, Test & Rehearsal
Rationale: Intersubjective Replication Check operates as a bounded trial, probe, simulation, or rehearsal that generates evidence from performance because it tests whether independent observers, following the same access, arrive at the same experience—turning a private observation into a shared one
Independent corroboration: The frozen evidence defines Intersubjective Replication Check as 'Tests whether independent observers, following the same access, arrive at the same experience—turning a private observation into a shared one', so its operative form is Experiment, Test & Rehearsal.
Nearest alternative: Assessment, Review & Assurance — Genuinely independent observers repeat a controlled observation procedure, making replication the operative evidence form.
Review outcome: Independent reviewer agreement; medium confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Convergent development
Present-day reach: Universal
Rationale: Independent-observer agreement and chance-corrected inter-rater reliability are established statistical validation practices.
Related originating lineages:
- Philosophy — Intersubjective testability materially supplies the epistemic move from private observation to publicly checkable claim.
- Psychology — Observer-coding and perception research materially provide many repeat-observation protocols.
Review resolution: Both independent reviews place the primary lineage in statistics_experimental_design. The queued differences (domain_reach_disagreement) concern secondary metadata rather than primary provenance. The final retains philosophy, psychology only where a reviewer supplied a formative-lineage rationale; this does not convert downstream applicability into origin. origin_mode=convergent because the reviewers document independently established or materially co-developing traditions. domain_reach=universal records application breadth separately from provenance.
Review outcome: Reconciled after independent review; medium confidence.
References¶
[1] Cohen, J. "A Coefficient of Agreement for Nominal Scales". Educational and Psychological Measurement 20(1), 37–46 (1960). Introduces kappa as a statistic for the reproducibility, or reliability, of judgments by two raters on nominal categories. registry ↩