{"schema_version":1,"research_id":"eoa_inverse_innovation_exp06_external_evaluation_20260803","source_assessment_id":"predictive_residual_processing__sociology_anthropology:P2:v0","cell_id":"predictive_residual_processing__sociology_anthropology","search_queries":["qualitative research researcher influence reflexivity prospective predictions fieldwork intervention effects","participatory action research reflexivity researcher intervention co-production longitudinal ethnography","official qualitative research reporting standards reflexivity researcher influence SRQR COREQ","ethnography member checking participant validation limitations researcher influence","site:ukri.org ESRC framework research ethics reflexivity participatory research researcher role","site:americananthro.org ethics ethnography researcher influence community collaboration code ethics","qualitative research preregistration prospective predictions reflexivity fieldwork study protocol","researcher effects intervention ethnography reflexivity action research causal attribution","\"prospective\" reflexivity ethnography researcher effects","\"researcher effects\" \"participant observation\" prospective","\"prediction error\" reflexivity qualitative research","\"researcher intervention\" \"field diary\" ethnography","site:americananthro.org ethics code do no harm informed consent anthropological research official","site:americananthro.org statement ethics community collaboration anthropological research researchers obligations","\"Ethnographic methods for process evaluations\" Savage open access","\"Benefits of Observer Effects\" Lessons from the Field full text","\"Be(com)ing a reflexive researcher\" full text pdf","site:europepmc.org/articles/PMC3032358","site:europepmc.org/articles/PMC4855482","\"Ethnographic methods for process evaluations of complex health behaviour interventions\" DOI","site:sussex.figshare.com \"Be(com)ing a reflexive researcher\"","site:research.manchester.ac.uk \"Be(com)ing a reflexive researcher\"","\"Be(com)ing a reflexive researcher\" \"33-45\""],"sources":[{"source_id":"S1","title":"Framework for research ethics — Our core principles","publisher":"UK Research and Innovation, Economic and Social Research Council","url":"https://www.ukri.org/councils/esrc/guidance-for-applicants/research-ethics-guidance/framework-for-research-ethics/our-core-principles/","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2025-05-12","accessed_at":"2026-08-03","claims_supported":["ESRC identifies researchers, research organizations, and research ethics committees as responsible actors.","ESRC-funded social research is expected to protect rights and dignity, minimize harm, maintain transparency and integrity, define accountability, and address conflicts throughout the project lifecycle.","An ethics committee is a credible authorizer for the proposed shadow pilot, but project-specific approval remains necessary."]},{"source_id":"S2","title":"Anthropological Ethics","publisher":"American Anthropological Association","url":"https://americananthro.org/about/anthropological-ethics/","source_class":"OFFICIAL_GUIDANCE","publication_date":"2012-11-01","accessed_at":"2026-08-03","claims_supported":["AAA professional principles require doing no harm, openness, informed consent and permissions, weighing obligations to collaborators, record protection, and respectful relationships.","These duties support participant challenge, complete-record preservation, consent bypasses, and ethics oversight rather than automated cancellation of sensitive observations."]},{"source_id":"S3","title":"Ethnographic methods for process evaluations of complex health behaviour interventions","publisher":"Trials, Springer Nature","url":"https://link.springer.com/article/10.1186/s13063-016-1340-2","source_class":"AUTHORITATIVE_SECONDARY","publication_date":"2016-05-04","accessed_at":"2026-08-03","claims_supported":["Qualitative process evaluation is used to understand how interventions are delivered, received, and produce outcomes.","Ethnography can add contextual and theory-building value but requires reflexivity, adaptation to the research question, and sufficient resources.","Research access, trust, rapport, and researcher participation affect what is observed and reported."]},{"source_id":"S4","title":"Benefits of “Observer Effects”: Lessons from the Field","publisher":"Qualitative Research, SAGE Publications","url":"https://doi.org/10.1177/1468794110362874","source_class":"PRIMARY_RESEARCH","publication_date":"2010-06-01","accessed_at":"2026-08-03","claims_supported":["Researcher or observer effects are a visible and longstanding methodological concern in ethnography.","Researcher-influenced performances may themselves be meaningful data rather than removable contamination.","The paper directly cautions against assuming that researcher-associated change can be cleanly subtracted from the phenomenon under study."]},{"source_id":"S5","title":"Be(com)ing a reflexive researcher: a developmental approach to research methodology","publisher":"Open Review of Educational Research, Taylor & Francis","url":"https://doi.org/10.1080/23265507.2017.1300068","source_class":"PRIMARY_RESEARCH","publication_date":"2017-03-16","accessed_at":"2026-08-03","claims_supported":["Prospective reflexivity examines how the researcher affects research, while reflexivity more generally involves ongoing mutual shaping of researcher and context.","Reflexive practice already asks researchers to step back, theorize their action, and document decisions.","It does not specify locked action-level consequence predictions, residual routing, or quantitative cancellation thresholds."]},{"source_id":"S6","title":"Preregistering Qualitative Research: A Delphi Study","publisher":"International Journal of Qualitative Methods, SAGE Publications","url":"https://doi.org/10.1177/1609406920976417","source_class":"PRIMARY_RESEARCH","publication_date":"2020-12-09","accessed_at":"2026-08-03","claims_supported":["Qualitative preregistration can timestamp expectations, expose design and analysis decisions, and distinguish ex-ante beliefs from ex-post interpretation.","The Delphi study developed a template using input from active qualitative researchers and methodologists.","The authors caution that preregistration is not useful for every qualitative approach and cannot be presumed generally useful across all qualitative communities."]},{"source_id":"S7","title":"Member Checking: A Tool to Enhance Trustworthiness or Merely a Nod to Validation?","publisher":"Qualitative Health Research; indexed by the U.S. National Library of Medicine","url":"https://pubmed.ncbi.nlm.nih.gov/27340178/","source_class":"PRIMARY_RESEARCH","publication_date":"2016-07-10","accessed_at":"2026-08-03","claims_supported":["Member checking is an established way to return interpretations to participants for accuracy, resonance, and elaboration.","Synthesized member checking explicitly addresses co-constructed knowledge.","Participant review is therefore established prior art, although it does not create prospective action-consequence predictions or residual queues."]},{"source_id":"S8","title":"Participatory action research","publisher":"Nature Reviews Methods Primers, Springer Nature","url":"https://www.nature.com/articles/s43586-023-00214-1","source_class":"AUTHORITATIVE_SECONDARY","publication_date":"2023-04-27","accessed_at":"2026-08-03","claims_supported":["Participatory action research already combines systematic inquiry, collaborative analysis, and action led by people affected by the issue.","Documented challenges include co-option, power inequality, institutional mismatch, and decentralizing researcher control.","Critical communities, critical reflexivity, situated expertise, and redistribution of power are established safeguards relevant to the proposal."]}],"problem_evidence":{"support":"STRONG","rationale":"The underlying problem visibly exists: ethnographic observation and researcher action can alter conduct and knowledge production, and intervention process evaluations must interpret researcher and contextual effects. However, the literature also treats these effects as meaningful, co-produced data; it does not validate the proposal's additive separation of predicted researcher effects from an unexplained remainder.","source_ids":["S3","S4","S5","S8"]},"stakeholder_evidence":{"support":"MODERATE","rationale":"Identifiable adopters are longitudinal qualitative research teams and principal investigators; identifiable authorizers or funders include institutional research ethics committees and ESRC. UKRI and AAA express needs for transparent, accountable, reflexive, consent-respecting research. No source documents demand for this particular prediction-residual workflow or commitment by a tenant assembly, institution, or funder to pilot it.","source_ids":["S1","S2","S3","S6"]},"prior_art":{"proximity":"ADJACENT_PRIOR_ART","closest_analogues":[{"name":"Prospective and retrospective reflexivity","similarity":"Already makes the researcher's effects and mutual shaping with the field an explicit, continuing object of analysis.","remaining_difference":"Does not lock bounded consequence predictions before each action, compute structured prediction errors, or use residuals to allocate review attention.","source_ids":["S5"]},{"name":"Qualitative preregistration and audit trails","similarity":"Prospectively timestamps expectations and decisions and exposes movement from ex-ante beliefs to ex-post interpretation.","remaining_difference":"Typically registers study-level plans rather than maintaining a synchronized action-level forward model with residual, fallback, and raw-audit machinery.","source_ids":["S6"]},{"name":"Ethnographic process evaluation","similarity":"Examines intervention delivery, mechanisms, context, researcher participation, and how outcomes arise.","remaining_difference":"Retains contextual interpretation and triangulation rather than treating a model-relative unexplained remainder as the primary attribution-review signal.","source_ids":["S3"]},{"name":"Member checking or participant validation","similarity":"Allows participants to contest, elaborate, or reshape researcher interpretations of co-constructed knowledge.","remaining_difference":"Is an interpretive validation practice, not a prospective prediction-and-residual loop.","source_ids":["S7"]},{"name":"Participatory action research","similarity":"Treats research and action as co-produced and uses collaborative analysis, critical reflexivity, and community accountability.","remaining_difference":"Does not ordinarily attempt action-effect cancellation; its non-additive account of co-production is also the strongest conceptual challenge to the proposal.","source_ids":["S8"]}],"distinctive_claim_remaining":"After accounting for reflexive journals, qualitative preregistration, process evaluation, member checking, and participatory analysis, the remaining testable claim is that locking bounded predictions about each authorized researcher action and routing structured mismatches to review will improve attribution traceability and reduce unsupported causal interpretations, relative to ordinary full-record reflexive analysis, without increasing participant disagreement, subgroup-specific misses, privacy incidents, or total analytic burden.","confidence":"MODERATE"},"implementation_evidence":{"support":"MODERATE","rationale":"The component practices—prospective documentation, audit trails, contextual ethnography, independent review, member checking, community challenge, and ethics oversight—are implementable. The novel assembly is workflow-heavy rather than technically difficult. Feasibility is unresolved for semantic coding reliability, recursive or delayed effects, secure handling of identifiable dissent, participant acceptability, model-version discipline, analyst blinding, and site-specific ethics/data authorization. The proposal's full-record fallback is essential because researcher and participant actions may not be additively separable.","source_ids":["S1","S2","S3","S4","S5","S6","S7","S8"]},"scores":{"meaningful_impact":{"score":3,"rationale":"On a 1–5 scale, better traceability could materially improve intervention-adjacent fieldwork, but prevalence and realized harm from the stated attribution problem are not quantified.","source_ids":["S3","S4","S5"]},"stakeholder_pull":{"score":3,"rationale":"Funders, ethics committees, and professional bodies clearly value transparency, reflexivity, participant rights, and accountability, but no adopter has requested this exact architecture.","source_ids":["S1","S2","S6"]},"incremental_advantage":{"score":2,"rationale":"Prospective action-level predictions may constrain hindsight repair, but added value over preregistration, reflexive logs, process evaluation, member checking, and collaborative analysis is untested and may be offset by workload or false separability.","source_ids":["S3","S5","S6","S7","S8"]},"distinctiveness_plausibility":{"score":2,"rationale":"The action-copy/residual combination is a distinctive arrangement in the searched literature, but most constituent practices are established and the search cannot establish world novelty.","source_ids":["S3","S5","S6","S7","S8"]},"technical_implementability":{"score":3,"rationale":"A shadow workflow can be built with ordinary secure qualitative-research tools, versioned forms, coding schemas, and blinded review; reliable semantic comparison and bounded modeling of recursive effects remain difficult.","source_ids":["S3","S5","S6","S7"]},"adoption_authority_feasibility":{"score":3,"rationale":"A PI can operate a shadow analysis only within approved data uses; an ethics committee, data steward, and participant governance process provide credible authorization routes, but no project-specific determination is available.","source_ids":["S1","S2"]},"evidence_readiness":{"score":3,"rationale":"A six-meeting comparative shadow pilot is well bounded and measurable, but the decisive evidence requires live field records, analyst labor, and participant or advisory review.","source_ids":["S3","S6","S7"]},"safety_net_benefit":{"score":4,"rationale":"Complete-record retention, participant challenge, independent raw review, protected bypasses, and full-context fallback directly address known ethical and epistemic risks if they are genuinely independent and enforced.","source_ids":["S1","S2","S4","S7","S8"]},"scalability":{"score":2,"rationale":"Forms and versioning can be reused, but predictions, semantic coding, ethics review, community interpretation, and raw audits are site- and relationship-specific and labor intensive.","source_ids":["S3","S5","S8"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"50K_TO_250K","scope":"Six-meeting, single-site shadow pilot with preregistration, two analysts, transcription or authorized record preparation, community-advisory review, ethics/data-governance work, secure storage, and evaluation reporting.","confidence":"LOW","assumptions":["No new participant-facing intervention or recording is introduced.","Two analysts contribute approximately 0.2–0.4 FTE each for six months.","Existing authorized recordings and meeting infrastructure are available.","Community advisers and participants are compensated for additional review.","The estimate is a resource-equivalent planning range, not a vendor quote or observed project cost."],"source_ids":["S1","S2","S3","S6","S7"]},"initial_deployment_startup":{"band_2026_usd":"50K_TO_250K","scope":"One-study deployment setup after a successful pilot: protocol amendment, secure templates and versioning, coding manual, staff training, calibration set, governance rules, and fallback testing.","confidence":"LOW","assumptions":["Deployment remains decision-support for analysts and does not automate causal conclusions.","The institution already has secure qualitative-data infrastructure.","Community governance is incorporated into the existing study rather than created as a new organization.","Pricing evidence was not located; the band reflects labor and governance scope."],"source_ids":["S1","S2","S5","S6","S8"]},"operational_launch":{"band_2026_usd":"250K_TO_1M","scope":"Launch across two to four longitudinal field studies, including dedicated coordination, analyst calibration, independent audits, participant-governance support, privacy review, and cross-site evaluation.","confidence":"LOW","assumptions":["Each site preserves full records and appoints an independent reviewer.","Local ethics and data-use determinations are required.","No predictive output changes participant-facing fieldwork.","Costs rise mainly with skilled analyst time and governance rather than software."],"source_ids":["S1","S2","S3","S7","S8"]},"annual_recurring":{"band_2026_usd":"250K_TO_1M","scope":"Annual operation for a small multi-study program: prediction preparation, dual coding, discrepancy review, community consultation, raw-record audits, secure data stewardship, retraining, and evaluation.","confidence":"LOW","assumptions":["Two to four active studies run concurrently.","At least one independent qualitative analyst and part-time data steward are retained.","High-risk cases receive full-context review rather than automated filtering.","No validated workload or salary model was found, so actual cost could cross an adjacent band."],"source_ids":["S1","S2","S3","S7","S8"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"YES","reason":"Direct methodological literature documents observer effects, reflexive mutual shaping, and the need to interpret interventions in context.","source_ids":["S3","S4","S5"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"Qualitative research teams and PIs are credible adopters; research ethics committees and ESRC are identifiable authorizers or funders with explicit expectations for transparent, accountable, ethically reflective social research.","source_ids":["S1","S2"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The incremental claim can be tested against ordinary full-record reflexive analysis using attribution traceability, unsupported interpretations, disagreement, harms, and analytic time.","source_ids":["S3","S5","S6","S7"]},"bounded_next_evidence_step":{"status":"YES","reason":"The proposed six-meeting shadow pilot has a fixed scope, same-record comparator, blinded conventional analysis, community review, measurable outcomes, and explicit falsifiers.","source_ids":["S6","S7","S8"]},"no_unresolved_safety_or_authority_stop":{"status":"UNCERTAIN","reason":"Shadow mode and full-record fallback reduce risk, but site-specific ethics approval, consent compatibility, raw-record access, privacy protection for identifiable residuals, participant acceptability, and authority over model boundaries are unresolved.","source_ids":["S1","S2","S7","S8"]},"credible_cost_scope_and_range":{"status":"YES","reason":"All four ranges have bounded operational scopes and explicit assumptions, but confidence is low because no direct price or observed labor evidence was found.","source_ids":["S1","S3","S6","S7","S8"]}},"next_evidence_step":"Secure a tenant-assembly research partner and project-specific ethics/data-steward determination, then preregister a six-meeting shadow pilot. Before each already-approved researcher action, lock a bounded prediction, null expectation, rival pattern, uncertainty, model version, and validity horizon. Preserve the complete authorized record. Randomly assign same-record meeting analyses to (A) the residual workflow and (B) ordinary full-record reflexive analysis by an analyst blinded to predictions; cross over analysts where feasible. Have the existing community advisory group review discrepant interpretations. Compare preregistered traceability ratings, unsupported causal attributions, analyst and participant/advisory disagreement, role-specific misses, structured residuals, fallback frequency, privacy incidents, and total analytic time. Falsify the incremental claim if the residual arm does not improve traceability or unsupported-attribution outcomes, if gains disappear after accounting for time, or if it increases participant disagreement, subgroup errors, privacy risk, over-cancellation, or mechanical causal language.","blocking_evidence":["No comparative field evidence shows that action-level residual processing outperforms ordinary reflexive logs, preregistration, process evaluation, member checking, or collaborative analysis.","No evidence establishes the prevalence or material magnitude of the stated attribution problem specifically in longitudinal tenant-assembly fieldwork.","No tenant assembly, principal investigator, ethics committee, or funder has expressed demand for or committed to this particular pilot.","Project-specific consent, ethics, privacy, data-access, and community-governance determinations are unavailable.","Semantic coding reliability, analyst blinding feasibility, delayed and recursive effect handling, subgroup error, participant acceptability, and net analytic workload require live testing.","The resource bands are planning estimates without bids, salary modeling, or observed implementation costs."],"research_disposition":"PARTNERED_RESEARCH_PROGRAM","world_novelty_boundary":"The search supports only a contrast against the eight reviewed sources: no exact action-copy, forward-model, structured-residual, independent-raw-audit workflow was identified, while several close component practices are established. World novelty, patentability, freedom to operate, market size, and realized impact remain unmeasured.","arm":"COMPLETE_PROPOSAL_PORTFOLIO","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":false,"material_progress_observed":true,"progress_targets":["Obtain a written project-specific ethics and data-governance determination for shadow use of complete meeting records.","Recruit a consenting fieldwork partner and establish participant or community-advisory authority over protected classes, model boundaries, disagreement review, and halt decisions.","Preregister operational definitions, comparator procedures, analyst blinding, primary outcomes, subgroup checks, analytic-time accounting, and falsification thresholds.","Run the six-meeting same-record comparative pilot without changing meetings or using residuals for causal claims.","Report whether any traceability benefit survives workload adjustment and whether recursive co-production makes cancellation conceptually or operationally invalid.","Replace planning cost bands with observed analyst time, governance effort, audit burden, and privacy-control costs."],"reason":"Web evidence verifies the methodological problem, credible authorizers, adjacent prior art, and a bounded test, but cannot establish comparative benefit, participant acceptability, semantic reliability, safety, or net workload. Those questions require proprietary field records, live workflow execution, and participant/community review; therefore the appropriate terminal recommendation is an empirical-research stop."},"proposal_index":2}