{"schema_version":1,"research_id":"eoa_inverse_innovation_exp06_external_evaluation_20260803","source_assessment_id":"representation_independent_interface_contract__sociology_anthropology:P4:v0","cell_id":"representation_independent_interface_contract__sociology_anthropology","search_queries":["site:unece.org GSIM Sampling Frame Sample statistical standard","site:census.gov Frames Program sampling frame modernization administrative records","probability sampling frame duplicates inclusion probabilities research","site:aapor.org sampling frames coverage duplication address based sampling report","UNECE Generic Statistical Information Model Sampling Frame object official","GSIM Sampling Frame information object official PDF","survey sampling software reproducible random seed official sampling frame documentation","site:stats.govt.nz sampling frame duplicate coverage survey design guide","official survey sampling frame management system duplicate units unique identifiers software","Data Documentation Initiative sampling procedure universe confidentiality standard official","survey sampling frame versioning reproducibility random seed software official documentation","national statistical office sampling frame quality duplicates unique identifiers guideline","site:ddialliance.org DDI Lifecycle sampling frame sample metadata standard","site:ddialliance.org specification sampling frame universe sample design DDI","site:hhs.gov 45 CFR 46 identifiable private information human subjects definition official","site:worldbank.org reproducibility random seed sampling survey official"],"sources":[{"source_id":"S1","title":"Frames Program","publisher":"U.S. Census Bureau","url":"https://www.census.gov/about/what/transformation/maximizing-operational-efficiency/data-centric-business-ecosystem/frames-overview/frames.html","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2026-03-19","accessed_at":"2026-08-03","claims_supported":["The Census Bureau is actively modernizing infrastructure for demographic, economic, and geospatial frames.","The program organizes enterprise frames around addresses, businesses, jobs, and people.","The stated institutional benefits include reducing redundancy, improving quality, reusing data, and enabling more targeted samples."]},{"source_id":"S2","title":"Statistics Canada Quality Guidelines: Coverage and Frames","publisher":"Statistics Canada","url":"https://www150.statcan.gc.ca/n1/pub/12-539-x/steps-etapes/4147788-eng.htm","source_class":"OFFICIAL_GUIDANCE","publication_date":"Undated archived guidance","accessed_at":"2026-08-03","claims_supported":["Omissions, erroneous inclusions, duplications, and misclassifications are recognized sampling-frame coverage errors.","Frame imperfections can bias or reduce the reliability of estimates and increase collection costs.","Official guidance recommends eliminating duplication, retaining sampling and rotation information, monitoring quality, and updating frames.","Frame identifiers, classifications, addresses, and size variables affect selection and downstream statistical operations."]},{"source_id":"S3","title":"DDI-Codebook 2.5 XML Schema Documentation: sampleFrame","publisher":"DDI Alliance","url":"https://docs.ddialliance.org/DDI-Codebook/2.5/xmlschema/schemas/codebook_xsd/elements/sampleFrame.html","source_class":"STANDARD","publication_date":"2012 (DDI-Codebook 2.5 release; page does not display an exact date)","accessed_at":"2026-08-03","claims_supported":["An established social-science metadata standard already models a sample frame.","The standard records frame custodian, validity period, universe, frame-unit type, reference period, use statements, and update procedures.","The schema supports identifiers and element-version metadata but is descriptive rather than an executable behavioral oracle."]},{"source_id":"S4","title":"DDI-Lifecycle (DDI-L)","publisher":"DDI Alliance","url":"https://ddialliance.org/ddi-lifecycle","source_class":"STANDARD","publication_date":"2026 page; current specification version 3.3 released in 2020","accessed_at":"2026-08-03","claims_supported":["DDI-Lifecycle supports longitudinal and linked datasets and manages metadata over time.","It covers conceptual and methodological objects, processing, physical storage, provenance, access, and reusable metadata.","It supports metadata-driven statistical systems and quality-control consistency across repeated studies."]},{"source_id":"S5","title":"Preparing Sample Files Before Collecting CAHPS Survey Data","publisher":"Agency for Healthcare Research and Quality","url":"https://www.ahrq.gov/cahps/surveys-guidance/survey-methods-research/preparing-sample-files.html","source_class":"OFFICIAL_GUIDANCE","publication_date":"2024-08","accessed_at":"2026-08-03","claims_supported":["Operational survey guidance requires duplicate removal before fieldwork.","It recommends assigning unique sampled-person identifiers that are not derived from existing sensitive identifiers.","Combining conflicting frame sources is described as complex, multistep work requiring coordination and rigorous quality control.","Survey vendors are expected to minimize linkage between identities and responses and maintain confidentiality procedures."]},{"source_id":"S6","title":"The SURVEYSELECT Procedure: PROC SURVEYSELECT Statement","publisher":"SAS Institute Inc.","url":"https://support.sas.com/documentation/cdl/en/statug/66859/HTML/default/statug_surveyselect_syntax01.htm","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","publication_date":"2013 (SAS/STAT 13.1 documentation)","accessed_at":"2026-08-03","claims_supported":["Existing software implements multiple probability-sampling methods, stratification, inclusion probabilities, weights, and sample-unit grouping.","A seed can reproduce a sample when the same input data and selection parameters are supplied.","Reproducibility can depend on the random-number-generator version, input ordering, control sorting, and matching stratum order.","The documentation demonstrates technical feasibility while also exposing why a seed alone is not a representation-independent contract."]},{"source_id":"S7","title":"45 CFR Part 46—Protection of Human Subjects","publisher":"Electronic Code of Federal Regulations, National Archives and Records Administration","url":"https://www.ecfr.gov/current/title-45/subtitle-A/subchapter-A/part-46","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"Current through 2026-07-30 at access","accessed_at":"2026-08-03","claims_supported":["The Common Rule applies to covered research involving human subjects.","A human subject includes a living individual about whom an investigator obtains or uses identifiable private information.","Identifiability includes information for which identity may readily be ascertained or associated with the information.","Institutional or IRB determinations remain necessary where applicability or exemption is uncertain; a fictional-data exercise avoids interaction with people and use of their private information but does not pre-authorize a later live-frame pilot."]},{"source_id":"S8","title":"Survey Methods for Businesses, Farms, and Institutions, Part I: Unbiased Estimation in the Presence of Frame Duplication","publisher":"National Agricultural Statistics Service, U.S. Department of Agriculture","url":"https://www.nass.usda.gov/Education_and_Outreach/Reports%2C_Presentations_and_Conferences/Survey_Reports/Survey%20Methods%20for%20Businesses%2C%20Farms%2C%20and%20Institutions%20Part%20I%20%28Pages%201-100%29.pdf","source_class":"PRIMARY_RESEARCH","publication_date":"Circa 1993-1994; exact compilation date was not displayed","accessed_at":"2026-08-03","claims_supported":["The research identifies frame duplication as a serious problem that undermines assumed inclusion probabilities.","Multiple frame records linked to one population element create multiple selection opportunities and can bias estimators that assume equal probabilities.","The paper formalizes the mapping between frame units and population elements and discusses inclusion-probability corrections, weight adjustment, and unique counting rules.","It shows that duplicate handling and abstract unit identity are established substantive sampling concerns, not merely software-cleanliness issues."]}],"problem_evidence":{"support":"MODERATE","rationale":"The component problem is visible and consequential: official guidance identifies duplication, erroneous inclusion, misclassification, stale frame information, and identifier quality as sources of bias, reduced reliability, and higher cost; primary USDA research shows how duplicate records alter selection opportunities and inclusion probabilities. AHRQ also documents duplicate removal, confidential identifiers, and difficult reconciliation across sources. However, the exact claimed prevalence of representation-only migrations, row permutations, or backend replacements changing multi-wave household draws was not found in these eight sources, so the candidate's specific failure mode remains plausible rather than directly measured.","source_ids":["S2","S5","S8"]},"stakeholder_evidence":{"support":"MODERATE","rationale":"The U.S. Census Bureau is an identifiable, credible institutional adopter or funder in the problem class: its active Frames Program explicitly seeks modernized frame infrastructure, lower redundancy, higher quality, and better targeted sampling. Statistics Canada and AHRQ identify frame custodians, survey sponsors, vendors, and survey staff as responsible workflow actors. None of the sources expresses demand for this exact opaque API, multi-backend conformance suite, or proposed pilot, and no adoption commitment was found.","source_ids":["S1","S2","S5"]},"prior_art":{"proximity":"ADJACENT_PRIOR_ART","closest_analogues":[{"name":"Official sampling-frame quality-management practice","similarity":"Statistics Canada already requires frame-quality testing, deduplication, retained sampling history, monitoring, updating, and coordination; AHRQ requires duplicate removal, unique confidential identifiers, and rigorous reconciliation before fieldwork.","remaining_difference":"These are procedural controls around particular frame workflows, not a representation-independent abstract state machine with one executable oracle applied to substitutable backends.","source_ids":["S2","S5"]},{"name":"DDI sample-frame and lifecycle metadata standards","similarity":"DDI already standardizes frame identity, custodian, universe, units, validity, update procedures, version metadata, methodology, processing, provenance, physical storage, and repeated-study metadata.","remaining_difference":"DDI primarily describes and exchanges metadata. The reviewed material does not require equivalent physical encodings to produce identical opaque handles, seeded draws, probabilities, error behavior, and side-effect limits under a shared black-box and metamorphic test suite.","source_ids":["S3","S4"]},{"name":"SAS PROC SURVEYSELECT","similarity":"The product already executes probability samples, records selection probabilities and weights, supports strata and clusters, and reproduces draws from a seed when input and parameters are held constant.","remaining_difference":"Its guarantee is conditional on the same input dataset, ordering-relevant parameters, and generator behavior; it does not abstract multiple frame representations into governed units or certify backend substitutability.","source_ids":["S6"]},{"name":"USDA duplicate-frame corrections and unique counting rules","similarity":"USDA research explicitly maps multiple frame records to unique population elements and evaluates inclusion-probability correction, weight adjustment, and unique counting rules.","remaining_difference":"It addresses statistical correction and counting rules, not storage-independent snapshot immutability, opaque handles, operation-level errors, leakage auditing, or cross-backend conformance.","source_ids":["S8"]}],"distinctive_claim_remaining":"Given a governance-approved definition of unit identity, eligibility, and sampling design, two independently implemented frame backends that map valid source states to the same immutable abstract snapshot will return identical selected opaque handles, inclusion probabilities, validation results, errors, and non-contact side effects for every contract-level draw, including after row permutation, lossless reserialization, and internal-key replacement. This is falsified by any held-out contract-visible divergence, any necessary dependence on source ordering or direct identifiers, or any valid state that cannot be represented without concealing an unresolved substantive judgment.","confidence":"MODERATE"},"implementation_evidence":{"support":"MODERATE","rationale":"The pieces are technically conventional: DDI supplies versioned frame and lifecycle concepts; SAS demonstrates seeded probability selection, strata, clusters, probabilities, and weights; AHRQ demonstrates deduplication, confidential identifiers, source reconciliation, and divided sponsor/vendor responsibilities. A synthetic test avoids participant contact and identifiable private information, while later use of live frames would require institutional privacy, data-governance, security, and potentially IRB determinations. No source demonstrates the complete proposed contract, mutation-tested conformance oracle, or successful replacement of heterogeneous household-frame backends.","source_ids":["S3","S4","S5","S6","S7"]},"scores":{"meaningful_impact":{"score":7,"rationale":"If present, duplicate chances, silent eligibility changes, altered probabilities, and identifier exposure can affect inferential validity and participant handling. The exact frequency and realized magnitude in multi-wave household studies are unmeasured.","source_ids":["S2","S5","S8"]},"stakeholder_pull":{"score":6,"rationale":"Major statistical organizations visibly invest in frame modernization and quality, but no named organization requested or committed to this exact intervention.","source_ids":["S1","S2","S5"]},"incremental_advantage":{"score":6,"rationale":"The proposal adds executable, representation-transforming conformance and opacity to existing metadata, quality-control, and seeded-sampling practices. Advantage over a carefully governed canonical table has not been tested.","source_ids":["S2","S3","S4","S6"]},"distinctiveness_plausibility":{"score":5,"rationale":"The integrated behavioral contract and cross-backend oracle were not located, but most constituent practices are established and the search was neither a world-novelty nor patent search.","source_ids":["S2","S3","S4","S5","S6","S8"]},"technical_implementability":{"score":8,"rationale":"Versioned frame metadata, sampling algorithms, seeds, probabilities, unit grouping, and synthetic fixtures are all supported by established standards or products. Identity-resolution semantics and algorithm-version determinism remain difficult.","source_ids":["S3","S4","S6"]},"adoption_authority_feasibility":{"score":7,"rationale":"Sampling statisticians, frame custodians, survey sponsors, governance bodies, and IRBs have recognizable roles. Software maintainers cannot legitimately settle contested unit identity, eligibility, or live-data authority by themselves.","source_ids":["S1","S2","S5","S7"]},"evidence_readiness":{"score":6,"rationale":"A safe synthetic comparison can be run promptly with explicit outputs and falsifiers, but no contract, independent adapters, mutation suite, adopter agreement, or pilot results currently exist.","source_ids":["S3","S5","S6"]},"safety_net_benefit":{"score":7,"rationale":"Opaque non-derived identifiers, non-contact draw semantics, and a halt on unresolved identity decisions could reduce disclosure and unauthorized recruitment risk. Deterministic handles or exposed seeds could themselves create reidentification or selection-prediction risks.","source_ids":["S5","S7"]},"scalability":{"score":6,"rationale":"Standards and reusable conformance tests favor reuse across studies and backends, but each study must govern population definitions, identity resolution, sampling designs, legal permissions, and local frame mappings.","source_ids":["S1","S3","S4","S5"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"10K_TO_50K","scope":"Two-week synthetic exercise: contract workshop, 200 fictional units, flat-file and relational adapters, fixed and generated tests, deliberate mutants, leakage review, and findings report.","confidence":"MODERATE","assumptions":["Approximately 4-8 resource-equivalent person-weeks across a sampling statistician, frame custodian, data engineer, and ethics or community-governance reviewer.","Existing development environments and sampling libraries are available.","No live data, participant recruitment, procurement, or production security accreditation is included."],"source_ids":[]},"initial_deployment_startup":{"band_2026_usd":"50K_TO_250K","scope":"Production-grade contract, reference model, two backend adapters, conformance and mutation suites, version registry, access controls, audit logging, documentation, and governance review for one study.","confidence":"LOW","assumptions":["One bounded study with two moderately documented backends.","Existing identity-resolution policy and sampling algorithms can be wrapped rather than invented.","Excludes major source-data remediation, enterprise procurement, and participant contact."],"source_ids":["S3","S4","S6"]},"operational_launch":{"band_2026_usd":"250K_TO_1M","scope":"Controlled launch for one multi-wave study, including live-frame mapping, independent statistical verification, privacy and security review, workflow integration, staff training, parallel-run comparison, rollback, and production support.","confidence":"LOW","assumptions":["Live data trigger institutional authorization and security controls.","At least one legacy workflow must run in parallel through a wave or equivalent acceptance period.","The band excludes rebuilding an enterprise frame or resolving large-scale source-coverage deficiencies."],"source_ids":["S1","S5","S7"]},"annual_recurring":{"band_2026_usd":"50K_TO_250K","scope":"Contract stewardship, adapter maintenance, regression certification, dependency and generator-version updates, access review, incident response, and certification for later waves.","confidence":"LOW","assumptions":["One to three active backends and several releases or frame freezes per year.","Substantive design changes receive separate statistical review.","Major identity-resolution campaigns, source licensing, and fieldwork are excluded."],"source_ids":["S2","S4","S6"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"YES","reason":"Official guidance and USDA research verify that duplication, misclassification, stale frames, and weak unit identity can alter inclusion probabilities, bias estimates, and raise costs. The narrower representation-migration prevalence remains unmeasured.","source_ids":["S2","S5","S8"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"The Census Bureau Frames Program is a credible adopter/funder in the problem class, and official guidance identifies survey sponsors, frame custodians, statisticians, vendors, governance bodies, and IRBs as relevant authorizers. Exact proposal interest is not established.","source_ids":["S1","S2","S5","S7"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The remaining claim predicts contract-level equality across independently represented frames under specified transformations, beyond DDI metadata and SAS same-input reproducibility, and has explicit divergence and hidden-dependency falsifiers.","source_ids":["S3","S4","S6"]},"bounded_next_evidence_step":{"status":"YES","reason":"A two-week fictional-data, two-backend exercise can compare the contract against status-quo scripts and a canonical-table rival without touching live participants or frames.","source_ids":["S5","S6"]},"no_unresolved_safety_or_authority_stop":{"status":"YES","reason":"The immediate exercise uses only fictional units, initiates no contact, and gives substantive identity and eligibility decisions to authorized reviewers. This does not clear a later live-data pilot, which would require institution-specific determinations.","source_ids":["S5","S7"]},"credible_cost_scope_and_range":{"status":"YES","reason":"All four bands state bounded resource-equivalent scope and exclusions. Confidence is low beyond the synthetic step because no organization-specific architecture, labor rates, procurement constraints, or source-data quality assessment was available.","source_ids":[]}},"next_evidence_step":"Run a pre-registered, two-week synthetic trial with one sampling statistician, one frame custodian, one data engineer, and one authorized ethics or community-governance reviewer. Create 200 fictional abstract units with duplicates, unresolved identities, withdrawals, household changes, strata, clusters, and certainty units. Independently encode them as a shuffled flat file and normalized relational store. Compare three approaches: (A) status-quo backend-specific filter/sort/draw scripts, (B) a single canonical-table import with ordinary validation, and (C) the proposed opaque contract with independent adapters. Freeze rules before observing results. For 100 fixed seeds plus generated operation sequences, compare abstract snapshots, opaque selected handles, inclusion probabilities, errors, mutations, and side effects under row permutation, lossless reserialization, internal-key replacement, ineligible-record insertion, and duplicate consolidation. Inject order dependence, duplicate selection chances, partial-error output, stale eligibility, and identifier-leaking mutants. Success requires zero held-out contract-visible divergence between conforming backends, detection of every injected mutant, no direct-identifier dependency, and an explicit unresolved state for every case reviewers cannot legitimately map. Falsify incremental value if approaches A and B already meet those criteria at materially lower effort; falsify the intervention if its adapters pass the declared suite yet diverge on held-out cases, conceal unresolved identity judgments, change approved probabilities, or leak identifiers. Use no real records and initiate no recruitment.","blocking_evidence":["No field or synthetic result yet shows that the proposed contract prevents a divergence missed by a canonical-table baseline or current quality controls.","The prevalence and magnitude of representation-only draw changes in actual multi-wave household studies are not externally measured.","No named adopter has committed staff, authority, data access, or funding to the exact intervention.","No independent implementation has demonstrated that valid source states can be mapped to unique abstract units without suppressing contested identity or eligibility judgments.","No privacy or security assessment has tested whether opaque handles, output ordering, metadata, timing, or seed management enable reidentification or prediction.","No organization-specific cost evidence, architecture inventory, or procurement estimate was available."],"research_disposition":"PILOT_OR_ADOPTION_INQUIRY","world_novelty_boundary":"The search establishes only that sampling-frame quality management, duplicate correction, versioned social-science metadata, lifecycle standards, confidential unique identifiers, and seeded sampling software already exist. It did not locate the exact integrated opaque behavioral contract with cross-backend black-box, property-based, metamorphic, mutation, and leakage testing. World novelty, patentability, freedom to operate, market size, and realized impact remain unmeasured; absence from this bounded eight-source review is not evidence of novelty.","arm":"COMPLETE_PROPOSAL_PORTFOLIO","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":false,"material_progress_observed":true,"progress_targets":["Obtain written interest and authority boundaries from one sampling organization willing to participate in a fictional-data trial.","Pre-register the abstract state model, comparator implementations, transformation set, acceptance thresholds, protected observables, and falsifiers before coding.","Complete two independent backend adapters and a canonical-table comparator without sharing mapping code that could create correlated defects.","Demonstrate zero held-out contract-level divergence and detection of all deliberately broken adapters; otherwise reject or narrow the claim.","Document every unmappable or contested unit-state decision and verify that none is silently converted into ineligibility, deduplication, or a technical default.","Complete a handle, ordering, timing, metadata, and seed leakage assessment before requesting any live-frame pilot.","Produce measured engineering effort and workflow burden for all three comparators to replace the current assumption-based cost bands."],"reason":"Bounded web research supports the problem class, identifies credible institutional actors, and differentiates the claim from nearby standards, guidance, and software. The decisive evidence is now empirical: only an independently implemented dual-backend trial with comparators and injected faults can establish incremental benefit, detect oracle weakness, and expose hidden governance or privacy dependencies. Under the controller rule, evidence requiring live execution rather than further bounded web search requires STOP_EMPIRICAL_RESEARCH_NEEDED; every STOP is non-repairable in this evaluation cycle."},"proposal_index":4}