{"schema_version":1,"research_id":"eoa_inverse_innovation_exp06_external_evaluation_20260803","source_assessment_id":"catalytic_pathway_enablement__religious_studies_theology:P3:v0","cell_id":"catalytic_pathway_enablement__religious_studies_theology","search_queries":["ritual studies audiovisual annotation segmentation ELAN ethnography comparative ritual database","multimodal annotation ELAN ISO standard annotation graph qualitative research intercoder disagreement","ritual dynamics lab database annotation video ritual research","ethnographic video archive metadata consent indigenous restricted access annotation","site:eviada.org annotation process ethnographic video annotator scene metadata EVIA","EVIA Digital Archive annotation tool ethnographic video depositor annotation process paper","ritual video annotation comparative ritual studies database project annotation","religious ritual database coded variables comparative ritual studies database","Campbell Quincy Osserman Pedersen coding in-depth semistructured interviews problems unitization intercoder reliability full text","BLS occupational employment wage social science research assistants 2025","Local Contexts TK Labels sacred secret materials official","CARE Principles Indigenous data governance official authority control responsibility ethics","site:bls.gov/oes/current social science research assistants hourly mean wage 19-4061","site:bls.gov/ooh anthropologists archeologists median pay 2025","site:bls.gov/oes current anthropology and archeology teachers postsecondary wage"],"sources":[{"source_id":"S1","title":"Abstracts: AV in DH 2014 Workshop—EVIA Digital Archive presentation","publisher":"AV in DH 2014 Workshop","url":"https://avindh2014.wordpress.com/abstracts/","source_class":"OFFICIAL_ORGANIZATION_DATA","publication_date":"2014-07","accessed_at":"2026-08-03","claims_supported":["EVIA scholars used video annotation for ethnomusicology, cultural-performance analysis, and comparison of text- and media-based ethnography.","EVIA reported ongoing funding, maintenance, obsolescence, administrative-support, depositor-time, and publication-delay problems.","Depositors spent an intensive two weeks segmenting and annotating ten hours of video using a three-level event/scene/action hierarchy.","An identifiable institution already operated a closely analogous ethnographic-video annotation workflow."]},{"source_id":"S2","title":"ELAN Manual: Introduction","publisher":"Max Planck Institute for Psycholinguistics","url":"https://www.mpi.nl/corpus/html/elan/pr01.html","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","publication_date":"undated","accessed_at":"2026-08-03","claims_supported":["ELAN supports audio/video annotation, time-linked and interlinked annotations, user-defined tiers, multiple character sets, search, and export.","The core multilayer temporal-annotation technology required by the proposal already exists."]},{"source_id":"S3","title":"The Multimedia Yasna","publisher":"Digirati Cultural Heritage","url":"https://digirati.com/showcasing-collections/the-multimedia-yasna/","source_class":"COMMERCIAL_FIRST_PARTY","publication_date":"undated","accessed_at":"2026-08-03","claims_supported":["A live religious-studies project at SOAS combined a five-hour Zoroastrian ritual film with an ELAN transcript, ritual-action annotations, object identifications, translations, and encyclopedic context.","The project synchronized transcript, ritual actions, objects, and semantic ritual structure on a unified timeline.","The implementation encountered data-integration, temporal-granularity, scale, and runtime-performance challenges."]},{"source_id":"S4","title":"EVIA Digital Archive Project Media: Search","publisher":"Indiana University EVIA Digital Archive Project","url":"https://media.eviada.org/eviadasb/search.html","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","publication_date":"undated","accessed_at":"2026-08-03","claims_supported":["EVIA provides scene-level playback with descriptive annotations.","It uses ethnographic-video-specific controlled vocabularies, depositor-assigned terms, segment rankings, and faceted search.","A maintained annotation and retrieval system for ethnographic recordings is established prior art."]},{"source_id":"S5","title":"Cataloging Your Video: Types of Metadata","publisher":"WITNESS","url":"https://archiving.witness.org/archive-guide/catalog/types-metadata/","source_class":"OFFICIAL_GUIDANCE","publication_date":"undated","accessed_at":"2026-08-03","claims_supported":["Useful video records require contextual, descriptive, rights, security-restriction, provenance, and chain-of-custody metadata.","Cataloging is labor-intensive and requires training and quality control.","Standards and controlled vocabularies improve interoperability, but incomplete context reduces reliability.","Accurate cataloging should identify disputed or unverified information and avoid editorializing."]},{"source_id":"S6","title":"Coding In-depth Semistructured Interviews: Problems of Unitization and Intercoder Reliability and Agreement","publisher":"Sociological Methods & Research (SAGE)","url":"https://paperzz.com/doc/8232682/coding-in-depth-semistructured-interviews--problems-of-un...","source_class":"PRIMARY_RESEARCH","publication_date":"2013","accessed_at":"2026-08-03","claims_supported":["Unitization—choosing the segment to which a code applies—can seriously disrupt intercoder reliability.","Qualitative coding requires knowledgeable coders, explicit code development, reliability assessment, and adjudication.","Coder stability, reproducibility, contextual ambiguity, limited budgets, and the time required to develop agreement are visible methodological problems.","The authors propose staged codebook development, adjudication, and deployment after acceptable reliability or agreement."]},{"source_id":"S7","title":"Anthropologists and Archeologists: Occupational Outlook Handbook","publisher":"U.S. Bureau of Labor Statistics","url":"https://www.bls.gov/ooh/life-physical-and-social-science/anthropologists-and-archeologists.htm","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2025-08-28","accessed_at":"2026-08-03","claims_supported":["The May 2024 median annual wage for anthropologists and archeologists was $64,910.","The occupation generally requires at least a master's degree, supporting the assumption that qualified specialist time is a material cost.","BLS notes limited project funding as a common work constraint."]},{"source_id":"S8","title":"Local Contexts TK and BC Labels: Information and Activity Packet","publisher":"Local Contexts","url":"https://localcontexts.org/wp-content/uploads/2023/04/Local-Contexts-Labels-Intro-and-Adapt-Activity_Fillable.pdf","source_class":"OFFICIAL_GUIDANCE","publication_date":"2023-04","accessed_at":"2026-08-03","claims_supported":["Secret or sacred materials may have community-defined access and circulation restrictions even when copies are already circulating.","Responsibility and authority may be distributed across multiple communities and require collective negotiation.","Community Voice labels recognize that archival descriptions may be incomplete or partial and should preserve multiplicity.","Permission labels distinguish approved activities from uses requiring direct engagement with primary cultural authorities.","Consent verification and culturally sensitive access conditions must remain external authority constraints rather than annotation-workflow friction."]}],"problem_evidence":{"support":"STRONG","rationale":"The problem is visible in both direct practice and methods research. EVIA reports that ten hours of ethnographic video required an intensive two-week annotation institute and identifies depositor time, maintenance, funding, and publication delay as continuing problems. Campbell et al. independently document unitization, coder-knowledge, agreement, and reliability burdens, while WITNESS characterizes contextual cataloging as labor-intensive and quality-control dependent. Evidence is not ritual-only prevalence data, so the frequency and magnitude across religious-studies projects remain unmeasured.","source_ids":["S1","S5","S6"]},"stakeholder_evidence":{"support":"STRONG","rationale":"Identifiable institutional users exist: Indiana University's EVIA project built and operated a structured ethnographic-video annotation workflow and explicitly described unresolved time and sustainability needs; SOAS's Multimedia Yasna project commissioned a synchronized interface for transcript, ritual actions, objects, and a five-hour ritual film. These demonstrate adopter pull for the core function, although neither source expresses intent to adopt this exact proposed bench or fund a new cross-project service.","source_ids":["S1","S3","S4"]},"prior_art":{"proximity":"SUBSTANTIAL_COLLISION","closest_analogues":[{"name":"EVIA Annotators Workbench and Digital Archive","similarity":"Very close functional analogue: a reusable institutional workflow for ingesting ethnographic video, hierarchically segmenting it into events/scenes/actions, attaching scholarly annotations, applying controlled vocabularies, and publishing searchable scene-level outputs.","remaining_difference":"The reviewed material does not demonstrate the proposal's complete rights-aware intake contract, explicit separation of emic and analyst layers, logged mechanical macro, rotating specialist exception lane, regeneration assay, capacity dashboard, and matched causal evaluation as one maintained service.","source_ids":["S1","S4"]},{"name":"ELAN","similarity":"Established technical substrate for time-aligned, multilayer, linked, searchable audio/video annotation with user-defined tiers.","remaining_difference":"ELAN is a general annotation tool, not a governed ritual-sequence service with eligibility screening, cultural-authority controls, specialist adjudication, workload management, regeneration, and counterfactual performance measurement.","source_ids":["S2"]},{"name":"Multimedia Yasna","similarity":"Direct domain analogue integrating an ELAN transcript, ritual actions, translations, object annotations, encyclopedic context, and semantic navigation for a recorded religious ritual.","remaining_difference":"It is a bespoke single-tradition project rather than evidence that a reusable cross-project bench can preserve heterogeneous local categories and restrictions while reducing total labor across repeated cycles.","source_ids":["S3"]},{"name":"WITNESS metadata and cataloging workflow","similarity":"Adjacent governed audiovisual practice specifying source context, rights, security restrictions, provenance, controlled vocabularies, integrity, and cataloging quality control.","remaining_difference":"It addresses archival description and evidentiary video rather than comparative ritual segmentation, multimodal scholarly annotation, specialist adjudication, or catalytic turnover.","source_ids":["S5"]}],"distinctive_claim_remaining":"Relative to the strongest existing tool-assisted project workflow—ELAN plus a project-specific codebook and ordinary expert review—a maintained cross-project bench combining authority-aware intake, mechanically logged alignment, separated observational/participant/analyst layers, bounded specialist adjudication, and measured recalibration will reduce labor-adjusted time to an independently acceptable annotation package by at least 20% across successive authorized records, without more authorization incidents, layer-conflation errors, false boundary precision, uncertainty loss, or downstream correction. This is contrastive and falsifiable, but untested.","confidence":"HIGH"},"implementation_evidence":{"support":"MODERATE","rationale":"Technical feasibility is strong because ELAN already supplies tiered time alignment and Multimedia Yasna demonstrates synchronization of ritual actions, text, objects, and video at substantial scale. EVIA demonstrates institutional workflow feasibility. WITNESS and Local Contexts supply usable rights, restriction, provenance, plurality, and cultural-authority patterns. However, no source validates the proposed integrated cross-project workflow, regeneration cycle, fairness of eligibility rules, security implementation, workload model, or performance under multilingual and culturally heterogeneous ritual material. Legal and ethical permission remains record- and community-specific and cannot be created by the bench.","source_ids":["S1","S2","S3","S5","S8"]},"scores":{"meaningful_impact":{"score":3,"rationale":"Reducing duplicated setup and adjudication could preserve scarce interpretive labor and improve auditability, but corpus-wide prevalence and realized effect size are unmeasured.","source_ids":["S1","S5","S6"]},"stakeholder_pull":{"score":4,"rationale":"EVIA and Multimedia Yasna demonstrate concrete institutional demand for closely analogous functions, with EVIA explicitly reporting time and sustainability problems.","source_ids":["S1","S3","S4"]},"incremental_advantage":{"score":2,"rationale":"The core transformation, hierarchy, annotation technology, controlled vocabulary, and ritual-video integration already exist; advantage rests on an unevaluated governance and service-integration increment.","source_ids":["S1","S2","S3","S4"]},"distinctiveness_plausibility":{"score":2,"rationale":"A testable integrated-service claim remains, but the proposal substantially collides with EVIA, ELAN, and Multimedia Yasna and cannot claim a novel annotation paradigm.","source_ids":["S1","S2","S3","S4"]},"technical_implementability":{"score":4,"rationale":"Existing tools and deployments establish feasibility for tiered temporal annotation, multimodal synchronization, controlled vocabularies, and searchable outputs; integration and secure operations remain engineering work.","source_ids":["S2","S3","S4","S5"]},"adoption_authority_feasibility":{"score":3,"rationale":"A PI, research-ethics officer, repository, or data steward can authorize workflow use only within existing mandates. Rights holders and communities retain nondelegable, case-specific control, limiting universal intake.","source_ids":["S5","S8"]},"evidence_readiness":{"score":3,"rationale":"The candidate specifies observable labor, quality, queue, error, and recovery measures and can be tested in a small matched study, but those data require live work logs and qualified blinded review.","source_ids":["S1","S6"]},"safety_net_benefit":{"score":4,"rationale":"Explicit restriction metadata, provenance, separate descriptive layers, uncertainty, rerouting, and shutdown rules could prevent unauthorized or overconfident use, provided they supplement rather than displace community authority.","source_ids":["S5","S8"]},"scalability":{"score":2,"rationale":"Reusable software and templates scale, but language competence, contextual interpretation, specialist adjudication, cultural authority, and annotation labor remain scarce and may become bottlenecks.","source_ids":["S1","S3","S6","S8"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"50K_TO_250K","scope":"Design and execute the 24-excerpt matched pilot, including protocol and grammar freeze, secure workspace, annotation and comparator labor, specialist adjudication, independent blinded review, analysis, and incident review.","confidence":"MODERATE","assumptions":["Twenty-four short, already-digitized and already-authorized excerpts; no new field recording or consent acquisition.","Approximately 0.4-0.8 FTE combined annotator/project-management effort for 4-6 months plus paid fractional language, ritual, data-governance, and statistical expertise.","Uses existing ELAN-class tooling rather than developing a new annotation platform.","Includes institutional overhead and 2026 labor escalation from the cited 2024 wage benchmark."],"source_ids":["S1","S2","S7"]},"initial_deployment_startup":{"band_2026_usd":"250K_TO_1M","scope":"Convert a successful probe into a secure maintained service: harden intake and access controls, build macros and validation, define tier schemas and module versioning, create training and anchor sets, integrate repository export, and complete governance and security review.","confidence":"LOW","assumptions":["One institution, two to four supported language/modality modules, and reuse of existing media and identity infrastructure.","Does not include acquiring new recordings, negotiating rights for legacy corpora, or building a general-purpose annotation application from scratch.","Requires specialist and software/data-engineering time beyond ordinary research-assistant labor."],"source_ids":["S2","S3","S5","S7","S8"]},"operational_launch":{"band_2026_usd":"250K_TO_1M","scope":"Launch for several research teams with staffed intake, annotator calibration, specialist rotation, incident response, module validation, audit logging, user support, and downstream repository integration.","confidence":"LOW","assumptions":["Roughly 2-4 resource-equivalent FTEs during the launch year, including coordination, annotation, technical operations, and fractional scholarly/language specialists.","Moderate authorized corpus volume; no mass digitization or bespoke translation program.","Service-level and community-engagement costs vary substantially by corpus."],"source_ids":["S1","S3","S5","S7","S8"]},"annual_recurring":{"band_2026_usd":"250K_TO_1M","scope":"Maintain a small institutional bench: compensated annotator pool, fractional language/ritual experts, steward, technical maintenance, secure storage, calibration, audits, refresh and retirement of modules, and protected recovery capacity.","confidence":"LOW","assumptions":["Approximately 2-5 resource-equivalent FTEs plus infrastructure and paid external/community expertise.","Throughput is metered; this is not a high-volume transcription factory.","Specialist compensation, institutional overhead, language mix, and rights-holder engagement dominate uncertainty.","The BLS anthropologist median is only a wage anchor, not a quote for the proposed multidisciplinary service."],"source_ids":["S1","S5","S7","S8"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"YES","reason":"Direct operational evidence documents intensive annotation time and sustainability problems, while independent methods research documents unitization and intercoder burdens.","source_ids":["S1","S5","S6"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"Indiana University's EVIA project and SOAS's Multimedia Yasna project are identifiable institutional implementers of closely analogous workflows. PIs and institutional data/ethics officers are credible procedural authorizers within existing rights mandates.","source_ids":["S1","S3","S4"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The remaining claim compares the integrated maintained bench against an ELAN-assisted project-specific workflow on labor-adjusted cycle time, quality, authority integrity, and recovery across cycles. It is measurable and can fail.","source_ids":["S1","S2","S3","S6"]},"bounded_next_evidence_step":{"status":"YES","reason":"A 24-excerpt, 12-pair randomized matched pilot with blinded review, a strong comparator, successive-cycle anchor checks, and precommitted pass/kill rules is bounded.","source_ids":["S1","S2","S6"]},"no_unresolved_safety_or_authority_stop":{"status":"YES","reason":"There is no generic prohibition if every excerpt is already authorized for the precise processing, retention, and review purpose; restrictions travel with the record; nonqualifying material is rerouted; and rights holders retain control. Any unverified permission, prohibited shared-tool handling, or secret/sacred restriction is an immediate case-level stop.","source_ids":["S5","S8"]},"credible_cost_scope_and_range":{"status":"YES","reason":"The ranges are resource-equivalent planning estimates grounded in EVIA's observed labor intensity, demonstrated technical complexity, and an official specialist-wage anchor. Precision is low, but scope, exclusions, staffing assumptions, and bands are explicit.","source_ids":["S1","S3","S7"]}},"next_evidence_step":"With a willing PI, data steward, ethics officer, and relevant rights holders, freeze the intake contract, permissions, retention rules, grammar, supported modules, assignment, and analysis plan before viewing pilot records. Select 24 short, fully authorized excerpts as 12 matched pairs stratified by modality, language support, contextual completeness, and anticipated ambiguity. Randomize one excerpt per pair to the complete bench and the other to the strongest feasible comparator: the originating project's ordinary expert workflow using ELAN or equivalent time-aligned software and its existing codebook, but without the proposed maintained intake, macro, specialist lane, or regeneration package. Record total elapsed time; every annotator, setup, specialist, and downstream-review minute; accepted and disputed boundaries; emic/analyst conflations; missing uncertainty; reroutes; corrections; access events; and workload breaches. Have qualified reviewers blinded to route assess every package and reprocess a fixed authorized anchor set after scheduled recalibration. Proceed to a larger partnered trial only if the bench shows at least 20% lower labor-adjusted time per acceptable annotated minute, zero unauthorized exposure, no more than a five-percentage-point increase in serious layer/boundary/uncertainty errors, no material increase in downstream correction, and recovery to within 10% of initial anchor performance. Falsify or stop the intervention if gains disappear after all labor and setup are counted, any permission boundary is breached, quality crosses the noninferiority margin, disagreement is suppressed rather than represented, or anchor performance cannot be restored.","blocking_evidence":["No comparative field data show that the integrated bench improves labor-adjusted cycle time over existing ELAN/EVIA-style workflows.","No prevalence estimate establishes how many religious-studies projects have repeated segmentation and reconciliation—rather than missing permission, language competence, context, or theoretical definition—as the dominant bottleneck.","No participating institution, PI, ethics office, repository, or rights-holder group has committed to the proposed pilot.","Project-specific authority, retention, shared-tool, community-protocol, and confidentiality conditions cannot be verified by public web research.","No evidence establishes stable cross-tradition annotation selectivity or successful regeneration across repeated cycles.","Cost estimates lack an institutional architecture, corpus volume, language mix, overhead rate, security classification, and compensation plan for community expertise."],"research_disposition":"PARTNERED_RESEARCH_PROGRAM","world_novelty_boundary":"World novelty, patentability, freedom to operate, market size, and realized impact were not measured. The search establishes substantial collision with EVIA, ELAN, Multimedia Yasna, and established audiovisual metadata practice; it does not establish that no identical integrated bench exists elsewhere. The only remaining claim evaluated here is the bounded incremental performance of the proposed governance-and-service integration against the strongest existing workflow.","arm":"COMPLETE_PROPOSAL_PORTFOLIO","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":false,"material_progress_observed":true,"progress_targets":["Secure a named PI, data steward, ethics officer, and applicable rights-holder or community-authority approvals for the exact pilot purpose and tooling.","Obtain baseline work logs or prospectively collect complete labor and correction data for the strongest ELAN-assisted project-specific comparator.","Run the pre-registered 24-excerpt matched pilot with blinded qualified review and zero-tolerance authority-incident monitoring.","Demonstrate at least 20% labor-adjusted improvement without crossing quality, authority, workload, or downstream-correction falsifiers.","Demonstrate that calibration restores anchor-set performance across successive cycles and report any tradition-, language-, modality-, or resource-specific failure.","Produce an institution-specific bottom-up budget covering staffing, specialist and community compensation, security, storage, overhead, module maintenance, and incident response."],"reason":"Bounded web research has answered the visible-problem, adopter, feasibility, safety-pattern, and prior-art questions. It cannot establish the remaining incremental claim because that requires proprietary workflow data, authorized ritual records, live annotation, qualified blinded assessment, and observation across regeneration cycles. The core practice substantially overlaps established systems, so only field evidence can show whether the proposed integration adds value rather than additional process."},"proposal_index":3}