{"schema_version":1,"research_id":"eoa_inverse_innovation_exp06_external_evaluation_20260803","source_assessment_id":"predictive_residual_processing__accounting_auditing:P3:v0","cell_id":"predictive_residual_processing__accounting_auditing","search_queries":["site:pcaobus.org AS 2201 evidence control testing monitoring recurring controls","site:pcaobus.org inspection observations internal controls evidence audit deficiencies 2024","continuous controls monitoring recurring control evidence Workiva AuditBoard official","process mining conformance checking internal controls audit evidence research paper","PCAOB AS 2201 official testing controls evidence frequency recurring control","COSO guidance monitoring internal control ongoing evaluations official","site:workiva.com products controls management evidence collection automated controls official","site:auditboard.com continuous controls monitoring evidence automation official","Workiva controls management product evidence requests control testing automation","AuditBoard controls management automated evidence collection product","Diligent HighBond continuous controls monitoring product evidence","SAP process control continuous control monitoring automated testing official","site:theiia.org Global Internal Audit Standards monitoring controls technology data analytics 2024","site:sec.gov management internal control monitoring evidence recurring controls guidance","site:pcaobus.org 2024 inspection spotlight testing controls review evidence official PDF","audit evidence collection burden survey internal controls 2024 official report","10.2308/ISYS-2022-028 integrating process mining machine learning internal control evaluation auditing publisher","\"A Framework for the Structured Implementation of Process Mining for Audit Tasks\" DOI","site:publications.aaahq.org process mining internal control evaluation auditing 2025 Duan Vasarhelyi"],"sources":[{"source_id":"S1","title":"AS 2201: An Audit of Internal Control Over Financial Reporting That Is Integrated with An Audit of Financial Statements","publisher":"Public Company Accounting Oversight Board","url":"https://pcaobus.org/oversight/standards/auditing-standards/details/AS2201","source_class":"STANDARD","publication_date":"2007-06-12","accessed_at":"2026-08-03","claims_supported":["Control-effectiveness conclusions require sufficient evidence and cannot rest on inquiry or metadata alone.","Evidence needs vary with control risk, nature, timing, extent, operating frequency, and changes to the control.","Management is responsible for ICFR, while the external auditor retains independent testing and opinion authority.","Inspection, observation, and re-performance remain necessary safeguards against treating an expected evidence envelope as proof of effectiveness."]},{"source_id":"S2","title":"Inspection Data for U.S. Global Network Firms","publisher":"Public Company Accounting Oversight Board","url":"https://pcaobus.org/oversight/inspections/inspection-data-us-global-network-firms","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2025","accessed_at":"2026-08-03","claims_supported":["Recent inspections continue to identify issuer audits lacking sufficient appropriate evidence for financial-statement or ICFR opinions.","Audit-evidence and ICFR-testing deficiencies are consequential enough to receive Part I.A classification.","The inspection population is risk-weighted and therefore does not establish population prevalence."]},{"source_id":"S3","title":"Monitoring Internal Control Systems","publisher":"Committee of Sponsoring Organizations of the Treadway Commission","url":"https://www.coso.org/monitoring-internal-control-system","source_class":"OFFICIAL_GUIDANCE","publication_date":"2009","accessed_at":"2026-08-03","claims_supported":["Organizations are expected to monitor the quality of internal-control systems.","Corporate management and controllership are credible authorizers for an internal-control monitoring process."]},{"source_id":"S4","title":"2024 Global Internal Audit Standards","publisher":"The Institute of Internal Auditors","url":"https://www.theiia.org/en/standards/2024-standards/global-internal-audit-standards/","source_class":"STANDARD","publication_date":"2024-01-09","accessed_at":"2026-08-03","claims_supported":["Internal-audit functions are expected to conform to standards emphasizing board authorization, independence, objectivity, confidentiality, resource management, evidence-based engagement work, and monitoring action plans.","The board, chief audit executive, and senior management are identifiable governance actors for authorizing and overseeing a shadow monitoring pilot.","Independent internal-audit review and protected access to complete evidence are necessary workflow and safety constraints."]},{"source_id":"S5","title":"Performing Automated Testing and Monitoring","publisher":"SAP","url":"https://help.sap.com/docs/SAP_PROCESS_CONTROL/f77342ea45c24d3f81032575e6f50d8b/01a9db9d07b54eba8d4d99e91d7f3322.html","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","publication_date":"2026","accessed_at":"2026-08-03","claims_supported":["SAP Process Control already supports recurring automated and semi-automated control testing and continuous monitoring.","Business rules can detect exceptions in transaction, configuration, and master data and send exception reports into workflow.","Existing practice includes schedules, thresholds, control-rule assignments, historical change logs, manual review, ratings, and remediation issues."]},{"source_id":"S6","title":"Internal Controls Automation for SOX Compliance","publisher":"Diligent","url":"https://www.diligent.com/products/internal-controls","source_class":"COMMERCIAL_FIRST_PARTY","publication_date":"2026","accessed_at":"2026-08-03","claims_supported":["Commercial platforms already automate control testing, monitoring, evidence collection, gap detection, workflows, dashboards, and remediation tracking.","Whole-population analytics and real-time exception flagging materially overlap the proposed monitoring value proposition.","The page expresses vendor-observed demand for reducing manual follow-up, but supplies no independent effectiveness or pricing evidence."]},{"source_id":"S7","title":"Process Mining of Event Logs: A Case Study Evaluating Internal Control Effectiveness","publisher":"Accounting Horizons / Hasselt University Document Server","url":"https://documentserver.uhasselt.be/handle/1942/28125","source_class":"PRIMARY_RESEARCH","publication_date":"2019","accessed_at":"2026-08-03","claims_supported":["A real-world bank case used full-population event logs to identify process variants, segregation-of-duty violations, personnel patterns, and timing anomalies.","Process mining can compare actual executions with expected process behavior and generate audit-relevant evidence.","The study demonstrates adjacent technical feasibility but does not test an evidence-envelope residual ledger with raw-package sampling and fallback."]},{"source_id":"S8","title":"AS 1105: Audit Evidence","publisher":"Public Company Accounting Oversight Board","url":"https://pcaobus.org/oversight/standards/auditing-standards/details/AS1105","source_class":"STANDARD","publication_date":"2010-08-05","accessed_at":"2026-08-03","claims_supported":["Audit evidence must be sufficient, relevant, and reliable; more volume cannot compensate for poor quality.","Company-produced information requires accuracy, completeness, precision, and applicable IT-control testing.","Evidence includes both corroborating and contradictory information, and inconsistencies or reliability doubts require additional procedures.","Automated 100-percent examination is permitted when effective, but sampling, inspection, and independent re-performance remain distinct evidence methods."]}],"problem_evidence":{"support":"MODERATE","rationale":"The problem visibly matters: PCAOB standards require sufficient, risk-responsive evidence, and current inspection data show consequential evidence deficiencies persist. COSO calls for monitoring, while commercial products explicitly target manual evidence collection and follow-up. However, no relied-upon source measures how often recurring control-evidence gaps are discovered late, how much reviewer capacity full-package monitoring consumes, or whether this is the binding constraint in a defined adopter population.","source_ids":["S1","S2","S3","S6","S8"]},"stakeholder_evidence":{"support":"MODERATE","rationale":"Corporate management or controllership can authorize operational ICFR monitoring, and a board-authorized chief audit executive can govern independent sampling and safeguards. COSO and IIA express a general need for monitoring, resource management, independence, and evidence-based work; vendors demonstrate an active buyer category. No named organization, funder, audit committee, or controllership function was found expressing demand for this exact residual-ledger design or committing data and staff to a pilot.","source_ids":["S3","S4","S5","S6"]},"prior_art":{"proximity":"SUBSTANTIAL_COLLISION","closest_analogues":[{"name":"SAP Process Control automated testing and continuous monitoring","similarity":"Schedules recurring control tests, applies configured business rules to ERP data, identifies exceptions, retains change history, generates workflow, and supports manual or automated ratings.","remaining_difference":"The documentation does not establish a frozen occurrence-level evidence-envelope predictor reconstructed from model version plus residual, independent random full-package inspection of quiet cases, or residual-driven decompression rules.","source_ids":["S5"]},{"name":"Diligent Internal Controls Management","similarity":"Automates control testing, monitoring, evidence collection, whole-population analytics, exception detection, dashboards, and remediation workflow.","remaining_difference":"The public product page does not establish synchronized predictive evidence envelopes, signed reconstruction residuals, protected raw-evidence sampling, or a rule that model matches cannot support effectiveness conclusions.","source_ids":["S6"]},{"name":"Process-mining conformance analysis for internal controls","similarity":"Compares full-population event logs with expected process behavior and detects variants, timing anomalies, and segregation-of-duty violations relevant to auditors.","remaining_difference":"The published case analyzes event-log conformance rather than prospectively freezing expected evidence schemas and lineage, suppressing only primary-queue redundancy, and empirically governing suppression through blind full-evidence inspection and fallback.","source_ids":["S7"]},{"name":"Risk-based control testing under PCAOB standards","similarity":"Already varies testing effort by risk, expected operation, control frequency, changes, and evidence quality, while using sampling and complete-population automated procedures where appropriate.","remaining_difference":"The standards do not prescribe an operational residual queue or model-plus-residual reconstruction layer for control owners and controllership.","source_ids":["S1","S8"]}],"distinctive_claim_remaining":"For stable recurring controls, a prospectively frozen and version-synchronized evidence-envelope model, combined with a residual-only primary monitoring queue, will reduce total analyst review time versus both complete-package review and calendar-plus-required-fields or conventional rule-based monitoring, while independent blind full-package inspection shows non-inferior recall of consequential evidence and control-operation issues. This claim fails if quiet-case inspections reveal a materially higher missed-issue rate, reconstruction exceeds tolerance, or model maintenance plus fallback consumes the saved review time.","confidence":"HIGH"},"implementation_evidence":{"support":"MODERATE","rationale":"Rules engines, recurring schedules, event-log comparison, exception workflows, whole-population analytics, change logs, and evidence repositories are established and make an offline prototype technically credible. Standards also define the necessary evidence-quality, independence, authority, confidentiality, and human-judgment boundaries. Feasibility remains unverified for source completeness, identity and role feeds, model-version synchronization, stable evidence schemas, independent sample selection, privacy access, false-positive burden, and integration across heterogeneous repositories.","source_ids":["S1","S4","S5","S6","S7","S8"]},"scores":{"meaningful_impact":{"score":3,"rationale":"Earlier detection and lower monitoring effort could matter, but affected volume, delay, and avoidable harm are unmeasured.","source_ids":["S1","S2","S6"]},"stakeholder_pull":{"score":3,"rationale":"Standards and commercial offerings show monitoring demand and credible governance actors, but no adopter has requested this specific design.","source_ids":["S3","S4","S5","S6"]},"incremental_advantage":{"score":2,"rationale":"Exception monitoring, automated evidence collection, conformance checking, and whole-population analytics already cover much of the value proposition; the remaining advantage depends on empirical net-attention savings and blind-sample recall.","source_ids":["S5","S6","S7"]},"distinctiveness_plausibility":{"score":3,"rationale":"The combination of reconstructive evidence envelopes, synchronized versions, quiet-case raw inspection, and mandatory fallback is contrastive, but component-level novelty and world novelty are unmeasured.","source_ids":["S5","S6","S7"]},"technical_implementability":{"score":4,"rationale":"All core computation and workflow components have close operational analogues; heterogeneous evidence lineage and completeness heartbeats are the principal unresolved engineering risks.","source_ids":["S5","S6","S7","S8"]},"adoption_authority_feasibility":{"score":3,"rationale":"Management, boards, and chief audit executives have identifiable roles, but external-auditor reliance and audit conclusions cannot be delegated to the system.","source_ids":["S1","S4","S8"]},"evidence_readiness":{"score":3,"rationale":"A read-only historical replay is bounded and low risk, but it requires proprietary workflow metadata, complete evidence, blind professional review, and measured labor.","source_ids":["S4","S7","S8"]},"safety_net_benefit":{"score":4,"rationale":"Independent raw-package sampling, human conclusions, protected-control bypasses, and fallback directly address the risk of false reassurance, provided they are actually independent and tested.","source_ids":["S1","S4","S8"]},"scalability":{"score":3,"rationale":"Rules and workflow automation can scale, but per-control modeling, integrations, audit sampling, and model-change governance may scale poorly across heterogeneous controls.","source_ids":["S5","S6","S7"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"50K_TO_250K","scope":"Pre-register and execute a one-quarter offline replay for 12 controls, including data extraction, frozen expectation construction, a simple-rules comparator, blind full-package inspection, time measurement, and analysis.","confidence":"LOW","assumptions":["Copied read-only data are available without new production connectors.","Approximately 8-20 person-weeks are needed across analytics, controllership, internal audit, and security/privacy review.","No external-audit reliance opinion or production control conclusion is purchased."],"source_ids":["S4","S7","S8"]},"initial_deployment_startup":{"band_2026_usd":"250K_TO_1M","scope":"Design and integrate a governed shadow system for roughly 12-30 controls, including repository and identity connectors, versioning, heartbeats, case workflow, access controls, sampling, logging, and fallback.","confidence":"LOW","assumptions":["The organization already has a GRC workflow and authoritative evidence repository.","Two to five source systems require integration.","Existing platform capabilities can be configured rather than replaced."],"source_ids":["S5","S6"]},"operational_launch":{"band_2026_usd":"250K_TO_1M","scope":"Parallel-run the monitored controls through at least two cycles, train owners and reviewers, validate bypass and rollback, remediate data-quality gaps, and obtain controllership and internal-audit acceptance.","confidence":"LOW","assumptions":["Launch remains shadow-only until acceptance criteria pass.","Full-package review continues during validation.","Scope is limited to one entity or process family and no more than roughly 50 controls."],"source_ids":["S1","S4","S5","S8"]},"annual_recurring":{"band_2026_usd":"250K_TO_1M","scope":"Operate connectors, evidence storage, monitoring queues, independent raw sampling, model reviews, control-change management, privacy/security controls, incident response, and periodic validation for a limited production portfolio.","confidence":"LOW","assumptions":["One to three combined full-time-equivalent roles plus platform and integration support are required.","External-audit procedures remain outside this amount.","Frequent source-system or control redesign could move cost into a higher band."],"source_ids":["S4","S5","S6"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"YES","reason":"Authoritative standards require sufficient reliable control evidence, current regulator data show consequential evidence deficiencies, and first-party products target evidence-collection and monitoring workload. Exact prevalence and delay are still unmeasured.","source_ids":["S1","S2","S3","S6","S8"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"Management or controllership is responsible for ICFR monitoring, while a board-authorized chief audit executive can govern independent sampling and acceptance. These are credible authorizing roles even though no named pilot partner was found.","source_ids":["S1","S3","S4"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The proposal can be compared against complete-package review and calendar-plus-required-fields or existing rule-based monitoring on total labor, reconstruction error, escalations, and consequential issues missed in blind full-package inspections.","source_ids":["S5","S6","S7","S8"]},"bounded_next_evidence_step":{"status":"YES","reason":"A read-only one-quarter replay of 12 preselected controls is finite, non-operational, comparator-based, and produces explicit falsification measures.","source_ids":["S4","S7","S8"]},"no_unresolved_safety_or_authority_stop":{"status":"YES","reason":"For an offline replay, complete evidence can remain authoritative, internal audit can inspect blindly, and neither the model nor control owners need authority to conclude effectiveness, modify evidence, or change audit results. Production suppression would require a new approval decision.","source_ids":["S1","S4","S8"]},"credible_cost_scope_and_range":{"status":"UNCERTAIN","reason":"Implementation components and staffing scopes can be bounded, but relied-upon sources disclose neither prices nor adopter-specific integration effort. All four bands are resource-equivalent planning estimates with low confidence.","source_ids":["S5","S6"]}},"next_evidence_step":"With a named controllership and internal-audit partner, pre-register an offline replay of one completed quarter for 12 controls selected before residual inspection. Freeze each expectation using only documentation available before the occurrence. Run (A) complete-package review, (B) calendar-plus-required-fields or conventional configured-rule monitoring, and (C) the proposed versioned evidence-envelope residual ledger. Internal audit, blinded to B and C scores, inspects every package and independently labels consequential execution, population, role, parameter, lineage, source, and evidence-quality issues. Measure total analyst and model-maintenance time, issue recall and precision, time-to-triage, reconstruction error, quiet-case misses, missing heartbeats, checksum failures, privacy incidents, and fallback frequency. Do not advance if C misses any protected-class issue, has a consequential-issue recall confidence bound below the predeclared non-inferiority margin, exceeds a predeclared reconstruction tolerance, or fails to save at least 20% total labor versus A after maintenance and fallback; also reject the incremental claim if C does not materially outperform B on missed consequential issues per reviewer-hour.","blocking_evidence":["No independent estimate of the prevalence or discovery delay of recurring control-evidence failures was found.","No named adopter, funder, or external auditor has committed proprietary data, reviewer time, or acceptance criteria.","No field evidence shows that evidence-envelope similarity predicts which packages can safely receive less primary-queue attention.","The missed-issue rate among low-residual occurrences is unknown, especially for substantive review quality, manipulated evidence, incomplete populations, management override, and shared source-system blind spots.","Source completeness, identity assurance, model-version synchronization, privacy access, and fallback reliability have not been tested across heterogeneous repositories.","No direct 2026 pricing or measured implementation-effort evidence supports the cost bands.","World novelty, patentability, freedom to operate, market size, and realized impact remain unmeasured."],"research_disposition":"PARTNERED_RESEARCH_PROGRAM","world_novelty_boundary":"The search establishes substantial collision with continuous controls monitoring, automated evidence collection, exception workflows, whole-population analytics, process-mining conformance checks, and risk-based audit testing. It did not establish whether any public or private system combines a prospectively frozen occurrence-level control-evidence envelope, model-plus-residual reconstruction, residual-only primary attention routing, independently selected full-package inspection of quiet cases, and governed decompression. This is only a search boundary: world novelty, patentability, freedom to operate, market size, and realized impact are unmeasured.","arm":"COMPLETE_PROPOSAL_PORTFOLIO","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":false,"material_progress_observed":true,"progress_targets":["Secure a named controllership and independent internal-audit partner with authority to provide read-only historical data and blind reviewers.","Pre-register comparator definitions, consequential-issue taxonomy, non-inferiority margin, reconstruction tolerance, protected classes, labor-saving threshold, and stopping rules.","Demonstrate complete occurrence heartbeats, compatible version checks, evidence-lineage access, independent sample selection, privacy controls, and tested fallback for all 12 controls.","Report quiet-case missed issues and protected-class recall separately from aggregate accuracy and queue reduction.","Measure total resource cost including model construction, maintenance, investigation, raw inspection, and fallback—not merely primary-queue volume.","Differentiate empirically from configured continuous-control rules and evidence-workflow products before any operational suppression or adoption inquiry."],"reason":"Bounded web research verifies a meaningful regulated context, credible authorizers, implementable components, and a falsifiable contrast, but also finds substantial prior-art collision. The remaining benefit and safety claims depend on proprietary control evidence, blind professional review, workflow measurement, and live or historical replay; they cannot be resolved by further bounded web search. Under the controller rule, this requires STOP_EMPIRICAL_RESEARCH_NEEDED and repairable false."},"proposal_index":3}