{"schema_version":1,"research_id":"eoa_inverse_innovation_exp06_external_evaluation_20260803","source_assessment_id":"bounded_rivalry_governance__futurism_foresight:P2:v0","cell_id":"bounded_rivalry_governance__futurism_foresight","search_queries":["site:calpers.ca.gov asset liability management forecast assumptions board consultant RFP","state pension fund asset liability management vendor long term capital market assumptions consultant RFP","pension fund model risk vendor lock-in forecasting data portability procurement","site:gao.gov vendor lock-in data rights procurement interoperability","pension economic scenario generator model risk compare alternative models challenger benchmark independent validation","pension asset liability model economic scenario generator transparency reproducibility model risk report","site:federalreserve.gov SR 11-7 challenger model benchmarking model validation vendor model","site:oecd.org public procurement vendor lock-in interoperability data portability multiple suppliers","\"champion challenger\" model risk management official guidance","\"challenger model\" pension fund forecasting","\"shadow model\" model risk management forecasting procurement","\"parallel run\" vendor transition procurement model system","site:find-tender.service.gov.uk \"DN781589\"","site:vermont.gov VPIC 2025 investment consultant RFP asset liability","site:treasury.gov state pension investment consultant RFP asset liability","site:gov \"Asset Liability Management Model\" pension procurement contract","site:calpers.ca.gov investment consultant RFP asset liability management contract award","site:opengov.com pension investment consultant RFP asset liability public retirement","site:mass.gov pension investment consultant RFP asset liability","site:pa.gov pension investment consultant RFP capital market assumptions","\"ocds-h6vhtk-055b30\"","\"Border to Coast\" \"£3,600,000\" ALM","\"DN781589\" \"3,600,000\"","\"Asset Liability Management Model\" \"Ortec Finance\" \"£3,600,000\"","Actuarial Standard of Practice 56 modeling governance validation official pdf"],"sources":[{"source_id":"S1","title":"Model Risk Management for Pension Funds Investment Practitioners","publisher":"Society of Actuaries Research Institute","url":"https://www.soa.org/globalassets/assets/files/resources/research-report/2025/2025-12-ar170-rockefeller-college.pdf","source_class":"PRIMARY_RESEARCH","publication_date":"2025-12","accessed_at":"2026-08-03","claims_supported":["Pension ALM models and economic scenario generators materially influence contribution, allocation, and risk decisions.","Different economic-scenario model choices and parameters can produce materially different pension-risk projections.","Comparison with alternative models, holdout testing, stress testing, documentation, independent review, and diversified modeling approaches are recommended risk controls."]},{"source_id":"S2","title":"Asset Liability Management Model—Contract Details Notice 2026/S 000-021415","publisher":"UK Find a Tender Service","url":"https://www.find-tender.service.gov.uk/Notice/021415-2026","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2026-03-10","accessed_at":"2026-08-03","claims_supported":["Border to Coast Pensions Partnership is an identifiable public-sector adopter that procured an ALM model including an economic scenario generator, capital-market assumptions, stochastic projections, stress testing, and reporting.","The contract was awarded to Ortec Finance for £3.6 million excluding VAT, with an initial three-year term and possible extensions totaling nine years.","The procurement demonstrates an authorized buyer, an operating supplier, and a useful order-of-magnitude cost anchor, but not demand for the proposed dual-lane governance design."]},{"source_id":"S3","title":"Supporting Public Service Delivery with Efficient ICT Public Procurement","publisher":"OECD","url":"https://www.oecd.org/en/publications/towards-agile-ict-procurement-in-the-slovak-republic_b0a5d50f-en/full-report/component-5.html","source_class":"AUTHORITATIVE_SECONDARY","publication_date":"2022","accessed_at":"2026-08-03","claims_supported":["Public-authority vendor lock-in can arise when supplier-specific knowledge, proprietary compatibility requirements, long contracts, and migration costs impair future competition.","Lock-in can reduce supplier participation, innovation, continuity, and long-term value for money.","Open standards, flexible specifications, interoperability, and public control of long-lived data are established countermeasures."]},{"source_id":"S4","title":"SR 11-7 Attachment: Supervisory Guidance on Model Risk Management","publisher":"Board of Governors of the Federal Reserve System","url":"https://www.federalreserve.gov/boarddocs/srletters/2011/sr1107a1.pdf","source_class":"OFFICIAL_GUIDANCE","publication_date":"2011-04-04","accessed_at":"2026-08-03","claims_supported":["Benchmarking against alternative internal or external models is an established model-validation practice.","Outcomes analysis should use multiple tests, including out-of-sample backtesting where appropriate.","Parallel outcomes analysis of original and adjusted models is recommended before replacement, while differences between models require investigation rather than automatic winner selection."]},{"source_id":"S5","title":"Generator of Economic Scenarios (GOES) (E/A) Subgroup","publisher":"National Association of Insurance Commissioners","url":"https://content.naic.org/committees/a-e/generator-economic-scenarios-sg","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"Undated; current page accessed 2026-08-03","accessed_at":"2026-08-03","claims_supported":["A formal shared economic-scenario governance framework with designated oversight is operationally credible in a regulated financial context.","The framework includes periodic review, a public update timeline, and acceptance criteria reflecting both history and plausibly extreme scenarios.","This is prior art for governed common scenario infrastructure, though not for competitive pension-forecast provider displacement."]},{"source_id":"S6","title":"Economic Scenario Generator","publisher":"Ortec Finance","url":"https://www.ortecfinance.com/en/solutions/economic-scenario-generator","source_class":"COMMERCIAL_FIRST_PARTY","publication_date":"Undated; current page accessed 2026-08-03","accessed_at":"2026-08-03","claims_supported":["Commercial economic-scenario services already support pension funds, asset allocation, liability analysis, stress testing, and alternative capital-market views.","Existing delivery modes include flat files for customer-controlled workflows and a platform with API connectivity.","This supports interface feasibility while showing that scenario generation and portable delivery are established products rather than novel elements."]},{"source_id":"S7","title":"Actuarial Standard of Practice No. 56: Modeling","publisher":"Actuarial Standards Board","url":"https://www.actuarialstandardsboard.org/asops/modeling-3/","source_class":"STANDARD","publication_date":"2019-12","accessed_at":"2026-08-03","claims_supported":["Material pension projection models are within the standard's scope.","Model controls include holdout validation, sensitivity tests, comparison with alternative models, documentation, reproducibility, change management, and qualified review.","For externally developed models, actuaries should understand and disclose reliance, limitations, dependencies, and sensitivities.","Continuity controls should periodically address access to data, software, staff, hardware, and vendor relationships; confidential information need not be disclosed."]},{"source_id":"S8","title":"PSERS Board Adopts 5-year Experience Study Review Results","publisher":"Pennsylvania Public School Employees' Retirement System","url":"https://www.pa.gov/agencies/psers/newsroom/psers-board-adopts-5-year-experience-study-review-results","source_class":"OFFICIAL_ORGANIZATION_DATA","publication_date":"2021-08-07","accessed_at":"2026-08-03","claims_supported":["A state pension board is an identifiable authorizer that adopts demographic and economic assumptions affecting contributions and benefits funding.","PSERS obtained long-range economic-assumption presentations from three consultants and retained final decision authority at the Board.","Assumption changes had large reported liability and contribution consequences, supporting materiality and the feasibility of plural expert comparison."]}],"problem_evidence":{"support":"MODERATE","rationale":"Pension economic-scenario model choice demonstrably affects consequential projections, and public-sector supplier lock-in through proprietary knowledge, interfaces, and migration costs is documented. Public pension systems also depend on contracted consultants for long-range assumptions. However, no direct source located an actual state pension fund where one forecasting contractor controls the precise archive, transformation logic, interface, and evaluation feedback described by the candidate. The sector-specific lock-in prevalence and magnitude therefore remain unverified.","source_ids":["S1","S3","S8"]},"stakeholder_evidence":{"support":"STRONG","rationale":"Border to Coast is an identifiable public pension organization that procured the relevant ALM and scenario capability, while PSERS demonstrates that a state pension board authorizes consequential assumptions after receiving multiple consultant forecasts. These sources establish credible adopters and authority, but neither expresses demand for a funded shadow-provider lane or recurring displacement mechanism.","source_ids":["S2","S8"]},"prior_art":{"proximity":"SUBSTANTIAL_COLLISION","closest_analogues":[{"name":"Public procurement of an integrated pension ALM and economic-scenario platform","similarity":"An authorized public pension buyer competitively procured the same underlying forecasting, stochastic ALM, stress-testing, and reporting capability from a commercial provider.","remaining_difference":"The notice describes one supplier and does not document a fund-owned neutral archive, funded challenger lane, transition bond, or evidence-based recurring displacement.","source_ids":["S2","S6"]},{"name":"Alternative-model benchmarking and parallel outcomes analysis","similarity":"Official model-risk guidance already calls for comparison with alternative models, outcome testing, investigation of discrepancies, and parallel analysis before model replacement.","remaining_difference":"The candidate applies those controls to recurring competition between independent service providers and finances the challenger as an operational lane.","source_ids":["S4","S7"]},{"name":"Governed shared economic-scenario infrastructure","similarity":"NAIC maintains a formal economic-scenario governance process with acceptance criteria, update review, and implementation oversight.","remaining_difference":"It is regulatory scenario infrastructure, not a contestable public-pension provider role with procurement, protests, and switching duties.","source_ids":["S5"]},{"name":"Plural public-pension consultant review","similarity":"PSERS used three consultants' long-range economic assumptions while its board retained authority and adopted the final assumptions.","remaining_difference":"The consultants were inputs to board deliberation; there was no common operational harness, production-versus-challenger lane, or precommitted displacement threshold.","source_ids":["S8"]},{"name":"Open-standard anti-lock-in procurement","similarity":"Public-procurement practice already uses interoperability, flexible specifications, standard formats, and buyer-controlled data to limit incumbent entrenchment.","remaining_difference":"The candidate adds prospective forecast comparison, funded access, transition security, and recurring competition around a long-lived forecasting service.","source_ids":["S3"]}],"distinctive_claim_remaining":"World novelty is not established. The remaining contrastive, falsifiable claim is that, relative to conventional single-provider procurement and an open-format-only comparator, combining a fund-owned neutral data/evaluation layer with a separately funded parallel challenger and precommitted displacement windows will materially reduce challenger onboarding and exit dependency while producing interpretable comparative evidence without degrading continuity or privileging one forecasting method.","confidence":"HIGH"},"implementation_evidence":{"support":"MODERATE","rationale":"Flat-file and API delivery, common scenario governance, holdout testing, alternative-model comparison, reproducibility controls, documentation, and parallel outcomes analysis are all established and make an offline harness technically credible. The hard unresolved issues are whether one common schema can faithfully represent materially different forecasting methods, whether hidden long-horizon evaluations remain informative, whether the fund can maintain the shared layer securely, and whether state procurement, trade-secret, protest, bonding, and records rules permit the full workflow.","source_ids":["S3","S4","S5","S6","S7"]},"scores":{"meaningful_impact":{"score":4,"rationale":"Forecast and assumption choices affect pension liabilities, contributions, allocation, and long-horizon risk; PSERS reports changes with hundreds of millions to billions of dollars in estimated consequences. The candidate could matter substantially where the hypothesized lock-in exists.","source_ids":["S1","S8"]},"stakeholder_pull":{"score":4,"rationale":"Public pension organizations demonstrably buy ALM and economic-scenario capability, and pension boards use multiple consultants. Pull for the exact dual-lane contestability package is not yet documented.","source_ids":["S2","S8"]},"incremental_advantage":{"score":3,"rationale":"The integration of portability, funded parallel operation, and precommitted displacement plausibly improves on one-vendor procurement or report-only diversification, but no comparative outcome evidence establishes lower switching risk, better forecasts, or better value.","source_ids":["S2","S3","S4","S8"]},"distinctiveness_plausibility":{"score":2,"rationale":"Nearly every component has strong prior art: open-standard procurement, public control of data, alternative-model benchmarking, parallel outcomes analysis, plural consultants, governed scenario infrastructure, and commercial portable delivery. Only the particular institutional bundle remains differentiated.","source_ids":["S2","S3","S4","S5","S6","S7","S8"]},"technical_implementability":{"score":3,"rationale":"An offline archive, standardized package, flat-file/API interface, hidden evaluation set, and reproducibility audit are technically ordinary. A genuinely method-neutral schema and meaningful long-horizon scoring remain nontrivial and require testing.","source_ids":["S4","S5","S6","S7"]},"adoption_authority_feasibility":{"score":3,"rationale":"Pension boards and authorized procurement functions can select consultants and adopt assumptions, but the precise state-law authority for dual awards, bonds, bid protests, data rights, confidential methods, and structural access remedies must be established jurisdiction by jurisdiction.","source_ids":["S2","S8"]},"evidence_readiness":{"score":2,"rationale":"The first offline exercise is bounded and measurable, but the central problem prevalence, incumbent dependencies, realistic challenger costs, and effects on contestability require proprietary records and partner participation.","source_ids":["S1","S3","S7"]},"safety_net_benefit":{"score":4,"rationale":"Board-retained authority, offline testing, parallel rather than abrupt replacement, reproducibility controls, and explicit non-automatic use reduce operational and decision risk. Misinterpretation of comparative scores remains a material hazard.","source_ids":["S4","S7","S8"]},"scalability":{"score":3,"rationale":"Rulebooks, schemas, audit harnesses, and transition clauses can be templated across funds, but each plan's liabilities, legal regime, security environment, data history, and forecasting methods require substantial local adaptation.","source_ids":["S2","S3","S6","S7"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"50K_TO_250K","scope":"One 8-12 week preregistered offline portability and model-comparison exercise covering two independently packaged systems, a limited historical/synthetic dataset, evaluator time, security review, and a short legal/procurement review.","confidence":"LOW","assumptions":["No live production integration or investment use.","Existing historical or synthetic data can be lawfully used.","Two providers participate under capped fixed-fee test agreements.","The range is a resource-equivalent planning estimate, not a vendor quotation."],"source_ids":["S6","S7"]},"initial_deployment_startup":{"band_2026_usd":"250K_TO_1M","scope":"Design and build the fund-controlled archive, canonical metadata and export schema, evaluation harness, access controls, rulebook, procurement documents, governance roles, and transition test environment.","confidence":"LOW","assumptions":["The fund reuses existing secure hosting and identity infrastructure.","The scope excludes purchasing a full replacement ALM platform.","Method-specific adapters are limited in number.","External counsel and independent validation are included at planning-level effort."],"source_ids":["S2","S3","S6","S7"]},"operational_launch":{"band_2026_usd":"1M_TO_5M","scope":"Run the first competitive cycle, contract production and funded challenger lanes, complete onboarding and parallel validation, establish monitoring and protest capacity, and execute transition rehearsals.","confidence":"MODERATE","assumptions":["The £3.6 million public ALM contract is treated only as an order-of-magnitude anchor, not as a directly comparable U.S. quote.","Two-provider launch costs more than a single platform but remains within the broad anchor after adding shared-layer operations and independent review.","Refundable bond principal is excluded; premiums and administration are included.","No live investment-management authority is delegated."],"source_ids":["S2","S6"]},"annual_recurring":{"band_2026_usd":"250K_TO_1M","scope":"Production and shadow service fees, shared-layer maintenance, periodic audit and security work, evaluator and governance staff, annual outcome review, and amortized biennial recompete activity.","confidence":"LOW","assumptions":["The public £3.6 million contract's potential nine-year duration implies a single-system annual order of magnitude in the mid-six figures before currency conversion.","The challenger lane is smaller than the production lane.","Internal pension staff and infrastructure are partially shared with existing ALM operations.","Material custom redevelopment or litigation would exceed this band."],"source_ids":["S2","S5","S6","S7"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"UNCERTAIN","reason":"Model risk and generic public-sector vendor lock-in are supported independently, but no direct evidence verifies the candidate's combined incumbent-control pattern in a state pension forecasting contract.","source_ids":["S1","S3","S8"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"Public pension organizations procure ALM and scenario systems, and pension boards demonstrably authorize long-range assumptions after consultant input.","source_ids":["S2","S8"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The shared-layer-plus-funded-challenger design can be compared against single-provider and open-format-only baselines on onboarding effort, portability, reproducibility, continuity, evaluator agreement, and method neutrality.","source_ids":["S3","S4","S7"]},"bounded_next_evidence_step":{"status":"YES","reason":"A read-only offline exercise can test ingestion, reproduction, scoring, export, security, and comparative interpretation without changing a contract or live pension assumption.","source_ids":["S4","S6","S7"]},"no_unresolved_safety_or_authority_stop":{"status":"UNCERTAIN","reason":"The offline test has a credible safety boundary, but full adoption still requires jurisdiction-specific findings on procurement authority, records and confidentiality duties, intellectual property, bonding, protests, cybersecurity, and the board's nondelegable fiduciary role.","source_ids":["S7","S8"]},"credible_cost_scope_and_range":{"status":"UNCERTAIN","reason":"A directly relevant public ALM contract anchors the broad order of magnitude, but no quotes or bottom-up estimates were found for the added shared infrastructure, funded challenger, validation, and governance functions.","source_ids":["S2","S6"]}},"next_evidence_step":"Secure one pension-fund partner and preregister an 8-12 week, read-only offline test using the current provider-format workflow and two independently packaged forecast systems. Compare three conditions: (A) current vendor-specific ingestion and reproduction, (B) common open export format without a funded parallel-service workflow, and (C) the proposed fund-controlled archive, evaluation harness, and challenger package. Measure staff hours and elapsed time to ingest, undocumented dependencies, reproduction error, provenance completeness, security findings, export completeness, evaluator agreement, score changes after audit, sensitivity to undisclosed historical cutoffs and synthetic regime perturbations, method-specific schema exceptions, and stability of the proposed displacement result. Falsify the incremental claim if condition C does not materially reduce onboarding or exit dependencies relative to both comparators; if it loses decision-relevant information, requires incumbent-specific extensions, or produces unstable rankings under reasonable scoring weights; if independent evaluators cannot reproduce results; or if legal/security review prohibits the necessary data, audit, or transition rights. Do not change a live contract, publish provider identities, or use outputs for trades, contributions, benefits, or actuarial certification.","blocking_evidence":["No direct evidence that the precise incumbent-controlled archive, transformation, interface, and feedback problem exists in a state pension forecasting contract.","No externally validated estimate of the prevalence, switching cost, or qualified-bidder suppression caused by this pattern.","No test showing that a common schema can represent materially different forecasting approaches without encoding incumbent assumptions.","No empirical comparison of the proposed dual-lane design against conventional procurement or open-format-only procurement.","No jurisdiction-specific legal determination covering dual awards, data and model rights, public records, trade secrets, bid protests, transition bonds, and structural access remedies.","No provider quotations or fund-specific staffing estimates for the shared layer, funded challenger, audit, and recurring competition.","No evidence that long-horizon displacement thresholds remain stable, decision-relevant, and resistant to repeated-contest overfitting or provider coordination."],"research_disposition":"PARTNERED_RESEARCH_PROGRAM","world_novelty_boundary":"The search assessed visible prior art and found substantial collisions at the component and workflow levels. It did not measure world novelty, patentability, freedom to operate, market size, or realized impact. No exact-match finding should be interpreted as novelty, and the distinctive claim is limited to a testable institutional combination rather than ownership of the underlying mechanisms.","arm":"COMPLETE_PROPOSAL_PORTFOLIO","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":false,"material_progress_observed":true,"progress_targets":["Obtain proprietary contract, architecture, archive-ownership, onboarding, and prior-bid evidence from one partner fund to verify or falsify the stated lock-in problem.","Complete the preregistered three-condition offline portability and comparison exercise with independent provider packages.","Demonstrate method neutrality, reproducibility, evaluator agreement, export completeness, and ranking stability under sensitivity analysis.","Obtain jurisdiction-specific procurement, fiduciary, confidentiality, intellectual-property, cybersecurity, bonding, and protest review.","Collect comparable vendor quotes and internal staffing estimates for startup, launch, and annual operation.","Predefine decision thresholds and falsifiers before any provider identities or scores are revealed."],"reason":"Bounded web research establishes consequential pension model risk, credible public-pension adopters, and extensive prior art for nearly every component. It cannot establish the precise state-pension lock-in pattern, the method neutrality of a shared interface, the comparative benefit of a funded challenger lane, or fund-specific legal and cost feasibility. Those questions require proprietary evidence and live partner testing, so further web research is not the controlling next step."},"proposal_index":2}