{"schema_version":1,"research_id":"eoa_inverse_innovation_exp04_external_evaluation_20260802","source_assessment_id":"deadweight_loss_reduction__systems_cybernetics:PROPOSAL_FIRST:v0","cell_id":"deadweight_loss_reduction__systems_cybernetics","search_queries":["hardware-in-the-loop laboratory testbed scheduling utilization reservation unused time research","site:nasa.gov hardware in the loop laboratory scheduling testbed facility user","research facility beamtime scheduling readiness unused slots allocation policy","hardware-in-the-loop test scheduling optimization laboratory","site:energy.gov user facility scheduling maximize utilization beam time readiness review policy","site:anl.gov user facility unused beamtime reassign schedule policy readiness","site:nasa.gov hardware in loop laboratory facility test validation demand schedule","laboratory equipment scheduling utilization idle time primary research constraint scheduling test lab","site:bls.gov/oes/current software developers operations research analysts median annual wage 2025","ABB Robotics constraint-guided test execution scheduling experience report official paper","Time-aware test case execution scheduling cyber physical systems shared resources paper DOI","Cisco XPRESSO reservations priority queue testbed official documentation","\"Integrating Reservations and Queuing in Remote Laboratory Scheduling\" DOI IEEE","site:ieeexplore.ieee.org \"Integrating Reservations and Queuing\"","site:arxiv.org/abs/2306.01529 publication date authors ABB Robotics DynTest results","site:arxiv.org/abs/1911.04766 real world data test laboratory scheduling results"],"sources":[{"source_id":"S1","title":"6.1 Technical Planning","publisher":"NASA Office of the Chief Engineer","url":"https://www.nasa.gov/reference/6-1-technical-planning/","source_class":"OFFICIAL_GUIDANCE","publication_date":"2023","accessed_at":"2026-08-02","claims_supported":["NASA treats laboratories and testbeds as resources that must be planned explicitly.","NASA describes HWIL as a costly environment where additional facility time can create significant cost and schedule impacts.","A laboratory could therefore realize meaningful value from avoiding genuinely wasted HWIL time."]},{"source_id":"S2","title":"R080 - Software Test Fidelity","publisher":"NASA Software Engineering Handbook","url":"https://swehb.nasa.gov/spaces/SITE/pages/215777511/R080%2B-%2BSoftware%2Btest%2Bfidelity","source_class":"OFFICIAL_GUIDANCE","publication_date":"2025","accessed_at":"2026-08-02","claims_supported":["NASA identifies limited test hardware as a cause of contention among development, integration, and testing teams.","NASA warns that inadequate high-fidelity or hardware-based testing can create safety, reliability, certification, rework, and schedule risks.","Reallocation must not substitute utilization for test fidelity or safety controls."]},{"source_id":"S3","title":"Integrating Reservations and Queuing in Remote Laboratory Scheduling","publisher":"ERIC, U.S. Department of Education; paper published in IEEE Transactions on Learning Technologies","url":"https://eric.ed.gov/?id=EJ1016719","source_class":"AUTHORITATIVE_SECONDARY","publication_date":"2013","accessed_at":"2026-08-02","claims_supported":["Shared laboratory scheduling is an observed problem involving scarce resources.","The Sahara remote-laboratory study analyzed an integrated reservation-and-queue mechanism.","The paper reports that optimized scheduling can reduce waiting and improve utilization and per-use cost."]},{"source_id":"S4","title":"Constraint-Guided Test Execution Scheduling: An Experience Report at ABB Robotics","publisher":"arXiv; related SafeComp 2023 publication","url":"https://arxiv.org/abs/2306.01529","source_class":"PRIMARY_RESEARCH","publication_date":"2023-06-02","accessed_at":"2026-08-02","claims_supported":["ABB Robotics is an identifiable industrial adopter of automated, constraint-guided scheduling for hardware-dependent tests.","The authors report that manual scheduling is time-consuming and error-prone and that static schedules become infeasible as tests and hardware change.","A constraint-based scheduling layer was deployed within an industrial continuous-integration process."]},{"source_id":"S5","title":"Investigating Constraint Programming and Hybrid Methods for Real World Industrial Test Laboratory Scheduling","publisher":"arXiv; related Journal of Scheduling publication","url":"https://arxiv.org/abs/1911.04766","source_class":"PRIMARY_RESEARCH","publication_date":"2019-11-12; revised 2024-08-14","accessed_at":"2026-08-02","claims_supported":["Industrial test laboratories face scheduling constraints involving specialized equipment, qualified personnel, deadlines, and many tests.","Constraint-programming and hybrid optimization models were evaluated with real-world laboratory data.","The study found feasible schedules for all evaluated instances and several optimal solutions, supporting technical implementability."]},{"source_id":"S6","title":"Beamtime Requests at SER-CAT","publisher":"SER-CAT at the Advanced Photon Source, Argonne National Laboratory","url":"https://www3.ser.aps.anl.gov/beamtime-requests","source_class":"OFFICIAL_GUIDANCE","publication_date":"n.d.","accessed_at":"2026-08-02","claims_supported":["An operating scientific facility already allows unused usable slots to be reassigned by its beamline scheduling manager.","Scheduled users may give or trade slots, while banking and equal-usage rules preserve allocation interests.","The facility distinguishes unusable capacity from unused usable capacity and identifies a scheduling authority and dispute process."]},{"source_id":"S7","title":"Reserving Testbeds and Topologies","publisher":"Cisco DevNet XPRESSO Documentation","url":"https://developer.cisco.com/docs/xpresso/reservations/","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","publication_date":"n.d.","accessed_at":"2026-08-02","claims_supported":["Cisco identifies shared testbeds as scarce resources whose scheduling affects time, cost, availability, and utilization.","XPRESSO combines recurring or one-time reservations with persistent queues, priorities, expiration, release, and administrator-defined limits.","A commercial-grade testbed scheduler therefore already implements much of the proposed reservation-plus-dynamic-queue logic."]},{"source_id":"S8","title":"National Employment and Wage Data by Occupation, May 2025","publisher":"U.S. Bureau of Labor Statistics","url":"https://www.bls.gov/news.release/ocwage.t01.htm","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2026","accessed_at":"2026-08-02","claims_supported":["May 2025 mean wages were $47.95 per hour for operations research analysts, $71.20 for software developers, and $53.60 for software quality-assurance analysts and testers.","These wage anchors support order-of-magnitude labor estimates, subject to benefits, overhead, safety-review, and institutional procurement multipliers."]}],"problem_evidence":{"support":"MODERATE","rationale":"Official NASA guidance establishes that HWIL time is costly, limited hardware creates contention, and inadequate hardware access can force lower-fidelity testing with safety and schedule consequences. Research on remote laboratories, ABB hardware-dependent testing, and industrial test laboratories independently confirms scheduling, waiting, specialized-resource, and staffing constraints. However, no source or supplied field record shows that the specific proposed condition—unused safety-eligible reservations coexisting with ready, compatible queued experiments—occurs at any identified HIL laboratory, let alone at material frequency. The general problem exists and matters; the candidate's scoped wedge remains unverified.","source_ids":["S1","S2","S3","S4","S5","S7"]},"stakeholder_evidence":{"support":"MODERATE","rationale":"NASA engineering authorities explicitly require planning for costly HWIL access and hardware contention, while ABB Robotics is an identifiable adopter of constraint-guided hardware-test scheduling. Cisco supplies a first-party scheduling system addressing the stated shared-testbed need, and SER-CAT assigns an official scheduling manager authority to reassign unused capacity. These establish credible adopter classes and expressed operational need, but the candidate names no actual laboratory, data owner, allocation committee, safety officer, sponsor, or committed funder.","source_ids":["S1","S2","S4","S6","S7"]},"prior_art":{"proximity":"ESTABLISHED_PRACTICE","closest_analogues":[{"name":"SER-CAT unused-slot reassignment, user transfer, banking, and equal-usage rules","similarity":"Very close governance analogue: it separates unusable from unused usable capacity, authorizes reassignment of unused slots, permits transfers and swaps, and preserves member allocation interests.","remaining_difference":"It concerns synchrotron beamtime rather than a safety-certified HIL testbed and does not document the proposed readiness-checkpoint telemetry or team access-floor metric.","source_ids":["S6"]},{"name":"Cisco XPRESSO reservation and priority-queue manager","similarity":"Close product analogue: recurring reservations coexist with queued jobs, priority rules, release, expiration, time limits, notifications, and administrator controls for scarce testbeds.","remaining_difference":"Its documented priority system does not establish safety-clearance compatibility, equitable minimum-access floors, or the candidate's shadow-replay evaluation design.","source_ids":["S7"]},{"name":"Sahara integrated reservations and queuing","similarity":"Close research analogue explicitly combining booking and queuing to improve scarce laboratory utilization and waiting time.","remaining_difference":"It addresses remote educational laboratories rather than safety-sensitive HIL experiments and does not establish the proposed incidence and safety gates.","source_ids":["S3"]},{"name":"ABB Robotics DynTest","similarity":"Close industrial-test analogue using constraint-guided dynamic scheduling across hardware test machines and changing test repositories.","remaining_difference":"The reported intervention optimizes test execution rather than specifically reclaiming unconfirmed equal-quota reservations under a protected access floor.","source_ids":["S4"]},{"name":"Constraint-based industrial test-laboratory scheduling","similarity":"Adjacent-to-close technical analogue scheduling specialized equipment and qualified personnel under hard constraints using real laboratory data.","remaining_difference":"It is a general scheduling optimizer, not a governance rule for transferable reserved windows with independent safety authority and sunset conditions.","source_ids":["S5"]}],"distinctive_claim_remaining":"For one named HIL testbed that currently uses equal nontransferable recurring quotas, and only after removing windows infeasible because of safety clearance, reset, thermal, staffing, duration, or configuration constraints, a predeclared readiness checkpoint plus access-floor-aware reassignment will produce more feasible safety-valid completed tests and shorter ready-queue delay than both the existing manual-exception baseline and a notification-only vacancy comparator, without worsening any team's protected access, operator burden, abort rate, handoff errors, interlock events, or test fidelity. This is narrow and falsifiable but is an application-specific effectiveness claim, not demonstrated world novelty.","confidence":"HIGH"},"implementation_evidence":{"support":"MODERATE","rationale":"Technical feasibility is credible: reservation-plus-queue systems exist, scientific facilities reassign unused slots, ABB deployed constraint-guided hardware-test scheduling, and real industrial laboratory scheduling has been solved under equipment and qualified-person constraints. A retrospective replay is lower risk than live reassignment and needs only timestamped reservation, readiness, eligibility, configuration, staffing, reset, execution, and access data. Workflow feasibility still depends on defining readiness without manipulable self-attestation, encoding configuration and reset constraints, handling short-notice operator work, and auditing team incidence. Authority is institution-specific: a scheduler or allocation committee may change allocation rules, but the safety officer, operator stop-work authority, data custodian, cybersecurity/privacy reviewers, and affected teams must separately approve their domains. The read-only step creates no direct equipment hazard; a live pilot could create setup errors, rushed resets, strategic readiness claims, inequitable capture, sensitive-plan disclosure, or lower-fidelity tests and therefore requires independent safety gates, access floors, audit logging, sunset, and automatic rollback. No general legal prohibition was found, but employment, research-security, proprietary-data, records-retention, and institutional policy reviews remain unperformed.","source_ids":["S2","S3","S4","S5","S6","S7"]},"scores":{"meaningful_impact":{"score":3,"rationale":"HIL access can affect cost, schedules, validation fidelity, and safety, but the amount of recoverable capacity at the proposed lab is unknown.","source_ids":["S1","S2"]},"stakeholder_pull":{"score":3,"rationale":"NASA, ABB, Cisco, and SER-CAT show explicit demand or adoption around scarce test resources, but there is no named partner for this candidate.","source_ids":["S1","S2","S4","S6","S7"]},"incremental_advantage":{"score":2,"rationale":"The proposal sensibly combines readiness, hard constraints, access floors, and rollback, but reservations-plus-queues, reassignment, and constraint scheduling already exist; advantage over simpler release or notification rules is unmeasured.","source_ids":["S3","S4","S5","S6","S7"]},"distinctiveness_plausibility":{"score":2,"rationale":"The remaining distinction is a narrow HIL governance-and-evaluation configuration, not a clearly new mechanism.","source_ids":["S3","S4","S6","S7"]},"technical_implementability":{"score":4,"rationale":"Existing products, facility practices, and industrial research demonstrate the core scheduling and constraint-handling capabilities; local data quality and safety-rule encoding are the principal uncertainties.","source_ids":["S4","S5","S6","S7"]},"adoption_authority_feasibility":{"score":3,"rationale":"Comparable facilities have scheduling authorities and administrators, but the candidate has not identified or verified the actual committee, safety officer, data custodian, or operator authority.","source_ids":["S6","S7"]},"evidence_readiness":{"score":3,"rationale":"The retrospective replay is bounded and observable, but the existence, completeness, timestamp semantics, and accessibility of the required proprietary logs are unknown.","source_ids":["S4","S5","S7"]},"safety_net_benefit":{"score":3,"rationale":"Preserving independent safety clearance, access floors, stop-work authority, and rollback could protect against efficiency-driven harm, but those controls have not been reviewed or tested locally.","source_ids":["S2","S6"]},"scalability":{"score":3,"rationale":"The software pattern can extend across testbeds, but configuration constraints, safety rules, operator practices, and fairness policy must be validated separately for each facility or testbed class.","source_ids":["S4","S5","S7"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"10K_TO_50K","scope":"Eight-week retrospective distortion review and shadow replay for one testbed: data extraction, constraint specification, blinded eligibility review, incidence analysis, and safety-review sign-off; no live reassignment.","confidence":"MODERATE","assumptions":["Approximately 120-300 combined hours from an operations-research analyst, software/data engineer, testbed operator, allocation representative, and independent safety reviewer.","BLS mean wages are converted to resource-equivalent loaded rates using roughly 1.6-2.0 times direct wages for benefits, overhead, management, and institutional support.","Required logs already exist and can be exported without buying hardware or conducting major data reconstruction."],"source_ids":["S8"]},"initial_deployment_startup":{"band_2026_usd":"50K_TO_250K","scope":"Design and integrate a one-testbed scheduling layer after positive retrospective evidence: interfaces, rule engine, identity and permissions, safety-constraint validation, audit logs, dashboards, security review, user consultation, and rollback implementation.","confidence":"MODERATE","assumptions":["Roughly 0.4-1.2 loaded FTE-years distributed across software, operations research, quality assurance, safety, operator, project-management, and cybersecurity roles.","Existing calendar, identity, clearance, and telemetry systems expose usable interfaces.","No new HIL rack, operator shift, or major certification campaign is required."],"source_ids":["S4","S7","S8"]},"operational_launch":{"band_2026_usd":"50K_TO_250K","scope":"Prepare and operate a time-bounded live pilot on one testbed, including training, supervised handoffs, on-call engineering, monitoring, incident review, independent evaluation, and restoration of the prior rule if halted.","confidence":"LOW","assumptions":["The pilot lasts about 8-16 weeks and uses existing authorized operating hours.","Additional operator coverage and safety oversight are required during early reassignments.","No intervention-linked event triggers an extended investigation or equipment recertification."],"source_ids":["S1","S2","S8"]},"annual_recurring":{"band_2026_usd":"10K_TO_50K","scope":"One-testbed steady-state administration, software maintenance, rule and access-floor audits, quarterly safety and incidence review, user support, and data retention after successful authorization.","confidence":"LOW","assumptions":["Approximately 0.1-0.25 combined loaded FTE plus modest hosting and monitoring costs.","The implementation reuses institutional infrastructure and does not require dedicated 24-hour support.","Material configuration changes trigger separate validation work rather than being absorbed into routine operations."],"source_ids":["S7","S8"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"YES","reason":"Official NASA guidance and independent laboratory and industrial-test research verify that scarce hardware access, scheduling, waiting, and lower-fidelity substitution are real and consequential, although the exact local wedge is not yet shown.","source_ids":["S1","S2","S3","S4","S5"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"ABB Robotics is a documented adopter of related scheduling, NASA engineering guidance expresses the need, and SER-CAT and Cisco identify facility schedulers or administrators capable of governing comparable allocation rules. No adopter is yet committed to this specific candidate.","source_ids":["S1","S2","S4","S6","S7"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The claim can compare access-floor-aware readiness reassignment against both the current rule and notification-only vacancy handling after hard infeasibility filters, with explicit throughput, delay, access, burden, fidelity, and safety outcomes.","source_ids":["S3","S4","S6","S7"]},"bounded_next_evidence_step":{"status":"YES","reason":"An eight-week, one-testbed, read-only historical replay is bounded, reversible, comparator-based, and has predeclared problem and intervention falsifiers.","source_ids":["S4","S5","S6","S7"]},"no_unresolved_safety_or_authority_stop":{"status":"YES","reason":"The proposed next step changes no live allocation or operating condition. It still requires data-custodian and independent safety-review authorization; any live pilot remains stopped pending separate committee, safety, operator, security, and access-floor approval.","source_ids":["S2","S6"]},"credible_cost_scope_and_range":{"status":"YES","reason":"All four phases are scoped to one testbed and anchored to current federal occupational wage data with explicit loaded-labor, integration, and no-new-hardware assumptions; launch and recurring bands remain low-confidence until a host is named.","source_ids":["S8"]}},"next_evidence_step":"Secure a named laboratory partner and written data-access approval, then preregister and execute a read-only shadow replay over one fixed eight-week period for one HIL testbed. Freeze definitions before inspection: readiness must be evidenced by artifacts available at the checkpoint; feasibility must require experiment clearance, configuration compatibility, duration fit, operator availability, thermal/reset compliance, and no safety-rule relaxation. Compare (A) actual equal-quota/manual-exception outcomes, (B) notification-only voluntary release, and (C) readiness-checkpoint, access-floor-aware reassignment. Report recoverable hours, feasible additional tests, ready-queue delay, team-level access share, operator/setup burden, aborts, handoff errors, interlock or stop-work signals, and fidelity changes. Use a blinded independent safety reviewer for feasibility classification. Falsify the problem if no unused feasible window has a contemporaneous ready eligible claimant; falsify incremental advantage if C produces no material improvement over B, moves waiting elsewhere, breaches an access floor, increases adverse operational signals, or requires relaxing a protected constraint. Do not execute live assignments in this step.","blocking_evidence":["No direct evidence establishes equal nontransferable weekly quotas at a named HIL laboratory.","No reservation, readiness, queue, execution, safety, staffing, reset, configuration, or team-access dataset was available.","The frequency and value of contemporaneous unused feasible windows and ready eligible claimants are unknown.","No laboratory allocation committee, safety officer, operator lead, data custodian, cybersecurity reviewer, or funder has agreed to participate.","Readiness definitions, access-floor thresholds, operator-burden limits, safety triggers, and minimum worthwhile effect sizes have not been preregistered.","The incremental value over a simpler notification-and-voluntary-release rule has not been tested.","Live behavioral effects—including strategic readiness claims, capture by well-resourced teams, congestion displacement, setup errors, and fidelity loss—require a later authorized field test.","Institution-specific legal, research-security, proprietary-data, privacy, records-retention, labor, and policy reviews remain undone."],"research_disposition":"KNOWN_PRACTICE_DIFFUSION","world_novelty_boundary":"The search establishes only that closely related reservation-plus-queue systems, unused-slot reassignment rules, fairness-aware scientific-facility practices, and constraint-guided industrial test scheduling already exist. It does not measure world novelty, patentability, freedom to operate, market size, or realized impact. The only remaining candidate distinction is the particular combination and local causal effect of readiness checkpoints, safety/compatibility filtering, protected access floors, feedback adjustment, and sunset/rollback on one equal-quota HIL testbed.","arm":"PROPOSAL_FIRST","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":true,"material_progress_observed":false,"progress_targets":["Name one HIL laboratory, one testbed, the allocation-rule owner, the independent safety authority, the operator lead, and the data custodian.","Obtain written authorization and an eight-week extract of timestamped reservation, readiness, queue, execution, staffing, reset, configuration, access, abort, interlock, and stop-work records.","Preregister hard feasibility filters, readiness evidence, access floors, operator-burden and safety limits, minimum worthwhile effect sizes, missing-data rules, and the actual-versus-notification-only-versus-reassignment comparators.","Complete a blinded retrospective replay and quantify how many apparently idle windows remain recoverable after every protected constraint is applied.","Reject or revise the candidate if no contemporaneous ready eligible demand exists, if notification-only release performs similarly, or if modeled gains depend on weakened safety, fidelity, staffing, reset, or access protections.","Only after a positive replay, obtain separate authority for a sunsetted live pilot with immutable audit logs, stop-work authority, incident triggers, access-floor monitoring, and automatic rollback."],"reason":"Bounded web research has reached its limit. The general problem and implementability are supported, but close prior art makes this primarily a diffusion and local-effect question. Whether the exact allocation wedge exists, whether the proposed rule beats a simpler release mechanism, and whether it preserves safety and equitable access can only be answered from proprietary laboratory logs and, later, an authorized live test."}}