{"schema_version":1,"research_id":"eoa_inverse_innovation_exp05_external_evaluation_20260803","source_assessment_id":"negative_space_design__marine_science:P3:v0","cell_id":"negative_space_design__marine_science","search_queries":["site:ioos.noaa.gov QARTOD quality flags missing data ocean observations manual","marine ocean time series visualization missing data gaps connect lines misleading research","official chart guidance missing data line chart gaps interpolation dashed line","time series missing data visualization user study line charts gaps","OceanSITES data format reference manual quality control flags fill value official PDF","site:ioos.us ERDDAP plotting quality flags missing data line graphs","site:help.marine.copernicus.eu time series missing values gaps quality flags","ocean observing dashboard missing data visualization quality control flags","\"Where's My Data?\" Song Szafir DOI IEEE 2019","site:vega.github.io missing invalid values line chart defined gaps documentation","site:w3.org WAI charts accessibility text alternative data visualization","site:bls.gov occupational employment software developers May 2025 wages","Vega Lite invalid values line mark gaps documentation defined predicate","Plotly connectgaps reference official scatter line missing data","Highcharts connectNulls API official","Matplotlib masked arrays line plot gaps official gallery"],"sources":[{"source_id":"S1","title":"Where's My Data? Evaluating Visualizations with Missing Data","publisher":"University of Colorado Boulder VisuaLab / IEEE Transactions on Visualization and Computer Graphics","url":"https://cmci.colorado.edu/visualab/MissingData/","source_class":"PRIMARY_RESEARCH","publication_date":"2019-01-01","accessed_at":"2026-08-03","claims_supported":["Incomplete datasets arise from collection failures and other causes.","Two crowdsourced studies tested how visualization and imputation choices affect trend and average estimates and perceptions of confidence, credibility, reliability, and completeness.","The source supplies replicable stimuli and response data, but does not test marine users or the proposed cause-coded gap design."]},{"source_id":"S2","title":"Data over time: Line chart","publisher":"UK Office for National Statistics","url":"https://service-manual.ons.gov.uk/data-visualisation/chart-types/line-chart","source_class":"OFFICIAL_GUIDANCE","publication_date":"not stated","accessed_at":"2026-08-03","claims_supported":["Official chart guidance says to leave gaps where otherwise regular data are missing to represent continuity accurately.","The guidance recommends markers when observations occur at irregular intervals.","Breaking a line at missing data is already established visualization practice."]},{"source_id":"S3","title":"OceanSITES Data Format Reference Manual, Version 1.4","publisher":"OceanSITES / Ocean Best Practices System Repository","url":"https://repository.oceanbestpractices.org/bitstream/handle/11329/874.2/oceansites_data_format_reference_manual.pdf?isAllowed=y&sequence=5","source_class":"STANDARD","publication_date":"2020-05-11","accessed_at":"2026-08-03","claims_supported":["OceanSITES time-series files use per-measurement quality-control flags and required flag meanings.","The standard distinguishes good, bad, nominal, interpolated, and missing values; interpolated values are not equivalent to observations.","Existing standardized QC states provide part, but not all, of the proposed cause taxonomy."]},{"source_id":"S4","title":"Quality Assurance / Quality Control of Real Time Oceanographic Data","publisher":"U.S. Integrated Ocean Observing System, NOAA","url":"https://ioos.noaa.gov/project/qartod/","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"not stated","accessed_at":"2026-08-03","claims_supported":["QARTOD flags are intended to let users and automated processes control subsequent use of oceanographic data.","IOOS establishes authoritative QA/QC procedures and engages federal agencies and regional observing systems expected to implement common procedures.","IOOS and participating regional programs are identifiable authorities for QC-related data-product changes, although this source does not request the proposed rendering."]},{"source_id":"S5","title":"Quality Flags (QARTOD) — Ocean Data Explorer Documentation","publisher":"Alaska Ocean Observing System / Axiom Data Science","url":"https://portal.aoos.org/help/how-to/map/data-charts-QARTOD.html","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","publication_date":"2023","accessed_at":"2026-08-03","claims_supported":["An operating ocean-data portal already exposes roll-up and individual QARTOD flags inside data charts for visual exploration.","Users can filter QC results and rescale them with time controls.","AOOS and its portal operator are identifiable candidate adopters or test partners, but the documentation does not express demand for cause-coded line gaps."]},{"source_id":"S6","title":"Modes for Handling Invalid Data","publisher":"Vega-Lite","url":"https://vega.github.io/vega-lite/docs/invalid-data.html","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","publication_date":"not stated","accessed_at":"2026-08-03","claims_supported":["A mainstream visualization system can either connect valid points across invalid rows or break line and area paths at invalid values.","Conditional encodings, layering, and imputation transforms are supported implementation mechanisms.","The core trace-breaking behavior is technically routine; cause mapping, endpoint rules, estimate-layer separation, and cross-view consistency require application logic."]},{"source_id":"S7","title":"Images Tutorial","publisher":"World Wide Web Consortium Web Accessibility Initiative","url":"https://www.w3.org/WAI/tutorials/images/","source_class":"OFFICIAL_GUIDANCE","publication_date":"2026-04-08","accessed_at":"2026-08-03","claims_supported":["Graphs and diagrams conveying detailed information need a complete text equivalent.","Gap meaning and interval status cannot safely depend only on visual spacing, color, or hover interaction.","Accessibility testing and equivalent textual interval summaries are required implementation guardrails."]},{"source_id":"S8","title":"Occupational Employment and Wages — May 2025","publisher":"U.S. Bureau of Labor Statistics","url":"https://www.bls.gov/news.release/archives/ocwage_05152026.pdf","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2026-05-15","accessed_at":"2026-08-03","claims_supported":["May 2025 national mean wages were $148,100 for software developers, $111,490 for software QA analysts and testers, $98,770 for web developers, and $117,490 for web and digital interface designers.","These wage data anchor resource-equivalent labor assumptions; they do not include benefits, overhead, participant recruitment, or portal-specific integration costs."]}],"problem_evidence":{"support":"MODERATE","rationale":"The general interpretive problem is externally visible: experimental research shows that missing-data visualization choices affect judgments, official chart guidance treats uninterrupted lines across missing regular observations as inaccurate continuity, and Vega-Lite documents that a filtering mode can connect valid endpoints as though invalid rows did not exist. Marine standards and portals also show that missing, bad, and interpolated states occur and must be communicated. No relied-upon source demonstrates that a particular ocean portal currently spans qualifying intervals, measures resulting event-timing errors, or establishes prevalence or realized harm, so product-specific existence and magnitude remain unverified.","source_ids":["S1","S2","S3","S4","S5","S6"]},"stakeholder_evidence":{"support":"MODERATE","rationale":"IOOS is an authoritative QA/QC standard setter whose participating agencies and regional systems implement common procedures, while AOOS operates a portal that already visualizes QARTOD flags. They are credible authorizers or adopters for a non-production comparison. Their published material expresses a need to communicate and control use based on quality state, but neither source requests cause-coded gaps or commits staff, data, or funding.","source_ids":["S4","S5"]},"prior_art":{"proximity":"ESTABLISHED_PRACTICE","closest_analogues":[{"name":"Broken line for missing regular observations","similarity":"The ONS already directs chart authors to leave gaps where regular data are missing, matching the proposal's central omission of an unsupported connecting segment.","remaining_difference":"It does not prescribe marine QC cause codes, bounded endpoint marks, a separately activated estimate layer, or a comparative interpretation test.","source_ids":["S2"]},{"name":"Vega-Lite break-path invalid-data modes","similarity":"The renderer natively breaks line and area paths at null or invalid values and supports layered and conditional encodings.","remaining_difference":"It does not infer an operational cause, distinguish rejected/offline/communications/pending states, enforce marine acceptance policy, or specify accessible interval summaries.","source_ids":["S6"]},{"name":"OceanSITES QC-state representation","similarity":"The standard separates missing, bad, nominal, interpolated, and accepted observations in machine-readable per-value flags.","remaining_difference":"Its standard states do not provide the full proposed outage/communications/processing cause taxonomy or dictate primary-chart gap rendering and estimate activation.","source_ids":["S3"]},{"name":"AOOS Ocean Data Explorer QARTOD overlays","similarity":"A deployed ocean portal exposes individual and roll-up QC flags within interactive data charts.","remaining_difference":"The documentation does not show cause-coded protected intervals, mandatory trace breaks, endpoint semantics, or an observation-versus-estimate layer contract.","source_ids":["S5"]},{"name":"Missing-data visualization experiments","similarity":"Existing experiments compare visualization and imputation choices for incomplete time-series line graphs and measure judgments and perceived data quality.","remaining_difference":"They do not test the proposed marine cause taxonomy, operational states, event-timing inference, exports, or accessibility behavior.","source_ids":["S1"]}],"distinctive_claim_remaining":"Conditional on a portal that presently connects accepted endpoints across qualifying intervals, a primary observed-data trace that is broken and bounded at those intervals, labels the interval with a non-color-dependent operational cause, and keeps any estimate in an opt-in separately styled layer will produce higher correct classification of observed, unknown, rejected, zero, and estimated states—and fewer unsupported event-timing claims—than both (a) a connected trace with warnings or QC symbols and (b) an unexplained broken trace, without materially degrading interpretation of fully observed trends.","confidence":"HIGH"},"implementation_evidence":{"support":"STRONG","rationale":"Trace breaking, endpoint marks, conditional encodings, and separate layers are supported by existing visualization software; OceanSITES/QARTOD flags provide a starting data model; and AOOS demonstrates that QC data can be surfaced in an ocean portal. Implementation must add a provenance-backed cause mapper, avoid treating absent rows as merely filtered rows, preserve true zeros, define cadence-aware endpoints, synchronize interactive and exported views, and provide textual equivalents. No special legal prohibition was found, but production authority, records governance, and availability of outage/communications/processing logs were not verified. A non-production archived-data test that changes no source data is low risk; operational and alarm displays should remain excluded until separately authorized.","source_ids":["S3","S4","S5","S6","S7"]},"scores":{"meaningful_impact":{"score":3,"rationale":"Correctly exposing observational boundaries could prevent unsupported scientific or operational inference, but occurrence frequency, decision consequences, and realized impact in ocean portals are unmeasured.","source_ids":["S1","S2","S3"]},"stakeholder_pull":{"score":2,"rationale":"Relevant authorities and product owners visibly care about communicating QC, but no direct request, commitment, funding signal, or adoption intent for cause-coded gaps was found.","source_ids":["S4","S5"]},"incremental_advantage":{"score":3,"rationale":"Cause labels and opt-in estimates plausibly improve on generic gaps and connected traces with warnings, but the advantage is the central untested empirical claim.","source_ids":["S1","S2","S3","S5"]},"distinctiveness_plausibility":{"score":2,"rationale":"Breaking lines at missing values and displaying QC flags are established practices; distinctiveness survives only in their cause-aware, accessible, estimate-separated combination and its claimed comparative effect.","source_ids":["S2","S3","S5","S6"]},"technical_implementability":{"score":5,"rationale":"Existing chart systems directly support path breaks, layering, and conditional encodings; remaining work is ordinary data mapping, interface, export, and accessibility engineering.","source_ids":["S6","S7"]},"adoption_authority_feasibility":{"score":4,"rationale":"An observing-program product owner can authorize a reversible non-production view while QC leads retain acceptance authority; production and operational-display approvals remain unverified.","source_ids":["S4","S5"]},"evidence_readiness":{"score":4,"rationale":"The claim can be tested on archived records with matched renderings, blinded tasks, explicit comparators, and preregistered falsifiers without changing source measurements.","source_ids":["S1","S3","S6"]},"safety_net_benefit":{"score":2,"rationale":"The intervention offers an epistemic safeguard against invented continuity and can be rolled back, but direct protection of life, essential services, or vulnerable populations is not established.","source_ids":["S2","S4"]},"scalability":{"score":4,"rationale":"A reusable rendering and cause-mapping rule could extend across variables, stations, zoom levels, and exports, though heterogeneous metadata and portal stacks will constrain transfer.","source_ids":["S3","S4","S6"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"10K_TO_50K","scope":"Preregistered archived-series study: cause mapping, three matched prototypes, accessibility/export audit, powered recruitment capped at 120 participants, analysis, and decision memo.","confidence":"MODERATE","assumptions":["Four to eight staff-weeks split among a developer, visualization/data specialist, marine QC reviewer, and study lead.","Existing archived data and a non-production chart harness are available.","Participant incentives, recruitment, overhead, and benefits are included as resource equivalents above direct wages."],"source_ids":["S8"]},"initial_deployment_startup":{"band_2026_usd":"50K_TO_250K","scope":"One-portal, one-station or small-variable-set production-adjacent pilot with provenance mapping, feature flag, estimate layer, tests, documentation, telemetry, and accessibility/export validation.","confidence":"MODERATE","assumptions":["Approximately three to nine blended person-months.","Existing QC fields cover at least missing, rejected, and interpolated states; limited integration with operational logs is required.","No redesign of raw-data acceptance or alarm systems."],"source_ids":["S3","S5","S6","S7","S8"]},"operational_launch":{"band_2026_usd":"250K_TO_1M","scope":"Governed rollout across a regional portal's supported stations and variables, including metadata remediation, static exports, screen-reader output, training, monitoring, rollback, and independent scientific/operational review.","confidence":"LOW","assumptions":["Roughly two to six fully loaded staff-years across engineering, QA, design, marine science, program management, and accessibility.","Some sources require new mappings from instrument, communications, and processing logs.","Operational and safety-critical products undergo separate authorization and may remain excluded."],"source_ids":["S3","S4","S5","S7","S8"]},"annual_recurring":{"band_2026_usd":"50K_TO_250K","scope":"Rule and taxonomy maintenance, metadata exception handling, regression tests, accessibility/export audits, user support, and annual scientific governance review for one regional portal.","confidence":"LOW","assumptions":["About 0.4 to 1.4 fully loaded FTE equivalents plus modest infrastructure and user-research expense.","No new observing hardware or source-data processing system is included.","The estimate varies strongly with station heterogeneity and cause-log quality."],"source_ids":["S4","S5","S7","S8"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"YES","reason":"Independent research, official chart guidance, marine QC standards, and renderer documentation support the general risk that missing-data representation affects interpretation and that connected endpoints may conceal missing intervals. Product-specific prevalence remains a blocking evidence gap, not a negation of the general problem.","source_ids":["S1","S2","S3","S6"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"IOOS and participating regional programs are recognized QA/QC authorities, and AOOS operates a portal that already visualizes QARTOD flags. No adoption commitment is established.","source_ids":["S4","S5"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The remaining claim specifies two comparators, observable classification and event-timing outcomes, and a no-material-degradation condition.","source_ids":["S1","S2","S5"]},"bounded_next_evidence_step":{"status":"YES","reason":"One archived mooring series, three renderings, a capped powered sample, predefined tasks, export/accessibility audits, and quantitative falsifiers form a bounded study.","source_ids":["S1","S3","S6","S7"]},"no_unresolved_safety_or_authority_stop":{"status":"YES","reason":"A non-production archived-data study can preserve source records, alarms, status displays, and existing production behavior; production or operational adoption still requires owner and QC approval.","source_ids":["S4","S5","S7"]},"credible_cost_scope_and_range":{"status":"YES","reason":"All four bands state deployment scope and person-time assumptions and are anchored to current government occupational wage estimates, although integration uncertainty makes rollout and recurring estimates low-confidence.","source_ids":["S8"]}},"next_evidence_step":"With one observing-program partner, first audit one archived mooring variable to verify that the current renderer actually spans qualifying intervals and that provenance can distinguish no sample, QC rejection, instrument offline, communications outage, processing pending, true zero, and interpolation. If verified, preregister a three-arm randomized study: A, connected accepted endpoints plus existing warnings/QC symbols; B, broken trace with a generic missing label; C, the proposed bounded cause-coded gap with estimates off by default in a separately styled layer. Use cadence-matched intervals and unchanged source data. Power the study for the primary endpoint and cap recruitment at 120 marine scientists, data/QC staff, and operational users. Primary outcomes are correct classification of interval state and rejection of unsupported event-timing claims; secondary outcomes are task time, confidence calibration, interpretation of fully observed trends, outage awareness, and estimate retrieval. Advance only if C improves correct state classification by at least 15 percentage points over A and 10 points over B, reduces unsupported timing claims by at least 15 points versus A, and causes no more than a 10-point loss on fully observed trend tasks. Falsify the incremental claim if powered confidence intervals exclude any required improvement, if C increases confusion between true zero and missingness, if users overlook current outage status more often, or if any critical cause, estimate, screen-reader, zoom, stacked-variable, or static-export representation diverges from the underlying flags. Do not test in production or alter source records.","blocking_evidence":["No direct audit shows that an identified ocean dashboard currently connects accepted samples across qualifying unobserved intervals.","No source measures how often ocean-data users infer false continuity, unsupported event timing, or false zeros from existing displays.","No adopter has expressed demand for this specific design, committed a product owner, supplied data, or offered funding.","Availability and reliability of provenance for instrument-offline, communications-outage, and processing-pending causes are unverified.","The comparative benefit of cause-coded gaps over both connected warnings and generic gaps requires live user testing.","Production authority, operational-display exclusions, and alarm/status governance require partner confirmation.","Cross-view consistency for zoom, stacked variables, screen readers, downloads, and static exports has not been tested.","Cost estimates lack a portal-specific architecture review and inventory of metadata remediation work."],"research_disposition":"PARTNERED_RESEARCH_PROGRAM","world_novelty_boundary":"This evaluation searched and opened exactly eight direct sources to assess problem support, stakeholders, prior practice, feasibility, accessibility, and costs. It found established practice for breaking lines at missing observations, standardized marine QC states, deployed QC overlays, and common renderer support. It did not conduct a systematic literature review, product census, code audit, standards-completeness review, patent search, claims analysis, or jurisdictional freedom-to-operate analysis. World novelty, patentability, freedom to operate, market size, product-specific prevalence, and realized impact remain unmeasured.","arm":"COMPLETE_PROPOSAL_PORTFOLIO","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":false,"material_progress_observed":true,"progress_targets":["Obtain written participation from one observing-program product owner and QC lead, including access to one archived series and its operational provenance.","Verify by product audit that at least one qualifying connected interval exists and quantify its frequency before recruiting users.","Resolve a mutually exclusive cause taxonomy and document fallback behavior when cause provenance is missing or conflicting.","Preregister the three-arm study, powered sample, primary metrics, minimum effect thresholds, multiplicity handling, and listed falsifiers.","Complete screen-reader, keyboard, color-independent, zoom, stacked-variable, estimate-layer, and static-export conformance tests with zero critical semantic mismatches.","Secure separate authority and safety review before any production or operational-dashboard exposure.","Replace broad cost assumptions with a partner architecture estimate covering metadata remediation, implementation, QA, governance, and recurring maintenance."],"reason":"Bounded web research establishes that the general problem is credible, the core rendering behavior is technically straightforward, and close prior art makes generic line breaking an established practice. The only meaningful remaining advantage—the effect of cause coding plus estimate separation relative to connected warnings and generic gaps—depends on partner data, a product audit, and live comparative user testing. Those questions cannot be resolved by further bounded web search."},"proposal_index":3}