Data Integrity Preservation

Preserve the accuracy, consistency, and traceability of data or records across their lifecycle.

Solution archetype #

299

Problem family

Identity, Provenance & Integrity Failure

Problem subfamily

Conservation, Record Integrity & Reconciliation

The Diagnostic Story

Symptom: Different systems report different values for what should be the same record, and nobody can explain where the discrepancy came from or which version is current. Users compensate with spreadsheets, manual cross-checks, and screenshots because the official record is unreliable. Backups and logs exist but cannot establish a trusted state when an incident or dispute occurs.

Pivot: Define what trustworthy data state means for the context, identify the lifecycle boundaries where integrity breaks down, constrain authorized mutation paths, validate state and transitions, and preserve the provenance evidence needed to investigate and correct discrepancies.

Resolution: Decision-makers can tell which data is trusted, suspect, stale, or under repair. Corruption and unauthorized change are detected earlier and propagate less widely. Trust in records becomes inspectable and maintainable rather than assumed, and recovery after incidents restores trusted state rather than merely restoring bytes.

Reach for this when you hear…

[financial reconciliation] “The ledger and the sub-ledger disagree by forty thousand dollars and I cannot tell which one to trust or when they diverged.”

[clinical data management] “The patient record shows a contraindicated drug was never given, but the nursing note says it was — someone edited something without a trail.”

[data engineering] “The pipeline ran successfully but the output totals do not match the source, and without lineage I have no idea where the transformation went wrong.”

When This Archetype Applies

Complete catalog groundingAt least one sufficient condition set is fully represented by existing primes or domain-specific abstractions.

Diagnostic problem

Data or records that support decisions can lose accuracy, consistency, provenance, authority, or recoverability as they are created, transformed, copied, integrated, edited, migrated, archived, or restored.

What this problem means

The structural problem is that data must remain stable enough to guide action while also moving through imperfect people, systems, formats, permissions, transformations, and time. A record that was once accurate can become stale. A field that is valid in isolation can contradict another system. A file can be unchanged but wrong at source. A log can exist while the real change path went around it.

This produces a trust gap: the system has data, but it cannot confidently say what the data means, where it came from, whether it is current, who changed it, or how to repair it when it disagrees with another representation.

Show the applicability expression

Applicability expression5 distinct conditions

Decision-critical unreliable dataandDuplicated cross-system entityandBoundary-crossing dataandSilent unauthorized mutationandRequired provenance evidence

Algebraic1∧2∧3∧4∧5

groundedpartly groundedopen

5 conditions, all required.

5Required in every casenumbered 1–5

These hold no matter which pattern applies.

1

Decision-critical unreliable data · grounded · any one of 6

A decision, obligation, calculation, eligibility rule, safety action, or report depends on data that may be wrong, stale, incomplete, or inconsistent. more

and — also required

2

Duplicated cross-system entity · grounded

The same entity, event, transaction, account, or record appears in multiple systems or versions. more

and — also required

3

Boundary-crossing data · grounded

Data crosses a boundary: manual-to-automated, upstream-to-downstream, old-system-to-new-system, source-to-derivation, or internal-to-external. more

and — also required

4

Silent unauthorized mutation · grounded · any one of 4

Unauthorized or poorly controlled changes could silently alter trusted state. more

and — also required

5

Required provenance evidence · grounded

Downstream users need evidence of origin, change history, review, or recovery before relying on records. more

Other requirements and context (1)

Why these sit outside the expression

Supporting context — it may accompany or help interpret the situation, but it is not a load-bearing condition in a sufficient diagnostic set.

• Supporting contextIntegrity failure would be costly, unsafe, unfair, legally consequential, or difficult to detect after the fact.

Coverage

5 of 5 conditions grounded.

Read the methodology · Download the trigger-logic data

Mechanisms / Implementations

• Access Control Enforcement: Restricts who or what may read, write, approve, delete, or restore protected data, so records change only through authorized paths and never through hidden side doors.
• Audit Log: Keeps an append-only, attributable record of every action on protected data — who, when, and what changed — so integrity events can be investigated and reconstructed after the fact.
• Backup and Restore Verification: Proves that protected data can actually be restored and that the restored records still satisfy their integrity invariants — not merely that a backup file exists.
• Checksum or Hash Validation: Detects unintended alteration, transmission error, or corruption by comparing a freshly computed hash against a trusted reference value.
• Data Lineage Capture: Records how each value moved through sources, transformations, joins, and derivations, so a suspect output can be traced back to the upstream step that produced it.
• Data Validation Schema: Encodes the structure, types, allowed values, and cross-field rules a record must satisfy, rejecting malformed data at the boundary before it is trusted.
• Integrity Anomaly Monitoring: Watches trusted data for impossible values, unexpected drift, duplication spikes, missing records, or staleness, and raises a visible exception when something looks wrong.
• Reconciliation Workflow: Compares two records or states that should agree, classifies each discrepancy, and drives it to a repair, quarantine, or accepted-divergence decision that is recorded.
• Referential Integrity Constraint: Prevents a record from pointing to a nonexistent or invalid related record, so links between data never dangle.
• Source-of-Truth Registry: Documents, per field or claim, which system or role is authoritative — the reference that integrity checks and reconciliation consult to know which value should win.

▸ Show 1 more

Related Abstractions

Abstractions this archetype builds on — directly (a source ingredient) or as a related pattern. Links follow the typed catalog namespace.

Built directly on (2)

• Data Integrity: Accuracy and consistency preserved.
• Invariance: Properties unchanged under transformation.

Also references 11 related abstractions

• Access Control: Restrict system access.
• Accountability: Responsibility for actions.
• Boundary: Defines system limits.
• Closure: Ensures operations remain within a set.
• Constraint: Limits possibilities to guide outcomes.
• Fault Tolerance: Continue operating under failure.
• Feedback: Outputs influence inputs.
• Observability: Infer internal state externally.
• Relation: Describes associations or dependencies.
• Transaction: All-or-nothing operations.

▸ Show 1 more

Variants

Narrower or domain-specific specializations that share this archetype's core structure. Recognized variants are established; candidate variants are provisional.

Record Integrity Preservation · domain variant · recognized

Preserve the trustworthiness of official, legal, clinical, financial, administrative, or operational records across their lifecycle.

Provenance Integrity Preservation · implementation variant · recognized

Preserve trustworthy origin, authorship, transformation, and custody information for data or records.

Cross-System Consistency Preservation · scale variant · recognized

Preserve agreement among related records, states, ledgers, schemas, or reports distributed across systems or organizations.

Transactional Integrity Preservation · mechanism family variant · recognized

Preserve data integrity while related changes are written, committed, reversed, retried, or exposed to readers.

Recovery Integrity Preservation · risk or failure variant · candidate

Preserve or re-establish trusted data state after corruption, loss, unauthorized change, failed migration, or incident response.

Editorial Notes

Problem Classification

Classification: Identity, Provenance & Integrity Failure → Conservation, Record Integrity & Reconciliation

Problem kernel: records lose authoritative correspondence across transformations

Rationale: Creation, copying, editing, migration, and restoration can introduce corruption or divergence without governed integrity and reconciliation.

Independent corroboration: The earliest necessary condition in the frozen evidence is: Data or records that support decisions can lose accuracy, consistency, provenance, authority, or recoverability as they are created, transformed, copied, integrated, edited, migrated, archived, or restored. That is a conservation record integrity and reconciliation problem because Supposedly equivalent records, descendants, or transformations permit hidden loss, duplication, corruption, inheritance drift, or unresolved authoritative conflict.

Review outcome: Independent reviewer agreement; high confidence.