Skip to content

Round-Trip Validation Test

An audit procedure — instantiates Structure-Preserving Embedding Design

Sends a curated set of source items through the embedding and back, then checks whether what returns equals what left — and where it differs, names the structure that was lost.

This is the empirical check that the whole embed-then-recover path is faithful, run over a set of cases chosen on purpose. A Round-Trip Validation Test does not declare the contract and does not perform the recovery; it exercises them end to end — source → host → source — and compares the output against the input under an explicit notion of equality. A case that comes back identical is evidence the structure survived the crossing; a case that comes back different is a diff that points at exactly which structure did not. Its whole value is turning "something seems off" into "this specific construct breaks."

Example

A rich-text editor imports Markdown into its internal model and exports it back out, and the test diffs the exported file against the original. The case set is deliberately adversarial rather than average: a nested list, a table inside a blockquote, a link whose visible text contains an asterisk, a hard line break, an emoji. Most cases round-trip character-for-character. Two do not — the nested list comes back with one indent level flattened, and the table inside the blockquote returns as a plain table stripped of its quote. What is being checked is the round-trip property: decode after encode should return the original.[1]

Those two diffs are the outcome that matters. Before any user ever pastes a document and watches its formatting quietly degrade, the team knows precisely which two structures their internal model cannot hold, and can decide to fix them, or to document them as known limitations — a choice made from evidence rather than a support ticket.

How it works

  • Curate a case set that targets edge structure — the constructs most likely to distort — rather than the inputs that happen to be typical.
  • Run each case through the real path source → host → source, using the actual embedding and recovery machinery, not a stand-in.
  • Compare the return to the original under an explicit equality relation — exact, or equal-up-to-normalization — stated in advance.
  • Localize every failure. A broken round trip names the specific structure lost, so "it broke" becomes "the nested list lost an indent level."

Tuning parameters

  • Equality relation — byte-exact versus semantic or normalized equality. Too strict flags cosmetic differences as failures; too loose waves real loss through. This is the highest-leverage dial on the whole test.
  • Case-set adversariality — ordinary inputs versus constructed edge cases. Harder cases catch more but can overstate breakage for constructs that never actually occur.
  • Round-trip length — a single hop versus repeating the crossing N times to expose drift that accumulates only over several passes.
  • Coverage target — how much of the source's structural space the cases must exercise before a pass is trusted to mean anything.

When it helps, and when it misleads

Its strength is that it is the cheapest honest evidence an embedding is faithful, and its output is actionable — a diff points straight at the construct to fix, not at a vague fidelity score.

Its classic misuse is choosing an equality relation loose enough, or a case set easy enough, that the round trip always passes: a reassuringly green test that proves nothing, sometimes stood up specifically to bless a mapping already shipped. And "round-trips cleanly" only ever certifies the constructs the cases covered; an untested one can still be silently mangled. The discipline is to make equality as strict as the meaning demands, grow the case set from real failures as they surface, and read a pass as "no loss found here," never as "no loss."

How it implements the components

This mechanism realizes the empirical-check side of the archetype — it runs the crossing and reads the result:

  • round_trip_check — the core operation: push a case source → host → source and compare what returns against what left.
  • validation_case_set — the curated, edge-weighted set of inputs the check is run over, and the thing whose coverage bounds what a pass can claim.

It does not provide the recovery step it exercises — that comes from Serialization with Reconstruction Schema and the interpretation/recovery map; relation- and operation-level preservation is the Invariant Preservation Test Suite's; and distinct items colliding onto one host point is the Embedding Collision Probe's.

Notes

A round trip verifies recoverability, not usefulness in the host. Data can return byte-perfect and still be useless for the host operations the embedding existed to enable — if the map preserved the wrong structure faithfully. A clean round trip is necessary evidence of fidelity, not sufficient evidence of a good embedding.

References

[1] The round-trip property — that decode(encode(x)) returns x — is a standard target of property-based testing, where many varied inputs are generated automatically and checked against the property rather than against a fixed list of expected outputs. ↩