Skip to content

Representation Leakage Probe

Diagnostic audit — instantiates Representation-Independent Interface Contract

Hunts for behaviour clients can observe but the contract never promised — the accidental internals that quietly become an unofficial interface.

Reachable and observable are not the same thing. A boundary can make a representation unreachable — no field to touch, no constructor to call — while the representation still shows through in observable behaviour: the order a set iterates, how long an operation takes, the exact bytes of an opaque token, the wording of an error. Representation Leakage Probe is the audit that hunts for exactly this: behaviour a client can observe and start depending on that the contract never promised. Its defining move is to treat the unpromised-but-observable surface as the thing under test — not whether the component is correct, but whether it reveals more of itself than it means to, turning accidental internals into an unofficial contract.

Example

A service returns paginated results with an opaque cursor: clients are told to treat it as a black box and pass it back to fetch the next page. A leakage probe sits in front of the API and asks a blunt question — what can a client observe about this cursor beyond "it works"? It base64-decodes one and finds {"offset": 40} in plain sight. Nothing stops a client from decoding it, doing arithmetic on the offset, and jumping to page 500 directly — and the probe flags it, because the day the team switches to keyset pagination (where no integer offset exists) every client that did so breaks.

The probe also checks the sanctioned channels: the contract does permit an X-Debug header that returns internal timing, so that leak is on the allowlist, not the watchlist. The output is a two-column ledger — observable-and-promised versus observable-but-not — and the second column is the list of accidents waiting to become dependencies.

How it works

  • Enumerate the observable surface. Everything a client can see — return values, ordering, error text and codes, response headers, the decoded contents of an opaque token, coarse timing — not just the documented outputs.
  • Subtract the contract. Cross the observable surface against what the contract actually promises; whatever is observable but unpromised is a candidate leak.
  • Check against the sanctioned allowlist. Deliberate introspection windows — a debug endpoint, a documented ordering guarantee — are legitimate; the probe distinguishes these sanctioned escape hatches from accidental ones so the watchlist holds only the accidents.
  • Rank by depend-ability. A leak clients can easily observe and would find useful (a decodable offset) is more dangerous than one technically present but hard to reach.

Tuning parameters

  • Observation breadth — how many channels are inspected: values only, or also timing, ordering, memory, error text, token bytes. Wider catches more leaks and raises more noise.
  • Leak threshold — how easily observable something must be before it counts. Flag everything and you get paralysis; flag only the blatant and the subtle leaks slip through.
  • Allowlist scope — what introspection is declared sanctioned. A generous allowlist cuts false positives but risks blessing a leak you will regret.
  • Adversarial stance — whether the probe merely observes or actively tries to provoke leakage: forcing hash collisions to expose iteration order, timing operations to detect a cache. More adversarial finds the subtle side-channels, at more cost.

When it helps, and when it misleads

Its strength is catching the failure the other mechanisms structurally cannot see. An opaque boundary blocks reach; conformance tests check promised behaviour; neither notices when an unpromised observable quietly becomes load-bearing. The probe is the only sibling pointed at the gap between "unreachable" and "unobservable" — the place where Hyrum's Law is fought before it is lost.[1]

Its failure modes are calibration. Flag every observable difference and the watchlist becomes noise no one acts on; flag too few and the dangerous leak — the decodable cursor, the relied-upon ordering — sails through. It can only find leaks in the channels it thinks to inspect, so a novel side-channel (a new timing oracle) stays invisible until someone adds it to the probe. The classic misuse is triaging a real leak as "harmless, nobody depends on that" — the precise assumption Hyrum's Law exists to refute — and closing the ticket. The discipline is to fix or formally sanction every confirmed leak — seal it, or move it onto the allowlist as a real promise — rather than leave it in the ambiguous middle where dependence silently forms.

How it implements the components

Representation Leakage Probe realises the detection side of representation-hiding — it watches the wall from outside for cracks:

  • observable_leakage_watchlist — its primary output: the ledger of behaviour that is observable but unpromised, ranked by how likely clients are to depend on it.
  • escape_hatch_or_introspection_policy — it operates the allowlist of sanctioned introspection, the line that separates a deliberate debug window from an accidental leak.

It does not build the hiding boundary it audits — that is Opaque Type / Module Boundary — and it does not check functional conformance, which is the test siblings' work (Property-Based Conformance Test, Reference-Implementation Differential Test).

Notes

Every entry the probe sanctions becomes a real promise, and every promise constrains future versions — so the watchlist is an input to Semantic Versioning & Deprecation Gate. Deciding a leak is "fine" is therefore never free: it silently adds a clause to the contract that a later release must either keep or break.

References

[1] Hyrum's Law — with enough users of an interface, every observable behaviour, promised or not, ends up depended on by someone. The probe exists because unpromised behaviour is precisely where that dependence forms. ↩