Skip to content

Neighborhood Trustworthiness and Continuity Metric

Neighbourhood-fidelity metric — instantiates Neighborhood-Preserving Substrate Mapping

Scores how faithfully each point's map-neighbours match its true source-neighbours, separating the false neighbours a map invents from the real neighbours it tears apart.

Once a map exists, the question is whether its local neighbourhoods can be trusted — and this metric answers it with a matched pair of rank-based scores rather than one blurry quality number. Trustworthiness penalises false neighbours: items that sit close on the map but are actually far apart in the source (intrusions the projection invented). Continuity penalises torn neighbours: items close in the source that the map has flung apart (true relationships it broke). Its defining idea is that these are the two opposite errors an embedding makes, and they must be measured separately — both computed by comparing the K-nearest-neighbour sets in the source against those in the map, at a neighbourhood scale K chosen to match the locality the task actually cares about.

Example

A team ships a 2D projection of a word-embedding space so analysts can browse a vocabulary. Before trusting it, they score it at K = 15. For each word they compare its fifteen nearest neighbours on the 2D map against its true fifteen nearest in the original high-dimensional space. Trustworthiness comes out high where map-neighbours are genuine — "bank" sits beside "loan" and "deposit" — but drops where the flattening jammed unrelated words together, dropping "river" next to "bank" purely from the 2D crush. Continuity catches the reverse: "teacher" and "professor," true neighbours in the source, flung to opposite corners. The two scores together tell the analysts the map is safe for coarse browsing but not for the claim "these two words are unrelated because they're far apart" — which is exactly the reading low continuity invalidates.

How it works

  • Fix a neighbourhood size K — the locality scale being judged.
  • Take both neighbour sets — for each item, its K nearest in the source and its K nearest on the map.
  • Score trustworthiness — penalise each map-neighbour that is not a true source-neighbour, weighted by how far it truly ranks (a distant intruder hurts more).
  • Score continuity — penalise each source-neighbour that is missing from the map's neighbourhood, weighted the same way.
  • Aggregate — into two global scores, and optionally a per-region field showing where each kind of error concentrates.

It operationalises the source neighbourhood relation and reports fidelity against it; it does not build the map or produce a single grid-topology violation count.

Tuning parameters

  • Neighbourhood size K — the scale scored. Small K judges the very-local; large K judges mid-scale structure. Match it to the neighbourhood the map will actually be read at.
  • Trust vs. continuity emphasis — which error is worse for the use at hand: false neighbours (dangerous when the map is used to assert closeness) or torn neighbours (dangerous when used to assert distance).
  • Rank weighting — how steeply to penalise by how far an intruder truly ranks; steep weighting punishes egregious errors far more than near-misses.
  • Per-item vs. aggregate — a single global pair, or a per-region field that localises where trust fails and feeds a legend.
  • Source-distance metric — which source dissimilarity defines a "true neighbour"; it must be the notion of nearness the map is meant to convey.

When it helps, and when it misleads

Its strength is naming the specific way a map lies — invented closeness versus broken closeness — instead of a single fuzzy "quality," and tying that verdict to the neighbourhood scale that matters to the reader.

Its failure mode is that both numbers depend entirely on K and the source metric: a score quoted without its K is meaningless, and the choice of K can be tuned to flatter. It also speaks only to local fidelity — a map can score high trustworthiness and continuity and still misrepresent large-scale distances. The classic misuse is reporting trustworthiness alone (the map "looks clean") while hiding poor continuity (it is silently tearing true neighbours apart). The discipline is to report both, always with K, and ideally across a sweep of K so the reader sees at which scale the map holds.[1]

How it implements the components

  • source_neighborhood_relation — it operationalises the source's K-nearest-neighbour sets as the ground truth every score is measured against.
  • task_relevance_and_sensitivity_profile — the choice of K and of trust-vs-continuity emphasis encodes which locality scale, and which error, the task is sensitive to.

It does not produce the discrete grid-topology violation rate (Topographic Error Measure), does not build the map (Multidimensional Scaling Layout, Self-Organizing Map Training), and does not enumerate individual folds (Map Fold and Collision Scan).

Notes

Trustworthiness and continuity are asymmetric and must never be collapsed into one averaged score: doing so erases which failure is happening, and the two demand opposite fixes. Low trustworthiness (false neighbours) usually means the map is too compressed and needs room; low continuity (torn neighbours) usually means it folded or split a region that should have stayed whole.

References

[1] Trustworthiness and continuity are an established pair of rank-based measures for embedding quality, built precisely to separate intrusions (false neighbours) from extrusions (lost neighbours). They are diagnostic scores, not a recipe for a good map — a projection can be improved on one only by spending fidelity on the other. ↩