Skip to content

Nonlinear Dependence Screen

Diagnostic screen — instantiates Correlation Structure Characterization

Runs form-agnostic dependence statistics to catch relationships a linear or rank coefficient scores as near-zero, so real structure isn't dismissed as no-relationship.

Pearson's r answers a narrow question — how linear is the association — and answers it as zero for a great many relationships that are strongly dependent but curved, humped, or threshold-shaped. Nonlinear Dependence Screen computes model-free dependence statistics — distance correlation, mutual information, the maximal information coefficient — across variable pairs to flag exactly the dependence that linear and rank coefficients miss. Its defining target is the false negative: the coefficient that reads ≈ 0 over a genuine U-shape or periodic relation, tempting an analyst to conclude "no relationship" when the truth is "no linear relationship." Where the diagnostic panel shows the data to the eye, this screen returns a number — a form-agnostic dependence score you can rank, threshold, and act on — together with the shape that linear correlation left out.

Example

A process chemist relates reaction yield to temperature across a run of experiments. Yield climbs with temperature to an optimum, then falls off as side-reactions take over — an inverted-U. Because the rising and falling halves cancel, Pearson's r across the full range sits near zero, and a linear screen would flag temperature as irrelevant to yield and drop it from the model. The nonlinear screen tells the opposite story: distance correlation is high, marking temperature as strongly dependent, and the profile it returns shows the non-monotone hump. The variable that a linear scan discarded is in fact one of the most important, and only a form-agnostic measure could see it.

How it works

The screen's engine is a class of statistics that are zero only under genuine independence — unlike Pearson's r, which is zero for many dependent forms. Run as a wide sweep over many variable pairs, it separates "no dependence" from "no linear dependence," then hands the flagged pairs on for their actual form to be characterized. It deliberately answers the coarse question first — is there any dependence here? — before the fine one — what shape is it? — and it complements linear coefficients rather than replacing them: the linear number still says how much of the association is straight-line, while the screen says how much total dependence exists.

Tuning parameters

  • Statistic choice — distance correlation, mutual information, MIC, or a kernel measure (HSIC), each with its own bias, variance, and interpretability. More powerful detectors are often harder to read.
  • Bandwidth or binning — how mutual-information-style estimators discretize the data. Fine resolution catches subtle structure but manufactures dependence from noise at small n.
  • Screening threshold — how strong a nonlinear score must be to flag. Lower thresholds miss less and cry wolf more.
  • Multiplicity awareness — when the sweep spans many pairs, the flags must be passed to a chance-baseline test rather than trusted individually.
  • Detection resolution — for grid-based measures, how fine a grid to search, trading detection of intricate structure against false alarms.

When it helps, and when it misleads

Its strength is that it rescues real relationships a linear coefficient would bury, and it makes the crucial distinction the archetype keeps insisting on — between no relationship and no linear relationship — into something measurable rather than merely warned about.

Its weaknesses are the price of model-free power. A high score says "dependent," not "how" — it buys sensitivity at the cost of interpretability. The estimators are data-hungry, and at small n or fine binning they can conjure dependence out of noise; being more sensitive, they also raise more false positives across a wide sweep. The classic misuse is reading a high nonlinear score as if it established a usable mechanism or a cause. The discipline that guards against this is to pair every flag with the diagnostic panel (to see the form), with a chance-baseline test (to check it survives permutation across many pairs), and with a labeling step before any of it is called causal. A near-zero linear coefficient is never evidence of independence — only of no straight-line relationship.[1]

How it implements the components

Nonlinear Dependence Screen realizes the form-and-strength side of the archetype — quantifying dependence the linear lens omits:

  • relation_strength_indicator — its model-free score (distance correlation, mutual information) is a strength indicator that registers curved and non-monotone association a Pearson or Spearman coefficient reads as weak.
  • correlation_or_dependence_profile — a flagged pair's returned shape (hump, threshold, periodicity) is the dependence profile that the linear coefficient failed to represent.

It does NOT choose the measure a priori (dependence_measure_policy, measurement_context_record — Dependence-Measure Selection Matrix) nor vet raw-data integrity (paired_observation_alignment, missingness_and_measurement_error_record — Joint-Distribution Diagnostic Panel); and it does not test a flag against a chance baseline (multiplicity_control_record — Permutation Null and Multiplicity Check).

Notes

A flag from the screen is a beginning, not a conclusion. It establishes that dependence exists, at which point the form still has to be seen (panel), the chance baseline cleared (permutation), and the causal reading withheld (labeling). Strength is not shape, and shape is not mechanism.

References

[1] Pearson's correlation measures only linear association, so r ≈ 0 is fully consistent with strong nonlinear dependence — the textbook case is y = x² over a range symmetric about zero, where the positive and negative slopes cancel to a linear correlation of zero. Model-free measures such as distance correlation and mutual information, which vanish only under true independence, are the standard correctives. ↩