Skip to content

Control Group Comparison

Quasi-experimental method — instantiates Rebound-Aware Efficiency Governance

Compares treated units against otherwise-similar untreated ones to recover what total use would have been without the efficiency program — separating the real saving from the rebound and from what would have happened anyway.

Control Group Comparison answers the question the whole archetype turns on: did the efficiency intervention actually cut total resource use, once you subtract what would have happened anyway? It compares units that received the measure — buildings, stores, households — against otherwise-similar units that did not, and reads the difference as the causal effect. Its distinctive contribution is the counterfactual. A before-and-after look credits the program with every change, including weather, prices, and secular trend; the untreated group holds those constant, so what stands out is the net saving with rebound already subtracted. Where an elasticity experiment deliberately varies a lever to trace a response curve forward, this looks backward at an intervention already made and asks whether it worked.

Example

A retail chain retrofits lighting in 200 stores and wants the real energy saving, not the brochure figure. A simple before-after would mislead: that winter was milder and foot traffic was down, each cutting energy on its own. So the chain compares the 200 retrofit stores against a matched set of non-retrofit stores over the same window — a difference-in-differences read.[1] The retrofit stores did use less than the controls — but by noticeably less than the engineering estimate promised, because managers, seeing that lighting was now cheap, left displays and signage lit longer. That gap between the engineered number and the control-adjusted number is the rebound, and only the comparison makes it visible. The output is the true net saving and a quantified take-back.

How it works

The mechanism's whole job is to construct a credible stand-in for the world without the intervention. The treated units supply the observed outcome; the control units supply the counterfactual; the difference — ideally a difference of differences, netting out any pre-existing gap — is the causal effect. Its validity rests entirely on the control group being genuinely comparable, which is why the design work is really the selection and testing of controls: matching on the traits that drive use, and checking that treated and untreated moved in parallel before the intervention.

Tuning parameters

  • Control selection — randomized assignment, matched comparison, or a synthetic control built from many untreated units. Randomization is cleanest; matching and synthetic methods rescue evaluation when randomizing is impossible.
  • Matching covariates — which traits the control is balanced on. Miss a driver of use and the estimate absorbs its effect as if it were the program's.
  • Observation window — how much pre- and post-period is compared. Too short misses slow rebound; too long lets unrelated shocks creep in.
  • Spillover handling — whether treatment leaks into the control (shared managers, shared suppliers). Contaminated controls shrink the measured effect toward zero.

When it helps, and when it misleads

Its strength is isolating the causal net effect: it subtracts confounds and secular trend that fool a naïve comparison, and it exposes rebound precisely as the gap between the engineering estimate and the control-adjusted result. When an efficiency program has already been deployed and the question is whether it truly moved the total, this is the honest way to answer.

It misleads when the control is not what it claims to be. Selection bias — treated and untreated differing on something unobserved — biases the effect in unknown directions; contamination and spillover blur the two groups; and for a universal policy there may be no untreated group to compare against. It also cannot see resource use that leaked across the system boundary. The classic misuse is to choose a control that flatters the program, or to run a bare before-after and dress it up as a controlled study. The discipline is to pre-specify controls, test the parallel-trends assumption, and prefer randomization wherever it is feasible.

How it implements the components

  • baseline_and_counterfactual_measure — the untreated group is the counterfactual; constructing and validating it is the mechanism's core act.
  • elasticity_and_behavior_estimate — the recovered treatment effect is an estimate of the behavioral response net of trend, including the rebound visible as the shortfall against the engineering forecast.

It does not vary the lever prospectively to trace a full response curve — that is Elasticity Experiment — nor value the measured burdens in money, which is Full-Cost Accounting.

References

[1] Difference-in-differences compares the change in a treated group with the change in a comparable untreated group over the same period, cancelling out shocks common to both. Its key assumption is parallel trends — that, absent the intervention, the two groups would have moved together — which is testable on pre-intervention data and must be checked, not assumed. ↩