Coarse Grid Search¶
Method — instantiates Coarse-to-Fine Search
Evaluates a bounded parameter or design space on a rough regular grid first, then places a finer grid around the most promising cells and repeats until improvement stalls.
Coarse Grid Search discretizes a bounded continuous space into a regular lattice, evaluates every cell of a rough lattice first, then zooms: it lays a finer lattice only over the neighborhood of the best cell (or cells) and evaluates that, iterating the zoom until the grid is fine enough that further refinement no longer moves the objective. Its distinctive premise is regularity — the coarse representation is a grid whose spacing is the resolution knob, so refinement is literally "shrink the spacing near the winner." Unlike a mechanism that scans an irregular or perceptual space, everything here hangs on the bounds of the box and the coarseness of the mesh laid across it.
Example¶
An engineer is tuning a gradient-boosted model with three knobs: learning rate, tree depth, and subsample fraction. Trying every combination at fine spacing would mean tens of thousands of training runs. Instead she bounds each knob (learning rate 0.01–0.3 on a log axis, depth 2–10, subsample 0.5–1.0) and lays a coarse 4×4×3 grid — 48 runs she can launch in parallel overnight. Validation scores cluster: the best cell sits at learning-rate 0.1, depth 6, subsample 0.8, with its neighbors close behind and the far corners clearly worse.
The next morning she throws away the rest of the box and lays a finer grid only around that cell — halving the spacing on each axis over a small window. Twenty-seven more runs. The winner nudges to learning-rate 0.08, depth 7. A third, still-finer zoom moves the score by less than the noise between reruns, so she stops. Three cheap grids, roughly a hundred runs total, land within a whisker of what exhaustive fine search would have found — because the objective was smooth enough that a coarse mesh pointed reliably at the right basin.
How it works¶
- Fix the box. State each dimension's lower and upper bound (and whether it is linear or log-scaled); this boundary is what the coarse mesh must cover.
- Lay a coarse regular grid across the whole box and evaluate every cell — the exhaustive-but-cheap first pass.
- Rank cells and select the promising neighborhood(s) — usually the top cell and its immediate neighbors, sometimes the top k to hedge against a rugged surface.
- Refine by re-gridding locally. Lay a finer lattice over the selected neighborhood, shrinking the spacing by a fixed zoom factor, and evaluate.
- Stop when the best cell's improvement between successive zooms falls below a tolerance, or the spacing reaches the resolution beyond which differences are meaningless.
Tuning parameters¶
- Coarse grid spacing — the resolution of the first pass. Fewer, wider cells cost less but risk stepping over a narrow optimum; denser first grids are safer but erode the whole efficiency argument.
- Number of cells refined — zoom around only the single best cell (fast, greedy, basin-trapping) or the top k (robust to a bumpy surface, more evaluations).
- Zoom factor — how much the spacing shrinks each round. Aggressive zoom converges quickly but can lock onto a local basin; gentle zoom explores the neighborhood more thoroughly.
- Axis scaling — linear vs log (or other) spacing per dimension. The right scaling is what makes a fixed number of cells informative; the wrong scaling wastes the whole grid on a dead region.
- Stop tolerance — the improvement-per-zoom below which search halts. Tight tolerances chase noise; loose ones quit while real gains remain.
When it helps, and when it misleads¶
Its strengths are simplicity, perfect parallelism, and reproducibility: every cell is independent, the whole pass fans out across machines, and the procedure leaves an exact record of what was tried. It shines on smooth, low-dimensional objectives where a coarse mesh is a trustworthy pointer toward the good basin.
It misleads in two classic ways. First, cost explodes with dimensions — a grid of g points per axis over d axes is g^d evaluations, so even a coarse grid becomes unaffordable past a handful of dimensions, the curse of dimensionality.[n1] Second, a narrow or off-lattice optimum can fall cleanly between coarse grid lines and never be seen; greedy zoom onto the best coarse cell then confidently refines the wrong basin. The classic misuse is trusting a too-coarse grid on a rugged surface. The standard corrective — well documented for hyperparameter tuning — is to prefer random sampling over a fixed lattice in high dimensions, and to refine around several top cells rather than one, so a single unlucky mesh alignment cannot decide the outcome.
How it implements the components¶
search_space_boundary— the explicit per-dimension bounds (and scaling) of the box define exactly what the grid must cover.coarse_representation— the rough regular lattice is the low-resolution model of the whole space.promising_region_filter— ranking cells by objective and selecting the top cell(s) is the filter that marks where to refine.refinement_step— re-gridding at finer spacing over the selected neighborhood adds the targeted detail.stop_rule— the improvement-per-zoom tolerance ends refinement when further detail stops paying.
It carries no audit of the cells it skipped over (false_negative_check, backtracking_path) and no notion of maintaining a varied slate (diversity_quota) — those belong to Diagnostic Narrowing and Portfolio Screening.
Related¶
- Instantiates: Coarse-to-Fine Search — the mesh-then-zoom procedure for continuous parameter and design spaces.
- Sibling mechanisms: Multi-Resolution Search · Funnel Process · Diagnostic Narrowing · Portfolio Screening · Progressive Candidate Review · Design Downselection · Search Tree Pruning with Refinement
Editorial Notes¶
Form Classification¶
Form family: Analysis, Modeling & Optimization
Rationale: Evaluates a bounded parameter or design space on a rough regular grid first, then places a finer grid around the most promising cells and repeats until improvement stalls, making its operative form a computation, comparison, model, or analytic representation used to infer, estimate, or choose.
Independent corroboration: The frozen evidence defines Coarse Grid Search as 'Evaluates a bounded parameter or design space on a rough regular grid first, then places a finer grid around the most promising cells and repeats until improvement stalls', so its operative form is Analysis, Modeling & Optimization.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Computer Science & Software Engineering
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Computer science and machine learning established exhaustive grid search over bounded hyperparameter boxes as a reproducible, parallelizable tuning procedure, with resolution set by the values sampled on each axis.
Related originating lineages:
- Mathematics — Numerical analysis contributes discretization, mesh resolution, and successive local refinement of a regular grid.
- Operations Research — Optimization contributes bounded search spaces, objective evaluation, promising-region refinement, and stopping tolerances.
Review resolution: Machine-learning research documents regular grid search over bounded hyperparameter boxes and its reproducible parallel evaluation, directly matching the source mechanism. Operations research supplies the optimization frame and numerical mathematics supplies mesh resolution and local refinement, so computer science is primary with both formal lineages retained.
Attribution caveat: Regular-grid numerical search also belongs to operations research and numerical analysis, but the source mechanism's hyperparameter example, parallel runs, and contrast with random search track the machine-learning lineage most closely.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
Notes¶
[n1] The curse of dimensionality names the exponential growth of a regular grid's cell count with the number of dimensions, which makes uniform grid search intractable in high-dimensional spaces. Bergstra and Bengio's finding that random search often outperforms grid search for neural-network hyperparameters is the standard reason practitioners abandon fixed lattices as dimensions grow. ↩