Optimization & Search Algorithms

Primes about formal methods for finding a best solution under constraints, spanning algorithmic techniques (dynamic programming, greedy algorithms, branch and bound, linear/integer programming), landscape and representation concepts (optimization landscape, local optimum, problem representation), and tradeoff framing (regularization, multiobjective optimization, sensitivity analysis).

21 primes in this family — primes that sit near one another in abstraction space (k-means over structural-signature embeddings). Each is shown with its short description.

• Backcasting — Work backward from desired future.
• Branch and Bound — Systematic search with pruning.
• Dynamic Programming — Solve via subproblem reuse.
• Greedy Algorithm — Committing irrevocably to the locally best choice at each step, with no lookahead.
• Heuristic — Mental shortcuts.
• Integer Linear Programming (ILP) — Discrete optimization with integer variables.
• Linear Programming (LP) — Optimize linear objective with constraints.
• Local Optimum — A point best within its neighborhood but not across the whole landscape.
• Marginal Analysis — Incremental effects.
• Markov Decision Processes (MDPs) — Sequential decision-making under uncertainty.
• Multiobjective Optimization — Balance competing objectives.
• Network Flow Models — Optimize flow across networks.
• No Free Lunch Theorem — Averaged over all problem instances, no search, optimization, or learning procedure outperforms any other; a method's value is the match between its inductive bias and the problems it faces.
• Optimization — Finds best solution under constraints.
• Optimization Landscape — The topology of a value function over a configuration space, whose shape predicts which search strategies will succeed or get stuck.
• Problem Representation — The chosen encoding of a problem fixes which operations and intermediate states are available, and therefore which solutions can be found at all, before any solving begins.
• Problem Space — Range of possibilities.
• Regularization — Add a tunable soft penalty on the complexity of a candidate solution to a fitting procedure, trading data-fit against complexity by an explicit weight chosen for out-of-sample performance.
• Scheduling — Organizing tasks over time.
• Sensitivity Analysis (in Operations Research) — Analyze impact of parameter variation.
• Simulated Annealing — Probabilistic search escaping local optima.