Nelder-Mead Algorithm Visualization

Nelder-Mead is a derivative-free optimization method that finds a local minimum by iteratively modifying a simplex — a geometric shape with one more vertex than the space has dimensions. In 2D, the simplex is a triangle. Each step reflects, expands, contracts, or shrinks the triangle to move it toward the minimum. Here we minimize f(x, y) = −exp(−(√(x²+y²) − 2)² + x), which has a global minimum near (2.5, 0).

pseudocode
Initialize simplex with three points Evaluate f at each vertex Sort: f(u) ≤ f(v) ≤ f(w) Compute centroid c = (u + v) / 2 Reflect: r = 2c − w Evaluate f(r) If f(u) ≤ f(r) < f(v): replace w ← r Else if f(r) < f(u): Expand: e = c + 2(r − c) Evaluate f(e) If f(e) < f(r): replace w ← e Else: replace w ← r Else (f(r) ≥ f(v)): p₁ = (w + c) / 2 and p₂ = (c + r) / 2 Evaluate f(p₁) and f(p₂) If min(f(p₁), f(p₂)) < f(v): w ← best of p₁, p₂ Else: shrink — p ← u + 0.5(p − u) for all p Check convergence Repeat ↑
visualization · f(x, y) = −exp(−(√(x²+y²) − 2)² + x)
Loading…
u — best vertex
v — second vertex
✕ w — worst vertex
c — centroid
r — reflected
e — expanded
p₁, p₂ — contraction pts
✓ — accepted