Concept

Optimisation

Finding the value of a variable that makes a quantity as small or as large as possible, subject to constraints. Whether it is tractable turns almost entirely on convexity, which decides not how fast an answer is found but whether a found answer can be recognised.

Named by 12 essays across 6 fields — each of them below, with the objects they name alongside it.

A cheapest assignment, and the proof that it is cheapest. A 4 by 4 cost table with the cheapest assignment marked, and a row price and column price beside each. Every used cell's two prices add to its cost, and the prices total the assignment's cost.

A price for every person and task

The cheapest assignment can be found without comparing it to any other. Attach a number to each person and each task so that no pair's two numbers exceed its cost, and if the numbers add to an assignment's total, that assignment is cheapest — proved, by an argument that never mentions the alternatives.

applied · Assignment
A line under every point of x². The curve x² with 4 tangent lines drawn, each extended across the whole interval and each staying below the curve throughout.

A line under every point

The chord above the curve is one definition of convexity. There is a second — a line under the curve at every point, staying under everywhere — and it is the one that turns a statement about a derivative at a point into a statement about the whole function.

analysis · Convexity
x² and its conjugate. Two panels: the curve x² with tangent lines of several slopes, and the conjugate function plotted against slope, whose value at each slope is the intercept of the corresponding tangent.

The function seen from its tangents

A convex function is the upper envelope of its own tangent lines, so it can be described by giving, for each slope, how far the line of that slope has to be pushed down. That description is a second function, and applying the construction twice returns the original.

analysis · Convexity
One minimum, or several. Two curves side by side with their local minima marked: a convex one with a single minimum, and a fourth-power well with 2.

Where the guarantee stops

Convexity converts every downhill method into a correct one, and its absence removes the guarantee entirely rather than degrading it. What is left is a collection of partial answers, and knowing which of them apply to a given problem is most of what non-convex optimisation is.

analysis · Convexity
Zykov's moves, from 10 edges to 16. 6 panels showing a graph on 7 points with no complete graph on 4, changed one move at a time; the edge count rises 10, 11, 12, 13, 14, 16 and ends at Turán's graph.

Moves that only ever add edges

Turán's theorem says the densest graph avoiding a complete graph on r + 1 points is the balanced r-part graph. Zykov's proof finds it by a sequence of moves — turn a point into a copy of a better-connected one it is not joined to — each of which adds edges and none of which can create the forbidden clique. A second proof spreads a unit of weight over the points and gets the same bound from a maximum.

discrete · Extremal graphs
The postman's route: 24 blocks of street, walked in 28. A street network with its odd-degree vertices marked and the streets a shortest closed route must walk twice drawn doubled, dashed in a second colour, pairing up the odd vertices.

The streets a postman walks twice

A postman must walk every street of a district and come back. If every corner has an even number of streets, no street needs walking twice. If not, some must — and the ones repeated always join the odd corners in pairs. Pricing every way of pairing them finds the shortest round; pairing the nearest corners first does not.

discrete · Eulerian paths
The largest value on each plane, over every direction of space. A longitude–latitude map of directions u in three dimensions, each shaded by the largest value of xᵀAx on the plane perpendicular to u. The smallest such value is the middle eigenvalue 1.829, reached at ±v₁; the largest is 3.622, reached along a great circle.

The highest point on the sphere is an eigenvalue

For a symmetric matrix, walk a unit arrow over every direction and record the value of xᵀAx. The highest value reached is the largest eigenvalue, the lowest is the smallest, and every eigenvalue in between is a saddle height, a minimum of maxima. From that one description comes a theorem no formula for the roots could give: delete a row and its column, and every eigenvalue of what is left sits between two of the original's.

algebra · Eigenvectors
3 moves along the edges to the corner that maximises 2x₁ + 3x₂. The simplex method on a two-variable program with 5 constraints, started at the origin. It visits (0, 0), (0, 8), (1, 8), (5/2, 15/2), with objective values 0, 24, 26, 55/2, and stops where the prices on both binding constraints are non-negative.

Prices at every corner

The duality theorem says a linear program's best value equals its dual's, and says nothing about how to find either. The simplex method finds both at once — it walks from corner to corner, and at each one asks the constraints that meet there for prices. A negative price names an edge that climbs; when none is negative, the prices are the proof.

applied · Duality
Eight corners of a squashed cube, visited in order by the simplex method. Klee and Minty's program in 3 variables drawn as its own deformed cube and as a plain one. The simplex method with the largest-price rule visits all 8 corners, with objective values 0, 4, 6, 10, 15, 19, 21, 25; the optimum is one edge from the start.

The cube that takes every corner

The simplex method is fast on every program anybody meets in practice. In 1972 Victor Klee and George Minty squashed a cube so that the method, choosing the steepest edge each time, visits all of its corners — 2ⁿ − 1 moves in n variables, with the optimum one edge from the start.

applied · Duality
The most nearly equal dissections of a square into 3, 5 and 7 triangles found. 3 triangles, areas 0.25000, 0.50000, 0.25000, spread 2.5000e-1; 5 triangles, areas 0.19098, 0.21353, 0.19098, 0.19098, 0.21353, spread 2.2543e-2; 7 triangles, areas 0.14151, 0.14458, 0.14151, 0.14458, 0.14151, 0.14174, 0.14458, spread 3.0711e-3.

How nearly equal an odd number of triangles can be

A square cannot be cut into an odd number of triangles of equal area. It can be cut into five triangles whose areas differ by about two hundredths, seven that differ by three thousandths, nine by a ten-thousandth and a half — the spread falling by a factor of seven or more with every two triangles added, closing on equality and never reaching it. A search for the closest finds that the best five it can make have the golden ratio in their areas.

computation · Scissors congruence
Malfatti's three circles against the three largest, one at a time. A triangle with angles 40, 70, 70 degrees: Malfatti's circles cover 69.1% of it, the greedy choice of incircle then largest remaining circles covers 73.7%.

Three circles that touch and are not the largest

In 1803 Gian Francesco Malfatti asked how to cut three round columns from a triangular block of marble with as little waste as possible, and answered with three circles each touching the other two and two sides. The circles exist in every triangle and are found by solving three equations. They are never the answer to his question: the plain greedy rule — the inscribed circle first, then the largest circle that still fits, twice — always does better, by 1.4 per cent on the equilateral triangle and by almost double on a thin one.

geometry · Triangle centres
The point nearest in total to three corners, and the lines that find it. A triangle with equilateral triangles on its sides, the three lines from each corner to the opposite apex meeting at the Fermat point, where the sides subtend 120°; least total distance 7.7090.

The point nearest in total to three corners

Three towns want one depot with the least total road to all of them. The answer is not the centroid or any of the classical centres: it is the point from which every side of the triangle is seen at 120°, found by building equilateral triangles on the sides. Three straight lines through it have the same length, and that length is the least total. When an angle of the triangle reaches 120°, the answer jumps to that corner — and the same 120° rule shapes soap films and the shortest networks joining many points.

geometry · Triangle centres

Named alongside it

The objects these essays reach for when they reach for this one.

ConvexityDualityCertificateComplementary slacknessCounterexampleGradientGraphLinear programLocal minimumMatchingSimplex methodSupporting line

All concepts