Gradient
Named by 4 essays across 2 fields — each of them below, with the objects they name alongside it.
A line under every point
The chord above the curve is one definition of convexity. There is a second — a line under the curve at every point, staying under everywhere — and it is the one that turns a statement about a derivative at a point into a statement about the whole function.
Where the guarantee stops
Convexity converts every downhill method into a correct one, and its absence removes the guarantee entirely rather than degrading it. What is left is a collection of partial answers, and knowing which of them apply to a given problem is most of what non-convex optimisation is.
Moving a map across a product
The transpose looks like a fact about a matrix: reflect its entries in the diagonal. It is a fact about the inner product. Measure lengths and angles differently and the map that slides to the other side of the product is a different matrix, and a matrix that was symmetric stops being so.
A slope without a subtraction
The definition of a slope subtracts two nearly equal numbers and divides by a tiny one, and a computer that follows it loses half its digits however carefully the step is chosen. Two pieces of arithmetic avoid the subtraction altogether. Step sideways into the complex numbers by 10⁻²⁰⁰ and read off the imaginary part; or calculate with numbers a + bε in which ε² = 0, and the slope rides along exactly. Run backwards, the second gives all thousand slopes of a function of a thousand variables for under five times the cost of the function.
Named alongside it
The objects these essays reach for when they reach for this one.
AdjointConvexityDerivativeLocal minimumOptimisationChain ruleComplex numbersComplexityCounterexampleDifference quotientEigenvectorFloating-point