Chain rule
Named by 2 essays across one field — each of them below, with the objects they name alongside it.
A slope without a subtraction
The definition of a slope subtracts two nearly equal numbers and divides by a tiny one, and a computer that follows it loses half its digits however carefully the step is chosen. Two pieces of arithmetic avoid the subtraction altogether. Step sideways into the complex numbers by 10⁻²⁰⁰ and read off the imaginary part; or calculate with numbers a + bε in which ε² = 0, and the slope rides along exactly. Run backwards, the second gives all thousand slopes of a function of a thousand variables for under five times the cost of the function.
Running a computation backwards with little memory
The reverse sweep that gives a whole gradient for the price of a few evaluations needs the computation's intermediate states in reverse order, and storing them all can cost more memory than the machine has. Keep only a few and recompute the rest, and the cheapest schedule has an exact answer found by Andreas Griewank in 1992: with c stored states and no step run more than r times, the longest chain that can be reversed is the binomial coefficient C(c + r, c). Pascal's triangle is the table of what memory buys.
Named alongside it
The objects these essays reach for when they reach for this one.
DerivativeAdjointBinomial coefficientComplex numbersDifference quotientDynamic programmingFloating-pointGradientLower boundRecursion