Concept

Floating-point

The way computers store real numbers: a fixed number of significant binary digits and an exponent. Every operation rounds to the nearest representable value, so long calculations can lose accuracy through cancellation even when each step is correct.

Named by 2 essays across 2 fields — each of them below, with the objects they name alongside it.

$The orbit a computer draws, and the orbit. Two orbits of the tent map from the same starting fraction plotted against the step number — one computed exactly in whole-number arithmetic and periodic, one computed in double precision and reaching zero.$

The orbit a computer draws

A chaotic orbit computed in floating point is not the orbit of the point it started from. Sometimes it is the true orbit of a nearby point, which is enough; sometimes the arithmetic simply runs out, and the picture is of the rounding.

dynamics · Iteration

Four ways to a slope, and the one that subtracts. f'(1.5)=4.053427893898620; forward best 2.60e-9 at h=1.0e-8; central best 6.41e-12 at h=3.2e-6; complex step at 1e-200: 1.00e-17; dual: 1.00e-17.

A slope without a subtraction

The definition of a slope subtracts two nearly equal numbers and divides by a tiny one, and a computer that follows it loses half its digits however carefully the step is chosen. Two pieces of arithmetic avoid the subtraction altogether. Step sideways into the complex numbers by 10⁻²⁰⁰ and read off the imaginary part; or calculate with numbers a + bε in which ε² = 0, and the slope rides along exactly. Run backwards, the second gives all thousand slopes of a function of a thousand variables for under five times the cost of the function.

analysis · The derivative

Named alongside it

The objects these essays reach for when they reach for this one.

Adjoint Binary expansion Chain rule Complex numbers Derivative Difference quotient Exact arithmetic Gradient Iteration Numerical artefact Periodic orbit Rounding error

All concepts