Concept

Convergence rate

How quickly a sequence closes on its limit, measured by how the remaining error shrinks each step. It is what decides whether an iteration is worth running, and it can be geometric, quadratic or as slow as one over a square root.

Named by 42 essays across 9 fields — each of them below, with the objects they name alongside it.

120 needles on a lined floor. 120 needles dropped at random across evenly spaced lines; 83 of them cross a line.

Getting pi by dropping needles on the floor

Throw a needle at a lined floor enough times, count how often it crosses a line, and pi falls out. There is no circle anywhere in the experiment.

probability · Monte Carlo
Waiting for all 6 kinds. One bar per new kind: the expected number of draws needed to see a kind not yet seen, rising as fewer of them are left, and adding to 14.70 draws in total.

How long until every one turns up

Draw at random from six equally likely kinds until all six have appeared. The wait is not six draws, and it is not sixty; it is fourteen point seven, and the number is a harmonic sum wearing a hat.

probability · Expectation
A square cut into 7 pieces and a remainder. A square divided by cutting off a fixed fraction of what is left, over and over, so that the pieces are the terms of a geometric series and the uncut corner is the tail.

The sum that fits in one square

Half, then a quarter, then an eighth, forever. Adding infinitely many things sounds like it should give infinity, and the picture that says otherwise is a square with a corner left uncut.

analysis · Geometric series
five values, unevenly weighted, and the mass outside 3 standard deviations. A distribution drawn as bars, with the windows one and a half, two and three standard deviations wide marked. The probability outside each window is summed and compared with the bound that knows only the variance.

How far from the average a thing can be

Knowing only an average and a spread — nothing about the shape, nothing about the number of outcomes, nothing about symmetry — the chance of landing three standard deviations out is at most one in nine. And there is a distribution that lands there exactly that often, so the bound cannot be improved.

probability · Concentration
One set of sums, two scalings, two different limits. The exact distribution of a sum of n independent copies, scaled two ways. Divided by n it collapses onto the mean; divided by the square root of n it holds a fixed width and settles into a shape.

The average settles and the wobble does not

Two theorems are usually met a page apart and sound as though one is a sharper version of the other. They are the same sums looked at through two different magnifying glasses: divide by the number of them and everything collapses to a point, divide by its square root and a shape appears.

probability · Central limit
x ↦ cos x: two starts, one destination. A map whose graph is nowhere steeper than a fixed factor under one, with staircases from two different starting points converging on the same crossing, and the distance to it falling under a geometric bound.

A map that shrinks everything

One extra hypothesis — that every distance is shortened by at least a fixed factor — turns the existence of a fixed point into its uniqueness, an algorithm for finding it, and a bound on the error after any number of steps.

analysis · Fixed points
π(x) against its two estimates, up to 20,000. The ratio of the prime counting function to x over the logarithm of x, and to the logarithmic integral, plotted against x. The first is above one and coming down slowly; the second is close to one throughout.

Counting what has no formula

There is no expression that gives the nth prime, and yet the number of primes below a bound is predictable to within a fraction of a per cent — by a function that is not a formula for the primes but an integral of the wrong-looking quantity.

number · Prime distribution
How fast a sum becomes a bell curve. The largest gap between the distribution of a standardised sum and the bell curve, against the number of terms, on logarithmic axes. Both summands fall along a line of slope about minus a half.

How fast the bell arrives

The limit theorem says a standardised sum approaches the bell curve and says nothing about when. The rate is one over the square root of the number of terms, the constant in front is made of the third moment, and both are visible.

probability · Central limit
The same map, written in the basis of its own eigenvectors. Three panels: the map [2, 1, 1, 2] on the standard grid, the diagonal stretch by 3 and 1 it becomes on the eigenvector grid, and the two put back together.

The same map in a better basis

Measured along its own invariant directions, a linear map stops shearing and becomes two independent stretches. Nothing about the map has changed; the grid it is described against has.

algebra · Eigenvectors
How fast each way of averaging closes in, as the dimension grows. Relative error against the number of points, both on logarithmic scales, for a regular grid in 1, 4, 8 dimensions and for random points in 8; the grid's lines steepen or flatten with the dimension and the random one does not move from a slope of a half.

The error that does not care how many dimensions

A grid gets rapidly better in one dimension and hopelessly worse in twenty. Random points get better at the same slow rate whatever the dimension, which is why a method that is bad everywhere ends up being the only one that works.

probability · Monte Carlo
Two unbiased estimates of one integral, and their spread. The sharply peaked integrand with the proposal density that follows it, above a strip plot of 200 estimates from each of two methods; the weighted estimates cluster 4.2 times more tightly about the same value.

Sampling where the answer lives

Monte Carlo error cannot be made to fall faster than the square root, so the only thing left to attack is the constant in front of it. Drawing points where the integrand is large, and dividing by how often they were drawn, leaves the answer alone and can shrink the noise many times over.

probability · Monte Carlo
Points too even to be random. 256 independent random points beside 256 points of a Halton sequence, with the largest mismatch between a box's share of points and its area plotted against the number of points for both.

Points too even to be random

Independent random points clump, and the clumping is what makes the error fall only as the square root. Points chosen to be evenly spread rather than independently beat that rate, and the price is that nothing about them is random at all.

probability · Monte Carlo
The tail that would have to be a whole number. For each denominator, the value of q! times the tail of the series for e, plotted against the band between zero and one where no whole number lies, with the bound 1/q above it.

A tail too small to be a whole number

If e were a fraction with denominator q, then q! times e would be a whole number. It splits into a whole part and a tail, the tail is squeezed strictly between nothing and one, and there is no whole number there.

number · Irrationality
How fast an iteration arrives. The distance from the fixed point plotted against the step number on a logarithmic vertical axis, for two ordinary iterations and for Newton's method, whose curve bends downward.

How fast the staircase arrives

The slope at a crossing decides whether an orbit reaches it. The same number decides how fast — and when the slope is zero the arithmetic changes kind, from a fixed factor per step to a doubling of the correct digits.

dynamics · Iteration
Sampling the orders, and how fast the answer arrives. The largest error in the estimated shares against the number of orderings sampled, both on logarithmic axes, with the square-root rate drawn through the first point.

Too many orders to list

The rule is an average over every order the players could have arrived in. At seven players that is five thousand orders and at twenty it is more than there are seconds in the age of the universe — so the average is sampled, and the error falls at a rate that can be measured.

applied · Shapley value
What one input can do, over 1024 cases. A table of functions of several inputs with the largest effect any single input has on each, the bound that effect implies, and the true tail probability — computed by enumerating every input.

No single input can move it far

Independence was never the hypothesis doing the work. A quantity built from many separately drawn inputs concentrates whenever changing one of them moves it only a little — and that covers quantities which are not sums of anything and have no formula at all.

probability · Concentration
n / 2ⁿ held under a geometric series. A bar for each term of the series n / 2ⁿ with a decaying geometric curve above them, the curve lying above every bar from term 2 onwards.

The series everything else is measured against

A geometric series is not one series among many. It is the yardstick: a total exists if its terms eventually fit under one, so a single comparison settles infinitely many questions — and the test built from it says nothing at all in exactly the place where the interesting cases are.

analysis · Geometric series
Runs of doubling length in 1/n^2, and the geometric series that bounds them. A bar for the total of each run of terms of 1/n to the 2, with an outlined bar above it for the bound obtained by replacing every term in the run with its largest, the bounds forming a geometric series.

The repair at the boundary

Where the geometric yardstick says nothing, compare a series with itself at doubled spacing. That one move turns every 1/n^p back into a geometric series, reads the threshold off at p = 1, and then produces an infinite hierarchy of boundaries with no slowest divergent series anywhere in it.

analysis · Geometric series
How rare a sum of two squares is. Two curves against the logarithm of the bound: the fraction of numbers below it that are sums of two squares, falling; and that count times the square root of the logarithm, divided by the bound, which is nearly constant.

Almost no number is one

Sums of two squares look common — a sixth of all numbers up to a million are one. The fraction is falling to nothing, at a rate so slow that no computation will ever make it obvious, and the constant in front of it has been computed to fifty places and identified with nothing.

number · Sums of two squares
Waiting for all 6 when they are not equally likely. One bar per kind giving the expected wait for that kind on its own, with the rarest much the tallest, and the expected wait for the whole collection printed above them.

The one that hardly ever comes up

Make the kinds unequally likely and the tidy decomposition into stages fails, because a stage's rate now depends on which kinds turned up rather than on how many. What replaces it is an alternating sum over every subset — and the rarest kind turns out to be nearly the whole answer.

probability · Expectation
A circle trapped between two 12-sided polygons. A circle with a regular polygon of 12 sides inscribed in it and another circumscribed about it, beside a table of the bounds on pi obtained by doubling the side count.

Pinned between two sequences

The ring dissection makes the answer obvious and proves nothing. Archimedes' method proves it and makes nothing obvious — it never exhibits the area at all, it rules out every other value — and the recursion that drives it computes π by hand with one square root a step.

geometry · Circle area
How many steps Newton's method needs on (z − 1)²(z + 1). A square of the complex plane shaded in bands by the number of iterations a root-finding method needs from each starting point, darker meaning slower, with the roots marked.

A double root halves the error instead of squaring it

Near an ordinary root, Newton's method squares its error at every step and a handful of steps reach full precision. Near a double root it only halves the error — twenty steps where five would do, and a ceiling of about eight correct digits that no number of steps can break through. Doubling the step repairs the double root and ruins the simple one.

dynamics · Newton basins
Newton's and Halley's basins for z³ − 1. Two squares of the complex plane side by side, each coloured by which root a starting point converges to, the left under Newton's method and the right under Halley's, with non-converging starts marked.

A cubic method that is Newton's in disguise

Halley's method, from 1694, uses the second derivative as well as the first and cubes the error at every step where Newton's squares it. It is also, exactly, Newton's method applied to a different function — p divided by the square root of p′ — and that single fact explains why its basins are calmer, why it walks out of the trap that holds Newton for ever, and why its boundaries are still fractal.

dynamics · Newton basins
Where a jump puts the error of evenly spread points. Three unit squares with 16, 64, 256 scrambled Sobol' points, one per box, and the line y = 0.15 + 0.7x crossing 6, 12, 26 boxes.

What a jump costs evenly spread points

Scrambled Sobol' points integrate a smooth function with an error that falls like N to the power −3/2. Put a jump in the integrand along a tilted line and the exponent halves, to −3/4, because about √N of the N boxes the points fill are cut by the jump and each of those is left to chance. A jump along an axis costs almost nothing, the cost grows with the length of the jump and the dimension, and integrating across the jump before sampling gives the whole rate back.

probability · Monte Carlo
Sperner's corridor to the same fixed point, on a gentle map and a twisted one. Two 32 by 32 labelled grids; the corridor crosses 44 triangles reading 100 corners on the gentle map and 148 triangles reading 204 corners on the twisted one.

What it costs to find the point that stays put

Brouwer's theorem promises a point a map leaves where it is, and Sperner's corridor walks to it. On a gentle map the walk is short; twist the map about its fixed point and the corridor follows every turn, so its cost grows like the square of the grid. Halving the square by winding number finds the same point in a number of readings proportional to the grid's side whatever the map does — and no method that only reads the map can do fundamentally better.

topology · Fixed points
The Mandelbrot set counted cell by cell. A 96-cell-wide grid over the Mandelbrot set: 2034 cells certainly inside (1.3794), 290 undecided after 60 steps (0.1967).

The area of the Mandelbrot set, from two sides

The Mandelbrot set has an area, and nobody knows it. Counting grid cells closes in on 1.5066 from both sides, but every count judges a cell by a single point, and points on the boundary never declare themselves. An exact formula exists — π times one minus a sum over the coefficients of the map onto the set's outside — and every partial sum is a guaranteed upper bound; after sixteen thousand coefficients the bound is still 1.786, and at the rate it falls it would need something like 10²⁵ more.

algebra · Complex numbers
Polynomials climbing to the square root, each above the last. The iterates of p ↦ p + (x − p²)/2 at steps 1, 2, 3, 4, 6, 9 on [0, 1] beneath the curve √x, with largest gaps 1.000, 0.500, 0.259, 0.176, 0.108, 0.068.

Settling in order settles everywhere at once

A sequence of functions can settle at every point and never settle uniformly — the largest gap stays put while each point escapes it. Dini found the circumstances in which that cannot happen: continuous functions, a continuous limit, every member above the next, and a closed interval. Under those four conditions convergence at each point is convergence everywhere at once, and Weierstrass's iteration for the square root becomes a sequence of polynomials converging uniformly to |x|.

analysis · Uniform convergence
Running totals of random steps, stopped as they pass 4. Four staircases of uniform steps stopped on passing 4, taking 11, 8, 8, 10 steps; the exact mean is 8.666604.

Two-thirds of a step past the line

Add random numbers between nought and one until the total passes one, and it takes e of them on average. Ask the same about passing any t and the answer is an exact finite sum that settles, within a few units, onto the line 2t + 2/3. The slope is one over the average step. The two-thirds is the interesting part: it is what the one step that crosses the line charges for being chosen by its length, and it is set entirely by how much the steps vary.

probability · Expectation
The longest run of heads at three sizes, never the same shape. n=65536: 12:0.018 13:0.117 14:0.233 15:0.239 16:0.172 17:0.104 18:0.057 19:0.030 20:0.015; n=82273: 13:0.075 14:0.204 15:0.249 16:0.197 17:0.124 18:0.070 19:0.037 20:0.019; n=104408: 13:0.040 14:0.162 15:0.248 16:0.221 17:0.148 18:0.086 19:0.046 20:0.024 21:0.012.

The longest run has no limit

Toss a fair coin n times and the longest run of heads is about log₂ n. The average settles: it is log₂ n − 0.667 to within a few millionths by a million tosses. The distribution does not. Because the run is a whole number and log₂ n is not, the chance of each value swings round the same cycle once every doubling of n, for ever, and computing it exactly to a million tosses shows the swing as large at the end as at the start.

probability · Expectation
The share of numbers beginning with 1, never settling. 10^1.0: 0.2000, 10^1.8: 0.1746, 10^2.3: 0.5487, 10^2.8: 0.1841, 10^3.3: 0.5231, 10^3.8: 0.1931, 10^4.3: 0.5002, 10^4.7: 0.2022, 10^5.2: 0.4766, 10^5.7: 0.2117, 10^6.2: 0.4519, 10^6.7: 0.2217, 10^7.2: 0.4261, 10^7.7: 0.2321, 10^8.2: 0.3990, 10^8.7: 0.2431, 10^9.2: 0.3707, 10^9.6: 0.2545, 10^10.1: 0.3411, 10^10.6: 0.2665, 10^11.1: 0.3100, 10^11.6: 0.2791.

A share that depends on the average

What share of the whole numbers begin with the digit 1? Counted up to N, the answer swings between one ninth and five ninths for ever, and averaging the swing over N, even three times over, narrows it without removing it. Weight each number n by 1/n instead and the share settles at log₁₀ 2 — Benford's value — and so does every other sensible weighting that counts each decade alike. The primes behave the same way. Benford's law for the counting numbers is not a fact about them; it is a fact about how they are averaged.

probability · Central limit
Two random strings of bits and the longest thread between them. Two random binary strings of length 36; longest common subsequence 27.

The longest thread between two random strings

Write down two random strings of a thousand bits each and find the longest sequence that can be read in both, skipping freely. It is about 81% of each, and the exact share — the Chvátal–Sankoff constant — has resisted computation for fifty years. Its fluctuations are a smaller mystery with a sharper edge: the variance is known to be at most n/2, and over every length that can be computed it grows like n to the power 0.7, with nobody able to say what it does after that.

probability · Concentration
Two rays land at the root of every bulb. Parameter rays 1/3 & 2/3, 1/7 & 2/7, 5/7 & 6/7, 1/15 & 2/15, 13/15 & 14/15, 9/31 & 10/31, 21/31 & 22/31 to depth 160; furthest ray end from its root 0.0197.

Two rays for every fraction

Seen from far away, every point of the Mandelbrot set's outside has an angle, and the curves of constant angle come in to land on the set. At the root of the p/q bulb exactly two of them land, and their angles are written by the staircase under a line of slope p/q. The limbs those pairs fence off share out the whole circle of angles between them, by the identity Σ φ(q)/(2^q − 1) = 1.

algebra · Complex numbers
Strings that overlap themselves leave more behind. 00: 253.293; 01: 230.283; 02: 230.299; 03: 230.315; 04: 230.330; 05: 230.345; 06: 230.361; 07: 230.376; 08: 230.390; 09: 230.405; 10: 220.887; 11: 244.784; 12: 222.442; 13: 223.049; 14: 223.575; 15: 224.033; 16: 224.438; 17: 224.798; 18: 225.120; 19: 225.410; 20: 225.673; 21: 225.913; 22: 249.163; 23: 226.334; 24: 226.520; 25: 226.692; 26: 226.852; 27: 227.001; 28: 227.140; 29: 227.270; 30: 227.393; 31: 227.508; 32: 227.617; 33: 250.747; 34: 227.817; 35: 227.910; 36: 227.998; 37: 228.081; 38: 228.161; 39: 228.237; 40: 228.310; 41: 228.380; 42: 228.446; 43: 228.510; 44: 251.598; 45: 228.631; 46: 228.688; 47: 228.743; 48: 228.796; 49: 228.847; 50: 228.896; 51: 228.944; 52: 228.990; 53: 229.035; 54: 229.078; 55: 252.146; 56: 229.161; 57: 229.201; 58: 229.239; 59: 229.277; 60: 229.313; 61: 229.348; 62: 229.383; 63: 229.416; 64: 229.449; 65: 229.481; 66: 252.538; 67: 229.543; 68: 229.572; 69: 229.601; 70: 229.630; 71: 229.657; 72: 229.684; 73: 229.711; 74: 229.737; 75: 229.762; 76: 229.787; 77: 252.837; 78: 229.835; 79: 229.859; 80: 229.882; 81: 229.904; 82: 229.927; 83: 229.948; 84: 229.970; 85: 229.991; 86: 230.011; 87: 230.031; 88: 253.077; 89: 230.071; 90: 230.090; 91: 230.109; 92: 230.128; 93: 230.146; 94: 230.164; 95: 230.182; 96: 230.199; 97: 230.216; 98: 230.233; 99: 253.275.

What a missing string of digits leaves behind

Strike from the harmonic series every term whose denominator contains 42 and the rest adds up to 228.45. Strike out 99 instead and it adds up to 253.28. The difference is not about which numbers are lost but about waiting: 99 overlaps itself, so it takes 110 random digits on average to turn up where 42 takes 100, and the sum is almost exactly ln 10 times that wait.

analysis · Harmonic series
Fictitious play that never settles. Shapley's 3 × 3 game, 34 runs in 2,000,000 rounds; final mixtures row 0.371, 0.456, 0.173, column 0.253, 0.544, 0.203.

A best reply to the past

Let two players each answer the whole history of the other's play as though it were a fixed plan. In a zero-sum game the history closes in on the equilibrium, as Julia Robinson proved in 1951. In Lloyd Shapley's three-by-three game it never does: play runs round six profiles for ever, each run the one before plus the one three before plus two, so the runs grow by the real root of x³ = x² + 1 and every run is a fixed share of all the history.

applied · Equilibrium
Four generators climbing out of zeroland. Output ones-density from a one-bit start reaches 0.45 after: MT19937 374000, TT800 4500, WELL512 44, xorshift128 37.

Three hundred thousand outputs of nearly nothing

Start the Mersenne twister from a state with a single 1 among its 19,937 bits and it puts out zeros, then sparse ones, and only after about 340,000 outputs anything that looks random. Every linear generator has such a zeroland around the all-zero state; xorshift128 crosses it in 37 outputs, WELL512 in 44, and the twister's three-word twist makes it the slowest of all by four orders of magnitude.

computation · Pseudorandomness
Where the samples go matters more than how many. n=10: trap -2.65, simp -1.70, cc -1.74, gauss -2.07; n=20: trap -3.69, simp -3.35, cc -3.89, gauss -4.21; n=40: trap -4.31, simp -7.10, cc -8.16, gauss -8.51; n=80: trap -4.93, simp -8.28, cc -14.49, gauss -15.26

Where to take the samples

Every numerical integral is a weighted sum of samples of the curve. Spacing the samples evenly gives errors that shrink like a power of their number; choosing where they go, as Gauss did in 1814, makes six samples exact for every polynomial up to degree eleven and turns the power into a geometric rate. The surprise is that the humble trapezoid rule, evenly spaced and exact only for straight lines, does the same thing on any smooth curve that goes once round a circle — and that a single corner defeats every one of them.

analysis · The integral
Two thinned harmonic series that never stop growing. every n: 10d 23.603067, 40d 92.680619, 120d 276.887427; no digit 9: 10d 15.109789, 40d 22.589564, 120d 22.920604; as many 7s as 3s: 10d 11.327995, 40d 24.391523, 120d 43.428896; digit sum a prime: 10d 7.156214, 40d 21.096266, 120d 51.331792

A rule that counts lets the sum escape

Strike out every number containing a 9 and the harmonic series converges, to 22.92. Keep only the numbers with as many 7s as 3s and it diverges — slowly, like the square root of the number of digits. Ask for two such balances and it still diverges; ask for three and it converges. The dividing line is Pólya's theorem that a random walk comes home in one and two dimensions and not in three.

analysis · Harmonic series
Where the secant method goes, from every starting pair. x^3−x, 180×180 on [−2,2]²: to −1 10164, to 0 11954, to 1 10164, none 118.

Newton's method without a derivative

Replace Newton's tangent by the chord through the last two guesses and the method no longer needs a derivative — but it needs two starting values, so its basins live in a plane of pairs instead of on a line. The errors multiply instead of squaring, their logarithms add like Fibonacci numbers, and the digits gained grow by the golden ratio: 1, 3, 5, 9, 15, 25, 41, 67, 109.

dynamics · Newton basins
Each doubling adds a quarter of what the last one added. n=6: A=2.5980762114; n=12: A=3.0000000000; n=24: A=3.1058285412; sliver ratio 0.263305.

The error that predicts itself

Archimedes' polygons close on π slowly — six doublings of the sides for two decimals. But the error does not fall at random. Each doubling adds almost exactly a quarter of the area the doubling before it added, so the rest of the slivers can be summed before they are drawn. Liu Hui saw the quarter in 263; Snell and Huygens turned it into bounds; and the 96-gon that gave Archimedes two decimals gives six once its error is known.

geometry · Circle area
A disc that turns for ever. The map λz + z² at the golden rotation number: invariant curves from the linearising series at four fractions of its radius 0.3284, the critical orbit on the edge of the disc, and the boundary of the filled set.

A disc that turns for ever

At each fraction on the Mandelbrot set's cardioid a bulb is born. At the irrational numbers between them something rarer can happen: a disc round the fixed point rotates rigidly for ever, with no point falling in or flying out. Whether it exists depends on nothing but the digits of one continued fraction, and on one sum of their logarithms.

algebra · Complex numbers
Root two, raised to itself for ever. The cobweb of x to the power of root two from 1: 1.00000, 1.41421, 1.63253, 1.76084, 1.84091, 1.89271, 1.92700, settling on 2; the other solution 4 repels, with slope ln 4.

Root two, raised to itself for ever

Stack √2 to the power √2 to the power √2 without end and the tower settles at exactly 2. Solve the equation for a tower equal to 4 and √2 appears again, and that answer is false. The bases that give a limit run from e^(−e) to e^(1/e), the slope at the crossing decides everything, and a sum over labelled trees turns out to be the same number.

analysis · The exponential
A table with harmonic rows. The Hilbert table of size 8 and its largest stretch 1.6959390, against pi.

A table that never reaches π

Fill a square table with 1/1, 1/2, 1/3 along the rows, each row starting one place further on. Every row is a piece of the harmonic series, and yet the table stretches no vector by as much as π. At a thousand rows the stretch is 2.42, at a trillion it is near 2.94, and one integral explains both the number and the failure to arrive.

analysis · Harmonic series

Named alongside it

The objects these essays reach for when they reach for this one.

ExpectationVarianceApproximationHarmonic seriesLimitConvergenceGeometric seriesIterationFixed pointMonte CarloSamplingExact arithmetic

All concepts