Central limit — the series

A Galton board after 600 balls. 600 balls fall through 12 rows of pegs, each bouncing left or right at random, and pile up in a bell-shaped heap.

A bell curve assembled out of coin flips

Drop six hundred balls through a board of pegs, each bouncing left or right at random, and they pile up in a shape that can be predicted precisely. Nothing coordinated them.

part 1 · probability

One set of sums, two scalings, two different limits. The exact distribution of a sum of n independent copies, scaled two ways. Divided by n it collapses onto the mean; divided by the square root of n it holds a fixed width and settles into a shape.

The average settles and the wobble does not

Two theorems are usually met a page apart and sound as though one is a sharper version of the other. They are the same sums looked at through two different magnifying glasses: divide by the number of them and everything collapses to a point, divide by its square root and a shape appears.

part 2 · probability

A lopsided distribution added to itself, and the shape that returns. On the left, the exact distribution of a sum of copies of one lopsided distribution, standardised, for several counts: the shapes converge. On the right, the bell curve convolved with itself, which is the bell curve again.

The shape that averaging leaves alone

Adding independent quantities blurs their distributions together, and rescaling restores the width. Almost every shape is changed by that operation. Exactly one is returned unaltered, and that is why sums of unrelated things keep arriving at it.

part 3 · probability

Averages of a heavy-tailed quantity, which never settle. Running averages of draws from a Cauchy distribution, which jump rather than converge, beside the cumulative distributions of averages of 1, 4 and 16 draws, which lie on top of one another.

An average that never settles

The average of many independent quantities is supposed to steady as their number grows. For one famous distribution it does not steady at all — the average of a thousand draws has exactly the same distribution as a single draw, and no amount of further averaging changes it.

part 4 · probability

How fast a sum becomes a bell curve. The largest gap between the distribution of a standardised sum and the bell curve, against the number of terms, on logarithmic axes. Both summands fall along a line of slope about minus a half.

How fast the bell arrives

The limit theorem says a standardised sum approaches the bell curve and says nothing about when. The rate is one over the square root of the number of terms, the constant in front is made of the third moment, and both are visible.

part 5 · probability

The chance the average clears 0.75, against the number of draws. The exact probability that the average of n draws exceeds a fixed level, on a logarithmic scale, falling along a straight line whose slope is the rate function, with the normal approximation drawn beside it and diverging.

The tail is not a bell

The limit theorem describes a window of width one over the root of n around the mean; ask instead for the chance that an average lands a fixed distance away and the answer falls exponentially, at a rate computed from the summand before any n is chosen.

part 6 · probability

The 91 histograms 12 draws can produce. A triangle whose points are the possible histograms of a fixed number of draws over three faces, each drawn as a dot shaded by how far it is from the true distribution.

When the whole histogram deviates

A rare average has a price, an exponent that grows with the number of trials. Ask instead for the chance that the whole tally of outcomes comes out wrong, and the exponent is no longer a function of one number — it is a distance between two distributions, and every rare-average rate is a shadow of it.

part 7 · probability

Leading digits of products of random numbers, against Benford's law. 1 factor: ones 11.1%, nines 11.1%; 2 factors: ones 24.1%, nines 3.4%; 3 factors: ones 30.1%, nines 4.2%; 6 factors: ones 30.1%, nines 4.6%; the law gives 30.1% and 4.6%.

Multiplying makes the digit one common

Multiply a few random numbers together and the product starts with 1 about 30 per cent of the time and with 9 under 5 per cent — Benford's law, which no factor contains. The logarithm of a product is a sum, the central limit theorem spreads that sum across many powers of ten, and once it is spread its fractional part is uniform. The approach is geometric, at a rate fixed by a single number for each kind of factor; sums never get there, and the powers of two get there with no randomness at all.

part 8 · probability

A share that depends on the average

What share of the whole numbers begin with the digit 1? Counted up to N, the answer swings between one ninth and five ninths for ever, and averaging the swing over N, even three times over, narrows it without removing it. Weight each number n by 1/n instead and the share settles at log₁₀ 2 — Benford's value — and so does every other sensible weighting that counts each decade alike. The primes behave the same way. Benford's law for the counting numbers is not a fact about them; it is a fact about how they are averaged.

part 9 · probability