Concept

Central limit theorem

The statement that a standardised sum of many independent contributions is approximately bell-shaped. It describes a window of width one over the square root of the count around the mean, and says nothing about probabilities further out than that.

Named by 8 essays across 3 fields — each of them below, with the objects they name alongside it.

One set of sums, two scalings, two different limits. The exact distribution of a sum of n independent copies, scaled two ways. Divided by n it collapses onto the mean; divided by the square root of n it holds a fixed width and settles into a shape.

The average settles and the wobble does not

Two theorems are usually met a page apart and sound as though one is a sharper version of the other. They are the same sums looked at through two different magnifying glasses: divide by the number of them and everything collapses to a point, divide by its square root and a shape appears.

probability · Central limit
The chance the average clears 0.75, against the number of draws. The exact probability that the average of n draws exceeds a fixed level, on a logarithmic scale, falling along a straight line whose slope is the rate function, with the normal approximation drawn beside it and diverging.

The tail is not a bell

The limit theorem describes a window of width one over the root of n around the mean; ask instead for the chance that an average lands a fixed distance away and the answer falls exponentially, at a rate computed from the summand before any n is chosen.

probability · Central limit
The 91 histograms 12 draws can produce. A triangle whose points are the possible histograms of a fixed number of draws over three faces, each drawn as a dot shaded by how far it is from the true distribution.

When the whole histogram deviates

A rare average has a price, an exponent that grows with the number of trials. Ask instead for the chance that the whole tally of outcomes comes out wrong, and the exponent is no longer a function of one number — it is a distance between two distributions, and every rare-average rate is a shadow of it.

probability · Central limit
How fast each way of averaging closes in, as the dimension grows. Relative error against the number of points, both on logarithmic scales, for a regular grid in 1, 4, 8 dimensions and for random points in 8; the grid's lines steepen or flatten with the dimension and the random one does not move from a slope of a half.

The error that does not care how many dimensions

A grid gets rapidly better in one dimension and hopelessly worse in twenty. Random points get better at the same slow rate whatever the dimension, which is why a method that is bad everywhere ends up being the only one that works.

probability · Monte Carlo
Descents of 10 as a sum of 9 hidden coins. Bars of 9 coin probabilities beside a bar chart of the descents distribution for n = 10, with dots giving the coin-sum distribution landing on every bar.

Coins hidden in the roots

The polynomial that counts permutations by their descents has no product formula, and nothing in the definition of a descent is a coin toss. But every root of the polynomial is real and negative, and a polynomial like that is a product of coins in disguise: each root r is a coin landing heads with chance 1/(1 − r). The descent count of a random permutation is exactly a sum of independent coins nobody can point to — which is why it is bell-shaped, and why its coefficients obey inequalities the inversion count breaks.

discrete · Generating functions
How many different primes divide a number up to ten million. The share of numbers up to 10⁷ with each count of distinct prime factors (6.7%, 25.4%, 36.4%, 23.9%, 6.9%, 0.7%, 0.0%, 0.0%), against the Erdős–Kac normal curve with mean and variance 2.78.

How many primes a typical number has

A typical number near N has about log log N different prime factors, and the count is spread around that in a bell curve whose variance is log log N as well. Both facts are theorems. Neither is visible at any size anyone can count: up to ten million the average is right and the spread is less than half what the limit says.

number · Unique factorisation
Five urns, each drawn as walks. Simulated walks of 2000 draws for urns with replacement matrices (0,1,1,0), (2,1,1,2), (3,1,1,3), (7,1,1,7), (1,0,0,1).

An urn forgets its start only below one half

Let each draw from an urn add balls of both colours in fixed amounts, and the long run depends on a single ratio of two eigenvalues. Below one half the urn behaves like a coin, its fluctuations spread like the square root of the draws and settle into a bell. Above one half the first few draws decide most of the outcome, the spread grows faster, and the shape that results is not a bell and depends on how the urn began.

probability · Random walk
Leading digits of products of random numbers, against Benford's law. 1 factor: ones 11.1%, nines 11.1%; 2 factors: ones 24.1%, nines 3.4%; 3 factors: ones 30.1%, nines 4.2%; 6 factors: ones 30.1%, nines 4.6%; the law gives 30.1% and 4.6%.

Multiplying makes the digit one common

Multiply a few random numbers together and the product starts with 1 about 30 per cent of the time and with 9 under 5 per cent — Benford's law, which no factor contains. The logarithm of a product is a sum, the central limit theorem spreads that sum across many powers of ten, and once it is spread its fractional part is uniform. The approach is geometric, at a rate fixed by a single number for each kind of factor; sums never get there, and the powers of two get there with no randomness at all.

probability · Central limit

Named alongside it

The objects these essays reach for when they reach for this one.

VarianceConvergence rateLarge deviationsLaw of large numbersNormal distributionRate functionScalingTail boundCharacteristic functionConcentration inequalityConvergenceCounting argument

All concepts