Concept

Law of large numbers

The statement that the average of independent repetitions of a measurement settles on the underlying mean as their number grows. It is what makes an average an estimate at all, and it says nothing about how fast — that is a separate and finer question.

Named by 2 essays across one field — each of them below, with the objects they name alongside it.

One set of sums, two scalings, two different limits. The exact distribution of a sum of n independent copies, scaled two ways. Divided by n it collapses onto the mean; divided by the square root of n it holds a fixed width and settles into a shape.

The average settles and the wobble does not

Two theorems are usually met a page apart and sound as though one is a sharper version of the other. They are the same sums looked at through two different magnifying glasses: divide by the number of them and everything collapses to a point, divide by its square root and a shape appears.

probability · central limit

How fast each way of averaging closes in, as the dimension grows. Relative error against the number of points, both on logarithmic scales, for a regular grid in 1, 4, 8 dimensions and for random points in 8; the grid's lines steepen or flatten with the dimension and the random one does not move from a slope of a half.

The error that does not care how many dimensions

A grid gets rapidly better in one dimension and hopelessly worse in twenty. Random points get better at the same slow rate whatever the dimension, which is why a method that is bad everywhere ends up being the only one that works.

probability · monte carlo

Named alongside it

The objects these essays reach for when they reach for this one.

Central limit theorem Convergence rate Variance Convergence Curse of dimensionality Estimator bias Expectation Independence Integral Monte Carlo Normal distribution Quadrature

All concepts