Concept

Large deviations

The study of how fast the chance of a fixed-size departure from the typical falls as the sample grows. The answer is exponential in the sample size, and the exponent is computed from the summand before any sample size is chosen.

Named by 2 essays across one field — each of them below, with the objects they name alongside it.

Also named here as rate function — the same set of essays touches all of them, so they are one junction rather than several.

The chance the average clears 0.75, against the number of draws. The exact probability that the average of n draws exceeds a fixed level, on a logarithmic scale, falling along a straight line whose slope is the rate function, with the normal approximation drawn beside it and diverging.

The tail is not a bell

The limit theorem describes a window of width one over the root of n around the mean; ask instead for the chance that an average lands a fixed distance away and the answer falls exponentially, at a rate computed from the summand before any n is chosen.

probability · central limit

The 91 histograms 12 draws can produce. A triangle whose points are the possible histograms of a fixed number of draws over three faces, each drawn as a dot shaded by how far it is from the true distribution.

When the whole histogram deviates

The rung below priced a rare average. Ask instead for the chance that the whole tally of outcomes comes out wrong, and the exponent is no longer a function of one number — it is a distance between two distributions, and every rare-average rate is a shadow of it.

probability · central limit

Named alongside it

The objects these essays reach for when they reach for this one.

Central limit theorem Rate function Tail bound Concentration inequality Counting argument Empirical distribution Entropy Heavy tails Legendre transform Moment generating function Multinomial Relative entropy

All concepts