Concept

Entropy

A number measuring how spread out a distribution is, computed as the average logarithm of one over each outcome's probability. It is the shortest average length any code for those outcomes can achieve, in the units the logarithm is taken in.

Named by 2 essays across 2 fields — each of them below, with the objects they name alongside it.

Rotating by √2 − 1 of a turn, 40 times. Points on a circle produced by repeatedly turning through the same angle.

The orbit that must come back

A system with finitely many states has to repeat itself. Poincaré showed the same thing holds when the states are a continuum — almost every starting point returns arbitrarily close to where it began, however complicated the rule, and the argument is the pigeonhole principle with volume in place of counting.

dynamics · pigeonhole

The 91 histograms 12 draws can produce. A triangle whose points are the possible histograms of a fixed number of draws over three faces, each drawn as a dot shaded by how far it is from the true distribution.

When the whole histogram deviates

The rung below priced a rare average. Ask instead for the chance that the whole tally of outcomes comes out wrong, and the exponent is no longer a function of one number — it is a distance between two distributions, and every rare-average rate is a shadow of it.

probability · central limit

Named alongside it

The objects these essays reach for when they reach for this one.

Central limit theorem Counting argument Empirical distribution Irrational rotation Iteration Large deviations Measure Multinomial Orbit Pigeonhole Rate function Recurrence

All concepts