Entropy
Named by 6 essays across 4 fields — each of them below, with the objects they name alongside it.
The orbit that must come back
A system with finitely many states has to repeat itself. Poincaré showed the same thing holds when the states are a continuum — almost every starting point returns arbitrarily close to where it began, however complicated the rule, and the argument is the pigeonhole principle with volume in place of counting.
When the whole histogram deviates
A rare average has a price, an exponent that grows with the number of trials. Ask instead for the chance that the whole tally of outcomes comes out wrong, and the exponent is no longer a function of one number — it is a distance between two distributions, and every rare-average rate is a shadow of it.
Fair bits from an unfair coin
Read a biased coin's flips in pairs, keep 01 as 0 and 10 as 1, and throw away the rest: the output is exactly fair, whatever the bias, and nobody needs to know the bias. The trick wastes most of the coin, the waste can be recycled almost up to the ceiling Shannon's entropy sets — and it fails quietly the moment the flips remember each other.
The rate a noisy channel allows
A channel that flips one bit in ten can still carry messages with as few errors as anyone likes — at up to 0.531 message bits per transmitted bit, and at no rate above that. The number is Shannon's capacity, 1 − H(p). Repetition reaches reliability only by sending nothing; a code chosen at random gets there at any rate below the limit; and the reason there is a limit at all is a count of how many flip patterns a block of noise can hold.
Two sign patterns that land together
At λ = 1/φ the sign patterns + − − and − + + land in exactly the same place, because λ² + λ = 1. That one coincidence, repeated wherever it fits, puts 2ⁿ patterns onto a Fibonacci number of points, leaves the random sum's transform ringing at the same height forever, and makes a distribution that fills a whole interval live on a set of no length.
Two weak sources make one fair bit
No fixed rule can turn every weakly random source into fair bits: for any rule, some source with almost full unpredictability makes it constant. Two independent sources are different. Multiply their bits in pairs, add, and keep the parity — and if the two together carry more unpredictability than the length of one, the result is nearly fair, whatever else the sources do. The reason is that the table of those parities is balanced on every large rectangle.
Named alongside it
The objects these essays reach for when they reach for this one.
IndependenceAlgebraic integerBernoulli convolutionBiasBinaryBinomial distributionCentral limit theoremChannel capacityCharacteristic functionCounting argumentEmpirical distributionError-correcting code