Matpic
mathematics, in pictures
Concept

Posterior

The probability distribution over hypotheses after the data are taken into account, proportional to the prior times the likelihood. It is what Bayes' theorem produces, and it is only as trustworthy as the list of hypotheses and the model behind the likelihood.

Named by 3 essays across one field — each of them below, with the objects they name alongside it.

A fair coin judged by someone who thinks it is biased. Posterior probabilities of three wrong coin biases over two thousand tosses of a fair coin, for five runs; the bias 0.6 wins every run.

Certain of a coin that is not there

Bayes' theorem is exact, and it is only as good as the list of hypotheses it is given. Hand it a list that leaves out the truth and it does not hesitate: it becomes certain of the entry that is least wrong, in a precise sense — the one closest in Kullback–Leibler divergence — and if two entries are equally wrong it never settles at all. Hand it a model that assumes independence where there is none, and its intervals shrink as fast as they would for honest data while covering the truth less and less often.

probability · Bayes A fixed p-value favours the null more as the sample grows. Even odds at p=0.05 near n=76, at p=0.01 near n=1202, at p=0.001 near n=83176; p=0.05 at n=10⁶ gives 116.9 to one for the fair coin; PEAR data z=3.614, p=0.00030, BF₀₁=11.90.

A result at five per cent that favours the null

Toss a coin a million times and find a count of heads exactly at the edge of significance, p = 0.05. A test at the usual level rejects fairness. A Bayes factor comparing a fair coin with a coin of unknown bias says the opposite, by 117 to one. Both computations are right, and the sample size that separates them is about seventy-six tosses.

probability · Bayes Where to look after looking and not finding. A prior over a 16 by 12 grid and the posterior after 60 looks by the greedy rule that found nothing: the chance of having found the object by then was 0.425.

Where to look after looking and not finding

A lost object is somewhere on a map of probabilities, and every search that fails redraws the map. Bayes' rule says how: the cell searched loses probability in proportion to how good the search was, and every other cell gains. Looking where the chance of finding it on the next look is highest is provably best, and a search that overrates its own eyesight looks in the wrong place — which is how the wreck of Air France 447 was missed for two years.

probability · Bayes

Named alongside it

The objects these essays reach for when they reach for this one.

Bayes' theoremModel misspecificationBayes factorCentral limit theoremConditional probabilityCoverageCredible intervalGreedy algorithmIndependenceLikelihoodOptimisationP-value

All concepts