Matpic
mathematics, in pictures
Concept

Bayes' theorem

The rule for turning a probability around: how likely a cause is, given that its effect has been observed. It is computed from the prior chance of the cause and how likely the effect is under each cause, and is most easily drawn as two rectangles.

Named by 9 essays across 2 fields — each of them below, with the objects they name alongside it.

Bayes' theorem as two rectangles. A unit square split by how common the condition is (1.0%) and then by how the test behaves. Of everyone who tests positive, the fraction who have it is 16.7%.

Bayes' theorem is a picture of a square

A test that is 99% accurate returns a positive result. The chance it is right can easily be under one in five, and the reason is visible the moment the population is drawn as a square rather than described as a formula.

probability · Bayes Three doors, as areas. Staying wins 33.3% of the time and switching wins 66.7%, because the host's choice is constrained by what the host can see, so opening a door rules a region out without moving any boundary.

The door that was not opened

Three doors, one prize, a host who opens a losing door and offers a swap. Switching wins two times in three, and the reason is not about doors — it is about what the host was allowed to do.

probability · Bayes Two children, and at least one is a boy. Four equally likely families drawn as quarters of a square: the question “is at least one a boy?” rules out only the girl–girl family, and leaves three equal quarters; the chance of two boys is 33.3%.

Two children and the sentence about one of them

A family has two children and at least one is a boy. The chance that both are boys is one in three — or one in two, or anything from one in three to certainty — and every one of those answers is right for some way the sentence could have come to be said. There is no host and no door, and the protocol is still the whole problem.

probability · Bayes One experiment, measured by runs and by awakenings. Two unit squares for the same coin and the same schedule: the same experiment weighed two ways: by runs, heads keeps half the square; by awakenings, heads is one of 3 equal slices.

One coin, counted by runs and by wakings

Beauty is put to sleep and a fair coin is tossed. Heads, she is woken once; tails, twice, with the first waking erased from her memory. Each time she wakes she is asked how likely heads is. One half, say some; one third, say others; and unlike every earlier puzzle of this kind, stating the protocol exactly does not end the argument.

probability · Bayes A majority of independent voters, more often right than any of them. The probability that a simple majority of n independent voters is right, for n from 1 to 201, when each voter is right with probability 0.45, 0.51, 0.55, 0.6, 0.7.

A majority wiser than its members

Condorcet's other theorem turns voting round: the voters no longer have preferences but judgements about a single fact, each a little more likely right than wrong. Then a simple majority of many of them is almost certainly right — 6,763 voters who are each right 51% of the time make a majority right 95% of the time. The theorem survives voters worse than a coin, if the average is better. It does not survive voters who share their mistakes, and when their skills differ the right rule weighs votes rather than counting them.

applied · Voting rules Evidence as steps on a scale of decibans. A horizontal decibel scale of odds with a starting dot at the prior and one arrow per piece of evidence, ending at a posterior probability of 44.2%.

Evidence measured in decibans

Write a probability as odds and take the logarithm, and every piece of evidence becomes a length. A positive result on a good test is thirteen decibans; a negative one is minus twenty. Lay the lengths end to end from the prior and the posterior is where they stop, in any order. The rule fails in exactly one way — when two pieces of evidence share a cause — and Turing built a code-breaking method on the arithmetic.

probability · Bayes Switching envelopes when the largest amount is 64. A bar chart of the probability-weighted gain from switching at each amount that might be seen, 1 to 64: small positive bars and one large negative bar, adding to zero.

The envelope that always looks better

Two envelopes, one holding twice as much as the other. Open one, see an amount, and reason that the other holds double or half with equal chance — so switching gains a quarter on average. By symmetry the same argument says switch back. The step that fails is not the arithmetic; it is the claim that double and half are equally likely whatever amount is seen, which no honest prior allows — and there is one prior under which the other envelope really does look better at every amount.

probability · Bayes A fair coin judged by someone who thinks it is biased. Posterior probabilities of three wrong coin biases over two thousand tosses of a fair coin, for five runs; the bias 0.6 wins every run.

Certain of a coin that is not there

Bayes' theorem is exact, and it is only as good as the list of hypotheses it is given. Hand it a list that leaves out the truth and it does not hesitate: it becomes certain of the entry that is least wrong, in a precise sense — the one closest in Kullback–Leibler divergence — and if two entries are equally wrong it never settles at all. Hand it a model that assumes independence where there is none, and its intervals shrink as fast as they would for honest data while covering the truth less and less often.

probability · Bayes A fixed p-value favours the null more as the sample grows. Even odds at p=0.05 near n=76, at p=0.01 near n=1202, at p=0.001 near n=83176; p=0.05 at n=10⁶ gives 116.9 to one for the fair coin; PEAR data z=3.614, p=0.00030, BF₀₁=11.90.

A result at five per cent that favours the null

Toss a coin a million times and find a count of heads exactly at the edge of significance, p = 0.05. A test at the usual level rejects fairness. A Bayes factor comparing a fair coin with a coin of unknown bias says the opposite, by 117 to one. Both computations are right, and the sample size that separates them is about seventy-six tosses.

probability · Bayes

Named alongside it

The objects these essays reach for when they reach for this one.

Conditional probabilityIndependenceAreaLikelihoodCounting argumentLaw of large numbersRandom walkSample spaceExpectationModel misspecificationPosteriorBase rate

All concepts