Generator

Bayes' theorem as two rectangles

A generator in the probability library, called 59 times across 10 essays. Below: what it draws with nothing chosen and at each mode an essay asks for, what it checks while drawing, and everywhere it is used.

bayes is one function. Everything below came out of it during this build, at parameters taken from the essays rather than invented for this page — so a figure here is the same figure a reader meets in an essay, and if the generator changes, this page changes with it.

With nothing chosen

Bayes' theorem as two rectangles. A unit square split by how common the condition is (1.0%) and then by how the test behaves. Of everyone who tests positive, the fraction who have it is 16.7%.

A fixed p-value favours the null more as the sample grows

A fixed p-value favours the null more as the sample grows. Even odds at p=0.05 near n=76, at p=0.01 near n=1202, at p=0.001 near n=83176; p=0.05 at n=10⁶ gives 116.9 to one for the fair coin; PEAR data z=3.614, p=0.00030, BF₀₁=11.90.

The same peak over a narrower strip

The same peak over a narrower strip. n=100: peak 6.912, BF₁₀ 0.8433; n=10000: peak 6.827, BF₁₀ 0.0855.

The most a p-value can say against the null

The most a p-value can say against the null. At p=0.05: best single bias 6.826 to one; calibration 2.456; uniform n=100 0.843; uniform n=10⁴ 0.086.

The bar for evidence rises with the sample

The bar for evidence rises with the sample. n=100: z 2.045 for even odds, 2.955 for ten to one; n=10000: z 2.959 for even odds, 3.655 for ten to one; n=1000000: z 3.656 for even odds, 4.239 for ten to one; n=100000000: z 4.239 for even odds, 4.751 for ten to one.

Tuning the alternative buys at most two to one

Tuning the alternative buys at most two to one. n=100: minimum BF₀₁ 0.4685 at spread 0.08318; n=10000: minimum BF₀₁ 0.4734 at spread 0.00851; n=1000000: minimum BF₀₁ 0.4734 at spread 0.00085; normal-theory bound 0.4734.

What it checks while it draws

Collected by running the family and recording what it asserted, not written here. The count is how many separate times the claim was put to the test while these drawings were made.

Where it is called

Every figure on this list is drawn by the same rule, so a change to the rule changes all of them at once. That is why the list is published.

Probability

A result at five per cent that favours the null

Toss a coin a million times and find a count of heads exactly at the edge of significance, p = 0.05. A test at the usual level rejects fairness. A Bayes factor comparing a fair coin with a coin of unknown bias says the opposite, by 117 to one. Both computations are right, and the sample size that separates them is about seventy-six tosses.

Probability

Bayes' theorem is a picture of a square

A test that is 99% accurate returns a positive result. The chance it is right can easily be under one in five, and the reason is visible the moment the population is drawn as a square rather than described as a formula.

Probability

Certain of a coin that is not there

Bayes' theorem is exact, and it is only as good as the list of hypotheses it is given. Hand it a list that leaves out the truth and it does not hesitate: it becomes certain of the entry that is least wrong, in a precise sense — the one closest in Kullback–Leibler divergence — and if two entries are equally wrong it never settles at all. Hand it a model that assumes independence where there is none, and its intervals shrink as fast as they would for honest data while covering the truth less and less often.

Probability

Evidence measured in decibans

Write a probability as odds and take the logarithm, and every piece of evidence becomes a length. A positive result on a good test is thirteen decibans; a negative one is minus twenty. Lay the lengths end to end from the prior and the posterior is where they stop, in any order. The rule fails in exactly one way — when two pieces of evidence share a cause — and Turing built a code-breaking method on the arithmetic.

Probability

Evidence that can be multiplied

A p-value from one study and a p-value from the next cannot simply be multiplied, and a researcher who decides whether to run the next study by looking at the last one quietly breaks the rule that combines them. An e-value — a number whose average under the null is at most one — can be multiplied, study after study, under any rule for continuing, and never passes twenty to one more than one time in twenty. What it costs is tosses, and what it demands is a decision about which alternatives to be good at finding.

Probability

One coin, counted by runs and by wakings

Beauty is put to sleep and a fair coin is tossed. Heads, she is woken once; tails, twice, with the first waking erased from her memory. Each time she wakes she is asked how likely heads is. One half, say some; one third, say others; and unlike every earlier puzzle of this kind, stating the protocol exactly does not end the argument.

Probability

Peeking until the answer is yes

Toss a fair coin, test for bias after every toss, and stop the moment the result is significant at 5%. Within a thousand tosses nearly half of fair coins have been declared biased, and with no limit on the tosses every one of them is. Watch a Bayes factor the same way, stopping at twenty to one, and no amount of watching gets more than one fair coin in twenty past the line.

Probability

The door that was not opened

Three doors, one prize, a host who opens a losing door and offers a swap. Switching wins two times in three, and the reason is not about doors — it is about what the host was allowed to do.

Probability

The envelope that always looks better

Two envelopes, one holding twice as much as the other. Open one, see an amount, and reason that the other holds double or half with equal chance — so switching gains a quarter on average. By symmetry the same argument says switch back. The step that fails is not the arithmetic; it is the claim that double and half are equally likely whatever amount is seen, which no honest prior allows — and there is one prior under which the other envelope really does look better at every amount.

Probability

Two children and the sentence about one of them

A family has two children and at least one is a boy. The chance that both are boys is one in three — or one in two, or anything from one in three to certainty — and every one of those answers is right for some way the sentence could have come to be said. There is no host and no door, and the protocol is still the whole problem.

The whole library · What the figures prove