Bayes factor
Named by 3 essays across one field — each of them below, with the objects they name alongside it.
Also named here as p-value — the same set of essays touches all of them, so they are one junction rather than several.
A result at five per cent that favours the null
Toss a coin a million times and find a count of heads exactly at the edge of significance, p = 0.05. A test at the usual level rejects fairness. A Bayes factor comparing a fair coin with a coin of unknown bias says the opposite, by 117 to one. Both computations are right, and the sample size that separates them is about seventy-six tosses.
Peeking until the answer is yes
Toss a fair coin, test for bias after every toss, and stop the moment the result is significant at 5%. Within a thousand tosses nearly half of fair coins have been declared biased, and with no limit on the tosses every one of them is. Watch a Bayes factor the same way, stopping at twenty to one, and no amount of watching gets more than one fair coin in twenty past the line.
Evidence that can be multiplied
A p-value from one study and a p-value from the next cannot simply be multiplied, and a researcher who decides whether to run the next study by looking at the last one quietly breaks the rule that combines them. An e-value — a number whose average under the null is at most one — can be multiplied, study after study, under any rule for continuing, and never passes twenty to one more than one time in twenty. What it costs is tosses, and what it demands is a decision about which alternatives to be good at finding.
Named alongside it
The objects these essays reach for when they reach for this one.
LikelihoodP-valueCentral limit theoremMartingaleStopping timeBayes' theoremBeta distributionEntropyModel misspecificationPosteriorRandom walk