Matpic
mathematics, in pictures
Concept

Stationary distribution

A distribution over a Markov chain's states that the chain leaves unchanged: as much flows into each state as out. On a finite chain that can reach everywhere it exists, is unique, and is what a single long run's time averages settle on.

Named by 4 essays across one field — each of them below, with the objects they name alongside it.

A walk that visits each state as often as its weight says. The target distribution over 12 states with the share of a 40,000-step Metropolis run beside each bar, above the first 300 steps of the walk itself; the two distributions differ by 0.5 per cent in total.

A walk that samples a distribution

When a distribution can be evaluated but not drawn from, a wandering point can be arranged to visit each state as often as its weight says. The rule needs no normalising constant, compares two weights and steps or stays.

probability · Monte Carlo A walk on a weighted graph, and a cycle whose traffic goes one way. A weighted graph with the long-run share of each state read off its total weight, beside a three-state cycle whose shares are equal and whose traffic circulates.

The chain that runs the same backwards

Put weights on the edges of a graph, step to a neighbour in proportion to them, and the long-run share of a state is its own weight over the total — read straight off the picture, with nothing to solve. The condition that makes that work is strictly stronger than being stationary.

probability · Markov chains The time a single walk spends in each state, against the share it should hold. Paired bars for each state, one the fraction of a long run's time spent there and one the computed stationary share, above a table of expected return times.

The time spent and the share held

The first rung's shares were a limit of distributions — where the walk probably is after many steps. This one is about a single walk: the fraction of its time spent in each state is that state's share, and the expected wait between visits is exactly the reciprocal.

probability · Markov chains One walk on the whole numbers, three chances, three different fates. The relative weight of each state for three step-up chances, drawn as bars, with the running total of those weights and what each case means beneath.

Where the shares have nowhere to go

On finitely many states, a chain that can reach everywhere and is not forced into a rhythm settles down. Give it infinitely many and both conditions can hold while the walk leaves and never returns — or returns with certainty and takes an unbounded average time about it.

probability · Markov chains

Named alongside it

The objects these essays reach for when they reach for this one.

Markov chainProbabilityRandom walkDetailed balanceExpectationInvariantLimitEigenvectorErgodicityGeometric seriesGraphMetropolis algorithm

All concepts