A mixture that is a population

A mixed equilibrium has always been an awkward object. Between two choosers it says each randomises in exactly the proportion that leaves the other indifferent — and indifferent is the whole trouble, because a chooser indifferent between two options has no reason to mix in that proportion rather than any other. The equilibrium is consistent, and nothing holds anyone to it.

There is a second reading of the same arithmetic in which the trouble disappears. Replace the two choosers by a large population, each member playing one pure strategy. The mixture is now the population’s composition — the share playing each strategy — and nobody chooses it. What moves it is success: strategies earning more than the population’s average grow in share, and strategies earning less shrink.

In this game the mixed equilibrium is not a knife-edge at all. It is where every population ends up.

Shares that grow with their payoff #

The game on the page is the classic hawk–dove contest. Two members meet over a prize worth 4. An aggressive member — a hawk — against a peaceful one — a dove — takes the whole prize; two doves share it, 2 each; two hawks fight, one wins the prize and one pays an injury of 6, which averages to $-1$ each.

Let $x$ be the share of hawks. A hawk meeting a random member earns $-x + 4(1-x) = 4 - 5x$; a dove earns $2(1-x) = 2 - 2x$. The difference is $2 - 3x$, positive when hawks are rare and negative when they are common, and nought at $x = 2/3$.

The replicator dynamic turns that difference into motion:

$$\frac{dx}{dt} = x(1-x)\,\bigl(\text{hawk's payoff} - \text{dove's payoff}\bigr).$$

The factor $x(1-x)$ is there because a strategy nobody plays cannot grow and neither can the other one when everybody plays the first; the bracket is the reason to grow at all. Nothing about the population is rational, forward-looking or even aware of the game. Members who do better leave more members like themselves, and the rule is nothing more than that.

The rest points are exactly the equilibria plus the edges. The rate is nought where the bracket is nought — the mixed equilibrium, where both strategies earn the same — and at the two ends, where one strategy is extinct. That is the same set the definition of equilibrium produces, with two extra points the dynamic cannot leave because there is nobody left to change.

The line that decides #

Everything about the population’s fate is in one straight line: the payoff difference, as a function of the share. The rest point sits where the line crosses nought, and what happens nearby depends on which way it crosses.

In hawk–dove the line slopes down. Just below $2/3$ hawks do better, so their share rises towards $2/3$; just above, doves do better, so it falls back. A line crossing nought from above makes the crossing an attractor, and the left panel of the figure shows the growth rate falling through nought there with the arrows on the axis pointing inwards from both sides.

The stag hunt of the essay on two equilibria crosses the other way.

Here the payoff difference is $4x - 3$, rising through nought at $3/4$. Below it hare-hunters do better and the stag share collapses to nought; above it stag-hunters do better and it climbs to one. The mixed equilibrium is a point that pushes, not one that pulls: it is the watershed between two basins, and its location — three-quarters of the way to all-stag — is the same number the two-chooser analysis read off an axis as the boundary between beliefs. The population reading confirms the conclusion and changes its meaning. The worse equilibrium’s basin is three times the better one’s, and a population is not choosing anything; it simply falls into whichever basin it starts in.

Stable against invaders #

Biologists came at the same question from a different direction, and their answer turns out to be identical in this setting and different in general.

Maynard Smith and Price asked, in 1973, what makes a behaviour persist in a population once it is established. Their answer was a test against invaders: a small group of mutants playing some other strategy appears. The resident strategy is evolutionarily stable if, whatever the mutants play, the residents do better — either strictly better against the resident population, or equally well against it and strictly better against the mutants.

For a mixture $x^*$ in a two-strategy game the test reduces to one inequality: $u(x^*, y) > u(y, y)$ for every other mixture $y$, where $u(p, q)$ is what $p$ earns against $q$. The resident mixture must do better against any invader than the invader does against itself.

Hawk–dove’s $2/3$ passes. The stag hunt’s $3/4$ fails: a population slightly richer in stag-hunters does better against itself than $3/4$ does against it. And in a two-strategy game the invader test and the attractor test are the same test — both reduce to the sign of the slope of that one line. The figures compute both, in exact fractions, for every invading mixture on a lattice, and require them to agree.

That agreement is a theorem, and it is the reason the two literatures could merge. An evolutionarily stable strategy is always an attractor of the replicator dynamic. In two dimensions the converse holds too; in more it does not, and the gap between them is where the next figure lives.

Rock, paper and scissors, which circle for ever #

Three strategies, each beating the next and losing to the one before. The only equilibrium is a third each, and it is perfectly good as an equilibrium: against that mixture every pure strategy earns the same, so nobody can gain by deviating.

A population rich in rock is invaded by paper, which then feeds scissors, which then feeds rock, and the composition goes round. It does not spiral in and it does not spiral out: the product of the three shares stays exactly constant along every orbit, so each population is confined to one level curve of that product for ever. The centre, where the product is largest, is reached by no population that did not start there.

The reason is short. The rate at which the logarithm of the product changes is the sum of the three strategies’ advantages over the average, and when a win and a loss are worth the same, the three advantages sum to nought at every composition. Nothing is gained or lost around the cycle, so nothing drifts.

And the invader test explains why the equilibrium does not attract: a third each is not evolutionarily stable. Against it, every invading mixture earns exactly what the residents do and does exactly as well against itself — so the invaders are never driven out, merely never favoured. An equilibrium can be one no deviator profits from and one no population converges to, and rock–paper–scissors is the standard demonstration that those are separate properties.

Tilt the payoffs and the orbits open #

The conserved product is a knife-edge of its own, and moving off it in either direction produces the two other things a rest point can do.

When wins outweigh losses, the sum of the three advantages is positive away from the centre, the product grows, and every population spirals inwards. The centre is now evolutionarily stable: invaders do strictly worse against themselves than the balanced mixture does against them, because meeting their own kind wastes the larger payoff for winning.

When losses outweigh wins the product shrinks, the orbits open outwards, and populations end up running round near the edges — nearly all rock, then nearly all paper, then nearly all scissors, with the swings becoming longer and more extreme. The centre is still the game’s only equilibrium, still a point no deviator profits from, and a point every population flees.

Three games with the same equilibrium in the same place, and three different fates for every population near it. What decides between them is not the location of the equilibrium, which the definition fixes, but the sign of one quantity the definition never computes.

Two extremes: a quantity climbed, and a quantity kept #

The conserved product and the populations that settle are the two ends of a single classification, and the classification is by the shape of the payoff table.

When the table is symmetric — when a member playing $i$ against $j$ earns exactly what a member playing $j$ against $i$ earns, so that every meeting is a partnership — the population’s average payoff rises along every trajectory, and it rises at a rate equal to the variance of the payoffs across the members present. That is Fisher’s fundamental theorem of natural selection, in the form the replicator dynamic gives it, and it makes the average payoff a quantity the population climbs. Every trajectory ends at a peak of it, exactly as every sequence of improving moves on a congestion game’s landscape ended at a valley of Rosenthal’s potential. A game of pure partnership cannot cycle, for the same reason water cannot flow round a closed loop downhill.

When the table is antisymmetric — when one member’s gain in each meeting is the other’s loss, which is rock–paper–scissors with a win worth a loss — the average payoff is nought at every composition and there is nothing to climb. What the dynamic keeps instead is a weighted sum of the logarithms of the shares, with the equilibrium’s own shares as the weights. For rock–paper–scissors those weights are a third each, and a third of the logarithm of the product is the product in disguise: the conserved quantity drawn in the triangle is this one.

Most games are neither, and the tilted versions of rock–paper–scissors show what happens between: part of the table is a partnership and part is a contest, and whichever part dominates near the equilibrium decides whether the orbits close in or open out. A symmetric table gives a potential and an antisymmetric one gives a conservation law, and those are the only two ways a dynamic can be understood without solving it.

What the definition of equilibrium cannot see #

Every argument about equilibrium so far has been built on one definition: a state from which no single participant gains by changing alone. The population reading shows exactly what that definition leaves out.

It is a statement about the state and not about its neighbours. It says nothing about what happens a little way off, and “a little way off” is where every real population lives, because compositions are never exact and mutations never stop. Equilibrium is necessary for a population to rest and not sufficient for it to arrive. Every stable composition is an equilibrium — a composition where some strategy did strictly better would change — but the rock–paper–scissors centre and the stag hunt’s watershed are equilibria that no population ever reaches.

That is the population counterpart of two equilibria and no way to choose. There, the definition could not say which of two equilibria happens; here, it cannot say whether an equilibrium happens at all. Both gaps are closed by the same move: specify a process and ask where it goes. Existence is what the fixed-point argument buys, and existence is all it buys.

The process is an assumption #

The replicator dynamic is one process among several, and the conclusions above belong to it rather than to the game.

It assumes reproduction in proportion to payoff. That fits organisms whose payoff is offspring, and it fits imitation, where members copy more successful members with a probability that grows with the difference. It fits less well where members choose best replies to the current composition, and the best-reply dynamic gives different pictures: in the rock–paper–scissors game with balanced payoffs, for instance, best-reply dynamics converge to the centre in a way the replicator orbits never do.

It assumes an infinite, well-mixed population. Finite populations drift by chance, and on the conserved orbits above, any randomness at all makes the product wander until some strategy goes extinct. A population on a spatial grid, where members meet their neighbours rather than a random partner, can keep all three strategies alive indefinitely — which is what was found for strains of bacteria that produce a toxin, resist it, or are sensitive to it: grown on a plate, where neighbours interact, all three persisted, and mixed together in a flask, one was lost.

And it assumes the strategies are fixed. New strategies do not appear. Evolutionary stability is the one concept that admits them, but only as small groups of a strategy already in the list of possibilities, and it says nothing about what happens when the invasion is large.

Lizards on three strategies #

Rock, paper and scissors is not a toy. Sinervo and Lively reported in 1996 that males of the side-blotched lizard in a population in central California come in three throat colours with three mating strategies: orange males hold large territories and many females, blue males hold small territories and guard their mates closely, and yellow males hold no territory and mimic females to sneak matings.

Orange beats blue by taking territory; yellow beats orange by sneaking into its large, poorly guarded harem; blue beats yellow by guarding closely. In the population studied, the commonest colour changed over the years in the order the cycle predicts, returning to its starting point over a few breeding seasons.

The cycle is exactly the one the triangle draws. An abstract claim about a three-strategy game with no evolutionarily stable strategy turned out to describe the mating system of a lizard, measured in the field, twenty-three years after the concept was written down for quite different reasons.

Where the account needs care #

The attractor is not the stable strategy in more than two dimensions. Every evolutionarily stable strategy attracts under the replicator dynamic, but there are games with more strategies where a rest point attracts without passing the invader test. The two coincide on the page because every figure here has two strategies or the cyclic symmetry of three.

Conservation is exact only on the balanced game. A win worth $1.001$ and a loss worth $1$ turn the closed loops into spirals that take a very long time to reveal themselves. On any finite run the orbits of a nearly balanced game look closed, and the distinction is invisible without the product being tracked.

The trajectories are computed and the stability is not. Every orbit drawn is a numerical integration, checked for staying on the triangle and for the drift of the product. Whether the centre attracts is decided by exact arithmetic on the payoffs, and the orbits are drawn to agree with that decision, not to make it.

And a population is not a pair of choosers. The mixed equilibrium of the two-chooser game and the rest point of the population have the same coordinates and describe different situations, and the stability results transfer only to the population. A pair of rational choosers does not become more likely to play $2/3$ because a population would drift there.

Maynard Smith, Price, and a letter to Nature #

George Price, a chemist turned self-taught geneticist, wrote a long paper in the late 1960s on why animal contests so often stop short of serious injury, and sent it to Nature. John Maynard Smith was one of its referees. The paper was far too long to be published as it stood, but the idea in it stayed with him, and the two eventually wrote it up together: The Logic of Animal Conflict appeared in 1973 and introduced the evolutionarily stable strategy.

The dynamic came five years later. Taylor and Jonker wrote down the replicator equation in 1978 and showed that evolutionarily stable strategies are attractors of it, joining the invader test to a process. The rock–paper–scissors conservation law is older than either, being a special case of the invariants of predator–prey equations that Lotka and Volterra had found in the 1920s.

The idea arrived as an explanation of restraint — why a stag with lethal antlers fights by pushing — and the arithmetic is the hawk–dove figure at the top of this page: an aggressive strategy that cannot take over, because its own success fills the population with members it must fight.

What an orbit cannot show #

A trajectory drawn for thirty units of time is thirty units of time, and the claims are about all time. The conserved product is checked to six figures at the end of each run; that it is exactly conserved is the one-line argument, not the drawing.

The triangle is also silent about noise. A closed orbit is closed only in a population so large that chance plays no part, and any real population on one of those loops is doing a random walk between them. No figure here shows that drift, and it is the thing that decides the fate of real rock–paper–scissors populations.

And the flow along a single line shows a population of two strategies. With three or more strategies the space of compositions is a triangle or a higher simplex, the paths can wind, and the one-line test for stability becomes a matrix condition that no picture on this page draws.

Still open: when the strategies can change #

The replicator dynamic moves shares among strategies that already exist. Adaptive dynamics lets the strategies themselves drift in small steps, and asks whether a population evolving that way reaches an evolutionarily stable strategy at all — sometimes it reaches a point where it splits into two diverging types instead, which is one model of how a species becomes two. The folk theorems for repeated play, where almost every outcome becomes an equilibrium, are where the invader test is most needed and where it most often fails to single anything out. And what moving first is worth is the question of what happens when the other kind of asymmetry — not between strategies, but between the times the choosers act — is allowed.

A property of the neighbourhood, not the point #

The move worth keeping is to ask what a condition says about nearby states rather than about the state itself.

A definition of rest — no gain from deviating, no net force, no change in the payoff difference — is a statement about a single point, and it is satisfied equally by points that attract, points that repel and points that are circled. Which of those a point is depends on how the condition changes as the state moves, which is a derivative, or a slope, or a sign the definition does not mention.

Before trusting that a system will be found at its equilibrium, ask which way the imbalance points just off it. Every attracting equilibrium in this essay had an imbalance pointing back towards it, and every equilibrium that failed to attract had one pointing away or pointing round.