A difference too small to draw

Take two numbers that differ in the fourth decimal place. Apply the same rule to both, over and over. Nothing random happens; nothing is measured; the same arithmetic is done to each.

For the first fifteen steps the two lines are one line. By step thirty they are visibly apart. By step forty they are unrelated: the position of one says nothing about the position of the other, beyond the fact that both are between zero and one.

That is sensitive dependence, and it is the property that makes a completely determined system unpredictable.

The two words that do the work #

Two claims are easy to confuse and it is worth separating them at once.

Deterministic means the future is a function of the present. Give the rule the same number and it returns the same answer, every time, forever. Both orbits above are deterministic, and running either of them again reproduces it exactly.

Predictable means that an approximate present determines the future approximately. That is a different claim, and it is the one that fails. The two orbits above start approximately equal and end up not approximately equal, so an approximate present is worthless.

Everything a person actually knows about a physical system is approximate. So a system can be perfectly determined and completely unforecastable, and there is no contradiction — the two words are answers to different questions.

Where the separation comes from #

The mechanism is the same stretch-and-fold that produced the whole cascade, and it can be watched in one step.

Near a point where the map’s slope is $m$, a small gap $d$ becomes a gap of about $|m|d$. If $|m|$ is bigger than one, the gap grows. The logistic map at $3.9$ has $|f'|$ bigger than one over most of the interval, so most steps stretch, and stretching compounds.

Growth by a constant factor each step is exponential, which is why the two lines in the first figure stay together for a while and then part suddenly. A gap of $10^{-4}$ multiplied by $2$ each step reaches $0.1$ after about ten steps and reaches the size of the whole interval after fourteen. The separation looks abrupt because exponential growth always does.

Half a lifetime per decimal place #

The practical consequence is a piece of arithmetic worth doing once.

If gaps grow by a factor $e^{\lambda}$ per step, then a gap of $\varepsilon$ becomes order one after about $\frac{1}{\lambda}\ln(1/\varepsilon)$ steps. The logarithm is the important part: improving the measurement by a factor of ten buys a fixed additional number of steps, not a proportional one.

For the logistic map at $r = 4$, where $\lambda = \ln 2$, each extra decimal place of initial precision buys about $3.3$ more steps of usable prediction. Measuring a thousand times more accurately buys ten steps.

That is the whole of the forecasting problem, and it is why the answer to “why can nobody forecast the weather three weeks out” is not “the models are bad”. A perfect model with a measurement error of one part in a million, in a system with a doubling time of a day and a half, is useless at three weeks, and a measurement error of one part in a billion moves that to four and a half.

Where it does not happen #

Sensitivity is not a property of complicated systems. It is a property of some systems, and the same map without it is a useful control.

That figure is the argument that the sensitivity belongs to the parameter and not to the experiment. Same map, same starting points, same separation, same number of steps — and the opposite outcome, because at $2.9$ the slope at the fixed point is under one and steps contract instead of stretching.

Most systems most of the time are like this. A pendulum with friction, a ball in a bowl, a hot object cooling: perturb them and the perturbation dies. Sensitivity is the exception, and the reason it matters is that the exceptions include the weather, the solar system over long enough times, and a great many things people would like to forecast.

Sensitivity is not enough on its own #

A rule that merely separates nearby points is not chaotic, and the missing ingredient is what makes the subject non-trivial.

The map $x \mapsto 2x$ on the whole line separates every pair of points exponentially and is completely predictable: everything runs off to infinity, and a rough starting position gives a rough final one, because “roughly” scales with everything else. Nothing interesting happens because nothing comes back.

What is needed alongside stretching is mixing: the orbit must return, and points that were far apart must be brought back together. That is the folding half of stretch-and-fold, and it is why every map in this field sends its interval into itself rather than out of it.

The standard definition of chaos — Devaney’s — asks for three things: sensitive dependence, a dense orbit, and periodic points that are dense. The second is the mixing, the third is the structure the last essay counted, and the first is this one. It later turned out that the first follows from the other two, which is a tidy result about a definition that was assembled by hand.

What the two orbits have in common #

After they separate, the two orbits are unrelated as sequences. They are not unrelated as sets.

Both wander over the same interval. Both visit any given subinterval about the same proportion of the time — a statement about a limiting distribution rather than about any particular step, of the kind the bell curve makes for sums of coin flips. Both have the same long-run average, the same distribution of values, and the same statistical description in every respect anybody has measured. What has been lost is the correspondence between step numbers, not the behaviour.

That is the standing consolation of this subject and the basis of everything useful done with it. A weather forecast for next Tuesday is hopeless; a statement about the distribution of next August’s temperatures is not, and it is not hopeless because the underlying system, while unpredictable in detail, is extremely well behaved in aggregate.

Two ways to measure a difference #

There is a subtlety in the first figure that is easy to read past: the two orbits separate, and then they stop separating.

They have to. Both are confined to the unit interval, so the gap between them cannot exceed one. Exponential growth applies to small gaps only, and once the difference is of order the size of the space the growth stops and the gap starts wandering — sometimes large, occasionally small again by coincidence.

That is why sensitivity has to be defined as a statement about arbitrarily small perturbations rather than about the eventual gap. The correct statement is that there is some fixed distance such that, however close two starting points are, their orbits eventually get at least that far apart. Nothing is claimed about what happens afterwards, and nothing needs to be.

The practical version of the same point: the useful quantity is the number of steps before the error reaches a tolerance, not the error itself. A forecast horizon is a time, and this whole field’s contribution to forecasting is a way of computing one.

The picture is not of the orbit it names #

There is an uncomfortable fact about every chaotic figure on this site and it is worth stating in the essay that is about it.

A computed orbit is wrong. Arithmetic is done to sixteen decimal places, so every step introduces an error of about $10^{-16}$, and errors grow by the same exponential factor as anything else. After fifty steps of a map that doubles gaps, an error in the sixteenth place has grown to the size of the interval. The orbit in the first figure is not the orbit of $0.4$; it is the orbit of $0.4$ plus rounding, which is a different number.

What rescues the pictures is a theorem rather than an argument from care. The shadowing lemma says that for maps of this kind, a computed orbit — wrong at every step — stays uniformly close to the true orbit of some nearby starting point. So the figure is an honest picture of the system’s behaviour and a dishonest picture of the number in its caption.

Which of those a reader needs depends on the question. For “what does this map do”, the figure is exactly right. For “where is $0.4$ after fifty steps”, nothing on this page answers it and nor does any computer.

The century-long delay #

The reason this was not noticed until the 1960s is worth a paragraph, because it is not that nobody looked.

Poincaré looked. In 1890, working on whether the solar system is stable, he found that the three-body problem has orbits whose behaviour cannot be described by any series expansion, and wrote — in a passage that reads as though it were written seventy years later — that a small difference in initial conditions can produce a very large one in the final phenomenon, and that prediction becomes impossible.

Then almost nothing happened for seventy years. The mathematics was correct, it was published, and it went nowhere, because the objects it described could not be seen. The only access to a chaotic orbit is to compute a few thousand steps of it, and until there was a machine that would do that, the subject consisted of theorems about pictures nobody had.

Lorenz found it again in 1961 by accident: restarting a weather simulation from a printout that carried three decimal places where the machine held six, and watching the run diverge from the original. What made it a discovery rather than a bug report was that he had the earlier run to compare against — the accident was a controlled experiment, and he recognised it.

That is a case worth keeping beside the four-colour theorem and the Feigenbaum constant: three results in this collection that existed only once machines could produce the evidence, and all three initially distrusted for that reason.

Two things a forecaster can still do #

Given all that, it is worth being precise about what remains possible, because “chaos” is often taken to mean “give up” and it does not.

Run the forecast many times. If a single starting point is untrustworthy, start from a cloud of them — a few dozen states, all consistent with the measurements, spread through the error bars. Run each. Where the resulting orbits agree, the forecast is good; where they scatter, it is not. That converts an unanswerable question about a number into an answerable one about a spread, and it is what every operational weather centre now does.

The picture of the cloud spreading is exactly the first figure with more than two lines in it, and the useful output is not the average of the lines but the time at which they stop agreeing.

Looked at without its twin, an orbit gives no sign of the sensitivity at all. That is the practical difficulty with the property: it is invisible in any single run, and detecting it requires either two runs or a derivative.

Ask a different question. The quantity that is unpredictable is the state at a named future time. Many things people actually want are not that: whether the orbit stays in a region, how often it visits one, what its average is. Those are questions about the invariant distribution and are answerable to arbitrary accuracy by running one long orbit — the wrong orbit, as established above, and it does not matter, since every orbit has the same statistics.

The habit generalises past this subject. When a question is unanswerable, the useful move is often to find the nearby question that is not: Buffon’s needle answers a geometry question by throwing sticks, and one point’s worth of information gets a number out of a system by asking what a measurement reduces rather than what it reveals.

What it is not #

Three things get attached to this idea that do not belong to it, and each is worth removing.

It is not randomness. A random sequence and a chaotic orbit look identical over any finite window, and they are entirely different objects: the chaotic one is reproducible from its starting point and has a rule that fits on a line. The resemblance is a statement about what pictures can show, not about the systems.

It is not complexity. The map here has one parameter and one multiplication. Simple rules producing behaviour nobody can summarise is the pattern this whole field runs on, and it appears elsewhere in this collection too — an eight-line rule building a fractal, three shuffles generating a group nobody can hold in their head. Sensitivity requires stretching, and stretching is the simplest thing a rule can do.

It is not the butterfly effect as usually told. The popular version — a butterfly’s wingbeat causing a hurricane — implies a chain of causation from a small event to a large one, which is not the claim. The claim is that two whole system states differing by a butterfly’s worth of air diverge into different weather. Nothing is caused by the butterfly; the butterfly is a label for the size of the difference.

Lorenz’s own title was more careful than its reputation: Does the flap of a butterfly’s wings in Brazil set off a tornado in Texas? — a question, asked in 1972, whose intended answer was that the question is not well posed.

Where the ladder goes next #

The obvious follow-up is quantitative. The orbits separate; how fast, exactly, and can the rate be attached to a number that describes the map rather than the experiment? That is the Lyapunov exponent, and measuring it turns “sensitive” from an adjective into a quantity with a value.

The other direction is the one this collection has already been down. A random walk is unpredictable for a completely different reason — the randomness is in the rule rather than in the measurement — and yet its long-run statistics are as well behaved as a chaotic orbit’s. Two unpredictable systems with different sources of unpredictability and the same kind of consolation is worth noticing, because it suggests the consolation is the more general fact.