A closer start buys only time

Sensitive dependence on initial conditions is usually stated as a slogan and demonstrated with a picture of two curves that were together and then are not. That is a true statement and it is not a measurement, and everything interesting about the phenomenon is in the measurement.

The question to ask is: how fast. If two trajectories a millionth apart take a hundred time units to separate visibly, and two a billionth apart take a hundred and one, then the system is chaotic in a strong sense and improving instruments buys almost nothing. If the second pair takes a thousand, the phenomenon is real and manageable. The two possibilities are distinguished by a single number.

The measurement #

Take two starting points a distance $\delta_0$ apart on the attractor and follow both. If the separation behaves like

$$\delta(t) \approx \delta_0 e^{\lambda t},$$

then plotting $\log \delta$ against $t$ gives a straight line of slope $\lambda$, and the whole content of sensitive dependence is in the sign and size of that number.

For the Lorenz system at its standard parameters $\lambda$ is about nine tenths, in the units of the equations. That is the largest Lyapunov exponent, and three things about it are worth separating.

The line is straight. Not “roughly exponential” — the growth is exponential over several orders of magnitude, and the logarithm of the gap is linear in time to the accuracy of the fit. That is a stronger statement than “the trajectories diverge”.

The slope does not depend on $\delta_0$. Three pairs started at three separations give three parallel lines. Starting closer moves the line down and does not tilt it, which is the sentence in this essay’s title stated as a picture.

And the line stops. The separation cannot exceed the size of the attractor, so once the gap is comparable with the object the growth saturates and the trajectories become independent. Everything the exponent predicts happens before that, and how long “before that” lasts is precisely what the next section computes.

It is worth noticing what would falsify the claim. If the three lines had different slopes, the exponent would be a property of the experiment rather than of the system, and the whole quantity would be meaningless. If the lines were curved, the growth would not be exponential and the single number would be the wrong summary. The figure asserts the first and the fit reports the second, so the two ways the picture could have been uninformative are both checked.

What it costs to gain a decimal place #

The arithmetic that follows is the practically important part.

Suppose the initial uncertainty is $\delta_0$ and predictions are useful until the error reaches some tolerance $\Delta$. The time available is

$$T = \frac{1}{\lambda}\ln\frac{\Delta}{\delta_0}.$$

Improving the measurement by a factor of ten reduces $\delta_0$ tenfold and increases $T$ by $\ln 10 / \lambda$, which for the Lorenz system is about two and a half time units — a fixed amount, the same for every factor of ten, however many have already been bought.

That is the shape of the whole difficulty. Going from three digits of initial accuracy to six does not double the forecast horizon; it adds three fixed steps. Going to twelve adds six. Reaching twice the horizon needs squaring the accuracy, which for a physical measurement is not a matter of effort but of what is possible.

Put the other way round, the horizon grows like the logarithm of the precision, and a logarithm is the slowest growing function anybody uses. An instrument a million times better than today’s would extend a Lorenz forecast by about fifteen time units — from, say, twenty to thirty-five — and would cost more than every instrument ever built. That is the practical content of chaos, and it is an arithmetic statement rather than a philosophical one.

That figure is the arithmetic of the previous section drawn rather than computed. The vertical spacing of the three lines at $t = 0$ is the difference in initial precision; the horizontal spacing between them at any level is the extra time that precision buys. The lines being parallel is what makes the second quantity a constant, and if they converged or diverged the whole calculation would change.

Why the rate is a property of the system #

The independence of $\delta_0$ is not obvious and has a clean explanation.

A small separation vector evolves, to first order, by the derivative of the flow — the same linear map whatever the vector’s length, because a derivative is linear. So the growth factor over a short time depends on where the trajectory is and not on how big the separation is, and the total growth over a long time is the product of many such factors along the orbit.

The exponent is the long-run average of the logarithm of those factors, and since the trajectory spends its time on the attractor in a statistically definite way, the average converges to a number that depends on the attractor and not on the starting pair. That convergence is the content of Oseledets’ theorem, and it is what turns a slogan into a quantity.

There is a consequence worth noticing: the exponent is an average, so short stretches of an orbit can contract rather than expand. A pair of trajectories can come closer for a while — the figure’s lines wobble — and the exponential growth is what happens over the long run.

Where the number comes from #

The return map of the previous rung gives a second route to the same number, and comparing them is worth doing.

That map has slope everywhere greater than one, and applying it once multiplies a small separation by that slope. So over $k$ returns the separation grows by the product of the slopes, and the exponent per return is the average of $\log|f'|$. Dividing by the average return time converts it to an exponent per unit of time, which comes out near the number measured directly.

Two routes, one number, and the routes have nothing in common: one integrates the full three-dimensional system and measures a gap, the other averages the slope of a one-dimensional model. That they agree is evidence that the reduction of the previous rung is doing what it claims.

The second route also explains a feature the first only displays. The average of $\log|f'|$ is taken with respect to how often the orbit visits each part of the interval, so a steep region visited rarely contributes little. That weighting is the attractor’s own invariant measure, and it is why the exponent is not simply the average slope of the curve: the system’s own statistics decide which parts of the map matter, and computing the exponent requires knowing them.

Saturation is worth understanding rather than treating as an artefact, because it is where the exponent’s regime ends and the geometry begins. Exponential growth of a separation inside a bounded set is a contradiction if it continues, so something must intervene, and the something is that the two trajectories eventually land on opposite sides of the fold. After that their separation is not a small perturbation of anything and the linear theory has nothing to say.

What it does not say #

Three misreadings are common enough to be worth heading off.

It does not say the system is unpredictable. It says a specific prediction has a horizon. The statistics of the system — how often it visits each lobe, the average value of a coordinate, the shape of the attractor — are perfectly predictable and are not affected by the exponent at all. Weather is chaotic and climate is not, and the distinction is exactly this one.

It does not say the system is random. Every trajectory is a deterministic solution of three equations, and given exact initial conditions the future is determined. What fails is the usefulness of that determination under any finite precision, which is a different claim.

It does not say every pair of nearby points separates at that rate. The exponent is an average over the attractor, and there are places where a pair contracts and places where it expands much faster than the average. The finite-time exponent varies from region to region, and its variation is what makes some forecasts much worse than others from the same model — a phenomenon forecasters see as “predictable and unpredictable weather regimes” and which the single average number hides completely.

And it does not mean small causes have large effects. A small change produces a small change, for a while. What is unusual is the rate at which “a while” runs out, and the rate is a fixed number rather than a catastrophe.

The full spectrum, and what the other exponents say #

A three-dimensional system has three exponents, one per direction, and the largest is only part of the story.

For the Lorenz system they are approximately $0.9$, $0$ and $-14.6$. The zero is generic and means something specific: along the direction of the flow itself, two points on the same trajectory stay the same distance apart in the sense that matters, so there is no growth or decay. Every continuous flow with a bounded non-fixed orbit has a zero exponent, and its presence is a check on any computation.

The large negative exponent is the volume contraction, and it is the reason the return map is thin. Their sum, $0.9 + 0 - 14.6$, is negative, which is the statement that volumes shrink — and it equals the divergence of the vector field, which for the Lorenz system is the constant $-(\sigma + 1 + \beta) = -13.7$, close to the sum measured.

One direction stretches, one does nothing, and one squashes hard. Everything about the attractor’s geometry follows from that combination: the stretching produces the sensitivity, the squashing produces the thinness, and the fact that the object stays bounded forces the folding that the next rung is about.

What the number means outside this system #

The exponent is a property of a particular set of equations, and the reason it is worth measuring is that the same measurement is available anywhere.

Weather. Estimates of the largest exponent for the atmosphere put the doubling time of a small error at around a day and a half. With the tenfold rule above, that puts a hard limit on deterministic forecasting somewhere around two weeks, and no improvement in models or observations moves it much — which is why forecasts beyond that point are probabilistic rather than deterministic, and why the change was a change of ambition rather than a temporary state.

The solar system. The exponent for the planetary orbits corresponds to a Lyapunov time — the time for an error to grow by a factor of $e$ — of about five million years. The orbits are stable enough to be predicted for millions of years and not for hundreds of millions, and the question of whether Mercury’s orbit can destabilise over the age of the sun is genuinely open for this reason.

And a driven pendulum has a measurable exponent on a bench, which is the same stretch-and-fold behaviour in an apparatus small enough to hold, which is the reason this is a subject with experiments rather than only simulations.

The uniform thing across those three is the shape of the conclusion. Not “unpredictable”, but “predictable for a time that grows like the logarithm of the precision”. A logarithm is a very slow function, and that single fact is what makes chaos a practical limit rather than a philosophical one.

What the pictures cannot show #

The exponential is drawn on a logarithmic axis and looks linear. That is the point of the axis and it is also a trap: a reader who does not notice reads the figure as saying the gap grows steadily, when it is doubling in a fixed time. The straightness of the line is the exponential growth.

The slope is fitted over the middle stretch. At the start the separation is dominated by the particular direction the two points were offset in; at the end it saturates. Both ends are excluded from the fit, which is the right thing to do and means the number reported comes from a chosen window.

The three lines are three runs and the claim is about all runs. Three pairs agreeing is three pieces of evidence for a statement whose content is that every pair behaves this way, and the theorem that makes it so — that the average converges for almost every starting point — is not something a picture of three orbits can establish.

And a numerical trajectory is not a trajectory. The computed orbits diverge from the true ones at the very rate being measured, so the two curves being compared are both wrong. What saves the measurement is that the growth rate is a property of the attractor and the computed orbit stays on it; the number is right and the orbits are not.

Where the ladder goes next #

The next rung asks how a system can stretch every distance at every step and stay bounded, and the answer is the mechanism this rung’s exponents describe: stretching and folding. The rung after that asks what kind of set repeated folding leaves behind.

Also named as a debt: the finite-time exponents and their distribution over the attractor, which is the quantity a forecaster actually wants and which a single average conceals. Nothing above measures it.

Sideways, the observation this rung quantifies is the one Lorenz made by accident, the averaging of a logarithm along an orbit is the ergodic average applied to a derivative, the folded map’s own coding is what makes a pair’s histories diverge letter by letter, and the reduction that gives the second computation of the exponent is the return map.

How the exponent is actually computed #

The direct measurement in the figures is the honest one and it is not how the number is computed in practice, because it runs out of room.

The gap saturates once it reaches the size of the attractor, so a single pair of trajectories gives only the few time units before that happens. The standard method is renormalisation: run the pair, and whenever the gap grows beyond a small threshold, pull the second trajectory back along the separation direction to the original distance, recording the factor by which it had grown. Repeat for a long time, and the exponent is the average of the logarithms of the recorded factors, divided by the time.

That procedure measures the same quantity and never lets the linear approximation break down, which is what makes it correct. It also makes clear what is being averaged: the exponent is the mean rate of stretching along the orbit, and the pair of trajectories is a device for sampling it.

A subtlety worth knowing: the separation direction rotates as the trajectory moves, and it converges to the most-expanding direction whatever it started as — which is why a single arbitrary offset gives the largest exponent rather than some combination. Getting the smaller exponents needs several directions kept orthogonal to each other as they evolve, which is what the standard algorithms do.

What is worth carrying away #

The interesting content of sensitive dependence is a rate, and a rate is a number that can be measured, compared and used.

Knowing that a system is chaotic says almost nothing. Knowing its largest exponent says how long a forecast survives, how much a tenfold improvement in instruments buys, and — through the sum of the exponents — how fast the system forgets a volume of initial conditions. All three are practical questions with numerical answers.

The habit worth taking is to replace a qualitative claim by the quantity it is about. “Sensitive to initial conditions” becomes “the exponent is nine tenths per time unit”, and the second sentence supports arithmetic while the first supports only agreement.