How fast two orbits part

The last essay established that two nearby orbits separate. It did not say how fast, and “fast” needs a unit before it means anything.

The unit turns out to be per step, in the exponent: gaps do not grow by a fixed amount each step, they grow by a fixed factor. The number that describes the growth is therefore a rate of exponential growth, and the standard way to extract one is to take a logarithm.

That curve is the whole subject in one picture: negative on the left, crossing zero at the end of the cascade, and positive but ragged after it. The raggedness is not noise — every downward spike is a window where order comes back.

One step, then all of them #

Start with a single step, which is calculus and nothing more.

Two points a distance $d$ apart, with $d$ small, are mapped to two points about $|f'(x)|\,d$ apart. That is the definition of the derivative, read as a statement about a gap rather than about a limit.

Now do it $n$ times, from $x_0$. The gap gets multiplied by $|f'(x_0)|$, then by $|f'(x_1)|$, then by $|f'(x_2)|$, and so on, so after $n$ steps it is

$$d_n \approx d_0 \prod_{i=0}^{n-1} |f'(x_i)|.$$

A product of $n$ numbers is awkward. Its logarithm is a sum of $n$ numbers, which is not, so take it:

$$\ln\frac{d_n}{d_0} \approx \sum_{i=0}^{n-1} \ln|f'(x_i)|.$$

Divide by $n$ and the right-hand side becomes an average, of the quantity $\ln|f'|$, over the points the orbit visits. That average is the Lyapunov exponent:

$$\lambda = \lim_{n\to\infty} \frac 1n \sum_{i=0}^{n-1} \ln|f'(x_i)|.$$

Everything else in this essay is a consequence of that line. The logarithm is there because multiplication is the natural operation and addition is the tractable one; the average is there because the derivative is different at every point of the orbit and only the typical value matters over a long run.

What the sign means #

Read the definition backwards: $d_n \approx d_0 e^{\lambda n}$.

Negative $\lambda$. Gaps shrink exponentially. Nearby orbits converge, the system forgets where it started, and the long-run behaviour is a fixed point or a cycle. This is the entire left half of the figure.

Positive $\lambda$. Gaps grow exponentially. This is sensitive dependence, and the number says how fast.

Zero $\lambda$. The borderline, and it is not a technicality. It happens exactly at the parameters where something changes — every period-doubling, the end of the cascade, and the birth of every window — because the marginal case is what a bifurcation is.

Placing those two pictures on top of each other is the strongest argument this field has that its two descriptions are the same description. One is drawn from where orbits go, the other from the average of a derivative, and their features line up parameter for parameter. Two independent constructions agreeing on every feature is the same kind of evidence as the constant that does not care which map turning up in families that share nothing but a shape.

The clearest single-step version of the same statement is the picture the stability essay was built around, since a fixed point is the case where the orbit visits one place and the average is just the derivative there.

For a fixed point the definition collapses to $\lambda = \ln|f'(x^*)|$, and the condition $|f'| < 1$ becomes $\lambda < 0$. For a $k$-cycle it collapses to the average of the logarithm over the $k$ points, which is $\frac1k \ln|(f^k)'|$ — the multiplier of the cycle, which is the quantity that decides its stability. So the exponent is not a new criterion; it is the old one, written so that it still means something when there is no cycle to take a derivative of.

The one number that can be checked #

Everything in the Lyapunov curve is numerical: run an orbit for a few thousand steps, average a logarithm, plot the answer. Numerical curves are exactly the kind of thing that can be quietly wrong, so the generator wants one value it can verify against arithmetic rather than against another computation.

At $r = 4$ there is one. The logistic map at $4$ is conjugate to the tent map at slope $2$ — the substitution $x = \sin^2(\pi\theta/2)$ carries one to the other exactly — and the tent map at slope $2$ multiplies every gap by exactly $2$ at every step, since its derivative is $\pm 2$ everywhere. So

$$\lambda(4) = \ln 2 = 0.693147\ldots$$

with no approximation anywhere. The generator computes $\lambda(4)$ the same way it computes every other point on the curve, and asserts that the answer is $\ln 2$. If the sampling, the transient discard, the averaging or the derivative were wrong, that one point would be wrong, and the assertion would fail.

A curve with one exactly known point on it is a different object from a curve with none. This is the same discipline as checking a numerical integral against a case with a closed form: the check does not prove the other values, but a method that gets the checkable one right is a method that has been tested.

Why not just measure the gap #

There is an obvious alternative: start two orbits close together, watch the gap, fit an exponential. It is what the last essay’s first figure does by eye.

It does not work as a measurement, for the reason that essay ended on. The gap stops growing once it reaches the size of the interval, so an exponential fit over a long run reports the wrong answer, and over a short run it reports whatever the first few steps happened to do.

The derivative formulation has neither problem. It never mentions a second orbit at all — it is the infinitesimal separation rate, extracted from one orbit — so there is no gap to saturate. That is a recurring trade in this collection: a quantity defined as a limit is often easier to compute than the finite thing it describes.

The practical algorithm still uses a second orbit, but renormalises: step both, measure the gap, record its logarithm, then pull the second point back to the original distance and repeat. That keeps the gap infinitesimal in effect while doing arithmetic at a usable scale, and it generalises to systems where no formula for $f'$ is available.

The windows are visible in the exponent #

The Lyapunov curve’s downward spikes are the most informative part of it, and they are the same phenomenon the period-three window showed as a clear band in the bifurcation diagram.

Inside a window, the map has an attracting cycle. An attracting cycle has $|{\left(f^k\right)}'| < 1$ around it — that is what attracting means — so the average of $\ln|f'|$ is negative there. The exponent must dip below zero, and it does, sharply, at every window.

That figure is worth comparing to the same range of the bifurcation diagram. The window there is a striking visual event — three clean curves in the middle of a smear — but its boundaries are hard to read off, because a smear fading into three curves has no obvious edge. In the exponent the boundaries are the zero crossings, which are unambiguous.

This is a general use for the quantity: it converts a qualitative judgement about a picture into a sign test on a number, and sign tests can be automated.

Reading the curve honestly #

Two features of the plotted curve are artefacts of how it is computed, and saying so is part of publishing it.

The negative spikes have no bottom. Where the map has a superstable cycle — one that contains the critical point $x = 1/2$, where $f' = 0$ — the logarithm is $-\infty$ and the true exponent dives without limit. The plot shows a finite spike because the average is taken over finitely many steps and the exact critical point is never hit. Every window contains one of these, and the deepest visible dip in each is the drawn version of an infinity.

The positive values are noisy. An average over six thousand steps converges like $1/\sqrt{n}$, so neighbouring columns disagree by a little even where the true curve is smooth. The generator discards eight hundred steps of transient first, which removes the larger error — an orbit that has not yet reached the attractor is averaging the derivative over places the system does not actually visit.

Neither is hidden by drawing more columns; both are properties of the estimator. The one defence against reporting a numerical artefact as a result is the exact value at $r = 4$, which is why the generator asserts it rather than merely mentioning it.

Where the average comes from #

There is a step in the derivation that was passed over quickly and is the one a careful reader should object to.

The exponent was defined as a limit of averages along an orbit. Which orbit? A different starting point visits different places, so it averages $\ln|f'|$ over a different set of points, and there is no obvious reason the answer should be the same.

It is the same, for almost every starting point, and the reason is the consolation the last essay ended on. A chaotic orbit visits the interval according to a fixed distribution — the invariant density — and the fraction of its time spent near any point converges to the same thing regardless of where it started. An average along the orbit therefore converges to an average against that density, which does not mention the starting point at all:

$$\lambda = \int_0^1 \ln|f'(x)|\,\rho(x)\,dx.$$

This is Birkhoff’s ergodic theorem, and the word “almost” is doing real work: the exceptions are the periodic points, which are dense, infinite in number, and measure zero. Start exactly on the fixed point and the average is $\ln|f'|$ at that one place, which is not $\lambda$. Start anywhere a computer is likely to land and it is.

For the logistic map at $r = 4$ the invariant density is known in closed form — $\rho(x) = 1/\left(\pi\sqrt{x(1-x)}\right)$, the arcsine distribution — and doing that integral by hand gives $\ln 2$ again, by a completely different route from the tent-map conjugacy. Two derivations agreeing is the kind of evidence this collection puts weight on, in the same way that two ways of counting the same set prove an identity that neither proves alone.

More than one dimension #

The definition above uses $f'$, which is a single number, and that is a one-dimensional convenience.

In higher dimensions the derivative is a matrix, and it stretches different directions by different amounts. So there is not one exponent but a spectrum of them, one per dimension, each describing the growth rate along one direction of an evolving frame. The largest one dominates any generic perturbation, and is what “the” Lyapunov exponent usually means.

The signs of the whole spectrum classify the behaviour. All negative: everything converges to a point. One zero, the rest negative: a periodic orbit, the zero being the direction along the orbit, which neither grows nor shrinks. At least one positive with the sum negative: chaos on an attractor — stretching in some direction, overall contraction, and the two together are precisely the stretch-and-fold that makes a strange attractor. That is the case the Lorenz system sits in, and its exponents are the standard way of establishing that what its picture shows is genuinely chaotic rather than a very long cycle.

The spectrum has an exact constraint, and a dimension in it #

The three-number spectrum quoted for the Lorenz system is a numerical measurement like every other value on the curve above, and it has the same problem: nothing about it can be checked by eye. It has two properties that can be checked, and both are worth having.

The sum is known in closed form. Adding the exponents gives the rate at which a small blob of initial conditions loses volume, since the blob stretches along one direction by one exponent, along the next by the next, and so on. For a flow that rate is the divergence of the vector field — and for the Lorenz equations the divergence is the same at every point of space, coming to $\sigma + 1 + \beta$ of contraction, which at the standard parameters is $10 + 1 + \tfrac83$, a little under fourteen.

So the three measured exponents must add to that number, exactly, and they do. That is the same kind of anchor the value at $r = 4$ provides for the curve above: one quantity computed from the equations rather than from an orbit, against which the whole numerical spectrum can be tested. A measurement that missed it would be wrong somewhere, and the check costs nothing.

Notice what the sum being negative and one exponent being positive say together. Volume shrinks and lengths grow — which is the stretch-and-fold in arithmetic form, and the reason the attractor can be neither a point (all directions would shrink) nor a solid region (volume would be preserved). It has to be something in between, with no volume and more than a surface’s worth of structure.

And the spectrum says how much in between. The Kaplan–Yorke formula computes a dimension from the exponents alone: add them from the largest downwards until the running total would go negative, then add a fraction — how much of the next exponent it would take to bring the total to zero. For Lorenz the first two add to about nine-tenths, and the third would cancel that after contributing about a sixteenth of itself, so the attractor’s dimension is about 2.06.

That number is a little above two and far below three, which is exactly what the picture shows without being able to say: a surface, essentially, but one folded into infinitely many sheets, so that it is not quite a surface. It is a dimension that is not a whole number, computed here from stretching rates rather than from box counting — and where both have been measured on the same attractor, they agree.

That agreement is the payoff. A dimension is a property of the attractor’s shape, defined by covering it with boxes and counting; an exponent is a property of the motion on it, defined by averaging a derivative. Neither definition mentions the other, and one determines the other.

A quantity that measures how much is lost #

There is a second reading of $\lambda$ that arrives from an unrelated direction and lands on the same number.

If two starting points a distance $2^{-k}$ apart are indistinguishable to a measurement of $k$ binary digits, then each step of a map with exponent $\lambda$ makes $\lambda/\ln 2$ previously invisible digits visible — the stretching brings detail up from below the measurement threshold. Equivalently, each step destroys that many digits of the future: what was known becomes unknown at a fixed rate.

So the exponent is a rate of information production, measured in bits per step, and for the logistic map at $r = 4$ the rate is exactly one bit per step. That is not a metaphor. The map at $r = 4$ is conjugate to the doubling map, the doubling map shifts a binary expansion left by one place, and shifting left by one place reveals exactly one new digit per step and discards exactly one.

Pesin’s theorem makes the correspondence general: for a large class of systems, the sum of the positive Lyapunov exponents equals the Kolmogorov–Sinai entropy, which is the information-theoretic quantity defined without any reference to derivatives. A geometric rate of stretching and a rate of information generation turn out to be the same number, which is the sort of coincidence that usually means one definition was the wrong way to think about it.

Both of them are downstream of the same idea this collection has met before — that a measurement is worth what it rules out, and that counting what remains is often easier than describing it.

What the number is worth #

The exponent gives the forecast arithmetic its constant. With $\lambda$ known, an initial uncertainty $\varepsilon$ stays under a tolerance $\delta$ for about

$$n \approx \frac{1}{\lambda}\ln\frac{\delta}{\varepsilon}$$

steps, and that is the number a forecaster actually wants. For the logistic map at $r = 4$, with $\varepsilon = 10^{-16}$ and $\delta = 0.1$, it is about fifty steps — which is why every chaotic figure on this site stops well before it.

One more consequence is worth stating because it is the reason the number appears in fields that have nothing to do with maps on an interval. The exponent is a property of the system, not of the units it is written in: rescale the variable, change the clock, conjugate the map to another one, and $\lambda$ per unit time is unchanged. Quantities that survive a change of coordinates are the ones worth naming, which is the same reason the directions a map leaves alone are the ones worth finding.

For the atmosphere, the largest exponent corresponds to a doubling time of a day or two, and the usable horizon lands where the observed one does. That is the strongest evidence that this quantity means anything outside its own definition: it was defined for maps on an interval, computed for a system with millions of variables, and it predicted a fact about weather forecasting that was already known independently.