The exponential of a square

The equation $y' = ky$ has exactly one solution through each starting point, and it is $y_0e^{kx}$. That is a statement about one quantity growing at a rate proportional to itself.

Almost nothing in the world is one quantity. Two chemicals feeding into each other, two populations preying on one another, a current and a voltage in a circuit, a position and a velocity — each of these is a pair whose rates depend on both members, and the natural way to write that is

$$\mathbf{y}' = A\mathbf{y},$$

with $\mathbf{y}$ a pair and $A$ a matrix. The question this rung answers is whether the solution is still an exponential, and the answer is that it is, of exactly the same shape, once the notation is taken seriously.

Taking the series seriously #

The exponential has a series,

$$e^x = 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!} + \cdots,$$

and every operation in it — adding, multiplying by itself, dividing by a whole number — is something a square matrix can do. So define

$$e^A = I + A + \frac{A^2}{2!} + \frac{A^3}{3!} + \cdots,$$

with $I$ in the place of 1. Nothing has been assumed; a definition has been written down, and the only question is whether it converges. It does, and comfortably: the entries of $A^n$ grow at most like $(n\|A\|)^n$ while the denominator is $n!$, which wins.

The figure sums sixty terms and asserts that the sixtieth is smaller than $10^{-14}$ in every entry, which for the matrices drawn here it is by a wide margin.

Three routes to the same matrix #

A definition by a series is worth having and hard to trust. The figure therefore computes $e^A$ by three routes that share no arithmetic and requires all three to agree to seven decimal places.

By the series, as above.

By the eigenvalues. If $A = PDP^{-1}$ then $A^n = PD^nP^{-1}$, so every term of the series has a $P$ on the left and a $P^{-1}$ on the right and they can be factored out:

$$e^A = P e^D P^{-1},$$

where $e^D$ is the diagonal matrix of $e^{\lambda_1}$ and $e^{\lambda_2}$. Exponentiating a matrix means exponentiating its eigenvalues, which is diagonalisation doing what it always does — turning a hard operation on a matrix into an easy operation on two numbers.

By following the flow. Integrate the differential equation numerically from $(1, 0)$ and from $(0, 1)$, and see where each arrives after one unit of time. Those two arrival points are the columns of $e^A$, because a linear map is determined by what it does to the two basis arrows.

That third route is the one that makes the picture evidence rather than decoration. The curves drawn on the page are that integration, so the paths and the matrix in the caption are one computation, and the agreement with the series is a claim that could have failed.

Doubling the time is not a new idea and it is worth drawing, because it makes the object visible as a family: there is one matrix for each $t$, they compose, and the differential equation is the statement of how the family changes with $t$. That family is what the next section is about.

Why it solves the equation #

The claim is that $\mathbf{y}(t) = e^{At}\mathbf{y}_0$ solves $\mathbf{y}' = A\mathbf{y}$ with $\mathbf{y}(0) = \mathbf{y}_0$, and the proof is the series differentiated term by term:

$$\frac{d}{dt}\left(I + At + \frac{A^2t^2}{2} + \cdots\right) = A + A^2 t + \frac{A^3t^2}{2} + \cdots = A\left(I + At + \cdots\right),$$

which is $A$ times the original. Setting $t = 0$ gives $I$, so the initial condition holds. And the uniqueness argument from the previous rung transfers unchanged: if $\mathbf{y}' = A\mathbf{y}$ then $(e^{-At}\mathbf{y})' = 0$, so $e^{-At}\mathbf{y}$ is constant.

Every linear system with constant coefficients is therefore solved, completely, by one formula — and the work of understanding a particular system reduces to understanding one matrix.

That is worth pausing on as a matter of method. The series was derived for real numbers and is being applied to objects that are not numbers, and the licence for doing so is not analogy — it is that every operation appearing in it is defined for the new objects, and the convergence argument survives with sizes in place of absolute values. Wherever those two conditions hold, the exponential comes along: for complex numbers, for matrices, for bounded operators on a space of functions, for quaternions. The definition travels because it asks for so little.

The one law that fails #

This is where the analogy has to be watched, and the failure is not a technicality.

For numbers, $e^{a+b} = e^ae^b$. For matrices this is false in general, and it is false for the reason everything about matrices is: $A$ and $B$ need not commute. Multiplying out the two series and comparing term by term, the quadratic terms are $(A+B)^2/2$ on one side and $(A^2 + 2AB + B^2)/2$ on the other, and those differ by $(AB - BA)/2$.

So $e^{A+B} = e^Ae^B$ exactly when $A$ and $B$ commute, and the discrepancy is measured by the commutator $AB - BA$ — a quantity that is zero for numbers and is the whole subject of Lie theory for matrices. What replaces the addition law is the Baker–Campbell–Hausdorff formula, an infinite series of nested commutators, and its existence is the reason the exponential is the bridge between a group of transformations and the flat space of matrices near its identity.

The failure is not a blemish to be worked around; it is where the subject is. Two flows that do not commute — a rotation about one axis and a rotation about another — compose into something that depends on the order, and the commutator measures by how much. Doing $A$ then $B$ then undoing $A$ then undoing $B$ leaves a residue proportional to $AB - BA$, which is why a car can be parked sideways by a sequence of forward and turning motions none of which moves it sideways. The matrix exponential is the setting in which that observation becomes an equation.

The single fact that does survive is $e^{A}e^{-A} = I$, because $A$ commutes with itself. So $e^A$ is always invertible, whatever $A$ is — including matrices with zero determinant, whose exponentials are perfectly well behaved.

A rotation from no trigonometry #

Here is the case that makes the whole construction feel inevitable.

Take the matrix $J$ that sends $(x, y)$ to $(-y, x)$ — a quarter turn — and note $J^2 = -I$. Then $J^3 = -J$, $J^4 = I$, and the powers cycle with period four exactly as the powers of $i$ do. Feeding $\theta J$ into the series and separating the terms with $I$ from the terms with $J$:

$$e^{\theta J} = \left(1 - \frac{\theta^2}{2!} + \frac{\theta^4}{4!} - \cdots\right)I + \left(\theta - \frac{\theta^3}{3!} + \cdots\right)J,$$

and the two brackets are the series for cosine and sine. So

$$e^{\theta J} = \begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix},$$

which is a rotation by $\theta$. Nothing trigonometric was assumed; the two series fell out of sorting the powers of $J$ into even and odd.

This is Euler’s identity with a matrix where the imaginary unit was, and the correspondence is exact: the matrices $aI + bJ$ multiply exactly as the complex numbers $a + bi$ do. The complex plane is a subring of the two-by-two matrices, and the exponential agrees on both sides of that identification. So $e^{i\pi} = -1$ and $e^{\pi J} = -I$ are the same statement, one of them written with an $i$ and the other with a half turn drawn on a page.

Reading the behaviour off two numbers #

Because the eigenvalues decide everything, the qualitative behaviour of a two-dimensional linear system is decided by the trace and the determinant — the same two numbers that make up the characteristic polynomial — and the classification is short enough to hold in the head.

If the determinant is negative the eigenvalues are real with opposite signs, and the origin is a saddle: points run out along one invariant line and in along the other, as in the second figure above. If the determinant is positive, the sign of the trace decides whether everything runs out or in, and the discriminant decides whether it does so along straight lines or by spiralling. A zero trace with positive determinant gives closed orbits and nothing else happens at all.

That is the whole of it: four behaviours, decided by two numbers and one inequality between them. The reason a subject as large as linear systems has so short a classification is that the diagonal form leaves nothing to happen beyond two independent one-dimensional stories, and the only extra possibilities come from the eigenvalues being complex or the eigenvectors being missing.

It also means the boundaries of the classification matter more than the interior. A system whose trace is exactly zero has closed orbits; nudge the trace and every orbit becomes a spiral, inwards or outwards according to the sign. A closed orbit is therefore not a robust feature of a linear system, which is why closed orbits in nature are almost always evidence of something nonlinear holding them in place.

What the trace and determinant do #

Two properties survive the exponential in a clean form.

$$\det(e^A) = e^{\operatorname{tr} A}.$$

For a diagonalisable matrix this is immediate: the determinant of $e^A$ is the product of $e^{\lambda_i}$, which is $e^{\sum \lambda_i}$, and the sum of the eigenvalues is the trace. The figure checks it for every matrix it draws, including the ones with complex eigenvalues where the eigen-route is not available.

The identity has a direct reading in the picture. The determinant of $e^{At}$ is the factor by which the flow multiplies areas after time $t$, so a system whose matrix has zero trace preserves area exactly — every blob of starting points is carried to a blob of the same size, however distorted. That is why the skew matrix above, whose trace is zero, has circles for orbits and why an area-preserving flow is the normal state of affairs in mechanics rather than a coincidence.

The defective case is worth seeing because the series does not care. Diagonalisation fails and the definition does not depend on it; $e^A$ exists for every square matrix, and for a nilpotent one it is a finite sum. Where a defective matrix does show is in the behaviour: a repeated eigenvalue with a missing eigenvector produces solutions with a factor of $t$ in front of the exponential, which is growth no diagonal system can produce.

Where this is used, and why it is not optional #

Three settings, chosen because in each the matrix exponential is the definition rather than a technique.

Mechanics. A system near equilibrium is described by a linear system, and its behaviour over time is $e^{At}$ applied to the initial state. Whether a bridge oscillates, damps or diverges is the sign of a real part; whether the oscillation is at one frequency or several is how many complex pairs there are.

Markov chains in continuous time. A process that jumps between states at constant rates has a matrix of rates whose exponential is the matrix of probabilities after time $t$. The stationary distribution is again the eigenvector belonging to eigenvalue zero of the rate matrix, which becomes eigenvalue one after exponentiating — the same object as in the discrete case, reached by a different door.

Rotations in three dimensions. Every rotation is $e^{\theta K}$ for a skew matrix $K$ built from its axis, which is Rodrigues’ formula and is the reason rotations are stored and interpolated as axis-and-angle rather than as nine numbers. The skew matrices are the tangent space to the rotations at the identity, and the exponential is the map from the flat thing to the curved one — the two-dimensional case above, one dimension up.

The common thread is worth stating. In each case there is a set of transformations that forms a group under composition and is curved, and a set of “generators” that is flat and easy to compute with, and the exponential is the bridge. Everything hard about the group is pushed into the failure of $e^{A+B} = e^Ae^B$, which is where the curvature went.

What the picture cannot show #

The flow is drawn for one unit of time, and $e^{At}$ for other $t$ is a different matrix. A picture of one moment cannot show that the family $e^{At}$ is a group — that running for time $s$ and then for time $t$ is the same as running for $s + t$ — although that is the property doing most of the work whenever this object is used. Drawing it would require several panels showing the same thing.

Nor does the two-dimensional picture suggest what happens with a large matrix, where the interesting question is which of many eigenvalues dominates and how long the transient behaviour lasts before it does. The rate at which a solution settles is governed by the eigenvalue with the largest real part, and the size of the excursion before it settles is governed by something else entirely — the same gap between eventual behaviour and immediate behaviour that separates eigenvalues from singular values.

And there is a practical warning the drawing gives no hint of. Computing $e^A$ by summing the series is a poor idea for a real matrix: for a matrix with large entries the terms grow enormously before the factorials take over, and the answer is a small number obtained by cancelling large ones. There is a well-known paper called Nineteen Dubious Ways to Compute the Exponential of a Matrix, and summing the definition is one of them.

The ladder from here #

Rungs above: the logarithm of a matrix, and when it exists. The one-parameter groups that every matrix generates, and Lie’s correspondence between them and their generators. The Baker–Campbell–Hausdorff series in full. Second-order equations turned into first-order systems by carrying the derivative as an extra coordinate. Stability, read off the real parts of the eigenvalues. And the exponential of an operator on functions, where the same series generates the heat flow and the translation of a graph along its own axis.