The directions a map leaves alone

A linear map picks up the plane and redraws it. Almost every arrow it touches comes out pointing in a new direction — that is what makes it a transformation rather than a scaling.

But not every arrow. For most maps there are a few special directions along which the arrow comes out pointing exactly where it went in, merely longer or shorter, and those directions turn out to carry more information about the map than the four numbers in its matrix do.

What the picture is showing #

Every arrow in the figure starts on the unit circle and ends where the map sends it. Read one at a time, each says this went to that.

Most of them turn. The ones that do not are drawn heavier, and they lie on two lines. Along the first, the map multiplies everything by $3$. Along the second, it does nothing at all — the multiplier is $1$.

Those lines are the eigenvectors, and the multipliers are the eigenvalues. The word means own or characteristic in German, and the awkwardness of the half-translation has survived a century of attempts to replace it.

Stated as an equation, an eigenvector is a non-zero $v$ with

$$Av = \lambda v$$

for some number $\lambda$ — the map’s action on $v$ is indistinguishable from multiplying by a number. The generator checks exactly this before drawing: it applies the matrix to each claimed eigenvector and requires the result to be $\lambda v$ to machine precision, because a line drawn from a slightly wrong calculation looks entirely convincing.

The same map, drawn the other way #

It is worth putting this figure beside the one the grid essay would draw for the same matrix, because the two pictures show the same transformation and answer different questions.

The grid shows what happened to everything and makes the global deformation obvious. The eigen-picture shows what happened to almost nothing — two lines out of infinitely many — and makes the structure obvious instead. The grid answers “how much did this distort”; the eigen-picture answers “what did it not distort”.

Neither contains the other. A reader shown only the grid could stare at it a long time without finding the diagonal lines, because a parallelogram does not advertise which of its directions were fixed. A reader shown only the eigen-picture would not know the area factor without multiplying the eigenvalues.

That is the general condition of this subject and worth stating once: a transformation has several honest pictures, each keeping a different part of it, and the skill is knowing which question each one answers. It is the same relationship the determinant has with the rank — different lenses, neither reducible to the other, both computed from the same four numbers.

Why the special directions matter more than the matrix #

The matrix has four numbers and they are not intrinsic to the map. Change the axes and all four change, while the transformation itself does nothing different — the grid was relabelled, not deformed.

The eigen-directions do not change. They are places in the plane, and rotating the coordinate system does not move a place. Neither do the eigenvalues: a stretch by $3$ is a stretch by $3$ in any labelling.

So the eigen-data is what belongs to the map rather than to the description of it, in the same way the determinant is. And it determines a great deal. In this example the two eigenvalues are $3$ and $1$, so their product is $3$ — which is the determinant, the area factor. Their sum is $4$, which is the trace, the sum of the diagonal entries. Both of those hold generally: the eigenvalues multiply to the determinant and add to the trace, which is a useful pair of checks and a hint that these numbers are the map’s real parameters.

The contrast between those two is the essay’s second theme in miniature: the triangular map’s directions lean, the symmetric one’s do not, and nothing about the pictures suggests which property of the entries decides it.

What they are for #

The reason eigenvectors are worth finding is that they make repetition trivial.

Applying a map many times is ordinarily an expensive and opaque business: multiply the matrix by itself, watch the entries turn into something unrecognisable, and have no idea where a given point ends up. Along an eigen-direction it is arithmetic. Apply the map $n$ times to an eigenvector and the result is $\lambda^n v$ — the direction never changes and the length is a power.

Now take any starting vector at all and write it as a combination of the eigenvectors, which is possible whenever there are enough of them. The map acts on each piece independently, so applying it $n$ times gives

$$A^n x = c_1\lambda_1^n v_1 + c_2\lambda_2^n v_2.$$

The complicated problem has become two separate scalar problems, and it is immediately obvious what happens in the long run: the term with the largest $|\lambda|$ swamps the others, so almost every starting vector ends up pointing along the dominant eigenvector. That is why the eigenvalues are the right thing to know about a repeated process — a population model, a random walk, the state of a physical system left to itself, a web-ranking algorithm — and it is one calculation rather than a simulation.

The Fibonacci numbers as a matrix power #

The cleanest instance of that argument is a sequence with no matrices anywhere in its statement.

Write the step from one pair of Fibonacci numbers to the next as a map: $(a, b) \mapsto (a + b, a)$, which as a matrix is $\begin{pmatrix}1&1\\1&0\end{pmatrix}$. Applying it $n$ times to $(1, 0)$ produces $(F_{n+1}, F_n)$, so the whole sequence is the powers of one small matrix.

Its eigenvalues are the roots of $\lambda^2 - \lambda - 1 = 0$, which is the golden ratio’s defining equation — so they are $\varphi = 1.6180\ldots$ and $-1/\varphi = -0.6180\ldots$.

Everything about the sequence follows from those two numbers. Since $F_n$ is a combination of $\varphi^n$ and $(-1/\varphi)^n$, and the second term shrinks to nothing, the Fibonacci numbers grow like $\varphi^n$ and the ratio of consecutive terms converges to $\varphi$. The second eigenvalue is negative, which is why the convergence alternates — the shrinking term flips sign at every step, so the ratios overshoot above and below in turn. That alternation was observed in the golden-ratio essay as a fact about the picture; here it is a consequence of one eigenvalue being negative.

Writing the combination out gives Binet’s formula, an expression full of $\sqrt5$ that returns whole numbers for every input. It looks like a conjuring trick and it is a change of basis: the Fibonacci numbers are being expressed in the coordinate system where the map is diagonal, and that system has irrational axes.

Change of basis is the formal version. Choosing the eigenvectors as axes turns the matrix diagonal, and a diagonal matrix is one that stretches each axis separately and does nothing else. Diagonalising is finding the coordinate system in which the map is boring, and the whole difficulty was that the original axes were the wrong ones.

The maps with no such direction #

Not every map has one, and the counterexample is the most familiar transformation there is.

A rotation by ninety degrees turns every direction. There is no line it leaves alone, because leaving a line alone is precisely what a rotation does not do. So it has no real eigenvectors, and the figure has nothing to draw.

The algebra agrees and then says something more interesting. Eigenvalues are roots of the characteristic polynomial $\det(A - \lambda I) = 0$, which for this rotation is $\lambda^2 + 1 = 0$ — no real solutions. But it has complex ones, $\pm i$, and those are exactly the numbers that perform a quarter turn.

That is not a coincidence dressed up. Over the complex numbers every polynomial has roots, so every matrix has eigenvalues, and a real matrix with no real eigen-direction has a complex pair whose argument is the angle it rotates through and whose modulus is the amount it scales. The eigenvalue does not stop existing; it stops being real, and the rotation is what a complex eigenvalue looks like when the picture is confined to the real plane.

There is a second, different failure worth separating from it.

The shear $\begin{pmatrix}1 & 1\\ 0 & 1\end{pmatrix}$ has the repeated real eigenvalue $1$ — and only one eigen-direction, the horizontal, rather than two. It cannot be diagonalised: there is no basis of eigenvectors, because there are not enough of them. So maps fall into three cases rather than two, and has no real eigenvector and has too few eigenvectors are separate diseases with separate cures.

Where the perpendicular ones come from #

The first figure’s two eigen-directions are at right angles, and the second figure’s are not. Which happens is decided by a property that has nothing obviously to do with directions.

A matrix is symmetric when it equals its own transpose — when the entry in row $i$, column $j$ matches the one in row $j$, column $i$. The first example, $\begin{pmatrix}2&1\\1&2\end{pmatrix}$, is symmetric. The triangular one is not.

The spectral theorem says that a real symmetric matrix always has real eigenvalues and a perpendicular set of eigenvectors. Always — no exceptions, no shortage, no complex pairs.

The reason routes through the dot product, and it is short enough to sketch. Symmetry means $Av \cdot w = v \cdot Aw$ for every pair. Apply that to two eigenvectors with different eigenvalues: the left side is $\lambda_1 (v \cdot w)$ and the right is $\lambda_2 (v \cdot w)$. Since $\lambda_1 \ne \lambda_2$, the only escape is $v \cdot w = 0$ — the eigenvectors are perpendicular, and the whole argument is one line of moving a matrix across a dot product.

This matters far beyond tidiness. Symmetric matrices are the ones that arise from quadratic forms, from covariances, from moments of inertia, from energies — anywhere a quantity depends on a direction and does not care which way along it. In every one of those settings the guarantee applies: there is a set of perpendicular axes in which the situation decouples completely, and finding them is the standard first move. Principal axes of a rotating body, principal components of a data set, and normal modes of a vibrating structure are three names for this one theorem.

What the picture cannot show #

The figure draws sixteen arrows and the claim is about all directions. A near-eigenvector — one turned by a tenth of a degree — would be indistinguishable from a real one at any drawn size, and the figure marks a direction as eigen when the cross product vanishes to machine precision rather than when it looks parallel.

More seriously, the picture cannot show the repeated-eigenvalue case honestly. The shear’s single eigen-direction looks like an ordinary eigen-direction; nothing visible says and there is no second one. An absence is invisible, and the difference between a map with two eigen-directions very close together and a map with one is the difference between diagonalisable and not — which governs whether the whole apparatus of the previous sections applies. That distinction is discontinuous in the entries and has no visual signature whatever.

Nor can two dimensions carry the reason any of this is worth doing. The payoff is that an $n \times n$ problem becomes $n$ scalar problems, and at $n = 2$ that saves almost nothing. The figures show the mechanism at the one size where the mechanism is not needed.

The ladder from here #

Rungs above: the characteristic polynomial derived, and why $\det(A - \lambda I) = 0$ is the right equation. Diagonalisation carried out, with the change-of-basis matrices drawn as grids. The power method, which finds the dominant eigenvector by doing nothing but applying the map repeatedly. Repeated eigenvalues and Jordan form, where diagonalisation fails and the nearest available thing is described. The spectral theorem proved. Singular value decomposition, which gives every map — square or not, symmetric or not — a pair of perpendicular frames, and is what to reach for when eigenvectors are not available. Complex eigenvalues read as rotation and scaling. Markov chains and their steady states, where the dominant eigenvalue is $1$ and its eigenvector is the long-run distribution. And the connection back to the determinant and trace, which are the eigenvalues’ product and sum.

The habit #

The move this essay describes is one of the most transferable in mathematics: look for the parts of a problem the operation does not mix up.

A linear map mixes directions together, and finding the ones it leaves alone converts one hard problem into several easy ones. A Fourier decomposition does exactly the same thing for differentiation, which is why heat diffusion becomes a family of independent exponential decays: the sine waves are the eigenvectors of the second-derivative operator, and each one is scaled rather than reshaped.

The general recipe is the same in both cases. Find the objects the operation merely scales, express whatever is at hand as a combination of them, and the operation becomes multiplication. Whether that is available is usually the difference between a problem being tractable and not, and it is worth asking early rather than after the direct approach has failed.