The dot product is a shadow

The dot product is introduced as an instruction. Given two lists of numbers, multiply the entries that line up and add the results:

$$(3, 2) \cdot (4, -1) = 3 \times 4 + 2 \times (-1) = 10.$$

Correct, quick, and entirely unmotivated. Nothing about that procedure suggests why it should matter, why the answer is a single number rather than another vector, or why the same expression turns up in the definition of work, in the test for perpendicularity, in the projection formula, and in the definition of an angle. The rule is a rule, and the meaning arrives later or not at all — the same complaint that can be made of a matrix presented as a box of numbers.

The picture supplies the meaning in one figure.

The shadow #

Shine a light perpendicular to $b$ and let $a$ cast a shadow onto $b$’s line. The shadow has a length, and it has a direction — it either points along $b$ or against it — so call it a signed length.

The dot product is that signed length multiplied by $|b|$:

$$a \cdot b = (\text{shadow of } a \text{ on } b) \times |b|.$$

Both readings of the picture give $10$, and they must, because they are the same number computed two ways. The generator checks it: it drops the perpendicular, verifies that the foot really is perpendicular by requiring $(a - \text{proj}) \cdot b = 0$, and then requires the shadow-times-length product to match the coordinate rule exactly.

Written with an angle, the shadow of $a$ is $|a|\cos\theta$ — which is what a cosine is, the horizontal reach of something at angle $\theta$ — and the whole thing becomes the form usually quoted:

$$a \cdot b = |a|\,|b|\cos\theta.$$

That form and the coordinate form look like different definitions and are the same one. The coordinate version is what to compute with; the shadow version is what it means; and every property below follows from the second without effort.

Everything falls out #

It is zero exactly at right angles. A shadow has zero length precisely when $a$ points straight up from $b$’s line. So $a \cdot b = 0$ means perpendicular — not as a convention or a coincidence of the formula, but because a right angle is the one configuration that casts no shadow. That is the single most-used fact about the dot product and it needs no calculation at all.

Its sign says which side. If $a$ leans the same way as $b$, the shadow points along $b$ and the product is positive. Lean the other way and it is negative.

It is symmetric, which is not obvious. The picture is lopsided: it projects $a$ onto $b$, which treats them differently. Yet $a \cdot b = b \cdot a$. Projecting the other way gives a shadow of a different length, on a line of a different length, and the two products agree — because both equal $|a||b|\cos\theta$, which does not care about the order. This is a case where the coordinate formula is the more obvious of the two, and it is worth noticing that the illuminating description is not always the one that makes every property visible.

It gives lengths. Dot a vector with itself and the angle is zero, so $a \cdot a = |a|^2$. In coordinates that is $x^2 + y^2$ — which is the Pythagorean theorem, arriving as a special case of a multiplication rule.

Why multiplying coordinates computes a shadow #

The two descriptions have been asserted to agree and it is worth seeing why, because the reason explains what the coordinate rule is really doing.

A coordinate is already a shadow. Writing $a = (3, 2)$ says that $a$ reaches $3$ along the horizontal axis and $2$ along the vertical — which is to say, its shadows on the two axis directions are $3$ and $2$. Coordinates are not an arbitrary labelling; they are projections onto a chosen set of directions.

The chosen directions have two properties that make everything work: each is a unit length, and they are perpendicular to each other. So when $a$ and $b$ are broken into their pieces along those directions and the products expanded, every cross term is a shadow of one axis on a different axis — which is zero — and every matching term is a product of two shadows on the same unit direction. Nine possible pairings in three dimensions, six of which vanish, three of which survive.

That is the coordinate rule. Multiply the matching entries and add is what is left of a general expansion after perpendicularity has deleted everything else, and the reason the entries have to match is that non-matching ones are at right angles.

It also explains a fact that otherwise looks like a coincidence: the rule gives the same answer in any rotated set of axes. The expansion only ever used that the axes are perpendicular unit vectors, and a rotated set is too, so the answer cannot depend on which one was picked.

What leaves it alone #

If the dot product measures a genuine geometric relationship, the transformations that preserve geometry ought to preserve it — and turning that around gives one of the more useful definitions in linear algebra.

A map that preserves all dot products preserves all lengths, since $a \cdot a = |a|^2$, and preserves all angles, since the angle is built from the product and the lengths. The converse holds as well. So preserving the dot product and being a rigid motion of space fixing the origin are the same condition, and the maps satisfying it are called orthogonal.

That is a definition worth having because it is checkable. A matrix is orthogonal exactly when its columns are perpendicular unit vectors — which is a set of dot products, computed directly from the entries with no geometry required. Rotations qualify, reflections qualify, and their determinants are $+1$ and $-1$ respectively, which is the handedness bit separating the two families.

The practical payoff is numerical. Orthogonal matrices do not amplify anything: applied to a vector they leave its length alone, so errors present in the input come out the same size rather than magnified. That single property is why so much of numerical linear algebra is organised around factoring matrices into orthogonal pieces, and it traces directly back to a shadow not being longer than the arrow that casts it.

The angle is a definition, not a measurement #

Rearranging gives

$$\cos\theta = \frac{a \cdot b}{|a|\,|b|},$$

which looks like a formula for computing an angle already present. In two dimensions it is. In higher dimensions it is the definition of the angle, and that is a much bigger claim than it appears.

There is no protractor in a thousand dimensions and no way to look at two vectors and see how far apart they point. The quantity on the right, however, is a perfectly ordinary arithmetic expression that can be evaluated in any number of dimensions at all. So the angle is defined to be whatever it says.

For that to work, the right-hand side must always lie between $-1$ and $1$ — otherwise it names no angle. That it does is the Cauchy–Schwarz inequality,

$$|a \cdot b| \le |a|\,|b|,$$

and it is the load-bearing inequality of the subject. Geometrically it is unremarkable — a shadow cannot be longer than the thing casting it — and that is precisely the intuition it is generalising.

The pattern is worth naming because it recurs constantly. A quantity is understood in a familiar setting, an arithmetic expression is found that computes it there, and the expression is then promoted to the definition in settings where the original intuition has nothing to hold on to. Radians work this way, and so does the extension of $\sin$ past the angles a triangle can supply. The move is safe exactly when the promoted expression keeps the properties that mattered, which is what Cauchy–Schwarz guarantees here.

Where it stops being about arrows #

Once the angle is defined by a formula rather than measured, nothing requires the vectors to be arrows.

Take two functions on an interval and define their dot product as an integral of their product. Every property survives: it is symmetric, it is linear, dotting a function with itself gives a non-negative number that is zero only for the zero function, and Cauchy–Schwarz still holds. So functions have lengths, and functions have angles, and two functions can be perpendicular.

That is the machinery underneath Fourier series, and it explains the one thing that essay leaves as a recipe. Why can the coefficient of each harmonic be computed independently of all the others? Because $\sin(mx)$ and $\sin(nx)$ are perpendicular whenever $m \ne n$ — their integral-product is exactly zero — so each harmonic casts no shadow on any other, and asking how much of one is present is a projection that the rest cannot contaminate.

The coefficient formula in that essay, an integral of the function against $\sin(nx)$, is this essay’s shadow calculation performed in a space where the axes are waves. The two pictures are the same picture.

What it throws away #

A dot product takes two vectors and returns one number, which is a spectacular amount of discarding — everything about where the two arrows are pointing has been reduced to a single quantity.

What survives is exactly one thing: how much of one lies along the other. Everything perpendicular is gone, and gone completely. Two very different vectors with the same shadow on $b$ give the same answer, and no operation on the answer will ever tell them apart.

That is a virtue when the perpendicular part is genuinely irrelevant. Work in physics is force dotted with displacement precisely because a force at right angles to the motion does no work — carrying a suitcase along a level corridor costs nothing against gravity, and the dot product is the statement that gravity’s shadow on the horizontal is zero. The lost information is lost because the question does not depend on it.

It is the same economy as the determinant, which reduces a whole transformation to an area factor, and Euler’s degree count, which reduces a city to parities. Each keeps one number and answers one question completely.

What the picture cannot show #

The figure is two-dimensional and the concept’s real work is done in many dimensions, where no picture exists. That is the ordinary limitation, and here it is unusually consequential: the two-dimensional case is the one where the angle can be checked against a protractor, and it is therefore the only case where the formula is a computation rather than a definition. Every drawing on this page is of the situation that does not need the definition.

The figure also cannot show that the projection is unique. It draws one perpendicular from the tip of $a$ and lands at one point, which invites the reading that this is the obvious place for it to land. What makes that point special is that it is the nearest point of $b$’s line to the tip of $a$ — a minimisation, and a claim about every other point on the line, none of which is drawn. That characterisation is the one that generalises: in a space of functions there is no perpendicular to drop, and the projection is defined as the nearest point, which is why least-squares fitting is a projection and why the normal equations look the way they do.

The ladder from here #

Rungs above: Cauchy–Schwarz proved rather than asserted, by the quadratic that cannot have two real roots. The cross product, which keeps the perpendicular information the dot product throws away, and why it exists only in three dimensions. Orthogonal bases and Gram–Schmidt. Projection as least squares, with the normal equations derived from the picture. Inner product spaces in general, and what the axioms are for. Fourier coefficients as projections, made precise. The adjoint of a matrix, defined by moving it across a dot product. And the connection to eigenvectors, where symmetric matrices turn out to have perpendicular ones and the dot product is what makes that statement mean anything.

The move #

The general habit here is one of the most reliable in the subject: when a rule looks arbitrary, look for the quantity it is measuring.

Multiply the matching coordinates and add is a rule. How far one thing reaches along another is a quantity. The rule computes the quantity, and once that is known, every property of the rule becomes a property of the quantity — obvious rather than memorised, and portable to settings where the original rule would have made no sense at all.

The reverse move is the failure mode. A rule learned as a rule can be applied and cannot be extended, because there is nothing to ask of it when the setting changes. Nobody would think to define the angle between two functions from multiply the matching coordinates. From how much of one lies along the other, it is the obvious next question.