Matpic
mathematics, in pictures
Concept

Inner product

The operation pairing two vectors or functions into a number, from which length and the angle between them are defined. Only one of the p-norms comes from one, which is why angles and projections exist for ordinary distance and for no other exponent.

Named by 10 essays across 4 fields — each of them below, with the objects they name alongside it.

The dot product as a shadow. Two vectors and the shadow the first casts on the second. The shadow is 2.425 long and b is 4.123, so the dot product is 10.000.

The dot product is a shadow

Multiply the matching coordinates and add them up. That rule explains nothing, and it hides the fact that the answer is a length — how far one arrow reaches along another, times how long that other one is.

algebra · Inner product The target, multiplied by one harmonic at a time. Four panels, each showing the square wave multiplied by a single sine. The areas cancel exactly except against the harmonics the wave actually contains.

Where the coefficients come from

The recipe for a square wave has a four over pi in front and a one over three on the second term, and the essay that built a square wave from sines used them without saying where they came from. They come from multiplying by one harmonic and taking the area.

analysis · Fourier series The unit ball at p = 2.00. The set of points one unit from the origin, when distance is measured by the p-th power sum. At p = 1 it is a diamond, at p = 2 a circle, and as p grows it fills out a square.

Circles that are diamonds and squares

The theorem hands over a formula for distance. Take the formula as a definition, change the exponent in it, and the set of points one unit from the origin stops being round — while remaining, in every sense that matters, a circle.

geometry · Pythagoras The level curve, and the axes the matrix chooses. The curve xᵀAx = 1 for the matrix [2, 0.8, 0.8, 1.4], drawn by solving for the radius at each angle, with the two eigen-directions marked; they cross at a right angle and are the axes of the curve.

Symmetry forces a right angle

A matrix equal to its own reflection across the diagonal always has real stretches and always has perpendicular directions to stretch along. Neither is true of matrices in general, and both follow from one line of algebra.

algebra · Eigenvectors The parabola that proves |a·b| ≤ |a||b|. The squared length of a − t b plotted against t. It is a parabola opening upward whose least value is 7.118; that this is never negative is exactly the Cauchy–Schwarz inequality.

The square that cannot be negative

Cauchy–Schwarz is the load-bearing inequality of the whole subject and is nearly always stated without proof. It is one line away from a fact nobody would argue with, and the line is a parabola with no room to cross the axis.

algebra · Inner product One subtraction clears a direction. Gram–Schmidt on two planar vectors, in 3 panels: the pair as given, the shadow of the second on the first, and the perpendicular pair that is left when the shadow is removed.

One subtraction clears a direction

A basis is a set of directions to measure along, and most bases are awkward because the directions get in each other's way. Removing one shadow at a time turns any basis into one where every coordinate is a shadow and nothing interferes.

algebra · Inner product The nearest point of the plane the columns span. A target vector in space, the plane spanned by two columns, the point of that plane nearest the target, and the residual joining them, which meets the plane at a right angle.

The nearest point of a flat thing

More equations than unknowns almost never have a solution. Asking instead for the point of a plane nearest to where the answer should have been turns an unanswerable question into a shadow, and the shadow is what a line of best fit is.

algebra · Inner product The map moved to the other side of the product. Two panels over the unit curve of the product u₁v₁ + u₂v₂. The left applies A to u and measures it against v, giving 2.328; the right applies the adjoint to v and measures it against u, giving the same number.

Moving a map across a product

The transpose looks like a fact about a matrix: reflect its entries in the diagonal. It is a fact about the inner product. Measure lengths and angles differently and the map that slides to the other side of the product is a different matrix, and a matrix that was symmetric stops being so.

algebra · Inner product Perpendicular to both, and as long as their parallelogram. The vectors (2, 0.4, 0.2) and (0.6, 1.8, 0.3), the parallelogram they span, and their cross product (−0.24, −0.48, 3.36), drawn perpendicular to both with a length of 3.403, which is the parallelogram's area.

A plane disguised as an arrow

The cross product of two arrows is an arrow perpendicular to both, as long as the area of their parallelogram. Reflect everything in a mirror and it points the wrong way, because it was never an arrow: it is a plane, written as the one direction a plane in three dimensions leaves over.

algebra · Inner product A rectangle of the parity table, nearly balanced. A 32 by 32 grid of +1 and −1 entries — the parity of the inner product of row and column — with a 15 by 14 rectangle of chosen rows and columns highlighted; its entries sum to 8.

Two weak sources make one fair bit

No fixed rule can turn every weakly random source into fair bits: for any rule, some source with almost full unpredictability makes it constant. Two independent sources are different. Multiply their bits in pairs, add, and keep the parity — and if the two together carry more unpredictability than the length of one, the result is nearly fair, whatever else the sources do. The reason is that the table of those parities is balanced on every large rectangle.

computation · Pseudorandomness

Named alongside it

The objects these essays reach for when they reach for this one.

OrthogonalityProjectionDot productBasisCauchy schwarzLeast squaresPositive definiteResidualSubspaceApproximationEigenvectorGram schmidt

All concepts