Matpic
mathematics, in pictures
Series

Inner product — the series

8 essays on one idea, from the one that introduces it to the one that assumes the rest.
  1. The dot product as a shadow. Two vectors and the shadow the first casts on the second. The shadow is 2.425 long and b is 4.123, so the dot product is 10.000.

    The dot product is a shadow

    Multiply the matching coordinates and add them up. That rule explains nothing, and it hides the fact that the answer is a length — how far one arrow reaches along another, times how long that other one is.

    part 1 · algebra
  2. The parabola that proves |a·b| ≤ |a||b|. The squared length of a − t b plotted against t. It is a parabola opening upward whose least value is 7.118; that this is never negative is exactly the Cauchy–Schwarz inequality.

    The square that cannot be negative

    Cauchy–Schwarz is the load-bearing inequality of the whole subject and is nearly always stated without proof. It is one line away from a fact nobody would argue with, and the line is a parabola with no room to cross the axis.

    part 2 · algebra
  3. One subtraction clears a direction. Gram–Schmidt on two planar vectors, in 3 panels: the pair as given, the shadow of the second on the first, and the perpendicular pair that is left when the shadow is removed.

    One subtraction clears a direction

    A basis is a set of directions to measure along, and most bases are awkward because the directions get in each other's way. Removing one shadow at a time turns any basis into one where every coordinate is a shadow and nothing interferes.

    part 3 · algebra
  4. The nearest point of the plane the columns span. A target vector in space, the plane spanned by two columns, the point of that plane nearest the target, and the residual joining them, which meets the plane at a right angle.

    The nearest point of a flat thing

    More equations than unknowns almost never have a solution. Asking instead for the point of a plane nearest to where the answer should have been turns an unanswerable question into a shadow, and the shadow is what a line of best fit is.

    part 4 · algebra
  5. The map moved to the other side of the product. Two panels over the unit curve of the product u₁v₁ + u₂v₂. The left applies A to u and measures it against v, giving 2.328; the right applies the adjoint to v and measures it against u, giving the same number.

    Moving a map across a product

    The transpose looks like a fact about a matrix: reflect its entries in the diagonal. It is a fact about the inner product. Measure lengths and angles differently and the map that slides to the other side of the product is a different matrix, and a matrix that was symmetric stops being so.

    part 5 · algebra
  6. Perpendicular to both, and as long as their parallelogram. The vectors (2, 0.4, 0.2) and (0.6, 1.8, 0.3), the parallelogram they span, and their cross product (−0.24, −0.48, 3.36), drawn perpendicular to both with a length of 3.403, which is the parallelogram's area.

    A plane disguised as an arrow

    The cross product of two arrows is an arrow perpendicular to both, as long as the area of their parallelogram. Reflect everything in a mirror and it points the wrong way, because it was never an arrow: it is a plane, written as the one direction a plane in three dimensions leaves over.

    part 6 · algebra
  7. Shrinking beats the nearest point at every true mean. p=3: ratio at |θ|²/p = 0, 1, 5: 0.667, 0.863, 0.976; p=5: ratio at |θ|²/p = 0, 1, 5: 0.400, 0.732, 0.931; p=10: ratio at |θ|²/p = 0, 1, 5: 0.200, 0.621, 0.886; p=25: ratio at |θ|²/p = 0, 1, 5: 0.080, 0.549, 0.855

    The nearest point is not the best guess

    Measure ten unrelated quantities once each, with noise, and report the measurements. Nothing seems better. Yet pulling all ten towards nought by an amount the data choose gives a smaller total error whatever the true values are — in three dimensions or more, and never in one or two. The reason is a right angle: in many dimensions noise is nearly perpendicular to the truth, so a measurement is always too long, and the best correction is the shadow of the truth on it.

    part 7 · algebra
  8. Six lines all at one angle. The six diagonals of the icosahedron: every pair meets at cosine 0.447214, an angle of 63.43 degrees.

    Six lines all at one angle

    Take the six long diagonals of an icosahedron: every pair meets at the same angle. Squaring each line into the matrix that projects onto it removes every sign and leaves a table no vector can kill, so at most d(d + 1)/2 lines can do this in d dimensions. The bound is reached at 2, 3, 7 and 23, and at no other dimension anyone can find.

    part 8 · algebra

All series