Concept

Least squares

The rule that fits a model to data by minimising the total of the squared discrepancies. Squaring is what makes the answer a projection: the fitted values are the point of the subspace the model can reach that is nearest the data, and the leftover meets that subspace at a right angle.

Named by 2 essays across one field — each of them below, with the objects they name alongside it.

Also named here as linear independence, orthonormal basis, residual — the same set of essays touches all of them, so they are one junction rather than several.

One subtraction clears a direction

A basis is a set of directions to measure along, and most bases are awkward because the directions get in each other's way. Removing one shadow at a time turns any basis into one where every coordinate is a shadow and nothing interferes.

algebra · Inner product

The nearest point of the plane the columns span. A target vector in space, the plane spanned by two columns, the point of that plane nearest the target, and the residual joining them, which meets the plane at a right angle.

The nearest point of a flat thing

More equations than unknowns almost never have a solution. Asking instead for the point of a plane nearest to where the answer should have been turns an unanswerable question into a shadow, and the shadow is what a line of best fit is.

algebra · Inner product

Named alongside it

The objects these essays reach for when they reach for this one.

Approximation Basis Dot product Gram schmidt Inner product Linear independence Orthogonality Orthonormal basis Projection Residual Subspace Cauchy schwarz

All concepts