Where the coefficients come from

The square wave is built from the odd harmonics, each weighted by the reciprocal of its number, with a factor of $4/\pi$ across the whole sum. Those numbers were handed over as a recipe. Here is where they come from, and the answer is a picture rather than a calculation.

To find out how much of a given harmonic is in a function, multiply the function by that harmonic and take the area under the result. That is the entire procedure. If the harmonic is present, the product spends more time positive than negative and the area survives; if it is absent, the positive and negative parts cancel exactly and the area is zero.

Why the cancellation is exact #

The cancellation is not approximate and not a matter of luck. It is a property of the sines themselves, and the statement is

$$\int_{-\pi}^{\pi} \sin(mx)\sin(kx)\,dx = \begin{cases}\pi & m = k\\ 0 & m \ne k.\end{cases}$$

Two different sines, multiplied and integrated over a full period, give exactly nothing. Any sine multiplied by itself gives $\pi$, since the square is never negative and averages a half.

That property has a name — the harmonics are orthogonal — and the name is borrowed on purpose.

The dot product is a shadow, and the integral above is the same operation with a function in place of a vector. Where a vector has finitely many coordinates to multiply and add, a function has a continuum of them, and the sum becomes an integral. Everything else transfers unchanged: there is a notion of length, a notion of angle, a notion of perpendicular, and a component along each direction.

The harmonics are the perpendicular directions. Extracting the coefficient of one of them is taking a shadow, and the reason the extraction does not disturb the others is that they are all at right angles to it.

Running the procedure #

For the square wave $f$, which is $+1$ on $(0,\pi)$ and $-1$ on $(-\pi,0)$, the projection onto $\sin(mx)$ is

$$\int_{-\pi}^{\pi} f(x)\sin(mx)\,dx = 2\int_{0}^{\pi}\sin(mx)\,dx = \frac{2\bigl(1-\cos m\pi\bigr)}{m},$$

which is $4/m$ when $m$ is odd and $0$ when $m$ is even. Dividing by the $\pi$ that a sine gives against itself:

$$b_m = \frac{4}{\pi m} \quad (m \text{ odd}), \qquad b_m = 0 \quad (m \text{ even}).$$

There is the recipe. The $4/\pi$ and the $1/m$ are not chosen and not fitted; they are what the areas come out at. The vanishing of the even terms, which rung one explained by a symmetry argument, is here simply a computation that returns zero.

The figures do not take that arithmetic on trust. Each panel’s area is computed numerically, on a grid split at the discontinuities so that no interpolation runs across a jump, and then compared against the amplitude of the harmonic the rest of this family draws — so the picture, the arithmetic and the curves in the neighbouring essay are checked against one another rather than against a formula written out twice.

What the coefficients then say #

Once every coefficient is available, the whole function can be replaced by the list of them, and the list is worth looking at as an object in its own right.

The rate at which those bars fall is a measurement of how smooth the original function is, and the correspondence is exact enough to be used as a diagnostic.

A jump gives $1/m$. A corner — continuous, with a discontinuous slope — gives $1/m^2$. A function with $p$ continuous derivatives gives $1/m^{p+1}$, and an infinitely smooth periodic function gives coefficients that fall faster than any power at all, usually exponentially.

The mechanism is one integration by parts per derivative: each one moves a factor of $m$ into the denominator and can be performed exactly as often as the function has derivatives to give up. Where that supply runs out, the decay stops improving.

The energy, split among the harmonics #

Orthogonality has one more consequence, and it is the one that makes the coefficients feel like a description of the function rather than a set of numbers extracted from it.

Square the function and take the area. Because the harmonics do not interfere, every cross term integrates to nothing, and what is left is the sum of the squares of the coefficients:

$$\frac{1}{\pi}\int_{-\pi}^{\pi} f(x)^2\,dx = \sum_{m} b_m^2.$$

This is Parseval’s identity, and it is Pythagoras in a space with one dimension per harmonic — the square of the length is the sum of the squares of the components, which is the theorem this collection meets in three dimensions with the sum continued forever.

For the square wave, whose square is $1$ everywhere, the left side is $2$. The right side is $\sum_{m\ \text{odd}} (4/\pi m)^2$, so

$$\frac{16}{\pi^2}\left(1 + \frac19 + \frac1{25} + \cdots\right) = 2 \quad\Longrightarrow\quad 1 + \frac19 + \frac1{25} + \cdots = \frac{\pi^2}{8}.$$

A sum of reciprocal odd squares, evaluated exactly, obtained by squaring a square wave. Euler’s more famous $\pi^2/6$ for the sum over all the squares follows in two more lines, and the whole computation is a by-product of the projection machinery rather than an attack on the series.

That is a fair advertisement for the method: the coefficients are not merely a way to rebuild the function, they are a second description of it, and quantities that are awkward in one description are sometimes trivial in the other.

What it costs #

The projection integral is exact, elegant, and expensive.

Computing one coefficient requires integrating the function against one harmonic over a whole period. Computing $N$ coefficients from $N$ sampled values costs $N$ multiplications each, so $N^2$ in total — which for a second of audio at forty-four thousand samples is two billion operations for one second of sound.

That cost is the reason the discrete transform was a specialist tool for a century and a half and then suddenly was not: the fast Fourier transform reorganises the same $N^2$ operations into $N\log N$ by noticing that the same products are being computed repeatedly, which takes the two billion to about seven hundred thousand. The mathematics is unchanged. Only the bookkeeping is different, and the difference is the whole of digital signal processing.

The second cost is subtler and is the one this rung has to be honest about. The procedure assumes the integral exists. For a function with a jump it does; for functions that are merely integrable it does; but the series built from those coefficients need not converge to the function, and as rung one records, continuity is not enough to guarantee it. Orthogonality delivers the coefficients unconditionally and says nothing about what their sum does.

The best approximation, without trying #

There is a second dividend, and it explains why truncating the series is a reasonable thing to do rather than a compromise.

Take the first $N$ harmonics and ask which combination of them comes closest to the target, closest meaning the smallest total squared error. That is a genuine minimisation problem with $N$ unknowns, and in a general basis it would have to be solved.

In an orthogonal basis the answer is the coefficients already computed. Nothing is gained by re-fitting: the partial sum of the Fourier series is the best approximation available from those harmonics, and it is best by the same argument that makes a shadow the nearest point of a line to a point off it. The error is what remains after the projection, and it is perpendicular to everything kept.

That has a consequence worth stating because it is easy to expect the opposite. Adding a term never requires the earlier terms to be revised. A fit to a polynomial of degree five and a fit to a polynomial of degree six share none of their coefficients in general; a Fourier series to five terms and one to six share all five. The expansion is stable under extension, and that is why it can be truncated anywhere and why coefficients can be stored, transmitted and discarded independently — which is the basis of every transform-coded image and audio format there is.

Where it needs a condition #

Orthogonality is a statement about an interval, and it is false on the wrong one.

Over $[-\pi, \pi]$ the harmonics are orthogonal. Over $[0, 1]$ they are not — $\sin x$ and $\sin 2x$ have a non-zero product integral there — and any attempt to extract coefficients by projection over an interval that is not a whole period returns numbers that are contaminated by every other harmonic.

That failure has a practical name. Sampling a signal over a stretch that is not a whole number of periods and then transforming it produces energy at frequencies that are not in the signal, because the projections no longer cancel; it is called spectral leakage, and it is the reason windowing functions exist, which is the same family of tapers that fixes the Gibbs overshoot approaching the problem from the other side.

The deeper condition is completeness. Orthogonality says the harmonics do not overlap; it does not say they span — that every function is a combination of them and nothing has been left out. A basis that is orthogonal but incomplete would extract coefficients perfectly and reconstruct the wrong function, with the missing part invisible to every projection. That the trigonometric system is complete is a genuine theorem and a much harder one than orthogonality, and the whole method depends on it.

Who worked out the mechanism #

Fourier had the coefficients in 1807 and did not have this argument for them. His derivation ran through an infinite system of linear equations in infinitely many unknowns, solved by a limiting process that his examiners — Lagrange, Laplace and Legendre — found unconvincing, and they were right to: the manipulation is not justified by anything Fourier supplied.

The projection argument, which is a page long and needs nothing but the integral of a product of sines, appears to have been available to him and was used explicitly by Euler in the 1770s for a related expansion. Why Fourier took the harder road is not settled; the usual explanation is that he was thinking of the coefficients as the solution to a physical problem rather than as components in a space of functions, and the space-of-functions picture did not exist yet.

That picture was assembled over the following century, and the vocabulary now used to state the argument in three lines — inner product, orthogonality, projection, completeness, basis — was invented in large part because of Fourier series, mostly by Hilbert and Schmidt in the 1900s. So the clean derivation is not the one that was overlooked; it is the one that became sayable afterwards, using words the problem itself forced into existence.

The order is worth registering, because it recurs. A calculation is done first by whatever means work, the calculation turns out to matter, and the apparatus that makes it obvious is then built to explain it. The apparatus usually outlives the problem: Hilbert space was built for integral equations and Fourier series and is now the language of quantum mechanics, which had not been thought of.

What the picture cannot show #

The shaded areas in the first figure are the argument, and they show only that the areas look equal on the two sides for the even harmonics. Equality by eye is worth very little: a lobe two percent taller than its partner would be indistinguishable in the drawing and would make the coefficient two percent of the first harmonic’s, which is a visible amount of extra wave.

What closes that gap is the checking rather than the drawing. Each panel’s area is integrated numerically and asserted against the value it claims, and a figure whose lobes did not cancel would refuse to be drawn. The picture shows the mechanism; the assertion supplies the precision, and it is worth saying which is doing which.

The other invisible thing is the infinite-dimensionality. The analogy to shadows is a picture of two vectors in a plane, and the actual space has one dimension per harmonic. Nothing about the finite picture prepares a reader for the properties that fail there — that a sum of components need not converge, that a set of perpendicular directions can be infinite and still miss something — and those failures are exactly where the subject’s difficulty lives.

A last practical note about the procedure, since it is often met as a formula and rarely as a measurement. Nothing in it requires the function to be known in closed form. The projection is an integral of the function against a sine, so a function known only by samples — a recorded sound, a tide gauge, a column of numbers — is projected by multiplying the samples by the sine’s values and adding. That is what makes the method an instrument rather than an identity, and it is why the first serious users of Fourier analysis were tide predictors and seismologists rather than analysts.

The ladder from here #

Below: the construction itself, and, in another field, the dot product this whole rung is an analogy to. Above: what happens when the period is allowed to grow without bound, where the sum over harmonics becomes an integral over frequencies, and the problem the whole apparatus was invented for, where each harmonic decays at its own rate and the coefficients earn their keep.

Sideways: the same projection with different orthogonal families gives Legendre, Chebyshev and Hermite expansions, each adapted to a different interval and weight; and the same picture of a shadow is what least squares is doing when it fits a line to data.

Choosing the directions to look along #

The lasting point is what orthogonality buys, and it is not elegance.

In a general basis, finding the coefficients of a function means solving a system: every coefficient depends on every other, and changing one changes them all. In an orthogonal basis the system is diagonal. Each coefficient is found by a single integral that ignores the rest, adding a term never disturbs the terms already found, and truncating the series leaves the best approximation available from the terms kept.

That is why the whole subject is organised around orthogonal families rather than around convenient-looking ones. The property being chased is not that the pieces are simple — sines are not especially simple — but that they do not interfere. It is the same reason eigenvectors are worth finding: a set of directions along which a hard operation acts independently turns one coupled problem into many separate ones, and that decoupling is most of what applied mathematics does with a hard problem.