A square wave built entirely out of round ones

A sine wave is the smoothest repeating shape there is. It has no corners, no flat stretches, no sudden anything — it is a circle seen from the side, and circular motion is not capable of jerking.

A square wave is the opposite in every respect: perfectly flat, then instantaneously somewhere else, then perfectly flat again.

Fourier’s claim, made in 1807 and disbelieved by most of the people who first read it, is that the second is a sum of the first. Lagrange, on the review committee, objected; the paper was not published for fifteen years. The objection was not stupidity — it was that nobody yet had a definition of what it means for a sum of infinitely many things to equal something, and Fourier was cheerfully asserting one.

The recipe #

Only odd multiples of the base frequency are used, and each is weighted by the reciprocal of its multiple. Both of those facts have reasons, and both reasons are visible later in this essay rather than assumed here:

$$\frac{4}{\pi}\left(\sin x + \frac{\sin 3x}{3} + \frac{\sin 5x}{5} + \frac{\sin 7x}{7} + \cdots\right)$$

That is the whole construction. The first term is a plain sine wave of the right period. It is a bad square wave — too round in the middle, too gentle at the crossings — but it is right in the broad strokes.

The third harmonic, at a third of the amplitude, is added in the phase that pushes the middle of each hump down and the edges up. It flattens the top and steepens the crossing simultaneously, which is exactly what is needed.

Each new harmonic is smaller and faster than the last. The smallness means the sum keeps converging; the fastness means each new term can correct finer features than the one before. Coarse shape first, then edges, then corners. It is the same division of labour as a coarse pass followed by finishing passes.

Note where $\pi$ has appeared. There is no circle in a square wave, and there is no circle in the statement of the problem — and yet the overall scale factor is $4/\pi$. It is there because the coefficients come from integrating over a full period, and a full period is a lap of a circle. $\pi$ turns up uninvited so often that its appearance is nearly a signature: something in the problem is going round, even when nothing visible is.

Why odd harmonics only #

The square wave chosen here has a symmetry: shift it by half a period and it turns upside down. Formally, $f(x + \pi) = -f(x)$.

The even harmonics do not have that symmetry — $\sin 2x$ shifted by $\pi$ comes back as itself, not as its negative. So they cannot possibly contribute, and their coefficients come out as exactly zero without anyone having to arrange it.

This is characteristic of Fourier analysis: symmetries in the target function turn into whole families of coefficients vanishing. A function symmetric about the origin needs no cosines. A function symmetric about the vertical axis needs no sines. The decomposition respects whatever structure the function already had, which is one of the reasons it is useful rather than merely true.

The division of labour also explains why truncating the series is a reasonable thing to do. The early terms carry the shape and the later ones carry the detail, so stopping early loses sharpness rather than correctness — which is precisely the property that makes lossy compression possible. Discarding the high harmonics of an image or a sound file is discarding the fine detail and keeping the shape, which is a bargain most viewers and listeners accept without noticing they have made it.

The overshoot that will not go away #

Now the part that is genuinely surprising, and that caused an argument.

Look closely at the corners in any of the pictures above. Just before each jump the sum shoots past the target, then rings briefly before settling. Adding more terms makes the ringing narrower and pushes it closer to the jump — so the natural expectation is that it shrinks away in the limit.

It does not.

The first spike always overshoots by approximately $8.95\%$ of the jump, no matter how many terms are used. Sixty-one terms, six thousand, six million: the same 9%.

What does happen in the limit is that the overshoot moves. It gets squeezed into an ever-narrower window right at the discontinuity, so for any fixed point away from the jump, the sum converges to the right value. The series converges pointwise everywhere. It simply does not converge uniformly, and the difference between those two notions is precisely the room in which a permanent 9% error can hide.

This is the Gibbs phenomenon. It was first noticed by Henry Wilbraham in 1848, forgotten, and rediscovered around 1898 during a dispute about whether a mechanical harmonic analyser built by Albert Michelson was malfunctioning. Michelson’s machine summed 80 harmonics mechanically and drew the result; on square-wave input it drew the spikes. Michelson suspected a fault in the apparatus. J. Willard Gibbs wrote to Nature to explain that the machine was working perfectly and the spikes were real.

The episode is a good advertisement for taking instruments seriously. The overshoot was in the mathematics all along, and it took a physical device drawing an unexpected picture for anyone to look again.

It also has practical consequences that are not merely curiosities. Sharp transitions in digital images and audio are reconstructed from finitely many frequency components, and Gibbs overshoot shows up as ringing artefacts — the faint halos around hard edges in an over-compressed JPEG are this phenomenon, visible on a screen.

The nine percent is a specific number #

“About 9%” is the figure usually quoted, and rounding it hides where it comes from. The overshoot has a closed form.

The partial sum near a jump is, in the limit, a rescaled copy of the sine integral $\mathrm{Si}(x) = \int_0^x \tfrac{\sin t}{t}\,dt$, and the height of its first peak is $\mathrm{Si}(\pi) = 1.851937\ldots$ against a limiting value of $\pi/2 = 1.570796\ldots$. The ratio of the two, minus one, is the overshoot:

$$\frac{2}{\pi}\,\mathrm{Si}(\pi) - 1 = 0.0894898\ldots$$

So the number is $8.94898\%$ of the jump — the Wilbraham–Gibbs constant, and one of the more obscure constants to have a name.

The important part is what it does not depend on. Not on the square wave: the same constant governs the overshoot at a jump in a sawtooth, in a rectified sine, in any function with a jump discontinuity whatever.

Nor on the size of the jump, since the overshoot is a fixed fraction of it. Nor on anything the function does elsewhere. What the partial sums are actually doing near a discontinuity is reproducing one universal shape, scaled to the jump and compressed as the term count rises, and every jump in every function gets the same shape.

That is worth registering as a claim about the method rather than about the example. A finite Fourier sum does not know what function it is approximating in the neighbourhood of a jump; it knows only that there is a jump, and it responds identically in every case. The overshoot is not the series struggling with a difficult target. It is the series doing the one thing it does at a discontinuity, which happens to be wrong by a fixed amount.

That last comparison is the useful generalisation. How fast a Fourier series converges is a direct readout of how smooth the target is. A jump discontinuity gives coefficients falling like $1/m$ and permanent Gibbs overshoot; a kink gives $1/m^2$ and no overshoot; an infinitely smooth function gives coefficients that fall faster than any power. The spectrum knows the smoothness, which is one of the reasons the frequency picture is worth having.

The overshoot belongs to the truncation #

A qualification belongs next to all of that, because “the nine percent never goes away” is true of one particular way of adding the terms up and false of the obvious alternative.

Take the partial sums $S_1, S_2, \ldots, S_N$ and average them. That is the Fejér mean, and Fejér proved in 1900 that for any continuous periodic function these averages converge uniformly — no exceptions and no conditions beyond continuity, which is precisely the guarantee the partial sums themselves fail to give. At a jump they do something better again: they do not overshoot at all. Summing the square wave with 61 harmonics, the ordinary partial sum peaks at $1.17907$ against a target of $1$, which is the $8.95\%$ of the jump described above; the Fejér mean of the same 61 terms peaks at $0.98973$ — below the target, climbing towards it. At 201 terms it reaches $0.99685$. It arrives from underneath and never rings.

The mechanism is worth a sentence, because it explains an entire family of fixes. Averaging the partial sums is the same thing as multiplying the $m$-th coefficient by $1 - m/(N+1)$: a taper that fades the high harmonics out instead of cutting them off. A truncated series is the full series multiplied by a rectangular window, and it is the rectangle’s own sharp edge that produces the ringing. Round the window off and the ringing goes with it.

Signal processing does exactly this and calls it windowing. The Lanczos factors, which multiply term $m$ by $\sin\!\big(\pi m/(N{+}1)\big) \big/ \big(\pi m/(N{+}1)\big)$, cut the square wave’s overshoot from $8.95\%$ of the jump to $1.19\%$, and hold it there at any term count. Fejér’s taper removes the overshoot entirely at the price of a blurrier edge; the sharper tapers keep the edge and leave a little ringing. Every window function in a signal-processing library is a chosen position on that trade.

So the honest form of the Gibbs statement is narrower than the usual one and more interesting for it. It is not that a jump cannot be approximated well by sinusoids. It is that truncating the Fourier series — discarding every harmonic past the $N$-th and keeping the rest at full weight — overshoots by $8.95\%$ of the jump however large $N$ becomes. The defect belongs to the truncation, and a different summation of the very same coefficients does not have it.

What the picture cannot show #

The figures show partial sums, and every one of them is a finite sum of smooth functions — therefore itself perfectly smooth, with no jump anywhere. The square wave has a jump. So no picture in this essay contains the object under discussion; they contain a sequence of things approaching it, and the thing itself is not in the sequence.

This is not pedantry. It is precisely why the Gibbs overshoot is possible: a property can hold for every term of a sequence and fail in the limit, or fail for every term and hold in the limit. Pointwise convergence, uniform convergence, and convergence in the mean-square sense all differ here, and the pictures cannot distinguish them — they look identical. The distinctions took nineteenth-century analysis decades to sort out, and they were sorted out with inequalities, not diagrams.

The ladder from here #

Later rungs: computing the coefficients, as an integral that projects onto each harmonic. Orthogonality, and why that projection works — the dot-product-as-shadow idea applied to functions. The complex form, which halves the bookkeeping once multiplication is rotation. Parseval’s identity, where energy in time equals energy in frequency. The Fourier transform, when the period goes to infinity. The FFT, and how $n^2$ became $n\log n$. Heat diffusion, the problem Fourier was actually solving. And the uncertainty principle, which is a theorem about Fourier pairs before it is anything about physics.

What is actually being claimed #

The strong version of Fourier’s idea is that essentially any periodic function can be written this way. The precise conditions took a century to pin down and are genuinely delicate — the same century, and largely the same people, that made the integral respectable — Dirichlet gave the first satisfactory sufficient conditions in 1829, and the question of exactly which functions have convergent Fourier series drove a great deal of nineteenth and twentieth century analysis. Cantor invented set theory while working on a problem in this area — a reminder that the questions which look like housekeeping are sometimes the ones that open a field.

It is worth being blunt about how that century turned out, because the answer is not the reassuring one. The strong version of Fourier’s claim — that a continuous periodic function is the sum of its Fourier series — is false.

Paul du Bois-Reymond constructed a counterexample in 1873: a function that is continuous everywhere, with no jump and no corner anywhere at all, whose Fourier series diverges at a point. Not converges slowly, not converges to the wrong value — diverges. Continuity, which is exactly the property a reader would expect to be sufficient, is not sufficient.

It gets worse before it gets better. Kolmogorov produced in 1926 an integrable function whose Fourier series diverges at every point, everywhere, without exception. Nothing about the series recovers the function at any point of its domain.

The rescue, when it came, took another forty years. Lennart Carleson proved in 1966 that if a function is square-integrable — if the total energy is finite, which every physical signal satisfies — then its Fourier series converges at almost every point. The exceptional set has measure zero. This had been an open question since 1913, was widely expected to be false, and Carleson’s proof is famously difficult; Hunt extended it the following year to a slightly larger class.

So the honest statement of the theory is layered. For any function with finite energy, the series converges in the mean-square sense — the total squared error goes to zero — and that is easy, and it is the version applications rely on. Pointwise convergence almost everywhere is true and was a fifty-three-year problem. Pointwise convergence everywhere is false, and continuity does not buy it.

What all of this says about the pictures is uncomfortable and worth sitting with. Every figure in this essay shows a partial sum tracking a target more and more closely, which is a perfectly accurate depiction of the well-behaved case and a completely misleading one about the general theory. Nothing looks different, in a drawing, about a function whose series diverges. Convergence is not a visible property.

But the working version is robust enough to have reorganised applied mathematics around it: any repeating signal can be described either by its shape over time or by how much of each frequency it contains, and those two descriptions carry the same information. Neither is more fundamental. Some questions are much easier in one than the other.

That is the deep content. Not that sines can build a square, but that a function and its spectrum are the same object, and problems that are intractable in one representation sometimes dissolve in the other. Differentiation becomes multiplication. Convolution becomes multiplication. Heat diffusion, which is what Fourier was actually working on, becomes a set of independent exponential decays, one per frequency, instead of a coupled partial differential equation.

The square wave is the demonstration piece. The idea is that there are two ways to look at everything, and it is worth knowing both.

That is a pattern worth naming, because the site keeps running into it. Four curves turn out to be one cone. Two constructions turn out to be one recursion. Here a signal and its spectrum turn out to be one object. In each case the second description is not a translation of the first — it is the same thing, and questions that were opaque in one view become trivial in the other. Recognising when a problem is being asked in the wrong representation is most of what makes a problem tractable.