Close in height and nowhere close in slope

The two essays before this one were about what uniform convergence buys. A limit that forgets to be continuous showed that it carries continuity: a uniform limit of continuous functions is continuous, because a function within $\varepsilon$ of a continuous one everywhere cannot jump by more than $2\varepsilon$ plus what the continuous one does. Settling in order settles everywhere at once found circumstances in which it comes free. And integrals pass to uniform limits too: if every member is within $\varepsilon$ of the limit on an interval of length $L$, the integrals differ by at most $\varepsilon L$.

So it is natural to expect the next property along to follow. If smooth functions converge uniformly, surely the limit is smooth, and its slope is the limit of their slopes. It is not, and the failure is total: the slopes can do anything at all. This essay draws four ways it fails, states what has to be added to make it work, and puts a number on the asymmetry — integration and differentiation sit on opposite sides of the function, and one shrinks what the other inflates.

A wiggle that shrinks in height and grows in slope #

The simplest example fits in one formula. Let

$$f_n(x) = \frac{\sin(n^2 x)}{n}.$$

The sine never exceeds 1, so $|f_n(x)| \le 1/n$ everywhere: the largest gap from the zero function is exactly $1/n$, and the sequence converges to $0$ uniformly, on the whole line at once. The limit is as smooth as a function can be, and its slope is $0$ at every point.

Now differentiate. By the chain rule

$$f_n'(x) = n\cos(n^2 x),$$

which swings between $-n$ and $n$, faster and faster. At $x = 0$ it equals $n$ for every member, so the slopes at that single point run off to infinity while the heights at that point are all exactly $0$. At most other points the slopes oscillate without settling on anything. The limit’s slope, $0$, is the limit of nothing.

The mechanism is in the two factors of $n$. Dividing by $n$ shrinks the height; putting $n^2$ inside the sine squeezes the oscillation, so that each wiggle is $n^2$ times narrower. A slope is height divided by width, and the height has gone down by $n$ while the width has gone down by $n^2$. The wiggles get smaller and steeper at once, and uniform convergence, which only measures height, cannot see the steepness at all.

Smooth curves converging to a corner #

The first example has a perfectly good limit whose slope the members fail to approach. Worse is possible: smooth members whose uniform limit has no slope at some point.

The functions $g_n(x) = \sqrt{x^2 + 1/n}$ are hyperbolas, differentiable everywhere, with a rounded bottom of width about $1/\sqrt n$. They lie above $|x|$, and the largest gap is at $x = 0$, where it is $1/\sqrt n$; so they converge uniformly to $|x|$, which has a corner at $0$. Their slopes are

$$g_n'(x) = \frac{x}{\sqrt{x^2 + 1/n}},$$

continuous for every $n$, and at each fixed $x$ they converge to $-1$, $0$ or $+1$ according to the sign of $x$. The pointwise limit of the slopes is a step, and a slope can swing but never jump proved that a step is never the slope of anything — so no function, smooth or otherwise, has these limiting slopes everywhere. The corner is exactly where the slopes’ convergence fails to be uniform: near $0$ there are points with slope $\tfrac12$ for every $n$, at $x = \pm 1/\sqrt{3n}$, and the step’s value there is $\pm 1$.

The essay on Dini’s theorem used this same corner from the other direction. Weierstrass’s square-root iteration produced polynomials converging uniformly to $|x|$, which is the opening move in approximating every continuous function by polynomials. The approximation of a corner by smooth functions is the point, and it would be impossible if uniform convergence carried slopes.

A Fourier series and the series of its slopes #

The same failure appears in the setting where functions are most often built from pieces: Fourier series. The sawtooth function, equal to $x$ on $(-\pi, \pi)$ and repeated, has the series

$$x = 2\Big(\sin x - \frac{\sin 2x}{2} + \frac{\sin 3x}{3} - \cdots\Big),$$

which converges to $x$ at every point strictly inside the interval, and uniformly on every closed piece of it — the slow convergence near the jump at $\pm\pi$ being the corner that goes first, with the Gibbs overshoot of a square wave built from round ones.

Differentiate term by term and the series becomes

$$2\big(\cos x - \cos 2x + \cos 3x - \cdots\big).$$

The derivative of $x$ is $1$, so if differentiation could be done inside the sum this series would add up to $1$. It does not add up to anything. At $x = 0$ its partial sums are $2, 0, 2, 0, \dots$, alternating forever. Near $x = \pm\pi$ every term is close to $-2$ and the partial sums run down like $-2N$. Elsewhere they oscillate with an amplitude that does not shrink. A series that converges can have a series of derivatives that converges nowhere.

The reason is the same as in the first example. The $k$-th term $\sin(kx)/k$ is small because of the $1/k$; differentiating removes the $1/k$ and leaves $\cos kx$, of full size. Each differentiation multiplies the $k$-th coefficient of a Fourier series by $k$, so a series whose coefficients decay like $1/k$ — enough to converge, barely — becomes one whose coefficients do not decay at all.

Heights settling, slopes running away #

Push the construction to its limit and the result is the function with a corner at every point. Take

$$W(x) = \sum_{k=0}^{\infty} \Big(\tfrac12\Big)^k \cos(3^k \pi x).$$

The $k$-th term has height $2^{-k}$, so the series converges uniformly — the tail after $N$ terms is at most $2^{-N}$ everywhere — and $W$ is continuous. But the $k$-th term has slope up to $\pi\,(3/2)^k$, and the slopes of the partial sums grow without bound.

On the logarithmic scale the two lines are mirror images, and that symmetry is the whole construction: each new term shrinks the height gap by a factor of 2 and multiplies the steepest slope by about $1.5$, because $3 \times \tfrac12 = 1.5 > 1$. The measured steepest slope is $32.3$ with five terms and $414$ with eleven, against a bound of $\pi \sum (1.5)^k$ that the figure checks. Weierstrass’s theorem, and Hardy’s sharpening of it, says that when $ab \ge 1$ the limit has a slope nowhere at all. The uniform convergence is perfect, and it delivers a continuous function with no derivative at any point.

A mistake with a history #

The sawtooth series has a history that makes it the right example. In his Cours d’analyse of 1821 Augustin-Louis Cauchy stated that a convergent series of continuous functions has a continuous sum. Niels Henrik Abel pointed out in 1826 that the series $\sin x - \tfrac12\sin 2x + \tfrac13\sin 3x - \cdots$ is a counterexample: every term is continuous, and the sum is $x/2$ inside the interval and jumps at $\pm\pi$. Cauchy’s statement needed a hypothesis nobody had yet named, and the name, uniform convergence, came out of the next thirty years of repairing it — in work by Seidel, Stokes and Weierstrass, who made it the centre of his Berlin lectures.

Once the hypothesis was in place, continuity and integration were safe, and the same habit of thought assumed that differentiation was too. Through most of the nineteenth century series were differentiated term by term whenever it was convenient, and the results were usually right, because the series that arose in physics usually had coefficients decaying fast enough. The example that ended the habit was Weierstrass’s own, presented to the Berlin Academy in 1872: a series converging uniformly, with every partial sum smooth, whose sum has no derivative at any point. Charles Hermite’s often-quoted reaction, that he turned away “with fright and horror from this lamentable plague of continuous functions which have no derivatives”, is a fair measure of how firmly the opposite had been believed.

The figures above are that history compressed. Abel’s series is the left panel of the Fourier figure; the habit it did not cure is the right panel; and Weierstrass’s function is the pair of diverging lines in the figure after it.

What does carry a slope #

The repair is to ask for convergence of the right thing. The theorem is: suppose the $f_n$ are differentiable on an interval, their derivatives $f_n'$ converge uniformly to some $g$, and the values $f_n(x_0)$ converge at one point $x_0$. Then the $f_n$ converge uniformly to a function $f$, and $f$ is differentiable with $f' = g$.

The proof goes through the integral, which is the operation that does respect uniform limits. By the fundamental theorem of calculus — area is the undoing of slope —

$$f_n(x) = f_n(x_0) + \int_{x_0}^{x} f_n'(t)\,dt.$$

On the right the first term converges by assumption and the integral converges, uniformly in $x$, because its integrand does; so the left side converges uniformly to $f(x_0) + \int_{x_0}^{x} g$, and that expression is differentiable with derivative $g$, by the same fundamental theorem. Slopes pass to the limit only by being integrated first. The hypothesis is about derivatives because the conclusion is about derivatives, and convergence of the functions alone is the wrong kind of information.

Each of the four counterexamples fails exactly this hypothesis. The slopes $n\cos(n^2 x)$ do not converge at all. The slopes $x/\sqrt{x^2 + 1/n}$ converge, but not uniformly. The differentiated Fourier series does not converge. The Weierstrass partial sums’ slopes grow without bound. And the theorem cannot be weakened to pointwise convergence of the slopes, as the corner shows: there the slopes converge at every point, to a step, and the limit has no slope at $0$.

Integral, function and slope on one scale #

The asymmetry can be measured on the first example. Take $f_n(x) = \sin(n^2 x)/n$ and set its integral, the function itself and its slope side by side.

The integral from $0$ is $(1 - \cos n^2 x)/n^3$, whose largest value is $2/n^3$; the function’s is $1/n$; the slope’s is $n$. On the logarithmic plot these are three straight lines with gradients $-3$, $-1$ and $+1$, two powers of $n$ apart at each step. Each integration divides by the frequency and each differentiation multiplies by it. A wiggle of frequency $n^2$ and height $1/n$ has an integral of height $1/n^3$ and a slope of height $n$; uniform convergence, which measures the middle line, says nothing about the line above it and implies everything about the line below.

This is why the analysis of differential equations is so often done backwards. A differential equation is turned into an integral equation, as in the existence proof that the subsequence that has to exist supported, because approximate solutions converge in the integral form, where wiggles die, and the derivative of the limit is then read off afterwards. The same reason explains why numerical differentiation of noisy data is notoriously unstable and numerical integration is not: noise is high-frequency wiggle, and differentiation is the operation that amplifies high frequencies.

Differentiation has no bound, and integration does #

There is a way of saying all this in one sentence about operations on functions. Measure a continuous function on an interval by its largest absolute value, its sup norm $\|f\|$, which is the quantity uniform convergence makes small. Integration from a fixed point is then a bounded operation: $\big\|\int_{x_0}^x f\big\| \le L\,\|f\|$ on an interval of length $L$, so a small function always has a small integral. Differentiation has no such bound. The ratio $\|f_n'\|/\|f_n\|$ for $f_n = \sin(n^2x)/n$ is $n^2$, and no constant can sit on the right-hand side of an inequality $\|f'\| \le C\|f\|$.

Unbounded operations are not hopeless; they are merely delicate. Differentiation has a weaker property that is exactly the theorem of the previous section: it is closed. If $f_n \to f$ and $f_n' \to g$, both uniformly, then $f' = g$. A closed operation cannot send a convergent sequence to a sequence converging to the wrong answer; it can only fail to send it anywhere at all, as every counterexample above does. That distinction, between operations that cannot be trusted and operations that can be trusted when they converge, is the starting point of the theory of unbounded operators, and differentiation is its first and most important example — the reason quantum mechanics, whose momentum is a derivative, needs that theory to be stated precisely.

How fast a wiggle has to be #

The examples suggest a rule of thumb that can be made precise. A sequence that converges uniformly fails to carry slopes when its members carry oscillation whose frequency grows faster than its amplitude shrinks. For $\sin(n^2x)/n$ the amplitude shrinks like $1/n$ and the frequency grows like $n^2$, and the slopes behave like the ratio, $n$. Had the frequency grown like $n$ — the sequence $\sin(nx)/n$ — the slopes would be $\cos nx$, bounded but not converging; with frequency $\sqrt n$, the sequence $\sin(\sqrt n\,x)/n$, the slopes would be at most $1/\sqrt n$ and would converge uniformly to $0$, and the theorem would apply.

So there is a borderline, and it is where amplitude times frequency stays of fixed size. The same borderline separates the Fourier series whose term-by-term derivatives behave from those whose do not: coefficients $c_k$ times frequencies $k$ must still decay, so that $\sum k|c_k|$ converges, before the differentiated series converges uniformly. And it is the same borderline Weierstrass’s function sits beyond, with height ratio $a$ and frequency ratio $b$ and the product $ab$ deciding whether slopes survive. Three examples that looked unrelated are one inequality between how fast something shrinks and how fast it oscillates.

What the measurements show and do not show #

The steepest slopes in the Weierstrass figure are measured on a grid of up to two and a half million points, and a grid maximum can only fall short of the true one; the figure checks every measured value against the bound $\pi\sum(1.5)^k$, which no measurement may exceed. The growth it shows is therefore real growth of a lower bound, and the theorem that the limit has no derivative anywhere is Weierstrass’s and Hardy’s, not the figure’s. A picture of eleven partial sums cannot show a function with no slope; it can show the two rates whose competition produces one.

The Fourier figure shows twenty terms. That the differentiated partial sums alternate $2, 0, 2, 0$ at $x = 0$ and grow like $-2N$ near $\pm\pi$ is exact and checked; that they fail to converge at every point is a statement about all $N$, which the pictures support and an argument proves — the terms $\cos kx$ do not tend to $0$ at any $x$, so the series cannot converge anywhere. And the levels figure measures one sequence. Its three gradients are exact for that sequence; the claim that integration helps and differentiation hurts in general is the theorem of the section before it, not a consequence of one plot.

Still open: which rough limits keep some slope #

Uniform limits of smooth functions can be anything continuous, and in a precise sense almost all continuous functions have no derivative anywhere — Stefan Banach proved in 1931 that the nowhere-differentiable functions form all but a negligible part of the space of continuous functions, in the sense of Baire category that a limit can jump at every fraction introduced. So the interesting questions are about how much of a slope survives. A function may fail to be differentiable but still be Hölder continuous of some exponent, its increments bounded by $C|h|^{\alpha}$; the Weierstrass function with ratios $a$ and $b$ has exponent $\log(1/a)/\log b$, here $\log 2/\log 3 \approx 0.63$.

Where the questions remain open is in the fine structure: at which points a given rough function has one-sided derivatives, infinite derivatives or none, and how large those sets are. For Weierstrass’s own function the set of points with a finite or infinite one-sided derivative is known to be small, and for random functions such as the paths of Brownian motion the analogous sets have been mapped in great detail; but for many explicit series, including some studied since Riemann proposed $\sum \sin(k^2x)/k^2$ as a candidate in the 1860s — which turned out, as Joseph Gerver showed in 1970, to have a derivative at a very sparse set of rational multiples of $\pi$ and nowhere else — the complete description of where any slope survives was found only after long effort, and for most series of this kind it is not known.

Integration forgives and differentiation remembers #

Uniform convergence is a statement about heights, and heights are what integrals see: two functions that are close everywhere have close integrals, because an integral is an average and an average of small differences is small. A slope is the opposite of an average. It is a ratio of two small numbers, height over width, and two functions that are close in height can have that ratio anywhere at all, by wiggling on a scale smaller than their distance apart.

That is the whole content of the four counterexamples and the one theorem. The theorem works by turning the question about slopes into one about integrals, where uniformity is enough. The counterexamples work by hiding fast oscillation under small height, where uniformity cannot see it. And at the far end of the construction sits the continuous function with no slope anywhere — not a pathology at the edge of analysis, but the typical shape of a uniform limit about which nothing more than height is known.