The subsequence that has to exist

A bounded sequence of numbers always has a subsequence that converges. The reason is the pigeonhole principle run for ever: cut the interval holding the numbers in half, keep a half that still holds infinitely many of them, cut that in half, and keep going. The halves close in on a single point, and picking one term from each half, later and later in the sequence, gives a subsequence that approaches it. Bolzano used exactly this argument, and Weierstrass made it one of the foundations of analysis.

For functions the same question has a different answer. The supremum distance — the largest vertical gap between two graphs — turns the continuous functions on an interval into the points of a space, and a sequence converges uniformly exactly when it converges as a sequence of points there. So the question can be asked word for word: does every bounded sequence of continuous functions have a uniformly convergent subsequence?

It does not, and the waves $\sin 2\pi kx$ are the standard reason.

Bounded, and nowhere to settle #

Every member of $\sin 2\pi kx$ lies between $-1$ and $1$, so the sequence is as bounded as a sequence of functions can be. And every two different members are at least 1 apart somewhere on the interval.

The proof comes from averaging rather than from looking. Square the difference $\sin 2\pi mx - \sin 2\pi nx$ and average it over the interval. Each sine squared averages to a half, and the cross term $\sin 2\pi mx \cdot \sin 2\pi nx$ averages to nothing when $m \ne n$ — the orthogonality that makes Fourier coefficients computable. So the square of the difference averages to exactly 1, and a function whose square averages to 1 cannot stay below 1 in size everywhere. At some point the two members differ by at least 1. The table measures the actual largest gaps on a fine grid, which can only understate them, and finds every one between 1.76 and 2.

A sequence whose members are all at least 1 apart has no subsequence that settles, since a settling subsequence would eventually have its members within a half of each other. So the functions bounded by 1 contain infinitely many points of their space, pairwise at least 1 apart.

In a plane or in space that is impossible: a bounded region can hold only finitely many points that far apart, which is why the halving argument works there. The space of continuous functions is infinite-dimensional, and this is what that means in practice — boundedness no longer confines anything. Each new wave points in a direction none of the earlier ones used.

Why the pigeonhole fails for functions #

The halving argument works for a number because a number is decided, to any precision, by finitely many choices of which half it lies in. A function is a value at every point of an interval, which is infinitely many numbers, and the obvious repair runs into trouble at once.

The repair is to look at finitely many points. Fix some nodes across the interval, record each member’s values there rounded to a coarse grid, and call that the member’s profile. There are only finitely many possible profiles, so infinitely many members of any sequence share one. That much is the pigeonhole principle and it is fine. The trouble is what sharing a profile says.

Two members can agree exactly at every node and be as far apart as the bound allows in between. A member that oscillates fast enough can do anything at all between the nodes, and making the nodes finer only invites a member that oscillates faster still. What the argument lacks is a promise that between nearby nodes nothing much happens — and the promise has to hold for every member at once, with one spacing of nodes serving the whole family.

One stretch length for every member #

A family of functions is equicontinuous when, for every tolerance $\varepsilon$, there is a single stretch length $\delta$ such that in every member, any two points closer than $\delta$ have values closer than $\varepsilon$.

Each continuous function on a closed interval has such a $\delta$ on its own — that is uniform continuity — so the entire content of the definition is that one $\delta$ serves every member. It is the order of the quantifiers once more: for every member there is a $\delta$, against there is a $\delta$ for every member.

The figure measures the modulus of continuity of each member: the largest change across any stretch of length $0.05$. For the phase shifts $\sin(2\pi x + k)$ it is 0.313 at every $k$, because every member is the same wave slid sideways and none is steeper than another. For $x^k$ it climbs towards 1, because the members steepen near $x = 1$, which is the same failure that cost $x^k$ its continuity in the limit. For $\sin 2\pi kx$ it reaches 2 at $k = 10$: once half an oscillation fits inside a stretch of $0.05$, that stretch holds a rise from $-1$ to $1$.

Only the first family is equicontinuous. The usual way to establish the property is a common bound on the slope: if every member has its derivative between $-M$ and $M$, then by the mean value theorem no member changes by more than $M\delta$ across a stretch $\delta$, and one stretch length serves them all.

The theorem, and why two conditions are enough #

The Arzelà–Ascoli theorem. On a closed bounded interval, a sequence of continuous functions that is bounded and equicontinuous has a uniformly convergent subsequence. Conversely, a family in which every sequence has a uniformly convergent subsequence must be bounded and equicontinuous. Ascoli introduced the condition in the 1880s and Arzelà completed the characterisation in the next decade.

The argument is the pigeonhole repair with the missing promise supplied. Fix a tolerance $\varepsilon$ and take the $\delta$ that equicontinuity provides. Place nodes closer together than $\delta$, and round every member’s values at the nodes to multiples of $\varepsilon$. Finitely many profiles again, so infinitely many members share one. Now take two members $f$ and $g$ with the same profile, and any point $x$. The nearest node $a$ is within $\delta$, so $f(x)$ is within $\varepsilon$ of $f(a)$; $f(a)$ is within $\varepsilon$ of $g(a)$, since their rounded values agree; and $g(a)$ is within $\varepsilon$ of $g(x)$. Three steps of $\varepsilon$, at every point at once, so the two members are within $3\varepsilon$ of each other everywhere.

Then halve $\varepsilon$ and repeat the argument inside the infinitely many members that survived, and halve again, and so on. Taking one member from each stage, each later in the sequence than the last, gives a subsequence whose members from stage $j$ onwards lie within $3\varepsilon/2^j$ of one another everywhere. That is a uniformly Cauchy sequence, it converges uniformly, and its limit is continuous. The step that picks one member per stage is Cantor’s diagonal move, turned from an argument about what cannot be listed into a way of building something.

Watching the choice being made #

The family of shifted waves shows the argument without any rounding, because a member of it is decided by a single number: the angle $k$ it is shifted by, taken round a circle of length $2\pi$.

The largest gap between $\sin(2\pi x + a)$ and $\sin(2\pi x + b)$ is exactly $2|\sin((a - b)/2)|$, so members whose phases lie in the same arc are close everywhere, not merely at a few points. Sorting sixty members into eight arcs forces one arc to hold at least eight of them, and those eight are within 0.765 of each other across the whole interval. That is equicontinuity and the pigeonhole principle doing their jobs together in one picture.

The table is the diagonal choice carried out. Each stage halves the arc, keeps the half holding more of the remaining members, and picks the first member in it later than the last pick. Everything picked from then on lies inside that arc, so the gaps are bounded by a quantity that halves every stage, and the picks form a uniformly convergent subsequence.

The sequence itself never converges. The phases $k$ are whole numbers of radians taken round a circle whose length $2\pi$ is irrational, so they never repeat and spread evenly over the circle. For every angle $c$ there is a subsequence converging uniformly to $\sin(2\pi x + c)$, and the theorem promises only that some convergent subsequence exists. Which limit is reached depends entirely on which choices are made, and here there is a whole circle of possible limits.

Solutions that have to exist #

The theorem’s most famous use is to show that differential equations have solutions, and the family it is applied to is a sequence of broken lines.

Take an equation $y' = f(t, y)$ and follow it in straight steps: from each point, move along the slope the equation gives there for a short time, then look again. Those are Euler polygons. When $f$ is continuous and bounded by $M$, every polygon has every slope between $-M$ and $M$, so the polygons are bounded on a finite interval and equicontinuous with one modulus, $M\delta$, for all of them. Arzelà–Ascoli hands over a uniformly convergent subsequence.

The limit solves the equation. Each polygon nearly satisfies the integral form $y(t) = y(0) + \int_0^t f(s, y(s))\,ds$ — area as the undoing of slope — and uniform convergence carries both sides of that equation to the limit. That is Peano’s existence theorem, from 1890: a continuous rule for the slope is enough for a solution to exist.

It is not enough for there to be only one. The equation $y' = 3y^{2/3}$ from $y(0) = 0$ is solved by $y = 0$ and by $y = t^3$, and by the zero function followed by $t^3$ from any later start. Uniqueness needs a stronger hypothesis, a bound on how fast the slope changes with $y$, and under that hypothesis the solution is unique and can be found by an iteration that shrinks every distance. Peano’s argument proves existence and says nothing about which solution the subsequence found.

The same move answers an objection met in a quite different place. The most area a fence can hold records Weierstrass’s complaint that Steiner’s arguments assumed a best shape exists. One standard repair is a compactness theorem for shapes rather than for functions — Blaschke’s selection theorem of 1916, which says that convex shapes confined to a bounded region always contain a sequence converging to a convex shape. Take shapes whose areas approach the best possible, extract such a sequence, and the shape it converges to is the optimum whose existence Steiner needed. The argument is Arzelà–Ascoli’s, applied to boundaries.

A ball that is not compact #

A set in which every sequence has a convergent subsequence is called compact, and the theorem is a description of the compact sets of continuous functions on a closed interval: the ones that are closed, bounded and equicontinuous.

In the plane, and in any space of finitely many dimensions, compact means closed and bounded and nothing more. That is the Heine–Borel theorem, and it is the halving argument run in each coordinate at once. The waves $\sin 2\pi kx$ show that the equivalence breaks for continuous functions, and in 1918 Riesz proved that it breaks everywhere it could: in a space where distance is measured by a norm, the closed ball of radius one is compact exactly when the space has finitely many dimensions. Any space with infinitely many dimensions contains a sequence like the waves, every member at least a fixed distance from all the others.

So Arzelà–Ascoli is not a technicality about functions. It says what has to be added to boundedness when dimension runs out, and the answer — a uniform bound on how quickly the members change — is available only because the points of this space are functions, for which “how quickly” means something. In a space whose points were arbitrary infinite lists of numbers, no such bound would be on offer.

Where boundedness is enough after all #

In one setting the extra condition comes free. A function of a complex variable that has a derivative in the complex sense cannot be small on a disc and steep in the middle of it: Cauchy’s integral formula bounds its derivative at a point by its size on a circle round that point, divided by the circle’s radius. So a family of such functions bounded by a single number on a region is automatically equicontinuous on every closed disc inside the region, and Arzelà–Ascoli applies with nothing more assumed.

That is Montel’s theorem, from 1907, and it carries the Riemann mapping theorem: the map carrying a region of the plane onto a disc is found as the limit of a subsequence chosen from a bounded family of maps, exactly as the solution of a differential equation was found from a family of Euler polygons.

The waves are no counterexample there, and the reason is instructive. As a function of a complex number $z$, $\sin 2\pi kz$ is not bounded at all: a short step off the real line makes it grow like $e^{2\pi k|\mathrm{Im}\, z|}$. Confined to the real line the waves are bounded and steep. Allowed to leave it they are no longer bounded, and the obstruction that made them separate disappears with the hypothesis.

Where the guarantee stops #

The interval must be closed and bounded. On the whole half-line, the bumps $f_k(x) = \max(0, 1 - |x - k|)$ are bounded by 1 and have slope at most 1, so they are equicontinuous; each sits one unit further along than the last, every two are 1 apart, and no subsequence converges uniformly. The finite set of nodes the argument needs does not exist on an unbounded interval.

Existence is not construction. The theorem says a subsequence converges and gives no rule for finding it. Among the shifted waves the diagonal choice could be made explicitly only because a member is determined by one number; for a general family the nodes are infinitely many, the stages go on for ever, and the subsequence exists in the way a number defined by infinitely many halvings exists.

Different subsequences can have different limits. Nothing makes the limit unique, and the Euler polygons show why that matters: when an equation has several solutions, different subsequences of polygons can in principle close on different ones, and the theorem cannot say which.

Finite families and the easiest diagonal #

Every family drawn has finitely many members. The table of gaps stops at ten, the modulus at forty, the diagonal search at five thousand. The claims are about all members, and each figure states a closed form — the mean square, the modulus, the gap between two shifted waves — that the finite measurement is checked against.

The grid understates every maximum. A largest gap measured at four thousand points is a lower bound on the true one, which is why the gaps are reported as at least 1.76 and the proof of at least 1 is given by averaging.

And the diagonal argument is shown in its easiest case. A family decided by one number needs one halving per stage. The general proof needs a node at every rational point, one stage for each, and the diagonal taken across all of them; no picture holds that.

The question it leaves: what survives a limit that is not uniform #

Arzelà–Ascoli produces uniform convergence, and uniform convergence is what lets continuity and integrals pass to a limit. Many of the limits analysis actually meets are not uniform. The partial sums of a power series converge uniformly inside their interval and may do anything at its edge; the sequences that define integrals of rough functions converge only outside small sets.

For the first, the question is whether a series that converges at the edge of its interval takes the value its function takes there, and it has a clean answer, due to Abel: a sum can be read from inside the interval. For the second, the answer replaces the uniform bound on the difference with a fixed function bounding the size of every member, which is Lebesgue’s dominated convergence theorem and the reason the integrable functions are the ones they are.

A bound on how fast, not on how big #

Boundedness says how far a function may go. Equicontinuity says how fast it may get there, and for functions on a closed interval the second is what the first was expected to do.

The argument that uses it is the oldest one in the subject — divide, keep the part with infinitely many members, divide again — and it needed only one new ingredient to work for functions: a promise, shared by every member, that knowing the values at close enough points is knowing the values everywhere. When a compactness argument fails in infinitely many dimensions, look for the missing uniform bound on how quickly things change, because that is almost always what the finite-dimensional version was getting for free.