The slope of a single point

Slope is the least mysterious idea in elementary mathematics. Take two points, divide the rise by the run, and that is the slope of the line through them. Nothing about it needs defending.

Ask for the slope at a point and the definition falls apart immediately. One point has no rise and no run. The formula becomes $0/0$, which is not a small number or a large one but no number at all — the one arithmetic expression that carries no information whatever, since every number $q$ satisfies $0 \cdot q = 0$.

So the question is either meaningless or needs a different kind of answer. The construction that supplies one is a sequence.

The sequence, and what it is not #

Fix a point on the curve. Take a second point a distance $h$ along, and draw the line between them. Its slope is a perfectly ordinary quotient:

$$\frac{f(x+h) - f(x)}{h}.$$

Now shrink $h$. Each value gives a genuine line through two genuine points, and the slopes in the figure form a sequence: $3.2, 2.8, 2.5, 2.28, 2.12$. They are heading somewhere, and where they are heading is $2$.

The derivative is defined as that destination:

$$f'(x) = \lim_{h \to 0} \frac{f(x+h) - f(x)}{h}.$$

The two things this definition is not are worth stating, because they are where the standard confusions live.

It is not the value at $h = 0$. The quotient at $h = 0$ is $0/0$ and remains meaningless. The limit is a statement about the values at $h \ne 0$ — about what they approach — and it never evaluates the expression at the forbidden point. Every secant in the figure is a real line; the tangent is not one of them.

It is not a very small quotient. There is no smallest $h$, no last secant, and no stage at which the approximation becomes the answer. This is the same structure as rectangles under a curve, where the staircase is never the area and the area is what the staircases converge to. The limit is exact because the limit is exact, not because $h$ got small enough.

Computing one, without limits doing the work #

For $f(x) = x^2$ the algebra is short enough to run in full, and running it shows where the limit actually enters.

$$\frac{(x+h)^2 - x^2}{h} = \frac{x^2 + 2xh + h^2 - x^2}{h} = \frac{2xh + h^2}{h} = 2x + h.$$

That last cancellation is legal because $h \ne 0$ — dividing by $h$ requires it, and the whole point of the secant construction is that $h$ never is zero. What is left, $2x + h$, is an ordinary expression with no division problem in it. It has a limit as $h \to 0$, and the limit is $2x$, obtained by substitution rather than by any subtle process.

At $x = 1$ that gives $2$, which is where the printed slopes were heading. The figure’s five numbers are $2 + h$ for $h = 1.2, 0.8, 0.5, 0.28, 0.12$, and the generator computes them from the function rather than from the formula — so if the algebra above were wrong, the picture and the caption would disagree.

The structure generalises. The difference quotient is always $0/0$ in appearance and always simplifies to something harmless before the limit is taken, and the simplification is the actual work. When it cannot be simplified, there is usually no derivative.

From a number to a function #

The construction so far produces one number at one point. The step that makes calculus a subject rather than a technique is noticing that it produces one at every point, and that those numbers assemble into a new function.

Running the algebra at a general $x$ gave $2x$, and the two figures are that formula sampled twice. So $x^2$ has a derivative function $2x$: an object of the same kind as the thing it came from, which can be plotted, added, and differentiated again.

That closure is what makes the subject productive. If differentiating produced numbers, calculus would be a table of answers. Because it produces functions, differentiation can be iterated — giving acceleration from velocity, curvature from slope — and it can be inverted, which is the question the integral answers. It also means equations can be written about the operation: an equation relating a function to its own derivative is a differential equation, and almost every quantitative law of nature is one.

The most famous such equation is the shortest. Ask for a function equal to its own derivative and the answer is the exponential — a question that cannot even be posed until the derivative is a function rather than a number.

Local straightness #

There is a second way to read the same picture, and for many people it is the one that makes the derivative feel like an object rather than a procedure.

As the second point slides in, the secants stop changing much. Past a certain closeness they are all nearly the same line, and the curve between the two points is nearly indistinguishable from it. A differentiable function is one that is locally straight: zoom in far enough at any point and the curve becomes a line, and the derivative is that line’s slope.

That reading explains why the derivative is useful rather than merely defined. Straight lines are easy; curves are not; and a differentiable function is one that can be treated as straight over a small enough region, with an error that is second order and therefore negligible first. Nearly every applied use of calculus is that substitution and nothing more.

It also explains what can go wrong. A function fails to be differentiable exactly when zooming in does not produce a line — because there is a corner, because there is a jump, or because the curve oscillates so violently that no amount of zooming settles it.

The point with two slopes #

The cleanest failure is a corner, and $|x|$ at the origin is the standard one.

Approach from the right and every difference quotient is $h/h = 1$. Approach from the left and every one is $-h/(-h) \cdot -1 = -1$. Both sequences are constant. Both converge instantly. They converge to different numbers.

A limit is required to be independent of how it is approached — that is not a technicality but the whole content of the word. Two approaches giving two answers means there is no limit, and no derivative at zero.

Notice what this does not say. $|x|$ is perfectly continuous at zero: it has no jump, the value is $0$, and nearby values are near $0$. Continuity and differentiability are different properties, and this is the standard witness that the second is strictly stronger. Zooming in on the corner never straightens it — magnify by a million and the corner is still a corner, at the same angle, because the function is made of two straight pieces and scaling does not change an angle.

The failure has a shape worth recognising, because it is the same one that appears wherever a limit is claimed. In the needle-dropping experiment the estimator converges but is not centred; here the quotients converge from each side and to different places. Both are cases where “it approaches something” is true of every sub-collection of the evidence and false of the whole — and the definition has to be stated carefully enough to notice.

Corners are also not rare. A function can have corners at every rational point and be continuous everywhere. Weierstrass produced, in 1872, a function continuous at every point and differentiable at none — a curve with no tangent anywhere, which was received as an outrage and is now understood to be the typical case rather than the exception. Differentiable functions are the unusual ones; the intuition that a continuous curve must have a slope almost everywhere is a report on the examples people draw, not on the class of continuous functions.

The trouble with computing it #

The definition suggests an obvious numerical method — pick a small $h$ and evaluate the quotient — and that method is a good illustration of why definitions and algorithms are different things.

Two errors are in competition. The truncation error is the mathematics: the quotient differs from the derivative by roughly $h f''/2$, so it shrinks as $h$ does. The rounding error is the arithmetic: $f(x+h)$ and $f(x)$ are nearly equal numbers, so subtracting them cancels the leading digits and leaves a result carrying far fewer significant figures than either input, and dividing by a tiny $h$ magnifies what is left.

The cancellation is worth making concrete. Evaluating $\sin$ at $1$ and at $1 + 10^{-10}$ gives two numbers agreeing in their first ten significant digits; subtracting them destroys those ten and leaves a difference whose leading digit was the eleventh — so of sixteen digits held, six survive. Dividing by $10^{-10}$ does not restore them. The information was destroyed by the subtraction and no later operation can recover it, which is what makes this a different situation from ordinary rounding error.

One error falls as $h$ shrinks and the other rises. Their sum is minimised somewhere in the middle, and in double precision that optimum sits near $h \approx 10^{-8}$ — the square root of the machine epsilon — where about eight of the sixteen available digits survive. Half the precision is lost, unavoidably, by the best possible choice of $h$. Choosing $h$ smaller makes the answer worse, and choosing $h = 10^{-16}$ returns zero, because $x + h$ rounds to $x$.

This is the same shape of failure as computing $e$ from the compound-interest limit: an expression that converges in exact arithmetic and turns round and walks away in floating point, because two operations are fighting and the arithmetic decides before the limit does. It is also why real code does not do this. Symbolic differentiation, automatic differentiation, or a complex-step trick will each give the derivative to full precision, and all three work by never forming the difference of two nearly equal numbers. The complex-step method is the prettiest: evaluate $f$ at $x + ih$ for a tiny imaginary step and take the imaginary part divided by $h$. No subtraction of nearly equal quantities occurs anywhere, so $h$ can be made as small as desired, and the trick works because multiplying by $i$ is a quarter turn that moves the perturbation into a direction the real part cannot cancel against.

What the picture cannot show #

The figure draws five secants and the claim is about all of them. That is the usual gap, and here it is doing more damage than usual, because the sequence’s behaviour — not merely its endpoint — is what has to be established.

Five slopes heading toward $2$ is consistent with a limit of $2$. It is equally consistent with a limit of $1.99$, or with a sequence that approaches $2$ and then turns away, or with no limit at all. Nothing in a finite set of secants distinguishes these, and the distinction is the entire content of the definition. The generator addresses this by requiring each drawn secant to be strictly closer to the analytic derivative than the one before, which is a stronger and more checkable statement than “the last one is close”.

The corner figure has the opposite problem: it shows a failure that is real, and it makes failure look conspicuous. Most functions that fail to be differentiable do not fail in a visible way. A function differentiable everywhere except on a set of measure zero looks exactly like a smooth one at every scale a page can render, and Weierstrass’s nowhere-differentiable curve looks, at a glance, like a rough line — which is precisely why it took until 1872 for anyone to construct one and why it caused the reaction it did.

The ladder from here #

Rungs above: the derivative of $\sin$ via the squeeze argument, where the inequality that does the work is invisible at every drawable scale. The product and chain rules, drawn as areas and as compositions rather than derived. The mean value theorem, which is this construction run backwards — every secant’s slope is achieved by some tangent. Weierstrass’s function, plotted at several magnifications. Differentiability in several variables, where directional derivatives can all exist and the function still fail to be differentiable. The derivative as a linear map, which is the definition that generalises. Numerical differentiation done properly. And the fundamental theorem, where this construction and the rectangles turn out to undo each other.

The two limits, side by side #

It is worth putting this essay next to the integral one, because between them they are the whole of elementary calculus and they are the same manoeuvre.

The integral takes a quantity that is easy for straight-sided shapes — area — and extends it to curved ones by a limit of things that are not the answer. The derivative takes a quantity that is easy for two points — slope — and extends it to one point by a limit of things that are not the answer. In both cases the object being defined is absent from every stage of its own construction, and in both cases nineteenth-century analysis was the work of making that respectable.

The pleasing part is that the two turn out to be inverse. Nothing in either construction hints at it: one chops an interval into pieces and adds, the other takes two points and divides. That they undo each other is the fundamental theorem, and it is the reason the subject is called calculus rather than two subjects with separate names.