The flat map that fits closest

The derivative of a function of one variable is a number, and the number is the slope of the line that fits the curve best near a point. Nothing about that description requires the answer to be a number — what is being asked for is a straight thing that fits, and in more than one dimension a straight thing is a linear map.

That last number is the definition doing its work. A map $f$ is differentiable at a point $p$ when there is a linear map $A$ with

$$f(p + h) = f(p) + Ah + r(h), \qquad \frac{|r(h)|}{|h|} \to 0,$$

and the requirement is not merely that the error shrinks — every continuous map has that — but that it shrinks faster than $h$ does. The figure measures exactly that: the residual falls by four when the patch halves, so relative to the patch it falls by two.

Why the definition is about a map and not about slopes #

The obvious alternative is to differentiate one variable at a time and collect the answers. That gives the partial derivatives, and they are not enough.

The standard example is the function equal to $xy/(x^2 + y^2)$ away from the origin and to zero at it. Along either axis it is identically zero, so both partial derivatives at the origin exist and are zero — and along the diagonal it takes the constant value one half, so it is not even continuous there. No linear map approximates it: the only candidate is the zero map, and the error does not shrink at all along the diagonal.

So partial derivatives measure the behaviour along two special directions and can miss everything in between. The linear-map definition asks for a single object that works in every direction at once, and that requirement is what makes the derivative useful rather than merely computable.

When the partial derivatives exist and are continuous, they do assemble into a derivative — the matrix of them is the map $A$ — and that is the working criterion. But the theorem there is a genuine theorem, and the counterexample above is what it is protecting against.

Straight, under magnification #

The one-dimensional statement of the same idea is the one everybody has seen, and it deserves a picture because the general case is invisible without it.

The claim is not that the curve looks straight — a picture can be made to show that for any continuous curve by zooming far enough on a screen with finite pixels. The claim is about a rate, and the figure measures the widest gap in each window rather than reporting an impression.

That rate is the whole distinction between a curve with a tangent and a curve without one.

A curve with a corner is exactly as continuous as one without, and no amount of zooming makes it straight. The curve that has a corner at every point is the extreme version, and it is what the definition is written to exclude.

The matrix, and what it does to area #

Once the derivative is a linear map, it can be written as a matrix of partial derivatives — the Jacobian — and everything already known about matrices applies to it.

The first figure measures this on the drawing rather than quoting it: the image of the patch is traced, its area computed by the shoelace formula, and the ratio to the patch’s own area compared with $|\det J|$ as the patch shrinks. It converges on the determinant — at the drawn size the curvature contributes a fifth of the answer, and at the smallest patch drawn the two agree to a per cent.

That is why the determinant of the Jacobian appears in every change of variables in a multiple integral. Chopping a region into small patches, mapping each, and adding up the images requires knowing what each patch’s area becomes, and the answer is the area of the patch multiplied by the local area factor. The factor is a property of the derivative and not of the map, which is what makes the substitution rule work at all.

The chain rule becomes multiplication #

The strongest argument for the definition is what happens to the chain rule.

If $f$ is differentiable at $p$ with derivative $A$, and $g$ is differentiable at $f(p)$ with derivative $B$, then $g \circ f$ is differentiable at $p$ with derivative $BA$ — the matrix product. In one dimension that is the familiar rule about multiplying slopes; in general it is the statement that composing two maps composes their best linear approximations.

The proof is two lines with this definition and a page of index-juggling without it. Substituting one approximation into the other gives $g(f(p+h)) \approx g(f(p) + Ah) \approx g(f(p)) + B(Ah)$, and the errors combine to something still smaller than $|h|$.

Once that is in place, the whole apparatus of linear algebra becomes available for questions about non-linear maps near a point. Whether the map is locally invertible is the question of whether the matrix is; how much the map distorts angles is a question about the matrix’s singular values; which directions are stretched most is a question about its eigenvectors. None of those questions could be asked of a list of partial derivatives.

Where the approximation is used #

Naming the derivative a linear map is not a change of vocabulary; it is what makes several standard procedures possible, and each is worth stating because each is invisible from the one-variable picture.

Solving equations by repeated linearisation. To solve $f(x) = 0$ for a map of several variables, replace $f$ near a guess by its linear approximation, solve the resulting linear system exactly, and repeat. That is Newton’s method in several dimensions, and it exists only because the approximation is a map: a list of partial derivatives could not be solved against.

Deciding local invertibility. If the derivative at a point is an invertible matrix, the map itself is invertible on some neighbourhood of that point. That is the inverse function theorem, and it converts a question about a non-linear map, which is hard, into a question about a determinant, which is arithmetic. The converse fails — a map can be invertible with a singular derivative, as $x \mapsto x^3$ is on the line — so the theorem is a one-way street and worth reading carefully.

Following a curve defined implicitly. A relation like $F(x, y) = 0$ defines a curve, and whether it can locally be written as a graph is decided by whether one column of the derivative is invertible. The tangent to the curve is then read off the same matrix. That is the implicit function theorem, and it is the reason a curve given by an equation can be handled with the same tools as one given by a formula.

The common thread is that a hard local question about a non-linear object becomes a linear-algebra question about one matrix. The approximation is not an approximation to be tolerated; it is the bridge between the two subjects.

Two derivatives that are not this one #

Two other objects wear the name and are weaker, and confusing them with the derivative is the most common error in this material.

The directional derivative measures the rate of change along one chosen direction, and is a number for each direction. The derivative determines all of them — the directional derivative along $u$ is $Au$ — but the reverse fails: a map can have a directional derivative in every direction and still not be differentiable, since the directional derivatives need not fit together linearly. The example above is of exactly that kind.

The gradient applies to a map into the line rather than into the plane, and it is the derivative rewritten as a vector: the single row of the matrix, stood on end. That rewriting is convenient and slightly dishonest, because a row and a column are different kinds of object and the identification depends on a choice of inner product. The dot product is what does the identifying, and in settings where there is no natural one — or where the coordinates carry different units — the gradient is not available and the derivative still is.

Both are useful. Neither is the object with the approximation property, and the approximation property is what all the theorems need.

What is being thrown away #

The derivative discards everything about the map except its first-order behaviour at one point, and it is worth being explicit about how much that is.

The two maps in the figures differ everywhere and can have the same derivative at a point. Any information about curvature, about where the map sends distant points, and about whether it is one-to-one anywhere beyond an immediate neighbourhood is gone. What survives is a matrix — four numbers, in the plane — and the claim is that those four numbers control the behaviour near the point to first order.

That is the same trade every good approximation makes: a complicated object is replaced by a simple one, exactly enough of the original is kept to answer a class of questions, and the rest is quantified as an error rather than ignored. What makes this instance so productive is that the simple object is a linear map, and linear maps are the best-understood objects in mathematics.

Where it came from #

The definition took an unusually long time to settle, and the delay is instructive about what was hard.

Through the eighteenth and most of the nineteenth century, functions of several variables were handled entirely through partial derivatives, one variable at a time, with the others held fixed. That machinery is enough for computation and it hides the concept: nothing in it says what object the derivative is, and the pathological examples that distinguish partial derivatives from differentiability were not constructed until the subject had reasons to look for them.

The modern definition — a linear map with a residual small compared with the increment — is due to Fréchet, in the early 1900s, and was written for spaces of functions rather than for the plane. That is the usual direction of travel for a good definition: it was formulated where the old one was unavailable, since a space of functions has no coordinates to differentiate one at a time, and then turned out to be the right definition in the finite-dimensional case as well.

The consequence is worth stating. The derivative in several variables is not a generalisation of the one-variable derivative that happened to work; it is the definition that had to be found in order to say anything at all in infinite dimensions, and the finite-dimensional case inherited it. That is why the notation still bears traces of the older, coordinate-bound approach, and why the matrix is so often introduced before the map it represents.

What the picture cannot show #

The patch in the figures is $0.6$ across and its image is visibly curved, which is the opposite of what the theorem says: the whole content is what happens as the patch shrinks, and at a size where the two grids were indistinguishable there would be nothing to see. The drawing therefore shows a patch large enough to be wrong, with the residual column supplying the behaviour at sizes too small to draw.

The plane is also the smallest interesting case. The definition is identical for a map between spaces of any dimensions, where the derivative is a rectangular matrix; a map from three dimensions to two has a derivative that is a $2 \times 3$ matrix and no determinant at all, and the area factor is replaced by something more elaborate. None of that is visible here.

And the residual is measured over a square patch by sampling it on a grid. That is a check rather than a proof: the definition quantifies over all $h$ approaching zero from every direction, and a grid of six hundred sample points at five sizes tests a finite portion of that.

The ladder from here #

Below: the slope of a single point, which is this in one dimension, and the product rule, whose corner term is the residual seen in the simplest possible case. Sideways: matrices as grids and the determinant as an area factor, which are what the derivative turns out to be made of. Above: the slope of a mirror image, where inverting the map inverts the matrix, and the implicit and inverse function theorems, which say that the invertibility of the linear approximation is inherited by the map itself near the point.

What the residual measures #

The column of residuals in the figures is the definition made numerical, and reading it takes a moment’s care.

The quantity tabulated is the largest distance between the curved image of a point and its flat image, over the whole patch. It falls by four at each halving, which is the signature of a remainder controlled by the square of the step — and the definition asks for less than that, only that the remainder beat the step itself.

So the maps drawn here are better behaved than differentiability requires. A map can be differentiable with a residual falling like the step to the power one and a half, and no picture would distinguish that from the quadratic case at any single size. What distinguishes them is the ratio between successive sizes, which is why the figure reports a column rather than a number.

A number that was always a matrix #

The lasting point is that generalising did not complicate the object; it revealed what the object had been.

The derivative in one dimension is a linear map from the line to the line, and every such map is multiplication by a number — which is why the answer could be recorded as a number and why the notation hides the map. In two dimensions the same definition, word for word, produces a matrix, and in infinitely many dimensions it produces an operator between function spaces with no matrix at all.

This pattern — where the familiar case turns out to be a degenerate instance whose structure was invisible for lack of room — recurs throughout this collection. The theorem about right triangles is a statement about one particular norm; a determinant is a volume before it is a formula. In each case nothing was added by the generalisation, and something that had been there all along became visible.