Gradient and Directional Change

Define the rate along a unit direction

For a point $\mathbf a$ and unit vector $\mathbf u$,

when the limit exists. Because $\|\mathbf u\|=1$, $|h|$ measures input-space distance. Domain boundaries can prevent an ordinary two-sided limit in some directions.

If $f$ is differentiable,

This follows by substituting $h\mathbf u$ into the full linear approximation. The directional-derivative reference develops the formula and its geometry.

Normalise before calculating

Let $f=x^2+2y^2$. At $(1,2)$, the gradient is $(2,8)$. A direction described by $(3,4)$ has magnitude five, so its unit vector is $(3/5,4/5)$. Therefore

Using $(3,4)$ directly would give $38$. That is the rate along $(1,2)+t(3,4)$, a path moving through input space at speed five. It is not the rate per unit distance.

Derive the direction of greatest increase

For a unit vector, $\nabla f\cdot\mathbf u=\|\nabla f\|\cos\phi$. When the gradient is nonzero, the maximum rate is its norm, attained in its direction. The minimum is the negative norm, attained in the opposite direction.

In the example, the greatest rate is $2\sqrt{17}$ and its unit direction is

This is a local first-order statement. A finite move also encounters curvature and changing derivatives, so it need not have the largest finite effect among all moves of the same length.

Relate the gradient to a level curve

The vector $(-4,1)$ is perpendicular to $(2,8)$ and gives zero first-order change. It is a tangent direction to the level curve $x^2+2y^2=9$ through $(1,2)$.

The tangent line is

Moving along this straight line has zero first-order output change at the point, but the line does not remain on the curved level set. A path staying on the level curve must continually adjust its tangent direction.

Use a gradient as a level-surface normal

For a differentiable $F(x,y,z)$, a regular level surface $F=c$ has normal $\nabla F$ wherever that gradient is nonzero.

For $F=x^2+y^2+z^2$, the point $(1,2,3)$ lies on level fourteen and has gradient $(2,4,6)$. Its tangent plane is

or $x+2y+3z=14$. For a graph $z=f(x,y)$, use $F=f(x,y)-z$ to obtain normal $(f_x,f_y,-1)$. The two-component gradient of $f$ is an input-plane vector, not already a three-component surface normal.

A zero gradient does not classify a point

At a differentiable point with zero gradient, every unit directional derivative is zero. The point is stationary to first order, but it need not be a minimum or maximum.

At the origin, $x^2+y^2$ has a minimum, $-x^2-y^2$ has a maximum and $x^2-y^2$ has a saddle. All have zero gradient. Higher-order behaviour and the domain determine the classification.

A zero gradient also fails to supply a normal direction for a level set. Regular-level conclusions require a nonzero gradient; special points need their own analysis.

All directional derivatives can exist without differentiability

Define $g(0,0)=0$ and $g(x,y)=x^3/(x^2+y^2)$ elsewhere. Its coordinate partials at the origin are $(1,0)$. For a unit vector $(u_1,u_2)$,

Every directional derivative exists, but its value is $u_1^3$. Differentiability would require the linear gradient rule $u_1$ instead. In direction $(1/\sqrt2,1/\sqrt2)$, the actual derivative is $1/(2\sqrt2)$, not $1/\sqrt2$.

The function is continuous at the origin because $|g(x,y)|\le|x|$. Thus even continuity plus all directional derivatives does not guarantee the complete linear approximation. The failure is the nonlinear dependence of the rate on the direction.

Combine gradient with path velocity

For a differentiable field and differentiable path,

Here the velocity is not normalised because the result is change per unit time or parameter. At nonzero speed, divide by that speed to obtain change per unit distance.

An object can travel rapidly along a level set without changing the scalar value. Another can move slowly across sharply changing levels and experience a substantial rate. Both gradient and velocity matter.

Check the distance scale behind “steepest”

The usual steepest-direction result uses Euclidean distance in the stated coordinates. It has a direct spatial interpretation when axes use compatible length units. If one input is time and another is money, their unscaled coordinates do not automatically define a meaningful physical distance.

Changing an input’s numerical units changes its partial derivative and can change an apparent steepest direction. For differently scaled parameters, choose a justified scaling or metric before comparing directions. Partial rates remain meaningful with their own units, but geometric optimisation needs a distance convention.

A descending direction still needs a suitable step

A basic update is

For $f=x^2+y^2$ at $(1,1)$, the gradient is $(2,2)$. With $\eta=0.1$, the new point is $(0.8,0.8)$ and the value drops from two to $1.28$. With $\eta=1.5$, it moves to $(-2,-2)$ and the value rises to eight.

The negative gradient is locally descending at a differentiable nonstationary point, but an arbitrarily large step is not guaranteed to decrease the function. A usable optimisation method also needs domain handling and a stopping criterion.

Related topics

Partial Derivatives supplies differentiability and the chain rule. Motion Along a Curve interprets path velocity, and Approximation and Error in Calculus explains the limits of finite-step predictions.