Approximation and Error in Calculus

Define the comparison

Let $Q$ be the exact mathematical target and $\widehat Q$ its approximation. Signed error is $\widehat Q-Q$ and absolute error is $|\widehat Q-Q|$. For $Q\ne0$, relative error is

If $Q=2$ and $\widehat Q=2.03$, absolute error is $0.03$ and relative error is $0.015$, or $1.5\%$. Near a zero target, relative error can be large or undefined even when the absolute difference is small.

The exact target belongs to a mathematical model. A perfectly evaluated model may still describe reality poorly. Numerical and model error need different evidence.

Separate error sources

Truncation or discretization error comes from replacing a limiting process with a finite calculation. Roundoff comes from finite-precision representation and arithmetic. Data error comes from uncertain inputs. Model error comes from an imperfect description of the underlying process.

These contributions interact. Numerical differentiation amplifies sample errors by dividing a difference by a small step. A stable algebraic reformulation can reduce roundoff without changing the target.

Distinguish a proven bound under assumptions, a heuristic estimate and an observed error against a trusted reference. They do not warrant the same claims.

Bound a tangent-line approximation

For $f(x)=\sqrt{x}$, the tangent line at four is

At $4.1$, it gives $2.025$. Since $f”=-1/(4x^{3/2})$, its magnitude on $[4,4.1]$ is at most $1/32$. Taylor’s bound yields

The exact value is approximately $2.0248456731$, so the observed error is about $0.0001543269$, below the bound. Concavity explains why the tangent approximation is high; a magnitude bound alone does not give its direction.

Include a Taylor remainder

The degree-$n$ Taylor polynomial about $a$ is

If the derivative of order $n+1$ is continuous on the interval joining $a$ and $x$, with magnitude at most $M$, then

The Taylor remainder reference states the theorem. The interval and derivative bound are part of the calculation.

For $e^{0.2}$,

The error is at most $e^{0.2}(0.2)^4/24\approx0.0000814269$. Its observed value, approximately $0.0000694248$, is smaller. To guarantee correct rounding, check whether the entire certified interval rounds to the same value. A small tolerance does not settle rounding when that interval straddles a rounding boundary.

Balance step-size effects

For a central difference, suppose truncation error is bounded by $Ch^2$ and each sample has absolute error at most $\delta$. A combined bound is

The first term decreases with a smaller step; the second increases. For positive $C$ and $\delta$,

With $C=1$ and $\delta=10^{-6}$, the bound-minimising step is approximately $0.007937$. This minimises the stipulated envelope, not necessarily the unknown actual error. Available sample positions, units and the assumptions behind $\delta$ still matter.

Remove a leading term by extrapolation

Suppose a valid error expansion is

Then $D(h/2)=D+ch^2/4+O(h^4)$, giving

For central differences of $x^3$ at two, $D(0.1)=12.01$ and $D(0.05)=12.0025$. The extrapolation equals twelve exactly because this cubic has no higher surviving error term.

Richardson extrapolation needs the assumed expansion and consistently computed values. Two meshes can miss the same feature, and extrapolation can amplify uncertain inputs. Without remainder control, identify a leading-order estimate as an estimate rather than a rigorous bound.

Avoid cancellation with equivalent algebra

For small $x$, $\sqrt{1+x}-1$ subtracts nearly equal values. Rationalisation gives

The second expression avoids that subtraction. For $(\sqrt{1+x}-1)/x$, cancellation yields $1/(\sqrt{1+x}+1)$ only when $x\ne0$. The latter expression additionally defines a continuous extension at zero with value $1/2$.

Stable algebra does not silently change a domain. Distinguish equality on the original domain from an explicitly defined extension. It also does not remove uncertainty in the input or shortcomings in the model.

A residual is not a solution error

For $0.001x=1$, the exact solution is $1000$. At $\widehat x=1000.001$, the residual is

while the solution error is $0.001$. Converting residual to solution error multiplies it by $1000$ in this scale.

Conditioning concerns sensitivity of the mathematical problem to data changes. Stability concerns the numerical algorithm’s handling of those changes and roundoff. A stable algorithm cannot make an inherently sensitive problem insensitive.

Residuals are useful diagnostics, but their size must be interpreted through the problem’s scaling. A small residual alone is not a complete accuracy certificate.

Report what the calculation supports

Name the target, method, interval, step and assumptions behind any bound. Retain units and round once at the end. Distinguish an exact reference from another approximation.

When no rigorous bound is available, report refinement behaviour and supporting checks honestly. More elaborate computation does not justify stronger certainty by itself. The goal is to understand what the approximation supports, not to avoid approximation altogether.

Related topics

Numerical Differentiation and Numerical Integration supply finite formulas. Initial-Value Problems examines sensitivity, while Partial Derivatives extends local approximation to combined input changes.