Numerical integration and differentiation · Definition

Finite differences for derivatives

numerical methodsnumerical integration and differentiation

When a domain is discretized, the derivative stops being a limit and becomes a quotient of differences. This idea underlies many methods for ODEs and PDEs.

Definition

Finite differences are approximations of the derivatives of a function \(f\) from its values at discrete points separated by a step \(h\). They turn differential problems into algebraic systems that a computer can solve.

\[f'(x)\approx \frac{f(x+h)-f(x)}{h}\]

The forward, backward and centered formulas are the most used. By Taylor's theorem, the forward one has error \(O(h)\), while the centered one has error \(O(h^2)\) because the odd-order terms cancel.

Take \(f(x)=\sin x\) at \(x=1\) with \(h=0.1\). The forward formula gives \(\frac{\sin 1.1-\sin 1}{0.1}\approx 0.4974\), versus \(\cos 1\approx 0.5403\), while the centered one gives \(\frac{\sin 1.1-\sin 0.9}{0.2}\approx 0.5394\), much more accurate.

A typical mistake is reducing \(h\) indefinitely. For very small values, rounding in the subtraction of nearly equal numbers dominates the total error, so an optimal \(h\) exists.

Think about it

Why does the centered formula have error \(O(h^2)\) while the forward one only has \(O(h)\), if both use the same step \(h\)?

Because subtracting \(f(x+h)\) and \(f(x-h)\) cancels the odd-order terms of the Taylor expansion, while the forward formula keeps a term of order \(h\).

Exercises 10

  1. What is the forward finite difference formula for \(f'(x)\)?
  2. What is the backward finite difference formula for \(f'(x)\)?
  3. What is the centered finite difference formula for \(f'(x)\)?
  4. In finite differences, what does \(h\) represent?
  5. You have \(f(1)=2\) and \(f(1.1)=2.5\). What is the forward approximation of \(f'(1)\) with \(h=0.1\)?
  6. Let \(f(x)=x^2\). Compute the centered approximation of \(f'(2)\) with \(h=0.5\).
  7. If the step \(h\) is halved in the centered formula, how does the truncation error change approximately?
  8. Expanding \(f(x+h)=f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\cdots\), what is the leading term of the error in \(\frac{f(x+h)-f(x)}{h}-f'(x)\)?
  9. When a domain is discretized, what replaces the derivative in the finite difference method?
  10. With \(f(0)=1\), \(f(0.2)=1.44\) and \(f(-0.2)=0.84\), what is the centered approximation of \(f'(0)\) with \(h=0.2\)?

References

  • Richard L. Burden, J. Douglas Faires, Annette M. Burden, Numerical Analysis, 10th edition, Cengage Learning, 2016.
  • Lloyd N. Trefethen, David Bau III, Numerical Linear Algebra, 1st edition, SIAM, 1997.