Approximation of derivatives using finite difference methods

This post is part of a series of articles on the Finite Difference Method, with other posts in the series focusing on solving the heat equation/diffusion equation explicitly, the implicit Crank-Nicolson method, and solving tridiagonal matrix systems using Thomas's algorithm.

• Approximation of derivatives using finite difference methods
• Explicit method for solving the diffusion equation
• Implicit Crank-Nicolson method
• Solving a tridiagonal matrix system using Thomas's algorithm

This is the first in a multi-part tutorial series on using Finite Difference Methods (FDM) to numerically solve parabolic partial differential equations. In particular, the heat equation will be solved. Given that the Black-Scholes pricing equation can be transformed into a heat equation, these methods can be applied to solve the equation numerically.

If we let $f\in C^2(\mathbb{R})$, i.e. let f be a continuous function of the real numbers which is twice differentiable, then the heat equation for f is given by:

$$\frac{\partial f}{\partial t}=\frac{{\partial }^{2}f}{\partial x^2}$$

Discretising Derivatives

The Finite Difference Method provides a numerical solution to this equation via the discretisation of its derivatives. The derivatives will be approximated via a Taylor Series expansion. Recall that a Taylor Series provides a value for a function $f=f(x)$ when the dependent variable $x\in \mathbb{R}$ is translated by an amount $\mathrm{\Delta }x$, in terms of its derivatives at that point. Thus, $f(x+\mathrm{\Delta }x)$ can be calculated by:

$$f(x+\mathrm{\Delta }x)=f(x)+f^{\prime }(x)\mathrm{\Delta }x+\frac{f^{″}(x)(\mathrm{\Delta }x{)}^2}{2!}+\frac{f^{‴}(x)(\mathrm{\Delta }x{)}^3}{3!}+O(\mathrm{\Delta }x{)}^4$$

Note that $O(\mathrm{\Delta }x)$ represents additional terms that are bounded by a constant term multipled by $\mathrm{\Delta }x$.

In a similar manner, we can determine f when the dependent variable x is translated by an identical amount $\mathrm{\Delta }x$ in the opposite direction:

$$f(x-\mathrm{\Delta }x)=f(x)-f^{\prime }(x)\mathrm{\Delta }x+\frac{f^{″}(x)(\mathrm{\Delta }x{)}^2}{2!}-\frac{f^{‴}(x)(\mathrm{\Delta }x{)}^3}{3!}+O(\mathrm{\Delta }x{)}^4$$

These two equations when rearranged allow a first order approximation for the derivative $f^{\prime }(x)$:

$$\begin{array}{rl}{f}^{\prime }(x)& =\frac{f(x+\mathrm{\Delta }x)-f(x)}{\mathrm{\Delta }x}+O(\mathrm{\Delta }x)\\ & =\frac{f(x)-f(x-\mathrm{\Delta }x)}{\mathrm{\Delta }x}+O(\mathrm{\Delta }x)\end{array}$$

In addition, subtracting the backward Taylor expansion from the forward Taylor expansion provides a second order approximation for the derivative $f^{\prime }(x)$:

$$f^{\prime }(x)=\frac{f(x+\mathrm{\Delta }x)-f(x-\mathrm{\Delta }x)}{2\mathrm{\Delta }x}+O(\mathrm{\Delta }x{)}^2$$

To obtain a second order approximation for the second derivative of $f$, $f^{″}(x)$, we can add the forward and backward Taylor expansions to receive:

$$f^{″}(x)=\frac{f(x+\mathrm{\Delta }x)-2f(x)+f(x-\mathrm{\Delta }x)}{(\mathrm{\Delta }x{)}^2}+O(\mathrm{\Delta }x{)}^2$$

Domain Discretisation

Now that the derivatives have been approximated in a one-dimensional continuous setting, it is time to discretise the domain of the heat equation. The function f is now a function of two real variables x and t which represent space and time, respectively. The function $f=f(x,t)$ can be thought of as the quantity of heat at any point along a one-dimensional bar.

The 1D continuous bar can be discretised into I separate points $1,2,...,i,...,I$. Time can be discretised into N separate points $1,2,...,n,...,N$, each separated by a time-spacing $\mathrm{\Delta }t$. The approximation $f_i^n$ to $f$ at time n and location $x=i\mathrm{\Delta }x$ is given by:

$$f_i^n\approx f(i\mathrm{\Delta }x,n\mathrm{\Delta }t)$$

Note that the superscript n does not refer to a power of f or its nth derivative, instead it refers to its value at the time $n\mathrm{\Delta }t$. Thus, the aforementioned Taylor approximations to the derivatives can be written as:

$$\begin{array}{rl}{f}^{\prime }(i\mathrm{\Delta }x,n\mathrm{\Delta }t)& \approx \frac{f_{i+1}^n-f_i^n}{\mathrm{\Delta }x}\\ & \approx \frac{f_i^n-f_{i-1}^n}{\mathrm{\Delta }x}\\ & \approx \frac{f_{i+1}^n-f_{i-1}^n}{2\mathrm{\Delta }x}\\ f^{″}(i\mathrm{\Delta }x,n\mathrm{\Delta }t)& \approx \frac{f_{i+1}^n-2f_i^n+f_{i-1}^n}{(\mathrm{\Delta }x{)}^2}\end{array}$$

They are, respectively, the forward difference, backward difference, centered difference and centered second difference.

Errors and Conclusion

Given that the differences are approximations it is natural to question how errors can arise in the overall solution. In fact, two types of errors can occur. They are the truncation error and the roundoff error. Truncation error is the error associated with the discretisation of the derivatives, i.e. those occuring due to the additional higher order derivative terms.

Roundoff error is that which occurs due to the precision of the solution being stored on a computer. Usually double precision floating point values are used which store 16 digits for each value. These errors can accumulate, particularly if single precision is used, which only stores 8 digits for floating point values.

In the next tutorial the heat equation will be solved numerically. It will be seen how truncation error can rapidly accumulate if certain conditions are not enforced.

Reference: Derivative Approximation via Finite Difference Methods

From https://www.quantstart.com/articles/Derivative-Approximation-via-Finite-Difference-Methods/