The method of lines (MOL) for the diffusion equation

Chapter 1 The method of lines (MOL) for the diffusion equation The method of lines refers to an approximation of one or more partial differential equations with ordinary differential equations in just one of the independent variables. The assumption is the ordinary differential equations are easier to analyze and solve than the partial differential equations. The approximation can be based on finite differences, finite elements, collocation or Fourier series like expansions. A method of lines obtained with finite differences seems easiest to apply and will be used exclusively in this text. In order to introduce the basic ideas used later for pricing options let us consider the diffusion equation a(x, t)u xx + b(x, t)u x c(x, t)u d(x, t)u t = f(x, t) (1.1) and the boundary and initial conditions u(0, t) = α(t), u(x, 0) = u 0 (x). u(l, t) = β(t) Options, bonds and their Greeks are described by equations like (1.1). Consistent with these applications we shall make the assumption that all coefficients in (1.1) are continuous in x and t and that a(x, t) > 0 for 0 < x < L and all t, d(x, t) > 0 for 0 < x < L and all t. 1 c 2006 2009 Gunter H. Meyer

1.1. MOL WITH CONTINUOUS TIME CHAPTER 1. The coefficients b and c may change sign in specific problems. The two independent variables are x and t. In later applications the variable x will stand for an asset price or an interest rate. We shall call x the spatial variable. t stands for time, usually time to expiry of the option or bond. It is our choice whether to approximate this problem with ordinary differential equations in x or t. In subsequent chapters we shall always retain x as the continuous variable and discretize t. However, in the engineering literature the term MOL invariably refers to ordinary differential equations in t. This version is sometimes called the vertical method of lines []. For completeness, we shall give first a brief exposition of a method of lines approximation of (1.1) valid for all t and at discrete values of x, and then turn to the MOL in x. 1.1 The method of lines with continuous time (the vertical MOL) We define a mesh with 0 = x 0 < < x P = L x = L/M and x = i x, i = 0, 1,..., P, and approximate (1.1) along the line x = x i with the ordinary differential equation a(x i, t) u i+1(t) + u i 1 (t) 2u i (t) x 2 + b(x i, t) u i+1(t) u i 1 (t) c(x i, t)u i (t) u i 2 x (t) = f(x i, t) for i = 1,..., P 1, where u 0 (t) = α(t) and u M (t) = β(t), and where u i (0) = u 0 (x i ). It is possible to write these P 1 ordinary differential equations for the vector u(t) = (u 1 (t),...,u P 1 (t)) in matrix form u (t) = A(t) u + b(t) u(0) = u 0 (1.2) 2 c 2006 2009 Gunter H. Meyer

CHAPTER 1. 1.1. MOL WITH CONTINUOUS TIME where A(t) is a tridiagonal matrix and the vector b(t) is determined by the source term f(x, t) and the boundary conditions {α(t), β(t)}. The initial condition u 0 = (u 0,1,...u 0,P 1 ) is defined by u 0i = u 0 (x i ). Taylor expansions can be used to show that (1.2) is a consistent approximation of (1.1). Moreover, it can be shown that under reasonable hypotheses on the data of problem (1.1) the analytic solution of (1.2) satisfies u(x i, t) u i (t) K x 2, i = 1,...,P 1 where u(x, t) is the analytic solution of (1.1) and K depends on the smoothness of u and the interval of integration [0, T] []. Of course, in general an analytic solution of (1.2) is not available. Then the system must be solved numerically. It is possible to define a numerical integrator for (1.2), for example a backward Euler method or the trapezoidal rule, so that the resulting algebraic equations are identical to standard finite difference approximations to (1.1). In this case nothing has been gained by introducing the method of lines. The attraction of (1.2) is due to the fact that the theory of ordinary differential equations yields insight into the analytic solution of (1.2), and that many efficient adaptive black box numerical codes exist for the approximate solution of initial value problems for linear and nonlinear ordinary differential equations. Thus, in principle, a highly accurate numerical of solution of (1.2) is obtainable. Coupled with the fact that the MOL discretization is readily extended to nonlinear problems and to other classes of equations, it is natural that this two step solution approach to time dependent problems has become commonplace and the basis of software for the numerical solution of time dependent partial differential equations []. For a discussion of the method of lines leading to initial value problems for ordinary differential equations, and some caveats against its thoughtless application we refer the reader to []. The transformation of a so-called parabolic initial/boundary value problem like (1.1) to (1.2) is not so straightforward when the domain of the equation changes with time, e.g. when the boundary conditions are given in the form u(s(t), t) = α(t), u(s(t), t) = β(t) (1.3) where x = s(t) and x = S(t) describe non-constant boundaries. The problem is exacerbated when s(t) or S(t) is not known a priori but a free boundary 3 c 2006 2009 Gunter H. Meyer

1.2. MOL WITH CONTINUOUS x CHAPTER 1. like an early exercise boundary in an American option. It is possible to define an analog to (1.2) where {u i (t)} is an approximate solution along curves {x i (t)} (see the method of arbitrary lines [Xanthis]). Alternatively, one can map the irregular domain into a rectangle at the expense of complicating the differential equation (1.1). For this approach in a financial setting see [Tavella, Wu] where the resulting equations are solved with finite differences. We have no experience with the time-continuous MOL for financial applications and can give no insight into how well it solves, for example, American option problems. In this book, our goal is to work as much as possible with the original equation in the natural variables of the application. We believe that we can reach this goal readily by discretizing t instead of x. 1.2 The method of lines with continuous x (the horizontal MOL) For options with discontinuous pay-off, barriers, early exercise features or jumps, the solution of the Black Scholes equation can vary strongly with x at any given time. In this setting it is useful to base the method of lines on discretizing time and solving the resulting ordinary differential equation in x to a high degree of accuracy. This approach will be followed consistently throughout the remainder of this book. Assuming that problem (1.1) is to be solved over the time interval [0, T], where T is arbitrary but fixed (usually the time to expiry of the option), we introduce a partition in time where 0 = t 0 < t 1 < < t N = T t n = t n t n 1 is usually, but not necessarily, constant with respect to n. For a method of lines approximation of (1.1) at t = t n we simply replace u t by a time implicit (i.e. backward) difference quotient. Two approximations are used repeatedly. If u n (x) denotes the approximation of the solution u(x, t n ) then we employ either the backward Euler approximation u t (x, t n ) = Du n (x) = u n(x) u n 1 (x) t n (1.4) 4 c 2006 2009 Gunter H. Meyer

CHAPTER 1. 1.2. MOL WITH CONTINUOUS x or the three-level difference quotient u t (x, t n ) = Du n (x) = c n u n (x) u n 1 (x) t n + d n u n 1 (x) u n 2 (x) t n 1. (1.5) A Taylor series expansion of the right hand side of (1.5) shows that the weights {c n, d n } should be chosen such that ( ) ( ) ( ) 1 1 cn 1 t n 1 t n 2 t n 1 t n + t 2 =. n 1 d n 0 In most applications the time step is constant (i.e. t n = t n 1 ). For this case the weights are c n = 3 2, d n = 1 2. In general it follows from Taylor expansions applied to (1.4) that for any smooth function φ(x, t) φ t (x, t n ) Dφ(x, t n ) = K 1 t n so that the approximation of φ t is of order t, or of first order. Similarly it follows from (1.5) that φ t (x, t) Dφ(x, t n ) = K 2 max{ t 2 n, t2 n 1 } so that the approximation of φ t is of order t 2, i.e. of second order. Here K 1 and K 2 are constants which depend only on the smoothness of φ. The method of lines approximation of (1.1) at time t = t n is given by the two point boundary value problem a(x, t n )u n(x)+b(x, t n )u n(x) c(x, t n )u n (x) d(x, t n )Du n (x) = f(x, t n ) (1.6) subject to u n (0) = α(t n ), u n (L) = β(t n ). u 0 (x) = g(x) is the given initial condition. It is well known that standard finite difference methods based on a backward Euler or an implicit three-level time discretization are unconditionally stable. As the spatial mesh parameter in the finite difference approximation decreases to zero the equation (1.6) results. Hence we may infer that both (1.4) and (1.5) yield a stable numerical method for the solution of the diffusion equation (1.1). A direct proof of 5 c 2006 2009 Gunter H. Meyer

1.2. MOL WITH CONTINUOUS x CHAPTER 1. stability is provided in Appendix 1.1. We note that at the first time level t = t 1 the expression is only defined for the backward Euler quotient (1.4). Once u 1 (x) is found then for n 2 the equation (1.6) is defined for either difference quotient Du n (x). If equation (1.1) is to be solved subject to boundary data like (1.3) then the boundary conditions for (1.6) simply become u n (s(t n )) = α(t n ), u n (S(t n )) = β(t n ). However, if the interval [s(t n 1 ), S(t n 1 )] does not contain the interval [s(t n ), S(t n )] then u n 1 (x) must be extended beyond its boundary points in order to be able to define Du n (x) on [s(t n ), S(t n )]. For example, if S(t n ) > S(t n 1 ) then we would use the (smooth pasting) linear extension u n 1 (x) = u n 1 (S(t n 1 ) + u n 1 (S(t n 1)(x S(t n 1 )), x > S(t n 1 ). Analogous extensions apply to u n 2 (x) and at the lower boundary s(t). The accuracy of the approximation of u t generally implies the same accuracy for the solution of (1.1), meaning that for an analytic solution of (1.6) we can assert that u(x, t n ) u n (x) K max t 1 k or 2 k where K depends solely on the smoothness of the analytic solution u(x, t) of (1.1). A second order method is preferable, and in fact essential for an efficient MOL code for problems in finance. It comes basically for free because numerical methods for (1.6) differ little whether (1.4) or (1.5) is used. Numerical experiments verify that the performance of the method of lines is greatly improved by switching over to a second order method for n 3. In applications we shall routinely approximate u t (x, t) with the backward quotient (1.4) for n = 1 and n = 2, and then change over to the three level quotient (1.5) for n 3. The reader familiar with the Crank-Nicolson method for the diffusion equation will be aware that one could equally well define an MOL approximation involving the spatial differential operator at times t n 1 and t n. However, the Crank-Nicolson approximation requires care when initial data are non-smooth (typical for options) or the boundary and initial data are discontinuous (typical for barrier options). In fact, even though self-starting in principle, it often has to be combined with a few implicit Euler steps (sometimes referred to as Rannacher s method) to suppress unwarranted initial 6 c 2006 2009 Gunter H. Meyer

CHAPTER 1. 1.3. MOL FOR MULTI-DIMENSIONAL PROBLEMS oscillations. Hence the self-starting feature is lost. The three level scheme appears simpler to apply and to give comparable results. For this reason a Crank-Nicolson based MOL will not be considered here. 1.3 The method of lines with continuous x for multi-dimensional problems Asian and basket options, stochastic volatility models and multi-factor interest rate bond options lead to multi-dimensional diffusion equations. For such equations the simplicity of the MOL described in Section 1.2 is retained, in principle, if we combine the MOL with an analog of standard line iterative methods known for algebraic approximations of multi-dimensional diffusion equations. In practice, such approach is limited to very low-dimensional problems, and we shall treat applications which involve only two spatial variables x and y. We shall consider problems of the form a 11 (x, y, t)u xx + a 12 (x, y, t)u xy + a 22 (x, y, t)u yy (1.7) + b 1 (x, y, t)u x + b 2 (x, y, t)u y c(x, y, t)u d(x, y, t)u t = f(x, y, t) on a domain D(t) = {(x, y) : s(y, t) < x < S(y, t), y 0 < y < Y }, t 0 where y 0 and Y are given numbers and s and S are smooth curves in y and t. The boundary of D(t) will be denoted by D(t). A schematic of D(t) is given in Fig.?? In applications we impose boundary conditions on u(x, y, t) on all or part of the boundary D(t), and an initial condition on u(x, y, 0). For definiteness we shall use so-called Dirichlet boundary data u(x, y, t) = α(x, y, t), (x, y) D(t) and the initial condition where α and u 0 are given functions. u(x, y, 0) = u 0 (x, y) 7 c 2006 2009 Gunter H. Meyer

1.3. MOL FOR MULTI-DIMENSIONAL PROBLEMS CHAPTER 1. As in the preceding section we begin by discretizing time in (1.7). Let u n (x, y) denote an approximation to u(x, y, t n ) at time t n. We employ the difference quotients introduced in Section 1.2 u t (x, y, t n ) Du n (x, y) = { un u n 1 3 2 for n = 1, 2 t for n = 3,...,N u n u n 1 t 1 2 u n 1 u n 2 t where t = t n t n 1 is assumed to be constant for all n. For notational convenience the argument (x, y) of u n will generally be suppressed. Moreover, if the time index n does not appear explicitly it is assumed to be the index n of the latest time step so that u(x, y) u n (x, y). The time index will be written only when it is needed for clarity. The (parabolic) initial/boundary value problem for (1.7) at time t n is approximated by the sequence of time discrete (elliptic) problems a 11 (x, y, t n )u xx + a 12 (x, y, t n )u xy + a 22 (x, y, t n )u yy + b 1 (x, y, t n )u x (1.8) + b 2 (x, y, t n )u y ĉ(x, y, t n )u = ˆf(x, y, t n ) where and ĉ(x, y, t n ) = c(x, y, t n ) + { d(x, y, t n ) 1 t for n = 1, 2 d(x, y, t n ) 3 2 t for n = 3,...,N ˆf(x, y, t n ) = f(x, y, t n ) { d(x, y, tn ) u n 1(x,y) for n = 1, 2 t ] d(x, y, t n ) for n = 3,...,N [ 3un 1 (x,y) 2 t + u n 1(x,y) u n 2 (x,y) 2 t u 0 (x, y) is given and the boundary data are evaluated at t n. In order to approximate (1.8) with ordinary differential equations in x we impose a uniform partition y 0 < y 1 < < y M = Y The points {y m } of the partition will always be indexed by m. Next we replace derivatives with respect to y in (1.8) by central difference quotients. 8 c 2006 2009 Gunter H. Meyer

CHAPTER 1. 1.3. MOL FOR MULTI-DIMENSIONAL PROBLEMS If u m (x) ( u m,n (x)) denotes an approximation to u(x, y m, t n ) then with y = y m y m 1 we write u y (x, y m, t n ) = u m+1(x) u m 1 (x) 2 y u yy (x, y m, t n ) = u m+1(x) + u m 1 (x) 2u m (x) y 2 u xy (x, y m, t n ) = u m+1 (x) u m 1 (x) 2 y The method of lines approximation to (1.7) with continuous x at discrete time t n along the line y = y m for m = 1,...,M 1 takes on the form Lu(x) = a 11 (x, y m, t n )u m (x) + b 1(x, y m, t n )u m (x) c(x, y m, t n )u m (x) (1.9) = F(x, y m, t n, u m 1 (x), u m+1 (x), u m 1 (x), u m+1 (x), u m,n 1 (x), u m,n 2 (x)) where and c(x, y m, t n ) = ĉ(x, y m, t n ) + a 22 (x, y m, t n ) 2 y 2 F(x, y m, t n ) = ˆf(x, y m, t n ) a 22 (x, y m, t n ) u m+1(x) + u m 1 (x) y 2 b 2 (x, y m, t n ) u m+1(x) u m 1 (x) 2 y a 12 (x, y m, t n ) u m+1(x) u m 1(x) 2 y (We recall the notation: u m = u(x, ym, t n ), u m,n 1 (x) = u(x, y m, t n 1 ).) The initial condition yields {u m,0 (x)} M m=0 and the boundary conditions give {u m (s(y m, t n ))} and {u m (S(y m, t n ))}, u 0 (x), u M (x) and their derivatives. Equations (1.9) represent a boundary value problem for a system of M 1 coupled ordinary differential equations. If s(y, t n ) or S(y, t n ) is not constant with respect to y then the equations are defined over different x-intervals and the boundary value problem is a so-called multipoint boundary value problem. Its numerical solution is often as complicated as the numerical solution of the partial differential equation (1.8). However, for problems 9 c 2006 2009 Gunter H. Meyer

1.3. MOL FOR MULTI-DIMENSIONAL PROBLEMS CHAPTER 1. arising in finance it generally is possible to solve the system (1.9) iteratively as a sequence of scalar second order equations like (1.6). We introduce a new index k for the iteration count. It appears as a superscript of u so that u k m (x) stands for the kth iterate approximating u(x, y m, t n ). At time t n let {u 0 m(x)} M 1 m=1 denote an initial guess for the solution {u m (x)} of (1.9). (Typically one would choose the solution from the preceding time step u 0 m(x) = u m,n 1 (x).) In the kth iteration for k 1 we compute a solution {u k m (x)} for m = 1,...,M 1 by solving L m u k m (x) = F(x, y m, t n, u k m 1, uk 1 m+1, u k m 1, u k 1 m+1, u m,n 1, u m,n 2 ). (1.10) Note that at any stage the right hand side F is a known source term. Assuming that the sequence {u k m (x)} converges as k we obtain u m,n (x) = lim u k m (x) for m = 1,...,M 1. k The reader familiar with iterative methods for linear algebraic systems will recognize that our iterative method is a line Gauss-Seidel iteration, except that along each line y = y m a two point boundary value problem for a second order ordinary differential equation instead of a matrix problem has to be solved. In practice we set where K is an integer such that u m,n (x) = u K m (x) max m uk m(x) u K 1 m (x) ǫ. The choice of the convergence tolerance ǫ is dictated by the accuracy required by the application and the computer time it takes for convergence. If a a 12 (x, y, t) 0 then it is possible to prove that the line Gauss Seidel method will indeed converge to a solution of the multipoint problem at each time level []. Numerical experience suggests that the iteration will also converge for problems in finance where a 12 0. On occasion spurious oscillatory solutions can arise from the central difference approximation of u y (x, y, t). They can be suppressed by either decreasing y, or if that is infeasible, by replacing b 2 (x, y m, t n )u y (x, y m, t n ) 10 c 2006 2009 Gunter H. Meyer

CHAPTER 1. 1.3. MOL FOR MULTI-DIMENSIONAL PROBLEMS with a lower order one-sided difference quotient. For example, if b 2 0 we would choose u y (x, y m, t n ) = u m+1 u, y if b 2 < 0 we would use u y (x, y m, t n ) = u u m 1 y These first order quotients represent an upwinding of the convection term b 2 u y in (1.7). Upwinding is a well understood and important tool in the theory of numerical methods for partial differential equations []. We also remark that if t is very large then the convergence of the line Gauss Seidel iteration may be unacceptably slow. In this case a line SOR modification of the Gauss Seidel method may prove helpful. We consider the solution of (1.10) an intermediate solution and denote it by ũ. The desired solution u k m is found from u k m = uk 1 m + ω[ũ m u k 1 m ] for some ω [1, 2]. The optimum relaxation factor ω is not known a priori but can be found by trial and error. ω = 1 yields again the Gauss Seidel iteration. Finally, we remark that it is straightforward to generalize this approach to multi-dimensional equations like. p a ij u xi x j + i,j=1 p b i u xi cu du t = f( x, t) i=1 where x = (x 1,...,x p ). Such equation would arise, for example, in connection with basket options depending on p assets. We discretize t and then x 2,...,x p and replace all derivates with respect to these variables by appropriate finite difference quotients. A system of the order of M p 1 coupled second order ordinary differential equations in the independent variable x 1 results to which a line Gauss Seidel iteration can be applied. Some parallelization of the line Gauss Seidel method is possible, but we have no practical experience with financial applications for p > 2. In summary then, the method of lines appximation for the parabolic equation describing financial derivatives is reduced at each time level to a 11 c 2006 2009 Gunter H. Meyer

1.3. MOL FOR MULTI-DIMENSIONAL PROBLEMS CHAPTER 1. single linear second order differential equation (1.6), or to a sequence of such equations. This is a sensible approach only if such second order equation can be solved accurately and efficiently. As stated in Section 1.1, a standard approach would be to replace the derivatives in (1.6) by their finite difference analogs and to set up a matrix system. The resulting algorithm is again a standard finite difference algorithm for the diffusion equation found in every textbook on numerical methods for partial differential equations. Such method would not merit the detour via the method of lines approximation. Here we shall employ a less familiar algorithm for the solution of (1.6), but one which is readily adapted to the class of problems associated with American options. This algorithm is the subject of the next chapter. 12 c 2006 2009 Gunter H. Meyer