Numerical Analysis by Dr. Anita Pal Assistant Professor Department of Mathematics National Institute of Technology Durgapur Durgapur-71309 email: anita.buie@gmail.com 1
. Chapter 8 Numerical Solution of Ordinary Differential Equations Module No. 1 Runge-Kutta Methods
...................................................................................... One of the most useful tools of applied mathematics is differential equation, it may be ordinary differential equation (ODE) or partial differential equation (PDE). Both ODE and PDE are widely used to model lots of mathematical and engineering problems. Unfortunately, it is not possible to find analytical solution of all ODEs or PDEs. Finding of analytic solution of an ODE or a PDE is a very difficult task. But, several numerical techniques are available to solve ODEs and PDEs. Let us consider the following first order differential equation with initial condition dy = f(x, y) (1.1) dx y(x 0 ) = y 0. (1.) If the analytic solution of an ODE is not available then we will go for numerical solution. The numerical solution of a differential equation can be determined in one of the following two forms: (i) A power series solution for y in terms of x. Then the values of y can be obtained by substituting x = x 0, x 1,..., x n. (ii) A set of tabulated values of y for x = x 0, x 1,..., x n with spacing h. The general solution of an ODE of order n contains n arbitrary constants. To find such constants, n conditions are required. The problems in which all the conditions are specified at the initial point only, are called initial value problems (IVPs). The ODEs of order two or more and for which the conditions are specified at two or more points are called boundary value problems (BVPs). There may not exist a solution of an ordinary differential equation always. sufficient condition for existence of unique solution is provided by Lipschitz. The methods to find approximate solution of an initial value problem are called finite difference methods or discrete variable methods. The solutions are determined at a set of discrete points called a grid or mesh of points. That is, a given differential equation is converted to a discrete algebraic equation. By solving such equation, we get an approximate solution of the given differential equation. The 1
........................................................... Runge-Kutta Methods A finite difference method is convergent if the solution of the finite difference equation approaches to a limit as the size of the grid spacing tends to zero. But, in general, there is no guarantee, that this limit corresponds to the exact solution of the differential equation. Mainly, two types of numerical methods are used, viz. explicit method and implicit method. If the value of y i+1 depends only on the values of y i, h and f(x i, y i ) then the method is called explicit method, otherwise the method is called implicit method. In this chapter, we discuss some useful methods to solve ODEs. 1.1 Euler s method Euler s method is the most simple but crude method to solve the differential equation of the form dy dx = f(x, y), y(x 0) = y 0. (1.3) Let x 1 = x 0 + h, where h is small. Then by Taylor s series ( dy y 1 = y(x 0 + h) = y 0 + h + dx )x0 h ( d y ) dx, c 1 where c 1 lies between x 0 and x Thus, by neglecting second order term, we get = y 0 + hf(x 0, y 0 ) + h y (c 1 ) (1.4) y 1 = y 0 + hf(x 0, y 0 ). (1.5) In general, y n+1 = y n + hf(x n, y n ), n = 0, 1,,... (1.6) This is a very slow method. To find a reasonable accurate solution, the value of h must be taken as small. The Euler s method is less efficient in practical problems because if h is not sufficiently small then this method gives inaccurate result. This method is modified to get more better result as follows: The value of y 1 obtained by Euler s method is repeatedly modified to get a more accurate result. For this purpose, the term f(x 0, y 0 ) is replaced by the average of f(x 0, y 0 ) and f(x 0 + h, y (0) 1 ), where y(0) 1 = y 0 + hf(x 0, y 0 ).
1.1. Euler s method.............................................................. That is, the modified Euler s method is y (1) 1 = y 0 + h [f(x 0, y 0 ) + f(x 1, y (0) 1 )] (1.7) where y (0) 1 = y 0 + hf(x 0, y 0 ). Note that the value of y 1 depends on the value of y 1 obtained in previous iteration. So this method is known as implicit method, whereas Euler method is explicit method. Other simple methods, viz. Taylor s series, Picard, Runge-Kutta, etc. are also available to solve the differential equation of the form (1.1). Now, we describe Runge-Kutta method. In this method, several function evaluations are required at each step and it avoids the computation of higher order derivatives. There are several types of Runge-Kutta methods, such as second, third, fourth, fifth, etc. order. The fourth-order Runge-Kutta method is more popular. These are singlestep explicit methods. 1. Second-order Runge-Kutta method First we deduce second order Runge-Kutta method from modified Euler s method. The modified Euler s method is y 1 = y 0 + h [f(x 0, y 0 ) + f(x 1, y (0) 1 )] (1.8) where y (0) 1 = y 0 + hf(x 0, y 0 ). Now, we substitute the value of y (0) 1 to the equation (1.8). Then Let y 1 = y 0 + h [f(x 0, y 0 ) + f(x 0 + h, y 0 + hf(x 0, y 0 ))]. k 1 = hf(x 0, y 0 ) and k = hf(x 0 + h, y 0 + hf(x 0, y 0 )) = hf(x 0 + h, y 0 + k 1 ). (1.9) Using this notation the equation (1.8) becomes y 1 = y 0 + 1 (k 1 + k ). (1.10) 3
........................................................... Runge-Kutta Methods This method is known as second-order Runge-Kutta method. Note that this is an explicit formula, whereas modified Euler s method is an implicit method. The local truncation error of this method is O(h 3 ). This formula can also be derived independently. Independent derivation of second-order Runge-Kutta method Suppose the solution of the equation is of the following form: where By Taylor s theorem k 1 = hf(x 0, y 0 ) and dy dx = f(x, y), y(x 0) = y 0 (1.11) y 1 = y 0 + ak 1 + bk, (1.1) k = hf(x 0 + αh, y 0 + βk 1 ), a, b, α and β are constants. y 1 = y(x 0 + h) = y 0 + hy 0 + h 0 + = y 0 + hf(x 0, y 0 ) + h [( f ) ( f ) ] x + f(x 0, y 0 ) + O(h 3 ) (x 0,y 0 ) y (x 0,y 0 ) [ since df dx = f ] + f(x, y) f x y k = hf(x 0 + αh, y 0 + βk 1 ) [ ( f ) ( f ) ] = h f(x 0, y 0 ) + αh x + βk 1 (x 0,y 0 ) y + (x 0,y 0 ) O(h ) = hf(x 0, y 0 ) + αh ( f ) ( f ) x + (x 0,y 0 ) βh f(x 0, y 0 ) y + (x 0,y 0 ) O(h3 ). y 0 + h3 6 y Substituting these values to the equation (1.1), we get 4 y 0 + hf(x 0, y 0 ) + h [f x(x 0, y 0 ) + f(x 0, y 0 )f y (x 0, y 0 )] + O(h 3 ) = y 0 + (a + b)hf(x 0, y 0 ) + bh [αf x (x 0, y 0 ) + βf(x 0, y 0 )f y (x 0, y 0 )] + O(h 3 ).
1.1. Euler s method.............................................................. Equating the coefficients of f, f x and f y both sides we get the following equations. a + b = 1, bα = 1 and bβ = 1. (1.13) Obviously, α = β. Here, number of equations is three and variables is four. Therefore, the system of equations has many solutions. However, usually the parameters are chosen as α = β = 1, then a = b = 1. For this set of parameters, the equation (1.1) becomes y 1 = y 0 + 1 (k 1 + k ) + O(h 3 ), (1.14) where k 1 = hf(x 0, y 0 ) and k = hf(x 0 + h, y 0 + k 1 ). (1.15) 1.3 Fourth-order Runge-Kutta method The fourth-order Runge-Kutta formula to calculate y 1 is y 1 = y 0 + 1 6 (k 1 + k + k 3 + k 4 ) (1.16) where k 1 = hf(x 0, y 0 ) k = hf(x 0 + h/, y 0 + k 1 /) k 3 = hf(x 0 + h/, y 0 + k /) k 4 = hf(x 0 + h, y 0 + k 3 ). Using the value of y 1 one can find the value of y as y = y 1 + 1 6 (k 1 + k + k 3 + k 4 ) (1.17) where k 1 = hf(x 1, y 1 ) k = hf(x 1 + h/, y 1 + k 1 /) k 3 = hf(x 1 + h/, y 1 + k /) k 4 = hf(x 1 + h, y 1 + k 3 ). 5
........................................................... Runge-Kutta Methods In general, y i+1 = y i + 1 6 (k(i) 1 + + 3 + 4 ) (1.18) where 1 = hf(x i, y i ) = hf(x i + h/, y i + 1 /) 3 = hf(x i + h/, y i + /) 4 = hf(x i + h, y i + 3 ), for i = 0, 1,,.... Example 1.1 Given y = x y with x = 0, y = 1. Find y(0.) by second and fourthorder Runge-Kutta methods. Solution. Here, x 0 = 0, y 0 = 1, f(x, y) = x y. By Second order Runge-Kutta method Let h = 0.1. k 1 = hf(x 0, y 0 ) = 0.1 (0 1 ) = 0.10000. k = hf(x 0 + h, y 0 + k 1 ) = 0.1 f(0.1, 0.9) = 0.07100. Now, we calculate, y 1 = y(0.1) = y(0) + 1 (k 1 + k ) = 1 + 0.5 ( 0.1 0.071) = 0.91450. To determine y = y(0.), x 1 = 0.1 and y 1 = 0.91450. k 1 = hf(x 1, y 1 ) = 0.1 {0.1 (0.91450) } = 0.073633. k = hf(x 1 + h, y 1 + k 1 ) = 0.1 [0. (0.9145 0.07363) ] = 0.05071. Therefore, y = y(0.) = y(0.1) + 1 (k 1 + k ) = 0.91450 + 0.5 ( 0.07363 0.05071) = 0.8533. 6
1.1. Euler s method.............................................................. By fourth-order Runge-Kutta methods: Here h = 0.1, x 0 = 0, y 0 = 1, f(x, y) = x y. Then, k 1 = hf(x 0, y 0 ) = 0.1 (0 (1) ) = 0.1. Therefore, k = hf(x 0 + h/, y 0 + k 1 /) = 0.1 {( 0.1 0.1 ) (1 ) } = 0.0855. k 3 = hf(x 0 + h/, y 0 + k /) = 0.1 {( 0.1 k 4 = 0.1 {( 0.1 0.0855 ) (1 ) } = 0.08666. 0.08666 ) (1 ) } = 0.0734. y 1 = y(0.1) = y 0 + 1 6 (k 1 + k + k 3 + k 4 ) = 1 + 1 [ 0.1000 + {( 0.0855) + ( 0.08666)} 0.0734] = 0.91379. 6 To find, y = y(0.), h = 0.1, x 1 = 0.1, y 1 = 0.91379. Then, k 1 = hf(x 1, y 1 ) = 0.1 {0.1 (0.91379) } = 0.07350. k = hf(x 1 + h/, y 1 + k 1 /) = 0.1 {(0.1 + 0.1 0.07350 ) (0.91379 ) } = 0.0619. k 3 = hf(x 1 + h/, y 1 + k /) = 0.1 {(0.1 + 0.1 0.0619 ) (0.91379 ) } = 0.0694. k 4 = hf(x 1 + h, y 1 + k 3 ) = 0.1 {(0.1 + 0.1) (0.91379 0.0694) } = 0.0539. Therefore, y = y(0.) = y(0) + 1 6 (k 1 + k + k 3 + k 4 ) = 0.91379 + 1 [ 0.07350 + {( 0.0619) + ( 0.0694)} 0.0539] 6 = 0.85119. 7
........................................................... Runge-Kutta Methods Note 1.1 The fourth order Runge-Kutta method gives better result, though, this method has some disadvantages. In this method, a lot of function calculations are required. Thus, if the function f(x, y) is complicated, then the Runge-Kutta method is very laborious. Error in fourth-order Runge-Kutta method The fourth-order Runge-Kutta formula can be written as y 1 = y 0 + (h/) [ k 1 + 4(k + k 3 ) + k 4 ]. 3 This form is similar to the Simpson s 1/3 formula with step size h/. Thus, the local truncation error of this formula is h5 880 yiv (c 1 ), i.e. of O(h 5 ). After n steps, the accumulated error is n i=1 h 5 880 yiv (c i ) = x n x 0 5760 yiv (c)h 4 = O(h 4 ). 1.4 Runge-Kutta method for a pair of equations The second and fourth-order Runge-Kutta methods can also used to solve a pair of first order differential equations. Let us consider a pair of first-order differential equations dy = f(x, y, z) dx dz = g(x, y, z) dx (1.19) with initial conditions x = x 0, y(x 0 ) = y 0, z(x 0 ) = z 0. (1.0) Here, x is the independent variable and y, z are the dependent variables. So, in this problem we have to determine the values of y, z for different values of x. 8
1.1. Euler s method.............................................................. The fourth-order Runge-Kutta method to find the values of y i and z i for x = x i from a pair of equations is y i+1 = y i + 1 6 [k(i) 1 + + 3 + 4 ], z i+1 = z i + 1 6 [l(i) 1 + l(i) + l(i) 3 + l(i) 4 ], (1.1) where 1 = hf(x i, y i, z i ) l (i) 1 = hg(x i, y i, z i ) = hf(x i + h/, y i + 1 /, z i + l (i) 1 /) l (i) = hg(x i + h/, y i + 1 /, z i + l (i) 1 /) 3 = hf(x i + h/, y i + /, z i + l (i) /) l (i) 3 = hg(x i + h/, y i + /, z i + l (i) /) 4 = hf(x i + h, y i + 3, z i + l (i) 3 ) l (i) 4 = hg(x i + h, y i + 3, z i + l (i) 3 ). (1.) Example 1. Solve the following pair of differential equations dy dz = z and = y + xz at x = 0. given y(0) = 1, z(0) = 0. dx dx Solution. Let h = 0.. Here f(x, y, z) = z, g(x, y, z) = y + xz. We calculate the values of k 1, k, k 3, k 4 ; l 1, l, l 3, l 4 as follows. k 1 = hf(x 0, y 0, z 0 ) = 0. 0 = 0 l 1 = hg(x 0, y 0, z 0 ) = 0. (1 + 0) = 0. k = hf(x 0 + h/, y 0 + k 1 /, z 0 + l 1 /) = 0. 0. = 0.0 l = hg(x 0 + h/, y 0 + k 1 /, z 0 + l 1 /) = 0. [(1 + 0) + (0 + 0. = 0. (1 + 0.1 + 0.1) = 0.00 ) (0 + 0. )] k 3 = hf(x 0 + h/, y 0 + k /, z 0 + l /) = 0. 0.0 = 0.000 l 3 = hg(x 0 + h/, y 0 + k /, z 0 + l /) = 0. [(1 + 0.0 0. 0.00 ) + (0 + ) (0 + )] = 0.040 k 4 = hf(x 0 + h, y 0 + k 3, z 0 + l 3 ) = 0. 0.040 = 0.04080 l 4 = hg(x 0 + h, y 0 + k 3, z 0 + l 3 ) = 0. [(1 + 0.000) + (0 + 0.) (0 + 0.040)] = 0.10. 9
........................................................... Runge-Kutta Methods Hence, y(0.) = y 1 = y(0) + 1 6 [k 1 + (k + k 3 ) + k 4 ] = 1 + 1 [0 + (0.0 + 0.00) + 0.0408] = 1.00. 6 z(0.) = z 1 = z(0) + 1 6 [l 1 + (l + l 3 ) + l 4 ] = 0 + 1 [0. + (0.00 + 0.040) + 0.10] = 0.0404. 6 1.5 Runge-Kutta method for a system of equations The above method can be extended to solve a system of first order differential equations. Let us consider the following system of first order differential equations with initial conditions dy 1 dx = f 1(x, y 1, y,..., y n ) dy dx = f (x, y 1, y,..., y n ) (1.3) dy n dx = f n(x, y 1, y,..., y n ) y 1 (x 0 ) = y 10, y (x 0 ) = y 0,..., y n (x 0 ) = y n0. (1.4) In this problem, x is the independent variable and y 1, y,..., y n are n dependent variables. The above system of equations can be written as a vector notation as follows: dy dx = f(x, y 1, y,..., y n ) with y(x 0 ) = y 0. (1.5) The fourth-order Runge-Kutta method for the equation (1.5) is y j+1 = y j + 1 6 [k 1 + k + k 3 + k 4 ] (1.6) where 10 k 1 = k 11 k 1. k n1, k = k 1 k. k n, k 3 = k 13 k 3. k n3, k 4 = k 14 k 4. k n4 (1.7)
1.1. Euler s method.............................................................. and k i1 = hf i (x j, y 1j, y j,..., y nj ) k i = hf i (x j + h/, y 1j + k 11 /, y j + k 1 /,..., y nj + k n1 /) k i3 = hf i (x j + h/, y 1j + k 1 /, y j + k /,..., y nj + k n /) k i4 = hf i (x j + h, y 1j + k 13, y j + k 3,..., y nj + k n3 ) for i = 1,,..., n. Here, y kj is the value of the variable y k evaluated at x j. 1.6 Runge-Kutta method for second order IVP The Runge-Kutta methods can be used to solve second order IVP by converting the IVP to a pair of first order differential equations. Let the second order differential equation be and the initial conditions be To convert it to first order equation, let a(x)y + b(x)y + c(x)y = f(x) (1.8) x = x 0, y(x 0 ) = y 0, y (x 0 ) = z 0. (1.9) y (x) = z(x). (1.30) Then y (x) = z (x). Therefore, the equation (1.9) reduces to with initial conditions dy dx = z dz dx = 1 [f(x) b(x)z c(x)y] = g(x, y, z) (1.31) a(x) x = x 0, y(x 0 ) = y 0, z(x 0 ) = z 0. (1.3) This is a pair of first order differential equations. Now, Runge-Kutta methods may be used to solve this pair of equations (1.31) with initial conditions (1.3). The value of y is the solution of the given IVP and z is the first order derivatives. 11
........................................................... Runge-Kutta Methods Example 1.3 Find the value of y(0.1) for the following second order differential equation by fourth-order Runge-Kutta method. y (x) y (x) + y(x) = x + 1 with y(0) = 1, y (0) = 0. Solution. Substituting y = z. Then the given equation reduces to z z + y = x + 1, or z = z y + x + 1 = g(x, y, z) (say). Therefore, y = z and z = (z y + x + 1). Here, x 0 = 0, y 0 = 1, z 0 = 0, h = 0.1. k 1 = h z 0 = 0.1 0 = 0 l 1 = hg(x 0, y 0, z 0 ) = 0.1 [( 0) 1 + 0 + 1] = 0 k = h (z 0 + l 1 /) = 0.1 0 = 0 l = hg(x 0 + h/, y 0 + k 1 /, z 0 + l 1 /) = 0.1 [( 0) (1 + 0 0.1 ) + (0 + ) + 1] = 0.1 [ 1 + 0.05 + 1] = 0.005 k 3 = h (z 0 + l /) = 0.1 0.005 = 0.0005 l 3 = hg(x 0 + h/, y 0 + k /, z 0 + l /) = 0.1 [0.005 1 + 0.05 + 1] = 0.0055 k 4 = h (z 0 + l 3 ) = 0.1 0.0055 = 0.00055 l 4 = hg(x 0 + h, y 0 + k 3, z 0 + l 3 ) = 0.1 [{ (0 + 0.0055)} (1 + 0.0005) + (0 + 0.1) + 1] = 0.011075. Therefore, 1 y(0.1) = y 1 = y 0 + 1 6 [k 1 + (k + k 3 ) + k 4 ] = 1 + 1 [0 + (0 + 0.0005) + 0.00055] = 1.000175 6 z(0) = y (0.1) = z(0) + 1 6 [l 1 + (l + l 3 ) + l 4 ] = 0 + 1 [0 + (0.005 + 0.0055) + 0.011075] = 0.005346. 6 The required value of y(0.1) is 1.000175. In addition, y (0.1) is 0.005346.
1.1. Euler s method.............................................................. 1.7 Runge-Kutta-Fehlberg method Many variant of Runge-Kutta methods are available in literature. The Runge-Kutta- Fehlberg method gives better result by solving the differential equation twice using step sizes h and h/. In this method, at each step two different approximations for the solution are calculated. If the two approximations are closed then the solution is obtained. If the approximation does not meet the required accuracy, then the step size may be reduced. In each step the following six values are required. k 1 = hf(x i, y i ) ( k = hf x i + h 4, y i + k ) 1 4 ( k 3 = hf x i + 3h 8, y i + 3 3 k 1 + 9 ) 3 k ( k 4 = hf x i + 1 13 h, y i + 193 197 k 1 700 197 k + 796 ) 197 k 3 ( k 5 = hf x i + h, y i + 439 16 k 1 8k + 3680 513 k 3 845 ) 4104 k 4 ( k 6 = hf x i + h, y i 8 7 k 1 + k 3544 565 k 3 + 1859 4104 k 4 11 ) 40 k 5. (1.33) Then the approximate value of y i+1 using fourth-order Runge-Kutta method is y i+1 = y i + 5 16 k 1 + 1408 565 k 3 + 197 4104 k 4 1 5 k 5. (1.34) In this formula, the value of k is not used. Another value of y is determined by the following fifth order Runge-Kutta method: y i+1 = y i + 16 135 k 1 + 6656 185 k 3 + 8561 56430 k 4 9 50 k 5 + 55 k 6. (1.35) If these two values are closed, i.e. if y i+1 yi+1 is small enough, then the method gives the solution of the given problem, otherwise, the computation is repeated by reducing the step size h. 13