Numerical Methods for Ordinary Differential Equations

CHAPTER 1 Numerical Methods for Ordinary Differential Equations In this chapter we discuss numerical method for ODE. We will discuss the two basic methods, Euler s Method and Runge-Kutta Method. 1. Numerical Algorithm and Programming in Mathcad 1.1. Numerical Algorithm. If you look at dictionary, you will the following definition for algorithm, 1. a set of rules for solving a problem in a finite number of steps; 2. a sequence of steps designed for programming a computer to solve a specific problem. A numerical algorithm is a set of rules for solving a problem in finite number of steps that can be easily implemented in computer using any programming language. The following is an algorithm for compute the root of f(x) = 0, Input f, a, N and tol. Output: the approximate solution to f(x) = 0 with initial guess a or failure message. Step One: Set x = a Step Two: For i=0 to N do Step Three - Four Step Three: Compute x = x f(x) f (x) Step Four: If f(x) tol return x Step Five return failure. In analogy, a numerical algorithm is like a cook recipe that specify the input cooking material, the output the cooking product, and steps of carrying computation cooking steps. In an algorithm, you will see loops (for, while), decision making statements(if, then, else(otherwise)) and return statements. for loop: Typically used when specific number of steps need to be carried out. You can break a for loop with return or break statement. 1

2 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS while loop: Typically used when unknown number of steps need to be carried out. You can break a while loop with a break statement or a return statement. if else(otherwise): Used when condition(s) must meet before carrying out some steps. The else(otherwise) provide alternative steps if any. 1.2. Programming in Mathcad. To translate a numerical algorithm into a program in Mathcad, you need to define a function name with function arguments and function body. The function name can be any sequence of alphabetic letter begin with an English letter, such any myfunction, Myfunction1 etc. The function arguments specify the input to the function, typically the items specified in the input statement of an algorithm and is an comma(,) separated list enclose by () after function name, such as myfunction(f, a, N, tol), myone(a,b). Notice the function argument is optional, you can define a function with out argument as myfunction(). The function body implements the step section of an algorithm and the output statement. In Mathcad the function body is on the right side of := with vertical bars as grouping symbol. The entire function body is considered as a group, inside, it might has many subgroups. For the algorithm of finding f(x) = 0, there is an subgroup that contains step three and four, as shown in the following screen shot, the screen shot also shows how to get the programming toolbar from the math toolbar. Figure 1. mysolver and Programming toolbar

1. NUMERICAL ALGORITHM AND PROGRAMMING IN Mathcad 3 To enter code in Mathcad, (1) First enter the function name with arguments list and assignment operator :=, with sequence of typing, mysolver(f,a,n,tol): (2) Click the Add Line button or press ] to get a vertical bar and two (you should add more by press ] several times). (3) enter x and hold [Shift] type ] to get the local assignment or click on the programming toolbar, at the type a. Notice x a reads as assign a to x. (4) Click for on the programming menu or [Shift][Ctrl][ ] to get for in after for enter i, in after, enter 1;N, and in under for, press left bracket ] to get a vertical line and several. (5) To get if operator, you need either click on the toolbar or press [Shift][Ctrl][ ] ]. you will get if enter condition in after if and other statements in before if. (6) To get return operator, you need either click on the toolbar or press [Shift][Ctrl][ ], you get return in type any information you want to return. After defining a function, we call it with concrete input data, as shown in the screen shot, that call the mysolver function to find zero for both f(x) = x 2 1, with initial guess x = 2 and g(t) = t 3 3t 2 2t, with initial guess t = 2. Notice there is two ways to call a function, one way is to define all arguments before calling the function, another is to define some arguments and call the function with concrete values for the undefined arguments. As in all programming language, Mathcad allows you to call one function from within another. For example, we could create a function called myderivative which will compute derivative of a given function f(x) at a given number x and step size h > 0 using the forward difference formula: f (s) f(x+h) f(x). The following screen shot shows h the call of myderivative inside mysolver, Figure 2. Call another function

4 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS The algorithm for this function is very simple (here, we break down the computation in three steps), Input f, x, and h. Output: the approximation to f (x). Step One: Set w = f(x + h) Step Two: Set d = f(x) Step Three: return w d h 2. Euler s Method 2.1. Euler s Method. Euler s method is the simplest method in find approximate solutions to first order equations. From the forward difference formula f f(x + h) f(x) (x), h we have (1) f(x + h) f(x) + f (x)h Now if x (t) = f(t, x) is a first order differential equations, apply (1), we have x(t + h) x(t) + f(t, x(t))h. Suppose we want to find approximate solution over interval [a, b] with initial value x(a) = x 0, we divided the interval into n subintervals each with length h = b a, the ends of the subintervals is t n 0 = a, t 1 = a + h, t 2 = a + 2h,, t n = a + nh = b. Start with x 0 we can compute x 1 = x 0 + f(t 0, x 0 )h x 2 = x 1 + f(t 1, x 1 )h.x n = x n 1 + f(t n 1, x n 1 )h Example 2.1. Find approximate to x(1) if x (t) = t 2 e x sin(t), x(0) = 1 with h = 0.25. Solution Here the interval is [0, 1], so a = 0, b = 1. Since h = b a = 0.25 we have n = 4 and we need to compute x n 1, x 2, x 3, x 4 starting with x 0 = x(0) = 1. x 1 = x 0 + f(t 0, x 0 )h = 1 + f(0, 1) 0.25 = 0.15853 x 2 = x 1 + f(t 1, x 1 )h = 0.15853 + f(0.25, 0.15853) 0.25 = 0.01833 x 3 = x 2 + f(t 2, x 2 )h = 0.01833 + f(0.5, 0.01833) 0.25 = 0.23811 x 4 = x 3 + f(t 3, x 3 )h = 0.23811 + f(0.75, 0.15853) 0.25 = 0.30128 So the approximation to x(1) is x(1) x 4 = 0.30128. Also x(0.25) x 1 = 0.15853, x(0.5) x 2 = 0.01833, x(0.75) x 3 = 0.23811.

2. EULER S METHOD 5 2.2. Mathcad implementation of Euler s Method. First, we have the following simple algorithm for the Euler s method, Input f, a, b, x0 n. Output: the approximate solution to x = f(t, x) with initial guess x0 over interval [a, b]. Step One: Initialization Set h = b a n Set x 0 = x0 Set t 0 = a Step Two: For i=1 to n do Step Three Step Three: Set x i = x i 1 f(t i 1, x i 1 ) h Set t i = t i 1 + h Step Four return x. Notice, algorithm return an array of values, the ith element of the return array is an approximations of x(t) at t = a + ih. The following screen shot shows Mathcad code for implementing the algorithm. Notice, we also create a function tmesh(a, b, N) to compute the ending of subinterval for graphing purpose. Figure 3. Mathcad code for Euler s Method Notice the line to line corresponding between the Mathcad and the algorithm. Since Mathcad programming language is a scripting language, the translation between algorithm and code is straight forward, and you don t need to worry about the variable type, io, etc. Also, without explicit return statement, the result of last is, by default, will be returned.

6 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS The next screen shot shows a call to the myeuler and tmesh for the equation x = t 2 x + t 2 sin(t 3 ). It also shows the graph of approximate solution comparing with the exact solution x(t) = 3 10 cos(t3 ) 1 10 sin(t3 ) + 3 e 1 10 t3 10 Figure 4. Mathcad Euler s Approximation to x = t 2 x + t 2 sin(t 3 ) 2.3. Error analysis of Euler s Method. There are two source of error in every numerical method used to approximate a solution of x (t) = f(t, x), Local Truncation Error The roundoff error. The local truncation error is due to the method and roundoff error is due to computer that is used. For Euler s method, we use x i = x i 1 + f(t i 1, x i 1 ) h to approximate the value of x(t i ), the difference, assuming no rounding is introduced, is x i x(t i ), and is called the local truncation error. The following theorem shows how the local truncation error is depending on f(t, x) and h, Theorem 2.1. Suppose f(t, x), f(t,x) t on [a, b], and h = b 1 n, and f(t,x) are continuous x is the step size. Furthermore let x(t) be the solution of initial value problem x = f(t, x), x(a) = x 0 and x i = x i 1 + f(t i 1, x i 1 ) h be the approximate of x(t i ), and let e i = x(t i ) x i be

the local truncation error, then 3. RUNGE-KUTTA METHOD 7 e i+1 (1 + hk) e i + 1 2 h2 M, i = 1,...n., where f(t,x) K, and x (t) M. x Theorem 2.2 indicates that, without roundoff error, the smaller h gives better approximate solution. However, due the roundoff error introduced in each step of computation, (such as compute can only give approximate value for 2) we can t choose h arbitrarily small. The following result gives the optimal step size h, Theorem 2.2. With the same assumption of Theorem 2.2,and suppose at each step the roundoff error is bounded by ɛ then the optimal value for the step length is 2ɛ h opt = M 3. Runge-Kutta Method 3.1. Runge-Kutta Method. There are several Runge-Kutta methods, but the fourth order Runge-Kutta method is most popular. Suppose x (t) = f(t, x), x(a) = x 0. and h = b a is the step length, the n fourth order Runge-Kutta method is given below, k 1 = hf(t i, x i ) k 2 = hf(t i + 1 2 h, x i + 1 2 k 1) k 3 = hf(t i + 1 2 h, x i + 1 2 k 2) k 4 = hf(t i + h, x i + k 3 ) x i+1 = x i + 1 6 (k 1 + 2k 2 + 2k 3 + k 4 ) In general Runge-Kutta method gives more accurate result than Euler s method at the same step length. However, Runge-Kutta method is more expensive to implement, that is, at each step, Runge-Kutta method requires more computation than Euler s method( four function evaluations compare one in Euler s method). Example 3.1. Example 3.2. Find approximate to x(1) if x (t) = t 2 e x sin(t), x(0) = 1 with h = 0.25. Solution Here the interval is [0, 1], so a = 0, b = 1. Since h = b a = 0.25 we have n = 4 and we need to compute x n 1, x 2, x 3, x 4 starting with x 0 = x(0) = 1, t 0 = 0

8 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS x 1 : k 1 = hf(t 0, x 0 ) = 0.25 f(0, 1) = 0 k 2 = hf(t 0 + 1h, x 2 0 + 1k 2 1) = f(0.125, 1) = 0.373276 k 3 = hf(t 0 + 1h, x 2 0 + 1k 2 2) = 0.25f(0.125, 1 + 0.5( 0.373276)) = 0.373276 k 4 = hf(t 0 + h, x 0 + k 3 ) = 0.25f(0.25, 1 0.373276) = 0.309854 x 1 = x 0 + 1(k 6 1 + 2k 2 + 2k 3 + k 4 ) = 1 + 1 (0 + 2 ( 0.373276) + 2 ( 0.373276) 0.309854) 6 = 0.923911 t 1 = t 0 + h = 0 + 0.25 = 0.25 x 2 : k 1 = hf(t 1, x 1 ) = 0.25 f(.25, 0.923911) = 0.560741 k 2 = hf(t 1 + 1h, x 2 1 + 1k 2 1) = f(0.125, 0.923911 + 0.5( 0.560741)) = 0.719601 k 3 = hf(t 1 + 1h, x 2 1 + 1k 2 2) = 0.25f(0.125, 0.923911 + 0.5( 0.719601)) = 0.702688 k 4 = hf(t 1 + h, x 1 + k 3 ) = 0.25f(0.5, 0.923911 0.702688) = 0.763158 x 2 = x 1 + 1(k 6 1 + 2k 2 + 2k 3 + k 4 ) = 0.923911 + 1 ( 0.560741 + 2 ( 0.719601) + 2 ( 0.702688) 0.763158) 6 = 0.750224 t 2 = t 1 + h = 0.25 + 0.25 = 0.5 x 3 : k 1 = hf(t 2, x 2 ) = 0.25 f(.5, 0.750224) = 0.765171 k 2 = hf(t 2 + 1h, x 2 2 + 1k 2 1) = f(0.625, 0.750224 + 0.5( 0.765171)) = 0.735295 k 3 = hf(t 2 + 1h, x 2 2 + 1k 2 2) = 0.25f(0.625, 0.750224 + 0.5( 0.735295)) = 0.739508 k 4 = hf(t 2 + h, x 2 + k 3 ) = 0.25f(0.75, 0.750224 0.739508) = 0.637224 x 3 = x 2 + 1(k 6 1 + 2k 2 + 2k 3 + k 4 ) = 0.750224 + 1 ( 0.765171 + 2 ( 0.735295) + 2 ( 0.739508) 0.637224) 6 = 0.568891 t 3 = t 2 + h = 0.5 + 0.25 = 0.75

x 4 : 3. RUNGE-KUTTA METHOD 9 k 1 = hf(t 3, x 3 ) = 0.25 f(0.75, 0.568891) = 0.641483 k 2 = hf(t 3 + 1h, x 2 3 + 1k 2 1) = f(0.875, 0.568891 + 0.5( 0.641483)) = 0.485628 k 3 = hf(t 3 + 1h, x 2 3 + 1k 2 2) = 0.25f(0.875, 0.568891 + 0.5( 0.485628)) = 0.510243 k 4 = hf(t 3 + h, x 3 + k 3 ) = 0.25f(1, 0.568891 0.510243) = 0.308297 x 4 = x 3 + 1(k 6 1 + 2k 2 + 2k 3 + k 4 ) = 0.568891 + 1 ( 0.641483 + 2 ( 0.485628) + 2 ( 0.510243) 0.308297) 6 = 0.446327 t 4 = t 3 + h = 0.75 + 0.25 = 1.0 So the approximation to x(1) is x(1) x 4 = 0.446327. 3.2. Mathcad implementation. First, we have the following simple algorithm for the Euler s method, Input f, a, b, x0 n. Output: the approximate solution to x = f(t, x) with initial guess x0 over interval [a, b]. Step One: Initialization Set h = b a n Set x 0 = x0 Set t 0 = a Step Two: For i=1 to n do Step Three Step Three: Set k1 = hf(t i 1, x i 1 ) Set k2 = hf(t i 1 + 1h, x 2 i 1 + 1k 2 1) Set k3 = hf(t i 1 + 1h, x 2 i 1 + 1k 2 2) Set k4 = hf(t i 1 + h, x i 1 + k 3 ) Set x i = x i + 1(k 6 1 + 2k 2 + 2k 3 + k 4 ) Set t i = t i 1 + h Step Four return x. This algorithm returns an array of values, the ith element of the return array is an approximations of x(t) at t = a + ih. The following screen shot shows Mathcad code for implementing the algorithm. Again notice the line to line corresponding between the Mathcad and the algorithm.

10 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS Figure 5. Mathcad code for Euler s Method The next screen shot shows a call to the myrungekutta and tmesh for the equation x = t 2 x+t 2 sin(t 3 ). It also shows the graph of approximate solution comparing with the exact solution x(t) = 3 10 cos(t3 ) 1 10 sin(t3 ) + 3 e 1 10 10 t3, and with the approximate solution using Euler s method. In this case Runge-Kutta provides much superior result, an almost exact match! Figure 6. Mathcad Runge-Kutta s Approximation to x = t 2 x + t 2 sin(t 3 )

4. NUMERICAL METHOD FOR SYSTEM OF EQUATIONS 11 4. Numerical Method for System of Equations Both Euler s method and Runge-Kutta method can be used to find the approximate solution to the system of first order differential equations. In fact, the Mathcad codes for system of first differential equations are exactly the same as Mathcad does not differentiate scalar and vectors when performs most computations. The only thing needs to be taken care is that at each step the result is vector instead of a scalar. Suppose x(t) is a vector-valued function that satisfies x (t) = F (t, x(t)), where F (t, x) is also vector-valued function, t i = a+ih then the Euler s method compute the ith approximate, (2) x i = x i 1 + F (t i 1, x i 1 ). Example 4.1. Let x (t) = t 2 sin(x(t)) + e t cos(y(t)) y (t) = 2tx(t) + e y(t) x(0) = 1, y(0) = 1 Find approximate solution for x(t), y(t) over interval [0, 1] with h = 0.1 [ ] [ ] x(t) t Solution Set x(t) = and F (t, x) = 2 sin(x) + e t cos(y) y(t) 2tx + e y The equation can then be written, in vector form, x(0) = [ 1 1 x (t) = F (t, x(t)), ] From (2) we can find, with t 0 = 0, x 1 = [ x 0 + h] F [(t 0, x 0 ) ] 1 t 2 = + 0 sin(x 0 ) + e t 0 cos(y 0 ) 1 2t 0 x 0 + e y 0 [ ] cos( 1) = e 1 The following screen shot shows the results obtain by calling our Mathcad implementation of Euler s method. Notice that in defining the vector-valued function F (t, X), for Mathcad to know that X is an vector, in the definition, we use subscript to access the component of X, as shown in the screen shot.

12 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS Figure 7. Euler s method for system of equations

4. NUMERICAL METHOD FOR SYSTEM OF EQUATIONS 13 Similarly, we can use the same Mathcad code of Runge-Kutta method to find approximate solution as shown in the following screen shot for the above system of equations, Figure 8. Runge-Kutta s method for system of equations 4.1. Numerical Method for Higher Order Equations. To find numerical approximate for solutions of higher order differential equations using either Euler s method or Runge-Kutta method, we first transform an higher order differential equations into a system of first order differential equations and then apply the methods. Example 4.2. Rayleigh Equation In modeling the oscillations of a clarinet reed. Lord Rayleigh introduced an equation of the form mx + kx = ax b(x ) 3 Find approximate solution for m = 1, k = 1, a = 2, b = 3. and initial conditions x(0) = 1, x (0) = 2

14 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS Solution Suppose y(t) = x (t) is the velocity, then Rayleigh equation becomes x = y y = 2y 3y 3 2x [ ] y So we apply the numerical method for F = 2y 3y 3 and 2x [ ] [ ] x(t) = x(t) 1 x (t) = F (t, x) with, x(0) =. The following y(t) 2 screen shot gives approximate solution using Runge-Kutta method, Figure 9. Runge-Kutta s method for Rayleigh s equation we also plot the solution in xy-plane, which is called the velocityposition phase plane, with y-axis represent the velocity and x-axis is the

4. NUMERICAL METHOD FOR SYSTEM OF EQUATIONS 15 position. The velocity-position plane shows that all velocity-position solution pairs are eventually being attracted to an closed orbit, the closed orbit is called the global attractor.