Applied Math for Engineers Ming Zhong Lecture 15 March 28, 2018 Ming Zhong (JHU) AMS Spring 2018 1 / 28
Recap Table of Contents 1 Recap 2 Numerical ODEs: Single Step Methods 3 Multistep Methods 4 Method for Systems and Higher Order ODEs 5 Homework Ming Zhong (JHU) AMS Spring 2018 2 / 28
Recap Summary We have discussed: How to solve non-homogeneous linear First Order ODE system. Method of Undetermined Coefficients. Method of Variation of Parameters, u = Y 1 g. How to solve linear ODEs with power series. y + p(x)y + q(x)y = 0, p, q analytic, then y = m=0 a mx m. y + b(x) x y + c(x) x y = 0, b, c analytic at x = 0, then 2 y = x r m=0 a mx m. Ming Zhong (JHU) AMS Spring 2018 3 / 28
Numerical ODEs: Single Step Methods Table of Contents 1 Recap 2 Numerical ODEs: Single Step Methods 3 Multistep Methods 4 Method for Systems and Higher Order ODEs 5 Homework Ming Zhong (JHU) AMS Spring 2018 4 / 28
Numerical ODEs: Single Step Methods Why Numerical Methods? Not all ODEs can be solved analytically. Consider, x i = N φ( xj x )(xj i x i ), for 1 i N. j=1 Here, φ(r) = 1 if 0 r < 1 2 and φ(r) = 0.1 if 1 2 r < 1 and φ(r) = 0 otherwise. Ming Zhong (JHU) AMS Spring 2018 5 / 28
Numerical ODEs: Single Step Methods Notations Consider the Initial Value Problem (IVP) for (f is nice for the IVP to have unique solution), y = f (x, y), y(x 0 ) = y 0. We will try to obtain a numerical solution for x (a, b) containing x 0 : We consider some equidistant mesh points: x 1 = x 0 + h, x 2 = x 0 + 2h,, x n = x 0 + nh. h is called the step size, it is fixed. It is derived from Taylor Series expansion (step by step): y(x + h) = y(x) + hy (x) + h2 2 y + The Euler method (or Euler-Cauchy method) uses only the first derivative: y(x + h) y(x) + hy (x), y n+1 = y n + hf (x n, y n ). Ming Zhong (JHU) AMS Spring 2018 6 / 28
Numerical ODEs: Single Step Methods Error of the Euler Method Local Truncation Error By Taylor series expansion, y(x n ) y n O(h 2 ), assuming y n 1 = y(x n 1 ), called the Local Truncation Error. The number of steps is proportional to 1 h. The total error (global error) is proportional to h 2 1 h = h. The Euler method is a first-order method. There are roundoff errors in computer arithmetics, might accumulate in y 1, y 2,. Ming Zhong (JHU) AMS Spring 2018 7 / 28
Numerical ODEs: Single Step Methods Varaible Step Size Selection Since y = f, y = f = f x + f y y = f x + f y f. We can use it to esimate y, given a certain Tolerance τ, we want 1 y (ɛ n ) 2τ τ, thus h n y (ɛ n ). h 2 n y must not be zero on the interval. We can take the maximum of y over the whole solution interval. Ming Zhong (JHU) AMS Spring 2018 8 / 28
Numerical ODEs: Single Step Methods Improved Euler Method (Heun s Method) Predictor and Corrector Consider the Taylor series expansion of y, y(x + h) = y(x) + hy + h2 2 y + h3 3! y +. Repalcing y with f and y(x + h) = y(x) + hf + h2 2 f + h3 3! f +. However, f requires partial derivatives. The general strategy is to avoid computing f s directly, which brings us the improved Euler method. First, we preditc yn+1 = y n + hf (x n, y n ); use yn+1 to correct, y n+1 = y n + h 2 (f (x n, y n ) + f (x n+1, y n+1)). Ming Zhong (JHU) AMS Spring 2018 9 / 28
Numerical ODEs: Single Step Methods Local Truncation Error of Heun s Method Theorem The local truncation error of the Heun s method is of order h 3. Can we go beyound second order? The Runge-Kutta Methods (RK4 methods) k 1 = hf (x n, y n ). k 2 = f (x n + h 2, y n + k 1 2 ). k 3 = f (x n + h 2, y n + k 2 2 ). k 4 = f (x n + h, y n + k 3 ). y n+1 = y n + 1 6 (k 1 + 2k 2 + 2k 3 + k 4 ). It is a 4 th order method. Ming Zhong (JHU) AMS Spring 2018 10 / 28
Numerical ODEs: Single Step Methods General Explicit RK Methods Consider a s stage RK method of the following, y n+1 = y n + h s i=1 b ik i, where k 1 = f (x n, y n ) k 2 = f (x n + c 2 h, y n + h(a 21 k 1 )) k 3 = f (x n + c 3 h, y n + h(a 31 k 1 + a 32 k 2 )). k s = f (x n + c s h, y n + h(a s1 k 1 + a s2 k 2 + a s,s 1 k s 1 )) To specify a particular method, one needs to provide the integer s (the number of stages), the coefficients a ij (for 1 j < i s), b i (for i = 1,, s), and c i (for i = 2, s). The matrix A = [a ij ] is called the Runge-Kutta matrix, b i s are known as the weights and the nodes. Ming Zhong (JHU) AMS Spring 2018 11 / 28
Numerical ODEs: Single Step Methods Butcher Tableau 0 c 2 a 21 c 3 a 31 a 32..... c s a s1 a s2 a s,s 1 b 1 b 2 b s 1 b s The Runge-Kutta method is consistent if i 1 j=1 a ij = c i for i = 2,, s. order 1 2 3 4 5 6 7 8 min s 1 2 3 4 6 7 9 11 Ming Zhong (JHU) AMS Spring 2018 12 / 28
Numerical ODEs: Single Step Methods Step Size Control Runge-Kutta-Fehlberg Consider any first order method, and two approximations, y h and y 2h, at the same x n obtained from two different step sizes, h and 2h. Then, y yh e h and y y2h e 2h. Moreover, e 2h 2 1 e h, then, y h y 2h e 2h e h e h. So we can estimate e h by taking the difference of y h and y 2h. And, If y h y 2h τ (a given tolerance), we keep y 2h and move on. Otherwise, we calculate y h 2 and compare. Ming Zhong (JHU) AMS Spring 2018 13 / 28
Numerical ODEs: Single Step Methods The RKF Method However the RKF method can do it better. RKF is an embedded RK method. The Fehlberg s fifth-order method, y n+1 = y n + γ 1 k 1 + + γ 5 k 5 + γ 6 k 6. [γ 1,, γ 6 ] = [ 16 134 The Fehlberg s fourth-order method,, 0, 6656 12825, 28561 56430, 9 50, 2 55 ] yn+1 = y n + γ1 k 1 + γ5 k 5. [γ1,, γ 5 ] = [ 25 1408 216, 0, 2565, 2197 4104, 1 5 ] They use the same k i s. Ming Zhong (JHU) AMS Spring 2018 14 / 28
Numerical ODEs: Single Step Methods RKF Method, Cont. It also comes with an error estimator, These are explicit methods, where the evaluation of f s depend on previous information, (x n, y n ). Some problems are very sensitive to step size (explicitly). We need to consider implicit method for stiff problems, the implicit Euler method (Backward Euler), y n+1 = y n + hf (x n+1, y n+1 ). Each step, we need to solve an equation, y n+1 hf (x n+1, y n+1 ) = y n. Use Newton s method (if it is non-linear). It gives only first order accuracy however allows for bigger step size. Ming Zhong (JHU) AMS Spring 2018 15 / 28
Numerical ODEs: Single Step Methods Example Example Numerically solve the IVP, y = 20y + 20x 2 + 2x, y(0) = 1. The true solution is: y = e 20x + x 2, e 20x decays too fast. Using Backward Euler: y n+1 = y n + hf (x n+1, y n1 ). Solve for y n+1, we obtain y n+1 = yn+h(20(xn+h)2 +2(x n+h)) 1+20h. Solution is stable for Backward Euler with h = 0.2. Solution is stable for Euler with h = 0.05 but unstable for h = 0.1. Solution is stable for RK4 with h = 0.1 but unstable for h = 0.2. Ming Zhong (JHU) AMS Spring 2018 16 / 28
Multistep Methods Table of Contents 1 Recap 2 Numerical ODEs: Single Step Methods 3 Multistep Methods 4 Method for Systems and Higher Order ODEs 5 Homework Ming Zhong (JHU) AMS Spring 2018 17 / 28
Multistep Methods Multistep Methods We have discussed, A family of Single Step Methods: they are all part of the Runge-Kutta family (of different orders). They are self-starting, they only need y n 1 to get to y n. The Adams-Bashforth Methods, Consider the IVP, y = f (x, y) with y(x 0 ) = y 0. By integration, y(x n+1 ) y(x n ) = x n+1 x=x n y (x)(= f (x, y)) dx. We only know f at (x 0, y 0 ), (we do have expression for f, but y is unknown). We approximate f using polynomial at unknown (x, y). So y(x n+1 ) y(x n ) x n+1 x=x n p(x) dx. Ming Zhong (JHU) AMS Spring 2018 18 / 28
Multistep Methods Adams-Bashforth Methods, Cont. Continue on, Suppose we are given (x n 3, y n 3 ), (x n 2, y n 2 ), (x n 1, y n 1 ), and (x n, y n ). Let f n = f (x n, y n ) and f n s = f (x n s, y n s ), for s = 1, 2, 3. Suppose also x n 3, x n 2, x n 1, x n, are equidistant points. We will approximate f at the 4 pairs of points using cubic polynomial, p 3 (x), defined as p 3 (x) = f n + r f n + r(r + 1) 2 f n + 2 r(r + 1)(r + 2) 3 f n. 6 Here, r = x xn h, and is the Newton s backward difference formula, f n = f n f n 1, and k f n = k 1 f n k 1 f n 1. y n+1 = y n + h 24 (55f n 59f n 1 + 37f n 2 9f n 3 ), Adams-Bahsforth method of fourth order. Ming Zhong (JHU) AMS Spring 2018 19 / 28
Multistep Methods Adams-Moulton Formula The Adams-Bashforth method uses all past information, we can also include the information at (x n+1, y n+1 ). p 3 (x) = f n+1 + r f n+1 + r(r+1) 2 2 f n+1 + r(r+1)(r+2) 6 3 f n+1. y n+1 = y n + h 24 (9f n+1 + 19f n 5f n 1 + f n 2 ). It is an implicit method. In order to avoid solving an equation, we can use the predictor-corrector of AB4. Predict: y n+1 = y n + h 24 (55f n 59f n 1 + 37f n 2 9f n 3 ). Correct: y n+1 = y n + h 24 (9f (x n+1, y n+1 ) + 19f n 5f n 1 + f n 2 ). This is called the Adams-Moulton s method of fourth order. How do we get started? We are only given (x 0, y 0 ), but we will need (x 1, y 1 ), (x 2, y 2 ), and (x 3, y 2 ) to start the method. Just use any method to come up with those points. Ming Zhong (JHU) AMS Spring 2018 20 / 28
Method for Systems and Higher Order ODEs Table of Contents 1 Recap 2 Numerical ODEs: Single Step Methods 3 Multistep Methods 4 Method for Systems and Higher Order ODEs 5 Homework Ming Zhong (JHU) AMS Spring 2018 21 / 28
Method for Systems and Higher Order ODEs First Order System of ODEs Let us consider the IVP: y = f ( x, y) with y( x 0 ) = y 0. x is of size n 1 and y is of size m 1. f and y have to have the same size, but not necessarily x and y. f is nice so that the IVP has a unique solution on some open interval containing x 0. We only focus on first order ODEs or system of ODEs. For any higher order, y (m) = f (x, y, y, y,, y (m 1) ). Conversion: y 1 = y, y 2 = y y m = y (m 1). Then, y 1 = y 2,, y m 1 = y m, and y m = f (x, y 1, y 2,, y m 1 ). Ming Zhong (JHU) AMS Spring 2018 22 / 28
Method for Systems and Higher Order ODEs Single Step Methods The Euler method, The RK4 method, y n+1 = y n + h f ( x n, y n ). k1 = h f ( x n, y n ) k2 = h f ( x n + h 2, y n + k 1 2 ) k3 = h f ( x n + h 2, y n + k 2 2 ) k4 = h f ( x n + h, y n + k 3 ) y n+1 = y n + 1 6 ( k 1 + 2 k 2 + 2 k 3 + k 4 ) Ming Zhong (JHU) AMS Spring 2018 23 / 28
Method for Systems and Higher Order ODEs Runge-Kutta-Nystrom Methods The RKN methods, RKN methods are direct extensions of RK methods for second order ODEs: y = f (x, y, y ). k 1 = h 2 f (x n, y n, y n) k 2 = h 2 f (x n + h 2, y n + K, y n + k 1 ), K = h 2 (y n + 1 2 k 1) k 3 = h 2 f (x n + h 2, y n + K, y n + k 2 ) k 4 = h 2 f (x n + h, y n + K, y n + L), L = h(y n + k 3 ) y n+1 = y n + h(y n + 1 3 (k 1 + k 2 + k 3 )) y n+1 = y n + 1 3 (k 1 + 2k 2 + 2k 3 + k 4 ) Ming Zhong (JHU) AMS Spring 2018 24 / 28
Method for Systems and Higher Order ODEs RKN without y If y = f (x, y) not containing y, we have a simplify RNK formulas k 1 = h 2 f (x n, y n ) k 2 = h 2 f (x n + h 2, y n + h 2 (y n + k 1 2 ) = k 3 k 4 = h 2 f (x n + h, y n + h(y n + k 2 ) y n+1 = y n + h(y n + 1 3 (k 1 + 2k 2 )) y n+1 = y n + 1 3 (k 1 + 4k 2 + k 4 ) Backward Euler method for stiff systems, y n+1 = y n + h f ( x n+1, y n+1 ). At each step, we will have to solve a (possible) non-linear system for y n+1. Ming Zhong (JHU) AMS Spring 2018 25 / 28
Homework Table of Contents 1 Recap 2 Numerical ODEs: Single Step Methods 3 Multistep Methods 4 Method for Systems and Higher Order ODEs 5 Homework Ming Zhong (JHU) AMS Spring 2018 26 / 28
Homework Homework 7 Please do the following: Chapter 4 Review Questions: 20, 21, 22, 23. Chapter 5 Review Questions: 12, 14, 16, 18. Ming Zhong (JHU) AMS Spring 2018 27 / 28