What we ll do: Lecture Ordinary Differential Equations J. Chaudhry Department of Mathematics and Statistics University of New Mexico Review ODEs Single Step Methods Euler s method (st order accurate) Runge-Kutta methods (higher order accuracy) Accuracy and stability of the methods J. Chaudhry (UNM) CS 357 / 34 Differential Equations J. Chaudhry (UNM) CS 357 / 34 Ordinary Differential Equations (ODEs) Its hard to model the state of a system Temperature in the room Pressure on the wing of an airplane Concentration of ions in a solvent and a million more However, its much simpler to model the rate of change of the state: differential equations Could involve change w.r.t to one variable (e.g. time): Ordinary Differential Equations Could involve change w.r.t multiple variables (e.g. x and y axis): Partial Differential Equations An ODE involves One function u(t) Its derivatives u (t), u (t), u (t), etc Order of ODE is the higest order derivative Examples: Generic First-order ODE u = f (t, u(t)) Harmonic Oscillator (Second-order ODE) mu (t) + cu (t) + ku(t) = f (t) m: mass, c damping coeff., k: elastic stiffness coeff, f (t): external force J. Chaudhry (UNM) CS 357 3 / 34 J. Chaudhry (UNM) CS 357 4 / 34
Initial Value Problems (IVPs) Numerical Solution of IVPs ODEs in previous examples have infinitely many solutions Get unique solution by specifying initial conditions (ICs) Examples: IVP = ODE + IC u = f (t, u(t)), t (a, b] u(a) = u a mu (t) + cu (t) + ku(t) = f (t), t (0, T] u(0) = A, u (0) = B Many important ODEs have no closed form solution Analytical methods of limited use in such scenarios Approximate the solution using numerics Consider first order ODE u = f (t, u(t)), t (a, b] u(a) = u a Discretize [a, b] into equi-spaced points: with h = t i t i = (b a)/n a = t 0 < t < t < < t n < t n = b J. Chaudhry (UNM) CS 357 5 / 34 Euler s method J. Chaudhry (UNM) CS 357 6 / 34 Euler Example Simplest numerical methods to solve IVPs Taylor s thm u(t + h) = u(t) + hu (t) + (h /)u (c), Let u i be approximation to exact solution u(t i ) Drop O(h ) terms and utilize u (t) = f (t, u) t < c < t + h Apply Euler s method with h = 0. to the IVP u = tu + t 3, t [0, ] u(0) = h = (b a)/n = n = 5. Step i t i u i f (t i, u i ) u i+ = u i + hf (t i, u i ) u 0 = u a for i = 0 to n 3 u i+ = u i + h f (t i, u i ) Listing : Euler Algorithm 0 0 0 0. 0.080.046 0.4.046 0.4806.377 3 0.6.377 0.8986.375 4 0.8.375.5660.6306 5.6306 J. Chaudhry (UNM) CS 357 7 / 34 J. Chaudhry (UNM) CS 357 8 / 34
Euler Example Accuracy of Euler s method Exact solution is u(t) = 3e t / t Euler method with h = 0., 0., 0.05 As h decreases, approximation converges to exact solution Compute the error at final time, t = Error, e = O(h p ) loglog plot 0 0.9.8.7 Exact Soln Euler (h=/5) Euler (h=/0) Euler (h=/0) 0 -.6 Error u(t).5.4 0 -.3.. 0 0. 0. 0.3 0.4 0.5 0.6 0.7 0.8 0.9 t p appears to be 0-3 0-4 0-3 0-0 - h J. Chaudhry (UNM) CS 357 9 / 34 Accuracy of Euler s method J. Chaudhry (UNM) CS 357 0 / 34 Accuracy of Euler s method: Some Theory Compute the error at final time, t = Error, e = O(h p ) p p log(e k/e k ) log(h k /h k ) Experiment k e k e k h k h k p 0 0.78 - /0 - - 0.0899 0.78 /0 /0 0.93 0.04603 0.0899 /40 /0 0.96 3 0.039 0.04603 /80 /40 0.98 4.07 0.039 /60 /80 0.99 Euler s method is first-order accurate method Can theoretically show this too Recall the derivation using Taylor s thm Exact solution Numerical soln u(t + h) = u(t) + hf (t, u) + O(h ) u i+ = u i + hf (t i, u i ) Assume u(t) = u i. Then the local error, u(t + h) u i+ is u(t + h) u i+ = O(h ) Global error is the error at final time: u(t n ) u n Local errors accumulates to give O(nh ) = O(h) accuracy since nh = (b a). The analysis for glocal error is complicated and we omit it. J. Chaudhry (UNM) CS 357 / 34 J. Chaudhry (UNM) CS 357 / 34
Systems of ODEs Euler method for first-order system of ODEs A first-order system of m ODEs has the form u = f (t, u, u,..., u m ) u = f (t, u, u,..., u m ) t (a, b]. u m = f m (t, u, u,..., u m ) In IVPs, each variable has its own initial condition: u (a) = u a u (a) = u a. u m (a) = u ma Apply scalar Euler s method to each component Example: apply Euler s method to u = u + u = f (t, u, u ), u (0) = 0 u = tu = f (t, u, u ), u (0) = Euler s method: u,i+ = u,i + h( u,i + u,i ) u,i+ = u,i + h( t i u,i ) J. Chaudhry (UNM) CS 357 3 / 34 Higher order equations J. Chaudhry (UNM) CS 357 4 / 34 Euler s method for Higher order equations Convert higher order ODE to system to first order ODEs Consider u + au + btu = c Introducte y = u y = u This leads to the following first-order ODE system y = y y = ay bty + c Now apply Euler s method! We get approximation to y = u. Solve the following IVP using Euler s method: u = (u ) uu + sin t, u(0) =, u (0) = 0, u (0) = Convert to first-order system: y = u, y = u, y 3 = u Euler s method: y = y, y (0) = y = y 3, y (0) = 0 y 3 = y 3 y y + sin t, y 3 (0) = y,i+ = y,i + hy,i y,i+ = y,i + hy 3,i y 3,i+ = y 3,i + h(y 3,i y,iy,i + sin t i ) J. Chaudhry (UNM) CS 357 5 / 34 J. Chaudhry (UNM) CS 357 6 / 34
Runge-Kutta Methods (RK) Two-stage RK methods Integration both sides of u (t) = f (t, u(t)) from t i to t i+ u(t i+ ) u(t i ) = ti+ t i f (t, u(t)) dt Runge-Kutta methods are popular methods to solve IVPs Euler method is a single-stage RK method (RK) Higher order methods derived using higher order Taylor series, higher order integration techniques etc. Higher order methods have more stages Now apply (simple) trapezoidal rule on the RHS Let u i u(t i ) u(t i+ ) u(t i ) = h [f (t i, u(t i )) + f (t i+, u(t i+ ))] + O(h 3 ) u i+ = u i + h [f (t i, u i ) + f (t i + h, u i+ )] + O(h 3 ) Approx u i+ on RHS by Euler s method u i+ = u i + h [f (t i, u i ) + f (t i + h, u i + f (t i, u i ) + O(h )] + O(h 3 ) J. Chaudhry (UNM) CS 357 7 / 34 RKa method J. Chaudhry (UNM) CS 357 8 / 34 RKb method RKa Drop O(h 3 ) terms. Get method called RKa u i+ = u i + h [f (t i, u i ) + f (t i + h, u i + hf (t i, u i )] Written as a two-stage method K = f (t i, u i ) K = f (t i + h, u i + hk ) u i+ = u i + h( K + K ) K and K are called stages. RKa has has local error of O(h 3 ) for a global error of O(h ). Integration both sides of u (t) = f (t, u(t)) from t i to t i+ u(t i+ ) u(t i ) = Now apply midpoint rule on the RHS ti+ t i f (t, u(t)) dt u(t i+ ) u(t i ) = h[f (t i + h, u(t i + h ))] + O(h3 ) Use Euler s method to approx u(t i + h ) u(t i) + (h/)f (t i, u(t i ) Get RKb method u i+ = u i + hf (t i + h, u i + h f (t i, u i )) J. Chaudhry (UNM) CS 357 9 / 34 J. Chaudhry (UNM) CS 357 0 / 34
RKb method RK4 method RKb Two-stage RKb method K = f (t i, u i ) K = f (t i + h, u i + h K ) u i+ = u i + hk Second order accurate as well RK4 Four stages Derived using Simpson s rule with two subintervals [t i, t i + h/] and [t i + h/, t i + h] K = f (t i, u i ) K = f (t i + h, u i + h K ) K 3 = f (t i + h, u i + h K ) K 4 = f (t i + h, u i + hk 3 ) u i+ = u i + h 6 (K + K + K 3 + K 4 ) Fourth order accurate Arguably the most widely use ODE integrator J. Chaudhry (UNM) CS 357 / 34 Geometric Interpretation of RK methods J. Chaudhry (UNM) CS 357 / 34 Stability: Euler method on a model problem Model problem (Dahlquist model problem) u (t) = λu(t), λ > 0, t [0, T] Euler (or RK) uses the slope at t i : u i = f (t i, u i ) RKa uses the slopes at t i and t i+ which are K and K RKb uses the slopes at t i and t i+ which are K and K RK4 uses the slopes t i K t i+ K +K 3 t i+ K 4 u(0) = u 0 Exact solution is: u(t) = u 0 e λt u 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0. 0. 0 0 0.05 0. 0.5 0. 0.5 0.3 0.35 0.4 0.45 0.5 t Solution u decays exponentially as t increases J. Chaudhry (UNM) CS 357 3 / 34 J. Chaudhry (UNM) CS 357 4 / 34
Stability: Euler method on a model problem What went wrong? Euler method for the model problem: u i+ = u i + h( λu i ) = ( hλ)u i, i = 0,,,... E.g. choose λ = 30, u 0 =, h = 0. u 0 0 0-0 -0-30 Euler method converges when h is small enough What about when h is not sufficiently small? Local accuracy of Euler method is nd order from Taylor series But these local errors can propagate and increase exponentially! Stability quantifies the propagation of errors In the example above we chose a step size for which Euler method was not stable -40 0 0. 0. 0.3 0.4 0.5 t Instead of exponential decay, the approximate solution has a growing oscillatory behavior! J. Chaudhry (UNM) CS 357 5 / 34 Amplification factor J. Chaudhry (UNM) CS 357 6 / 34 Amplification factor of Euler Method Euler method was u i+ = ( hλ)u i u i+ u i = hλ = G i = u i+ u i = hλ G i = u i+ /u i is called the amplification factor at time step i For Euler method, G i = hλ If amplification factor is less than one then numerical solution does not grow without bound. We say the numerical solution is stable For Euler method we need for stability G i < = hλ < = < hλ < = 0 < λh < This gives a step size restriction 0 < h < /λ In the earlier example h = 0., λ = 30 so hλ = 3 > = unstable J. Chaudhry (UNM) CS 357 7 / 34 J. Chaudhry (UNM) CS 357 8 / 34
Region of Stability Region of Stability for Euler s method Let with λ R > 0 λ = λ R + iλ I Why complex number? For system of ODEs λ represents the eigenvalues of the coefficient matrix. Remember the eigenvalues can be complex even if the matrix is real Region of stability: Region in the λh plane where the numerical solver is stable. That is, the region in which the amplification factor G i < G j = hλ = h(λ R + iλ I ) = ( hλ R ) ihλ i = ( hλ R ) + (hλ I ) < Region of stability is circle with center (0, ) and radius : hλ I hλ R J. Chaudhry (UNM) CS 357 9 / 34 Region of Stability for RKb For the model problem we get K = λu j, K = λ(u j h λu j) u j+ = u j ( hλ + (hλ) ) Amplification factor G j = u j+ u j = (hλ) + (hλ) < Want all z = hλ in the complex place such that z + z < Or, find all z such that for all φ [0, π] z + z < e iφ J. Chaudhry (UNM) CS 357 30 / 34 Region of Stability for RKb Solve the quadratic equation Two solutions Im(z) 0 - - z z + ( e iφ ) = 0 z, = ± ze iφ -0.5 0 0.5.5.5 Re(z) Region of stability is the region inside the curves J. Chaudhry (UNM) CS 357 3 / 34 J. Chaudhry (UNM) CS 357 3 / 34
Region of Stability for RK4 Region of Stability for RK4 After similar derivations as earlier we get G j = u j+ u j = (hλ) + (hλ) 6 (hλ)3 + 4 (λh)4 Want z = hλ such that G(z) = z + z 6 z3 + 4 z4 < Plot G j (z) for all possible z values and then plot the -contour of G j Listing : RK4 Stability [x,y] = meshgrid(linspace(-4,4,000), linspace(-4,4,000)); z = x + i*y; 3 contour(x,y, abs( - z + (/)*z.ˆ - (/6)*z.ˆ3 + (/4)*z.ˆ4 ),[, ], k- ); Im(z) 4 3 0 - - -3-4 - 0 3 4 Re(z) J. Chaudhry (UNM) CS 357 33 / 34 J. Chaudhry (UNM) CS 357 34 / 34