Lecture 5: Single Step Methods

Lecture 5: Single Step Methods J.K. Ryan@tudelft.nl WI3097TU Delft Institute of Applied Mathematics Delft University of Technology 1 October 2012 () Single Step Methods 1 October 2012 1 / 44

Outline 1 Review Difference Formula vs. Interpolation Richardson s Extrapolation 2 Initial Value Problems (IVP s) Single-step methods Amplification factor Stability Nonlinear Stability () Single Step Methods 1 October 2012 2 / 44

Difference Formula vs. Interpolation The difference formula for approximating the k th derivative, f (k) (x), is f (k) (x) Q k (h) = 1 h k n α i f (x i ). i=0 This relates to the derivative of the Lagrange interpolant by α i h k = d k dx k L i,n(x), k n () Single Step Methods 1 October 2012 3 / 44

Richardson s Extrapolation Assume the error has the form M N(h) = K 1 h α 1 + K 2 h α 2 where K 1, K 2,... are unknown constants. M = a function value. N(h) = an approximation to M We can use this knowledge to either: Create a high order approximation from low order ones; or Obtain an error estimate () Single Step Methods 1 October 2012 4 / 44

Initial Value Problems (IVP s) Goal: We want to solve the equation y (t) = f (t, y(t)) y(t 0 ) = y 0. If f (t, y(t)) is linear, i.e., f (t, y(t)) = p(t)y + g(t), there is no problem, we learned how to solve this using integrating factors. () Single Step Methods 1 October 2012 5 / 44

Initial Value Problems (IVP s) However, if f (t, y(t)) is nonlinear, we have a problem. There is no general theory for solving nonlinear differential equations. Must solve numerically. Last time, we began discussing using single-step numerical methods to solve this equation. t y(t) = y 0 + t 0 y 0 + F }{{} c constant t y (τ) dτ = y 0 + }{{} t 0 f (τ,y(τ)) (t t 0 ) }{{} interval length f (τ, y(τ)) dτ }{{} treat as a constant,f (t 0,y 0 ) = y 0 + hf c = y(t) () Single Step Methods 1 October 2012 6 / 44

Single-step methods The Euler Forward method is an explicit method and is given by w n+1 = w n + hf (t n, w n ), n = 0,..., N 1 The Euler Backward method is an implicit method and is given by w n+1 = w n + hf (t n+1, w n+1 ), n = 0,..., N 1 () Single Step Methods 1 October 2012 7 / 44

Trapezoidal Method If we take the average of the Euler Forward and Euler Backward method, then we have the Trapezoidal method: w n+1 = w n + h 2 (f (t n, w n ) + f (t n+1, w n+1 )) This is an implicit method. () Single Step Methods 1 October 2012 8 / 44

Modified Euler Predictor-corrector schemes We can also use variations on this. Such as the Modified Euler method. w n+1 = w n + hf (t n, w n ) (predict) w n+1 = w n + h 2 (f (t n, w n ) + f (t n+1, w n+1 )) (correct) Uses an Euler forward step to predict. Uses a modified Trapezoidal method to correct. It is an explicit method. () Single Step Methods 1 October 2012 9 / 44

Single-step methods Summary We now have 4 schemes to solve y (t) = f (t, y(t)) + g(t) }{{} F (t,y) y(t 0 ) = y 0. Forward Euler: w n+1 = w n + hf (t n, w n ) Backward Euler: w n+1 = w n + hf (t n+1, w n+1 ) Trapezoidal method: w n+1 = w n + h 2 (F (t n, w n ) + F (t n+1, w n+1 )) Modified Euler: w n+1 = w n + hf (t n, w n ) w n+1 = w n + h 2 (F (t n, w n ) + F (t n+1, w n+1 )) () Single Step Methods 1 October 2012 10 / 44

Single-step methods How good are these methods? What are the differences between them? () Single Step Methods 1 October 2012 11 / 44

Single-step methods Convergence Theorem (Lax equivalence theorem) A method converges if and only if it is stable and consistent. Stability + Consistency = Convergence Today, we concentrate on stability. Requires a bit of ODE theory and some new concepts. () Single Step Methods 1 October 2012 12 / 44

Single-step methods Convergence Recall: A well-posed initial value problem (IVP) is one where a solution exists the solution is unique the solution depends continuously on the initial data. We need something similar for the numerical solution. () Single Step Methods 1 October 2012 13 / 44

Single-step methods Amplification factor Definition (Amplification factor) The amplification factor, Q(λh), is given by w n+1 = Q(λh)w n and is used to determine stability of a numerical scheme. To study this, consider a test IVP y (t) = λy(t) y(0) = y 0 where f (t, y) = λy and the solution is given by y(t) = y 0 e λ(t t 0). () Single Step Methods 1 October 2012 14 / 44

Amplification factor Euler forward Applying the Euler forward method to y (t) = λy(t), we would have w n+1 = w n + h f (t n, w n ) }{{} f (t n,w n)=λw n = w n + hλw n = (1 + hλ)w n w n+1 = (1 + hλ)w n The amplification factor for the forward Euler method is given by Q(hλ) = 1 + hλ () Single Step Methods 1 October 2012 15 / 44

Amplification factor Euler backward For the Euler backward method, we have to work a bit harder: w n+1 = w n + h f (t n+1, w n+1 ) }{{} f (t n+1,w n+1 )=λw n+1 = w n + hλw n+1 w n+1 hλw n+1 = w n 1 w n+1 = 1 hλ w n The amplification factor for the backward Euler method is given by Q(hλ) = 1 1 hλ () Single Step Methods 1 October 2012 16 / 44

Amplification factor Trapezoidal method The amplification factor for trapezoidal method can be written as: w n+1 = w n + h 2 w n+1 = w n + hλ 2 (w n + w n+1 ) f (t n, w n ) + f (t n+1, w n+1 ) }{{}}{{} λw n λw n+1 w n+1 h 2 λw n+1 = (1 + h 2 λ)w n w n+1 = 1 + h 2 λ 1 h 2 λw n The amplification factor for the trapezoidal method is given by Q(hλ) = 1 + h 2 λ 1 h 2 λ () Single Step Methods 1 October 2012 17 / 44

Amplification factor Modified Euler To find the amplification factor for modified Euler, we need to rewrite the method as: w n+1 = w n + h 2 f (t n, w n ) + f (t n+1, w n + hf (t n, w n )) }{{}}{{} λw n λ(w n+hλw n) = w n + (hλ + 1 ) 2 (hλ)2 w n w n+1 = (1 + hλ + 1 ) 2 (hλ)2 w n So that the amplification factor for the modified Euler method is given by Q(hλ) = 1 + hλ + 1 2 (hλ)2 () Single Step Methods 1 October 2012 18 / 44

Stability Using the amplification factor, we can get insight into the stability of the numerical method. Definition (Stability) A method is stable if small perturbations of the parameters results in only a small difference in the solution. () Single Step Methods 1 October 2012 19 / 44

Stability Initial Value Problem The exact initial value problem is given as y (t) = λy(t) + g(t) y(0) = y 0 Consider a small perturbation of the initial condition: ỹ (t) = λỹ(t) + g(t) ỹ(0) = y 0 + ɛ 0 where ɛ 0 is small. Define ɛ(t) = ỹ(t) y(t). () Single Step Methods 1 October 2012 20 / 44

Stability Initial Value Problem The problem is stable if ɛ(t) = ỹ(t) y(t) is finite for all t > 0. That is lim t ɛ(t) < ỹ (t) y (t) = (λỹ(t) + g(t)) (λy(t) + g(t)) ɛ (t) = λɛ(t) = λ(ỹ(t) y(t)) () Single Step Methods 1 October 2012 21 / 44

Stability Initial Value Problem We have the new PDE: ɛ (t) = λɛ(t) ɛ(0) = ɛ 0, which has solution ɛ(t) = ɛ 0 e λt. Clearly, if and lim ɛ(t) = lim ɛ 0e λt = 0 t t λ < 0 lim ɛ(t) = ɛ 0 if λ = 0. t () Single Step Methods 1 October 2012 22 / 44

Numerical Stability We need to apply these concepts numerically. To review: for the stability of the IVP We need lim t ɛ(t) = 0. This means that λ < 0. Remark: The IVP needs to be stable to study the stability of the numerical method. () Single Step Methods 1 October 2012 23 / 44

Numerical Stability Exact Perturbed w n+1 = Q(hλ)w n w 0 = y 0 w n+1 = Q(hλ) w n w 0 = y 0 + ɛ 0 Define ɛ n+1 = w n+1 w n+1. The numerical stability is given by examining ɛ n+1 = Q(hλ)ɛ n. The numerical stability condition is then lim ɛ n+1 = 0. n () Single Step Methods 1 October 2012 24 / 44

Numerical Stability We can simplify this. ɛ n+1 = Q(hλ)ɛ n = Q(hλ) (Q(hλ)ɛ n 1 ) = (Q(hλ)) 2 ɛ n 1 = (Q(hλ)) 3 ɛ n 2 = For numerical stability, This only happens if ɛ n+1 = (Q(hλ)) n+1 ɛ 0 lim ɛ n+1 = lim Q(hλ) n+1 ɛ 0 n h 0 }{{} 0. Q(hλ) < 1. want () Single Step Methods 1 October 2012 25 / 44

Numerical Stability To summarize, we can write the numerical method in terms of the amplification factor, The method is stable if w n+1 = Q(hλ)w n. Q(hλ) < 1. We use this to determine the maximum h =step size for which the method is stable. () Single Step Methods 1 October 2012 26 / 44

Numerical Stability Euler Forward: (Explicit) For stability, we must have w n+1 = Q(hλ)w n = (1 + hλ)w n Q(hλ) = 1 + hλ < 1 where λ < 0. This gives 0 < h < 2 λ () Single Step Methods 1 October 2012 27 / 44

Numerical Stability Euler Backward: (Implicit) w n+1 = Q(hλ)w n = 1 1 hλ w n For stability, we must have Q(hλ) = 1 1 hλ < 1 where λ < 0. This gives h > 0. So that the Euler Backward method is unconditionally stable. () Single Step Methods 1 October 2012 28 / 44

Numerical Stability Trapezoidal Method: (Implicit) w n+1 = Q(hλ)w n = 1 + h 2 λ 1 h 2 λw n For stability, we must have 1 + h 2 Q(hλ) = λ 1 h 2 λ < 1 where λ < 0. This gives h > 0. So that the Trapezoidal method is unconditionally stable. () Single Step Methods 1 October 2012 29 / 44

Numerical Stability Modified Euler: (Explicit) w n+1 = Q(hλ)w n = (1 + hλ + 1 2 (hλ)2 )w n For stability, we must have Q(hλ) = 1 + hλ + 1 2 (hλ)2 < 1 where λ < 0. This gives 0 < h < 2 λ. () Single Step Methods 1 October 2012 30 / 44

Nonlinear Stability A general nonlinear IVP can be written as y (t) = f (t, y(t)) y(t 0 ) = y 0 To find the stability, we have to use Taylor expansion. Expand around (ˆt, ŷ) : f (t, y) f (ˆt, ŷ) + (y ŷ)f y (ˆt, ŷ) + (t ˆt)f t (ˆt, ŷ) + y (t) f (ˆt, ŷ) + (y ŷ)f y (ˆt, ŷ) + (t ˆt)f t (ˆt, ŷ) + So that y (t) f (ˆt, ŷ) (y ŷ) f y (ˆt, ŷ) +(t ˆt)f t (ˆt, ŷ) + }{{}}{{}}{{} ɛ (t) ɛ(t) λ () Single Step Methods 1 October 2012 31 / 44

Nonlinear Stability For nonlinear equations, the stability is given by λ = f y (ˆt, ŷ). Stability depends on ˆt, ŷ. Recall that λ should be negative. () Single Step Methods 1 October 2012 32 / 44

Nonlinear Stability Example Given the initial value problem y (t) = 1 + (t y) 2, t [2, 3] y(2) = 1, (1) what is the stability condition using modified Euler? () Single Step Methods 1 October 2012 33 / 44

Nonlinear Stability In this example, It is a nonlinear equation with f (t, y) = 1 + (t y) 2. f (t, y) f (2, 1) + (t 2)f t (2, 1) + (y 1)f y (2, 1) = 2 + 2(t 2) 2(y 1) = 2y 2t = 2y + g(t) (2) It is almost like the linear problem y (t) = 2y + g(t) with λ = 2. () Single Step Methods 1 October 2012 34 / 44

Nonlinear Stability Therefore, for stability of the the nonlinear problem y (t) = 1 + (t y) 2, we actually consider the stability of the linear problem y (t) = 2y + g(t). But, we know that the stability of modified Euler is given by 0 < h < 2 λ < 2 2 = 1. () Single Step Methods 1 October 2012 35 / 44

Example Modified Euler Example Use the modified Euler method to approximate the solution to the initial value problem y (t) = 1 + (t y) 2, t [2, 3] y(2) = 1, (3) with h = 0.5. The exact solution is given by y(t) = t + 1 1 t. Determine the error in the numerical approximation. () Single Step Methods 1 October 2012 36 / 44

Example Modified Euler From the previous, we know that h = 0.5 falls within the stability range, 0 < h < 1. The modified Euler method is given by: w n+1 = w n + hf (t n, w n ) w n+1 = w n + h 2 (F (t n, w n ) + F (t n+1, w n+1 )) h = 0.5 t 0 = 2, t 1 = 2.5, t 2 = 3 with w 0 = y 0 = 1, and F (t, y) = 1 + (t y) 2. We have to determine w 1, w 2. () Single Step Methods 1 October 2012 37 / 44

Example Modified Euler For w 1 : This becomes w 1 = w 0 + h F (t 0, w 0 ) }{{} 1+(t 0 w 0 ) 2 w 1 = w 0 + h 2 (F (t 0, w 0 ) + F (t 1, w 1 )) }{{} 1+(t 1 w 1 ) 2 w 1 = 1 + 0.5(1 + (2 1) 2 ) = 2 w 1 = 1 + 0.25((1 + (2 1) 2 ) + (1 + (2.5 2) 2 )) = 1.8125 = w 1 The actual value is y(2.5) = 2.5 + 1 1 2.5 = 1.8333 () Single Step Methods 1 October 2012 38 / 44

Example Modified Euler For w 2 : This becomes w 2 = w 1 + h F (t 1, w 1 ) }{{} 1+(t 1 w 1 ) 2 w 2 = w 1 + h 2 (F (t 1, w 1 ) + F (t 2, w 2 )) }{{} 1+(t 2 w 2 ) 2 w 2 = 2.5488 w 2 = 2.4816 The actual value is y(3) = 3 + 1 1 3 = 2.5 () Single Step Methods 1 October 2012 39 / 44

Example Modified Euler To summarize: n t n y(t) w n Error 0 2 1 1 0 1 2.5 1.8333 1.8125 0.0208 2 3 2.5 2.4816 0.0184 where w n is obtained using the modified Euler () Single Step Methods 1 October 2012 40 / 44

Summary Today, we have addressed the following important topics: Lax Equivalence Theorem: For a linear scheme, Stability + Consistency = Convergence Stability is how much small perturbations affect the solution. For this we have to look at Stability of the initial value problem (IVP) Stability of the numerical method () Single Step Methods 1 October 2012 41 / 44

Summary IVP Stability For the stability of the initial value problem (IVP), we consider ɛ(t) = There are three possibilities: ỹ(t) y(t) }{{} perturbed problem 1 It is unstable if lim t ɛ(t) 2 It is stable if lim t ɛ(t) < (finite) 3 It is unconditionally stable if lim t ɛ(t) 0 Usually, this equates to λ < 0. () Single Step Methods 1 October 2012 42 / 44

Summary Numerical Stability For numerical stability, we must consider the amplification factor, Q(hλ) in The possiblities are: w n+1 = Q(hλ)w n = (Q(hλ)) n+1 w 0. 1 The method is unstable if lim h 0 Q(hλ) > 1. 2 The method is stable if lim h 0 Q(hλ) < 1. We use these conditions to choose an appropriate bound on h. () Single Step Methods 1 October 2012 43 / 44

Material addressed 1 Review Difference Formula vs. Interpolation Richardson s Extrapolation 2 Initial Value Problems (IVP s) Single-step methods Amplification factor Stability Nonlinear Stability Material in book: Chapter 6 Useful exercises: 1,3,4,5b,5c,6 () Single Step Methods 1 October 2012 44 / 44