Module 4: Numerical Methods for ODE. Michael Bader. Winter 2007/2008

Size: px

Start display at page:

Download "Module 4: Numerical Methods for ODE. Michael Bader. Winter 2007/2008"

Barnaby Eugene Hopkins
5 years ago
Views:

1 Outlines Module 4: for ODE Part I: Basic Part II: Advanced Lehrstuhl Informatik V Winter 2007/2008

2 Part I: Basic 1 Direction Fields 2 Euler s Method Outlines Part I: Basic Part II: Advanced 3 Discretized Model vs. Discrete Model 4 Implicit Euler 5 Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence

3 Part II: Advanced 6 Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Outlines Part I: Basic Part II: Advanced 7 Multistep Adams-Bashforth Adams-Moulton 8 Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary

4 Direction Fields Euler s Method Part I Basic for ODE Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence

5 Motivation: Direction Fields given: initial value problem: Direction Fields Euler s Method ẏ(t) = f(t,y(t)), y(t 0 ) = y 0 easily computable: direction field 5 4 p(t) Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence t idea: follow the arrows

6 Following the Arrows direction field illustrates slope for given time t n and value y n : ẏ n = f(t n,y n ) follow arrows = make a small step in the given direction: y n+1 := y n + τẏ n = y n + τ f(t n,y n ) Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence motivates numerical scheme: y 0 := y 0 y n+1 := y n + τ f(t n,y n ) for n = 0,1,2,...

7 Euler s Method numerical scheme is called Euler s method: y n+1 := y n + τ f(t n,y n ) results from finite difference approximation: y n+1 y n τ ẏ n = f(t n,y n ) Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence (difference quotient instead of derivative) or from truncation of Taylor expansion: y(t n+1 ) = y(t n ) + τ ẏ(t n ) + O(τ 2 )

8 Euler s Method and Direction Fields Direction Fields p(t) Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence t use direction at begin of the timestep

9 Euler s Method 1D examples model of Malthus, ṗ(t) = αp(t): p n+1 := p n + τα p n Logistic Growth, ṗ(t) = α (1 p(t)/β)p(t): ( p n+1 := p n + τα 1 p ) n p n β Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence Logistic growth with threshold: ( p n+1 := p n + τα 1 p ) ( n 1 p ) n p n β δ

10 Euler s Method in 2D Euler s method is easily extended to systems of ODE use vector notation: y n+1 := y n + τ f(t n,y n ) example: nonlinear extinction model ṗ(t) = q(t) = ( ( p(t) q(t) )p(t) p(t) q(t) )q(t) Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence Euler s method: p n+1 = p n + τ ( p n q n q n+1 = q n + τ ( p n q n ) ) p n q n

11 Discretized Model vs. Discrete Model simplest example: model of Malthus p n+1 := p n τα p n, α > 0 compare to discrete model: p n+1 := p n δp n, δ > 0 with decay rate δ ( percentage ) obvious restriction in the discrete model: δ < 1 obvious restriction for τ in the discretized model? τα < 1 τ < α 1 Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence not that simple in non-linear models or systems of ODE!

12 Implicit Euler Euler s method ( explicit Euler ): y n+1 := y n + τ f(t n,y n ) implicit Euler: y n+1 := y n + τ f(t n+1,y n+1 ) results from finite difference approximation: Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence y n+1 y n τ ẏ n = f(t n+1,y n+1 ) explicit formula for y n+1 not immediately available to do: solve equation for y n+1

13 Implicit Euler and Direction Fields Direction Fields Euler s Method p(t) Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error 20 Order of Consistency/Convergence t use direction at end of the timestep

14 Implicit Euler Examples example: Model of Malthus p n+1 := p n + τα p n+1 p n+1 = 1 1 τα p n correct (discrete) model? α < 0 : then 0 < (1 τα) 1 < 1 for any τ α > 0 : then τ < α 1 required! Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence in physics α < 0 is more frequent! (damped systems, friction,... ) implicit schemes preferred when explicit schemes require very small τ

15 Implicit Euler 2D Example Direction Fields example: arms race p n+1 = b 1 + a 11 p n+1 + a 12 q n+1 q n+1 = b 2 + a 21 p n+1 + a 22 q n+1 solve linear system of equations: Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence (1 a 11 )p n+1 a 12 q n+1 = b 1 a 21 p n+1 + (1 a 22 )q n+1 = b 2 (for each time step n n + 1)

16 Local Discretisation Error local influence of using differences instead of derivatives example: Euler s method { } y(t + τ) y(t) l(τ) = max f(t,y(t)) [a,b] τ memory hook: insert exact solution y(t) into Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence y n+1 y n τ ẏ n A numerical scheme is called consistent, if l(τ) 0 for τ 0

17 Global Discretisation Error compare numerical solution with exact solution example: Euler s method e(τ) = max [a,b] { y k y(t k ) } y(t) the exact solution; y k the solution of the discretized equation (depends on τ) Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence A numerical scheme is called convergent, if e(τ) 0 for τ 0

18 Order of Consistency/Convergence Direction Fields A numerical scheme is called consistent of order p (p-th order consistent), if l(τ) = O(τ p ) A numerical scheme is called convergent of order p (p-th order convergent), if Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence e(τ) = O(τ p )

19 Runge-Kutta- Part II Advanced for ODE 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary

20 Runge-Kutta- 1st idea: use additional evaluations of f, e.g.: y n+1 = g(y n,f(t n,y n ),f(t n+1,y n+1 )) open question: where to obtain y n+1, how to choose g 2nd idea: numerical approximations for missing values of y: Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary y n+1 y n + τf(t n,y n ) y n+1 = g ( y n,f(t n,y n ),f(t n+1,y n + τf(t n,y n )) )

21 Runge-Kutta- of 2nd order 3rd idea: choose g such that order of consistency is maximal example: 2nd-order Runge-Kutta: y n+1 = y n + τ 2 ( method of Heun ) ( f(tn,y n ) + f(t n+1,y n + τf(t n,y n )) ) further example: modified Euler (also 2nd order) y n+1 = y n + τ f ( t n + τ 2,y n + τ 2 f(t n,y n ) ) Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary

22 Runge-Kutta-Method of 4th order classical 4th-order Runge-Kutta: intermediate steps: k 1 = f(t n,y n ) ( k 2 = f t n + τ 2,y n + τ ) 2 k 1 ( k 3 = f t n + τ 2,y n + τ ) 2 k 2 k 4 = f (t n + τ,y n + τk 3 ) Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary explicit scheme: y n+1 = y n + τ 6 ( k1 + 2k 2 + 2k 3 + k 4 )

23 Multistep 1st idea: use previous steps for computation: y n+1 = g(y n,y n 1,...,y n q+1 ) 2nd idea: use integral form of ODE t n+1 ẏ(t) = f(t, y(t)) t n ẏ(t)dt = y(t n+1 ) y(t n ) = t n+1 t n t n+1 f(t,y(t))dt t n f(t,y(t))dt =? Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary

24 Multistep and Quadrature 3rd idea: use numerical method for integration interpolate f using a polynomial p: y(t n+1 ) y(t n ) = where p(t j ) = f(t j,y(t j )) t n+1 t n f(t,y(t))dt t n+1 t n p(t)dt for j = n s + 1,...,n. Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary compute integral and obtain quadrature rule: y n+1 = y n + n j=n s+1 α j f(t j,y j )

25 Adams-Bashforth s = 1 use y n only (leads to Euler s method): p(t) = f(t n,y n ), y n+1 = y n + τ f(t n,y n ) s = 2 use y n 1 and y n : p(t) = t n t f(t n 1,y n 1 ) + t t n 1 f(t n,y n ), τ τ y n+1 = y n + τ ( 3f(tn,y n ) f(t n 1,y n 1 ) ) 2 Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary usually consistent of s-th order modified at start (no previous values available)

26 Adams-Moulton use idea of Adams-Bashforth, but: include value y n+1 implicit scheme first order: implicit Euler p(t) = f(t n+1,y n+1 ), y n+1 = y n +τ f(t n+1,y n+1 ) second order: y n+1 = y n + τ 2 (f(t n,y n ) + f(t n+1,y n+1 )) Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary how to obtain y k+1? solve (nonlinear) equation difficult! easier and more common: predictor-corrector approach

27 Problems for for ODE Possible problems: Ill-Conditioned Problems: small changes in the input big changes in the exact solution of the ODE Instability: big errors in the numerical solution compared to the exact solution (for arbitrarily small time steps although the method is consistent) Stiffness: small time steps required for acceptable errors in the approximate solution (although the exact solution is smooth) Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary

28 Ill-Conditioned Problems small changes in input entail completely different results treatment of such problems is always difficult! discriminate: only at critical points? everywhere? possible risks: non-precise input round-off errors,... question: what are you interested in? really the solution for specific initial condition? statistical info on the solution? general behaviour (patterns)? Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary

29 Stability Example: ẏ(t) = 2y(t) + 1, y(0) = 1 exact solution: y(t) = 1 2 (e 2t + 1) well-conditioned: y ε (0) = 1 + ε y ε (t) y(t) = εe 2t use midpoint rule (multistep scheme): Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary y n+1 = y n 1 + 2τ f(x n,y n ) leads to numerical scheme: y n+1 = y n 1 + 2τ (1 2y n )

30 Stability (2) Observation: 2-step rule: y n+1 = y n 1 + 2τ (1 2y n ) start with exact initial values: y 0 = y(0) and y 1 = y(τ) numerical results for different sizes of τ: τ = 1.0 y 9 = , y 10 = τ = 0.1 y 79 = , y 80 = τ = 0.01 y 999 = 154.6, y 1000 = midpoint rule is 2nd-order consistent, but does not converge here: oscillations or instable behaviour Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary

31 Stability (3) reason: difference equation generates spurious solutions analysis: roots µ i of characteristic polynomial y 2 = y 0 + 4τ(1 y); all µ i < 1? Stability of ODE schemes: single step schemes: always stable multistep schemes: additional stability conditions in general: consistency + stability = convergence Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary

32 Stiff Equations Example: ẏ(t) = 1000y(t) , y(0) = y 0 = 2 exact solution: y(t) = e 1000t + 1 explicit Euler (stable): y k+1 = y k + τ( 1000y k ) = (1 1000τ)y k τ = (1 1000τ) k Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary oscillations and divergence for δ t > Why that? Consistency and stability are asymptotic terms!

33 Stiff Equations Summary Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Typical situation: one term in the ODE demands very small time step but does not contribute much to the solution Remedy: use implicit (or semi-implicit) methods Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary

34 Summary Runge-Kutta-methods: multiple evaluations of f (expensive, if f is expensive to compute) stable, well-behaved, easy to implement Multistep methods: higher order, but only evaluations of f (interesting, if f is expensive to compute) stability problems; behave like wild horses in practice: do not use uniform τ and s Implicit methods: for stiff equations most often used as corrector scheme Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary

Review Higher Order methods Multistep methods Summary HIGHER ORDER METHODS. P.V. Johnson. School of Mathematics. Semester

Review Higher Order methods Multistep methods Summary HIGHER ORDER METHODS. P.V. Johnson. School of Mathematics. Semester HIGHER ORDER METHODS School of Mathematics Semester 1 2008 OUTLINE 1 REVIEW 2 HIGHER ORDER METHODS 3 MULTISTEP METHODS 4 SUMMARY OUTLINE 1 REVIEW 2 HIGHER ORDER METHODS 3 MULTISTEP METHODS 4 SUMMARY OUTLINE