Outlines Module 4: for ODE Part I: Basic Part II: Advanced Lehrstuhl Informatik V Winter 2007/2008
Part I: Basic 1 Direction Fields 2 Euler s Method Outlines Part I: Basic Part II: Advanced 3 Discretized Model vs. Discrete Model 4 Implicit Euler 5 Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence
Part II: Advanced 6 Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Outlines Part I: Basic Part II: Advanced 7 Multistep Adams-Bashforth Adams-Moulton 8 Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary
Direction Fields Euler s Method Part I Basic for ODE Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence
Motivation: Direction Fields given: initial value problem: Direction Fields Euler s Method ẏ(t) = f(t,y(t)), y(t 0 ) = y 0 easily computable: direction field 5 4 p(t) Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence 3 2 1 0 0 2 4 6 8 10 t idea: follow the arrows
Following the Arrows direction field illustrates slope for given time t n and value y n : ẏ n = f(t n,y n ) follow arrows = make a small step in the given direction: y n+1 := y n + τẏ n = y n + τ f(t n,y n ) Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence motivates numerical scheme: y 0 := y 0 y n+1 := y n + τ f(t n,y n ) for n = 0,1,2,...
Euler s Method numerical scheme is called Euler s method: y n+1 := y n + τ f(t n,y n ) results from finite difference approximation: y n+1 y n τ ẏ n = f(t n,y n ) Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence (difference quotient instead of derivative) or from truncation of Taylor expansion: y(t n+1 ) = y(t n ) + τ ẏ(t n ) + O(τ 2 )
Euler s Method and Direction Fields Direction Fields 60 50 p(t) 40 30 20 Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence 10 0 0 50 100 150 200 250 t use direction at begin of the timestep
Euler s Method 1D examples model of Malthus, ṗ(t) = αp(t): p n+1 := p n + τα p n Logistic Growth, ṗ(t) = α (1 p(t)/β)p(t): ( p n+1 := p n + τα 1 p ) n p n β Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence Logistic growth with threshold: ( p n+1 := p n + τα 1 p ) ( n 1 p ) n p n β δ
Euler s Method in 2D Euler s method is easily extended to systems of ODE use vector notation: y n+1 := y n + τ f(t n,y n ) example: nonlinear extinction model ṗ(t) = q(t) = ( 71 8 23 12 ( 73 8 25 12 p(t) 25 12 q(t) )p(t) p(t) 23 12 q(t) )q(t) Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence Euler s method: p n+1 = p n + τ ( 718 23 12 p n 25 12 q n q n+1 = q n + τ ( 738 25 12 p n 23 12 q n ) ) p n q n
Discretized Model vs. Discrete Model simplest example: model of Malthus p n+1 := p n τα p n, α > 0 compare to discrete model: p n+1 := p n δp n, δ > 0 with decay rate δ ( percentage ) obvious restriction in the discrete model: δ < 1 obvious restriction for τ in the discretized model? τα < 1 τ < α 1 Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence not that simple in non-linear models or systems of ODE!
Implicit Euler Euler s method ( explicit Euler ): y n+1 := y n + τ f(t n,y n ) implicit Euler: y n+1 := y n + τ f(t n+1,y n+1 ) results from finite difference approximation: Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence y n+1 y n τ ẏ n = f(t n+1,y n+1 ) explicit formula for y n+1 not immediately available to do: solve equation for y n+1
Implicit Euler and Direction Fields Direction Fields Euler s Method 60 50 p(t) 40 30 Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error 20 Order of Consistency/Convergence 10 0 0 50 100 150 200 250 t use direction at end of the timestep
Implicit Euler Examples example: Model of Malthus p n+1 := p n + τα p n+1 p n+1 = 1 1 τα p n correct (discrete) model? α < 0 : then 0 < (1 τα) 1 < 1 for any τ α > 0 : then τ < α 1 required! Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence in physics α < 0 is more frequent! (damped systems, friction,... ) implicit schemes preferred when explicit schemes require very small τ
Implicit Euler 2D Example Direction Fields example: arms race p n+1 = b 1 + a 11 p n+1 + a 12 q n+1 q n+1 = b 2 + a 21 p n+1 + a 22 q n+1 solve linear system of equations: Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence (1 a 11 )p n+1 a 12 q n+1 = b 1 a 21 p n+1 + (1 a 22 )q n+1 = b 2 (for each time step n n + 1)
Local Discretisation Error local influence of using differences instead of derivatives example: Euler s method { } y(t + τ) y(t) l(τ) = max f(t,y(t)) [a,b] τ memory hook: insert exact solution y(t) into Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence y n+1 y n τ ẏ n A numerical scheme is called consistent, if l(τ) 0 for τ 0
Global Discretisation Error compare numerical solution with exact solution example: Euler s method e(τ) = max [a,b] { y k y(t k ) } y(t) the exact solution; y k the solution of the discretized equation (depends on τ) Direction Fields Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence A numerical scheme is called convergent, if e(τ) 0 for τ 0
Order of Consistency/Convergence Direction Fields A numerical scheme is called consistent of order p (p-th order consistent), if l(τ) = O(τ p ) A numerical scheme is called convergent of order p (p-th order convergent), if Euler s Method Discretized Model vs. Discrete Model Implicit Euler Analysis of Schemes for ODE Local Discretisation Error Global Discretisation Error Order of Consistency/Convergence e(τ) = O(τ p )
Runge-Kutta- Part II Advanced for ODE 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary
Runge-Kutta- 1st idea: use additional evaluations of f, e.g.: y n+1 = g(y n,f(t n,y n ),f(t n+1,y n+1 )) open question: where to obtain y n+1, how to choose g 2nd idea: numerical approximations for missing values of y: Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary y n+1 y n + τf(t n,y n ) y n+1 = g ( y n,f(t n,y n ),f(t n+1,y n + τf(t n,y n )) )
Runge-Kutta- of 2nd order 3rd idea: choose g such that order of consistency is maximal example: 2nd-order Runge-Kutta: y n+1 = y n + τ 2 ( method of Heun ) ( f(tn,y n ) + f(t n+1,y n + τf(t n,y n )) ) further example: modified Euler (also 2nd order) y n+1 = y n + τ f ( t n + τ 2,y n + τ 2 f(t n,y n ) ) Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary
Runge-Kutta-Method of 4th order classical 4th-order Runge-Kutta: intermediate steps: k 1 = f(t n,y n ) ( k 2 = f t n + τ 2,y n + τ ) 2 k 1 ( k 3 = f t n + τ 2,y n + τ ) 2 k 2 k 4 = f (t n + τ,y n + τk 3 ) Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary explicit scheme: y n+1 = y n + τ 6 ( k1 + 2k 2 + 2k 3 + k 4 )
Multistep 1st idea: use previous steps for computation: y n+1 = g(y n,y n 1,...,y n q+1 ) 2nd idea: use integral form of ODE t n+1 ẏ(t) = f(t, y(t)) t n ẏ(t)dt = y(t n+1 ) y(t n ) = t n+1 t n t n+1 f(t,y(t))dt t n f(t,y(t))dt =? Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary
Multistep and Quadrature 3rd idea: use numerical method for integration interpolate f using a polynomial p: y(t n+1 ) y(t n ) = where p(t j ) = f(t j,y(t j )) t n+1 t n f(t,y(t))dt t n+1 t n p(t)dt for j = n s + 1,...,n. Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary compute integral and obtain quadrature rule: y n+1 = y n + n j=n s+1 α j f(t j,y j )
Adams-Bashforth s = 1 use y n only (leads to Euler s method): p(t) = f(t n,y n ), y n+1 = y n + τ f(t n,y n ) s = 2 use y n 1 and y n : p(t) = t n t f(t n 1,y n 1 ) + t t n 1 f(t n,y n ), τ τ y n+1 = y n + τ ( 3f(tn,y n ) f(t n 1,y n 1 ) ) 2 Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary usually consistent of s-th order modified at start (no previous values available)
Adams-Moulton use idea of Adams-Bashforth, but: include value y n+1 implicit scheme first order: implicit Euler p(t) = f(t n+1,y n+1 ), y n+1 = y n +τ f(t n+1,y n+1 ) second order: y n+1 = y n + τ 2 (f(t n,y n ) + f(t n+1,y n+1 )) Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary how to obtain y k+1? solve (nonlinear) equation difficult! easier and more common: predictor-corrector approach
Problems for for ODE Possible problems: Ill-Conditioned Problems: small changes in the input big changes in the exact solution of the ODE Instability: big errors in the numerical solution compared to the exact solution (for arbitrarily small time steps although the method is consistent) Stiffness: small time steps required for acceptable errors in the approximate solution (although the exact solution is smooth) Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary
Ill-Conditioned Problems small changes in input entail completely different results treatment of such problems is always difficult! discriminate: only at critical points? everywhere? possible risks: non-precise input round-off errors,... question: what are you interested in? really the solution for specific initial condition? statistical info on the solution? general behaviour (patterns)? Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary
Stability Example: ẏ(t) = 2y(t) + 1, y(0) = 1 exact solution: y(t) = 1 2 (e 2t + 1) well-conditioned: y ε (0) = 1 + ε y ε (t) y(t) = εe 2t use midpoint rule (multistep scheme): Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary y n+1 = y n 1 + 2τ f(x n,y n ) leads to numerical scheme: y n+1 = y n 1 + 2τ (1 2y n )
Stability (2) Observation: 2-step rule: y n+1 = y n 1 + 2τ (1 2y n ) start with exact initial values: y 0 = y(0) and y 1 = y(τ) numerical results for different sizes of τ: τ = 1.0 y 9 = 4945.5, y 10 = 20953.9 τ = 0.1 y 79 = 1725.3, y 80 = 2105.7 τ = 0.01 y 999 = 154.6, y 1000 = 158.7 midpoint rule is 2nd-order consistent, but does not converge here: oscillations or instable behaviour Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary
Stability (3) reason: difference equation generates spurious solutions analysis: roots µ i of characteristic polynomial y 2 = y 0 + 4τ(1 y); all µ i < 1? Stability of ODE schemes: single step schemes: always stable multistep schemes: additional stability conditions in general: consistency + stability = convergence Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary
Stiff Equations Example: ẏ(t) = 1000y(t) + 1000, y(0) = y 0 = 2 exact solution: y(t) = e 1000t + 1 explicit Euler (stable): y k+1 = y k + τ( 1000y k + 1000) = (1 1000τ)y k + 1000τ = (1 1000τ) k+1 + 1 Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary oscillations and divergence for δ t > 0.002 Why that? Consistency and stability are asymptotic terms!
Stiff Equations Summary Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Typical situation: one term in the ODE demands very small time step but does not contribute much to the solution Remedy: use implicit (or semi-implicit) methods Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary
Summary Runge-Kutta-methods: multiple evaluations of f (expensive, if f is expensive to compute) stable, well-behaved, easy to implement Multistep methods: higher order, but only evaluations of f (interesting, if f is expensive to compute) stability problems; behave like wild horses in practice: do not use uniform τ and s Implicit methods: for stiff equations most often used as corrector scheme Runge-Kutta- 2nd-order Runge-Kutta 4th-order Runge-Kutta Multistep Adams-Bashforth Adams-Moulton Problems for for ODE Ill-Conditioned Problems Stability Stiff Equations Summary