2 Numerical Methods for Initial Value Problems

Transcription

1 Numerical Analysis of Differential Equations 44 2 Numerical Methods for Initial Value Problems Contents 2.1 Some Simple Methods 2.2 One-Step Methods Definition and Properties 2.3 Runge-Kutta-Methods 2.4 Linear Multi-Step Methods 2.5 Convergence of One-Step Methods 2.6 Zero-Stability of Linear Multistep Methods 2.7 Stepsize Control 2 Numerical Methods for Initial Value Problems TU Bergakademie Freiberg, SS 2012

2 Numerical Analysis of Differential Equations Some Simple Methods Consider an IVP y = f (t, y), y(t 0 ) = y 0. (IVP) Under the assumptions of Theorem 1.1, (IVP) possesses a unique solution y = y(t), say, on the interval I. We will approximate y(t) for t [t 0, t end ] I using the Euler method a (sometimes called the explicit Euler method), which can be regarded as a prototype for all numerical schemes for solving IVPs. As all methods we shall encounter in this course, Euler s method is based on discretization, i.e., approximating the solution y on only a discrete subset {t n : n = 0, 1,..., N} of the interval [t 0, t end ]. a LEONHARD EULER ( ) 2.1 Some Simple Methods TU Bergakademie Freiberg, SS 2012

3 Numerical Analysis of Differential Equations 46 We fix N N, set h := (t end t 0 )/N and define t n := t 0 +nh, n = 0, 1,..., N, i.e. t 0 < t 1 < t N 1 < t N = t end. The number h > 0 is called the stepsize, which we choose to be constant for convenience only. One step of Euler s method requires an approximation y n to y(t n ), n = 0, 1,..., N 1. Noting that, under the assumption y C 2 [t 0, t end ], y(t n+1 ) = y(t n + h) = y(t n ) + hy (t n ) h2 y (ξ) we obtain the y(t n ) + hy (t n ) = y(t n ) + hf (t n, y(t n )) y n + hf (t n, y n ), (Explicit) Euler scheme: y 0 given, y n+1 := y n + h f (t n, y n ) n = 0, 1,..., N Some Simple Methods TU Bergakademie Freiberg, SS 2012

4 Numerical Analysis of Differential Equations 47 A common way to visualize a first order ODE y = f(t, y) is provided by its associated direction field, obtained by drawing an arrow with slope y = f(t, y) at every point in the (t, y)-plane. The graph of the solution to the IVP y = f(t, y), y(t 0 ) = y 0, must pass through the point (t 0, y 0 ) and its tangents at every point (t, y(t)) must be parallel to the direction field. The following figure illustrates the Euler method approximating the solution of the logistic equation y = y(1 y) with IC y(0) = 1/10 using the step size h = 1. Rather than following its exact trajectory (which is, of course, impossible), the Euler scheme may be viewed at producing a piecewise linear approximation. At the starting point t 0 the Euler approximation uses the correct slope f(t 0, y 0 ) = 9/100. However, from the next point t 1 = h = 1 onwards the slope is wrong, and the approximation may deviate further and further from the exact solution. 2.1 Some Simple Methods TU Bergakademie Freiberg, SS 2012

5 Numerical Analysis of Differential Equations Euler Verfahren y =y(1 y), y(0)=1/ Some Simple Methods TU Bergakademie Freiberg, SS 2012

6 Numerical Analysis of Differential Equations 49 4 Richtungsfeld von y =x(y 2) und Loesungen mit Anfangswerten y(0) = 1:0.5:1 2.1 Some Simple Methods TU Bergakademie Freiberg, SS 2012

7 Numerical Analysis of Differential Equations 50 Does Euler s method converge, i.e., do the approximations converge to the exact solution y(t) as h 0? Note: each value of the step size h is associated with a (finite) collection of approximations y n = y n (h), n = 0, 1,..., N(h) := (t end t 0 )/h. The method is said to converge (in [t 0, t end ]) if lim h 0+ max y n(h) y(t n ) = 0. 0 n N(h) (Conv) Theorem 2.1 (Convergence of Euler s method) Under the assumptions of Theorem 1.1 the Euler method is convergent. More precisely, max y n(h) y(t n ) = O(h) as h 0. 0 n N(h) 2.1 Some Simple Methods TU Bergakademie Freiberg, SS 2012

8 Numerical Analysis of Differential Equations 51 exakte Loesung Schrittweite h 0 Schrittweite h 1 Schrittweite h 2 t_0 t 2.1 Some Simple Methods TU Bergakademie Freiberg, SS 2012

9 Numerical Analysis of Differential Equations 52 Modified Euler method: y 0 given, y n+1 := y n + h f ( t n h, y n hf (t n, y n ) ) n = 0, 1,..., N 1. Improved Euler method: y 0 gegeben, y n+1 := y n h[f (t n, y n ) + f (t n + h, y n + hf (t n, y n ))] n = 0, 1,..., N 1. The construction of these variants of Euler s method is also easily interpreted using the direction field. Both are convergent schemes in fact (by a trivial modification of the proof of Theorem 2.1) one can show that for both methods max y n(h) y(t n ) = O(h 2 ), as h 0. 0 n N(h) 2.1 Some Simple Methods TU Bergakademie Freiberg, SS 2012

10 Numerical Analysis of Differential Equations exakte Loesung Euler Verfahren modifiziertes Euler Verf. verbessertes Euler Verf Some Simple Methods TU Bergakademie Freiberg, SS 2012

11 Numerical Analysis of Differential Equations 54 Implicit Euler method: y 0 given, y n+1 := y n + h f (t n + h, y n+1 ) n = 0, 1,..., N 1. In contrast with the methods introduced so far, this is an implicit method, which means that determining y n+1 from y n requires the solution of a system of algebraic equations, which, unless f is linear in y, is generally nonlinear. As in Theorem 2.1 one can show that for the implicit Euler method max y n(h) y(t n ) = O(h) as h 0. 0 n N(h) 2.1 Some Simple Methods TU Bergakademie Freiberg, SS 2012

12 Numerical Analysis of Differential Equations 55 The schemes we have considered up to now involved two approximations of the solution at consecutive times. Such schemes are called one-step methods. Multistep methods are based on difference formulas involving more than two solution values. Examples are y(t + h) y(t h) 2h 3y(t) 4y(t h) + y(t 2h) 2h = y (t) + h2 6 y (t) + O(h 4 ), = y (t) h2 3 y (t) + O(h 3 ). When these formulas are evaluated at t = t n+1, we obtain Some Simple Methods TU Bergakademie Freiberg, SS 2012

13 Numerical Analysis of Differential Equations the explicit Midpoint-rule: (leapfrog method) y 0, y 1 given, y n+1 = y n 1 + 2h f (t n, y n ), n = 1, 2,..., N 1. (2.1) as well as an implicit method belonging to the so-called BDF-family (Backward Differentiation Methods) A BDF method: y 0, y 1 given, 3y n+1 4y n + y n 1 = 2hf (t n + h, y n+1 ), n = 1, 2,..., N 1. We observe that these methods can only proceed once, in addition to the initial vaue y 0, an approximation y 1 is available. All multistep methods require such a startup calculation. 2.1 Some Simple Methods TU Bergakademie Freiberg, SS 2012

14 Numerical Analysis of Differential Equations One-Step Methods Definition and Properties A one-step method (OSM) has the general form y n+1 = y n + h Φ f (y n+1, y n, t n ; h). pos- We will consider exclusively methods whose increment function Φ f sesses the following properties: (OSM) Φ f 0 (y n+1, y n, t n ; h) 0 (V 1 ) and Φ f (y n+1, y n, t n ; h) Φ f (y n+1, y n, t n ; h) M 1 y n+j yn+j. (V 2 ) j=0 For reasonable OSM property (V 2 ) follows from the Lipschitz continuity of f (cf. Theorem 1.1), which we always assume to hold. 2.2 One-Step Methods Definition and Properties TU Bergakademie Freiberg, SS 2012

15 Numerical Analysis of Differential Equations 58 Examples of more complicated OSMs: y n+1 y n = 1 4 h(k 1+3k 3 ), where k 1 = f (t n, y n ), k 2 = f (t n h, y n hk 1), k 3 = f (t n h, y n hk 2), (Example 1) an explicit OSM belonging to the class of Runge-Kutta methods a b (cf. 2.3) and the implicit Runge-Kutta method y n+1 y n = 1 2 h(k 1+k 2 ), where k 1 = f (t n, y n ), k 2 = f (t n + h, y n hk hk 2). (Example 2) a CARLE DAVID TOLMÉ RUNGE ( ) b MARTIN WILHELM KUTTA ( ) 2.2 One-Step Methods Definition and Properties TU Bergakademie Freiberg, SS 2012

16 Numerical Analysis of Differential Equations 59 We say the method (OSM) is convergent if lim h 0 t=t 0 +nh y n = lim h 0 t=t 0 +nh y n (h) = y(t) holds and does so for all IVPs which satisfy the assumptions of Theorem 1.1 where y(t) denotes the solution of such an IVP, uniformly for all t [t 0, t end ], for all solutions {y n (h)} = {y n } of (OSM) with initial values y 0 (h) satisfying lim h 0 y 0 (h) = y 0. Equivalent: The global discretisation error converges to 0 uniformly (as h 0): lim h 0 e n = e n (h) := y(t n ) y n (h) max e n(h) = lim 0 n N h 0 max y(t n) y n (h) = 0. 0 n N 2.2 One-Step Methods Definition and Properties TU Bergakademie Freiberg, SS 2012

17 Numerical Analysis of Differential Equations 60 We define the local discretion error (local truncation error) T n = T n (h) of a method at step n to be the difference of the left and right-hand sides of the expression defining the method (scaled in such a way that the differential equation is approximated as h 0) when the exact solution is substituted for the approximation. A method is said to be consistent of order p if T n (h) = O(h p ) as h 0. Example: Midpoint rule T n = y(t n+1) y(t n 1 ) 2h f(t n, y(t n )) = ] [y (t n ) + h2 6 y (t n ) + O(h 4 ) y (t n ) = h2 6 y (t n ) + O(h 4 ), i.e., we have consistency of order p = One-Step Methods Definition and Properties TU Bergakademie Freiberg, SS 2012

18 Numerical Analysis of Differential Equations 61 Warning: In the literature one often finds a slightly different definition of the local discretization error based on the method s defining equation in the form (OSM). This increases the order of consistency by one power of h. Motivation: Description of the error incurred in a single step independent of that accumulated from previous steps using the localization assumption y n = y(t n ). In this case one step of the scheme (OSM) compared with the exact solution compares as ŷ n+1 (h) = y(t n ) + hφ f (ŷ n+1, y(t n ), t n ; h), y(t n+1 ) = y(t n ) + hφ f (y(t n+1 ), y(t n ), t n ; h) + ht n+1 (h). Denoting the difference S n+1 (h) := y(t n+1 ) ŷ n+1 as step error, we obtain using (V 2 ), S n+1 h Φ f (y(t n+1 ), y(t n ), t n ; h) Φ f (ŷ n+1, y(t n ), t n ; h) + h T n+1 hm S n+1 + h T n+1, implying (1 hm) S n+1 ht n+1 so that S n behaves like ht n as h One-Step Methods Definition and Properties TU Bergakademie Freiberg, SS 2012

19 Numerical Analysis of Differential Equations 62 Relevance of S n : The integration of (IVP) from t 0 to t end requires (t end t 0 )/h steps. If step error S n is committed at step n, this results, under the simplifying assumption that individual error have no effect on each other, in an accumulated (global) error of t end t 0 h S(h) T (h). As we shall see shortly, the crucial property for the validity of this simplifying assumption is the stability of the method. 2.2 One-Step Methods Definition and Properties TU Bergakademie Freiberg, SS 2012

20 Numerical Analysis of Differential Equations Runge-Kutta Methods Derivation via Quadrature Fundamental theorem of calculus: y(t + h) = y(t) + [y(t + h) y(t)] = y(t) + Change of variables: s = t + τh, 0 τ 1 t+h t y (s) ds = y(t) + h 1 0 y (t + τh) dτ. Approximate integral with quadrature formula: 1 0 g(τ) dτ m β j g(γ j ). j=1 ( ) To integrate at least g 1 exactly, we require m j=1 β j = Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

21 Numerical Analysis of Differential Equations 64 We obtain y(t + h) y(t) + h = y(t) + h m β j y (t + γ j h) j=1 m β j f (t + γ j h, y(t + γ j h)) j=1 (RK-1) Problem now: don t know y(t + γ j h) = y(t) + h γ j 0 y (t + τh) dτ. Again approximate by quadrature, but using same nodes {γ j } m j=1 to avoid introducing new unknowns y(t + node h): as in ( ) γj 0 g(τ) dτ m α j,l g(γ l ), j = 1,..., m. ( ) l=1 To integrate at least g 1 exactly, require m l=1 α j,l = γ j, j = 1,..., m. 2.3 Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

22 Numerical Analysis of Differential Equations 65 This yields y(t + γ j h) y(t) + h = y(t) + h m α j,l y (t + γ l h) l=1 m α j,l f (t + γ l h, y(t + γ l h)) l=1 (RK-2) Introduce kj := f ( t + γ j h, y(t + γ j h) ), j = 1,..., m. (RK-2): (RK-1): kj f ( t + γ j h, y(t) + h y(t + h) y(t) + h m α j,l kl ), j = 1,..., m. l=1 m β j kj. j=1 2.3 Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

23 Numerical Analysis of Differential Equations 66 m-stage Runge-Kutta method (RKM): m y n+1 = y n + h β j k j ( j=1 k j = f t n + γ j h, y n + h where ) m α j,l k l, j = 1,..., m. l=1 (RKM) Shorthand notation for RKMs: Butcher Tableau a γ 1 α 1,1 α 1,m... γ m α m,1 α m,m a JOHN CHARLES BUTCHER ( 1933) β 1 β m 2.3 Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

24 Numerical Analysis of Differential Equations 67 Examples. The Butcher Tableau /2 1/2 represents a two-stage explicit RKM, the improved Euler method: [ y n+1 = y n + h 2 f(tn, y n ) + f(t n + h, y n + hf(t n, y n )) ] = y n + h 2 (k 1 + k 2 ), k 1 = f(t n, y n ), k 2 = f(t n + h, y n + hk 1 ) Note: a RKM is explicit when α j,l = 0 j l, i.e., if the coefficients α j,l form a strictly lower triangular matrix. 2.3 Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

25 Numerical Analysis of Differential Equations /4 1/4 2/3 1/4 5/12 1/4 3/4 represents a two-stage implicit RKM: k 1 = f ( t n, y n + h( 1 4 k k 2) ), k 2 = f ( t n h, y n + h( 1 4 k k 2) ), ( two generally nonlinear equations for k 1 and k 2 ) y n+1 = y n + h 4 (k 1 + 3k 2 ). (Example 2 in 2.2 is a further example of an implicit two-stage RKM.) 2.3 Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

26 Numerical Analysis of Differential Equations /2 1/ /6 4/6 1/6 represents an explicit three-stage RKM Kutta s third-order method: k 1 = f (t n, y n ), k 2 = f ( t n h, y n hk 1), k 3 = f ( t n + h, y n + h( k 1 + 2k 2 ) ), y n+1 = y n h(k 1 + 4k 2 + k 3 ). 2.3 Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

27 Numerical Analysis of Differential Equations /3 1/ /3 0 2/3 0 1/4 0 3/4 represents another explicit three-stage RKM Heun s third-order method: k 1 = f (t n, y n ), k 2 = f ( t n h, y n hk 1), k 3 = f ( t n h, y n hk 2), y n+1 = y n + h 4 (k 1 + 3k 3 ) (cf. Example 1 in 2.2). 2.3 Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

28 Numerical Analysis of Differential Equations /2 1/ /2 0 1/ /6 2/6 2/6 1/6 represents an explicit four-stage RKM Classical Runge-Kutta method: k 1 = f (t n, y n ), k 2 = f (t n h, y n hk 1), k 3 = f (t n h, y n hk 2), k 4 = f (t n + h, y n + hk 3 ), y n+1 = y n h(k 1 + 2k 2 + 2k 3 + k 4 ). 2.3 Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

29 Numerical Analysis of Differential Equations 72 An (equivalent) alternative way of representing (RKM) is m y n+1 = y n + h β j f (t n + γ j h, ỹ j ) j=1 m where ỹ j = y n + h α j,l f (t n + γ l h, ỹ l ), l=1 j = 1,..., m (RKM ) (simply set k j = f (t n + γ j h, ỹ j )). Interpretation: ỹ j : approximate solution y at time t n + γ j h k j : approximate slopes y at time t n + γ j h 2.3 Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

30 Numerical Analysis of Differential Equations Consistency of Runge-Kutta Methods All RKM can be written y n+1 = y n +h Φ f (y n+1, y n, t n ; h) where Φ f (y n+1, y n, t n ; h) = m β j k j. j=1 A RKM is consistent (and therefore convergent) if, and only if, m β j = 1. j=1 Determining the order of consistency of RKMs (or constructing m-stage RKMs with maximal order of consistency) often leads to rather involved calculations. 2.3 Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

31 Numerical Analysis of Differential Equations 74 As a tractable example, we shall analyze all explicit three-stage RKMs. Any such method has a Butcher Tableau of the form γ 2 γ γ 3 γ 3 α 3,2 α 3,2 0. We expand the local truncation error β 1 β 2 β 3 T n+1 (h) = y(t n+1) y(t n ) h = y(t n+1) y(t n ) h Φ f (y(t n ), t n ; h) 3 β j k j j=1 in powers of h (assuming y and f sufficiently differentiable). 2.3 Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

32 Numerical Analysis of Differential Equations 75 We consider scalar IVPs and, introducing the abbreviations F := f t + f y f and G := f tt + 2f ty f + f yy f 2, all derivatives of f evaluated at (t n, y(t n )) employ the chain rule to obtain y(t n+1 ) y(t n ) h On the other hand, we also have k 1 = f(t n, y(t n )) = f, = f F h (G + f yf )h 2 + O(h 3 ). k 2 = f(t n + hγ 2, y(t n ) + hγ 2 k 1 ) = f + hγ 2 F h2 γ 2 2G + O(h 3 ), k 3 = f(t n + hγ 3, y(t n ) + h(γ 3 α 3,2 )k 1 + hα 3,2 k 2 ) = f + hγ 3 F + h 2 (γ 2 α 3,2 F f y γ2 3G) + O(h 3 ). 2.3 Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

33 Numerical Analysis of Differential Equations 76 This means: T n+1 (h) = Consequences: [ 1 ] 3 j=1 β j f + [ 1 2 β ] 2γ 2 β 3 γ 3 F h + [ ( 1 3 β 2γ 2 2 β 3 γ 2 3) 1 2 G + ( 1 6 β 3γ 2 α 3,2 )F f y ] h 2 + O(h 3 ) (1) The Euler method is the only explicit one-stage RKM of order one (β 1 = 1). (2) All explicit two-stage RKM of order 2 are characterized by β 1 + β 2 = 1 and β 2 γ 2 = 1 2. Examples are the modified (β 1 = 0, β 2 = 1, γ 2 = 1 2 ) and the improved (β 1 = β 2 = 1 2, γ 2 = 1) Euler methods. There are no explicit two-stage RKMs of order three or higher. 2.3 Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

34 Numerical Analysis of Differential Equations 77 (3) Explicit three-stage RKM of order 3 are characterized by the four equations β 1 + β 2 + β 3 = 1, β 2 γ β 3 γ 2 3 = 1 3, β 2 γ 2 + β 3 γ 3 = 1 2, β 3γ 2 α 3,2 = 1 6. (One can show that none of these methods have order 4.) Examples include Heun s method (β 1 = 1 4, β 2 = 0, β 3 = 3 4, γ 2 = 1 3, γ 3 = α 3,2 = 2 3 ) and Kutta s method (β 1 = 1 6, β 2 = 2 3, β 3 = 1 6, γ 2 = 1 2, γ 3 = 1, α 3,2 = 2). (4) Similar (complicated) calculations reveal the existence of a two-parameter family of explicit four-stage RKMs, none of which have order 5. An example is the classical RKM; further examples are Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

35 Numerical Analysis of Differential Equations /3 1/ /3 1/ /8 3/8 3/8 1/8 (3/8-rule) /5 2/ /5 3/20 3/ /44 15/44 40/ / / /360 55/360 (Kuntzmann s method). 2.3 Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

36 Numerical Analysis of Differential Equations 79 This way of analyzing consistency of explicit RKMs becomes increasingly tedious for higher orders: order 3: 4 nonlinear equations for coefficients (see above) order 8: 200 nonlinear equations for coefficients. Butcher Theory a uses graph theory (trees) for systematic book-keeping of partial derivatives of f for computing the order of a given RKM. It does not, however, provide a technique for constructing RKMs of a desired order. In the following, we derive a sequence of necessary order conditions, obtained from the specific family of IVPs y = y + t l 1, y(0) = 0, (l N). a J. C. Butcher, The Numerical Analysis of Ordinary Differential Equations. Runge-Kutta and General Linear Methods. John Wiley & Sons, Chichester Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

37 Numerical Analysis of Differential Equations 80 Theorem 2.2 (Necessary order conditions for RKM) In order for the RKM defined by the Butcher tableau c A b to possess consistency order p there must hold b A k C l 1 e = (l 1)! (l + k)! = 1 l(l + 1)... (l + k) for l = 1, 2,..., p and k = 0, 1,..., p l. Notation: b := [β 1, β 2,..., β m ], A := [α j,ν ] 1 j,ν m, C := diag(γ 1, γ 2,..., γ m ) and e := [1, 1,..., 1] R m. 2.3 Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

38 Numerical Analysis of Differential Equations 81 Special cases of the necessary conditions of Theorem 2.2 are (for k = 0) b C l 1 e = m j=1 β j γ l 1 j = 1 l for l = 1, 2,..., p as well as (for l = 1 with k k + 1) b A k 1 e = 1 k! for k = 1, 2,..., p. Remark. An explicit m-stage RKM can have at most order m, since in this case A m = O (A ist a strictly lower trianglular matrix). In fact, for m 5 the maximal order p(m) attainable by an explicit m-stage RKM is bounded by p(m) m 1. The exact maximal orders for 1 m 12 are m p(m) Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012

39 Numerical Analysis of Differential Equations Linear Multistep Methods Multistep methods are characterized by using solution approximations earlier than y n in the update formula for advancing the numerical approximation from y n to y n+1. This is in contrast with one-step methods, which use only y n, besides possibly additional intermediate (e.g. stage) quantities. Of these, the most important class of linear multistep methods (LMM) possesses the general form r α j y n+j = h j=0 r β j f (t n+j, y n+j ), n = 0, 1, 2,.... (LMM) j=0 defined by the parameters {α j } r j=0 and {β j} r j=0. Since these are unique only up to a (nonzero) scaling factor, they are typically normalized by the condition α r = 1. The method (LMM) is explicit if β r = 0, otherwise implicit. We first study various subfamilies of LMM. 2.4 Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

40 Numerical Analysis of Differential Equations Adams-Methods Adams-Methods have the specific form y n+r y n+r 1 = h r β j f (t n+j, y n+j ), (2.2) j=0 i.e., they are LMMs characterized by α r = 1, α r 1 = 1, in addition to α 0 = α 1 = = α r 2 = 0. The coefficients β j result from maximizing the order of consistency for a given value of r, which is r in the explicit case and r + 1 in the implicit case. The former class of methods are known as Adams-Bashforth methods, the latter Adams-Moulton methods. 2.4 Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

41 Numerical Analysis of Differential Equations 84 Alternative derivation of Adams methods: From the differential equation, we have y(t n+r ) = y(t n+r 1 ) + tn+r tn+r y (τ) dτ = y(t n+r 1 ) + f (τ, y(τ)) dτ. t n+r 1 t n+r 1 Choosing {β j } r j=0 in such a way that tn+r t n+r 1 g(τ) dτ h r β j g(t n+j ) j=0 is an interpolatory quadrature formula a yields the coefficiants of the corresponding (explicit or implicit) Adams method. a i.e., replace g by the unique interpolation polynomial passing through the points {(t n, g(t n )),..., (t n+r 1, g(t n+r 1 ))} of degree r 1 (explicit case) or {(t n, g(t n )),..., (t n+r, g(t n+r ))} of degree r (implicit case) 2.4 Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

42 Numerical Analysis of Differential Equations 85 For r = 1 to 4 the explicit Adams-Bashforth methods are as follows: r = 1 : y n+1 = y n + hf (t n, y n ), r = 2 : y n+2 = y n+1 + h 2 ( f (t n, y n ) + 3f (t n+1, y n+1 )), r = 3 : y n+3 = y n+2 + h 12 (5f (t n, y n ) 16f (t n+1, y n+1 ) +23f (t n+2, y n+2 )), r = 4 : y n+4 = y n+3 + h 24 ( 9f (t n, y n ) + 37f (t n+1, y n+1 ) 59f (t n+2, y n+2 ) + 55f (t n+3, y n+3 )). Forr r = 1 we recover the explicit Euler method. 2.4 Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

43 Numerical Analysis of Differential Equations 86 The implicit Adams-Moulton methods for r = 1 to 4 read: r = 1 : y n+1 = y n + h 2 (f (t n, y n ) + f (t n+1, y n+1 )), r = 2 : y n+2 = y n+1 + h 12 ( f (t n, y n ) + 8f (t n+1, y n+1 ) +5f (t n+2, y n+2 )), r = 3 : y n+3 = y n+2 + h 24 (f (t n, y n ) 5f (t n+1, y n+1 ) +19f (t n+2, y n+2 + 9f (t n+3, y n+3 ))), r = 4 : y n+4 = y n+3 + h ( 19f (tn, y n ) + 106f (t n+1, y n+1 ) f (t n+2, y n+2 ) + 646f (t n+3, y n+3 ) + 251f (t n+4, y n+4 ) ). For r = 1 we recover the trapezoidal rule. 2.4 Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

44 Numerical Analysis of Differential Equations Nyström Methods Nyström methods have the form y n+r y n+r 2 = h r β j f (t n+j, y n+j ). (2.3) j=0 The explicit Nyström method of order r = 2 is the midpoint rule (2.1). The implicit Nyström method of order r = 2 is Simpson s rule y n+2 y n = h 3 ( ) f (t n y n ) + 4f (t n+1, y n+1 ) + f (t n+2, y n+2 ). 2.4 Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

45 Numerical Analysis of Differential Equations BDF Methods In contrast with Runge-Kutta and Adams methods, which were derived using quadrature, the BDF methods (backward differentiation formulas) are based on numerical differentiation. Construction of linear r-step method: (1) Assume approximations y n, y n 1,..., y n r+1 available. (2) To compute new approximation y n+1, denote by p P r the unique interpolating polynomial through the r + 1 points {(t j, y j )} n+1 j=n r+1. (3) The missing interpolation condition p(t n+1 ) = y n+1 (we have not yet computed y n+1 ) is replaced by the requirement p (t n+1 ) = f(t n+1, y n+1 ), (2.4) i.e., that p satisfy the differential equation at t n Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

46 Numerical Analysis of Differential Equations 89 The Newton form of the resulting interpolation polynomial is given by p(t) = y n+1 + y n+1,n (t t n+1 ) + y n+1,n,n 1 (t t n+1 )(t t n ) + + y n+1,n,...,n r+2,n r+1 (t t n+1 ) (t t n r+2 ) in terms of the r + 1 divided differences y n+1,..., y n+1,n,...,n+1 r. Under the simplifying assumption of constant stepsize h, its derivative at t = t n+1 is r p (t n+1 ) = y n+1,n,...,n+1 j h j 1 (j 1)!. (2.5) j=1 Introducing the backward differences 0 y n+1 := y n+1 y n+1 := y n+1 y n, j y n+1 := ( j 1 y n+1 ) = j 1 y n+1 j 1 y n, j = 2, 3,..., 2.4 Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

47 Numerical Analysis of Differential Equations 90 p (t n+1 ) may be expressed as p (t n+1 ) = 1 h r j=1 j y n+1 j, which, combined with condition (2.4), yields the BDF method of order r as r j=1 1 j j y n+1 = hf(t n+1, y n+1 ), r N, (2.6) denoted BDFr for short. The index translation n+1 n+r and dividing both sides by the coefficient of y n+1 transforms the BDF formulas (2.6) to our standard notation (LMM) for linear multistep methods. 2.4 Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

48 Numerical Analysis of Differential Equations 91 The BDF methods for r = 1 to 6 are: r = 1 : y n+1 y n = hf (t n+1, y n+1 ), r = 2 : 3 2 y n+2 2y n y n = hf (t n+2, y n+2 ) r = 3 : 11 6 y n+3 3y n y n y n = hf (t n+3, y n+3 ), r = 4 : y n+4 4y n+3 + 3y n y n y n = hf (t n+4, y n+4 ), r = 5 : y n+5 5y n+4 + 5y n y n y n u n = hf (t n+5, y n+5 ), r = 6 : y n+6 6y n y n y n y n y n u n = hf (t n+6, y n+6 ). For r = 1 we recover the implicit Euler method. 2.4 Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

49 Numerical Analysis of Differential Equations The Local Discretization Error of LMM Inserting the exact solution into (LMM), using the fact that y(t) satisfies y (t n ) = f (t n, y(t n )), we obtain after dividing by h T n+r (h) = 1 r r α j y(t n+j ) h β j y (t n+j ). h j=0 j=0 Expanding all evaluations of y and y at t = t n y(t n+j ) = y(t n ) + jhy (t n ) + (jh)2 2 y (t n+j ) = y (t n ) + jhy (t n ) + (jh)2 2 y (t n ) + (jh)3 6 y (t n ) + (jh)3 6 y (t n ) +... and collecting all terms with the same power of h, we obtain... y (t n ) +..., 2.4 Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

50 Numerical Analysis of Differential Equations 93 T n+r (h) = 1 r r α j y(t n ) + (jα j β j ) y (t n ) h j=0 j=0 r ( ) + h j 2 2 α j jβ j y (t n ) j=0 r ( ) + + h q j q+1 (q + 1)! α j jq q! β j y (q+1) (t n ) + O(h q+1 ). j=0 The LMM is consistent, i.e., T n (h) 0 as h 0, if and only if r α j = 0 as well as r jα j = r β j. (2.7) j=0 j=0 j=0 Its order of consistency is p N whenever the first p + 1 terms in brackets vanish. 2.4 Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

51 Numerical Analysis of Differential Equations Characteristic Polynomials The polynomials formed with the coefficients {α j } r j=0 and {β j} r j=0 of a LMM, known as its characteristic polynomials ρ(ζ) := r α j ζ j and σ(ζ) := j=0 r β j ζ j, (2.8) j=0 play an important role in its analysis. For an r-step method we have ρ P r and, for an implicit r-step method, also σ P r. When an r step method is explicit the degree of σ ist strictly less than r. The two consistency conditions (2.7) can be formulated in terms of the two characteristic polynomials as ρ(1) = 0 und ρ (1) = σ(1). (2.9) 2.4 Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

52 Numerical Analysis of Differential Equations 95 Example: For the Adams-Moulton method with r = 2 y n+2 = y n+1 + h ( ) f (t n, y n ) + 8f (t n+1, y n+1 ) + 5f (t n+2, y n+2 ), 12 we have α 0 = 0, α 1 = 1, α 2 = 1, and the characteristic polynomials are ρ(ζ) = ζ 2 ζ, σ(ζ) = 1 12 β 0 = 1 12, β 1 = 8 12, β 2 = 5 12 ( 5ζ 2 + 8ζ 1 ). 2.4 Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

53 Numerical Analysis of Differential Equations 96 Because of α 0 + α 1 + α 2 = 0, 0 α α α 2 (β 0 + β 1 + β 2 ) = 0, 0 α α α 2 (0 β β β 2 ) = 0, 1 6 (0 α α 1 + 8α 2 ) 1 2 (0 β β β 2 ) = 0, 1 24 (0 α α α 2 ) 1 6 (0 β β β 2 ) = this method is consistent of order p = Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

54 Numerical Analysis of Differential Equations Starting Values When an r-step method (r > 1) is started, it is necessary to calculate, besides y 0, the additional solution approximations y 1, y 2,..., y r 1. To maintain the method s order p, it is sufficient to calculate these using any other method of order at least p 1. Explanation: Since only a fixed number of steps (r 1) is performed with this startup method, independently of h, no power of h is lost in the accumulation of step errors as h 0; for this reason the order of accuracy depends only on that of the method used from step r onwards. Example: To generate y 1 to start the midpoint rule (2.1), which is consistent of order p = 2, one can use the explicit Euler method without loss of order. 2.4 Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

55 Numerical Analysis of Differential Equations Predictor-Corrector Methods A popular technique for avoiding the expensive solution of algebraic equation systems required by implicit LMM consists of (a) first computing an initial approximation ŷ n+r of y(t n+r ) using an explicit method with the same step size (predictor) and (b) then inserting ŷ n+r in place of y n+r in an implicit method (corrector). Example: Combination of r = 1 Adams-Bashforth method (explicit Euler) with Adams-Moulton method with same step number (trapezoidal rule): ŷ n+1 = y n + hf (t n, y n ), y n+1 = y n + h 2 (f (t n, y n ) + f (t n+1, ŷ n+1 )), (predictor step), (corrector step). It can be shown: this predictor/corrector combination has consistency order p = Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

56 Numerical Analysis of Differential Equations One-Step vs. Multistep Methods Advantages of one-step methods: One-step methods are self-starting. Changing the stepsize h possible witout extra effort. Integration across discontinuities of the solution possible without loss of order if these are grid points. Advantage of multistep methods: one evaluation of right hand side per time step (important if f expensive to evaluate). 2.4 Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

57 Numerical Analysis of Differential Equations Convergence of One-Step Methods Theorem 2.3 (Relation between local and global discretization error) Under the given assumptions (cf. Theorem 1.1 as well as (V 2 )) there holds ( ) e n ( e 0 + (t n t 0 ) max T j exp(m(t n t 0 )). 1 j n In particular: a one-step method is convergent if, and only if, it is consistent. 2.5 Convergence of One-Step Methods TU Bergakademie Freiberg, SS 2012

58 Numerical Analysis of Differential Equations 101 Theorem 2.4 (Total error of explicit OSM) Let the explicit one-step method (OSM) be realized in floating point arithmetic with machine precision ε as ỹ n+1 = ỹ n + hφ f (ỹ n, t n ; h) + ε n+1, ỹ 0 = y 0 + ε 0. If ε n ε and T n T for all n = 0, 1,..., then y(t n ) ỹ n ( ε 0 + (t n t 0 )(T + ε h )) exp(m(t n t 0 )). Dilemma: For h large, in general (local error) T large; for h small T becomes small, however ε h grows, i.e., total error dominated by rounding error. This fact favors high order methods, which achieve small T for moderate values of the step size h. 2.5 Convergence of One-Step Methods TU Bergakademie Freiberg, SS 2012

59 Numerical Analysis of Differential Equations Zero-Stability of Linear Multistep Methods Convergence of one-step methods: Contribution of n-th step error S n bounded by e M(t end t 0 ) S n, which still goes to zero as h 0. This property of the error is called zero-stability, and it always holds for OSM. This is not the case for LMM. Example: For the LMM y n+2 3y n+1 + 2y n discretization error is = hf (t n, y n ) the local T n+2 (h) = h 2 y (t n ) + O(h 2 ). We apply it to IVP y (t) = 0, y(0) = 0 with exact solution y(t) 0. (2.10) For starting values y 0 = y 1 = 0 it indeed yields the exact solution. 2.6 Zero-Stability of Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

60 Numerical Analysis of Differential Equations 103 However, setting y 1 = h to simulate an error of order h when approximating y(t 1 ), one obtains for the approximation of the solution at t = 1 the following values (h = 1/N): N y N Since y 1 0 as h 0 these initial data are admissible. Nonetheless, y N fails to converge to y(t N ) = y(1) = 0. The approximations satisfy the difference equation with solution y n+2 3y n+1 + 2y n = 0, n = 0, 1, 2,..., (2.11) y n = (2 n 1)y 1 (2 n 2)y 0 = 2y 0 y n (y 1 y 0 ), n Zero-Stability of Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

61 Numerical Analysis of Differential Equations Solution of Linear Difference Equations Solutions of a homogeneous linear difference equation of order r α r y n+r + α r 1 y n+r α 0 y n = r α j y n+j = 0 (2.12) j=0 can be obtained by hypothesizing solutions of the form y n = ζ n ( Ansatz ). Inserting in (2.12) and dividing by ζ n leads to the condition r α j ζ j = 0, j=0 i.e., y n = ζ n solves (2.12) if ζ is a root of the first characteristic polynomial ρ(ζ) (cf. (2.8)). 2.6 Zero-Stability of Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

62 Numerical Analysis of Differential Equations 105 If the characteristic polynomial ρ has the factored form ρ(ζ) = α r (ζ ζ 1 )(ζ ζ 2 ) (ζ ζ r ) with r distinct roots {ζ j } r j=1, then {ζn j }r j=1 constitute r linearly independent solutions of (2.12). Due to the linearity and homogeneity of (2.12), any linear combination y n = c 1 ζ n 1 + c 2 ζ n c r ζ n r, c 1,..., c r arbitrary, (2.13) of these is also a solution. Coefficients {c j } r j=1 determined by fixing first r values of y n: c 1 + c c r = y 0, c 1 ζ 1 + c 2 ζ c r ζ r = y 1, c 1 ζ r c 2 ζ r c r ζ r 1 r = y r Zero-Stability of Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

63 Numerical Analysis of Differential Equations 106 Example: The characteristic equation of the difference equation (2.11) is ρ(ζ) = ζ 2 3ζ + 2 = (ζ 1)(ζ 2). Conclusion: If ρ(ζ) has roots of modulus > 1, the associated LMM cannot be convergent. If two (or more) roots coincide, i.e., ζ i = ζ j, then ζi n and ζj n are linearly dependent. A further, linearly independent solution is y n = nζ n 1 i. For a root ζ i of multiplicity three, y n = n 2 ζ n 2 i Example: The (consistent) LMM applied to y (t) = 0. is linearly independent etc. y n+2 2y n+1 + y n = h 2 [ f (tn+2, y n+2 ) f (t n, y n ) ]. Conclusion: If ρ(ζ) has multiple roots of modulus 1, the associated LMM is also not convergent. 2.6 Zero-Stability of Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

64 Numerical Analysis of Differential Equations 107 Example: The consistent LMM applied to (2.10). y n+3 2y n y n y n = h 4 f(t n, y n ) Conclusion: For multiple roots of modulus > 1, the associated LMM is convergent (linear growth is outweighed by exponential decay). A LMM is called zero-stable, if the roots {ζ j } r j=1 polynomial ρ(ζ) satisfy (a) ζ j 1 for all roots. (b) ζ j < 1 for all multiple roots. of its first characteristic Examples: All Adams-methods are zero-stable. The BDF-methods are zero-stable for 1 r Zero-Stability of Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

65 Numerical Analysis of Differential Equations 108 These examples indicate that zero-stability is a necessary condition for convergence of LMMs. In fact, this condition is also sufficient. Theorem 2.5 (Dahlquist, 1956) For LMM applied to (IVP), consistency + zero-stability convergence. Remarks 2.6 (a) The first characteristic polynomial of a consistent LMM always possesses the root ζ = 1 (cf. (2.9)). (b) We will see that, besides zero-stability, there are further important stability properties for numerical methods for IVPs. (c) All single-step methods are zero-stable. 2.6 Zero-Stability of Linear Multistep Methods TU Bergakademie Freiberg, SS 2012

66 Numerical Analysis of Differential Equations Stepsize Control In practice, numerical methods for solving IVPs for ODEs almost never employ a constant stepsize h. Rather, it is much more efficient to adapt h to the local behavior of the solution y in the sense that rapid change in y necessitates small h slow variation of y allows large h. Here we will introduce one (of many) stepsize control schemes aimed at maintaining the (easily estimated) local discretization error T n near a prescribed tolerance tol T n tol, n = 1, 2,.... For systems of ODEs (particularly when solution components vary strongly in magnitude) absolute error tolerances for each solution component as well as a global error tolerance are typically supplied. 2.7 Stepsize Control TU Bergakademie Freiberg, SS 2012

67 Numerical Analysis of Differential Equations 110 Theorem 2.3 states that bounding the local discretization error also results in bounding the global discretization error (the actual quantity of interest). The local discretization error may be estimated by using two methods of different consistency orders, say, p and q where p < q, for computing y n from y n 1 : y n = y n 1 + hφ f (y n 1, t n 1 ; h) and ŷ n = y n 1 + h Φ f (y n 1, t n 1 ; h) with associated local discretization errors This implies T n = y(t n) y(t n 1 ) h T n = y(t n) y(t n 1 ) h Φ f (y(t n 1 ), t n 1 ; h) = O(h p ), Φ f (y(t n 1 ), t n 1 ; h) = O(h q ). T n T n = Φ f (y n 1, t n 1 ; h) Φ f (y n 1, t n 1 ; h) = 1 h (ŷ n y n ). 2.7 Stepsize Control TU Bergakademie Freiberg, SS 2012

68 Numerical Analysis of Differential Equations 111 Because T n T n = T n (1 + O(h q p )) T n, the quantity provides a (rough) estimate for T n. 1 h y n ŷ n T n Whenever 1 h y n ŷ n > tol, we discard the stepsize h, replacing it with h determined by ) p ( h = α h tol h y n ŷ n. ( ) This choice of h is motivated as follows: discarded stepsize h: 1 h y n ŷ n T n = ch p + O(h p+1 ) ch p, desired stepsize h: tol = T n = c h p + O( h p+1 ) c h p. (α is added as a safety factor, e.g. α = 0.9.) 2.7 Stepsize Control TU Bergakademie Freiberg, SS 2012

69 Numerical Analysis of Differential Equations 112 Starting again from y n 1, approximations y n and ŷ n now of y(t n 1 + h) 1 are then generated and this process repeated until h y n ŷ n tol holds. When this is achieved, ( ) is used to propose a (trial) stepsize for the next step (n n + 1). An elegant way to limit the computing effort is to compute y n and ŷ n using two RKM (of different orders) whose Butcher tableaus differ only in the vector b (same A and c). As a result, the slopes k j of their stages are also the same. Such pairs of RKMs are called embedded RKMs and denoted c A b b e.g /2 1/2. Here the explicit Euler method (p = 1) is embedded in the improved Euler method (p = 2). 2.7 Stepsize Control TU Bergakademie Freiberg, SS 2012

70 Numerical Analysis of Differential Equations 113 Another popular embedded pair is the Fehlberg 4(5)-formula: consisting of two six-stage RKMs of orders 4 and Stepsize Control TU Bergakademie Freiberg, SS 2012

71 Numerical Analysis of Differential Equations 114 A further example is the Dormand-Prince 4(5)-formula: Here a six-stage RKM of order 4 is embedded in a seven-stage method of order Stepsize Control TU Bergakademie Freiberg, SS 2012