Initial value problems for ordinary differential equations

Transcription

1 Initial value problems for ordinary differential equations Xiaojing Ye, Math & Stat, Georgia State University Spring 2019 Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 1

2 IVP of ODE We study numerical solution for initial value problem (IVP) of ordinary differential equations (ODE). A basic IVP: dy dt = f (t, y), for a t b Remark with initial value y(a) = α. f is given and called the defining function of IVP. α is given and called the initial value. y(t) is called the solution of the IVP if y(a) = α; y (t) = f (t, y(t)) for all t [a, b]. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 2

3 IVP of ODE Example The following is a basic IVP: y = y t 2 + 1, t [0, 2], and y(0) = 0.5 The defining function is f (t, y) = y t Initial value is y(0) = 0.5. The solution is y(t) = (t + 1) 2 et 2 because: y(0) = (0 + 1) 2 e0 2 = = 1 2 ; We can check that y (t) = f (t, y(t)): y (t) = 2(t + 1) et 2 f (t, y(t)) = y(t) t = (t + 1) 2 et 2 t2 + 1 = 2(t + 1) et 2 Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 3

4 IVP of ODE (cont.) More general or complex cases: IVP of ODE system: dy 1 = f 1 (t, y 1, y 2,..., y n ) dt dy 2 = f 2 (t, y 1, y 2,..., y n ) dt. for a t b dy n dt = f n (t, y 1, y 2,..., y n ) with initial value y 1 (a) = α 1,..., y n (a) = α n. High-order ODE: y (n) = f (t, y, y,..., y (n 1) ) for a t b with initial value y(a) = α 1, y (a) = α 2,..., y (n 1) (a) = α n. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 4

5 Why numerical solutions for IVP? ODEs have extensive applications in real-world: science, engineering, economics, finance, public health, etc. Analytic solution? Not with almost all ODEs. Fast improvement of computers. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 5

6 Some basics about IVP Definition (Lipschitz functions) A function f (t, y) defined on D = {(t, y) : t R +, y R} is called Lipschitz with respect to y if there exists a constant L > 0 f (t, y 1 ) f (t, y 2 ) L y 1 y 2 for all t R +, and y 1, y 2 R. Remark We also call f is Lipschitz with respect to y with constant L, or simply f is L-Lipschitz with respect to y. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 6

7 Some basics about IVP Example Function f (t, y) = t y is Lipschitz with respect to y on the set D := {(t, y) t [1, 2], y [ 3, 4]}. Solution: For any t [1, 2] and y 1, y 2 [ 3, 4], we have f (t, y 1 ) f (t, y 2 ) = t y 1 t y 2 t y 1 y 2 2 y 1 y 2. So f (t, y) = t y is Lipschitz with respect to y with constant L = 2. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 7

8 A set D R 2 is said to be convex if whenever (t 1, y 1 ) and (t 2, y 2 ) belong to D, then ((1 λ)t 1 + λt 2, (1 λ)y 1 + λy 2 ) also belongs to D for every λ in [0, 1]. Some basics about IVP In geometric terms, Definition 5.2 states that a set is convex provided that whenever two points belong to the set, the entire straight-line segment between the points also belongs to the set. (See Figure 5.1.) The sets we consider in this chapter are generally of the form Definition (Convex sets) A set D R 2 is convex if whenever (t 1, y 1 ), (t 2, y 2 ) D there is (1 λ)(t 1, y 1 ) + λ(t 2, y 2 ) D for all λ [0, 1]. D ={(t, y) a t b and < y < } for some constants a and b. It is easy to verify (see Exercise 7) that these sets are convex. (t 2, y 2 ) (t 1, y 1 ) (t 2, y 2 ) (t 1, y 1 ) Convex Not convex Suppose f(t, y) is defined on a convex set D R 2. If a constant L > 0 exists with Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 8

9 Some basics about IVP Theorem If D R 2 is convex, and f y (t, y) L for all (t, y) D, then f is Lipschitz with respect to y with constant L. Remark This is a sufficient (but not necessary) condition for f to be Lipschitz with respect to y. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 9

10 Some basics about IVP Proof. For any (t, y 1 ), (t, y 2 ) D, define function g by g(λ) = f (t, (1 λ)y 1 + λy 2 ) for λ [0, 1] (need convexity of D!). Then we have g (λ) = y f (t, (1 λ)y 1 + λy 2 ) (y 2 y 1 ) So g (λ) L y 2 y 1. Then we have g(1) g(0) = 1 0 g (λ) dλ L y 2 y dλ = L y 2 y 1 Note that g(0) = f (t, y 1 ) and g(1) = f (t, y 2 ). This completes the proof. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 10

11 Some basics about IVP Theorem Suppose D = [a, b] R, a function f is continuous on D and Lipschitz with respect to y, then the initial value problem y = f (t, y) for t [a, b] with initial value y(a) = α has a unique solution y(t) for t [a, b]. Remark This theorem says that there must be one and only one solution of the IVP, provided that the defining f of the IVP is continuous and Lipschitz with respect to y on D. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 11

12 Some basics about IVP Example Show that y = 1 + t sin(ty) for t [0, 2] with y(0) = 0 has a unique solution. Solution: First, we know f (t, y) = 1 + t sin(ty) is continuous on [0, 2] R. Second, we can see f t y = 2 cos(ty) t 2 4 So f (t, y) is Lipschitz with respect to y (with constant 4). From theorem above, we know the IVP has a unique solution y(t) on [0, 2]. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 12

13 Some basics about IVP Theorem (Well-posedness) An IVP y = f (t, y) for t [a, b] with y(a) = α is called well-posed if It has a unique solution y(t); There exist ɛ 0 > 0 and k > 0, such that ɛ (0, ɛ 0 ) and function δ(t), which is continuous and satisfies δ(t) < ɛ for all t [a, b], the perturbed problem z = f (t, z) + δ(t) with initial value z(a) = α + δ 0 (where δ 0 ɛ) satisfies z(t) y(t) < kɛ, t [a, b]. Remark This theorem says that a small perturbation on defining function f by δ(t) and initial value y(a) by δ 0 will only cause small change to original solution y(t). Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 13

14 Some basics about IVP Theorem Let D = [a, b] R. If f is continuous on D and Lipschitz with respect to y, then the IVP is well-posed. Remark Again, a sufficient but not necessary condition for well-posedness of IVP. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 14

15 Euler s method Given an IVP y = f (t, y) for t [a, b] and y(a) = α, we want to compute y(t) on mesh points {t 0, t 1,..., t N } on [a, b]. 5 Initial-Value Problems for Ordinary Differential Equations re 5.2 To this end, we partition [a, b] into N equal segments: set h = b a N, and define t i = a + ih for i = 0, 1,..., N. Here h is method appears in Figure 5.3, and a series of steps appears in Figure 5.4. called the step size. The graph of the function highlighting y(t i ) is shown in Figure 5.2. One step in Euler s y y(t N ) y(b) y f (t, y), y(a) α... y(t 2 ) y(t 1 ) y(t 0 ) α t 0 a t 1 t 2... t N b t Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 15

16 Euler s method From Taylor s theorem, we have y(t i+1 ) = y(t i ) + y (t i )(t i+1 t i ) y (ξ i )(t i+1 t i ) 2 for some ξ i (t i, t i+1 ). Note that t i+1 t i = h and y (t i ) = f (t i, y(t i )), we get y(t i+1 ) y(t i ) + hf (t, y(t i )) Denote w i = y(t i ) for all i = 0, 1,..., N, we get the Euler s method: { w0 = α w i+1 = w i + hf (t i, w i ), i = 0, 1,..., N 1 Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 16

17 Euler s method y(t 1 ) y(t 0 ) α y(t 1 ) y(t 0 ) α t 0 a t 1 t 2.. t. 0 t N a bt 1 t 2... t N b t igure 5.3 y y f (t, y), y(a) α y f (t, y), y(a) α Figure 5.4 y y(b) w N Figure 5.4 y y f (t, y), y(a) y(b) α w N y f (t, y), y(a) α Slope y (a) f (a, α) Slope y (a) f (a, α) w 1 α a t 1 t 2.. t 0. t N a tb 1 t 2 t... t N b t w 2 w 1 α w 2 w 1 α t 0 a t 1 t 2... t 0 t N a bt 1 t 2 t... t N b Euler s Example method 1 was Euler s used in method the first was illustration used in the with first h illustration = 0.5 to approximate with h = 0.5 the to solution approximate th to the initial-valuetoproblem the initial-value problem y = y t 2 + 1, y 0 = ty 2, t 2 + y(0) 1, = t 2, y(0) = 0.5. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 17

18 Euler s method Example Use Euler s method with h = 0.5 for IVP y = y t for t [0, 2] with initial value y(0) = 0.5. Solution: We follow Euler s method step-by-step: t 0 = 0 : w 0 = y(0) = 0.5 t 1 = 0.5 : w 1 = w 0 + hf (t 0, w 0 ) = ( ) = 1.25 t 2 = 1.0 : w 2 = w 1 + hf (t 1, w 1 ) = ( ) = 2.25 t 3 = 1.5 : w 3 = w 2 + hf (t 2, w 2 ) = ( ) = t 4 = 2.0 : w 4 = w 3 + hf (t 3, w 3 ) = ( ) = Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 18

19 Error bound of Euler s method Theorem Suppose f (t, y) in an IVP is continuous on D = [a, b] R and Lipschitz with respect to y with constant L. If M > 0 such that y (t) M (y(t) is the unique solution of the IVP), then for all i = 0, 1,..., N there is y(t i ) w i hm ( ) e L(t i a) 1 2L Remark Numerical error depends on h (also called O(h) error). Also depends on M, L of f. Error increases for larger t i. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 19

20 Error bound of Euler s method Proof. Taking the difference of we get y(t i+1 ) = y(t i ) + hf (t i, y i ) y (ξ i )(t i+1 t i ) 2 w i+1 = w i + hf (t i, w i ) y(t i+1 ) w i+1 y(t i ) w i + h f (t i, y i ) f (t i, w i ) + Mh2 2 y(t i ) w i + hl y i w i + Mh2 2 = (1 + hl) y i w i + Mh2 2 Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 20

21 Error bound of Euler s method Proof (cont). Denote d i = y(t i ) w i, then we have d i+1 (1 + hl)d i + Mh2 2 = (1 + hl) ( for all i = 0, 1,..., N 1. So we obtain d i + hm 2L d i+1 + hm ( 2L (1 + hl) d i + hm ) ( 2L (1 + hl) 2 d i 1 + hm 2L (1 + hl) i+1 ( d 0 + hm 2L and hence d i (1 + hl) i hm 2L hm 2L (since d 0 = 0). ) ) ) hm 2L Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 21

22 Error bound of Euler s method Proof (cont). Note that 1 + x e x for all x > 1, and hence (1 + x) a e ax if a > 0. Based on this, we know (1 + hl) i e ihl = e L(t i a) since ih = t i a. Therefore we get d i e L(t i a) hm 2L hm 2L = hm 2L (el(t i a) 1) This completes the proof. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 22

23 Error bound of Euler s method Example Estimate the error of Euler s method with h = 0.2 for IVP y = y t for t [0, 2] with initial value y(0) = 0.5. Solution: We first note that f y = 1, so f is Lipschitz with respect to y with constant L = 1. The IVP has solution y(t) = (t 1) 2 et 2 so y (t) = et e =: M. By theorem above, the error of Euler s method is y(ti ) w i hm ( ) e L(t i a) 1 = 0.2(0.5e2 2) ( ) e t i 1 2L 2 Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 23

24 Error bound of Euler s method Example Estimate the error of Euler s method with h = 0.2 for IVP y = y t for t [0, 2] with initial value y(0) = 0.5. Solution: (cont) 5.2 Euler s Me Table 5.1 t i w i y i = y(t i ) y i w i Numerical Analysis II XiaojingNote Ye, Math that & Stat, thegeorgia error State grows University slightly as the value of t increases. 24 This co

25 Round-off error of Euler s method Due to round-off errors in computer, we instead obtain { u0 = α + δ 0 u i+1 = u i + hf (t i, u i ) + δ i, i = 0, 1,..., N 1 Suppose δ > 0 such that δ i δ for all i, then we can show y(t i ) u i 1 ( hm L 2 + δ ) ( ) e L(t i a) 1 + δe L(t i a). h Note that hm 2 + δ h does not approach 0 as h 0. hm 2 + δ h 2δ reaches minimum at h = M (often much smaller than what we choose in practice). Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 25

26 Higher-order Taylor s method Definition (Local truncation error) We call the difference method { w0 = α + δ 0 w i+1 = w i + hφ(t i, w i ), i = 0, 1,..., N 1 to have local truncation error where y i := y(t i ). τ i+1 (h) = y i+1 (y i + hφ(t i, y i )) h Example Euler s method has local truncation error τ i+1 (h) = y i+1 (y i + hf (t i, y i )) h = y i+1 y i h f (t i, y i ) Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 26

27 Higher-order Taylor s method Note that Euler s method has local truncation error τ i+1 (h) = y i+1 y i h f (t i, y i ) = hy (ξ i ) 2 for some ξ i (t i, t i+1 ). If y M we know τ i+1 (h) hm 2 = O(h). Question: What if we use higher-order Taylor s approximation? y(t i+1 ) = y(t i ) + hy (t i ) + h2 2 y (t i ) + + hn n! y (n) (t i ) + R where R = hn+1 (n+1)! y (n+1) (ξ i ) for some ξ i (t i, t i+1 ). Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 27

28 Higher-order Taylor s method First note that we can always write y (n) using f : y (t) = f y (t) = f = t f + ( y f )f y (t) = f = t 2 f + ( t y f + ( y 2 f )f )f + y f ( t f + ( y f )f ) y (n) (t) = f (n 1) = albeit it s quickly getting very complicated. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 28

29 Higher-order Taylor s method Now substitute them back to high-order Taylor s approximation (ignore residual R) y(t i+1 ) = y(t i ) + hy (t i ) + h2 2 y (t i ) + + hn n! y (n) (t i ) = y(t i ) + hf + h2 2 f + + hn n! f (n 1) We can get the n-th order Taylor s method: { w0 = α + δ 0 where w i+1 = w i + ht (n) (t i, w i ), i = 0, 1,..., N 1 T (n) (t i, w i ) = f (t i, w i ) + h 2 f (t i, w i ) + + hn 1 f (n 1) (t i, w i ) n! Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 29

30 Higher-order Taylor s method Euler s method is the first order Taylor s method. High-order Taylor s method is more accurate than Euler s method, but at much higher computational cost. Together with Hermite interpolating polynomials, it can be used to interpolate values not on mesh points more accurately. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 30

31 Higher-order Taylor s method Theorem If y(t) C n+1 [a, b], then the n-th order Taylor method has local truncation error O(h n ). Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 31

32 Runge-Kutta (RK) method Runge-Kutta (RK) method attains high-order local truncation error without expensive evaluations of derivatives of f. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 32

33 Runge-Kutta (RK) method To derive RK method, first recall Taylor s formula for two variables (t, y): where t n k y k f = n f (t 0,y 0 ) and t n k y k f (t, y) = P n (t, y) + R n (t, y) P n (t, y) = f (t 0, y 0 ) + ( t f (t t 0 ) + y f (y y 0 )) + 1 ( t 2 f (t t 0 ) y t f (t t 0 )(y y 0 ) + y 2 f (y y 0 ) 2) n ( ) n n k t y k f (t t 0 ) n k (y y 0 ) k n! k R n (t, y) = 1 (n + 1)! k=0 n+1 ( ) n + 1 k k=0 n+1 k t y k f (ξ, µ) (t t 0 ) n+1 k (y y 0 ) k Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 33

34 Runge-Kutta (RK) method The second order Taylor s method uses T (2) (t, y) = f (t, y) + h 2 f (t, y) = f (t, y) + h 2 ( tf + y f f ) to get O(h 2 ) error. Suppose we use af (t + α, y + β) (with some a, α, β to be determined) to reach the same order of error. To that end, we first have ( ) af (t + α, y + β) = a f + t f α + y f β + R where R = 1 2 ( 2 t f (ξ, µ) α2 + 2 y t f (ξ, µ) αβ + 2 y f (ξ, µ) β 2 ). Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 34

35 Runge-Kutta (RK) method Suppose we try to match the terms of these two formulas (ignore R): then we have T (2) (t, y) = f + h 2 tf + hf 2 yf af (t + α, y + β) = af + aα t f + aβ y f So instead of T (2) (t, y), we use a = 1, α = h 2, β = h f (t, y) 2 af (t + α, y + β) = f (t + h 2, y + h ) 2 f (t, y) Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 35

36 Runge-Kutta (RK) method Note that R we ignored is R = 1 2 ( ( h )2 ( h )2 ( h )2 ) t 2 f (ξ, µ) + 2 y t f (ξ, µ) f + y 2 f (ξ, µ) f which means R = O(h 2 ). Also note that R = T (2) (t, y) f (t + h 2, y + h ) 2 f (t, y) = O(h 2 ) and T (2) (t, y) = O(h 2 ), we know f (t + h 2, y + h ) 2 f (t, y) = O(h 2 ) Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 36

37 Runge-Kutta (RK) method This is the RK2 method (Midpoint method): w 0 = α ( w i+1 = w i + h f t i + h 2, w i + h ) 2 f (t i, w i ), i = 0, 1,..., N 1. Remark If we have (t i, w i ), we only need to evaluate f twice (i.e., compute k 1 = f (t i, w i ) and k 2 = f (t i + h 2, w i + h 2 k 1)) to get w i+1. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 37

38 Runge-Kutta (RK) method We can also consider higher-order RK method by fitting T (3) (t, y) = f (t, y) + h 2 f (t, y) + h 6 f (t, y) with af (t, y) + bf (t + α, y + β) (has 4 parameters a, b, α, β). hf Unfortunately we can make match to the 6 term of T (3), which contains h2 6 f ( yf ) 2, by this way But it leaves us open choices if we re OK with O(h 2 ) error: let a = b = 1, α = h, β = hf (t, y), then we get the modified Euler s method: w 0 = α w i+1 = w i + h 2 ( ) f (t i, w i ) + f (t i+1, w i + hf (t i, w i )), i = 0, 1,..., N 1. Also need evaluation of f twice in each step. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 38

39 Runge-Kutta (RK) method Example Use Midpoint method (RK2) and Modified Euler s method with h = 0.2 to solve IVP y = y t for t [0, 2] and y(0) = 0.5. Solution: Apply the main steps in the two methods: ( Midpoint : w i+1 =w i + h f t i + h 2, w i + h ) 2 f (t i, w i ) Modified Euler s : w i+1 =w i + h ( ) f (t i, w i ) + f (t i+1, w i + hf (t i, w i )) 2 Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 39

40 Runge-Kutta (RK) method Example = , Use Midpoint method (RK2) and Modified Euler s method with Table 5.6 lists all the results h = 0.2 to solve IVP y of the calculations. = y t 2 For this problem, the Midpoint method + 1 for t [0, 2] and y(0) = 0.5. Solution: (cont) Midpoint method: w 2 = 1.22(0.828) (0.2) (0.2) = ; Modified Euler method: w 2 = 1.22(0.826) (0.2) (0.2) is superior to the Modified Euler method. Table 5.6 Midpoint Modified Euler t i y(t i ) Method Error Method Error Midpoint (RK2) method is better than modified Euler s method. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 40

41 Runge-Kutta (RK) method We can also consider higher-order RK method by fitting T (3) (t, y) = f (t, y) + h 2 f (t, y) + h 6 f (t, y) with af (t, y) + bf (t + α 1, y + δ 1 (f (t + α 2, y + δ 2 f (t, y)) ) (has 6 parameters a, b, α 1, α 2, δ 1, δ 2 ) to reach O(h 3 ) error. For example, Heun s choice is a = 1 4, b = 3 4, α 1 = 2h 3, α 2 = h 3, δ 1 = 2h 3 f, δ 2 = h 3 f. Nevertheless, methods of order O(h 3 ) are rarely used in practice. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 41

42 4-th Order Runge-Kutta (RK4) method Most commonly used is the 4-th order Runge-Kutta method (RK4): start with w 0 = α, and iteratively do k 1 = f (t i, w i ) k 2 = f (t i + h 2, w i + h 2 k 1) k 3 = f (t i + h 2, w i + h 2 k 2) k 4 = f (t i+1, w i + hk 3 ) w i+1 = w i + h 6 (k 1 + 2k 2 + 2k 3 + k 4 ) Need to evaluate f for 4 times in each step. Reach error O(h 4 ). Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 42

43 4-th Order Runge-Kutta (RK4) method Example Use RK4 (with h = 0.2) to solve IVP y = y t for t [0, 2] and y(0) = 0.5. Solution: With h = 0.2, we have N = 10 and t i = 0.2i for i = 0, 1,..., 10. First set w 0 = 0.5, then the first iteration is k 1 = f (t 0, w 0 ) = f (0, 0.5) = = 1.5 k 2 = f (t 0 + h 2, w 0 + h 2 k 1) = f (0.1, ) = 1.64 k 3 = f (t 0 + h 2, w 0 + h 2 k 2) = f (0.1, ) = k 4 = f (t 1, w 0 + hk 3 ) = f (0.2, ) = w 1 = w 0 + h 6 (k 1 + 2k 2 + 2k 3 + k 4 ) = So w 1 is our RK4 approximation of y(t 1 ) = y(0.2). Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 43

44 k 3 = 0.2f(0.1, 0.664) = th Order Runge-Kutta k 4 = 0.2f(0.2, (RK4) ) method = Example Use RK4 (with The remaining h = 0.2) results to solve and IVP their errors y = are y listed t 2 + in1 Table for t5.8. [0, 2] and y(0) = 0.5. w 1 = ( (0.328) + 2(0.3308) ) = Solution: (cont) Continue with i = 1, 2,, 9: Table 5.8 Runge-Kutta Exact Order Four Error t i y i = y(t i ) w i y i w i Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 44

45 High-order Runge-Kutta method Can we use even higher-order method to improve accuracy? #f eval n 7 8 n 9 n 10 Best error O(h 2 ) O(h 3 ) O(h 4 ) O(h n 1 ) O(h n 2 ) O(h n 3 ) So RK4 is the sweet spot. Remark Note that RK4 requires 4 evaluations of f each step. So it would make sense only if it s accuracy with step size 4h is higher than Midpoint with 2h or Euler s with h! Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 45

46 High-order Runge-Kutta method Example Use RK4 (with h = 0.1), Midpoint (with h = 0.05), and Euler s method (with h = 0.025) to solve IVP y = y t for t [0, 0.5] and y(0) = 0.5. Solution: Table 5.10 Modified Runge-Kutta Euler Euler Order Four t i Exact h = h = 0.05 h = Runge-Kutta Methods RK4 is better with same computation cost! Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 46

47 Error control Can we control the error of Runge-Kutta method by using variable step sizes? Let s compare two difference methods with errors O(h n ) and O(h n+1 ) (say, RK4 and RK5) for fixed step size h, which have schemes below: w i+1 = w i + hφ(t i, w i, h) O(h n ) w i+1 = w i + h φ(t i, w i, h) O(h n+1 ) Suppose w i w i y(t i ) =: y i. Then for any given ɛ > 0, we want to see how small h should be for the O(h n ) method so that its error τ i+1 (h) ɛ? Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 47

48 Error control We recall that the local truncation errors of these two methods are: τ i+1 (h) = y i+1 y i h τ i+1 (h) = y i+1 y i h φ(t i, y i, h) O(h n ) φ(t i, y i, h) O(h n+1 ) Given that w i w i y i and O(h n+1 ) O(h n ) for small h, we see τ i+1 (h) τ i+1 (h) τ i+1 (h) = φ(t i, y i, h) φ(t i, y i, h) φ(t i, w i, h) φ(t i, w i, h) = w i+1 w i h w i+1 w i+1 h Kh n w i+1 w i h for some K > 0 independent of h, since τ i+1 (h) O(h n ). Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 48

49 Error control Suppose that we can scale h by q > 0, such that τ i+1 (qh) K (qh) n = q n Kh n q n w i+1 w i+1 h ɛ So we need q to satisfy ( ɛh q w i+1 w i+1 )1/n q < 1: reject the initial h and recalculate using qh. q 1: accept computed value and use qh for next step. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 49

50 Runge-Kutta-Fehlberg method The Runge-Kutta-Fehlberg (RKF) method uses specific 4th-order and 5th-order RK schemes, which share some computed values and together only need 6 evaluation of f, to estimate ( ɛh q = 2 w i+1 w i+1 )1/4 ( ɛh = 0.84 w i+1 w i+1 )1/4 This q is used to tune step size so that error is always bounded by the prescribed ɛ. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 50

51 Multistep method Definition Let m > 1 be an integer, then an m-step multistep method is given by the form of w i+1 = a m 1 w i + a m 2 w i a 0 w i m+1 + h [ b m f (t i+1, w i+1 ) + b m 1 f (t i, w i ) + + b 0 f (t i m+1, w i m+1 ) ] for i = m 1, m,..., N 1. Here a 0,..., a m 1, b 0,..., b m are constants. Also w 0 = α, w 1 = α 1,..., w m 1 = α m 1 need to be given. b m = 0: Explicit m-step method. b m 0: Implicit m-step method. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 51

52 Multistep method Definition The local truncation error of the m-step multistep method above is defined by τ i+1 (h) = y i+1 (a m 1 y i + + a 0 y i m+1 ) h [ b m f (t i+1, y i+1 ) + b m 1 f (t i, y i ) + + b 0 f (t i m+1, y i m+1 ) ] where y i := y(t i ). Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 52

53 Adams-Bashforth Explicit method Adams-Bashforth Two-Step Explicit method: w 0 = α, w 1 = α 1, w i+1 = w i + h [ ] 3f (t i, w i ) f (t i 1, w i 1 ) 2 for i = 1,..., N 1. The local truncation error is for some µ i (t i 1, t i+1 ). τ i+1 (h) = 5 12 y (µ i )h 2 Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 53

54 Adams-Bashforth Explicit method Adams-Bashforth Three-Step Explicit method: w 0 = α, w 1 = α 1, w 2 = α 2, w i+1 = w i + h [ ] 23f (t i, w i ) 16f (t i 1, w i 1 ) + 5f (t i 2, w i 2 ) 12 for i = 2,..., N 1. The local truncation error is for some µ i (t i 2, t i+1 ). τ i+1 (h) = 3 8 y (4) (µ i )h 3 Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 54

55 Adams-Bashforth Explicit method Adams-Bashforth Four-Step Explicit method: w 0 = α, w 1 = α 1, w 2 = α 2, w 3 = α 3 w i+1 = w i + h 24 for i = 3,..., N 1. The local truncation error is for some µ i (t i 3, t i+1 ). [ 55f (t i, w i ) 59f (t i 1, w i 1 ) + 37f (t i 2, w i 2 ) 9f (t i 3, w i 3 ) τ i+1 (h) = y (5) (µ i )h 4 ] Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 55

56 Adams-Bashforth Explicit method Adams-Bashforth Five-Step Explicit method: w 0 = α, w 1 = α 1, w 2 = α 2, w 3 = α 3, w 4 = α 4 w i+1 = w i + h 720 [1901f (t i, w i ) 2774f (t i 1, w i 1 ) f (t i 2, w i 2 ) 1274f (t i 3, w i 3 ) + 251f (t i 4, w i 4 )] for i = 4,..., N 1. The local truncation error is for some µ i (t i 4, t i+1 ). τ i+1 (h) = y (6) (µ i )h 5 Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 56

57 Adams-Moulton Implicit method Adams-Moulton Two-Step Implicit method: w 0 = α, w 1 = α 1, w i+1 = w i + h 12 [5f (t i+1, w i+1 ) + 8f (t i, w i ) f (t i 1, w i 1 )] for i = 1,..., N 1. The local truncation error is for some µ i (t i 1, t i+1 ). τ i+1 (h) = 1 24 y (4) (µ i )h 3 Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 57

58 Adams-Moulton Implicit method Adams-Moulton Three-Step Implicit method: w 0 = α, w 1 = α 1, w 2 = α 2 w i+1 = w i + h 24 [9f (t i+1, w i+1 ) + 19f (t i, w i ) 5f (t i 1, w i 1 ) + f (t i 2, w i 2 )] for i = 2,..., N 1. The local truncation error is for some µ i (t i 2, t i+1 ). τ i+1 (h) = y (5) (µ i )h 4 Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 58

59 Adams-Moulton Implicit method Adams-Moulton Four-Step Implicit method: w 0 = α, w 1 = α 1, w 2 = α 2, w 3 = α 3 w i+1 = w i + h 720 [251f (t i+1, w i+1 ) + 646f (t i, w i ) 264f (t i 1, w i 1 ) + 106f (t i 2, w i 2 ) 19f (t i 3, w i 3 )] for i = 3,..., N 1. The local truncation error is for some µ i (t i 3, t i+1 ). τ i+1 (h) = y (6) (µ i )h 5 Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 59

60 Steps to develop multistep methods Construct interpolating polynomial P(t) (e.g., Newton s backward difference method) using previously computed (t i m+1, w i m+1 ),..., (t i, w i ). Approximate y(t i+1 ) based on ti+1 y(t i+1 ) = y(t i ) + y (t) dt = y(t i ) + t i y(t i ) + ti+1 t i f (t, P(t)) dt and construct difference method: ti+1 t i f (t, y(t)) dt w i+1 = w i + hφ(t i,..., t i m+1, w i,..., w i m+1 ) Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 60

61 Explicit vs. Implicit Implicit methods are generally more accurate than the explicit ones (e.g., Adams-Moulton three-step implicit method is even more accurate than Adams-Bashforth four-step explicit method). Implicit methods require solving for w i+1 from w i+1 = + h xxx f (t i+1, w i+1 ) + which can be difficult or even impossible. There could be multiple solutions of w i+1 when solving the equation above in implicit methods. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 61

62 Predictor-Corrector method Due to the aforementioned issues, implicit methods are often cast in predictor-corrector form in practice. In each step i: Prediction: Compute w i+1 using an explicit method φ to get w i+1,p using w i+1,p = w i + hφ(t i, w i,..., t i m+1, w i m+1 ) Correction: Substitute w i+1 by w i+1,p in the implicit method φ and compute w i+1 using w i+1 = w i + h φ(t i+1, w i+1,p, t i, w i,..., t i m+1, w i m+1 ) Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 62

63 Predictor-Corrector method Example Use the Adams-Bashforth 4-step explicit method and Adams-Moulton 3-step implicit method to form the Adams 4th-order Predictor-Corrector method. With initial value w 0 = α, suppose we first generate w 1, w 2, w 3 using RK4 method. Then for i = 3, 4,..., N 1: Use Adams-Bashforth 4-step explicit method to get a predictor w i+1,p : w i+1,p = w i + h 24 [ ] 55f (t i, w i ) 59f (t i 1, w i 1 ) + 37f (t i 2, w i 2 ) 9f (t i 3, w i 3 ) Use Adams-Moulton 3-step implicit method to get a corrector w i+1 : w i+1 = w i + h 24 [9f (t i+1, w i+1,p ) + 19f (t i, w i ) 5f (t i 1, w i 1 ) + f (t i 2, w i 2 )] Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 63

64 respectively, compared to those of Runge-Kutta, which were accurate, Predictor-Corrector method = and Example Use Adams The Predictor-Corrector remaining predictor-corrector Method approximations with h = 0.2 towere solve generated us IVP y = yare tshown in fortable t [0, ] and y(0) = 0.5. Table 5.14 Error t i y i = y(t i ) w i y i w i Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 64

65 Other Predictor-Corrector method We can also use Milne s 3-step explicit method and Simpson s 2-step implicit method below: w i+1,p = w i 3 + 4h 3 [ ] 2f (t i, w i ) f (t i 1, w i 1 ) + 2f (t i 2, w i 2 ) w i+1 = w i 1 + h 3 [f (t i+1, w i+1,p ) + 4f (t i, w i ) + f (t i 1, w i 1 )] This method is O(h 4 ) and generally has better accuracy than Adams PC method. However it is more likely to be vulnurable to sound-off error. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 65

66 Predictor-Corrector method PC methods have comparable accuracy as RK4, but often require only 2 evaluations of f in each step. Need to store values of f for several previous steps. Sometimes are more restrictive on step size h, e.g., in the stiff differential equation case later. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 66

67 Variable step-size multistep method Now let s take a closer look at the errors of the multistep methods. Denote y i := y(t i ). The Adams-Bashforth 4-step explicit method has error τ i+1 (h) = y (5) (µ i )h 4 The Adams-Moulton 3-step implicit method has error τ i+1 (h) = y (5) ( µ i )h 4 where µ i (t i 3, t i+1 ) and µ i (t i 2, t i+1 ). Question: Can we find a way to scale step size h so the error is under control? Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 67

68 Variable step-size multistep method Consider the their local truncation errors: y i+1 w i+1,p = y (5) (µ i )h 5 y i+1 w i+1 = y (5) ( µ i )h 5 Assume y (5) (µ i ) y (5) ( µ i ), we take their difference to get w i+1 w i+1,p = ( )y (5) (µ i )h y (5) (µ i )h 5 So the error of Adams-Moulton (corrector step) is τ i+1 (h) = y i+1 w i+1 h 19 w i+1 w i+1,p 270h = Kh 4 where K is independent of h since τ i+1 (h) = O(h 4 ). Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 68

69 Variable step-size multistep method If we want to keep error under a prescribed ɛ, then we need to find q > 0 such that with step size qh, there is τ i+1 (qh) = y(t i + qh) w i+1 qh This implies that 19q4 w i+1 w i+1,p 270h < ɛ ( 270hɛ q < 19 w i+1 w i+1,p )1/4 ( hɛ 2 w i+1 w i+1,p To be conservative, we may replace 2 by 1.5 above. In practice, we tune q (as less as possible) such that the estimated error is between (ɛ/10, ɛ) )1/4 Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 69

70 System of differential equations The IVP for a system of ODE has form du 1 = f 1 (t, u 1, u 2,..., u m ) dt du 2 = f 2 (t, u 1, u 2,..., u m ) dt. for a t b du m dt = f m (t, u 1, u 2,..., u m ) with initial value u 1 (a) = α 1,..., u m (a) = α m. Definition A set of functions u 1 (t),..., u m (t) is a solution of the IVP above if they satisfy both the system of ODEs and the initial values. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 70

71 System of differential equations t j = a + jh, for each j = 0, 1,..., N. Use the notation w ij, for each j = 0, 1,..., N and i = 1, 2,..., m, to denote an approximation to u i (t j ). That is, w ij approximates the ith solution u i (t) of (5.45) at the jth mesh point t j. For the initial conditions, set (see Figure 5.6) w 1,0 = α 1, w 2,0 = α 2,..., w m,0 = α m. (5.48) In this case, we will solve for u 1 (t),..., u m (t) which are interdependent Figure 5.6 according to the ODE system. y y y w 11 w 12 w 13 w 23 w 22 u u w 21 1 (a) α 1 (t) 1 u 2 (a) α 2 u 2 (t) w m3 w m2 w m1 u m (a) α m u m (t) a t 0 t 1 t 2 t 3 t a t 0 t 1 t 2 t 3 t a t 0 t 1 t 2 t 3 t Suppose that the values w 1, j, w 2, j,..., w m, j have been computed. We obtain w 1, j+1, w 2, j+1,..., w m, j+1 by first calculating k 1,i = hf i (t j, w 1, j, w 2, j,..., w m, j ), for each i = 1, 2,..., m; (5.49) k 2,i = hf i (t j + h 2, w 1, j k 1,1, w 2, j k 1,2,..., w m, j + 1 ) 2 k 1,m, (5.50) Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 71

72 System of differential equations Definition A function f is called Lipschitz with respect to u 1,..., u m on D := [a, b] R m if there exists L > 0 s.t. f (t, u 1,..., u m ) f (t, z 1,..., z m ) L for all (t, u 1,..., u m ), (t, z 1,..., z m ) D. m u j z j j=1 Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 72

73 System of differential equations Theorem If f C 1 (D) and f u j L for all j, then f is Lipschitz with respect to u = (u 1,..., u m ) on D. Proof. Note that D is convex. For any (t, u 1,..., u m ), (t, z 1,..., z m ) D, define g(λ) = f (t, (1 λ)u 1 + λz 1,..., (1 λ)u m + λz m ) for all λ [0, 1]. Then from g(1) g(0) 1 0 g (λ) dλ and the definition of g, the conclusion follows. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 73

74 System of differential equations Theorem If f C 1 (D) and is Lipschitz with respect to u = (u 1,..., u m ), then the IVP with f as defining function has a unique solution. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 74

75 System of differential equations Now let s use vector notations below a = (α 1,..., α m ) y = (y 1,..., y m ) w = (w 1,..., w m ) f(t, w) = (f 1 (t, w 1 ),..., f m (t, w m )) Then the IVP of ODE system can be written as with initial value y(a) = a. y = f(t, y), t [a, b] So the difference methods developed above, such as RK4, still apply. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 75

76 System of differential equations Example Use RK4 (with h = 0.1) to solve IVP for ODE system { I 1 (t) = f 1 (t, I 1, I 2 ) = 4I 1 + 3I I 2 (t) = f 2(t, I 1, I 2 ) = 2.4I I with initial value I 1 (0) = I 2 (0) = 0. Solution: The exact solution is { I1 (t) = 3.375e 2t e 0.4t for all t 0. I 2 (t) = 2.25e 2t e 0.4t Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 76

77 As a consequence, System of differential equations I 1 (0.1) w 1,1 = w 1, (k 1,1 + 2k 2,1 + 2k 3,1 + k 4,1 ) Example Use RK4 (with h = 0.1) to solve IVP for ODE system = ( (0.534) + 2( ) ) = and { I 1 (t) = f 1 (t, I 1, I 2 ) = 4I 1 + 3I I I 2 (t) f 2 (0.1) w 2,1 = 2(t, w 2,0 + I 1, 1 I 2 ) = 2.4I I (k 1,2 + 2k 2,2 + 2k 3,2 + k 4,2 ) = withthe initial remaining valueentries I 1 (0) in = Table I 2 (0) 5.19 = 0. are generated in a similar manner. Solution: (cont) The result by RK4 is Table 5.19 t j w 1,j w 2,j I 1 (t j ) w 1,j I 2 (t j ) w 2,j Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 77

78 High-order ordinary differential equations A general IVP for mth-order ODE is y (m) = f (t, y, y,..., y (m 1) ), t [a, b] with initial value y(a) = α 1, y (a) = α 2,..., y (m 1) (a) = α m. Definition A function y(t) is a solution of IVP for the mth-order ODE above if y(t) satisfies the differential equation for t [a, b] and all initial value conditions at t = a. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 78

79 High-order ordinary differential equations We can define a set of functions u 1,..., u m s.t. u 1 (t) = y(t), u 2 (t) = y (t),..., u m (t) = y (m 1) (t) Then we can convert the mth-order ODE to a system of first-order ODEs: u 1 = u 2 u 2 = u 3 for a t b. u m = f (t, u 1, u 2,..., u m ) with initial values u 1 (a) = α 1,..., u m (a) = α m. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 79

80 High-order ordinary differential equations Example Use RK4 (with h = 0.1) to solve IVP for ODE system y 2y + 2y = e 2t sin t, t [0, 1] with initial value y(0) = 0.4, y (0) = 0.6. Solution: The exact solution is y(t) = u 1 (t) = 0.2e 2t (sin t 2 cos t). Also u 2 (t) = y (t) = u 1 (t) but we don t need it. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 80

81 High-order ordinary differential equations Example Use RK4 (with h = 0.1) to solve IVP for ODE system y 2y + 2y = e 2t sin t, t [0, 1] with initial value y(0) = 0.4, y (0) = CHAPTER 5 Initial-Value Problems for Ordinary Differential Equations Solution: (cont) The result by RK4 is Table 5.20 tj y(tj) = u1(tj) w1,j y (tj) = u2(tj) w2,j y(tj) w1,j y (tj) w2,j In Maple the nth derivative y (n) (t) We can also use dsolve from Maple on higher-order equations. To define the differential is specified by (D@@n)(y)(t). equation in Example 1, use Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 81 2t

82 A brief summary The difference methods we developed above, e.g., Euler s, midpoints, RK4, multistep explicit/implicit, predictor-corrector methods, are based on step-by-step derivation and easy to understand; widely used in many practical problems; fundamental to more advanced and complex techniques. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 82

83 Stability of difference methods Definition (Consistency) A difference method is called consistent if ( ) lim max τ i(h) = 0 h 0 1 i N where τ i (h) is the local truncation error of the method. Remark Since local truncation error τ i (h) is defined assuming previous w i = y i, it does not take error accumulation into account. So the consistency definition above only considers how good φ(t, w i, h) in the difference method is. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 83

84 Stability of difference methods For any step size h > 0, the difference method w i+1 = w i + hφ(t i, w i, h) can generate a sequence of w i which depend on h. We call them {w i (h)} i. Note that w i gradually accumulate errors as i = 1, 2,..., N. Definition (Convergent) A difference method is called convergent if ( ) lim max y i w i (h) = 0 h 0 1 i N Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 84

85 Stability of difference methods Example Show that Euler s method is convergent. Solution: We have showed before that for fixed h > 0 there is y(t i ) w i hm ( ) e L(t i a) 1 hm ( ) e L(b a) 1 2L 2L for all i = 0,..., N. Therefore we have max y(ti ) w i hm 2L 1 i N ( ) e L(b a) 1 0 as h 0. Therefore lim h 0 (max 1 i N y(t i ) w i ) = 0. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 85

86 Stability of difference method Definition A numerical method is called stable if its results depend on the initial data continuously. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 86

87 Stability of difference methods Theorem For a given IVP y = f (t, y), t [a, b] with y(a) = α, consider a difference method w i+1 = w i + hφ(t i, w i, h) with w 0 = α. If there exists h 0 > 0 such that φ is continuous on [a, b] R [0, h 0 ], and φ is L-Lipschitz with respect to w, then The difference method is stable. The difference method is convergent if and only if it is consistent (i.e., φ(t, y, 0) = f (t, y)). If there exists bound τ(h) such that τ i (h) τ(h) for all i = 1,..., N, then y(t i ) w i τ(h)e L(t i a) /L. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 87

88 Stability of difference methods Proof. Let h be fixed, then w i (α) generated by the difference method are functions of α. For any two values α, ˆα, there is w i+1 (α) w i+1 (ˆα) = (w i (α) hφ(t i, w i (α))) (w i (ˆα) hφ(t i, w i (ˆα))) w i (α) w i (ˆα) + h φ(t i, w i (α)) φ(t i, w i (ˆα)) w i (α) w i (ˆα) + hl w i (α) w i (ˆα) = (1 + hl) w i (α) w i (ˆα) (1 + hl) i+1 w 0 (α) w 0 (ˆα) = (1 + hl) i+1 α ˆα (1 + hl) N α ˆα Therefore w i (α) is Lipschitz with respect to α (with constant at most (1 + hl) N ), and hence is continuous with respect to α. We omit the proofs for the other two assertions here. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 88

89 Stability of difference method Example Use the result of Theorem above to show that the Modified Euler s method is stable. Solution: Recall the Modified Euler s method is given by w i+1 = w i + h 2 ( ) f (t i, w i ) + f (t i+1, w i + hf (t i, w i )) So we have φ(t, w, h) = 1 2 (f (t, w) + f (t + h, w + hf (t, w))). Now we want to show φ is continuous in (t, w, h), and Lipschitz with respect to w. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 89

90 Stability of difference method Solution: (cont) It is obvious that φ is continuous in (t, w, h) since f (t, w) is continuous. Fix t and h. For any w, w R, there is φ(t, w, h) φ(t, w, h) = 1 f (t, w) f (t, w) f (t + h, w + hf (t, w)) f (t + h, w + hf (t, w)) 2 L 2 w w + L (w + hf (t, w)) ( w + hf (t, w)) 2 L w w + Lh f (t, w) f (t, w) 2 L w w + L2 h w w 2 = (L + L2 h ) w w 2 So φ is Lipschitz with respect to w. By first part of Theorem above, the Modified Euler s method is stable. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 90

91 Stability of multistep difference method Definition Suppose a multistep difference method given by w i+1 = a m 1 w i + a m 2 w i a 0 w i m+1 + hf(t i, h, w i+1,..., w i m+1 ) Then we call the following the characteristic polynomial of the method: λ m (a m 1 λ m a 1 λ + a 0 ) Definition A difference method is said to satisfy the root condition if all the m roots λ 1,..., λ m of its characteristic polynomial have magnitudes 1, and all of those which have magnitude =1 are single roots. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 91

92 Stability of multistep difference method Definition A difference method that satisfies root condition is called strongly stable if the only root with magnitude 1 is λ = 1. A difference method that satisfies root condition is called weakly stable if there are multiple roots with magnitude 1. A difference method that does not satisfy root condition is called unstable. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 92

93 Stability of multistep difference method Theorem A difference method is stable if and only if it satisfies the root condition. If a difference method is consistent, then it is stable if and only if it is covergent. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 93

94 Stability of multistep difference method Example Show that the Adams-Bashforth 4-step explicit method is strongly stable. Solution: Recall that the method is given by w i+1 = w i + h 24 [ ] 55f (t i, w i ) 59f (t i 1, w i 1 ) + 37f (t i 2, w i 2 ) 9f (t i 3, w i 3 ) So the characteristic polynomial is simply λ 4 λ 3 = λ 3 (λ 1), which only has one root λ = 1 with magnitude 1. So the method is strongly stable. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 94

95 Stability of multistep difference method Example Show that the Milne s 3-step explicit method is weakly stable but not strongly stable. Solution: Recall that the method is given by w i+1 = w i 3 + 4h 3 [ ] 2f (t i, w i ) f (t i 1, w i 1 ) + 2f (t i 2, w i 2 ) So the characteristic polynomial is simply λ 4 1, which have roots λ = ±1, ±i. So the method is weakly stable but not strongly stable. Remark This is the reason we chose Adams-Bashforth-Moulton PC rather than Milne-Simpsons PC since the former is strongly stable and likely to be more robust. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 95

96 Stiff differential equations Stiff differential equations have e ct terms (c > 0 large) in their solutions. These terms 0 quickly, but their derivatives (of form c n e ct ) do not, especially at small t. Recall that difference methods have errors proportional to the derivatives, and hence they may be inaccurate for stiff ODEs. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 96

97 Stiff differential equations Example Use RK4 to solve the IVP for a system of two ODEs: u 1 = 9u u cos t 1 3 sin t u 2 = 24u 1 51u 2 9 cos t sin t with initial values u 1 (0) = 4/3 and u 2 (0) = 2/3. Solution: The exact solution is u 1 (t) = 2e 3t e 39t cos t for all t 0. u 2 (t) = e 3t + 2e 39t 1 3 cos t Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 97

98 u Stiff differential 1 (t) = 2e equations 3t e 39t cos t, u 2(t) = e 3t + 2e 39t 1 cos t. 3 The transient term e 39t in the solution causes this system to be stiff. Applying Algorithm 5.7, the Runge-Kutta Fourth-Order Method for Systems, gives results listed in Table When h = 0.05, stability results and the approximations are accurate. Increasing the step Solution: size to h= (cont) 0.1, however, When leads to we theapply disastrousrk4 resultsto shown this in the stiff table. ODE, we obtain Table 5.22 w 1 (t) w 1 (t) w 2 (t) w 2 (t) t u 1 (t) h= 0.05 h= 0.1 u 2 (t) h= 0.05 h= which can blow up for larger step size h. Although stiffness is usually associated with systems of differential equations, the approximation characteristics of a particular numerical method applied to a stiff system can be predicted by examining the error produced when the method is applied to a simple test equation, Numerical Analysis II Xiaojing Ye, Math y = & λy, Stat, Georgia y(0) = State α, University where λ < 0. (5.64) 98

99 Stiff differential equations Now let s use a simple example to see why this happens: consider an IVP y = λy, t 0, and y(0) = α. Here λ < 0. We know the problem has solution y(t) = αe λt. Suppose we apply Euler s method, which is w i+1 = w i + hf (t i, w i ) = w i + hλw i = (1 + λh)w i = = (1 + λh) i+1 w 0 = (1 + λh) i+1 α Therefore we simply have w i = (1 + λh) i α. So the error is y(t i ) w i = αe λih (1 + λh) i α = e λih (1 + λh) i α In order for the error not to blow up, we need at least 1 + λh < 1, which yields h < 2 λ. So h needs to be sufficiently small for large λ. Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 99