Solving Ordinary Differential Equations

Solving Ordinary Differential Equations Sanzheng Qiao Department of Computing and Software McMaster University March, 2014

Outline 1 Initial Value Problem Euler s Method Runge-Kutta Methods Multistep Methods Implicit Methods Hybrid Method 2 Software Packages

Problem setting Initial Value Problem (first order) find y(t) such that y = f(y, t), given initial value y(t 0 ). Usually we assume t 0 = 0

Problem setting Initial Value Problem (first order) find y(t) such that y = f(y, t), given initial value y(t 0 ). Usually we assume t 0 = 0 Generalization 1: system of first order ODEs: y is a vector and f a vector-valued function. Example { y 1 = f 1(y 1, y 2, t) y 2 = f 2(y 1, y 2, t) or in vector notations: y = f(y, t)

Problem setting (cont.) Generalization 2: high order equation Let u = g(u, u, t). y 1 = u y 2 = u and transform the above into the following system of first order ODEs: { y 1 = y 2 Initial conditions: u(0) and u (0) y 2 = g(y 1, y 2, t)

Solution family A differential equation has a family of solutions, each corresponds to an initial value. 3.5 3 2.5 2 1.5 1 0.5 0-0.1-0.05 0 0.05 0.1 0.15 0.2 0.25 0.3 y = y, solution family y(t) = Ce t, y(0) = C.

Euler s method We consider the initial value problem: y = f(y, t), y(t 0 ) = y 0 Numerical solution: find approximations y n y(t n ), for n = 1, 2,... Note: y 0 = y(t 0 ) (initial value)

Euler s method We consider the initial value problem: y = f(y, t), y(t 0 ) = y 0 Numerical solution: find approximations Note: y 0 = y(t 0 ) (initial value) y n y(t n ), for n = 1, 2,... A k-step method: Compute y n+1 using y n k+1,..., y n 1, y n.

Euler s method (cont.) A single-step method: Euler s method. f(y 0, t 0 ) = y (t 0 ) y(t 1) y(t 0 ) h 0, where h 0 = t 1 t 0. The first step: y 1 = y 0 + h 0 f(y 0, t 0 )

Euler s method (cont.) A single-step method: Euler s method. f(y 0, t 0 ) = y (t 0 ) y(t 1) y(t 0 ) h 0, where h 0 = t 1 t 0. The first step: y 1 = y 0 + h 0 f(y 0, t 0 ) Euler s method y n+1 = y n + h n f(y n, t n ) Produces: y 0 = y(t 0 ), y 1 y(t 1 ), y 2 y(t 2 ),...

Example y = y, y(0) = 1.0. (Solution y = e t )

Example y = y, y(0) = 1.0. (Solution y = e t ) h = 0.4 Step 1: y 1 = y 0 hy 0 = 1.0 0.4 1.0 = 0.6 ( y(0.4) = e 0.4 0.6703) 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Example u 1 (t) = 0.6e t+0.4 0.8951e t in the solution family. u 1 = u 1, u 1 (0) 0.8951 (u 1 (0.4) = 0.6) 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Example Step 2: y 2 = y 1 hy 1 = 0.6 0.4 0.6 = 0.36

Example Step 2: y 2 = y 1 hy 1 = 0.6 0.4 0.6 = 0.36 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Example u 2 (t) = 0.36e t+0.8 0.8012e t in the solution family. u 2 = u 2, u 2 (0) 0.8012 (u 2 (0.8) = 0.36) 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Euler s method In general u n(t) = f(u n (t), t), in the solution family u n (t n ) = y n, passing (t n, y n ) u n (t n+1 ) u n (t n )+h n u n(t n ) = y n + h n f(u n (t n ), t n ) = y n + h n f(y n, t n ) = y n+1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Euler s method Starting with t 0 and y 0 = y(t 0 ), as we proceed, we jump from one solution in the family to another. 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Errors Two sources of errors: discretization error and roundoff error. Discretization error: caused by the method used, independent of the computer used and the program implementing the method.

Errors Two sources of errors: discretization error and roundoff error. Discretization error: caused by the method used, independent of the computer used and the program implementing the method. Two types of discretization error: Global error: e n = y n y(t n ) Local error: the error in one step

Local error Consider t n as the starting point and the approximation y n at t n as the initial value, if u n (t) is the solution of then the local error is u n = f(u n, t), u n (t n ) = y n d n = y n+1 u n (t n+1 )

Example Step 1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Local error d 0 = y 1 y(t 1 ) = 0.6 e 0.4 0.0703. Global error e 1 is the same as the local error d 0.

Example Step 2 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Local error d 1 = y 2 u 1 (t 2 ) = 0.36 u 1 (0.8) 0.0422. Global error e 2 = y 2 y(t 2 ) = 0.36 e 0.8 0.0893.

Example 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 d 0 = y 1 y(t 1 ) = 0.6 e 0.4 0.0703 d 1 = y 2 u 1 (t 2 ) = 0.36 u 1 (0.8) 0.0422 e 2 = y 2 y(t 2 ) = 0.36 e 0.8 0.0893

Stability Relation between global error e n and local error d n If the differential equation is unstable, e N > N 1 n=0 If the differential equation is stable, e N N 1 n=0 d n d n In this case, N 1 n=0 d n is an upper bound for the global error e N.

Example In the previous example: Local errors d 0 = 0.0703 and d 1 = 0.0422 Global error e 2 = 0.0893 e 2 < d 0 + d 1

Example In the previous example: Local errors d 0 = 0.0703 and d 1 = 0.0422 Global error e 2 = 0.0893 e 2 < d 0 + d 1 More generally, y = αy, solution family y = Ce αt. Stable when α < 0.

Accuracy A measurement for the accuracy of a method An order p method: C: independent of n and h n. d n Ch p+1 n (or O(h p+1 n ))

Example: Euler s method y n+1 = y n + h n f(y n, t n ) Local solution u n (t) u n (t) = f(u n(t), t), u n (t n ) = y n Taylor expansion at t n : u n (t) = u n (t n )+(t t n )u (t n )+O((t t n ) 2 ) Since y n = u n (t n ) and u (t n ) = f(y n, t n ), we get u n (t n+1 ) = y n + h n f(y n, t n )+O(hn 2 ) Local error d n = y n+1 u n (t n+1 ) = O(hn) 2 Euler s method is a first order method (p = 1)

Accuracy (cont.) Consider the interval [t 0, t N ] and partition t 0, t 1,..., t N. Roughly, the global error e N N 1 n=0 d n N O(h p+1 ) (t N t 0 ) O(h p ) at the final point t N is roughly O(h p ) for a method of order p. For a pth order method, if the subintervals h n are cut in half, then the average local error is reduced by a factor of 2 p+1, the global error is reduced by a factor of 2 p. (But double the number of steps, i.e., more work.)

Roundoff Error Each step of the Euler s method y n+1 = y n + h n f(y n, t n )+ǫ ǫ = O(u). Total rounding error: Nǫ = bǫ/h (b = t N t 0, fixed step size h) ( total error b Ch+ ǫ ) h

Roundoff Error Each step of the Euler s method y n+1 = y n + h n f(y n, t n )+ǫ ǫ = O(u). Total rounding error: Nǫ = bǫ/h (b = t N t 0, fixed step size h) ( total error b Ch+ ǫ ) h Remarks If h is too small, the roundoff error is large If h is too large, the discretization error is large

Runge-Kutta methods Idea: Sample f at several spots to achieve high order. Cost: More function evaluations

Runge-Kutta methods Idea: Sample f at several spots to achieve high order. Cost: More function evaluations Example. A second-order Runge-Kutta method Suppose y n+1 = y n +γ 1 k 0 +γ 2 k 1 where γ 1 and γ 2 to be determined and k 0 = h n f(y n, t n ) k 1 = h n f(y n +βk 0, t n +αh n ) α and β to be determined.

Runge-Kutta methods (cont.) Taylor series (two variables y and t): Thus k 1 = h n (f n +βk 0 f y (y n, t n )f n +αh n f t (y n, t n )+ ) y n+1 = y n +(γ 1 +γ 2 )h n f n +γ 2 βh 2 nf n f y(y n, t n ) +γ 2 αh 2 nf t(y n, t n )+... The Taylor expansion of the true local solution: u n (t n+1 ) = u n (t n )+h n u n(t n )+ h2 n 2 u n(t n )+... = y n + h n f n + h2 n 2 (f y (y n, t n )f n + f t (y n, t n ))+...

Second order RK method Comparing the two expressions, we set γ 1 +γ 2 = 1 γ 2 β = 1/2 γ 2 α = 1/2 Then the local error The global error O(h 2 ) d n = y n+1 u n (t n+1 ) = O(h 3 n)

Second order RK method Let γ 1 = 1 1 2α, γ 2 = 1 2α, β = α Second-order RK method y n+1 = y n + where k 0 = h n f(y n, t n ) k 1 = h n f(y n +αk 0, t n +αh n ) ( 1 1 ) k 0 + 1 2α 2α k 1

Second order RK method Let γ 1 = 1 1 2α, γ 2 = 1 2α, β = α Second-order RK method y n+1 = y n + where k 0 = h n f(y n, t n ) k 1 = h n f(y n +αk 0, t n +αh n ) ( 1 1 ) k 0 + 1 2α 2α k 1 When α = 1/2, related to the rectangle rule When α = 1, related to the trapezoid rule

Classical fourth-order Runge-Kutta method where k 0 = hf(y ( n, t n ) ) k 1 = hf y n + 1 2 k 0, t n + h 2 ( ) k 2 = hf y n + 1 2 k 1, t n + h 2 k 3 = hf(y n + k 2, t n + h) y n+1 = y n + 1 6 (k 0 + 2k 1 + 2k 2 + k 3 )

Multistep Methods Compute y n+1 using y n, y n 1,... and f n, f n 1,... possibly f n+1 (f i = f(y i, t i )). General linear k-step method: y n+1 = k k α i y n i+1 + h β i f n i+1 i=1 i=0 β 0 = 0 (no f n+1 ), explicit method β 0 0, implicit method

Examples Adams-Bashforth methods (explicit). tn+1 y n+1 = y n + p k 1 (t)dt t n where p k 1 (t) is a polynomial of degree k 1 which interpolates f(y, t) at (y n j, t n j ), j = 0,..., k 1.

Examples Adams-Bashforth methods (explicit). tn+1 y n+1 = y n + p k 1 (t)dt t n where p k 1 (t) is a polynomial of degree k 1 which interpolates f(y, t) at (y n j, t n j ), j = 0,..., k 1. For example. p 0 (t) = f n, Euler s method p 1 (t) = f n 1 + fn f n 1 h n 1 (t t n 1 )

Adams-Bashforth family y n+1 = y n + hf n local error h2 2 y(2) (η), order 1 y n+1 = y n + h 2 (3f n f n 1 ) local error 5h3 12 y(3) (η), order 2 y n+1 = y n + h 12 (23f n 16f n 1 + 5f n 2 ) local error 3h4 8 y(4) (η), order 3 y n+1 = y n + h 24 (55f n 59f n 1 + 37f n 2 9f n 3 ) local error 251h5 720 y(5) (η), order 4

Multistep methods (cont.) The start-up issue in multistep methods: How to get the k 1 start values f j = f(y j, t j ), j = 1,..., k 1?

Multistep methods (cont.) The start-up issue in multistep methods: How to get the k 1 start values f j = f(y j, t j ), j = 1,..., k 1? Use a single step method to get start values, then switch to multistep method.

Exmaple The motion of two bodies under mutual gravitational attraction. A coordination system: origin: position of one body x(t), y(t): position of the other body Differential equations derived from Newton s laws of motion: x (t) = α2 x(t) r(t) y (t) = α2 y(t) r(t) where r(t) = [x(t) 2 + y(t) 2 ] 3/2 and α is a constant involving the gravitational constant, the masses of the two bodies, and the units of measurement.

Example If the initial conditions are chosen as x(0) = 1 e, x (0) = 0, y(0) = 0, y (0) = α( 1+e 1 e ) 1/2 for some e with 0 e < 1, then the solution is periodic with period 2π/α. The orbit is an ellipse with eccentricity e and with one focus at the origin. To write the two second-order differential equations as four first-order differential equations, we introduce y 1 = x, y 2 = y, y 3 = x, y 4 = y

Exmaple We have a system of first-order euquations y 1 y 2 y 3 y 4 = y 3 y 4 y 1 s y 2 s with the initial condition s = (y2 1 + y2 2 )3/2 α 2, = y 1 (0) y 2 (0) y 3 (0) y 4 (0) = 0 0 1 0 0 0 0 1 1/s 0 0 0 0 1/s 0 0 1 e 0 0 α( 1+e 1 e ) 1/2 y 1 y 2 y 3 y 4

Example The function defining the system of equations; function yp = orbit(y, t) global a global e yp = zeros(size(y)); r = y(1)*y(1) + y(2)*y(2); r = r*sqrt(r)/(a*a); yp(1) = y(3); yp(2) = y(4); yp(3) = -y(1)/r; yp(4) = -y(2)/r; endfunction

Example A script filetwobody.m (Octave) global a global e a = input( a = ); e = input( e = ); # initial value x0 = [1-e; 0.0; 0.0; a*sqrt((1+e)/(1-e))]; # time span t = [0.0:0.1:(2*pi/a)] ; # solve ode [x, state, msg] = lsode( orbit, x0, t); Matlab [t, x] = ode45( orbit, [0.0 2*pi/a], x0);

Example Output x: matrix of four columns x(:,1): x(t) x(:,2): y(t) x(:,3): x (t) x(:,4): y (t) plot(x(:,1), x(:,2)) 1 0.5 0-0.5-1 -1-0.5 0 0.5 e = 1/4, α = π/4

Implicit methods Example y = 20y, y(0) = 1 (Solution e 20t ) Euler s method, h = 0.1 y n+1 = y n 20 0.1y n = y n 2y n 2 1.5 1 0.5 0-0.5-1 -1.5-2 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35

Implicit methods Example y = 20y, y(0) = 1 (Solution y 20t ) Euler s method, h = 0.1 y n+1 = y n 20 0.1y n = y n 2y n 2 1.5 1 0.5 0-0.5-1 -1.5-2 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35

Backward Euler s method Example y = 20y, y(0) = 1 (Solution y 20t ) Backward Euler s method, h = 0.1 y n+1 = y n 20 0.1y n+1 = y n 2y n+1 0.8 0.6 0.4 0.2 0-0.2 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35

Backward Euler s method Taylor expansion of the solution y(t) about t = t n+1 (instead of t = t n ) y(t n+1 + h) y(t n+1 )+y (t n+1 )h = y(t n+1 )+f(y(t n+1 ), t n+1 )h and set h = h n = (t n t n+1 ), then we get y(t n ) y(t n+1 ) h n f(y(t n+1 ), t n+1 )

Backward Euler s method (cont.) y(t n ) y(t n+1 ) h n f(y(t n+1 ), t n+1 ) Substituting y n for y(t n ) and y n+1 for y(t n+1 ), we have Backward Euler s method y n+1 = y n + h n f(y n+1, t n+1 )

Backward Euler s method (cont.) y(t n ) y(t n+1 ) h n f(y(t n+1 ), t n+1 ) Substituting y n for y(t n ) and y n+1 for y(t n+1 ), we have Backward Euler s method y n+1 = y n + h n f(y n+1, t n+1 ) Implicit methods tend to be more stable than their explicit counterparts. But there is a price, y n+1 is a zero of a usually nonlinear function.

Example A system of two differential equations. { u = 998u+1998v v = 999u 1999v If the initial values u(0) = v(0) = 1, then the exact solution is { u = 4e t 3e 1000t v = 2e t + 3e 1000t 4 3.5 3 2.5 2 1.5 1 0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04

Example (cont.) Suppose we use (forward) Euler s method u n+1 = u n + h(998u n + 1998v n ) v n+1 = v n + h( 999u n 1999v n ) with u 0 = v 0 = 1.0, then we get h = 0.01 h = 0.001 -------------------------------- u0 1.0 1.0 u1 30.96 3.996 u2-239.1 3.992 u3-9785 -7.988 u4 52420-31.92

Example (cont.) ui ui+1 ui+2 ti ti+1 ti+2 ti+3 t ui+3

Example (cont.) If we use the backward Euler method u n+1 = u n + h(998u n+1 + 1998v n+1 ) v n+1 = v n + h( 999u n+1 1999v n+1 )

Example (cont.) If we use the backward Euler method u n+1 = u n + h(998u n+1 + 1998v n+1 ) v n+1 = v n + h( 999u n+1 1999v n+1 ) Solving the linear system [ 1 998h 1998h 999h 1+1999h ] [ un+1 v n+1 ] = [ un v n ]

Example (cont.) With initial values u 0 = v 0 = 1.0, h = 0.01 h = 0.001 ------------------------------- u0 1.0 1.0 u1 3.688 2.496 u2 3.896 3.242 u3 3.880 3.613 u4 3.844 3.797

Example (cont.) For comparison, the solution values h = 0.01 h = 0.001 ------------------------------- u(0) 1.0 1.0 u(1) 3.960 2.892 u(2) 3.921 3.586 u(3) 3.882 3.839 u(4) 3.843 3.929

Hybrid methods One of the difficulties in implicit methods is that they involve solving usually nonlinear systems.

Hybrid methods One of the difficulties in implicit methods is that they involve solving usually nonlinear systems. Combining both explicit and implicit: Use an explicit method for predictor and an implicit method for corrector Example. Fourth-order Adam s method Predictor (fourth-order, explicit): y n+1 = y n + h 24 (55f n 59f n 1 + 37f n 2 9f n 3 ) Corrector (fourth-order, implicit): y n+1 = y n + h 24 (9f n+1 + 19f n 5f n 1 + f n 2 )

PECE methods Algorithm: 1 Prediction: use the predictor to calculate y (0) n+1 2 Evaluation: f (0) n+1 = f(y(0) n+1, t n+1) 3 Correction: use the corrector to calculate y (1) n+1 Steps 2 and 3 are repeated until y (i+1) n+1 y(i) n+1 tol. Then set y n+1 = y (i+1) n+1.

Outline 1 Initial Value Problem Euler s Method Runge-Kutta Methods Multistep Methods Implicit Methods Hybrid Method 2 Software Packages

Software packages IMSL ivprk (RK), ivpag (AB) MATLAB ode23, ode45 (RK), ode113 (AB), ode15s, ode23s (stiff), bvp4c (BVP) NAG d02baf (RK), d02caf (AB), d02eaf (stiff) Netlib dverk (RK), ode (AB), vode, vodpk (sfiff) Octave lsode

Summary Family of solutions of a differential equation, initial value problem Transform a high order ODE into a system of first order ODEs Errors: Discretization errors (global, local), stability of a differential equation (mathematical stability), roundoff error, total error Accuracy: Order of a method

Summary (cont.) Euler s method: Explicit, single step, first order Runge-Kutta methods: Explicit, single step Multistep methods: Adams-Bashforth family Implicit methods: Backward Euler s method Prediction-Correction scheme: Combination of explicit and implicit methods

References [1 ] George E. Forsyth and Michael A. Malcolm and Cleve B. Moler. Computer Methods for Mathematical Computations. Prentice-Hall, Inc., 1977. Ch 6.