Ordinary Differential Equations

Ordinary Differential Equations Introduction: first order ODE We are given a function f(t,y) which describes a direction field in the (t,y) plane an initial point (t 0,y 0 ) We want to find a function y(t) for t t 0,T such that y(t 0 ) y 0 initial condition y (t) f(t,y(t)) for t t 0,T ordinary differential equation (ODE) This is called an initial value problem (IVP). The partial derivative f y (t,y) f (t,y) is important for the behavior of the differential equation. y Theorem.. Assume that f(t,y) and f y (t,y) are continuous for t t 0,T, y R. For a given initial value y 0 there is a unique solution y(t) of the initial value problem. Either the solution exists for all t 0,T, or it only exists on a smaller interval t 0,t ) with t 0 < t < T. We can solve the IVP in Matlab with ode45: f @(t,y)... % define function f(t,y) ts,ys ode45(f,t0,t,y0); % find column vectors ts,ys with values of solution ys(end) % value y(t) of solution plot(ts,ys) % plot solution Example: Find a function y(t) for t, 3 such that Here we have t 0, y 0 0, f(t,y) t y. y (t) t y(t) y( ) 0 Numerical solution in Matlab: (using m-file dirfield.m from course web page) f @(t,y) t-y^ % define function f(t,y) dirfield(f, -:.:3, -:.:.6); hold on % plot direction field ts,ys ode45(f,-,3,0); % solve IVP for t from - to 3, initial value 0 % this gives vectors ts,ys plot(ts,ys, b ); hold off % plot solution

.5 0.5 0-0.5 - - -0.5 0 0.5.5.5 3 What happens if we perturb the initial value y 0? Theorem.. Let y(t) denote the solution of the IVP with initial condition y(t 0 ) y 0, let ỹ(t) denote the solution of the IVP with initial condition ỹ(t 0 ) ỹ 0. Assume f y (t,y) M for t t 0,T, y R. Then ỹ(t) y(t) ỹ 0 y 0 e M(t t 0) for t t 0,T For M < 0 the difference ỹ(t) y(t) decays exponentially for increasing t. For M > 0 the difference may increase exponentially. We call the ODE unstable if we have f y (t,y) > 0 for all t t 0,T, y R. We call the ODE stable if we have f y (t,y) < 0 for all t t 0,T, y R. System of ODEs, higher order ODEs We want to find n functions y (t),...,y n (t) for t t 0,T satisfying the differential equations y (t) f (t,y (t),...,y n (t)). y n(t) f n (t,y (t),...,y n (t)) and the initial conditions y (t 0 ) y (0),...,y n(t 0 ) y (0) n. We use vector notation: E.g., for n we want to find y(t) y (t) y (t) f (t,y (t),y (t)) f (t,y (t),y (t)) y (t) f (t, y(t)), y(t 0 ) y (0), y (t) y (t) y (t 0 ) y (t 0 ) such that y (0) y (0)

We will omit the vector arrows from now on. We denote by D y f(t,y) the Jacobian of f(t,y) with respect to y: f y D y f(t,y). f n y It is important for the behavior of the differential equation. Theorem.. Assume that f(t,y) and D y f(t,y) are continuous for t t 0,T, y R n. For a given initial value y (0) there is a unique solution y(t) of the initial value problem. Either the solution exists for all t 0,T, or it only exists on a smaller interval t 0,t ) with t 0 < t < T. We can solve the IVP in Matlab with ode45: For n we use f @(t,y)... ;... % define function f(t,y) using t, y(), y() ts,ys ode45(f,t0,t,y0); % find column vector ts, array ys with values of solution ys(end,:) % values y, y at final time T plot(ts,ys(:,)) % plot solution y(t) f y n. f n y n nd order ODE So far the differential equations only contained the first derivative y (t). But in many applications (e.g. Newton s law) we have differential equations containing y (t). We then need initial conditions for y(t 0 ) and y (t 0 ). We can rewrite this as a first order system: Let y (t) : y(t) and y (t) : y (t). Then we have y y and y where we solve the nd order ODE for y. Example: Find a function y(t) for t 0, 4 such that y (t) y (t)+3y(t) t () y(0), y (0) () This gives the first order system y y y t+y 3y, y (0) y (0) (3) Numerical solution in Matlab: Print out y(t) and plot the function y(t) f @(t,y) y(); t+y()-3*y(); % define function f(t,y) ts,ys ode45(f,0,4,-;); % solve IVP for t from 0 to 4, initial value -; finalval ys(end,) % value of y at final time T plot(ts,ys(:,)); % plot solution y(t) 3 Euler method Consider a first order system of ODEs: We want to find y(t) for t t 0,T such that y (t) f (t,y(t)), y(t 0 ) y (0) For the Euler method we divide the interval t 0,T into N subintervals of equal length h (T t 0 )/N (we can also use subintervals of different length). Let t j t 0 +jh. We then want to find approximations y (),...,y (N) for y(t j ). 3

start at the initial value t 0,y (0) for k 0,...,N do s : f(t k,y (k) ) y (k+) : y (k) +hs t k+ : t k +h Example: Consider the Initial Value Problem (), (). Use steps of the Euler method with h to find approximations for y() and y (). We use y (t) : y(t) and y (t) ( y (t)) and obtain the first order ODE (3). Step : s f(t 0,y (0) ) f 0,, y () y (0) +h s + 5 0 5 4.5 ( ) Step : s f(t,y () ) f, 0 4.5, y () y () 0 +h s + 4.5 5 4.5 4.5.5 5 7 This gives y().5 and y () 7. Errors for Euler method We consider the case n. At time t k the Euler method gives an approximation y k for the exact value y(t k ). We denote the error by e k : y k y(t k ) By Taylor s theorem we have with a remainder term r k y (τ k )h y (t k ) {}}{ y(t k+ ) y(t k )+h f (t k,y(t k ))+r k y k+ y k + h f(t k,y k ) The second equation is just the definition of the Euler approximation y k+. Subtracting the first from the second equation gives e k+ e k +h f(t k,y k ) f(t k,y(t k )) r k Using the mean value theorem for g(y) : f(t k,y) f(t k,y k ) f(t k,y(t k )) f y (t k,η k ) y k y(t k ), hence the new error is e k+ +hf y (t k,η k ) e k r k }{{} a k with the amplification factor a k +hf y (t k,η k ) and the local truncation error r k y (τ k )h. For an unstable ODE we have f y (t,y) > 0 and hence a k >. For a stable ODE we have f y (t,y) < 0 and hence a k <. However, in this case we want a k <, i.e., < +hf y (t,y) < The right inequality is true for any h > 0. The left inequality is true if the following stability condition holds: h < f y 4

() General case: We assume f y (t,y) C fort t 0,T, y R y (t) C fort t 0,T Then we get bounds a k A for the amplification factor and r k R for the local truncation error: a k +hf y (t k,η k ) +hc : A r k y (τ k )h C h : R yielding Since e 0 0 we obtain e k+ A e k +R e R e AR+R (+A)R e 3 A(+A)R+R ( +A+A ) R. e k ( +A+ +A k ) R (4) We have for the geometric series +A+ +A k Ak A Ak A (+hc ) k hc The function e x satisfies + x e x, hence with x hc we get + hc e hc. Using this and R C h in (4) gives the error bound This shows: y k y(t k ) C C e C (t k t 0 ) h if we keep taking Euler steps with a fixed value h for t the error can increase exponentially. This is not surprising: For an unstable ODE any tiny initial error can cause an exponentially increasing error for t. if we only want to find the solution for t t 0,T: We use h T t 0 N ch c N : The Euler method is a method of order. Example: The initial value problem y y sint cost, y(0) has the solution y(t) cost. Here f y (t,y) > 0. We use the Euler method with h 0.: f @(t,y) y-sin(t)-cos(t) tv 0:.:5; plot(tv,cos(tv), b ); hold on % plot exact solution dirfield(f,0:.3:5,-:.:3); ts,ys Euler(f,0,5,,50); % use 50 steps of size 5/50 plot(ts,ys, r.- ); hold off and obtain errors bounded by 5

3.5.5 0.5 0-0.5-0 0.5.5.5 3 3.5 4 4.5 We see that the Euler values go exponentially to + as t gets larger than 4, whereas the exact solution y(t) cos t stays bounded. () Special case: Stable ODE where h satisfies stability condition: We assume f y (t,y) < 0: We have C C 0 > 0 such that C f y (t,y) C 0 fort t 0,T, y R y (t) C fort t 0,T We now want to have an amplification factor with a k C 0 h, i.e., ( C 0 h) +hf y (t k,η k ) C 0 h The right inequality holds for any h > 0. We have hc +hf y (t k,η k ), therefore the left inequality holds if ( C 0 h) hc or h (5) C 0 +C If h satisfies this stability condition we have a k C 0 h : A <, hence and (4) now gives This shows: +A+ +A k Ak A A y k y(t k ) C C 0 h if we keep taking Euler steps with a fixed value h for t the error is bounded by Ch with a fixed constant C. Example: The initial value problem y y sint+cost, y(0) has the solution y(t) cost. Here f y (t,y) < 0. We use the Euler method with h 0.: 6

f @(t,y) -y-sin(t)+cos(t) tv 0:.:0; plot(tv,cos(tv), b ); hold on % plot exact solution dirfield(f,0:.3:0,-.:.:.); ts,ys Euler(f,0,0,,50); % use 50 steps of size 0/50 plot(ts,ys, r.- ); hold off 0.5 0-0.5-0 3 4 5 6 7 8 9 0 Here we have an error of size Ch, but the error stays bounded as t. 4 Improved Euler method (aka RK method) For the Euler method we obtained a local truncation error r k satisfying r k ch. Since we use N (T t 0 )/h steps to get from t 0 to the final time T, we obtained the error at time T was bounded by Ch : The Euler method is a method of order. This means that in order to improve the error by a factor of 0, we need to use 0 times as many steps. If we want to achieve an error of size 0 8 we may need of the order of 0 8 steps (assuming e.g. a stable problem with C and C 0 of about ). We would like to have a method of order, i.e., the error at time T is bounded by Ch. This means that the local truncation error should satisfy r k ch 3. How can we do this? We need more than one evaluation of the direction field per step. We start at the initial point (t 0,y 0 ). We take a step of size h to t h and want to find an approximation y for y(t ). We know that y (t) f (t,y(t)), so by the fundamental theorem of calculus we have y(t ) y 0 + t t 0 f (t,y(t))dt Let g(t) : f (t,y(t)). Then we have to approximate the integral I g(t)dt with an error ch 3. One way t 0 to do this is to use the trapezoid rule: Recall that on an interval of size h we have Q Trap I h 3 max g. Therefore we want to use I Q Trap h g(t 0)+g(t ) h f(t 0,y 0 )+f(t,y(t )) }{{}? But we cannot evaluate the second term f(t,y(t )) since we don t know y(t ). So we use the best approximation we have: the Euler approximation y Euler y 0 +h f(t 0,y 0 ). Since y Euler y(t ) Ch we get from the mean value theorem f(t,y Euler ) f(t,y(t )) f y (t,η) (y Euler y(t ) ) C Ch t 7

We now approximate y(t ) by y : y 0 + h f(t 0,y 0 )+f(t,y Euler ) (6) and obtain for the local truncation error y y(t ) Ch 3 since the error of the trapezoid rule is bounded by ch 3, and replacing y(t ) by y Euler causes an additional error h C Ch. The iteration (6) gives the Improved Euler method: we divide the interval t 0,T into N subintervals of equal length h (T t 0 )/N (we can also use subintervals of different length). Let t j t 0 +jh. We then want to find approximations y (),...,y (N) for y(t j ). start at the initial value t 0,y (0) for k 0,...,N do s () : f(t k,y (k) ) y E : y (k) +hs () s () : f(t k +h,y E ) y (k+) : y (k) + s () +s () t k+ : t k +h The local truncation error of the improved Euler method is of order O(h 3 ). Hence the error at a time t T is of order O(h ) O(N ): The improved Euler method is a method of order. Note that we use two evaluations of the function f per step: s () f(t k,y (k) ) and s () f(t k +h,y E ). Example: Consider the Initial Value Problem (), (). Use step of the Improved Euler method with h to find approximations for y( ) and y ( ). We use y (t) : y(t) and y (t) y (t) and obtain the first order ODE (3). s () f(t 0,y (0) ) f ( 0, ) 5, y E y (0) +h s + 5 4.5 0 s () f(t,y E ) f (, 4.5 0 ) 4.5 5, y() y (0) + h This gives y( ) 0.65 and y ( ) 4.5. ( s () +s ()) + 4 ( 5 +4.5 5 ) 0.65 4.5 5 Stiff ODE and ode5s Consider a very stable ODE where f y (t,y) is very negative. Then the Euler method only works if the step size h satisfies the stability condition h < f y. This can force use to use very tiny steps even if the solution y(t) is almost constant. This is called a stiff ODE. In this case ode45 uses many tiny steps and takes a long time. Example: A flame propagation model gives the following IVP: y y y 3 fort 0, δ y(0) δ Here δ is very small, e.g., δ 0 4. In this case we want to solve the problem for t 0, δ 0, 04. The solution approaches y. But there the problem becomes stiff: We have near y that f y (t,y) y 3y, so the stability condition for the Euler method requires h < f y. This means that we need N 0 4 steps of size h to get from t 0 0 to T /δ, despite the fact that the solution is almost constant for most of 0,T. The adaptive method ode45 (with default settings) also requires about 0 4 steps: 8

delta e-4; f @(t,y) y^-y^3; y0 delta; t0 0; T /delta; ts,ys ode45(f,0,t,y0); length(ts) % print number of steps This prints out 3 for the number of steps. Matlab has a special ode solver ode5s for stiff ODEs: We try this for our problem ts,ys ode5s(f,0,t,y0); length(ts) and get 08 for the number of steps. % print number of steps 6 Backward Euler method (aka implicit Euler method) For stable problems the Euler method gives magnification factors a k +hf y < for small h, but a k +hf y > for large h. If I look at a problem with f y < 0 from right to left with decreasing t, then an Euler method in decreasing t direction always has a magnification factor a >, for any step size h > 0. This suggests to use a backward Euler step : At time t k we have the value y (k). For time t k+ t k +h we want to find a value y (k+) such that an Euler step to the left takes us to t k and y (k) : Find y (k+) such that y (k+) h f ( t k+,y (k+)) y (k) (7) Note that the unknown vector y (k+) occurs also inside the function f. If the function f(t,y) is linear in y this gives linear equations for y. If the function f(t,y) is nonlinear in y this gives nonlinear equations for y. We can e.g. use or steps of the Newton method (note that we have a local truncation error of size O(h ) ). Example: Consider the Initial Value Problem (), (). Use step of the backward Euler method with h to find approximations for y( ) and y ( ). We use y (t) : y(t) and y (t) y (t) and obtain the first order ODE (3). Note that the function f(t,y) y t+y 3y depends linearly on y,y. We want to find a vector y () y y such that y y h f(t, y y ) y (0) y y y +y 3y This gives the linear system y y 3 y + y.5 which has the solution y () gives y( ) 0.5 and y ( ) 3. y y 3. This Claim: Let n. For a stable problem with f y (t,y) < 0 the nonlinear equation has a unique solution. Proof: The left hand side F(y k+ ) : y k+ h f(t k+,y k+ ) is strictly increasing for increasing y k+, with F(y) for y and F(y) for y. 9