Differential Equations

Differential Equations Definitions Finite Differences Taylor Series based Methods: Euler Method Runge-Kutta Methods Improved Euler, Midpoint methods Runge Kutta (2nd, 4th order) methods Predictor-Corrector Methods Euler-Trapezoidal, Milne Simpson Methods ITCS 4133/5133: Numerical Comp. Methods/Analysis 1 Ordinary Differential Equations

Motivation Engineering problems require estimates of derivatives of functions for analysis Approaches: 1. Use function differences between neighboring points, divided by distance between the points, 2. Fit a function to the relationship between the independent and dependent variable (say an nth order polynomial) and use its derivative ITCS 4133/5133: Numerical Comp. Methods/Analysis 2 Ordinary Differential Equations

Differential Equations An equation that defines a relationship between an unknown function and one or more derivatives Definitions: dy dx d 2 y dx 2 = f(x, y) ( = f x, y, dy ) dx Order: is the order of the highest derivative f(.) may be a function any combination of x, y, and (in case of second order) dy/dx Ordinary Diff. Eq.: f(.) is a function of a single variable ITCS 4133/5133: Numerical Comp. Methods/Analysis 3 Ordinary Differential Equations

Definitions(contd) Linearity: determined by whether f(.) is linear in x, y, dy/dx, Solution: is a function of the independent variable. Boundary Conditions: constraints placed on the solution space. ITCS 4133/5133: Numerical Comp. Methods/Analysis 4 Ordinary Differential Equations

Example Applications 1. Electrical Circuit: Relationship between current and time 2. 1D Heat Flow: L di dt + Ri = E, i = 0 at t = 0 H = KA dt dr where K is the coeff. of thermal conductivity, H is the quantity of heat, A is the area perpendicular to heat flow, T is the temperature. ITCS 4133/5133: Numerical Comp. Methods/Analysis 5 Ordinary Differential Equations

Example Applications ITCS 4133/5133: Numerical Comp. Methods/Analysis 6 Ordinary Differential Equations

Reminder:Finite Differences Forward Difference Backward Difference df(x) dx f(x + x) f(x) = x Two Step Method df(x) dx f(x) f(x x) = x df(x) dx f(x + x) f(x x) = 2 x ITCS 4133/5133: Numerical Comp. Methods/Analysis 7 Ordinary Differential Equations

Taylor Series Based Methods:Euler s Method Given y = f(x, y) treat f(x, y) as a constant and the derivative as a tangent (quotient) Thus, y = y 1 y 0 x 1 x 0 y 1 y 0 = f(x 0, y 0 )(x 1 x 0 ) y 1 = y 0 + hf(x 0, y 0 ) where h = (b a)/n, n is the number of values of x. ITCS 4133/5133: Numerical Comp. Methods/Analysis 8 Ordinary Differential Equations

Taylor Series Based Methods:Euler s Method y 1 = y 0 + hf(x 0, y 0 ) ITCS 4133/5133: Numerical Comp. Methods/Analysis 9 Ordinary Differential Equations

Euler s method: From Taylor Series Taylor s series, truncated to the first term is given by y(x + h) = y(x) + hy (x) + h2 2 y (η) Euler s method follows, since y(x + h) = y i+1, y(x) = y i, y (x) = f(x i, y i ), x < η < x + h ITCS 4133/5133: Numerical Comp. Methods/Analysis 10 Ordinary Differential Equations

Euler s Method: Example 1 y = x + y, 0 x 1, y(0) = 2 ITCS 4133/5133: Numerical Comp. Methods/Analysis 11 Ordinary Differential Equations

y = Euler s Method: Example 2 { y( 2x + 1/x), x 0 1 x = 0 where y(0) = 0.0, 0 x 2.0. ITCS 4133/5133: Numerical Comp. Methods/Analysis 12 Ordinary Differential Equations

Euler s Method: Algorithm ITCS 4133/5133: Numerical Comp. Methods/Analysis 13 Ordinary Differential Equations

Euler s Method: Notes, Errors dy dx is evaluated at beginning of interval Error e increases with the width of (x x 0 ), as higher order terms become more important Also known as one-step Euler method Local Error: Range over a single step size; measure difference between numerical solution at end of step (starting with exact solution at beginning of the step) and the exact solution at end of step. Global Error: Accumulates over the range of the solution; measured as the difference between numerical and exact solutions. Errors using Euler s method can be approximated using the second order term of the Taylor series: ɛ = h2 2 y (η) ITCS 4133/5133: Numerical Comp. Methods/Analysis 14 Ordinary Differential Equations

Runge-Kutta Methods To improve on Euler s method, we can use additional terms of the Taylor series. Problem: Need to compute additional higher order derivatives, which can be problematic for complex functions. Runge-Kutta methods determine the y value (dependent variable) based on the value at the beginning of the interval, step size and some representative slope over the interval Euler s and the mnodified Euler s methods are special cases of these techniques Runge-Kutta methods are classified based on their order; fourth order is the most commonly used. Higher order derivates are not required of these methods ITCS 4133/5133: Numerical Comp. Methods/Analysis 15 Ordinary Differential Equations

Second Order Runge-Kutta Methods General Form: k 1 k 2 = hf(x n, y n )//slope at beginning of interval = hf(x n + c 2 h, y n + a 21 k 1 )//slope at end of interval Iteration: y n+1 = y n + w 1 k 1 + w 2 k 2 Example Methods: Improved Euler: c 2 = 1, a 21 = 1, w 1 = w 2 = 0.5 Midpoint: c 2 = 1, a 21 = 2/3, w 1 = 0, w 2 = 1 Heun: c 2 = 2/3, a 21 = 2/3, w 1 = 1/4, w 2 = 3/4 ITCS 4133/5133: Numerical Comp. Methods/Analysis 16 Ordinary Differential Equations

Second Order Runge Kutta Methods: Midpoint Method Idea: Approximate the value of y at (x + h/2) by summing current value of y and one-half the change in y from Euler s method: k 1 k 2 = hf(x i, y i )// change in y : Euler s method = hf(x i + h/2, y i + k 1 /2)// change in y: slope est. at midpoint Iteration: y i+1 = y i + k 2 ITCS 4133/5133: Numerical Comp. Methods/Analysis 17 Ordinary Differential Equations

Runge Kutta Methods:Improved Euler s Method A second order Runge-Kutta method. Estimates of y at start and midpoint of interval are averaged to produce a revised estimate of y at end of interval. Procedure: k 1 = hf(x n, y n ) k 2 = hf(x n + h, y n + k 1 ) y n+1 = y n + 1 2 k 1 + 1 2 k 2 ITCS 4133/5133: Numerical Comp. Methods/Analysis 18 Ordinary Differential Equations

Improved Euler s Method: Procedure Evaluate y at start of interval Estimate y at end of interval using Euler s method Evaluate y at end of interval Compute average slope Compute a revised y at end of interval using average slope ITCS 4133/5133: Numerical Comp. Methods/Analysis 19 Ordinary Differential Equations

Example : Midpoint and Improved Euler s Methods ITCS 4133/5133: Numerical Comp. Methods/Analysis 20 Ordinary Differential Equations

Classic Runge-Kutta Method: Fourth Order A commonly used class of Runge-Kutta methods. k 1 = hf(x i, y i ) k 2 = hf(x i + 0.5h, y i + 0.5k 1 ) k 3 = hf(x i + 0.5h, y i + 0.5k 2 ) k 4 = hf(x i + 0.5h, y i + k 3 ) Iteration: y i+1 = y i + 1 6 (k 1 + 2k 2 + 2k 3 + k 4 ) ITCS 4133/5133: Numerical Comp. Methods/Analysis 21 Ordinary Differential Equations

Classic Runge-Kutta Method: Algorithm ITCS 4133/5133: Numerical Comp. Methods/Analysis 22 Ordinary Differential Equations

Classic Runge-Kutta Method: Example 1 ITCS 4133/5133: Numerical Comp. Methods/Analysis 23 Ordinary Differential Equations

Classic Runge-Kutta Method: Example 2 ITCS 4133/5133: Numerical Comp. Methods/Analysis 24 Ordinary Differential Equations

Predictor-Corrector Methods Euler s method and Runge-Kutta methods generally require step sizes to be small, else might not yield precise solutions Predictor-Corrector methods can be used to increase the accuracy of solutions These methods use solutions from previous intervals to project to the end of the next interval, followed by iterative refinement. Disadvantage: Requires values from previous intervals - one-step methods such as the Euler s method have to be used. Predictor: Gets an initial estimate at the end of the interval. Corrector: Improves the estimate by iteration. ITCS 4133/5133: Numerical Comp. Methods/Analysis 25 Ordinary Differential Equations

Euler-Trapezoidal Method Uses Euler s method as predictor and Trapezoidal rule as corrector Predictor y i+1,0 = y i, + h dy dx i, Corrector y i+1,j = y i, + h 2 [ dy dx + dy i, dx i+1,j 1 ] ITCS 4133/5133: Numerical Comp. Methods/Analysis 26 Ordinary Differential Equations

Euler-Trapezoidal Method: Example ITCS 4133/5133: Numerical Comp. Methods/Analysis 27 Ordinary Differential Equations

Euler-Trapezoidal Method: Example ITCS 4133/5133: Numerical Comp. Methods/Analysis 28 Ordinary Differential Equations

Milne-Simpson Method Uses Milne s method as predictor and Simpson s rule as corrector Predictor y i+1,0 = y i 3, + 4h 3 [ 2 dy dx dy i, dx + 2 dy i 1, dx i 2, ] Corrector y i+1,j = y i 1, + h 3 [ dy dx + 4 dy i+1,j 1 dx + dy i, dx i 1, ] ITCS 4133/5133: Numerical Comp. Methods/Analysis 29 Ordinary Differential Equations

Milne-Simpson Method: Example ITCS 4133/5133: Numerical Comp. Methods/Analysis 30 Ordinary Differential Equations

Milne-Simpson: Example ITCS 4133/5133: Numerical Comp. Methods/Analysis 31 Ordinary Differential Equations

ITCS 4133/5133: Numerical Comp. Methods/Analysis 32 Ordinary Differential Equations