Comparison of Numerical Ordinary Differential Equation Solvers

Transcription

1 Adrienne Criss Due: October, 008 Comparison of Numerical Ordinary Differential Equation Solvers Many interesting physical systems can be modeled by ordinary differential equations (ODEs). Since it is always possible to transform an ordinary differential equation into a set of coupled first-order differential equations, the problem of modeling the physical system can be reduced to solving a set of coupled first-order differential equations. In this project, we use the motion of a planet around a sun as our physical system in order to study three methods of solving a set of first order ODEs. The methods studied are a fourth-order Runge-Kutta with fixed step size, a fourth-order Runge-Kutta with variable step size, and the Bulirsch-Stoer method. We find that for our test system the Bulirsch-Stoer method gave the most accurate results, as determined by percent error from the true energy and angular momentum of the system. Introduction An Nth order ODE can be reduced to N coupled first-order ODEs through the process of rewriting the equations with new variables. Generally, the new variables are chosen to be the derivatives of the original ones and each other [1]. As an example, consider a general second-order equation dy dy + q(x) = r(x) dx dx (1) This can be rewritten as dy = z( x ) dx () dz = r ( x ) q ( x )z( x ) dx Therefore, the general set of equations to solve are dyi( x ) = ai( x, yi...yn ) dx (3) where i runs from 1 to N. In order to solve this system, we must have boundary conditions. The two classes of boundary conditions are initial value conditions, where we know the values of all the yi at a starting value of x, and two-point boundary value conditions, where the values of the yi are given at two values of x [1]. In this project we deal exclusively with initial value problems. Assuming we have the necessary boundary conditions, the simplest way to solve (3) is to let the dyi's and dx's go to finite steps Δyi and Δx. Then the solution at some time t + Δt is yi ( t + t ) = yi ( t ) + ai ( t ) t (4) This is called the Euler algorithm. While conceptually simple, it has very limited accuracy and stability and thus is not considered as a stand-alone ODE solver here. One problem with the Euler algorithm is that is only uses the derivative at the beginning of the interval. A modification, called the mid-point

2 algorithm, see (5), is similar to the Euler algorithm, but uses the derivative at the mid-point of the interval []. vn + 1 = vn + an ( t ) (5) 1 xn + 1 = xn + ( vn vn ) t For simplicity, we have dropped the yi notation and assumed one-dimensional motion of a particle. The subscript n denotes the value at time tn and the subscript (n+1) denotes the value at time tn + Δt. The order of an algorithm corresponds to how the global error depends on time. If the global error goes as (Δt)n, the algorithm is said to be nth order. The Euler algorithm is first-order. The midpoint method is first-order in velocity and second-order in position, which can be seen by plugging the first equation of (5) into the second; terms up to (Δt) are kept in velocity and (Δt) in position []. In the model presented here, we use the motion of one planet around a sun as our physical system. Motion is confined to the xy-plane. We assume an initial state and evolve it forward in time to a specified final time. In this example, Newton's equation of motion is d r GM = dt r (6) Rewriting the equation and working in x and y components, we obtain dx = vx ( t ) dt dvx GMx = = ax ( t ) dt r3 dy = vy ( t ) dt dvy GMy = = ay ( t ) dt r3 (7) Three separate methods of solving ODEs are compared: a fourth-order Runge-Kutta with fixed step size, a fourth-order Runge-Kutta with variable step size, and the Bulirsch-Stoer method. The methods are fed the same initial conditions. Two sets of initial conditions are considered: a stable moderate orbit and an orbit close to the sun. The ODE solvers are run from time t = 0 to time t = 40 years. The number of steps taken as well as the time and accuracy of each method are then compared. The Runge-Kutta methods and the Bulirsch-Stoer method are described below along with how they are implemented in the code. Method The first method implemented is the fourth-order Runge-Kutta method with fixed step size. To describe the fourth-order Runge-Kutta method, we will start by explaining the second-order RungeKutta method, following [1]. The basic idea is that the Euler method is used to take a step halfway across the desired interval. The derivative at that midpoint is then used to cross the full interval. This method is equivalent to the midpoint method. Writing it a different way k1 = ( t )a ( xn, yn ) k = ( t )a ( xn t, yn + k1) yn + 1 = yn + k + O( t 3 ) (8)

3 it is clear that the lowest order error term is of order (Δt)3. Lower orders have been canceled out. This is the general idea for higher order Runge-Kutta methods. The right hand side is evaluated in different ways that differ in terms higher than first-order. Judicious choices of combinations of these evaluations can cancel error terms one order at a time [1]. The form for the fourth-order Runge-Kutta is k1 = ( t )a ( xn, yn ) k = ( t )a ( xn + t k1, yn + ) k 3 = ( t )a ( xn + t k, yn + ) (9) k 4 = ( t )a ( xn + t, yn + k 3) yn + 1 = yn + k1 k k 3 k O( t 5 ) This is represented graphically by figure 1. Figure 1: The fourth-order Runge-Kutta evaluates the derivative four times over a given time step. By choosing the weights for each derivative, lower orders of error can be canceled out. [1] There are two main ways of implementing a Runge-Kutta with adaptive step size control. The first method is by step doubling. Step-doubling consists of taking each step twice, once as the full step, and again as two half steps. The truncation error is given as the difference of these two measurements. This error is kept within set bounds by adjusting Δt [1]. The second method of variable step size is called an embedded Runge-Kutta. In this method, a fifth-order Runge-Kutta that requires six evaluations of the function is also used to calculate a fourth-

4 order Runge-Kutta with the same six evaluations, but in a different combination. The form for the fifthorder Runge-Kutta is k1 = ( t )a ( xn, yn ) k = ( t )a ( xn + a t, yn + b 1k 1) (10)... k 6 = ( t )a ( xn + a 6 t, yn + b 61k1 + b 6 k + b 63k 3 + b 64 k 4 + b 65 k 5) yn + 1 = yn + c1k1 + c k + c3 k 3 + c 4 k 4 + c5 k 5 + c 6 k 6 + O(h 6 ) The embedded fourth-order formula is y * n + 1 = yn + c * 1k1 + c * k + c * 3k 3 + c * 4 k 4 + c * 5 k 5 + c * 6 k 6 + O(h 5 ) (11) The difference between these two answers gives an estimate of the error. The coefficients are listed in table 1 [1]. Cash-Karp Parameters for Embedded Runge-Kutta Method i ai bij ci c*i 37/378 85/ / / / / / /5 1/5 3 3/10 3/40 9/40 4 3/5 3/10-9/10 6/ /54 5/ -70/7 6 7/8 1631/ /51 35/7 575/ / / /1771 1/4 j= Table 1: List of the coefficients for the embedded Runge-Kutta method [1]. The embedded Runge-Kutta method is implemented in this project, since it is more efficient than stepdoubling [1]. Once the error Δ = yn+1 y*n+1 has been calculated, the step is rescaled. The relation between Δ1 corresponding to a step size Δt1 and Δ corresponding to a step size Δt is given by 0. 1 t1 = t The time steps are adjusted to maintain Δ within a prescribed level of accuracy. (1)

5 The last method implemented is the Bulirsch-Stoer method. While the first two methods take rather small steps in time, the Bulirsch-Stoer method is designed to cover much larger steps. The method first spans a large range, T, using n1 = steps and the modified midpoint method. This yields a result y1. Then the same range, T, is covered using n = 4 steps, yielding a result y. This process is repeated for some series of n, here chosen to be n=,4,6,8,10,1,14,... [1]. See figure for a schematic of this process. Figure : The modified midpoint method with increasing number of intervals is used to span a large step, H. The solutions to each modified midpoint cycle are fitted with a polynomial to extrapolate to a solution for y(h) [1]. After each repetition, a polynomial is fitted though the yi's. If the error in the fitting exceeds the accuracy desired, another repetition, with the next higher n in the sequence is calculated, and the process repeated up to a maximum n. In this program, n was chosen to be 16. If the accuracy at this point is still unacceptable, the size of the step, T, is decreased to Tk = T ( ε ε k + 1, k )1/( k + 1) (13) Here k represents the position in the sequence of n, ε is the error required, and εk+1,k is the error returned [1]. Verification Energy and angular momentum are conserved for this system. These two quantities can be calculated explicitly at every time using the formulas E= 1 GM ( vx + vy ) 3 r (14) Lz = xvy yvx Verification of each method at every step is performed by comparing the values calculated at that step with the initial values obtained using the initial conditions.

6 Data For the first run, the initial conditions were set to x = 1.0 Au, vx = 0.0 Au/yr, y = 0.0 Au, vy = 8.0 Au/yr and t = 0.0 yr. The simulation was run until t = 40 years. The step size was set to 0.01 yr for the fixed step Runge-Kutta. The same step size was suggested for the two variable step size programs. The plots produced are shown below. Figure 3: Orbit of the planet for each of the three different models. Note that the blue points are essentially covered by the green and red.

7 Figure 4: Close up of the orbits shown in the previous figure. Note that the green points tend to spiral outward in time. Also note that only two red points are visible on this plot.

8 Figure 5: Only the orbit obtained from the Bulirsch-Stoer method is plotted here in order that the spacing between the points may be seen.

9 Figure 6: Energy versus time for the different methods.

10 Figure 7: A close-up of the previous plot so that the variation in the energy from the fixed Runge-Kutta and Bulirsch-Stoer methods may be seen.

11 Figure 8: Angular momentum versus time for the different methods.

12 Figure 9: A close-up of the previous plot so that the lack of variation in the angular momentum from the fixed Runge-Kutta and Bulirsch-Stoer methods may be seen.

13 For the second run, the initial conditions were set to x = 0.5 Au, vx = 0.0 Au/yr, y = 0.0 Au, vy = 6.8 Au/yr and t = 0.0 yr. The simulation was run until t = 40 years. The step size was set to 0.01 yr for the fixed step Runge-Kutta. The same step size was suggested for the two variable step size programs. The plots produced are shown below. Figure 10: Orbit of the planet for each of the three different models. Note that the blue orbit is highly unstable and the planet eventually leaves the system.

14 Figure 11: Close up of the orbits shown in the previous figure. Note that the green points tend to spiral outward in time, while, at least initially, the blue points spiral inward.

15 Figure 1: Only the orbit obtained from the Bulirsch-Stoer method is plotted here in order that the spacing between the points may be seen.

16 Figure 13: Energy versus time for the different methods.

17 Figure 14: A close-up of the previous plot so that the variation in the energy from the Runge-Kutta adaptive step size and Bulirsch-Stoer methods may be see. Note that the blue points do not appear because they have already gone off scale.

18 Figure 15: Angular momentum versus time for the different methods.

19 Figure 16: A close-up of the previous plot so that the variation in the angular momentum from the Runge-Kutta adaptive step size and Bulirsch-Stoer methods may be see. Note that the blue points do not appear because they have already gone off scale.

20 Run 1 (time(msec) # steps) Run (time(msec) # steps) Fixed Runge-Kutta Adaptive Runge-Kutta Bulirsch-Stoer 0.0 Table : The run time and number of steps for each method. Energy (> 0.5% > 5.0%) Angular Mom. (> 0.5% > 5.0%) Run Run Adaptive Runge-Kutta Run Run Run Fixed Runge-Kutta Bulirsch-Stoer Run Table 3: The time in years at which each method reaches a certain percent error in energy and angular momentum.

21 Analysis Run 1 Looking at figure 3, the orbits of the planet for run 1, the orbits all look very similar. However, upon taking a closer look, figures 4 and 5, we see that the orbit calculated by the adaptive Runge-Kutta tends to increase, while the orbit of the fixed Runge-Kutta and the Bulirsch-Stoer do not. Indeed, from the energy and angular momentum versus time graphs, we see that the energy and angular momentum are not conserved for the adaptive Runge-Kutta, but increase rapidly relative to the other methods. From table 3 we learn that it takes approximately 4.4 years for the energy to differ from the true energy of the system by 0.5 percent and 13 years for the angular momentum to differ from the true angular momentum by 0.5 percent in the adaptive Runge-Kutta method. For these initial conditions, the other two methods never reached this error and no method reached 5 percent error within to 40 year model time. By looking at table, we can compare the run times for the different methods. Both the fixed Runge-Kutta and the Bulirsch-Stoer ran in less than one microsecond, while the adaptive Runge-Kutta took.3 microseconds to run. The adaptive Runge-Kutta also took the most steps to finish by almost a factor of 7. Given that the adaptive Runge-Kutta was the least accurate and slowest of the three methods for the initial conditions, it is the worst method for use in this particular situation with these parameters. Run From figure 10, we can already see that the fixed step Runge-Kutta breaks down. The planet spirals closer to the sun and then shoots out of the system. Looking at the energy and angular momentum graphs confirm this breakdown. The values diverge and then jump as the planet becomes unbound. From table 3, we see that the error of this method surpasses 0.5% in 0.1 years and 5.0 percent within one year, remarkably poor accuracy. The fast runtime does not make up for the lack of realism in this model. The adaptive step Runge-Kutta behaved much as it did in the previous run. The orbit gained energy and angular momentum as can be seen from the close up graph of the orbit and the energy and angular momentum graphs. However, the errors increased much faster. It took.7 years for the error in the energy to reach 5.0 percent and 6. years for the error in the angular momentum to reach 5.0 percent. This run took slightly over 00,000 steps and 30 microseconds. The Bulirsch-Stoer method performed as in run one. The errors never increased above 0.5 percent. The only difference between the runs was that this run required 3738 steps while the first run required 18 and this run took 0.6 milliseconds, while the first one required less than one microsecond. The Bulirsch-Stoer method is clearly the best of the three methods for this case; it is the most accurate and fastest method that is stable. Interpretation We can conclude that the Bulirsch-Stoer method was the most accurate and fastest method that gave reasonable results for our model of a planet around a sun for our two sets of initial conditions. We cannot conclude that the Bulirsch-Stoer method is the best in all models though. It comes with the caveat that it does not perform well with non-smooth functions [1]. The model here had smooth functions. What would happen in the case of non-smooth functions is something that can be explored at a later date. Between the fixed step Runge-Kutta and the adaptive Runge-Kutta, we found that the performance depended heavily on the initial conditions. While the adaptive Runge-Kutta gained energy and angular momentum in both cases, although at different rates, the fixed Runge-Kutta either

22 performed brilliantly for failed miserably depending on initial conditions. Therefore, we can say that the fixed Runge-Kutta seems to be more sensitive to initial conditions. In a further study, we could explore this dependence more. It should be noted that if the step size, dt in the program, is changed from 0.01 to 0.001, the fixed Runge-Kutta no longer diverges. The behavior of the other two methods does not appear to change. Critique I gained a much better understanding of the inner workings of these three ODE solvers. I learned that there is no one best method for all systems. The characteristics of the equations as well as the initial conditions dictate what method is most appropriate. These factors can have a huge impact on the performance of the solvers. While I already knew Fortran, I learned how to implement the above mentioned ODE solvers in Fortran thanks to Numerical Recipes. References [1] W. Press, S. Teukolsky, W. Vetterling, and B. Flannery, Numerical Recipes in Fortran 77, Cambridge University Press, New York, (003). [] H. Gould, J. Tobochnik, W. Christian, An Introduction to Computer Simulation Methods, Pearson Addison Wesley, New York, (007). Log Date Start Time End Time Time (hr) Activity 9/5 10:00 am 1:30 pm.5 Reading 9/5 1:15 pm :15 pm 1.0 Reading 9/9 10:00 am 7:00 pm 9.0 Programming 9/30 10:00 am 5:00 pm 7.0 Programming 9/30 10:45 pm 1:45 am.0 Reading/Programming/Writing 10/1 10:30 am 1:00 pm.5 Writing 10/1 4:30 pm 5:15 pm 0.75 Writing 10/1 10:30 pm 1:30 am.0 Writing 10/ 1:30 am 5:30 am 4.0 Writing 10/ 1:30 pm 1:00 pm 0.5 Proofreading