Multistage Methods I: Runge-Kutta Methods Varun Shankar January, 0 Introduction Previously, we saw that explicit multistep methods (AB methods) have shrinking stability regions as their orders are increased. Further, the second Dalhquist barrier stopped us from generating high-order A-stable multistep methods. In this chapter, we will introduce Runge-Kutta methods. Unlike the multistep methods, these methods generate information within a time-step to get highorder convergence; these methods are called multistage methods due to the generation of these intermediate stages. We will show how both explicit and implicit Runge-Kutta (RK) methods can be generated from quadrature rules. We will then discuss and analyze the general form of RK methods, abstracting away the polynomial interpolation idea. Explicit RK Methods from Quadrature Let us now derive some popular explicit RK methods. Unlike multistep methods, we will only use information over a single time-step to generate these. For this, we will return to the integral form of the simple ODE problem. This integral form is given by: y n+ = y n + t n+ t n f(t, y(t))dt. (). Forward Euler Using the left endpoint rule to approximate the integral, we get y n+ y n + t n+ t n f(t n, y n ), () = y n+ y n + tf(t n, y n ). (3)
Forward Euler is thus both AB and RK.. Second-order RK methods We now use the trapezoidal rule to approximate the integral. This gives us y n+ y n + t [f(t n, y n ) + f(t n+, y n+ )]. (4) This, however, is an implicit method. To make it explicit, we will simply approximate the y n+ on the right hand side with Forward Euler! This gives y n+ y n + t f(t n, y n ) + t f(t n+, y n + tf(t n, y n )). (5) This is typically written as y n+ y n + t (k + k ), () k = f(t n, y n ), (7) k = f(t n + h, y n + tk ). (8) This is the classical second-order Runge-Kutta method, referred to as RK. It is also known as Improved Euler or Heun s Method. This is not the only RK method. To generate a second RK method, all we need to do is apply a different quadrature rule of the same order to approximate the integral. For example, if we use the Midpoint rule, we get ( ) y n+ y n + tf t n+, yn+. (9) To make this a fully-explicit method, we simply replace y n+ by a Forward Euler approximation. This yields ( y n+ y n + tf t n+, yn + t ) f(t n, y n ). (0) This is called the Midpoint method. There is no real consistent naming scheme. This is simply because there were (and continue to be) armies of researchers developing RK methods of different orders and properties..3 RK4 If f is not dependent on y, RK4 can be derived from Simpson s rule. You will show this in your assignment.
3 General derivations of explicit RK methods As we go to higher-order explicit RK methods, the quadrature approach becomes ridiculously cumbersome. Instead, it is fairly common to generate explicit RK methods using the general form of RK methods. This is typically written as s y n+ = y n + t b i k i, () i= k = f(t n, y n ), () k = f(t n + c t, y n + t(a k )), (3) k 3 = f(t n + c 3 t, y n + t(a 3 k + a 3 k )), (4). (5) k s = f (t n + c s t, y n + t (a s k + a s k +... + a s,s k s )). () It is straightforward to collect the coefficients a ij into the Runge-Kutta matrix A. We also have the condition s c i = a ij. (7) j= For explicit RK methods, A is strictly lower-triangular. This implies that the first row of A is always filled with zeros! Consequently, for explicit RK methods, c = 0. 3. Example: Generating explicit RK methods Let s see how one goes about generating explicit RK methods with the above general formula. To do this, we first write down the general form of an RK method: y n+ = y n + t(b k + b k ) + O( t 3 ), (8) k = f(t n, y n ), (9) k = f(t n + c t, y n + ta k ). (0) Our strategy to find b, b, c and a is straightforward: we will match the Taylor expansion of y n+ to the Taylor expansion of the general RK method. This will give us conditions to find the unknown coefficients. Taylor Expansion for y n+ First, let s Taylor expand y n+. y n+ = y n + t t yn (t) + t t yn (t) + O( t 3 ). () 3
From the ODE, we have y (t) = f(t, y). We can use this to obtain an expression for y (t): y (t) = f t (t, y) + f y (t, y)y (t), () Plugging back into the Taylor series for y n+, we get = f t (t, y) + f y (t, y)f(t, y). (3) y n+ = y n + tf(t n, y n ) + t [f t(t n, y n ) + f y (t n, y n )f(t n, y n )] + O( t 3 ), = y n + t f(t n, y n ) + t [f(t n, y n ) + tf t (t n, y n ) + tf y (t n, y n )f(t n, y n )] + O( t 3 ). (4) (5) Taylor Expansion for the general RK scheme Now that we have an exact Taylor expansion for y n+, we can expand the RK scheme, and compare. First, we Taylor expand f(t n + c t, y n + ta k ): f(t n + c t, y n + ta k ) = f(t n, y n ) + c tf t (t n, y n ) + ta f y (t n, y n )k + O( t ), () = f(t n, y n ) + c tf t (t n, y n ) + ta f y (t n, y n )f(t n, y n ) + O( t ). (7) We then plug in this expression into the general RK scheme and simplify: y n+ = y n + (b + b ) tf(t n, y n ) + b t [c f t (t n, y n ) + a f y (t n, y n ), f(t n, y n )] + O( t 3 ). (8) Comparing Taylor expansions We can now compare the Taylor expansion of y n+ to the Taylor-expanded general RK scheme, and require the terms to match up to O( t 3 ). This gives us three conditions: b + b =, (9) c b =, (30) a b =. (3) These are three nonlinear equations for four unknowns! There are thus multiple choices of the unknowns that satisfy these equations. 4
In practice, we typically pick the c values, then solve the b values, and then solve for the a values. Remember, the c values dictate where between t n and t n+ we place our stages. b = b =, c = a = yields the classical RK method. b = 0, b =, c = a = leads to the previously seen Midpoint method. b = 4, b = 3 4, c = a = 3 describes the so-called Ralston method, which has been shown to minimize the local truncation error for the class of RK schemes. In general, the higher the number of stages s of the RK method, the more the number of conditions and unknowns, and the trickier it becomes to solve for the unknowns. 3. Butcher Tableaux It becomes extremely tedious to write down the different a, b, c values for RK methods. John Butcher designed an elegant solution to this problem: it is now common to collect these coefficients in a table, also called a Butcher tableau. c a a... a s....... Butcher tableaux are represented as follows: c s a s a s... a ss b b... b s A huge list of Butcher tableaux for both explicit and implicit RK methods is given at https://en.wikipedia.org/wiki/list_of_runge%e%80%93kutta_ methods. 4 Implicit RK Methods From Quadrature If you recall the derivation of explicit RK methods from quadrature, we initially ended up with something implicit when applying the quadrature rule, then threw out the implicitness by replacing appropriate terms with a Forward Euler approximation. To generate implicit RK methods (IRK methods), we simply apply the quadrature rule over the interval, and do no replacing. Why use implict RK methods at all? For one thing, we explicit implicit RK methods to be more stable than explicit methods, just like AM and BDF methods were more stable than AB methods. For another, explicit RK methods have a non-monotonic relationship between order p and number of stages s. For p > 4, s > p. An eighth order explicit RK method actually requires stages! However, for implicit RK methods, it is possible to get a much higher order than the number of stages used. 5
Here, we will discuss a few IRK methods and how to generate them. First, we present a definition and theorem that reassure us about the connection between RK methods and polynomial interpolation. Definition: Let c,..., c s be distinct real numbers in [0, ]. The collocation polynomial p(t) is a polynomial of degree s satisfying p(t 0 ) = y 0, (3) p t (t 0 + c i t) = f(t 0 + c i t, p(t 0 + c i t)), i =,..., s, (33) and the numerical solution of the corresponding collocation method is defined by y = p(t 0 + t). Theorem, Wright 970: The collocation method of the above definition is equivalent to the s-stage RK method with coefficients a ij = b i = c i 0 0 l i (t) = k i l j (t)dt, (34) l i (t)dt, (35) t c k c i c k. (3) Practically speaking, this theorem simply tells us that we can generate the RK coefficients for an IRK method by picking the c values, then using the Lagrange interpolating polynomial over the interval to generate a and b values. Recall that the c values are where the intermediate stages are located in time between t n and t n+. 4. Gauss methods Gauss methods are based on Gaussian quadrature. The idea is simple: we select c... c s to be the zeros of some orthogonal polynomial. Some common choices are Legendre polynomials, Chebyshev polynomials of the first kind and Chebyshev polynomials of the second kind. We will briefly discuss Gauss-Legendre IRK methods. A Gauss-Legendre method based on s stages has an order of p = s. Further, all Gauss-Legendre methods are A-stable! These methods therefore can go to arbitrarily high order, making them superior in one way to BDF methods. Unfortunately, they can be very expensive to compute; recall that for an implicit method, one has to solve a linear system. In the case of an s step implicit multistep method that solves an ODE system of M components, the linear system is M M in size. However, for an s stage IRK method, the linear
system is Ms Ms in size. Thus, in practice, Gauss-Legendre methods of p > are rarely used. This is why even BDF3 and BDF4 are popular despite their lack of A-stability; on large systems of ODEs, they are very efficient to solve (at the cost of extra storage). Here is the Butcher tableau for the two-stage Gauss-Legendre method of order 4. 3 + 3 4 4 + 3 4 3 4 Expanding the stages from the Butcher tableau, we have y n+ = y n + t (k + k ), (37) ( ( ) ( ( ) )) 3 k = f t n + t, y n + t 4 k 3 + 4 k, (38) ( ( ) (( ) )) 3 k = f t n + + 3 t, y n + t 4 + k + 4 k. (39) This is obviously implicit! method! It also doesn t look as nice as an explicit RK 4. Radau methods Radau methods have the highest possible order among quadrature formulae with either c = 0 or c s =. In these cases, p = s. Radau formulae allow either the first or last RK stage to be explicit, thereby offering lower costs than Gauss-Legendre RK methods while also allowing for a high-order A-stable method. If c s =, the corresponding collocation method is called a Radau IIA method. 4.3 Lobatto methods Lobatto methods have the highest possible order with c = 0 and c s =. For this to be possible, the c values must be zeros of d s dx s ( x s (x ) s ). (40) The order is p = s. When s =, we recover the implicit Trapezoidal rule. 7