Strang-type Preconditioners for Solving System of Delay Dierential Equations by Boundary Value Methods Raymond H. Chan Xiao-Qing Jin y Zheng-Jian Bai

Transcription

1 Strang-type Preconditioners for Solving System of Delay Dierential Equations by oundary Value Methods Raymond H. han iao-qing Jin y Zheng-Jian ai z ugust 27, 22 bstract In this paper, we survey some of the latest developments in using boundary value methods (VMs) for solving systems of delay dierential equations (DDEs). These methods require the solutions of nonsymmetric, large and sparse linear systems. The GMRES method with the Strang-type preconditioner is proposed for solving these systems. One of the main results is thatifan 2 -stable VM is used for a system of DDEs, then the preconditioner is invertible and the preconditioned matrix can be decomposed as I + L where I is the identity matrix and L is a low rank matrix. It follows that when the GMRES method is applied to solving the preconditioned systems, the method will converge fast. Introduction Let us begin with the initial value problem (IVP) of ordinary dierential equations >< >: dy(t) dt y(t )=z = J m y(t)+g(t) t 2 (t T] () where y(t), g(t) : R! R m, z 2 R m, and J m is the stiness matrix in R mm. The initial value methods (IVMs), such as the Runge-Kutta methods, are well-known methods for solving (), see []. Recently, another class of methods called the boundary value methods (VMs) has been proposed, see [5]. Using VMs to discretize (), we get a linear system Mu = b where u, b are vectors and M is a matrix depending on the multistep rule we used. The advantage of using VMs is that the methods are more stable and the resulting linear system is hence more wellconditioned. However, the system is in general large and sparse (with band-structure), and solving it is a maor problem in the application of the VMs. We use the GMRES method, which is one of Krylov subspace methods, for solving the discrete system. In order to speed up the convergence of the GMRES iterations, the Strang-type preconditioner S is proposed to precondition the discrete Department of Mathematics, The hinese University of Hong Kong, Shatin, Hong Kong. The research was partially supported by the Hong Kong Research Grant ouncil grant UHK4243/P and UHK DG y Faculty of Science and Technology, The University of Macau, Macau. The research was partially supported by the research grant RG24/-2S/JQ/FST from the University of Macau. z Department of Mathematics, The hinese University of Hong Kong, Shatin, Hong Kong, hina.

2 system. The advantage of the Strang-type preconditioner is that if an 2 -stable VM is used in (), then S is invertible and the preconditioned matrix can be decomposed as S ; M = I + L where the rank of L is at most 2m( + 2 ) which is independent of the integration step size. It follows that the GMRES method applied to the preconditioned system will converge in at most 2m( + 2 ) + iterations in exact arithmetic. VMs for the solution of ordinary dierential equations of the form () have been studyed in [6]. In this paper, we discuss the use of VMs to solve three dierent types of delay dierential equations, such as the neutral delay dierential equations, the dierential equations with multidelay, and the singular perturbation delay dierential equations. The GMRES method with the Strang-type preconditioners is proposed to solve the resulting linear systems. The outline of the paper is as follows. In x2, we give some background knowledge about the linear multistep formulae. Then, we investigate the properties of the Strang-type block-circulant preconditioner for the neutral delay dierential equations, the dierential equations with multi-delay, and the singular perturbation delay dierential equations in x3{x5, respectively. The convergence analysis of the method is given with some illustrative numerical examples. 2 Linear Multistep Formulae onsider a single -dimensional IVP y = f(t y) y(t )=y : t 2 (t T] (2) Over a uniform mesh t = t + h = r with step size h =(T ; t )=r, the -step linear multistep formula (LMF) is dened as follows: = y n+ ; h = f n+ = n = r; : (3) Here, y n is the discrete approximation to y(t n ), f n = f(t n y n ). To get the solution by (3), we need initial conditions y, y,, y ;. Since only y is provided from the original problem, we have to nd additional conditions for the remaining values y, y 2,,, y ;. The method in (3) with the ( ; ) additional conditions is called Initial Value Methods (IVMs). n IVM is called implicit if 6=and explicit if =. If an IVM is applied to the IVP (2) on the interval [t t r+; ] with uniform stepsize h, we have the following discrete problem where y =(y y + y r+; ) T, f =(f f + f r+; ) T, r y = h r f + g (4) ; g =( ( i y i ; h i f i ) y ; ; h f ; ) T 2

3 r =. r =. : rr rr Note that the matrices r and r are lower triangular Toeplitz matrices. We recall that a matrix is said to be Toeplitz if its entries are constant along its diagonals. Moreover, the linear system (4) can be solved easily by forward recursion. classical example of IVM is the second order backward dierentiation formulae: 3y n+2 ; 4y n+ + y n =2hf n+ which isatwo-step method with =, = ;4, 2 = 3 and =2. Instead of using IVM which sets initial conditions for (3), we can also use the so-called oundary Value Methods (VMs). Given, 2 such that + 2 =, then the corresponding VM requires initial addition conditions y, y,, y ; and 2 nal addition conditions y r, y r+,, y r+2 ;, which are called ( 2 )-boundary conditions. Note that the class of VMs contains the class of IVMs (i.e. =, 2 =). The discrete problem generated by a -step VM with ( 2 )-boundary conditions can be written in the following matrix form y = hf + g where y =(y y + y r; ) T, f =(f f + f r; ) T, g = ; ( i y i ; h i f i ) y ; ; h f ; y r ; h f r 2 i= ( +iy r;+i ; h +if r;+i )! T and 2 R (r; )(r; ) are dened as follows, =.. =.. : (5) Note that the coecient matrices are Toeplitz with lower bandwidth and upper bandwidth 2. n example of VMs is the third order generalized backward dierentiation formulae (GDF), 2y n+ +3y n ; 6y n; + y n;2 =6hf n which isathree-step method with (2,)-boundary conditions, and that =, = ;6, 2 =3, 4 =2and =6. 3

4 In order to study the stability properties of the VM, we introduce the characteristic polynomials (z) and(z) of the VM which are dened by (z) = z and (z) = z (6) where f i g and f i g are given by (3). The 2 -stability polynomial is dened by where z, q 2. Let ; fq 2 : Re(q) < g. Denition [5] The region (z q) (z) ; q(z) (7) D 2 = fq 2 : (z q) has zeros inside z =and 2 zeros outside z =g is called the region of 2 -stability of a given VM with ( 2 )-boundary conditions. Moreover, the VM is said to be 2 -stable if ; D 2 : lthough IVMs are more ecient than VMs (which cannot be solved by forward recursion), the advantage in using VMs over IVMs comes from their stability properties. For example, the usual backward dierentiation formulae are not -stable for >2 but the GDF are ; -stable for any, see for instances [, 3] and [5, p. 79 and Figures 5.{5.3]. 3 Neutral Delay Dierential Equations In this section, we consider the solution of neutral delay dierential equations: < : y (t) =L n y (t ; )+M n y(t)+n n y(t ; ) y(t) =(t) t t t t () where y(t), (t) :R! R n L n, M n, N n 2 R nn,and > is a constant. y applying a VM, the discrete solution of () is given by the solution of a linear system where H depends on the LMF used. Hy = b 3. VMs and Their Stability Properties In order to nd a reasonable numerical solution, we require that the solution of () is asymptotically stable. Let () denote the spectrum of a matrix, I n the n-by-n identity matrix, and kk 2 the 2-norm. We have the following lemma, see [9, 3]. Lemma Let L n, M n and N n be any matrices and kl n k 2 <. Then the solution of () is asymptotically stable if Re( i ) < for any i, where i 2 ; (I n ; L n ) ; (M n + N n ) with. 4

5 Let h = =k be the step size where k is a positive integer. For (), by using a VM with ( 2 )-boundary conditions over a uniform mesh t = t + h = r on the interval [t t + r h], we have i y p+i; = i L n y p+i; ;k + h i (M n y p+i; + N n y p+i; ;k ) (9) for p = ::: r ;, where = + 2, and f i g, f ig VM, see [2]. y providing the values are the coecients of the given y ;k ::: y y ::: y ; y r ::: y r + 2 ; () (9) can be written in a matrix form as Hy = b where H I n ; () L n ; h M n ; h () N n () y T =(y T y T + ::: yt r ;) 2 R n(r ; ) and b 2 R n(r ; ) is a vector that depends on the boundary values and the coecients of the method. In (), the matrices, 2 R (r ; )(r ; ) are dened as in (5), and (), () 2 R (r ; )(r ; ) are given as follows: () =. () = see [2]. We remark that the rst column of () is given by: ( ::: ::: k + ; and the rst column of () is given by ::: r ;k ;2 ; ( ::: ::: ::: k + ; r ;k ;2 ;. ) T ) T : (2) 5

6 3.2 Strang-type Preconditioner We rst dene the Strang's preconditioner for Toeplitz matrix. For any given Toeplitz matrix T l =[t i; ] l i = =[t q], Strang's circulant preconditioner s(t l ) is a circulant matrix with diagonals given by t q q bl=2c [s(t l )] q = >< >: t q;l [s(t l )] l+q bl=2c <q<l < ;q <l see [7, ]. The Strang-type block-circulant () preconditioner for () is dened as follows: S s() I n ; s( () ) L n ; hs() M n ; hs( () ) N n (3) = (F I n )( I n ; () L n ; h M n ; h () N n )(F I n ) (4) where s(e) is Strang's preconditioner of Toeplitz matrix E, E is the diagonal matrix given by E = Fs(E)F for E =,, (), () respectively, and F is the Fourier matrix. Now we discuss the invertibility of the Strang-type preconditioner. Let w = e 2i r ; i p ;, we have [ ] = (w )=w [ ] = (w )=w [ ()] = w ;k ; w ;k ; = (w )=w k + and [ ()] = w ;k ; w ;k ; = (w )=w k + where (z) and(z) are dened as in (6). Thus the th-block of where I n ; () L n ; h M n ; h () N n in (4) is given by S = [ ] I n ; [ ()] L n ; h[ ] M n ; h[ ()] N n h = w k ((w )I n ; h(w )M n ) ; (w )L n ; h(w )N n i w k + for = 2 ::: r ;. In order to prove that S is invertible, we need to establish that S is invertible for = 2 ::: r ;. Let where e ik ((e i )I n ; h(e i )M n ) ; (e i )L n ; h(e i )N n = e ik (I n ; e ;ik L n )D D (e i )I n ; h(i n ; e ;ik L n ) ; (M n + e ;ik N n )(e i ): (5) So, to prove that S is invertible, we onlyneedtoshowthatisinvertible for any 2 R. ssume that kl n k 2 <, we have I n ; e ;ik L n is nonsingular for any 2 R. Therefore, we only need to show forany 2 R, D is invertible. We have the following theorem. 6

7 Theorem [2] If the VM with ( 2 )-boundary conditions is 2 -stable and the conditions in Lemma hold, then for any 2 R, the matrix D dened by (5) is invertible. It follows that the Strang-type preconditioner S dened as in (3) is also invertible. 3.3 Spectral nalysis In this section, we discuss the convergence rate of the preconditioned GMRES method with the Strang-type preconditioner. First of all, we recall the following well-known result. Lemma 2 [] Let W be invertible and can be decomposed asw = I +L. If the GMRES method is applied to solving the linear system W x = b, then the method will converge in at most rank(l)+ iterations in exact arithmetic. Using Lemma 2, we have the following result for the spectra of preconditioned matrices and for the convergence rate of our method. Theorem 2 [2] Let H be given by () and S be given by (3), then we have S ; H = I n(r ; ) + L where I n(r ; ) 2 R n(r ; )n(r ; ) is the identity matrix and L is a low rank matrix with rank(l) 2( + k + +)n: Therefore, when the GMRES method is applied to solving the preconditioned system S ; Hy = S ; b the method will converge in at most 2( + k + +)n + iterations in exact arithmetic. We observe from Theorem 2 that if the step size h = =k is xed, the number of iterations for convergence of the GMRES method, when applied to solving S ; Hy = S ; b, will be independent of r,thatistosay, it is independent of the length of the interval that we considered. Regarding the cost per iteration, the main work in each GMRES iteration is the matrix-vector multiplication S ; Hy. Since () () are band matrices and L n M n N n are assumed to be sparse, the matrix{vector multiplication Hy can be done very fast. To compute S ; z for any vector z, by (4), it follows that S ; z =(F I n )( I n ; () L n ; h M n ; h () N n ) ; (F I n )z: This product can be obtained by using Fast Fourier Transforms and solving r ; linear systems of order n, see [4]. Since L n M n N n are sparse, the coecient matrices of the n-by-n linear systems will also be sparse. Thus S ; z can be obtained by solving r ; sparse n-by-n linear systems. 7

8 3.4 Numerical Result We illustrate the eciency of our preconditioner by solving Example below. We used the MTL-provided M-le \gmres" (see MTL on-line documentation) to solve the preconditioned systems. In all of our tests, the zero vector is the initial guess and the stopping criterion is r q 2 < ;6 r 2 where r q is the residual after the q-th iteration. Example. onsider < where and L n = n : y (t) =L n y (t ; ) + M n yt + N n y(t ; ) y(t) =( ) T N n = n M n = t t ; ; 2 ; ; ; : ; 2 Example is solved by using the fth order generalized dams method for t 2 [ 4]. Table lists the number of iterations required for convergence of the GMRES method for dierent n and k. In the table, I means no preconditioner is used and S denotes the Strang-type preconditioner dened in (3). We see that the number of iterations required for convergence, when a circulant preconditioner is used, is always less than that when no preconditioner is used. We should emphasize that our numerical example shows a much faster convergence rate than that predicted by the estimate provided by Theorem 2. 4 Dierential Equations with Multi-delays onsider the solution of dierential equations with multi-delays: >< >: dy(t) dt y(t) =(t) = J n y(t)+d n () y(t ; )++ D n (s) y(t ; s )+f(t) t t where y(t), f(t), (t) : R! R n J n, D n () D n (s) numbers. t t (6) 2 R nn and s > are some rational

9 n k I S * 7 n k I S * 6 Table : Number of iterations for convergence (`' means out of memory). 4. VMs and Strang-type Preconditioner s in x3., we want to nd an asymptotically stable solution for (6). We have the following lemma, see [5, 9]. Lemma 3 For any s, if (J n ) 2 max(j n + J T n ) < and (J n )+ then solution of (6) is asymptotically stable. s = D () n 2 < (7) In the following, for simplicity, we only consider the case of s = 2 in (6). The generalization to arbitrary s is straightforward. Let h = =m = 2 =m 2 be the step size where m and m 2 are positive integers with m 2 >m ( 2 > ). For (6), by using a VM with ( 2 )-boundary conditions over a uniform mesh t = t + h = r 2 on the interval [t t + r 2 h], we have i y p+i; = h i (J n y p+i; + D () n y p+i; ;m + D (2) n y p+i; ;m 2 + f p+i; ) () for p = ::: r 2 ;, where = + 2, and f i g, f ig VM. y providing the values are the coecients of the given y ;m2 ::: y ;m ::: y y ::: y ; y r2 ::: y r2 + 2 ; (9) () can be written in a matrix form as where Ry = b (2) R I n ; h J n ; h () D () n ; h(2) D (2) n (2) 9

10 y T =(y T y T + ::: y T r 2 ;) 2 R n(r 2 ; ) and b 2 R n(r 2 ; ) depends on f, the boundary values, and the coecients of the method. The matrices, 2 R (r 2 ; )(r 2 ; ) are dened as in (5) and (), (2) 2 R (r 2 ; )(r 2 ; ) in (2) are dened as the matrix () in (2). We remark that the rst column of () is given by ( ::: ::: m + ; and the rst column of (2) is given by ( ::: ::: m 2 + ; ::: r 2 ;m ;2 ; ::: r 2 ;m 2 ;2 ; The Strang-type preconditioner for (2) is dened as follows: ~S s() I n ; hs() J n ; hs( () ) D () n ; hs((2) ) D (2) n (22) where s(e) is Strang's circulant preconditioner of matrix E, fore =,, () and (2). We have the following theorem for the invertibility of our preconditioner ~ S and for the convergence rate of our method. Theorem 3 [] If the VM for (6) is 2 -stable and (7) holds, the Strang-type preconditioner ~ S dened as in (22) is invertible. Moreover, when the GMRES method is applied to solving the preconditioned system ~S ; Ry = ~ S ; b the methods will converge in at most (2+m +m )n = O(n) iterations in exact arithmetic. We know from Theorem 3 that if the step size h = =m = 2 =m 2 is xed, the number of iterations for convergence of the GMRES method, when applied to the preconditioned system ~S ; Ry = ~ S ; b, will be independent ofr 2 and therefore is independent of the length of the interval that we considered. For the operation cost of our algorithm, we refer to [, 4]. 4.2 Numerical Test We illustrate the eciency of our preconditioner by solving the following problem. In the example, the VM we used is the third order GDF for t 2 [ 4]. ) T ) T : Example 2. onsider where J n = < : y (t) =J n y(t)+d () n y(t ; :5) + D (2) n y(t ; ) t y(t) =(sint ::: ) T ; ; D () n = n t 2 ; ; ; ; 2

11 and D (2) n = n Table 2 shows the number of iterations required for convergence of the GMRES method with dierent combinations of matrix sizes n and u = =h. In the table, I means no preconditioner is used and ~ S denotes the Strang{type preconditioner dened in (22). We see that the numbers of iterations required for convergence increase slowly for increasing n and u under the column ~ S. : n u I S ~ n u I S ~ Table 2: Number of iterations for convergence. 5 Singular Perturbation Delay Dierential Equations In this section, we study the solution of singular perturbation delay dierential equations: >< >: x (t) =V () x(t)+v (2) x(t ; )+ () y(t)+ (2) y(t ; ) t t y (t) =F () x(t)+f (2) x(t ; )+G () y(t)+g (2) y(t ; ) t t < x(t) =(t) t t y(t) = (t) t t (23) where x(t), (t) : R! R m y(t), (t) : R! R n V (), V (2) 2 R mm (), (2) 2 R mn F (), F (2) 2 R nm G (), G (2) 2 R nn and > is a constant. We can rewrite (23) as the following IVP, >< >: z (t) =P z(t)+qz(t ; ) (t) z(t) = (t) t t t t where P and Q 2 R (m+n)(m+n) are dened as follows, V () P = () V (2) ; F () ; G () Q = (2) ; F (2) ; G (2) : (24) see [2]. Let h = =k 2

12 be the step size where k 2 is a positive integer. For (24), by using a VM with ( 2 )-boundary conditions over a uniform mesh t = t + h = v on the interval [t t + vh], we have i z p+i; = h i (P z p+i; + Qz p+i; ;k 2 ) (25) for p = ::: v;, where = + 2, and f i g, f ig see [5]. y providing the values are coecients of the given VM, (25) can be written in a matrix form as where The vector v in (27) is dened by z ;k2 ::: z z ::: z ; z v ::: z v+2 ; (26) Kv = b (27) K I m+n ; h P ; hu Q: (2) v T =(z T z T + ::: z T v;) 2 R (m+n)(v; ) : The right-hand side b 2 R (m+n)(v; ) of (27) depends on the boundary values and the coecients of the method. The matrices, 2 R (v; )(v; ) in (2) are dened as in (5) and U 2 R (v; )(v; ) in (2) is dened as the matrix () given by (2), see [5, 4, 6, ]. The Strang-type preconditioner can be constructed for solving (27): ^S s() I m+n ; hs() P ; hs(u) Q (29) where s(e) is Strang's circulant preconditioner of matrix E, for E =,, and U. We have the following theorem for the invertibility ofour preconditioner ^S and for the convergence rate of our method. Theorem 4 [2] If the VM with ( 2 )-boundary conditions is 2 -stable and the conditions in Lemma 3 hold, i.e., (P ) 2 max(p + P T ) < and (P )+Q 2 <, then the Strang-type preconditioner ^S dened by (29) is invertible. Moreover, when the GMRES method is applied to solving the preconditioned system ^S ; Kv = ^S; b the method will converge in at most O(m + n) iterations in exact arithmetic. We observe from Theorem 4 that if the step size h = =k 2 is xed, the number of iterations for convergence of the GMRES method applied to the system ^S; Kv = ^S; b will be independent of v, i.e., it is independent of the length of the interval we considered. For numerical examples, we refer to [2]. 2

13 References [] P. modio, F. Mazzia, and D. Trigiante, Stability of Some oundary Value Methods for the Solution of Initial Value Problems, IT, vol. 33 (993), pp. 434{45. [2] Z. ai,. Jin, and L. Song, Strang-type Preconditioners for Solving Linear Systems from Neutral Delay Dierential Equations, submited to alcolo. [3] L. rugnano and D. Trigiante, Stability Properties of Some VM Methods, ppl. Numer. Math., vol. 3 (993), pp. 29{34. [4] D. ertaccini, irculant Preconditioner for the Systems of LMF-ased ODE odes, SIM J. Sci. omput., vol. 22 (2), pp.767{76. [5] L. rugnano and D. Trigiante, Solving Dierential Problems by Multistep Initial and oundary Value Methods, Gordon and erach Science Publishers, msterdam, 99. [6] R. han,. Jin, and Y. Tam, Strang-type Preconditioners for Solving System of ODEs by oundary Value Methods, Electron. J. Math. Phys. Sci., vol. (22), pp. 4{46. [7] R. han and M. Ng, onugate Gradient Methods for Toeplitz Systems, SIM Review, vol. 3 (996), pp. 427{42. [] R. han, M. Ng, and. Jin, Strang-type Preconditioners for Systems of LMF-ased ODE odes, IM J. Numer. nal., vol. 2 (2), pp. 45{462. [9] G. Hu and T. Mitsui, Stability nalysis of Numerical Methods for Systems of Neutral Delay- Dierential Equations, IT, vol. 35 (995), pp. 54{55. []. Jin, Developments and pplications of lock Toeplitz Iterative Solvers, Kluwer cademic Publishers & Science Press, eiing, 22. []. Jin, S. Lei, and Y. Wei, irculant Preconditioners for Solving Dierential Equations with Multi-Delays, submited to omput. Math. ppl.. [2]. Jin, S. Lei, and Y. Wei, irculant Preconditioners for Solving Singular Perturbation Delay Dierential Equations. submited to Numer. Linear lgebra.. [3] J. Kuang, J. iang, and H. Tian The symptotic Stability of One Parameter Methods for Neutral Dierential Equations, IT, vol. 34 (994), pp. 4{4. [4] F. Lin,. Jin, and S. Lei, Strang-type Preconditioners for Solving Linear Systems form Delay Dierential Equations, IT, to appear. [5] T. Mori, N. Fukuma, and M. Kuwahara, Simple Stability riteria for Single and omposite Linear Systems with Time Delays, Int. J. ontrol, vol. 34 (9), pp. 75{4. [6] T. Mori, E. Noldus, and M. Kuwahara, Way to Stabilize Linear Systems with Delayed State, utomatica, vol. 9 (93), pp. 57{573. [7] Y. Saad and M. Schultz, GMRES: a Generalized Minimal Residual lgorithm for Solving Non-Symmetric Linear Systems, SIM J. Sci. Stat. omput., vol. 7 (996), pp. 56{69. 3

14 [] S. Vandewalle and R. Piessens, On Dynamic Iteration Methods for Solving Time-Periodic Dierential Equations, SIM J. Num. nal., vol. 3 (993), pp. 26{33. [9] S. Wang, Further Results on Stability of _(t) =(t) +(t ; ), Syst. ont. Letters, vol. 9 (992), pp. 65{6. 4