The Best Circulant Preconditioners for Hermitian Toeplitz Systems II: The Multiple-ero Case Raymond H. Chan Michael K. Ng y Andy M. Yip z Abstract In [0, 4], circulant-type preconditioners have been proposed for ill-conditioned Hermitian Toeplitz systems that are generated by nonnegative continuous functions with a zero of even order. The proposed circulant preconditioners can be constructed without requiring explicit knowledge of the generating functions. It was shown that the spectra of the preconditioned matrices are uniformly bounded except for a xed number of outliers and that all eigenvalues are uniformly bounded away from zero. Therefore the conjugate gradient method converges linearly when applied to solving the circulant preconditioned systems. In [0, 4], it was mentioned that this result can be extended to the case where the generating functions have multiple zeros. The main aim of this paper is to give a complete convergence proof for this class of generating functions. Key Words. Toeplitz systems, circulant preconditioners, kernel functions, the preconditioned conjugate gradient method AMS(MOS) Subject Classications. 65F0, 65F5, 65T0 Introduction An n-by-n matrix A n with entries a ij is said to be Toeplitz if a ij a ij for all i; j. Toeplitz systems of the form A n x b occur in a variety of applications in mathematics and engineering [8]. In this paper, we consider the solutions of Hermitian positive denite Toeplitz systems. There are a number of specialized fast direct methods for solving such systems in O(n ) operations, see for instance [9]. Faster methods requiring O(n log n) operations have also been developed, see []. E-mail: rchan@math.cuhk.edu.hk. Department of Mathematics, The Chinese University of Hong Kong, Shatin, Hong Kong. Research supported in part by Hong Kong Research Grants Council Grant No. CUHK 4/99P and CUHK DAG Grant No. 0604. y E-mail: mng@maths.hku.hk. Department of Mathematics, The University of Hong Kong, Pokfulam Road, Hong Kong. Research supported in part by HKU 747/99P and HKU CRCG Grant No. 0070. z E-mail: mhyipa@hkusua.hku.hk. Department of Mathematics, The University of Hong Kong, Pokfulam Road, Hong Kong.
Strang in [8] proposed using the preconditioned conjugate gradient method with circulant matrices as preconditioners for solving Toeplitz systems. The number of operations per iteration is of order O(n log n) as circulant systems can be solved eciently by fast Fourier transforms. Several successful circulant preconditioners have been introduced and analyzed; see for instance [, 6]. In these papers, the given Toeplitz matrix A n is assumed to be generated by a generating function f, i.e., the diagonals a j of A n are given by the Fourier coecients of f. It was shown that if f is a positive function in the Wiener class (i.e., the Fourier coecients of f are absolutely summable), then the corresponding circulant preconditioned systems converge superlinearly [6]. However, if f has zeros, the Toeplitz systems will be ill-conditioned. In fact, for the Toeplitz matrices generated by a nonnegative function having multiple zeros with maximal order p, their condition numbers grow like O(n p ), see [7, 6]. Hence the number of iterations required for convergence will increase like O(n p ), see [, p.4]. Tyrtyshnikov [0] has proved that the Strang [8] and the T. Chan [] preconditioners both fail in this case. To tackle this problem, non-circulant type preconditioners have been proposed, see [5, 7, 5]. The basic idea behind these preconditioners is to nd a function g that matches the zeros of f. Then the preconditioners are constructed based on this function g. These approaches work when the generating function f is given explicitly. However, when we are given only a nite n-by-n Toeplitz system, i.e., only fa j g jjj<n are given and the underlying f is unknown, then these preconditioners cannot be constructed. Recently, Chan, Yip and Ng [0] and Potts and Steidl [4] have independently proposed circulant-type preconditioners for these ill-conditioned Toeplitz systems. In [0], we construct the preconditioners by approximating f with the convolution product K m;r f that matches the zeros of f, where K m;r is chosen to be the generalized Jackson kernels, see []. We have proved that if f has a zero of order p, then K m;r f matches the zero of f when r > p. Using this result, we showed that the spectra of the circulant preconditioned matrices are uniformly bounded except for a xed number of outliers, and that their smallest eigenvalues are bounded uniformly away from zero. The construction of the preconditioners only requires the entries of A n and does not require explicit knowledge of f. In the numerical examples of [0], it has been shown that these circulant preconditioners are also eective for Toeplitz matrices that are generated by functions with multiple zeros. It was mentioned in both papers [0, 4] that this result can be extended to the case where the generating functions have multiple zeros of even order. The main aim of this paper is to give the convergence proof for this class of generating functions. In particular, we will prove that if f has s zeros with maximal order p, then K m;r f matches the zero of f when r > p. Using this result, we show that the spectra of the circulant preconditioned matrices are uniformly bounded except for a xed number of outliers, and that their smallest eigenvalues are bounded uniformly away from zero. It follows that the conjugate gradient method, when applied to solving these circulant preconditioned systems, will converge linearly. Since the cost per iteration is O(n log n) operations, see [8], the total complexity of solving these ill-conditioned Toeplitz systems is of O(n log n) operations. In contrast, the non-preconditioned systems will have condition number growing like O(n p )
and hence the complexity for solving the systems will be of order O(n p+ log n). The outline of the paper is as follows. In x, we introduce our circulant preconditioners from the generalized Jackson kernel K m;r. Some preliminary results on the approximation properties of K m;r are given in x. In x4, we show that K m;r f matches the zeros of f. In x5, we analyze the spectra of the circulant preconditioned matrices and their convergence rates. Finally, numerical examples are given in x6. Preconditioners from Generalized Jackson Kernel Let C be the space of all -periodic continuous real-valued functions. The Fourier coecients of a function f in C are given by a k f()e ik d; k 0; ; ; : Clearly a k a k for all k. Let A n [f] be the n-by-n Hermitian Toeplitz matrix with the (i; j)th entry given by a ij, i; j 0; : : : ; n. We will use C + to denote the space of all nonnegative functions in C which are not identically zero. We remark that the Toeplitz matrices A n [f] generated by f C + are positive denite for all n, see [7, Lemma ]. Conversely, if f C takes both positive and negative values, then A n [f] will be nondenite for large n. In this paper, we consider f C +. We say that 0 is a zero of f of order p if f( 0 ) 0 and p is the smallest positive integer such that f (p) ( 0 ) 6 0 and f (p+) () is continuous in a neighborhood of 0. By Taylor's theorem, f() f (p) ( 0 ) ( 0 ) p + O(( 0 ) p+ ) p! for all in that neighborhood. Since f is nonnegative, f (p) ( 0 ) > 0 and p must be even. In this paper, we consider f having zeros of order p j at j [; ] for j s. Such f can be written in the general form f() j ( j ) pj h(); () where h() is a positive function in C. We remark that the condition number of A n [f] generated by such an f grows like O(n pmax ) where p max max j p j, see [6]. The systems A n [f]x b will be solved by the preconditioned conjugate gradient method with circulant preconditioners. In the following, we will use the generalized Jackson kernel functions K m;r () k m;r m r! sin( m ) r sin( ) ; r ; ; : : : ()
to construct our circulant preconditioners. Here k m;r is a normalization constant such that It is known that 8 K m;r ()d : r k r m;r ; () 4 see [, p.57] or [0]. We note that K m; () is the Fejer kernel and K m;4 () is the Jackson kernel [, p.57]. For any m >, the kernel K m;r () can be expressed as K m;r () r(m) X kr(m) b (m;r) k e ik ; where the coecients b (m;r) k can be obtained by convolving the vector m ; ; m m ; ; m m ; ; m with itself for r times and this can be done by fast Fourier transforms, see [7, pp.94{ 96]. Thus the cost of computing the coecients fb (m;r) k g for all jkj r(m ) is of order O(rm log m) operations. In order to guarantee that K m;r ()d ; we normalize b (m;r) 0 to () by dividing all coecients b (m;r) k by b (m;r) 0. For a given function g, we dene the circulant preconditioner C n [g] to be the n-by-n circulant matrix with its j-th eigenvalue given by j j (C n [g]) g ; 0 j < n: (4) n We note that C n [g] Fndiag( 0 ; ; : : : ; n )F n, where F n is the n-by-n discrete Fourier matrix and F n is the conjugate transpose of F n, see [8]. Hence the matrix-vector multiplication C n [g]v, which is required in each iteration of the preconditioned conjugate gradient method, can be done in O(n log n) operations by fast Fourier transforms, see [8]. Clearly by (4), if g is a positive function, then C n [g] is positive denite for all n. For a given n-by-n Toeplitz matrix A n [f], our proposed circulant preconditioner is C n [K m;r f], where m dnre, i.e., r(m ) < n rm: (5) 4
Since f P k a ke ik, the convolution product of K m;r f is given by where (K m;r f)() d k ( r(m) X kr(m) a k b (m;r) k e ik n X kn+ a k b (m;r) k ; jkj r(m ); 0; otherwise: d k e ik ; Thus K m;r f depends only on a k for jkj < n, i.e., only on the entries of the given n-by-n Toeplitz matrix A n [f]. Notice that the cost of constructing C n [K m;r f] is of O(n log n) operations, see [0]. We rst note that since K m;r is a positive kernel, the preconditioners are all positive denite. Lemma. [0, Lemma.] Let f C +. The preconditioner C n[k m;r f] is positive denite for all positive integers m, n and r. Preliminary Lemmas In this section, we derive two results on the approximation properties of K m;r that will be required in the subsequent sections. In the following, we will use to denote the function dened on the whole real line R. For clarity, we will use to denote the periodic extension of dened on [; ], i.e. () ~ if ~ (mod ) and ~ [; ]. It is clear that p C+ We rst note that K m;r p matches the order of the zero of p Lemma. [0, Lemma.] Let p and r be positive integers. If r > p, then K m;r p (0) K m;r (t)t p dt c p;r m p ; where p p + for any integer p. at 0 if r > p. 4r 4r c p;r p+ : (6) However, if r p, then the order of the zeros of K m;r p and p may not be equal. Lemma. If r p, then where K m;r (t)t p dt c p;r m r ; p 4r m rp p r+p+ c p;r : (7) p + p r + 5
Proof: Since sin() on [0; ], we see from () that K m;r (t)t p dt r+ k m;r m r 0 sin r mt t rp dt p+ k m;r sin r u m p du 0 urp p+ k m;r r m p u p du 0 p+ k m;r r (p + )m p : In view of (), we have the rst inequality of (7). On the other hand, we also have K m;r (t)t p dt r k m;r m r pr+ r k m;r m p 0 p+ k m;r m p sin r mt t rp m 0 r m p+ k m;r m p (p r + ) Now the second inequality of (7) follows from (). 0 dt sin r u u u rp du rp du r m pr+ : Combining Lemmas. and., and noting that K m;r (t) is an even function whereas t j is an odd function for odd j, we have the following corollary. Corollary. For all positive integers j, m and r, we have In the next section, we have to estimate The following lemma will come in handy. c j;r K m;r (t)t j dt ; (8) mminfj;rg where c j;r 0 for odd j and the bounds for c j;r when j is even are given by (6) and (7). i () for [; ]. h Q s K m;r j ( j) p j Lemma.4 Let fp j g s j be positive integers and p P s ( ) p h K m;r Q s j p j. Then i j ( j) p j (( j ) () ) p j () (5 ) p ; (9) " K m;r ( j ) p j j 6 # ()
for all, j [; ]. In particular, if j k j, then " ( ) ppk for all, k [; ]. K m;r ( k ) p k " j;j6k ( j ) p j (( j ) () ) p j K m;r ( j ) p j j # () # () (5 ) p ; (0) Proof: For simplicity, let us write j(x) x p j (x ) p j (x + ) p j x p j (x () ) pj : () We rst claim that for each j it holds that ( ) pj j ( t j ) ( t j ) p j Let j j [; ], () is equivalent to ( ) pj j ( j t) ( j t) p j (5 ) pj ; 8t; ; j [; ]: () (5 ) pj ; 8t [; ]; j [; ]: () We prove () only for the case j [; 0]. The other case with j [0; ] can be proved similarly. By the denition of ( j t) p j, for any j [; 0], we have j( j t) (j t ) p j ( j t + ) p j ; t [; j + ]; ( j t) p j ( j t ) p j ( j t) p j ; t [ j + ; ]: For t [; j + ] and j [; 0], we have j t. Hence by considering the maximum and minimum of j(x )(x + )j on [; ], we obtain ( ) pj ( j t ) p j ( j t + ) pj (4 ) pj : For t [ j + ; ] and j [; 0], we have j t. Hence by considering the graph of jx(x )j on [; ], we get ( ) pj ( j t ) p j ( j t) pj (5 ) pj : Thus we have () and hence () too. By (), we see that ( ) p j j j( t j ) ( t j ) p j (5 ) p ; 7 8t; ; j [; ];
P s where p j p j. Hence we have 4 Km;r j ( j ) p j 5 () K m;r (t) ( ) p j ( t j ) p j K m;r (t) 4 Km;r ( ) p j j dt j( t j )dt j( j ) 5 (): Similarly, we obtain 4 Km;r j ( j ) p j5 () 4 Km;r (5 ) p j j( j ) 5 (): Hence (9) follows. To prove (0), we rst note that for j k tj, we have For j k tj, we have which implies that ( k t) p k ( k t) p k : (4) ( k t) p k (j k tj ) p k ; ( k t) p k ( k t) p pk k j k tj Together with (4), we have, for all j k j j k j, ( k t) p k ( k t) p k pk (5 ) p k ; Combining this with (), we see that for all j k j, pk : 8t [; ]: ( ) ppk ( t k ) p k Q s j;j6k j( t j ) j ( t j ) p j (5 ) p ; 8t [; ]: By using these two inequalities and the same arguments in establishing (9) above, we easily get (0). 8
4 Approximation by Using K m;r In this section, we use the results in x to show that for f given in (), the zeros of K m;r f match the order of the zeros of f and that the two functions are essentially the same away from the zeros of f. For simplicity, we divide the interval [; ] into subintervals according to the location of the zeros f j g s j of f. Let > 0 and L j (j ; j + ); if j 6 ; [; + ) [ ( ; ]; if j ; for j s. Note that there exists such that all the intervals L j for j s are disjoint and contained in [; ]. Clearly is less than half of the minimum gaps between the zeros. Dene, for j s and n >, I j;n [j n ; j + n ]; if j 6 ; [; + ] n [ [ n ; ]; if j ; J j;n L j n I j;n (j ; j n ) [ ( j + n ; j + ); if j 6 ; ( + n ; + ) [ ( ; n ); if j ; L [; ] n [ s j L j: In summary, the interval [; ] is divided in such a way that (a) I j;n is an arbitrary small interval containing the zero j ; (b) L j is a xed interval containing I j;n ; (c) L is a xed closed interval in which f takes positive values. Q s 4. The Function j ( j) p j on [s j J j;n Q s In this subsection, we show that K m;r j ( j) p j the same on [ s j J j;n. Q and s j ( j) p j are essentially Lemma 4. Let fp j g s j and r be positive integers with r > max j p j. Let m dnre. Then there exist positive numbers and independent of n such that for all suciently large n, " K m;r ( j ) p j j j ( j ) p j # () ; 8 [ s j J j;n: (5) 9
Proof: Without loss of generality, it suces to prove only for J ;n. In this case, by the construction of J ;n, we have n < j j and j j j for j s. In view of (0), it suces to show that 4 Km;r ( ) p j Q s j ( j) p j j( j ) 5 () ; 8 [ s j J j;n; (6) for some positive constants and. Here j () is dened in (). We begin with the upper bound. For, t, j [; ], we have j t j j. Hence by considering the graph of jx(x 4 )j on [; ], we have a bound for j : j( t j ) ( t j ) p j ( t j ) p j ( t j + ) pj (5 ) pj : Let p P s j p j. We rst note that 4 Km;r ( ) p (5 ) pp j K m;r (t)( t ) p K m;r (t)( t ) p K m;r (t) j( j ) 5 () j j p X p j0 j( t j ) dt (5 ) pj dt j ( ) p j (t) j dt: R Since K m;r(t)t j dt 0 for odd j and j j j < for j s, we have 4 Km;r ( ) p (5 ) pp Q s j ( j) p j 5 5 j Q s j ( j) p j pp pp j( j ) 5 () K m;r (t) K m;r (t) K m;r (t) 0 p X p j0 p p X j j0 p X p j0 j j ( ) p j t j dt ( ) j t j dt n j t j dt
5 pp p X j0 p j j n K m;r (t)t j dt: (7) Since r > max j p j p, by Lemma. and (5), j K m;r (t)t j dt n r j cj;r for each j 0; : : : ; p. Hence (7) is bounded above by a constant independent of n. Therefore we get an upper bound for (6). Next we establish a lower bound for (6). We have 4 Km;r ( ) p j K m;r (t)( t ) p K m;r (t) 4 ( ) p j( j ) 5 () j j j( t j ) dt j( j ) + u(t) 5 dt where u(t) ( t ) p j j( t j ) ( ) p j j( j ) 6p4p X j u j t j is a degree 6p 4p polynomial without the constant term. By (8), we have Thus 4 Km;r ( ) p j K m;r (t)u(t)dt pp X j u j j( j ) 5 () ( ) p c j;r m minfj;rg : j pp X u j c j;r j( j ) + m minfj;rg : Since j j and j j j for j 6, we have ( ) p ( ) p and pj ( j ) p j < ()p j for j 6. Hence, we have " K m;r ( ) p j j ( j ) p j j( j ) # () j
Q s j j( j ) Q s j ( j) p j ( ) pp () pp () pp p pp X p pp X j j ju j jc j;r m minfj;rg ju j jc j;r ; (8) mminfj;rg where the last inequality follows from () with t 0 there. Clearly, the second term of (8) tends to zero as m tends to innity. Thus for suciently large m (and hence large n), (8) is bounded uniformly from below by a positive constant. We therefore have established a lower bound for (6). We remark that if the gaps between j are small, then is small. Hence the upper bound in (7) (and therefore in (5)) will be large. Q s 4. The Function j ( j) p j on L Q s In this subsection, we show that K m;r j ( j) p j the same on L. Q and s j ( j) p j are essentially Lemma 4. Let fp j g s j, r be positive integers and m dnre. Then there exist positive numbers and independent of n such that for all suciently large n, " K m;r ( j ) p j j j ( j ) p j # () ; 8 L: Proof: First note that by the construction of L, there exist positive constants 0 and 0 such that 0 < 0 j ( j ) p j 0; 8 L: Let be a positive number less than 0. By using the fact that K m;r is a summation kernel [] with the property: lim m! kk m;r f f k 0; 8f C + ; where k k denotes the supremum norm, there exists a positive integer m 0 such that for m m 0 4 Km;r j ( j ) p j 5 () j ( j ) p j ; 8 L:
It follows that for all suciently large n (and hence m) 0 4 Km;r j j ( j ) p j5 () ( j ) p j + 0 ; 8 L: 4. Functions in C + on [s j J j;n [ L Now we extend the results of Lemmas 4. and 4. to any functions in C + zeros of even order. with multiple Lemma 4. Let f C + and have s zeros of order p j at j [; ] for j s. Let r > max j p j be any integer and m dnre. Then there exist positive numbers and, independent of n, such that for all suciently large n, (K m;r f) () f() ; 8 [ s j J j;n [ L: (9) Proof: Q s By denition (), f() g() dened on [; ]. Write j ( j) p j g() for some positive continuous function (K m;r f) () f() h K m;r Q s h Q s K m;r j ( j) p j i j ( j) p j g() () i () h Q i s K m;r j ( j) p j () Q s j ( j) p j g() : (0) Clearly the last factor is uniformly bounded above and below by positive constants. By Lemmas 4. and 4., the same holds for the second factor when [ s j J j;n [ L. As for the rst factor, by the mean value theorem for integrals [], there exists a [; ] such that Hence [K m;r 0 < g min j ( j ) p j g()]() g()[k m;r h K m;r Q s h Q s K m;r j ( j) p j j i j ( j) p j g() () i g max ; () ( j ) p j ](): 8 [; ];
where g min and g max are the minimum and maximum of g respectively. Thus the theorem follows. We remark that if the gaps between the zeros are small, then the upper bound of the second factor in (0) will be large (see the remark at the end of x4.). Hence in (9) will be large. 4.4 Functions in C + on [s j I j;n So far we have considered only the intervals [ s j J j;n [ L, i.e., away from the zeros of f. For [ s j I j;n, i.e., around the zeros of f, we now show that the convolution product K m;r f matches the order of the zeros of f. Lemma 4.4 Let f C + and have s zeros of order p j at j [; ] for j s. Let r > max j p j be any integer and m dnre. If I k;n for some k s, then we have (K m;r f) () O : Proof: We prove only for the case I ;n. The cases for I j;n for j s can be proved similarly. In this case we have j j n, Q j j j s and j j j 4 for j n. Since by (), f() j ( j) p j g() for some positive continuous function g() dened on [; ], by the mean value theorem for integrals again, we have g min 4 Km;r j n p k ( j ) p j5 () (Km;r f)() Hence, together with (0), it suces to show that 4 Km;r ( ) p j j( j ) 4 Km;r j g max 4 Km;r 5 () O ( j ) p j g() 5 () n p j ( j ) p j5 (): ; 8 I ;n : () We note that 4 Km;r ( ) p j j( j ) 5 () 4
j K m;r (t)( t ) p K m;r (t)( t ) p j Y 4 s j( t j )dt j( j ) + v(t) 5 dt j j( j )[K m;r ( ) p ]() K m;r (t)( t ) p v(t)dt; () where v(t) j j( t j ) j j( j ) 6p6p X is a degree 6p 6p polynomial without the constant term. For the rst term of (), since j j j and j j j 4 for j n, we have 6p6p j j v j t j j( j ) ( ) pp ; 8 I ;n : Moreover, by [0, Theorem.7] and the fact that r > max j p j p, we also have [K m;r ( ) p ]() O : Thus j n p j( j )[K m;r ( ) p ]() O For the second term of (), since j j n, we have p X j0 p X X j0 K m;r (t)( t ) p v(t)dt K m;r (t) 6p6p X k 6p6p k jv k j p j n p p X 6p6p p X ( ) pj t j v k t k dt j j0 k p v k ( ) p j K m;r (t)t j+k dt j j j p j K m;r (t)t j+k dt : () 5
p X j0 X 6p6p X k p 6p6p j0 X k jv k j p j jv k j p j p j n n p j K m;r (t)t j+k dt : c j+k;r m minfj+k;rg ; where the last equality follows from (8). Thus, we have p K m;r (t)( t ) p v(t)dt X O j0 n p j Putting this together with () into (), we get (). O m j+ O n p : + 5 Spectral Properties of the Preconditioned Matrices With the results in the previous sections, we now analyze the spectra and convergence rates of the preconditioned systems when the generating function has multiple zeros. We begin with the following lemma. Lemma 5. [5, ] Let f C +. Then A n[f] is positive denite for all n. Moreover if g C + is such that 0 < fg for some constants and, then for all n, x A n [f]x x A n [g]x ; 8x 6 0: With that, we have our rst main theorem which states that the spectra of the preconditioned matrices are essentially bounded. Theorem 5. Let f C + and have s zeros of order p j at j for j s. Let r > max j p j, m dnre, and p P s j p j. Then there exist positive numbers ; ( ), independent of n, such that for all suciently large n, at most p + s eigenvalues of C n [K m;r f]a n [f] are outside the interval [; ]. Proof: For any function g C, we let Cn ~ [g] to be the n-by-n circulant matrix with the j-th eigenvalue given by j ( ~ Cn [g]) 8 >< >: n p ; k j g n if j n ; otherwise; k < n for some k ; ; s for j 0; : : : ; n. Since, for large n, there is at most one j such that jjn k j < n for each k s, by (4), ~ Cn [g] C n [g] is a matrix of rank at most s. 6 (4)
Q s By assumption (), f() j sinp j (( j ))g() for some positive function g in C. We use the following decomposition of the Rayleigh quotient to prove the theorem: x A n [f]x x C n [K m;r f]x x A n [f]x h x Qs A n j sinp j j x ~ Cn h Qs j sinp j j x ~ C n [f]x h x i Qs A n j sinp j j i x i x x i x x ~ C n h Qs j sinp j x ~ Cn [f]x x ~ C n [K m;r f]x j x ~ Cn [K m;r f]x x C n [K m;r f]x : (5) We remark that by Lemma 5. and the denitions (4) and (4), all matrices in the factors in the right hand side of (5) are positive denite. As g is a positive function in C, by Lemma 5., the rst factor in the right hand side of (5) is uniformly bounded above and below. Similarly, by denition (4), the third factor is also uniformly bounded. The eigenvalues of the two circulant matrices in the fourth factor dier only when jjn k j n for all k s. But by Lemma 4., the ratios of these eigenvalues are all uniformly bounded when n is large. The eigenvalues of the two circulant matrices in the last factor dier only when jjn k j < n for some k s. But by Lemma 4.4, their Q ratios are also uniformly bounded. Regarding the second factor, Q s since A n [ j sinp j ( j )] is a banded Toeplitz matrix with Q s half bandwidth p +, C n [ j sinp j ( j )] is just the Strang preconditioner for s A n [ j sinp j ( j )] when n > p, see [9]. Thus the two matrices dier only by a rank p matrix. It follows that the matrix A n [ Q s j sinp j ( j )] ~ Cn [ Q s j sinp j ( j )] is of rank at most p + s. The rest of the proof now follows exactly the same line of the proof in [0, Theorem 4.]. In view of the remark at the end of x4., we see that if the gaps between the zeros are small, then the lower bound for the fourth factor in (5) will be small. Hence the condition numbers of our preconditioned matrices, althought independent of n, will be large. We will observe this in the numerical examples in x6. Finally, by using exactly the same argument used in [0, Theorem 4.], we show that all the eigenvalues of the preconditioned matrices are uniformly bounded away from zero. More precisely, we have Theorem 5. Let f C + and have s zeros of order p j at j for j s. Let r > max j p j and m dnre. Then there exists a positive constant c independent of n, such that for all n suciently large, all eigenvalues of the preconditioned matrix C n [K m;r f]a n [f] are larger than c. By combining Theorems 5. and 5., the number of preconditioned conjugate gradient (PCG) iterations required for convergence is of O(), see [4]. We recall that each PCG 7
iteration requires O(n log n) operations (see [8]) and so is the construction of the preconditioner (see x). Thus the total complexity of the PCG method for solving an n-by-n Toeplitz systems generated by f C + with multiple zeros of even order is of O(n log n) operations. In contrast, for the original matrix A n [f], since its condition number is growing like O(n pmax ), where p max max j p j, the complexity of solving the corresponding system will be of order O(n pmax+ log n). 6 Numerical Experiments In [0], we have already illustrated by numerical examples the eectiveness of the preconditioner C n [K m;r f] in solving Toeplitz systems. In particular, we have tested Toeplitz systems generated by functions having multiple zeros, and found that the circulant preconditioned system converge linearly, i.e., the number of iterations required for convergence is independent of n. This is in line with the theoretical results we have proven in x5 above. In this section, we further consider the function f (x d) (x + d) for dierent values of d. By the remark after Theorem 5., d will aect the condition number of the preconditioned matrices. In the test, m was set to dnre. The right-hand side vectors b were formed by multiplying random vectors to A n [f]. The initial guess is the zero vector and the stopping criteria is jjr q jj jjr 0 jj 0 7 where r q is the residual vector after q iterations. We also compared the performance of our preconditioners with the Strang [8] and the T. Chan [] circulant preconditioners. Tables and show the numbers of iterations required for convergence for dierent choices of preconditioners. In the tables, I denotes no preconditioner, S is the Strang preconditioner [8], K m;r are the preconditioners from the generalized Jackson kernel K m;r dened in () and T K m; is the T. Chan preconditioner [9]. Iteration numbers more than 6,000 are denoted by \y". We note that S in general is not positive denite as the Dirichlet kernel is not positive, see [9]. When some of its eigenvalues are negative, we denote the iteration number by \{" as the PCG method does not apply to non-denite systems and the solution thus obtained may be inaccurate. The test function in Table is a nonnegative function with two well separated zeros of order on [; ]. Thus the condition number of the Toeplitz matrices is O(n ) and hence the number of iterations required for convergence without using any preconditioners is increasing like O(n). We see that the T. Chan preconditioner does not work. This is to be expected from the fact that the order of K m; does not match that of at 0, see (7). However, we see that K m;4, K m;6 and K m;8 all work very well as predicted from our convergence analysis in x5. The zeros of the test function in Table are close together and hence the function behaves like 4 since ( 0:00) ( + 0:00) 4. Thus the eective condition numbers of the Toeplitz matrices is like O(n 4 ) and the systems are very ill-conditioned even for moderate n. As in the rst example, the T. Chan preconditioner does not work. However, K m;6 and K m;8 do work very well. This can also be explained by our Theorems 5. 8
( ) ( + ) n 64 8 56 5 04 I 5 4 9 547 S 0 { 9 0 8 T 8 4 0 7 6 46 K N;4 4 K N;6 4 4 K N;8 5 5 4 Table : Numbers of iterations required for dierent preconditioners. and 5.. Since the separation d in this example is small, the condition numbers of our preconditioned matrices are large, cf. the remark after Theorem 5.. We see from Table that the numbers of iterations required for convergence by using our preconditioners are larger than those in Table. ( 0:00) ( + 0:00) n 64 8 56 5 04 I 64 09 77 55 y S { { { { { { T 6 4 7 0 67 47 K N;4 5 7 0 4 6 6 K N;6 5 6 8 8 8 8 K N;8 6 7 9 9 9 0 Table : Numbers of iterations for dierent preconditioners. Example illustrates that for a function with zeros close together, it has eectively one zero with order being the sum of the orders of the zeros. To further illustrate this, we give in Figures and the spectra of the preconditioned matrices for all ve preconditioners for the two test functions when n 8. Notice the dierence between the spectra of the two preconditioned matrices for K m;4. We also see that the spectra of the preconditioned matrices for K m;6 and K m;8 are clustered in a small interval around except for one to two large outliers and that all the eigenvalues are well bounded away from 0. We note that the Strang preconditioned matrices in both cases have negative eigenvalues that are not depicted in the gures. 9
7 6 K Jackson Preconditioner m,8 5 K Jackson Preconditioner m,6 4 K Jackson Preconditioner m,4 T. Chan Preconditioner Strang Preconidtioner No Preconidtioner 0 0 4 0 0 0 0 0 0 0 0 Figure : Spectra of preconditioned matrices for f() ( ) ( + ) when n 8. 7 6 K Jackson Preconditioner m,8 5 K Jackson Preconditioner m,6 4 K Jackson Preconditioner m,4 T. Chan Preconditioner Strang Preconidtioner (has negative eigenvalues) No Preconidtioner 0 0 6 0 4 0 0 0 0 0 4 0 6 Figure : Spectra of preconditioned matrices for f() ( 0:00) ( + 0:00) n 8. when 0
References [] G. Ammar and W. Gragg, Superfast solution of real positive denite Toeplitz systems, SIAM J. Matrix Anal. Appl., 9 (988), pp. 6{67. [] T. Apostol, Mathematical Analysis, Addison-Wesley, 97. [] O. Axelsson and V. Barker, Finite Element Solution of Boundary Value Problems, Theory and Computation, Academic Press, Orlando, 984. [4] F. Di Benedetto, Analysis of preconditioning techniques for ill-conditioned Toeplitz matrices, SIAM J. Sci. Comput., 6 (995), pp. 68{697. [5] F. Di Benedetto, G. Fiorentino and S. Serra, C.G. preconditioning for Toeplitz matrices, Comput. Math. Appl., 5 (99), pp. 5{45. [6] R. Chan, Circulant Preconditioners for Hermitian Toeplitz Systems, SIAM J. Matrix Anal. Appl., 0 (989), pp. 54{550. [7] R. Chan, Toeplitz Preconditioners for Toeplitz Systems with Nonnegative Generating Functions, IMA J. Numer. Anal., (99), pp. {45. [8] R. Chan and M. Ng, Conjugate Gradient Methods for Toeplitz Systems, SIAM Review, 8 (996), pp. 47{48. [9] R. Chan and M. Yeung, Circulant Preconditioners Constructed from Kernels, SIAM J. Numer. Anal., 9 (99), pp. 09{0. [0] R. Chan, A. Yip and M. Ng, The Best Circulant Preconditioners for Hermitian Toeplitz Systems, Math. Dept. Res. Rept. 99-, The Chinese University of Hong Kong, 999. [] T. Chan, An Optimal Circulant Preconditioner for Toeplitz Systems, SIAM J. Sci. Statist. Comput., 9 (988), pp. 766{77. [] G. Lorentz, Approximation of Functions, Holt, Rinehart and Winston, New York, 966. [] D. Potts and G. Steidl, Preconditioners for Ill-Conditioned Toeplitz Matrices, BIT, 9 (999), pp. 5{5. [4] D. Potts and G. Steidl, Preconditioners for Ill-Conditioned Toeplitz Systems Constructed from Positive Kernels, preprint, 999. [5] S. Serra, Preconditioning Strategies for Hermitian Toeplitz Systems with Nondenite Generating Functions, SIAM J. Matrix Anal. Appl., 7 (996), pp. 007{09.
[6] S. Serra, On the extreme eigenvalues of Hermitian (block) Toeplitz matrices, Lin. Alg. Appl., 70 (998), pp. 09{9. [7] G. Strang, Introduction to Applied Mathematics, Wellesley-Cambridge Press, Cambridge, 986. [8] G. Strang, A Proposal for Toeplitz Matrix Calculations, Stud. Appl. Math., 74 (986), pp. 7{76. [9] W. Trench, An Algorithm for the Inversion of Finite Toeplitz Matrices, SIAM J. Appl. Math., (964), pp. 55{5. [0] E. Tyrtyshnikov, Circulant Preconditioners with Unbounded Inverses, Lin. Alg. Appl., 6 (995), pp. {4.