On the Properties for Iteration of a Compact Operator with Unstructured Perturbation

Transcription

1 On the Proerties for Iteration of a Comact Oerator with Unstructured Perturbation Jarmo Malinen Helsinki University of Technology Institute of Mathematics Research Reorts A360 (1996) 1

2 2 Jarmo Malinen: On the Proerties for Iteration of a Comact Oerator with Unstructured Perturbation, Helsinki University of Technology, Institute of Mathematics, Research Reorts A360 (1996) Abstract. We consider certain seed estimates for Krylov subsace methods (such as GMRES) when alied uon systems consisting of a comact oerator K with small unstructured erturbation B. Information about the decay of singular values of K is also assumed. Our main result is that the Krylov method will erform initially at suerlinear seed when alied uon such reconditioned system. However, with large iteration numbers the effect of B is seen to be dominant. As a byroduct, we resent several uer seed estimates with exlicit constant that should be useful for numerical uroses. AMS subject classification. 65F10, 65F40, 47A55, 47A30 Key words. Krylov subsace methods, reconditioning, comact oerator, Schatten class, seed estimate, suerlinear, linear

3 1 INTRODUCTION 3 1 Introduction Given a bounded linear oerator L L(H), what kind of comutational roblem is the inversion of equation (1.1) x = Lx + g by iterative means? How fast can it be done by Krylov subsace methods? How to make good seed estimates for the iterations? The answers to these (and related) questions give us much information about L. In articular the roerties for iteration of a comact oerator are by now rather well known, see [1]. In the theory of reconditioning and iterative inversion of Toelitz oerators we meet another kind of roblem, which is related to the case of the comact oerator but not quite, see [2]. We get linear oerators consisting of a large comact art K and a small non-comact erturbation B to invert. In the Toelitz context we have a comlicated interaction between K and B that we cannot lay our hands on. However, we may always generalize and look at K with a small unstructured erturbation B, whose norm alone is known. The study of such systems is the subject of this aer. Another asect on the roerties for iteration of K + B can be found in [3]. We shall see that an abstract Krylov subsace method will start with a suerlinear convergence rate corresonding to the structure of the comact art K. When the comact art has been deleted, solver will attack the unstructured small erturbation, and convergence rate is now linear. In alications the small erturbation art could be a reconditioning error (see [2]) or a linear oerator arising from noise. In both cases, the comact art is what matters and iteration should be stoed once it enters the linear hase - at least if the small erturbation is noise. So we may say, that the roerties for iteration of K are in a sense reserved under small erturbations. The notion of determinant is quite crucial in our aroach. In our language, the determinant of a comact oerator K is an entire function φ K (λ), which has zeroes dee enough to remove the singularities of (λ K) 1 other that the origin. In the constructions of determinants we shall need convergence estimates for the singular values of K. The additional requirement that K lies in some Schatten class S (H), as defined below, can serve as such. Definition 1.1. Let H be Hilbert, (0, ) and σ j be the singular values of K LC(H). (i) By. S denote the number in [0, ] given by: (1.2) K S := ( σ j ) 1 for each K LC(H). j=0

4 4 1 INTRODUCTION (ii) By S (H) denote the set of such K LC(H) that K S <. We call the sace S (H) the Schatten -class. At the end, in Lemma 6.5, we have analytic (but no longer entire) functions that we wish to call generalized determinants for the non-comact oerators K + B, where K S (H) and B is small. The general organization of this aer is as follows: In Section 2 we give the basic definitions of the theory of olynomial acceleration of iterations, following the guidelines of [1]. In Section 3 we define a determinant for K S (H), (0, 1] and use it rove a so called Carleman inequality (Theorem 3.8) for such K. Section 4 is devoted to the study of determinants for K S (H), (1, ). The results of Section 4 are used in Section 5 to rove the Carleman inequality for K S (H), (1, ) (Theorem 5.4). These Carleman inequalities (giving uer bounds for the resolvents of certain comact oerators) are by now classical (see [4]), but here we give recise roofs with exlicit, carefully analysed constants. These results are very useful for numerical uroses, and to our knowledge not reviously ublished. Finally in Section 6 we aly the results of all revious sections in order to roduce our main results (Theorems 6.7 and 6.9) that show us what an iterative solver can do on a comact oerator with small erturbation. As a byroduct we obtain a generalization of Carleman inequality for oerators of form K + B with K S (H) and B of small norm (formula 6.12). The idea of studying this roblem, and here resented aroaches to solve is originated from the author s Master s Thesis about certain roerties for iteration of Toelitz oerators, and are to our knowledge new. The construction of the olynomial sequence in Theorem 6.7 has been searately ublished in [3]. 2 Polynomial acceleration of iterations By X denote a Banach sace. Let L L[X] such that 1 / σ(l). Assume that the following fixed oint roblem is to be solved: (2.1) x = Lx + g Given x 0, define the sequence {x j } j=0 of elements of X by the recursion: (2.2) x j+1 := Lx j + g It is clear that if {x j } converges in the toology of X, then the limit x is the solution on (2.1). This is how we have constructed a rudimentary iterative solver for equation (2.1), called the method of successive aroximations. Given a sequence of iterates {x j }, the convergence of iteration (2.2) (and any other iteration that we shall later study) can be studied in terms of the following two sequences, which we shall call residuals and errors, resectively. (2.3) d j := Lx j x j + g

5 2 POLYNOMIAL ACCELERATION 5 (2.4) e j := x x j After n stes of iteration (2.2), we have a n + 1 elements {x j } n j=0 of X in hand. Also we know n residuals {d j } n 1 j=0, but the error sequence is not known {e j} n 1 j=0, because we simly do not have x. What we have is some information about the roblem (2.1) the Krylov data. One might justifiably ask three questions: (i) Assuming that we want to calculate Krylov data in n stes, is it not true that we can get a better aroximation to the solution of (2.1) than the last in the sequence {x j } of the iterates of the simlest iterative solver of all? (ii) If the above question can be answered in ositive, how do we find a better aroximation than x n? (iii) Assuming that we could use the Krylov data of the roblem otimally, how good an aroximation to the solution x of (2.1) could we get? In many cases, the first question can be answered ositively, and further study would reveal details that at least artially answer the latter two questions. We might wish to search for the solution of form: n 1 (2.5) x n = x 0 + γ jn L j d 0 j=0 for some coefficients γ jn. For notational convenience, define two olynomials n and q n 1 as follows: n 1 (2.6) q n 1 (λ) := γ jn λ j and j=0 (2.7) n (λ) := 1 (1 λ)q n 1 (λ) Note that n satisfies the normalization condition n (1) = 1. Each olynomial n normalized in this way corresonds to some (not necessarily very efficient) method of using Krylov data to roduce an aroximate solution to (2.1). Thus the olynomials normalized in this way will be imortant, and so we shall give the following definition: Definition 2.1. By P n denote the set of olynomials of degree n, with comlex coefficient, satisfying ( n P n ) : n (1) = 1. In the language of the olynomials q n 1, n, formula (2.5) takes now form: (2.8) x n = x 0 + q n 1 (L)d 0 Furthermore, the following roosition in [1, Pro ] will show how the residuals, iterates and errors (as defined in (2.3) and (2.4)) relate to olynomials q n 1, n :

6 6 2 POLYNOMIAL ACCELERATION Proosition 2.2. Define the Krylov subsace method for the solution of (2.1) method by (2.8), with olynomials q n 1, n satisfying (2.7). By x 0 X denote the initial vector of the iteration. Then the residuals, iterates and errors satisfy: (2.9) d n = n (L)d 0 (2.10) x n = n (L)x 0 + q n 1 (L)g (2.11) e n = n (L)e 0 Proof. By definitions in formulae (2.3),(2.8) and (2.7) we get: (2.12) d n := (I L)x n + g = (I L)(x 0 + q n 1 (L)d 0 ) + g = [I (I L)q n 1 (L)] d 0 + [g (I L)x 0 d 0 ] = n (L)d 0 and (2.9) is roved. To attack (2.10), we calculate by using (2.8), (2.3) and (2.7): (2.13) x n := x 0 + q n 1 (L)d 0 = x 0 + q n 1 (L)[(L I)x 0 + g] = [I + (L I)q n 1 (L)] x 0 + q n 1 (L)g = n (L)x 0 + q n 1 (L)g Finally we use (2.4), (2.8) and (2.7): (2.14) e n := x x n = [(I L) 1 q n 1 (L)]g n (L)x 0 = (I L) 1 [I (I L)q n 1 (L)]g n (L)x 0 = (I L) 1 n (L)g n (L)x 0 = n (L)[(I L) 1 g x 0 ] = n (L)(x x 0 ) = n (L)e 0 Formula (2.11) now follows. It is also true that the olynomials q n 1, n related by (2.7) transform the original roblem (2.1) into a roblem with the same solution. The following roosition is from [1, Pro ]: Proosition 2.3. Let q n 1 be an arbitrary olynomial and set n (λ) := 1 (1 λ)q n 1 (λ). If x is the solution of (2.15) x = Lx + g then it also solves the roblem (2.16) x = n (L)x + q n 1 (L)g Conversely, if additionally N(q n 1 (L)) = 0, then (2.15) follows from (2.16) (N denotes the kernel of a linear oerator). Proof. See [1,. 4] Proosition 2.3, in essence, can be regarded as the core of the theory of olynomial acceleration in the inversion of linear oerators. By choosing olynomial

7 3 CARLEMAN INEQUALITY FOR (0, 1] 7 n P n in a clever way, we transform equation (2.15) into the equivalent equation (2.16) with more leasant invertibility roerties. It is of course desirable to choose n (L) to be as small in norm as ossible. Then, by (2.16), the solution x of (2.15) is about q n 1 (L)g, rovided that n (L)x x. If this is the case, then also q n 1 (L) is close to (I L) 1, by (2.1). So it is our task to search for olynomials n such that n (L) is small. In ractice we usually do not calculate with oerators in order to minimize the oerator norm n (L). This follows from the fact that tyically oerators on large saces are clumsy, if not quite imossible objects to handle. Even if that was ossible, oerator owers for olynomials might be exensive to calculate, or the calculation and minimization of the oerator norms (that we need when finding the otimal use for the Krylov information) might be an undesirable task. Because of these reasons it is customary not to invert oerators themselves, but only equations (2.1) with given fixed g. Proosition 2.2 gives us the hint to minimize the errors or residuals, instead. However, by formula (2.11), the minimization of errors is not feasible, because the initial error e 0 := x x 0 is never known. However, the initial residual d 0 := (L I)x 0 +g is known, because it is determined by our initial guess x 0 that does not contain information. This is exactly what the GMRES (generalized minimal residual) algorithm does, giving of course better erformance with better initial guess x 0. 3 The Carleman inequality for (0, 1] Let H be a searable Hilbert sace and K LC(H). Certain conclusions on the size of the resolvent can be made, roviding we have some knowledge on the distribution of the singular values σ j (K) of K. The Schatten norm of Definition is this kind of information the corresonding inequalities are called Carleman inequalities. The natural starting oint is a result concerning oerators on a finite dimensional Hilbert sace, i.e. matrices. However, before that let us define the determinant for oerators in S (H), (0, 1]. Definition 3.1. Let (0, 1] and K S (H). Let λ C\{0} be arbitrary. Then define the determinant of (I K λ ) by: (3.1) det(i K λ ) := (1 λ j(k) ) λ where λ j (K) is the sequence of eigenvalues for K. It is well known that we must require some seed of decay from the eigenvalues λ j (K) in order to get the infinite roduct in (3.1) converge. One way to deal with this is require that K lies in some Schatten class S (H) for (0, 1]. Consider the following definition and lemma: 1 Actually we may seak about norm only if 1. However, we shall not let this disturb ourselves. To make the recise distinction here would only be a trick of technical nature.

8 8 3 CARLEMAN INEQUALITY FOR (0, 1] Definition 3.2. Let K LC(H) with eigenvalues λ j (K). Let (0, ). Define: (3.2) Λ (K) := λ j (K) There is relation between the Schatten norm K S of a given comact oerator K and Λ (K), which encodes the decay of the eigenvalues of K. The following classical result is by H. Weyl: Lemma 3.3. Let (0, ) and K S (H). Then the series λ j(k) converges absolutely and (3.3) Λ (K) K S Proof. This is [4, Corollary 1,. 1093]. It can be roved by standard argument that the roduct (3.1) defining the determinant converges uniformly on comact subsets of C \ {0} with resect to λ C if K S (H). In articular this imlies that det(i K λ ) is a holomorhic function in C \ {0}. Proosition 3.4. For all x R + and (0, 1]): (3.4) 1 + x e x Proof. Write f(t) := e t 1 t 1. For t > 0, (3.5) f (t) = 1 (e t t 1 ) Quite easily f (t) > 0 if t > 0, and consequently f is an increasing function if t [0, ). Clearly f(0) = 0. It follows that f is a ositive function in [0, ) which is equivalent with the inequality: (3.6) e t 1 + t 1 By setting t = x formula (3.4) follows. The following lemma is the first in the series of Carleman inequalities. It is a finite dimensional statement that will be generalized later in Theorem 3.8. Lemma 3.5. Let 0 < 1 and K L(E d ), where E d is a d-dimensional Hilbert sace. Then for λ / σ(k): (3.7) det(i K λ ) (I K K S λ ) 1 e λ Proof. Let S E d be invertible. Let us first show that: (3.8) det(s)s 1 d 1 σ j (S)

9 3 CARLEMAN INEQUALITY FOR (0, 1] 9 where σ j (S) are the singular values of S in non-increasing order. We may diagonalize S by unitary matrices U,U to get S = UMU, where M is diagonal and contains the singular values of S in non-increasing order. Now because the determinant of a unitary matrix has modulus 1: (3.9) det(s)s 1 det(u) det(m) det(u ) U 1 M 1 U 1 det(m) M 1 Clearly it suffices to show (3.8) for diagonal invertible matrices M only. Because det(m) is the roduct of its singular values, det(m)m 1 is a diagonal matrix with nth diagonal element equal to d ;j n σ j(m). By the fact that the oerator norm on finite dimensional Hilbert sace equals the largest singular value: (3.10) det(m)m 1 = max d n {1,2...,d} ;j n d 1 σ j (M) = σ j (M) and (3.8) has been roved for invertible diagonal matrices, and hence for all invertible matrices. To rove the claim of the lemma, set S := I K λ, λ / σ(k). We get: (3.11) det(i K λ ) (I K d 1 λ ) 1 σ j (I K λ ) d 1 d 1 (I σ d 1 j(k) ) λ e σ j (K) λ e P d 1 σ j (K) λ (I + σ j(k) ) λ λ e K where we have used Proosition 3.4. The lemma has been roved. It is our intention to generalize the revious lemma to any searable Hilbert sace H. The following lemma and roosition are tools that we shall need. Lemma 3.6. Let (0, 1] and K S (H), where H is a searable Hilbert sace. Then: S λ (3.12) det(i K λ ) e K Proof. This is a direct consequence of Lemma 3.3 and Proosition 3.4

10 10 3 CARLEMAN INEQUALITY FOR (0, 1] Proosition 3.7. Let X be a Banach sace and assume W n, W L(X) such that (i) ( n N) : W n C n (ii) ( x X) : lim n W n x Wx (i.e. W n W in strong oerator toology) (iii) lim n C n = C Then W C. Proof. For contradiction, assume that there is x 0 X; x 0 = 1 such that Wx 0 > C + ǫ for some ǫ > 0. Then by (ii) ( N 1 N) : (3.13) ( n > N 1 ) : W n x 0 > C + ǫ 2 Furthermore, by (iii) ( N 2 N) : (3.14) ( n > N 2 ) : C n < C + ǫ 2 Now, if n > max(n 1, N 2 ) we have: (3.15) W n x 0 > C n The contradiction against (i) roves the roosition.. Now we are ready to state and rove the Carleman inequality for Schatten classes S (H), (0, 1]. For arallel exositions, see [1] Section 5.6 and [4]. Theorem 3.8. Let (0, 1] and K S (H). For each λ / σ(k) {0} we have: (3.16) det(i K λ ) (I K K S λ ) 1 e λ Proof. Fix λ / σ(k) {0}. Pick a sequence S (H) K n K in S (H) such that dimk n (H) = n. Define the Hilbert subsace H n of H by: (3.17) H n := K n (H) + K n(h) Clearly dimh n 2n. By P n denote the orthogonal rojector onto H n. One easily checks that K n = P n K n P n and (3.18) (I K n λ ) 1 = P n (I K n λ ) 1 P n + (I P n ) because H n is the smallest reducing subsace of K n, outside of which K n vanishes. So we may regard K n as a matrix of dimension at most 2n 2n, and use

11 GENERALIZED DETERMINANTS 11 Lemma 3.5. By Lemmas 3.5, 3.6 and the definition of oerator norm we have: (3.19) Denote: det(i K n λ ) (I K n λ ) 1 = max { det (I K n λ ) P n(i K n λ ) 1 P n, det (I K n λ )(I P n) } = max { det (I K n λ ) P n(i K n λ ) 1 P n, det (I K n λ ) } max { Kn Kn S S e λ, e λ S λ } = e Kn (3.20) W n (λ) := P n (I K n λ ) 1 P n + (I P n ) As n, (3.21) W n (λ) W(λ) := det (I K λ )(I K λ ) 1 in strong oerator toology for each fixed λ / σ(k) {0}. Denote: Kn S (3.22) C n := e λ Because K n K in S (H), we have: K (3.23) lim C S n = C := e λ n Now the hyotheses of Proosition 3.7 have been fulfilled, and it follows: (3.24) W = det(i K λ ) (I K K S λ ) 1 e λ This roves the theorem.. = C Remark 3.9. Note that in Theorem 3.8 the Carleman inequality has been roved only for λ / σ(k) {0}. The sectrum on K, however, consists of discrete oints (with the excetion of the origin), and by continuity we may exand the result to aly for all λ 0. 4 Generalized determinants We have stated in Section 3 that the requirement K S (H) for (0, 1] enforces sufficient amount of decay on the eigenvalues so that roduct (3.1) converges. On the contrary, it is true that a weaker requirement K S (H) for (1, ) alone does not guarantee the convergence of the classical determinant. So it must be acceted that the classical determinant of Definition 3.1 simly will not do for saces S (H), > 1. However, the concet of an analytic function whose zeroes kill the singularities in the resolvent of K seems very aealing

12 12 GENERALIZED DETERMINANTS to us. Fortunately, it is ossible to construct generalized determinants that satisfy our requirements. The building blocks for these new determinants are the Weierstrass elementary factors whose definitions and certain growth estimates are given in the following: Definition 4.1. Let m N. The Weierstrass elementary factor is the entire function defined by: (4.1) E m (z) := (1 z)e P m zj j The study convergence questions of the above roducts are to be found in almost any book of basic function theory, see for examle [5, Theorem 15.9]. The following roositions will be used in the roof of Lemma (4.2) Proosition 4.2. For any m N we have: sin mθ m sin θ Proof. We shall give an induction roof. Clearly (4.2) holds for m = 1. Assume that it holds for m N. Then: (4.3) sin (m + 1)θ = sinmθ cosθ + cosmθ sin θ sin θ sin θ sin mθ cosθ + cosmθ sin mθ + 1 m + 1 sin θ sin θ This roves the claim. Certain radial size estimates for the Weierstrass elementary factors will be needed in the study on the generalized determinants. The following lemma is a result of O. Blumenthal and E. Lindelöf, see [6, ]. Lemma 4.3. Let m N and α [0, 1]. Then: Cm,α z m+α (4.4) E m (z) e where (4.5) C m,α mα (m + 1) 1 α m + α 2 Proof. The modulus of the Weierstrass elementary factor can be written in form: (4.6) E m (r e iθ ) = e log 1 r eiθ + P m rj j cos jθ So our task is clearly equivalent with finding an uer bound for the exression:

13 GENERALIZED DETERMINANTS 13 (4.7) G m,α (r, θ) := log 1 r eiθ + m rj j cos jθ r m+α = 1 2 log (1 2r cosθ + r2 ) + m rj j cos jθ r m+α In order to maximize this function G m,α, we calculate the artial derivatives. By a straightforward calculation we get the following identities: (4.8) and (4.9) G m,α (r, θ) = m + α G m,α (r, θ) + 1 r cosmθ cos(m + 1)θ r r r α 1 2r cosθ + r 2 G m,α 1 α r sin mθ + sin (m + 1)θ (r, θ) = r r 1 2r cosθ + r 2 where we have used the helful sum formulae: m (4.10) r j 1 cosjθ = rm+1 sin mθ r cos(m + 1)θ r + cosθ 1 2r cosθ + r 2 and (4.11) m Let (r 0, θ 0 ) satisfy Gm,α r r j sinjθ = rm+2 sin mθ r m+1 sin (m + 1)θ + s sin θ 1 2r cosθ + r 2 (r 0, θ 0 ) = Gm,α r (r 0, θ 0 ) = 0. From (4.8) we get: (4.12) G max = r1 α 0 r 0 cosmθ 0 cos(m + 1)θ 0 + α 1 2r 0 cosθ 0 + r0 2 where G max := G m,α (r 0, θ 0 ). Formula (4.9) imlies (4.13) r 0 = sin (m + 1)θ 0 sin(m)θ 0 (or r 0 = 0, which is not a valid maximum). By inserting this into (4.12) we find: (4.14) G max = r1 α 0 sin (m + 1)θ 0 cosmθ 0 cos(m + 1)θ 0 sin mθ 0 m + α sinmθ 0 (1 2r 0 cosθ 0 + r0 2) = 1 m + α sin θ 0 cos 1 α (m + 1)θ 0 sin 2 α mθ 0 (1 2r 0 cosθ 0 + r 2 0 ) = 1 sin α mθ 0 sin 1 α (m + 1)θ 0 m + α sin θ 0 where the last equality follows from: (4.15) 1 2r 0 cosθ 0 + r 2 0 = sin2 θ 0 sin 2 mθ 0

14 14 GENERALIZED DETERMINANTS Now we have immediately (4.16) G max = 1 (sin mθ 0 ) α (sin (m + 1)θ 0 ) 1 α m + α sinθ 0 sinθ 0 1 m + α mα (m + 1) 1 α where the last uer estimate is by Proosition 4.2. The uer bound 2 in formula (4.5) is easy to show. This roves the lemma. The following corollary is nothing but the result of the revious Lemma in a form that will be useful for our uroses. Corollary 4.4. Let m N and 1 m. Then (4.17) E m (z) e cm, z where (4.18) c m, := m m (m + 1) 1 +m 2 Proof. Trivial. Now we know enough about the nature of things to define the generalized determinants det m for each m N: Definition 4.5. Let m N and 1 m <. Let K S (H) and by λ j (K) denote the jth eigenvalue of K, ordered in the non-increasing order of absolute values, with multilicities. Let λ 0. Then (4.19) det m (I K λ ) := is called the generalized determinant of order m. E m ( λ j(k) ) λ The roduct defining the generalized determinant of order m converges uniformly on comact subsets of C \ {0}. It follows that the limit is a holomorhic function in C \ {0}. It should not arrive as a surrise that the growth estimates on the Weierstrass elementary factors will imly corresonding growth estimates on the generalized determinants. The following lemma will state the immediate consequences of Corollary 4.4: Lemma 4.6. Let m N and 1 m <. Let K S (H). Then (4.20) det m (I K ) ecm, λ where K S λ (4.21) c m, := m m (m + 1) 1 +m 2

15 5 CARLEMAN INEQUALITY FOR (1, ] 15 Proof. Use Corollary 4.4 as follows: (4.22) det m (I K λ ) = = e cm, Λ(K) λ e cm, K E m ( λ j(k) ) e P λ S λ cm, λ j (K) λ where the final conclusion is by Lemma The Carleman inequality for (1, ) In this section we shall use the tools of Section 4 to rove Theorem 5.4; the Carleman inequality for comact oerators in S (H) for (1, ). Lemma 5.1. Let m N and K, B S m (H). Let λ / σ(k) {0}. Then the following determinant is analytic and satisfies: (5.1) d dz det m (I K λ + zb) z=0 = det m (I K λ )Tr [{(I K λ ) ( K λ )m 2 }B] Proof. See [4, Lemma 23,.1110] Lemma 5.2. Let m N and 1 m <. Then (5.2) det m (I K λ ){(I K λ ) ( K λ )m 2 } Sq e cm, where c m, satisfies (4.21) and q = 1. K S λ Proof. Let B S (H), B S = 1 and q Residue Theorem: = 1. By Lemma 5.1 and the (5.3) det m (I K λ )Tr[{(I K λ ) ( K λ )m 2 }B] = 1 ξ 2 det m (I K 2πi λ + ξb)dξ ξ =ν for all ν > 0. By Lemma 4.6 (set λ = 1): (5.4) det m (I + S) e cm, S S for any S S. Now take S := K λ + ξb and aly this uon (5.3): (5.5) det m (I K λ ) Tr [{(I K λ ) ( K λ )m 2 }B] e cm, max ξ =ν K λ +ξb S e c m, [ K S +ν B λ S ]

16 16 5 CARLEMAN INEQUALITY FOR (1, ) Now let ν 0. Then the revious formula gives: (5.6) det m (I K λ ) Tr [{(I K λ ) ( K λ )m 2 }B] e cm, By the inverse Hölder inequality ([4, Lemma 14,. 1098]) we have: (5.7) A Sq = su T r(ab) B S 1 This and (5.6) give: (5.8) det m (I K λ ){(I K λ ) ( K λ )m 2 }B Sq e cm, K S λ K S λ This roves the lemma. One more constant will have to be calculated, before we are able to rove the Carleman inequality. Proosition 5.3. For all x 0 we have: (5.9) x j e j e x Proof. Trivial. Theorem 5.4. Let m N and 1 m <. Let K S (H). Then (5.10) det m (I K λ ) (I K λ ) 1 m e (cm,+ 1 e ) K S λ where (5.11) c m, = m m (m + 1) 1 +m Proof. By Lemma 5.2: 2 (5.12) det m (I K λ ){(I K λ ) ( K λ )m 2 } det m (I K λ ){(I K λ ) ( K λ )m 2 } Sq e cm, K S λ By the triangle inequality, we get immediately: (5.13) det m (I K λ ) (I K λ ) 1 det m (I K λ ){ ( K K ) m 2} S + e c m, λ λ det m (I K m 2 λ ) j=0 K j λ j + e cm, K S λ

17 6 ON THE POLYNOMIAL ACCELERATION 17 Let us study more closely the sum in the revious exression. We have by Proosition 5.3: (5.14) and moreover: (5.15) m 2 K j λ j K j λ j K j λ j From Lemma 4.6 we remember: (m 1)e m 2 e e j K e λ K S λ (5.16) det m (I K ) ecm, λ e j K S e λ (m 1)e 1 e K S λ K S λ To conclude the roof, let us combine (5.13), (5.15) and (5.16) in the following manner: (5.17) det m (I K λ ) (I K λ ) 1 e cm, = e cm, K S λ or a more beautiful uer bound: e cm, This roves the theorem. K K + e cm, S ( λ 1 S ) (m 1)e e λ K K S ( λ 1 S ) 1 + (m 1)e e λ K K S ( λ 1 S m e e λ j ) = m e (c m,+ 1 K e ) S λ 6 On the olynomial acceleration of a comact oerator with a small erturbation By availing the Carleman inequalities and certain oints of vector valued function theory, conclusions on the olynomial acceleration roerties of oerator L := K + B (as resented in Section 2) can be drawn, where K LC(H) for a searable Hilbert sace H, B L(H) is small when comared to K and 1 / σ(k +B). Everything before Theorem 6.7 will be just technical rearations. Definition 6.1. Let K LC(H) and B L(H). Denote: (6.1) K λ := (λ B) 1 K for all λ / σ(b). Remark 6.2. In this section we are keeing the small erturbation B always the same. That is the reason why we have chosen not to write the deendency of K λ on B exlicitly.

18 18 6 ON THE POLYNOMIAL ACCELERATION The ideal structure of the set of finite dimensional oerators and the fact that the singular values of a given comact oerator K LC(H) are equal to the aroximation numbers makes it ossible to rove that the information about the Schatten class of K is reserved under the maing K K λ. The following Proosition makes this oint recise. Proosition 6.3. Let (0, ), K S (H), B L(H) and λ / σ(b). Then K λ S (H) and it alies (6.2) K λ S (λ B) 1 K S Proof. Let λ / σ(b). The core of the roof is in the following calculation: (6.3) σ j (K λ ) = inf K λ F rankf n = inf rankf j (λ B) 1 (K (λ B)F) (λ B) 1 (λ B) 1 σ j (K) inf K (λ B)F rankf j because rank (λ B)F rank F. Raising to ower and summing u the singular values give formula (6.2). Lemma 6.4. Let m N. Let K n LC(H) be an oerator with n-dimensional range. Then (6.4) φ (n) m (λ) := det m (I K n λ ) is a holomorhic function of λ C \ σ(b). Proof. For brevity, denote D := C \ σ(b). Because dim K n (H) = n, we have dim(k n ) λ (H) n for all λ D. Moreover, there is an r: 1 r n + 1 such that r := max λ D card σ((k n ) λ ). By E D denote the set of such λ s that card σ((k n ) λ ) < r. By [7, Theorem 6.25] we know that E is a discrete closed set. Pick any λ 0 D\E. Because E is closed, we have a neighborhood N 1 (λ 0 ) of λ 0 such that (6.5) ( λ N 1 (λ 0 )) : cardσ((k n ) λ ) = r It follows that in N 1 (λ 0 ) the oerator (K n ) λ has exactly r 1 non-zero distinct eigenvalues {λ j ((K n ) λ ))} n. Define ǫ > 0 to be the minimum distance between any two eigenvalues in {λ j ((K n ) λ ))} n. By [7, Theorem 6.4] there exists a δ > 0 such that (6.6) λ λ 0 < δ = {λ j ((K n ) λ ))} n n B(λ j(k n λ0 ), ǫ 3 ) Now by [7, Theorem 6.20] we see that each maing λ λ j ((K n ) λ ) is holomorhic in N(λ 0 ) := {λ λ λ 0 < δ}. From the holomorhicity of Weierstrass

19 6 ON THE POLYNOMIAL ACCELERATION 19 elementary factors and the definition of det m we conclude that φ m (n) (λ) is indeed holomorhic in N(λ 0 ) and consequently in D \ E. Because φ (n) m (λ) is a continuous function on D, the Rado extension theorem ([5, Theorem 12.14]) imlies that φ m (n) (λ) is in fact holomorhic in whole D. This roves the lemma. In the following lemma we resent an analytic function φ m that can be regarded as a generalized determinant of the non-comact oerator K + B. Lemma 6.5. Let m N and 1 m <. Take K S (H). Then φ m (λ) := det m (I K λ ) is a holomorhic function of λ C \ σ(b). Proof. Pick a sequence of n-dimensional oerators K n K in S (H). Then by a continuity argument for the determinant we may show that for all λ C \ σ(b): (6.7) φ m (λ) = lim n φ(n) m (λ) If we are able to show that {φ (n) m } is a normal family of holomorhic functions, then the limit φ m (λ) itself is holomorhic by [5, Definition 14.5 and Theorem 10.28], thus roving the lemma. By Lemma 6.4 we already know that {φ (n) m } is a family of holomorhic functions. By [5, Theorem 14.6] it suffices to show that {φ (n) m } is uniformly bounded on the comact subsets of C \ σ(b). The following calculation makes the trick: (6.8) φ (n) m (λ) = det m (I + (λ B) 1 K n ) = E m ( λ j ((λ B) 1 K n )) = e cm, Λ((λ B) 1 K n) e cm, λj((λ B) 1 K n) where the inequality has been written by Corollary 4.4. We may continue the estimation by Lemma 3.3 as follows: (6.9) e cm, Λ((λ B) 1 K n) e cm, (λ B) 1 K n S e cm, (λ B) 1 K n S for all λ / σ(b); the last inequality is by Proosition 6.3. Because K n K in S (H), it follows that the family {K n } is uniformly bounded in the norm. S. Now the uer bound for each φ (n) m given by the right side of (6.9) is uniformly bounded on comact subsets of C \ σ(b) because the resolvent of B is. This roves the lemma. In the similar way we may roceed to rove also the following analogous lemma to Lemma 6.5:

20 20 6 ON THE POLYNOMIAL ACCELERATION Lemma 6.6. Let 0 < 1. Take K S (H). Then φ(λ) := det(i K λ ) is a holomorhic function of λ C \ σ(b). Proof. Omitted. It is time to resent a lemma that has something to do with the olynomial acceleration of the oerator K + B. We shall construct a sequence of nonnormalized olynomials { k } k=0 such that these olynomials see the oerator K + B small with increasing k. The existence of such non-normalized sequence itself is naturally a triviality, but later (in Lemma 6.8) we shall see that almost all of these olynomials in fact are almost normalized in the sense of Definition 2.1; i.e. { k (1)} k=0 remains bounded from some k on. The aroach resented here is due to O. Nevanlinna ([[]3]): Theorem 6.7. Let m N and 1 m <. Take K S (H) and let B L(H) be small such that 1 / σ(k + B). Then there exists a sequence of essentially monic olynomials { k } k=1 such that for all β (0, 1]: (6.10) k (K + B) 1 k ( m 1 k B + K S k )( B β k β + 1 ) 1 k e cm,+1 e k K 1 β S where c m, is defined in (4.21). Proof. Let us start with the easily roved fact for λ > B : (6.11) (λ (K + B)) 1 We also have by Theorem 5.4 and Proosition 6.3: (6.12) 1 λ B (I K λ) 1 φ m (λ) (I K λ ) 1 me (cm,+ 1 e ) K λ S me (cm,+ 1 K e )( S λ B ) On the other hand we can show that (I K λ ) 1 is a holomorhic oerator valued function outside σ(k+b) by studying the identity λ (K+B) = (λ B)(I K λ ). Also note that σ(k + B) σ(b) σ (K + B) for details see [1, Theorem ]. Estimate (6.12) together with the fact that φ m is holomorhic outside σ(b) (Lemma 6.5) allow us to conclude that φ m (λ)(i K λ ) 1 is holomorhic in C \ σ(b) and same alies also for: (6.13) φ m (λ)(λ (K + B)) 1 φ m (λ)(i K λ ) 1 (λ B) 1 Now the following estimate is a direct consequence of (6.12) and (6.13) for λ > B : (6.14) φ m (λ) (λ (K + B)) 1 m λ B e(cm,+ 1 e )( K S λ B )

21 6 ON THE POLYNOMIAL ACCELERATION 21 Because φ m is holomorhic in C\σ(B), we may write for λ > B the convergent Laurent series: a j (6.15) φ m (λ) = λ j Moreover, if λ > K + B, we may write the resolvent of K + B in form: (6.16) (λ (K + B)) 1 (K + B) j = j=0 j=0 λ j+1 By multilying the two revious formulae we get: (6.17) φ m (λ)(λ (K + B)) 1 = k (K + B)λ k 1 k=0 where the olynomials { k } k=0 are defined by: k (6.18) k (ξ) := a k j ξ j j=0 Formula (6.17) reresents a convergent series for λ > K + B. On the other hand, we know by the argument receding formula (6.13) that the left side of (6.17) is holomorhic for all λ > B ; not just for those λ s that satisfy λ > K + B. It follows that the right side of (6.17) converges for all λ > B. Now formula (6.17) and the standard theorem of residues gives: (6.19) k (K + B) = 1 2πi λ =r for any r > B. This gives immediately: (6.20) λ k φ m (λ)(λ (K + B)) 1 dλ k (K + B) 1 λ k φ m (λ) (λ (K + B)) 1 d λ 2π 1 2π rk λ =r m 1 r B e(cm,+ K e )( S r B ) 2πr where the first inequality is justified by formula (6.14). By setting r := B + K S k β and taking the kth root from the both ends on the revious formula gives (6.10). This roves the theorem. In Section 2 we found out that the olynomials k (λ) satisfying the normalization condition k (1) = 1 are of secial interest when studying the olynomial acceleration roerties of a given linear system. On the other hand, olynomials k are clearly not normalized in this manner - one might consider as a oor consolation that the coefficient of the highest degree term of k is always a 0 of formula (6.15). In this sense these olynomials are monic.

22 22 6 ON THE POLYNOMIAL ACCELERATION It follows from formula (6.18) that (6.21) k (1) = where a j s have the same meaning as in formula (6.15). It also follows that k j=0 (6.22) lim k k(1) = φ m (1) From the assumed nonsingularity of the roblem (i.e. 1 / σ(k + B)) and the general roerties of the generalized determinant φ m it follows that φ m (1) 0. For large enough k s, k (1) is bounded away from the origin, and we may define the correctly normalized olynomial sequence: (6.23) k (λ) := k(λ) k (1) This is exactly what we mean by stating before Theorem 6.7 that almost all of the olynomials k are almost almost normalized. It would be very satisfactory indeed to be able to give owerful results on the convergence roerties of the sequence k (1) with only relatively general conditions imosed on the oerators K and B. In articular, we would like to know how small the nonzero entity φ m (1) actually is. In order to study this question we should make further assumtions about K and B; this is no longer under the subject of this aer. We are satisfied with giving a lemma that collects the asymtotic result of the above discussion and Theorem 6.7: Lemma 6.8. Let m N and 1 m <. Take K S (H) and let B L(H) be small such that 1 / σ(k +B). Then there exist an integer m and a sequence of olynomials { k } k=m satisfying k(1) = 1 such that for all β (0, 1]: a j (6.24) k (K + B) 1 k (m C k ) 1 k ( B + K S k )( B β k β + 1 ) 1 k e cm,+ e 1 k K 1 β S where c m, is defined in (4.21) and C k satisfies: (6.25) lim k C k = C < Proof. Define k (λ) := k(λ) k (1) (6.24), where C k := 1 k (1) as in (6.23). Theorem 6.7 gives now formula. The existence of the finite C in formula (6.24) follows from the assumed nonsingularity of the roblem as roosed just after formula (6.23). This roves the lemma. We could rove a theorem quite similar to Theorem 6.7 by using the determinant and Schatten classes (0, 1]. Nothing essential would change in the

23 7 CONCLUDING REMARKS 23 roof, we would just use Theorem 3.8 instead of Theorem 5.4. The result is stated without roof. Theorem 6.9. Let (0, 1]. Take K S (H) and let B L(H) be small such that 1 / σ(k + B). Then there exists a sequence of essentially monic olynomials { k } k=1 such that for all β (0, 1]: (6.26) k (K + B) 1 k ( B + K S k β )( B k β K S + 1) 1 k e 1 (k 1 β ) 7 Concluding remarks Given a bounded linear oerator L, the otimal reduction factor η(l) defined by (7.1) η(l) := inf k 0; k k k (L) 1 k limits the attainable asymtotic seed of the all Krylov subsace methods alied uon L. It can be roved that η(l) is a function of the set σ(l), and furthermore that it does not deend on the isolated sectral values of L (see [1;Theorem 3.3.4]). Now, if L := K + B, then η(k + B) = η(b), because a comact erturbation can only add isolated oints to the sectrum of the oerator B. So it is true, that even a large comact art dies, because it cannot be seen in the otimal reduction factor. There is certain interest to look at the asymtotics of k (K+B) 1 k of Lemma 6.8 as k. It is true that our sequence is asymtotically otimal in the sense that one cannot construct another sequence k (K + B) whose limit is always smaller for all B of fixed size. For such B whose sectrum fills the disk of radius B, the otimal reduction factor would equal ρ(b) = B, by [1;Theorem 3.3.4(v)] and [1;Theorem 3.6.3(iii)]. So the asymtotic effect of K is inessential, but on the other hand we are not so interested in the asymtotics it is only the small number of iterations that should ever get calculated in the real life. Lemma 6.8 should tell us that in the first stages the iteration the convergence factor k (K + B) 1 k of order B + K s k β decreases (the suerlinear stage) and is asymtotically only of order B (the linear stage). Moreover, the rate of decrease of the convergence factor is dictated by the Schatten class of K. Note that the concet suerlinear is usually used to describe something that haens in the asymtotics of the seed estimates. Here we are a bit unorthodox and regard suerlinear stage of an iteration as those iteration stes when seed is being gained. By the linear stage we of course refer at the analogous henomenon.

24 24 7 CONCLUDING REMARKS The GMRES method for the inversion of non-symmetric roblems can be regarded as a minimization algorithm that (at least imlicitly) generates olynomial sequences to aroximate the values of resolvents at given oints; this is the minimization of residuals. If the GMRES generates the olynomial sequence s k (corresonding to our sequence k ), then the residual d k after k stes is roortional to s k (K + B)d 0 (see Proosition 2.2), and we have: (7.2) s k (K + B)d 0 k (K + B)d 0 k (K + B) d 0 The former inequality is true because s k is otimal at d 0, and k is suerotimal for the initial residual d 0. This is to say that the uer estimates we have for k are as well uer estimates for the GMRES residuals. The same kind of result is true so as to the error sequences with quite obvious modifications for the reasoning we again refer at Proosition 2.2.

25 7 CONCLUDING REMARKS 25 References 1. Nevanlinna, O., Convergence of Iterations for Linear Equations, Monograh in series Lectures in Mathematics ETH Zürich, Birkhäuser Verlag, Basel, Boston, Berlin, Malinen J., On Certain Proerties or Iteration of Toelitz Oerators, Helsinki University of Technology, Institute of Mathematics, Research Reorts A361 (1996). 3. Nevanlinna O., Convergence of Krylov-Methods for Sums of Two Oerators, Helsinki University of Technology, Institute of Mathematics, Research Reort A353 (1995). 4. Dunford, N & Schwarz, J., Linear Oerators; Part II: Sectral Theory, Interscience Publishers, Inc. (J. Wiley & Sons), New York, London, Rudin, W., Real and Comlex Analysis, 3rd edition, McGraw-Hill Book Comany, New York, Blumenthal, O., Princies de la thèorie des fonctions entieres d ordre infini, Paris, Auetit, B., A Primer on Sectral Theory, Sringer Verlag, Berlin, 1991.