Angles Between Infinite Dimensional Subspaces

1 Angles Between Infinite Dimensional Subspaces Abram Jujunashvili Department of Mathematics and Center for Computational Mathematics University of Colorado at Denver and Health Sciences Center Ph.D. Thesis Defense November 14, 2005 Supported by the Lawrence Livermore National Laboratory and by the National Science Foundation

2 Introduction In this dissertation, we introduce angles between infinite dimensional subspaces in two different ways, based on: spectra of product of orthogonal projectors that may result, e.g., in a set of angles that fills the whole interval [0, π/2]; Courant-Fischer numbers of operators that results in a finite number of angles or in a monotonically nondecreasing or nonincreasing sequence of angles. Courant-Fischer numbers are defined by the Courant-Fischer (min max, max min) principle. We call the second kind of angles the discrete angles. Connection to the finite dimensional case?

3 Main new results A new concept of principal invariant subspaces, which generalizes an existing concept of principal vectors; Lidskiĭ Mirsky Wielandt theorem for Courant Fischer numbers of bounded selfadjoint operators; Majorization for sines/cosines (squared) for discrete angles; Majorization for discrete Ritz values. We generalize some known results in the finite dimensional case (e.g., Knyazev and Argentati [2002, 2005]).

4 Outline Angles between subspaces; Estimate for proximity of angles; Discrete angles between subspaces; Estimates for proximity and majorization of discrete angles; Estimates for proximity of Ritz values; Analysis of convergence of domain decomposition methods; Application of angles between subspaces in microarray data analysis.

5 Why angles are important? Canonical correlations for stochastic processes; Information retrieval for functional data; Perturbations of subspaces related to Rayleigh Ritz approximations; Analysis of convergence of domain decomposition methods; Information about solution quality in eigensolvers; Microarray data analysis.

6 Preliminaries Let H be a (real or complex) Hilbert space, F and G be its proper nontrivial subspaces. A subspace is defined as a closed linear manifold. Let P F and P G be the orthogonal projectors onto F and G, respectively. We denote by B(H) the Banach space of bounded linear operators defined on H with a norm T = sup Tu. u H u =1 For T B(H) we define T = (T T) 1/2, using the positive square root. T U denotes the restriction of the operator T to its invariant subspace U. Σ(T) and Σ p (T) denote the spectrum and point spectrum of the operator T, respectively. Σ p (T) is defined as the set of numbers λ in Σ(T), for which λi T is not one-to-one, where I denotes the identity.

7 Angles between subspaces. Main new results: Angles from to ˆΘ(F, G) and between Θ(F, G); Connections: ˆΘ(F, G), ˆΘ(F, G ), etc. and Θ(F, G), Θ(F, G ), etc.; Principal invariant subspaces. Recursive relationships and isometries; Changes in the angles where one of the subspaces varies.

8 Definitions and properties of angles Definition A set ˆΘ(F, G) = {θ : θ = arccos(σ), σ 0, σ 2 Σ((P F P G ) F )} [0, π 2 ] is called the set of angles from the subspace F to the subspace G. Definition A set Θ(F, G) = ˆΘ(F, G) ˆΘ(G, F) is called the set of angles between the subspaces F and G. Remark In general ˆΘ(F, G) ˆΘ(G, F), i.e. the set-valued function ˆΘ(F, G) is non-symmetric, but Θ(F, G) is a symmetric function. Connection to the finite dimensional case?

9 Definitions of angles in the finite dimensional case Denote q = min{dimf, dimg}. Definition The principal angles θ 1,...,θ q [0, π/2] between F and G are defined recursively for k = 1,...,q by cos(θ k ) = max u F max v G (u, v) = (u k, v k ) subject to u = v = 1, u u i = v v i = 0, i = 1,...,k 1. In the infinite dimensional case this definition gives a finite number of angles (q < ) or infinite sequence of angles (q = ), but can not give a whole interval of angles. We use this approach later to define the discrete angles.

10 Definitions and properties of angles (cont.) Theorem 1 ˆΘ(F, G) = π 2 ˆΘ(F, G ). Theorem 1 leads to an equivalent definition of the angles using sines. Equivalent definition The set ˆΘ(F, G) = {θ : θ = arcsin(µ), µ 0 µ 2 Σ((P F P G ) F )} [0, π 2 ] is called the set of angles from the subspace F to the subspace G.

11 The connections between ˆΘ(F, G), ˆΘ(F, G ), ˆΘ(G, F), etc. Theorem 2 For any pair of subspaces F and G of H: 1. ˆΘ(F, G) = π 2 ˆΘ(F, G ); 2. ˆΘ(G, F) \ { π 2 } = ˆΘ(F, G) \ { π 2 }; 3. ˆΘ(F, G) \ ({0} { π 2 }) = π 2 {ˆΘ(F, G) \ ({0} { π 2 })}; 4. ˆΘ(F, G ) \ ({0} { π 2 }) = ˆΘ(F, G) \ ({0} { π 2 }); 5. ˆΘ(G, F ) \ {0} = π 2 {ˆΘ(F, G) \ { π 2 }}; 6. ˆΘ(G, F) \ { π 2 } = π 2 {ˆΘ(F, G) \ {0}}; 7. ˆΘ(G, F ) \ {0} = ˆΘ(F, G) \ {0}. Where do differences in the zero and right angles come from?

12 The connections between Θ(F, G), Θ(F, G ) and Θ(F, G ) Theorem 3 For any subspaces F and G of H the following equalities Θ(F, G) \ ({0} { π 2 }) = {π 2 Θ(F, G )} \ ({0} { π 2 }), Θ(F, G) \ {0} = Θ(F, G ) \ {0} and Θ(F, G ) \ {0} = Θ(F, G) \ {0} hold.

13 Spectra of the sum and the difference of projectors Theorem 4 Σ(P F P G ) \ ({ 1} {0} {1}) = ± sin(θ(f, G)) \ ({ 1} {0} {1}). Theorem 5 Σ(P F + P G ) \ ({0} {1}) = {1 ± cos(θ(f, G))} \ ({0} {1}). P G = 0 1 1 0 0 B @ 0 0 0C A ; P F 0 0 0 (1) = 0 1 1 0 0 B @ 0 1 0C A 0 0 0 or (2) = 0 1 0 0 0 B @ 0 0 0C A. 0 0 1 (1) Σ(P F + P G ) = {0} {1} {2}; Θ(F, G) = {0}; 1 ± cos(θ(f, G)) = {0} {2}. (2) Σ(P F + P G ) = {0} {1}; Θ(F, G) = {π/2}; 1 ± cos(θ(f, G)) = {1}.

14 Angles between subspaces and the gap gap(f, G) = P F P G = max{ P F P G, P G P F } The classical definition, see the textbooks Akhiezer and Glazman [1981], Gohberg and Kreĭn [1969], Kato [1995]. Theorem 6 min{min{cos 2 (ˆΘ(F, G))}, min{cos 2 (ˆΘ(G, F))}} = 1 (gap(f, G)) 2

15 Angles between subspaces and the minimum gap For subspaces F and G of H let us define Deutsch [1995] c(f, G) = sup{ (u, v) u F (F G), u 1, and the minimum gap Kato [1995] γ(f, G) = v G (F G), v 1}, inf u F, u/ G d(u, G) d(u, F G). Theorem 7 c(f, G) = sup { cos ( Θ(F, G) \ {0} )} ; γ(f, G) = inf { sin ( Θ(F, G) \ {0} )}.

16 Principal vectors Definition ˆΘ p (F, G) = {θ Θ(F, G) : cos 2 (θ) is an eigenvalue of (P F P G ) F }. Θ p (F, G) = ˆΘ p (F, G) ˆΘ p (G, F). Definition Normalized vectors u = u(θ) F and v = v(θ) G form a pair of principal vectors for subspaces F and G corresponding to the angle θ Θ(F, G), if P F v = cos(θ)u and P G u = cos(θ)v. Definition Assume θ Θ p (F, G) \ { π 2 }. Then, the multiplicity of cos2 (θ) as an eigenvalue of the operator (P F P G ) F, is called the multiplicity of θ. Theorem 8 There exists a pair of principal vectors for subspaces F and G corresponding to a given angle θ Θ(F, G) \ { π 2 } if and only if θ Θ p (F, G) \ { π 2 }. Principal vectors are eigenvectors of (P F P G ) F and (P G P F ) G. What s wrong with π/2?

17 Principal vectors (cont.) Theorem 9 (In the finite dimensional case, e.g., Wedin [1983], Knyazev and Argentati [2002]) Let u and v form a pair of principal vectors for subspaces F and G, corresponding to the angle θ Θ p (F, G) \ ({0} { π 2 }). Define u = (v cos(θ)u)/ sin(θ), v = (u cos(θ)v)/ sin(θ). Then u, v are the principal vectors for subspaces F and G, corresponding to the angle π 2 θ; u, v are the principal vectors for subspaces F and G, corresponding to the angle π 2 θ; u, v are the principal vectors for subspaces F and G, corresponding to the angle θ.

18 Principal subspaces Definition A pair of subspaces U F, V G is called a pair of principal subspaces for subspaces F and G corresponding to the angle θ Θ(F, G), if (P F P V ) F = cos 2 (θ)p U, (P G P U ) G = cos 2 (θ)p V. Theorem 10 Let U F be an eigenspace of an operator (P F P G ) F, corresponding to the eigenvalue cos 2 (θ), where θ Θ p (F, G) \ { π 2 }, P U be an orthogonal projector onto the subspace U, and V = P G U. Then the operator P V = 1 cos 2 (θ) P GP U P G is an orthogonal projector onto the subspace V.

19 Principal subspaces (cont.) Theorem 11 U F and V G form a pair of the principal subspaces for subspaces F and G corresponding to the angle θ Θ(F, G) \ { π 2 }, if and only if U and V are eigenspaces of (P F P G ) F and (P G P F ) G, respectively, corresponding to θ Θ p (F, G) \ { π 2 }. Then Θ(U, V) = Θ p(u, V) = {θ}. Theorem 12 If U(θ), U(φ) F, and V(θ), V(φ) G are the principal subspaces for subspaces F and G corresponding to the angles θ, φ Θ p (F, G) \ { π 2 }, then the following relations hold: P U(θ) P U(φ) = P U(θ) U(φ) ; P V(θ) P V(φ) = P V(θ) V(φ) ; P U(θ) and P V(φ) are mutually orthogonal if θ φ. Otherwise, if θ = φ, then we can choose V(θ) such that P U(θ) P V(θ) = P U(θ) P G ; For given U(θ) we can choose V(θ) such that P V(θ) P U(θ) = P V(θ) P F.

20 Principal subspaces (cont.) Theorem 13 Let U and V be the principal subspaces for subspaces F and G, corresponding to the angle θ Θ p (F, G) \ ({0} { π 2 }). Define U = R((P F P V ) F ), V = R((P G P U ) G ). Then U and V are closed and U, V are the principal subspaces for subspaces F and G, corresponding to the angle π 2 θ; U, V are the principal subspaces for subspaces F and G, corresponding to the angle π 2 θ; U, V are the principal subspaces for subspaces F and G, corresponding to the angle θ.

21 Principal invariant subspaces Definition A pair of subspaces U F, V G is called a pair of principal invariant subspaces for subspaces F and G, if P F V = U and P G U = V. Theorem 14 Let U be an invariant subspace of the operator (P F P G ) F and P U be an orthogonal projector on this subspace. Let the operator (P F P G ) U be invertible and its inverse be bounded. Then the orthogonal projector P V on the invariant subspace V = P G U of the operator (P G P F ) G is given by P V = P G P U ((P F P G ) U ) 1 P U P G. Theorem 15 Let π 2 / Θ(U, V) for a pair of subspaces U and V. U and V form a pair of principal invariant subspaces for subspaces F and G if and only if U is an invariant subspace for (P F P G ) F and V = P G U.

22 Principal invariant subspaces (cont.) Principal invariant subspaces can be determined recursively: Theorem 16 Let U F and V G be a pair of principal invariant subspaces for subspaces F and G, and U U, V V be a pair of the principal invariant subspaces for subspaces U and V. Then U, V form a pair of the principal invariant subspaces for subspaces F, G, and Θ(U, V) Θ(U, V) Θ(F, G).

23 Principal invariant subspaces and isometries Theorem 17 Let U and V be a pair of principal invariant subspaces of the subspaces F and G. Let W be a (partial) isometry from the polar decomposition P G P F = W P F P G P F. Then V = W U, U = W V, P V = WP U W and P U = W P V W. Remark If the inequality P F P G < 1 holds, then the assertions of Theorem 17 hold for both choices of W = W and W = Ŵ, where W = P G (I P F + P F P G P F ) 1/2 P F (Riesz and Sz.-Nagy [1990]); Ŵ = [P G P F + (I P G )(I P F )][I (P F P G ) 2 ] 1/2 [1970]; Kato [1995]). (Davis and Kahan

24 Spectral decompositions and isometries Theorem 18 Let {E 1 } and {E 2 } be the spectral measures of the operators (P F P G ) F and (P G P F ) G, respectively. Further, let Θ Θ(F, G) \ { π 2 } be the Borel set, P U(Θ) = cos(θ) E 1(dλ) and P V(Θ) = cos(θ) E 2(dλ). Then P V(Θ) = P G { cos(θ) } 1 λ E 1(dλ) P G. Theorem 19 Let E 1 (s) and E 2 (s) be the spectral families of the operators P F P G P F and P G P F P G, respectively, and the polar decomposition of the operator P G P F is given by P G P F = W P F P G P F. Then E 2 (s) = WE 1 (s)w.

25 Changes in the angles from to where one of the subspaces varies Hausdorff distance dist(s 1, S 2 ) = max{ sup u S 1 d(u, S 2 ), sup v S 2 d(v, S 1 )}. Theorem 20 (Kato [1995], Pták [1978]). Let A, B B(H) be selfadjoint operators. Then Theorem 21 dist(σ(a), Σ(B)) A B. dist(cos 2 (ˆΘ(F, G)), cos 2 (ˆΘ(F, G))) gap(g, G). Angles are stable with respect to a perturbation of a subspace.

26 Discrete angles: estimates and majorization Changes in the smallest discrete angles where the subspaces vary; Generalization of the Lidskiĭ Mirsky Wielandt theorem; Majorization for the sines of largest and the cosines of smallest discrete angles; Majorization for the squared sines and cosines of the discrete angles.

27 Preliminaries: Courant Fischer numbers The Courant Fischer numbers for a bounded selfadjoint operator A (Dunford and Schwartz [1988]; Reed and Simon [1978]): from the top: λ k (T) = sup H k H inf u Hk (u, Tu), k = 1, 2,.... dimh k =k u =1 The resulting sequence Λ (T) is ordered nonincreasingly; from the bottom: λ k (T) = inf H k H sup u Hk (u, Tu), k = 1, 2,.... dimh k =k u =1 The resulting sequence Λ (T) is ordered nondecreasingly. Connection to the spectrum of T? Connection to matrices?

28 Recursive definition of discrete principal angles Definition Denote q = min{dimf, dimg}. The smallest discrete principal angles θ 1,...,θ q [0, π/2] between F and G are defined recursively for k = 1,...,q by cos(θ k ) = sup u F sup (u, v) = (u k, v k ) v G subject to u = v = 1, (u, u i ) = (v, v i ) = 0, i = 1,...,k 1, provided that the pair of vectors u k, v k exists. If for some k = n the pair u k, v k does not exist, we set θ k = θ n for all k > n. Θ d (F, G) = (θ 1,...,θ q ), q is finite or infinite. As a set {Θ d (F, G)} Θ(F, G). Connection to the finite dimensional case?

29 Definition of smallest discrete angles using Courant Fischer numbers Equivalent definition The sequence ( ) ) Θ d (θ (F, G) = k : θ k = arccos λ k ((P FP G ) F ), k = 1, 2,...,q where λ k ((P FP G ) F ) are the Courant-Fischer numbers of the operator (P F P G ) F from the top, is called the sequence of smallest discrete principal angles between subspaces F and G.

30 Definition of largest discrete angles using Courant Fischer numbers Definition The sequence ( ) ) Θ d (θ (F, G) = k : θ k = arcsin λ k ((P FP G ) F ), k = 1,...,dimF ( ( ) ) θ k : θ k = arcsin λ k ((P GP F ) G ), k = 1, 2,...,dimG where λ k ((P FP G ) F ) and λ k ((P GP F ) G )are the Courant-Fischer numbers from the top of the operators (P F P G ) F and (P G P F ) G, is called the sequence of largest discrete principal angles between subspaces F and G. (1, 1/2,... ) (π/2, π/2, 1, 1/2,... ) = (1, 1/2,... ). Principal vectors, subspaces and invariant subspaces can be defined in the discrete case and have similar properties.

31 Changes in the smallest discrete angles where subspaces vary Theorem 22 Let θ k and ˆθ k be the k-th smallest discrete angles between the subspaces F and G, F and G, respectively, k = 1,...,q, q = min{dimf, dimg}. Then for k = 1,...,q, cos(θ k ) cos(ˆθ k ) κ gap(g, G), where κ = max{cos(θ min {(G + G) G, F}); cos(θ min {(G + G) G, F})}. We were not able to obtain similar result for the cosines of largest discrete angles. The proof is based on s-numbers which are defined only from the top. In the finite dimensional case both results are the same and are proven in Knyazev and Argentati [2002].

32 Majorization preliminaries Definition (Markus [1964], Marshall and Olkin [1979]) Assume that r = (r 1, r 2,...) and t = (t 1, t 2,...) are (finite or infinite) bounded sequences of real numbers. We say that t weakly majorizes r and write r w t, if sup π k i=1 r πi sup π k t πj, j=1 where the suprema are taken over all k-element permutations. r and t nonincreasing, r w t; Φ(x) convex, Φ( ) = 0. Then P k j=1 Φ(r j) P k j=1 Φ(t j), (k=1,2,...) (Gohberg and Kreĭn [1969]). Theorem 23 p w r and s w t imply (p s) w (r t). In the finite dimensional case this result is known.

33 Generalization of the Lidskiĭ Mirsky Wielandt theorem Theorem 24 Let A, B B(H) be selfadjoint operators on H. Then for any choice of indices 1 i 1 < < i k, for any k 1 k λ i j (A + B) j=1 k λ i j (A) + j=1 k λ j (B). j=1 The Lidskiĭ Mirsky Wielandt theorem is one of the major tools. Corollary 1 Under the assumptions of Theorem 24, λ (A + B) λ (A) w λ (B), λ (A) λ (B) w λ (A B), λ (A) λ (B) w λ (A B).

34 Majorization for discrete angles Theorem 25 Let F, G, G be subspaces of H, and P G P G < 1 holds. Assume also that max{dimf G, dimf G} = max{dimf G, dimf G} <. Then sinθ d (F, G) sinθ d (F, G) w sinθ d (G, G) holds, where Θ d (F, G), Θ d (F, G) and Θ d (G, G) are the sequences of largest discrete angles between corresponding subspaces.

35 Majorization for discrete angles (cont.) Theorem 26 Let F, G, G be subspaces of H, P G P G < 1, and dimf G <, dimf G <, dimf G = dimf G <. Then cos Θ d (F, G) cos Θ d (F, G) w sinθ d (G, G), holds, where Θ d (F, G) and Θ d (F, G) are the sequences of smallest discrete angles and Θ d (G, G) is the sequence of largest discrete angles between corresponding subspaces. We were not able to obtain the result of Theorem 25 for the sines of smallest angles and the result of Theorem 26 for the cosines of largest angles. In finite dimensional case both results are known Knyazev and Argentati [2005].

36 Majorization for discrete angles (cont.) Theorem 27 Under the assumptions of Theorem 25, sin 2 Θ d (F, G) sin2 Θ d (F, G) w sinθ d (G, G), cos 2 Θ d (F, G) cos2 Θ d (F, G) w sinθ d (G, G), and under the assumptions of Theorem 26, cos 2 Θ d (F, G) cos2 Θ d (F, G) w sinθ d (G, G), sin 2 Θ d (F, G) sin2 Θ d (F, G) w sinθ d (G, G).

37 Estimates for proximity of Ritz values Estimates for proximity of the sets of Ritz values; Majorization for Ritz values in the discrete case. Definition Let X H be a subspace of the Hilbert space H. Then Σ((P X A) X ) is called the set of Ritz values of an operator A with respect to the trial subspace X. Connection to the angles?

38 Extending to an orthogonal projector The first step Theorem 28 (Halmos [1969]; Riesz and Sz.-Nagy [1990]) A B(H) selfadjoint nonnegative contraction. Then Â, defined on H H as A A(I A) Â = A(I A) I A is an orthogonal projector.

39 Connection of angles to Ritz values The second step Theorem 29 The Ritz values of A and  are the same: Σ((P X A) X ) = Σ((P ˆXÂ) ˆX), ˆX = X 0 Ĥ = H H H. 0 The third step: define subspace Ẑ s.t. P Ẑ =  Σ((P X A) X ) = Σ((P ˆXPẐ) ˆX) = cos 2 (Θ( ˆX, Ẑ)). For the finite dimensional case see Knyazev and Argentati [2005].

40 Estimates for proximity of the sets of Ritz values Theorem 30 Let A B(H) be a selfadjoint operator and X and Y be subspaces of H. Then the following inequality dist(σ(p X AP X ), Σ(P Y AP Y )) (M m) P X P Y holds, where m and M are the bounds of the spectrum of A. Theorem 31 Assume A and B are linear bounded operators on a Hilbert space H and the equalities Σ(AA ) = Σ(A A) and Σ(BB ) = Σ(B B) hold. Then the following inequality holds: dist(σ( A ), Σ( B )) A B.

41 Majorization for Ritz values in discrete case Definition Let A B(H) be selfadjoint and X be a subspace of H. The sequence of Courant-Fischer numbers Λ ((P X A) X ) is called the sequence of the discrete Ritz values from the bottom of the operator A with respect to the trial subspace X. Theorem 32 Let A B(H) be selfadjoint and X and Y be subspaces of H and codimx = codimy <. Then Λ ((P X A) X ) Λ ((P Y A) Y ) w (M m) sinθ d (X, Y), where m and M are the greatest lower and least upper bounds of Σ(A). For Ritz values from the top we use an identity Λ ((P X A) X ) = Λ ( (P X A) X ).

42 1 0 Analysis of convergence of domain decomposition methods u v dx = 1 0 fvdx, v H = H 1 0([0, 1]); u H. Ω 1 = [0, α]; Ω 2 = [β,1]; 0 < β < α < 1; an overlap [β, α]. F = {f H f(x) = 0, x [α,1]} G = {g H g(x) = 0, x [0, β]} e k+1 = P G P F e k, k = 0, 1,...; (I P F P G P F )e = 0. e k+1 = [I (P F + P G )]e k, k = 0, 1,...; (P F + P G )e = 0.

43 Let θ = arccos Theorem 33 Analysis of the convergence (cont.) ( β(1 α) α(1 β) ). Σ(I P F P G P F ) = Σ(I P G P F ) = {sin 2 (θ)} {1}. Following Bjørstad and Mandel [1991], Vidav [1977], we prove Theorem 34 Σ(P F + P G ) = {1, 1 ± cos(θ), 2}. The conjugate gradient method converges for multiplicative Schwarz in two iterations and for additive Schwarz in four iterations.

44 Application of angles between subspaces in microarray data analysis The angles between subspaces are closely related to canonical correlations in statistics. We propose a new application of the angles between subspaces for microarray data analysis. For this purpose, we develop a public supplement to MALTAB s BioInformatics toolbox that performs single-array and comparison analysis for Affymetrix microarray data analogously to the Affymetrix software. We propose and implement a novel multi-comparison analysis of the Affymetrix microarray data using angles between subspaces.

45 References N. I. Akhiezer and I. M. Glazman. Theory of linear operators in Hilbert space. Vol. I, volume 9 of Monographs and Studies in Mathematics. Pitman (Advanced Publishing Program), Boston, Mass., 1981. ISBN 0-273-08495-X. Translated from the third Russian edition by E. R. Dawson, Translation edited by W. N. Everitt. Petter E. Bjørstad and Jan Mandel. On the spectra of sums of orthogonal projections with applications to parallel computing. BIT, 31(1):76 88, 1991. ISSN 0006-3835. Chandler Davis and W. M. Kahan. The rotation of eigenvectors by a perturbation. III. SIAM J. Numer. Anal., 7:1 46, 1970. ISSN 1095-7170. Frank Deutsch. The angle between subspaces of a Hilbert space. In

46 Approximation theory, wavelets and applications (Maratea, 1994), volume 454 of NATO Adv. Sci. Inst. Ser. C Math. Phys. Sci., pages 107 130. Kluwer Acad. Publ., Dordrecht, 1995. Nelson Dunford and Jacob T. Schwartz. Linear operators. Part II. Wiley Classics Library. John Wiley & Sons Inc., New York, 1988. ISBN 0-471-60847-5. Spectral theory. Selfadjoint operators in Hilbert space, With the assistance of William G. Bade and Robert G. Bartle, Reprint of the 1963 original, A Wiley-Interscience Publication. I. C. Gohberg and M. G. Kreĭn. Introduction to the theory of linear nonselfadjoint operators. Translated from the Russian by A. Feinstein. Translations of Mathematical Monographs, Vol. 18. American Mathematical Society, Providence, R.I., 1969.

47 P. R. Halmos. Two subspaces. Trans. Amer. Math. Soc., 144:381 389, 1969. ISSN 0002-9947. Tosio Kato. Perturbation theory for linear operators. Classics in Mathematics. Springer-Verlag, Berlin, 1995. ISBN 3-540-58661-X. Reprint of the 1980 edition. A. V. Knyazev and M. E. Argentati. Principal angles between subspaces in an A-based scalar product: algorithms and perturbation estimates. SIAM J. Sci. Comput., 23(6):2008 2040 (electronic), 2002. ISSN 1064-8275. A. V. Knyazev and M. E. Argentati. Majorization for changes in angles between subspaces, Ritz values, and graph laplacian spectra. SIAM Review, accepted, 2005. A. S. Markus. Eigenvalues and singular values of the sum and product of

48 linear operators. Uspehi Mat. Nauk, 19(4 (118)):93 123, 1964. ISSN 0042-1316. Albert W. Marshall and Ingram Olkin. Inequalities: theory of majorization and its applications, volume 143 of Mathematics in Science and Engineering. Academic Press Inc. [Harcourt Brace Jovanovich Publishers], New York, 1979. ISBN 0-12-473750-1. Vlastimil Pták. Perturbations and continuity of the spectrum. In Proceedings of the International Conference on Operator Algebras, Ideals, and their Applications in Theoretical Physics (Leipzig, 1977), pages 177 186, Leipzig, 1978. Teubner. Michael Reed and Barry Simon. Methods of modern mathematical physics. IV. Analysis of operators. Academic Press [Harcourt Brace Jovanovich Publishers], New York, 1978. ISBN 0-12-585004-2.

49 Frigyes Riesz and Béla Sz.-Nagy. Functional analysis. Dover Books on Advanced Mathematics. Dover Publications Inc., New York, 1990. ISBN 0-486-66289-6. Translated from the second French edition by Leo F. Boron, Reprint of the 1955 original. Ivan Vidav. The norm of the sum of two projections. Proc. Amer. Math. Soc., 65(2):297 298, 1977. ISSN 0002-9939. P. A. Wedin. On angles between subspaces of a finite-dimensional inner product space. In Bo Kȧgström and Axel Ruhe, editors, Matrix Pencils. Proceedings of the Conference Held at Pite Havsbad, March 22 24, 1982, pages 263 285. Springer Verlag, Berlin, 1983.