Matrix Perturbation Theory

5 Matrix Perturbation Theory Ren-Cang Li University of Texas at Arlington 5 Eigenvalue Problems 5-5 Singular Value Problems 5-6 53 Polar Decomposition 5-7 54 Generalized Eigenvalue Problems 5-9 55 Generalized Singular Value Problems 5-56 Relative Perturbation Theory for Eigenvalue Problems 5-3 57 Relative Perturbation Theory for Singular Value Problems 5-5 References 5-6 There is a vast amount of material in matrix (operator) perturbation theory Related books that are worth mentioning are [SS90] [Par98] [Bha96] [Bau85] and [Kat70] In this chapter we attempt to include the most fundamental results up to date except those for linear systems and least squares problems for which the reader is referred to Section 38 and Section 396 Throughout this chapter UI denotes a general unitarily invariant norm Two commonly used ones are the spectral norm and the Frobenius norm F 5 Eigenvalue Problems The reader is referred to Sections 43 4 and 4 for more information on eigenvalues and their locations Definitions: Let A C n n A scalar vector pair (λ x) C C n is an eigenpair of A if x 0 and Ax = λx A vector scalar vector triplet (y λ x) C n C C n is an eigentriplet if x 0 y 0 and Ax = λx y A = λy The quantity cond(λ) = x y y x is the individual condition number for λ where (y λ x) C n C C n is an eigentriplet Let σ (A) ={λ λ λ n } the multiset of A s eigenvalues and set = diag(λ λ λ n ) τ = diag(λ τ () λ τ () λ τ (n) ) 5-

5- Handbook of Linear Algebra where τ is a permutation of { n} For real ie all λ j s are real = diag(λ λ λ n ) is in fact a τ for which the permutation τ makes λ τ ( j ) = λ j for all j Given two square matrices A and A the separation sep(a A )betweena and A is defined as [SS90 p 3] sep(a A ) = inf XA A X X = A is perturbed to à = A + A The same notation is adopted for à except all symbols with tildes Let X Y C n k with rank(x) = rank(y ) = k Thecanonical angles between their column spaces are θ i = arc cos σ i where{σ i } i= k are the singular values of (Y Y ) / Y X(X X) / Define the canonical angle matrix between X and Y as (X Y ) = diag(θ θ θ k ) For k = ie x y C n (both nonzero) we use (x y) instead to denote the canonical angle between the two vectors Facts: [SS90 p 68] (Elsner) max min λ i λ j ( A + à i j ) /n A /n [SS90 p 70] (Elsner) There exists a permutation τ of { n} such that n τ ( A + à ) /n A /n 3 [SS90 p 83] Let (y µ x) be an eigentriplet of A A changes µ to µ + µ with µ = y ( A)x + O( A y x ) and µ cond(µ) A + O( A ) 4 [SS90 p 05] If A and A + A are Hermitian then UI A UI 5 [Bha96 p 65] (Hoffman Wielandt)IfA and A+ A are normal then there exists a permutation τ of { n} such that τ F A F 6 [Sun96] If A is normal then there exists a permutation τ of { n} such that τ F n A F 7 [SS90 p 9] (Bauer Fike) IfA is diagonalizable and A = X X is its eigendecomposition then max min λ i λ j X ( A)X p κ p (X) A p i j 8 [BKL97] Suppose both A and à are diagonalizable and have eigendecompositions A = X X and à = X X (a) There exists a permutation τ of { n} such that τ F κ (X)κ ( X) A F (b) UI κ (X)κ ( X) A UI for real and

Matrix Perturbation Theory 5-3 9 [KPJ8] Let residuals r = A x µ x and s = ỹ A µỹ where x = ỹ = and let ε = max { r s } The smallest error matrix A in the -norm for which (ỹ µ x) isanexact eigentriplet of à = A + A satisfies A = ε and µ µ cond( µ) ε + O(ε ) for some µ σ (A) 0 [KPJ8] [DK70][Par98 pp 73 44] Suppose A is Hermitian and let residual r = A x µ x and x = (a) The smallest Hermitian error matrix A (in the -norm) for which ( µ x) is an exact eigenpair of à = A + A satisfies A = r (b) µ µ r for some eigenvalue µ of A (c) Let µ be the closest eigenvalue in σ (A)to µ and x be its associated eigenvector with x = and let η = µ λ Ifη>0 then min µ λ σ (A) µ µ r η sin ( x x) r η Let A be Hermitian X C n k have full column rank and M C k k be Hermitian having eigenvalues µ µ µ k Set R = AX XM There exist k eigenvalues λ i λ i λ ik of A such that the following inequalities hold Note that subset {λ i j } k j= may be different at different occurrences (a) [Par98 pp 53 60] [SS90 Remark 46 p 07] (Kahan Cao Xie Li) max µ j λ i j R j k σ min (X) k (µ j λ i j ) R F σ min (X) j= (b) [SS90 pp 54 57] [Sun9] If X X = I and M = X AX and if all but k of A s eigenvalues differ from every one of M s by at least η>0 and ε F = R F /η < then k (µ k λ i j ) R F η ε F j = (c) [SS90 pp 54 57] [Sun9] If X X = I and M = X AX and there is a number η>0 such that either all but k of A s eigenvalues lie outside the open interval (µ η µ k + η)orallbut k of A s eigenvalues lie inside the closed interval [µ l + η µ l+ η] for some l k and ε = R /η < then max µ j λ i j R j k η ε [DK70] Let A be Hermitian and have decomposition X A[X X ] = X [ A A ]

5-4 Handbook of Linear Algebra where [X X ] is unitary and X C n k LetQ C n k have orthonormal columns and for a k k Hermitian matrix M set R = AQ QM Let η = min µ ν over all µ σ (M) and ν σ (A ) If η>0 then sin (X Q) F R F η 3 [LL05] Let M E M 0 A = E H Ã = 0 H be Hermitian and set η = min µ ν over all µ σ (M) and ν σ (H) Then max j n λ j λ j E η + η + 4 E 4 [SS90 p 30] Let [X Y ] be unitary and X C n k and let X A G A[X Y ] = E A Y Assume that σ (A ) σ (A ) = and set η = sep(a A ) If G E <η /4 then there is a unique W C (n k) k satisfying W E /η such that [ X Ỹ] is unitary and X [Ã A[ X G ] Ỹ] = where Thus tan (X X ) < E η Ỹ 0 Ã X = (X + Y W)(I + W W) / Ỹ = (Y X W )(I + WW ) / Ã = (I + W W) / (A + GW)(I + W W) / Ã = (I + WW ) / (A WG)(I + WW ) / Examples: Bounds on τ UI are in fact bounds on λ j λ τ ( j ) in disguise only more convenient and concise For example for UI = (spectral norm) τ = max j λ j λ τ ( j ) and for UI = F (Frobenius norm) [ n τ F = j = λ j λ τ ( j ) ] / Let A Ã Cn n as follows where ε>0 µ µ µ A = Ã = µ µ ε µ It can be seen that σ (A) = {µ µ} (repeated n times) and the characteristic polynomial det(ti Ã) = (t µ)n ε which gives σ (Ã) ={µ + ε/n e ijπ/n 0 j n } Thus

Matrix Perturbation Theory 5-5 λ µ =ε /n = A /n This shows that the fractional power A /n in Facts and cannot be removed in general 3 Consider 3 A = 0 4 5 0 0 400 A s eigenvalues are easily read off and 0 0 0 is perturbed by A = 0 0 0 000 0 0 λ = x = [ 0 0] T y = [0885 0553 0090] T λ = 4 x = [05547 083 0] T y = [0 0000 0000] T λ 3 = 400 x 3 = [05547 083 0000] T y 3 = [0 0 ] T On the other hand à s eigenvalues computed by MATLAB s eig are λ = 000 λ = 3947 λ 3 = 4058 The following table gives λ j λ j with upper bounds up to the st order by Fact 3 j cond(λ j ) cond(λ j ) A λ j λ j 070 000 0000 60 0 3 60 0057 3 60 0 3 60 0057 We see that cond(λ j ) A gives a fairly good error bound for j = but dramatically worse for j = 3 There are two reasons for this: One is in the choice of A and the other is that A s order of magnitude is too big for the first order bound cond(λ j ) A to be effective for j = 3 Note that A has the same order of magnitude as the difference between λ and λ 3 and that is too big usually For better understanding of this first order error bound the reader may play with this y j x example with A = ε j y j x for various tiny parameters ε j 4 Let = diag(c c c k ) and Ɣ = diag(s s s k ) where c j s j 0 and c j + s j = for all j The canonical angles between I k X = Q 0 V Y = Q Ɣ U 0 0 are θ j = arccos c j j = k whereq U V are unitary On the other hand every pair of X Y C n k with k n and X X = Y Y = I k having canonical angles arccos c j can be represented this way [SS90 p 40] 5 Fact 3 is most useful when E is tiny and the computation of A s eigenvalues is then decoupled into two smaller ones In eigenvalue computations we often seek unitary [X X ] such that X M E X M 0 A[X X ] = Ã[X X ] = E H 0 H X and E is tiny Since a unitarily similarity transformation does not alter eigenvalues Fact 3 still applies 6 [LL05] Consider the Hermitian matrix α ε A = ε β X

5-6 Handbook of Linear Algebra where α>βand ε>0 It has two eigenvalues and λ ± = α + β ± (α β) + 4ε { } λ 0 < + α ε = β λ (α β) + (α β) + 4ε The inequalities in Fact 3 become equalities for the example 5 Singular Value Problems The reader is referred to Section 56 Chapters 7 and 45 for more information on singular value decompositions Definitions: B C m n has a (first standard form) SVD B = U V whereu C m m and V C n n are unitary and = diag(σ σ ) R m n is leading diagonal (σ j starts in the top-left corner) with all σ j 0 Let SV(B) ={σ σ σ min{mn} } the set of B s singular values and σ σ 0 and let SV ext (B) = SV(B) unless m > n for which SV ext (B) = SV(B) {0 0} (additional m n zeros) A vector scalar vector triplet (u σ v) C m R C n is a singular-triplet if u 0 v 0 σ 0 and Bv = σ u B u = σ v B is perturbed to B = B + B The same notation is adopted for B except all symbols with tildes Facts: [SS90 p 04] (Mirsky) UI B UI Let residuals r = Bṽ µũ and s = B ũ µṽ and ṽ = ũ = (a) [Sun98] The smallest error matrix B (in the -norm) for which (ũ µ ṽ) is an exact singulartriplet of B = B + B satisfies B = εwhereε = max { r s } (b) µ µ εfor some singular value µ of B (c) Let µ be the closest singular value in SV ext (B)to µ and (u σ v) be the associated singular-triplet with u = v = and let η = min µ σ over all σ SV ext (B) and σ µ Ifη>0 then µ µ ε /η and [SS90 p 60] r sin (ũ u) + sin + s (ṽ v) η 3 [LL05] Let B F B = C m n B B 0 = E B 0 B where B C k k and set η = min µ ν over all µ SV(B ) and ν SV ext (B ) and ε = max{ E F } Then ε max σ j σ j j η + η + 4ε

Matrix Perturbation Theory 5-7 4 [SS90 p 60] (Wedin) LetB B C m n (m n) have decompositions [Ũ ] U B B[V V ] = 0 B B[Ṽ 0 B Ṽ] = 0 0 B U where [U U ] [V V ] [Ũ Ũ] and [Ṽ Ṽ] are unitary and U Ũ C m k V Ṽ C n k Set If SV( B ) SV ext (B ) = then Ũ R = B Ṽ Ũ B S = B Ũ Ṽ B R sin (U Ũ) F + sin (V Ṽ) F F + S F η where η = min µ ν over all µ SV( B ) and ν SV ext (B ) Examples: Let B = [ ] [ 3 0 3 4 0 3 B = ] [ = [e e ] ] e T e T Then σ = 0000 σ = 0999988 and σ = σ = Fact gives max σ j σ j 4 0 3 σ j σ j 5 0 3 j Let B be as in the previous example and let ṽ = e ũ = e µ = Then r = Bṽ µũ = 3 0 3 e and s = B ũ µṽ = 4 0 3 e Fact applies Note that without calculating SV(B) one may bound η needed for Fact (c) from below as follows Since B has two singular values that are near and µ = respectively with errors no bigger than 4 0 3 then η (+4 0 3 ) = 4 0 3 3 Let B and B be as in Example Fact 3 gives max σ j σ j 6 0 5 a much better bound j than by Fact 4 Let B and B be as in Example Note B s SVD there Apply Fact 4 with k = to give a similar bound as by Fact (c) 5 Since unitary transformations do not change singular values Fact 3 applies to B B C m n having decompositions U B B[V V ] = F U B B[V E B V ] = 0 0 B U where [U U ] and [V V ] are unitary and U C m k V C n k U j= 53 Polar Decomposition The reader is referred to Chapter 7 for definition and for more information on polar decompositions Definitions: B F m n is perturbed to B = B + B and their polar decompositions are B = QH B = Q H = (Q + Q)(H + H) where F = R or C B is restricted to F for B F

5-8 Handbook of Linear Algebra Denote the singular values of B and B as σ σ and σ σ respectively The condition numbers for the polar factors in the Frobenius norm are defined as cond F (X) = lim δ 0 sup B F δ X F for X = H or Q δ B is multiplicatively perturbed to B if B = D L BD R for some D L F m m and D R F n n B is said to be graded if it can be scaled as B = GS such that G is well-behaved (ie κ (G) isof modest magnitude) where S is a scaling matrix often diagonal but not required so for the facts below Interesting cases are when κ (G) κ (B) Facts: [CG00] The condition numbers cond F (Q) and cond F (H) are tabulated as follows where κ (B) = σ /σ n R C Factor Q m = n /(σ n + σ n ) /σ n m > n /σ n /σ n Factor H [Kit86] H F B F 3 [Li95] If m = n and rank(b) = n then m n ( + κ (B) ) + κ (B) 4 [Li95] [LS0] If rank(b) = n then ( Q UI Q F Q UI + σ n + σ n B F σ n + σ n σ n + σ n B UI ) B UI max{σ n σ n } 5 [Mat93] If B R n n rank(b) = n and B <σ n then Q UI B UI B ln ( B ) σ n + σ n where is the Ky Fan -norm ie the sum of the first two largest singular values (See Chapter 73) 6 [LS0] If B R n n rank(b) = n and B <σ n + σ n then Q F 4 σ n + σ n + σ n + σ n B F 7 [Li97] Let B and B = D BD L R having full column rank Then Q F I D + D L F L I + I D F R + D F R I F 8 [Li97] [Li05] Let B = GS and B = GS and assume that G and B have full column rank If G G < then

Matrix Perturbation Theory 5-9 where γ = Q F γ G G F ( ) ( H)S F γ G G + G F + ( G G ) Examples: Take both B and B to have orthonormal columns to see that some of the inequalities above on Q are attainable Let B = [ ] 0 50 = [ ] 0 99 498 5 0 and [ ] 43 35497 0 B = 407 354 0 obtained by rounding each entry of B to have five significant decimal digits B = QH can be read off above and B = Q H can be computed by Q = Ũ Ṽ and H = Ṽ Ṽ where B s SVD is Ũ Ṽ One has and SV(B) ={500 0 00 0 3 } SV( B) ={500 0 04 0 3 } B B F Q Q F H H F 3 0 3 3 0 3 0 6 3 0 6 0 3 0 3 Fact gives H F 3 0 3 and Fact 6 gives Q F 0 5 3 [Li97] and [Li05] have examples on the use of inequalities in Facts 7 and 8 54 Generalized Eigenvalue Problems The reader is referred to Section 43 for more information on generalized eigenvalue problems Definitions: Let A B C m n Amatrix pencil is a family of matrices A λb parameterized by a (complex) number λ The associated generalized eigenvalue problem is to find the nontrivial solutions of the equations Ax = λbx and/or y A = λy B where x C n y C m and λ C and (x λ y) is called a generalized eigentriplet A λb is regular if m = n and det(a λb) 0 for some λ C In what follows all pencils in question are assumed regular An eigenvalue λ is conveniently represented by a nonzero number pair so-called a generalizedeigenvalue α β interpreted as λ = α/β β = 0 corresponds to eigenvalue infinity

5-0 Handbook of Linear Algebra A generalized eigenpair of A λb refers to ( α β x) such that β Ax = α Bx wherex 0 and α + β > 0 A generalized eigentriplet of A λb refers to (y α β x) such that β Ax = α Bx and βy A = αy Bwherex 0 y 0 and α + β > 0 The quantity cond( α β ) = x y y Ax + y Bx is the individualconditionnumber for the generalized eigenvalue α β where (y α β x) is a generalized eigentriplet of A λb A λb is perturbed to à λ B = (A + A) λ(b + B) Let σ (A B) ={ α β α β α n β n } be the set of the generalized eigenvalues of A λb and set Z = [A B] C n n A λb is diagonalizable if it is equivalent to a diagonal pencil ie there are nonsingular X Y C n n such that Y AX = Y BX = where = diag(α α α n ) and = diag(β β β n ) A λb is a definite pencil if both A and B are Hermitian and γ (A B) = min x C n x = x Ax + i x Bx > 0 The same notation is adopted for à λ B except all symbols with tildes The chordal distance between two nonzero pairs α β and α β is χ ( α β α β ) = βα αβ α + β α + β Facts: [SS90 p 93] Let (y α β x) be a generalized eigentriplet of A λb[ A B] changes α β = y Ax y Bx to α β = α β + y ( A)x y ( B)x +O(ε ) where ε = [ A B] and χ ( α β α β ) cond( α β ) ε + O(ε ) [SS90 p 30] [Li88] If A λb is diagonalizable then 3 [Li94 Lemma 33] (Sun) max min χ ( α i β i α j β j ) κ (X) sin (Z Z ) i j sin (Z Z Z ) UI Z UI max{σ min (Z) σ min ( Z)} where σ min (Z) isz s smallest singular value 4 The quantity γ (A B) is the minimum distance of the numerical range W(A + ib) to the origin for definite pencil A λb 5 [SS90 p 36] Suppose A λb is a definite pencil If à and B are Hermitian and [ A B] < γ (A B) then à λ B is also a definite pencil and there exists a permutation τ of { n} such that max χ( α j β j α τ ( j ) β τ ( j ) ) [ A B] j n γ (A B) 6 [SS90 p 38] Definite pencil A λb is always diagonalizable: X AX = and X BX = and with real spectra Facts 7 and 0 apply

Matrix Perturbation Theory 5-7 [Li03] Suppose A λb and à λ B are diagonalizable with real spectra ie Y AX = Y BX = and Ỹ à X = Ỹ B X = and all α j β j and all α j β j are real Then the follow statements hold where = diag(χ( α β α τ () β τ () ) χ( α n β n α τ (n) β τ (n) )) for some permutation τ of { n} (possibly depending on the norm being used) In all cases the constant factor π/ can be replaced by for the -norm and the Frobenius norm (a) UI π κ (X)κ ( X) sin (Z Z ) UI (b) If all α j + β j = α j + β j = in their eigendecompositions then UI π X Y X Ỹ [ A B] UI 8 Let residuals r = β A x α B x and s = βỹ A αỹ Bwhere x = ỹ = The smallest error matrix [ A B] in the -norm for which (ỹ α β x) is an exact generalized eigentriplet of à λ B satisfies [ A B] = ε whereε = max { r s } and χ ( α β α β ) cond( α β ) ε + O(ε ) for some α β σ (A B) 9 [BDD00 p 8] Suppose A and B are Hermitian and B is positive definite and let residual r = A x µb x and x = (a) For some eigenvalue µ of A λb µ µ r B x B B r where z M = z Mz (b) Let µ be the closest eigenvalue to µ among all eigenvalues of A λb and x its associated eigenvector with x = and let η = min µ ν over all other eigenvalues ν µ of A λb Ifη>0 then µ µ ( η r B x B ) B r η sin ( x x) B κ (B) r η 0 [Li94] Suppose A λb and à λ B are diagonalizable and have eigendecompositions ] ] Y [ A[X X ] = Y [ B[X X ] = Y Y X = [W W ] and the same for à λ B except all symbols with tildes where X Y W C n k C k k Suppose α j + β j = α j + β j = for j n in the eigendecompositions and set η = min χ ( α β α β ) taken over all α β σ ( ) and α β σ ( ) If η>0 then sin (X X ) F X W η Ỹ ( Z Z) [ X X ] F

5- Handbook of Linear Algebra 55 Generalized Singular Value Problems Definitions: Let A([ ]) C m n and B C l n A matrix pair {A B} is an (m l n)-grassmann matrix pair if rank A B = n In what follows all matrix pairs are (m l n)-grassmann matrix pairs A pair α β is a generalized singular value of {A B} if det(β A A α B B) = 0 α β 0 0 α β 0 ie α β = µ ν for some generalized eigenvalue µ ν of matrix pencil A A λb B Generalized Singular Value Decomposition (GSVD) of {A B}: U AX = A V BX = B where U C m m V C l l are unitary X C n n is nonsingular A = diag(α α )isleading diagonal (α j starts in the top left corner) and B = diag( β n β n )istrailing diagonal (β j ends in the bottom-right corner) α j β j 0 and α j + β j = for j n (Setsomeα j = 0 and/or some β j = 0 if necessary) {A B} is perturbed to {Ã B} ={A + A B + B} Let SV(A B) [ ={ α ] β α β α n β n } be the set of the generalized singular values of {A B} A and set Z = C B (m+l) n The same notation is adopted for {Ã B} except all symbols with tildes Facts: If {A B} is an (m l n)-grassmann matrix pair then A A λb B is a definite matrix pencil [Van76] The GSVD of an (m l n)-grassmann matrix pair {A B} exists 3 [Li93] There exist permutations τ and ω of { n} such that n max χ( α i β i α τ (i) β τ (i) ) sin (Z Z) j n j= [ χ ( α i β i α ω(i) β ω(i) )] sin (Z Z) F 4 [Li94 Lemma 33] (Sun) sin (Z Z) Z UI Z UI max{σ min (Z) σ min ( Z)} where σ min (Z) isz s smallest singular value 5 [Pai84] If αi + βi = α i + β i = for i = n then there exists a permutation ϖ of { n} such that n [(α j α ϖ ( j ) ) + (β j β ] ϖ ( j ) ) min Z 0 Z 0 Q F Q unitary j = where Z 0 = Z(Z Z) / and Z 0 = Z( Z Z) /

Matrix Perturbation Theory 5-3 6 [Li93] [Sun83] Perturbation bounds on generalized singular subspaces (those spanned by one or a few columns of U V and X in GSVD) are also available but it is quite complicated 56 Relative Perturbation Theory for Eigenvalue Problems Definitions: Let scalar α be an approximation to α and p Define relative distances between α and α as follows For α + α 0 d(α α) = α α α α = (classical measure) α α α ϱ p (α α) = p α p + α ([Li98]) p ζ (α α) = α α α α ([BD90] [DV9]) ς(α α) = ln( α/α) for αα > 0 ([LM99a] [Li99b]) and d(0 0) = ϱ p (0 0) = ζ (0 0) = ς(0 0) = 0 A C n n is multiplicatively perturbed to à if à = D L AD R for some D L D R C n n Denote σ (A) ={λ λ λ n } and σ (Ã) ={ λ λ λ n } A C n n is said to be graded if it can be scaled as A = S HS such that H is well-behaved (ie κ (H) is of modest magnitude) where S is a scaling matrix often diagonal but not required so for the facts below Interesting cases are when κ (H) κ (A) Facts: [Bar00] ϱ p ( ) is a metric on C for p Let A à = D AD C n n be Hermitian where D is nonsingular (a) [HJ85 p 4] (Ostrowski) There exists t j satisfying λ min (D D) t j λ max (D D) such that λ j = t j λ j for j = n and thus (b) [LM99] [Li98] max j n d(λ j λ j ) I D D diag (ς(λ λ ) ς(λ n λ ) n ) UI ln(d D) UI diag (ζ (λ λ ) ζ (λ n λ ) n ) UI D D UI 3 [Li98] [LM99] Let A = S HS be a positive semidefinite Hermitian matrix perturbed to à = S (H + H)S Suppose H is positive definite and H / ( H)H / < and set M = H / SS H / M = DMD where D = [ I + H / ( H)H /] / = D Then σ (A) = σ (M) and σ (Ã) = σ ( M) and the inequalities in Fact above hold with D here Note that

5-4 Handbook of Linear Algebra D D UI H / ( H)H / UI H / ( H)H / H H H H UI 4 [BD90] [VS93] Suppose A and à are Hermitian and let A =(A ) / be the positive semidefinite square root of A If there exists 0 δ< such that x ( A)x δx A x for all x C n then either λ j = λ j = 0or δ λ j /λ j + δ 5 [Li99a] Let Hermitian A à = D AD have decompositions ] X [A A[X X ] = X Ã[ X X ] = A X X [à à ] where [X X ] and [ X X ] are unitary and X X C n k Ifη = min ϱ (µ µ) > 0 µ σ (A ) µ σ (à ) then sin (X X ) F (I D )X F + (I D )X F η 6 [Li99a] Let A = S HSbe a positive semidefinite Hermitian matrix perturbed to à = S (H + H)S having decompositions in notation the same as in Fact 5 Let D = [ I + H / ( H)H /] / Assume H is positive definite and H / ( H)H / < If η ζ = ζ (µ µ) > 0 then sin (X X ) F D D F η ζ min µ σ (A ) µ σ (à ) Examples: [DK90] [EI95] Let A be a real symmetric tridiagonal matrix with zero diagonal and off-diagonal entries b b b n Suppose à is identical to A except for its off-diagonal entries which change to β b β b β n b n where all β i are real and supposedly close to Then à = DAD where D = diag(d d d n ) with d k = β β 3 β k β β 4 β k d k+ = β β 4 β k β β 3 β k Let β = n j= max{β j /β j } Then β I D βi and Fact and Fact 5 apply Now if all ε β j + ε then ( ε) n β β ( + ε) n Let A = SHS with S = diag( 0 0 0 3 ) and 0 0 A = 0 0 0 4 0 4 H = 0 0 0 0 0 4 0 6 0

Matrix Perturbation Theory 5-5 Suppose that each entry A ij of A is perturbed to A ij (+δ ij ) with δ ij ε Then ( H) ij ε H ij and thus H ε Since H 0/8 Fact 3 implies ζ (λ j λ j ) 5ε/ 5ε 5ε 57 Relative Perturbation Theory for Singular Value Problems Definitions: B C m n is multiplicatively perturbed to B if B = D BD L R for some D L C m m and D R C n n Denote the singular values of B and B as SV(B) ={σ σ σ min{mn} } SV( B) ={ σ σ σ min{mn} } B is said to be (highly) graded if it can be scaled as B = GS such that G is well-behaved (ie κ (G) is of modest magnitude) where S is a scaling matrix often diagonal but not required so for the facts below Interesting cases are when κ (G) κ (B) Facts: Let B B = D BD L R C m n whered L and D R are nonsingular (a) [EI95] For j n (b) [Li98] [LM99] σ j D L D R σ j σ j D L D R diag (ζ (σ σ ) ζ (σ n σ n )) UI D L D L UI + D R D R UI [Li99a] Let B B = D BD L R C m n (m n) have decompositions [Ũ ] U B B[V V ] = 0 B B[Ṽ 0 B Ṽ] 0 = 0 B U where [U U ] [V V ] [Ũ Ũ] and [Ṽ Ṽ] are unitary and U Ũ C m k V Ṽ C n k Set U = (U Ũ) V = (V Ṽ) If SV(B ) SV ext ( B ) = then sin U F + sin V F η [ (I D L )U F + (I D R )V F Ũ + (I D )U L F + (I D R )V F] / where η = min ϱ (µ µ) overallµ SV(B ) and µ SV ext ( B ) 3 [Li98] [Li99a] [LM99] Let B = GS and B = GS be two m n matrices where rank(g) = n and let G = G G Then B = DBwhereD = I + ( G)G Fact and Fact apply with D L = D and D R = I Note that ( ) D D ( G)G UI UI + ( G)G

5-6 Handbook of Linear Algebra Examples: [BD90] [DK90] [EI95] B is a real bidiagonal matrix with diagonal entries a a a n and offdiagonal (the one above the diagonal) entries are b b b n B is the same as B except for its diagonal entries which change to α a α a α n a n and its off-diagonal entries which change to β b β b β n b n Then B = D BD L R with ( D L = diag α α α α α α 3 β ( D R = diag β β β α α α ) β β ) Let α = n j= max{α j /α j } and β = n j= max{β j /β j } Then (αβ) ( D L D R ) DL D R αβ and Fact and Fact apply Now if all ε α i β j (αβ) ( + ε) n Consider block partitioned matrices + ε then ( ε) n (αβ) B B B = 0 B B 0 B = = B 0 B I B B = BD R 0 I By Fact ζ (σ j σ j ) B B Interesting cases are when B B is tiny enough to be treated as zero and so SV( B) approximates SV(B) well This situation occurs in computing the SVD of a bidiagonal matrix Author Note: Supported in part by the National Science Foundation under Grant No DMS-050664 References [BDD00] Z Bai J Demmel J Dongarra A Ruhe and H van der Vorst (Eds) Templates for the Solution of Algebraic Eigenvalue Problems: A Practical Guide SIAM Philadelphia 000 [BD90] J Barlow and J Demmel Computing accurate eigensystems of scaled diagonally dominant matrices SIAM J Numer Anal 7:76 79 990 [Bar00] A BarrlundThe p-relative distance is a metric SIAM J Matrix Anal Appl ():699 70 000 [Bau85] HBaumgärtel Analytical Perturbation Theory for Matrices and Operators Birkhäuser Basel 985 [Bha96] R Bhatia Matrix Analysis Graduate Texts in Mathematics Vol 69 Springer New York 996 [BKL97] R Bhatia F Kittaneh and R-C Li Some inequalities for commutators and an application to spectral variation II Lin Multilin Alg 43(-3):07 0 997 [CG00] F Chatelin and S Gratton On the condition numbers associated with the polar factorization of a matrix Numer Lin Alg Appl 7:337 354 000 [DK70] C Davis and W Kahan The rotation of eigenvectors by a perturbation III SIAM J Numer Anal 7: 46 970 [DK90] J Demmel and W Kahan Accurate singular values of bidiagonal matrices SIAM J Sci Statist Comput :873 9 990 [DV9] J Demmel and K Veselić Jacobi s method is more accurate than QR SIAM J Matrix Anal Appl 3:04 45 99

Matrix Perturbation Theory 5-7 [EI95] SC Eisenstat and ICF Ipsen Relative perturbation techniques for singular value problems SIAM J Numer Anal 3:97 988 995 [HJ85] RA Horn and CR Johnson Matrix Analysis Cambridge University Press Cambridge 985 [KPJ8] W Kahan BN Parlett and E Jiang Residual bounds on approximate eigensystems of nonnormal matrices SIAM J Numer Anal 9:470 484 98 [Kat70] T Kato Perturbation Theory for Linear Operators nd ed Springer-Verlag Berlin 970 [Kit86] F Kittaneh Inequalities for the schatten p-norm III Commun Math Phys 04:307 30 986 [LL05] Chi-Kwong Li and Ren-Cang Li A note on eigenvalues of perturbed Hermitian matrices Lin Alg Appl 395:83 90 005 [LM99] Chi-Kwong Li and R Mathias The Lidskii Mirsky Wielandt theorem additive and multiplicative versions Numer Math 8:377 43 999 [Li88] Ren-Cang Li A converse to the Bauer-Fike type theorem Lin Alg Appl 09:67 78 988 [Li93] Ren-Cang Li Bounds on perturbations of generalized singular values and of associated subspaces SIAM J Matrix Anal Appl 4:95 34 993 [Li94] Ren-Cang Li On perturbations of matrix pencils with real spectra Math Comp 6:3 65 994 [Li95] Ren-Cang Li New perturbation bounds for the unitary polar factor SIAM J Matrix Anal Appl 6:37 33 995 [Li97] Ren-Cang Li Relative perturbation bounds for the unitary polar factor BIT 37:67 75 997 [Li98] Ren-Cang Li Relative perturbation theory: I Eigenvalue and singular value variations SIAM J Matrix Anal Appl 9:956 98 998 [Li99a] Ren-Cang Li Relative perturbation theory: II Eigenspace and singular subspace variations SIAM J Matrix Anal Appl 0:47 49 999 [Li99b] Ren-Cang Li A bound on the solution to a structured Sylvester equation with an application to relative perturbation theory SIAM J Matrix Anal Appl :440 445 999 [Li03] Ren-Cang Li On perturbations of matrix pencils with real spectra a revisit Math Comp 7:75 78 003 [Li05] Ren-Cang Li Relative perturbation bounds for positive polar factors of graded matrices SIAM J Matrix Anal Appl 7:44 433 005 [LS0] W Li and W Sun Perturbation bounds for unitary and subunitary polar factors SIAM J Matrix Anal Appl 3:83 93 00 [Mat93] R Mathias Perturbation bounds for the polar decomposition SIAM J Matrix Anal Appl 4:588 597 993 [Pai84] CC Paige A note on a result of Sun Ji-Guang: sensitivity of the CS and GSV decompositions SIAM J Numer Anal :86 9 984 [Par98] BN Parlett The Symmetric Eigenvalue Problem SIAM Philadelphia 998 [SS90] GW Stewart and Ji-Guang Sun Matrix Perturbation Theory Academic Press Boston 990 [Sun83] Ji-Guang Sun Perturbation analysis for the generalized singular value decomposition SIAM J Numer Anal 0:6 65 983 [Sun9] Ji-Guang Sun Eigenvalues of Rayleigh quotient matrices Numer Math 59:603 64 99 [Sun96]Ji-Guang Sun On the variation of the spectrum of a normal matrix Lin Alg Appl 46:5 3 996 [Sun98] Ji-Guang Sun Stability and accuracy perturbation analysis of algebraic eigenproblems Technical Report UMINF 98-07 Department of Computer Science Umeå Univeristy Sweden 998 [Van76] CF Van Loan Generalizing the singular value decomposition SIAM J Numer Anal 3:76 83 976 [VS93] Krešimir Veselić and Ivan Slapničar Floating-pointperturbations of Hermitian matrices Lin Alg Appl 95:8 6 993