PERTURBATION ANALYSIS OF THE EIGENVECTOR MATRIX AND SINGULAR VECTOR MATRICES. Xiao Shan Chen, Wen Li and Wei Wei Xu 1. INTRODUCTION A = UΛU H,

Transcription

1 TAIWAESE JOURAL O MATHEMATICS Vol. 6, o., pp , ebruary 0 This paper is available online at PERTURBATIO AALYSIS O THE EIGEVECTOR MATRIX AD SIGULAR VECTOR MATRICES Xiao Shan Chen, Wen Li and Wei Wei Xu Abstract. Let A be an n n Hermitian matrix and A = UΛU H be its spectral decomposition, where U is a unitary matrix of order n and Λ is a diagonal matrix. In this note we present the perturbation bound and condition number of the eigenvector matrix U of A with distinct eigenvalues. A perturbation bound of singular vector matrices is also given for a real n n or (n +) n matrix. The results are illustrated by numerical examples.. ITRODUCTIO Let A be a n n Hermitian matrix with the spectral decomposition (.) A = UΛU H, where U is an n n unitary matrix, U H denotes the conjugate transpose of U and Λ=diag(λ,λ,...,λ n ). Let B be an m n(m n) complex matrix with the singular value decomposition (.) B = W ΣV H, where the m m left singular vector matrix W and the n n right singular vector matrix V are unitary, and ( ) Σ (.3) Σ=, Σ 0 = diag(σ,...,σ n ), σ i 0, i=,...,n. Received ebruary 3, 00, accepted October 3, 00. Communicated by Wen-Wei Lin. 00 Mathematics Subject Classification: 655, Key words and phrases: Eigenvector matrix, Singular vector matrix, Condition number, robenius norm. This work is supported by the atural Science oundation of Guangdong Province ( , ) and by the ational atural Science oundation of China (097075). 79

2 80 Xiao Shan Chen, Wen Li and Wei Wei Xu The spectral decomposition of a Hermitian matrix and the singular value decomposition of a general matrix are very useful tools in many matrix computation problems (see, e.g.[, 5, 6]). In this paper we focus on perturbation analysis for the eigenvector matrix U in (.) and singular vector matrices W and V in (.). As to perturbation of eigenvalues, singular values, eigenspaces and singular subspaces have been studied [,, 4, 9, 0,, 5, 6, 7]. We know that eigenspaces and singular subspaces are usually much less sensitive to perturbations compared with the corresponding basis. It is well-known that the maximum eigenspace spanned by column vectors of U and the maximum singular subspaces spanned by column vectors of W and V are is not sensitive at all, but the eigenvector matrix U and the singular vector matrices W and V may be infinitely sensitive. The following simple example illustrates this case. Example.. Let A = I (i.e. identity matrix). Then A has the eigendecomposition (.) with U = I, Λ=A. If its perturbed matrix à has following form ( ) ε à =, ε where ε is a nonzero real number. It is easy to see that the eigenvector matrix of à has the following form d d Ũ = d d, where d = d =. Then for any nonzero real number ε, we have Ũ U =4 (d + d + d + d ) 4, where a denotes the conjugate of a complex number a. This example shows that the eigenvector matrix U isn t a continuous function of matrix elements. Based on the technique of splitting operators and Lyapunov majorants, perturbation bounds of four kinds of Schur decompositions have been presented by some authors(see, e.g.[3, 7, 8, 3, 4]). In this paper, we give the absolute and relative perturbation bounds and condition numbers of the eigenvector matrix U of the matrix A with distinct eigenvalues under Hermitian perturbations. We also present absolute and relative perturbation bounds and condition numbers of singular vector matrices W and V for the singular value decomposition of a real n n matrix with

3 Eigenvector Matrix and Singular Vector Matrices 8 distinct singular values and a real (n +) n matrix with distinct nonzero singular values, respectively. We use the following notation: Let A H and A T stand for the conjugate transpose and transpose of a matrix A, respectively. I n is the identity matrix of order n. The robenius norm and spectral norm of a matrix A are denoted by A and A, respectively. Let C m n (R m n ), C m n D (Rm n D ) and Cm n (Rm n ) denote the sets of complex(real) m n matrices, complex(real) m n diagonal matrices, and complex(real) m n matrices whose diagonal entries are zeros, respectively. Let X =(x ij ) C m n (R m n ), we defined X and X D by { { xij i j 0 i j (X ) ij = and (X 0 i = j D ) ij = i = j respectively. It is evident that any X C m n can be split uniquely as X = X + X D, or example we have a c e x ij X C m n, X D C m n D. b 0 b d = c 0 + f e f a 0 0 d A PERTURBATIO BOUD O THE EIGEVECTOR MATRIX In this section we study the perturbation bound of the eigenvector matrix U of A in the decomposition (.). irst we define the linear operator L : C n n C n n by ( ) (.) L τ X = (XΛ ΛX), ν where Λ=diag(λ,λ,...,λ n ) and τ,ν are positive parameters. The following lemma, whose proof is omitted, is similar to Theorem in [6]. Lemma.. The linear operator L is defined by (.). Then L is nonsingular if and only if all eigenvalues of Λ are simple, i. e. λ i λ j,i j, i, j =,,...,n. Let Λ=diag(λ,λ,...,λ n ), and let L be the operator defined by (.). ow we define the function { } η(λ) = min (XΛ ΛX) ν : X C n n, τ X =. It is easy to see that if L is nonsingular then

4 8 Xiao Shan Chen, Wen Li and Wei Wei Xu (.) η(λ) = L = τ ν min i j λ i λ j, where L is defined by L =max{ L (X) : X C n n, X =}. Lemma.. Let X C n n. Then we have (.3) (X H X) n X. Proof. [7]. oting that X H X is a Hermitian matrix, (.3) follows from (6) in In this section we have the main theorem below. Theorem.. Let A C n n be a Hermitian matrix with distinct eigenvalues and have the spectral decomposition (.). Let à = A+ A be a Hermitian matrix, and let ( ɛ = A, α = τ ) (.4) + A, η(λ) ν νη(λ) n where η(λ) is defined by (.). If ɛ satisfies the following condition (.5) ɛ α + τ +4α, then à has a spectral decomposition à = Ũ ΛŨ H such that (.6) Ũ U τ ɛ αɛ + 4αɛ τ ɛ b u(ɛ). Proof. otice that the robenius norm is unitarily invariant norm. Hence without loss of generality we may assume that U = I n in (.). Write Λ = Λ Λ and U = Ũ U = Ũ I n. Then by ÃŨ AU = Ũ Λ UΛ, we can obtain (.7) AŨ + A U = Ũ Λ + UΛ. Let (.8) E = Ũ H AŨ. oting that Ũ H = I n + U H, then from (.7) we get (.9) UΛ Λ U = E + U H Λ U U H UΛ Λ.

5 Eigenvector Matrix and Singular Vector Matrices 83 By the definition (.) of the operator L and (.9), we can get ( ) (.0) L τ U = ν E + ν ( U H Λ U U H UΛ). Choose a spectral decomposition of à so that the diagonal elements of Ũ are real. Then by U + U H + U H U =0, we get (.) U D = diag( U H U). rom Lemma., we know that L is nonsingular. Hence (.0)-(.) can be rewritten as a continuous mapping Φ:C n n C n n expressed by (.) τ U = ν L (E )+ ν L ( ( U H Λ U U H UΛ) ), U D = diag( U H U). Since ( U H Λ U U H UΛ) Lemma. we have = ( U H Λ U) ( U H U) Λ, from ( U H Λ U U H UΛ) ( U H Λ U) + ( U H U) Λ ) τ (+ A n τ U. otice that diag( U H U) U, E A. Hence the mapping Φ expressed by (.) satisfies τ U ɛ + α τ U, τ U D τ τ U (.3) where ɛ, α are defined by (.4). Let z =(c,c ) T C. Consider the system, (.4) c = ɛ + α(c + c ), c = τ (c + c ). rom (.4) we get the equation (.5) ( 4α τ If ɛ satisfies (.5), then (.6) + τ)c +(αɛ )c + τɛ =0. c = τɛ αɛ + 4αɛ τ ɛ

6 84 Xiao Shan Chen, Wen Li and Wei Wei Xu is a solution to (.5). By c = ɛ+ α τ c and (.6), we get a solution c =(c,c )T to the system (.4). Let X c = {X : X C n n, X τc, X D τc }. Then, X c is a bounded closed convex set of C n n, and the relation (.3) shows that the continuous mapping Φ maps X c into X c. By the Brouwer fixed-point theorem, the mapping Φ has a fixed point U X c such that τ U c = τ c, which yields the desired result (.6). Remark.. Taking τ = ν =and ν = A,τ = U = n, the bound in (.6) is called the absolute and relative perturbation bound, respectively. Remark.. or small ɛ the upper bound b u (ɛ) defined by (.6) has the Taylor expansion ( ) τ b u (ɛ) =ɛ + αɛ (.7) + 8 +α ɛ 3 + O(ɛ 4 ), ɛ 0. Combining (.7) with (.4) we see that the quantities A η(λ) = min i j n λ i λ j for ν = τ =and η(λ) = n min λ for ν = A i λ j,τ = U = n can be i j n regarded as absolute and relative condition numbers of the eigenvector matrix U of the spectral decomposition (.), respectively. Remark.3. Let A C n n have the Schur decomposition A = UTU H, where U is an n n unitary matrix and T is an n n upper triangular matrix. Let Y be strictly lower triangular matrix and the quantity η A is defined by η A =min{ low(ty YT) : Y =}. Konstantinov, Petkov and Christov[7] show that absolute and relative condition numbers of the Schur factor U are A η A and n η A, respectively. When both A and its perturbed matrix are Hermitian, from Remark. it is obvious that condition numbers of the eigenvector matrix U in (.) improve ones of the Schur factor by improving a numerical factor.

7 Eigenvector Matrix and Singular Vector Matrices A PERTURBATIO BOUD O SIGULAR VECTOR MATRICES In this section we investigate perturbation analysis of singular vector matrices for a m n real matrix. Let B R m n (m n) have the singular value decomposition (.)-(.3). Since B is a real matrix, V,Σ, and W may all be taken to be real. irst we define the linear operator P : R m m ( P δ X, ) ( (3.) ω Y = µ (ΣT X Y Σ T ), R n n R n m (ΣY XΣ) µ ), R m n where Σ is defined by (.3), and δ, ω, µ are positive parameters. Let T =(t ij ) R n m,r =(r ij) R m n (m n). or i =,,...,n we define row vectors l i with respect to T and R by (3.) l i =(t i,r i,...,t i,i,r i,i,t i,i+,r i,i+,...,t in,r in ), i =,,...,n. The following lemma determine when the operator P is nonsingular. Lemma 3.. Let P be the linear operator defined by (3.). () If m = n, then P is nonsingular if and only if all the singular values of B are simple, i. e. σ i σ j, i j, i, j =,...,n. () If m = n +, then P is nonsingular if and only if all the singular values of B are simple and σ i > 0,i =,,...,n. (3) If m>n+, then P is singular. by (3.3) and Proof. Let d ij = ( δσi ωσ j µ δσ j ωσ i ),i j, i, j =,...,n (3.4) P =diag(d,...,d n,...,d i,...,d i,i,d i,i+,...,d in...,d n,...,d n,n ). () If m = n, we define a column vector nvec(t,r) of T =(t ij ),R=(r ij ) C n n by nvec(t,r)=(l,l,...,l n ) T, where l i are defined by (3.). Then we have ( (3.5) nvec µ (ΣT X Y Σ T ), ) ( (ΣY XΣ) = P nvec µ δ X, ) ω Y. Hence the operator P is nonsingular if and only if P is a nonsingular matrix. Obviously, P is nonsingular if and only if σ i σ j,i j, i, j =,...,n.

8 86 Xiao Shan Chen, Wen Li and Wei Wei Xu () If m = n +, we define nvec(t,r) of T =(t ij ) R n (n+) and R =(r ij ) R (n+) n by nvec(t,r)=(l,...,l n,t,n+,...,t n,n+,r n+,,...,r n+,n ) T and also define nvec(t,r) of T =(t ij ) R (n+) (n+) and R =(r ij ) R n n by nvec(t,r)=(l,...,l n,t,n+,...,t n,n+,t n+,,...,t n+,n ) T, where l i,i=,,...,nare defined by (3.). Hence we have ( ) ( (3.6) nvec µ (ΣT X Y Σ T ), (ΣY XΣ) = Q nvec µ δ X, ) ω Y, where (3.7) ( Q = diag P, δ µ Σ, ) δ µ Σ. So the operator P is nonsingular if and only if Q is a nonsingular matrix. Obviously, Q is nonsingular if and only if σ i σ j,i j, i, j =,...,n and σ i > 0,i =,,...,n. (3) If m>n+, we take X =(x ij ) R m m and Y R n n as the following form x ij = { 0, (i, j) (n +,n+), (i, j)=(n +,n+), Y =0. It is easy to verify that P annihilates the nonzero matrix pair ( δ X, ω Y ) and must be singular. The proof is complete. Let P be the operator defined by (3.). ow we define the function { ( η(σ) = min µ (ΣT X Y Σ T XΣ)) ), µ (ΣY : ( X R m m,y R n n, δ X, ) } ω Y =. When the operator P is nonsingular, from (3.3)-(3.7) we can obtain (3.8) η(σ)= P µ min ϕ(σ i,σ j ), i j n { = µ min min ϕ(σ i,σ j ), i j n min i=,,...,n δσ i m=n, }, m=n+,

9 Eigenvector Matrix and Singular Vector Matrices 87 where P is defined by P =max{ P (X, Y ) : X R n m,y Rm n, (X, Y ) =} and δω(σi +σ j ) σ i σ j (3.9) ϕ(σ i,σ j )=. (δ +ω )(σ i +σj )+ (σi +σ j ) (δ ω ) +6δ ω σi σ j ow we give the following main result. Theorem 3.. Let m = n or m = n +and B R m n have the singular value decomposition (.)-(.3) with for m = n, σ i σ j, i j n and for m = n +, σ i σ j,σ i > 0, i j n. Moreover, let B = B + B R m n, and B ε =, ζ =max{δ, ω}, η(σ) µ (3.0) ( ) γ = max{δ,ω } m µη(σ) + δω B, where η(σ) is defined by (3.8). If ε satisfies the following condition (3.) ε γ + ζ +4γ, then B has the singular value decomposition (3.) B = W ΣṼ T, where W and Ṽ are m m and n n orthogonal matrices, respectively, and ) Σ Σ =(, Σ = diag( σ, σ,..., σ n ), σ i 0,i=,,...,n 0 such that (3.3) W W δ + Ṽ V ε ω γε+ b w,v (ε). 4γε ζ ε Proof. otice that the robenius norm is unitarily invariant norm. Hence without loss of generality we may assume that W = I m and V = I n in (.). By (.) and (3.) we obtain (3.4) ( B B)Ṽ + B(Ṽ I n)= W ( Σ Σ) + ( W I m )Σ.

10 88 Xiao Shan Chen, Wen Li and Wei Wei Xu Let (3.5) = W T BṼ, Σ = Σ Σ, W = W I m, V = Ṽ I n. (3.4) can be written as the following form (3.6) + W T Σ V = Σ+ W T W Σ. Similarly, from W T ( B B)+( W T I m )B =( Σ Σ)Ṽ T +Σ(Ṽ T I n ) we have (3.7) + W T ΣṼ = Σ+Σ V T Ṽ. Since W = I m + W and Ṽ = I n + V, from (3.6)-(3.7) we can obtain (3.8) Σ T W V Σ T = T + Σ T + V T V Σ T V T Σ T W, Σ V W Σ= + Σ+ W T W Σ W T Σ V. Moreover, the matrices W and V satisfy (3.9) W + W T + W T W =0, V + V T + V T V =0. In terms of the definition (3.) of the operator P, we can obtain from (3.8) ( P δ W, ) ω V ( ) (3.0) = µ ( V T V Σ T V T Σ T W ), µ ( W T W Σ W T Σ V ) rom (3.9) we have µ (( T ), ). (3.) ( W D, V D )= ( ( W T W ) D, ( V T ) V ) D. Since the operator P is nonsingular, then (3.0) can be rewritten as ( δ W, ) ω V (3.) = µ P ( ( V T V Σ T V T Σ T W ), ( W T W Σ W T Σ V ) ) µ P ( ( T ), ). By Lemma. we have

11 Eigenvector Matrix and Singular Vector Matrices 89 ( V T V Σ T ), ( W T W Σ) and = (( V T V ) Σ T, ( W T W ) Σ) max{δ,ω } Σ ( ω ( V T V ), δ ( W T W ) ) max{δ,ω } B ( m δ W, ) ω V ( V T Σ W ), ( W T Σ V ) δω B ( δ W, ω V ). Hence we get ( ( V T V Σ T V T Σ W ), ( W T W Σ W T ) Σ V ) ( ) (3.3) max{δ,ω } m ( + δω B δ W, ) ω V. rom (3.5) we have (3.4) (( T ), ) B. Hence by (3.)-(3.4), we get ( ( δ W, ) ω V ε + γ δ W, ) (3.5) ω V. By (3.) we get ( (3.6) δ W D, ω V D) ( ) ζ δ W, ω V. where ζ is defined by (3.0). ext in terms of (3.5) and (3.6), we take the similar proof as Theorem.3 to get the bound (3.3). The proof is complete. Remark 3.. Taking δ = ω = µ = and µ = B,δ = W,ω = V, the bound in (3.3) is called the absolute and relative perturbation bounds, respectively. Remark 3.. or small ε the upper bound b w,v (ε) defined by (3.3) has the Taylor expansion ( ) ζ b w,v (ε) =ε + γε (3.7) + 8 +γ ε 3 + O(ε 4 ), ε 0.

12 90 Xiao Shan Chen, Wen Li and Wei Wei Xu Combining (3.7) with (3.8)-(3.9) we see that the quantity η(σ) can be regarded as a condition number of the left singular vector matrix W and the right singular vector matrix V of the singular value decomposition (.) of a real n n or (n +) n matrix B. Taking µ = δ = ω =, the quantity min η(σ) = σ i σ j, m = n, i j n { max min σ i σ j, min σ i i j n i=,,...,n }, m = n + is regarded as the absolute condition number; Taking µ = B,δ = W,ω = V, the quantity B n min σ i σ j, m = n, η(σ) = i j n { } B max min min n+σi, m = n +, i j n ϕ(σ i,σ j ), i=,,...,n where ϕ(σ i,σ j )= n(n +)(σi + σ j ) σ i σ j (n +)(σ i + σj )+ (σi + σ j ) +6n(n +)σi σ j regarded as the relative condition number. 4. UMERICAL EXAMPLES In this section we use two simple examples to illustrate the results of previous two sections. All computations were performed by using MATLAB 6.5. The relative machine precision is Example 4.. Let A = diag(,,.00) and its perturbed matrix 0. 5 A = Taking U = I, Λ=A, then A + A has the spectral decomposition A + A = Ũ Ũ H, where Ũ = and

13 Eigenvector Matrix and Singular Vector Matrices 9 = diag( , , ). Computation gives Ũ U = , Ũ U U = Taking ν = τ = in Theorem.3, we obtain the following absolute perturbation bound b u (ɛ) = ; Taking ν = A,τ = U relative perturbation bound = 3 in Theorem.3, we obtain the following b u (ɛ) = The numerical results shows that the upper bound of (.6) is fairly sharp. Example 4.. Let B = , B = , and B =B+ B. Let B =W ΣV T be the singular value decomposition of B, where W =I 4,V =I 3. Then B has the singular value decomposition B = W ΣṼ T, where W = , Σ = and Ṽ =

14 9 Xiao Shan Chen, Wen Li and Wei Wei Xu Computation gives Ũ U + Ṽ V = , Ũ U U + Ṽ V V = Taking µ = δ = ω =in Theorem 3., we obtain the absolute perturbation bound b w,v (ε) = ; Taking µ = B,δ = U =,ω = V = 3 in Theorem 3., we obtain the relative perturbation bound b w,v (ε) = The numerical results show that the upper bound of (3.3) is correct. 5. COCLUSIO In this paper we have presented perturbation bounds of the eigenvector matrix of a Hermitian matrix and the singular vector matrices of a n n or (n +) n real matrix. The analysis is based on techniques proposed in [3, 7, 8, 3, 4]. ote that for the singular value decomposition B = W ΣV H of a complex matrix B, the diagonal elements of W and V may not be real at the same time. Hence for complex matrices, (3.) maybe not holds. It remains an open problem how the result in Theorem 3. is extended to the complex case. ACKOWLEDGMETS The authors are grateful to the referee for his/her many useful suggestions. REERECES. Z. Bai, J. Demmel, J. Dongarra, A. Ruhe and H. Van Der Vorst, Templates for the solution of Algebraic Eigenvalue Problems: A Practical Guide, SIAM, Philadelphia, R. Bhatia, C. Davis and A. Mcintosh, Erturbation of spectral subspaces and solution of linear operator equations, PLinear Algebra Appl., 5/53 (983), X. S. Chen, Perturbation bounds for the periodic Schur decomposition, BIT umer. Math., 50 (00), 4-58.

15 Eigenvector Matrix and Singular Vector Matrices C. Davis and W. Kahan, The rotation of eigenvectors by a perturbation, III, SIAM J. umer. Anal., 7 (970), R. A. Horn and C. R. Johnson, Matrix Analysis, Cambrdge University Press, ew York, R. A. Horn and C. R. Johnson, Topics in Matrix Analysis, Cambridge University Press, Cambridge, MA, M. Konstantinov, P. Hr. Petkov and. D. Christov, on-local perturbation analysis of the Schur system of a matrix, SIAM J. Matrix Anal. Appl., 5 (994), M. Konstantiov, V. Mehrmann and P. Petkov, Perturbation analysis of Hamilton Schur and block-schur forms, SIAM J. Matrix Anal. Appl., 3 (00), R. C. Li, On perturbations of matrix pencils with real spectra, Math. Compt., 6 (994), R. C. Li, Relative perturbation theory: II. Eigenspace and singular subspace variations, SIAM J. Matrix Anal. Appl., 0 (998), J. M. Ortega and W. C. Rheinboldt, Iterative solution of nonlinear equations in several variables, Academic Press, ew York, B.. Parlett, The Symmetric Eigenvalue Problem, Prentice-Hall, Englewood Cliffs, J, J. G. Sun, Perturbation bounds for the Schur decomposition, Report UMI-9.0, ISS Institute of Information Processing, University of Umea, Sweden, J. G. Sun, Perturbation bounds for the generalized Shur decomposition, SIAM J. Matrix Anal. Appl., 6 (995), G. W. Stewart and J. G. Sun, Matrix Perturbation Theory, Academic Press, Boston, G. W. Stewart, Error and perturbation bounds for subspaces associated with certain eigenvalue problems, SIAM Rev., 5 (973), P. A. Wedin, Perturbation bounds in connection with singular value decomposition, BIT umer. Math., (97), 99-. Xiao Shan Chen and Wen Li School of Mathematics South China ormal University Guangzhou 5063 P. R. China chenxs33@63.com liwen@scnu.edu.cn

16 94 Xiao Shan Chen, Wen Li and Wei Wei Xu Wei Wei Xu Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing P. R. China