ON THE QR ITERATIONS OF REAL MATRICES

Transcription

1 Unspecified Journal Volume, Number, Pages S????-????(XX- ON THE QR ITERATIONS OF REAL MATRICES HUAJUN HUANG AND TIN-YAU TAM Abstract. We answer a question of D. Serre on the QR iterations of a real matrix with nonreal eigenvalues whose moduli are distinct except for the conjugate pairs. Numerical experiments by MATLAB are performed. 1. Introduction There are many numerical methods for the computation of the eigenvalues of a given A GL n (K with K = R or C. One of the most efficient methods is the QR method [3, p ]. Define a sequence {A } N GL n (K of matrices with A 1 := A A j+1 := R j Q j if A j = Q j R j is the QR decomposition of A j, j = 1,,... Notice that (1.1 A j+1 = Q 1 j A j Q j. So the eigenvalues of each A j are identical with those of A, counting multiplicities. One hopes to have some sort of convergence on the sequence {A } N so that the limit would provide the eigenvalues of A. If we write then [3] P = Q 1 Q Q, U = R R 1 R 1, (1. A = P U, Q = P 1 1 P, R = U U 1 1, (1.3 A = P 1 1 AP 1 = U 1 AU 1 1. In Wilinson s boo [6, p ] one finds the following classical result. Theorem 1.1. Let A GL n (C such that the moduli of the eigenvalues λ 1,..., λ n of A are distinct, that is, (1.4 λ 1 > λ > > λ n (>. Let A = Y 1 diag (λ 1,..., λ n Y. Assume that Y admits an LU decomposition Y = LU. Then the strictly lower triangular part of A converges to zero the diagonal part of A converges to D := diag (λ 1,..., λ n. Though Theorem 1.1 is a rather satisfactory result, in many applications one encounters A GL n (R. If A has nonreal eigenvalues, then they occur in complex conjugate pairs the assumption (1.4 does not hold for A. D. Serre [3, p.174] asserts that AMS Mathematics Subject Classification. Primary 15A3, 65F1 Key words: QR iterations, eigenvalues, QR decomposition, LU decomposition 1 c (copyright holder

2 H. HUANG AND T.Y. TAM When A M n (R, one maes the following assumption. Let p be the number of real eigenvalues q that of nonreal eigenvalues; then there are p + q distinct eigenvalue moduli. In that case, {A } N might converge to a bloc-triangular form, the diagonal blocs being or 1 1. The limits of the diagonal blocs provide trivially the eigenvalues of A. The assertion has never been proved nor disproved, as pointed out by Serre [3, p.175]. Evidently the above quoted paragraph is interpreted as the strictly lower triangular bloc part of {A } N converges to zero. Indeed the diagonal blocs of {A } N may not converge, even though the eigenvalues of these diagonal blocs converge to the eigenvalues of A (see Proposition 4.. In Section, Theorem.1 gives an affirmative answer to the question of Serre under a very mild condition. Namely, if a real matrix A = Y 1 DY has distinct moduli eigenvalues (up to conjugate pairs, where D is given in (., Y admits a certain bloc LU decomposition, then the strictly lower triangular bloc part of {A } N converges to zero, where the diagonal blocs (of or 1 1 forms provide the eigenvalues of A. In Section 3, we exhibit that if Y (A = Y 1 DY does not have the bloc LU decomposition as in Theorem.1, then the conclusion of Theorem.1 may not hold, based on some numerical experiments. In other words, Serre s assertion is not true for this ind of matrices. In Section 4 we provide some quantitative analysis for the case. In Section 5 we prove that unlie the real case, the complex case still behaves well even if Y does not admit LU decomposition as long as (1.4 is satisfied.. An answer to Serre s question The assumption of Serre on A GL n (R amounts to that the eigenvalues of A have distinct moduli except for the conjugate pairs. It may be interpreted as the real counterpart of (1.4 in Theorem 1.1. By the real Jordan canonical form [, Theorem 3.4.5, p.15], A admits the following decomposition (.1 A = Y 1 DY where (. D := diag (λ 1 E θ1,, λ m E θm, λ 1 > > λ m >, E θi= := 1, E θi=π := 1, E θi (,π := cos θi sin θ i. sin θ i cos θ i In general Y has the Bruhat decomposition Y = LωU where L is unit lower triangular, U is upper triangular, ω is a permutation matrix uniquely determined by Y. If Y admits a bloc LU decomposition analogous to that in Theorem 1.1, we have the following result. Since such matrices Y form a dense subset of GL n (R, a romly chosen A GL n (R almost surely satisfies the above requirements. Theorem.1. Let A GL n (R be a matrix such that the eigenvalues of A have distinct moduli except for the conjugate pairs. With the above notations, let γ = (γ 1,, γ m where γ i is the size of E θi, i = 1,..., m. Let [M] γ be the bloc form

3 QR ITERATIONS OF REAL MATRICES 3 of M corresponding to the partitions γ. Let { λ t := max,, λ m }. λ 1 λ m 1 If Y = LωU [ω] γ is bloc diagonal (for example, if ω is the identity matrix, then the strictly lower triangular bloc part of [A ] γ converges to zero in O(t, the eigenvalues of the ith diagonal bloc of [A ] γ converge to the eigenvalues of λ i E θi in O(t. Proof: Let Y 1 = QR so that A = Y 1 D Y = QRD LωU = QR(D LD D ωu. Denote [L] γ = (L ij m m where the (i, j bloc of L is L ij of size γ i γ j. Then ( [D LD ] γ = λi λ j E θi L ij E. θ j Let D := diag [L] γ = L m m L mm, where diag [L] γ denotes the bloc diagonal part of [L] γ. Denote Eθ 1 L 11 E θ 1 D := D D D =... E θ m L mm E θ m, = 1,,.... Using L ij = for i < j E θi = 1 for all i, where is the spectral norm, So we have D LD = (I n + O(t D. A = QR(I n + O(t D D ωu = Q(I n + O(t RD D ωu = QO T RD D ωu. Here O T is the QR decomposition of the last I n + O(t. By the Gram-Schmidt process one has (.3 O = I n + O(t, T = I n + O(t. Since [T RD D ωu] γ is a bloc upper triangular matrix, its Q-component C in the QR decomposition is a bloc diagonal matrix according to γ. So the QR decomposition of A is A = P U = (QO C (C 1 T RD D ωu. Hence by the uniqueness of the QR decomposition P = QO C, U = C 1 T RD D ωu.

4 4 H. HUANG AND T.Y. TAM Therefore, by (.3 (.4 A = Q R = P 1 1 P U U 1 1 = C 1 1 O 1 1 O T RDR 1 T 1 1 C 1 = C 1 1 RDR 1 C 1 + O(t. Because C 1 1 RDR 1 C 1 is bloc upper triangular, the entries of the strictly lower triangular blocs of A approach zero in O(t. Moreover, by bloc multiplication the ith diagonal bloc of C 1 1 RDR 1 C 1 is similar to that of D, namely λ i E θi. So the eigenvalues of the ith diagonal bloc of [A ] γ approach those of λ i E θi in O(t. Numerical experiments denomstrate the convergence rate in Theorem.1. From the computational point of view, the assumption that [ω] γ is in bloc diagonal form does not impose any difficulty: A will first be reduced to an Hessenberg form to achieve drastic cost reduction [3, p.176]. Thus we may assume that A GL n (R is in irreducible (nonreduced Hessenberg form. Those nonsingular Y for which A = Y 1 DY would have the required LωU decomposition in Theorem.1, according to the following result. Proposition.. Suppose that A GL n (R in Theorem.1 is in irreducible Hessenberg form. Then for any Y GL n (R such that A = Y 1 DY, it has the decomposition Y = LωU, where [ω] γ is in diagonal bloc form, D is given in (.. Proof: For any θ R, if P := 1 i 1, then 1 i ( diag (e iθ, e iθ cos θ sin θ = P sin θ cos θ P 1. Let S GL n (C be in bloc diagonal form such that the diagonal blocs of [S] γ are P the 1 1 blocs are 1, according to the partition γ. Then A = Y 1 S 1 DSY where D is a diagonal matrix such that the diagonal blocs of [ D] γ are either ±λ j or λ j diag (e iθj, e iθj. We claim that the matrix Z = SY admits LU decomposition the argument follows from [3, p.179] (there are some typos in the proof. First notice that the rows of Z are left eigenvectors of A, that is, if z 1,..., z n denote the rows of Z, then z j A = µz j, j = 1,..., n, µ = ±λ j or λ j e ±iθj since ZA = DZ. Then {z1,..., zq } is invariant under A one has {z1,..., zq } {e 1,..., e q } = C n. In other words, det(z j e 1 j, q. In other words, the leading principal minors of Z of order q is nonzero. So Z = SY admits an LU decomposition. Thus SY = LU for some unit lower triangular matrix L upper triangular matrix U. Then Y = S 1 LU. Now the matrix S 1 L is in lower triangular bloc form with diagonal blocs 1 1 or. Applying Gaussian elimination on S 1 L, one has Y = L ωu where L is (real unit lower triangular, U is (real upper triangular ω is a diagonal bloc permutation matrix corresponding to the partition γ. The permutation matrix ω is unique. In general the strictly lower triangular part of the (real sequence {A } N does not converge to zero (Compare [, p.114].

5 QR ITERATIONS OF REAL MATRICES 5 Proposition.3. Suppose that A M n (R has nonreal eigenvalues. strictly lower triangular part of {A } N does not converge to zero. Proof: The sequence {A } N is contained in the compact set {X M n (R : X F = A F }, Then the where A F = (tr A A 1/ denotes the Frobenius norm of A. So there is a convergent subsequence {A i } i N. If the strictly lower triangular part of the sequence {A } N converged to zero, then the subsequence would converge to a real upper triangular matrix U. By the continuity of the eigenvalues (counting multiplicities [3, p.44], the eigenvalues of A would be the diagonal entries of U would be real, a contradiction. The argument in the above proof wors for real singular matrices having nonreal eigenvalues as well. 3. Numerical experiments We now discuss some numerical experiments which show that the conclusion of Theorem.1 may not hold if the condition on Y in Theorem.1 is not satisfied. Let a cos c a sin c A = Y 1 a sin c a cos c b cos d b sin d Y, b sin d b cos d where Y = LωU, 1 1 L = , ω = 1 1, U = I Clearly the condition of Theorem.1 is not satisfied for Y. With a =, b = 1/, numerically we have the following pattern convergence (not actual convergence of the corresponding matrices. We use the formula in (1.3 A = P 1 1 AP 1 instead of A = R 1 Q 1 to compute A via MAPLE MATLAB. (1 If c =, d = 1 (the eigenvalues of A occur as two distinct complex conjugate pairs, A, Q, P. ( If c =, d = π ( 1/ is a double eigenvalue of A, A, 1/

6 6 H. HUANG AND T.Y. TAM Q, 1 P. ( 1 (3 If c = π, d = 1 ( is a double eigenvalue of A, A, 1 Q, P. (4 If c =, d = π/ (the eigenvalues of A occur as two distinct complex conjugate pairs, Q. Then for all N, A, A 1, P, ( 1 P 1. (5 If c = d = π/, then for all N, A A In the above cases, no desired convergence (in the fashion of Theorem.1 occurs for the lower triangular bloc part of A. We also used A = R 1 Q 1 to compute A. The computed lower triangular bloc part of A tends to zero. Probably the roundoff errors perturb A so that the computed Y has bloc LU decomposition in the computational process. Denote

7 QR ITERATIONS OF REAL MATRICES 7 L to be the maximal entry in module of the lower left bloc of A. The convergence rate of L to is exactly the convergence rate of A to the bloc upper triangular form. Denote c( := L /t, where t = λ λ 1 = 1 4. When A meets the conditions in Theorem.1, we now c( M, = 1,,..., for some constant M depending on A alone. However, for the above five cases in which A does not satisfy the conditions in Theorem.1, numerical experiments show that we still have c( M, where M is determined by A the digit number used in floating-point computations. We apply A = R 1 Q 1 in MAPLE to compute A for the first case (c =, d = 1, using 1,, 3, 35-digit number floating-point arithmetic, respectively. Then we plot c( against for 1 1 as follow: Plot of c( by 1-digit computation Plot of c( by -digit computation 1E1.5E1 8E11 E1 6E11 1.5E1 4E11 1E1 E11 5E Plot of c( by 3-digit computation Plot of c( by 35-digit computation 7E3 E36 6E3 5E3 1.5E36 4E3 3E3 1E36 E3 5E35 1E The plots of c( display similar pattern in different floating point precisions. Roughly speaing, when using n-digit floating-point arithmetic, the upper bound M of the computed c( is around the scale 1 n. Similar phenomenon holds for the other four cases. 4 1

8 8 H. HUANG AND T.Y. TAM 4. Analysis of the real case with nonreal eigenvalues In Theorem.1, we see that the QR iterations for almost all real matrices converge to a bloc upper triangular form with or 1 1 diagonal blocs. Thus it is important to study the real matrix with nonreal eigenvalues in a quantitative fashion. Proposition 4.1. Suppose A GL (R has nonreal eigenvalues. Let A = Y 1 cos θ sin θ Y det A sin θ cos θ where Denote Y = y11 y 1 SL y 1 y (R, u := y 11 y 1 + y 1 y, θ (, π \ {π}. v := y11 + y1, u r := + v ( u + v + 1 v. Then the modulus of the (, 1 entry c of A satisfies (4.1 sv r sin θ c min{ sr sin θ, A }, v where s := det A A is the spectral norm of A. Proof: The singular values of A A are the same thus the entries of A are bounded above by A. Notice A /s = Y 1 cos θ sin θ Y sin θ cos θ y y = 1 cos θ sin θ y11 y 1 y 1 y 11 sin θ cos θ y 1 y (4. = Let A = P U = P ( a (4.3 ( cos θ + u sin θ v sin θ 1/a. s. By the Gram-Schmidt process, a = (cos θ + u sin θ + v (sin θ u + v = + 1 u + v 1 cos θ + u sin θ u = + v ( u + v 1 + u cos (θ + ζ, where ζ is a constant. Since u + v + 1 ( u + v 1 + u = ( u + v + 1 v, u + v ( u + v + 1 v a ( u + v + 1 v.

9 QR ITERATIONS OF REAL MATRICES 9 In other words, (4.4 On the other h, from (.4 v r a r. A = P 1 1 P U U 1 1 = P 1 1 A 1 AU 1 1 = U 1AU 1 1, (4.5 U 1 = so that (4.6 a 1 s 1 1/a 1 ( A = s 1/a 1 v sin θ ( = s v sin θ/a. 1 s +1 1/a 1 By (4.4 (4.6, the modulus of the (, 1 entry c of A is bounded by sv r sin θ c = sv sin θ/a 1 sr sin θ. v This completes the proof of (4.1. Now we are able to study the convergence of the QR iterations of the matrix A GL (R. It is sufficient to consider A SL (R. Proposition 4.. (1 Suppose A SL (R has nonreal eigenvalues. Then A converges if only if A is an orthogonal matrix. In this case, Q = A, R = I, = 1,,... ( If A SL (R has nonreal eigenvalues is not an orthogonal matrix, then each of the sequences {A } N, {P } N, {U } N, {Q } N, {R } N is bounded below above but not convergent. Proof: We adopt the notations from Proposition 4.1. (1 From (4.6, if A converges, then c has to converge. Then by (4.3, we have two possibilities: (a ( u +v 1 + u =, that is, u = v = 1. So by the definitions of u v, the matrix Y is orthogonal thus A is an orthogonal matrix. (b θ = π/ or 3π/, cos ζ = u +v 1 / ( u +v 1 + u =. So u + v = 1, a = 1 by (4.3. We have ( cos η sin η A = P 1 U 1 = sin η cos η 1 t, 1 for some t R η (, π\{π}. If t = then A is an orthogonal matrix. If t we have 1 t cos η sin η A = 1 sin η cos η cos η t sin η sin η + t cos η =. sin η cos η

10 1 H. HUANG AND T.Y. TAM So a = (cos η t sin η + ( sin η = 1 t cos η sin η + t sin η = 1. Hence t = cos η/ sin η. In such situation, we have A 1 = A 3 = A = A 4 =. Moreover, A 1 = A if only if cos η =, contradict with t. The converse is obviously true. ( By (1. (4.5 R = U U 1 1 = a /a 1, a 1 /a since s := det A = 1. By the first part A does not converge. So in (4.3 ( u +v 1 + u. Thus a /a 1 has finite positive upper bound lower bound but it does not converge. Thus the entries of R are bound above below in absolute value but not convergent. Now ( ( Q = A R 1 a 1 /a = v sin θ/a =. 1 (v/a 1 a sin θ If a 1 a does not converge, then Q does not converge. If a 1 a converges, then A belongs to (1(b thus Q does not converge. Neither P nor U converges since A does not converge ( Some remars on Theorem 1.1 Theorem 1.1 does not say that {A } N converges to an upper triangular matrix. In fact it is pointed out in [3, p.178], when the eigenvalues of A have distinct arguments, the sequence {A } N does not converge, in contrast to an incorrect assertion in [, p.114]. In general Y in Theorem 1.1 may not admit LU decomposition. Instead, it has the Bruhat decomposition Y = LωU for some permutation matrix ω I n. However, this will not cause any trouble based on two observations. First the set of nonsingular matrices without LU decompositions is of measure zero in GL n (C [1, p.47]. So a romly chosen A GL n (C almost surely satisfies the conditions in Theorem 1.1. Secondly, in practice a preliminary reduction of A GL n (C to an Hessenberg form drastically reduces the cost of each QR step [3, p.176]. So A will first be turned into a Hessenberg form [3, p , p ], thus we may assume that A is in irreducible (nonreduced Hessenberg form. Those nonsingular Y satisfying A = Y 1 DY have LU decomposition [3, p.179]. Nevertheless, for the general case Y = LωU, the diagonal part of {A } N converges to D ω := ω 1 Dω the strictly lower triangular part of A converges to zero, which are parts of the statements of Theorem 5.1. Theorem 5.1 is obviously a generalization of Theorem 1.1. We now introduce some notations. Given an n n permutation ω, we denote the permutation on {1,..., n} by the same notation ω such that ωe j = e ω(j, j = 1,..., n, where {e 1,..., e n } is the stard basis of R n. We write O(t for a matrix whose entries are less than or equal to C t in absolute value for some constant C >.

11 QR ITERATIONS OF REAL MATRICES 11 Theorem 5.1. Let A GL n (C such that the moduli of the eigenvalues λ 1,..., λ n of A are distinct, that is, (5.1 λ 1 > λ > > λ n (>. Let A = Y 1 DY where D := diag (λ 1,, λ n. Let ω be the permutation matrix uniquely determined by Y = LωU, where L is unit lower triangular U is upper triangular. Denote H := ( u11 diag u 11,, u nn, u nn D ω := diag (λ ω(1,, λ ω(n, C ω := ( λω(1 diag λ ω(1,, λ ω(n. λ ω(n Then where Cω 1 A Cω +1 = H 1 RD ω R 1 H + O(t, Q = C ω + O(t, { λ t := max,, λ n } < 1, λ 1 λ n 1 Y 1 ω = QR is the QR decomposition of Y 1 ω. In particular (1 lim C 1 ω A C +1 ω = lim C ωr C +1 ω = H 1 RD ω R 1 H. ( lim Q = C ω. (3 The strictly lower triangular part of A converges to zero in O(t. (4 The diagonal part of A converges to D ω in O(t. Proof: Under the assumption, A = Y 1 D Y. Let Y = LωU where L is unit lower triangular, U is upper triangular, ω is a permutation matrix uniquely determined by Y. Let Y 1 ω = QR be the QR decomposition of Y 1 ω. Then A = QRω 1 D LωU = QRω 1 (D LD D ωu. Notice that the unit lower triangular matrix D LD = I n + O(t since the (i, j entry of D LD where i > j is l ij (λ i /λ j whose absolute value is less than or equal to M t, where M = max 1 j<i n l ij. Hence A = QRω 1 (I n + O(t D ωu = Q(I n + O(t Rω 1 D ωu = QO T RD ωu where O T is the QR decomposition of the last I n + O(t. By the Gram-Schmidt process we have O = I n + O(t T = I n + O(t. So A = P U = (QO HCω(C ω H 1 T RDωU

12 1 H. HUANG AND T.Y. TAM is the QR decomposition of A since C ω H are diagonal unitary. By the uniqueness of the QR decomposition, By (1. (5. (5.3 P = QO HC ω = QHC ω + O(t, U = Cω H 1 T RDωU = Cω H 1 RDωU + O(t. Q = P 1 1 P = C ω + O(t, R = U U 1 ω H 1 RD ω R 1 HCω 1 + O(t, 1 = C (5.4 A = Q R = Cω +1 H 1 RD ω R 1 HCω 1 + O(t. Since Cω 1 is diagonal unitary, (5.5 Cω 1 A Cω +1 = H 1 RD ω R 1 H + O(t. Conclusion ( follows from (5., (1 from (5.5. From (5.4 the diagonal part of A is diag A = diag [Cω +1 H 1 RD ω R 1 HCω 1 + O(t ] = D ω + O(t (4 follows immediately. Similarly (3 follows from (5.4. Corollary 5.. The following statements are equivalent. (1 The sequence {A } N converges. ( The sequence {R } N converges. (3 The arguments of λ ω(i λ ω(j are equal whenever the (i, j entry of RD ω R 1 is nonzero for i < j. Proof: (1 ( follows from Theorem 5.1 (1 (. From (5.4 A = H 1 Cω +1 RD ω R 1 Cω 1 H + O(t. So A converges if only if Cω +1 RD ω R 1 Cω 1 converges. Thus (1 (3 are equivalent. Our proof of Theorem 5.1 is a modification of the proof in [3, Theorem 1..1]. Part of the theorem has been discussed in [6, p.519-5] wherein the proof relies on a very careful observation of the pivoting procedure. Acnowledgement: The authors are thanful to the referee s helpful comments. References [1] S. Helgason, Differential Geometry, Lie Groups, Symmetric Spaces, Academic Press, New Yor, [] R.A. Horn C.R. Johnson, Matrix Analysis, Cambridge Univ. Press, [3] D. Serre, Matrices: Theory Applications, Springer, New Yor,. [4] G.W. Stewart, Introduction to Matrix Computations, Academic Press, New Yor, [5] D. Watins, Understing the QR algorithm, SIAM Rev. 4 (198, [6] J.H. Wilinson, The Algebraic Eigenvalues Problem, Oxford Science Publications, Oxford, Department of Mathematics Statistics, Auburn University, AL , USA address: huanghu@auburn.edu, tamtiny@auburn.edu