AN ASYMPTOTIC BEHAVIOR OF QR DECOMPOSITION

Size: px

Start display at page:

Download "AN ASYMPTOTIC BEHAVIOR OF QR DECOMPOSITION"

Kerry Jefferson
5 years ago
Views:

1 Unspecified Journal Volume 00, Number 0, Pages S????-????(XX) AN ASYMPTOTIC BEHAVIOR OF QR DECOMPOSITION HUAJUN HUANG AND TIN-YAU TAM Abstract. The m-th root of the diagonal of the upper triangular matrix R m in the QR decomposition of AX m B = Q m R m converges and the limit is given by the moduli of the eigenvalues of X with some ordering, where A, B, X C n n are nonsingular. The asymptotic behavior of the strictly upper triangular part of R m is discussed. Some computational experiments are discussed. 1. Introduction The QR decomposition [] of a nonsingular X C n n asserts that X = QR, where Q C n n is unitary and R C n n is upper triangular with positive diagonal entries and the decomposition is unique. It is simply a matrix version of the traditional Gram-Schmidt process on the columns of X. The diagonal entries of R have very nice geometric interpretation, that is, r ii, i = 1,..., n, is equal to the distance from the i-th column of X to the space spanned by the first i 1 columns of X. We denote by (1) a(x) := diag (a 1 (X),..., a n (X)) = diag (r 11,..., r nn ), the diagonal matrix of R. In this paper, it is shown in Section that given nonsingular A, B, X C n n, and the QR decomposition AX m B = Q m R m, the sequence of matrices { () [a(ax m B)] 1/m} = { (diag R m=1 m ) 1/m} converges and the limit is given by the moduli of the eigenvalues of X. The asymptotic behavior of the strictly upper triangular part of R m is studied in Section 3. Some computational experiments using MAPLE and MATLAB are discussed in the last section. m=1 000 AMS Mathematics Subject Classification. Primary 15A3, 15A18 Key words: Eigenvalues, QR decomposition 1 c 0000 (copyright holder)

2 H. HUANG AND T.Y. TAM. Convergence of [a(ax m B)] 1/m Let {e 1,..., e n } be the standard basis of C n, that is, e i has 1 as the only nonzero entry at the i-th position. We identify a permutation ω S n with the unique permutation matrix (also written as ω) in the general linear group GL n (C), where ωe i = e ω(i). The matrix representation of ω under the standard basis is ω = [ e ω(1),, e ω(n) ]. Given a matrix A C n n, let A(i j) denote the submatrix formed by the first i rows and the first j columns of A, 1 i, j n. Theorem.1. Let A, B, X GL n (C). Let X = Y 1 DY be the Jordan decomposition of X, where D is the Jordan form of X, diag D = diag (λ 1,..., λ n ) satisfying λ 1 λ n. Then (3) lim a(axm B) 1/m = diag ( λ ω(1),..., λ ω(n) ), where the permutation ω is uniquely determined by Y B: () rank ω(i j) = rank (Y B)(i j) for 1 i, j n. Proof. Let X m := AX m B have the QR decomposition X m = Q m R m. Then (5) a 1 (X m ) a k (X m ) = det R m (k k) = det (X m (n k) X m (n k)). That is, the product a 1 (X m ) a k (X m ) is uniquely determined by the first k columns of X m. Set D 0 := diag D. Then D = CD 0 for a unit upper triangular matrix C commuting with D 0. By LU decomposition [, p.1], Y B = LωU, for some (unique) permutation matrix ω, some unit lower triangular matrix L, and some nonsingular upper triangular matrix U. By block multiplication (Y B)(i j) = [ L(i i) 0 ] [ ω(i j) = L(i i)ω(i j)u(j j). ] [ ] U(j j) 0 So ω satisfies rank ω(i j) = rank (Y B)(i j) for 1 i, j n. Obviously rank ω(i j) is the number of nonzero entries in ω(i j). Thus it is easy to verify that ω ij is a nonzero entry 1 if and only if rank ω(i j) rank ω(i j 1) rank ω(i 1 j) + rank ω(i 1 j 1) = 1. So the permutation matrix ω is uniquely determined by rank ω(i j), 1 i, j n. Hence ω is uniquely determined by Y B.

3 AN ASYMPTOTIC BEHAVIOR OF QR DECOMPOSITION 3 Let L = [ l pq ]n n. Then from X m = AX m B, X m (n k) = AY 1 C m D0 m (L ω) U(n k) = AY 1 C m diag (λ 1,..., λ n ) [ m l pω(q) ]n k U(k k) () Denote = AY 1 C m [ ( λp ) m lpω(q) ]n k diag (λ ω(1),..., λ ω(k) ) m U(k k). [ ( ) m (7) H m := AY 1 C m λp lpω(q). ]n k From (5) and the expression of X m (n k), m a1 (X m ) a k (X m ) = m det(x m (n k) X m (n k)) = m det U(k k) λ ω(1) λ ω(k) m det(h mh m ). m Clearly lim det U(k k) = 1 since U(k k) is a nonsingular constant matrix. It remains to prove lim m det(hmh m ) = 1 since it implies m lim a1 (X m ) a k (X m ) = λ ω(1) λ ω(k), m and thus lim ai (AX m B) = λ ω(i) for 1 i n. Viewing C as a constant matrix, the entries in C m are polynomials of m since C is unit upper triangular. In (7), each entry in AY 1 C m is a polynomial of m, and l pω(q) = 0 for those p < ω(q). Therefore, the (q, q) entry of HmH m has the form (8) p ω(q ) p ω(q) ( λ p λ ω(q ) ) m ( ) m λp f p p(m), where f p p(m) is a polynomial of m. So det(hmh m ) is a sum of summands in which each summand is a product of terms of the following form ( ) m ( ) m λ p λp f p p(m), for p ω(q ), p ω(q). λ ω(q ) Notice that λp 1 in each of the summands. If some λ p < 1 in a summand, then the summand approaches 0 as m goes to infinity.

4 H. HUANG AND T.Y. TAM Let E pq C n k whose only nonzero entry 1 is at the (p, q) position. Let Ω := { (p, q) {1,..., n} {1,..., k} : l pω(q) 0, λ p = }, L 0 := l pω(q) E pω(q), (p,q) Ω H m := AY 1 C m (p,q) Ω = AY 1 C m (p,q) Ω ( λp ) m l pω(q)e pω(q) ( ) m ( ) m λp λω(q) l pω(q)e pω(q) λ p ( ) m ( ) = AY 1 C m D0 λω(1) L 0 diag D 0 λ ω(1),..., λ m ω(k), λ ω(k) ( ) where D 0 := diag λ 1,..., λn D 0 λ 1 λ n. We remark that L 0 C n k is of full rank since it is obtained from L by picking some of the columns after the column permutation by ω and some entry deletions (but not the diagonal entries) of L. The discussion on det(hmh m ) in the preceding paragraph says that (9) lim [det(h mh m ) det(h mh m)] = 0. We have det(h mh m) = det [ ( ) D L m ( ) m ] 0 0 (C ) m (AY 1 ) (AY 1 )C m D0 L 0. D 0 D 0 Given two positive semi-definite matrices P, Q C n n, recall the Löwner partial order [, p.1] [5, p.1] : P Q whenever P Q is positive semi-definite. It follows that det P det Q. Now suppose that (AY 1 ) (AY 1 ) has the minimal eigenvalue α > 0 and the maximal eigenvalue β. Then Thus αi n (AY 1 ) (AY 1 ) βi n. ( ) D L m ( ) m 0 0 (C ) m (AY 1 ) (AY 1 )C m D0 L 0 D 0 D 0 ( ) D L 0(C ) m m ( ) m 0 D0 (αi n ) C m L 0 D 0 D 0 = αl 0(C ) m C m L 0. Now det[l 0(C ) m C m L 0 ] = f(m), where f is a polynomial. Then [, p.19] for all m N (10) det(h mh m) det[αl 0(C ) m C m L 0 ] = α k f(m).

5 Likewise, AN ASYMPTOTIC BEHAVIOR OF QR DECOMPOSITION 5 (11) det(h mh m) β k f(m). From (9), (10), (11), we get lim m det(hmh m ) = lim m det(h mh m) = 1. This completes the proof. Corollary.. Let X = Y 1 DY be the Jordan decomposition of X GL n (C), where D is the Jordan form of X, diag D = diag (λ 1,..., λ n ) satisfying λ 1 λ n. Then lim a(xm ) 1/m = diag ( λ ω(1),..., λ ω(n) ), where the permutation ω is uniquely determined by Y : rank ω(i j) = rank Y (i j) for 1 i, j n. Remark.3. In Theorem.1, there are many choices for the Jordan decomposition of X. Thus the permutation matrix ω may not be uniquely fixed by X. However, the result of Theorem.1 implies that diag ( λ ω(1),..., λ ω(n) ) is uniquely fixed by X and B. The phenomenon may be understood in the following way: Let X = Y 1 D Y be another Jordan decomposition of X, where the moduli of the diagonal entries of D are in non-increasing order. We obtain another permutation ω by rank ω (i j) = rank (Y B)(i j), for 1 i, j n. So Y B = L ω U for some unique lower triangular matrix L and upper triangular matrix U. Suppose ε 1 I t1 ε (1) D 0 := diag ( λ 1,, λ n ) = I t, ε 1 > > ε s.... ε s I ts That is, there are t i copies of the eigenvalue modulus ε i of X. Both D and D are block diagonal according to the row and column partitions γ := (t 1,..., t s ). There is a block diagonal matrix P according to γ such that D = P 1 DP. It follows from X = Y 1 DY = Y 1 D Y that (P Y Y 1 )D = D(P Y Y 1 ), that is, P Y Y 1 commutes with D. Note that D 0 = diag D is a polynomial of D [3, p.17], and D 0 in (1) is a polynomial of D 0 by the Lagrange-Sylvester interpolation polynomial [1, Chapter V]. Thus P Y Y 1 commutes with D 0, which

6 H. HUANG AND T.Y. TAM implies that P Y Y 1 is block diagonal according to γ, and so is T := Y Y 1. So Y = T Y and L ω U = Y B = T Y B = T LωU = L 1 T 1 ωu, where L 1 is unit lower triangular, and T 1 is block diagonal according to γ. From the LU decomposition discussed in the proof of Theorem.1, rank ω (i j) = rank (T 1 ω)(i j) for 1 i, j n. In particular, denote p i = t t i, then since T 1 is block diagonal, (13) rank ω (p i j) = rank (T 1 ω)(p i j) = rank ω(p i j), 1 i s, 1 j n. Partition the rows of ω and ω by γ, and partition the columns of ω and ω by (1, 1,..., 1), respectively. The (i, j)-block of ω has a nonzero entry (clearly 1) if and only if rank ω(p i j) rank ω(p i 1 j) rank ω(p i j 1) + rank ω(p i 1 j 1) 0, where p 0 := 0, 1 i s, 1 j n. Similar result holds for ω. By (13), the nonzero entries of ω and ω are located in the same block positions. This implies that λ ω(j) and λ ω (j) have the same moduli for 1 j n. Remark.. There is no similar convergence pattern for AW m X m B in general. For example, let A = B = I n, W = diag (1,,, n), and X = ω I n be a permutation. Then [ 1 m ] [ω m (1)] m AW m X m B = m... ω m = ω m [ω m ()] m.... nm [ω m (n)] m 3. Asymptotic behavior of the off-diagonal entries of R m Theorem.1 presents the asymptotic behavior of the diagonal entries of R m in the QR decomposition of AX m B = Q m R m. We now investigate the entries in the strictly upper triangular part of R m. [ ] Theorem 3.1. Under the same assumption as in Theorem.1, let R m = r (m) ij n n in the QR decomposition of AX m B = Q m R m. Then (1) lim r(m) ij 1/m max i k j Proof. From () with k = n, X m = AY 1 C m [ ( λp ) m lpω(q) ]n n { λω(k) } = λ min i k j ω(k), 1 i j n. diag (λ m ω(1),..., λ m ω(n)) U.

7 AN ASYMPTOTIC BEHAVIOR OF QR DECOMPOSITION 7 We know that l pω(q) = 0 for those p < ω(q), and λ 1 λ n. So l pω(q). The entries of C m are polynomials of m. Thus H m = AY 1 C m [ ( λp ) m lpω(q) ]n k ( λp ) m lpω(q) has the norm H m g(m) for a polynomial g and every m N. So in the QR decomposition H m = Q m Rm, the entries of R [ ] m = are bounded by the polynomial g(m). Now r (m) ij n n Therefore, Q m R m = X m = Q m Rm diag (λ m ω(1),..., λ m ω(n))u. R m = diag ( ) λω(1) λ ω(1),..., λ m ω(k) Rm diag ( λ m λ ω(n) ω(1),..., λω(n)) m U, and (15) r (m) n ij = r (m) ik λm ω(k)u kj = j r (m) j ik λm ω(k)u kj r (m) ik u kj λ ω(k) m. k=1 k=i k=i In other words, r (m) ij j k=i g ikj(m) λω(k) m for some polynomials gikj and every m N. This leads to inequality (1). In AX m B = Q m R m, define the matrix [ (1) R m (1/m) := r (m) ij ]n n 1/m. In general { R m (1/m) } m=1 may not converge. See the following example: [ ] 1 1 Example 3.. Let A := I, X := and B := I 0 1. Then { AX m X, m odd; B = I, m even. and R m+1 (1/(m+1)) = [ ] 1 1, R 0 1 m (1/(m)) = Clearly { r (m) 1 1/m } m=1 = {1, 0, 1, 0, } does not converge. [ ] Even { R m (1/m) } m=1 converges for some A, X and B, unlike the diagonal entries, given 1 i < j n, the sequence { r (m) ij 1/m } m=1 may not converge to any eigenvalue modulus of X. The following is an example.

8 8 H. HUANG AND T.Y. TAM Example 3.3. The example here indicates that although lim R m (1/m) may exist, { r (m) 1 1/m } m=1 does not necessarily converge to any eigenvalue modulus of X. Let 1 > a > b > 0 and A := I 3, X := a 0, B := b so that ω = By direct computation a m + b m So R m = b m a m +b m 0 a m b m a m +b m a m +b m a m +b m a m b m a m +b m lim R m (1/m) = a b /a a 0 b b and lim r (m) 1 1/m = b /a is not any eigenvalue modulus of X. Remark 3.. Needless to say, there is no convergence of the sequence {Q m } m=1, for example, if [ ] 0 1 A = B := I, X :=, 1 0 then {Q m } m=1 = {X, I, X, I,... } does not converge.. Numerical experiments In this section, we use MATLAB and MAPLE to investigate { R m (1/m) } m=1 in AX m B = Q m R m. First we study the convergence of { R m (1/m) } m=1 whenever A, X, B, are randomly generated (with probably some restrictions on ω). Then we discuss the convergence of [a(ax m B)] 1/m in the floating point computation. If A, X, B, are randomly generated, then it is almost surely that ω = I n, and R m (1/m),. λ 1 λ 1 λ 1 λ λ...., λ n as m. However, if ω is fixed first, and randomly generate nonsingular A, X = Y 1 DY, unit lower triangular L and upper triangular U, and construct B by Y B = LωU,

9 then usually we still have (17) lim r(m) ij AN ASYMPTOTIC BEHAVIOR OF QR DECOMPOSITION 9 1/m = max i k j { λω(k) } = λ min i k j ω(k), for 1 i j n. In other words, usually the above limit exists and the equality in (1) holds. This phenomenon can be understood from the proof of Theorem 3.1. According to (15), r (m) j ij = r (m) ik u kjλ m ω(k). k=i When A, X, B, are randomly generated as above, the right side of the above equation is almost surely dominated by those λ m ω(k) terms with the highest modulus. Thus (17) holds in almost all situations. Example.1. This example illustrates the convergence pattern (17) for randomly generated matrices with certain fixed ω. Let X := diag (10, 9, 8, 7, ). Let ω denote the permutation matrix corresponding to ( 5 1 3). Thus ( λ ω(1),, λ ω(5) ) = (9,, 7, 10, 8). Suppose B = LωU, and A, L, U, are randomly generated as follow: ] ] A := [ , L := [ , U := [ Using symbolic computation for AX m B = Q m R m in MAPLE, we see that lim R m (1/m) = This is exactly the limit in (17). Next we compute the discrepancy between a(ax m B) 1/m and the eigenvalue moduli of X for randomly generated A, X, B. Again it is almost surely that ω = I n. Therefore, theoretically we have lim a(axm B) 1/m = λ(x). However, the following example reveals that the floating point computation differs vastly from the symbolic one. We randomly generate A, X, B GL (C) as below. ].

10 10 H. HUANG AND T.Y. TAM Example i 1 1i 3i 19 + i A := 3 + i 8i + 3i i 7 7i 19 19i 1i 1 + i, 3 + 3i 1 i 31 i i 33 0i 1 i 5 + 3i 1 7i X := i + 9i 3 i 1 + 8i i i + 7i 9 + 7i, 8 i i i 5 + 8i B := 37 5i 5 3i 3 8i i 30 + i 3 0i 5 9i i 35 38i i 1 + 8i i i i 1 i The moduli of eigenvalues of X are: λ (8.31, 7.09,.3, 1.5). We compare the floating point plot (Figure 1b) of a(ax m B) λ(x) with the symbolic one (Figure 1a) in MAPLE: Figure 1a (symbolic) Figure 1b (floating point) Evidently the symbolic plot matches Theorem.1 but the floating point plot does not. The discrepancy may be caused by the instability of the QR decomposition compounded by power taking. In order to identify the components in which computation departs from our theoretical result, for each 1 i, we compare the floating point computation of a i (AX m B) λ i (X) with the symbolic one in MAPLE. The plots for a i (AX m B) λ i (X) versus m (m = 10,, 300) in symbolic and floating point computations are given below: (1) a 1 (AX m B) λ 1 (X) versus m: (notice that λ 1 / λ 1 = 1)

11 AN ASYMPTOTIC BEHAVIOR OF QR DECOMPOSITION Figure a (symbolic) Figure b (floating point) () a (AX m B) λ (X) versus m: (notice that λ 1 / λ 1.17) Figure 3a (symbolic) Figure 3b (floating point) (3) a 3 (AX m B) λ 3 (X) versus m: (notice that λ 1 / λ 3 1.3) Figure a (symbolic) Figure b (floating point) () a (AX m B) λ (X) versus m: (notice that λ 1 / λ.3)

12 1 H. HUANG AND T.Y. TAM Figure 5a (symbolic) Figure 5b (floating point) In general, we observe that for generic nonsingular randomly generated A, X, B, when λ k = λ 1, the floating point plot of a k (AX m B) 1/m λ k (X) 0 as m ; Otherwise, a k (AX m B) 1/m λ k (X) does not approach 0. Moreover, the divergence appears earlier whenever λ 1 / λ k is larger. The phenomenon may be interpreted in this way: Theoretically, a k (AX m B) is dominated by f(m)λ m k for certain f(m) bounded by polynomials. However, in the floating point computation, the round-off errors may disturb a k (AX m B) to add some λ m 1 terms with small modulus coefficients. When m is sufficiently large, the sequence {[a k (AX m B)] 1/m } m=1 will go to λ 1 (X) instead of λ k (X) in the floating point computation. References [1] F.R. Gantmacher, The Theory of Matrices, I, Chelsea Publishing Company, New York, [] R.A. Horn and C.R. Johnson, Matrix Analysis, Cambridge University Press, [3] J.E. Humphreys, Introduction to Lie Algebras and Representation Theory, Springer, New York, 197. [] D. Serre, Matrices: Theory and Applications, Springer, New York, 00. [5] X. Zhan, Matrix Inequalities, Lecture Notes in Mathematics 1790, Springer, Berlin, 00. [] F. Zhang, Matrix Theory: Basic Results and Techniques, Springer, New York, Department of Mathematics and Statistics, Auburn University, AL , USA address: huanghu@auburn.edu, tamtiny@auburn.edu

ON THE QR ITERATIONS OF REAL MATRICES

Unspecified Journal Volume, Number, Pages S????-????(XX- ON THE QR ITERATIONS OF REAL MATRICES HUAJUN HUANG AND TIN-YAU TAM Abstract. We answer a question of D. Serre on the QR iterations of a real matrix