CHAPTER 2. Unitary similarity and unitary equivalence

Transcription

1 CHAPTER 2 Unitary similarity and unitary equivalence 2.0 Introduction yin Capter 1, we made an initial study of similarity of A 2 M n via a general nonsingular matrix S, tat is, te transformation A! S 1 AS. For certain very special nonsingular matrices, called unitary matrices, te inverse of S as a simple form: S 1 = S. Similarity via a unitary matrix U, A! U AU, is not only conceptually simpler tan general similarity (te conjugate transpose is muc easier to compute tan te inverse), but it also as superior stability properties in numerical computations. A fundamental property of unitary similarity is tat every A 2 M n is unitarily similar to an upper triangular matrix wose diagonal entries are te eigenvalues of A. Tis triangular form can be furter rened under general similarity; we study te latter in Capter 3. Te transformation A! S AS, in wic S is nonsingular but not necessarily unitary, is called *congruence; we study it in Capter 4. Notice tat similarity by a unitary matrix is bot a similarity and a *congruence. For A 2 M n;m, te transformation A! UAV, in wic U 2 M m and V 2 M n are bot unitary, is called unitary equivalence. Te upper triangular form acievable under unitary similarity can be greatly rened under unitary equivalence and generalized to non-square matrices: every A 2 M n;m is unitarily equivalent to a nonnegative diagonal matrix wose diagonal entries (te singular values of A) are of great importance. y Matrix Analysis, second edition by Roger A. Horn and Carles R. Jonson, copyrigt Cambridge University Press

2 102 Unitary similarity and unitary equivalence 2.1 Unitary matrices and te QR factorization Denition. A set of vectors fx 1 ; : : : ; x k g C n is ortogonal if x i x j = 0 for all i 6= j, i; j = 1; : : : ; k. If, in addition, x i x i = 1 for all i = 1; : : : ; k (tat is, te vectors are normalized), ten te set is ortonormal. Exercise. If fy 1 ; : : : ; y k g is an ortogonal set of nonzero vectors, sow tat te set fx 1 ; : : : ; x k g dened by x i = (y i y i) 1=2 y i, i = 1; : : : ; k, is an ortonormal set Teorem. Every ortonormal set of vectors in C n is linearly independent. Proof: Suppose tat fx 1 ; : : : ; x k g is an ortonormal set, and suppose tat 0 = 1 x k x k. Ten 0 = ( 1 x k x k ) ( 1 x k x k ) = i;j i j x i x j = k i=1 j ij 2 x i x i = k i=1 j ij 2 because te vectors x i are ortogonal and normalized. Tus, all i = 0 and ence fx 1 ; : : : ; x k g is a linearly independent set. Exercise. Sow tat every ortogonal set of nonzero vectors in C n is linearly independent. Exercise. If fx 1 ; : : : ; x k g 2 C n is an ortogonal set, sow tat eiter k n or at least k n of te vectors x i are equal to zero. An independent set need not be ortonormal, of course, but one can apply te Gram Scmidt ortonormalization procedure (0.6.4) to it and obtain an ortonormal set wit te same span as te original set. Exercise. Sow tat any nonzero subspace of R n or C n as an ortonormal basis (0.6.5) Denition. A matrix U 2 M n is unitary if U U = I. If, in addition, U 2 M n (R), U is real ortogonal. Exercise. Sow tat U 2 M n and V 2 M m are unitary if and only if U V 2 M n+m is unitary. Exercise. Verify tat te matrices Q; U; and V in Problems 19, 20, and 21 in (1.3) are unitary. Te unitary matrices in M n form a remarkable and important set. We list some of te basic equivalent conditions for U to be unitary in (2.1.4) Teorem. If U 2 M n, te following are equivalent: (a) U is unitary;

3 2.1 Unitary matrices and te QR factorization 103 (b) U is nonsingular and U = U 1 ; (c) UU = I; (d) U is unitary; (e) Te columns of U form an ortonormal set; (f) Te rows of U form an ortonormal set; and (g) For all x 2 C n, kxk 2 = kuxk 2, tat is, x and Ux ave te same Euclidean norm. Proof: (a) implies (b) since U 1 (wen it exists) is te unique matrix, left multiplication by wic produces I (0.5); te denition of unitary says tat U is suc a matrix. Since BA = I if and only if AB = I (for A; B 2 M n (0.5)), (b) implies (c). Since (U ) = U, (c) implies tat U is unitary; tat is, (c) implies (d). Te converse of eac of tese implications is similarly observed, so (a) (d) are equivalent. Partition U = [u 1 : : : u n ] according to its columns. Ten U U = I means tat u i u i = 1 for all i = 1; : : : ; n and u i u j = 0 for all i 6= j. Tus, U U = I is anoter way of saying tat te columns of U are ortonormal, and ence (a) is equivalent to (e). Similarly, (d) and (f) are equivalent. If (a) olds and y = Ux, ten y y = x U Ux = x Ix = x x, so (a) implies (g). To prove te converse, let U U = A = [a ij ], let z; w 2 C be given, and take x = z + w in (g). Ten x x = z z + w w + 2 Re z w and y y = x Ax = z Az + w Aw + 2 Re z Aw; (g) ensures tat z z = z Az and w w = w Aw, and ence Re z w = Re z Aw for any z and w. Take z = e p and w = ie q and compute Re ie T p e q = 0 = Re ie T p Ae q = Re ia pq = Im a pq, so every entry of A is real. Finally, take z = e p and w = e q and compute e T p e q = Re e T p e q = Re e T p Ae q = a pq, wic tells us tat A = I and U is unitary Denition. A linear transformation T : C n! C m is called a Euclidean isometry if x x = (T x) (T x) for all x 2 C n. Teorem (2.1.4) says tat a square complex matrix U 2 M n is a Euclidean isometry (via U : x! Ux) if and only if it is unitary. See (5.2) for oter kinds of isometries. Exercise. Let T () = cos sin sin cos i, in wic is a real parameter. (a) Sow tat a given U 2 im 2 (R) is real ortogonal if and only if eiter U = T () or U = T () for some 2 R. (b) Sow tat a given U 2 i M 2 (R) is real ortogonal if and only if eiter U = T () or U = T () for some 2 R. Tese are two different presentations, involving a parameter, of te 2-by-2 real ortogonal matrices. Interpret tem geometrically.

4 104 Unitary similarity and unitary equivalence Observation. If U; V 2 M n are unitary (respectively, real ortogonal), ten UV is also unitary (respectively, real ortogonal). Exercise. Use (b) of (2.1.4) to prove (2.1.6) Observation. Te set of unitary (respectively, real ortogonal) matrices in M n forms a group. Tis group is generally referred to as te n-by-n unitary (respectively, ortogonal) group, a subgroup of GL(n; C) (0.5). Exercise. A group is a set tat is closed under a single associative binary operation ( multiplication ) and is suc tat te identity for and inverses under te operation are contained in te set. Verify (2.1.7). Hint: Use (2.1.6) for closure; matrix multiplication is associative; I 2 M n is unitary; and U = U 1 is again unitary. Te set (group) of unitary matrices in M n as anoter very important property. Notions of convergence and limit of a sequence of matrices will be presented precisely in Capter 5, but can be understood ere in terms of convergence and limit of entries. Te dening identity U U = I means tat every column of U as Euclidean norm 1, and ence no entry of U = [u ij ] can ave absolute value greater tan 1. If we tink of te set of unitary matrices as a subset of C n2, tis says it is a bounded subset. If U k [u (k) ij ] is an innite sequence of unitary matrices, k = 1; 2; : : : suc tat lim k!1 u (k) ij u ij exists for all i; j = 1; 2; : : : ; n, ten from te identity Uk U k = I for all k = 1; 2; : : : we see tat lim k!1 Uk U k = U U = I, in wic U = [u ij ]. Tus, te limit matrix U is also unitary. Tis says tat te set of unitary matrices is a closed subset of C n2. Since a closed and bounded subset of a nite dimensional Euclidean space is a compact set (see Appendix E), we conclude tat te set (group) of unitary matrices in M n is compact. For our purposes, te most important consequence of tis observation is te following selection principle for unitary matrices Lemma. Let U 1 ; U 2 ; : : : 2 M n be a given innite sequence of unitary matrices. Tere exists an innite subsequence U k1 ; U k2 ; : : :, 1 k 1 < k 2 <, suc tat all of te entries of U ki converge (as sequences of complex numbers) to te entries of a unitary matrix as i! 1. Proof: All tat is required ere is te fact tat from any innite sequence in a compact set one may always select a convergent subsequence. We ave already observed tat if a sequence of unitary matrices converges to some matrix, ten te limit matrix must be unitary.

5 2.1 Unitary matrices and te QR factorization 105 Te unitary limit guaranteed by te lemma need not be unique; it can depend upon te subsequence cosen. i Exercise. Consider te sequence of unitary matrices U k = 0 1 k; 1 0 k = 1; 2; : : : : Sow tat tere are two possible limits of subsequences. Exercise. Explain wy te selection principle (2.1.8) applies as well to te (real) ortogonal group; tat is, an innite sequence of real ortogonal matrices as an innite subsequence tat converges to a real ortogonal matrix. A unitary matrix U as te property tat U 1 equals U. One way to generalize te notion of a unitary matrix is to require tat U 1 be similar to U. Te set of suc matrices is easily caracterized as te range of te mapping A! A 1 A for all nonsingular A 2 M n Teorem. Let A 2 M n be nonsingular. Ten A 1 is similar to A if and only if tere is a nonsingular B 2 M n suc tat A = B 1 B. Proof: If A = B 1 B for some nonsingular B 2 M n, ten A 1 = (B ) 1 B and B A 1 (B ) 1 = B(B ) 1 = (B 1 B ) = A, so A 1 is similar to A via te similarity matrix B. Conversely, if A 1 is similar to A, ten tere is a nonsingular S 2 M n suc tat SA 1 S 1 = A and ence S = A SA. Set S e i S for 2 R, so tat S = A S A and S = A S A. Adding tese two identities gives H = A H A, in wic H S + S is Hermitian. If H were singular, tere would be a nonzero x 2 C n suc tat 0 = H x = S x + S x, so 1 x = S x = e 2i S 1 S x and S 1 S x = e 2i x. Coose a value of = 0 2 [0; 2) suc tat e 2i0 is not an eigenvalue of S 1 S ; te resulting Hermitian matrix H H 0 is nonsingular and as te property tat H = A HA. Now coose any complex suc tat jj = 1 and is not an eigenvalue of A. Set B (I A )H, in wic te complex parameter 6= 0 is to be cosen, and observe tat B is nonsingular. We want to ave A = B 1 B, or BA = B. Compute B = H( I A), and BA = (I A )HA = (HA A HA) = (HA H) = H(A I). We sall be done if we can select a nonzero suc tat =, but if = e i i(, ten = e )=2 will do. If a unitary matrix is presented as a 2-by-2 block matrix, ten te ranks of its off-diagonal blocks are equal; te ranks of its diagonal blocks are related by a simple formula Lemma. Let a unitary U 2 M n be partitioned as U = U11 U 12 U 21 U 22 i,

6 106 Unitary similarity and unitary equivalence in wic U 11 2 M k. Ten rank U 12 = rank U 21 and rank U 22 = rank U 11 + n 2k. In particular, U 12 = 0 if and only if U 21 = 0, in wic case U 11 and U 22 are unitary. Proof: Te two assertions about rank follow immediately from te ilaw of complementary nullities (0.7.5) using te fact tat U 1 =. U 11 U 21 U 12 U 22 Plane rotations and Houseolder matrices are special (and very simple) unitary matrices tat play an important role in establising some basic matrix factorizations Example. plane rotations. Let U(; i; j) = cos 0 : : : 0 sin row j sin 0 : : : 0 cos row i 7 5 mn i column j Tis is simply te identity matrix, wit te i; i and j; j entries replaced by cos and te i; j entry (respectively j; i entry) replaced by sin (respectively, sin ). Exercise. Verify tat U(; i; j) 2 M n (R) is real ortogonal for any pair of indices 1 i < j n and any parameter 0 < 2. Te matrix U(; i; j) carries out a rotation (troug an angle ) in te i, j coordinate plane of R n. Left multiplication by U(; i; j) affects only rows i and j of te matrix multiplied; rigt multiplication by U(; i; j) affects only columns i and j of te matrix multiplied Example. Houseolder matrices. Let w 2 C n be a nonzero vector.

7 2.1 Unitary matrices and te QR factorization 107 Te Houseolder matrix U w 2 M n is dened by U w = I 2(w w) 1 ww. If w is a unit vector, ten U w = I 2ww. Exercise. Sow tat a Houseolder matrix U w is bot unitary and Hermitian; if w 2 R n ten U w is real ortogonal and symmetric. Exercise. Sow tat a Houseolder matrix U w acts as te identity on te subspace w? and tat it acts as a reection on te one-dimensional subspace spanned by w; tat is, U w x = x if x? w and U w w = w. Exercise. Use ( ) to sow tat det U w = 1 for all n. Tus, for all n and every nonzero w 2 R n, te Houseolder matrix U w 2 M n (R) is a real ortogonal matrix tat is never a proper rotation matrix (a real ortogonal matrix wose determinant is +1). Exercise. Use (1.2.8) to sow tat te eigenvalues of a Houseolder matrix are always 1; 1; : : : ; 1 and explain wy its determinant is always 1. Houseolder matrices provide a simple way to construct a unitary matrix tat takes a given vector into any oter vector tat as te same Euclidean norm Teorem. Let x; y 2 C n be given and suppose kxk 2 = kyk 2. If y = e i x for some real, let U(y; x) = e i I n ; oterwise let 2 [0; 2) be suc tat x y = e i jx yj (take = 0 if x y = 0), let w = e i x y, and let U(y; x) = e i U w, in wic U w is te Houseolder matrix U w = I 2(w w) 1 ww. Ten U(y; x) is unitary and essentially Hermitian, U(y; x)x = y, and U(y; x)z? y wenever z? x. If x and y are real, ten U(y; x) is real ortogonal: U(y; x) = I if y = x, and U(y; x) is te real Houseolder matrix U x y oterwise. Proof: Te assertions are readily veried if x and y are linearly dependent, tat is, if y = e i x for some real. If x and y are linearly independent, te Caucy-Scwarz inequality (0.6.3) ensures tat x x 6= jx yj. Compute and w w = (e i x y) (e i x y) = x x e i x y e i y x + y y = 2(x x Re(e i x y)) = 2(x x jx yj) w x = e i x x y x = e i x x e i jy xj = e i (x x jx yj) and, nally, e i U w x = e i (x 2(w w) 1 ww x) = e i (x (e i x y)e i ) = y

8 108 Unitary similarity and unitary equivalence If z is ortogonal to x, ten w z = y U(y; x)z = e i y z kxk 2 2 y z and 1 jx yj = e i (y z + ( y x)) = 0! e i y x kyk 2 2 ( y x) Since U w is unitary and Hermitian, U(y; x) = (e i I)U w is unitary (as a product of two unitary matrices) and essentially Hermitian (0.2.5). Exercise. Let y 2 C n be a given unit vector and let e 1 be te rst column of te n-by-n identity matrix. Construct U(y; e 1 ) using te recipe in te preceding teorem and verify tat its rst column is y (wic it sould be, since y = U(y; e 1 )e 1 ). Exercise. Let x 2 C n be a given nonzero vector. Explain wy te matrix U(kxk 2 e 1 ; x) constructed in te preceding teorem is an essentially Hermitian unitary matrix tat takes x into kxk 2 e 1. Te following QR factorization of a complex or real matrix is of considerable teoretical and computational importance Teorem. (QR factorization) Suppose A 2 M n;m and n m. Ten (a) Tere is a Q 2 M n;m wit ortonormal columns and an upper triangular R 2 M m wit non-negative main diagonal entries suc tat A = QR. (b) If rank A = m, ten te factors Q and R in (a) are uniquely determined and te main diagonal entries of R are all positive. (c) If m = n, ten te factor Q in (a) is unitary. (d) If A is real, ten bot of te factors Q and R in (a) may be taken to be real. Proof: Let a 1 2 C n be te rst column of A, let r 1 = ka 1 k 2, and let U 1 be a unitary matrix suc tat U 1 a 1 = r 1 e 1. Teorem (2.1.13) gives an explicit construction for suc a matrix, wic is eiter a unitary scalar matrix or te product of a unitary scalar matrix and a Houseolder matrix. Partition r1 F U 1 A = 0 A 2 in wic A 2 2 M n 1;m 1. Let a 2 2 C n 1 be te rst column of A 2 and let r 2 = ka 2 k 2. Use (2.1.13) again to construct a unitary V 2 2 M n 1 suc tat V 2 a 2 = r 2 e 1 and let U 2 = [I 1 ] V 2. Ten 2 r 1 F 3 U 2 U 1 A = 4 0 r A 3

9 2.1 Unitary matrices and te QR factorization 109 Repeat tis construction m times to obtain U m U m 1 U 2 U 1 A = R 0 in wic R 2 M m is upper triangular. Its main diagonal entries are r 1 ; : : : ; r m are all nonnegative. Let U = U m U m 1 U 2 U 1. Partition U = U 1 U 2 U m 1U m = [Q Q 2 ], in wic Q 2 M n;m as ortonormal columns since it comprises te rst m columns of a unitary matrix. Ten A = QR, as desired. If A as full column rank, ten R is nonsingular, so its main diagonal entries are all positive. Suppose tat rank A = m and A = QR = ~ Q ~ R, in wic R and ~ R are upper triangular and ave positive main diagonal entries, and Q and ~ Q ave ortonormal columns. Ten A A = R (Q Q)R = R IR = R R and also A A = ~ R ~ R, so R R = ~ R ~ R and ~ R R = ~ RR 1. Tis says tat a lower triangular matrix equals an upper triangular matrix, so bot must be diagonal: ~RR 1 = D is diagonal, and it must ave positive main diagonal entries because te main diagonal entries of bot ~ R and R 1 are positive. But ~ R = DR implies tat D = ~ RR 1 = ~ R R = (DR) R = D 1 R R = D 1, so D 2 = I and ence D = I. We conclude tat ~ R = R and ence ~ Q = Q. Te assertion in (c) follows from te fact tat a square matrix wit ortonormal columns is unitary. Te nal assertion (d) follows from te construction in (a) and te assurance in (2.1.13) tat te unitary matrices U i may all be cosen to be real. Exercise. Sow tat any B 2 M n of te form B = A A, A 2 M n, may be written as B = LL, in wic L 2 M n is lower triangular and as nonnegative diagonal entries. Explain wy tis factorization is unique if A is nonsingular. Tis is called te Colesky factorization of B; every positive denite matrix may be factored in tis way (see Capter 7). For square matrices A 2 M n, tere are some easy variants of te QR factorization tat can be useful. Let K be te (real ortogonal and symmetric) n-by-n reversal matrix ( ), wic as te pleasant property tat K 2 = I. Moreover, KRK is lower triangular if R is upper triangular (te main diagonal entries are te same, but te order is reversed), and of course KLK is upper triangular if L is lower triangular. If we factor KAK = QR as in (2.1.14), ten A = (KQK)(KRK) = Q 1 L, in wic Q 1 = KQK is unitary and L is lower triangular wit nonnegative main diagonal entries; we call tis a QL factorization of A. Now let A = QL be a QL factorization of A, and observe tat A = L Q, wic is an RQ factorization of A. Finally, factoring

10 110 Unitary similarity and unitary equivalence KA K = QL gives A = (KLK) (BQB), wic is an LQ factorization of A Corollary. Let A 2 M n be given. Ten tere are unitary matrices Q 1 ; Q 2 ; Q 3, lower triangular matrices L 2 ; L 3 wit nonnegative main diagonal entries, and an upper triangular matrix R 2 wit nonnegative main diagonal entries suc tat A = Q 1 L 1 = R 2 Q 2 = L 3 Q 3. If A is nonsingular, ten te respective unitary and triangular factors are uniquely determined and te main diagonal entries of te triangular factors are all positive. If A is real, ten all of te factors Q 1 ; Q 2 ; Q 3 ; L 2 ; L 3 ; R 2 may be cosen to be real. Problems 1. If U 2 M n is unitary, sow tat j det Uj = Let U 2 M n be unitary and let be a given eigenvalue of U. Sow tat (a) jj = 1 and (b) x is a (rigt) eigenvector of U corresponding to if and only if x is a left eigenvector of U corresponding to. Hint: Use (2.1.4g) and Problem 1 in (1.1). 3. Given real parameters 1 ; 2 ; : : : ; n, sow tat U = diag(e i1 ; e i2 ; : : : ; e in )is unitary. Sow tat every diagonal unitary matrix as tis form. 4. Caracterize te diagonal real ortogonal matrices. 5. Sow tat te permutation matrices (0.9.5) in M n are a subgroup (a subset tat is itself a group) of te group of real ortogonal matrices. How many different permutation matrices are tere in M n? 6. Give a presentation in terms of parameters of te 3-by-3 ortogonal group. Two presentations of te 2-by-2 ortogonal group are given in (2.1). 7. Suppose A; B 2 M n and AB = I. Provide details for te following argument tat BA = I: Every y 2 C n can be represented as y = A(By), so rank A = n and ence dim(nullspace(a)) = 0 ( ). Compute A(AB BA) = A(I BA) = A (AB)A = A A = 0, so AB BA = A matrix A 2 M n is complex ortogonal if A T A = I. A real ortogonal matrix is unitary, i but a nonreal ortogonal matrix need not be unitary. (a) Let K = 2 M 2 (R). Sow tat A(t) = (cost)i + (i sin t)k 2 M is complex ortogonal for all t 2 R, but tat A(t) is unitary only for t = 0. Te yperbolic functions are dened by cos t = (e t + e t )=2, sin t = (e t e t )=2. (b) Sow tat, unlike te unitary matrices, te set of complex ortogonal matrices is not a bounded set, and it is terefore not a compact set. (c) Sow tat te set of complex ortogonal matrices of a given size forms

11 2.1 Unitary matrices and te QR factorization 111 a group. Te smaller (and compact) group of real ortogonal matrices of a given size is often called te ortogonal group. (d) If A 2 M n is complex ortogonal, sow tat j det Aj = 1; consider A(t) in (a) to sow tat A can ave eigenvalues wit jj 6= 1. (e) If A 2 M n is complex ortogonal, sow tat A, A T, and A are all complex ortogonal and nonsingular. Do te rows or columns of A form an ortogonal set? (f) Caracterize te diagonal complex ortogonal matrices. Compare wit Problem 4. (g) Sow tat A 2 M n is bot complex ortogonal and unitary if and only if it is real ortogonal. 9. If U 2 M n is unitary, sow tat U, U T, and U are all unitary. 10. If U 2 M n is unitary, sow tat x; y 2 C n are ortogonal if and only if Ux and Uy are ortogonal. 11. A nonsingular matrix A 2 M n is skew ortogonal if A 1 = A T. Sow tat A is skew-ortogonal if and only if ia is ortogonal. More generally, if 2 R, sow tat A 1 = e i A T if and only if e i=2 A is ortogonal. Wat is tis for =? for = 0? 12. Sow tat if A 2 M n is similar to a unitary matrix, ten A 1 is similar to A. 13. Consider diag (2; 1 2 ) 2 M 2 and sow tat te set of matrices tat are similar to unitary matrices is a proper subset of te set of matrices A for wic A 1 is similar to A. 14. Sow tat te intersection of te group of unitary matrices in M n wit te group of complex ortogonal matrices in M n is te group of real ortogonal matrices in M n. Hint: U 1 = U T = U. 15. If U 2 M n is unitary, f1; : : : ; ng, and U[; c ] = 0, (0.7.1) sow tat U[ c ; ] = 0, and U[] and U[ c ] are unitary. 16. Let x; y 2 R n be given linearly independent unit vectors and let w = x + y. Consider te Palais matrix P x;y = I 2(w T w) 1 ww T + 2yx T. Sow tat: (a) P x;y = (I 2(w T w) 1 ww T )(I 2xx T ) = U w U x is a product of two real Houseolder matrices, so it is a real ortogonal matrix; (b) det P x;y = +1, so P x;y is always a proper rotation matrix; (c) P x;y x = y and P x;y y = x + 2(x T y)y; (d) P x;y z = z if z 2 R n, z? x; and z? y; (e) P x;y acts as te identity on te (n 2)-dimensional subspace (spanfx; yg)? and it is a proper rotation on te 2-dimensional subspace spanfx; yg tat takes x into y; (f) If n = 3, explain wy P x;y is te unique proper rotation tat takes x into y and leaves xed teir vector cross product x y; (g) te eigenvalues of P x;y are x T y i(1 (x T y) 2 ) 1=2 = e i ; 1; : : : ; 1, in wic cos = x T y.

12 112 Unitary similarity and unitary equivalence Hint: (1.3.23) te eigenvalues of [w x] T [ (w T w) 1 w y] 2 M 2 (R) are 1 2 (xt y 1 i(1 (x T y) 2 ) 1=2 ). 17. Suppose tat A 2 M n;m, n m, and rank A = m. Describe te steps of te Gram-Scmidt process applied to te columns of A, proceeding from left to rigt. Explain wy tis process produces, column-by-column, an explicit matrix Q 2 M n;m wit ortonormal columns and an explicit upper triangular matrix R 2 M m suc tat Q = AR. How is tis factorization related to te one in (2.1.14)? 18. Let A 2 M n be factored as A = QR as in (2.1.14), partition A = [a 1 : : : a n ] and Q = [q 1 : : : q n ] according to teir columns and let R = [r ij ] n i;j=1. (a) Explain wy fq 1; : : : ; q k g is an ortonormal basis for spanfa 1 ; : : : ; a k g for eac k = 1; : : : ; n. (b) Sow tat r kk is te Euclidean distance from a k to spanfa 1 ; : : : ; a k 1 g for eac k = 2; : : : ; n. 19. Let X = [x 1 : : : x m ] 2 M n;m, suppose rank X = m, and factor X = QR as in (2.1.14). Let Y = QR = [y 1 : : : y m ]. Sow tat te columns of Y are a basis for te subspace S = spanfx 1 ; : : : ; x m g and tat Y X = I m, so y i x j = 0 if i 6= j and eac y i x i = 1. One says tat fy 1 ; : : : ; y m g is te basis of S tat is dual (reciprocal) to te basis fx 1 ; : : : ; x m g. 20. If U 2 M n is unitary, sow tat adj U = det(u)u. 21. Explain wy Lemma remains true if unitary is replaced wit complex ortogonal. 22. Suppose tat X; Y 2 M n;m ave ortonormal columns. Sow tat X and Y ave te same range (column space) if and only if tere is a unitary U 2 M m suc tat X = Y U. Hint: (0.2.7). 23. Let A 2 M n, let A = QR be a QR factorization, let R = [r ij ], and partition bot A and R according to teir columns: A = [a 1 : : : a n ] and R = [r 1 : : : r n ]. Explain wy ka i k 2 = kr i k 2 for eac i = 1; : : : ; n, j det Aj = det R = r 11 r nn, and r ii kr i k 2 for eac i = 1; : : : ; n. Conclude tat j det Aj Q n i=1 ka ik 2. Tis is known as Hadamard's inequality. Furter Reading. For more information about matrices tat satisfy te conditions of (2.1.9), see C. R. DePrima and C. R. Jonson, Te Range of A 1 A in GL(n; C), Linear Algebra Appl. 9 (1974) Unitary similarity Since U = U 1 for unitary U, te transformation on M n given by A! U AU is a similarity transformation if U is unitary. Tis special type of similarity is called unitary similarity.

13 2.2 Unitary similarity Denition. Let A; B 2 M n be given. We say tat A is unitarily similar to B if tere is a unitary U 2 M n suc tat A = UBU. If U may be taken to be real (and ence is real ortogonal), ten A is said to be real ortogonally similar to B. We say tat A is unitarily diagonalizable if it is unitarily similar to a diagonal matrix; A is real ortogonally diagonalizable if it is real ortogonally similar to a diagonal matrix. Exercise. Sow tat unitary similarity is an equivalence relation Teorem. Let U 2 M n and V 2 M m be unitary, let A = [a ij ] 2 M n;m and B = [b ij ] 2 M n;m, and suppose A = UBV. Ten P n;m i;j=1 jb ijj 2 = P n;m i;j=1 ja ijj 2. In particular, tis identity is satised if m = n and V = U, tat is, if A is unitarily similar to B. Proof: It sufces to ceck tat tr B B = tr A A. (0.2.5) Compute tr A A = tr(ubv ) (UBV ) = tr(v B U UBV ) = tr V B BV = tr B BV V = tr B B. Exercise. Sow tat te matrices unitarily similar i and i are similar but not Unitary similarity implies similarity, but not conversely. Te unitary similarity equivalence relation partitions M n into ner equivalence classes tan te similarity equivalence relation. Unitary similarity, like similarity, corresponds to a cange of basis, but of a special type it corresponds to a cange from one ortonormal basis to anoter. Exercise. Using te notation of (2.1.11), explain wy only rows and columns i and j are canged under real ortogonal similarity via te plane rotation U(; i; j). Exercise. Using te notation of (2.1.13), explain wy U(y; x) AU(y; x) = U wau w for any A 2 M n, tat is, a unitary similarity via an essentially Hermitian unitary matrix of te form U(y; x) is a unitary similarity via a Houseolder matrix. Unitary (or real ortogonal) similarity via a Houseolder matrix is often called a Houseolder transformation. For computational or teoretical reasons, it is often convenient to transform a given matrix by unitary similarity into anoter matrix wit a special form. Here are two examples Example. Suppose A = [a ij ] 2 M n is given. We claim tat tere is a unitary U 2 M n suc tat all te main diagonal entries of U AU = B = [b ij ] are equal; if A is real, ten U may be taken to be real ortogonal. If tis claim

14 114 Unitary similarity and unitary equivalence is true, ten tr A = tr B = nb 11, so every main diagonal entry of B is equal to te average of te main diagonal entries of A. Begin by considering te complex case and n = 2. Since we can replace A 2 M 2 by A ( 1 2 tr A)I, tere is no loss of generality to assume tat tr A = 0, in wic case te two eigenvalues of A are for some 2 C. We wis to determine a unit vector u suc tat u Au = 0. If = 0, let u be any unit vector suc tat Au = 0. If 6= 0, let w and z be any unit eigenvectors associated wit te distinct eigenvalues. Let x() = e i w + z, wic is nonzero for all 2 R since w and z are linearly independent. Compute x() Ax() = (e i w + z) (e i w z) = 2i Im(e i z w). If z w = e i jz wj,ten x( ) Ax( ) = 0. Let u = x( )= kx( )k 2. Now let v 2 C 2 be any unit vector tat is ortogonal to u and let U = [u v]. Ten U is unitary and (U AU) 11 = u Au = 0. But tr(u AU) = 0, so (U AU) 22 = 0 as well. Now suppose n = 2 and A is real. If te diagonal entries of A = i [a ij ] are not equal, consider te plane rotation matrix U = cos sin. A calculation reveals tat te diagonal entries of U AU T are equal if (cos2 sin 2 )(a 11 a 22 ) = 2 sin cos (a 12 + a 21 ), so equal diagonal entries are acieved if 2 (0; =2) is cosen so tat cot 2 = (a 12 + a 21 )=(a 11 a 22 ). We ave now sown tat any 2-by-2 complex matrix A is unitarily similar to a matrix wit bot diagonal entries equal to te average of te diagonal entries of A; if A is real, te similarity may be taken to be real ortogonal. Now suppose n > 2 and dene f(a) = maxfja ii a jj j : i; j = 1; 2; : : : ; ng. If f(a) > 0, let A 2 = aii a ij for a pair of indices i; j for wic a ji a jj i f(a) = ja ii a jj j (tere could be several pairs of indices for wic tis maximum positive separation is attained; coose any one of tem). Let U 2 2 M 2 be unitary, real if A is real, and suc tat U2 A 2 U 2 as bot main diagonal entries equal to 1 2 (a ii + a jj ). Construct U(i; j) 2 M n from U 2 in te same way tat U(; i; j) was constructed from a 2-by-2 plane rotation in (2.1.11). Te unitary similarity U(i; j) AU(i; j) affects only entries in rows and columns i and j, so it leaves uncanged every main diagonal entry of A except te entries in positions i and j, wic it replaces wit te average 1 2 (a ii + a jj ). For any k 6= i; j te triangle inequality ensures tat sin ja kk 1 2 (a ii + a jj )j = j 1 2 (a kk a ii ) (a kk a jj )j 1 2 ja kk a ii j ja kk a jj j cos 1 2 f(a) + 1 f(a) = f(a) 2

15 2.2 Unitary similarity 115 wit equality only if te scalars a kk a ii and a kk a jj bot lie on te same ray in te complex plane and ja kk a ii j = ja kk a jj j. Tese two conditions 1 imply tat a ii = a jj, so it follows tat ja kk 2 (a ii + a jj )j < f(a) for all k 6= i; j. Tus, te unitary similarity we ave just constructed reduces by one te nitely many pairs of indices k; ` for wic f(a) = ja kk a``j. Repeat te construction, if necessary, to deal wit any suc remaining pairs and acieve a unitary U (real if A is real) suc tat f(u AU) < f(a). Finally, consider te compact set R(A) = fu AU : U 2 M n is unitary}. Since f is a continuous nonnegative-valued function on R(A), it acieves its minimum value tere, tat is, tere is some B 2 R(A) suc tat f(a) f(b) 0 for all A 2 R(A). If f(b) > 0, we ave just seen tat tere is a unitary U (real if A is real) suc tat f(b) > f(u BU). Tis contradiction sows tat f(b) = 0, so all te diagonal entries of B are equal Example. Suppose A = [a ij ] 2 M n is given. Te following construction sows tat A is unitarily similar to an upper Hessenberg matrix wit nonnegative entries in its rst subdiagonal. Let a 1 be te rst column of A, partitioned as a T 1 = [a 11 T ] wit 2 C n 1. Let U 1 = I n 1 if = 0; oterwise, use (2.1.13) to construct U 1 = U(kk 2 e 1 ; ) 2 M n 1, a unitary matrix tat takes into a positive multiple of e 1. Form te unitary matrix V 1 = I 1 U 1 and observe tat te rst column of V 1 A is te vector [a 11 kk 2 0] T. Moreover, A 1 = (V 1 A)V1 as te same rst column as V 1 A and is unitarily similar to A. Partition it as 2 3 a 11 F A 1 = 4 kk2 5, A 2 2 M n 1 A 2 0 Use (2.1.13) again to form, in te same way, a unitary matrix U 2 tat takes te rst column of A 2 into a vector wose entries below te second are all zero and wose second entry is nonnegative. Let V 2 = I 2 U 2 and let A 2 = V 2 AV2. Tis similarity does not affect te rst column of A 1. After at most n 1 steps, tis construction produces an upper Hessenberg matrix A n 1 tat is unitarily similar to A and as nonnegative subdiagonal entries. Exercise. If A is Hermitian or skew-hermitian, explain wy te construction in te preceding example produces a tridiagonal Hermitian or skew-hermitian matrix tat is unitarily similar to A. Teorem (2.2.2) provides a necessary but not sufcient condition for two given matrices to be unitarily similar. It can be augmented wit additional identities tat collectively do provide necessary and sufcient conditions. A

16 116 Unitary similarity and unitary equivalence key role is played by te following simple notion. Let s, t be two given noncommuting variables. We refer to any nite formal product of nonnegative powers of s, t W (s; t) = s m1 t n1 s m2 t n2 s m k t n k ; m 1 ; n 1 ; : : : ; m k ; n k 0 (2.2.5) as a word in s and t. Te degree of te word W (s; t) is te nonnegative integer m 1 + n 1 + m 2 + n m k + n k, tat is, te sum of all te exponents in te word. If A 2 M n is given, we dene a word in A and A as W (A; A ) = A m1 (A ) n1 A m2 (A ) n2 A m k (A ) n k Since te powers of A and A need not commute, it may not be possible to simplify te expression of W (A; A ) by rearranging te terms in te product. Suppose A is unitarily similar to B 2 M n, so tat A = UBU for some unitary U 2 M n. For any word W (s; t) we ave W (A; A ) = (UBU ) m1 (UB U ) n1 (UBU ) m k (UB U ) n k = UB m1 U U(B ) n1 U UB m k U U(B ) n k U = UB m1 (B ) n k B m k (B ) n k U = UW (B; B )U so W (A; A ) is unitarily similar to W (B; B ). Tus, tr W (A; A ) = trw (B; B ). If we take te word W (s; t) = ts, we obtain te identity in (2.2.2). If one considers all possible words W (s; t), tis observation gives innitely many necessary conditions for two matrices to be unitarily similar. A teorem of W. Spect, wic we state witout proof, guarantees tat tese necessary conditions are also sufcient Teorem. Two matrices A; B 2 M n are unitarily similar if and only if tr W (A; A ) = tr W (B; B ) (2.2.7) for every word W (s; t) in two noncommuting variables. Spect's teorem can be used to sow tat two matrices are not unitarily similar by exibiting a specic word tat violates (2.2.7). However, except in special situations (see Problem 6), it may be useless in sowing tat two given matrices are unitarily similar because innitely many conditions must be veried. Fortunately, a renement of Spect's teorem says tat it sufces to ceck te trace identities (2.2.7) for only nitely many words, wic gives a practical criterion to assess unitary similarity of matrices of small size.

17 2.2 Unitary similarity Teorem. Two matrices A; B 2 M n are unitarily similar if and only if tr W (A; A ) = tr W (B; B ) for every word W (s; t) in two noncommuting variables wose degree is at most r 2n 2 n n n 2 2 For n = 2, it sufces to verify (2.2.7) for te tree words W (s; t) = s; s 2, and st. For n = 3, it sufces to verify (2.2.7) for te seven words W (s; t) = s; s 2 ; st; s 3 ; s 2 t; s 2 t 2, and s 2 t 2 st. Problems 1. Let A = [a ij ] 2 M n (R) be symmetric but not diagonal, and suppose tat indices i; j wit i < j are cosen so tat ja ij j is as large as possible. Dene by cot 2 = (a jj a ii )=2a ij, let U(; i; j) be te plane rotation (2.1.11), and let B = U(; i; j)au(; i; j) T = [b pq ]. Sow tat b ij = 0 and use (2.2.2) to sow tat P p6=q jb pqj 2 < P p6=q ja pqj 2. Indeed, it is not necessary to compute ; just take cos = a jj (a 2 ij + a2 jj ) 1=2 and sin = a ij (a 2 ij + a2 jj ) 1=2. Sow tat repeated real ortogonal similarities via plane rotations (cosen in te same way for B and its successors) strictly decrease te sums of te squares of te off-diagonal entries wile preserving te sums of te squares of all te entries; at eac step, te computed matrix is (in tis sense) more nearly diagonal tan at te step before. Tis is te metod of Jacobi for calculating te eigenvalues of a real symmetric matrix. It produces a sequence of matrices tat converges to a real diagonal matrix. Wy must te diagonal entries of te limit be te eigenvalues of A? How can te corresponding eigenvectors be obtained? 2. Te eigenvalue calculation metod of Givens for real matrices also uses plane rotations, but in a different way. For n 3, provide details for te following argument sowing tat every A = [a ij ] 2 M n (R) is real ortogonally similar to a real lower Hessenberg matrix, wic is necessarily tridiagonal if A is symmetric; see (0.9.9) and (0.9.10). Coose a plane rotation U 1;3 of te form U(; 1; 3), as in te preceding problem, so tat te 1,3 entry of U1;3AU 1;3 is 0. Coose anoter plane rotation of te form U 1;4 = U(; 1; 4) so tat te 1,4 entry of U1;4(U 1;3AU 1;3 )U 1;4 is 0; continue in tis way to zero out te rest of te rst row wit a sequence of real ortogonal similarities. Ten start on te second row beginning wit te 2,4 entry and zero out te 2,4, 2,5,..., 2,n entries. Explain wy tis process does not disturb previously manufactured 0 entries, and wy it preserves symmetry if A is symmetric. Proceeding in tis way troug row n 3 produces a lower Hessenberg matrix after nitely many

18 118 Unitary similarity and unitary equivalence real ortogonal similarities via plane rotations; tat matrix is tridiagonal if A is symmetric. However, te eigenvalues of A are not displayed as in Jacobi's metod; tey must be obtained from a furter calculation. 3. Sow tat every A 2 M 2 is unitarily similar to its transpose. Hint: Consider te tree words W (s; t) = s; s 2 ; st. 4. Let 2 A = Any matrix is similar to its transpose (3.2.3), but A is not unitarily similar to A T. For wic of te seven words listed in (2.2.8) do A and B = A T fail te test (2.2.7)? 5. If A 2 M n and tere is a unitary U 2 M n suc tat A = UAU, sow tat A + A = U(A + A )U, tat is, U commutes wit A + A. Apply tis observation to te 3-by-3 matrix in te preceding problem and conclude tat if it is unitarily similar to its transpose, ten any suc unitary similarity must be diagonal. Sow tat no diagonal unitary similarity can take tis matrix into its transpose. 6. Let A 2 M n and B; C 2 M m be given. Use eiter (2.2.6) or (2.2.8) to sow tat B and C are unitarily similar if and only if any one of te following conditions olds: i (a) A 0 0 B and A 0 0 C i are unitarily similar. (b) B B and C C are unitarily similar if bot direct sums contain te same number of terms. (c) A B B and A C C are unitarily similar if bot direct sums contain te same number of terms. 7. Give an example of two 2-by-2 matrices tat satisfy te identity (2.2.2) but are not unitarily similar. Explain wy. 8. Let A; B 2 M 2 and let C = AB BA. Use Example to sow tat i 2 C 2 = I for some scalar. Hint: tr C =?; 0 =? b a 0 9. Let A 2 M n and suppose tr A = 0. Use Example to sow tat A can be written as a sum of two nilpotent matrices. Conversely, if A can be written as a sum of nilpotent matrices, explain wy tr A = 0. Hint: Write A = UBU, in wic B = [b ij ] as zero main diagonal entries. Ten write B = B L + B R, in wic B L = [ ij ], ij = b ij if i j and ij = 0 if j > i.

19 2.3 Unitary triangularizations Let n 2 be a given integer and dene! = e 2i=n. (a) Explain wy P n 1 k=0!k` = 0 unless ` = mn for some m = 0; 1; 2; : : :, in wic case te sum is equal to n. (b) Let F n = n 1=2 [! (i 1)(j 1) ] n i;j=1 denote te n- by-n Fourier matrix. Sow tat F n is symmetric, unitary, and coninvolutory: F n Fn = F n F n = I. (c) Let C n denote te basic circulant permutation matrix ( ). Explain wy C n is unitary (real ortogonal). (d) Let D = diag(1;!;! 2 ; : : : ;! n 1 ) and sow tat C n F n = F n D, so C n = F n DFn and Cn k = F n D k Fn for all k = 1; 2; : : : : (e) Let A denote te circulant matrix ( ), expressed as te sum in ( ). Explain wy A = F n Fn, in wic = diag( 1 ; : : : ; n ), eac ` = P n 1 k=0 a k+1! k(` 1), and te diagonal entries of are te entries of te vector n 1=2 FnAe 1. Tus, te Fourier matrix provides an explicit unitary diagonalization for every circulant matrix. (f) Write F n = C n + is n, in wic C n and S n are real. Wat are te entries of C n and S n? Let H n = C n + S n denote te n-by-n Hartley matrix. (g) Sow tat Cn 2 + Sn 2 = I, C n S n = S n C n = 0, H n is symmetric, and H n is real ortogonal. () Let K n denote te reversal matrix ( ). Sow tat C n K n = K n C n = C n, S n K n = K n S n = S n, and H n K n = K n H n, so C n, S n, and H n are centrosymmetric. It is known tat H n AH n = is diagonal for any matrix of te form A = E + K n F, in wic E and F are real circulant matrices, E = E T, and F = F T ; te diagonal entries of are te entries of te vector n 1=2 H n Ae 1. In particular, te Hartley matrix provides an explicit real ortogonal diagonalization for every real symmetric circulant matrix. Furter Readings and Notes. For te original proof of (2.2.6), see W. Spect, Zur Teorie der Matrizen II, Jaresber. Deutsc. Mat.-Verein. 50 (1940) 19 23; tere is a modern proof in [Kap]. For a survey of te issues addressed in (2.2.8), see D. Ðjoković and C. Jonson, Unitarily Acievable Zero Patterns and Traces of Words in A and A, Linear Algebra Appl. 421 (2007) Unitary triangularizations Peraps te most fundamentally useful fact of elementary matrix teory is a teorem attributed to I. Scur: any square complex matrix A is unitarily similar to a triangular matrix wose diagonal entries are te eigenvalues of A. Te proof involves a sequential deation by unitary similarity Teorem. (Scur) Let A 2 M n ave eigenvalues 1 ; : : : ; n in any prescribed order and let x be a unit vector suc tat Ax = 1 x. Ten tere is a unitary U = [x u 2 : : : u n ] 2 M n suc tat U AU = T = [t ij ] is upper triangular wit diagonal entries t ii = i ; i = 1; : : : ; n. Tat is, every square

20 120 Unitary similarity and unitary equivalence complex matrix A is unitarily similar to an upper triangular matrix wose diagonal entries are te eigenvalues of A in any prescribed order. Furtermore, if A 2 M n (R) and if all its eigenvalues are real, ten U may be cosen to be real ortogonal. Proof: Let x be a normalized eigenvector of A associated wit te eigenvalue 1, tat is, x x = 1 and Ax = 1 x. Let U 1 = [x u 2 : : : u n ] be any unitary matrix wose rst column is x. For example, one may take U 1 = U(x; e 1 ) as in (2.1.13) or see Problem 1. Ten U1 AU 1 = U1 Ax Au2 : : : Au n = U 1 1 x Au 2 : : : Au n = = x u 2. u n x Au 2 : : : Au n 5 1 x x x Au 2 : : : x Au n 1 u 2x. A 1 1 u nx = 1 F 0 A 1 because te columns of U 1 are ortonormal. Te eigenvalues of te submatrix A 1 = [u i Au j] n i;j=2 2 M n 1 are 2 ; : : : ; n. If n = 2, we ave acieved te desired unitary triangularization. If not, let 2 C n 1 be a normalized eigenvector of A 1 corresponding to 2, and perform te preceding reduction on A 1. If U 2 2 M n 1 is any unitary matrix wose rst column is, ten we ave seen tat U 2 A 1 U 2 = 2 F 0 A 2 Let V 2 = [1] U 2 and compute te unitary similarity (U 1 V 2 ) AU 1 V 2 = V2 U1 AU 1 V 2 = A 2 Continue tis reduction to produce unitary matrices U i 2 M n i+1 ; i = 1; : : : ; n 1 and unitary matrices V i 2 M n ; i = 2; : : : ; n 2. Te matrix U = U 1 V 2 V 3 V n 2

21 2.3 Unitary triangularizations 121 is unitary and U AU is upper triangular. If all te eigenvalues of A 2 M n (R) are real, ten all of te eigenvectors and unitary matrices in te preceding algoritm can be cosen to be real (Problem 3 in (1.1) and (2.1.13)). Exercise. Follow te proof of (2.3.1) to see tat upper triangular can be replaced by lower triangular in te statement of te teorem wit, of course, a different unitary similarity. Exercise. If te eigenvector x in te proof of (2.3.1) is also a left eigenvector of A, we know tat x A = x. (1.4.7a) Explain wy U1 AU 1 = A 1 i. If every rigt eigenvector of A is also a left eigenvector, explain wy te upper triangular matrix T constructed in (2.3.1) is actually a diagonal matrix Example. If te eigenvalues of A are re-ordered and te corresponding upper triangularization (2.3.1) is performed, te entries of T above te main diagonal can look very different. Consider 2 T 1 = p p 2 5 ; T 2 = ; U = p p Explain wy U is unitary and T 2 = UT 1 U. Exercise. If A = [a ij ] 2 M n as eigenvalues 1 ; : : : ; n and is unitarily similar to an upper triangular matrix T = [t ij ] 2 M n, te diagonal entries of T are te eigenvalues of A in some order. Apply (2.2.2) to A and T to sow tat nx nx X nx j i j 2 = ja ij j 2 jt ij j 2 ja ij j 2 (2.3.2a) i=1 i;j=1 i<j wit equality if and only if T is diagonal. i;j=1 Exercise. If A = [a ij ] and B = [b ij ] 2 M 2 ave te same eigenvalues and if P 2 i;j=1 ja ijj 2 = P 2 i;j=1 jb ijj 2, use te criterion in (2.2.8) to sow tat A and B are unitarily similar. However, consider 2 A = and B = (2.3.2b) wic ave te same eigenvalues and te same sums of squared entries. Use te criterion in (2.2.8) or te exercise following ( ) to sow tat A and B are not unitarily similar. Neverteless, A and B are similar. Wy?

22 122 Unitary similarity and unitary equivalence It is a useful adjunct to (2.3.1) tat a commuting family of matrices may be simultaneously upper triangularized Teorem. Let F M n be a commuting family. Tere is a unitary U 2 M n suc tat U AU is upper triangular for every A 2 F. Proof: Return to te proof of (2.3.1). Exploiting (1.3.19) at eac step of te proof in wic a coice of an eigenvector (and unitary matrix) is made, coose an eigenvector tat is common to every A 2 F and coose a single unitary matrix tat as tis common eigenvector as its rst column; it deates (via unitary similarity) every matrix in F in te same way. Similarity preserves commutativity, and a partitioned multiplication calculation reveals tat, if two matrices of te form A11 A 12 0 A 22 and B11 B 12 0 B 22 commute, ten A 22 and B 22 commute also. Tus, te commuting family property is inerited by te submatrix A i at eac reduction step in te proof of (2.3.1). We conclude tat all ingredients in te U of (2.3.1) may be cosen in te same way for all members of a commuting family, tus verifying (2.3.3). In (2.3.1) we are permitted to specify te main diagonal of T (tat is, we may specify in advance te order in wic te eigenvalues of A appear as te deation progresses), but (2.3.3) makes no suc claim even for a single matrix in F. At eac stage of te deation, te common eigenvector used is associated wit some eigenvalue of eac matrix in F, but we may not be able to specify wic one. We simply take te eigenvalues as tey come, using (1.3.19). If a real matrix A as any non-real eigenvalues, tere is no ope of reducing it to upper triangular form T by a real similarity because te diagonal entries of T would be te eigenvalues of A. However, we can always reduce A to a real quasi-triangular form by a real similarity as well as by a real ortogonal similarity Teorem. Suppose tat A 2 M n (R) as p complex conjugate pairs of non-real eigenvalues 1 = a 1 + ib 1 ; 1 = a 1 ib 1 : : : ; p = a p + ib p ; p = a p ib p in wic all a j ; b j 2 R and all b j 6= 0, and, if 2p < n, an additional n 2p real eigenvalues 1 ; : : : ; n 2p. Ten tere is a nonsingular

23 2.3 Unitary triangularizations 123 S 2 M n (R) suc tat 2 S 1 AS = 6 4 A 1 A 2... F (2.3.5) 0 A n p is real and block upper triangular, and eac diagonal block is eiter 1-by-1 or aj b j 2-by-2. Tere are p real diagonal blocks of te form b j a j i, one for eac conjugate pair of non-real eigenvalues j ; j = a j ib j. Tere are n 2p diagonal blocks of te form [ j ], one for eac of te real eigenvalues 1 ; : : : ; n 2p. Te p 2-by-2 diagonal blocks and te n 2p 1-by-1 diagonal blocks may appear in (2.3.5) in any prescribed order. Te real similarity S may be taken to be a real ortogonal matrix Q; in tis case te 2-by-2 diagonal blocks of Q T AQ ave te form R aj b j j nonsingular real upper triangular matrix. i b j a j R 1 j, in wic eac R j is a Proof: Te proof of (2.3.1) sows ow to deate A by a sequence of real ortogonal similarities corresponding to its real eigenvalues, if any. Problem 33 in (1.3) describes te deation step corresponding to a complex conjugate pair of non-real eigenvalues; repeating tis deation p times acieves te form (2.3.5), wose 2-by-2 diagonal blocks ave te asserted form (wic reveals teir corresponding conjugate pair of eigenvalues). It remains to consider ow te 2-by-2 diagonal blocks would be modied if we were to use only real ortogonal similarities in te 2-by-2 deations. If = a + ib is a non-real eigenvalue of A wit associated eigenvector x = u + iv; u; v 2 Ri n, we ave seen tat fu; vg is linearly independent and A[u v] = [u v] a b. If fu; vg is not an ortonormal set, use te QR factorization (2.1.14) to write [u v] = Q 2 R 2, in wic Q 2 = [q 1 q 2 ] 2 M n;2 (R) as ortonormal columns and R 2 2 M 2 (R) i is nonsingular and upper triangular. Ten A[u v] = AQ 2 R 2 =, so Q 2 R 2 a b b a a b AQ 2 = Q 2 R 2 b a R 1 2 If we let S be a real ortogonal matrix wose rst two columns are q 1 and q 2, we obtain a deation of te asserted form. Of course, if fu; vg is ortonormal, te QR factorization is unnecessary and we can take S to be a real ortogonal matrix wose rst two columns are u and v. Tere is also a real version of (2.3.3). b a

24 124 Unitary similarity and unitary equivalence Teorem. Let F M n (R) be a commuting family. Tere is a real ortogonal Q M n (R) suc tat Q T AQ as te form (2.3.5) for every A 2 F. Exercise. Modify te proof of (2.3.3) to prove (2.3.6) as follows: First deate all members of F using all te common real eigenvectors. Ten consider te common non-real eigenvectors and deate two columns at a time as in te proof of (2.3.4). Notice tat different members of F may ave different numbers of 2-by-2 diagonal blocks after te common real ortogonal similarity, but if one member as a 2-by-2 block in a certain position and anoter member does not, ten commutativity requires tat te latter must ave a pair of equal 1-by-1 blocks tere. Problems 1. Let x 2 C n be a given unit vector and write x = [x 1 y T ] T, in wic x 1 2 C and y 2 C n 1. Coose 2 R suc tat e i x 1 0 and dene z = e i x = [z 1 T ] T, in wic z 1 2 R is nonnegative and 2 C n 1. Consider te Hermitian matrix " # z1 V x = I z 1 Use partitioned multiplication to compute Vx V x = Vx 2. Conclude tat U = e i V x = [x u 2 : : : u n ] is a unitary matrix wose rst column is te given vector x. 2. If x 2 R n is a given unit vector, sow ow to streamline te construction described in Problem 1 to produce a real ortogonal matrix Q 2 M n (R) wose rst column is x. Prove tat your construction works. 3. Let A 2 M n (R). Explain wy te nonreal eigenvalues of A (if any) must occur in conjugate pairs. n i io 4 Consider te family F = ; and sow tat te ypotesis of commutativity in (2.3.3), wile sufcient to imply simultaneous unitary upper triangularizability of F, is not necessary. 5. Let F = fa 1 ; : : : ; A k g M n be a given family, and let G = fa i A j : i; j = 1; 2; : : : ; kg be te family of all pair-wise products of matrices in F. If G is commutative, it is known tat F can be simultaneously unitarily upper triangularized if and only if every eigenvalue of every commutator A i A j A j A i is zero. Sow tat assuming commutativity of G is a weaker ypotesis tan assuming commutativity of F. Sow tat te family F in Problem 4