Matrix Theory, Math6304 Lecture Notes from October 25, 2012 take by Maisha Bhardwaj Last Time (10/23/12) Example for low-rak perturbatio, re-examied Relatig eigevalues of matrices ad pricipal submatrices Covetio : We follow our usual order of arragig eigevalues of a matrix A i a o-decreasig fashio λ 1 (A) λ 2 (A) λ (A). Warm-up : Test for positive defiiteess First we recall what a positive defiite matrix is. 4.5.1 Defiitio. AmatrixA M with A = A is positive defiite (or positive semi-defiite) if all eigevalues of A are strictly positive (or o-egative). Last time, we saw a iterlacig relatio betwee eigevalues of a matrix  M +1 ad those of its pricipal submatrix A M.Wealsokowthatthedetermiatadeigevaluesofamatrix A are related i the followig maer: det(a) = λ j (A), where λ j (A), 1 j are the eigevalues of A. Weowseekifthereisarelatiobetweea positive defiite matrix ad the determiats of its pricipal submatrices. To this ed, we have the followig theorem: 4.5.2 Theorem. AmatrixA M with A = A is positive defiite if ad oly if all leadig pricipal submatrices {A r } Ar r=1 of A with A = satisfy det(a r ) > 0. I some sese, this theorem is useful because we are ot required to compute the eigevalues of matrix A i order to check for its positive defiiteess. Istead we ca check the determiats of all leadig pricipal submatrices, however, efficiet computatio of determiats is aother issue. 1
Proof. ( ) LetusassumethatthematrixA is positive defiite. Thus all the eigevalues of A are strictly positive, i particular, the smallest eigevalue satisfy λ 1 (A )=>0. We recall our techique of borderig, ifa M,  M +1 be Hermitia matrices with A y  = y ad a R. Let {λ a j }, {ˆλ j } +1 be the eigevalues of A ad  arraged i o-decreasig order, the ˆλ 1 λ 1 ˆλ 2 λ 2 λ ˆλ +1. Now, we observe that A is a borderig of A 1,heceλ 1 (A 1 ) λ 1 (A ).Similarly,A 1 beig a borderig of A 2 gives λ 1 (A 2 ) λ 1 (A 1 ) λ 1 (A ). Proceedig i this maer, we obtai that λ 1 (A r ) λ 1 (A )=>0 for each 1 r. Thisimpliesthatalleigevalues of matrix A r are strictly positive. Thus, for each leadig pricipal submatrix A r, 1 r det(a r )= r λ j (A r ) > 0 ( ) Coversely,weassumethatdet(A r ) > 0 for each leadig pricipal submatrix A r of A for each r {1, 2,,}. WewattoshowthatthematrixA is positive defiite, that is, all its eigevalues are strictly positive. The proof proceeds by usig iductio over r. For r =1,thereisothigtoshowasA 1 =[ ], det(a 1 )= > 0 by assumptio. We ow assume that for all A r,r k, A r is positive defiite. For the matrix A k+1,wehave det(a k+1 )= k+1 λ j (A k+1 )=λ 1 (A k+1 ) k+1 j=2 λ j (A k+1 ) (1) As A k+1 is a borderig of A k,applyigtheborderigresultadusigthefactthateigevalues of A k are strictly positive, we obtai for each j {2,,k+1}, λ j (A k+1 ) λ j 1 (A k ) > 0. (2) Thus, from Eqs. (1), (2) ad usig det(a k+1 ) > 0, wecocludethat λ 1 (A k+1 )= k+1 j=2 det(a k+1) λ j (A k+1 ) > 0. This gives us that all eigevalues of A k+1 are strictly positive. Hece, by iductio, we have completed the proof. 4.6 Geeralized Iterlacig ad Rayleigh-Ritz Theorem By ow, we kow the relatio betwee eigevalues of a matrix  M +1 ad those of its pricipal submatrix A M. I the followig, we will ivestigate the iterlacig relatioship 2
betwee eigevalues of A ad those of its pricipal submatrices of all smaller sizes. These ca be thought of as a aalogue of rak-r perturbatio results we have see before. 4.6.3 Theorem. Let A M with A = A ad for each 1 r, A r M r be a pricipal submatrix of A, theforeachk, k r, λ k (A) λ k (A r ) λ k+ r (A). Proof. We cosider a sequece of pricipal submatrices, as A r beig a pricipal submatrix of A r+1, A r+1 beig a pricipal submatrix of A r+2,. A 1 beig a pricipal submatrix of A. The, by precedig corollary o borderig, we have ad λ k (A) λ k (A 1 ) λ k (A r ) λ k (A r ) λ k+1 (A r+1 ) λ k+ r (A r+ r )=λ k+ r (A). Combiig the above two iequalities gives the required result. So far, we have see two approaches of iterlacig - oe usig rak perturbatios ad aother by relatig eigevalues of a matrix with those of its pricipal submatrices. These two approaches seem to be depedet o the choice of basis vectors, however, we will establish that this is ot the case via the followig result. It also shows a iterlacig relatio betwee the eigevalues of amatrixa with those of a matrix B which is merely uitarily equivalet to a pricipal submatrix of A. 4.6.4 Corollary. Let A M with A = A ad {u 1,u 2,,u r } C be a orthoormal system with r. Let B r = Au j,u i r i, M r (3) ad the eigevalues of A ad B r be arraged i a o-decreasig order, the for each 1 k r, λ k (A) λ k (B r ) λ k+ r (A) Proof. We first exted {u 1,u 2,,u r } to a orthoormal basis {u 1,u 2,,u } of C by appedig {u r+1,u r+2,,u }.LetU = u 1 u 2 u,theuisauitarymatrix. We defie A = U AU= Au j,u i i, As the matrix A is uitarily equivalet to matrix A, theybothhavesameeigevalues. Wealso observe that B r = Au j,u i r is a pricipal submatrix of i, A ad hece the result follows from the previous theorem. 3
From Rayleigh-Ritz theorem, we have characterized the miimum ad maximum eigevalues of ahermitiamatrixa i terms of a quadratic form as follows : λ mi =miax, x x=1 λ max =maxax, x x=1 For x C with x =1,wehaveP x = xx as a rak-oe projectio oto the spa of {x}. I this view, we write Ax, x = x Ax = tr[x Ax] =tr[xx A]=tr[P x A] (sice tr[ab] =tr[ba]). Therefore, the above optimizatio formulatio ca be writte as λ mi =mi x=1 tr[p xa] λ max =max x=1 tr[p xa] Now, we ask if there is some way to characterize the sum of r smallest eigevalues or that of r- largest eigevalues of Hermitia A i terms of tr[pa], wherep is a rak-r orthogoal projectio oto a subspace of dim r. The followig corollary allows us to formulate a geeralized versio of Rayleigh - Ritz. 4.6.5 Corollary. For a subspace C with dim( )=r, letp Sr deote the orthogoal projectio of C oto.leta M with A = A ad eigevalues {λ j } beig arraged i ao-decreasigorder,the λ 1 (A)+λ 2 (A)+ + λ r (A) = mi tr[p Sr A] ad λ r+1 (A)+λ r+2 (A)+ + λ (A) =max tr[p Sr A] Proof. The orthogoal projectio for each is give by choosig ay orthoormal spaig set of size r, {u 1,u 2,,u r } ad lettig (from Parseval s idetity) r P Sr x = x, u j u j. Thus, we have P Sr = UU M with U = u 1 u 2 u r M,r. Now, we exted the set of vectors {u j } r to a orthoormal basis {u j } for C. We defie Ũ = u 1 u 2 u M,whichisauitarymatrix. Sice uitarily equivalece preserves the trace, we have tr[p Sr A]=tr[Ũ P Sr A Ũ] = P Sr Au j,u j = = A u j,ps r u j A u j,p Sr u j (P Sr beig a projectio implies P Sr = PS r ) 4
By defiitio of P Sr,wehave,P Sr u j =0for j>rad P Sr u j = u j, 1 j r. This implies that r tr[p Sr A]= A u j,u j = tr[b r ], where B r = Au j,u i r i, M r. Usig the result from corollary 4.6.4, we have λ k (A) λ k (B r ) for each 1 k r. Takigsum over all k, weobtai λ 1 (A)+λ 2 (A)+ + λ r (A) λ 1 (B r )+λ 2 (B r )+ + λ r (B r ) = tr[b r ] So, we have show that λ 1 (A)+λ 2 (A)+ + λ r (A) mi tr[p Sr A] = mi tr[b r ] (4) We ow establish that the mi/if over all achieves equality here. To this ed, we eed to get asubspace with dim( )=r such that λ 1 (A)+λ 2 (A)+ + λ r (A) =tr[p Sr A]. Wechoose {u 1,u 2,,u r } to be eigevectors correspodig to eigevalues {λ 1 (A),λ 2 (A),,λ r (A)} with u i u j for all i = j. Thethematrix B r correspodig to the subspace is give by λ 1 (A) 0 λ 2 (A) B r =...,adhecetheequalityieq.(4)isachieved. 0 λ r (A) Similarly, the proof for λ r+1 (A) +λ r+2 (A) + + λ (A) =max precedig part by observig that tr[p Sr A] follows from λ +1 j (A) = λ j ( A) ad max tr[p Sr A]= mi tr[p Sr ( A)]. From the above geeralized versio of Rayleigh-Ritz theorem, we also get a ituitio that the sum/itegrated eigevalues of a Hermitia matrix may be more compatible with the structure of M i cotrast to idividual eigevalues themselves. 4.7 Majorizatio 4.7.6 Questio. Ca we extract iformatio about eigevalues without solvig optimizatio problems such as i Rayleigh- Ritz or Courat- Fischer theorem? Aswer. Yes, we ca extract some useful iformatio about sums of eigevalues of a Hermitia matrix. We will develop some results i this directio i ext few classes. I this regard, we eed some termiologies ad the first oe is the cocept of mazorizatio. 4.7.7 Defiitio. Let {α j }, {β j } be real sequeces, the β majorizes α, deotedby β α, if k k β mi α ti, 1 k, i=1 i=1 5
where for a give k, m i ad t i are permutatios of {1, 2,,} such that α t1 α t2 α t ad β m1 β m2 β m. 6