Cauchy-Schwarz Inequaltes ssocated wth Postve Semdente Matrces Roger. Horn and Roy Mathas Department of Mathematcal Scences The Johns Hopkns Unversty, altmore, Maryland 228 October 9, 2 Lnear lgebra and ts pplcatons 42:63-82, 99. bstract Usng a quaslnear representaton for untarly nvarant norms, we prove a basc nequalty: Let = L be postve semdente, where 2 Mm;n. Then M k jj p k 2 kl p k km p k for all p > and all untarly nvarant norms k k. We show how several nequaltes of Cauchy- Schwarz type follow from ths bound and obtan a partal analog of our results for lp norms. Introducton In ths paper we descrbe a technque for provng nequaltes of Cauchy-Schwarz type for matrces, and use t to derve varous nequaltes, some of whch are new. We also use a result for Hadamard products to translate nequaltes for the l norm nto nequaltes for certan other l p norms. We concentrate on untarly nvarant norms and l p norms. For nequaltes of Cauchy-Schwarz type for certan other norms see [9]. We use M m;n to denote the space of m-by-n complex matrces, and dene M n M n;n. The space of n-by-n Hermtan matrces s denoted by H n. For ; 2 H n we wrte f? s postve semdente. For p we dene the l p norm on M n by kk p = @ ;j =p ja j j p for 2 M n : norm kk on M m;n s called untarly nvarant f kuv k = kk for all 2 M m;n and all untary U 2 M m ; V 2 M n. Gven a postve semdente matrx and p >, p denotes the unque postve
semdente p th power of. Gven 2 M m;n dene jj ( ) =2. The ordered sngular values, () 2 () mnfm;ng, of 2 M m;n are the largest mnfm; ng egenvalues of jj. The cone of all nonnegatve real n-vectors wth entres arranged n non-ncreasng order s denoted by R n +#. When statng nequaltes nvolvng matrces of derent szes t s convenent to employ the followng natural conventon: f k k s a norm on M n and 2 M k;l wth k; l n then kk k ^k where ^ = 2 M n : For each nonzero 2 R n +# we dene the untarly nvarant norm k k on M n by kk n (): These are the generalzed spectral norms ntroduced n []. Gven a norm k k on M m;n, we dene ts dual norm, k k D, on M m;n by kk D = maxfjtr j : kk g: It s easy to see that the dual norm of a untarly nvarant norm s also untarly nvarant, and that any untarly nvarant norm s self-adjont. The dualty theorem for norms states that k k = (k k D ) D for any norm k k. The Hadamard product of ; 2 M m;n s [a j b j ] 2 M m;n. Let 2 M m;n and p >. If the entres of are nonnegatve or f p s a postve nteger, then we dene the p th Hadamard power of by (p) [a p j ] 2 M m;n. The Schur Product Theorem ([6], or [8, Theorem 7.5.3]) states that the Hadamard product of two postve semdente matrces s postve semdente. 2 Quaslnear Representaton and a asc Inequalty for Untarly Invarant Norms In ths secton we rst prove a useful quaslnear representaton for untarly nvarant norms, Theorem 2., and then gve our man result, Theorem 2.3. Theorem 2. Let k k be a untarly nvarant norm on M m;n. kk R n +# such that Then there s a compact set kk = maxfkk : 2 kk g for all 2 M m;n : Proof: Consder the compact convex set kk f( (); : : :; n ()) T : kk D and 2 M m;n g: Let ; 2 M m;n be gven wth kk D. From the denton of the dual norm and the dualty theorem, we have jtr j kk. Thus, by the cyclc nvarance of the trace and the untary 2
nvarance and self-adjontness of k k D we have, for any untary matrces U 2 M m, V 2 M n jtr (U 2 V )(V 2 U )j = jtr (V V 2 U U 2 )j kk k(v V 2 U U 2 ) k D = kk kk D kk: Now choose U ; V so that (U 2 V ) and (V 2 U ) are \dagonal" matrces wth ther sngular values on the man dagonal. Thus for any wth kk D we have mnfm;ng () () kk or equvalently maxfkk : 2 kk g kk: (2.) y the dualty theorem agan, there s some 2 M m;n wth kk D = for whch jtr j = kk. Combnng ths wth an nequalty of von Neumann [9, Theorem ] (or [3, Theorem 2..]) we have kk = jtr j n from whch t follows that equalty holds n (2.). In the proof we chose () ( ) = n () (); kk = f( (); : : :; n ()) T : kk D and 2 M m;n g = fx : x 2 R n +# and g D (x) g = fx : g D (x) g \ R n +# where g s the symmetrc gauge functon assocated wth the untarly nvarant norm k k. It would have been sucent to use only the smaller set of extreme ponts of the compact convex set kk. See [4] for a smlar representaton theorem for untary smlarty nvarant norms. s an mmedate consequence of Theorem 2. we have Corollary 2.2 Let f : R k +! R + be non-decreasng n each component, and let ; ; : : :; k 2 M m;n be gven. Then kk f(k k; : : :; k k k) for all untarly nvarant norms k k: (2.2) f and only f kk f(k k ; : : :; k k k ) for all nonzero 2 R n +# (2.3) We are now ready to prove a basc nequalty for untarly nvarant norms. 3
Theorem 2.3 Suppose that the block matrx L = M s postve semdente, where L 2 M m, M 2 M n, and 2 M m;n. Then L and M are postve semdente and (a) For all p > and all k = ; : : :; mnfm; ng we have p () k Y (b) For all p > and every untarly nvarant norm k k we have p=2 (L) p=2 (M): (2.4) k jj p k 2 kl p k km p k: (2.5) Proof: Let be postve semdente and parttoned as ndcated. To prove (2.4) and (2.5) t suces to prove the theorem for square, as the general case follows by augmentng wth zero blocks to make square. Let 2 M n. Then = L =2 CM =2 (2.6) for some contracton C [2, Lemma 2]. y an nequalty of. Horn [7] we have () for each k = ; :::; n, and hence (L =2 ) (C) (M =2 ) p () k Y for any p > and any k = ; : : :; n. Ths proves (2.4). Gven any 2 R n, from (2.7) we have +# p () k Y (L) =2 (M) =2 p=2 (L) p=2 (M) (2.7) p=2 (L) p=2 (M): (2.8) ecause the entres of and the sngular values of a matrx are nonnegatve and are arranged n decreasng order we also have and p () nn p () (2.9) p=2 (L) p=2 (M) n p=2 n (L) p=2 n (M) : (2.) 4
y a result of Weyl ([2] or [3, 5..2.b]), the weak multplcatve majorzaton relaton n (2.8)- (2.) mples the weak addtve majorzaton k k p () p=2 (L) p=2 (M) for k = ; : : :; n: If we now take k = n and apply the Cauchy-Schwarz nequalty we have That s, n p () n = n p=2 (L) p=2 h =2 p=2 n (L) (M) h =2 p=2 p (L)! =2 n kjj p k (kl p k km p k ) =2 (M) p (M)! =2 : for all 2 R n +#, from whch (2.5) follows by Corollary 2.2 wth f(x ; x 2 ) = p x x 2. If we factor as =q =r wth r? + q? = ; r ; q and use Holder's nequalty nstead of the Cauchy-Schwarz nequalty, we get the nequalty kjj p k kl pr=2 k =r km pq=2 k =q (2.) of Holder-type for all untarly nvarant norms k k, p >, and conjugate ndces q; r. We now gve some matrx-valued nequaltes assocated wth parttoned postve semdente matrces. Theorem 2.4 Suppose that the block matrx L = M s postve semdente, where L 2 M m, M 2 M n and 2 M m;n. Then L and M are postve semdente and (a) If M s nonsngular, then f L s nonsngular, then M? L; (2.2) L? M: (2.3) (b) There s some C 2 M m;n such that (C) and = L =2 CM =2. 5
(c) If m n then there s some U 2 M m;n such that UU = I 2 M m and M =2 U LUM =2 ; (2.4) f m n then there s some V 2 M n;m such that V V = I 2 M n and L =2 V MV L =2 : (2.5) Proof: Let be postve semdente and parttoned as ndcated. The nequaltes (2.2) and (2.3) are well-known [8, Theorem 7.7.6], and ether mples (b) as shown n [2, Lemma 2]. To prove (2.4) and (2.5), rst assume that s square and that M and L are non-sngular; the sngular case follows by a contnuty argument, and we wll show how the non-square case follows from the square case. Use (2.2) to wrte L M? = (M?=2 )(M?=2 ) = (M?=2 )(M?=2 ) : ecause Y Y and Y Y are untarly smlar for any square matrx Y, there s a untary matrx U such that Thus, n the square case we have (M?=2 )(M?=2 ) = U(M?=2 )(M?=2 )U = (UM?=2 )( )(M?=2 U ): M =2 U LUM =2 : smlar argument proves (2.5) n the square case. Now consder the non-square case. ssume that m n and dene the n-by-n matrces ^ and ^L L : ecause s postve semdente, the matrx ^L ^ s also postve semdente. y the square case of (2.4) there s a untary ^U 2 Mn such that Partton ^U as U ^U = W ^ M ^ ^ M =2 ^U ^L ^UM =2 : (2.6) wth U 2 M m;n ; W 2 M n?m;n : Parttoned multplcaton shows that = ^ ^ and ^U ^L ^U = U LU. Thus, (2.6) s M =2 U LUM =2 : 6
Fnally, snce ^U ^U = I 2 M n, t follows that UU = I 2 M m. The proof of (2.5) s smlar when m n: one forms ^ = ( ); ^M = M ; and ^V = V Y wth V 2 M n;m and Y 2 M m?n;m and proceeds from the square case of (2.5). See [5, 2] for other nequaltes, of a derent character, also nvolvng parttoned postve semdente matrces. Corollary 2.2 may be thought of as a generalzaton of a well-known theorem of Ky Fan [8, Corollary 7.4.47], to whch t s equvalent when k = and f(t) = t. Usng summaton by parts we have kk = q () = q (? + )N (); q = mnfm; ng where q+ and N k () () + + k () s the Ky Fan k-norm. Ths dentty shows that kk kk for all nonzero 2 R n +# f and only f N () N () for all = ; : : :; n, so ether of these crtera s necessary and sucent to have kk kk for every untarly nvarant norm k k. 3 pplcatons We now gve some examples of applcatons of Theorems 2.3 and 2.4. Example 3. Let 2 M m;l, 2 M m;n. Then = ( ) ( ) ; and hence k j j p k 2 k( ) p k k( ) p k = k jj 2p k kjj 2p k (3.) for all untarly nvarant norms k k. Takng p = 2 gves whle the choce p = gves k j j =2 k 2 kk kk for all untarly nvarant norms k k; (3.2) k k 2 k k k k for all untarly nvarant norms k k: (3.3) The nequalty (3.2) s Proposton 5 n [3], whch was proved by a smlar technque. Inequalty (3.3) s the rst asserton n [9, Theorem 3.], for whch we have gven another proof. The case of equalty n (3.3) was determned n [9, Theorem 3.9]. One may wonder how, or whether, the cases of equalty n (3.2) and (3.3) are related. They are unrelated, as shown n the followng theorem. 7
Theorem 3.2 There s a untarly nvarant norm k k on M 3 wth the followng property: Gven any postve real numbers p and p 2 wth p 6= p 2 there are ; 2 M 3 such that but Proof: k j j p k 2 = k( ) p k k( ) p k (3.4) k j j p 2 k 2 < k( ) p 2 k k( ) p 2 k: (3.5) Let [; =4; ] T and [7=6; =6; =6] T. Dene k k on M 3 by kk maxfkk ; kk g = maxf 3 (); 3 ()g: It s clear that k k s a untarly nvarant norm on M 3. Suppose p 2 >p >. Choose s 2 (; ) such that s p 2 <=2 but s p >=2 and check that and p + 2 s p + 3 s p > p + 2 s p + 3 s p (3.6) p 2 + 2 s p 2 + 3 s p 2 < p 2 + 2 s p 2 + 3 s p 2 : (3.7) Let t = p s and dene @ Then, usng (3.6), we have (3.4): we have C t ; t @ C t : k j j p k 2 = k j j p k 2 = k( ) p k k( ) p k = k( ) p k k( ) p k: Whle, by (3.7), we have and k j j p 2 k 2 = ( + (=4)s p 2 ) 2 < (7=6 + (=6)s p 2 + (=6)s p 2 ) ( + (=4)s p 2 ) = k( ) p 2 k k( ) p 2 k k( ) p 2 k k( ) p 2 k; k j j p 2 k 2 = (7=6 + (=6)s p 2 ) 2 < (7=6 + (=6)s p 2 + (=6)s p 2 ) (7=6 + (=6)s p 2 ) = k( ) p 2 k k( ) p 2 k k( ) p 2 k k( ) p 2 k: The strct nequalty (3.5) follows from these two strct nequaltes. smlar constructon gves a counterexample when p > p 2 >. 8
Example 3.3 Snce the sum of postve semdente matrces s postve semdente, for any 2 M m ;l; 2 M m ;n It now follows from Theorem 2.3 that p (j j) P P ( P P ) p=2 ( ) p=2 ( : ); p > ; k = ; 2; : : : : (3.8) Ths s a generalzaton of Lemma 4 n [8], where the proof gven was consderably more complcated. Example 3.4 If ; Y 2 M n are Hermtan and Y, then [6, Lemma 2.6] Y Y and hence f we take p = n (2.5) we obtan Ths nequalty has been noted n [4]. kk k for all untarly nvarant norms k k: (3.9) We now lst some examples that use the Schur Product Theorem. Notce that for any matrx 2 M m;n the matrx I I s postve semdente. = I Example 3.5 Snce the Hadamard product of postve semdente matrces s postve semdente, for any ; 2 M m;n we have I ( ) I I and hence for all untarly nvarant norms k k = ( ) ( ) I k k 2 k( ) Ik k( ) Ik k k k k = k k k k: (3.) The second nequalty holds because the sngular values (whch are also the egenvalues) of a postve semdente matrx majorze ts man dagonal entres. The nequalty (3.) has been observed n [5] and [9], and s proved by an alternate method n [9, Theorem 3.]. It has been generalzed to a famly of products that ncludes the Hadamard product and the usual product n []. 9
Example 3.6 Let ; 2 M m;n. Then I I ( ) ( ) = ( ) I ; and hence (2.2) gves j j 2 = ( ) ( ) ( ) ( ): (3.) In [2] Zhang proves (3.) by computng ( ) ( )? ( ) ( ) n terms of the entres of and, then showng that the resultng expresson s a sum of rank one postve semdente matrces. If and are square and postve semdente, then (3.) reduces to () 2 2 2, whch s a specal case of ( ) p p p for p 2 [; 2] (3.2) whch s true for all postve semdente matrces ; [, Theorem ()]. However, t seems that (3.2) cannot be derved from Theorem 2.3 easly. Example 3.7 If s normal then a drect calculaton usng the spectral theorem shows that jj : jj Thus, f ; 2 M n are normal then jj jj ( ) ( ) jj jj and hence k k k jj jj k (3.3) for all untarly nvarant norms k k, provded and are normal. The hypothess of normalty s essental. Consder the followng counterexample found usng MTL: = = @ @ 3?6?9?6 4 46?9 46 56?4?2 5 7 9? 4 8? The matrx s not only normal but s actually postve semdente. However, s not normal. For ths choce of and we have k ( ) > k C C : ( jj) for k = ; 2; 3;
and hence, by a smple generalzaton of Ky Fan's theorem [8, Corollary 7.4.47], we have kk > k jjjj k for all untarly nvarant norms k k on M 3. Note that even f both and are Hermtan the matrx-valued nequalty j j jj jj s not true because the weaker statement ( ) (jj jj) = ; : : :; n; s not true. See [7, Theorem 3.2] for a 2-by-2 counterexample. 4 Hadamard Products and l p Norms In ths secton we use a result about Hadamard products to prove a Cauchy-Schwarz nequalty for l p norms. From the Schur Product Theorem t follows that for any postve nteger k and any postve semdente matrces we have (k) and (k) (k) : less well-known result of FtzGerald and Horn [6, Theorems 2.2 and 2.4] extends ths to non-ntegral powers. Theorem 4. Let n be an nteger greater than, and let ; 2 M n matrces wth real nonnegatve entres. Then be postve semdente. (p) for any real number p n? 2. 2. If then (p) (p) for any real number p n?. Furthermore, the lower bounds n these two assertons are sharp: 3. If < p < n? 2 and p s not an nteger, then there s a postve semdente matrx C 2 M n wth nonnegatve entres such that C (p) s not postve semdente. 4. If < p < n? and p s not an nteger, then there are postve semdente matrces D; E 2 M n wth nonnegatve entres such that D E but D (p) 6 E (p). It s not known whether Theorem 4. holds for a sutable denton of (p) for matrces wth real (though possbly negatve), or more generally, complex entres and p nonntegral. However, t does follow from Theorem 4. that for any p n? 2 and q n? and any postve semdente matrces, possbly wth complex entres, we have ( ) (p) and ( ) (q) ( ) (q) : (4.) Usng these deas we can prove a Cauchy-Schwarz nequalty for l p norms.
Theorem 4.2 Let L = be an n-by-n real or complex matrx ( need not be square), and let p be ether a postve nteger or a real number wth p n? 2. Then If, n addton, has nonnegatve real entres then M ; kk 2 2p klk 2p kmk 2p : (4.2) kk 2 p klk p kmk p : (4.3) Proof: Let be postve semdente and parttoned as above. We wll prove (4.3) when has nonnegatve real entres; the nequalty (4.2) follows when ths s appled to the postve semdente matrx, whch always has nonnegatve entres. Let L = [l j ] 2 M k, M = [m j ] 2 M n?k, and = [x j ] 2 M k;n?k. For t > let y = [t; :::; t;?=t; :::;?=t] T, (k +'s and n? k?'s). Snce s postve dente, we have y T y = t 2 ;j l j + (=t 2 ) ;j m j? 2 ;j x j : ecause s entrywse nonnegatve ths s equvalent to Now set t 2 = p kmk =klk and square to gve kk 2 ft2 klk + (=t 2 )kmk g: kk 2 klk kmk : If p s a postve nteger or p n? 2, Theorem 4. ensures that (p), and hence Fnally, take p th roots to obtan the result. kk 2p p = k (p) k 2 kl (p) k km (p) k = klk p pkmk p p: If and p satsfy the hypotheses of the theorem, and f has nonnegatve entres and all the entres of M and L have nonnegatve real part, then one can apply the nequalty (4.3) to + and show that (4.3) stll holds. If n 3 and = [a j ] n ;j= then all the prncpal mnors of [ja jj] n ;j= are nonnegatve and hence [ja j j] n ;j= (see [2, pg. 238] for the explct calculaton). Thus, f 2 M 3 s postve semdente and s parttoned as L = ; M then kk 2 p klk p kmk p for all p : 2
However, f n > 3, then t s not true that [a j ] n ;j= mples [ja jj] n ;j=. The followng 4-by-4 matrx s a counterexample [8, Problem 7.5.6]: @ 3?2 3 9?2 4 9 4 The next result s a partal converse of Theorem 4.2. Theorem 4.3 For each postve nteger n and non-ntegral p 2 (; n? 2) there s a postve semdente matrx 2 M n wth nonnegatve entres that can be parttoned as L = M such that kk 2 p > klk p kmk p : Proof: Let p 2 (; n?2) be non-ntegral. y Theorem 4. there s a postve semdente 2 M n wth nonnegatve entres such that (p) s not postve semdente. Let x 2 R n be a non-zero vector such that x T (p) x <. Wthout loss of generalty one may assume that all the entres of x are non-zero, and that the rst k entres are postve and the remanng n? k are negatve. Partton as L = M where L 2 M k. pplyng a dagonal congruence to f necessary we may assume that jx j =. Now use the same algebrac manpulatons used n the proof of Theorem 4.2 to gve C > x T (p) x = klk p p + kmk p p? 2kk p p: Rearrange ths, and apply the arthmetc-geometrc mean nequalty to get 2kk p p > klk p p + kmk p p 2 q klk p pkmk p p: Fnally, square and take p th roots to obtan the desred nequalty. We now apply Theorem 4.2 to gve a partal answer to a queston rased n [9]. Example 4.4 Let 2 M l;n ; 2 M l;m. We have and hence ; k k 2 p k k p k k p (4.4) for p = 2; 4; : : : or p 2(m + n? 2). If, n addton,, and all have only nonnegatve real entres, then (4.4) holds for all postve ntegers p and any real number p m + n? 2. 3
References [] T. ndo. Concavty of certan maps on postve dente matrces and applcatons to Hadamard products. Ln. lg. ppl., 26:23{4, 979. [2] T. ndo, R.. Horn, and C. R. Johnson. The sngular values of a Hadamard product: basc nequalty. Ln. Multln. lg., 2:345{65, 987. [3] R. hata. Perturbaton nequaltes for the absolute value map n norm deals of operators. J. Operator Theory, 9:29{36, 988. [4] R. hata and F. Kttaneh. On the sngular values of a product of operators. To appear n SIM J. Matrx nal. ppl. [5] J. de Plls. Transformatons on parttoned matrces. Duke Math. J., 36(3):5{5, 969. [6] C. H. FtzGerald and R.. Horn. On fractonal Hadamard powers of postve dente matrces. J. Math. nal. ppl., 6(3):633{42, 977. [7]. Horn. On the sngular values of a product of completely contnuous operators. Proc. Nat. cad. Sc., 36:374{5, 95. [8] R.. Horn and C. R. Johnson. Matrx nalyss. Cambrdge Unversty Press, New York, 985. [9] R.. Horn and R. Mathas. n analog of the Cauchy-Schwarz nequalty for Hadamard products and untarly nvarant norms. to appear n SIM J. Matrx nal. ppl. [] R.. Horn, R. Mathas, and Y. Nakamura. Inequaltes for untarly nvarant norms and blnear matrx products. Techncal Report 56, Department of Mathematcal Scences, The Johns Hopkns Unversty, May, 989. [] C.-K. L, T.-Y. Tam, and N.-K. Tsng. The generalzed spectral radus, numercal radus and spectral norm. Ln. lg. ppl., 6:25{37, 984. [2] M. Marcus and W. Watkns. Parttoned Hermtan matrces. Duke Math. J., 38(2):237{49, 97. [3]. W. Marshall and I. Olkn. Inequaltes: Theory of Majorzaton and ts pplcatons. cademc Press, London, 979. [4] R. Mathas. representaton of untary smlarty nvarant norms. Techncal Report 55, Department of Mathematcal Scences, The Johns Hopkns Unversty, altmore, 989. [5] K. Okubo. Holder-type nequaltes for Schur products of matrces. Ln. lg. ppl., 9:3{28, 987. 4
[6] J. Schur. ermerkungen zur Theore der beschrankten lnearformen mt unendlch velen Veranderlchen. J. fur Rene und ngewandte Mathematk, 4:{28, 9. [7] G. H. P. Styan. Hadamard products and multvarate statstcal analyss. Ln. lg. ppl., 6:27{4, 973. [8] R. C. Thompson. Matrx type metrc nequaltes. Ln. Multln. lg., 5:33{9, 978. [9] J. von Neumann. Some matrx nequaltes and metrzaton of matrc space. Tomsk. Unv. Rev., :286{3, 937. lso n John von Neumann Collected Works (. H. Taub, ed.) Vol. IV, pp. 25-8, Pergamon Press, Oxford, 962. [2] H. Weyl. Inequaltes between the two knds of egenvalues of a lnear transformaton. Proc. Natonal cad. Sc., 35:48{, 949. [2] F. Zhang. Notes on Hadamard products. To appear n Ln. Multln. lg. 5