An Analog of the Cauchy-Schwarz Inequality for Hadamard. Products and Unitarily Invariant Norms. Roger A. Horn and Roy Mathias

Transcription

1 An Analog of the Cauchy-Schwarz Inequality for Hadamard Products and Unitarily Invariant Norms Roger A. Horn and Roy Mathias The Johns Hopkins University, Baltimore, Maryland SIAM J. Matrix Analysis and Applications, Vol. 11, No. 4, , Abstract We show that for any unitarily invariant norm k k on Mn (the space of n-by-n complex matrices) and ka Bk 2 ka Ak kb Bk for all A; B 2 Mm;n (1) ka Bk 2 ka Ak kb Bk for all A; B 2 Mn where denotes the Hadamard (entrywise) product. inequality for absolute norms on C n These results are a consequence of an kx yk 2 kx xk ky yk for all x; y 2 C n : (2) We also characterize the norms on C n that satisfy (2), characterize the unitary similarity invariant norms on Mn that satisfy (1), and obtain related results on norms on C n and unitary similarity invariant norms on Mn that are of independent interest. 1 Introduction and Notation Let M m;n denote the space of m-by-n complex matrices and write M n M n;n ; let A A denote the conjugate transpose of a matrix in M m;n. Recently, Harald Wimmer [20, p. 315] conjectured that an analog of the Cauchy-Schwarz inequality holds for any unitarily invariant norm k k on M m;n : ka Bk 2 ka Ak kb Bk for all A; B 2 M m;n (1.1) For three special choices of norm k k (the trace norm, the Frobenius norm, and the spectral norm), Wimmer proved (1.1) and identied the cases of equality. In Section 3 we give a proof of (1.1) and a similar inequality for Hadamard products; both results follow from a simple norm inequality (Theorem 2.3) for the Hadamard product of vectors. We identify the cases of equality in this inequality and in (1.1). In Sections 3 and 4 we prove some 1

2 results of independent interest on unitary similarity invariant norms. In Section 4 we provide a variety of examples and show that the set of norms satisfying (1.1) is a convex set that strictly contains the unitarily invariant norms. We use A 0 to mean that A is positive semidenite. If A 0 then A denotes the unique positive semidenite square root of A. Given A 2 M m;n we dene jaj (A A). The real vector space of n-by-n Hermitian matrices is denoted by H n. If A; B 2 H n, we write A B if A? B 0. Recall that the Hadamard product of two matrices A = [a ij ] and B = [b ij ] of the same size is AB [a ij b ij ]. We denote the ordered singular values of any A 2 M m;n by 1 (A) : : : q (A) 0 (where q = minfm; ng) and dene (A) [ 1 (A); : : :; q (A)] T 2 R q +; for A 2 H n we denote the ordered eigenvalues of A by 1 (A) : : : n (A) and dene (A) [ 1 (A); : : :; n (A)] T 2 R n. The eigenvalues and singular values of a positive semidenite matrix are identical. A complex matrix is a partial isometry if each of its singular values is 0 or 1. The trace of a square matrix A (the sum of its main diagonal entries, or, equivalently, the sum of its eigenvalues) is denoted by tr A. Given x 2 C n and an index set I f1; : : :; ng we dene x(i) 2 C n by ( xi if i 2 I x(i) i = 0 if i 62 I and we dene jxj [jx i j] n. Given a vector x 2 Cn we dene diag(x) 2 M n to be the diagonal matrix with i; i entry x i. Given vectors x; y 2 R n we use x y to mean that x i y i for i = 1; : : :; n. A norm k k on C n is absolute if kxk = k jxj k for all x 2 C n, and is monotone if jxj jyj implies kxk kyk. These two notions were introduced by Bauer, Stoer and Witzgall in [2], where they arose naturally in the study of induced norms on M n. It is a fact that a norm is absolute if and only if it is monotone [2, Theorem 2] or [7, Theorem ]. If vectors x; y 2 R n are given, and if and are permutations of f1; 2; : : :; ng such that x (1) x (2) : : : x (n) and y (1) y (2) : : : y (n), we say that x is weakly majorized by y if kx x (i) kx y (i) for all k = 1; : : :; n: If, in addition, equality holds when k = n, then we say that x is majorized by y. A function g() : C n! R + is called a symmetric gauge function if it is a permutation invariant absolute norm on C n. We will make frequent use of the fact that for x; y 2 C n and any symmetric gauge function g() jxj is weakly majorized by jyj implies g(x) g(y): (1.2) Given a norm k k on C m we dene its dual (with respect to the Euclidean inner product) by kxk D maxfjy xj : y 2 C m ; kyk 1g: (1.3) Given a norm k k on M m;n < A; B > tr B A) by we dene its dual (with respect to the Frobenius inner product kak D maxfjtr (B A)j : B 2 M m;n ; kbk 1g: 2

3 If we take n = 1, then this denition specializes to (1.3). The duality theorem for norms [7, Theorem ] states that k k = (k k D ) D for any norm k k. A norm k k on M m;n is unitarily invariant if kak = kuav k for all A 2 M m;n and all unitary U 2 M m and V 2 M n. A theorem of von Neumann [19] (or [7, Theorem ], or [16, Theorem V.5]) states that a norm k k on M m;n is unitarily invariant if and only if there is a symmetric gauge function g such that kxk = g((x)) for all X 2 M m;n. A norm k k on M n is unitary similarity invariant if kak = kuau k for all A; U 2 M n with U unitary. See [7] for further information on Hadamard products, norms, dual norms, unitarily invariant norms, symmetric gauge functions, singular values, and other concepts discussed in this paper. See [13, p. 263] for a general discussion of the connection between majorization and unitarily invariant norms. 2 An Inequality for Absolute Norms In this section we are interested in an inequality for Hadamard products of vectors that leads directly to a proof of the matrix inequality (1.1). To obtain Theorem 2.3, the main result in this section, it is helpful to know two lemmata, whose proofs we omit. The rst result is Theorem 1 in [2]; the second can be proved by an argument very similar to the proof of Lemma 3.7. Lemma 2.1 A norm on C n is absolute if and only if its dual norm is absolute. Lemma 2.2 Let k k be an absolute norm on C n and let x 2 C n be given. Then kxk = maxfjy xj : y 2 C n and kyk D 1g = maxfz T jxj : z 2 R n + and kzk D 1g: We use the following notation in Theorem 2.3. Given x 2 C n and an index set I f1; : : :; ng, dene x(i) 2 C n by ( xi if i 2 I x(i) i = 0 if i 62 I : Theorem 2.3 Let k k be an absolute norm on C n. Then kx yk 2 kx xk ky yk for all x; y 2 C n : (2.1) If x; y 6= 0 then equality holds in (2.1) if and only if there is a positive constant c and an index set I f1; : : :; ng such that jx(i)j = cjy(i)j; kx x(i)k = kxk; and ky y(i)k = kyk: Proof: Use Lemma 2.2 to compute kx yk 2 = [maxfz T (jxj jyj) : z 2 R n + and kzk D 1g] 2 (2.2) 3

4 = maxf[(z jxj) T (z jyj)] 2 : z 2 R n + and kzk D 1g maxf[(z jxj) T (z jxj)][(z jyj) T (z jyj)] : z 2 R n + and kzkd 1g (2.3) maxf(z T jx xj)] : z 2 R n + and kzk D 1g maxf(z T jy yj)] : z 2 R n + and kzkd 1g (2.4) = k jx xj k k jy yj k = kx xk ky yk: For z = [z i ] 2 R n +, we have written z [z i ] 2 R n + for the Hadamard (entrywise) nonnegative square root of z. That the the stated conditions are sucient for equality is clear from the monotonicity of k k: kx yk 2 kx y(i)k 2 = kx (1=c)x(I)k kcy y(i)k = kx x(i)k ky y(i)k = kx xk ky yk and hence equality holds in (2.1). Conversely, suppose that equality holds in (2.1) for given non-zero vectors x; y. Then any z 2 R n + that attains the maximum in (2.2) must also attain the maximum in (2.3) and both maxima in (2.4). Let ~z be such a vector and dene the index set I fi : ~z i > 0g. Equality in (2.3) implies that there is a positive scalar c such that ~z i jxj i = c~z i jyj i for i = 1; : : :; n and hence, by the denition of I, it follows that jx(i)j = cjy(i)j. Because ~z attains the rst maximum in (2.4), we have kx xk = ~z T jx xj, and we can use Lemma 2.2 again to compute kx x(i)k = maxfz T jx x(i)j : z 2 R n + and kzk D 1g ~z T (x x) = kx xk: But kx xk kx x(i)k by the monotonicity of k k, so kx xk = kx x(i)k. The same argument shows that ky yk = ky y(i)k. Notice that the inequality (2.1) with the l 1 norm is the heart of the classical Cauchy-Schwarz inequality: j nx x i y i j 2 ( nx jx i y i j) 2 = kx yk 2 1 kx xk 1 ky yk 1 = nx jx i j 2 n X It is of interest to characterize the norms on C n that satisfy the conclusion of Theorem 2.3. We discuss the converse of the following preliminary lemma in Theorem jy i j 2 :

5 Lemma 2.4 Let k k be a norm on C n such that kxk k jxj k for all x = [x i ] 2 C n, where jxj [jx i j]. Then the function (x) k jxj k is a norm on C n. Proof: Since the function (x) k jxj k is positive denite and homogeneous on C n, we need only show that it obeys the triangle inequality. We claim that it suces to prove that kuk kvk whenever u; v 2 R n + and u v: (2.5) Since jx + yj jxj + jyj for all x; y 2 C n, (2.5) and the triangle inequality for k k give the desired result: (x + y) = k jx + yj k k jxj + jyj k k jxj k + k jyj k = (x) + (y): To prove (2.5), let u; v 2 R n + be given with u v. If u = v, there is nothing to prove, so assume that u 6= v. Some corresponding entries of u and v may be equal but at least one entry of u must be strictly less than the corresponding entry of v. We shall construct a vector w 2 R n + such that u w v, kwk kvk, and w has one more entry than v that is equal to the corresponding entry of u. A nite induction then leads to the conclusion that kuk kvk. Dene k = minfi : u i < v i ; i = 1; : : :; ng and dene v 0 ; v 00 2 R n by ( ( v 0 vi for i 6= k i =?v i for i = k ; vi for i 6= k v00 i = 0 for i = k : Notice that v 00 = 1 2 (v0 + v) and jv 0 j = v. Using the hypothesis on k k, we have kv 0 k k jv 0 j k kvk; and hence kv 00 k = 1 kv + 2 v0 k 1(kvk + 2 kv0 k) 1 (kvk + kvk) = kvk: (2.6) 2 Now dene u k =v k, so 0 < 1. Dene w v + (1? )v 00 and notice that w i = v i if i 6= k and that w k = u k. Thus, w has one more entry than v that is equal to the corresponding entry of u. Using (2.6) we obtain as desired. kwk = kv + (1? )v 00 k kvk + (1? )kv 00 k kvk + (1? )kvk = kvk; We can now characterize the norms that satisfy the inequality (2.1). Theorem 2.5 Let k k be a norm on C n. Then kx yk 2 kx xk ky yk for all x; y 2 C n (2.7) if and only if kzk k jzj k for all z = [z i ] 2 C n ; (2.8) where jzj [jz i j]. 5

6 Proof: Suppose kzk k jzj k for all z 2 C n. Lemma 2.4 guarantees that (x) k jxj k is an absolute norm on C n, so we may apply Theorem 2.3 to () and obtain kx yk 2 k jx yj k 2 = 2 (x y) (x x)(y y) = kx xk ky yk: Conversely, suppose (2.7) holds and let z 2 C n be given. Dene x; y 2 C n by ( zi =jz i j if z i 6= 0 x i 0 if z i = 0 ; y i jz i j i = 1; : : :; n: Then which is (2.7). kzk 2 = kx yk 2 kx xk ky yk = k jzj k k jzj k = k jzj k 2 ; An example of a norm on C 2 that is not absolute but nevertheless satises the condition (2.8), and hence (2.7) as well, is kxk maxfjx 1 + x 2 j; jx 1 j; jx 2 jg: Although we have characterized the norms for which the inequality (2.7) holds in terms of the natural condition (2.8), it is not always easy to determine whether a particular norm has this property. For example, it is not known which unitarily invariant norms on M m;n satisfy (2.8). 3 Inequalities for Matrices We are now ready to prove (1.1), as well as an analogous inequality for the Hadamard product of matrices, and to discuss the cases of equality. Theorem 3.1 Let k k be a unitarily invariant norm on M n. Then ka Bk 2 ka Ak kb Bk for all A; B 2 M m;n (3.1) and ka Bk 2 ka Ak kb Bk for all A; B 2 M m;n : (3.2) The inequality (3.2) has also been obtained by Okubo [14, Theorem 4.3], while (3.1) can be derived by using an argument similar to that used by Bhatia to prove Proposition 5 (another Cauchy- Schwarz type inequality) in [4]. Both of these inequalities can also be derived as corollaries of Theorem 2.3 in [10]. Proof: Let g be the symmetric gauge function associated with the unitarily invariant norm k k. A theorem of A. Horn [6, Theorem 3] gives the weak majorization relation kx i (A B) kx i (A) i (B) k = 1; : : :; n (3.3) 6

7 between the singular values of the product A B and those of A and B. Compute ka Bk 2 = g 2 ( 1 (A B); : : :; n (A B)) g 2 ( 1 (A) 1 (B); : : :; n (A) n (B)) g( 2 1(A); : : :; 2 n(a)) g( 2 1(B); : : :; 2 n(b)) = g( 1 (A A); : : :; n (A A)) g( 1 (B B); : : :; n (B B)) = ka Ak kb Bk: The rst inequality comes from combining (3.3) and (1.2), the second is an application of Theorem 2.3 to the monotone norm g(), and the penultimate equality is because 2 i (A) = i(a A). To prove (3.2) we use the weak majorization relation [8, Lemma 1] kx i (A B) for the Hadamard product and apply exactly the same argument. kx i (A) i (B) k = 1; : : :; n (3.4) The inequality (3.2) is a generalization to all unitarily invariant norms of a classical inequality of Schur for the spectral norm [17, Satz III, pg 8]: if k k is chosen to be the spectral norm jx j 2 1 (X), then (3.2) is Schur's inequality 1 (A B) 1 (A) 1 (B): Theorem 3.1 allows us to make the following generalization of Theorem 2.3 in [20]. Corollary 3.2 Let k k be a given unitarily invariant norm on M n. Then for all A 2 M n, kak = minfkb Bk kc Ck : B; C 2 M n and B C = Ag: (3.5) Proof: For any A 2 M n, Theorem 3.1 gives kak inffkb Bk kc Ck : B; C 2 M n and B C = Ag: (3.6) That the inmum is attained and is equal to kak follows by setting B = P and C = P U, where A = P U is a polar decomposition of A, i.e., P; U 2 M n ; P 0, and U is unitary. In Example 4.4 we give a non-unitarily invariant norm that satises 3.5. It is possible to characterize the unitary similarity invariant norms that satisfy (1.1); to do so, we require the following analog of Lemma 2.4. Lemma 3.3 Let k k be a unitary similarity invariant norm on M n such that kak k jaj k for all A 2 M n ; (3.7) where jaj (A A). Then N(A) k jaj k is a unitarily invariant norm on M n. 7

8 Proof: The unitary invariance, positivity and homogeneity of the function N() are clear, so it suces to prove that N() obeys the triangle inequality. We have the weak majorization (see [5], or [13, p. 243], or [9, Chapter3]): kx i (A + B) kx i (A) + kx i (B) k = 1; : : :; n; which expresses the subadditivity of the Ky Fan k-norms, i.e., the vector (A + B) is weakly majorized by the vector (A) + (B). Dene a norm k k 0 on C n by kxk 0 kdiag(x)k. Condition (3.7) guarantees that kxk 0 k jxj k 0 for all x 2 C n, so Lemma 2.4 ensures that the function kxk 00 k jxj k 0 is a monotone norm on C n. Since the given norm k k is unitary similarity invariant, the norm k k 00 is permutation invariant. Thus, the norm k k 00 is actually a symmetric gauge function. If C 2 M n is a given positive semidenite matrix, then there is a unitary U 2 M n such that C = UU, where = diag((c)). Because the norm k k is unitary similarity invariant, kck = kuu k = kdiag((c))k = k(c)k 0 = k(c)k 00 : Now use this identity with C ja+bj, noting that the eigenvalues and singular values of a positive semidenite matrix are identical, to write N(A + B) = k ja + Bj k = k(a + B)k 00 k(a) + (B)k 00 k(a)k 00 + k(b)k 00 = k jaj k + k jbj k = N(A) + N(B): The rst inequality uses weak majorization and the fact that k k 00 is a symmetric gauge function, while the second is just the triangle inequality for k k 00. The condition (3.7) is sucient for a unitary similarity invariant norm on M n to satisfy the conclusion of Lemma 3.3, but it is not necessary. See Theorem 4.9 for a stronger version of Lemma 3.3, which provides a necessary and sucient condition. Another way to prove the triangle inequality for N(A) k jaj k 00 is to use the matrix-valued triangle inequality [18, Therem 2]. We demonstrate this technique in the proof of Theorem 4.9. The following characterization is a matrix analog of Theorem 2.5 for the usual matrix product. Theorem 3.4 Let k k be a unitary similarity invariant norm on M n. Then ka Bk 2 ka Ak kb Bk for all A; B 2 M n (3.8) if and only if kak k jaj k for all A 2 M n ; (3.9) 8

9 where jaj (A A). If either (3.8) or (3.9) holds, then ka Bk 2 k ja Bj k 2 ka Ak kb Bk for all A; B 2 M n : (3.10) Proof: Let k k be a given unitary similarity invariant norm on M n. If condition (3.9) holds, then the function k j j k is a unitarily invariant norm on M n by Lemma 3.3. It now follows from (3.1) that for any A; B 2 M n ka Bk 2 k ja Bj k 2 k ja Aj k k jb Bj k = ka Ak kb Bk: Thus (3.9) implies both (3.8) and (3.10). Conversely, suppose that (3.8) holds and let A 2 M n be given. Let A = UP be a polar decomposition of A. Using the condition (3.8) and the hypothesis of unitary similarity invariance, we obtain the desired inequality: kak 2 = k(p U ) P k 2 k(p U ) (P U )k k(p ) (P )k = kup U k kp k = kp k kp k = k jaj k 2 : The hypothesis that the norm k k be unitary similarity invariant is essential, as we show in Example 4.2. In Example 4.13 we exhibit a unitary similarity invariant norm that does not satisfy the condition (3.9). We now determine the case of equality in (3.1), and to do so we require two preliminary results that are analogs of Lemmata 2.1 and 2.2. There is an analogy between absolute norms on C n and unitarily invariant norms on M n. A unitarily invariant norm k k on M n is a function only of the singular values, and hence kak = k jaj k for all A 2 M n because A and jaj have the same singular values. Lemma 3.5 A norm on M m;n is unitarily invariant if and only if its dual norm is unitarily invariant. Proof: Let k k be a given unitarily invariant norm and let A 2 M m;n ; U 2 M m, and V 2 M n be given with U and V unitary. Then kuav k D = maxfjtr B UAV j : B 2 M m;n ; kbk 1g = maxfjtr (U BV ) Aj : B 2 M m;n ; kbk 1g = maxfjtr C Aj : C 2 M m;n ; kck 1g = kak D ; which shows that k k D is unitarily invariant. The hypothesis that k k is unitarily invariant is used to obtain the penultimate equality in this series of identities. The converse now follows from the duality theorem for norms. 9

10 Before proceeding it is convenient to isolate some simple but useful facts about the Frobenius inner product on M n. Lemma 3.6 Let A; B 2 M n be given. (a) If A and B are positive semidenite then tr AB 0. (b) If A and B are Hermitian then tr AB is real. (c) Let A be positive semidenite and let B = H + ik, where H; K 2 H n. Then Re tr B A = tr HA tr jhja. Proof: If A and B are positive semidenite then so is A BA, and hence tr AB = tr A BA 0, which veries (a). The assertion in (b) follows from applying (a) to the positive semidenite matrices A + ja j 2 I and B + jb j 2 I and noting that the trace of a Hermitian matrix is real. The assertion in (c) that Re tr B A = Re (tr HA? itr KA) = tr HA follows from (b), while the inequality tr HA tr jhja follows from the observation that (jhj?h) is positive semidenite and hence tr (jhj?h)a 0. The following is a matrix analog of Lemma 2.2. Lemma 3.7 Let k k be a given unitarily invariant norm on M n and let A 2 M n be given. Then kak = maxfjtr (C A)j : C 2 M n and kck D 1g = maxftr CjAj : C 2 H n ; C 0; and kck D 1g: Proof: The rst identity is the duality theorem for norms. To show the second, compute kak = k jaj k = maxfjtr C jaj j : C 2 M n and kck D 1g = maxfre tr C jaj : C 2 M n and kck D 1g = tr C 0jAj for some C 0 2 M n with kc 0 k D 1. Then tr C jaj tr 0 jc 0jjAj by Lemma 3.6 (c), and hence, using Lemma 3.5 for the second inequality, kak tr jc 0jjAj maxftr CjAj : C 2 H n ; C 0; and kck D 1g maxfre tr C jaj : C 2 M n and kck D 1g = k jaj k = kak: Thus, all the inequalities must be equalities and the asserted identity follows. We also need a well-known result expressing the monotonicity of a unitarily invariant norm with respect to multiplication by a partial isometry [3, Prop ]. We provide a proof that uses only the existence of the singular value decomposition. 10

11 Lemma 3.8 Let k k be a unitarily invariant norm on M m;n, and let A 2 M m;n, P 1 2 M m, and P 2 2 M n be given. Then kp 1 AP 2 k 1 (P 1 ) 1 (P 2 )kak: In particular, if P 1 and P 2 are partial isometries then kp 1 AP 2 k kak: Proof: Let A 2 M m;n, P 1 2 M m, and P 2 2 M n and let k k be a unitarily invariant norm on M m;n. Assume without loss of generality that 1 (P 1 ) = 1 (P 2 ) = 1. We will show that The inequality kp 1 Ak kak: kap 2 k kak can be proved by a very similar argument. Combining these two results gives the desired conclusion. Let P 1 = UV be a singular value decomposition of P 1, i.e., U; V are unitary and = diag((p 1 )). Dene s 2 C m by s j = j (P 1 ) + i q 1? 2 j (P 1) j = 1; : : :; m and dene S = diag(s). Then S is unitary, since 0 j (P 1 ) 1 for all j = 1; : : :; m, and = 1(S + 2 S ). Using the unitary invariance of k k, the triangle inequality, and the fact that S and S are unitary we compute: kp 1 Ak = kuv Ak = kv Ak = k 1 2 (S + S )V Ak 1 2 ksv Ak ks V Ak = 1kAk + 1 kak = kak: 2 2 We are now ready to identify the cases of equality in (3.1), with a result that is an analog of the last part of Theorem 2.3. Theorem 3.9 Let k k be a given unitarily invariant norm on M n, and let A; B 2 M m;n be given non-zero matrices. Then ka Bk 2 = ka AkkB Bk if and only if there is a positive constant c and there are partial isometries P 1 and P 2 such that AP 1 = cbp 2 ; kp 1 A AP 1 k = ka Ak; and kp 2 B BP 2 k = kb Bk: (3.11) Proof: Let k k be a unitarily invariant norm on M n, let A; B 2 M m;n. The polar decomposition [7, Theorem 7.3.2] guarantees that there is a unitary U 2 M n such that A BU is positive semidenite. Use Lemma 3.7 and the Cauchy-Schwarz inequality for the Frobenius inner product to 11

12 compute ka Bk 2 = ka BUk 2 = maxf[tr CA BU] 2 : C 2 H n ; C 0; and kck D 1g = maxf[tr C A BUC ] 2 : C 2 H n ; C 0; kck D 1g maxf(tr C A AC )(tr C U B BUC ) : C 2 H n ; C 0; kck D 1g (3.12) maxftr C A AC : C 2 H n ; C 0; kck D 1g maxftr C U B BUC : C 2 H n ; C 0; kck D 1g (3.13) = maxftr CA A : C 2 H n ; C 0; kck D 1g maxftr CU B BU : C 2 H n ; C 0; kck D 1g = ka Ak ku B BUk = ka Ak kb Bk: Now suppose ka Bk 2 = ka AkkB Bk, so that the preceding inequalities must all be equalities. If inequality (3.12) is an equality, then there must be a positive semidenite C ~ 2 H n such that k Ck ~ D 1 and [tr ~ C A BU ~ C ] 2 = tr ( ~ C A A ~ C ) tr ( ~ C U B BU ~ C ): This last condition states that equality holds in the Cauchy-Schwarz inequality, which can occur only if A ~ C and BU ~ C are dependent, i.e., there is a non-zero scalar such that If inequality (3.13) is also an equality it is necessary that and A ~ C = BU ~ C : (3.14) ka Ak = tr ~ C A A ~ C (3.15) kb Bk = tr ~ C U B BU ~ C : (3.16) Let E 2 M n be the Hermitian projection onto the range of ~ C, so E is a partial isometry, E = E, and E ~ C = ~ C E = ~ C. We now show that c jj is the required positive constant and P 1 E, P 2 (=jj)ue are the required partial isometries. By the denition of E and (3.14) we have AE ~ C = A ~ C = BU ~ C = BUE ~ C and hence AE = BUE, which is the same as AP 1 = cbp 2. Now use Lemma 3.8, Lemma 3.7, and (3.15) to compute ka Ak kp 1 A AP 1 k = kea AEk 12

13 = maxftr C(EA AE) : C 2 H n ; C 0; and kck D 1g = maxftr C EA AEC : C 2 H n ; C 0; and kck D 1g tr ~ C EA AE ~ C = tr ~ C A A ~ C = ka Ak: Thus, kp 1 A AP 1 k = ka Ak, as asserted. The same argument shows that kp 2 B BP 2 k = kb Bk. Conversely, if there is a positive constant c and there are partial isometries P 1 and P 2 such that then by Lemma 3.8 we have AP 1 = cbp 2 ; kp 1 A AP 1 k = ka Ak; and kp 2 B BP 2 k = kb Bk ka Bk 2 ka AkkB Bk Thus, both inequalities must be equalities. = kp 1 A AP 1 kkp 2 B BP 2 k = kp 1 A cbp 2 kk(1=c)p 1 A BP 2 k = kp 1 A BP 2 k 2 ka Bk 2 : 4 Examples, Counterexamples and Corollaries Although Wimmer's conjecture (1.1) is now settled, there are several interesting points to be noted. The rst is that not every norm on M n satises (1.1) and there are norms satisfying (1.1) that are not unitarily invariant. Moreover, the set of norms satisfying (1.1) is a convex set. Example 4.1 Consider the l p norms dened on M m;n by kak p ( X i;j ja ij j p ) 1=p 1 p < 1 Let m = n = 2 and kak 1 maxfja ij j : 1 i m; 1 j ng: A = 1?1 ; B = : 0 1 Then (1.1) does not hold for the l p norms when 1 p < 2 since ka Bk 2 p = 24=p 6 (2 2 1=p )(2 1=p ) = ka Ak p kb Bk p : Example 4.2 Let A = [a ij ]; B = [b ij ] 2 M n. The l 1 norm k k 1 on M m;n is not unitarily invariant, but does satisfy (1.1). Let A = [a 1 : : :a n ] and B = [b 1 : : : b n ] be partitioned according to their columns. Then ka Bk 2 1 = max ja i b j j 2 (max a i a i )(max b jb j ) ka Ak 1 kb Bk 1 : 13

14 Notice that k k 1 does not satisfy condition (3.7); for example, consider 1 1 q 1 1 A = ; jaj = : For this choice of A we have kak 1 = 1 6 1= p 2 = k jaj k 1. However this does not contradict Theorem 3.4 because k k 1 is not unitary similarity invariant. Example 4.3 Let C 2 M n be given. The C-numerical radius is dened on M n by r C (A) maxfjtr CU AUj : U 2 M n is unitaryg: If C is not a scalar matrix and tr C 6= 0 then it is known [12] that r C () is a norm on M n. The function r C () is unitary similarity invariant but is never unitarily invariant when n > 1 and C 6= 0 because, under these conditions, one can always construct A 2 M n such that r C (A) 6= r C (jaj). The classical numerical radius, r(a) maxfjx Axj : x 2 C n and kxk 2 = 1g; is an example of a norm of the form r C (); it corresponds to the positive semidenite matrix C = [1] 0 n?1. Note that r C (A) = maxfjtr CU AU)j : U 2 M n is unitaryg (4.1) = maxf maxf nx nx nx i (CU AU) : U 2 M n is unitaryg i (C) i (U AU) : U 2 M n is unitaryg i (C) i (A): The rst inequality is an application of the inequality [13, Theorem 20.b.1] jtr Xj 1 (X) + + n (X) for any X 2 M n ; while the second is a special case of (3.3). Suppose A and C are both positive semidenite. Then there are unitary matrices U 1 ; U 2 2 M n such that A = U 1 diag((a)) U 1 and C = U 2 diag((c)) U 2 : The choice U U 1 U 2 in (4.1) then shows that the preceding inequalities are equalities in this case. Hence for positive semidenite C 2 M n and any A 2 M n we have r C (jaj) = nx i (C) i (A) r C (A): Thus, Theorem 3.4 guarantees that whenever C 2 M n is a non-scalar positive semidenite matrix the unitary similarity invariant (but not unitarily invariant when n > 1) norm r C () on M n satises (1.1). The numerical radius is an example of such a norm. 14

15 Example 4.4 The Hadamard operator norm k k H on M m;n is kak H maxf 1 (A B) : B 2 M m;n and 1 (B) = 1g: Although k k H is not unitarily invariant, it satises not only (1.1) [1, Section 5] but also a rectangular version of (3.5) [15, Section 7.7] : for any A 2 M m;n kak H = minf(kb Bk H kc Ck H ) : B; C 2 M m;n ; B C = Ag: (4.2) If A 0 then it is known that kak H = maxfa ii : i = 1; : : :; ng. For general A 2 M m;n, however there is no known explicit formula to calculate kak H, so (4.2) may provide a useful bound, or may provide the basis for a practical algorithm. Theorem 4.5 Let N 1 () and N 2 () be given norms on M n that satisfy the inequality (1.1) and let 2 [0; 1] be given. Then N() N 1 () + (1? )N 2 () also satises (1.1), so the set of norms satisfying (1.1) is a convex set that does not include all norms and is strictly larger than the set of unitarily invariant norms. Proof: Since any convex combination of norms is a norm, we need only show that N() satises (1.1). Compute N(A B) 2 = [N 1 (A B) + (1? )N 2 (A B)] 2 = 2 N 2 1 (A B) + 2(1? )N 1 (A B)N 2 (A B) +(1? ) 2 N 2 2 (A B) 2 N 1 (A A)N 1 (B B) +2(1? )([N 1 (A A)N 1 (B B)][N 2 (A A)N 2 (B B)]) +(1? ) 2 N 2 (A A)N 2 (B B) = [N 1 (A A) + (1? )N 2 (A A)] [N 1 (B B) + (1? )N 2 (B B)] = N(A A)N(B B): These ideas suggest a way to generate new norms on M n. A pre-norm is a continuous, homogeneous, positive function on a real or complex vector space; it does not necessarily satisfy the triangle inequality. Theorem 4.6 Let k k be a given norm on M n, and dene N(A) ka Ak. always a pre-norm on M n. If the norm k k also satises the inequality Then N() is ka Bk 2 ka Ak kb Bk for all A; B 2 M m;n (4.3) then N() is a norm on M n. In particular, N(A) ka Ak is a unitarily invariant norm on M n if k k is a unitarily invariant norm, or if k k is a unitary similarity invariant norm such that kak k jaj k for all A 2 M n, where jaj (A A). 15

16 Proof: The function N(A) ka Ak is clearly positive, homogeneous, and continuous for any norm k k, so it is always a pre-norm on M n. It is a straightforward computation to show that N() satises the triangle inequality if it satises the inequality (4.3). If one chooses for k k the spectral norm, the trace norm (kak tr jaj), the numerical radius, the l 1 norm, or the Hadamard operator norm, then the respective norms N(A) (ka Ak) are the spectral norm, the Frobenius norm (N(A) [tr A A] ), the spectral norm, and N(A) the largest Euclidean column length in the last two cases. See Example 4.13 for a unitary similarity invariant norm that does not satisfy the monotonicity condition at the end of Theorem 4.6. We have the following analog of Theorem 4.6 for vector norms on C n. Its proof is very similar to that of Theorem 4.6. Theorem 4.7 Let k k be a norm on C n such that kzk k jzj k for all z = [z i ] 2 C n, where jzj [jz i j]. Then (x) (kx xk) is an absolute norm on C n. We will now consider the converses of some of the results proved so far. First we characterize the norms k k on C n such that k j j k is also a norm, and the unitary similarity invariant norms k k on M n such that k j j k is a norm on M n. Theorem 4.8 Let k k be a given norm on C n, and let jxj [jx i j] for all x = [x i ] 2 C n. Then () k j j k is a norm if and only if kuk kvk whenever u; v 2 R n + and u v: (4.4) This result is Theorem 5 in [2] (see also [7, Theorem ]), where norms that satisfy the condition (4.4) are referred to as monotone on the positive orthant. Notice that (2.8) provides a sucient condition for k j j k to be a norm on C n (Lemma 2.4), while the condition (4.4) is both necessary and sucient. In Example 4.10, we show that the condition (4.4) is strictly weaker than (2.8). Theorem 4.9 Let k k be a given unitary similarity invariant norm on M n, and let jaj (A A) for A 2 M n. Then N(A) k jaj k is always a unitarily invariant function on M n, and it is a norm on M n if and only if kxk ky k whenever X; Y 2 H n and 0 X Y: (4.5) Proof: The unitary invariance of N() is clear. If N() is a norm then it is unitarily invariant and agrees with k k on the positive semidenite matrices. The inequality (4.5) is true for any unitarily invariant norm because if 0 X Y, then i (X) i (Y ) for i = 1; : : :; n; in particular, the singular values of X are weakly majorized by those of Y. Conversely, let A; B 2 M n be given and assume (4.5). By the matrix-valued triangle inequality [18, Theorem 2], there are unitary U; V 2 M n such that ja + Bj UjAjU + V jbjv 16

17 and hence (4.5), the ordinary triangle inequality, and the unitary similarity invariance of k k give N(A + B) = k ja + Bj k kujaju + V jbjv k kujaju k + kv jbjv k = k jaj k + k jbj k = N(A) + N(B): Since positivity and homogeneity are clear, it follows that N() is a norm. In Example 4.13 we show that there are unitary similarity invariant norms that do not satisfy the monotonicity condition (4.5). As one might suspect from Theorem 4.8, the converses of Lemma 2.4 and Theorem 4.7 are not true. There are norms on C n that satisfy the condition (4.4) but not (2.8). Example 4.10 Consider the function k k dened on C 2 by kxk maxfjx 1 j; jx 2 j; jx 1? x 2 jg: Then k k is easily shown to be a norm, but it does not satisfy the monotonicity condition kxk k jxj k. Consider, for example, x = [1;?1] T. However, 1 (x) kx xk, and 2 (x) k jxj k are both norms since 1 (x) = 2 (x) = kxk 1 = maxfjx 1 j; jx 2 jg: Thus, k k is a norm on C 2 that satises the condition (4.4) but not (2.8). Similarly, one might suspect from Theorem 4.9 that the converses of Lemma 3.3 and Theorem 4.6 are also false. There are unitary similarity invariant norms on M n that satisfy the condition (4.5), but not (3.7). To construct one we rst prove Lemma 4.11 Let k k be a given norm on the real vector space H n. Then the function k k 0 : M n! R + dened by kak 0 maxfk 1 2 (A + A )k : 2 C; and jj = 1g (4.6) is a self-adjoint norm on M n that agrees with k k on H n, i.e., kak 0 = ka k 0 for all A 2 M n and kak 0 = kak for all A 2 H n. If the given norm k k is unitary similarity invariant on H n, then the norm k k 0 is also unitary similarity invariant on M n. Furthermore, the norm k k 0 is minimal in the following sense: if N() is any self-adjoint norm on M n that agrees with k k on H n, then N(A) kak 0 for all A 2 M n. Proof: The positivity and homogeneity of k k 0 follow from the positivity and homogeneity of k k. For the triangle inequality, take A; B 2 M n and compute ka + Bk 0 = maxfk 1 2 [(A + B) + (A + B) ]k : 2 C jj = 1g 17

18 maxfk 1 2 (A + A )k + k 1 2 (B + B )k : 2 C; and jj = 1g maxfk 1 2 (A + A )k : 2 C; jj = 1g = kak 0 + kbk 0 : + max fk 1 2 (B + B )k : 2 C; jj = 1g We now know that k k 0 is a norm on M n, and the fact that kak 0 = ka k 0 is immediate from the denition, as is the assertion about unitary similarity invariance. Suppose that A 2 H n and jj = 1. Then k 1(A + 2 A )k = k 1 (A + A)k = k(re )Ak = jre j kak kak 2 with equality for = 1. This shows that kak 0 = kak whenever A 2 H n. Finally, consider the assertion about the minimality of k k 0. Let N() be a given norm on M n such that N(A) = N(A ) for all A 2 M n and N(A) = kak for all A 2 H n. Then for any 2 C with jj = 1 we have N(A) = 1 2 [N(A) + N((A) ) N( 1 2 [A + A ]) = k 1 2 [A + A ]k and hence N(A) maxfk 1 2 [A + A ]k : 2 C; and jj = 1g = kak 0 : Example 4.12 We shall exhibit a norm that shows that the implications in Theorem 4.6 and Lemma 3.3 cannot be reversed. Let 1 (X) 2 (X) denote the algebraically ordered eigenvalues of X 2 H 2. Dene the function k k : H 2! R + by kxk = maxfj 1 (X)j; j 2 (X)j; j 1 (X)? 2 (X)jg: Notice the similarity between this function and the norm on C 2 dened in Example One can easily verify that the function k k is a norm on the real vector space H n : Either use the Weyl inequalities [7, Theorem 4.3.1] 1 (A + B) 1 (A) + 1 (B) 2 (A + B) 2 (A) + 2 (B) for the eigenvalues of any A; B 2 H 2, or note that the function kxk is a Schur-convex function of the eigenvalues of X and apply Theorem 3 in [11]. Notice that kak = 1 (A) = ja j 2 if A 2 H 2 is positive semidenite. Let the norm k k 0 be derived from k k as dened in (4.6), and use Lemma 4.11 to observe that k k 0 is a unitary similarity invariant norm on M 2. For X 2 M 2 set Then N(X) (kx Xk 0 ) and (X) k jxj k 0 ; where jxj (X X) : N(X) = (kx Xk 0 ) = (kx Xk) = ( jx X j 2 ) = jx j 2 18

19 and (X) = k jxj k 0 = k jxj k = j jxj j 2 = jx j 2 : Thus, both N(X) and (X) are unitarily invariant norms on M 2. However, the choice A = ; B = 0 1 0?1 shows that the norm k k 0 satises neither (4.5) nor (3.7) since ka Ak 0 = kb Bk 0 = 1, ka Bk 0 = 2, kbk 0 = 2 and k jbj k 0 = 1. Example 4.13 There is a unitary similarity invariant norm on M 2 that satises neither (3.7) nor (4.5). Let C = 2 0 0?1 and consider the unitary similarity invariant norm r C () on M 2. There is no unitary U for which UjCjU is a scalar multiple of C. By the Cauchy-Schwarz inequality for the Frobenius inner product and the denition of r C () we have jtr CUjCjU j 2 < (tr C 2 )(tr UjCj 2 U ) = (tr C 2 ) 2 r 2 C(C) for any unitary U, and hence r C (jcj) < r C (C). Thus, the unitary similarity invariant norm r C () does not satisfy (3.7). To see that r C () does not satisfy (4.5) either, set X = ; Y = and note that 0 X Y, but r C (X) = 2 > 1 = r C (Y ) Open Questions In Theorem 3.4 we characterized the unitary similarity invariant norms on M n for which (1.1) holds, and in Example 4.2 we showed that the norm k k 1, which is not unitary similarity invariant, satises (1.1). Is there a useful characterization of the norms on M m;n that satisfy (1.1)? In Corollary 3.2 we have shown that for any unitarily invariant norm k k on M n and all A 2 M n, kak = minfkb Bk kc Ck : B; C 2 M n and B C = Ag: (5.1) If k k is a unitary similarity invariant norm, then the right-hand side of (5.1) is a unitarily invariant function of A 2 M n. Thus, a unitary similarity invariant norm satises (5.1) if and only if it is unitarily invariant. We showed in Example 4.4 that the non-unitarily invariant norm k k H also obeys (5.1). How can one characterize the norms that satisfy (5.1)? Consider the l p norms k k p on M m;n, dened in Example 4.1. We have shown that the inequality ka Bk 2 p ka Ak p kb Bk p for all A; B 2 M m;n (5.2) 19

20 is false for p 2 [1; 2) (Example 4.1), and true for p = 2 (this is the Cauchy-Schwarz inequality for the Frobenius inner product). Does (5.2) hold for p 2 (2; 1)? This question is partially answered in [10, Example 4.4]. For p 1, the Schatten-p norm on M n is dened by kak Sp ( nx p i (A))1=p : If p = 2k for some integer k, then the Schatten-p norm can also be dened by From this representation it is clear that kak Sp (tr (A A) k ) k : whenever p is an even integer. Thus Theorem 2.5 ensures that k[a ij ]k Sp k[ja ij j]k Sp (5.3) ka Bk 2 Sp ka AkSp kb BkSp for all A; B 2 M n (5.4) whenever p is an even integer. If n = 2, then (5.3) holds for all p 2. To see that (5.3) is not true for 1 p < 2, consider 1 1 A = : 1?1 What are the values of p for which (5.3), and hence (5.4) holds? One can show that the answer depends on n. More generally, what are the unitarily invariant norms k k for which k[a ij ]k k[ja ij j]k for all [a ij ] 2 M m;n? Notice that the spectral norm and the Frobenius norm satisfy this inequality. References [1] T. Ando, R. A. Horn, and C. R. Johnson. The singular values of a Hadamard product: A basic inequality. Lin. Multilin. Alg., 21:345{65, [2] F. L. Bauer, J. Stoer, and C. Witzgall. Absolute and monotonic norms. Numerische Mathematik, 3:257{64, [3] R. Bhatia. Perturbation Bounds for Matrix Eigenvalues. Pitman Research Notes in Mathematics 162. Longman Scientic and Technical, New York, [4] R. Bhatia. Perturbation inequalities for the absolute value map in norm ideals of operators. J. Operator Theory, 19:129{36, [5] K. Fan. Maximum properties and inequalities for the eigenvalues of completely continuous operators. Czech. J. Math., 12:382{400,

21 [6] A. Horn. On the singular values of a product of completely continuous operators. Proc. Nat. Acad. Sci., 36:374{5, [7] R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, New York, [8] R. A. Horn and C. R. Johnson. Hadamard and conventional submultiplicativity for unitarily invariant norms on matrices. Lin. Multilin. Alg., 20:91{106, [9] R. A. Horn and C. R. Johnson. Topics in Matrix Analysis. Cambridge University Press, New York, [10] R. A. Horn and R. Mathias. Cauchy-Schwarz inequalities associated with positive semidenite matrices. Technical Report 514, Department of Mathematical Sciences, The Johns Hopkins University, Baltimore, [11] C.-K. Li and N.-K. Tsing. Norms that are invariant under unitary similarities and the C- numerical radii. Lin. Multilin. Alg., 24:209{22, [12] M. Marcus and M. Sandy. Three elementary proofs of the Goldberg-Strauss theorem on numerical radii. Lin. Multilin. Alg., 11:243{52, [13] A. W. Marshall and I. Olkin. Inequalities: Theory of Majorization and its Applications. Academic Press, London, [14] K. Okubo. Holder-type inequalities for Schur products of matrices. Lin. Alg. Appl., 91:13{28, [15] V. I. Paulsen. Completely Bounded Maps and Dilations. Pitman Research Notes In Mathematics 146. Longman Scientic and Technical, Harlow, [16] R. Schatten. Norm Ideals of Completely Continuous Operators. Springer-Verlag, berlin, [17] J. Schur. Bermerkungen zur Theorie der beschrankten Bilinearformen mit unendlich vielen Veranderlichen. J. fur Reine und Angewandte Mathematik, 140:1{28, [18] R. C. Thompson. Convex and concave functions of singular values of matrix sums. Pacic J. of Math., 66:285{90, [19] J. von Neumann. Some matrix inequalities and metrization of matric space. Tomsk. Univ. Rev., 1:286{300, Also in John von Neumann Collected Works (A. H. Taub, ed.) Vol. IV, pp , Pergamon Press, Oxford, [20] H. Wimmer. Extremal problems for Holder norms of matrices and realizations of linear systems. SIAM J. Matrix Anal. Appl., 9:314{322,