Sophomoric Matrix Multiplication

Transcription

1 Sophomoric Matrix Multiplication Carl C. Cowen IUPUI (Indiana University Purdue University Indianapolis) Universidad de Zaragoza, 3 julio 2009

2 Linear algebra students learn, for m n matrices A, B, y C, matrix addition is A + B = C if and only if a ij + b ij = c ij. Expect matrix multiplication is AB = C if and only if a ij b ij = c ij, But, the professor says No! It is much more complicated than that! Today, I want to explain why this kind of multiplication not only is sensible but also is very practical,very interesting, and has many applications in mathematics and related subjects. Definition If A y B are m n matrices, the Schur (o Hadamard o naïve o sophomoric) product of A y B is the m n matrix C = A B with c ij = a ij b ij.

3 These ideas go back more than a century to Moutard (1894), who didn t even notice he had proved anything(!), Hadamard (1899), and Schur (1911). Hadamard considered analytic functions f(z) = n=0 a nz n and g(z) = n=0 b nz n that have singularities at {α i } and {β j } respectively. He proved that if h(z) = n=0 a nb n z n which has singularities {γ k }, then {γ k } {α i β j }.

4 This seems a little less surprising when you consider convolutions: Let f and g be 2π-periodic functions on R and so that a k = 2π 0 e ikθ f(θ) dθ 2π y b k = 2π 0 e ikθ g(θ) dθ 2π f a k e ikθ y g b k e ikθ If h(θ) = 2π 0 f(θ t)g(t) dt 2π, then h a k b k e ikθ y f 0 y g 0 implies h 0.

5 Schur s name is most often associated with the matrix product because he published the first theorem about esta typo de matrix multiplicación. Definition A real ( or complex) n n matrix is called positive or positive semidefinite if A = A Ax, x 0 for all x in R n ( or C n ) Then For any m n matrix A, both AA and A A are positive. Conversely, if B is positive, then B = AA for some A. In statistics, every variance-covariance matrix is positive.

6 Examples: A = B = 1 2 is NOT positive: but BC = 2 1, y C = = =, are positive is not. = 1

7 Schur Product Theorem (1911) Si A y B son positive n n matrices, then A B is positive also. Applications: Experimental design: Si A y B son variance-covariance matrices, then A B is positive also. P.D.E. s: Let Ω be a domain in R 2 and let L be the differential operator Lu = a 11 x + 2a a 22 y + b u 2 1 x + b u 2 y + cu L is called elliptic if a 11 a 12 a 21 a 22 is positive definite.

8 Lu = a 11 x + 2a a 22 y + b u 2 1 x + b u 2 y + cu Weak Minimum Principle (Moutard, 1894) Si L is elliptic, c < 0, and L 0 in Ω, then u cannot have a negative minimum value in Ω. By contradiction: If u has a minimum at (x 0, y 0 ) and u(x 0, y 0 ) < 0, then and u x (x 0, y 0 ) = u y (x 0, y 0 ) = 0 0 = Lu = a 11 x + 2a a 22 y + b u 2 1 x + b u 2 y + cu = a 11 x + 2a a 22 y + cu 2

9 Weak Minimum Principle (Moutard, 1894) Si L is elliptic, c < 0, and L 0 in Ω, then u cannot have a negative minimum value in Ω. 0 = Lu = a 11 x + 2a a 22 y + cu 2 = a 11 2 u x 2 a 12 = a 11 a 12 a 12 a 22 a 12 a 22 y 2 x 2 y 2, + cu, + cu

10 Weak Minimum Principle (Moutard, 1894) Si L is elliptic, c < 0, and L 0 in Ω, then u cannot have a negative minimum value in Ω. 0 = Lu = a 11 x + 2a a 22 y + cu 2 = a 11 2 u x 2 a 12 = a 11 a 12 a 12 a 22 a 12 a 22 y 2 x 2 y 2, + cu, + cu

11 Weak Minimum Principle (Moutard, 1894) Si L is elliptic, c < 0, and L 0 in Ω, then u cannot have a negative minimum value in Ω. 0 = Lu = a 11 x + 2a a 22 y + cu 2 = a 11 2 u x 2 a 12 = a 11 a 12 a 12 a 22 a 12 a 22 y 2 x 2 y 2, + cu, + cu

12 Weak Minimum Principle (Moutard, 1894) Si L is elliptic, c < 0, and L 0 in Ω, then u cannot have a negative minimum value in Ω. 0 = Lu = a 11 x + 2a a 22 y + cu 2 = a 11 2 u x 2 a 12 a 12 = a 11 a 12 a 12 a 22 a 22 y 2 x 2 y 2, + cu, + cu

13 Weak Minimum Principle (Moutard, 1894) Si L is elliptic, c < 0, and L 0 in Ω, then u cannot have a negative minimum value in Ω. 0 = Lu = a 11 x + 2a a 22 y + cu 2 = a 11 2 u x 2 a 12 = a 11 a 12 a 12 a 22 a 12 a 22 y 2 x 2 y 2, + cu, + cu > 0

14 Weak Minimum Principle (Moutard, 1894) Si L is elliptic, c < 0, and L 0 in Ω, then u cannot have a negative minimum value in Ω. 0 = Lu = a 11 x + 2a a 22 y + cu 2 = a 11 2 u x 2 a 12 = a 11 a 12 a 12 a 22 a 12 a 22 y 2 x 2 y 2, + cu, + cu > 0 Contradiction!

15 Fejer s Uniqueness Theorem Si L is elliptic y c < 0 on Ω, then there is at most one solution to the boundary value problem Lu = f u = g en Ω en Ω that is continuous on Ω and smooth en Ω. Si u 1 y u 2 son both solutíones con u 1 u 2, then since u 1 = g = u 2 en Ω and u 1 u 2 = 0 = u 2 u 1 en Ω, either u 1 u 2 or u 2 u 1 must have a negative minimum value en Ω. But Lu 1 = f = Lu 2 en Ω, so L(u 1 u 2 ) 0 L(u 2 u 1 ) en Ω Moutard: neither has negative minimum value en Ω so must have u 1 u 2.

16 Goal: Prove Schur s theorem Recall (AB) t = B t A t y (AB) = B A Do for case of real scalars: same pero less comfortable para most matématicos con complex scalars Use column vectors: x, y = x 1 y 1 + x 2 y 2 + x 3 y x n y n = x t y Lemma An n n matrix A is positive if and only if A = v 1 v t 1 + v 2 v t 2 + v 3 v t v k v t k for vectors v 1, v 2, v 3,, v k and k n.

17 Lemma An n n matrix A is positive if and only if A = v 1 v t 1 + v 2 v t 2 + v 3 v t v k v t k for vectors v 1, v 2, v 3,, v k and k n. ( ) Let x be in R n and A = v 1 v t 1 + v 2 v t v k v t k for vectors v 1, v 2,, v k. Then Ax, x = (v 1 v t 1 + v 2 v t v k v t k)x, x = j v j v t j x, x = j (v j v t j x) t x = j = j x t ((v t j ) t v t j x = j v j, x v j, x = j (v t j x) t (v t j x) v j, x 2 0

18 Lemma An n n matrix A is positive if and only if A = v 1 v t 1 + v 2 v t 2 + v 3 v t v k v t k for vectors v 1, v 2, v 3,, v k and k n. ( ) Let A be positive. Since A = A t, there is an orthonormal basis for R n that consists of eigenvectors for A, call it w 1, w 2,, w n. Then, for each j, let α j be the eigenvalues of A, that is, Aw j = α j w j. Because A is positive, α j 0 for all j. We suppose they have been numbered so that for 1 j k, α j > 0 and for j k + 1, α j = 0. For 1 j k, choose β j > 0 with α j = βj 2 and let v j = β j w j. Then, we show that A = v 1 v t 1 + v 2 v t v k v t k = α 1 w 1 w t 1 + α 2 w 2 w t α k w k w t k

19 Lemma An n n matrix A is positive if and only if A = v 1 v t 1 + v 2 v t 2 + v 3 v t v k v t k for vectors v 1, v 2, v 3,, v k and k n. ( ) To show that A = α 1 w 1 w t 1 + α 2 w 2 w t α k w k w t k we will show that for each x in R n, Ax and (α 1 w 1 w t 1 + α 2 w 2 w t α k w k w t k )x are the same. If x is a vector in R n, then x is a linear combination of the w j s, say x = x 1 w 1 + x 2 w x n w n. Then Ax is given by Ax = A(x 1 w 1 + x 2 w x n w n ) = x 1 Aw 1 + x 2 Aw x n Aw n

20 Lemma An n n matrix A is positive if and only if A = v 1 v t 1 + v 2 v t 2 + v 3 v t v k v t k for vectors v 1, v 2, v 3,, v k and k n. ( ) which is Ax = x 1 Aw 1 + x 2 Aw x n Aw n = x 1 α 1 w 1 + x 2 α 2 w x k α k w k + + x n α n w n = x 1 α 1 w 1 + x 2 α 2 w x k α k w k + + x n 0w n = x 1 α 1 w 1 + x 2 α 2 w x k α k w k Notice that w t i w j = w i, w j = δ ij.

21 Lemma An n n matrix A is positive if and only if A = v 1 v t 1 + v 2 v t 2 + v 3 v t v k v t k for vectors v 1, v 2, v 3,, v k and k n. ( ) Similar to the above calculation, (v 1 v t 1 + v 2 v t v k vk)x t = (α 1 w 1 w t 1 + α 2 w 2 w t α k w k wk)x t = ( i α i w i w t i )( j x j w j ) = i,j (α i w i w t i )x j w j = i,j α i x j w i (w t i w j ) = i α i x i w i = x 1 α 1 w 1 + x 2 α 2 w x k α k w k Thus, for each x, Ax = (v 1 v t 1 + v 2 v t v k v t k )x and A = v 1 v t 1 + v 2 v t v k v t k

22 Lemma If u and v are vectors in R n, then (uu t ) (vv t ) = (u v)(u v) t. u 1 u 1 u 1 u 2 u 1 u n If u = (u 1, u 2,, u n ), then uu t u 2 u 1 u 2 u 2 u 2 u n =..... u n u 1 u n u 2 u n u n Thus, u 1 u 1 u 1 u 2 u 1 u n v 1 v 1 v 1 v 2 v 1 v n (uu t ) (vv t u 2 u 1 u 2 u 2 u 2 u n v 2 v 1 v 2 v 2 v 2 v n ) = u n u 1 u n u 2 u n u n v n v 1 v n v 2 v n v n =

23 Lemma If u and v are vectors in R n, then (uu t ) (vv t ) = (u v)(u v) t. Thus, (uu t ) (vv t ) = u 1 u 1 v 1 v 1 u 1 u 2 v 1 v 2 u 1 u n v 1 v n u 2 u 1 v 2 v 1 u 2 u 2 v 2 v 2 u 2 u n v 2 v n..... u n u 1 v n v 1 u n u 2 v n v 2 u n u n v n v n = u 1 v 1 u 1 v 1 u 1 v 1 u 2 v 2 u 1 v 1 u n v n u 2 v 2 u 1 v 1 u 2 v 2 u 2 v 2 u 2 v 2 u n v n..... = (u v)(u v) t u n v n u 1 v 1 u n v n u 2 v 2 u n v n u n v n

24 Schur Product Theorem (1911) Si A y B son positive n n matrices, then A B is positive also. There are vectors u 1, u 2,, u k so that A = u 1 u t 1 + u 2 u t u k u t k and vectors v 1, v 2,, v l so that B = v 1 v t 1 + v 2 v t v l v t l. Now, ( ) A B = u i u t i j v j v t j i = i,j (u i u t i ) (v j v t j ) = i,j (u i v j ) (u i v j ) t 0

25 Corollary Si A = (a ij is a positive n n matrix, then (a 2 ij ), (a3 ij ), (ea ij) are positive also. For example, is positive, so , , and e3 e 2 e 2 e 2 are also positive!

26 Application: Matrix completion problems 15 2 a b c Are there numbers a, b, and c so that A = a b c 3 13 is positive? In this case, yes: a = 7, b = 5, and c = 8.

27 General Problem For B is a fixed n n matrix, compute the Schur multiplier norm of B, that is, find the smallest constant K B such that X B K B X Moreover, we want to have a computationally effective way to find K B.

28 Schur (1911) Si B is a positive n n matrix, then its Schur multiplier norm is its largest diagonal entry. If β is the largest diagonal entry of B, then B I = β, and K B β. Note that A α if and only if Schur s Theorem implies, 0 B B B B αi A A αi I A A I = 0. B I B A B A B I = B I B A (B A) B I βi B A (B A) βi