7 Matrix Operations. 7.0 Matrix Multiplication + 3 = 3 = 4

Transcription

1 7 Matrix Operations Copyright 017, Gregory G. Smith 9 October 017 The product of two matrices is a sophisticated operations with a wide range of applications. In this chapter, we defined this binary operation, visualize it using directed graphs, and start to exploration of invertible matrices. 7.0 Matrix Multiplication How should we define matrix multiplication? When a matrix A multiplies a vector ~v, it transforms ~v into the vector A~v. If this vector is then multiplied by the matrix B, the resulting vector is B(A~v). We want to represent the composite mapping ~v 7! B(A~v) as multiplication by a single matrix BA, so that B(A~v) =(BA)~v. To describe matrix multiplication, let A be an (m n)-matrix, let B an (` m)-matrix, and let ~v K n. If ~a 1,~a,...,~a n denote the columns of A, then we have A~x = v 1 ~a 1 + v ~a + + v n ~a n K m. By the linearity of matrix multiplication, we obtain B(A~x) =B(v 1 ~a 1 )+B(v ~a )+ + B(v n ~a n )=v 1 (B~a 1 )+v (B~a )+ + v n (B~a n ) K`. In particular, we have B(A~x) =[B~a 1 B~a B~a n ]~v and we define BA := [B~a 1 B~a B~a n ] Problem. Compute the product Solution. We have = = so the product is = 1 5 = = = = ()()+()(1) ()()+()( ) ()(6)+()() (1)()+( 5)(1) (1)()+( 5)( ) (1)(6)+( 5)() = Matrix multiplication may be reinterpreted using the dot product. Assuming that the product BA is defined, the (i, k)-entry of BA is dot

2 90 linear algebra product of the i-th row in B with the k-th column in A. Indeed, if b i,j for 1 6 i 6 ` and 1 6 j 6 m are the entries in B and a j,k for 1 6 j 6 m and 1 6 k 6 n are the entries in A, then the (i, k)-entry of the product AB is  m j=1 b i,ja j,k = b i,1 a 1,k + b i, a,k + + b i,m a m,k Problem. Find the entries in the second row of the product Solution. If denotes an arbitrary scalar, then we have = = Problem. Compute both products of the following matrices: Solution. We have = = For n N, the identity matrix is the (n n)-matrix I n := [~e 1 ~e ~e n ] = The subscript on the square matrix I will be omitted when the number of rows or columns is clear from the context Proposition (Properties of matrix multiplication). Let A, B, C be matrices for which the indicated sums and products are defined. For any scalar c K, we have the following properties: (identity) I m A = A = AI n (BA) T = A T B T (associativity) C(BA)=(CB)A c(ba)=(cb)a = B(c A) (distributivity) C(B + A)=CB + CA (C + B)A = CA + BA. Proof. Let a j,k for 1 6 j 6 m and 1 6 k 6 n be the entries in the matrix A and let ~a 1,~a,...,~a n be the columns of A; it follows that ~a k = [a 1,k a,k a m,k ] T. Since ~e j ~a k = a j,k and ~a k ~e j = a k,j,we

3 matrix operations 91 deduce that I m A = A = AI n. If b i,j for 1 6 i 6 ` and 1 6 j 6 m denotes the entries in the matrix B, then the (i, k)-entry in BA is  m j=1 b i,ja j,k and the (k, i)-entry in A T B T is  m j=1 a j,kb i,j, so the commutativity of multiplication of scalars implies that (BA) T = A T B T. Finally, the general definition of matrix multiplication and the linearity of matrix multiplication with a vector yield C(BA) =[C(B~a 1 ) C(B~a ) C(B~a n )] = [(CB)~a 1 (CB)~a (CB)~a n ] =(CB)A, c(ba) =[c(b~a 1 ) c(b~a ) c(b~a n )] = [(cb)~a 1 (cb)~a (cb)~a n ] =(cb)a = [c(b~a 1 ) c(b~a ) c(b~a n )] = [B(c~a 1 ) B(c~a ) B(c~a n )] = B(cA), C(B + A) = C(~ b1 +~a 1 ) C(~ b +~a ) C(~ bn +~a n ) = C~ b1 + C~a 1 C~ b + C~a C~ bn + C~a n = CB + CA, (C + B)A = [(C + B)~a 1 (C + B)~a (C + B)~a n ] = [C~a 1 + B~a 1 C~a + B~a C~a n + B~a n ] = CA + BA. There are some important differences between products of matrices and products of scalars. The product of two matrices depends on the order of the factors. For example, we have = 0 0 6= 0 0 = The product of two nonzero matrices may equal zero. For example, we have = 0 0. The cancellation property does not hold for matrix multiplication, that is the matrix equation AB = AC does not imply B = C. For example, we have = 0 0 = , but = If A is an square matrix and k N, then the k-fold product of A is A k := AA {z A } is the product of k copies of A. In particular, the k times empty product is A 0 = I Problem. If N = , then compute N k for all k N Solution. We have N 0 = I, N 1 = N, N = = , N = = , and N k = 0 for k >.

4 9 linear algebra Exercises Problem. Determine which of the following statements are true. If a statement is false, then provide a counterexample. i. Matrix multiplication is defined for any two matrices. ii. The product of two matrices is a matrix. iii. The entry in a product of matrices equals the dot product of the corresponding row and column. iv. The identity matrix can have any size. v. Matrix multiplication is never commutative Problem. Let A be a square matrix. If the linear system A ~x =~0 has infinitely many solutions, then prove that the linear system A~x =~0 also has infinitely many solutions Problem. The trace of an (n n)-matrix A := [a j,k ] is the sum of its diagonal entries: tr(a) := a 1,1 + a, + + a n,n = n  a j,j. j=1 i. Prove that the trace is linear. In other words, show that, for any (n n)-matrices A, B and any scalars c, d K, we have tr(c A + d B) =c tr(a)+d tr(b). ii. If A and B are (n n)-matrices, then prove that tr(ab) =tr(ba). iii. For n > 1, show that the matrix equation XY YX = I has no solutions for (n n)-matrices X and Y.

5 matrix operations Adjacency Matrices* How can we visualize matrix multiplication for square matrices? A directed graph G consists of a set V(G) := {1,,..., n} of vertices and a set E(G) of ordered pairs of vertices called edges. The edge (j, k) is drawn as an arrow from the jth vertex to the kth vertex. Given a directed graph G, the adjacency matrix A G is a square matrix where rows and columns correspond to the vertices of G. The (j, k)-entry in A G is the number of edges in G from the kth vertex to the jth vertex. The adjacency matrices for our two examples are A T = A H = From the definition of the adjacency matrix, we see that every square matrix with entries in {0, 1} is the adjacency matrix of a direct graph. A walk in the directed graph G is a sequence of edges e 1, e,...,e` where the tail of e j+1 equals the head of e j for 1 6 j <`. The length of a walk is the number of edges in the sequence, and each vertex corresponds to a trivial walk of length 0. For example, the directed graph T has a unique path of length from the second vertex to first vertex. How are the following matrix products related to paths? Proposition. If A G is the adjacency matrix of a directed graph G and ` N, then the (j, k)-entry of A`G is the number of walks with length ` from the kth vertex to the jth vertex. Proof. We proceed by induction on `. When ` = 0, we have A 0 G = I and the vertices correspond to the walks of length 0, which establishes the base case of the induction. Suppose that `>0, so A`G = A` 1 G A G. Let n be the number of vertices in G. If a j,k denotes the (j, k)-entry in A G for all 1 6 j, k 6 n, then the definition of the adjacency matrix guarantees that a j,k equals the number of walks of length 1 in G from the kth vertex to the jth vertex. Similarly, if b i,j denotes the (i, j)-entry in A` 1 G for all 1 6 i, j 6 n, then the induction hypothesis implies that b i,j equals the number of walks of length ` 1 in G from the jth vertex to the kth vertex. By concatenating these walks at the jth vertex, we see that there are b i,j a j,k walks of length ` in G from the kth vertex to the ith vertex passing through the jth vertex at second step. By summing over all possible choices for the second vertex, we see that there are b i,1 a 1,k + b i, a,k + + b i,n a n,k walks of length ` in G from the kth vertex to the ith vertex. From the dot-product reinterpretation of matrix multiplication, we recognize this sum as the (i, k)-entry in the product A` 1 G A G = A`G. T Figure 7.0: A small directed graph H Figure 7.1: A larger directed graph " # A T = A T = " # 1 0 A H = A H = A H = A 5 H =

6 9 linear algebra P 1 5 Figure 7.: A bipartite directed graph Q Figure 7.: Another bipartite graph Figure 7.: Merging two bipartite graphs B Q B P = " 1 # How can we visualize matrix multiplication for arbitrary matrices? A bipartite directed graph is a directed graph such that no vertex is both a tail and head. In other words, the set of vertices that are tails for some edge is disjoint from the set of vertices that are heads. For a bipartite directed graph, the biadjacency matrix B G is a matrix in which the columns correspond to the vertices that are tails of some edge and the rows correspond to the vertices that are heads of some edge. The (j, k)-entry of B G is the number of edges in G from the kth vertex to the jth vertex. The biadjacency matrices for our examples are B P = B Q = From the definition of the biadjacency matrix, we see that every matrix with entries in {0, 1} is the biadjacency matrix of a unique bipartite directed graph Proposition. Let G and H be bipartite directed graphs such that the heads in G corresponds the tails in H. If B G and B H are the associated biadjacency matrices, then the (i, k)-entry of the product B H B G is the number of walks from the k-th tail in G to the i-th head in H in the directed graph obtained by merging the heads in G with the tails in H. Proof. If a j,k denote the (j, k)-entry in the adjacency matrix B G for all 1 6 j 6 m and all 1 6 k 6 n, then the definition of the biadjacency matrix ensures that a j,k equals the number of walks of length 1 in G from the kth vertex to the jth vertex. Similarly, if b i,j denote the (i, j)-entry in the adjacency matrix B H for all 1 6 i 6 ` and all 1 6 j 6 m, then the definition of the biadjacency matrix ensures that b i,j equals the number of walks of length 1 in H from the jth vertex to the kth vertex. By concatenating these walks at the jth vertex, we see that there are b i,j a j,k walks of length in the merged graph from the kth vertex the ith vertex passing through the jth vertex. Summing over all possible choices for the intermediate vertex, we see that there are b i,1 a 1,k + b i, a,k + + b i,m a m,k walks of length in the merged graph from the kth vertex to the ith vertex. From the dot-product reinterpretation of matrix multiplication, we recognize this sum as the (i, k)-entry in the product B H B G. To create an equivalence between directed graphs and matrices, one normally considers weighted graphs. A weighted directed graph is a directed graph such that each edge (k, j) is labelled by an element of a j,k K. In the weighted adjacency matrix A G or the weighted biadjacency matrix B G, the (j, k)-entry equals the label a j,k K. It follows that every square matrix is the weighted adjacency matrix of some weighted directed graph, and every matrix is the weighted biadjacency matrix of some weighted bipartite directed graph.

7 matrix operations 95 Exercises 7.1. Problem. Determine which of the following statements are true. If a statement is false, then provide a counterexample. i. The adjacency matrix of a directed graph is a square matrix. ii. The adjacency matrix of a directed graph is always symmetric. iii. The adjacency matrix of a nonempty directed graph is never skewsymmetric. iv. A biadjacency matrix can have any size Problem. A complete bipartite directed graph is a graph whose vertices can be partitioned into two subsets and such that no edge has both endpoints in the same subset, and every possible edge that could connect vertices in different subsets is part of the graph. Describe the biadjacency matrix of a complete bipartite directed graph.

8 96 linear algebra

9 matrix operations Invertible Matrices Do anymatriceshaveamultiplicativeinverse? Let A be a square matrix. If there is a matrix B such that AB = I and BA = I, then B is an inverse of A and we write B = A 1. When there exists a inverse, we say that A is invertible. For example, we have = 0 1 and = , so both of these matrices are invertible Problem. Show that 6 1 is not invertible. Solution. Assuming that wy xy were the inverse, we would have I = 6 1 w y w + 6x y + 6z x z = w + x y+ z () 8 9 w + 6x = 1 >< >= w + x = 0 >: y + 6z = 0>;. y + z = 1 Elementary row operations give R 1 R 7! R R R 7! R ! R 1 7! R R 7! R 1 R 7! R R 7! R! R + R 7! R! Since the linear system is inconsistent, no inverse exists Lemma. If A is an invertible matrix, then its inverse is unique. Proof. If B and C are both inverses of the matrix A then the definition of an inverse matrix and the properties of matrix multiplication give C = CI = C(AB) =(CA)B = IB = B. 7.. Proposition. If A is an invertible matrix, then linear system A~x = ~ b has a unique solution given by ~x = A 1 ~ b. Proof. To establish existence, we have A(A 1 ~ b)=(aa 1 )~ b = I ~ b = ~ b, so A 1 ~ b is a solution. On the other hand, for any solution ~v, we have ~v = I~v =(A 1 A)~v = A 1 (A~v) =A 1 ~ b which proves uniqueness. 7.. Problem. Consider the matrix A := ab cd where a, b, c, d K. If ad bc 6= 0, then show that A is invertible where A 1 = ad 1 d b bc c a. If ad bc = 0 then prove that A is not invertible. For the matrix A := ab cd, the expression ad bc is called the determinant of A and written det(a) =ad bc. Solution. If ad a b c d bc 6= 0, then we have d b ad bc ab + ba c a = cd dc cb + da =(ad bc) I,

10 98 linear algebra which proves that A 1 = ad 1 d b bc c a. Next, suppose that ad bc = 0. If a 6= 0, then we would have d = bc a and a b a b a b A = c d = ac/a bc/a = ka kb, where k = a c. If wx y z were an inverse of A, then we would obtain a I = b ka kb w x y z aw + by = ax + bz k(aw + by) k(ax + bz). a 0 b a 0 b 07 ka 0 kb ka 0 kb 1 R k R 1 7! R R k R 7! R! Applying elementary row operations to the associated augmented matrix yields a 1 R 1 7! R 1 a a 0 b 0 1 R 1 R 7! R 1 a 0 b 00 1 R 7! R R 7! R 10 b a 60 a 0 b 07 R + k R 7! R 60 a 0 b 07 R 0000 k 5! 7! R ! 6010 b a Since the linear system is inconsistent, we deduce that the matrix A does not have an inverse in this case. On the other hand, if a = 0, then we have bc = 0 which implies that either b = 0 or c = 0. Thus, A has or 0 b 0 d neither of which has an inverse, because no the form 00 cd matrix with a zero row or zero column has a inverse. 7.. Proposition (Properties of invertible matrices). Let A and B be invertible matrices of the same size. If 0 6= c K and k N, then we have the following: (involution) (A 1 ) 1 = A (compatibility with matrix multiplication) (AB) 1 = B 1 A 1 (compatibility with scalar multiplication) (c A) 1 = 1 c A 1 (compatibility with transpose) (A T ) 1 =(A 1 ) T (compatibility with powers) (A k ) 1 =(A 1 ) k Proof. Since AA 1 = I = A 1 A, the uniqueness of the inverse for the matrix A 1 implies that A =(A 1 ) 1. Since the properties of matrix multiplication give (B 1 A 1 )(AB) =B 1 (A 1 A)B = B 1 IB = B 1 B = I (AB)(B 1 A 1 )=A 1 (B 1 B)A = A 1 A = I, the uniqueness of the inverse establishes that (AB) 1 = B 1 A 1. As 1 c A 1 (c A) = c c (A 1 A)=I and (c A) 1 c A 1 = c c (AA 1 )=I, the uniqueness of the inverse shows that (c A) 1 = 1 c A 1. The properties of the transpose give I =(AA 1 ) T =(A 1 ) T A T and I =(A 1 A) T = A T (A 1 ) T, so we deduce that (A T ) 1 =(A 1 ) T. We proceed by induction on k. For the base case when k = 0, we have (A 0 ) 1 = I 1 = I =(A 1 ) 0. Suppose k > 0. The compatibility with matrix multiplication and the induction hypothesis give (A k ) 1 =(AA k 1 ) 1 =(A k 1 ) 1 A 1 =(A 1 ) k 1 A 1 =(A 1 ) k.

11 matrix operations 99 Exercises 7..5 Problem. Determine which of the following statements are true. If a statement is false, then provide a counterexample. i. Every matrix has a multiplicative inverse. ii. Every square matrix has a multiplicative inverse. iii. The identity matrix is equal to its own inverse. iv. The inverse of a sum of invertible matrices equals the sum of there inverses Problem. Let P be an invertible (m m)-matrix, let Q be an invertible (n n)-matrix, let U be an (m n)-matrix, and let V be an (n m)-matrix. Directly verify the Woodbury matrix identity (P + UQV) 1 = P 1 P 1 U(Q 1 + VP 1 U) 1 VP Problem. Let A be an invertible (m m)-matrix, let B be an (m n)-matrix, let C be an (n m)-matrix, and let D be an (n n)- matrix. If the Schur complement S := D CA 1 B is invertible, then establish the blockwise inversion formula: " # 1 " # A B A = 1 + A 1 BS 1 CA 1 A 1 BS 1 C D S 1 CA 1 S Problem. Consider the matrix M := i. Verify that M 5M + 9M I = 0. ii. If N := 1 (M 5M + 9I), then prove that N = M 1. iii. Explain how N in part i can be obtained from the equation in part ii.