ICS 6N Computational Linear Algebra Matrix Algebra Xiaohui Xie University of California, Irvine xhx@uci.edu February 2, 2017 Xiaohui Xie (UCI) ICS 6N February 2, 2017 1 / 24
Matrix Consider an m n matrix a 11 a 12... a 1n a 21 a 22... a 1n A =...... = [ ] a 1 a 2... a n a m1 a m2... a mn a ij is the scalar entry in the ith row and jth column, called the (i, j)-entry. Each column is a vector in R m. Two matrices are equal if they have the same size and the corresponding entries are equal a 11, a 22,... are called the diagonal entries A is called diagonal if all non-diagonal entries are zero The identity matrix I n is a square diagonal matrix with diagonal being 1 The zero matrix is a matrix in which all entries are zero, written as 0. Xiaohui Xie (UCI) ICS 6N February 2, 2017 2 / 24
Matrix operations Given two m n matrices A and B, Sum: A + B is an m n matrix whose (i, j)-entry is a ij + b ij Multiplication by a scalar: ra = Ar is an m n matrix whose (i, j)-entry is ra ij, where r is a scalar. Matrix vector product: Ax = x 1 a 1 + x 2 a 2 +... + x n a n Xiaohui Xie (UCI) ICS 6N February 2, 2017 3 / 24
Examples Given A = A + B 2A A + C [ ] 1 2 3, B = 4 5 6 [ ] 1 0 1, C = 0 1 1 [ ] 1 0, compute 0 1 Xiaohui Xie (UCI) ICS 6N February 2, 2017 4 / 24
Examples [ ] 1 2 3 Given A =, B = 4 5 6 [ ] 2 2 4 A + B = 0 6 7 [ ] 2 4 5 2A = 8 10 12 A + C = Not defined. [ ] 1 0 1, C = 0 1 1 [ ] 1 0, 0 1 Xiaohui Xie (UCI) ICS 6N February 2, 2017 5 / 24
Properties of matrix operations Given A, B, C matrices of the same size, and scalars r and s, (A + B) + C = A + (B + C) A + B = B + A A + 0 = A r(a + B) = ra + rb (r + s)a = ra + sa r(sa) = (rs)a Xiaohui Xie (UCI) ICS 6N February 2, 2017 6 / 24
Matrix Multiplication When a matrix B multiplies a vector x, it transforms x into the vector Bx. If this vector is then multiplied in turn by a matrix A, the resulting vector is A(Bx). Thus A(Bx) is produced from x by a composition of mappings the linear transformations. Our goal is to represent this composite mapping as multiplication by a single matrix, denoted by AB, so that A(Bx) = (AB)x. Xiaohui Xie (UCI) ICS 6N February 2, 2017 7 / 24
Matrix Multiplication Suppose A is m n, B is n p, and x is in R p Denote B = [ b 1 b 2... b p ]. Xiaohui Xie (UCI) ICS 6N February 2, 2017 8 / 24
Matrix Multiplication Suppose A is m n, B is n p, and x is in R p Bx is a vector in R n, A(Bx) is a vector in R m Denote B = [ b 1 b 2... b p ]. Then Bx = x1 b 1 + x 2 b 2 +... + x p b p A(Bx) = A(x 1 b 1 + x 2 b 2 +... + x p b p ) = A(x 1 b 1 ) + A(x 2 b 2 ) +... + A(x p b p ) = x 1 (Ab 1 ) + x 2 (Ab 2 ) +... + x p (Ab p ) linear combination = [ Ab 1 Ab 2 Ab p ] x So AB = [ Ab 1 Ab 2 Ab p ], an m p matrix. Xiaohui Xie (UCI) ICS 6N February 2, 2017 9 / 24
MATRIX MULTIPLICATION Definition: If A is an m n matrix, and if B is an n p matrix with columns b 1,, b p, then the product AB is the m p matrix whose columns are Ab 1,, Ab p. That is AB = [ Ab 1 Ab 2 Ab p ] Each column of AB is a linear combination of the columns of A using weights from the corresponding column of B. Xiaohui Xie (UCI) ICS 6N February 2, 2017 10 / 24
Example Given A = [ ] 1 1 1 2 1 and B = 1 1, compute AB. 2 1 1 0 0 Xiaohui Xie (UCI) ICS 6N February 2, 2017 11 / 24
Example [ ] 1 1 1 2 1 Given A = and B = 1 1, compute AB. 2 1 1 0 0 Solution: AB = Ab 1 Ab 2 [ ] 1 [ ] [ ] 1 2 1 Ab 1 = 1 1 1 2 1 =, Ab 2 1 1 1 2 = 1 1 = 2 1 1 0 0 [ ] 1 3 So AB = 1 3 [ ] 3 3 Xiaohui Xie (UCI) ICS 6N February 2, 2017 12 / 24
Row-column rule for computing AB Now let s check the (i, j)-entry of AB: (AB) ij = the i-th entry of the j-th column = the i-th entry of Ab j = b j (the i-th row of A) = a i1 b 1j + a i2 b 2j +... + a in b nj The (i, j)-entry of AB is the sum of the products of corresponding entries from row i of A and column j of B (AB) ij = a i1 b 1j + a i2 b 2j +... + a in b nj = n a ik b kj k=1 Xiaohui Xie (UCI) ICS 6N February 2, 2017 13 / 24
Example Given A = [ ] 1 1 1 2 3, B = 1 0, compute AB. 4 5 6 0 1 Xiaohui Xie (UCI) ICS 6N February 2, 2017 14 / 24
Example Given A = Solution: [ ] 1 1 1 2 3, B = 1 0, compute AB. 4 5 6 0 1 Are the sizes consistent? Yes Based on definition, AB = Ab 1 Ab 2 = Or we can also calculate this entry by entry (AB) 11 = [ 1 2 3 ] [ 1 1 0 ] = 3 (AB) 21 = [ 4 5 6 ] [ 1 1 0 ] = 9 And so[ on until ] we get 3 2 AB = 9 2 [ ] 3 2 9 2 Xiaohui Xie (UCI) ICS 6N February 2, 2017 15 / 24
Special Cases An nx1 matrix can be viewed as a vector in R n (column vector) A row vector can be viewed as a 1xn matrix. (Dot product) A row vector times a column vector produces a scalar if they are of the same size. b 1 [ ] b 2 a1 a 2... a n. = a 1b 1 + a 2 b 2 +... + a n b n b n Xiaohui Xie (UCI) ICS 6N February 2, 2017 16 / 24
Special Cases (Out product) A column vector times a row vector produces a matrix. a 1 a 1 b 1 a 1 b 2... a 1 b n a 2. [ ] a 2 b 1 a 2 b 2... a 2 b n b 1 b 2... b n =...... a m a m b 1 a m b 2... a m b n Xiaohui Xie (UCI) ICS 6N February 2, 2017 17 / 24
Special cases Let A be an mxn matrix, AI n = A = I m A A0 = 0. Xiaohui Xie (UCI) ICS 6N February 2, 2017 18 / 24
Theorems If the sizes are consistent a) (AB)C = A(BC) b) A(B + C) = AB + AC c) (B + C)A = BA + BC d) (ra)b = A(rB) e) I m A = AI n Xiaohui Xie (UCI) ICS 6N February 2, 2017 19 / 24
Warnings AB BA in general. They are not even of the same size! Example Even[ if they ] are the [ same ] size it is in general not true 1 1 0 1 A =, B = 0 0 1 0 [ ] [ ] 1 1 0 0 AB =, BA = 0 0 1 1 If AB = BA then A and B are commutable, but in general they are not. Xiaohui Xie (UCI) ICS 6N February 2, 2017 20 / 24
Warnings If AB = AC and A 0, we cannot conclude B = C Example Even[ if they ] are the [ same ] size it is[ in general ] not true 1 1 1 2 3 4 A =, B =, C = 1 1 1 2 3 4 [ ] [ ] 0 0 0 0 AB =, BA = 0 0 0 0 But we can clearly see B C Xiaohui Xie (UCI) ICS 6N February 2, 2017 21 / 24
Powers of a matrix Definition: If A is an n n matrix and if k is a positive integer, then A k denotes the product of k copies of A. A k = A A (k times) A 0 = I by convention. Xiaohui Xie (UCI) ICS 6N February 2, 2017 22 / 24
Transpose Given an mxn matrix A, the transpose of A is the nxm matrix, denoted by A T, whose columns are formed from the corresponding rows of A. a 11 a 12... a 1n a 11 a 21... a n1 a 21 a 22... a 2n If A =......, then a 12 a 22... a n2 AT =...... a m1 a m2... a mn a 1m a 2m... a nm (A T ) ij = a ji Xiaohui Xie (UCI) ICS 6N February 2, 2017 23 / 24
Properties of matrix transpose If the sizes are consistent (A T ) T = A (A + B) T = A T + B T (ra) T = ra T (ra)b = A(rB) (AB) T = B T A T (note the reverse order!) Xiaohui Xie (UCI) ICS 6N February 2, 2017 24 / 24