Lecture 4 Matrices University of British Columbia, Vancouver Yue-Xian Li February 27
Def: An m n matrix is a 2D array of scalars having m rows and n columns and enclosed between a pair of either square or round brackets A a a 2 a n a 2 a 22 a 2n a m a m2 a mn m n [a ij, ( i m, j n) A matrix is usually represented by a capital letter in printed text and handwriting A matrix can also be defined by its entries in the form A [a ij, ( i m, j n), where a ij is the entry located at the i th row and j th column A row vector with n components is a n matrix A column vector with m components is a m matrix A scalar is a matrix All rules derived for matrices should all apply to vectors and scalars 2
4 Matrix operations 4 Equality between matrices A, B A B iff { (a) A, B have the same size; (b) a ij b ij for all i, j Thus, A B iff they have the same size and all corresponding entries of the two are identical Eg 4 Determine A what values of x, y [ 2 4 2 B [ 2 x 2 + y x for Ans: { x 2 + y 4, x 2 { x 2, y 3 3
42 Scalar multiplication αa α[a ij [αa ij, for any scalar α Thus, when a matrix is multiplied a scalar, each entry of the matrix is multiplied by the scalar Eg 42 x t 2t 4t t 2 4 ; 2 [ 3 2 5 [ 2 6 4 4
43 Matrix addition/subtraction For two matrices A, B: C A ± B is defined only if A, B have the same size and c ij a ij ± b ij, for all i, j Thus, adding A to B is to add each corresponding entries between the two Eg 43 Let A [ 2 3 4, B [ 2 2 3 A + B [ + ( 2) 2 + 3 + 2 4 + ( 3) [ 3 5, A B [ ( 2) 2 3 2 4 ( 3) [ 3 7 5
44 Matrix multiplication Let A m l, B k n be of sizes m l, k n, respectively, then is defined if l k and that C m n A m l B k n (i) Each entry in C is a dot product between a row in A and a column in B c ij (i th row of A) (j th column of B) l a ip b pj a i b j + a i2 b 2j + + a il b lj p (ii) This size of C is m n C m n A m l B l n, c ij l a ip b pj p c c l c n c i c ij c in c m c ml c mn a a l a k a i a il a ik a m a ml a mk b b j b n b i b lj a in b k b kj b kn 6
Eg : If A [ 2 3 2 2, B [ 2 3 2, then C AB [ 2 3 [ 2 3 [ ( 2)(2) + (3)( 3) ()(2) + ()( 3) [ 3 2 2 But BA [ 2 3 2 [ 2 3 2 2 is not defined! Usually, AB BA Therefore, matrix multiplication is not commutative C 2 A 2 2 B 2 [ c c 2 [ 2 c [ 2 3 3 (st row of A) (st column of B); [ 2 c 2 [ 3 (2st row of A) (st column of B) 7
Matrix multiplication is not commutative! Eg : If A 2 3 5 3 2, B [ 2 3 2 2, then C AB 2 3 5 [ 2 3 Also, BA [ 2 3 2 2 + 3 2 3 + 3 2 + 3 + 2 + 5 3 + 5 2 2 2 3 5 3 2 is not defined! 7 6 2 3 3 3 3 2 [ [ [ [ 2 3 2 3 Eg : Let A, B, AB 4 4 [ [ [ [ 2 3 2 3 BA 4 4 Even if AB, BA both are defined, usually AB BA 8
Eg 44 Write the linear system in matrix form x + 2x 2 + 3x 3, x 2x 2 + 3x 3 2, 2x + x 2 x 3 3 Ans: n x, n 2 x 2, n 3 x 3, n [ 2 3, n 2 [ 2 3, n 3 [2, x x x 2 x 3, n n 2 n 3 x 2 3 2 3 2 3 2 3 3 x x 2 x 3 3 2 3 3 A x b where A 2 3 2 3 2, x x x 2 x 3, b 2 3 9
Therefore, any linear system can be written in matrix form as follows a x + a 2 x 2 + + a n x n b, a 2 x + a 22 x 2 + + a 2n x n b 2, a m x + a m2 x 2 + + a mn x n b m, where A a a n a m a mn, x x x m A x b,, b b b m If m > n, there are more equations (m) than the numb of unknowns (n) If m < n, there are less equations (m) than the numb of unknowns (n) If m n, there are the same number of equations and unknowns A is a square matrix Solving linear systems of the form A x b is one important topic of matrix algebra
When A n n is a square matrix, there is a strong resemblance between ax b and A x b Cases ax b A x b Unique If a, If A, solution x a b x A b, A is inverse of A Infinitely many If a b, If A, Rank[ARank[A b, solutions x t, t R x x h + x p NO If a, b, If A, Rank[A<Rank[A b, solution none none
Basic properties of matrix operations: Let A, B, C be matrices, and α, β be scalars A + B B + A 2 A + (B + C) (A + B) + C 3 α(a + B) αa + αb 4 (α + β)a αa + βa 5 (αβ)a α(βa) β(αa) 6 A A 7 A + O A, where O is a matrix with all zero entries 8 A + ( A) A A O 9 A(B + C) AB + AC (A + B)C AC + BC A(BC) (AB)C 2 α(ab) (αa)b A(αB) 3 If A is square, then A n A } {{ A}, (n integer) n times 4 If A, I are square, AI IA A, I is the identity matrix with in diagonal entries and everywhere else 2
Eg 45 Let A [ a A 2 2 b 2 c 2 d 2? [ a b c d For what values of a, b, c, d, Ans: A 2 [ a b c d [ a b c d [ a 2 + bc (a + d)b (a + d)c d 2 + bc Thus, bc which gives rise to 3 possible cases b, c, a + d c; b, c, a + d b; b c, [ 2 [ a a 2 a + d d (a + d) 2 d [ 2 2 [ a a + d a 2 (a + d) 2 d d [ 2 2 [ a a 2 d d 2 Square diagonal matrices behave like scalars: [ 2 [ [ a a 2 n [ a a (i) d d 2, n d d n (ii) [ a d [ α β [ aα bβ [ α β Thus, AB BA for diagonal matrices! 3 [ a d
45 Matrix as a row of column vectors An m n matrix can be expressed as a row of n column vectors Rules of matrix multiplication hold for such expressions provided that the operation is defined In the following text, no proof will be provided Instead, examples will be provided to clarify the points AEg : [ A [a a 2, where a [, a 2 [ Thus, if defined, one can do the following A [ 2 3 [a a 2 [ 2 3 2a +3a 2 2 [ [ +3 [ 5 3 Thus, A x is a LC of the columns of A Let s us verify the result using normal matrix multiplication: A [ 2 3 [ [ 2 3 [ ()(2) + ()(3) ()(2) + ()(3) [ 5 3 4
AEg 2: If A is identical as in AEg and B [ 2 2 [b b 2, where b [, b 2 [ 2 2, then, if defined, one can do the multiplication as follows BA B [a[ a 2 [Ba [ Ba 2 [[b b 2 a [b b[ 2 a 2 3 [[b b 2 [b b 2 [b b + b 2 3 Notice that the first column of BA is equal to b while the second is equal to b + b 2 Let s us verify the result using normal matrix multiplication: BA [ 2 2 [ [ 3 3 A x a column vector that is a LC of columns of A If the operation is defined, BA B [a a 2 [Ba Ba 2 BA a matrix whose columns are LCs of columns of B Therefore, columns of the matrix product T n T n T are LCs of columns of the leftmost (last) matrix T n 5
Notice also when matrices are expressed in such a form some multiplications that are not defined appear doable but leads to ridiculous results Thus, results discussed here apply only to those multiplications that are defined in normal matrix operations [ 2 [a a 2 2 [ 2 [ 2 2 is not defined! But [ 2 2 [3 4 2 [ 3 4 6 8 2 2 is defined! 6
Summary: () m n matrix A, one can express it as a row of column vectors A [a a 2 a n, a j R m ( j n) (2) Let x [x x 2 x n T be a column vector in R n, then x Ax [a a 2 a n x 2 x a + x 2 a 2 + + x n a n x n is a linear combination (LC) of column vectors of A (3) Let B be a l m matrix, then BA B[a a 2 a n [Ba Ba 2 Ba n Since Ba j ( j n) is a linear combination of columns of B, each column of BA is a linear combination of columns of B 7
46 Matrix as a column of row vectors An m n matrix can be expressed as a column of m row vectors Rules of matrix multiplication hold for such expressions provided that the operation is defined AEg3: A [ [ a a 2, where a [, a 2 [ Then, if defined, [3 4 A [3 4 [ a a 2 3a + 4a 2 3 [ + 4 [ [3 7 Thus, xa is a LC of the rows of A Let s us verify the result using normal matrix multiplication: [3 4 A [3 4 [ [(3)() + (4)() (3)() + (4)() [3 7 8
[ AEg4: If A is identical as in AEg3 and B [ 3 3 b, where b [, b 2 [3 3, then, if defined b 2 AB [ a a 2 [ b + b 2 b 2 B [ a B a 2 B [ 4 4 3 3 a [ b b 2 a 2 [ b b 2 [ [ [ b [ b 2 b Notice that the first row of AB is equal to b + b 2 while the second is equal to b 2 Let s us verify the result using normal matrix multiplication: AB [ [ 3 3 [ 4 4 3 3 xa a row vector that is a LC of rows of A [ a If the operation is defined, AB B a 2 [ a B a 2 B AB a matrix whose rows are LCs of rows of B b 2 Therefore, rows of the matrix product T n T n T are linear combinations of rows of the rightmost (first) matrix T 9
Notice also when matrices are expressed in such a form some multiplications that are not defined appear doable but leads to ridiculous results Thus, results discussed here apply only to those multiplications that are defined in normal matrix operations [ a a 2 2 [3 4 2 [ 2 2 [3 4 2 is not defined! But [ 2 [3 4 2 [ 3 4 3 4 2 2 is defined! 2
Summary: () m n matrix A, one can express it as a column of row vectors a A a 2, at i R n ( i m) a m (2) Let x [x x 2 x m be a row vector in R m, then a xa [x x 2 x m a 2 x a +x 2 a 2 + +x m a m a m is a linear combination (LC) of row vectors of A (3) Let B be a n l matrix, then a AB a 2 B a m a B a 2 B a m B Since a i B ( i m) is a linear combination of rows of B, each row of AB is a linear combination of rows of B 2
42 Linear transformation Def: A transformation or map or function is a rule that assigns one output to each input Eg: Matrix A m n defines a transformation from R n to R m This is because for each x R n, when treated as an input, A x y R m is the output assigned to input x Thus, A defines a transformation from R n to R m : A : R n R m Similar to functions learned in calculus, some transformations are linear but others are not linear Similarly, some transformations are one-to-one, ie for each output y there exists one and only one input x that is mapped onto it Some are not Eg: Rotating each vector in R 2 counter-clockwise (CC) by 9 is both linear and one-to-one from R 2 to R 2 : T R 9 : R 2 R 2 This is because rotation keeps any pgram unchanged before and after the transformation One and only one vector is rotated into each vector 22
Eg: Function y x 2 (y T (x) : R R) is a transformation that is neither linear nor one-to-one This is because x, x 2 2x 2 are transformed into y, y 2 4 2y Or, simply x 2 is nonlinear since it is not a scalar multiple of x Also, both x and x 3 are mapped into the same number y, indicating that it is not one-to-one 23
Def of linearity: A transformation T : R n R m is linear iff T (α x + β x 2 ) αt ( x ) + βt ( x 2 ) OR T ( x + x 2 ) T ( x ) + T ( x 2 ) and T (α x) αt ( x) T is linear iff the transformation of a LC of two inputs (ie T (αx + βy)) is identical to the LC of the outputs of the two (ie αt (x) + βt (y)) Geometrically speaking, one can treat x + y as the diagonal of the pgram with sides x and y After transformation, it is the same as the diagonal of the pgram with sides T ( x) and T ( y) Thus, the output of the diagonal of the pgram with sides x and y is the same as the diagonal of the pgram with sides T ( x) and T ( y) Also, the output of a scalar multiple of an input (ie T (α x)) is identical to the scalar multiple of the output of x (ie αt ( x)) Matrix multiplication always defines a linear transformation Also, any linear transformation between vector 24
spaces of finite dimensions can be expressed as a matrix multiplication 25
Rotation of vectors in R 2 is linear: T( x + y ) T( x ) y x + y 2T( y ) 2 y T( x ) x T( y ) x Counter clockwise rotation by 9 Proof: Matrix multiplication is linear Let A be an m n matrix, x, y R n, α, β be scalars A(α x + β y) A(α x) + A(β y) α(a x) + β(a y) 26
Eg 42 Show that [ (a) matrix multiplication by M T defines a transformation that rotates vectors in R 2 by 9 CC (b) The transformation is linear Ans: (a) One only needs to show that it rotates the basis vectors CC by 9 The basis vectors in R 2 are [ [ e, e 2 [ [ [ M T e e 2 e rotated CC 9 [ [ [ M T e 2 e e 2 rotated CC 9 Alternatively, let v [v v 2 T, M T v [ [ v v 2 [ v2 v It is easy to check that [ v 2 v T is [v v 2 T rotated CC by 9 27
(b) M T (α x +β x 2 ) M T (α x )+M T (β x 2 ) α(m T x )+β(m T x 2 ) 28
Theorem 42: In vector spaces of finite dimensions, linear transformation (LT) matrix multiplication (MM) (are equivalent to each other) Thus, (i) Given any matrix M m n, it defines a LT from R n to R m (ii) Given any LT : R n R m, one can always find a corresponding matrix M T that carries out the transformation, T ( x) M T x, for all x R n Proof of (i): M : R n R m is a transformation since for all x R n as a input, there always exists one output y M x R m Since matrix multiplication is linear, multiplication by matrix M is a linear transformation Proof of (ii) will come later MM is a linear transformation due to basic properties of matrix multiplication Any LT between vector spaces of finite dimensions can be realized through MM Main question: Given a LT, how to find the matrix that does the job? 29
Eg 422 Show that in R 2 (a) Proj φ ( x), ie the projection of a vector onto a line at an angle φ wrt the x-axis, is a linear transformation (b) Find the matrix, M Projφ such that Proj φ ( x) M Projφ x Proj ( e ) sin φ u φ 2 Proj φ ( ) cosφ u e sin φ φ cosφ u e 2 φ φ e Using projection formula Geometric projection of basis Ans: [ cos φ (a) Let u be the unit vector of the direction of sin φ the line For all x, x 2 R 2 and scalars α, β R, Proj φ (α x + β x 2 ) def of proj [(α x + β x 2 ) u u (α x u + β x 2 u) u α( x u) u + β( x 2 u) u αproj φ ( x ) + βproj φ ( x 2 ) 3
(b) x [ x x 2 x e +x 2 e 2 x [ +x 2 [, thus Proj φ ( x) Proj φ (x e +x 2 e 2 ) x Proj φ ( e )+x 2 Proj φ ( e 2 ) [ Proj φ ( e ) Proj φ ( e 2 ) [ x M x Projφ x, 2 where M Projφ [ Proj φ ( e ) Proj φ ( e 2 ) is a matrix expressed as a row of column vectors Therefore, the projection matrix is a matrix whose column vectors are outputs/images of the basis vectors, Proj φ ( e ) and Proj φ ( e 2 ) If we find the images of the basis using any method, we can construct the matrix by putting them in the columns of the matrix Theorem 422 and (ii) of 42: For any LT, T : R n R m, there exists a matrix representation M T [T ( e ) T ( e 2 ) T ( e n ) whose columns are images of the basis vectors { e, e 2,, e n } of the R n 3
Proof: For each x x x n x e + +x n e n R n, T ( x) T (x e + + x n e n ) x T ( e ) + + x n T ( e n ) x [T ( e ) T ( e n ) M T x, x n where M T [T ( e ) T ( e n ) is a matrix whose columns are T ( e ),, T ( e n ) Therefore, finding the matrix M T is reduced to finding the images of the basis vectors For the projection problem, one can use two methods to calculate T ( e ),, T ( e n ) Method : Use the projection formula T ( e ) Proj φ ( e ) ( e u) u cos φ [ cos φ sin φ [ ([ cos 2 φ cos φ sin φ [ cos φ sin φ ) [ cos φ sin φ 32
([ [ cos φ T ( e 2 ) Proj φ ( e 2 ) ( e 2 u) u sin φ [ [ cos φ cos φ sin φ sin φ sin φ sin 2 φ Thus, ) [ cos φ sin φ M Projφ [ Proj φ ( e ) Proj φ ( e 2 ) [ cos 2 φ cos φ sin φ cos φ sin φ sin 2 φ Method 2: Graphic/geometric determination (preferred if easily doable!) Proj ( e ) sin φ u φ 2 Proj φ ( ) cosφ u e sin φ φ cosφ u e 2 φ φ e Using projection formula Geometric projection of basis Based on the left graph above, T ( e ) Proj φ ( e ) cos φ u [ cos 2 φ cos φ sin φ 33
T ( e 2 ) Proj φ ( e 2 ) sin φ u [ cos φ sin φ sin 2 φ Therefore, M Projφ [ Proj φ ( e ) Proj φ ( e 2 ) [ cos 2 φ cos φ sin φ cos φ sin φ sin 2 φ Question: Is Proj φ one-to-one and invertible? Ans: No, it is not Why? 34
Eg 423 Find the matrix that rotates vectors in R 2 CC by an angle φ Ans: Based on Theorem 422 above, M Rotφ [Rot φ ( e ) Rot φ ( e 2 ) Rot ( ) φ e 2 φ e 2 φ Rot φ( e ) e According to the graph above, Rot φ ( e ) [ cos φ sin φ, Rot φ ( e 2 ) [ sin φ cos φ Therefore, [ cos φ sin φ M Rotφ [Rot φ ( e ) Rot φ ( e 2 ) sin φ cos φ M Rot9 [ cos 9 sin 9 sin 9 cos 9 35 [, same as in Eg 42
Question: Is Rot φ one-to-one and invertible? Ans: Yes, it is Why? Actually, one can easily find the inverse matrix M Rot φ M Rot φ [ cos( φ) sin( φ) sin( φ) cos( φ) [ cos φ sin φ sin φ cos φ Let us verify this: M Rotφ M Rot φ [ cos φ sin φ sin φ cos φ [ cos φ sin φ sin φ cos φ [ M Rot φ M Rotφ [ cos φ sin φ sin φ cos φ [ cos φ sin φ sin φ cos φ [ 36
Eg 424 Find the matrix that reflects vectors in R 2 in the line at an angle φ Ans: Based on Theorem 422 above, M Refφ [Ref φ ( e ) Ref φ ( e 2 ) x x x φ x Ref φ( x ) According to the graph above, Ref φ ( x) x +( x ) x x x ( x x ) 2 x x 2Proj φ ( x) x 2M Projφ x I x (2M Projφ I) x M Refφ x Therefore, [ M Refφ 2M Projφ I 2 cos 2 φ cos φ sin φ cos φ sin φ sin 2 φ [ 37
[ cos(2φ) + sin(2φ) sin(2φ) cos(2φ) + [ [ cos(2φ) sin(2φ) sin(2φ) cos(2φ) Question: Is Ref φ one-to-one and invertible? Ans: Yes, it is Why? Actually, it is obvious that the inverse matrix if itself Let us verify this: M Ref φ M refφ M refφ M refφ [ [ cos(2φ) sin(2φ) cos(2φ) sin(2φ) sin(2φ) cos(2φ) sin(2φ) cos(2φ) [ 38
Eg 425 Find the matrices that project and reflect a vector in R 3 in plane P defined by x + y + z n x x x P x Ans: Based on the graph above, Ref ( ) P x Projˆn ( x) x, Proj P ( x) x x x, Ref P ( x) x + ( x ) x 2 x Thus, one only needs to find the matrix for Projˆn, where ˆn is the unit vector of the normal direction of P : ˆn 3 According to Theorem 422 M Projˆn [Projˆn ( e ) Projˆn ( e 2 ) Projˆn ( e 3 ) 39
Using the projection formula Projˆn ( e ) Projˆn ( e 2 ) Projˆn ( e 3 ) ( e ˆn) ˆn 3 Thus, M Projˆn [Projˆn ( e ) Projˆn ( e 2 ) Projˆn ( e 3 ) 3 Therefore, M ProjP I M Projˆn 3 M RefP I 2M Projˆn 3 2 2 2 2 2 2 2 2 2 M RefP is one-to-one and invertible with M Ref P M RefP However, M ProjP and M Projˆn are not one-to-one and do not have an inverse 4
Eg 426 Show that (a) The set of all polynomials up to degree 3 are equivalent to vectors in R 4 (b) Differentiation of the these polynomials is a LT (c) Differentiation in (b) can be carried out by MM Find the matrix Ans: (a) Polynomials up to degree 3 are defined by P (x) p + p x + p 2 x 2 + p 3 x 3 p p p 2 p 3 Note that the basis vectors are defined as e + x + x 2 + x 3 e 2 + x + x 2 + x 3 4,, p R4
e 3 + x + x 2 + x 3 e 4 + x + x 2 + x 3, (b) Let D : R 4 R 4 be the differentiation operator Let p (x) p, p 2 (x) p 2 be two polynomials up to degree 3, α, β be scalars Then, D(α p + β p 2 ) (αp (x) + βp 2 (x)) αp (x) + βp 2(x) αd( p ) + βd( p 2 ) Thus, D is a linear transformation (c) Based on Theorem 422, where M D [D( e ) D( e 2 ) D( e 3 ) D( e 4 ), 42
D( e ) ( + x + x 2 + x 3 ) + + + D( e 2 ) ( + x + x 2 + x 3 ) + + + D( e 3 ) (+x+x 2 +x 3 ) +2x++ D( e 4 ) (+x+x 2 +x 3 ) ++3x 2 + Therefore, M D [D( e ) D( e 2 ) D( e 3 ) D( e 4 ) Let s verify the result For 2 3 2 3,,, 43
p P (x) p + p x + p 2 x 2 + p 3 x 3 D( p) P (x) p + 2p 2 x + 3p 3 x 2 + Now, using matrix multiplication D( p) M D p 2 3 p p p 2 p 3 p p p 2 p 3, p 2p 2 3p 3 p 2p 2 3p 3 Question: Is D one-to-one and invertible? Ans: No, because for each p p (p R is any scalar), D( p) Thus, infinitely many vectors are transformed to 44
Composition of linear transformations Def: Suppose one transformation, T : R n R m is applied to vectors in R n followed by another transformation, S : R m R k Then, the combined actions of the two is called a composite transformation of the two denoted by S(T ( x)) or S T ( x), where the order is of great importance, the right-most being the st one and left-most the last one Remarks: () If both S, T are linear, then the composite S(T ( x)) (S T ( x)) is also linear Proof: S(T (α x + β x 2 )) S(αT ( x ) + βt ( x 2 )) αs(t ( x )) + βs(t ( x 2 )) (2) If T ( x) M T x, S( y) M S y, then S(T ( x)) S T ( x) M S M T x, where the order is of crucial importance (c) T n T n T ( x) is the composition of a sequence of n transformations starting from T to T n, the right-most is alway the st one 45
AEg 42 A LT T : R 4 R 3 is as follows Find its matrix representation x T x 2 x x 2 x 3 x 2 x 3 x x 3 x 4 4 Ans: One only needs to turn the image vector into the product of a matrix to vector x T x x 2 x 3 x 4 x x 2 x 2 x 3 x 3 x 4 M T x x 2 x 3 x 4 M T x A more tedious (and totally unnecessary) way is to use the definition to find out T ( e ) T, T ( e 2 ) T, 46
T ( e 3 ) T, T ( e 4 ) T, thus, M T 47
AEg 422 Find the matrix representation of T given that ([ ) [ ([ ) [ 3 2 T, T 2 2 Ans: One needs to find T ( e ), T ( e 2 ) In order to do that, one needs to express e and e 2 as LCs of the two vectors whose images are known Notice that x [ 3 2 3 e + 2 e 2, x 2 [ 2 2 e + e 2 Thus, e 2 x 2 x, e 2 2 x 3 x 2 Therefore, T ( e ) 2T ( x 2 ) T ( x ) 2 T ( e 2 ) 2T ( x ) 3T ( x 2 ) 2 [ 2 [ 3 [ [ 2 [ 3 3 [ 5 4 ; M T [T ( e ) T ( e 2 ) [ 3 5 3 4 An alternative method would be 48
T [ 3 2 2 [ 2 T [ 2 [ 3 2 2 [ 2 [ 2 2 3 [ 3 5 3 4 This method works only when the original vectors are LI which is usually the case 49
Eg 427 In R 2, let Rπ 4 be the matrix that rotates vectors CC by π 4 and P π be the matrix that projects vectors into 4 the line at an angle π 4 wrt the x-axis Find the matrices of the composed action of the two Rπ 4 P π 4 and P πrπ 4 4 Ans: P π 4 [ Rπ 4 [ cos π 4 sin π 4 sin π 4 cos π 4 2 [ cos 2 ( π 4 ) cos( π 4 ) sin( π 4 ) cos( π 4 ) sin( π 4 ) sin2 ( π 4 ) 2 [ Therefore, Rπ 4 P π 4 2 2 P π 4 Rπ 4 2 2 [ [ Note that Rπ 4 P π 4 P π 4 Rπ 4 [ [ 2 [ 2 [ Question: Is the composite one-to-one and invertible? Ans: No, if any one of the two is not one-to-one and invertible, so wouldn t be the composite transformation 5
43 The transpose of a matrix AA T The transpose for each matrix A, denoted by A T, is obtained by turning its rows into columns and columns into rows Alternatively, A T is obtained by rotating the array of numbers around its diagonal entries, leaving the diagonal entries unchanged Eg : for A 2 3 5 3 2, B [ 2 3 2 2, then A T [ 2 3 5 2 3, B T [ 2 3 2 2 Def: Let A [a ij be an m n matrix Its transpose, denoted by A T of size n m, is obtained by turning its rows to columns and vice versa while maintaining their original orders, ie A T [a T ij, a T ij a ji It is obvious that a T ii a ii, thus the diagonal entries of the two are identical 5
Some properties of the transpose () (A T ) T A (2) (AB) T B T A T, notice that A m l B l n C m n, B T n la T l m C T n m (3) If x, y are two column vectors in R n, then x y x T y y T x (4) If x R n, y R m, A A m n Then, y (A x) (A T y) x Proof: y (A x) y T (A x) scalar scalar T ( y T (A x)) T x T A T y x (A T y) 52
Eg 43 (Taken from midterm exam in 28) Given that [ [ 5 3 [ 5 2 5 2 y, find the scalar α y 2 8 9 3 8 8 5 5 8 Ans: α α T [ 9 5 [9 5 T [ 2 y 5 3 2 8 5 8 T [ 9 5 5 3 2 8 ( y T ) T [9 5 5 8 8 + 5 23 T [ 5 2 5 3 8 8 y T y 5 3 2 8 5 8 [ 9 5 T For this problem, trying to solve y first is not practical One would be doing much lengthier calculations 53
44 Fundamental Theorem of Linear Algebra Results presented above can be summarized as follows () Any LT, T : R n R m can be realized through MM Conversely, MM by any matrix A m n defines a LT from R n to R m (2) Given an LT, T : R n R m, its matrix representation is given by A [T ( e ) T ( e 2 ) T ( e n ), whose columns are images of the orthonormal basis vectors in R n, e,, e n (3) If the LT, T : R n R m, is represented by A x y, then x R n defines the domain space, y R m defines the target space Fundamental Theorem of Linear Algebra elaborates the structure of the domain space R n and the target space R m for any given matrix A m n This material is not required material for Math 52, however, I put it here so that those who would like to know can get some information on this very important result in linear algebra 54
Fundamental Theorem of Linear Algebra: Given any m n matrix, it defines a LT from R n to R m Each space can be dived into two subspaces such that R n Ker(A) Row(A), R m Ker(A T ) Col(A), where is the direct sum, Ker(A), Row(A), Ker(A T ), Col(A) are the four fundamental subspaces of A defined as follows () Kernel or nullspace of A: Ker(A) { x R n : A x R m }, which contains all vectors in R n that are mapped/transformed into by A Also called the nullspace of A and denoted by N (A) (2) Rowspace of A or image of A T : Row(A) { x R n : x LCs of row vectors of A}, which contains all vectors in R n that are linear combinations of the row vectors of A Also called the range or image of A T and denoted by R(A T ) or I(A T ) (3) Kernel or nullpsace of A T : Ker(A T ) { y R m : A T y R n }, 55
which contains all vectors in R m that are mapped/transformed into by A T Also called the nullspace of A T and denoted by N (A T ) (4) Columnspace or image/range of A: Col(A) { x R m : x LCs of column vectors of A}, which contains all vectors in R m that are linear combinations of the column vectors of A Also called the range or image of A and denoted by R(A) or I(A) Now, let s clearly define the key mathematical terms that appeared above Def of subspace: If V, W are vector spaces with identical definitions of vector addition and scalar multiplication and that V W is a subset of W, then V is a subspace of W Def of direct sum : Let U, V be subspaces of W W is a direct sum of U, V, ie W U V, if W U + V and U V {} 56
Graphic illustration of the theorem R n A rrank(a) R m Row(A) x R dim r A dim r Col(A) A y C x +x K x R A Ker(A) x K dim n r dim m r Ker(A T) () A is one-to-one between Row(A) and Col(A), both are r dimensional, r Rank(A) (2) For all x K Ker(A), A x K Similarly, for all y Ker(A T ), A T y (3) All x R n can be expressed as x x R + x K, where x R Col(A), x K Ker(A) Thus A x A( x R + x K ) A x R + A x K A x R y C (4) Col(A) contains all possible images of A because A xlcs of column vectors of A 57
[ 2 2 Eg 44 Matrix Pπ 4 2 [ is a matrix that 2 2 projects all vectors in R 2 onto the line at an angle φ π 4 Find the four fundamental subspaces and determine if Pπ 4 is invertible Def: span{ v, v 2,, v n }the set of all possible vectors generated as a LC of the set of vectors { v, v 2,, v n } span{ v}the set of all points on the line that contain v Ans: Based on the graph below [ Row(Pπ 4 ) span{[ T } t, which is line L [ [ Col(Pπ 4 ) span{ } t, which is also line L L s(, ) Lt(,) x x x 58
Note that P T π 4 Pπ 4, thus [ Ker(A) Ker(A T ) span{ [ } s, which is line L Thus, for this example, Row(Pπ 4 ) Col(Pπ 4 ) which is line L, and Ker(A) Ker(A T ) which is line L Also, because Pπ 4 x x, Pπ 4 is one-to-one between Row(Pπ 4 ) and Col(Pπ 4 ) Because Ker(A) s [ { }, Pπ 4 is not invertible This is obvious since the two rows and two columns of Pπ 4 are identical which implies that its determinant must be zero 59
45 Application to random walks Eg 45 Consider a particle that jumps among 3 distinct states at predetermined probabilities (see figure) in discrete time steps Use matrix equation to describe the probability of finding it in each state after n (n, 2, ) time steps P P 3 P 2 P 3 P 2 P 33 3 P 23 P 32 2 P 22 To P ij Prob of jumping from state j to state i From Notices that: () P ij for all i, j 3 (2) P ij P j + P 2j + P 3j i Prob of finding it in of the 3 states is equal to irrespective of its location at the previous time step 6
Let x i (n) prob of being in state i at time n where i, 2, 3; n,, 2, As time evolves to n + : x i (n + ) prob of being in state i at time n + P i x (n) + P i2 x 2 (n) + P i3 x 3 (n) Let x(n) x(n+) P [P ij x (n) x 2 (n) x 3 (n) x (n + ) x 2 (n + ) x 3 (n + ) P P 2 P 3 P 2 P 22 P 23 P 3 P 32 P 33 P P 2 P 3 P 2 P 22 P 23 P 3 P 32 P 33 Then, P x (n) + P 2 x 2 (n) + P 3 x 3 (n) P 2 x (n) + P 22 x 2 (n) + P 23 x 3 (n) P 3 x (n) + P 32 x 2 (n) + P 33 x 3 (n) x (n) x 2 (n) x 3 (n) The sum of each column is equal to : P x(n) is the transition (Markov) matrix 6 3 P ij i
Therefore, random walk of a particle between m states (m 3 for this example) is described by the following matrix iteration equation: x(n + ) P x(n), (n,, 2, ) where n is usually chosen as the initial time when the process starts This equation is also called a difference equation or a recursive map It is also called a Markov process Markov process is a random process in which the future is independent of the past, given the present In other words, it is a process with only a one-step memory The present state only depends on the state of immediate past but independent of the states two or more steps in the past In most literature, a Markov matrix is often expressed in form of P T We use P to make it easier for beginners 62
Eg 452 Suppose P 3 4 2 2 3 4 2 5 5 2 5 is the Markov matrix for a 3-state random walk Note that the sum of each column is equal to and that it is not positive since there exists a zero entry The particle is in state 2 initially Find (a) prob of it being in state after one time step; (b) prob of it being in state after two time steps Ans: We know that at t : x() (a) Using the iteration equation x() P x() 3 4 2 2 3 4 2 5 5 2 5 4 2 4 x () 4 (b) Again using the iteration equation x(2) P x() 3 4 2 2 3 4 2 5 5 2 5 4 2 4 37 2 36 2 47 2 x (2) 37 2 63
46 Matrix inverse Def of one-to-one LT: If T : R n R m is a LT and if for all x, x 2 R n, T ( x ) T ( x 2 ) only if x x 2, then T is one-to-one In other words, a one-to-one LT transforms distinct vectors into distinct image vectors Clarification of the concept of one-to-one f(x) f(x) f(x) f( x )? x f( )? f( x )? f( x ) f( x ) x x x x x 2 x x x one to many (not defined!) many to one (not one to one!) one to one Def of matrix inverse: For a square matrix A n n, if det A, then it is invertible and defines a one-to-one LT Matrix B is called the inverse of A iff AB BA I We use A to denote the inverse of A Thus, AA A A I 64
Question: Given an invertible matrix A, how to find A? Answer: Tow methods (i) Row operation/reduction (ii) Formula (easy for 2 2 matrices, not so for larger ones) Theorem (Method ): If A is invertible (ie det A ), then finding the inverse is equivalent to solving A x y for x A : x y A : y x A x I y I x A y Therefore, A is solved by [ A I row operations RREF of A [ I A Remark: Matrix I is the RREF of an invertible matrix A Thus finding the inverse is equivalent to finding the RREF of the augmented matrix [ A I 65
Eg 46 Find the inverse of the following matrices [ 4 (a) A ; (b) B 2 3 2 7 2 Ans: (a) [A I ()()+4(2) (2) (2) [ 4 2 7 [ 7 4 2 [ 4 2 [ 7 4 A 2 (2)(2) 2() Let s verify [ 4 AA 2 7 [ 7 4 2 [ 7 + 8 4 4 4 + 4 8 7 [ (b) [B I ()() (3) (2)(2) (3) 2 3 2 2 (2)(2) 2() (2)(3) () B 2 2 66
Formula for A for 2 2 matrices: For A [ a b c d, if det A, then A det A [ d b c a Proof: Since det A det ad bc, let s divide the augmented matrix by det: [ a b c d ()d() b(2) (2)a(2) c() divide by det [ a det c det [ d det c det b det det d det det The above two row operations must be done simultaneously on the previous matrix Here is () d() b(2): [ ad d() det [ bc b(2) det [ ad bc () d() b(2) det bd det bd det d det b det a det d det b det b det [ d det b det Similarly, (2) a(2) c() can be calculated to yield the results above 67
Eg 462 For A [ 4 2 7, A? Ans: det A 4 2 7 7 8 Thus, A [ 7 4 [ 7 4 det A 2 ( ) 2 [ 7 4 2 which is identical to the result in 46(a) 68
Important properties of matrix inverse: If A is invertible, then A exists and is unique Proof: If AB BA I and AC CA I, then B BI B(AC) (BA)C IC C 2 If A, B are both invertible, so is AB and that (AB) B A Proof: AB(B A ) A(BB )A AA I, (B A )AB B (A A)B B B I 3 If A is invertible, then A x b x A b 4 If A n n is invertible, the following statements are all equivalent (a) det A (b) x is the unique solution of A x (c) Rank(A)n (d) RREF(A)I (e) There exists a A such that AA A A I (f) Columns of A are LI and form a basis of R n (g) Rows of A are LI and form a basis of R n 69
Inverse of LTs in geometry and row operations () M Projφ [ cos 2 φ cos φ sin φ cos φ sin φ sin 2 φ, M Proj φ? Ans: Does not exist because det M Projφ which suggests that it is not invertible! Notice an important property of M Projφ : M 2 Proj φ M n Proj φ M Projφ for all integer n >! (2) M Rotφ [ cos φ sin φ sin φ cos φ, M Rot φ? Ans: M Rot φ M Rot φ M Rot2π φ [ cos φ sin φ sin φ cos φ A T Other important properties of M Rotφ include (a) M Rot2π I 7
(b) M Rotπ M Rot π M Rot π, thus M 2 Rot π I (c) M 6 Rotπ 3 I, thus M Rotπ 3 M 5 Rotπ 3 (d) Columns (and rows) of M Rotφ form an orthonormal basis of R 2, thus is a called a real unitary matrix/orthogonal matrix Such matrices have the following properties: (i) det A or - (ii) A x x (iii) A A T (3) M Refφ [ cos(2φ) sin(2φ) sin(2φ) cos(2φ), M Ref φ? Notice that M Refφ is also an orthogonal matrix, thus M Ref φ M T Ref φ M Refφ, and that M 2 Ref φ I (4) Row operations that we perform in Gaussian elimination are all LTs Thus, each row operation can be carried out by MM Such matrices are referred to as elementary matrices 7
[ Eg S is an elementary matrix that swaps the two rows of a 2 2 matrix [ [ [ a b c d Check : c d a b Notice that S is an orthogonal matrix and that det S Thus, S S T [ S, and that S 2 I Question: Is there a formula for calculating A for all A n n (for n > 2)? Answer: Yes, will be covered later But the computation is rather demanding even for n 3 (comparable to using row operations!) 72
47 Determinant Determinant is a scalar that measures the magnitude or size of a square matrix Notice that conclusions presented here are focused on rows and row expansions Identical results apply when the rowfocus is changed to column-focus In other words, if the word row(s) is replaced by column(s) in the following text, all results will stand Definition (verbal): For an n n (square) matrix A [a ij, det A is the sum of products of n entries, ie det A (±)a j a 2j2 a njn, where (j j 2 j n ) is one permutation of integers 2 n The sum exhausts a total of n! such permutations which exhaust all possible ways of picking one and only one entry from each row and each column Terms with even permutations carry a + sign and those with odd permutations a - sign Permutation: A rearrangement of an ordered list of numbers, say S, is a permutation of S 73
Eg 47 For S {, 2, 3}, there are a total of 3! 6 permutations Let P (23) represents the set of all possible permutations of S P (23) {23, 23, 32, 32, 23, 32} Note: the total number of permutations of S (23) is equal to the total number of outcomes in a 3-team sport competition It is the total number of ways of putting 3 distinct objects into 3 ordered slots It is also the total number of distinct passwords one can generate out of the three digits,2,3 Definition: For each permutation (j j 2 j n ) of integers (, 2,, n), σ(j j 2 j n ) minimal number of pairwise exchanges of to recover the order (2 n) Even/odd permutations: a permutation (j j 2 j n ) is even/odd if the value of σ(j j 2 j n ) is even/odd Eg 472 For S {, 2, 3}: σ(23), σ(23) 2, σ(32) 2 are even; σ(32), σ(23), σ(32) are odd 74
[ a a Eg 473 For 2 2 matrix A 2 a 2 a 22, det A (±)a j a 2j2 (+)a a 22 +( )a 2 a 2 a a 22 a 2 a 2 Eg 474 Use definition to show the determinant of diagonal and triangular matrices is equal to the product of diagonal entries Answer: In each one of these matrices, there is only one way of picking one and only one entry from each row and each column that is not a zero a a 22 a nn a a 22 a nn a a 22 a nn a a 22 a nn 75
Eg 475 Use definition to show that if A has a zero row (or column), det A Answer: Since in every term of the determinant, there is one and only one number from each row and each column Thus, there is at least a in each product which makes every term equal to! a a j a n a n a nj a nn a a n a i a in a n a nn 76
Definition (formula): For all n n matrix A [a ij ( i, j n) a a 2 a n a det A 2 a 22 a 2n a n a n2 a nn P (n) ( ) σ(j j 2 j n ) a j a 2j2 a njn, where P ( n) is the set of all possible permutations of integers ( n) (a total of n!) and that { ( ) σ(j j 2 j n +, if ) σ(j j 2 j n ) is even,, if σ(j j 2 j n ) is odd Eg 475 For a 3 3 matrix A [a ij ( i, j 3), det A a a 2 a 3 a 2 a 22 a 23 a 3 a n2 a 33 ( ) σ(j j 2 j 3 ) a j a 2j2 a 3j3 P (23) ( ) σ(23) a a 22 a 33 + ( ) σ(23)2 a 2 a 23 a 3 + ( ) σ(32)2 a 3 a 2 a 32 ( ) σ(32) a a 23 a 32 + ( ) σ(23) a 2 a 2 a 33 + ( ) σ(32) a 3 a 22 a 3 a a 22 a 33 + a 2 a 23 a 3 + a 3 a 2 a 32 a a 23 a 32 a 2 a 2 a 33 a 3 a 22 a 3 77
Remark: For matrices larger than 3 3, this formula is not practical to use by hand So, it shall be mainly employed in demonstrating important properties of determinants Row expansion formula As an engineer, you may not need to know how to prove each formula that we learn In this section, I will focus on the how-to without saying why For all practical purposes, an engineer should know how to calculate the determinant of a large matrix using either row or column expansions We have already learned how to calculate a 3 3 matrix using expansion in the first row of a matrix As a matter of fact, the first row is not always the best to use in an expansion formula So, we here extend that formulas to expansions in any row of a matrix 78
a a j a n a a j a n a i a ij a in a i ( ) i+ a i a ij a in + a n a nj a nn a n a nj a nn a a j a n a a j a n +a ij ( ) i+j a i a ij a in + +a in ( ) i+n a i a ij a in a n a nj a nn a n a nj a nn a i c i + + a ij c ij + + a in c in row expansion formula, where A ij is the (n ) (n ) matrix obtained by deleting the i th row and j th column of A: A ij a a j a n a i a ij a in a n a nj a nn, det A ij a a j a n a i a ij a in a n a nj a nn det A ij is called the (i, j) th minor of A c ij ( ) i+j det A ij is called the (i, j) th cofactor of A Similarly, we can expand in the j th column of a matrix 79
Eg 476 det A 2 2 4 3 2 2 2 4 4 3 6 2? Ans: Using expansion in the first row, we obtain det A ( ) + 2 2 2 2 4 3 2 2 2 4 4 3 6 2 3 2 2 4 4 6 2 +( ) +3 2 + 2 2 2 2 4 3 2 2 2 4 3 2 2 2 4 4 3 6 2 4 3 2 2 4 3 6 +( ) +4 4 2 2 4 3 2 2 2 4 4 3 6 2 Using expansion in the second row, we obtain det A ( ) 2+3 3 2 2 4 3 2 2 2 4 4 3 6 2 +( ) 2+4 2 2 2 4 3 2 2 2 4 4 3 6 2 3 2 4 2 4 4 3 2 +2 2 2 2 2 4 3 6 8
But the simplest is to expand in the second column, det A ( ) 3+2 2 2 2 4 3 2 2 2 4 4 3 6 2 ( ) 5 2 2 2 4 3 2 3 6 2 ( 2)(2+2 36 24) 72 Remarks: (i) Row (column) expansion is the sum of the products between all entries of that row (column) multiplied by the corresponding cofactors (ii) The (i, j) th cofactor c ij ( ) i+j det A ij, where A ij is the (n ) (n ) matrix obtained by deleting the i th row and the j th column of matrix A 8
Important Properties of Determinant: All properties can be proved using the formula definition of the determinant However, we shall not cover them in this course det A det A T, A and A T have the same determinant Eg: 2 3 4 4 6 3 2 4 2 det A k l det A, where A k l is the matrix obtained by exchanging the k th and the l th row (or column) Thus, swapping between two rows/columns reverses the sign of det A Eg: 2 3 4 4 6 2, but 3 4 2 2 4 3 6 4 2 82
3 det A kl, where A kl is a matrix whose k th and l th rows (or columns) are identical Based on the previous property, det A det A k l det A, thus det A Eg: 2 2 2 2 4 det(αa) α n det A, where A is n n Eg: 2 3 4 2, 2 [ 2 3 4 2 4 6 8 6 24 8 22 ( 2) 83
5 Determinant is linear in each row/column Suppose a row vector is a linear combination of two row vectors, say the k th row a k sb + tc (k, 2,, n), where s, t are scalars Then, det a a 2 sb + tc a n s det a a 2 ḅ a n + t det a a 2 c a n Equivalently, we can also express this property in the following form a a 2 a n a 2 a 22 a 2n sb + tc sb 2 + tc 2 sb n + tc n a n a n2 a nn s a a 2 a n a 2 a 22 a 2n b b 2 b n a n a n2 a nn +t a a 2 a n a 2 a 22 a 2n c c 2 c n a n a n2 a nn Thus, multiplying a row/column by a scalar results in the change in det A by the same factor 84
Eg: a b 3 5 5a 3b, 2a 2b 3 5 a 6b 2(5a 3b) 2 a b 3 5 a + c b + d 3 5 5(a + c) 3(b + d) 5a + 5c 3b 3d, a b 3 5 + c d 3 5 5a 3b + 5c 3d Thus, a + c b + d 3 5 a b 3 5 + c d 3 5 85
6 det A kk+αl det A, Adding a scalar multiple of one row/column to another row/column leaves det A unchanged This is a direct consequence of properties 3 and 5 Eg: a b c d ad bc a b c + 2a d + 2b a b c d + 2 a b a b a b c d a b c + 2a d + 2b a(d + 2b) b(c + 2a) ad + 2ab dc 2ab ad bc 86
7 det AB det A det B Eg: det A det B 2 3 4 8 3 5, 4 3 3 ( 4) 7 AB [ 2 5 3 5, det AB 2 5 3 5 3 ( 65) 35 det A det B 87
Formula for the inverse of n n matrices: For all n n matrices A [a ij : () A ij (n ) (n ) obtained by deleting i th row and j th column of A (2) m ij det A ij the (i, j) th minor of A (3) c ij ( ) i+j det A ij the (i, j) th cofactor of A Definition of cofactor matrix: For all n n matrices A [a ij, its cofactor matrix is C [c ij c c 2 c n c 2 c 22 c 2n, c n c n2 c nn where c ij is the (i, j) th cofactor of matrix A Formula for A : A ( ) det A C T where C is the cofactor matrix of A 88
Eg 477 Given A (a) Show that it is invertible 2 2 5 2, (b) Find A using the cofactor formula Ans: (a) First calculate the cofactors for the expansion in row : c + 5 2 2; c 2 2 2 2; c 3 + 2 5 3 det A ()c + (2)c 2 + ()c 3 4 + 3 7! (b) Then, calculate all the other cofactors: c 2 2 5 2 9; c 22 + 2 2 2; c 23 2 2 5 4 c 3 + 2 ; c 32 ; c 33 + 2 2 89
Therefore, C 2 2 3 9 2 4 2 A ( ) C T det A 7 2 9 2 2 3 4 2 Verify, A A 7 2 9 2 2 3 4 2 2 2 5 2 7 7 7 7 I 3 9