Linear Algebra: Lecture notes from Kolman and Hill 9th edition.

Transcription

1 Linear Algebra: Lecture notes from Kolman and Hill 9th edition Taylan Şengül March 20, 2019 Please let me know of any mistakes in these notes Contents Week Systems of Linear Equations 1 12 Matrices 2 13 Matrix Multiplication 2 Week Algebraic Properties of Matrix Operations 3 15 Special Types of Matrices and Partitioned Matrices 4 Week Echelon Form of a Matrix 6 22 Solving Linear Systems 7 Week 4 8 Week Elementary Matrices; Finding A Definition of Determinant Properties of Determinant 14 Week Cofactor Expansion Inverse of a Matrix Other Applications of Determinants 19 Week 1 11 Systems of Linear Equations Examples of a system of m linear equations in n variables

2 m = n = 1 m = 2, n = 1 Geometry: every point on the line is a solution m = n = 2 Geometry: solutions lie in the intersection of two lines 3 Cases: 1 Lines coincide: Infinitely many solutions 2 Lines intersect at exactly 1 point: unique solution 3 Lines are parallel and do not intersect: no solution m = n = 3 Geometry: solutions lie in the intersection of three planes Cases: 1 Planes all coincide: in finitely many solution 2 Planes intersect on a line: infinitely many solutions 3 Planes intersect at exactly 1 point: unique solution 4 Two of the planes are parallel and do not intersect: no solution 12 Matrices Representation of an m n matrix Rows and columns of a matrix Square matrix Main diagonal of a matrix An n 1 matrix is called an n-vector When are two matrices equal? Matrix addition Scalar multiplication of a matrix Difference of two matrices Linear combination of matrices Transpose of a matrix Exercises 12 5, 7, 9, 11, Matrix Multiplication Dot product of two vectors Matrix multiplication Which matrices can be multiplied? If the matrix product AB is defined, BA may not be defined If it is defined it may not be equal to AB

3 Matrix vector product a 11 a 12 a 1n c 1 a 21 a 22 a 2n c 2 Ac = a m1 a m2 a mn c n Ac = c 1 a 11 a 21 a m1 + c 2 a 12 a 22 a m2 = + + c n (row 1 (A)) T c (row 2 (A)) T c (row m (A)) T c a 1n a 2n a mn Ac = c 1 col 1 (A) + c 2 col 2 (A) + + c n col n (A) Express the linear system in the matrix form Augmented matrix Exercises 13: 1, 5, 9, 11, 15, 17, 19, 23, 30, 33, 37, 41, 51 Week 2 14 Algebraic Properties of Matrix Operations Properties of matrix addition Theorem 11 Commutative, associative, zero matrix, additive inverse Properties of matrix multiplication Theorem 12 Associative, distributive property over addition Properties of scalar multiplication Theorem 13 r(sa) = (rs)a (r + s)a = ra + sa r(a + B) = ra + rb A(rB) = r(ab) = (ra)b Properties of transpose Theorem 14 ( A T ) T = A (A + B) T = A T + B T proof for (AB) T = B T A T (AB) T = B T A T (ra) T = ra T (AB) T ij = (AB) ji = k a jk b ki = k b T ika T kj = (B T A T ) ij Exercises 14: 22, 30, 31

4 15 Special Types of Matrices and Partitioned Matrices A square matrix is called a diagonal matrix if its entries satisfy a ij = 0 if i j A scalar matrix is a diagonal matrix whose diagonal elements are equal Identity matrix I n is a scalar matrix with diagonal elements equal to 1 An n n square matrix is called the identity matrix I n if its entries satisfy { 0, i j a ij = 1, i = j If A is m n: AI n = A, and I m A = A For a square matrix, define powers of A A 0 = I n, A p = AA }{{ A }, p 1 p times Example Let A be a diagonal matrix with diagonal entries {2, 3, 5} Compute A 4 By definition, we set A 0 = I n For any non-negative integers p, q A p A q = A p+q, (A p ) q = A pq Note that in general, (AB) p A p B p An n n matrix is called upper triangular if its entries satisfy a ij = 0 if i > j, that is all its entries below the main diagonal are zero Similarly we can define lower triangular matrices A square matrix is symmetric if A T = A and skew-symmetric if A T = A Write the general form of 3 3 symmetric and skew symmetric matrices Example Any square matrix A can be written as a sum of symmetric and skewsymmetric matrices This decomposition is unique ( ) 1 2 = S + K 3 4 Write S = ( ) ( a b b c and K = 0 d ) d 0 and solve for a, b, c, d to find a = 1, b = 5/2, c = 4, d = 1/2 Partitioned matrices We skip this topic for now If for an n n matrix A there exists an n n matrix B such that AB = BA = I n, then A called a nonsingular or invertible matrix and B is called an inverse of A If A is not invertible then it is called singular or non-invertible

5 The inverse matrix is unique proof Suppose Then AB = BA = I n and AC = CA = I n B = BI n = B(AC) = (BA)C = I n C = C Because of uniqueness, we write the inverse of a matrix as A 1 Example Find the inverse of Solution A = [ [ ] [ ] [ ] 1 2 a b 1 0 AA 1 = = I 3 4 c d 2 = 0 1 [ ] [ ] a + 2c b + 2d 1 0 = 3a + 4c 3b + 4d 0 1 [ ] 2 1 A 1 = ] Theorem 16 If A and B have inverses, then so does AB And its inverse is (AB) 1 = B 1 A 1 proof Check the inverse identity from left and right Theorem 17 In general if A 1,, A n are invertible then so does A 1 A n and (A 1 A n ) 1 = A 1 n A 1 1 Theorem 18 Inverse of inverse is itself proof This is evident from the identity AA 1 = A 1 A = I Theorem 19 Taking transpose and inverse commutes proof ( A 1) T A T = I T n = I n and ( A T ) ( A 1) T = In Linear Systems and Inverses Start with the linear system Ax = b Suppose that A is nonsingular A 1 (Ax) = A 1 b ( A 1 A ) x = A 1 b I n x = A 1 b x = A 1 b We showed that if A is an n n matrix, then the linear system Ax = b has the unique solution x = A 1 b Moreover if b = 0, then the unique solution to the homogeneous system Ax = 0 is x = 0 Example Solve the systems x + 2y = 8 3x + 4y = 6, and x + 2y = 10 3x + 4y = 2 Solution Write the system as Ax = b Then pre-multiply with A 1 Once A 1 is computed Ax = b can be solved repeatedly for different b

6 Exercises 15: 3, 4, 5, 9, 11, 17, 19, 21, 22, 27, 31, 32, 33, Section 16 Matrix Transformations We briefly touched upon the transformations of vectors when multiplied by matrices In particular we talked about transformations such as dilation, contraction and rotation You are not responsible from this section when studying for the exams Week 3 21 Echelon Form of a Matrix An m n matrix A is said to be in reduced row echelon form (RREF) if it satisfies the following properties: a) All zero rows, if there are any, appear at the bottom of the matrix b) The first nonzero entry from the left of a nonzero row is a 1 This entry is called a leading one of its row c) For each nonzero row, the leading one appears to the right and below any leading ones in preceding rows d) If a column contains a leading one, then all other entries in that column are zero An,m n matrix satisfying properties (a), (b), and (c) is said to be in row echelon form (REF) example Give examples where a matrix breaks one of each rule There are three types of elementary row operations 1 Interchange row i and row j: r i r j 2 Replace row i by k 0 times row i: kr i r i 3 Replace row j by k time row i + row j: kr i + r j r j If the matrix B m n can be obtained from the matrix A m n then B is row equivalent to A If A is row equivalent to B then B is row equivalent to A since each elementary row operation has an inverse which is again an elementary row operation For example the inverse of the operation r i r j is itself, since applying it twice gives the original matrix Every non-zero matrix is row-equivalent to a matrix in REF The REF of a matrix may not be unique Moreover, every matrix is equivalent to a unique matrix in RREF Algorithm for bringing a matrix to a REF and RREF by using elementary row operations See example 5 in the book Exercises 21: 1, 3, 5

7 22 Solving Linear Systems Take a system of m equations and n unknowns Then write it in matrix form Ax = b Then write it in the augmented matrix form [A b] Theorem Let Ax = b and Cx = d be two linear systems If the augmented matrices [A b] and [C d] are row equivalent then the linear systems are equivalent, ie they have the same solutions To solve a linear system: 1 Write the linear system in the augmented form: [A b] 2 Gaussian Elimination: Reduce [A b] into row echelon form and solve by back substitution 3 Gauss-Jordan reduction: Reduce [A b] into reduced row echelon form Example 6 from Section 22 The system x + 2y + 3z = 6 2x 3y + 2z = 14 3x + y z = 2 in augmented matrix form is Use the row operations r 1 + r 2 r 2, , r 1 + r 3 r 3, , r 3 r 3, r 2 r 3, , r 2 + r 3 r 3, r 3 r 3,

8 Then solve by back substitution That is Week 4 z = 3, y = 4 2z = 2 x = 6 2y 3z = 1 Examples of systems with exactly one solution, no solution, infinitely many solutions with 1, 2 free parameters If a system has no solutions we say that the system is inconsistent, otherwise we say that the system is consistent Example If [ C d ] = then Cx = d has no solutions since the last equation is Example The solutions of the system are in the form 0x 1 + 0x 2 + 0x 3 + 0x 4 = x 1 = 1 10r x 2 = 2 + 5r x 3 = 11 + r x 4 = 9 2r x 5 = r, any real number Example The solutions of the system are of the form x 1 = 2 3 r 2s t x 2 = r x 3 = s x 4 = t x 5 = t

9 with r, s, t being real numbers Application: quadratic interpolation Find the quadratic polynomial passing from the points (1, -5), (-1, 1), (2, 7) Consider a system of the form Ax = b, where A m n is a matrix The system is called a homogeneous system if b = 0 where 0 is the m 1 zero vector If b 0 then the system is called a non-homogeneous system A homogeneous system Ax = 0, where A m n is a matrix, always has a solution Namely the solution x = 0 which is the n 1 zero vector This solution is called the trivial solution Hence, a homogeneous system is always consistent We discussed that for linear systems, there are either 0, 1 or many solutions Hence for homogeneous systems the question is whether the trivial solution is the only solution or there are infinitely many solutions Theorem The system has infinitely many solutions if m < n, that is there are fewer equations than unknowns proof Consider the RREF form B of A Then each column of B which does not have a leading one will be a free parameter Since each row has at most 1 leading one, if there are more columns than there are rows, each column can not have a leading one Example x + y + z + w = 0 x + w = 0 x + 2y + z = 0 whose has augmented matrix in REF is Thus the solution is x = r y = r z = r w = r, any real number Example Balance the chemical reaction xnaoh + yh 2 SO 4 zna 2 SO 4 + wh 2 O That is, find x, y, z Solution Na: x = 2z O : x + 4y = 4z + w H : x + 2y = 2w S : y = z

10 A non-trivial solution is NaOH + H 2 SO 4 Na 2 SO 4 + 2H 2 O Example Section 22, exercise 14 Determine all values of a such that the resulting linear system has (a) no solution; (b) a unique solution; (c) infinitely many solutions: x + y z = 2 x + 2y + z = 3 x + y + ( a 2 5 ) z = a Example Section 22, exercise 27 Find an equation relating a, b, and c so that the linear system 2x + 2y + 3z = a 3x y + 5z = b x 3y + 2z = c is consistent for any values of a, b, and c that satisfy that equation The relationship between homogeneous and non-homogeneous linear systems If Ax = b has a particular solution x p and Ax = 0 has a solution x h, then x h + x p is also a solution to Ax = b Exercises 22: 1, 3, 5, 7, 10, 15, 20, 26, 30, 39 Week 5 23 Elementary Matrices; Finding A 1 An m n elementary matrix of type I, type II, or type III is a matrix obtained from the identity matrix I n by performing a single elementary row or elementary column operation of type I, type II, or type III, respectively example The following are elementary matrices E 1 = E 3 = , E 2 = , and E 4 =

11 Theorem Let A be an m n matrix, and let an elementary row operation of type I, type II, or type III be performed on A to yield matrix B Let E be the elementary matrix obtained from I m by performing the same elementary row operation as was performed on A Then B = EA example Give an example Theorem If A and B are m n matrices, then A is row equivalent to B if and only if there exist elementary matrices E 1, E 2,, E k such that B = E k E k 1 E 2 E 1 A Theorem An elementary matrix E is invertible, and its inverse is an elementary matrix of the same type proof This follows from the fact that every row operation can be inverted Theorem A n n is nonsingular if and only if A is a product of elementary matrices proof If A = E 1 E k then since E i is invertible, A 1 = E 1 k E1 1 Conversely, if A is nonsingular, then Ax = 0 has only zero solution since x = A 1 0 = 0 Then each column of RREF(A) must have a leading one, otherwise the system would have a non-trivial solution Thus RREF (A) = I n and A is row equivalent to I n By the above theorem A = E k E 1 I n Summary For A n n the following are equivalent 1 A is nonsingular 2 Ax = 0 has only the trivial solution 3 A is row equivalent to I n (RREF of A is I n ) 4 The linear system Ax = b has a unique solution for every b n 1 proof x = A 1 b 5 A is a product of elementary matrices [ ] 1 2 Let A = Show that A is singular Solution Show that RREF of A is not 2 4 I 2 To find A 1 Suppose A is nonsingular Then there exists elementary matrices E 1,, E k such that E k E 1 A = I n This means A 1 = E k E 1 From this observation, we get (E k E k 1 E 2 E 1 ) [A I n ] = [E k E k 1 E 2 E 1 A E k E k 1 E 2 E 1 I n ] = [ I n A 1] Example Find the inverse of A = if it exists Solution Write [ A I3 ] =

12 Use elementary row operations 5r 1 +r 3 r 3, 1 2 r 2 r 2, 1 4 r 3 r 3, 3 2 r 3+r 2 r 2, r 3 + r 1 r 1, 1r 2 + r 1 r 1 to bring it into the form = [ I 3 A ] Theorem A n n is singular, if and only if it is row equivalent to a matrix which has a row of zeros proof In this case Ax = 0 has a nontrivial solution and hence can not be invertible example Show that the matrix is singular A = Theorem If A and B are n n matrices such that AB = I n, then BA = I n Thus B = A 1 proof Read from the book Remark By definition A and B are inverses of each other if AB = I n and BA = I n This theorem says that to be inverses of each other, it suffices check only one equation AB = I n Exercises 23: 2, 7, 8, 9, 11, 17, 19, Definition of Determinant Let S n = {1, 2,, n} be A rearrangement (with no repetation of elements) j 1 j 2 j n of the elements of S n is called a permutation of S n The set S n has a total A permutation is said to have an inversion if a larger integer comes before than a smaller integer If the total number of inversions is even, then the permutation is called even otherwise it is called odd Example The permutation 4132 of S 4 has 4 inversions: 4 > 1, 4 > 3, 4 > 2, 3 > 2 so it is even Example The permutations of the set S 2 = {1, 2} are 12 and 21 The permutation 12 is even (zero inversions), and the permutation 21 is odd (1 inversion) Example The permutations of the set S 3 = {1, 2, 3} are 123, 231, 312 which are even and 132, 321, 213 which are odd Let A = [a ij ] be an n n matrix The determinant of A is det(a) = (±)a 1j1 a 2j2 a njn where the summation is over all permutations j 1 j 2 j n of the set S = {1, 2,, n} The sign is taken as + or according to whether the permutation j 1 j 2 j n is even or odd

13 For we have [ ] a11 a A = 12 a 21 a 22 det(a) = a 11 a 22 a 12 a 21 Example For det(a) = (2)(5) ( 3)(4) = 22 A = [ ] For we have A = a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 det(a) =a 11 a 22 a 33 + a 12 a 23 a 31 + a 13 a 21 a 32 a 11 a 23 a 32 a 12 a 21 a 33 a 13 a 22 a 31 a 11 a 12 a 13 a 11 a 12 a 21 a 22 a 23 a 21 a 22 a 31 a 32 a 33 a 31 a 32 Figure 1: Sarrus rule to compute 3 3 determinants This method does not work for 4 4 matrices Example The determinant of is The definition of determinant is not practical when n is as large as 10 It involves the sum of 10! terms each of which is a product of 10 terms Around operations Exercises 31: 8, 11, 13

14 32 Properties of Determinant det(a) = det ( A ) T Example A = a c b d = a b c d = A T = ad bc det(a ri r j ) = det(a) if i j Example a b c d = c d a b If two rows (or columns) of A are equal then det(a) = 0 proof Suppose row i = row j, i j then A = A ri r j so that det A = det A ri r j = det A Example a b a b = and = 0 If a row (or column) of A consists entirely of zeros then det(a) = 0 proof This follows from the fact that each summand in the determinant sum formula contains exactly one element from each row (or column) Example = 0 det(a kri r i ) = k det(a) Example ka kb c d = k(ad bc) = k a c b d det(a ri+kr j r i ) = det(a), i j Example a b ka + c kb + d = (a(kb + d) b(ka + c) = a b c d Example = = 4, obtained by adding twice the second row to the first row Example If det(a) = 4 and B = A r1+2r 2 r 1 then det(b) = 4 Let A be an upper (or lower) triangular matrix Then det(a) is equal to the product of the diagonal entries of A Since a diagonal matrix is both lower and upper triangular, the theorem for triangular matrices applies to diagonal matrices as well

15 Example Compute After r 1 r 3 determinant changes by 1 and the matrix is in triangular form The answer is = 10 Since det(a) = det(a T ), column operations can be used too instead of row operations That is 1 det(a) = det(a ci c j ), i j 2 det(a kci c i ) = k det(a) 3 det(a ci+kc j c i ) = det(a), i j Example Compute Second column can be made zero by c 2 2c 1 c 2 So the determinant is zero Example Compute det(a) for After the operations 1 1/2r 1 r 1 2 2r 1 + r 3 r 3, 3 7r 2 + r 3 r A = det(a) = = 2(1)(1)9 = This method is called to as computation of determinant via reduction to triangular form Week 6 Determinant of three types of elementary matrices: det(i ri r j ) = 1,

16 det (I kri r i ) = k, det ( I krj+r i r i ) = 1 Lemma 31 If E is an elementary matrix then det(ea) = det(e) det(a) proof Take E = I ri r j then EA is the matrix A with row i and j swapped Thus det(ea) = det(a) On the other hand det(e) det(a) = det(a) Thus det(ea) = det(e) det(a) The same is true for other types of elementary matrices Suppose B is row equivalent to A We know B = E k E k 1 E 1 A det(b) = det(e k E k 1 E 1 A) = det(e k ) det(e k 1 E 1 A) = det(e k ) det(e k 1 ) det(e 1 ) det(a) A n n is nonsingular if and only if det(a) 0 proof If A is nonsingular then A is a product of elementary matrices and hence its determinant is non zero If A is singular, then A is row equivalent to a matrix B that has a row of zeros Hence its determinant is zero Corollary 31 If A is n n matrix, then Ax = 0 has a nontrivial solution if and only if det(a) = 0 Example 9 Let A be a 4 4 matrix with det (A) = 2 1 Describe the set of all solutions to the homogeneous system Ax = 0 answer since det (A) 0, the homogeneous system has only the trivial solution 2 If A is transformed to reduced row echelon form B, what is B? answer I n 3 Can the linear system Ax = b have more than one solution? Explain answer no the system has unique solution x = A 1 b 4 Does A 1 exist? answer Yes Theorem 39 If A and B are n n matrices then det(ab) = det(a) det(b) proof If A is nonsingular then A = E k E 1, product of elementary matrices det(ab) = det(e k E 1 B) = det(e k ) det(e 1 ) det(b) = det(a) det(b) If A is singular then A is row equivalent to C which has a row of zeros Thus det(ab) = det(e k E 1 CB) = det(e k ) det(e 1 ) det(cb) Notice that CB has a row of zeros and det(cb) = 0 Example If det(a) = 5, det(b) = 3 then det(ab) = 15 and det(a 2 ) = 25 Corollary 32 If A is nonsingular then det(a 1 ) = 1 det(a) proof 1 = det(i) = det(aa 1 ) = det(a) det(a 1 ) In general (most usually) det(a + B) det(a) + det(b) Exercises 32: 1-5, 8, 9, 10, 13, 14, 15, 17, 22, 26, 30, 31

17 33 Cofactor Expansion Let A = [a ij ] be an n n matrix Let M ij be the (n 1) (n 1) submatrix of A obtained by deleting the ith row and jth column det(m ij ) is called the minor of a ij The cofactor A ij of a ij is A ij = ( 1) i+j det(m ij ) Theorem 310 Let A = [a ij ] be an n n matrix Then expansion of det(a) along the ith row is det(a) = a i1 A i1 + a i2 A i2 + + a in A in and the expansion of det(a) along the jth column is det(a) = a 1j A 1j + a 2j A 2j + + a nj A nj proof Prove the statement for the 3 3 case example Use cofactor expansion method to solve Solution Best to expand along 2nd row Becomes ( 1) 2+2 ( 2)( ) = 38 Best way to compute a matrix determinant is to expand it along a row/column with most zeros Can combine row operations with cofactor method to evaluate determinants Example Find all values of t for which t t t + 1 = 0 Solution ( 1) 3+3 (t + 1)((t 1)(t + 2) 0) = 0 gives t = 2, 1, 1 Exercises 33 3, 11, Inverse of a Matrix Theorem 311 If A = [a ij ]is an n n matrix then a i1 A k1 + a i2 A k2 + + a in A kn = 0 for i k proof If B is a matrix obtained from A by replacing the kth row of A by its ith row Thus B has two identical rows and det(b) = 0 On the other hand det(b) = a i1 A k1 + a i2 A k2 + + a in A kn

18 For A = , A 21 = ( 1) = 19, A 22 = ( 1) = 14, A 23 = ( 1) = 3 a 31 A 21 + a 32 A 22 + a 33 A 23 = (4)(19) + (5)( 14) + ( 2)(3) = 0, a 11 A 21 + a 12 A 22 + a 13 A 23 = (1)(19) + (2)( 14) + (3)(3) = 0 a i1a k1 + a i2 A k2 + + a in A kn = det(a) if i = k = 0 if i k Let A = [a ij ] be an n n matrix The adjoint of A is the n n matrix A 11 A 12 A 1n A 21 A 22 A 2n adja = A n1 A n2 A nn where A ij is the cofactor of a ji (Be careful about the transpose) Theorem 312 If A is n n matrix Thus if det(a) 0 then example Let A = answer adj A = A(adjA) = (adja)a = det(a)i n A 1 = det(a) (adja) Compute adj A Verify the above identity Corollary 34 If A is an n n matrix and det (A) 0, then A 1 = 1 det(a) (adj A) Example Use the adjoint method to find the inverse of T,

19 Solution The answer is This method of inverting is much less efficient than the method given in Chapter 2 ([A I n ] [I n A 1 ]) Exercises 34: 2, 9, 10, Other Applications of Determinants Theorem 313 Cramer s Rule Let a 11 x 1 + a 12 x a 1n x n = b 1 a 21 x 1 + a 22 x a 2n x n = b 2 a n1 x 1 + a n2 x a nn x n = b n If det(a) 0, then the system has the unique solution x 1 = det (A 1) det(a), x 2 = det (A 2) det(a),, x n = det (A n) det(a) where A i is the matrix A with ith column replaced by b, b 1 b 2 b = b n proof Follows from Corollary 34 Use Cramer s rule to solve Example 2x 1 + 3x 2 x 3 = 1 x 1 + 2x 2 x 3 = 4 2x 1 x 2 + x 3 = 3 solution A = x 1 = A = 2 = 4 2 = 2

20 x 2 = x 3 = A A = 6 2 = 3 = 8 2 = 4 Cramer s rule is applicable only when there are n equations in n unknowns and the coefficient matrix A is nonsingular Otherwise we must use the Gaussian elimination or Gauss-Jordan reduction methods Cramer s rule becomes computationally inefficient for n 4 Exercises 35 1, 3, 5 Optional Read section 36 for computational complexity of computing determinants with various methods, inverse matrix, etc Supplementary Exercises 1, 2, 3, 4, 5, 8 Quiz 1, 2, 3, 4, 5