MTH 0A - Linear Algebra - 05-6-II Semester Arbind Kumar Lal P Field A field F is a set from which we choose our coefficients and scalars Expected properties are ) a+b and a b should be defined in it ) a+b and a b must be inside the field 3) Both operations are commutative: a+b = b+a; a b = b a 4) There should be identity elements for both operations Identity element for + is called 0 and that for is called 5) Inverse for a wrt +: a F, b F st a+b = 0 a 0 wrt : a F \{0}, b F st a b = 6) Value of (a+b)+c and a+(b+c) are equal Value of (a b) c and a (b c) are equal 7) distributes itself over +: a (b+c) = (a b)+(a c) Examples: R, C, Q Also Z 5 = {0,,,3,4} with a+b := (a+b) (mod 5) and a b := (a b) (mod 5) Here, 3+4 =, 4 = 3 and 4 4 = Q[ ] := {a+b : a,b Q} (a+b )+(c+d ) = (a+c)+(b+d) (a+b ) (c+d ) = (ac+bd)+(ad+bc) Also, whenever (a+b ) 0, we have (a+b ) a = a b is irrational P Linear Equations A linear equation (over some field F) is an expression of the form b a b as Indian Institute of Technology Kanpur a x +a x + +a n x n = b
Example: x +3x +7x 3 = 6 is a linear equation Over what? Over any field What does mean in F? +! Anyway we never mention that!! Come on, isn t it obvious? A system of linear equations is a collection of m linear equations in the same n variables It has the form: a x + +a n x n = b a m x + +a mn x n = b m } x+y +z = is a system of linear equations in 3 variables y+3z = 7 Matrix form of a system (of linear equations): a a n x b = a m a mn x n b m TheproductAB correspondstooperatingonthecolumns of the matrix A, usingentries of B Thus, is same as a a x + a m a a n x b = a m a mn x n b m a a x + + a m a n a n x n = a mn b b (Similar to does there exist integers x,y such that 3x+5y =??? ) P The old story: A new way b m Q Solve x+y = x+y = A x+y = x+y = () x+y = x+4y = () () x+y = 3y = 3 So Write the Augmented coefficient matrix of the system with a matrix:
Above solution procedure is nothing but () () () Continue? 4 0 3 3 3 () 0 () () 0 0 () 0 0 } x x+y +z = = System: That is, y y +3z = 7 0 3 7 z Augmented Coefficient Matrix: 0 3 7 Apply elimination 0 3 7 () 3 7 0 () () 0 5 3 7 0 Highlighted positions are called pivots/leading terms Corresponding variables are basic variables Others are free variables [ x 5 + t ] 7 Here z is free For solution: put z = t; then y = 3 t z t 5 7 For example, t = 0 gives and t = gives 0 P We don t want to solve it Let the computer do it! In that case we have to provide an algorithm What is an algorithm? A step by step instruction to carry out the task Actually, we do not have to!! Gauss-Jordan have already done it!! To describe that we need 3 elementary row operations Given the Augmented Coefficient Matrix (ACM) of a system: ) E i (α) Multiply ith row by α 0 For example, 0 0 3 E ( ) 0 3 0 3 0 0 3 a a n b a m a mn b m 3
0 0 0 Result is the same as: 0 0 0 3 0 0 0 3 Elementary Matrix E ( ) P Elementary row operations E ij Interchange rows i and j 0 3 0 0 3 0 3 For example, 0 3 E 4 0 3 3 0 0 5 5 0 0 0 3 Result is the same as: 0 3 0 3 0 5 Elementary Matrix E 4 E ij (α) Replace ith row R i by R i +αr j 0 3 0 0 3 0 3 For example, 0 3 E 35( 5 ) 0 3 0 5 5 7 5 0 5 5 0 0 3 Result is the same as: 5 0 3 3 0 5 Elementary Matrix E 35 ( 5 ) P Echelon-form(EF) Let A be a matrix A pivot/leading term is the first (from left) nonzero element of a nonzero row in A We use a ij to denote it A matrix A is in echelon form (EF) (ladder like) if ) Pivot of the i+th row comes to the right of the ith ) Entries below the pivot in a pivotal column are 0 3) The zero rows are at the bottom 4
In EF: 3 3 0 0 0, 0 0 0 3 0 3 3 0 Not in EF: 0 0 0 0 0 0 0 5 0 0 6 0 0 0 0 0 0 0 (rule, fail); P Row-reduced-echelon-form(RREF) A matrix A is in rref if ) It is in EF ) Pivot of each nonzero row is 3) Other entries in a pivotal column are 0 In rref: Not in rref: [ 0 3 0 0 0 0 0 0 0 0 [ 0 3 3 0 0 0 0 0 0 0 0, 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 (rule 3 fails) ] 0 0 5 0 0 6,, 0 0 0 0 0 0 0 0 0 0 0 0 0 ] 0 3 0 0 3, 0 0 0 0, 0 0 0 0 0 0 0 0 0 0 A is row equivalent to B if B is the result of k elementary row operations on A That is, if there exists some elementary matrices E,,E k st E k E k E A = B P Gauss-Jordan elimination(gje) GJE An algorithm Uses row operations only Input: A Output: a matrix B in RREF st B is row equivalent to A 0) Put region = A ) If all entries in the region are 0, STOP Else, in the region, find the leftmost nonzero column and find its topmost nonzero entry Suppose it is a ij Box it This is a pivot Take it to the top row of the region Make it Make other entries of the whole matrix in it s column 0 ) Put region=the submatrix below and to the right of the current pivot Go to step ) 5
The process will stop, aswe can get at most min{m,n} pivots 0 3 7 Apply GJE: 3 4 8 E 0 3 7 3 4 8 E 3( ) 0 3 7 0 3 7 E ( ) 0 0 0 0 0 0 0 0 0 0 5 0 5 3 7 0 0 3 7 E 3 7 ( ),E 3 ( ) 0 0 0 0 0 E 3 7 34 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 E 3 (?),E 3 (?) 0 0 0 0 0 0 0 0 0 Th Each matrix A m n is row equivalent to some matrix in rref!! P Gauss elimination GE It is the same as GJE except that ) pivots need not be made and ) entries above the pivots need not be made 0 3 E ( ),E 3 ( ) 4 9 0 E 3 ( 3) 0 3 8 GJE may be viewed as an extension of GE 0 0 0 P Inverses of elementary matrices Sometimes it helps to imagine an elementary matrix as an elementary row operation Recall Invertibility: A n n is invertible if there is a B st AB = BA = I What is the inverse of E i (α)? That is, if I have multiplied ith row by α, how do i get back the original? Must be E i ( ) α What is the inverse of E ij? That is, if I have interchanged R i and R j, how do i get back the original? Must be E ji = E ij What is the inverse of E ij (α)? That is, if I have added αr j to R i, how do i get back the original? Must be E ij ( α) Ex Inverse of an elementary matrix is an elementary matrix Ex If A is invertible, then it has a unique inverse Recall that for any invertible matrix A(i,:) 0 Th Let A n n be invertible Then, the rref of A is I Po A rref of A is nothing but E k E A = EA (say), where E i s are elementary 6
As EA is invertible, it has no zero row How many pivots should it have? As EA is in rref, it must be I P RREF Th If A is row equivalent to B then, the systems Ax = 0 and Bx = 0 have the same solution set!! Q Is A= ] 0 0 0 0 row equivalent to B= 0 0 0 a 0 b? 0 0 0 a No,[ b is a solution of Bx=0, not of Ax=0 Th Let A and B be two row equivalent matrices in rref Then A = B P Rank of a Matrix Cor Each matrix A is row equivalent to a unique matrix in rref!! We use rrefa to denote this matrix The rank of a matrix A is the number of pivots in rrefa Notation: ranka Thus, ranka m n m,n and rank(0) = 0 P Gauss-Jordan say it Th Take a system Ax = b and an invertible matrix B Then, y is a solution of Ax = b if and only if y is a solution of BAx = Bb!! Gauss-Jordan idea Take the ACM [A b] of a system Keep on applying elementary row operations Solution space stays same!!! STOP at the rref[a b ] If A (i,:) = 0 and b i 0, then conclude that the system has no solution (inconsistent) Note that, here, rank(a) < rank([a b]) Otherwise, a general solution is obtained by assigning the free-variables arbitrary values and by evaluating the values of the basic variables Note that, here, rank(a) = rank([a b]) P Very Important Ideas Th Let A be a matrix of Rank r Then, there exist(s) 7
B Invertible P st PA = 0 Invertible P and Q such that I P m m A m n Q n n = PAQ == r 0 0 0 Ex[Rank Factorization] Let ranka m n = r Then there exist B m r,c r n st rankb = rankc = r and A = BC as [ A = P I r 0 Q I = [P 0 0 P r 0 ] 0 0 Th Consider the homogeneous linear system Ax = 0 Then, ][ ] Q = P Q Q The zero vector, 0 = (0,,0) t, is always a solution, called the trivial solution Suppose x,x 0 are two solutions of Ax = 0 Then, k x +k x is also a solution of Ax = 0 for any k,k R ThConsider thelinearsystem Ax = b, where Aisanm n matrixandx t = (x,,x n ) If [C d] = rref([a b]) then, Ax = b is inconsistent (has no solution) if [C d] has a row of the form [0 t ], where 0 t = (0,,0) That is, Rank(A) < Rank([A b]) = Rank([C d]) Ax = b is consistent (has a solution) if [C d] has NO row of the form [0 t ] That is, Rank(A) = Rank([A b]) = Rank([C d]) Furthermore, (recall n = Number of unknowns implies) if Rank(A) = n then, Ax = b has a unique solution if Rank(A) < n then, Ax = b has infinite number of solutions P Example x+y +z = System y +3z = x+z = ACM: A = 0 3 0 RREF(A) = 0 0 0 3 0 0 0 0 Matlab command rref(a) 8
System is inconsistent (has no solution) x+y +z = System y +3z = x+z = 0 ACM: A = 0 3 rref(a) = 0 3 0 0 0 0 0 { +t } x,y are basic, z is free General solution: 3t t : t R Let [C d] = rref([a b]) Assume C(i,:) 0 and the system is consistent Notice that in C(i,:) all entries are zero, except the pivot and entries corresponding to free variables Thus, if we assign all free variables 0, then we get x i = d i What happens if we put z = t = 0, in the above example? If rank(a m n ) < n, then Ax = 0 has a nonzero solution!! P Invertibility and Gauss-Jordan Let A be a square matrix of order n Then the following statements are equivalent A is invertible The homogeneous system Ax = 0 has only the trivial solution The (rref) row-reduced echelon form of A is I n A is a product of elementary matrices If A is invertible then rref(a) = I Thus, E k E k E E A = I for some elementary matrices Hence, A = E k E k E E Implication Given a matrix A n n, apply GJE to [A I n ] Then, we get elementary matrices E,E,,E k such that E k E k E [A I] = [E k E k E A E k E k E I] = [I A ] P Inverse - Example [ 0 0 ] E ( ) 0 0 E 0 3 ( /3) 0 3 3 0 3 3 0 3 3 E ( ) 3 0 0 4 7 0 0 0 0 E ( ) E 3 ( ) 9 [ 3 0 0 0 0 0 0 0 ]
[,4,7,0,,0] [,,3,,0,0] = [0,0,,,,0] [,,,0,0,] [,,3,,0,0] = [0,,,,0,] 3 0 0 E 3 0 0 E ( ) E ( ) 0 0 0 E 3 ( ) E 3 () [ 0 0 3 0 0 5 0 0 0 [ 0 0 0 0 0 0 0 Thus, ] we[ have ] solved ] three linear systems ] [ simultaneously, ] [ namely] x α 0 u 0 3 A[ y = 0, A[ β =, and A[ v = 0, where A = 4 7 z 0 γ 0 w (x,y,z) = ( 3,5, ), (α,β,γ) = (,,) and (u,v,w) = (,,0) P Determinant 3 Notation:- Let A = 3 4 7 3 Then, A( ) = = 7 3, 4 7 3 3 A( 3) = = 4 3, 4 7 and A(,,3) = [4] Determinant: Determinant of a square matrix A = [a ij ], denoted det(a) (or A ) is defined by a, if A = [a] (n = ), det(a) = n ( ) +j a j det ( A( j) ), otherwise j= Let A = [ ] Then det(a) = A = a b Let A = Then, det(a) = A = a A( ) b A( ) = ad bc c d For A =, det(a) = = 5 3 = 3 5 3 5 ] ] 0
P Singular, Non-Singular Singular, Non-Singular: A matrix A is said to be a singular if det(a) = 0 It is called non-singular if det(a) 0 Th Let A be an n n matrix If B = E ij A then, det(b) = det(a), B = E i (c),c 0 then, det(b) = cdet(a), B = E ij (c),c 0 then, det(b) = det(a), all the elements of one row of A are 0 then, det(a) = 0, two rows of A are equal then det(a) = 0 A is a triangular matrix then det(a) is product of diagonal entries Th As det(i n ) =, det(e ij ) = det(e i (c)) = c whenever c 0 det(e ij (c)) =, whenever c 0 Let A n n = [a ij ] Then, for any k, k n det(a) = n ( ) k+j a kj det ( A(k j) ) j= LetA n n matrix Then, det(a) equalsthevolumeofthen-dimensional parallelepiped formed by the rows of A u v T w R γ Sv Qu P θ Figure 3: Parallelepiped with vertices P,Q,R and S as base Let u t = (u,u,u 3 ),v t = (v,v,v 3 ) and w t = (w,w,w 3 ) Then, u v = (u v 3 u 3 v,u 3 v u v 3,u v u v ) and w w w 3 volume (P) = Area(PQRS) height = w (u v) = ± u u u 3 v v v 3
P Cofactor (i,j) th minor of A Denoted A ij equals det(a(i j)) (i,j) th cofactor of A: Denoted C ij equals ( ) i+j A ij Adjoint of a Matrix: Denoted Adj(A) = [C ji ] 3 4 7 Example A = 3 Then, Adj(A) = 4 5 as C = ( ) + A = 4,C = ( ) + A =,,C 33 = ( ) 3+3 A 33 = 3 4 7 A Adj(A) = 3 4 5 0 0 = 6 0 0 = Adj(A) A 0 0 Note that det(a) = 6 So, Important Observation: Is it always TRUE? A Adj(A) = Adj(A) A = det(a)i P Determinant-cofactor Th Let A be an n n matrix Then, for i n, for i l, n a ij C ij = n a ij ( ) i+j A ij = det(a), j= j= n a ij C lj = n a ij ( ) l+j A lj = 0, j= j= A(Adj(A)) = det(a)i n Thus, whenever det(a) 0 one has A = det(a) Adj(A) Th A square matrix A is non-singular if and only if A is invertible Th Let A and B be square matrices of order n Then det(ab) = det(a) det(b) = det(ba)
Th Let A be a square matrix Then det(a) = det(a t ) P Cramer s Rule Th The following statements are equivalent for a square matrix A: A is invertible The linear system Ax = b has a unique solution for every b det(a) 0 Cramer s Rule Let A be an n n matrix If det(a) 0 then, the unique solution of the linear system Ax = b is x j = det(a j), for j =,,,n, det(a) where A j is the matrix obtained from A by replacing the jth column of A by the column vector b 3