row operation. = elementary matrix E with B = EA [B = AE]. with Id E by the same elementary operation.

Transcription

1 . Elementary matrix operations and systems of linear equations goal: computing rank of matrix solving systems of linear equations.. Elementary matrix operations and elementary matrices. goal: transform a matrix by elementary operations into a simpler one of same rank Definition.. An elementary row-column operation is () interchanging two rows/columns () multiplying a row /column by a non-zero scalar () adding a scalar multiple of a row /column to another row /column 4 Ex A 4 Rem If P Q by elementary operation, then Q P by el. oper. type : type : type : interchange rows and multiply second column by add 4 third row to first row need later 4 A A A 4 Definition.. An n nelementarymatrix is one obtained from I n by performing an elementary operation. Ex. ր ց exch. row and add row to row Theorem.. Let A M m n (F) A B by elementary [col.] row operation. elementary matrix E with B EA [B AE]. with Id E by the same elementary operation. A A A A 4 A A Theorem.4. Elementary matrices are invertible, and their inverses are of the same type.

2 Proof....λ / λ λ... -λ... Rank of matrix and matrix inverses. Definition.5. A M m n (F) rk(a) : rk(l A : F n F m ). Theorem.6. Let V, W be VS /F Then rk(t) rk ( β ) γ OB [T] γ β. T : V W linear transformation. Proof. The following diagram commutes. V T W φ β φ γ φ β (x) [x] β φ β, φ γ isomorphism F n L [T] γ β F m rk(t) dim(imt) dim(φ γ (ImT)) def of rank φ γ iso () using def of rank ( dim(im(l [T] γ )) rk β L [T] γ β previous def ) ( ) : rk [T] γ β φ γ (ImT) Im(φ γ T) Im(L [T] γ φ β) L β [T] γ (Imφ β) L β [T] γ (Fn ) Im(L β [T] γ ) β diagram commutes φ β iso () Theorem.7. Let A M m n (F) P, Q invertible. M m m Then rk(pa) rk(a) rk(aq) rk(paq). M n n L Q onto Proof. R(L AQ ) R(L A L Q ) L A L Q (F n ) L A (L Q (F n )) L A (F n ) R(L A ) rk(aq) dim(r(l AQ )) dim(r(l A )) rk(a). dim R(L PA ) rk(pa) diml P (L A (F n )) dim(l A (F n )) dimr(l A ) rk(a). L P iso

3 Corollary.8. Elementary row and column operations on a matrix are rankpreserving. Theorem.9. The rank of a matrix is the maximal number of linearly independent columns dim(subspace generated by columns). Proof. Let A M m n (F). Then L A : F n F m L A (x) A x L A (e i ) A e i i-th column of A. L A linear So Im(L A ) L A (span({e i })) span({l A (e i )}) span(columns of A). Example rk first two cols are linearly independent, Corollary.. A M m n (F) is invertible rk(a) m n. Theorem.. (important!) A M m n (F). Then A el. op. m r n r {}}{ Id r {. r rk(a). : I r,m,n Proof. By example p : pivot element row : pivot element col : if all elements in pivot rectangle are end } row col p exchange rows and columns so that pivot (type ) divide row by pivot element so it gets (type )

4 zero out below pivot by type row operations zero out right of pivot by type col operations (make entries to ) I,4,5 done 5 5 move pivot one row down & one col right Corollary.. A M m n (F) rk(a) r then P, M m m Q M n n products of elementry matricies: A PI r,m,n Q Proof. P E E k product of elementary matrices E i for row operations in Q col 5. Then P E k E, E i are elementary. Corollary.4. Let A M m n (F). (a) rk(a T ) rk(a) (b) rk(a T ) dimspan(columns of A) dimspan(rows of A) Corollary.5. A M n n (F) is invertible A is a product of elementary matrices Proof. A invertible rk(a) n cor.. A P P, Q products of elementary matrices I n,n,n I n Q PQ, elementary matrices are invertible their products are invertible

5 M m m Corollary.6. Let A M m n (F) rk(a) r P, M n n Q invertible with A PI r,m,n Q. 5 Proof. By Corollary., P, Q products of elementary matrices. elementary matrices are invertible P, Q invertible assume A PI r,m,n Q, P,Q invertible by Corollary.5, P, Q are products of elementary matrices and rk(ea) rk(a) rk(ae ) when E, E elementary so rk(a) rk(pi r,m,n Q) rk(i r,m,n ) r. Proof of Cor.4. (a) When P is invertible, (P T ) (P ) T, so A PI r,m,n Q rk(a) r A T Q T invertible (b) follows from (a). I r,n,m P T invertible rk(a T ) r. Theorem.7. Let T : V W, U : W Z linear transformations, A,B matrices such that AB defined. Then (a) rk(ut) rk(u) rk(t) (b) rk(ab) rk(a) rk(b) Proof. (a) rk(ut) ( ) T(V) W U(T(V)) U(W) dim(u(t(v))) dim(u(w)) dimim(u) rk(u). U : W Z. Let Ũ U T(V ) : T(V) Z dimim(ũ)+dim(ker(ũ)) dim(t(v)) }{{} dimim(ũ) dim(t(v)) ( ) dim(ũ(t(v))) dim(t(v)) dimim(t) rk(t) (b) for matrices apply to L A, L B. The inverse of a matrix Definition.8. A M m n, B M m p, (A B) (AB) M m (n+p) augmented matrix (A B) Let A M n n (F) consider (A I n ). Assume A is invertible. Then A is product of elementary matrices (Cor.5 above) A E... E g.

6 6 (A B) E(A B) (EA EB) Now do this for E g,e g,...,e on (A Id n ). for an elementary matrix. E is an elementary row operation (A Id n ) A (A Id n ) (A A A Id n ) (Id A ). Corollary.9. A is invertible there exists a sequence of row operations that turn (A Id n ) into (Id n B). In this case B A. Proof. we proved that (A Id n ) (Id n A ) by row operations. Now assume it is possible. (A Id n ) (Id n X) by row operations () row operations on (A X) preserve X A: invariant (EA EX) (A X) (X E )(EA) X A. Thus if (), then X Id n Id n A A X X A If (A Id n ) (Id n B) by row operations, then forget right block: A Id n by row operations, so rk(a) rk(id n ) n A is invertible. this means: if A is non-invertible, any attempt to turn (A Id n ) row (Id n B) will produce a left block with a zero row/column. How to find row operations for (A Id n ) (Id n B)? (this is the first method to compute B A ; we may see another) 4 Ex 4? (We start as in the algorithm to determine rank.) 4 (A I) 4 exch row and pivot 4 if pivot + all below pivot 4 then stop. column: matrix not invertible

7 / 4 / 4 / multiply first row by / pivot zero out below pivot & above pivot (if pivot is in column > ) comes first important difference to row-column reduction: cannot zero out right of pivot by column operation! 7 / 4 / 4 / / / / / / / / / / 8 / 8 / 4 / 8 5/ 8 / 4 / 4 / 4 / / 8 / 8 / 4 4 change shape of pivot rectangle unnecessary (now what is above pivot is also in rectangle) divide by pivot zero above and below pivot 4 change pivot skip done Systems of linear equations - Theoretical aspects.

8 8 a x a n x n b () a m x a mn x n b m A (a ij ) m i,j n x x. x n b coefficient matrix. b. b m () Ax b. linear equation system (LES) a ij, b i F field const x j F unknowns s F n solution As b {solutions} : solution set { consistent : solution set () inconsistent: solution set system of m linear equations with n unknowns over F we had this before! given v,...,v n, x, find λ i with λ i v i x. Now we do this more systematically x +x x +x +x x +x x x x x +x 6 x +x { x x ( 6,,7) (x unique sol,x,x ) sol. no sol. inconsistent (8, 4, ) How do we find solution? Definition.. A system Ax b is { homogeneous ifb nonhomogeneousifb Theorem.. K sol. set of Ax. K ker(l A ) : kera. dimk n rk(l A ) Proof. L A : R n R m dimker(l A ) dim R n dimim(l A ). fewer eqn than variables Corollary.. If m < n, then K {}. Theorem.. Let Ax b (4) be a (consistent) LES and Ax (5) be the homogeneous system corresponding to (4). Then K (4) s+k (5) {s+k : k K (5) }, where s is a concrete solution of (4). Proof. Let s K (4) s K (4) K (4) s+k (5). As b As b A(s s ) s s K (5) s ( s+s }{{} )+s s+k (5). K (5)

9 9 Nowlets s+k (5) s s+k Ak Example.4. As As+Ak As b s K (4) s+k (5) K (4). x +x x x + x x A dim(kera) dim(r ) rk(a). check that ( ) how to find later ( ) : R R kera kera { t rk(a) : t R } x +x x x + x x K 4 { 4 + t special solution : t R } solution set One special case Theorem.5. Let Ax b be a LES with A M n n. Then Ax b has exactly one solution A is invertible In that case the solution is x A b. Proof. Let A be invertible x A b is a solution. {sol. of Ax b} A b+ { } sol of Ax {A b} }{{} kera {} A invertible injective Let s be the unique solution of Ax b. s unique {solutions to Ax b} s+ { sol of Ax } {s}. }{{} kera ker(a) {} A injective A : F n F n bijective A invertible Ex. x +4x x / x +4x +x x 8 5/ 8 / 4 A b / 4 / 4 / x +x +x x / 8 / 8 / 4 / 4. / 4 Definition.6. (A b) the augmented matrix of LES Ax b.

10 Theorem.7. Ax b is consistent LES rk(a b) rk(a). x : Ax b b Im(A) span( columns of A ) [ ] if V W, then V W dimv dimw : (ImL A ) V {}}{ span(columns of A {b} ) dim(span( col of A W {}}{ span(columns of A ) ) ) dim(span( col of A {b} ) ) Example.8. x +x x +x rk(a) rk(a b) ( ) ( ) ( ) A b (A b). rk(a) inconsistent Example.9. Is a linear combination of,? x +x x +x x x x rk(a b) no solution no closed economic model (Application) producer & consumer : Farmer (food), Tailor (cloth), carpenter (house) no capital enters and exits (closed model) Food Clothing Housing Farmer.4.. Tailor..7. Carpenter.5..6 consumption (... ) e.g. Farmer consumes 4% of food and % of cloth. Qu How much must F,T,C produce to attain an equilibrium ( income spending society survives )? Let food cloth house p, p, p be incomes of farmer/tailor/carpenter when farmer must buy % of housing, he spends.p

11 spending {}}{ income {}}{.4p +.p +.p p.p +.7p +.p p.5p +.p +.6p p consumption matrix.4.. A..7. p.5..6 p p Ap p..5 In this example p.5 p ker(a Id) equilibrium condition.4 so income of farmer:taylor:carpenter 5:7:8 p Rem. what we want is p (positive; all entries p i ) Thm. If A (B.Noble 97) n {}}{ {}}{ B C D E } n with C,D >, then (I A)x has a -dimensional dolsution set generated by a non-negative vector. open model outside demand d food d d cloth d d d house { }} { income(carpenter) spending(carpenter)+ outside demand (house) farmer taylor farmer taylor food cloth d win(carpenter) d+ap p p (I A) d (and again p must be non-negative).4. LES - computational aspects. /. one sol we know: to solve Ax b we need \. basis of sol. set of Ax we use row operations to accomplish & by transforming Ax b into a simple system where we see solutions Definition.. (A b) (A b ) equivalent if x sol. of Ax b x sol. of A x b.

12 Theorem.. C M m m invertible, A M m n, b F m (CA Cb) (A b) (same solutions) Corollary.. Row operations on (A b) give equivalent system. Qu: How to find proper row operations? Gaussian elimination by example x +x +x x 4 x +x +x x +x +x x 4 (6) I part Forward pass 4 5 move pivot make pivot to (but not down) (but not above) right as long as pivot + below can divide by but better exch row & zero out below pivot move pivot right & down skip make pivot ( second row) pivot + below is zero move right we achieved now that the first non-zero entry in each row & always right of first entry in the previous row II part Backward pass zero out above using row operations of type from right to left (so that in columns with this is the only non-zero entry)

13 + + 5 (7) x +x x x 4 Definition.. A matrix is in reduced row echelon form (r.r.e. form, RREF) if the following conditions are satisfied: (a) any non-zero row lies above any zero row (b) first (leftmost) non-zero entry in a row is the only non-zero entry in its column (c) first (leftmost) non-zero entry in a row is and occurs in a column (above) right to leftmost entry in previous row What is the solution of a LES in r.r.e. form? for every variable that does not occur leftmost in an equation take arbitrary value; in our example x t then solve in terms of t x x t x t x x 4 t solution set of (7) t solution t R set of (6) +t : t R Example.4. x x 5 x +x x 5 x x +x 5 x 4 x 5 entries that do not occur first in a row: x and x 5 x s, x 5 t and solve for x,x,x 4

14 4 x 4 +x 5 +t x x 5 +x t+s x +x 5 x +t s +s +t general solution x x x x 4 x 5 (s,t R arbitrary) +t s t+s s +t t Theorem.5. (A b) LES in r.r.e. form with r non-zero equations. (a) rk(a) r (b) the above procedure gives the general solution x x +sx +tx +ux + s,t,u, R where x is one solution of non-homogeneous system & system sx +tx + is the general solution of corresponding homogeneous if (A b) consistent (c) (A b) is inconsistent zero row in A with non-zero element in b; Rem r.r.e. form is unique. (Why?) An interpretation of r.r.e.f. Let A be a matrix and B be the r.r.e.f. of A Let be the leftmost non-zero entries in each row of B. # rkb # non-zero rows in B r.r.e.f. column of #k are k-th row : j k e k. Claim : j,...,j r -columns of A are linearly independent because c a j + +c r a jr B invertible mtx MA c Ma j e + +c r Ma jr e r c i because A B by row operations

15 5 Let A be invertible x A b is a solution. {sol. of Ax b} A b+ { } sol of Ax {A b} }{{} kera {} A invertible injective Claim : when b k a k d.. d r... (8) is te k-th column of B then r d i a ji is the k-th column of A i because b k r d i e i, so i ( r ) a k M (8) b k M d i e i i i.e., all other j i -th columns of A are linear combinations of a ji b ji e i r d i M b ji i r d i a ji i all columns of A are linear combinations of a ij {a j,...,a jr } are a basis of the image of A Theorem.6. Let A M m n rk(a) r (r > ) and let. Then B be r.r.e.f. of A (a) The number of non-zero rows in B is r. (b) i,,...,r a column b ji of B s.t. b ji e i (c) a j,...,a jr (columns of A numbered j,...,j r ) are linearly independent (d) For each k,,...,n: column k of B is b k d e +...+d r e r a k d a j +...+d r a jr Corollary.7. r.r.e.f of A is unique. Proof. go from k, k,..., k n Let l be the number of linearly independent columns,...,k. k-th column a k of A is not lin. combination of previous columns k-th column of B is not linear combination of previous columns B r.r.e.f. b k e l+ if so, +l is uniquely determined: it is the number in order of such k if not, a j,...,a jl is linearly independent

16 6 so uniquely determined d i with then l d m a jm a k m l d m b jm m e m b k Applications: choose a basis from a generating set d d d l... complete a linearly independent set to a basis Example A B 4 b, b, b 5 linearly independent, b b, b 4 4b b a, a, a 5 linearly independent, a a, a 4 4a a {a,a,a 5 } basis of span{a,...,a 5 } S {}}{ Question: find a basis of span, 4, 8,, which is a subset of S: answer {,, }. Solution: write the vectors in S as columns of a matrix A find reduced r.e.f. B of A choose the columns of A which correspond to columns of B with Question: Complete a linearly independent set of vectors S V to a basis of V. write S as columns of a matrix A append to A on the right as columns some generating set H of A, A (A H) find reduced r.e.f. B of A choose the columns of A which correspond to columns with of B: S {...} is a basis

17 7 Example.9. V {(x,...,x 5 ) R 5 : x +7x +5x 4x 4 +x 5 } S {(,,,, ), (,,,, ), ( 5,,,,)} can check using that S is linearly independent i.e. if we write vectors in S into a matrix A then every column in r.r.e.f. B of A has a : 5 A find a generating set H of V {x 7x 5x +4x 4 x 5 } (x,x,x,x 4,x 5 ) ( 7t 5t +4t t 4,t,t,t,t 4 ) t +t +t +t 4 answer: basis of V S is 5 }{{} S S {4,,,,)} }{{} H / / summary: we can determine rank determine inverse solve a LES complete a linearly independent set to a basis choose a basis from a generating set B H {,,, 4} A B r.r.e.f.