Math 18.02: Notes on vector and matrix forms of a system of linear equations, matrix algebra, inverse matrices, and related facts

Transcription

1 Math 80: Notes o ector ad matrix forms of a system of liear equatios, matrix algebra, ierse matrices, ad related facts This sulemet roides more details about the ector ad matrix formulatios for a system of liear equatios, liear fuctios defied by matrices, the meaig of the colums of a matrix, ad how to fid matrices for seeral imortat geometrically defied liear trasformatios Vector form of a system of liear equatios ax + + a x b Ay system of m liear equatios i ukows is of the form If we choose to am x+ + amx b m rereset ectors i R m as colums ad use oly the defiitios of scalar multilicatio of a ector ad ector x tx x y x+ y additio, ie t ad +, we ca exress these liear equatios i the form: xm txm xm ym xm + ym If we deote the colum ectors as a a b x + + x (ector form of the liear system} am am bm a a,, ad am am b b, we ca the rewrite this more bm succictly as x+ + x b This ca be uderstood geometrically What this says is that this system will hae a solutio (or may solutios) if the ector b o the right-had-side ca be exressed as a liear combiatio of the ectors {,, }, ie the ector b ca be built out of these ectors by aroriate scalig ad ector additio So x 4 ad y 00, ad this agrees with our exectatios 3x+ y 4 Examle: The liear system x y 3 ca be writte i ector form as 3 4 x + y 3 With 3 (i red) ad (i blue), ad 4 b 3 (i black), we wat to kow what x ad y must be so that x+ y b Visually, we might guess that this ca be doe with x betwee erhas ad 5, ad y a small egatie umber We sole for these alues usig row reductio methods: x y+ z 4 Examle #: If we write the system 4x+ 4y+ z 3 i ector form, we hae 3 4 x + y + z If 3 4 we write 4 (i red) ad 4 (i blue) ad 3 (i gree), ad b 3 (i black), we are the seekig alues for xyz,, so that x+ y + z 3 b There are (ifiitely) may ways to do this

2 This agrees with what we foud whe we soled a similar system last week usig row reductio: x 0 0 t y 7 0 t z 4t t R Eery choice of t gies a differet way to costruct the ector b out of these three saig ectors (see icture at right) 3x+ y 5 Examle #3: The system x+ y 7 ca be writte i ector form x+ y as x + y 7 Writig,, ad b 7 as ectors i R 3, there is some doubt as to whether it s ossible to do this, ad this agrees with the fact that we reiously foud this system to be icosistet This situatio is illustrated i the diagram at right (where the axes hae bee rotated for a better iew) The red ad blue ectors, ad, sa a lae, ad the third ector, b, does ot lie i this lae I liear algebra we exress this by sayig b sa{, } Matrix form of a liear system If we take the ector form aboe ad assemble the ectors {,, } side-by-side to form a m matrix a a x A, ad if we write x, we ca defie the roduct of this matrix ad the am am x x ector as Ax x + + x Usig this defiitio, we ca exress the liear system x ax + + a x b simly as Ax b This is called the matrix form of the liear system am x+ + amx b m This ca also be uderstood i terms of (liear) fuctios Note that if we write T ( x) Ax, we hae the iut m ector x R ad the outut ector T ( x) Ax b R We ca therefore uderstad such a system of liear m equatios i terms of the fuctio T : R R We also sometimes rereset this by writig either: T m R R or m R A R or A fuctio defied i this maer is called a liear trasformatio A m x R Ax b R m Defiitio: A fuctio T : R R is called a liear trasformatio if for all ectors, R ad for all scalars c, c R, T satisfies the liearity roerty T( c+ c) ct ( ) + ct ( ) This ca also be exressed more geometrically by sayig that T reseres ector additio, ie T( + ) T( ) + T( ), ad T reseres scalar multilicatio, ie T ( c ) ct ( ) We call the iut sace R the domai (as exected), ad we refer to the outut sace m R as the codomai

3 Note : Oe thig worth metioig here is that this otio of a liear fuctio may ot etirely agree with reious usage of the term liear as see i calculus courses ad before Secifically, ay fuctio of the form L( x) ax + b is first-order, but it is ot liear uless b 0 Note that Lx ( + y) ax ( + y) + b Lx ( ) + Ly ( ) Also, reseratio of scalar multilicatio meas that it would hae to be the case that L(0) b 0, so i order to be liear it must be the case that L( x) ax (the grah of this lie would hae to ass through the origi) I articular, ot that a fuctio like f( x) x+ 3 is ot a liear fuctio! More geerally, a liear fuctio T : R R would hae to be of the form T( x,, x) cx + + cx, ie a m ure first-order exressio without costat term For a liear fuctio T : R R, all m (outut) comoets of the alue of this fuctio would hae to be of this form Note : I the case of a fuctio defied by T ( x) Ax for a m matrix A, the liearity roerty simly becomes the distributie law: A( cx+ cx) cax ( ) + cax ( ) Proositio: T ( x) Ax (for a m matrix A) is a liear trasformatio x y Proof: If we write the matrix A i terms of its colums, A ad let x, y ad let x y x y αx βy αx+ βy αβ, R, the αx+ βy α + β + usig basic facts about scalig ad x y αx βy αx + βy addig ectors Usig our defiitio of the roduct of a matrix ad a ector, we hae: αx+ βy T( αx+ βy) A( αx+ βy) ( αx+ βy) + + ( αx + βy) αx + βy αx + βy + + αx + βy αx + + αx + βy + + βy α( x + + x) + β( y+ + y) αax + βay αt( x) + βt( y) As you ca see, the liearity roerty ultimately flows from the distributie law for ector additio Imortat Note: We bega by lookig at systems of liear equatios ad itroduced matrices iitially as a coeiet way of keeig track of the maiulatio of equatios e route to a solutio of the system A matrix was effectiely just a box of umbers We ow hae a ery differet ad extremely imortat ew iew of a m matrix as a liear fuctio This fuctioal iew of a m matrix as a liear fuctio from R to R will be with us from ow o Meaig of the colums of a matrix Now that we are able to thik of a matrix as a fuctio, it s ossible to roide a simle iterretatio of the colums of a matrix that will allow us to costruct matrices based o iformatio about how they act o ectors I R we itroduce the stadard or elemetary basis ectors e, e,, e You hae 0 0 robably see these ectors before uder differet ames For examle, i R, we hae e 0 i ad 0 e j, ad we ca write ay ector i R 0 as x x x + y x + y y 0 i j 3

4 0 0 Similarly, i R 3, we hae e 0 i, e j, ad e k, ad we ca write ay ector i R 3 as 0 x 0 0 x y x0+ y+ z0 xi+ yj+ zk z 0 0 x This same decomositio ca be doe i R as x xe+ + xe Usig our defiitio of the roduct x of a matrix ad a ector, we see that: 0 Ae {st colum of the matrix A} 0 0 Ae {d colum of the matrix A} Ae { th colum of the matrix A} I other words, the colums of a matrix tell us how the corresodig liear fuctio acts o the basic ectors { e,, e} ad, quite sigificatly, these comletely determie the matrix I fact, for ay ector x xe+ + xe, we hae Ax A( xe + + x e ) xae+ + x Ae x + + x We ca ow begi writig dow some imortat examles of matrices Idetity i R : The idetity fuctio Id : R R is simly Id ( x) x This is clearly liear (it reseres eerythig, icludig scalig ad additio of ectors) ad we hae all we eed to determie its corresodig (square) matrix, deoted by I (ofte just as I), ad called the Idetity matrix Id( e) e {st colum of the matrix} Id( e 0 ) e {d colum of the matrix} I e e 0 Id( e) e { th colum of the matrix} This matrix has 0 s eerywhere excet o the mai diagoal, ad all of the diagoal etries are equal to Dilatio (scalig) i R : This is a trasformatio of the form T( x) rx for some fixed scalar r We hae: T( e) re {st colum of the matrix} T( e 0 ) re {d colum of the matrix} r A re re 0 r T( e) re { th colum of the matrix} This also yields aother diagoal matrix, oce agai with equal etries o the mai diagoal 4

5 Ierse of a liear trasformatio Defiitio: We call a liear trasformatio T : R R iertible (also called osigular) if it is both oe-tooe (if T( x) T( y ) the ecessarily x y) ad oto the codomai (for eery ector z R there is a (uique) x R such that ( x) T z) It s relatiely easy to see why iertibility will oly make sese for liear trasformatios T : R R gie by (square) matrices, T ( x) Ax ; ad certaily ot all such trasformatios will hae ierses This is the same otio of iertibility we hae for fuctios elsewhere Howeer, i the cotext of liear trasformatios gie by T ( x) Ax we hae a simle algorithmic way of ot oly determiig if this liear trasformatio is iertible, but also for determiig the matrix of this ierse fuctio (referred to as its ierse matrix A ) if it exists It all comes dow to a ehaced iew of row reductio ad what iertibility meas i terms of rak ad the reduced row-echelo form of a associated matrix Some of you may already kow about ierse matrices ad may be temted to use them to sole arbitrary systems of liear equatios This is a ery bad idea! Liear systems ca be icosistet, ad they ca also hae ifiitely may solutios If you restrict yourself to usig ierse matrices for solig all liear systems, you will ery soo come to regret this Row reductio is uiersally alid 3 x Cosider a simle examle like T ( x) Ax where A Gie ay iut ector x x, this 3 x y trasformatio will gie the outut ector T ( x ) Ax x y y To be iertible, gie ay y x ector y y, we would hae to be able to sole uiquely for x x i terms of the comoets of y What does this mea i terms of algebra? T ( ) 3 x 3x + x y x Ax x x x y I e staggered the right-had-sides a bit to suggest the aroach All we hae to do is augmet the matrix a little more ad rereset these two equatios by eterig the coefficiets o both the left-had-side ad the righthad side This gies 3 0 [ ] 0 0 A I where I is the aroriate Idetity matrix If it s ossible to x sole uiquely for x x, we ll discoer this by row reductio That is: x 5 y+ 5 y This last array reads: x 5 y 5 y I A We discoer two thigs from this examle: x () If the matrix A has full rak, the we will be able to sole uiquely for x x () If the matrix A has full rak, the matrix of its ierse will aear i the right half of rref [ A I ] I A 5

6 The situatio i geeral is o differet If T : R R is gie by a (square) matrix, T ( x) Ax, ad if we write T ( x) Ax y, the we write the matrix [ ] A I ad carry out row reductio to determie whether this has full rak If it does t hae full rak, the we ca t sole uiquely for x, ad the trasformatio (ad its matrix) is ot iertible Howeer, x () If the matrix A has full rak, the we will be able to sole uiquely for x i terms of x () If the matrix A has full rak, the matrix of its ierse will aear i the right half of rref [ A I ] I A y y y Note: This is geerally the simlest way to fid the ierse of a matrix by had There is a formulaic way of doig this usig determiats (based o Cramer s Rule), but it s imractical for matrices larger tha 3 3 O a matrix-caable calculator, the recirocal butto ofte also seres to fid the ierse of a square matrix a b There is a eer-so-simle way to fid the ierse of a matrix A c d First, calculate its determiat det( A ) ad bc You ca easily show usig our row reductio method that if det( A ) ad 0 bc 0, the the matrix A will ot hae full rak ad will ot be iertible If det( A ) ad 0bc 0, the A will hae full rak ad will be iertible, ad d b d b A det( ) c a ad bc c a A 3 For examle, if A, the det( A ) (3)( 0) 0 ()() 05 0 ad A The corresodig method for 3 3 matrices has similar elemets to this, but ioles far more calculatio Matrix algebra a a Defiitio: Gie ay scalar k R ad a m matrix A, we defie the scalar multile of am am ka ka the matrix as ka I the case of a m matrix (a colum ector) or a matrix (a row kam kam ector), this is the same as the ordiary scalig of a ector 6 3 Examle: a a b B b Defiitio: Gie two m matrices A ad B B, we defie the sum of these am am bm B bm two like matrices by addig their resectie etries That is: a B a b B b ( a + b) B ( a + b ) A+ B B + B B am B am bm B bm ( am + bm ) B ( am + bm) 6

7 Examle: Proositio: For ay m matrix A, ay scalar k, ad ay colum ector x, ( ka) x k( Ax ) Proof: This is a straightforward calculatio Writig A i terms of its colum ectors, we hae x x ( kax ) k k k x( k ) x( k ) kx ( x ) x x x k k( ) Ax x Proositio: For ay m matrices A ad B ad ay colum ector x, ( A + B) x Ax + Bx x Proof: If we write A ad B w B w, ad x, the x x x ( A+ Bx ) ( ) ( ) B + w B w + w B + w x x x + w + B+ x + w x + xw + B+ x + x w ( ) ( ) x x B B B + w B w Ax + Bx x x x+ + x + xw+ + xw + Matrix roducts Though it s ossible to take a formulaic aroach to the multilicatio of matrices, it s much better to thik of each matrix as reresetig a liear trasformatio ad to defie matrix roduct by cosiderig the comositio of these liear trasformatios Proositio: Where defied, the comositio of liear trasformatios is a liear trasformatio m Proof: Suose A is a m matrix that corresods to a liear trasformatio T : R A R, ie T ( y) Ay A Also, suose B is a matrix that corresods to a liear trasformatio T : R B R, ie T ( x) Bx B m We ca the defie the comositio T A TB : R R by ( T T )( ) T ( T ( )) A B x A B x Sice both of these fuctios are liear, for ay scalars c, c ad ectors, R, we hae: T T ( c + c ) T T ( c + c ) T ct ( ) + c T ( ) ( A B) A( B ) A( B B ) ct ( T ( )) + c T ( T ( )) c ( T T ) + c ( T T ) A B A B A B A B m So TA TB : R R is also liear ad is rereseted by a m matrix Call this matrix AB Corollary (really a restatemet of the defiitio): For ay ector x R, ( ) ( ) AB x A Bx 7

8 This statemet look ery much like a associatie law for multilicatio, but it s really just the statemet that AB is defied to be the matrix of the comositio Calculatio of the matrix roduct How do we actually calculate the matrix roduct AB (where defied)? Perhas the simlest way to do this is to recall the meaig of the colums of ay matrix The colums tell us where the corresodig liear fuctio takes the elemetary ectors { e,, e}, so AB AB( e) B AB( e ) A( Be) B A( Be ) But if we write B Be ( ) B Be ( ) B, we the see that AB A ( ) B A ( ) That is, AB A B A B A I other words, the matrix A simly idiidually multilies each of the colum ectors of B 0 4 Examle: If A 5 ad B 0, the the roduct AB is defied (though BA is ot), ad AB It should be relatiely easy to see that each etry is calculated as ( AB) ( ith row of A) ( jth colum of B ) ij This dot roduct is oly defied whe the umber of colums of A matches the umber of rows of B Matrix multilicatio (where defied) is ot commutatie: AB BA It s easy to uderstad why matrix multilicatio caot be commutatie ee i the case where both roducts are defied Matrix roduct is just the comositio of fuctios, ad comosig fuctios i reerse order does ot geerally gie the same fuctios, ie f g g f This is most simly uderstood by thikig about it i less mathematical terms For examle, if you ut o your socks ad the ut o your shoes, this is clearly differet tha first uttig o your shoes ad the uttig o your socks Sometimes you ca get the same result, just as it is the case that there are some matrices A ad B such that AB BA, but this will ot geerally be the case Matrix multilicatio (where defied) is associatie: ( AB) C A ( BC ) This follows from the corresodig fact about comositio of fuctios, amely that ( f g) h f ( g h) The Idetity matrix acts as a multilicatie idetity: For a m matrix A, Im A A ad AI A Though this is easy to see by calculatio, it follows from the geeral fact about fuctios that Id d f f ad f d Id f, ie for ay x i the domai of f, we hae ( Id d f )( x) Id( f ( x)) f ( x) ad ( f d Id )( x) f ( Id( x)) f ( x) Proositio: If A is a iertible matrix with ierse matrix A, the A A I ad AA I Proof: These follow directly from the fact that matrix roduct reresets the comositio of liear fuctios ad the fact that a fuctio comosed with its ierse (i either order) yields the idetity fuctio 8

9 Proositio: If both A ad B are iertible matrices, the AB is also iertible ad ( ) AB B A Proof: This also follows directly from the geeral fact about fuctios, ie ( ) f g g f I omathematical terms, if you first ut o your socks ad the ut o your shoes, the ierse of this is to first take off your shoes ad the take off your socks More easy-to-roe matrix algebra facts: For ay scalar k ad aroriate sized matrices A( C + D) AC + AD (left-had distributie law) ( A + B) C AC + BC (right-had distributie law) ka C k AC ( ) ( ) These ad the facts reiously stated are ot meat to be exhaustie Excet for the fact that matrix multilicatio is ot commutatie, most of the familiar algebraic rules are also true for matrices Notes by Robert Witers 9