GEOMETRY OF MATRICES. FIGURE1. VectorinR 2.

Transcription

1 GEOMETRY OF MATRICES. SPACES OF VECTORS..DefinitionofR n.thespacer n consistsofallcolumnvectorswithncomponents.thecomponents are real numbers...representationofvectorsinr n....r. ThespaceR isrepresentedbytheusualx,x plane.thetwocomponentsofthevector givethex andx coordinatesofapointandthevectorisadirectedlinesegmentthatgoesout from(,) x. FIGURE. VectorinR.,.. 4 x...r. Wecanthinkofthisgeometricallyasthex andx axesbeinglaidoutonatabletopwith thex axisbeingperpendiculartothishorizontalsurfaceasinfigure.thevectorisadirectedline segmentfrom(,,)tothepoint(x,x,x ).Wefrequentlyattachanarrowtotheendoftheline segment, at other times we simply connect the line segment to the dot representing the point. Date: February 8, 8.

2 GEOMETRY OF MATRICES FIGURE. VectorinR. 4 x 4 x x..definitionofavectorspace.avectorspaceconsistsofasetvofobjects(calledvectors) togetherwithtwooperations,vectoraddition(assigningavector x+ ytoanyvectors xand yinv) andscalarmultiplication)assigningavector α xor α xtoanyscalar αandavector xinv)such that following eight axioms are satisfied. V: x+ y= y+ x (commutativelawofvectoraddition)

3 GEOMETRY OF MATRICES V:( x+ y)+ z= x+( y+ z (associativelawofvectoraddition) V:ThereisanelementofV,denotedbyor suchthat + x= x+ = x. V4:Foreach xinv,thereisanelement xinv,suchthat x+ x= x+ x=. V:(α + β) x=α x+β x V6: α( x+ y)=α x+α y V7:(α β) x=α(β x) V8: x= xwhereisascalar. Notethatthedefinitionsof+inV.aredifferentandthattherearetwotypesofmultiplication in V Properties of a vector space..4.. Geometric representation of scalar multiplication. A scalar multiple of a vector a, is another vector,sayu,whosecoordinatesarethescalarmultiplesofa scoordinates,u=ca. Considerthe example below. ( ) a =, c = ( ) () 6 u =ca = Geometrically,ascalarmultipleofavectoraisasegmentofthelinethatpassesthroughand aandcontinuesforeverinbothdirections.forexampleifwemultiplythevector(,)by.we obtain(7.,4.)asshowninfigure. 4 x FIGURE. Scalar Multiple of Vector 7.,4.4, x

4 4 GEOMETRY OF MATRICES Ifwemultiplyavectorbyascalarwithanegativesign,thedirectionchangesasinfigure4. FIGURE 4. Negative Scalar Multiple of Vector x, 4 x,.4..geometricrepresentationofadditionofvectors.thesumoftwovectorsaandbisathirdvector whose coordinates are the sums of the corresponding coordinates of a and b. Consider the example below. ( ) ( ) a =, b = ( ) () c =a +b = Geometrically, vector addition can be represented by the parallelogram law which is that the sum vector(a+b) corresponds to the directed line segment along the diagonal of the parallelogram havingaandbassides.anotherwaytosaythisistomoveacopyofthevector a parallelto a sothatits tail restsattheheadofvector b,andletthegeometricalvectorconnectingtheorigin andtheheadofthisshifted a bethevectora+b.yetanotherwaytothinkofthisistomoveinthe directionanddistancedefinedbyafromthetipofb.considerfigurewhere ( ( ) a =, b = ) ( ) () 7 c =a +b = 6

5 GEOMETRY OF MATRICES x FIGURE. Vector Addition 6 7,6 4, c, Copy of a a b x Thecopyofaextendsfromthetipofbtothepoint(7,6). Theparallelogramisformedbyextendingacopyofbfromthetipofawhichalsoendsatthepoint(7,6).Thelinesegmentfromthe origintothispoint,whichisthediagonaloftheparallelogramisthevectorc..4..subspaces. Asubspaceofavectorspaceisasetofvectors(including)thatsatisfiestwo requirements:ifaandbarevectorsinthesubspaceandcisanyscalar,then :a+bisinthesubspace :caisinthesubspace Note that a subspace necessarily contains the zero vector Subspaces and linear combinations. A subspace containing the vectors a and b must contain all linear combinations of a and b..4..columnspaceofamatrix.thecolumnspaceofthem nmatrixaconsistsofalllinearcombinationsofthecolumnsofa.thecombinationscanbewrittenasax.thecolumnspaceofais asubspaceofr m becauseeachofthecolumnsofahasmrows.thesystemofequationsax=bis solvableifandonlyifbisinthecolumnspaceofa.whatthismeansisthatthesystemissolvable ifthereissomewaytowritebasalinearcombinationofthecolumnsofa.considerthefollowing examplewhereaisaxmatrix,xisaxvectorandbisaxvector. A = 4,x = ( x x ) b = b (4) b b

6 6 GEOMETRY OF MATRICES ThematrixAhastwocolumns.LinearcombinationsofthesecolumnswilllieonaplaneinR. AnyvectorsbwhichlieonthisplanecanbewrittenaslinearcombinationsofthecolumnsofA. OthervectorsinR cannotbewritteninthisfashion.forexampleconsiderthevectorb =[,,8]. Thiscanbewrittenasfollows A = ( ) 4 = 8 () 4 + () = () 8 wherethevectorx =[,]. Thevectorb =[,,9]canalsobewrittenasalinearcombinationofthecolumnsofA.Specifically, A = ( ) 4 = 9 () 4 + () = (6) 9 wherex =[,]. Tofindthecoefficientsxthatallowustowriteavectorbasalinearcombinationofthecolumns ofawecanperformrowreductionontheaugmentedmatrix[a,b].forthisexampleweobtain à = 4 (7) 9 Multiplythefirstrowby4toyield andsubtractitfromthesecondrow ( 4 ) 4 4 Thiswillgiveanewmatrixonwhichtooperate. à = (8) 9 Multiplythefirstrowby

7 GEOMETRY OF MATRICES 7 ( 6 ) and subtract from the third row This will give Nowdividethesecondrowby Nowmultiplythesecondrowby 9 6 à = à = (9) () ( ) and subtract from the third row This will give à = () Atthispointwehaveanidentitymatrixintheupperleftandknowthatx =andx =.The thirdequationimpliesthat()(x )+()(x )=.Sothevectorb =[,,9]canbewrittenaslinear combination of the columns of A. Nowconsiderthevectorb =[,,].Writeouttheaugmentedmatrixsystemandperform row operations. Multiplythefirstrowby4 and subtract from the second row à = 4 () ( 4 8 )

8 8 GEOMETRY OF MATRICES This will give Multiplythefirstrowby à = () ( 4 ) and subtract from the third row This will give Nowdividethesecondrowby Nowmultiplythesecondrowby 4 6 à = à = (4) 6 () 6 ( ) and subtract from the third row This will give 6 4 à 4 = Nowmultiplythethirdrowby/4.Thiswillgive (6) 4

9 GEOMETRY OF MATRICES 9 à = / (7) Atthispointwehaveaproblembecausethethirdrowofthematriximpliesthat()(x )+()(x ) =,whichisnotpossible. Considernowthegeneralcasewhereb =[b,b,b]. Theaugmentedmatrixsystemcanbe written à = b 4 b (8) b Multiplythefirstrowby4 and subtract from the second row ( 4 4b ) This will give Multiplythefirstrowby 4 b 4 4b à = b 4b b b 4b (9) b ( b ) and subtract from the third row This will give Nowdividethesecondrowby b b à = à = b b b b 4b () b b b b 4b () b b

10 GEOMETRY OF MATRICES Nowmultiplythesecondrowby ( b 4b ) and subtract from the third row b b b 4b b b +b This will give b à 4 = b 4b () b b +b Wehaveanidentitymatrixintheupperlefthandcorner.Thegeneralsolutionisoftheform x =b b x = b 4b b +b +b = Consider first the case from equation. For this system b = b b = ( ) b, x = b 4b = 8 If we then substitute we obtain ( ) 4 = ( ) () (4) Now consider the case from equation 6 b = b b =, x = b 9 If we then substitute we obtain b +b +b = +8+ = () ( b b 4b ) = ( ) = ( ) (6) b +b +b = +9+6 = (7) Soforthesystemsinequationsand6,thegivenbvectorisinthecolumnspaceofthematrix Aandcanbewrittenasalinearcombinationofthesecolumnswithcoefficients(,)and(,) respectively. For the system in equation we have b = b = b b If we then substitute we obtain, x = ( b b 4b ) = ( ) 8 = ( ) (8) b +b +b = + +4 = (9)

11 GEOMETRY OF MATRICES TheconditionontheelementsofbisnotsatisfiedandsobcannotbewrittenasalinearcombinationofthecolumnspaceofA.Thefirsttwoelementsarecorrectlywrittenascombinationsofthe firsttwoelementsofeachcolumnofa,butthelastelementofthelinearcombinationis6rather than.thatis () 4 + (/) =.4.6.Nullspaceofamatrix.Thenullspaceofanm nmatrixaconsistsofallsolutionstoax=. ThesevectorsxareinR n.theelementsofxarethemultipliersofthecolumnsofawhoseweighted sumgivesthezerovector.thenullspacecontainingthesolutionsxisdenotedbyn(a).thenull spaceisasubspaceofr n whilethecolumnspaceisasubspaceofr m.formanymatrices,theonly solutiontoax=isx=. Ifn>m,thenullspacewillalwayscontainvectorsotherthanx=. Consider a few examples. ( ) :ConsiderthematrixA=.Multiplythefirstrowby 6 ( ) 6 andsubtractitfromthesecondrow Thiswillgivethenewmatrix. Ã = 6 6 ( ) Thisimpliesthatx canbeanything.theconventionistosetitequaltoone.ifx =, thenwehave so x + ()() = x = ( ( ( ( ) + () = ) 6) ) ( ) ThenullspaceofthematrixAisthevector. Allvectorsofthisformwherethe firstelemementis-timesthesecondelementwillmaketheequationax=true. ( ) :ConsiderthematrixA=.Multiplythefirstrowby 4 ( ) 6 andsubtractitfromthesecondrow

12 GEOMETRY OF MATRICES Thiswillgivethenewmatrix. 6 4 à = ( ) Thisimpliesthatx mustbezero.ifx =,thenwehave so x + ()() = x = ( ( ( () + () = ) 4) ) ThenullspaceofthematrixAisthevector equationax=isx=. ( ).Theonlyvectorxthatwillmakethe : Consider the matrix Multiplythefirstrowby A = ( ) andsubtractitfromthesecondrow Thiswillgivethenewmatrix à = Nowmultiplythefirstrowby ( 6 9 ) andsubtractitfromthethirdrow

13 GEOMETRY OF MATRICES Thiswillgivethenewmatrix à = 4 Nowsubtractthesecondrowfromthethirdrow Thiswillgivethenewmatrix. à 4 = Ifwewritethesystemusingà 4 weobtain 4 4 x x x = x 4 Thevariablesx andx 4 canbeanyvaluesbecauseofthezeroesinthepivotcolumns. Theconventionistosetthemequaltozeroandone.Firstwritethesystemwithx =and andx 4 =.Thiswillgive 4 x x = Thisimpliesthatx mustbezero.ifx =,thenwehave 4 x = x + = x =

14 4 GEOMETRY OF MATRICES Thevectorx = the system as follows isinthenullspaceofthematrixaascanbeseenbywritingout ( ) + () 4 + () 8 + () 4 = 6 4 Nowwritethesystemwithx =andandx 4 =.Thiswillgive 4 x x = Thisimpliesthatx +mustbezero.thisimpliesthatx =-.Makingthissubstitution we have 4 x = x +4 = x = The vector isinthenullspaceofthematrixaascanbeseenbywritingoutthe system as follows ( ) + () 4 + ( ) 8 + () 4 = 6 4 Wecanthenwriteageneralxasfollows x =x +x 4 = 4: Consider the matrix Multiplythefirstrowby A = ( 4 6 ) x x 4 x x 4 x 4

15 GEOMETRY OF MATRICES andsubtractitfromthesecondrow Thiswillgivethenewmatrix. Ã = Nowmultiplythefirstrowby ( ) 6 9 andsubtractitfromthethirdrow Thiswillgivethenewmatrix. Ã = Nowmultiplythefirstrowbynegativeoneandsubtractitfromthefourthrow 7 8 Thiswillgivethenewmatrix. Ã 4 = Nowmultiplythesecondrowbyoneandsubtractitfromthethirdrow Thiswillgivethenewmatrix. 8 8

16 6 GEOMETRY OF MATRICES à = 8 8 Nowmultiplythesecondrownegativeoneandsubtractitfromthefourthrow Thiswillgivethenewmatrix. 8 8 à 6 = Ifwewritethesystemusingà 6 weobtain 8 8 x x x x 4 = x +x +x +x 4 = 8x 4 = Thisimpliesthatx 4 =.Thesystemhasthreevartiablesandonlyoneequation.Assume thatx andx arefreevariablesgiventhezeroesinthepivotcolumns.setx =andx = to obtain So the vector system as follows x +x + = x + = x = isinthenullspaceofthematrixaascanbeseenbywritingoutthe ( ) + () () () = 7 Nowsetx =andx =toobtain

17 GEOMETRY OF MATRICES 7 So the vector system as follows.. Basis vectors. ( ) x +x +x = x + = x = isinthenullspaceofthematrixaascanbeseenbywritingoutthe + () () Wecanthenwriteageneralxasfollows x =x +x 6 9 = + () = 7 x x x x...definitionofabasis. Asetofvectorsinavectorspaceisabasisforthatvectorspaceifany vectorinthevectorspacecanbewrittenasalinearcombinationofthem.theminimumnumber ofvectorsneededtoformabasisforr k isk.forexample,inr,weneedtwovectorsoflengthtwo toformabasis.onevectorwouldonlyallowforotherpointsinr thatliealongthelinethrough that vector....exampleforr.considerthefollowingthreevectorsinr a = [ ] a a,a = [ ] a, b = a [ b b ]. () Ifa anda areabasisforr,thenwecanwriteanyarbitraryvectorasalinearcombinationof them.thatiswecanwrite [ ] [ ] [ ] a a b α + α a =. () a b Toseetheconditionsunderwhicha anda formsuchabasiswecansolvethissystemfor α and α asfollows.appendingthecolumnvectorbtothematrixa,weobtaintheaugmented matrixforthesystem.thisiswrittenas [ ] a a à = b () a a b Wecanperformrowoperationsonthismatrixtoreduceittoreducedrowechelonform.Wewill dothisinsteps.thefirststepistodivideeachelementofthefirstrowbya.thiswillgive [ ] a b à = a a () a a b

18 8 GEOMETRY OF MATRICES Nowmultiplythefirstrowbya toyield [ a a a a a b a ] andsubtractitfromthesecondrow a a b a a a a b a a a a a a b a b a Thiswillgiveanewmatrixonwhichtooperate. à = Nowmultiplythesecondrowby [ a a a a a a a a b a b b a a a a a a a toobtain ] à = [ a b a a a b a b a a a a ] (4) Nowmultiplythesecondrowby a a andsubtractitfromthefirstrow.firstmultiplythesecond rowby a a toyield. [ ] a a (a b a b ) a a (a a a a ) () Nowsubtracttheexpressioninequationfromthefirstrowofà toobtainthefollowingrow. [ ] [ a b a a ] [ a a (a b a b ) a = a (a a a a ) ] b a a (a b a b ) a (a a a a ) (6) Nowreplacethefirstrowinà withtheexpressioninequation6ofobtainã 4 à 4 = b a a (a b a b ) a (a a a a ) a b a b a a a a (7) Thiscanbesimplifiedasbyputtingtheupperrighthandtermoveracommondenominator, and canceling like terms as follows

19 GEOMETRY OF MATRICES 9 Ã 4 = = b a a b a a a a a b +a a a b a (a a a a ) b a a a a b a (a a a a ) a b a b a a a a a b a b a a a a (8) = b a a b a a a a a b a b a a a a Wecannowreadoffthesolutionsforx andx.theyare x = b a a b a a a a (9) x = a b a b a a a a Aslongasthisdeterminantisnotzero,wecanwritebasafunctionofa anda.ifthedeterminantofaisnotzero,thena anda arelinearlyindependent.geometricallythismeansthatthey donotlieonthesameline.soinr,anytwovectorsthatpointindifferentdirectionsareabasis for the space....lineardependence.asetofvectorsislinearlydependentifanyoneofthevectorsintheset can be written as a linear combination of the others. The largest number of linearly independent vectorswecanhaveinr k isk...4. Linear independence. A set of vectors is linearly independent if and only if the only solution to is α a + α a + + α k a k = (4) α = α = = α k = (4) Wecanalsowritethisinmatrixform.ThecolumnsofthematrixAareindependentiftheonly solutiontotheequationax=isx=....spanningvectors.thesetofalllinearcombinationsofasetofvectorsisthevectorspace spannedbythosevectors. Bythiswemeanallvectorsinthisspacecanbewrittenasalinear combinationofthisparticularsetofvectors.considerforexamplethefollowingvectorsinr. a =,a =,a = (4)

20 GEOMETRY OF MATRICES Thevectora canbewrittenasa +a.butvectorsinr havinganon-zerothirdelementcannot bewrittenasacombinationofa anda.thesethreevectorsdonotformabasisforr.theyspan thespacethatismadeupofthe floor ofr.similarlythetwovectorsfromequation4makeup abasisforaspacedefinedastheplanepassingthroughthosetwovectors...6.rowspaceofamatrix.therowspaceofthem nmatrixaconsistsofalllinearcombinationsof therowsofa.thecombinationscanbewrittenasx AorA xdependingonwhetheroneconsiders theresultingvectorstoberowsorcolumns.therowspaceofaisasubspaceofr n becauseeachof therowsofahasncolumns.therowspaceofamatrixisthesubspaceofr n spannedbytherows...7.linearindependenceandthebasisforavectorspace.abasisforavectorspacewithkdimensions isanysetofklinearlyindependentvectorsinthatspace...8.formalrelationshipbetweenavectorspaceanditsbasisvectors.abasisforavectorspaceisa sequence of vectors that has two properties simultaneously. : The vectors are linearly independent :Thevectorsspanthespace Therewillbeoneandonlyonewaytowriteanyvectorinagivenvectorspaceasalinear combinationofasetofbasisvectors. Thereareaninfinitenumberofbasisvectorsforagiven space,butonlyonewaytowriteanygivenvectorasalinearcombinationofaparticularbasis...9.basesandinvertiblematrices. Thevectors ξ, ξ,..., ξ n areabasisforr n exactlywhentheyare thecolumnsofann ninvertiblematrix.thereforer n hasinfinitelymanybases,oneassociated with every different invertible matrix....pivotsandbases. Whenreducinganm nmatrixatorow-echelonform,thepivotcolumns formabasisforthecolumnspaceofthematrixa.thepivotrowsformabasisfortherowspaceof the matrix A....Dimensionofavectorspace.Thedimensionofavectorspaceisthenumberofvectorsinevery basis.forexample,thedimensionofr is,whilethedimensionofthevectorspaceconsistingof pointsonaparticularplaneisr isalso....dimensionofasubspaceofavectorspace.thedimensionofasubspacespaces n ofanndimensionalvectorspacev n isthemaximumnumberoflinearlyindependentvectorsinthesubspace.... Rank of a Matrix. :Thenumberofnon-zerorowsintherowechelonformofanm nmatrixaproducedby elementaryoperationsonaiscalledtherankofa. :Therankofanm nmatrixaisthenumberofpivotcolumnsintherowechelonformof A. :Thecolumnrankofanm nmatrixaisthemaximumnumberoflinearlyindependent columns in A. 4:Therowrankofanm nmatrixaisthemaximumnumberoflinearlyindependentrows ina :Thecolumnrankofanm nmatrixaisequaltotherowrankofthem nmatrixa.this commonnumberiscalledtherankofa. 6:Ann nmatrixawithrank=nissaidtobeoffullrank.

21 GEOMETRY OF MATRICES..4.DimensionandRank.Thedimensionofthecolumnspaceofanm nmatrixaequalsthe rankofa,whichalsoequalsthedimensionoftherowspaceofa.thenumberofindependentcolumns ofaequalsthenumberofindependentrowsofa.asstatedearlier,thercolumnscontainingpivotsin therowechelonformofthematrixaformabasisforthecolumnspaceofa....vectorspacesandmatrices.anm nmatrixawithfullcolumnrank(i.e.,therankofthe matrix is equal to the number of columns) has all the following properties. : The n columns are independent :TheonlysolutiontoAx=isx=. :Therankofthematrix=dimensionofthecolumnspace=n 4:Thecolumnsareabasisforthecolumnspace...6. Determinants, Minors, and Rank. Theorem. Therankofanm nmatrixaiskifandonlyifeveryminorinaoforderk +vanishes, whilethereisatleastoneminoroforderkwhichdoesnotvanish. Proposition.Consideranm nmatrixa. :deta=ifeveryminorofordern-vanishes. :Ifeveryminorofordernequalszero,thenthesameholdsfortheminorsofhigherorder. (restatement of theorem): The largest among the orders of the non-zero minors generated byamatrixistherankofthematrix...7.nullityofanm nmatrixa.thenullspaceofanm nmatrixaismadeupofvectorsin R n andisasubspaceofr n.thedimensionofthenullspaceofaiscalledthenullityofa.itisthe the maximum number of linearly independent vectors in the nullspace...8.dimensionofrowspaceandnullspaceofanm nmatrixa. Consideranm nmatrixa.the dimension of the nullspace, the maximum number of linearly independent vectors in the nullspace, plustherankofa,isequalton.specifically, Theorem. rank(a) + nullity(a) = n (4) Proof.Letthevectors ξ, ξ,...,ξ k beabasisforthenullspaceofthem nmatrixa.thisisasubset ofr n withnorlesselements.thesevectorsare,ofcourse,linearlyindependent.thereexistvectors ξ k+, ξ k+,..., ξ n inr n suchthat ξ, ξ,...,ξ k, ξ k+,...,ξ n formabasisforr n.wecanprovethat A ( ξ k+ ξ k+ ξ n ) = ( Aξk+ Aξ k+... Aξ n ) isabasisforthecolumnspaceofa.thevectorsaξ,aξ,...,aξ n spanthecolumnspaceofa because ξ, ξ,...,ξ k, ξ k+,...,ξ n formabasisforr n andthuscertainlyspanr n.becauseaξ j = forj k,weseethataξ k+,aξ k+,...,aξ n alsospanthecolumnspaceofa.specificallyweobtain A ( ξ ξ ξ k ξ k+... ξ n ) = (... Aξk+ Aξ k+... Aξ n ) We need to show that these vectors are independent. Suppose that they are not independent. Thenthereexistscalarsc i suchthat Wecanrewrite44asfollows n c i (A ξ i ) = (44) i=k+

22 GEOMETRY OF MATRICES A ( n i=k+ c i ξ i ) = (4) This imples that the vector n ξ = i=k+ c i ξ i (46) isinthenullspaceofa.because ξ, ξ,...,ξ k formabasisforthenullspaceofa,theremustbe scalars, γ, γ,...,γ k suchthat k ξ = γ i ξ i (47) i= Ifwesubstractoneexpressionfor ξfromtheother,weobtain ξ = k γ i ξ i i= n c i ξ i = (48) j=k+ andbecause ξ, ξ,...,ξ k, ξ k+,..., ξ n arelinearlyindependent,wemusthave γ, = γ =... = γ k =c k+ =c k+ =... =c n = (49) IfristherankofA,thefactthatAξ k+ Aξ k+... Aξ n formabasisforthecolumnspaceofa tellsusthatr=n-k.becausekisthenullityofathenwehave rank(a) + nullity(a) = n n k +k =n ().. Orthogonal vectors.. PROJECTIONS...Definitionoforthogonalvectors.Twovectorsaandbaresaidtobeorthogonaliftheirinner productiszero,thatisifa b =.Geometricallytwovectorsareorthogonaliftheyareperpendicular to each other....exampleinr. Considerthetwounitvectors [ [ a =,b = ] ] a b = [ ] [ ] = () () + () () = Consider two different vectors that are also orthogonal [ [ ] a =,b = ] a b = [ ] [ ] = () ( ) + () () = Wecanrepresentthissecondcasegraphicallyasinfigure6 () ()

23 GEOMETRY OF MATRICES FIGURE6. OrthogonalVectorsinR x,, x...anexampleinr.considerthetwovectors a =,b = a b = [ ] = () () + () () + ( )() = Wecanrepresentthisthreedimensionalcaseasinfigure7oralternativelyasinfigure8. ()..4.AnotherexampleinR.Considerthethreevectors 4 a =,b =,c = c a = [ 4/9 /9 9 ] = ( ) ( ) ( 9 ) ( ) + ( 9 ) ( ) = (4)

24 4 GEOMETRY OF MATRICES FIGURE7. OrthogonalVectorsinR x x x,, Hereaandcareorthogonalbutaandbareobviouslynot.Wecanrepresentthisthreedimensionalcasegraphicallyinfigure9.Notethatthelinesegmentdisparalleltocandperpendicular toa,justascisorthogonaltoa....anotherexampleinr. Considerthefourvectorsbelowwherepisascalarmultipleofa. Wecanseethateisisorthogonaltop. 6 a =,b =,p =,e = 6 e p = [ ] = e a = [ ] = ( ) ( ) ( ) () + andtoa.thisexampleisrepresentedinfigure. + ( ) ( ) 6 () + () () = ( ) () + () () = (6)

25 GEOMETRY OF MATRICES FIGURE8. OrthogonalVectorsinR x,, x x.. Projection onto a line....ideaanddefinitionofaprojection.consideravectorb =[b,b,...b n ]inr n.alsoconsider alinethroughtheorigininr n thatpassesthroughthepointa=[a,a,... a n ]. Toprojectthe vectorontothelinea,wefindthepointonthelinethroughthepointathatisclosesttob. We findthispointonthelinea(callitp)byfindingthelineconnectingbandthelinethroughathat isperpendiculartothelinethrougha.thispointp(whichisonthelinethrougha)willbesome multipleofa,i.e. p=cawherecisascalar. Considerfigure. Theideaistofindthepoint alongthelineathatisclosesttothetipofthevectorb.inotherwords,findthevectorb-pthatis perpendiculartoa.giventhatpcanbewrittenasscalarmultipleofa,thisalsomeansthatb-cais perpendicularororthogonaltoa.thisimpliesthata (b-ca)=,ora b-ca a=.giventhata band a aarescalarswecansolvethisforthescalarcasinequation7. a (b ca) = a b ca a = c = a b =a b a a a a Definition.Theprojectionofthevectorbontothelinethroughaisthevectorp = ca = a b a a a. (7)

26 6 GEOMETRY OF MATRICES FIGURE9. OrthogonalVectorsinR. x.. d a b. x,, c. x...anexample.projectthevectorb =[,,]ontothelinethrougha =[,,]. Firstfindthe scalar needed to construct the projection p. For this case a =,b = a b = [,,] a a = [,,] = = (8) Theprojectionpisthengivenbycaor c = a b a a =

27 GEOMETRY OF MATRICES 7 FIGURE. OrthogonalVectorsinR a b.. p e x x.,,. x p = ca = [ ] = 6 (9) Thedifferencebetweenbandp(theerrorvector,ifyouwill)isb-p.Thisiscomputedasfollows. e = b p = 6 = (6) Wecanthinkofthevectorbasbeingsplitintotwoparts,thepartalongthelinethrougha,that isp,andthepartperpendiculartothelinethrougha,thatise.wecanthenwriteb=p+e.... The projection matrix. We can write the equation for the projection p in another useful fashion.

28 8 GEOMETRY OF MATRICES FIGURE. ProjectionontoaLine a b p b. x. x p.,, e. x p =ac =a a b a a = aa a a b =Pb wherep = aa a a We call the matrix P the projection matrix. (6)..4.Exampleprojectionmatrix.Considerthelinethroughthepointawherea=[,,].Wefind theprojectionmatrixforthislineasfollows.firstfinda a. a =,a a = [,,] = 9 (6) Thenfindaa.

29 GEOMETRY OF MATRICES 9 Then construct P as follows aa = [,,] = P = (6) 4 4 (64) Considertheprojectionofthepointb=[,,]ontothelinethrougha.Weobtain p =Pb This is represented in figure. = = (6) FIGURE. ProjectionontoaLine x.. x. x. b p b e a p,,

30 GEOMETRY OF MATRICES.. Projection onto a more general subspace....ideaanddefinition.consideravectorb =[b,b,...b m ]inr m.thenconsiderasetofnmx vectors a, a,... a n.thegoalistofindthecombinationc a +c a +...+c n a n thatisclosestto the vector b. This is called the projection of vector b onto the n-dimensional subspace spanned by the a s.ifwetreatthevectorsascolumnsofamatrixa,thenwearelookingforthevectorinthe columnspaceofathatisclosesttothevectorb,i.e.,thepointc,suchthatbandacareclosest.we findthepointinthissubspacesuchthattheerrorvectorb-acisperpendiculartothesubspace. Thispointissomelinearcombinationofvectors a, a,... a n..considerfigurewhereweplotthe vectorsa =[,,]anda =[,,]. FIGURE. TwoVectorsinR x. x. a a.. x,,

31 GEOMETRY OF MATRICES ThesubspacespannedbythesetwovectorsisaplaneinR.Theplaneisshowninfigure4. FIGURE4. TwoVectorsonaPlaneinR. a x. a. x.,, x

32 GEOMETRY OF MATRICES Nowconsiderathirdvectorgivenbyb=[6,,].Werepresentthethreevectorsinfigure. FIGURE. AVectorinR notontheplane b x. a a 4 6. x. x.,, Thisvectordoesnotlieinthespacespannedbya anda.theideaistofindthepointinthe subspacespannedbya anda thatisclosesttothepoint(6,,).theerrorvectore=b-acwillbe perpendiculartothesubspace.infigure6thevector(labeleda )inthespacespannedbya and a thatisclosesttothepointbisdrawn.thediagramalsoshowsthevectorb-a....formulafortheprojectionpandtheprojectionmatrixp.theerrorvectore=b-acwillbe orthogonaltoeachofthevectorsinthecolumnspaceofa,i.e.,itwillbeorthogonalto a, a,... a n. Wecanwritethisasnseparateconditionsasfollows. a (b Ac) = a (b Ac) =. a n(b Ac) = (66) Wecanalsowritethisasamatrixequationinthefollowingmanner.

33 GEOMETRY OF MATRICES FIGURE 6. Projection onto a Subspace 6 x 4 x.. a a,, a b p b. x. A (b Ac) = A b A Ac = A Ac =A b Equation67canthenbesolvedforthecoefficientscbyfindingtheinverseofA A.Thisisgiven by (67) Theprojectionp=Acisthen A Ac =A b c = (A A) A b (68) p =Ac Theprojectionmatrixthatproducesp=Pbis =A(A A) A b (69) P = A(A A) A (7) Givenanym nmatrixaandanyvectorbinr m,wecanfindtheprojectionofbontothe columnspaceofabypremultiplyingthevectorbbythisprojectionmatrix.thevectorbissplitinto

34 4 GEOMETRY OF MATRICES twocomponentsbytheprojection.thefirstcomponentisthevectorpwhichliesinthesubspace spannedbythecolumnsoftheamatrix. Thesecondcomponentisthevectore,whichisthe differencebetweenpandb.theideaisthatwegettobbyfirstgettingtopandthenmovingina perpendiculardirectionfromptobasinfigure6....example.projectthevectorb =[6,,]ontothespacespannedbythevectorsa =[,,]and a =[,,].Thematricesareasfollows. A = A AandA Bareasfollows,A = [ ],b = [ ] A A = [ ] = [ ] A b = 6 [ 6 = ] TocomputecweneedtofindtheinverseofA A.Wecandosoasfollows Wecannowsolveforc (A A) = TheprojectionpisthengivenbyAc = [ ] = [ 6 ] [ ] 6 (7) (7) (7) [ 6 c = (A A) A ] [6 ] b = [ ] (74) = andwhilee=b-pis p =Ac = = [ ] (7)

35 GEOMETRY OF MATRICES e =b p = 6 = (76) Thissolvestheproblemforthisparticularvectorb.Forthemoregeneralproblemisitusefulto construct the projection matrix P. P =A (A A) A = [ 6 ] [ ] 6 [ ] = = Noticethateisperpendiculartobotha anda. a e = [,, ] a e = [,, ] = = Wecanthinkofthevectorbasbeingsplitintotwoparts,thepartinthespacespannedbythe columnsofa,thatisp,andthepartperpendiculartothissubspace,thatise.wecanthenwriteb =p+e.alsonotethatpb=p Pb =p 6 =.4. Projections, linear independence, and inverses. To find the coefficients c, the projection p, andtheprojectionmatrixp,weneedtobeabletoinvertthematrix,a A. Theorem.ThematrixA AisinvertibleifandonlyifthecolumnsofAarelinearlyindependent. (77) (78) (79)