Revew of lner lgebr Nuno Vsconcelos UCSD
Vector spces Defnton: vector spce s set H where ddton nd sclr multplcton re defned nd stsf: ) +( + ) (+ )+ 5) λ H 2) + + H 6) 3) H, + 7) λ(λ ) (λλ ) 4) H, - + 8) λ(+ ) λ + λ 9) (λ+λ ) λ + λ the cnoncl emple s R d wth stndrd vector ddton nd sclr multplcton e d e d α + e e e 2 e 2
Vector spces but there re much more nterestng emples e.g., the spce of functons f:x R wth (f+g)() f() + g() R d s vector spce of fnte dmenson f (f,..., f d ) when d goes to nfnt we hve functon f f(t) the spce of functons s n nfnte dmensonl vector spce (λf)() λf()
Dt n ths course we wll tlk lot bout dt n ll cses dt wll be represented n vector spce: n emple s rell just pont ( dtpont ) on such spce from bove we know how to perform bsc opertons on dtponts ths s nce, becuse dtponts cn be qute bstrct e.g. mges: n mge s functon on the mge plne t ssgns color f(,) to ech ech mge locton (,) the spce Ψ of mges s vector spce (note: ssumes tht mges cn be negtve) ths mge s pont on Ψ
Imges becuse of ths we cn mnpulte mges s we would mnpulte vectors e.g. I wnt to morph (,) nto b(,): ths s the pth long the lne from to b: c(α) + α(b-) (-α) + α b for α we hve for α we hve b for α n (,) we hve pont on the lne between nd b to morph we cn smpl ppl the sme rule to the mges! b b- α(b-)
Imges when we mke c(,) (-α) (,) + α b(,) we get mge morphng α α.2 α.4 b b- α(b-) α.6 α.8 α the pont s tht ths s possble onl becuse we re n vector spce
Imges note tht mges cn lso be represented s ponts n R d smple mge on fnte grd to get rr of pels (,) (,j) mges re lws stored lke ths on computers stck ll the rows nto vector, e.g. 3 3 mge s converted nto 9 vector n generl n m mge s trnsformed nto nm vector note tht ths s et nother vector spce the pont s tht there re multple vector spces nto whch the dt cn be represented
Tet nother common tpe of dt s tet documents re represented b word counts: ssocte counter wth ech word slde wndow through the tet whenever the word occurs ncrement ts counter ths s the w serch engnes represent web pges
Tet e.g. word counts for three documents n certn corpus (onl 2 words shown for clrt) note tht: ech document s 2 dmensonl vector f I dd two word count vectors (documents), I get new word count vector (document) f I multpl word count vector (document) b sclr, I get word count vector note: once gn we ssume word counts could be negtve (to mke ths hppen we cn smpl subtrct the verge vlue) ths mens: we re once gn on vector spce (postve subset of R d ) document s pont n ths spce
Blner forms one reson to use vector spces s tht the llow us to mesure dstnces between dtponts we wll see tht ths s crucl for clssfcton the mn tool for ths s the dot-product to defne dot-product we frst need to recll the noton of blner form Defnton: blner form on vector spce H s mppng Q: H H R (, ) Q(, ) such tht,, H ) Q[(λ+λ ), ] λq(, ) + λ Q(, ) ) Q[,(λ+λ )] λq(,) + λ Q(, )
Dot products Defnton: dot-product on vector spce H s blner form <.,.>: H H R (, ) <, > such tht ) <,>, H ) <,> f nd onl f ) <,> <,> for ll nd note tht ) nd ) mke the dot-product nturl mesure of smlrt nothng cn be more smlr to thn tself ths becomes more precse wth ntroducton of norm
Dot products nd norms n dot-product defnes norm v 2 <,> the norm hs the followng propertes non-negtvt:, sclr multplcton: λ λ trngle neqult: + + ths defnes metrc d(,) - whch s mesure of the dstnce between nd lws remember tht the norm chnges wth the dotproduct!
Dot products bck to our emples: n R d the stndrd dot-product s whch leds to the stndrd Euclden norm the dstnce between two vectors s the stndrd Euclden dstnce d T, d T 2 d T d 2 ) ( ) ( ) ( ), (
Dot-products nd norms note tht ths mmedtel gves mesure of smlrt between web pges compute word count vector from pge, for ll dstnce between pge nd pge j s smpl j T d(, j ) j ( j ) ( j ) ths llows us to fnd, n the web, the most smlr pge to n gven pge ou would be mzed b how close to ths s the mesure of smlrt used b most serch engnes! wht bout functons, e.g. mges?
Dot products recll tht the spce of functons s n nfnte dmensonl vector spce the stndrd dot-product s the nturl etenson of tht n R d (just replce summtons b ntegrls) the norm becomes the energ of the functon f ( ), g( ) f ( ) g( ) d f ( ) 2 f ( ) d nd the dstnce between functons the energ of the dfference between them d( f ( ), g( )) 2 f ( ) g( ) [ f ( ) g( )] d
Bss we know how to mesure dstnces n vector spce nother nterestng propert s tht we cn full chrcterze the spce b one of ts bses set of vectors,, k re bss of vector spce H f the re lnerl ndependent c c, the spn H : for n v n H, v cn be wrtten s v c
Bss note tht b mkng the the columns of mtr X ths cn be compctl wrtten s lner ndependent: Xc c spn H v, c v Xc lso, ll bss of H hve the sme number of vectors, whch s clled the dmenson of H ths s vld for n vector spce!
Bss emple the bss of the vector spce of mges of fces the fgure onl show the frst 6 bss vectors but there ctull more these vectors re orthonorml
Orthogonlt two vectors re orthogonl f nd onl f ther dot product s zero 2π 2 2π e.g. sn sn( )cos( ) d 2 n the spce of functons defned on [,2π], cos() nd sn() re orthogonl two subspces V nd W re orthogonl f ever vector n V s orthogonl to ever vector n W set of vectors,, k s clled orthogonl f ll prs of vectors re orthogonl orthonorml f the vectors lso hve unt norm, j,, f f j j
tr m n mtr s n opertor tht mps vector from R m nto vector n R n e.g. the equton A, or sends n R n to n R m m m O n mn n e n e m A e e e 2 note tht there s nothng mgcl bout ths, t follows rther mechncll from the defnton of mtr-vector multplcton
tr vector multplcton consder A,.e. jn j j but we cn thnk of ths s where ( ) mens the th row of A hence the th component of s the dot product of ( ) nd s the projecton of on the spce spnned b the rows of A ( ) L n j j j n n n e e 2 e m A - m - - - m - 2-2
tr vector multplcton but there s more. Let A,.e. jn j j whch cn be seen s where wth bove nd below mens the th column of A hence s the th component of n the spce spnned b the columns of A s lner combnton of the columns of A n n n mn m n n n j j j m + + + + + + L L L e e 2 e m A n n R n
tr vector multplcton two lterntve pctures coordntes of n row spce of A (belongs to R m ) coordntes of n column spce of A (belongs to R n ) e e 2 e m A - m - - - m - 2-2 n n ( ) n n m + + L
A cool trck the mtr multplcton formul C AB lso pples to block mtrces when these re defned properl for emple, f A,B,C,D,E,F,G,H re mtrces c j k k b kj A C BE D G F H AE + BG CE + DG AF CF + + BD DH onl but mportnt cvet: the szes of A,B,C,D,E,F,G,H hve to be such tht the ntermedte opertons mke sense!
tr vector multplcton ths mkes t es to derve the two lterntve pctures the row spce pcture s just lke sclr multplcton, wth blocks ( -) nd the column spce pcture s just dot product, wth blocks nd the columns of A ( ) ( ) L n n n n n { { ( ) ( ) n m m n n n n L L
Orthogonl mtrces mtr s clled orthogonl f t s squre nd hs orthonorml columns mportnt propertes: ) the nverse of n orthogonl mtr s ts trnspose ths cn be esl shown wth the block mtr trck 2) n orthogonl mtr s rotton mtr ths follows from the fct tht t does not chnge the norms of the vectors on whch t opertes ( ) K O K K K n j n T T A A ) ( ) ( A A A A A T T T T
Rotton mtrces the combnton of. opertor nterpretton 2. block mtr trck s useful n mn stutons poll: wht s the mtr R tht rottes R 2 b θ degrees? e 2 e 2 e θ e
Rotton mtrces the ke s to consder how the mtr opertes on the vectors of the cnoncl bss note tht R sends e to e e 2 e' r r 2 usng the column spce pcture e' r r r r 2 22 r + r 2 2 22 r r from whch we hve the frst column of the mtr 2 e 2 sn θ θ cos θ e e R e' r r 2 22 cosθ snθ r r 2 22
Rotton mtrces nd we do the sme for e 2 R sends e 2 to e 2 e' r r r r r r + r 2 2 2 2 r22 2 22 r r 2 22 from whch R cosθ snθ snθ [ e e' ] ' 2 cosθ check cos θ e cosθ snθ cosθ snθ R T R I snθ cosθ snθ cosθ e 2 -sn θ e 2 cos θ sn θ θ θ e
Anlss/snthess one nterestng cse s tht of mtrces wth orthogonl columns note tht, n ths cse, the columns of A re bss of the column spce of A bss of the row spce of A T ths leds to n nterestng nterpretton of the two pctures the projecton of nto the column spce of A s equl to tht nto the row spce of A T A T cn then be sntheszed b usng the column spce pcture A Q: s the sntheszed equl to?
Projectons A: not necessrl! Recll A T nd A f nd onl f AA T I! ths mens tht A hs to be orthonorml. wht hppens when ths s not the cse? we get the projecton of on the column spce of A e.g. let then nd A 2 3 2 + ' 2 2 2 e e 2 e 3 column spce of A row spce of A T
Null spce wht hppens to the prt tht s lost? ths s the null spce of A T N ( T ) { T A A } n the emple ll vectors of the tpe snce A T α α note tht N(A) s (lws!) orthogonl to row spce of A: s n null spce f t s orthogonl to ll rows of A here ths mens tht N(A T ) s orthogonl to the column spce of A α e 3 e e 2 column spce of A row spce of A T null spce of A T
Orthonorml mtrces Q: wh s the orthonorml cse specl? becuse here there s no null spce of A T recll tht for ll n N(A T ) A T A the onl vector n the null spce s ths mkes sense: A hs n orthonorml columns, e.g. A these spn ll of R n there s no etr room for n orthogonl spce the null spce of A T hs to be empt the projecton nto row spce of A T (column spce of A) s the vector tself n ths cse, we s tht the mtr hs full rnk
The three fundmentl spces there re ctull four, but here we re onl concerned wth three these hold for n mtr column spce: spce spnned b the columns row spce: spce spnned b the rows null spce: spce of vectors orthogonl to ll rows (lso known s the orthogonl complement of the row spce) squre smmetrc mtrces (A A T ): column spce s the sme s row spce null spce orthogonl to column spce nn, orthonorml (A T A I) mtrces column spce row spce R n null spce {}
Squre mtrces n ths cse the two spces re n R n - n - n e m A 2-2 - - - e e 2 n n 2 2