Linear Algebra Concepts

Transcription

1 Ler Algebr Cocepts Ke Kreutz-Delgdo (Nuo Vscocelos) ECE 75A Wter 22 UCSD

2 Vector spces Defto: vector spce s set H where ddto d sclr multplcto re defed d stsf: ) +( + ) (+ )+ 5) l H 2) + + H 6) 3) H, + 7) l(l ) (ll ) 4) H, - + 8) l(+ ) l + l (lsclr;,, H ) 9) (l+l ) l + l the cocl emple s R d wth stdrd vector ddto d sclr multplcto e d + e d e e e 2 e 2

3 Vector spces But there re much more terestg emples E.g., the spce of fuctos f:x R wth (f + g)() f() + g() (lf)() lf() R d s vector spce of fte dmeso,e.g. f ( f,..., f d ) T Whe d goes to ft we hve fucto f f (t ) The spce of ll fuctos s fte dmesol vector spce

4 Dt Vector Spces I ths course we wll tlk lot bout dt d fetures Dt/fetures wll lws be represeted vector spce: emple s rell just pot ( dtpot ) o such spce from bove we kow how to perform bsc opertos o dtpots ths s ce, becuse dtpots c be qute bstrct e.g. mges: mge s fucto o the mge ple t ssgs color f(,) to ech ech mge locto (,) the spce Y of mges s vector spce (ote: ssumes tht mges c be egtve) ths mge s pot Y

5 Imges Becuse of ths we c mpulte mges b mpultg ther equvlet vector represettos E.g., Suppose oe wts to morph (,) to b(,): Oe w to do ths s v the pth log the le from to b. c() + (b-) (-) + b for we hve (,) for we hve b(,) for (,) we hve pot o the le betwee (,) d b(,) To morph mge we c smpl ppl ths rule to the mge vector represettos! b b- (b-)

6 Imges Whe we mke c(,) (-) (,) + b(,) we get mge morphg :.2.4 b b- (b-).6.8 The pot s tht ths s possble becuse we eplot the structure of vector spce.

7 Imges Imges re usull ppromted s pots R d Smple (dscretze) mge o fte grd to get rr of pels (,) (,j) Imges re lws stored lke ths o dgtl computers We c ow stck ll the rows (or colums) to vector. E.g. 3 3 mge c be coverted to 9 vector s follows: I geerl m mge vector s trsformed to m vector Note tht ths s et other vector spce The pot s tht there re geerll multple dfferet, but somorphc, vector spces whch the dt c be represeted

8 Tet Aother commo tpe of dt s tet Documets re represeted b word couts: ssocte couter wth ech word slde wdow through the tet wheever the word occurs cremet ts couter Ths s the w serch eges represet web pges

9 Tet E.g. word couts for three documets cert corpus (ol 2 words show for clrt) Note tht: Ech documet s d 2 dmesol vector If I dd two word cout vectors (documets), I get ew word cout vector (documet) If I multpl word cout vector (documet) b sclr, I get word cout vector Note: oce g we ssume word couts could be egtve (to mke ths hppe we c smpl subtrct the verge vlue) Ths mes: We re oce g vector spce (postve subset of R d ) A documet s pot ths spce

10 Bler forms Oe reso to use er product vector spces s tht the llow us to mesure dstces betwee dt pots We wll see tht ths s crucl for clssfcto The m tool for ths s the er product ( dot-product ). We c defe the dot-product usg the oto of bler form (ssumg rel vector spce). Defto: bler form o rel vector spce H s bler mppg Q: H H R (, ) Q(, ) B-ler mes tht ",, H ) Q[(l+l ), ] lq(, ) + l Q(, ) ) Q[,(l+l )] lq(,) + l Q(, )

11 Ier Products Defto: er product o rel vector spce H s bler form <.,. >: H H R (, ) <, > such tht ) <,>, " H ) <,> f d ol f ) <,> <,> for ll d The postve-defteess codtos ) d ) mke the er product turl mesure of smlrt Ths becomes more precse wth troducto of orm

12 Ier Products d Norms A er product duces orm v the ssgmet 2 <,> B defto, orm must obe the followg propertes Postve-defteess:, & ff Homogeet: l l Trgle Ieqult: + + A orm defes correspodg metrc d(,) - whch s mesure of the dstce betwee d Alws remember tht the duced orm chges wth dfferet choce of er product!

13 Ier Product Bck to our emples: IR d the stdrd (or uweghted) er product s Whch leds to the stdrd (uweghted) Euclde orm R d The dstce betwee two vectors s the stdrd (uweghted) Euclde dstce R d d T, d T 2 d T d 2 ) ( ) ( ) ( ), (

14 Ier Products d Norms Note, e.g., tht ths mmedtel gves mesure of smlrt betwee web pges compute word cout vector from pge, for ll dstce betwee pge d pge j c be smpl defed s: T d(, j ) j ( j ) ( j ) Ths llows us to fd, the web, the most smlr pge to gve pge j, t lest wth respect to ths smple metrc. I fct, ths s ver close to the mesure of smlrt used b most serch eges! Wht bout orms o fucto spces, s used to represet, e.g., mges d other cotuous vlued sgls?

15 Ier Products o Fucto Spces Recll tht the spce of fuctos s fte dmesol vector spce The stdrd (uweghted) er product s the turl eteso of tht R d (just replce summtos b tegrls) f ( ), g( ) f ( ) g( ) d The orm s relted to the eerg of the fucto 2 2 f ( ) f ( ) d The dstce betwee fuctos s relted to the eerg of the dfferece betwee them 2 2 d( f ( ), g( )) f ( ) g( ) [ f ( ) g( )] d

16 Bss Vectors We kow how to mesure dstces vector spce Aother terestg propert s tht we c usull full chrcterze vector spce b oe of ts bses A set of vectors,, k s bss of vector spce H f d ol f (ff) the re lerl depedet c c, " d the sp H : I.e., for v H, v c be wrtte s v c These two codtos me tht uquel represeted ths form. v c be

17 Bss Note tht B mkg the cocl represettos for the vectors the colums of mtr X, these two codtos c be compctl wrtte s Codto. The vectors re ler depedet: Xc c Codto 2. The vectors sp H " v, c such tht v Xc Also, ll bses of H hve the sme umber of vectors, whch s clled the dmeso of H Ths s vld for vector spce!

18 Bss emple A bss of the vector spce of mges of fces The fgure ol show the frst 6 bss vectors but there ctull more These vectors re orthoorml

19 Orthogolt Two vectors re orthogol ff ther er product s zero e.g. 2p 2 the spce of fuctos defed o [,2p], cos() d s() re orthogol Two subspces V d W re orthogol, V W, f ever vector V s orthogol to ever vector W set of vectors,, k s clled orthogol f ll prs of vectors re orthogol. orthoorml f ll of the orthogol vectors lso hve ut orm. 2p s s( )cos( ) d 2, j, f, f j j

20 Mtr m mtr represets ler opertor tht mps vector from the dom X R to vector the codom Y R m E.g. the equto A seds R to R m ccordg to X Y m m m e e 2 e e e m A

21 Mtr-Vector Multplcto I Cosder A,.e. j j j,,,m Ths s equvlet to j j j where ( ) mes the th row of A. Hece the th compoet of s the er product of ( ) d. (m rows) The m compoets of re obted b projectg oto (.e., tkg the er product wth) the m rows of A the dom spce e m A s cto X m - m - e 2 e

22 Mtr-Vector Multplcto II But there s more. Let A,.e. j j j, ow be wrtte s where wth bove d below mes the th colum of A. The compoet weghts the th colum of A codom ( colum spce spce sped b the colums of A). I.e, s ler combto of the colums of A the codom m m j j j m e e 2 e A mps from X to Y m m m

23 Mtr-Vector Multplcto I & II Thus there re two ltertve (dul) pctures of A: Coordtes of projected oto row spce of A (The X R vewpot) Dom X R e A m - m Dom X R vewpot - - ( rows) m e 2 e Codom Y R m vewpot Compoets of coordtes of log colums of A (Y R m vewpot)

24 Block Mtr Multplcto the mtr multplcto formul C AB c j lso pples to block mtrces whe these re defed to be coforml. for emple, f A,B,C,D,E,F,G,H re coforml mtrces, To be coforml mes tht the szes of the mtrces A,B,C,D,E,F,G,H hve to be such tht the termedte opertos mke sese! k A B E F AE BG AF BH C D G H CE DG CF DH k b kj

25 Mtr-Vector Multplcto I & II Ths mkes t es to derve the two ltertve pctures The row spce pcture (or vewpot): Sclr multplcto betwee the row blocks ( -) d The colum spce pcture (or vewpot): Ier products betwee blocks gve b the (sclr) blocks d the colum blocks of A. m m

26 Squre mtrces ths cse m d the row d colum subspces re both equl to (copes of) R - - e A e e 2 2 2

27 Orthogol mtrces A mtr s clled orthogol f t s squre d hs orthoorml colums. Importt propertes: ) The verse of orthogol mtr s ts trspose ths c be esl show wth the block mtr trck. (Also see lter.) T T A A j 2) A proper (det(a) ) orthogol mtr s rotto mtr ths follows from the fct tht t s utr,.e., does ot chge the orms ( szes ) of the vectors o whch t opertes, 2 T T T T 2 A ( A) ( A) A A, AND does NOT duce reflecto.

28 Rotto mtrces The combto of. opertor terpretto 2. block mtr trck s useful m stutos Emple: Wht s the mtr R tht rottes the ple R 2 b degrees? e 2 e

29 Rotto mtrces The ke s to cosder how the mtr opertes o the vectors e of the cocl bss ote tht R seds e to e e 2 e' r r 2 usg the colum spce pcture e' r r r r 2 22 r r r r 2 s cos e from whch we hve the frst colum of the mtr R e' r r 2 22 cos s r r 2 22

30 Rotto Mtrces d we do the sme for e 2 R seds e 2 to e 2 e' r r r r r r r r r r 2 22 from whch R cos s s e e' ' 2 cos check cos e cos s cos s R T R I s cos s cos -s e 2 cos s

31 Projectos Wht f A s ot orthogol? Cosder A T d A (Note tht ) for ll f d ol f AA T I! ths mes tht A hs to be orthogol to hve Wht hppes whe ths s ot the cse? The tke ECE 74!! E.g., f, the s dempotet (d lso obvousl smmetrc) so we get orthogol projecto of oto the colum spce of A e.g., let, the d A ' e e 2 e 3 colum spce of A row spce of A T 2 T T AA AA T AA colum s c ' p e

32 Null Spce of Mtr Wht hppes to the prt tht s lost? For the prevous emple ths prt belogs to the ull spce of A T T T A N A I the emple, ths s comprsed of ll vectors of the tpe sce A T FACT: N(A) s lws orthogol to the row spce of A: s the ull spce ff t s orthogol to ll rows of A For the prevous emple ths mes tht N(A T ) s orthogol to the colum spce of A e 3 e e 2 colum spce of A row spce of A T ull spce of A T

33 Orthogol Mtrces Cot. A orthogol mtr hs lerl depedet colums d therefore must hve verse. T Note tht A A I (prove erler) d the estece of verse AA I mples T T T A I A A AA A I A. Thus Ths mes tht T T A A AA I A hs orthoorml colums d rows Ech of these two sets of vectors sp ll of R There s o etr room for orthogol subspce the rowspce The ull spce of A T hs to be empt The squre mtr A hs full rk

34 The Four Fudmetl Subspces These est for mtr: Colum Spce: spce sped b the colums Row Spce: spce sped b the rows Nullspce: spce of vectors orthogol to ll rows (lso kow s the orthogol complemet of the row spce) Left Nullspce: spce of vectors orthogol to ll colums (lso kow s the orthogol complemet of the colum spce) Assume Dom of A Codom of A. The: Specl Cse I: Squre Smmetrc Mtrces (A A T ): Colum Spce s equl to the Row Spce Nullspce s equl to the Left Nullspce, d s therefore orthogol to the Colum Spce Specl Cse II: Orthogol Mtrces (A T A AA T I) Colum Spce Row Spce R Nullspce Left Nullspce {} the Trvl Subspce

35 END