Inner Product Spaces (Chapter 5)

Size: px

Start display at page:

Download "Inner Product Spaces (Chapter 5)"

Hubert King
5 years ago
Views:

1 Ier Product Spces Chpter 5 I this chpter e ler out :.Orthogol ectors orthogol suspces orthogol mtrices orthogol ses. Proectios o ectors d o suspces

2 Orthogol Suspces We ko he ectors re orthogol ut ht out suspces? Defiitio :o suspces W d V of ector spce re orthogol if eery ector i V is perpedicu lr to eery ector i W i.e. V & W. he Emple : z-is is suspce orthogl to the -y ple suspce heorem : Eery ector i the ull spce of is perpedicu lr to eery ro of ecuse re orthogol suspces. eso : m ull spce d ro spce is perpedicu lr to ll lier comitio s of ros perpediculr to

3 Orthogol Suspces heorem spce eso of : Defiitio eery ect is the : Eery ect or y i the ull is to eery colum of C zero d Cosider : he or to ector. d re orthogol V isted is orthogol of deoted d spce d complemet y V of pply suspces of. heir i.e. i preious C V cotis left m. it itersectiti ull theorem Fct : If set of ectors re orthogol the they re lierly idepedet ut the coerse is ot true gie couter-emple. Proof : put these orthogol ectors s colums of mtri d set their lier comitio to o

4 Fudmetl heorem of Lier lger Prt II he ullspce is the orthogol complemet of the ro spce their uio is. he left ull spce is the orthogol complemet of the colum spce their uio is m. dim = r r = dim = r = r + = m dim = - r dim = m - r

5 Emple o Orthogol Complemet p g p 7 y sped of suspce the e Let 5 d u W : Sol for sis Fid W W W di free piots dim r r

6 Emple Cot d ull Bsis Vectors for Orthogol Complemet Suspce

7 Orthogolity d Proectios y-is ssume tht the folloig reltio holds etee the sides of trigle sho tht it is right trigle t=u+ u u u u u u u u u hy? u -is u represets the proectio of the ector t o the -is =t-u is the differece ector error etee t d its proectio is u o key properties of proectios : is orthogol to the -is u is the closest ector o the -is to ector t

8 Proectio o Lie F t K lie : oto Proectio e = - p p p. Key Fct : p p e.... Key Fct i : Cses Specil. iii ii i

9 p P he sclr proectio Proectio Mtri proectio mtri is rk mtri hich descries the s lier trsformtio from to p. proectio mtri ti is sigulr epli ituitiely i i Key Property he proectio ector p is the closest ector to log

10 P P i : otes Epli ituitiely tr P tr ii m deotes do t cre tr his is useful check e p P P I iii his is useful check o proectio mtri clcultios P I p ple to perpediculr suspce o the proects

11 Proectios o Suspces We ko ho to proect o ector ut ho do e proect o suspce? Use the diide d coquer priciple! Ide : represet the suspce i terms of its sis ectors. he proectio ector is gie y lier comitio of these sis ectors. he error ector is orthogol to ll of the sis ectors

12 Proectio reduces dimesiolity Proectio oto Suspce : Prolem : Fid the proectio ector p ssumig tht 's re idepedet i form suspce Sol : i sis for the d defie i - dimesiol the colums e is m here m of Proect oto the colum spce of Best descriptio of suspce is its sis p e m-dimesiol m > -dimesiol p is the closest ector to i the -dimesiol proectio suspce e is orthogol to eery ector i the -dimesiol proectio suspce. I prticulr it is orthogol to ll of its sis ectors i ' s

13 p P iertile proects ote simplifictio he colums of re orthogol P! P is rk- proectio mtri of size m m oto the colum spce of P I P is tll mtri ith lierly idepedet colums hece is iertile proof o et slide otes : Shortest distce from to colum spce of e rce P k P Cosider specil cse of proectio o lie is proof hit : use the idetity trceb=trceb his is geerliztio for proectio o lie here the proectio mtri s rk- mtri ith trce equl to

14 iff i til i H : heorem t colums lierly idepede hs m & m iff iertile is Hece. :if Proof {} cse i this d if Hece dim dim lier lger of theorem l fudmet From the Hece rk rk dim dim rk rk rk rk

15 Summry : here i.e We t to proect oto suspce sped y C Error ector C C ti t fl ti M t i Coectio to eflectio Mtri r : eflectio p Prllelogrm eflectio Mtri : I I P r hy? proe it!? Wht is I hy? proe it!

16 Emple : Fid the proectio of the ector o the suspce sped y the lierly idepedet ectors clcultelte the distce etee ee d its proectio. Sol : d

17 P p e P p Check tr P k P s check erify tht

18 Orthoorml Mtrices Defiitio :he ectors q q q re orthoorml : i orthororml iff q q i : i uit - orm qi ote : m m mtri ith orthororml colums stisfies for i I d is the it is q q q clled orthoorml mtri. clled orthogol mtri If the orm is ote tht if m= I q q q ot

19 ottios Emples of Orthoorml Mtrices : cos si cos si si cos Permuttios : Colums of si cos Wht out the proectio mtri? Is it orthogol? hy? re uit ectors E : Gie emple of tll orthoorml mtri eflectios : uu I here u is uit - orm colum ector I uu uu uu I lso d e i I

20 Orthogol Bses Fct : Orthoorm l mtrices presere y y y y legth & dot products Proectio o suspce ith orthogol sis I ote tht p set o iersio I Emple : Ese of proectig o d y es ersus other ses

21 Why Orthogol Bsis? We ko tht y ector c e ritte uiquely i terms of the lierly- idepedet sis * But he the sis ectors re orthogol fidig i ectors Gie emple of orthoorml sis for the i 's ecomes ery esy sice from * i ii i i : i Ho to geerte orthoorml sis? i i

22 he Grm-Schmidt Procedure V We. spce for ector sis is Suppose : follos s sis orthogol costruct P Illustrte grphiclly for P t the ed to get orthoorml sis e ust ormlize diide ech sis ector y its orm mgitude

23 Emple pply Grm - Schmidt procedure to ; 5 ; Sol : ;

24 Fctoriztio Fctoriztio / / Writig the Grm-Schmidt / / / Equtios i mtri form e get ithout ormliztio sis Orthogol idepedet ectors gie y origil colums / / mtri Upper-rigulr o digol ormlized 's represets the lier trsformtio etee origil sis & orthoorml sis & is lys iertile hy? / y y ormlized Bsis orthoorml upper-trigulr ithout s o digol

25 Fctoriztio : pplictio Solig systems of lier equtios : geertes orthogol sis used i prctice he solig lier equtios i fied precisio orthoorml mtrices re roust to roud-off errors eltio to Cholesky trigulr fctoriztio of ithout the eed to compute "loer times upper trigulr" fctoriztio of iduces Cholesky fctoriztio!

26 emple preious From : Emple got e Schmidt Grm From

27 Emple Cot d ormlized digol hich is elemets the orthoorml ot equl to Cholesky fctoriztio

28 pplictio : Lest Squres I prctice e he: e here e is "oise" ector. Equiletly e c rite e p proectios!! Fid closest ector to i C; i.e. proect oto C Fid to miimize ii i e his is chieed he e C i. e. e e

29 Lest Squres Usig Fctoriztio herefore ecomes ; trigulr system of equtios Emple: Lie Fittig of Eperimetl Dt Fit lie c m y to dt poits ; ; 5 usig lest squres more equtios th ukos

30 Emple Cot d c m 5 rk uique solutio c m y residul fittig error e. 8. his ector hs the smllest orm of ll other residul error ectors for y lier fittig ote tht of the gie dt. his miimum squred orm is e.7

Orthogonality, orthogonalization, least squares

Orthogonality, orthogonalization, least squares ier Alger for Wireless Commuictios ecture: 3 Orthogolit, orthogoliztio, lest squres Ier products d Cosies he gle etee o-zero vectors d is cosθθ he l of Cosies: + cosθ If the gle etee to vectors is π/ (90º),