CS3 Nuercl Alss Lecture 7 Lest Sures d Curve Fttg Professor Ju Zhg Deprtet of Coputer Scece Uverst of Ketuc Legto KY 456 633 Deceer 4
Method of Lest Sures Coputer ded dt collectos hve produced treedous out of dt tht re possle to uderstd wthout soe sort of postprocessg Gve set of dt If we ssue tht the dt for ler fuctol relto we c wrte the fucto s wth the coeffcets d to e detered For ech prs we c reuest For =. Note tht f > we hve ore th two ler eutos to detere ust two uows
Lest Sures Ft M sple pots whch e ccurte. Not good for terpolto. 3
Over Detered Sste I geerl we hve ore eutos th uows. here s o ect soluto to the prole. However we could detere soluto tht zes the totl error Suppose the ler euto s gve s If the pot s o the strght le defed the fucto we hve I ost cses pot s ot o the le we hve For =. r s the curve fttg error r 4
Mze the Errors Most sple pots re ot o the curve. Hopefull the totl dstce etwee the sple pots d the fttg curve s zed. 5
Mzg otl Error Oe c reso tht f the su of the errors s zed the dt should ft the le s est s t c he totl error c e represeted s r We c ze the ove fuctol to select the coeffcets d. hs c e solved the techues of ler progrg hs s l pproto or pproto he shortcog of ths forulto s tht the fucto of the totl error s ot dfferetle. M tools clculus ot e used 6
Method of Lest Sures A ltertve strteg s to ze dfferet error fucto whch s cotuousl dfferetle hs s lso clled l pproto. It s specl cse of the l p pproto wth the l p or s defed s p where = s desol vector p Fro sttstcl cosdertos f the errors follow orl prolt dstruto the zto of φ produces est estte of d / p p 7
How to Copute Mu 8 We use techue clculus to detere the etree pot of fucto hs gves us two eutos he re clled the orl eutos d c e wrtte eplctl s whch c e solved for d
Lest Sures Soluto 9 he soluto of the prevous two two ler sste c e solved s where Lots of sple coputtos Ler Eple d d d
Lest Sures Ft he dt the prevous slde led to.5 8.5 8.5 4. 37.65 7. whch c e solved for =.6 d = 3.
Nopolol Eple We c ft tle opolol fucto
Bss Fuctos A geerl lest sures fttg c e wrtte s c g whch the fuctos g g g re clled ss fuctos. he re ow d ept fed Gve set of dt we wt to fd the vlues of c to ze the totl error s c c c c g We g set the prtl dervtves to e zero c
Bss Fuctos II 3 he prtl dervtves re for Settg the sulteousl zero we hve hs s the orl euto whch s sste of ler eutos wth uows c c c he coeffcets of the ler sste re he coeffcet tr s osgulr f the ss fucto g g g re lerl depedet. he ss fuctos should e pproprte for the prole uesto d e the resultg coeffcet tr well codtoed g g c c g c g g g g
4 Orthoorl Bss Fuctos Gve set of ss fuctos {g g g } the set of ll fuctos tht re ler cotos of the ss fuctos re We re loog for prtculr g Є G such tht the fttg totl error s zed A + fuctos tht re lerl depedet c e used s ss fuctos. Dfferet choces of ss fuctos e the orl euto for eser or ore dffcult to solve } { g c g g G tht such : g c g g c
Choose Bss Fuctos We s tht ss {g g g } hs the propert of orthoorlt f I ths cse the orl euto s splfed s whch c e evluted strghtforwrdl g g c g he Gr Schdt procedure c e used to orthoorlze gve ss order to hve the ove propert. hs procedure e epesve We c lso choose soe ss fuctos so tht the coeffcet tr s es to solve ot ecessrl s dett tr 5
6 Polol Bss Fuctos Cosder G s the spce of ll polols of degree. We turll choose A polol G c e represeted s he sple ss s however ot ver good sce the re too uch le Assue we hve the dt restrcted the tervl [ ] wth We c defe set of Cheshev polols tht for good ss g g g c g c g
Polol Bss Fuctos he re too uch le d glol ture. A sll chge vlue ffects ll ss fuctos 7
Orthogol Polols Orthoorl ss polols ssocted wth the Legeder polols 8
9 Cheshev Polols he frst few Cheshev polols re he Cheshev polols c e geerted recursvel s he c lso e wrtte s A fucto c e represeted s ler coto of the Cheshev polols A fucto f wrtte s ler coto of the Cheshev polols c e evluted effcetl 8 8 3 4 4 4 3 3 rccos cos c f
Cheshev Polols he frst few Cheshev polols
Evlutg Cheshev Polols o evlute f for gve We use cwrd recurso procedure he Cheshev polols re defed o the tervl [ ] we would lso le the scsss { } to le the tervl [ ].e. { } = d { } =. If the le dfferet tervl [] we c use trsforto to p the tervl [ ] oto [] c f w w f w w c w w w z
Evlutg Cheshev Polols w w w w w w w w w w w w w w w w w w w w w w w w c f
3 Algorth of Polol Fttg. Fd the sllest tervl cotg ll wth = { } d = { }. Me trsforto to the tervl [ ] usg the p 3. Decde o the order of the polols roud 8 or 4. Usg the Cheshev polols s ss geerte the + + orl eutos for z z c z z
4 Algorth II 5. Use euto solvg route to solve the orl eutos for coeffcets c c c to ot the fucto 6. rsfor the fucto c to the orgl vrle s he coputtol etesve prt s to for the coeffcet tr of the orl euto Specfc procedures re detled oo c f f : : : z z z A Ac d let
Polol Regresso Assue the dt collected cot errors the procedure for soothg dt s to reove the eperetl errors s uch s possle Soothg dt s dfferet fro terpolto sce the ltter ssues tht the dt re ccurte Gve tle of eperetl dt We wt to fd polol tht represets the orgl dt fetures We hve where ε s the oservtol error P N P N N 5
Polol Regresso II We c use the ethod of lest sures through solvg sste of orl eutos to detere P. A utt clled vrce p c e coputed to see how good the pproto s If the orgl dt rell represet polol of degree N wth ose the N N We c copute σ σ utl we see for soe N tht σ N σ N+ σ N+ the we chose the polol P N s the oe represetg the orgl dt tred he drwc s tht we eed to copute p p
A Eple of Regresso A reltoshp etwee the hours studed d the test scores
Ier Product Let two fuctos f d g whose dos cot { } we defe f g f g s the er product of the fuctos f d g A er product of two fuctos hs the followg propertes. f g g f. f f uless f for ll 3. f g f g where s sclr 4. f g h f g f h 8
9 Orthogol Polols A set of fuctos s orthogol f fg = for two dfferet fuctos the set We c geerte set of orthogol fuctos s For where he polols { - } c sp ler spce whch the re ss
3 Orthogol Polols We c chec the orthogolt of polols s he reg prt c e proved usg ducto for Assue
Well Defed? We eed to show If ths s ot the cse the hs es tht It follows tht t hs + roots. If s sller th the we ow tht s the zero polol whch s ot true sce d [ lower order ter s ] for 3
Represetg Fucto A polol of degree the sped ler spce c e represeted s p If we for the er product wth respect to o oth sdes For d usg the fct tht = f wh? we hve Hece for = re the eeded coeffcets p p p 3
Solvg Icosstet Eutos A sste of ler eutos of the for wth > s cosstet f there s o possle vector to e the resdul zero. here s o soluto the covetol sese stsfg ths sste It s of soe terest pplctos to fd the vector tht zes the or resdul We c te the prtl dervtves wth respect to d set the eul to zero to ot the orl eutos for 33
Drect Fctorztos he orl eutos oted c e solved Guss elto d ts soluto of the orgl sste the lest sures sese If we wrte the orgl ler sste s A = It s lso possle to drectl fctor the tr A s A = Q R where Q s + + orthogol tr stsfg Q Q = d R s upper trgulr + + tr stsfg r > d r = for <. We the hve R = Q whch c e solved c susttuto 34
35 Sgulr Vlue Decoposto he QR fctorzto c e oted lgorth clled odfed Gr Schdt procedure to orthogolze the row vectors A ore volved lgorth eeds to copute the Sgulr Vlue Decoposto SVD of the tr A s whch U d V re orthogol.e. d Σ s + + dgol tr hvg oegtve etres V U A I V V I U U r o
36 Pseudo Iverse If A s osgulr sure tr wth We c copute the true verse of A.e. A to solve the ler sste s If A s + + rectgulr tr we frst copute the sgulr vlue decoposto of A s the for of We the vert Σ s V U A A A r
Pseudo Iverse II We c defe the pseudo verse of A s Ad the soluto of the rectgulr ler sste s defed to e It c e show tht ths defto of soluto does ze the resdul or of the orgl cosstet sste Note tht QR fctorzto d Sgulr Vlue Decoposto re ore epesve to perfor cses th solvg the orl eutos. I cse tht the tr A s sprse tr we e le to solve t ore effcetl usg cert tertve ethods Forg the orl euto A A s opto A A V V U U 37
Proof I Cosder sste of ler eutos A = d A s tr. he l soluto of the sste s A V U wth Proof. Let e vector. Defe Usg the orthogolt propertes we hve A V f U f V Note tht s dgol tr V UV d U c U f A f UV U f c 38
Fro prevous pge we hve Proof r c r c o ze the epresso we eed to ze the frst ters o the rght hd sde defg c for r c Other copoets of c re uspecfed. We c set So we set d c for r V V c V U A 39
4 Soe Propertes here re soe terestg propertes for the pseudoverse. he re clled Perose Propertes. Proof: A A A A AA AA AA A A A AA A A V U V U V U U V V U A AA