8.6 Order-Recursive LS s[n]

Transcription

1 8.6 Order-Recurive LS [] Motivate ti idea wit Curve Fittig Give data: 0,,,..., - [0], [],..., [-] Wat to fit a polyomial to data.., but wic oe i te rigt model?! Cotat! Quadratic! Liear! Cubic, Etc. ry eac model, loo at J mi wic oe wor bet J mi p Cotat Lie Quadratic Cubic p 3 4 # of parameter i model

2 Cooig te Bet Model Order Q: Sould you pic te order p tat give te mallet J mi?? A: O!!!! Fact: J mi p i mootoically o-icreaig a order p icreae If you ave ay data poit you ca perfectly fit a p model to tem!!!! [] poit defie a lie 3 poit defie a quadratic 4 poit defie a cubic poit defie a a - - a a 0 Warig: Do t Fit te oie!!

3 Cooig te Order i ractice ractice: ue implet model tat adequately decribe te data Sceme: Oly icreae order if cot reductio i igificat " Icreae to order p oly if J mi p J mi p > ε uer-et treold " Alo, i practice you may ave ome idea of te epected level of error tu ave ome idea of epected J mi ue order p uc tat J mi p Epected J mi Wateful to idepedetly compute te LS olutio for eac order Drive eed for: Efficiet way to compute LS for may model Q: If we ave computed p-order model, ca we ue it to recurively compute p-order model? A: YES!! Order-Recurive LS 3

4 4 Defie Geeral Order-Icreaig Model Defie: p [ p p ],, 3,... Etc. 3 Order-Recurive LS wit Ortoormal Colum If all i are EASY!!!! p p p

5 Order-Recurive Solutio for Geeral If i are ot arder, but oible! Baic Idea: Give curret-order etimate: map ew colum of ito a O verio ue it to fid ew etimate, te traform to correct for ortogoalizatio Quote ere becaue ti etimate i for te ortogoalized model Ortogoalized verio of S -D pace paed by & Rage ote: i ot ow ere it i i a iger dimeioal pace!! 5

6 Geometrical Developmet of Order-Recurive LS e Geometry of Vector Space i idipeable for DS! Curret-Order [... ] ot ecearily Recall: %"""" $ """"# rojector oto S Rage Give et colum: Fid, wic i to S See App. 8A for Algebraic Developmet Yu! Geometry i Eaier! I %"$"# S ŝ 3 S 6

7 7 So our approac i ow: project oto ad te add to ŝ e projectio of oto i give by!, calar ue " "# " %" $ Divide by orm to ormalize θ ow add ti to curret igal etimate:

8 8 " "# " "$ % I θ θ Scalar ca move ere ad trapoe Write out calar defie a b for coveiece ow we ave: Write out. ad ue tat i idempotet [ ] b b b b θ θ %"$"# Fially: Clearly ti i θ

9 9 θ θ Order-Recurive LS Solutio Drawbac: eed Iverio Eac Recurio See Eq. 8.9 ad 8.30 for a way to avoid iverio Commet:. If implifie problem a we ve ee i equatio implifie to our earlier reult. ote: above i reidual of -order model part of ot modeled by -order model Update recurio wor olely wit ti Mae See!!!

10 8.7 Sequetial LS I Lat Sectio: Data Stay Fied Model Order Icreae I i Sectio: Data Legt Icreae Model Order Stay Fied You ave received ew data ample! Say we ave θ[ ] baed o {[0],..., [-]} If we get [] ca we compute θ [ ] baed o θ [ ] ad []? w/o olvig uig full data et! We wat θ [ ] f θ [ ], [ ] Approac ere:. Derive for DC-Level cae. Iterpret Reult 3. Write Dow Geeral Reult w/o roof 0

11 Sequetial LS for DC-Level Cae 0 ] [ A We ow ti: Re-Write ] [ ] [ ] [ ] [ 0 0 A A %$# ad ti: "" " # "" %" $ %$# %$# predictio error te ew data predictio of old etimate ] [ A A A

12 Weigted Sequetial LS for DC-Level Cae i i a eve better illutratio Aumed model: Stadard WLS give: [ ] A w[ ] var{ w[ ]} σ A 0 0 [ ] Wit maipulatio imilar to te above cae we get: σ σ w[] a uow DF but a ow timedepedet variace A A %$# old etimate σ 0 %"$"# σ [ ] A %"" $ "# " predictio error i a Gai term tat reflect goode of ew data

13 Eplorig e Gai erm We ow tat we get tat var A 0 σ var A var A ad uig it i & σ variace of te ew data poore of curret etimate poore of ew data ote: 0 K[] Gai deped o Relative Goode Betwee: o Curret Etimate o ew Data oit 3

14 Etreme Cae for e Gai erm A [ ] % A[ "$"# ] K[ ] [ ] A[ ] %"" $ """ # old etimate predictio error If var A[ ] << σ Good Etimate Bad Data K[ ] 0 ew Data a Little Ue Mae Little "Correctio" Baed o ew Data If var A[ ] >> σ Bad Etimate Good Data K[ ] ew Data Very Ueful Mae Large "Correctio" Baed o ew Data 4

15 Geeral Sequetial LS Reult At time ide - we ave: θ Σ [ [0] [] ' [ ] ] LS Etimate uig cov θ w { } θ quality meaure of etimate C 0 diag{ σ, σ,', σ See App. 8C for derivatio } Diagoal Covariace Sequetial LS require ti At time ide we get []: θ w θ w ac o row at bottom to ow ow θ map to [] 5

16 Iterate tee Equatio: Give te Followig: θ Σ [ ] σ Update te Etimate: θ θ [ ] θ %"$"# Compute te Gai: σ Σ Σ redictio of [] uig curret parameter etimate Update te Et. Cov.: Σ I Σ Iitializatio: Aume p parameter Collect firt p data ample [0],..., [p-] Ue Batc LS to compute: θ p Σ p e tart equetial proceig Gai a ame id of depedece o Relative Goode betwee: o Curret Etimate o ew Data oit 6

17 Sequetial LS Bloc Diagram Σ,, σ Obervatio Compute Gai Updated Etimate [] Σ [ ] θ Σ θ [ ] θ θ θ θ z - redicted Obervatio reviou Etimate 7