Ch. 12 Linear Bayesian Estimators

Transcription

1 h. Lier Byesi stimtors

2 Itrodutio I hpter we sw: the MMS estimtor tkes simple form whe d re joitly Gussi it is lier d used oly the st d d order momets (mes d ovries). Without the Gussi ssumptio, the Geerl MMS estimtor requires itegrtios to implemet udesirble! So wht to do if we t ssume Gussi but wt MMS? Keep the MMS riteri But restrit the form of the estimtor to be LINAR LMMS stimtor LMMS stimtor Wieer Filter Somethig similr to BLU!

3 Byesi Approhes MMS Squred ost Futio (Nolier stimte) stimte : ˆ rr. ov.: M ˆ MAP Hit-or-Miss ost Futio Other ost Futios rr. ov.: Joitly Gussi d (Yields Lier stimte) M ˆ + ( ) stimte : ˆ Sme! LMMS Fore Lier stimte Kow:,, M ˆ + ( ) stimte : ˆ rr. ov.: stimte : ˆ rr. ov.: M ˆ Byesi Lier Model (Yields Lier stimte) + H ( HH + w ) ( Hμ ) H ( HH + ) H w 3

4 .3 Lier MMS stimtor Solutio Slr Prmeter se: stimte:, rdom vrible reliztio Give: dt vetor [[0] []...[N-] ] Assume: Joit PDF p(, ) is ukow But its st two momets re kow here is some sttistil depedee betwee d.g., ould estimte slry usig 0 pst yers tes owed.g., t estimte slry usig 0 pst yers umber of hristms rds set Gol: Mke the best possible estimte while usig ffie form for the estimtor N ˆ 0 [ ] + N hoose to miimize Bmse( ˆ) ( ˆ) Hdles No-Zero Me se 4

5 5 Derivtio of Optiml LMMS oeffiiets Usig the desired ffie form of the estimtor, the Bmse is + 0 ] [ ˆ) ( N N Bmse Step #: Fous o N 0 ˆ) ( N Bmse 0 ] [ 0 + N N Pssig / N through gives 0 ] [ N N Note: N 0 if [] 0

6 6 Step #: Plug-I Step # Result for N 0 ) ( ) ( ) ( ]) [ ] [ ( ˆ) ( slr slr N Bmse where [ 0... N- ] Note: ( ) ( ) sie it is slr Oly up to N-

7 hus, epdig out [ ( ) ( )] gives Bmse(ˆ) ( )( ) + t. ( )( ) + t. + t. + N N N N ross-ovrie vetors ( )( ) ( )( ) ( ) ovrie mtri vrie Note: Bmse (ˆ) + 7

8 Step #3: Miimize w.r.t.,,, N- Bmse( ˆ) 0 Oly up to N- 0 Step #4: ombie Results N ˆ 0 + [ ] + N [ ] + ( ) his is where the sttistil depedee betwee the dt d the prmeter is used vi ross-ovrie vetor So the Optiml LMMS stimte is: ˆ + ( ) If Mes 0 ˆ Note: LMMS stimte Oly Needs st d d Momets ot PDFs!! 8

9 Step #5: Fid Miimum Bmse Substitute ito Bmse result d simplify: Bmse(ˆ) Bmse( ˆ) Note: If d re sttistilly idepedet the 0 ˆ Bmse (ˆ) otlly bsed o prior ifo the dt is useless 9

10 .. D Level i WGN with Uiform Prior Rell: Uiform prior gve o-losed form requirig itegrtio but hgig to Gussi prior fied this. Here we keep the uiform prior d get simple form: by usig the Lier MMS For this problem the LMMS estimte is: ( )( ) A + w A + w ˆ A A Need A σ + σ I A ( ) A A + w A & w re uorrelted σ A Aˆ σ A σ + σ A / N 0

11 .4 Geometril Iterprettios Abstrt Vetor Spe Mthemtiis first tkled physil vetor spes like R N d N, et. But the bstrted the bre essee of these strutures ito the geerl ide of vetor spe. We ve see tht we iterpret Lier LS i terms of Physil vetor spes. We ll ow see tht we iterpret Lier MMS i terms of Abstrt vetor spe ides.

12 Abstrt Vetor Spe Rules A bstrt vetor spe osists of set of mthemtil objets lled vetors d other set lled slrs tht obey:. here is well-defied opertio of dditio of vetors tht gives vetor i the set, d Addig is ommuttive d ssoitive here is vetor i the set ll it 0 for whih ddig it to y vetor i the set gives bk tht sme vetor For every vetor there is other vetor s.t. whe the re dded you get the 0 vetor. here is well-defied opertio of multiplyig vetor by slr d it gives vetor i the set, d Multiplyig is ssoitive Multiplyig vetor by the slr gives bk the sme vetor 3. he distributive property holds Multiplitio distributes over vetor dditio Multiplitio distributes over slr dditio

13 mples of Abstrt Vetor Spes. Slrs Rel Numbers Vetors N th Degree Polyomils w/ Rel oeffiiets. Slrs Rel Numbers Vetors M N Mtries of Rel Numbers 3. Slrs Rel Numbers Vetors Futios from [0,] to R 4. Slrs Rel Numbers Vetors Rel-Vlued Rdom Vribles with Zero Me ollidig ermiology slr RV is vetor!!! 3

14 Ier Produt Spes A etesio of the ide of Vetor Spe must lso hve: here is well-defied oept of ier produt s.t. ll the rules of ordiry ier produt still hold <,y> <y, > * Not eeded for Rel IP Spes < +,y> <,y > + <,y> <,> 0; <,> 0 iff 0 Note: ier produt idues orm (or legth mesure): <,> So ier produt spe hs:. wo sets of elemets: Vetors d Slrs. Algebri Struture (Vetor Additio & Slr Multiplitio) 3. Geometri Struture Diretio (Ier Produt) Diste (Norm) 4

15 Ier Produt Spe of Rdom Vribles Vetors: Set of ll rel RVs w/ zero me & fiite vrie (ZMFV) Slrs: Set of ll rel umbers Ier Produt: <X,Y> XY lim his is Ier Produt Spe First this is vetor spe Additio Properties: X+Y is other ZMFV RV. It is Assoitive d ommuttive: X+(Y+Z) (X+Y)+Z; X+Y Y+X. he zero RV hs vrie of 0 (Wht is RV with vr 0???) 3. he egtive of RV X is X Multiplitio Properties: For y rel #, X is other ZMFV RV. It is Assoitive: (bx) (b)x. X X Distributive Properties:. (X+Y) X + Y. (+b)x X + bx Ier Produt is orreltio! Uorrelted Orthogol Net his is ier produt spe < X + X,Y> ( X + X )Y X Y+ X Y X <X, X> X vrx 0 5

16 Use IP Spe Ides for Setio.3 Apply to the stimtio of zero-me slr RV: ˆ ryig to estimte the reliztio of RV vi lier ombitio of N other RVs [0], [], [], [N-] N 0 [ ] Zero-Me do t eed N Now usig our ew vetor spe view of RVs, this is the sme struturl mthemtis tht we sw for the Lier LS! N se ( ) ( ) Miimize: ˆ ˆ Bmse ˆ oets to Geometry oets to MS [0] [] ˆ h RV is viewed s vetor Rell Orthogolity Priiple!!! stimtio rror Dt Spe ) ( ˆ [ ] 0 6

17 Now pply this Orthogolity Priiple ) ) ( ˆ 0 with ˆ ( 0 Assumig tht is ivertible he Norml qutios Sme s before!!! ˆ 7

18 .5 Vetor LMMS stimtor Meig Physil Vetor stimte: Reliztio of [ ] p Lier stimtor: ˆ A + Gol: Miimize Bmse for eh elemet View i th row i A d i th elemet i s formig slr LMMS estimtor for i Alredy kow the idividul elemet solutios! Write them dow ombie ito mtri form 8

19 Solutios to Vetor LMMS he Vetor LMMS estimte is: ˆ + [ ] Now p N Mtri ross-ovrie Mtri Still N N Mtri ovrie Mtri If 0 & 0 show similrly tht Bmse Mtri is M ˆ ( ˆ)( ˆ ˆ) M ˆ p p prior ov. Mtri p N N N N p 9

20 wo Properties of LMMS stimtor. ommutes over ffie trsformtios If he α A + b d ˆ is LMMS stimte α ˆ A ˆ + b α ˆ ˆ +. If α + the is LMMS stimte for α ˆ 0

21 Byesi Guss-Mrkov heorem Let the dt be modeled s ˆ μ + H kow H + w p rdom me µ ov Mt (Not Gussi) ( ) H H + [ Hμ ] w Like G-M heorem for the BLU N rdom zero me ov Mt w (Not Gussi) Applitio of previous results, evluted for this dt model gives: ε H ( ) HH + w H MMS Mtri: Sme forms s for Byesi Lier Model (whih ilude Gussi ssumptio) ept here the result is suboptiml uless the optiml estimte is lier I prtie geerlly do t kow if lier estimte is optiml but we use LMMS for its simple form! M ˆ he hllege is to guess or estimte the eeded mes & ov mtries ε