Detection and Estimation Theory

Transcription

1 ESE 54 Detecto ad Etmato Theoy Joeph A. O Sullva Samuel C. Sach Pofeo Electoc Sytem ad Sgal Reeach Laboatoy Electcal ad Sytem Egeeg Wahgto Uvety Ubaue Hall (Lyda awe) jao@wutl.edu J. A. O'S. ESE 54, Lectue 9, 0/7/09

2 Aoucemet o Cla /9 Poblem Set 3 Due Cla /0 We wll chedule two make-up clae o Fday Fday Feb. 0 ad Feb. 7 Aothe Fday afte pg beak Mdtem Exam? Othe aoucemet? Queto? J. A. O'S. ESE 54, Lectue 9, 0/7/09

3 Stattcal t t Ifeece Tato Pobablty o pdf p(r θ) Ifeece Algothm - Log-lkelhood lh ato tett - Paamete etmate Paamete Space - Hypothe - Cotuou Data Space -Cotuou -Dcete - Radom poce Ifeece Space - Hypothe - Cotuou J. A. O'S. ESE 54, Lectue 9, 0/7/09

4 Outle: Itoducto to Etmato Theoy Rage of poblem tuded Mmum mea cot poblem Mmum mea quae eo etmato Mmum abolute eo etmato Maxmum a poteo etmato Othe Maxmum lkelhood fo oadom paamete Fhe fomato ad the Came-Rao boud J. A. O'S. ESE 54, Lectue 9, 0/7/09 4

5 Rage of Etmato Theoy Poblem Studed d Radom Paamete oadom paamete Cot Fucto Mea Squae Eo Mea Abolute Eo Lkelhood Fucto: Maxmum a poteo Othe mea cot Lkelhood Fucto: Maxmum lkelhood Othe cot Soluto Poteo Mea Meda Lkelhood equato Geealzed mea Lkelhood equato J. A. O'S. ESE 54, Lectue 9, 0/7/09 5

6 Radom Paamete Etmato Po o the paamete p ( S ) Codtoal pdf o the p ( R S) data gve the C [ SR, ˆ ( )] paamete E{ C[,()] ˆ } Baye Rule gve poteo pdf ˆ* ag m E C [,()] ˆ Cot fucto gve ˆ Select that etmato that mmze the mea cot p (S) p ( R S) ŝ() { } J. A. O'S. ESE 54, Lectue 9, 0/7/09 6

7 Radom Paamete Etmato Etmato a fucto Fo each data pot, etmato gle- valued Mmze codtoal mea cot Geealzed oto of mea p (S) p ( R S) ŝ() p ( S) p ( R S ) E C[,()] ˆ { } { } ˆ * ag m E C[, ˆ()] { ˆ } ˆ { [,()] ˆ } E E{ C[,()] ˆ } ˆ E C[,()] C[ SR,( )] p ( R S) p ( S) dsr d E C { } ˆ C[ S, ( R)] p ( S R) ds p ( R) dr R ˆ( ) agm C[ S, ˆ] p ( S R) ds ˆ J. A. O'S. ESE 54, Lectue 9, 0/7/09 7

8 Mmum Mea Squae Eo Etmato Cot equal quaed eo Awe the MMSE etmato t Mmze equal poteo mea Hold fo adom vaable ad adom vecto C [ S, ˆ] S ˆ ˆ ˆ R ˆ( ) ag m C[ S, ˆ] p ( S R) ds ag m S ˆ p ( S R) ds { } E ( ) { } ˆE ˆ ˆ ( E ˆ ) [ ] 0 R ˆ( ) E[ R] J. A. O'S. ESE 54, Lectue 9, 0/7/09 8

9 Mmum Mea Squae Eo Etmato Repeated meauemet of a Gaua adom vaable Gaua oe Poteo mea a lea combato of data Pefomace? MSE(mea eo) +(eo vaace) ame: MMSE MSE acheved a mmum meauemet of a ˆ( R ) E [ R ] p p (0, σ ) + w,,,...,, w..d. (0, σ ) w p ( R S) p( S) ( S R) p ( R) ( ) R S S exp exp πσ σ σ πσ ( S R) Z( R) S S RS exp σ σ σ Z ( ) R ( S μ ) exp σ a Rσ a, μ, + Z 3( R) σ σ a σ σ Z ( 3 R ) πσ a J. A. O'S. ESE 54, Lectue 9, 0/7/09 9

10 MMSE Etmato fo a Gaua Poteo vaace equal po vaace dvded by oe plu the gal to oe ato (SR) SR equal gal eegy dvded by oe powe Etmate equal a caled veo of the mea of the meauemet Scale equal SR dvded by +SR Poteo tadad d devato deceae a the quae oot of the umbe of depedet meauemet Z (0, σ ) w w w +,,,...,,..d. (0, σ ) [ ] ˆ( R ) E R p 3 exp ( S μ ) σ a Rσ a ( S R), μ, + Z3( R) σ σa σ σ ( R) πσ a σσ σ σ σ + + SR a σ σ σ + σ σ ˆ( R) μ a σ σ SR σ SR R R R σ + + σ If SR, σa σ σ, ad σ σ a SR J. A. O'S. ESE 54, Lectue 9, 0/7/09 0

11 Mmum Abolute Eo Etmato Defed hee fo a gle vaable Etmate the meda of the poteo MAE etmate moe obut to outle the data, pealze lage eo le tha MMSE Pefomace? CS [, ˆ] S ˆ ˆ ( R ) agm C [ S, ˆ] p ( S R) ds agm S ˆ p ( S R) ds ˆ ˆ ˆ ( ˆ ) ( ˆ + ) ˆ ˆ R ˆ { ˆ } ( ) ˆ ( ) 0 R R ag m S p ( S ) ds S p ( S R) ds E p S ds p S ds ˆ ˆ R ˆ p ( S ) ds p ( S R) ds ˆ( R) meda of poteo J. A. O'S. ESE 54, Lectue 9, 0/7/09

12 Rage of Etmato Theoy Poblem Studed d Radom Paamete oadom paamete Cot Fucto Mea Squae Eo Mea Abolute Eo Lkelhood Fucto: Maxmum a poteo Othe mea cot Lkelhood Fucto: Maxmum lkelhood Othe cot Soluto Poteo Mea Meda Lkelhood equato Geealzed mea Lkelhood equato J. A. O'S. ESE 54, Lectue 9, 0/7/09

13 Maxmum a Poteo Etmato Cot oe f eo geate tha ε ad zeo othewe Code ε to be mall Maxmze the poteo pobablty dety fucto: MAP etmate e Alo called the mode of the poteo Pefomace?, S ˆ >ε CS [, ˆ ] 0, othewe ˆ( R) agm C[ S, ˆ] p ( S R) ds ˆ ˆ ˆ ε ag m p ( S R) ds+ p ( S R) ds ˆ + ε ˆ + ε R ε J. A. O'S. ESE ˆ54, Lectue 9, 0/7/09 ˆ ˆ ε ag max p ( S ) ds ag max p ( ˆ R) 3

14 Maxmum a Poteo Etmato Maxmzg a oegatve-valued fucto equvalet to maxmzg t logathm Thk about the fucto plotted o a db cale Ue Baye Rule to epeet the poteo tem of the po ad the codtoal lkelhood Deomato ot a fucto of S Mut olve the ˆ MAP ( R ) ag max ε p ( S R) S lkelhood equato p ( S R) MAP ad MAE 0 etmate fo the Gaua poblem equal the MMSE etmate, the poteo mea S S ˆ MAP p ( R S) p( S) p ( S R) p ( R) l p ( S R ) l p ( R S ) + l p ( S ) l p ( R) l p ( R S) l p ( S) 0 + S S ˆ S S ˆ J. A. O'S. ESE 54, Lectue 9, 0/7/09 MAP MAP 4

15 Othe Cot Fucto Paamete may take may fom Ampltude, fequecy, phae Itety of a Poo (cocetato of adoactve ubtace) Vaace of oe a amplfe o ccut Decto: SO(3); dtace ad decto: SE(3) Subpace gal pace Defomato o wapg: mage o volume wapg Dtace o othe dcepacy mut be defed o the paamete pace oegatve, zeo at tuth, mootoc ome ee Example: map paamete to a matx ad ue a matx-dtace (o dtace quaed lke um of quae eo) to duce a dcepacy paamete pace J. A. O'S. ESE 54, Lectue 9, 0/7/09 5

16 Outle: Itoducto to Etmato Theoy Maxmum lkelhood fo oadom paamete Fhe fomato o ad the Camé-Rao ao boud J. A. O'S. ESE 54, Lectue 9, 0/7/09

17 Rage of Etmato Theoy Poblem Studed d Radom Paamete oadom paamete Cot Fucto Mea Squae Eo Mea Abolute Eo Lkelhood Fucto: Maxmum a poteo Othe mea cot Lkelhood Fucto: Maxmum lkelhood Othe cot Soluto Poteo Mea Meda Lkelhood equato Geealzed mea Lkelhood equato J. A. O'S. ESE 54, Lectue 9, 0/7/09 7

18 Rage of Etmato Theoy Poblem Studed d Cot Fucto Soluto Radom Paamete oadom paamete Mea Squae Eo Mea Abolute C [ S, Eo ˆ] S ˆ Poteo Mea Meda Lkelhood ˆFucto: ( Lkelhood ˆ MMSE R ) agm C [ S, ] p ( ) S R ds ˆ Maxmum a poteo equato Othe mea cot ag m Geealzed S ˆ p ( Smea R) ds ˆ { ˆ } E ( ˆ) { } Lkelhood ˆ Fucto: ˆ Lkelhood ˆ E Maxmum lkelhood equato ( E[ ] ˆ ) 0 Othe coṱ cot ( R) E[ R] MMSE J. A. O'S. ESE 54, Lectue 9, 0/7/09 8

19 Rage of Etmato Theoy Poblem Studed d Cot Fucto Soluto Radom Paamete oadom paamete Mea Squae Eo Poteo Mea Mea Abolute Meda Eo CS [, ˆ] S ˆ Lkelhood Fucto: Lkelhood ˆ Maxmum a poteo p equato ( S ) ds p ˆ ( S ) ds R R Othe mea cot ˆ ( R) Geealzed meda of poteo mea MAE Lkelhood Fucto: Maxmum lkelhood Othe cot J. A. O'S. ESE 54, Lectue 9, 0/7/09 Lkelhood equato 9

20 Rage of Etmato Theoy Poblem Studed d Radom Paamete Cot Fucto Mea Squae Eo Mea Abolute Eo Soluto Poteo Mea Meda oadom paamete Lkelhood Fucto: Maxmum a poteo ˆ Othe ( R) ag mea maxcot l p ( S R) MAP S Lkelhood equato Geealzed mea Lkelhood ag max Fucto: l p ( S) Lkelhood l p( S) l p ( ) R + R S Maxmum lkelhood equato l p ( R S) l p ( S) Othe cot 0 S + S ˆ S S ˆ J. A. O'S. ESE 54, Lectue 9, 0/7/09 MAP MAP 0

21 Commet o Radom Paamete Etmato If the poteo ymmetc aoud t mea, the the poteo mea (MMSE etmate) equal the poteo meda (MAE etmate). If the poteo mea alo the maxmum, the MAP equal the MMSE etmate. If the cot fucto ymmetc the eo ad the poteo ymmetc aoud the mea, the the mmum cot etmate equal the MMSE etmate. J. A. O'S. ESE 54, Lectue 9, 0/7/09

22 oadom Paamete Etmato Thee o po o the paamete. Cocetate o the maxmum lkelhood ule: fd the paamete that maxmze the lkelhood fucto o equvaletly the loglkelhood fucto. oadom paamete veo of MAP etmato Pefomace? ˆ ( R) agmax p ( R S) agmaxl p ( R S) ML S S 0 l p ( R S ) S S ˆ ML J. A. O'S. ESE 54, Lectue 9, 0/7/09

23 oadom Paamete Etmato Thee o po o the paamete. Cocetate o the maxmum lkelhood ule: fd the paamete that maxmze the lkelhood fucto o equvaletly the loglkelhood fucto. oadom paamete veo of MAP etmato Sgle vaable ad S S multple vaable Pefomace? S p ( R S) ŝ( ( ) ˆ ( R) ag max p ( R S) ag max l p ( R S) ML 0 l p ( R S) S S ˆ 0 S l p ( R S ) S ML ˆML J. A. O'S. ESE 54, Lectue 9, 0/7/09

24 Maxmum Lkelhood Etmato Repeated R meauemet of a + w w σ detemtc w vaable Gaua oe ( R S) Solve lkelhood p ( S) exp R πσ σ equato ML etmate the ( R S) lmt of the MMSE l p ( R S) + cotat σ etmate t a SR goe to fty l p ( ) R S ( R S) 0 Po vaace S S ˆ σ ML S ˆ ML goe to fty Pefomace? ˆ ML R MSE(mea eo) +(eo SR ˆ MMSE ( R) R vaace) ) + SR,,,...,,..d. d (0, ) J. A. O'S. ESE 54, Lectue 9, 0/7/09

25 Pefomace: Detemtc Paamete Etmate adom [ ] Ba equal the mea of the etmate mu the tuth Vaace o covaace matx of the etmate Fo the example, the etmate ubaed ad the vaace ealy computed. I may cae, computg the ba ad vaace may be had. E ˆ( ) S+ BS () ( )( ) cov( ˆ( )) E ˆ( ) S BS ( ) ˆ( ) S BS ( ) T w w ˆ ML R E[ ˆ ML () ] E[ ] E ( ˆ ) ML () E ( ) σ E w +,,,...,,..d. (0, σ ) J. A. O'S. ESE 54, Lectue 9, 0/7/09

26 Fhe Ifomato ad the Camé-Rao Boud Actual pefomace tem of vaace may be dffcult to compute, o boud o pefomace ae ought. The Camé-Rao Boud a lowe boud o the vaace of ay ubaed etmato. It deped oly o the pobablty dtbuto fo the data, ot o ay patcula etmato. Late, we code algothm to compute etmate. It mpotat to ote that the Camé-Rao Boud (ad elated boud) ae depedet of the algothm. If the lowe boud achevable, the ay etmato that acheve that lowe boud called effcet. Thee a veo of the Camé-Rao Boud fo baed etmato t a well, but t ot a ueful becaue ba ot kow (othewe t could be ubtacted). Pefomace boud uch a the Camé-Rao Boud may be ued fo ytem deg ad aaly by evaluatg how the boud deped o ytem paamete. J. A. O'S. ESE 54, Lectue 9, 0/7/09

27 Fhe Ifomato ad the Camé-Rao Boud Theoem: Let ˆ ( ) be ay ubaed etmate of. Aume that p( R ) p( R ) ad ext ad ae abolutely tegable. The va( ˆ ( )) l p( ) E o, equvaletly va( ˆ ( )). l p ( ) E J. A. O'S. ESE 54, Lectue 9, 0/7/09

28 Fhe Ifomato ad the Camé-Rao Boud Theoem: va( ˆ ( )) l p E CRB/FI o, equvaletly va( ˆ ( )). Poof: l p( ) E E [ ˆ() ] ( ˆ( R) ) p( R ) dr 0 p( R ) ( ˆ( ) ) p( ) d ( ˆ( ) ) d 0 R R R R R p( R ) l p( R ) p( R ) l p( R ) ( ˆ( R ) ) p ( R ) dr ( R ) l p ( ˆ( R) ) p( R ) p( R ) dr ( ) ( ) ( R ) ( ) l p ˆ( R) p R dr p( R ) dr Schwaz equalty J. A. O'S. ESE 54, Lectue 9, 0/7/09

29 Commet Codto fo equalty Schwaz equalty Codto fo a ML etmate Equalty acheved the CRB f the Schwaz equalty hold wth equalty. If a etmato ext that acheve equalty ( effcet), the the ML etmato effcet. If o effcet etmato ext, the vaace may be abtaly lage tha the CRB. The ecod devatve fom of Fhe Ifomato ealy foud. Fo baed etmato, the boud chage. The vaace of a baed etmato may be lowe tha a ubaed etmato. Code the zeo etmato. ( R ) l p k ()( ˆ ( R) ) l p( R ) k ()( ˆ ( R) ) 0 ˆ ML ( R) ˆ( R) ˆ ( R), k( ˆ ( R)) 0 ML J. A. O'S. ESE 54, Lectue 9, 0/7/09 va( (()) ˆ ()) ML Baed Etmato: + db() d l p E ( ) ˆ ( R) ML

30 Maxmum Lkelhood Etmato..d. meauemet of a fucto of a detemtc vaable Gaua oe Solve lkelhood equato ue ome pefeed oluto techque Fhe fomato ealy computed. ote the depedece of pefomace o the tue value of the paamete. p R R g w w w ( ) +,,,...,,..d. (0, σ ) ( R g ( S )) ( R S) exp πσ σ l p ( R S ) ( R g ( S) ) σ ( ) l p ( R S) R g ( S) dg ( S) 0 S S ˆ σ ds S ˆ ML J E ML l p ( ) S dg ( S) S σ ds J. A. O'S. ESE 54, Lectue 9, 0/7/09

31 Maxmum Lkelhood Etmato: Example Computato Ampltude Sgal Eegy dvded by oe powe Fequecy Sgal eegy popotoal to the quae of the umbe of cycle Expoet Potve v. egatve expoet J E dg ( S) g () J ds σ l ( ) p S dg ( S) S σ ds dg ( S ) E k g() k k J ( k) ds σ σ dg ( S) π ( ) g ( ) co( π( ) / M) ( ) / M ds M 4π π J ( ) ( π ( )/ M) M M σ σ dg ( S ) g () e ( ) e J e... ( ) ( ) J ds σ 0 α e α e, e, α α e α α e α 0 e 0 ( π ) α α α α α e ( e ) e ( e α e + ) J. A. O'S. eese 54, Lectue 9, 0/7/09 α 0 α e α α ( e )