ESE 54 Detecton and Etmaton Theoy Joeph A. O Sullvan Samuel C. Sach Pofeo Electonc Sytem and Sgnal Reeach Laboatoy Electcal and Sytem Engneeng Wahngton Unvety 411 Jolley Hall 314-935-4173 (Lnda anwe) jao@wutl.edu 1
Announcement Poblem Set 3 Due n Cla /7 We wll chedule two make-up clae on Fday Fday Ma. Anothe Fday afte png beak Mdtem Exam: Mach 7 o 9? Othe announcement? Queton?
Outlne: Intoducton to Etmaton Theoy Range of poblem tuded Mnmum mean cot poblem Mnmum mean quae eo etmaton Mnmum abolute eo etmaton Maxmum a poteo etmaton Othe Maxmum lkelhood fo nonandom paamete Fhe nfomaton and the Came-Rao bound 3
Range of Etmaton Theoy Poblem Studed d Random Paamete Cot Functon Mean Squae Eo Mean Abolute Eo Lkelhood Functon: Maxmum a poteo Othe mean cot Soluton Poteo Mean Medan Lkelhood equaton Genealzed mean Nonandom Lkelhood Functon: Lkelhood paamete Maxmum lkelhood equaton Othe cot 4
Random Paamete Etmaton Po on the paamete p ( S ) Condtonal pdf on the p ( R S) data gven the C [ SR, ˆ ( )] paamete E{ C[,()] ˆ } Baye Rule gve poteo pdf ˆ* ag mn E C [,()] ˆ Cot functon gven ˆ Select that etmato that mnmze the mean cot p (S) p ( R S) ŝ() { } 5
Random Paamete Etmaton Etmato a functon Fo each data pont, etmato ngle- valued Mnmze condtonal mean cot Genealzed noton of mean p (S) p ( R S) ŝ() p ( S) p ( R S ) E C[,()] ˆ { } { } ˆ * ag mn E C[, ˆ()] { ˆ } ˆ { [,()] ˆ } E E{ C[,()] ˆ } ˆ E C[,()] C[ SR,( )] p ( R S) p ( S) dsr d E C { } ˆ C[ S, ( R)] p ( S R) ds p ( R) dr R ˆ( ) agmn C[ S, ˆ] p ( S R) ds ˆ 6
Mnmum Mean Squae Eo Etmaton Cot equal quaed eo Anwe the MMSE etmato t Mnmze equal poteo mean Hold fo andom vaable and andom vecto C [ S, ˆ] S ˆ ˆ ˆ R ˆ( ) ag mn C[ S, ˆ] p ( S R) ds ag mn S ˆ p ( S R) ds { } E ( ) { } ˆE ˆ ˆ ( E ˆ ) [ ] 0 R ˆ( ) E[ R] 7
Mnmum Mean Squae Eo Etmaton Repeated meauement of a Gauan andom vaable n Gauan noe Poteo mean a lnea combnaton of data Pefomance? MSE(mean eo) +(eo vaance) Name: MMSE MSE acheved a mnmum meauement of a ˆ( R ) E [ R ] p p N (0, σ ) + w, 1,,..., N, w..d. N (0, σ n) w p ( R S) p( S) ( S R) p ( R) N ( ) 1 R S 1 S exp exp 1 πσ σ n σ n πσ ( S R) Z1( R) N S S RS exp σ 1 σn σn Z ( ) R ( S μ ) exp σ N a Rσ a 1 1 N, μ, + Z 3( R) 1 σ n σ a σ σ n Z ( 3 R ) πσ a 8
MMSE Etmaton fo a Gauan Poteo vaance equal po vaance dvded by one plu the gnal to noe ato (SNR) SNR equal gnal enegy dvded by noe powe Etmate equal a caled veon of the mean of the meauement Scale equal SNR dvded by 1+SNR Poteo tandad d devaton deceae a the quae oot of the numbe of ndependent meauement Z N (0, σ ) w N w w +, 1,,...,,..d. N (0, σ n) [ ] ˆ( R ) E R p 3 exp ( S μ ) σ N a Rσ a 1 1 N ( S R), μ, + Z3( R) 1 σn σa σ σn ( R) πσ a σσ 1 1 σ σ σ + 1+ SNR n a σn Nσ Nσ 1+ σ n σ ˆ( R) μ a σ n Nσ N N N SNR N N N N σ n 1 SNR 1 R R R 1 Nσ 1 1 1 1 + + σ n n n If SNR 1, σa σ σ, and σ σ a SNR N N 9
Mnmum Abolute Eo Etmaton Defned hee fo a ngle vaable Etmate the medan of the poteo MAE etmate moe obut to outle n the data, penalze lage eo le than MMSE Pefomance? CS [, ˆ] S ˆ ˆ ( R ) agmn C [ S, ˆ] p ( S R) ds agmn S ˆ p ( S R) ds ˆ ˆ ˆ ( ˆ ) ( ˆ + ) ˆ ˆ R ˆ { ˆ } ( ) ˆ ( ) 0 R R ag mn S p ( S ) ds S p ( S R) ds E p S ds p S ds ˆ ˆ R ˆ p ( S ) ds p ( S R) ds ˆ( R) medan of poteo 10
Range of Etmaton Theoy Poblem Studed d Random Paamete Cot Functon Mean Squae Eo Mean Abolute Eo Lkelhood Functon: Maxmum a poteo Othe mean cot Soluton Poteo Mean Medan Lkelhood equaton Genealzed mean Nonandom Lkelhood Functon: Lkelhood paamete Maxmum lkelhood equaton Othe cot 11
Maxmum a Poteo Etmaton Cot one f eo geate than ε and zeo othewe Conde ε to be mall Maxmze the poteo pobablty denty functon: MAP etmate e Alo called the mode of the poteo Pefomance? 1, S ˆ >ε CS [, ˆ] 0, othewe ˆ ( R ) agmn CSp [, ˆ ] ( S R ) ds ˆ ˆ ˆ ε ag mn p ( S R) ds + p ( S R) ds ˆ + ε ˆ + ε J. A. O'S. ESE 54, Lectue ˆ 8, R ε 0/05/09 ˆ ˆ ε ag max p ( S ) ds ag max p ( ˆ R) 1
Maxmum a Poteo Etmaton Maxmzng a nonnegatve-valued functon equvalent to maxmzng t logathm Thnk about the functon plotted on a db cale Ue Baye Rule to epeent the poteo n tem of the po and the condtonal lkelhood Denomnato not a functon of S Mut olve the ˆ MAP ( R ) ag max ε p ( S R) S lkelhood equaton p ( S R) MAP and MAE 0 etmate fo the Gauan poblem equal the MMSE etmate, the poteo mean S S ˆ MAP p ( R S) p( S) p ( S R) p ( R) ln p ( S R ) ln p ( R S ) + ln p ( S ) ln p ( R) ln p ( R S) ln p ( S) 0 + S S ˆ S S ˆ MAP MAP 13
Othe Cot Functon Paamete may take many fom Ampltude, fequency, phae Intenty of a Poon (concentaton of adoactve ubtance) Vaance of noe n an amplfe o ccut Decton: SO(3); dtance and decton: SE(3) Subpace n gnal pace Defomaton o wapng: mage o volume wapng Dtance o othe dcepancy mut be defned on the paamete pace Nonnegatve, zeo at tuth, monotonc n ome ene Example: map paamete nto a matx and ue a matx-dtance (o dtance quaed lke um of quae eo) to nduce a dcepancy n paamete pace 14
Range of Etmaton Theoy Poblem Studed d Random Paamete Cot Functon Mean Squae Eo Mean Abolute Eo Lkelhood Functon: Maxmum a poteo Othe mean cot Soluton Poteo Mean Medan Lkelhood equaton Genealzed mean Nonandom Lkelhood Functon: Lkelhood paamete Maxmum lkelhood equaton Othe cot 15
Nonandom Paamete Etmaton Thee no po on the paamete. Concentate on the maxmum lkelhood ule: fnd the paamete that maxmze the lkelhood functon o equvalently the loglkelhood functon. Nonandom paamete veon of MAP etmaton Pefomance? ˆ ( R) ag max p ( R S) ag max ln p ( R S) ML S S 0 ln p ( R S) S S ˆ ML 16