Detection and Estimation Theory

Transcription

1 ESE 54 Detecto ad Estmato heory Joseph A. O Sullva Samuel C. Sachs Professor Electroc Systems ad Sgals Research Laboratory Electrcal ad Systems Egeerg Washgto Uversty Urbauer Hall (Lyda aswers jao@wustl.edu J. A. O'S. ESE 54, Lecture -3, /4,6,7/9

2 Aoucemets Problem Set 4 s Due uesday Mar. 3 Make-up class Frday Feb. 7, here AotherFrdayaftersprgbreak after Mdterm Exam? Other aoucemets? Questos? J. A. O'S. ESE 54, Lecture -3, /4,6,7/9

3 Outle: Itroducto to Estmato heory Maxmum lkelhood for oradom parameters Bouds o MSE for radom parameters a Matrx Fsher formato ad Cramer-Rao bouds J. A. O'S. ESE 54, Lecture -3, /4,6,7/9 3

4 Maxmum Lkelhood Estmato..d. measuremets of a fucto of a determstc varable Gaussa ose Solve lkelhood equato use some preferred soluto techque Fsher formato s easly computed. ote the depedece of performace o the true value of the parameter. p s R R r g s w w w = ( +, =,,...,,..d. (, σ s ( R g ( S ( R S = exp = πσ σ r s l p ( R S = ( R g ( S r s = σ ( l p ( R S R g ( S dg ( S = = r s S S sˆ = σ ds = S= sˆ ML J = E = ML l pr s ( r S dg ( S S = σ ds J. A. O'S. ESE 54, Lecture -3, /4,6,7/9 4

5 Maxmum Lkelhood Estmato: Example Computatos Ampltude Sgal Eergy dvded by ose power Frequecy Sgal eergy s proportoal to the square of the umber of cycles Expoet Postve vs. egatve expoet J = E = dg ( S g ( s = s = J = ds σ l ( pr s r S dg ( S S = σ ds dg ( S E k g( s = sk = k J = ( k = ds σ σ dg ( S π ( g ( s = cos( πs( / M = s s( / M ds M 3 4π π J = ( s ( π s ( / M M 3M σ σ = dg ( S g ( s = e = ( e J = e =... = s ( s ( s J ds σ = α e α e =, e =, α α e α α e α = e = α α J. A. O'S. ESE 54, Lecture -3, e /4,6,7/9 α = α e ( π ( ( α ( e e e e + e e = = α α α α α 5

6 Frequecy Estmato Suppose that a susod s observed addtve whte Gaussa ose Estmate the frequecy; uque frequeces are <s<.5 because cose s a eve fucto he Fsher formato J really defes the performace. A proxy for the performace s a approxmate sgal-to-ose rato that takes s.5 r = ag( s + w, =,,..., he performace s much dg ( S g ( s = cos( π s ( = π ( s π s ( better tha ths SR due to the olearty of the susod ds ( l ( p s S r r a dg ( S J = E = S = σ ds 4π a π a J = ( s ( π s( σ = 3σ 3 a ( + ( + SR =, k = π J SR 3 J. A. O'S. ESE 54, Lecture -3, /4,6,7/9 3 σ k = 6

7 Frequecy Estmato See Matlab Code SR=5 to 5 db depedet trals Sgal legth= Frequecy=/6; estmates restrcted to the rage from.5 ote threshold h performace behavor typcal of frequecy estmato Lower rght shows the approxmate CR Boud usg the formula o the prevous slde ad the exact Sg al Value ad CRLB for Frequecy Estmato ScaleMSE MSE of Estma ate, db wo sgals ad Oe Data Realzato me Sample SR J. A. O'S. ESE 54, Lecture -3, /4,6,7/9 db MSE Ampltude Sgal SR db CR Boud (red ad Approx CR Boud SR db 7

8 Frequecy Estmato Matlab Code fucto [sampperf,crboud,srrage]=freqsgals3(s,,k,srm,srmax srrage]=freqsgals3(s SRmax SRstep=(SRmax-SRm/; srrage=srm:srstep:srmax; legthsr=legth(srrage; % SR=*log(a^*M*K/(*sgma^ a=sqrt(*exp(log(*srrage//(k; ll=:(k-; sgal=a'*cos(*p*s*ll; ose=rad(,k; scaddate=:.:.5; : ds=.; scadll=scaddate'*ll; caddatesgal=cos(*p*scadll; dervsg=-s(*p*s*ll*dag(*p*ll; g( p derveergy=sum(dervsg.^; crboud=./(derveergy*a.^; kle=; sampperf=zeros(,legthsr; whle kle<legthsr, kle=kle+; r=oes(,*sgal(kle,:+ose; % Fd the max over caddate frequeces % hs ca be a straght maxmzato or a soluto of the lkelhood % equato. Here, we mplemet a straght maxmzato over % *r*s'-s*s'. lkelhoods=*a(kle*caddatesgal*r'-((a(kle^*... dag(caddatesgal*caddatesgal'*oes(,; [lkemax,dexs]=max(lkelhoods; sdex=scaddate(dexs; sampperf(kle=sum((s-sdex.^/; ed J. A. O'S. ESE 54, Lecture -3, approxfsherfo=(*k^3*p^*a.^/3; /4,6,7/9 8

9 Example: Ampltude ad Phase Estmato kπ r k = as + φ + k, k =,,...,,,..d. depedet of a ad φ. s eve. k k (, σ ( a Suppose a ad φ are depedet radom varables. A, β A β a e a φ, π φ π π Fd the MAP estmates for a ad φ. ( b Gve π φ π ad a, fd the maxmum lkelhood estmates for a ad φ. J. A. O'S. ESE 54, Lecture -3, /4,6,7/9 9

10 Example: Ampltude ad Phase Estmato here are multple approaches to a soluto. We outle both a drect ad a drect approach, argug that the drect approach s easer. For the drect approach, ( gve a = A ad φ =Φ, r s μ, σ I, where μ s the mea vector. ( k kπ kπ kπ ( Acos Φ, k odd μk = As +Φ = As cosφ+ Acos s Φ = ( Φ μ AcosΦ B = B, = = AsΦ B B k - As, k eve l pr a, φ ( R A, Φ = ( R μ ( l ( πσ σ I R μ = ( RR Rμ+ μ μ l ( πσ σ Acos Φ Acos Φ A μμ= [ Acos Φ As Φ ] BB = [ Acos Φ As Φ ] = As Φ As Φ Acos Φ R μ = R B R B As Φ J. A. O'S. ESE 54, Lecture -3, /4,6,7/9

11 Example: Ampltude ad Phase Estmato For MAP estmato, there are two equatos to be solved jotly: l p( R, A, Φ l p( R, A, Φ =, =, A Φ l p( R, A, Φ = l p( R A, Φ + l p( A + l p( Φ A β A a e a β,, φ, π φ π, π l p( R, A, Φ A cos Φ A = + B B A σ σ R R s + = Φ A β Set ths to zero. Solve the quadratc equato for the MAP estmate. he ampltude estmate depeds o the phase estmate, where the phase solves l p( R, A, Φ A s Φ = B B Φ σ R R cos = Φ ˆ R B ( ˆ R B φmap ( R = ta cos φmap ( R = R B J. A. O'S. ESE 54, Lecture -3, R B + R B /4,6,7/9 ( (

12 Example: Ampltude ad Phase Estmato he alteratve or drect approach s to chage coordates o the varables tobe estmated, defg ew radom varables x ad y. Estmate them, the fd a ad φ. x acosφ x β, y = asφ y,, β X p( R X, Y B, σ I Y l p( R X, Y l p( X, Y R B X X + = = X X σ σ β R B R B xˆ (, ˆ ( MAP R = y, MAP R = σ σ + + β β ( ˆ ˆ ymap R φ ta, ˆ ˆ ( ˆ ( ( MAP = amap = xmap R + ymap R = R B + R xˆ MAP ( R σ + β ˆ ˆ J. A. O'S. ESE 54, Lecture -3, φ ˆ ˆ ( ˆ ML = φmap, aml = xml R + yml ( R = ( R B ( /4,6,7/9 + R B ( B

13 Example: Ampltude ad Phase Estmato Maxmum lkelhood estmates Suffcet statstcs ( R XY Loglkelhood ratos relatve to a fcttous p ( R ull hypothess (ose oly. For a properly r (, σ I. chose ull p XY p hypothess, ths removes terms that = R μ μ μ do ot deped o the σ parameters. l p ( R X, Y, XY, ( R X, Y p( R p arg max p X, Y = arg max, for ay p Select basedodetectotheoryresults detecto theory results. Usethe "ose oly" hypothess, ( R l ( R, l ( R = ( R μ I ( R μ RR σ σ ( R B X X R σ σ = xˆ ( ˆ ML R = R B, yml ( R = R B, ˆ ( ˆ yml R φ ta, ˆ ˆ ˆ ML = aml = xml R + yml R = xˆ R R B + R B ( ML J. A. O'S. ESE 54, Lecture -3, /4,6,7/9 ( ( ( ( 3

14 Asymptotc t Propertes rt of MLE Cosstecy (covergece to truth as gets large aˆ ( r a probablty as Asymptotc effcecy (asymptotc varace a equal to CRB ML lm E [ aˆ r ] var ( ML l p ( r a A Asymptotc Gaussa aˆ r a σ dstrbuto ML ( (, CRB Commet: for fte sample szes, other estmators may be better = J. A. O'S. ESE 54, Lecture -3, /4,6,7/9 4

15 Example: Expoetal Dstrbutob R λ r e, R, d..d. λ R l p( R λ = l λ+ λ Expoetally = dstrbuted radom p R λ varables Estmate t the mea Maxmum lkelhood estmate s ubased estmate s ubased ( Cramer-Rao Boud s acheved l ( R = λ λ λ = ˆ λml ( R = R λ = ˆ E λml ( R λ = E r λ = R R R R λ λ λ E( r = e dr = R e + Re dr λ Ca we do better? E[ r] = λ, E( r, E ( r E ( ˆ λml ( R λ = λ λ = λ λ = = CRB J. A. O'S. ESE 54, Lecture -3, /4,6,7/9 5

16 Example: Expoetal Dstrbutob Cosder multplyg the MLE by a costat Fd the costat to mmze the mea square error over all meas Geeralzato: that mmzato usually depeds o the varable, so select the worst case varable Result: MSE < CRB; varace < CRB; based estmate Uformly better performace tha the MLE ( ˆ ML R = + ( λ m E αλ ( λ m α α λ α α + m = λ α α + α λ = + α = + ˆ( λ R = R + = E ˆ( λ r = λ = λ λ + + var ˆ λ λ λ( r = < + ( J. A. O'S. ESE 54, Lecture -3, /4,6,7/9 6

17 Radom Parameter r MSE Bouds heorem: Let sˆ ( r be estmate of the radom varable s. Assume that p( R, S p( R, S ad exst ad are absolutely tegrable. he S S Es [( ˆ r s] = l p( r, s l p( r, s E E S S S wth equalty f ad oly f - Posteror s l p ( R, S l p ( S R Gaussa = k =. S S - MAP=MMSE - Both effcet he p s ( S R = exp ks + c( R S + c( R. r (meet MSE J. A. O'S. ESE 54, Lecture -3, /4,6,7/9 boud 7

18 Performace Bouds for Vectors of Parameters: oradom Parameters Matrx equalty: dfferece of matrces s oegatve defte Fsher formato matrx Covarace matrx for ay ubased estmator s greater tha or equal to the verse of the Fsher formato matrx l p( l p( E r s r s J = j s sj J = E s p r s s p r s = E ( l p( s s r s sr ˆ( s x = l s p( r s Κε I E xx = I J Bouds o dvdual varaces σ, J ε J Bouds o ay square submatrx (mors Κε I ε Bouds o bouds by = vertg part of Fsher formato matrx J. A. O'S. ESE 54, Lecture -3, /4,6,7/9 ( l ( ( l ( = ( Κ J ( I JΚε ( I Κε J ( J Κε I J 8

19 Mtr Matrx Bouds Matrx equalty: dfferece of matrces oegatve defte Bouds o dvdual varaces Bouds o ay square submatrx Bouds o bouds by vertg part of Fsher formato matrx sr ˆ( s x = l p( s r s Κε I E xx = I J σ ε, J = J I ( Κε J ( I JΚε Κε = I J ( I ΚεJ ( J Κε ( ( ( B Κ B Κ B = J. A. O'S. ESE 54, Lecture -3, B Κ ( /4,6,7/9 Κ B Κ B Κ B Κ + Κ B Κ Κ B Κ B Κ B Κ B B Κ B Κ 9

20 Mtr Matrx verso r lemma A rak m update to a matrx correspods to a rak m update of ts verse Subtractg a oegatve e defte matrx from a matrx correspods to addg a oegatve defte matrx to ts verse Κ ( ( B = ( + ( ( Κ BΚ B = Κ + Κ B ( Κ B Κ B B Κ Κ B Κ B Κ B Κ B B Κ Κ B Κ B Κ B Κ B B Κ B Κ Κ B Κ B Κ B Κ Κ B Κ B Κ B B Κ J. A. O'S. ESE 54, Lecture -3, /4,6,7/9

21 Lear Estmato X ad Y are jotly Gaussa. Problem : Fd the expected value of X gve Y. X ad Y have kow secod order statstcs Problem : Amog all lear estmates of X as a fucto of Y, fd the oe that mmzes MSE Aswer = Aswer Fudametal Property: he error the estmate s orthogoal to the varables used the estmate. Error covarace matrx J. A. O'S. ESE 54, Lecture -3, /4,6,7/9