11.5 MAP Estimator MAP avoids this Computational Problem!

Transcription

1 .5 MAP timator ecall that the hit-or-mi cot function gave the MAP etimator it maimize the a oteriori PDF Q: Given that the MMS etimator i the mot natural one why would we conider the MAP etimator? A: If and are not jointly Gauian, the form for MMS etimate require integration to find the conditional mean. MAP avoid thi omutational Problem! Note: MAP doen t require thi integration Trade natural criterion v. comutational eae What ele do you gain? More fleibility to chooe the rior PDF

2 Notation and Form for MAP arg ma ˆ MAP ˆMAP Notation: maimize the oterior PDF arg ma etract the value of that caue the maimum quivalent Form via Baye ule: ] arg ma ˆ MAP Proof: Ue ] arg ma arg ma ˆ MAP Doe not deend on

3 Vector MAP < Not a traight-forward a vector etenion for MMS > The obviou etenion lead to roblem: hooe to minimize ˆi ˆ i i i. over, i ˆ i arg ma i -D marginal i conditioned on ˆ Need to integrate to get it!! d d Problem: The whole oint of MAP wa to avoid doing the integration needed in MMS!!! I there a way around thi? an we find an Integration-Free Vector MAP? 3

4 ircular Hit-or-Mi ot Function Not in Book Firt look at the -dimenional cot function for thi troubling verion of a vector ma: It conit of individual alication of -D Hit-or-Mi -δ δ δ -δ, 0,,, in quare, not in quare The corner of the quare let too much in ue a circle! δ 0,, < δ δ Thi actually eem more natural than the quare cot function!!! 4

5 MAP timate uing ircular Hit-or-Mi So what vector Bayeian etimator come from uing thi circular hit-or-mi cot function? Back to Book an how that it i the following Vector MAP ˆ MAP arg ma Doe Not equire Integration!!! That i find the maimum of the joint conditional PDF in all i conditioned on 5

6 How Do Thee Vector MAP Verion omare In general: They are NOT the Same!! amle: /6 /3 /6, The vector MAP uing ircular Hit-or-Mi i:.5 0.5] T To find the vector MAP uing the element-wie maimization: /3 /6 /3 /3 ˆ ˆ.5.5] T

7 Bayeian ML ecall A we kee getting good data, become more concentrated a a function of. But ince: ˆ arg ma arg ma ] MAP hould alo become more concentrated a a function of. arg ma ] Note that the rior PDF i nearly contant where i non-zero Thi become truer a N, and get more concentrated arg ma MAP Bayeian ML Ue conditional PDF rather than the arameterized PDF 7

8 .6 Performance haracterization The erformance of Bayeian etimator i characterized by looking at the etimation error: ˆ andom due to a riori PDF andom due to Performance characterized by error PDF We ll focu on Mean and Variance If i Gauian then thee tell the whole tory Thi will be the cae for the Bayeian Linear Model ee Thm. 0.3 We ll alo concentrate on the MMS timator 8

9 Performance of Scalar MMS timator The etimator i: ˆ d Function of So the etimation error i: f, General eult for a function of two V: Z f X, Y Function of two V Z f, y XY, y d dy Z Z f, y Z, y d dy var Z XY 9

10 0 valuated a een below So alying the mean reult give: 0 0 ] ] ] ], See hart on Decomoing Joint ectation in Note on V Pa through the term Two Notation for the ame thing ] deend on doe not d d 0 i.e., the Mean of the timation rror over data & arm i Zero!!!!

11 And alying the variance reult give: var ˆ ˆ,, d d Ue 0 Bme ˆ So the MMS etimation error ha: mean 0 var Bme So when we minimize Bme we are minimizing the variance of the etimate If i Gauian then ˆ ~ N 0, Bme

12 ..6: D Level in WGN w/ Gauian Prior We aw that Bme Aˆ N / σ + / σ A with σ σ N A A / ˆ + µ σ A σ N σ A σ N + / + / contant contant thi i Gauian becaue it i a linear combo of the jointly Gauian data amle A If X i Gauian then Y ax + b i alo Gauian So  i Gauian ~ N 0, N / σ + / σ A Note: A N get large thi PDF collae around 0. Thi etimate i conitent in the Bayeian ene Bayeian onitency: For large N Aˆ A regardle of the realization of A!

13 Performance of Vector MMS timator Vector etimation error: ˆ Mut etend the variance reult: cov Look ome more at thi: M ˆ,, ] T The mean reult i obviou. M ] ] ˆ ] T T Some New Notation Bayeian Mean Square rror Matri In general thi i a function of See hart on Decomoing Joint ectation General Vector eult: 0 M ˆ 3

14 Why do we call the error covariance the Bayeian MS Matri? The Diagonal lement of To ee thi: M ˆ are Bme of the timate ] T, ii i ] i Bme i ] i i i,, d i d d i d Integrate over all the other arameter marginalizing the PDF 4

15 Perf. of MMS t. for Jointly Gauian ae Let the data vector and the arameter vector be jointly Gauian. Nothing new to ay about the mean reult: 0 Now look at the rror ovariance i.e., Bayeian MSq Matri: ecall General eult: M ˆ Thm 0. ay that for Jointly Gauian Vector we get that doe NOT deend on M ˆ Thm 0. alo give the form a: M ˆ 5

16 Perf. of MMS t. for Bayeian Linear Model ecall the model: H + w ~Nµ, Nothing new to ay about the mean reult: ~N0, w 0 Now for the error covariance thi i nothing more than a ecial cae of the jointly Gauian cae we jut aw: eult for Jointly Gauian ae M ˆ valuation for Bayeian Linear H T H H + T w Alternate Form ee 0.33 M ˆ H T + H H T T H H + w w H 6

17 Summary of MMS t. rror eult. For all cae: t. rror i zero mean 0. rror ovariance for three Neted ae: Bme i M ] ˆ ii General ae: M ˆ Jointly Gauian: M ˆ Bayeian Linear: Jointly Gauian & Linear Obervation M ˆ H T + H H T T H H + w w H 7

18 Main Bayeian Aroache MMS Squared ot Function In General: Nonlinear timate timate: timate: ˆ rr. ov.: M rr. ov.: M timate : rr. ov.: M ˆ Jointly Gauian and Yield Linear timate ˆ + ˆ Bayeian Linear Model Yield Linear timate ˆ ˆ Hard to Imlement numerical integration T T + H HH + w Hµ T T H HH + w H timate : aier to Imlement Determining can be hard to find MAP Hit-or-Mi ot Function ˆ arg ma ay to Imlement Performance Analyi i hallenging ay to Imlement Only need accurate model:, w, H 8

19 .7 amle: Bayeian Deconvolution Thi eamle how the ower of Bayeian aroache over claical method in ignal etimation roblem i.e. etimating the ignal rather than ome arameter t ht Σ t Meaured Data Samle of t Model a a zero-mean WSS Gauian Proce w/ known AF τ Aumed Known wt Gauian Bandlimited White Noie w/ Known Variance So model a D-T Sytem Goal: Oberve t & timate t Note: At Outut t i Smeared & Noiy 9

20 0 Samled-Data Formulation + ] ] 0] ] ] 0] ] ] ] 0 0 0] ] ] ] ] 0] N w w w n n N h N h N h h h h N Meaured Data Vector Known Obervation Matri H Signal Vector to timate AWGN: w w σ I We have modeled t a zero-mean WSS roce with known AF So n] i a D-T WSS roce with known AF m] So vector ha a known covariance matri Toelitz & Symmetric given by: 0] ] ] ] ] ] 0] ] ] ] 0] ] ] ] ] 0] n n Model for Prior PDF i then ~ N0, and w are indeendent

21 MMS Solution for Deconvolution We have the cae of the Bayeian Linear Model o: ˆ T H H + I H σ T Note that thi i a linear etimate Thi matri i called The Weiner Filter The erformance of the filter i characterized by: T + H H M σ ˆ /

22 Sub-amle: No Invere Filtering, Noie Only Direct obervation of with H I + w ˆ t Σ wt t Goal: Oberve t & De-Noie t Note: At Outut t with Noie + σ I M ˆ + I / σ Note: Dimenionality Problem # of arm # of obervation laical Method Fail ˆ Bayeian method can olve it!! For inight conider ingle amle cae: ŝ0] 0] 0] + σ η 0] 0] η η + σ 0] High SN Low SN ˆ 0] 0] ˆ0] 0 Data Driven Prior PDF Driven SN

23 3 Sub-Sub-amle: Secific Signal Model Direct obervation of with H I + w But here the ignal follow a ecific random ignal model ] ] ] n u n a n + un] i White Gauian Driving Proce Thi i a t -order auto-regreive model: A Such a random ignal ha an AF & PSD of ] k u a a k σ f j u e a f P π σ + See Figure.9 &.0 in the Tetbook