Estimation Theory. Chapter 10

Size: px

Start display at page:

Download "Estimation Theory. Chapter 10"

Audra Francis
5 years ago
Views:

1 Estimation hor hatr

2 Basian Estimation ow assum that w hav som rior nowldg about. o incororat it, assum is random variabl with a givn df and stimat as a raliation of this random variabl PRIOR KOWLEDGE Eaml: D Lvl in WG but instad of - << Â still MVU but w can do bttr b incororating this rior nowldg on & introducing bias Pdf of Â Imossibl Valus Â

3 Â > omar MSEs Basian Estimation lads to imossiblstimats for onsidr truncatd saml manbiasd - : - of : < : > Â and r { < } onsistnt with nown constraints ; { } r > - Ă

4 Basian Estimation { } { } ; ; ; ; ; ; ; ˆ ] [ ; ; ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ms d d d d d d d ms u u r r > > < δ δ bo function

5 Basian Estimation For - bttr than Â in trms of MSE Basian roach considrd to b a Random Variabl with rior PDF, w attmt to stimat th raliation of PDF - W[n] hoos [n] Basian Philosoh

6 Basian Estimation Otimalitcritrion is Basian MSE BMSEÂ E[Â - ], Joint PDF lassical : MSEÂ With rsct to, Â ; d Basian : BMSEÂ Â - ; d d MSE dnds on, BMSE dos not. Basic data gnrating mchanism is diffrnt

7 omutr Erimnt lassical Gnrat M raliations of W[n], add to ach on a Givn Basian Gnrat W[n], thn gnrat from U[-, ], assuming is indndnt of W[n], rat rocdur M tims MSE will dnd on, BMSE will not. doting BMSE as otimalit critrion will not lad to Â which is a function of no raliabilit roblms

8 o Minimi BMSEÂ Â Â Basian Estimation ˆ Â - [ ] Â - d Minimi for givn sinc,dd d ˆ d / d sinc Â E conditional man d / d Man of ostrior PDF minimis BMSE. onlinar in gnral!

9 ot : is continuing aml Basian Estimation rior PDF bfor data is obsrvd d ormaliing Factor d But W[n]is assumd to b indndnt of [n] W W Bas' rul this is wh rocdur calld Basian aroach [ n] [ n].

10 Basian Estimation π π [ n] n [ n] for WG Idntical form as ; in classical cas - conditional PDF would not b th sam if indndnt of W[n] But this is a wr not

11 Basian Estimation ] [ ] [ : : d " ] [ ] [ [n]- But : : d " n n n n n > > π π

12 Basian Estimation, toif intgrats normaliing constant so that > π P - Prior PDF

13 Basian Estimation P/ Postrior PDF - ow Â E - π d - π an t b valuatd in closd form vn for this siml aml! d

14 Data vs. Prior Knowldg P/ - E/ Â is biasd toward ro man of rior PDF of Unlss o truncation or no >> rior nowldg ot that bfor data is obsrvd, all w now about is in whos man is ro

15 Data vs. Prior Knowldg an show that EÂ- Â- d s SK.8 whr Â now is minimid for ÂE Basian stimator E/ is a comromis btwn rior nowldg stimator ÂE and data nowldg stimator Â s P/ ˆ E

16 Data vs. Prior Knowldg Data nowldg swams out rior nowldg. In Gnral, ˆ E d In gnral, this stimator is non-linar and can t b comutd in closd form ct for scial cass th most imortant of which is th Gaussian rior df

17 hoosing Prior PDF Should b basd on hsical constraints of roblm if ossibl. lso choos rior df to allow as intgration o find /d o find E//d Eaml D Lvl in WG Gaussian Prior PDF Us π for -

18 Gaussian Prior Eaml [n] ] [ ] 3, 3 [ or in th intrval, b nar that will Strong blif π π π n d S SK for roof

19 Gaussian Prior Eaml E Â MMSE stimator is Gaussian but with diffrnt man and varianc. Postrior PDF Whr

20 Â For Â For stimat. rior stimat and data comromis btwn, wighting factor is ot Â >> << < < α α α α α α Gaussian Prior Eaml

21 Gaussian Prior Eaml lso as, >> and w gt "lassical"estimator. < < 3 Â 3 Â Â μ o show that rior nowldg imrovs stimation accuracon th avrag as masurd b Basian MSE : BMSEÂ EÂ -

22 Gaussian Prior Eaml ˆ ˆ E d d, d d d d Var/ BMSEÂ Var d Varianc of ostrior PDF avragd ovr all raliations of. BMSEÂ d

23 hings Gaussian Prior Eaml o rior nowldg BMSEÂ whn word out wll sinc < Gaussian Gaussian or Rroducing Prort

24 Bacground :Bivariat Gaussian PDF - E E E E Lt, Eign - dcomosition : E E E E whr, U U U U U U > π π,,, E -is -is E

25 Bivariat Gaussian PDF [ ]. b th unitar transformation llismajor and minor as and rotation of E, E origin to of llis through transformation rlatd to thiscanonical llis is Original llis "canonical", and - E ; - E Dfin ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ U U ~ ~,

26 Bivariat Gaussian PDF If w lot a scattr diagram of a bivariat RV, w gt an llis sinc th most lil, airs satisf, o which dscribs oints insid an llis ~ o ~ o -is -is Show that! Positivl orrlatd gativl orrlatd -is -is

27 Bivariat Gaussian PDF,, whr,, th bivariat Gaussian PDF is givn b It can b shown that ltrnativ form : ρ ρ ρ π ρ ρ

28 Bivariat Gaussian PDF Summar:. Whn ρ uncorrlatd & ; U I th as of th llis coincid with th and as.. If in addition, circl 3. If in addition to&, circl cntrd at origin

29 Prortis of Bi-Variat Gaussian PDF, E,Var, ovr or w find B intgrating,,,, Var E Var ov ov Var E E E E π E E E/ ontours of onstant Probabilit onstant ] [ E E and ar jointl Gaussian ach llis corrsonds to diffrnt P/

30 Prortis of Bi-Variat Gaussian PDF onditional df normalid tointgrat to on. lso, Gaussian sinc E Var, is cross -sction of,, isgaussian in. E Var cov, var cov, var,, d whn suitabl E must b horm s ndi for roof : If & ar jointl Gaussian, thn th conditional PDF P/ is also Gaussian and

31 Prortis of Bi-Variat Gaussian PDF Bfor obsrving, man changs and varianc dcrass - lss uncrtainit about. Var whr ρ E, Var. cov, Var - Var Var Var- ρ cov, VarVar ftr obsrving, orrlatio officin n t ρ Rcall E is MMSE stimator of. Ŷ E cov, var E Linar!

32 Prortis of Bi-Variat Gaussian PDF Or Ŷ - E Var Dnots ŷ n ormalid cov, Var Var ρ E Var n ˆ E ρ ρ Minimum BMSE stimator loits corrlation btwn random variabls to stimat th raliation of on basd on th raliation of th othr

33 Prortis of Bi-Variat Gaussian PDF lso, BMSEŷ Var/ Var- ρ If ρ ±, W stimat rfctl If ρ,, ar uncorrlatd indndnt Ŷ E BMSEŶ Var

34 Gnraliing o Multivariat Gaussian for roof s ndi whr Gaussian multivariat is also hn, Jointl Gaussian, If l l l l E E E, E E l l π

35 lication : Basian Linar Modl In introductor aml [n] W Loos lilinar modl ct for bing random variabl. onsidr, ow assum is random vctor with PDF, and indndnt of W. n,,... - H Known Matri Random Vctor W, W W[n] WG-Indndnt of W

36 Basian Linar Modl d for Basian stimator. But ostrior PDF is Gaussian sinc, ar jointl Gaussian. Lt Z H W H I Z I W, W ar indndnt and marginall Gaussian h ar jointl Gaussian. Sinc Z is a linar function of, it is also Gaussian W

37 Basian Linar Modl Lt H W nd al MultivariatGaussian Rsults. E E H W HE E E E E E [ ] E E [ ] H W H [ ] H W H μ

38 [ ] [ ] H H H H H H H H E H W H E E E E W H H E WW H HE W W W w hav ar indndnt,, Sinc Basian Linar Modl

39 W Â E Â α is calld th bias factor. Bac to Eaml D Lvlin WG - Gaussian rior [n] W[n] n,,..., -, W[n] ar indndnt Us Woodbur's idntit to invrt matri s ttboo Prior Estimat gain Data orrction "Error" It has th formα α H W I SR SR I α is a biasd stimator!! unbiasdmvu unbiasdmvu

40 Eaml ont d lso, MSE for a ovr is givn b MSE ˆ Var var var ˆ MSEˆ α If w us th Basian aroach to stimat, α articular α α raliation of E ˆ I Find th intrsction oints thn th ; i.. without avraging α α

41 Eaml ont d Basian stimator with Gaussian rior df is biasd & hibits lss MSE than th classical MVU stimator onl if is clos to th rior man. Howvr, if is random, thn th Basian stimator will b on avrag bttr than th classical stimator sinc Basian MSE is givn b BMSE ˆ α E α [ MSE ˆ] <

42 linar modl. Sam form as fficint stimator for classical For no rior nowldg, 3 W - H H H E W oncluding Rmars indndnt columns t hav H ndn' linar stimation, Unli classical Using standard matri invrsion idntitis E invrs! an also show that si - vs.si - H H H H H H W W W

43 hatr Rviw ε ε ε ε ε ε < < > > > > ; ] [ ; ]'s [ all ; onntiall distributd.3: Prob. Δ ]- [ channl d d, n min n n n w lso, onntial!

44 ^ ^ E Eaml ont d - ε ε ε ε ε - [ ε ε - ] ε min ε [ n] d [ ] min [ n] ε non - linar in data

45 Ercis : Otimum Basian MMSE Soft Dtctor for Uniforml- Distributd M-PM Motivation : In codd communication sstms, th outut of th dtctor is fd to a dcodr to furthr rduc th rror robabilit. Fding soft information to th dcodr rsults in much bttr rformanc than fding hard.g. and - for BPSK information. In this rcis, w driv th otimum Basian MMSE soft dtctor for M-PM and comar it to th subotimal slicr. Problm Statmnt : givn whr is a discrt random variabl drawn with qual robabilit from an M-PM signal constllation, comut th otimum stimator for. Sciali our rsult to th cas of ro-man Gaussian. Furthr sciali our rsult to BPSK both quall and non-quall robabl signaling and 4-PM and comar with th hard slicr.

46 Soft Equi-Probabl M-PM WG MMSE Dtctor / ˆ / / /,, /, / /, /, f f d f f f f d f f f f M d f f f M f M odd M Z M odd M Z M odd M Z M odd M Z M odd M δ

47 Soft Equi-Probabl M-PM WG MMSE Dtctor for larg SR sign sign which aroachs tanh ˆ w gt For BPSK, ˆ For Gaussian ro - man nois ,, M odd M M odd M Etnd th rsults to 4-PM

Data Assimilation 1. Alan O Neill National Centre for Earth Observation UK

Data Assimilation 1. Alan O Neill National Centre for Earth Observation UK Data Assimilation 1 Alan O Nill National Cntr for Earth Obsrvation UK Plan Motivation & basic idas Univariat (scalar) data assimilation Multivariat (vctor) data assimilation 3d-Variational Mthod (& optimal