Chapter 7 Maximum Likelihood Estimate (MLE)

Transcription

1 Chapter 7 aimum Likelihood Estimate (LE)

2 otivatio for LE Problems:. VUE ofte does ot eist or ca t be foud <See E. 7. i the tetbook for such a case>. BLUE may ot be applicable ( Hθ w) Solutio: If the PDF is kow, the LE ca always be used!!! This makes the LE oe of the most popular practical methods dvatages: Disadvatages:. It is a Tur-The-Crak method. Optimal for large eough data size. ot optimal for small data size. Ca be computatioally comple - may require umerical methods

3 Ratioale for LE Choose the parameter value that: makes the data you did observe the most likely data to have bee observed!!! Cosider possible parameter values: θ & θ sk the followig: If θ i were really the true value, what is the probability that I would get the data set I really got? Let this probability be P i So if P i is small it says you actually got a data set that was ulikely to occur! ot a good guess for θ i!!! But p p(;θ ) d p p(;θ ) d θ p ; θ ) pick so that L ( L is largest 3

4 Defiitio of the LE θl is the value of θ that maimizes the Likelihood Fuctio p(;θ) for the specific measured data p(;θ) θl θ θl maimizes the likelihood fuctio ote: Because l(z) is a mootoically icreasig fuctio θl maimizes the log likelihood fuctio l{p(; θ)} Geeral alytical Procedure to Fid the LE. Fid log-likelihood fuctio: l p(;θ). Differetiate w.r.t θ ad set to : l p(;θ)/ θ 3. Solve for θ value that satisfies the equatio 4

5 5 E. 7.3: E. of LE Whe VUE o-eistet w ~ (,) WG ~(,) Likelihood Fuctio:!!!!!! "!!!!! $! # ) ( ep ) ( ) ; ( p π To take l of this use log properties: Take /, set, ad chage to  > Epad this : ) ( ) ( Cacel

6 6 aipulate to get: 4 L Solve quadratic equatio to get LE: Ca show this estimator biased (see bottom of p. 6) But it is asymptotically ubiased Use the Law of Large umbers : Sample ea True ea } { E as { } { } { } E E E E L 4 4 $!#!" CRLB ) var( So ca use this to show: symptotically Ubiased & Efficiet

7 7.5 Properties of the LE (or Why We Love LE ) The LE is asymptotically:. ubiased. efficiet (i.e. achieves CRLB) 3. Gaussia PDF lso, if a truly efficiet estimator eists, the the L procedure fids it! The asymptotic properties are captured i Theorem 7.: If p(;θ ) satisfies some regularity coditios, the the LE is asymptotically distributed accordig to θ L a ~ ( θ, I ( θ)) where I(θ ) Fisher Iformatio atri 7

8 Size of to chieve symptotic This Theorem oly states what happes asymptotically whe is small there is o guaratee how the LE behaves Q: How large must be to achieve the asymptotic properties? : I practice: use ote Carlo Simulatios to aswer this 8

9 ote Carlo Simulatios: see ppedi 7 methodology for doig computer simulatios to evaluate performace of ay estimatio method Illustrate for determiistic sigal s; θ i WG ote Carlo Simulatio: Data Collectio: ot just for the LE!!!. Select a particular true parameter value, θ true - you are ofte iterested i doig this for a variety of values of θ so you would ru oe C simulatio for each θ value of iterest. Geerate sigal havig true θ: s;θ t (call it s i matlab) 3. Geerate WG havig uit variace w rad ( size(s) ); 4. Form measured data: s sigma*w; - choose σ to get the desired SR - usually wat to ru at may SR values do oe C simulatio for each SR value 9

10 Data Collectio (Cotiued): 5. Compute estimate from data 6. Repeat steps 3-5 times - (call # of C rus or just # of rus ) 7. Store all estimates i a vector EST (assumes scalar θ) Statistical Evaluatio:. Compute bias. Compute error RS 3. Compute the error Variace b 4. Plot Histogram or Scatter Plot (if desired) ( θ θtrue ) i i RS VR ( θ θt ) i ow eplore (via plots) how: Bias, RS, ad VR vary with: θ value, SR value, value, Etc. Is B? Is RS (CRLB) ½? i θ i i i θ i

11 E. 7.6: Phase Estimatio for a Siusoid Some pplicatios:. Demodulatio of phase coheret modulatios (e.g., DSB, SSB, PSK, Q, etc.). Phase-Based Bearig Estimatio Sigal odel: cos(πf o φ) w,,,, - Recall CRLB: ad f o kow, φ ukow var () φ σ SR White ~(,σ ) For this problem all methods for fidig the VUE will fail!! So try LE!!

12 So first we write the likelihood fuctio: p( ; φ ) ep cos( πf ) o φ σ ( ) $! πσ!!!! #!!!!! " GOL: Fid φ that maimizes this equivalet to miimizig this Ed up i same place if we maimize LLF So, miimize: J ( φ ) cos( πf φ ) o Settig ( φ ) J φ gives ( πf φ ) si( πf φ ) cos( πf φ ) si o o o $!!!!! #!!!!!! " si ad cos are whe summed over full cycles So LE Phase Estimate satisfies: ( ) si πf o φ Iterpret via ier product or correlatio

13 ow usig a Trig Idetity ad the re-arragig gives: Or cos( ) φ si o L ta φ ( πf ) si( ) φ cos( πf ) Recall: I-Q Sigal Geeratio LPF si cos y i (t) ( πf ) o ( πf o) o Recall: This is the approimate LE Do t eed to kow or σ but do eed to kow f o (t) cos(πf o t) -si(πf o t) LPF y q (t) The sums i the above equatio play the role of the LPF s i the figure (why?) Thus, L phase estimator ca be viewed as: ata of ratio of Q/I 3

14 ote Carlo Results for L Phase Estimatio See figures 7.3 & 7.4 i tet book 4