Suppose we have observed values t 1, t 2, t n of a random variable T.

Transcription

1 Sppose we have obseved vales, 2, of a adom vaable T. The dsbo of T s ow o belog o a cea ype (e.g., expoeal, omal, ec.) b he veco θ ( θ, θ2, θp ) of ow paamees assocaed wh s ow (whee p s he mbe of ow paamees). Le he desy fco be we as f ( ; θ ). Fo example, f T has omal dsbo, whee θ µ ad θ 2 σ. 2 µ f ( ; µσ, ) exp σ 2π 2 σ MLE page D.L.Bce, 2002 MLE page 2 D.L.Bce, 2002 We wa o esmae he ow paamees by choosg hose vales of θ whch mae he lelhood of he obseved vales as lage as possble. Sce he obseved vales ae depede, he lelhood fco Lθ (, ) s he podc of he pobably desy fco evalaed a each obseved vale: (, θ ) f ( ; L The maxmm lelhood esmao ˆθ s fod by maxmzg L(, wh espec o θ. Ths ˆθ coespods o he dsbo ha s mos lely o have yelded he obseved daa, 2,. The poblem ( Maxmze L, ; θ s a olea opmzao poblem whch mgh be solved by ay appopae NLP algohm (Newo s mehod, he cojgae gade mehod, ec.) MLE page 3 D.L.Bce, 2002 MLE page 4 D.L.Bce, 2002

2 Fo compaoal coveece, s sally pefeable o maxmze he logahm of he maxmm lelhood (whch wll yeld he same maxmzg ˆ: ( Maxmze L θ l, ;.e., becase l L ( ; l f ( ; θ ) l f ( ; we solve he poblem: ( Maxmze l f ; θ Example: Expoeal Dsbo The pobably desy fco (pdf) of he expoeal dsbo wh paamee λ s ( ; ) f λ λe λ We have a se of obsevaos, 2,. Wha s he vale of he paamee λ whch maes hs se of obsevaos mos lely? Sample daa: Tmes o fale of sx elecoc compoes ae ( hos): 25, 75, 50, 230, 430, ad 700. MLE page 5 D.L.Bce, 2002 MLE page 6 D.L.Bce, 2002 Solo: The lelhood fco s λ L(, ; λ ) λ e λ exp λ The logahm of he lelhood s l L ( ; λ ) log λ λ whch has devave d L ( ; λ ) λ λ d I he case, he, we ca solve he olea opmzao poblem (wh oe vaable) by fdg a saoay po,.e., a vale of λ fo whch he above devave s zeo. d ( ; ) 0 ˆ Lλ ˆ ˆ λ dλ λ λ Tha s, he case of he expoeal dsbo, he maxmm lelhood esmao s smply he ecpocal of he aveage of he obseved vales. Fo he sample daa, he, λ ˆ 6fales 6fales fales / h hs 60hs MLE page 7 D.L.Bce, 2002 MLE page 8 D.L.Bce, 2002

3 I he case of he omal dsbo (wh wo paamees, µ & σ), he opmaly codos fo maxmm of he log lelhood s a pa of olea eqaos, b aga hey ca be solved closed fom, ad he esls ae as oe mgh expec: he MLE fo µ s he aveage of he obsevaos, ad he MLE fo σ s he sqae oo of he sample vaace. I geeal, howeve, oe cao fd a closed-fom solo fo he maxmm lelhood esmao(s), eqg a eave algohm. (Fo example, MLE fo Webll & Gmbel dsbos.) Webll Dsbo: pdf: f () exp Sppose, 2, ae mes o fale of a gop of mechasms. The lelhood fco s L( ;, ) exp exp We wsh o choose vales of & whch maxmze L (o eqvalely, he logahm of L),.e., whch mae he obseved vales of as lage as possble! MLE page 9 D.L.Bce, 2002 MLE page 0 D.L.Bce, 2002 The log-lelhood fco s l L ;, l l + l ad so he opmaly codos ae l L ( ;, ) 0 l L ( ;, ) 0 Ths gves s a pa of olea eqaos wo ows: ˆ + ˆˆ 0 l ˆ + l 0 B he lef sde of he fs eqao ca be facoed: ˆˆ + 0 Sce he fs faco cao be zeo, we se he secod faco eqal o zeo ad solve fo û ems of ˆ : Elmag û he secod eqao by sbsg he fs, we ge he followg olea eqao ˆ aloe: l ˆ + l 0 Ths ca ow be solved by, fo example, he seca mehod. MLE page D.L.Bce, 2002 MLE page 2 D.L.Bce, 2002

4 Maxmm Lelhood Esmao wh cesoed daa Sppose ha a expeme was emaed a me τ afe oly of he s a lfees had faled. Ths s accoed fo by defg he lelhood as sce F ( τ; (, θ ) ( τ; θ ) ( ; L F f s he pobably ha he s svve l me τ. The log-lelhood fco s heefoe ( θ ) ( ) ( θ ) + ( l L; l F; l f ; Example: MLE of Webll paamees, gve cesoed daa The CDF of he Webll dsbo s F( ;, ) exp ad so he lelhood fco s τ L ;, exp exp exp ( ) + τ MLE page 3 D.L.Bce, 2002 MLE page 4 D.L.Bce, 2002 The log-lelhood fco s l L ;, l l + l + τ Aga, he ecessay codos fo a maxmm of he log-lelhood a ( ˆ, ˆ ) ae ( ˆ ˆ) ( ˆ ˆ) l L ;, 0 l L ;, 0 A esl smla o he cesoed case ca be deved: ad ˆ ˆ + τ ˆ l l + τ τ + τ + l 0 Ths secod eqao ca be solved fo ˆ by he seca mehod, ad he ˆ sed o calclae û by he fs eqao. MLE page 5 D.L.Bce, 2002 MLE page 6 D.L.Bce, 2002

5 EXAMPLE: Twey devces ae esed smlaeosly l 500 days have passed, a whch me he followg fale mes ( days) have bee ecoded: A plo of Y vs X, obaed by he asfomaos: Y log log whee R() s he obseved faco of he devces whch have R () svved l me, ad X log shold be a le f he Webll model wee o f he daa pefecly. Esmae he lfeme fo whch he devce s 90% elable. MLE page 7 D.L.Bce, 2002 MLE page 8 D.L.Bce, 2002 LEAST SQUARES REGRESSION RESULTS: (scale paamee) (shape paamee) so ha mea sadad devao Noe: hs s deemed by mmzg he sm of he sqaed eos he ( ) leazed veso of F () e, amely y x l, whee x l & y l l, ahe ha he ogal eqao! R () If we se hese paamees fod by lea egesso, he elably fco wold have he vales: F() -F() Hece, accodg o hs model, 90% of he devces shold be opeag a 54.8 (appoxmaely 55) days. MLE page 9 D.L.Bce, 2002 MLE page 20 D.L.Bce, 2002

6 SECANT METHOD Maxmm Lelhood esl: Solvg he olea eqao fo : l l + τ τ + τ g + l 0 If o fs wo gesses a he vale of ae 0.5 ad 2.0, he we deeme ha g(0.5).3739 & ad g(2.0) The seca jog he wo pos o he gaph of g coss he axs a.4787., We he epea, wh he 2 mpoved gesses 0.5 ad MLE page 2 D.L.Bce, 2002 MLE page 22 D.L.Bce, 2002 SECANT METHOD RESULTS: eo E E 2 Oce we deeme he vale of ˆ whch maxmzes he lelhood fco, he he coespodg vale of he paamee û s fod by ˆ ˆ + τ ˆ MAXIMUM LIKELIHOOD RESULT: (scale paamee) , (shape paamee) F() -F() Accodg o hs model, he, 90% of he devces shold be opeag a (appoxmaely 74) days. MLE page 23 D.L.Bce, 2002 MLE page 24 D.L.Bce, 2002