Sppose we have obseved vales, 2, of a adom vaable T. The dsbo of T s ow o belog o a cea ype (e.g., expoeal, omal, ec.) b he veco θ ( θ, θ2, θp ) of ow paamees assocaed wh s ow (whee p s he mbe of ow paamees). Le he desy fco be we as f ( ; θ ). Fo example, f T has omal dsbo, whee θ µ ad θ 2 σ. 2 µ f ( ; µσ, ) exp σ 2π 2 σ MLE page D.L.Bce, 2002 MLE page 2 D.L.Bce, 2002 We wa o esmae he ow paamees by choosg hose vales of θ whch mae he lelhood of he obseved vales as lage as possble. Sce he obseved vales ae depede, he lelhood fco Lθ (, ) s he podc of he pobably desy fco evalaed a each obseved vale: (, θ ) f ( ; L The maxmm lelhood esmao ˆθ s fod by maxmzg L(, wh espec o θ. Ths ˆθ coespods o he dsbo ha s mos lely o have yelded he obseved daa, 2,. The poblem ( Maxmze L, ; θ s a olea opmzao poblem whch mgh be solved by ay appopae NLP algohm (Newo s mehod, he cojgae gade mehod, ec.) MLE page 3 D.L.Bce, 2002 MLE page 4 D.L.Bce, 2002
Fo compaoal coveece, s sally pefeable o maxmze he logahm of he maxmm lelhood (whch wll yeld he same maxmzg ˆ: ( Maxmze L θ l, ;.e., becase l L ( ; l f ( ; θ ) l f ( ; we solve he poblem: ( Maxmze l f ; θ Example: Expoeal Dsbo The pobably desy fco (pdf) of he expoeal dsbo wh paamee λ s ( ; ) f λ λe λ We have a se of obsevaos, 2,. Wha s he vale of he paamee λ whch maes hs se of obsevaos mos lely? Sample daa: Tmes o fale of sx elecoc compoes ae ( hos): 25, 75, 50, 230, 430, ad 700. MLE page 5 D.L.Bce, 2002 MLE page 6 D.L.Bce, 2002 Solo: The lelhood fco s λ L(, ; λ ) λ e λ exp λ The logahm of he lelhood s l L ( ; λ ) log λ λ whch has devave d L ( ; λ ) λ λ d I he case, he, we ca solve he olea opmzao poblem (wh oe vaable) by fdg a saoay po,.e., a vale of λ fo whch he above devave s zeo. d ( ; ) 0 ˆ Lλ ˆ ˆ λ dλ λ λ Tha s, he case of he expoeal dsbo, he maxmm lelhood esmao s smply he ecpocal of he aveage of he obseved vales. Fo he sample daa, he, λ ˆ 6fales 6fales 0.0037267 fales / h. 25+75+50+230+430+700 hs 60hs MLE page 7 D.L.Bce, 2002 MLE page 8 D.L.Bce, 2002
I he case of he omal dsbo (wh wo paamees, µ & σ), he opmaly codos fo maxmm of he log lelhood s a pa of olea eqaos, b aga hey ca be solved closed fom, ad he esls ae as oe mgh expec: he MLE fo µ s he aveage of he obsevaos, ad he MLE fo σ s he sqae oo of he sample vaace. I geeal, howeve, oe cao fd a closed-fom solo fo he maxmm lelhood esmao(s), eqg a eave algohm. (Fo example, MLE fo Webll & Gmbel dsbos.) Webll Dsbo: pdf: f () exp Sppose, 2, ae mes o fale of a gop of mechasms. The lelhood fco s L( ;, ) exp exp We wsh o choose vales of & whch maxmze L (o eqvalely, he logahm of L),.e., whch mae he obseved vales of as lage as possble! MLE page 9 D.L.Bce, 2002 MLE page 0 D.L.Bce, 2002 The log-lelhood fco s l L ;, l l + l ad so he opmaly codos ae l L ( ;, ) 0 l L ( ;, ) 0 Ths gves s a pa of olea eqaos wo ows: ˆ + ˆˆ 0 l ˆ + l 0 B he lef sde of he fs eqao ca be facoed: ˆˆ + 0 Sce he fs faco cao be zeo, we se he secod faco eqal o zeo ad solve fo û ems of ˆ : Elmag û he secod eqao by sbsg he fs, we ge he followg olea eqao ˆ aloe: l ˆ + l 0 Ths ca ow be solved by, fo example, he seca mehod. MLE page D.L.Bce, 2002 MLE page 2 D.L.Bce, 2002
Maxmm Lelhood Esmao wh cesoed daa Sppose ha a expeme was emaed a me τ afe oly of he s a lfees had faled. Ths s accoed fo by defg he lelhood as sce F ( τ; (, θ ) ( τ; θ ) ( ; L F f s he pobably ha he s svve l me τ. The log-lelhood fco s heefoe ( θ ) ( ) ( θ ) + ( l L; l F; l f ; Example: MLE of Webll paamees, gve cesoed daa The CDF of he Webll dsbo s F( ;, ) exp ad so he lelhood fco s τ L ;, exp exp exp ( ) + τ MLE page 3 D.L.Bce, 2002 MLE page 4 D.L.Bce, 2002 The log-lelhood fco s l L ;, l l + l + τ Aga, he ecessay codos fo a maxmm of he log-lelhood a ( ˆ, ˆ ) ae ( ˆ ˆ) ( ˆ ˆ) l L ;, 0 l L ;, 0 A esl smla o he cesoed case ca be deved: ad ˆ ˆ + τ ˆ l l + τ τ + τ + l 0 Ths secod eqao ca be solved fo ˆ by he seca mehod, ad he ˆ sed o calclae û by he fs eqao. MLE page 5 D.L.Bce, 2002 MLE page 6 D.L.Bce, 2002
EXAMPLE: Twey devces ae esed smlaeosly l 500 days have passed, a whch me he followg fale mes ( days) have bee ecoded: 3.5 74.0 87.5 00. 03.3 8.9 279.9 297. 462.5 465.4 A plo of Y vs X, obaed by he asfomaos: Y log log whee R() s he obseved faco of he devces whch have R () svved l me, ad X log shold be a le f he Webll model wee o f he daa pefecly. Esmae he lfeme fo whch he devce s 90% elable. MLE page 7 D.L.Bce, 2002 MLE page 8 D.L.Bce, 2002 LEAST SQUARES REGRESSION RESULTS: (scale paamee) 653.504 (shape paamee) 0.90833 so ha mea 630.396 sadad devao 754.336 Noe: hs s deemed by mmzg he sm of he sqaed eos he ( ) leazed veso of F () e, amely y x l, whee x l & y l l, ahe ha he ogal eqao! R () If we se hese paamees fod by lea egesso, he elably fco wold have he vales: F() -F() 4.2824 0.0 0.99 8.90435 0.02 0.98 3.993 0.03 0.97 9.363 0.04 0.96 24.837 0.05 0.95 30.5337 0.06 0.94 36.3925 0.07 0.93 42.4042 0.08 0.92 48.5623 0.09 0.9 54.8622 0.0 0.90 Hece, accodg o hs model, 90% of he devces shold be opeag a 54.8 (appoxmaely 55) days. MLE page 9 D.L.Bce, 2002 MLE page 20 D.L.Bce, 2002
SECANT METHOD Maxmm Lelhood esl: Solvg he olea eqao fo : l l + τ τ + τ g + l 0 If o fs wo gesses a he vale of ae 0.5 ad 2.0, he we deeme ha g(0.5).3739 & ad g(2.0) 0.68085. The seca jog he wo pos o he gaph of g coss he axs a.4787., We he epea, wh he 2 mpoved gesses 0.5 ad.4787. MLE page 2 D.L.Bce, 2002 MLE page 22 D.L.Bce, 2002 SECANT METHOD RESULTS: eo 0.5.3739 2.0 0.68085.4787 0.397478 0.5203.0548.2083 0.27582.09244 0.08608 0.974445 0.025006 0.996528 0.00227829 0.994684 0.0000438242 0.994648 7.83302E 8 0.994648 2.68896E 2 Oce we deeme he vale of ˆ whch maxmzes he lelhood fco, he he coespodg vale of he paamee û s fod by ˆ ˆ + τ ˆ MAXIMUM LIKELIHOOD RESULT: (scale paamee) 70.339, (shape paamee) 0.994648 F() -F() 6.9646 0.0 0.99 4.0526 0.02 0.98 2.2337 0.03 0.97 28.5026 0.04 0.96 35.8579 0.05 0.95 43.2993 0.06 0.94 50.8272 0.07 0.93 58.4427 0.08 0.92 66.467 0.09 0.9 73.9408 0.0 0.90 Accodg o hs model, he, 90% of he devces shold be opeag a 73.94 (appoxmaely 74) days. MLE page 23 D.L.Bce, 2002 MLE page 24 D.L.Bce, 2002