Parameter Estimatio Samples from a probability distributio F () are: [,,..., ] T.Theprobabilitydistributio has a parameter vector [,,..., m ] T. Estimator: Statistic used to estimate ukow. Estimate: Observed value of the estimator. Maimum Likelihood Estimator The likelihood for idepedet samples is defied as L(; ) The maimum likelihood estimator is defied as Y f( i ; ) ˆ ML argmal(; ) To estimate the value of that maimizes L or equivaletly l L we will set Eample. For Beroulli distributio, @ l L @ i 0, for i,,...,m P (X ) p ( p) Hece, amog observatios, the likelihood is defied as L(; p) Y p i ( p) i p P i ( p) P i p ( ) ( p) l L l p + ( )l( p)
Takig derivative with respect to the parameter p d l L ( ) 0 dp p p ( p) ( )p 0 Hece, the ML estimator is ˆp Eample. ) ˆp P i For Poisso distributio P (X )! e Hece, amog observatios, the likelihood is defied as L(; p) Y i i! ep( ) P i Q i! ep( ) l L l Takig derivative with respect to the parameter X l( i!) Hece, the ML estimator is ˆ. Eample 3. For Gaussia distributio d l L 0 d P ) ˆ i f() p ep apple ( µ) Hece, amog observatios, the likelihood is defied as L(; µ, ) Y p ep apple (i µ) " ep ( ) / # X ( i µ) Subhaya De
l L l( ) l( ) X ( i µ) Takig derivative with respect to the parameter µ @ l L X @µ ( i µ) 0 P ) ˆµ i Hece, the ML estimator is ˆµ. Takig derivative with respect to the parameter @ l L @( ) X + ) ˆ Hece, the ML estimator is ˆ P i0 ( i ˆµ). Eample 4. ( i µ) 4 0 X ( i ˆµ) i0 For Gamma distributio f() ( ) e Hece, amog observatios, the likelihood is defied as L(;, ) Y i ( ) ( ) Y e i! e P i X l L ( ) l i X i +( )l l ( ) Takig derivative with respect to the parameter @ l L @ X i + 0 ) ˆ ˆ P i Subhaya De
Hece, the ML estimator is ˆ. ˆ P i Takig derivative with respect to the parameter ) l ˆ @ l L X @ l i + l 0 (ˆ ) (ˆ ) l! X i This is a oliear equatio eeded to be solved to get ˆ. Eample 5. 0 ( ) ( ) X l i If the observatios {0.3, 0., 0.5, 0.8, 0.9} are obtaied from a distributio with f(), 0theestimatethevalueof usig Maimul Likelihood method. The likelihood is defied as The log likelihood is L(; ) 5Y l L 5l +( ) Takig derivative of l L with respect to i 5X l i 0 @ l L @ 5 5X + l i 0 ) ˆ 5 P 5 l i.3038 Eample 6. For Uiform distributio i (0,) f(), 0 << Hece, amog observatios, the likelihood is defied as L(; ) l L Y l This is maimized whe is miimum but ma(,,..., ). Hece, the ML estimator is ˆ ma(,,..., ). Subhaya De
Iterval Estimate Let X,X,...,X are samples from a Gaussia distributio with mea µ ad variace. The poit estimator X is Gaussia with mea µ ad variace /. Hece, P.96 < X µ / p <.96 0.95 P X.96p <µ< X +.96p 0.95 Based o the observatios, with 95% we ca say that the populatio mea µ lies withi the iterval.96 p, +.96 p kow as the 95 percet cofidece iterval estimate of µ. I geeral, 00( )percettwo-sidedcofideceitervalfor µ is z / p, + z / p. Oe-sided upper cofidece iterval for µ is z p, +. Oe-sided lower cofidece iterval for µ is, + z p. Sample size: If we wat the 00( ) percettwo-sidedcofideceitervalforµ to be withi ( ± ) weeedasamplesize (z / / ). Quick referece: 00( )% two-sided cofidece iterval: 90% cofidece: 0,z /.65 95% cofidece: 5,z /.96 98% cofidece:,z /.33 99% cofidece:,z /.58 Similarly, the followig Table shows a variety of cases for samples from a ormal populatio: Note that, s P ( i ). Table 6.3: Di eret cases. Case Parameter Cofidece iterval Lower iterval Upper iterval kow µ ± z / p, + z p z p, + ukow µ s ± t /, p s, + t, p s t, p, + ( )s µ ukow ( )s, ( )s 0, ( )s, + /, /, /, /, Eample 6. Estimate the sample size eeded for mea to be withi ±0.5 where iterval of 95%. ad a cofidece Subhaya De
z /.96 0.5 984 Eample 7. The lifetime X of light bulbs are epoetially distributed. Based o observatio of 8 light bulbs we obtai their average lifetime is 00 hours. Estimate the 95% cofidece iterval for the mea lifetime. For epoetially distributed radom variable X, f() e The mea of X is / ad variace is /. For large umber of samples, thesamplemea is Gaussia with mea / ad variace. Hece, we ca write P.96 < X! <.96 0.95 p P.96 p < X < +.96 p 0.95.96 P p < X < +.96 p 0.95 X P +.96/ p < X <.96/ p 0.95 Hece, the 95% cofidece iterval for the mea lifetime of the bulbs is 00 +.96/ p 8 < < 00.96/ p 8 or 64 < < 56. Eample 8. For Poisso distributed radom variable get the 00( The p.m.f. is give by The mea E[X] i ) cofidece iterval. P (X i) e i! Var(X). Hece, for large X is approimately Gaussia with mea Subhaya De
ad variace /. Thishelpsiwritig r! P X.96 r < < X +.96 0.95 r! P X <.96 P X 0.95 (.96) < 0.95 Therefore, the cofidece iterval is the two solutios of the followig quadratic equatio ( ) (.96) Subhaya De