Stat 400, sectio 6. Methods of Poit Estimatio otes by Tim Pilachoski A oit estimate of a arameter is a sigle umber that ca be regarded as a sesible value for The selected statistic is called the oit estimator of. The symbol ˆ is used for both the radom variable ad the calculated value of the oit estimate. Ideally, the oit estimator ˆ is ubiased, i.e. E ( ˆ ). I ords, the samlig distributio based o the statistic has a exected value equal to the actual (but uko) oulatio arameter. he e have a choice betee oit estimators hich are all ubiased, ho do e ick the oe e should use? e already have oe criterio. he cosiderig the oulatio arameter mea µ, both radom variable X ad radom variable X are ubiased. Hoever, sice the variace of X V ( X ) [ σ ] is larger tha σ X the variace of X V ( X ), the samle statistic x has a loer robability of beig reresetative of the oulatio tha the samle statistic x. More imortatly, ho do e fid a ossibility i the first lace? Sectio 6. formalizes this choice rocess by lookig at to methods of ickig ad estimator: a) the method of momets ad b) the method of maximum likelihood. a) the method of momets Defiitio. Give a radom variables X, X, X ad ositive iteger k the k th oulatio momet of X is k ( X ) E ( X ), k mk. X E X σ µ Thus, ( X ) E ( X ) µ. [For the derivatio of the secod assertio, see Lecture 6.c.] Defiitio. Give radom samle values x, x, x, from the samle sace of a radom variable X ad oegative iteger k the k th samle momet S k is Thus, m ad m S x. k Sk xi, k. i It is imortat to ote that the the k th oulatio momets of X are fuctios of arameter hile the k th samle momet S k is ot. Rather S k is a average sum of oers of the samle values x i. Here s ho to imlemet the method of momets: a) Set the k th oulatio momet of X, m k ( X ), equal to the k th samle momet S k, the b) solve the resultig equatio for. That is, solve m X S, k. k k Lecture 6.c Examle F revisited. You toss a coi times. Defie a radom variable 0 if a toss is tails, ad if a toss is heads. Use the method of momets to determie a estimator ˆ for the oulatio arameter roortio of successes. [Although this sceario is described i terms of fliig a coi, the mathematics ould be the same for ay Beroulli/biomial distributio.] For each toss, e have a biomial robability desity fuctio. 0 P( ) X
Lecture 6.c Examle F. (cotiued) Examle A (see Lecture 6.b). Use the method of momets to determie estimators for arameters mea, µ, ad variace, σ, for radom variable X for hich e ko a robability distributio. s because (as demostrated i A better estimator ould be samle variace xi ( x i ) Lecture 6.b) it is ubiased. Dr. Millso develos a momet estimator for a uiform distributio (Lecture 3. 9-5). The text covers exoetial, gamma ad egative biomial distributios (Examles 6.-6.4).
b) the method of maximum likelihood. Radom variables X, X,, X form a (simle) radom samle of size if they meet to (imortat) requiremets:. The X i s are ideedet radom variables.. Every X i has the same robability distributio. Give radom samle values x, x, x, from the samle sace of a discrete radom variable X ith a robability mass fuctio X ( x; ) [the mf is a fuctio of uko arameter ], hat is P P(X x, X x, X x )? I other ords, hat is the robability of gettig (by radom chace) the samle e actually got? Sice the X i s are ideedet, P P X x, X x X x P X x P X x P X x., Sice every X i has the same robability distributio, P P X x X x, X x x ; x ; x ;., X X X Because the samle values x, x, x are umbers, P is a fuctio oly of arameter, the likelihood fuctio, hich e ill desigate L(). e at to fid the value of that maximizes L(). The maximum likelihood estimator ˆ ill be a critical value, such that L ( ˆ ) 0. short side tri: logarithmic differetiatio Let h( ) l[ L ( )] he robability equals 0.) The by the chai rule, d h( ) dx. (Domai is ot a roblem because 0 ( ; ) X x i ( l[ L( )]), ad e are t iterested i occasios h ( ) L ( ) L( ) Here s the imortat thig: h ad L share the same critical values. Also, sice a b l ( a) < l( b) reserved ad both h ad L ill have a maximum at the same critical value! d dx To use the method of maximum likelihood, Let h( ) l[ L( )]. Fid h ( ). h ad solve for i terms of x, x,, x. Set 0 <, order is Lecture 6.c Examle F revisited. You toss a coi times. Defie a radom variable Y roortio of heads. Use the method of maximum likelihood to determie a estimator ˆ for the oulatio arameter. [Although this sceario is described i terms of fliig a coi, the mathematics ould be the same for ay Beroulli/biomial distributio.] e re lookig for a estimator of the oulatio arameter (oulatio roortio, or robability of success). umber of heads Recall from Lecture 6.c, e have Y umber of tosses That is, for each toss, e have a biomial robability desity fuctio. 0 P( ) X ( ; ) ( ) 0, 0 otherise.
First ste: rite out ad simlify the likelihood fuctio usig our geeric arameter. otes o the roof: ; ; ; L Secod ste: Create h() ad simlify. otes o the roof: L h l l l l l l Fial ste: Differetiate h(), set equal to 0, ad solve for. otes o the roof: h 0 ˆ heads umber of ˆ
Examle B. A radom variable X has a exoetial robability distributio. Use the method of maximum likelihood to determie a estimator for arameter λ. Examle B exteded. A radom variable X has a exoetial robability distributio. Determie a estimator for arameter λ. e ll eed the Ivariace Pricile: Let ˆ ˆ, ˆ, be the maximum likelihood estimators for arameters,,. The the maximum likelihood estimator of ay fuctio h(,, ) of these arameters is the fuctio h ( ˆ ˆ ˆ,, ) of the maximum likelihood estimators.