PROPERTIES OF GOOD ESTIMATORS

Transcription

1 ESTIMATION INTRODUCTION Estmato s the statstcal process of fdg a appromate value for a populato parameter. A populato parameter s a characterstc of the dstrbuto of a populato such as the populato mea, the populato varace or populato proporto. Whe o formato s avalable o the parameter uder vestgato, a sample has to be selected from the populato order to obta some dea of the value of ths parameter. Obvously, we are assumg that a cesus would be ot oly mpractcal but also mpossble, gve that the populato sze s of fte magtude. The samplg method to be adopted depeds o the structure of the populato ad the sample s to be chose so as to be as ubased as possble. It should cota all, f ot most of, the characterstcs of ts paret populato. It has to be metoed that t s very hard to select the perfect sample, as t s mpossble to elmate samplg errors completely. It s therefore evdet that a sample statstc wll always devate from ts correspodg parameter. A sample statstc s ay fucto of observed data, especally used to estmate a parameter for eample, the sample mea ad the sample varace. There are two ways of estmatg a populato parameter: pot ad terval estmato. PROPERTIES OF GOOD ESTIMATORS A pot estmator s a sgle-valued sample statstc whch s used to appromate a populato parameter. A questo of terest s: whch statstc should oe use to estmate a parameter? For eample, suppose we wat to estmate the populato mea μ. Should we use the sample mea, the meda or the mode? The soluto s to pck the statstc that teds to produce a estmate closest to the true value. Ths ca be epected to occur f the estmator possesses four propertes whch we wll dscuss terms of populato ad sample meas.. Ubasedess A estmator s sad to be ubased for the parameter estmated f t s cetered at the rght spot. Mathematcally, the average value of the estmator should be equal to the parameter that t s estmatg. I mathematcal otato, f the sample statstc T s a ubased estmator of the populato parameter θ, the E[T] = θ. May estmators are asymptotcally ubased the sese that the bases reduce to practcally sgfcat values (close to zero) whe becomes suffcetly large. The estmator s, the sample varace, s such a eample, as wll be see later.

2 . Cosstecy If a estmator approaches the parameter t s estmatg as the sample sze creases, t s the sad to be cosstet. Stated more rgorously, a estmator s sad to be cosstet f, as approaches fty, the probablty that t wll dffer from the parameter s ot more tha a arbtrary small costat. For stace, the sample mea s a ubased estmator of μ, o matter what form the populato dstrbuto assumes, whle the sample meda s ubased oly f the dstrbuto s symmetrcal. I case of large samples, cosstecy s a desrable property for a estmator to possess. However, small samples, cosstecy s of lttle mportace uless the lmt of the probablty defg cosstecy s reached eve wth a relatvely small sze of the sample..3 Effcecy The cocept of effcecy refers to the samplg varablty of a estmator. If two competg estmators are both ubased, the oe wth smaller varace (for a gve sample sze) s sad to be relatvely more effcet. Stated a somewhat dfferet laguage, estmator T s sad to be more effcet estmator U f the varace of the frst s less tha that of the secod. The smaller the varace of the estmator, the more cocetrated s the dstrbuto of the estmator aroud the parameter beg estmated ad, therefore, the better ths estmator s. For eample, f the populato s symmetrcally dstrbuted, the both the sample mea ad the sample meda are cosstet ad ubased estmators of μ. Yet the sample mea s better tha the sample meda as a estmator of μ sce t s more effcet..4 Suffcecy A estmator s sad to be suffcet f t coveys as much formato as s possble about the parameter whch s cotaed the sample. The sgfcace of suffcecy les the fact that, f a suffcet estmator ests, t s absolutely uecessary to cosder ay other estmator; a suffcet estmator esures that all formato a sample ca fursh wth respect to the estmato of a parameter s beg utlsed. May methods have bee devsed for estmatg parameters that may provde estmators satsfyg these propertes. The two most mportat methods are the least-squares (OLS) ad the mamum lkelhood (MLE).

3 3 POINT ESTIMATION I ths course, we shall fd the pot estmators for the populato mea μ, varace σ ad proporto p. The dervato of these estmators wll requre a basc kowledge of the propertes of epectato ad varace (read Sectos 4.. ad 4.. below); the reader mght eve fd these dervatos qute mathematcally trcate sometmes. 3. Propertes of epectato The epectato of a radom varable s just ts arthmetc mea or average. The epectato of X s deoted by E[X ] ad defed by E [ X ] = P [ X = ] Gve a costat c ad radom varables X ad Y, the. E [ c] = c. E [ cx ] = ce[ X ] 3. E [ X ± Y ] = E[ X ] ± E[ Y ] Note. E [ XY ] E[ X ] E[ Y ] ecept f X ad Y are depedet.. X E Y E[ X ] E[ Y ] 3. Propertes of varace The varace of a radom varable s a measure of ts spread or dsperso. The varace of X s deoted by var[x ] ad defed as var[ X ] = E[ X ( E[ X ) ] ] Gve a costat c ad radom varables X ad Y, the. var[ c] = 0. var[ cx ] = c var[ X ] 3. var[ X ± Y ] = E = var[ X ] + var[ Y ] oly f X ad Y are depedet. Note. var[ XY ] var[ X ] var[ Y ]. X var Y var[ X ] var[ Y ] 3

4 3.3 Assumptos Durg the comg dervatos, the followg assumptos wll be made:. The populato varable s X where E [X ] = μ ad var[ X ] = σ.. Each observato from the set {.,, } s depedetly ad detcally dstrbuted (..d), that s, for ay observato, E [ X ] = μ ad var[ X ] = σ just lke for the populato varable. Note that these assumptos are etremely mportat ad wll have to be remembered durg the dervatos. 3.4 Estmato of the populato mea μ If we wsh to have a dea of the value of the populato mea μ, t s atural to select a sample ad calculate the sample mea. These values should ot be dfferg by much f the sample s ubased. Ca we coclude that the populato mea s very close to? Ca we use as a substtuto for μ wheever the latter s ukow? Frst, we must show that the sample mea s a ubased estmator for the populato mea by provg that E[ X ] = μ. From defto, E =. We therefore fd ts epectato as follows: X = E[ X ]= E[ X + X X = { E[ X ] + E[ X ] E[ X ]} = { μ + μ μ} = ( μ) = μ. We ca thus coclude that s a ubased pot estmator of μ. I pla ad smple Eglsh, t meas that, wheever the populato mea μ s ukow, we may select a sample (as ubased as possble), calculate ts sample mea ad cosder t as a worthy replacemet for μ. ] 4

5 3.5 Estmato of the populato varace σ The sample varace s seems to be a deal caddate for beg the pot estmator of the populato varace σ. It remas to be checked whether s s ubased for σ. From the chapter o Descrptve Statstcs, we kow that s = The epectato of E s s derved as follows: = X X [ s ] E X = E E[ X ] The rght-had sde of the above wll be splt order to fd the epectato of each term. Let us look at the frst term: [ X ] = E[ X + X +... ] X E = E + = { E[ X ] + E[ X ] E[ X ]} Usg the defto of varace ( terms of epectato, Secto 4.3.), we have var X = E X E X so that [ ] [ ] ( [ ] ) [ X ] var[ X ] ( E[ X ] ) E = + for ay. Thus, [ ] [ ] [ ] [ ] E X X = E X = E X 3 =... = E X = σ + μ X E = E X = ( σ + μ ) = σ + μ Equato (I) ad [ ] The secod term ca be smplfed smlarly, that s, usg the defto of varace: E [ X ] = var[ X ] + ( E[ X ]) = var[ X ] + μ 5

6 The term var [ X ] ca further be smplfed as follows: X var = 3 [ X ] var = var[ X + X + X... ] + X = { var[ X ] + var[ X ] + var[ X ] +... var[ ]} 3 + X sce t was already assumed that the observatos are..d. Thus, var[ X ] ( ) σ = σ =. σ E Equato (II) so that [ X ] = var[ X ] + μ = + μ Combg Equatos (I) ad (II), we have E σ σ [ ] ( σ μ ) μ σ s = + + = = σ It s clear that s s a based estmator of σ sce ts epectato s ot equal to σ. However, t s ot very far from beg ubased gve that ( ) ad would appromately equal f were very large ths s what asymptotc ubasedess s all about! More terestg would the be to determe the statstc whch s the ubased estmator of σ. Usg oe of the laws of epectato (Secto 4.3.), we have, startg from E [ s ] = σ that, multplyg both sdes by, E [ s ] = σ, mplyg that s E = σ. Hece, s s a ubased estmator of σ. 6

7 3.6 Estmato of the populato proporto p We ofte wat to kow the proporto of dvduals a populato whch satsfes a certa characterstc. For eample, t would be terestg to kow the percetage of left-haded people Maurtus or the proporto of books a lbrary whch cota more tha 500 pages. As usual, t wll be assumed that the populato s fte so that formato may oly be obtaed by selectg a sample. The populato proporto s deoted by p. I geeral, whe we select dvduals, they ether satsfy or do ot satsfy the characterstc uder vestgato. If t ever happes that a dvdual falls both categores smultaeously (for eample, someoe ambdetrous), the that dvdual s automatcally dscarded for the sake of calculatos. It s thus qute atural to use the bomal dstrbuto because each dvdual wll ether be labelled as success or falure, depedg o whether t satsfes the characterstc or ot. If we wat to have a dea of the value of p, we select a sample of sze ad cout the umber,, of dvduals satsfyg the requred characterstc. It s obvous that a atural pot estmate for p, usually deoted by p s, would be. For the reader who s ufamlar wth the bomal dstrbuto (dscrete), t s suffcet to kow that f X s a bomal varable wth parameters ad p, the E [ X ] = p ad var[ X ] = p( p). These results wll certaly help the followg dervato. The epectato of the sample proporto s obtaed as follows: X E = E = [ X ] = ( p) p Thus, the sample proporto s a ubased estmator of the populato proporto. We summarse our fdgs the followg table. Parameter Sample statstc Ubased estmator Mea μ = μˆ = Varace σ s ˆ σ = Proporto p p s = p ˆ = Fg

8 4 INTERVAL ESTIMATION Ths aspect of estmato s a attempt to fd the lower ad upper boudares of a terval that may cota the parameter uder vestgato. The legth of ths terval depeds o the cofdece level as specfed the problem. The cofdece level s the degree of certaty wth whch we ca say that the terval wll cota the parameter. It s gve the 00( α ) % form, where α s kow as the sgfcace level or the marg of error. More formally, a 95% cofdece terval would be defed as oe that has a probablty of 0.95 of cotag the parameter. It has to be metoed here that 0.95 s ot the probablty that the parameter les the terval. There s a subtle dfferece betwee these two statemets the sese that t s ot the parameter whch vares but the boudares of the terval. Let us frst become famlar wth the above theory ad otatos by meas of a eample. Eample Image that we wsh to fd a terval estmate for the populato mea weght of people who are 45 years old a gve populato. The frst step would be to select a sample of reasoable sze, as ubased as possble, ad fd the pot estmate of the populato mea, that s, the sample mea. Ths pot estmate wll be a gudele to the costructo of the terval, whch also requres the cofdece level. If the sample mea s, say, 57. kg, the the populato mea should ot be very far from ths fgure, takg to cosderato the fact that the sample s as ubased as possble. To mamse the probablty of beg correct our terval estmato, t s logcal to place the sample mea the mddle of the terval. The obvous reaso s that, f there dd est some samplg error durg the samplg process, the ay amout of devato from the sample mea wll stll yeld a true fgure for the populato mea whch s cotaed the terval (thk about t very carefully). Furthermore, t s clear that the greater the cofdece level, the larger wll be the terval sce the marg of error has to be mmsed. The logcal argumet gve above ca also be statstcally eplaed sce we always select relatvely large samples order to obta mamum formato o the populato parameter, we ca make use of a etremely powerful theorem to support our argumet. 8

9 4. The Cetral Lmt Theorem If.,, are observatos of a radom sample of sze from ay dstrbuto wth mea μ ad varace, the, for large, the dstrbuto of the sample mea =. σ X s appromately ormal such that σ X ~ N μ, where 4. Costructo of a terval We ca thus make use of the ormal dstrbuto theory to show that the probablty of beg correct our estmato s mamsed wheever the terval s symmetrc about the pot estmate of the parameter. The followg dagrams show two tervals of the same legth but placed at dfferet locatos o the -as of a ormal curve. It s obvous that mamum probablty s acheved whe the terval s placed the cetre of the dstrbuto (LB ad UB stad for the lower boudary ad upper boudary of each terval respectvely). LB UB Fg. 4.. LB UB Fg

10 It s clear that the shaded area Fg. 4.. s larger tha that of Fg. 4.., hece justfyg our choce of ceterg the terval o the pot estmator. Fg below s a overall vew of a cofdece terval. Cofdece level α α LB E E UB Cofdece terval Fg The quatty E s just half of the legth of the cofdece terval. To obta the respectve values of the lower ad upper boudares, t suffces to evaluate E ad, tur, subtract t from, ad add t to, the value of the pot estmate (sce the boudares are equdstat to the cetre of the dstrbuto). The procedure for terval calculato s as follows:. Gve the cofdece level, we subtract t from ad dvde by two to obta the half of the sgfcace level.. Use ths ew value to get ts correspodg z-value from the stadard ormal table ths s the umber of stadard devatos betwee ay boudary ad the cetre. 3. Calculate the value of oe stadard devato of the estmator. 4. Multply the stadard devato by the z-value order to obta E. 5. The cofdece terval wll thus be (Pot Estmate E, Pot Estmate + E) Ths procedure wll be used for calculatg cofdece tervals for both populato meas ad proportos. The major dffereces wll just be the pot estmates ad ther stadard devatos. 0

11 4.3 Estmato of the populato mea μ Whe fdg a terval estmate for the populato mea μ, we should frst select a sample ad determe the value of the sample mea or pot estmate. From the Cetral Lmt Theorem, the stadard devato for s σ. However, f the populato stadard devato σ s ukow, we have to replace t by σˆ, ts ubased estmate. We the follow the procedure as gve the prevous secto. Ths s llustrated by the eample below. Eample A radom sample of 50 adult me udergog a route medcal specto had ther heght ( cm) measured to the earest cetmetre ad the followg data were obtaed: = 4305, = Calculate ubased estmates for the populato mea ad varace ad hece a 99% symmetrc cofdece terval for the populato mea. Soluto We use the followg formato: = 50, = 4305, = The ubased estmate for the populato mea s 4305 = = = The ubased estmate for the populato varace s s = = (7.8) = Sce the cofdece level s 99%, half the sgfcace level s 0.005, whch gves us a z-value of.576 from the stadard ormal table. z ˆ σ (.576)( 9.744) Thus, E = = = 0. 5 ( decmal places, that s, the 50 same degree of accuracy of the sample mea). A 99% cofdece terval for the populato mea s therefore < μ < , 7.3 < μ <

12 4.4 Estmato of the populato proporto p As has bee prove Secto 3.6, the ubased estmator of the populato proporto p s the sample proporto. Gve that the bomal dstrbuto s used to determe the ubasedess of the sample proporto, we use the same dstrbuto to fd ts varace (or stadard devato). We kow that f X s a bomal varable, var[ X ] = p( p). Thus, the varace of would be X p( p) p( p var var[ X ] ) = = =. But the aga, we ask ourselves the questo: How ca we use the value of p the formula for the varace of f we are precsely lookg for a cofdece terval for p? The aswer s smple wheever p s ukow, we replace t by ts ubased estmator! Hece, the varace of pˆ, that s,, wll be p ˆ( pˆ ) p ˆ( pˆ ) or ts stadard devato s. Let us ow calculate the cofdece terval for a populato proporto by meas of a eample. Eample A survey was carred out to vestgate the proporto of people who are left-haded a populato. To that effect, a sample of 000 people revealed that oly 5 of them were left-haded. Calculate 95% cofdece lmts for the populato proporto of left-haded people. Soluto We use the followg formato: = 000, = 5. 5 The sample proporto, p ˆ, s thus = Sce the cofdece level s 95%, half the sgfcace level s 0.05, whch gves us a z-value of.96 from the stadard ormal table. pˆ( pˆ) (0.5)(0.885) Thus, E = z = (.96) = (3 decmal 000 places, that s, the same degree of accuracy of the sample proporto). A 99% cofdece terval for the populato mea s therefore < p < , < p < 0.35.