2.1.1 The Art of Estimation Examples of Estimators Properties of Estimators Deriving Estimators Interval Estimators

Transcription

1 . ploatoy Statstcs. Itoducto to stmato.. The At of stmato.. amples of stmatos..3 Popetes of stmatos..4 Devg stmatos..5 Iteval stmatos

2 . Itoducto to stmato Samplg - The samplg eecse ca be epeseted by a collecto of depedet ad detcally dstbuted adom vaables say { }. - Whe the sample s tae we ed up wth a set of ealzato { } Samplg Assumptos d-assumpto - The obsevatos ae ealzatos of depedet adom vaables - The adom vaables ae detcally dstbuted Note The dea behd detcally dstbuted adom vaables s that the sample could be tae aga esultg aothe set of ealzato say of. The depedece assumpto caot be made whe mag feece about tme sees o stochastc pocesses. Oe uses stead the assumpto of statoay ad egodcty.

3 . Itoducto to stmato The At of stmato The ey of a statstcal aalyss s to mae statstcal feece whch s the pocess of etactg fomato fom the data about the pocess udelyg the obsevatos. The success of statstcal feece depeds o the estmatos chose A estmato α s a fucto o the space of obsevatos α s a adom vaable A patcula value of α obtaed fom a gve sample of the adom vaable s a ealzato The at s to fd good estmatos that yeld estmates a specfed eghbohood of the tue value wth some ow lelhood - a estmato s subjected to samplg vaablty. It caot be ght o wog but some ae bette othe the othes - amples of bad estmatos of mea ad vaace ae s s They ae bad sce they vay stogly fom sample to sample /

4 . Itoducto to stmato amples of estmatos: Fequecy hstogams ad empcal dstbuto fucto estmatos of the tue pobablty desty fucto ad c.d.f. of They ae obtaed by pattog the eal le K Θ R ad Θ I Θ l 0 fo l R to K subsets such that The umbe of obsevatos fallg to each obsevatos s H Θ { : Θ } Θ dvded by the total umbe of Fo Θ [ ] H Θ s a estmato of F. Sce H Θ f s a estmato of H Θ d Θ Θ P Θ f d Θ s a estmato of f Wth sutable egulaty codtos coveges to the tue f desty fucto as the sample sze whe the umbe of elemets each Θ ad the umbe of subsets teds to fty as Θ

5 . Itoducto to stmato amples of estmatos: estmato of the mea The sample mea epessed as a adom vaable o epessed as a ealzato obtaed fom a patcula sample s a estmato of the tue mea sce Va / -A mpotat fomula statstcs! - assume the depedece! - The sample mea has a dstbuto whch appoaches a omal dstbuto as the sample sze ceases CT - The spead of the dstbuto of the estmato.e. the ucetaty of the estmato deceases wth ceasg sample sze The spead of the dstbuto of the sample mea as a fucto of the sample sze

6 . Itoducto to stmato amples of estmatos: stmatos of the vaace covaaces ad coelatos Thee ae two estmatos fo vaace two fo covaace mat two fo coelato whee S o T j j T j j C o Σ jj j j jj j j S S S ρ ρ o th compoet of the ad th adom vecto beg the wth ad ; ; ; ; j j S j ;j j j j j j j

7 . Itoducto to stmato Popetes of stmatos A good estmato wll poduce estmates α the eghbohood of the tue paamete α. Ths s measued by the mea squaed eo M ; α α α α et α ad ~ α be two competg estmatos of a paamete α. The s sad to be moe effcet tha ~ α α f M ; α α < M ~ α ; α -a estmato havg small mea squaed eo s desable - the mea squaed eo ca be wtte as the sum of the mea squaed bas ad the vaace of the estmato - statstcas ofte seach fo ubased estmatos wth mmum vaace

8 . Itoducto to stmato Bas a measue of accuacy et α be a paamete of a statstcal model ad let be a estmato of ths paamete. The the bas of estmato α s ts epected o mea eo gve by B α α α α - postve bas dcates that the estmato oveestmates α. Smlaly egatve bas dcates a udeestmato - a estmato that has o bas s sad to be ubased - postve bas does ot mply that all ealzatos of the estmato ae geate tha α although that could be tue f the bas s lage compaed to the vaablty of the estmato - to assess the lelhood fo obtag a patcula value of the estmato oe eeds to ow the dstbuto of the estmato - t s hghly desable to have estmatos wth lttle o o bas. It may sometmes ecessay to balace small bas agast othe desable popetes

9 . Itoducto to stmato Bas of some estmatos -The empcal dstbuto fucto s a ubased estmato of the c.d.f - The sample mea s a ubased estmato of -The sample vaace S s a ubased estmato of whle s a based estmato of : 0 F B 0 B B S B / 0 4. dstbuto the bomal has valued adom vaable - tege depedece the de the assumpto of 3. of dstbutos detcal de the assumpto of. sample wth the adom vaables umbe of. 0 Pood of y F y F y F B y F y F y P y F F B < S S Va Sce / 0 of Poof

10 . Itoducto to stmato Bas of Some stmatos Smla to the uvaate case oe obtas fo the sample mea vecto ad the sample covaace mat B 0 B C 0 B Σ Σ

11 . Itoducto to stmato Asymptotcally ubased estmatos A estmato wth the popety lm B α 0 s sad to be asymptotcally ubased. May based estmatos e.g. ae asymptotcally ubased

12 The vaace of the empcal dstbuto fucto at The vaace of the sample mea The vaace of the two dffeetly defed sample vaaces. Itoducto to stmato Vaace of some estmatos a measue of ucetaty Assume that the sample cossts of d adom vaables F F F Va / Va 4 4 * 4 * 3 4 * 4 * adom vaables d omal the sample cossts of If momet the fouth cetal 3 γ γ γ γ + S Va Va Va S Va Va + j Va

13 . Itoducto to stmato Both bas ad vaace cotbute to the epected mea squaed eo [ B α ] Va α M ; α α + Poof : M ; α α α α α α α α α - α + α α α α α α Cosstecy A estmato s cosstet f ts mea squaed eo goes to zeo wth ceasg sample sze.e. f lm M ; α α 0 Ay asymptotcally ubased estmato wth vaace that s asymptotcally zeo s cosstet

14 . Itoducto to stmato Bas Coecto We showed that s a based estmato wth B / ad S coects ths bas by multplyg the estmato by /-. May bas coectos ae of ths fom by scalg α a based estmato of α by a costat c so that the esultg estmato ~ α α / c s ubased. Howeve oe should be caeful about ths type of mpovemet sce the mpoved estmato ~ α may ot always be moe effcet tha the ogal oe α If c> ~ α s moe effcet tha α due to the educto of both the bas ad vaace If c< the bas s educed but the vaace s ehaced The scalg facto that tus based to ubased S s c-/< 4 M S ; Va S M ; + M S ; > M ; < The based estmato s slghtly moe effcet tha the ubased estmato S.

15 . Itoducto to stmato Devg stmatos: The Method of Momets The oldest method of detemg estmatos If thee ae paametes to be estmated the method cossts of epessg the fst momets tems of these paametes equatg them to the coespodg sample momets ad tag the solutos of the esultg equatos as estmates of the paametes Sce the sample momets ae cosstet estmatos the estmatos deved fom the method of momets ae geeally cosstet

16 . Itoducto to stmato Devg stmatos: Mamum elhood Method The Mamum elhood Method deves estmatos systemcally It uses the assumpto that the sample cossts of d adom vaables { } all dstbuted as. ; ; s jot pobablty desty fucto fo.the the dstbuto of of vecto cotag the paametes a s whee be the desty fucto of ; et α α α α N T f f f α α α α α α α α wth espect to ad s obtaed by mamzg The mamum lelhood estmato M of. ; l ad the coespodg log - lelhood fucto ; values s gve by the lelhood fucto obsevg these patcula The lelhood of. Suppose we have obseved l f l f

17 . Itoducto to stmato Mamum elhood stmato of the Paamete of the Bomal Dstbuto Suppose that the sample cossts of d Beoull adom vaables { } whch tae values betwee 0 ad wth pobabltes -p ad p. The pobablty fo s f p p p ad the jot pobablty fo { } to have a patcula set of ealzatos { } s P h h p p whee h s the sum of. The pobablty dstbuto of dstbuto h h f H h; p p p h H s the bomal Suppose we have obseved Hh. The lelhood of obsevg h fo a patcula value of p s gve by the lelhood fucto p f h; p H The M of p s obtaed by detemg the value of the paamete p fo whch the obseved value h of H s most lely.e. H p o l H p gve by l s mamzed wth espect to p to yeld H h p l + h l p + hl p H p h /

18 . Itoducto to stmato Mamum elhood stmato of the Mea ad the Vaace of a Nomal Radom Vaable Cosde a sample cosstg of d omal adom vaables { }. The jot pobablty fo { } to have the patcula ealzatos { } s The log-lelhood fucto s gve by ep π f l l π The mamzato pocedue yelds + l l 4 0 0

19 . Itoducto to stmato The Appeal of Mamum elhood stmato The Mamum elhood Method s a systematc appoach to seach fo estmatos Ms ted to have pleasg asymptotc popetes. They ca be show to be cosstet ad asymptotcally omal ude faly geeal codtos

20 . Itoducto to stmato Ms of Related stmatos Cosde a adom vecto wth two paametes α β elated to each othe though g α β ad g β α whee g ad g - ae cotuous. If α s a M of α the β g α s a M of β. Smlaly f β s a M of β the α s a M of g β α. ample Suppose s a omal adom vecto wth covaace mat Σ. v et λ λ be the egevalues of Σ ad let e e be the coespodg egevectos. Both the covaace mat Σ α ad ts egevalues ad egevectos ae paametes of. Thee s a cotuous elatoshp betwee these two epesetatos of the covaace stuctue of. Theefoe sce the covaace estmato Σ s the mamum lelhood estmato of Σ the egevalues adegevectos of Σ ae Ms of the egevalues ad egevectos of Σ.

21 . Itoducto to stmato Iteval stmatos: Cofdece Iteval fo a Paamete A teval estmato deals wth a teval o a ego that wll cove the uow but fed paamete wth a gve pobablty A ~ p 00% cofdece teval fo the paamete α s costucted fom two statstcs estmatos α P ~ p α α α ad α α < α such that dcates that the set o the left coves the pot o the ght p~ s chose to be elatvely lage e.g. p~ 0.95 The uppe ad lowe lmts of the cofdece teval ae adom vaables; they ae fuctos of the adom vaables. The cofdece teval dcates the aveaged behavo of the epotg pocedue: It wll cove the fed pot α ~ p 00 % of the tme Gve a patcula sample eveythg about the cofdece teval s fed We delbeately avod the oto cota sce t mples that α s adom Te ealzatos of a 95% cofdece teval fo uow paamete α. O aveage 9 out of 0 tevals wll cove α. I ths eample α0. The cuve shows the desty fucto of the sampled adom vaable.

22 . Itoducto to stmato Iteval stmatos: Cofdece Iteval fo a Radom Vaable It s the teval o ego that wll cove a adom vaable wth a gve pobablty Cosde a epemet whch + obsevatos ae obtaed such a way that they ca be epeseted by + d adom vaables wth that coves ~ + p ad 00% of the tme s a cofdece teval of the adom vaable. The teval The teval s costucted such that epeated samplg of the pobablty of coveage s P ~ < p < + The adom tevals ae wde tha cofdece tevals fo a fed paamete sce they eed to be able to cove a movg athe tha fed taget + Te ealzato of a 95% cofdece teval fo a adom vaable. O aveage 9 out of 0 tevals wll cove. The cuve shows the desty fucto of.

23 . Itoducto to stmato Costuctg Cofdece Itevals A cofdece teval s defed as a set Θ ~ p such that P Θ A ~ ~ p p That s Θ ~ p s costucted so that t coves A tme epeated samplg ~ 00% p of the A deotes ethe a fed paamete o a adom vaable Θ ~ p depeds o the assumed statstcal model the atue of the taget ad the cofdece level p~. Fo uvaate poblems t ca oly be a teval

24 . Itoducto to stmato amples et epeset a sample of d omal adom vaables wth mea ad vaace. What s the cofdece teval fo the mea wth ow vaace? The adom vaable Z has the stadad omal dstbuto Ctcal values z cofdece level such that ~ p P z < Z < z. The shotest ~ p 00% cofdece teval s obtaed by choosg z ad z P Z < z ~ p / z z sg the defto of Z yelds ~ p P z < / < z P. / ad z z / ae defed fo a gve < < such that N0. z /. What s the cofdece teval fo the mea wth uow vaace? The adom vaable T has a t dstbuto wth -degees of Poceedg as ~ p P t < P t / S the left pael oe fds S / / S < t < < t S / feedom..

25 . Theoetcal Dstbutos

26 The t dstbuto

27 . Itoducto to stmato amples et epeset a sample of d omal adom vaables wth mea ad vaace. What s the cofdece teval fo the vaace? The adom vaable ψ - S has a χ lowe tal ctcal values ψ adψ ae chose so that P ψ < ψ 0.5 ~ p / P ψ < ψ ~ p /. ~ The p 00% cofdece teval fo s - S ψ / dstbuto wth - df. - S ψ The uppe ad

28 The Χ Dstbuto