ENGI 4421 Central Limit Theorem Page Central Limit Theorem [Navidi, section 4.11; Devore sections ]

Transcription

1 ENGI 441 Cetral Limit Therem Page Cetral Limit Therem [Navidi, secti 4.11; Devre sectis ] If X i is t rmally distributed, but E X i, V X i ad is large (apprximately 30 r mre), the, t a gd apprximati, X ~ N, At " is a demstrati web prgram t illustrate hw the sample mea appraches a rmal distributi eve fr highly -rmal discrete distributis f X. Csider the expetial distributi, whse p.d.f. (prbability desity fucti) is x 1 1 f x; e, x 0, 0 E X, VX It ca be shw that the exact p.d.f. f the sample mea fr sample size is 1 x x e f x;,, x 0, 0, X 1! 1 1 with mea ad variace E X, V X. [A -examiable derivati f this p.d.f. is available at " Fr illustrati, settig = 1, the p.d.f. fr the sample mea fr sample sizes = 1,, 4 ad 8 are: 1: f x e x x : f x 4xe 3 4x 44x e 4: 8: f x f x X 3! X X 7 8 x e 88 7! x The ppulati mea = E[X] = 1 fr all sample sizes. The variace ad the psitive skew bth dimiish with icreasig sample size. The mde ad the media apprach the mea frm the left.

2 ENGI 441 Cetral Limit Therem Page 11-0 Fr a sample size f = 16, the sample mea X has the p.d.f. f X x x e x ad parameters E X 1 ad V X 15!. 16 A plt f the exact p.d.f is draw here, tgether with the rmal distributi that has the same mea ad variace. The apprach t rmality is clear. Beyd = 40 r s, the differece betwee the exact p.d.f. ad the Nrmal apprximati is egligible. It is geerally the case that, whatever the prbability distributi f a radm quatity may be, the prbability distributi f the sample mea X appraches rmality as the sample size icreases. Fr mst prbability distributis f practical iterest, the rmal apprximati becmes very gd beyd a sample size f 30. Example A radm sample f 100 items is draw frm a expetial distributi with parameter = Fid the prbabilities that (a) a sigle item has a value f mre tha 30; (b) the sample mea has a value f mre tha 30. (a) x.0430 X e e P 30 e

3 ENGI 441 Cetral Limit Therem Page Example (ctiued) (b) >> 30, s CLT ~ N 5,.5 X t a gd apprximati. z x Z P X 30 P [The exact value, usig the Erlag distributi 100,.04, is , a abslute errr f less tha 0.005, but a relative errr f ver 18%.] [The relative errrs imprve clser t the mea.] Recall that, fr X X ~ N,, Z : a X b z Z z z z P P a b b a

4 ENGI 441 Cetral Limit Therem Page Sample Prprtis A Berulli radm quatity has tw pssible utcmes: x = 0 (= failure ) with prbability q1 p ad x = 1 (= success ) with prbability p. Suppse that all elemets f the set X 1, X, X 3,, X are idepedet Berulli radm quatities, (s that the set frms a radm sample). Let T X 1 X X 3 X umber f successes i the radm sample T ad Pˆ = prprti f successes i the radm sample, the T is bimial (parameters:, p) E T p VT pq 1 p EP E T p 1 V pq pq P V T Fr sufficietly large, CLT P ~ N p, pq I practice, ˆP is sufficietly clse t a rmal distributi prvided the expected umbers f bth successes ad failures exceed te: p > 10 ad q > 10.

5 ENGI 441 Cetral Limit Therem Page Example % f all custmers prefer brad A. Fid the prbability that a majrity i a radm sample f 100 custmers des t prefer brad A. p =.55 = P ~ N.55, 100 pˆ p P P.50 P Z P Z pq Ф( 1.01).156 [There is therefre a sigificat pssibility that a majrity f a radm sample will t prefer brad A, eve thugh that clearly disagrees with the ppulati!]

6 ENGI 441 Cfidece Itervals (Oe Sample) Page Cfidece Itervals (Oe Sample) [Navidi, sectis ; Devre chapter 7] S far we have cstructed prbability statemets hw likely certai sample values are, give kwledge f the ppulati frm which the radm sample came. Nw we shall reverse that situati: we have a kw sample i frt f us, frm which we ca ifer the values f the parameters f the ppulati frm which the sample was draw. This is the realm f iferetial statistics. If the radm quatity X is such that ~ N, X, the it is highly ulikely that X will be mre tha three stadard deviatis away frm its mea: P X 3 PX 3 PZ Mre tha 99.7% f the time, X will be clser tha three stadard deviatis t its mea. Fr a sufficietly large radm sample, the cetral limit therem assures us that the sample mea X ~ N, (either exactly r t a excellet apprximati). Therefre we have 99.7% cfidece that a bserved sample mea x is withi three stadard errrs f the ppulati mea. This lie f reasig allws us t replace a pit estimate by a rage f plausible values f a ukw parameter a cfidece iterval. Mre geerally, whe X ~ N,, P z/ X z/ 1

7 ENGI 441 Cfidece Itervals (Oe Sample) Page Subtract, X : P X z X z Multiply by ( 1): P X z X z Rearrage the iequalities t btai: / / / / The cfidece iterval estimatr fr (at a level f cfidece f 1 ) is X z X z The 1 cfidece iterval estimatr fr is a radm iterval X z, X z The prbability is (1 ) that the abve radm iterval icludes the true value f cfidece f all radm samples will prduce a iequality, (the iterval estimate fr ) x z x z that is true. Nte that the cfidece iterval estimate ctais radm quatities at all! The statemet is either abslutely certai t be true r abslutely certai t be false, (depedig the values f,, x, ad ). Iterpretati f a cfidece iterval [ = cfidece iterval estimate ] Oly 5% f all 95% cfidece iterval estimates fr fail t iclude.

8 ENGI 441 Cfidece Itervals (Oe Sample) Page A ccise expressi fr the C.I. (cfidece iterval estimate fr ) is x z A Bayesia view f iterval estimati: If the ly quatity amg {,, x, ad } that we d t kw is, the represet the ukw by the radm quatity A. The x A P z z 1 P x z A x z 1 which is a valid prbability statemet abut the radm quatity A ( decisi thery). A te abut the stadard rmal distributi ad the t distributi Let Z ~ N(0, 1) (stadard rmal distributi), s that P Z z z (cumulative distributi fucti fr the stadard rmal distributi). th The the percetile f the stadard rmal distributi is z, which satisfies PZ z. It als fllws that 1 z z. 1 z The t distributi with ν degrees f freedm is als a bell shaped curve, with a mea, media ad mde at t 0, but with a greater variace tha the stadard rmal distributi. As the umber f degrees f freedm icreases, the t distributi appraches the z (stadard rmal) distributi. The graphs f t 1 ad t 5 are shw here, tgether with z, which is idistiguishable t the eye frm t fr abve 30 r s. Therefre lim t, t, z. Use the t distributi ly if the true ppulati variace is ukw.

9 ENGI 441 Cfidece Itervals (Oe Sample) Page percetile z, use the fial rw i the table f critical values T fid the th f the t distributi ( page 17-0 r the iside back cver f the textbk): z t., The fial rw f the t tables is Therefre PZ r equivaletly z ; P Z r equivaletly z ; etc. Example The rate f eergy lss X (watt) i a mtr is kw t be a rmally distributed radm quatity with stadard deviati 3.0 W. A radm sample f 100 such mtrs prduces a sample mea rate f eergy lss f 58.3 W. Fid a 99% cfidece iterval estimate fr the true mea rate f eergy lss. Edpits f classical CI: x z z.005 t.005, Therefre the edpits are % CI fr μ is μ (W) (t d.p.) [Reas fr " " istead f "<": The stated edpits are just iside the exact iterval. The exact lwer limit is , which places iside. The aswer culd als be quted crrectly as "57.5 < μ < 59.08" r "57.53 μ < 59.08".]

10 ENGI 441 Cfidece Itervals (Oe Sample) Page Example (ctiued) Hw large must be fr the width f the 99% cfidece iterval estimate fr t be less tha 1.0? w x z x z z 3.0 w.57 We require w < Therefre mi = 39. Chice f sample size The width f the cfidece iterval x z, x z is w z z w The sample size is iversely related t the square f the desired width. Edpits f a (1 ) CI fr : (a) (b) (c) kw: x z ukw, large: s 1 x z ; s xi x sample s.d. 1 ukw, small: x t s, 1 i1 Whe is small, X must be early (r exactly) rmal.

11 ENGI 441 Cfidece Itervals (Oe Sample) Page Example The lifetime X f a particular brad f filamets is kw t be rmally distributed. A radm sample f six filamets is tested t destructi ad they are fud t last fr a average f 1,008 hurs with a sample stadard deviati f 6. hurs. (a) Fid a 95% cfidece iterval estimate fr the ppulati mea lifetime. (b) Is the evidece csistet with 1000? (c) Is the evidece csistet with > 1000? X ~ N, X Z ~ N0,1 X but is t kw. Use T t degrees f freedm, S ~ (a) A 95% CI fr μ is s 6. x t 1008 t, 1.05, 5 6 Frm the table page 17-0, t.05, 5.57 s x t , 1 The 95% CI fr μ is < μ < (hurs) ( d.p.) (b) μ = 1000 is t i the CI. Therefre yes, the evidece is csistet with μ 1000 (c) We eed a e-sided CI (t test μ > 1000): t.05, c x s 6. t, = % CI is μ > [Expressed lsely, we are 95% sure that μ > ] Yes, the CI is csistet with μ >

12 ENGI 441 Cfidece Itervals (Oe Sample) Page 11-1 Prperties f a cfidece iterval If we thik f the width f the cfidece iterval as specifyig its precisi, the the cfidece level (r reliability) f the iterval is iversely related t its precisi. Estimati f Ppulati Prprti Whe a radm sample f size is draw frm a ppulati i which a prprti p f the items are successes, the, as we saw page 11.04, pq P ˆ ~ N p, fr sufficietly large p ad q, (amely, p > 10 ad q > 10 ). Cmpare this with X ~ N,, fr which the crrespdig cfidece iterval has edpits x z / s. Hwever, the variace pq is ukw because p ad q are ukw. A bvius remedy is t replace the ukw parameters p ad q by their pit estimates ˆp ad ˆq.

13 ENGI 441 Cfidece Itervals (Oe Sample) Page Therefre, a simple 100(1 )% cfidece iterval estimatr fr p is P z / PQ ad the 100(1 )% cfidece iterval estimate fr p is pˆ z / pq ˆˆ Hwever, these cfidece itervals ca exhibit sigificat errrs whe either p r q is much less tha 100. Durig the 1990 s, mre reliable cfidece itervals fr p were develped. Oe f them is (Devre, sixth editi, secti 7., page 66) : z pq ˆˆ z pˆ 4 z / 1 / / z / Ather iterval, frm the Navidi textbk (page 339), is the Agresti-Cull iterval. If x is the bserved umber f successes i a radm sample f idepedet Berulli trials, the defie x* x ad * 4 s that p* x* x * 4 ad q* 1 p* The the 100(1 )% cfidece iterval estimate fr p is p* z / p* q* * It turs ut that the Bayesia pit estimate fr p is p*, t ˆp.

14 ENGI 441 Cfidece Itervals (Oe Sample) Page Example Frm a radm sample f e thusad silic wafers, 750 pass a quality ctrl test. Fid a 99% cfidece iterval estimate fr p (the true prprti f wafers i the ppulati that are gd). = 1000 ad x = 750 x p ˆ q ˆ 1 pˆ z.005 t.005,.576 Edpits f the C.I.: pq ˆˆ.75.5 pˆ z = Therefre the 99% cfidece iterval estimate fr p is crrect t three sigificat figures. 71.5% p 78.5% Usig the mre precise Agresti-Cull versi f the cfidece iterval yields x* 75 x* = = 75, * = = 1004 p* * 1004 p* q* * 1004 The 99% CI is therefre p* q* p* z * 71.4% p 78.4% [With a sample size f 1000 ad the bserved umbers f successes ad failures bth exceedig 100 by a large margi, it is surprise that the tw versis f the classical cfidece iterval fr p agree t better tha 1%.]

15 ENGI 441 Cfidece Itervals (Oe Sample) Page (1)100% Bayesia Cfidece Iterval fr [t i the Navidi r Devre textbks] Suppse that previus evidece leads us t believe that. The stregth f this belief is represeted by the variace (lwer variace crrespds t strger belief). We wish t update that estimate after a radm sample f size has bee examied. Assume that 30 (s that the Cetral Limit Therem will apply). Prir distributi: New evidece: X ~ N, Sample size = Sample mea = x Sample stadard deviati = s Calculate where wd, w x d w d w w 1, w w w are the weights f the data ad rigial ifrmati respectively, give by wd 1 1, w s d [weights ~ precisi] Psterir distributi: X ~ N, 1 100% Bayesia iterval fr : z / Cmpare with the classical 1 100% cfidece iterval fr : s x z / ( 30) r x z / I may applicatis, the Bayesia iterval is fte arrwer tha the classical cfidece iterval, because the Bayesia iterval icrprates mre ifrmati (previus evidece r belief abut the true value f ). [Nte: it is easy t shw that as (r if x the), * = x ad that as, * s, which are the classical expressis.]

16 ENGI 441 Cfidece Itervals (Oe Sample) Page Examples f Bayesia Cfidece Itervals These examples are mdificatis f the previus examples f classical cfidece itervals fr. Example (mdificati f Example 11.03) The rate f eergy lss X (watt) i a mtr is kw t be a rmally distributed radm quatity ad prir experiece suggests that the mea is 60 W with stadard deviati 3.0 W. A radm sample f 100 such mtrs prduces a sample mea rate f eergy lss f 58.3 W with sample stadard deviati.8 W. Fid a 99% cfidece iterval estimate fr the true mea rate f eergy lss. 1 1 Prir: X ~ N60, 3, 60, 3 Weight: w Data: x 58.3, s.8, 100 Weight: wd * w w d ad w x w wd w The 99% CI fr μ is * z * w d x w d * * = [57.60, 59.03] (W) ( d.p.) Cmpare this with the classical CI: [57.53, 59.07]. [The Bayesia CI is mre precise, due t gd use f prir ifrmati.] [Nte that the true value f is t kw. Therefre the true umber f degrees f freedm is betwee 1 = 99 ad ifiity. Hwever, t.005, ad z , which shifts the budaries f the CI by less tha The errr caused i replacig t by the apprximati z is therefre egligible.] s

17 ENGI 441 Cfidece Itervals (Oe Sample) Page Example (mdificati f Example 11.04) The lifetime X f a particular brad f filamets is kw t be rmally distributed. Prir experiece suggests that 1000 ad 6.0. A radm sample f six filamets is tested t destructi ad they are fud t last fr a average f 1,008 hurs with a sample stadard deviati f 6. hurs. (a) Fid a 95% cfidece iterval estimate fr the ppulati mea lifetime. (b) Is the evidece csistet with 1000? (a) Data: 1 Prir: 1000, 6 w 36 6 x 1008, s 6., 6 w d 6. 1 * * The 95%CI is * t * , hr d.p..05, 5 [Nte that the true umber f degrees f freedm is betwee ( 1) ad, because f the presece f the prir ifrmati. The CI quted here is therefre cservative (wider tha the true iterval).] (b) μ = 1000 is just utside the CI; therefre YES. [This is the same cclusi as that draw frm the classical cfidece iterval fr μ: ( , ). A fairly strg prir belief i μ = 1000, tgether with a small sample size, has dragged the Bayesia CI dw clser t μ = 1000; almst clse eugh t brig μ = 1000 iside the CI.] [The half width f the Bayesia CI is The half width f the classical CI is 6.51] See als [Ed f Chapter 11]

18 ENGI 441 Cfidece Itervals (Oe Sample) Page [Space fr Additial Ntes]