Statistical inference: example 1. Inferential Statistics

Transcription

1 Statistical iferece: example 1 Iferetial Statistics POPULATION SAMPLE A clothig store chai regularly buys from a supplier large quatities of a certai piece of clothig. Each item ca be classified either as good quality or top quality. The agreemets require that the delivered goods comply with stadards predetermied quality. I particular, the proportio of good quality items must ot exceed 25% of the total. From a cosigmet 40 items are extracted ad 29 of these are of top quality whereas the remaiig 11 are of good quality. Statistical iferece is the brach of statistics cocered with drawig coclusios ad/or makig decisios cocerig a populatio based oly o sample data. INFERENTIAL PROBLEMS: 1. provide a estimate of π ad quatify the ucertaity associated with such estimate; 2. provide a iterval of reasoable values for π; 3. decide whether the delivered goods should be retured to the supplier Statistical iferece: example 1 Statistical iferece: example 2 Formalizatio of the problem: POPULATION: all the pieces of clothes of the cosigmet; VARIABLE OF INTEREST: good/top quality of the good biary variable; PARAMETER OF INTEREST: proportio of good quality items π; SAMPLE: 40 items extracted from the cosigmet. The value of the parameter π is ukow, but it affects the samplig values. Samplig evidece provides iformatio o the parameter value. 189 A machie i a idustrial plat of a bottlig compay fills oe-liter bottles. Whe the machie is operatig ormally the quatity of liquid iserted i a bottle has mea µ = 1 liter ad stadard deviatio σ =0.01 liters. Every workig day 10 bottles are checked ad, today, the average amout of liquid i the bottles is x = with s = INFERENTIAL PROBLEMS: 1. provide a estimate of µ ad quatify the ucertaity associated with such estimate; 2. provide a iterval of reasoable values for µ; 3. decide whether the machie should be stopped ad revised. 190

2 Formalizatio of the problem: Statistical iferece: example 2 POPULATION: all the bottles filled by the machie; VARIABLE OF INTEREST: amout of liquid i the bottles cotiuous variable; PARAMETERS OF INTEREST: mea µ ad stadard deviatio σ of the amout of liquid i the bottles; SAMPLE: 10 bottles. The values of the parameters µ ad σ are ukow, but they affect the samplig values. Samplig evidece provides iformatio o the parameter values. 191 The sample Cesus survey: attempt to gather iformatio from each ad every uit of the populatio of iterest; sample survey: gathers iformatio from oly a subset of the uits of the populatio of iterest. Why usig a sample? 1. Less time cosumig tha a cesus; 2. less costly to admiister tha a cesus; 3. measurig the variable of iterest may ivolve the destructio of the populatio uit; 4. a populatio may be ifiite. 192 Probability samplig A probability samplig scheme is oe i which every uit i the populatio has a chace greater tha zero of beig selected i the sample, ad this probability ca be accurately determied. Probabilistic descriptio of a populatio SIMPLE RANDOM SAMPLING: every uit has a equal probability of beig selected ad the selectio of a uit does ot chage the probability of selectig ay other uit. For istace: extractio with replacemet; extractio without replacemet. For large populatios compared to the sample size the differece betwee these two samplig techiques is egligible. I the followig we will always assume that samples are extracted with replacemet from the populatio of iterest. 193 Uits of the populatio; variable X measured o the populatio uits; sometimes the distributio of X is kow, for istace i X Nµ,σ 2 ; ii X Beroulliπ. 194

3 Probabilistic descriptio of a sample The observed samplig values are x 1,x 2,...,x ; BEFORE the sample is observed the samplig values are ukow ad the sample ca be writte as a sequece of radom variables Samplig distributio of a statistic 1 Suppose that the sample is used to compute a give statistic, for istace i the sample mea X; ii the sample variace S 2 ; iii the proportio P of uits with a give feature; X 1,X 2,...,X for simple radom samples with replacemet: 1. X 1,X 2,...,X are i.i.d.; 2. the distributio of X i is the same as that of X for every i = 1,...,. 195 geerically, we cosider a arbitrary statistic T = gx 1,...,X where g is a give fuctio. 196 Samplig distributio of a statistic 2 Suppose that X Nµ,σ 2 Normal populatio Oce the sample is observed, the observed value of the statistic is give by t = gx 1,...,x ; suppose that we draw all possible samples of size from the give populatio ad that we compute the statistic T for each sample; the samplig distributio of T is the distributio of the populatio of the values t of all possible samples. i this case the statistics of iterest are: i the sample mea X = 1 X i i=1 ii the sample variace S 2 = 1 X i X 2 1 i=1 the correspodig observed values are x = 1 x i ad s 2 = 1 x i x 2, i=1 1 i=1 respectively

4 Expected value of the sample mea The sample mea The sample mea is a liear combiatio of the variables formig the sample ad this property ca be exploited i the computatio of the expected value of X, that is E X; the variace of X, that is Var X ; the probability distributio of X. For a simple radom sample X 1,...,X, the expected value of X is X1 +X E X = E 2 + +X = 1 EX 1 +X 2 + +X = 1 [EX 1+EX 2 + +EX ] = 1 µ = µ Variace of the sample mea For a simple radom sample X 1,...,X, the variace of X is Var X X1 +X = Var 2 + +X = 1 2VarX 1 +X 2 + +X Samplig distributio of the mea For a simple radom sample X 1,X 2,...,X, the sample mea X has expected value µ ad variace σ 2 /; if the distributio of X is ormal, the = 1 2 [VarX 1+VarX 2 + +VarX ] = 1 2 σ2 = σ2 X N µ, σ2 more geerally, the cetral limit theorem ca be applied to state that the distributio of X is APPROXIMATIVELY ormal

5 The sample variace The chi-squared distributio The sample variace is defied as S 2 = 1 X i X 2 1 i=1 if X i Nµ,σ 2 the 1S 2 σ 2 χ 2 1 Let Z 1,...,Z r be i.i.d. radom variables with distributio N0;1; the radom variable X = Z Z2 r is said to follow a CHI-SQUARED distributio with r degrees of freedom d.f.; we write X χ 2 r; X ad S 2 are idepedet. EX = r ad VarX = 2r Coutig problems The variable X is biary, i.e. it takes oly two possible values; for istace success ad failure ; the radom variable X takes values 1 success ad 0 failure; the parameter of iterest is π, the proportio of uits i the populatio with value 1 ; Formally, X Beroulliπ so that EX = π ad VarX = π1 π. The sample proportio 1 Simple radom sample X 1,...,X ; the variable X takes two values: 0 ad 1, ad the sample proportio is a special case of sample mea i=1 X i P = the observed value of P is p

6 The sample proportio 2 The sample proportio is such that Estimatio EP = π ad VarP = π1 π for the cetral limit theorem, the distributio of X is approximatively ormal; sometimes the followig empirical rules are used to decide if the ormal approximatio is satisfyig: 1. π > 5 ad 1 π > 5. Parameters are specific umerical characteristics of a populatio, for istace: a proportio π; a mea µ; a variace σ 2. Whe the value of a parameter is ukow it ca be estimated o the basis of a radom sample. 2. p1 p > Poit estimatio A poit estimate is a estimate that cosists of a sigle value or poit, for istace oe ca estimate a mea µ with the sample mea x; a proportio π with a sample proportio p; Estimator vs estimate A estimator of a populatio parameter is a radom variable that depeds o sample iformatio, whose value provides a approximatio to this ukow parameter. a poit estimate is always provided with its stadard error that is a measure of the ucertaity associated with the estimatio process. A specific value of that radom variable is called a estimate

7 Estimatio ad ucertaity Poit estimatio of a mea σ 2 kow Parameter θ; the samplig statistics T = gx 1,...,X o which estimatio is based is called the estimator of θ ad we write ˆθ = T the observed value of the estimator, t, is called a estimate of θ ad we write ˆθ = t; it is fudametal to assess the ucertaity of ˆθ; a measure of ucertaity is the stadard deviatio of the estimator, that is SDT = SDˆθ. This quatity is called the STANDARD ERROR of ˆθ ad deoted by SEˆθ. Cosider the case where X 1,...,X is a simple radom sample from X Nµ,σ 2 ; Parameters: µ, ukow; assume that the value of σ 2 is kow. the sample mea ca be used as estimator of µ: µ = X; the distributio of the estimator is ormal with STANDARD ERROR µ E µ = µ ad Var µ = σ2 SE µ = σ Poit estimatio of a mea σ 2 ukow Poit estimatio of a mea with σ 2 kow: example I the bottlig compay example, assume that the quatity of liquid i the bottles is ormally distributed. The a poit estimate of µ is Typically the value of σ 2 is ot kow; i this case we estimate it as ˆσ 2 = s 2 ; this ca be used, for istace, to estimate the stadard error of ˆµ µ = ad the stadard error of this estimate is SEˆµ = σ 10 = = ŜEˆµ = ˆσ. I the bottlig compay example, if σ is ukow it ca be estimated as ŜEˆµ = =

8 Poit estimatio of a proportio Parameter: π; Poit estimatio for the mea of a o-ormal populatio X 1,...,X i.i.d. with EX i = µ ad VarX i = σ 2 ; the distributio of X i is ot ormal; for the cetral limit theorem the distributio of X is approximatively ormal. the sample proportio P is used as a estimator of π π = P this estimator is approximately ormally distributed with E π = π ad Var π = π1 π the STANDARD ERROR of the estimator is π1 π SE π = ad i this case the value of stadard error is ever kow Estimatio of a proportio: example For the clothig store chai example the estimate of the proportio π of good quality items is π = = ad a ESTIMATE of the stadard error is ŜEˆπ = = Properties of estimators: ubiasedess A poit estimator ˆθ is said to be a ubiased estimator of the parameter θ if the expected value, or mea, of the samplig distributio of ˆθ is θ, formally if Eˆθ = θ Iterpretatio of ubiasedess: if the samplig process was repeated, idepedetly, a ifiite umber of times, obtaiig i this way a ifiite umber of estimates of θ, the arithmetic mea of such estimates would be equal to θ. However, ubiasedess does ot guaratees that the estimate based o oe sigle sample coicides with the value of θ

9 Poit estimator of the variace The sample variace S 2 is a ubiased estimator of the variace σ 2 of a ormally distributed radom variable ES 2 = σ 2. Bias of a estimator Let ˆθ be a estimator of θ. The bias of ˆθ, Biasˆθ, is defied as the differece betwee the expected value of ˆθ ad θ O the other had S 2 is a biased estimator of σ 2 Biasˆθ = Eˆθ θ E S 2 = 1 σ 2. The bias of a ubiased estimator is Properties of estimators: Mea Squared Error MSE For a estimator ˆθ of θ the ukow estimatio error is give by θ ˆθ The Mea Squared Error MSE is the expected value of the square of the error MSEˆθ = E[θ ˆθ 2 ] = Var ˆθ +[θ Eˆθ] 2 = Var ˆθ +Biasˆθ 2 Hece, for a ubiased estimator, the MSE is equal to the variace. 221 Most Efficiet Estimator Let ˆθ 1 ad ˆθ 2 be two estimator of θ, the the MSE ca be use to compare the two estimators; if both ˆθ 1 ad ˆθ 2 are ubiased the ˆθ 1 is said to be more efficiet tha ˆθ 2 if Var ˆθ 1 < Var ˆθ 2 ote that if ˆθ 1 is more efficiet tha ˆθ 2 the also MSEˆθ 1 < MSEˆθ 2 ad SEˆθ 1 < SEˆθ 2 ; the most efficiet estimator or the miimum variace ubiased estimator of θ is the ubiased estimator with the smallest variace. 222

10 Iterval estimatio A poit estimate cosists of a sigle value, so that if X is a poit estimator of µ the it holds that P X = µ = 0 more geerally, Pˆθ = θ = 0. Iterval estimatio is the use of sample data to calculate a iterval of possible or probable values of a ukow populatio parameter. Cofidece iterval for the mea of a ormal populatio σ kow X 1,...,X simple radom sample with X i Nµ,σ 2 ; assume σ kow; a poit estimator of µ is ˆµ = X N µ, σ2 the stadard error of the estimator is SEˆµ = σ Before the sample is extracted... The sample distributio of the estimator is completely kow but for the value of µ; the ucertaity associated with the estimate depeds o the size of the stadard error. For istace, the probability that ˆµ = X takes a value i the iterval µ±1.96 SE is 0.95 that is 95%. Cofidece iterval for µ The probability that X belogs to the iterval µ 1.96SE, µ+1.96se is 95%; this ca be also stated as: the probability that the iterval X 1.96SE, X +1.96SE area 95% cotais the parameter µ is 95% SE SE µ 3 SE µ 2 SE µ 1 SE µ µ + 1 SE µ + 2 SE µ + 3 SE µ µ P µ 1.96 SE X µ+1.96 SE =

11 Formal derivatio of the 95% cofidece iterval for µ σ kow It holds that so that X µ SE 0.95 = P N 0,1 where SE = σ 1.96 X µ SE 1.96 = P 1.96 SE X µ 1.96 SE = P X 1.96 SE µ X SE = P X 1.96 SE µ X SE Cofidece iterval for µ with σ kow: example I the bottlig compay example, if oe assumes σ = 0.01 kow, a 95% cofidece iterval for µ is that is so that ; ; ; After the sample is extracted... O the basis of the sample values the observed value of µ = x is computed. x may belog to the iterval µ±1.96 SE or ot. For istace a differet sample... A differet sample may lead to a sample mea x that, as i the example below, does ot belog to the iterval µ±1.96 SE ad, as a cosequece, also the iterval x 1.96 SE; x+1.96 SE will ot cotai µ. area 95% x area 95% x µ 3 SE µ 2 SE µ 1 SE µ µ + 1 SE µ + 2 SE µ + 3 SE ad i this case x belogs to the iterval µ±1.96 SE ad, as a cosequece, also the iterval x 1.96 SE; x+1.96 SE will cotai µ. 229 µ 3 SE µ 2 SE µ 1 SE µ µ + 1 SE µ + 2 SE µ + 3 SE The iterval x 1.96 SE; x+1.96 SE will cotai µ for the 95% of all possible samples. 230

12 Iterpretatio of cofidece itervals Probability is associated with the procedure that leads to the derivatio of a cofidece iterval, ot with the iterval itself. A specific iterval either will cotai or will ot cotai the true parameter, ad o probability ivolved i a specific iterval. Cofidece iterval: defiitio A cofidece iterval for a parameter is a iterval costructed usig a procedure that will cotai the parameter a specified proportio of the times, typically 95% of the times. Cofidece itervals for five differet samples of size = 25, extracted from a ormal populatio with µ = 368 ad σ = 15. A cofidece iterval estimate is made up of two quatities: iterval: set of scores that represet the estimate for the parameter; cofidece level: percetage of the itervals that will iclude the ukow populatio parameter A wider cofidece iterval for µ Sice it also holds that P µ 2.58 SE X µ+2.58 SE = 0.99 Cofidece level The cofidece level is the percetage associated with the iterval. A larger value of the cofidece level will typically lead to a icrease of the iterval width. The most commoly used cofidece levels are area 99% x area 99% x 68% associated with the iterval X ±1SE; 95% associated with the iterval X ±1.96SE; 99% associated with the iterval X ±2.58SE. µ 3 SE µ 2 SE µ 1 SE µ µ + 1 SE µ + 2 SE µ + 3 SE µ 3 SE µ 2 SE µ 1 SE µ µ + 1 SE µ + 2 SE µ + 3 SE the the probability that X 2.58SE, X+2.58SE cotais µ is 99%. Where the values 1, 1.96 ad 2.58 are derived from the stadard ormal distributio tables

13 Z N0,1; Notatio: stadard ormal distributio tables α value betwee zero ad oe; z α value such that the area uder the Z pdf betwee z α ad + is equal to α; formally furthermore PZ > z α = α ad PZ < z α = 1 α P z α/2 < Z < z α/2 = 1 α 235 Cofidece iterval for µ with σ kow: formal derivatio 1 It holds that so that X µ SE or, equivaletly, N 0,1 where SE = σ P µ z α/2 SE X µ+z α/2 SE = 1 α P z α/2 X µ SE z α/2 = 1 α 236 Cofidece iterval at the level 1 α for µ with σ kow Cofidece iterval for µ with σ kow: formal derivatio 2 1 α = P z α/2 X µ SE z α/2 = P z α/2 SE X µ z α/2 SE = P X z α/2 SE µ X +z α/2 SE = P X z α/2 SE µ X +z α/2 SE A cofidece iterval at the cofidece level 1 α, or 1 α%, for µ is give by Sice SE = σ the X z α/2 SE; X +z α/2 SE X z α/2 σ ; X +z α/2 σ

14 Margi of error The cofidece iterval Reducig the margi of error x±z α/2 σ ca also be writte as x±me where ME = z α/2 σ ME = z α/2 σ is called the margi of error. The margi of error ca be reduced, without chagig the accuracy of the estimate, by icreasig the sample size. the iterval width is equal to twice the margi of error Cofidece iterval for µ with σ ukow The Studet s t distributio 1 X N µ, σ2 ; X µ SE N0,1; i this case the stadard error is ukow ad eeds to be estimated. For Z N0;1 ad X χ 2 r, idepedet; the radom variable T = Z X/r is said to follow a Studet s t distributio with r degrees of freedom; the pdf of the t distributio differs from that of the stadard ormal distributio because it has heavier tails. SEˆµ = σ is estimated by ŜEˆµ = ˆσ where ˆσ = S Studet s t ormal ad it holds that X µ ŜE t t 1 ad N0;1 compariso. 242

15 Cofidece iterval for µ with σ ukow The Studet s t distributio 2 For r + the Studet s t distributio coverges to the stadard ormal distributio t 25 ad N0;1 compariso.. 1 α = P t 1,α/2 X µ ŜE t 1,α/2 = P t 1,α/2 ŜE X µ t 1,α/2 ŜE = P X t 1,α/2 ŜE µ X +t 1,α/2 ŜE = P X t 1,α/2 ŜE µ X +t 1,α/2 ŜE where t 1,α/2 is the value such that the area uder the t pdf, with 1 d.f. betwee t 1,α/2 ad + is equal to α/2. Hece, a cofidece iterval at the level 1 α for µ is X t 1,α/2 S ; X +t 1,α/2 S Cofidece iterval for µ with σ ukow: example For the bottlig compay example, if the value of σ is ot kow, the s = e t 9;0.025 = ad a 95% cofidece iterval for µ is that is so that ; ; Cofidece iterval for the mea of a o-ormal populatio X 1,...,X i.i.d. with EX i = µ ad VarX i = σ 2 ; the distributio of X i is ot ormal; for the cetral limit theorem the distributio of X is approximatively ormal; if oe uses the procedures described above to costruct a cofidece iterval for µ the omial cofidece level of the iterval is oly a approximatio of the true cofidece level ;

16 For the cetral limit theorem so that P π SE 1 α P Cofidece iterval for π N 0,1 where SE = z α/2 P π SE z α/2 π1 π = P z α/2 SE P π z α/2 SE = P P z α/2 SE π P +z α/2 SE = P P z α/2 SE π P +z α/2 SE Cofidece iterval for π: example For the clothig store chai example, a 95% cofidece iterval for π is SEˆπ; SEˆπ 40 so that π1 π SEˆπ = is estimated by ŜEˆπ = ˆπ1 ˆπ where ˆπ = x = ad oe obtais ; so that Sice π is always ukow, it is always ecessary to estimate the stadard error ; Example of decisio problem Problem: i the example of the bottlig compay, the quality cotrol departmet has to decide whether to stop the productio i order to revise the machie. Hypothesis: the expected mea quatity of liquid i the bottles is equal to oe liter. The stadard deviatio is assumed kow ad equal to σ = The decisio is based o a simple radom sample of = 10 bottles. Statistical hypotheses A decisioal problem i expressed by meas of two statistical hypotheses: the ull hypothesis H 0 the alterative hypothesis H 1 the two hypotheses cocer the value of a ukow populatio parameter, for istace µ, { H0 : µ = µ 0 H 1 : µ µ

17 Distributio of X uder H 0 If H 0 is true that is uder H 0 the distributio of the sample mea X has expected value equal to µ 0 = 1; has stadard error equal to SE = σ/ 10 = if X 1,...,X 10 is a ormally distributed i.i.d. sample tha also X follows a ormal distributio, otherwise the distributio of X is oly approximatively ormal by the cetral limit theorem. Observed value of the sample mea The observed value of the sample mea is x. x is almost surely differet form µ 0 = 1. uder H 0, the expected value of X is equal to µ 0 ad the differece betwee µ 0 ad x is uiquely due to the samplig error. HENCE THE SAMPLING ERROR IS x µ 0 that is observed value mius expected value Decisio rule The space of all possible sample meas is partitioed ito a Outcomes ad probabilities There are two possible states of the world ad two possible decisios. This leads to four possible outcomes. rejectio regio also said critical regio; orejectio regio. H 0 IS REJECTED H 0 TRUE H 0 FALSE Type I error α OK H 0 IS NOT REJECTED OK Type II error β The probability of the type I error is said sigificace level of the test ad ca be arbitrarily fixed typically 5%

18 Test statistic A test statistic is a fuctio of the sample, that ca be used to perform a hypothesis test. for the example cosidered, X is a valid test statistics, which is equivalet to the, more commo, z test statistic Hypothesis testig: example 5% sigificace level arbitrarily fixed; Z = X N0,1 Z = X µ 0 σ/ N0,1 the observed value of Z is z = = 2.055; the empirical evidece leads to the rejectio of H p-value approach to testig z test for µ with σ kow the p-value, also called observed level of sigificace is the probability of obtaiig a value of the test statistic more extreme tha the observed sample value, uder H 0. decisio rule: compare the p-value with α: X 1,...,X i.i.d. with distributio Nµ,σ 2 ; the value of σ is kow. p-value < α = reject H 0 p-value α = oreject H 0 for the example cosidered p-value=pz PZ = 0.04; p-value < 5% = statistically sigificat result. Hypotheses: test statistic: { H0 : µ = µ 0 H 1 :... Z = X µ 0 σ/ p-value < 1% = highly sigificat result. uder H 0 the test statistic Z has distributio N0;

19 z test: two-sided hypothesis z test: oe-sided hypothesis right H 1 : µ µ 0 H 1 : µ > µ 0 i this case i this case p value = PZ > z p value = PZ > z 3.5 z z z z test: oe-sided hypothesis left t test for µ with σ ukow H 1 : µ < µ 0 Hypotheses: i this case { H0 : µ = µ 0 H 1 : µ µ 0 p value = PZ < z test statistic: t = X µ 0 S/ 3.5 z p-value: PT 1 > t where T 1 follows a Studet s t distributio with 1 degrees of freedom

20 z test for π Test for a proportio Hypotheses: Null hypotheses: H 0 : π = π 0 ; Uder H 0 the samplig distributio of P is approximately ormal with expected value EP = π 0 ad stadard error π SEP = 0 1 π 0 Note that uder H 0 there are o ukow parameters. test statistic: { H0 : π = π 0 H 1 : π π 0 P π Z = 0 π 0 1 π 0 / P-value: PZ > z z test for π: example For the clothig store chai example, the hypotheses are { H0 : π = 0.25 H 1 : π > 0.25 Hece, uder H 0 the stadard error is SE = = so that z = = ad the p-value is PZ 0.37 = 0.36 ad the ull hypothesis caot be rejected. 265