FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals for meas 6. Hypothesis testig. Mea ad stadard error of sample data Two differet radom variables ca be measured for the same object: e.g. the height ad weight of a perso. Both these radom variables have distributios, mea values ad variaces. However, taller people are usually heavier tha shorter people so these two variables are depedet. Variables ca also be idepedet: idepedet evets P (A B) = P(A)P(B) idepedet discrete radom variables P (X = x i Y = y j )=P(X=x i )P(Y =y j ) The latter defies a joit distributio for the two radom variables. Ex. A ew plat at a maufacturig site has to be istalled ad the commissioed. The times required for the two steps deped upo differet radom factors, ad ca therefore be regarded as idepedet. Based o past experiece the respective distributios for X (istallatio time) ad Y (commissioig time), both i days, are P (X =)=0., P(X =4)=0.5, P(X =5)=0. P(Y =)=0.4, P(Y =)=0.6 Fid the joit distributio for X ad Y, ad the probability that the total time will ot exceed 6 days. Sice the factors are idepedet P (X = x i Y = y j )=P(X=x i )P(Y =y j ). Thus the joit probability table is Y/X 4 5 Y/X 4 5 0. 0.4 0.5 0.4 0. 0.4 i.e. 0.08 0.0 0. 0. 0.6 0.5 0.6 0. 0.6 0. 0.0 0.8 Note that the colum ad row totals give the idividual distributios for X ad Y. For the secod part of the questio P (X + Y 6) = P (X = Y =)+P(X= Y =)+P(X=4 Y =) : o other combiatios are possible. The joit distributios ca the be read from the table givig P (X + Y 6) = 0.08 + 0. + 0.0 = 0.40 It would be easy to calculate P (X +Y = w i )foreachw i, ad the evaluate E(X +Y ) usig the expressio E(X + Y )= w i w i P(X+Y =w i ).
However, we ca use a more geeral result. Give that E(X) = x i P(X=x i ) ad that E(Y) satisfies x i a similar formula, the it ca be show that for idepedet variables E(X + Y )=E(X)+E(Y) ad Var(X +Y)=Var(X)+Var(Y). Give some data we usually do ot kow the exact distributio but it would be useful to try to estimate the mea ad variace from the data Def. For a sample {X,X,..., X } of data the sample average is defied by X = X i i= Def. For sample data {X,X,..., X } the sample variace is defied by S X = (X i X). i= [Note that is used i the deomiator i S X because the differeces X i X sum to zero ad so are ot idepedet.] Ex. A die was tossed six times producig the followig results 6,, 4,,, 5. Fid the sample average ad the stadard variace. X = 0 (6++4+++5)= 6 6 = 0. [ ( S X = 6 0 ) ( + 0 ) ( + 4 0 ) ( + 0 ) ( + 0 ) ( + 5 0 ) ] 6 [ (8 = ) ( + 4 ( + + 5 ) 4 ( + ) ) 7 ] 5 + = 64+6+4+6+49+5 = 74 5 9 45 = 58 5.87 S X.97 [For ubiased die it ca be show that the theoretical results for X ad S X are.5 ad.708 respectively. ] for a large umber of tosses. Biomial distributio Specifyig the exact distributio of a radom variable requires a lot of iformatio. Good estimates of mea, variace etc. ca be obtaied from data. Ofte probability distributios ca be determied by formulae usig the estimated values of the parameters. Cosider the simple coi tossig experimet where oly two outcomes are possible success (say ) or failure (say 0). A Beroulli trial is a simple observatio of a radom variable X, say, that ca take the values or0:suppose P(X=)=p, P (X =0)= p. The E(X) =X=(p)+0( p)=p+0=p, ad Var(X) =E(X X) =E(X p) =(0 p) P(X=0)+( p) P(X=) =p ( p)+( p) p=p( p)(p +( p)) = p( p). Now let {X,..., X } deote idepedet Beroulli trials, each with success probability p.the umber of successes Y = X + X +... +X. Suppose Y = k,where 0 k,the k of the X i values equal ad k equal 0. The probability of k s at first with k 0 s followig is p k ( p) k, sice the outcomes are idepedet. The umber of ways of distributig k successes amog trials = hece P (Y = k) = = k p k ( p) k. k! ( k)!k!,
This is the geeral form of the biomial distributio ad leads to mea = p, variace = p( p). Ex. If o average i 0 of a certai type of colum fails uder loadig, what is the probability that amog 6 such colums at most will fail? Give P (colum failig) = P (F )= 9 =0.05, P(colum ot failig) = P (F )= 0 0 =0.95, therefore P (0 F )=(0.95) 6 =0.440, P ( F )=(0.05) (0.95) 5 6 P ( F )=(0.05) (0.95) 4 6 P (at most fail) = 0.440 + 0.706 + 0.460 = 0.957 =(0.05)(0.469) 6! =(0.05)(0.469)(6) = 0.706 5!! =(0.005)(0.48767) 6! 4!! =(0.005)(0.48767)(6)(5) =0.460. Normal distributio This occurs very frequetly i practice. Def. A cotiuous radom variable X has a ormal distributio (or Gaussia distributio) with mea µ X ad variace σ X if the probability desity fuctio satisfies f X (x) = exp [ (x µ X) ] π σ X ( <x< ). Write X N (µ X,σ X ) to represet a radom variable with ormal distributio which has mea µ X variace σ X (see figure ). f X 0.4 µ =0, σ = X X ad µ X =0, = -6-4 - 0 4 figure 6 x The stadard ormal distributio has mea 0 ad variace, ad leads to Def. Stadard ormal cumulative distributio is Φ(z) = z e x / dx. π Φ(z) is usually tabulated, ad a summary of values appears o the Formula Sheet. Note that Φ(z) deotes the area uder the probability distributio curve to the left of x = z. Suppose that X is a ormal variable with mea µ X ad variace σ X the it ca be show that Z = X µ X is also a ormal variable with mea 0 ad variace. (The latter result is very useful i applicatios.)
Ex 4. The burig time X of a experimetal rocket is a radom variable havig (approximately) a ormal distributio with mea 600s ad stadard deviatio 5s. Fid the probability that such a rocket will bur for (a) less tha 550s, (b) more tha 67.5s. (a) Give µ X = 600 ad = 5, therefore where Z = X 600 5 (b) ( X 600 P (X <550) = P < 5 ) 550 600 = P (Z < ), 5. From symmetry (look at figure ), ad the usig the results o the Formula Sheet, P (Z < ) = P (Z >) = P (Z ) = Φ() = 0.977 = 0.08 X 600 67.5 600 P (X >67.5) = P > = P (Z >.5) 5 5 = P(Z.5) = Φ(.5) = 0.9 = 0.0668 Whe >0 the it ca be show that the ormal distributio with mea p ad variace p( p) provides a very accurate approximatio to the biomial distributio. 4. Samplig Oe of the major problems i statistics, estimatig the properties of a large populatio from the properties of a sample of idividuals chose from that populatio, is cosidered i this sectio. Select at radom a sample of observatios X,X,..., X take from a populatio. From these observatios you ca calculate the values of a umber of statistical quatities, for example the sample mea X. If you choose aother radom sample of size from the same populatio, a differet value of the statistic will, i geeral, result. I fact, if repeated radom samples are take, you ca regard the statistic itself as a radom variable, ad its distributio is called the samplig distributio of the statistic. For example, cosider the distributio of heights of all adult me i Eglad, which is kow to coform very closely to the ormal curve. Take a large umber of samples of size four, draw at radom from the populatio, ad calculate the mea height of each sample. How will these mea heights be distributed? We fid that they are also ormally distributed about the same mea as the origial distributio. However, a radom sample of four is likely to iclude me both above ad below average height ad so the mea of the sample will deviate from the true mea less tha a sigle observatio will. This importat geeral result ca be stated as follows: If radom samples of size are take from a distributio whose mea is µ X ad whose stadard deviatio is, the the sample meas form a distributio with mea µ X ad stadard deviatio = /. Note that the theorem holds for all distributios of the paret populatio. However, if the paret distributio is ormal the it ca be show that the samplig distributio of the sample mea is also ormal. The stadard deviatio of the sample mea, defied above, is usually called the stadard error of the sample mea. Let us ow preset three worked examples. Ex 5. A radom sample is draw from a populatio with a kow stadard deviatio of.0. Fid the stadard error of the sample mea if the sample is of size (i) 9, (ii) 00. What sample size would give a stadard error equal to 0.5? Usig the result stated earlier (i) stadard error = σx = =0.667, 9 to decimal places, 4
(ii) stadard error = σx = 00 =0. If the stadard error equals 0.5, the / =0.5. Squarig the implies that 4/ =0.5 or =6, (i.e. the sample size is 6). Ex 6. The diameters of shafts made by a certai maufacturig process are kow to be ormally distributed with mea.500 cm ad stadard deviatio 0.009 cm. What is the distributio of the sample mea diameter of ie such shafts selected at radom? Calculate the percetage of such sample meas which ca be expected to exceed.506 cm. Sice the process is ormal we kow that the samplig distributio of the sample mea will also be ormal, with the same mea,.500 cm, but with a stadard error (or stadard deviatio) =0.009/ 9=0.00 cm. I order to calculate the probability that the sample mea is bigger tha.506, i.e. X>.506, we stadardise i the usual way by puttig Z =(X.500)/0.00, ad the X.500.506.500 P (X >.506) = P > = P (Z >.0) 0.00 0.00 = P(Z.0) = Φ(.0) =.977 = 0.08, usig the formula sheet Hece,.8% of the sample meas ca be expected to exceed.506 cm. Ex 7. What is the probability that a observed value of a ormally distributed radom variable lies withi oe stadard deviatio from the mea? Give ormally distributed radom variable, X, has mea µ X ad stadard deviatio, i.e. X N(µ X,σ X ). We eed to calculate P (µ X X µ X + ). Defie Z = X µ X,the Z N(0, ). It follows that ( (µx ) µ X P (µ X X µ X + )=P X µ X (µ ) X + ) µ X =P( Z ) = P (0 Z ), by symmetry = (Φ() Φ(0)) = (0.84 0.5000) =(0.4) = 0.686 It was stated above that whe the paret distributio is ormal the the samplig distributio of the sample mea is also ormal. Whe the paret distributio is ot ormal, the obtai the followig theorem (surprisig result?): Cetral limit theorem If a radom sample of size, ( 0), is take from ANY distributio with mea µ X ad stadard deviatio, the the samplig distributio of X is approximately ormal with mea µ X ad stadard deviatio /, the approximatio improvig as icreases. Ex 8. It is kow that a particular make of light bulb has a average life of 800 hrs with a stadard deviatio of 48 hrs. Fid the probability that a radom sample of 44 bulbs will have a average life of less tha 790 hrs. Sice the umber of bulbs i the sample is large, the sample mea will be ormally distributed with mea = 800 ad stadard error = 48 =4. Put Z= X µ X = (X 800),the 44 4 X 800 790 800 P(X<790) = P < 4 4 = P (Z <.5) = P (Z >.5), by symmetry = P(Z.5) = 0.998 = 0.006. 5
To coclude this sectio the mai results cocerig the distributio of the sample mea X are summarised. Cosider a paret populatio with mea µ X ad stadard deviatio. From this populatio take a radom sample of size with sample mea X ad stadard error /. Defie Z = X µ X / the (i) for all the distributio of Z is N(0, ) if the distributio of the paret populatio is ormal; (ii) < 0 the distributio of Z is approximately N(0, ) if the distributio of the paret populatio is approximately ormal; (iii) 0 the distributio of Z is a good approximatio to N(0, ) for all distributios of the paret populatio. 5. Cofidece itervals for meas Choose a sample at radom from a populatio, the the mea, X, of the sample is said to provide a poit estimator of the populatio mea µ X. The accuracy of this estimate is measured by the cofidece iterval, which is the iterval withi which you ca be reasoably sure the value of the populatio mea µ X lies. Oe usually calculates k by specifyig that the probability that the iterval (X k, X + k) cotais the populatio mea is 0.95, or 0.99. For example, if k is calculated so that P (X k µ X X + k) =.95 the the iterval is called the 95% cofidece iterval if the iterval is calculated for very may samples the 95 out of 00 itervals would cotai µ X. O replacig 95 by 99 you obtai the defiitio of a 99% cofidece iterval. To proceed further, assume that the stadard deviatio is kow ad that Z = X µ X / is distributed with the stadard ormal distributio N(0, ) (the coditios for this to hold were stated at the ed of sectio 4). The results o the Formula Sheet show that 95% of the stadard ormal distributio lies betwee.96 ad.96. Hece P (.96 Z.96) = 0.95. Rearragig the iequalities iside the bracket to obtai coditios o X yields Z = X µ X /.96 X µ X.96 / µ X X.96 / ; Z = X µ X /.96 X µ X.96 / µ X X +.96 /. It follows that the earlier expressio ca be re-writte as P ( X.96 σx µ X X +.96 σx ) =0.95, ad hece the iterval X.96 σx µ X X +.96 σx is the 95% cofidece iterval for µ X. Similarly X.58 σx µ X X +.58 σx is the 99% cofidece iterval for µ X. 6
Ex 9. The percetage of copper i a certai chemical is to be estimated by takig a series of measuremets o radomly chose small quatities of the chemical ad usig the sample mea to estimate the true percetage. From previous experiece idividual measuremets of this type are kow to have a stadard deviatio of.0%. How may measuremets must be made so that the stadard error of the estimate is less tha 0.%? If the sample mea w of 45 measuremets is foud to be.9%, give a 95% cofidece iterval for the true percetage, ω. Assume that measuremets are made. The stadard error of the sample mea is (/ )%. For the required precisio require / <0., i.e. >(/0.) =4/0.09 = 44.4. Sice must be a iteger, at least 45 measuremets are ecessary for required precisio. With a sample of 45 measuremets, you ca use the cetral limit theorem ad take the sample mea percetage W to be distributed ormally with mea ω ad stadard error / 45. Hece, if ω is the true percetage, it follows that Z = W ω / is distributed as N(0, ). Sice 95% of the area uder the stadard 45 ormal curve lies betwee Z =.96 ad Z =.96, ( P.96 W ω ) / 45.96 =0.95. Re-arragig, we obtai P ( ) W.96 ω W +.96 =0.95. 45 45 Hece, the 95% cofidece iterval for the true percetage is (.9.96(0.98),.9 +.96(0.98)) = (.,.49). To complete this sectio we defie the sample variace. Def. Give a sample of observatios X,X,..., X the sample variace, s,isgiveby s = ( Xi X ),where X deotes the sample mea. i= I our discussio of cofidece itervals for the mea it was assumed that the populatio variace σ X was kow. What happes whe this it is ot kow? For samples of size >0, a good estimate of σ X is obtaied by calculatig the sample variace s ad usig this value. (For small samples, <0, eed to use the t-distributio ot cosidered i this module). 6. Hypothesis testig A assumptio made about a populatio is called a statistical hypothesis. From iformatio cotaied i a radom sample we try to decide whether or ot the hypothesis is true: if evidece from the sample is icosistet with the hypothesis, the hypothesis is rejected; if the evidece is cosistet with the hypothesis, the the hypothesis is accepted. The hypothesis beig tested is called the ull hypothesis (usually deoted by H 0 ) it either specifies a particular value of the populatio parameter or specifies that two or more parameters are equal. A cotrary assumptio is called the alterative hypothesis (usually deoted by H ) ormally specifies a rage of values for the parameter. A commo example of the ull hypothesis is H 0 : µ X = µ 0. The three alterative hypotheses are (i) H : µ X >µ 0, (ii) H : µ X <µ 0, (iii) H : µ X µ 0. Types (i) ad (ii) are said to be oe-sided (or oe-tailed, see figure 6b) type (iii) is two-sided (or two-tailed, see figure 6a). The result of a test is a decisio to choose H 0 or H. This decisio is subject to ucertaity, ad two types of error are possible: 7
(i) a type I error occurs whe we reject H 0 o the basis of the test although it happes to be true the probability of this happeig is called the level of sigificace of the test ad this is prescribed before testig most commoly chose values are 5% or %. (ii) a type II error occurs whe you accept the ull hypothesis o the basis of the test although it happes to be false. The above ideas are ow applied to determie whether or ot the mea, X, of a sample is cosistet with a specified populatio mea µ 0. The ull hypothesis is H 0 : µ X = µ 0 ad a suitable statistic to use is Z = X µ 0 /, where is the stadard deviatio of the populatio ad is the size of the sample. Fid the rage of values of Z for which the ull hypothesis would be accepted kow as acceptace for the test depeds o the pre-determied sigificace level ad the choice of H. Correspodig rage of values of Z for which H 0 is rejected (i.e. ot accepted) is called the rejectio. Ex 0. A stadard process produces yar with mea breakig stregth 5.8 kg ad stadard deviatio.9 kg. A modificatio is itroduced ad a sample of 0 legths of yar produced by the ew process is tested to see if the breakig stregth has chaged. The sample mea breakig stregth is 6.5 kg. Assumig the stadard deviatio is uchaged, is it correct to say that there is o chage i the mea breakig stregth? Here H 0 : µ X = µ 0, H : µ X µ 0, where µ 0 =5.8adµ X is the mea breakig stregth for the ew process. If H 0 is true (i.e. µ X = µ 0 ), the Z = X µ 0 / has approximately the N(0, ) distributio, where X is the mea breakig stregth of the 0 sample values ad =0. At the 5% sigificace level there is a rejectio of.5% i each tail, as show i figure 6a (sice, uder H 0, P (Z <.96) = P (Z >.96) = P (Z.96) = Φ(.96) = 0.05, i.e..5%). This is a example of a two-sided test leadig to a two-tailed rejectio. rejectio acceptace rejectio % -.96 0.96 % Figure 6a The test is therefore: accept H 0 if.96 Z.96, otherwise reject. 6.5 5.8 From the data, Z =.9/ =.08. Hece, H 0 is rejected at the 5% sigificace level: i.e. the 0 evidece suggests that there IS a chage i the mea breakig stregth. Let us ow cosider a slightly differetly worded questio. Suppose the modificatio was specifically desiged so as to icrease the stregth of the yar. I this case H 0 : µ X = µ 0, H : µ X >µ 0, ad H 0 is rejected if the value of Z is ureasoably large. I this situatio the test is oe-sided ad acceptace ad (oe-tailed) rejectio s at the 5% sigificace level are show below. 8
acceptace 0 Figure 6b rejectio.64 5% At 5% sigificace level, test is accept H 0 if Z.64, otherwise reject. From earlier work Z =.08 ad agai the ull hypothesis is rejected. [Compare the two diagrams above, which illustrate the statemet that the rejectio for a test depeds o the form of both the alterative hypothesis ad the sigificace level.] rec/0ls 9