CONFIDENCE INTERVALS

CONFIDENCE INTERVALS CONFIDENCE INTERVALS Documets prepared for use i course B01.1305, New York Uiversity, Ster School of Busiess The otio of statistical iferece page 3 This sectio describes the tasks of statistical iferece. Simple estimatio is oe form of iferece, ad cofidece itervals are aother. The derivatio of the cofidece iterval page 5 This shows how we get the iterval for the populatio mea, assumig a ormal populatio with kow stadard deviatio. This situatio is ot realistic, but it does a ice job of layig out the algebra. Discussio of cofidece itervals ad examples page 7 This gives some basic backgroud ad the uses illustratios of cofidece itervals for a ormal populatio mea ad for a biomial proportio. Some examples page 13 Here are illustratios of itervals for a ormal populatio mea ad for a biomial proportio. Cofidece itervals obtaied through Miitab page 14 Miitab ca prepare a cofidece iterval for ay colum of a worksheet (spreadsheet). Miitab also has a special provisio for computig cofidece itervals directly from x ad s or, i the biomial case, from p. More details o biomial cofidece itervals page 16 revisio date 18 NOV 005 Gary Simo, 005 Cover photo: IBM 79 tape drive, Computer Museum, Moutai View, Califoria. 1

CONFIDENCE INTERVALS

THE NOTION OF STATISTICAL INFERENCE A statistical iferece is a quatifiable statemet about either a populatio parameter or a future radom variable. There are may varieties of statistical iferece, but we will focus o just four of them: parameter estimatio, cofidece itervals, hypothesis tests, ad predictios. Parameter estimatio is coceptually the simplest. Estimatio is doe by givig a sigle umber which represets a guess at a ukow populatio parameter. If X 1, X,, X is a sample of values from a populatio with ukow mea µ, the we might cosider usig X as a estimate of µ. We would write µ = X. This is ot the oly estimate of µ, but it makes a lot of sese. A cofidece iterval is a iterval which has a specified probability of cotaiig a ukow populatio parameter. If X 1, X,, X is a sample of values from a populatio which is assumed to be ormal ad which has a ukow mea µ, the a 1 - α cofidece iterval for µ s is X ± t α/; 1. Here t α/;-1 is a poit from the t table. Oce the data leads to actual umbers, you ll make a statemet of the form I m 95% cofidet that the value of µ lies betwee 484.6 ad 530.8. A hypothesis test is a yes-o decisio about a ukow populatio parameter. There is cosiderable formalism, itese otatio, ad jargo associated with hypothesis testig. If X 1, X,, X is a sample of values from a populatio which is assumed to be ormal ad which has a ukow mea µ, we might cosider the ull hypothesis H 0 : µ = µ 0 versus alterative H 1 : µ µ 0. The symbol µ 0 is a specified compariso value, ad it will be a umber i ay applicatio. Based o data, we will decide either to accept H 0 or to reject H 0. We work with a level of sigificace (usually oted as α ad early always 0.05) such that the probability of rejectig H 0 whe it is really true is limited to the level of sigificace. I the situatio illustrated here, suppose that = 4, H 0 : µ = 310, H 1 : µ 310, ad α = 0.05. The H 0 will be rejected if ad oly if t t 0.05;3 =.069. The X 310 symbol t refers to the t statistic, ad t =. s 3

THE NOTION OF STATISTICAL INFERENCE Predictios are guesses about values of future radom variables. We ca subdivide this otio ito poit predictios ad iterval predictios, but poit predictios are usually obvious. If X 1, X,, X is a sample of values from a populatio which is assumed to be ormal ad which has a ukow mea µ, we might wish to predict the ext value X +1. Implicit i this discussio is that we have observed X 1 through X, but we have ot yet observed X +1. The poit predictio is certaily X ; i fact, we would write X +1 = X. The 1 - α predictio iterval works out to be X ± t /; 1 s α + 1 1. 4

THE DERIVATION OF THE CONFIDENCE INTERVAL The ormal table gives us the fact that P[ -1.96 < Z < 1.96 ] = 0.95. With a sample of values from a populatio with mea µ ad stadard deviatio σ, the Cetral Limit theorem gives us the result that Z = X µ = X µ is approximately σ σ ormally distributed with mea 0 ad with stadard deviatio 1. If the populatio is assumed to be exactly ormal to start with, the X µ σ automatically ormally distributed. (This is ot a use of the Cetral Limit theorem.) is If oe does ot make the assumptio that the populatio is exactly ormal to start with, the X µ is approximately ormal, provided is large eough. (This σ is precisely the Cetral Limit theorem.) The official stadard is that should be at least 30. The result works well for as small as 10, provided that oe is ot workig with probabilities like 0.006 or 0.9974 which are too close to zero or oe. Start from P[ -1.96 < Z < 1.96 ] = 0.95 ad the substitute for Z the expressio X µ σ. This will give us X µ P 1.96 < < 1.96 σ = 0.95 We ca rewrite this as σ σ P 1.96 < X µ < 1.96 = 0.95 Now subtract X from all items to get σ σ P X 1.96 < µ < X + 1.96 = 0.95 Multiply by -1 (which requires reversig iequality directio) to obtai σ σ P X + 1.96 > µ > X 1.96 = 0.95 5

THE DERIVATION OF THE CONFIDENCE INTERVAL This is i the form large > medium > small. Rewrite as small < medium < large to get σ σ P X 1.96 < µ < X + 1.96 = 0.95 This is ow a probability statemet i which µ is i the middle! If we kew σ ad if we observe X, the this is a special kid of probability statemet about µ. We call this a cofidece iterval. We are 95% cofidet that µ is i this iterval. This is a very useful statistical iferetial statemet. We ofte say simply, we are 95% cofidet that σ µ is i the iterval X ± 19. 6. Most cofidece itervals are used with 95% cofidece. To make thigs more geeral, we use the z γ otatio. Specifically, z γ is the upper gamma poit from the stadard ormal distributio. This meas that z γ has γ of the probability to its right; we ca write this as P[ Z > z γ ] = γ. The value 1.96 is the upper.5% poit, i that we would write P[ Z > z 0.05 ] = 0.05. The most commoly ecoutered symbol is z α/. Here are some simple examples. α α/ z α/ 0.10 0.05 1.645 0.05 0.05 1.96 0.01 0.005.575 Whe we wated 95% cofidece, we put.5% of the probability i each of the upper ad lower tails. I geeral, if we wat 1 - α cofidece, we put α of probability i each of the upper ad lower tails. The statemet that creates the 1 - α cofidece iterval is the σ P X z < µ < X + z α/ α/ σ = 1 - α The 1 - α cofidece iterval is the X ± z σ α/. The statemet is hard to use i practice, because it is very rare to be able to claim kowledge of σ but o kowledge of µ. There is a simple ext step to cover this shortcomig. 6

DISCUSSION OF CONFIDENCE INTERVALS AND EXAMPLES A cofidece iterval is a statemet used to trap ukow populatio parameters. A example... We are 95% cofidet that the September mea sales per store is betwee 46 thousad dollars ad 8 thousad dollars. The speaker is tryig to estimate µ, the true-but-ukow populatio mea sales per store. Presumably he or she has used a sample to make the statemet. The statemet is either true or false (ad we do t kow which), but it would be reasoable to give 95-to-5 bettig odds that the statemet is true. The simplest approximate cofidece iterval statemet is this: We are approximately 95% cofidet that the ukow parameter is i the iterval [estimate] ± [stadard error]. This is approximate for a umber of reasos. First of all, we are tryig to make a all-purpose statemet that really might ot apply perfectly to everythig. Also, the should really be 1.96, which has the property that P[ -1.96 Z 1.96 ] = 0.95 for a stadard ormal radom variable Z. Oe ca refie the statistical theory a bit, tuig up ay or all of choice for the estimate techique for obtaiig the stadard error refiig the multiplier Let s give ext what may be the most-commoly used cofidece iterval. Suppose that you have a sample X 1, X, X 3, X 4,, X. As usual, we will use the upper case letters whe we thik of the X i s as radom variables ad x 1, x, x 3, x 4,, x whe we thik of these as observed umerical values. I practice, this distictio will be hard to eforce. The procedure which we will use depeds o the assumptio that the X i values costitute a sample from a populatio which follows the ormal distributio. As usual, statig a assumptio does ot make it true. Noetheless, it is importat to state a assumptio that you are goig to exploit, eve if you do ot completely believe the assumptio. It is importat to ivoke the coditios o which the work depeds. The procedure which follows, by the way, happes to work rather well eve if the sampled populatio does ot follow the ormal distributio. Moreover, whe the sample size exceeds 30, the ormal distributio assumptio is ot critical. Let s use µ for the mea of the populatio ad use σ for the stadard deviatio of the populatio. I all realistic problems, the values of µ ad σ are both ukow. The statistical work will be based o the sample mea X ad the sample stadard deviatio s. 7

DISCUSSION OF CONFIDENCE INTERVALS AND EXAMPLES Clearly X, the sample mea, will be used as the estimate for µ. Sice SD( X ) = σ, s we ll use SE( X ) = for the stadard error. As a cosequece of the fact that X µ s = X µ follows the t distributio with - 1 degrees of freedom, we ca give the s 95% cofidece iterval for µ as X ± t s 0.05; 1. The value t 0.05;-1 refers to the poit from the t-table. The t-table is Table 4 of Hildebrad, Ott, ad Gray, ad it appears o page 79 ad o the iside back cover. The 0.05 idicates the colum to be used. The - 1 refers to the degrees of freedom. Thus, if = 17, the - 1 = 16 ad we look up t 0.05;16 =.10. The 0.05 i the subscript refers to probability excluded to the right. If T -1 refers to the t radom variable with - 1 degrees of freedom, the the mathematical descriptio of this is P[ T -1 > t 0.05;-1 ] = 0.05. Said aother way, t 0.05;-1 is the upper.5 percet poit for the t distributio with - 1 degrees of freedom. We use the 0.05 for 95% cofidece itervals because we exclude the most extreme 5% of the distributio, meaig.5% i each tail. By established custom, the t table gives probabilities for oe tail. The use of 95% cofidece itervals is most commo, but there are occasios whe we eed 90% or 99% itervals. For a 99% iterval, oe would use t 0.005;-1. I geeral, if oe seeks cofidece level 1 - α, the correspodig poit from the t table is t α/; -1. Some people like to express the cofidece as a percet, rather tha as a decimal. That is, they d rather talk about a 100(1 - α)% cofidece iterval tha a 1 - α cofidece iterval. This is a exceedigly petty issue. Hildebrad, Ott, ad Gray use the symbol t α/ where we have used t α/; -1. Apparetly they believe that the degrees of freedom umber is obvious i most cases. They are correct, but we ll use the more detailed subscript ayhow. Sice 95% cofidece itervals are the most commo, we usually use the 0.05 colum i Hildebrad, Ott, ad Gray s Table 4. It should be oted that oce gets to be as big as about 15, the the values i this colum are rather close to. This reflects the empirical rule that most quatities are withi two stadard deviatios of their expected values with probability about 95%. 8

DISCUSSION OF CONFIDENCE INTERVALS AND EXAMPLES EXAMPLE: A sample of 0 five-poud potato bags, sampled from a huge shipmet of such bags, had a mea weight of 5.1 pouds, with a stadard deviatio of 0.14 poud. Give a 95% cofidece iterval for the true mea weight of all the bags i the shipmet. SOLUTION: Assume that the distributio of bag weights is ormal. The the iterval is s X ± t0.05 ; 1 which is 51 093 014.. ±. 0 ad umerically this becomes 5.1 ± 0.066. This ca also be give as (5.054, 5.186). The value.093 is foud i the row for 19 degrees of freedom i the t table, usig the colum for two-sided 95% itervals (the oe headed 0.05). I describig a series of steps leadig to a cofidece iterval, try to avoid usig the equals sig. That is, do NOT write somethig like 51 093 014.. ±. = 5.1 ± 0.066 = (5.054, 5.186). 0 This iterval is foud as defiitio 7.9 o page 78 of Hildebrad, Ott, ad Gray. This box otes the hierarchy of assumptios that are ivolved. ad if you are willig to assume that the sample comes from a ormal populatio ad if you are ot willig to assume that the sample comes from a ormal populatio If is small ( < 30) use defiitio 7.9 o page 78 you caot give a cofidece iterval of this variety If is large ( 30) use defiitio 7.9 o page 78 use defiitio 7.9 o page 78 Cosider ow a biomial problem. Suppose that X is the umber of successes i idepedet trials, each with success probability p. Let p = X be the sample proportio of successes. (May other authors use π for the success probability ad use π = X for the sample proportio.) The stadard error of p, meaig the estimate of the stadard 9

DISCUSSION OF CONFIDENCE INTERVALS AND EXAMPLES deviatio of p is SE( p ) = ) iterval for π ca be give the as. The covetioal, or Wald, 95% cofidece p ± 1.96 ) If you wished to use somethig other that 95% as the cofidece value, say you wat the cofidece level to be 1 - α, the iterval would be give as p ± z α / ) The value z α/ represets the upper oe-sided α poit from the ormal distributio. For istace, if 1 - α = 90%, the α = 10%, α = 5%, ad we use z0.05 = 1.645. To make the correspodece to the 95% iterval, thik of 1 - α = 95%, α = 5%, α =.5% ad we use z0.05 = 1.96. This is the most commo form of the iterval, but it is ot recommeded. The forms below are better. The otio of better is that they are more likely to come close to the target 1 - α coverage. It would be better to use the slightly loger cotiuity-corrected iterval p ± 1.96 ( ) 1 1 + The use of the fractio 1 is called a cotiuity correctio, the cosequece of usig a (cotiuous) ormal distributio approximatio o a (discrete) biomial radom variable. Alas, most texts do ot use this correctio. The iterval is certaily simpler without this correctio, ad perhaps that is the reaso that most 1 texts avoid it. However, omittig this cheats o the cofidece, i that a claimed 95% iterval might really oly be a 9% iterval. The best (simple) optio, ad the oe recommeded for most cases, is the Agresti-Coull x + iterval. It s based o p =. The correspodig 1 - α cofidece iterval is the + 4 10

DISCUSSION OF CONFIDENCE INTERVALS AND EXAMPLES p ± z α / p p ) + 4 This is computatioally idetical to ivetig four additioal trials, of which two are success. EXAMPLE: I a sample of 00 purchasers of Megabrew Beer there were 84 who purchased the beer specifically for cosumptio durig televised sportig evets. Give a 95% cofidece iterval for the ukow populatio proportio. SOLUTION: Note that p = 84 00 ( ) = 0.4. The stadard error associated with this 1 0.4 05. 8 estimate is SE( p) = = = 0. 0018 0.0349. The 00 covetioal 95% cofidece iterval is the 0.4 ± 1.96 0.0349 or 0.4 ± 0.0684. This ca be writte as (0.3516, 0.4884). This is urealistically short. Miitab will produce this iterval. Use Stat Basic Statistics 1 Proportio... Provide the values 00 ad 84 i the Summarized Data area. Click o Optios ad the check off Use test ad iterval based o ormal distributio. If you ucheck Use test ad iterval based o ormal distributio you will get a elaborate calculatio based o the exact distributio. The method is idetified as Clopper-Pearso. Details are i Distributios i Statistics: Discrete Distributios, by Johso ad Kotz, pages 58-59. The iterval foud by Miitab will be (0.3507, 0.4917). L The cotiuity-corrected form is 0.4 ± 1. 96 0. 0349 NM + 1 400 or 0.4 ± 0.0709, which may be writte as ( 0.3491, 0.4909). O QP The Agresti-Coull form based o p begis by otig that p = 0.416. The the 95% iterval is 84 + 00 + 4 = 86 04 11

DISCUSSION OF CONFIDENCE INTERVALS AND EXAMPLES 0.416 ± 1.96 0.416 0.5784 04 or 0.416 ± 0.0678. This is (0.3538, 0.4894). This is the form that we d recommed. EXAMPLE: A examiatio of 5 fiished televisio sets resulted i 18 beig classified as OK, meaig ready for shipmet to stores. Give a 95% estimate for the correspodig populatio fractio. SOLUTION: First ote that p = 18 = 0.7. With this small, we should really use the 5 iterval based o p. We have p = 0 0.6897, ad the 95% iterval is 9 0.6897 ± 1.96 0.6897 0.3103 9 or 0.6897 ± 0.1695. This is (0.50, 0.859). This is the preferred solutio. By way of compariso, the covetioal iterval is 0.7 ± 1.96 0.0900 which is 0.7 ± 0.1764, meaig (0.5436, 0.8964). The cotiuity-corrected iterval is L NM 0.7 ± 1. 96 0. 0900 + 1 50 or 0.7 ± 0.1964, ad this is (0.536, 0.9164). The exact distributio versio give by Miitab is (0.5061, 0.879). O Q P Oe-sided cofidece itervals are occasioally used i accoutig frameworks i which oe eeds a oe-sided statemet about a value estimated by samplig. Examples of such situatios iclude auditig eviromets i which you eed statemets like We are 95% cofidet that the percetage of ivoices i error is at most 8.0%. 1

SOME EXAMPLES EXAMPLE: A sample of 8 subjects from a cosumer pael reported weekly orage juice cosumptio. These values had a average of 0.4 oz with a stadard deviatio of 1. oz. Give a 95% cofidece iterval for the populatio mea. SOLUTION: To get started...be sure that you ca idetify µ, σ, x, s,. Now assume that the populatio is at least approximately ormal. ASIDE: We kow that the populatio ca t be ormal. The sample mea 0.4 is oly 1.67 estimated stadard deviatios away from zero. There are o data values two stadard deviatios below the mea! With a sample size of 8, we could probably justify use of the Cetral Limit theorem, though 8 is just a bit short of the recommeded 30. The iterval is 0.4 ±.05 1., which is 0.4 ± 4.73. We re 95% cofidet that the 8 value of µ is i the iterval 0.4 ± 4.73, which is (15.67, 5.13). The value.05 is t 0.05;-1, the oe-sided.5% poit (two-sided 5% poit) from the t distributio with - 1 = 7 degrees of freedom. EXAMPLE: Paelists were asked if they added a liquid bleach to their washig. Of the 8, there were 17 who used liquid bleach. Give a 95% cofidece iterval for the populatio proportio. SOLUTION: Before startig...what are p, p, σ,? The Agresti-Coull p versio is recommeded. Note that p = 17 + 8 + 4 p p ) The 95% iterval is p ± zα /, which i this case is + 4 = 19 3 = 0.59375. 0.59375 ± 1.96 0.59375 0.4065 3 This is 0.59375 ± 0.170169. This ca be reasoably give as 0.59 ± 0.17, or (0.4, 0.76). This is much more likely to hit the 95% target cofidece. Just for compariso, the iterval based o p ± is about (0.4, 0.79). z α / ) is 0.6071 ± 0.1809. This 13

CONFIDENCE INTERVALS OBTAINED THROUGH MINITAB oooooooooooooooooooooooooooooooooooo The program Miitab ca do the computatioal work of obtaiig cofidece itervals. Suppose that i a weight loss study, the subjects iitial percets of body fat were oted. Suppose that we d like a 95% cofidece iterval for the mea of the populatio from which these subjects are claimed to represet a sample. I Miitab, just do Stat Basic Statistics 1-Sample t. If we have = 80, x = 31.771, ad s = 5.676, we ca use the Summarized data area to eter the iput. The default cofidece is 95%, but you ca chage it if you wat. The output looks like this: Oe-Sample T N Mea StDev SE Mea 95% CI 80 31.7710 5.6760 0.6346 (30.5079, 33.0341) s The figure 0.635 represets, as may be easily checked. The 95% cofidece iterval is give as (30.5079, 33.0341). This meas that we re 95% sure that the ukow populatio mea µ for the body fat percets is i this iterval. (It should be oted that these subjects were specially recruited by virtue of beig overweight.) If the data were give i a colum of a Miitab worksheet, the that colum would be amed (istead of usig the Summarized data area). You ca also get Miitab to give a cofidece iterval for the differece betwee two meas. I this case, there are two subject groups, idetified as 1 ad accordig to the treatmet regime which they are give. Let s give a 95% cofidece iterval for µ 1 - µ, where the µ s represet the populatio mea drops i body fat over four weeks. For this situatio, give Stat Basic Statistics -Sample t. Geerally you will mark the radio butto for Samples i oe colum. The for Samples: idicate the colum umber (or ame) for the variable to be aalyzed. (The variable ame is drop4 for this example.) Next to Subscripts: give the colum umber (or ame) for the variable which idetifies the groups. (I this example, the actual ame Group is used.) Miitab also allows for the possibility that your two samples appear i two differet colums. For most applicatios, this teds to be a very icoveiet data layout, ad you should probably avoid it. There is a box Assume equal variaces which is uchecked as a default. You should however be very willig to check this box. (More about this below.) The Miitab output is this: o 14

CONFIDENCE INTERVALS OBTAINED THROUGH MINITAB oooooooooooooooooooooooooooooooooooo Two Sample T-Test ad Cofidece Iterval Two sample T for drop4 Group N 33 Mea 0.5 StDev 4.08 SE Mea 0.71 1 31 0.76 3.43 0.6 95% CI for mu () - mu (1): ( -.13, 1.65) T-Test mu () = mu (1) (vs ot =): T= -0.6 P=0.80 DF= 6 Both use Pooled StDev = 3.78 This listig first shows iformatio idividually for the two groups. (There are fewer tha 80 subjects because some did ot make the four-week evaluatio.) You might be amused to kow that the value 0.5 does ot represet 5% i this study; it represets 0.5%. The subjects did ot lose a lot of weight. We have the 95% cofidece iterval for µ - µ 1 as (-.13, 1.65). Of course, the iterval for µ 1 - µ would be (-1.65,.13). The fact that the iterval icludes zero should covice you that the two groups do ot materially differ from each other. You might woder what would have happeed had you ot check off the box Assume equal variaces. Here is that output: Two Sample T-Test ad Cofidece Iterval Two sample T for drop4 Group N Mea StDev SE Mea 33 0.5 4.08 0.71 1 31 0.76 3.43 0.6 95% CI for mu () - mu (1): ( -.1, 1.64) T-Test mu () = mu (1) (vs ot =): T= -0.6 P=0.80 DF= 61 The cofidece iterval is ow give as (-.1, 1.64), obviously ot very differet. The first of these rus, the oe with the iterval (-.13, 1.65), assumed that the two groups were samples from ormal populatios with the same stadard deviatio σ. The estimate of σ is called s p ad here its value is 3.78. Whe the stadard deviatio is assumed equal for the two groups, the iterval is give as x x 1 ± t s α /; 1+ p + 1 1 Whe the Assume equal variaces box is ot checked, the cofidece iterval is s1 s x x 1 ± z α/ + 1 For most situatios, the two versios of the cofidece iterval are very close. o 15

MORE DETAILS ON BINOMIAL CONFIDENCE INTERVALS T T T T T T T T T T T T T T T T T T T T T T T T T T T T T Cosider a biomial experimet, resultig i X evets i trails. Let x be the observed value of the radom variable X. We seek here a 1 - α cofidece iterval for the ukow populatio parameter p. We use ˆp = x as the sample proportio. We have E ˆp = p ad SD( ˆp ) = p(1 p). Sice ˆp is the obvious estimate of p, we have SE( ˆp ) = ) The importat cocept is that sample quatities acquire a probability law of their ow. The stadard error of the mea is critical here. The covetioal (Wald) cofidece iterval for the biomial proportio is ˆp ± z α / ) [Wald; ot recommeded] This covetioal iterval is give i most textbooks, but it is uacceptably overreachig. That is, its coverage probability teds to be less that the claimed 1 - α. A repair that you ll sometimes see replaces z α/ with t α/;-1, sice t α/;-1 > z α/ gives a slightly loger iterval ad thus gets closer to the claimed 1 - α. This is misguided because we have lack the statistical theory to use the t distributio. A improved aswer: ˆp ± z α / ( ) 1 1 + [cotiuity-corrected] This uses the biomial-to-ormal cotiuity correctio. This is a little wider that the covetioal iterval, ad it is more hoest i terms of the cofidece. This procedure is OK, but we ca do better. The itervals oted above ca have some aoyig problems. If x = 0, the Wald iterval will be 0 ± 0. If x =, the Wald iterval will be 1 ± 0. These do ot make sese. Either of these itervals ca sometimes have left eds below 0 or right eds above 1. There is a strategy more precise tha usig this cotiuity correctio. The cofidece iterval is based o the approximatio that p p p) T 16

MORE DETAILS ON BINOMIAL CONFIDENCE INTERVALS T T T T T T T T T T T T T T T T T T T T T T T T T T T T T is approximately ormally distributed. This gives the approximate probability statemet p P z < < z p p) α / α / = 1 - α It s importat to realize that ˆp represets the (radom) quatity observed i the data, while p deotes the ukow parameter value. I the covetioal derivatio of the biomial cofidece iterval, the ukow deomiator p ( 1 p) sample estimate ) is replaced by the. This time we ll do somethig differet. Sice the statemet { a < x < a } is equivalet to { x < a }, the iterval above ca be recast as p P < z p p) α / = 1 - α A little rearragemet gives ( p) ) P < zα p p / = 1 - α Oe more step provides ( ˆ ˆ ) α / ( P p pp+ p < z p 1 p = 1 - α We ll collect the terms accordig to powers of p. This gets us to ) ( ) ( ˆ α/ α/) P z p p z p + + + < 0 = 1 - α The expressio i [ ] is a quadratic iequality i p. The correspodig equality ( ) ( ˆ α/ α/) z p p z p + + + 0 = [roots give score iterval] has two roots, call them p lo ad p hi. The iequality holds betwee these roots, ad we would the give the 1 - α cofidece iterval as (p lo, p hi ) [score iterval] I practice, we ca collect umeric values for, iterval umerically. z α /, ad ˆp ad the solve the score T 17

MORE DETAILS ON BINOMIAL CONFIDENCE INTERVALS T T T T T T T T T T T T T T T T T T T T T T T T T T T T T If you wat to see the score iterval directly, it s this: 1 z α / z + + α/ + z α/ ± z ( ˆ) 1 p 1 z + α / α / + zα/ 4 + zα/ The ceter of this iterval, the expressio before the ±, is a weighted average of p ad 1. This ceter ca also be writte as 1 x + zα /, which looks like + zα / umber of successes 1 after creatig / umber of trials z 1 α fake successes ad z α / fake failures. This leads to the Agresti-Coull iterval, oted below. The half-width of the iterval, the expressio after the ±, is a slight adjustmet from z α / ), the half-width of the Wald iterval. You will also see a style based o creatig four artificial observatios, two successes (1s) ad two failures (0s). Let p = x + + 4 The use the covetioal iterval based o p ad the sample size + 4. The iterval is p ± z α / p p ) + 4 [Agresti-Coull; recommeded] This is the Agresti-Coull iterval. If you use cofidece 95% i the score iterval above, you ll use z α/ = z 0.05 = 1.96 ad this produces almost exactly the Agresti-Coull iterval. There are helpful discussios of these itervals i Aalyzig Categorical Data, by Jeffrey Simooff, Spriger Publicatios, 003. See especially pages 57 ad 65-66. Here s a example. The umeric results are summarized i the chart at the ed of this sectio. Suppose that we have a sample of = 400 ad that we observe x = 190 successes. This leads immediately to ˆp = 190 400 = 0.475 T 18

MORE DETAILS ON BINOMIAL CONFIDENCE INTERVALS T T T T T T T T T T T T T T T T T T T T T T T T T T T T T The covetioal (Wald) 95% cofidece iterval is ˆp ± z 0.05 ) 0.475 0.55 ad this computes to 0.475 ± 1.96, or about 0.475 ± 0.048939. This is 400 0.46061 to 0.53939. The 1 cotiuity-corrected form is 0.475 0.55 1 0.475 ± 1.96 +, or about 400 800 0.475 ± 0.050189. This is 0.44811 to 0.55189. We could also use the Agresti-Coull p form. Note that p = 190 + 400 + 4 0.47548. The compute = 19 404 p ± z α / p p ) + 4 0.47584 0.54716 This is 0.47584 ± 1.96, which is about 0.47584 ± 0.048697. 404 This iterval is 0.46587 to 0.53981. We ca also give the score iterval based o the quadratic iequality: ( ) ( ˆ α/ α/) + z p p+ z p + = 0 Usig z α/ = 1.96, = 400, ˆp = 0.475, we get this as 403.8416 p - 383.8416 p + 90.5 = 0 The roots of this quadratic are 383.8416 ± 39.338991987086 807.683 These are about 0.4653 ad 0.53944; thus 0.4653 to 0.53944 is the iterval. T 19

MORE DETAILS ON BINOMIAL CONFIDENCE INTERVALS T T T T T T T T T T T T T T T T T T T T T T T T T T T T T What does Miitab do for this problem? Use Stat Basic Statistics 1 Proportio. If the data appear i a worksheet colum, just eter the colum ame or umber. If you have just x ad, you ca use the Summarized data box. Select Optios, ad click the radio butto for Use test ad iterval based o ormal approximatio. This will produce exactly the covetioal cofidece iterval. For these data, the Miitab versio usig the ormal approximatio gives the iterval 0.4606 to 0.53938. If you ucheck the ormal approximatio butto, you get the Clopper-Pearso iterval based o the exact biomial distributio. This souds like a good idea, but it s a procedure also fraught with cotroversy. For these data, it produces the iterval 0.45155 to 0.5517. These data were x = 190, = 400. At p = 0.45155, we fid exactly P[ X 190 ] = 0.05. At p = 0.5517, we fid exactly P[ X 190 ] = 0.05. Cosider the oe-sided hypothesis test problem H 0 : p = p 0 versus H 1 : p > p 0, ad suppose that sigificace level 1 α = 0.05 is used. With = 400 ad x = 190, the use of a large value for p 0 would lead to acceptig H 0. The smallest value of p 0 at which H 0 could be accepted is 0.45155, the lower ed of the iterval. As for the other ed, cosider the oe-sided hypothesis test problem H 0 : p = p 0 versus H 1 : p < p 0, ad suppose agai that sigificace level 1 α = 0.05 is used. With = 400 ad x = 190, the use of a small value for p 0 would lead to acceptig H 0. The largest value of p 0 at which H 0 could be accepted is 0.5517, the upper ed of the iterval. Thus, the Clopper-Pearso method for a 1 - α cofidece iterval gives the ed poits which are the limits for p 0 at which separate oe-sided ull hypotheses (each at level 1 α) would be accepted. Miitab uses a iterestig modificatio of the Clopper-Pearso method if x = 0 or x =. Suppose that we wat a 95% iterval for p, ad the data give us = 0 ad x = 0. The lower ed of the iterval should certaily be 0.00. Cosider the hypothesis testig problem at level 1 α = 0.05 of H 0: p = p 0 versus H 1 : p > p 0 i which a very, very small value of p 0 (say 10-10 ) appears. There is o way to take advatage of a sigificace level of 0.05, sice for obvious rejectio rule { X 1 } we have P[ X 1 ] = 1 P[ X = 0 ] = 1 (1 p 0 ) 0 0 0 j 0 0 0 3 = 1 ( p0 ) = 1 j= 0 j 1 p0 + p0 p0 +... 1 3 T 0

MORE DETAILS ON BINOMIAL CONFIDENCE INTERVALS T T T T T T T T T T T T T T T T T T T T T T T T T T T T T 0 p 0. This uses the biomial theorem ad the igores miuscule terms ivolvig p, p, ad so o. Miitab the ivokes 0.00 as the lower ed of the iterval, ad uses 0 3 0 the etire α = 0.05 o the upper ed. The iterval is the give as (0.000000, 0.139108), ad the correspodig probability statemet is this: At p = 0.139108, we fid exactly P[ X 0 ] = P[ X = 0 ] = 0.05. Now, let s check this agai for a similar result, obtaied with a smaller sample size. Suppose that we have a sample of = 40 ad that we observe x = 19 successes. This leads to ˆp = 19 40 = 0.475 This is exactly the same ˆp that we had i the larger problem. The covetioal 95% cofidece iterval is ˆp ± z 0.05 ) ad this computes to 0.475 0.55 0.475 ± 1.96, or about 0.475 ± 0.154758. This is 0.304 to 0.69758. 40 1 0.475 0.55 1 The cotiuity-corrected form is 0.475 ± 1.96 +, or about 40 80 0.475 ± 0.16758. This is 0.30774 to 0.6458. We could also use the Agresti-Coull p form. Note that p = 19 + 40 + 4 The we compute p ± z α / p p ) + 4. This is = 1 44 0.47778. 0.47773 0.577 0.47773 ± 1.96, which is about 0.47773 ± 0.147588. This 44 iterval is 0.39685 to 0.64861. T 1

MORE DETAILS ON BINOMIAL CONFIDENCE INTERVALS T T T T T T T T T T T T T T T T T T T T T T T T T T T T T We could also give the score iterval based o the quadratic iequality. Usig z α/ = 1.96, = 40, ˆp = 0.475, we get this as 43.8416 p - 41.8416 p + 9.05 = 0. The roots are about 0.3516 ad 0.647838; thus 0.3516 to 0.647838 is the iterval. The Miitab versio, usig the ormal approximatio is 0.3045 to 0.69755. Without the ormal approximatio it is 0.31510 to 0.63870. Here is a summary table: Method = 400, x = 190 = 40, x = 19 Low Ed High Ed Low Ed High Ed Covetioal (Wald) 0.46061 0.53939 0.304 0.69758 Miitab with ormal approximatio 0.4606 0.53938 0.3045 0.69755 1 correctio 0.44811 0.55189 0.30774 0.6458 Agresti-Coull p method 0.46587 0.53981 0.39685 0.64861 Score iterval 0.4653 0.53944 0.3516 0.647838 Miitab without ormal approximatio 0.45155 0.5517 0.31510 0.63870 For the = 400 problem with ˆp = 0.475, the differeces amog the methods are trivial. For the much smaller problem with = 40 ad ˆp = 0.475, the differeces are material. Please avoid the covetioal ad Miitab with ormal approximatio methods. T