Power and Type II Error

Transcription

1 Statistical Methods I (EXST 7005) Page 57 Power ad Type II Error Sice we do't actually kow the value of the true mea (or we would't be hypothesizig somethig else), we caot kow i practice the type II error rate (). However, it is affected by a umber of thigs, ad we ca kow about these. 1) Power is affected by the distace betwee the hypothesized mea ( 0 ) ad true mea (). The Power Curve 1 Power 0 Differece betwee true ad hypothesized mea 2) Power is affected by the value chose for Type I error ()

2 Statistical Methods I (EXST 7005) Page 58 3) Power is affected by the variability or spread of the distributio Ifluecig the power of a test of hypothesis The capability of the test to reject H 0 whe it is false is called Power = 1. Aythig doe to ehace this value will improve your ability to test for differeces amog populatios. Which of the 3 factors ifluecig power ca you cotrol? For testig meas you may be able to cotrol sample size (). This reduces the variability ad icreases power. ou probably caot ifluece the differece betwee ad 0. ou ca choose ay value of. However, this caot be too small or Type II error becomes more likely. Too large ad Type I error becomes likely. Methods of icreasig the power of a test How would we use our kowledge of factors affectig power to icrease the power of our tests of hypothesis? Icrease the sigificace level (e.g. from = 0.01 to = 0.05) If H 0 is true we would icrease, the probability of a Type I error. If H 0 is false the we decrease, the probability of a Type II error, ad by decreasig, we are icreasig the POWER of test. For a give, the POWER ca be icreased by... 2 Icreasig, so decreases, ad the amout of overlap betwee the real ad hypothesized distributios decreases. For example, let s suppose we are coductig a test of the hypothesis H 0 : = 0 agaist the alterative H 1 : 0. We believe 0 = 50 ad we set = We also kow that 2 = 100 ad that = 25. From this iformatio we ca calculate The critical regio i 5 terms of Z is the P( Z Z 0 ) = 0.05 ad Z 0 = 1.96, ad the critical value o the origial scale variable scale is i = + Z i = (2) = If the REAL populatio mea is 54, calculate P( 53.92), give that the TRUE mea is 54 we calculate the Z value as Z = ( )/2 = 0.08 / 2 = 0.04.

3 Statistical Methods I (EXST 7005) Page 59 Summary The probability of a TPE II error () is the probability of ot drawig a sample that falls above this value ad ot rejectig the false ull hypothesis. The value is = P(Z 0.04) = So for a experimet with = 25, the power is 1 = = But suppose we had a larger sample, say = 100. Now The critical 10 regio stays at Z 0 = 1.96, but o the origial scale this is ow i = + Z i = (1) = For a true mea of 54 we ow get Z = ( )/1 = 2.04/1 = The value of = is P(Z 2.04) = , ad the power for this test is 1 = The bottom lie, With = 25, the power is With = 100, the power is This is why statisticias recommed larger sample sizes so strogly. We may ever really kow what power is, but we kow how to icrease it ad reduce the probability of TPE II error. Hypothesis testig is proe to two types of errors, oe we cotrol () ad oe we do ot (). Type I error is the REJECTION of a true ull hypothesis. Type II error is the FAILURE TO REJECT a ull hypothesis that is false. The Power of a test is 1 Not oly do we ot cotrol TPE II error, we probably do ot eve kow its value. However, we ca hopefully reduce this error, ad icrease power, by Cotrollig the distace betwee ad 0 (ot really likely) Selectig a value of that is ot too small (0.05 ad 0.01 are the usual values) Gettig a larger sample size (), this is the factor that is usually uder the most cotrol of the ivestigator. The t-test of hypotheses The t distributio is used the same as Z distributio, except it is used where sigma (),is ukow (or where is used istead of to calculate deviatios). The t distributio is a bell shaped curve, like the Z distributio, but ot the same. The Z distributio is ormal because it has a ormal distributio i the umerator ( i ) ad all other terms i the trasformatio are costat. The t distributio has a ormal distributio i the umerator but the sample variace i the deomiator is aother statistic with a chi square distributio. i ti ; the t distributio applied to idividual observatios S

4 Statistical Methods I (EXST 7005) Page t ; the t distributio used for hypothesis testig S S where; S = the sample stadard deviatio, (calculated usig istead of ) S = the sample stadard error The variace of the t distributio is greater tha that of the Z distributio (except where ), sice S 2 estimates 2, but is ever as good (reliability is less) Z distributio t distributio Mea 0 0 variace 1 1 Characteristics of the t distributio E(t) = 0, the expected value of the t distributio is zero. It is symmetrically distributed about a mea of 0 with t values ragig betwee ± (i.e. t + ) There is a differet t distributio for each degree of freedom (df), sice the distributio chages as the degrees of freedom chage. It has a broader spread for smaller df, ad arrows (approachig the Z distributio) as df icrease. As the df (, gamma) approaches ifiity ( ), the t distributio coverges the Z distributio. For example; Z (o df associated); middle 95% is betwee ± 1.96 t with 1 df; middle 95% is betwee ± t with 10 df; middle 95% is betwee ± t with 30 df; middle 95% is betwee ± t with df; middle 95% is betwee ± 1.96 How does the test for the t distributio differ from the Z distributio? For the Z distributio, sice i is ormally distributed, subtractig a costat ( ) ad dividig by a costat () does ot affect the distributio ad Z is ormal. For the t distributio we also have a ormally distributed i ad we subtract a costat ( ), but we divide by a statistic (S), ot a costat (). This alters the distribtuio so that it is ot quite a ormal distributio. The extra icertaity causes the t distributio to be broader tha the Z distriutio. However, as sample size icreases the value of S approaches ad the t distributio coverges o the Z distributio.

5 Statistical Methods I (EXST 7005) Page 61 Probability distributio tables i geeral The tables we will use will ALL be givig the area i the tail (). However, if you examie a umber of tables from other sources you will fid that this is ot always true. Eve whe it is true, some tables will give the value of as if it were i two tails, ad some as if it were i oe tail. For example, we wat to coduct a two-tailed Z test at the = 0.05 level. We happe to kow that Z = If we look at this value i the Z tables we expect to see a value of 0.025, or 2. But may tables would show the probability for 1.96 as 0.975, ad some as Why the differece? It just depeds o how the tables are preseted. Some of the alteratives are show below. Some tables give cumulative distributio startig at ifiity. ou wat to fid the probability correspodig to 1 2. The value that leaves.025 i the upper tail would be Some tables may start at zero (0.0) ad give the cumulative area from this poit for the upper half of the distributio. This would be less commo. The value that leaves.025 i the upper tail would be Amog the tables like ours, that give the area i the tail, some are called two tailed tables ad some are oe tailed tables. Table value, Oe tailed table. 1- Table value, Two tailed table. 1- Why the extra cofusio at this poit? All our tables will give the area i the tail. The Z tables we used gave the area i oe tail. For a two tailed test you eeded to doubled the probability. For the F tables ad Chi square tables covered later, this area will be a sigle tail as with the Z tables. This is because these distributios are ot symmetric. Traditioally, may t-tables have give the area i TWO TAILS istead of o oe tail. May textbooks have this type of tables. SAS will also usually give two-tailed values for t-tests.

6 Statistical Methods I (EXST 7005) Page 62 The t tables Our tables will have both two-tailed probabilities (top row) ad oe-tailed probabilities (bottom row), so you my use either. The same patters are true for may of the computer programs that you may use to get probabilities. For example i EXCEL If you use the NORMDIST(1.96) fuctio it returs 0.975, oe tail, cumulative from If you eter NORMSINV(0.025) it returs 1.96, the two tailed value If you eter TINV(0.05,9999) it returs 1.96, so it is also two-tailed. The TDIST(1.96,9999,1) fuctio allows you to specify 1 or 2 tails i the fuctio call. My t-tables are created i EXCEL, but pattered after Steel & Torrie, 1980, pg The degrees of freedom, d.f. or, are give o the left side of the table. The probability of radomly selectig a larger value of t is give at the top (ad bottom) of the page. P(t t 0 ) give at the bottom, this is a oe-tailed probability. P( t t 0 ) give at the top, this is a two-tailed probability (ot the absolute value sigs) Each row represets a differet t distributio (with differet d.f.). The Z table had may probabilities, correspodig to Z values of 0.00, 0.01, 0.02, 0.03, etc. About 400 probabilities occurred i the tables we used. They all fit o oe page because the whole Z table was a sigle distributio. The t table has may differet distributios so less iformatio is give about each distributio. If we are goig to give may differet t-distributios o oe page, we lose somethig. We will oly give a few selected probabilities, the oes we are most likely to use. e.g., 0.10, 0.05, 0.025, 0.01, Oly the POSITIVE side of the table is give, but as with the Z distributio, the t distributio is symmetric, so the lower half of the table ca be determied by usig the upper half. Our t-tables Partial t-table 1 or 2 tails? df

7 Statistical Methods I (EXST 7005) Page 63 Note the selected d.f. o the left side. The table stabilizes fairly quickly. May tables do't go over about d.f. = 30. The Z tables give a good approximatio for larger d.f. Our tables will give d.f. as follows dow the left most colum of the table, 1, 2, 3, 4, 5,6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19,20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 34, 36, 38, 40, 45, 50, 75, 100, Selected probabilities I the topmost row of the table selected probabilities will be give as for a TWO TAILED TEST. I the bottom-most row of the table selected probabilities will be give as for a ONE TAILED TEST. Probabilities i our tables are, Top row: Bottom row: HELPFUL HINT: Do't try to memorize two tail top, oe tail bottom, just recall the characteristics of the distributio whe df = the t = This leaves 5% i both tails ad 2.5% i oe tail. So take ay t-table ad look to see what probability correspods to df= ad t = If the value is 0.025, it is the area i oe tail of the distributio ad if it is it is a two tailed table. If the area is it is cumulative from, etc. This trick of recallig 1.96 also works for Z tables. The tables we use give the area i the tail of the distributio, Z = 1.96 correspods to a probability of Some Z tables give the cumulative area uder the curve startig at, the probability at Z = 1.96 would be Other Z tables give the cumulative area startig at 0, the probability at Z = 1.96 would be Workig with our t-tables Example 1. Let d.f. = = 10 H 0 : 0 versus H 1 : 0 ad = 0.05 P( t t 0 ) = 0.05; 2P(t t 0 )=0.05; P(t t 0 )=0.025 (Probabilities at the top of the table) t 0 =2.228 Example 2. Let d.f. = = 10 H 0 : 0 versus H 1 : 0 ad = 0.05 P(t t 0 ) = 0.05 (probabilities at the bottom of the table) t 0 =1.812

8 Statistical Methods I (EXST 7005) Page 64 Look up the followig values. Fid the t value for H 1 : 0, =0.050, d.f. = Fid the t value for H 1 : > 0, = 0.025, d.f. = Fid the t value for H 1 : 0, = 0.010, d.f. = Fid the t value for H 1 : > 0, = 0.025, d.f. = Fid the t value for H 1 : 0, = 0.200, d.f. = Fid the t value for H 1 : 0, = 0.002, d.f. = Fid the t value for H 1 : < 0, = 0.100, d.f. = Fid the t value for H 1 : < 0, = 0.010, d.f. = Fid the P value for t = 1.740, H 1 : < 0, d.f. = Fid the P value for t = 4.587, H 1 : 0, d.f. = t-test of Hypothesis We wat to determie if a ew drug has a effect o blood pressure of rhesus mokeys before ad after treatmet. We are lookig for a et chage i pressure, either up or dow (two-tailed test). Example 1 of the t-test We obtai a radom sample of 10 idividuals. Note: = 10, but d.f. = = 9 1) H 0 : 0 where 0 = 0 2) H 1 : 0 3) Assume: Idepedece (radomly selected sample) ad that the CHANGE i blood pressure is ormally distributed. 4) We set = 0.01, but split betwee two tails (to meet the alterate hypothesis). P( t t 0 ) = 0.01; 2P(t t 0 ) = 0.01; P(t t 0 ) = i each tail The critical value of t is: Give that it is a 2 tailed test, with 9 d.f. ( = 10, but d.f. = = 9) ad we set = 0.01 Uder these coditios, the critical limit from the t-table is t 0 = ) Obtai values from the sample of 10 idividuals ( = 10). The values for chage i blood pressure were; 0, 4, 3, 2, 0, 1, 4, 5, 1, 4 i1 i i1 2 i i 8 i

9 Statistical Methods I (EXST 7005) Page 65 S i 2 2 i 1 ( i ) i 64 2 i1 i ( 1) ( 1) 9 9 S S S Fially, the value of the test statistic, a t value i this case, is t with 9 d.f. S ) Compare the critical limit to the test statistic ad decide to reject or fail to reject. The critical limit from the t-table is t 0 = The test statistic calculated from the sample was (9 d.f.) The area leavig i each tail is almost too small to show o our usual graphs. The test statistic is clearly i the regio of acceptace, so we fail to reject the H 0. 7) Coclude that the ew drug does ot affect the blood pressure of rhesus mokeys. Is there a error? Maybe a Type II error, but ot a Type I error sice we did ot reject the ull hypothesis. Example 2 of the t-test A compay maufacturig evirometal moitorig equipmet claims that their thermograph (a machie that records temperature) requires (o the average) o more tha 0.8 amps to operate uder ormal coditios. We wish to test this claim before buyig their equipmet. We wat to reject the equipmet if the electricity demad exceeds 0.8 amps. 1) H 0 : 0, where 0 = 0.8 2) H 1 : 0 3) Assume (1) idepedece ad (2) a ormal distributio of amp values, or at least of the mea that we will test. We do ot assume a kow variace with the t-test, we use a variace calculated from the sample. 4) We set = The critical value of t for cosiders that we are doig a 1 tailed test (see H 1 :) with 15 d.f. ( = 16, but d.f. = = 15) ad = 0.05 P(t t 0 ) = 0.05 from the table is t 0 = 1.753

10 Statistical Methods I (EXST 7005) Page 66 5) Draw a sample. We have 16 machies for testig. The idividual values for amp readigs were ot recorded. Summary statistics are give below; 0.96 S S t test with SAS 0.32 S t 2.00 with 15 d.f. S ) Compare the critical limit ad to the test statistic. The critical limit from the table is t 0 = ad the calculated test statistic was t = 2 (with 15 d.f.) Clearly, the test statistic exceeds the oe tailed critical limit ad falls i the upper tail of the distributio i area of rejectio. 7) Coclusio: We would coclude that the machies require more electricity tha the claimed 0.8 amperes. Of course, there is a possibility of a Type I error. SAS example (#2a) Recall our test of blood pressure chage of Rhesus mokeys. We ca take the values of blood pressure chage, ad eter them i SAS PROC UNIVARIATE. Values: 0, 4, 3, 2, 0, 1, 4, 5, 1, 4 SAS PROGRAM DATA step t0 = OPTIONS NOCENTER NODATE NONUMBER LS=78 PS=61; TITLE1 't-tests with SAS PROC UNIVARIATE'; DATA mokeys; INFILE CARDS MISSOVER; TITLE2 'Aalysis of Blood Pressure chage i Rhesus Mokeys'; INPUT BPChage; CARDS; RUN; The data would follow the cards statemet edig with a semicolo PROC PRINT DATA=mokeys; RUN; PROC UNIVARIATE DATA=mokeys PLOT; VAR BPChage; TITLE2 'PROC Uivariate o Blood Pressure Chage'; RUN; The PROC UNIVARIATE from SAS will perform a two-sample t-test. See SAS PROGRAM output.