One-Way ANOVA Calculations: In-Class Exercise Psychology 311 Spring, 2013 1. You are planning an experiment that will involve 4 equally sized groups, including 3 experimental groups and a control. Each group will contain n observations. Your expectation is that each of the 3 experimental treatments will have approximately the same effect, and that this effect will be small roughly one-third a standard deviation improved performance over the control. (a) Calculate the 4 effects α j = µ j µ. Note that you will only be able to express them in standard deviation units. (Hint. At first this may seem impossible, but recall that if the effects must sum to zero, the knowledge that 3 group means all differ from the control by σ/3 allows you to define the 4th effect so that the 4 effects sum to zero.) (b) Once you have the α j values, you should immediately be able to specify the standardized effect values αj = α j/σ. What are they? (c) Suppose that n = 10 per group. Recall from lecture that the F statistic for testing the null hypothesis of equal group means has a general distribution that is noncentral F with a 1 and a(n 1) degrees of freedom and a noncentrality parameter λ given by λ = n a What is the value of λ in this case? (d) If the F test is to be conducted with α = 0.05, what is the critical value (i.e., rejection point)? When calculating the rejection point H 0 is true and λ = 0. (e) What is the power of the test with the proposed value of n = 10? Use R to perform the calculation, then verify it with Gpower 3. α 2 j (f) How large an n would you need to obtain a power of.90? 1
2. Suppose that you run the above experiment, and obtain the data shown below. Control Exp1 Exp2 Exp3 1 118 107 133 134 2 121 165 154 176 3 97 121 91 171 4 86 126 63 159 5 118 87 62 118 6 45 135 164 125 7 119 83 96 100 8 92 100 129 60 9 91 144 128 163 10 72 119 105 111 (a) Perform a 1-way ANOVA on the data. (b) Given the result, compute a 95% confidence interval on the noncentrality parameter λ. Hint. You can use the MBESS routine conf.limits.ncf or my noncentral distribution calculator NDC. (c) Cohen s f can be written as a f = (α j/σ) 2 a Examine the formula for λ, and note that f can be written as a monotonic, strictly increasing function of λ. Derive a formula for converting λ to f, and use it to compute a 95% confidence interval on f. 2
Key to One-Way ANOVA Calculations: In-Class Exercise Psychology 311 Spring, 2013 1. You are planning an experiment that will involve 4 equally sized groups, including 3 experimental groups and a control. Each group will contain n observations. Your expectation is that each of the 3 experimental treatments will have approximately the same effect, and that this effect will be small roughly one-third a standard deviation improved performance over the control. (a) Calculate the 4 effects α j = µ j µ. Note that you will only be able to express them in standard deviation units. (Hint. At first this may seem impossible, but recall that if the effects must sum to zero, the knowledge that 3 group means all differ from the control by σ/3 allows you to define the 4th effect so that the 4 effects sum to zero.) Answer. The group means can be written in terms of their relationship to each other as 0, σ/3, σ/3, σ/3. To express them as effects α j, you have to transform them by subtracting a constant so that their mean is zero. Since the current mean is σ/4, we need to subtract σ/4 from each one. The effects then become σ/4, σ/12, σ/12, σ/12. (b) Once you have the α j values, you should immediately be able to specify the standardized effect values αj = α j/σ. What are they? Answer. We simply divide them all by σ, obtaining 1/4, 1/12, 1/12, 1/12. (c) Suppose that n = 10 per group. Recall from lecture that the F statistic for testing the null hypothesis of equal group means has a general distribution that is noncentral F with a 1 and a(n 1) degrees of freedom and a noncentrality parameter λ given by λ = n a (α j /σ) 2 = n What is the value of λ in this case? Answer. The value of λ is a α 2 j 10(1/16 + 1/144 + 1/144 + 1/144) = 10(12/144) = 5/6 = 0.8333 (d) If the F test is to be conducted with α = 0.05, what is the critical value (i.e., rejection point)? When calculating the rejection point H 0 is true and λ = 0. Answer. The F statistic has 3 and 36 degrees of freedom, and the critical value is 1
> F.crit <- qf(.95,3,36) > F.crit [1] 2.866266 (e) What is the power of the test with the proposed value of n = 10? Use R to perform the calculation, then verify it with Gpower 3. Answer. Power is the area to the right of the rejection point we determined above in a noncentral F distribution with 3 and 36 degrees of freedom and λ = 0.8333. > lambda <- 5/6 > Power <- 1 - pf(f.crit,3,36,lambda) > Power [1] 0.09797938 Power is a pathetic 0.098. We can duplicate those calculations in Gpower 3 as shown in the following screen. Note how we open a side menu. To keep things simple, I defined σ as 3 and chose the means to be 0,1,1,1 yielding the same standardized effects as we calculated above. (f) How large an n would you need to obtain a power of.90? Answer. A total sample size of 688 for the 4 groups, or an n of 172 per group, is required. 2
3
2. Suppose that you run the above experiment, and obtain the data shown below. Control Exp1 Exp2 Exp3 1 118 107 133 134 2 121 165 154 176 3 97 121 91 171 4 86 126 63 159 5 118 87 62 118 6 45 135 164 125 7 119 83 96 100 8 92 100 129 60 9 91 144 128 163 10 72 119 105 111 (a) Perform a 1-way ANOVA on the data. Answer. Typing in the data yields a data frame with 40 rows and 2 columns. The first and last few lines are shown below with the head and tail commands. > head(data) Y Group 1 118 Control 2 121 Control 3 97 Control 4 86 Control 5 118 Control 6 45 Control > tail(data) Y Group 35 118 Exp3 36 125 Exp3 37 100 Exp3 38 60 Exp3 39 163 Exp3 40 111 Exp3 To analyze the data, one standard ANOVA approach is as follows. > results <- anova(lm(y ~ factor(group))) > xtable(results) Note that, in the above code, I use the function xtable to produce typeset tables within L A TEX. The standard R output shown below would be produced simply by typing results. 4
Df Sum Sq Mean Sq F value Pr(>F) factor(group) 3 6632.80 2210.93 2.31 0.0927 Residuals 36 34457.60 957.16 > results Analysis of Variance Table Response: Y Df Sum Sq Mean Sq F value Pr(>F) factor(group) 3 6633 2210.93 2.3099 0.09274. Residuals 36 34458 957.16 --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 (b) Given the result, compute a 95% confidence interval on the noncentrality parameter λ. Hint. You can use the MBESS routine conf.limits.ncf or my noncentral distribution calculator NDC. Answer. Here is MBESS output > library(mbess) > lambda.ci <- conf.limits.ncf(f.value = 2.3099, df.1 = 3, df.2 = 36 ) > lambda.ci $Lower.Limit [1] NA $Prob.Less.Lower [1] NA $Upper.Limit [1] 19.17194 $Prob.Greater.Upper [1] 0.025 Note that it returns a missing value code NA for the lower value. Since the lower limit cannot be lower than zero, I chose in my routine to return a value of zero. Here is the NDC screen. 5
(c) Cohen s f can be written as a f = (α j/σ) 2 a Examine the formula for λ, and note that f can be written as a monotonic, strictly increasing function of λ. Derive a formula for converting λ to f, and use it to compute a 95% confidence interval on f. Answer. We can write λ f = na > n <- 10; a <- 4 > upper <- sqrt(lambda.ci$upper.limit/(n*a)) > upper [1] 0.6923138 So our confidence interval for f has limits of 0 and 0.6923. 6