Math 141. Lecture 16: More than one group. Albyn Jones 1. jones/courses/ Library 304. Albyn Jones Math 141

Transcription

1 Math 141 Lecture 16: More than one group Albyn Jones 1 1 Library 304 jones@reed.edu jones/courses/141

2 Comparing two population means If two distributions have the same shape and spread, then the only difference between the two is location: the difference between their means µ x µ y. Two populattions differing only in location density x

3 The Generic t-test Every t-test has the form t df = ˆθ θ 0 SE(ˆθ) where ˆθ is an estimate for some parameter θ, θ 0 is the value specified by the null hypothesis, and the df come from the divisor used in computing SE(ˆθ).

4 The Two Sample t-test To test the null hypothesis that two populations have the same mean H 0 : µ X = µ Y the parameter of interest is the difference between the means θ = µ X µ Y and we need to know the SE of the estimator X Y.

5 The Variance of a Difference Suppose that X 1, X 2,... X n and Y 1, Y 2,... Y m are two independent random samples. Then E(X Y ) = µ x µ y and or Var(X Y ) = Var(X) + Var(Y ) Var(X Y ) = σ2 x n + σ2 y m

6 The Variance of a Difference, Continued If Var(X Y ) = σ2 x n + σ2 y m and both groups have the same variance, then σ 2 x = σ 2 y = σ 2 and σx 2 ( n + σ2 y 1 m = σ2 n + 1 ) m

7 The SE of a difference Suppose that X 1, X 2,... X n and Y 1, Y 2,... Y m are random samples from two populations with the same variance, so σ 2 x = σ 2 y = σ 2. Then we can estimate the common variance σ 2 with a pooled sum of squared deviations: s 2 p = (Xi X) 2 + (Y i Y ) 2. n + m 2 In other words, a weighted average of the two sample variances: sp 2 = (n 1)s2 x + (m 1)sy 2. n + m 2

8 Finally, a t-test! Suppose that X 1, X 2,... X n and Y 1, Y 2,... Y m are two independent random samples from populations with with the same variance. Then the t-statistic is t n+m 2 = X Y s p 1/n + 1/m with n + m 2 degrees of freedom. The 95% CI is (X Y ) ± t s p 1/n + 1/m where t = qt(.975, n + m 2)

9 Model Comparisons: H 0 The Null Hypothesis corresponds to the model X i = µ + ɛ i and Y j = µ + ɛ j where the error terms satisfy ɛ IID N(0, σ 2 )

10 Model Comparisons,H 1 The Alternative Hypothesis corresponds to the model X i = µ x + ɛ i and Y j = µ y + ɛ j where µ x µ y and the error terms satisfy ɛ IID N(0, σ 2 )

11 Anorexia data Anorexia Data Control Family Cog/Behav Y stripchart(y Therapy, data=anorexia, pch=19, cex=.5, col= red )

12 Anorexia data density.default(x = Control) Density Control Family Cog/Behav N = 26 Bandwidth = 3.711

13 Two Sample t-test, Anorexia Therapies > t.test(cogb,control,var.equal=true) Two Sample t-test data: CogB and Control t = , df = 52, p-value = alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: sample estimates: mean of x mean of y

14 Welch Test, Anorexia Therapies > t.test(cogb,control) Welch Two Sample t-test data: CogB and Control t = , df = , p-value = alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: sample estimates: mean of x mean of y

15 Should we Trust the t-test here? Cognitive/Behavioral Theoretical Quantiles Sample Quantiles Control Theoretical Quantiles Sample Quantiles

16 Summary: Two Sample T Test ˆθ θ 0 SE(ˆθ) = t n+m 2 = X Y s p 1/n + 1/m Generic Two Sample Estimates θ µ X µ Y X Y SE(ˆθ) SE(X Y ) s p 1/n + 1/m

17 Paired Samples Also: repeated measures, within subject designs, split-plot... We have two measurements for each case: before vs after treatment, pre-test and post-test scores, subjects measured under two different conditions, etc., where the measurements are positively correlated. This is just a one-sample t-test on the difference scores: H 0 : µ x µ y = 0 is the same as H 0 : µ X Y = 0

18 Paired Sample Example: Control Group Subj Before After Diff

19 Paired Sample Example: Continued > t.test(after[1:26],before[1:26],paired=true) Paired t-test data: After[1:26] and Before[1:26] t = , df = 25, p-value = > t.test(after[1:26]-before[1:26]) One Sample t-test data: After[1:26] - Before[1:26] t = , df = 25, p-value =

20 Paired Samples, Within Subjects, Repeated Measures Before After Control Family CogBehav Anorexia Data: Before and After

21 Paired Samples, Within Subjects, Repeated Measures :71 Before Before and After Pairs

22 Testing Several Group Means for Equality There is an omnibus test for two or more groups, called Analysis of Variance or ANOVA. We will study this test in detail later. Suppose we have g groups, the null hypothesis is that all group means are equal: H 0 : µ 1 = µ 2 =... = µ g The alternative: not all are equal.

23 Anova: Anorexia Therapies > summary(aov(y Therapy,data=Anorexia)) Df Sum Sq Mean Sq F value Pr(>F) Therapy Residuals Conclusion: not all groups have the same mean. Question: which are different?

24 Graphical Anova: Anorexia Therapies Anorexia data Control Family Cog/Behav

25 Post-hoc Comparisons There are various methods, like the Bonferroni correction for multiple comparisons. Probably the best after doing an anova is Tukey s Honest Significant Difference method. > Anorexia.aov <- aov(y Therapy,data=Anorexia) > TukeyHSD(Anorexia.aov) Tukey multiple comparisons of means 95% family-wise confidence level Fit: aov(formula = Y Therapy, data = Anorexia) $Therapy diff lwr upr p adj Family-Control Cog/Behav-Control Cog/Behav-Family

26 Multiple Comparisons and False Discoveries Multiple comparisons and the Scientific Method!

27 Two Samples: Anova vs t-test > summary(aov(y Therapy,data=Anorexia, subset = (Therapy!= "Family"))) Df Sum Sq Mean Sq F value Pr(>F) Therapy Residuals > t.test(y Therapy,data=Anorexia, subset = (Therapy!= "Family"),var.equal=TRUE) data: Y by Therapy t = , df = 52, p-value = > ( )ˆ2 # the square of t is F [1]

28 Anova as model Comparison Analysis of Variance is really a method of comparing two models: in this case Y ij = µ + ɛ ij versus Y ij = µ i + ɛ ij. In other words, all groups have the same mean µ versus each group has it s own mean µ i. As with the t-test, the validity of anova depends on the distribution of the deviations from the mean(s): ɛ ij N(µ i, σ 2 )

29 Summary two sample t-test: independent samples, normally distributed data, constant variance. paired sample t-test: positively correlated samples, normally distributed data Analysis of Variance: two or more independent samples, normally distributed data, constant variance.