An Old Research Question

Size: px

Start display at page:

Download "An Old Research Question"

Tamsyn Garrison
6 years ago
Views:

1 ANOVA

2 An Old Research Question The impact of TV on high-school grade Watch or not watch Two groups The impact of TV hours on high-school grade Exactly how much TV watching would make difference Multiple groups Not watch, watch a little, watch regularly

3 Then we could have something like this

4 What Should We Do?

5 Should t-test Be Used? Multiple comparison Increasing the chance of Type I error

6 Multiple Comparison Is Common In particular in factorial design Single factor Multiple levels: previous example Multiple factors Impact of TV watching and library visit

7 Terminology Factor The independent variable that designates the groups being compared TV watching and library visit Levels Individual conditions or values that make up a factor Factorial design A study that combines two or more factors

8 Figure 12.2 Two-Factor Research Design The research study uses two factors One factor uses two levels of therapy technique (I versus II) The second factor uses three levels of time (before, after, and 6 months after).

9 Figure 12.2 Two-Factor Research Design

10 Figure 12.2 Two-Factor Research Design Also notice that the therapy factor uses two separate groups (independent measures) and the time factor uses the same group for all three levels (repeated measures). 1 We have 15 comparisons!

11 How to deal with this problem?

12 Analysis of Variance Analysis of variance Also called ANOVA Used to evaluate mean differences between two or more treatments (advantage over t- test) Uses sample data as basis for drawing general conclusions about populations

13 Analysis of Variance Null hypothesis: the level or value on the factor does not affect the dependent variable In the population, this is equivalent to saying that the means of the groups do not differ from each other H : Alternative hypothesis: There is at least one mean difference among the populations All means are different from every other mean Some means are not different from some others, but other means do differ from some means

14 ANOVA: Statistics F test F variance variance (difference) between sample means (difference) expected with no treatment effect (error by chance) t obtained difference between sample means difference expected by chance F-ratio: based on variance instead of sample mean difference Numerator: Variance caused by differences among sample means Denominator: Variance be expected if there is no treatment effect

15 Logic of ANOVA A study with three treatments

16 Sources of Variability Between Treatments Systematic differences caused by treatments Random, unsystematic differences Individual differences Experimental (measurement) error

17 What if the null hypothesis is true?

18 F-Ratio The ratio of the variance between treatments to the variance within treatments (treatment effects + chance) / (chance) If no treatment effect, F should be 1 Otherwise, F should be larger than 1.

19 Experimental Design Simple experiments Single factor Between-subjects design Within-subjects design Factorial experiments More factors 2 x 2 These design all involve multiple treatments ANOVA would be needed.

21 Numerator of F-ratio Denominator of F-ratio Numerator of F-ratio Denominator of F-ratio

22 Logic of Repeated-Measures ANOVA Comparing variance Between-treatments vs. within-treatments Removing the difference between subjects F variance variance between expected treatments by chance F variance variance between expected treatments( without individual differences) by chance ( without individual differences)

23 ANOVA Notation and Formulas

24 k: the number of treatment n: the number of scores in each treatment N: the number of total scores in the study SX or T: the sum of the scores for each treatment G: the sum of all the scores in the study G = S(SX) = ST SX 2, SS, s 2, df,

25 Figure 12.4 ANOVA Calculation Structure and Sequence

26 Figure 12.5 Partitioning SS for Independent-measures ANOVA

27 ANOVA equations 2 G SS total X 2 N treatments SS within SS insideeach treatment SS between treatments T n 2 G N 2

28 Degrees of Freedom Analysis Total degrees of freedom df total = N 1 Within-treatments degrees of freedom df within = N k Between-treatments degrees of freedom df between = k 1

29 Figure 12.6 Partitioning Degrees of Freedom

30 Mean Squares and F-ratio MS between s 2 between SS df between between MS within s 2 within SS df within within F 2 sbetween s 2 within MS MS between within

31 ANOVA Summary Table Concise method for presenting ANOVA results Helps organize and direct the analysis process Convenient for checking computations Standard statistical analysis program output Source SS df MS F Between Treatments Within Treatments Total 60 12

32 Distribution of F-ratios If the null hypothesis is true, the value of F will be around 1.00 Because F-ratios are computed from two variances, they are always positive numbers Table of F values is organized by two df df numerator (between) shown in table columns df denominator (within) shown in table rows

33 Figure 12.7 Distribution of F-ratios

34 ANOVA Test Uses the same four steps that have been used in earlier hypothesis tests. Computation of the test statistic F is done in stages Compute SS total, SS between, SS within Compute MS total, MS between, MS within Compute F

35 Measuring Effect size for ANOVA Compute percentage of variance accounted for by the treatment conditions In published reports of ANOVA, effect size is usually called η 2 ( eta squared ) r 2 concept (proportion of variance explained) 2 SS betweentreatments SS total

36 In the Literature Treatment means and standard deviations are presented in text, table or graph Results of ANOVA are summarized, including F and df p-value η 2 E.g., F(3,20) = 6.45, p<.01, η 2 = 0.492

37 Example For each experiment N = 14

38 Experiment A Source SS df MS F Between Treatments Within Treatments Total

39 Experiment B Source SS df MS F Between Treatments Within Treatments Total

41 post hoc Tests ANOVA compares all individual mean differences simultaneously, in one test A significant F-ratio indicates that at least one difference in means is statistically significant Does not indicate which means differ significantly from each other! post hoc tests are follow up tests done to determine exactly which mean differences are significant, and which are not

42 Tukey s Honestly Significant Difference A single value that determines the minimum difference between treatment means that is necessary to claim statistical significance a difference large enough that p < α experimentwise Honestly Significant Difference (HSD) HSD q MS within n

45 A vs. B: M A M B = 2.44 > HSD significant B vs. C: M B M C = 1.66 < HSD A vs. C: M A M C = 4.00 > HSD significant

46 The Scheffé Test The Scheffé test is one of the safest of all possible post hoc tests Uses an F-ratio to evaluate significance of the difference between two treatment conditions F A versus B MS MS between within calculated with SS of twogroups

47 Between A & B

48 A & B F(2,24) = 3.36 B & C F(2,24) = 1.36 A & C F(2,24) = 9.00 df = 2, 24 and α =.05 the critical value for F: 3.40 Only the difference between A&C is significant.

49 Relationship between ANOVA and t tests For two independent samples, either t or F can be used Always result in same decision F = t 2 For any value of α, (t critical ) 2 = F critical

50 Figure Distribution of t and F statistics

51 Independent Measures ANOVA Assumptions The observations within each sample must be independent The population from which the samples are selected must be normal The populations from which the samples are selected must have equal variances (homogeneity of variance) Violating the assumption of homogeneity of variance risks invalid test results

52 To Report ANOVA Result The subjects averaged M A = 3, M B = 5.44, and M C = 7 in three treatments respectively. ANOVA indicated a significant difference, F(2, 24) = 9.15, p<.05, 2 =. Post hoc analysis (Tukey s HSD) indicated significant difference between Treatments A and B, as well as between Treatments A and C (HSD = 2.36). or Post hoc analysis (Sheffé) indicated significant difference between Treatments A and C only, F A vs. C (2,24) = 9, p<.05.

53 Homework 12.22

Multiple t Tests. Introduction to Analysis of Variance. Experiments with More than 2 Conditions

Multiple t Tests. Introduction to Analysis of Variance. Experiments with More than 2 Conditions Introduction to Analysis of Variance 1 Experiments with More than 2 Conditions Often the research that psychologists perform has more conditions than just the control and experimental conditions You might