Prepared by: Prof. Dr Bahaman Abu Samah Department of Professional Development and Continuing Education Faculty of Educational Studies Universiti Putra Malaysia Serdang
Use in experiment, quasi-experiment and field studies in which the same subject is measured under all levels of one or more trials The dependent variable is interval or ratio Trials is referred to as repeated-measure factor or withinsubjects factor
Test the effect of within-subjects factor (trial), betweensubjects factor (treatment) and interaction on the dependent variable
1. Treatment main effect: Is there any significant difference in sentence construction scores among the three treatment groups? 2. Trial main effect: Is there any significant difference in sentence construction scores among the three trials? 3. Interaction between treatment and trial: Do the differences in means for the sentence construction scores among the treatment groups vary depending on the trials?
1 DV is normally distributed in the population for each level of the within-subjects factor (trial) 2 The population variances of the difference scores computed between any two levels of a within-subjects factor are the same value regardless of which two levels are chosen. This assumption is also known as the sphericity assumption or homogeneity of variance of differences assumption 3 The cases represent a random samples from the populations, and the scores are independent of each other
Set Alpha Criteria sig-f> α sig-f α Decision Reject H O Fail to reject H O State H O and H A (3 Hypotheses) Report F & sig-f Decision Conclusion - Post-Hoc Comparison - Effect size (Partial Eta 2 ) Next
Steps in:
1 State H O and H A 2 4 3 Set confidence level (α) Run analysis & report F and sig-f Decision 5 Conclusion Next
1. Trial Main Effect (Within-Subjects) H O : μ t1 = μ t2 = μ t3 H A : Not all means are equal 2. Treatment Main Effect (Between-Subjects) H O : μ 1 = μ 2 = μ 2 H A : Not all means are equal 3. Interaction (I*J) H O : μ ij = μˈij H A : μ ij μˈij Next
α =.05
F and sig-f
Only two (2) possible decisions. Reject or Fail to Reject H O Reject H O : sig-f α Fail to reject H O : sig-f > α Criteria sig-f α sig-f > α Decision Reject H O Fail to reject H O
Treatment (Group) Main effect Reject H O Fail to reject H O There is a significant treatment (group) main effect on the DV There is no significant treatment (group) main effect on the DV Decision criteria Criteria sig-f α sig-f > α Decision Reject H O Fail to reject H O
Trial Main effect Reject H O Fail to reject H O There is a significant trial main effect on the DV There is no significant trial main effect on the DV Interaction: Treatment x Trial Reject H O Fail to reject H O There is a significant treatment-by-trial interaction effect on the DV There is no significant treatment-by-trial interaction effect on the DV
Use partial eta squared (η 2 ) as a measure of effect size Formula to calculate partial η 2 Partial 2 Main or interactio n SSB SSB Main Main or interactio n or interactio n SSE Effect size Conventions: <.10 Trivial.10 Small.25 Medium.40 Large Cohen, 1992 Partial η 2 indicated relationship between repeated-measures factor and the dependent variable; ranges between 0 to 1 0 indicates no relationship; 1 constitutes highest possible relationship between repeated measures factor and the dependent variable
In a study, a researcher is interested to access the effectiveness of a training program to improve students thinking skill. Students were randomly assigned into three groups based on their academic achievement (low, moderate, and high). Data were collected at pre, post1, and post2. ACHIEVE 1 2 3 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 PRE POST1 POST2 16 17 25 9 17 22 10 18 26 6 18 25 8 17 24 17 19 28 16 18 27 10 17 26 9 15 24 10 14 23 9 13 22 8 12 22 9 13 21 8 13 21 9 13 21 8 12 22 7 8 9 2 3 4 1 7 9 4 10 20 8 10 12 9 11 13 5 10 14 4 11 14 Data set: D8 Twowar Repeated Measure ANOVA THINKING SKILL
1. Treatment main effect: Is there any significant difference in thinking skill scores among the three treatment groups? 2. Trial main effect: Is there any significant difference in thinking skill scores among the three trials? 3. Interaction between treatment and trial: Do the differences in means for the thinking skill scores among the treatment groups vary depending on the trials?
1. Treatment main effect (Between group) H O : μ 1 = μ 2 = μ 3 H A : Not all means are equal 2. Trial main effect (Within-Subjects factor) H O : μ 1 = μ 2 = μ 3 H A : Not all means are equal 3. Interaction treatment x trial H O : μ ij = μ ij H A : μ ij μ ij
α =.05
Multivariate Tests Use of Multivariate tests does not require the assumption of sphericity Effect TRIAL TRIAL * ACHIEVE a. Exact statistic Pillai's Trace Wilks' Lambda Hotelling's Trace Roy's Largest Root Pillai's Trace Wilks' Lambda Hotelling's Trace Roy's Largest Root Multivariate Tests c Value F Hypothesis df Error df Sig..940 157.836 a 2.000 20.000.000.060 157.836 a 2.000 20.000.000 15.784 157.836 a 2.000 20.000.000 15.784 157.836 a 2.000 20.000.000.749 6.288 4.000 42.000.000.313 7.879 a 4.000 40.000.000 1.999 9.494 4.000 38.000.000 1.894 19.890 b 2.000 21.000.000 b. The statistic is an upper bound on F that yields a lower bound on the significance level. c. Design: Intercept+ACHIEVE Within Subjects Design: TRIAL Report F-ratio. Decision is based on sig-f Sig-F (.000) <.05; Significant trial main effect
Tests of Sphericity Mauchly's Test of Sphericity b Measure: MEASURE_1 Within Subjects Effect TRIAL Epsilon a Approx. Greenhous Mauchly's W Chi-Square df Sig. e-geisser Huynh-Feldt Lower-bound.628 9.316 2.009.729.844.500 Tests the null hypothesis that the error covariance matrix of the orthonormalized transformed dependent variables is proportional to an identity matrix. a. May be used to adjust the degrees of freedom for the averaged tests of significance. Corrected tests are displayed in the Tests of Within-Subjects Effects table. b. Design: Intercept+ACHIEVE Within Subjects Design: TRIAL Sig-value < α indicates violation of sphericity assumption
Tests of Within-Subjects Effects Use this value if the test meets the sphericity assumption Tests of Within-Subjects Effects Measure: MEASURE_1 Source TRIAL TRIAL * ACHIEVE Error(TRIAL) Sphericity Assumed Greenhouse-Geisser Huynh-Feldt Lower-bound Sphericity Assumed Greenhouse-Geisser Huynh-Feldt Lower-bound Sphericity Assumed Greenhouse-Geisser Huynh-Feldt Lower-bound Type III Sum of Squares df Mean Square F Sig. 1554.778 2 777.389 200.411.000 1554.778 1.457 1066.857 200.411.000 1554.778 1.687 921.405 200.411.000 1554.778 1.000 1554.778 200.411.000 137.639 4 34.410 8.871.000 137.639 2.915 47.223 8.871.000 137.639 3.375 40.784 8.871.000 137.639 2.000 68.819 8.871.002 162.917 42 3.879 162.917 30.604 5.323 162.917 35.435 4.598 162.917 21.000 7.758 Use any of the other three values if the sphericity assumption is violated
Tests of Between-Subjects Effects Measure: MEASURE_1 Transformed Variable: Average Source Intercept ACHIEVE Error Tests of Between-Subjects Effects Type III Sum Partial Eta of Squares df Mean Square F Sig. Squared 13667.556 1 13667.556 1064.675.000.981 1137.528 2 568.764 44.306.000.808 269.583 21 12.837 Report the F-value. However, decision is based on sig-f Sig-F (.000) <.05; reject the null hypothesis. Significant treatment effect Effect size
Treatment (Group) Main effect F = 44.306, sig F =.000 sig-f (.000) is smaller than α (.05) Reject H O There is a significant treatment (group) main effect on sentence construction scores at.05 level of significance Decision criteria Criteria sig-f> α sig-f α Decision Reject H O Fail to reject H O
Trial Main effect F = 200.411, sig-f =.000 sig-f (.000) is smaller than α (.05) Reject H O There is a significant trial main effect on sentence construction scores at.05 level of significance Interaction: Treatment x Trial F = 8.871, sig-f =.000 sig-f (.000) is smaller than α (.05) Reject H O There is a significant treatment-by-trial interaction effect on sentence construction scores at.05 level of significance
If the ANOVA reveals a significant result, proceed with the pairwise comparisons to assess which means differ from each other
Pairwise comparison: Treatment There are significant differences for the following pairs of groups: 1. 1 and 2 2. 1 and 3 3. 2 and 3 Measure: MEASURE_1 Pairwise Comparisons (I) Academic achievement 1 2 3 (J) Academic achievement 2 Based on estimated marginal means *. The mean difference is significant at the.05 level. 3 1 3 1 2 Mean Difference 95% Confidence Interval for Difference a (I-J) Std. Error Sig. a Lower Bound Upper Bound 3.542* 1.034.003 1.391 5.693 9.625* 1.034.000 7.474 11.776-3.542* 1.034.003-5.693-1.391 6.083* 1.034.000 3.932 8.234-9.625* 1.034.000-11.776-7.474-6.083* 1.034.000-8.234-3.932 a. Adjustment for multiple comparisons: Least Significant Difference (equivalent to no adjustments).
Pairwise comparison: Trial There are significant differences for the following pairs of trials: 1. 1 and 2 2. 1 and 3 3. 2 and 3 Measure: MEASURE_1 Pairwise Comparisons (I) TRIAL 1 2 3 (J) TRIAL 2 3 1 3 1 2 Mean Difference Based on estimated marginal means *. The mean difference is significant at the.05 level. 95% Confidence Interval for Difference a (I-J) Std. Error Sig. a Lower Bound Upper Bound -4.750*.560.000-5.915-3.585-11.333*.706.000-12.802-9.865 4.750*.560.000 3.585 5.915-6.583*.397.000-7.408-5.759 11.333*.706.000 9.865 12.802 6.583*.397.000 5.759 7.408 a. Adjustment for multiple comparisons: Least Significant Difference (equivalent to no adjustments).
1. Click Analyze General Linear Model Repeated Measures
2. At the dialog box below, type trial as within-subject factor name. and type 3 for the number of levels. - Click Add button. - Then click Define button.
3. Block all the within-subjects factors (Pre, Post1 and Post2), click the right arrow button.
4. Click the Academic achievement and enter into Between- Subject Factor(s) box. Then click the Option button
5. In the following Option dialog box, tick and select the following options. Click the Continue button
6. In the following Option dialog box, click OK