WITHIN-PARTICIPANT EXPERIMENTAL DESIGNS

Transcription

1 1 WITHIN-PARTICIPANT EXPERIMENTAL DESIGNS I. Single-factor designs: the model is: yij i j ij ij where: yij score for person j under treatment level i (i = 1,..., I; j = 1,..., n) overall mean βi treatment effect for treatment i (where βi = i ) j subject effect for subject j (where j = j ) βij interaction of treatment and subject ij random error we will only deal with the case where factor B is fixed and the subjects factor is random; we assume that βi is constant, βi = 0, j is independent N(0, 2 ), ij is independent N(0, 2 ), βij is N(0, [(I1)/I]β 2 ) with iβij = 0, and the j, βij, and ij are independent of each other; in addition to the previous assumptions, it is also assumed that the variance/covariance matrix of the observations is circular (which implies that the variances of all pairwise difference scores are equal in the population); (a) ANOVA table: SS df MS 2 2 Factor B ( y y ) n ( y y ) (I1) SSB/df i j i i i 2 2 Subjects (S) ( y y ) I ( y y ) (n1) SSS/df i j j j j 2 Interaction (B*S) ( y y y y ) (I1)(n1) SS(B*S)/df i j ij i j Total ( y y ) 2 y 2 ni y 2 I n 1 i j ij i j ij

2 2 the expected mean squares are: 2 2 E[MSB] E[MSS] n i E[MS(B *S)] 2 2 to test H0: βi = 0 vs. H1: βi 0 use the fact that under H0 MSB ~ F MS(BS) 1, 1 n1 (b) Comparisons among treatment means: to conduct comparisons among the treatment means, use any of the tests described for one-way between-participants designs, except that MS(BS) is substituted for MSE; to construct a 100(1-α)% confidence interval for some contrast ˆ use the following formula: / 2 1 MS( B S) ( c ) 1)( n 1) n 2 ˆ t ( I i if the homogeneity of variance assumption is in doubt, the error term may be computed from only those observations which are used in the comparison of interest (in the case of pairwise comparisons, this corresponds to the use of paired t-tests); (c) Trend analysis: again, the discussion of one-way between-participants designs is applicable, except that MS(BS) is used instead of MSE;

3 3 (d) Effect size measures: (i) standardized mean difference effect size : same as in the single-factor between-participants case; (ii) proportion of total variance effect sizes: 2 ˆ SSB SS total ( I 1)( MSB MS( B* S)) ˆ2 SS total ( I 1)( MSB MS( B* S)) ˆ 2 SS MSS total (e) ANOVA assumptions: in order for the statistical tests to be valid, it is critical that the variance/covariance matrix of the observations be circular (which implies that the variance of the difference between any two scores is a constant); tests for this assumption are available; alternatively, one may use (i) a conservative adjustment to the F-test (i.e., the significance of the F-ratio is evaluated based on F1, (n-1) instead of F(I-1), (I-1)(n-1)), or (ii) the Greenhouse-Geisser and Huynh-Feldt adjusted degrees of freedom tests based on estimates of the sphericity parameter; Note: repeated-measures data can also be analyzed using multivariate analysis of variance (MANOVA); because it requires less restrictive assumptions, MANOVA is often regarded as the preferred method of analysis (see Iacobucci 1994 for an overview);

4 4 II. Multi-factor designs: distinguish the following two cases: (i) pure within-participant designs: all factors included in the design are within-participant factors; (ii) mixed between-/within-participants designs: both between- and withinparticipants factors are included in the design; here we will only deal with the mixed two-factor design, with factor A being the (fixed) between-participants factor and factor B being the (fixed) within-participant (or repeatedmeasures) factor; the model is: y ijk i k j ij jk ijk where: yijk score of person k for level i of factor A and level j of factor B (i = 1,..., I; j = 1,..., J, k = 1,..., n) overall mean i effect of factor A k effect for subject k j effect of factor B ij interaction effect of factors A and B jk interaction effect of factor B and the subjects factor ijk random error in addition to the usual assumptions, it is assumed that the J x J variance/covariance matrices are homogeneous over all levels of factors A and that the pooled covariance matrix is circular;

5 5 (a) ANOVA table: SS df MS 2 2 Factor A ( y y ) nj ( y y ) (I1) SSA/df i j k i i i S(A) ( y y ) 2 J ( y y ) 2 I (n1) SS[S(A)]/df i j k ik i ik i i k 2 2 Factor B ( y y ) ni ( y y ) (J1) SSB/df i j k j j j 2 A*B n ( y y y y ) (IJ) SS[A*B]/df i j ij i j 2 B*S(A) ( y y y y ) I (J(n ) SS[B*S(A)]/df i j k ijk ij ik i Total ( y y ) y ni J y i j k ijk ijk i j k I J n 1 SST the expected mean squares are: 2 E[MSA] J 2 n J i 1 E[MSS(A)] J j E[MSB] n J E[MS(A *B)] n E[MS(B*S(A))] ij 1 J 1

6 6 to test H0: i = 0 use the fact that under H0 MSA MSS(A) ~ F, ) 1 n 1 to test H0: j = 0 and H0: ij = 0 use the fact that under H0 MSB MS[B*S(A)] ~, F J J n MS AB MS[B*S(A)] ~ F J, J n (b) Comparisons among treatment means and trend analysis: the analysis is similar to the two-factor between-participants design, except that the error terms have to be chosen appropriately; in general, the appropriate error term is the one used in the overall ANOVA for the particular factor under consideration (see, for example, Winer et al. 1991, pp. 526 ff. for details): y y y y y i y y y y y i' j j' ij ij ij i' j ij' i' j' MSS A MS BS A MS MS B S A MS pooled pooled SSS A SS BSA I(n 1) I(J 1)(n 1) to construct confidence intervals for contrasts, use the following formula involving the relevant contrast weights (cij) and the number of respondents whose data were averaged (nij), the appropriate error term (MS) as shown above, and the corresponding degrees of freedom (df): 2 / 2 cij ˆ t MS( )( ) df n ij if the homogeneity of variance assumption is in doubt, the error term may be computed from only those observations which are used in the comparison of interest;

7 7 (c) Effect size measures: (i) standardized mean difference effect size : same as before; (ii) proportion of total variance effect sizes (where N is the total number of respondents participating in the experiment): ˆ 2 A SSA SS total ˆ 2 B SSB SS total ˆ 2 A*B SS(A*B) SS total ˆ 2 A SS total ( I 1)( MSA MSS( A)) MSS( A) N MS( B* S( A)) ( J 1)( MSB MS( B* S( A)) ˆ 2 B SS MSS( A) N MS( B* S( A)) total ˆ 2 A* B ( I 1)( J 1)( MS( A* B) MS( B* S( A)) SS MSS( A) N MS( B* S( A)) total (d) Assumptions: in addition to the usual assumptions, two further assumptions have to be met for the tests of the within-participant effects to be valid: the variance/covariance matrices have to be homogeneous over all levels of factor A, and the pooled covariance matrix has to be circular; tests for these assumptions are available; alternatively, either (i) a conservative adjustment to the F-test can be employed: for H0: j = 0 the critical value for F is F1, I(n1) for H0: ij = 0 the critical value is F(I1), I(n1) or (ii) the Greenhouse-Geisser and Huynh-Feldt adjusted-df tests can be used;

8 8 Note: repeated-measures data can also be analyzed using multivariate analysis of variance (MANOVA); because it requires less restrictive assumptions, MANOVA is often regarded as the preferred method of analysis (see Iacobucci 1994 for an overview); Note: the most flexible approach to analyzing repeated-measures data is based on mixed models; mixed models can be estimated with PROC MIXED within SAS; although within-participant designs have several advantages (esp. gains in power), they have to be used with care because of the possibility of context effects (cf. Greenwald 1976): practice sensitization carry-over

9 9 Single-factor design: DATA ANOVAW; INPUT SUBJ Interference RECALL1-RECALL4; CARDS; ; proc ttest data=anovaw ci=none; paired RECALL1*RECALL2 RECALL1*RECALL3 RECALL1*RECALL4 RECALL2*RECALL3 RECALL2*RECALL4 RECALL3*RECALL4; run; [pp ] PROC GLM data=anovaw; MODEL RECALL1-RECALL4 = / NOUNI; REPEATED Repet 4 contrast (1) / MSTAT=EXACT SUMMARY PRINTE; run; [pp ] DATA ANOVAL; INPUT SUBJ DO Repet=1 TO 4; INPUT OUTPUT; END; CARDS; ;

10 10 PROC GLM data=anoval; CLASS Repet SUBJ; MODEL RECALL = Repet SUBJ Repet*SUBJ; TEST H=Repet E=Repet*SUBJ; LSMEANS Repet / T PDIFF TDIFF CL E=Repet*SUBJ; run; [pp ] PROC STANDARD M=0 DATA=ANOVAL OUT=ANOVALMC; VAR RECALL; BY SUBJ; DATA ANOVALMC; SET ANOVALMC; RECALL=RECALL ; proc sgplot data=anovalmc; title "vbar plot with confidence intervals by condition"; title2 "subject effect eliminated "; vbar Repet / response=recall stat=mean limitstat=clm; run; [p. 17] proc mixed data=anoval method=reml; class repet subj; model recall = repet / s ; repeated / subject=subj type=cs r; run; [p. 18] quit;

11 11 The TTEST Procedure Difference: RECALL1 - RECALL2 N Mean Std Dev Std Err Minimum Maximum Mean 95% CL Mean DF t Value Pr > t Difference: RECALL1 - RECALL3 N Mean Std Dev Std Err Minimum Maximum Mean 95% CL Mean DF t Value Pr > t Difference: RECALL1 - RECALL4 N Mean Std Dev Std Err Minimum Maximum Mean 95% CL Mean DF t Value Pr > t <.0001

12 12 Difference: RECALL2 - RECALL3 N Mean Std Dev Std Err Minimum Maximum Mean 95% CL Mean DF t Value Pr > t Difference: RECALL2 - RECALL4 N Mean Std Dev Std Err Minimum Maximum Mean 95% CL Mean DF t Value Pr > t Difference: RECALL3 - RECALL4 N Mean Std Dev Std Err Minimum Maximum Mean 95% CL Mean DF t Value Pr > t

13 13 The GLM Procedure Repeated Measures Analysis of Variance Sphericity Tests Mauchly's Variables DF Criterion Chi-Square Pr > ChiSq Transformed Variates Orthogonal Components MANOVA Tests for the Hypothesis of no Repet Effect H = Type III SSCP Matrix for Repet E = Error SSCP Matrix S=1 M=0.5 N=1.5 Statistic Value P-Value Wilks' Lambda <.0001 Pillai's Trace <.0001 Hotelling-Lawley Trace <.0001 Roy's Greatest Root <.0001 The GLM Procedure Repeated Measures Analysis of Variance Univariate Tests of Hypotheses for Within Subject Effects Repet <.0001 Error(Repet) Adj Pr > F Source G - G H - F Repet <.0001 <.0001 Error(Repet) Greenhouse-Geisser Epsilon Huynh-Feldt Epsilon

14 14 The GLM Procedure Repeated Measures Analysis of Variance Analysis of Variance of Contrast Variables Repet_N represents the contrast between the nth level of Repet and the 1st Contrast Variable: Repet_2 Mean Error Contrast Variable: Repet_3 Mean Error Contrast Variable: Repet_4 Mean <.0001 Error

15 15 The GLM Procedure Dependent Variable: RECALL Sum of Source DF Squares Mean Square F Value Pr > F Model Error Corrected Total R-Square Coeff Var Root MSE RECALL Mean Source DF Type I SS Mean Square F Value Pr > F Repet SUBJ Repet*SUBJ Repet SUBJ Repet*SUBJ Tests of Hypotheses Using the Type III MS for Repet*SUBJ as an Error Term Repet <.0001

16 16 The GLM Procedure Least Squares Means Standard Errors and Probabilities Calculated Using the Type III MS for Repet*SUBJ as an Error Term RECALL LSMEAN Repet LSMEAN Number Least Squares Means for Effect Repet t for H0: LSMean(i)=LSMean(j) / Pr > t Dependent Variable: RECALL i/j <.0001 < < < RECALL Repet LSMEAN 95% Confidence Limits Least Squares Means for Effect Repet Difference Between 95% Confidence Limits for i j Means LSMean(i)-LSMean(j)

17 17

18 18 The Mixed Procedure Covariance Parameter Estimates Cov Parm Subject Estimate CS SUBJ Residual Fit Statistics -2 Res Log Likelihood AIC (smaller is better) AICC (smaller is better) BIC (smaller is better) Null Model Likelihood Ratio Test DF Chi-Square Pr > ChiSq <.0001 Solution for Fixed Effects Standard Effect Repet Estimate Error DF t Value Pr > t Intercept <.0001 Repet <.0001 Repet Repet Repet The Mixed Procedure Type 3 Tests of Fixed Effects Num Den Effect DF DF F Value Pr > F Repet <.0001

19 19 Two-factor design: DATA ANOVAW; INPUT SUBJ Interference RECALL1-RECALL4; CARDS; ; PROC MEANS MEAN VAR; VAR RECALL1-RECALL4; BY Interference; run; [p. 21] PROC GLM data=anovaw; TITLE 'overall analysis'; CLASS Interference; MODEL RECALL1-RECALL4=Interference / NOUNI; REPEATED Repet 4 / MSTAT=EXACT SUMMARY; run; [pp ] PROC GLM data=anovaw; TITLE 'main effect of interference'; CLASS Interference; MODEL RECALL1-RECALL4=Interference / NOUNI; REPEATED Repet 4 / MSTAT=EXACT; CONTRAST 'Low vs. High Interference' Interference -1 1; MANOVA H=INTERFERENCE M=RECALL1+RECALL2+RECALL3+RECALL4 / MSTAT=EXACT SUMMARY; run; [p. 24] PROC GLM data=anovaw; TITLE 'main effect of repetition'; CLASS Interference; MODEL RECALL1-RECALL4=Interference / NOUNI; MANOVA H=INTERCEPT M=RECALL2-RECALL1, RECALL3-RECALL1, RECALL4-RECALL1 / MSTAT=EXACT SUMMARY; /* or REPEATED Repet 4 CONTRAST (1) / MSTAT=EXACT SUMMARY; */ run; [p. 25] PROC GLM data=anovaw; TITLE 'interference by specific repetition contrasts'; CLASS Interference; MODEL RECALL1-RECALL4=Interference / NOUNI; MANOVA H=INTERFERENCE M=RECALL2-RECALL1, RECALL3-RECALL1, RECALL4-RECALL1 / MSTAT=EXACT SUMMARY; run; [p. 26] PROC GLM data=anovaw; TITLE 'repetition by specific interference contrasts'; CLASS Interference; MODEL RECALL1-RECALL4=Interference / NOUNI; CONTRAST 'low vs. high interference' INTERFERENCE -1 1; REPEATED Repet 4; run; [p. 27]

20 20 PROC GLM data=anovaw; TITLE 'specific contrasts for both interference and repetition'; CLASS Interference; MODEL RECALL1-RECALL4=Interference / NOUNI; CONTRAST 'low vs. high interference' INTERFERENCE -1 1; MANOVA H=INTERFERENCE M=RECALL2-RECALL1, RECALL3-RECALL1, RECALL4-RECALL1 / MSTAT=EXACT SUMMARY; LSMEANS Interference / pdiff cl; run; [p. 28] DATA ANOVAL; INPUT SUBJ DO Repet=1 TO 4; INPUT OUTPUT; END; CARDS; etc. ; PROC GLM data=anoval; CLASS Interference Repet SUBJ; MODEL RECALL = Interference SUBJ(Interference) Repet Interference*Repet Repet*SUBJ(Interference); TEST H=Interference E=SUBJ(Interference); TEST H=Repet E=Repet*SUBJ(Interference); TEST H=Interference*Repet E=Repet*SUBJ(Interference); LSMEANS Interference / T TDIFF PDIFF CL E=SUBJ(Interference); LSMEANS Repet / T TDIFF PDIFF CL E=Repet*SUBJ(Interference); LSMEANS Interference*Repet / T TDIFF PDIFF CL E=Repet*SUBJ(Interference); run; [pp ] PROC STANDARD M=0 DATA=ANOVAL OUT=ANOVALMC; VAR RECALL; BY SUBJ; DATA ANOVALMC; SET ANOVALMC; IF Interference=1 THEN RECALL=RECALL ; IF Interference=2 THEN RECALL=RECALL ; proc sgpanel data=anovalmc; title "vbar plot with confidence intervals by condition"; title2 "subject effect eliminated "; panelby Interference; vbar Repet / response=recall stat=mean limitstat=clm; run; [p. 34] proc mixed data=anoval method=reml; class Interference Repet subj; model recall = Interference Repet Interference*Repet / s; repeated / subject=subj type=cs r; run; [p. 35] quit;

21 Interference= The MEANS Procedure Variable Mean Variance ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ RECALL RECALL RECALL RECALL ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ Interference= Variable Mean Variance ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ RECALL RECALL RECALL RECALL

22 22 overall analysis The GLM Procedure Repeated Measures Analysis of Variance Repeated Measures Level Information Dependent Variable RECALL1 RECALL2 RECALL3 RECALL4 Level of Repet MANOVA Tests for the Hypothesis of no Repet Effect H = Type III SSCP Matrix for Repet E = Error SSCP Matrix S=1 M=0.5 N=1 Statistic Value P-Value Wilks' Lambda Pillai's Trace Hotelling-Lawley Trace Roy's Greatest Root MANOVA Tests for the Hypothesis of no Repet*Interference Effect H = Type III SSCP Matrix for Repet*Interference E = Error SSCP Matrix S=1 M=0.5 N=1 Statistic Value P-Value Wilks' Lambda Pillai's Trace Hotelling-Lawley Trace Roy's Greatest Root

23 23 overall analysis The GLM Procedure Repeated Measures Analysis of Variance Tests of Hypotheses for Between Subjects Effects Interference Error The GLM Procedure Repeated Measures Analysis of Variance Univariate Tests of Hypotheses for Within Subject Effects Repet <.0001 Repet*Interference Error(Repet) Adj Pr > F Source G - G H-F-L Repet <.0001 Repet*Interference Error(Repet) Greenhouse-Geisser Epsilon Huynh-Feldt-Lecoutre Epsilon

24 24 main effect of interference The GLM Procedure Multivariate Analysis of Variance M Matrix Describing Transformed Variables RECALL1 RECALL2 RECALL3 RECALL4 MVAR MANOVA Tests for the Hypothesis of No Overall Interference Effect on the Variables Defined by the M Matrix Transformation H = Type III SSCP Matrix for Interference E = Error SSCP Matrix S=1 M=-0.5 N=2 Statistic Value P-Value Wilks' Lambda Pillai's Trace Hotelling-Lawley Trace Roy's Greatest Root MANOVA Tests for the Hypothesis of No Overall Low vs. High Interference Effect on the Variables Defined by the M Matrix Transformation H = Contrast SSCP Matrix for Low vs. High Interference E = Error SSCP Matrix S=1 M=-0.5 N=2 Statistic Value P-Value Wilks' Lambda Pillai's Trace Hotelling-Lawley Trace Roy's Greatest Root The GLM Procedure Multivariate Analysis of Variance Dependent Variable: MVAR1 Interference Error Contrast DF Contrast SS Mean Square F Value Pr > F Low vs. High Interference

25 25 Main effect of repetition The GLM Procedure Multivariate Analysis of Variance M Matrix Describing Transformed Variables RECALL1 RECALL2 RECALL3 RECALL4 MVAR MVAR MVAR Dependent Variable: MVAR1 Intercept Error Dependent Variable: MVAR2 Intercept Error Dependent Variable: MVAR3 Intercept <.0001 Error

26 26 interference by specific repetition contrasts M Matrix Describing Transformed Variables RECALL1 RECALL2 RECALL3 RECALL4 MVAR MVAR MVAR Dependent Variable: MVAR1 Interference Error Dependent Variable: MVAR2 Interference Error Dependent Variable: MVAR3 Interference Error

27 27 repetition by specific interference contrasts MANOVA Test Criteria and Exact F Statistics for the Hypothesis of no Repet*low vs. high interference Effect H = Contrast SSCP Matrix for Repet*low vs. high interference E = Error SSCP Matrix S=1 M=0.5 N=1 Statistic Value F Value Num DF Den DF Pr > F Wilks' Lambda Pillai's Trace Hotelling-Lawley Trace Roy's Greatest Root Contrast DF Contrast SS Mean Square F Value Repet*low vs. high interference Adj Pr > F Contrast Pr > F G - G H-F-L Repet*low vs. high interference

28 28 specific contrasts for both interference and repetition M Matrix Describing Transformed Variables RECALL1 RECALL2 RECALL3 RECALL4 MVAR MVAR MVAR MANOVA Tests for the Hypothesis of No Overall low vs. high interference Effect on the Variables Defined by the M Matrix Transformation H = Contrast SSCP Matrix for low vs. high interference E = Error SSCP Matrix S=1 M=0.5 N=1 Statistic Value P-Value Wilks' Lambda Pillai's Trace Hotelling-Lawley Trace Roy's Greatest Root Dependent Variable: MVAR1 Interference Error Contrast DF Contrast SS Mean Square F Value Pr > F low vs. high interference Dependent Variable: MVAR2 Interference Error Contrast DF Contrast SS Mean Square F Value Pr > F low vs. high interference Dependent Variable: MVAR3 Interference Error Contrast DF Contrast SS Mean Square F Value Pr > F low vs. high interference

29 29 The GLM Procedure Dependent Variable: RECALL Sum of Source DF Squares Mean Square F Value Pr > F Model Error Corrected Total R-Square Coeff Var Root MSE RECALL Mean Source DF Type I SS Mean Square F Value Pr > F Interference SUBJ(Interference) Repet Interference*Repet Repet*SUBJ(Interfer) Interference SUBJ(Interference) Repet Interference*Repet Repet*SUBJ(Interfer) Tests of Hypotheses Using the Type III MS for SUBJ(Interference) as an Error Term Interference Tests of Hypotheses Using the Type III MS for Repet*SUBJ(Interfer) as an Error Term Repet <.0001 Interference*Repet

30 30 The GLM Procedure Least Squares Means Standard Errors and Probabilities Calculated Using the Type III MS for SUBJ(Interference) as an Error Term RECALL H0:LSMean1=LSMean2 Interference LSMEAN t Value Pr > t RECALL Interference LSMEAN 95% Confidence Limits Least Squares Means for Effect Interference Difference Between 95% Confidence Limits for i j Means LSMean(i)-LSMean(j)

31 31 The GLM Procedure Least Squares Means Standard Errors and Probabilities Calculated Using the Type III MS for Repet*SUBJ(Interfer) as an Error Term RECALL LSMEAN Repet LSMEAN Number Least Squares Means for Effect Repet t for H0: LSMean(i)=LSMean(j) / Pr > t Dependent Variable: RECALL i/j <.0001 < < < RECALL Repet LSMEAN 95% Confidence Limits Difference Between 95% Confidence Limits for i j Means LSMean(i)-LSMean(j)

32 32 The GLM Procedure Least Squares Means Standard Errors and Probabilities Calculated Using the Type III MS for Repet*SUBJ(Interfer) as an Error Term RECALL LSMEAN Interference Repet LSMEAN Number Least Squares Means for Effect Interference*Repet t for H0: LSMean(i)=LSMean(j) / Pr > t Dependent Variable: RECALL i/j < <.0001 <.0001 < < < < < < < <.0001 <.0001 < < RECALL Interference Repet LSMEAN 95% Confidence Limits

33 33 Difference Between 95% Confidence Limits for i j Means LSMean(i)-LSMean(j)

34 34

35 35 The Mixed Procedure Covariance Parameter Estimates Cov Parm Subject Estimate CS SUBJ Residual Fit Statistics -2 Res Log Likelihood AIC (smaller is better) AICC (smaller is better) BIC (smaller is better) Null Model Likelihood Ratio Test DF Chi-Square Pr > ChiSq <.0001 Type 3 Tests of Fixed Effects Num Den Effect DF DF F Value Pr > F Interference Repet <.0001 Interference*Repet