Outline. Analysis of Variance. Acknowledgements. Comparison of 2 or more groups. Comparison of serveral groups

Transcription

1 Outline Analysis of Variance Analysis of variance and regression course Comparison of serveral groups Model checking Marc Andersen, Two-way ANOVA Analysis of variance and regression for health researchers, November 25, 2010 Interaction Advanced designs Acknowledgements 1 / 68 Comparison of 2 or more groups 2 / 68 written by Lene Theil Skovgaard , 2007 updated by Julie Lyng Forman , November 2009 updated by Marc Andersen 2 April 2009, April 2010, November 2010 number different same of groups individuals individual 2 unpaired paired t-test t-test 2 oneway twoway analysis of variance analysis of variance 1 Dept. of Biostatistics 2 StatGroup 3 / 68 4 / 68

2 7 / 68 8 / 68 One-way analysis of variance Example: ventilation during anaesthesia Do the distributions differ between the groups? Do the levels differ between the groups? Data: 22 bypass-patients randomised to 3 different kinds of ventilation during anaesthesia Outcome: measurement of red cell folate Group I Group II Group III 50% N 2 O, 50% O 2 for 24 hours 50% N 2 O, 50% O 2 during operation 30 50% O 2 (no N 2 O) for 24 hours Gr.I Gr.II Gr.III n Mean SD Example: ventilation during anaesthesia 5 / 68 One-way ANOVA 6 / 68 Red cell folate I II III Group One-way because we only have one critera for classification of the observations, here ventilation method ANalysis Of VAriance because we compare the variance between groups with the variance within groups

3 11 / / 68 The one-way ANOVA model Hypothesis testing Notation The j th observation from group i is described by: Y ij = µ i + ε ij j th observation mean individual in group no. i group i deviation i.e. as consisting of mean of the group plus an individual deviation, with ε ij N(0, σ 2 ) or equivalently Y ij N(µ i, σ 2 ). Assumptions Observations are assumed be independent and to follow a normal distribution with mean µ i withing group i with the same variance. Model assumptions should be investigated! Investigate difference between groups Null hypothesis: group means are equal, H 0 : µ i = µ Alternative hypothesis: group means are not equal We conclude that the means are not equal when we reject the null hypothesis of equality (ref DGA, 8.5 Hypothesis Testing) ANOVA math: Sums of squares Decomposition of deviation from grand mean y ij ȳ = (y ij ȳ i ) + (ȳ i ȳ ) Decomposition of variation (sums of squares) (y ij ȳ ) 2 = (y ij ȳ i ) 2 i,j i,j }{{}}{{} total variation within groups + i,j (ȳ i ȳ ) 2 }{{} between groups 9 / 68 Decomposition of variation total = between + within F-test statistic SS total = SS between + SS within (n 1) = (k 1) + (n k) F = MS between = SS between/(k 1) MS within SS within /(N k) 10 / 68 y ij ȳ i ȳ. j th observation in i th group average in i th group overall average, or grand mean Hypothesis test Reject the null hypothesis if F is large, i.e. if the variation between groups is too large compared to the variation within groups.

4 15 / / 68 Analysis of variance table Analysis of variance table - Anaestesia example ANOVA table ANOVA table Variation df SS MS F P Between k 1 SS b SS b /df b MS b /MS w P(F(df b, df w ) > F obs ) Within n k SS w SS w /df w Total n 1 SS tot F test statistics The F test statistics follows and F-distribution with df b and df w degrees of freedom: F obs F (df b, df w ). df SS MS F P Between Within Total F test statistics F = 3.71 F (2, 19) P = 0.04 Interpretation Weak evidence of non-equality of the three means Analysis of variance in SAS 13 / 68 Analysis of variance program 14 / 68 To define the anaestesia data in SAS, we write data ex_redcell; input grp redcell; cards; ; The variable redcell contains all the measurements of the outcome and grp contains the method of ventilation for each individual. proc glm data=ex_redcell; class grp; model redcell=grp / solution; General Linear Models Procedure Dependent Variable: REDCELL Sum of Mean Source DF Squares Square F Value Pr > F Model Error Corrected Total R-Square C.V. Root MSE REDCELL Mean Source DF Type I SS Mean Square F Value Pr > F GRP Source DF Type III SS Mean Square F Value Pr > F GRP

5 19 / / 68 Parameter estimates PROC glm box plot The option solution outputs parameter estimates T for H0: Pr > T Std Error of Parameter Estimate Parameter=0 Estimate INTERCEPT B GRP B B B... NOTE: The X X matrix has been found to be singular and a generalized inverse was used to solve the normal equations. Estimates followed by the letter B are biased, and are not unique estimators of the parameters. Group 3 (the last group) is the reference group The estimates for the other groups refer to differences to this reference group Interpreting the estimates 17 / 68 Multiple comparisons 18 / 68 What is the scientific question Clinical significance Statistical significance Provide confidence interval Does it make sense? The F -test show, that there is a difference but where? Pairwise t-tests are not suitable due to risk of mass significance A significance level of α = 0.05 means 5% chance of wrongfully rejecting a true hypothesis (type I error) The chance of at least one type I error goes up with the number of tests. (for k groups, we have m = k(k 1)/2 possible tests, the actual significance level can be as bad as: 1 (1 α) m, e.g. for k=5: 0.40)

6 23 / / 68 Adressing multiplicity Tukey: multiple comparisons in SAS There is no completely satisfactory solution. Approximative solutions 1. Select a (small) number of relevant comparisons in the planning stage. 2. Make a graph of the average ±2 SEM and judge visually (!), perhaps supplemented with F -tests on subsets of groups. 3. Modify the t-tests by multiplying the P-values with the number of tests, the socalled Bonferroni correction (conservative) 4. Use a correction for multiple testing (Dunnett, Tukey) or a (prespecified) multiple testing procedure proc glm data=ex_redcell; class grp; model redcell=grp / solution; lsmeans grp / adjust=tukey pdiff cl; The GLM Procedure Least Squares Means Adjustment for Multiple Comparisons: Tukey-Kramer Least Squares Means for effect grp Pr > t for H0: LSMean(i)=LSMean(j) Dependent Variable: redcell i/j Least Squares Means for Effect grp Difference Simultaneous 95% Between Confidence Limits for i j Means LSMean(i)-LSMean(j) / / 68 Visual assessment (1/3) Visual assessment (2/3) The bars represent 95 % confidence intervals for the means using the standard deviation for each group (std2mjt in symbol1 statement). The bars represent 95 % confidence intervals for the means using the pooled standard deviation for each group (std2mpjt in symbol1 statement). proc gplot data=ex_redcell; plot redcell*grp / haxis=axis1 vaxis=axis2 frame; axis1 order=(1 to 3 by 1) offset=(8,8) label=(h=3) value=(h=2) minor=none; axis2 offset=(1,1) value=(h=2) minor=none label=(a=90 R=0 H=3); symbol1 v=circle i=std2mjt l=1 h=2 w=2; Red cell folate I II III Group proc gplot data=ex_redcell; plot redcell*grp / haxis=axis1 vaxis=axis2 frame; axis1 order=(1 to 3 by 1) offset=(8,8) label=(h=3) value=(h=2) minor=none; axis2 offset=(1,1) value=(h=2) minor=none label=(a=90 R=0 H=3); symbol1 v=circle i=std2mpjt l=1 h=2 w=2; Red cell folate I II III Group

7 27 / / 68 Visual assessment (3/3) The bars represent 95 % confidence intervals for the means using the pooled standard deviation for each group obtained from PROC glm. Model checking Check if the assumptions are reasonable: (If not the analysis is unreliable!) Variance homogeneity may be checked by performing Levenes test (or Bartletts test). In case of variance inhomogeneity, we may also perform a weighted analysis (Welch s test), just as in the T-test Normality may be checked through probability plots (or histograms) of residuals, or by a numerical test on the residuals. In case of non-normality, we may use the nonparametric Kruskal-Wallis test Transformation (often logarithms) may help to achieve variance homogeneity as well as normality Check of variance homogeneity and normality in SAS 25 / 68 Output from proc glm: Test for variance homogeneity 26 / 68 proc glm data=ex_redcell; class grp; model redcell=grp; means grp / hovtest=levene welch; output out=model p=predicted r=residual; Store residuals in a dataset for further model checking proc univariate data=model normal ; var residual; histogram residual/ normal(mu=0); ppplot residual / normal(mu=0) square; Levene s Test for Homogeneity of redcell Variance ANOVA of Squared Deviations from Group Means Sum of Mean Source DF Squares Square F Value Pr > F grp Error Weighted anova in case of variance heterogeneity: Welch s ANOVA for redcell Source DF F Value Pr > F grp Error So we are not too sure concerning the group differences...

8 31 / / 68 Test for normality Output from proc univariate: Histogram and probability plot Output from proc univariate Tests for Normality Test --Statistic p Value---- Shapiro-Wilk W Pr < W Kolmogorov-Smirnov D Pr > D > Cramer-von Mises W-Sq Pr > W-Sq > Anderson-Darling A-Sq Pr > A-Sq > The 4 tests focus on different aspects of non-normality. For small data sets, we rarely get significance For large data sets, we almost always get significance Could look at a probability plot instead Non-parametric ANOVA, the Kruskal-Wallis test 29 / 68 Two-way analysis of variance 30 / 68 SAS code proc npar1way wilcoxon; exact; class grp; var redcell; Wilcoxon Scores (Rank Sums) for Variable redcell Classified by Variable grp Sum of Expected Std Dev Mean grp N Scores Under H0 Under H0 Score Kruskal-Wallis Test Chi-Square DF 2 Asymptotic Pr > Chi-Square Exact Pr >= Chi-Square Again, we have lost the significance... Two criterias for subdividing observations, A og B Data in two-way layout: B A 1 2 c r. Effect of both factors Perhaps even interaction (effect modification) One factor may be individuals or experimental units (e.g. different treatments tried on same person)

9 35 / / 68 Repeated measurements Line plot ( Spaghettiogram ) Example: Short term effect of enalaprilate on heart rate Time Subject average average Ideally the time courses are parallel. The additive model 33 / 68 Analysis of variance table - enalaprilate example 34 / 68 The two effects (s and t) work in an additive way. Y st = µ + α s + β t + ε st The ε st s are assumed to be independent, normally distributed with mean 0, and identical variances, ε st N(0, σ 2 ). (This assumption should be investigated!) Variational decomposition: SS total = SS subject + SS time + SS residual df SS MS F P Subjects < Times Residual Total Highly significant difference between subjects (not very interesting) Significant time differences.

10 39 / / 68 Two-way ANOVA in SAS Two-way ANOVA output proc glm data=ex_pulse; class subject times; model hrate=subject times / solution; General Linear Models Procedure Class Level Information Class Levels Values SUBJECT TIMES Number of observations in data set = 36 General Linear Models Procedure Dependent Variable: HRATE Sum of Mean Source DF Squares Square F Value Pr > F Model Error Corrected Total R-Square C.V. Root MSE HRATE Mean Source DF Type I SS Mean Square F Value Pr > F SUBJECT TIMES Source DF Type III SS Mean Square F Value Pr > F SUBJECT TIMES Parameter estimates T for H0: Pr > T Std Error of Parameter Estimate Parameter=0 Estimate 37 / 68 Expected values and residuals Expected values for subject=3, times=30 38 / 68 INTERCEPT B SUBJECT B B B B B B B B B... TIMES B B B B... Residuals ŷ st = ˆµ + ˆα s + ˆβ t = = NOTE: The X X matrix has been found to be singular and a generalized inverse was used to solve the normal equations. Estimates followed by the letter B are biased, and are not unique estimators of the parameters. subject 9 at time 120 minutes is the reference r st = observed expected = y st ŷ st ε st Residual for subject 3, time 30: r 32 = = 0.84

11 43 / / 68 Model checking Residual based diagnostics Look for: differences in variances (systematic?) Non-normality Lack of additivity (interaction). Can only be tested if there is more than one observation for each combination Serial correlation? (Neighboring observations look more alike) Use the residuals for model checking Probability plot of residuals. Plot residuals vs expected values. Plot residuals vs group. Look for outliers (a large residual means observed and expected values deviate a lot). Enalaprilate example 41 / 68 Interaction 42 / 68 Example of two criterias for subdividing individuals: sex and smoking habits Outcome: FEV 1 Here, we see an interaction between sex and smoking. No systematic patterns should be present.

12 47 / / 68 Possible explanations for interaction Example: The effect of smoking on birth weight Biologically different effects of smoking on males and females Perhaps the women do not smoke as much as the men Perhaps the effect is relative (to be expressed in %) Example: The effect of smoking on birth weight 45 / 68 Interpreting interaction 46 / 68 There is an effect of smoking, but only for those who have been smoking for a long time. There is an effect of duration, and this effects increases with amount of smoking The effect of duration depends upon... amount of smoking and the effect of amount depends upon... duration of smoking

13 51 / / 68 Example: Fibrinogen after spleen operation Example: Fibrinogen after spleen operation 34 rats are randomized, in 2 ways 17 have their spleen removed (splenectomy=yes/no) 8/17 in each group are kept in altitude chambers ( ft) (place=altitude/control) fibrinogen Outcome Fibrinogen level in mg% at day no_altitude no_control yes_altitude yes_control group ANOVA model with interaction 49 / 68 Two-way ANOVA with interaction in SAS 50 / 68 The usual additive model: Y spr = µ + α s + β p + ε spr, ε spr N(0, σ 2 ) splenectomy (s=yes/no) and place (p=altitude/control) have an additive effect. Model with interaction Y spr = µ + α s + β p + γ sp + ε spr, ε spr N(0, σ 2 ) Here, we specify an interaction between splenectomy and place, i.e. the effect of living in a high altitude may be thought to depend upon whether or not you have an intact spleen. and vice versa.. proc glm data=ex_fibrinogen; class splenectomy place; model fibrinogen=place splenectomy place*splenectomy / solution; output out=model p=predicted r=residual; The GLM Procedure Class Level Information Class Levels Values splenectomy 2 no yes place 2 altitude control Number of observations 34

14 55 / / 68 Output: two-way ANOVA table Output: Parameter estimates Dependent Variable: fibrinogen Sum of Source DF Squares Mean Square F Value Pr > F Model Error Corrected Total R-Square Coeff Var Root MSE fibrinogen Mean Source DF Type I SS Mean Square F Value Pr > F place splenectomy splenectomy*place Source DF Type III SS Mean Square F Value Pr > F Standard Parameter Estimate Error t Value Pr > t Intercept B <.0001 place altitude B place control B... splenectomy no B splenectomy yes B... splenectomy*place no altitude B splenectomy*place no control B... splenectomy*place yes altitude B... splenectomy*place yes control B... NOTE: The X X matrix has been found to be singular, and a generalized inverse was used to solve the normal equations. Terms whose estimates are followed by the letter B are not uniquely estimable. place splenectomy splenectomy*place Computing expected values 53 / 68 Expected fibrinogen levels 54 / 68 The reference levels are place=control, splenectomy=yes (as SAS chooses the reference levels as last level based on alphabetic ordering) so the expected fibrinogen level for these animals is intercept= For all other groups, we have to add one or more extra estimates, as shown in the table below: place splenectomy control altitude yes = no = = Note: expected value for splenectomy=no, place=altitude - rounding issue

15 59 / / 68 Model checking Variance homogeneity may be judged from a one-way anova Normality assumption for residuals Result from proc univariate normal) The GLM Procedure Class Level Information Class Levels Values group 4 no_altitude no_control yes_altitude yes_control Number of observations 34 Levene s Test for Homogeneity of fibrinogen Variance ANOVA of Squared Deviations from Group Means Sum of Mean Source DF Squares Square F Value Pr > F group E Error E Tests for Normality Test --Statistic p Value Shapiro-Wilk W Pr < W Kolmogorov-Smirnov D Pr > D > Cramer-von Mises W-Sq Pr > W-Sq Anderson-Darling A-Sq Pr > A-Sq Conclusion No reason to suspect non-normality No reason to suspect inhomogeneity Mode simplification 57 / 68 Assessing the main effects 58 / 68 In the two-way anova, the interaction was not significant (P=0.77), so we omit it from the model: proc glm data=ex_fibrinogen; class splenectomy place; model fibrinogen=place splenectomy / solution clparm; Dependent Variable: fibrinogen Sum of Source DF Squares Mean Square F Value Pr > F Model <.0001 Error Corrected Total R-Square Coeff Var Root MSE fibrinogen Mean Source DF Type III SS Mean Square F Value Pr > F place splenectomy Standard Parameter Estimate Error t Value Pr > t Intercept B <.0001 place altitude B place control B... splenectomy no B splenectomy yes B... Parameter 95% Confidence Limits Intercept place altitude place control.. splenectomy no splenectomy yes.. Removal of spleen leads to a decrease in fibronogen of approx mg% at day 21 Placing in altitude leads to an increase in fibronogen of approx mg% at day 21

16 63 / / 68 Residual plots More complicated analyses of variances Normality Variance homogeneity Three- or more-sided analysis of variance. Latin squares Residual I A B C II B C A III C A B Expected (Cochran & Cox (1957): Experimental Designs, 2.ed., Wiley) Cross-over designs Variance component models Example of a latin square: A rabbit experiment 61 / 68 Example of a latin square: A rabbit experiment 62 / 68 6 rabbits Vaccination at 6 different spots on the back 6 different orders of vaccination Swelling is area of blister (cm 2 ) spot rabbit order swelling

17 67 / / 68 Illustrations 3-way analysis of variance, with additive effects swelling a b c d e f spot swelling order proc glm; class rabbit spot order; model swelling=rabbit spot order; The GLM Procedure Class Level Information Class Levels Values rabbit spot 6 a b c d e f order Number of observations / / 68 3-way analysis of variance How about possible interactions? Dependent Variable: swelling Sum of Source DF Squares Mean Square F Value Pr > F Model Error Corrected Total R-Square Coeff Var Root MSE swelling Mean proc glm; class rabbit spot order; model swelling=rabbit spot order spot*order; Dependent Variable: swelling Sum of Source DF Squares Mean Square F Value Pr > F Source DF Type III SS Mean Square F Value Pr > F Model Error Corrected Total rabbit spot order The design is balanced, so the test of the effect of one variable (covariate) does not depend on which of the others are still in the model. Source DF Type I SS Mean Square F Value Pr > F rabbit spot order spot*order There is no room for interaction, since there is only one observation for each combination of spot and order!