Multi-factor analysis of variance

Transcription

1 Faculty of Health Sciences Outline Multi-factor analysis of variance Basic statistics for experimental researchers 2015 Two-way ANOVA and interaction Mathed samples ANOVA Random vs systematic variation Julie Lyng Forman Department of Biostatistics, University of Copenhagen Mixed models Repeatability and reproducibility 2 / 68 Two-way analysis of variance Example: Fibrinogen after spleen removal What is the effect of two treatments in combination? B A 1 2 c 1 2. r. Effect of treatment A? Effect of treatment B? Do the two treatments interact? Randomized experiment with 34 rats. Two treatments: splenectomy (yes/no): 17 have their spleen removed. altitude (high/normal) 8 in each group at high altitude. Outcome: Serum fibrinogen (mg) after 21 days. Quantification / test of interaction Possible with multiple observations for each combination. not possible with one observation for each combination. Note: one measurement on each rat - we have independent data. 3 / 68 4 / 68

2 Data from combined treatment groups Interactionplot Sample means for combined treatments: Altitude Spleen Mean (SD) High No 440 (99) High Yes 352 (93) Normal No 366 (66) Normal Yes 261 (49) 5 / 68 Do we see the same effect of altitude with and without spleen? 6 / 68 Interaction What is interaction? Interaction between two treatments (or factors) means that The effect of the two treatments depend on one another. When interaction is present: Quantify differences between treatment combinations as in a one-way ANOVA. Estimated effects of the one treatment must be presented for each value of the other treatment in turn (and vice versa). Interaction is also called effect modification Because the effect of one treatment is modified by the other. 7 / 68 8 / 68

3 Model with or without interaction Parameter estimates (FIX ME) Model: for k th rat with combination of i th and j th treatment. Y ijk = µ + α i + β j + γ ij + ε ijk µ: mean of reference group (no splenectomy, normal altitude). α: effect of splenectomy (at normal altitude) β: effect of altitude (without splenectomy) γ: possible interaction (i.e. increase/decrease in anticipated effect when the two treatments are combined). ε ijk s are the error terms assumed independent N (0, σ 2 ) In case γ = 0 we say that there is no interaction or that the treatment effects are additive. We can test this as a hypothesis. Model: With interaction Without interaction Effect Estimate (95% CI) Estimate (95% CI) Intercept (312.8;419.5) (316.5;408.3) Altitude 74.8 (-3.0;152.5) 82.7 (28.6;136.8) Splenectomy (-179.9;-29.0) (-151.0;-43.0) Interaction 15.8 (-94.2;125.8) assumed = 0! Hence the estimated effect of altitude is 15.8 higher (95% CI to 125.8) for a rat who has its spleen removed compared to one who has not. The interaction is not significant, but the confidence is wide so we cannot completely rule out a possible effect modification. 9 / / 68 Interpretation of parameter estimates. Expected fibrinogen levels for each treatment combination altitude splenectomy normal high no = = yes = = Here we have added the estimated treatment effects to the mean of the reference group. 11 / 68 Testing interaction in SAS PROC GLM DATA=fibrinogen PLOTS=(DIAGNOSTICS RESIDUALS); CLASS splenectomy (ref= no ) altitude (ref= normal ); MODEL fibrinogen=altitude splenectomy altitude*splenectomy / SOLUTION CLPARM; OUTPUT OUT=fit expected=predicted; 12 / 68 Use PROC GLM for two-way ANOVA Both treatment factors must be declared with CLASS. If reference groups are not chosen SAS uses the last in alphabetic order. Both the main effects and the interaction must be included in the MODEL-statement Options SOLUTION CLPARM prints parameter estimates and confidence intervals. Use the OUTPUT-statement to save fitted values in a dataset.

4 Output: test of any treatment effect Output: test for interaction The GLM Procedure Class Level Information Class Levels Values altitude 2 high norm splenectomy 2 yes no Number of Observations Read 34 Number of Observations Used 34 Dependent Variable: fibrinogen R-Square Coeff Var Root MSE fibrinogen Mean Source DF Type I SS Mean Square F Value Pr > F altitude splenectomy altitude*splenectomy Source DF Type III SS Mean Square F Value Pr > F Sum of Source DF Squares Mean Square F Value Pr > F Model Error Corrected Total altitude splenectomy altitude*splenectomy Type III tests: Drop one factor while keeping the others. Type I tests: (read buttom up) Drop one factor after the other. 13 / / 68 Output: parameter estimates Post hoc testing: Interaction Standard Parameter Estimate Error t Value Pr > t 95% Confidence Limits Intercept B < altitude high B altitude norm B..... splenectomy yes B splenectomy no B..... altitude*splenectomy high yes B altitude*splenectomy high no B..... altitude*splenectomy norm yes B..... altitude*splenectomy norm no B..... NOTE: The X X matrix has been found to be singular, and a generalized inverse was used to solve the normal equations. estimates are followed by the letter B are not uniquely estimable. The warning means that other parameter estiates could have been obtained if an others reference groups were chosen; not a problem! If interaction is present, compare all treatment combinations with: PROC GLM DATA=fibrinogen; CLASS splenectomy (ref= no ) altitude (ref= normal ); MODEL fibrinogen=altitude*splenectomy; LSMEANS altitude*splenectomy / PDIFF CL; Or assess the effect of each treatment given the other with: Terms whose PROC GLM DATA=fibrinogen; CLASS splenectomy (ref= no ) altitude (ref= normal ); MODEL fibrinogen=altitude*splenectomy; LSMEANS altitude*splenectomy / SLICE=altitude; LSMEANS altitude*splenectomy / SLICE=splenectomy; 15 / / 68

5 Post hoc testing: No interaction Model checking If there is no interaction, the treatment effects are additive. Hence, asses each treatment in turn with: PROC GLM DATA=fibrinogen; CLASS splenectomy (ref= no ) altitude (ref= normal ); MODEL fibrinogen=altitude splenectomy; LSMEANS altitude / PDIFF CL; LSMEANS splenectomy / PDIFF CL; or in this case just use / SOLUTION CL with the model statement since there are only two levels of each treatment factor. The error terms ε st s are assumed to be independent (this we know to be true). normally distributed with zero mean and equal variances Use the residuals for model checking: Probability or QQ-plot of residuals. Plot of residuals vs expected values and/or factors. Any outliers in the data? 17 / / 68 Expected values and residuals Diagnostic plots Expected value: for altitude=normal, splenectomy=yes: ŷ ij = ˆµ + ˆα i + ˆβ j + ˆγ ij = = Residual: for the first rat in this group: ε ijk r ijk = observed expected = y ijk ŷ ij ε st = = / / 68

6 Outline Overview: comparison of treatment groups Two-way ANOVA and interaction Mathed samples ANOVA Random vs systematic variation Mixed models Repeatability and reproducibility number independent paired of groups samples samples 2 unpaired paired t-test t-test 2 one-way two-way analysis of variance analysis of variance or mixed model Analysis of variance: Last week: t-tests and one-way ANOVA. Today: Two-way ANOVA and mixed models. 21 / / 68 Example + exercise: Gene expressions Spaghettigram Four treatments applied to five cell lines (from lecture 2). Treatment ctrl A B C Do we see: differences among treatments? differnces among cell lines? (Is this interesting?) Interaction? (Not possible to test and not interesting) The cell lines should be roughly parallel and equally variable Log-transformed seems better than raw data. 23 / / 68

7 Two-way ANOVA model Test of treatment effect Measurement for subject s with treatment t: Y st = µ + α s + β t + ε st µ is the intercept (mean of reference) α s describe expected differences between cell lines. β s describe expected differences between treatments. The error terms ε st s are assumed to be independent normally distributed with equal variances The model assumptions should be checked / 68 PROC GLM DATA=expression12; CLASS cell_line treatment; MODEL log_expression=cell_line treatment / SOLUTION CLPARM; Source DF Type III SS Mean Square F Value Pr > F cell_line treatment We find significant differences among treatments (interesting) and among cell lines (not that interesting... ). 26 / 68 Parameter estimates Estimates of treatment effect As compared to the control group: Standard Parameter Estimate Error t Value Pr > t 95% Confidence Limits Intercept B < cell_line B cell_line B cell_line B cell_line B cell_line B..... Treatment A B Treatment B B Treatment C B Treatment ctrl B..... NOTE: The X X matrix has been found to be singular, and a generalized inverse was used to solve the normal equations. estimates are followed by the letter B are not uniquely estimable. Note: The control treatment has been chosen as reference. Treatment A, B, and C parameter estimates are expected differences wrt the control... on log-scale!. Treatment log-scale back-transformed A 1.16 (0.58;1.74) +218% (+79%;+467%) B 0.17 (-0.41;0.75) +19% (-33%;+111%) C 0.13 (-0.45;0.70) +13% (-36%;+102%) i.e. treatment A approximately triples the gene expression level 100 {exp( ) 1} 100 ( ) 218. Terms whose Multiple comparisons: could be performed by e.g. adding the following statment to the program: LSMEANS treatment / PDIFF=CONTROL ADJUST=DUNNETT CLPARM; (here for all comparisons to the control). 27 / / 68

8 Outline Gene expressions again Two-way ANOVA and interaction Mathed samples ANOVA Random vs systematic variation Mixed models Repeatability and reproducibility Source DF Type III SS Mean Square F Value Pr > F cell_line Standard Parameter Estimate Error t Value Pr > t 95% Confidence Limits Intercept B < cell_line B cell_line B cell_line B cell_line B cell_line B..... NOTE: The X X matrix has been found to be singular, and a generalized inverse was used to solve the normal equati estimates are followed by the letter B are not uniquely estimable. Overall significant differences in gene expression levels were found among the cell lines (P=0.0212), and estimates show that e.g. cell lines 1, 2, and 3, but not 4, differ significantly from cell line 5. Should this be reported as an interesting finding? 29 / / 68 Fixed and random effects Example: experiment with rabbits Fixed effects such as treatment, dose, and time. Typically a limited number of carefully selected groups. Group names are specific and cannot be shuffled. Each group must have a decent size in order to reach interesting conclusions (statistical power). Random effect such as rat, cell line, experiment or operator. Possibly a large number of different groups. Group names are non-informative (number of rat or cell line) and could be shuffled without consequence. Allows inference to be extended beyond the subjects in the experiment and to the population they were sampled from. The number of groups matters not the size of the groups. R = 6 rabbits vaccinated. In S = 6 spots on the back. Response: swelling in cm 2 Model: Which one? A one-way ANOVA with a random effect; the rabbit-factor 31 / / 68

9 One-way ANOVA with random variation Comparison of k groups/clusters, satisfying: The groups are of no individual interest and it is of no relevance to test whether they have identical means. The groups may be thought of as a random sample from a population, that we want to describe. Example: Swelling was measured 6 times consecutively on a sample of 6 rabbits. 33 / 68 What response can we expect in the population? Test for identical rabbits means: P = (one-way ANOVA) is not very helpfull in this regard, and neither are the estimates of differences between specific rabbits. Mean swelling with 95% CL (and normal range) is better. Random effects ANOVA model Model for response of s th spot on r th rabbit: Y rs = µ + a r + ε rs µ is the grand mean (i.e. of the rabbit population). a r is the between-rabbit deviation (i.e. how does rabbit r deviate from the grand mean). ε rs is the within-rabbit deviation (i.e. how does spot s deviate from its rabbit s mean). It is assumed that all error terms (a r s and ε rs s) are independent and normally distributed: a r N (0, ω 2 B), ε rs N (0, σ 2 W ) The deviations between rabbits are considered random and their variance ωb 2, is called the between-rabbit variance component. 34 / 68 Implications of random effects anova Each single observations is sampled from the same population assumed to follow the normal distribution: Y rs N (µ, ω 2 B + σ 2 W ) Population mean µ (the grand mean). Population variance ω 2 B + σ2 W (the total variation). But: Measurements made on the same rabbit are correlated with the so-called intra-class correlation More about correlation next lecture Parameter estimates Grand mean (µ): 7.37 (6.68;8.05). Variance components: Variation Variance component Estimate (95% CI) %of variation Between ω 2 B 0.33 (0.03; 1.04) 36% Within ω 2 W 0.58 (0.06; 2.48) 64% Total ω 2 B + σ2 W % Corr(y r1, y r2 ) = ρ = ω 2 B ω 2 B + σ2 W Warning: Confidence intervals for the variance components may be invalid due to the tiny sample size (only six rabbits). I.e. measurements made on the same rabbit tend to look more alike than measurements made on different rabbits 35 / / 68

10 Interpretation of variance components Typical difference between spots on the same rabbit: y rs1 y rs2 = µ + α r + ε rs1 (µ + α r + ε rs2 ) = ε rs1 + ( ε rs2 ) N (0, 2 ω 2 W ) Normal region: ± = ± 2.16 cm 2 Typical difference between spots on different rabbits: y r1 s 1 y r2 s 2 = α r1 + ( α r2 ) + ε rs1 + ( ε rs2 ) N (0, 2 (σ 2 B + ω 2 W )) Normal region: ± 2 2 ( ) = ± 2.70 cm 2 Note: If Y 1 N (µ 1, σ 2 1) and Y 2 N (µ 2, σ 2 2) are independent normal variables, then their difference is normal Y 1 Y 2 N (µ 1 µ 2, σ σ 2 2). 37 / 68 SAS: PROC MIXED PROC MIXED DATA=rabbit; CLASS rabbit; MODEL swelling = / SOLUTION CL DDFM=SATTERTHWAITTE; RANDOM rabbit; 38 / 68 Syntax is similar to PROC GLM with a MODEL-statement specifying the relationship between outcome and covariates. Categorical variables must be declared with CLASS. Random effects are specified in a RANDOM-statement. The technical option DDFM=SATTERTHWAITTE ensures that correct degrees of freedom are used in computation e.g. when data is unbalanced. When in doubt, use it! SAS: proc mixed output SAS: proc mixed output Model Information Data Set Dependent Variable Covariance Structure Estimation Method Residual Variance Method Fixed Effects SE Method Degrees of Freedom Method Class Level Information Class Levels Values rabbit Dimensions Covariance Parameters 2 Columns in X 1 Columns in Z 6 Subjects 1 Max Obs Per Subject / 68 WORK.A2 swelling Variance Components REML Profile Model-Based Satterthwaitte Number of Observations Number of Observations Read 36 Number of Observations Used 36 Number of Observations Not Used 0 Iteration History Iteration Evaluations -2 Res Log Like Criterion Convergence criteria met. Always check that numerical optimisation has converged. 40 / 68

11 SAS: proc mixed output Negative variance components Covariance Parameter Estimates Cov Parm Estimate rabbit Residual Fit Statistics -2 Res Log Likelihood 91.5 AIC (smaller is better) 95.5 AICC (smaller is better) 95.9 BIC (smaller is better) 95.1 Solution for Fixed Effects Effect Estimate StdError DF t Value Pr > t Alpha Lower Upper Intercept < Finally: Parameter estimates, tests, and some overall goodness of fit 41 / 68 statistics (that could be used to compare to other models). Warning: It may happen that SAS reports a zero-estimate for the variation between, ω 2 B. By coincidence. Thus the model is OK. As a result of competition within clusters. 42 / 68 Example: yield of plants grown in the same pot. Thus, the model is wrong as the clustering leads to dissimilarities (negative correlation) rather than similarities (positive correlation) in outcome. Comparison of modeling strategies Comments on the strategies: Quantifying overall swelling Four strategies for estimating the grand mean of the rabbit population method estimate (s.e.) 1: forget rabbit (0.155) 2: fixed rabbit (0.127) 3: rabbit averages (0.267) 4: random rabbit (0.267) 1. We (wrongfully) assume independence all 36 measurements 2. We estimate the mean swelling by classical one-way anova. 3. We reduce the data to six averages from the individual rabbits and then compute mean and SE. 4. We estimate the mean swelling in the random effects anova model. 1. Ignoring the clustering is wrong! leads to systematic underestimation of the standard error. 2. In the fixed effect one-way anova the grand mean has a different interpretation!... as the mean swelling of these six particular rabbits. leads to systematic underestimation of the standard error. 3. Looking at the sample of averages may be OK. At least in balanced designs (otherwise the individual averages have unequal variances and the standard error may be affected) But we loose information on within subject variation. 43 / / 68

12 Unbalanced data We delete the 3 smallest measurements from rabbit 2 (largest level) so that the data becomes unbalanced and the results change: method estimate (s.e.) 1: forget rabbit (0.163) 2: fixed rabbit (0.136) 3: rabbit averages (0.333) 4: random rabbit (0.298) Full sample (0.267) 1 we have omitted some of the largest observations 2 rabbit 2 has a lower weight in the average (only 3 observations) 3 average for rabbit 2 has increased 4 rabbit 2 has a lower weight in the average due to a larger standard error Design considerations Plan an experiment with: R rabbits (independent or true replicates). S spots for each rabbit (repeated measurements or pseudo replicates). R S measurements. Then variance of mean estimate var(ȳ) = ω2 B R + σ2 W RS, decreases with R and S. standard error rabbits The different curves correspond to S varying from 1 to / / 68 Effective sample size Outline How many rabbits would we need to obtain the same precision in estimating the grand mean if we had only one measurement on each of R 1 rabbits? Solve the equation for var(ȳ): R 1 = R S 1 + ρ(s 1) where ρ is the within rabbit correlation. Two-way ANOVA and interaction Mathed samples ANOVA Random vs systematic variation Mixed models Estimate: ρ = ω2 B ω 2 B +σ2 W = = R 1 = 12.8 Repeatability and reproducibility I.e. one measurement on each of thirteen rabbits gives the same precision as six measurements on each of six rabbits! 47 / / 68

13 Linear mixed models Generalisations of ANOVA and GLM models involving both fixed effects (covariates) and several sources of random variation, the so-called variance components. Environmental variation. Between clinics, regions or countries. Biological variation. Between patients, animals, or cell lines. Within-individual variation. Between injection sites, tumors, slices. Variation due to uncontrollable circumstances. E.g. day to day, assay, observer. Measurement error. E.g. duplicates, triplicates. Multi-level models Mixed models are also called variance component models. Often we have a multi-level model with hierarchical ordering of the levels. We have variation (i.e. a variance component) on each level. And possibly fixed effects (covariates) on each level. individual context/cluster context/cluster level 1 level 2 level 3 spots rabbits slices tumors mice duplicates experiments operators Arrows indicate simplification or grouping. 49 / / 68 Merits of mixed models Drawbacks of mixed models We get a better understanding of the various sources of variation. Certain effects may be estimated more precisely (higher power), since some sources of variation are eliminated, e.g. by making comparisons within the same subject. This is analogous to the paired comparison situation. When planning subsequent investigations, the knowledge of the relative sizes of the variance components will (in principle) be of help in deciding the number of repetitions needed at each level. Independent (sometimes called true) replicates Repeated measurements (called pseudo replicates) Their statistical analysis is more difficult. When making inference (estimation and testing), it is important to take all sources of variation into account. Results may be biased if one or more sources of variation are disregarded! Only few statistical software can do the correct analyses. 51 / / 68

14 Testing fixed effects Testing fixed effects with PROC MIXED Imagine that rabbits are grouped in two (e.g. treatments): level variation covariates 1 within rabbit spot 2 between rabbits group Part of the variation between rabbits could be explained by systematic differences between groups. Part of the variation within rabbits could be explained by systematic differences between spots. PROC MIXED DATA=rabbit; CLASS grp rabbit spot; MODEL swelling = grp spot / SOLUTION CL DDFM=SATTERTH; RANDOM rabbit; Output: Covariance Parameter Estimates Cov Parm Estimate rabbit < smaller than before Residual < smaller than before 53 / / 68 Testing fixed effects with PROC MIXED Type 3 Tests of Fixed Effects Num Den Effect DF DF F Value Pr > F group spot Solution for Fixed Effects Effect spot group Estimate StdError DF t Value Pr > t Alpha Lower Upper Intercept group group spot a spot b Disregarding repeated measurements When the random rabbit variation is ignored: PROC GLM DATA=rabbit; CLASS group spot; MODEL swelling=group spot / SOLUTION CLPARM; Source DF Type III SS Mean Square F Value Pr > F group spot Standard Parameter Estimate Error t Value Pr > t 95% Confidence Limits Intercept B < group B group B..... spot a B spot b B / 68 Too small standard errors for estimates of difference between groups and too large standard errors for estimates of differences 56 / 68 between spots!

15 Summary Outline Measurements belonging together in the same cluster tend to look alike (they are correlated). If we fail to take thid into account, we will experience: Possible bias in estimates (in unbalanced data). Too small standard errors (type 1 error) for estimates of level 2 effects (between-cluster effects). Too low efficiency (type 2 error) for evaluation of level 1 covariates (within-cluster effects) Two-way ANOVA and interaction Mathed samples ANOVA Random vs systematic variation Mixed models Repeatability and reproducibility 57 / / 68 Comparing measurement devices Illustration of all data Example: Peak expiratory flow rate, l/min: 17 subjects, 2 measurement devices, two replicates with each method. subject Wright mini Wright id Y 1p1 Y 1p2 Y 2p1 Y 2p Average SD Reference: Bland and Altman, Lancet (1986). 59 / / 68

16 Aim of investigation Simple approaches Quantify the precision of each measuring device Repeatability (variability=measurement error) Quantify the agreement between the two devices. Bias of one method compared to the other. Variance of one method compared to the other. Can the devices be used interchangably? For reliability of each method separately we could: make Bland Altman plots of differences vs averages. compute limits of agreement, i.e. the 95% normal range of the differences. For reproducibility (method comparison) we might: compare the averages in a Bland-Altman plot? Not good - unless you also do averages in clinic! For both at the same time: Mixed model for variance between and within methods. 61 / / 68 Variance component models Stratified analyses For each method (i = 1, 2) we have a variance component model Y ijk = µ i + a ij + ε ijk µ i population mean as anticipated by method i. a ij deviation of subject j from population mean, assumed normally distributed N (0, σ 2 i ). ε ijk deviation for replicate k (measurement error), assumed normally distributed N (0, ω 2 i ). PROC MIXED DATA=wright; BY method; CLASS id; MODEL flow = / SOLUTION CL; RANDOM id; method=mini Cov Parm Subject Estimate Intercept id Residual Effect Estimate Error DF t Value Pr > t Alpha Lower Upper Intercept < method=wright Cov Parm Subject Estimate Intercept id Residual Effect Estimate Error DF t Value Pr > t Alpha Lower Upper Intercept < / / 68

17 Joint model for both methods For methods (i = 1, 2): Y ijk = µ i + a ij + ε ijk ε ijk assumed normally distributed N (0, ω 2 i ) and independent across methods. a ij assumed normally distributed N (0, σ 2 i ) and correlated with ρ = Cor(a i1, a i2 ). Anticipated means for the same subject ought to look a lot like each other, so the a ij s are likely to be correlated across methods. Note that SAS outputs the covariance parameter σ 12 = Cov(a 1j, a 2j ) = σ 1 σ 2 ρ. Advanced analysis From the statistical analysis of repeated measurements-course. PROC MIXED DATA=wright; CLASS method id; MODEL flow=method / SOLUTION CL; RANDOM method / TYPE=UN SUBJECT=id; REPEATED / TYPE=simple GROUP=method SUBJECT=id*method; Covariance Parameter Estimates Cov Parm Subject Group Estimate UN(1,1) id UN(2,1) id UN(2,2) id Residual method*id method mini Residual method*id method wright Solution for Fixed Effects Effect method Estimate StdError DF t Value Pr > t Alpha Lower Upper Intercept < method mini method wright / / 68 Repeatability Reproducibility Typical differences (approximate 95% normal range) between two measurement with the same method: Wright: ˆω 2 1 = ±2 2ω 2 1 ±43.3 Mini: ˆω 2 2 = ±2 2ω 2 2 ±56.3 Seemingly Wright is more precise, but is the difference significant? F = = 1.69 F(17, 17) P = 0.14 Don t form too firm a conclusion with too small data. No evidence of systematic differences between the two methods. Estimated bias +6.0 (-10.4;22.4) for mini vs wright. P=0.46. Typical differnces between the two methods: var(y 1jk Y 2jk ) = var(a 1j a 2j + ε 1jk ε 2jk ) = σ σ 2 2 2σ 12 + ω ω 2 2 = = Limits-of-agreement: 6.03 ± = ( 69.3, 81.3). 67 / / 68