Analyzing More Complex Experimental Designs

Transcription

1 Analyzing More Complex Experimental Designs

2 Experimental Constraints In the real world, you may find it impossible to obtain completely independent samples We already talked about some ways to handle simple situations: paired tests and blocked designs, for instance Things can get more complex, requiring even more sophisticated models

3 Some Important Models Randomized Block Design Repeat the entire experiment under different values of the blocking variable, which is a nuisance variable You don t care about the effects of the blocking variable (if you do care, then you want a factorial design) Repeated Measures Design Measure the same individual under different conditions Heart rate resting, on stationary bike, on treadmill for each of 20 people taking or not taking a drug Nested Design Some values of one treatment only occur nested within values of another treatment Testing the effects of drug 1 and drug 2 with rats in many different cages cages are nested within drug

4 Example of Randomized Block Design Effects of Salinity on Plant Growth From Six different salt treatments 10, 15, 20, 25, 30, 35 ppt. Four blocks located in slightly different locations: (randomized using random.org)

5 Traditional Analysis Two-factor ANOVA without interaction term Usual assumptions: Normality within groups Homoscedasticity R code: lm or aov, like any two-factor ANOVA

6 Randomized Block Example Obs salt block biomass Decide whether you want your factors to be treated as numerical or categorical (i.e., salt and block) -Depending on your decision, you will get different answers -Block has no inherent meaning, so if you use numbers to indicate blocks, you must tell R to treat them categorically > salt_data$block <- factor(salt_data$block) > class(salt_data$block) [1] factor -If you think there s a linear relationship between salt and growth rate, you can leave it as an ordered, numerical variable. Otherwise: > salt_data$salt <- factor(salt_data$salt)

7 Randomized Block Example > salt_lm <- lm(biomass ~ block + salt, data=salt_data) > anova(salt_lm) Analysis of Variance Table Response: biomass Df Sum Sq Mean Sq F value Pr(>F) block *** salt e-06 *** Residuals Signif. codes: 0 *** ** 0.01 *

8 Repeated Measures ANOVA This design is used when the same subject is measured multiple times under different conditions It s kind of like a paired t-test, but with more than two conditions The model can also be more complex, with interaction terms and multiple factors within subjects Basically, we are correcting for baseline differences among subjects

9 Repeated Measures ANOVA Example from Experiment: Five subjects are asked to memorize a list of words Words are of three types: positive, negative and neutral Response variable is the number of words of a given type recalled

10 Repeated Measures Example Observation Subject Valence Recall 1 1 Jim Neg Jim Neu Jim Pos Victor Neg Victor Neu Victor Pos Faye Neg Faye Neu Faye Pos Ron Neg Ron Neu Ron Pos Jason Neg Jason Neu Jason Pos 35

11 Repeated Measures Example To run a repeated measures ANOVA, you have to tell R how to partition the error. Recall that ANOVA is all about the error terms (e.g., Mean Squared Error), which are deviations from the predictions of the model. R code: > aov_rm_1 <- aov(recall ~ Valence + Error(Subject/Valence), data=rm_data) > summary(aov_rm_1)

12 Repeated Measures Output Error: Subject Df Sum Sq Mean Sq F value Pr(>F) Residuals Error: Subject:Valence Df Sum Sq Mean Sq F value Pr(>F) Valence e-07 *** Residuals Signif. codes: 0 *** ** 0.01 * No test for subject we don t care Significant effect of Valence (type of word) Get the means for the different levels of the factor: > print(model.tables(aov_rm_1,"means"),digits=3)

13 Repeated Measures Example 2 X Observation Subject Task Valence Recall Jim Free Neg Jim Free Neu Jim Free Pos Jim Cued Neg Jim Cued Neu Jim Cued Pos Victor Free Neg Victor Free Neu Victor Free Pos Victor Cued Neg Victor Cued Neu Victor Cued Pos Faye Free Neg Faye Free Neu Faye Free Pos 12

14 Repeated Measures Example 2 Now there are two factors: Task and Valence The entire experiment is repeated within each subject It s a slightly more complex repeated measures design R code: (almost the same as before) > aov_rm2 <- aov(recall ~ (Task*Valence)+Error(Subject/(Task*Valence)), data=rm2_data) > summary(aov_rm2)

15 Repeated Measures Example 2 Error: Subject Df Sum Sq Mean Sq F value Pr(>F) Residuals Error: Subject:Task Df Sum Sq Mean Sq F value Pr(>F) Task Residuals Signif. codes: 0 *** ** 0.01 * Error: Subject:Valence Df Sum Sq Mean Sq F value Pr(>F) Valence Residuals Error: Subject:Task:Valence Df Sum Sq Mean Sq F value Pr(>F) Task:Valence Residuals

16 Repeated Measures Assumptions The distribution of the response variable should be approximately normal within treatments Sphericity: the variances of the differences between all combinations of related groups must be equal (can use Mauchly s test of sphericity) Balanced design: P-values may be inaccurate for unbalanced designs

17 Nested ANOVA A nested design is necessary when values of one factor are nested within values of another factor In other words, some values of the nested factor only occur for one value of the factor higher in the hierarchy

18 Nested ANOVA Example In principle, there can be many more levels in the nesting hierarchy (really any number)

19 Nested ANOVA Example From: Two technicians taking measurements of uptake of fluorescently labeled protein in rat kidneys The question is whether the measurements made by the two technicians are significantly different (so technician is the factor we care about) Each technician uses different rats and makes multiple measurements per rat kidney, so rat is nested within technician

20 Nested ANOVA Example Technician: Brad Janet Rat: Arnold Ben Charlie Dave Eddy Frank

21 Nested ANOVA Example Tech Rat Protein 1 Janet Janet Janet Janet Janet Janet Janet Janet Janet Janet Janet Janet Janet Janet Once again, it s all about telling R how to handle the error. In this case, we need to tell it that when we test for an effect of Tech, we are using the error at the level of Rat. > rat_data <- read.csv("rat_nested_ex.csv") > rat_aov <- aov(protein ~ Tech + Error(Rat), data=rat_data) > summary(rat_aov) Error: Rat Df Sum Sq Mean Sq F value Pr(>F) Tech Residuals Error: Within Df Sum Sq Mean Sq F value Pr(>F) Residuals

22 Other Experimental Designs Latin-Square Design: Split-Plot Design:

23 Summary Lots of different types of experimental designs Many can be analyzed with standard linear models Normality within groups and balanced designs are common assumptions If these assumptions cannot be met, then there s another class of model: Generalized Linear Models and Generalized Linear Mixed Models (which are different than general linear models)

24 Fixed versus Random Effects Fixed: an explanatory variable is a fixed effect if the groups are predefined and of direct interest Random: an explanatory variable is a random effect if the groups are randomly sampled from a population of possible groups

25 Example of Fixed versus Random Effects We are interested in the effects of a drug on disease progression the drug dose is a fixed effect The drug is being administered at 10 randomly chosen clinics the clinic is a random effect Random effects are often nuisance variables of no intrinsic interest

26 GLMM A generalized linear model allows response variables to have distributions other than a normal distribution A generalized linear mixed model included fixed and random factors GLMMs accommodate hierarchical designs and repeated measures The model-fitting procedure usually involves maximum likelihood or restricted maximum likelihood (REML), as opposed to merely calculating sums of squares as in ANOVA.

27 GLMM GLMMs can also accommodate heteroscedasticity GLMMs use distributions in the exponential family, which includes Gaussian (normal), poisson, binomial, negative binomial and Gamma If you have a complex experimental design with random factors, you should probably expect to use a GLMM in your analysis

28 Poisson, Binomial, Negative Binomial Discrete distributions Similar but with different relationships between the mean and variance Binomial: variance is smaller than the mean Poisson: variance equals the mean Negative Binomial: variance is larger than the mean

29 Gamma Distribution Continuous distribution (like Gaussian) Asymmetric (unlike Gaussian)