One-way ANOVA Model Assumptions STAT:5201 Week 4: Lecture 1 1 / 31
One-way ANOVA: Model Assumptions Consider the single factor model: Y ij = µ + α }{{} i ij iid with ɛ ij N(0, σ 2 ) mean structure random Here, the assumptions are coming from the errors: 1 Normally distributed 2 Constant variance 3 Independent We will use the estimated errors ˆɛ ij = Y ij Ŷij or residuals from the model to check the assumptions. (Though internally studentized residuals or externally studentized residuals can also be used.) 4 Adequacy of the model is about the mean structure, and here, we are fitting the most complex mean structure for this single factor study, which is a separate mean for each group, so in the full model, this shouldn t be a concern. 2 / 31
One-way ANOVA: Model Assumptions Consider the single factor model: Y ij = µ + α }{{} i ij iid with ɛ ij N(0, σ 2 ) mean structure random If the model is correct, our inferences are good. If the assumptions are not true, our inferences may not be valid. Confidence intervals might not cover at the stated level. p-values are not necessarily valid. Type I error rates could be larger (or smaller) than stated. Assumptions need to be checked for the validity of the tests. 3 / 31
One-way ANOVA: Model Assumptions Consider the single factor model: Y ij = µ + α }{{} i ij iid with ɛ ij N(0, σ 2 ) mean structure random Some procedures work reasonably well even if some of the assumptions are violated (we ll explore this for the two-sample t-test in homework). This is called robustness of validity. More mild violations are of course better than more extreme violations, with respect to validity. 4 / 31
One-way C(.dl~ ANOVA: Checking Constant Variance c~ VfVt/ov~ Checking constant variance Plot residuals vs. fitted values PLo-/ 01- rm/~ V5, /I/J V~. 7f f'i1 ozid! OK I s.~ 5 /'a.d~ :;ea IIPi I/;:; If the model is OK for constant variance, then this plot should show a random 1jJN'ds scattering ~~ of points ~ above ~Iow andfk- belowvp[-/-'w the reference r~ line at a horizontal 0, as on the left below. The right one shows nonconstant variance. ;;~ a/~ a. hoy'" 2-crJd 0, r. e.. ~ d.,... " 0,, " #.. ' r:,. r ',, +;.\\--J v~ '" y LlVheJ ~ /J ~ pet 4~VY':'\ (=? v"" itl VIee Jl/2 f.v.d.s 6Yl ~ <'am ) Megaphone pattern violation (variance depends on mean) 5 / 31
One-way ANOVA: Checking Constant Variance There are some statistical tests that will perform a hypothesis test for the equality of variances across groups. H 0 : σ 2 i is equal for all i. Levene s Test Brown-Forsythe Test (a.k.a. modified Levene s test) Example (SAS: levene s test) proc glm data=upsit plots=diagnostics; class agegroup; model smell = agegroup; means agegroup / hovtest=levene; run; For the other homogeneity of variance (HOV) tests in SAS, just google SAS HOVTEST. You can use HOVTEST=BF for brown-forsythe test. 6 / 31
R-Square Coeff Var Root MSE smell Mean 0.275670 14.52664 0.179200 1.233594 One-way ANOVA: Checking Constant Variance Source DF Type I SS Mean Square F Value Pr > F agegroup 4 2.13878141 0.53469535 16.65 <.0001 Example Source (SAS: DF Type III SSlevene s Mean Square F Value Pr > test) F agegroup 4 2.13878141 0.53469535 16.65 <.0001 Distribution of smell 1.4 F 16.65 Prob > F <.0001 1.2 The SAS System 12:31 Thursda smell 1.0 The GLM Procedure 0.8 Levene's Test for Homogeneity of smell Variance ANOVAof Squared Deviations from Group Means 0.6 Source DF Sum of Squares Mean Square F Value Pr > F agegroup 4 0.0799 0.0200 6.35 <.0001 1 2 3 4 5 agegroup Error 175 0.5503 0.00314 The plot suggests we have nonconstant variance, and the null hypothesis of H 0 : σ 1 = σ 2 = σ 5 is strongly rejected by Levene s test. Oehlert has some issues with using statistical tests for nonconstant variance due to sensitivity to non-normality, but you may still be asked about these tests. 7 / 31
One-way ANOVA: Model Adequacy Plot residuals vs. fitted values The residuals vs. fitted plot can also give you some information about the adequacy of the model in a multi-factor ANOVA (we ll see this later in multi-factor factorials, plot shown below). For instance, if you are missing an important interaction term in the mean structure, then this plot will often display a curved trend. 8 / 31
One-way ANOVA: Dealing with Nonconstant Variance Dealing with nonconstant variance When the variance depends on the mean (like in the megaphone pattern), the usual approach is to apply a transformation to the response variable. These are called variance-stabilizing transformations. Suppose Var(y) mean or Var(y) µ I want a transformation of y, or function h(y), such that Var[h(y)]=constant. Consider a Taylor Series expansion of h around µ 9 / 31
One-way ANOVA: Dealing with Nonconstant Variance We now have the first-order approximation: Var[h(y)] Var[h(µ) + h (µ)(y µ)] = [h (µ)] 2 Var(y) = c 0 [h (µ)] 2 µ {as Var(y) µ} And we want Var[h(y)] to be a constant. So, set [h (µ)] 2 µ equal to a constant and solve for h. 10 / 31
One-way ANOVA: Dealing with Nonconstant Variance Setting equal to a constant and solving for the unknown h: [h (µ)] 2 µ =constant constant h (µ) = µ 1 h(µ) =c 1 dµ µ 1/2 h(µ) =c 2 µ So, if Var(y) µ, then use a square root transformation to achieve constant variance. 11 / 31
One-way ANOVA: Dealing with Nonconstant Variance This is built on the Delta Method. If Var(y) mean, use h(y) = y. If Var(y) mean 2, use h(y) = ln(y) Many times it s hard to tell from the data what the specific relationship between the variance and the mean is, so a trial-and-error process is applied. Other possibilities if the spread increases with µ: h(y) = 1 y. h(y) = log 10 (y) Other possibilities if the spread decreases with µ: h(y) = y 2. h(y) = y 1.5 12 / 31
One-way ANOVA: Dealing with Nonconstant Variance The Box-Cox procedure chooses a transformation based on the observed data. The λ parameter dictated the transformation. The following form for the transformation is suggested to create the new outcome variable y (λ) : y (λ) = { y λ 1 λ when λ 0 log(y) when λ = 0 Though λ is continuous, in practice we usually use a convenient λ that is near to the optimal, like 0.5 or 0, etc. 13 / 31
One-way ANOVA: Dealing with Nonconstant Variance Example (SAS: Box-Cox for 2-factor factorial, perceived as single factor or a superfactor. Weeks has 5 levels, Water has 2 levels.) proc transreg data=germ; model boxcox(germination)=class(superfactor); run; A transformation using λ = 0.25 is suggested, but the convenient λ = 0.5 looks to be in the confidence interval (we will check constant variance after the transformation). 14 / 31
One-way ANOVA: Dealing with Nonconstant Variance Example (R: Box-Cox for 2-factor factorial, perceived as single factor or a superfactor. Weeks has 5 levels, Water has 2 levels.) > library(mass) > bc.fit <- boxcox(germination as.factor(superfactor)) > bc.fit 15 / 31
One-way ANOVA: Dealing with Nonconstant Variance So, if we know the proportional relationship between the variance and the mean, then we can analytically find an appropriate transformation to achieve constant variance (or near constant variance). When we do not know the relationship, then we can apply the Box-Cox transformation to give us a suggestion of an appropriate transformation. In practice, if I observe a nonconstant variance that can be corrected through transformation (not all of them are), I mostly see variance increasing with the mean, and I just try a square-root or log transformation right away (or log(y + 1) if there are zeros.). REMINDER: In a two-sample t-test with nonconstant variance, we do have an availalbe method called Welch s Approximate t or or Welch-Satterthwaite t. 16 / 31
One-way ANOVA: Checking Normality Checking Normality This is usually done by plotting a normal probability plot or normal QQ-plot. If the data were generated from a normal distribution, then the normal probability plot will show the data points falling approximately along the diagonal reference line (this is not a best-fit line, it simply connects the 25th and 75th percentile points). 17 / 31
One-way ANOVA: Checking Normality Chi-squared, right-skewed Chi-squared (df=2) Normal Q-Q Plot Frequency 0 100 200 300 Sample Quantiles 0 2 4 6 8 10 0 5 10 15 rchisq(1000, 2) -3-2 -1 0 1 2 3 Theoretical Quantiles Uniform, thin tails Uniform(0,1) Normal Q-Q Plot Frequency 0 10 20 30 40 50 60 Sample Quantiles 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 y -3-2 -1 0 1 2 3 Theoretical Quantiles 18 / 31
One-way ANOVA: Checking Normality There are a number of statistical tests that test for non-normality: Anderson-Darling test Shapiro-Wilk test Many others One issue with normality tests is that as your N gets larger, you start to get a lot of power for detecting very small deviations from normality. In small samples, you ll probably never reject. In practice, I feel like the visual normal probability plot is most useful. But clients will commonly ask how to perform certain tests, such as these, in software. 19 / 31
One-way ANOVA: Dealing with Non-normality Dealing with Non-normality Try a transformation. If there s an outlier that a transformation does not fix, do a sensitivity analysis where you perform the analysis with and without the outlier. These can both be reported. If the important items do not change (e.g. significance) the outlier is perhaps not a big issue. DO NOT simply remove an outlier because it is unusual!! Not OK. (See link on webpage). And in fact, that data point could be telling you something very interesting. Try a non-parametric test: Randomization test (pretty general) Wicoxon rank-sum test/mann-whitney test (for 2-sample t-test) Wilcoxon signed-rank test (for paired t-test) Kruskal-Wallis test (for 1-way ANOVA, extends Mann-Whitney test) Freidman test (for RCBD) 20 / 31
One-way ANOVA: Checking Independence Checking Independence Many times, a client brings the data to you and you have to rely on their description of the data collection, and that independence holds. Ideally, it should be part of the design in terms of randomly assigning treatments to EUs and randomly assigning the order of the runs. If you happen to know the order in which data were collected, or the time the observations were collected, it s a good idea to check for correlation in the residuals with respect to order or time (e.g. plot resids vs. time and/or resids vs. order). 21 / 31
One-way ANOVA: Checking Independence If a pattern appears in the plots, then these items are sources of variation that can be added to the model (though it makes the model a bit more complex). Above we see sequential observations over time, and an observable trend. 22 / 31
One-way ANOVA: Checking Independence The Durbin-Watson test statistic can be used to check for time dependence or serial dependence. The residuals e i are used to calculate DW: DW = n 1 i (e i e i 1 ) 2 n i e 2 i Independent data tend to have DW around 2. A positive correlation makes DW smaller and negative correlation makes it bigger. If DW gets as low as 1.5 or as high as 2.5, you should start worrying about time correlation and it s affect on the inference. 23 / 31
One-way ANOVA: Checking Independence Example (Checking residuals for time correlation) The SAS data set diags contains a vector of residuals under the column name resid. Below, nothing is listed as a predictor in the model statement because only an intercept is used. proc reg data=diags; model resid= /DW; The SAS System 14:15 Thursday, February 1, 2018 2 run; The REG Procedure Model: MODEL1 Dependent Variable: Resid Residual Durbin-Watson D 1.319 Number of Observations 48 1st Order Autocorrelation 0.334 An approximate 95% CI for ρ is 0.334 ± 2 1 n or 0.334 ± 0.289, so there appears to be correlation in the residuals over time. 24 / 31
One-way ANOVA: Dealing with Dependence Another type of correlation is a spatial correlation, which could be checked using a variogram. If there is some kind of non-independence, we should incorporate this into our model. Perhaps there is time-correlation or spatial-correlation, and we have models that will incorporate this correlation structure. If there are repeated measures on a single subject, then this also represents correlation (within the observations on a person), and we can incorporate that into our model. If the residuals are non-independent because you were missing an important factor, then we can include that factor in the model. 25 / 31
Importance of Assumption Violations Biggest Issue: Non-independence Standard errors for treatments can be biased, and this can greatly affect the type I error rate of our test. Next Biggest Issue: Nonconstant variance If you have balanced data, then the affect on p-values is potentially small. For unbalanced data, the error rates can be greatly affected. Smallest Issue: Non-normality If you have moderate non-normality, the p-values are only slightly affected. If it s very non-normal, inference can be affected. Similarly, one very strong outlier can greatly affect the results. Again, this will have the least impact on error rates in balanced data. 26 / 31
Checking Assumptions: Example Example (Response time for circuit types) Returning to our previous 1-way ANOVA example to check assumptions... Three different types of circuit are investigated for response time in milliseconds. Fifteen are completed in a balanced CRD with the single factor of Type (1,2,3). Circuit Type Response Time 1 9 12 10 8 15 2 20 21 23 17 30 3 6 5 8 16 7 From D.C Montgomery (2005). Design and Analysis of Experiments. Wiley:USA 27 / 31
Checking Assumptions: Example Example (Response time for circuit types) Normality looks violated. 28 / 31
Checking Assumptions: Example Example (Response time for circuit types) We ll apply the natural log-transformation and perform the 1-way ANVOA on the transformed response. 29 / 31
Checking Assumptions: Example Example (Response time for circuit types) Constant variance seems to be worse here. We will go back and try a nonparametric test on the original data. 30 / 31
Checking Assumptions: Example Example (Response time for circuit types) Perform a 1-way ANOVA using a nonparametric test: Kruskal-Wallis. Kruskal-Wallis Test Chi-Square 10.3735 DF 2 Asymptotic Pr > Chi-Square 0.0056 Exact Pr >= Chi-Square 0.0005 Reject H 0 : α i = 0 for all i, where H A : at least one group different 31 / 31