Topic 23: Diagnostics and Remedies

Topic 23: Diagnostics and Remedies

Outline Diagnostics residual checks ANOVA remedial measures

Diagnostics Overview We will take the diagnostics and remedial measures that we learned for regression and adapt them to the ANOVA setting Many things are essentially the same Some things require modifications

Residuals Predicted values are cell means Yˆ = Y Residuals are the differences between the observed values and their cell means e = Y Yˆ = Y Y ij ij ij ij i ij i

Basic plots Plot the responses vs the factor levels (the values of the explanatory variable X) Plot the residuals vs the factor levels Construct a normal quantile plot and/or histogram of the residuals

KNNL p 777 KNNL Example Compare 4 brands of rust inhibitor (X has r=4 levels) Response variable is a measure of the effectiveness of the inhibitor There are 10 units per brand (n i =n=10)

Fit ANOVA Model proc glm data=a1; class brand; model eff=brand; output out=a2 r=resid; run;

Plots Responses versus the factor Residuals versus the factor Normal quantile plot or histogram of the residuals

Plots vs the factor symbol1 v=circle i=none; proc gplot data=a2; plot (eff resid)*brand; run;

Data vs the factor Means look different common spread in Y s

Residuals vs the factor Odd dist of points

Snake shape due to odd (lack of and large)spread Can try nonparametric analysis last slides

Looks reasonably Normal

Response conditionally assumed to be Normal not marginally. This plot noninformative.

General Summary Look for Outliers Variance that depends on level Non-normal errors Plot residuals vs time and other variables if available

Homogeneity tests Homogeneity of variance (also called homoscedasticity) H 0 : σ 12 = σ 22 = = σ 2 r H 1 : not all σ i2 are equal Several significance tests are available.

Homogeneity tests Text discusses Hartley, modified Levene SAS has several including Bartlett s (essentially the likelihood ratio test) and several versions of Levene

Homogeneity tests There is a problem with assumptions ANOVA is robust with respect to moderate deviations from Normality ANOVA results can be sensitive to the homogeneity of variance assumption Some homogeneity tests are sensitive to the Normality assumption

Levene s Test Do ANOVA on the squared residuals from the original ANOVA Modified Levene s test uses absolute values of the residuals Modified Levene s test is recommended Another quick and dirty rule of thumb ( 2) ( 2) i i Concern if max s / min s > 2

KNNL p 785 KNNL Example Compare the strengths of 5 types of solder flux (X has r=5 levels) Response variable is the pull strength, force in pounds required to break the joint There are 8 solder joints per flux (n=8)

Scatterplot Equal spread?

Levene s Test proc glm data=a1; class type; model strength=type; means type/ hovtest=levene(type=abs); run;

ANOVA Table Source DF Sum of Squares Mean Square F Value Pr > F Model 4 353.612085 88.4030213 41.93 <.0001 Error 35 73.7988250 2.1085379 Corrected Total 39 427.410910 Common variance estimated to be 2.11

Output Levene's Test for Homogeneity of strength Variance ANOVA of Absolute Deviations from Group Means Source Sum of DF Squares Mean Square F Value Pr > F type 4 8.6920 2.1730 3.07 0.0288 Error 35 24.7912 0.7083 We reject the null hypothesis and assume nonconstant variance

Means and SDs Level of strength type N Mean Std Dev 1 8 15.4200000 1.23713956 2 8 18.5275000 1.25297076 3 8 15.0037500 2.48664397 4 8 9.7412500 0.81660337 5 8 12.3400000 0.76941536 s / s = 10.44 >> 2 2 2 3 5

Remedies Delete outliers Is their removal important? Is their removal valid? Use weights (weighted regression) Transformations Nonparametric procedures

What to do here? Not really any obvious outliers Do not see pattern of increasing or decreasing variance or skewed dists Will consider Weighted ANOVA Mixed model ANOVA

Weighted least squares We used this with regression Obtain model for how the sd depends on the explanatory variable (plotted absolute value of residual vs x) Then used weights inversely proportional to the estimated variance

Weighted Least Squares Here we can 1. Compute the variance for each level of X 2. Use these as weights in PROC GLM We will illustrate with the soldering example

Obtain the variances and weights proc means data=a1; var strength; by type; output out=a2 var=s2; data a2; set a2; wt=1/s2; NOTE. Data set a2 has 5 cases

Merge and then use the weights in PROC GLM data a3; merge a1 a2; by type; proc glm data=a3; class type; model strength=type; weight wt; lsmeans type / cl; run;

Output Source DF Sum of Squares Mean Square F Value Pr > F Model 4 324.213099 81.0532747 81.05 <.0001 Error 35 35.0000000 1.0000000 Corrected Total 39 359.213099 Data have been standardized so all groups have variance of 1

LSMEANS Output Because of weights, standard errors simply based on sample variances of each level(but uses df=35!) type strength LSMEAN Standard Error Pr > t 95% Confidence Limits 1 15.4200000 0.4373949 <.0001 14.532041 16.307959 2 18.5275000 0.4429921 <.0001 17.628178 19.426822 3 15.0037500 0.8791614 <.0001 13.218957 16.788543 4 9.7412500 0.2887129 <.0001 9.155132 10.327368 5 12.3400000 0.2720294 <.0001 11.787751 12.892249

Mixed Model ANOVA Relax the assumption of constant variance rather than including a known weight This involves moving to a mixed model procedure Topic will not be on exam but wanted you to be aware of these model capabilities for your own research

SAS Code proc glimmix data=a1; class type; model strength=type / ddfm=kr; random residual / group=type; lsmeans type; run; This allows the variance to differ in each level and a degrees of freedom adjustment is used to account for this

GLIMMIX OUTPUT Fit Statistics -2 Res Log Likelihood 122.11 AIC (smaller is better) 132.11 AICC (smaller is better) 134.18 BIC (smaller is better) 139.88 CAIC (smaller is better) 144.88 HQIC (smaller is better) 134.79 Generalized Chi-Square 35.00 Gener. Chi-Square / DF 1.00 Covariance Parameter Estimates Standard Cov Parm Group Estimate Error Residual (VC) type 1 1.5305 0.8181 Residual (VC) type 2 1.5699 0.8392 Residual (VC) type 3 6.1834 3.3052 Residual (VC) type 4 0.6668 0.3564 Residual (VC) type 5 0.5920 0.3164 Really looks like 3 groups of variances though

GLIMMIX OUTPUT Type III Tests of Fixed Effects Effect Num DF Den DF F Value Pr > F type 4 14.81 71.78 <.0001 type Least Squares Means type Estimate Standard Error DF t Value Pr > t 1 15.4200 0.4374 7 35.25 <.0001 2 18.5275 0.4430 7 41.82 <.0001 3 15.0038 0.8792 7 17.07 <.0001 4 9.7413 0.2887 7 33.74 <.0001 5 12.3400 0.2720 7 45.36 <.0001 Same estimates but now DF have been adjusted for estimating the variances

SAS Code proc glimmix data=a1; class type; model strength=type / ddfm=kr; random residual / group=type1; lsmeans type; run; Variable Type1 was created to identify Type 1 and 2, Type 3, and Type 4 and 5 as 3 different groups

GLIMMIX OUTPUT Fit Statistics -2 Res Log Likelihood 122.13 AIC (smaller is better) 128.13 AICC (smaller is better) 128.91 BIC (smaller is better) 132.80 CAIC (smaller is better) 135.80 HQIC (smaller is better) 129.74 Generalized Chi-Square 35.00 Gener. Chi-Square / DF 1.00 Covariance Parameter Estimates Standard Cov Parm Group Estimate Error Residual (VC) Grp 1 1.5502 0.5859 Residual (VC) Grp 2 6.1834 3.3052 Residual (VC) Grp 3 0.6294 0.2379 Better BIC so better fitting model but same general conclusions

GLIMMIX OUTPUT Type III Tests of Fixed Effects Effect Num DF Den DF F Value Pr > F type 4 19.8 77.68 <.0001 type Least Squares Means type Estimate Standard Error DF t Value Pr > t 1 15.4200 0.4402 14 35.03 <.0001 2 18.5275 0.4402 14 42.09 <.0001 3 15.0037 0.8792 7 17.07 <.0001 4 9.7412 0.2805 14 34.73 <.0001 5 12.3400 0.2805 14 43.99 <.0001 Why are df for some groups 14?

Transformation Guides When σ i2 is proportional to μ i, use When σ i is proportional to μ i, use log(y) Y When σ i is proportional to μ i2, use 1/y For proportions, use arcsin( Y ) arsin(sqrt(y)) in a SAS data step Box-Cox transformation

Example Consider study on KNNL pg 790 Y: time between computer failures X: three locations data a3; infile 'u:\ch18ta05.txt'; input time location interval; proc gplot; plot time*location; run;

Scatterplot Outlier or skewed distribution? Can consider transformation first

Box-Cox Transformation Can consider regression log( s ) vs log( y ) i i and 1-b 1 is the power to raise Y by Can try various convenient powers Can use SAS directly to calculate the power

E(logsig) = 0.90 +.79 logmu Power should be 1-.79 0.20

Using SAS proc transreg data=a3; model boxcox(time / lambda=-2 to 2 by.2) = class(location); run;

Box-Cox Transformation Information for time Lambda R-Square Log Like Output -1.0 0.24-73.040-0.8 0.27-67.330-0.6 0.31-62.316-0.5 0.33-60.144-0.4 0.35-58.239-0.2 0.38-55.346 * 0.0 + 0.39-53.830 * 0.2 0.38-53.769 < 0.4 0.36-55.118 * 0.5 0.34-56.273 0.6 0.32-57.712 0.8 0.29-61.314 1.0 0.25-65.675 < - Best Lambda * - 95% Confidence Interval + - Convenient Lambda

Transforming data in SAS data a3; set a3; transtime = time**0.20; symbol1 v=circle i=none; proc gplot; plot transtime*location; run;

Much more constant spread in data!

Nonparametric approach Based on ranks See KNNL Section 18.7, p 795 See the SAS procedure NPAR1WAY

Rust Inhibitor Analysis Source DF Sum of Squares Mean Square F Value Pr > F Model 3 15953.4660 5317.82200 866.12 <.0001 Error 36 221.03400 6.13983 Corrected Total 39 16174.5000 Highly significant F test. Even if there is a violation of Normality, the evidence is overwhelming

Nonparametric Analysis Wilcoxon Scores (Rank Sums) for Variable eff Classified by Variable abrand abrand N Sum of Scores Expected Under H0 Std Dev Under H0 Mean Score 1 10 128.0 205.0 32.014119 12.80 2 10 355.0 205.0 32.014119 35.50 3 10 255.0 205.0 32.014119 25.50 4 10 82.0 205.0 32.014119 8.20 Average scores were used for ties. Kruskal-Wallis Test Chi-Square 33.7041 DF 3 Pr > Chi-Square <.0001

Last slide We ve finished most of Chapters 17 and 18. We used program topic23.sas to generate the output.