Lecture 7 Remedial Measures

Transcription

1 Lecture 7 Remedial Measures STAT 512 Spring 2011 Background Reading KNNL: , Chapter 4 7-1

2 Topic Overview Review Assumptions & Diagnostics Remedial Measures for Non-normality Non-constant variance Non-linearity Other Miscellaneous Topics (Chapter 4) 7-2

3 Regression Assumptions X and Y are related linearly (scatter plot, residuals vs. X) Assumptions on the Errors... Constancy of Variance (residuals vs. X) Normality (normal probability plot) Independent (sequence plot) 7-3

4 Remedial Measures Two basic choices when assumptions are violated: Use some more appropriate model (often more complicated) Find a transformation of the data for which the regression model is appropriate 7-4

5 Non-linear Relationships Can potentially still use a linear model. For example, Y X X Y ln X This model is still linear in terms of the regression coefficients (parameters). Simply consider a new predictor variable 2 or ln X X, and just treat this like any usual predictor. 7-5

6 Non-linear Relationships Can use nonlinear regression models (beyond the scope of this course, but discussed in Chapter 13). For now, we will try to guess at a good transformation and see if it works. 7-6

7 Variance Not Constant Might be able to model the change in variance (if it is related to X). In this case, can use a weighted analysis (Chapter 11.1) Sometimes a variance-stabilizing transformation can be found (log, square-root are common) Box-Cox procedure can help to find a transformation Note: In this class, we use natural logs, unless specified otherwise 7-7

8 Errors Not Normal Knowledge of error distribution known? If so, can use SAS GENMOD procedure (Chapter 14) Binomial (Yes/No or Categorical Resp.) Poisson (Response is a Count) Knowledge of error distribution unknown? Sometimes a transformation will help Often, non-normality/non-constant variance occur together and transformations can sometimes help both! 7-8

9 Other Remedies Correlated errors (not independent) Use a model for correlated error structure (Chapter 12) Omission of Important Predictors Multiple Regression (starts in Chapter 6) 7-9

10 Other Remedies Outliers Determine whether to keep in analysis (e.g., was there a recording error? Be very cautious of deleting observations!) Determine influence on parameter estimates and standard errors Perform more robust estimation procedure that puts less emphasis on outliers (Chapter 11.3) 7-10

11 Transformations Finding a good transformation gets easier with practice Generally the method is to make an educated guess at a useful transformation and then try it to see if it works by rechecking diagnostic plots Transformations have a tendency to stabilize variance and normality. 7-11

12 Transformations (X) (For nonlinear relationship issues) Log or Square-root Square or Exp(x) Reciprocal or Exp(-x) 7-12

13 Transformations (Y) (For non-constant variance issues) See page 132 Standard transformations if increasing variance: square-root or reciprocal If decreasing variance: log Simultaneous transformations on X may also be useful 7-13

14 Box-Cox Procedure Automated procedure to determine a best power transformation for the response Chooses from different, Y 1 (No transformation) 0.5 (Square Root) 0 (Natural Log) 0.5 (Reciprocal Square Root) 1 (Reciprocal) Use TRANSREG procedure in SAS 7-14

15 Example (1) boxcox.sas X - Age Y Plasma Level 25 Healthy children data orig; input age datalines; ; proc print data=orig; run; 7-15

16 Example (2) First, let s look at the scatterplot to see the relationship goptions ftitle=centb ftext=swissb htitle=3 htext=1.5 ctitle=blue ctext=black; title1 'Original Variables'; symbol1 v=dot c=blue ; axis1 label=('age (Years)'); axis2 label=(angle=90 'Plasma Level'); proc gplot data=orig; plot plasma*age / haxis=axis1 vaxis=axis2; run; Note, method for obtaining titles, axis labels. 7-16

17 Example (3) 7-17

18 Example (4) Run SLR model and check diagnostic plots proc reg data=orig; model plasma=age; output out = notrans r = resid; run; axis1 label=('age (Years)'); axis2 label=(angle=90 'Residual'); proc gplot data = notrans; plot resid*age / vref = 0 haxis=axis1 vaxis=axis2; run; proc univariate data=notrans; var resid; qqplot/normal (L=1 mu = est sigma = est); run; Note: Reference line in residual plot, 45-degree line in normal probability plot 7-18

19 Example (5) Root MSE R-Square

20 Example (6) 7-20

21 Example (7) Residuals do not appear to have constant variance Relationship not quite linear Use Box-Cox procedure to suggest a possible transformation of the Y variable 7-21

22 Example (8) proc transreg data = orig; model boxcox(plasma)=identity(age); run; The TRANSREG Procedure Box-Cox Transformation Information for plasma Lambda R-Square Log Like * * < * * < - Best Lambda * - 95% Confidence Interval + - Convenient Lambda 7-22

23 Example (9) + indicates the most convenient ; < indicates the best as determined by the log-likelihood function. Try ln ( ) Y and 1 Y data trans; set orig; logplasma = log(plasma);*in SAS log=ln, log10=log base 10; rsplasma = plasma**(-0.5); proc print data = trans; run; 7-23

24 Example (10) Re-run regression, diagnostic plots with transformed variables title1 'Natural Log Transformation'; proc reg data = trans; model logplasma = age; output out = logtrans r = logresid; run; axis1 label=('age (Years)'); axis2 label=(angle=90 'ln(plasma)'); proc gplot data = logtrans; plot logplasma * age/ haxis=axis1 vaxis=axis2; run; axis1 label=('age (Years)'); axis2 label=(angle=90 'Residual'); proc gplot data = logtrans; plot logresid * age / vref = 0 haxis=axis1 vaxis=axis2; run; proc univariate data=logtrans; var logresid; qqplot/ normal (L=1 mu = est sigma = est); run; 7-24

25 Example (11) 7-25

26 Example (12) Root MSE R-Square

27 Example (13) 7-27

28 Example (14) 7-28

29 Example (15) Root MSE R-Square

30 Example (16) 7-30

31 Example (17) Both transformations ln ( Y) and 1 Y : Led to a reasonably linear regression relation R-square improvement Improved non-constant variance problem Normality assumption supported in all cases 7-31

32 Summary of Remedial Measures Nonlinear Relationships Sometimes a transformation on X will fix this. Nonconstant Variance If we can model the way in which the error changes, we can use weighted regression. Sometimes a transformation on Y will work instead. Nonnormal Errors Could use a procedure that allows different distributions for the error term. Often, a transformation on Y will help. 7-32

33 Summary of Remedial Measures Often, a transformation on Y may help with more than one issue (e.g., normality and non-constant variance). Box-Cox Transformations Suggests some possibly Y transformations to try. Sometimes a transformation on X and Y may help. Remember - Assumptions still need to be satisfied (on the transformed scale) if we are to use linear regression model. So, we must always recheck diagnostic plots after transforming any variable. 7-33

34 Chapter 4 Covers some miscellaneous but important topics Joint (family) confidence levels ( ) Regression through the origin (4.4) Measurement Errors (4.5, optional reading) Inverse predictions when Y becomes X (4.6) 7-34

35 Summary of Inference - Reminder % Confidence Intervals b t s b 1 crit 1 b t s b 0 crit 0 Where tcrit t(1 ; n 2)

36 Family Confidence Levels Separate confidence intervals for 0 and 1 Now: Joint estimation of 0 and 1 If k 95% CI s are independent then their family confidence coefficient is given by 0.95 k. Note, here k=2 and 0.95 k = Usually not independent, so family confidence coefficient will be somewhat larger than 0.95 k, but certainly less than

37 Bonferroni Adjustment We want the probability that both intervals are correct to be 0.95 Basic idea is that we have an error budget (α =.05), so spend half on β 0 and half on β 1 We use α =.025 for each CI (97.5% CI), leading to b b t s b 0 c 0 t s b where 1 c tc t 1, n 2 2. We start with a 5% error budget and we have two intervals so we give 2.5% to each Each interval has two ends (tails) so we again divide by

38 Bonferroni Adjustment (Summary) Want to control family confidence level at 95%, then need to make adjustments Instead of, use /k. This is often more conservative than necessary, but will work in all cases and for any number of CIs. We can use this method for simultaneous estimation of mean responses and predictions of new observations too. 7-38

39 Mean Response CIs We already talked about simultaneous estimation for all X h with a confidence band: use Working-Hotelling Y ˆ ˆ h Ws Yh where W 2 2F 1 ;2, n 2 For simultaneous estimation for a few X h, say k different values, we may use Bonferroni instead. Y ˆ ˆ h Bs Yh where B t 1 / 2 k, n 2 Similar for simultaneous prediction intervals. 7-39

40 Regression Through Origin Y i = β 1 X i + ε i Should be very cautious using something like this. Forcing regression line through (0,0) can introduce bias, especially if X=0 isn t in the scope of the model. Problems with r 2 and other statistics Generally safer not to use this method; if when X=0 it is true that Y=0, then probably 0 will not test as significantly different from zero anyway. 7-40

41 Inverse Predictions From equation Yˆ b0 b1x instead of estimating Y based on X, want to estimate X based on Y Sometimes called calibration Example: A regression analysis was performed on the amount of decrease in cholesterol level (Y) achieved with a given dose (X) of a new drug based on observations of 50 patients. A physician needs to know the dose to give if a new patient s cholesterol needs to be decreased by a certain amount ( Y h( new )). 7-41

42 Inverse Predictions Natural point estimate is Xˆ h( new) Y h( new) 0 b 1 b Approximate confidence limits are obtained using the standard error: SE predx MSE b1 n SS X ˆ 2 h( new) X X 7-42

43 Upcoming in Lecture 8... Review of Matrix Algebra in the context of simple linear regression (Chapter 5) 7-43