Introduction to Regression

Transcription

1 Introduction to Regression Using Mult Lin Regression Derived variables Many alternative models Which model to choose? Model Criticism Modelling Objective Model Details Data and Residuals Assumptions 1

2 Introduction to Regression: Criticism Objective? 1. Informal Analyse/explore the data Find patterns/exceptions 2. Use for prediction Precision 3. Use for testing/forming aspects of theory Understand how the variables are interrelated Understand coefficients What variables are important? 2

3 1 Informal Objectives Analyse/explore the data; Find patterns/exceptions Single response variable clear? No: Use scatterplot matrix; smoothers, correlation Generally: Use Multivariate analysis Yes: Also use regression; What predictors are most important? Interpret Coefficients Add new x vars; lurking vars? too many vars? Check residuals Exceptions? Patterns? 3

4 Stat Anal Alg Vect Mech Matrix Plot of Stat, Anal, Alg, Vect, Mech Math Marks 88 Students Descriptive Statistics: Stat, Anal, Alg, Vect, Mech Variable Mean StDev Stat Anal Alg Vect Mech Stat Correlation: Stat, Anal, Alg, Vect, Mech Anal Alg Stat Anal Alg Vect Anal Alg Vect Mech Vect Cell Contents: Pearson correlation Mech

5 2 Prediction Best? Seek large R 2 ; small S Simple model Small number of predictors; natural scale Interpretable coefficients No important exceptions No pattern in residuals later Prediction Intervals Great care with extrapolation 5

6 Standard Errors Data Like This Regression Analysis: LogVol versus LogHt, LogDiam The regression equation is LogVol = LogHt LogDiam Predictor Coef SE Coef T P Constant LogHt LogDiam S = R-Sq = 97.8% 95% of Analyses of Data Like This LogHt coeff in (1.12 2(0.22)) ie 1 LogDiam coeff in (1.98 2(0.075)) ie 2 6

7 Simple is Good The regression equation is ie LogVol = Log (Ht Diam 2 ) 2 (0.035) = Log (Ht Diam 2 ) 2 (0.035) Vol = (Ht Diam 2 ) 10 2 (0.035) Meaning Vol in interval based on (Ht Diam 2 ) = Roughly (0.035) = % 10-2 (0.035) = % 95% of Tree Vols = (Ht Diam 2 ) to within about 16% 7

8 Simple is good Predict Vol from Ht AND Diam S=3.88 R-sq = 9.80% logvol from loght, logdiam S = R-sq = 97.87% logvol from log (Ht Diam 2 ) S = R-Sq = 97.7% 8

9 Residuals - Trees Patterns? Exceptions? Vol vs Ht, Diam in linear scale Vol vs Ht Diam 2 in log scale Unusual Observations Obs Vol Fit Resid Std Resid R R Large residual Unusual Observations Obs logvol Fit Resid Std Resid R R Large residual 9

10 3 Prediction for Theory Means of comparing/testing coeffs ( evolving theory) Compared to what? Controlling for external variation? Coefficients Interpretation One-at-a-time - SEs, t-ratios, Conf Intervals Many-at-once - Analysis of variance Entire model R 2 Confidence bands 10

11 Gas: Modelled with Interaction Regression Analysis: Gas versus Temperature, Insulated, Ins X Temp Model Summary S R-sq R-sq(adj) R-sq(pred) % 92.35% 91.% Coefficients Term Coef SE Coef T-Value P-Value VIF Constant Temperature Insulated Ins X Temp S = R-Sq = 92.8% 11

12 Stat Significance Simplest for MTB Coeff value is 0: Var plays no role Coeff 0 Var plays some role Statistically Sig 95% interval does not includes 0 Data Like This Var seems to play some role in pred Formal Logic T ˆ Hyp SE ˆ T stat inconsistent with Explanation: β Hyp =0 and chance 12

13 Stat Significance Statistically Sig 95% interval does not includes 0 Alt Data Like This If in fact Null Hyp true then obs T is large in sense that Pr( T >17.03) when Null Hyp true Contrast Scientifically Sig Var plays important role in prediction in theory 13

14 What are Significant implications? Term Coef SE Coef T-Value P-Value VIF Constant Temperature Insulated Ins X Temp

15 Residuals Fits and Diagnostics for Unusual Observations Obs Gas Fit Resid Std Resid X X R R R R Large residual X Unusual X 15

16 Model Criticism Variables are columns; cases are rows Which variables are important? in what way? Which cases are important? in what way? Outliers, residuals, influence Errors Normal? Independent? 16

17 Frequency Residual Percent Residual The regression equation is Diagnostics Volume = Diameter Height Ht*Diam^2 Unusual Observations Obs Diameter Volume Fit SE Fit Residual St Resid R X R denotes an observation with a large standardized residual. X denotes an observation whose X value gives it large leverage. Residual Plots for Vol 99 Normal Probability Plot 5.0 Versus Fits Residual Fitted Value Histogram Versus Order Residual Observation Order

18 SRES1 Vol Unusual Observations Obs Diameter Volume Outliers Fit SE Fit Residual St Resid R X R denotes an observation with a large standardized residual. X denotes an observation whose X value gives it large leverage. 2 Scatterplot of SRES1 vs FITS Fitted Line Plot Vol = FITS FITS FITS

19 Leverage x values far from the centre have high leverage. More difficult to see when many x variables 19

20 Outlying and Influential cases Outlying cases Unusual y values, for these x-values Influential cases Unusual combination of x-values Perhaps erroneous Omit and re-analyse? Certainly worth double-checking 20

21 Gas Consumption The regression equation is Gas = Temperature Insulated Ins X Temp Unusual Observations Obs Temperature Gas Fit SE Fit Residual St Resid X X R R R R denotes an observation with a large standardized residual. X denotes an observation whose X value gives it large leverage. 21

22 Diagnostics for Gas Obs Temperature Gas Fit SE Fit Residual St Resid X X R R R 22

23 Compare Models Gas Consumption Previous Analysis The regression equation is Gas = Temperature Insulated Ins X Temp Predictor Coef SE Coef T P Constant Temperature Insulated Ins X Temp S = R-Sq = 92.8% R-Sq(adj) = 92.% New Analysis The regression equation is Gas = Temperature Insulated Ins X Temp Predictor Coef SE Coef T P Constant Temperature Insulated Ins X Temp Case 55 excluded S = R-Sq = 93.6% R-Sq(adj) = 93.2% 23

24 Compare diagnostics for Gas Previous Diagnostics Obs Temperature Gas Fit SE Fit Residual St Resid X X R R R New Diagnostics Obs Temperature Gas Fit SE Fit Residual St Resid X R R R R R R denotes an observation with a large standardized residual. X denotes an observation whose X value gives it large leverage. 2

25 Comparing diagnostics for Gas New Diagnostics Obs Temperature Gas Fit SE Fit Residual St Resid X R R R R R 25

26 Normal Distribution Diagnostics Prob Plot Normal Distribution Provides a scale for large outliers Relatively unimportant for T-ratios, p-values 26

27 Diagnostics and Analysis Residuals/leverage point to cases worth careful examination. Plot the data. Use labels to identify cases in different plots Ask questions Reduction in SS as more variables added can lead to VIF Difficulties with correlation in x-vars. T-ratios for coeffs provide a very unreliable guide. Proceed carefully. 27

28 Interpreting Coefficients What do coefficients mean? Recall Scale log Units - dimensions SEs 95% Conf Ints 28

29 Are coefficients important? Any? Some? Omitted variables? How judge? Science SE ANOVA Are (all) data to be relied on? Does part of the data dominate some conclusions 29