Regression Analysis By Example

Transcription

1 Regression Analysis By Example Third Edition SAMPRIT CHATTERJEE New York University ALI S. HADI Cornell University BERTRAM PRICE Price Associates, Inc. A Wiley-Interscience Publication JOHN WILEY & SONS, INC. New York Chichester Weinheim Brisbane Singapore Toronto

2 Contents Preface Introduction 1.1 What Is Regression Analysis? 1.2 Publicly Available Data Sets 1.3 Selected Applications of Regression Analysis 1.4 Steps in Regression Analysis 1.5 Scope and Organization of the Book Exercises Simple Linear Regression Introduction 2.2 Covariance and Correlation Coefficient 2.3 Example: Computer Repair Data 2.4 The Simple Linear Regression Model 2.5 Parameter Estimation 2.6 Tests of Hypotheses 2.7 Confidence Intervals 2.8 Predictions 2.9 Measuring the Quality of Fit xiii vii

3 CONTENTS 2.10 Regression Line Through the Origin Trivial Regression Models Bibliographie Notes 4$ Exercises 4$ Multiple Linear Regression Introduction Description of the Data and Model Example: Supervisor Performance Data Parameter Estimation Interpretations of Regression Coefficients Properties of the Least Squares Estimators Multiple Correlation Coefficient Inference for Individual Regression Coefficients Tests of Hypotheses in a Linear Model Predictions Summary 74 Exercises 75 Appendix 80 Regression Diagnostics: Detection of Model Violations Introduction The Standard Regression Assumptions Various Types of Residuais Graphical Methods Graphs Before Fitting a Model Graphs After Fitting a Model Checking Linearity and Normality Assumptions Leverage, Influence, and Outliers Measures of Influence The Potential-Residual Plot What to Do with the Outliers? Role of Variables in a Regression Equation Effects of an Additional Predictor Robust Regression 116 Exercises 116

4 CONTENTS ix 5 Qualitative Variables as Predictors Introduction Salary Survey Data Interaction Variables Systems of Regression Equations Other Applications of Indicator Variables Seasonality Stabilüy of Regression Parameters Over Time 142 Exercises Transformation of Variables Introduction Transformations to Achieve Linearity Bacteria Deaths Due to X-Ray Radiation Transformations to Stabilize Variance Detection of Heteroscedastic Errors Removal of Heteroscedasticity Weighted Least Squares Logarithmic Transformation of Data Power Transformation Summary 176 Exercises Weighted Least Squares Introduction Heteroscedastic Models Two-Stage Estimation Education Expenditure Data Fitting a Dose-Response Relationship Curve 197 Exercises The Problem of Correlated Errors Introduction: Autocorrelation Consumer Expenditure and Money Stock Durbin-Watson Statistic Removal of Autocorrelation by Transformation Iterative Estimation With Autocorrelated Errors Autocorrelation and Missing Variables 209

5 x CONTENTS 8.7 Analysis of Housing Starts Limitations of Durbin-Watson Statistic Indicator Variables to Remove Seasonality Regressing Two Time Series 219 Exercises Analysis of Collinear Data Introduction Effects on Inference Effects on Forecasting Detection of Multicollinearity Centering and Scaling Principal Components Approach Imposing Constraints Searching for Linear Functions of the ß 's Computations Using Principal Components Bibliographie Notes 258 Exercises 258 Appendix: Principal Components Biased Estimation of Regression Coefßcients Introduction Principal Components Regression Removing Dependence Among the Predictors Constraints on the Regression Coefficients Principal Components Regression: A Caution Ridge Regression Estimation by the Ridge Method Ridge Regression: Some Remarks Summary 279 Exercises 279 Appendix: Ridge Regression Variable Selection Procedures Introduction Formulation of the Problem Consequences of Variables Deletion Uses of Regression Equations 288

6 CONTENTS XI 11.5 Criteria for Evaluating Equations Multicollinearity and Variable Selection Evaluating All Possible Equations Variable Selection Procedures General Remarks on Variable Selection Methods A Study of Supervisor Performance Variable Selection With Collinear Data The Homicide Data Variable Selection Using Ridge Regression Selection of Variables in an Air Pollution Study A Possible Strategy for Fitting Regression Models Bibliographie Notes 312 Exercises 312 Appendix: Effects of Incorrect Model Specifications Logistic Regression Introduction Modeling Qualitative Data The Logit Model Example: Estimating Probability of Bankruptcies Logistic Regression Diagnostics Determination of Variables to Retain Judging the Fit of a Logistic Regression Classification Problem: Another Approach 330 Exercises 331 Appendix: Statistical Tables 335 References 347 Index 355