Applied Regression Modeling

Size: px

Start display at page:

Download "Applied Regression Modeling"

Lorraine Lynch
6 years ago
Views:

3 Applied Regression Modeling

5 Applied Regression Modeling A Business Approach Iain Pardoe University of Oregon Charles H. Lundquist College of Business Eugene, Oregon WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION

Copyright O 2006 by John Wiley & Sons, Inc. Allrightsreserved. Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada.

6 Copyright O 2006 by John Wiley & Sons, Inc. Allrightsreserved. Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Denvers, MA 01923, (978) , fax (978) , or on the web at Requests to the Publisher for pennission should be addressed to the Permissions Department, John Wiley & Sons, Inc., Ill River Street, Hoboken, NJ 07030, (201) , fax (201) , or online at Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no tepiesentalions or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability orfitnessfor a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) , outside the United States at (317) or fax (317) Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic format For information about Wiley products, visit our web site at Library of Congress CaUdoging-in-Publication Datei Pardoe, Iain Applied regression modeling: a business approach / Iain Pardoe. p. cm. Includes bibliographical references and index. ISBN 13:978-0^ (alk. paper) ISBN 10: (alk. paper) 1. Regression analysis. 2. Statistics. I. Title. QA278.2.P *36 dc

7 To Tanya, Bethany, and Sierra

9 CONTENTS Preface Acknowledgments xiii xv Introduction xvii 1.1 Statistics in business xvii 1.2 Learning statistics xix 1 Foundations Identifying and summarizing data Population distributions Selecting individuals at random probability Random sampling Central limit theorem normal version Student's t-distribution Central limit theorem t version Interval estimation Hypothesis testing The rejection region method The p-value method Hypothesis test errors Random errors and prediction 23 vii

10 Vlll CONTENTS 1.8 Chapter summary 26 Problems 27 2 Simple linear regression Probability model for X and Y Least squares criterion Model evaluation Regression standard error Coefficient of determination R Slope parameter Model assumptions Checking the model assumptions Model interpretation Estimation and prediction Confidence interval for the population mean, E(K) Prediction interval for an individual K-value Chapter summary Review example 66 Problems 70 3 Multiple linear regression Probability model for (X\,X 2,...) and Y Least squares criterion Model evaluation Regression standard error Coefficient of determination R Regression parameters global usefulness test Regression parameters nested model test Regression parameters individual tests Model assumptions Checking the model assumptions Model interpretation Estimation and prediction Confidence interval for the population mean, E(Y) Prediction interval for an individual K-value Chapter summary 114 Problems Regression model building I Transformations Natural logarithm transformation for predictors 122

11 CONTENTS IX Polynomial transformation for predictors Reciprocal transformation for predictors Natural logarithm transformation for the response Transformations for the response and predictors Interactions Qualitative predictors Qualitative predictors with two levels Qualitative predictors with three or more levels Chapter summary 158 Problems Regression model building II Influential points Outliers Leverage Cook's distance Regression pitfalls Autocorrelation Multicollinearity Excluding important predictor variables Overfitting Extrapolation Missing Data Model building guidelines Model interpretation using graphics Chapter summary 194 Problems Case studies Home prices Data description Exploratory data analysis Regression model building Results and conclusions Further questions Vehicle fuel efficiency Data description Exploratory data analysis Regression model building Results and conclusions Further questions 219

12 X CONTENTS 7 Extensions Generalized linear models Logistic regression Poisson regression Discrete choice models Multilevel models Bayesian modeling Frequentist inference Bayesian inference 235 Appendix A: Computer software help 237 A.l SPSS 238 A. 1.1 Getting started and summarizing univariate data 238 A. 1.2 Simple linear regression 241 A. 1.3 Multiple linear regression 243 A.2 Minitab 245 A.2.1 Getting started and summarizing univariate data 245 A.2.2 Simple linear regression 248 A.2.3 Multiple linear regression 249 A.3 SAS 251 A.3.1 Getting started and summarizing univariate data 252 A.3.2 Simple linear regression 254 A.3.3 Multiple linear regression 255 A.4 R and S-PLUS 257 A.4.1 Getting started and summarizing univariate data 258 A.4.2 Simple linear regression 260 A.4.3 Multiple linear regression 261 A.5 Excel 263 A.5.1 Getting started and summarizing univariate data 263 A.5.2 Simple linear regression 265 A.5.3 Multiple linear regression 265 Problems 267 Appendix B: Critical values for t-dlstributions 269 Appendix C: Notation and formulas 273 C.l Univariate data 273 C.2 Simple linear regression 274 C.3 Multiple linear regression 275 Appendix D: Mathematics refresher 277

13 CONTENTS Xi D. 1 The natural logarithm and exponential functions 277 D.2 Rounding and accuracy 278 Appendix E: Brief answers to selected problems 279 References 287 Glossary 291 Index 297

PREFACE This book has developed from class notes written for the "Business Statistics" course taken primarily by undergraduate business majors in their junior year at the University of Oregon.

15 PREFACE This book has developed from class notes written for the "Business Statistics" course taken primarily by undergraduate business majors in their junior year at the University of Oregon. This course is essentially an applied regression course, and incoming students have already taken an introductory probability and statistics course. The book is suitable for any undergraduate second statistics course in which regression analysis is the main focus. It would also be suitable for use in an applied regression course for nonstatistics major graduate students, including MBAs. Mathematical details have deliberately been kept to a minimum, and the book does not contain any calculus. Instead, emphasis is placed on applying regression analysis to data using statistical software, and understanding and interpreting results. Chapter 1 reviews essential introductory statistics material, while Chapter 2 covers simple linear regression. Chapter 3 introduces multiple linear regression, while Chapters 4 and 5 provide guidance on building regression models, including transforming variables, using interactions, incorporating qualitative information, and using regression diagnostics. Each of these chapters includes homework problems, mostly based on analyzing real datasets provided with the book. Chapter 6 contains two in-depth case studies, while Chapter 7 introduces extensions to linear regression and outlines some related topics. The appendices contain instructions on using statistical software (SPSS, Minitab, SAS, and R/S-PLUS) to carry out all the analyses covered in the book, a table of critical values for the t-distribution, notation and formulas used throughout the book, a glossary of important terms, a short mathematics refresher, and brief answers to selected homework problems. The first five chapters of the book have been successfully used in quarter-length courses over the last several years. An alternative approach for a quarter-length course would be to skip some of the material in Chapters 4 and 5 and substitute one or both of the case studies xlli

Applied Regression Modeling

Applied Regression Modeling A Business Approach Iain Pardoe University of Oregon Charles H. Lundquist College of Business Eugene, Oregon WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION CONTENTS