Wiley. Methods and Applications of Linear Models. Regression and the Analysis. of Variance. Third Edition. Ishpeming, Michigan RONALD R.

Transcription

1 Methods and Applications of Linear Models Regression and the Analysis of Variance Third Edition RONALD R. HOCKING PenHock Statistical Consultants Ishpeming, Michigan Wiley

2 Contents Preface to the Third Edition xvii Preface to the Second Edition xix Preface to the First Edition xxi PART I REGRESSION 1 1 Introduction to Linear Models Background Information, Mathematical and Statistical Models, Definition of the Linear Model, Examples of Regression Models, Single-Variable, Regression Model, Regression Models with Several Inputs, Discrete Response Variables, Multivariate Linear Models, Concluding Comments, 21 Exercises, 21 2 Regression on Functions of One Variable The Simple Linear Regression Model, Parameter Estimation, Least Squares Estimation, Maximum Likelihood Estimation, Coded Data: Centering and Scaling, The Analysis of Variance Table, 33 vii

3 viii CONTENTS 2.3 Properties of the Estimators and Test Statistics, Moments of Linear Functions of Random Variables, Moments of Least Squares Estimators, Distribution of the Least Squares Estimators, The Distribution of Test Statistics, The Analysis of Simple Linear Regression Models, Two Numerical Examples, A Test for Lack-of-Fit, Inference on the Parameters of the Model, Prediction and Prediction Intervals, Examining the Data and the Model, Residuals, Outliers, Extreme Points, and Influence, Normality, Independence, and Variance Homogeneity, Polynomial Regression Models, The Quadratic Model, Higher Ordered Polynomial Models, Orthogonal Polynomials, Regression through the Origin, 72 Exercises, 72 3 Transforming the Data The Need for Transformations, Weighted Least Squares, Variance Stabilizing Transformations, Transformations to Achieve a Linear Model, Transforming the Dependent Variable, Transforming the Predictors, Analysis of the Transformed Model, Transformations with Forbes Data, 93 Exercises, 95 4 Regression on Functions of Several Variables The Multiple Linear Regression Model, Preliminary Data Analysis, Analysis of the Multiple Linear Regression Model, Fitting the Model in Centered Form, Estimation and Analysis of the Original Data, Model Assessment and Residual Analysis, Prediction, Transforming the Response, 110

4 CONTENTS ix 4.4 Partial Correlation and Added-Variable Plots, Partial Correlation, Added-Variable Plots, Simple Versus Partial Correlation, Variable Selection, The Case of Orthogonal Predictors, Criteria for Deletion of Variables, Nonorthogonal Predictors, Computational Considerations, Selection Strategies, Model Specification, Application to Subset Selection, Improved Mean Squared Error, Development of the Cp Statistic, 136 Exercises, Collinearity in Multiple Linear Regression The Collinearity Problem, Introduction, A Simple Example, The Picket Fence, Rotation of Coordinates, An Example with Collinearity, Preliminary Data Analysis, Initial Regression Analysis, Collinearity Diagnostics, Variance Inflation Factors, Eigenvalues, Eigenvectors, and Principal Component Plots, Remedial Solutions: Biased Estimators, Variable Deletion, Regression on Principal Components, Ridge Regression, 174 Exercises, Influential Observations in Multiple Linear Regression The Influential Data Problem, The Hat Matrix, The Centered and Uncentered Hat Matrices, Properties of the Hat Matrices, The Effects of Deleting Observations, Estimation of 0, 189

5 6.3.2 Computation of Residuals, Computation of Predicted Values, Estimation of the Error Variance, a2, Elements of the Hat Matrix, The Determinant of XTX, Deletion of More Than One Case, Numerical Measures of Influence, The Diagonal Elements of the Hat Matrix, Residuals, The Mean Square Ratio, Cook's Distance, Other Indicators of Influential Data, The Dilemma Data, Plots for Identifying Unusual Cases, The Projection Ellipse, The Augmented Hat Matrix, Multiple Extremes: The Masking Problem, Robust/Resistant Methods in Regression Analysis, M-Estimation, Iterative, Reweighted Least Squares, Regression with Bounded Influence, 212 Exercises, 213 Polynomial Models and Qualitative Predictors 7.1 Polynomial Models, The Quadratic Model with Two Predictors, Quadratic Surfaces, The Analysis of Response Surfaces, Analysis with First-Order Models, Analysis with Second-Order Models, Models with Qualitative Predictors, Indicator Variables to Identify Groups of Data, Indicator Variables to Fit Segmented Polynomials, 240 Exercises, 247 Additional Topics 8.1 Nonlinear Regression Models, Some Linearizeable Functions, The Modified Gauss-Newton Method, 258

6 CONTENTS xi 8.2 Nonparametric Model-Fitting Methods, Locally Weighted Average Predictors, Projection Pursuit Regression, Generalized Linear Models, Logistic Regression, Poisson Regression, Random Input Variables, Errors in the Inputs, Calibration, 277 Exercises, 278 PART II THE ANALYSIS OF VARIANCE Classification Models I: Introduction Background Information, The One-Way Classification Model, The Cell Means Model, Specification of Hypotheses, The Numerator Sum of Squares, Pairwise Comparisons of Means, Orthogonal Contrasts Among the Means, The Acceptance Ellipsoid, A Reparameterized Model, The Analysis of Covariance, The Two-Way Classification Model: Balanced Data, The Cell Means Model for the Two-Way Model, Hypotheses for the Two-Way Model, Simultaneous Inference on Marginal the Means, Reparameterizations of the Two-Factor Model, Parameter Estimation and Hypothesis Testing, A Non-Full Rank Model, The Two-Way Classification Model: Unbalanced Data, Discussion in Terms of the Cell Means Model, Discussion in Terms of the Reparameterized Model, The Case of Zero Cell Frequencies, The Two-Way Classification Model: No Interaction, Parameter Estimation, Tests of Hypotheses, Simultaneous Inference, 338

7 xii CONTENTS The No-Interaction Model: Unbalanced Data, Missing Cells: Estimation, Missing Cells: Testing Hypotheses, Connected Designs, Concluding Comments, 347 Exercises, The Mathematical Theory of Linear Models The Distribution of Linear and Quadratic Forms, The Distribution of Linear Functions of Normal Variables, The Distribution of Quadratic Functions of Normal Variables, Estimation and Inference for Linear Models, Estimation of Parameters, Optimality Properties of the Estimators, Estimation for the Constrained Model, A Partitioned Form of the Model, Reparameterized Models, Tests of Linear Hypotheses on /3, Unconstrained Model, The Constrained Model, Confidence Regions and Intervals, Confidence Regions, Confidence Intervals, 393 Exercises, Classification Models II: Multiple Crossed and Nested Factors The Three-Factor Cross-Classified Model, The Analysis with Balanced Data, Tests of Hypotheses, Unbalanced Data, nijk ^ 0, Estimability and Testability with Missing Cells, A General Structure for Balanced, Factorial Models, The Hypotheses, Numerator Sums of Squares, The Reparameterized Model, The Sum of Squares Identity, The Twofold Nested Model, A General Structure for Balanced, Nested Models, The Hypotheses, 426

8 CONTENTS xiii The Hypothesis Sums of Squares, The Reparameterized Model, A Three-Factor, Nested-Factorial Model, The Analysis with Balanced Data, The Analysis with Unbalanced Data, A General Structure for Balanced, Nested-Factorial Models, The Hypotheses, The Reparameterized Model, The Hypotheses Sums of Squares, The Sum of Squares Identity, Summary, 438 Exercises, Mixed Models I: The AOV Method with Balanced Data Introduction, Examples of the Analysis of Mixed Models, The One-Way Classification, Random Model, The Two-Way Classification, Mixed Model, The Three-Factor, Nested-Factorial Model, The Randomized Block Design, The General Analysis for Balanced, Mixed Models, A Description of the Model, Parameter Estimation and Inference, Additional Examples, The Twofold Nested, Random Model, Mixed Models for Split-Plot Designs, Repeated Measures Designs, Longitudinal Studies, Alternative Developments of Mixed Models, The Graybill Mixed Model, The Scheffe Mixed Model, A Randomization Theory, 492 Exercises, Mixed Models II: The AVE Method with Balanced Data Introduction, The Two-Way Cross-Classification Model, The Mixed Model, The Random Model, 508

9 xiv CONTENTS 13.3 The Three-Factor, Cross-Classification Model, Nested Models, Nested-Factorial Models, A General Description of the AVE Table, The AVE Table for Factorial Models, The AVE Table for Nested Models, The AVE Table for Nested-Factorial Models, The AVE Method for General Mixed Effects Models, Additional Examples, The Computational Procedure for the AVE Method, 537 Exercises, Mixed Models III: Unbalanced Data Introduction, Parameter Estimation: Likelihood Methods, Maximum Likelihood Estimation, Restricted Maximum Likelihood Estimation, Minimum Norm Quadratic Unbiased Estimators, A Numerical Illustration of the Methods, ML and REML Estimates with Balanced Data, ML Estimation with Balanced Data, REML Estimation with Balanced Data, The EM Algorithm for REML Estimation, A Review of the EM Algorithm, The EM Algorithm Applied to REML Estimation, The Estimation of Fixed Effects, Inferences on Variance Components and Fixed Effects, Numerical Examples to Illustrate the EM-AOV Algorithm, Diagnostic Analysis with the EM Algorithm, Numerical Examples to Illustrate the Diagnostics, The Computation of Individual, Pseudo Degrees of Freedom, Additional Numerical Examples, Models with Covariates, The Development of the Analysis, A Numerical Example, Summary, 585 Exercises, 585

10 CONTENTS XV 15 Simultaneous Inference: Tests and Confidence Intervals Simultaneous Tests, Simultaneous Tests: General Methods, Simultaneous Tests: Cell Means Models, Simultaneous Confidence Intervals, The Bonferroni Confidence Intervals, The Scheffe Confidence Intervals, Tukey Studentized-Range Intervals, 611 Exercises, 612 Appendix A Mathematics 615 A.I Matrix Algebra, 615 A.I.I Notation, 615 A.I.2 The Rank of a Matrix, 616 A.I.3 The Trace of a Matrix, 617 A.I.4 Eigenvalues and Eigenvectors, 617 A.1.5 Quadratic Forms and Definite Matrices, 618 A.I.6 Special Matrices, 619 A.I.7 The Diagonalization of Matrices, 620 A.I.8 Kronecker Products of Matrices, 620 A.I.9 Factorization of Matrices, 621 A.I. 10 Matrix Inversion, 622 A.I. 11 The Solution of Linear Equations, 624 A.I. 12 Generalized Inverses, 627 A.I. 13 Cauchy-Schwartz Inequalities, 630 A.II Optimization, 630 A.II.l The Differentiation of Matrices and Determinants, 630 A.II.2 The Differentiation of a Function with Respect to a Vector, 631 A.II.3 The Optimization of a Function, 632 Appendix B Statistics 634 B.I Distributions, 634 B.I.I The Normal Distribution, 634 B.I.2 The x^distribution, 637 B.I.3 The r-distribution, 638 B.I.4 The F-distribution, 639 B.II The Distribution of Quadratic Forms, 639

11 XVI CONTENTS Bill Estimation, 642 B.III.l Maximum Likelihood Estimation, 642 B.III.2 Constrained Maximum Likelihood Estimation, 642 B.III.3 Complete, Sufficient Statistics, 643 B. IV Tests of Hypotheses and Confidence Regions, 643 B.IV.l Tests of Hypotheses, 643 B.IV.2 Confidence Intervals and Regions, 644 Appendix C Data Tables 645 C. I Downloading Data Files from FTP Server, 645 C.II Listing of Data Set Files, 645 Appendix D Statistical Tables 660 References 669 Index 677