Measurement Error Models

Transcription

1 Measurement Error Models

2 Measurement Error Models WAYNE A. FULLER Iowa State University Ames, Iowa JOHN WILEY & SONS New York Chichester Brisbane Toronto Singapore

3 A NOTE TO THE IEADER This book has been electronically reproduced from digital itiforniation stored at Jolui Wiley I% Sons, hic. We are pleased that the use of this new technology will enable 11s to keep works of enduring scholarly value in print as long as there is a reasonable demand for them. The content of this book is identical to previous printings by John Wiley & Sons, Inc. All rights reserved. Published simultaneously in Canada. Reproduction or translation of any part of this work beyond that permitted by Section 107 or 108 of the 1976 United States Copyright Act without the permission of the copyright owner is unlawful. Requests for permission or further information should be addressed to the Permissions Department, John Wiley & Sons, Inc. Library of Congress Cataloging in Publication Data: Fuller, Wayne A. Measurement error models. (Wiley series in probability and mathematical statistics. Applied probability and statistics, ISSN ) Bibliography: p. Includes index. 1. Error analysis (Mathematics) 2. Regression analysis. I. Title. 11. Series. QA275.FS ISBN Printed and bound in the United States of America by Braun-Brumfield, lnc

4 To Doug and Bret

5 Preface The study of regression models wherein the independent variables are measured with error predates the twentieth century. There has been a continuing interest in the problem among statisticians and there is considerable literature on the subject. Also, for over 80 years, studies have documented the presence of sizable measurement error in data collected from human respondents. Despite these two lines of research, only a fraction of the statistical studies appearing in the literature use procedures designed for explanatory variables measured with error. This book is an outgrowth of research on the measurement error, also called response error, in data collected from human respondents. The book was written with the objective of increasing the use of statistical techniques explicitly recognizing the presence of measurement error. To this end, a number of real examples have been included in the text. An attempt has been made to choose examples from a variety of areas of application, but the reader will understand if many of the examples have an agricultural aspect. The book may be used as a text for a graduate course concentrating on statistical analyses in the presence of measurement error. It is hoped that it will also find use as an auxiliary text in courses on statistical methodology that heretofore have ignored, or given cursory treatment to, the problems associated with measurement error. Chapter 1 was developed to provide an introduction to techniques for a range of simple models. While the models of Chapter 1 are special cases of models discussed in later chapters, it is felt that the concepts are better communicated with the small models. There is some flexibility in the order in which the material can be covered. One can move from a section in Chapter 1 to the corresponding section in Chapter 2 or Chapter 4. To facilitate flexible use, Sections 1.2, 1.3, and 1.4 are largely self-supporting. As a result, there is some duplication in the treatment of topics such as prediction. Some repetition seems advantageous because the vii

6 viii PREFACE models of this book differ from those typically encountered by students in courses on regression estimation. The proofs of most of the theorems require a background of statistical theory. One will be comfortable with the proofs only if one has an understanding of large sample theory. Also, the treatment assumes a background in ordinary linear regression methods. In attempting to make the book useful to those interested in the methods, as well as to those interested in an introduction to the theory, derivations are concentrated in the proofs of theorems. It is hoped that the text material, the statements of the theorems, and the examples will serve the person interested in applications. Computer programs are required for any extensive application of the methods of this book. Perhaps the most general program for normal distribution linear models is LISREL@ VI by Joreskog and Sorbom. LISREL VI is available in SPSSXTM and can be used for a wide range of models of the factor type. A program with similar capabilities, which can also perform some least squares fitting of the type discussed in Section 4.2, is EQS developed by Bentler. EQS is available from BMDP@ Statistical Software, Inc. Dan Schnell has placed the procedures of Chapter 2 and Section 3.1 in a program for the IBM@ Personal Computer AT. This program, called EV CARP, is available from the Statistical Laboratory, Iowa State University. The packages SAP and BMDP contain algorithms for simple factor analysis. A program, ISU Factor, written with Proc MATRIX of SAS by Sastry Pantula, Department of Statistics, North Carolina State University, can be used to estimate the factor model, to estimate multivariate models with known error variances, and to estimate the covariance matrix of the factor estimates. A program for nonlinear models, written with Proc MATRIX of SAS by Dan Schnell, is available from Iowa State University. I have been fortunate to work with a number of graduate students on topics related to those of this text. Each has contributed to my understanding of the field, but none is to be held responsible for remaining shortcomings. I express my sincere thanks to each of them. In chronological order they are James S. DeGracie, Angel Martinez-Garza, George E. Battese, A. Ronald Gallant, Gordon D. Booth, Kirk M. Wolter, Michael A. Hidiroglou, Randy Lee Carter, P. Fred Dahm, Fu-hua Yu, Ronald Mowers, Yasuo Amemiya, Sastry Pantula, Tin-Chiu Chua, Hsien-Ming Hung, Daniel Schnell, Stephen Miller, Nancy Hasabelnaby, Edina Miazaki, Neerchal Nagaraj, and John Eltinge. I owe a particular debt to Yasuo Amemiya for proofs of many theorems and for reading and repair of much of the manu- LISREL is a registered trademark of Scientific Software, Inc. SPSS' is a trademark of SPSS, Inc. BMDP is a registered trademark of BMDP Statistical Software, Inc. SAS is a registered trademark of SAS Institute, Inc. IBM AT is a registered trademark of International Business Machines, Inc.

7 PREFACE script. I thank Sharon Loubert, Clifford Spiegelman, and Leonard Stefanski for useful comments. I also express my appreciation to the United Kingdom Science and Engineering Research Council and the U.S. Army European Research Ofice for supporting the Workshop on Functional and Structural Relationships and Factor Analysis held at Dundee, Scotland, August 24 through September 9, Material presented at that stimulating conference had an influence on several sections of this book. I am grateful to Jane Stowe, Jo Ann Hershey, and Christine Olson for repeated typings of the manuscript. A part of the research for this book was supported by joint statistical agreements with the United States Bureau of the Census and by cooperative research agreements with the Statistical Reporting Service of the United States Department of Agriculture. ix WAYNE A. FULLER Ames, Iowa February 1987

8 Contents List of Examples List of Principal Results List of Figures xv xix xxiii 1. A Single Explanatory Variable Introduction, Ordinary Least Squares and Measurement Error, Estimation with Known Reliability Ratio, Identification, Measurement Variance Known, Introduction and Estimators, Sampling Properties of the Estimators, Estimation of True x Values, Model Checks, Ratio of Measurement Variances Known, Introduction, Method of Moments Estimators, Least Squares Estimation, Tests of Hypotheses for the Slope, Instrumental Variable Estimation, Factor Analysis, Other Methods and Models, Distributional Knowledge, 72 xi

9 xii The Method of Grouping, Measurement Error and Prediction, Fixed Observed X, 79 Appendix 1.A. Appendix l.b. Appendix l.c. Appendix l.d. CONTENTS Large Sample Approximations, 85 Moments of the Normal Distribution, 88 Central Limit Theorems for Sample Moments, 89 Notes on Notation, Vector Explanatory Variables Bounds for Coefficients, The Model with an Error in the Equation, Estimation of Slope Parameters, Estimation of True Values, Higher-Order Approximations for Residuals and True Values, The Model with No Error in the Equation, The Functional Model, The Structural Model, Higher-Order Approximations for Residuals and True Values, Instrumental Variable Estimation, Modifications to Improve Moment Properties, An Error in the Equation, No Error in the Equation, Calibration, 177 Appendix 2.A. Language Evaluation Data, Extensions of the Single Relation Model Nonnormal Errors and Unequal Error Variances, Introduction and Estimators, Models with an Error in the Equation, Reliability Ratios Known, Error Variance Functionally Related to Observations, The Quadratic Model, Maximum Likelihood Estimation for Known Error Covariance Matrices. 217

10 CONTENTS 3.2. Nonlinear Models with No Error in the Equation, Introduction, Models Linear in x, Models Nonlinear in x, Modifications of the Maximum Likelihood Estimator, The Nonlinear Model with an Error in the Equation, The Structural Model, General Explanatory Variables, Measurement Error Correlated with True Value, Introduction and Estimators, Measurement Error Models for Multinomial Random Variables, 272 Appendix 3.A. Data for Examples, Multivariate Models 4.1. The Classical Multivariate Model, Maximum Likelihood Estimation, Properties of Estimators, Least Squares Estimation of the Parameters of a Covariance Matrix, Least Squares Estimation, Relationships between Least Squares and Maximum Likelihood, Least Squares Estimation for the Multivariate Functional Model, Factor Analysis, Introduction and Model, Maximum Likelihood Estimation, Limiting Distribution of Factor Estimators, 360 Appendix 4.A. Matrix-Vector Operations, 382 Appendix 4.B. Properties of Least Squares and Maximum Likelihood Estimators, 396 Appendix 4.C. Maximum Likelihood Estimation for Singular Measurement Covariance, 404 Bibliography Author Index Subject Index xiii

11 List of Examples Number Topic Corn-nitrogen. Error variance of explanatory variable known. Estimates, 18 Corn-nitrogen. Estimated true values, 23 Corn-nitrogen. Residual plot, 26 Pheasants. Ratio of error variances known, 34 Rat spleens. Both error variances known, 40 Rat spleens. Tests and confidence intervals, 48 Earthquake magnitudes. Instrumental variable, 56 Corn hectares. Factor model, 63 Corn hectares. Standardized factors, 69 Corn-nitrogen. Prediction for random model, 75 Earthquakes. Prediction in another population, 77 Coop managers. Error variances estimated, 110 Coop managers. Estimated true values, 114 Corn-nitrogen. Variances of estimated true values and residuals, 121 Apple trees. Estimated error covariance, 130 Corn-moisture experiment. Estimated error covariance, 134 Coop managers. Test for equation variance, 138 xv

12 xvi Rat spleens. Variances of estimated true values and residuals, 142 Language evaluation. Instrumental variables, 154 Firm value. Instrumental variables, 158 Corn-nitrogen. Calibration, 179 LIST OF EXAMPLES Corn-nitrogen. Duplicate determinations used to estimate error variance, 197 Farm size. Reliability ratios known, 201 Textiles. Different slopes in different groups, 204 Pig farrowings. Unequal error variances, 207 Tonga earthquakes. Quadratic model, 214 Quadratic. Both error variances known, 21 5 Supernova. Unequal error variances, 221 Created data. Linear in true values, 226 Berea sandstone. Nonlinear, 230 Berea sandstone. Nonlinear multivariate, 234 Hip prosthesis. Implicit nonlinear, 244 Quadratic, maximum likelihood. Large errors, 247 Pheasants. Alternative estimators of variance of estimated slope, 255 Pheasants. Alternative form for estimated variance of slope, 257 Quadratic. Large errors. Modified estimators, 257 Quadratic. Error in the equation. Weighted, 266 Moisture response model. Nonlinear, 268 Unemployment. Binomial, 275 Mixing fractions. Known error covariance matrix, 308 Cattle genetics. Error covariance matrix estimated, 3 13 Two earthquake samples. Least squares estimation, 325 Corn hectares. Estimation of linear model, 330 Corn hectares. Distribution-free variance estimation, 332 Earthquakes. Least squares iterated to maximum likelihood, 337

13 LIST OF EXAMPLES xvii Corn hectares. Least squares estimation fixed and random models, 344 Bekk smoothness. One factor, 364 Language evaluation. Two factors, 369 Language evaluation. Not identified, 374

14 List of Principal Results Theorem A. 1 1.c. 1 1.c.2 1.C Topic Approximate distribution of estimators for simple model with error variance of explanatory variable known, 15 Approximate distribution of estimators for simple model with ratio of error variances known, 32 Approximate distribution of instrumental variable estimators for simple model, 53 Distribution of estimators when the observed explanatory variable is controlled, 8 1 Large sample distribution of a function of sample means, 85 Large sample distribution of first two sample moments, 89 Limiting distribution of sample second moments containing fixed components, IID observations, 92 Limiting distribution of sample second moments containing fixed components, independent observations, 94 Limiting distribution of estimators for vector model with error covariance matrix of explanatory variables known, 108 Maximum likelihood estimators for vector model with no error in the equation, 124 Limiting distribution of estimators for vector model with no error in the equation. Limit for small error variances and (or) large sample size, 127 Limiting distribution of instrumental variable estimator, 15 1 xix

15 xx I B. 1 LIST OF PRINCIPAL RESULTS Moment properties of modified estimator for simple model with known error variance and an error in the equation, 164 Vector version of Theorem 2.5.1, 171 Moment properties of modified estimator for model with no error in the equation, 173 Limiting distribution of weighted estimator for unequal error variances and an error in the equation, 187 Limiting distribution of weighted estimator for unequal error variances with no error in the equation, 190 Limiting distribution of maximum likelihood estimator for model with unequal known error covariance matrices, 218 Limiting distribution of estimators for nonlinear model with no error in the equation and known covariance matrix, 240 Limiting distribution of estimator of variance for nonlinear model, 243 Multivariate maximum likelihood estimator, error covariance matrix known up to a constant, 293 Multivariate maximum likelihood estimator, error covariance matrix estimated, 296 Multivariate likelihood ratio statistic, 301 Strong consistency of multivariate maximum likelihood estimators, 303 Limiting distribution of multivariate maximum likelihood estimators, 305 Limiting distribution of least squares estimator for structural model, 323 Least squares estimation of a diagonal covariance matrix, 329 Limiting distribution of least squares for normal functional model, 339 Equivalence of least squares estimators for the normal functional and structural models, 342 Limiting distribution of estimators for factor model, 360 Condition under which the factor model is not identified, 372 Almost sure convergence of the estimator maximizing a continuous function, 398

16 LIST OF PRINCIPAL RESULTS xxi 4.B.2 Limiting distribution of estimator maximizing the normal distribution likelihood-structural model, B.3 Limiting distribution of estimator maximizing normal distribution likelihood-functional model, B.4 Limiting distribution of least squares estimator computed from sample covariances, c. 1 Maximum likelihood estimator for model with known singular error covariance matrix, 404

17 List of Figures Figure Title Residual plot for the corn-nitrogen data, 26 Pheasant data and estimated structural line, 36 Estimated minimum distance line and estimated true values for two types of cells, 42 Plot of residuals against estimated true x values, 67 Prediction in a second population, 80 Plot of residual against estimated true value for value orientation, 116 Normal probability plot for residuals, 117 Histogram for 2000 maximum likelihood estimates, 168 Histogram for 2000 modified estimates, 168 Plot of mean of two Y observations against mean of two X observations for pig farrowing data, 209 Plot of weighted residuals against estimated true values, 212 Estimated function and observed value for wave velocity, 233 Estimated ellipse and observed data for image of hip prosthesis, 247 True function and maximum likelihood estimated function for 120 observations generated by quadratic model, 249 Deviation Z,, - it, plotted against ftl, 368 xxiii