Discriminant Analysis and Statistical Pattern Recognition

Transcription

1 Discriminant Analysis and Statistical Pattern Recognition GEOFFREY J. McLACHLAN Department of Mathematics The University of Queensland St. Lucia, Queensland, Australia A Wiley-Interscience Publication JOHN WILEY & SONS, INC. New York Chichester Brisbane Toronto Singapore

2 Contents Preface 1. General Introduction 1.1. Introduction, Basic Notation, Allocation Rules, Decision-Theoretic Approach, Unavailability of Group-Prior Probabilities, Training Data, Sample-Based Allocation Rules, Parametric Allocation Rules, Assessment of Model Fit, Error Rates of Allocation Rules, Posterior Probabilities of Group Membership, Distances Between Groups, Likelihood-Based Approaches to Discrimination 2.1. Maximum Likelihood Estimation of Group Parameters, A Bayesian Approach, Estimation of Group Proportions, Estimating Disease Prevalence, Misclassified Training Data, Partially Classified Training Data, Maximum Likelihood Estimation for Partial Classification, 39

3 viii CONTENTS 2.8. Maximum Likelihood Estimation for Partial Nonrandom Classification, Classification Likelihood Approach, Absence of Classified Data, Group-Conditional Mixture Densities, Discrimination via Normal Models Introduction, Heteroscedastic Normal Model, Homoscedastic Normal Model, Some Other Normal-Theory Based Rules, Predictive Discrimination, Covariance-Adjusted Discrimination, Discrimination with Repeated Measurements, Partially Classified Data, Linear Projections of Homoscedastic Feature Data, Linear Projections of Heteroscedastic Feature Data, Distributional Results for Discrimination via Normal Models Introduction, Distribution of Sample NLDF (JT-Statistic), Moments of Conditional Error Rates of Sample NLDR, Distributions of Conditional Error Rates of Sample NLDR, Constrained Allocation with the Sample NLDR, Distributional Results for Quadratic Discrimination, Some Practical Aspects and Variants of Normal Theory-Based Discriminant Rules Introduction, Regularization in Quadratic Discrimination, Linear Versus Quadratic Normal-Based Discriminant Analysis, Some Models for Variants of the Sample NQDR, Regularized Discriminant Analysis (RDA), Robustness of NLDR and NQDR, Robust Estimation of Group Parameters, 161

4 CONTENTS ix 6. Data Analytic Considerations with Normal Theory-Based Discriminant Analysis Introduction, Assessment of Normality and Homoscedasticity, Data-Based Transformations of Feature Data, Typicality of a Feature Vector, Sample Canonical Variates, Some Other Methods of Dimension Reduction to Reveal Group Structure, Example: Detection of Hemophilia A Carriers, Example: Statistical Diagnosis of Diabetes, Example: Testing for Existence of Subspecies in Fisher's Iris Data, Parametric Discrimination via Nonnormal Models Introduction, Discrete Feature Data, Parametric Formulation for Discrete Feature Data, Location Model for Mixed Features, Error Rates of Location Model-Based Rules, Adjustments to Sample NLDR for Mixed Feature Data, Some Nonnormal Models for Continuous Feature Data, Case Study of Renal Venous Renin in Hypertension, Example: Discrimination Between Depositional Environments, Logistic Discrimination Introduction, Maximum Likelihood Estimation of Logistic Regression Coefficients, Bias Correction of MLE for g = 2 Groups, Assessing the Fit and Performance of Logistic Model, Logistic Versus Normal-Based Linear Discriminant Analysis, Example: Differential Diagnosis of Some Liver Diseases, 279

5 CONTENTS 9. Nonparametric Discrimination Introduction, Multinomial-Based Discrimination, Nonparametric Estimation of Group-Conditional Densities, Selection of Smoothing Parameters in Kernel Estimates of Group-Conditional Densities, Alternatives to Fixed Kernel Density Estimates, Comparative Performance of Kernel-Based Discriminant Rules, Nearest Neighbor Rules, Tree-Structured Allocation Rules, Some Other Nonparametric Discriminant Procedures, Estimation of Error Rates Introduction, Some Nonparametric Error-Rate Estimators, The Bootstrap, Variants of the Bootstrap, Smoothing of the Apparent Error Rate, Parametric Error-Rate Estimators, Confidence Intervals, Some Other Topics in Error-Rate Estimation, Assessing the Reliability of the Estimated Posterior Probabilities of Group Membership Introduction, Distribution of Sample Posterior Probabilities, Further Approaches to Interval Estimation of Posterior Probabilities of Group Membership, Selection of Feature Variables in Discriminant Analysis Introduction, Test for No Additional Information, Some Selection Procedures, Error-Rate-Based Procedures, The F-Test and Error-Rate-Based Variable Selections, 406

6 CONTENTS Xl Assessment of the Allocatory Capacity of the Selected Feature Variables, Statistical Image Analysis Introduction, Markov Random Fields, Noncontextual Methods of Segmentation, Smoothing Methods, Individual Contextual Allocation of Pixels, ICM Algorithm, Global Maximization of the Posterior Distribution of the Image, Incomplete-Data Formulation of Image Segmentation, Correlated Training Data, 443 References 447 Author Index 507 Subject Index 519