Applied Multivariate Statistical Analysis

Transcription

1 Applied Multivariate Statistical Analysis

2

3 Wolfgang Karl HRardle Léopold Simar Applied Multivariate Statistical Analysis Fourth Edition 123

4 Wolfgang Karl HRardle C.A.S.E. Centre f. Appl. Stat. & Econ. School of Business and Economics Humboldt-UniversitRat zu Berlin Berlin, Germany Léopold Simar Center of Operations Research & Econometrics (CORE) Katholieke Univeristeit Leuven Inst. Statistics Leuven, Belgium The majority of chapters have quantlet codes in Matlab or R. These quantlets may be downloaded from or via a link on and from ISBN ISBN (ebook) DOI / Library of Congress Control Number: Mathematics Subject Classification (2000): 62H10, 62H12, 62H15, 62H17, 62H20, 62H25, 62H30, 62F25 Springer Heidelberg New York Dordrecht London Springer-Verlag Berlin Heidelberg 2003, 2007, 2012, 2015 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. Printed on acid-free paper Springer-Verlag GmbH Berlin Heidelberg is part of Springer Science+Business Media ( com)

5 Preface to the Fourth Edition The fourth edition of this book on Applied Multivariate Statistical Analysis offers a new sub-chapter on Variable Selection by using least absolute shrinkage and selection operator (LASSO) and its general form the so-called Elastic Net. All pictures and numerical examples have been now calculated in the (almost) standard language R & MATLAB. The code for each picture is indicated with asmall sign near the picture, e.g. MVAdenbank denotes the corresponding quantlet for reproduction of Fig. 1.9, where we display the densities of the diagonal of genuine and counterfeit bank notes. We believe that these publicly available quantlets (see also create a valuable contribution to distribution of knowledge in the statistical science. The symbols and notations have also been standardised. In the preparation of the fourth edition, we received valuable input from Dedy Dwi Prastyo, Petra Burdejova, Sergey Nasekin and Awdesch Melzer. We would like to thank them. Berlin, Germany Louvain la Neuve, Belgium January 2014 Wolfgang Karl Härdle Léopold Simar v

6

7 Preface to the Third Edition The third edition of this book on Applied Multivariate Statistical Analysis offers the following new features. 1. A new Chap. 8 on Regression Models has been added. 2. Almost all numerical examples have been reproduced in MATLAB or R. The chapter on regression models focuses on a core business of multivariate statistical analysis. This contribution has not been subject of a prominent discussion in earlier editions of this book. We now take the opportunity to cover classical themes of ANOVA and ANCOVA analysis. Categorical responses are presented in Sect The spectrum of log linear models for contingency tables is presented in Sect , and applications to count data, e.g. in the economic and medical science are presented there. Logit models are discussed in great detail, and the numerical implementation in terms of matrix manipulations is presented. The majority of pictures and numerical examples has been now calculated in the (almost) standard language R & MATLAB. The code for each picture is indicated with a small sign near the picture, e.g. MVAdenbank denotes the corresponding quantlet for reproduction of Fig. 1.9, where we display the densities of the diagonal of genuine and counterfeit bank notes. We believe that these publicly available quantlets (see also create a valuable contribution to distribution of knowledge in the statistical science. The symbols and notations have also been standardised. In the preparation of the third edition, we received valuable input from Song Song, Weining Wang and Mengmeng Guo. We would like to thank them. Berlin, Germany Louvain la Neuve, Belgium June 2011 Wolfgang Karl Härdle Léopold Simar vii

8

9 Contents Part I Descriptive Techniques 1 Comparison of Batches Boxplots Histograms Kernel Densities Scatterplots Chernoff-Flury Faces Andrews Curves Parallel CoordinatesPlots HexagonPlots Boston Housing Exercises Part II Multivariate Random Variables 2 A Short Excursion into Matrix Algebra ElementaryOperations Spectral Decompositions QuadraticForms Derivatives Partitioned Matrices GeometricalAspects Exercises Moving to Higher Dimensions Covariance Correlation SummaryStatistics LinearModel fortwo Variables Simple Analysis of Variance ix

10 x Contents 3.6 Multiple Linear Model Boston Housing Exercises Multivariate Distributions Distributionand Density Function Momentsand Characteristic Functions Transformations The MultinormalDistribution SamplingDistributionsand Limit Theorems Heavy-TailedDistributions Copulae Bootstrap Exercises Theory of the Multinormal ElementaryProperties ofthe Multinormal The Wishart Distribution Hotelling s T 2 -Distribution Spherical and Elliptical Distributions Exercises Theory of Estimation The Likelihood Function The Cramer Rao Lower Bound Exercises Hypothesis Testing Likelihood Ratio Test Linear Hypothesis Boston Housing Exercises Part III Multivariate Techniques 8 Regression Models GeneralANOVA and ANCOVA Models ANOVA Models ANCOVA Models Boston Housing Categorical Responses MultinomialSamplingandContingencyTables Log-LinearModelsfor ContingencyTables Testing Issues with Count Data LogitModels Exercises

11 Contents xi 9 Variable Selection Lasso Lasso in the LinearRegression Model Lasso in HighDimensions Lasso in LogitModel Elastic Net Elastic Net in Linear RegressionModel Elastic Net in Logit Model GroupLasso Exercises Decomposition of Data Matrices by Factors The GeometricPoint of View Fitting the p-dimensionalpoint Cloud Fitting the n-dimensionalpoint Cloud Relations Between Subspaces Practical Computation Exercises Principal Components Analysis StandardisedLinear Combination Principal Components in Practice Interpretationof the PCs AsymptoticPropertiesof the PCs Normalised Principal Components Analysis Principal Components as a Factorial Method Common Principal Components Boston Housing MoreExamples Exercises Factor Analysis The Orthogonal Factor Model Estimationof the Factor Model FactorScores and Strategies Boston Housing Exercises Cluster Analysis The Problem The ProximityBetween Objects Cluster Algorithms Boston Housing Exercises

12 xii Contents 14 Discriminant Analysis AllocationRules forknowndistributions DiscriminationRules in Practice Boston Housing Exercises Correspondence Analysis Motivation Chi-Square Decomposition Correspondence Analysis in Practice Exercises Canonical Correlation Analysis Most InterestingLinearCombination Canonical Correlation in Practice Exercises Multidimensional Scaling The Problem Metric MDS NonmetricMDS Exercises Conjoint Measurement Analysis Introduction Design of Data Generation Estimationof PreferenceOrderings Exercises Applications in Finance PortfolioChoice EfficientPortfolio EfficientPortfolios in Practice The Capital Asset Pricing Model Exercises Computationally Intensive Techniques Simplicial Depth ProjectionPursuit Sliced InverseRegression Support Vector Machines Classification and Regression Trees Boston Housing Exercises

13 Contents xiii Part IV Appendix 21 Symbols and Notations Data Boston HousingData Swiss Bank Notes Car Data Classic Blue PulloversData US CompaniesData FrenchFoodData Car Marks French Baccalauréat Frequencies JournauxData US Crime Data Plasma Data WAIS Data ANOVA Data Timebudget Data Geopol Data US Health Data VocabularyData Athletic Records Data UnemploymentData Annual Population Data BankruptcyData I BankruptcyData II References Index