THE EFFICIENCIES OF THE SPATIAL MEDIAN AND SPATIAL SIGN COVARIANCE MATRIX FOR ELLIPTICALLY SYMMETRIC DISTRIBUTIONS

Transcription

1 THE EFFICIENCIES OF THE SPATIAL MEDIAN AND SPATIAL SIGN COVARIANCE MATRIX FOR ELLIPTICALLY SYMMETRIC DISTRIBUTIONS BY ANDREW F. MAGYAR A issertation submitte to the Grauate School New Brunswick Rutgers, The State University of New Jersey in partial fulfillment of the requirements for the egree of Doctor of Philosophy Grauate Program in Statistics Written uner the irection of Davi E. Tyler an approve by New Brunswick, New Jersey May, 01

2 ABSTRACT OF THE DISSERTATION The Efficiencies of the Spatial Meian an Spatial Sign Covariance Matrix for Elliptically Symmetric Distributions by Anrew F. Magyar Dissertation Director: Davi E. Tyler The spatial meian an spatial sign covariance matrix SSCM are popularly use robust alternatives for estimating the location vector an scatter matrix when outliers are present or it is believe the ata arises from some istribution that is not multivariate normal. When the unerlying istribution is an elliptical istribution, it has been observe that these estimators perform better uner certain scatter structures. This issertation is a etaile stuy of the efficiencies of the spatial meian an the SSCM uner the elliptical moel, in particular the epenence of their efficiencies on the population scatter matrix. For the spatial meian, it is shown this estimator is asymptotically most efficient compare to the MLE for the location vector when the population scatter matrix is proportional to the ientity matrix. Furthermore, it is possible to construct an affinely equivariant version of the spatial meian that is asymptotically more efficient than the spatial meian. Asymptotic relative efficiencies of these two estimators are calculate to emonstrate how inefficient the spatial meian can be as the unerlying scatter structure becomes more elliptical. A simulation experiment is carrie out to provie evience of analogous result for finite samples. When the goal is estimating ii

3 eigenprojection matrices, it is proven that uner the elliptical moel the eigenprojection estimates obtaine from the Tyler matrix are asymptotically more efficient than those corresponing to the SSCM. Calculations of asymptotic relative efficiencies are presente to emonstrate the loss of efficiency in using eigenprojection estimates of the SSCM as oppose to the Tyler matrix, particularly when the scatter structure of the ata is far from spherical. To assess the performance of these estimators in the finite sample setting, the notion of principal angles is use to efine a means to compare eigenprojection estimators. Using this concept, simulations are implemente that support finite sample results similar to those for the asymptotic case. The implications of the above results are iscusse, particularly in the application of principal component analysis. Future research irections are then propose. iii

4 Acknowlegements First an foremost, I woul like to acknowlege my octoral avisor, Dr. Davi E. Tyler, for mentoring me throughout my grauate stuies. Not only has he introuce me to the fiel of robust statistics but also provie me with a topic that I have enjoye researching these last few years. This issertation coul not have been complete without his guiance. Seconly, I woul like to acknowlege the entire Statistics Department at Rutgers University for giving me the chance to pursue grauate stuies an embarking me on an interesting an rewaring career. My time here has enable me to evelop my love of mathematics an apply it to a fiel that I have grown to love equally. It was a privilege to stuy uner such an outstaning faculty, each of whom has helpe me in some capacity whether it be acaemically, intellectually or professionally. I woul also like to acknowlege Dr. Daniel Ocone from the Department of Mathematics as well as Dr. Richar McLean from the Department of Economics, both of whom I was fortunate to have ha the opportunity to learn uner their tutelage. Aitionally, I woul like to acknowlege my high-school mathematics teacher, Micheal O Boyle, who sparke the fire of curiosity for mathematics an the esire to pursue it further. Lastly, I woul like to thank all my family an friens, both those from before grauate school an those I met uring my time at Rutgers. I am truly blesse to have such a collection of people in my life. You have all provie me with a neee escape from work an a means to simply enjoy life in your company. iv

5 Deication This issertation is eicate to the two people whose love an support has helpe me evelop into the person I am toay, my parents. To my mother, Dorothea, from whom I inherite the abilities necessary to pursue a career in the fiel of statistics an see this eneavor through. To my father, Ruolph, who always encourage me to o what mae me happy. While wors alone cannot escribe all that you have one for me, I have only these two to express my appreciation, Thank You. -Any v

6 Table of Contents Abstract ii Acknowlegements iv Deication v List of Figures ix 1. Preliminary Material Introuction Spherically an Elliptically Symmetric Distributions Multivariate Estimation an Equivariance The Goal of this Dissertation The Spatial Meian Introuction The Spatial Meian Theoretical Results Efficiency an Relative Efficiency of Location Estimators The Asymptotic Efficiency of the Spatial Meian The Oracle Spatial Meian an Affine Spatial Meian Asymptotic Efficiency Calculations Finite Sample Performance Finite Sample Theory Results Bivariate Distributions Trivariate Distributions vi

7 10 Dimensional Distributions Appenix The Influence Function an Asymptotic Variance-Covariance Matrix of M-estimators of Location The Asymptotic Variance-Covariance of the Spatial Meian The Asymptotic Variance-Covariance of the Oracle Spatial Meian Proof of Theorem Simulation Results The Spatial Sign Covariance Matrix Introuction Tyler s Scatter Matrix Spatial Sign Covariance Matrix Theoretical Results Optimality of the Tyler Matrix uner the Elliptical Moel Asymptotic Calculations Finite Sample Performance Finite Sample Theory Results Appenix Asymptotic Distribution of Eigenprojections an Eigenvectors of a Scatter Estimate Tyler s Scatter Matrix The Spatial Sign Covariance Matrix Asymptotic Calculations Conclusion Implications Future Work References vii

8 Vita viii

9 List of Figures.1. Asymptotic Relative Efficiencies of the Spatial Meian to the Oracle Spatial Meian in R Asymptotic Relative Efficiencies of the Spatial Meian to the Oracle Spatial Meian in R 3 : Λ 0 = iag 1, 1, r Asymptotic Relative Efficiencies of the Spatial Meian to the Oracle Spatial Meian in R 3 : Λ 0 = iag 1, r, r Asymptotic Relative Efficiencies of the Spatial Meian to the Oracle Spatial Meian in R Finite Sample Efficiencies of Affine Spatial Meians to the Oracle Spatial Meian for the Bivariate Spherical Normal Finite Sample Efficiencies of the Spatial Meian to the Oracle Spatial Meian for the Bivariate Normal Finite Sample Efficiencies of Affine Spatial Meians to the Oracle Spatial Meian for the Bivariate Spherical t Finite Sample Efficiencies of the Spatial Meian to the Oracle Spatial Meian for the Bivariate t Finite Sample Efficiencies of the Affine Spatial Meian using Dümbgen s matrix to the Oracle Spatial Meian for the Bivariate Spherical Cauchy an Slash Finite Sample Efficiencies of the Spatial Meian to the Oracle Spatial Meian for the Bivariate Cauchy Finite Sample Efficiencies of Affine Spatial Meians to the Oracle Spatial Meian for the Trivariate Spherical Normal an Cauchy ix

10 .1. Finite Sample Efficiencies of the Spatial Meian to the Oracle Spatial Meian for the Trivariate Normal an Cauchy - I Finite Sample Efficiencies of the Spatial Meian to the Oracle Spatial Meian for the Trivariate Normal an Cauchy - II Finite Sample Efficiencies of the Spatial Meian to the Oracle Spatial Meian for the Trivariate Normal an Cauchy - III Finite Sample Efficiencies of Affine Spatial Meians to the Oracle Spatial Meian for the Spherical Normal an Cauchy in R Finite Sample Efficiencies of the Spatial Meian to the Oracle Spatial Meian for the Normal an Cauchy in R 10 - I Finite Sample Efficiencies of the Spatial Meian to the Oracle Spatial Meian for the Normal an Cauchy in R 10 - II Finite Sample Efficiencies of the Spatial Meian to the Oracle Spatial Meian for the Normal an Cauchy in R 10 - III Asymptotic Relative Efficiencies of the Eigenprojection Estimate of the SSCM to the Corresponing Eigenprojection Estimate of the Tyler Matrix in R Asymptotic Relative Efficiencies of the Eigenprojection Estimate of the SSCM to the Corresponing Eigenprojection Estimate of the Tyler Matrix in R Asymptotic Relative Efficiencies of the Eigenprojection Estimate of the SSCM to the Corresponing Eigenprojection Estimate of the Tyler Matrix in R Finite Sample Relative Efficiencies of the Eigenprojection Estimate of the SSCM to the Corresponing Eigenprojection Estimate of the Tyler Matrix in R Finite Sample Relative Efficiencies of the Eigenprojection Estimate of the SSCM to the Corresponing Eigenprojection Estimate of the Tyler Matrix in R x

11 3.6. Finite Sample Relative Efficiencies of the Eigenprojection Estimate of the SSCM to the Corresponing Eigenprojection Estimate of the Tyler Matrix in R xi

12 1 Chapter 1 Preliminary Material 1.1 Introuction Many proceures in multivariate statistics make the assumption that the ata arises as realizations from some multivariate normal istribution. A multivariate normal istribution is entirely specifie by two parameters, its population mean vector an population covariance matrix, commonly symbolize by the Greek letters µ an Σ respectively. Most multivariate methos involve inference on one or both of these parameters with either one or both being assume unknown. Thus it is often necessary to obtain estimates of these parameters from the observe ata. The most popularly use estimators of µ an Σ are the sample mean vector an sample covariance matrix. Unfortunately, when the ata eviates from the assumption of multivariate normality the reliability of these estimators, an any metho that relies on them, is compromise. For instance, the presence of outliers in the ata coul have a significant impact on these estimates; extreme observations ten to pull the sample mean vector away from µ an towars themselves. Their effect on the sample covariance matrix is ouble in that they effect the sample mean vector use in calculating the sample covariance matrix as well as possibly biasing the estimate variation in their particular irection thus not giving an accurate estimate of Σ. One attempt to aress these short-comings is to implement methos from nonparametric statistics. Non-parametric proceures make minimal or no assumptions about the unerlying istribution from which the ata arises. Consequently, their performance oes not epen on the unerlying istribution hence will still prouces reliable results. However, when the ata oes inee come from a multivariate normal istribution, proceures involving estimation of µ an or Σ with the sample mean an

13 sample covariance matrix often rastically out-perform any non-parametric proceure. Recall that in this situation, the sample mean vector an sample covariance matrix are functions of the complete, sufficient statistics thus contain the most information about the parameters µ an Σ. For this reason, alternative approaches have been taken to evelop reliable proceures without sacrificing performance; robust statistics is one such instance. As oppose to assuming the ata is from one particular istribution or iscaring istributional assumptions altogether, robust statistics consiers methos evelope for a family of istributions. Because of this, these methos will prouce reliable results an perform well for all istributions within the class they were evelope for. In robust multivariate statistics, consierable attention is given to the stuy of proceures uner the assumption the ata comes from some elliptically symmetric istribution or elliptical istribution. This family of istributions is one generalization of the multivariate normal istribution. The characterization of an elliptical istribution involves specifying parameters that act as generalizations of µ an Σ; these are the location vector an scatter matrix respectively an are represente with the same symbols. Similar to before, one or both of these parameters is assume unknown an must be estimate from the ata. Several estimators of location an scatter have been propose to estimate the location vector an scatter matrix. Perhaps the most natural are the maximum likelihoo estimators MLE s for µ an Σ assuming the ata comes from a particular elliptical istribution. As a extension of these estimators, Maronna [3] evelope M-estimators of location an scatter. Inepenently, Huber [14] evelope an even broaer class of estimators using arguments base on the concept of affine equivariance. 1. Spherically an Elliptically Symmetric Distributions Elliptically symmetric istributions have playe a central role in the evelopment of robust multivariate statistics by serving as alternatives to the multivariate normal istribution with which to stuy the robustness properties of multivariate methos. Elliptical istributions arise by taking affine transformations of spherically symmetric

14 3 istributions. A multivariate ranom vector, z R, is sai to be spherically symmetric or spherical about the origin, 0, if z Qz for any orthogonal matrix Q. The istribution of a spherically symmetric ranom variable will be enote by G. If the measure inuce by G is absolutely continuous with respect to Lebesgue measure in R, then there exists a probability ensity function for the ranom vector z of the form C g, g z = C g, g z t z where g is a fixe Lebesgue integrable function in R an C g, is a constant epening on both g an that ensures that the expression is inee a ensity i.e. integrates to 1 in R ; that is C g, = g z t z z R In this form, the motivation of the name spherically symmetric is obvious in that the contours of equal ensity are concentric hyper-spheres in R. The vector 0 serves not only as the point of symmetry for the concentric hyper-spheres, but is also the mean vector if the istribution has finite first moments. If in aition the istribution has finite secon moments, the covariance matrix of the istribution is proportional to the ientity matrix I. If the ranom vector z R is spherically symmetric, then for any nonsingular matrix, M, an vector µ R, the vector x = Mz + µ is sai to be elliptically istribute in -imensions with parameters µ an Σ = MM t. When a ranom vector x R has an elliptical istribution it will be notate x E µ, Σ; G; this istribution will be referre to as F. If the measure inuce by F is absolutely continuous with respect to Lebesgue measure in R, then x has a ensity, enote f, that is given by 1 f x; µ, Σ, g = C g, et Σ 1/ g x µ Σ 1 = C g, et Σ 1/ g x µ t Σ 1 x µ

15 4 where x A = x t Ax. Analogously, the name elliptically symmetric originates from the fact the contours of equal ensity are concentric hyper-ellipsois in R, namely the hyper-ellipsoi given by the equation x µ t Σ 1 x µ = c. Similarly, the parameter µ is the location vector an correspons to the center of symmetry of the hyper-ellipsois an also the mean vector when the istribution has finite first moments. The parameter Σ is referre to as the scatter matrix since it etermines the sprea an orientation of the concentric hyper-ellipsois. In general, the istribution nee not have finite secon moments, but in cases where it oes Σ is also referre to as the pseuocovariance matrix since the covariance matrix of the istribution is then proportional to Σ. Note, in general the scatter matrix is not well-efine within the class of elliptical istributions. For a given elliptical istribution, the function g s can be replace with the function g c s = c g cs. It is possible to impose restrictions on the function g s to eliminate any ambiguities such as take the constant c such that g c s has covariance matrix given by I, however, will not be necessary for aims of this issertation. Necessarily, a scatter matrix is symmetric an positive efinite. Recall from linear algebra that any symmetric, positive efinite matrix has a spectral ecomposition unique up to multiplication of the eigenvectors by ±1. For the matrix Σ, let λ 1 λ λ enote its eigenvalues an p 1,..., p enote an orthonormal set of eigenvectors with p i belonging to the eigenspace corresponing to λ i. Let the eigenprojection associate with the eigenvalue λ i be enote P i. Recall this is an projection matrix in to the eigenspace corresponing to the eigenvalue λ i, enote P i. For repeate eigenvalues i.e. an eigenvalue with algebraic multiplicity greater than 1, then the imension of the corresponing eigenspace the geometric multiplicity coul also be greater than 1. Suppose λ j = λ j+1 = = λ j+k 1 = λ. In this is instance the notation P λ an P λ will be use to enote the corresponing eigenprojection an eigenspace associate with the eigenvalue λ. Any pair of orthonormal vectors that span P λ coul be taken for p j,..., p j+k 1. If a given eigenvalue, say λ ι, has geometric multiplicity 1, then the eigenvector p ι is uniquely efine up to multiplication by ±1. Defining the matrices P = [p 1,..., p ] an Λ = iag λ 1,..., λ, then one can write Σ = PΛP t. Using the spectral ecomposition, it is possible to efine the unique, symmetric, positive efinite

16 5 square root of Σ to be Σ 1/ = PΛ 1/ P t, where Λ 1/ = iag λ 1,..., λ. There is a useful representation for any spherical istribution that will be utilize on several occasions in the proceeing chapters. Every spherical istribution in R has a stochastic representation of the form z R G u with R G an u inepenent. R G is a nonnegative ranom variable referre to as the raial component of z an u is a ranom vector that is uniformly istribute on S, the unit hyper-sphere in R. This implies that if x E µ, Σ; G, then it has stochastic representation x R G Mu + µ, where M is any matrix such that MM t = Σ. One such choice is M = Σ 1/. If x 1,..., x n is an i.i.. sample from some elliptical istribution E µ, Σ; G, this will be referre to as the elliptical moel. 1.3 Multivariate Estimation an Equivariance Intimately connecte with elliptical istributions is the concept of affine equivariance. The location vector an scatter matrix for elliptical istributions are affinely equivariant; that is if x E µ, Σ; G, then the ranom vector x = Ax + b will also be elliptically istribute with x E µ, Σ ; G where µ = Aµ + b an Σ = AΣA t for any non-singular, matrix A an b R. Because of this property, when it is assume that ata arises from an elliptical istribution, it is natural to consier estimators of µ an Σ that possess the property of affine equivariance. That is, if x 1,..., x n yiels estimates of the location vector an scatter matrix µ n an Σ n respectively, then the estimators obtaine for the transforme ata x i = Ax i + b, i = 1,..., n, will be µ n = A µ n + b an Σ n = A Σ n A t. M-estimators are examples of affinely equivariant estimators of location an scatter. However, in [3] the author showe the breakown points of affinely equivariant M-estimators is at most 1/ + 1. Consequently, much work has been on the evelopment of high breakown affinely equivariant estimates such as the minimum volume ellipsoi estimate MVE an minimum covariance eterminant estimate MCD [31], S-estimates [9] & [18], projection base estimates [10], [5] & [37], CM-estimates [16], MM-estimates [3] & [37], τ-estimates [19], an one-step versions of these estimates [0]. Unfortunately, high breakown affinely equivariant estimators ten to

17 6 be computationally intensive, especially for large an n; current algorithms are only approximate an probabilistic in nature. Another pitfall of affinely equivariant estimators is that when n <, any affinely equivariant estimate of location an scatter reuce to the sample mean vector an sample covariance matrix respectively, the latter being singular in this situation [38]. Consequently, these short-comings have le to the evelopment of methos that iscar the property of affine equivariance. To the goal of the previous paragraph, one such approach has been to evelop estimators that are only orthogonally equivariant. Estimators of the location vector an scatter matrix are sai to be orthogonally equivariant if the ata x 1,..., x n yiel estimates µ n an Σ n respectively, then the estimates obtaine for the transforme ata x i = Qx i + b, i = 1,..., n will be µ n = Q µ n + b an Σ n = Q Σ n Q t for any, orthogonal matrix Q an b R. Orthogonal transformations are special cases of affine transformations, thus an analogous result hols for the parameters µ an Σ uner the class of elliptical istributions. When consiering the class of elliptical istributions, a benefit of using affinely equivariant estimators of location an scatter is that the form of the influence function can be erive by just consiering the spherical case. Furthermore, the efficiencies of such estimators oes not epen on either µ or Σ. These properties o not carry over to estimators that lack the property of affine equivariance. On the contrary, uner elliptical moels it has been observe that non-affinely equivariant estimators perform better uner certain scatter structure than for others. Unfortunately, this fact is usually ignore when eciing which estimator to use. Using these estimators is rather Procrustean in that they are favoring certain scatter structures over others; to a egree this is letting the metho etermine the moel. 1.4 The Goal of this Dissertation As mentione in section 1.4, the performance of non-affinely equivariant estimators uner the elliptical moel is epenent on Σ. One popular metho for evaluating the performance of as estimator is to stuy the variability with which the estimator measure

18 7 the parameter of interest. While the nature of the parameter ictates the criterion of interest, it usually involves the variances or variance-covariance matrix of the estimators being stuie. Naturally, the evaluation of the performance necessitates some sort of benchmark or alternative metho that achieves the same goal, thus usually estimators are stuie in reference to competing estimators. The evaluation of estimators via the prior paraigm is the basis of a notion calle efficiency. This issertation will stuy the efficiencies of two popularly use orthogonally equivariant estimators uner elliptical moels an their epenencies on Σ. The two consiere are the estimators of location an scatter, the spatial meian an the spatial sign covariance matrix SSCM respectively. These will be aresse in separate parts; the spatial meian being iscusse in Chapter II whereas the SSCM in Chapter III. The concept of efficiency for location an scatter estimates will be efine more explicitly in the subsequent sections. The efficiencies will be consiere in both the asymptotic an finite sample cases. For both estimators, uner elliptical moels it will be shown that these estimators are asymptotically most efficient when the unerlying scatter structure of the ata is spherically symmetric. Simulation results will be presente in the finite sample case to support an analogous hypothesis. Technical etails of proofs omitte from the boy of the issertation will be presente in two separate appenices at the ens of chapters an 3. The first part of chapter 4 will iscuss the implications of the above finings in robust Principal Component Analysis PCA, a commonly use imension reuction technique with broa applications [7], [8], [1], [17] & []. The issertation will conclue with a iscussion of future research irections.

19 8 Chapter The Spatial Meian.1 Introuction For estimating the location parameter of multivariate ata, the spatial meian is a commonly use robust alternative to the sample mean vector when it is believe that the ata being analyze either contains outlying observations or comes from a istribution that is not multivariate normal. Since the estimation of the spatial meian oes not require an estimate of the scatter matrix in its calculation, the spatial meian has the ae benefit that it exists even when the sample size is less than imension thus making it a popular estimator of the location parameter for sparse ata. However, unlike the sample mean vector, the spatial meian is not affinely equivariant but only equivariant uner translations, rescaling an orthogonal transformations. Because of this property, the spatial meian is commonly use in orthogonally equivariant multivariate methos that require estimation of a location parameter as an intermeiary such as principle component analysis PCA. The reason the spatial meian lacks the property of affine equivariance is that in its calculation it own-weighs observations in terms of their Eucliean istances as oppose to their Mahalanobis istance from the estimate center of the ata. Thus in the the setting where the ata is assume to arise from some elliptical istribution, one might conjecture that the spatial meian is less efficient in situations where the istribution is not spherical. This chapter is broken own into the following sections. In section, the spatial meian is iscusse an it is shown that the estimating equation for it is simply the MLE for µ when the istribution for the elliptical moel is a spherical Laplace Double Exponential istribution. In section 3, the main theoretical results are presente. The first subsection iscusses the concept of relative efficiency an efficiency of a vector

20 9 estimator, both for finite samples an asymptotically. It is then shown that for the class of elliptically symmetric istributions, the spatial meian is asymptotically most efficient when Σ I, that is the istribution is in fact spherically symmetric. In aition, it is possible for one to construct an affinely equivariant version of the spatial meian that has the same asymptotic istribution as the spatial meian at spherical symmetry but is asymptotically more efficient than the spatial meian for all nonspherical elliptical istributions. Lastly, some calculations are presente to emonstrate the severity of the asymptotic inefficiencies. Section 4 carries out a simulation stuy of the efficiencies of the spatial meian for finite samples. The first subsection contains theoretical results neee to carry out the simulations an escribes how they were implemente. The secon subsection contains the results an iscussion.. The Spatial Meian Given a multivariate ata set, x 1,..., x n, in R, the spatial meian is the vector µ SM that satisfies the following objective function n µ SM = argmin η R x i η..1 Recall in the univariate case, = 1, above reuces to an expression whose solution is given by the sample meian of the ataset. Consequently, the spatial meian can be thought of as one possible generalization of the meian to the multivariate case. i=1 Perhaps the earliest reference of the spatial meian was in [4]. In the literature, the spatial meian is also referre to as the meian centre or L 1 meian since the minimization of the objective function involves minimizing the sum of the L 1 norms to the observations [13]. Brown [5] stuie the asymptotic efficiency of the spatial meian for the bivariate normal istribution as well as for stanar multivariate normal istributions in imensions greater than. The objective function in..1 has no explicit solution, however, it was shown to be uniquely efine when > [15] & [7]. The spatial meian is a special case of a monotonic M-estimate of multivariate location an thus can be compute via a simple re-iterate least squares IRLS algorithm [43].

21 10 In [39], the authors propose a useful moification to improve the prior algorithm. A summary of the spatial meian can be foun in [8]. As mentione in the introuction, the spatial meian is not affinely equivariant, but only equivariant uner orthogonal transformations, rescaling as well as translations. That is, for any b R, c R an orthogonal matrix Q, if the ata is transforme as x i = cqx i + b for i = 1,..., n, then the spatial meian transforms as µ SM = cq µ SM + b. Referring back to the theory of maximum likelihoo estimation, the sample meian is the MLE of the location parameter when the ata comes from a Laplace istribution. Analogously, the spatial meian arises as the MLE when the ata comes from an elliptical Laplace istribution with Σ I. Recall the ensity function of the multivariate Laplace istribution with location vector µ an scatter matrix Σ is given by f x; µ, Σ, g L = π/ Γ Γ / et Σ 1/ exp x µ t Σ 1 x µ Given n observations, the likelihoo function is then given by L µ, Σ; X, g L = = et Σ 1/ exp x i µ t Σ 1 x i µ n π / Γ Γ / i=1 n π / Γ et Σ 1/ exp Γ / n i=1 x i µ t Σ 1 x i µ If Σ were known a priori, then maximizing the above likelihoo entails maximizing the argument in the exponential, which is the same as minimizing the sum. Hence the the MLE for µ is n µ Σ = argmin η R x i η Σ 1 If Σ = σ I, then the estimating equation for µ σ I woul yiel the same solution as equation..1, that is µ σ I = µ SM. This characterization will be utilize in establishing the theoretical results to follow. i=1

22 11.3 Theoretical Results.3.1 Efficiency an Relative Efficiency of Location Estimators When sampling from elliptical istributions, it was conjecture in the introuction that the spatial meian is asymptotically most efficient when Σ is proportional to the ientity matrix. The spatial meian is intene to give an estimate of the location vector, thus before proving the aforementione result it is necessary to iscuss the notions of efficiency an relative efficiency for vector estimators both in the finite sample case an asymptotically. Let θ n an θ n be two ifferent unbiase estimators of the vector parameter θ base ] ] on samples of size n. Let = V ar [ θn = V ar [ θn, assuming θ V θn n an an V θn θ n have finite secon moments. In this situation, comparing the estimators θ n an θ n reuces to a comparison of how they estimate linear combinations of the parameter being estimate, that is a t θ for some a R. The natural estimators for a t θ are a t θn ] ] an a t θn with variances given by V ar [a t θn = a t a an V ar [a V θn t θn = a t a V θn respectively. Since a t θ is a univariate parameter, convention is to focus on the ratio of variances, a t V θn a/a t V θn a. Comparison of θ n to θ n involves locating the vector a such that ratio a t V θn a/a t V θn a is maximal or minimal. Results from linear algebra gives that the value of the ratio at its minimum/maximum is the same as the smallest/largest eigenvalue of the matrix V 1 θ n V θn with the vector a giving the minimum/maximum being the corresponing eigenvector of the aforementione matrix. Comparing efficiencies in the asymptotic sense is analogous to the finite sample case. For the asymptotic case, assume θ n an θ n are n consistent, that is n θn θ D Norm 0, AV θ, n θn θ D Norm 0, AV θ where AV θ an AV θ are the asymptotic variance-covariance matrix of the estimators θ n an θ n respectively. For any a R, it follows that n a t θn a t θ D Norm 0, a t AV θa, n a t θn a t θ D Norm 0, a t AV θa

23 1 Again the focus will be on the ratio a t AV θa/a t AV θa for a R. This value of this ratio at its minimum/maximum equals the smallest/largest eigenvalue of the matrix AV 1 θ AV θ with the vector a being the corresponing eigenvectors of the aforementione matrix. Provie the istribution is such that the asymptotic version of the multivariate information inequality hols, the MLE of θ is the best one can o, hence one is often intereste in the asymptotic efficiency of an estimator relative to the MLE. The asymptotic variance-covariance matrix of the MLE of θ is given by the inverse of the Fisher information matrix, enote I θ. Thus one has AV θ I 1 θ where refers to the usual partial orering of symmetric matrices. Recall for two symmetric matrices, A an B, > is a partial orering of these matrices such that A B A > B if an only if A B is positive semi-efinite efinite. The asymptotic efficiency of an estimator is efine to be a AE θ t I 1 θ a = max a R As mentione above, the value of AE θ a t AV θa is given by the largest eigenvalue of the matrix AV 1 θ I 1 θ with a being the corresponing eigenvector..3. The Asymptotic Efficiency of the Spatial Meian Let x E µ, Σ; G be a ranom vector that is absolutely continuous with respect to Lebesgue measure. The Fisher information matrix for µ for a fixe Σ an G is given by I µ; Σ, G = 1 α G Σ 1 where α G = E [ u R G] 1 is a scalar that epens only on G with u s = sg s /g s [30]. Let µ be any estimator of µ. By the asymptotic version of the information inequality one has that AV µ I 1 µ; Σ, G = α G Σ. It is shown in section.5. of the appenix that the asymptotic variance-covariance matrix of the spatial meian uner the elliptical moel is given below as

24 13 AV SM Σ, G = β G PV Λ P t where β G = 1/E [1/R G ] an V Λ = iag ν 1 Λ,..., ν Λ with νi Λ = [ E [ E λ i u,i Λ 1/ u 3 λ i u,i Λ 1/ u ] 1 Λ 1/ u I ] an u,i representing the i th component of the ranom unit vector u. For the family of elliptical istributions E µ, Σ; G with fixe G further suppose that the following conitions are also satisfie Conitions i E [1/R G ] > 0 an ii g is boune. The following theorem formally states that for any family of elliptical istributions with fixe G that satisfies the conition given above, the asymptotic efficiency of the spatial meian is uniquely maximize at a spherically symmetric member of this family, Theorem Let x 1,..., x n represent an i.i.. sample from E µ, Σ; G that satisfies conitions Then with equality holing if an only if Σ I. AE µ SM ; σ I, G AE µ SM ; Σ, G Proof. See section.5.4 of the appenix..3.3 The Oracle Spatial Meian an Affine Spatial Meian As mentione in the introuction to this chapter, the calculation of the spatial meian involves own-weighing observations in terms of their Eucliean istances as oppose to their Mahalanobis istance from the estimate center of the ata. Consequently, uner the elliptical moel one might surmise that an estimator that oes the latter

25 14 woul work better in the case when Σ I. The following estimator oes just the above. Consier the situation that Σ is known an not proportional to I, efine, n µ OSM = µ Σ = argmin η R x i η Σ 1 Note that this is the MLE for µ when sampling from an elliptical Laplace istribution when Σ is known; this follows from the likelihoo function compute in section. Denote this estimator as the oracle spatial meian since it requires knowing the scatter i=1 matrix of the ata a priori. It is shown in section.5.3 of the appenix that the asymptotic variance-covariance matrix of the oracle spatial meian is given by AV µosm Σ, G = 1 β G Σ As allue to, one might suspect that for elliptical istributions, this estimator for µ is more efficient than the spatial meian. This is state in the following theorem, Theorem Let x 1,..., x n represent an i.i.. sample from E µ, Σ; G that satisfies conitions Then AV µosm Σ, G < AV µsm Σ, G unless Σ I in which case equality hols. Proof. From the theory of maximum likelihoo estimation, for the case when the particular istribution is the elliptical Laplace istribution, i.e. G = G L, then AV µosm Σ, G L < AV µsm Σ, G L unless µ OSM an µ SM are asymptotically equivalent. However in the proof of Theorem 3.3.1, when Σ I then AV µsm Σ, G Σ whereas AV µosm Σ, G L is. Thus µ ASM an µ SM have ifferent asymptotic variancecovariance matrices in this instance, thus strict inequality hols when Σ I. It follows that AV µosm Σ, G L = 1 β G L Σ < β G L PV Λ P t = AV µsm Σ, G L Canceling β G L from both sies gives then multiplying by β G gives

26 15 AV µosm Σ, G = 1 β G Σ < β G PV Λ Pt = AV µsm Σ, G the esire result. When Σ I, then µ OSM = µ SM, thus the two estimators will have the same asymptotic variance-covariance matrix, hence equality hols. QED In practice Σ is not known, however, can be estimate from the ata. Let Σ n be an affinely equivariant estimate of Σ base on a sample of size n. Replacing Σ with Σ n in the efinition of the oracle spatial meian yiels an affinely equivariant estimate of the location parameter, that is µ ASM = µ Σn = argmin η R n i=1 x i η Σ 1 n Refer to this estimator as an affinely equivariant spatial meian base on the scatter estimate Σ n, or simply an affine spatial meian. If the elliptical istribution uner consieration is such that Σ n = Σ + O P n 1/.3.1 then it follows by theorem 3 of [9] that n µ Σn µ OSM P 0, thus n µ Σn µ D Norm 0, AV µ Σ Σ, G where AV µ Σ Σ, G = AV µosm Σ, G. This leas to the following corollary to Theorem Corollary Let x 1,..., x n be a ranom sample from E µ, Σ; G such that conitions 3..1 are satisfie. For any affinely equivariant estimate of scatter Σ n, such that Σ n = Σ + O P n 1/, it follows AV µ Σ G < AV µsm G, Σ unless Σ I in which case equality hols.

27 16 This corollary states that the spatial meian is asymptotically inamissible over the class of elliptical istributions. Note however for a particular estimator of scatter, this result hols true only when the conitions in equation.3.1 are met. For instance, if Σ n is the sample covariance matrix, then this result hols only if the elliptical istribution has finite fourth moments. However, for a broa class of M-estimates of multivariate scatter efine in [3], the conition in equation.3.1 hols without any further assumptions on G..3.4 Asymptotic Efficiency Calculations In the previous section, it was mentione that the oracle spatial meian an affine spatial meian are asymptotically equivalent provie that Σ n = Σ + O P n 1/. Furthermore, uner the elliptical moel it was shown that both of these were asymptotically more efficient at estimating the µ than the spatial meian. To unerstan more precisely how inefficient the spatial meian can be compare to either of the aforementione estimators uner the elliptical moel, the asymptotic relative efficiency of the spatial meian relative to the oracle spatial meian will be compute uner various imensions an scatter structures. As will be seen, the values of the asymptotic efficiencies will not epen on the particular elliptical istribution. The results in section 3. an 3.3 give, AV µosm Σ, G = 1 β G Σ an AV µ SM Σ, G = β G PV Λ P t with V Λ = iag ν1 Λ,..., ν Λ where [ ] λ E i u,i νi Λ 1/ u Λ = [ ] E λ i u,i 1 I Λ 1/ u 3 Λ 1/ u Since AV µsm Σ, G has the same eigenvectors as Σ, the asymptotic relative efficiency of µ SM to µ OSM reuces to a comparison of the eigenvalues of AV µosm Σ, G an AV µsm Σ, G corresponing to the same eigenvectors. Thus without loss of generality one can reuce consieration to the simple case where P = I. As a consequence of this

28 17 simplification there is a convenient interpretation of the aforementione comparison of eigenvalues. For uncorrelate scatter structure, the eigenvalues of the asymptotic variance-covariance matrix correspon to the variances of the estimate components. Thus a comparison of the eigenvalues reuces to a comparison of the variability with which each estimator estimates the components of µ, i.e. the asymptotic relative efficiency for the components of the spatial meian to the oracle spatial meian. The asymptotic relative efficiency of each component can be compute an is equal to [ AV µλ Λ, G ] i ARE i µ SM, µ OSM ; Λ, G = [ AV µsm Λ, G] i Note that for the expression νi Λ, one has = 1 λ i ν i Λ ν i c Λ = = = = [ ] c E λ i u,i cλ 1/ u [ ] E c λ i u,i 1 I cλ 1/ u 3 cλ 1/ u [ ] c E λ i u,i c Λ 1/ u [ ] E c λ i u,i 1 I c 3 Λ 1/ u 3 c Λ 1/ u [ ] λ E i u,i Λ 1/ u [ ] E λ i u,i 1 I c Λ 1/ u 3 c Λ 1/ u [ ] λ E i u,i Λ 1/ u [ ] 1 E c λ i u,i 1 I Λ 1/ u 3 Λ 1/ u = c ν i Λ thus ARE i c Λ = 1 c λ i c ν i Λ = 1 λ i ν i Λ = ARE i Λ this implies the size parameter oes not matter in the calculation of the asymptotic relative efficiency. Define r i = λ i λ 1, that is r i is the ratio of the scale of the largest

29 18 ARE of Component ARE of Component r_ r_ Figure.1: Asymptotic Relative Efficiencies of the Spatial Meian to the Oracle Spatial Meian in R component to the scale of the i th component. Consequently, when consiering the asymptotic efficiency of each component one can take the case when the scale of the largest component is fixe; for simplicity assume it is equal to 1. Consier the situation Σ = Λ 0 = iagλ,..., λ, r λ,......, r λ where 0 r }{{}}{{} Using the results in section.5.4 of the appenix, it follows from Lemma 5..3 that the efficiency of the any of the first 1 components is given by ARE 1 r = F1 1, 1 ; + ; 1 r F 1 1, 1 ; + ; 1 r The efficiency of the last 1 components is given by ARE 1 r = F1 1, 1 ; + ; 1 r F 1 1, 1 ; + ; 1 r where F 1 a, b; c; k = B 1 b, c b 1 0 xb 1 1 x c b 1 1 kx a x is the Gauss hypergeometric function. efficiencies of each component as a function of r. Starting with two imensions, plotte in Figure.1 are the For the component associate with the higher scale, the spatial meian oes not o

30 19 ARE of Components 1& ARE of Component r_ r_3 Figure.: Asymptotic Relative Efficiencies of the Spatial Meian to the Oracle Spatial Meian in R 3 : Λ 0 = iag 1, 1, r much worse than the oracle spatial meian in estimating this component, even in the most extreme cases when Λ is nearly singular. However, for the component associate with the smaller scale, the asymptotic relative efficiency of the spatial meian to the oracle spatial meian is quite low when r is small inicating the rastic inferiority of the precision with which the spatial meian estimates this component compare to the oracle spatial meian. For three imensions, perhaps the most interesting scatter structures are those in which two of the λ s are equal, that is λ 1 = λ or λ = λ 3. The first case correspons to r = 1, the efficiency for each of the components as a function of r is given in Figure., For the secon case, r = r 3 = r. Presente in Figure.3 are the asymptotic efficiencies of the components as a function of r. In both cases note that the efficiency for the larger components of the spatial meian relative to the oracle spatial meian is still relatively high even for nearly singular scatter structures. However for the smaller components as the scatter structure gets more singular, the relative efficiencies iminish rastically. Of the two situations, the scenario with which λ = λ 3 is the most eleterious on the relative efficiencies of the

31 0 ARE of Component ARE of Components & r r Figure.3: Asymptotic Relative Efficiencies of the Spatial Meian to the Oracle Spatial Meian in R 3 : Λ 0 = iag 1, r, r components. As the imension increases, the number of possible scatter structures to consier greatly increases. The last imension examine will be = 10 uner the following scatter structures, Λ 1 = iag1, r, r, r, r, r, r, r, r, r Λ = iag1, 1, r, r, r, r, r, r, r, r Λ 3 = iag1, 1, 1, 1, 1, r, r, r, r, r Λ 4 = iag1, 1, 1, 1, 1, 1, 1, 1, r, r Λ 5 = iag1, 1, 1, 1, 1, 1, 1, 1, 1, r In an 3 imensions, it was always that case the components associate with the larger scale ha higher efficiencies. However, this is not true in general. In fact it is not only the scale that affects the efficiency of a given component, but also the number of components that have scales of similar magnitues. Presente in Figure.4 are the efficiencies of the components for scatter structures given above.

32 1 ARE of Component LAMBDA_1 ARE of Components ARE of Components 1& LAMBDA_ ARE of Components ARE of Components LAMBDA_3 ARE of Components ARE of Components LAMBDA_4 ARE of Components 9& ARE of Components LAMBDA_5 ARE of Component LAMBDA_1 LAMBDA_ LAMBDA_3 LAMBDA_4 LAMBDA_5 Figure.4: Asymptotic Relative Efficiencies of the Spatial Meian to the Oracle Spatial Meian in R 10 As the number of components with larger scales increases, the efficiencies of all the components, not just the ones with the larger scales, improves. This was also the case in three imensions. Also of note, for scatter structures Λ 1 an Λ, there are values of r in which the efficiencies are slightly higher for the components with the smaller scales, whereas for the rest of the scatter structures Λ the efficiencies are always higher for the components with the larger scales..4 Finite Sample Performance.4.1 Finite Sample Theory For elliptical istributions it was proven that asymptotically, one can always fin a more efficient estimator than the spatial meian by using an affinely equivariant version of it. In section 3.5, exactly how asymptotically inefficient the spatial meian is compare to the oracle spatial meian or affine spatial meian was consiere uner various imensions an scatter structures. However, for finite samples, working out the exact istribution of the aforementione estimators is intractable, thus one must resort to simulations in orer to ascertain the efficiencies. For finite samples, there are two factors that must be consiere when comparing the efficiency of the spatial meian to an affinely equivariant version. The first is how the efficiency is affecte by the fact

33 one is estimating Σ with an affinely equivariant estimator of it. Asymptotically, this was shown not to matter provie the estimator of scatter converges in probability to Σ, however, for finite samples how one estimates Σ will be of consequence. The secon consieration is how much efficiency is lost by sacrificing affine equivariance for only orthogonal equivariance an oes this epen on the particular affine equivariant estimator of Σ. It will be shown that these two factors can be consiere separately. To this en the following theorem is neee. The proof is relegate to section.5.5. of the appenix. Theorem Let x 1,..., x n represent an i.i.. sample from E µ, Σ; G. Let λ 1 > λ > > λ m be the istinct eigenvalues of Σ where m is the number of mutually orthogonal eigenspaces of Σ. For any orthogonally equivariant estimator of µ base on a sample of size n, µ n, with variance-covariance matrix V n µ = V ar [ µ n ], the following are true, 1 V n µ an Σ have the same eigenspaces; consequently they have the same eigenprojections an/or eigenvectors. Let λ 1,n λ,n λ,n, enote the eigenvalues of V n µ. It follows the eigenspace associate with λ i,n is the same that is associate with λ i. Consequently, λ i,n = λ j,n if an only if λ i = λ j. Proof. See section.5.5 of the appenix. The above theorem, couple with affine equivariance arguments, implies that when sampling from an elliptical istribution, the variance-covariance matrix of the oracle spatial meian, affine spatial meian an spatial meian have the following forms, provie they exist,

34 3 V n µ Σ = α n G Σ, V n µ Σ = β Σn G Σ, V n µ SM = PV n Λ; G P t with α n G being a positive scalar that epens only on n an G whereas β Σn G epens on n, G an the choice of scatter estimator, Σ n. The matrix V n Λ; G is a iagonal an epens on n, G an Λ, that is V n Λ; G = iag ν 1,n Λ; G,..., ν,n Λ; G. As a consequence of the fact that the variance-covariance matrix of the oracle an affine spatial meian are both proportional to Σ, the finite sample relative efficiency of µ Σn to µ Σ reuces to a scalar quantity, namely RE n µ Σ, µ Σ = αn G /β Σn G. Furthermore, since the variance-covariance matrix of the spatial meian has the same eigenvectors as Σ, the finite sample relative efficiency of µ SM to either µ Σ to µ Σ reuces to a comparison of eigenvalues of variance-covariance matrices corresponing to the same eigenvectors. Thus without loss of generality one can reuce consieration to the simple case where P = I. This simplification yiels the same convenient interpretation as it i for comparing the asymptotic efficiencies in the previous section, namely the comparison of the eigenvalues reuces to a comparison of the variability with which each estimator estimates the components of µ, i.e. the finite sample relative efficiency for the components of the spatial meian to an affine spatial meian. This can be expresse as, RE i,n µ SM, µ Σn = β Σn G λ j ν j,n Λ; G = α n G λ i ν i,n Λ; G β Σn G α n G = RE i,n µ SM, µ Σ RE n µ Σ, µ Σ for i = 1,...,. Note that RE i,n µ SM, µ Σ only epens on n, Σ an G, but not the choice of estimator use for Σ. For a fixe n an G, this reflects how the relative efficiency of the j th components is effecte by the fact Σ is not proportional to the ientity matrix. The term RE n µ Σ, µ Σ epens only on n, Σn an G, thus can be viewe as a measure of how the relative efficiency of the components is affecte by the choice of the scatter estimate, Σ n. For the simulations, four elliptical istributions will be consiere: the normal, Cauchy, t 3 an slash istributions; the reaer is referre to [41]. The imensions being