Proceedigs of the World Cogress o Egieerig 2008 Vol II WCE 2008, July 2-4, 2008, Lodo, UK Multivariate Aalysis of Variace Usig a Kotz Type Distributio Kusaya Plugpogpu, ad Dayaad N Naik Abstract Most stadard iferetial statistical methods for multivariate data are developed uder the fudametal assumptio that the data are from a multivariate ormal distributio Ufortuately, oe ca ever be sure whether a set of data is really from a multivariate ormal distributio There are umerous methods for checkig (testig) multivariate ormality, but these tests are geerally ot very powerful, especially for smaller sample sizes Hece it is always beeficial to have alterative multivariate distributios ad the methodology for usig them I this article, we cosider a Kotz type multivariate distributio which has fatter tail regios tha that of multivariate ormal distributio ad show how multivariate aalysis of variace ca be performed usig this distributio as model Keywords: Geeralized spatial media, Kotz type distributio, simultaeous cofidece itervals, testig the equality of mea vectors Itroductio Multivariate ormal distributio is fudametal for multivariate aalysis of variace Elegat results are obtaied uder this model However, i practice, the assumptio of this distributio may ot be valid Numerous classes of multivariate distributios have bee used i practice i place of multivariate ormal distributio See [3]-[5] ad [] I this article, we cosider a Kotz type multivariate distributio (of a p variate radom vector X) which has fatter tail regios tha that of multivariate ormal distributio ad its probability desity fuctio (pdf ) is give by: f(x, µ, Σ) = c p Σ 2 exp { [(x µ) Σ (x µ)] 2 }, () where µ R p, Σ is a positive defiite matrix (pd) ad c p = Γ( p 2 ) 2π p 2 Γ(p) This pdf has appeared i the literature i differet forms For example, the pdf is a special case of the followig families of distributios: Mauscript submitted March 20, 2008 K Plugpogpu is with Departmet of Statistics, Silpakor Uiversity, Bagkok, Thailad (e-mail: kusaya@suacth), ad D N Naik is with Departmet of Mathematics ad Statistics, Old Domiio Uiversity, Norfolk, Virgiia 2329, USA (e-mail: daik@oduedu) Multivariate distributios proposed by Simoi (see [8]): These have the pdf proportioal to exp{ r [(x µ) A(x µ)] r 2 }, where A is p d ad r For r = oe obtais our multivariate distributio Elliptically symmetric distributios (see [8]): Let X be a p radom vector, µ be a p vector i R p, ad Σ be a p p o-egative defiite matrix The X has a elliptically cotoured distributio, deoted by EC p (µ, Σ, ψ) if the characteristic fuctio φ X µ(t) = E[exp(it (X µ))] of X µ is a fuctio of the quadratic form t Σt as φ X µ(t) = ψ(t Σt) for some fuctio ψ Therefore, the elliptically symmetric distributios deoted by EC p (µ, Σ, g), have the pdf (here Σ is pd) i the form f(x) = k p Σ 2 g[(x µ) Σ (x µ)], where g is a oe-dimesioal real-valued fuctio idepedet of p ad k p = pγ( p 2 ) π p 2 Γ( + p 2β )2+ p 2β For our distributio g(t) = exp{ t 2 } Power expoetial distributios (see [5]): A radom vector X has a p-dimesioal power expoetial distributio, deoted by P E p (µ, Σ, β), with µ, Σ, ad β, where µ R p, Σ is a p p p d matrix, ad β (0, ) Its desity fuctio is f(x, µ, Σ, β) = k Σ 2 exp{ 2 [(x µ) Σ (x µ)] β }, where k = pγ( p 2 ) π p 2 Γ(+ p p 2β )2+ 2β For β = 2 oe obtais our distributio This fuctio is actually the pdf of a elliptically cotoured radom vector EC p (µ, Σ, g) Kotz type distributios (see [4]): If X EC p (µ, Σ, g) ad the desity geerator g is of the form g(u) = c p u N exp( ru s ), r, s > 0, 2N + p > 2 the we say
Proceedigs of the World Cogress o Egieerig 2008 Vol II WCE 2008, July 2-4, 2008, Lodo, UK that X possesses a symmetric Kotz distributio The pdf of X is give by f(x, µ, Σ) = c p Σ 2 [(x µ) Σ (x µ)] N where c p = exp { r[(x µ) Σ (x µ)] s }, sγ( p 2 ) r 2N+p 2 π p 2 Γ( 2N+p 2 2s 2s ) Whe N =, s = 2, ad r = the distributio reduces to our distributio The pdf () ca also be writte as a multivariate ormal mixtures as i [9] ad [0] The Kotz type distributio with the pdf give i () has heavier tail regios tha those covered by the multivariate ormal distributio ad hece ca be useful i providig robustess agaist outliers (see [3]) For p =, the pdf () reduces to that of a double expoetial (or Laplace) distributio Hece we may treat this distributio as a multivariate geeralizatio of double expoetial distributio However, this is ot a multivariate double expoetial distributio because, its margial distributios are ot double expoetial distributios We ote that double expoetial distributio is symmetric aroud a locatio parameter µ, ad the maximum likelihood estimate of µ is the media It is well kow that a media is more robust estimator of a locatio parameter tha the mea For this reaso, may times i practice double expoetial (Laplace) distributio is used for data aalysis istead of ormal distributio I our earlier paper, [5], we have discussed various characteristics of the distributio (), icludig its margial ad coditioal distributios ad momets We ote that E(X) = µ, ad V ar(x) = (p+)σ Also, we provided a algorithm for simulatig samples from this distributio The maximum likelihood estimators of µ ad Σ, were also derived ad the asymptotic distributio of the maximum likelihood estimate of µ was give Further, usig Mardia s multivariate measures of skewess ad kurtosis, we provided a goodess-of-fit test for Kotz type distributio Iferece for parameter vector µ was also discussed i [5] I this article we discuss how this distributio ca be used to perform multivariate aalysis of variace I the ext sectio, for ease of readig, we will provide the maximum likelihood estimate of µ ad its asymptotic distributio ad also provide some details o how to costruct simultaeous cofidece itervals usig Boferroi probability iequality It is worth otig that the most iterestig property of the distributio i had is that the maximum likelihood estimators uder this distributio are the geeralized spatial media (GSM) estimators as defied i [6] Sectios 3 ad 4 discuss oe way multivariate aalysis of variace A example to illustrate the methods is cosidered i Sectio 5 ad cocludig remarks are provided i Sectio 6 2 Estimatio of Parameters May researchers have discussed statistical iferece usig elliptical distributios For example, see [3] ad the refereces therei However, the maximum likelihood theory developed i [3] assumes that the joit distributio of the radom sample, X,, X, is elliptically symmetric I fact, i this case the maximum likelihood estimators of µ ad Σ are essetially same as those i the multivariate ormal case (see [3]) Several authors have performed statistical iferece based o certai elliptical distributios For example, [2] used multivariate t-distributio ad maximum likelihood method to aalyze certai regressio ad repeated measuremets, ad [3] used multivariate power expoetial distributio to aalyze a certai repeated measuremets I each case umerical algorithms are used to fid the estimates of the parameters I the followig we discuss estimatio of parameters usig maximum likelihood methods whe a idepedet idetically distributed sample from () is available Suppose X,, X is a radom sample from Kotz type distributio () The the log-likelihood fuctio is give by l L(µ, Σ) = l c 2 l Σ (x i µ) Σ (x i µ) The MLEs of µ ad Σ are obtaied by miimizig 2 l Σ + (x i µ) Σ (x i µ) (2) simultaeously wrt µ ad Σ Whe Σ = I, the solutio to the above problem or the MLE of µ is the spatial media itroduced i [7] ad for geeral Σ it is geeralized spatial media itroduced i [6] ad studied i [4] J B S Haldae defied (see [7]) the spatial media of multivariate data vectors x,, x, as a poit (vector) ˆµ R p which miimizes x i µ = (xi µ) (x i µ) with respect to µ For p >, the vector ˆµ is uique except whe all the mass of the distributio is cocetrated o a lie ad is ivariat uder orthogoal trasformatio, but ot uder affie trasformatio (see [], [2]) C R Rao (see [6]) defied two geeralized spatial medias which are ivariat uder affie trasformatio as: (i) a vector ˆµ which miimizes (x i µ) S (x i µ)
Proceedigs of the World Cogress o Egieerig 2008 Vol II WCE 2008, July 2-4, 2008, Lodo, UK with respect to µ, where S is the usual sample variace covariace matrix, ad (ii) a vector ˆµ which miimizes 2 l Σ + (x i µ) Σ (x i µ) simultaeously with respect to µ ad Σ Thus, we ote that the MLE of µ uder the assumptio of Kotz type distributio () for X,, X is same as the geeralized spatial media defied i [6] Computatio of GSM ad Σ: Let X,, X be a radom sample from () The the GSM of µ which miimizes (2) ca be computed i two stages as follows (see [4]) Suppose Σ is kow or set to a iitial value ad Σ = GG, for a osigular G The the geeralized spatial media ˆµ which miimizes (x i µ) Σ (x i µ) wrt µ ca be obtaied as ˆµ = Gˆν, where ˆν is the spatial media which miimizes (yi ν) (y i ν) wrt ν Here y i = G x i ad ν = G µ Spatial media ca be computed usig a algorithm give i [6] Next usig ˆµ the maximum likelihood estimate of Σ is obtaied as the matrix Σ which miimizes (2) with respect to Σ as a solutio to the o-liear equatios give by Σ = (x i ˆµ)(x i ˆµ) (x i ˆµ) Σ (x i ˆµ) We use oliear optimizatio methods to obtai maximum likelihood estimates of all the parameters We have adopted SAS IML procedure for writig the computer programs Usig the Newto Raphso method the optimizatio yields uique estimates i the feasible regios uder most covariace structures Theorem (The asymptotic distributio of GSM): Let X,, X be a radom sample from p variate (p > ) Kotz type distributio () with parameters µ ad Σ ad ˆµ be the maximum likelihood estimate of µ The D (ˆµ µ) N(0, ΣA BA Σ), [ ] (X µ)(x µ) where B = E ad (X µ) Σ (X µ) (X µ)(x µ) A = E[ (Σ (X µ) Σ (X µ) (X µ) Σ (X µ) )] Further B ad A ca be estimated by B =  = (x i ˆµ)(x i ˆµ) (x i ˆµ) Σ, (x i ˆµ) (x i ˆµ) Σ (x i ˆµ) [ Σ (x i ˆµ)(x i ˆµ) (x i ˆµ) Σ ], (x i ˆµ) where Σ is the maximum likelihood estimate of Σ May times i practice, we may be iterested i performig simultaeous iferece o a set of k parameters, for example, o compoets of vector µ Oe coveiet ad easy way to build simultaeous cofidece itervals o these parameters is usig the Boferroi method It is a simple method that allows the costructio of may cofidece itervals maitaiig a overall cofidece coefficiet The method is based o Boferroi s probability iequality, P ( k A i) k P (Ac i ), where A i is the evet that the i th cofidece iterval cotais the correspodig parameter ad A c i is the complemet of that evet Hece the left had side of Boferroi s iequality is the probability that all the cofidece itervals simultaeously cotai their correspodig true parameter values ad the right had side is oe mius the sum of the probabilities that the itervals do ot cotai the correspodig true values Thus if we wat the overall cofidece coefficiet to be α the we should costruct the idividual cofidece iterval with a cofidece level of α/k Propositio (Simultaeous Cofidece Itervals): The 00( α)% Boferroi simultaeous cofidece itervals for m liear fuctios of µ, is, are give by ( ) a a i ˆµ z i τ a i a α/2m, a i ˆµ + z i τ a i α/2m, i =,, m, where a, i s are vectors of kow costats ad z α/2m is the upper 00( α/2m) th percetile of a stadard ormal distributio, ad τ = Σ BÂ Σ I [5], we have used these results to perform iferece o the compoets of vector µ 3 Testig Equality of Mea Vectors Suppose X i, X i2,, X ii is a radom sample of size i from Kotz type populatio with the parameters µ i ad Σ i, i =,, g The radom samples from differet g populatios are assumed to be idepedet Let µ i = (µ i, µ i2,, µ ip ) ad σ i = (σ i,,, σ i,p,, σ i,p,p, σ i,pp ), i =,, g Let θ = (µ,, µ g, σ,, σ g) be the vector of all ukow parameters
Proceedigs of the World Cogress o Egieerig 2008 Vol II WCE 2008, July 2-4, 2008, Lodo, UK Cosider the problem of testig H 0 : µ = µ 2 = = µ g = µ whe Σ = Σ 2 = = Σ g = Σ, that is, whe σ = = σ g = σ, where σ = (σ,, σ p,, σ p,p, σ pp ) Uder H 0, let θ = ( µ, σ ) be the MLE of θ The the maximum of the likelihood fuctio uder H 0 is give by L( θ) = (c p ) Σ /2 e g i j= (x ij µ) Σ (x ij µ), where = ( g i) Next, let θ = (ˆµ,, ˆµ g, σ ) be the MLE of θ uder o restrictios The the maximum of the likelihood fuctio is give by L( θ) = (c p) Σ /2 e g i j= (x ij ˆµ i ) Σ (x ij ˆµ ) i The the likelihood ratio test for testig H 0 rejects H 0 if Λ = L( θ) L( θ) < c, where c is the critical value to be obtaied appropriately If σ, i s are differet ad we wated to test H 0 : µ = µ 2 = = µ g = µ, the likelihood ratio test for testig H 0 is give as follows Let θ = ( µ, σ,, σ g) be the MLE of θ The the maximum of the likelihood fuctio is give by L( θ) = (c p ) g Σ i i/2 e g i j= (x ij µ) Σ i (x ij µ) Uder o restrictios, let θ = (ˆµ,, ˆµ g, σ,, σ g) be the MLE of θ The the maximum likelihood fuctio is give by L( θ) = (c p) g Σ i i/2 e g i j= (x ij ˆµ i ) Σ i (x ij ˆµ ) i The the likelihood ratio test rejects H 0 if Λ = L( θ) L( θ) < c, where c is a suitably chose costat Whe the sample size is large, ( L( θ) ) 2 l Λ = 2 l is approximately distributed as χ 2 r L( θ) radom variable, where the degrees of freedom, r = (dimesio of θ uder o restrictios) - (dimesio of θ uder H 0 ) 4 Simultaeous Cofidece Itervals Let X i, X i2,, X ii, i =,, g, be g idepedet radom samples of size i each from Kotz type distributios with parameters µ i, ad Σ i, i =,, g Suppose the tests have revealed that a sigificat differece exists betwee the populatio meas I order to pipoit the differeces we costruct simultaeous cofidece itervals o various cotrasts of differece betwee ay two mea vectors The followig results provide distributioal results that eable costructig simultaeous cofidece itervals for liear combiatios of µ, ij s Propositio 2 (Simultaeous Cofidece Itervals): Let X i, X i2,, X ii be a radom sample of size i from Kotz type distributio with parameters µ i, ad Σ i, i =,, g ad suppose the samples from differet g populatios are idepedet Suppose Σ = = Σ g = Σ Usig the Theorem ad Propositio, the 00( α)% Boferroi simultaeous cofidece itervals for m liear combiatios of µ l µ l, l < l =,, g are give by a k(ˆµ l ˆµ l ) ± z α/2m a k ( ˆτ l + ˆτ l )a k, k =,, m (3) l l where a, k s are vectors of kow costats, z α/2m is the upper 00( α/2m)th percetile of the stadard ormal distributio, τ i = Σ i B i  i Σ, i = l, l =,, g, ad B i = i i j=  i = i i j= (x ij ˆµ i )(x ij ˆµ i ) (x ij ˆµ i ) Σ, (x ij ˆµ i ) (x ij ˆµ i ) Σ (x ij ˆµ i ) [ Σ (x ij ˆµ i )(x ij ˆµ i ) (x ij ˆµ i ) Σ ] (x ij ˆµ i ) If the variace covariace matrices from differet populatios are differet the we have the followig result Propositio 3 (Simultaeous Cofidece Itervals): Like before, let X i, X i2,, X ii be a radom sample of size i from Kotz type distributio with µ i, ad Σ i, i =,, g, ad the samples from differet g populatios are idepedet The 00( α)% Boferroi simultaeous cofidece itervals for m liear combiatios of µ l µ l, l < l =,, g are give by a k(ˆµ l ˆµ l ) ± z α/2m a k ( ˆτ l + ˆτ l )a k, k =,, m (4) l l where a, k s are vectors of kow costats, z α/2m is the upper 00( α/2m)th percetile of the stadard ormal distributio, τ i = Σ i  i B i  i Σ i, i = l, l =,, g, ad B i = i i j= (x ij ˆµ i )(x ij ˆµ i ) (x ij ˆµ i ) Σ i (x ij ˆµ i ),
Proceedigs of the World Cogress o Egieerig 2008 Vol II WCE 2008, July 2-4, 2008, Lodo, UK  i = i i j= (x ij ˆµ i ) Σ i (x ij ˆµ i ) [ Σ i (x ij ˆµ i )(x ij ˆµ i ) ] (x ij ˆµ i ) Σ i (x ij ˆµ i ) Usig the followig example, we illustrate the computatio of the maximum likelihood estimates ad perform some statistical iferece uder Kotz type distributio () All the computatios are doe usig programs writte i SAS/IML software 5 A Example I the followig, we illustrate the procedure for testig the equality of several populatio meas usig the Football helmet data give i the example below Before testig the equality of the meas, we first test the equality of the variace covariace matrices usig the likelihood ratio test Data o three variables, x = eyeto-top-of-head measuremet, x 2 = ear-to-top-of-head measuremet, ad x 3 = jaw width are give for three groups of players, amely, high school football players, college football players, ad o-football players There are 30 observatios i each group The helmet data collected as part of a prelimiary study of a possible lik betwee football helmet desig ad eck ijuries are provided i [7] The hypotheses ad the results from testig are as follows: (i) Test H 0 : Σ = Σ 2 = Σ 3 = Σ The test statistic = 2 lλ = 3054878 Usig 2 lλ χ 2 2, the P-value = 03650646 Hece, we do ot reject H 0 ad coclude that the variace covariace matrices are the same for the three groups (ii) Next, we test H 0 : µ = µ 2 = µ 3 = µ, give Σ = Σ 2 = Σ 3 = Σ The test statistic = 2 lλ = 9703 The P-value = P [χ 2 6 > 9703] < 0000 Hece we reject H 0 ad coclude that at least two µ, is are differet (iii) Sice we rejected the hypothesis of equality of meas we wat to fid simultaeous cofidece itervals for liear fuctios of µ l µ l, l < l =, 2, 3 Let µ i = (µ i,, µ ip ), i =, 2, 3 The with the choices, a = (, 0, 0), a 2 = (0,, 0), ad a 3 = (0, 0, ) usig (3), the 95% Boferroi simultaeous cofidece itervals for µ j µ 2j, j =, 2, 3 are: µ µ 2 (22932274, 3934077), µ 2 µ 22 (04294995, 20009086), µ 3 µ 23 ( 022608, 0923095) The 95% Boferroi simultaeous cofidece itervals for µ j µ 3j, j =, 2, 3 are: µ µ 3 (387555, 327359), µ 2 µ 32 (03040446, 628665), µ 3 µ 33 ( 0000264, 065952) The 95% Boferroi simultaeous cofidece itervals for µ 2j µ 3j, j =, 2, 3 are: µ 2 µ 3 ( 70963, 0002829), µ 22 µ 32 ( 0835963, 03224655), µ 23 µ 33 ( 0374984, 0730842) 6 Cocludig Remarks I our earlier paper [5], we proposed the Kotz type distributio give i () as a alterative to the multivariate ormal distributio for performig multivariate iferece We itroduced various properties of the distributio there ad discussed the maximum likelihood estimatio of the parameters Further we have provided a goodess-of-fit test ad a simulatio algorithm Applicatio of this distributio to multivariate aalysis of variace ad simultaeous cofidece iterval costructio is provided here i this article Usig a example, we have illustrated the computatios However, oe eed to perform a i-depth study, perhaps usig a extesive simulatio, to compare the performace of this distributio uder the presece of outliers ad other sceario agaist the multivariate ormal (the gold stadard) distributio We ited to udertake such a study i the ear future Refereces [] Brow, B M, Statistical Use of the Spatial Media, Joural of the Royal Statistical Society, Series B, vol 45, 983, pp 25-30 [2] Ducharme, G R, ad Milasevic, P, Spatial Media ad Directioal Data, Biometrika, vol 74, 987, pp 22-25 [3] Fag, K T, ad Aderso, T W, Statistical Iferece i Elliptically Cotoured ad Related Distributios, Allerto Press, New York, 990 [4] Fag, K T, Kotz, S, ad Ng, K W, Symmetric Multivariate ad Related Distributios, Chapma ad Hall, Lodo, 990 [5] Gómez, E, Gómez-Villegas, M A, ad Marí, J M, A Multivariate Geeralizatio of the Power Expoetial Family of Distributios, Commuicatios i Statistics, Theory ad Methods, vol 27, 998, pp 589-600
Proceedigs of the World Cogress o Egieerig 2008 Vol II WCE 2008, July 2-4, 2008, Lodo, UK [6] Gower, J S, The Mediacetre, Applied Statistics, vol 23, 974, pp 466-470 [7] Haldae, J B S, Note o the Media of a Multivariate Distributio, Biometrika, vol 35, 948, pp 44-45 [8] Johso, M E, Multivairiate Statistical Simulatio, J Wiley, New York, 987 [9] Kao, Y, Cosistecy Property of Elliptical Probability Desity Fuctios, Joural of Multivariate Aalysis, vol 5, 994, pp 39-47 [0] Kariya, T K, ad Siha, B K, Robustess of Statistical Tests, Academic Press, Bosto, 989 [] Kotz, S, Multivariate Distributios at a Cross- Road, i Statistical Distributios i Scietific Work, Patil, G P, Kotz, S, ad Ord, J K, Eds, D Reidel, The Netherlads, 975, pp 247-270 [2] Lage, K L, Little, R J A, ad Taylor, J M G, Robust Statistical Modelig Usig the t Distributio, Joural of the America Statistical Associatio, vol 84, 989, pp 88-896 [3] Lidsey, J K, Multivariate Elliptically Cotoured Distributios for Repeated Measuremets, Biometrics, vol 55, 999, pp 277-280 [4] Naik, D N, Multivariate Medias: A Review, i Probability ad Statistics, Basu, S K, ad Siha, B K, Eds, Narosa Publishig House, New Delhi, 993, pp 80-90 [5] Naik, D N, ad Plugpogpu, K, A Kotz Type Distributio for Multivariate Statistical Iferece, i Advaces i Distributio Theory, Order Statistics, ad Iferece, Balakrisha, N, Castillo, E, ad Sarabia, J M, Eds, Birkhäuser, Bosto, 2006, pp -24 [6] Rao, C R, Methodology Based o the L -Norm i Statistical Iferece, Sakhyā, Series A, vol 50, 988, pp 289-33 [7] Recher, A C, Methods of Multivariate Aalysis, 2d ed, J Wiley, New York, 2002, pp 280-28 [8] Simoi, S de, Su ua Estesioe dello Schema delle Curve Normalize di Ordia r alle Variabili Doppie, Statistica (Bologa), vol 28, 968, pp 5-70