A Matrix Variate Skew-t Distribution

Transcription

1 A Matrx Varate Skew-t Dstrbuton Mchael P.B. Gallaugher and Paul D. McNcholas arv:73.364v3 [stat.me] 3 Apr 7 Dept. of Mathematcs & Statstcs, McMaster Unversty, Hamlton, Ontaro, Canada. Abstract Although there s ample work n the lterature dealng wth skewness n the multvarate settng, there s a relatve paucty of work n the matrx varate paradgm. Such work s, for example, useful for modellng three-way data. A matrx varate skew-t dstrbuton s derved based on a mean-varance matrx normal mxture. An expectaton-condtonal maxmzaton algorthm s developed for parameter estmaton. Smulated data are used for llustraton. Keywords: Matrx varate dstrbuton; skew-t dstrbuton Introducton Matrx varate dstrbutons have proven to be useful for modellng three-way data, such as multvarate longtudnal data. However, n most cases, the underlyng dstrbuton has been ellptcal such as the matrx varate normal and the matrx varate t dstrbutons. However, there has been relatvely lttle work done on matrx varate data that can account for skewness present n the data. The work that has been carred out n the area of matrx varate skew dstrbutons s mostly lmted to the matrx varate skew-normal dstrbuton. Heren, we derve a matrx varate skew-t dstrbuton. The remander of ths paper s lad out as follows. In Secton, some background s presented. In Secton 3, the densty of the matrx varate skew-t dstrbuton s derved and a parameter estmaton procedure s gven. Secton 4 looks at some smulatons, and we conclude wth a summary and some future work (Secton 5). Background. Matrx Varate Dstrbutons One natural method to model three-way data s to use a matrx-varate dstrbuton. There are many examples n the lterature of such dstrbutons, the most well-known beng the

2 matrx-normal dstrbuton. For notonal clarty, we use to denote a realzaton of a random matrx. An n p random matrx follows a matrx varate normal dstrbuton wth locaton parameter M and scale matrces Σ and Ψ of dmensons n n and p p, respectvely. We wrte N n p (M, Σ, Ψ) to denote such a random matrx and the densty of can be wrtten f( M, Σ, Ψ) = (π) np Σ p Ψ n exp tr ( Σ ( M)Ψ ( M) )}. () One well known property of the matrx varate normal dstrbuton (Harrar and Gupta, 8) s N n p (M, Σ, Ψ) vec( ) N np (vec(m), Ψ Σ) () where N np ( ) s the multvarate normal densty wth dmenson np, vec(m) s the vectorzaton of M, and s the Kronecker product. Although the matrx varate normal s arguably the most mathematcally tractable, there are examples of non-normal cases. One famous example s the Wshart dstrbuton (Wshart, 98) arsng as the dstrbuton of the sample covarance matrx of a multvarate normal sample. More recently, however, there has been some work done n the area of matrx skew dstrbutons such as the matrx-varate skew normal dstrbuton, e.g., Chen and Gupta (5), Domínguez-Molna et al. (7), and Harrar and Gupta (8). More nformaton on matrx varate dstrbutons can be found n Gupta and Nagar (999). Very recently, there has also been work done n the area of fnte mxtures. Specfcally, Anderlucc et al. (5) looked at clusterng and classfcaton of multvarate longtudnal data usng a mxture of matrx varate normal dstrbutons. Also, Doğru et al. (6), looked at mxtures of matrx varate t dstrbutons.. Normal Varance-Mean Mxtures Varous multvarate dstrbutons such as the multvarate t, and skew-t, the shfted asymmetrc Laplace dstrbuton, and the generalzed hyperbolc dstrbutons arse as specal cases of a normal varance-mean mxture (cf. McNcholas, 6, Ch. 6). In ths formulaton, the densty of a p-dmensonal random vector takes the form f(x) = whch s equvalent to the representaton φ p (x µ + wα, wσ)h(w θ)dw, = µ + W α + W V, (3) where V N p (, Σ) and W > s a latent random varable wth densty h(w θ). The multvarate skew-t dstrbuton wth ν degrees of freedom arses as a specal case wth W IG ( ν, ν ), where IG( ) denotes the nverse Gamma dstrbuton wth densty functon f(x α, β) = βα Γ(α) x α exp β x }.

3 .3 The Generalzed Inverse Gaussan Dstrbuton A random varable Y has a generalzed nverse Gaussan (GIG) dstrbuton wth parameters a, b and λ f ts densty functon can be wrtten as where f(y a, b, λ) = K λ (x) = ( a ) λ y λ b K λ ( ab) exp ay + } b y, y λ exp x ( y + )} dy y s the modfed Bessel functon of the thrd knd wth ndex λ. Several functons of GIG random varables have tractable expected values, e.g., b K λ+ ( ab) E(Y ) = a K λ ( ab), (4) ( ) a K λ+ ( ab) E = Y b K λ ( λ ab) b, (5) ( ) b E(log Y ) = log + a K λ ( ab) λ K λ( ab), (6) where a, b R +, λ R, and K λ ( ) s the modfed Bessel functon of the thrd knd wth ndex λ. These results wll prove to be useful for parameter estmaton for the matrx-varate skew-t dstrbuton. 3 Methodology 3. A Matrx Varate Skew-t Dstrbuton We wll say that an n p random matrx has a matrx varate skew-t dstrbuton, MVST n p (M, A, Σ, Ψ, ν), f can be wrtten = M + W A + W V, (7) where M and A are n p matrces, V N n p (, Σ, Ψ), and W IG ( ν, ν ). Analogous to ts multvarate counterpart, M s a locaton matrx, A s a skewness matrx, Σ and Ψ are scale matrces, and ν s the degrees of freedom. It then follows that w N n p (M + wa, wσ, Ψ) 3

4 and thus the jont densty of and W s f(, w ϑ) = f( w)f(w) = ν ν (π) np Σ p Ψ n Γ( ν )w exp ν+np [ ( tr Σ ( M wa)ψ ( M wa) ) + ν ]}, (8) w where ϑ = (M, A, Σ, Ψ, ν). We note that the exponental term n (8) can be wrtten as exp tr(σ ( M)Ψ A ) } exp [ ]} δ(; M, Σ, Ψ) + ν + wρ(a, Σ, Ψ), w where δ(; M, Σ, Ψ) = tr(σ ( M)Ψ ( M) ) and ρ(a, Σ, Ψ) = tr(σ AΨ A ). Therefore, the margnal densty of s f() = = f(, w)dw ν ν Σ p Ψ n Γ( ν ) exp tr(σ ( M)Ψ A ) } (π) np w ν+np exp [ δ(; M, Σ, Ψ) + ν Makng the change of varables gven by ρ(a, Σ, Ψ) y = w δ(; M, Σ, Ψ) + ν we can wrte f MVST ( ϑ) = ( ) ν ν exp tr(σ ( M)Ψ A )} (π) np Σ p Ψ n Γ( ν ) K ν+np w ]} + wρ(a, Σ, Ψ) dw. ( δ(; M, Σ, Ψ) + ν ) ν+np 4 ρ(a, Σ, Ψ) ( ) [ρ(a, Σ, Ψ)] [δ(; M, Σ, Ψ) + ν]. The densty of, as derved here, s consdered a matrx varate extenson of the multvarate skew-t densty used by Murray et al. (4a,b). As dscussed by Dutlleul (999) and Anderlucc et al. (5) n the matrx varate normal case, the estmates of Σ and Ψ are unque only up to a multplcatve constant. Indeed, f we let Σ = vσ and Ψ = (/v)ψ, v, the lkelhood s unchanged. Ths dentfablty ssue can be resolved, for example, 4

5 by settng tr(σ) = n or tr(ψ) = p. Note that Ψ Σ = Ψ Σ, so the estmate of the Kronecker product s unque. For the purposes of parameter estmaton, note that the condtonal densty of W s f(w ) = f( w)f(w) f() = [ρ(a, Σ, Ψ)/(δ(; M, Σ, Ψ) + ν)] λ w λ K λ ( ρ(a, Σ, Ψ)[δ(; M, Σ, Ψ) + ν]) exp Therefore, where λ = (ν + np)/. Fnally, we note that W GIG (ρ(a, Σ, Ψ), δ(; M, Σ, Ψ) + ν, λ), } ρ(a, Σ, Ψ)w + [δ(; M, Σ, Ψ) + ν]/w. MVST n p (M, A, Σ, Ψ, ν) vec( ) MST np (vec(m), vec(a), Ψ Σ, ν), (9) where MST np ( ) denotes the multvarate skew-t dstrbuton wth locaton parameter vec(m), skewness parameter vec(a), scale matrx Ψ Σ, and ν degrees of freedom. Ths can be easly seen from the representaton gven n (3) and the property of the matrx normal dstrbuton gven n (). Note that the normal varance-mean mxture representaton (7) as well as the relatonshp wth the multvarate skew-t dstrbuton (9) present two convenent methods to generate random matrces from the matrx varate skew t dstrbuton. The former s used n Secton Parameter Estmaton Suppose we observe a sample of N matrces,,... N from an n p matrx varate skewt dstrbuton. As wth the multvarate skew-t dstrbuton, we proceed as f the observed data s ncomplete, and ntroduce the latent varables W,..., W n. The complete-data loglkelhood s then l c (ϑ) =C + Nν ( ν ) log Np Nn ( ( ν )) log Σ log Ψ N log Γ ν + tr ( Σ ( M)Ψ A ) + tr ( Σ AΨ ( M) ) w [ tr(σ ( M)Ψ ( M) ) + ν ] log(w ) w tr(σ AΨ A ), where C s constant wth respect to the parameters. We proceed by usng an expectatoncondtonal maxmzaton (ECM) algorthm (Meng and Rubn, 993) descrbed overleaf. 5

6 ) Intalzaton: Intalze the parameters M, A, Σ, Ψ, ν. Set t = ) E Step: Update a, b, c, where a (t+) where and = E(W, ˆϑ (t) ) b (t+) c (t+) δ( ; ˆM (t), ˆΣ (t), ˆΨ (t) ) + ˆν (t) K λ (t) +(κ (t) ) K λ (t)(κ (t) ) = ρ(â(t), ˆΣ (t), ˆΨ (t) ) ( ) (t) = E, ˆϑ W = ρ(â(t), ˆΣ (t), ˆΨ (t) ) K +(κ (t) ) λ(t) δ( ; ˆM (t), ˆΣ (t), ˆΨ + (t) ) + ˆν (t) K λ (t)(κ (t) ) = E(log(W ), ˆϑ (t) ) ( δ( ; = log ˆM (t), ˆΣ (t), ˆΨ ) (t) ) + ˆν (t) ρ(â(t), ˆΣ (t), ˆΨ + (t) ) K λ (t)(κ (t) ) () ˆν (t) + np δ( ; ˆM (t), ˆΣ (t), ˆΨ (t) ) + ν (t) () λ K λ(κ (t) ) κ (t) = [ρ(â(t), ˆΣ (t), ˆΨ (t) )][δ( ; ˆM (t), ˆΣ (t), ˆΨ (t) ) + ˆν (t) ], λ=λ (t) λ (t) = ν(t) + np 3) Frst CM Step: Update the parameters M, A, ν. ) N ˆM (t+) (a (t+) b (t+) = N a(t+) b (t+) N, (3) () Â (t+) = ) N (b (t+) b (t+) N a(t+) b (t+) N, (4) where a (t+) = (/N) N a(t+) and b (t+) = (/N) N b(t+). The update for the degrees of freedom cannot be obtaned n closed form. Instead we solve (5) for ν to obtan ˆν (t+). ( ν ) ( ν ) log + ϕ N (b (t+) + c (t+) ) =, (5) where ϕ( ) s the dgamma functon. 6

7 4) Second CM Step: Update Σ [ ˆΣ (t+) = N ( Np b (t+) ( ˆM (t+) ) ˆΨ(t) ( ˆM (t+) ) Â(t+) ˆΨ(t) ( ˆM (t+) ) ( ˆM (t+) ) ˆΨ(t) Â (t+) + a (t+) Â (t+) ˆΨ(t) (t+) )] Â (6) 5) Thrd CM Step: Update Ψ ˆΨ (t+) = Nn [ N ( b (t+) ( Â(t+) ˆΣ(t+) ˆM ) (t+) )] + a (t+) Â (t+) ˆΣ(t+) Â (t+) ( ˆM (t+) ) ˆΣ(t+) ( ˆM (t+) ) ( ˆM (t+) ) ˆΣ(t+) Â (t+) (7) 6) Check Convergence: If not converged, set t = t + and return to step. Note that there are several optons for determnng convergence of ths ECM algorthm. In the smulatons n Secton 4, a crteron based on the Atken acceleraton (Atken, 96) s used. The Atken acceleraton at teraton t s a (t) = l(t+) l (t), (8) l (t) l (t ) where l (t) s the (observed) log-lkelhood at teraton t. The quantty n (8) can be used to derve an asymptotc estmate (.e., an estmate of the value after very many teratons) of the log-lkelhood at teraton t +,.e., l (t+) = l (t) + a (t) (l(t+) l (t) ) (cf. Böhnng et al., 994; Lndsay, 995). As n McNcholas et al. (), we stop our EM algorthms when l (t+) l (t) < ɛ, provded ths dfference s postve. As dscussed n Dutlleul (999) and Anderlucc et al. (5) for parameter estmaton n the matrx varate normal case, the estmates of Σ and Ψ are unque only up to a multplcatve constant. Indeed, f we let Σ = vσ and Ψ = (/v)ψ, v, the lkelhood s unchanged. However, we notce that, Ψ Σ = Ψ Σ, so the estmate of the Kronecker product s unque. 7

8 4 Smulatons We conduct two smulatons to llustrate the estmaton of the parameters. In both smulatons, we take 5 dfferent datasets of sze, from a 3 4 matrx skew-t dstrbuton. Also, n both smulatons, Σ = , Ψ = and ν = 4. In smulaton, we took the locaton and skewness matrx to be M and A, respectvely, and M and A n smulaton, where M =, A =, M = , A = In Fgures and, we show lne plots of the margnals for each column (labelled V, V, V3, V4) of a typcal dataset from smulatons and, respectvely. The dashed red lnes denote the means. In Fgure, the skewness n columns,, and 4, for smulaton, s very promnent when vsually compared to column 3, whch has zero skewness. The skewness s also apparent n the lneplots for smulaton, however, because the values of the skewness are generally less than those for smulaton, t s not as promnent. The component-wse means of the parameters as well as the component wse standard devatons are gven n Table. We see that the estmates of the mean matrx and skewness matrx are very close to the true value for both smulatons. Moreover, we see that the estmates of Σ and Ψ also correspond approxmately to the ther true values, and thus so would the Kronecker product, whch s not shown. 5 Dscusson The densty of a matrx varate skew-t dstrbuton was derved. Ths dstrbuton can be consdered as a three-way extenson of the multvarate skew-t dstrbuton. Parameter estmaton was carred out usng an ECM algorthm. Because the formulaton of multvarate skew-t dstrbuton ths work s based on s a specal case of the generalzed hyperbolc dstrbuton, t s reasonable to postulate an extenson to a broader class of matrx varate dstrbutons. Ongong work consders a fnte mxture of matrx varate skew-t dstrbutons for clusterng and classfcaton of three-way data. 8,.

9 a b V V c d.5 V3. V Fgure : Typcal Margnals for Smulaton for (a) V, (b) V, (c) V3 and (d) V4. The red dashed lnes denote the means. Table : Component wse averages and standard devatons for the estmated parameters for smulatons and. Smulaton M(sd) A(sd) Σ(sd) Ψ(sd) ν(sd) (.63) 4. (.9) Acknowledgements The authors gratefully acknowledge the fnancal support provded by the Vaner Canada Graduate Scholarshps (Gallaugher) and the Canada Research Chars program (McNcholas). 9

10 5 a b 5 5 V V c d 4 5 V3 V Fgure : Typcal Margnals for Smulaton for (a) V, (b) V, (c) V3 and (d) V4. The red dashed lnes denote the means. References Atken, A. C. (96). A seres formula for the roots of algebrac and transcendental equatons. Proceedngs of the Royal Socety of Ednburgh 45, 4. Anderlucc, L., C. Vrol, et al. (5). Covarance pattern mxture models for the analyss of multvarate heterogeneous longtudnal data. The Annals of Appled Statstcs 9 (), Böhnng, D., E. Detz, R. Schaub, P. Schlattmann, and B. Lndsay (994). The dstrbuton of the lkelhood rato for mxtures of denstes from the one-parameter exponental famly. Annals of the Insttute of Statstcal Mathematcs 46, Chen, J. T. and A. K. Gupta (5). Matrx varate skew normal dstrbutons. Statstcs 39 (3), Doğru, F. Z., Y. M. Bulut, and O. Arslan (6). Fnte mxtures of matrx varate t dstrbutons. Gaz Unversty Journal of Scence 9 (),

11 Domínguez-Molna, J. A., G. González-Farías, R. Ramos-Quroga, and A. K. Gupta (7). A matrx varate closed skew-normal dstrbuton wth applcatons to stochastc fronter analyss. Communcatons n Statstcs Theory and Methods 36 (9), Dutlleul, P. (999). The MLE algorthm for the matrx normal dstrbuton. Journal of Statstcal Computaton and Smulaton 64 (), 5 3. Gupta, A. K. and D. K. Nagar (999). Matrx Varate Dstrbutons. Boca Raton: Chapman & Hall/CRC Press. Harrar, S. W. and A. K. Gupta (8). On matrx varate skew-normal dstrbutons. Statstcs 4 (), Lndsay, B. G. (995). Mxture models: Theory, geometry and applcatons. In NSF-CBMS Regonal Conference Seres n Probablty and Statstcs, Volume 5. Calforna: Insttute of Mathematcal Statstcs: Hayward. McNcholas, P. D. (6). Mxture Model-Based Classfcaton. Boca Raton: Chapman & Hall/CRC Press. McNcholas, P. D., T. B. Murphy, A. F. McDad, and D. Frost (). Seral and parallel mplementatons of model-based clusterng va parsmonous Gaussan mxture models. Computatonal Statstcs and Data Analyss 54 (3), Meng,.-L. and D. B. Rubn (993). Maxmum lkelhood estmaton va the ECM algorthm: a general framework. Bometrka 8, Murray, P. M., R. B. Browne, and P. D. McNcholas (4a). Mxtures of skew-t factor analyzers. Computatonal Statstcs and Data Analyss 77, Murray, P. M., P. D. McNcholas, and R. B. Browne (4b). A mxture of common skew-t factor analyzers. Stat 3 (), Wshart, J. (98). The generalsed product moment dstrbuton n samples from a normal multvarate populaton. Bometrka, 3 5.