Linear Models for Multivariate Repeated Measures Data

Transcription

1 THE UNIVERSITY OF TEXAS AT SAN ANTONIO, COLLEGE OF BUSINESS Working Paper SERIES Date December 3, 200 WP # 007MSS Linear Models for Multivariate Repeated Measures Data Anuradha Roy Management Science and Statistics University of Texas at San Antonio San Antonio, Texas United States Copyright 200, by the author(s Please do not quote, cite, or reproduce without permission from the author(s ONE UTSA CIRCLE SAN ANTONIO, TEXAS BUSINESSUTSAEDU

2 Linear Models for Multivariate Repeated Measures Data Anuradha Roy Department of Management Science and Statistics The University of Texas at San Antonio San Antonio, Texas 78249, USA Abstract We study the general linear model (GLM with doubly exchangeable distributed error for m observed random variables The doubly exchangeable linear model (DEGLM arises when the m dimensional error vectors are doubly exchangeable (defined later, jointly normally distributed, which is much weaker assumption than the independent and identically distributed error vectors as in the case of GLM or classical GLM (CGLM We estimate the parameters in the model and also find their distributions Key Words: Multivariate repeated measures; Linear model; Replicated observations JEL Classification: C0, C3 Introduction A generalization of the general linear model (GLM or the classical general linear model (CGLM is considered by Arnold in 979 when the m error vectors are unobserved and exchangeable, jointly normally distributed; not independent and identically distributed (iid as in the case of CGLM He named his new model as exchangeable linear model (EGLM EGLM is especially appropriate for doubly multivariate data or two-level multivariate data The variance-covariance matrix Σ (partitioned with exchangeable distributed error is of the form U 0 U U U U 0 U Σ U U U 0 I u (U 0 U + J u U, where I u is the u u identity matrix, u is a u vector containing all elements as unity, J u u u, U 0 is a m m positive definite symmetric matrix, and U is a symmetric m m matrix Leiva (2007 also used this variance-covariance matrix Σ for classification problems, and named this as equicorrelated partitioned

3 matrix with equicorrelation parameters U 0, U The m m block diagonals U 0 represent the variancecovariance matrix of the m response variables at any given site, whereas the m m block off diagonals U represent the covariance matrix of the m response variables between any pair of sites We assume U 0 is constant for all sites Also, U is constant between any pair of sites In this article we extend Arnold s (979 generalization of the EGLM when m error vectors are unobserved and doubly exchangeable (defined in Section 2 Doubly exchangeable data is common in repeated measures designs in biomedical, medical, engineering, and in many other research areas In repeated measures designs, in particular those employed in the clinical trial study of skin care products, the data are collected on a vector of measurements (m at different body positions (u and at different points (v in time For example, consider a clinical trial study where measurements are taken on the characteristics of wrinkling, pigmentation, inflammation, and hydration on hands, face, neck, and arms once in every month for four consecutive months Occasionally, biomedical researchers measure levels of fat byproducts at different parts of the body (sites in an eight-week clinical trial for their research In other words, these data are multivariate in three levels In these examples the variables at different sites and at different time points are not independent, but are stochastically dependent in nature Different sites and different time points may be interchangeable or exchangeable (equicorrelated among themselves; in other words it is reasonable to assume that the variables have doubly exchangeable structure Doubly exchangeable linear model (DEGLM is suitable for data that have doubly exchangeable structure In this article we develop DEGLM for three-level multivariate data by using doubly exchangeable structure or jointly equicorrelated covariance structure (Leiva, 2007; Roy and Leiva, 2007 Jointly equicorrelated covariance structure (defined in Section 2 assumes a block circulant covariance structure, consisting of three unstructured covariance matrices for three multivariate levels This jointly equicorrelated covariance structure can capture double exchangeability in the data structure in a longitudinal study both in time and space Another advantage of this covariance structure is that the measurements over time need not be of equally spaced Let y be the m-variate vector of all measurements We partition this vector y as follows: y y y v, where y t y t y tu, with y ts for s,, u, t,, v The m-dimensional vector of measurements y ts represents the replicate on the s th location and at the t th time point y ts, y ts,m, 2

4 2 Basic results 2 Jointly equicorrelated vectors Definition Let y be an m variate partitioned real-valued random vector y (y,, y v, where y t (y t,, y tu for t,, v, and y ts (y ts,,, y ts,m for s,, u Let E [y] µ y R m, and Γ y be the (m m dimensional partitioned covariance matrix Cov [y] ( Γ yt,y t (Γtt, where Γ tt Cov [y t, y t ] for t, t,, v The m variate vectors y,, y u,, y v,, y vu are said to be jointly equicorrelated if Γ y is given by Γ y I vu (U 0 U +I v J u (U W +J vu W, ( where U 0 is a positive definite symmetric m m matrix, and U and W are symmetric m m matrices The variance covariance matrix Γ y is then said to have a jointly equicorrelated covariance structure with equicorrelation parameters U 0, U and W The matrices U 0, U and W are all unstructured Thus, the vectors y,, y u,, y v,, y vu jointly equicorrelated covariance matrix are jointly equicorrelated if they have the following Γ y U 0 U U W W W W W W U U 0 U W W W W W W U U U 0 W W W W W W W W W U 0 U U W W W W W W U U 0 U W W W W W W U U U 0 W W W W W W W W W U 0 U U W W W W W W U U 0 U W W W W W W U U U 0, (2 that is, U 0 if t t and s s, Cov [y ts ; y t s ] U if t t and s s, W if t t, The m m block diagonals U 0 in (2 represent the variance-covariance matrix of the m response variables at any given site and at any given time point, whereas the m m block off diagonals U in (2 represent the covariance matrix of the m response variables between any two sites and at any given time point We 3

5 assume U 0 is constant for all sites and time points, and U is the same for all site pairs and for all time points The m m block off diagonals W represent the covariance matrix of the m response variables between any two time points It is assumed to be the same for any pair of time points, irrespective of the same site or between any two sites 22 Matrix-variate normal distribution The random matrix X(p n is said to have a matrix-variate normal distribution with mean matrix M(p n and covariance matrix Σ Ψ, where Σ > 0, and Ψ > 0 are p p and n n matrices respectively if and only if Vec(X N pn (Vec(M, Σ Ψ We will use the notation X N p,n (M, Σ Ψ or X N p,n (M, Σ, Ψ Note that, if n (thus, Ψ is a scalar, then X follows a p variate normal distribution with mean vector M and variance-covariance matrix ΨΣ The matrix variate normal distribution arises when sampling from multivariate normal population Let x, x 2,, x N be a random sample of size N from N p (µ, Σ Define the observation matrix as follows: x x 2 x N x 2 x 22 x 2N X x p x p2 x pn then X N N,p ( N µ, I N Σ We will use the following results of matrix-variate normal distribution (Pan and Fang, (2002; Gupta and Nagar, (2002 in this article Result : X N p,n (M, Σ, Ψ, Then X N n,p (M, Ψ, Σ Result 2 : X N p,n (M, Σ, Ψ, and that D(m p is of rank m p, and C(n t is of rank t n, and A(m t, then DXC + A N m,t (DMC + A, DΣD, C ΨC Result 3 : If X N p,n (M, Σ, Ψ and X 2 N p,n (M 2, Σ 2, Ψ 2, then ( X + X 2 N p,n M + M 2, (Σ Ψ + (Σ 2 Ψ 2 23 Matrix results The jointly equicorrelated variance-covariance matrix Γ y (Roy and Leiva, 2007 in ( can be written as Γ y I v (V 0 V + J v V, with V 0 I u (U 0 U + J u U, 4

6 and V J u W, where V 0 is a positive definite (mu mu dimensional symmetric matrix, V is a (mu mu dimensional symmetric matrix, and the m m matrices U 0, U and W are defined as in Section 2 3 The Model We study the doubly exchangeable general linear model (DEGLM for three-level multivariate data by considering an m u(v dimensional random matrix Y What do I mean by this notation? The matrix has m rows, u columns and v depths In other words, this notation means m u dimensional matrices are stacked one after another v times We now write the model as or Y α m u(v + m u(v γ T + e, m r r u(v m u(v Y u(v m u(v α m + T γ u(v r r m + e u(v m, (3 where Y is a m u(v dimensional random matrix α is an m dimensional vector γ is a (r m matrix T is an (r u(v dimensional matrix such that the design matrix X [, T ] has rank r We assume > r The error matrix e is such that the m dimensional components of Vec(e, e,, e u,, e v,, e vu are doubly exchangeable, ie, E(e ts 0, for s,, u, t,, v, and U 0 if t t and s s, Cov [e ts ; e t s ] U if t t and s s, W if t t, Arnold (979 showed that the usual methods for making inferences about α in the CGLM are not valid for the EGLM He also mentioned that it was difficult to extract much information about U 0 from the data Thus, there is no sensible way to test hypotheses about α in the EGLM Our model (3 is an improvement over Arnold s model as with W 0 and min(u, v m + r one can test α in the EGLM To compute the model parameters and their distributions we first need to prove some lemmas Lemma Let Γ C I mu and Γ I v (C I m where C and C are orthogonal matrices whose v v u u first columns are proportional to s Let Γ y be a jointly equicorrelated covariance matrix as in equation (2 5

7 of Def, then Γ Γ(Γ y Γ Γ is a diagonal matrix as follows: I u Γ Γ(Γ y [Γ Γ I u I u, where U 0 U, 2 U 0 + (u U uw (U 0 U + u (U W, and 3 U 0 + (u U + u (v W (U 0 U + u (U W + W Proof: It can be easily shown that Γ and Γ are orthogonal We see that The determinant of Γ y is given by Γ(Γ y Γ ( C I mu(i v (V 0 V + J v V ( C I mu, v v v v v I v (V 0 V + V, [ ] V o + (v V 0 (4 0 I v (V o V Γ(Γ y Γ Γ y V o + (v V V o V v Therefore, the matrix Γ y is non-singular, if both V o + (v V and V o V are non-singular matrices Now, from (4, we have [ (C I m(v o + (v V (C I ] m 0 Γ Γ(Γ y Γ Γ u u u u 0 I v (C I m(v o V (C I (5 m u u u u Now, (C I m(v 0 + (v V (C I m (C I m ( I u (U 0 U + J u [(U W + vw] (C I m u u u u u u u u [ ] (U o U + u(u W + W 0 0 I u (U o U [ ] I u 6

8 Similarly, (C I m(v 0 V (C I m (C I m ( I u (U 0 U + J u (U W (C I m u u u u u u u u [ ] (U o U + u(u W 0 0 I u (U o U [ ] I u Therefore, from (5 we have Γ Γ(Γ y Γ Γ I u I u I u It follows that if, 2 and 3 are non-singular then Γ y is non-singular Corollary If W 0, then Thus, we see that 2 3 U 0 U, 2 U 0 + (u U (U 0 U + uu and 3 U 0 + (u U (U 0 U + uu Lemma 2 Let Y is a m u(v dimensional random matrix Then Γ Γ(Vec Γ Γ(Vec Y m u(v Vec(Z : Z 2 m m (u Vec(Z 2 m Vec(Z v m : Z 22 m (u : Z v2 m (u Y m u(v can be expressed as 7

9 Proof: Γ Γ(Vec Y m u(v Γ ( C I u I m V ec( Y v v m u(v Vec( Y C m uu u Vec( Y 2 C m uu u Vec( Y v C m uu u Vec( u Y u : Z 2 m u m (u Vec( u Y 2 u : Z 22 m u m (u Vec( u Y v u : Z v2 m u m (u Vec(Z : Z 2 m m (u Vec(Z 2 m Vec(Z v m : Z 22 m (u : Z v2 m (u Lemma 3 Γ Γ(Vec( γ Γ Γ(Vec( γ Proof: m r Γ Γ(Vec( γ m r T m r r ( T Vec r can be expressed as γ U m r r T r Γ Γ(Vec( γ : γ U 2 m r r u Γ ( C v v I mu(vec( γ : : γ T m r r ( T m r r (I v C I m(vec( γ u u Vec( γ T C m r r uu u Vec( γ T 2 C m r r uu u Vec( γ C ( Vec ( Vec ( Vec ( Vec T v m r r uu u γ U m r r γ U 2 m r r γ U v m r r γ U m r r T m r r u : γ U 2 m r r u U 22 m r r u : γ U v2 m r r u : γ : γ U 2 m r r u U v m r r : γ T 2 m r r u : : γ : γ U v m r r U v2 m r r u : : γ T v m r r u : γ U v2 m r r u 8

10 Lemma 4 Γ Γ(Vec( α can be expressed as m Proof: Γ Γ(Vec( α m Vec ( α m : 0 : : 0 Γ Γ(Vec( α Γ ( Γ(Vec( α m m Γ [ (Vec( α (C I m u ] We will first calculate Vec( α m (C I u Now, (C I u m α v v v (v v (v v v u 0 m α 0 v α u m 0 0 (v (v v I u u u u α m Therefore, ( Vec( α (C I m u Vec v α : 0 : : 0 m u Thus, Γ [ (Vec( α m (I v C u u I mvec (C I u ] ( v α m u : 0 : : 0 Vec ( v α m u C u u : 0 : : 0 9

11 Now, we see that Therefore we have and the lemma is proved C u u [ v 0 0 u ] ( v Vec α α m 0, 0 m u u u C Lemma 5 Proof: (I v C u u (C I u 0 u 0 u 0 u (I v C u u (C I u v v v (I v C u u (v v (v v (I v C 0 u 0 u 0 u u u v + u v 2 + u 2 + (v v u (v (v v u + + v u I u u + (v (v v u (v v 0 u u u u

12 Theorem Let Y represent the three-level multivariate data If m u(v Γ Γ(V ec Y m ( Vec(Z : Z 2 m m (u Vec(Z 2 m Vec(Z v m : Z 22 m (u : Z v2 m (u then all the components Z, Z 2, Z 2,, Z v2 are independently normally distributed such that Z N m, ( α + γ U, 3,, (6a Z i N m, (γ U i, 2, i 2, 3,, v, (6b and Z i2 N m,u (γ U i2,, I u i, 2,, v (6c Proof: Follows from Lemmas and 2 Comments: Thus, we see that CGLMs are nested in one DEGLM The model involving Z has covariance matrix 3, but has only one sample, thus estimation of 3 is not possible Testing of α is therefore not possible This case is similar to Arnold s (979 EGLM, where the intercept α could not be tested Each of the models involving Z i i 2, 3,, v has no intercept and has only one sample too, however, they have common variance-covariance matrix 2 Thus, γ and 2 can be estimated and γ can be tested too Similarly, each of the models involving Z i2, i, 2,, v, are all independent and have a common covariance matrix and have no intercept term Thus, one can calculate estimates of γ and from each of the models, and test any hypothesis about γ Corollary 2 If W 0, then Z N m, ( α + γ U, 2,, Z i N m, (γ U i, 2, i 2, 3,, v, and Z i2 N m,u (γ U i2,, I u i, 2,, v Comments: In this case there are v independent samples to estimate 2 Thus, testing any hypothesis about α is possible in the Arnold s EGLM when one use DEGLM with W 0 and min(u, v m + r Corollary 3 If v, then the DEGLM reduces to the EGLM For v, we have Z N m, ( uα + γ U, U 0 (u U,, and Z 2 N m,u (γ U 2, U 0 U, I u

13 We see that the above model is exactly same as Arnold s (979 EGLM Theorem 2 The estimates of α and γ in the model (3 are given by r m ( j α ( /2( Z U, (7 m U ij U ij U r Proof: The model (3 can be written as U ( Y α + T m m j γ r r m [ T ] [ α X r B r m + e Therefore, the least square estimate of B is given by Now, and γ (X X B r m X Y X X r r X Y r m r m ] + e, [ ] [ T ] T [ T ] T T T Y m T Y r m We will now compute each of the partitioned matrices in X X r r and X r Y have U ij Z ij U Z, (8 r + e m, (9 m r m T r T ( C I u(i v C (I v C ( C I u r v v u u u u v v ( U : U 2 : : U v : U v2 (I v C ( C I u r r u r r u u u v v ( U : U 2 : : U v : U v2 r r u r r u U, r 2 0 u 0 u 0 u Using Lemmas 3 and 5 we

14 and using Lemma 3 we have T T r r T ( C I u(i v C (I v C r v v u u u u ( U : U 2 : : U v : U v2 r r u r r u U i U i + j Now, using Lemmas 2 and 5 we have Again using Lemmas 2 and 3 we get U i2 U i2 U ij U ij ( C I u T v v U r U 2 u r U v r U v2 u r r Y m ( C I u (I v C (I v C ( C I v v u Y u u u u v v m [ ] : 0 u : 0 u : : 0 u (Iv C ( C I u Y u u v v m Z m T Y T ( C I u(i v C (I v C r m r v v u u u u ( U : U 2 : : U v : U v2 r r u r r u U i Z i + j U i2 Z i2 U ij Z ij 3 ( C I uy v v Z m Z 2 u m Z v m Z v2 u m

15 Therefore, (X X B U r 2 j U r v U iju ij and this is equal to X Y, which can be expressed as follows [ Z X Y m r m 2 v r m j U ijz ij Therefore, equating the corresponding terms we get ] α m r m, Now, from (0a we have U α + m r α m + U j U ij U ij r r m r m α ( /2( Z U m j Thus, (7 is proved Now, substituting the value of α in (0b we get, Therefore, U α + m r U ( /2( Z U + r r m j j U Z U U + r r j ( U ij U ij U U r j ( j r m U ij U ij U r j U ij U ij U ij U ij U ij U ij r m U ( r m r m r m j j Z m U ij Z ij j j j U ij Z ij U ij Z ij U ij Z ij U ij Z ij U Z r U ij Z ij U Z r If v, the model reduces to Arnold s (979 EGLM For v we see that α (u /2( Z U, m ( U j U j U U ( r ( U 2 U 2 r r ( U 2 Z 2 r u u m, j U j Z j U Z r (0a (0b 4

16 and these estimates are exactly same as obtained by Arnold (979 Theorem 3 The distributions of α and are as follows: where α N,m (α,, A ( U i Z i + A [ 3 + U A [ (U i (U i ] A U 2 i2 +U A U i2 U i2a U ], ( U i2 Z i2 N (r,m γ, [ (U i (U i ] A 2 + A U i2 U i2a A ( 2 v r r j U iju ij U U r Proof: We will first find the distribution of, and then the distribution of α We note that A A, so A is symmetric Now, from (8 we have r m ( j A ( U ij U ij U r j U ( U ij Z ij U Z r j U ij Z ij U Z r A [U 2 Z U v Z v + U 2 Z 2 + U 22 Z U v2 Z v2] A U i Z i + A U i2 Z i2 i2 To get the distribution of we will find the distributions of A v i2 U iz i and A v U i2z i2 separately Using (6b and (6c we have and, 2 A A U i Z i N r,m (A (U i (U iγ, i2 U i2 Z i2 N r,m (A A i2 [ (U i (U i ] A, 2, i2 U i2 U i2γ, A 5 U i2 U i2a,

17 Therefore, We have A U i Z i N (r,m (A U i U iγ, i2 i2 (A [ (U i (U i ] A 2, i2 and 2 A U i2 Z i2 N (r,m (A U i2 U i2γ, A U i2 U i2a Therefore, A ( (U i Z i + i2 + 2 N (r,m ( A A U i2 Z i2 (U i U iγ + A U i2 U i2γ, i2 [ (U i (U i ] A 2 + A U i2 U i2a i2 Now, A U i U iγ + A U i2 U i2γ A [ U i U i + i2 γ i2 U i2 U i2]γ Therefore, A ( U i Z i + A ( U i2 Z i2 N (r,m γ, [ (U i (U i ] A 2 + A U i2 U i2a We see that is an unbiased estimate of γ We will now find the distribution of α Now from (7 we have α Z m U, α Z U ( + 2, Z U U 2 ( 6

18 Now, from (6a we have and, ( Z N,m ( α + U γ,, 3, ( U N,m U A U A i2 ( U 2 N,m U A U A (U i (U iγ,, i2 [ (U i (U i ] A U 2, U i2 U i2γ,, U i2 U i2a U Now, ( α + U γ U A [ α + U γ A α Therefore, from ( we have (U i U iγ U A i2 U i U iγ A i2 α N,m (α,, i2 ] U i2 U i2γ [ 3 + U A [ (U i (U i ] A U 2 +U A U i2 U i2a U ] Here also we see that α is an unbiased estimate of α Corollary 4 If W 0, the distributions of α and are as follows: α N,m (α,, [ 2 + U A [ (U i (U i ] A U 2 i2 +U A U i2 U i2a U ], U i2 U i2γ 7

19 and A ( ( U i Z i + U i2 Z i2 N (r,m γ, A [ (U i (U i ] A 2 + A U i2 U i2a i2 where A 2 v r r j U iju ij U U Corollary 5 If v, A r r U ij U ij U U j U U + U 2 U 2 U U U 2 U 2, and the distributions of α and are as follows: α N,m (α,, [ 3 + U A [ (U i (U i ] A U 2 i2 ] [((U 0 U + uu + U u (U 2 U 2 U N,m (α,, U + ( + U u (U 2 U 2 U (U 0 U and ( N (r,m γ, A [ (U i (U i ] A 2 + A U i2 U i2a i2 ( N (r,m γ, (U 2 U 2 U 2 U 2(U 2 U 2 N (r,m (γ, (U 2 U 2, (U 0 U These estimates are exactly the same as those of Arnold s (979 Acknowledgements The author would like to acknowledge the generous support for the summer grant from the College of Business at the University of Texas at San Antonio 8

20 References [] Anderson, TW An introduction to multivariate statistical analysis, Third Ed Wiley, New Jersey, 2003 [2] Arnold, SF, Linear Models with Edchangeably Distributed Errors Journal of the American Statistical Association 74 (365 ( [3] Gupta, AK, Nagar, DK Matrix Variate Distributions, Chapman & Hall/CRC, Florida, 2000 [4] Leiva, R, Linear Discrimination with Equicorrelated Training Vectors, J Multivariate Analysis, 98(2 ( [5] Pan, JX, Fang, KT Growth Curva Models and Statistical Diagnostics, Springer-Verlag, New York, 2002 [6] Roy, A, Leiva, R Discrimination with jointly equicorrelated multi-level multivariate data Advances in Data Analysis and Classification, (3, (