Orthogonal decompositions in growth curve models

Transcription

1 ACTA ET COMMENTATIONES UNIVERSITATIS TARTUENSIS DE MATHEMATICA Volume 4, Orthogonal decompositions in growth curve models Daniel Klein and Ivan Žežula Dedicated to Professor L. Kubáček on the occasion of his eightieth birthday Abstract. The article shows advantage of orthogonal decompositions in the standard and extended growth curve models. Using this, distribution of estimators of ρ and σ in the standard GCM with uniform correlation structure is derived. Also, equivalence of Hu and von Rosen conditions in the extended GCM under mild conditions is shown.. The standard growth curve model with uniform correlation structure The basic model that we consider is the following one: Y = XBZ + e, Vece) N,Σ I n ), Σ = θ G + θ ww, ) where Y n p is a matrix of independent p-variate observations, X n m is an ANOVA design matrix, Z p r is a regression variables matrix, and e is a matrix of random errors. As for the unknown parameters, B m r is an location parameters matrix, and θ,θ are scalar) variance parameters. Matrix G p p > and vector w R p are known. The Vec operator stacks elements of a matrix into a vector column-wise. Assumed correlation structure is called generalized uniform correlation structure. It was studied in the context of the growth curve model GCM) in [6], and recently in [4]. A special case was studied also in [3]. As for the estimation of unknown parameters, Žežula in [6] used directly model ), whereas Ye and Wang [4] used modified model with orthogonal Received March 5,. Mathematics Subject Classification. Primary 6H; Secondary 6J5. Key words and phrases. Growth curve model, extended growth curve model, orthogonal decomposition, uniform correlation structure. The research was supported by grants VEGA MŠ SR /3/9, VEGA MŠ SR /35/, and by the Agency of the Slovak Ministry of Education for the Structural Funds of the EU, under project ITMS:67. 35

2 36 DANIEL KLEIN AND IVAN ŽEŽULA decomposition: Y G = Y + Y, Y = Y G P F = XBZ G P F + e, ) Y = Y G MF = XBZ G MF + e, where F = G w, P F is the orthogonal projection matrix onto the column space RF) of F, and M F = I P F onto its orthogonal complement. Let us denote S = n rx) Y M X Y, W = P F G SG P F, W = M F G SG M F. The estimators of Žežula are ˆθ = w) TrS) Sw w w) TrG) Gw w, ˆθ = STrG) GTrS) w) TrG) Gw w, 3) and the estimators of Ye and Wang are ˆθ = TrW ) p = w G w.tr G S ) w G SG w p )w G, w) ˆθ = p )TrW ) TrW ) p )w G = p.w G SG w w G w.tr G S ) w p )w G w). 4) These pairs of estimators are both unbiased, but different. Naturally, we would like to know the variances. Since S W p n rx), n rx) Σ ), it is easy to establish that VarVec S) = ) Ip + K pp Σ Σ), 5) n rx) where K pp is the commutation matrix, see e.g. [5]. This immediately implies Var [ Tr G S )] = n rx) Tr G ΣG Σ ), Var [ w G SG w ] = w G ΣG w ), n rx) Cov [ Tr G S ),w G SG w ] = n rx) w G ΣG ΣG w.

3 ORTHOGONAL DECOMPOSITIONS IN GROWTH CURVE MODELS 37 Necessary formulas for Var [TrS)], Var S, and Cov [TrS), S] are special cases. Short computation gives Var ˆθ = n rx) w) 4 Tr Σ ) w) w w. Σ + w w) Σ) [ w) TrG) Gw w], Var ˆθ = n rx) [TrG) Σ] TrG) G Σ + G) TrΣ ) [ w) TrG) Gw w], and Var ˆθ = n rx) [ w p ) w G w) G w ) Tr G ΣG Σ ) + Var ˆθ = + w G ΣG w ) w G w ) w G ΣG ΣG w )], 8) n rx) [ w p ) w G w) 4 G w) Tr G ΣG Σ ) + +p w G ΣG w) pw G w) w G ΣG ΣG w )]. 9) Analytical comparison of these quantities is quite difficult. Few simulations performed suggest that in general Ye and Wang s estimators tend to have smaller variance. Very important special case of the previous model is the model with 6) 7) Σ = σ [ ρ)i p + ρ ]. ) This correlation structure is called the uniform correlation structure or the intraclass correlation structure. It is the case with G = I p and w =, slightly reparametrized. It must hold p ρ. As a special case of ), estimators of σ and ρ can be then obtained by a simple transformation of ˆθ and ˆθ : ˆσ = ˆθ + ˆθ and ˆρ = ˆθ ˆθ + ˆθ. ) This implies the following form of estimators due to Žežula: ˆσ Z = TrS), ˆρ Z = ) S p p TrS), ) and due to Ye and Wang: ˆσ Y W = Tr V ) + Tr V ) p, ˆρ Y W = p Tr V ) p )Tr V ) + Tr V )), 3)

4 38 DANIEL KLEIN AND IVAN ŽEŽULA where V = P SP, V = M SM. Ye and Wang recognized that ˆσ Z = ˆσ Y W, but they failed to recognize that also ˆρ Z = ˆρ Y W. and Lemma. ˆσ Z = ˆσ Y W and ˆρ Z = ˆρ Y W for any Y. Proof. Trivially, Tr V ) = Tr P SP ) = Tr SP ) = p TrS ) = p S, Tr V ) = Tr M SM ) = Tr SM ) = TrS) Tr SP ) = TrS) p S. Substituting these values into 3), we easily get ). Thus, in the following we can write only ˆσ and ˆρ. This orthogonal decomposition is very useful for derivation of the distribution of the estimators. Lemma. Let H W p l,ξ), Ξ >, and let T k p be an arbitrary matrix. Then, rt) TrTHT ) λ i χ l, where λ,...,λ rt) are all positive eigenvalues of TΞT and rt) is the rank of T. In particular, rt) E TrTHT ) = l λ i, i= i= rt) Var TrTHT ) = l λ i. Proof. There must exist independent r.v. X,...,X l distributed as N p,ξ) such that H = l i= X ix i = G G, where G = X,...,X l ). Then, TX i N k,tξt ) i which may be singular). According to the Theorem in [] it holds TrTHT ) = Tr GT ) GT ) ) rt) λ i χ l, where λ,...,λ rt) are all positive eigenvalues of TΞT. Since χ -statistics are independent, the claims about mean and variance are trivial. The results concerning distributions of Tr V ) and Tr V ) can be found in [4], but without proof. We extend these results to the parameters of interest. i= i=

5 ORTHOGONAL DECOMPOSITIONS IN GROWTH CURVE MODELS 39 Theorem 3. Distributions of Tr V ) and Tr V ) are independent, so that ˆσ σ [ pn rx)) ρ + p )ρ TrV ) σ [ + p )ρ] n rx) χ n rx), Tr V ) σ ρ) n rx) χ p )n rx)), + p )ρ)χ n rx) + ρ)χ p )n rx)) [ ] + p )ˆρ F ˆρ n rx),p )n rx)). Proof. It is well known that under normality ) S W p n rx), n rx) Σ. see e.g. Theorem 3.8 in [5]). We want to make use of Lemma with l = n rx), Ξ = n rx) Σ, T = P, and also with T = M. Since ) P n rx) Σ P = σ [ + p )ρ] P n rx) is a multiple of idempotent matrix of rank, its only positive eigenvalue is equal to σ [ + p )ρ]/n rx)). Similarly, since M is idempotent with rank p and ) M n rx) Σ M = σ ρ) n rx) M, it has p positive eigenvalues which are all equal to σ ρ)/n rx)). Now the results for Tr V ) and Tr V ) follow from Lemma, perpendicularity of M and P, and properties of χ -distribution. This, together with 3), immediately implies the result for ˆσ. The second formula in 3) can be transformed to + p )ˆρ ˆρ = p )Tr V ) Tr V ) Because the distributions of Tr V ) and Tr V ) are independent, clearly σ ρ)n rx)) σ [ + p )ρ]n rx)) p )Tr V ) F TrV ) n rx),p )n rx)).. ],

6 4 DANIEL KLEIN AND IVAN ŽEŽULA This result is not very useful with respect to ˆσ, since its distribution depends on both σ and ρ, but enables us to test for any specific value of ρ. Using simple transformation, we can even derive directly the probability density function of ˆρ: ) fx) = ρ p )[ + p )ρ] + ρ p )[ + p )ρ] + p )x x ) n rx) Γ p n rx)) ) Γ n rx) Γ ) n rx) + p )x x p x). p )n rx)) ) pn rx)) Also, α confidence interval for ρ is given by ) c c ;, 4) + p )c + p )c where and c = c = ˆρ + p )ˆρ F n rx),p )n rx)) α ) ˆρ α ) + p )ˆρ F n rx),p )n rx)). Figures 4 below show histograms and theoretical densities of ˆρ for a special case of the model quadratic growth in three groups, 5 simulations) for various true values of unknown parameter. Example 4. Let us consider random sample from bivariate normal distribution with the same variances in both dimensions. It can be formally written as GCM with the uniform correlation structure: Y Y Y =.. Y n Y n = n µ,µ )I +e, e N n n,σ ρ ρ Using the above mentioned estimator we get ˆρ = s s +, s ) ) ) I n. where s is sample covariance of the two variables, and s and s are sample variances. This estimator is slightly more effective than the standard sample correlation coefficient in the sense of MSE).

7 ORTHOGONAL DECOMPOSITIONS IN GROWTH CURVE MODELS ,5 -, -,5 -, -,5 -,,,4,6,8 x x Figure. ρ =, Figure. ρ = ,,,4,6,8,7,75,8,85,9,95 x x Figure 3. ρ =,5 Figure 4. ρ =, 95. The extended growth curve model The extended growth curve model ECGM) with fixed effects, called also the sum-of-profiles model, is Y = X i B i Z i + e, e N n p,σ I n ). 5) i= The dimensions of matrices X i, B i, and Z i are n m i, m i r i, and p r i, respectively. Usually it is supposed that column spaces of X i s are ordered, RX k ) RX ), 6) while nothing is said about different matrices Z i. Recently, Hu see []) came up with modification of the model, assuming X ix j = i j. 7) His idea is to separate groups rather then models. We will show that the two models are under certain conditions equivalent. Example 5. Let us consider EGCM with two groups with different growth patterns linear and quadratic: { β + β t j + e ij, i =,...,n, j =,...,p, Y ij = β 3 + β 4 t j + β 5 t j + e ij, i = n +,...,n + n, j =,...,p.

8 4 DANIEL KLEIN AND IVAN ŽEŽULA This model can be written as Y = ) ) ) n β β... + n β 3 β 4 t... t p )β 5 t... t ) p + e, n or, by the new way, as ) ) )... n... Y = β,β ) + β t... t p 3,β 4,β 5 ) t... t p + e. n t... t p Note that in the second form in the previous example RZ ) RZ ). This leads to the idea that we can consider a model in which the column spaces of all matrices Z i are nested, which naturally arises in situations when different groups use polynomial regression functions of different order. Let us consider model 5) with condition 6), such that n p r X ). 8) Since X i are - matrices whose columns are indicators of different groups, without loss of generality we can assume that all columns of X are mutually perpendicular, and columns of every X i+ are a subset of columns of X i. Let us define Xk = X k and Xi = X i X i+, i =,...,k, where the symbol X i X i+ denotes the matrix consisting of those columns of X i which are not in X i+. It is easy to see, that X i = Xi,...,X k ) and P X i P Xi+ = P X i+ X i = P X i. Then, we can reformulate the model 5) with von Rosen s condition 6) in the following way: EY = = X i B i Z i = i= j= i= Xi,...,Xk ) i= j Xj BijZ i = j= X j B ii. B ik Z i = Z B j,...,bjj). Z j i= j=i df = Xj BijZ i = Xj Bj Zj j= 9) matrices X j have dimensions n m j and B ij m j r i, where m i = k j=i m j ). It is now easy to see that model 9) satisfies Hu s condition: Moreover, now we have X i X j = i j. ) Z i = Z,...,Z i ), which implies RZ ) RZ k ). i =,...,k,

9 ORTHOGONAL DECOMPOSITIONS IN GROWTH CURVE MODELS 43 ECGM with Hu s condition is much easier to handle. If all matrices X i and Z i are of full rank, then all B i are estimable, and unbiased LSE ˆB i depend only on X i and Z i : ˆB i = Xi Xi ) X i Y Σ Zi Z i Σ Zi ), ) see []. Such a closed form was difficult to obtain in the von Rosen model. Even for two components the estimators are rather complicated: ˆB = X X ) X Y Σ Z Z Σ Z ) X X ) X P X Y P Σ M Σ Z Z ˆB = X X ) X Y Σ Z Z Σ M Σ Z Z ), ) Σ Z Z Σ Z ), see [7]. Each ˆB and ˆB depends on both Z and Z, and ˆB even on X. Estimator of common variance matrix can be split into perpendicular pieces: ˆΣ = n rx ) Y M X Y = n k i= rx i )Y I P X i )Y. ) In the last expression, the left-hand side term is the estimator using von Rosen s model and right-hand side one using Hu s model. It is easy to see, that the estimators are equivalent, since X = X,...,X k ). The situation in Hu s model is much easier also for a special correlation structure Σ = σ R with R known. The unbiased estimator of residual variance σ is i= ˆσ = n k i= rx i)tr P R Z i ] ) Tr [Y Ŷ ) Y Ŷ ), R where Ŷ = ) k i= P X i Y PZ R i is the unbiased estimator of EY. For comparison, in the von Rosen model the unbiased estimator of residual variance is ˆσ = [ ] m Tr Y Ŷ ) Y Ŷ ),

10 44 DANIEL KLEIN AND IVAN ŽEŽULA where k ) m = n rx )) TrR) + rx i ) rx i+ )) Tr MZ R,...,Z i ) R + i= ) + r X k )Tr MZ R,...,Z k ) R = k ) ) = n rx i ) rx i+ ))Tr PZ R,...,Z i ) R r X k )Tr PZ R,...,Z k ) R, see [7]. i= 3. Conclusion The method of orthogonal decomposition is very promising in complex models. Many tasks, which are very difficult or impossible to handle in basic models, can be done with ease in models consisting of mutually orthogonal components. As it is shown above, simple transformation can change a model into an equivalent which allows to determine explicit forms of estimators and/or their distribution. We hope the method will prove even more useful in the future, either in the models investigated here or in some others. References [] Glueck D.H. and Muller K.E. 998), On the trace of a Wishart, Comm. Statist. Theory Methods 79), [] Hu J. 9), Properties of the explicit estimators in the extended growth curve model, Statistics 445), [3] Klein D. and Žežula I. 7), On uniform correlation structure; In: Mathematical Methods In Economics And Industry, Herľany, Slovakia, pp. 94. [4] Ye R.D. and Wang S.G. 9), Estimating parameters in extended growth curve models with special covariance structures, J. Statist. Plann. Inference 39, [5] Žežula I. 993), Covariance components estimation in the growth curve model, Statistics 4, [6] Žežula I. 6), Special variance structures in the growth curve model, J. Multivariate Anal. 97, [7] Žežula I. 8), Remarks on unbiased estimation of the sum-of-profiles model parameters, Tatra Mt. Math. Publ. 39, Institute of Mathematics, P. J. Šafárik University, Jesenná 5, SK-454 Košice, Slovakia address: daniel.klein@upjs.sk address: ivan.zezula@upjs.sk