Afrik Sttistik Vol. 02), 205, pges 827 837. DOI: http://dx.doi.org/0.6929/s/205.827.77 Afrik Sttistik ISSN 236-090X Biliner regression model with Kronecker nd liner structures for the covrince mtrix Joseph Nzbnit, Dietrich von Rosen 2,3 nd Mrtin Singull 2 Deprtment of Mthemtics, University of Rwnd, PO.Box 3900 Kigli, Rwnd. 2 Deprtment of Mthemtics, Linköping University, SE 58 83 Linköping, Sweden 3 Deprtment of Energy nd Technology, Swedish University of Agriculturl Sciences, SE 750 07 Uppsl, Sweden Received August 4, 205; Accepted December 5, 205 Copyright c 205, Afrik Sttistik. All rights reserved Abstrct. In this pper, the biliner regression model bsed on normlly distributed rndom mtrix is studied. For these models, the dispersion mtrix hs the so clled Kronecker product structure nd they cn be used for exmple to model dt with sptio-temporl reltionships. The im is to estimte the prmeters of the model when, in ddition, one covrince mtrix is ssumed to be linerly structured. On the bsis of n independent observtions from mtrix norml distribution, estimting equtions in flip-flop reltion re estblished nd the consistency of estimtors is studied. Résumé. Nous bordons dns ce ppier du model de regession bilinéire bsé sur une mtrice létoire gussienne. Dns les modèles que nous étudions, les mtrices de dispersion ont l structure du produit de Kronecker si bien qu elles sont cpbles de modéliser les données présentnt des reltions sptio-temporelles. Le but de cette étude est d estimer les prmètres du modèle, lorsqu en plus une mtrice de covrince est supposée être linéirement structurée. Etnt données n observtions normlement distribuées, les équtions d estimtion sont étblies trvers une reltion flip-flop. L consistence des estimteurs est étudiée. Key words: Biliner regression; Estimting equtions; Flip-flop lgorithm; Kronecker product structure; Liner structured covrince mtrix; Mximum likelihood estimtion. AMS 200 Mthemtics Subject Clssifiction : 62G08; 62J05. Corresponding uthor Joseph Nzbnit : j.nzbnit@ur.c.rw, nzbnit@gmil.com Dietrich von Rosen : dietrich.von.rosen@liu.se Mrtin Singull : mrtin.singull@liu.se
Nzbnit et l., Afrik Sttistik, Vol. 02), 205, pges 827 837. Biliner regression model with Kronecker nd liner structures for the covrince mtrix. 828. Introduction Multivrite repeted mesures dt sets, which correspond to multiple mesurements tht re tken over time on more thn one response vrible on ech subject or unit, re common in vrious reserch fields such s medicine, phrmcy, environment, engineering, business, etc. This kind of dt is known to hve covrince mtrix with the so clled Kronecker product structure Ψ Σ, where stnds for the mtrix Kronecker product. The positive definite mtrices Ψ nd Σ re often referred to s the temporl nd sptil covrince mtrix, respectively. The Kronecker product structure my lso occur in ny sptio-temporl process like multivrite time series or stochstic processes. It comes nturlly to bse sttisticl nlyses for this kind of dt on the mtrix norml model. Aprt from the Kronecker product covrince structure, one or both of the mtrices Ψ nd Σ my be structured. In this pper we re interested in the cse where Σ is linerly structured. In this pper we consider n independent nd identiclly distributed observtion mtrices X i N p,q M, Σ, Ψ), i =, 2,..., n. It follows tht the dispersion mtrix of X i hs Kronecker product structure, i.e, D[X i ] = Ψ Σ, where D[X i ] = D[vecX i ] nd vec is the usul vec-opertor. For the interprettion we note tht Ψ describes the covrinces between the columns of X. These covrinces will be the sme for ech row of X. The other covrince mtrix Σ describes the covrinces between the rows of X which will be the sme for ech column of X. The product Ψ Σ tkes into ccount the covrinces between columns s well s the covrinces between rows. Therefore, Ψ Σ indictes tht the overll covrince consists of the products of the covrinces in Ψ nd in Σ, respectively, i.e., Cov[x ij, x kl ] = σ ik ψ jl, where X = x ij ), Σ = σ ik ) nd Ψ = ψ jl ). In Dutilleul 999) the mtrix norml distribution is reviewed nd two-stge lgorithm to find mximum likelihood estimtors is proposed. In ddition we suppose tht the men M of X i hs biliner structure, i.e., E[X i ] = ABC, where A : p r nd C : s q re known design mtrices of regressors. Throughout this pper, without loss of generlity, it is ssumed tht mtrices A nd C re of full rnk, i.e., rnka) = r nd rnkc) = s. Also we ssume tht the covrince mtrices Σ nd Ψ re positive definite. When Ψ = I, the identity mtrix, we get the well known growth curve model s introduced in Pothoff nd Roy 964). The growth curve model hs been extensively studied nd useful references re Khtri 966); Kollo nd von Rosen 2005); Srivstv nd Khtri 979). When Ψ is known, the sitution is lmost similr to the clssicl growth curve model. In this cse, explicit mximum likelihood estimtors MLEs) were derived by Srivstv et l. 2009) nd their uniqueness were proved under full rnk condition of design mtrices. 2. Explicit estimtors of linerly structured Σ with unknown prmeters nd known Ψ We observe tht for mtrices X i N p,q ABC, Σ, Ψ), i =, 2,..., n, we my form new mtrix X = X : X 2 : : X n ), such tht X N p,qn AB n C), Σ, I n Ψ), where n is the n dimensionl vector of ones, nd I n is the n n identity mtrix.
Nzbnit et l., Afrik Sttistik, Vol. 02), 205, pges 827 837. Biliner regression model with Kronecker nd liner structures for the covrince mtrix. 829 When Ψ is known nd positive definite, we trnsform the dt by letting Y i = X i Ψ /2, where Ψ /2 is symmetric positive definite squre root of Ψ. This yields Y = Y : Y 2 : : Y n ) : p qn, nd the model becomes Y N p,qn AB n CΨ /2 ), Σ, I), which is similr to the clssicl growth curve model. Now we ssume tht the mtrix Σ is linerly structured see Kollo nd von Rosen 2005), Definition.3.7). The commonly encountered liner structures for the covrince mtrix re the uniform structure, the compound symmetry structure, the bnded structure, the Toeplitz structure, etc. The linerly structured covrince mtrix will be denoted Σ s). For convenience we define nd denote by vecσk) the column-wise vectoriztion of Σ s) where ll 0 s nd repeted elements by modulus) hve been disregrded. Then there exists, see Kollo nd von Rosen 2005), trnsformtion mtrix T such tht vecσk) = T vecσ s) or vecσ s) = T + vecσk), ) where T + denotes the Moore-Penrose generlized inverse of T. Put G = n CΨ /2 : s qn. Then the problem boils down to find estimtors in the model Y N p,qn ABG, Σ s), I). 2) This problem hs been studied by Ohlson nd von Rosen 200). From results in Ohlson nd von Rosen 200), simple mnipultions give explicit estimtors of prmeters in the model 2). The explicit estimtor of the men structure is A BG = A A s) A A s) Y G GG G, 3) where s) is consistent estimtor of Σ s) obtined from s) vec = qn s T + T + ) T + T + ) vecs, in which s = rnkg) = rnkc) nd S = Y I G GG G)Y. Agin, following Ohlson nd von Rosen 200), nother consistent estimtor of Σ s) is obtined from vec s) = T + T + ) Υ ΥT + T + ) Υ vecs + Ĥ Ĥ ), where Ĥ = I A ) A s) A A s) Y G GG G, Υ = qn s)i + s I A ) A s) A A boldsymbolσ s)
Nzbnit et l., Afrik Sttistik, Vol. 02), 205, pges 827 837. Biliner regression model with Kronecker nd liner structures for the covrince mtrix. 830 I A ) ) A s) A A s). From 3) we obtin n unbised estimtor of the prmeter mtrix B, B = A s) A A s) Y G GG. 3. Estimtors of linerly structured Σ with unknown prmeters nd unknown Ψ In this section we consider the model X N p,qn AB n C), Σ, I n Ψ), 4) in which we ssume tht Σ hs liner structure nd is unknown, nd Ψ is unknown. In the unstructured cse with the only dditionl estimbility condition ψ qq =, the mximum likelihood estimtion equtions were derived in Srivstv et l. 2009). Those re nâbc = AA Ŝ A A Ŝ X n Ψ C C Ψ C C), 5) nd nq = X A B n C))I Ψ )X A B n C)), 6) Ψ = np where Ŝ nd B re given by X i ÂBC) Xi ÂBC), 7) n B = A Ŝ A A Ŝ X n Ψ C C Ψ C ), 8) Ŝ = XI Ψ n n n Ψ C C Ψ C C Ψ )X, 9) nd Ŝ is ssumed to be positive definite. Moreover, it hs been shown in Srivstv et l. 2009) tht solving these equtions, using the flip-flop lgorithm, the estimtes in the lgorithm converge to the unique mximum likelihood estimtors of the prmeters. However, when Σ is linerly structured this pproch will hrdly produce n estimtor with the desired structure. In fct, we would chieve the originl structure if n is lrge enough which is not the cse for rel dt sets. The im of this pper is to build up flip-flop reltion tht will hndle the liner structure of Σ. To find the estimting eqution for the linerly structured Σ, herefter denoted Σ := Σ s), let Ψ in 4) be fixed. Then, we know tht S = XI Ψ n n n Ψ C CΨ C CΨ )X W p Σ s), nq s),
Nzbnit et l., Afrik Sttistik, Vol. 02), 205, pges 827 837. Biliner regression model with Kronecker nd liner structures for the covrince mtrix. 83 nd hence E[S] = nq s)σ s). Here W p, ) denotes the Wishrt distribution. So it is nturl to use S when finding n estimtor of Σ s). We pply lest squres pproch, i.e., we minimize { tr S nq s)σ s)) S nq s) Σ s))} 0) with respect to Σ s). Here, tr stnds for the trce of mtrix. To find the minimizer of 0), we use techniques bsed on differentitions. For more detils on mtrix differentition one cn consult Kollo nd von Rosen 2005). The expression 0) cn be rewritten s { } tr S nq s)σ s) ) ) = vecs nq s)vecσ s) ) ), ) where the nottion Q) ) stnds for Q) Q). We differentite ) with respect to vecσk) nd equlize to 0 to get in which dσs) dσk) 2nq s) dσs) dσk) vecs nq s) Σs) ) = 0, 2) is given see Kollo nd von Rosen, 2005) by dσ s) dσk) = T + ). 3) Combining equtions in ), 2) nd 3) we get the liner eqution T + ) vecs = nq s) T + ) T + vecσk), which is consistent. Its generl solution is given by vecσk) = T + ) T + T + ) vecs + T + ) T +) o z, nq s where z is n rbitrry vector nd the nottion Q o stnds for ny mtrix of full rnk spnning the orthogonl complement of the column spce of Q. Hence, using ) we obtin the unique minimizer of 0) given by vecσ s) = T + vecσk) = nq s T + T + ) T + T + ) vecs. Thus, first estimtor for Σ s) is given by s) vec = nq s T + T + ) T + T + ) vecŝ, 4) where Ψ in S hs been replced with its estimtor Ψ to get Ŝ s in 9). s) Now we suppose tht is positive definite which lwys holds for lrge n) nd use it in 8) insted of Ŝ to find n estimtor of B, given by n B = A s) A A s) X n Ψ C C Ψ C ). 5)
Nzbnit et l., Afrik Sttistik, Vol. 02), 205, pges 827 837. Biliner regression model with Kronecker nd liner structures for the covrince mtrix. 832 To derive the finl estimtor of Σ s), we follow similr ides s in Nzbnit et l. 202) or Ohlson nd von Rosen 200). Let Q = I A A S A A S nd S = n Q X n n Ψ C CΨ C CΨ )X Q. We wnt to use the sum of S nd S to find the finl estimtor of Σ s). We first observe tht X n n Ψ C CΨ C CΨ )X is independent of S nd thus S S W p Q Σ s) Q, s). Agin we pply lest squres pproch nd minimize { } tr S + S [nq s)σ s) + sq Σ s) Q ]) ) with respect to Σ s). This is equivlent to the minimiztion with respect to Σ s) of vecs + S ΥvecΣ s)) vecs + S ΥvecΣ s)), where Υ = nq s)i + sq Q. Using differentition techniques, s bove, unique minimizer is obtined vecσ s) = T + T + ) Υ ΥT + T + ) Υ vecs + S ). Hence, replcing Ψ with its estimtor we get n estimting eqution for Σ s) given by vec s) = T + T + ) Υ ΥT + T + ) Υ vec Ŝ + Ŝ), 6) where Ŝ = n Q X n n Ψ C C Ψ C C Ψ )X Q, 7) Q = I A A s) A A s), 8) nd Υ = nq s)i + s Q Q. 9) The reltion 6) gives us n estimting eqution for Σ s). Thus, modifying 5) using 5), replcing 6) with 6) nd modifying 7) using 5) we obtin the following theorem which is the min result of this pper. Note tht the estimtors to be uniquely determined, the estimbility condition is dded.
Nzbnit et l., Afrik Sttistik, Vol. 02), 205, pges 827 837. Biliner regression model with Kronecker nd liner structures for the covrince mtrix. 833 Theorem. Let X N p,qn AB n C), Σ s), I n Ψ). Assume tht σ s) pp estimtion equtions for the prmeters Σ s) nd Ψ re given by nâbc = A A s) A A =. The s) X n Ψ C C Ψ C C), 20) nd where vec s) = T + T + ) + Υ ΥT T + ) Υ vec Ŝ + Ŝ), 2) Ψ = np X i ÂBC) s) X i ÂBC), 22) s), Ŝ, Ŝ nd Υ re given by 3), 9), 7) nd 9) respectively. These equtions re nested nd cnnot be solved explicitly. Therefore n itertive lgorithm, like the so clled flip-flop lgorithm, is required to get estimtes of prmeters. In ddition, these equtions re modifiction of equtions presented in Srivstv et l. 2009) for n unstructured dispersion mtrix where it hs been shown tht only one solution exists. By construction this still holds true. Few simultions not included in this pper) hve supported tht nd the solution does not depend on the strting vlues. Next, we show the consistency of the proposed estimtors. Theorem 2. The estimtors ÂBC, Ψ nd s), given in Theorem, re consistent estimtors of ABC, Ψ nd Σ s), respectively. In the sequel, for two rndom sequences X n nd Y n, the nottion X n = Yn mens tht p X n Y n 0, n nd the nottion p mens converges in probblity. The following lemm will be utilized in the proof of Theorem 2. Lemm. Let following hold s), s) nd Ψ be given in 4), 2) nd 22), respectively. Then, the Ψ = Ψ p trσs) s) ), 23) s) = Σ s) p trσs) s) ), 24) s) = Σ s) p trσs) s) ). 25) Proof. The Crmér-Slutsky s theorem Crmér 946) will be utilized severl times. From eqution 22), vec Ψ = [X i ÂBC) X i ÂBC) ]vec np s). p Since, ÂBC ABC see the proof lter), this eqution gives vec Ψ = [X i ABC) X i ABC) ]vec np s). 26)
Nzbnit et l., Afrik Sttistik, Vol. 02), 205, pges 827 837. Biliner regression model with Kronecker nd liner structures for the covrince mtrix. 834 As E[X i ABC) X i ABC)] = vecσ s) vec Ψ, the lw of lrge numbers yields n X i ABC) X i ABC) p vecψvec Σ s). Hence, the reltion 26) becomes vec Ψ = vecψ p trσs) s) ), or equivlently Ψ = Ψ p trσs) s) ), which estblishes 23). Since nd S = XI Ψ n n n Ψ C CΨ C CΨ )X 27) using 23) in 9), it follows tht nq s S p Σ s), 28) Ŝ = nq s)σ s) p trσs) s) ). 29) From ) nd use of 29) in 4) yield s) vec = T + T + ) T +) T + ) vecσ s) p trσs) s) ) = T + T + ) T +) T + ) T + vecσk) p trσs) s) ) = T + vecσk) p trσs) s) ) = vecσ s) p trσs) s) ). Hence s) = Σ s) p trσs) s) ). Using 24) in 8), it follows tht Q = I A A Σ s) A A Σ s) =: Q. 30) Note tht S = n Q X n n Ψ C CΨ C CΨ )X Q
Nzbnit et l., Afrik Sttistik, Vol. 02), 205, pges 827 837. Biliner regression model with Kronecker nd liner structures for the covrince mtrix. 835 = Q XI n Ψ )X S) Q n ) = Q X i Ψ X i S Q, where S is given in 27). But, by the lw of lrge numbers n X i Ψ X p i qσ s), which together with 28) imply tht S = Q nqσ s nq s)σ s) ) Q Thus, use 23) nd 30) in 7) to get From 29) nd 3), vecŝ + Ŝ) = Ŝ = s Q Σ s) Q = s Q Σ s) Q. p trσs) s) ). 3) nq s)i + s Q Q ) vecσ s) p trσs) s) ) = ΥvecΣ s) p trσs) s) ), where Υ = nq s)i + s Q Q. Hence, from 2) nd ), vec s) = T + T + ) + Υ ΥT ) ) T + ) s) Υ ΥvecΣ p trσs) s) ) = T + T + ) + Υ ΥT ) ) T + ) Υ ΥT + vecσk) p trσs) s) ) = T + vecσk) p trσs) s) ) = vecσ s) p trσs) s) ), since Υ hs full rnk nd hence which proves 25). s) = Σ s) p trσs) s) = ) s),
Nzbnit et l., Afrik Sttistik, Vol. 02), 205, pges 827 837. Biliner regression model with Kronecker nd liner structures for the covrince mtrix. 836 Proof Proof of Theorem 2). From 20) we hve ÂBC = n A A s) A A s) X n Ψ C C Ψ C C) ) = A A s) A A s) n ) X i Ψ C C Ψ C C = A A s) A A s) ABC Ψ C C Ψ C C since the lw of lrge numbers gives n X i = ABC, p E [X i ] = ABC nd, A nd C re of full column rnk nd full row rnk, respectively. This proves tht ÂBC p ABC. A pre-multipliction of 25) with Σ s) gives Σ s) s) I p trσs) s) ) = 0, which implies tht tr Σ ) s) s) p 2 trσ s) s) ) ) = 0. This is equivlent to p p ˆλ i p 2 ˆλ i = 0, where ˆλ i, i =, 2,..., p re eigenvlues of Σ s) s) nd ˆλi > 0 for ll i with probbility one. So, we must hve ˆλ p i for ll i, which implies tht Σ s) s) = I. 32) Using 32) nd Lemm we get tht 2 is complete. s) p Σ s) nd Ψ p Ψ nd the proof of Theorem In principle the resoning in the proof of Theorem 2 cn be used to prove the consistency of estimtors in Srivstv et l. 2009) for the unstructured Σ wht hs not been done yet) or ny other estimtors bsed on the flip-flop lgorithm. For exmple, eqution 6) gives nq = X A B n C))I Ψ )X A B n C))
Nzbnit et l., Afrik Sttistik, Vol. 02), 205, pges 827 837. Biliner regression model with Kronecker nd liner structures for the covrince mtrix. 837 = = X i ÂBC) Ψ X i ÂBC) Using the lw of lrge numbers nd we hve [X i ABC)Ψ X i ABC) ] p tr Σ ). E [ X i ABC)Ψ X i ABC) ] = qσ = Σ p tr Σ ), which cn be used in similr wy s in the proof of Theorem 2 to prove the consistency of estimtors in Srivstv et l. 2009). Acknowledgements The reserch of Joseph Nzbnit hs been supported through the UR-Sweden Progrm for Reserch, Higher Lerning nd Institution Advncement nd ll persons nd institutions involved re hereby cknowledged. References Dutilleul, P., 999. The MLE lgorithm for the mtrix norml distribution. J. Stt. Comput. Simul., 642), 05-23. Pothoff, R.F. nd Roy, S.N., 964. A generlized multivrite nlysis of vrince model useful especilly for growth curve problems. Biometrik, 5, 33-326. Khtri, C.G., 966. A note on MANOVA model pplied to problems in growth curve, Ann. Inst. Sttist. Mth., 8), 75-86. Kollo, T. nd von Rosen, D. 2005. Advnced Multivrite Sttistics with Mtrices. Mthemtics nd Its Applictions New York), 579. Springer, Dordrecht. Srivstv, M.S. nd Khtri, C.G., 979. An Introduction to Multivrite Sttistics. North- Hollnd, New York-Oxford. Srivstv, M.S., von Rosen, T. nd von Rosen, D., 2009. Estimtion nd testing in generl multivrite liner models with Kronecker product covrince structure. Snkhyā. Ser. A, 72), 37-63. Ohlson, M. nd von Rosen, D., 200. Explicit estimtors of prmeters in the growth curve model with linerly structured covrince mtrices, J. Multivrite Anl., 05), 284-295. Nzbnit, J., Singull, M. nd von Rosen, D., 202. Estimtion of prmeters in the extended growth curve model with linerly structured covrince mtrix, Act Comment. Univ. Trtu. Mth., 6), 3-32. Crmér, H., 946. Mthemticl Methods of Sttistics. Princeton Mthemticl Series, vol. 9. Princeton University Press, Princeton, N. J.