High-Dimensional Asymptotic Behaviors of Differences between the Log-Determinants of Two Wishart Matrices

Transcription

1 TR-No. 4-0, Hiroshima Statistical Research Grou, 9 High-Dimensional Asymtotic Behaviors of Differences between the Log-Determinants of Two Wishart Matrices Hirokazu Yanagihara,2,3, Yusuke Hashiyama and Yasunori Fujikoshi,2 Deartment of Mathematics, Graduate School of Science, Hiroshima University -3- Kagamiyama, Higashi-Hiroshima, Hiroshima , Jaan 2 Hiroshima University Statistical Science Research Core -3- Kagamiyama, Higashi-Hiroshima, Hiroshima , Jaan 3 Tokyo Kantei Co., Ltd., Kami-Osaki, Shinagawa-ku, Tokyo 4-002, Jaan (Last Modified: December 3, 204) Abstract In this aer, we evaluate the asymtotic behaviors of the differences between the logdeterminants of two random matrices distributed according to the Wishart distribution by using a high-dimensional asymtotic framework in which the sizes of the matrices and the degrees of freedoms aroach simultaneously. We consider two structures of random matrices: a matrix is comletely included in another matrix, and a matrix is artially included in another matrix. As an alication of our result, we derive the condition needed to ensure consistency for a given loglikelihood-based information criterion for selecting variables in a canonical correlation analysis. Key words: Canonical correlation analysis, Consistency of information criterion, High-dimensional asymtotic framework, Information criterion, Model selection. Corresonding author address: yanagi@math.sci.hiroshima-u.ac.j (Hirokazu Yanagihara). Introduction Let W and W 2 be symmetric random matrices distributed according to the Wishart distribution (Wishart matrices). In this aer, we study the asymtotic behavior of the difference between the log-determinants of two Wishart matrices, i.e., log W 2 log W = log W 2 W. () The difference between the log-determinants of two Wishart matrices lays a key role in multivariate analysis, because many statistics in multivariate analysis (e.g., the log-likelihood ratio statistic under the normality assumtion) can be exressed as a difference (see, e.g., Muirhead, 982; Siotani et al., 985; Anderson, 2003). Hence, to rove the asymtotic behavior of log W 2 / W is of major interest in multivariate analysis. A common aroach to the study of asymtotic behavior is to use

2 Asymtotic Behaviors of Log-Determinants of Two Wishart Matrices a large samle (LS) asymtotic framework such that the samle size n, which is also the number of degrees of freedom of the Wishart matrix, aroaches. Since high-dimensional data analysis has been attracting the attention of many researchers in recent years, it is imortant to study the asymtotic behavior in terms of the following high-dimensional (HD) asymtotic framework. HD asymtotic framework: n and /n are aroaching and c [0, ), resectively. It should be emhasized that the ordinary LS asymtotic framework is included in the HD asymtotic framework as a secial case. An aim of this aer is to evaluate the asymtotic behavior of log W 2 / W, using the HD asymtotic framework. Let O d, be a d matrix of zeros. We will consider the following structures for Wishart matrices: Case. The W 2 includes W comletely, i.e., W W (n k, I ), W 2 = W + Z Z W (n k + d, I ), (2) where k and d are ositive integers indeendent of n and, and W and Z are indeendent random matrices defined by Z N d (O d,, I I d ). Case 2. The W 2 includes W 2 artially, i.e., W = U U W (n k, I ), W 2 = (ZΓ + U) (ZΓ + U) W (n k, I + ΓΓ ), (3) where Γ is a r constant matrix, k and r are ositive integers indeendent of n and, and U and Z are indeendent random matrices defined by U N (n k) (O n k,, I I n k ), Z N (n k) r (O n k,r, I r I n k ). Cases and 2 corresond to the log-likelihood ratio statistics of models with an inclusion relation and without an inclusion relation, resectively. As an alication of our result, we derive the condition for ensuring the consistency of a loglikelihood-based information criterion (LLBIC) for selecting variables in a canonical correlation analysis (CCA) that analyzes the correlation of two linearly combined variables; note that this is an imortant method in multivariate analysis. An otimized solution of can be found by solving an eigenvalue roblem. CCA has been introduced in many textbooks on alied statistical analysis (see, e.g., Srivastava, 2002, cha. 4.7; Timm, 2002, cha. 8.7), and it is widely used in many alied fields (e.g., Doeswijk et al., 20; Khalil et al., 20; and Vahedia, 20). The family of LLBICs includes many famous information criteria, e.g., Akaike s information criterion (AIC), the bias-corrected AIC (AIC c ), Takeuchi s information criterion (TIC), the Bayesian information criterion (BIC), the consistent AIC (CAIC), and the Hannan and Quinn information criterion (HQC). Under a general model, the AIC, AIC c, TIC, BIC, CAIC, and HQC were roosed by Akaike (973; 2

3 Yanagihara, H., Hashiyama, Y. & Fujikoshi, Y. 974), Hurvich and Tsai (989), Takeuchi (976), Schwarz (978), Bozdogan (987), and Hannan and Quinn (979), resectively. The AIC and AIC c for selecting variables in CCA were roosed by Fujikoshi (985), and the TIC for selecting variables in CCA was roosed by Hashiyama et al. (204). By using the AIC for CCA and the definitions of the original information criteria, we formulate the BIC, CAIC, and HQC for selecting variables in CCA. In this aer, if the asymtotic robability that an information criterion selects the true model aroaches, then we say that the information criterion is consistent. Under the HD asymtotic framework, Yanagihara et al. (202) and Fujikoshi et al. (204) studied the consistency of an information criterion in a multivariate linear regression model. For CCA, there are no results in the literature on the consistency of an information criterion under the HD asymtotic framework, although several authors (e.g., Nishii et al., 988) have studied consistency under the LS asymtotic framework. This aer is organized as follows: In Section 2, we resent our main results. In Section 3, we show a condition for ensuring the consistency of the LLBIC for CCA. Technical details are rovided in the Aendix. 2. Main Results In this section, we evaluate the asymtotic behavior of log W 2 / W in () within the HD asymtotic framework. We begin by considering case, and we have the following theorem (the roof is given in Aendix A): Theorem Suose that W and W 2 are Wishart matrices given by (2). Then, we have log W 2 ( = d log( /n) + ) tr(v ) + O (n 2 ), (4) W n n where V is a d d random matrix with the order O (), which is given by V = n ( ZW Z n k I d and tr(v ) is asymtotically distributed as tr(v ) d (χ 2 d d)/ ( is bounded) N(0, 2d/( c) 3 ) ( ) ), (5). (6) Wakaki (2006) derived a result similar to Theorem by using a roerty of Wilks lambda distribution, whereas we used a roerty of the Wishart distribution to rove it. and Notice that n ( ) n n ( log ) = + O(n ), n (X d) = n (X d) + O ( 3/2 n 2 ), where X is a random variable distributed according to the chi-square distribution with d degrees of freedom. Hence, from Theorem, the following corollary is obtained: 3

4 Corollary and where R is defined by Asymtotic Behaviors of Log-Determinants of Two Wishart Matrices Suose that W and W 2 are Wishart matrices given by (2). Then, we have R = log W 2 W = d log( c) + o (), (7) n log W 2 W = R + o (), X/, X χ 2 d ( is bounded) dg(c) ( ) Here, g(x) is a function with domain x [0, ), which is given by (x = 0) g(x) = x log( x) (x (0, )) From elementary calculus calculations,.. (8) it turns out that g(0) is defined to be equal to lim x 0+ { x log( x)} and that g(x) is a strictly monotonically increasing function in x [0, ). In this aer, we have derived Corollary through Theorem. Although it is necessary to clarify the asymtotic distribution of tr(v ) in order to rove Theorem, (7) can be derived without the asymtotic distribution of tr(v ), using only the exectation of tr(v ). Next, we consider case 2, and we have the following theorem (the roof is given in Aendix B): Theorem 2 Suose that W and W 2 are Wishart matrices given by (3). Then, we have log W 2 W = log I + ΓΓ + o (). (9) 3. Alication 3.. Redundancy Model in CCA In this section, we show an examle of an alication of Theorems and 2 to an actual statistical roblem. We derive conditions to searately ensure consistency and inconsistency of the LLBIC for selecting variables in CCA. Let z = (x, y ) = (x,..., x q, y,..., y ) be a (q + )-dimensional random vector distributed according to the (q+ )-variates multivariate normal distribution with the following mean vector and covariance matrix: E[z] = µ = µ x µ y, Cov[z] = Σ = Σ xx Σ xy Σ xy Σ yy Suose that j denotes a subset of ω = {,..., q} containing q j elements, and x j denotes the q j - dimensional vector consisting of x indexed by the elements of j. For examle, if j = {, 2, 4}, then x j consists of the first, second, and fourth elements of x. We will also let j denote the comlement of the set j, i.e., j = j c. Of course, it holds that x ω = x and q ω = q. Without loss of generality, 4.

5 Yanagihara, H., Hashiyama, Y. & Fujikoshi, Y. we divide x into two subvectors x = (x j, x j ), where x j is a (q q j )-dimensional random vector. Exressions of Σ xx and Σ xy corresonding to the division of x are Σ j j Σ j j Σ xx =, Σ xy = Σ jy. Σ j j Σ jy These imly that another exression of Σ corresonding to the division is Σ j j Σ j j Σ jy Σ j j Σ = Σ j j Σ jy Σ j j Σ jy Σ jy Σ yy. (0) Let z,..., z n+ be (n + ) indeendent random vectors from z, and let S be the usual unbiased estimator of Σ given by S = n n+ i= (z i z)(z i z), where z = (n + ) n+ i= z i. Following the same method that we used for Σ in (0), we divide S as S = S S j j S j j S jy xx S xy S xy S = S j j S j j S jy. yy One of the interests in CCA is to determine whether x j is irrelevant. Fujikoshi (985) determined that x j is irrelevant if the following equation holds: In articular, we note that () is equivalent to S jy S jy tr(σ xx Σ xy Σ yy Σ xy) = tr(σ j j Σ jyσ yy Σ jy ). () S yy Σ jy Σ j j Σ j j Σ jy = O q q j,. (2) Consequently, the candidate model in which x j is irrelevant can be exressed as ns W +q (n, Σ) s.t. tr(σ xx Σ xy Σ yy Σ xy) = tr(σ j j Σ jyσ yy Σ jy ). (3) In CCA, the above model is called the redundancy model j or simly the model j. If the model j is selected as the best model, then we can regard x j as irrelevant and x j as relevant. An estimator of Σ in (3) is given by ˆΣ j = arg min Σ where F(S, Σ) is given by { F(S, Σ) s.t. tr(σ xx Σ xy Σ yy Σ xy) = tr(σ j j Σ jyσ yy Σ jy )}, F(S, Σ) = { tr(σ S) log Σ S ( + q) }. It is easy to see that nf(s, Σ) is the Kullback Leibler (KL) discreancy function assessed by the Wishart density. In the analysis of covariance structure, the discreancy function is frequently called the maximum likelihood discreancy function (Jöreskog, 967). Although we do not resent it here, ˆΣ j can be derived in a closed form (see, e.g., Fujikoshi & Kurata, 2008; Fujikoshi et al., 200, cha..5). 5

6 Asymtotic Behaviors of Log-Determinants of Two Wishart Matrices 3.2. A Class of Information Criteria for CCA Let J be a set of candidate models denoted by J = { j,..., j K }, where K is the number of models. We searate J into two sets such that one is a set of oversecified models, and the other is a set of undersecified models. Let J + denote the set of oversecified models, which is defined by J + = { j J tr(σ xx Σ xy Σ yy Σ xy) = tr(σ j j Σ jyσ yy Σ jy )}. Suose that the true model is exressed as the oversecified model having the smallest number of elements, i.e., j = arg min j J + q j. For simlicity, we write q j as q. On the other hand, let J denote the set of undersecified models, which is defined by J = J + J. For the general exression for any j J, Σ yy j and S yy j are defined as Σ yy j = Σ yy Σ jy Σ j j Σ jy, S yy j = S yy S jy S j j S jy. (4) In articular, we write Σ yy ω = Σ yy x and S yy ω = S yy x. From Fujikoshi et al. (200, cha..5) and the fact that tr( ˆΣ j S) = + q, the minimum value of F(Σ, S) under the model j is given by F min ( j) = F(S, ˆΣ j ) = log S yy j S yy x. In the model j, various information criteria can be defined by adding a enalty term for the model comlexity m( j) to nf min ( j), i.e., several information criteria are included in the following class of information criteria: IC m ( j) = nf min ( j) + m( j). (5) By changing m( j), (5) can exress the following secific criteria: 2{q j + (q q + )/2} (AIC) n{b(q) + b(q j + ) b(q j ) (q + )} (AIC c ) 2{q j + (q q + )/2} + ˆκ x + ˆκ ( jy) ˆκ j (TIC) m( j) = {q j + (q q + )/2} log n (BIC) {q j + (q q + )/2}(log n + ) (CAIC) 2{q j + (q q + )/2} log log n (HQC), (6) where b(q) is a function of q defined by nq/(n q ), and ˆκ x, ˆκ j, and ˆκ ( jy) are estimators of multivariate kurtoses of x, x j, and (x j y ), resectively, which are defined by ˆκ x = ˆκ(D x ), ˆκ j = ˆκ(D j ), ˆκ ( j,y) = ˆκ(D ( j,y) ). Here ˆκ(D) is an estimator of multivariate kurtosis of the d-variates extracted from z by D z, which 6

7 Yanagihara, H., Hashiyama, Y. & Fujikoshi, Y. is defined by ˆκ(D) = n+ { (zi z) D(D SD) D (z i z) } 2 d(d + 2), (7) n + i= where D is a (q + ) d matrix whose elements are 0 or, and satisfies D D = I d, e.g., D x = I q O,q, D y = O q, I, D j = I q j O +q q j,q j, D ( j,y) = (D j, D y ). Additionally, we assume that there exists a nonzero function a m (n, ) such that β m ( j) = lim n,/n c m( j) m( j ), 0 < β m ( j) <. (8) a m (n, ) We regard as the best model the candidate model that makes IC m the smallest, i.e., the best model chosen by IC m can be exressed as ĵ m = arg min j J IC m( j). Let α m be an asymtotic robability such that P( ĵ m = j ) α m as n and /n c. The IC m is consistent if α m =, and it is inconsistent if α m < Conditions for Consistency in CCA We begin with the following lemma (the roof is given in Aendix C): Lemma. Let The following equations are derived: W = W, W 2 = W + Z Z, where W and W 2 are indeendent random matrices defined by W W (n q j, I ), Z N (q j q ) (O q j q,, I I q j q ). Then, for any j J + \{ j }, F min ( j) F min ( j ) can be rewritten as F min ( j) F min ( j ) = log W 2 W. (9) 2. Let W = U U = W 3 + U 2 U 2, W 2 = (ZΓ j + U) (ZΓ j + U), W 3 = U U, where U = (U, U 2 ), U, U 2, and Z are mutually indeendent random matrices defined by U N (n q) (O n q,, I n q I ), U 2 N (q q j ) (O q q j,, I q q j I ), Z N (n q j ) (q q j )(O n q j,q q j, I q q j I n q j ), 7

8 Asymtotic Behaviors of Log-Determinants of Two Wishart Matrices and Γ j is a (q q j ) nonrandom matrix defined by Here Σ yy x is given by (4), and Σ j j j is defined by Γ j = Σ /2 yy x (Σ jy Σ j j Σ j j Σ jy) Σ /2. (20) Then, for any j J, F min ( j) F min (ω) can be rewritten as j j j Σ j j j = Σ j j Σ j j Σ j j Σ j j. (2) F min ( j) F min (ω) = log W 2 W + log W W 3. (22) Let δ j = log I + Γ j Γ j. It is known that δ j 0 with equality if and only if j J +, because Γ j = O,q q j holds if and only if j J +. Moreover, δ j can be rewritten as δ j = log I + Γ j Γ j = log Σ yy j Σ yy x, (23) (the roof is given in Aendix D). Notice that δ j deends on not n. It should be ket in mind that sometimes lim δ j becomes infinite and other times it is finite (see examles in Aendix E). The size of a convergent value and the order of δ j lay an imortant role in deciding whether a criterion is consistent. In addition, we assume that lim δ j > 0 for j J. When j J + \{ j }, it follows from Corollary and Lemma that F min ( j) F min ( j ) = o (). Notice that F min ( j) F min ( j ) = F min ( j) F min (ω) + F min (ω) F min ( j ). Since ω J +, F min (ω) F min ( j ) = o () holds. By using this result, Theorem 2, and Lemma, we have Let us define R j by F min ( j) F min ( j ) = δ j + o () ( j J ). (24) R j = X j /, X j χ 2 (q j q ) ( is bounded) (q j q )g(c) ( ), (25) where g(x) is the function given by (8). By alying Corollary to the case of CCA through Lemma, we derive n {F min( j) F min ( j )} = R j + o () ( j J + \{ j }). (26) From Lemma A.3 in Yanagihara (203), we can see that the LLBIC is consistent if the following equations hold: lim n,/n c lim n,/n c {IC m( j) IC m ( j )} > 0 ( j J + \{ j }), n {IC m( j) IC m ( j )} > 0 ( j J ). Besides, if there is a model j such that either of the above two inequities is not satisfied, the LLBIC is not consistent. Recall that IC m ( j) = nf min ( j) + m( j). Hence, from the above equations, (24), and (26), conditions for consistency are obtained as in the following theorem: 8

9 Theorem 3 Yanagihara, H., Hashiyama, Y. & Fujikoshi, Y. The IC m is consistent when n and /n c [0, ) if the following conditions are satisfied simultaneously: C. For any j J + \{ j }, R j ( lim n,/n c where R j is given by (25), and a m (n, ) and β m ( j) are given in (8). ) < β m ( j), (27) a m (n, ) C2. For any j J, where δ j is given by (23 ). lim n,/n c nδ j a m (n, ) > β m( j), (28) If either of the above two conditions is not satisfied, the IC m is not consistent when n and /n c [0, ). If is bounded, P( R j > ϵ) 0 ϵ > 0, because R j is a ositive random variable distributed according to the chi-square distribution. Hence, when is bounded, the condition C is equivalent to lim n a m (n, ) =. Although a condition for consistency has been derived, we still do not know which criteria satisfy that condition. Therefore, the conditions for the consistency of secific criteria in (6) are clarified in the following corollary (the roof is given in Aendix F): Corollary 2 Let S = { j J q q j > 0}, c a = arg solve{ g(x) + 2 = 0} (29) x [0,) Necessary and sufficient conditions for the consistency of secific criteria are as follows: nδ j AIC & TIC:, c [0, c a ), and for any j S, lim n,/n c 2(q q j ) >. nδ j ( c) 2 AIC c :, and for any j S, lim n,/n c (q q j )(2 c) >. nδ j BIC & CAIC: for any j S, lim n,/n c (q q j ) log n >. nδ j HQC: for any j S, lim n,/n c 2(q q j ) log log n >. Acknowledgment The first author s research was artially suorted by the Ministry of Education, Science, Sorts, and Culture, and a Grant-in-Aid for Challenging Exloratory Research, # , The third author s research was artially suorted by the Ministry of Education, Science, Sorts, and Culture, a Grant-in-Aid for Scientific Research (C), # ,

10 Asymtotic Behaviors of Log-Determinants of Two Wishart Matrices References Akaike, H. (973). Information theory and an extension of the maximum likelihood rincile. In 2nd. International Symosium on Information Theory (Eds. B. N. Petrov and F. Csáki), , Akadémiai Kiadó, Budaest. Akaike, H. (974). A new look at the statistical model identification. IEEE Trans. Automatic Control, AC-9, Anderson, T. W. (2003). An Introduction to Multivariate Statistical Analysis (3rd. ed.). John Wiley & Sons, Hoboken, New Kersey. Bozdogan, H. (987). Model selection and Akaike s information criterion (AIC): the general theory and its analytical extensions. Psychometrika, 52, Doeswijk, T. G., Hageman, J. A., Westerhuis, J. A., Tikunov, Y., Bovy. A. & van Eeuwijk, F. A. (20). Canonical correlation analysis of multile sensory directed metabolomics data blocks reveals corresonding arts between data blocks. Chemometr. Intell. Lab., 07, Fujikoshi, Y. (985). Selection of variables in discriminant analysis and canonical correlation analysis. In Multivariate Analysis VI (Ed. P. R. Krishnaiah), , North-Holland, Amsterdam. Fujikoshi, Y. & Kurata, H. (2008). Information criterion for some indeendence structures. In New Trends in Psychometrics (Eds. K. Shigemasu, A. Okada, T. Imaizumi & T. Hoshino), 69 78, Universal Academy Press, Tokyo. Fujikoshi, Y., Sakurai, T. & Yanagihara, H. (204). Consistency of high-dimensional AIC-tye and C -tye criteria in multivariate linear regression. J. Multivariate Anal., 23, Fujikoshi, Y., Shimizu, R. and Ulyanov, V. V. (200). Multivariate Statistics: High-Dimensional and Large-Samle Aroximations. John Wiley & Sons, Inc., Hoboken, New Jersey. Hannan, E. J. and Quinn, B. G. (979). The determination of the order of an autoregression. J. Roy. Statist. Soc. Ser. B, 4, Harville, D. A. (997). Matrix Algebra from a Statistician s Persective. Sringer-Verlag, New York. Hashiyama, Y., Yanagihara, H. & Fujikoshi, Y. (204). Jackknife bias correction of the AIC for selecting variables in canonical correlation analysis under model missecification. Linear Algebra Al., 455, Hurvich, C. M. and Tsai, C.-L. (989). Regression and times series model selection in small samles. Biometrika, 50, Jöreskog, K. G. (967). Some contributions to maximum likelihood factor analysis. Psychometrika, 32, Khalil, B., Ouarda, T. B. M. J. & St-Hilaire, A. (20). Estimation of water quality characteristics at ungauged sites using artificial neural networks and canonical correlation analysis. J. Hydrol., 405, Mardia, K. V. (974). Alications of some measures of multivariate skewness and kurtosis in testing normality and robustness studies. Sankhy ā Ser.B, 36, Muirhead, R. J. (982). Asects of Multivariate Statistical Theory. John Wiley & Sons, Inc., New York. Nishii, R., Bai, Z. D. & Krishnaiah, P. R. (988). Strong consistency of the information criterion for model selection in multivariate analysis. Hiroshima Math. J., 8, Schwarz, G. (978). Estimating the dimension of a model. Ann. Statist., 6, Siotani, M., Hayakawa, T. & Fujikoshi, Y. (985). Modern Multivariate Statistical Analysis: A Graduate Course and Handbook. American Sciences Press, Columbus, Ohio. Srivastava, M. S. (2002). Methods of Multivariate Statistics. John Wiley & Sons, New York. Srivastava, M. S. & Khatri, C. G. (979). An Introduction to Multivariate Statistics. North Holland, New York. Takeuchi, K. (976). Distribution of information statistics and criteria for adequacy of models. Math. Sci., 53, 2 8 (in Jaanese). Timm, N. H. (2002). Alied Multivariate Analysis. Sringer-Verlag, New York. Vahedia, S. (20). Canonical correlation analysis of rocrastination, learning strategies and statistics anxiety among Iranian female college students. Procedia Soc. Behav. Sci., 30, Wakaki, H. (2006). Edgeworth exansion of Wilks lambda statistic. J. Multivariate Anal., 97, Yanagihara, H. (203). Conditions for consistency of a log-likelihood-based information criterion in normal multivariate linear regression models under the violation of normality assumtion. TR No. 3-, Statistical Research Grou, Hiroshima University. Yanagihara, H., Wakaki, H. & Fujikoshi, Y. (202). A consistency roerty of the AIC for multivariate linear models when the dimension and the samle size are large. TR No. 2-08, Statistical Research Grou, Hiroshima University. 0

11 Yanagihara, H., Hashiyama, Y. & Fujikoshi, Y. Aendix A. Proof of Theorem It follows from a basic roerty of a determinant (see, e.g., Harville, 997, cor. 8..2) that Notice that log W 2 W = log I + W /2 Z ZW /2 = log Id + ZW Z. (A.) E[ZZ ] = I d, E [ tr { (ZZ ) 2}] = d( + d + ), E [ {tr(zz )} 2] = d(d + 2). By using the above results, the assumtion that Z and W are indeendent, and th in Fujikoshi et al. (200), we derive E [ ZW Z ] = h E[ZZ ] = h I d, E [ tr { (ZW Z ) 2}] = h 0 h 3 E [ tr { (ZZ ) 2}] + = d( + d + ) h 0 h 3 + E [ { tr(zz ) } ] 2 h 0 h h 3 d(d + 2) h 0 h h 3, where h i = n k i. Hence, the following equation is obtained: [ ZW ] E Z (/h )I 2 d(d + ) d{(d + ) + 2} d = + + 2d2 h 0 h 3 h 0 h h 3 h 0 h 2 h = O(n 2 ). 3 The above equation imlies that ZW Z = (/h )I d +O ( /2 n ). Hence, V = O () is derived, where V is given by (5). Moreover, ( + /h ) = O(), because 0 < ( + ) = n k <. h n k Substituting (5) into (A.) yields log W 2 W = log ( + /h )I d I d + ( + /h )n V ( ) (n k ) = d log + tr(v ) + O (n 2 ). n k n(n k ) (A.2) Notice that ( ) log = log( /n) + O(n 2 ), n k (n k ) ( = ) + O( 3/2 n 3 ). n(n k ) n n From the above equations and (A.2), (4) is roved. Next, we rove (6). Notice that for sufficiently large,

12 Hence, we have Asymtotic Behaviors of Log-Determinants of Two Wishart Matrices ZW Z = { (ZW Z ) } = (ZZ ) /2 { (ZZ ) /2 (ZW Z ) (ZZ ) /2} (ZZ ) /2. tr(zw Z ) = tr(u U 2 ), where U and U 2 are d d random matrices defined by U = ZZ, U 2 = (ZZ ) /2 (ZW Z ) (ZZ ) /2. From a basic roerty of a Wishart distribution and th in Fujikoshi et al. (200), we can see that U and U 2 are indeendent, and U W d (, I d ), U 2 W d (n k + d, I d ). (A.3) Let V = (U I d ), V 2 = h0 + d {U 2 (h 0 + d)i d }. (A.4) Then, it is easy to see that V = O () and V 2 = O (), and Notice that tr(v ) d N(0, 2d) as, tr(v 2 ) d N(0, 2d) as (n ). (A.5) n h 0 + d = n n + O(n 2 ), d h 0 + d d = O(n 2 ). h h 0 + d = n + O(/2 n 3/2 ), By using the above equations, V, and V 2, V can be exanded as tr(v ) = n { tr(u U 2 ) d } h = n h 0 + d tr (I d + ) ( ) V I d + h0 + d V 2 d h ( ) { } n = tr(v ) n n tr(v 2) + O ( /2 n ). (A.6) Thus, from (A.3), (A.4), (A.5), and (A.6), (6) is roved. B. Proof of Theorem 2 Let H(Λ, O r, r ) Q be a singular value decomosition of Γ, where H is a th orthogonal matrix satisfying H H = H H = I, Q is an rth orthogonal matrix satisfying Q Q = QQ = I r, and Λ is an rth diagonal matrix whose ath diagonal element is a singular value λ a, i.e., Λ = diag(λ,..., λ r ) (0 λ λ r ). Then, it is easy to see that V = UH and B = ZQ are indeendent, and 2

13 Yanagihara, H., Hashiyama, Y. & Fujikoshi, Y. V N (n k) (O n k,, I I n k ), B N (n k) r (O n k,r, I r I n k ). Let us artition V as V = (V, V 2 ), where V and V 2 are (n k) r and (n k) ( r) matrices, resectively. Then, we have U U = H V V V V 2 V 2 V V 2 V H, 2 (ZΓ + U) (ZΓ + U) = H (BΛ + V ) (BΛ + V ) (BΛ + V ) V 2 V 2 (BΛ + V ) V 2 V H. 2 (B.) By alying the formula for the determinant of a artitioned matrix (see, e.g., Harville, 997, th ) to (B.), we derive It follows from (B.2) that W = U U = V 2 V 2 V {I n k V 2 (V 2 V 2) V 2 }V, W 2 = (ZΓ + U) (ZΓ + U) T = log W 2 W log I + ΓΓ = log = V 2 V 2 (BΛ + V ) {I n k V 2 (V 2 V 2) V 2 }(BΛ + V ). V {I n k V 2 (V 2 V 2) V 2 }V (BΛ + V ) {I n k V 2 (V 2 V 2) V Notice that V, V 2, and B are mutually indeendent, and 2 }(BΛ + V ) log I + ΓΓ. (B.2) (B.3) Hence, we derive V {I n k V 2 (V 2 V 2) V 2 }V W r (n k + r, I r ), (BΛ + V ) {I n k V 2 (V 2 V 2) V 2 }(BΛ + V ) W r (n k + r, I r + Λ 2 ). n k + r V {I n k V 2 (V 2 V 2) V 2 }V I r, n k + r (I r + Λ 2 ) /2 (BΛ + V ) {I n k V 2 (V 2 V 2) V 2 }(BΛ + V )(I r + Λ 2 ) /2 I r. These equations imly that (n k + r) r V {I n k V 2 (V 2 V 2) V 2 }V ), (n k + r) r I r + Λ 2 (BΛ + V ) {I n k V 2 (V 2 V 2) V 2 }(BΛ + V ). (B.4) Notice that Γ Γ = QΛ 2 Q. By using this equation and a basic roerty of a determinant (see e.g., Harville, 997, cor. 8..2), we derive I r + Λ 2 = I r + Γ Γ = I + ΓΓ. (B.5) By substituting (B.4) and (B.5) into (B.3), T roved. 0 is obtained. This means that equation (9) is 3

14 Asymtotic Behaviors of Log-Determinants of Two Wishart Matrices C. Proof of Lemma We first describe a lemma about another exression of S yy j ; this is required for roving Lemma (the roof is given in Aendix G). Lemma A. Let E, A j, and B be mutually indeendent random matrices, which are defined by E N n q (O n,, I I n ), A j N n (q q j )(O n,q q j, I q q j I n ), B N n q (O n,q, Σ xx I n ), and let B j denote the n q j matrix consisting of the columns of B indexed by the elements of j. Then, for any j J, ns yy j can be rewritten as ns yy j = Σ /2 yy x(a j Γ j + E) (I n P j )(A j Γ j + E)Σ/2 yy x, (C.) where P j is the rojection matrix to the subsace sanned by the columns of B j, i.e., P j = B j (B j B j) B j, and Γ j is given by (20). In articular, when j J +, ns yy j = Σ /2 yy xe (I n P j )EΣ /2 yy x. (C.2) When j J + \{ j }, it follows from Lemma A. that F min ( j) F min ( j ) = log ns yy j ns yy j = log E (I n P j )E E (I n P j )E = log E (I n P j )E + E (P j P j )E E. (I n P j )E Notice that I n P j and P j P j are idemotent matrices, and (I n P j )(P j P j ) = O n,n holds. Hence, (9) is roved. When j J, it follows from Lemma A. that F min ( j) F min (ω) = log ns yy j log ns yy x = log (A j Γ j + E) (I n P j )(A j Γ j + E) log E (I n P ω )E = log (A jγ j + E) (I n P j )(A j Γ j + E) E (I n P j )E = log (A jγ j + E) (I n P j )(A j Γ j + E) E (I n P j )E + log E (I n P ω )E + E (P ω P j )E E. (I n P ω )E + log E (I n P j )E E (I n P ω )E Notice that I n P j, I n P ω, and P ω P j are idemotent matrices, and (I n P ω )(P ω P j ) = O n,n holds. Hence, (22) is roved. D. Proof of Equation (23) It follows from the general formula for the inverse of a block matrix, e.g., th in Harville 4

15 Yanagihara, H., Hashiyama, Y. & Fujikoshi, Y. (997), that Σ xx = Σ j j Σ j j Σ j j Σ j j = Σ j j + Σ j j Σ j jσ j j j Σ j j Σ j j Σ j j j Σ j j Σ j j Σ j j Σ j jσ j j j Σ j j j, where Σ j j j are given by (2). By using the above equation, we have Σ xyσ xx Σ xy = ( Σ jy, Σ jy) It follows from (20) and (D.) that Hence, we have Σ j j + Σ j j Σ j jσ j j j Σ j j Σ j j Σ j j j Σ j j Σ j j Σ j j Σ j jσ j j j Σ j j j = Σ jy Σ j j Σ jy + (Σ jy Σ j j Σ j j Σ jy) Σ j j j (Σ jy Σ j j Σ j j Σ jy). By using the above equation, (23) is roved. Σ /2 yy x Σ yy j Σ /2 yy x = I + Γ j Γ j. I + Γ j Γ j = Σ /2 yy x Σ yy j Σ /2 yy x = Σ yy j Σ yy x. Γ j = O,q q j holds when j J +. Hence, δ j = 0 holds when j J +. Σ jy Σ jy (D.) On the other hand, it follows from (2) that E. Two Examles of δ j Two examles of δ j are resented in this section. In both examles, we assume that j is the subset of j such that j = { j, j j }. First, we show the case that a limiting value of δ j is bounded. Let Σ xx = Ω(q), Σ yy = Ω(), Σ xy = ρ q ρ/{ + (q )ρ}, q q where ρ (, ) and Ω(m) is an m m symmetric matrix defined by Ω(m) = ( ρ)i m + ρ m m. From the general formula of the inverse of the sum of two matrices, e.g., cor in Harville (997), we have Then, we can see that q Ω(m) = { } ρ I m ρ + (m )ρ m m. Σ yy x = Σ yy j = Ω() (ρ q )Ω(q ) (ρ q ) { } = ( ρ) ρ I + + (q )ρ = ( ρ) { + ( + q )}ρ. + (q )ρ (E.) It follows from the same calculations as in (E.) that 5

16 Asymtotic Behaviors of Log-Determinants of Two Wishart Matrices Σ yy j = Ω() (ρ q j )Ω(q j ) (ρ q j ) = ( ρ) { + ( + q j )}ρ. (E.2) + (q j )ρ Hence, from (E.) and (E.2), the following equation is derived: δ j = log Σ yy j Σ yy x = log { + ( + q j )}{ + (q )ρ} { + ( + q )}{ + (q j )ρ} { } + (q )ρ log. + (q j )ρ This equation indicates that the limiting value of δ j is bounded as. Next, the case that δ j aroaches is shown. Let Σ xx = I q, Σ yy = I, Σ xy = where α is a -dimensional vector defined by ρ α = 2 q ρ (ρ,..., 2 ρ ). Here ρ (, ). Notice that q 0 q q α, α α = ρ2 q ρ 2 (ρ 2 ) k = { (ρ 2 ) }. q k= Hence, we can see that Σ yy x = Σ yy j = I (α q )I q (ρ q α ) = I q αα = { (ρ 2 ) } = (ρ 2 ). (E.3) It follows from the same calculations as in (E.3) that Σ yy j = I (α q j )I q j (ρ q j α ) = From (E.3) and (E.4), the following equation is derived: δ j = log Σ [ yy j Σ yy x = log q j { (ρ 2 ) } ] q This equation indicates that δ j aroaches as. [ q j { (ρ 2 ) } ]. (E.4) q log(ρ 2 ). F. Proof of Corollary 2 From a simle calculation, we have the following exansion: n { b(q j + ) b(q j ) b(q + ) + b(q ) } = 6 n(2n ) (n ) 2 (q j q ) + O(n 2 ).

17 It follows from the above equation and (6) that 2(q j q ) Yanagihara, H., Hashiyama, Y. & Fujikoshi, Y. (AIC) (2 /n)(q j q )/( /n) 2 + O(n ) (AIC c ) 2(q j q ) + ˆκ ( jy) ˆκ j ˆκ ( j y) + ˆκ j m( j) m( j ) = (q j q ) log n (q j q )( + log n) 2(q j q ) log log n (TIC) (BIC) (CAIC) (HQC). (F.) On other hand, by using the results in Mardia (974), the mean and variance of ˆκ(D) in (7) are calculated as d(d + 2) E [ˆκ(D)] = n(n + 2), Var [ˆκ(D)] = 8d(d + 2)(n 2)(n + )4 (n d)(n d + 2). n 4 (n + 2) 2 (n + 4)(n + 6) These imly that for any j J, ˆκ j 0, ˆκ ( jy) 0. It follows from the above equations and (F.) that 2(q j q ) (AIC, TIC) {(2 c)/( c) 2 }(q j q ) (AIC c ) a m (n, ) =, β m ( j) = log n q j q (BIC, CAIC) log log n 2(q j q ) (HQC) Hence, it is easy to see that lim n,/n c a m (n, ) = (AIC, AIC c, TIC) 0 (BIC, CAIC, HQC).. (F.2) By using the above results and (27), we can see that condition C holds for the BIC, CAIC, and HQC. Condition C holds for the AIC, AIC c, and TIC if goes to and the following equation is satisfied: g(c) < 2 (AIC, TIC) /( c) + /( c) 2 (AIC c ) where g(x) is given by (8). The above equation imlies that condition C holds for the AIC and TIC if goes to and c [0, c a ), and it holds for the AIC c if goes to, because g(x) + 2 > 0 holds when x [0, c a ) and g(x) + ( x) + ( x) 2 > 0 holds when x [0, ), where c a is given in (29). Moreover, from (28) and (F.2), we can see that condition 2 holds if the following equation is satisfied: q q j < lim n,/n c nδ j /(2), (AIC, TIC) nδ j ( c) 2 /{(2 c)} (AIC c ) nδ j /( log n) nδ j /(2 log log n) (BIC, CAIC) (HQC) It is easy to see that the above equation is satisfied if q q j 0, because δ j > 0. Hence, it is sufficient to consider the case of j S, where S is given in (29). Consequently, Corollary 2 is roved. 7.

18 Asymtotic Behaviors of Log-Determinants of Two Wishart Matrices G. Proof of Lemma A. Let Y = (y,..., y n+ ) and X = (x,..., x n+ ), where y i and x i are the ith individuals from y and x, resectively, and X j denotes an (n + ) q j matrix consisting of the columns of X indexed by the elements of j. Then, ns yy j is exressed as ns yy j = Y H n+ { I n+ H n+ X j ( X j H n+ X j ) X j H n+ } H n+ Y, where H n is the rojection matrix to the orthocomlement of the subsace sanned by n, i.e., H n = I n n n/n, and n is an n-dimensional vector of ones. Let V be an n random matrix such that (B, V ) N n (q+) (O n,q+, Σ), where B is an n q random matrix. From a roerty of a multivariate normal distribution, we can rewrite ns yy j using V and B, as follows: ns yy j = V (I n P j )V, (G.) where P j = B j (B j B j) B j, and B j is an n q j matrix consisting of the columns of B indexed by the elements of j. From a roerty of a conditional distribution of a multivariate normal distribution, e.g., th in Srivastava and Khatri (979), we have V B N n (BΣ xx Σ xy, Σ yy x I n ), B j B j N n q (B j Σ j j Σ j j, Σ j j j I n ), where Σ j j j is given by (2). Hence, we can exress V as V = BΣ xx Σ xy + EΣ /2 yy x, B j = B j Σ j j Σ j j + A j Σ /2 j j j, (G.2) where E, A j, and B j are mutually indeendent random matrices. It should be ket in mind that any S yy j ( j J) can be reresented by using the common E, because E is indeendent of j. Without loss of generality, we assume that B is arranged as (B j, B j). It follows from the same calculation as in (D.) that BΣ xx Σ xy = (B j, B j) Σ j j + Σ j j Σ j jσ j j j Σ j j Σ j j Σ j j j Σ j j Σ j j Σ j j Σ j jσ j j j Σ j j j Σ jy Σ jy { = B j Σ j j Σ jy Σ j jσ j j j (Σ jy Σ j j Σ j j Σ jy) } + B jσ j j j (Σ jy Σ j j Σ j j Σ jy). Substituting B j in (G.2) into the above equation yields { BΣ xx Σ xy = B j Σ j j Σ jy Σ j jσ j j j (Σ jy Σ j j Σ j j Σ jy) } + (B j Σ j j Σ j j + A j Σ /2 j j j )Σ j j j (Σ jy Σ j j Σ j j Σ jy) = B j Σ j j Σ jy + A j Γ j Σ/2 yy x, where Γ j is the (q q j ) matrix defined by (20). Substituting the above equation into V in (G.2) 8

19 yields Yanagihara, H., Hashiyama, Y. & Fujikoshi, Y. V = B j Σ j j Σ jy + A j Γ j Σ/2 yy x + EΣ /2 yy x. (G.3) Notice that (I n P j )B j = O n,q j. By using (G.) and (G.3), (C.) is roved. Moreover, Γ j = O,q q j if j J +, because then (2) holds. This imlies that for any j J + V = B j Σ j j Σ jy + EΣ /2 yy x. (G.4) Hence, from (G.) and (G.4), (C.2) is roved. 9