-dimensional space. d 1. d k (1 ρ 2 j,j+k;j+1...j+k 1). Γ(2m) m=1. 2 ) 2 4d)/4. Γ d ( d 2 ) (d 2)/2. Γ d 1 (d)

Transcription

1 Generating random correlation matrices based on partial correlation vines and the onion method Harry Joe Department of Statistics University of British Columbia Abstract Partial correlation vines and the onion method are presented for generating random correlation matrices. As a special case, a uniform distribution over the set of d d positive definite correlation matrices obtains. Byproducts are: (a) For a uniform distribution over the space of d d correlation matrices, the marginal distribution of each correlation is Beta(d/, d/) on (, ). (b) An identity is obtained for the determinant of a correlation matrix R via partial correlations in a vine. (c) A formula is obtained for the volume of the set of d d positive definite correlation matrices in ( d ) -dimensional space. Outline. Statement of key results on generating random correlation matrices. Parametrization with partial correlations 3. Regular vines, D-vines, C-vines 4. Random correlation matrices based on partial correlations 5. Random correlation matrices based on onion method Key results: R is a d d correlation matrix. Several simple ways to generate a random R that is uniform over set of positive definite d d correlation matrices; more generally with density [det(r)] α.. Identity: ( ρ )( ρ 3)( ρ 3;) for d = 3 det(r) = ( ρ i,i+) i= d k ( ρ j,j+k;j+...j+k ). k= j= [partial correlations in the double product] 3. Volume of d d correlation matrices in d()/ dimensional space is: ()/ π (d )/4 Γ(m) m=, if d is odd; (3d d(d )/4 π ()/4 Γ ( d+ ) 4d)/4 Γ d ( d ) (d )/ Γ(m) m=, if d is even. Γ (d) Partial correlations correlations ρ i,i+ for i =,..., d the partial correlations ρ ij;i+,...j for j i For example, ρ, ρ 3, ρ 34, ρ 3;, ρ 4;3, ρ 4;3 for d = 4. For multivariate normal random variables, partial correlations are conditional correlations. In general, they are functions of a correlation matrix. Advantage: the ( d ) parameters can independently take values in the interval (, ); different vines can be used. Disadvantage: the reparametrization is non-unique, and depends on the order of indexing, e.g.: ρ, ρ 3, ρ 3; or ρ, ρ 3, ρ 3;

2 d = 5: 3 different vines + matrix for onion method are shown. d-vine which is like Markovian dependence c-vine regular partial correlation vine onion method #distinct vines increases quickly as d increases, partial correlations independently in (, ). ρ ρ 3; ρ 4;3 ρ 5;34 ρ ρ 3 ρ 4;3 ρ 5;34 ρ 3 ρ 3 ρ 34 ρ 35;4 ρ 4 ρ 4 ρ 43 ρ 45 ρ 5 ρ 5 ρ 53 ρ 54 ρ ρ 3 ρ 4 ρ 5 ρ ρ 3; ρ 4; ρ 5; ρ 3 ρ 3 ρ 34; ρ 35; ρ 4 ρ 4 ρ 43 ρ 45;3 ρ 5 ρ 5 ρ 53 ρ 54 ρ ρ 3; ρ 4; ρ 5;4 ρ ρ 3 ρ 4 ρ 5;4 ρ 3 ρ 3 ρ 34; ρ 35;4 ρ 4 ρ 4 ρ 43 ρ 45 ρ 5 ρ 5 ρ 53 ρ 54 ρ ρ 3 ρ 4 ρ 5 ρ ρ 3 ρ 4 ρ 5 ρ 3 ρ 3 ρ 34 ρ 35 ρ 4 ρ 4 ρ 43 ρ 45 ρ 5 ρ 5 ρ 53 ρ 54 D-vine for d = 5: specify densities separately for each partial correlation; then density of the correlation matrix R is f R (r) = f ρ (r )f ρ3 (r 3 )f ρ34 (r 34 )f ρ45 (r 45 ) f ρ3; (r 3; )f ρ4;3 (r 4;3 )f ρ35;4 (r 35;4 ) f ρ4;3 (r 4;3 )f ρ5;34 (r 5;34 )f ρ5;34 (r 5;34 ) Jacobian Under appropriate choices, this leads to a uniform density. References for vines D. Kurowicka, R. Cooke, Uncertainty Analysis with High Dimensional Dependence Modelling, Wiley, 006. Vines are a graphical method for modeling multivariate dependencies 3 {} {3} {34} 3 4 {3 } {3} {} {4} 4 {3 } {4 3} {34 } {4 } {4 3}

3 3 {3 } {3} {} {4} {45} 4 5 {34 } {4 } {5 4} {35 4} {5 4} Random correlation matrices [Joe, 006, JMVA] Partial correlation approach: distributions separately for ρ i,i+ for i =,..., d, and ρ ij;i+,...j for j i. This leads to a joint distribution of {ρ ij : i < j d}. More generally, choose distributions separately for partial correlation in a regular vine. Previous papers : combination of random eigenvalues and random (orthogonal) matrices, or a random T matrix leading to T T, but cannot do distribution theory? Motivations: generation of ellipsoidal clusters to study clustering methods efficiency of estimating eq n methods for multiv probit model assess probability that a correlation matrix is a rank correlation matrix. prior for correlation matrix in Bayesian statistics? Main result With appropriate choices of beta distributions for ρ i,i+ for i =,..., d, and ρ ij;i+,...j for j i, can get a joint density for {ρ ij : i < j d} that is R α, where α > 0. More generally, same for partial correlations on a regular vine. In this case, the joint density is invariant to the order of indexing of variables for the partial correlations (and to the vine), and each ρ ij marginally has a Beta(α + d/, α + d/) distribution on (, ). Special case with α = : uniform density over the set of positive definite correlation matrices. where Anderson (958, p. 80): partial corr ρ j,j+k;j+,...,j+k ( k d ) is ρ j,j+k r (j, k)(r (j, k)) r 3 (j, k) [ ] / [ r (j, k)(r (j, k)) r (j, k) r 3 (j, k)(r (j, k)) r 3 (j, k) R[j : j + k] = r (j, k) ρ j,j+k r (j, k) R (j, k) r 3 (j, k), ρ j+k,j r 3(j, k) with r (j, k) = (ρ j,j+,..., ρ j,j+k ), r 3(j, k) = (ρ j+k,j+,..., ρ j+k,j+k ), and R (j, k) consisting of the middle k rows / columns. Hence ρ j,j+k = r (j, k)(r (j, k)) r 3 (j, k) + ρ j,j+k;j+,...,j+k D jk, ] / 3

4 where D jk = [ ][ ] r (j, k)(r (j, k)) r (j, k) r 3(j, k)(r (j, k)) r 3 (j, k). d = 3 [ρ for rv, r for dummy variable of function] Consider (ρ, ρ 3, ρ 3 ) (ρ, ρ 3, ρ 3 ). Since ρ 3 = ρ 3 ρ ρ 3 ( ρ )( ρ 3 ), the Jacobian of the transformation is: r 3 / r 3 = [( r )( r 3)] /. Note that r3 = r r3 r3 + r r 3 r 3 det(r) ( r )( r 3 ) = ( r )( r 3 ) det(r(r, r 3, r 3 )) = r r 3 r 3 + r r 3 r 3 Beta(α, α) density on (, ): [B(α, α)] ( u ) α or U = V, V Beta(α, α) on (0, ). ρ, ρ 3 Beta(α, α ), ρ 3 Beta(α, α ) independently. = ( r )( r 3)( r 3 ) (D) f ρ,ρ 3,ρ 3 (r, r 3, r 3 ) ( r 3 ) α [( r )( r 3)] α 3/ If α = α +, then the density is [det(r)] α (D) This is symmetric in r, r 3, r 3, which means the same joint density obtains from ρ, ρ 3 Beta(α, α ), ρ 3 Beta(α, α ) etc. α =, α = 3/ Uniform over set of correlation matrices. d > 3 : computer proof before math proof Based on the results for the d = 3 case, conjectured an extension of (D) as an identity for det(r), and conjectured the Beta distributions on ρ ij;i+,...,j needed to get joint density of (ρ ij ) to be [det(r)] α. The identity was verified numerically and the univariate margins of the random correlation matrix were also checked numerically before proving the results for general d. Results for d > 3. Thm : det(r) = ( ρ i,i+) i= d k ( ρ j,j+k;j+...j+k ). k= j= Thm 4: The determinant J d of the Jacobian for the transform of (ρ, ρ 3, ρ 3, ρ 34, ρ 4, ρ 4, ρ 45,..., ρ d ) to (ρ, ρ 3, ρ 3;, ρ 34, ρ 4;3, ρ 4;3, ρ 45,..., ρ d;... ) is: ( ρ i,i+) d [ i= d d k ( ρ i,i+k;i+...i+k ) k] /. k= i= 4

5 Thm 5: If α k = α + (d k), k =,..., d, and ρ i,i+k;i+...i+k is Beta(α k, α k ) on (, ) for i < i + k d, then the joint density becomes where the normalizing constant c d is (α +d k)(d k) k= c [ { }] α d det (rij ) i,j d, [ ( B α + (d k), α + (d k))] d k. k= If α = and α k = (d + k) for k =,..., d, leading to uniform joint density for {ρ ij, i < j}, then the normalizing constant is c d = k= (d k) [ ( B (d k + ), (d k + ))] d k k= and the recursion is = k= k [ ( B (k + ), (k + ))] k, k= c d = c () [ B( d, d)]. ( d ) Byproduct: normalizing constant is the volume of the set of d-dim positive definite correlation matrices in -dim space c d / d()/ decreases quickly as d increases. d c d c d / d()/ = π (/) = π (3/7) = π 6 (3/8) = π 6 (89/535) = π (n 7 /d 7 ) = π (n 8 /d 8 ) = π 0 (n 9 /d 9 ) = π 0 (n 0 /d 0 ) Random correlation matrix based on vines [Lewandowski, Kurowicka and Joe, 007] Extensions of Theorems,4,5 in Joe (006, JMVA). V = (e) is a partial correlation vine; for example, V = (, 3, 4, 3;, 4;, 34; ) with depth=(order )=(#conditioned variables): (0, 0, 0,,, ). For a vine based on d variables, there are d edges with depth k e = 0, d edges with depth k e =,..., and edge with depth k e = d. Let R be a (random) correlation matrix based on distributions for {ρ e : e V }. T: det(r) = e V ( ρ e) [Kurowicka and Cooke, 006, LAA] (product over edges of the vine) 5

6 T4: Jacobian of transform from correlations {ρ ij } to partial correlations {ρ e } is { } / e V ( ρ e) d k e T5. Let ρ e Beta(α ke +, α ke +) on (, ) independently, where α ke + = α + (d k e). Get uniform over space of correlation matrices if α =, and then marginal distribution for each correlation is Beta(d/, d/). The C-vine algorithm for generating correlation matrices with density [det(r)] η, η >.. Initialization β = η + (d )/.. Loop for k =,..., d. a) β β ; b) Loop for i = k +,..., d; i) generate partial corr p k,i;,...,k Beta(β, β) on (, ); ii) use recursive formula on p k,i;,...,k to get r k,i = r i,k [no matrix inversions]; 3. Return r, a d d correlation matrix. Onion method for random correlation matrices [Ghosh & Henderson, 003; extended LKJ 007] Result. Consider the spherical density c( w w) β for w R m, w w, where c = Γ(β+m/)π m/ /Γ(m/). If W has this density, then W = V U where V Beta(m/, β) and U is uniform on the surface of the m- dimensional hypersphere. If Q = AW, where A is an m m non-singular matrix, then the density of Q is c[det(aa )] / ( q [AA ] q) β, q q [A A] q. ( ) rm q Result. Partition r m+ = q where r m is an m m correlation matrix and q is a m-vector of correlations such that r m+ is an (m + ) (m + ) correlation matrix. Then det(r m+ ) = det(r m ) ( q r m q ). We use upper case letter of r m, q, r m+ to denote random vectors and matrices. Let β, β m > 0. If R m has density [det(r m )] βm and Q given R m = r m has density [det(r m )] / ( q r m q) β, then the density of R m+ is [det(r m )] βm 3/ ( q r m q) β. If β m = β +, then the density of R m+ is [det(r m+ )] β. Algorithm for the extended onion method to get random correlation matrices in dimension d with density [det(r)] η, η > 0. ( ) r. Initialization. β = η + (d )/, r u, where u Beta(β, β), r r. Loop for m =,..., d. 6

7 (a) β β ; (b) generate y Beta(m/, β), generate u = (u,..., u m ) uniform on the surface of m-dimensional hypersphere; (c) w y / u, obtain Cholesky decomp AA = r, set q Aw; ( ) r q (d) r q. 3. Return r, a d d correlation matrix. Why is this called the onion method??? By the symmetry, each ρ jk = R jk in the correlation matrix has a marginal Beta(η + [d ]/, η + [d ]/) density on (, ). For the special case of η = leading to uniform over the space of correlation matrices, the marginal density of each R jk is Beta(d/, d/) on (, ). Computational time: In C, onion method with incremental Cholesky is fastest, and C-vine is faster than onion method without incremental Cholesky. In Matlab, C-vine (which avoids matrix inversions) is fastest. D-vine is slower because of matrix inversions. Other comments: If partial correlations in a particular vine are needed, then use the appropriate vine. Choosing the vine and the densities for the partial corr in this vine, one could get random correlation matrices that have larger correlations at a few particular pairs. References H. Joe, Generating random correlation matrices based on partial correlations, Journal of Multivariate Analysis 97 (006) S. Ghosh, S.G. Henderson, Behavior of the NORTA method for correlated random vector generation as the dimension increases. ACM Transactions on Modeling and Computer Simulation (TOMACS) 3 (003), T. Bedford, R. Cooke, Vines - a new graphical model for dependent random variables, Annals of Statistics 30 (00) D. Kurowicka, R. Cooke, Uncertainty Analysis with High Dimensional Dependence Modelling, Wiley, 006. D. Kurowicka, R. M. Cooke, Completion problem with partial correlation vines, Linear Algebra and Its Applications 48 (006) D. Lewandowski, D. Kurowicka, H. Joe, Generating random correlation matrices based on vines and extended Onion method, (007), submitted. 7