Teacher s Corner. f (x 1,x 2 ) =

Transcription

1 Teacher s Corner Uniform Correlation Miture of Bivariate Normal Distributions and Hypercubically Contoured Densities That Are Marginally Normal Kai ZHANG, LawrenceD.BROWN,EdwardGEORGE, and Linda ZHAO Downloaded by [University North Carolina - Chapel Hill] at 6:4 3 August 4 The bivariate normal density with unit variance and correlation ρ is well known. We show that by integrating out ρ, the result is a function of the maimum norm. The Bayesian interpretation of this result is that if we put a uniform prior over ρ, then the marginal bivariate density depends only on the maimal magnitude of the variables. The square-shaped isodensity contour of this resulting marginal bivariate density can also be regarded as the equally weighted miture of bivariate normal distributions over all possible correlation coefficients. This density links to the Khintchine miture method of generating random variables. We use this method to construct the higher dimensional generalizations of this distribution. We further show that for each dimension, there is a unique multivariate density that is a differentiable function of the maimum norm and is marginally normal, and the bivariate density from the integral over ρ is its special case in two dimensions. KEY WORDS: Bivariate normal miture; Khintchine miture; Uniform prior over correlation.. INTRODUCTION It is well known that a multivariate distribution, which has normal marginal distributions, is not necessarily jointly multivariate normal (in fact, not even when the distribution is conditionally normal; see Gelman and Meng 99), that is, a p-dimensional Kai Zhang is Assistant Professor, Department of Statistics and Operations Research, the University of North Carolina, 36 Hanes Hall, CB# 36, Chapel Hill, NC 7599 ( zhangk@ .unc.edu). Lawrence D. Brown ( lbrown@wharton.upenn.edu), Edward George ( edgeorge@wharton.upenn.edu), and Linda Zhao ( lzhao@wharton.upenn.edu) are Professors, Department of Statistics, the Wharton School, the University of Pennsylvania, Philadelphia, PA 94. The authors appreciate insightful comments and suggestions from Robert Adler, Barry Arnold, James Berger, Richard Berk, Shankar Bhamidi, Peter Bickel, Amarjit Budhiraja, Andreas Buja, Dennis Cook, Bradley Efron, Jianqing Fan, Andrew Gelman, Ben Goodrich, Shane Jensen, Tiefeng Jiang, Iain Johnstone, Abba Krieger, J. Steve Marron, Henry McKean, Xiao-Li Meng, Dylan S. Small, J. Michael Steele, Yazhen Wang, and Cun-Hui Zhang. The authors also thank the editors and reviewers for careful reading and helpful comments. This work was supported by grants DMS-3969 and DMS-3795 from the U.S. National Science Foundation. Color versions of one or more of the figures in the article can be found online at multivariate distribution X (X,...,X p ) that has marginal standard normal densities φ( ),φ( ),...,φ( p ) and marginal distribution functions ( ), ( ),..., ( p ) may not have a jointly multivariate normal density f (,..., p ) (π) p/ / ep T /} (.) for some p by p correlation matri. Classical eamples of such distributions can be found in Feller (97) and Kotz, Balakrishnan, and Johnson (4). In this article we focus on one particular class of such distributions that arises from uniform mitures of bivariate normal densities over the correlation matri. When p, the bivariate normal density with unit variances is well known: f (, ρ) π ρ ep + ρ ( ρ ) } (.) for some ρ. By a uniform correlation miture of bivariate normal density we mean the bivariate density function f (, ) below: f (, ) f (, ρ)dρ. (.3) This type of continuous miture of bivariate normal distributions has been used in applications such as imaging analysis (Aylward and Pizer 997). We show that such a uniform correlation miture results in a bivariate density that depends on the maimal magnitude of the two variables: f (, ) ( ( )), (.4) where ( ) is the cdf of standard normal distribution, and ma, }. This bivariate density has a natural Bayesian interpretation: it can be regarded as the marginal density of (X,X ) if we put a uniform prior over the correlation ρ (this type of density is referred to as the marginal predictive distribution in Bayesian literature). Moreover, one interesting feature of this density is that its isodensity contours consists of concentric squares. Although we were not able to find the above result in the literature, we noticed that the bivariate density f (, ) ( ( ) ) is first obtained in a different manner by Bryson and Johnson (98). In this article, we consider constructing multivariate distributions through the Khintchine miture method (Khintchine 938). The bivariate density f (, ) is listed as an 4 American Statistical Association DOI:.8/ The American Statistician, August 4, Vol. 68, No. 3 83

2 eample of their construction. But the link between this density and the uniform miture over correlations is not addressed. Through the Khintchine miture approach, we show that the resulting mied density is a function of. Moreover, we show that for each p, this resulting density is the unique multivariate density that is a differentiable function of and is marginally normal. It thus becomes interesting to investigate the connection between the Khintchine miture and the uniform miture over correlation matrices. ( ( )) + ( ( )) + π e / d ( ( )) ( ( )) π e / π e / φ( ). (.) Downloaded by [University North Carolina - Chapel Hill] at 6:4 3 August 4. THE UNIFORM CORRELATION MIXTURE INTEGRAL Our first main result is the following theorem: Theorem. f (, ) f (, ρ)dρ ( ( )). (.) The proof can be found in Appendi A. Note that f (, )is a proper bivariate density, and it is marginally standard normal: f (, )d R + ( ( ))d (, ) (,) ( ( ))d The form of this bivariate density implies that its isodensity contours consists of concentric squares. Thus, an intuitive interpretation of this result is that if we average the isodensity contours of bivariate normal distributions which are concentrically elliptic, we shall get an isodensity contour of concentric squares. The plot of f (, )isgiveninfigure. This result also indicates that if we have a uniform prior over the correlation ρ, the resulting marginal density depends only on the maimal magnitude of the two variables. The application of this result to Bayesian inference needs to be further investigated. But from this marginal density of and, the posterior distribution for ρ can be derived in fully eplicit form as the ratio of the bivariate normal density and this marginal density. Thus this result immediately leads to the Bayes factor of testing ρ, in this very special situation. Moreover, the uniform prior has been used (see Barnard, McCulloch, and Meng for theory and applications to shrinkage estimation) to model covariance matrices. We shall report these types of applications in future work Figure. The plot of the bivariate density function f (, ) ( ( )). Note that this bivariate density has a contour of squares and is marginally normal. 84 Teacher s Corner

3 Downloaded by [University North Carolina - Chapel Hill] at 6:4 3 August 4 3. CONNECTION TO THE KHINTCHINE MIXTURE AND HYPERCUBICALLY CONTOURED DISTRIBUTIONS THAT ARE MARGINALLY NORMAL In Bryson and Johnson (98), the authors developed a method of generating multivariate distributions through Khintchine s theorem. Khintchine s theorem states that any univariate continuous random variable X has a single mode if and only if it can be epressed as the product X YU, where Y and U are independent continuous variables and U has a uniform distribution over [, ]. This result and its etensions can be used to construct multivariate distributions that have specified marginal distributions. As an eample of such a distribution with standard normal marginal distributions, the authors considered the construction with mutually independent U i Uniform[, ],i, and Y χ 3 (so that Y>and Y χ3 ). The random variables X and X are then generated as X YU and X YU. The authors showed that with this construction from Y and U i s, the density of (X,X )iseactlyf(, ) ( ( )). This density can also be generalized to higher dimensions through Khintchine s method. In fact, for any p, one generates U i Uniform[, ],i,...,pand Y χ 3, and considers X i YU i,i,...,p. Then each X i is standard normally distributed. By using a p + dimensional transformation with X i YU i and W Y and then integrating out W, we derive the joint density of X (X,...,X p )as f p (,..., p ) p y p e y / dy. (3.) π Note that f p (,...,) for p 3. Nevertheless, f p is a proper p-dimensional density for every p that has standard normal marginal distributions. Since f p is a function of, the isodensity contour of f p consists of concentric hypercubes, which generalizes f (, ) ( ( )). We further show below that f p is the only density that possesses this property of being hypercubically contoured and having marginally normal distributions. Proposition. Consider a p-dimensional density that is a function of, that is, g p (,..., p ) h p ( )forsome differentiable function h p : R + R +. If g p (,..., p ) has standard normal marginal densities, then the unique epression of g p (,..., p )is g p (,..., p ) p y p e y / dy. (3.) π The proof can be found in Appendi B. 4. DISCUSSION: EQUIVALENCE BETWEEN THE UNIFORM CORRELATION MIXTURE AND THE KHINTCHINE MIXTURE IN HIGHER DIMENSIONS? In this article, we have shown the equivalence of three bivariate densities: the uniform correlation miture of bivariate normal densities, the unique square-contoured bivariate density with normal marginals, and the joint density of the Khintchine miture of χ 3 densities. We have also shown the equivalence of hypercubically contoured densities and the Khintchine miture of χ 3 densities in higher dimensions. It thus becomes interesting to investigate whether the uniform correlation miture of bivariate normal densities is equivalent to them in higher dimensions. Intuitively, this equivalence should carry over, as we would just be averaging the ellipsoid-shaped contours of multivariate normal densities instead of elliptical ones. However, directly integrating the multivariate normal density over the uniform measure over positive definite matrices is not a transparent task. Furthermore, if this relationship holds for normal distributions in higher dimensions, one may be curious whether this equivalence holds also for other distributions. The fact f p (,...,) < only for p is also interesting. Due to the relationship between the normal distribution and random walks, it would be interesting to see if the finiteness of f p (,...,) connects with the recurrence or transience of random walks. Due to the natural Bayesian interpretation of this miture, we will also investigate its Bayesian applications. For eample, the Bayesian inference about ρ discussed in this article relies on the fact that the means and variances are known, and the correlation ρ is the only parameter of interest. However, the important case with unknown means and variances (as studied in Geisser 964) needs to be further studied, and it is not clear whether a fully eplicit form of the posterior distribution eists in this case. We will study these problems in future work. APPENDIX A: PROOF OF THEOREM. The derivation of f (, ) consists of the following steps:. To show f (, ) 4 ( ()).. To show that if and are not both s, then the function g(ρ) + ρ is monotone decreasing for ρ ( ρ ) (,a] and is monotone increasing for ρ [a,), where a a(, ) sgn( ). 3. To treat I a f (, ρ)dρ and I a f (, ρ)dρ separately, and to show that + ρ } dρ ( ρ ) 4π ρ ep I + I ( ( )). Step. For, f (, ) 4π ρ dρ 4π arcsin ρ 4 ( ()). (A.) Step. If and are not both s, then consider the derivative of the eponent in f (, ρ): g(ρ) + ρ. ( ρ ) The American Statistician, August 4, Vol. 68, No. 3 85

4 Downloaded by [University North Carolina - Chapel Hill] at 6:4 3 August 4 After some algebra, it can be shown that d dρ g(ρ) ( ρ ) (ρ )(ρ ), (A.) and d d ρ g(ρ) ( + 3ρ ) ( ρ ) 3 ρ(ρ + 3) + ( + 3ρ ) }. (A.3) Therefore, the minimum of g(ρ) is attained at a a(, ) sgn( ). For eample, if >, then a. We should also note that the minimum value of g(ρ) is g(a). (A.4) Step 3. Without loss of generality, we consider the case >. We split the integral into two pieces: f (, ρ)dρ a f (, ρ)dρ + a f (, ρ)dρ I + I. (A.5) We start with I a 4π ρ ep + ρ } dρ. ( ρ ) (A.6) For ρ [a,),g(ρ) is monotone increasing in ρ. Therefore, we consider the transformation y g(ρ). Solving the quadratic equation for ρ yields that (y )( ) + ρ g y (y). (A.7) y Denote (y) (y )(y ). Note that ( + ) d 4y (y). (A.8) dy (y) Thus, by differentiating (A.7)iny, we obtain ( ) dρ + y (y) dy. (A.9) y (y) Moreover, by (A.7), ρ (( ) y + y (y) ). (A.) By applying the last two equations to I, we have ( + ) y (y) I 4 e y dy. πy (y) (A.) A similar argument for I yields that ( + ) y + (y) I 4 e y dy. πy (y) (A.) We show net that ( + ) y + (y) ( ) y ( + ) y (y) + ( ) y. (A.3) This is seen by noting that ( + ) y + (y) ( ) y ( + ) y (y) + ( ) y 4 y ( ) y. (A.4) Now the integral is simplified as f (, ρ)dρ I + I e y dy 4πy y 4πz z e z dz, (A.5) where z y/. By classical Laplace transform results in Erdélyi et al. (954), the last integral is found to be 4πz z e z dz ( ( ) ). (A.6) APPENDIX B: PROOF OF PROPOSITION 3. We shall prove the proposition by induction. For p,, and the marginal density is the density in. Thus g ( ) φ( ) is the unique density that is marginally normal and is a function of. This is the only case where the marginal normality is needed. Suppose (3.) is true for,...p. For p +, write (,..., p ). Suppose g p+ () h p+ ( ) for some differentiable function h p+ : R + R + and has standard normal marginal distributions. 86 Teacher s Corner

5 Downloaded by [University North Carolina - Chapel Hill] at 6:4 3 August 4 Note that if we integrate out p+, that is, g p+ ()d p+ h p+ ( ) + h p+ ( p+ )d p+, (B.) then this resulting density on the right-hand side is a p- dimensional joint density that depends only on. By the uniqueness from the induction hypothesis, h p+ ( ) + p π h p+ ( p+ )d p+ z p e z / dz. Now consider the equation in a positive t that th p+ (t) + t h p+ ( p+ )d p+ (B.) p z p e z / dz. (B.3) π t Differentiating in t on both sides, we get h p+ (t) p π t p e t /. (B.4) Thus, h p+ (y) y (p+) π t (p+) e t / dt + c (B.5) for some constant c. Since lim y h p+ (y), c. Thus the induction proof is complete. [Received December 3. Revised March 4.] REFERENCES Aylward, S., and Pizer, S. (997), Continuous Gaussian Miture Modeling, in Information Processing in Medical Imaging (Vol. 3), eds. J. Duncan and G. Gindi, Berlin, Heidelberg: Springer, pp [83] Barnard, J., McCulloch, R., and Meng, X.-L. (), Modeling Covariance Matrices in Terms of Standard Deviations and Correlations, With Application to Shrinkage, Statistica Sinica,, 8 3. [84] Bryson, M. C., and Johnson, M. E. (98), Constructing and Simulating Multivariate Distributions Using Khintchine s Theorem, Journal of Statistical Computation and Simulation, 6, [83,85] Erdélyi, A., Magnus, W., Oberhettinger, F., and Tricomi, F. G. (954), Tables of Integral Transforms, (Vols. and ), New York: McGraw-Hill. [86] Feller, W. (97), An Introduction to Probability Theory and its Applications, (Vol., nd ed.), New York: Wiley. [83] Geisser, S. (964), Posterior Odds for Multivariate Normal Classifications, Journal of the Royal Statistical Society, Series B, 6, [85] Gelman, A., and Meng, X.-L. (99), A Note on Bivariate Distributions That Are Conditionally Normal, The American Statistician, 45, 5 6. [83] Khintchine, A. Y. (938), On Unimodal Distributions, Izvestiya Nauchno- Issledovatel skogo Instituta Matematiki i Mekhaniki,, 7. [83] Kotz, S., Balakrishnan, N., and Johnson, N. L. (4), Continuous Multivariate Distributions, Models and Applications, New York: Wiley. [83] The American Statistician, August 4, Vol. 68, No. 3 87