arxiv: v1 [math.st] 4 Oct 2018

Transcription

1 arxiv:8.36v [math.st] 4 Oct 8 Gaussian approximation of Gaussian scale mixtures Gérard Letac, Hélène Massam May 6th, 8 Abstract For a given positive random variable V > and a given Z N(,) independentofv, we computethescalar t suchthatthedistance between Z V andz t,inthel (R)sense, isminimal. Wealsoconsider thesame problem in several dimensions. Keywords: Normal approximation, Gaussian scale mixture, lancherel theorem. Introduction Let Z N(,I n ) is standard Gaussian in and consider an independent random positive definite matrix V of order n with distribution µ. We call the distribution of VZ a Gaussian scaled mixture. Denote by f the density in of VZ. For n > several µ can yield the same density f. In many practical circumstances, µ is not very well known, and f is complicated. On the other hand, for n =, histograms of the symmetric density f(x) = e x v () πv look like the histogram of a normal distribution since logf( x) is convex. The aim of the present note is to say something of the best normal approximation N(,t ) of f in the sense of L ( ). In Section, we recall some known facts and examples about the pair (f,µ) when n =. In Section 3, our main result, for n =, is roposition 3. which shows the existence, the uniqueness of t and the fact that t < E(V). This proposition also gives the equation, see (6), that has to be solved to obtain t when µ is known. In Section 4, we consider the more difficult case when n >. In that case, t is a positive definite matrix, and roposition 4. shows the existence of t. A basic tool we use in this note is the lancherel identity. Review of Gaussian scaled mixtures in the uni-dimensional case A probability density f on R is called a discrete Gaussian scale mixture if there exist numbers < v < < v n and p,...,p n > such that p + +p n =

2 and f(x) = n i= p i πvi e x v i. It easy to see that if V n i= p iδ vi is independent of Z N(,) then the density of Z V is f. A way to see this is to observe that for all s R we have e sx f(x)dx = n p i e s vi = E(E(e szv V)) = E(e szv ). i= More generally, we will say that the density f is a Gaussian scale mixture if there exists a probability distribution on (, ) such that () holds. As in the finite mixture case, if V µ is independent of Z N(,) the density of Z V is f. To see this denote Then For instance if a > and if L V (u) = e uv. () e sx f(x)dx = L V ( s /) = E(e szv ). (3) f(x) = a e a x (4) is the double exponential density, then for s < a we have where L V (u) = e sx f(x)dx = a a s = L V( s /) a a +u = a e vu a v dv. This means that the mixing measure is an exponential distribution with mean /a. There are other examples of pairs (f,µ) (Z V,V) in the literature. For instance, almer, Kreutz-Delgado and Makeig() offer an interesting catalog containing also some examples for n >. Note that if f is known then the distribution of logz + logv is known and finding the distribution µ or the distribution of logv is a problem of deconvolution. If its solution exists, it is unique, as shown for instance by (3). An example of such a deconvolution is given by West (987), who extends (4) to f(x) = Ce a x α where < α < as follows: he observes that for A > and < α <, there exists a probability density g, called a positive stable law, such that, for θ >, e tθ g(t)dt = Ce Aθα. If we define = C πg(/v)v 3 dv where C is such that µ is a probability and replace θ by x /, we get, for a = α A, e x v πv = Ce a x α.

3 For α /, the Laplace transform L V is not elementary anymore. Another elegant example of deconvolution is given by Stefanski (99) and Monahan and Stefanski (99) with the logistic distribution f(x) = e x (+e x ) = n= ( ) n+ ne n x. (5) Using (4) to represent ne n x, one can deduce that, if µ exists here, it must be = ( ) n+ n e n v dv n= which indeed exists since this is the Kolmogorov-Smirnov distribution. 3 Normal approximation Such a mixing f keeps some characteristics of the normal distribution: It is a symmetric density, f(x) = e κ(x ) where u κ(u) is convex since e κ(u) = e u/v πv = e uw ν(dw) is the Laplace transform of the positive measure ν(dw) defined as the image of πv by the map u w = /v. As said in the introduction, in some practical applications, the distribution of V is not very well known, and it is interesting to replace f by the density of an ordinary normal distribution N(,t ). The L (R) distance is well adapted to this problem. We are going to prove the following result. roposition 3.. If f is defined by (), then. f L (R) if and only if ( ) E < V +V when V and V are independent with the same distribution µ.. If f L (R), there exists a unique t = t (µ) > which minimizes t I V (t) = [ f(x) πt e x t ] dx. 3. The scalar y = /t the unique positive solution of the equation (+vy) = 3/ 3/. (6) In particular, if µ λ is the distribution of λv, then t = t (µ λ ) = λt (µ). 3

4 4. The value of I V (t ) is ( ( ( ) I V (t ) = E ) E π V +V V +t In particular 5. Finally t E(V). + ) t I λv (t ) = λ I V (t ). (7) roof. Recall that if g L (R) L (R) and if ĝ(s) = eisx g(x)dx, then lancherel theorem says that π ĝ(s) ds = g(x) dx. (8) Furthermore if g L (R) then g L (R) if and only if ĝ L (R). Let us apply (8) first to g = f. From (), we have ˆf(s) = L V (s /). Then ˆf (s)ds = = L V(s /)ds = L(u) du u E(e u(v+v) ) du = πe( ). u V +V This proves statement. of the proposition. To prove., 3. and 4., we apply (8) to g(x) = f(x) πt e x t for which ĝ(s) = L(s /) e ts /. As a consequence and I V (t) = π [ L V (s /) e ts / ] ds = π I V (t) = π Since e tu udu = Γ(3/) (t) 3/ and since L V (u)e tu udu = [ LV (u) e tu] e tu udu. [ LV (u) e tu] du u e u(v+t) udu = Γ(3/) then I V (t) = if and only if (t+v) = 3/ (t) 3/. We can rewrite this equation in t as F(/t) = / 3/ where F(y) = (+vy) 3/. 4 (t+v) 3/,

5 Since < / 3/ <, F() =, lim y F(y) = and F (y) = 3 v <, (+vy) 5/ it follows that I has only one zero t on (, ) and it is easy to see from the sign of I that I reaches its minimum at t. Toshow5, wewillapplyjenseninequalityf(e(x)) E(f(X))totheconvex function f(x) = x 3/ and the random variable X = +y V. From (+y E(V)) 3/ E ( ) (+y V) 3/ it follows that +y E(V) and t = /y < E(V). = 3/ Example. Suppose that r(v = ) = r(v = ) = /. Let us compute t and I(t ). With the help of Mathematica, we see that the solution of (+t) 3/ + (+t) 3/ = (t) 3/ is t = Finally ( I V (t ) = π ) =.9, +t +t t a satisfying result. Example. Suppose that V is uniform on (,) Then t =.36678, I V (t ) =.8. If V is uniform on [,a], then from art 4 of roposition 3., we have t = a Example 3. Suppose that r(v > v) = e v. Then t =.54, I V (t ) =.7. 4 Extension to the Euclidean space Denote by the convex cone of real positive definite matrices of order n. A scaled Gaussian mixture f on is the density of a random variable X on of the form X = V / Z where V µ is a random matrix in independent of the standard random Gaussian variable Z N(,I n ). In this section, we study the conditions that the distribution must satisfy for f to be in L ( ), and we find a Gaussian law N(,t ) which is the closest to f in the L ( ) sense. 4. Non identifiability An important remark is in order: for n >, the measure µ which generates a given f is not unique. 5

6 Example 4. Let p > (n )/ and consider the Wishart distribution µ p with shape parameter p and expectation pi n. Then since e trace(vt) µ p (dv) = (det(i n +t)) p we can claim that e s vs µ p (dv) = (det(i n + ss )) = p (+ s ) p. Similarly, consider X following a gamma distribution with shape parameter p and mean p/. Consider also the distribution µ of V = XI n. Then E(e λx ) = (+ E(e trace(s Vs) ) = λ)p, (+ s ) p. This example shows that µ and µ p generate the same scaled Gaussian mixture distribution. 4. Some integrals for the standard Gaussian distribution We recall here two simple formulas. We use the convention that if s it is written aa a column matrix and s is its transposed matrix and is a row vector. Lemma 4.. Let A. Then e s As ds = (π)n/, deta e s As ss ds = (π)n/ A. deta roof. Without loss of generality, we may assume that A is diagonal, and the proof is obvious in this particular case. 4.3 Existence of the best normal approximation roposition 4.. Let be a probability distribution on the convex cone ofpositivedefinite matricesofordern. Let f(x) deotethe densityofthe random variable X = V / Z of where V µ is independent of Z N(,I n ). Then ( ). f L ( ) if and only if E det < where V and V V+V are independent with the same distribution µ.. For f L ( ), consider the function I defined on by [ t I(t) = f(x) (π)n dett e x t x] dx. Then I reaches its minimum at some t, and this t is a solution in of the following equation in t : (v +t) = t. (9) det(v +t) + n dett 6

7 roof. We have ˆf(s) = e i s,x f(x)dx = E(e i V / Z,s ) = E(e s V s ). () Now using lancherel Theorem and Lemma 4., we prove part. as follows: f (x)dx = ˆf(s) R (π) n ds = ( ) n R (π) n E(e s (V +V )s )ds = n R (π) n/e n det. V +V To prove part, we use lancherel theorem again and obtain I(t) = [ˆf(s) e ] (π) n s ts ds. We then want to show that the minimum of I(t) is reached at some t. Let Then from Lemma 4., I (t) = (π) n/ I(t) ˆf. I (t) = ˆf,g t + g t = ˆf,g t + n/ π n dett where g t (s) = e s ts. We show that is compact. Writing K = {y ;I (y ) } I (y) = ˆf,g y (π) n/ dety, we see that y K, i.e. I (y ) if and only if πn/ I (y). From (), the definition of g t (s) and Lemma 4., we have that I (y) = = (π) n/ E(e s dety ( ) E = dety det(v +y ) For C > let us show that V s )e s y s K = {y ;I (y) C} ( ds = (π) n/ dety E det(in +vy). is compact. Note that K = K for C = π n/ /. Since I is continuous, K is closed. Let us prove that K is bounded. Denote y = (tracey ) /. Suppose that y (k) K is such that y (k) k and let us show that for such a y (k), I (y (k) ), which is a contradiction. Indeed, trace(vy (k) ) k if v. To see this, assume that v = diag(v,...,v n ). Then ) ds e s (V +y )s trace(vy (k) ) = v y (k) + +v ny (k) nn trace(y (k) ) min i v i y (k) min i v i k. 7

8 Moreover, if (λ...,λ n ) are the eigenvalues of vy (k), det(i n +vy (k) ) = (+λ )...(+λ n ) +λ + +λ n = +trace(vy (k) ) k By dominated convergence, it follows that I (y (k) ) k and this proves that K is bounded. We have therefore shown that K is compact. This proves that the minimum of I (t) and I(t) is reached at some point t of. The last task is to show that t is a solution of equation (9). Since I(t) is differentiable and reaches its minimum on the open set, the differential of I(t) must cancel at t. Denote by S the linear space of symmetric real matrices of dimension n equipped with the scalar product s,s = trace(ss ). The differential of I is the following linear form on S h I (t)(h) = (π) n/ The equality I (t) = is equivalent to ˆf(s)e s ts ss ds = [ˆf(s) e s ts ] e s ts s hsds. e s ts ss ds. Using the second formula in Lemma 4. and the fact that ˆf(s) = E(e s Vs ), we obtain (v +t) det(v +t) = (t) det(t) = + n t dett. Comment. While it is highly probable that the value t at which I(t) reaches its minimum is unique, it is difficult to show for n > that the complicated equation (9) has a unique solution: there is no reason to think that the function t I(t) is convex. This is not the case for n =. 5 References Monahan, J. F. and Stefanski, L. A. (99). Normal Scale Mixture Approximations to F (z) and Computation of the Logistic-Normal Integral, in Handbook of the Logistic Distribution, N. Balakrishnan, Ed., Marcel Dekker, New York. almer, J.A, Kreutz-Delgado, K. and Maleig, S. () Dependency models based on generalized Gaussian scale mixtures. DRAFT UCSD-SCCN v., Sept 7. Stefanski, L. A. (99). A Normal Scale Mixture Representation of the Logistic Distribution, Statistics & robability Letters, West, M. (987) On scale mixture of normal distributions Biometrika 74, 3,