Holonomic Gradient Method for Multivariate Normal Distribution Theory. Akimichi Takemura, Univ. of Tokyo August 2, 2013

Transcription

1 Holonomic Gradient Method for Multivariate Normal Distribution Theory Akimichi Takemura, Univ. of Tokyo August 2, 2013

2 Items 1. First example (non-central t-distribution, by T.Sei) 2. Principles of HGM for normal distribution theory 3. Orthant probability 4. Wishart distribution and hypergeometric function of a matrix argument 5. Sum of weighted independent non-central χ 2 (1) variables (in preparation) 1

3 Joint works with many people, in particular Tamio Koyama, T.Sei, N.Takayama. Orthant probability discussed in Koyama and T. arxiv: (Submitted) Wishart discussed in arxiv: v3. Published in Journal of Multivariate Analysis. Our first paper (N 3 OST 2 ): Nakayama-Nishiyama-Noro-Ohara-Sei- Takayama-Takemura: Holonomic gradient descent and its application to the Fisher-Bingham integral, Advances in Applied Mathematics, 47, ,

4 The first example: Non-central t distribution by T.Sei X: χ 2 variable with k degrees of freedom Y : N(λ, 1) variable Let T = Y/ X/k. Compute Pr(T q). From standard textbooks (e.g. Johnson-Kotz Vol.2), Pr(T q) is usually expressed as an infinite series with each term involving incomplete beta function. 3

5 Regard q and λ as variables (and leave k fixed). Let F (q, λ) = Pr(T q). Our strategy: derive partial differential equations satisfied by F. Then integrate the differential equation to evaluate F. We call this method Holonomic Gradient Method (HGM). 4

6 We can obtain three partial differential equations: λf 2 = q q F λ λ F (1) ( ) qλ q 2 2 k q(k + 1) F = (k + q 2 ) 2 k + q 2 q F λk2 (k + q 2 ) λf (2) 2 q λ F = λk k + q qf qk 2 k + q λf (3) 2 Consider any higher-order derivative i λ j qf of F. Repeatedly applying the above three partial differential equations, any i λ j qf can be reduced to a rational function combination of q F, λ F and F. 5

7 The Pfaffian system for F (q, λ) is given as F F q q F = qλ 0 2 k q(k+1) λk2 (k+q 2 ) 2 k+q 2 (k+q 2 ) 2 q F λ F λ F λ 0 λk k+q 2 qk k+q 2, (4) F F q F = λk qk 0 k+q 2 k+q 2 q F. (5) λ F 0 q λ λ F (there is no singularity in this Pfaffian since k > 0) 6

8 With initial values ( q F )(0, 0) = 1 Γ( k+1 kπ Γ( k), 2 ( λ F )(0, 0) = 1 2π, 2 ) (4) and (5) can be solved by standard numerical solver for differential equations. 7

9 Maybe our first example is not very convincing. The standard method already works OK. The point is that our method is general. There is a general theory and algorithm to obtain the partial differential equations (1) (3) and the Pfaffian system (4), (5), based on Gröbner bases theory for D modules. In this example, although we derived (1)-(5) by hand, it is very important that the general theory tells us what to expect. 8

10 Principles of HGM for normal distribution theory Let X 1,..., X n be i.i.d. random vectors from the multivariate normal distribution N m (µ, Σ). Let R(q) R m n be a parameterized region. Problem: compute F (q, µ, Σ) def = Pr((X 1,..., X n ) R(q)) exp( 1 n (X i µ) T Σ 1 (X i µ))dx 1... dx n R(q) 2 i=1 9

11 FACT: If the family of indicator functions I A(θ) is nice (such as polyhedral region, region defined by polynomial inequalities), i.e., if they are holonomic as Schwartz distributions, then F (q, µ, Σ) is holonomic, i.e., it satisfies a system of partial differential equations with rational function coefficients. This fact means, that the whole sampling distribution theory under the multivariate normal distribution can be treated by HGM. Total rewriting of the sampling theory is in order. 10

12 Orthant probability Let R m + = {x R m x i 0} be the positive orthant. Compute Φ m (Σ, µ) exp ( 1 ) 2 (z µ)t Σ 1 (z µ) dz R m + Define x = 1 2 Σ 1, y = Σ 1 µ and ( m g(x, y) = exp x ij t i t j + R m + i,j=1 m i=1 y i t i ) dt. 11

13 For a subset J of [m] = {1,..., m} let ( g J (x, y) = exp x ij t i t j + ) y k t k dt j. R J + i J j J k J j J Furthermore for J = {j 1,..., j s } write x J = (x jk j l ) 1 k,l s, y J = (y j1,..., y js ) T Σ J = 1 2 x 1 J = (σ J ij), µ J = Σ J y J = (µ J j 1,..., µ J j s ). 12

14 Then g J (x, y) satisfies µ J i g J + j J yi g J = σj ijg J\{j} i J 0 i / J 2 yi yj g J {i, j} J, i < j xij g J = y 2 i g J {i} J, i = j 0 else. (6) (7) These are basically the same as the recurrence relations by Plackett (1954). But our recurrence relations are more suitable for HGM. 13

15 Numerical performance of HGM is good: Table 1: Averages of computational times dim Miwa HGM dim Miwa HGM

16 Wishart distribution and hypergeometric function of a matrix argument W : m m symmetric positive definite (W > 0) W = n i=1 X ix T i, X i N m (0, Σ), i.i.d. Density of Wishart distribution: f(w ) = C n m 1 W 2 Σ n 2 exp( 1 2 trw Σ 1 ) C is known (containing gamma functions). 15

17 l 1 : the largest root of W We want to evaluate the probability Pr(l 1 < x). l 1 < x W < xi m, where I m : m m is the identity matrix Hence the probability is given in the incomplete gamma form: Pr(l 1 < x) = C n m 1 2 W 0<W <xi m Σ n 2 exp( 1 2 trw Σ 1 )dw 16

18 Pr(l 1 < x) is written as C exp ( x ) 2 trσ 1 x 1 2 nm 1F 1 ( m ; n + m + 1 ; x ) 2 2 Σ 1 Hypergeometric function of a matrix argument (Herz(1955)): Γ m (c) 1F 1 (a; c; Y ) = exp(trxy ) Γ m (a)γ m (c a) 0<X<I m X a (m+1)/2 I m X c a (m+1)/2 dx, where Γ m (a) = π 1 4 m(m 1) m i=1 Γ ( a i 1 ). 2 17

19 1 F 1 (a; c; Y ) is a symmetric function of characteristic roots of Y its series expression is written in terms of symmetric polynomials. Zonal polynomials (A.T.James) C κ (Y ), κ k homogeneous symmetric polynomial of degree k in the characteristic roots of Y. 18

20 Series expansion of 1 F 1 (Constantine(1963)) 1F 1 (a; c; Y ) = k=0 κ k (a) κ C κ (Y ). (c) κ k! This is a beautiful mathematical result. However for numerical computation, zonal polynomials have enormous combinatorial difficulties and statisticians pretty much forgot zonal polynomials. 19

21 The partial differential equations satisfied by F (Y ) = 1 F 1 (a; c; y 1,..., y m ) were obtained by Muirhead(1970). g i F = 0, i = 1,..., m, where g i = y i 2 i + (c y i ) i j i y j y i y j ( i j ) a. We can derive Pfaffian system from these equations. 20

22 Can we use this PDE for numerical computation? (People never tried this for 40 years). Works! HGM works very well up to dimension m = 10. Plot of the cumulative distribution: 1.2 by hg

23 Sum of weighted non-central χ 2 (1) variables Let X N m (µ, Σ). Problem: compute Pr( X r), where X = (X Xm) 2 1/2 is the Euclidean norm. The distribution of X 2 is the distribution of the sum of weighted independent non-central χ 2 (1) variables. 22

24 Given r = X, the conditional density of X on is proportional to S m 1 (r) = {x x = r} exp ( 1 2 (x µ)t Σ 1 (x µ) ) It is the Fisher-Bingham distribution on S m 1 (r) (cf. our first paper N 3 OST 2.) Hence we can just integrate the holonomic system given in N 3 OST 2 w.r.t. r and evaluate Pr( X r). Directional statistics multivariate normal 23

25 Summary HGM is very relevant for the whole sampling distribution theory under the multivariate normal distribution. HGM is practical if we implement it efficiently. HGM is general and can be applied to many problems. We stand at the beginning of applications of D-module theory to statistics! 24