Holonomic gradient method for distribution function of a weighted sum of noncentral chi-square random variables

Transcription

1 Holonomic gradient method for distribution function of a weighted sum of noncentral chi-square random variables T.Koyama & A.Takemura March 3,

2 The Ball Probability (1). Let X be a d-dimensional normal random vector with the mean vector µ and the covariance matrix Σ: X N(µ, Σ) Then the probability P (X X2 d r2 ) of the ball with radius r is written as 1 x x2 d r2 (2π) d/2 exp Σ 1/2 ( 1 ) 2 (x µ) Σ 1 (x µ) dx. 2

3 The Ball Probability (2). By rotation, we can assume that Σ is a diagonal matrix without loss of generality. Hence X 2 = X X2 d is the weighted sum of independent noncentral chi-square random variables. The integral for the infinitesimal interval is the Fisher-Bingham integral. r < x x2 d < r + dr In statistical interpretation, the conditional distribution of X given X = r is the Fisher-Bingham distribution. 3

4 Fisher Bingham Integral. The Fisher Bingham Integral is an integral on the sphere S d 1 (r) = { t R d t t2 d = r2} with parameters λ 1,..., λ d, τ 1,..., τ d defined by f(λ, τ, r) = d exp S d 1 (r) i=1 λ i t 2 i + Here, dt is the volume element on S d 1 (r) with d i=1 τ i t i dt. Sd 1(r) dt = rd 1 S d 1, S d 1 = Vol(S d 1 (1)) = 2πd/2 Γ(d/2). 4

5 Related works with the Fisher Bingham Integral [KW] K. Kume, S. G. Walker, On the Fisher-Bingham distribution, Statistics and Computing 19, , (2009) H. Nakayama, K. Nishiyama, M. Noro, K. Ohara, T. Sei, N. Takayama, A. Takemura, Holonomic Gradient Descent and its Application to the Fisher-Bingham Integral, Advances in Applied Mathematics 47, , (2011) [KNNT] T. Koyama, H. Nakayama, K. Nishiyama, and N. Takayama, Holonomic gradient descent for the Fisher-Bingham distribution on the d-dimensional sphere, Computational Statistics, 29, , (2014) T.Sei, A.Kume, Calculating the Normalising Constant of the Bingham Distribution on the Sphere using the Holonomic Gradient Method, Statistics and Computing, (2013) 5

6 The Ball Probability and the Fisher Bingham Integral. Let Σ = diag(σ 2 1,..., σ2 d ), µ = (µ 1,..., µ d ). Put new parameters as λ i := 1 2σi 2, τ i := µ i σi 2. Then the ball probability is written by the Fisher Bingham integral: P (X X2 d r2 ) di=1 λi = exp 1 4 π d/2 d i=1 τi 2 r λ i 0 f(λ, τ, s)ds. 6

7 The Ordinary Differential Equation w.r.t r. Let f(λ, τ, r) be the Fisher Bingham integral. Put F = ( f,..., f, f,..., f ). τ 1 τ d λ 1 λ d Then, the vector F satisfies the ODE[KNNT]: r F = P r F. The matrix P r = (p ij ) is explicitly given by rp ij = (2λ i r 2 + 1)δ ij + d k=1 τ i δ j(k+d) (1 i d), rp (i+d)j = τ i r 2 δ ij + (2λ i r 2 + 2)δ j(i+d) + k i δ j(k+d) (1 i d) for 1 j 2d. 7

8 P r = 1 r 2r 2 x O y 1 y O 2r 2 x d + 1 y d y d r 2 y 1 O 2r 2 x O r 2 y d 1 2r 2 x d + 2 where O denotes off-diagonal block of 0 s and 1 denotes off-diagonal block of 1 s., 8

9 Remark. Since the Fisher Bingham integral f(λ, τ, r) satisfies the equation f(λ, τ, r) = S d 1 (r) 1 r 2(t t2 d ) exp = 1 r 2 ( f λ f λ d ), d i=1 λ i t 2 i + d i=1 τ i t i dt we can obtain the value of f(λ, τ, r) from the value of the vector F. 9

10 Series Expansion of the Fisher Bingham Integral. The Fisher Bingham integral has the following series expansion [KW]: f(λ, τ, r) = r d 1 S d 1 Here, α,β N d 0 S d 1 = r 2 α+β (d 2)!! d i=1 (2α i + 2β i 1)!! (d α + 2 β )!!α!(2β)! λα τ 2β. Sd 1(1) dt = 2πd/2 Γ(d/2) and N 0 = {0, 1, 2,... }. For a multi-index α N d 0, we define α! = d i=1 α i!, α!! = d i=1 α i!!, and α = d i=1 α i. 10

11 Laplace Approximation (1). Suppose σ 1 > σ 2 σ d, i.e., λ 1 > λ 2 λ d, then we have the Laplace approximation for the Fisher Bingham integral as follows: f(λ, τ, r) f(λ, τ, r). Here, we put f(λ, τ, r) = (e rτ 1 + e rτ 1) exp r 2 λ 1 d i=2 π (d 1)/2 1 di=2 (λ 1 λ i ) 1/2. τ 2 i 4(λ i λ 1 ) 11

12 Laplace Approximation (2). Under the same assumption, we have the Laplace approximation for derivatives of the Fisher Bingham integral: f λ 1 r 2 f, f τ 1 r erτ 1 e rτ 1 e rτ 1 + e rτ 1 where j = 2,..., d. f, f λ j f τ j ( τ j 2(λ j λ 1 ) τ j 2(λ 1 λ j ) f, ) (λ 1 λ j ) f, 12

13 Computation of Initial Values. By the power series expansion of the Fisher Bingham integral, we obtain an approximation for 0 < r 1: f = r d+1 τ i + O(r d+2 ) τ i (i = 1,..., d), f = r d+1 + O(r d+2 ) λ i (i = 1,..., d) In our implementation of the HGM, we take r = for computing the initial value. 13

14 HGM for 0 < r 1. When 0 < r 1, we utilize the ordinary differential equation F r = P rf (1) to evaluate the Fisher Bingham integral by HGM. Here, we put ( f F =,..., f, f,..., f ), τ 1 τ d λ 1 λ d and P r is an explicitly given 2d 2d matrix with rational function elements, as shown above. 14

15 Procedure. Compute the value of r (d+1) F for r = by the approximate expression. The reason of computing r (d+1) F is that the values of elements of F is too small for floating point number. Solve the differential equation (1) numerically. In our implementation, we utilize Explicit embedded Runge-Kutta Prince-Dormand (8, 9) method, and we put the accuracy In order to prevent the elements of F becoming too large, we multiply a constant to F several times when we applying Runge- Kutta method. We use different set of standard monomials for small r and large r. 15

16 HGM for 1 r. When parameter r is large, we utilize the following ordinary differential equation: ( ) Q D r = r D 1 (2rλ 1 + τ 1 )I 2d + DP r D 1 Q (2) where ( 1 Q = exp( r 2 f λ 1 r τ 1 ), f,..., f, 1 f r τ 1 τ 2 τ d r 2, f,..., f λ 1 λ 2 λ d I 2d is the identity matrix with size 2d, and we put ), D = diag( 1 r, 1,..., 1, 1 r2, 1,..., 1). By the Laplace approximation, we expect that Q converges to a vector when r. 16

17 Procedure. Compute the value of F for r = 1.0. Compute the value of Q from F. Solve the differential equation (2) numerically. Compute the value of F from Q. Tamio s code will be made open. 17

18 Cumulative Distribution Function. Let d = 3 and σ 1 = 3.0, σ 2 = 2.0, σ 3 = 1.0, µ 1 = 0.01, µ 2 = 0.02, µ 3 = 0.03, (3) i.e., λ 1 = , λ 2 = 0.125, λ 3 = 0.5, τ 1 = , τ 2 = 0.005, τ 3 = By the HGM, we can compute the probability P ( di=1 X 2 i < r 2) for each r. 18

19 Ball Probability Figure 1: A graph of r and P ( di=1 X 2 i < r 2) r 19

20 Ratio of FB Integral and It s Laplace Approximation. When r 1, the Laplace approximation for the Fisher Bingham integral is f(λ, τ, r) (e rτ 1 + e rτ 1) exp r 2 λ 1 π (d 1)/2 1 di=2 (λ 1 λ i ) 1/2. d i=2 τ 2 i 4(λ i λ 1 ) If HGM is numerically accurate enough, the ratio of both sides of the above expression goes to 1 when r goes to infinity. 20

21 The following numerical calculations confirm this conjecture experimentally. For f/ τ i and f/ λ i, we also obtained the Laplace approximations. We check by numerical calculations that the ratio of the differential of f and it s Laplace approximation goes to 1 for r. 21

22 For the parameter (3), the graph of the ratio are the following: (Fisher Bingham Integral)/(Laprace Approximation) r Figure 2: Ratio of FB Integral and It s Laplace Approximation 22

23 The graph of the ratio of differentials are the following: (Differantial of FB)/(Laprace Approximation) df/d\tau_1 df/d\tau_2 df/d\tau_3 df/d\lambda_1 df/d\lambda_2 df/d\lambda_ r Figure 3: Ratio of differentials of FB and It s LA 23

24 Let d be dimension, σ (1) and σ (2) be diagonal matrices. Suppose the diagonal elements of σ (1) and σ (2) are σ (1) d + 1 i = (1 k d) (4) k(k + 1) and σ (2) i = 2(d + 2)(d + 3) k(k + 1)(k + 2)(k + 3) respectively (based on cumulative χ 2 statistic ). Let µ (1) = 0, µ (2) = ( (d 1) ). (1 k d) (5) 24

25 For each d, we computed the probability P ( d i=1 X 2 i < 40.0 ) and measured the computational time. We considered the following four patterns of parameters: (Σ (1), µ (1) ), (Σ (1), µ (2) ), (Σ (2), µ (1) ), (Σ (2), µ (2) ). 25

26 For Σ (1), we obtained dim µ = µ (1) = 0 µ = µ (2) 0 p time(s) p time(s)

27 For Σ (2), we obtained dim µ = µ (1) = 0 µ = µ (2) 0 p time(s) p time(s)

28 In the case where the dimension is large. Let d be the dimension, and parameter σ k = 1 k(k + 1), µ k = 0 (1 k d). This corresponds to Anderson-Darling statistic, where in fact d =. We computed the probability and measured it s computational time. Here, we fixed the radius as r =

29 dim p time(s) Computational time r 29

30 Summary and Discussion. HGM is again amazingly accurate. It is important to re-scale the elements of the standard monomial vector as needed, so that they do not become too small or too large. Asymptotic evaluation is useful for re-scaling. Question: can we start from infinity and integrate in the reverse direction? In this problem it was not practical. 30