Use of Asymptotics for Holonomic Gradient Method

Transcription

1 Use of Asymptotics for Holonomic Gradient Method Akimichi Takemura Shiga University July 25, 2016 A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 1 / 30

2 Some disclaimers In this talk, I consider numerical aspects of HGM (Holonomic Gradient Method). I will only discuss some examples (no general theory). A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 2 / 30

3 Outline 1 Map of topics for HGM and asymptotic methods 2 A simple example of univariate normal distribution 3 HGM for ball probability under multivariate normal distribution. This part is based on the following paper: Holonomic gradient method for distribution function of a weighted sum of noncentral chi-square random variables. Computational Statistics. Tamio Koyama and A.T A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 3 / 30

4 Holonomic function and Pfaffian system Let f (x 1,..., x n ) be holonomic. Let i = / x i be the differential operator w.r.t. x i, i = 1,..., n. There exists some finite set F of lower-order derivatives ( standard monomials ) of f F = (f, 1 f, 2 f,..., 2 1f,... ) (column vector) such that i F = Q i F, i = 1,..., n (1) where Q i is n n matrix of rational functions in x 1,..., x n. (1) is called the Pfaffian system. A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 4 / 30

5 Holonomic function and Pfaffian system Each i F = Q i F is an ordinary differential equation for the x i -axis. Once the Pfaffian system is obtained, we can solve F by a standard ODE solver along any path, by the chain rule. Any path is OK theoretically, but with floating number computations we need to choose an appropriate path. A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 5 / 30

6 A figure of topics for HGM asymptotic evaluation chambers singularity 0 initial values by series A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 6 / 30

7 Numerical issues for asymptotics As we integrate F towards infinity (in certain direction), usually some elements of F diverges to infinity (or converges to zero, where we lose precision). This causes trouble, because with computer we work with floating point numbers with limited precision. The way F diverges should depend on how we approach the infinity. In order to keep elements of F within the precision of floating point numbers, we should re-scale elements of F as needed. We need theoretical evaluation of asymptotics for the elements of F. A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 7 / 30

8 Asymptotics abounds in statistics literature In statistics, asymptotic approximation has been one of the major subjects for research. Most results are based on the central limit theorem, where the sample size goes to infinity. For HGM, the Laplace method is more useful, because the function f is often defined by integration as a parameterized integral. Laplace method approximates an integral by the contribution of the integrand around its maximum. A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 8 / 30

9 A simple example of univariate normal distribution This is a simple example, where HGM is not really needed. I use it just for the purpose of illustration. Let φ(x) = 1 2π e x2 2 denote the density of the standard normal distribution N(0, 1). Let Φ(x) = x φ(u)du denote the upper probability of N(0, 1). The behavior of Φ(x) is not trivial. The ratio m(x) = Φ(x)/φ(x) is called the Mills ratio. 1 1 sometime written as Mill s ratio, but Mills ratio is more prevalent. A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 9 / 30

10 Mills ratio Integration by parts gives the following asymptotics (as x ). Φ(x) = = = How about differential equation? 1 x u uφ(u)du [ 1 ] u φ(u) u=x x ( 1 x 1 ) x 3 + o(x 3 ) 1 u 2 φ(u)du φ(x). Because φ (x) = xφ(x), we have Φ (x) = x Φ (x). A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 10 / 30

11 Differential equation of the upper probability Numerically solving differential equation for φ(x) or Φ(x) is probably not good for large x: φ(x + x) φ(x) xφ(x) (x) = (1 x x)φ(x). We need to take x smaller and smaller as x gets larger. Since we know the asymptotic behavior of Φ(x), let us re-scale Φ(x) as f (x) = x 1 φ(x) Φ(x) = x 2πe x2 /2 Φ(x). Then f (x) 1 1 x 2, f (x) = O(x 3 ). A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 11 / 30

12 Re-scaling the upper probability We can write f (x) as f (x) = 1 φ(x) Φ(x) + x 2 2πe x2 /2 Φ(x) x = 1 f (x) + xf (x) x. x Divide both sides by x and further differentiate 2 to obtain 1 x 2 f (x) + 1 x f (x) = 2 x 3 f (x) + 1 x 2 f (x) + f (x). Multiplying by x gives f (x) = 2 x 2 f (x) + ( 2 x + x)f (x). 2 I am not sure if this is a reasonable thing to do or not. A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 12 / 30

13 In the Pfaffian form ( ) ( f (x) 0 1 x f = (x) 2 x 2 2 x + x ) ( ) f (x) f (x) Numerically solving this differential equation should be OK, because xf (x) 0 as x. Question: can we do something like the following? ( ) ( ) ( ) f (x) 0 1 x f = x f (x) (x) xf (x) x 2 x 2 But the column vectors on both sides are not the same... A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 13 / 30

14 Question in more general form So far we only considered re-scaling of the function f itself. Given the set of standard monomials F and the asymptotic behavior of the elements of F, can we re-scale each element of F differently? A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 14 / 30

15 HGM for ball probability under multivariate normal distribution Let X be a d-dimensional normal random vector with the mean vector µ and the covariance matrix Σ: X N(µ, Σ) Then the probability P(X X d 2 r 2 ) of the ball with radius r is written as 1 x1 2+ +x2 d r 2 (2π) d/2 exp Σ 1/2 ( 1 ) 2 (x µ) Σ 1 (x µ) dx. A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 15 / 30

16 Ball probability By rotation, we can assume that Σ is a diagonal matrix without loss of generality. Hence X 2 = X X 2 d is the weighted sum of independent noncentral chi-square random variables. The integral for the infinitesimal interval is the Fisher-Bingham integral. r < x x 2 d < r + dr In statistical interpretation, the conditional distribution of X given X = r is the Fisher-Bingham distribution. A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 16 / 30

17 Fisher Bingham Integral The Fisher Bingham Integral is an integral on the sphere S d 1 (r) = { t R d t t2 d = r 2} with parameters λ 1,..., λ d, τ 1,..., τ d defined by ( d ) d f (λ, τ, r) = exp λ i ti 2 + τ i t i dt. S d 1 (r) i=1 Here, dt is the volume element on S d 1 (r) with i=1 S d 1 (r) dt = r d 1 S d 1, S d 1 = Vol(S d 1 (1)) = 2πd/2 Γ(d/2). A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 17 / 30

18 The Ball Probability and the Fisher Bingham Integral Let Σ = diag(σ 2 1,..., σ 2 d ), µ = (µ 1,..., µ d ). Put new parameters as λ i := 1 2σi 2, τ i := µ i σi 2. Then the ball probability is written by the Fisher Bingham integral: P(X Xd 2 r 2 ) d ( i=1 λi 1 = exp 4 π d/2 d i=1 ) τi 2 r f (λ, τ, s)ds. λ i 0 A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 18 / 30

19 Pfaffian for ball probability Let f (λ, τ, r) be the Fisher Bingham integral. Put ( f F =,..., f, f ) f,...,. τ 1 τ d λ 1 λ d Then, the vector F satisfies the differential equation r F = P r F. (Pfaffian system) The matrix P r = (p ij ) is explicitly given by d rp ij = (2λ i r 2 + 1)δ ij + τ i δ j(k+d) (1 i d), k=1 rp (i+d)j = τ i r 2 δ ij + (2λ i r 2 + 2)δ j(i+d) + δ j(k+d) (1 i d) k i for 1 j 2d. A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 19 / 30

20 Pfaffian for ball probability P r = 1 r 2r 2 x O y 1 y O 2r 2 x d + 1 y d y d r 2 y 1 O 2r 2 x O r 2 y d 1 2r 2 x d + 2, where O denotes off-diagonal block of 0 s and 1 denotes off-diagonal block of 1 s. A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 20 / 30

21 Laplace approximation Suppose σ 1 > σ 2 σ d, i.e., λ 1 > λ 2 λ d, then we have the Laplace approximation for the Fisher Bingham integral as follows: Here, we put f (λ, τ, r) f (λ, τ, r). f (λ, τ, r) = (e rτ 1 + e rτ 1 ) exp ( r 2 λ 1 d i=2 π (d 1)/2 1 d i=2 (λ 1 λ i ) 1/2. ) τi 2 4(λ i λ 1 ) A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 21 / 30

22 Laplace approximation for the derivatives Under the same assumption, we have the Laplace approximation for derivatives of the Fisher Bingham integral: { ( f r 2 f ) } f τ 2 j 1, + f, λ 1 λ j 2(λ j λ 1 ) 2(λ 1 λ j ) f τ 1 r erτ1 e rτ1 e rτ 1 + e rτ 1 f, where j = 2,..., d. f τ j τ j 2(λ 1 λ j ) f, A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 22 / 30

23 Switching Pfaffian at r = 1. When 0 < r 1, we utilize the original differential equation F r = P r F When r > 1, we utilize the following differential equation: Q r = ( D r D 1 (2rλ 1 + τ 1 )I 2d + DP r D 1 ) Q (2) where ( 1 Q = exp( r 2 f λ 1 r τ 1 ), f,..., f, 1 f r τ 1 τ 2 τ d r 2, f f,..., λ 1 λ 2 λ d I 2d is the identity matrix with size 2d, and D = diag( 1 1, 1,..., 1,, 1,..., 1). r r 2 This is re-scaling by the Laplace approximation, which was essential. A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 23 / 30 )

24 Numerical results Let d = 3 and σ 1 = 3.0, σ 2 = 2.0, σ 3 = 1.0, µ 1 = 0.01, µ 2 = 0.02, µ 3 = 0.03, (3) i.e., λ 1 = , λ 2 = 0.125, λ 3 = 0.5, τ 1 = , τ 2 = 0.005, τ 3 = ( d ) By the HGM, we can compute the probability P i=1 X i 2 < r 2 for each r. A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 24 / 30

25 Numerical results Ball Probability r ( d ) Figure 1: A graph of r and P i=1 X i 2 < r 2 A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 25 / 30

26 Ratio of FB Integral and It s Laplace Approximation When r 1, the Laplace approximation for the Fisher Bingham integral is ( ) d f (λ, τ, r) (e rτ 1 + e rτ 1 ) exp r 2 τi 2 λ 1 4(λ i λ 1 ) π (d 1)/2 1 d i=2 (λ 1 λ i ) 1/2. i=2 If HGM is numerically accurate enough, the ratio of both sides of the above expression goes to 1 when r goes to infinity. The following numerical calculations confirm this conjecture experimentally. A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 26 / 30

27 Ratio of FB Integral and It s Laplace Approximation For the parameter (3), the graph of the ratio are the following: (Fisher Bingham Integral)/(Laprace Approximation) Figure 2: Ratio of FB Integral and It s Laplace Approximation r A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 27 / 30

28 Ratio of FB Integral and It s Laplace Approximation The graph of the ratio of differentials are the following: (Differantial of FB)/(Laprace Approximation) df/d\tau_1 df/d\tau_2 df/d\tau_3 df/d\lambda_1 df/d\lambda_2 df/d\lambda_ r Figure 3: Ratio of differentials of FB and It s LA A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 28 / 30

29 Observations from the numerical results HGM was fast and very accurate in this problem. It is important to re-scale the elements of the standard monomial vector as needed, so that they do not become too small or too large. Asymptotic evaluation is useful for re-scaling. Question: can we start from infinity and integrate in the reverse direction? In this problem it was not practical. A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 29 / 30

30 Summary I showed a map of topics for HGM and the importance of asymptotic methods. I believe that HGM sits in the middle of all kinds of asymptotics. It is important to incorporate the asymptotic results in the implementation of HGM, because the input to the program can be an extreme case. Obviously then people can use asymptotic approximation, but HGM should give more accurate results. I my personal memos, I have some scattered asymptotic formulas, which was not used for HGM in our published papers, because HGM works well enough for moderate input values. A.Takemura (Shiga U) HGM and asymptotics 2015/3/3 30 / 30