Holonomic gradient method for hypergeometric functions of a matrix argument Akimichi Takemura, Shiga University

Transcription

1 Holonomic gradient method for hypergeometric functions of a matrix argument Akimichi Takemura, Shiga University (joint with H.Hashiguchi and N.Takayama) June 20, 2016, RIMS, Kyoto

2 Items 1. Summary of multivariate normal distribution theory 2. Definition of hypergeometric functions of a matrix argument 3. Largest root of a Wishart matrix and 1 F 1 4. Holonomic Gradient Method (HGM) for 1 F 1 5. Restriction to diagonal region 6. Largest root of ratio of two Wishart matrices and 2 F 1 7. HGM for 2 F 1 1

3 Main references Holonomic gradient method for the distribution function of the largest root of a Wishart matrix. Journal of Multivariate Analysis, 117, Hashiguchi, Numata, Takayama and Takemura Distribution of ratio of two Wishart matrices and evaluation of cumulative probability by holonomic gradient method. Hashiguchi, Takayama and Takemura. In preparation. 2

4 1. Summary of multivariate normal distribution theory Sampling distributions under multivariate normal distribution are mostly holonomic a : The density f(x; µ, Σ) is holonomic in the random variables and the parameters Probability of regions defined by polynomial inequalities is holonomic w.r.t. parameters. a satisfies linear differential equations with rational function coefficients 3

5 Various standard results and formulas under multivariate normality should satisfy differential equations. The differential equations are basic features of the standard formulas. Let us obtain differential equation to each standard result. This also helps numerically via HGM. Why multivariate normal? 4

6 There are many discussions on the role of multivariate normality (Gaussianity), but it is still of basic importance and useful as ingredients. One reason may be that Gaussianity scales. Density of multivariate normal distribution N (µ, Σ). Let x = (x 1,..., x ) R. f(x; µ, Σ) = 1 (2π) /2 Σ 1/2 exp( 1 2 (x µ) Σ 1 (x µ)). µ = 0: central vs. µ 0: non-central 5

7 The distinction of central vs. non-central usually refers to the mean vector. In this talk, I consider the central case µ = 0. Noncentral cases will be discussed in the talks by Siriteanu and Vidunas on Friday. They are more difficult, even with Σ = I. When the inference on the covariance matrix Σ is considered, let us call Σ = I the null case and Σ I the non-null case. I consider the non-null case. Ultimate target: µ 0, Σ I. 6

8 Complex normal distribution There is a complex version of the normal distribution CN (µ, Σ), common in communication theory. Let x C f(x; µ, Σ) = 1 π Σ exp( (x µ) Σ 1 (x µ)) In contrast to the real case, where half integers appear, we only have integers in the complex case. The complex case is easier. Siriteanu and Vidunas consider the complex case. 7

9 Wishart distribution Let X : n m, where n m. Let the rows of X be independent normal random vectors with common covariance matrix Σ. Let W = X X : m m positive definite with probability 1. sample covariance matrix, sum of squares and cross products matrix, Gram matrix. The distribution of W is the Wishart distribution with n degrees of freedom. 8

10 Let M = E(X). If M 0, then the distribution of W is non-central Wishart, otherwise it is central Wishart. Notation for non-central Wishart: W (n; Σ, Ψ), Ψ = Σ 1 M M. Notation for central Wishart: W (n; Σ). 9

11 Density of the central Wishart W (n; Σ). f(w ) = where 1 2 /2 Γ ( 2 ) Σ /2 W ( 1)/2 exp( 1 2 trσ 1 W ), Γ (a) = π ( 1)/4 1 =ى Γ(a i 1 2 ) is called the multivariate Gamma function. f(w ) is w.r.t. ي ى dw يى and it is only positive for W > 0. Non-central density involves 0 F 1 and complicated. 10

12 Two-sample problem When we consider a single Wishart matrix W, then the problem is one-sample. When we consider two independent Wishart matrices W 1 W (n 1, Σ 1 ), W 2 W (n 2, Σ 2 ), then the problem is two-sample. Then null case in the two-sample problem is Σ 1 = Σ 2 ( equality of covariance matrices ). 1 F 1 appears in the one-sample case and 2 F 1 appears in the two-sample case. (This was not totally clear to me until half a year ago.) 11

13 2. Definition of hypergeometric functions of a matrix argument Three definitions: 1) integral, 2) series, 3) differential equations. Recursive integral definition by Herz (1955). symmetric : س.(سexp(tr = (س) 0 ء 0 ) ش ; q ق... 1 ق ;ف p ف... 1 ف) q ءp+1 = 1 سل( شس ; q ق... 1 ق ; p ف... 1 ف) q ءp a (m+1)/2 س trs م Γ m (ف) S>0 (س ;ق q ق... 1 ق ; p ف... 1 ف) q+1 ءp (ق) = 2m(m 1)/2 Γ m m(m+1)/2 (ىل 2 ).شل(س 1 ش ; q ق... 1 ق ; p ف... 1 ف) q ءp b ش trt م R(T )=X 0 12

14 Particular cases 1F 1 (a; b; S) = 2F 1 (a 1, a 2 ; b; S) = Γ (b) Γ (a)γ (b a) ة 0 exp(trst ) T 1)/2 + ) ف I T 1)/2 + ) ف ق dt Γ (b) Γ (a 1 )Γ (b a 1 ) ة 0 2 ف ST I T ف 1 ( +1)/2 I T ف ق 1 ( +1)/2 dt We can take these as definitions of 1 F 1 and 2 F 1. These are not trivial from the recursive integral definition. In fact, reconsideration of integral formula for 2 F 1 led to our new result. 13

15 Series expansion in terms of zonal polynomials 1 F 1 (a; c; Y ) is a symmetric function of characteristic roots of Y its series expression is written in terms of symmetric polynomials. Zonal polynomials (A.T.James) C κ (Y ), κ k homogeneous symmetric polynomial of degree k in the characteristic roots of Y. 14

16 Series expansion of 1 F 1 (Constantine(1963)) (a) κ C κ (Y ) 1F 1 (a; c; Y ) =. (c) κ k! More generally =0 κ = (س ; q ق... 1 ق ; p ف... 1 ف) q ءp k=0 κ k ) ع) p ) κ C κ ف)... κ ) 1 ف).! q ) κ ق)... κ ) 1 ق) Zonal polynomials are beautiful mathematical objects with group representation background. However for numerical computation, zonal polynomials have enormous combinatorial difficulties and statisticians pretty much forgot them. 15

17 Differential equation by Muirhead The partial differential equation satisfied by F (Y ) = 1 F 1 (a; c; y 1,..., y ) was obtained by Muirhead(1970). g ى F = 0, i = 1,..., m, where ى ) ى (c y + ى 2 ى = y ى g ى ي ي y ي y ى y a. ) ي ى ( He also obtained partial differential equation satisfied by 2 F 1 (shown later). 16

18 With suitable regularity conditions and initial values a, these differential equations determine 1F 1 and 2 F 1. Hence they can also be regarded as definitions. Can we use this PDE for numerical computation? (People never tried this for 40 years). Works! HGM works well up to dimension m = 20 for 1 F 1. a more precisely analytic at the origin, symmetric in y 1,..., y m, and its value is 1 at the origin. 17

19 3. Largest root of a Wishart matrix and 1 F 1 W : m m Wishart matrix. l 1 : the largest root of W We want to evaluate the probability Pr(l 1 < x). l 1 < x W < xi, where I : m m is the identity matrix 18

20 Hence the probability is given in the incomplete gamma form: Pr(l 1 < x) = C n m 1 2 W m ة ڤ طڤ 0 Σ n 2 exp( 1 2 trw Σ 1 )dw From general theory Pr(l 1 < x) is holonomic. Just as in dim=1, Pr(l 1 < x) is written as C exp ( x ) 2 trσ 1 x 1 2 1F 1 ( m ; n + m + 1 ; x ) 2 2 Σ 1 19

21 4. HGM for 1 F 1 We illustrate HGM for m = 2. Two partial differential equations [ g 1 F = y (c y 1 ) y 2 2 g 2 F = [ y (c y 2 ) ] ( 1 2 ) a F = 0, y 1 y 2 y ] 1 ( 2 1 ) a F = 0. y 2 y 1 Let us compute higher-order derivative from these equations. 20

22 Divide the second equation by y 2 and write ( c F = y y 1 y 2 (y 2 y 1 ) ( 2 1 ) + a y 2 ) F. The RHS becomes messy, but an important fact is that the number of differentiations is reduced by 1. We can reduce the number of differentiations as long as there are more than 1 differentiations with respect to each variable. 21

23 This implies that all higher-order derivatives can be written as a rational function combination of the following 4 square-free mixed derivatives: F (Y ), 1 F (Y ), 2 F (Y ), 1 2 F (Y ). In algebraic terminology, let K = C(y 1, y 2 ) the field of rational functions and let R = K 1, 2 = C(y 1, y 2 ) 1, 2 be the ring of differential operators. 22

24 Theorem 1 {g 1, g 2 } is a Gröbner basis w.r.t. graded lexicographic term order and {1, 1, 2, 1 2 } is the set of standard monomials. (This theorem holds for general dimension.) Hence we only keep F (Y ), 1 F (Y ), 2 F (Y ), 1 2 F (Y ) in memory. We can always compute higher-order derivatives from these 4 values. 23

25 For example 1 2 2F = a 2y 2 (y 2 y 1 ) F + ( (y 2 y 1 ) 2 + a y 2 1 (y 2 y 1 ) 2 2F ( c y 2 y c y 1 ) 1 F 2y 2 (y 2 y 1 ) y 1 ) 1 2 F y 2 (y 2 y 1 ) = h (1 2) 00 F + h (1 2) 10 1 F + h (1 2) 01 2 F + h (1 2) F. (Actually this is all we need for HGM) 24

26 5. Restriction to diagonal region We have so far assumed non-diagonal region. ي y ى y On the diagonal y ى = y ي, the PDE is singular: ى ) ى (c y + ى 2 ى = y ى g ى ي ي y ي y ى y a. ) ي ى ( Let m = 2. Consider letting y 1 y 2 in [ y (c y 1 ) y ] 2 ( 1 2 ) a F = 0. 2 y 1 y 2 25

27 We apply l Hospital s rule to 1 2 y 1 y 2. L Hospital s rule results in 1 2 lim = , 1 2 y 1 y 2 where y 2 is regarded fixed and we let y 1 y 2. 26

28 After applying L Hospital s rule several times, we can show that f(y) = F (y, y) satisfies the following ODE: y 8 f (y) + (c 1 y) ( 3 8 f (y) + c y 4y f (y) a 2y f(y)) f (y) a 2 f (y) = 0. Actually this computation can be performed by Oaku s restriction algorithm(1997) of a holonomic ideal. 27

29 The following asir program import( names.rr )$ import("nk_restriction.rr")$ dp_gr_print(1)$ dp_ord(0)$ G1=y1*dy1^2 + (c-y1)*dy1+(1/2)*(y2/(y1-y2))*(dy1-dy2)-a; G1=red((y1-y2)*G1); G2=base_replace(G1,[[y1,y2],[y2,y1],[dy1,dy2],[dy2,dy1]]); F=base_replace([G1,G2],[[y1,y],[y2,y+z2],[dy1,dy-dz2],[dy2,dz2]]); A=nk_restriction.restriction_ideal(F,[z2,y],[dz2,dy],[1,0] param=[a,c]); end$ outputs the following, which coincides with the by-hand computation! -y^2*dy^3+(3*y^2+(-3*c+1)*y)*dy^2+(-2*y^2+(4*a+4*c-2)*y -2*c^2+2*c)*dy-4*a*y+(4*c-4)*a In hindsight, this program (Oaku s algorithm) worked only for m = 2, 3. 28

30 Clear the denominator and consider g ى ي = ى (y ى y ي ) g ى, i = 1,..., m. Conjecture in 2013: g 1,..., g generate a holonomic ideal in C y 1,..., y, 1,...,. True for m 3, but not true for m 4. This was a very surprising result, after our 2013 paper. L Hospital s rule has been implemented by Masayuki Noro and he now computed all blocking patterns up to m = 20. He will present his result in ISSAC

31 6. Largest root of ratio of two Wishart matrices and 2 F 1 Let W 1 W (n 1, Σ 1 ), W 2 W (n 2, Σ 2 ), be two independent Wishart matrices. We want to compute the distribution function of the largest root l 1 of W 1 W 1 2. Distribution of l 1 (W 1 W 1 2 ) was known by Chikuse (1977) in terms of 2 F 1. However we have derived a stronger result in a matrix form. 30

32 Let W 1/2 2 denote the unique positive definite square root of W 2 and define U = W 1/2 2 W 1 W 1/2 2 Let Σ 1 = I without loss of generality. Result: U has the density f(u) = 1+ 2 Γ ( ) 2 Γ ( 1 2 )Γ ( 2 2 ) I + Σ 2U ( 1+ 2)/2 U ( 1 1)/2. 31

33 Corollary: P (U Ω) = C 1 Ω 1/2 Γ ( 1 2 )Γ ( +1 2 ) Γ ( ) ( n1 2 F 1 2, n 1 + n 2 2 ; n 1 + m + 1 ; Σ 2 Ω 2 ), where C 1 = Γ ( n 1 + n 2 2 ) Σ 2 1/2 /(Γ ( n 1 2 )Γ ( n 2 2 )) Hence we have a result in a matrix form. 32

34 Many results are known from 1970 s. Sometimes it is hard to tell whether those formulas are the same as our result. For example 2 F 1 reduces to a polynomial, if n 2 m 1 is a positive even integer. We use computer algebra to check that two polynomials the same. 33

35 HGM for 2 F 1 Put ى ي )+ ى +[p(x ى 2 = ى g ى ي ى )] ي, x ى q 2 (x ), ى r(x ي ) ي, x ى q(x where p(x i ) = c 1 2 (m 1) (a + b (m 1))x i x i (1 x i ) q 2 (x i, x j ) = 1/2 x i x j q(x i, x j ) = (1/2)x j(1 x j ) x i (1 x i )(x i x j ) ab r(x i ) = x i (1 x i ) 34

36 2 F 1 (a, b, c; x 1,..., x ) is annihilated by the linear partial differential operator g ى s. (Muirhead) Result: The set {g ى } is a Gröbner basis in the ring of differential operators with rational function coefficients R = C(x 1,..., x ) 1,...,. Implementation of HGM by N.Takayama: Implementation of HGM for 2 F 1 is similar to that of 1 F 1, but it is more difficult requiring lots of tuning. 35

37 Summary and things to do We gave a somewhat heavy summary of multivariate normal distribution theory. We have derived a distributional result in a matrix form involving 2 F 1. We have implemented HGM for 2 F 1, but some implementation issues remain. Diagonalization for 2 F 1 needs further investigation. 36