THE EFFICIENT EVALUATION OF THE HYPERGEOMETRIC FUNCTION OF A MATRIX ARGUMENT

Transcription

1 MATHEMATICS OF COMPUTATION Volume 75, Number 54, Pages S (6)184- Article electronically published on January 19, 6 THE EFFICIENT EVALUATION OF THE HYPERGEOMETRIC FUNCTION OF A MATRIX ARGUMENT PLAMEN KOEV AND ALAN EDELMAN Abstract. We present new algorithms that efficiently approimate the hypergeometric function of a matri argument through its epansion as a series of Jack functions. Our algorithms eploit the combinatorial properties of the Jack function, and have compleity that is only linear in the size of the matri. 1. Introduction The hypergeometric function of a matri argument has a wide area of applications in multivariate statistical analysis [17], random matri theory [7], wireless communications [8], [1], etc. Ecept in a few special cases, it can be epressed only as a series of multivariate homogeneous polynomials, called Jack functions. This series often converges very slowly [16, p. 39], [17], and the cost of the straightforward evaluation of a single Jack function is eponential [3]. The hypergeometric function of a matri argument has thus acquired a reputation of being notoriously difficult to approimate even in the simplest cases [], [1]. In this paper we present new algorithms for approimating the value of the hypergeometric function of a matri argument. We eploit recursive combinatorial relationships between the Jack functions, which allow us to only update the value of a Jack function from other Jack functions computed earlier in the series. The savings in computational time are enormous; the resulting algorithm has compleity that is only linear in the size of the matri argument. In the special case when the matri argument is a multiple of the identity, the evaluation becomes even faster. We have made a MATLAB [15] implementation of our algorithms available [13]. This implementation is very efficient (see performance results in Section 6), and has lead to new results [1], [6]. The hypergeometric function of a matri argument is defined as follows. Let p andq be integers, and let X be an n n comple symmetric matri with eigenvalues 1,,..., n.then (1.1) p F (a q 1 ) (a p ) (a 1,...,a p ; b 1,...,b q ; X) C k!(b 1 ) (b q ) (X), k= k Received by the editor September 16, 4 and, in revised form, February 6, 5. Mathematics Subject Classification. Primary 33C, 65B1; Secondary 5A99. Key words and phrases. Hypergeometric function of a matri argument, Jack function, zonal polynomial, eigenvalues of random matrices. This work was supported in part by NSF Grant DMS c 6 American Mathematical Society

2 834 PLAMEN KOEV AND ALAN EDELMAN where α> is a parameter, k means =( 1,,...) is a partition of k (i.e., 1 are integers such that = k), (1.) (a) ( a i 1 ) α + j 1 (i,j) is the generalized Pochhammer symbol, andc (X) is the Jack function. The Jack function C (X) =C ( 1,,..., n ) is a symmetric, homogeneous polynomial of degree in the eigenvalues 1,,..., n of X [17, Rem., p. 8], []. For eample, when α =1,C (X) becomes the (normalized) Schur function, and for α =,thezonal polynomial. 1 There are several normalizations of the Jack function which are scalar multiples of one another: C (X) is normalized so that k C (X) = (tr X) k ; in Section 3 we epress (1.1) in terms of the Jack function (X), which is normalized so that the coefficient of 1 is ( )!. The functions C (X) andj (X) can be defined recursively, e.g., J (1.3) J ( 1,,..., n )= J µ ( 1,,..., n 1 ) /µ n β µ, µ where β µ is a rational function of α (see Section 3 for details). The relationship (1.3) becomes key in achieving efficiency in our algorithms. The Jack functions C (X) andj (X), and in turn the hypergeometric function of a matri argument, depend only on the eigenvalues 1,,..., n of X. Many authors, however, have found the matri notation in (1.1) and the use of a matri argument to be more convenient. We follow the same practice. The hypergeometric function of a matri argument is scalar-valued, which is a major distinction from other functions of a matri argument (e.g., the matri eponential), which are matri-valued. The hypergeometric function of a matri argument generalizes the classical hypergeometric function to which it reduces for n = 1. In general, however, there is no eplicit relationship between these two functions for n. We approimate the series (1.1) by computing its truncation for m: (1.4) m p F q (a 1,...,a p ; b 1,...,b q ; X) m k= k (a 1 ) (a p ) k!(b 1 ) (b q ) C (X). The series (1.1) converges for any X when p q; itconvergesifma i i < 1 when p = q + 1, and diverges when p>q+ 1, unless it terminates [17, p. 58]. When it converges, its -term converges to zero as. In these cases (1.4) is a good approimation to (1.1) for a large enough m. The computational difficulties in evaluating (1.4) are: (A) the series (1.1) converges slowly in many cases [16]; thus a rather large m may be needed before (1.4) becomes a good approimation to (1.1); (B) the number of terms in (1.4) (i.e., the number of partitions m) grows, roughly, as O(e m ) (see Section 5); (C) the straightforward evaluation of a single Jack function, C (X) for = m, has compleity that grows as O(n m )[3]. 1 Some authors define the hypergeometric function of a matri argument through the series (1.1) for α =1[9,(4.1)]orα = [17, p. 58] only. There is no reason for us to treat the different α s separately (see also [4], [5], [7] for the uniform treatment of the different α s in other settings).

3 HYPERGEOMETRIC FUNCTION OF A MATRIX ARGUMENT 835 While there is little we can do about (A) (which is also a major problem even in the univariate (n = 1) case [18]) or (B), our major contribution is in improving (C), the cost of evaluating the Jack function. We eploit the combinatorial properties of the Pochhammer symbol and the Jack function to only update the -term in (1.4) from the µ-terms, µ. As a result the compleity of our main algorithm for computing (1.4), Algorithm 4., is only linear in the size n of the matri argument X, eponentially faster than the previous best algorithm [1] (see Sections, 5 and 6.3 for details). In the special case when X is a multiple of the identity, we present an even faster algorithm, Algorithm 4.1, whose compleity is independent of n. A number of interesting problems remain open. Among these are: detecting convergence; selecting the optimal value of m in (1.4) for a desired accuracy; selecting the optimal truncation of the series (1.1). We do not believe that a uniform answer to these problems eists for every α and every p and q. Therefore, we leave the choice of m and an appropriate truncation to the user. We elaborate more on these open problems in Section 7. With minimal changes our algorithms can approimate the hypergeometric function of two matri arguments (1.5) pf q (a 1:p ; b 1:q ; X, Y ) k= k and more generally functions of the form (1.6) G(X) = k= k (a 1 ) (a p ) k!(b 1 ) (b q ) a C (X), C (X) C C (I) (Y ), for arbitrary coefficients a at a similar computational cost (see, e.g., (6.5) in subsection 6.). In (1.5) and throughout this paper, we denote a vector (z 1,...,z t )asz 1:t. This paper is organized as follows. We survey previous algorithms for computing the hypergeometric function of a matri argument in Section. In Section 3 we describe our approach in computing the truncation (1.4). We present our new algorithms in Section 4, and analyze their compleity in Section 5. We present numerical eperiments in Section 6. Finally, we draw conclusions and present open problems in Section 7.. Previous algorithms Butler and Wood [] used Laplace approimations to compute the integral representations [17, Thm. 7.4., p. 64]: 1 (a; c; X) = Γ () n (c) Γ () n (a)γ () n (c a) 1F () <Y <I e tr(xy ) n+1 a (det Y ) n+1 c a det(i Y ) (dy ),

4 836 PLAMEN KOEV AND ALAN EDELMAN valid for real symmetric X, R(a) > n 1 1 (a, b; c; X) = Γ () n (c) F () n 1, R(c) > Γ () n (a)γ () n (c a) <Y <I,andR(c a) > n 1 ;and det(i XY ) b n+1 n+1 a (det Y ) c a det(i Y ) (dy ), valid for R(X) <I, R(a) > n 1 n 1,andR(c a) >. This approach, however, is restricted to the cases p = 1 or, q =1,andα =. Gutiérrez, Rodriguez, and Sáez presented in [1] (see also [19] for the implementation) an algorithm for computing the truncation (1.4) for α = (then the Jack functions are called zonal polynomials). For every k = 1,,..., m, the authors form the upper triangular transition matri [14, p. 99] K (indeed by all partitions of k) between the monomial symmetric functions (m ) k and the zonal polynomials (C ) k. Then for every partition =( 1,,...) k they compute m = µ µ 1 1 µ (where µ ranges over all distinct permutations of ( 1,,...) [, p. 89]) and form the product (C ) k = K (m ) k. Computing the vector (m ) m alone costs m ( ) n+m 1 m = O(n m ) since every term in every m is of degree m, andforevery nonstrictly increasing sequence of m numbers from the set {1,,..., n} we obtain adistinctterminsomem. The overall cost is thus at least eponential (O(n m )), which eplains the authors observation: We spent about 8 days to obtain the 67 zonal polynomials of degree with a 35 MHz Pentium II processor. In contrast, our Algorithm 4. takes less than a hundredth of a second to do the same. Its compleity is only linear in n and subeponential in m (see Section 5). 3. Our approach We make the evaluation of m p F q (a 1:p ; b 1:q ; X) efficient by only updating the -term from the µ-terms, µ, instead of computing it from scratch. We first epress m p F q (a 1:p ; b 1:q ; X) in terms of the Jack function J (X), which is normalized so that the coefficient of 1 in J (X) =J ( 1,,..., n ) equals ( )! [, Thm. 1.1]. The Jack functions C (X) andj (X) are related as (3.1) C where (3.) j = (X) = α ( )! (i,j) j J (X), h (i, j)h (i, j), and h (i, j) j i + α( i j +1)andh (i, j) j i +1+α( i j) arethe upper and lower hook lengths at (i, j), respectively. Denote (3.3) Q α (a 1 ) (a p ) j (b 1 ) (b q ).

5 HYPERGEOMETRIC FUNCTION OF A MATRIX ARGUMENT 837 Since J (X) =when n+1 >, we need to sum only over partitions with at most n parts: (3.4) m p F q (a 1:p ; b 1:q ; X) = m, n+1 = Q J (X). When computing (3.4), we recursively generate all partitions m in such a way that consecutively generated partitions differ in only one part. Therefore it is convenient to introduce the notation (i) ( 1,..., i 1, i 1, i+1,...) for any partition such that i > i+1. In the following subsections we derive formulas for updating the -term in (3.4) from the µ-terms, µ Updating the coefficients Q. We update Q from Q (i) using the following lemma. Lemma 3.1. (3.5) p Q j=1 = (a j + c) i 1 Q q (i) j=1 (b j + c) (g j α)e j g j=1 j (e j + α) i 1 j=1 l j f j l j + h j, where c = i 1 α + i 1, d = i α i, e j = d jα + j, g j = e j +1, f j = j α j d, h j = f j + α, andl j = h j f j. Proof. From (1.), (a) j (i) (3.6) = 1 i 1 j α h (i) (i, j) h (i) (i, j) h j=1 (i, j) h (i, j) which along with (3.3) imply (3.5). =(a) (i) (a (i 1)/α + i 1), and from (3.), i 1 j=1 h (i) (j, i ) h (i) (j, i ) h (j, i ) h, (j, i ) The seemingly complicated notation of Lemma 3.1 is needed in order to minimize the number of arithmetic operations needed to update Q. A straightforward evaluation of (3.3) costs 6 (+p+q); in contrast, (3.5) costs only (p+q)+1 i +9i 11 arithmetic operations. 3.. Updating the Jack function. When =(),J () ( 1,..., n )=1. For >(), we update J ( 1,..., n )fromj µ ( 1,..., r ),µ, r n. When X is a multiple of the identity we have an easy special case [, Thm. 5.4]: J (I) = (n (i 1) + α(j 1)). (i,j) Therefore we can update J (I) fromj (i) (I) as (3.7) J (I) =J (i) (I) (n i +1+α( i 1)). In the general case, we update the Jack function using the identity (see, e.g., [, Prop. 4.]), (3.8) J ( 1,,..., n )= J µ ( 1,,..., n 1 ) /µ n β µ, µ

6 838 PLAMEN KOEV AND ALAN EDELMAN where the summation is over all µ such that /µ is a horizontal strip, and { (i,j) (3.9) β µ B µ(i, j) h (i,j) µ Bµ µ(i, j), where Bν µ(i, j) ν (i, j), if j = µ j, h ν (i, j), otherwise. The skew partition /µ is a horizontal strip when 1 µ 1 µ [, p. 339]. We borrow the idea for updating the Jack function from [3], but make two important improvements. We only update the coefficients β µ, and store the precomputed Jack functions much more efficiently than in [3]. The coefficients β µ are readily computable using (3.9) at the cost of 6( + µ ) arithmetic operations. The following lemma allows us to start with β =1and update β µ(k) from β µ at the cost of only 1k +6µ k 7 arithmetic operations. Lemma 3.. Let, µ, and ν = µ (k) be partitions such that /µ and /ν are horizontal strips, and r = µ r for r k 1. Then β ν k k 1 u r (3.1) = α β µ u r=1 r + α µ v r + α k 1 w r + α, v r=1 r w r=1 r where α = α 1, t= k αµ k,q= t +1,u r = q r + α r,v r = t r + αµ r,and w r = µ r t αr. Proof. Denote l µ k.wehaveν = µ, ecept for ν k = µ k 1. Since /µ and /ν are horizontal strips, l = µ l and l ν l = µ l 1. Then β ν µ = Bµ µ B ν β µ B µ ν Bν ν = Bν Bµ ν B µ µ B ν ν B µ µ(k, l) = Bν B B µ µ (3.11) µ B ν α. ν ν We transform the first term of (3.11) by observing that Bµ(i, j) =Bν(i, j), ecept for j = l: k k k Bν B B = ν(r, l) µ B r=1 µ(r, l) = h (r, l) h r=1 (r, l) = k r +1+α( r l) k r + α( r=1 r l +1). To simplify the second term of (3.11), we observe that B µ(i, µ j) =Bν(i, ν j), ecept for i = k and j = l. Also r = µ r = ν r for r k 1, and k = µ k µ k 1=ν k. Therefore ν B µ µ B ν ν = = = which yields (3.1). k r=1 l 1 r=1 l 1 r=1 B µ µ(r, l) B ν ν(r, l) l B µ(k, µ r) r=1 B ν ν(k, r) h k 1 µ(k, r) h ν(k, r) h µ(r, l) h ν r=1 (r, l) µ r k + α(µ k r +1) µ r k + α(µ k r) k 1 r=1 k r + α(µ r l +1), k r + α(µ r l)

7 HYPERGEOMETRIC FUNCTION OF A MATRIX ARGUMENT Algorithms for efficient evaluation of m p F q (a 1:p ; b 1:q ; X) Algorithm 4.1 computes m p F q (a 1:p ; b 1:q ; X) in the easy special case when X is a multiple of the identity. Algorithm 4. handles the general case. Both algorithms recursively generate all partitions m with at most n parts by allowing 1 to take all values 1,,...,m and, independently, i,i =1,,...,n, to take all values 1,,..., i 1, subject to the restriction m. The coefficients Q are updated using (3.5). The Jack functions are updated using (3.7) or (3.8) as appropriate ThecasewhenX is a multiple of the identity. Algorithm 4.1 (Hypergeometric function, X = I). The following algorithm computes m p F q (a 1:p ; b 1:q ; I). The variables, n, α, s, a, b, and are global. function s = hgi(m, α, a, b, n, ) s =1 summation(1, 1, m) function summation(i, z, j) for i =1:min( i 1,j) (defaults to j for i =1) z = z (n i +1+α( i 1)) T (where T is the right-hand side of (3.5)) s = s + z if (j > i ) and (i <n) then summation(i +1,z,j i ) endif endfor i = In Algorithm 4.1 the variable z equals the -term in (3.4); it is updated using (3.5) and (3.7). The parameter j in summation equals m. The hypergeometric function of two matri arguments (1.5) is in this case pf q (a 1:p ; b 1:q ; I, yi) = p F q (a 1:p ; b 1:q ; yi). 4.. The general case. We use the identity (3.8) to update the Jack function, but in order to use it efficiently, we need to store and reuse the Jack functions computed earlier. Therefore we inde all partitions m with at most n parts ( n+1 =)by linearizing the m-tree that they form (each node =( 1,,..., k ) has at most m children ( 1,,..., k, k+1 ), k+1 =1,,...,min( k,m ), k < n). In other words, if P mn =#{ m, n+1 =}, we assign a distinct integer inde N {, 1,...,P mn } to every such partition. We start by assigning the indees, 1,...,m to partitions with one part: N (i) i, i =, 1,,...,m. Then, recursively, once an inde N has been assigned to a partition =( 1,..., k ) with k parts, we assign m consecutive unassigned indees to the partitions ( 1,..., k, k+1 ), k+1 =1,,...,m. We record the tree structure in an array D such that D(N (1,..., k ))=N (1,..., k,1). Now given, we can compute N by starting with N (1 ) = 1 and using the recurrence (4.1) N (1,..., i ) = D(N (1,..., i 1 ))+ i 1. We store every computed J ( 1:i )( m, n+1 =,i =1,,...,n)inthe (N,i)th entry of an P mn n array, which we call J in Algorithm 4. below.

8 84 PLAMEN KOEV AND ALAN EDELMAN We compute the value of P mn as follows. Let p k (i) be the number of partitions of i with eactly k parts. The p k (i),i =1,,...,m,k =1,,...,min(m, n), are computed using the recurrence p k (i) =p k 1 (i 1) + p k (i k) [1, p. 8]. Then (4.) P mn = m i=1 Net, we present our main algorithm. min(m,n) k=1 p k (i). Algorithm 4. (Hypergeometric function). The following algorithm computes m p F q (a 1:p ; b 1:q ; X), where X = diag(). The variables m, n, a, b, α,, J, D, H,, N,µ,N µ,ands are global. function s = hg(m, α, a, b, ) Compute P mn using (4.) n = length(); H = m +1; s =1; D = zeros(p mn, 1) J = zeros(p mn,n); J(1, :) = 1 (J ( 1:i ) is stored in J(N,i)) summation(1, 1, 1, m) function summation(i, z, j) r =min( i 1,j) (defaults to j for i =1) for i =1:r if ( i =1) and (i >1) then D(N )=H; N = H; H = H + r else N = N +1 endif z = z T (where T is the right-hand side of (3.5)) if 1 =1 then J(N, 1) = 1 (1 + α( 1 1)) J(N 1, 1) endif for t =:n do jack(, 1,,t) endfor s = s + z J(N,n,1) if (j > i ) and (i <n) then summation(i +1,z,j i ) endif endfor function jack(k, β µ,c,t) for i = k : µ 1 if (k >) and (µ i >µ i+1 ) d = N µ µ i = µ i 1; Compute N µ using (4.1) Update β µ using (3.1) if µ i > then jack(i, β µ,c+1,t) else J(N,t)=J(N,t)+β µ J(N µ,t 1) c+1 t endif µ i = µ i +1; N µ = d endif endfor if k = then J(N,t)=J(N,t)+J(N,t 1) else J(N,t)=J(N,t)+β µ c t endif (computes J ( 1:t ))

9 HYPERGEOMETRIC FUNCTION OF A MATRIX ARGUMENT 841 Algorithm 4. generates all partitions m with at most n parts analogously to Algorithm 4.1. The parameter z in summation is now just Q. For every the function jack recursively generates all partitions µ such that /µ is a horizontal strip. The Jack function is computed using (3.8); β µ is updated using (3.1). The parameter c equals µ at all times Implementation notes. In our implementation of Algorithms 4.1 and 4. in [13] we made a few fairly straightforward, but important improvements: (1) we precompute the values of j i for i =1,,...,n,andj =1,,...,m; () we keep and update the conjugate partitions and µ along with and µ, so we never need to recover and µ when computing (3.5) and (3.1); (3) when two sets of arguments ( 1:n and y 1:n ) are passed, the hypergeometric function of two matri arguments, (1.5), is computed; (4) if a vector = (t 1,...,t r ) is passed as a parameter in Algorithm 4.1, the output is also a vector with the values of m p F q (a 1:p ; b 1:q ; t j I)forj = 1,,...,r. 5. Compleity analysis Algorithm 4.1 (the case X = I) costso(p mn ) arithmetic operations (where again, P mn is the number of partitions, m with at most n parts). To bound the compleity of Algorithm 4. (general case) we observe that the formula (3.8) represents the summation of at most P mn terms (in fact a lot less, but we have been unable to obtain a better bound than P mn that is easy to work with). We use (3.8) to compute the Jack function J ( 1:t ) for all m and t n, i.e., P mn n Jack functions, each of which costs at most O(P mn ) arithmetic operations. The overall cost of Algorithm 4. is thus bounded by O(P mn n). There is no eplicit formula for P mn, so we use Ramanujan s asymptotic formula [11, p. 116] for number of partitions of m p(m) 1 4m 3 ep ( π m/3 ) to obtain P mn m i=1 ( p(i) O ep ( π m/3 )) ( and Pmn O ep ( π m/3 )). Therefore the compleity of Algorithm 4. is linear in n and subeponential in m. 6. Numerical eperiments We performed etensive numerical tests to verify the correctness and compleity of our Algorithms 4.1 and 4.. We compared the output of our algorithms for F and 1 F 1 with eplicit epressions (subsection 6.1). We also compared the probability distributions of the eigenvalues of certain random matrices (which are epressed as hypergeometric functions) against the results of Monte Carlo eperiments (subsection 6.). Finally, we present performance results in subsection 6.3.

10 84 PLAMEN KOEV AND ALAN EDELMAN 6.1. Eplicit epressions. We compared the output of Algorithms 4.1 and 4. against the epressions [17, p. 6]: (6.1) F (X) = e tr X, (6.) 1F (a; X) = det(i X) a, for n = 1 and random uniformly distributed values i [, 1 ], i =1,,...,n.For m = 3 it took less than one second per test. The results agreed to at least 1 decimal digits with (6.1), and at least 8 decimal digits with (6.), reflecting the slower convergence of (6.). 6.. Eigenvalue statistics. We tested Algorithms 4.1 and 4. against the eigenvalue statistics of the β-laguerre and Wishart matrices. The n nβ-laguerre matri of parameter a is defined as L = BB T,where B = χ a χ β(n 1) χ a β χ β χ a β(n 1), a > β (n 1). The n n Wishart matri with l degrees of freedom (l >n) and covariance matri Σ, A = W n (l, Σ), is defined as A =Σ 1/ Z T Z Σ 1/,whereZ ij = N(, 1) for i =1,,...,l; j =1,,...,n. The eigenvalue distributions of A and L are the same when β =1, a = l,and Σ=I. The cumulative distribution functions of the largest eigenvalues, λ L and λ A,of L and A, are ( Γ n 1 n α P (λ L <) = +1) ( ( Γ n a + n 1 α +1) ( n+1 ) P (λ A <) = Γ() n Γ () n ( l+n+1 ) det ( 1 Σ 1) l/ ) an ( 1F 1 a; a + n 1 α +1; 1 I), ( 1F () l 1 ; n+l+1 ; 1 Σ 1), respectively [4, Thm. 1..1, p. 147], [17, Thm , p. 4], where α =/β, and Γ n is the multivariate Gamma function of parameter α: n ( (6.3) Γ n (c) π n(n 1) α Γ c i 1 ) for R(c) > n 1 α α. i=1 We use the Kummer relation [17, Thm , p. 65] 1F 1 (a; c; X) =e tr X 1F 1 (c a; c; X) to obtain the equivalent, but numerically more stable epressions P (λ L <)= P (λ A <)= ( n 1 Γ n Γ n α ( +1) ( ) an a + n 1 e n α +1) ) ( n+1 Γ() n ( Γ () l+n+1 n 1F 1 ( n 1 α ) det ( 1 Σ 1) l/ e tr( Σ 1 ) 1 F () 1 n 1 +1;a + α +1;1 I), ( n+1 ; n+l+1 ; 1 Σ 1), which we plot on Figure 1 along with the Monte Carlo results from a sample of 1 random matrices.

11 HYPERGEOMETRIC FUNCTION OF A MATRIX ARGUMENT 843 cdf of λ ma of 3 3 Laguerre, a=3 cdf of λ ma of Laguerre, a= predicted eperimental cdf of λ ma of W (4,diag(1,)). predicted eperimental predicted eperimental cdf of λ ma of W 3 (4,diag(.5,1,1.1)). predicted eperimental 1 3 Figure 1. The c.d.f. of λ ma of the β-laguerre matri (top plots) and the Wishart matri (bottom plots). Net we consider the distribution of the smallest eigenvalue of L and that of tr A. If c = a β (n 1) 1 is a nonnegative integer, then the probability density function of the smallest eigenvalue of L (see, e.g., [4, Thm , p. 146]) is proportional to (6.4) f() = cn e ( n F (/β) c, β n +1; I n 1). Since c is a nonpositive integer, the series epansion of F (/β) in (6.4) terminates, even though it diverges in general. The probability density function of tr A is g f() = det(λ 1 Σ) l/ ln +k,λ() ( l () k! ) C() (I λσ 1 ) (6.5) where k= = det(λ 1 Σ) l/ k= k g ln +k,λ() k ( l ) () g r,λ () = e /λ r 1 (λ) r, ( >), Γ(r) k j 1 J () (I λσ 1 ),

12 844 PLAMEN KOEV AND ALAN EDELMAN.5 pdf of λ min of Laguerre, a=5.5 pdf of λ min of Laguerre, a= pdf of tr(a), A=W 3 (4,diag(.5,.5,1)) pdf of tr(a), A=W 4 (5,diag(.5,1,1.1,)) Figure. The p.d.f. of λ min of the β-laguerre matri (top plots) and the p.d.f. of the trace of the Wishart matri (bottom plots). and λ is arbitrary [17, p. 341]. We follow the suggestion by Muirhead to use λ =λ 1 λ l /(λ 1 + λ l ), where λ 1 and λ l are the largest and smallest eigenvalues of Σ, respectively. Although the epression (6.5) is not a hypergeometric function of a matri argument, its truncation for m has the form (1.6), and is computed analogously. We plot (6.4) and (6.5) on Figure and compare the theoretical predictions of these formulas with eperimental data Performance results. In Figure 3 we demonstrate the efficiency of Algorithms 4.1 and 4. on an 1.8GHz Intel Pentium 4 machine. In the left plot we present the performance data for Algorithm 4.1 (whose compleity is independent of the size n of the matri X = I). Its efficiency is evident we need to sum beyond partitions of size m = 5 before this algorithm takes a full second (for reference, the k! in the denominator of the -term in (1.1) then reaches up to 5! ). Algorithm 4. is also very efficient. The right plot of Figure 3 clearly demonstrates its linear compleity in n. It also takes at most a few seconds on matrices of size n 1 and partitions of size m 3.

13 HYPERGEOMETRIC FUNCTION OF A MATRIX ARGUMENT 845 Time to compute m F () (I n ) 5 Time to compute m F () (X) m=3 m=5 m= time in seconds time in seconds ma size partition m 5 1 n = size of X Figure 3. The performance of Algorithms 4.1 (left) and 4. (right). 7. Conclusions and open problems We have presented new algorithms for computing the truncation of the hypergeometric function of a matri argument. They eploit the combinatorial properties of the Pochhammer symbol and the Jack function to achieve remarkable efficiency, and have lead to new results [1], [6]. Several problems remain open, among them automatic detection of convergence. The -term in (3.4) does approach zero as, but it need not monotonically decrease. Although we have i 1 j=1 in (3.5), it is not always true that Q J i 1 (g j α)e j g j (e j + α) l j f j < 1 l j=1 j + h j Q (i) J (i), and it is unclear how to tell when convergence sets in. Another open problem is to determine the best way to truncate the series (1.1). Our choice to truncate it for m seems to work well in practice, but one can imagine selecting a partition λ and truncating for λ instead of, or in addition to m. Acknowledgments We thank Brian Sutton and Per-Olof Persson for several discussions that resulted in the simplification of the implementation of our algorithms, as well as the

14 846 PLAMEN KOEV AND ALAN EDELMAN anonymous referee for the useful comments, which lead to improvements in the eposition. References 1. P.-A. Absil, A. Edelman, and P. Koev, On the largest principal angle between random subspaces, Linear Algebra Appl., to appear.. R. W. Butler and A. T. A. Wood, Laplace approimations for hypergeometric functions with matri argument, Ann. Statist. 3 (), no. 4, MR19617 (3h:676) 3. J. Demmel and P. Koev, Accurate and efficient evaluation of Schur and Jack functions, Math. Comp., 75 (5), no. 53, I. Dumitriu, Eigenvalue statistics for the Beta-ensembles, Ph.D. thesis, Massachusetts Institute of Technology, I. Dumitriu and A. Edelman, Matri models for beta ensembles, J.Math.Phys.43 (), no. 11, MR (4g:844) 6. A. Edelman and B. Sutton, Tails of condition number distributions, SIAMJ.MatriAnal. Appl., accepted for publication, P. Forrester, Log-gases and random matrices, matpjf.html 8. H. Gao, P.J. Smith, and M.V. Clark, Theoretical reliability of MMSE linear diversity combining in Rayleigh-fading additive interference channels, IEEE Transactions on Communications 46 (1998), no. 5, K. I. Gross and D. St. P. Richards, Total positivity, spherical series, and hypergeometric functions of matri argument, J. Appro. Theory 59 (1989), no., MR1118 (91i:335) 1. R. Gutiérrez, J. Rodriguez, and A. J. Sáez, Approimation of hypergeometric functions with matricial argument through their development in series of zonal polynomials, Electron. Trans. Numer. Anal. 11 (), MR17997 (b:334) 11. G. H. Hardy, Ramanujan: Twelve lectures on subjects suggested by his life and work, AMS Chelsea, New York, MR486 (3:71d) 1. M. Kang and M.-S. Alouini, Largest eigenvalue of comple Wishart matrices and performance analysis of MIMO MRC systems, IEEE Journal on Selected Areas in Communications 1 (3), no. 3, P. Koev, I. G. Macdonald, Symmetric functions and Hall polynomials, Second ed., Oford University Press, New York, MR (96h:57) 15. The MathWorks, Inc., Natick, MA, MATLAB reference guide, R.J.Muirhead,Latent roots and matri variates: a review of some asymptotic results, Ann. Statist. 6 (1978), no. 1, MR (56:16919) 17., Aspects of multivariate statistical theory, John Wiley & Sons Inc., New York, 198. MR6593 (84c:673) 18. K. E. Muller, Computing the confluent hypergeometric function, M(a, b, ), Numer. Math. 9 (1), no. 1, MR (3a:3344) 19. A. J. Sáez, Software for calculus of zonal polynomials, R. Stanley, Some combinatorial properties of Jack symmetric functions, Adv. Math. 77 (1989), no. 1, MR11473 (9g:5) 1., Enumerative combinatorics. Vol. 1, Cambridge Studies in Advanced Mathematics, vol. 49, Cambridge University Press, Cambridge, MR1446 (98a:51)., Enumerative combinatorics. Vol., Cambridge Studies in Advanced Mathematics, vol. 6, Cambridge University Press, Cambridge, MR16768 (k:56) Department of Mathematics, Massachusetts Institute of Technology, Cambridge, Massachusetts 139 address: plamen@math.mit.edu Department of Mathematics, Massachusetts Institute of Technology, Cambridge, Massachusetts 139 address: edelman@math.mit.edu