Multidimensional Fast Gauss Transforms by Chebyshev Expansions

Transcription

1 Multiimensional Fast Gauss Transforms by Chebyshev Expansions Johannes Tausch an Alexaner Weckiewicz May 9, 009 Abstract A new version of the fast Gauss transform FGT) is introuce which is base on a truncate Chebyshev series expansion of the Gaussian. Unlike the traitional fast algorithms, the scheme oes not subivie sources an evaluation points into multiple clusters. Instea, the whole problem geometry is treate as a single cluster. Estimates for the error as a function of the imension an the expansion orer p will be erive. The new algorithm has orer `+p+1 N +M) complexity, where M an N are the number of source- an evaluation points. For a fixe p, this estimate is only polynomial in. However, to maintain accuracy it is necessary to increase p with. The precise relationship between an p is investigate analytically an numerically. 1 Introuction The Gauss transform is important in many applications, ranging from financial calculus [4], image processing [5, 14], multivariate ata analysis [11], machine learning [15, 9] an approximation by raial basis function [8], to name only a few. The task is to fin the potentials Φx i ), 1 i N, for given sources q j, 1 j M, such that ) M Φx i ) exp x i y j q j 1) δ j1 where x i R is the location of the i-th evaluation point, y j R is the location of the j-th source, δ > 0 is the variance, is the imension an is the Eucliean norm. We assume that the scaling is such that all sources an evaluation points are locate in the unit cube C : [ 1, 1]. The irect computation of the sum at every evaluation point is an ONM) algorithm which is prohibitively large even for moerate values of N an M. However, it was recognize early on that a variant of the fast multipole metho can be use to evaluate the potentials in optimal ON + M) complexity. This is the fast Gauss transform, introuce by Greengar an Strain [6]. 1

2 The algorithm subivies the unit cube into smaller cubes an approximates interactions between the small cubes using moments-to-local MtL) translation operators. The complete algorithm consists of computing the moments, computing the MtL translations, an evaluating the truncate series to obtain the potentials. The fast Gauss transform is simpler than the tree coes evelope for potential theory in that it suffices to have only one level of interacting cubes. The cost of the fast Gauss transform epens on the complexity of one MtL translation an the number of interacting cubes. Since the Gauss kernel ecays exponentially, only nearby interactions have to be compute, thus it is possible to choose the parameters number of cubes in a linear irection, number of expansion terms) such that the complexity is optimal an inepenent of the variance. The fast algorithm computes only an approximation of the potential, because the Gauss kernel is replace by a truncate series expansion. The conventional approach is to work with the Taylor series of the Gaussian. This is also known as the Hermite expansion since erivatives of the Gaussian involve Hermite polynomials. After the original papers, several optimizations an extensions of this algorithm have been iscusse, e.g., [, 7, 10, 1, 14]. In aition to the original papers, there is a number of papers that iscuss the error that is introuce by the truncation of the Hermite expansion, e.g., [3, 13]. From these papers it is evient that it is ifficult to obtain realistic error estimates in imensions larger than one. In this article we are primarily intereste in case of high imensions. Although the optimal ON + M) complexity of the original FGT hols in any imension, the constants grow exponentially with an thus the algorithm suffers from the curse of imensionality. There are two aspects of the algorithm that contribute to this exponential epenence. The first is that if the omain is subivie into L cubes in each irection then the number of interacting cubes grows at least like L, assuming that the istribution of sources an evaluation points is fairly uniform. The secon aspect is that the number of terms that have to be retaine in the expansion grows exponentially with the imension. To aress the first issue, more efficient space subivision schemes have been consiere [10]. In our work we avoi the first problem altogether by using only one cube that contains all sources an evaluation points. In this setting it is necessary to have an expansion of the Gauss kernel that converges rapily in the whole unit cube, especially for small values of the variance. It is well known that the Hermite expansion converges rapily only in a neighborhoo of the expansion point. On the other han, we will show that the Chebyshev expansion has goo global convergence properties. To aress the secon issue, we retain the terms in the multivariate Chebyshev series where the sum of the orers is less than p. A similar approach been consiere for the Taylor series expansion in [10]. Thus the number of terms is +p ), which grows only like polynomial in. Furthermore, we will show how to exploit the Kronecker prouct form to compute the MtL translation using less than p + 1) ) +p multiplications. Thus for a given orer the algorithm has polynomial complexity.

3 However, in the complexity estimates it is important to take approximation error uner consieration. Our analysis shows that to control the error of the potential in a weighte L -norm it is necessary that the orer must be proportional to the imension. Thus the complexity still has an exponential epenence on the imension. However, our numerical results suggest that this epenence is very mil an hence we are able to compute Gauss transforms in imensions as high as 8 on an eight gigabyte workstation. The outline of the paper is as follows. In Section we erive the approximation theory of the Gaussian using Chebyshev expansions. an emonstrate that the expansion coefficients can be represente in close form using Bessel functions. Then in Section 3 we escribe the one-imensional fast Gauss transform. Section 4 extens the methoology to arbitrary imensions an escribes how to write the MtL transform in Kronecker prouct form. Error an complexity estimates for the multiimensional case are erive in Sections 5 an 6. Section 7 conclues with numerical results. Chebyshev Expansion of the Gaussian We consier the expansion of the one-imensional Gaussian in Chebyshev polynomials. It is well known that the Chebyshev polynomials T n x) cosnacosx)), x [ 1, 1], are L w [ 1, 1]-orthogonal with weight function wx) 1 x ) 1. That is, 1 1 T n x)t m x)wx) x π γ n δ n,m, where γ 0 1 an γ n for n 1. Thus the Gaussian has the expansion ) exp r E n δ)t n r), r [ 1, 1] ) δ with coefficients E n δ) γ n π 1 n0 1 exp ) x T n x)wx) x. 3) δ The integral can be expresse in close form, using Bessel functions. To see this, recall Formulas an of [1], which can be combine as expiz cos θ) γ n J n z)i n cosnθ). n0 Here J n enotes the Bessel function of orer n. If we replace θ θ, set z i/δ) an multiply by exp 1/δ)), we obtain exp 1 ) δ cos θ γ n i n exp 1 ) ) i J n cosnθ). δ δ n0 3

4 If r cos θ then the left han sie is the Gaussian, an the right han sie is the expansion in Chebyshev polynomials. Thus the coefficients in ) are { γn i n exp ) 1 E n δ) δ J n i ) δ, n even, 0 n o. 4) To obtain error bouns of the truncate series in ), we nee to estimate the coefficients. Since it is har to fin uniform bouns for Bessel functions of arbitrary orer an argument, it is more promising to work with the integral efinition of the coefficients in 3). The estimate below epens on the following technical result. Lemma.1 Let Then hols for b > 0. Ib) : 1 π π 0 exp b cos θ) ) θ. Ib) 1 π b Proof. It is easily checke that Now thus Ib) 1 π exp b sin θ) ) θ 1 π exp b sin θ) ) θ π π π π Ib) 1 π π π sin θ θ π, 0 θ π, exp 4b ) π θ θ 1 exp 4b ) π π θ θ 1 π b. Lemma. For every a > 0 the estimate E n δ) γ n δπ a n+1 exp hols. 1 4δ a 1 ) ) a Proof. After change of variables x cos θ the integral in 3) becomes E n δ) γ n π π 0 exp 1 ) δ cos θ) expinθ) θ 4

5 This integran is perioic with perio π an is an entire function in the complex θ-plane. By the Cauchy integral theorem, the contour of integration can be move by ã units in the upper imaginary half plane. Hence E n δ) γ n π an we may estimate π E n δ) γ n π e nã 0 [ exp 1 ] [ ] δ cos θ + iã) exp inθ + iã) θ, π Elementary manipulations lea to Re { cos θ + iã) } 1 where a expã). Thus E n δ) γ n πa n exp 1 4δ γ n a n exp 1 4δ 0 exp 1 ) δ Re cos θ + iã) θ. a + 1a ) cos θ) 1 a 1 ) 4 a a 1 ) ) π exp b cos θ) ) θ, a 0 ) ) a 1 a Ib), where Ib) was efine in Lemma.1 an b 1 a + 1a ) δ. 5) Because of 1 δ b a the assertion can be erive from Lemma.1. Since the estimate of Lemma. is vali for any positive a, we can choose a that minimizes the last estimate. Simple calculus shows that the optimal value of a is [ a δñ δñ) ) ] 1 1, where ñ n + 1. Thus where κt) 1 ln t t ) 1 E n δ) γ n δπ exp ñκñδ) ) 6) ) 1 4t [ ) t t ) 1 1 t t ) 1 ) 1 ]. 5

6 The function κt) is monotonically increasing for t 0, furthermore κt) t, t 0, 7) 4 κt) 1 lnt), t. 8) We enote the p-term expansion of the Gaussian by G p, i.e., G p δ, r) p E n δ)t n r) 9) n0 an the remainer by R p. Since T n 1 in [ 1, 1] it follows that R p δ, r) E n δ)t n r) E n δ), r 1. 10) np+1 np+1 Using estimate 6) an the remainer of the geometric series the following boun can be erive δπ δπ exp pκδ p)) R p δ, r) exp ñκδñ)) 1 exp κδ p)), 11) np+1 where p p + if p is o an p p + 3 if p is even. The secon step is justifie because κ ) is monotonically increasing. If δ is fixe an p then we obtain from the asymptotic 8) that δπ p R p δ, r) [δp]. 1) Thus the convergence is super-exponential, which has to o with the fact that the Gauss kernel is an entire function. Furthermore, the estimate makes clear that the convergence is slower when δ gets smaller. The Chebyshev coefficients an their estimates are shown in Figure 1. 3 One Dimensional Gauss Transform The fast Gauss transform escribe below epens on a truncate series expansion of the Gauss kernel, which is the exponential in 1) consiere as a function of the x- an the y-variable. We will consier two ifferent truncation methos, the first is to truncate the Chebyshev expansion in both variables, the other is to truncate the expansion of the Gaussian in the r-variable an then use an aition theorem of the Chebyshev polynomials. 6

7 δ1/4, exact δ1/4, estimate δ1/16, exact δ1/16, estimate δ1/64, exact δ1/64, estimate 10 6 E n orer Figure 1: Comparison of E n, with the estimate 6) 3.1 L w-orthogonal approximation If x, y [ 1, 1], the two-variate Chebyshev expansion of the Gauss kernel is ) x y) exp E k,l δ)t k x)t l y) 13) δ k,l N 0 where the coefficients are E k,l δ) γ kγ l π exp ) x y) T k x)t l y)wx)wy) yx 14) δ Unlike the case of the Gaussian, there oes not appear to be a close-form expression for the coefficients of the Gauss kernel. Thus the integrals have to be 7

8 compute by quarature when the metho is implemente. We will comment on this issue in Section 7. The expansion 14) is truncate by retaining all terms smaller than a given orer p, the resulting approximation is enote by G p x, y) E k,l δ)t k x)t l y) 15) k+l p an the remainer is enote by R p x, y). approximation in the L w L w-norm. By orthogonality, Gp is the best 3. Expansion base on an Aition Theorem An alternative way to obtain an approximation of the Gauss kernel is base the following aition theorem of Chebyshev polynomials. Lemma 3.1 There are coefficients a n) k,l such that ) x y T n a n) k,l T kx)t l y). 16) k+l n For any p n, the coefficients are given by where x p) j a n) k,l γ p 1 kγ l p cosπ j 1 p ). i0 j0 p) x i T n x p) j ) T k x p) i )T l x p) j ), 17) Proof. Since T n is a polynomial of egree n it follows that T 1 n x y)) is a linear combination of the monomials x k y l, 0 k + l n. Since T k x)t l y), 0 k+l n is also a basis for this subspace assertion 16) follows. By orthogonality, the coefficients are given by a n) k,l γ kγ l π ) x y T n T k x)t l y)wy)wx) yx. If the integrals are replace by the p-th orer Gauss-Chebyshev quarature rule, which is exact for this integran when p n, then assertion 17) follows. To approximate the Gauss kernel, begin with the expansion of the Gaussian in equation 9) exp 1δ ) x y) exp 4 δ ) ) x y δ G p 4, x y ) δ +R p 4, x y ), where, by the aition theorem, δ G p 4, x y ) p ) ) δ x y E n T n E p) k,l 4 δ)t kx)t l y). n0 k+l p 8

9 The coefficients are given by E p) k,l δ) p nk+l a n) k,l E n ) δ, 18) 4 an the remainer of the truncate series of the Gauss kernel is δ R p 4, x y ) ) ) δ x y E n T n. 19) 4 np+1 The convergence results of Section apply since x, y Fast Gauss Transform To compute the sum in 1) efficiently, replace the Gauss kernel by one of the two truncate Chebyshev series expansions escribe above. Thus the potential at the evaluation points Φx i ) is approximate by Φ p x i ), given by Φ p x i ) M j1 k+l p E k,l T k x i )T l y j )q j. Here, E k,l E k,l δ) in the case of the two-variate expansion an E k,l E p) k,l δ) in case of expansion with the aition theorem. Rearranging the orer of summation, we see that the approximate potential has an expansion in terms of Chebyshev polynomials p Φ p x i ) λ k T k x i ), 0) where the expansion coefficients are k0 p k λ k E k,l µ l. 1) Here µ l are the moments of the sources, which are given by µ l l0 M T l y j )q j. ) j0 Instea of evaluating the exact potentials Φx i ) via the sum in 1), we use the following proceure for computing the approximate potentials Φ p x i ). Fast Gauss Transform 1. QtM translation: Compute the moments in ).. MtL translation: Compute the expansion coefficients in 1). 9

10 3. LtP translation: Compute the truncate series 0) at the evaluation points. The complexities of Steps 1 an 3 are OMp) an ONp), respectively; the complexity of the MtL translation is Op ). Thus the metho will be faster than the irect evaluation if the expansion orer satisfies pn + M) NM. 3.4 Approximation Error If the Gauss kernel is approximate with the aition theorem, error estimates for the potential follow irectly from the previously erive estimates for the remainer of the Gaussian. The case of the L w-orthogonal expansion is more ifficult, since it is har to estimate the magnitue of the coefficients in the twovariate Chebyshev series 13). Instea of fining bouns for these coefficients, we will use the optimality of the approximation in the L w-norm. Since this type of argument will reappear in the analysis of the multi-imensional transform, we will iscuss the etails below. In view of 0) the error of the potential when using the L w -orthogonal expansion can be expresse by Chebyshev series Φx) Φ p x) ˆλ k T k x), where the expansion coefficients are ˆλ k By orthogonality, it follows that k+l p Φ Φ p L w k+l p k0 k0 E k,l δ)µ l. 3) π γ k ˆλ k : ˆλ l w 4) To estimate the lw -norm of ˆλ we apply the Cauchy-Schwarz inequality to 3) ˆλ k 4 π π E γ k,lδ) π µ l. l γ l Multiplying by π γ k l N 0 an aing over k leas to ˆλ l 4 π w π E γ k γ k,lδ) µ l. 5) l w k+l p The expression in the brackets is the L w -norm of the resiual. Since the L w - orthogonal approximation is optimal, we can use the approximation base on the aition theorem as the upper boun, hence, k+l p π γ k γ l E k,l δ) R L w L w π max x, y 1 R p δ 4, x y ). 10

11 Combining the last estimate with 4), 5) an 11) leas to δπ exp[ pκ pδ/4)] Φ Φ p L w 1 exp[ κ pδ/4)] µ l. w 4 Multiimensional Transforms If the sources an evaluation points are locate in a -imensional Eucliean space then the Gauss kernel separates ) x y exp exp x 1 y 1 ) )... exp x y ) ). δ δ δ Expaning each term in the prouct shows that the expansion of the multivariate Gauss kernel are proucts of single-variate expansion coefficients ) x y exp E α,β δ)t α x)t β y), x, y [ 1, 1]. 6) δ α,β Here an in the following we use the usual multi inex notation α α 1,..., α ), α α α, an efine the multivariate Chebyshev polynomial by T α x) T α1 x 1 )... T α x ) an the multivariate coefficients by E α,β δ) E α1,β 1 δ)... E α,β δ). 7) We seek an approximation of the Gauss kernel in the space of -variate polynomials of egree p Π p : span { x α y β : α + β p } span {T α x)t β y) : α + β p}. The inex set of the basis of Π p is efine by S p is the -imensional simplex Sp. Here, S p } { α 1,..., α ) : α p 4.1 L w-orthogonal approximation In analogy to the one-imensional case we can simply truncate the Chebyshev series, an obtain G p x, y) E α,β δ)t α x)t β y). 8) α+β p The corresponing resiual is enote by R p x, y). By orthogonality of the Chebyshev polynomials, 8) is the best approximation of the Gauss kernel in Π p with respect to the L w-norm, an the resiual is L w-orthogonal to G p. Furthermore, the coefficients are proucts of the single-variate coefficients, which. 11

12 will be important for efficient MtL translations. Thus the approximation 8) will be the starting point for the multivariate fast Gauss transform. Because of the proucts in 7) it is ifficult to estimate the approximation error. Therefore we will consier a secon approximation scheme to the multivariate Gauss kernel, for which the error estimates for the one-imensional Gaussian apply. 4. Raially symmetric approximation The erivation of the raially symmetric approximation begins with the oneimensional Chebyshev series of the Gaussian ) x y exp exp 4 ) ) x y δ δ ) ) δ x y δ G p, 4 x y + R p, 4, where R p is the remainer of the Gaussian, efine in 10), an ) δ x y G p, 4 p n0 ) ) δ x y E n T n 4. 9) Note that if x, y C then the argument to the Chebyshev polynomials is less than unity, therefore the estimates of the remainer erive in Section apply. Furthermore, because of 4), the sum contains only terms of even orer, thus 9) is in Π p. This follows irectly from the following lemma. Lemma 4.1 If n is even then T n x y ) Π n. Proof. For even n, T n z) is a polynomial in z, hence there are coefficients a k such that ) n/ x y T n a k x y k. From the binomial formula it follows that 1 k! x y k 1 [ x1 y 1 ) x y ) ] k k! x y) γ [α + β)]! 1) β x α y β. γ! α + β)!α!β! γ k k0 α+β k Combining the last two results implies the assertion. 1

13 Because of the Lemma, we have coefficients E p) α,β such that ) δ x y G p, 4 E p) α,β δ)t αx)t β y). α+β p Unfortunately, the coefficients E p) α,β are not proucts of single-variate coefficients, thus the tensor-prouct form of the Chebyshev series is lost, an therefore the raially-symmetric approximation is not attractive for numerical computations. It will be important for error estimates, which will be iscusse in Section Multivariate Fast Gauss Transform Replacing the Gauss kernel in 1) by G p leas to the multiimensional Gauss transform. Similar to the one-imensional case, the potential has the expansion Φ p x) α p λ α T α x). 30) The expansion coefficients are etermine by the MtL-translation λ α E α,β µ β, α Sp, 31) where the moments are given by µ β β+α p M T β x j )q j, β Sp. 3) j1 We illustrate the issues of evaluating the MtL-translation for the case of three imensions. Writing out all inices explicitly, the translation is λ α1,α,α 3 p α β 30 p α β 3 E α3,β 3 β 0 p α β β 3 E α,β β 10 E α1,β 1 µ β1,β,β 3. 33) The expansion coefficients can be compute by a sequence of one-imensional transforms. Set λ 0) µ an compute λ 1) α 1,α,α 3,β,β 3 λ ) α 1,α,α 3,β 3 λ 3) α 1,α,α 3 p α β β 3 β 10 p α β 3 β 0 p α β 30 E α1,β 1 λ 0) β 1,β,β 3 E α,β λ 1) α 1,α,α 3,β,β 3 E α3,β 3 λ ) α 1,α,α 3,β 3, 13

14 then λ λ 3). Unfortunately, λ 1) has five inices an λ ) has four inices instea of just three. Thus their computation is expensive because of the large number of terms that have to be compute. It is possible to better exploit the Kronecker prouct form of the Gauss kernel by incluing more terms in the computation of the λ s. In the three-imensional case this can be accomplishe by evaluating the following expression λ α1,α,α 3 p α 1 α β 30 p α 1 β 3 E α3,β 3 β 0 In this case λ 1) epens only on three inices β 10 p β β 3 E α,β β 10 p β β 3 λ 1) α 1,β,β 3 E α1,β 1 λ 0) β 1,β,β 3. E α1,β 1 λ 0) β 1,β,β 3. 34) If the λ 1) s are compute for inices α 1, β, β 3 ) S 3 p it is sufficient to compute the λ 0) s for inices in S 3 p, since the inices in the above sum satisfy β 1 + β + β 3 p. Now λ ) is given by p α 1 β 3 λ ) α 1,α,β 3 E α,β λ 1) α 1,β,β 3. β 0 If the λ ) s are compute for inices α 1, α, β 3 ) Sp 3 then by the same token, we only nee the coefficients of λ 1) in Sp. 3 Finally, λ λ 3) is given by p α 1 α λ 3) α 1,α,α 3 E α3,β 3 λ ) α 1,α,β 3. β 30 As before, an inspection of the upper boun in the summation shows that we only nee the coefficients of λ ) in S 3 p to compute the coefficients of λ 3) in S 3 p. The same principle generalizes to any imension, thus the algorithm in the general case is 14

15 Multivariate MtL-translation for j 1 : for α 1,..., α j, β j+1,..., β ) Sp n p α 1... α j 1 β j+1... β en en λ j) α 1,...,α j,β j+1,...,β n β j0 E αj,β j λ j 1) α 1,...,α j 1,β j,...,β 5 Complexity an Implementation Details It is well known that the carinality of Sp is given by ) p + #Sp + p)!.!p! A boun for #Sp can be obtaine with Stirling s formula Equation in[1]) n! πnn n exp n + θ ), n > 0, 1n for some θ 0, 1). To obtain an upper boun for #S p set θ 1 for the factorials in the numerator an θ 0 in the enominator. This leas to #S p µ 0 1 p + 1 ) p ) 1 + p, 35) p) where µ 0 exp1/1)/ π Since the mile term in the above estimate is boune, the carinality of Sp grows like a polynomial in the imension if p is fixe. However, the subsequent error analysis shows that the expansion orer must be increase when the imension is increase. We will provie more accurate complexity estimates later on. We will now iscuss the complexity of the three translations in the FGT algorithm. MtL translation. The sum in the inner loop of the MtL operation is compute once for every α 1,..., α j, β j+1,..., β ) Sp. The number of terms in the sum is variable, but boune above by p + 1. Finally, the j-loop is execute once for every irection. We choose the number of multiplications as a measure for the complexity, thus we see that there are ) p + N MtL #Spp + 1) p + 1) operations. 15

16 QtM an LtP translations. The computation of the moments is one by a straightforwar evaluation of the sum 3). Likewise, the computation of the potentials is one by evaluation of the sum 30) for every evaluation point. Hence the complexities for these operations are N QtM N T M an N MtL N T N, respectively, where N T is the cost to compute T α x) for a given x an α S p. The obvious metho for evaluating the multivariate Chebyshev polynomial is the following algorithm. Direct computation of T α x), α S p. 1. Compute single-variate Chebyshev polynomials T k x j ) for k 0,..., p an j 1,...,.. Set T α x) T α1 x 1 )... T α x ) for α S p. The cost of Step 1 is of lower orer an will be be neglecte. The main contribution comes from Step which entails ) p + N T D 1)#Sp 1) multiplications. Since T α must be compute once for every source- an every evaluation point it is worthwhile to optimize this calculation. An alternative algorithm is base on the observation that multi inices can be generate by recurrence. From the efinitions it is clear that p Sp j { } ˆα, α j ) : ˆα S j 1 p α j. α j0 This motivates the following algorithm. Computation of T α x), α S p by recurrence. 1. Compute single-variate Chebyshev polynomials T k x j ) for k 0,..., p an j 1,...,.. Set T 1) α 1 T α1 x 1 ), for α 1 0,..., p. 3. Compute for j : for α j 0 : p for ˆα S j 1 p α j T j) ˆα,α T j) α j x j )T j 1) ˆα en en en 16

17 To assess the complexity of this algorithm we count the number of multiplications in Step 3. N T R p #S j 1 p α j j α j0 j α j0 p ) p + j 1 αj ) p + j p j p α j ) p + j j ) p + j p + 1 j 1 j0 p j ) ) p 0 ) lower orer terms. In lines 3 an 5 of the above calculation we have use a well known property of binomial coefficients, [1], Formula Comparing the irect computation of T α x) an the computation by recurrence leas to the ratio N T R p N T D p + 1) 1), which shows that the computation by recurrence can be significantly faster than the irect computation. Neglecting lower orer terms, the total cost of the fast Gauss transform is N F GT N MtL + N T R N + M) ) ) p + p p + 1) + M + N). In a typical situation N, M, p, thus the cost of computing the QtM an LtP translations ominates over the cost of computing the MtL transform. 6 Multivariate Approximation Error To estimate the error introuce by the fast Gauss transform we precee in a similar manner as in Section 3.4. We begin with the expansion of the error Φx) Φ p x) ˆλ α T α x). α α N 0 With a calculation similar to the one-imensional case in Section 3.4 we obtain ) 1 π ) Φ Φ p L ˆλ w γ α ˆλ l α w π ˆR p L µ w L w l w, 17

18 where ˆR p is the resiual ˆR p x, y) α,β) I p E α,β T α x)t β y). Here, the inex set Ip enotes the inices that are inclue in the MtL translation, see equation 34). In the estimate of the resiual below, it is not necessary to completely characterize this set. The only important property is that by construction S p I p hols. There are fewer terms in ˆR p than there are in R p, efine by 8), hence it follows from orthogonality that ˆR p L w L R p w L w L 36) w. Because of the best approximation property we can use the raially symmetric approximation as an upper boun. R p L w L w ) R p δ L R x y w L p, w 4 wx)wy) xy C C ) δ π max R x y p, x C 4 y C. Thus estimate 11) for the remainer of the one imensional Gaussian can be applie. It follows that Φ Φ p L w ) 1 δπ exp pκ δ 4 p)) 8 1 exp κ δ 4 p)) µ l. 37) w We now iscuss how the orer shoul be ajuste to the imension such that the error is controlle. We consier a relationship of the form p µ 38) for some constant µ > 0. Then simple algebra leas to the estimate ) 1 δπ exp pκ δ 4 p)) 8 1 exp κ δ 4 p)) c ) 1 [ δπ exp ln µκ 8 )] ) δµ 4 39) where c 1 expδµ/4)) 1 is an unimportant constant. For error control, the argument to the exponential function must not be positive. This leas to a conition on µ ) δµ µκ ln. 4 Since κ is positive an monotonically increasing this conition can be satisfie for any given δ. With conition 38) the complexity of the FGT oes no longer 18

19 grow algebraically in the imension. In fact, estimate 35) implies that the growth is exponential #Sp µ 1 µ with constants µ 1 µ ) 1 µ an µ 1 + µ) 1 + µ) 1 µ. 7 Numerical Results We have implemente the fast Gauss transform for arbitrary imension an expansion orer. The coe was written in C, compile with the gcc compiler an teste on a single core of a ual core AMD Opteron with 400 MHz clock spee. The system s memory of eight Gbyte was sufficient to run all examples escribe below in core memory. We compute the Gauss transform for the case that the source- an evaluation points coincie. Their location an the source strengths are generate ranomly with the C-library routine ran). The output of the ranom number generator is normalize such that the points are in C an the source strengths are in the interval [0, 1]. We o not re-see this routine between successive runs which ensures that we obtain the same sources when comparing results with ifferent parameters. To assess the accuracy we compute the potentials at the source points using the irect ON ) algorithm an compute the maximal relative error on the noe points max Φx i) Φ p x i ) i1,...,n ɛ : max Φx i) i1,...,n Note that our theoretical results mostly pertain the L w-error. However, this quantity is not irectly available an hence we resort to the maximal error. Since the expansion coefficients of the Gauss kernel E k,l have no close-form expression, we investigate computing the integrals 14) by Gauss-Chebyshev quarature an replacing the coefficients by E p) k,l efine 18). This resulte in only marginal ifferences in the results. To illustrate the behavior of the fast Gauss transform, we have performe three experiments, which are escribe as follows. 1. Test of the error as a function of the variance δ an expansion orer. The results are shown in Figure. The imension is three an there are 10,000 points. The error ecreases super-exponentially with the orer an the rate of ecrease eteriorates if the variance is reuce. Thus the behavior is similar to the Chebyshev coefficients illustrate in Figure 1. The CPU time for the fast Gauss transform varies between 0.03 sec p4) an 1.63 sec p48), the CPU time for the irect computation is about

20 sec. Note that the plots only show errors for even values of p. The error for the next-larger o value is harly smaller because there are only even terms in the Chebyshev series of the Gaussian. If o values of p were inclue, the plots woul exhibit staircase-like behavior rel. error 10 4 δ1/4 δ1/16 δ1/ orer Figure : Error vs. orer for various values of δ an fixe 3.. Depenence of the CPU time as a function of the number of points an the imension for variance δ 0.5 fixe. We select the expansion orer such that the relative error is not greater than 0.0. By experimentation we fin empirically that this can be accomplishe if the orers are chosen as in Table 1. The results in Figure 3 clearly show the linear scaling in Dimension Orer Table 1: Parameters for results of Figure 3. the number of points for a given imension. For a higher imension the constant factor is larger, resulting in parallel lines on the logarithmic plot. The figure also isplays the quaratic scaling of the irect evaluation. Here the constant factor epens in a much weaker way on the imension. As a result, the fast Gauss transform is only faster if the number of points is sufficiently large, especially in high imensions. As it is evient from the plot, the cross over for imensions less or equal eight occurs before, 000 an for imension ten at about N 30,

21 times) number points 4F) 4D) 6F) 6D) 8F) 8D) 10F) 10D) Figure 3: CPU time vs. number of points, δ Depenence of the CPU time as a function of the imension for a fixe number of points an fixe expansion orer. To compensate for bigger errors in higher imensions we increase the variance by the formula δ 0.. Accoring to the theory, the L w-error for a fixe expansion orer shoul still increase with the imension. However, as Figure 4 suggests, the relative maximum oes not appear to grow. Accoring to our complexity estimates the CPU time shoul grow like a polynomial. This is well reprouce in the logarithmic plot, that shows that the curve for the CPU time approximates a straight line well when the imension is high enough. 8 Acknowlegments The proof of Lemma.1 was suggeste by an anonymous reviewer. Alexaner Weckiewicz research was supporte by a grant from John McCaw. References [1] M. Abramowitz an I. Stegun, eitors. Hanbook of mathematical functions. U.S. Govt. Print. Off., [] F. Anersson an G. Beylkin. The fast Gauss transform with complex parameters. J. Comput. Phys., 031):74 86,

22 rel. Error x) Time in sec. o) imension Figure 4: CPU time an error vs. imension, when p 9, N 10 4 an δ 0.. [3] B. Baxter an G. Roussos. A new error estimate of the fast Gauss transform. SIAM J. Sci. Statist. Comput., 41):57 59, 00. [4] M. Broay an Y. Yamamoto. Application of the fast Gauss transform to option pricing. Mamagement Science, 498): , 003. [5] A. Elgammal, R. Duraiswami, an L.S. Davis. Efficient kernel ensity estimation using the fast Gauss transform with applications to color moeling an tracking. IEEE Trans. Pattern Anal. an Mach. Intel., 511): , 003. [6] L. Greengar an J. Strain. The fast Gauss transform. SIAM J. Sci. Statist. Comput., 1:79 94, [7] L. Greengar an X. Sun. A new version of the fast Gauss transform. Doc. Math. J. DMV, Extra Volume ICM 1998, III: , [8] O. Livne an B. Wright. Fast evaluation of smooth raial basis function expansions. Electronic Transactions on Numerical Analysis, 3:63 87, 006. [9] M. Mahaviani, N. e Freitas, B. Fraser, an F. Hamze. Fast computational methos for visually guie robots. In Proc. IEEE Intnl. Conf. on Robotics an Automation, pages , 005. [10] V.C. Raykar, C. Yang, R. Duraiswami, an N. Gumerov. Fast computation of sums of Gaussians in high imensions. Technical Report CS-TR-4767,

23 Department of Computer Science, University of Marylan, CollegePark, 005. [11] D.W. Scott. Multivariate Density Estimation. Wiley, 199. [1] X. Sun an Y. Bao. A Kronecker prouct representation of the fast Gauss transform. SIAM J. Matrix Anal. Appl., 43): , 003. [13] X. Wan an G.E. Karniaakis. A sharp error estimate for the fast Gauss transform. J. Comput. Phys., 19:7 1, 006. [14] C. Yang, R. Duraiswami, N.A. Gumerov, an L. Davis. Improve fast gauss transform an efficient kernel ensity estimation. In Proceeings. Ninth IEEE International Conference on Computer Vision, pages , 003. [15] X. Zhu, Z. Ghahramani, an J. Lafferty. Semi-supervise learning using Gaussian fiels an harmonic functions. In 0th International Conference on Machine Learning,