Faster Gaussian Summation: Theory and Experiment

Transcription

1 Faster Gaussian Summation: Teory and Experiment Dongryeol Lee College of Computing Georgia Institute of Tecnology Atlanta, GA 3033 Alexander Gray College of Computing Georgia Institute of Tecnology Atlanta, GA 3033 Abstract e provide faster algoritms for te problem of Gaussian summation, wic occurs in many macine learning metods. e develop two new extensions - an OD p Taylor expansion for te Gaussian kernel wit rigorous error bounds and a new error control sceme integrating any arbitrary approximation metod - witin te best discretealgoritmic framework using adaptive ierarcical data structures. e rigorously evaluate tese tecniques empirically in te context of optimal bandwidt selection in kernel density estimation, revealing te strengts and weaknesses of current state-of-te-art approaces for te first time. Our results demonstrate tat te new error control sceme yields improved performance, wereas te series expansion approac is only effective in low dimensions five or less. Fast Gaussian Summation Kernel summations occur ubiquitously in bot old and new macine learning algoritms, including kernel density estimation, kernel regression, radial basis function networks, spectral clustering, and kernel PCA Gray & Moore, 00; de Freitas et al., 006. Tis paper will focus on te most common form Gx q = N Kδ qr in wic we desire te sum for r= M different query points x q s, eac using N reference qr points x r s weigted by > 0. Kδ qr = e is te Gaussian kernel, were δ qr = x q x r wit scaling parameter, or bandwidt. For concreteness we will take as our main example kernel density estimation Silverman, 986, te most widely used distributionfree metod for te fundamental task of density estimation. Because te Gaussian kernel as infinite tail, we must pursue approximation in order to acieve runtimes less tan tat of exaustive summation. Our goal is to compute eac Gx q as quickly as possible e Gx q Gx q Gx q wile ensuring tat x q ɛ were ɛ is a user-supplied error tolerance. In practice we wis to perform tis computation for a range of bandwidts, from small to large, for example in order to do optimal bandwidt selection by cross-validation. Te basic idea in kernel summation is to approximate te kernel sum contribution G x q of some subset of te reference points X of size N, lying in some compact region of space wit centroid x, to a query point. In more efficient scemes te approximate contribution is made to an entire subset of te query points X of size N lying in some region of space, wit centroid x. Metods from computational pysics. Te successful Fast Multipole Metod FMM Greengard & oklin, 987 developed for te Coulombic kernel, used multipole expansions for te continuous approximation, octtrees a form of ierarcical grid for te discrete data structure, and an explicit level-by-level enumeration of te node-node comparisons. Since te expansions only old locally, Greengard & oklin, 987 developed a set of tree translation operators for converting between expansions centered at different points in order to create teir ierarcical algoritm. Te original Fast Gauss Transform FGT Greengard & Strain, 99 was developed in te same style, but for te Gaussian kernel using two different expansions. Te first one is te multivariate Hermite expansion wic expresses a sum as an expansion about a representative centroid x in te reference region : In tis paper we use te multi-index notation Greengard & Strain, 99; Yang et al., 003. A multiindex α = α, α,..., α D is a D-tuple of integers. For any multi-index α, β and any x D, α = α + α + + α D, = α!α! α D!, 3x α = x α x α x α D, 4D α = α α α D D, 5α + β = α +

2 Gx q = x r Tis can be re-written as: Gx q = x r xr x α xr x αα xq x xq x α as a Taylor local expansion about a representative centroid x in te query region. Dual-tree recursion. In terms of discrete algoritmic structure, te dual-tree framework of Gray & Moore, 00, in te context of kernel summation, generalizes all of te well-known algoritms, including te Barnes-Hut algoritm Barnes & Hut, 986, te Fast Multipole Metod Greengard & oklin, 987, Appel s algoritm Appel, 985, and te SPD Callaan & Kosaraju, 995: it is a node-node algoritm considers query regions rater tan points, is fully recursive, can use adaptive data structures suc as kdtrees, and is bicromatic can specialize for differing query and reference sets. Te idea is to represent bot te query points and te reference points respectively wit a tree and recurse on a pair of query and reference node. Tis is sown in dept-first form in Figure toug it can also be performed using a priority queue Gray & Moore, 003a. It was applied Dualtree, if Can-approximate,, ɛ Approximate,, return if leaf and leaf, DualtreeBase, else Dualtree.l,.l,Dualtree.l,.r Dualtree.r,.l,Dualtree.r,.r Figure : Generic structure of a dual-tree algoritm. to te problem of kernel density estimation in Gray & Moore, 003b using a finite-difference approximation, a variant of a monopole approximation. Partially by avoiding series expansions, wic depend explicitly on te dimension, te result was te fastest suc algoritm for general dimension, wen operating at te optimal bandwidt. However, wen performing cross-validation to determine te initially unknown optimal bandwidt, bot suboptimally small and large bandwidts must be evaluated. Tis finite-differencebased metod tends to be efficient around or below te optimal bandwidt, and at very large bandwidts, but for intermediately-large bandwidts it suffers. β,, α D +β D, 6α β = α β,, α D β D, were i is a i-t directional partial derivative. e define α > β if α i > β i, and α p for p Z if α i p for i D. e define te Hermite functions nt by nt = e t H nt, were te Hermite polynomials H nt are defined by te odrigues formula: H nt = n e t D n e t, t. Te multivariate Hermite function is ten defined as a product of its univariate versions: αt = D αd t. d= Automatic error control. Among te existing metods, dual-tree metod is te only one to automatically acieve te user s error tolerance ɛ. Oter metods are overridden wit many tweak parameters wose values ave to be canged simultaneously wit little or no guidance. Tese parameters waste uman time and offer no error tolerance guarantee unless verified by a procedure tat computes density estimate exaustively. Tis issue is discussed in Section 7. Series expansion. Expansions in Greengard & Strain, 99 require te computation of Op D subterms. ile effective in te context of computational pysics problems, tis is problematic in statistical/data mining applications, in wic D may be larger tan or 3. Lee et al., 006 developed te translation operators and error bounds necessary to perform te original FGT-style Op D approximation witin te context of te dual-tree framework, demonstrating te first ierarcical fast Gauss transform. However, te new algoritm sowed efficiency over any of te aforementioned metods over te entire range of bandwidts necessary in cross-validation, only in very low dimensions 3 or less. Te Improved Fast Gauss Transform IFGT Yang et al., 003 introduced a rearranged series approximation requiring OD p subterms, wic seemed promising for iger dimensions wit an associated error bound, wic was unfortunately incorrect. Te IFGT was based on a flat set of clusters and did not provide any translation operators. Tis paper. e demonstrate for te first time te OD p rater tan Op D expansion of te Gaussian kernel different from tat of te IFGT witin a ierarcical dual-tree algoritm. e also introduce a more efficient mecanism for automatically acieving te user s error tolerance wic works wit bot discrete and continuous approximation scemes. e evaluate tese new tecniques empirically on real datasets, revealing te strengts and weaknesses of te main current approaces for te first time. OD p and Op D Expansions For concreteness, we first discuss te difference between Op D and OD p expansion by approximating te -D Gaussian kernel using its Hermite expansion at order p =. Its Op D expansion is: e xq xr = xqd x d 0 + xrd x d xqd x d + d= xq x xq x xr x xq x 0 xq x + xr x xq x xq x 0 + xr x xr x xq x xq x

3 Figure 3: HL operator converts te far field expansion centered at x into te local expansion centered at x. Figure : Left: HH operator combines two finer scaled far field expansions centered at x and x into a coarser scaled one centered x. igt: LL operator converts te local expansion centered at x into two finer scaled ones centered at x and x. On te oter and, OD p expansion uses graded lexicograpic ordering Yang et al., 003, and yields: xq xr e xq x xq x xr x xq x xq x 0 + xr x xq x xq x 0 In bot cases, D d= xqd x d αd forms basis functions for te expansion. In actual codes, te factors in front of tese basis functions are stored as coefficients in te corresponding reference node. Op D expansion requires exactly p D coefficients, wile OD p one requires D+p D. If we iterate over all reference points in te reference node wit teir weigts taken N N into account, we store:, w xr x r, N r= N w xr x N r,, r= N r= r= xr x r= xr x N, r= r= xr x and w xr x r for Op D and OD p expansions respectively. Te Taylor expansion works similarly, except tat te coefficients are stored in te corresponding query node. 3 Translation Operators Since te properties of te Gaussian kernel do not require tat approximation be made in te local fasion, te original FGT used a flat grid wit only HL operator wose associated incorrect error was corrected by Baxter & oussos, 00. Lee et al., 006 derived two additional translation operators necessary for a ierarcical FGT and te associated error bounds for Op D expansion of Hermite/Taylor coefficients. e briefly review all tree translation operators. Te first translation operator transfers te contribution of a reference node into te Taylor series cen- tered about x in a query node. Lemma. Hermite-to-local HL translation operator in Lemma. in Greengard & Strain, 99: Given a reference node, a query node, and te Hermite expansion centered at a centroid x of : Gx q = xq x A α α were Aα = N r= xr x α, te Taylor expansion at te centroid x of is given by: Gx q = xq x B β β were Bβ = β β 0 A α α+β x x Te next operator allows efficient precomputation of te Hermite moments in te reference tree in a bottomup fasion from its cildren. Lemma. Hermite-to-Hermite HH translation operator: Given te Hermite expansion centered at a centroid x in a reference node : Gx q = A xq x α α tis same Hermite expansion sifted to a new location x of te parent node is given by: Gx q = xq x A γ γ were A γ = 0 α γ γ A α. γ 0 x x γ α. Te final operator combines te approximations at different scales troug one breadt-first traversal. Lemma 3. Local-to-local LL translation operator: Given a Taylor expansion centered at a centroid x of a query node : Gx q = xq x B β β te β α β B β β 0 Taylor expansion obtained by sifting tis expansion to te new centroid x of te cild node is: Gx q = [ ] x x β α xq x α. 4 Error Bounds for OD p Expansions Because Hermite/Taylor expansions are truncated after a finite number of terms, we incur an error in approximation. In order to bound te total approximation error, we need one error bound for eac translation operator. In Lee et al., 006, te Hermite and

4 te Taylor expansion were treated as products of D univariate Hermite/Taylor expansions. Te trailing sum in eac univariate expansion was bounded using te property of infinite geometric series, wic in turn limited te size of te query/reference node for pruning to be valid. Here, we use te same tranlsation operators, but instead view eac expansion as a vector function and use te OD p expansion advocated in Yang et al., 003. Te new error bounds based on tis new expansion sceme depend on te multidimensional Taylor s Teorem, and effectively eliminate te node size restriction imposed by te Op D expansion Greengard & Strain, 99; Lee et al., 006. Teorem. Multidimensional Taylor s Teorem: Let O D be an open set. Let x O and f be a function wic is n times differentiable in O. For any x O, tere exists θ wit 0 < θ < suc tat fx = Dα fx x x α + last term n = Dα fx + θx x x x α. Te Dα fx + θx x x x α is called te Lagrange remainder and n D α fx + θx x D xd x d α d sup 0<θ< d= Te first lemma gives an upper bound on te absolute error on estimating a reference node contribution by evaluating a truncated Hermite expansion. Te second lemma gives an upper bound on te absolute error incurred from approximating te contribution of a reference node by evaluating te Taylor series formed via direct local accumulation of eac reference point. Lemma 4. Given a query node, a reference node wit an Hermite expansion about its centroid x : Gx q = xq x A α α, and xq, te absolute truncating error after taking te first OD p terms is min e 4 D+p r p D bounded by: E DH p = r D p p were pd! pd! x r = max r x x and p = p mod D. r Proof. By Teorem and te triangle inequality, Gx q xq x A α α x r e Kδ qr min 4 max x q,x r α α xq x xq x r α D xr x α D x rd x d d= x rd x d α d d= α d e e D min 4 x rd x d α d d= min min 4 r α e 4 D+p D r p r pd! D p p D! p Lemma 5. Given te following Taylor expansion about te centroid x of a query node : Gx q = xq x B β β were Bβ = β β 0 x x A α α+β, for any xq, te absolute error due to truncating te series after OD p terms is min e 4 D+p r p D bounded by: E DL p = r D p p were pd! pd! x r = max q x x and p = p mod D. q Proof. Te derivation is similar to one in Lemma 4. Te final lemma gives an upper bound on te absolute error incurred by approximating te reference node contribution by te Taylor expansion converted from te truncated Hermite expansion. Lemma 6. A truncated Hermite expansion about te centroid x of a reference node given by: Gx q = xq x A α α as te following Taylor expansion about te centroid x of a query node : Gx q = xq x C β β were Cβ = β α<p β 0 A α α+β x x. Te absolute truncation error after taking OD p terms is: min 4 D+p D e = r D p p pd! p D! I r r were r = max r p + r p x q x q x, x r = max r x x, p = p mod D and r { 0, 0 x Ix = p, oterwise. Proof. Let E = xq x β β and E = Ten, Gx q E + E β β p α p xr x E HL p D+p D αα+β x x β x x r xq x β β x x A α α+β

5 min e 4 D+p r p D Clearly, E r D p p. In addition, pd! pd! e e e E max D x rd x d d= xq x r α+β x q,x r min 4 α+ α+β D x rd x d d= min 4 D x rd x d d= min 4 α d x qd x d α d x qd x d β d xqd x d min e 4 D+p D p r r D p p pd! p D! min e 4 D+p D r pd! D p p D! p DUALTEE BASE DIECTL + LL + EVALL β d α+β α+β β d D α d x qd x d x rd x d α d d= β d D xqd x d d= β d p D+p I r r D r DIECTM HL + LL + EVALL Figure 4: Four ways a reference node can send its contributions to a query node using te original FGT style pruning. In te clockwise order starting from te top left, exaustive computation feeference/query points, multipole evaluation many reference/few query points, direct Taylor accumulation feeference/many query points, HL-translation many reference/query points. 5 New Error Guarantee ule Let us first revisit our metod of automatically guaranteeing te user s error tolerance ɛ defined in Section. e now specify te function Can-approximate,, ɛ, wic only as local information contained in te query node and te reference node available to it, but must guarantee a global error criterion. In te dual-tree finite-difference algoritm DFD Gray & Moore, 003b, te function Approximate, approximates te contribution of to eac query point x q in, G x q, by G x q = Kδ K = max +Kδmin, were = x r and δ min and δmax are lower and upper bounds on te distance between x q and x r, respectively. Tese distances are easily obtained using te bounding boxes of te nodes. By using tese bounds DFD algoritm maintains a running lower bound G min on G x q wic olds for all x q. In Section 4, we laid out more approximation metods in addition to finite-difference approximation FD: evaluating a truncated Hermite expansion centered at x DH, forming a truncated Taylor expansion centered at x using eac reference point DL, and forming an approximated truncated Taylor expansion centered at x by converting te truncated Hermite expansion centered at x HL. Te following specifies Canapproximate,,ɛ, wic incorporates te new approximation metods. Teorem. Given te following metods for approximating te contribution of a reference node : A = {EX, DH, DL, HL, FD} were DH, DL, HL, and FD are denoted as above, and EX for exaustive computation, A A wit a maximum absolute error of E A can be used to guarantee te global error tolerance ɛ if: E A ɛ Gmin were = N. r= Proof. Given x q, suppose Gx q was computed using k reference nodes i s, wose contribution was approximated using A i. By te triangle inequality: Gx q Gx q = k G i x q G i x q k G i x q G i x q i= k E Ai k i= i= i= iɛ Gmin ɛgmin ɛgx q Tis neule generalizes te previous local approximation condition Gray & Moore, 003b: Kδ min Kδmax /Gmin ɛ were E F D = Kδ min Kδmax. Clearly, E EX = 0, and E DH, E DL, and E HL are given as Lemma 4, 5, 6 respectively. Te approximation rule above essentially gives eac reference node a maximum relative error proportional to te sum of te weigts of reference points it contains. In considering te i-t reference node contribution, wen A i = EX, te maximum allowable

6 relative error of iɛ N is not used up; Oterwise, if G min > 0, pruning requires only a relative error of i ɛ were i = EAi. Our new approximation ɛg min rule notes tat te portion of te weigts not used to cover te incurred pruning error can be stored into a field T initialized to zero before te computation and denoted. T ereon in eac query node to use tem in future pruning opportunities. Te first case yields i as te leftover, wile te second case pruned case yields i i. Given A A wit te maximum absolute error of E A, we now modify te approximation condition to: E A /G min ɛ+t. Solving for T yields: T EA. enever a pruning is attempted, te ɛg min modified algoritm will evaluate te rigt andside of te inequality. If te evaluated value is negative, it represents te leftover token after pruning is performed and. T of te current query node will be incremented by E A ɛg. If positive, it represents min te required extra token from te. T slot of te current query node, in order to prune te given query and reference node pair. If. T T, pruning succeeds and. T is decremented by EA ɛg min 6 New Dual-tree Algoritm. e first introduce an extra field in eac query node storing contributions from reference nodes obtained by finite-difference approximation and direct Hermite evaluations. Te contributions from Taylor coefficients obtained via direct local accumulation and HL translation opeator will be accounted for during te post-processing step. In preprocessing, we construct two trees, one for te query dataset and one for te reference dataset. In tis paper an efficient form of spere-rectangle trees Katayama & Sato, 997 is used, wit idea of caced sufficient statistics as in mrkd-trees Deng & Moore, 995. Te Hermite moments of order PLIMIT is pre-computed for te reference tree. For te experimental results, we ave fixed PLIMIT = 8 for D =, PLIMIT = 6 for D = 3, PLIMIT = 4 for D = 5, PLIMIT = for D = 6. e presume tat PLIMIT = for D > 6. During te recursive function call DITO, an optimized version of finite-difference pruning is first attempted. In case of failure, we attempt FMMtype pruning in wic we coose te ceapest operation given a query node and a reference node from te followings: direct Hermite evaluation DIECTMHermite coefficients, truncation order, query point, direct local accumulation DIbuildInternal n = empty node {, } = Partition into two n.l = buildeferencetree n.r = buildeferencetree n.mcoeffs = HHn.l.mcoeffs, n.l.x, n.x n.mcoeffs+ = HHn.r.mcoeffs, n.r.x, n.x return n buildleaf n = empty node n.mcoeffs = Compute te Hermite series of order PLIMIT using eac x r centered at x return n buildeferencetree if size < leaf tresold, return buildleaf else, return buildinternal Figure 5: Building te tree from te reference dataset. ECTLte set of reference points, truncation order, query node centroid, HL translation HLHermite coefficients, truncation order, reference node centroid, query node centroid, and exaustive computations. ougly, direct Hermite evaluations at eac x q is ON D pdh +, direct local accumulation ON D pdl+, HL translation OD phl+, an exaustive metod ODN N. In our algoritm, if an exaustive metod is selected, we let te recursion continue, oping pruning can occur in te finer level of recursion. It is possible to and-tune te exact cutoffs for determining te optimal coice, but tese roug approximations seem to work well. In te post-processing step, we perform a breadt-first traversal of te query tree. For an internal node, its Taylor expansion is sifted to te centers of its cildren via LL translation operator LLTaylor coefficients, truncation order, old query node centroid, new query node centroid; te estimated contributions are propagated downward directly to its cildren as well. For a leaf node, we evaluate te Taylor expansion at every x q using EVALLTaylor coefficients, truncation order, query node centroid, query point and add in te far-field contribution. 7 Experiments and Conclusions e empirically evaluated te runtime performance of six algoritms on six real-world datasets astronomy -D, pysical simulation 3-D, parmaceutical 5- D, biology 7-D, forestry 0-D, image textures 6- D scaled to fit in [0, ] D ypercube, for kernel density estimation at every query point wit a range of bandwidts, from 3 orders of magnitude smaller tan optimal to tree orders larger tan optimal, according to te standard least-squares cross-validation scores Silverman, 986. In our case, te set of reference points

7 bestmetod,, maxerr p DH = te smallest p PLIMIT suc tat E DH p maxerr, oterwise p DH = p DL = te smallest p PLIMIT suc tat E DLp maxerr, oterwise p DL = p HL = te smallest p PLIMIT suc tat E HLp maxerr, oterwise p HL = c DH = N D p DH +, c DL = N D p DL+ c HL = D p HL+, c Direct = DN N c = minc DH, c DL, c HL, c Direct if c = c DH, return {DH, p DH, ɛ DH p DH} else if c = c DL, return {DL, p DL, ɛ DLp DL} else if c = c HL, return {HL, p HL, ɛ HLp HL} else, return {DIECT,, } Figure 6: Coosing te FMM-type approximation wit te least cost for a query and reference node pair. is te same as te set of query points. All datasets ave 50K points so tat te exact exaustive metod can be tractably computed. e set te tolerance ɛ = 0.0. e compare: FGT Fast Gauss Transform Greengard & Strain, 99, IFGT Improved Fast Gauss Transform Yang et al., 003, DFD dualtree wit finite-difference Gray & Moore, 003b, DFDO dual-tree wit finite-difference and improved error control Section 3., DFTO dual-tree wit Op D expansion Lee et al., 006 and improved error control, and DITO dual-tree wit OD p expansion and improved error control. All times wic include preprocessing but exclude parameter selection time are in CPU seconds on a dual Intel Xeon 3 GHz wit Gb of main memory/ Mb of CPU cace. Codes are in C/C++, compiled under O6 funroll loops flags on Linux kernel Te measurements in columns two to eigt are obtained by running te algoritms at te bandwidt k were 0 3 k 0 3 is te constant in te corresponding column. Te dual-tree algoritms all acieve te error tolerance automatically. e also note tat te FGT uses a different error tolerance definition: Gx q Gx q τ. e first set τ = ɛ, alving it until te error tolerance ɛ was met. For te IFGT, we created an automatic sceme to tweak its multiple parameters based on recommendations given in te paper and software documentation: For D =, use p = 8; for D = 3, use p = 6; set ρ x =.5; start wit K = N and double K until te error tolerance is met. en tis failed to meet te tolerance, we resorted to additional trial and error by and. e are primarily concerned wit te sum of te times over all te bandwidts, sown in te last column of te table. Entries in te tables of X denote cases were te algoritm exausted AM and caused a segmentation fault. Entries of denote cases were no setting of te algoritm s parameters was able to satisfy te DITOBase, forall x q forall x r c = K x q x r, G min G max q G max q q + = c, + = c, q + = c = = min q Gmin q,. T + =, G min G max = max q Gmax q DITO, dl = K δ max, du = K δ min K δ T = ` min K δ max ɛg min // Optimized finite difference pruning first, if T 0 or. T T. T + = T, G min + = dl, G max + = du, + = 0.5dl + du +, return else // FMM-type pruning {A, p, E A} = bestmetod,, ɛ +. T if A = DH forall x q G min q + = EVALM.mcoeffs, p, x, x q else if A = DL.lcoeffs+ = DIECTL x r, p, x else if A = HL.lcoeffs+ = HL.mcoeffs, p, x, x if A DIECT T = EA `. ɛg min T,. T = T,G min + = dl, G max return if leaf and leaf, DITOBase, else DITO.l,.l,DITO.l,.r DITO.r,.l,DITO.r,.r + = du, Figure 7: Te main procedure implementing a new error-control and OD p expansion. DITOPost if leaf forall x q q + = EVALL.lcoeffs,.p L,.x, x q + else.l+ =,.r + =.l.lcoeffs+ = LL.lcoeffs,.p L,.x,.l.x.r.lcoeffs+ = LL.lcoeffs,.p L,.x,.r.x DITOPost.l,DITOPost.r Figure 8: Combining different types of approximations on different scales, using a breadt-traversal. error tolerance. Our results demonstrate tat te OD p expansion elps reduce te computational time on datasets of dimensionality up to 5. For example, on te -D dataset, te new algoritm DITO performed about times as fast as te original DFD algoritm, wic is in it-

8 self an improvement over te naive algoritm. Te datasets above five dimensions, owever, present difficulty for te series expansion idea to be effective, and te new algoritm is slower tan DFD algoritm. Yet te algoritm wit te optimized pruning rule DFDO consistenyl yields about 0 % to 5 % improvement over DFD algoritm in iger dimensions. sj 50000, D =, N = 50000, = Alg\ Σ Naive FGT X X X X IFGT 7.05 DFD DFDO DFTO DITO mockgalaxy D M rnd, D = 3, N = 50000, = Alg\ Σ Naive FGT X X X X X DFD DFDO DFTO DITO bio5 rnd, D = 5, N = 50000, = Alg\ Σ Naive FGT X X X X X X X X DFD DFDO DFTO DITO pall7 rnd, D = 7, N = 50000, = Alg\ Σ Naive FGT X X X X X X X X DFD DFDO DFTO DITO covtype rnd, D = 0, N = 50000, = Alg\ Σ Naive FGT X X X X X X X X DFD DFDO DFTO DITO CoocTexture rnd, D = 6, N = 50000, = Alg\ Σ Naive FGT X X X X X X X X DFD DFDO DFTO DITO eferences Appel, A An efficient program for manybody simulations. SIAM Journal on Scientific and Statistical Computing, 6, Barnes, J., & Hut, P A ierarcical on log n force-calculation algoritm. Nature, 34. Baxter, B., & oussos, G. 00. A new error estimate for te fast gauss transform. SIAM Journal on Scientific Computing, 4, Callaan, P., & Kosaraju, S A decomposition of multidimensional point sets wit applications to k- nearest-neigbors and n-body potential fields. Journal of te ACM, 6, de Freitas, N., ang, Y., Madaviani, M., & Lang, D Fast krylov metods for n-body learning. In Y. eiss, B. Scölkopf and J. Platt Eds., Advances in neural information processing systems 8, Cambridge, MA: MIT Press. Deng, K., & Moore, A Multiresolution instance-based learning. Proceedings of te Twelft International Joint Conference on Artificial Intelligence pp San Francisco, CA: Morgan Kaufmann. Gray, A., & Moore, A n-body problems in statistical learning. Advances in Neural Information Processing Systems 3 December 000. MIT Press. Gray, A. G., & Moore, A.. 003a. Nonparametric density estimation: Toward computational tractability. Proceedings of SIAM Data Mining. Gray, A. G., & Moore, A.. 003b. apid evaluation of multiple density models. Artificial Intelligence and Statistics. Greengard, L., & oklin, V A fast algoritm for particle simulations. Journal of Computational Pysics, 73. Greengard, L., & Strain, J. 99. Te fast gauss transform. SIAM J. Sci. Stat. Comput.,, Katayama, N., & Sato, S Te sr-tree: An index structure for ig-dimensional nearest neigbor queries. Proceedings of te 997 ACM SIGMOD International Conference on Management of Data pp Lee, D., Gray, A., & Moore, A Dual-tree fast gauss transforms. In Y. eiss, B. Scölkopf and J. Platt Eds., Advances in neural information processing systems 8, Cambridge, MA: MIT Press. Silverman, B Density estimation for statistics and data analysis. Capman and Hall. Yang, C., Duraiswami,., Gumerov, N. A., & Davis, L Improved fast gauss transform and efficient kernel density estimation. International Conference on Computer Vision.