Problem descriptions: Non-uniform fast Fourier transform

Size: px

Start display at page:

Download "Problem descriptions: Non-uniform fast Fourier transform"

Donna Anderson
5 years ago
Views:

1 Problem descriptions: Non-uniform fast Fourier transform Lukas Exl 1 1 University of Vienna, Institut für Mathematik, Oskar-Morgenstern-Platz 1, A-1090 Wien, Austria December 11, 2015 Contents 0.1 Software for NUFFT Notation and setting NUDFT - Non-uniform discrete Fourier transform NUFFT Matrix form Window functions Gridding Type NUFFT in micromagnetic computation List of Algorithms 1 Type-1 NUFFT Type-2 NUFFT Corresponding author, lukas.exl@univie.ac.at 1

2 NUFFT - Non-uniform FFT Non-uniform fast Fourier transform (NUFFT) dates back to the 90 s [1, 2]. It is a fast computational scheme for non-equispaced/uniform discrete Fourier transforms (NDFT), like fast Fourier transform (FFT) is a fast scheme for (ordinary) discrete Fourier transform (DFT). According to a distinction of the input/output objects, there are three main types of non-uniform discrete Fourier transform. One that gets non-uniformly located data points as input and creates a multi-index object 1. This type is often referred to as type-1 NUFFT [3] (or transposed NUFFT [4 6]). Another one (type-2) gets a multi-index object and creates a vector with indices related to non-equispaced locations in physical space. Finally, if both input and output data are non-equispaced, a combination of the type-1 and type-2 NUFFT leads to the so-called type-3 NUFFT [4, 7]. One possible application for NUFFT is the so-called discrete convolution with non-equispaced data, which appears for example in particle physics in the context of the computation of potentials or non-local (force-)fields. The reason for this application is the fact that the convolution theorem for discrete convolution 2 of equispaced data with radial kernels generalizes to nonequispaced data [5]. This makes this fast convolution method a quasi-linearly scaling method (N log N). Consider a discrete convolution of non-equispaced data, i.e. φ(x i ) = M α j K(x i y j ), α j R, x i, y j ( 1, )3, j = 1... M, i = 1... N, (1) j=1 where K is some radial kernel function like, for example, the Newtonian kernel N(x) := 1/ x. Approximation of (a smoothed version of) the kernel by a Fourier series leads to a computational scheme similar to that derived from the convolution theorem for equispaced data, where (inverse) Fourier transform is replaced by the (type-2) discrete Fourier transform, i.e. there is the following analogy F 1( F (α) F (k) ) NUFFT H( NUFFT(α) F (k) ), (2) where NUFFT and NUFFT H denotes the type-1 and type-2 NUFFT, respectively, and the symbol means element-wise multiplication, cf. Sec. 0.2 for notations. Such a method uses NUFFT for the so-called far-field part of the computation 3 in Fourier space, while the near field part (correction for the smoothing step) is computed in physical space. 1 tensor. 2 Convolution is inverse Fourier transform of the product of the Fourier transforms. 3 The vector/tensor k consists of equidistant samples of some smoothed version of the kernel function K. 2

3 Another application arises in the numerical treatment of higher dimensional integrals, where one typically has to deal with the so-called curse of dimensionality, i.e. the computations scale exponentially with the dimension in the exponent, e.g. S 3 for three dimensions and S grid (or quadrature) points in each principal direction. NUFFT was successfully applied for the efficient and highly accurate (higher dimensional) quadrature of Fourier integrals related to the computation of potentials/fields [8]. Such methods split the Fourier formulation of the potential problem into a near and a far field part, where NUFFT is used for the computation of the near field part in Fourier space. Further applications can be found in [3, 7, 9, 10] and references therein, which include fast image reconstruction in magnetic resonance imaging (MRI), fast summation, fast Gaussian transform, polar FFT, CT/Radon transform, quadrature on manifolds, etc. 0.1 Software for NUFFT Courant Mathematics and Computing Laboratory: [10] TU Chemnitz: [9] University of Michigan Image reconstruction toolbox: [11] Air Force Research Laboratories via Matlab Central: [12] 0.2 Notation and setting For the description we use three dimensions, generalization to arbitrary dimension is readily possible. We basically need two ingredients: Non-equispaced data points: M arbitrarily located data points (scaled because of technical reason), i.e. y j ( 1 2, 1 2 )3 =: T, j = 1... M. Multi-index set: For the grid size n = (n 1, n 2, n 3 ) with n i 2N, we define the multi-index set I n := {k Z 3 n/2 c k c n/2 1}, where c means that the relation holds for each component. The indices k I n label the grid points of a Cartesian grid (tensor grid) with size I n = n 1 n 2 n 3. 3

4 Vectors are understood as columns, the symbol (.) T means transpose, i.e. the vector k = (k 1, k 2, k 3 ) T is a transposed row. We also use the short notation k T y := k 1 y 1 + k 2 y 2 + k 3 y 3 for the inner product. For component-wise multiplication we use the notation k y := (k 1 y 1, k 2 y 2, k 3 y 3 ) T ; for componentwise inversion simply k 1 := (1/k 1, 1/k 2, 1/k 3 ) T. 0.3 NUDFT - Non-uniform discrete Fourier transform The sums f k = M f j e 2πikT y j, f j C, y j T j=1 g j = f k e 2πikT y j, f k C, y j T k I n (type-1), (type-2), are analogues of the discrete Fourier transform (DFT), so-called non-uniform discrete Fourier transforms (NUDFT). Observe, however, that a big difference to ordinary discrete Fourier transform makes the fact that these sums are not inverse or unitary transformations to each other in general. An exception is the case where the data y j, j = 1... M j n 1, j I n are equispaced on a tensor grid with I n ( M) grid points, which makes it a discrete Fourier transform. (3) 0.4 NUFFT Fast approximate methods have been invented, which break down the quadratic computational complexity O(M I n ) of the sums (3) to only O( I n log I n + M) in the case of non-uniform fast Fourier transform (NUFFT). The basic idea 4 is to smear the data over a tensor grid by a gridding process and then apply FFT. The data-smearing is undone afterwards (deconvolution). In the case of convolution with, e.g., a Gaussian heat kernel as gridding procedure, this deconvolution step is a simple scalar division in Fourier space. Type-2 NUFFT consists of the transposed operations. For a complete derivation the reader is referred to e.g. [4]. We consider the problem of computing type-1 NUFFT, i.e. f k = M f j e 2πikT y j, f j C, y j T, (4) j=1 and briefly give an algorithmic description (again dimension d = 3 is easily adapted to any other): 4 In the case of type-1 NUFFT; type-2 NUFFT is the transposed operation of type-1 NUFFT, see. Sec

5 Algorithm 1 Type-1 NUFFT Given n 2N 3, α, M N (e.g. α = 2), data y j T, j = 1... M, sources f j C, j = 1... M. Define N = αn. 1. Gridding/ Sparse convolution : For l I N compute g l = f j ψ(y j l N 1 ), (5) by exploiting the locality of the window function ψ. j 2. FFT: For k I n compute DFT (utilizing multivariate FFT) ĝ k := g l e 2πikT (l N 1). (6) l I N 3. Deconvolution: For k I n compute f k := ĝ k I N c k (ψ), (7) where c k (ψ) are the precomputed Fourier coefficients of the window function ψ. The first part in Alg. 1 scales linearly in M (see Sec. 0.7), where the constant depends on the accuracy of the sparse approximation of the sum in (5). The scaling of the FFT part in Alg. 1 is O( I N log I N ). The deconvolution scales linearly in the size of the tensor grid. Overall Alg. 1 scales like O( I N log I N + M). 0.5 Matrix form The three steps in Alg. 1 can be summarized in matrix vector notation as f = D F G f, (8) where G is the (sparse) gridding matrix of size I N M, F denotes the Fourier transform and D the diagonal matrix which realizes step 3 in Alg. 1. The vectors f and f consist of entries f j and f k, respectively. The sparsity of the matrix G is discussed in Sec. 0.6 and Window functions The sum in (5) is too complex to be calculated exactly (would yield O( I N M) complexity). Hence, approximations have to be considered which yield truncation errors and aliasing errors 5

6 (cf. Eqn. (5) and (7), respectively). In order to keep those errors small, common window functions are local in time and frequency space. Moreover, tensor products of 1-dimensional functions are used to construct higher-dimensional functions. Typical examples for window functions are so-called (dilated) Gaussian functions, i.e. ψ(x) = c 1 e c 2 (nx)2 m, c1, c 2 > 0, m N, (9) with known Fourier transform (also Gaussian) which is used in step 3 of Alg. 1. The higher dimensional function is simply constructed as tensor product, i.e. ψ(x) = ψ 1 (x 1 )ψ 2 (x 2 )ψ 3 (x 3 ). Possible are also so-called (dilated) cardinal central B-splines, (dilated) sinc functions or (dilated) Kaiser-Bessel functions, cf. D. Potts [13] where also exponential error decay w.r.t. the truncation parameter m is shown. As mentioned before the gridding step (cf. step 1 in Alg. 1 and Sec. 0.7) includes truncation, which yields a sparse approximation for the matrix G in the matrix-vector multiplication G f in (8). The non-zeros of the matrix G are contributions of the window function which are numerically significant. There are a few different ways to deal with this matrix-vector evaluation, depending on whether the matrix is fully precomputed or some compressed storage variant is used. Fully precomputed window functions All the values ψ(y j l N 1 ) for j = 1... M and l I N,m (y j ) 5 are precomputed. These are at most (2m + 1) 3 M real numbers, where m is a small number (about 4 20) depending on the desired accuracy and the used type of window function. The complexity of the matrix-vector evaluation is O ( (2m + 1) 3 M ) too. Tensor product precomputation If the window functions are tensor products 6 it is also possible to only precompute the values ( ) ψ q (y j ) q l q /N q for all components q. In three dimensions this strategy yields a storage demand of 3(2m + 1)M, where the complexity for the matrix-vector product increases by 2(2m + 1) 3 extra multiplications (for each of the M nodes y j ) for the construction of the multivariate function from its product structure. The overall asymptotic complexity of the matrix-vector evaluation is still O ( (2m + 1) 3 M ). 5 A certain subset of I N, cf. Sec This is the case for all above mentioned types of window functions, but not for those used in [14]. 6

7 Table interpolation While the tensor product structure decreases the storage costs for the rows of G, (linear) interpolation of the window functions can reduce costs for the columns. This is especially useful if M is very large. A large amount of equidistant samples of the components ψ q are precomputed and stored. During the NUFFT algorithm these precomputed samples are used for the ( ) approximation of the desired values ψ q (y j ) q l q /N q by linear interpolation (two neighbors of (y j ) q ). See e.g. [15] for a paper on that topic. 7 Of course, higher order interpolation is also possible. In any case, storage requirements are drastically reduces because only the equidistant samples have to be kept in memory, i.e. 3S in three dimensions for S equidistant samples in each direction (independent of M and n). These approaches require a multiple of (2m + 1) 3 extra multiplications per node y j. Fast Gaussian gridding Fast Gaussian gridding was introduced in [3] for the case of Gaussian window functions. It uses the tensor structure of the window function and, especially, a certain splitting of the exponential function for Gaussian window functions. Storage is reduced to 2dM + d(2m + 1) (d dimensions) without any approximation. The splitting (in d = 1 dimension) which is used for the evaluation of the values ψ(y j l/n), l I n,m (y j ) 8 for fixed node y j reads e c(ny j l) 2 = e c ((ny j u) l ) 2 = e c(ny j u) 2 (e 2c(ny j u) ) l e cl 2, (10) where u := min I n,m (y j ), l = m and c = c(m) > 0 is some constant depending on m. The first two factors on the r.h.s. of (10) are constant for fixed y j. The last factor is even independent of the node y j, which simply allows precomputing those 2m + 1 values in the beginning of the NUFFT. The l -powers in the second term (brackets) can be calculated successively by 2m 1 multiplications from the l = 1 value. Hence, it is enough to store only 2M + (2m + 1) values (d = 1) and 2dM + d(2m + 1) (d dimensions) in general. 0.7 Gridding In the gridding step in Alg. 1 the summation is only over those indices j where ψ(y j l N 1 ) contributes. So, one way to realize this would be to identify, for all l in the multi-index set individually, all corresponding j where ψ(y j l N 1 ) is essentially zero. Afterwards simply restrict summation to those indices j. But, since I N is typically much larger than M a transposed strategy is useful: Consider the window function to be (numerically) zero outside the hypercube 3 q=1[ m/n q, m/n q ] 7 In reference [15] the term gridding is used for the whole type-1 NUFFT. 8 See Sec. 0.7 for the description of the index set I n,m (y j ). 7

8 for m N. For fixed j we collect those indices l for which y j l N 1 lies inside the hypercube 3 q=1[ m/n q, m/n q ] and call this set I N,m (y j ), i.e. I N,m (y j ) := {l I N mn 1 c N 1 l y j c mn 1 } (11) = {l I N y j N m1 c l c y j N + m1}. (12) This set has at most (2m + 1) 3 elements if y j T. As a result the computation of the convolution in (0.7) is realized as follows: Initialize all g l with zeros For j = 1... M calculate the vector ( f j ψ(y l N 1 ) ) and add it to (g l I N,m l) l IN,m (y (y j ) j ), which scales linearly in M, i.e. at most µm operations are needed, with a constant µ which is smaller or equal a uniform bound for the sizes of the sets I N,m (y j ), j = 1... M, i.e. µ (2m+1) Type-2 The type-2 NUFFT, cf. formula (3), is the adjoint of the type-1 NUFFT, i.e. in matrix form we have g = G T F H D f. (13) An algorithm to efficiently compute the sum g j = f k e 2πikT y j, f k C, y j T, (14) k I n is given in Alg. 2. The scaling is also O( N log N + M) like for type-1 NUFFT. 0.9 NUFFT in micromagnetic computation In computational micromagnetics a large community uses fast Fourier transform (FFT) methods for the computation of the non-local stray field on rectangular domains [16]. Such a method strongly relies on a uniform Cartesian grid, since only for uniform data the computationally beneficial FFT applies. Even adaption to curved geometries by local corrections was considered [17] in order to maintain the core of the method. However, ordinary FFT algorithms do not apply anymore for non-uniformly distributed sources (magnetic moments), like due to a finite element discretization of the magnet. Quadrature approximation of the Green s function form of the magnetic scalar potential yields discrete convolution with non-equidistantly distributed sources similar to that in (1). This approach was reported in [18] for a P1-FE discretization (in 8

9 Algorithm 2 Type-2 NUFFT Given n 2N 3, α, M N (e.g. α = 2), data y j T, j = 1... M, sources f k C, k I n. Define N = αn. 1. For k I n compute ĝ k := f k I N c k (ψ), (15) where c k (ψ) are the precomputed Fourier coefficients of the window function ψ. 2. For l I n compute DFT (utilizing multivariate FFT) g l := ĝ k e 2πikT (l N 1). (16) k I N 3. For j = 1... M compute g j = g l ψ(y j l N 1 ), (17) l by exploiting the locality of the window function ψ. two dimensions). The method applies NUFFT as in algorithm 1 and 2. A different method was introduced in [14, 19]. Here, the type-1 NUFFT is modified in a way that the window functions already incorporate the geometry of the finite element mesh. As a consequence, the number of non-uniform sources (M) is greatly reduced. On the other hand, the window functions lose the convenient tensor product property, and hence, only a full storage scheme is possible, cf. Sec Further approximation was considered, e.g. compression for tensors, like TT-format, etc., see [14] and related references therein. Due to the FFT on the auxiliary tensor grid, both approaches are not well suited/efficient for very inhomogeneous distributions of sources, as it would arise for instance for local FE-mesh refinement. For such cases the linearly scaling fast multipole method (FMM) is widely used. However, in the case of reasonably uniformly distributed sources, the rather large constant in the O(M)-scaling of the FMM makes a NUFFT method more favorable (see remark of Greengard himself in [8]). References [1] A. Dutt and V. Rokhlin. Fast Fourier transforms for nonequispaced data. SIAM Journal on Scientific computing, 14(6): ,

10 [2] G. Beylkin. On the fast Fourier transform of functions with singularities. Applied and Computational Harmonic Analysis, 2(4): , [3] L. Greengard and J.-Y. Lee. Accelerating the nonuniform fast Fourier transform. SIAM review, 46(3): , [4] D. Potts, G. Steidl, and M. Tasche. Fast Fourier transforms for nonequispaced data: A tutorial. In Modern sampling theory, pages Springer, [5] D. Potts and G. Steidl. Fast summation at nonequispaced knots by NFFT. SIAM Journal on Scientific Computing, 24(6): , [6] D. Potts, G. Steidl, and A. Nieslony. Fast convolution with radial kernels at nonequispaced knots. Numerische Mathematik, 98(2): , [7] J.-Y. Lee and L. Greengard. The type 3 nonuniform FFT and its applications. Journal of Computational Physics, 206(1):1 5, [8] S. Jiang, L. Greengard, and W. Bao. Fast and accurate evaluation of nonlocal Coulomb and dipole-dipole interactions via the nonuniform FFT. SIAM J. Sci. Comput., 36(5): B777 B794, [9] J. Keiner, S. Kunis, and D. Potts. NFFT from TU Chemnitz. URL tu-chemnitz.de/~potts/nfft/. [10] CMCL. NUFFT software from CMCL. URL nufft/nufft.html. [11] J. Fressler. Image reconstruction toolbox. URL ~fessler/code/. [12] M. Ferrara. NUFFT software via Matlab Central. URL matlabcentral/fileexchange/25135-nufft--nfft--usfft. [13] D. Potts. Schnelle Fourier Transformationen für nichtäquidistante Daten und Anwendungen. Habilitationsschrift, Universität zu Lübeck, [14] L. Exl and T. Schrefl. Non-uniform FFT for the finite element computation of the micromagnetic scalar potential. Journal of Computational Physics, 270: , [15] P. J Beatty, D. G. Nishimura, and J. M. Pauly. Rapid gridding reconstruction with a minimal oversampling ratio. Medical Imaging, IEEE Transactions on, 24(6): ,

11 [16] M. J. Donahue and D. G. Porter. OOMMF User s Guide, Version 1.0. Interagency Report, NISTIR 6376, sep [17] M. J. Donahue and R. D. McMichael. Micromagnetics on curved geometries using rectangular cells: Error correction and analysis. IEEE Trans. Magn., 43(6): , June doi: /TMAG [18] E. Kritsikis, J.-C. Toussaint, O. Fruchart, H. Szambolics, and L. Buda-Prejbeanu. Fast computation of magnetostatic fields by nonuniform fast Fourier transforms. Applied Physics Letters, 93(13): , [19] L. Exl. Tensor grid methods for micromagnetic simulations. PhD thesis, Vienna UT,

Introduction to Practical FFT and NFFT

Introduction to Practical FFT and NFFT Michael Pippig and Daniel Potts Department of Mathematics Chemnitz University of Technology September 14, 211 supported by BMBF grant 1IH81B Table of Contents 1 Serial