Fast Algorithms for the Computation of Oscillatory Integrals

Transcription

1 Fast Algorithms for the Computation of Oscillatory Integrals Emmanuel Candès California Institute of Technology EPSRC Symposium Capstone Conference Warwick Mathematics Institute, July 2009

2 Collaborators Lexing Ying Laurent Demanet

3 Our problem Evaluate numerically a Fourier integral operator (FIO) (T f)(x) = a(x, k)e 2πiΦ(x,k) ˆf(k) dk R d at points x given on a Cartesian grid k R d : frequency variable ( ˆf(k) = e 2πix k f(x) dx) R d a(x, k): (smooth) amplitude Φ(x, λk): homogeneous (smooth) phase function as large as k Φ(x, λk) = λ Φ(x, k), λ > 0 e.g. Φ(x, k) = g(x) k

4 A motivating example: wave propagation 2 u t 2 (x, t) = c2 u(x, t), u(x, 0) = u 0 (x) u/ t(x, 0) = 0

5 A motivating example: wave propagation 2 u t 2 (x, t) = c2 u(x, t), u(x, 0) = u 0 (x) u/ t(x, 0) = 0 Solution operator is u(x, t) = 1 ( ) e 2πi(x k+c k t) û 0 (k) dk + e 2πi(x k c k t) û 0 (k) dk 2 R 2 R 2 Two FIOs with phase functions Φ ± (x, k) = x k ± c k t Inhomogeneous medium c(x) solution operator = sum of two FIOs (small times)

6 Importance of FIOs Arise in many (inverse) problems Applying FIOs is often the computational bottleneck

7 Example in seismics Marine survey

8 Kirchhoff migration Wave measurements f s (t, x r ) parametrized by time t receiver location x r source coordinate x s Forward map: F δc = δp Imaging operator is F : FIO under general assumptions Approximations by generalized Radon transform (GRT): integration of f s over fixed set of curves parametrized by travel times g s (x) = δ(t τ(x, x r ) τ(x, x s )) f s (t, x r ) dt dx r Followed by stack operation over the s index

9 Other examples Transmission electron microscopy Radar imaging Ultrasound imaging

10 Discrete computational problem Discrete grids X = {(i 1 /N, i 2 /N) : 0 i 1, i 2 < N} [0, 1] 2 Ω = {(k 1 /N, k 2 /N) : N/2 k 1, k 2 < N} [ 1/2, 1/2] 2 Given input {f(k)} k Ω, evaluate (T f)(x) := 1 a(x, k) e 2πiNΦ(x,k) f(k), N at all x X k Ω with Φ smooth and homogeneous in k

11 Peek at the results (T f)(x) := 1 a(x, k) e 2πiNΦ(x,k) f(k), N k Ω Kernel is not analytic and is highly oscillatory x X Naive evaluation O(N 4 ) (O(N 2 ) inputs/outputs) Algorithm for fast summation O(N 2.5 log N) (C., Demanet and Ying, 06)

12 Peek at the results (T f)(x) := 1 a(x, k) e 2πiNΦ(x,k) f(k), N k Ω Kernel is not analytic and is highly oscillatory x X Naive evaluation O(N 4 ) (O(N 2 ) inputs/outputs) Algorithm for fast summation O(N 2.5 log N) (C., Demanet and Ying, 06) Today: Novel algorithm with optimal complexity for accurate summation O(N 2 log N) flops O(N 2 ) storage

13 Agenda The butterfly structure Fast butterfly algorithm for the evaluation of FIOs

14 The Butterfly Structure

15 Butterfly algorithm General algorithmic structure for evaluating certain types of integrals u i = j K(x i, p j )f j Introduced by Michielssen and Boag ( 96) Generalized by O Neil and Rokhlin ( 07)

16 Butterfly algorithm General algorithmic structure for evaluating certain types of integrals u i = j K(x i, p j )f j Introduced by Michielssen and Boag ( 96) Generalized by O Neil and Rokhlin ( 07) Example: K(x, p) = e 2πiNxp {x i }: N points in [0, 1] {p j }: N points in [0, 1] {f j }: sources at {p j } Applications FFT nonuniform FFTs many others Kernel is dense and oscillatory

17 Low-rank approximation (K(x, p) = exp(2πin xp)) A interval in x B interval in p obeying length(a) length(b) 1/N The submatrix {K(x i, p j ) : x i A, p j B} has approximately low rank: K(x, p) with r = O(log(1/ɛ)) r a t (x)b t (p) ɛ t=1

18 Example DFT Top left block 10 0 Top singular values

19 Interpolative decompositions O Neil and Rokhlin suggest using interpolative decompositions a t (x) := K(x, p t ), p t B Rank-revealing QR decomposition: Gu and Eisenstat ( 96) Interpolative decomp.: Cheng, Gimbutas, Martinsson and Rokhlin ( 05) Interpolative representation r K(x, p) K(x, p t )b t (x) Cost for an m n matrix is O(mn 2 ) t=1

20 Multiscale decompositions Compute low-rank approximations of all submatrices obeying Use two-scale relations for efficiency length(a) length(b) = 1/N

21 Definition of partial sums and equivalent sources Partial sums u B (x) = K(x, p j )f j p j B Approximation for x A and length(a) length(b) 1/N r u B (x) K(x, p AB (p j )f j, x A t=1 t ) p j B b AB t

22 Definition of partial sums and equivalent sources Partial sums u B (x) = K(x, p j )f j p j B Approximation for x A and length(a) length(b) 1/N r u B (x) K(x, p AB (p j )f j, x A t=1 t ) p j B Equivalent sources for (A, B): {ft AB } 1 t r = f AB t p j B b AB t b AB t (p j )f j

23 Definition of partial sums and equivalent sources Partial sums u B (x) = K(x, p j )f j p j B Approximation for x A and length(a) length(b) 1/N r u B (x) K(x, p AB (p j )f j, x A t=1 t ) p j B Equivalent sources for (A, B): {ft AB } 1 t r = f AB t p j B b AB t b AB t (p j )f j Compact representation of K AB : B A r u B (x) = K(x, p AB t )ft AB t=1

24 Definition of partial sums and equivalent sources Partial sums u B (x) = K(x, p j )f j p j B Approximation for x A and length(a) length(b) 1/N r u B (x) K(x, p AB (p j )f j, x A t=1 t ) p j B Equivalent sources for (A, B): {ft AB } 1 t r = f AB t p j B b AB t b AB t (p j )f j Compact representation of K AB : B A r u B (x) = K(x, p AB t )ft AB Butterfly structure Recursive computation of {p AB t }, {ft AB } for length(a) length(b) = 1/N t=1

25 Initialization For all (A, B), l(a) = 1 & l(b) = 1/N, construct {p AB t = f AB t p j B b AB t (p j )f j } 1 t r and {ft AB } 1 t r cost of constructing {p AB t } is O(r 2 N)/pair cost of constructing {ft AB } is O(r 2 )/pair tot. cost is O(r 2 N 2 ) Interpretation: {p AB t } selected columns, {ft AB } column weights

26 Recursion: merge, split, compress For all (A, B), l(a) = 1/2 & l(b) = 2/N, get {p AB t } 1 t r and {ft AB } 1 t r u B (x) 2 r i=1 t=1 K(x, p ApBi t )f ApBi t, x A A p

27 Recursion: merge, split, compress For all (A, B), l(a) = 1/2 & l(b) = 2/N, get {p AB t } 1 t r and {ft AB } 1 t r u B (x) 2 r i=1 t=1 Can reduce the rank of K(x, p ApBi t Treat {f ApBi t K(x, p ApBi t )f ApBi t, x A A p } t,i as sources at {p ApBi t } t,i B f AB t = 2 i=1 s=1 ) : x A, {p ApBi t } t,i K(x, p AB t ) r b AB t (p ApBi s )fs ApBi cost of constructing {p AB t } is O(r 2 N)/pair cost of constructing {ft AB } is O(r 2 )/pair tot. cost is O(r 2 N 2 )

28 Schematic representation

29 Schematic representation u Bi (x) = r t=1 K(x, p ApBi t )f ApBi t u B (x) = r t=1 K(x, p AB t )ft AB

30 Next step... until l(b) = 1 and l(a) = 1/N

31 Termination In the end, l(a) = 1/N and l(b) = 1, and u B (x) r t=1 K(x, p AB t )ft AB cost of evaluating u B (x) is O(r)/pair tot. cost is O(rN)

32 Multiscale recursion

33 Summary: complexity analysis If low-rank expansions available If not O(r 2 N log N) evaluation time O(r 2 N 2 ) evaluation time O(r 2 N log N) storage O(r 2 N log N) storage

34 Summary: complexity analysis If low-rank expansions available If not O(r 2 N log N) evaluation time O(r 2 N 2 ) evaluation time O(r 2 N log N) storage O(r 2 N log N) storage Powerful architecture but Precomputation time may be prohibitive Storage may be prohibitive

35 Summary: complexity analysis If low-rank expansions available If not O(r 2 N log N) evaluation time O(r 2 N 2 ) evaluation time O(r 2 N log N) storage O(r 2 N log N) storage Powerful architecture but Precomputation time may be prohibitive Storage may be prohibitive Our contribution O(r 2 N log N) evaluation time and storage complexity is O(r 2 N)

36 Fast Evaluation of FIOs

37 Recall oscillatory integral u(x) = k Ω e 2πiNΦ(x,k) f k x X k Ω

38 Polar coordinates for frequency variable k Phase may be singular at k = 0 because of homogeneity (e.g. Φ(x, k) = k )

39 Polar coordinates for frequency variable k Phase may be singular at k = 0 because of homogeneity (e.g. Φ(x, k) = k ) Polar coordinates p = (p 1, p 2 ) k 1 = p 1 cos(2πp 2 ) k 2 = p 2 sin(2πp 2 ) Cartesian Polar Partitioning in k Slight abuse of notations u(x) = p Ω e 2πiNΨ(x,p) f p, smooth phase Ψ

40 Hierarchical structure l(a) l(b) = 1/N

41 Low-rank interactions If l(a)l(b) 1/N, kernel e 2πiNΨ(x,p) : x A, p B has approx. low rank Assume wlog A and B centered at 0

42 Low-rank interactions If l(a)l(b) 1/N, kernel e 2πiNΨ(x,p) : x A, p B has approx. low rank Assume wlog A and B centered at 0 Residual phase in 1D (for simplicity) R AB (x, p) = Ψ(x, p) Ψ(0, p) Ψ(x, 0) + Ψ(0, 0) = x p Ψ(x, p ) xp = O(1/N) Same calculation in higher dimensions

43 Low-rank interactions If l(a)l(b) 1/N, kernel e 2πiNΨ(x,p) : x A, p B has approx. low rank Assume wlog A and B centered at 0 Residual phase in 1D (for simplicity) R AB (x, p) = Ψ(x, p) Ψ(0, p) Ψ(x, 0) + Ψ(0, 0) = x p Ψ(x, p ) xp = O(1/N) Same calculation in higher dimensions Shows that e 2πiNRAB (x,p) approx. low rank

44 Low-rank interactions If l(a)l(b) 1/N, kernel e 2πiNΨ(x,p) : x A, p B has approx. low rank Assume wlog A and B centered at 0 Residual phase in 1D (for simplicity) R AB (x, p) = Ψ(x, p) Ψ(0, p) Ψ(x, 0) + Ψ(0, 0) = x p Ψ(x, p ) xp = O(1/N) Same calculation in higher dimensions Shows that e 2πiNRAB (x,p) approx. low rank Factorization [ e 2πiNΨ(x,p) = e 2πiNΨ(0,0) e 2πiNΨ(0,x) e 2πiNRAB (x,p) e 2πiNΨ(0,p)]

45 Oscillatory Chebyshev interpolation Chebyshev interpolation of e 2πiNRAB (x,p) in x when l(a) 1/ N p when l(b) 1/ N E.g. interpolation in p {p t } B : tensor-chebyshev grid in B L B t (p) : Lagrange interpolant with inputs at {p B t }, L B t (p B t ) = δ(t = t )

46 Oscillatory Chebyshev interpolation Chebyshev interpolation of e 2πiNRAB (x,p) in x when l(a) 1/ N p when l(b) 1/ N E.g. interpolation in p {p t } B : tensor-chebyshev grid in B L B t (p) : Lagrange interpolant with inputs at {p B t }, L B t (p B t ) = δ(t = t ) With grid of logarithmic size in 1/ɛ (x,p) e e2πinrab 2πiNRAB (x,p B t ) L B t (p) ɛ t on A B When interpolation in x, low-rank approximation is P t e2πinrab (x A t,p) L A t (x)

47 Demodulation/Interpolation/Remodulation From R AB (x, p) = Ψ(x, p) Ψ(0, p) Ψ(x, 0) + Ψ(0, 0) e 2πiNΨ(x,p) e 2πiNΨ(x,pB t }{{} ) e i2πnψ(0,pb t ) L B t (p) e i2πnψ(0,p) ɛ }{{} t a t(x) b t(x) Similar structure (with different interpretation) when interpolating kernel in x

48 Overall structure and two-scale relation 1 Initialize equivalent sources 2 Propagate equivalent sources (interpolation in p) until mid-level 3 Switch representation at mid-level 4 Propagate equivalent sources (interpolation in x) 5 Terminate by evaluating output Step 2: propagation of equivalent sources f AB t = c,t b B t (p Bc t ApBc )ft = e 2πiNΨ(x0(A),pB t ) c t L B t (p Bc t ) e2πinψ(x0(a),pbc t ) f ApBc t Step 4: similar

49 Finer points and summary {p B t } and polynomials L t (p) are computed all at once Only need to store equivalent sources at pairs of consecutive scales Separation rank higher than that of the interpolative decomposition

50 Finer points and summary {p B t } and polynomials L t (p) are computed all at once Only need to store equivalent sources at pairs of consecutive scales Separation rank higher than that of the interpolative decomposition Complexity is O(polylog(1/ɛ) N 2 log N) Storage is O(polylog(1/ɛ) N 2 for ɛ-accurate computation Easy extensions to varying amplitudes a(x, k)e i2πnφ(x,k)

51 Numerical results Generalized Radon transform integrating f along ellipses centered at x and with axes of length c 1 (x) and c 2 (x) is the sum of two FIOs with phases Φ ± (x, k) = x k ± c 2 1 (x)k2 1 + c2 2 (x)k2 2

52 Numerical results Generalized Radon transform integrating f along ellipses centered at x and with axes of length c 1 (x) and c 2 (x) is the sum of two FIOs with phases Φ ± (x, k) = x k ± c 2 1 (x)k2 1 + c2 2 (x)k2 2 First example (constant amplitude) u(x) = k Ω e 2πiNΦ+(x,k) ˆf(k) with c 1 (x) = 1 3 (2 + sin(2πx 1) sin(2πx 2 )), c 2 (x) = 1 3 (2 + cos(2πx 1) cos(2πx 2 ))

53 (N, Chby. grid) Alg. Time Dir. Time Speedup Error (1024, 5) 1.48e e e e-2 (2048, 5) 6.57e e e e-2 (4096, 5) 3.13e e e e-2 (1024, 7) 2.76e e e e-4 (2048, 7) 1.23e e e e-4 (4096, 7) 5.80e e e e-4 (1024, 9) 4.95e e e e-5 (2048, 9) 2.21e e e e-5 (4096, 9) 1.02e e e e-5 (1024, 11) 8.33e e e e-7 (2048, 11) 3.48e e e e-7 Conclusion: scales like O(log(1/ɛ)) O(N 2 log N)

54 Numerical results II Second example (variable amplitude): exact integration u(x) = a + (x, k)e 2πiNΦ+(x,k) ˆf(k) + a (x, k)e 2πiNΦ (x,k) ˆf(k) k Ω k Ω a ± (x, k) = (J 0 (2πc(x) k ) ± iy 0 (2πc(x) k ) e 2πic(x) k Φ ± (x, k) = x k + c(x) k J 0 and Y 0 are Bessel functions and spheres radii c(x) = 1 4 (3 + sin(2πx 1) sin(2πx 2 ))

55 (N, Chby. grid) Alg. Time Dir. Time Speedup Error (256, 5) 1.39e e e e-2 (512, 5) 7.25e e e e-2 (1024, 5) 3.45e e e e-2 (256, 7) 2.69e e e e-4 (512, 7) 1.38e e e e-4 (1024, 7) 6.43e e e e-4 (256, 9) 5.23e e e e-5 (512, 9) 2.49e e e e-5 (1024, 9) 1.15e e e e-5 (256, 11) 1.04e e e e-7 (512, 11) 4.10e e e e-7 (1024, 11) 1.84e e e e-7 Conclusion: scales like O(log(1/ɛ)) O(N 2 log N)

56 Summary Accurate and near-optimal numerical evaluation of FIOs Operating characteristics Butterfly structure Residue phase Oscillatory Chebyshev interpolation Many applications/extensions Other types of oscillatory kernels K(x, p) Other types of problems: e.g. sparse Fourier transform (Ying, 08) Reference: E. J. Candès, L. Demanet and L. Ying (2008). A Fast Butterfly Algorithm for the Computation of Fourier Integral Operators, to appear in Mult. Model. Sim