Multiscale Geometric Analysis: Theory, Applications, and Opportunities

Transcription

1 1 Multiscale Geometric Analysis: Theory, Applications, and Opportunities Emmanuel Candès, California Institute of Technology Wavelet And Multifractal Analysis 2004, Cargese, July 2004

2 Report Documentation Page Form Approved OMB No Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE 07 JAN REPORT TYPE N/A 3. DATES COVERED - 4. TITLE AND SUBTITLE Multiscale Geometric Analysis: Theory, Applications, and Opportunities 5a. CONTRACT NUMBER 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) California Institute of Technology 8. PERFORMING ORGANIZATION REPORT NUMBER 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR S ACRONYM(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release, distribution unlimited 11. SPONSOR/MONITOR S REPORT NUMBER(S) 13. SUPPLEMENTARY NOTES See also ADM001750, Wavelets and Multifractal Analysis (WAMA) Workshop held on July 2004., The original document contains color images. 14. ABSTRACT 15. SUBJECT TERMS 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT UU a. REPORT unclassified b. ABSTRACT unclassified c. THIS PAGE unclassified 18. NUMBER OF PAGES 96 19a. NAME OF RESPONSIBLE PERSON Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18

3 2 Collaborators David Donoho (Stanford) Laurent Demanet (Caltech)

4 3 Classical Multiscale Analysis Wavelets: Enormous impact Theory Applications Many success stories Deep understanding of the fact that wavelets are not good for all purposes Consequent constructions of new systems lying beyond wavelets Overview Other multiscale constructions Problems classical multiscale ideas do not address effectively

5 4 Fourier Analysis Diagonal representation of shift-invariant linear transformations; e.g. solution operator to the heat equation t u = a 2 u Fourier, Théorie Analytique de la Chaleur, Truncated Fourier series provide very good approximations of smooth functions if f C k. f S n (f) L2 C n k,

6 5 Limitations of Fourier Analysis Does not provide any sparse decomposition of differential equations with variable coefficients (sinusoids are no longer eigenfunctions) Provides poor representations of discontinuous objects (Gibbs phenomenon)

7 6 Wavelet Analysis Almost eigenfunctions of differential operators (Lf)(x) = a(x) x f(x) Sparse representations of piecewise smooth functions

8 Wavelets and Piecewise Smooth Objects 1-dimensional example: 7 Fourier series: g(t) = 1 {t>t0 }e (t t 0) 2. g(t) = c k e ikt Fourier coefficients have slow decay: c (n) c 1/n. Wavelet series: g(t) = θ λ ψ λ (t) Wavelet coefficients have fast decay: θ (n) c (1/n) r for any r > 0. as if the object were non-singular

9 8 Wavelets and Operators Calderòn-Zygmund Operator (Kf)(x) = K(x, y)f(y) dy K singular along the diagonal K smooth away from the diagonal Sparse representation in wavelet bases (Calderòn, Meyer, etc.) Applications to numerical analysis and scientific computing (Beylkin, Coifman, Rokhlin)

10 9 Our Viewpoint Sparse representations of point-singularities Sparse representation of certain matrices Simultaneously Applications Approximation theory Data compression Statistical estimation Scientific computing More importantly: new mathematical architecture where information is organized by scale and location

11 10 New Challenges Intermittency in higher-dimensions Evolution problems CHA has not addressed these problems.

12 11 Agenda Limitations of existing image representations Curvelets: geometry and tilings in Phase-Space Representation of functions, signals Representation of operators, matrices

13 12 Three Anomalies Inefficiency of Existing Image Representations Limitations of Existing Pyramid Schemes Limitations of Existing Scaling Concepts

14 13 I: Inefficient Image Representations Edge Model: Object f(x 1, x 2 ) with discontinuity along generic C 2 smooth curve; smooth elsewhere. Fourier is awful Best m-term trigonometric approximation f m f f m 2 2 m 1/2, m Wavelets are bad Best m-term approximation by wavelets: f f m 2 2 m 1, m

15 14 Optimal Behavior There is a dictionary of atoms with best m-term approximant f m f f m 2 2 m 2, m No basis can do better than this. No depth-search limited dictionary can do better. No pre-existing basis does anything near this well.

16 15 (a) Wavelets (b) Triangulations

17 16 II. Limitations of Existing Pyramids Canonical Pyramid Ideas (1980-present) Laplacian Pyramid (Adelson/Burt) Orthonormal Wavelet Pyramid (Mallat/Meyer) Steerable Pyramid (Adelson/Heeger/Simoncelli) Multiwavelets (Alpert/Beylkin/Coifman/Rokhlin) Shared features Elements at dyadic scales/locations FIXED Number of elements at each scale/location

18 Wavelet Pyramid 17

19 18 Limitations of Existing Scaling Concepts Traditional Scaling f a (x 1, x 2 ) = f(ax 1, ax 2 ), a > 0. Curves exhibit different kinds of scaling Anisotropic Locally Adaptive If f(x 1, x 2 ) = 1 {y x2 } then f a (x 1, x 2 ) = f(a x 1, a 2 x 2 ) In Harmonic Analysis called Parabolic Scaling.

20 19 x4 Fine Scale Rectangle x2 Identical Copies of Planar Curve Figure 1: Curves are Invariant under Anisotropic Scaling

21 20 Curvelets C. and Guo, New multiscale pyramid: Multiscale Multi-orientations Parabolic (anisotropy) scaling width length 2 Earlier construction, C. and Donoho (2000)

22 21 Philosophy (Slightly Inaccurate) Start with a waveform ϕ(x) = ϕ(x 1, x 2 ). oscillatory in x 1 lowpass in x 2 Parabolic rescaling D j ϕ(d j x) = 2 3j/4 ϕ(2 j x 1, 2 j/2 x 2 ), D j = 2j 0, j j/2 Rotation (scale dependent) 2 3j/4 ϕ(d j R θjl x), θ jl = 2π l2 j/2 Translation (oriented Cartesian grid with spacing 2 j 2 j/2 ); 2 3j/4 ϕ(d j R θjl x k), k Z 2

23 j Rotate Translate 2 -j/2 Parabolic Scaling 2 -j/2 2 -j

24 23 Curvelet Construction, I Decomposition in space and angle Orthonormal partition of unity (scale): smooth pair of windows (w 0, w 1 ) w 2 0 (r) + w 2 1 (2 j r) = 1, r > 0. j=0 Orthonormal partition of unity (angle): v 2 (t l) = 1, t R. l= For each pair J = (j, l), define the two-dimensional window W J (ξ) = w 1 (2 j ξ ) v(2 j/2 (θ θ j,l )), θ j,l = π l 2 j/2. Note W j,l (ξ) = W j,0 (R θj,l ξ)

25 24 W J is an orthonormal partition of unity (ξ) = 1. J W 2 J Interpretation Divide frequency domain into annuli x [2 j, 2 j+1 ) Subdivide Annuli into wedges. Split every other scale. Oriented local cosines on wedges

26 25 Partition: Frequency-side Picture 2 j/2 2 j

27 Curvelet Construction, II 26 Local bases Fix scale/angle pair (j, l) suppw j,l D j,l = R 1 θ j,l D j,0 where D j,0 is a rectangle of sidelength 2π2 j 2π2 j/2, say. Rectangular grid Λ j of step-size 2 j 2 j/2 Λ j = {k : k = (k 1 2 j, k 2 2 j/2 ), k 1, k 2 Z} Local Fourier basis of L 2 (D j,0 ) e j,0,k (ξ) = 1 2π2 3j/4 eikξ, and likewise local Fourier basis of L 2 (D j,l ) k Λ j e j,l,k = e j,0,k (R θj,l ξ)

28 27 Curvelets (frequency-domain definition) ˆϕ j,l,k (ξ) = W j,0 (R θj,l ξ)e j,0,k (R θj,l ξ)

29 Properties 28 Reproducing formula f = J,k f, ϕ J,k ϕ J,k Why? Let g be the RHS and apply Parseval ĝ = J,k ˆf, ˆϕ J,k ˆϕ J,k = J,k ˆf, W J e J,k W J e J,k = J = J W J ˆfW J, e J,k e J,k k ˆfW J W J = ˆf W 2 J = ˆf J Conclusion ĝ = ˆf and, therefore, f = g. Parseval relation (similar argument) f 2 = J,k f, ϕ J,k 2

30 29 Curvelet: Space-side Viewpoint In the frequency domain In the spatial domain ˆϕ j,0,k (ξ) = 2 3j/4 2π W j,0(ξ)e i k,ξ, k Λ j ϕ j,0,k (x) = 2 3j/4 ϕ j (x k), W j,0 = 2π ˆϕ j and more generally ϕ j,l,k (x) = 2 3j/4 ϕ j (R θj,l (x R 1 θ j,l k)), All curvelets at a given scale are obtained by translating and rotating a single mother curvelet.

31 30 Further Properties Tight frame f = j,l,k f, ϕ j,l,k ϕ j,l,k f 2 2 = j,l,k f, ϕ j,l,k 2 Geometric Pyramid structure Dyadic scale Dyadic location Direction New tiling of phase space

32 31 Needle-shaped Scaling laws length 2 j/2 width 2 j width length 2 #Directions = 2 j/2 Doubles angular resolution at every other scale Unprecedented combination

33 32 Second Dyadic Decomposition Fefferman (70 s), boundedness of Riesz spherical means Stein, and Seeger, Sogge and Stein (90 s), L p boundedness of Fourier Integral Operators. Smith (90 s), atomic decomposition of Fourier Integral Operators.

34 33 Digital Curvelets Source: DCTvUSFFT (Digital Curvelet Transform via USFFT s), C. and Donoho (2004).

35 34 Digital Curvelets: Frequency Localization Time domain Frequency domain (modulus) Source: DCTvUSFFT (Digital Curvelet Transform via USFFT s), C. and Donoho (2004).

36 35 Digital Curvelets: : Frequency Localization Time domain Frequency domain (real part) Source: DCTvUSFFT (Digital Curvelet Transform via USFFT s), C. and Donoho (2004).

37 Curvelets and Edges 36

38 37 Optimality Suppose f is smooth except for discontinuity on C 2 curve Curvelet m-term approximations, naive thresholding f f curve m 2 2 C m 2 (log m) 3 Near-optimal rate of m-term approximation (wavelets m 1 ).

39 Idea of the Proof I 38 s f(x,y) Edge fragment -s Scale 2 Coefficient ~ 0 Decomposition of a Subband

40 39 Idea of the Proof II Frequency 2 s Scale 2 -s Microlocal behavior

41 40 Edge Fragments Parabolic edge Square root of the truncated absolute value of the FT Edge fragment FT of an edge fragment (abs)

42 41 Curvelets and Warpings C 2 change of coordinates preserves sparsity (C. 2002). Let ϕ : R 2 R 2 be a one to one C 2 function such that J ϕ is bounded away from zero and infinity. Curvelet expansion f = µ θ µ (f)γ µ, θ µ (f) = f, γ µ Likewise, f ϕ = µ θ µ (f ϕ)γ µ The coefficient sequences of f and f ϕ are equally sparse. Theorem 1 Then, for each p > 2/3, we have θ(f ϕ) lp C p θ(f) lp.

43 42 x µ(t) ξ µ(t) 2 j/2 ξ µ x µ 2 j Curvelets are nearly invariant through a smooth change of coordinates

44 43 Curvelets and Curved Singularities f smooth except along a C 2 curve f n, n-term approximation obtained by naive thresholding f f n 2 L 2 C (log n) 3 n 2 Why? 1. True for a straight edge 2. Deformation preserves sparsity φ Optimal arbitrary singularity straight singularity

45 Curvelets and Hyperbolic Differential Equations 44

46 45 Representation of Evolution Operators Wave Equation 2 t u = c2 (x) u, x R 2, R 3,... Evolution operator S(t; x, y) is dense! Modern viewpoint Basis ϕ n of L 2. Representation of S: A(n, n ) = ϕ n, Sϕ n F u 0 S u = Su 0 F, θ(u) = Sθ(u 0 ) θ(u 0 ) A θ(u) Find a representation in which S is sparse

47 46 Range of Influence t (x,t) x 0 Domain of Dependence The solution operator S is a dense integral.

48 47 Potential for Sparsity If the matrix A is sparse, potential for fast multiplication fast inversion Example: convolutions and Fourier transforms

49 48 Current Multiscale Thinking Traditional ideas: Multigrids Wavelets Adaptive FEM s etc. Traditional multiscale ideas are ill-adapted to wave problems: 1. they fail to sparsify oscillatory integrals like the solution operator 2. they fail to provide a sparse representation of oscillatory signals which are the solutions of those equations.

50 49 Peek at the Results Sparse representations of hyperbolic symmetric systems Connections with geometric optics Importance of parabolic scaling

51 50 Symmetric Systems of Differential Equations t u + A i (x) i u + B(x)u = 0, u(0, x) = u 0 (x), x R n, i u is m-dimensional and the A i s are symmetric Examples Maxwell Acoustic waves Linear elasticity

52 Example: Acoustic waves 51 System of hyperbolic equations: v = (u, p) t u + (c(x)p) = 0 t p + c(x) u = 0 Dispersion matrix 0 0 k 1 A j (x)k j = c(x) 0 0 k 2 j k 1 k 2 0 Eigenvalues (λ ν ): λ ± = ±c(x), λ 0 = 0. Eigenvectors (R ν ) R 0 (k) = k / k, R ± (k) = ±k/ k. 1

53 52 Geometric Optics: Lax, 1957 High-frequency wave-propagation approximation v(x, t) = ( e iωφ ν (x,t) a 0ν (x, t) + a 1ν(x, t) ω ν + a ) 2ν(x, t) +... ω 2 Plug into wave equation Eikonal equations t Φ ν + λ ν (x, x Φ) = 0. λ ν (x, k) are the eigenvalues of the dispersion matrix j A j(x)k j And transport equations for amplitudes Turns a linear equation into a nonlinear evolution equation!

54 53 Hamiltonian Flows Dispersion matrix A j (x)k j j Eigenvalues λ ν (x, k), eigenvectors R ν (x, k) Hamiltonian flows (in general, m of them) in phase-space ẋ(t) = k λ ν (x, k), x(0) = x 0, k(t) = x λ ν (x, k), k(0) = k 0. Eikonal equations from geometric optics t Φ ν + λ ν (x, x Φ) = 0. Φ is constant along the Hamiltonian flow Φ(t, x(t)) = Cste. Problem: valid before caustics, i.e. before Φ becomes multi-valued (ray-crosses).

55 54 Hyper-curvelets Frequency definition ˆΦ νµ (k) = E ν ˆϕ µ (k). Hyper-curvelets build-up a (vector-valued) tight-frame, namely u 2 L 2 = ν,µ [u, Φ νµ ] 2, and u = ν,µ[u, Φ νµ ]Φ νµ Other possibilities: ˆΦ νµ (k) = R ν (k) ˆϕ µ (k), Φ µν (x) = R ν (x, k) ˆϕ µν (k)e i k,x dk.

56 55 Approximation The action of the wave propagator on a curvelet is well- approximated by a rigid motion. Initial data: Φ νµ (x) Approximate solution ˆΦ νµ (t, k) = R ν (k) ˆϕ µ(t) (k) where ϕ µ(t) (x) = ϕ µ (U µ (t)(x x µ (t)) + x µ ) U µ (t) is a rotation x µ (t) is a translation

57 56 θ µ (t) x µ (t) θ µ' (t) x µ' (t) Rays θ µ ' θ µ x µ ' x µ

58 57 Main Result Curvelet representation of the propagator S(t): A(ν, µ; ν, µ ) = Φ νµ, S(t)Φ ν µ Theorem 2 (C. and Demanet) Suppose the coefficients of a general hyperbolic system are smooth, i.e. C. The matrix is sparse. Suppose a is either a row or a column of A, and let a (n) be the n-largest entry of the sequence a, then for each r > 0, a (n) obeys a (n) C r n r. The matrix is well-organized. There is a natural distance over the curvelet indices such that for each M, a(ν, µ; ν, µ ) C M,t (ν, ν ) ν (1 + d(µ, µ ν (t)) M The constant is growing at most like C 1 e C 2t, and for ν ν, C 1 << 1.

59 58 φ µ φ t(µ) Sketch of the curvelet representation of the wave propagator A(ν, µ ; ν, µ) = Φ νµ, S(t)Φ ν µ.

60 59 Our Claim Curvelets provide an optimally sparse representation of wave propagators. Fourier matrix is dense FEM matrix is dense Wavelet matrix is dense Second-Order Scalar Equations 2 t u i,j a ij (x) i j u = 0 Sparsity comes for free (in the scalar curvelet system)

61 60 Why Does This Work? The purpose of calculations is insight, not numbers... (adapted from R. W. Hamming) Wave operator (constant coefficients) 2 t u = c2 u, u(x, 0) = u 0 (x), t u(x, 0) = u 1 (x). Fourier transform û(t, k) = u(t, x)e ik x dx. Solution û(t, k) = cos(c k t)û 0 (k) + sin(c k t) û 1 (k) k û(t, k) = e ±ic k t û 0 (k) + e±ic k t û 1 (k) k

62 61 Set t = ct e i k t = e ik 1t +δ(k)t, δ(k) = k k 1. Because of parabolic scaling δ(k) = O(1) Frequency modulation is nearly linear cos( k t') ~ cos(k t') 1 e i k t = ĥ(k) eik 1t. ĥ smooth and non-oscillatory Space-side picture: 1. modulation corresponds to a displacement of ±ct in the codirection. 2. multiplication by ĥ: gentle convolution x µ Displacement in the codirection

63 62 cos( k t) ~ cos(k t) µ k µ x µ Displacement in the codirection

64 63 Warpings Variable coefficients: u tt = c 2 (x) u. At small scales, the velocity field varies varies little over the support of a curvelet. Model the effect of variable velocity field as a warping g: φ µ (g(x)). Warpings do not distort the geometry of a curvelet.

65 64 x µ(t) ξ µ(t) 2 j/2 ξ µ x µ 2 j

66 Importance of Parabolic Scaling 65 Consider arbitrary scaling (anisotropy incresases as α decreases) width 2 j, length 2 jα, 0 α 1. ridgelets α = 0 (very anisotropic), curvelets α = 1/2 (parabolic anisotropy), wavelets α = 1 (roughly isotropic). For wave-like behavior, need width length 2 For particle-like behavior, need width length 2 For both (simultaneously), need width length 2 Only working scaling: α = 1/2

67 Examples I j t = 0 t = T

68 Examples II 67

69 68 Examples III 2 -j 2 -αj 2 -(1-α)j t = 0 t = T

70 69 Architecture of the Argument 1. Decoupling into polarized components: Decompose the wavefield u(t, x) into m one-way components f ν (t, x) u(t, x) = m ν=1 R ν f ν (t, x), R ν : pseudo-differential operator (independent of time), maps scalar fields into m-dimensional vector fields. S(t) = m ν=1 R ν e itλ ν L ν + negligible, L ν : pseudo-differential operator, maps m-dimensional vector fields into scalar fields.

71 70 Curvelet Splitting = ( ) ( )

72 71 S(t) ν S ν (t), S ν (t) = R ν e itλ ν L ν 2. Fourier Integral Operator parametrix. We then approximate for small times t > 0 each e itλ ν, ν = 1,..., m, by an oscillatory integral or Fourier Integral Operator (FIO) F ν (t) F ν (t)f(x) = e iφ ν (t,x,k) σ ν (t, x, k) ˆf(k) dk Key issue: decoupling is crucial because this what makes it work for large times (not an automatic consequence of Lax s construction) S ν (nt) = S ν (t) n 3. Sparsity of FIO s. General FIO s F (t) are sparse and well-structured when represented in tight frames of (scalar) curvelets ϕ µ a result of independent interest.

73 72 Potential for Scientific Computing Transform this theoretical insight into effective algorithms Sources of inspiration Rokhlin Engquist & Osher Others

74 73 Our Viewpoint u t = e P t u 0 F u 0 e P t u t F For any t, A(t) is sparse θ 0 A(t) θ t Background: Fast and accurate Digital Curvelet Transform is available (with D. Donoho).

75 74 Preliminary Work Grid size N Accuracy ɛ Complexity O(N 1+δ ɛ δ ), any δ > 0, Conjecture O(N log N ɛ δ ) Arbitrary initial data Works in any dimension General procedure

76 Propagating curvelets is not geometric optics! 75

77 76 Summary New geometric multiscale ideas Key insight: geometry of Phase-Space New mathematical architecture Addresses new range of problems effectively Promising potential

78 Numerical Experiments, I 1

79 (a) Noisy Phantom (b) Curvelets

80 (c) Curvelets and TV (d) Curvelets and TV: Residuals

81 (e) Wavelets (f) Curvelets

82

83 (l) Noisy (m) Curvelets & TV: Residuals

84 (n) Noisy Scanline (o) True Scanline and Curvelets and TV Reconstruction

85 (a) Noisy Scanline (b) Curvelets (c) Curvelets and TV (d) True Scanline and Curvelets and TV Reconstruction Figure 10: Scanline Plots

86 (p) Original (q) Noisy

87 (r) Curvelets (s) Curvelets and TV

88 (t) Noisy Detail (u) Curvelets (v) Curvelets and TV

89 Numerical Experiments, II: Work of Jean-Luc Starck (CEA) 2

90

91

92 Curvelet Wavelet Curvelet

93

94

95 a) Simulated image (Gaussians+lines) b) Simulated image + noise c) A trous algorithm d) Curvelet transform e) coaddition c+d f) residual = e-b

96 Separation of Texture from Piecewise Smooth Content The separation task: decomposition of an image into a texture and a natural (piecewise smooth) scene part. = +

97 Data X n X t