Optimization-based sparse recovery: Compressed sensing vs. super-resolution

Transcription

1 Optimization-based sparse recovery: Compressed sensing vs. super-resolution Carlos Fernandez-Granda, Google Computational Photography and Intelligent Cameras, IPAM 2/5/2014

2 This work was supported by a Fundación La Caixa Fellowship and a Fundación Caja Madrid Fellowship Joint work with Emmanuel Candès (Stanford)

3 Optimization-based recovery Optimization Object Sensing system Data

4 Outline Two inverse problems When is the problem well posed? When do optimization-based methods succeed?

5 Two inverse problems When is the problem well posed? When do optimization-based methods succeed?

6 Compressed sensing Object Data Compressible Randomized Mathematical model: Random Fourier coecients of sparse signal

7 Super-resolution Object Data Point sources Low-pass blur Mathematical model: Low-pass Fourier coecients of sparse signal (Figures courtesy of V. Morgenshtern)

8 Motivation: Limits of resolution in imaging The resolving power of lenses, however perfect, is limited (Lord Rayleigh) Diraction imposes a fundamental limit on the resolution of optical systems

9 Super-resolution Spectrum Object Data

10 Super-resolution Optics: Data-acquisition techniques to overcome the diraction limit Image processing: Methods to upsample images onto a ner grid while preserving edges and hallucinating textures This talk: Estimation/deconvolution from low-pass measurements

11 Compressed sensing Super-resolution Spectrum interpolation Spectrum extrapolation

12 Two inverse problems When is the problem well posed? When do optimization-based methods succeed?

13 Compressed sensing x = Spectrum of x Fourier series of a measure/function x with domain [0, 1]

14 Compressed sensing x = Spectrum of x Discrete Fourier transform (DFT) of a vector x R N

15 Compressed sensing x = Spectrum of x Data: Random DFT coecients

16 Compressed sensing = x Data: Random DFT coecients

17 Compressed sensing = x What is the eect of the measurement operator on sparse vectors?

18 Compressed sensing = x Crucial insight: restricted operator is well conditioned when acting upon any sparse signal (restricted isometry property) [Candès, Tao 2006]

19 Compressed sensing = x Crucial insight: restricted operator is well conditioned when acting upon any sparse signal (restricted isometry property) [Candès, Tao 2006]

20 Super-resolution Signal: superposition of spikes (Dirac measures) in the unit interval x = j a j δ tj a j C, t j [0, 1] Data: low-pass Fourier coecients with cut-o frequency f c y(k) = 1 0 e i 2πkt x (dt) = j a j e i 2πkt j, k Z, k f c

21 Super-resolution x = Spectrum of x Fourier series of a measure x with domain [0, 1]

22 Super-resolution y x = Data: Low-pass Fourier coecients

23 Super-resolution = y x Data: Low-pass Fourier coecients

24 Super-resolution = y x Problem: If the support is clustered, the problem may be ill posed In super-resolution sparsity is not enough!

25 Super-resolution = y x If the support is spread out, there is still hope We need conditions beyond sparsity

26 Minimum separation The minimum separation of the support of x is = inf t t (t,t ) support(x) : t t

27 Conditioning of submatrix with respect to If < λ c := 1/f c the problem is ill posed λ c is the diameter of the main lobe of the point-spread function (twice the Rayleigh distance)

28 Example: 25 spikes, f c = 10 3, = 0.8/f c Signals Data (in signal space)

29 Example: 25 spikes, f c = 10 3, = 0.8/f c 0.3 Signal 1 Signal Signal 1 Signal Signals Data (in signal space)

30 Example: 25 spikes, f c = 10 3, = 0.8/f c The dierence is almost in the null space of the measurement operator ,000 1, ,000 2,000 Dierence Spectrum

31 Two inverse problems When is the problem well posed? When do optimization-based methods succeed?

32 Estimation via convex programming Measurement model: Underdetermined linear system (we need further assumptions) Ax y Idea: Impose nonparametric assumptions on structure by minimizing a cost function min x C ( x) subject to A x = y, In our case: l 1 norm to enforce sparsity

33 Subgradients Denition: g x any vector v is a subgradient of a convex function C at x if for C (x + v) C (x) + g x, v Lemma If there is a subgradient of C at x in the range of A, g x = A u, and Ax = y then x is a solution of min x C ( x) subject to A x = y Proof: For all x such that Ax = y, so that A (x x) = 0, C ( x ) C (x) + g x, x x = C (x) + A u, x x = C (x) + ( u, A x x) = C (x)

34 Dual certicate for the l 1 norm Lemma x is a solution to min x x 1 subject to A x = y if Ax = y and there exists g x = A u such that g x (j) = sign {x (j)} if j support (x) g x (j) 1 if j / support (x)

35 Dual certicate for the l 1 norm Lemma x is the unique solution to min x x 1 subject to A x = y if Ax = y and there exists g x = A u such that g x (j) = sign {x (j)} if j support (x) g x (j) < 1 if j / support (x)

36 Dual certicate for the l 1 norm Lemma x is solution to min x x 1 subject to A x = y if Ax = y and there exists g x = A u such that g x (j) = sign {x (j)} if j support (x) g x (j) < 1 if j / support (x) The range of A corresponds to Compressed sensing: Random sinusoids Super-resolution: Low-pass sinusoids

37 Dual certicate for compressed sensing Least-squares interpolator

38 Dual certicate for compressed sensing Works out for linear levels of sparsity (up to logarithmic factors) [Candès, Romberg, Tao 2006]

39 Dual certicate for super-resolution Least-squares interpolator does not work

40 Dual certicate for super-resolution st idea: Interpolation with a low-frequency fast-decaying kernel K g x (t) = α j K(t t j ), t j support(x)

41 Dual certicate for super-resolution st idea: Interpolation with a low-frequency fast-decaying kernel K g x (t) = α j K(t t j ), t j support(x)

42 Dual certicate for super-resolution st idea: Interpolation with a low-frequency fast-decaying kernel K g x (t) = α j K(t t j ), t j support(x)

43 Dual certicate for super-resolution st idea: Interpolation with a low-frequency fast-decaying kernel K g x (t) = α j K(t t j ), t j support(x)

44 Dual certicate for super-resolution st idea: Interpolation with a low-frequency fast-decaying kernel K g x (t) = α j K(t t j ), t j support(x)

45 Dual certicate for super-resolution Problem: Magnitude of polynomial locally exceeds 1

46 Dual certicate for super-resolution Problem: Magnitude of polynomial locally exceeds 1 Solution: Add correction term and force g x(t k ) = 0 for all t k support (x) g x (t) = α j K(t t j ) + β j K (t t j ) t j support(x)

47 Dual certicate for super-resolution Problem: Magnitude of polynomial locally exceeds 1 Solution: Add correction term and force g x(t k ) = 0 for all t k support (x) g x (t) = α j K(t t j ) + β j K (t t j ) t j support(x)

48 Guarantees for super-resolution Theorem [Candès, F. 2012] If the minimum separation of the signal support obeys 2 /f c then recovery via l 1 norm minimization is exact

49 Guarantees for super-resolution Theorem [Candès, F. 2014] If the minimum separation of the signal support obeys 1.28 /f c, then recovery via l 1 norm minimization is exact

50 Guarantees for super-resolution Theorem [Candès, F. 2014] If the minimum separation of the signal support obeys 1.28 /f c, then recovery via l 1 norm minimization is exact Theorem [Candès, F. 2012] In 2D l 1 -norm minimization super-resolves point sources with a minimum separation of 2.38 /f c where f c is the cut-o frequency of the low-pass kernel

51 Guarantees for super-resolution Results hold for continuous version of the l 1 norm (no discretization) Numerical simulations show that the method works for 1/f c Generalizations of dual certicate allow to prove robustness to noise [Candès, F. 2013], [F. 2013] If the signal is sparse, we can randomly undersample low-pass measurements [Tang, Bhaskar, Shah, Recht 2013]

52 Conclusion Characterizing the interaction between the measurement operator and the structure of the object of interest is crucial to understand When the problem is well posed (conditioning of restricted operator) When optimization-based methods succeed (dual certicates)

53 For more details Towards a mathematical theory of super-resolution. E. J. Candès and C. Fernandez-Granda. Communications on Pure and Applied Math 67(6), Super-resolution from noisy data. E. J. Candès and C. Fernandez-Granda. Journal of Fourier Analysis and Applications 19(6), Support detection in super-resolution. C. Fernandez-Granda. Proceedings of SampTA 2013,