Sparse Recovery Beyond Compressed Sensing

Transcription

1 Sparse Recovery Beyond Compressed Sensing Carlos Fernandez-Granda Applied Math Colloquium, MIT 4/30/2018

2 Acknowledgements Project funded by NSF award DMS

3 Separable Nonlinear Inverse Problems Sparse Recovery for SNL Problems Super-resolution Deconvolution SNL Problems with Correlation Decay

4 Separable Nonlinear Inverse Problems Sparse Recovery for SNL Problems Super-resolution Deconvolution SNL Problems with Correlation Decay

5 Separable nonlinear inverse (SNL) problems Aim: estimate parameters θ 1,..., θ s R d from data y R n Relation between data and each θ j governed by nonlinear function φ Contributions of θ 1,..., θ s combine linearly with unknown coeffs c R s s y = c (j) φ(θ j ) j=1 = [ φ(θ 1 ) φ(θ 2 ) φ(θ s ) ] c n > s, easy if we know θ 1,..., θ s

6 SNL problems Super-resolution Deconvolution Source localization in EEG Direction of arrival in radar / sonar Magnetic-resonance fingerprinting

7 Magnetic-resonance fingerprinting (Ma et al, 2013) Goal: Estimate magnetic relaxation-time constants of tissues in a voxel 0.1 φ(θ) time (s) θ (ms)

8 Separable Nonlinear Inverse Problems Sparse Recovery for SNL Problems Super-resolution Deconvolution SNL Problems with Correlation Decay

9 Methods to tackle SNL problems Nonlinear least-squares solved by descent methods Drawback: local minima Prony-based / Finite-rate of innovation Drawback: challenging to apply beyond super-resolution Reformulate as sparse-recovery problem (drawbacks discussed at the end)

10 Linearization Linearize problem by lifting to a higher-dimensional space True parameters: θ T1,..., θ Ts Grid of parameters: θ 1,..., θ N, N >> n 0 y = [ φ(θ 1 ) φ(θ T1 ) φ(θ Ts ) φ(θ N ) ] c (1) c (s) 0 s = c (j) φ(θ Tj ) j=1

11 Sparse Recovery for SNL Problems Find a sparse c such that y = Φ grid c Underdetermined linear inverse problem with sparsity prior

12 Popular approach: l 1 -norm minimization minimize c 1 subject to Φ grid c = y

13 Popular approach: l 1 -norm minimization Deconvolution: Deconvolution with the l 1 norm, Taylor et al (1979) EEG: Selective minimum-norm solution of the biomagnetic inverse problem, Matsuura and Okabe (1995) Direction-of-arrival in radar / sonar: A sparse signal reconstruction perspective for source localization with sensor arrays, Malioutov et al (2005) and many, many others...

14 Magnetic-resonance fingerprinting Multicompartment magnetic resonance fingerprinting (Tang, F., Lannuzel, Bernstein, Lattanzi, Cloos, Knoll, Asslaender 2018) φ(θ 1 ) φ(θ 2 ) Data time (s) Ground truth Min. l 1 est θ (s)

15 Main question Under what conditions can SNL problems be solved by l 1 -norm minimization?

16 Continuous dictionary Analysis should apply to arbitrarily fine grids We model the coefficients / parameter values as an atomic measure s x := c (j) δ θtj j=1 s y = c (j) φ(θ Tj ) j=1 = φ(θ) x( dθ) = Φx Intuitively, Φ is a continuous dictionary with n rows

17 Sparse Recovery for SNL Problems Find a sparse x such that y = φ(θ) x( dθ) (Extremely) underdetermined linear inverse problem with sparsity prior

18 Total-variation norm Continuous counterpart of the l 1 norm Not the total variation of a piecewise-constant function c 1 = x TV = sup v, c v 1 sup f (t) x ( dt) f C [0,1], f 1 [0,1] If x = j c jδ θj then x TV = c 1

19 Main question For an SNL problem, when does minimize subject to x TV φ(θ) x( dθ) = y achieve exact recovery? Wait, isn t this just compressed sensing?

20 Compressed sensing Recover s-sparse vector x of dimension m from n < m measurements y = Ax Key assumption: A is random, and hence satisfies restricted-isometry properties with high probability

21 Restricted isometry property (Candès, Tao 2006) An m n matrix A satisfies the restricted isometry property (RIP) if there exists 0 < κ < 1 such that for any s-sparse vector x (1 κ) x 2 Ax 2 (1 + κ) x 2 2s-RIP implies that for any s-sparse signals x 1, x 2 Ax 2 Ax 1 2

22 Restricted isometry property (Candès, Tao 2006) An m n matrix A satisfies the restricted isometry property (RIP) if there exists 0 < κ < 1 such that for any s-sparse vector x (1 κ) x 2 Ax 2 (1 + κ) x 2 2s-RIP implies that for any s-sparse signals x 1, x 2 Ax 2 Ax 1 2 = A (x 2 x 1 ) 2

23 Restricted isometry property (Candès, Tao 2006) An m n matrix A satisfies the restricted isometry property (RIP) if there exists 0 < κ < 1 such that for any s-sparse vector x (1 κ) x 2 Ax 2 (1 + κ) x 2 2s-RIP implies that for any s-sparse signals x 1, x 2 Ax 2 Ax 1 2 = A (x 2 x 1 ) 2 (1 κ) x 2 x 1 2

24 Separable nonlinear problems If φ is smooth, nearby columns in Φ grid := [ φ(θ 1 ) φ(θ 2 ) φ(θ N ) ] are highly correlated so RIP does not hold! There are x 1, x 2 such that Ax 1 Ax 2 Sparsity is not enough, we need additional restrictions!

25 Separable Nonlinear Inverse Problems Sparse Recovery for SNL Problems Super-resolution Deconvolution SNL Problems with Correlation Decay

26 Super-resolution Joint work with Emmanuel Candès (Stanford)

27 Limits of resolution in imaging The resolving power of lenses, however perfect, is limited (Lord Rayleigh) Diffraction imposes a fundamental limit on the resolution of optical systems

28 Fluorescence microscopy Data Point sources Low-pass blur (Figures courtesy of V. Morgenshtern)

29 Sensing model for super-resolution Point sources Point-spread function Data = Spectrum =

30 Super-resolution s sources with locations θ 1,..., θ s, modeled as superposition of spikes x = j c (j)δ θj c j C, θ j T [0, 1] We observe Fourier coefficients up to cut-off frequency f c y(k) = = 1 0 exp ( i2πkt) x (dt) s c (j) exp ( i2πkθ j ) j=1 SNL problem where exp ( i2πθ j ( f c )) φ(θ j ) = exp ( i2πθ j f c )

31 Fundamental questions 1. Is the problem well posed? 2. Does TV -norm minimization work?

32 Is the problem well posed? = Measurement operator = low-pass samples with cut-off frequency f c

33 Is the problem well posed? = Effect of measurement operator on sparse vectors?

34 Is the problem well posed? = Submatrix can be very ill conditioned!

35 Is the problem well posed? = If support is spread out there is hope

36 Minimum separation The minimum separation of the support of x is = inf θ θ (θ,θ ) support(x) : θ θ

37 Conditioning of submatrix with respect to If < 1/f c the problem is ill posed If > 1/f c the problem becomes well posed Proved asymptotically by Slepian and non-asymptotically by Moitra 1/f c is the diameter of the main lobe of the point-spread function (twice the Rayleigh distance)

38 Example: 25 spikes, f c = 10 3, = 0.8/f c Signals Data (in signal space)

39 Example: 25 spikes, f c = 10 3, = 0.8/f c 0.3 Signal 1 Signal Signal 1 Signal Signals Data (in signal space)

40 Example: 25 spikes, f c = 10 3, = 0.8/f c The difference is almost in the null space of the measurement operator ,000 1, ,000 2,000 Difference Spectrum

41 Theoretical questions 1. Is the problem well posed? 2. Does TV -norm minimization work?

42 Super-resolution via TV-norm minimization minimize subject to x TV φ(θ) x( dθ) = y

43 Dual certificate for TV-norm minimization v R n is a dual certificate associated to x = j c j δ θj c j R, θ j T if Q (θ) := v T φ (θ) Q (θ j ) = sign (c j ) if θ j T Q (θ) < 1 if θ / T Dual variable guaranteeing that x TV is optimal

44 Dual certificate For any x + h such that φ(θ) h( dθ) = 0 x + h TV = sup f (θ) x ( dθ) + f (θ) h ( dθ) f 1 [0,1] [0,1]

45 Dual certificate For any x + h such that φ(θ) h( dθ) = 0 x + h TV = sup f (θ) x ( dθ) + f 1 [0,1] [0,1] Q (θ) x ( dθ) + [0,1] [0,1] f (θ) h ( dθ) Q (θ) h ( dθ)

46 Dual certificate For any x + h such that φ(θ) h( dθ) = 0 x + h TV = sup f (θ) x ( dθ) + f 1 [0,1] θ j T [0,1] Q (θ) x ( dθ) + [0,1] [0,1] [0,1] Q (θ) h ( dθ) Q (θ j ) c j δ θj ( dθ) + v T f (θ) h ( dθ) [0,1] φ (θ) h ( dθ)

47 Dual certificate For any x + h such that φ(θ) h( dθ) = 0 x + h TV = sup f (θ) x ( dθ) + f 1 [0,1] θ j T [0,1] Q (θ) x ( dθ) + = x TV [0,1] [0,1] [0,1] Q (θ) h ( dθ) Q (θ j ) c j δ θj ( dθ) + v T f (θ) h ( dθ) [0,1] φ (θ) h ( dθ)

48 Dual certificate For any x + h such that φ(θ) h( dθ) = 0 x + h TV = sup f (θ) x ( dθ) + f 1 [0,1] θ j T [0,1] Q (θ) x ( dθ) + = x TV [0,1] [0,1] [0,1] Q (θ) h ( dθ) Q (θ j ) c j δ θj ( dθ) + v T f (θ) h ( dθ) [0,1] φ (θ) h ( dθ) Existence of Q for any sign pattern implies that x is the unique solution

49 Dual certificate for super-resolution v C n is a dual certificate associated to x = j c j δ θj c j C, θ j T if Q (θ) := v φ (θ) = f c k= f c v k exp (i2πkθ) Q (θ j ) = sign (c j ) if θ j T Q (θ) < 1 if θ / T Linear combination of low pass sinusoids

50 Certificate for super-resolution Aim: Interpolate sign pattern

51 Certificate for super-resolution Interpolation with a low-frequency fast-decaying kernel F q(t) = θ j T α j F (t θ j )

52 Certificate for super-resolution Interpolation with a low-frequency fast-decaying kernel F q(t) = θ j T α j F (t θ j )

53 Certificate for super-resolution Interpolation with a low-frequency fast-decaying kernel F q(t) = θ j T α j F (t θ j )

54 Certificate for super-resolution Interpolation with a low-frequency fast-decaying kernel F q(t) = θ j T α j F (t θ j )

55 Certificate for super-resolution Interpolation with a low-frequency fast-decaying kernel F q(t) = θ j T α j F (t θ j )

56 Certificate for super-resolution Technical detail: Magnitude of certificate locally exceeds 1

57 Certificate for super-resolution Technical detail: Magnitude of certificate locally exceeds 1 Solution: Add correction term and force derivative to vanish on support Q(θ) = θ j T α j F (θ θ j ) + β j F (θ θ j )

58 Certificate for super-resolution Technical detail: Magnitude of certificate locally exceeds 1 Solution: Add correction term and force derivative to vanish on support Q(θ) = θ j T α j F (θ θ j ) + β j F (θ θ j )

59 Guarantees for super-resolution Theorem [Candès, F. 2012] If the minimum separation of the signal support obeys 2 /f c then recovery via convex programming is exact Theorem [Candès, F. 2012] In 2D convex programming super-resolves point sources with a minimum separation of 2.38 /f c where f c is the cut-off frequency of the low-pass kernel

60 Guarantees for super-resolution Theorem [F. 2016] If the minimum separation of the signal support obeys 1.26 /f c, then recovery via convex programming is exact Theorem [Candès, F. 2012] In 2D convex programming super-resolves point sources with a minimum separation of 2.38 /f c where f c is the cut-off frequency of the low-pass kernel

61 Numerical evaluation of minimum separation f c = 30 f c = 40 f c = 50 1 Number of spikes Number of spikes Number of spikes Minimum separation min fc Minimum separation min fc Minimum separation min fc 0 Numerically TV-norm minimization succeeds if 1 f c

62 Separable Nonlinear Inverse Problems Sparse Recovery for SNL Problems Super-resolution Deconvolution SNL Problems with Correlation Decay

63 Deconvolution Joint work with Brett Bernstein (Courant)

64 Seismology

65 Reflection seismology Geological section Acoustic impedance Reflection coefficients

66 Reflection seismology Sensing Ref. coeff. Pulse Data Data convolution of pulse and reflection coefficients

67 Model for the pulse: Ricker wavelet σ σ

68 Toy model for reflection seismology

69 Toy model for reflection seismology

70 Toy model for reflection seismology

71 Toy model for reflection seismology

72 Toy model for reflection seismology

73 Toy model for reflection seismology

74 Toy model for reflection seismology

75 Toy model for reflection seismology

76 Toy model for reflection seismology

77 Deconvolution s sources with locations θ 1,..., θ s, modeled as superposition of spikes x = j c (j)δ θj c j R, θ j T [0, 1] We observe samples of convolution with kernel K y(k) = (K x) (s k ) s = c (j) K (s k θ j ) j=1 SNL problem where K (s 1 θ j ) φ(θ j ) = K (s n θ j )

78 Theoretical questions 1. Is the problem well posed? 2. Does TV -norm minimization work?

79 Minimum separation Kernels are approximately low-pass The support cannot be too clustered

80 Sampling proximity We need two samples per spike Convolution kernel decays: at least two samples close to each spike Samples S and support T have sample proximity γ if for every θ i T there exist s i, s i S such that θ i s i γ and θ i s i γ We consider arbitrary non-uniform sampling patterns with fixed γ

81 Sampling proximity γ γ γ γ

82 Theoretical questions 1. Is the problem well posed? 2. Does TV -norm minimization work?

83 Deconvolution via TV-norm minimization minimize subject to x TV φ(θ) x( dθ) = y

84 Dual certificate for SNL problems v R n is a dual certificate associated to x = j c j δ θj c j R, θ j T if n Q (θ) := v T φ (θ) = v k K (s k θ) k=1 Q (θ j ) = sign (c j ) if θ j T Q (θ) < 1 if θ / T Linear combination of shifted copies of K fixed at the samples

85 Certificate for deconvolution +1 θ 1 θ 2 θ 3 1

86 Certificate construction Only use subset S containing 2 samples close to each spike Q(θ) = sj S v j K (s j θ) Fit v so that for all θ i T Q (θ i ) = sign (c i ) Q (θ i ) = 0

87 It works! +1 s 2,1 θ2 s 2,2 s 1,1 θ 1 s 1,2 s 3,1 θ 3 s 3,2 Gaussian Kernel 1

88 It works! +1 s 2,1 θ2 s 2,2 s 1,1 θ 1 s 1,2 s 3,1 θ 3 s 3,2 Ricker Kernel 1

89 Certificate construction Problem: The construction is difficult to analyze (coefficients vary) Solution: Reparametrization into bumps and waves Q(θ) = sj S v j K (s j θ) = θ i T α i B θi (θ, s i,1, s i,2 ) + β i W θi (θ, s i,1, s i,2 ),

90 Bump function (Gaussian kernel) +1 Bump Gaussian s 1 θ i s 2 B θi (θ, s i,1, s i,2 ) := b i,1 K( s i,1 θ) + b i,2 K( s i,2 θ) B θi (θ i, s i,1, s i,2 ) = 1 θ B θ i (θ i, s i,1, s i,2 ) = 0

91 Wave function (Gaussian kernel) Wave Gaussian s 1 s 2 θ i W θi (θ, s i,1, s i,2 ) = w i,1 K( s i,1 θ) + w i,2 K( s i,2 θ) W θi (θ i, s i,1, s i,2 ) = 0 θ W θ i (θ i, s i,1, s i,2 ) = 1

92 Bump function (Ricker wavelet) +1 Bump Ricker s 1 θ i s 2 B θi (θ, s i,1, s i,2 ) := b i,1 K( s i,1 θ) + b i,2 K( s i,2 θ) B θi (θ i, s i,1, s i,2 ) = 1 θ B θ i (θ i, s i,1, s i,2 ) = 0

93 Wave function (Ricker wavelet) Wave Ricker s 1 s 2 θ i W θi (θ, s i,1, s i,2 ) = w i,1 K( s i,1 θ) + w i,2 K( s i,2 θ) W θi (θ i, s i,1, s i,2 ) = 0 θ W θ i (θ i, s i,1, s i,2 ) = 1

94 Certificate construction Reparametrization decouples the coefficients Q(θ) = sj S v j K (s j θ) = θ i T α i B θi (θ, s i,1, s i,2 ) + β i W θi (θ, s i,1, s i,2 ) sign (c i ) B θi (θ, s i,1, s i,2 ) θ i T

95 Certificate for deconvolution (Gaussian kernel) +1 s 2,1 θ2 s 2,2 s 1,1 θ 1 s 1,2 s 3,1 θ 3 s 3,2 Gaussian Kernel 1

96 Certificate for deconvolution (Gaussian kernel) +1 s 2,1 θ 2 s 2,2 s 1,1 θ 1 s 1,2 s 3,1 θ 3 s 3,2 Bump Wave 1

97 Certificate for deconvolution (Ricker wavelet) +1 s 2,1 θ2 s 2,2 s 1,1 θ 1 s 1,2 s 3,1 θ 3 s 3,2 Ricker Kernel 1

98 Certificate for deconvolution (Ricker wavelet) +1 s 2,1 θ2 s 2,2 Bump Wave s 1,1 θ 1 s 1,2 s 3,1 θ 3 s 3,2 1

99 Exact recovery guarantees [Bernstein, F. 2017] 1.2 Gaussian Kernel Sample Proximity Numerical Recovery Exact Recovery Spike Separation

100 Exact recovery guarantees [Bernstein, F. 2017] 0.8 Ricker Kernel Sample Proximity Numerical Recovery Exact Recovery Spike Separation

101 Separable Nonlinear Inverse Problems Sparse Recovery for SNL Problems Super-resolution Deconvolution SNL Problems with Correlation Decay

102 SNL problems Joint work with Brett Bernstein (Courant) and Sheng Liu (CDS, NYU)

103 General SNL problems The function φ may not be available explicitly but can often be computed numerically by solving a differential equation Source localization in EEG Direction of arrival in radar / sonar Magnetic-resonance fingerprinting

104 Mathematical model Signal: superposition of Dirac measures with support T x = j c j δ θj c j R, θ j T [0, 1] Data: n measurements following SNL model y = φ (θ) x (dθ)

105 Diffusion on a rod with varying conductivity θ 1

106 Diffusion on a rod with varying conductivity θ 2 θ 1

107 Diffusion on a rod with varying conductivity θ 2 θ 1 θ 3

108 Diffusion on a rod with varying conductivity

109 Diffusion on a rod with varying conductivity φ(θ) can be computed by solving differential equation s 0 s 1 s 2 s 3 s 4 s 5 s 6 s 7 s 8 s 9 s 10s 11s 12s 13s 14s 15s 16s 17s 18s 19s 20s 21s 22s 23s 24s 25s 26s 27s 28 θ 0 θ 1 θ 2

110 Time-frequency pulses θ 1

111 Time-frequency pulses θ 2 θ 1

112 Time-frequency pulses θ 1 θ 2 θ 3

113 Time-frequency pulses

114 Time-frequency pulses Gabor wavelets in 1D φ(θ) k = exp ( (s k θ) 2 ) sin(150 s k (s k θ)) θ [0, 1] 2σ θ 0 θ 1 θ 2 s 0 s 1 s 2 s 3 s 4 s 5 s 6 s 7 s 8 s 9 s 10 s 11 s 12 s 13 s 14 s 15 s 16 s 17 s 18 s 19 s 20 s 21 s 22

115 Sparse estimation for general SNL problems Problem: Sparse recovery requires RIP-like properties that do not hold for SNL problems with smooth φ (even if we discretize) We cannot hope to recover all sparse signals How about signals such that φ(θ i ) T φ(θ j ) is small for all θ i θ j in T? Challenge: Prove guarantees for general SNL problems that only depend on correlation structure

116 SNL problems with correlation decay Aim: Guarantees for signals under separation conditions with respect to support-centered correlations ρ 1,..., ρ s ρ i (θ) := φ(θ i ) T φ(θ)

117 Diffusion on a rod with varying conductivity s 0 s 1 s 2 s 3 s 4 s 5 s 6 s 7 s 8 s 9 s 10s 11s 12s 13s 14s 15s 16s 17s 18s 19s 20s 21s 22s 23s 24s 25s 26s 27s 28 θ 0 θ 1 θ 2

118 Support-centered correlations θ 0 θ 1 θ 2

119 Time-frequency pulses φ(θ) k = exp ( (s k θ) 2 ) sin(150 s k (s k θ)) θ [0, 1] 2σ θ 0 θ 1 θ 2 s 0 s 1 s 2 s 3 s 4 s 5 s 6 s 7 s 8 s 9 s 10 s 11 s 12 s 13 s 14 s 15 s 16 s 17 s 18 s 19 s 20 s 21 s 22

120 Support-centered correlations θ 0 θ 1 θ 2

121 Correlation decay Parametrized by F i < N i < N + i < F + i and σ i Parameters can be different at each θ i Fi 2σ i θ i 1σ Fi Ni i N i + F i + F i + i + 1σ i F i + + 2σ i F

122 Correlation decay ρ i is concave in [N i, N + i ]: ρ i (θ) < γ 0 ρ i is bounded outside [N i, N + i ]: ρ i (θ) < γ 1 ρ i decays for θ < F i and θ > F + i : ρ i (θ) < γ 2 e ( θ F i )/σ i for θ < F i ρ i (θ) < γ 2 e ( θ F + i )/σ i for θ > F + Choice of exponential decay is arbitrary i

123 Correlation decay Additional condition on correlation derivatives ρ (q,r) i (θ) := φ (q) (θ i ) T φ (r) (θ) ρ (q,r) i ρ (q,r) i ρ (q,r) i decays for θ < F i and θ > F + i, for q = 0, 1, r = 0, 1, 2: (θ) < γ 2 e ( θ F i )/σ i for θ < F i (θ) < γ 2 e ( θ F + i )/σ i for θ > F + Open question: Is this necessary? i

124 Normalized distance Normalized distance from θ to θ j > θ d j (θ) := F j σ j θ If d j (θ) is large, φ(θ) and φ(θ j ) are not very correlated θ 1 θ F2 σ2 d 2 (θ) θ 2 F 3 σ 3 d 3 (θ) θ 3

125 Minimum separation conditions Normalized distance between spikes is equal to θ 1 F 1 + σ F2 2 θ 2 F 3 θ 3 2 σ 3 θ 1 θ 1 2 F 2 σ2 2 θ 2 F 3 θ 3 3 σ3 2

126 Guarantees for SNL problems with decaying correlation Theorem [Bernstein, F. 2018] For any SNL problem with decaying correlation TV-norm minimization achieves exact recovery under the separation conditions if > C for a fixed constant C depending on the decay bounds γ 0, γ 1, γ 2

127 Dual certificate for SNL problems v R n is a dual certificate associated to x = j c j δ θj c j R, θ j T if Q (θ) := v T φ (θ) Q (θ j ) = sign (c j ) if θ j T Q (θ) < 1 if θ / T

128 Dual certificate construction Use support-centered correlations to interpolate sign pattern Q (θ) := s α i ρ i (θ) i=1 +1 1

129 Dual certificate construction Use support-centered correlations to interpolate sign pattern Q (θ) := s α i ρ i (θ) i=1 +1 1

130 Dual certificate construction Use support-centered correlations to interpolate sign pattern Q (θ) := s α i ρ i (θ) i=1 +1 1

131 Dual certificate construction Use support-centered correlations to interpolate sign pattern Q (θ) := s α i ρ i (θ) i=1

132 Dual certificate construction Use support-centered correlations to interpolate sign pattern s Q (θ) := α i ρ i (θ) i=1 s = α i φ(θ i ) T φ(θ) i=1

133 Dual certificate construction Use support-centered correlations to interpolate sign pattern s Q (θ) := α i ρ i (θ) i=1 s = α i φ(θ i ) T φ(θ) i=1 = v T φ(θ) v := s α i φ(θ i ) i=1

134 Dual certificate construction Use support-centered correlations to interpolate sign pattern s Q (θ) := α i ρ i (θ) i=1 s = α i φ(θ i ) T φ(θ) i=1 = v T φ(θ) v := s α i φ(θ i ) Technical detail: Correction term to ensure derivative vanishes i=1

135 Robustness to noise / outliers Variations of dual certificates establish robustness at small noise levels (Candès, F. 2013), (F. 2013), (Bernstein, F. 2017) Exact recovery with constant number of outliers (up to log factors) (F., Tang, Wang, Zheng 2017), (Bernstein, F. 2017) Open questions: Analysis of higher-noise levels and discretization error, robustness for positive amplitudes

136 Drawbacks Solving convex program is computationally expensive Approach doesn t scale well at high dimensions In practice, reweighting is need to obtain sparse solutions for noisy data Open question: Analysis of other techniques (reweighting methods, descent methods on nonconvex cost functions)

137 Conclusion Previous works focus mostly on random operators For deterministic problems sparsity is not enough! Under separation conditions: 1. Sharp guarantees for super-resolution and deconvolution 2. General guarantees for SNL problems with correlation decay

138 References Prolate spheroidal wave functions, Fourier analysis, and uncertainty V V - The discrete case. D. Slepian. Bell System Technical Journal, 1978 Super-resolution, extremal functions and the condition number of Vandermonde matrices. A. Moitra. Symposium on Theory of Computing (STOC), 2015 The recoverability limit for superresolution via sparsity. L. Demanet, N. Nguyen Robust recovery of stream of pulses using convex optimization. T. Bendory, S. Dekel, A. Feuer. J. of Math. Analysis and App., 2016 Super-resolution without separation. G. Schiebinger, E. Robeva, B. Recht. Information and Inference, 2016

139 References Towards a mathematical theory of super-resolution. E. J. Candès, C. Fernandez-Granda. Comm. on Pure and Applied Math., 2013 Super-resolution from noisy data. E. J. Candès, C. Fernandez-Granda. J. of Fourier Analysis and Applications, 2014 Support detection in super-resolution. C. Fernandez-Granda. Proceedings of SampTA 2013 Super-resolution of point sources via convex programming. C. Fernandez-Granda. Information and Inference, 2016

140 References Demixing sines and spikes: robust spectral super-resolution in the presence of outliers. C. Fernandez-Granda, G. Tang, X. Wang, L. Zheng. Information and Inference, 2017 Multicompartment magnetic-resonance fingerprinting. S. Tang, C. Fernandez-Granda, S. Lannuzel, B. Bernstein, R. Lattanzi, M. Cloos, F. Knoll, J. Asslaender Deconvolution of point sources: A sampling theorem and robustness guarantees. B. Bernstein, C. Fernandez-Granda Sparse recovery beyond compressed sensing: Separable nonlinear inverse problems. B. Bernstein, C. Fernandez-Granda, S. Liu