Compressive Sensing and Beyond

Transcription

1 Compressive Sensing and Beyond Sohail Bahmani Gerorgia Tech.

2

3 Signal Processing Compressed Sensing

4 Signal Models

5 Classics: bandlimited The Sampling Theorem Any signal with bandwidth B can be recovered from its samples collected uniformly at a rate no less that f s = 2B. Claude Shannon Harry Nyquist 1 x(t) = n Z x n sinc f s t n 0 x n = x t, sinc f s t n

6 Classics: time-frequency X(τ, ω) = x t, w t τ e jωt x t = 1 2π R R X τ, ω e jωt dωdτ ψ m,n (t) = a m 2 ψ a m t nb x t = x t, ψ m,n (t) ψ m,n (t) m Z n Z D Gabor A. Haar I. Daubechies R. Coifman 2

7 Sparsity x F x = x Ω F Ω x T. Tao E. Candès J. Romberg = D. Donoho 3

8 Structured sparsity Tree Structured Models Graph Structured Models Discrete Wavelet Transform Protein-Protein Interactions Gene Regulatory Network Couch Lab, George Mason University, 4

9 Low rank Model for Images B. Recht Other applications minimum order linear system realization low-rank matrix completion low-dimensional Euclidean embedding M.Fazel P. Parrilo 5

10 Compressive Sensing

11 Linear inverse problems y A x Ax = y m = n x x = A y m < n No unique inverse, but 6

12 Sparse solutions argmin x 0 x subject to Ax = y x 1 x 0 = supp x supp x = i x i 0 x 2 Ax 1 Generally NP-hard! Tractable for many interesting A Ax 2 7

13 Specialize A Many sufficient conditions proposed Nullspace property, Incoherence, Restricted Eigenvalue, Restricted Isometry Property 1 δ k x 2 2 Ax δ k x 2 2, x: x 0 k x 1 d Ax 1 1 ± δd x 2 Ax 2 8

14 Randomness comes to rescue Random matrices can exhibit RIP: Random matrices with iid entries Gaussian, Rademacher (symmetric Bernoulli), Uniform Structured random matrices Random partial Fourier matrices Random circulant matrices RIP holds with high probability if m k log γ n 9

15 Basis Pursuit (l 1 -minimization) argmin x 1 x subject to Ax = y x 1 = i x i Theorem [Candès] If A satisfies δ 2k -RIP with δ 2k < 2 1, then BP recovers any k-sparse target exactly. 10

16 Value Basis Pursuit (l 1 -minimization) 1.5 Spar se Signal ` 1 Reconst r uct ion 1 CVX I ndex 11

17 Greedy algorithms OMP, StOMP, IHT, CoSAMP, SP, Iteratively estimate the support and the values on the support Many of them have RIP-based convergence guarantees CoSaMP (Needell and Tropp) Proxy vector z = A T (Ax y) supp x Z = supp z 2k 2 b = argmin x Ax y 2 s. t. x T c= 0 Converges with δ 4k 0.1 x z b Z T b k Update 12

18 Rice single-pixel camera Wakin et al oroginal object dim. 20% dim. 40% 13

19 Compressive MRI Traditional MRI Lustig et al CS MRI Acquisition Speed 14

20 Image super-resolution Yang et al D h α Sparse representation α D l α CS-based reconstruction Original 15

21 Low-Rank Matrix Recovery

22 Linear systems w/ low-rank solutions r y i = X = n 2 n U V 1 r, A i Simple least squares requires n 1 n 2 measurements Degrees of freedom is r n 1 + n 2 1 Can we close the gap? 16

23 Rank minimization argmin rank X X subject to A X = y If identifiable, it recovers the low-rank solution exactly Just like l 0 -minimization, it is generally NP-hard Special measurement operators A admit efficient solvers 17

24 Random measurements A i with iid entries exhibit low-rank RIP Gaussian Rademacher Universal Uniform Some structured matrices Rank-one A i = a i b i A i with dependent entries Instance optimal Standard RIP usually doesn t hold, but other approaches exist 18

25 Nuclear-norm minimization argmin X X subject to A X = y A X = y X = i σ i X Theorem [Recht, Fazel, Parrilo] If A obeys δ 5r -RIP with δ 5r < 0.1 then nuclear-norm minimization recovers any rank-r target exactly. 19

26 Matrix completion Alice 4 5? 3?? 1 Bob? 2? 4 5??... Yvon 5? ? Zelda 3 2?? Theorem [Candès and Recht] If M is a rank-r matrix from the random orthogonal model, then w.h.p. the nuclear-norm minimization recovers M exactly from O(rn 5 4 log n) uniformly observed entries, where n = n 1 n 2. 20

27 Robust PCA Ordinary PCA is sensitive to noise and outliers low-rank component Background Modeling M = L + S argmin L,S L + λ S 1 subject to P Ωobs L + S = Y sparse component Candès et al 21

28 Compressive multiplexing Ahmed and Romberg B x m t = α m ω e j2πωt ω= B Ω~R M + W log 3 MW W = 2B

29 Nonlinear CS

30 Sparsity-constrained minimization Squared error isn t always appropriate example : non-gaussian noise models argmin x f x subject to x 0 k It s challenging in its general form Objectives with certain properties allow approximations convex relaxation: l 1 -regularization greedy methods 23

31 Gene classification a i : microarray sample y i : binary sample label (healthy/diseased) model: p y a; x DNA Microarray Sparse Logistic Regression m x : sparse weights for genes argmin log 1 + e a Tx i y i a T i x x i=1 subject to x 0 k, x

32 Imaging under photon noise Photon noise follows a Poisson distribution y Ax i p y A; x = i y i! i Physical constraints A 0, Ax 0, x 0 SPIRAL-TAP argmin x 0 e Ax i log p y A; x + τr(x) Harmany et al x 1 W T x 1 x TV 25

33 Coherent diffractive imaging Szameit et al * I : diffracted field intensity Cx : real image * C : Shifts of generating function L : low-pass filter F : Fourier transform f x = LFCx 2 I 2 2 * Lima et al., ESRF ( 26

34 Gradient Support Pursuit w/ Raj and Boufounos Generalizes CoSaMP x 1 z 4 b 5 b k Proxy vector z = f x Z = supp z 2k b = argmin x f x s. t. x T c= 0 supp x 2 3 Z T Converges under SRH the Hessian 2 f x is well-conditioned when restricted to sparse subspaces a generalization of the RIP Update 27

35 There s much more 28

36 Resources a collection of CS papers, tutorials, and softwares sparse and low-rank algorithms wiki a weblog focusing on CS and broader computational areas 29