Compressive sensing in the analog world Mark A. Davenport Georgia Institute of Technology School of Electrical and Computer Engineering
Compressive Sensing A D -sparse Can we really acquire analog signals with CS?
Challenge 1 If Map analog sensing to matrix multiplication x(t) is bandlimited, x(t) = y[m] = há m (t); x(t)i = 1X n= 1 1X n= 1 x[n]sinc(t=t s n) x[n] há m (t); sinc(t=t s n)i
Challenge 2 Map analog sparsity into a sparsifying dictionary x D
Candidate Analog Signal Models Model for 1 Sparsifying dictionary for a Sparsity level for x S multitone sum of tones overcomplete DFT? -sparse S X(F) - Typical model in CS - Coherence - Off-grid tones B nyq 2 0 B nyq 2
Candidate Analog Signal Models Model for 1 Sparsifying dictionary for a Sparsity level for x S multitone sum of tones overcomplete DFT? -sparse S K multiband sum of bands?? X(F) B - Landau - Bresler, Feng, Venkataramani - Eldar and Mishali B nyq 0 2 B nyq 2
The Problem with the DFT DTFT x[n] = Z W W X(f)e j2¼fn df; 8n X(f) 2W time-limiting 1 2 0 1 2 x = N 1 X k=0 X k e k N NOT SPARSE ; e f := 2 6 4 e j2¼f0 e j2¼f.. e j2¼f(n 1) 3 7 5 DFT 1 2 0 1 2
Discrete Prolate Spheroidal Sequences (DPSS s) Slepian [1978]: Given an integer and, the DPSS s are a collection of vectors that satisfy The DPSS s are perfectly time-limited, but when they are highly concentrated in frequency.
DPSS Eigenvalue Concentration ` ` The first eigenvalues. The remaining eigenvalues.
Another Perspective: Subspace Fitting Suppose that we wish to minimize over all subspaces of dimension. Optimal subspace is spanned by the first DPSS vectors.
DPSS Examples
DPSS s for Bandpass Signals
DPSS Dictionaries for CS Modulate DPSS vectors to center of each band: D = [D 1 ; D 2 ; : : : ; D J ] X(f) 2W approximately square if a 1 2 0 possible bands 1 2 Most multiband signals, when sampled and time-limited, are well-approximated by a sparse representation in D.
Empirical Results: DFT Comparison [Davenport and Wakin - 2012]
Empirical Results: DFT Comparison [Davenport and Wakin - 2012]
Recovery Guarantees? A D y b bx = Db
The Treachery of Images
The Treachery of A A D x Given, choice of is no longer unique Correlations in make it difficult to establish guarantees via standard tools If D D is poorly conditioned, we can have or kdb D k 2 À kb k 2 kdb D k 2 kb k 2
Signal-focused Recovery Strategy Focus on x instead of Measure error in terms of kbx xk 2 instead of kb k 2 p 1 ±k k k 2 kad k 2 p1 + ± k k k 2 p 1 ±k kd k 2 kad k 2 p1 + ± k kd k 2
initialize: until converged: proxy: identify: merge: update: CoSaMP r = y; x 0 = 0; ` = 0; = ; h = A r = f2s largest elements of jhjg T = [ ex = arg min ky Azk 2 supp(z)µt = fs largest elements of jexjg output: bx = x` x`+1 = exj r`+1 = y Ax`+1 ` = ` + 1
Key Steps = f2s largest elements of jhjg ex = arg min ky Azk 2 supp(z)µt = fs largest elements of jexjg x`+1 = exj R n Given a vector in, use hard thresholding to find best sparse approximation P R(D ) : orthogonal projector onto opt (z; S) = arg min j j=s kz P zk 2
Approximate Projection P R(D ) : orthogonal projector onto opt (z; S) = arg min j j=s kz P zk 2 S(z; S) : estimate of opt (z; S) kp opt z P S zk 2 min ² 1 kp opt zk 2 ; ² 2 kz P opt zk 2 measure quality of approximation in signal space, not coefficient space
initialize: until converged: output: proxy: identify: merge: update: Signal Space CoSaMP r = y; x 0 = 0; ` = 0; = ; bx = x` h = A r = S(h; 2S) T = [ ex = arg min ky Azk 2 z2r(d T ) = S(ex; S) x`+1 = P (ex) r`+1 = y Ax`+1 ` = ` + 1
Recovery Guarantees S x = D A D 4S Suppose there exists an -sparse such that and satisfies the -RIP of order. If we observe y = Ax + e, then kx x`+1 k 2 C 1 kx x`k 2 + C 2 kek 2 For ± 4k = 0:029; ² 1 = 0:1; ² 2 = 1; kx x`k 2 2 `kxk 2 + 25:4kek 2 [Davenport, Needell, and Wakin - 2013]
Practical Choices for S(z; S) Given z, we want to find an S -sparse such that z ¼ D Any sparse recovery algorithm! CoSaMP Orthogonal Matching Pursuit (OMP) -minimization followed by hard-thresholding `1 S(z; S) = H S µarg min w:dw=z kwk 1
Remaining Gaps None of the practical choices are proven to provide the desired approximate projections Experimental results suggest that (at least for certain dictionaries) none of these choices are sufficient Recent progress Hegde and Indyk (2013) Giryes and Needell (2013) weaker requirements on approximate projection
Conclusion Dealing with analog signals in the traditional compressive sensing framework requires new sparsifying dictionaries modified algorithms signal-focused analysis Many open questions remain provably near-optimal algorithms for computing approximate projections may actually involve the use of DPSS s efficient methods for handling both multiband and multitone signals simultaneously
Thank You!