Part IV Compressed Sensing

Transcription

1 Aisenstadt Chair Course CRM September 2009 Part IV Compressed Sensing Stéphane Mallat Centre de Mathématiques Appliquées Ecole Polytechnique

2 Conclusion to Super-Resolution Sparse super-resolution is sometime possible but... don t be fooled. It requires a signal that is sufficiently sparse and a transformed dictionary sufficiently incoherent. Does not work as is in many inverse-problems, such as image interpolation. More structured representations are needed, which incorporate more prior information on the signal.

3 Representation from Linear Sampling Linear sampling: an analog signal is projected on a basis { φ n } n<n of an approximation space V N, which specifies a linear approximation: Uniform sampling: P U f(x) = f, φ n φn n<n f(x) φ n (x) = φ(x Tn) Sparse representation of the discrete signal in a basis {g p } p Γ f[n] = f, φ n R N with the M largest coefficients { f, g p } p Λ with Λ = M.

4 Compressive Sensing Sparse random analog measurements Y [q] =Ū f[q]+w [q] = f,ū q + W [q] where ū q (x) are realizations of a random process. Discretization: if then ū q (x) V N Y [q] =Uf[q]+W [q] = f, u q + W [q] where f[n] = f, φ and u q [n] R N n is a random vector. If f is sparse in {g p } p Γ can we recover f from Y?

5 Compressive Sensing Recovery From measurements with a random operator sparse super-resolution estimation: where decomposition of Y in Y [q] =Uf[q]+W [q] F = ã[p] g p p Λ ã[p] has a support Λ D U = {Ug p } p Γ or with an orthogonal matching pursuit. computed with a sparse by minimizing: 1 2 a[p] Ug p Y 2 + T a[p] p Γ p Γ

6 Restricted Isometry and Incoherence Riesz basis condition for recovery stability (1 δ Λ ) p Γ Restricted isometry condition: (1 δ M ) p Γ any such family a[p] 2 a[p] 2 p Λ a[p] Ug p 2 (1 + δλ ) p Γ {Ug p } p Λ p Λ a[p] Ug p 2 (1 + δm ) p Γ is nearly orthogonal. a[p] 2. δ Λ δ M (D U ) < 1 if Λ M a[p] 2. Relation to incoherence: δ M (D U ) (M 1) µ(d U ) with µ(d U ) = max p q Ug p, Ug q.

7 Exact Recovery Theorem: If f = p Λ a[p] g p with Λ = M and δ 3M < 1/3 1 then a = arg min b 2 b[p] Ug p Y 2 + T b 1 p Γ Sparse signals are exactly recovered.

8 Gaussian Random Matrices We want to have {Ug p } p Γ nearly uniformly distributed over the unit sphere of R Q so that {Ug p } p Λ is as orthogonal as possible even for Λ not small. The distribution of a Gaussian white noise of variance 1 is a uniform measure in the neighborhood of the unit sphere of R Q. If {g p } p Γ is an orthonormal basis of R N and if U is a matrix of Q by N values taken by independant Gaussian random variables (white noise) then {Ug p } p Γ are values taken by Q independant Gaussian random variables.

9 RIP Stability for Gaussian Matrices Theorem: If U is a Gaussian random matrix then for any δ < 1 there exists β > 0 such that δ M (D U ) δ if M β Q log(n/q). Valid for random Bernouilli matrices (random 1 and -1). We need Q CM l on/m) g ( measurements to recover M values and M unknown indices among N. Coding would require of the order of M log(n/m) bits.

10 Perfect Recovery Constants Monte-Carlo experiments for recovering signals with M non-zero coefficients out of N with Q random Gaussian measurements: Q CM l on/m) g ( Ratio of recovered signals with Q = Perfect for Q/M > 6.2 Q/M > 7.7 Q/M = 11 Q/M =

11 Other Random Operators Storing a random Gaussian matrix U and computing Uh requires memory and calculations, too much. O(N 2 ) RIP theorem valid for Bernouilli matrices (random 1 and -1). Still too much memory and computations. Similar RIP theorem valid for a random projector in an orthonormal basis {g m} m which is highly incoherent with the sparsity basis {g p } p. May require only O(N) memory and O(N log N) computations.

12 Measurements Random Sparse Spike Inversion Y = u f + W with f[n] = p Λ a[p] δ[n p]. Random wavelet makes a random sampling of the Fourier coefficients of f : û[k] is the indicator of random set of frequencies. Fourier and Dirac bases have a low coherence.

13 Random Sparse Spike Inversion 0.25 Deterministic wavelet 0.1 Random wavelet Seismic wavelet u[n] Incoherence u ũ[p] Original signal Recovery Recovery

14 Non-Linear Approximation Error sorted with decreasing amplitude { f, g mk } k f, g mk+1 f, g mk. Non-linear approximation in an orthonormal basis: and f M = M f, g mk g mk k=1 f f M 2 = N f, g mk 2. k=m+1 If f, g mk = O(k α ) then f f M 2 = O(M 1 2α ).

15 Theorem: then and if Stability and Recovery Error 1 ã = arg min b 2 Y b[p] Ug p 2 + T b 1 p Γ f p Γ There exists C such that if δ 3M < 1/3 and ã[p] g p 2 C N 1 M k=m f, g mk + C W f, g mk = O(k s ) with s>1 and W =0 f p Γ ã[p] g p 2 = O(M 2s+1 ). Requires Q C M log(n/m) random measurements.

16 Approximation Constants For Q random measurements, M is the number basis coefficients defining an approximation having the same error. The ratio Q/M is computed with a Monte-Carlo experiment for different decay exponents s for N=1024. Q/M Q

17 Recovery Efficiency Compressive sensing efficiency in random Gaussian and Fourier dictionaries. Comparison of Basis Pursuit, Matching Pursuit and Orthogonal Matching Pursuits

18 Image Compressive Sampling Wavelet coefficients of images often have the decay exponent s = 1 of bounded variation images. Q measurements with a linear uniform sampling satisfy Q/M < 5, relatively to a non-linear approximation with M coefficients. Direct compressive is worst with typically Q/M > 7. Improvements by using some prior information on coefficients. Grouping wavelet coefficients scale by scale improves image approximations.

19 Compressive Sensing Applications Robustness: compressive sensing is a democratic acquisition process where all samples are equally important. A missing sample introduce an error that is diluted across the signal. Analog to Digital Converters: for signals that have a sparse Fourier transform, with a random time sampling. For utrawide band signals having a Nyquist rate which is too large. Single pixel camera: random Bernouilli (1 or -1) mesurements of images with a single pixel at very high sampling rate. Medical Resonance Imaging: randomize as much as possible the Fourier sampling of images obtained with MRI, and use their wavelet sparsity to improve their resolution.

20 Conclusion to Compressive Sensing Sparse super-resolution becomes stable with randomized measurements. Large potential applications. Asymptotic performance equivalent to a non-linear approximation with a known signal. The devil is in the constants, compared to a linear uniform sampling. Technological difficulties for the signal recovery: large amount of memory and computations are required.