Compressed Sensing - Near Optimal Recovery of Signals from Highly Incomplete Measurements

Transcription

1 Compressed Sensing - Near Optimal Recovery of Signals from Highly Incomplete Measurements Wolfgang Dahmen Institut für Geometrie und Praktische Mathematik RWTH Aachen and IMI, University of Columbia, SC Joint work with Albert Cohen and Ron DeVore W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

2 Outline 1 Some Background/Motivation Why CS in TEM, STEM,...? Shannon Paradigm Sparsity - Image Compression 2 A New Paradigm The Objective 3 What does it have to offer? How to Measure Performance? The Maximal Sparsity Range Good Sensing Matrices Dependence on Norms The Role of Randomness Instance Optimality in Probability Instance Optimal Decoders in Probability 4 Questions/Diskussion W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

3 Why CS in TEM, STEM,...? Key Issue: Complex but information sparse signals sample them at possibly low rate while recovering essential information Are those sparse? W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

4 Why CS in TEM, STEM,...? Or are those sparse?... Symyx Technologies Inc. Isolation of M1 using Combinatorial Screening [001] SAED Pattern of Phase M1 a* b* h00; h 2n 0k0; k 2n a-glide b and b- glide a or 2 a 21.2 Å 1 screw b 26.6 Å along both a and b axes DeSanto et al., Topics in Catal. 23 (1-4), 23 (2003); Z. Krist 219, (2004). W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

5 Shannon Paradigm Classical model: bandlimited signals recovery requires sampling at twice the Nyquist rate...too much in many situations... But: what about such signals? how to exploit structure? sparse information content W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

6 Image Compression left: Original Pixel 8 Bit grey depth 384 KB with naiv storage right: compressed version ca. 3.5 % of origonal storage W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

7 Images Are just Functions Image histogram function f (x) = p χ (x) W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

8 The Haar-Wavelet a b = + Feinstruktur = Mittelung + Detail averaging profile φ(x) oscillation profile ψ(x) W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

9 Change of Bases: Fast Wavelet-Transform 2 J 1 X k=0 p J,k φ J,k (x) = 2 J 1 1 X k=0 p J 1,k φ J 1,k (x) 2 J 1 X 1 + d J 1,k ψ J 1,k (x) k=0 p J 1,k = 1 2 (p J,2k + p J,2k+1 ), d J 1,k = 1 2 (p J,2k p J,2k+1 ) T : p J d J p J p J 1 p J 2 p 1 p 0 p = T 1 d d J 1 d J 2 d 1 d 0 d l2 = f L2 W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

10 Back to the Images W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

11 Wavelet-Decomposition W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

12 The Principle Sample at highest level of resolution Transform image Quantize - retain 3%, say Transmit compressed data What if measurements are costly or... cause damage? W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

13 Could it Suffice to Sample at the Rate of Information Content? (Candes/Romberg/Tao) W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

14 A Principal Effect y l = 1 2π 2π 0 f (t)e ilt dt 1 N 1 f (2πj/N) e N }{{}} il2πj/n {{} j=0 =:x j =:φ l,j = (Φx) l Exact reconstruction: f = argmin { g 1 : ĝ(l) = y l, l k} W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

15 Sampling, Sparsity, etc. - The Setting Assumption: signal x R N is (quasi) sparse in some basis x = Mz, #suppz = k N to recover x allow for more general notions of measurements x y i = φ i (x) = φ i x, i = 1,..., n N try to keep n k N for which N, n, k is this possible while retaining the essential information of x? W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

16 The Model x R N, y i = φ i x + e i, i = 1,..., n, y = Φx + e N y e = n Φ T k k = card T x W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

17 Some Guiding Questions How to build efficient decoders? : y x! x What are good sensing matrices Φ? y e = n T N Φ k When x = Mz, z sparse, when is Φ = ΦM good too? k = card T x How to measure performance of CS? What got things started:... x = x for k-sparse x, k < n Candés/Romberg/Tao, Donoho, Gilbert/Strauss/Tropp, Tanner, Rice group, Needell/Vershynin,... W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

18 How to Measure Performance? Exact recovery of k-sparse signals: Σ k := {z R N : #supp (z) k} y = Φx, x Σ k (y) = x But real signals are rarely truly sparse! Compare x (y) lp with σ k (x) lp := inf z Σ k x z lp Instance Optimality: Best k-term approximation as a Benchmark Given l p, N, n, how large can we have k s.t. (Φ, ) with x (Φx) lp C 0 σ k (x) lp, x R N (IO(l p, k)) (1) Note: implies exact recovery of k-sparse signals! W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

19 The Maximal Sparsity Range THEOREM: Consider K := U(l N 1 ) = {x RN : x l1 1} inf (Φ, ) A n,n sup x (Φx) l2 C 0 max σ k(x) l2 x K x K k c 0 n/ log(n/n) Maximal sparsity range k c 0 n/ log(n/n) Gelfand widths: (Kashin, Gluskin/Garnaev) d n (K ) X := inf sup{ x X ; x K Y } Y :codim Y n Y Y K W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

20 Good Sensing Matrices - l 1 -Minimization Restricted Isometry Property - RIP(k, δ) (1 δ) x l2 Φx l2 (1 + δ) x l2, x Σ k THEOREM [CT]: l 1 -minimization RIP(3k, δ) implies (y) := argmin Φz=y z l1 σ = x (Φx) l2 < k (x) l1 k Questions: This is not quite instance optimality in l 2 Given n, N, for how large k can RIP(3k, δ) hold? W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

21 A Geometric Explanation - n = 1, N = 2 {x : Φx = y} W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

22 A Geometric Explanation - n = 1, N = 2 {x : Φx = y} W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

23 Caution: The Choice of the Norm Matters THEOREM: The case X = l N 1 : RIP(3k, δ), δ < δ 0 implies that l 1 -minimization is instance optimal of order k in l 1. THEOREM: The case X = l N 2 : No matter how Φ and are chosen (Φ, ) is IO(l 2, 1) = n an. IO(l 2, k) does not seem to be a viable concept However... W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

24 Which Matrices satisfy RIP for large k? Random Matrices All constructions for the optimal range k < n/ log(n/n) involve random matrices 1 Gaussian matrices: Φ i,j = N (0, 1 n ) are i.i.d. Gaussian variables of variance 1/n 2 Bernoulli matrices: Φ i,j = ±1 n are i.i.d. Bernoulli variables 3 Uniformly distributed vectors on S n 1 relaxed concept: Instance Optimality in Probability (IOP) W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

25 Concentration of Measure Property Φ = Φ(ω): n N matrices on probability space (Ω, ρ) CoMP : For any x R N, δ (0, 1], ρ {ω : Φ(ω)x 2 l2 x 2 } l2 δ x 2 l2 1 be cnδ2 Note: M orthogonal {Φ(ω)M} also satisfy CoMP THEOREM: CoMP implies RIP(k, δ) for any δ (0, 1] with high probability when k c 0 n/ log(n/n) THEOREM: CoMP holds for sub-gaussian distributions as well as uniform vectors on the unit sphere W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

26 Instance Optimal Decoders in Probability The following are instance optimal in l 2 in probability for CoMP families of random matrices: l 1 -minimization (DeVore/Petrova/Wojtaszczyk) Greedy-thresholding algorithms (Cohen/D./DeVore, Needell/Tropp) W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38

27 What to do with it? Some questions: CS without denoising seems to be infeasible Does it pay to work in reciprocal space? sparse phase retrieval? Are defects sparse? Are there other data dependent dictionaries for which TEM/STEM images are sparse signals? is random beam placement an option? W. Dahmen (RWTH Aachen) Compressed Sensing Feb. 22, / 38