Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information

Transcription

1 1 Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information Emmanuel Candès, California Institute of Technology International Conference on Computational Harmonic Analysis Nashville, Tennessee, May 2004 Collaborators: Justin Romberg (Caltech), Terence Tao (UCLA)

2 2 Incomplete Fourier Information Observe Fourier samples ˆf(ω) on a domain Ω. 22 radial lines, 8% coverage

3 Classical Reconstruction Backprojection: essentially reconstruct g with ˆf(ω) ĝ ω Ω (ω) = 0 ω Ω 3 Original Phantom (Logan Shepp) Naive Reconstruction original g

4 4 Interpolation? 25 A Row of the Fourier Matrix original g

5 Total Variation Reconstruction 5 Reconstruct g with min g g T V s.t. ĝ(ω) = ˆf(ω), ω Ω Original Phantom (Logan Shepp) Reconstruction: min BV + nonnegativity constraint original g = original perfect reconstruction!

6 6 Sparse Spike Train Sparse sequence of N T spikes Observe N Ω Fourier coefficients

7 Interpolation? 7

8 l 1 Reconstruction 8 Reconstruct by solving min g t g s.t. ĝ(ω) = ˆf(ω), ω Ω For N T N Ω /2, we recover f perfectly. t original recovered from 30 Fourier samples

9 Extension to TV 9 g T V = i g t+1 g t = l 1 -norm of finite differences Given frequency observations on Ω, using min g T V s.t. ĝ(ω) = ˆf(ω), ω Ω we can perfectly reconstruct signals with a small number of jumps.

10 Reconstructed perfectly from 30 Fourier samples 10

11 11 Model Problem Signal made out of T spikes Observed at only Ω frequency locations Extensions Piecewise constant signal Spikes in higher-dimensions; 2D, 3D, etc. Piecewise constant images Many others

12 12 Sharp Uncertainty Principles Signal is sparse in time, only T spikes Solve combinatorial optimization problem (P 0 ) min g g l0 := #{t, g(t) 0}, ĝ Ω = ˆf Ω Theorem 1 N (sample size) is prime (i) Assume that T Ω /2, then (P 0 ) reconstructs exactly. (ii) Assume that T > Ω /2, then (P 0 ) fails at exactly reconstructing f; f 1, f 2 with f 1 l0 + f 2 l0 = Ω + 1 and ˆf 1 (ω) = ˆf 2 (ω), ω Ω

13 13 l 1 Relaxation? Solve convex optimization problem (LP for real-valued signals) (P 1 ) min g g l1 := t g(t), ĝ Ω = ˆf Ω Example: Dirac s comb N equispaced spikes (N perfect square). Invariant through Fourier transform ˆf = f Can find Ω = N N with ˆf(ω) = 0, ω Ω. Can t reconstruct More dramatic examples exist But all these examples are very special

14 14 Dirac s Comb f(t) N f(ω) N t ω f ˆf

15 15 Main Result Theorem 2 Suppose T α(m) Ω log N Then min-l 1 reconstructs exactly with prob. greater than 1 O(N M ). (n.b. one can choose α(m) [29.6(M + 1)] 1. Extensions T, number of jump discontinuities (TV reconstruction) T, number of 2D, 3D spikes. T, number of 2D jump discontinuities (2D TV reconstruction)

16 16 Heuristics: Robust Uncertainty Principles f unique minimizer of (P 1 ) iff f(t) + h(t) > t t f(t), h, ĥ Ω = 0 Triangle inequality f(t) + h(t) = Sufficient condition T f(t) + h(t) + T c h t T f(t) h(t) + T c h t h(t) h(t) T c T h(t) 1 2 h l 1 T Conclusion: f unique minimizer if for all h, s.t. ĥ Ω = 0, it is impossible to concentrate h on T

17 17 Connections Donoho & Stark (88) Donoho & Huo (01) Gribonval & Nielsen (03) Tropp (03) and (04) Donoho & Elad (03) Santosa & Symes (86) Dobson & Santosa (96) Bressler (98?) Vetterli et. al. (03) Gilbert et al. (04)

18 18 Dual Viewpoint Convex problem has a dual Dual polynomial P (t) = ω Ω ˆP (ω)e iωt P (t) = sgn(f)(t), t T P (t) < 1, t T c ˆP supported on set Ω of visible frequencies Theorem 3 (i) If F T Ω and there exits a dual polynomial, then the (P 1 ) minimizer (P 1 ) is unique and is equal to f. (ii) Conversely, if f is the unique minimizer of (P 1 ), then there exists a dual polynomial.

19 19 Dual Polynomial P(t) ^ P(ω) t ω Space Frequency

20 20 Construction of the Dual Polynomial P (t) = ω Ω ˆP (ω)e iωt P interpolates sgn(f) on T P has minimum energy

21 Auxilary matrices: 1 Ω H = I F 1 P Ω FR T Hf(t) := ω Ω t E:t t e iω(t t ) f(t ), 21 Restriction: R T is the restriction map, R T f := f T R T is the obvious embedding obtained by extending by zero outside of T Identity: R T R T = I T. P := (R T 1 Ω H)(I T 1 Ω R T H) 1 R T sgn(f). Frequency support. P has Fourier transform supported in Ω Spatial interpolation. P obeys R T P = (R T R T 1 Ω R T H)(R T R T 1 Ω R T H) 1 R T sgn(f) = R T sgn(f), and so P agrees with sgn(f) on T.

22 22 Hard Things P := (R T 1 Ω H)(I T 1 Ω R T H) 1 R T sgn(f). (I T 1 Ω R T H) invertible P (t) < 1, t / T Interpretation I T 1 Ω R T H = [F T Ω ] F T Ω i.e. invertibility means that F T Ω = R Ω FR T is of full rank.

23 23 Invertibility (I T 1 Ω R T H) = I T 1 Ω H 0, H 0 (t, t ) = 0 t = t ω Ω eiω(t t ). t t Fact: H 0 (t, t ) Ω H 0 2 Tr(H 0 H 0) = t,t H 0 (t, t ) 2 T 2 Ω Want H 0 Ω, and therefore T 2 Ω = O( Ω 2 ) T = O( Ω )

24 24 Key Estimates Want to show largest eigenvalue of H 0 (self-adjoint) is less than Ω. Take large powers of random matrices Tr(H 2n 0 ) = λ2n λ2n T Key estimate: develop bounds on E[Tr(H 2n 0 )] γ2n n n T n+1 Ω n. Key intermediate result: with large-probability A lot of combinatorics! H 0 γ log T T Ω

25 Numerical Results 25 Signal length N = 1024 Randomly place N t spikes, observe N w random frequencies Measure % recovered perfectly red = always recovered, blue = never recovered N w N t /N w

26 26 Other Phantoms, I Original Phantom Classical Reconstruction original g = classical reconstruction

27 27 Original Phantom Total Variation Reconstruction original g = TV reconstruction = Exact!

28 28 Other Phantoms, II Original Phantom Classical Reconstruction original g = classical reconstruction

29 29 Original Phantom Total Variation Reconstruction original g = TV reconstruction = Exact!

30 30 Scanlines 2 A Scanline of the Original Phantom 2 Classical (Black) and TV (Red) Reconstructions original g = classical reconstruction

31 31 Robust Uncertainty Principles T = supp(f) Ω = supp( ˆf) Discrete uncertainty principle (Donoho & Stark 88) T + Ω 2 N Robust uncertainty principle (C. & Romberg 04): for nearly all T, Ω T + Ω β N/ log N

32 32 Summary Exact reconstruction Tied to new uncertainty principles Stability Robustness Optimality Many extensions: e.g. arbitrary synthesis/measurement pairs Contact:

33 33 Extentions Two bases Φ and Ψ f has a sparse decomposition in Φ, f = t α tφ t N T terms in the expansion Only able to observe measurements f, ψ ω on small (random) set Ω. N Ω measurements Incoherence: M = N sup t,ω φ t, ψ ω. Then perfect reconstruction occurs provided N T N Ω M 2 log N Tight Twist: both T AND Ω need to be randomized.