Breaking the coherence barrier - A new theory for compressed sensing

Transcription

1 Breaking the coherence barrier - A new theory for compressed sensing Anders C. Hansen (Cambridge) Joint work with: B. Adcock (Simon Fraser University) C. Poon (Cambridge) B. Roman (Cambridge) UCL-Duke Workshop, September 4, / 42

2 Compressed Sensing in Inverse Problems Typical analog/infinite-dimensional inverse problem where compressed sensing is/can be used: (i) Magnetic Resonance Imaging (MRI) (ii) X-ray Computed Tomography (iii) Thermoacoustic and Photoacoustic Tomography (iv) Single Photon Emission Computerized Tomography (v) Electrical Impedance Tomography (vi) Electron Microscopy (vii) Reflection seismology (viii) Radio interferometry (ix) Fluorescence Microscopy 2 / 42

3 Compressed Sensing in Inverse Problems Most of these problems are modelled by the Fourier transform Ff (ω) = f (x)e 2πiω x dx, R d or the Radon transform Rf : S R C (where S denotes the circle) Rf (θ, p) = f (x) dm(x), x,θ =p where dm denotes Lebesgue measure on the hyperplane {x : x, θ = p}. Fourier slice theorem both problems can be viewed as the problem of reconstructing f from pointwise samples of its Fourier transform. g = Ff, f L 2 (R d ). (1) 3 / 42

4 Compressed Sensing Given the linear system If Solve Ux 0 = y. min z 1 subject to P Ω Uz = P Ω y, where P Ω is a projection and Ω {1,..., N} is subsampled with Ω = m. m C N µ(u) s log(ɛ 1 ) log (N). then P(z = x 0 ) 1 ɛ, where µ(u) = max U i,j 2 i,j is referred to as the incoherence parameter. 4 / 42

5 Pillars of Compressed Sensing Sparsity Incoherence Uniform Random Subsampling In addition: The Restricted Isometry Property + uniform recovery. Problem: These concepts are absent in virtually all the problems listed above. Moreover, uniform random subsampling gives highly suboptimal results. Compressed sensing is currently used with great success in many of these fields, however the current theory does not cover this. 5 / 42

6 Uniform Random Subsampling 1 U = Udft Vdwt. 5% subsamp-map Reconstruction Enlarged 6 / 42

7 Sparsity The classical idea of sparsity in compressed sensing is that there are s important coefficients in the vector x 0 that we want to recover. The location of these coefficients is arbitrary. 7 / 42

8 Sparsity and the Flip Test Let x = and y = U df x, A = P Ω U df V 1 dw, where P Ω is a projection and Ω {1,..., N} is subsampled with Ω = m. Solve min z 1 subject to Az = P Ω y. 8 / 42

9 Sparsity - The Flip Test 2 Truncated (max = ) Figure : x10 Wavelet coefficients and subsampling reconstructions from 10% of Fourier coefficients with distributions (1 + ω12 + ω22 ) 1 and (1 + ω12 + ω22 ) 3/2. If sparsity is the right model we should be able to flip the coefficients. Let 2 Truncated (max = ) 1.5 zf = x10 9 / 42

10 Sparsity - The Flip Test Let Solve to get ẑ f. min z 1 ỹ = U df V 1 dw z f subject to Az = P Ω ỹ Flip the coefficients of ẑ f back to get ẑ, and let ˆx = V 1 dw ẑ. If the ordering of the wavelet coefficients did not matter i.e. sparsity is the right model, then ˆx should be close to x. 10 / 42

11 Sparsity- The Flip Test: Results Figure : The reconstructions from the reversed coefficients. Conclusion: The ordering of the coefficients did matter. Moreover, this phenomenon happens with all wavelets, curvelets, contourlets and shearlets and any reasonable subsampling scheme. Question: Is sparsity really the right model? 11 / 42

12 Sparsity - The Flip Test 512, 20% U Had V 1 dwt Fluorescence Microscopy CS reconstr. CS reconstr, w/ flip Subsampling coeffs. pattern 1024, 12% U Had V 1 dwt Compressive Imaging, Hadamard Spectroscopy 12 / 42

13 Sparsity - The Flip Test (contd.) CS reconstr. CS reconstr, w/ flip coeffs. Subsampling pattern 1024, 20% 1 Udft Vdwt Magnetic Resonance Imaging 512, 12% 1 Udft Vdwt Tomography, Electron Microscopy 13 / 42

14 Sparsity - The Flip Test (contd.) 1024, 10% U dft V 1 dwt Radio interferometry CS reconstr. CS reconstr, w/ flip Subsampling coeffs. pattern 14 / 42

15 Lesson Learned! The optimal sampling strategy depends on the sparsity structure of the signal. Compressed sensing theorems must therefore show how the optimal sampling strategy will depend on the structure. Such theorems currently do not exists. 15 / 42

16 What about the RIP? Did any of the matrices used in the examples satisfy the RIP? 16 / 42

17 Images are not sparse, they are asymptotically sparse How to measure asymptotic sparsity: Suppose f = β j ϕ j. j=1 Let N = k N{M k 1 + 1,..., M k }, where 0 = M 0 < M 1 < M 2 <... and {M k 1 + 1,..., M k } is the set of indices corresponding to the k th scale. Let ɛ (0, 1] and let { s k := s k (ɛ) = min K : K β π(i) ϕ π(i) ɛ i=1 M k i=m k 1 +1 β j ϕ j }, in order words, s k is the effective sparsity at the k th scale. Here π : {1,..., M k M k 1 } {M k 1 + 1,..., M k } is a bijection such that β π(i) β π(i+1). 17 / 42

18 Images are not sparse, they are asymptotically sparse Sparsity, sk(ǫ)/(mk Mk 1) Level 1 Level 2 Level 3 Level 4 Level 5 Level 6 Level 7 Level 8 Worst sparsity Best sparsity Sparsity, sk(ǫ)/(mk Mk 1) Level 1 Level 2 Level 3 Level 4 Level 5 Level 6 Level 7 Level 8 Worst sparsity Best sparsity Relative threshold, ǫ Relative threshold, ǫ Figure : Relative sparsity of Daubechies 8 wavelet coefficients. 18 / 42

19 Analog inverse problems are coherent Let U n = U df V 1 dw Cn n where U df is the discrete Fourier transform and V dw is the discrete wavelet transform. Then µ(u n ) = 1 for all n and all Daubechies wavelets! 19 / 42

20 Analog inverse problems are coherent, why? Note that where Thus, we will always have WOT-lim n U dfv 1 dw = U, ϕ 1, ψ 1 ϕ 2, ψ 1 U = ϕ 1, ψ 2 ϕ 2, ψ µ(u df V 1 dw ) c. 20 / 42

21 Analog inverse problems are asymptotically incoherent Fourier to DB4 Fourier to Legendre Polynomials Figure : Plots of the absolute values of the entries of the matrix U 21 / 42

22 We need a new theory Such theory must incorporates asymptotic sparsity and asymptotic incoherence. It must explain the two intriguing phenomena observed in practice: The optimal sampling strategy is signal structure dependent The success of compressed sensing is resolution dependent The theory cannot be RIP based (at least not with the classical definition of the RIP) 22 / 42

23 New Pillars of Compressed Sensing Asymptotic Sparsity Asymptotic Incoherence Multi-level Subsampling 23 / 42

24 Sparsity in levels Definition For r N let M = (M 1,..., M r ) N r with 1 M 1 <... < M r and s = (s 1,..., s r ) N r, with s k M k M k 1, k = 1,..., r, where M 0 = 0. We say that β l 2 (N) is (s, M)-sparse if, for each k = 1,..., r, k := supp(β) {M k 1 + 1,..., M k }, satisfies k s k. We denote the set of (s, M)-sparse vectors by Σ s,m. 24 / 42

25 Sparsity in levels Definition Let f = j N β jϕ j H, where β = (β j ) j N l 1 (N). Let σ s,m (f ) := min β η l 1. (2) η Σ s,m 25 / 42

26 Multi-level sampling scheme Definition Let r N, N = (N 1,..., N r ) N r with 1 N 1 <... < N r, m = (m 1,..., m r ) N r, with m k N k N k 1, k = 1,..., r, and suppose that Ω k {N k 1 + 1,..., N k }, Ω k = m k, k = 1,..., r, are chosen uniformly at random, where N 0 = 0. We refer to the set Ω = Ω N,m := Ω 1... Ω r. as an (N, m)-multilevel sampling scheme. 26 / 42

27 Local coherence Definition Let U C N N. If N = (N 1,..., N r ) N r and M = (M 1,..., M r ) N r with 1 N 1 <... N r and 1 M 1 <... < M r we define the (k, l) th local coherence of U with respect to N and M by µ N,M (k, l) = where N 0 = M 0 = 0. µ(p N k 1 N k UP M l 1 M l ) µ(p N k 1 N k U), k, l = 1,..., r, 27 / 42

28 The optimization problem inf η l 1 subject to P Ω Uη y δ. (3) η l 1 (N) 28 / 42

29 Theoretical Results Let U C N N be an isometry and β C N. Suppose that Ω = Ω N,m is a multilevel sampling scheme, where N = (N 1,..., N r ) N r and m = (m 1,..., m r ) N r. Let (s, M), where M = (M 1,..., M r ) N r, M 1 <... < M r, and s = (s 1,..., s r ) N r, be any pair such that the following holds: for ɛ > 0 and 1 k r, ( r ) 1 N k N k 1 log(ɛ 1 ) µ N,M (k, l) s l log (N). (4) m k Suppose that ξ C N is a minimizer of (3) with δ = δ K 1 and K = max 1 k r {(N k N k 1 )/m k }. Then, with probability exceeding 1 sɛ, where s = s s r, we have that ξ β C ( δ (1 + L s ) ) + σ s,m (f ), for some constant C, where σ s,m (f ) is as in (2), L = 1 + K = max k=1,...,r { Nk N k 1 m k }. l=1 log 2(6ɛ 1 ) log 2 (4KM and s) 29 / 42

30 Fourier to wavelets Number of samples in each level: m k log(ɛ 1 ) log(n) Nk N k 1 N k 1 ( k 2 ŝ k + s l 2 α(k l) + l=1 where ŝ k = max{s k 1, s k, s k+1 }. Note that r l=k+2 s l 2 v(l k) ) m s log(n), m = m m r, s = s s r. Note that this shows how the success of CS will be resolution dependent., 30 / 42

31 r-level Sampling Scheme Figure : The typical sampling pattern that will be used. 31 / 42

32 Resolution Dependence, 5% subsampling Size: , Error = 10.8% Original CS reconstruction Subsamp. map 32 / 42

33 full sampling and 5% subsampling (DB4) MRI Data courtesy of Andy Ellison, Boston University. Numerics taken from: On asymptotic structure in compressed sensing, B. Roman, B. Adcock, A. C. Hansen, arxiv: / 42

34 The GLPU-Phantom The Guerquin-Kern,Lejeune, Pruessmann, Unser-Phantom (ETH and EPFL) 34 / 42

35 Seeing further with compressed sensing Figure : The figure shows full sampling (= samples) with zero padding. Numerics taken from: On asymptotic structure in compressed sensing, B. Roman, B. Adcock, A. C. Hansen, arxiv: / 42

36 Seeing further with compressed sensing Figure : The figure shows 6.25% subsampling from (= samples) and DB4. Numerics taken from: On asymptotic structure in compressed sensing, B. Roman, B. Adcock, A. C. Hansen, arxiv: / 42

37 Key Question Question: Will compressed sensing ever be used in commercial MRI scanners? 37 / 42

38 Siemens Siemens has implemented our experiments and verified our theory experimentally on their scanners. See the Siemens report: Novel Sampling Strategies for Sparse MR Image Reconstruction, in Proceedings of the International Society for Magnetic Resonance in Medicine. 38 / 42

39 Siemens Conclusion: Significant differences in the spatial resolution can be observed. The image resolution has been greatly improved. Current results practically demonstrated that it is possible to break the coherence barrier by increasing the spatial resolution in MR acquisitions. This likewise implies that the full potential of the compressed sensing is unleashed only if asymptotic sparsity and asymptotic incoherence is achieved. Therefore, compressed sensing might better be used to increase the spatial resolution rather than accelerating the data acquisition in the context of non-dynamic 3D MR imaging. 39 / 42

40 Key Question Question: Will Compressed sensing ever be used in commercial MRI scanners? Answer: We are closer than ever. 40 / 42

41 Other work Structured sampling: Calderbank, Carin, Carson, Chen, Rodrigues (2012) Structured sparsity: Tsaig & Donoho (2006), Eldar (2009), He & Carin (2009), Baraniuk et al. (2010), Krzakala et al. (2011), Duarte & Eldar (2011), Som & Schniter (2012), Renna et al. (2013), Chen et al. (2013), Donoho & Montanari (2013) + others Existing algorithms: Model-based CS: Baranuik, Cevher, Duarte, Hegde (2010) Bayesian CS: Ji, Xue & Carin (2008), He & Carin (2009) Turbo AMP: Som & Schniter (2012) Variable density sampling: Krahmer & Ward (2013), Kutyniok & Lim (2014) 41 / 42

42 Related Papers For details see [4, 5, 2, 1, 3] B. Adcock and A. C. Hansen. Generalized sampling and infinite-dimensional compressed sensing. Found. Comp. Math., (in revision). B. Adcock, A. C. Hansen, C. Poon, and B. Roman. Breaking the coherence barrier: A new theory for compressed sensing. arxiv: , A. C. Hansen. On the solvability complexity index, the n-pseudospectrum and approximations of spectra of operators. J. Amer. Math. Soc., 24(1):81 124, B. Roman, B. Adcock, and A. C. Hansen. On asymptotic structure in compressed sensing. arxiv: , Q. Wang, M. Zenge, H. E. Cetingul, E. Mueller, and M. S. Nadar. Novel sampling strategies for sparse mr image reconstruction. Proc. Int. Soc. Mag. Res. in Med., (22), / 42