Determinis)c Compressed Sensing for Images using Chirps and Reed- Muller Sequences

Transcription

1 Determinis)c Compressed Sensing for Images using Chirps and Reed- Muller Sequences Kangyu Ni Mathematics Arizona State University Joint with Somantika Datta*, Prasun Mahanti, Svetlana Roudenko, Douglas Cochran Arizona State University, Princeton University*

2 Outline Introduc)on Determinis)c CS using chirps and Reed- Muller sequences Mo)va)on Need a method suitable for images Method Construc)on of the CS matrix Reconstruc)on algorithm Results Best ini)al approxima)on Fast chirp transforms

3 Compressed Sensing*: overview Φ x = y n N n 1 Φ sensing matrix y data/measurements x sparse signal want to recover! N 1 1. x : k-sparse k < n << N 2. Φ : RIP (Restricted Isometry Property), e.g. random matrices 3. Prac)cal reconstruc)on algorithms, e.g. l1 minimiza)on *Candes, Romberg, Tao 2006 and Donoho 2006

4 Mo)va)on Random matrices (Gaussian, ) sa)sfy RIP high probability of successful recovery Why determinis)c sensing Explicit reconstruc)on algorithm Efficient storage Smaller error in reconstruc)on Exis)ng works Chirp matrices Applebaum, Howard, Searle, Calderbank 2 nd - order RM sequences Howard, Calderbank, Searle, Jafarpour DeVore, Indyk, Iwen, Herman, Need a method suitable for images

5 Sta)s)cal Restricted Isometry Property* Φ is (k, ε, δ) -StRIP if for k-sparse x R N (1 ε) x 2 2 Φx (1+ε) x 2 holds with prob. exceeding 1- δ (w.r.t. a uniform distribution of x among all k-sparse vectors) *Calderbank, Howard, Jafarpour, Construction of a Large Class of Deterministic Sensing Matrices that Satisfy a Statistical Isometry Property

6 CS with Chirps and RM Sequences discrete chirp signal 2 nd - order Reed- Muller func)ons υ r,m (l) = 1 n e 2πi n ml + 2πi n rl 2 φ P,b (a) = 1 T 2 m i2b a + ( Pa) T a r, m, l Ζ n a, b Ζ 2 m (binary vectors of length m) P : m m binary symmetric matrix complex sinusoids Walsh func)ons υ 0,m (l) = 1 n e 2πi n ml φ 0,b (a) = 1 2 m i2b T a

7 CS with Chirps and RM Sequences inverse Fourier matrix n=3! e 2πi e 2πi 1 e 2πi e 2πi Fast Fourier Transform *chirp matrix * Applebaum et al Φ = [U r1 U r2 U rn ] n X n 2 Hadamard matrix m=2! Fast Hadamard Transform (FHT) **Reed- Muller matrix (P is zero- diagonal) ** Howard et al Φ = [U P1 U P2 U P2 m( m 1) /2 ] 2 m X 2 m (m+1)/2

8 Chirp Sensing Matrix Φ = [U r1 U r2 U rn ] n X n 2 n = 3 size: 3 X 3 2 Φ = [U r1 U r2 U r3 ] = m = 0, 1, 2 l = 0, 1, l m e 2πi e 2πi e 2πi 1 e 2πi e 2πi e 2πi r 1 = 0 l m e 2πi e 2πi 3 2πi e 2πi 2πi πi e 2πi 2πi r 2 = 1 l e 2πi e 2πi m e 2πi 3 2πi e 2πi 2πi e 2πi 2πi e 2πi 2πi r 3 = 2

9 RM Sensing Matrix Φ = [U P1 U P2 U P2 m( m 1) /2 ] 2 m X 2 m (m+1)/2 m = 2 size: 2 2 X 2 3 Φ = [U P1 U P2 ] = 1 2 m = X 2 55 a a, b Ζ 2 2 = 0 0 b ,,, a b P 1 P 2 P 1 = 0 0, P =

10 Quadra)c Reconstruc)on Algorithm Chirp version* Φ x = y y(l) = z 1 v r1,m 1 (l) + + z k v rk,m k (l) O(kn 2 log n) 1. Find r i -- shift-and-multiply & FFT, find peak 2. Find m i -- dechirp & FFT, find peak FFT{ y(l)v rj,0 (l)} FFT{ y(l) y(l + T) } 3. Find z i -- least squares min z y(l) z j v rj,m j (l) 2 4. Repeat until residual < epsilon * Applebaum et al.

11 Quadra)c Reconstruc)on Algorithm RM version* Φ x = y y(a) = z 1 φ P1,b 1 (a) + + z k φ Pk,b k (a) 1. Find P i -- shift-and-multiply & FHT, find peak 2. Find b i -- dechirp & FHT, find peak Fast Hadamard Transform (FHT) (projection onto Walsh basis) FHT{ y(a)φ Pj,0 (a)} FHT{ y(a) y(a + e j )} O( kn(logn) 2 ) 3. Find z i -- least squares min z y(a) z j φ Pj,b j (a) 2 4. Repeat until residual < epsilon * Howard et al.

12 Chirp Results comparison with MP + random matrices n=67 Reconstruc)on of sparse signals Reconstruc)on of sparse signals with noise Sparsity k Matching Pursuit - - O(knN) Deterministic CS with Chirp alg.- - O(kn 2 log n) * Applebaum et al.

13 RM Results **Howard et al. Rule of thumb*: n > k log 2 (1+ N /k) ;;;(high prob. of successful recon. using l1 min. + random matrix) m = X 2 55 n =1,024 and N =3.6 X ;;by rule of thumb k < 20 *Baron et al., Donoho and Tanner

14 Outline Introduc)on Determinis)c CS using chirps and Reed- Muller sequences Mo)va)on Need a method suitable for images Method Construc)on of the CS matrix Reconstruc)on algorithm Results Best ini)al approxima)on Fast chirp transforms

15 Chirp and RM Quadra)c Reconstruc)on Algorithms k = sparsity, n = #measurements N = signal size Pros: Outperform MP in recon. error and computational complexity MP O(knN) det CS with RM pros & cons O( kn(logn) 2 ) Cons: Not suited for images image with 10% sparsity k = 6,554, N = 65,536 by rule of thumb, n > k log 2 (1+ N /k) n 22,670 but n N = 2 m X 2 m (m+1)/2 least squares problem becomes too large N n = image not sparse enough

16 Outline Introduc)on Determinis)c CS using chirps and Reed- Muller sequences Mo)va)on Need a method suitable for images Method Construc)on of the CS matrix Reconstruc)on algorithm Results Best ini)al approxima)on Fast chirp transforms

17 chirp Construc)on of Sensing Matrices N = image size (expl: 256 X 256 = 2 16 ) n = prime greater than and closest to N/4 (expl: 16,411) Sensing matrix: Φ = [U r1 U r2 U r3 U r4 ] coherence = 1 n RM N = image size (expl: 256 X 256 = 2 16 ) n = N/4 (expl: 2 14 = 16,384) Sensing matrix: Φ = [U P1 U P2 U P3 U P4 ] = max ϕ j,ϕ l j l

18 Reconstruc)on Algorithm for Images Get initial best approximation solution Repeat 1-3 until residual is sufficiently small 1 Find multiple (P i, b i ) pairs 2 Determine z i by least squares solutions 3 Get residual y(a) = y(a) z l φ Pl,b l (a) j l =1

19 Ini)al Best Approxima)on of Solu)on Detection of the bulk of a signal Based on energy of wavelets concentrate on upper-left region y = Φx = [U P1 U P2 U P3 U P4 ] x 1 = U P1 x 1 +U P2 x 2 +U P3 x 3 +U P4 x 4 x 2 x 3 x 4 U * P1 y = U * P1 U P1 x 1 +U * P1 U P2 x 2 +U * P1 U P3 x 3 +U * P1 U P4 x 4 x 1

20 Example: Ini)al Best Approxima)on X 256 Shepp- Logan phantom image x x 1 in ascending order wavelet coefficients U P1 T y U P1 T y in ascending order No prior knowledge of individual image required

21 Find Mul)ple (P i, b i ) Pairs Discrete Chirp-Hadamard Transform (DCHT) : find (P i, b i ) pairs DCHT = de-chrip + FHT Only 4 P-matrices used FHT (( i) (P j a )T a y(a) ), j =1, 2, 3, 4 Pick d 100 peaks in the FHT Computational complexity improves from O(kn(logn) 2 ) to O( 1 d 4kn logn)

22 Determine z i by Least Squares Φ x = y Φ sub z = y n N n k min x Φ sub z y 2 Φ = [U P1 U P2 U P3 U P4 ] solved by LSQR [Paige & Saunders] Each U Pj is 0 n n 0 Hadamard matrix n n

23 Outline Introduc)on Determinis)c CS using chirps and Reed- Muller sequences Mo)va)on Need a method suitable for images Method Construc)on of the CS matrix Reconstruc)on algorithm Results Best ini)al approxima)on Fast chirp transforms

24 Reconstruc)on SNR x SNR(dB) = 10 log actual 2 10 x actual x reconstructed 2

25 256 X 256 images cameraman knee. original original 14% Haar wavelets 10% Haar wavelets

26 Reconstruc)on SNR image sparsity k n/k chirp RM cameraman 14% db 44 db knee 10% db 108 db n : N = 1 : 4 n/n = 25% Φ = [ U 1 U 2 U 3 U ] 4

27 brain vessel man 512 X X X 1024 original original original 7% sparse 5% sparse 2.38% sparse

28 Result 1: 512 X 512 Medical Image x actual x reconstructed x reconstructed 7% sparse noiselets*+ l1** SNR = 25 db chirp SNR = 120 db n : N = 1 : 4 Φ = [ U 1 U 2 U 3 U ] 4 n/n = 25% **Zhang, Yang, and Yin, YALL1: Your ALgorithms for L1 *Candes and Romberg, sparsity and incoherence in compressive sampling

29 Result 2: 512 X 512 Medical Image x actual x reconstructed x reconstructed 5% sparse noiselets*+ l1** SNR = 9.1 db chirp SNR = 50 db n : N = 1 : 8 Φ = [ U 1 U 2 U 3 U 4 U 5 U 6 U 7 U 8 ] n/n = 12.5% **Zhang, Yang, and Yin, YALL1: Your ALgorithms for L1 *Candes and Romberg, sparsity and incoherence in compressive sampling

30 Result 3: 1024 X 1024 real image x actual x reconstructed x reconstructed 2.38% sparse noiselets*+ l1** SNR = 4.4 db chirp SNR = 112 db n : N = 1 : 16 Φ = [ U 1 U 2 U 3 U 4 U 5 U 6 U 7 U 8 U 9 U 10 U 11 U 12 U 13 U 14 U 15 U 16 ] n/n = 6.25% **Zhang, Yang, and Yin, YALL1: Your ALgorithms for L1 *Candes and Romberg, sparsity and incoherence in compressive sampling

31 Results n/n Image sparsity n/k noiselets chirp RM 25% Brain 7% Φ = [ U 1 U 2 U 3 U 4 ] 12.5% Vessel 5% Φ = [ U 1 U 2 U 3 U 4 U 5 U 6 U 7 U 8 ] db 120 db 116 db db 50 db 10 db 6.25% Man 2.38% Φ = [ U 1 U 2 U 3 U 4 U 5 U 6 U 7 U 8 U 9 U 10 U 11 U 12 U 13 U 14 U 15 U 16 ] db 112 db 109 db

32 Stability of Algorithm y = Φ x k y = Φ x k + µ y = Φ x + µ µ : noise x k : k - sparse x : compressible

33 1. y = Φ x k + µ resilient to noise

34 2. y = Φ x + µ works for compressible signals

35 Conclusion New reconstruc)on algorithm by determinis)c CS method _especially suitable for images Ini)al best approxima)on method Speed up solu)on Decrease reconstruc)on error Beker computa)onal complexity for finding nonzero loca)ons A fast matrix- vector mul)plica)on via FFT dras)cally improves the least- squares solu)on )me Extend u9lity of CS with chirps and RM into regime of less sparsity Support imaging applica9ons

36 Acknowledgement This work was par)ally supported by NSF- DMS grant # NSF- DUE # ONR- BRC grant #N Robert Calderbank, Sina Jafarpour, Stephen Howard, Stephen Searle discussions of their work in determinis)c CS Jus)n Romberg advice about noiselets and l 1 algorithms Jim Pipe guidance about medical imaging, the knee, vessel, and brain images kangyu.ni@gmail.com