Application to Hyperspectral Imaging

Transcription

1 Compressed Sensing of Low Complexity High Dimensional Data Application to Hyperspectral Imaging Kévin Degraux PhD Student, ICTEAM institute Université catholique de Louvain, Belgium 6 November, 2013

2 Hyperspectral imaging is the fusion of spectrometry and imaging Spectrometry + Imaging Emission (or absorption) spectrum λ Light Wavelength in nm 800

3 RGB imaging mimics the Human Visual System 3 large bands to get chrominance Short, Medium and Long retina cones

4 Multi/Hyper-spectral imaging goes beyond Full spectrum (a lot of narrow bands) at every pixel

5 HSI mixes spatial and spectral information Classify several materials spatially on an image thanks to their unique spectral signature («fingerprint»).

6 Applications Microscopy, Spectroscopy Counterfeit detection Agriculture, Environmental Monitoring Biotechnology, Skin health, Endoscopy,... Surveillance, Security and Defense Non-contact quality control (e.g. Thin Films, Food, Pharmaceutical,...)...

7 How is it usually done? chromatic scanning electronic sensor imec s CMOS imager with Fabry-Perot filter spatial scanning dispersive element electronic sensor

8 What are the issues? Spatially very restrictive (low resolution, line scanning...) Low speed and power consuming acquisition Big amount of data (e.g. 200 spectral bands for every pixel) to acquire, compress, transmit and store. Complex devices

9 Why Low Complexity HD signals? The paradox Huge amount of data Big effort to acquire (expensive sensors) $ m But high level of redundancy Big effort to compress (expensive DSP) $ m example : K-sparse in Ψ

10 HD Data is the ideal field for Compressed Sensing Compression at the acquisition (e.g. optically). x R N Signal Φ Optical ( linear) compressed sensing Big gain both in the sensor and in the DSP. y R M measurements (sensor)

11 HD Data is the ideal field for Compressed Sensing Asymptotic theory Works better at high dimensions Classically in CS theory if K is the signal sparsity, Compression rate M N C K N log N K relative sparsity For HD data K increases slower than N so, K N and M N

12 Compressed Sensing of High Dimensional Data How to?

13 Model good signal priors Build and assess accurate signal model Use suited sparsity basis Ψ (wavelets, DCT,...) Combine sparsity bases Ψ = Ψ x Ψ y Ψ λ Append bases Ψ = [Ψ 1, Ψ 2, ] dictionaries Learn or design application specific dictionaries Use other low complexity priors e.g. TV, low rank (see later)

14 Design efficient sensing In terms of physical implementation Physically (e.g. optically) feasible Simple efficient electronics

15 Design efficient sensing In terms of mathematical properties Linear... Or not? (e.g. quantization); Fidel to reality; Restricted Isometry Property 0 < δ < 1 such that for any low complexity (sparse, low rank,...) signal x, 1 δ x 2 Φx δ x 2 x 2 y = Φx y 2 x 2

16 Design efficient sensing In terms of numerical algorithms efficiency Φ = Bottleneck of reconstruction algorithms Sparse, block sparse, block diagonal, binary matrices, FFT, FWT,... NOT full random (e.g. Gaussian) matrices!! For a 512x512x32 hyperspectral volume (8M voxels) sensing matrix with 2 46 entries i.e. 512 TB!

17 Reconstruct Use prior to build an optimization program Either convex relaxation and exact solution assume x is sparse: low l e.g. BPD 0 norm N arg min x x 0 s. t. Φx y 2 ε non convex hard to solve Or non convex original problem, approximate solution using fast greedy algorithm e.g. L0-LASSO (IHT, OMP, CoSaMP,...) arg min x Φx y 2 2 s. t. x 0 K

18 Reconstruct Use prior to build an optimization program Either exact solution of convex relaxation e.g. BPDN arg min x x 1 s. t. Φx y 2 ε convex Or approximate solution of non convex problem e.g. L0-LASSO arg min x 2 Φx y 2 s. t. x 0 K non convex

19 Reconstruct Choose the solver see Parikh and Boyd s monograph for a good intro Generic slow convex optimization (proximal algorithms,...) e.g. Douglas-Rashford, Chambolle-Pock Dedicated non convex fast greedy methods e.g. IHT, OMP, CoSaMP

20 Reconstruct Implement: HD data is challenging Optimized libraries (BLAS, LAPACK, ), Randomization (power method, truncated SVD,...) Parallel computing Multi-core (e.g. OpenMP) GPU (CUDA, OpenCL ) HW accelerators (Xeon Phi,...) Clusters (MPI )...

21 Some examples

22 CS of a monochromatic image 512x512 Smile Log Scale Haar DWT coefficients α/ α Sensor provides only 256x256 measurements ( M N = 25%). We can play with optics. K N 4% (K, ε) compressible ε = 10 4

23 CS of a monochromatic image Choice of the sensing Spread Spectrum Random Fourier (DCT) Ensemble Φ = S F H

24 CS of a monochromatic image Choice of the sensing Spread Spectrum Random Fourier (DCT) Ensemble Φ = S F H Random selection DCT Diagonal ±1 spread spectrum

25 CS of a monochromatic image Choice of the sensing Spread Spectrum Random Fourier (DCT) Ensemble Φ = S F H Random selection DCT Diagonal ±1 spread spectrum Good math property if M N = 0.25 C K N ln N C 0.66

26 CS of a monochromatic image Choice of the sensing Spread Spectrum Random Fourier (DCT) Ensemble Φ = S F H Random selection DCT Diagonal ±1 spread spectrum Good math property if M N = 0.25 C K N ln N C 0.49 Small (?) constant 1? < 1? High

27 CS of a monochromatic image Choice of the sensing Spread Spectrum Random Fourier (DCT) Ensemble Φ = S F H Random selection DCT Diagonal ±1 spread spectrum Good math property if M N = 0.25 C K N ln N C 0.49 Numerically efficient (optically feasible?) Small (?) constant 1? < 1? High Let s try anyway

28 CS of a monochromatic image Result : BPDN solved with CP super-resolution without CS (but with BPDN) Φ is a simple average over 4px blocks CS Φ is the SSRFE +18dB PSNR = 32dB PSNR = 50dB

29 Result : CS of a monochromatic image Original Bi-cubic interpolation (not BPDN) super-resolution BPDN without CS CS blurred blocks perfect

30 Result : CS of a monochromatic image Original This is a good small & sparse toy example. More natural small (512x512) images are not sparse enough (Lena, ) Bicubic interpolation (not BPDN) super-resolution BPDN without CS Works better when N and K N CS blurred blocks perfect

31 Reshape A discrete signal and in particular an image can also be mathematically treated as a matrix x R N X R n 1 n 2

32 Low-Rank Prior Rank and Singular Value Decomposition (SVD) of X R n 1 n 2 σ 1 X = USV = u 1 u n1 σ r 0 v 1 v n2 output unitary matrix r = σ i u i v i i=1 rank 1 matrices singular values rank r input unitary matrix Low rank means highly redundant (generalization of sparsity for matrices) Best rank r approximation (LS) = SV hard thresholding or truncated SVD efficient algorithms e.g. Matlab svds(x,r) or approx. with randomization : D. Achlioptas, F. Mcsherry, Fast Computation of Low Rank Matrix Approximations, 2007

33 Reshape A discrete signal and in particular an hyperspectral image can also be treated as a 3-ways tensor X R n 1 n 2 n 3

34 Tensors and Low Rank Prior Rank of a tensor : minimum r such that there exists a decomposition X = r i=1 The rank is NP-complete (J. Håstad 1990). a i b i c i Instead, we can use the n-rank i.e. rank of the n-unfolding matrix X n Unfolding: n 2 n 1 rank 1 tensors Exterior tensor product n 2 n 3 a i R n 1 b i R n 2 c i R n 3 n 3 n 1 X X (1) R n 1 n 2 n 3 X (2) R n 2 n 3 n 1 X (3) R n 3 n 1 n 2

35 Tensors and Low Rank Prior We would like to solve arg min X NP-complete rank X s. t. ΦX y ε n-rank relaxation arg min 1 X + rank X 2 + rank X 3 s. t. ΦX y ε This problem is not convex and (like the l 0 minimization) combinatorial. Nuclear norm convex relaxation (analogy with l 1 ) rank X = # σ i σ i > 0 X = σ i i arg min X X 1 + X 2 + X 3 s. t. ΦX y ε

36 Tensors and Low Rank Prior We would like to solve arg min X NP-complete rank X s. t. ΦX y ε n-rank relaxation arg min 1 X + rank X 2 + rank X 3 s. t. ΦX y ε This problem is not convex and (like the l 0 minimization) combinatorial. Nuclear norm convex relaxation (analogy with l 1 ) rank X = # σ i σ i > 0 X = σ i i arg min X X 1 + X 2 + X 3 s. t. ΦX y ε OK But why low rank?

37 Tensors and Low Rank Prior Hyperspectral images are low rank : the source mixing model materials ρ Source image h 1 h 3 materials ρ n 1 n 2 pixel list h 2 h ρ=4 n 3 wavelength ρ spectra, each with n 3 spectral samples : the mixing matrix H n 1 n 2 pixels, each with a concentration of the ρ sources : the source matrix ρ X 3 = HS T T = h i s i rank X rank X 3 ρ i=1 S

38 Tensors and Low Rank Prior Rank in the spatial directions (X 1 or X 2 ) may not be a good proxy for tensor rank and is not particularly relevant for natural images. r = 16, PSNR = 24.3dB K = 16384, PSNR = 33dB SVT Same number of degrees of freedom K = r(n 1 + n 2 ) Haar wavelets

39 Low Rank and Joint Sparse [M. Golbabee, P. Vandergheynst 2012] Joint sparsity of the unfolded tensor X 3 (in sparsity basis Ψ) n 1 n 2 n 3 k non zero columns Ψ 1 X 3 Same support for wavelet transform of each spectral band (= union of supports of each spectral band). Low-rank and joint sparse with the source mixing model : Source matrix is joint sparse (in the sparsity basis). ρ n 1 n 2 Ψ 1 (S T )

40 Low Rank and Joint Sparse Some theoretical results from Golbabee & Vandergheynst : Sparsity : X vec 0 = # x i,j x i,j 0 (number of non zero entries) Convex relaxation X vec 1 = x i,j i,j Convex minimization for sparse signal (BPDN) arg min X X vec 1 s. t. Φ X y ε (for nice (sub)gaussian Φ) M O K log n 1n 2 n 3 K = O kn 3 log n 1n 2 k NB : we note X = X 3 = x i,j and assume Ψ = Id

41 Low Rank and Joint Sparse Some theoretical results from Golbabee & Vandergheynst : Joint sparsity : X p,0 = # x.,i x.,i p > 0 (number of non zero columns) Convex relaxation X 2,1 = x i 2 i Convex minimization for joint sparse signal arg min X X 2,1 s. t. Φ X y ε (for nice (sub)gaussian Φ) M O k log n 1n 2 k + kn 3 O kn 3

42 Low Rank and Joint Sparse Some theoretical results from Golbabee & Vandergheynst : Low rank: rank(x) = # σ i σ i > 0 (number of non zero singular values) Convex relaxation X = i σ i Convex minimization for low rank signal arg min X X s. t. Φ X y ε (for nice (sub)gaussian Φ) M O r n 1 n 2 + n 3

43 Low Rank and Joint Sparse Some theoretical results from Golbabee & Vandergheynst : Convex minimization for low rank and joint sparse signal (one example) X = arg min X X 2,1 + λ X s. t. Φ X y ε (for nice (sub)gaussian Φ) M O k log n 1n 2 k + kr + n 3 r Error bound X X F κ 0 if not exactly joint-sparse X X r,k 2,1 k + X X r,k 2r + κ 1 ε if not exactly low rank if noisy measurements

44 Low Rank and Joint Sparse Demo example 512x512x32 Block diagonal sensing matrix Φ = Φ Id n3 Where Φ is the SSRFE (applied spatially) with m n 1 n 2 = 25% No spectral mixing no dispersive optical element. quite far from full Gaussian. Generated (k, ε) joint-compressible low rank HS image (from smile). With r = 6 and k n 1 n 2 = 4% for ε = 10 4

45 Low Rank and Joint Sparse Demo example 512x512x32 (target)

46 Low Rank and Joint Sparse Demo example 512x512x32 (reconstruction) PSNR = 43,5dB

47 Source Separation [M. Golbabee, S. Arberet, P. Vandergheinst 2012] There exist extensive spectral databases with spectra associated to a lot of materials. We can use a subset of these databases as a dictionary for sparsity prior. X = SH T unknown concentration Image R n 1n 2 ρ known spectral database R n 3 ρ

48 Source Separation [M. Golbabee, S. Arberet, P. Vandergheinst 2012] Modified reconstruction model: y = ΦX vec = Φ SH T vec = Φ H Id n1 n 2 R M S vec Φ R M ρn 1n 2 arg min X S vec 1 s. t. Φ S vec y ε

49 Source Separation [M. Golbabee, S. Arberet, P. Vandergheinst 2012] When Φ = Φ Id n3 (no spectral mixing) we can write Y = ΦX R M n 1n 2 R m n 1n 2 M = m n 3 R m n 3 Decorrelation before reconstruction (dimensionality reduction) Y = Y H T = ΦS R m ρ No dependency in H and smaller dimension. Much easier to handle in reconstruction algorithms!

50 Thank You!

51 References N. Parikh and S. Boyd, Proximal Algorithms, 2013 D. Brady, Optical Imaging and Spectroscopy, 2009 D. Achlioptas, F. Mcsherry, Fast Computation of Low Rank Matrix Approximations, 2007 J. Håstad, Tensor rank is NP-complete, 1990 M. Golbabee, P. Vandergheynst, Compressed Sensing of Simultaneous Low-Rank and Joint-Sparse Matrices, 2012 M. Golbabee, S. Arberet, P. Vandergheynst, Compressive Source Separation: Theory and Methods for Hyperspectral Imaging, 2012 For additional references, feel free to ask me at