Robust multichannel sparse recovery

Transcription

1 Robust multichannel sparse recovery Esa Ollila Department of Signal Processing and Acoustics Aalto University, Finland SUPELEC, Feb 4th, 2015

2 1 Introduction 2 Nonparametric sparse recovery 3 Simulation studies 4 Source localization with Sensor Arrays

3 2/24 Single Measurement Vector Model (SMV) y = Φ M 1 M N x N 1 + e N 1 Φ = ( φ 1 φ N ) = ( φ(1) φ (M) ) H, M N known measurement (design, system) matrix x = (x 1,...,x N ), unobserved signal vector e = (e 1,...,e M ), random noise vector. Objective recover K-sparse or compressible signal from y knowing Φ, K NOTE: typically, M < N (ill-posed model) and K N.

4 3/24 Single Measurement Vector Model (SMV) y = Φ M 1 M N x N 1 + e M 1 If we can determine Γ = supp(x), then the problem reduces to conventional regression problem (more responses than predictors ): y Φ x e

5 Multiple Measurement Vectors Model (MMV) y i = Φx i +e i, i = 1,...,Q In matrix form Y = Φ M Q M N X N Q + E M 1 Y = ( y 1 y Q ), observed measurement matrix X = ( x 1 x Q ), unobserved signal matrix E = ( e 1 e Q ), noise matrix Objective recover K-sparse (or compressible) signal matrix X knowing Y, Φ, K. N Q matrix X is called K-sparse if X 0 = supp(x) K, where Q Γ = supp(x) = supp(x i ) = {j {1,...,N} : x jk 0 for some k} i=1 4/24

6 5/24 Multiple Measurement Vectors Model (MMV) Key idea: signals are all predominantly supported on a common support set = Joint estimation can lead both to computational advantages and increased reconstruction accuracy See [Tropp, 2006, Chen and Huo, 2006, Eldar and Rauhut, 2010, Duarte and Eldar, 2011, Blanchard et al., 2014]. Applications EEG/MEG [Gorodnitsky et al., 1995] equalization of sparse communications channels [Cotter and Rao, 2002] blind source separation [Gribonval and Zibulevsky, 2010] source localization using sensor arrays [Malioutov et al., 2005]...

7 6/24 Matrix norms The mixed l p,q norm of X for p,q [1, ) ( ( ) q/p ) ( 1/q 1/q X p,q = x ij p = x (i) p) q. i j i If p = q = matrix p-norm X p,p = X p. l 2 (Frobenius) norm 2 is denoted shortly as. The mixed l -norms of X C N Q X p, = max i {1,...,N} x (i) p X,q = ( i (max j {1,...,Q} x ij ) q) 1/q. The row l 0 quasi-norm of X X 0 = supp(x) = nr. of number of nonzero rows of X

8 7/24 Optimization problem min X Y ΦX 2 subject to X 0 K is combinatorial (NP-hard). More feasible approaches: Optimization (Convex relaxation) Greedy pursuit: Orthogonal Matching Pursuit (OMP) Normalized Iterative Hard Thresholding (NIHT) Normalized Hard Thresholding Pursuit (NHTP) Compressive Sampling MP (CoSaMP) Guaranteed to perform very well under suitable conditions (restricted isometry property on Φ, non impulsive noise e) NIHT is simple and scalable, convenient for problems where the support is of primary interest

9 7/24 Optimization problem min X Y ΦX 2 subject to X 0 K is combinatorial (NP-hard). More feasible approaches: Optimization (Convex relaxation) Greedy pursuit: Orthogonal Matching Pursuit (OMP) Normalized Iterative Hard Thresholding (NIHT) Normalized Hard Thresholding Pursuit (NHTP) Compressive Sampling MP (CoSaMP) Guaranteed to perform very well under suitable conditions (restricted isometry property on Φ, non impulsive noise e) NIHT is simple and scalable, convenient for problems where the support is of primary interest

10 8/24 IDEA: replace X 0 by l p,1 norm Convex relaxation [Tropp, 2006] uses p = : where X,1 = i x (i). [Malioutov et al., 2005] uses p = 2: where X 2,1 = i x (i). min X Y ΦX 2 +λ X,1 min X Y ΦX 2 +λ X 2,1 good recovery depends on the choise of egularization parameter λ > 0 not robust due to the used data fidelity term (LS-loss) not as scalable as greedy methods

11 1 Introduction 2 Nonparametric sparse recovery 3 Simulation studies 4 Source localization with Sensor Arrays

12 9/24 Nonparametric approach based on mixed norms min X Y ΦX q p,q data fidelity subject to X 0 K sparsity constraint (P p,q ) = mixed l 1 norms provide robustness, i.e., give larger weights on small residuals and less weight on large residuals. We consider the cases (p,q) = (1,1): Y ΦX 1 = i j y ij φ H (i) x j robust norm for errors i.i.d. in space and time (p,q) = (2,1): Y ΦX 2,1 = i y (i) X H φ (i) robust norm for errors i.i.d. in space only and devise greedy simultaneous NIHT (SNIHT) algorithm

13 10/24 Normalized iterative hard thresholding (NIHT) Conventional l 2,2 approach : X n+1 = H K hard thresholding stepsize ( X n + µ n+1 Φ H (Y ΦX n ) neg. gradient X 2 ) ψ 2,2 (X) = X = Conventional l 2,2 approach ψ 1,1 (X) = [sign(x ij )] (elementwise operation of S( )) ψ 2,1 (X) = [sign(x (i) )] (rowwise operation of S( ))

14 10/24 Normalized iterative hard thresholding (NIHT) In our mixed norm l p,q approach: X n+1 = H K hard thresholding stepsize ( X n + µ n+1 Φ H ψ p,q (Y ΦX n ) neg. gradient X q p,q ) ψ 2,2 (X) = X = Conventional l 2,2 approach ψ 1,1 (X) = [sign(x ij )] (elementwise operation of S( )) ψ 2,1 (X) = [sign(x (i) )] (rowwise operation of S( ))

15 10/24 Normalized iterative hard thresholding (NIHT) In our mixed norm l p,q approach: X n+1 = H K hard thresholding stepsize ( X n + µ n+1 Φ H ψ p,q (Y ΦX n ) neg. gradient X q p,q ) ψ 2,2 (X) = X = Conventional l 2,2 approach ψ 1,1 (X) = [sign(x ij )] (elementwise operation of S( )) ψ 2,1 (X) = [sign(x (i) )] (rowwise operation of S( )) Spatial sign sign(x) = { x/ x, for x 0 0, for x = 0

16 10/24 Normalized iterative hard thresholding (NIHT) In our mixed norm l p,q approach: X n+1 = H K hard thresholding stepsize ( X n + µ n+1 Φ H ψ p,q (Y ΦX n ) neg. gradient X q p,q ) ψ 2,2 (X) = X = Conventional l 2,2 approach ψ 1,1 (X) = [sign(x ij )] (elementwise operation of S( )) ψ 2,1 (X) = [sign(x (i) )] (rowwise operation of S( )) Spatial sign sign(x) = { x/ x, for x 0 0, for x = 0

17 11/24 SNIHT(p,q) algorithm input : Y, Φ, sparsity K, mixed norm indices (p,q) output : (X n+1,γ n+1 ) as estimates of X and Γ = supp(x) initialize: X 0 = 0, µ 0 = 0, n = 0, Γ 0 = 1Γ 0 = supp ( H K (Φ H ψ p,q (Y)) ) while halting criterion false do 2 R n ψ = ψ p,q(y ΦX n ) 3 G n = Φ H R n ψ 4 µ n+1 = CompStepsize(Φ,G n,γ n,µ n,p,q) 5 X n+1 = H K (X n + µ n+1 G n ) 6 Γ n+1 = supp(x n+1 ) 7 n = n+1 end

18 12/24 Computation of the step size µ n+1 For convergence, it is important that the stepsize is adaptively controlled at each iteration. Given the found support Γ n is correct, one can choose µ n+1 as the min. of the objective fnc for the gradient ascent direction X n +µg n Γ n: L(µ) = ( Y Φ X n +µg n ) Γ n q p,q = Rn µb q p,q where R n = Y ΦX n and B = ΦG (Γ n ) For p = q: simple linear regression problem based on l p -norm, with response r = vec(r n ) and predictor b = vec(b). µ n+1 = G n (Γ n ) 2 / ΦG n (Γ n ) 2 for p = q = 2 For p = q = 1 and (p,q) = (2,1), minimizer of L(µ) can not be found in closed-form.

19 13/24 Computation of the step size µ n+1 When p = q = 1, the solution µ verifies a fixed point (FP) equation µ = H(µ) ( H(µ) = r ij 1 b ij 2) 1 r ij 1 Re(bij r ij), i,j i,j and R = R n µb (depends on unknown µ) and B = ΦG (Γ n ). We select µ n+1 as the 1-step FP iterate µ n+1 = H(µ n ), where µ n is the value of the stepsize at previous nth iteration. Often this approximation was within 1e 3 to minimizer of L(µ). Same approach is used in the case (p,q) = (2,1).

20 1 Introduction 2 Nonparametric sparse recovery 3 Simulation studies 4 Source localization with Sensor Arrays

21 14/24 Simulation study Y = Φ M Q M N X N Q + E M 1 Set-up Measurement matrix Φ: φ ij CN(0,1) with unit-norm columns Γ = supp(x) randomly drawn K elements from {1,...,N} signal matrix X: x ij = 10 and Arg(x ij ) Unif(0,2π) i Γ. For l = 1,...,L MC-trials (L = 1000), compute X [l] and Γ [l] Performance measures mean squared error MSE(^X) = 1 LQ L l=1 probability of exact recovery PER = 1 L ^X [l] X [l] 2. L I (^Γ [l] = Γ [l]). l=1

22 15/24 IIID Gaussian noise, e ij CN(0,σ 2 ) MSE (db) MSE vs SNR for Q=16 SNIHT(2, 2) SNIHT(2, 1) SNIHT(1, 1) HUB-SNIHT SNR (db) PER PER vs Q at SNR = 5 db SNIHT(2, 2) SNIHT(2, 1) SNIHT(1, 1) HUB-SNIHT Q (number of vectors) SNIHT(2, 2) but SNIHT(2, 1) and HUB-SNIHT (threshold c = 1.517) loose only 0.09 db in MSE (Q = 16)

23 16/24 IID t-distributed noise, e ij Ct ν (0,σ 2 ), σ 2 = Med F ( e i 2 ) SNR(σ) = 30 db SNR(σ) = 10 db, ν = 3 MSE (db) SNIHT(2, 2) SNIHT(2, 1) SNIHT(1, 1) HUB-SNIHT PER SNIHT(2, 2) SNIHT(2, 1) SNIHT(1, 1) HUB-SNIHT ν (degrees of freedom) Q (number of vectors) SNIHT(1, 1)

24 17/24 Row IIID compound Gaussian inverse Gaussian texture, e (i) CIG λ (0,σ 2 I Q ) SNIHT(2, 2) SNIHT(2, 1) SNIHT(1, 1) HUB-SNIHT MSE (db) SNR (db) SNIHT(2, 1)

25 1 Introduction 2 Nonparametric sparse recovery 3 Simulation studies 4 Source localization with Sensor Arrays

26 18/24 Sensor array and narrowband, farfield source model M sensors, K sources (M > K): Model for sensor measurements at time instant t y(t) = A(θ)x(t)+e(t), - steering matrix A(θ) = ( a(θ 1 )... a(θ K ) ) - unknown (distinct) Direction of Arrivals (DOA s) θ 1,...,θ K - known array manifold a(θ) (e.g., ULA) - unknown signal waveforms x(t) = (x 1 (t),...,x K (t)) Objective Estimate the DOA s of the sources given the snapshots y(t 1 ),...,y(t Q ) and the number of sources K impinging on the sensor array.

27 19/24 Multichannel sparse recovery approach to source localization Construct an M N overcomplete steering matrix A( θ) for a grid of all source locations θ 1,..., θ N of interest. Suppose that θ contains the true DOA s to some accuracy. The array model in matrix form then rewrites as Y = ( y(t 1 ) y(t Q ) ) = A( θ)x+e, where X C N Q is K-sparse, with K non-zero rows containing the source waveforms at time instants t 1,...,t Q. = K-sparse MMV model In the multichannel sparce recovery formulation A( θ) is completely known, and we can use SNIHT methods to identify the support.

28 19/24 Multichannel sparse recovery approach to source localization Construct an M N overcomplete steering matrix A( θ) for a grid of all source locations θ 1,..., θ N of interest. Suppose that θ contains the true DOA s to some accuracy. The array model in matrix form then rewrites as Y = ( y(t 1 ) y(t Q ) ) = A( θ)x+e, where X C N Q is K-sparse, with K non-zero rows containing the source waveforms at time instants t 1,...,t Q. = K-sparse MMV model In the multichannel sparce recovery formulation A( θ) is completely known, and we can use SNIHT methods to identify the support.

29 19/24 Multichannel sparse recovery approach to source localization Construct an M N overcomplete steering matrix A( θ) for a grid of all source locations θ 1,..., θ N of interest. Suppose that θ contains the true DOA s to some accuracy. The array model in matrix form then rewrites as Y = ( y(t 1 ) y(t Q ) ) = A( θ)x+e, where X C N Q is K-sparse, with K non-zero rows containing the source waveforms at time instants t 1,...,t Q. = Finding DOA s is equivalent to identifying Γ = supp(x) In the multichannel sparce recovery formulation A( θ) is completely known, and we can use SNIHT methods to identify the support.

30 19/24 Multichannel sparse recovery approach to source localization Construct an M N overcomplete steering matrix A( θ) for a grid of all source locations θ 1,..., θ N of interest. Suppose that θ contains the true DOA s to some accuracy. The array model in matrix form then rewrites as Y = ( y(t 1 ) y(t Q ) ) = A( θ)x+e, where X C N Q is K-sparse, with K non-zero rows containing the source waveforms at time instants t 1,...,t Q. = Finding DOA s is equivalent to identifying Γ = supp(x) In the multichannel sparce recovery formulation A( θ) is completely known, and we can use SNIHT methods to identify the support.

31 20/24 Simul set-up ULA of M = 20 sensors with half a wavelength interelement spacing. K = 2 independent Gaussian sources from θ 1 = 0 o and θ 2 = 8 o. Temporally/spatially white Gaussian error terms. Low SNR = 0 db. Uniform grid θ on [ 90,90] with 2 o degree spacing We compute PER rates and relative frequencies of found DOA estimates in the grid for 1000 Monte Carlo runs.

32 21/24 Number of snapshots Q = 4 Frequency DOA s at θ 1 = 0 o and θ 1 = 8 o. SNIHT(2, 2) SNIHT(1, 1) SNIHT(2, 1) DOA θ degrees (2 o sampling grid) PER rates 1 SNIHT(1,1) = 81% 2 SNIHT(2,1) = 73.1% 3 HUB-SNIHT = 42.6% 4 SNIHT = 37.6% SNIHT(1, 1)

33 22/24 Number of snapshots Q = 40 Frequency DOA s at θ 1 = 0 o and θ 1 = 8 o. SNIHT(2, 2) SNIHT(1, 1) SNIHT(2, 1) DOA θ degrees (2 o sampling grid) PER rates 1 SNIHT(1,1) = 100% 2 SNIHT(2,1) = 73.1% 3 HUB-SNIHT = 42.6% 4 SNIHT = 37.6% SNIHT(1, 1)

34 23/24 Conclusions robust and nonparametric appraches for simultaneous sparse recovery

35 23/24 Conclusions robust and nonparametric appraches for simultaneous sparse recovery Different method, SNIHT(1, 1), SNIHT(2, 1), HUB-SNIHT, can be selected for a problem at hand (iid/correlated noise, high-breakdown, etc)

36 23/24 Conclusions robust and nonparametric appraches for simultaneous sparse recovery Different method, SNIHT(1, 1), SNIHT(2, 1), HUB-SNIHT, can be selected for a problem at hand (iid/correlated noise, high-breakdown, etc) Fast and scalable greedy SNIHT algorithm: increase in robustness does not imply increase in computation time!

37 23/24 Conclusions robust and nonparametric appraches for simultaneous sparse recovery Different method, SNIHT(1, 1), SNIHT(2, 1), HUB-SNIHT, can be selected for a problem at hand (iid/correlated noise, high-breakdown, etc) Fast and scalable greedy SNIHT algorithm: increase in robustness does not imply increase in computation time! Applications in source localization and image denoising, etc.

38 24/24 This talk based on E. Ollila (2015a) Nonparametric Simultaneous Sparse Recovery: an Application to Source Localization PDF (ArXiv): EUSIPCO 15, submitted. E. Ollila (2015b). Robust Simultaneous Sparse Approximation A Festschrift in Honor of Hannu Oja, Springer, Sept

39 Blanchard, J. D., Cermak, M., Hanle, D., and Jin, Y. (2014). Greedy algorithms for joint sparse recovery. IEEE Trans. Signal Processing, 62(7). Chen, J. and Huo, X. (2006). Theoretical results on sparse representations of multiple-measurement vectors. IEEE Trans. Signal Processing, 54(12): Cotter, S. and Rao, B. (2002). Sparse channel estimation via matching pursuit with application to equalization. IEEE Trans. Comm., 50(3): Duarte, M. F. and Eldar, Y. C. (2011). Structured compressed sensing: From theory to applications. IEEE Trans. Signal Processing, 59(9): Eldar, Y. C. and Rauhut, H. (2010). Average case analysis of multichannel sparse recovery using convex relaxation. IEEE Trans. Inform. Theory, 56(1): Gorodnitsky, I., George, J., and Rao, B. (1995). Neuromagnetic source imaging with FOCUSS: a recursive weighted minimum norm algorithm. J. Electroencephalogr. Clin. Neurophysiol., 95(4):

40 24/24 Gribonval, R. and Zibulevsky, M. (2010). Handbook of Blind Source Separation, chapter Sparse component analysis, pages Academic Press, Oxford, UK. Malioutov, D., Çetin, M., and Willsky, A. S. (2005). A sparse signal reconstruction perspective for source localization with sensor arrays. IEEE Trans. Signal Processing, 53(8): Tropp, J. A. (2006). Algorithms for simultaneous sparse approximation. part II: Convex relaxation. Signal Processing, 86: