Fast Algorithms for Sparse Recovery with Perturbed Dictionary

Transcription

1 Fast Algorithms for Sparse Recovery with Perturbed Dictionary Xuebing Han, Hao Zhang, Gang Li arxiv:.637v3 cs.it May Abstract In this paper, we account for approaches of sparse recovery from large underdetermined linear models with perturbation present in both the measurements and the dictionary matri. Eisting methods have high computation and low efficiency. The total least-squares TLS criterion has welldocumented merits in solving linear regression problems while FOCal Underdetermined System Solver has lowcomputation compleity in sparse recovery. Based on TLS and methods, the present paper develops more fast and robust algorithms, TLS- and SD-. TLS- algorithm is not only near-optimum but also fast in solving TLS optimization problems under sparsity constraints, and thus fit for large scale computation. In order to reduce the compleity of algorithm further, another suboptimal algorithm named SD- is devised. SD- can be applied in MMV multiple-measurement-vectors TLS model, which fills the gap of solving linear regression problems under sparsity constraints. The convergence of TLS- algorithm and SD- algorithm is established with mathematical proof. The simulations illustrate the advantage of TLS- and SD- in accuracy and stability, compared with other algorithms. Inde Terms perturbation, linear regression model, sparse solution, optimal recovery, convergence, performance. I. INTRODUCTION The problem of finding sparse solutions to underdetermined system of linear equations has been a hot spot of researches in recent years, because of its widespread application in compressive sensing/sampling CS,, biomagnetic imagining3, source localization 4, signal reconstruction5, 6, etc. In the noise-free setup, CS theory holds promise to eplain the equivalence between l -norm minimization and l -norm minimization as solving eactly linear equations when the unknown vector is sparse7, 8. Variants of CS for noise setup of perturbed measurements are usually solved based on basis pursuit BP approach9, utilizing method of linear programming4 or Lasso, greedy algorithms e.g., R3, CoSaMP4, etc or least-squares methods with l -regularization e.g., 5, 5, 6. However, eiting BP, greedy algorithms and do not account for perturbations present in the dictionary matri, i.e. regression matri. Recently, only a little attention has been paid on the sparse problems with perturbations present both in measurements and dictionary matri. Performance analysis of CS and BP methods Xuebing Han is with Guilin Air-Force Academy, Guilin, P. R. China thuhb@gmail.com; Hao zhang and Gang Li are with the Department of Electronics Engineering, Tsinghua University, Beijing, P. R. China haozhang@tsinghua.edu.cn, Gangli@tsinghua.edu.cn; for the linear regression model under sparsity constraints was researched in 6, 7 and 8; a feasible approach in 9, named S-TLS, was devised to reconstruct sparse vectors based on Lasso from the fully-perturbed linear model. However, the research of 6, 7 and 8 are limited in theoretical aspect and do not devise systematic approaches. Due to its highly-computational burden, S-TLS is very time-consuming, and thus unsuitable for large scale problems. In this paper, an etension form of is devised solving sparse problems to fully-perturbed linear model. Belonging to categories of conve optimization, LP and Lasso have the stable results but their computational burden is the highest; greedy algorithms have low computation, but their performances can only be guaranteed when the dictionary matri satisfies some rigorous conditions, such as very small restricted isometry constants 7. was originally designed to obtain a sparse solution by successively solving quadratic optimization problems and was widely used to deal with compressed sensing problems. The obvious advantages of are its low computation and stable results. For, only a few iterations tends to be enough to obtain a rather good approimating solution. So it is an ecellent choice to develop to solve approimate sparse solutions to linear regression model, especially in large scale application. Our objective is to overcome the influence of perturbation present in dictionary matri and measurements on the accuracy of sparse recovery effectively. Meanwhile, the merits of, rapid convergence and good adaption to intrinsic properties of dictionary matri, are maintained. First, objective function to be optimized can be obtained under a Bayesian framework. Then the necessary condition for the optimizing solution is that each first-order partial derivative of objective function is equal to zero. Net we can get the iterative epression using iterative relaation algorithm. Finally, the new algorithms are proved to be convergent. The paper is organized as follows. In Section II, we introduce perturbed linear regression model for sparse recovery, and analyze the optimal problem simply. In Section III, we use a MAP estimate to obtain the objective function to be optimized, then yield an iterative algorithm to provide solutions, named TLS- for adopting TLS method and framework of. Convergence of TLS- is proved. In Section IV, we propose another algorithm based on and TLS model, named SD- to distinguish TLS-. Though SD- is a suboptimal optimal, its computation is low and it can be used in MMV case. In the simulation of Section V, the performances of mentioned algorithms are presented. Finally, we draw some conclusions in Section VI.

2 II. PERTURBED LINEAR REGRESSION MODEL Consider the underdetermined linear system of y = A, where A is an m n matri with m < n, y is the given m data vector, and is unknown n vector to be recovered. With being sparse, and A satisfying some property e.g., RIP7, CS theory asserts that eact recovery of can be guaranteed by solving the conve problem9, 7, : min s.t.subject to y = A. Suppose that data perturbations eist in the linear model A. The corresponding conve problem can be written as a Lagrangian form9, 4, 5: min y A +γ p p, where p p = p, γ > is a sparsity-tuning parameter9, and < p p is set to in 9, 9. What the present paper focusses on is how to reconstruct sparse vector efficiently from over- and especially under-determined linear regression models while perturbations are present in y and/or A. The perturbed linear regression model can be formulated as follows, : y = A+E+e, where e represents perturbation vector and E represents perturbation matri. Due to randomness and uncertainty, it is usually assumed that the components of noise in the same channel are independently and identically Gaussian distributed, e.g. e N,σ I and vece N,σ I, where vec is matri vectorizing operator. can be rewritten as B +D =, where B = y,a, D = e,e. Without eploiting sparsity, TLS has well documented merits solving above problem. For over-determined models TLS estimates are given by ˆ = argmin D, D F, s.t.b +D =, where F represents Forbenius-form operator. With the assumption of vecd N,σ, gives the equivalent solutions as y A ˆ = argmin + The distinct objective of the present paper is twofold: developing efficient solvers for fully-perturbed linear models, and accounting for sparsity of. To achieve these goals, following optimization problem must be sovled ˆ = argmin,d D F +γ p p, 3 where γ >, and < p. In 3, the l F -term forces the quadratic sum of perturbations to be minimal while thel p -term forces sparsity of recovery9, 9, and γ controls tradeoff between above two terms. Developing efficient algorithms to get the local even global optimum of 3 is the main goal. In net section, we will eplain how to get the objective function and estimate the value of γ with a bayesian formation, then develop the new method of optimization. III. TLS- ALGORITHM This section develops an etension of, TLS-, to solve using Bayesian framework 9 and main idea of TLS. For simplifying formulas, we assume σ = σ = σ, that is vecd N,σ. At the end of the section, we will introduce how to process the situation with σ σ. A. Bayesian Formulation From, we obtain y A = Gv, 4 where G =, T I m m, v = vecd, represents Kronecker product. Under Bayesian viewpoint, unknown vector is assumed to be random and independent of D. Then the MAP estimation of can be obtained as: ˆ MAP = argma ln p y = argmalnpy +lnp. 5 This formula is general and offers considerable fleibility. In order to obtain optimality of the resultant estimates, another assumption must be made on the distributions of the solution vector. As discussed in 5, the elements of sparse are assumed to be distributed as general Gaussian and independent, p = C ep β p m k p, 6 k= where C is constant, < p and β is constant depended on p with β = p Γ/p Γ3/p where Γ means Gamma function. Only one parameter characterizes the distribution in 6. The pdf moves toward a uniform distribution as p and toward a very peaky distribution as p. With v N,σ I andgg H = + I, we have lnpy = σ y A H y A + +C, 7 where C is constant. With the densities of the perturbation vector v and the solution vector, we can now proceed to find the MAP estimate as y A ˆ MAP = argmin + +γ p p, 8 where γ = σ /β p. B. Derivation of TLS- with The optimization problem 8 is equivalent to argmin z Jz where Jz Bz = +γ z p p, z = z 9, B = y,a. To simplify the objective function,we normalize z and get the equivalent form as min Bz z +γ z p p, s.t. z =.

3 3 Using Lagrange multiplier method, the objective function can be rewritten as Tz = Bz +γ z p p +λ zh z, where λ is the Lagrange multiplier. The factored gradient approach developed in 3, an iterative method can be derived to minimize Tz. A necessary condition for the optimum solution z is that it must satisfy z Tz =. We can get B H B +απz z = λz, 3 where α = pγ/, Πz = diag zi p i=,,n+. So the iterative relaation scheme can be constructed as B H B +απz k z k = λz k. 4 It is easily seen that λ should be the minimal eigenvalue of objective matri B H B + απz k. However, it s very hard to find it for two reasons: firstly, the minimal eigenvalue is likely close to zero because objective matri is approimately singular; secondly, the dimension of matri above is tremendous for most large scale application, which leads to a big computational burden for matri inversion. 4 implies that B H B +απz k zk = λ z k. 5 From 5, finding the minimal eigenvalue is taken place of by finding the maimal eigenvalue. The latter become much more well-posed. Moreover, with the aid of matri inversion formula, we have B H B +απz k = α W k W kb H αi BW kb H BW k, 6 where W k = Π z k. Let Φ k = W k W kb H αi BW kb H BW k, 7 then we obtain Φ k z k = α λ z k. 8 It should be mentioned that the dimension of matri αi BW k BH is much less than that of matri B H B + απz k, so the cost of matri inversion is etremely reduced. Besides, we need only calculate the maimal eigenvalue and corresponding eigenvector instead of all the eigenvalue and eigenvector of Φ k. That is to say, some highly efficient solver, such as Lanczos iteration, could be utilized to make the problem further simplified. Noting that the optimal problem 8 is not global conve, the TLS- algorithm guarantees convergence to a local optimum. Once the initial point z is close to the true point, estimation of true value can be found through iterations. In this paper, we set = A H AA H y, then z is set through substituting into and normalization of z. When the convergent solution z is obtained, we can get TLS = z,,z n+ T /z. 9 Algorithm is the algorithmic description of TLS-. Algorithm TLS-: Input: z, B, α, p. Set W k = diag z k i p, and p i=,,n+,; Calculate Φ k = Wk W k BH αi +BWk BH BWk. 3 Compute the largest eigenvalue λ k and corresponding eigenvector u k of Φ k using Lanczos method. 4 Set z k = u k. 5 If z k z k / z k < ǫ, eit; else goto step. C. Convergence and Sparsity To show that TLS- algorithm can approimately solve the sparse problem of through iterative method, two key results should be obtained: i TLS- is a convergent algorithm that it indeed reduces Jz at each iterate step; ii the convergence points of TLS- are sparse. proof of convergence: From 4 we have BW H B W q k +αq k λwk q k =, where B W = BW k, q k = W k z k. And q k can be treated as an optimal solution: q k = argmin q BW q +α q +λ q H W k q. From and the equivalence of optimization between 9 and, z k can be epressed a solution to an optimization problem: z k =argminq k z, where Q k z = Bz z +α W k z. So TLS- algorithm can be considered to be a method of re-weighted l -form minimization 5, 5. Since z k is the local unique solution to minimize Q k z, we have Q k z k < Q k z k 3 with z k, z k located in the same small domain and z k z k. And we can get the conclusion 5 that z i p z i p i p z i p z i z i i = p z T Πz z z T Πz z, 4 where Πz = diag zi p. With z k and z k z k z k obtained from the k th andkth iteration of TLS-, we have Jz k Jz k Bzk z k +αz T k W k z Bzk k z k +αz T k W k z k =Q k z k Q k z k <, 5

4 4 where z k and z k are obtained from the k-th and k -th iteration step of TLS-. The first inequality follows from 4 and the last inequality from 3. So the value of Jz k decreases as k increases. From 5 and Jz k, it can be concluded that TLS- is a convergent algorithm. proof of sparsity: Assuming z is a local minima of Jz, z is also a local minima to an optimization problem: min zi p s.t. B +Dz =, which can be rewritten as z i min i p s.t. y = A+E+e. 6 i Similarly shown in 4, 5, 4 especially p =, as an equivalence of l -norm optimization above optimization problem can obtain the local minima which are necessary sparse. The provement of equivalence between l -norm and l p -norm about fully-perturbed model is aslo an open problem. Let z be an fied point of the algorithm, and therefore a solution of 5. If z is not sparse, it is not a local minima of 6, so there must be other points close to z which can reduce Jz3. Thus it can be concluded that only sparse solutions are stable points of TLS- algorithm. D. Robust Modification Note that we assumed the components of perturbation matri e, E are i.i.d. independent and identically distributed. Actually, only noise eisting in the same channel is assumed to be i.i.d.. When e and E have the different distributed variances, it is necessary to normalize variances of perturbations before signal reconstruction. Assume that e and E are independent, and e N,σ I, vece N,σI. Then we have y A = Gv with G =, σ σ T I m m, v = e σ σ vece. It can be seen v N,σ I. For 9, instead of we have z =, σ σ T T, B = y, σ σ A. Now TLS- algorithm can be used to recover the sparse signal. IV. SD- ALGORITHM TLS- needs to compute the maimal eigenvalue and its corresponding eigenvector of matri Φ k in every iteration. By utilizing Lanczos algorithm, TLS- algorithm can be speeded up greatly. However, it is still possible to release much more the computation burden while the performance descends a little. In this section, a suboptimal algorithm, named SD- Synchronous Descending, is divised. Based on TLS model, Zhu in 9 devised a sparse recovery algorithm S-TLS. To optimize the objective function, S- TLS adopted iterative block coordinate descent method, yielding successive estimates of with E fied and alternately of E with fied until obtaining stable solutions. The algorithm needs several convergent procedures before final convergence. Different from S-TLS, SD- is more efficient, which only needs one convergent procedure, with estimating and E synchronously in each iteration; meanwhile, SD- has lower computation compleity. A. Bayesian Formulation In this section, and E in are both considered variants to be optimized. Assume that e N,σ I, vece N,σ I, and e, E are independent. So we have p e e = C 3 ep eh e p E E = C 4 ep σ veceh vece σ = ep E F σ +C 7 Where C, C are constant. The Bayesian formulation is described as ˆ MAP,Ê MAP = argma lnp,e y,e =argmalnpy,e+lnp+lnpe. 8,E Here we have lnpy,e = σ y A+E +lnc 3. 9 B. Derivation of SD- From 6 8 and 9, the objective function can be written as J,E = y A+E + σ σ tre H E+γ p p 3 where tr means trace of matri and tre H E = E F. The necessary condition of the optimal solution satisfies that partial differentiation to each component for J, E is equal to zero, that is: a E J,E =. We can get E J,E = E H y A H +σ σ E. So we can get the estimate of E as a function of : E = y AH σ σ + H. 3 Here the fact of λi +F H F F H = F H λi +FF H is used. b J,E =. Referring to 5, we can get the iterative relaation scheme of as k = W k A H k A k A H k +αi y, 3 k where α = pγ, W k = diag i p and i=,,n A k = A+E k W k. There eists error inevitably when we estimate E, thus accuracy of estimating will be affected. It is a suboptimal algorithm. Algorithm is the algorithmic description of SD-. Algorithm SD-:

5 5 Input: y,, E A, σ, σ, p. Set W k = diag k i p,; Calculate i=,,n, and p 5 normlized result recovered by regularized E k = y A k H k σ σ + k, and A k = A+E k W k ; 3 Calculate k = W k A H k A ka H k +αi y; 4 If k k / k < ǫ, eit; else goto step. amplitude/db 5 C. Proof of Convergence Formula 3 can be seen as k = W k b k, where b k can be treated as an optimal solution, that is b k = argmin y A k W k b +α b 33 Alternately and equivalently, k can be epressed a solution to an optimization problem: where k = argminq k, Q k = y A k +α W k. 34 Referring to 3-5, we can conclude that SD- is also a convergent algorithm. D. SD- Etension: MMV case Besides low computation, the breakthrough advantage of SD- is that it can be used in multiple measurement vectors MMV model, while TLS- and S-TLS 9 cannot fit this model or remain to be developed. Supposed y l = A+E l + e l, with l =,,L, where y l R m and l R n. Suppose that the vectors l,l =,,L are sparse and have the same sparsity profile, and let Y = y,, y L, X =,, L. The objective function for MMV case is epressed as JX,E = Y A+EX F + σ n E F +γ L p/ l i σ i= l= The weight matri W k can be re-epressed as 6 W k =diag c k i p/ L l with c k i = k i Then formula 3 can be rewritten as l= 35 / 36 X k = W k A H k A k A H k +αi Y 37 For E J,E = we can renew 3 as σ E k = Y AX k σ I +Xk H X k Xk H 38 Then the Algorithm can be modified to fit MMV model as Algorithm 3. Algorithm 3 MMV SD-: Input: y,, E A, σ, σ, p a Result recovered by Regularized : weak signal is loss amplitude/db 5 5 normlized result recovered by TLS b Result of recovered by TLS-: weak signal is found Fig.. Result of weak signal recovery with m =,n = 3 ck Set W k = diag i p where c k i = Calculate L l= i=,,n, l k i /, p,; E k = Y AX k σ σ I +XH k X k X H k and A k = A+E k W k ; 3 Calculate X k = W k A H k A ka H k +αi Y ; 4 If X k X k / X k < ǫ, eit; else goto step. V. SIMULATION RESULTS The parameters in this paper are set as: norm-factor p =.5, convergence threshold ǫ =.. In each Monte Carlo simulation, trials are carried out independently. In each trial, the m n dictionary A is chosen as Gaussian random matri, entries of which are independently, identically and normally distributed. In order to analyze the mentioned algorithms, the true sparse solution has to be known, and it is hard to know in practice problems. The algorithm in one simulation is considered to be successful if all nonzero-locations of are found eactly; otherwise, the algorithm is considered to be failed.

6 TLS SD Regularized TLS SD a success probability of algorithms in finding the support set correctly Fraction of sparsity entries k/n a percentage success with randiness distribution in sparse entries RMSE of recovered amplitude Regularized TLS SD TLS SD b RMSE of signal amplitude recovery Fig.. Performance of involved algorithms with m =,n = 3 A. Single Measurement Vector Case This subsection shows the advantages of recovering ability of new algorithms from TLS model with numerical simulation. Let be a s-sparse vector, i.e. = s, and let the average power of be normalized, i.e. i i =. In each trial, entries of matri e, E are also independently and identically Gaussian distributed with mean zero and variance σ. Then overall SNR can be represented as /σ. The indices of nonzero coordinate set T are chosen randomly from a discrete uniform distribution U, N without repetition. In following simulations, besides TLS- and SD-, other algorithms will be involved: standard FO- CUSS 5, Regularized 5, and S-TLS 9. In Fig. and Fig., the number of rows and columns of dictionary matri are set to and 3 respectively. In Fig., SNR is set to 5 db, T = 3,5,5 and T =.439,.986,.489 T. It can be seen from Fig. that TLS- does much better than in etracting weak signal when dictionary and measurement are both corrupted. For TLS-, the position and amplitude of if the variances of generalizing e and E are different, the performance of TLS- will not change, while the performances of the other algorithms will be affected Fraction of sparsity entries k/n b percentage success with the same amplitude in sparse entries Fig. 3. percentage success of involved algorithms with different k/m. m =,n = 3 signal are both recovered ecellently; the result of is failed, for weak signal is buried in False Peak brought by perturbation on dictionary and can not be distinguished correctly. Fig. A shows the statistical results of percentage success, and Fig. B shows the statistical root-mean-square error RMSE of signal amplitude recovery when algorithms can find the nonzero-coordinate T correctly under different SNR scenes. TLS- and SD- are presented to be more robust from Fig. a, and perform much better on amplitude recovery from Fig. b. Fig. 3 shows the percentage-success curves of algorithms with different k/m. In the simulation, m =, n = 3, k =,,,, SNR=5dB and entries of T are set to obey i.i.d. normal distribution in Fig. 3a and in Fig. 3b. It can be seen from Fig. 3 that, TLS- and SD- perform always better than common algorithms and S-TLS designed to solve fully-perturbed model as k/m changes. In the simulations of Fig. 4 and Table I, m = 8, n = 5, s = 3 and T =,, T / 3. With smooth curves, Fig. 4 shows that the recovery performance of TLS- in this scenario is much better than the other algorithms; SD-

7 7 RMSE of recovered amplitude Regularized TLS SD SNR RegFOC TLS-FOC SD-FOC S-TLS db sec sec sec sec sec TABLE I RUN-TIME OF ALGORITHMS WITH m = 8,n = 5. THE SIMULATIONS ARE DONE IN MATLAB 7.8 ON A CORE, 3.-GHZ, -GBYTE RAM PC Fig. 4. RMSE of signals recovery in the condition of m = 8, n = 5. is superior to S-TLS in low SNR, and inferior to S-TLS in high SNR. Table I shows run-times of mentioned algorithms under the same condition. In order to obtain a measure of the computational compleity, the average CPU times for each algorithm consumeing is tabulated in Table I. It can be seen that, as the same classified algorithms TLS- and SD- are much faster than S-TLS. By comparison with other algorithms, it can be concluded that TLS- and SD- have the complete advances in percentage succuss, accurate reconstruction and computational speed. And TLS- has the higher success percentage and more accurate reconstruction than SD- while SD- is faster than TLS-. B. MMV Case In this simulation we consider the performance of SD- in MMV case. X is a sparse matri with L columns and only s rows with nonzero entries. In each trial, the indices of nonzero rows in X are chosen randomly from a discrete uniform distribution, and the amplitudes of the row entries are generalized randomly from a standard normal distribution; entries of bothe ande l l=,,l are independently Gaussian distributed with mean zero and variance σ. The overall SNR is /σ. The measurement matri can epressed as Y = A+EX +e l l=,,l The relative MSE between the true and estimate solution is defined as 6 MSE = E ˆX X F X F In following simulations, besides SD-, the other algorithms will be involved, containing: MMV 6, Regularized MMV 6, and MMV 6. The number of rows and columns of dictionary A are set to and 3 respectively, and let s = 7. Two quantities are varied in this eperiment: SNR and L. Fig. 5 and Fig. 6 show success-probability curves and MSE curves respectively when L =,5,6. It can be found that as L becomes larger, success numbers become larger; however, MSE curves seem to be unchanged for it is related with perturbation and unrelated with L. MMV SD- performs better than other algorithms. VI. CONCLUSION In this paper, through etending algorithms, we have proposed two new algorithms, TLS- and SD-, to recover the sparse vector from an underdetermined system when the measurements and dictionary matri are both perturbed. The convergence of algorithms was considered. Then we applied SD- in MMV model with a row-sparsity structure. The simulations showed our approaches performed better than other present algorithms in computational compleity, percentage success and RMSE of signal amplitude recovery. The benefits of TLS- and SD- make them be good candidates of sparse recovery algorithms for more practical applications. REFERENCES D. L. Donoho, Compressed sensing, IEEE Trans. on Inf. Theory, vol. 5, pp , April 6. E. J. Candes, Compressive sampling, International Congress of Mathematicians, vol. 3, pp , 6. 3 I. F. Gorodnitsky, J. George, and B. D. Rao, Neuromagnetic source imaging with focuss: A recursive weighted minimum norm algorithm, Electroencephalogr. Clin. Neurophysiol., vol. 95, no. 4, pp. 3 5, Oct D. Malioutov, M. Cetin, and A. S. Willsky, A sparse signal reconstruction perspective for source localization with sensor arrays, IEEE Trans. Signal Process., vol. 53, pp. 3 3, Aug I. F. Gorodnitsky and B. D. Rao, Sparse signal reconstructions from limited data using focuss: A re-weighted minimum norm algorithm, IEEE Trans. Signal Process., vol. 45, pp. 6 66, March S. F. Cotter, B. D. Rao, K. Engan, and K. Kreutz-Delgado, Sparse solutions to linear inverse problems with multiple measurement vectors, IEEE Trans. Signal Process., vol. 53, pp , July 5. 7 E. Candes and T. Tao, Decoding by linear programming, IEEE Trans. Inf. Theory, vol. 5, pp , December 5. 8 E. J. Candès, The restricted isometry property and its implications for compressed sensing, Comptes Rendus Mathematique, vol. 346, no. 9-, pp , Oct S. Chen, D. L. Donoho, and M. A. Saunders, Atomic decomposition by basis pursuit, SIAM J. Sci. Comput., vol., pp. 33 6, 998. D. L. Donoho and X. Huo, Uncertainty principles and ideal atomic decomposition, IEEE Trans. on Inf. Theory, vol. 47, pp , Novermber. R. Tibshirani, Regression shrinkage and selection via the lasso, J. Roy. Statist. Soc., vol. 58, pp , 996.

8 L= Regularized SD MSE of recovered amplitude L= Regularized SD L=5.5. L=5 Regularized SD Regularized SD MSE of recovered amplitude L=6.5. L=6 Regularized SD Regularized SD MSE of recovered amplitude Fig. 5. Success probability of algorithms obtaining all s nonzero rows in MMV case, with m =, n = 3, s = 7 and Number of observation vectors is set to L =,5,6. Fig. 6. Relative MSE of amplitude recovery in MMV case, with m =, n = 3, s = 7 and Number of observation vectors is set to L =,5,6. J. A. Troppb and A. C. Gilbert, Signal recovery from random measurements via orthogonal matching pursuit, IEEE Trans. Inf. Theory, vol. 53, no., pp , 7. 3 D. Needell and R. Vershynin, Signal recovery from incomplete and inaccurate measurements via regularized orthogonal matching pursuit, IEEE J. Selected Topics Signal Process., vol. 4, pp. 3 36,. 4 D. Needell and J. A. Troppb, Cosamp: Iterative signal recovery from incomplete and inaccurate samples, Appl. Comput. Harmon. Anal., vol. 6, pp. 3 3, 9. 5 B. D. Rao, K. Engan, S. F. Cotter, J. Palmer, and K. Kreutz-Delgado, Subset selection in noise based on diversity measure minimization, IEEE Trans. Signal Process., vol. 5, pp , March 3. 6 M.A. Herman and T. Strohmer, General deviants: An analysis of perturbations in compressed sensing, IEEE Journal of Selected Topics in Signal Process., vol. 4, pp , April. 7 D. H. Chae, P. Sadeghi, and R. A. Kennedy, Effects of basis-mismatch in compressive sampling of continuous sinusoidal signals, in Proc. of nd Intl. Conf. on Future Computer and Communication, May Y. Chi, A. Pezeshki, L. Scharf, and R. Calderbank, Sensitivity to basis mismatch in compressed sensing,, Mar H. Zhu, G. Leus, and G. B. Giannakis, Sparsity-cognizant total least-squares for perturbed compressive sampling, IEEE Trans. Signal Process., vol. 59, pp. 6,. E. J. Candes, The restricted isometry property and its implications

9 9 for compressed sensing, Comptes Rendus Mathematique, vol. 346, no. 9-, pp , 8. G. H. Golub and C. F. Van Loan, An analysis of the total least squares problem, SIAM J. Numer. Anal., vol. 7, pp , December 98. X. Zhang, Matri analysis and applications, Tsinghua Univ. Press, Bejing, 4. 3 B. D. Rao and K. Kreutz-Delgado, An affine scaling methodology for best basis selection, IEEE Trans. Signal Processing, vol. 47, pp. 87, Jan E. J. Candès, J. K. Romberg, and T. Tao, Stable signal recovery from incomplete and inaccurate measurements, Comm. Pure Appl. Math., vol. 59, pp. 7 3, 6.