The Sparsest Solution of Underdetermined Linear System by l q minimization for 0 < q 1

Transcription

1 The Sparsest Solution of Underdetermined Linear System by l q minimization for 0 < q 1 Simon Foucart Department of Mathematics Vanderbilt University Nashville, TN Ming-Jun Lai Department of Mathematics, University of Georgia Athens, GA 3060 January 7, 008 Abstract We study tne l q approximation of the sparsest solution of underdetermined linear systems. Mainly we present a condition on the matrix associated with an underdetermined linear system under which the solution of l q minimization is the sparsest solution of the system. Our condition generalizes a similar condition in [Candés, Romberg and Tao 06] ensuring that the solution of the l 1 minimization is the sparsest solution. Our condition in l 1 case slightly improves the similar condition. We present a numerical method to compute the l q minimization. Our numerical experiments show that the l q method is better than many existing methods, to find the sparsest solution. 1 Introduction Given a matrix A of size m n with m < n, let R k = {Ax, x R n, x 0 k} be the range of A of all the k-component vectors, where x 0 denotes the number of nonzero entries in the vector x and k < m. Given a vector y R k, we are looking for a solution x R n with x 0 k such that y = Ax. In general, there may have many such solutions. The research problem is to find the sparsest one. That is, given y R k, solve the following minimization problem simon.foucart@vanderbilt.edu mjlai@math.uga.edu min{ x 0, x R n, Ax = y}. (1) 1

2 The solution of the above problem is called the sparsest solution of y = Ax. Although the problem in Eq. (1) is a NP problem (cf. [18]) in general, it can be solved by using many methods, e.g., l 1 minimization, OGA(orthogonal greedy algorithm) and BP(basis pursuit) under some assumptions such that the (P1) solution is also the (P0) solution. The l 1 minimization problem is the following min{ x 1, x R n, Ax = y}. () where x 1 = n i=1 x i for x = (x 1, x,, x n ) T. The solution 1 Ax is called the l 1 solution of y = Ax. There are several conditions on the matrix A that the l 1 solution is the sparsest solution (cf. [Donoho and Elad 03] [13]) and [Candes, Romberg, Tao 06][3]). One is to use the concept spark of matrix A. One is to use the so-called Restricted Isometry Property(RIP). Recall spark(a) is the smallest possible number σ such that there exists σ columns from A that are linearly dependent. It is clear that σ(a) rank(a) + 1. Theorem 1.1 ([Donoho and Elad 03]) A representation y = Ad is necessarily the sparsest possible if x 0 < spark(a)/. The concept of RIP can be explained as follows. Letting 0 < k < m be an integer and A T be a submatrix of A consisting of columns from A with column indices in T {1,,, n}, the k restricted isometry constant δ k of A is the smallest quantity such that (1 δ k ) x A Tx (1 + δ k) x for all subset T with #(T) k. If a matrix A has such a constant δ k > 0 for some k, A possesses RIP. With this concept, it is easy to see that if δ k < 1, then the solution of Eq. (1) is unique. Furthermore, Theorem 1. ([Candés, Romberg, and Tao 06]) Suppose that k 1 such that δ 3k + 3δ 4k < and let x R n be a vector with x 0 k. Then for y = Ax, the solution of Eq. () is unique and equal to x. This result is recently simplified slightly in the following way: Theorem 1.3 ([Candés 07]) Suppose that k 1 such that δ k < 1. Let x R n be a vector with x 0 k. Then for y = Ax, the solution of Eq. () is unique and equal to x.

3 In this paper, we study the following approach to approximate the (P 0 ) solution. Let x q = ( n x i q ) 1/q i=1 be the standard l q quasi-norm for 0 < q < 1. We consider the minimization for 0 < q < 1 min{ x q q, x Rn, Ax = y}. (3) A solution of the above minimization is denoted by q Ax. This is not a brand new approach. It was studied recently in [7]. It is shown that Theorem 1.4 ([Chartrand 07a]) Let q (0, 1]. Suppose that there exists a positive number b > 1 such that the matrix A has RIP such that δ ak + bδ (a+1)k < b 1 for a = b q/( q). Then the solution of Eq. (3) is the sparest solution. This above result was further simplified in [8] in the following sense. Corollary 1.1 ([Chartrand 07b]) Suppose that δ k+1 < 1. Then there is q > 0 such that for any y = Ax with x 0 k, the solution of l q minimization in Eq.(3) is unique and is x. However, the proof is incorrect. In his proof, a > 1 because of b and a < 1+1/k. Then no such a so that ak can be an integer. In this paper we study the l q minimization in Eq. (3) by generalizing and refining the proof in [Candes, Romberg, and Tao 06]. Comparing to the results above, our results are a slight improvement of Theorem 1. and Theorem 1.3 and can be derived a correct version of Corollary 1.1. For convenience, let us state our theorems here and leave the proof to a section later. To this end, we need the following notations: cond k (A) = max cond(a T ) = #(T) k max T {1,,n} #(T) k In addition, let α k, β k be the best constants such that largest eigenvalue of A T A T smallest eigenvalue of A T A. T α k z Az β z for any vector z R n with z 0 k. In fact, α k = 1 δ k and β k = 1 + δ k. Our main result in this paper is 3

4 Theorem 1.5 Fix q (0, 1]. Suppose that c l := cond l (A) < 1 + ( ) /q 1 l, k for some integer l > k. Then every k-sparse vector x is exactly recovered by q Ax, the minimization solution of Eq. (3). Moreover, one has for every x R n x q Ax q C(q, c l )σ k (x) q and x q Ax C(q, c l) k 1/q 1/σ k(x) q, (4) where σ k (x) q := min z 0 k x z q and C(q, c l ) is a positive constant. Two cases are of particular interest: one is when l = k and q = 1. That is, Corollary 1. If the condition cond k (A) < 3 (5) is satisfied, then the l 1 solution is the sparsest one and the estimates in Eq. (4) hold. In terms of the RIP, since cond k (A) = 1+δ k 1 δ k, the condition (5) is that δ k < 1/. This improves Theorem 1.. The other case of interest is when l = k +1 and all q > 0. In this case, we may assume cond k+ (A) < +. Indeed, there exists a q small enough such that ( ) 1/q 1/ k + 1 cond k+ (A) < 1 +. k It follows from Theorem 1.5 that Corollary 1.3 Suppose that cond k+ (A) < +, then the solution q (Ax) is exactly the x with x 0 k for q > 0 small enough and independent of x. It is easy to see that cond k+ (A) < + is equivalent to λ k+ < 1. The above corollary can be given in the formulation of Corollary 1.1. Corollary 1.4 Suppose that δ k+ < 1. Then there is q > 0 such that for any y = Ax with x 0 k, the solution of l q minimization in Eq. (3) is unique and is x. It is easy to see that our result is an extension of Theorem 1. from the l 1 minimization to l q minimization for 0 < q 1. Also we improve the result in Theorem 1.3 and correct Corollary 1.1. In addition, we also consider the sparsest solution from imperfect measurements. Under the assumption that the measurements y = Ax+e contain a noise vector e with e θ for a fixed amount θ > 0, we need to solve min{ x 0, x R n, Ax y θ}. (6) 4

5 We again approximate the sparsest solution using the l q minimization for 0 < q 1. For q = 1, the above problem was studied in [Candes, Romberg, and Tao 06]. In the same way, we generalize their results from q = 1 to a general q (0, 1]. min{ x 0, x R n, Ax y θ}. (7) The paper is organized as follows. We begin with discussing the existence of the solution of the above minimization. Then we give a proof of Theorem 1.5. Next we use a random matrix theory to show that there are matrices obey our condition for large values of S with overwhelming probability. Finally we derive a numerical algorithm to compute appproximate solution of Eq. (3) and Eq. (7). We numerically experiment our scheme and compare with the existing l 1 method based on the matlab programs available on Candes webpage, the newly available reweighted l 1 method, the orthogonal greedy algorithm ( OGA) (cf. [0]) and regularized optimial matching pursuit (ROMP) (cf. [xxx]). Numerical results show that our algorithm is better than the l 1 method, the reweighted l 1 method, and is also better than the orthogonal greedy method (OGA) (cf. [0]) and ROMP method. Proof of Main Results We first show that the minimization problem Eq. (3) has a solution for q > 0. That is, the existence of the solution is independent of the RIP and the spark of A. Theorem.1 Fix 0 < q < 1. There exists a solution q Ax solving Eq. (3). Proof. Without loss of generality, we may assume that A is of full rank and the first m columns of A are linearly independent. Then we can find a quick solution x 0 R n such that y = Ax 0 by setting all x i = 0 for m < i n. Let ν be the infinimum of Eq. (3). Then it is easy to see that 0 ν x 0 q q. Let x j R n be a sequence such that Ax j = y and x j q q ν, i.e., x j q q ν + 1/j as j +. Thus, writing x j = (x j 1,, xj n )T, for each i between 1 and n, we claim that x j i is bounded from the above by B = max{1, (ν + 1)1/q } < +. If x j i 1, we are done. Otherwise, we have x j i q ν + 1/j ν + 1. Therefore we only need to look for the minimizer over the bounded interval [ B, B] n. Since the minimization functional is continuous, it achieves its minimal over a bounded domain. Thus, the minimizer of Eq. (3) exists. 5

6 Next we study when the solution of the l q minimization i.e., the solution of Eq. (3) is the sparsest solution, the solution of Eq. (1). Mainly we now turn to the proof of Theorem 1.5. Proof of Theorem 1.5. We begin recalling the following properties for 0 < q < 1: x 1 x q, x q n 1/q 1/ x, x + y q q x q q + y q q for any x,y R n. Step 1: Null space property. We establish first that, for any index set S with S 0 k and any η ker(a), one has η S q ρ η S q, with ρ := condl (A) 1 ( ) 1/q 1/ l, (8) k where S stands for the complementary of S in {1,, n}. For an S and η being given, we partition S in the following way: S 1 := {indices of the l largest entries in absolute value of η in S}, S := {indices of the next l largest entries in absolute value of η in S},. Using one of the above inequalities We write η S + η S1 1 α l η S q k 1/q 1/ η S k 1/q 1/ η S + η S1. = 1 α l A(η S + η S1 ) = 1 α l [ ] A(η Sr ), A(η Sh ). h,r A( r η Sr ), A( h η Sh ) We renormalize η Sr, η Sh so that they have an l -norm equal to one by setting It follows that η Sr := η Sr / η Sr, η Sh := η Sh / η Sh for r, h. 1 A(η Sr ), A(η Sh ) = A( η Sr ), A( η Sh ) = 1 [ A( ηsr + η Sh ) η Sr η Sh 4 A( η S r η Sh ) ] 1 [ β 4 l η Sr + η Sh αl η Sr η Sh ] 1 = [β l αl]. 6

7 Thus, we have obtained A(η Sr ), A(η Sh ) β l α l η Sr η Sh. Consequently, we have A(η S + η S1 ) 1 [β l α l ] r,h η Sr η Sh = β l α l ( ) η Sh h and ( ) η S + η S1 1 A(η αl S + η S1 ) cond l(a) 1 η Sh. h Now consider an integer h. For i S h and j S h 1, the inequality η i η j yields, by raising to the power q and averaging over j, the inequality η i q 1 l η S h 1 q q, which in turns implies, by raising to the power /q and summing over i, the inequality η Sh l 1 /q η Sh 1 q. We derive h η Sh l 1/ 1/q h 1 [ 1/q η Sh q l 1/ 1/q η Sh q] q = l 1/ 1/q η S q. h 1 We then deduce η S + η S1 condl (A) 1 l 1/ 1/q η S q. In conclusion, we obtain the desired claim Eq. (8): η S q k 1/q 1/ η S k 1/q 1/ η S + η S1 condl (A) 1 ( l k) 1/ 1/q η S q. Step : l q -estimate. Let S be an index set corresponding to the k largest entries of x in absolute value. Note that x S minimizes x z q over all k-sparse vectors z, i.e., z 0 k. With x := q Ax, observe that η := x x belongs to the kernel of A. We have This implies x q q x q q = x S q q + x S q q x S q q η S q q + η S q q x S q q. η S q q η S q q + x S q q = η S q q + σ k(x) q q ρq η S q q + σ k(x) q q, hence η S q q 1 ρ qσ k(x) q q. 7

8 We finally derive η q q η S q q + η S q q (1 + ρq ) η S q q (1 + ρq ) 1 ρ q σ k (x) q q. Step 3: l -estimate. With η and S, S 1,... defined as before, we recall that η S S1 = η S + η S1 ρk 1/ 1/q η S q, ( ) 1/ η S S1 = η Sh η Sh k 1/ 1/q η S q. h We have now established the desired result: η = η S S1 + η S S1 (1 + ρ )k 1 /q η S q /q (1 + ρ ) (1 ρ q ) /q k1 /q σ k (x) q. This completes the proof. In fact, the result in Theorem 1.5 can be slightly improved. That is, letting c l = cond l (A), ( η S q ρ η S q with ρ cl 1 )( 1 + c l + 3 ) ( ) 1/q 1/ k := ( c l + 1 ). (9) l As in Eq. (8), we had an upper for η S + η S1. We now uncover a lower bound for η S + η S1. Indeed, since η = η S + η S1 + ker(a), η S1 1 A(η βl S1 ) = 1 A( η βl Sh ), A( η Sh ) h =1 h =1 = 1 [ A(ηS ), A(η βl S ) + A(η S ), A( η Sh ) + A( η Sh ), A( η Sh ) ] h h h 1 [ α βl l η S A( η 0 ), A(η Sh ) + A(η S + η S1 ) ]. h As before, we obtain A( η S ), A(η Sh ) β l α l η S η Sh. Thus, ( ) η S + η S α l η βl S β l α l η βl Sh η S + 1 A(η βl S + η S1 ). Using the estimates in the proof of Theorem 1.5, we may now write ( ( ) ( α l ) η S + 1 α l 1 η Sh η S + 1 β l β l h (1 + 1 ) η S (1 c + 1 ) Σ η S + 1 l c l h h α l (1 1 c l )( c l 1 β l ) Σ, ) A(η S + η S1 ) 8

9 where Σ = h η S h. Since the quadratic polynomial (c l + 1)x + (1 c l )Σ x 1 (c l 1) Σ takes a nonpositive value at x = η S, we derive It now remains to write η S (c l 1)Σ + (c l 1) Σ + (c l + 1)(c l 1) Σ (c l + 1) = (c l 1)Σ(1 + c l + 3) =: ρσ. (c l + 1) η S q k 1/q 1/ η S k 1/q 1/ ρσ k 1/q 1/ ρl 1/ 1/q η S q = ρ It follows that Theorem. Let c l = cond l (A). Suppose that ( cl 1 )( 1 + c l + 3 ) ρ := ( c l + 1 ) ( ) 1/q 1/ k < 1. l ( ) 1/q 1/ k η l S q. The solution of Eq. (3) is the sparsest solution. The estimates in Eq. (4) hold. Let us make the following remark. On the role of the null space property, the case q = 1 is a natural one. Given a vector x supported on S with #(S) k, we want to have x = argmin{ z 1, Az = Ax}, i.e. x 1 x η 1 for all η ker(a). Using a characterization of best l 1 -approximation, i.e. 0 being a best approximation to x from ker(a), the desired property is equivalent to η ker(a), sign(x i )η i η i. i S i S Since we require this to hold for any x supported on S, it is necessary and sufficient to ask η ker(a), η S 1 η S 1. As seen in the above proof, the slightly stronger condition η S 1 ρ η S 1 with ρ < 1 even yields instance optimality, not just exact recovery of sparse vectors. In fact, a very simple argument reveals that the null space property is also somewhat necessary in the l q -case. Indeed, if η belongs to ker(a), and if an index set S has cardinality at most k, then A(η S ) = A( η S ) and since η S should be recovered by l q -minimization, one should have η S q η S q. 9

10 We next follow the arguments in [Candes and Tao 05] to discuss the value cond k (A). Consider a matrix A of size m n whose entries are i.i.d. Gaussian with mean zero and variance 1/m. Let T be a subset of {1,,, n}. Following upon the work of Geman and Wachter as discussed in [Candes and Tao 05], we have (cf. [1]) and λ min (A T A T) (1 λ), a.s. λ max (A T A T) (1 + λ), a.s. (cf. [17]) as m and #(T) with #(T)/m λ < 1. Thus, c k = cond k (A) 1 + λ 1 λ, a.s. By Corollary,1, it follows that λ < 1/. That is, almost surely, when k = #(T) < m/4, the solution of Eq. () is the sparsest solution. Our numerical experiments in the last section verify this case. 3 Recovery From Imperfect Data We next consider the situation that the measurements y are imperfect. That is, y = Ax 0 +e with unknown perturbation vector e which is bounded by a known amount e η. In this case we consider the minimization for 0 < q < 1. min{ x q q, x R n, Ax y θ}. (10) A solution of the above minimization is denoted by q,θ y. When q = 1, we refer to [Donoho, Elad, and Temlyakov 06] ([14]) for results based on the mutual coherence of the columns of A and related literatures. As in the previous section, we have Theorem 3.1 Fix 0 < q < 1 and y. There exists a solution q,θ y solving Eq. (10) for every positive θ. Similar to the proof of Theorem 1.5 we can show Theorem 3. Fix q (0, 1]. Suppose that c l := cond l (A) < 1 + ( ) /q 1 l, k for some integer l > k. Then every k-sparse vector x is approximated by q,theta y, the minimization solution of Eq. (3) satisfying x q,θ y C(q, c l) k 1/q 1/σ k(x) q + C(k)θ, (11) where σ k (x) q := min z 0 k x z q, C(q, c l ) is a positive constant and C(k) is a positive constant dependent on α k. 10

11 Proof. Let q Ax be the solution of Eq. (3. Then Note that we have x q,θ y x q Ax + q,θ y q Ax q,θ y q Ax 1 α k A( q,θ y q Ax) = 1 α k A( q,θ y y 1 α k Combining with Theorem 1.5 we immediately obtain the desired result. 4 Numerical Computation In this section we propose an algorithm to compute the minimization problem Eq. (3). Since it is a nonconvex minimization, we have to approximate it. Fix a positive number ɛ > 0. Let n l q,ɛ (x) = x i ( x i + ɛ) q 1 be a functional of x R n. Instead of the minimization of Eq. (3), we consider i=1 min{l q,ɛ (x), x R n, Ax = y}. (1) for a fixed ɛ > 0 and q > 0. Similar to the proof of Theorem.1, we can see that the solution of the above minimization exists. We next show that the solution x ɛ of Eq.(1) will converge to the solution Eq. (3). For convenience, let x be a solution of Eq. (3). Theorem 4.1 Fix 0 < q 1. Let x ɛ be the solution of Eq. (1). Then x ɛ converges to one of the minimizers of Eq. (3) as ɛ 0 +. Proof. Let ɛ = 1/j and x j be the solution of the minimization problem Eq. (1) for ɛ = 1/j. Writing x j = (x j,1,, x j,n ) T, it is easy to see that x j,i are bounded for each j. Thus, there is a subsequence from {x j, j = 1,,, } which is convergent. For convenience, let us say x j converges to x. We claim that x is a solution in Eq. (3). Recall that x is a solution of Eq. (3). Clearly, l q,1/j (x j ) l q,1/j ( x) x q q x j q q. (13) It is easy to see that f ɛ (t) = t ( t + ɛ) q 1 are continuous functions for any ɛ > 0 and f ɛ (t) is decreasing for ɛ for ɛ > 0. By Dini s theorem, f ɛ (t) is uniformly convergent to f 0 (t) over a bounded interval [ B, B]. Note that t ( t + ɛ) q 1 t q 0 11

12 as ɛ 0 + uniformly for all t [ B, B]. Thus, we have x j q q l q,1/j(x j ) 0 as j. By using Eq. (13), it follows that x q = x q. This completes the proof. We now derive a simple algorithm to compute the minimization in Eq. (1). Let x 0 be any solution solving Ax 0 = y. Fix q > 0 and ɛ > 0. We solve the following minimization problem to find x j : min x 1,,x n { n i=1 x i ( x (j 1) i + ɛ) q 1, Ax = y} (14) for j = 1,,,. If x j x j 1 1 is very small enough, we stop the iterations. This method is very easy to implement, e.g., by using the l 1 method to a weighted matrix Ãj of A. More precisely, letting ˆx = (ˆx 1,, ˆx n ) T with ˆx i = x i ( x (j 1) i + ɛ) q 1 and Âj = AD j with D being a diagonal matrix with entries ( x (j 1) i + ɛ) 1 q. That is, we use an l 1 minimization algorithm to solve min{ ˆx 1, Â jˆx = y} to get ˆx, and hence, x (j). It is interesting to point out that when q = 0, this algorithm becomes the reweighted l 1 method discussed in [6]. However, the algorithm proposed above may not converge for some q. A steepest descent method with some variations for the minimization problem Eq. (1) was extensively experimented in [8]. In the following section we use our simple algorithm to do some modest experiments. Numerical experiments show that l q minimization can work much better. 5 Numerical Experiments We now compare our method with several existing methods: l 1 method, reweighted l 1 mehtod, the OGA (cf. [0]), ROMP(cf. [19]) and our simple iterative l q method. Let us describe our comparisons in detail as follows. Example 1. Fix an integer 1 k 60. We choose a matrix A of size with i.i.d. Gaussian random entrices with zero mean and variance 1. For each A, we use a vector p of size 51 1 which contains a random permutation of integers 1,,, 51. For an index set T = q(1 : k), we let x(t) = randn(1 : k)) (in MATLAB command) be the true solution with x 0 k, we let y = Ax be the given vector. Choose x 0 = A(1 : 18, 1 : 18) 1 y and pad x 0 by zeros to form an initial guess vector of size Then we use our new method as well as the existing l 1 method, the reweighted l 1 method, OGA method described in [0] (The authors thank Alex Petukhov for providing his MATLAB program for comparison) and the ROMP method to find x with given A, y, x 0 and accuracy For our method we vary the choices of 0 < ɛ < 1 and q < 1. If x x < 10 3, we assume that the method is successful, otherwise the method fails. We repeat the above experiment 1

13 Comparison of l 1, rwl 1, OGA, l q, ROMP methods l 1 0. rw l 1 OGA l q ROMP Figure 1: Comparison of l 1 and l q methods for sparest solutions 100 times for each k from 1 to 60 and record the number of success that a method can find the solution during the 100 experiment. We divide these numbers by 100 and plot the pecentages of successfully solving the solutions by the 5 methods in the following figure From Figure 1, we can see that our new method does provide a better way to solve the Eq. (1) problem than all the other methods. Examle. In addition, we performed another set of comparion. Our numerical experiment is exactly the same as above except for x(t) = sign(randn)(1 : k) to the exact sparsest vector. The percentages of successfully solving the sparsest solutions by the methods mentioned in Example 1 are shown in Fig.. The figure above again shows that our method is the best among the all methods discussed in this paper. References [1] Candès, E. J. Compressive sampling. International Congress of Mathematicians. Vol. III, , Eur. Math. Soc., Z urich, 006. [] Candès, E. J., J. K. Romberg, Quantitative robust uncertainty principles and optimally sparse decompositions. Found. Comput. Math. 6 (006),, [3] Candès, E. J., J. K. Romberg, J. K. and T. Tao, Stable signal recovery from incomplete and inaccurate measurements. Comm. Pure Appl. Math. 59 (006),

14 Comparison of l 1, rwl 1, OGA, l q, ROMP methods 1 l 1 rw l 1 OGA l q ROMP Figure : Comparison of l 1, l q, and OGA methods for sparest solutions [4] Candès, E. J. and T. Tao, Decoding by linear programming, IEEE Trans. Inform. Theory 51 (005), no. 1, [5] Candès, E. J. and T. Tao, Near-optimal signal recovery from random projections: universal encoding strategies, IEEE Trans. Inform. Theory 5 (006), no. 1, [6] Candès, E. J., M. Watkin, and S. Boyd, Enhancing Sparsity by Reweighted l 1 Minimization, manuscritp, 007. [7] R. Chartrand, Nonconvex compressed sensing and error correction, in International Conference on Acoustics, Speech, and Signal Processing, IEEE, 007. [8] R. Chartrans, Exact reconstruction of sparse signals via nonconvex minimization, manuscript, 007. [9] Donoho, D. L., Compressed sensing, IEEE Trans. Inform. Theory 5 (006), [10] Donoho, D. L., Sparse components of images and optimal atomic decompositions. Constr. Approx. 17 (001), [11] D. L. Donoho, Unconditional bases are optimal bases for data compression and for statistical estimation, Appl. Comput. Harmonic Anal., 1(1993),

15 [1] D. L. Donoho, Unconditional bases and bit-level compression, Appl. Comput. Harmonic Anal., 3(1996), pp [13] Donoho, D. L. and M. Elad, Optimally sparse representation in general (nonorthogonal) dictionaries via l 1 minimization, Proc. Natl. Acad. Sci. USA 100 (003), no. 5, [14] Donoho, D. L., M. Elad, and V. N. Temlyakov, Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Trans. Inform. Theory, 5 (006), [15] Donoho, D. L. and J. Tanner, Sparse nonnegative solution of underdetermined linear equations by linear programming. Proc. Natl. Acad. Sci. USA 10 (005), no. 7, [16] Elad, M. and A. M. Bruckstein, IEEE Trans. Inf. Theory 48(00), [17] Geman, S., A limit theorem for the norm of random matrices, Ann. Prob., 8(1980), [18] Natarajan, B. K., Sparse approximate solutions to linear systems, SIAM J. Comput., vol. 4, pp. 734, [19] Needell, D. and R. Vershynin, Uniform uncertainty principal and signal recovery via regularized orthogonal matching pursuit, manuscript, 007. [0] Petukhov, A., Fast implementation of orthogonal greedy algorithm for tight wavelet frames, Signal Processing, 86(006), [1] Wachter, K. W., The strong limits of random matrix spectra for sample matrices of independent elements, Ann. Prob., 6(1978),