A Note on Guaranteed Sparse Recovery via l 1 -Minimization

Transcription

1 A Note on Guaranteed Sarse Recovery via l -Minimization Simon Foucart, Université Pierre et Marie Curie Abstract It is roved that every s-sarse vector x C N can be recovered from the measurement vector y Ax C m via l -minimization as soon as the s-th restricted isometry constant of the matrix A is smaller than 3/ , or smaller than 4/ for large values of s. We consider in this note the classical roblem of Comressive Sensing consisting in recovering an s-sarse vector x C N from the mere knowledge of a measurement vector y Ax C m, with m N, by solving the minimization roblem P minimize z C N z subject to Az y. A much favored tool in the analysis of P has been the restricted isometry constants k of the m N measurement matrix A, defined as the smallest ositive constants such that z Az + z for all k-sarse vector z C N. This notion was introduced by Candès and Tao in 3], where it was shown that all s-sarse vectors are recovered as unique solutions of P as soon as 3s + 3 4s <. There are many such sufficient conditions involving the constants k, but we find a condition involving only s more natural, since it is known 3] that an algorithm recovering all s-sarse vectors x from the measurements y Ax exists if and only if s <. Candès showed in ] that s-sarse recovery is guaranteed as soon as s < This sufficient condition was later imroved to s < / in 5], and to s < / in ], with the roviso that s is either large or a multile of 4. The urose of this note is to show that the threshold on s can be ushed further we oint out that Davies and Gribonval roved that it cannot be ushed further than / in 4 Our roof relies heavily on a technique introduced in Let us note that the results of ], 5], and ], even though stated for R rather than C, are valid in both settings. Indeed, for disjointly suorted vectors u and v, instead of using a real olarization formula to derive the estimate Au, Av k u v, where k is the size of suu suv, we remark that k max { A K A K I, cardk k }, so that Au, Av A K u, A K v A KA K u, v A KA K Iu, v k u v.

2 Using, we can establish our main result in the comlex setting, as stated below. Theorem. Every s-sarse vector x C N is the unique minimizer of P with y Ax if s < , and, for large s, if s < This theorem is a consequence of the following two roositions. Proosition. Every s-sarse vector x C N is the unique minimizer of P with y Ax if s < when s, 3 s < 6s 4 + r / s when s 3n + r with r 3, 3 4 s < s 5 + 3r / s when s 4n + r with r 4, 4 s < s/ 8n + 8r/5 when s 5n + r with r 5. Proosition 3. Every s-sarse vector x C N is the unique minimizer of P with y Ax if 3 s < + s s / 8 s s3s s where s 3/ s. Proof of Theorem. For s 8, we determine which sufficient condition of Proosition is the weakest, using the following table of values for the thresholds s s 3 s 4 s 5 s 6 s 7 s 8 Case Case Case For these values of s, requiring s < is enough to guarantee s-sarse recovery. As for the values s 9, since the function of n aearing in Case 4 is nondecreasing when r is fixed, the corresonding sufficient condition holds for s as soon as it holds for s 5. Then, because

3 requiring s < is enough to guarantee s-sarse recovery from Case 4 when 4 s 8, it is also enough to guarantee it when s 9. Taking Case into account, we conclude that the inequality s < ensures s-sarse recovery for every integer s, as stated in the first art of Theorem. The second art of Theorem follows from Proosition 3 by writing s s 3/ 8 s s3s s s 8 3/ 3 3/ , 6 4 and substituting this limit into 3. A crucial role in the roofs of Proositions and 3 is layed by the following lemma, which is simly the shifting inequality introduced in ] when k 4l. We rovide a different roof for the reader s convenience. Lemma 4. Given integers k, l, for a sequence a a a k+l 0, one has l+k jl+ / ] k aj] max, a j 4l k j Proof. The case l + k follows from the facts that the left-hand side is at most k a l+ and that the right-hand side is at least k a k. We now assume that l + < k, so that the subsequences a,..., a k and a l+,..., a l+k overla on a l+,..., a k. Since the left-hand side is maximized when a k+,..., a l all equal a k, while the right-hand side is minimized when a,..., a l all equal a l+, it is necessary and sufficient to establish that / ] a l+ + + a k k] + l + a max, l + a l+ + a l+ + + a k 4l k By homogeneity, this is the roblem of maximization of the convex function fa l+,..., a k : ] / a l+ + + a k + l + a k over the convex olygon P : { a l+,..., a k R k l : a l+ a k 0 and l + a l+ + a l+ + + a k }. Because any oint in P is a convex combination of its vertices and because the function f is convex, its maximum over P is attained at a vertex of P. We note that the vertices of P are obtained as intersections of k l hyerlanes arising by turning k l of the k l + inequality constraints into equalities. We have the following ossibilities: 3

4 if a l+ a k 0, then fa l+,..., a k 0; if a l+ a j < a j+ a k 0 and l+a l+ +a l+ + +a k for l+ j k, then a l+ a j /j, so that fa l+,..., a k j l/j ] / /4l ] / when j < k and that fa l+,..., a k k/k ] / /k ] / when j k. It follows that the maximum of the function f over the convex olygon P does not exceed max / 4l, / ] k, which is the exected result. Proof of Proosition. It is well-known, see e.g. 6], that the recovery of s-sarse vectors is equivalent to the null sace roerty, which asserts that, for any nonzero vector v ker A and any index set S of size s, one has 4 v S < v S. The notation S stands for the comlementary of S in {,..., N}. Let us now fix a nonzero vector v ker A. We may assume without loss of generality that the entries of v are sorted in decreasing order v v v N. It is then necessary and sufficient to establish 4 for the set S {,..., s}. We start by examining Case 4. We artition S {s +,..., N} in two ways as S S T T... and as S S U U..., where S : {s +,..., s + s } is of size s, T : {s + s +,..., s + s + t}, T : {s + s + t +,..., s + s + t},... are of size t, U : {s + s +,..., s + s + u}, U : {s + s + u +,..., s + s + u},... are of size u. We imose the sizes of the sets S S, S T k, and S U k to be at most s, i.e. s s, t s, s + u s. Thus, with : s, we derive from and v S + v S Av S + v S ] Av S, Av S + v S + Av S, Av S + v S Av S, A v Tk + Av S, ] A v Uk k k 5 v S v Tk + v S v Uk k 4 k

5 Introducing the shifted sets T : {s +,..., s + t}, T : {s + t +,..., s + t},..., and Ũ : {s +,..., s + u}, Ũ : {s + u +,..., s + u},..., Lemma 4 yields, for k, v Tk max, t ] v 4s etk, v Uk max, u ] v 4s euk. Substituting into 5, we obtain 6 v S + v S v S max, t ] v 4s S + v S max, ] u v 4s S To minimize the first maximum in 6, we have all interest in taking the free variable t as large as ossible, i.e. t s. We now concentrate on the second maximum in 6. The oint s, u belongs to the region R : { s 0, u 0, s s, s + u s }. This region is divided in two by the line L of equation u 4s. Below this line, the maximum equals / u, which is minimized for a large u. Above this line, the maximum equals / 4s, which is minimized for large s. Thus, the maximum is minimized at the intersection of the line L with the boundary of the region R other than the origin which is given by s : s 5, u : 8s 5. If s is a multile of 5, we can choose s, u to be s, u. In view of 4s s, 6 becomes v S + v S v S v S + ] c v S with c 5 s 8. Comleting the squares, we obtain, with γ : v S / s, vs γ + vs c γ + c γ. Simly using the inequality v S c γ 0, we deduce v S + + c γ. Finally, in view of v S s v S, we conclude v S + + c v S. Thus, the null sace roerty 4 is satisfied as soon as + + c <, i.e. < c. 5

6 Substituting c 5/8 leads to the sufficient condition s < / 3 + 3/ , valid when s is a multile of 5. When s is not a multile of 5, we cannot choose s, u to be s, u, and we choose it to be a corner of the square s, s ] u, u ]. In all cases, the corner s, u is inadmissible since s + u > s, and among the three admissible corners, one can verify that the smallest value of max / 4s, / u ] is achieved for s, u s, u. With this choice, in view of 4s s, 6 becomes v S + v S v S v S + ] c v S s with c s 8s/5. The same arguments as before yield the sufficient condition s < / 3+ + s/ 8s/5, which is nothing else than Condition 4. We now turn to Cases and 3, which we treat simultaneously by writing s n+r, r, for 3 and 4. We artition S as S S T T... and S S U U..., where S : {s +,..., s + s } is of size s n +, T : {s + s +,..., s + s + t}, T : {s + s + t +,..., s + s + t},... are of size t s, U : {s + s +,..., s + s + u}, U : {s + s + u +,..., s + s + u},... are of size u s. Moreover, we artition S as S S, where S,..., S r are of size n + and S r+,..., S of size n. We then set w 0 : v S, w : v S + + v S + v S, w : v S + v S3 + + v S + v S,..., w : v S v S + v S, so that w j v S + v S and w j v S + v S. With : s, we derive from and v S + v S Av S + v S / A w j, A vs S r Awj, A v Tk + Awj, A v Uk ] 7 k r w j v Tk + k jr+ jr+ k w j v Uk Taking into account that s 4s and s 4s, Lemma 4 yields, for k, v Tk v etk, v Uk v euk, s s where T : {s +,..., s + t}, T : {s + t +,..., s + t},..., and Ũ : {s +,..., s + u}, k 6

7 Ũ : {s + u +,..., s + u},.... Substituting into 7, we obtain 8 v S + v S r v w j S r v S s v S + s + j jr+ w j + ] v w j S s jr+ ] s s w j. We use the Cauchy Schwarz inequality to derive r s w j + s w s r j r + r w j s j jr+ j s r 9 v S + v S s v S. Setting a : v S and b : v S + v S v S, 8 and 9 imly a + b v S a + ] c b, with c : s r s s. Comleting the squares, we obtain, with γ : v S / s, a γ + b c γ + cγ. Thus, the oint a, b is inside the circle C assing through the origin, with center γ, c γ. Since b a, this oint is above the line L assing through the origin, with sloe. If a, b denotes the intersection of C and L other than the origin we then have a a + c γ, as one can verify that the oint on the circle C with maximal abscissa is below the line L. Finally, in view of v S s v S s a, we conclude that v S + c v S. Thus, the null sace roerty 4 is satisfied as soon as + c <, i.e. < + + c. Secifying 3 and 4 yields Conditions and 3, resectively. As for Case, corresonding to s, we simly write, with :, v {} Av {} Av{}, A v {k} v {} v {k}, k 7 k

8 so that v {} v {} v {}, and the null sace roerty 4 is satisfied as soon as / <, i.e. < /. Proof of Proosition 3. As in the revious roof, we only need to establish that v S < v S for a nonzero vector v ker A sorted with v v N and for S {,..., s}. We artition S {s +,..., N} as S S T T..., where S : {s +,..., s + s } is of size s, T : {s + s +,..., s + s + t}, T : {s + s + t +,..., s + s + t},... are of size t. For an integer r s + s, we consider the r-sarse vectors w : v {,...,r}, w : v {,...,r+},..., w s+s r+ : v {s+s r+,...,s+s }, w s+s r+ : v {s+s r+,...,s+s,},..., w s+s that We imose w j rv S + v S s+s j and w j r v S + v S. s+s j s s, r + t s, t 4s, in order to justify the chain of inequalities, where : s, : v {s+s,,...,r }, so v S + v S Av S + v S A r s+s j s+s r j s + s rt s+s Aw j, A v Tk k w j v S t v S + v S v S. r / w j r, A v Tk k w j v Tk r j k s + s s+s w j v S t s+s j Simlifying by v S + v S and using v S s v S s v S + v S, we arrive at 0 v S ss + s rt v S + σ ρτ v S, where σ : s s, ρ : r s, and τ : t s. 8

9 Pretending that the quantities σ, ρ, τ are continuous variables, we first minimize +σ / ρτ, subject to σ, ρ+τ, and τ 4σ. The minimum is achieved when ρ is largest ossible, i.e. ρ τ. Subsequently, the minimun of + σ / ττ, subject to σ, τ, and τ 4σ, is achieved when σ is largest ossible, i.e. σ τ/4. Finally, one can easily verify that the minimum of + τ/4 / ττ subject to τ is achieved for τ 6 4. This corresonds to σ 3/ 0.47, and suggests the choice s 3/ s. The latter does not give an integer value for s, so we take s 3/ s s s, and in turn t 4s 4 s 4s and r s t 6s 4 s. Substituting into 0, we obtain s s v S 8 s s3s s v S. Thus, the null sace roerty 4 is satisfied as soon as Condition 3 holds. Remark. For simlicity, we only considered exactly sarse vectors measured with infinite recision. Standard arguments in Comressive Sensing would show that the same sufficient conditions guarantee a reconstruction that is stable with resect to sarsity defect and robust with resect to measurement error. References ] Cai, T., L. Wang, and G. Xu, Shifting inequality and recovery of sarse signals, IEEE Transactions on Signal Processing, to aear. ] Candès, E., The restricted isometry roerty and its imlications for comressed sensing, Comtes Rendus de l Académie des Sciences, Série I, , ] Candès, E. and T. Tao, Decoding by linear rograming, IEEE Trans. Inf. Theory 5005, ] Davies, M. and R. Gribonval, Restricted isometry constants where l sarse recovery can fail for 0 <, To aear in IEEE Trans. Inf. Th.. 5] Foucart, S. and M. J. Lai, Sarsest solutions of underdetermined linear systems via l q - minimization for 0 < q, Alied and Comut. Harmonic Analysis, 6009, ] Gribonval, R, and M. Nielsen, Sarse reresentations in unions of bases, IEEE Trans. Inform. Theory, 49003,