Rui ZHANG Song LI. Department of Mathematics, Zhejiang University, Hangzhou , P. R. China

Transcription

1 Acta Mathematica Sinica, English Series May, 015, Vol. 31, No. 5, pp Published online: April 15, 015 DOI: /s Acta Mathematica Sinica, English Series Springer-Verlag Berlin Heidelberg & The Editorial Office of AMS 015 Optimal D-RIP Bounds in Compressed Sensing Rui ZHANG Song LI Department of Mathematics, Zhejiang University, Hangzhou 31007, P. R. China zhangrui11358@yeah.net songli@zju.edu.cn Abstract This paper establishes new bounds on the restricted isometry constants with coherent tight frames in compressed sensing. It is shown that if the sensing matrix A satisfies the D-RIP condition δ < 1/3 orδ < /, then all signals f with D f are -sparse can be recovered exactly via the constrained l 1 minimization based on y = Af, whered is the conjugate transpose of a tight frame D. These bounds are sharp when D is an identity matrix, see Cai and Zhang s wor. These bounds are greatly improved comparing to the condition δ < or δ < Besides, if δ < 1/3 or δ < /, the signals can also be stably reconstructed in the noisy cases. Keywords Compressed sensing, D-restricted isometry property, coherent, tight frames MR010) Subject Classification 94A1, 94A15, 94A08, 68P30 1 Introduction In compressed sensing, sparse signals can be estimated accurately using fewer linear measurements. The most widely studied algorithm has been l 1 -minimization min x R m x 1 s.t. Ax y B, where A R n m n m) is the sensing matrix, y R n is the observed data, B is a bounded set relying on the noise structure. In [1], Candès and Tao introduced restricted isometry property RIP) to analyse the performance of sensing matrix and the recovery algorithm. A variety of sufficient conditions on the RIP for the exact/stable recovery of -sparse signals have been introduced in the literature, e.g., [1,, 5, 9, 14, 17, 0, 3]. But in many practical situations, there are numerous signals of interest which are not exactly sparse under nature bases, or other orthonormal bases [10, ]. For instance, signals are sparse under Gabor frames or curvelets in radar and sonar or image processing, respectively, while there may not be any sparsifying orthonormal basis. This means that the signal f R m is expressed as f = Dx, where D R m d d>m) is some coherent and redundant dictionary and x is a sparse vector in R d. On the other hand, researchers have come to rely on and be grateful for the convenience representation of the signals under overcomplete dictionaries [10]. Comparing to standard compressed sensing, it is more natural to consider signal recovering when the signal of interest is sparse in terms of some redundant dictionary. Received April 11, 014, accepted September 10, 014 Supported by NSFC Grant No )

2 756 Zhang R. and Li S. In this paper, we pay attention to the following model, ˆf =argmin{ D f 1 : Af y B}, 1.1) f where D R m d is a redundant dictionary which we assume that it is a tight frame here this is simply to mae the analysis easier) and B is a bounded set determined by the noise structure. In [10], Candès et al. considered the l 1 -analysis model in the noisy case ˆf =arg min f f R m D 1 s.t. Af y ɛ, 1.) where ɛ is an upper bound on the noise level. As proposed by Candès et al., one way to describe the conditions about the sensing matrix under which the l 1 -analysis model performs well is to use D-RIP, which is a natural extension to the standard RIP [11, 1, 16] if D is an identity matrix, the two definitions coincide). To state our main results of the paper, let us review the definition of D-RIP. Definition 1.1 D-RIP) Let D be a tight frame. A measurement matrix A is said to obey the RIP adapted to D abbreviated D-RIP) with constant δ if 1 δ ) Dv ADv 1 + δ ) Dv holds for all suppv), wheresuppv) ={i : v i 0} and δ is defined to be the smallest constant which satisfies the above inequalities. It is well nown that there are many random matrices satisfying D-RIP with high probability. For instance, the random matrices with Gaussian, subgaussian or Bernoulli entries satisfy D-RIP with the number of measurements n on the order of logd/), see, e.g., [3]. Interested readers are referred to [10, Subsection 1.5] for details. Candès et al. proposed a sufficient condition, i.e., δ < 0.08, to reconstruct all sparse signals stably. In [18, 19], this bound has been improved to δ < and δ < 0.307, respectively. Liu et al. considered the general frames and used the condition 9δ +4δ 4 < 5toguarantee reconstruction stably [0]. Aldroubi et al. discussed l p -minimization model 0 <p 1) in [1], and proposed a sufficient condition under which this model is robust to bounded noise, stable with respect to perturbations of the sensing matrix A and the dictionary D. In this paper, we focus on the upper bounds on δ and δ to guarantee the exact recovery or stable reconstruction of signal by model 1.). Our first aim is to show that when δ < / or δ < 1/3, then all signals f with D f being -sparse can be recovered exactly via model 1.) and the bounds are sharp in some sense. Furthermore, if δ < / orδ < 1/3, then the signals can also be stably reconstructed in the noisy cases by 1.). Our proof relies heavily on a technique introduced in [7, 8]. Notations and Lemmas To state and prove our main results, let us introduce some notations and lemmas firstly. Let A R n m be a sensing matrix, where n m in the usual case. D R m d m d) isa tight frame which means each row of D is orthonormal. Let ˆD R d m) d be its orthonormal complement, which is also a tight frame. So for arbitrary vector x R d, Dx + ˆDx = x.

3 Optimal D-RIP Bounds in CS 757 If A satisfies D-RIP with constant δ, then for all -sparse vectors x, wehave 1 δ ) x ADx + ˆDx 1 + δ ) x..1) The form is similar to the definition of the standard RIP. For T {1,...,d}, we will denote by D T the matrix D restricted to the columns indexed by T,andwriteDT to mean D T ), T c to mean the complement of T in {1,...,d}. Let T 0 denote the set of the largest coefficients of D f in magnitude. Denote D f) [] to be the best -term approximation of D f in some norms, which means that D f) [] = DT 0 f. The following lemmas are useful in proving our main results. Lemma.1 [8, Lemmas 3.1 and 5.3]) Let and m be positive integers with m, λ 0. Let a 1 a a m 0 be a sequence of non-increasing real numbers satisfying a w + λ m w=+1 Then there exist non-negative real numbers { } 1 i,+1 j m such that a w. and Besides 1 = a j, a w + λ a +i + +1 j m m a w + λ, 1 i..) m w=+1 a w..3) Lemma. [7, Lemma 1.1]) For a positive number α and a positive integer, define T α, ) R d by T α, s) ={v R d : v α, v 1 α}. For any v R d, define the set of sparse vectors Uα,, v) R d by Uα,, v) ={v R d : suppu) suppv), u 0, u 1 = v 1, u α}. Then any v T α, s) can be expressed as N v = λ i u i, with 0 λ i 1, N λ i =1, and u i Uα,, v). Lemma.3 Let ˆf f, wheref is the signal we want to reconstruct which is -sparse in terms of D. WederivethatforanyT {1,...,d}, DT h 1 + DT cf 1 DT ch 1..4) Proof Since ˆf is the minimizer of 1.1) with B =0,wehave D f 1 D ˆf 1.

4 758 Zhang R. and Li S. So D T f 1 + D T cf 1 = D f 1 D T f + h) 1 + D T cf + h) 1 The desired inequality can be deduced. D T f 1 D T h 1 + D T ch 1 D T cf 1. Lemma.4 Let D d a iu i and u i be the indicator vector which means a vector with 1 or 1 in only i-th entry and zeros elsewhere. For convenience, we limit a 1 a a d 0. Then there exist nonnegative sequences { } 1 i,+1 j d such that and Proof d = a j, + a +i 1 Using.4) and restricting T to be T 0,weget +1 j d a w + D f D f) [] 1, 1 i. a i + DT f 0 c 1 d i=+1 We can complete the proof of lemma by applying Lemma.1 with λ = DT f 0 c 1 to the above inequality. The following lemma is the real extension of Lemma 5. in [8], and plays an important role in the proof of main results. Lemma.5 Let A satisfy D-RIP with δ < 1. Suppose that g, h 0, g + h, {d i } g, {e j } l, {t ij} 1 i g,1 j l are non-negative real numbers satisfying and min d i max e i 1 i g 1 i l g t ij = e j, 1 j l. Let {b i } h, {c i} h be real numbers and {u 11,...,u 1h ; u 1,...,u h ; u 31,...,u 3g ; u 41,..., u 4l } be a set of indicator vectors with different support in R p. Define h g l β 1 = b i u 1i + d i u 3i + e j u 4j R p and a i. Then, we have h g l β = c i u i + d i u 3i + e j u 4j R p. ADβ 1 + ˆDβ 1 ) ADβ + ˆDβ )

5 Optimal D-RIP Bounds in CS 759 h 1 δ ) b i + g d i + l ) ) h t ij 1 + δ ) c i + g d i + l ) t ij )..5) Proof We can prove the lemma by induction on l. Some ideas of the proof are from Cai s wor [8]. If l = 0,.5) is a trivial consequence of the equation.1), since g + h. Suppose.5) holds for l 1, we want to deduce the case of l. We can show the following equality in l space: g Mβ 1 Mβ = μ[ MP 1 MP ]+ ν i [ MQ i1 MQ i ] for any M R q p q is an arbitrary integer), where ν i = t il d i +t il i =1,...,g), μ =1 g ν i 0, and h g l 1 P 1 = b i u 1i + d i + t il )u 3i + e j u 4j, P = Q i1 = Q i = h c i u i + g l 1 d i + t il )u 3i + e j u 4j, h [ ] l 1 b w u 1w + d w + t wl )u 3w +d i + t il )u 4l + e j u 4j, h w i [ c w u 1w + w i l 1 ] d w + t wl )u 3w +d i + t il )u 4l + e j u 4j, which corresponds with the assumption of l 1. Now by induction assumption of l 1, we have ADβ 1 + ˆDβ 1 ) ADβ + ˆDβ ) [ h g l ) ) h μ 1 δ ) b i + d i + t ij 1 + δ ) c i + + g h ν i [1 δ ) b w + h 1 + δ ) c w + h =1 δ ) b i + g g d i + g d w + d w + l ) ) t wj l ) )] t wj l ) ) h t ij 1 + δ ) c i + g g d i + d i + l ) )] t ij l ) t ij ). Hence,.5) holds for the case of l. We complete the proof of lemma by induction. 3 Exact Recovery In this section, we present the main results of the paper, which provide new D-RIP bounds that guarantee the exact recovery of signals by the model 1.) with B = {0}. These bounds improve the best nown bounds for δ < and δ < , respectively. Furthermore, these bounds are also sharp in some cases.

6 760 Zhang R. and Li S. Theorem 3.1 Suppose the sensing matrix A satisfies D-RIP with δ < 1 3 for some integer d. Let y = Af, whered f is a -sparse vector. Then the minimizer of 1.) with B = {0} recovers f exactly, i.e., ˆf = f. Theorem 3. Suppose the sensing matrix A satisfies D-RIP with δ < / for some integer 1 d. Let y = Af, where D f is a -sparse vector. Then the minimizer of 1.) with B = {0} recovers f exactly, i.e., ˆf = f. Remar 3.3 The bounds δ < 1 3 and δ < / are sharp in the case when D is an identity matrix see [7, 8]). 4 Recovery of Signals with Bounded Noise In this section, we consider reconstruction of sparse signals in terms of D) in the presence of bounded noise. Specially we consider two types of bounded errors, B l = {z : z ɛ} and Dantzig selector bounded B DS = {z : D A z λ} [13]. We shall use ˆf l to denote the solution of 1.) with B = B l, and use ˆf DS to denote the solution of 1.) with B = B DS. Theorem 4.1 Suppose the sensing matrix A satisfies D-RIP with δ < 1 3 for some integer d. Then ˆf l f C 0 D f D f) [] 1 + C 1 ɛ, 4.1) ˆf DS f C 0 D f D f) [] 1 + C λ, 4.) where C 0 = δ + 1 3δ )δ )+1 3δ ) 1 3δ, C 1 = 1+δ ) 1 3δ, and C = 1 3δ. Proof We just present the proof of 4.1), while the proof of 4.) is essentially the same see, e.g., [4, 19]). To finish the proof of 4.1), we discuss two cases for which when is even and is odd, respectively. When is even, note Let / DT 11 a i u i, DT 1 D T 1 D T 31 3/ i=+1 d a i u i, D T / i=/+1 a i u i, i=3/+1 u j ), DT 3 a i u i, d i=/+1 u j ). M 0 = DT 11 h + DT 1 h, M 11 = DT 11 h + DT 1 h + DT 31 h, M 1 = DT 1 h + DT 1 h + DT 31 h, M 1 = DT 1 h + DT h + DT 3 h, M = DT 11 h + DT h + DT 3 h. We can derive the following equality: Ah, ADM 0 = ADM 11 + ˆDM 11 ) ADM 1 + ˆDM 1 )

7 Optimal D-RIP Bounds in CS ADM 1 + ˆDM 1 ) ADM + ˆDM ) + ADM 0 + ˆDM 0 ). 4.3) Then applying Lemma.5 to 4.3), we derive that the right-hand side of 4.3) can be bounded from below / 3/ d ) ) RHS 1 δ ) a i + a i δ ) + 3/ i= δ ) =1 δ ) 1 δ ) a i + / d a i + i=+1 a i δ ) ) +1 δ ) i=3/+1 i=+1 a i + a i + d d i=/+1 a i + i=/+1 i=3/+1 a i a i + ) ) +1 δ ) ) a i d ) ) ) a i δ a i + D T f ) 0 c 1, 4.4) where the first inequality comes from Lemma.5 and.1), the second inequality comes from.) and the Cauchy Schwartz inequality. Note that Ah, ADM 0 Ah ADM 0 4ɛ 1+δ a i, 4.5) where Ah = A ˆf y) Af y) ɛ. We can estimate the upper bound of a i using δ and D T c f 1 0. Combining 4.4) with 4.5) implies that a i δ D T c f ɛ 1+δ 1 3δ δ D T c f ɛ ) 1+δ +41 3δ )δ DT f 0 c 1 / + 1 3δ ɛ 1+δ +δ + 1 3δ )δ ) DT f 1/ 0 c. 4.6) 1 3δ Finally, applying.3) with λ = D T 0 c f 1 to 4.6) yields h = D h a i + ) a i + D T f 0 c 1 ɛ 1 + δ ) 1 3δ + δ + 1 3δ )δ )+1 3δ ) 1 3δ D T c 0 f 1.

8 76 Zhang R. and Li S. When is odd, 3, note D T 11 a 1 u 1, D T 1 D T D T 3 Obviously, 3+1)/ i=+ d +1)/ i= a i u i, D T 3 +1)/ i= a i u i, D T 13 i=3+3)/ )u j, D T 33 i=+3)/ a i u i, D T 31 d i=+3)/ a i u i, D T 1 a +1 u +1, d )u j. s 1j u j, D D T 11 h + D T 1 h + D T 13 h + D T 1 h + D T h + D T 3 h + D T 31 h + D T 3 h + D T 33 h. Let M 0 = DT 11 h + DT 1 h + DT 13 h, M 11 = DT 1 h + DT 13 h + DT 1 h + DT 31 h, M 1 = DT 11 h + DT 1 h + DT 31 h, M 1 = DT 11 h + DT 13 h + DT h + DT 3 h, M = DT 1 h + DT h + DT 3 h, M 31 = DT 11 h + DT 1 h + DT 3 h + DT 33 h, M 3 = DT 13 h + DT 3 h + DT 33 h. Then we derive an equality and Ah, ADM 0 = ADM 11 + ˆDM 11 ) ADM 1 + ˆDM 1 ) + ADM 1 + ˆDM 1 ) ADM + ˆDM ) + ADM 31 + ˆDM 31 ) ADM 3 + ˆDM 3 ) + ADM 0 + ˆDM 0 ). 4.7) Similar as in the even case, we derive that ADM 11 + ˆDM 11 ) ADM 1 + ˆDM 1 [ d ) ] [ 1 δ ) a i + a +1 + s 1,j 1 + δ ) a 1 + i= ADM 1 + ˆDM 1 ) ADM + ˆDM ) [ +1)/ 1 δ ) a 1 + a i + d ) ] a i + [ +1) 1 + δ ) i= i=+3)/ a i + +1)/ i= i= a i + d ) ] a +1 + ADM 31 + ˆDM 31 ) ADM 3 + ˆDM 3 ) [ +1)/ d ) ] 1 δ ) a i + a i + i=+3)/ d s 1,j ) ],

9 Optimal D-RIP Bounds in CS 763 [ 1 + δ ) i=+3)/ a i + i=+3)/ a i + d ) ]. It follows that the right-hand side of 4.7) can be bounded from below, [ d ) ] RHS 1 δ ) 3 a i + a +i + [ 1 + δ ) a i + [ = 1 δ ) 1 δ ) a i δ a +i + a +i + d d ) ] ) ] a i δ a i + D T f ) 0 c 1. We have already shown that LHS 4ɛ 1+δ a i. The following steps are the same with the case when is even. Both cases show that δ < 1/3 implies stable recovery. This completes the proof of Theorem 4.1. We can also improve the D-RIP condition δ for the stable recovery of the -sparse signals. Theorem 4. Suppose the sensing matrix A satisfies D-RIP with δ < / for some integer d 1. Then ˆf l f C 0 D f D f) [] 1 + C 1 ɛ, ˆf DS D f D f) [] 1 f C 0 + C λ, where C 0 = δ + δ δ ) +1 ), C δ ) 1 = 1+δ 1, C δ = 1. δ Proof Firstly, we have.4) with T = T 0 which means that the l 1 norm of DT h is bounded. 0 c Furthermore, from the definition of support T 0, we have the following relationship D T c 0 h D T 0 h 1 / D T 0 h 1 + D T c 0 f 1)/. We now apply Lemma. with α = DT 0 h 1 + DT f 1)/, thend 0 c T h can be expressed 0 c as convex combinations of sparse vectors: DT N 0 c λ iv i, where v i is -sparse, N λ i =1 with λ i 0, 1] for each i, and v i v i D T 0 h 1 + D T c 0 f 1)/ D T 0 h + D T c 0 f 1/. Denote γ i = D T 0 h + μv i,where0 μ 1. Then N λ j γ j 1 γ i = 1 D T 0 h + μdt h 1 0 c μv i ) 1 = μd h + μ DT 0 h 1 μv i, 4.8)

10 764 Zhang R. and Li S. and γ i, N λ jγ j 1 γ i μd h are all -sparse vectors. It is also straightforward to verify N N λ i M λ j γ j 1 ) γ i = 1 N λ i Mγ i 4, where M R q d, q is an arbitrary integer. Specially, we obtain N N λ i AD λ j γ j 1 ) γ N i + ˆD λ j γ j 1 ) ) γ i = 1 4 N λ i ADγ i + ˆDγ i ). 4.9) We now estimate the left-hand side of 4.9) by applying 4.8) N N λ i AD λ j γ j 1 ) γ N i + ˆD λ j γ j 1 ) ) γ i N [ ) 1 = λ i AD μ DT 0 h 1 ] μv i + μd h N [ ) 1 + λ i ˆD μ DT 0 h 1 ] μv i + μd h N ) 1 = λ i AD μ DT 0 h μ ) ) v i + ˆD 1 μ DT 0 h μ ) v i) + μ1 μ) ADD T 0 h, Ah. 4.10) Since 3 )DT h 1 v i are -sparse vectors for each i, we apply.1) to 4.10), thus it gives N N λ i AD λ j γ j 1 ) γ N i + ˆD λ j γ j 1 ) ) γ i 1 + δ ) N [ ) ] 1 λ i μ DT 0 h + μ 4 v i +1 μ)μ 1+δ D T 0 h ɛ. On the other side, 1 N λ i ADγ i + 4 ˆDγ N i ) 1 δ ) λ i 4 D T0 h + μ v i ). Combining the above two inequalities, and applying 4.8), we deduce that μ δ N λ i v i [ μ + μ)+ 1 ] μ + μ )δ DT 0 h 1 μ)μ 1+δ D T 0 h ɛ.

11 Optimal D-RIP Bounds in CS 765 Note that v i DT 0 h + η, whereη = DT f 1/,soweobtain 0 c [ μ + μ)+ 1 3 ] μ + μ )δ DT 0 h μ ηδ +ɛ1 μ)μ 1+δ ) D T 0 h μ δ η 0, 4.11) which is a second-order inequality for D T 0 h. Our aim is to maximize δ 0, 1) and at the same time, D T 0 h is bounded, so it forces that δ μ + μ 3 μ μ + 1. By simple calculation, δ can achieve its maximal value / whenμ = 1. Set μ = 1 and solve 4.11). Then we derive DT 0 h ηδ + ɛ 1+δ δ ) ηδ + ɛ 1+δ ) +δ δ )η + δ ) δ + δ δ ) η + 1+δ δ ) 1 ɛ. δ The following steps are similar with those in Theorem 4.1. We may now complete the proof by applying.3) to the above equation, and derive the upper bound of h, h = DT h 0 c + D T 0 h DT 0 h + D T 0 h + η) D T 0 h + η 1+δ 1 δ + ɛ + δ This completes the proof of Theorem 4.. δ δ ) δ ) ) D T c 0 +1 f 1. References [1] Aldroubi, A., Chen, X., Powell, A. M.: Perturbations of measurement matrices and dictionaries in compressed sensing. Appl. Comput. Harmon. Anal., 33, ) [] Blanchard, J. D., Cartis, C., Tanner J.: Compressed sensing: How sharp is the RIP? SIAM Rev., 53, ) [3] Baraniu, R., Davenport, M., DeVore, R., et al.: A simple proof of the restricted isometry property for random matrices. Constr. Approx., 8, ) [4] Cai, T., Wang, L., Xu, G.: Shifting inequality and recovery of sparse signals. IEEE Trans. Signal Process., 58, ) [5] Cai, T., Wang, L., Xu, G.: New bounds for restricted isometry constants. IEEE Trans. Inform. Theory, 56, ) [6] Cai, T., Xu, G., Zhang, J.: On recovery of sparse signals via l 1 minimization. IEEE Trans. Infrom. Theory, 55, ) [7] Cai, T., Zhang, A.: Sparse representation of a polytope and recovery of sparse signals and low-ran matrices. IEEE Trans. Inform. Theory, 60, )

12 766 Zhang R. and Li S. [8] Cai, T., Zhang, A.: Sharp RIP bound for sparse signal and low-ran matrix recovery. Appl. Comput. Harmon. Anal., 35, ) [9] Candès, E. J.: The restricted isometry property and its implications for compressed sensing. C. R. Math. Acad. Sci. Paris, Serie I, 346, ) [10] Candès, E. J., Eldar, Y. C., Needell, D.: Compressed sensing with coherent and redundant dictionaries. Appl. Comput. Harmon. Anal., 31, ) [11] Candès, E. J., Romberg, J., Tao, T.: Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theory, 5, ) [1] Candès, E. J., Tao, T.: Decoding by linear programming. IEEE Trans. Inform. Theory, 51, ) [13] Candès, E. J., Tao, T.: The Dantzig selector: Statistical estimation when p is much larger than n. Ann. Statist., 35, ) [14] Davies, M. E., Gribonval, R.: Restricted isometry properties where l p sparse recovery can fail for 0 <p 1. IEEE Trans. Inform. Theory, 55, ) [15] Donoho, D. L.: Compressed sensing. IEEE Trans. Inform. Theory, 5, ) [16] Donoho, D. L.: For most large underdetermined systems of linear equations the minimal l 1 solution is also the sparsest solution. Comm. Pure Appl. Math., 59, ) [17] Foucart, S., Lai, M. J.: Sparsest solutions of underdetermined linear systems via l q minimization for 0 <q 1. Appl. Comput. Harmon. Anal., 6, ) [18] Li, S., Lin, J.: Compressed sensing with coherent tight frames via l q -minimization for 0 <q 1. Inverse Probl. Imaging, 8, ) [19] Lin, J., Li, S., Shen, Y.: New bounds for restricted isometry constants with coherent tight frames. IEEE Trans. Signal Process., 61, ) [0] Liu, Y., Mi, T., Li, S.: Compressed sensing with general frames via optimal-dual-based l 1 -analysis. IEEE Trans. Inform. Theory, 58, ) [1] Mo, Q., Li, S.: New bounds on the restricted isometry constant δ. Appl. Comput. Harmon. Anal., 31, ) [] Rauhut, H., Schnass, K., Vandergheynst, P.: Compressed sensing and redundant dictionaries. IEEE Trans. Inform. Theory, 54, ) [3] Saab, R., Yılmaz, Ö.: Sparse recovery by non-convex optimization-instance optimality. Appl. Comput. Harmon. Anal., 9, )