ONE of the central aims of compressed sensing is to efficiently recover sparse signals from underdetermined

Transcription

1 Stable Sparse Reovery with Three Unonstrained Analysis Based Approahes Huanmin Ge, Jinming Wen, Wengu Chen, Jian Weng and Ming-Jun Lai Abstrat Effiient reovery of sparse signals from underdetermined linear systems has reeived lots of attentions in reent deade. This paper onsiders the stable reovery of a signal x, whih is not -sparse itself but is -sparse in terms of a tight frame D, from the observations y = Ax + e. Three unonstrained minimization approahes will be studied. We show that if the sensing matrix A satisfies the restrited isometry property adapted to a tight t frame D with δ t < t+8 for any t >, then these three approahes an stably reover x. As a onsequene, when t = and 3, our result signifiantly improves existing best suffiient onditions in terms of δ and δ 3. Moreover, we theoretially haraterize the reovery error of these methods. Index Terms Compressed sensing, unonstrained analysis based approahes, restrited isometry property, signal reovery, tight frame. I. INTRODUCTION ONE of the entral aims of ompressed sensing is to effiiently reover sparse signals from underdetermined linear systems. Mathematially, it reasts to reover an unnown sparse signal x R n from the following linear measurements: y = Ax + e, ( where y R m is a measurement vetor, A R m n (m n is a nown sensing matrix and e R m is a vetor of measurement errors. If the signal x is -sparse, i.e., it has at most nonzero entries, then under some suitable onditions on the sensing matrix A, x an be stably reovered via some effiient sparse reovery algorithms, see, e.g., [] [7]. The theory for the sparse reovery has been extended to the setting of sparse ditionary reovery motivated by numerous pratial appliations, e.g. [8], [9]. That is, x in ( is no longer sparse itself, but it is sparse in terms of an overomplete ditionary D R n d with d >> n, i.e. the signal x is expressed as x = Dv, where v R d is sparse. Many sparse reovery results for ( have been extended to the sparse ditionary reovery setting. See, e.g., [0] [3] and referenes therein. In this paper, we assume D R n d (d n is a tight frame, i.e., D i = for i n, and for any w R n, it holds that w = d i= (w D i D i, where D i is the i-th olumn of D. We refer to [] for basi theory on tight frame. We say x is sparse in terms of an overomplete ditionary if x = Dv and v R d is sparse. One of the most ommonly used framewors to reovery sparse ditionary is the restrited isometry property adapted to D (D-RIP [] whih is defined as follows: Definition. A matrix A R m n is said to obey the restrited isometry property adapted to D R n d (abbreviated D-RIP of order with onstant δ if ( δ Dv ADv ( + δ Dv ( H. Ge is with Graduate Shool, China Aademy of Engineering Physis, Beijing, 00088, China ( gehuanmin@63.om. J. Wen is with the Edward S. Rogers Sr. Department of Eletrial and Computer Engineering, University of Toronto, Toronto, M5S 3G, Canada ( jinming.wen@utoronto.a. W. Chen is with Institute of Applied Physis and Computational Mathematis, Beijing, 00088, China ( henwg@iapm.a.n. J. Weng is with the Shool of Information Tehnology, Jinan University, Guangzhou 5063, China ( ryptjweng@gmail.om. M. Lai is with Department of Mathematis, University of Georgia, Athens, GA 3060, U.S.A., ( mjlai@uga.edu.

2 holds for all -sparse vetors v R d. The smallest onstant δ is alled as the the restrited isometry onstant adapted to D (abbreviated D-RIC. From Definition, one an easily see that the D-RIP redues to the standard RIP when D is the n n identity matrix. Thus, the D-RIP is an extension of the restrited isometry property (RIP. Similar to the definition of the standard RIP of order in [5], we define δ as δ when is not an integer, where is an integer satisfying < < +. In the rest of the paper, we use δ to denote the D-RIC of A with order of. The following onstrained analysis basis pursuit has been proposed in [] to reonstrut the signal x from (, where e satisfies e ε for a positive onstant ε: min D x subjet to A x y ε. (3 x R n This method has been well studied and a number of suffiient onditions of stably reovering x have been proposed. These inlude δ < 0.08 [], δ < 0. [6], δ < 0.7 [3], δ < 0.93 [7] and δ < [8]. In addition, the following unonstrained analysis based approah, whih is widely used in image proessing and omputer vision [9] [3], has been onsidered: min x R n λ D x + A x y, ( where s a positive parameter. The unonstrained analysis approah ( is equivalent to the analysisbased method (3 in the sense that for any ε > 0, there exists a λ suh that the solution of ( is that of (3, and vie versa. It is well nown that sine ( is an unonstrained minimization, the omputation of ( is more onvenient than that for (3. In addition, if the noise level is not given or annot be aurately estimated, we use ( instead of (3. See [3] for the more detailed relation between (3 and (. An overview of ( an be found in [33]. It has been shown in [3] and [35] that δ 3 < and δ < 0.907, are suffiient for the stable reovery of the signal x via the approah ( if D A e λ. (5 The best existing suffiient ondition, whih is based on δ for stably reovering x via ( is δ < 0. that was proposed by Shen et al. in [36]. Many effiient algorithms have been proposed to solve the unonstrained analysis problem (, see, e.g., [35], [37] [39]. In partiular, Zhao et al. [35] onsidered the monotone version of the fast iterative shrinage-thresholding algorithm to solve the minimization problem (. But the proximal operator of this algorithm for D x does not have a losed-form solution, so based on smoothing and deomposition transformations, the following general nonsmooth onvex optimization problem was introdued to reover x in [35]: min x R n,z R d λ z + A x y + ρ z D x. (6 More detailed disussions on the relation between (6 and ( an be found in [35]. It has been respetively shown in [35] and [36] that δ < and δ < 0. is a suffiient ondition of stably reovering the signal x from ( via the method (6 when e satisfies (5. Although ( and (6 an stably reover x from ( when A satisfies the D-RIP with ertain D-RIC if e satisfies (5, they may not be able to stably reover x based on y and A if the measurements y R m are orrupted by both the bounded noise e R m and the sparse noise e R m in term of ertain tight frame D R m d (i.e., D e (D e max (s = 0 for some integer s [0], that is y = Ax + e + e. (7

3 3 In order to reover signals x in (7, the following separation unonstrained analysis has been introdued in [3]: where min λ W ν + ν R n+m Φ ν y, (8 Φ = [ [ ] [ ] ] x D 0 A I m, ν =, W =. (9 e 0 D It has been shown that the onditions δ 3(+s < [3] and δ (+s < 0. [36] an guarantee the stable reovery of the signal x in (7 via the approah (8 if e satisfies W Φ e λ, (0 where W is defined in (9. Sine there are effiient algorithms for solving (, (6 and (8, we shall fous on theoretially investigating suffiient onditions of stable reovering x with these three approahes in this paper. From the above introdution, we an see that suffiient onditions an be haraterized by using δ and δ 3. Sine it is usually diffiult to he whih ondition is better than the other one for any two onditions whih are based on two different orders of D-RIC of A, our paper will use δ t (t > to haraterize new suffiient onditions. One of the main advantages is that the proposed onditions are valid for any order of D-RIP with t >, and hene the new onditions are more general. The main ontributions of this paper are summarized as follows: We will show that if A in ( satisfies the D-RIP of order t with t δ t <, t >, ( t + 8 then signals x in ( an be stably reovered via solving ( or (6 if e satisfies (5 (see Theorem and Theorem 3 below. It is not hard to see that, when t =, ( signifiantly improves the best existing δ -based suffiient ondition whih is δ < 0. given by [36, Theorem 3] for ( and [36, Theorem 5] for (6. Moreover, ( with t = 3 greatly improves the best existing suffiient ondition based on δ 3, whih is δ 3 < given by [3, Theorem 3.] for (. We will show that if Φ (see (9 satisfies the D-RIP with t δ t(+s < t + 8, t > ( and the noise vetor e in (7 obeys (0, then signals x in (7 an be stably reovered via solving (8 (see Theorem below. It an be easily he that ( with t = and t = 3 signifiantly improves existing best suffiient onditions whih are δ (+s < 0. given by [36, Theorem 6] and δ 3(+s < given by [3, Theorem.], respetively. In order to present our results, we need some notation and definitions. Notation: For a positive integer d, [d] denotes the index set {,,, d}. For S [d], let S [d] be the omplement of S and S be the number of elements in S. For a vetor x R d, define the l p ( p norm of x by x p = ( d i= x i p p with p < and x = max i [d] x i. The norm x 0 denotes the number of nonzero entries of x. The support of x is defined by supp(x = {i [d] : x i 0}. For a set S [d], denote D S as D with all but the olumns indexed by S set to zero vetor. Denote x S as x with all but the entries indexed by S set to zero. Let x max( as x with all but the largest entries in absolute value set to zero, and x max( = x x max(. The rest of the paper is organized as follows. In Setion II, we first introdue some tehnial lemmas. We present our main results in Setion III and show them in Setion IV. Finally, we summarize the results in this paper in Setion V.

4 II. TECHNICAL LEMMAS To prove our main results, let us start with some useful tehnial lemmas. We will use the following results established in [0, Lemma.]. Lemma. For a positive number α and a positive integer s, define the polytope T (α, s R d by T (α, s = {v R d : v α, v sα}. For any v R d, define the set of sparse vetors U(α, s, v R d by U(α, s, v = {u R d : supp(u supp(v, u 0 s, u = v, u α}. (3 Then any v T (α, s an be expressed as v = u i for some positive integer N, where u i U(α, s, v and i= = with 0. ( i= Next we need a null spae property. That is, the matrix A has the l -robust null spae property of order if it satisfies the D-RIP. See [, Definition 3]. Lemma. Let D R n d satisfy DD = I and A R m n satisfy the D-RIP of order t (t > with δ t (0,. Let T [d] be an index set with T. Then for any vetor h R n, it holds that D T h β Ah + β D T h, (5 where β = ( δ t + δ t, β = δ t ( δ t (t. (6 Proof. This is a ritial results. We inlude a proof in this paper. See Appendix A. In fat, Lemma is a nontrivial improvement of [36, Lemma ] as the proof is based on Lemma. More preisely, [36, Lemma ] indiates that if A R m n satisfies the D-RIP of order with δ t (0, 0.5, then where D T h β Ah + β D T h, β = + δ δ, β = δ δ. (7 From Lemma, one an see that (5 holds if A satisfy the D-RIP of order t for any t >, thus it is an extension of [36, Lemma ]. In the following, we will show that although β β when 0 < δ 3, β < β holds for all δ (0, 0.5 when t =. Thus, Lemma improves [36, Lemma ] when t =. By (6, when t = β = ( δ + δ, β = δ δ. (8

5 5 Fig.. The rates β /β and β /β versus δ show that β is always smaller than β and β is smaller than β when δ is larger enough. In the following, we ompare β with β and β with β. By (7, (8 and the fat that δ (0, 0.5, it is not hard to show that 0 < β = β ( δ δ = ( δ <, + δ + δ δ δ 0 < β = δ β δ δ = δ <. δ To learly see the relationships between β and β, and between β and β, we show them in Figure. From Figure one an see that although β /β > when 0 < δ 3, β /β < holds for all δ (0, 0.5. To show our main results in the next setion, the following lemma is also neessary. Lemma 3. Let D R n d satisfy DD = I n and S, T [d] be two index sets with S s and T. Suppose that for any given vetors x R n and h R n, it holds that D S h a D S h + b D T x + β, (9 where onstants a, b and β satisfy a > 0, b 0 and β 0. If the matrix A R m n satisfies the D-RIP with δ s+ < (0 + a s for ertain onstant > 0, then DS h [ (asδs+ + δ s+ s (b DT x + β + ( + a s s( + δ s+ Ah ], ( where Proof. Please see Appendix B. ( = ( + a s + a s δ s+. ( We remar that to the best of the authors nowledge, suh ind of result has not been reported in the literature before. III. MAIN RESULTS In this setion, we will show that if A in ( satisfy the D-RIP of order t with (, then signals x in ( an be stably reovered via solving ( or (6 if e satisfies (5. We will also prove that if Φ (see (9 satisfies the D-RIP with (, then signals x in (7 an be stably reovered via solving (8 if the noise vetor e in (7 obeys (0. These suffiient onditions signifiantly improves existing ones. In addition to give the suffiient onditions, upper bounds on the reovery errors of these three methods will also be presented in this setion.

6 6 A. A suffiient ondition of stably reovering x with solving ( This subsetion fouses on investigating the reovery performane of (, and our main results are stated as in Theorem below whih not only provides a suffiient ondition of stably reovering x based on solving (, but also haraterizes the reover errors of this method. Theorem. Let D R n d be a tight frame and A in ( satisfy the D-RIP of order t with (. Suppose that e in ( obeys (5. Then the solution ˆx of ( satisfies A(ˆx x C λ + C (D x max(, (3 ˆx x C 3 λ + C (D x max(, ( where the onstants C C are given preisely as follows: 6 C = ( δ t, (5 + δ t Proof. Please see Setion IV-A. C = ( δ t + δ t, (6 3 t C 3 = ( δ t (, (7 t t + 8 t+8 t ( δ ( t (t+8(t t + δ 3 t + δ t (t + 8 t+8 t C = ( t (t + 8 t+8 t +. (8 In the noise-free ase, i.e., when e = 0, we an hoose a λ whih is lose to 0 suh that C 3 s lose to 0. Then Theorem indiates that x an be approximately estimated by solving ( if (D x max( is small. Furthermore, from Theorem one an see that x an be aurately reovered by solving ( when D x is -sparse and e = 0. To ompare our suffiient ondition for stably reovery of signals x via solving ( with existing ones, we give the following result for t = whih an be easily obtained from Theorem. Corollary. Let D R n d be a tight frame and A in ( satisfy 0 δ < 0. (9 If e in ( satisfies (5, then the solution ˆx of ( satisfies A(ˆx x C λ + C (D x max(, and ˆx x C 3 λ + C (D x max(, where 6 C = ( δ, C = ( δ + δ, (30 + δ 3 C 3 = ( δ ( 0δ, (3 ( 0( δ 0 + δ 3 + δ 0δ C = +. (3 0 0δ

7 7 (a C /C and C /C (b C 3 /C 3 and C /C Fig.. The rates C /C, C/C, C3/C 3 and C /C versus δ whih shows that these oeffiients are similar In the following, we ompare our new suffiient ondition of stably reovery of x via solving ( with existing ones. To save spae, instead of omparing our suffiient ondition with all of those in [3] [36], we only ompare it with the best existing one ( [36, Theorem 3] whih is stated as follows: Theorem. Let D R n d be a tight frame and A in ( satisfy If e in ( satisfies (5, then the solution ˆx of ( satisfies δ < 0.. (33 A(ˆx x C λ + C (D x max( and ˆx x C 3 λ + C (D x max(, where C = 3 + δ δ, C = ( δ 3 + δ, (3 C 3 = 3( + δ (7 δ ( δ ( 5δ, (35 6 6δ C = 5( 5δ ( δ (36 Sine 0/0 0.36, by (9 and (33, one an easily see that our new D-RIP ondition on A is less restritive than that given by (33. To learly see the upper bounds on the reovery error, we draw Figure to show C /C, C /C, C3 /C 3 and C /C. From Figure (a, we an see that although C /C > for 0 δ < 0., C /C < for 0 δ < 0.. Figure (b shows that both C 3 /C 3 < and C /C < hold when 0.05 δ < 0.. Sine C is always larger than C when 0 δ < 0., how to improve the upper bound on A(ˆx x by dereasing C will be investigated in the future. B. A suffiient ondition of stably reovering x with solving (6 The reovery performane of (6 for reovering signals x an be haraterized by the following theorem. Theorem 3. Let D R n d be a tight frame and the matrix A in ( satisfy (. If v in ( obeys (5, then the solution ˆx of (6 satisfies A(ˆx x C λ + C (D x max( + C λd ρ, (37

8 8 (D x max( ˆx x C 3 λ + C + C λd, (38 ρ where d is the number of olumns of D and C, C, C 3 and C are defined in (5-(8. Proof. Please see Setion IV-B. Clearly, if t =, our suffiient ondition given by Theorem 3 redues to δ < for the method (6. This suffiient ondition is less restritive than the best existing suffiient ondition whih is δ < 0. that was given by [36, Theorem 5]. In the following, we loo into the upper bound on the reovery error of method (6. By [36, Theorem 5], we have (D x max( ˆx x C 3 λ + C + C λd ρ if A in ( satisfies the D-RIP with δ < 0., where the onstants C 3 and C are defined in (35 and (36. Our bound on the estimation error of the method (6 is given by (38, where C 3 and C are respetively defined in (3 and (3. By Figure (b, one an see that both C 3 and C are less than C 3 C when δ (0.05, 0., thus our bound on the estimation error is sharper than that given by [36, Theorem 5] in this ase. C. A suffiient ondition of stably reovering x with solving (8 In this subsetion, we analyze the reovery performane of (8 for reovering signals x from (7. Speifially, we have the following result. Theorem. Let frames D R n d and D R m d be tight frames suh that D x is sparse and D e (D e s = 0. (39 [ ˆx Suppose that Φ (see (9 satisfies the D-RIP with ( and e in (7 obeys (0. Then the solution ˆν = ê] of (8 satisfies Φ(ˆν ν C 5 + sλ + C6 (D x max(, (0 ˆx x C 7 + sλ + C8 (D x max( + s, ( where 6 C 5 = ( δ t(+s, ( + δ t(+s C 6 = ( δ t(+s + δ t(+s, (3 3 (t C 7 = ( δ t(+s (, ( t t + 8 δ t+8 t(+s ( δ ( (t+8(t t(+s t + δ 3 t(+s + δ t(+s (t + 8 δ t+8 t(+s C 8 = ( +. (5 t (t + 8 δ t+8 t(+s Proof. Please see Setion IV-C.

9 9 Obviously, if t =, our suffiient ondition given by Theorem is δ (+s < for the method (8. This suffiient ondition is less restritive than the best existing one whih is δ (+s < 0. that was given by [36, Theorem 6]. In the following, we loo into the upper bound on the estimation error of method (8. By (, our bound on the estimation error of the method (8 is ˆx x C 7 + sλ + C8 (D x max( + s, where C 7 = ( δ (+s ( 0δ (+s, ( 0( δ(+s 0 + δ 3 (+s + δ (+s 0δ(+s C 8 = δ(+s From [36, Theorem 6], it follows that ˆx x C 7 + sλ + C8 (D x max( + s where C 7 = 3( + δ (+s(7 δ (+s ( δ (+s ( 5δ (+s and C 8 = 6 6δ (+s 5( 5δ (+s ( δ (+s Notie that the ratios C 7 and C 8 versus δ C 7 C (+s (0, 0. are the same as C 3 and C versus δ 8 C 3 C (0, 0.. Therefore, by Figure (b, one an see that both C 7 and C 8 are less than when δ C 7 C (+s (0.05, 0., 8 whih means that our bound on the estimation error is sharper than that given by [36, Theorem 5] in this ase. IV. PROOFS In this setion, we prove our main results given in Setion III. A. Proof of Theorem Proof. To simplify notation, denote T = supp((d x max (, (6 then by the definition of (D x max (, we obtain T. Furthermore, denote then by [36, eq. (3.], we have Therefore, (a h = ˆx x, (7 Ah 3λ D T h + λ D T x λ D T h. (8 Ah 3λ DT h + λ DT x λ DT h (b 3λ ( DT β Ah + β h + λ DT x λ DT h =3λ β Ah + λ D T x λ( 3β D T h

10 0 ( 3λ β Ah + λ D T x, (9 where β and β are defined in (6, (a is from the Cauhy-Shwartz inequality and the fat that T, (b follows from (5 and ( is beause 3β > 0. Indeed, by ( and the seond equality in (6, we have By (9, we obtain 3δ 3 t 3β = ( δ t (t > = t t+8 Ah 3λ β + 9β λ + 6λ D T x t t+8 ( t t+8 ( t 9 t + 8 = 0. t + 8 3λ β + (t ( 3 β λ + 8 D T x 3 β = 3λ β + D T x 3 β = 3λ β + (D x max ( 3 β, (50 where the last equality is from (6, and the definitions of (D x max ( and (D x max (. Then, by the first equality in (6, (5, (6, (7 and (50, one an see that (3 holds. Sine D is a tight frame, by [, Chapter 3], we have DD T = I. Then h = D h = DS h + DS h, (5 where we denote S = supp((d h max (. (5 By (6, (5, and the definitions of (D h max ( and (D x max (, one an see that Thus D S h D T h. (53 D S h = D h D S h D h D T h = D T h 3 D T h + D T x 3 D S h + D T x, (5 where the seond to the last inequality is from (8 and the last inequality is due to (53. We show ( by giving an upper bound on h. Towards this aim, by (5, we give upper bounds on DS h and DS h. We first give an upper bound on DS h. Speifially, we have DS h DS h DS h (a (3 DS h + DT x D S h (b (3 D S h + D T x D S h = 3 D S h + D S x D T h, (55 where (a follows from (5 and D S h D S h

11 whih is based on the fats that the absolute values of the entries of DS h are not smaller than those of DS h and S, and (b is from the Cauhy-Shwarz inequality. In the following, we give an upper bound on DS h by using Lemma 3 with assuming a = 3, b =, β = 0, = (t, s =. By (6 and (5, we have T and S s. Moreover, by ( and (5, one an he that (9 and (0 hold. Then by (, we obtain ( (t ( = ((t + 9 (t + 9 δ t t = (t + 8 t + 8 δ t. (56 Therefore, aording to (, we obtain DS h [( δ t + D δ t T x + ] (t (t + 8( + δ t Ah [( δ t + δ t + ( δ t (t + 8(t D T x (t + 8(t ] λ, (57 δ t where is defined in (56, and the seond inequality follows from (3, (6 and (7. Then, we obtain (a( h DS h + 3 DS h + D S h DT x DS h + D T x (b [( δ t + δ t + ( δ t (t + 8(t 3 D + T x + (t + 8(t ] λ, (58 δ t where (a is from (5 and (55, and (b follows from (57. By (7, (8, (7, (56 and (58, we an see that ( holds. B. Proof of Theorem 3 Proof. Let T and h be defined as in (6 and (7, then T and by the proof of [35, Lemma IV.], one has Ah + λ D T h 3λ D T h + λ D T x + λ d ρ. (59 Hene, using the tehniques for deriving (9, we obtain Ah 3λ β Ah + λ D T x + λ d ρ, (60 where β is defined in (6. From the above inequality, it follows that (a A(ˆx x = Ah (b 3λ β + 3λ β + 9β λ + 6λ D T x + λ d ρ ( 3 β λ + 8 D T x + λd ρ 3 β

12 =3λ β + ( 3 DT β x + λd ρ ( =3λ β + ( 3 (D x max ( + λd β ρ (d 6 = ( δ t ( δ t + δt λ + (D x max( + δ t 3 + ( δ t + δ t λd 6 ρ (e (D x max( =C λ + C + λd, C ρ where (a, (b and (d follows respetively from (7, (60 and the first equality in (6, ( is due to (6 and the definitions of (D x max ( and (D x max (, and (e is from (5, (6. Thus, (37 is proved. Let S be defined as in (5, then S. Moreover, (5, (53 and (5 hold. Then, using the tehniques for deriving (5 and (59, one an obtain D S h 3 D S h + D T x + λd ρ. (6 Similar to the proof of (, in order to show (38, we first give an upper bound on DS h. Speifially, by the tehniques for deriving (55 and (6, one obtains ( DS h 3 DS h + DT x + λd D S h. (6 ρ In the following, we give an upper bound on DS h by applying Lemma 3 with a = 3, b =, β = λd ρ and = (t. By (6 and (5, we have T and S s. Moreover, by ( and (6, one also hes that (9 and (0 hold. Sine in ( is only depend on a, b and s, (56 holds. Hene, from (, it follows that DS h [ (δ t + δ t D T x + (6δ t + δ t λd ρ + ] (t (t + 8( + δ t Ah [( δ t + δ t + ( δ t (t + 8(t D T x 3 ( + 6δ t + δ t + ( δ t (t + 8(t λd 3 ρ + 6 (t + 8(t ] λ, (63 δ t where is defined in (56, and the last inequality is from (37, (7 and (6. Then, one obtains that (a[ ( h DS h + 3 DS h + DT x + λd D S h ] ρ ( DS h + DT x + λd ρ (b [ ( + δ t + δ t + ( δ t (t + 8(t D T x 3 ( + 6δ t + δ t + ( δ t (t + 8(t + (t + 8(t λ + ( δ t 3 λd ] ρ

13 ( D =C 3 λ + T x C + C λd, ρ where (a is from (5 and (6, (b is due to (63 and ( follows from (56, (7 and (8. Therefore, by the above inequality, (7, (5 and the definitions of (D x max ( and (D x max (, (37 holds. We omplete the proof of the theorem. 3 C. Proof of Theorem [ [ [ x x ˆx Proof. Sine ν =, ν =, ˆν =, W and Φ = [ ] A I e] ẽ] ê] m in (8 an be respetively viewed as x, x, ˆx, D and A in (, this theorem an be proved by utilizing Theorem. Speifially, by ( and (0, ( and (5 hold, then aording to Theorem, we obtain ( [ ] x (a W e max (+s Φ(ˆν ν C 5 + sλ + C6 + s ( [ (b D 0 ] [ ] x 0 D e =C 5 + sλ + C6 + s [ ] D x D e =C 5 + sλ + C6 C 5 + sλ + C6 max (+s [ + s ] (D x max ( (D e max (s + s ( =C 5 + sλ + C6 (D x max ( + s, max (+s where the onstants C 5 and C 6 are respetively defined in ( and (3, (a is from (3, (b follows from (9 and ( is due to (39. Furthermore, by using the tehniques for the above inequality and (, we an show that ( [ ] x W e max (+s (D ˆν ν C 7 + sλ + C8 = C 7 + sλ + x max ( C8, (6 + s + s where the onstants C 7 and C 8 are respetively defined in ( and (5. Sine (6 and [ [ ] ˆx x ˆν ν = ˆx x ê], e ( holds. We omplete the proof of the theorem. V. CONCLUSION In this paper, we have studied suffiient onditions for stable reovery of a signal x, whih is not sparse itself but it is sparse in terms of a tight frame, from noisy measurements by three unonstrained t analysis based approahes. We have proved that if the sensing matrix A satisfies δ t < with t >, t+8 then any signal x an be stably reovered from y = Ax + e bysolving ( or (6 under the noise level D A e λ. In addition, we have also proved that δ t t < is a suffiient ondition of stable t+8 reovering any signal x from y = Ax + e + e by solving (8 under some onstrains on the noises e and e.

14 APPENDIX A PROOF OF LEMMA Proof. We first show that (5 holds when t is an integer. For a given t >, define { T = i T (D T h i > D T h }, (65 (t { T = i T (D T h i D T h }. (66 (t Then learly T = T T whih implies that D h = D T h + D T h = D T h + D T h + D T h. (67 Furthermore, by (65, T T = whih implies that D T h D T T h. Thus, to show (5, it suffies to show that D T T h β Ah + β D T h, (68 where β and β are defined in (6. Let us use Lemma to prove (68. We first show that T < (t. (69 If T =, then T = 0 and hene (69 holds. Otherwise, by (65, one yields that DT h > T D T h (t = T D T T h (t T D T h (t. Thus a qui simplifiation of the above inequality yields the (69. By (65 and (66, T = T T and T T =. The above inequality implies DT h = DT h DT h DT h T D T h (t = ((t T D T h (t and we use (66 to have D T h D T h (t. Sine we assumed t is an integer, (t T is a positive integer. Then by Lemma with one an see that D T h an be expressed as α = D T h (t, s = (t T, v = D T h, D T h = for ertain positive integer N, where i N, satisfies ( and ( D u i U T h (t, (t T, DT h u i (70 with the set U being defined in (3. Moreover, by (3, one an see that ( D u i u i 0 u i ((t T T h D T h (t (t. (7 i=

15 5 To simplify notation, for i N, we define v i =( + δ t D T T h + δ t u i, (7 v i =( δ t D T T h δ t u i. (73 By (3, u i 0 (t T whih ombing with (69 and the assumption that T, we an see that both v i and v i are t-sparse for i N. Moreover, by (7 and (73, we have ( ADv i AD v i = i= = ( ADDT T h + δ t AD(u i + DT T h ADDT T h δ t AD(u i + DT T h i= i= (δ t (ADDT T h AD(u i + DT T h (a =δ t (ADD T T h AD( u i + DT T h i= (b =δ t (ADD T T h ADD h ( =δ t (ADD T T h Ah (d δ t + δt DD T T h Ah (e δ t + δt D T T h Ah, (7 where (a follows from (, (b is beause of (67 and (70, ( is from the assumption that DD = I, (d follows from (, (69 and the Cauhy-Shwarz inequality and (e is from the fat that the maximal singular value of D is not larger than whih an be obtained from DD = I. In the following, we give a lower bound on the left-hand side of (7 whih ombing with (7 an show (68. Toward this goal, we assume D R (d n d is the orthonormal omplement of D. Then DD = 0, v = Dv + Dv, (75 for any v R d. Moreover, by (, for any -sparse signal v R d, we have ( δ v ADv + Dv ( + δ v. (76 Sine DD = 0 (see (75, using the tehniques for deriving (7, one an obtain ( Dv i D v i = δ t ( DD T T h DD h = 0. i= Then, ( ADv i AD v i = i= (a (b = ( ADv i + Dv i i= ( AD v i + D v i i= (( δ t v i ( + δ t v i i= (δ t ( δt D T T h δt u 3 i i=

16 6 ( =δ t ( δ t D T T h δ 3 t ( u i i= (d δ t ( δt D T T h δt 3 DT h (t, where (a is from (76 and the fat that both v i and v i are t-sparse for i N (see the first sentene under (73, (b follows from (7 and (73, ( is beause of ( and (d is from ( and (7. Then, by (7, we have whih is equivalent to δ t ( δt D T T h δt 3 DT h (t δ t + δt DT T h Ah, ( δ t D T T h ( + δ t Ah D T T h δ t (t D T h 0. Therefore, [ DT T h ( + δ t Ah + ( + δ t Ah + δt( δt D T h ] (t ( δt δ t ( δ t (t D T h + ( δ t + δ t Ah, where the last inequality is based on the fat that f + g f + g for f, g 0. Then, by (6, (68 holds whih implies (5 holds. If t is not an integer, we define t = t, then t is an integer and δ t = δ t <. From the above proof and Definition, one an see that (5 holds in this ase, thus it holds no matter whether t is an integer or not. APPENDIX B PROOF OF LEMMA 3 Proof. To simplify notation, the right-hand side of (9 is denoted by η, i.e., η = a D S h + b D T x + β. (77 We first show that ( holds when > 0 is an integer. Define the index sets { S = i S : DS h(i > η }, (78 { S = i S : DS h(i η }. (79 Then it is easy to see whih imply S S = and S = S S, (80 D h = D S h + D S h = D S S h + D S h. (8 Moreover, by the definition of S, S S = whih implies that DS h DS S h. Thus to show (, it is suffies to show that [ DS S (asδs+ + δ s+ s (b DT h x + β + ] ( + a s s( + δ s+ Ah, (8

17 7 where is defined in (. In order to use Lemma to show (8, we first prove that S <. (83 If S =, then S = 0 and hene (83 holds. Otherwise, from (78, it follows that DS (a h > S η (b S D S h ( = S D S S h (d S D S h, where (a is from (78, (b is due to the assumption (9 and (77, ( and (d follow from (80. Thus (83 holds. By (79 and (80, we have and D S h η DS h = DS h DS h η η S = ( S η, where the inequality is from (9, (77 and (78. Using Lemma with α = η, s = S (note that this s is different from the s whih gives an upper bound on S as assumed in this lemma and v = DS h, then DS h an be expressed as D S h = for ertain positive integer N, where satisfies ( and ( η u i U, S, DS h u i (8 for i N. Moreover, for i N, we have u i u i 0 u i (a ( η η ( S (b = [ ] a DS h + (b DT x + β + a DS h (b DT x + β ( [ a s DS h + (b DT x + β + a ] s DS h (b DT x + β (d [ a s DS S h + (b DT x + β + a ] s DS S h (b DT x + β (e = ( a sx + sp + asxp, (86 where (a is from (3 and (85, (b follows from (77, ( is from the Cauhy-Shwarz inequality, (d is due to the fat that DS h DS S h whih an be seen from (80 and (e is beause we denote i= (85 X = D S S h, P = b D T x + β s. (87 To simplify notation, define the vetors γ i = DS S h + µu i, i N, (88 γ i = ( µd S S h µu i, i N, (89 ω = ( µds S h + µd h, (90

18 8 where the onstant µ = +a s. Thus a s ( µ µ = µ ( = µ µ ( = µ and a s( + a s + a s a s + a s [( µ = µ ] [( µ = µ + a s ] Then, by (88-(90, one has the following identities Moreover, ω γ i = ( µd S S h + µd h (D S S h + µu i γ i = i= = µ + a s (9 = µ a s. (9 = ( µd S S h µu i + µd h = γ i + µd h. (93 (DS S h + µu i = DS S h + µds h i= = D S S h + µ(d h D S S h = ω, (9 where the first equality is from (88, the seond equality is from (8 and N i= = in (, and the last two equalities are respetively from (8 and (90. By (3 and (85, u i 0 S whih ombing with (83 and the assumption that S s, one an see that γ i and γ i are (s + -sparse for i N. Thus i= (a = AD γ i, µadd h i= AD[( µd S S h µu i], µadd h i= (b =( µµ ADD S S h, Ah µ AD( u i, Ah i= ( =( µµ ADD S S h, Ah µ ADD S h, Ah (d =( µµ ADD S S h, Ah µ AD(D h D S S h, Ah =( µµ ADD S S h, Ah µ ( Ah ADD S S h, Ah =( µµ ADD S S h, Ah µ Ah, (95 where (a is from (89, (b is due to N i= = in ( and the assumption that DD T = I, ( and (d follow from (8 and (8, respetively. Let D R (d n d be the orthogonal ompliment of D, then ( AD (ω γ i + D ( ω γ i (a = ( AD( γ i + µd h + D( γ i + µd h i=

19 (b = ( AD γ i + D γ i + ( µ Ah + µ( µ ADDS S h, Ah µ Ah i= ( ( + δ s+ (d =( + δ s+ i= γ i + µ( µ + δ s+ DS S h Ah i= ( µd S S h µu i + µ( µ + δ s+ DS S h Ah i= (e =( + δ s+ (( µ D S S h + µ (f =( + δ s+ (( µ X + µ i= u i + µ( µ + δ s+ DS S h Ah i= u i + µ( µ + δ s+ Ah X, (96 i= where we use (93 in (a, (b is from (95 and the first equality in (75, ( is due to that γ i is (s + - sparse, the Cauhy-Shwarz inequality and (76, (d follows from (89, (e is from N i= = in (, (3, (80, (8 and (85 and (f follows from (87. Moreover, we have λ ( i ADγ i + Dγ i λ ( i = AD(DS S h + µu i + D(D S S h + µu i i= ( δ s+ = δ s+ = δ s+ i= D S S h + µu i N ( D S S h + µ u i i= N (X + µ u i, (97 where the inequality is from the fat that γ i is (s + -sparse and (76, the seond equality follows from (, (3, (80 and (85 and the last equality is by (87. By (9, we an get the following equations = ( AD (ω γ i ADγ i i= i= ( ADω ADω, ADγ i = ADω ADω, AD( where the last two equalities are from N i= = in ( and (9, respetively. Furthermore, by using the tehniques for deriving (98, we an show that By (97-(99, one has i= i= ( AD (ω γ i + D ( ω γ i i= 9 γ i = 0, (98 ( ( D ω γ i Dγ i = 0. (99 δ s+ i= N (X + µ u i. (00 i=

20 0 Therefore, one obtains 0 (a ( + δ s+ (( µ X + µ δ s+ u i + µ( µ + δ s+ Ah X i= N (X + µ u i i= = ((µ µ + ( µ + µ δ s+ X + µ( µ + δ s+ Ah X + µ δ s+ (b( (µ µ + ( µ + ( + a s ( µ δ s+ X + + µ δ s+ s P [ ( + a s u i i= µ( µ + δ s+ Ah + asµ δ s+ P X ( ( = µ ( + a s δ s+ X + a s + δ s+ Ah + asδ s+ P X δ ] s+s P [ (d ( = µ X ( + a s( + δ s+ Ah + asδ s+ P X δ ] s+s P, (0 where (a is from (96 and (00, (b is due to (86, ( is from (9 and (9 and (d follows from (. Using (0 and (0, one obtains X [ ( + a s( + δ s+ Ah + asδ s+ P + δs+ s P ] (( + ( + a s( + δ s+ Ah + asδ s+ P asδ s+ + δ s+ s ( + a P + s( + δ s+ Ah, where the last inequality uses f + g f + g with f, g 0. Sine S S, Then, by (87, (8 holds whih implies ( holds when > 0 is an integer. When the onstant is not an integer, taing =, then > and (s + is an integer, thus δ s+ = δ s+ < + a s < + a s. Hene, by using the above proof by woring on δ s+, one an show that ( holds. REFERENCES [] E. J. Candès, J. Romberg, and T. Tao, Stable signal reovery from inomplete and inaurate measurements, Commun. Pure Appl. Math., vol. 59, no. 8, pp. 07-3, 006. [] E. J. Candès, J. Romberg, and T. Tao, Robust unertainty priniples: exat signal reonstrution from highly inomplete frequeny information, IEEE Trans. Inform. Theory, vol. 5, no., pp , 006. [3] E. J. Candès, M. B. Wain, and S. P. Boyd, Enhaning sparsity by reweighted l minimization, J. Fourier Anal. Appl., vol., no. 5, pp , 007. [] E. J. Candès, The restrited isometry property and its impliations for ompressed sensing, Compte Rendus Math., vol. 36, no. 9-0, pp , 008. [5] D. L. Donoho, Compressed sensing, IEEE Trans. Inf. Theory., vol. 5, no., pp , 006. [6] D. L. Donoho, M. Elad, and V. N. Temlyaov, Stable reovery of sparse overomplete representations in the presene of noise, IEEE Trans. Inform. Theory., vol. 5, no., pp. 6-8, 006. [7] D. L. Donoho, I. Drori, Y. Tsaig, and J. L. Star, Sparse solution of underdetermined linear equations by stagewise orthogonal mathing pursuit, IEEE Trans. Inf. Theory., vol. 58, no., pp. 09-, 0. [8] T. Cai, G. Xu, and J. Zhang, On reovery of sparse signal via l minimization, IEEE Trans. Inform. Theory, vol. 55, no. 7, pp , 009.

21 [9] T. Cai and A. Zhang, Sharp RIP bound for sparse signal and low-ran matrix reovery, Appl. Comput. Harmon. Anal., vol. 35, no., pp. 7-93, 03. [0] T. Cai and A. Zhang, Spares representation of a polytope and reovery of sparse signals and low-ran matries, IEEE Trans. Inform. Theory, vol. 60, no., pp. -3, 0. [] Q. Mo and S. Li, New bounds on the restrited isometry onstant δ, Appl. Comput. Harmon. Anal., vol. 3, no. 3, pp , 0. [] J. Wen, D. Li, and F. Zhu, Stable reovery of sparse signals via l p-minimization, Appl. Comput. Harmon. Anal., vol. 38, no., pp. 6-76, 05. [3] J. F. Determe, J. Louveaux, L. Jaques, et al., On the noise robustness of simultaneous orthogonal mathing pursuit, IEEE Trans. Signal Proess., vol. 65, no., pp , 07. [] L. Yu,, G. Zheng and J.-P. Barbot, Dynamial sparse reovery with finite-time onvergene, IEEE Trans. Signal Proess., vol. 65, no. 3, pp , 07. [5] J. Wen, Z. Zhou, J. Wang, et al., A sharp ondition for exat support reovery with orthogonal mathing pursuit, IEEE Trans. Signal Proess., vol. 65, no. 6, pp , 07. [6] J. Wen, J. Wang, and Q. Zhang, Nearly Optimal Bounds for Orthogonal Least Squares, IEEE Trans. Signal Proess., vol. 65, no. 0, pp , 07. [7] R. Zhang and S. Li, A proof of onjeture on restrited isometry property onstants δ t (0 < t <, to appear in IEEE Trans. 3 Inform. Theory, 07. [8] E. J. Candès, L. Demanet, D. L. Donoho, and L. Ying, Fast disrete urvelet transforms, Multisale Model. Simul., vol. 5, no. 3, pp , 006. [9] E. J. Candès, and D. L. Donoho, New tight frames of urvelets and optimal representations of objets with pieewise C singularities, Comm. Pure Appl. Math., vol. 57, pp. 9-66, 00. [0] A. M. Brustein, D. L. Donoho, and M. Elad, From sparse solutions of systems of equations to sparse modeling of signals and images, SIAM Rev., vol. 5, pp. 3-8, 009. [] S. S. Chen, D. L. Donoho, and M. A. Saunders, Atomi deomposition by basis pursuit, SIAM Rev., vol. 3, pp. 9-59, 00. [] E. J. Candès, Y. C. Eldar, D. Needell, and P. Randall, Compressed sensing with oherent and redundant ditionaries, Appl. Comput. Harmon. Anal., vol. 3, no., pp , 0. [3] J. Lin, S. Li, and Y. Shen, New bounds for restited isometry onstants with oherent tight frames, IEEE Trans. Signal Proess., vol. 6, no. 3, pp. 6-6, 03. [] L. Daubehies, Ten letures on wavelets//cbms-nsf Regional Conf Ser Appl Math, Philadelphia: SIAM,99. [5] E. J. Candès and T. Tao, Deoding by linear programming, IEEE Trans. Inform. Theory, vol. 5, no., pp. 03-5, 005. [6] Y. Liu, T. Mi, and S. Li, Compressed sensing with general frames via optimal-dual-based l -analysis, IEEE Trans. Inform. Theory, vol. 58, no. 7, pp. 0-, 0. [7] S. Li and J. Lin, Compressed sensing with oherent tight frame via l q-minimination for 0 < q, Inverse Probl. Imaging, vol. 8, no. 3, pp , 0. [8] R. Zhang and S. Li, Optimal D-RIP bounds in ompressed sensing, Ata Mathematia Sinia( English Series, vol. 3, no. 5, pp , 05. [9] D. M. Higdon, J. E. Bowsher, V. E. Johnson, et al., Fully Bayesian estimation of Gibbs hyperparameters for emission omputed tomography data, IEEE Trans. Med. Imaging, vol. 6, no. 5 pp , 997. [30] M. Elad, J. L. Star, P. Querre, and D. L. Donoho, Simultaneous artoon and texture image inpainting using morphologial omponent analysis, Appl. Comput. Harmon. Anal., vol. 9, pp , 005. [3] S. Farsiu, M. Elad, and P. Milanfar Multi-frame demosaiing and super-resolution of olor images, IEEE Trans. Image Proess., vol. 5, no., pp. -59, 006. [3] H. Zhang, M. Yan, and W. Yin, One ondition for solution uniqueness and robustness of both l -synthesis and l -analysis minimizations, Adv. Comput. Math., vol., pp , 06. [33] M. Elad, P. Milanfar, and R. Rubinstein, Analysis versus synthesis in signal priors, Inverse Problems, vol. 3, no. 3, pp , 007. [3] J. Lin and S. Li, Sparse reovery with oherent tight frame via analysis dantzig seletor and analysis lasso, Appl. Comput. Harmon. Anal., vol. 37, no., pp. 6-39, 0. [35] T. Zhao, Y. C. Eldar, A. Be, et al., Smoothing and deomposition for analysis sparse reovery, IEEE Trans. Signal Proess., vol. 6, no. 7, pp , 0. [36] Y. Shen, B. Han, and E. Braverman, Stable reovery of analysis based approahes, Appl. Comput. Harmon. Anal., vol. 39, no., pp. 6-7, 05. [37] S. Boyd and L. Vandenberghe, Convex optimization, U.K.: Cambridge Univ. Press, 00. [38] M. V. Afonso, J. M. Biouas-Dias, and M. A. T. Figueiredo, Fast image reovery using variable splitting and onstrained optimization, IEEE Trans. Image Proess., vol. 9, no. 9, pp , 00. [39] S. Boyd, N. Parih, E. Chu, et al., Distributed optimization and statistial learning via alternating diretion method of multipliers, in Found. Trends Mah. Learn., vol. 3, no., pp. -, 00. [0] J. N. Lasa, M. A. Davenport, and R. G. Baraniu, Exat signal reovery from orrupted measurements through the pursuit of justie, Pro. Asilomar Conf. on Signals, Systems and Computers, pp , 009. [] S. Fouart, Stability and robustness of l -minimizations with Weibull matries and redundant ditionaries, Linear Algebra and Its Appliations, vol., pp. -, 0.