A Few Basic Problems in Compressed Sensing

Transcription

1 A Few Basic Problems in Compressed Sensing Song Li School of Mathematical Sciences Zhejiang University South China Normal University,May 19,2017

2 Outline Brief Introduction Sparse Representation Sparse Recovery Methods Compressed Sensing with Redundant Dictionaries Compressed Data Separation Connections with Statistics Analysis DS and Analysis LASSO Correlated Measurements and RE Condition D-RE Condition l 2 -RDNSP and Correlated Matrices

3 Outline Brief Introduction Sparse Representation Sparse Recovery Methods Compressed Sensing with Redundant Dictionaries Compressed Data Separation Connections with Statistics Analysis DS and Analysis LASSO Correlated Measurements and RE Condition D-RE Condition l 2 -RDNSP and Correlated Matrices

4 Problem: Reconstruct signal f : {0, 1,..., N 1} C from a partial collection ˆf (ξ 1 ),..., ˆf (ξ M ) of Fourier coefficients: ˆf (ξ j ) := 1 N 1 f (t)e 2πitξj /N. N t=0 Theorem (E. Candès, J. Romberg and T. Tao, IEEE.TIT,2004) Let ξ 1,..., ξ M {0, 1,... N 1} be chosen randomly. Then with high probability, every K-sparse signal f : {0, 1,..., N 1} C can be recovered from ˆf (ξ 1 ),..., ˆf (ξ M ), as long as M > CK log N for some absolute constant C by min g g 1, s.t. ĝ Ω = ˆf Ω.

5 E. Candès, J. Romberg and T. Tao, 2004 (a) Origin (b) Measurement Matrix (c) Recovered by least square (d) Recovered by l 1 Figure: The Logan-Shepp plantom and its reconstructions after Fourier-sampling along 22 radial lines (here M/N 0.05). This type of measurement is a toy model of that used in MRI.

6 General Models There has been considerable interest in understanding when it is possible to find structured solutions to underdetermined systems of linear equations. The compressive sensing and matrix completion, that have been developed to find sparse and low-rank solutions via convex programming techniques (E.Candes,ICM Plenary Speaker, 2014). The unknown signal x R N Measurement matrix Ψ, of size M N, and M < N. Measurements y := Ψx. Problem: Reconstruct signal x from the measurements y.

7 Sparsity and Approximate Sparsity Assume that x is sparse: In the coordinate basis: x 0 = supp(x) K N. x = (x 1,..., x n ) with x j Cj r for r > 1 (approximate sparsity). Or with respect to some other basis or directory D: x = Dz and z 0 K N. Fourier Basis Wavelet Frame.

8 Outline Brief Introduction Sparse Representation Sparse Recovery Methods Compressed Sensing with Redundant Dictionaries Compressed Data Separation Connections with Statistics Analysis DS and Analysis LASSO Correlated Measurements and RE Condition D-RE Condition l 2 -RDNSP and Correlated Matrices

9 Sparse Representation It is a centre problem in applicable harmonic analysis to find D such that vector z is sparse, which is called sparse representation in approximation theory S.Mallat and Z.Zhang, 1993, X.M.Huo and D.Donoho, 1999, T.Tao and E.Candès,2004. E.Candes, Sparsity of Mathematics, ICM Plenary Speaker,2014. Which means that x = Dz, where z is the sparsest coefficients and D is a transform, for example, linear transform. When D is a dictionary, to find the sparest z, they showed that the problem can be reduced to following linear programming problem min z 1, s.t., Dz = x.

10 Outline Brief Introduction Sparse Representation Sparse Recovery Methods Compressed Sensing with Redundant Dictionaries Compressed Data Separation Connections with Statistics Analysis DS and Analysis LASSO Correlated Measurements and RE Condition D-RE Condition l 2 -RDNSP and Correlated Matrices

11 Sparse Recovery Method Straightforward choice: l 0 minimization (combinatorial optimization problem, NP hard), i.e. 0 min x x 0, s.t. Ψx = y. For the underdetermined linear system of equations Ψx = y (a full-rank matrix Ψ R n m with n < m), we pose the following questions: Q1: When can uniqueness of the sparsest solution be claimed? Q2: Can a candidate solution be tested to verify its (global) optimality? Q3: Can the solution be reliably and efficiently found in practice? Q4: What performance guarantees can be given for various approximate and practical solvers?

12 Sparse Recovery Method Convex relaxation: l 1 minimization l 0 -regularization Lasso(least absolute shrinkage and selection operator) Dantzig Selector method Nonconvex optimization: l p -minimization with 0 < p < 1 Reweighted least square algorithm Greedy pursuit Other some approaches

13 When 0 1? l 1 minimization: 1 min x x 1, s.t. Ψx = y. Measurement matrix Ψ has the null space property of order K for γ > 0 if η T 1 γ η T c 1 holds for all sets T with T K and all η {ζ : Ψζ = 0}. (R.Gribonval, M. Nielsen, 2003) If γ < 1, the K-sparse signal x is the unique solution of basis pursuit.

14 RIP (E.Candès and T.Tao, IEEE.TIT, 2004) Measurement matrix Ψ has the restricted isometry property of order K if (1 δ K ) x 2 2 Ψx 2 2 (1 + δ K ) x 2 2, x 0 K.

15 RIP Matrices (E. Candès and T.Tao, 2004) For Sub-Gaussian matrices, with high probability, δ K δ when M Cδ 2 K log(n/k). (H.Rauhut, 2008) For partial random Fourier matrix, with high probability, δ K δ when M Cδ 2 K log 4 N. Explicit matrices satisfying the RIP were constructed by R.Devore et al. (J. Complexity, 2008) J.Bourgain et al. (Duke.J.Math,2011, J.Funct.Anal, 2014, Geom.Funct.Anal,2015)

16 Theorem (E. Candès, J. Romberg and T.Tao, CPAM, 2005) If x is K-sparse, Ψ has RIP of order 4K with δ 3K + 3δ 4K < 2. Then the l 1 minimizer x is the unique solution of 1, and is equal to this x(icm 2006, plenary speaker (R.Devore)and invited speaker (E.Candes)). (T.Cai and A.Zhang, IEEE. TIT, 2014): δ K < 1/3 or δ 2K < 2 2, the bounds are optimal. T.Cai and A.Zhang (IEEE.TIT,2014) t 1 δ tk <, t 4 t 3. Conjecture by T.Cai and A.Zhang (IEEE.TIT,2014) δ tk < t 4 t, 0 < t < 4 3.

17 Optimal Bound Theorem (R.Zhang and S.Li, IEEE.TIT,online,2017) For any 0 < p 1, if Ψ satisfies the RIP with δ tk < t 4 t, 0 < t 4 3, then any k sparse signal x can be exactly recovered by solving the model min x p p s.t. Ψx = Ψx. [Extensions] Dantzig Selector, Compressible signals, Noisy observations, Tight frame, Low rank matrix recovery. Confirmed the conjecture in [A.Zhang and T.Cai, 2014] completely.

18 l p Minimization l 0 norm is the limit as p 0 of the l p norms in the following sense: Another choice: x 0 = lim p 0 x p p p : min x x p p, s.t. Ψx = y. (Z.B.Xu.etc. IEEE.TNNLS, 2012, IEEE.TSP, 2014) M.J.Lai.ect, ACHA, 2011, Q.Y.Sun, ACHA, 2013, D.R.Chen. ect, IEEE.TIT,2013, J.M.Wen.ect, ACHA, 2014 for l p Minimization by RIP and J.G.Peng.ect,IEEE.TIT,2015.

19 Optimal Bound Theorem (R.Zhang and S.Li,to submit ACHA,2016) For any 0 < p < 1, if Ψ satisfies the RIP with δ 2k < η 2 p η, where η (1 p, 1 p 2 ) is the only positive solution of the equation p 2 η 2 p + η 1 + p 2 = 0, then any k sparse signal x can be exactly recovered by solving the model min x p p s.t. Ψx = Ψx. Combined with (M.Davies and R.Gribonval,IEEE.TIT,2009), the above bound is optimal for each 0 < p 1. When η is irrational and p < 1, < can be replaced by. [Extensions] Dantzig Selector, Compressible signals, Noisy observations, Tight frame, Low rank matrix recovery.

20 Outline Brief Introduction Sparse Representation Sparse Recovery Methods Compressed Sensing with Redundant Dictionaries Compressed Data Separation Connections with Statistics Analysis DS and Analysis LASSO Correlated Measurements and RE Condition D-RE Condition l 2 -RDNSP and Correlated Matrices

21 Compressed Sensing with Redundant Dictionaries Recover f from y = Ψf + e, where e 2 ε. Let D R N d be a tight frame in R N. Sparsity is expressed not in terms of an orthogonal basis but in terms of an overcomplete dictionary. f R n is now expressed as f = Dx where x is (approximately) sparse.

22 l 1 -Synthesis Approach Reconstruct ˆx from y = ΨDx + e via and then set ˆf = Dˆx. min x x 1 s.t. ΨDx y 2 ε, When D is highly correlated, ΨD does not satisfies the RIP condition or the MIP condition in general.

23 l 1 -Analysis Approach Reconstruct ˆf directly from y = Ψf + e. Method: min f D f 1 subject to Ψf y 2 ε. (Elad-Milanfar-Rubinstein, 2007) When D is orthogonal, l 1 -analysis and l 1 -synthesis are equivalent. When D is highly redundant, a remarkable difference.

24 D-RIP (Candès-Eldar-Needell-Randall,2011) Measurement matrix Ψ has restricted isometry property adapted to D (D-RIP) of order K if (1 δ K ) Dx 2 2 ΨDx 2 2 (1 + δ K ) Dx 2 2, x 0 K. Theorem (Candès-Eldar-Needell-Randall, ACHA,2011) Let f be the true signal and f be the l 1 minimizer. If the measurement matrix Ψ satisfies D-RIP with δ 2K < 0.08 (or δ 7K < 0.6). Then, f f 2 C 0 ε + C 1 σ K (D f ) 1 K.

25 D-RIP Matrices (E. Candès and T.Tao, 2004) For Sub-Gaussian matrices,whp, δ K δ when M Cδ 2 K log(d/k). (H.Rauhut, K.Schnass and P.Vandergheynst, 2008): Any matrix Ψ obeying for fixed v R N, P((1 δ) v 2 2 Ψv 2 2 (1 + δ) v 2 2) 1 Ce γm (γ is an arbitrary positive constant) will satisfy the D-RIP WHP. Randomizing the column signs of any matrix that satisfies the standard RIP results in a matrix which satisfies the D-RIP WHP.

26 p-d-rip Definition (S.Li and J.Lin, IEEE.TIT, 2016) Measurement matrix Ψ has the p-restricted Isometry Property adapted to D ((D, p)-rip) of order K if (1 δ) Dx p 2 ΨDx p p (1 + δ) Dx p 2, x 0 K. The (D, p)-rip constant δ K is defined as the smallest number δ such that the above inequality holds. Some notations: D R n d, f R n L f 2 2 D f 2 2 U f 2 2, D = (DD ) 1 D.

27 Theorem (S.Li and J.Lin, IEEE.TIT,2016) Assume that the (D, p)-rip constant of the measurement matrix Ψ satisfies ρ 1 p/2 (ρ 1 p/2 + 1) p/2 κ p (1 + δ a ) < 1 δ K+a, ρ = K a, κ = U L. Then the solution ˆf to l p satisfies ˆf f 2 C D h (D h) [K] p p K 1/p 1/2, where C is a positive constant depending only on the δ.

28 Theorem (S.Li and J.Lin, IEEE.TIT, 2016) Let Ψ be an m n matrix whose entries are i.i.d. random distributed normally with mean zero and variance σ 2. Then there exist constants C 1 (p) and C 2 (p) such that whenever 0 < p 1 and m C 1 (p)κ p 2 p K + pc2 (p)κ p 2 p K log(d/k), ( d with probability exceeding 1 1/ K ˆf f 2 C D h (D h) [K] p p K 1/p 1/2. κ = U/L, ), the solution ˆf of l p satisfies

29 Outline Brief Introduction Sparse Representation Sparse Recovery Methods Compressed Sensing with Redundant Dictionaries Compressed Data Separation Connections with Statistics Analysis DS and Analysis LASSO Correlated Measurements and RE Condition D-RE Condition l 2 -RDNSP and Correlated Matrices

30 Model: Compressed Data Separation y = Ψ(f 1 + f 2 ) + z, where Ψ is a known m n measurement matrix (with m n). Problem: to reconstruct the unknown constituents f 1 and f 2 based on y, Ψ and dictionaries D 1, D 2, where f 1 = D 1 x 1, f 2 = D 2 x 2. D.Donoho and X.M.Huo, 2001,D.Donoho and G.Kutyniok, 2013 with Ψ = I and E.Candes, J.Romberg and T.Tao,2004. The l 1 -Split-analysis algorithm(l 1 -SAS) (ˆf 1, ˆf 2 ) = arg min f 1, f 2 R N D 1 f D 2 f 2 1 subject to Ψ( f 1 + f 2 ) y 2 ε. Candès,Eldar-Needell-Randall (ACHA,2011) One would hope to have a result for Split-analysis similar to Theorem (CENR).

31 Gaussian Matrix Let D 1 = (d 1i ) 1 i d1 and D 2 = (d 2j ) 1 j d2. The mutual coherence of D 1 and D 2 is defined as µ 1 = µ 1 (D 1 D 2 ) = max d 1i, d 2j. i,j Theorem (S.Li, J.Lin and Y.Shen,IEEE.TIT,2013) Let D 1 R N d1 and D 2 R N d2 be two tight frames. Let Ψ be Gaussian matrix with M C(K 1 + K 2 ) log(d 1 + d 2 ). Assume that µ 1 (K 1 + K 2 ) < Then WHP, the solution (ˆf 1, ˆf 2 ) to l 1 -SAS obeys ˆf 1 f ˆf 2 f 2 2 C 0 ε + C 1 σ K1 (D 1 f 1) 1 + σ K2 (D 2 f 2) 1 K1 + K 2.

32 The l q -Split-analysis algorithm(l q -SAS)(0 < q 1) (ˆf 1, ˆf 2 ) = arg min f 1, f 2 R N D 1 f 1 q q + D 2 f 2 q q subject to y = Ψ( f 1 + f 2 ). Theorem (J.Lin and S.Li,IEEE.TIT,2016) Let D 1 R N d1 and D 2 R N d2 be two tight frames. Let ( Ψ be Gaussian ) d matrix with M C 1 (q)(k 1 + K 2 ) + qc 2 (q)(k 1 + K 2 ) log 1+d 2 K 1+K 2. ) ( Assume that µ 1 (K 1 + K 2 ) ( (2 3q/2 5) 2 2 q ) < 1. Then 8.5 2/q WHP, the solution (ˆf 1, ˆf 2 ) to l q -SAS obeys ˆf 1 f ˆf 2 f 2 2 C 1 σ K1 (D 1 f 1) q q + σ K2 (D 2 f 2) q q (K 1 + K 2 ) 1/q 1/2.

33 Outline Brief Introduction Sparse Representation Sparse Recovery Methods Compressed Sensing with Redundant Dictionaries Compressed Data Separation Connections with Statistics Analysis DS and Analysis LASSO Correlated Measurements and RE Condition D-RE Condition l 2 -RDNSP and Correlated Matrices

34 Analysis DS and Analysis LASSO (ADS) : f ADS = arg min f R n D f 1 s.t. D Ψ (Ψ f y) λ (ALASSO) : ˆf AL 1 = arg min f R n 2 (Ψ f y) µ D f 1 When D is an identity matrix, the famous model (DS) was established by E. Candès and T.Tao(ICM,invited speaker, 2006). There were more than ten papers published in Annals of Statistics to discuss the DS-model after their results were published. There were also many papers related to ALASSO in statistics, see [Ryan J.Tibshirani, etc., The Annals of Statistics, 2012] and many references there in.

35 Theorem (S.Li and J.Lin, ACHA,2014) Let f be the true signal, and λ obeys D Ψ (Ψf y) λ. If the measurement matrix Ψ satisfies the D-RIP with δ 3K < 1 2. Then, f ˆf ADS 2 C 0 Kλ + C1 σ K (D f ) 1 K. where C 0 and C 1 are constants depending only on the D-RIP constant δ 3k.

36 Theorem (S.Li and J.Lin, ACHA,2014) Let f be the true signal, and λ obeys D Ψ (Ψf y) µ 2. If the measurement matrix Ψ satisfies the D-RIP with δ 3K < 1 4. Then, f ˆf AL 2 C 0 Kµ + C1 σ K (D f ) 1 K. where C 0 and C 1 are constants depending only on the D-RIP constant δ 3k and D D 1,1. Related Works: Y.Wang,J.Wang and Z.B.Xu (Block ADS model, Discrete Dynamics in Nature and Society, 2013) Z.Tan, Y.Eldar, A.Beck and A.Nehorai(IEEE.TSP,2014) combined J.Lin,S.Li and E.Candes s techniques to study the stability of analysis sparse recovery problems. Y.Shen, B.Han and E.Braverman (ACHA, 2014) improved the constants C 0, C 1. J.Liu, L.Yuan and J.P.Ye, arxiv, 2013.

37 Restricted Eigenvalue Condition The Restricted Isometry Property or Incoherence Property are too strong for anisotropic measurements, where Ea i a i I. Definition (P.Bickel et. al 2009, Ann.Stat) Let s and k be a positive number. Ψ satisfies the Restricted Eigenvalue (abbreviated as RE) condition of order s and k with constant K(s, k, A), if Ax 2 K(s, k, A) := min min > 0. I s x I c 1 k x I 1 x I 2 When k = 1, K(s, 1, A) > 0 is equivalent to NullSpace Property of A with order s.

38 RE Condition Supplementary RE condition is weaker than RIP condition [P.Bickel, Y.Ritov and A.Tsybakov, 2009]. Besides, it is one of the most general assumptions on the correlated matrix to guarantee nice properties for the LASSO estimator and the Dantzig Selector, see [G.Rauskutt, M.J.Wainwright and B.Yu, 2010], [M.Rudelson and S.H.Zhou, 2013], etc. However, [P.Bickel et. al 2009] pointed out that RE condition fail to deal with signal in redundant correlated dictionaries.

39 D-RE Condition in Analysis Model Definition (S.Li and Y.Xia, IEEE TIT, 2016) Let D be a general frame. A satisfies the D-RE condition of order s and k with constant K(s, k, A), if Ax 2 K(s, k, A) := min min I s D I c x 1 k D x I 1 DI x 2 > 0. Taking u = D x, we can see that x = D u for a unique u Range(D ). It follows that the D-RE condition can be regarded as the RE condition for matrix AD restricted to {u Range(D ) s.t. u I c 1 k u I 1 }.

40 Analysis DS and Analysis LASSO (ADS) : f ADS = arg min f R n D f 1 s.t. (D ) A (Af y) η (ALASSO) : ˆf AL 1 = arg min f R n 2 Af y λ D f 1 Denote C(s, k) := {x R p I [d] and I s s.t. D I c x 1 k D I x 1 }. When y = Af + e, where D f 0 s and e N(0, σ 2 I). Let ˆx be the solution of the ALASSO (or the ADS). Then ˆf f C(s, k) holds with high probability provided with appropriate λ and η. Here k = 1 for the ADS (k = 3 for the ALASSO).

41 D-RE Condition and D-RIP Condition The D-RE condition is a weaker version of its D -RIP condition. Theorem (S.Li and Y.Xia, IEEE TIT, 2016) Let D R p d be a frame with frame bounds L and U. If A R n p satisfies the D -RIP with ( 1 (1 + k) 1 + δ s 1 1 δ s1 A satisfies the D-RE condition. U L s s 1 s2 ) s (1 + k) > 0, s 1 s1 When D = I, it is similar as [P.Bickel et al., Assumption 1-4]. If a i i.i.d. N (0, Σ), where Σ := (1 a)i + a11 T (a [0, 1)). When D is a sparse tight frame, the D-RE condition can hold with high probability, even when the D-RIP is unbounded as s increases.

42 D-RE Condition of Correlated Sub-Gaussian Matrices Theorem (S.Li and Y.Xia, IEEE. TIT, 2016) If for any fixed x R n, we have P((1 δ) x 2 2 Φx 2 2 (1 + δ) x 2 2) 1 2 exp( γδ 2 n). Σ is a positive definite matrix. D is a tight frame. Then with probability at least 1 2 exp( γδ 2 n/2), ΦΣ 1/2 holds the D-RE condition with K(s, k, ΦΣ 1/2 ) (1 18δ)K(s, k, Σ 1/2 ), provided with n Cδ 2 s 1 log(d/s 1 ), where s 1 = s[9k k 2 (3k + 1) 2 ρ 2 max j Σ 1/2 D j 2 2 /( 1 8 δ)2 ] and ρ = max x C(s,k) S p 1 Σ 1/2 x 2. C is an absolute constant.

43 D-RE Condition of Basis System Theorem (S.Li and Y.Xia,IEEE.TIT, 2016) Let A be matrix whose rows a 1,...,a m are independent copies of a such that max i [d] a, D i K and the covariance matrix Σ meeting the D-RE condition with constant K(s, k, Σ 1/2 ). Denote ρ s = min x US Σ 1/2 x 2, where U s = {Dx S p 1 x 0 s}. D is a tight frame. Then with probability at least 1 2 exp( δρs 1 n K 2 ηs 2 s 1 ), A holds 1 the D-RE condition with K(s, k, A) (1 18δ)K(s, k, Σ 1/2 ), provided with n Cs 1K 2 ηs 2 ( 1 log d log n ρ s1 δ 2 log 2 s1 K 2 ηs 2 ) 1, where and C is an absolute constant. ρ s1 D Dx 1 η s1 = sup. Dx 2=1, x 0 s 1 s1

44 When Σ = I, it return to the sample size in Theorem 1.2 in [F.Krahmer, D.Needell and R.Ward, 2015]. They focus on reweighted measurements choosing from orthonormal bases. η s is the factor by which sparsity is preserved under the gram matrix map D D. For instance, harmonic frame with d jk = 1 p exp( 2πijk n+l ) for j [p] and k [N], then η 1 + L 2. The Haar frame with redundancy 2 have η 2 log(p). [F.Krahmer, D.Needell and R. Ward, 2015]. When D is a general frame with frame bounds L and U, we have n C(U/L) 3 spoly log n in the above two theorems.

45 Sparse Signal and D-RE Estimation Theorem (S.Li and Y.Xia,IEEE.TIT, 2016) D R p d is a frame with frame bounds L and U. Let y = Af + e with e N (0, σ 2 I) and D f 0 s. Let ˆf AL and ˆf ADS be an optimal solution to the ALASSO and the ADS model with λ 4ασ log d and η 2ασ log d, where α = max i [d] AD i 2. Then with probability at least 1 ( π log dd) 1, we have L ˆf AL f 2 8λ s/k 2 (s, 3, A), L ˆf ADS f 2 12η s/k 2 (s, 1, A).

46 Remarks When D is a tight frame, applying the relationship between the D-RE condition and the D-RIP condition, we get sufficient condition δ 2s < 0.25 in AlASSO model, which is weaker than δ 2s < 0.2 in [Y.Shen, B.Han and E.Braverman, ACHA,2014] and δ 3s < 0.25 in [S.Li and J.Lin, ACHA,2014]. In the theorem, ˆx AL x 2 C 1 := 8λ s(1 + δ 2s ) 2 /(1 4δ 2s ) 2, If D x 0 s, compared with the error bound ˆx AL x 2 C 2 := 3(1 + δ 2s)(7 12δ 2s ) 2(1 2δ 2s ) 2 (1 5δ 2s ) λ s in [Y.Shen,B.Han and E.Braverman, ACHA,2014], C 1 is smaller than C 2 when δ 2s is fixed.

47 Robust D-NullSpace Property Definition (S.Fourcart, 2014) Ψ is said to satisfy the robust l 2 D-NullSpace Property (l 2 -RDNSP) of order s with constant ρ > 0 and 0 < γ < 1 2, if for any x Rp, we have D Sx 2 ρ Ψx 2 + γ D x 1 s, where D S x is the best s-sparse approximation of D x. It is weaker than restricted isometry property and stronger than NullSpace Property, besides,it is almost a necessary and sufficient condition for robust recovery [Q.Y.Sun, 2011], [S.Fourcart, 2014]. When A satisfies l 2 -RDNSP (e.g, Weibull random matrices) and D is a general frame, The stability of Dual l 1 analysis was established in [X.Y.Zhang and S.Li, 2013, IEEE SPL].

48 RDNSP and Correlated Matrices Theorem (S.Li and Y.Xia,IEEE.TIT, 2016) Consider Gaussian random matrix A R n p with i.i.d. N (0, Σ) rows. Let D R p d be a frame with frame bounds L and U. Σ 1 2 satisfies the l 2 -DNSP of order s 0. If n C ( α 1 U L ) 2 s 0 log d, where α 1 = max i [d] D Σ 1 2 D i 2. Then with high probability, A satisfies the l 2 -DNSP of order s 0. If Σ = I, we see that α 1 = 1 and n = O ( U L s 0 log d ). [H.Rauhut et al., 2015]. C is a constant depending on the NullSpace constant of Σ 1 2, which may depends on the ratio κ = U/L.

49 Stability Estimation with l 2 -RDNSP Theorem (S.Li and Y.Xia,IEEE.TIT, 2016) If Ψ satisfies the l 2 -RDNSP of order s with constant (ρ, γ). Let y = Af + e with (D ) A e η = λ/2. Let ˆf ADS and ˆf AL be optimal solutions to the ADS model and the ALASSO model, respectively. We have ˆf ADS f 2 ˆf AL f 2 12ρ 2 (1 2γ) 2 L η s + 2γ + 5 DS f c 1, (1 2γ)L s 15ρ 2 (1 4γ) 2 L λ 32 8γ DS s + f c 1, (3 12γ)L s where S is the index set of largest s elements of D f in magnitude. If e N (0, σ 2 I), η = 2ασ log d, where α = max i [d] AD i 2.

50 Suppose that the measurement error is e N (0, σ 2 I), we can take λ = 4ασ log d and η = (D ) D, λ, where α = max i [d] AD i 2, to predict the closeness of prediction loss, AˆxADS Ax 2 2 Aˆx AL Ax λ2 (ρ 2 + 1)s 0 (1 2γ) 2 (D ) D 2, + D ˆx AL (D ˆx AL ) [s0] 2 1/s 0. It generates the results in [P.Bickel et al. 2009] to the non-sparse signal case.

51 Thank You Song Li School of Mathematical Sciences, Zhejiang University

52 1. Qun Mo, and Song Li. New bounds on the restricted isometry constant δ 2k. Applied and Computational Harmonic Analysis, 31 (2011): Yi Shen and Song Li, Restricted p-isometry property and its application for nonconvex compressive sensing, Advance in Computational Mathematics, 37, (2012): Junhong Lin, Song Li, and Yi Shen. New bounds for restricted isometry constants with coherent tight frames. IEEE Transactions on Signal Processing, 61 (2013): Junhong Lin, Song Li, and Yi Shen. Compressed data separation with redundant dictionaries. IEEE Transactions on Information Theory, 59 (2013): Yun Cai, and Song Li. Compressed data separation via dual frames based split-analysis with Weibull matrices. Applied Mathematics-A Journal of Chinese Universities 28 (2013): Huimin Wang, and Song Li. The bounds of restricted isometry constants for low rank matrices recovery. Science China Mathematics 56 (2013): Junhong Lin, Song Li Block sparse recovery via mixed l 2 /l 1 minimization. Acta Mathematica Sinica, English Series, 29 (2013): Xiaoya Zhang and Song Li. Compressed Sensing via Dual Frame Based l 1 -Analysis With Weibull Matrices. IEEE Signal Processing Letters, 20 (2013): Junhong Lin, and Song Li. Nonuniform support recovery from noisy random measurements by Orthogonal Matching Pursuit. Journal of Approximation Theory, 165 (2013): Junhong Lin and Song Li. Convergence of projected Landweber iteration for matrix rank minimization. Applied and Computational Harmonic Analysis, 36 (2014): Junhong Lin, Song Li, Sparse recovery with coherent tight frame via analysis dantzig selector and analysis lasso, Applied and Computational Harmonic Analysis, 37 (2014): Junhong Lin, Song Li, Compressed sensing with coherent tight frames via lq minimization for 0 < q 1. Inverse Problems and Imaging, 8 (2014): Yun Cai, Song Li, Convergence analysis of projected gradient descent for Schatten-p nonconvex matrix recovery, Science China Mathematics, 58 (2015): Rui Zhang, Song Li, Optimal D-RIP bounds in compressed sensing, Acta Mathematica Sinica, English Series, to appear in Yi Shen, Song Li, Sparse Signals Recovery from Noisy Measurements by Orthogonal Matching Pursuit, Inverse Problem and Imaging, 9 (2015):