Weighted- 1 minimization with multiple weighting sets

Transcription

1 Weighted- 1 iniization with ultiple weighting sets Hassan Mansour a,b and Özgür Yılaza a Matheatics Departent, University of British Colubia, Vancouver - BC, Canada; b Coputer Science Departent, University of British Colubia, Vancouver - BC, Canada ABSTRACT In this paper, we study the support recovery conditions of weighted 1 iniization for signal reconstruction fro copressed sensing easureents when ultiple support estiate sets with different accuracy are available. We identify a class of signals for which the recovered vector fro 1 iniization provides an accurate support estiate. We then derive stability and robustness guarantees for the weighted 1 iniization proble with ore than one support estiate. We show that applying a saller weight to support estiate that enjoy higher accuracy iproves the recovery conditions copared with the case of a single support estiate and the case with standard, i.e., non-weighted, 1 iniization. Our theoretical results are supported by nuerical siulations on synthetic signals and real audio signals. Keywords: Copressed sensing, weighted 1 iniization, partial support recovery 1. INTRODUCTION A wide range of signal processing applications rely on the ability to realize a signal fro linear and soeties noisy easureents. These applications include the acquisition and storage of audio, natural and seisic iages, and video, which all adit sparse or approxiately sparse representations in appropriate transfor doains. Copressed sensing has eerged as an effective paradig for the acquisition of sparse signals fro significantly fewer linear easureents than their abient diension. 1 3 Consider an arbitrary signal x R N and let y R n be a set of easureents given by y = Ax + e, where A is a known n N easureent atrix, and e denotes additive noise that satisfies e 2 for soe known. Copressed sensing theory states that it is possible to recover x fro y (given A) evenwhen n N, i.e., using very few easureents. When x is strictly sparse, i.e. when there are only k<nnonzero entries in x, and when e =, one ay recover an estiate x of the signal x as the solution of the constrained iniization proble iniize u subject to Au = y. (1) u R N In fact, using (1), the recovery is exact when n 2k and A is in general position. 4 However, iniization is a cobinatorial proble and quickly becoes intractable as the diensions increase. Instead, the convex relaxation iniize u 1 subject to Au y 2 (2) u R N can be used to recover the estiate x. Candés, Roberg and Tao 2 and Donoho 1 show that if n k log(n/k), then 1 iniization (2) can stably and robustly recover x fro inaccurate and what appears to be incoplete easureents y = Ax+e, where, as before, A is an appropriately chosen n N easureent atrix and e 2. Contrary to iniization, (2), which is a convex progra, can be solved efficiently. Consequently, it is possible to recover a stable and robust approxiation of x by solving (2) instead of (1) at the cost of increasing the nuber of easureents taken. Further author inforation: (Send correspondence to Hassan Mansour) Hassan Mansour: E-ail: hassan@cs.ubc.ca; Özgür Yılaz: E-ail: oyilaz@ath.ubc.ca

2 Several works in the literature have proposed alternate algoriths that attept to bridge the gap between and 1 iniization. For exaple, the recovery fro copressed sensing easureents using p iniization with <p<1has been shown to be stable and robust under weaker conditions that those of 1 iniization. 5 9 However, the proble is non-convex and even though various siple and efficient algoriths were proposed and observed to perfor well epirically, 7, 1 so far only local convergence can be proved. Another approach for iproving the recovery perforance of 1 iniization is to incorporate prior knowledge regarding the support of the signal to-be-recovered. One way to accoplish this is to replace 1 iniization in (2) with weighted 1 iniization iniize u 1,w subject to Au y 2, (3) u where w [, 1] N and u 1,w := i w i u i is the weighted 1 nor. This approach has been studied by several groups and ost recently, by the authors, together with Saab and Friedlander [15]. In this work, we proved that conditioned on the accuracy and relative size of the support estiate, weighted 1 iniization is stable and robust under weaker conditions than those of standard 1 iniization. The works entioned above ainly focus on a two-weight scenario: for x R N, one is given a partition of {1,...,N} into two sets, say T and T c. Here T denotes the estiated support of the entries of x that are largest in agnitude. In this paper, we consider the ore general case and study recovery conditions of weighted 1 iniization when ultiple support estiates with different accuracies are available. We first give a brief overview of copressed sensing and review our previous result on weighted 1 iniization in Section 2. In Section 3, we prove that for a certain class of signals it is possible to estiate the support of its best k-ter approxiation using standard 1 iniization. We then derive stability and robustness guarantees for weighted 1 iniization which generalizes our previous work to the case of two or ore weighting sets. Finally, we present nuerical experients in Section 4 that verify our theoretical results. 2. COMPRESSED SENSING WITH PARTIAL SUPPORT INFORMATION Consider an arbitrary signal x R N and let x k be its best k-ter approxiation, given by keeping the k largest-in-agnitude coponents of x and setting the reaining coponents to zero. Let T =supp(x k ), where T {1,...,N} and T k. We wish to reconstruct the signal x fro y = Ax + e, wherea is a known n N easureent atrix with n N, and e denotes the (unknown) easureent error that satisfies e 2 for soe known argin >. Also let the set T {1,...,N} be an estiate of the support T of x k. 2.1 Copressed sensing overview It was shown in [2] that x can be stably and robustly recovered fro the easureents y by solving the optiization proble (1) if the easureent atrix A has the restricted isoetry property 16 (RIP). Definition 1. The restricted isoetry constant δ k of a atrix A is the sallest nuber such that for all k-sparse vectors u, (1 δ k )u 2 2 Au 2 2 (1 + δ k )u 2 2. (4) The following theore uses the RIP to provide conditions and bounds for stable and robust recovery of x by solving (2). Theore 2.1 (Candès, Roberg, Tao 2 ). Suppose that x is an arbitrary vector in R N, and let x k be the best k-ter approxiation of x. Suppose that there exists an a 1 k Z with a>1 and Then the solution x to (2) obeys δ ak + aδ (1+a)k <a 1. (5) x x 2 C + C 1 k 1/2 x x k 1. (6)

3 Reark 1. The constants in Theore 2.1 are explicitly given by C = 2(1+a 1/2 ) 1 δ(a+1)k a 1/2 1+δ ak, C 1 = 2a 1/2 ( 1 δ (a+1)k + 1+δ ak) 1 δ(a+1)k a 1/2. (7) 1+δ ak Theore 2.1 shows that the constrained 1 iniization proble in (2) recovers an approxiation to x with an error that scales well with noise and the copressibility of x, provided (5) is satisfied. Moreover, if x is sufficiently sparse (i.e., x = x k ), and if the easureent process is noise-free, then Theore 2.1 guarantees exact recovery of x fro y. At this point, we note that a slightly stronger sufficient condition copared to (5) that is easier to copare with conditions we obtain in the next section is given by 2.2 Weighted 1 iniization δ (a+1)k < a 1 a +1. (8) The 1 iniization proble (2) does not incorporate any prior inforation about the support of x. However, in any applications it ay be possible to draw an estiate of the support of the signal or an estiate of the indices of its largest coefficients. In our previous work, 15 we considered the case where we are given a support estiate T {1,...,N} for x with a certain accuracy. We investigated the perforance of weighted 1 iniization, as described in (3), where the weights are assigned such that w j = ω [, 1] whenever j T, and w j = 1 otherwise. In particular, we proved that if the (partial) support estiate is at least 5% accurate, then weighted 1 iniization with ω<1 outperfors standard 1 iniization in ters of accuracy, stability, and robustness. Suppose that T has cardinality T = ρk, where ρ N/k is the relative size of the support estiate T. Furtherore, define the accuracy of T T via α := T T,i.e.,αisthefraction of T inside T. As before, we wish to recover an arbitrary vector x R N fro noisy copressive easureents y = Ax + e, wheree satisfies e 2. To that end, we consider the weighted 1 iniization proble with the following choice of weights: 1, i T iniize z 1,w subject to Az y 2 with w i = c, z ω, i T. (9) Here, ω 1 and z 1,w is as defined in (3). Figure 1 illustrates the relationship between the support T, support estiate T and the weight vector w. Figure 1. Illustration of the signal x and weight vector w ephasizing the relationship between the sets T and T.

4 Theore 2.2 (FMSY 15 ). Let x be in R N and let x k be its best k-ter approxiation, supported on T. Let T {1,...,N} be an arbitrary set and define ρ and α as before such that T = ρk and T T = αρk. Suppose that there exists an a 1 k Z,witha (1 α)ρ, a>1, and the easureent atrix A has RIP with δ ak + a a 2 δ (a+1)k < 2 1, (1) ω +(1 ω) 1+ρ 2αρ ω +(1 ω) 1+ρ 2αρ for soe given ω 1. Then the solution x to (9) obeys x x 2 C + C1k 1/2 ωx x k 1 +(1 ω)x T c T c 1, (11) where C and C 1 are well-behaved constants that depend on the easureent atrix A, the weight ω, andthe paraeters α and ρ. Reark 2. The constants C and C1 are explicitly given by the expressions 2 1+ ω+(1 ω) 1+ρ 2αρ C a = 1 δ(a+1)k ω+(1 ω) 1+ρ 2αρ, C 2a 1/2 1 δ (a+1)k + 1+δ ak 1 = 1+δak a 1 δ(a+1)k ω+(1 ω) 1+ρ 2αρ. (12) 1+δak a Consequently, Theore 2.2, with ω =1, reduces to the stable and robust recovery theore of [2], which we stated above see Theore 2.1. Reark 3. It is sufficient that A satisfies δ (a+1)k < ˆδ (ω) := a ω +(1 ω) 1+ρ 2αρ 2 a + ω +(1 ω) 1+ρ 2αρ 2 (13) for Theore 2.2 to hold, i.e., to guarantee stable and robust recovery of the signal x fro easureents y = Ax + e. It is easy to see that the sufficient conditions of Theore 2.2, given in (1) or (13), are weaker than their counterparts for the standard 1 recovery, as given in (5) or (8) respectively, if and only if α>.5. A siilar stateent holds for the constants. In words, if the support estiate is ore than 5% accurate, weighted 1 is ore favorable than 1, at least in ters of sufficient conditions and error bounds. The theoretical results presented above suggest that the weight ω should be set equal to zero when α.5 and to one when α<.5 as these values of ω give the best sufficient conditions and error bound constants. However, we conducted extensive nuerical siulations in [15] which suggest that a choice of ω.5 results in the best recovery when there is little confidence in the support estiate accuracy. An heuristic explanation of this observation is given in [15]. 3. WEIGHTED 1 MINIMIZATION WITH MULTIPLE SUPPORT ESTIMATES The result in the previous section relies on the availability of a support estiate set T on which to apply the weights ω. In this section, we first show that it is possible to draw support estiates fro the solution of (2). We then present the ain theore for stable and robust recovery of an arbitrary vector x R N fro easureents y = Ax + e, y R n and n N, with ultiple support estiates having different accuracies. 3.1 Partial support recovery fro 1 iniization For signals x that belong to certain signal classes, the solution to the 1 iniization proble can carry significant inforation on the support T of the best k-ter approxiation x k of x. We start by recalling the null space property (NSP) of a atrix A as defined in [17]. Necessary conditions as well as sufficient conditions for the existence of soe algorith that recovers x fro easureents y = Ax with an error related to the best k-ter approxiation of x can be forulated in ters of an appropriate NSP. We state below a particular for of the NSP pertaining to the 1-1 instance optiality.

5 Definition 2. AatrixA R n N, n<n, is said to have the null space property of order k and constant c if for any vector h N(A), Ah =, and for every index set T {1...N} of cardinality T = k h 1 c h T c 1. Aong the various iportant conclusions of [17], the following (in a slightly ore general for) will be instruental for our results. Lea 3.1 ([17]). If A has the restricted isoetry property with δ (a+1)k < a 1 a+1 for soe a>1, then it has the NSP of order k and constant c given explicitly by 1+δak c =1+. a 1 δ(a+1)k In what follows, let x be the solution to (2) and define the sets S =supp(x s ), T =supp(x k ), and T = supp(x k ) for soe integers k s>. Proposition 3.2. Suppose that A has the null space property (NSP) of order k with constant c and where η = 2c 2 c. Then S T. The proof is presented in section A of the appendix. Reark 4. Note that if A has RIP so that δ (a+1)k < a 1 a+1 in x(j) (η + 1)x T c 1, (14) j S for soe a>1, then η is given explicitly by η = 2( a 1 δ (a+1)k + 1+δ ak ) a 1 δ(a+1)k 1+δ ak. (15) Proposition 3.2 states that if x belongs to the class of signals that satisfy (14), then the support S of x s i.e., the set of indices of the s largest-in-agnitude coefficients of x is guaranteed to be contained in the set of indices of the k largest-in-agnitude coefficients of x. Consequently, if we consider T to be a support estiate for x k, then it has an accuracy α s k. Note here that Proposition 3.2 specifies a class of signals, defined via (14), for which partial support inforation can be obtained by using the standard 1 recovery ethod. Though this class is quite restrictive and does not include various signals of practical interest, experients suggest that highly accurate support estiates can still be obtained via 1 iniization for signals that only satisfy significantly ilder decay conditions than (14). A theoretical investigation of this observation is an open proble. 3.2 Multiple support estiates with varying accuracy: an idealized otivating exaple Suppose that the entries of x decay according to a power law such that x(j) = cj p for soe scaling constant c, p>1 and j {1,...,N}. Consider the two support sets T 1 =supp(x k1 ) and T 2 =supp(x k2 ) for k 1 >k 2, T 2 T 1. Suppose also that we can find entries x(s 1 ) = cs p 1 c(η + 1) k1 p 1 p 1 and x(s 2)=cs p 2 c(η + 1) k1 p 2 p 1 that satisfy (14) for the sets T 1 and T 2,respectively,wheres 1 k 1 and s 2 k 2.Then s 1 s 2 = p 1 η+1 which follows because < 1 1/p < 1 and k 1 k /p k 1 1/p 1 k 1 1/p 2 p 1 η+1 1/p (k1 k 2 ).

6 Consequently, if we define the support estiate sets T 1 =supp(x k 1 ) and T 2 =supp(x k 2 ), clearly the corresponding accuracies α 1 = s1 k 1 and α 2 = s2 k 2 are not necessarily equal. Moreover, if 1/p p 1 <α 1, (16) η +1 1/p s 1 s 2 <α 1 (k 1 k2), and thus α 1 <α 2. For exaple, if we have p =1.3 and η = 5, we get p 1 η+1.1. Therefore, in this particular case, if α 1 >.1, choosing soe k 2 <k 1 results in α 2 >α 1,i.e.,weidentifytwo different support estiates with different accuracies. This observation raises the question, How should we deal with the recovery of signals fro CS easureents when ultiple support estiates with different accuracies are available? We propose an answer to this question in the next section. 3.3 Stability and robustness conditions In this section we present our ain theore for stable and robust recovery of an arbitrary vector x R N fro easureents y = Ax + e, y R n and n N, with ultiple support estiates having different accuracies. Figure 2 illustrates an exaple of the particular case when only two disjoint support estiate sets are available. Figure 2. Exaple of a sparse vector x with support set T and two support estiate sets T 1 and T 2.Theweightvector is chosen so that weights ω 1 and ω 2 are applied to the sets T 1 and T 2, respectively, and a weight equal to one elsewhere. Let T be the support of the best k-ter approxiation x k of the signal x. Suppose that we have a support estiate T that can be written as the union of disjoint subsets T j, j {1,...,}, each of which has cardinality T j = ρ j k, ρ j a for soe a>1 and accuracy α j = T j T T. j Again, we wish to recover x fro easureents y = Ax + e with e 2. To do this, we consider the general weighted 1 iniization proble in u 1,w subject to Au y (17) u R N

7 where u 1,w = N w i u i, and w i = i=1 ω 1, i T 1. ω, i T 1, i T c for ω j 1, for all j {1,...,} and T = T j. Theore 3.3. Let x R n and y = Ax + e, where A is an n N atrix and e is additive noise with e 2 for soe known >. Denote by x k the best k-ter approxiation of x, supportedont and let T 1,..., T {1,...,N} be as defined above with cardinality T j = ρ j k and accuracy α j = T j T T, j {1,...,}. j For soe given ω 1,...,ω 1, define γ := ω j ( 1) + (1 ω j ) 1+ρ j 2α j ρ j. If the RIP constants of A are such that there exists an a 1 k Z,witha>1, and δ ak + a γ 2 δ (a+1)k < a 1, (18) γ2 then the solution x # to (17) obeys x # x 2 C (γ) + C 1 (γ)k 1/2 ω j x Tj T c 1 + x T c T c 1. (19) The proof is presented in section B of the appendix. Reark 5. The constants C (γ) and C 1 (γ) are well-behaved and given explicitly by the expressions 2 1+ γ a C (γ) = 1 δ(a+1)k γ, C 1 (γ) = 2a 1/2 1 δ (a+1)k + 1+δ ak 1+δak a 1 δ(a+1)k γ. (2) 1+δak a Reark 6. Theore 3.3 is a generalization of Theore 2.2 for 1 support estiates. It is easy to see that when the nuber of support estiates =1, Theore 3.3 reduces to the recovery conditions of Theore 2.2. Moreover, setting ω j =1for all j {1,...,} reduces the result to that in Theore 2.1. Reark 7. The sufficient recovery condition (13) becoes in the case of ultiple support estiates δ (a+1)k < ˆδ (γ) := a γ2 a + γ 2, (21) where γ is as defined in Theore 3.3. It can be shown that when =1, γ reduces to the expression in (13). Reark 8. The value of γ controls the recovery guarantees of the ultiple-set weighted 1 iniization proble. For instance, as γ approaches, condition (21) becoes weaker and the error bound constants C (γ) and C 1 (γ) becoe saller. Therefore, given a set of support estiate accuracies α j for all j {1...}, it is useful to find the corresponding weights ω j that iniize γ. Notice that for all j, γ is a su of linear functions of ω j with α j controlling the slope. When α j >.5, the slope is positive and the optial value of ω j =. Otherwise, when α j.5, the slope is negative and the optial value of ω j =1. Hence, as in the single support estiate case, the theoretical conditions indicate that when the α j are known a choice of ω j equal to zero or one should be optial. However, when the knowledge of α j is not reliable, experiental results indicate that interediate values of ω j produce the best recovery results.

8 L 1 in 2 set wl 1 3 set wl 1 ( 1 =.8, 1 =.1, 2 = ) 12 =.3 2 =.5 45 = SNR (db) SNR (db) SNR (db) Figure 3. Coparison between the recovered SNR (averaged over 1 experients) using two-set weighted 1 with support estiate T and accuracy α, three-set weighted 1 iniization with support estiates T 1 T 2 = T and accuracy α 1 =.8 and α 2 <α,andnon-weighted 1 iniization. 4. NUMERICAL EXPERIMENTS In what follows, we consider the particular case of = 2, i.e. where there exists prior inforation on two disjoint support estiates T 1 and T 2 with respective accuracies α 1 and α 2. We present nuerical experients that illustrate the benefits of using three-set weighted 1 iniization over two-set weighted 1 and non-weighted 1 iniization when additional prior support inforation is available. To that end, we copare the recovery capabilities of these algoriths for a suite of synthetically generated sparse signals. We also present the recovery results for a practical application of recovering audio signals using the proposed weighting. In all of our experients, we use SPGL1 18, 19 to solve the standard and weighted 1 iniization probles. 4.1 Recovery of synthetic signals We generate signals x with an abient diension N = 5 and fixed sparsity k = 35. We copute the (noisy) copressed easureents of x using a Gaussian rando easureent atrix A with diensions n N where n = 1. To quantify the reconstruction quality, we use the reconstruction signal to noise ratio (SNR) average over 1 realizations of the sae experiental conditions. The SNR is easured in db and is given by x 2 SNR(x, x) = 1 log 2 1 x x 2, (22) 2 where x is the true signal and x is the recovered signal. The recovery via two-set weighted 1 iniization uses a support estiate T of size T = 4 (i.e., ρ = 1) where the accuracy α of the support estiate takes on the values {.3,.5,.7}, and the weight ω is chosen fro {.1,.3,.5}. Recovery via three-set weighted 1 iniization assues the existence of two support estiates T 1 and T 2, which are disjoint subsets of T described above. The set T 1 is chosen such that it always has an accuracy α 1 =.8 while T 2 = T \ T 1. In all experients, we fix ω 1 =.1 and set ω 2 = ω. Figure 4.1 illustrates the recovery perforance of three-set weighted 1 iniization copared to twoset weighted 1 using the setup described above and non-weighted 1 iniization. The figure shows that utilizing the extra accuracy of T 1 by setting a saller weight ω 1 results in better signal recovery fro the sae easureents.

9 4.2 Recovery of audio signals Next, we exaine the perforance of three-set weighted 1 iniization for the recovery of copressed sensing easureents of speech signals. In particular, the original signals are sapled at 44.1 khz, but only 1/4th of the saples are retained (with their indices chosen randoly fro the unifor distribution). This yields the easureents y = Rs, wheres is the speech signal and R is a restriction (of the identity) operator. Consequently, by dividing the easureents into blocks of size N, we can write y =[y T 1,y T 2,...] T. Here each y j = R j s j is the easureent vector corresponding to the jth block of the signal, and R j R nj N is the associated restriction atrix. The signals we use in our experients consist of 21 such blocks. We ake the following assuptions about speech signals: 1. The signal blocks are copressible in the DCT doain (for exaple, the MP3 copression standard uses a version of the DCT to copress audio signals.) 2. The support set corresponding to the largest coefficients in adjacent blocks does not change uch fro block to block. 3. Speech signals have large low-frequency coefficients. Thus, for the reconstruction of the jth block, we identify the support estiates T 1 is the set corresponding to the largest n j /16 recovered coefficients of the previous block (for the first block T 1 is epty) and T 2 is the set corresponding to frequencies up to 4kHz. For recovery using two-set weighted 1 iniization, we define T = T 1 T 2 and assign it a weight of ω. In the three-set weighted 1 case, we assign weights ω 1 = ω/2 on the set T 1 and ω 2 = ω on the set T \ T 1. The results of experients on an exaple speech signal with N = 248, and ω {, 1/6, 2/6,...,1} are illustrated in Figure 4. It is clear fro the figure that three-set weighted 1 iniization has better recovery perforance over all 1 values of ω spanning the interval [, 1] SNR 6 2 set wl1 3 set wl1 ( 1 =, 2 = /2) /6 2/6 3/6 4/6 5/6 1 Figure 4. SNRs of the two reconstruction algoriths two-set and three-set weighted 1 iniization for a speech signal fro copressed sensing easureents plotted against ω.

10 5. CONCLUSION In conclusion, we derived stability and robustness guarantees for the weighted 1 iniization proble with ultiple support estiates with varying accuracy. We showed that incorporating additional support inforation by applying a saller weight to the estiated subsets of the support with higher accuracy iproves the recovery conditions copared with the case of a single support estiate and the case of (non-weighted) 1 iniization. We also showed that for a certain class of signals the coefficients of which decay in a particular way it is possible to draw a support estiate fro the solution of the 1 iniization proble. These results raise the question of whether it is possible to iprove on the support estiate by solving a subsequent weighted 1 iniization proble. Moreover, it raises an interest in defining a new iterative weighted 1 algorith which depends on the support accuracy instead of the coefficient agnitude as is the case of the Candès, Wakin, and Boyd 2 (IRL1) algorith. We shall consider these probles elsewhere. APPENDIX A. PROOF OF PROPOSITION 3.2 We want to find the conditions on the signal x and the atrix A which guarantee that the solution x to the 1 iniization proble (2) has the following property in j S x (j) ax x (j) = x (k + 1). j T c Suppose that the atrix A has the Null Space property (NSP) 17 of order k, i.e., for any h N(A), Ah =, then h 1 c h T c 1, where T {1, 2,...N} with T = k, and N (A) denotes the Null-Space of A. If A has RIP with δ (a+1)k < a 1 a+1 for soe constant a>1, then it has the NSP of order k with constant c which can be written explicitly in ters of the RIP constant of A as follows 1+δak c =1+. a 1 δ(a+1)k Define h = x x, thenh N(A) and we can write the 1-1 instance optiality as follows h 1 2c 2 c x T c 1, with c < 2. Let η = 2c 2 c, the bound on h T 1 is then given by h T 1 (η + 1)x T c 1 x T c 1. (23) The next step is to bound x T c 1. Noting that T =supp(x k ), then x T 1 x T c 1, and x T c 1 x T c 1 x (k + 1). Using the reverse triangle inequality, we have j, x(j) x (j) x(j) x (j) which leads to in j S x (j) in x(j) ax x(j) j S j S x (j). But ax j S x(j) x (j) = h S h S 1 h T 1, so cobining the above three equations we get in j S x (j) x (k + 1) +in x(j) (η + 1)x T c 1. (24) j S Equation (24) says that if the atrix A has δ (a+1)k -RIP and the signal x obeys in x(j) (η + 1)x T c 1, j S then the support T of the largest k entries of the solution x to (2) contains the support S of the largest s entries of the signal x.

11 APPENDIX B. PROOF OF THEOREM 3.3 The proof of Theore 3.3 follows in the sae line as our previous work in [15] with soe odifications. Recall that the sets T j and disjoint and T = T j, and define the sets T jα = T T j, for all j {1,...,}, where T jα = α j ρ j k. Let x # = x + h be the iniizer of the weighted 1 proble (17). Then Moreover, by the choice of weights in (17), we have Consequently, x + h 1,w x 1,w. ω 1 x T1 + h T ω x T + h T 1 + x T c + h T c 1 ω 1 x T1 1 + ω x T 1 + x T c 1. x T c T + h T c T 1 + x T c T c + h T c T c 1 + x T c T 1 + x T c T c 1 + ω j x Tj T + h Tj T 1 + ω j x Tj T c ω j x Tj T 1 + ω j x Tj T c 1. + h Tj T c 1 Next, we use the forward and reverse triangle inequalities to get ω j h Tj T c 1 + h T c T c 1 h T c T 1 + ω j h Tj T 1 +2 x T c T c 1 + ω j x Tj T c 1. Adding (1 ω j )h T c j T c 1 on both sides of the inequality above we obtain h Tj T c 1 + h T c T c 1 ω j h Tj T 1 + ω j )h Tj T (1 c 1 + h T c T x T c T c 1 + ω j x Tj T c 1. Since h T c 1 = h T T c 1 + h T c T c 1 and h T T c 1 = h Tj T c 1, this easily reduces to h T c 1 ω j h Tj T 1 + (1 ω j )h Tj T c 1 + h T c T 1 +2 x T c T c 1 + ω j x Tj T c 1. (25) Now consider the following ter fro the left hand side of (25) ω j h Tj T 1 + (1 ω j )h Tj T c 1 + h T c T 1 Add and subtract (1 ω j)h T c j T 1, and since the set T jα = T T j, we can write h T c j T 1 +h Tj T c 1 = h T T \ T jα 1 to get = = ω j h Tj T 1 + h T c j T 1 + (1 ω j ) ω j h T 1 + h T c T 1 h T c j T 1 + ω j +1 h T 1 + (1 ω j )h T T \ T jα 1. h Tj T c 1 + h T c j T 1 + h T c T 1 (1 ω j )h T T \ T jα 1 h T c j T 1

12 The last equality coes fro h T T j c 1 = h T c T 1 +h T ( T \ T j) 1 and h T ( T \ T j) 1 =( 1)h T T 1. Consequently, we can reduce the bound on h T c 1 to the following expression: h T c 1 ω j +1 h T 1 + (1 ω j )h T T \ T jα 1 +2 x T c T c 1 + ω j x Tj T c 1. (26) Next we follow the technique of Candès et al. 2 and sort the coefficients of h T c partitioning T c it into disjoint sets T j,j {1, 2,...} each of size ak, wherea>1. That is, T 1 indexes the ak largest in agnitude coefficients of h T c, T 2 indexes the second ak largest in agnitude coefficients of h T c, and so on. Note that this gives = j 1 h T j,with h T c h Tj 2 akh Tj (ak) 1/2 h Tj 1 1. (27) Let T 1 = T T 1, then using (27) and the triangle inequality we have h T c 1 2 h Tj 2 (ak) 1/2 h Tj 1 j 2 (ak) 1/2 h T c 1. Next, consider the feasibility of x # and x. Both vectors are feasible, so we have Ah 2 2 and Ah T Ah T c Ah Tj 2 j δ ak h Tj 2. Fro (26) and (28) we get Ah T δ akak + 1+δ akak j 2 j 1 x T c T c 1 + ω j x Tj T c 1 ( ω j + 1)h T 1 + (1 ω j )h T T \ T jα 1. Noting that T T \ T jα =(1+ρ j 2α j ρ j )k, 1 δ(a+1)k h T δ akak x T c T c 1 + ω j x Tj T c δ aak ( ω j + 1)h T 2 + (1 ω j ) 1+ρ j 2α j ρ j h T T \ T jα 2. Since for every j we have h T T j\ T jα 2 h T1 2 and h T 2 h T1 2,thus δ ak ak x T c T c 1 + ω j x Tj T c 1 h T1 2 ω. (29) j +1+ (1 ω j) 1+ρ j 2α jρ j 1 δ(a+1)k 1+δak a Finally, using h 2 h T1 2 +h T c 1 2 and let γ = ω j +1+ (1 ω j ) 1+ρ j 2α j ρ j, we cobine (26), (28) and (29) to get 2 1+ a γ 1 δ(a+1)k + 1+δ ak +2 ak x T c T c 1 + ω j x Tj T c 1 h 2 1 δ(a+1)k γ, (3) 1+δak a (28)

13 with the condition that the denoinator is positive, equivalently δ ak + a γ 2 δ (a+1)k < a γ 2 1. ACKNOWLEDGMENTS The authors would like to thank Rayan Saab for helpful discussions and for sharing his code, which we used for conducting our audio experients. Both authors were supported in part by the Natural Sciences and Engineering Research Council of Canada (NSERC) Collaborative Research and Developent Grant DNOISE II ( ). Ö. Yılaz was also supported in part by an NSERC Discovery Grant. REFERENCES [1] Donoho, D., Copressed sensing., IEEE Transactions on Inforation Theory 52(4), (26). [2] Candès, E. J., Roberg, J., and Tao, T., Stable signal recovery fro incoplete and inaccurate easureents, Counications on Pure and Applied Matheatics 59, (26). [3] Candès, E. J., Roberg, J., and Tao, T., Robust uncertainty principles: exact signal reconstruction fro highly incoplete frequency inforation, IEEE Transactions on Inforation Theory 52, (26). [4] Donoho, D. and Elad, M., Optially sparse representation in general (nonorthogonal) dictionaries via 1 iniization, Proceedings of the National Acadey of Sciences of the United States of Aerica 1(5), (23). [5] Gribonval, R. and Nielsen, M., Highly sparse representations fro dictionaries are unique and independent of the sparseness easure, Applied and Coputational Haronic Analysis 22, (May 27). [6] Foucart, S. and Lai, M., Sparsest solutions of underdeterined linear systes via q -iniization for <q 1, Applied and Coputational Haronic Analysis 26(3), (29). [7] Saab, R., Chartrand, R., and Yilaz, O., Stable sparse approxiations via nonconvex optiization, in [IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)], (28). [8] Chartrand, R. and Staneva, V., Restricted isoetry properties and nonconvex copressive sensing, Inverse Probles 24(352) (28). [9] Saab, R. and Yilaz, O., Sparse recovery by non-convex optiization instance optiality, Applied and Coputational Haronic Analysis 29, 3 48 (July 21). [1] Chartrand, R., Exact reconstruction of sparse signals via nonconvex iniization, Signal Processing Letters, IEEE 14(1), (27). [11] von Borries, R., Miosso, C., and Potes, C., Copressed sensing using prior inforation, in [2nd IEEE International Workshop on Coputational Advances in Multi-Sensor Adaptive Processing, CAMPSAP 27.], ( ). [12] Vaswani, N. and Lu, W., Modified-CS: Modifying copressive sensing for probles with partially known support, arxiv:93.566v4 (29). [13] Jacques, L., A short note on copressed sensing with partially known signal support, Signal Processing 9, (Deceber 21). [14] Ain Khajehnejad, M., Xu, W., Salan Avestiehr, A., and Hassibi, B., Weighted l1 iniization for sparse recovery with prior inforation, in [IEEE International Syposiu on Inforation Theory, ISIT 29], (June 29). [15] Friedlander, M. P., Mansour, H., Saab, R., and Özgür Yılaz, Recovering copressively sapled signals using partial support inforation, to appear in the IEEE Transactions on Inforation Theory. [16] Candès, E. J. and Tao, T., Decoding by linear prograing., IEEE Transactions on Inforation Theory 51(12), (25). [17] Cohen, A., Dahen, W., and DeVore, R., Copressed sensing and best k-ter approxiation, Journal of the Aerican Matheatical Society 22(1), (29). [18] van den Berg, E. and Friedlander, M. P., Probing the pareto frontier for basis pursuit solutions, SIAM Journal on Scientific Coputing 31(2), (28). [19] van den Berg, E. and Friedlander, M. P., SPGL1: A solver for large-scale sparse reconstruction, (June 27). [2] Candès, E. J., Wakin, M. B., and Boyd, S. P., Enhancing sparsity by reweighted 1 iniization, The Journal of Fourier Analysis and Applications 14(5), (28).