An RIP-based approach to Σ quantization for compressed sensing

Transcription

1 An RIP-based approach to Σ quantization for copressed sensing Joe-Mei Feng and Felix Kraher October, 203 Abstract In this paper, we provide new approach to estiating the error of reconstruction fro Σ quantized easureents for copressed sensing. Our ethod is based on the restricted isoetry property (RIP) of a certain projection of the easureent atrix. The ain application of our result is the error analysis for partial rando circulant atrices. This is the first tie such bounds are provided for a structured easureent atrix with the fast ultiplication property. Our results also recover the best-known reconstruction error bounds for Gaussian and subgaussian easureent atrix. Introduction. Copressed sensing Copressed sensing has drawn significant attention since the seinal works by Candés, Roberg, Tao [6], and Donoho [0]. The theory of copressed sensing is based on the observation that various cases of natural signals are approxiately sparse with respect to certain bases or fraes. The basic idea is to recover such signals fro a sall nuber of linear easureents. Hence the proble turns into an undeterined linear syste. Various criteria have been proposed to deterine whether such a syste has a unique sparse solution. In this paper we will work with the restricted isoetry property (RIP), which has been shown by Candés et al. [8] to guarantee uniqueness. Definition. A atrix A R N has the restricted isoetry property (RIP) of order s if there exists 0 < δ < such that for all s-sparse vectors x C N, i.e., vectors that have at ost s non-zero coponents, ( δ) x 2 2 Ax 2 2 ( + δ) x 2 2. The sallest such δ is called the RIP-constant of order s denoted by δ s. Finding the RIP-constant of a easureent atrix is in general an NP hard proble [23]. That is why ost papers work with rando atrices. Exaples of rando atrices known to have the RIP include subgaussian, partial rando circulant, and partial Fourier atrices. A subgaussian atrix has independent rando entries whose tail is doinated by a Gaussian

2 (cf. Definition 3). Such atrices have been shown to have the RIP provided = Ω(s log(en/s)) [8]. This order of the ebedding diension is known to be optial. Exaples of subgaussian atrices include Gaussian and Bernoulli atrices. A partial rando circulant atrix is the atrix representation of a deterinistically subsapled rando convolution. Such a atrix has been shown to have the RIP provided = Ω(s(log s) 2 (log N) 2 ) []. A partial rando Fourier atrix consists of rows of a discrete Fourier atrix drawn at rando. The RIP for such a atrix has been shown provided = Ω(s log 3 s log N)) [2]. These ebedding diensions are slightly worse than for subgaussian atrices, but such structured atrices are preferred in any applications, e.g., because of their reduced randoness and their fast ultiplication properties..2 Quantized copressed sensing While the easureents in copressed sensing are usually not representable by finite bits, we need to quantize these easureents such that the syste can be operated on coputers. To quantize the easureents we seek to represent our easureents by finite any sybols fro a finite alphabet consisting of real nubers. The extree case of considering the set of only two eleents {, } is also called -bit quantization. The ost intuitive ethod to quantize the easureents is to ap each of the easureents to the closest eleent fro the alphabet set. Since this ethod processes the quantization independently for each easureent, it is also called eoryless scalar quantization (MSQ). Most of the literature on MSQ quantization copressed sensing up to date consider -bit quantization [5, 7, 9, ], which bias down to considering only the easureent signs. Jacques et al. [7] show for Gaussian easureents or easureents drawn uniforly fro the unit sphere, the worst case error is bounded by O( s N log s ). Later, for Gaussian easureents, Gupta et al. [4] deonstrate that one ay tractably recover the support of the signal fro O(s log n) easureents. Plan et al. [9] show that one can again for Gaussian easureents, reconstruct the direction of an s-sparse signal via convex optiization, with accuracy O( s ) 5 up to logarithic factors with high probability. Later Ai et al. [] derive siilar results for subgaussian easureents under additional assuptions on the size of the signal entries. However in [7] it is shown that the reconstruction l 2 -error can never be better than Ω( s ). To break this bottleneck of MSQ, Σ quantization for copressed sensing has drawn attention recently. Σ quantizes a vector as a whole rather than the coponents individually, i.e., the quantized values depend on previous quantization steps. Güntürk et al. [3] show that for rth order Σ quantization in Gaussian copressed sensing case, the l 2 -error is bounded by O(( s )α(r 2 ) ) for any 0 < α < with high probability. More recently, in [8], this result has been generalized to subgaussian easureents. Indeed for r large enough this breaks the MSQ bottleneck..3 Contributions The priary contribution of this paper is that the restricted isoetry property (RIP) is applied to estiate the error bound for Σ quantized copressed sensing. That is, once we know the RIP-constant of a odification of the easureent atrix, we can estiate the reconstruction error. 2

3 In the following results, we assue that the Σ quantized easureents with quantization alphabet Z = Z are given to us. We refer the readers to Section 2.2 for details on the quantization schee eployed. A special role is played by the rth power finite difference atrix D; denoting the singular value decoposition of D r by D r = U D rs D rv D r, we obtain our ain theore below. Theore. Given an s-sparse signal x R N, denoted by Φ R N a easureent atrix, and q the rth order Σ -quantized easureents of Φx with step size. Suppose Φ has the RIP such that the support set T can be deterined. Choose L as the Sobolev dual atrix of Φ T and reconstruct the signal by ˆx = Lq (see Section for details), if l P lvd Φ, l, has r RIP-constant δ s δ, where P l aps a vector to its first l coponents. then the reconstruction error is bounded above by x ˆx 2 2c 2 (r) ( δ) ( l ) r+ 2, where c 2 (r) > 0 is a constant depending only on r. Note fro Theore, the saller l is the better the bound. However, l has to be large enough such that (P l VD Φ) has the RIP-constant δ r s δ. This result can be applied to obtain recovery guarantees for various copressed sensing setting such as Gaussian, subgaussian and partial rando circulant easureents. We will show in later section that our result covers which in [3] and [8]. Since the result of reconstruction bound on rando circulant atrix has not been proposed so far, we state it as the following. Theore 2. Given an s-sparse signal x R N, let Φx = P Ω (ξ x) R N be a partial rando circulant atrix, where the ξ i s are independent ean-zero, ρ-subgaussian rando variables (see Definition 3) of variance one and q the rth order Σ -quantization of Φx with step size. If C s log 2 2α s log 2 2α N, where 0 α 2, C depends only on ρ, and large enough such that the support set T is deterined, choosing L as the Sobolev dual atrix of Φ T, reconstructing the signal by ˆx = Lq, then, with probability at least 0.99, the reconstructed error is bounded by x ˆx 2 ρ ( s ) α(r 2 ), the notation ρ eans less than or equal to up to a constant depending on ρ..4 Organization The following paper is organized as the following. We first introduce the background and previous results on Σ quantization, suprea of chaos process, and the restricted isoetry property for partial rando circulant atrices. Further we present our ain result in Section 3, where we show how the RIP is 3

4 used to estiate reconstruction error for quantized copressed sensing. In Section 4, we show that our result can cover the best-known bounds for Gausssian and subgaussian easureent atrices. In the last section, we apply our result on partial rando circulant easureent atrices. This is the first tie such bounds are provided for a structured easureent atrix with the fast ultiplication property. 2 Background and previous results 2. Notations Through out this paper we use the following notations. The set D s,n = {x C x 2, x 0 s} is the set of unit nor s-sparse vectors. l 0 -nor 0 counts the nuber of non-zero coponents of a vector. Denote the Frobenius nor by A F = tra A, and l 2 -operation nor A 2 2 = sup x 2 = Ax 2. Based on last two nors d F (B) = sup A B A F, and d 2 2 (B) = sup A B A 2 2. We also assign orders to singular value of a atrix as σ i ( ) and σ in ( ) indicating the ith largest and the sallest singular value of a atrix respectively. Also denoted by, up to a positive constant, while eans up to a positive constant. Denoted by the Moore-Penrose inversion operator. 2.2 Σ Quantization Σ quantization is originally introduced as an efficient quantizer for redundant representations of oversapled band-liited functions [6]. Later on the atheatical analysis is provided by [9, 2] and any follow-up papers, and further the schee has been extended to frae expansions [3]. Given an alphabet Z such that Z = Z, the idea of rth order Σ quantization is to quantize each coponent of a vector taking the previous r quantization steps into account. More explicitly, a greedy rth order Σ quantization aps a sequence of inputs (y j ) to eleents q i Z via an internal state variable u i, explicitly ( r u) i = r j=0 ( r j ) ( ) j u i j = y i q i, () where q i is chosen such that u i is iniized. Setting initial conditions (u i ) i=0 = 0, then equation () can be expressed as D r u = y q, where the finite difference atrix is given by, if i = j, D ij, if i = j +, 0, otherwise. (2) 2.2. Coarse recovery Given an s-sparse signal x, and an N easureent atrix Φ, where N, we acquire easureents y = Φx. Applying rth order Σ quantization to y, 4

5 we have q. If treat q as perturbed easureents, i.e., q = y + e = Φx + e, then by [3], we can deterine the support set. This is proved by a odified version of Proposition 4. in [3] and Theore in [7], which says Proposition. Given x R N an s-sparse signal, denote e a noise vector with e 2 ϵ, and let Φ R N be a easureent atrix. Reconstruct x fro q = Φx + e via l iniization we obtain x, i.e., ˆx = arg in z subject to Φz q 2 ϵ. If Φ has RIP-constants such that δ 3s + 3δ 4s < 2, then x ˆx 2 K η, and denote T the support set of x and s = T, and if in j T x j K2 r 2, j T, for soe positive constant K, then the largest s coponents of x is T. Note the factor which akes Φ a unit-nor-coluned atrix, while in the copressed sensing literature, this is coon to noralize the easureent atrix such that it has unit-nor coluns. However for quantization copressed sensing, if the easureent size varies depending on the scale, then to analysis the reconstruction error it is not justified to use a fix step size. To fairly copare the error while we let grow, we should leave our easureent entries independent of, therefore, in this paper the easureent atrices are not noralized, and instead, for convenience we set each entry of the easureent atrices to have variance one Σ error estiate and Sobolev dual According to the previous section, denote the support set of x by T, we solve x by ultiplying a left inverse of Φ T, say L. Then the reconstruction l 2 -error is given by x ˆx 2 = Ly Lq = L(y q) 2 = L(D r u) 2 LD r 2 2 u 2. The Sobolev dual atrix L sob,r, first introduced in [4], is a left inverse of Φ T defined to iniize LD r 2 2, i.e., in L LD r 2 2 subject to LΦ T = I. The geoetric intuition is that the frae is soothly varying. Using the explicit forula L sob,r D r = (D r Φ T ), we obtain the error bound x ˆx 2 (D r Φ T ) 2 2 u 2 = σ in (D r Φ T ) u 2. (3) Recall that u 2 2, once we find a bound for σ in (D r Φ T ) fro below we can bound x ˆx 2 fro above. A key ingredient to bound this singular value is the following result fro the study of Toeplitz atrices, which depends highly on Weyl s inequality [5] (see also for exaple in [3]). Proposition 2. Let r be any positive integer and D be as in (2). There are positive constants c s (r) and c s2 (r), independent of, such that c s (r)( j )r σ j (D r ) c s2 (r)( j )r, j =,...,. (4) 5

6 2.3 Suprea of chaos process Based on [], in this section we will see one of the ost iportant tools for our ain result, which is Theore 3. Definition 2. For a etric space (T, d), an adissible sequence of T is a collection of subsets of T, {T r : r }, T r 2 2r and T 0 =, define γ 2 function introduced by Talagrand [22] as γ 2 (T, d) = inf T r sup t T 2 r/2 d(t, T r ), γ 2 function can be bounded by a Dudley integral [22]. More specificaly, the γ 2 function used in the following theore is bounded by γ 2 (B, 2 2 ) d2 2(B) 0 r=0 log /2 N(B, 2 2, u)du, where N(B, 2 2, u) is the covering nuber of B with respect to the nor 2 2 and the radius u. A key ingredient for out ain result will be the following theore. Theore 3. [] Let B be a set of atrices, and let ξ be a rando vector whose entries are independent, ean zero, variance one, and ρ-subgaussian rando variables (see Definition 3). Then P(C B c E + t) 2 exp( c 2 in{ t2 V 2, t }), (5) U where c, c 2 > 0 are constants depending on ρ, C B := sup Aξ 2 2 E Aξ 2 2, A B where and E = γ 2 (B, 2 2 )(γ 2 (B, 2 2 ) + d F (B)) + d F (B)d 2 2 (B), V = d 2 2 (B)(γ 2 (B, 2 2 ) + d F (B)), U = d 2 2 2(B). 2.4 The restricted isoetry property for partial rando circulant atrices In this section we will show the RIP for partial rando atrices, which is selected fro []. The ain ingredient to define a partial rando circulant atrix is an ρ- subgaussian rando variable as below. Definition 3. A rando variable X is called ρ-subgaussian if P( X t) 2 exp( t 2 /2ρ 2 ) 6

7 Based on this definition we can define a partial rando circulant atrix corresponding to the atrix representation of a subsapled rando convolution. Then the definition of a partial rando circulant atrix is Definition 4. Given a rando vector ξ = (ξ i ) N i=, where the ξ i s are independent ean-zero, ρ-subgaussian rando variables of variance one. A partial rando circulant atrix Φ,N, generated by ξ is defined via Φ,N x = P Ω (ξ x), where P Ω is a deterinistic projection to coponents of a vector. [] states the RIP for partial rando circulant atrix Φ as below. Theore 4. Let ξ = (ξ j ) N j= be a rando vector with independent ean-zero, variance one, ρ-subgaussian entries. If for s N and η, δ (0, ), cδ 2 s ax{(log s) 2 (log N) 2, log(η )}, (6) then with probability at least η, the RIP-constant of the partial rando circulant atrix Φ R N generated by ξ satisfies δ s δ. The constant c depends only on ρ. 3 RIP-based error analysis In this section we will give the quantized copressed sensing proble a atheatical odel, and explain how we approach the reconstruction error via the restricted isoetry property. In the next two sections we show its applications. Fro Section 2.2.2, the ain issue to estiate the reconstruction error is to estiate σ in (D r Φ T ). Coparing to σ in (D r Φ), since the doain is restricted, σ in (D r Φ T ) σ in (D r Φ). The intuition is to find a suprea of the effective sallest singular value of σ in (D r Φ) while the doain is restricted on D s,n. This restriction otivates the concept of RIP. In the following proof we show how RIP can be applied to find this effective sallest singular value. Proof of Theore. Recall that D r = U D rs D rv D r. Then, as s is a diagonal atrix, σ in (D r Φ T ) = σ in (S D rvd rφ T ) σ in (P l S D rvd rφ T ) = σ in ((P l S D rpl )(P l VD rφ T ) l s ) s l σ in (P l VD rφ T ) l s, Next we need to bound σ in (P l VD Φ r T ) uniforly over all support set T. If l P l VD Φ RIP-constant δ r s δ then σ in (P l VD Φ r T ) l s is uniforly bounded fro below by l δ. (7) The theore follows by applying (3), (4), (7) as σ in (D r Φ T ) u 2 2c 2 (r) ( δ) ( l ) r+ 2 (8) 7

8 4 Gaussian and subgaussian atrices Given Φ a standard Gaussian rando atrix. Since (P l V D r Φ) is also a standard Gaussian rando atrix due to rotation invariance, with l = Ω(s log N), l (P l V D r Φ) has the RIP-constant δ s < δ with high probability [20]. Since s l, l ( s )α, α (0, ). Provided that l = Ω(s log N), Apply Theore directly, we obtain l ( s )α s log N ( s )α s(log N) α. x ˆx 2 ( s ) α(r 2 ), with high probability. This therefore recovers the result in [3]. By siilar steps, this can also be generalized to subgaussian easureent atrices, which recovering the result in [2]. As this arguent involves soe additional technical steps, we refrain fro presenting the details here. 5 Partial rando circulant atrices In this section we apply our ain theore, Theore, on partial rando circulant atrices to obtain Theore 2. To prove Theore 2 we need the following lea. Lea. Let ξ R N be a rando vector with independent ean-zero, variance one, ρ-subgaussian entries, and denote by Φ R n a partial rando circulant atrix generated by ξ. If l satisfies l C s(log s)(log N), where C is a constant depending only on ρ, then, with probability at least 0.99, l P lv D Φ has the restricted isoetry property with constant δ s 0.. Proof. The proof of this lea is based on Theore 3 Define B = { l P lvd V r x : x D s,n }, then δ s ( l P lvd Φ) = sup x D s,n l P lvd rv xξ 2 2 E l P lvd rv xξ 2 2 = sup x D s,n l P lvd rv xξ 2 2 E x 2 2 = sup Aξ 2 2 E Aξ 2 2 A B = C B, 8

9 since for A B E Aξ 2 2 = Eξ A Aξ = A 2 F = l P lvd rv x 2 F = l P lv x 2 F = x 2 2. Estiating the ters in Theore 3 using the definition of B and the corresponding bounds for A = { V x : x D s,n } derived in [], we obtain and d F (B) = d 2 2 (B) = df (A) l l, s l d 2 2(A) l. To estiate γ 2 (B, 2 2 ), we need to estiate the covering nubers of the set B with respect to the nor l /2 ˆ, while the nor used in the estiation of γ 2 (A, 2 2 ) in [] is /2 ˆ. Paralleling to the arguents to s γ 2 (A, 2 2 ) (log s)(log N), we obtain γ 2 (B, 2 2 ) s (log s)(log N). l Suppose l 3c s(log s)(log N), where c is the sae as in Theore 3, and assue t = δ 2 then by definition of E c E + δ s s s 2 c { (log s)(log N)[ (log s)(log N) + l l l ] + l l } + δ 2 δ, (9) Further by definition of V and U, one has for C large enough and t 2 2 exp( c 2 V 2 ) 2 exp( c 2( s l [ s l (log s)(log N) + ) 0.0 (0) l ])2 δ 2 t 2 exp( c 2 U ) 2 exp( c 2( δ/2 )) 0.0. () s/l Cobining equation (0) and (), we obtain 2 exp( c 2 in{ t2 V 2, t }) 0, 0. (2) U Setting δ = 0., we obtain fro equation (5) that This proves the lea. P (δ s 0.)

10 Proof of Theore 2. To deterine the support set of s-sparse signal x, we need fro Theore 4 that s log 2 s log 2 N. (3) Now choose δ = 0., and l C s(log s)(log N), where C is as in Lea, cobining Lea and Theore. We obtain that the reconstruction error is bounded by x ˆx 2 ( l ) r+ 2 ( ) r+ 2 s(log s)(log N) (( s )) α(r 2 ), This follows fro the fact that (/s) 2α 2 log s log N. This inequality only holds for 2α 2 0, and thus α 2. Together with equation (3), we obtain that 0 α 2 This concludes the proof. References [] Albert Ai, Alex Lapanowski, Yaniv Plan, and Roan Vershynin. One-bit copressed sensing with non-gaussian easureents. Linear Algebra and its Applications, 203. [2] R. Baraniuk, M. Davenport, R. DeVore, and M. Wakin. A siple proof of the restricted isoetry property for rando atrices. Constr. Approx., 28(3): , [3] John J Benedetto, Alexander M Powell, and Ozgur Yilaz. Siga-delta (Σ ) quantization and finite fraes. Inforation Theory, IEEE Transactions on, 52(5): , [4] Jaes Blu, Mark Laers, Alexander M Powell, and Özgür Yılaz. Sobolev duals in frae theory and siga-delta quantization. Journal of Fourier Analysis and Applications, 6(3):365 38, 200. [5] Petros T Boufounos and Richard G Baraniuk. -bit copressive sensing. In Inforation Sciences and Systes, CISS nd Annual Conference on, pages 6 2. IEEE, [6] Eanuel J Candés, Justin K Roberg, and Terence Tao. Stable signal recovery fro incoplete and inaccurate easureents. Counications on pure and applied atheatics, 59(8): , [7] Eanuel J Candés, Justin K Roberg, and Terence Tao. Stable signal recovery fro incoplete and inaccurate easureents. Counications on pure and applied atheatics, 59(8): , [8] Eanuel J Candes and Terence Tao. Decoding by linear prograing. Inforation Theory, IEEE Transactions on, 5(2): ,

11 [9] Ingrid Daubechies and Ron DeVore. Approxiating a bandliited function using very coarsely quantized data: A faily of stable siga-delta odulators of arbitrary order. The Annals of Matheatics, 58(2):679 70, [0] D. L. Donoho. Copressed sensing. IEEE Trans. Infor. Theory, 52(4): , [] Felix Kraher, Shahar Mendelson, and Holger Rauhut. Suprea of chaos processes and the restricted isoetry property. Co. Pure Appl. Math., to appear. [2] C Sinan Güntürk. One-bit siga-delta quantization with exponential accuracy. Counications on Pure and Applied Matheatics, 56(): , [3] C Sinan Güntürk, Mark Laers, Alexander M Powell, Rayan Saab, and Ö Yılaz. Sobolev duals for rando fraes and σδ quantization of copressed sensing easureents. Foundations of Coputational Matheatics, 3(): 36, 203. [4] Ankit Gupta, Robert Nowak, and Benjain Recht. Saple coplexity for -bit copressed sensing and sparse classification. In Inforation Theory Proceedings (ISIT), 200 IEEE International Syposiu on, pages IEEE, 200. [5] Roger A Horn and Charles R Johnson. Matrix analysis. Cabridge university press, 202. [6] Hi Inose, Y Yasuda, and J Murakai. A teleetering syste by code odulation- -Σ odulation. Space Electronics and Teleetry, IRE Transactions on, (3): , 962. [7] Laurent Jacques, Jason N Laska, Petros T Boufounos, and Richard G Baraniuk. Robust -bit copressive sensing via binary stable ebeddings of sparse vectors. 20. [8] Felix Kraher, Rayan Saab, and Rachel Ward. Root-exponential accuracy for coarse quantization of finite frae expansions. Inforation Theory, IEEE Transactions on, 58(2): , 202. [9] Yaniv Plan and Roan Vershynin. One-bit copressed sensing by linear prograing. Counications on Pure and Applied Matheatics, 66(8):63 333, 203. [20] Holger Rauhut. Copressive sensing and structured rando atrices. Theoretical foundations and nuerical ethods for sparse recovery, 9: 92, 200. [2] Mark Rudelson and Roan Vershynin. On sparse reconstruction fro fourier and gaussian easureents. Counications on Pure and Applied Matheatics, 6(8): , [22] Michel Talagrand. The generic chaining: upper and lower bounds for stochastic processes. Springer, 2005.

12 [23] Andreas M Tillann and Marc E Pfetsch. The coputational coplexity of the restricted isoetry property, the nullspace property, and related concepts in copressed sensing. arxiv preprint arxiv: ,