MEASUREMENT MATRIX DESIGN FOR COMPRESSIVE SENSING WITH SIDE INFORMATION AT THE ENCODER

Transcription

1 MEASUREMENT MATRIX DESIGN FOR COMPRESSIVE SENSING WITH SIDE INFORMATION AT THE ENCODER Pingfan Song João F. C. Mota Nikos Deligiannis Miguel Raul Dias Rodrigues Electronic and Electrical Engineering Departent, University College London, UK Departent of Electronics and Inforatics, Vrije Universiteit Brussel, Belgiu iminds vzw, Ghent, Belgiu ABSTRACT We study the proble of easureent atri design for Copressive Sensing (CS when the encoder has access to side inforation, a signal analogous to the signal of interest. In particular, we propose to incorporate this etra inforation into the signal acquisition stage via a new design for the easureent atri. The goal is to reduce the nuber of encoding easureents, while still allowing perfect signal reconstruction at the decoder. Then, the reconstruction perforance of the resulting CS syste is analysed in detail assuing the decoder reconstructs the original signal via Basis Pursuit. Finally, Gaussian width tools are eploited to establish a tight theoretical bound for the nuber of required easureents. Etensive nuerical eperients not only validate our approach, but also deonstrate that our design requires fewer easureents for successful signal reconstruction copared with alternative designs, such as an i.i.d. Gaussian atri. Inde Ters easureent atri design, side inforation, Copressive Sensing, Basis Pursuit. INTRODUCTION Copressive Sensing (CS [, ] is a signal acquisition paradig that leverages signal sparsity to acquire signals with far less easureents than classical sapling schees. It has been applied, for eaple, in edical iaging [3], radar detection [4], sensor networks [5], and copressive video [6]. In any of these applications, one has access to side inforation, a signal analogous to the signal we want to reconstruct. For eaple, in copressive video past reconstructed fraes can be used as side inforation to reconstruct the current frae [7,8]; in edical iaging, prior patient scans can also be used as side inforation in iage reconstruction [9]. One of the ain benefits of using side inforation in these applications, where easureents are epensive, is that it allows reducing the nuber of required easureents for reconstruction. Much research has used side or prior inforation to obtain even lower acquisition rates in CS. This typically involves odifying the reconstruction procedure, e.g., Basis Pursuit (BP [] by using estiates of the support of the signal, either deterinistically, as in odified-cs [ 5], probabilistically, as in [6, 7], or in the for of a signal analogous to the signal to be reconstructed, as in l -l and l -l iniization [8 ]. Other work, such as [], has used side inforation to perfor classification and reconstruction under This work is supported by China Scholarship Council(CSC, UCL Overseas Research Scholarship (UCL-ORS, the VUB-UGent-UCL-Duke International Joint Research Group grant (VUB: DEFIS4, and by EPSRC grant EP/K3366/. Encoder Decoder A y Optiization ˆ w Fig.. CS with Side Inforation at the encoder. probabilistic odels. In all these work, however, side inforation is used to aid only the sparse reconstruction process, not the acquisition one. In practice, side inforation ay be available both at the encoder and decoder. In this paper, the goal is to integrate side inforation into the easureent atri to aid the signal acquisition process. Note that our approach differs fro those where the encoder uses richer odels to iprove perforance, e.g., [3, 4]. We show that, conceptually, our easureent atri design is equivalent to solving a weighted l iniization proble at the decoder. In that sense, our work is closely related to [5], which establishes bounds on the nuber of easureents required to reconstruct sparse signals using weighted l iniization. Although we use tools siilar to the ones in [5] (naely the concept of Gaussian width [6 9] to establish related bounds, our bounds are uch sipler and, as our eperients show, also uch tighter. Proble stateent. Fig. illustrates the situation where side inforation is available at the encoder. Let R n be the signal of interest, fro which we take linear easureents y = A, where A R n is the easureent atri. Then the vector of easureents y is sent to the decoder. In turn, the decoder reconstructs a vector R n fro y by, e.g., solving an optiization proble like BP. If there are enough easureents, should be a good approiation to. In the figure, w R n coprises the side inforation, a vector that we assue to be siilar to. In this contet, we study the proble of designing the easureent atri A at the encoder using the side inforation w, and then analyze the reconstruction perforance of the resulting syste. Contributions. In the presence of side inforation at the encoder, we propose a new design schee for the easureent atri A in which each row vector is independently drawn fro the Gaussian distribution N (, Σ, where the covariance atri Σ R n n is, assued diagonal, is designed according to the side inforation w. Then, based on this design, we establish a bound on the nuber of easureents that guarantees perfect reconstruction of the original signal via solving BP. Eperiental results illustrate the gains of the proposed easureent atri design with respect to the scenarios where no side inforation is used.

2 . BACKGROUND Let R n be the sparse signal of interest. When there is no side inforation available, it is a standard CS proble to reconstruct fro a given vector of easureents y = A. One coon approach is to solve Basis Pursuit []: iniize f( = subject to y = A, where := n i= i denotes the l-nor of. Assuing A R n is coposed of i.i.d. Gaussian entries with zero ean and variance /, there are several different tools to analyse (, for eaple, the Restricted Isoetry Property (RIP [3] and the Gaussian width/distance [6 9]. We will use the latter, since it allows providing sharper perforance characterizations, although with the shortcoing of being applicable only to Gaussian atrices. Concretely, [8] establishes the following theore. Theore (Corollary 3.3 and Proposition 3. in [8]. Let R n be the signal of interest with sparsity s := {i : i }. Given a vector of easureents y = A, where atri A R n is coposed of i.i.d. zero-ean Gaussian rando variables with variance /, then, is the unique optial solution of ( with probability at least ep ( (λ ω(ω, provided ( n s ln + 7 s 5 s +. ( In the above theore, λ := [ g ] denotes the epected length of a zero-ean, unit-variance Gaussian ] vector g N (, I in R. Moreover, ω(ω := [sup g T z denotes the Gaussian width z Ω of the set Ω, where Ω = T f ( S n is the spherical part of the tangent cone T f ( and g N (, I n R n is a zero-ean, unit-variance Gaussian vector. Thus, when no prior inforation is available, the nuber of easureents necessary to recover is O(s ln n. Net, we will see that providing side inforation to the encoder allows obtaining a bound saller than (. 3. MEASUREMENT DESIGN WITH SIDE INFORMATION In this section, we consider the case where side inforation is available at the encoder. We start by presenting our design schee for the easureent atri; then, we analyse the resulting schee and establish a theoretical bound for the nuber of easureents required for perfect reconstruction in subsection 3., and give the proof for the bound in subsection Design Schee Assuing the encoder has access to w, the side inforation, we design the easureent atri A R n as follows: each row of A is generated independently as a realization of a rando Gaussian vector with zero ean and covariance atri Σ = diag(σ,..., σ i,..., σ n R n n, where each variance σ i is given by { if wi σ i = (3 ε (, ] if w i =. In (3, ε is a predefined paraeter that deterines the gains with respect to the no-side-inforation case. Driven by the side inforation w, the intuition of setting ε coes fro energy considerations: less energy should be spent on acquiring the coponents of that the side inforation indicates to be zero. ( bound of eas (/n * : w: I: = { i: * ¹ } I c : = { i: * i = } r i P: = { i: w i ¹ } Fig.. Visualization of the isatch paraeters r and v noralized sparsity (s/n v Bound in ( Bound in (5, ǫ =. Bound in (5, ǫ =.3 Bound in (5, ǫ =.5 Bound in (5, ǫ =.7 Bound in (5, ǫ =.9 Bound in [ref], ǫ =. Bound in [ref], ǫ =.3 Bound in [ref], ǫ =.5 Bound in [ref], ǫ =.7 Bound in [ref], ǫ =.9 Fig. 3. Coparison of our theoretical bound (5 with the classic CS bound (, and Mansour and Saab s bound [5], with r/s =., v/s = Main result: CS with side inforation at the encoder In this section, we analyse the perforance of the resulting CS syste and present a new bound on the nuber of easureents required for successful reconstruction when the easureent atri is designed as above. To present our result, we define two paraeters that capture the aount of isatch between and w: r := {i : i, w i = }, v := {i : i =, w i }. (4a (4b In other words, r counts the nuber of coponents issed by w, and v counts the nuber of coponents that w overestiates as shown in Fig.. Denoting the sparsity of by s and the sparsity of w by s w, there holds r s and v in{s w, n s}. Proposition. Let R n be the signal of interest with sparsity s := {i : i }, and w R n be the side inforation. Let r > and v >, defined as (4a and (4b, denote the two types of isatch between and w. Given a vector of easureents y = A, where A R n is designed as in Section 3., then is the unique optial solution of ( with probability at least ep ( (λ ω(ω, provided ( εs + ( εr ( n ln + 7 s 5 s + ( εv +. (5 Reark. Proposition establishes that our easureent atri design schee reduces the nuber of required easureents fro O(s ln n [cf.(] to O((ε s + ( εr ln n. In particular, if the nuber of coponents issed by w, naely r, is sall and the diension n is large, the reduction in the nuber of easureents can be significant, as the doinant logarithic ter in (5 is reduced rearkably copared to the counterpart ter in the CS bound (, as shown in (6a. On the other hand, (6b ensures that the unfavourable

3 increase fro the non-doinant linear ter in (5 is liited to no ore than the isatch v. ε ; r s = εs + ( εr s. (6a ε ; v = v ( εv. (6b Reark. Proposition also shows that ε deterines the sensitivity of the bound to the quality of the side inforation. For high quality side inforation with sall isatch r and v, a saller ε should be selected. Otherwise, a larger ε is ore favourable. Reark 3. It can also be seen that our bound (5 generalizes the classical CS bound (. Concretely, as shown in Fig. 3, bound (5 asyptotically approaches the classical CS bound ( with the increase of ε. Note that, taking ε =, (5 siplifies to (, as ε s + ( ε r = s and ( ε v =. Reark 4. Section 3.3 will show that solving BP with our easureent atri design as in Proposition is equivalent to solving a weighted l iniization proble at the decoder. In that contet, Mansour and Saab s work [5] established a result siilar to Proposition ; see Theore 5 in [5]. Their result is, however, considerably ore coplicated and looser than ours, as shown in Fig. 3. To copare the roughly, assue n is large enough and set ε. The doinant ter of the bound in (5 becoes r[ ln n, and the doinant ter of the bound in [5, Th.5] becoes (r + v ln n + (r + v ln n + ] (r + v ln n, where r and v are as in ( Outline of the proof of Proposition This section gives the proof outline of Proposition which involves ainly two stages: converting the BP optiization with our easureent atri design to a weighted l iniization with an i.i.d. Gaussian atri and coputing the upper bound for the Gaussian width/distance. Equivalence to weighted l iniization. We notice the equivalence between BP optiization with our easureent atri design A and the weighted l iniization proble with an i.i.d. Gaussian atri Ã. Concretely, proble ( with the easureent atri A as proposed in 3. can be forulated as in s.t. y = A in s.t. y = (AD(D in z Dz (7 s.t. y = Ãz where à := AD and z = D for D := diag(d,..., d n. Note that fro ( to (7, the optiization variable is changed fro to z = D, where the weights atri D is invertible because ε >. The goal of D is to transfor the current non-i.i.d. (anisotropic Gaussian easureent atri A to an i.i.d. Gaussian atri Ã, tha is isotropic in this case. In addition, the diagonal for of D ensures that the optial solution z of (7 has the sae support as. Recall that the rando colun vector X is called isotropic if EXX T = I n. In order to ake à = AD isotropic, each row of à needs to satisfy Eã T i ã i = I n, where ã i = [ã i,, ã ij,, ã in] is the i-th row vector of Ã. Thus, for each eleent, there holds E [ ã ij] = E [ a ij d i ] = d i [Da ij + (Ea ij ] = d i σ i =, since Ea ij =. The variance σ i = Da ij is the i-th eleent in the diagonal of the covariance atri Σ, defined as in (3. Finally, we take D = diag(d,..., d n with { d i = ; if wi = σi / ε; if w i = < ε. (8 Coputation of the bounds. The required nuber of easureents that guarantees successful reconstruction is upper bounded by the Gaussian width [8, 9]. As it is difficult to copute the Gaussian width in a closed for, an upper bound based on Gaussian distance is coputed instead. Concretely, f(z = Dz is a conve function. Suppose / f(z for a given z R n, then [dist ( g, cone f(z ] + = w(ω + (9 guarantees perfect recovery with high probability, where cone f(z is the cone generated by the subdifferential of the objective function f(z at the optial point z. Let dist(g, S := in{ z g : z S} denote the Euclidean distance between a point g and the set S. To copute an upper bound for the Gaussian distance in (9, the objective is decoposed as f(z = Dz = n i= f (i (z i = di zi and the corresponding cone is coputed as n i= ( cone f(z = t f ( (z, t f ( (z,, t f (n (zn, { t di sign(d i zi ; if i I t f (i (z i = I (, t d i =: [ t di, t d i]; if i / I where i =,..., n, I := {i : zi } is the support of z, and I(a, b denotes an interval with centre a and radius b. Then, Jensen s inequality is applied to derive that [ dist ( g, cone f(z ] i I + i / I i [dist ( g i, t d i sign(d i z i ] (a i [dist ( g i, I(, t d i ] (b which holds for arbitrary t >. In our case, t = ε ln (n/s is selected to copute the upper bound. For siplicity, an auiliary function A( for R n is defined as A( = ϕ( + ( + Q(. ( where ϕ( = ep ( / / π is the probability density function of a scalar Gaussian rando variable with zero-ean and unit variance, and Q( = + ϕ(t dt is the Q-function. Then, it is proved that for a scalar zero-ean Gaussian rando variable with unit variance, i.e., g N (,, the Gaussian distance can be epressed as [dist ( g, I(a, b ] = A(b a + A(b + a. ( Specifically, when b =, interval I(a, reduces to a point a, and [dist(g, a ] = A( a + A( a = a +. (3 When the interval is centered at, i.e., a =, there holds [dist ( g, I(, b ] = A(b (4

4 Then, (3 and (4 are plugged into (a and (b to obtain (a = [ + t d ] ( ( n i = s + εs + ( εr ln, s i I (b = i / I A(t d i = (n s va(t + va(t ε = (n sa(t + va(t ε va(t 5 s + ( εv, where the first ter involving s/5 is derived fro Lea 3 in [], / i.e., for all >, and the last ter is fro the π ln( 5 ( property of function A(, that is, a A(α A( ( α/ for < α. Finally, (a and (b are cobined to obtain the upper bound (5. 4. EXPERIMENTAL RESULTS A set of eperients has been conducted to evaluate the efficacy of the proposed easureent atri design. An i.i.d. Gaussian atri is used as the benchark. In the signal reconstruction stage, we use the SPGL Toolbo [3] to solve the sae BP with both an i.i.d. Gaussian atri and our easureent atri design. The paraeter setting is shown in Table. varies fro to with a step of and the relative sparsity s/n varies fro.5 to.6 with a step of.5 so that the nuber of easureents corresponding to the success rate eceeding 85% can be found as the epirical threshold. To indicate the phase transitions, s/ varies fro. to with a step of.5. In these eperients, instances of A are generated for each pair of s and. The eperient results are shown in Fig. 4. Fig. 4(a shows the success rate varying as a function of the nuber of easureents for different sparsity levels. Fig. 4(b copares the theoretical bounds and epirical bounds. Fig. 4(c and Fig. 4(d copares the phase transition for the two types of easureent atri. The results indicate that our easureent atri design outperfors the i.i.d. Gaussian atri for sall isatch, and our bound is tight and practically coincides with the epirical phase transition well. Table. Paraeters setting for the eperients n /n s/n s/ r/s v/s ε.:.5:.6.: CONCLUSIONS In this paper, a new easureent atri design schee is proposed to integrate side inforation at the encoder into a CS syste. Based on the design schee, the perforance of the resulting syste is analysed in ters of the nuber of easureents required for perfect reconstruction. Etensive eperients indicate that our easureent atri design allows reducing the nuber of easureents provided the side inforation has reasonable quality. In addition, it is deonstrated that our theoretical bound is siple and tight. We believe this work can contribute to better design of CS systes in scenarios where side inforation is available, such as edical iaging, sensor networks, and ulti-view caera systes. In the future, we will consider not only the case where the easureents are noisy, but also the scenario where the side inforation is available at both the encoder and the decoder..5 s = s = s = s = 4 6 s = s = s = s = s = (a Success ratio with easureents for different sparsity levels. ρ = s/ ρ = s/ bound of eas (/n noralized sparsity (s/n (b Theoretical (curves and epirical (arkers bounds theor, i.i.d Gauss theor, Our Matri epir, i.i.d Gauss epir, Our Matri theor, i.i.d Gauss δ = /n (c Phase transition for i.i.d. Gaussian atri. theor, Our Matri δ = /n (d Phase transition for proposed easureent atri. Fig. 4. Eperiental results. (a In each sub-figure, the black line with arker represents the recovery perforance of i.i.d. Gaussian atri and BP, the red line with arker represents our easureent atri design with BP. In (c and (d, the colorbar represents the success ratio. For each pair of /n and s/, the higher the succes ratio is, the whiter the point is

5 6. REFERENCES [] D. Donoho, Copressed sensing, IEEE Trans. Infor. Theory, vol. 5, no. 4, pp , 6. [] E. Candès, J. Roberg, and T. Tao, Robust uncertainty principles: Eact signal reconstruction fro highly incoplete frequency inforation, IEEE Trans. Infor. Theory, vol. 5, no., pp , 6. [3] M. Lustig, D. Donoho, and J. Pauly, Sparse MRI: The application of copressed sensing to rapid MR iaging, Magnetic Resonance in Medicine, vol. 58, pp. 8 95, 7. [4] M. Heran and T. Stroher, High-resolution radar via copressed sensing, IEEE Trans. Signal Process., vol. 57, no. 6, pp , 9. [5] W. Bajwa, J. Haupt, A Sayeed, and R Nowak, Copressive wireless sensing, in Intern. Conf. Infor. Process. in Sensor Networks (IPSN, 6, pp [6] V. Cevher, A. Sankaranarayanan, M. Duarte, D. Reddy, R. Baraniuk, and R. Chellappa, Copressive sensing for background subtraction, in European Conf. on Coputer Vision (ECCV, 8. [7] J. F. C. Mota, N. Deligiannis, A. C. Sankaranarayanan, V. Cevher, and M. R. D. Rodrigues, Dynaic sparse state estiation using l -l iniization: Adaptive-rate easureent bounds, algoriths, and applications, in IEEE Intern. Conf. Acoustics, Speech, and Sig. Processing (ICASSP, 5. [8] J. F. C. Mota, N. Deligiannis, A. C. Sankaranarayanan, V. Cevher, and M. R. D. Rodrigues, Adaptive-rate sparse signal reconstruction with application in copressive foreground etraction, IEEE Trans. Signal Process., 5, (in press. [9] L. Weizan, Y. C. Eldar, and D. Ben-Bashat, Copressed sensing for longitudinal MRI: An adaptive-weighted approach, Medical Physics, vol. 4, no. 9, pp , 5. [] S. Chen, D. Donoho, and M. Saunders, Atoic decoposition by basis pursuit, SIAM J. Sci. Cop., vol., no., pp. 33 6, 998. [] N. Vaswani and W. Lu, Modified-CS: Modifying copressive sensing for probles with partially known support, IEEE Trans. Signal Process., vol. 58, no. 9, pp ,. [] M Ain Khajehnejad, Weiyu Xu, Babak Hassibi, et al., Weighted l iniization for sparse recovery with prior inforation, in IEEE Intern. Syp. Infor. Theory (ISIT. [3] S. Oyak, M. A. Khajehnejad, and B. Hassibi, Recovery threshold for optial weight l iniization, in IEEE Intern. Syp. Infor. Theory (ISIT. IEEE,, pp [4] W. Lu and N. Vaswani, Eact reconstruction conditions and error bounds for regularized odified basis pursuit (regodified-bp, in Asiloar Conf. Signals, Systes and Coputers (ASILOMAR. [5] M. P. Friedlander, H. Mansour, R. Saab, and O. Yilaz, Recovering copressively sapled signals using partial support inforation, IEEE Trans. Infor. Theory, vol. 58, no., pp. 34,. [6] J. Scarlett, J. Evans, and S. Dey, Copressed sensing with prior inforation: Inforation-theoretic liits and practical decoders, IEEE Trans. Signal Process., vol. 6, no., pp , 3. [7] E. Zios, J. Mota, M. Rodrigues, and N. Deligiannis, Bayesian copressed sensing with heterogeneous side inforation, in 6 IEEE Data Copression Conference, (DCC6, Snowbird, Utah, USA, Apr. 6. [8] A. Charles, M. Asif, J. Roberg, and C. Rozell, Sparsity penalties in dynaical syste estiation, in IEEE Conf. Inforation Sciences and Systes,, pp. 6. [9] X. Wang and J. Liang, Side inforation-aided copressed sensing reconstruction via approiate essage passing, Preprint: 576v, 3. [] J. F. C. Mota, N. Deligiannis, and M. R. D. Rodrigues, Copressed sensing with prior inforation: Optial strategies, geoetry, and bounds, arxiv preprint arxiv:48.55, 4. [] J. Mota, N. Deligiannis, and M. Rodrigues, Copressed sensing with side inforation: Geoetrical interpretation and perforance bounds, in IEEE Global Conf. on Signal and Inforation Processing (GlobalSIP, 4, pp [] F. Renna, L. Wang, X. Yuan, J. Yang, G. Reeves, R. Calderbank, L. Carin, and M. R. D. Rodrigues, Classification and reconstruction of high-diensional signals fro lowdiensional noisy features in the presence of side inforation, 4. [3] B. Adcock, A. C. Hansen, C. Poon, and B. Roan, Breaking the coherence barrier: A new theory for copressed sensing, Preprint: 4. [4] W. Chen, M. R. D. Rodrigues, and I. J. Wassell, Projections design for statistical copressive sensing: A tight frae based approach, IEEE Trans. Signal Process., vol. 6, pp. 6 9, 3. [5] H. Mansour and R. Saab, Recovery analysis for weighted l -iniization using a null space property, arxiv preprint arxiv:4.565, 4. [6] Y. Gordon, On Milan s inequality and rando subspaces which escape through a esh in R n, Springer, 988. [7] M. Rudelson and R. Vershynin, On sparse reconstruction fro Fourier and Gaussian easureents, Counications on Pure and Applied Matheatics, vol. 6, no. 8, pp. 5 45, 8. [8] V. Chandrasekaran, B. Recht, P. Parrilo, and A. Willsky, The conve geoetry of linear inverse probles, Found. Coputational Matheatics, vol., pp ,. [9] D. Aelunen, M. Lotz, M. Mccoy, and J. Tropp, Living on the edge: Phase transitions in conve progras with rando data, Inforation and Inference: A Journal of the IMA, pp. 7, 4. [3] E. Candès and T. Tao, Decoding by linear prograing, IEEE Trans. Infor. Theory, vol. 5, no., pp , 5. [3] Ewout Van Den Berg and Michael P Friedlander, Probing the pareto frontier for basis pursuit solutions, SIAM Journal on Scientific Coputing, vol. 3, no., pp. 89 9, 8.