arxiv: v1 [cs.it] 12 Mar 2014

Transcription

1 COMPRESSIVE SIGNAL PROCESSING WITH CIRCULANT SENSING MATRICES Diego Valsesia Enrico Magli Politecnico di Torino (Italy) Dipartimento di Elettronica e Telecomunicazioni arxiv: v [cs.it] 2 Mar 204 ABSTRACT Compressive sensing achieves eective dimensionality reduction o signals, under a sparsity constraint, by means o a small number o random measurements acquired through a sensing matrix. In a signal processing system, the problem arises o processing the random projections directly, without irst reconstructing the signal. In this paper, we show that circulant sensing matrices allow to perorm a variety o classical signal processing tasks such as iltering, interpolation, registration, transorms, and so orth, directly in the compressed domain and in an exact ashion, i.e., without relying on estimators as proposed in the existing literature. The advantage o the techniques presented in this paper is to enable direct measurement-to-measurement transormations, without the need o costly recovery procedures. Index Terms Compressed sensing, circulant matrix, compressive iltering. INTRODUCTION Compressive sensing (CS) [] has successully shown that a small number o measurements, obtained through a suitable sensing matrix, is able to acquire a sparse signal and enable its exact recovery rom its measurements. Remarkably, knowledge o the sparsity basis is not explicitly needed at the moment o acquisition, but only or recovery. Since the beginnings o CS, it was clear that working in the compressed domain presented advantages in terms o complexity because many tasks do not really require recovery but rather to solve inerence problems. The seminal paper by Davenport et al. [2] on signal processing with compressive measurements provided techniques to perorm detection, classiication, estimation and intererence cancellation tasks. In this paper, we ocus on a dierent, although similar, problem. The goal is to perorm classical signal processing operations such as iltering, interpolation and others directly on the compressive measurements, thereby avoiding to irst apply the computationally expensive reconstruction algorithms in order to later apply the operator on the reconstructed samples. The resulting operation is a measurement-to-measurement transormation, which takes as input the acquired measurements and outputs This work is supported by the European Research Council under the European Communitys Seventh Framework Programme (FP7/ ) / ERC Grant agreement n the measurements o the processed signal, without ever perorming reconstruction. This is achieved by using a circulant sensing matrix, or a distributed circulation property in single node and multi-node scenarios respectively. Conversely, [2] is concerned with parameter estimation rom the measurements. This can also be applied to the estimation o transorm or iltered coeicients, but the estimate is always aected by some error, whereas our proposed method yields an exact solution. 2. BACKGROUND AND NOTATION We use the subscript A [a,b] to denote a submatrix o A composed by the rows indexed by interval [a, b]. The subscript x a denotes a circular shit to the right (downwards or column vectors) by a positions. In the standard CS ramework, introduced in [3], a signal x R n which has a sparse representation in some basis Ψ R n n, i.e., x = Ψθ, θ 0 = k, k n, can be recovered by a smaller vector y R m, k < m < n, o linear measurements y = Φx, where Φ R m n is the sensing matrix. The optimum solution, seeking the sparsest vector compliant with y, is an NP-hard problem, but one can resort to a convex optimization reconstruction problem by minimizing the l norm, provided enough measurements (m k log(n/k)) are available. This algorithm is robust when it is used to reconstruct signals which are not exactly sparse, but rather compressible, meaning that the magnitude o their sorted coeicients (in some basis Ψ) decays exponentially. The most used sensing matrices in the literature are random matrices whose elements are i.i.d. Gaussian random variables. However, circulant matrices have been recently proposed [4][5] because they can be implemented easily or arise naturally in various problems. Circulant matrices have been shown to be as eective as Gaussian sensing matrices, when the signal is acquired in its sparsity domain and have a slightly reduced perormance otherwise [6]. The orm o such matrices is φ φ 2 φ 3 φ n φ n φ φ 2 φ n Φ =. φ n m2 φ n m3 φ n m4 φ n m where the irst row, called seed in the ollowing, is drawn at

2 Φ H - = H ~ Fig.. Partial commutation condition random (e.g., i.i.d. Gaussian). Other constructions (e.g., randomly selecting m rows rom the ull matrix) are reported in [6], but we will not consider them in this paper. The n-point circular convolution o two sequences x R n and h R N is x = h x = Hx, where H is an n n circulant matrix. The commutator o two linear operators A R n n and B R n n is zero i and only i the two operators commute and is deined as Φ [A, B] = AB BA. () 3. SINGLE-NODE COMPRESSIVE FILTERING The main goal o this paper is to show that it is possible to compute a measurement-to-measurement transormation that allows us to ind the measurements o a iltered version o the acquired signal, directly rom its linear measurements. The classic iltering operation consists in convolving the ilter impulse response with the signal, and circular convolution can be used i periodic boundary conditions are considered. This amounts to taking the product between a square circulant matrix H and the signal o interest. It can be observed that square circulant matrices orm a commutative group with respect to matrix product, hence i we used a square circulant sensing matrix Φ we would have y = ΦHx = HΦx = Hy. CS uses rectangular sensing matrices in order to achieve compression, hence Φ is a partial circulant matrix. Nevertheless, we can still exploit a partial commutation property. Let us deine an extension o the commutator introduced in () to handle rectangular matrices. The m-partial commutator is deined as: = ΦH HΦ (2) where H is the submatrix o H restricted to the irst m rows and m columns. The ollowing theorem shows that obtaining the measurements o a iltered version o the signal is as straightorward as iltering the measurements. The price to pay or this operation is the corruption o N measurements in the positions corresponding to the non-zero rows o, thus the impulse response should not be too long to avoid corruption o too many measurements, and acquisition should take an extra N measurements to account or this. A graphical depiction o Theorem is shown in Fig.. Theorem. Let H be an n n circulant matrix obtained rom an impulse response o length N, Φ be an m n partial circulant sensing matrix, measurements y = Φx, ( and measurements o the iltered signal y = ΦHx. Then, Hy ) = (y ) i i and only i i [, m N ]. i 2 y 2 5 y 3 3 Fig. 2. Second derivative computation in multi-node scenario Proo. First, we notice that it is enough to prove that only that the irst m N = m rows o the m-partial commutator are zero. Let us call C R n n the square circulant matrix having the same seed as Φ, H T = H [,m ] and call HT = H [,m ]. Then, ([C, H]) [,m ] = (CH) ( ) [,m ] (HC) [,m ] = ( ) [,m ] HΦ (HC) [,m ] = y 4 [,m ] ( ) [,m ] H T Φ H T C = 0. Since, by assumption, the irst row o matrix H has at most N non-zero entries (all in the irst N positions), we have ( ) HT T = 0 and H [m,n] T = ( (H ) ) T ( T (H ) T T [,m], so that T HT C = T [,m]) Φ = HT Φ. We thus ind ( ) [,m ] = 0. The last N rows o are surely non-zero because ( ) HT T 0. [m,n] 4. MULTI-NODE COMPRESSIVE FILTERING In this section we deal with an extension o the previous ramework, which can encompass two scenarios. The irst one considers a multi-node distributed system in which agents cooperate by sharing some measurements in order to perorm the iltering operation. The second one is a single-node system having as many measurements as those rom all the nodes in the previous interpretation, but with a structured sensing matrix. Let us describe the principles o operation reerring to the ormer scenario or ease o explanation, but always keeping in mind the latter. In the multi-node scenario, each node is equipped with a random sensing matrix (e.g., Gaussian), not circulant. However, nodes are assigned matrices that have a circulant property over the network. Numbering the nodes rom to J, this means that Φ () is drawn at random, then row i in matrix Φ (j) is a circularly right-shited version o the same row in Φ (j ). I all the nodes observe the same signal x, then they can collaborate to perorm the iltering operation in the compressed domain. Given an impulse response h, node j can obtain the measurements o the iltered signal y (j) = Φ (j) x as: N y (j) = h i y (j i) or j [N, J] (3) i=0 We can derive a commutation theorem similar to Thm., by deining the (J, m)-distributed partial commutator: ] [ Φ, H = ΦH (H J I m ) Φ (4) J,m 4

3 where Φ [ ] T = Φ ()T Φ (2)T Φ (J)T, HJ is a submatrix o H restricted to the irst J rows and columns, I m is the m m identity matrix, and denotes Kronecker product. Theorem 2. Given J nodes, ( the ) l-th measurement o the iltered signal at node j is y (j) = N ( i=0 h ) i y (j i) l l i and only i j > N. We skip the proo or brevity and because it ollows that o Thm., now using the (J, m)-distributed partial commutator. The main advantage o this multi-node ramework is that each node has a ully random matrix on its own. Thereore, i its number o measurements is suiciently high, it can perorm reconstruction without suering the slight perormance degradation shown by circulant matrices when the sparsity domain is not the identity. Moreover, no measurement is corrupted in this process, except or N nodes, which cannot compute the iltered measurements with respect to their own sensing matrix. Moreover, multiple nodes can stack their measurements, corresponding to the second scenario discussed at the beginning o the section. In this case, the resulting sensing matrix would be obtained by stacking the individual sensing matrices, thus having a block-circulant structure (a total o m independent rows, used as seed o J circulant blocks). 5. MAIN APPLICATIONS OF THE PROPOSED FRAMEWORK The ollowing sections describe some applications o the presented ramework. For reasons o brevity, we shall ocus on the single-node case, but all the results can be readily extended to the multi-node scenario. The general case o iltering using circular convolution with a sequence h has already been treated in the previous section. The ollowing examples consider problems that can be cast in a similar manner. 5.. Finite dierences It is o interest to apply a discretized dierential operator to the signal and compute the measurements o the resulting sequence directly rom the original measurements. Let us consider a discretization o the second derivative, so that the n-th sample in the output is computed, in vector orm, as: x = x 2x x where the subscript k denotes a circular right-shit by k positions. It is easy to check that the measurements y = Φx can be obtained directly in the compressed domain as: y = y 2y y with the corruption o the irst measurement. In the multinode setting it is immediate to derive that node j can request the measurements o nodes j and j in order to compute Φ (j) x directly in the compressed domain: y (j) This example is shown in Fig. 2. = y (j ) 2y (j) y (j) 5.2. Compressive interpolation The objective is to compute the measurements o an interpolated version o the signal o interest rom its compressive measurements. Interpolation is modelled as the cascade o an upsampler and a low-pass ilter. Thereore it is possible to perorm the operation in the compressed domain i we assume that the original signal was acquired by a circulant matrix with a ew zeroed columns, whose number depends on the upsampling actor. The low-pass ilter is implemented by the standard compressive iltering technique explained in the previous sections, bearing in mind that N measurements will be corrupted. As an example, let us consider a piecewise linear interpolator with an interpolation actor o 2. It is easy to veriy that the interpolated sequence and the corresponding measurement-domain transormation are: x INT = 2 x 2 x x y INT = 2 y 2 y y In this example the original measurements have been acquired as y = ˆΦx, where the i-th column o ˆΦ is ˆΦ i = Φ 2i or a given circulant matrix Φ. The measurements o the interpolated signal with respect to matrix Φ are now available Shit retrieval and registration The problem o integer shit retrieval in the compressed domain was considered in [7]. The authors showed that the test max s R { z, ΦD s Φ v } exactly recovers the integer shit s rom as low as one measurement (z and v are the measurements o the signal and o its shited version using cyclic shit operator D s ), given some conditions on Φ. The partial Fourier matrix was considered as an example o sensing matrix satisying the prescribed properties. We notice that circulant matrices do not satisy the conditions listed in Theorem in [7]; in particular, αφ Φ I or any scaling actor α. However, we can exploit Theorem to perorm shit retrieval in a dierent way. One o the implications o Theorem is that a shit in the signal corresponds to a shit in the measurements by the same amount, being the cyclic shit operator D s a circulant matrix, with the corruption o s measurements. Theorem 3. Let Φ R m n be a partial circulant matrix. Let z = Φx and v = Φx s. I s < m, then (5) retrieves the correct shit ŝ = s. ŝ = arg min s z ṽ 2 (5) with z = z [,m s] and ṽ = v [s,m] or s 0, or z = z [ s,m] and ṽ = v [,m s ] or s < 0.

4 Proo. Let us call C R n n the square circulant matrix having the same seed as Φ. Suppose that 0 < s < m. I s = s, we have ṽ = C [s,m]x s = C [,m s ]x, and z = C [,m s ]x, hence ṽ = z, so z ṽ 2 = 0. I s > s or s < s we have ṽ = C [s,m] x s = C [s s,m]x [ T and ṽ = C[m s T,m] CT [,m s s]] x, respectively and z = C [,m s] x in both cases. Hence, by construction o C, we have ṽ z. The same reasoning applies or m < s < 0. Thus, we proved retrieval o the correct shit. When s m, ṽ and z cannot be constructed because ṽ and z contain measurements coming rom two disjoint submatrices o C. Notice that thanks to compressive interpolation (section 5.2), it is also possible to retrieve sub-integer shits; we leave this as uture work. A dierent problem is registration. I we know the shit s, we can register a signal, meaning that we can compute the measurements o the shited version simply by shiting the measurement vector. I want to compute the measurements with respect to the original matrix Φ, then s measurements will be corrupted by the registration. Otherwise, we can suppose that the registered measurements are measurements acquired with a dierent sensing matrix Φ, which is circulant, having as seed the (m s )-th row o the ull circulant matrix, that the partial Φ was extracted rom Compressive wavelet transorm We show that it is possible to obtain the measurements o the wavelet coeicients rom the measurements o the signal directly in the compressed domain. Changing the signal domain ater the sensing process can be useul in several ways. In [8], the authors propose a technique to improve the quality o the reconstruction o signals compressible in the wavelet domain by exploiting the tree-based structure within the wavelet coeicients. The method requires CS measurements o the wavelet coeicients in order to be applied. However, in many scenarios it is not practical to sense the wavelet coeicients, either because the sensing is implemented in a low-complexity, low-power hardware so that calculating the transorm would be computationally and energetically costly or because specialized hardware directly acquires random projections o the data (e.g., [9]). Moreover, being able to compute the measurements o the wavelet coeicients, which typically hold a sparse or compressible representation o the signal, allows to recover them directly, thus avoiding any issues regarding the coherence [0] o sensing matrix and spasiying wavelet dictionary. In order to implement the wavelet transorm in the compressed domain we exploit the liting scheme []. The liting method to compute the wavelet transorm consists in a sequence o liting steps, each composed by two ilters operating on two subsequences. Initially, the two subsequences are the even and odd samples o the signal; the predict ilter is used to predict the odd sequence rom the even one, and the Φ E Φ Ο y E y O λ y a y b y c λ 2 Fig. 3. Compressive wavelet transorm y d K 0 K Φθ dierence is then passed through the update ilter and the result summed back to the even sequence. It can be noticed that this ramework can be mapped in the compressed domain because the iltering steps can be implemented as explained in the previous sections under the choice o periodic boundary conditions. We thus only need to impose a structured sensing that acquires measurements o the even and odd subsequences separately. Let us call Φ (e) and Φ (o) the sensing matrices used to acquire the even and odd subsequences, respectively. Both have size m n, but the even (odd) columns are zeroed in Φ (o) (Φ (e) ). Moreover, the sum Φ (e) Φ (o) is a circulant matrix. The measurements o the even and odd sequences orm the two input sequences in the compressive scheme, depicted in Fig. 3. The prediction step applies the prediction ilter to the even sequence and the output is let-shited by. The update step applies the update ilter and right-shits the output by. This is the structure o a single liting step, which may be repeated depending on the particular transorm to be implemented. Finally, gains are applied and the two sequences are summed. It can be shown that the inal output is Φθ being θ the vector o wavelet coeicients. We remark that, as explained in section 3, 2(N ) 2(N ) measurements per liting step are corrupted. In more detail, let us consider a practical example taken rom [2]. The spline 5/3 biorthogonal transorm has the ollowing prediction and update ilters λ (z) = 2 (z) and λ 2(z) = 4 (z ) and gain actors K 0 =, K = 2. Applying the compressive transorm procedure we can obtain the measurements o the wavelet coeicients with a corruption o the irst 2 and last 2 measurements, which will be discarded. 6. CONCLUSIONS In this paper we have shown how endowing the sensing matrix with circulant properties (e.g., ully circulant as in the singlenode case or block-circulant as in the multi-node case) allows to translate a variety o classical signal processing tasks to the reduced dimensionality domain in an exact orm or corrupting ew measurements. We have discussed some applications o the presented paradigm, including various orms o iltering, shit retrieval and registration and a technique to transorm the measurements o a signal into the measurements o its wavelet coeicients. Future directions or the work presented in this paper include extending the theory to 2D signals.

5 7. REFERENCES [] D. L. Donoho, Compressed sensing, IEEE Transactions on Inormation Theory, vol. 52, no. 4, pp , [2] M. A. Davenport, P. T. Bouounos, M. B. Wakin, and R. G. Baraniuk, Signal Processing With Compressive Measurements, Selected Topics in Signal Processing, IEEE Journal o, vol. 4, no. 2, pp , 200. [3] E. J. Candes and T. Tao, Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? IEEE Transactions on Inormation Theory, vol. 52, no. 2, pp , [4] W. U. Bajwa, J. D. Haupt, G. M. Raz, S. J. Wright, and R. D. Nowak, Toeplitz-Structured Compressed Sensing Matrices, in Statistical Signal Processing, SSP 07. IEEE/SP 4th Workshop on, 2007, pp [5] H. Rauhut, Circulant and Toeplitz matrices in compressed sensing, arxiv preprint arxiv: , [6] W. Yin, S. Morgan, J. Yang, and Y. Zhang, Practical compressive sensing with Toeplitz and circulant matrices, in Proc. SPIE, vol. 7744, 200, p K. [7] H. Ohlsson, Y. C. Eldar, A. Y. Yang, and S. S. Sastry, Compressive shit retrieval, in Acoustics, Speech and Signal Processing, 203. ICASSP 203. IEEE International Conerence on, vol., 203, pp [8] L. He and L. Carin, Exploiting structure in waveletbased bayesian compressive sensing, Signal Processing, IEEE Transactions on, vol. 57, no. 9, pp , [9] J. Yoo, S. Becker, M. Monge, M. Loh, E. Candes, and A. Emami-Neyestanak, Design and implementation o a ully integrated compressed-sensing signal acquisition system, in Acoustics, Speech and Signal Processing (ICASSP), 202 IEEE International Conerence on, 202, pp [0] E. Candes and J. Romberg, Sparsity and incoherence in compressive sampling, Inverse problems, vol. 23, no. 3, p. 969, [] W. Sweldens, The liting scheme: a construction o second generation wavelets, SIAM J. Math. Anal., vol. 29, no. 2, pp , Mar [Online]. Available: [2] D. S. Taubman, M. W. Marcellin, and M. Rabbani, JPEG2000: Image compression undamentals, standards and practice, Journal o Electronic Imaging, vol., no. 2, pp , 2002.