Discrete Signal Reconstruction by Sum of Absolute Values

Transcription

1 JOURNAL OF L A TEX CLASS FILES, VOL., NO. 4, DECEMBER 22 Discrete Signal Reconstruction by Sum of Absolute Values Masaaki Nagahara, Senior Member, IEEE, arxiv: v [cs.it] 8 Mar 25 Abstract In this letter, we consider a roblem of reconstructing an unknown discrete signal taking values in a finite alhabet from incomlete linear measurements. The difficulty of this roblem is that the comutational comlexity of the reconstruction is exonential as it is. To overcome this difficulty, we extend the idea of comressed sensing, and roose to solve the roblem by minimizing the sum of weighted absolute values. We assume that the robability distribution defined on an alhabet is known, and formulate the reconstruction roblem as linear rogramming. Examles are shown to illustrate that the roosed method is effective. Index Terms Discrete signal reconstruction, sum of absolute values, digital signals, comressed sensing, sarse otimization. I. INTRODUCTION Signal reconstruction is a fundamental roblem in signal rocessing. Recently, a aradigm called comressed sensing [], [2], [3] has been roosed for signal reconstruction from incomlete measurements. The idea of comressed sensing is to utilize the roerty of sarsity in the original signal; if the original signal is sufficiently sarse, ractical algorithms such as the basis ursuit [4], the orthogonal matching ursuit [5], etc, may give exact recovery under an assumtion on the measurement matrix (see e.g. [3]). On the other hand, it is also imortant to reconstruct discrete signals whose elements are generated from a finite alhabet with a known robability distribution. This tye of reconstruction, called discrete signal reconstruction, arises in black-and-white or grayscale sarse image reconstruction [6], [7], blind estimation in digital communications [8], machinetye multi-user communications [9], discrete control signal design [], to name a few. The difficulty of discrete signal reconstruction is that the reconstruction has a combinatorial nature and the comutational time becomes exonential. For examle, 2-dimensional signal reconstruction with two symbols (i.e. binary signal reconstruction) needs at worst about.5 36 years with a comuter of 34 eta FLOPS (see Section II below), which cannot be executed, obviously. To overcome this difficulty, we borrow the idea of comressed sensing based on l otimization as used in the basis ursuit [4]. Our idea is that if the original discrete signal, say x, includes L symbols r,r 2,...,r L, then each vector x r i is sarse. For examle, a binary vector x on alhabet {, } includes a number ofand, and hence bothx andx+ are sarse. To recover such a discrete signal, we roose to minimize the sum of weighted absolute values of the elements in x r i. The weights are determined by the robability distribution on the alhabet. The roblem is reduced to a standard linear rogramming roblem, and effectively solved by numerical softwares such as cvx in MATLAB [], [2]. For discrete signal reconstruction, there have been researches based on comressed sensing, called integer comressed sensing: [3] has roosed a Bayesian-based method, which works only for binary sarse signals (i.e., - valued signals that contain many s). [9] considers arbitrary finite alhabet that contains. More recently, motivated by decision feedback equalization, [4] rooses to use l otimization for discrete signal estimation under the assumtion of sarsity. [5], [6], [7] also roose methods based on the finiteness of the measurement matrix (i.e., the elements of the measurement matrix are also in a finite alhabet). As mentioned in [5], this tye of integer comressed sensing is connected with error correcting coding. Comared with these researches, the roosed method in this aer considers arbitrary finite alhabet that does not necessarily contain. The remainder of this letter is organized as follows: Section II formulates the roblem of discrete signal reconstruction and discusses the difficulty of the roblem. Section III rooses to use the sum of weighted absolute values for discrete signal reconstruction, and show a sufficient and necessary condition for exact recovery by extending the notion of the null sace roerty [7]. Examles of one-dimensional signals and twodimensional images are included in Section IV. Section V draws conclusions. Notation For a vector x = [x,x 2,...,x N ] R N, we define the l and l 2 norms resectively as x N x n, x 2 x x, where denotes the transose. For a vector x and a scalar r, we define x r [x r,x 2 r,...,x N r]. Coyright (c) 25 IEEE. Personal use of this material is ermitted. However, ermission to use this material for any other uroses must be obtained from the IEEE by sending a request to ubs-ermissions@ieee.org. M. Nagahara is with Graduate School of Informatics, Kyoto University, Kyoto, 66-85, Jaan; nagahara@ieee.org (corresonding author) For a matrix Φ C M N, kerφ is the kernel (or the null sace) of Φ, that is, kerφ {x C N : Φx = }.

2 2 JOURNAL OF L A TEX CLASS FILES, VOL., NO. 4, DECEMBER 22 I n is the n-dimensional identity matrix, and k is a k- dimensional vector whose elements are all, that is, k [,,...,] R k. For two matrices A R M N and B R K L, A B denotes the Kronecker roduct, that is, A B A 2 B... A N B A B R MK NL, A M B A M2 B... A MN B where A ij is the ij-th element of A. For two real-valued vectors x and y of the same size, x y denotes the elementwise inequality, that is, x y means x i y i for all i. II. PROBLEM FORMULATION Assume that the original signal x is an N-dimensional vector whose elements are discrete. That is, x [x,x 2,...,x N ] X N, X {r,r 2,...,r L }, where r i R and we assume r < r 2 < < r L. () If a symbol, say r, is a comlex number, then taking r j Re(r) and r k Im(r) gives a real-valued alhabet, and hence the assumtion () is not restrictive. We here assume that the values of r i are known and the robability distribution of them is given by P(r i ) = i, i =,2,...,L, (2) where we assume i >, L =. (3) Then we consider a linear measurement rocess modelled by y = Φx C M, (4) where Φ C M N. Note that we consider a comlex-valued matrix Φ since Φ can be constructed from e.g. a comlexvalued DFT (Discrete Fourier Transform) matrix; see the image rocessing examle in Section IV. We assume incomlete measurements, that is, M < N. The objective here is to reconstruct x X N from the measurement vector y C M in (4). First of all, we discuss the uniqueness of the solution of the discrete signal reconstruction. We have the following roosition: Proosition : Given Φ C M N, the following roerties are equivalent: (A) If Φx = Φx 2 and both x and x 2 are in X N, then x = x 2. (B) Define the difference set of X as Then X {r i r j : i,j =,2,...,L}. kerφ X N = {}. (5) (C) The matrix Φ is injective as a ma from X N to C M. Proof: (A) (B): Assume (A) holds. Take any v kerφ X N. Since v X N, there exist x,x 2 X N such that v = x x 2. Then we have Φv = Φx Φx 2 = since v kerφ. Then from (A), we have x = x 2. It follows that v =. (B) (C): Assume (B) holds. Take any v X N and assume Φv =. Then from (B), we have v =. This roves Φ is an injective ma from X N to C M. (C) (A): Assume that (C) holds. Take any x,x 2 X N such that Φx = Φx 2. Since x x 2 X N and Φ is injective on X N, we have x x 2 =, or x = x 2. Throughout the letter, we assume the uniqueness of the solution, that is, the air (X,Φ) is chosen to satisfy (5). If the uniqueness assumtion holds, we can find the exact solution in a finite number of stes via an exhaustive comutation as follows. The set X N is a finite set, and we can write X N = {x,x 2,...,x µ }. For each x i, we comute y i = Φx i and check if y i = y. Thanks to the uniqueness assumtion, we can find the exact solution in a finite time. The roblem is that the size of X N is µ = L N, and hence the comutational comlexity is exonential. For examle, if L = 2 (two symbols) and N = 2, then µ = , which takes at worst about.5 36 years (much longer than the lifetime of the universe) by the current fastest comuter with 34 eta FLOPS. To overcome this, we adot a relaxation technique based on the sum of absolute values in the next section. III. SOLUTION VIA SUM OF ABSOLUTE VALUES We here roose a relaxation method for discrete signal reconstruction. By borrowing the idea of comressed sensing, we can assume that each vector x r i, i =,2,...,L, is sarse (if the robability i given in (2) is not so small), and the sarsity is roortional to the robability i. Hence, we consider the following minimization roblem: minimize z F(z) i z r i subject to y = Φz, (6) where we use the l norm for the measure of sarsity as in comressed sensing. By the definition of the l norm, we rewrite the cost function F(z) as F(z) = N i z n r i = N L(z n ), (7) where L(t) is the sum of weighted absolute values L(t) i t r i. An examle of this function is shown in Fig.. As is shown in this figure, the function L(t) is continuous, convex, and iecewise linear. In fact, we have the following roosition:

3 SHELL et al.: BARE DEMO OF IEEETRAN.CLS FOR JOURNALS 3 L(t) r r 2 r 3 Fig.. Piecewise linear function L(t) in the cost function (7). Proosition 2: The function L(t) is continuous and convex on R and is a iecewise linear function given by t+r, if t (,r ] L(t) = a i t+b i, if t (r i,r i+ ], i =,2,...,L t r, if t [r L, ). where a i i j r i r i, j, b i i j r j + t j r j. Proof: Since each function t r i in L(t) is continuous and convex on R, the function L(t), which is the convex combination of t r i, i =,2,...,L, is also continuous and convex on R. Suose t r. From the inequality (), we have t r i for i =,2,...,L, and hence L(t) = i (t r i ) = t+r, where we used (3). Next, suose r i < t r i+ (i =,2,...,L ). The inequality () gives t r j > for j =,2,...,i and t r j for j = i +,i + 2,...,L. It follows that i L(t) = j (t r j ) j (t r j ) = a i t+b i. Finally, if t r L, then t r i for i =,2,...,L due to (), and hence L(t) = i (t r i ) = t r. Note that the function L(t) is rewritten as L(t) = max i=,,...,l {a it+b i }, where a =, b = r, a L =, and b L = r. It follows that the otimization (6) is equivalently described as minimize θ R N subject to N θ y = Φz, Az +b Eθ, (8) where θ R N is an auxiliary variable, and A I N a R N(L+) N,a [a,a,...,a L ] R L+, b N b R N(L+),b [b,b,...,b L ] R L+, E I N L+ R N(L+) N. This is a standard linear rogramming roblem and can be efficiently solved by numerical otimization softwares, such as cvx in MATLAB [], [2]. Now, we discuss the validity of the relaxation otimization given in (6) or (8). To see this, we extend the notion of the null sace roerty [7] in comressed sensing to our roblem: Definition : A matrix Φ C M N is said to satisfy the null sace roerty for an alhabet X if F(x) < F(x v), (9) for any x X N and any v kerφ\{}. Then we have the following theorem: Theorem : Let Φ C M N. Every x X N is uniquely recovered from the l otimization (6) with y = Φx if and only if Φ satisfies the null sace roerty for X. Proof: ( ): Take any v ker Φ\{} and x X N. Put z x v. Since v kerφ, we have Φv = Φ(v x+x) =, or Φx = Φz. This means that z is in the feasible set of the otimization roblem (6) with y = Φx. Also, we have x z since v. Now, by assumtion, x is the unique solution of (6) with y = Φx, and hence we have F(x) < F(z) = F(x v). ( ): Take any x X N and z C N such that x z and Φx = Φz. Putv x z. Then we havev kerφ\{}. From the null sace roerty for X, we have F(x) < F(x v) = F(z). It follows that x is the unique solution of (6). IV. EXAMPLES In this section, we show two examles to illustrate the effectiveness of the roosed method. The first examle is one-dimensional signal reconstruction with multile symbols. Let the original signal x be a 2- dimensional vector (i.e. N = 2) and the elements are drawn from the following alhabets with robability distributions: X 2 {,} : P() =, P() =, X 3 {,,}: P() =, P() = P( ) = 2, () X 5 { 2,,,,2}: P() =, P( 2) = P( ) = P() = P(2) = 4, where [,]. We assume the measurement vector y is a -dimensional vector (i.e. M = ), and the measurement matrix Φ is generated such that each element is indeendently drawn from the Gaussian distribution with a mean of and a standard deviation of by using a MATLAB command, randn(,2). The original vector x is also generated such that each element is indeendently drawn from the distribution () with varying arameter [, ]. Fig. 2 shows the grahs of the (noise-to-signal ratio) x ˆx 2 / x 2 for X 2,X 3,X 5 in (), where ˆx is the reconstructed signal, with 2 trials of random Φ and x for

4 4 JOURNAL OF L A TEX CLASS FILES, VOL., NO. 4, DECEMBER 22.5 binary symbols symbols Fig. 3. Original image (left) and disturbed image by random noise (right) Fig. 2. Averaged NSR x ˆx 2 / x 2 vs robability for X 2 (to), X 3 (middle), X 5 (bottom), by the roosed (solid) and the basis ursuit (dash) each [,]. If is large (i.e., ), then the original vector x is sarse, and we also reconstruct the original signal by the basis ursuit min z R N z subject to y = Φz, and then round off the values by the basis ursuit to the nearest integer. The error grahs by the basis ursuit are also shown in Fig. 2. For the binary alhabetx 2 = {,}, one may exchange the roles of and before erforming the basis ursuit, and the error curve below =.5 is essimistic. However, such a simle strategy cannot be alied to X 3 and X 5 for the basis ursuit, while the roosed method works well for small. This is because the basis ursuit does not fully utilize the information of the alhabet (i.e., the basis ursuit only uses the information of the value through the sarsity). We also note that the basis ursuit can be used only when the alhabet includes, while the roosed method works as well when is not an element of the alhabet. Fig. 2 also imlies a conjecture that the erformance of the roosed method converges that of the basis ursuit as the size of the alhabet goes to infinity. Next, we see an examle from image rocessing. Let us consider a binary (or black-and-white) image shown in Fig. 3 (left), which is a ixel binary-valued image. We add random Gaussian noise with a mean of and a standard deviation of. to each ixel to obtain a disturbed image as shown in Fig. 3 (right). We reresent this disturbed image as a real-valued matrix X R Then we aly the discrete Fourier transform (DFT) to X to obtain ˆX = WXW () where W is the DFT matrix defined by... ω ω 2... ω K W , ω K ω 2(K )... ω (K )(K ) Fig. 4. Reconstructed images by the basis ursuit (left) and by the roosed method (right) where K = 37 and ω ex( j2π/k). The relation can be equivalently reresented by vec( ˆX) = (W W)vec(X) C 369. We then randomly down-samle the vector vec(ˆx) to obtain a half-sized vectory C 685. The measurement matrixφis then a matrix generated by randomly down-samling row vectors from W W. Fig. 4 shows the reconstructed images by the basis ursuit with rounding off (left) and by the roosed method (right). For the roosed method, we assumed P() = P() = /2. The results clearly show the effectiveness of our method also for image reconstruction. V. CONCLUSION In this letter, we have roosed a reconstruction method for discrete signals based on the sum of absolute values (or the weighted l norm). The reconstruction algorithm is described as linear rogramming, which can be solved effectively by numerical otimization softwares. Examles have been shown that the roosed method is much more effective than the basis ursuit which only uses the information of sarsity. Future work includes an accessible condition that ensures the null sace roerty, as the restricted isometry roerty in comressed sensing. ACKNOWLEDGMENT This research is suorted in art by the JSPS Grant-in-Aid for Scientific Research (C) No , Grant-in-Aid for Scientific Research on Innovative Areas No , and an Okawa Foundation Research Grant.

5 SHELL et al.: BARE DEMO OF IEEETRAN.CLS FOR JOURNALS 5 REFERENCES [] D. L. Donoho, Comressed sensing, IEEE Trans. Inf. Theory, vol. 52, no. 4, , Ar. 26. [2] Y. C. Eldar and G. Kutyniok, Comressed Sensing: Theory and Alications. Cambridge University Press, 22. [3] K. Hayashi, M. Nagahara, and T. Tanaka, A user s guide to comressed sensing for communications systems, IEICE Trans. on Communications, vol. E96-B, no. 3, , Mar. 23. [4] S. S. Chen, D. L. Donoho, and M. A. Saunders, Atomic decomosition by basis ursuit, SIAM J. Sci. Comut., vol. 2, no.,. 33 6, Aug [5] Y. C. Pati, R. Rezaiifar, and P. S. Krishnarasad, Orthogonal matching ursuit: Recursive function aroximation with alications to wavelet decomosition, in Proc. the 27th Annual Asilomar Conf. on Signals, Systems and Comuters, Nov. 993, [6] M. Duarte, M. Davenort, D. Takhar, J. Laska, T. Sun, K. Kelly, and R. Baraniuk, Single-ixel imaging via comressive samling, IEEE Signal Process. Mag., vol. 25, no. 2,. 83 9, Mar. 28. [7] V. Bioglio, G. Coluccia, and E. Magli, Sarse image recovery using comressed sensing over finite alhabets, in Proc. IEEE International Conference on Image Processing (ICIP), Oct. 24. [8] A.-J. van der Veen, S. Talwar, and A. Paulraj, Blind estimation of multile digital signals transmitted over FIR channels, Signal Processing Letters, IEEE, vol. 2, no. 5,. 99 2, May 995. [9] B. Knoo, F. Monsees, C. Bockelmann, D. Peters-Drolshagen, S. Paul, and A. Dekorsy, Comressed sensing K-best detection for sarse multi-user communications, in Proc. 22nd Euroean Signal Processing Conference (EUSIPCO), Se. 24. [] A. Bemorad and M. Morari, Control of systems integrating logic, dynamics, and constraints, Automatica, vol. 35, no. 3, , 999. [] M. Grant and S. Boyd, CVX: Matlab software for discilined convex rogramming, version 2., htt://cvxr.com/cvx, Mar. 24. [2], Grah imlementations for nonsmooth convex rograms, in Recent Advances in Learning and Control, ser. Lecture Notes in Control and Information Sciences, V. Blondel, S. Boyd, and H. Kimura, Eds. Sringer, 28, vol. 37,. 95. [3] Z. Tian, G. Leus, and V. Lottici, Detection of sarse signals under finitealhabet constraints, in Proc. IEEE Int. Conf. Acoust. Seech Signal Process (ICASSP), Ar. 29, [4] J. Ilic and T. Strohmer, Sarsity enhanced decision feedback equalization, IEEE Trans. Signal Process., vol. 6, no. 5, , May 22. [5] S. Sarvotham, D. Baron, and R. G. Baraniuk, Comressed sensing reconstruction via belief roagation, 26. [6] A. K. Das and S. Vishwanath, On finite alhabet comressive sensing, in Proc. IEEE Int. Conf. Acoust. Seech Signal Process (ICASSP), May 23, [7] A. Cohen, W. Dahmen, and R. DeVore, Comressed sensing and best k-term aroximation, J. Amer. Math. Soc., no. 22,. 2 23, 29.