Sparse signals recovered by non-convex penalty in quasi-linear systems

Transcription

1 Cui et al. Journal of Inequalities and Applications 018) 018:59 R E S E A R C H Open Access Sparse signals recovered by non-conve penalty in quasi-linear systems Angang Cui 1,HaiyangLi, Meng Wen and Jigen Peng 1* * Correspondence: jgpengjtu@16.com 1 School of Mathematics and Statistics, Xi an Jiaotong University, Xi an, China Full list of author information is available at the end of the article Abstract The goal of compressed sensing is to reconstruct a sparse signal under a few linear measurements far less than the dimension of the ambient space of the signal. However, many real-life applications in physics and biomedical sciences carry some strongly nonlinear structures, and the linear model is no longer suitable. Compared with the compressed sensing under the linear circumstance, this nonlinear compressed sensing is much more difficult, in fact also NP-hard, combinatorial problem, because of the discrete and discontinuous nature of the l 0 -norm and the nonlinearity. In order to get a convenience for sparse signal recovery, we set the nonlinear models have a smooth quasi-linear nature in this paper, and study a non-conve fraction function ρ a in this quasi-linear compressed sensing. We propose an iterative fraction thresholding algorithm to solve the regularization problem QP λ a ) for all a > 0. With the change of parameter a > 0, our algorithm could get a promising result, which is one of the advantages for our algorithm compared with some state-of-art algorithms. Numerical eperiments show that our method performs much better than some state-of-the-art methods. MSC: 34A34; 78M50; 93C10 Keywords: Compressed sensing; Quasi-linear; Non-conve fraction function; Iterative thresholding algorithm 1 Introduction In compressed sensing see, e.g., [1, ]), the problem of reconstructing a sparse signal under a few linear measurements which are far fewer than the dimension of the ambient space of the signal can be modeled into the following l 0 -minimization: P 0 ) min R n 0 subject to A = b, 1) where A R m n is an m n real matri of full row rank with m < n, andb R m is a nonzero real vector of m-dimension, and 0 is the l 0 -norm of the real vector, which countsthenumberofthenonzeroentriesin see, e.g., [3 5]). In general, the problem P 0 ) is computational and NP-hard because of the discrete and discontinuous nature of the l 0 - norm. However, many real-life applications in physics and biomedical sciences carry some strongly nonlinear structures [6], so that the linear model in problem P 0 )isnolongersuitable. In this nonlinear case, we consider a map A : R n R m, which is no longer necessarily The Authors) 018. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original authors) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

2 Cui et al. Journal of Inequalities and Applications 018) 018:59 Page of 11 linear, and reconstruct a sparse vector R n from the measurements b R m given by A)=b. ) Compared with the l 0 -minimization under the linear circumstance, this nonlinear minimization is much more difficult, in fact also NP-hard, combinatorial problem, because of the discrete and discontinuous nature of the l 0 -norm and the nonlinearity. In order to get a convenience for sparse signal recovery, in this paper, we set the nonlinear models have a smooth quasi-linear nature. By this means, there eists a Lipschitz map F : R n R m n 3) such that A)=F) 4) for all R n.so,thel 0 -minimization under the quasi-linear case can be mathematically viewed as the following form: QP 0 ) min R n 0 subject to F) = b. 5) In fact, the minimization QP 0 ) under the quasi-linear case is also combinatorial and NPhard [6, 7]. To overcome this problem, the authors in [6, 7]proposedthel 1 -minimization QP 1 ) min R n 1 subject to F) = b 6) for the constrained problem and QP λ 1 ) { min F) b R n + λ } 1 7) for the regularization problem, where 1 = n i=1 i is the l 1 -norm of vector. In [6, 7], the authors have shown that the l 1 -norm minimization can really make an eact recovery in some specific conditions. In general, however, these conditions are always hard to satisfied in practice. Moreover, the regularization problem QP λ 1 )alwaysleadsto a biased estimation by shrinking all the components of the vector toward zero simultaneously, and sometimes results in over-penalization in the regularization model QP λ 1 )as the l 1 -norm in linear compressed sensing. In pursuit of better reconstruction results, in this paper, we propose the following fraction minimization: QP λ a ) { min F) b R n + λp a) }, 8) where P a )= n ρ a i ), a >0 9) i=1

3 Cui et al. Journal of Inequalities and Applications 018) 018:59 Page 3 of 11 Figure 1 Behavior of the fraction function ρ a t)for various values of a >0 and ρ a t)= a t a t +1 10) is the fraction function which performs outstanding in image restoration [8], linear compressed sensing [9] and matri rank minimization problem [10]. Clearly, with the change of parameter a > 0, the non-conve function P a ) could approimately interpolate the l 0 -norm lim ρ 0 if i =0; a i )= 11) a + 1 if i 0. Figure 1 shows the behavior of the fraction function ρ a t) for various values of a >0. The rest of this paper is organized as follows. Some preliminary results that are used in this paper are given in Sect..InSect.3, we propose an iterative fraction thresholding algorithm to solve the regularization problem QPa λ ) for all a >0.InSect.3, we present some numerical eperiments to demonstrate the effectiveness of our algorithm. The concluding remarks are presented in Sect. 4. Preliminaries In this section, we give some preliminary results that are used in this paper. Define a function of β R as f λ β)=β γ ) + λρ a β) 1) and let pro β a,λ γ ) arg min f λβ). 13) β R Lemma 1 see [9 11]) The operator pro β a,λ defined in 13) can be epressed as pro β a,λ g a,λ γ ) if γ > ta,λ γ )= ; 0 if γ ta,λ, 14)

4 Cui et al. Journal of Inequalities and Applications 018) 018:59 Page 4 of 11 Figure Plot of the threshold function g a,λ for a =1 and λ =0.5 Figure 3 Plot of the threshold function g a,λ for a = and λ =0.5 where g a,λ γ ) is defined as 1+a γ 1 + cos φγ ) 3 g a,λ γ )=signγ ) a 7λa ) φγ )=arccos 41 + a γ ) 1 3 andthethresholdvaluesatisfies 3 π 3 )) 1 ), 15) ta,λ = ta,λ 1 if λ 1 ta,λ if λ > 1 a ; a, 16) where t 1 a,λ = λ a, t a,λ = λ 1 a. 17) Figures, 3, 4, and5 show the plots of the threshold function g a,λ for a =1,,3,5, and λ =0.5. Figures 6 and 7 show the plots of the hard/soft threshold functions with λ =0.5.

5 Cui et al. Journal of Inequalities and Applications 018) 018:59 Page 5 of 11 Figure 4 Plot of the threshold function g a,λ for a =3 and λ =0.5 Figure 5 Plot of the threshold function g a,λ for a =5,andλ =0.5 Figure 6 Plot of the hard threshold function with λ =0.5 Definition 1 The iterative thresholding operator G a,λ can be defined by G a,λ )= pro β a,λ 1),...,pro β a,λ n) ), 18) where pro β a,λ is defined in Lemma 1. 3 Thresholding representation theory and algorithm for problem QP λ a ) In this section, we establish a thresholding representation theory of the problem QPa λ), which underlies the algorithm to be proposed. Then an iterative fraction thresholding algorithm IFTA) is proposed to solve the problem QPa λ ) for all a >0.

6 Cui et al. Journal of Inequalities and Applications 018) 018:59 Page 6 of 11 Figure 7 Plot of the soft threshold function with λ = Thresholding representation theory For any fied positive parameters λ >0,μ >0,a >0and R n,let C 1 )= F) b + λp a) 19) and C, y)=μ Fy) b + λμp a) μ Fy) Fy)y + y. 0) It is clear that C, )=μc 1 ) for all μ >0. Theorem 1 For any λ >0and 0<μ < L 1 with F ) F ) L. If is the optimal solution of min R n C 1 ), then is also the optimal solution of min R n C, ), that is, C, ) C, ) for any R n. Proof By the definition of C, y), we have C, ) = μ F ) b + λμp a) μ F ) F ) + μ F ) b + λμp a) μc 1 ) = C, ). Theorem For any λ >0,μ >0and solution of min R n C 1 ), min R n C, ) is equivalent to { min Bμ ) R n + λμp a) }, 1) where B μ )= + μf ) b F ) ).

7 Cui et al. Journal of Inequalities and Applications 018) 018:59 Page 7 of 11 Proof By the definition, C, y)canberewrittenas C, ) = μf ) F ) + μf ) ) b + λμp a)+μ b + μ F ) μf ) F ) + μf ) b = Bμ ) + λμp a)+μ b + μ F ) Bμ ), which implies that min R n C, ) for any λ >0,μ >0isequivalentto min R n { Bμ ) + λμp a) }. Combining Theorem, Theorem 1 and Lemma 1, the thresholding representation of QP λ a )canbeconcludedby = G a,λμ Bμ )), ) where the operator G a,λμ is defined in Definition 1 and obtained by replacing λ with λμ. With the thresholding representations ), the IFTA for solving the regularization problem QPa λ ) can be naturally defined as k+1 = G a,λμ Bμ k )), k =0,1,,..., 3) where B μ k )= k + μf k ) b F k ) k ). 3. Adjusting the values for the regularization parameter λ > 0 In this subsection, the cross-validation method see [9, 10, 1]) is accepted to automatically adjust the value for the regularization parameter λ > 0. In other words, when some prior information is known for a regularization problem, this selection is more reasonable and intelligent. Suppose that the vector of sparsity r is the optimal solution of the regularization problem QPa λ ), and without loss of generality, set Bμ 1 Bμ Bμ r Bμ r+1 Bμ n 0. Then it follows from 14)that Bμ i > ta,λμ i {1,,...,r}, Bμ i ta,λμ i {r +1,r +,...,n}, where ta,λμ is obtained by replacing λ with λμ in t a,λ. By ta,λμ t1 a,λμ,wehave B μ ) r ta,λμ t a,λμ = λμ 1 a ; B μ ) r+1 < ta,λμ t1 a,λμ = λμ a. 4) It follows that B μ ) r+1 aμ λ a B μ ) r +1). 5) 4a μ

8 Cui et al. Journal of Inequalities and Applications 018) 018:59 Page 8 of 11 From 5), we obtain [ Bμ ) r+1 λ, a B μ ) r +1) ]. aμ 4a μ We denote by λ 1 and λ the left and the right of the above interval, respectively: λ 1 = B μ ) r+1 aμ and λ = a B μ ) r +1). 4a μ Achoiceofλ is λ 1 if λ 1 1 λ = λ if λ 1 > 1 a μ ; a μ. Since is unknown, and k is the best available approimation to,sowecantake λ 1,k = B μ k ) r+1 if λ aμ 1,k 1 λ = ; a μ λ,k = a B μ k ) r +1) if λ 4a μ 1,k > 1, 6) a μ in the kth iteration. That is, 6) can be used to automatically adjust the value of the regularization parameter λ > 0 during iteration. Remark 1 Notice that 6) isvalidforanyμ > 0 satisfying 0 < μ F k ). In general, we can take μ = μ k = 1 ɛ with any small ɛ 0, 1) below. Especially, the threshold value F k ) is ta,λμ = λμ a when λ = λ 1,k,andta,λμ = λμ 1 a when λ = λ,k. 3.3 Iterative fraction thresholding algorithm IFTA) Based on the thresholding representation 3) andthe analyses given in Sect. 3.,the proposed iterative fraction thresholding algorithm IFTA) for regularization problem QP λ a ) can be naturally described in Algorithm 1. Remark The convergence of IFTA is not proved theoretically in this paper, and this is our future work. 4 Numerical eperiments In the section, we carry out a series of simulations to demonstrate the performance of IFTA, and compare them with those obtained with some state-of-art methods iterative soft thresholding algorithm ISTA) [6, 7]), iterative hard thresholding algorithm IHTA) [6, 7]. In our numerical eperiments, we set F)=A 1 + ηf 0 ) A, 7) where A 1 R is a fied Gaussian random matri, 0 R 400 is a reference vector, f :[0, ) R is a positive and smooth Lipschitz continuous function with f t)=lnt +1), η is a sufficiently small scaling factor we set η = 0.003), and A R is a fied matri with every entry equals 1. Then the form of nonlinearity considered in 7) is a quasi-

9 Cui et al. Journal of Inequalities and Applications 018) 018:59 Page 9 of 11 Algorithm 1 Iterative fraction thresholding algorithm IFTA) Initialize: Given 0 R n, μ 0 = 1 ɛ 0 < ɛ <1)anda >0; F 0 ) while not converged do z k := B μk k )= k + μ k F k ) y F k ) k ), λ 1,k = B μ k k ) r+1, λ aμ,k = a B μ k k ) r +1) k μ k = 1 ɛ ; F k ) if λ 1,k 1 then a μ k λ = λ 1,k ; ta,λμ k = λμ ka for i =1:length) 1. zi k > t a,λμ k,then k+1 i = g a,λk μ k zi k);. zi k t a,λμ k,then k+1 i =0; else λ = λ,k ; t = λμ k 1 a ; for i =1:length) 1. zi k > t a,λμ k,then k+1 i = g a,λk μ k zi k);. zi k t a,λμ k,then k+1 i =0; end k k +1; end while return: k+1. 4a μ k, linear, and the more detailed accounts of the setting in the form of 7) canbefoundin [6, 7]. By randomly generating such sufficiently sparse vectors 0 choosing the nonzero locations uniformly over the support in random, and their values from N0, 1)), we generate vectors b. In this way, we know the sparsest solution to F 0 ) 0 = b,andweareableto compare this with algorithmic results. The stopping criterion is usually as follows: k k 1 k Tol, where k and k 1 are numerical results from two continuous iterative steps and Tol is a given small number. The success is measured by computing relative error = 0 0 Re, where is the numerical results generated by IFTA, and Re is also a given small number. In all of our eperiments, we set Tol = 10 8 to indicate the stopping criterion, and set Re = 10 4 to indicate a perfect recovery of the original sparse vector 0. Figure 8 shows the success rate of three algorithms in the recovery of a sparse signal with different cardinality. In this eperiment, we repeatedly perform 30 tests and present average results and take a =.5. Figure 9 shows the relative error between the solution and the given signal 0.Inthis eperiment, we repeatedly perform 30 tests and present average results and take a =.5. The graphs presented in Fig. 8 and Fig. 9 show the performance of the ISTA, IHTA and IFTA in recovering the true sparsest) signals. From Fig. 8, we can see that IFTA performs best, and IST algorithm the second. From Fig. 9, we see that the IFTA has the smallest relative error value with sparsity growing.

10 Cui et al. Journal of Inequalities and Applications 018) 018:59 Page 10 of 11 Figure 8 Success rate of three algorithms in the recovery of a sparse signal with different cardinality. In this eperiment, we repeatedly perform 30 tests and present average results and take a =.5 Figure 9 Relative error between the solution and the given signal 0. In this eperiment, we repeatedly perform 30 tests and present average results and take a =.5 5 Conclusion In this paper, we take the fraction function as the substitution for l 0 -norm in quasi-linear compressed sensing. An iterative fraction thresholding algorithm is proposed to solve the regularization problem QPa λ ) for all a > 0. With the change of parameter a >0,ouralgorithm could get a promising result, which is one of the advantages for our algorithm compared with some state-of-art algorithms. We also provide a series of eperiments to assess performance of our algorithm and the eperiment results have illustrated that our algorithms is able to address the sparse signal recovery problems in nonlinear systems. Compared with ISTA and IHTA, IFTA performs best in sparse signal recovery and has the smallest relative error value with sparsity growing. However, the convergence of our algorithm is not proved theoretically in this paper, and it is our future work. Acknowledgements The work was supported by the National Natural Science Foundations of China , , , ) and the Science Foundations of Shaani Province of China 016JQ109, 015JM101). Competing interests The authors declare that they have no competing interests. Authors contributions All authors contributed equally to this work. All authors read and approved the final manuscript. Author details 1 School of Mathematics and Statistics, Xi an Jiaotong University, Xi an, China. School of Science, Xi an Polytechnic University, Xi an, China.

11 Cui et al. Journal of Inequalities and Applications 018) 018:59 Page 11 of 11 Publisher s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Received: 11 December 017 Accepted: 6 March 018 References 1. Candes, E., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 598), ). Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 54), ) 3. Bruckstein, A.M., Donoho, D.L., Elad, M.: From sparse solutions of systems of equations to sparse modelling of signals and images. SIAM Rev. 511), ) 4. Elad, M.: Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. Springe, New York 010) 5. Theodoridis, S., Kopsinis, Y., Slavakis, K.: Sparsity-aware learning and compressed sensing: an overview Ehler, M., Fornasier, M., Sigl, J.: Quasi-linear compressed sensing. Multiscale Model. Simul. 1), ) 7. Sigl, J.: Quasilinear compressed sensing. Master s thesis, Technische University München, Munich, Germany 013) 8. Geman, D., Reynolds, G.: Constrained restoration and recovery of discontinuities. IEEE Trans. Pattern Anal. Mach. Intell. 143), ) 9. Li, H., Zhang, Q., Cui, A., Peng, J.: Minimization of fraction function penalty in compressed sensing Cui, A., Peng, J., Li, H., Zhang, C., Yu, Y.: Affine matri rank minimization problem via non-conve fraction function penalty. J. Comput. Appl. Math. 336, ) 11. Xing, F.: Investigation on solutions of cubic equations with one unknown. J. Cent. Univ. Natl. Nat. Sci. Ed.) 13), ) 1. Xu, Z., Chang, X., Xu, F., Zhang, H.: L1/ regularization: a thresholding representation theory and a fast solver. IEEE Trans. Neural Netw. Learn. Syst. 37), )