A Greedy Search Algorithm with Tree Pruning for Sparse Signal Recovery

Transcription

1 A Greedy Search Algorithm with Tree Pruning for Sparse Signal Recovery Jaeseok Lee, Suhyuk Kwon, and Byonghyo Shim Information System Laboratory School of Infonnation and Communications, Korea University Abstract-In this paper, we propose a new sparse recovery algorithm referred to as the matching pursuit with a tree pruning l (TMP) that performs efficient combinatoric search with the aid of Pre-selection 10 N greedy tree pruning. Two key ingredients of the TMP algorithm are pre-selection to put a restriction on the indices of columns in '----"""": ::"",",,,::::----_.../ <[> being investigated and tree pruning to avoid the investigation of unpromising paths in the search. In the noisy setting, we show that TMP identifies the support (index set of nonzero elements) accurately when the signal power is larger than the constant multiple of noise power. In the empirical simulations, we confirm this results by showing that TMP performs close to an ideal estimator (often called Oracle estimate) for high signal-to-noise ratio (SNR) regime. I. INTRODUCTION In recent years, compressive sensing (CS) has received much attention as a means to recover sparse signals in underdetermined system [1]-[4]. Key finding of the CS paradigm is that one can recover signals with far smaller measurements than traditional approaches use as long as the signals to be recovered is sparse and the sensing mechanism roughly preserves the energy of signals of interest. It is now well known that the problem to recover the sparest signal x using the measurements y cpx is formulated as the fa-minimization problem given by min Ilxllo subject to y cpx (1) x where cp E lft M xn is the sensing matrix. Since solving this problem is combinatoric in nature and known to be NP-hard [1], early works focused on the fl-relaxation method, such as Basis Pursuit (BP) [1], BP denoising (BPDN) [5] (also known as Lasso [3]), and Dantzig selector [4]. Another line of research, popularly studied in recent years, is greedy approach. In a nutshell, the greedy algorithm estimates the support T (index set of nonzero elements) of the sparse signals in an iterative fashion, generating a series of locally optimal updates. Although greedy algorithms, including orthogonal matching pursuit (OMP) algorithm and its variants [6]-[8], are relatively simple to implement and also computationally efficient, the performance is not so appealing, in particular for the noisy scenario. The purpose of this paper is to introduce an efficient greedy tree search algorithm for the sparse signal recovery referred to as the matching pursuit with a tree pruning (TMP). Our approach significantly reduces the computational burden of exhaustive search, yet achieves excellent recovery Pruning -,,,) : 1 : fl,(i) s,(2), 4 : 6 :, 7: r--i.. r--i 6 s,4(1) -4 : 5 : : 3 : 8.1',(2), 8 ' LJ 4!.. J h.ll, e 1h 11,, Ih1' Ih.ll, e Fig. I. Illustration of the proposed TMP algorithm. Path with dotted box is pruned from the tree since the magnitude of the residual is larger than the threshold E. perfonnance in terms of the mean squared error (MSE) in the noisy scenario. Two key ingredients of the TMP algorithm accomplishing this mission are pre-selection to put a restriction on the indices of columns in cp being investigated and tree pruning based sparse signal search. In the pre-selection stage, we choose a small number of promising indices using a conventional greedy algorithm. If we denote the set obtained in the pre-selection stage as 8, then we set N» 181 > K (in our empirical results, we could achieve near best performance with only 181 :: 2K). Once the pre-selection is finished, in the second stage, a tree search to reconstruct the sparse signal x is perfonned. Although the number of all possible candidates is (I I ) and this number is clearly much smaller than (Z) due to the pre-selection, searching all of these is still computationally demanding for nontrivial K. In order to alleviate the computational burden and at the same time preserves the effectiveness of the search, we exploit an aggressive tree pruning strategy, which is realized by the pruning of unpromising candidates in the middle of the search. Note that the prerequisite to perform the pruning operation is to construct the cost function J(x) Ily - cpxl12 that corresponds to the f2-norm of the residual for full-blown candidate. Since the direct evaluation of J(x) is not possible due to the causality of the search process, we combine already selected indices (henceforth referred to as /14/$ IEEE 1847

2 the causal indices) and roughly estimated indices (noncausal indices). When this 'roughly estimated' cost function is greater than the deliberately designed threshold E (i.e., J(x) > E), further investigation of the path is hopeless and hence we prune the path from the tree immediately. From the performance guarantee (expressed in terms of restricted isometry property (RIP)) in Section III and also empirical simulations in Section IV, we show that the proposed TMP algorithm is very effective in recovering sparse signals in both noiseless and noisy conditions. Choose noncausal set -K 5 itl arg l1).ax sc0\si,lslk-i II <fjir.iii s, 2 Construct estimated path -K 'i -K U 5 itl II. MATCHING PURSUIT WITH A TREE PRUNING A. Pre-selection: A First Stage Pruning As mentioned, the purpose of the pre-selection is to choose indices of columns in cp which are highly likely to be the elements of support T. Using the pre-selection, the search set is reduced from [2 {I, 2,...,N} to 8 (a subset of [2). In constructing 8, one can use the OMP algorithm with ck-iterations (c > 1) [9], [10] or the generalized OMP algorithm [8]. Recent studies show that by the overbooking of indices (selection of indices more than the sparsity level K), mis-detection probability of the support elements can be reduced significantly at the expense of a marginal increase in the false alarm probability. By careful examination of preselected indices in the second stage, indices not in the support T can be removed, and hence, the chance of identifying the support improves without increasing the false alarm event. B. Pruning in Tree Search In the second stage of the TMP algorithm, we reconstruct the sparse signal using the tree search. Since all combinations of K indices can be interpreted as candidates in a tree and each layer of the tree can be sorted by the magnitude of the correlation between the column of cp and the residual, the problem to find the candidate minimizing the residual is readily modeled as a combinatoric tree search problem. In order to save computational complexity and expedite the search, a pruning operation removing unpromising paths from the search tree is employed. If we denote the path 1 at layer (iteration) 'i as 81, then 81 {81' 82,... 8i} is the set of causal indices selected in the first 'i iterations. In a similar way, if we denote the rest of path (i.e., set of noncausal indices) as.5ti-1 then.5ti- 1 {8i +1' Si+2,'" SK}. Note that the noncausal set,5ti-1 is temporarily needed for the pruning operation and simply obtained by choosing K - 'i indices of columns in 8 \ 81 whose magnitude of the correlation with the residual r (8D is maximal. That is, where r(,51) 1 In this paper, we use path and candidate interchangeably, In particular, we denote a full-blown path sf by candidate. (2) (3) (4) no prune 51 from the tree yes store 51 Fig. 2. The pruning operation of TMP in the i-th layer. For example, if K - i 2, 8 \ 81 {5, 7, 9,11}, and then the noncausal set becomes Sti-1 {7, 11}. Once 'roughly' estimated candidate sf 81 U Sti-1 is obtained, we compute the residual r(,5f) y - CPsKXK 1 (where x K cp! K y) to decide whether to prune the path 8,,51 or not. To be specific, if the 2-norm of the residual is greater than the threshold E (i.e., Ilr(,5f)112 > E), then the path has little hope to survive and hence is pruned immediately (see Fig. 1). Note that the pruning threshold f is initialized to infinity and whenever the search of a layer is finished, updated to the minimum 2-norm of the residual among all survived paths (E min Ilr(sf)112)' Once the search for the final layer is finished, a path with the minimum cost function is chosen as the output of TMP. We summarize the proposed TMP algorithm in Table I. III. PERFORMANCE ANALYSIS In this section, we analyze a recovery condition under which TMP can accurately identify K-sparse signals in the noiseless and noisy scenarios. Identification implies the accurate recovery of the sparse signal in the noiseless scenario and the stable recovery of the noisy scenario. In our analysis, we use the restricted isometry property (RIP) of the sensing matrix CP. A sensing matrix cp is said to satisfy the RIP of order K if there exists a constant 6 ( cp) E (0, 1) such that (1-6(cp))llxll Ilcpxll (1 + 6(cp))llxll (5) 1848

3 In p u '7 t: m ea s ur em en t : Y, se n sl TABLE I THE TMP ALGORITHM ' n g m a tr "" ix <I> Output: Estimated signal x Initialization: i : 0, SO : 0 8 j p reseleclion (y, <I>, p) while i < K do i : i + 1, Si : 0, E i+l : Ei for I 1 to lsi-ii do 8: 8 \ Sl-I for j 1 to 181 do st : Sl-I U {s;} if st <f- Si then sft-l arg max II <I>;r(stll 2 8C8.181K-, sf st U Sft- I ' r(sf) P.LKy 8 1 if Ilr(sf)112 <:: C i then Si : S i U st, I* : sf if Ilr(I*) 112 <:: Ei+l then ci+l : Ilr(I*)112 end for end for end while return x* <I> 1. Y, sp ar s "" it y "' K "', "' i ni : ti "' al : t "" hr e "' sh o "" ld : c l indices: (preselection) (update j-th path) (check the duplicated path) (support estimation) (pruning decision) (update pruning threshold) (signal reconstruction) The j p rcsclcction ( -) is a function to choose multiple promising indices (see Section ILA). for any K -sparse vector x. In particular, the minimum of all constants 6(<I» satisfying (5) is called the restricted isometry constant (RIC) and denoted as 6 K ( <I> ). In the sequel, we use 6 K instead of 6 K (<I» for brevity. Also, in many places, we skip the proof due to the page limitation (see [11] for details). A. Exact Recovery Condition for Noiseless Scenario In order to reconstruct the original sparse signal accurately, TMP should satisfy following two conditions: 1) All the support elements should be chosen in the preselection process (i.e., T c 8). 2) In each iteration, at least one true path 2 should survive in the pruning process. In this work, we use the generalized OMP algorithm (gomp) [8] in the pre-selection stage. Note that gomp performs K iterations and selects L( > 1) indices per iteration (181 LK). The recovery condition of gomp is summarized as follows. Theorem I (Perfect recovery condition in noiseless scenario [12]): The gomp algorithm selects the support T if the sensing matrix <I> satisfies 6L7.8K < (6) Next, we investigate the condition that the final candidate sf of the tree search equals the support T. Note that to ensure. f T, at least one true path should be survived in each layer and further a support element (true index) should be added to this path. 2 lf st is a true path, it contains indices only in T (st c T). Before we proceed, we provide definitions useful in our analysis. Let Ai be the smallest correlation in magnitude between the residual r(. l) and columns associated to correct Further, let, i be the largest correlation in magnitude between r(. l) and columns associated to incorrect indices: Following lemmas provide a lower bound of A i and an upper bound of I i, respectively. Lemma 2: Suppose a path sl includes indices only in T (. 1 c T), then A i satisfies A i 1 -/ 6 6 M IIXT\sd2' (7) Lemma 3: Suppose a path sl includes indices only in T, then I i satisfies, i ::; 16: IIXT\slI12' In order to ensure that.5f (8) T, it is clear that all elements (indices) of noncausal set Sf't-l should be correct (i.e., Sf't-l C T). In other words, among all possible noncausal sets (say Sf't-l (1), Sf't-l (2), Sf't-l (3),... ) generated from s1, the set containing the true indices (say.5f't-l (3) should not be pruned. That is, Ilr(sl U sf't-l(3)) II I lrt11 < E. (9) Since (9) always holds true for the noiseless scenario (1lrril 0), what we need is a condition that noncausal set chosen from (2) is the part of support (i.e., Sf't-l (3) c T). A condition ensuring this requirement is stated in the following theorem. Theorem 4: Suppose a causal path. 1 contains correct indices only, then the noncausal set Sf't-l of TMP consist only of correct ones under (10) In other words,.5f T under 6 M <. Proof' (,From (2), it is clear that the noncausal set Sf't-l becomes the subset of T if (11 ) Further, using Lemma 2 and 3, a (sufficient) condition of (11) becomes 1 -/ 6M IIXT\S1112 > 16: IIXT\S,112' After some manipulations, we get the desired result. The overall recovery condition of TMP is identified by choosing the stricter condition between Theorem 1 and

4 Theorem 5 (Recovery condition of TMP): The TMP algorithm exactly recovers any K -sparse signal from y <I>x if the sensing matrix <I> satisfies the RIP with 6L7.8K < M < 0.16 if L7.8K-.l > M, otherwise. (12) (13) For Gaussian random measurements, this implies that TMP can exactly recover any K -sparse signal with m 0 (K log n) measurements. B. Reconstruction from Noisy Measurements In this subsection, we analyze the condition under which the TMP algorithm identifies the support in noisy scenario. Two requirements of TMP to identify the support are as follows. 1) All the support elements should be chosen in the preselection process (i.e., T c 8). 2) True path (sf T) should be survived in the pruning process. Proposition 9: The true path,5f T survives in the pruning process under Following theorem specifies the condition to satisfy the first requirement. (18) Proposition 6: The gomp algorithm selects the support if It is interesting to note that when the condition (18) is satisfied, the elements of the K -sparse signal x satisfy which is true for high SNR regime, the support is identified (a + vl)(l- 6LK)llv112 and hence all non-support elements and columns associated II xt\a II 2 (1 4 ) with these can be removed from the system. In doing so, one vl(l- 6(L+1)K6LK) - a(l- 6LK)6L+K can obtain the overdetermined system model y <I>TxT + v3 for any A -I- T. and the output of TMP becomes equivalent to the output of the In order to meet the second requirement, 1) under the condition that causal path is true (s1 c T), noncausal set should also be true (Si't- 1 C T) and further 2) this true path should not be removed by the tree pruning (i.e., if sf T, then Ilr(sfjl12 < E). Conditions capturing these observations are summarized in the following two lemmas. Lemma 7: Suppose a causal path s1 consists only of true indices (i.e., s1 C T), then the corresponding noncausal set Si't- 1 also contains the true ones (i.e., Si't- 1 T \ sd under (1-6K)( + VI +6M)llvI12 IlxT\Si OK - OK+1-6M (15) for any,51 C T. Lemma 8: The candidate whose magnitude of the residual is minimal becomes the support (i.e., Ilr(T)112 s: Ilr(sfjl12 for any sf) under 2(1-6K)llvI12 II xt\i3k II 1 2 > S:. 1-3U2K (16) In Lemma 7, we obtained the condition that the noncausal set is true when the causal path is true, and in lemma 8 we obtained the condition that the true path has the minimum residual magnitude (and hence survived from the pruning process). Therefore, when the conditions of Lemma 7 and 8 are jointly satisfied, the support T will be survived from the pruning process. Formal description of our finding is given in the following theorem. (17) where 2(1-bK) IL and w (l-bk)(+) t 1-302K 1-0K-bK+I-0lvI for any A -I- T. Proof' Immediate from Lemma 7 and 8. Using Proposition 6 and 9, we obtain an overall condition of TMP ensuring the identification of the support in the noisy scenario. Theorem 10: The TMP algorithm identifies the support from the noisy measurement y <I>x + v under IlxT\sll12,IIvl12 for any.5 C T where, max {v,/l,w} and _ (v'k+vl)(l-blk) _ 2(1-bK) and _ (l-bk)(+) W - 1-bK-bK+l -blvi Proof' Immediate from Proposition 6 and 9. Remark 11: Theorem 10 directly implies that TMP identifies the support from the noisy measurement v - VL(1 -b(l+l)k8lk)-vk(1-blk)bl+k' /L - 1-3b2K ' best possible estimator so called Oracle estimator x <I>y.4 Finally, by slightly modifying Theorem 10, one can show the stability of the TMP algorithm. Theorem 12: The output x of the TMP algorithm satisfies where T Ilx -xl12 s: Tllvl12 i+1+(r-1)bk (l-bk )v'1-b2k. IV. SIMULATION AND DISCUSSIONS (19) In this section, we evaluate the performance of TMP and conventional sparse recovery algorithms through numerical simulations. In the noiseless scenario, accuracy of the reconstruction algorithms is measured by comparing the maximal sparsity level of the underlying sparse signals at which the perfect recovery is ensured (this point is often called critical sparsity). In the noisy scenario, for each signal-to-noise ratio (SNR) point, we measure the MSE of the recovery algorithms 1 N MSE N L IXi -x il 2. il The simulation is based on randomly generated sensing matrix <I> E lr 1 00 x 2 56 whose entries are chosen independently from Gaussian distribution N(O,l/m). In addition, the nonzero elements of the K-sparse vector x are randomly 3 ci> A E rn: x I A I is a submatrix of ci> that contains columns indexed by A. 4 The estimator that has prior knowledge on the support is called the oracle estimator. 1850

5 o Q) IT: 0.4 >< w 0.3 ' OMP [ e-- CoSaMP [71 _StOMP[61 ---gomp (L 2) [ TMP (IElI 2K) K Fig. 3. Exact recovery ratio of the sparse recovery algorithms in the noiseless setting. w (j) ::i: Oracle LS -+-OMP[51 --e- CoSaMP [ StOMP [61 _gomp[ proposed TMP (IEJI4K) -+-- proposed TMP (IElI2K) 10-6 "----'----'----'-----'-----'---'" SNR (db) Fig. 4. MSE performance of sparse recovery algorithms in the noisy setting (K 35). generated in each trial. For each recovery algorithm, we perform 10,000 independent trials. Fig. 3 illustrates the exact recovery ratio as a function of the sparsity level K. Note that higher critical sparsity implies better recovery performance. As seen in the figure, the critical sparsity of the proposed TMP algorithm is larger than the conventional sparse recovery algorithms. Fig. 4 provides the MSE performance of the sparse recovery algorithms in the noisy scenario as a function of SNR where the SNR (in db) 50 is defined as SNR 10 log r. In this case, the system model is expressed as y cpx + v where v is the noise vector whose elements are generated from Gaussian N(O, 10-8;';/'). Overall, we see that the MSE performance of all recovery algorithms improves with the SNR. While the performance improvement of conventional greedy algorithms diminishes gradually as the SNR increases, the performance of TMP improves with the SNR and in fact it performs very close to Oracle estimator in high SNR regime (see Remark II). These results shows that that TMP is effective in estimating sparse signals in the noisy scenario. V. CONCLUSIONS In this paper, we proposed a new sparse signal recovery algorithm referred to as the TMP algorithm that is effective in recovering sparse signals for noisy scenarios. i,from the performance guarantees as well as the simulation results, we could observe that the proposed TMP algorithm is effective in recovering sparse signals in noisy scenarios. In particular, we observed that TMP performs close to the Oracle estimator in the high SNR regime. ACKNOWLEDGMENT This research was supported by the KCC (Korea Communications Commission) under the R&D program supervised by the KCA (Korea Communications Agency) (KCA ) and the NRF grant funded by the Korea government (MEST) (No. 2012RIA2A2AOI04751O). REFERENCES [1] E. 1. Candes, 1. Romberg, and T. Tao, "Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information," IEEE Trans. In! Theory, vol. 52, no. 2, pp , Feb [2] D. L. Donoho and X. Huo, "Uncertainty principles and ideal atomic decomposition," IEEE Trans. In! Theory, vol. 47, no. 7, pp , Nov [3] R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Royal Stat. Soc. Series B (Methodological), vol. 58, no. I, pp. pp , [4] E. Candes and T. Tao, "The dantzig selector: Statistical estimation when p is much larger than n," The Annal. Stat., vol. 35, no. 6, pp. pp , [5] S. S. Chen, D. L. Donoho, and M A. Saunders, "Atomic decomposition by basis pursuit," SIAM Journal on Scientific Computing, vol. 20, no. I, pp , [6] D. L. Donoho, Y. Tsaig, T. Drori, and 1. L. Starck, "Sparse solution of underdetermined systems of linear equations by stagewise orthogonal matching pursuit," IEEE Trans. In! Theory, vol. 58, no. 2, pp , Feb [7] D. Needell and J. A. Tropp, "Cos amp: iterative signal recovery from incomplete and inaccurate samples," Commun. ACM, vol. 53, no. 12, pp , Dec [8] 1. Wang, S. Kwon, and B. Shim, "Generalized orthogonal matching pursuit," IEEE Trans. Signal Process., vol. 60, no. 12, pp , Dec [9] T. Zhang, "Sparse recovery with orthogonal matching pursuit under rip," IEEE Trans. In! Theory, vol. 57, no. 9, pp , Sept [10] 1. Wang and B. Shim, "How many iterations are needed for the exact recovery of sparse signals using orthogonal matching pursuit?," submitted to IEEE Trans. In! Theory. [II] 1. Lee, S. Kwon, and B. Shim, "Sparse signal recovery with a tree pruning," Preparing for submission to IEEE In! Theory. [12] 1. Wang, S. Kwon, and B. Shim, "Recovery of sparse signals via generalized orthogonal matching pursuit: A new analysis," submitted to IEEE Trans. Signal Proc. 1851