Quasi Gradient Projection Algorithm for Sparse Reconstruction in Compressed Sensing

Transcription

1 Sensors & ransducers, Vol. 65, Issue, February 04, pp Sensors & ransducers 04 by IFSA Publshng, S. L. Quas Gradent Projecton Algorth for Sparse Reconstructon n Copressed Sensng Xn Meng, Mnhua Zhang School of Coputer and Councaton Engneerng, anjn Unversty of echnology, anjn , Chna el.: , fax: E-al: engxn@tjut.edu.cn Receved: 9 October 03 /Accepted: 7 Deceber 03 /Publshed: 8 February 04 Abstract: Copressed sensng s a novel sgnal saplng theory under the condton that the sgnal s sparse or copressble. he exstng recovery algorths based on the gradent projecton can ether need pror nowledge or recovery the sgnal poorly. In ths paper, a new algorth based on gradent projecton s proposed, whch s referred as Quas Gradent Projecton. he algorth presented quas gradent drecton and two step szes schees along ths drecton. he algorth doesn t need any pror nowledge of the orgnal sgnal. Sulaton results deonstrate that the presented algorth cans recovery the sgnal ore correctly than GPSR whch also don t need pror nowledge. Meanwhle, the algorth has a lower coputaton coplexty. Copyrght 04 IFSA Publshng, S. L. Keywords: Copressed sensng, Gradent projecton, Quas gradent, l steepest descent, Penalzng paraeter.. Introducton.. acground In recent years, copressve sensng (CS) [, ] has attracted consderable attenton n the areas of sgnal processng. CS bulds upon the fundaental fact that we can represent any sgnals usng only a few non-zero coeffcents n a sutable bass or dctonary. Nonlnear optzaton can then enable recovery of such sgnals fro very few easureents. Copressve Sensng nvolves three aspects of research, naely sgnal sparse decoposton, easureent atrx desgn and sgnal reconstructon algorth. Consder a real-valued, fnte-length, onedensonal, dscrete-te sgnal x, whch can be vewed as an N colun vector n R N wth eleents x [],,,, N. Any sgnal n R N can be represented n ters of a bass of N vectors { } N. For splcty, assue that the bass s orthonoral. Usng the N N bass atrx Ψ [ N ] wth the vectors { } as coluns, x can be expressed as N x Ψα, () where α s the N colun vector of weghtng coeffcents, x. Clearly, x and α are the equvalent representatons of the sgnal. Now we only consder K -sparse sgnal x, that s K coeffcents of x are nonzero and N K are Artcle nuber P_889 3

2 Sensors & ransducers, Vol. 65, Issue, February 04, pp zero. Consder the M N ( M N ) easureent atrx Φ, then the copressve sensng easureent vector can be obtaned by y Φxn, () where n s the whte Gaussan nose wth varance. Candès [] ponts out that the orgnal sgnal can be recovered fro y and Φ by the followng proble: n x st.. Φx y, (3) 0 where l 0 nor denotes the nuber of non-zero 0 coponents (hs s a standard abuse of ternology: s not postve hoogeneous, yet s referred to 0 as a nor). Stand { x : Φx y } for Ω. It s ore easly handled f the l 0 nor s replaced by the l nor, resultng n a convex proble [3]: n x st.. Φx y (4).. Prevous Algorth he l nzaton proble (4) s a constrant convex optzaton ssue. If the objectve functon s dfferental, the faous gradent projecton algorth [4] can be used to solve (4). Whereas the l nor s non-dfferentable, ts gradent doesn t exst. Prevous algorths based on gradent projecton to solve (4) are tred to convert the l nzaton proble to the equvalent one n whch the objectve functon s dfferentable. GPSR (Gradent Projecton for Sparse Reconstructon) [5] splts the varable x nto ts postve and negatve parts, and then transfors the PDN (asc Pursut de-nosng) [3] to a quadratc progra bounded on the non-negatve regon. Although the denson of the new proble s twce that of the orgnal one, the operatons nvolved s alost as before. However ts reconstructon effects are not very well. Although Iteratve Weghted Gradent Projecton for Sparse Reconstructon (IWGP) [6] proved the speed of GPSR by explorng the correlaton between the gradent and the resdual error, the effects of recovery proves lttle. PSD (Projected Steepest Descent) [7] s to nze the resdual error bounded on the l ball wth radus of, but the should be a pror nowledge. GPSS [8] (gradent projecton ethods by step length selecton rules) proves the convergence speed of PSD by adoptng adaptve step length selecton based on strateges for the alternaton of the well-nown arzla-orwen [9] rules, whle have no obvous proveent on the reconstructon accuracy. Actually, the purpose of the gradent n the gradent projecton ethod s only offerng a descent drecton of the objectve functon. Soe of the prevous algorths to solve the l nzaton proble can be understood for ths vew. Algorth GraDes [0] (Gradent Descent wth Sparsfcaton) can be seen to set all but s largest coordnates n absolute value to zero to gan a searchng pont and then project the searchng pont onto Ω. IH [] sets a threshold value and reserve the coordnates whose absolute value no less than threshold value whle others zero. In these two algorths, the drecton fro the current pont to the searchng pont s a descent drecton of the l nor. It s nature to as that f there exsts a steepest descent drecton of the l nor so that the gradent projecton algorth can be used drectly to solve (4) by replacng the gradent wth ths drecton. In ths paper, we wll show that sgn( x ) referred to be as l quas gradent just has ths property. It s also a concdence that sgn( x ) s one of the sub-gradents of l nor, but we derve t for an absolutely dfferent perspectve. A new step sze selecton rule s ntroduced followng the dea of nzaton rule by Cauchy [4]. esdes, an nterpretaton s gven for the Vanshng Postve Step Sze adopted n []. hen, we present our algorth naed l quas gradent projecton (QGP).. L Quas Gradent Projecton Algorth.. L Quas Gradent Suppose (,,, ) x s on the hyper plane: x x x x N and x sgn( ), so x x (5) he followng forula can be concluded for (5): x 0 x (6) [ sgn( )] ( ) So noral vector of the hyper plane at x that ponted to the orgn s sgn( x ). he case 6 n Fg. deonstrates that f we want to ove x to plane x ( s a lttle saller than ) n the nearest way, we ust ove along wth the drecton sgn( x ). he case that x s just the vertex or n the ntersecton of two planes also have the sae result as llustrated n Fg.. Fro the above analyss, we can conclude that: Propostons : sgn( x ) s the steepest descent 3

3 Sensors & ransducers, Vol. 65, Issue, February 04, pp drecton of x at pont x, and the correspondng slope s sgn( x ). P ( ) S ( ) x ( ) x () Obverse that (9) s equvalent to x x x x () * * * * [ ( )] * hen we have ( ) x. As x * s absolutely sallest of all non-zero coponents, the followed consequence can be ade Fg.. l Quas Gradent. Drecton,, 4, 5 represent the steepest descent drecton of the current ponts separately. It s easly seen that l nor value decreases fast along than the drecton of 3. Drecton 6 represents the noral vector of the plane that ponted to the orgn. So x ( ), x ( ) S ( ) ( x) 0, x ( ) x ( ), x ( ) x ( )sgn( x ) (3) Proof: Denote { x: x } by, where x. Consder an n-densonal vector x ( x, x,, x N ) wth non-zero coponents. When ovng x to soe pont x n the hyper plane, the change rate of l nor s ( x )/ x x. Obvously, when x P ( x ), the x x s nal, the axu rate can be acheved. So the steepest descent drecton s P ( ) x x. Sort the absolute values of the coponents of x, * resultng n the rearranged sequence ( x ),,, N. * * * hus x x 0 for all, and x 0. Usng lea 4. n [7], the projecton of x on s the sae as the one on { x: x }, so there exsts that * * * * ( x ) ( x x x ) (7) S ( ) ( x) x ( )sgn( x ) (4) Fro the above, we learn that the steepest descent drecton P ( ) x x has the sae drecton wth, and the correspondng slope wll be the length of sgn( x ), that s sgn( x ). * * () If ( x ) x, then ( * * x ) x, thus (7) holds for soe. Suppose sgn( x ) s the steepest descent drecton, that s sgn( x ) has the sae drecton wth P ( ) x x, thus x R x x x (5) * * * * [ ( )] he above forula (5) s equvalent to ( * * x ) x. It ples fro (7) that and P ( ) S ( ) x x, (8) * * * * ( x ) ( x x x ) (6) If, we have * * * where ( ) x [ ( x x )] () Consder the case where * * ( x ) x x (9) he above forula ples that, (7) holds for, that s * * * * * * x x x x x x x ( ) ( ) ( ) (0) It follows fro ths that * * * * ( x ) ( x x x ) (7) hs contradcts the forula (6). hus our assupton s ncorrect, and sgn( x ) s not the * * steepest descent drecton for ( x ) x. We conclude fro the above analyss that sgn( x ) corresponded to l nor have two slar propertes wth the opposte drecton of gradent for dfferentable functon: ) he negatve gradent s a vector that ponts n the drecton of the greatest rate of decrease of the objectve functon, and whose agntude s that rate 33

4 Sensors & ransducers, Vol. 65, Issue, February 04, pp of decrease. Slarly, sgn( x ) s the greatest descent drecton of l nor, and ts agntude s also the rate of decrease. ) Property ) holds only for the local area of the current pont, not globally. So we naed sgn( x ) as the l quas gradent of x at x... Step Sze Selecton Rule... Mnzaton Rule Step length selecton s an essental proble n the gradent related ethods. Inspred by the classcal steepest descent ethod [4], we search the ( 0 ) such that the cost l nor functon s nzed along the drecton AAA, that s, satsfes # n 0 x sgn( x) x sgn( x ) (8) Sort the absolute values of the nonzero coponents of x, resultng n the rearranged sequence ( x ),,,. hus 0 x x for all. hus x sgn( x ) x (9) Denote x by f ( ). If the absolute values of the nonzero coponents dffer fro each other, we can ae the followng conclusons: If 0 x, we have ( ) ( ) f x (0) Fg.. Mnzaton Rule (a) s even, (b) s odd.... Vanshng Postve Step Sze Lterature [] adopts vanshng postve step sze rule, whch convergences to zeros along wth the teraton. In what follows, we wll gve an nterpretaton for the reason of ths rule. Suppose x Ω, then the dstance between x sgn( x ) and ts projecton on Ω s ( xsgn( x)) P ( xsgn( x)) Φ sgn( x) Ω (3) he above forula ples that the dstance s n drect proporton to the step sze. Constraned n # the nterval (0, ]. On one hand the objectve functon l nor can decrease ore as the grows. Meanwhle the teraton pont s ore far away fro the constrant regon. hs can be llustrated n the Fg. 3. o balance ths nteracton, our ephass s set to the decrease l nor, that s settng consderable larger. As the teraton goes by, the l nor can decrease lttle, then we ephass on the latter aspect, that s settng sall step sze so that the teraton pont volate lttle for Ω. o fulfll ths thought, a nature ethod s justly vanshng postve step sze rule as suggested n []. If x x, we have () ( ) [( ) ( )] ( ) f x x If x, we have f x () ( ) ( ) Above analyss ples that f ( ) s a pecewse lnear functon as llustrated n Fg., and the correspondng slope s ( ) for all 0. If s even, f ( ) can be nzed at x /. If s odd, f ( ) can be nzed at x ( )/. he sae results can be obtaned for the case that several nonzero coponents have the dentcal absolute value. Fg. 3. Vanshng Postve Step Sze..3. Algorth Descrpton Our algorth naed l Quas Gradent Projecton Algorth (QGP) s defned as follows. Step : Intalzaton. Let x (0) Ω ; set 0. 34

5 Sensors & ransducers, Vol. 65, Issue, February 04, pp Step : Copute step sze such as Mnzaton Step Sze or Vanshng Postve Step Sze, then search along the drecton of sgn( x ) : ths scale of sparsty, QGP-MR and QGP-VP can get a ore accurate result. ( ) ( ) ( ) ( ) w x sgn( x ) (4) Step 3: Projecton. he next teraton pont s the projecton of w on Ω : x P ( w ) w Φ ( y Φw ) (5) Ω Step 4: Perfor convergence test and ternate wth approxate soluton x f t s satsfed; otherwse set and return to Step. Denote QGP algorth wth Mnzaton Rule and Vanshng Postve Step Sze as QGP-MR and QGP-VP separately. Fg. 4. Relatve error versus te. 3. Sulaton Results In ths secton we report experents whch deonstrate the copettve perforance of our approach on probles of the for (), and copared t wth the state-of-the-art algorth GPSR-. All the experents are carred out on a personal coputer wth an Intel Pentu(R) E5300 dual core.6 GHz processor and G of eory. In the sulaton, N=000, M=500, the orgnal sgnal x contans K randoly placed ± spes, and the observaton y s generated accordng to (), 4 wth 0. he easureent atrx Φ s obtaned by frst fllng t wth ndependent saples of a standard Gaussan dstrbuton and then orthonoralzng the rows. For the QGP-VP algorth, let ( ) wth, Noralze sgn( x ), that s the adopted descent drecton s sgn( x ) / sgn( x ). In all experents, we ntal x (0) Φ y. o evaluate the proposed ethod, relatve error s used whch s defned as x x / x. Set K=00 n the frst experent. We run QGP- MR and QGP-VP for 00 teratons and GPSR 00 teratons. he convergence results of the three algorths are showed n Fg. 4. Fg. 4 shows the relatve error of each teraton pont of the three ethods. he results ndcate that all algorths converge to stable pont. In the frst several teratons, the relatve errors of all algorths decrease fast. Whle the estatons ade by QGP-MR and QGP-VP are uch better than the GPSR- ethod. Fg. 5 shows the relatve errors of these algorths as the sparsty. Each result s ean value by 50 randoly ade easureent atrx. It ples that the relatve error of the GPSR- gets bgger as the ncrease of the sparsty nuber, whle the other two ethods rean relatvely stable. Otherwse, n Fg. 5. Least relatve error versus sparsty. In the thrd experent, we ternate the QGP algorths wth two dfferent step szes when the relatve error s saller than 0.05 that s x x / x he present runnng te of QGP-MR and QGP-VP s descrbed n Fg. 6. hs fgure deonstrates that the te needed for both ethods for alost exact reconstructon gets bgger as the ncrease of the sparsty nuber. Meanwhle QGP-VP s a lttle fast than QGP-MR. Fg. 6. e of exact reconstructon versus sparsty. 35

6 Sensors & ransducers, Vol. 65, Issue, February 04, pp Concludng Rears ased on the classcal gradent projecton algorth, l quas gradent projecton (QGP) s presented by the ntroducton of l quas gradent and two step length selecton. Sulaton results deonstrate that the ethod can reconstruct the sgnal fast and accurately. Acnowledgeent hs wor s supported by the Natonal Natural Scence Foundaton of Chna (No ). References []. Donoho D. L., Copressed sensng, IEEE ransactons on Inforaton heory, 5, 4, 006, pp []. Candès E., Roberg J., ao., Robust uncertanty prncples: exact sgnal reconstructon fro hghly ncoplete frequency nforaton, IEEE ransactons on Inforaton heory, 5,, 006, pp [3]. Chen S.., Donoho D. L., Saunders M. A., Atoc decoposton by bass pursut, SIAM Journal on Scentfc Coputng, 0,, 998, pp [4]. ertseas D. P., Nonlnear Prograng, Second edton, Athena Scentfc, oston, 999, pp [5]. Fgueredo M., Nowa R., Wrght S., Gradent projecton for sparse reconstructon: applcaton to copressed sensng and other nverse probles, IEEE Journal of Selected opcs n Sgnal Processng,, 4, 007, pp [6]. Deng J., Ren G., Jn Y., et al., Iteratve weghted gradent projecton for sparse reconstructon, Inforaton echnology Journal, 0, 7, 0, pp [7]. Daubeches I., Fornaser M., Lors I., Accelerated projected gradent ethod for lnear nverse proble wth sparsty constrants, Journal of Fourer Analyss and Applcatons, 4, 5/6, 008, pp [8]. Lors I., ertero M., Mol C. D., et al., Acceleratng gradent projecton ethods for l-constraned sgnal recovery by steplength selecton rules, Appled and Coputatonal Haronc Analyss, 7,, 009, pp [9]. Frassoldat G., Zann L., Zanghrat G., New adaptve stepsze selectons n gradent ethods, Journal of Industral and Manageent Optzaton, 4,, 008, pp [0]. Garg R., Khandear R., Gradent Descent wth Sparsfcaton: An teratve algorth for sparse recovery wth restrcted soetry property, n Proceedngs of the 6th Internatonal Conference on Machne Learnng, Montreal, Canada, 009. []. luensath., Yaghoob M., Daves M. E., Iteratve hard thresholdng and regularzaton, n Proceedngs of the IEEE Internatonal Conference on Acoustcs, Speech and Sgnal Processng, 007. []. Lorenz D. A., Pfetsch M. E., llan A. M., Solvng ass Pursut: Sub gradent algorth, heurstc optalty chec, and solver coparson, Optzaton Onlne E-Prnt ID, 0, Copyrght, Internatonal Frequency Sensor Assocaton (IFSA) Publshng, S. L. All rghts reserved. ( 36