A Fixed Point Method for Convex Systems

Transcription

1 Appled Mathematcs Publshed Ole October ( A Fed Pot Method for Cove Systems Morteza Kmae Farzad Rahpeyma Departmet of Mathematcs Raz Uversty Kermashah Ira Departmet of Mathematcs Payame Noor Uversty ehra Ira mal: mortezamae@malcom rahpeyma_83@yahoocom Receved Jue 6 ; revsed September 7 ; accepted October 6 ABSRAC We preset a ew fed pot techque to solve a system of cove equatos several varables Our approach s based o two powerful alorthmc deas: operator-spltt ad steepest descet drecto he quadratc coverece of the proposed approach s establshed uder some reasoable codtos Prelmary umercal results are also reported Keywords: Cove quatos; Least Squares; -Reularzato Problems; Fed Pot; Quadratcally Coverece Itroducto System of cove equatos s a class of problems that s coceptually close to both costraed ad ucostraed optmzato ad ofte arse the appled areas of mathematcs physcs boloy eeer eophyscs chemstry ad dustry Cosder the follow system of cove equatos F R m whch F : R R s a cove cotuously dfferetable fucto It s otced that f F Ab the system () s a lear system of equatos ad there are a lot of approaches to solve ths problem Oe of the most terest methods for solv lear system s fed pot methods that have bee comprehesvely studed by may authors For eample shrae subspace optmzato ad cotuato [] fed-pot cotuato method [] olear wavelet mae process [3] M method [4] teratve threshold method [5] ad fast teratve threshold [6] he system () s called a overdetermed system wheever m ad uder-determed for m If m we obta a square system of cove equatos Most of the tme we wsh to fd a proper R such that () holds as closely as possble hs meas that our obectve s to reduce F as much as ad f possble reduce t to zero Hece the system of cove quatos () ca be wrtte as a ucostraed optmzato problem whch () m f st R () f F (3) It s obvous that f J F where J s the Jacob matr of F I ths wor we cosder a -reularzed least squares problem for system (): m f (4) whch s a parameter We ote that f F F F Fm ad ay F s cove the F s cove O the other had covety of mples that f s cove herefore φ s a cove fucto As a eample Hale Y ad Zha [] preseted a fed-pot cotuato method for -reularzed mmzato that based o operator-spltt ad co tuato: h shr (5) where ad mapps shr h: R R are defed as : h I (6) shr s ma (7) he operator the rht had sde of relato (7) deote the compoet-wse product of s ad ma Because of the parameter τ s costat the umber of teratos ad computatoal costs crease ad so t s ot sutable o overcome the metoed dsadvatae we create ovato the parameter τ based o the steepest descet drecto: Copyrht ScRes

2 38 M KIMIAI F RAHPYMAII (8) J J he aalyss of the ew approach shows that t herts both stablty of fed pot methods ad low computatoal cost of steepest descet methods We also vestate the lobal coverece to frst-order statoary pots of the proposed method ad provde the quadratc coverece rate o show the effcecy of the proposed method practce some umercal results are also reported he rest of ths paper s orazed as follows: I Secto we descrbe the motvato behd the proposed alorthm the paper toether wth the alorthm s structure I Secto 3 we prove that the proposed alorthm s lobally coveret Prelmarly umercal results are reported Secto 4 Fally some coclusos are epressed Secto 5 he New Alorthm: Motvato ad Structure I ths secto we frst troduce a fed pot alorthm for small-scale cove systems of equatos he ve some propertes of the alorthm ad vestate ts lobal coverece as well as the quadratc coverece rate he obectve fucto (4) s a sum of two cove fuctos By cove aalyss mmz a cove fucto s equvalet to fd a zero of the subdfferetal Let X be the set of optmal solutos of (4) It s well-ow that a optmalty codto for (4) s X s or equvaletly X where deotes the zero vector R ad s -th compoet of It follows readly from (9) that s a optmal soluto of (4) f ad oly f or other words X herefore t s easy to chec whether s a soluto of (4) (see []) Oe of the smplest methods for solv (4) eerates a sequece that based o steepest de- scet drecto Here (9) J J ad J F Note that f the system () be a lear system of equatos the A A ad Here the system () s a cove ma A A system of equatos the J J ad for the purpose of our aalyss we wll always choose Us these formato we J J ma preset a promal reularzato of the learzed fucto f at for problem (4) (see [7]) ad wrtte t equvaletly as f ar m f () After or costat terms () ca be rewrtte as ar m () ar m Notce that the fucto the problem () s mmzed f ad oly f each fuctos q s mmzed If we tae the we ca smply obta the mmzer of as follows: q herefore obtaed Now based o above arumets a ew fed pot alorthm ca be outled as follows: Alorthm : Fed pot alorthm (FP) ad the soluto of (4) s Iput: Choose a tal pot R ad costats Be ; l Whle F ad ma {Start loop } Step : {Parameter shrae calculato} Copyrht ScRes

3 M KIMIAI F RAHPYMAII 39 ; Step : {Operato shrae calculato} h shr ; Step 3: {Parameters update} Calculate Geerate as (8); ; Icremet by oe ad o to Step ; d Whle {d loop} d 3 Coverece Aalyss I ths secto we wll ve the coverece aalyss of the proposed alorthm ve Secto I the coverece aalyss we eed the follow assumpto: (H) Problem (4) has a optmal soluto set X ad there ests a set : δ X for some X ad δ such that f s twce cotuously dfferetable o Ω ad ˆ J J ma J J for all ma Us the mea-value theorem we hav y J y t y J y t y J y J y y for ay y (H) here ests a costat M J M N Lemma 3 By the defto of (8) we have such that ad satsfy- for ay Proof Suppose that the ad J J J J So by (8) we coclude that Now suppose that he we show that By cotradcto we assume that he ad J J J J herefore we have that s a cotradcto I the follow lemma we show that the ew choce of satsfy the lemma 4 ad corollary 4 of [] whe ma J J Lemma 3 Uder assumpto (H) the choce of ad result h I s oepasve over Ω e for ay h h () Moreover Proof Let J J ad d Now we have two cases: ) If the Hece wheever equalty holds () h h d J J I J J d h h I J J d d I J J ˆ J J ma ˆ J J J J ˆ J J ˆ ma (3) Let p Jd By the lemma 3 we have d f ad oly f he h h d d d d d f by the equato (3) we obta J J J J p J Jp d J Jd p J Jp whch cotradcts to ˆ J J Hece p so that ) If the J p h h (4) Copyrht ScRes

4 33 M KIMIAI F RAHPYMAII Hece d J J I J J d h h I J J I J J d d ˆ J J ma ˆ J J J J ˆ J J ˆ ma Let p Jd By the lemma 3 we have d f ad oly f he h h d d d d d f by the quato (4) we obta J J J J p J Jp d J Jd whch cotradcts to so that p J Jp ˆ J J Hece p J p whch completes the proof Corollary 33 (Costat optmal radet) From (H) assumpto for ay X there s a vector R such that Let X be the soluto set of (4) X ad be the vector specfed corollary 3 he we defe : : : L where We wll show that the sequece eer- ated by (5) s fte coverece for compoets L ad s quadratc coverece for compoets It s obvous from the optmalty codto (9) that L { } ad for ay X we have supp : L Hale Y Zha [] establshes the fte coverece propertes of stated the follow of theorem he proof of the theorem 34 ad 35 s smlar to the theorem 4 ad 4 [] heorem 34 Uder assumpto (H) the sequece s eerated by the fed pot terato (5) appled to problem (4) from ay start pot coveres to some X I addto for all but ftely may teratos we have L (5) s s h h L (6) where the umbers of teratos ot satsfy (5) ad (6) do ot eceed ad respectvely heorem 35 (he quadratc case) Let f be a cove quadratc fucto that s bouded below J J be ts Hessa ad satsfy ma J J the the sequece s eerated by the fed pot terato (5) appled to problem (4) from ay start pot coveres to some X I addto for all but ftely may teratos we have (5) - (6) hold for all but ftely may teratos Lemma 36 Suppose assumptos (H) ad (H) holds he we have lm F Proof From (9) we ca obta the by (H) ad the above equalty we have M F F F M For suffcetly lare we coclude that lm F Now cosder the sequece eerated by the FP alorthm Accord to the fed pot teratos (5) t covere to some pot X We wll show that the coverece s quadratc I order to do the follow addtoal assumpto s requred: (H3) he follow codto G G O (7) Copyrht ScRes

5 M KIMIAI F RAHPYMAII 33 holds where G J J G J J ad Suppose that s eouh lare so that for all L Also suppose that C C deote the square sub-matr of the matr C correspod to the de set Frstly we suppose that the the mea-value theorem yelds G I G Sce shr h ad shr h h (8) s o-epasve [] us H (7) ad (8) we have that h h I G I G G G G G G G G G G G G M M Secodly we suppose that we coclude that O M O M O Smlar frst case heorem 37 Suppose that (H)-(H3) holds ad let s the sequece eerated by the FP alorthm start For suffcetly lare the sequece s coveres to some pot X quadratcally 4 Prelmary Numercal permets: hs secto reports some umercal results ad comparsos reard the mplemetatos of the ew proposed dea of the preset study wth some other alorthms for small-scale problems All codes are wrtte MALAB 9 proramm evromet wth double precso format by a same subroute I the epermets the preseted alorthms are stopped wheever 5 F est problems are as follows: ) rdaoal system lear s A F where A s a trdaoal matr ve by ad A trda 8 A : A: : A : ) Fve daoal system lear s A F where A s a fve matr ve by ad A trda 8 A : A: : A : 3) Loarthmc fucto [8] F l f that s the ra- tal pot: 4) Strctly cove fucto [8] det of h e tal pot: e F 5) Strctly cove fucto [8] f det of h e F e tal pot: 6) Strctly cove fucto 3 [8] f det of h e e F that s the ra- that s the ra- Copyrht ScRes

6 33 M KIMIAI F RAHPYMAII tal pot: 7) Lear fucto-full ra [8] F tal pot: 8) Pealty fucto [8] 5 F F 4 4 tal pot: 3 3 9) Sum square fucto [9] F tal pot: ) roometrc epoetal fucto [] F ep cos h F ep cos h F ep cos h where h h tal pot: 5 5 I ths secto we compare the umercal results obtaed by ru Alorthms follow: ) FP ) FP J J 3) DS (steepest descet drecto d J J where the Jacoba matr J s eact I ru the alorthm FP ad FP taes advataes of the parame - ters 5 ad ma he dmesos of problems are selected from to he results for small-scale problems are summarzed able I able ad f respectvely dcate the total umber of terates ad the total umber of fucto evaluatos able dcates the total umber of teratos ad fucto evaluatos for some small scale problems wth dmesos to vdetally oe ca see that FP performs better tha the other preseted alorthms the sese of both the total umber of teratos ad the total umber of fucto evaluatos From able we ob- serve that the proposed alorthm s the best oe o the all of test problems We ca deduce that our ew alorthm s more effcet ad robust tha the other cosdered alorthms for solv small scale system of cove equatos problems I more detals the results of able Fure are terpreted thas to the Dola ad More s performace profle [] I the procedure of Dola ad More the profle of each code s measured cosder the rato of ts computatoal outcome versus the best umercal outcome of all codes hs profle offers a tool for compar the performace of teratve processes statstcal structure I partcular let S s set of all alorthms ad P s a set of able Numercal results for small scale problems Problem Dm FP SD FP f f f 5 5/6 6/7 /3 7/8 6/7 /3 5 9/ 7/8 /3 /3 4/5 /3 5 /3 5/6 /3 5 /3 3/4 / /8 5/6 5/6 /3 5/6 4/5 5 8/9 5/6 4/ /33 4/5 8/9 3/33 4/5 8/9 5 3/33 4/5 8/ /84 55/56 3/ /798 3/38 3/4 77/77 8/9 3/ /845 5/6 7/ /866 5/6 7/8 844/845 5/6 7/ / / 9/ 9/ / 9/ 5 9/ / 9/ 8 5 4/4 5/6 5/6 484/485 5/6 5/6 97/973 5/6 5/6 9 5 Over 9/3 6/8 Over 59/6 3/3 5 Over 88/89 46/47 3 7/8 7/8 8/9 5 7/8 5/6 7/8 8/9 4/5 8/9 Copyrht ScRes

7 M KIMIAI F RAHPYMAII 333 small-scale systems of cove equatos that bled steepest descet drecto ad fed pot deas Prelmary umercal effort o the set of small-scale cove systems of equatos dcates that sfcat profts both the total umber of teratos ad the total umber of fucto evaluatos ca be acheved Fure Performace profle for the umber of terates test problems wth s solvers ad p problems For each problem p ad solver s ad t p s s the computato result reard to the performace de he the follow performace rato s defed t ps rps (9) m tps : s S If alorthm s s ot coveret for a problem p the procedure sets rps rfal where r fal should be strctly larer tha ay performace rato (9) For ay factor the overall performace of alorthm s s ve by s sze pp : rp s p I fact s s the probablty of alorthm s S that a performace rato r p s s wth a factor R of the best possble rato he fucto s s the dstrbuto fucto for the performace rato specally s ves the probablty that alorthm s ws over all other alorthms ad lm r ps s ves the probablty of that alorthm s solve a problem herefore ths performace profle ca be cosdered as a measure of the effcecy ad the robustess amo the alorthms I Fure the -as shows the umber whle the y-as hbts Prps : s s From Fure t s clear that FP had the most ws compared wth the other alorthm whle t solved about 6% of the test problems wth the reatest effcecy If oe cocetrates o the ablty of complet a ru successfully t ca be see that FP s the best alorthm amo the cosdered alorthms because t reaches faster tha the other 5 Cocluso I ths paper we have preseted a ew alorthm for RFRNCS [] Z We W Y D Goldfarb ad Y Zha A Fast Alorthm for Sparse Recostructo Based o Sharae Optmzato ad Cotuato CA echcal Report R9-9 [] Hale W Y ad Y Zha A Fed-Pot Cotuato Method for -Reularzed Mmzato wth Applcatos to Compressed Ses CA echcal Report R7-7 7 July 7 [3] A Chambolle R A De Vore N Y Lee ad B J Lucer Nolear Wavelet Imae Process: Varatoal Problems Compresso ad Nose Removal throuh Wavelet Shrae I rasactos o Imae Pro- cess Vol 7 No pp do:9/83668 [4] M A Fueredo ad R D Nowa A M Alorthm for Wavelet-Based Imae Restorato I rasactos o Imae Process Vol No 8 3 pp do:9/ip38455 [5] I Daubeches M Defrse ad C D Mol A Iteratve hreshold Alorthm for Lear Iverse Problems wth a Sparsty Costrat Commucatos o Pure ad Appled Mathematcs Vol 57 No 4 pp do:/cpa4 [6] C Voesch ad M User Fast Iteratve hreshold N Alorthm for Wavelet-Reularzed Decovoluto Proceeds of the SPI Optcs ad Photocs 7 Coferece o Mathematcal Methods: Wavelet XII Vol 67 Sa Deo 6-9 Auust 7 p 5 [7] B Martet Réularsato d Iéquatos Varato Nelles par Appromatos Successves Recherche Operatoelle Vol 4 No 3 97 pp [8] G L Yua Z X We ad S Lu Lmted Memory BFGS Method wth Bactrac for Symmetrc Nolear quatos Mathematcal ad Computer Modell Vol 54 No - pp do:6/mcm [9] M Moa ad C Smutc est Fuctos for Optmzato Need 5 wwwzsdtcpwrpc/fle/docs/fuctopdf [] L Luša ad J Vlče Sparse ad Partally Separable est Problems for Ucostraed ad qualty Costraed Optmzato echcal Report Vol [] D Dola ad J J Moré Bechmar Optmzato Software wth Performace Profles Mathematcal Proramm Vol 9 No pp -3 Copyrht ScRes