Varable Metrc Forward-Backward Splttg wth Applcatos to Mootoe Iclusos Dualty Patrck L. Combettes ad B`ăg Côg Vũ UPMC Uversté Pars 06 Laboratore Jacques-Lous Los UMR CNRS 7598 75005 Pars, Frace plc@math.jusseu.fr, vu@ljll.math.upmc.fr Abstract We propose a varable metrc forward-backward splttg algorthm ad prove ts covergece real Hlbert spaces. We the use ths framework to derve prmal-dual splttg algorthms for solvg varous classes of mootoe clusos dualty. Some of these algorthms are ew eve whe specalzed to the fxed metrc case. Varous applcatos are dscussed. Keywords: cocoercve operator, composte operator, demregularty, dualty, forward-backward splttg algorthm, mootoe cluso, mootoe operator, prmal-dual algorthm, quas-fejér sequece, varable metrc. Mathematcs Subject Classfcatos 2010 47H05, 49M29, 49M27, 90C25 1 Itroducto The forward-backward algorthm has a log hstory gog back to the projected gradet method see [1, 12] for hstorcal backgroud. It addresses the problem of fdg a zero of the sum of two operators actg o a real Hlbert space H, amely, fd x H such that 0 Ax+Bx, 1.1 uder the assumpto that A: H 2 H s maxmally mootoe ad that B: H H s β-cocoercve for some β ]0,+ [,.e. [4], x H y H x y Bx By β Bx By 2. 1.2 Cotact author: P. L. Combettes, plc@math.jusseu.fr, phoe:+33 1 4427 6319, fax:+33 1 4427 7200. The work of B`ăg Côg Vũ was partally supported by Grat 102.01-2012.15 of the Vetam Natoal Foudato for Scece ad Techology Developmet NAFOSTED. 1
Ths framework s qute cetral due to the large class of problems t ecompasses areas such as partal dfferetal equatos, mechacs, evoluto clusos, sgal ad mage processg, best approxmato, covex optmzato, learg theory, verse problems, statstcs, game theory, ad varatoal equaltes [1, 4,7,10,12, 15,18,20, 21, 23,24,29, 30,39,40, 42]. Theforward-backward algorthm operates accordg to the route x 0 H ad N x +1 = Id+γ A x γ Bx, where 0 < γ < 2β. 1.3 I classcal optmzato methods, the beefts of chagg the uderlyg metrc over the course of the teratos to mprove covergece profles has log bee recogzed [19, 33]. I proxmal methods, varable metrcs have bee vestgated mostly whe B = 0 1.1. I such staces 1.3 reduces to the proxmal pot algorthm x 0 H ad N x +1 = Id+γ A x, where γ > 0. 1.4 I the case whe A s the subdfferetal of a real-valued covex fucto a fte dmesoal settg, varable metrc versos of 1.4 have bee proposed [5, 11, 27, 35]. These methods draw heavly o the fact that the proxmal pot algorthm for mmzg a fucto correspods to the gradet descet method appled to ts Moreau evelope. I the same sprt, varable metrc proxmal pot algorthms for a geeral maxmally mootoe operator A were cosdered [8, 36]. I [8], superlear covergece rates were show to be achevable uder sutable hypotheses see also [9] for further developmets. The fte dmesoal varable metrc proxmal pot algorthm proposed [32] allows for errors the proxmal steps ad features a flexble class of exogeous metrcs to mplemet the algorthm. The frst varable metrc forward-backward algorthm appears to be that troduced [10, Secto 5]. It focuses o lear covergece results the case whe A + B s strogly mootoe ad H s fte-dmesoal. The varable metrc splttg algorthm of [28] provdes a framework whch ca be used to solve 1.1 staces whe H s fte-dmesoal ad B s merely Lpschtza. However, t does ot explot the cocoercvty property 1.2 ad t s more cumbersome to mplemet tha the forward-backward terato. Let us add that, the mportat case whe B s the gradet of a covex fucto, the Ballo-Haddad theorem asserts that the otos of cocoercvty ad Lpschtz-cotuty cocde [4, Corollary 18.16]. The goal of ths paper s two-fold. Frst, we propose a geeral purpose varable metrc forwardbackward algorthm to solve 1.1 1.2 Hlbert spaces ad aalyze ts asymptotc behavor, both terms of weak ad strog covergece. Secod, we show that ths algorthm ca be used to solve a broad class of composte mootoe cluso problems dualty by formulatg them as staces of 1.1 1.2 alterate Hlbert spaces. Eve whe restrcted to the costat metrc case, some of these results are ew. The paper s orgazed as follows. Secto 2 s devoted to otato ad backgroud. I Secto 3, we provde prelmary results. The varable metrc forward-backward algorthm s troduced ad aalyzed Secto 4. I Secto 5, we preset a ew varable metrc prmal-dual splttg algorthm for strogly mootoe composte clusos. Ths algorthm s obtaed by applyg the forwardbackward algorthm of Secto 4 to the dual cluso. I Secto 6, we cosder a more geeral class of composte clusos dualty ad show that they ca be solved by applyg the forwardbackward algorthm of Secto 4 to a certa cluso problem posed the prmal-dual product space. Applcatos to mmzato problems, varatoal equaltes, ad best approxmato are dscussed. 2
2 Notato ad backgroud We recall some otato ad backgroud from covex aalyss ad mootoe operator theory see [4] for a detaled accout. Throughout, H, G, ad G 1 m are real Hlbert spaces. We deote the scalar product of a Hlbert space by ad the assocated orm by. The symbols ad deote respectvely weak ad strog covergece, ad Id deotes the detty operator. We deote by BH, G the space of bouded lear operators from H to G, we set BH = BH,H ad SH = { L BH L = L }, where L deotes the adjot of L. The Loewer partal orderg o SH s defed by U SH V SH U V x H Ux x Vx x. 2.1 Now let α [0,+ [. We set P α H = { U SH U αid }, 2.2 ad we deote by U the square root of U P α H. Moreover, for every U P α H, we defe a sem-scalar product ad a sem-orm a scalar product ad a orm f α > 0 by x H y H x y U = Ux y ad x U = Ux x. 2.3 Notato 2.1 WedeotebyG = G 1 G m thehlbertdrectsumofthehlbertspacesg 1 m,.e., ther product space equpped wth the scalar product ad the assocated orm respectvely defed by : x,y x y ad : x m x 2, 2.4 where x = x 1 m ad y = y 1 m deote geerc elemets G. Let A: H 2 H be a set-valued operator. The doma ad the graph of A are respectvely defed by doma = { x H Ax } ad graa = { x,u H H u Ax }. We deote by zera = { x H 0 Ax } the set of zeros of A ad by raa = { u H x H u Ax } the rage of A. The verse of A s A : H 2 H : u { x H u Ax }, ad the resolvet of A s J A = Id+A. 2.5 Moreover, A s mootoe f x,y H H u,v Ax Ay x y u v 0, 2.6 ad maxmally mootoe f t s mootoe ad there exsts o mootoe operator B: H 2 H such that graa grab ad A B. The parallel sum of A ad B: H 2 H s A B = A +B. 2.7 The cojugate of f: H ],+ ] s f : H [,+ ] : u sup x u fx, 2.8 x H 3
ad the fmal covoluto of f wth g: H ],+ ] s f g: H [,+ ] : x f fy+gx y. 2.9 y H The class of lower semcotuous covex fuctos f: H ],+ ] such that domf = { x H fx < + } s deoted by Γ 0 H. If f Γ 0 H, the f Γ 0 H ad the subdfferetal of f s the maxmally mootoe operator f: H 2 H : x { u H y H y x u +fx fy } 2.10 wth verse f = f. Let C be a oempty subset of H. The dcator fucto ad the dstace fucto of C are defed o H as { 0, f x C; ι C : x ad d C = ι C : x f x y. 2.11 +, f x / C y C respectvely. The teror of C s tc ad the support fucto of C s σ C = ι C. Now suppose that C s covex. The ormal coe operator of C s defed as N C = ι C : H 2 H : x {{ u H y C y x u 0 }, f x C;, otherwse. 2.12 The strog relatve teror of C,.e., the set of pots x C such that the cocal hull of x+c s a closed vector subspace of H, s deoted by src; f H s fte-dmesoal, src cocdes wth the relatve teror of C, deoted by r C. If C s also closed, ts projector s deoted by P C,.e., P C : H C: x argm y C x y. Fally, l 1 + N deotes the set of summable sequeces [0,+ [. 3 Prelmary results 3.1 Techcal results The followg propertes ca be foud [26, Secto VI.2.6] see [17, Lemma 2.1] for a alterate short proof. Lemma 3.1 Let α ]0,+ [ ad µ ]0,+ [, ad assume that A ad B are operators SH such that µid A B αid. The the followg hold. α Id B A µ Id. x H A x x A x 2. A α. The ext fact cocers sums of composte cocoercve operators. Proposto 3.2 Let I be a fte dex set. For every I, let 0 L BH,G, let β ]0,+ [, ad let T : G G be β -cocoercve. Set T = I L T L ad β = 1/ I L 2 /β. The T s β-cocoercve. 4
Proof. Set I α = β L 2 /β. The I α = 1 ad, usg the covexty of 2 ad 1.2, we have x H y H x y Tx Ty = I whch cocludes the proof. = I x y L T L x L T L y L x L y T L x T L y β T L x T L y 2 I β L 2 L T L x L T L y 2 I = β 1 α L α T L x L T L y I β L T L x L T L y 2 I = β Tx Ty 2, 3.1 2 3.2 Varable metrc quas-fejér sequeces The followg results are from [17]. Proposto 3.3 Let α ]0,+ [, let W N be P α H, let C be a oempty subset of H, ad let x N be a sequece H such that η N l 1 +N z C ε N l 1 +N N x +1 z W+1 1+η x z W +ε. 3.2 The x N s bouded ad, for every z C, x z W N coverges. Proposto 3.4 Let α ]0,+ [, ad let W N ad W be operators P α H such that W W potwse as +, as s the case whe sup W < + ad η N l 1 + N N 1+η W W +1. 3.3 N Let C be a oempty subset of H, ad let x N be a sequece H such that 3.2 s satsfed. The x N coverges weakly to a pot C f ad oly f every weak sequetal cluster pot of x N s C. Proposto 3.5 Let α ]0,+ [, let W N be a sequece P α H such that sup N W < +, let C be a oempty closed subset of H, ad let x N be a sequece H such that ε N l 1 + N η N l 1 + N z C N x +1 z W+1 1+η x z W +ε. 3.4 The x N coverges strogly to a pot C f ad oly f lmd C x = 0. 5
Proposto 3.6 Let α ]0,+ [, let ν N l 1 +N, ad let W N be a sequece P α H such that sup N W < + ad N 1+ν W +1 W. Furthermore, let C be a subset of H such that tc, let z C ad ρ ]0,+ [ be such that Bz;ρ C, ad let x N be a sequece H such that ε N l 1 +N η N l 1 +N x Bz;ρ N The x N coverges strogly. x +1 x 2 W +1 1+η x x 2 W +ε. 3.5 3.3 Mootoe operators We establsh some results o mootoe operators a varable metrc evromet. Lemma 3.7 Let A: H 2 H be maxmally mootoe, let α ]0,+ [, let U P α H, ad let G be the real Hlbert space obtaed by edowg H wth the scalar product x,y x y U = x U y. The the followg hold. UA: G 2 G s maxmally mootoe. J UA : G G s 1-cocoercve,.e., frmly oexpasve, hece oexpasve. J UA = U +A U. Proof. : Set B = UA ad V = U. For every x,u grab ad every y,v grab, Vu VBx = Ax ad Vv VBy = Ay, so that x y u v V = x y Vu Vv 0 3.6 by mootocty of A o H. Ths shows that B s mootoe o G. Now let y,v H 2 be such that x,u grab x y u v V 0. 3.7 The, for every x,u graa, x,uu grab ad we derve from 3.7 that x y u Vv = x y Uu v V 0. 3.8 Sce A s maxmally mootoe o H, 3.8 gves y,vv graa, whch mples that y,v grab. Hece, B s maxmally mootoe o G. : Ths follows from ad [4, Corollary 23.8]. : Let x ad p be G. The p = J UA x x p + UAp U x U + Ap p = U +A U x. Remark 3.8 let α ]0,+ [, let U P α H, set f: H R: x U x x /2, ad let D: x,y fx fy x y fy be the assocated Bregma dstace. The Lemma 3.7 asserts that J UA = f +A f. I other words, J UA s the D-resolvet of A troduced [3, Defto 3.7]. 6
Let U P α H for some α ]0,+ [. The proxmty operator of f Γ 0 H relatve to the metrc duced by U s [25, Secto XV.4] prox U f : H H: x argm fy+ 1 y H 2 x y 2 U, 3.9 ad the projector oto a oempty closed covex subset C of H relatve to the orm U s deoted by PC U. We have prox U f = J U f ad P U C = proxu ι C, 3.10 ad we wrte prox Id f = prox f. I the case whe U = Id Lemma 3.7, examples of closed form expressos for J UA ad basc resolvet calculus rules ca be foud [4, 15, 18]. A few examples llustratg the case whe U Id are provded below. The frst result s a exteso of the well-kow resolvet detty J A +J A = Id. Example 3.9 Let α ]0,+ [, let γ ]0,+ [, ad let U P α H. The the followg hold. Let A: H 2 H be maxmally mootoe. The J γua = UJ γ UA U U = Id γujγ U A γ U. 3.11 Let f Γ 0 H. The prox U γf = U prox γf U U = Id γu prox U γ f γ U. Let C be a oempty closed covex subset of H. The prox U γσ C = U prox γσc U U = Id γu PC U γ U. Proof. : Let x ad p be H. The p = J γua x x p γuap U x U p γ UA U U p U p = J γ UA U U x p = UJ γ UA U U x. 3.12 Furthermore, by [4, Proposto 23.23], J UγA U = Id U U +γa U. Hece, 3.12 yelds However J γua = Id U U +γa. 3.13 p = U +γa x x Up+γA p γ p Ax Up x Up A γ p γ U x Id+γ U A γ p γ p = J γ U A γ U x. 3.14 7
Hece, U +γa = γj γ U A γ U ad, usg 3.13, we obta the rghtmost detty. : ApplytoA = f, aduse3.10adthefactthat f U = U f U = U f U [4, Corollary 16.42]. : Apply to f = σ C, ad use 3.10. Example 3.10 Defe G as Notato 2.1, let α R, ad, for every {1,...,m}, let A : G 2 G be maxmally mootoe ad let U P α G. Set A: G 2 G : x 1 m m A x ad U: G G: x 1 m U x 1 m. The UA s maxmally mootoe ad x 1 m G J UA x 1 m = J U A x 1 m. 3.15 Proof. Ths follows from Lemma 3.7 ad [4, Proposto 23.16]. Example 3.11 Let α ]0,+ [, let ξ R, let U P α H, let φ Γ 0 R, suppose that 0 u H, ad set H = { x H x u ξ } ad g = φ u. The g Γ 0 H ad x H prox U g x = x+ prox x u x u U u 2 φ U u 3.16 U u 2 ad x, f x u ξ; PHx U = ξ x u x+ u U u U u, f x u > ξ. 3.17 Proof. It follows from Example 3.9 that x H prox U g x = U prox g U Ux. 3.18 Moreover, g U = φ U u. Hece, usg 3.18 ad [4, Corollary 23.33], we obta x H prox U g x = U prox φ U u Ux = x+ prox x u x u U u 2 φ U u. 3.19 U u 2 Fally, upo settg φ = ι ],ξ], we obta 3.17 from 3.16. Example 3.12 Let α ]0,+ [, let γ R, let A P 0 H, let u H, let U P α H, ad set ϕ: H R: x Ax x /2+ x u +γ. The ϕ Γ 0 H ad x H prox U ϕ x = Id+U A x U u. 3.20 Proof. Let x H. The p = prox U ϕ x x p = U ϕp x p = U Ap+u x U u = Id+U Ap p = Id+U A x U u. 8
Example 3.13 Let α ]0,+ [ ad let U P α H. For every {1,...,m}, let r G, let ω ]0,+ [, ad let L BH,G. Set ϕ: x 1/2 m ω L x r 2. The ϕ Γ 0 H ad x H prox U ϕ x = m Id+U ω L L m x+u ω L r. 3.21 Proof. We have ϕ: x Ax x /2+ x u +γ, where A = m ω L L, u = m ω L r, ad γ = m ω r 2 /2. Hece, 3.21 follows from 3.20. 3.4 Demregularty Defto 3.14 [1, Defto 2.3] A operator A: H 2 H s demregular at x doma f, for every sequece x,u N graa ad every u Ax such that x x ad u u as +, we have x x as +. Lemma 3.15 [1, Proposto 2.4] Let A: H 2 H be mootoe ad suppose that x doma. The A s demregular at x each of the followg cases. A s uformly mootoe at x,.e., there exsts a creasg fucto φ: [0,+ [ [0,+ ] that vashes oly at 0 such that u Ax y,v graa x y u v φ x y. A s strogly mootoe,.e., there exsts α ]0,+ [ such that A αid s mootoe. J A s compact,.e., for every bouded set C H, the closure of J A C s compact. I partcular, doma s boudedly relatvely compact,.e., the tersecto of ts closure wth every closed ball s compact. v A: H H s sgle-valued wth a sgle-valued cotuous verse. v A s sgle-valued o dom A ad Id A s demcompact,.e., for every bouded sequece x N doma such that Ax N coverges strogly, x N admts a strog cluster pot. v A = f, where f Γ 0 H s uformly covex at x,.e., there exsts a creasg fucto φ: [0,+ [ [0,+ ] that vashes oly at 0 such that α ]0,1[ y domf f αx+1 αy +α1 αφ x y αfx+1 αfy. v A = f, where f Γ 0 H ad, for every ξ R, { x H fx ξ } s boudedly compact. 4 Algorthm ad covergece Our ma result s stated the followg theorem. Theorem 4.1 Let A: H 2 H be maxmally mootoe, let α ]0,+ [, let β ]0,+ [, let B: H H be β-cocoercve, let η N l 1 +N, ad let U N be a sequece P α H such that µ = sup U < + ad N 1+η U +1 U. 4.1 N 9
Let ε ]0,m{1,2β/µ +1}[, let λ N be a sequece [ε,1], let γ N be a sequece [ε,2β ε/µ], let x 0 H, ad let a N ad b N be absolutely summable sequeces H. Suppose that Z = zera+b, 4.2 ad set N y = x γ U Bx +b x +1 = x +λ JγU Ay +a x. 4.3 The the followg hold for some x Z. x x as +. N Bx Bx 2 < +. Suppose that oe of the followg holds. a lmd Z x = 0. b At every pot Z, A or B s demregular see Lemma 3.15 for specal cases. c tz ad there exsts ν N l 1 +N such that N 1+ν U U +1. The x x as +. Proof. Set N { A = γ U A B = γ U B ad p = J A y q = J A x B x s = x +λ q x. 4.4 The 4.3 ca be wrtte as N x +1 = x +λ p +a x. 4.5 O the other had, 4.1 ad Lemma 3.1& yeld N U 1 α, U P 1/µ H, ad 1+η U U+1 4.6 ad, therefore, N x H 1+η x 2 U x 2. 4.7 U +1 Hece, we derve from 4.5, 4.4, Lemma 3.7, 4.6 ad 4.1 that N x +1 s U λ a U + p q U a U a U U + y x +B x U +γ U b U a +γ U b 1 2β ε a + b. 4.8 α µ 10
Now let z Z. Sce B s β-cocoercve, N x z Bx Bz β Bx Bz 2. 4.9 O the other had, t follows from 4.1 that N B x B z 2 γ 2 U U Bx Bz 2 γ 2 µ Bx Bz 2. 4.10 We also ote that, sce Bz Az, 4.4 yelds N z = J A z B z. 4.11 Altogether, t follows from 4.4, 4.11, Lemma 3.7, 4.9, ad 4.10 that N q z 2 U x z B x B z 2 U x q B x B z 2 U = x z 2 U 2 x z B x B z U + B x B z 2 U x q B x B z 2 U = x z 2 U I tur, we derve from 4.7 ad 4.4 that 2γ x z Bx Bz + B x B z 2 U x q B x B z 2 U x z 2 U γ 2β µγ Bx Bz 2 x q B x B z 2 U x z 2 U ε2 Bx Bz 2 x q B x B z 2 U. 4.12 N 1+η s z 2 s U z 2 +1 U 1 λ x z 2 +λ U q z 2 U x z 2 ε 3 Bx U Bz 2 ε x q B x B z 2, 4.13 U whch mples that N s z 2 1+η U x z 2 ε 3 Bx +1 U Bz 2 ε x q B x B z 2 U 4.14 δ 2 x z 2 U 4.15 where δ = sup 1+η. N Next, we set 1 N ε = δ α a + 4.16 2β ε b. 4.17 µ 11
The our assumptos yeld ε < +. N 4.18 Moreover, usg 4.7, 4.14, ad 4.8, we obta N x +1 z U x +1 s +1 U + s z +1 U +1 1+η x +1 s U δ x +1 s U + 1+η x z U + 1+η x z U 1+η x z U +ε 1+η x z U +ε. 4.19 I vew of 4.6, 4.18, ad 4.19, we ca apply Proposto 3.3 to assert that x z U N coverges ad, therefore, that ζ = sup x z U < +. 4.20 N O the other had, 4.7, 4.8, ad 4.17 yeld N x +1 s 2 1+η U x +1 s 2 ε 2 +1 U. 4.21 Hece, usg 4.14, 4.15, 4.16, ad 4.20, we get N x +1 z 2 s U z 2 +2 s +1 U z +1 U x +1 s +1 U + x +1 s 2 +1 U +1 1+η x z 2 ε 3 Bx U Bz 2 ε x q B x +B z 2 U +2δζε +ε 2 x z 2 U ε3 Bx Bz 2 ε x q B x +B z 2 U +ζ 2 η +2δζε +ε 2. 4.22 Cosequetly, for every N N, ε 3 N =0 Bx Bz 2 x 0 z 2 U 0 ζ 2 + N =0 x N+1 z 2 U N+1 + N ζ 2 η +2δζε +ε 2 =0 ζ 2 η +2δζε +ε 2. 4.23 Appealg to 4.18 ad the summablty of η N, takg the lmt as N +, yelds N Bx Bz 2 1 ε 3 ζ 2 + N ζ 2 η +2δζε +ε 2 < +. 4.24 We lkewse derve from 4.22 that x q B x +B z 2 U N < +. 4.25 12
: Let x be a weak sequetal cluster pot of x N, say x k x as +. I vew of 4.19, 4.6, ad Proposto 3.4, t s eough to show that x Z. O the oe had, 4.24 yelds Bx k Bz as +. O the other had, sce B s cocoercve, t s maxmally mootoe [4, Example 20.28] ad ts graph s therefore sequetally closed H weak H strog [4, Proposto 20.33]. Ths mples that Bx = Bz ad hece that Bx k Bx as +. Thus, vew of 4.24, Bx Bx 2 < +. N Now set 4.26 N u = 1 γ U x q Bx. 4.27 The t follows from 4.4 that N u Aq. 4.28 I addto, 4.4, 4.6, ad 4.25 yeld u +Bx = 1 U γ x q B x +B x 1 εα x q B x +B x µ εα x q B x +B x U 0 as +. 4.29 Moreover, t follows from 4.4, 4.1, ad 4.26 that x q x q B x +B x + B x B x x q B x +B x +γ U Bx Bx x q B x +B x +2β ε Bx Bx 0 as +. 4.30 ad, therefore, sce x k x as +, that q k x as +. To sum up, { q k x as +, ad N q k,u k graa. 4.31 u k Bx Hece, usg the sequetal closedess of graa H weak H strog [4, Proposto 20.33], we coclude that Bx Ax,.e., x Z. : Sce x Z, the clam follows from 4.24. : We ow prove strog covergece. a: Sce A ad B are maxmally mootoe ad domb = H, A+B s maxmally mootoe [4, Corollary 24.4] ad Z s therefore closed [4, Proposto 23.39]. Hece, the clam follows from, 4.19, ad Proposto 3.5. 13
b: It follows from ad 4.30 that q x Z as + ad from 4.29 that u Bx Ax as +. Hece, f A s demregular at x, 4.28 yelds q x as +. I vew of 4.30, we coclude that x x as +. Now suppose that B s demregular at x. The sce x x Z as + by ad Bx Bx as + by, we coclude that x x as +. c: Suppose that z tz ad fx ρ ]0,+ [ such that Bz;ρ Z. It follows from 4.20 that θ = sup x Bz;ρ sup N x x U 1/ αsup N x z + sup x Bz;ρ x z < + ad from 4.22 that N x Bz;ρ x +1 x 2 x U x 2 +θ 2 η +1 U +2δθε +ε 2. 4.32 Hece, the clam follows from, Lemma 3.1, ad Proposto 3.6. Remark 4.2 Here are some observatos o Theorem 4.1. Supposethat NU = Id. The4.3relapsestotheforward-backwardalgorthmstuded [1, 12], whch tself captures those of [27, 29, 40]. Theorem 4.1 exteds the covergece results of these papers. As show [18, Remark 5.12], the covergece of the forward-backward terates to a soluto may be oly weak ad ot strog, hece the ecessty of the addtoal codtos Theorem 4.1. I Eucldea spaces, codto 4.1 was used [32] a varable metrc proxmal pot algorthm ad the [28] a more geeral splttg algorthm. Next, we descrbe drect applcatos of Theorem 4.1, whch yeld ew varable metrc splttg schemes. We start wth mmzato problems, a area whch the forward-backward algorthm has foud umerous applcatos, e.g., [15, 18, 21, 39, 40]. Example 4.3 Let f Γ 0 H, let α ]0,+ [, let β ]0,+ [, let g: H R be covex ad dfferetable wth a 1/β-Lpschtza gradet, let η N l 1 + N, ad let U N be a sequece P α H such that 4.1 holds. Furthermore, let ε ]0,m{1,2β/µ +1}[ where µ s gve by 4.1, let λ N be a sequece [ε,1], let γ N be a sequece [ε,2β ε/µ], let x 0 H, ad let a N ad b N be absolutely summable sequeces H. Suppose that Argmf +g ad set N y = x γ U gx +b x +1 = x +λ prox U γ y f +a x. 4.33 The the followg hold for some x Argmf +g. x x as +. N gx gx 2 < +. Suppose that oe of the followg holds. a lmd Argmf+g x = 0. 14
b At every pot Argmf +g, f or g s uformly covex see Lemma 3.15v. c targmf+g ad there exsts ν N l 1 + N such that N 1+ν U U +1. The x x as +. Proof. A applcato of Theorem 4.1 wth A = f ad B = g, sce the Ballo-Haddad theorem [4, Corollary 18.16] esures that g s β-cocoercve ad sce, by[4, Corollary 26.3], Argmf +g = zera+b. The ext example addresses varatoal equaltes, aother area of applcato of forwardbackward splttg [4, 23, 39, 40]. Example 4.4 Let f Γ 0 H, let α ]0,+ [, let β ]0,+ [, let B: H H be β-cocoercve, let η N l 1 +N, ad let U N be a sequece P α H that satsfes 4.1. Furthermore, let ε ]0,m{1,2β/µ +1}[ where µ s gve by 4.1, let λ N be a sequece [ε,1], let γ N be a sequece [ε,2β ε/µ], let x 0 H, ad let a N ad b N be absolutely summable sequeces H. Suppose that the varatoal equalty fd x H such that y H x y Bx +fx fy 4.34 admts at least oe soluto ad set N y = x γ U Bx +b x +1 = x +λ prox U γ y 4.35 f +a x. The x N coverges weakly to a soluto x to 4.34. Proof. Set A = f Theorem 4.1. 5 Strogly mootoe clusos dualty I [13], strogly covex composte mmzato problems of the form mmze x H fx+glx r+ 1 2 x z 2, 5.1 where z H, r G, f Γ 0 H, g Γ 0 G, ad L BH,G, were solved by applyg the forward-backward algorthm to the Fechel-Rockafellar dual problem mmze v G f z L v+g v+ v r, 5.2 where f = f 2 /2 deotes the Moreau evelope of f. Ths framework was show to capture ad exted varous formulatos areas such as sparse sgal recovery, best approxmato theory, ad verse problems. I ths secto, we use the results of Secto 4 to geeralze ths framework several drectos smultaeously. Frst, we cosder geeral mootoe clusos, ot just mmzato problems. Secod, we corporate parallel sum compoets see 2.7 the model. Thrd, our algorthm allows for a varable metrc. The followg problem s formulated usg the dualty framework of [16], whch tself exteds those of [2, 22, 31, 34, 37, 38]. 15
Problem 5.1 Let z H, let ρ ]0,+ [, let A: H 2 H be maxmally mootoe, ad let m be a strctly postve teger. For every {1,...,m}, let r G, let B : G 2 G be maxmally mootoe, let ν ]0,+ [, let D : G 2 G be maxmally mootoe ad ν -strogly mootoe, ad suppose that 0 L BH,G. Furthermore, suppose that z ra A+ The problem s to solve the prmal cluso fd x H such that z Ax+ together wth the dual cluso L B D L r +ρid. 5.3 L B D L x r +ρx, 5.4 fd v 1 G 1,..., v m G m such that {1,...,m} r L J ρ A ρ z L j v j j=1 B v D v. 5.5 Let us start wth some propertes of Problem 5.1. Proposto 5.2 I Problem 5.1, set x = J ρ M ρ z, where M = A+ The the followg hold. L B D L r. 5.6 x s the uque soluto to the prmal problem 5.4. The dual problem 5.5 admts at least oe soluto. Let v 1,...,v m be a soluto to 5.5. The x = J ρ A ρ z m L v. v Codto 5.3 s satsfed for every z H f ad oly f M s maxmally mootoe. Ths s true whe oe of the followg holds. a The cocal hull of { L E = x r v s a closed vector subspace. 1 m x doma ad v 1 m m ra B +D } 5.7 b A = f for some f Γ 0 H, for every {1,...,m}, B = g for some g Γ 0 G ad D = l for some strogly covex fucto l Γ 0 G, ad oe of the followg holds. 1/ r 1,...,r m sr { L x y 1 m x domf ad {1,...,m} y domg +doml }. 2/ For every {1,...,m}, g or l s real-valued. 16
3/ H ad G 1 m are fte-dmesoal, ad there exsts x r domf such that {1,...,m} L x r r domg +r doml. 5.8 Proof. : It follows from our assumptos ad [4, Proposto 20.10] that ρ M s a mootoe operator. Hece, J ρ M s a sgle-valued operator wth doma raid+ρ M [4, Proposto 23.9]. Moreover, 5.3 ρ z raid+ρ M = domj ρ M, ad, vew of 2.5, the cluso 5.4 s equvalet to x = J ρ Mρ z. &: It follows from 2.5 ad 2.7 that v 1 G 1 v m G m v 1 G 1 v m G m { v 1,...,v m solves 5.5 x = J ρ A ρ z m j=1 L j v j { {1,...,m} v B D L x r z m L v Ax+ρx { {1,...,m} r L x B v D x = J ρ A ρ z m j=1 L j v j v. 5.9 v: It follows from Mty s theorem [4, Theorem 21.1], that M +ρid s surjectve f ad oly f M s maxmally mootoe. va: Usg Notato 2.1, let us set L: H G: x L x 1 m ad B: G 2G : y B D y r 1 m. 5.10 The t follows from 5.6 that M = A+L B L ad from 5.7 that E = LdomA domb. Hece, sce coee = spae, vew of [6, Secto 24], to coclude that M s maxmally mootoe, t s eough to show that B s. For every {1,...,m}, sce D s maxmally mootoe ad strogly mootoe, domd = rad = G [4, Proposto 22.8] ad t follows from [4, Proposto 20.22 ad Corollary 24.4] that B D s maxmally mootoe. Ths shows that B s maxmally mootoe. vb: Ths follows from [16, Proposto 4.3]. Remark 5.3 I coecto wth Proposto 5.2v, let us ote that eve the smple settg of ormal coe operators fte dmeso, some costrat qualfcato s requred to esure the exstece of a prmal soluto for every z H. To see ths, suppose that, Problem 5.1, H s the Eucldea plae, m = 1, ρ = 1, G 1 = H, L 1 = Id, z = ζ 1,ζ 2, r 1 = 0, D 1 = {0}, A = N C, ad B 1 = N K, where C = { ξ 1,ξ 2 R 2 ξ 1 2 +ξ 2 2 1} ad K = { ξ 1,ξ 2 R 2 ξ 1 0 }. The doma +B 1 +Id = doma domb 1 = C K = {0} ad the prmal cluso z Ax+B 1 x+x reduces to ζ 1,ζ 2 N C 0+N K 0 = ],0] {0}+[0,+ [ {0} = R {0}, whch has o soluto f ζ 2 0. Here coedoma domb 1 = coec K = K s ot a vector subspace. I the followg result we derve from Theorem 4.1 a parallel prmal-dual algorthm for solvg Problem 5.1. 17
Corollary 5.4 I Problem 5.1, set β = max 1 m 1 1 + 1 ν ρ 1 m L 2. 5.11 Let a N be a absolutely summable sequece H, let α ]0,+ [, ad let η N l 1 +N. For every {1,...,m}, let v,0 G, let b, N ad d, N be absolutely summable sequeces G, ad let U, N be a sequece P α G. Suppose that µ = max sup 1 m N U, < + ad {1,...,m} N 1+η U,+1 U,. 5.12 Let ε ]0,m{1,2β/µ +1}[, let λ N be a sequece [ε,1], ad let γ N be a sequece [ε,2β ε/µ]. Set s = z m L v, x = J ρ Aρ s +a N For = 1,...,m w, = v, +γ U, L x r D 5.13 v, d, v,+1 = v, +λ J γu, B w, +b, v,. The the followg hold for the soluto x to 5.4 ad for some soluto v 1,...,v m to 5.5. {1,...,m} v, v as +. I addto, x = J ρ A ρ z m L v. x x as +. Proof. For every {1,...,m}, sce D s maxmally mootoe ad ν -strogly mootoe, D s ν -cocoercve wth domd = rad = G [4, Proposto 22.8]. Let us defe G as Notato 2.1, ad let us troduce the operators T : H H: x J ρ A ρ z x A: G 2 G : v B v 1 m D: G G: v r +D 5.14 v 1 m L: H G: x L x 1 m ad N U : G G: v U, v 1 m. 5.15 : I vew of 2.4 ad 5.14, A s maxmally mootoe, 5.16 D s m 1 m ν -cocoercve, Lemma 3.7 mples that T s ρ-cocoercve, 5.17 18
whle L 2 m L 2. Hece, we derve from 5.11 ad Proposto 3.2 that B = D LTL s β-cocoercve. 5.18 Moreover, t follows from 5.12, 5.15, ad 2.4 that sup U = µ ad N 1+η U +1 U P α G. 5.19 N Now set N a = b, 1 m b = d, L a 1 m v = v, 1 m w = w, 1 m. 5.20 The N a < +, N b < +, ad 5.13 ca be rewrtte as N w = v γ U Bv +b v +1 = v +λ JγU Aw +a v. 5.21 Furthermore, the dual problem 5.5 s equvalet to fd v G such that 0 Av +Bv 5.22 whch, vew of 5.16, 5.18, ad Proposto 5.2, ca be solved usg 5.21. Altogether, the clams follow from Theorem 4.1 ad Proposto 5.2. : Set N z = x a. It follows from, 5.13 ad 5.14 that x = TL v ad N z = TL v. 5.23 I tur, we deduce from 5.17,, 5.18, ad the mootocty of D that ρ z x 2 = ρ TL v TL v 2 L v v TL v TL v v v LTL v LTL v v v Dv Dv v v LTL v LTL v = v v Bv Bv δ Bv Bv, 5.24 where δ = sup N v v < + by. Therefore, t follows from 5.21 ad Theorem 4.1 that z x 0. Sce a 0 as +, we coclude that x x as +. Remark 5.5 Here are some observatos o Corollary 5.4. At terato, thevectors a, b,, ad d, model errors the mplemetato of the olear operators. Note also that, thaks to Example 3.9, the computato of v,+1 5.13 ca be mplemeted usg J γ U, B rather tha J γ U, B. 19
Corollary 5.4 provdes a geeral algorthm for solvg strogly mootoe composte clusos whch s ew eve the fxed stadard metrc case,.e., {1,...,m} N U, = Id. The followg example descrbes a applcato of Corollary 5.4 to strogly covex mmzato problems whch exteds the prmal-dual formulato 5.1 5.2 of [13] ad solves t wth a varable metrc scheme. It also exteds the framework of [14], where f = 0 ad {1,...,m} l = ι {0} ad N U, = Id. Example 5.6 Let z H, let f Γ 0 H, let α ]0,+ [, let η N l 1 +N, let a N be a absolutelysummablesequeceh, adletmbeastrctlypostveteger. Forevery {1,...,m}, let r G, let g Γ 0 G, let ν ]0,+ [, let l Γ 0 G be ν -strogly covex, let v,0 G, let b, N ad d, N be absolutely summable sequeces G, let U, N be a sequece P α G, ad suppose that 0 L BH,G. Furthermore, suppose that see Proposto 5.2vb for specal cases z ra f + The prmal problem s L g l L r +Id. 5.25 mmze x H fx+ g l L x r + 1 2 x z 2, 5.26 ad the dual problem s mmze f z v 1 G 1,...,v m G m L v + g v +l v + v r. 5.27 Suppose that 5.12 holds, let ε ]0,m{1,2β/µ +1}[, let λ N be a sequece [ε,1], ad let γ N be a sequece [ε,2β ε/µ], where β s defed 5.11 ad µ 5.12. Set N s = z m L v, x = prox f s +a For = 1,...,m w, = v, +γ U, L x r l v, d, v,+1 = v, +λ prox U, γ g w, +b, v,. 5.28 The 5.26 admts a uque soluto x ad the followg hold for some soluto v 1,...,v m to 5.27. {1,...,m} v, v as +. I addto, x = prox f z m L v. x x as +. Proof. Set ρ = 1, A = f, ad {1,...,m} B = g ad D = l. 5.29 20
It follows from [4, Theorem 20.40] that the operators A, B 1 m, ad D 1 m are maxmally mootoe. We also observe that 5.25 mples that 5.3 s satsfed. Moreover, for every {1,...,m}, D s ν -strogly mootoe [4, Example 22.3v], l s Fréchet dfferetable o G [4, Corollary 13.33 ad Theorem 18.15], ad D = l = l = { l } [4, Corollary 16.24 ad Proposto 17.26]. Sce, for every {1,...,m}, doml = G, [4, Proposto 24.27] yelds {1,...,m} B D = g l = g l, 5.30 whle [4, Corollares 16.24 ad 16.38] yeld {1,...,m} B +D = g +{ l } = g +l. 5.31 Moreover, 3.10 mples that 5.28 s a specal case of 5.13. Hece, vew of Corollary 5.4, t remas to show that 5.4 ad 5.5 yeld 5.26 ad 5.27, respectvely. Let us set q = 2 /2. We derve from [4, Example 16.33] that f +q z = f +Id z. 5.32 O the other had, t follows from 5.25 ad [4, Proposto 16.5] that f +q z + L g l L r f +q z+ g l L r 5.33 ad that x fx+ m g l L x r + x z 2 /2 s a strogly covex fucto Γ 0 H. Therefore [4, Corollary 11.16] asserts that 5.26 possesses a uque soluto x. Next, we deduce from 5.32, 5.29, 5.30, ad Fermat s rule [4, Theorem 16.2] that, for every x H, x solves 5.4 z fx+ L g l L x r +x 0 f +q z m x+ L g l L r x 0 f +q z+ g l L r x x solves 5.26. 5.34 Fally, set L: H G: x L x 1 m ad h: G ],+ ] : v m g v + l v + v r. Werecallthat f = f q sfréchet dfferetableohwth f = prox f [4,Remark14.4]. Hece, t follows from 5.29, 5.31, [4, Proposto 16.8 ad Theorem 16.37], ad Fermat s rule [4, Theorem 16.2] that, for every v = v 1 m G, v solves 5.5 {1,...,m} r L J A z L j v j j=1 {1,...,m} r L prox f z 0,...,0 L f z L v + B v D v L jv j g +l v j=1 m g +l + r v = L f z L v + hv = f z L +h v v solves 5.27, 5.35 21
whch completes the proof. We coclude ths secto wth a applcato to a composte best approxmato problem. Example 5.7 Letz H, let C beaclosed covex subsetofh, let α ]0,+ [, let η N l 1 + N, let a N be a absolutely summable sequece H, ad let m be a strctly postve teger. For every {1,...,m}, let r G, let D be a closed covex subset of G, let v,0 G, let b, N be a absolutely summable sequece G, let U, N be a sequece P α G, ad suppose that 0 L BH,G. The problem s mmze x C L 1 x r 1 +D 1. L mx r m+d m x z. 5.36 Suppose that 5.12 holds, that max 1 m sup N U, m L 2 < 2, ad that r 1,...,r m sr { L x y 1 m x C ad {1,...,m} y D }. 5.37 Set N s = z m L v, x = P C s +a For = 1,...,m w, = v, +U, L x r v,+1 = w, U, P U, D U, w, +b,. 5.38 The x N coverges strogly to the uque soluto x to 5.36. Proof. Set f = ι C ad {1,...,m} g = ι D, l = ι {0}, ad N γ = λ = 1 ad d, = 0. The 5.37 ad Proposto 5.2vb1/ mply that 5.25 s satsfed. Moreover, vew of Example 3.9, 5.38 s a specal case of 5.28. Hece, the clam follows from Example 5.6. 6 Iclusos volvg cocoercve operators We revst a prmal-dual problem vestgated frst [16], ad the [41] wth the scearo descrbed below. Problem 6.1 Let z H, let A: H 2 H be maxmally mootoe, let µ ]0,+ [, let C: H H be µ-cocoercve, ad let m be a strctly postve teger. For every {1,...,m}, let r G, let B : G 2 G be maxmally mootoe, let ν ]0,+ [, let D : G 2 G be maxmally mootoe ad ν -strogly mootoe, ad suppose that 0 L BH,G. The problem s to solve the prmal cluso fd x H such that z Ax+ L B D L x r +Cx, 6.1 22
together wth the dual cluso fd v 1 G 1,..., v m G m such that x H { z m L v Ax+Cx {1,...,m} v B D L x r. 6.2 Corollary 6.2 I Problem 6.1, suppose that ad set z ra A+ L B D L r +C, 6.3 β = m{µ,ν 1,...,ν m }. 6.4 Let ε ]0,m{1,β}[, let α ]0,+ [, let λ N be a sequece [ε,1], let x 0 H, let a N ad c N be absolutely summable sequeces H, ad let U N be a sequece P α H such that N U +1 U. For every {1,...,m}, let v,0 G, ad let b, N ad d, N be absolutely summable sequeces G, ad let U, N be a sequece P α G such that N U,+1 U,. For every N, set m δ = U, L U 2, 6.5 ad suppose that ζ = δ 1+δ max{ U, U 1,,..., U m, } 1 2β ε. 6.6 Set N p = J UA x U m L v, +Cx +c z +a y = 2p x x +1 = x +λ p x For = 1,...,m v, +U, L y D v, d, r +b, q, = J U, B v,+1 = v, +λ q, v,. 6.7 The the followg hold for some soluto x to 6.1 ad some soluto v 1,...,v m to 6.2. x x as +. {1,...,m} v, v as +. Suppose that C s demregular at x. The x x as +. v Suppose that, for some j {1,...,m}, D j s demregular at v j. The v j, v j as +. 23
Proof. Defe G as Notato 2.1 ad set K = H G. We deote the scalar product ad the orm of K by ad, respectvely. As show [16, 41], the operators A : K 2 K : x,v 1,...,v m m L v z +Ax r 1 L 1 x+b1 v 1... r m L m x+bm v m B : K K: x,v 1,...,v m Cx,D1 v 1,...,D m v m 6.8 m S : K K: x,v 1,...,v m L v, L 1 x,..., L m x are maxmally mootoe ad, moreover, B s β-cocoercve [41, Eq. 3.12]. Furthermore, as show [16, Secto 3], uder codto 6.3, zera+b ad x,v zera+b x solves 6.1 ad v solves 6.2. 6.9 Next, for every N, defe U : K K: x,v 1,...,v m U x,u 1, v 1,...,U m, v m V : K K: x,v 1,...,v m U x m L v, L x+u U1, T : H G: x L 1 x,..., U m, L m x., v 1 m 6.10 It follows from our assumptos ad Lemma 3.1 that N U +1 U P α K ad U 1 α. 6.11 Moreover, for every N, V SK sce U SK. I addto, 6.10 ad 6.11 yeld N V U + S ρ, where ρ = 1 α + m L 2. 6.12 O the other had, N x H T x 2 = where N β = m U, L U U x 2 x 2 U U, L U 2 = β x 2 U, 6.13 U, L U 2. Hece, 6.5 yelds N 1+δ β = 1 1+δ. 6.14 24
Therefore, for every N ad every x = x,v 1,...,v m K, usg 6.10, 6.13, 6.14, Lemma 3.1, ad 6.6, we obta x V x = x U x + = x 2 U = x 2 U + m + m v U, v 2 v 2 2 U, v 2 U, L x v U, L x U, v 2 1+δ β T x 1+δ β U1, v1,..., U m, vm x 2 + v U 2 T x 2 +1+δ U β v 2, 1+δ β U, x 2 U + m x 2 v 2 U U, 1+δ +1+δ β v 2 U, = δ x 2 + 1+δ U δ 1+δ U x 2 + v 2 U, U, v 2 ζ x 2. 6.15 I tur, t follows from Lemma 3.1 ad 6.6 that N V 1 ζ 2β ε. 6.16 Moreover, by Lemma 3.1, N U +1 U U U. Furthermore, we derve from Lemma 3.1 ad 6.12 that V Altogether, +1 V V +1 V +1 x K V x x V x 2 1 ρ x 2. 6.17 sup V 2β ε N ad N V +1 V P 1/ρ K. 6.18 Now set, for every N, x = x,v 1,,...,v m, y = p,q 1,,...,q m, a = a,b 1,,...,b m, c = c,d 1,,...,d m, d = U a,u1, b 1,,...,U m, b m, ad b = S +V a +c d. 6.19 25
The N a < +, N c < +, ad N d < +. Therefore 6.12 mples that N b < +. Furthermore, usg the same argumets as [41, Eqs. 3.22 3.35], we derve from 6.7 ad 6.8 that N x +1 = x +λ J V A x V Bx +b +a x. 6.20 We observe that 6.20 has the structure of the varable metrc forward-backward splttg algorthm 4.3, where N γ = 1. Fally, 6.16 ad 6.18 mply that all the codtos Theorem 4.1 are satsfed. &: Theorem 4.1 asserts that there exsts x = x,v 1,...,v m zera+b 6.21 such that x x as +. I vew of 6.9, the assertos are proved. &v: It follows from Theorem 4.1 that Bx Bx as +. Hece, 6.8, 6.19, ad 6.21 yeld Cx Cx ad {1,...,m} D v, D v as +. 6.22 Hece the results follow from & ad Defto 3.14. Remark 6.3 I the case whe C = ρid for some ρ ]0,+ [, Problem 6.1 reduces to Problem 5.1. However, the algorthm obtaed Corollary 5.2 s qute dfferet from that of Corollary 6.2. Ideed, the former was obtaed by applyg the forward-backward algorthm 4.3 to the dual cluso, whch was made possble by the strog mootocty of the prmal problem. By cotrast, the latter reles o a applcato of 4.3 a prmal-dual product space. Example 6.4 Let z H, let f Γ 0 H, let µ ]0,+ [, let h: H R be covex ad dfferetable wth a µ -Lpschtza gradet, let a N ad c N be absolutely summable sequeces H, let α ]0,+ [, let m be a strctly postve teger, ad let U N be a sequece P α H such that N U +1 U. For every {1,...,m}, let r G, let g Γ 0 G, let ν ]0,+ [, let l Γ 0 G be ν -strogly covex, let v,0 G, let b, N ad d, N be absolutely summable sequeces G, suppose that 0 L BH,G, ad let U, N be a sequece P α G such that N U,+1 U,. Furthermore, suppose that z ra f + The prmal problem s mmze x H fx+ ad the dual problem s L g l L r + h. 6.23 g l L x r +hx x z, 6.24 mmze f h z v 1 G 1,...,v m G m L v + g v +l v + v r. 6.25 26
Let β = m{µ,ν 1,...,ν m }, let ε ]0,m{1,β}[, let λ N be a sequece [ε,1], suppose that 6.6 holds, ad set x U m L v, + hx +c z +a N p = prox U f y = 2p x x +1 = x +λ p x For = 1,...,m q, = prox U, g v,+1 = v, +λ q, v,. v, +U, L y l v, d, r +b, 6.26 The x N coverges weakly to a soluto to 6.24, for every {1,...,m} v, N coverges weakly to some v G, ad v 1,...,v m s a soluto to 6.25. Proof. Set A = f, C = h, ad {1,...,m} B = g ad D = l. I ths settg, t follows from the aalyss of [16, Secto 4] that 6.24 6.25 s a specal case of Problem 6.1 ad, usg 3.10, that 6.26 s a specal case of 6.7. Thus, the clams follow from Corollary 6.2&. Remark 6.5 Supposethat, Corollary6.2adExample6.4, thereexstτ adσ 1 m ]0,+ [ such that N U = τ Id ad {1,...,m} U, = σ Id. The 6.7 ad 6.26 reduce to the fxed metrc methods appearg [41, Eq. 3.3] ad [41, Eq. 4.5], respectvely see [41] for further coectos wth exstg work. Refereces [1] H. Attouch, L. M. Brceño-Aras, ad P. L. Combettes, A parallel splttg method for coupled mootoe clusos, SIAM J. Cotrol Optm., vol. 48, pp. 3246 3270, 2010. [2] H. Attouch ad M. Théra, A geeral dualty prcple for the sum of two operators, J. Covex Aal., vol. 3, pp. 1 24, 1996. [3] H. H. Bauschke, J. M. Borwe, ad P. L. Combettes, Bregma mootoe optmzato algorthms, SIAM J. Cotrol Optm., vol. 42, pp. 596 636, 2003. [4] H. H. Bauschke ad P. L. Combettes, Covex Aalyss ad Mootoe Operator Theory Hlbert Spaces. Sprger, New York, 2011. [5] J. F. Boas, J. Ch. Glbert, C. Lemaréchal, ad C. A. Sagastzábal, A famly of varable metrc proxmal methods, Math. Programmg, vol. 68, pp. 15 47, 1995. [6] R. I. Boţ, Cojugate Dualty Covex Optmzato, Lecture Notes Ecoomcs ad Mathematcal Systems, vol. 637. Sprger, New York, 2010. [7] L. M. Brceño-Aras ad P. L. Combettes, Mootoe operator methods for Nash equlbra opotetal games, Computatoal ad Aalytcal Mathematcs, D. Baley, H. H. Bauschke, P. Borwe, F. Garva, M. Théra, J. Vaderwerff, ad H. Wolkowcz, eds.. Sprger, New York, 2013. [8] J. V. Burke ad M. Qa, A varable metrc proxmal pot algorthm for mootoe operators, SIAM J. Cotrol Optm., vol. 37, pp. 353 375, 1999. [9] J. V. Burke ad M. Qa, O the superlear covergece of the varable metrc proxmal pot algorthm usg Broyde ad BFGS matrx secat updatg, Math. Programmg, vol. 88, pp. 157 181, 2000. 27
[10] G. H.-G. Che ad R. T. Rockafellar, Covergece rates forward-backward splttg, SIAM J. Optm., vol. 7, pp. 421 444, 1997. [11] X. Che ad M. Fukushma, Proxmal quas-newto methods for odfferetable covex optmzato, Math. Program., vol. 85, pp. 313 334, 1999. [12] P. L. Combettes, Solvg mootoe clusos va compostos of oexpasve averaged operators, Optmzato, vol. 53, pp. 475 504, 2004. [13] P. L. Combettes, D- h Dũg, ad B. C. Vũ, Dualzato of sgal recovery problems, Set-Valued Var. Aal., vol. 18, pp. 373 404, 2010. [14] P. L. Combettes, D- h Dũg, ad B. C. Vũ, Proxmty for sums of composte fuctos, J. Math. Aal. Appl., vol. 380, pp. 680 688, 2011. [15] P. L. Combettes ad J.-C. Pesquet, Proxmal splttg methods sgal processg, Fxed-Pot Algorthms for Iverse Problems Scece ad Egeerg, H. H. Bauschke et al., eds, pp. 185 212. Sprger, New York, 2011. [16] P. L. Combettes ad J.-C. Pesquet, Prmal-dual splttg algorthm for solvg clusos wth mxtures of composte, Lpschtza, ad parallel-sum type mootoe operators, Set-Valued Var. Aal., vol. 20, pp. 307 330, 2012. [17] P. L. Combettes ad B. C. Vũ, Varable metrc quas-fejér mootocty, Nolear Aal., to appear. http://arxv.org/abs/1206.5705 [18] P. L. Combettes ad V. R. Wajs, Sgal recovery by proxmal forward-backward splttg, Multscale Model. Smul., vol. 4, pp. 1168 1200, 2005. [19] W. C. Davdo, Varable metrc method for mmzato, Argoe Natoal Laboratory research ad developmet report 5990, 1959; reprted SIAM J. Optm., vol. 1, pp. 1 17, 1991. [20] E. De Vto, V. Umatà, ad S. Vlla, A cosstet algorthm to solve Lasso, elastc-et ad Tkhoov regularzato, J. Complexty, vol. 27, pp. 188 200, 2011. [21] J. Duch ad Y. Sger, Effcet ole ad batch learg usg forward backward splttg, J. Mach. Lear. Res., vol. 10, pp. 2899 2934, 2009. [22] J. Eckste ad M. C. Ferrs, Smooth methods of multplers for complemetarty problems, Math. Programmg, vol. 86, pp. 65 90, 1999. [23] F. Facche ad J.-S. Pag, Fte-Dmesoal Varatoal Iequaltes ad Complemetarty Problems. Sprger-Verlag, New York, 2003. [24] R. Glowsk ad P. Le Tallec, Augmeted Lagraga ad Operator-Splttg Methods Nolear Mechacs. SIAM, Phladelpha, 1989. [25] J.-B. Hrart-Urruty ad C. Lemaréchal, Covex Aalyss ad Mmzato Algorthms. Sprger-Verlag, New York, 1993. [26] T. Kato, Perturbato Theory for Lear Operators, 2d ed. Sprger-Verlag, New York, 1980. [27] C. Lemaréchal ad C. Sagastzábal, Varable metrc budle methods: from coceptual to mplemetable forms, Math. Programmg, vol. 76, pp. 393 410, 1997. [28] P. A. Lotto, L. A. Parete, ad M. V. Solodov, A class of varable metrc decomposto methods for mootoe varatoal clusos, J. Covex Aal., vol. 16, pp. 857 880, 2009. [29] B. Mercer, Topcs Fte Elemet Soluto of Ellptc Problems Lectures o Mathematcs, o. 63. Tata Isttute of Fudametal Research, Bombay, 1979. 28
[30] B. Mercer, Iéquatos Varatoelles de la Mécaque Publcatos Mathématques d Orsay, o. 80.01. Uversté de Pars-XI, Orsay, Frace, 1980. [31] U. Mosco, Dual varatoal equaltes, J. Math. Aal. Appl., vol. 40, pp. 202 206, 1972. [32] L. A. Parete, P. A. Lotto, ad M. V. Solodov, A class of exact varable metrc proxmal pot algorthms, SIAM J. Optm., vol. 19, pp. 240 260, 2008. [33] J. D. Pearso, Varable metrc methods of mmsato, Comput. J., vol. 12, pp. 171 178, 1969. [34] T. Peae, Dualzato of geeralzed equatos of maxmal mootoe type, SIAM J. Optm., vol. 10, pp. 809 835, 2000. [35] L. Q ad X. Che, A precodtog proxmal Newto method for odfferetable covex optmzato, Math. Programmg, vol. 76, pp. 411 429, 1997. [36] M. Qa, The Varable Metrc Proxmal Pot Algorthm: Theory ad Applcato, Ph.D. thess, Uversty of Washgto, Seattle, WA, 1992. [37] S. M. Robso, Composto dualty ad maxmal mootocty, Math. Programmg, vol. 85, pp. 1 13, 1999. [38] R. T. Rockafellar, Dualty ad stablty extremum problems volvg covex fuctos, Pacfc J. Math., vol. 21, pp. 167 187, 1967. [39] P. Tseg, Further applcatos of a splttg algorthm to decomposto varatoal equaltes ad covex programmg, Math. Programmg, vol. 48, pp. 249 263, 1990. [40] P. Tseg, Applcatos of a splttg algorthm to decomposto covex programmg ad varatoal equaltes, SIAM J. Cotrol Optm., vol. 29, pp. 119 138, 1991. [41] B. C. Vũ, A splttg algorthm for dual mootoe clusos volvg cocoercve operators, Adv. Comput. Math., publshed o-le 2011. http://www.sprgerlk.com/cotet/m177247u22644173/ [42] D. L. Zhu ad P. Marcotte, Co-coercvty ad ts role the covergece of teratve schemes for solvg varatoal equaltes, SIAM J. Optm., vol. 6, pp. 714 726, 1996. 29