Multi parameter proximal poit algorithms Ogaeditse A. Boikayo a,b,, Gheorghe Moroşau a a Departmet of Mathematics ad its Applicatios Cetral Europea Uiversity Nador u. 9, H-1051 Budapest, Hugary b Departmet of Mathematics, Uiversity of Botswaa Pvt 00704, Gaboroe, Botswaa Abstract The aim of this paper is to prove a strog covergece result for a algorithm itroduced by Y. Yao ad M. A. Noor i 2008 uder a ew coditio o oe of the parameters ivolved. Further, covergece properties of a geeralized proximal poit algorithm which was itroduced i [5] are aalyzed. The results i this paper are proved uder the geeral coditio that errors ted to zero i orm. These results exted ad improve several previous results o the regularizatio method ad the proximal poit algorithm. Keywords: Firmly oexpasive operator, maximal mootoe operator, oexpasive map, proximal poit algorithm, resolvet operator 2000 MSC: 47H05, 47J25, 47H09 1. Itroductio Let H be a real Hilbert space with ier product, ad iduced orm. Recall that a map T : H H is called oexpasive if for every x, y H we have T x T y x y. I the case whe T x T y a x y holds for some a 0, 1), the T is called a cotractio with Lipschitz costat a. The map T is called firmly oexpasive if for ay x, y H, T x T y 2 T x T y, x y. Obviously, firmly oexpasive mappigs are oexpasive. A operator A : DA) H 2 H is said to be mootoe if x x, y y 0, x, y), x, y ) GA). That is, its graph GA) = {x, y) H H : x DA), y Ax} is a mootoe subset of H H. A operator A is called maximal mootoe if i additio to beig mootoe, its graph is ot properly cotaied i the graph of ay other mootoe operator. I oliear aalysis ad covex optimizatio, a importat ad iterestig problem is to Correspodig author Email addresses: boikayoa@gmail.com Ogaeditse A. Boikayo), morosaug@ceu.hu Gheorghe Moroşau) 1 Dedicated to Professor Viorel Barbu o the occasio of his 70th birthday Preprit submitted to J. Noliear ad Covex Aalysis February 29, 2012
fid zeroes of maximal mootoe operators. From the view poit of fixed poit theory, the problem fid a x DA) such that 0 Ax), 1) is equivalet to fid fixed poit of the firmly oexpasive mappig I + λa) 1 : H H, the so called resolvet of A for all λ > 0. Ideed, may problems that ivolve covexity ca be formulated as fidig zeroes of maximal mootoe operators. Such problems iclude, but are ot limited to covex miimizatio, variatioal iequalities ad cocavecovex mii-max problems. Oe of the most popular ad powerful solutio techiques for solvig oliear problems is the so called proximal poit algorithm which was iitially suggested by Martiet [13] ad later extesively developed by Rockafellar [16]. The mai idea of this method is to replace the origial problem 1) by a sequece of regularized problems fid a x DA) such that 0 Ax) + β 1 x x ), 2) so that at each step, problem 2) has a uique solutio x := x +1. Here β ) is a sequece of positive real umbers ad x 0 H is a give startig poit. Accordigly, x +1 solves problem 2) if ad oly if x +1 = J β x, where J β = I + β A) 1. I geeral, equatio 2) is solved oly approximately, i which case, x +1 is the iexact solutio of problem 2), i.e., x +1 = J β x + e, = 0, 1..., 3) where e ) is cosidered to be the sequece of computatioal errors. Rockafellar [16] proved weak covergece of algorithm 3 to a elemet i the fixed poit set F J c ) = {x H J c x = x} = A 1 0), for all c > 0, provided that this set is ot empty, with the coditios lim if β > 0 ad E1) e <. =0 Rockafellar also proved strog covergece if i additio, the operator A 1 is Lipschitz cotiuous at zero with modulus a 0), that is, A 1 0) = {y}, ad for some τ > 0, z y a z wheever z, z ) GA) ad z τ. Other relevat refereces cocerig ifiite products of resolvets with errors are [6, 14]. Güler s example [9] see also [1]) which revealed that the PPA fails i geeral to coverge strogly, gave rise to the atural questios): how ca the PPA be modified so as to obtai strog covergece? Or is it possible to costruct/desig strogly coverget proximal poit algorithms? There are several attempts made so far i order to address the above questios). Oe such attempt was made i [17] followed, for example by [11, 15]. Aother effort was made idepedetly by Xu [20], ad Kamimura ad Takahashi [10]. They proposed the followig iexact PPA which is simpler tha the oe obtaied by Solodov ad Svaiter [17] x +1 = u + 1 )J β x + e, for ay u, x 0 H ad all 0, 4) 2
where ) 0, 1) ad β ) 0, ). I fact, their versio of the PPA correspods to the case whe u = x 0, the iitial startig poit of the trajectory geerated by 4). They proved a strog covergece result whe e ) satisfies E1). The case whe e ) is ot summable ad u ot ecessarily the startig poit of the PPA was treated i [2]. A strog covergece result was obtaied i [3] by further weakeig the error coditio i [2]. It should however be metioed that ot as i [2], such a coditio depeds o the sequece of parameters ). The precise coditio o the sequece of error terms e ) was E2) lim e = 0. Aother algorithm of iterest which geerates strogly coverget sequeces is the regularizatio method which was itroduced by Lehdili ad Moudafi [12], ad exteded by Xu [21]. Give x 0, u H, this method accordig to Xu geerates a sequece x ) iteratively by x +1 = J β u + 1 )x + e ), for all 0. 5) The authors have observed i [3] see also [4]) that there is a strog coectio betwee the proximal poit algorithm 4) ad the regularizatio method 5). More precisely, they oted that takig v = x 1 u e 1 1 1, equatio 4) reduces to v +1 = J β 1 u + 1 1 )v + e 1 ), for all 1, 6) ad for 0 ad e 0, x ) defied by 4) coverges if ad oly if v ) does. Thus 4) ad 6) are equivalet. The regularizatio method was further exteded [5] to v +1 = J β 1 u + λ 1 v + γ 1 T v + e 1 ), 1, 7) where T : H H is a oexpasive map, 0, 1), λ, γ [0, 1] with +λ +γ = 1, ad β 0, ). Uder appropriate coditios o the cotrol parameters, λ, γ ad β, it was show [5] that v ) geerated by 7) coverges strogly to P A 1 0)u, provided that A 1 0) F T ), where F T ) = {x H T x = x} is the fixed poit set of T. Recetly, Yao ad Noor [22] proposed a algorithm which is defied by x +1 = u + λ x + γ J β x + e, 0, 8) where agai u, x 0 H are give, 0, 1), λ, γ [0, 1] with + λ + γ = 1, ad β 0, ). They showed that x ) is strogly coverget to P A 1 0)u, provided that A 1 0), 0 < lim if λ lim sup λ < 1, lim if β > 0, C1) lim β +1 β ) = 0, lim = 0 ad =0 =. We ote that for ubouded β ), coditio C1) fails to satisfy the atural choice β = for all N. Yao ad Noor s result brigs us 3
to the followig questio: Does [22, Theorem 3.3] remai true if λ ) is assumed to be bouded from above away from 1 ad/or β ) satisfies weaker coditios which iclude choices such as β = for all N? The purpose of this paper is to address the above questio ad to discuss the strog covergece of sequeces geerated by algorithm 7). Our mai result is Theorem 2 which is cocered with the followig coditios: lim if β > 0 ad 2. Prelimiaries C2) lim β +1 β = 1. Let T : H H be a oexpasive map, ad let A : DA) H 2 H be a maximal mootoe operator. Fix N, ad defie a map f : H H by the rule x J β u + λ x + γ T x + e ), where β > 0, ), λ ) ad γ ) are sequeces i 0, 1) such that + λ + γ = 1, ad u, e H are give. The oe ca easily check that f is a cotractio. Therefore it follows from the Baach cotractio priciple that f has a uique fixed poit z, say. I other words, z = J β u + λ z + γ T z + e ), 0. 9) We prove the covergece result associated with the sequece z ). Lemma 1. Let β 0, ) ad let, λ, γ 0, 1) with lim = 0 ad + λ + γ = 1 for all N. Assume that A 1 0) =: F F T ), where T : H H is a oexpasive map, ad either =0 e < or e / 0. The for ay fixed u, x 0 H, the sequece z ) geerated by 9) coverges strogly to P F u, the projectio of u o F. Proof. To show that z ) is bouded, we first ote that if e / ) is bouded, the there exists a positive costat C such that sup u p + e ) C. N For every p F, we have from 9) z p u p + e / ) + λ z p) + γ T z p) u p + e ) + λ z p + γ T z p 1 ) z p + C, where the first two iequalities follow from the fact that J β Therefore z ) is bouded. ad T are oexpasive. Let ω w z )) be the weak ω limit set of z ). That is, ω w z )) = {y H z k y for some subsequece z k ) of z )}. 4
We claim that ω w z )) F. Let z j ) be a subsequece of z ) covergig weakly to some z. Sice λ j ) is bouded, it has a coverget subsequece, agai deoted λ j ). There are two possibilities here: either λ j 1 or λ j θ [0, 1). I the first case, we derive from Az j j u + λ j 1)z j + γ j T z j + e j β j 0, as j, 10) that z F. I the secod case, we ote that from 9), we have 1 λ ) z T z, z p + β Az, z p = u T z + e /, z p, where p F. Usig the mootoicity of A, we have for some M > 0 M 21 λ ) z T z, z p = 1 λ ) z T z 2 + z p 2 T z T p 2 ) 1 λ ) z T z 2. Passig to the limit i the above estimate, with = j, we get lim z j T z j = 0. j Agai from 10), we get z F, showig that ω w z )) F. Therefore, there exists a subsequece z k ) of z ) covergig weakly to z F such that lim sup u P F u, z P F u = lim k u P F u, z k P F u = u P F u, z P F u 0. O the other had, z P F u 2 α 2 u P F u + e ) 2 + λ z P F u + γ T z P F u ) 2 α + 2 u P F u + e, λ z P F u) + γ T z P F u) 1 ) 2 z P F u 2 + α 2 u P F u + e ) 2 α + 2 u P F u + e, 1 )z P F u) + γ T z z ), where the secod iequality follows from the oexpasivity of T. Hece for some positive costat C, 2 ) z P F u 2 C + 2 u P F u + e, z P F u) + γ T z z ). Passig to the limit i the above iequality, we deduce the strog covergece of z ) to P F u as claimed. We leave it to the reader to verify the result i the case whe e ) l 1. We ote that Lemma 1 above cotais [4, Theorem 1] as a special case. We ext recall some lemmas which will be used i the sequel. 5
Lemma 2 Suzuki [18]). Let x ) ad y ) be bouded sequeces i a real Baach space ad let ρ ) be a sequece i 0, 1), with 0 < lim if ρ lim sup ρ < 1. Suppose that x +1 = ρ y +1 ρ )x for all itegers 0 ad lim sup y +1 y x +1 x ) 0. The lim y x = 0. Lemma 3 Goebel ad Kirk [7], Goebel ad Reich [8]). A map T : H H is firmly oexpasive if ad oly if 2T I where I is the idetity map) is oexpasive. Lemma 4 Xu [20]). Let s ) be a sequece of o-egative real umbers satisfyig s +1 1 a )s + a b + c, 0, where a ), b ) ad c ) satisfy the coditios: i) a ) [0, 1], with =0 a =, ii) c 0 for all 0 with =0 c <, ad iii) lim sup b 0. The lim s = 0. Cocerig the boudedess of the sequeces v ) ad x ) defied by 7) ad 8), we have the followig two lemmas, respectively. The proof of Lemma 6 is cotaied i the proof of [5, Theorem 5]. Lemma 5 cf. [5]). Let β 0, ), 0, 1) ad λ, γ [0, 1] with +λ +γ = 1 for all. Assume that A : DA) H 2 H is a maximal mootoe operator with F := A 1 0), ad either =0 e < or e / ) is bouded. The for ay fixed u, x 0 H, the sequece x ) defied by 8) is bouded. Lemma 6 cf. [5]). If i additio to the assumptios of Lemma 5, =1 = ad A 1 0) F T ), where T : H H is a oexpasive map, the for ay fixed u, v 1 H, the sequece v ) defied by 7) is bouded. We will eed the followig idetity, the proof of which is well kow ad ca easily be reproduced, see, e.g., [4]. Lemma 7 Resolvet Idetity). For ay β, γ > 0 ad x H, we have γ J β x = J γ β x + 1 γ ) ) J β x. β We coclude this sectio with a lemma kow i the literature as the subdifferetial iequality. Its proof is immediate. Lemma 8. For all x, y H, we have x + y 2 y 2 + 2 x, x + y. 6
3. Mai results Theorem 1. Assume that A : DA) H 2 H is a maximal mootoe operator ad A 1 0) =: F F T ), where T : H H is a oexpasive map. Fix u, v 1 H, ad let v ) be the sequece geerated by algorithm 7) uder the coditios: i) 0, 1), λ, γ [0, 1], + λ + γ = 1 with lim = 0 ad =0 =, ii) either E1) or E2), iii) β 0, ) with lim if β > 0, 1 C3) lim 1 1 1β +1 β The v ) coverges strogly to P F u. ) 1 = 0 ad C4) lim 1 Proof. Accordig to Lemma 6, the sequece v ) is bouded. Deote γ γ ) 1β +1 = 0. β w = J β 1 u + λ 1 w + γ 1 T w ), 11) the from Lemma 1 we have that w P F u. Now it follows from 7) that v +1 w +1 v +1 w + w w +1 λ 1 v w + γ 1 T v T w + e 1 + w w +1 1 1 ) v w + e 1 + w w +1, 12) where the last iequality follows from the oexpasivity of the map T. Usig the resolvet idetity, we ote that 11) ca be writte as ε w = J ε 1 u + λ 1 w + γ 1 T w ) + 1 ε ) ) w, β β where ε > 0 is the greatest lower boud of β ). This together with the fact that the resolvet operator J ε is oexpasive gives w +1 w 1 ε ) w +1 w + ελ w +1 w + εγ T w +1 T w β +1 β +1 β +1 + ε ε 1 β +1 β u w + εγ εγ 1 β +1 β T w w 1 εα ) w +1 w + ε β +1 ε 1 β +1 β K + εγ εγ 1 β +1 M, β for some positive costats K ad M. This last estimate reduces to w +1 w 1 α 1β +1 β K + γ γ 1β +1 β M. 13) 7
Usig this last iequality i 12) we arrive at v +1 w +1 1 1 ) v w + e 1 + 1 1β +1 β K + 1 γ γ 1β +1 β M. Therefore from Lemma 4, we get that v w 0, which i tur implies that v P F u. Example 1. Clearly, the sequeces ), β ) ad γ ) defied by = 1/ + 1, β = 1 + 1 ad γ = 1/ + 1) for 2 satisfy the Coditios C3) ad C4). Remark 1. A result similar to the above theorem was proved i [4] for T = I, the idetity operator, ad uder the additioal assumptio β +1 β. Therefore, Theorem 1 is a geeralizatio ad improvemet of [4, Theorem 4]. Note that [21, Theorem 3.2] which is similar i its ature to [4, Theorem 4] ca also be geeralized i the same way. We coclude this sectio by provig a strog covergece result associated with the ewly itroduced coditio C2). The ext result is a refiemet of [19, Theorem 4]. It also exteds [19, Theorem 4] to geeral errors. Although C2) is stroger tha the coditio 1 lim 1 ) = 0 14) β +1 β i [5, Theorem 1] ad [5, Theorem 2]), the mai advace i Theorem 2 below is that strog covergece of the sequece x ) is proved uder weaker coditios o both ) ad λ ) tha those of [5, Theorem 1] ad [5, Theorem 2]). For the compariso of coditios C2) ad 14), see Remark 3 below. Theorem 2. Assume that A : DA) H 2 H is a maximal mootoe operator ad F := A 1 0). Fix u, x 0 H, ad let x ) be the sequece geerated by algorithm 8) uder the coditios: i) 0, 1) with lim = 0 ad =0 =, ii) either E1) or E2), iii) λ, γ [0, 1], + λ + γ = 1 with lim if γ > 0 ad iv) β 0, ) with lim if β > 0 ad C2). The x ) coverges strogly to P F u. Proof. From Lemma 5, we kow that x ) is bouded. Deote y := T x + µ u x ) + σ, where T = 2J β I, µ = 2 /γ ad σ = 2e /γ. Obviously, the sequece y ) is bouded sice x ) is so), ad from the defiitio of T, 8) ca be writte as x +1 = u + λ x + γ 2 x + γ 2 T x + e = 1 γ ) x + γ T x + 2 u x ) + 2e ) 2 2 γ γ = 1 γ ) x + γ 2 2 y. 8
Sice T is oexpasive, we have for some positive costat M y +1 y T +1 x +1 T x + µ +1 u x +1 + µ u x + σ +1 σ T +1 x +1 T +1 x + T +1 x T x + µ +1 + µ )M + σ +1 σ x +1 x + 2 Jβ+1 x J β x + µ+1 + µ )M + σ +1 σ = x +1 x + 2 J β+1 β +1 x J β+1 x + 1 β ) ) +1 J β x β β + µ +1 + µ )M + σ +1 σ x +1 x + 2 1 β +1 x J β x β + µ +1 + µ )M + σ +1 σ, 15) where the equality follows from the applicatio of the resolvet idetity. Rearragig terms i 15) ad passig to the limit as, we get lim sup Therefore applyig Lemma 2 we get which implies that { y +1 y x +1 x } 0. lim x y = 0, lim x +1 x = 0. 16) Note that sice lim if γ > 0, there exists δ [0, 1) such that λ δ for all N. The from 8), we have x +1 J β x u J β x + e / + λ x J β x which implies that u J β x + e / + δ x x +1 + x +1 J β x ), lim x +1 J β x = 0. 17) O the other had, we observe that if β > 0 is the greatest lower boud of β ), the the applicatio of the resolvet idetity yields ) J β x J β x 1 ββ J β x x ) J β x x +1 + x +1 x. Passig to the limit as i the above iequality, ad oticig 16) ad 17), we get lim J β x J β x = 0. 18) 9
Moreover, from 16), 17) ad 18), we have lim sup x J β x lim sup x x +1 + x +1 J β x + J β x J β x ) = 0. 19) Now let x k ) be a subsequece of x ) covergig weakly to some z. The for some positive costat K, 2 x k J β z, z J β z = x k J β z 2 + z J β z 2 x k z 2 x k J β x k + x k z ) 2 + z J β z 2 x k z 2 K x k J β x k + z J β z 2. Passig to the limit i the above iequality as k, ad oticig 19), we arrive at z A 1 0). Hece for a subsequece x j ) of x ) covergig weakly to a poit x, we have lim sup u P F u, x P F u = lim j u P F u, x j P F u = u P F u, x P F u 0. Fially, from Lemma 8 ad equatio 8), we have x +1 P F u λ x P F u + γ J β x P F u ) 2 + 2 u P F u + e, x +1 P F u 1 ) x P F u 2 + 2 u P F u + e, x +1 P F u Therefore, from Lemma 4 we get strog covergece of x ) to P F u. I the case whe the error sequece e ) satisfies coditio E1), the we get from Lemma 8 ad equatio 8) x +1 P F u 1 ) x P F u 2 + 2 u P F u, x +1 P F u + e C, for some C > 0. As before, strog covergece of x ) to P F u ca be derived. Remark 2. Not as i [5, Theorem 4] ad [22, Theorem 3.3], we do ot require that γ be bouded above away from 1. I additio, we have used the weaker Coditio C2) istead of C1). Therefore, Theorem 2 improves sigificatly the results i [5, 22]. Remark 3. Note that for the sequece β ) satisfyig β ε for some ε > 0 ad all N, 1 1 β +1 β = 1 β +1 1 β +1 β 1 ε 1 β +1 β implyig that the Coditio 14) is weaker tha the Coditio C2) of the precedig theorem. Ideed, oe ca check that the sequece { 2 if is odd, β = 3 if is eve satisfies 14) but ot C2). These two coditios are however equivalet if β ) is bouded both from below away from zero ad from above). Ackowledgmet: May thaks are due to the referee for the useful commets ad for idicatig additioal refereces related to our paper. 10.
Refereces [1] H. H. Bauschke, E. Matoušková ad S. Reich, Projectio ad proximal poit methods: covergece results ad couterexamples, Noliear Aal. 56 2004), 715-738. [2] O. A. Boikayo ad G. Moroşau, Modified Rockafellar s algorithms, Math. Sci. Res. J. 5) 13 2009), 101-122. [3], A proximal poit algorithm covergig strogly for geeral errors, Optim. Lett. 4 2010), 635-641. [4], Iexact Halper-type proximal poit algorithms, J. Glob. Optim., i press. [5], Four parameter proximal poit algorithms, Noliear Aal. 74 2011), 544-555. [6] R. E. Bruck ad S. Reich, Noexpasive projectios ad resolvets of accretive operators i Baach spaces, Housto J. Math. 3 1977), 459-470. [7] K. Goebel ad W. A. Kirk, Topics i Metric Fixed Poit Theory, Cambridge Uiversity Press, Cambridge, 1990). [8] K. Goebel ad S. Reich, Uiform Covexity, Hyperbolic Geometry, ad Noexpasive Mappigs, Marcel Dekker, New York, 1984. [9] O. Güler, O the covergece of the proximal poit algorithm for covex miimizatio, SIAM J. Cotrol Optim. 29 1991), 403-419. [10] S. Kamimura ad W. Takahashi, Approximatig solutios of maximal mootoe operators i Hilbert spaces, J. Approx. Theory 106 2000), 226-240. [11], Strog covergece of a proximal-type algorithm i a Baach space, SIAM J. Optim. 13 2002), 938-945. [12] N. Lehdili ad A. Moudafi, Combiig the proximal algorithm ad Tikhoov regularizatio, Optimizatio 37 1996), 239-252. [13] B. Martiet, Régularisatio d iéquatios variatioelles par approximatios successives, Rev. Fraçaise Iformat. Recherche Opératioelle 4 1970), Ser. R-3, 154 158. [14] O. Nevalia ad S. Reich, Strog covergece of cotractio semigroups ad of iterative methods for accretive operators i Baach spaces, Israel J. Math. 32 1979), 44-58. [15] S. Reich ad Sabach, A strog covergece theorem for a proximal-type algorithm i reflexive Baach spaces, J. Noliear Covex Aal. 10 2009), 471-485. [16] R. T. Rockafellar, Mootoe operators ad the proximal poit algorithm, SIAM J. Cotrol Optim. 14 1976), 877-898. 11
[17] M. V. Solodov ad B. F. Svaiter, Forcig strog covergece of proximal poit iteratios i a Hilbert space, Math. Program. Ser. A 14 2000), 189-202. [18] T. Suzuki, Strog covergece theorems for ifiite families of oexpasive mappigs i geeral Baach spaces, Fixed Poit Theory Appl. 2005), 103-123. [19] F. Wag, A ote o the regularized proximal poit algorithm, J. Glob. Optim., i press. [20] H. K. Xu, Iterative algorithms for oliear operators, J. Lodo Math. Soc. 2) 66 2002), 240-256. [21], A regularizatio method for the proximal poit algorithm, J. Glob. Optim. 36 2006), 115-125. [22] Y. Yao ad M. A. Noor, O covergece criteria of geeralized proximal poit algorithms, J. Comp. Appl. Math. 217 2008), 46-55. 12