Two Strong Convergence Theorems for a Proximal Method in Reflexive Banach Spaces

Transcription

1 Two Strong Convergence Theorems for a Proxmal Method n Reflexve Banach Spaces Smeon Rech and Shoham Sabach Abstract. Two strong convergence theorems for a proxmal method for fndng common zeroes of maxmal monotone operators n reflexve Banach spaces are establshed. Both theorems take nto account possble computatonal errors. 1. Introducton In ths paper X denotes a real reflexve Banach space wth norm and X stands for the topologcal dual of X endowed wth the nduced norm. We denote the value of the functonal ξ X at x X by ξ, x. An operator A : X 2 X s sad to be monotone f for any x, y dom A, we have ξ Ax and η Ay = ξ η, x y 0. Recall that the set dom A = {x X : Ax } s called the effectve doman of such an operator A. A monotone operator A s sad to be maxmal f graph A, the graph of A, s not a proper subset of the graph of any other monotone operator. In ths paper f : X, + ] s always a proper, lower semcontnuous and convex functon, and f : X, + ] s the Fenchel conjugate of f. The set of nonnegatve ntegers wll be denoted by N. The problem of fndng an element x X such that 0 Ax s very mportant n Optmzaton Theory and related felds. For example, f A s the subdfferental f of f, then A s a maxmal monotone operator and the equaton 0 f x s equvalent to the problem of mnmzng f over X. One of the methods for solvng ths problem n Hlbert space s the well-known proxmal pont algorthm. Let H be a Hlbert space and let I denote the dentty operator on H. The proxmal pont algorthm generates, for any startng pont x 0 = x H, a sequence {x n } n N n H 2000 Mathematcs Subject Classfcaton. 47H05, 47J25. Key words and phrases. Banach space, Bregman projecton, Legendre functon, monotone operator, proxmal pont algorthm, resolvent, totally convex functon. 1

2 2 SIMEON REICH AND SHOHAM SABACH by the rule 1.1 x n+1 = I + λ n A 1 x n, n = 0, 1, 2,..., where {λ n } n N s a gven sequence of postve real numbers. equvalent to Note that 1.1 s 0 Ax n λ n x n+1 x n, n = 0, 1, 2,.... Ths algorthm was frst ntroduced by Martnet [28] and further developed by Rockafellar [38], who proves that the sequence generated by 1.1 converges weakly to an element of A 1 0 when A 1 0 s nonempty and lm nf n + λ n > 0. Furthermore, Rockafellar [38] asks f the sequence generated by 1.1 converges strongly. Ths queston was answered n the negatve by Güler [24], who presented an example of a subdfferental for whch the sequence generated by 1.1 converges weakly but not strongly; see [7] for a more recent and smpler example. Qute a few results regardng the proxmal pont algorthm and ts extensons can be found n the lterature. See, for example, [5, 6, 7, 10, 11, 15, 16, 19, 21, 22, 25, 26, 29, 30, 32, 33, 35, 39, 41, 43]. We menton, n partcular, the semnal papers [41, 21, 5, 6]. These papers ntroduce a new paradgm whch has snce led to many modfcatons. One such modfcaton has been proposed by Bauschke and Combettes [5] see also Solodov and Svater [41], who have modfed the proxmal pont algorthm n order to generate a strongly convergent sequence. They ntroduce, for example, the followng algorthm see [5, Corollary 6.1, p. 258] for a sngle operator and λ n = 1/2: x 0 H, 1.2 y n = R λna x n, C n = {z H : y n z x n z }, Q n = {z H : x 0 x n, z x n 0}, x n+1 = P Cn Q n x 0, n = 0, 1, 2,.... Here, for each x H and each nonempty, closed and convex subset C of H, the mappng P C s defned by x P C x = nf { x z : z C}. Ths mappng s called the metrc projecton of H onto C. The mappng R λa = I + λa 1 s the classcal resolvent of the maxmal monotone operator A. They prove that f A 1 0 s nonempty and lm nf n + λ n > 0, then the sequence generated by 1.2 converges strongly to P A 1 0. We and Zhou [42] generalze ths result to those Banach spaces X whch are both unformly convex and unformly smooth. They

3 STRONG CONVERGENCE THEOREMS 3 ntroduce the followng algorthm: x 0 X, 1.3 y n = J λn x n, C n = {z X : φ z, y n φ z, x n }, Q n = {z X : Jx 0 Jx n, z x n 0}, x n+1 = Q Cn Q n x 0, n = 0, 1, 2,..., where J s the normalzed dualty mappng of the space X, J λ x = J + λa 1 J and φ y, x = y 2 2 Jx, y + x 2. Here, for each nonempty, closed and convex subset C of X, Q C s a certan generalzaton of the metrc projecton P C n H. They prove that f A 1 0 s nonempty and lm nf n + λ n > 0, then the sequence generated by 1.3 converges strongly to Q A 1 0. In the present paper we extend Algorthms 1.2 and 1.3 to general reflexve Banach spaces usng a well chosen convex functon f. More precsely, we ntroduce the followng algorthm: x 0 X, 1.4 y n = Res f λ na x n, C n = {z X : D f z, y n D f z, x n }, Q n = {z X : f x 0 f x n, z x n 0}, x n+1 = proj f C n Q n x 0, n = 0, 1, 2,..., where {λ n } n N s a gven sequence of postve real numbers, Res f A s the resolvent of A relatve to f, ntroduced and studed n [4], f s the gradent of f and proj f C s the Bregman projecton of X onto C nduced by f see Secton 2.4. Algorthm 1.4 s more flexble than 1.3 because t leaves us the freedom of fttng the functon f to the nature of the operator A especally when A s the subdfferental of some functon and of the space X n ways whch make the applcaton of 1.4 smpler than that of 1.3. It should be observed that f X s a Hlbert space H, then usng n 1.4 the functon f x = 1/2 x 2, one obtans exactly Algorthm 1.2. If X s not a Hlbert space, but stll a unformly convex and unformly smooth Banach space X, then settng f x = 1/2 x 2 n 1.4, one obtans exactly 1.3. We also note that the choce f x = 1/2 x 2 n some Banach spaces may make the computatons n Algorthm 1.3 qute dffcult. These computatons can be smplfed by an approprate choce of f. For nstance, f X = l p or X = L p wth p 1,, and f x = 1/p x p n 1.4, then the computatons become smpler than those requred n 1.3, whch corresponds to f x = 1/2 x 2. As a matter of fact, we propose two extensons of Algorthm 1.4 see Algorthms 4.1 and 4.4 whch approxmate a common zero of several maxmal monotone operators and whch allow computatonal errors. These algorthms are smlar to but dfferent

4 4 SIMEON REICH AND SHOHAM SABACH from the one we have recently studed n [34]. They also dffer from the algorthm n [6] n the defnton of the sets C n and n our takng nto account possble computatonal errors. Our man results Theorems 1 and 2 are formulated and proved n Secton 4. The next secton s devoted to several prelmnary defntons and results. In secton 3 we prove two auxlary results whch are used n the proofs of our man results n Secton 4. The behavor of Algorthm 1.4 when the operator A s zero free s analyzed n Secton 5 see Theorem 3. The sxth secton contans three corollares of Theorems 1, 2 and 3. In the seventh and last secton we present an applcaton of Theorems 1, 2 and Prelmnares 2.1. Some facts about Legendre functons. Legendre functons mappng a general Banach space X nto, + ] are defned n [3]. Accordng to [3, Theorems 5.4 and 5.6], snce X reflexve, the functon f s Legendre f and only f t satsfes the followng two condtons: L1 The nteror of the doman of f, nt dom f, s nonempty, f s Gâteaux dfferentable see below on nt dom f, and dom f = nt dom f; L2 The nteror of the doman of f, nt dom f, s nonempty, f s Gâteaux dfferentable on nt dom f, and dom f = nt dom f. Snce X s reflexve, we always have f 1 = f see [8, p. 83]. Ths fact, when combned wth condtons L1 and L2, mples the followng equaltes: f = f 1, ran f = dom f = nt dom f and ran f = dom f = nt dom f. Also, condtons L1 and L2, n conjuncton wth [3, Theorem 5.4], mply that the functons f and f are strctly convex on the nteror of ther respectve domans. Several nterestng examples of Legendre functons are presented n [2] and [3]. Among them are the functons 1 s s wth s 1,, where the Banach space X s smooth and strctly convex and, n partcular, a Hlbert space. The functon f s called cofnte f dom f = X A property of gradents. For any convex f : X, + ] we denote by dom f the set {x X : f x < + }. For any x dom f and y X, we

5 STRONG CONVERGENCE THEOREMS 5 denote by f x, y the rght-hand dervatve of f at x n the drecton y, that s, f fx + ty fx x, y := lm. t 0 t The functon f s sad to be Gâteaux dfferentable at x f lm t 0 fx + ty fx /t exsts for any y. The functon f s sad to be Fréchet dfferentable at x f ths lmt s attaned unformly n y = 1. Fnally, f s sad to be unformly Fréchet dfferentable on a subset E of X f the lmt s attaned unformly for x E and y = 1. We wll need the followng result. Proposton 1 cf. [34, Proposton 2]. If f : X R s unformly Fréchet dfferentable and bounded on bounded subsets of X, then f s unformly contnuous on bounded subsets of X from the strong topology of X to the strong topology of X Some facts about totally convex functons. Let f : X, + ] be convex. The functon D f : dom f nt dom f [0, + ], defned by 2.1 D f y, x := fy fx f x, y x, s called the Bregman dstance wth respect to f cf. [18]. If f s a Gâteaux dfferentable functon, then the Bregman dstance has the followng mportant property, called the three pont dentty: for any x, y, z nt dom f, 2.2 D f x, y + D f y, z D f x, z = fz fy, x y. Recall that, accordng to [13, Secton 1.2, p. 17] see also [12], the functon f s called totally convex at a pont x nt dom f f ts modulus of total convexty at x, that s, the functon υ f : nt dom f [0, + [0, + ], defned by 2.3 υ f x, t := nf {D f y, x : y dom f, y x = t}, s postve whenever t > 0. The functon f s called totally convex when t s totally convex at every pont x nt dom f. In addton, the functon f s called totally convex on bounded sets f υ f E, t s postve for any nonempty bounded subset E of X and for any t > 0, where the modulus of total convexty of the functon f on the set E s the functon υ f : nt dom f [0, + [0, + ] defned by υ f E, t := nf {υ f x, t x E dom f}. Examples of totally convex functons can be found, for example, n [13, 17]. The followng proposton summarzes some propertes of the modulus of total convexty. Proposton 2 cf. [13, Proposton 1.2.2, p. 18]. Let f be a proper, convex and lower semcontnuous functon. If x nt dom f, then The doman of υ f x, s an nterval of the form [0, τ f x or [0, τ f x] wth τ f x 0, + ].

6 6 SIMEON REICH AND SHOHAM SABACH If c [1, + and t 0, then υ f x, ct cυ f x, t. The functon υ f x, s superaddtve, that s, for any s, t [0, +, we have υ f x, s + t υ f x, s + υ f x, t. v The functon υ f x, s ncreasng; t s strctly ncreasng f and only f f s totally convex at x. The followng proposton follows from [15, Proposton 2.3, p. 39] and [44, Theorem , p. 164]. cofnte. results. Proposton 3. If f s Fréchet dfferentable and totally convex, then f s The next proposton turns out to be very useful n the proof of our man Proposton 4 cf. [36, Proposton 2.2, p. 3]. If x dom f, then the followng statements are equvalent: The functon f s totally convex at x; For any sequence {y n } n N dom f, lm D f y n, x = 0 lm y n x = 0. n + n + Recall that the functon f s called sequentally consstent see [17] f for any two sequences {x n } n N and {y n } n N n X such that the frst one s bounded, lm D f y n, x n = 0 lm y n x n = 0. n + n + Proposton 5 cf. [13, Lemma 2.1.2, p. 67]. If dom f contans at least two ponts, then the functon f s totally convex on bounded sets f and only f the functon f s sequentally consstent The resolvent of A relatve to f. Let A : X 2 X be an operator and assume that f Gâteaux dfferentable. The operator Prt f A := f + A 1 : X 2 X s called the protoresolvent of A, or, more precsely, the protoresolvent of A relatve to f. Ths allows us to defne the resolvent of A, or, more precsely, the resolvent of A relatve to f, ntroduced and studed n [4], as the operator Res f A : X 2X gven by Res f A := Prtf A f. Ths operator s sngle-valued when A s monotone and f s strctly convex on nt dom f. If A = ϕ, where ϕ s a proper, lower semcontnuous and convex functon, then we denote Prox f ϕ := Prt f ϕ and proxf ϕ := Res f ϕ.

7 STRONG CONVERGENCE THEOREMS 7 If C s a nonempty, closed and convex subset of X, then the ndcator functon ι C of C, that s, the functon ι C x := { 0 f x C + f x / C s proper, convex and lower semcontnuous, and therefore ι C exsts and s a maxmal monotone operator wth doman C. The operator prox f ι C s called the Bregman projecton onto C wth respect to f cf. [9] and we denote t by proj f C. Note that f X s a Hlbert space and fx = 1 2 x 2, then the Bregman projecton of x onto C,.e., argmn { y x : y C}, s the metrc projecton P C. Recall that the Bregman projecton of x onto the nonempty, closed and convex set K dom f s the necessarly unque vector proj f K x K satsfyng D f proj f K x, x = nf {D f y, x : y K}. Smlarly to the metrc projecton n Hlbert spaces, Bregman projectons wth respect to totally convex and dfferentable functons have a varatonal characterzaton. Proposton 6 cf. [17, Corollary 4.4, p. 23]. Suppose that f s totally convex on nt dom f. Let x nt dom f and let K nt dom f be a nonempty, closed and convex set. If ˆx K, then the followng condtons are equvalent: The vector ˆx s the Bregman projecton of x onto K wth respect to f; The vector ˆx s the unque soluton of the varatonal nequalty f x f z, z y 0, y K; The vector ˆx s the unque soluton of the nequalty D f y, z + D f z, x D f y, x, y K. For the next techncal result we need to defne, for any λ > 0, the Yosda approxmaton of A by A λ = f f Res f λa /λ. We have the followng propertes of the Yosda approxmaton A λ. Proposton 7: For any λ > 0 and for any x X, we have Res f λa x, A λ x graph A; 0 Ax f and only f 0 A λ x.

8 8 SIMEON REICH AND SHOHAM SABACH Proof. Indeed, Res f λa x = f + λa 1 f x f x f + λa Res f λa x f f Res f λa x /λ A Res fλa x A λ x A Res fλa x. Indeed, 0 Ax 0 λax f x f + λa x x f + λa 1 f x f x f Res fλa x 0 f f Res f λa x 0 λa λ x 0 A λ x. Now we can prove the followng mportant property of the resolvent. Proposton 8: Let A : X 2 X be a maxmal monotone operator such that A 1 0. Then D f u, Res f λa x + D f Res f λa x, x D f u, x for all λ > 0, u A 1 0 and x X. Proof. Let λ > 0, u A 1 0 and x X be gven. By the monotoncty of A, the three pont dentty 2.2 and Proposton 7, we have D f u, x = D f u, Res f λa x + D f Res f λa x, x + f Res fλa x f x, u ResfλA x = D f u, Res f λa x + D f Res f λa x, x + λ A λ x, u Res fλa x D f u, Res f λa x + D f Res f λa x, x. 3. Auxlary Results In ths secton we prove two lemmata whch are used n the proofs of our man results n Secton 4. Lemma 1: Let f : X R be a totally convex functon. If {D f x n, x 0 } n N s bounded, then the sequence {x n } n N s bounded too. Proof. Snce the sequence {D f x n, x 0 } n N s bounded, there exsts M > 0 such that D f x n, x 0 < M for any n N. Therefore the sequence {ν f x 0, x n x 0 } n N s bounded by M too, because from the defnton of the

9 STRONG CONVERGENCE THEOREMS 9 modulus of total convexty see 2.3 we get that 3.1 ν f x 0, x n x 0 D f x n, x 0 M. Snce the functon f s totally convex, the functon ν f x, s strctly ncreasng and postve on 0, cf. Proposton 2v. Ths mples, n partcular, that ν f x, 1 > 0 for all x X. Now suppose by way of contradcton that the sequence {x n } n N s not bounded. Then there exsts a sequence {n k } k N of postve real numbers such that lm x n k = +. k + Consequently, lm k + x nk x 0 = +. Ths shows that the sequence {ν f x 0, x n x 0 } n N s not bounded. Indeed, there exsts some k 0 > 0 such that x nk x 0 > 1 for any k > k 0 and then, by Proposton 2, we see that ν f x 0, x nk x 0 x nk x 0 ν f x 0, 1 +, because, as noted above, ν f x 0, 1 > 0. Ths contradcts 3.1. Hence the sequence {x n } n N s ndeed bounded, as clamed. Lemma 2: Let f : X R be a totally convex functon and let C be a nonempty, closed and convex subset of X. Suppose that the sequence {x n } n N s bounded and any weak subsequental lmt of {x n } n N belongs to C. If D f x n, x 0 D f proj f C x 0, x 0 for any n N, then {x n } n N converges strongly to proj f C x 0. Proof. Denote proj f C x 0 = ũ. The three pont dentty see 2.2 and the assumpton D f x n, x 0 D f ũ, x 0 yelds D f x n, ũ = D f x n, x 0 + D f x 0, ũ fũ fx 0, x n x 0 D f ũ, x 0 + D f x 0, ũ fũ fx 0, x n x 0 = fũ fx 0, ũ x 0 fũ fx 0, x n x 0 = fũ fx 0, ũ x n. Snce {x n } n N s bounded there s a weakly convergent subsequence {x n } N and denote ts weak lmt by v. We know that v C. It follows from Proposton 6 that lm sup + D f x n, ũ lm sup fũ fx 0, ũ x n + = fũ fx 0, ũ v 0. Hence lm D f x n, ũ = 0. +

10 10 SIMEON REICH AND SHOHAM SABACH Proposton 4 now mples that x n ũ. It follows that the whole sequence {x n } n N converges strongly to ũ = proj f C x 0, as clamed Two Strong Convergence Theorems In ths secton we study the followng algorthm when Z := N x 0 X, ηn = ξn + 1 λ fy n n fx n, ξn A yn, w n = f λ nη n + fx n, C n = { z X : D f z, y n Df z, w n }, C n := N =1 C n, Q n = {z X : fx 0 fx n, z x n 0}, x n+1 = proj f C n Q n x 0, n = 0, 1, 2,..., =1 A 1 0 : Theorem 1: Let A : X 2 X, = 1, 2,..., N, be N maxmal monotone operators such that Z := N =1 A 1 0. Let f : X R be a Legendre functon whch s bounded, unformly Fréchet dfferentable and totally convex on bounded subsets of X. Assume further that f s bounded and unformly Fréchet dfferentable on bounded subsets of X. Then, for each x 0 X, there are sequences {x n } n N whch satsfy 4.1. If, for each = 1, 2,..., N, lm nf n + λ n > 0, and the sequences of errors { } ηn n N X satsfy lm n + ηn = 0, then each such sequence {x n } n N converges strongly to proj f Z x 0 as n +. Proof. Note that dom f = X because dom f = X and f s Legendre. Hence t follows from [4, Corollary 3.14, p. 606] that dom Res f λa = X. We begn wth the followng clam. Clam 1: There are sequences {x n } n N whch satsfy 4.1. As a matter of fact, we wll prove that, for each x 0 X, there exsts a sequence {x n } n N whch s generated by 4.1 wth η n = 0 for all = 1, 2,..., N and n N. It s obvous that C n are closed and convex sets for any = 1, 2,..., N. Hence C n s also closed and convex. It s also obvous that Q n s a closed and convex set. Let u Z. For any n N we have from Proposton 8 that D f u, y n = Df u, Res f λ n Aw n D f u, w n, whch mples that u C n. Snce ths holds for any = 1, 2,..., N, t follows that u C n. Thus Z C n for any n N. On the other hand t s obvous that Z Q 0 = X. Thus Z C 0 Q 0, and therefore x 1 = proj f C 0 Q 0 x 0 s well defned. Now suppose that Z C n 1 Q n 1 for some n 1. Then t follows that there exsts x n C n 1 Q n 1 such that x n = proj f C n 1 Q n 1 x 0 snce C n 1 Q n 1 s

11 STRONG CONVERGENCE THEOREMS 11 a nonempty, closed and convex subset of X. So from Proposton 6 we have fx 0 fx n, y x n 0, for any y C n Q n. Hence we obtan that Z Q n. Therefore Z C n Q n and hence x n+1 = proj f C n Q n x 0 s well defned. Consequently, we see that Z C n Q n for any n N. Thus the sequence we constructed s ndeed well defned and satsfes 4.1, as clamed. From now on we fx an arbtrary sequence {x n } n N satsfyng 4.1. It s clear from the proof of Clam 1 that Z C n Q n for each n N. Clam 2: The sequence {x n } n N s bounded. It follows from the defnton of Q n and Proposton 6 that proj f Q n x 0 = x n. Furthermore, by Proposton 6, for each u Z, we have 4.2 D f x n, x 0 = D f proj f Q n x 0, x 0 D f u, x 0 D f u, proj f Q n x 0 D f u, x 0. Hence the sequence {D f x n, x 0 } n N s bounded by D f u, x 0 for any u Z. Therefore by Lemma 1 the sequence {x n } n N s bounded too, as clamed. Clam 3: Every weak subsequental lmt of {x n } n N belongs to Z. It follows from the defnton of Q n and Proposton 6 that proj f Q n x 0 = x n. Snce x n+1 Q n, t follows from Proposton 6 that D f x n+1, proj f Q n x 0 + D f proj fqn x 0, x 0 D f x n+1, x 0 and hence 4.3 D f x n+1, x n + D f x n, x 0 D f x n+1, x 0. Therefore the sequence {D f x n, x 0 } n N s ncreasng and snce t s also bounded see Clam 2, lm n + D f x n, x 0 exsts. Thus from 4.3 t follows that lm D f x n+1, x n = 0. n + Proposton 5 now mples that lm n + x n+1 x n = 0. Snce wn = f λ nηn + fx n and f s unformly contnuous on bounded subsets of X by Proposton 1, t follows that lm w n x n = 0 n +

12 12 SIMEON REICH AND SHOHAM SABACH for any = 1, 2,..., N, and hence lm n + D f xn, wn = 0. For any = 1, 2,..., N, the three pont dentty see 2.2 mples that D f xn+1, w n = Df x n+1, x n D f xn, w n + fxn fw n, x n+1 x n. Therefore lm D f xn+1, w n = 0. n + Next, for any = 1, 2,..., N, t follows from the ncluson x n+1 C n that D f xn+1, y n Df xn+1, w n. Hence lm n + D f xn+1, y n = 0. Proposton 5 now mples that lm n + y n x n+1 = 0. Therefore, for any = 1, 2,..., N, we have y n x n y n x n+1 + xn+1 x n 0. Ths means that the sequence { } yn s bounded for any = 1, 2,..., N. Now n N let { x nj }j N be a weakly convergent subsequence of {x n} n N and denote ts weak } lmt by v. Then {y nj also converges weakly to v for any = 1, 2,..., N. Snce j N lm nf n + λ n > 0 and lm n + η n = 0, t follows from Proposton 1 that ξn = 1 fxn λ fyn + ηn 0 n for any = 1, 2,..., N. Snce ξn Ayn and A s monotone, t follows that η ξ n, z yn 0 for all z, η graph A. Ths, n turn, mples that η, z v 0 for all z, η graph A. Therefore, usng the maxmal monotoncty of A, we now obtan that v A 1 0 for each = 1, 2,..., N. Thus v Z and ths proves Clam 3. Clam 4: The sequence {x n } n N converges strongly to proj f Z x 0. Let ũ = proj f Z x 0. Snce x n+1 = proj f C n Q n x 0 and Z s contaned n C n Q n, we have D f x n+1, x 0 D f ũ, x 0. Therefore Lemma 2 mples that {x n } n N converges strongly to ũ = proj f Z x 0, as clamed. Ths completes the proof of Theorem 1. We now present another result whch s smlar to Theorem 1, but wth a dfferent type of errors. More precsely, we study the followng algorthm when

13 STRONG CONVERGENCE THEOREMS 13 Z := N =1 A : x 0 X, yn = Res f λ Ax n + e n, n Cn = { z X : D f z, y n Df z, xn + en}, C n := N =1 C n, Q n = {z X : fx 0 fx n, z x n 0}, x n+1 = proj f H n W n x 0, n = 0, 1, 2,...,. Theorem 2: Let A : X 2 X, = 1, 2,..., N, be N maxmal monotone operators such that Z := N =1 A 1 0. Let f : X R be a Legendre functon whch s bounded, unformly Fréchet dfferentable and totally convex on bounded subsets of X. Then, for each x 0 X, there are sequences {x n } n N whch satsfy 4.4. If, for each = 1, 2,..., N, lm nf n + λ n > 0, and the sequences of errors { } e n X satsfy lm n N n + e n = 0, then each such sequence {x n } n N converges strongly to proj f Z x 0 as n +. Proof. Note that dom f = X because dom f = X and f s Legendre. Hence t follows from [4, Corollary 3.14, p. 606] that dom Res f λa = X. We begn wth the followng clam. Clam 1: There are sequences {x n } n N whch satsfy 4.4. As a matter of fact, we wll prove that, for each x 0 X, there exsts a sequence {x n } n N whch s generated by 4.4 wth e n = 0 for all = 1, 2,..., N and n N. It s obvous that C n are closed and convex sets for any = 1, 2,..., N. Hence C n s also closed and convex. It s also obvous that Q n s a closed and convex set. Let u Z. For any n N, we obtan from Proposton 8 that D f u, y n = Df u, Res f λ Ax n + e n D f u, xn + e n, n whch mples that u C n. Snce ths holds for any = 1, 2,..., N, t follows that u C n. Thus Z C n for any n N. On the other hand t s obvous that Z Q 0 = X. Thus Z C 0 Q 0, and therefore x 1 = proj f C 0 Q 0 x 0 s well defned. Now suppose that Z C n 1 Q n 1 for some n 1. The t follows that there exsts x n C n 1 Q n 1 such that x n = proj f C n 1 Q n 1 x 0 snce C n 1 Q n 1 s a nonempty, closed and convex subset of X. So from Proposton 6 we have fx 0 fx n, y x n 0, for any y C n Q n. Hence we obtan that Z Q n. Therefore Z C n Q n and hence x n+1 = proj f C n Q n x 0 s well defned. Consequently, we see that Z C n Q n for any n N. Thus the sequence we constructed s ndeed well defned and satsfes 4.4, as clamed.

14 14 SIMEON REICH AND SHOHAM SABACH From now on we fx an arbtrary sequence {x n } n N satsfyng 4.4. It s clear from the proof of Clam 1 that Z C n Q n for each n N. Clam 2: The sequence {x n } n N s bounded. It follows from the defnton of Q n and Proposton 6 that proj f Q n x 0 = x n. Furthermore, by Proposton 6, for each u Z, we have 4.5 D f x n, x 0 = D f proj f Q n x 0, x 0 D f u, x 0 D f u, proj f Q n x 0 D f u, x 0. Hence the sequence {D f x n, x 0 } n N s bounded by D f u, x 0 for any u Z. Therefore by Lemma 1 the sequence {x n } n N s bounded too, as clamed. Clam 3: Every weak subsequental lmt of {x n } n N belongs to Z. It follows from the defnton of Q n and Proposton 6 that proj f Q n x 0 = x n. Snce x n+1 Q n, t follows from Proposton 6 that D f x n+1, proj f Q n x 0 + D f proj fqn x 0, x 0 D f x n+1, x 0 and hence 4.6 D f x n+1, x n + D f x n, x 0 D f x n+1, x 0. Therefore the sequence {D f x n, x 0 } n N s ncreasng and snce t s also bounded see Clam 2, lm n + D f x n, x 0 exsts. Thus from 4.6 t follows that 4.7 lm n + D f x n+1, x n = 0. Proposton 5 now mples that lm n + x n+1 x n = 0. For any = 1, 2,..., N, t follows from the defnton of the Bregman dstance see 2.1 that D f xn, x n + e n = f xn f x n + e n fxn + e n, x n x n + e n = f x n f x n + e n + fxn + e n, e n. The functon f s bounded on bounded subsets of X and therefore f s bounded on bounded subsets of X see [13, Proposton , p. 17]. In addton, f s unformly Fréchet dfferentable and therefore f s unformly contnuous on bounded subsets see [1, Theorem 1.8, p. 13]. Hence, snce lm n + e n = 0, t follows that 4.8 lm D f xn, x n + e n = 0. n +

15 STRONG CONVERGENCE THEOREMS 15 For any = 1, 2,..., N, t follows from the three pont dentty see 2.2 that D f xn+1, x n + e n = Df x n+1, x n + D f xn, x n + e n + fx n fx n + e n, x n+1 x n. Snce lm n + x n+1 x n = 0 and f s bounded on bounded subsets of X, 4.7 and 4.8 mply that lm D f xn+1, x n + e n = 0. n + For any = 1, 2,..., N, t follows from the ncluson x n+1 C n that D f xn+1, y n Df xn+1, x n + e n. Hence lm n + D f xn+1, y n = 0. Proposton 5 now mples that lm n + y n x n+1 = 0. Therefore, for any = 1, 2,..., N, we have y n x n y n x n+1 + xn+1 x n 0. Ths means that the sequence { } yn s bounded for any = 1, 2,..., N. Now n N let { x nj }j N be a weakly convergent subsequence of {x n} n N and denote ts weak } lmt by v. Then {y nj also converges weakly to v for any = 1, 2,..., N. j N Let ξ n Ay n, snce lm nf n + λ n > 0 and lm n + e n = 0, t follows from Proposton 1 that ξn = 1 fxn λ + e n fyn 0 n for any = 1, 2,..., N. Snce ξn Ayn and A s monotone, t also follows that η ξ n, z yn 0 for all z, η graph A. Ths, n turn, mples that η, z v 0 for all z, η graph A. Therefore, usng the maxmal monotoncty of A, we now obtan that v A 1 0 for each = 1, 2,..., N. Thus v Z and ths proves Clam 3. Clam 4: The sequence {x n } n N converges strongly to proj f Z x 0. Let ũ = proj f Z x 0. Snce x n+1 = proj f C n Q n x 0 and Z s contaned n C n Q n, we have D f x n+1, x 0 D f ũ, x 0. Therefore Lemma 2 mples that {x n } n N converges strongly to ũ = proj f Z x 0, as clamed. Ths completes the proof of Theorem 2.

16 16 SIMEON REICH AND SHOHAM SABACH 5. Zero Free Operators Ths secton concerns the case where our two algorthms are appled to a sngle zero free operator A. In ths case both our algorthms take the form 5.1 and 5.2 x 0 X, η n = ξ n + 1 λ n fy n fx n, ξ n Ay n, w n = f λ n η n + fx n, C n = {z X : D f z, y n D f z, x n }, Q n = {z X : fx 0 fx n, z x n 0}, x n+1 = proj f C n Q n x 0, n = 0, 1, 2,..., x 0 X, y n = Res f λ na x n + e n, C n = {z X : D f z, y n D f z, x n + e n }, Q n = {z X : fx 0 fx n, z x n 0}, x n+1 = proj f C n Q n x 0, n = 0, 1, 2,...,. We frst recall the followng lemma see [34, Lemma 1]: Lemma 3: If A : X 2 X s a maxmal monotone operator wth bounded doman, then A 1 0. Now we can prove that the generaton of an nfnte sequence by Algorthm 5.1 or 5.2 does not depend on the zero set A 1 0 of A beng not empty. Theorem 3. Let A : X 2 X be a maxmal monotone operator. Let f : X R be a Legendre functon whch s bounded, unformly Fréchet dfferentable and totally convex on bounded subsets of X. In case of Algorthm 5.1 assume, n addton, that f s bounded and unformly Fréchet dfferentable on bounded subsets of X. Then, for each x 0 X, there are sequences {x n } n N whch satsfy ether 5.1 or 5.2. If lm nf n + λ n > 0, and ether the sequence of errors {η n } n N X satsfes lm n + η n = 0 or the sequence of errors {e n } n N X satsfes lm n + e n = 0, then ether A 1 0 and each such sequence {x n } n N converges strongly to proj f A 1 0 x 0 or A 1 0 = and each such sequence {x n } n N satsfes lm n + x n = +. Proof. In vew of Theorem 1 and Theorem 2, we only need to consder the case where A 1 0 =. Frst of all we prove that n ths case, for each x 0 X, there s a sequence {x n } n N whch satsfes ether 5.1 wth η n = 0 or 5.2 wth e n = 0 for all n N.

17 STRONG CONVERGENCE THEOREMS 17 We prove ths by nducton. We frst check that the ntal step n = 0 s well defned. Indeed, the problem 0 Ax + 1 λ 0 fx fx 0 always has a soluton y 0, ξ 0 because t s equvalent to the problem x =Res f λ 0A x 0 and ths problem does have a soluton snce dom Res f λa = X see Proposton 3 and [4, Theorem 3.13v, p. 606]. Now note that Q 0 = X. Snce C 0 cannot be empty y 0 C 0, the next terate x 1 can be generated; t s the Bregman projecton of x 0 onto C 0 = Q 0 C 0. Note that whenever x n s generated, y n and ξ n can further be obtaned because the proxmal subproblems always have solutons. Suppose now that x n and y n, ξ n have already been defned for n = 0,..., ˆn. We have to prove that xˆn+1 s also well defned. To ths end, take any z 0 dom A and defne and ρ = max { y n z 0 : n = 0,..., ˆn} hx = { 0, x z 0 ρ + 1 +, otherwse. Then h : X, + ] s a proper, convex and lower semcontnuous functon, ts subdfferental h s maxmal monotone see [31, Theorem 2.13, p. 124], and A = A + h s also maxmal monotone see [37]. Furthermore, A z = A z for all z z 0 < ρ + 1. Therefore ξ n A y n for n = 0,..., ˆn. We conclude that x n and y n, ξ n also satsfy the condtons of Theorems 1 and 2 appled to the problem 0 A x. Snce A has a bounded effectve doman, ths problem has a soluton by Lemma 3. Thus t follows from Clam 1 n the proofs of Theorems 1 and 2 that xˆn+1 s well defned n both Algorthms 5.1 and 5.2. Hence the whole sequence {x n } n N s well defned, as asserted. If {x n } n N were to have a bounded subsequence, then t would follow from Clam 3 n the proofs of Theorems 1 and 2 that A had a zero. Therefore f A 1 0 =, then lm n + x n = +, as asserted. 6. Consequences of the Strong Convergence Theorems Algorthm 1.4 s a specal case of Algorthm 5.1 when η n = 0 for all n N, and a specal case of Algorthm 5.2 when e n = 0 for all n N. Hence as a drect consequence of Theorems 1, 2 and 3 we obtan the followng result:

18 18 SIMEON REICH AND SHOHAM SABACH Corollary 1. Let A : X 2 X be a maxmal monotone operator. Let f : X R be a Legendre functon whch s bounded, unformly Fréchet dfferentable and totally convex on bounded subsets of X, and suppose that lm nf n + λ n > 0. Then for each x 0 X, the sequence {x n } n N generated by 1.4 s well defned, and ether A 1 0 and {x n } n N converges strongly to proj f A 1 0 x 0 as n +, or A 1 0 = and lm n + x n = +. Notable corollares of Theorems 1, 2 and 3 occur when the space X s both unformly smooth and unformly convex. In ths case the functon fx = 1 2 x 2 s Legendre cf. [3, Lemma 6.2, p. 24] and unformly Fréchet dfferentable on bounded subsets of X. Accordng to [14, Corollary 1, p. 325], f s sequentally consstent snce X s unformly convex and hence f s totally convex on bounded subsets of X. Therefore Theorems 1, 2 and 3 hold n ths context and lead us to the followng two results whch, n some sense, complement Theorem 3.1 n [42] see also Theorem 3.5 n [29]. Corollary 2. Let X be a unformly smooth and unformly convex Banach space and let A : X 2 X be a maxmal monotone operator. Then, for each x 0 X, the sequence {x n } n N generated by 1.3 s well defned. If lm nf n + λ n > 0, then ether A 1 0 and {x n } n N converges strongly to Q A 1 0 x 0 as n +, or A 1 0 = and lm n + x n = +. Corollary 3. Let X be a Hlbert space and let A : X 2 X be a maxmal monotone operator. Then, for each x 0 X, the sequence {x n } n N generated by 1.2 s well defned. If lm nf n + λ n > 0, then ether A 1 0 and {x n } n N converges strongly to P A 1 0x 0 as n +, or A 1 0 = and lm n + x n = +. These corollares also hold n the presence of computatonal errors as n Theorems 1, 2 and An Applcaton of the Strong Convergence Theorems Let g : X, + ] be a proper, convex and lower semcontnuous functon. Recall that the subdfferental g of g s defned for any x X by g x := {ξ X : ξ, y x g y g x y X}. Applyng Theorems 1, 2 and 3 to the subdfferental of g, we obtan an algorthm for fndng mnmzers of g. Proposton 9. Let g : X, + ] be a proper, convex and lower semcontnuous functon whch attans ts mnmum over X. If f : X R s a Legendre functon whch s bounded, unformly Fréchet dfferentable, and totally convex on bounded subsets of X, and {λ n } n N s a postve sequence wth lm nf n + λ n > 0,

19 STRONG CONVERGENCE THEOREMS 19 then, for each x 0 X, the sequence {x n } n N generated by x 0 X, 0 = ξ n + 1 λ n fy n fx n, ξ n g y n, C n = {z X : D f z, y n D f z, x n }, Q n = {z X : fx 0 fx n, z x n 0}, x n+1 = proj f C n Q n x 0, n = 0, 1, 2,..., converges strongly to a mnmzer of g as n +. If g does not attan ts mnmum over X, then lm n + x n = +. Proof. The subdfferental g of g s a maxmal monotone operator because g s a proper, convex and lower semcontnuous functon see [31, Theorem 2.13, p. 124]. Snce the zero set of g concdes wth the set of mnmzers of g, Proposton 9 follows mmedately from Theorems 1, 2 and 3. Note that n ths case s equvalent to y n = arg mn x X { g x + 1 } D f x, x n λ n 0 g y n + 1 λ n fy n fx n. 8. Acknowledgements The frst author was partally supported by the Israel Scence Foundaton Grant 647/07, by the Fund for the Promoton of Research at the Technon and by the Technon Presdent s Research Fund. Both authors thank the referee for several helpful comments. References [1] Ambrosett, A. and Prod, G.: A prmer of nonlnear analyss, Cambrdge Unversty Press, Cambrdge, [2] Bauschke, H. H. and Borwen, J. M.: Legendre functons and the method of random Bregman projectons, J. Convex Anal , [3] Bauschke, H. H., Borwen, J. M. and Combettes, P. L.: Essental smoothness, essental strct convexty, and Legendre functons n Banach spaces, Commun. Contemp. Math , [4] Bauschke, H. H., Borwen, J. M. and Combettes, P. L.: Bregman monotone optmzaton algorthms, SIAM J. Control Optm , [5] Bauschke, H. H. and Combettes, P. L.: A weak-to-strong convergence prncple for Fejérmonotone methods n Hlbert spaces, Math. Oper. Res , [6] Bauschke, H. H. and Combettes, P. L.: Constructon of best Bregman approxmatons n reflexve Banach spaces, Proc. Amer. Math. Soc ,

20 20 SIMEON REICH AND SHOHAM SABACH [7] Bauschke, H. H., Matoušková, E. and Rech, S.: Projecton and proxmal pont methods: convergence results and counterexamples, Nonlnear Anal , [8] Bonnans, J. F. and Shapro, A.: Perturbaton analyss of optmzaton problems, Sprnger Verlag, New York, [9] Bregman, L. M.: A relaxaton method for fndng the common pont of convex sets and ts applcaton to the soluton of problems n convex programmng, USSR Comput. Math. and Math. Phys , [10] Brézs, H. and Lons, P.-L.: Produts nfns de résolvantes, Israel J. Math , [11] Bruck, R. E. and Rech, S.: Nonexpansve projectons and resolvents of accretve operators, Houston J. Math , [12] Butnaru, D., Censor, Y. and Rech, S.: Iteratve averagng of entropc projectons for solvng stochastc convex feasblty problems, Computatonal Optmzaton and Applcatons , [13] Butnaru, D. and Iusem, A. N.: Totally convex functons for fxed ponts computaton and nfnte dmensonal optmzaton, Kluwer Academc Publshers, Dordrecht, [14] Butnaru, D., Iusem, A. N. and Resmerta, E.: Total convexty for powers of the norm n unformly convex Banach spaces, J. Convex Anal , [15] Butnaru, D., Iusem, A. N. and Zălnescu, C.: On unform convexty, total convexty and convergence of the proxmal pont and outer Bregman projecton algorthms n Banach spaces, J. Convex Anal , [16] Butnaru, D. and Kassay, G.: A proxmal-projecton method for fndng zeroes of set-valued operators, SIAM J. Control Optm , [17] Butnaru, D. and Resmerta, E.: Bregman dstances, totally convex functons and a method for solvng operator equatons n Banach spaces, Abstr. Appl. Anal. 2006, Art. ID 84919, [18] Censor, Y. and Lent, A.: An teratve row-acton method for nterval convex programmng, J. Optm. Theory Appl , [19] Censor, Y. and Zenos, S. A.: Proxmal mnmzaton algorthm wth D-functons, J. Optm. Theory Appl , [20] Coranescu, I.: Geometry of Banach spaces, dualty mappngs and nonlnear problems, Kluwer Academc Publshers, Dordrecht, [21] Combettes, P. L.: Strong convergence of block-teratve outer approxmaton methods for convex optmzaton, SIAM J. Control Optm , [22] Ecksten, J.: Nonlnear proxmal pont algorthms usng Bregman functons, wth applcaton to convex programmng, Math. Oper. Res , [23] Gárcga Otero, R. and Svater, B. F.: A strongly convergent hybrd proxmal method n Banach spaces, J. Math. Anal. Appl , [24] Güler, O.: On the convergence of the proxmal pont algorthm for convex mnmzaton, SIAM J. Control Optm , [25] Kammura, S. and Takahash, W.: Approxmatng solutons of maxmal monotone operators n Hlbert spaces, J. Approx. Theory , [26] Kammura, S. and Takahash, W.: Weak and strong convergence of solutons to accretve operator nclusons and applcatons, Set-Valued Anal , [27] Kammura, S. and Takahash, W.: Strong convergence of a proxmal-type algorthm n a Banach space, SIAM J. Optm ,

21 STRONG CONVERGENCE THEOREMS 21 [28] Martnet, B.: Régularsaton d néquatons varatonelles par approxmatons successves, Revue Françase d Informatque et de Recherche Opératonelle , [29] Nakajo, K. and Takahash, W.: Strong convergence theorems for nonexpansve mappngs and nonexpansve semgroups, J. Math. Anal. Appl , [30] Nevanlnna, O. and Rech, S.: Strong convergence of contracton semgroups and of teratve methods for accretve operators n Banach spaces, Israel J. Math , [31] Pascal, D. and Sburlan, S.: Nonlnear mappngs of monotone type, Sjthoff & Nordhoff Internatonal Publshers, Alphen aan den Rjn, [32] Rech, S.: Weak convergence theorems for nonexpansve mappngs n Banach spaces, J. Math. Anal. Appl , [33] Rech, S.: A weak convergence theorem for the alternatng method wth Bregman dstances, n Theory and applcatons of nonlnear operators of accretve and monotone type, Marcel Dekker, New York, 1996, [34] Rech, S. and Sabach, S.: A strong convergence theorem for a proxmal-type algorthm n reflexve Banach spaces, preprnt, [35] Rech, S. and Zaslavsk, A. J.: Infnte products of resolvents of accretve operator, Topol. Methods Nonlnear Anal , [36] Resmerta, E.: On total convexty, Bregman projectons and stablty n Banach spaces, J. Convex Anal , [37] Rockafellar, R. T.: On the maxmalty of sums of nonlnear monotone operators, Trans. Amer. Math. Soc , [38] Rockafellar, R. T.: Monotone operators and the proxmal pont algorthm, SIAM J. Control Optm , [39] Rockafellar, R. T.: Augmented Lagrangans and applcatons of the proxmal pont algorthm n convex programmng, Math. Oper. Res , [40] Rockafellar, R. T. and Wets, R. J.-B.: Varatonal analyss, Sprnger Verlag, Berln, [41] Solodov, M. V. and Svater, B. F.: Forcng strong convergence of proxmal pont teratons n a Hlbert space, Math. Program , [42] We, L. and Zhou, H. Y.: Projecton scheme for zero ponts of maxmal monotone operators n Banach spaces, J. Math. Res. Exposton , [43] Yao, J. C. and Zeng, L. C.: An nexact proxmal-type algorthm n Banach spaces, J. Optm. Theory Appl , [44] Zălnescu, C.: Convex analyss n general vector spaces, World Scentfc Publshng, Sngapore, Smeon Rech: Department of Mathematcs, The Technon - Israel Insttute of Technology, Hafa, Israel E-mal address: srech@tx.technon.ac.l Shoham Sabach: Department of Mathematcs, The Technon - Israel Insttute of Technology, Hafa, Israel E-mal address: ssabach@tx.technon.ac.l