A Strong Convergence Theorem for a Proximal-Type. Algorithm in Re exive Banach Spaces

Transcription

1 A Strog Covergece Theorem for a Proxmal-Type Algorthm Re exve Baach Spaces Smeo Rech ad Shoham Sabach Abstract. We establsh a strog covergece theorem for a proxmal-type algorthm whch approxmates (commo) zeroes of maxmal mootoe operators re exve Baach spaces. Ths algorthm employs a well-chose covex fucto. The behavor of the algorthm the presece of computatoal errors ad the case of zero free operators s also aalyzed. Fally, we meto several corollares, varatos ad applcatos. 1. Itroducto I ths paper X deotes a real re exve Baach space wth orm kk ad X stads for the (topologcal) dual of X edowed wth the duced orm kk. We deote the value of the fuctoal 2 X at x 2 X by h; x. A operator A : X! 2 X s sad to be mootoe f for ay x; y 2 dom A, we have 2 Ax ad 2 Ay =) h ; x y 0: (Recall that the set dom A = fx 2 X : Ax 6=?g s called the e ectve doma of such a operator A.) A mootoe operator A s sad to be maxmal f graph A, the graph of A, s ot a proper subset of the graph of ay other mootoe operator. I ths paper f : X! ( 1; +1] s always a proper, lower semcotuous ad 2000 Mathematcs Subject Class cato. Prmary: 47H05; Secodary: 47J25. Key words ad phrases. Baach space, Bregma projecto, Legedre fucto, maxmal mootoe operator, mootoe operator, proxmal pot algorthm, resolvet, totally covex fucto. 1

2 2 SIMEON REICH AND SHOHAM SABACH covex fucto, ad f : X! ( 1; +1] s the Fechel cojugate of f. The set of oegatve tegers wll be deoted by N. The problem of dg a elemet x 2 X such that 0 2 Ax s very mportat Optmzato Theory ad related elds. For example, f A s the subd of f, the A s a maxmal mootoe operator ad the equato 0 (x) s equvalet to the problem of mmzg f over X. Oe of the methods for solvg ths problem Hlbert space s the well-kow proxmal pot algorthm. Let H be a Hlbert space ad let I deote the detty operator o H. The proxmal pot algorthm geerates, for ay startg pot x 0 = x 2 H, a sequece fx g 2N H by the rule (1.1) x +1 = (I + A) 1 x ; = 0; 1; 2; : : : ; where f g 2N s a gve sequece of postve real umbers. Note that (1.1) s equvalet to 0 2 Ax (x +1 x ) ; = 0; 1; 2; : : : : Ths algorthm was rst troduced by Martet [26] ad further developed by Rockafellar [34], who proves that the sequece geerated by (1.1) coverges weakly to a elemet of A 1 (0) whe A 1 (0) s oempty ad lm f!+1 > 0. Furthermore, Rockafellar [34] asks f the sequece geerated by (1.1) coverges strogly. Ths questo was aswered the egatve by Güler [22], who preseted a example of a subd eretal for whch the sequece geerated by (1.1) coverges weakly but ot strogly; see [6] for a more recet ad smpler example. More recetly, Solodov ad Svater [37] have mod ed the proxmal pot algorthm order to geerate a strogly coverget sequece. They troduce the followg algorthm:

3 PROXIMAL-TYPE ALGORITHM 3 (1.2) 8 >< >: x 0 2 H; 0 = v + 1 (y x ) ; v 2 Ay ; H = fz 2 H : hv ; z y 0g ; W = fz 2 H : hx 0 x ; z x 0g ; x +1 = P H\W (x 0 ); = 0; 1; 2; : : : : Here, for each x 2 H ad each oempty, closed ad covex subset C of H, the mappg P C s de ed by kx P C xk = f fkx zk : z 2 Cg. Ths mappg s called the metrc projecto of H oto C. They prove that f A 1 (0) s oempty ad lm f!+1 > 0, the the sequece geerated by (1.2) coverges strogly to P A 1 (0). Kammura ad Takahash [25] geeralze ths result to those Baach spaces X whch are both uformly covex ad uformly smooth. They troduce the followg algorthm: 8 x 0 2 X; (1.3) >< >: 0 = v + 1 (Jy Jx ) ; v 2 Ay ; H = fz 2 X : hv ; z y 0g ; W = fz 2 X : hjx 0 Jx ; z x 0g ; x +1 = Q H\W (x 0 ); = 0; 1; 2; : : : ; where J s the ormalzed dualty mappg of the space X. Here, for each oempty, closed ad covex subset C of X, Q C s a certa geeralzato of the metrc projecto P C H. They prove that f A 1 (0 ) s oempty ad lm f!+1 > 0, the the sequece geerated by (1.3) coverges strogly to Q A 1 (0 ). Other developmets regardg the proxmal pot algorthm ca be foud, for example, [4, 6, 9, 10, 14, 16, 18, 20, 23, 24, 27, 29, 30, 31, 35, 38]. I the preset paper we study a exteso of algorthms (1.2) ad (1.3) to all re exve Baach spaces usg a well-chose covex fucto f. More precsely, we

4 4 SIMEON REICH AND SHOHAM SABACH cosder the followg algorthm troduced by Gárcga Otero ad Svater [21]: 8 x 0 2 X; (1.4) >< >: 0 = + 1 (rf(y ) rf(x )) ; 2 Ay ; H = fz 2 X : h ; z y 0g ; W = fz 2 X : hrf(x 0 ) rf(x ); z x 0g ; x +1 = proj f H \W (x 0 ); = 0; 1; 2; : : : ; where f g 2N s a gve sequece of postve real umbers, rf s the gradet of f ad proj f C s the Bregma projecto (see Secto 2.4) of X oto C duced by f. Algorthm (1.4) s more exble tha (1.3) because t leaves us the freedom of ttg the fucto f to the ature of the operator A (especally whe A s the subd eretal of some fucto) ad of the space X ways whch make the applcato of (1.4) smpler tha that of (1.3). It should be observed that f X s a Hlbert space H, the usg (1.4) the fucto f (x) = (1=2) kxk 2, oe obtas exactly algorthm (1.2). If X s ot a Hlbert space, but stll a uformly covex ad uformly smooth Baach space X, the settg f (x) = (1=2) kxk 2 (1.4), oe obtas exactly (1.3). We also ote that the choce f (x) = (1=2) kxk 2 some Baach spaces may make the computatos algorthm (1.3) qute d cult. These computatos ca be smpl ed by a approprate choce of f. For stace, f X = `p or X = L p wth p 2 (1; +1), ad f (x) = (1=p) kxk p (1.4), the the computatos become smpler tha those requred (1.3), whch correspods to f (x) = (1=2) kxk 2. We propose a exteso of algorthm (1.4) (see algorthm (3.1)) whch approxmates a commo zero of several maxmal mootoe operators ad whch allows computatoal errors. Our ma result (Theorem 1) s formulated ad proved Secto 3. The ext secto s devoted to several prelmary de tos ad results. The behavor of the algorthm whe the operator A s zero free s aalyzed Secto 4 (see Theorem 2). The fth secto cotas three corollares of Theorems 1 ad 2. I the sxth ad last secto we preset a applcato of Theorems 1 ad 2.

5 PROXIMAL-TYPE ALGORITHM 5 2. Prelmares 2.1. Some facts about Legedre fuctos. Legedre fuctos mappg a geeral Baach space X to ( 1; +1] are de ed [3]. Accordg to [3, Theorems 5.4 ad 5.6], sce X re exve, the fucto f s Legedre f ad oly f t sats es the followg two codtos: (L1) The teror of the doma of f, t dom f, s oempty, f s Gâteaux d eretable (see below) o t dom f ad dom rf = t dom f; (L2) The teror of the doma of f, t dom f, s oempty, f s Gâteaux d eretable o t dom f ad dom rf = t dom f : Sce X s re exve, we always have (@f) 1 (see [7, p. 83]). Ths fact, whe combed wth codtos (L1) ad (L2), mples the followg equaltes: rf = (rf ) 1 ; ra rf = dom rf = t dom f ad ra rf = dom rf = t dom f: Also, codtos (L1) ad (L2), cojucto wth [3, Theorem 5.4], mply that the fuctos f ad f are strctly covex o the teror of ther respectve domas. Several terestg examples of Legedre fuctos are preseted [2] ad [3]. Amog them are the fuctos 1 s kks wth s 2 (1; 1), where the Baach space X s smooth ad strctly covex ad, partcular, a Hlbert space. The fucto f s called co te f dom f = X.

6 6 SIMEON REICH AND SHOHAM SABACH 2.2. A property of gradets. For ay covex f : X! ( 1; +1] we deote by dom f the set fx 2 X : f (x) < +1g. For ay x 2 dom f ad y 2 X, we deote by f (x; y) the rght-had dervatve of f at x the drecto y, that s, f f(x + ty) (x; y) := lm t&0 t f(x) : The fucto f s sad to be Gâteaux d eretable at x f lm t!0 (f(x + ty) f(x)) =t exsts for ay y. The fucto f s sad to be Fréchet d eretable at x f ths lmt s attaed uformly kyk = 1. Fally, f s sad to be uformly Fréchet d eretable o a subset E of X f the lmt s attaed uformly for x 2 E ad kyk = 1. We wll eed the followg result. Proposto 1. If f : X! R s uformly Fréchet d eretable ad bouded o bouded subsets of X, the rf s uformly cotuous o bouded subsets of X from the strog topology of X to the strog topology of X. Proof. If ths result were ot true, there would be bouded sequeces fx g 2N ad fy g 2N, ad a postve umber " such that kx y k! 0 ad hrf (x ) rf (y ) ; w 2", where fw g 2N s a sequece X wth kw k = 1 for each 2 N. Sce f s uformly Fréchet d eretable, there s a postve umber such that f (y + tw ) f (y ) t hrf (y ) ; w "t for all 0 < t < ad 2 N. We also have hrf (x ) ; (y + tw ) x f (y + tw ) f (x ) ; 2 N: I other words, t hrf (x ) ; w f (y + tw ) f (y ) + hrf (x ) ; x y + f (y ) f (x ) :

7 PROXIMAL-TYPE ALGORITHM 7 Hece 2"t t hrf (x ) rf (y ) ; w [f (y + tw ) f (y ) t hrf (y ) ; w ] + hrf (x ) ; x y + f (y ) f (x ) "t + hrf (x ) ; x y + f (y ) f (x ) : Sce rf s bouded o bouded subsets of X (see [12, Proposto , p. 17]), t follows that hrf (x ) ; x y coverges to zero, whle f (y ) f (x )! 0 sce f s uformly cotuous o bouded subsets (see [1, Theorem 1.8, p. 13]). But ths would yeld that 2"t "t, a cotradcto Some facts about totally covex fuctos. Let f : X! ( 1; +1] be covex. The fucto D f : dom f t dom f! [0; +1] de ed by D f (y; x) := f(y) f(x) f (x; y x); s called the Bregma dstace wth respect to f (cf. [17]). If f s a Gâteaux d eretable fucto, the the Bregma dstace has the followg mportat property, called the three pot detty: for ay x; y; z 2 t dom f, (2.1) D f (x; y) + D f (y; z) D f (x; z) = hrf(z) rf(y); x y : Recall that, accordg to [12, Secto 1.2, p. 17] (see also [11]), the fucto f s called totally covex at a pot x 2 t dom f f ts modulus of total covexty at x, that s, the fucto f : t dom f [0; +1)! [0; +1], de ed by (2.2) f (x; t) := f fd f (y; x) : y 2 dom f; ky xk = tg ; s postve wheever t > 0. The fucto f s called totally covex whe t s totally covex at every pot x 2 t dom f. I addto, the fucto f s called totally covex o bouded sets f f (E; t) s postve for ay oempty bouded subset E of X ad for ay t > 0, where the modulus of total covexty of the fucto f o

8 8 SIMEON REICH AND SHOHAM SABACH the set E s the fucto f : t dom f [0; +1)! [0; +1] de ed by f (E; t) := f f f (x; t) j x 2 E \ dom fg : Examples of totally covex fuctos ca be foud, for example, [12, 15]. The followg proposto summarzes some propertes of the modulus of total covexty. Proposto 2 (cf. [12, Proposto 1.2.2, p. 18]). Let f be a proper, covex ad lower semcotuous fucto. If x 2 t dom f, the () The doma of f (x; ) s a terval of the form [0; f (x)) or [0; f (x)] wth f (x) 2 (0; +1]. () If c 2 [1; +1) ad t 0, the f (x; ct) c f (x; t). () The fucto f (x; ) s superaddtve, that s, for ay s; t 2 [0; +1), we have f (x; s + t) f (x; s) + f (x; t). (v) The fucto f (x; ) s creasg; t s strctly creasg f ad oly f f s totally covex at x. The followg proposto follows from [14, Proposto 2.3, p. 39] ad [39, Theorem , p. 164]. Proposto 3. If f s Fréchet d eretable ad totally covex, the f s co te. The ext proposto turs out to be very useful the proof of our ma result. Proposto 4 (cf. [32, Proposto 2.2, p. 3]). If x 2 dom f, the the followg statemets are equvalet: () The fucto f s totally covex at x; () For ay sequece fy g 2N dom f, lm D f (y ; x) = 0 ) lm ky xk = 0:!+1!+1

9 PROXIMAL-TYPE ALGORITHM 9 Recall that the fucto f s called sequetally cosstet (see [15]) f for ay two sequeces fx g 2N ad fy g 2N X such that the rst oe s bouded, lm D f (y ; x ) = 0 ) lm ky x k = 0:!+1!+1 Proposto 5 (cf. [12, Lemma 2.1.2, p. 67]). If dom f cotas at least two pots, the the fucto f s totally covex o bouded sets f ad oly f the fucto f s sequetally cosstet The resolvet of A relatve to f. Let A : X! 2 X be a operator ad assume that f Gâteaux d eretable. The operator Prt f A := (rf + A) 1 : X! 2 X s called the protoresolvet of A, or, more precsely, the protoresolvet of A relatve to f. Ths allows us to de e the resolvet of A, or, more precsely, the resolvet of A relatve to f, troduced ad studed [5], as the operator Res f A : X! 2X gve by Res f A := Prtf A rf. Ths operator s sgle-valued whe A s mootoe ad f s strctly covex o t dom f. If A where ' s a proper, lower semcotuous ad covex fucto, the we deote Prox f ' := Prt ad proxf ' := Res : If C s a oempty, closed ad covex subset of X, the the dcator fucto C of C, that s, the fucto 8 >< 0 f x 2 C C (x) := >: +1 f x =2 C s proper, covex ad lower semcotuous, ad C exsts ad s a maxmal mootoe operator wth doma C. The operator prox f C s called the Bregma projecto oto C wth respect to f (cf. [8]) ad we deote t by proj f C. Note that f X s a Hlbert space ad f(x) = 1 2 kxk2, the the Bregma projecto of x oto C,.e., argm fky xk : y 2 Cg, s the metrc projecto P C.

10 10 SIMEON REICH AND SHOHAM SABACH Recall that the Bregma projecto of x oto the oempty, closed ad covex set K dom f, s the ecessarly uque vector proj f K (x) 2 K satsfyg D f proj f K (x); x = f fd f (y; x) : y 2 Kg : Smlarly to the metrc projecto Hlbert spaces, Bregma projectos wth respect to totally covex ad d eretable fuctos have varatoal characterzatos. Proposto 6 (cf. [15, Corollary 4.4, p. 23]). Suppose that f s totally covex o t dom f. Let x 2 t dom f ad let K t dom f be a oempty, closed ad covex set. If ^x 2 K, the the followg codtos are equvalet: () The vector ^x s the Bregma projecto of x oto K wth respect to f; () The vector ^x s the uque soluto of the varatoal equalty hrf (x) rf (z) ; z y 0; 8y 2 K; () The vector ^x s the uque soluto of the equalty D f (y; z) + D f (z; x) D f (y; x) ; 8y 2 K: 3. A Strog Covergece Theorem for a Proxmal-Type Algorthm I ths secto we study the followg algorthm whe Z := T N =1 A 1 (0 ) 6=?: 8 x 0 2 X; = + 1 rf(y ) rf(x ) ; 2 A y ; (3.1) >< H = z 2 X : ; z y 0 ; H := \ N =1 H ; W = fz 2 X : hrf(x 0 ) rf(x ); z x 0g ; >: x +1 = proj f H \W (x 0 ); = 0; 1; 2; : : : ;

11 where, for each = 1; 2; : : : ; N, ad 2N PROXIMAL-TYPE ALGORITHM 11 2N s a gve sequece of postve real umbers s the sequece of errors correspodg to the approxmate solutos of the resolvet equato. Note that f = 0, the y = Res f A (x ) : Theorem 1. Let A : X! 2 X, = 1; 2; : : : ; N, be N maxmal mootoe operators such that Z := T N =1 A 1 (0 ) 6=?. Let f : X! R be a Legedre fucto whch s bouded, uformly Fréchet d eretable ad totally covex o bouded subsets of X. The, for each x 0 2 X, there are sequeces fx g 2N whch satsfy (3.1). If, for each = 1; 2; : : : ; N, lm f!+1 > 0, ad the sequeces of errors 2N X satsfy lm!+1 = 0 ad lm sup!+1 ; y 0, the each such sequece fx g 2N coverges strogly to proj f Z (x 0) as! +1. Proof. Note that dom rf = X because dom f = X ad f s Legedre. Hece t follows from [5, Corollary 3.14(), p. 606] that dom Res f A = X. We beg wth the followg clam. Clam 1: There are sequeces fx g 2N whch satsfy (3.1). As a matter of fact, we wll prove that, for each x 0 2 X, there exsts a sequece fx g 2N whch s geerated by (3.1) wth = 0 for all = 1; 2; : : : ; N ad 2 N. It s obvous that H are closed ad covex sets for ay = 1; 2; : : : ; N. Hece H s also closed ad covex. It s also obvous that W s a closed ad covex set. Let u 2 Z. Sce dom Res f 0 A = X, there exsts y 0; 0 2 X X such that 0 = rf(y 0) rf(x 0 ) y0 = Res f 0 (x 0) ad 0 A 0 2 A y0. Sce A s mootoe, t follows that 0 ; y 0 u 0; whch mples that u 2 H 0. Sce ths holds for ay = 1; 2; : : : ; N, t follows that u 2 H 0. It s also obvous that u 2 W 0 = X. Thus u 2 H 0 \ W 0, ad therefore x 1 = proj f H 0\W 0 (x 0 ) s well de ed. Now suppose that u 2 H 1 \ W 1 for some

12 12 SIMEON REICH AND SHOHAM SABACH 1. Let x = proj f H 1\W 1 (x 0 ). Aga, there exsts y ; 2 X X such that 0 = + 1 rf(y ) rf(x ) ad 2 Ay. The mootocty of A mples that u 2 H. Sce ths holds for ay = 1; 2; : : : ; N, t follows that u 2 H. Now t follows from Proposto 6() that hrf(x 0 ) rf(x ); u x D E = rf(x 0 ) rf proj f H 1\W 1 (x 0 ) ; u proj f H 1\W 1 (x 0 ) 0; whch mples that u 2 W. Therefore u 2 H \W, ad hece x +1 = proj f H \W (x 0 ) s well de ed. Thus the sequece we costructed s deed well de ed ad sats es (3.1), as clamed. From ow o we x a arbtrary sequece fx g 2N satsfyg (3.1). It s clear from the proof of Clam 1 that Z H \ W for each 2 N. Clam 2: The sequece fx g 2N s bouded. It follows from the de to of W ad Proposto 6() that proj f W (x 0 ) = x. Furthermore, by Proposto 6(), for each u 2 Z, we have (3.2) D f (x ; x 0 ) = D f proj f W (x 0 ); x 0 D f (u; x 0 ) D f u; proj f W (x 0 ) D f (u; x 0 ) : Hece the sequece fd f (x ; x 0 )g 2N s bouded by D f (u; x 0 ) for ay u 2 Z. Therefore the sequece f f (x 0 ; kx x 0 k)g 2N s bouded by D f (u; x 0 ), because from the de to of the modulus of total covexty (see (2.2)) ad from (3.2) we get that (3.3) f (x 0 ; kx x 0 k) D f (x ; x 0 ) D f (u; x 0 ):

13 PROXIMAL-TYPE ALGORITHM 13 Sce the fucto f s totally covex, the fucto f (x; ) s strctly creasg ad postve o (0; 1) (cf. Proposto 2(v)). Ths mples, partcular, that f (x; 1) > 0 for all x 2 X. Now suppose by way of cotradcto that the sequece fx g 2N s ot bouded. The there exsts a sequece f k g k2n of postve real umbers such that lm kx k k = +1: k!+1 Cosequetly, lm k!+1 kx k x 0 k = +1. Ths shows that the sequece f f (x 0 ; kx x 0 k)g 2N s ot bouded. Ideed, there exsts some k 0 > 0 such that kx k x 0 k > 1 for ay k > k 0 ad the, by Proposto 2(), we see that f (x 0 ; kx k x 0 k) kx k x 0 k f (x 0 ; 1)! +1; because, as oted above, f (x 0 ; 1) > 0. Ths cotradcts (3.3). Hece the sequece fx g 2N s deed bouded, as clamed. Clam 3: Every weak subsequetal lmt of fx g 2N belogs to Z. It follows from the de to of W ad Proposto 6() that proj f W (x 0 ) = x. Sce x +1 2 W, t follows from Proposto 6() that D f x +1 ; proj f W (x 0 ) + D f proj fw (x 0 ); x 0 D f (x +1 ; x 0 ) ad hece (3.4) D f (x +1 ; x ) + D f (x ; x 0 ) D f (x +1 ; x 0 ) : Therefore the sequece fd f (x ; x 0 )g 2N s creasg ad sce t s also bouded (see Clam 2), lm!+1 D f (x ; x 0 ) exsts. Thus from (3.4) t follows that (3.5) lm!+1 D f (x +1 ; x ) = 0:

14 14 SIMEON REICH AND SHOHAM SABACH Proposto 5 ow mples that lm!+1 (x +1 x ) = 0. For ay = 1; 2; : : : ; N, t follows from the three pot detty (see (2.1)) that D f (x +1 ; x ) D f y ; x = D f x +1 ; y + rf(x ) rf(y); y x +1 rf(x ) rf(y); y x +1 = ; y = ; y x +1 x +1 ; y x +1 ; y x +1 because x +1 2 H. We ow have D f y ; x Df (x +1 ; x ) + ; y x +1 = D f (x +1 ; x ) + ; y ; x +1 D f (x +1 ; x ) + ; y + kx +1 k : Hece lm sup D f!+1 y; x lm sup D f (x +1 ; x )!+1 + lm sup!+1 ; y + lm sup!+1 kx +1 k : Sce lm!+1 = 0, lm sup!+1 ; y 0, ad lm!+1 D f (x +1 ; x ) = 0 (by (3.5)), we see that lm sup!+1 D f y ; x 0. Hece lm!+1 D f y ; x = 0. Proposto 5 ow mples that lm!+1 y x = 0. Now let xj j2n be a weakly coverget subsequece of fx g 2N ad deote ts weak lmt by o v. The y j also coverges weakly to v for ay = 1; 2; : : : ; N. Sce j2n lm f!+1 > 0 ad lm!+1 = 0, t follows from Proposto 1 that (3.6) = 1 rf(x ) rf(y ) +! 0; for ay = 1; 2; : : : ; N. Sce 2 Ay ad A s mootoe, t follows that ; z y 0

15 PROXIMAL-TYPE ALGORITHM 15 for all (z; ) 2 graph (A ). Ths, tur, mples that h; z v 0 for all (z; ) 2 graph (A ). Therefore, usg the maxmal mootocty of A, we ow obta that v 2 A 1 (0 ) for each = 1; 2; : : : ; N. Thus v 2 Z ad ths proves Clam 3. Clam 4: The sequece fx g 2N coverges strogly to proj f Z (x 0). Let ~u = proj f Z (x 0). Sce x +1 = proj f H \W (x 0 ) ad Z s cotaed H \ W, we have D f (x +1 ; x 0 ) D f (~u; x 0 ). The three pot detty (see (2.1)) yelds D f (x ; ~u) = D f (x ; x 0 ) + D f (x 0 ; ~u) hrf(~u) rf(x 0 ); x x 0 D f (~u; x 0 ) + D f (x 0 ; ~u) hrf(~u) rf(x 0 ); x x 0 = hrf(~u) rf(x 0 ); ~u x 0 hrf(~u) rf(x 0 ); x x 0 = hrf(~u) rf(x 0 ); ~u x : Now let fx g 2N be a weakly coverget subsequece of fx g 2N ad deote ts weak lmt by v. We already kow (see Clam 3) that v 2 Z. It follows from Proposto 6() that Hece lm sup D f (x ; ~u) hrf(~u) rf(x 0 ); ~u v 0:!+1 lm!+1 D f (x ; ~u) = 0: Proposto 4 ow mples that x! ~u. It follows that the whole sequece fx g 2N coverges strogly to ~u = proj f Z (x 0), as clamed. Theorem 1. Ths completes the proof of

16 16 SIMEON REICH AND SHOHAM SABACH 4. Zero Free Operators Suppose ow that the operators A, = 1; 2; : : : ; N, have o commo zero. If fx g 2N s a sequece satsfyg (3.1), the lm!+1 kx k = +1. Ths s because f fx g 2N were to have a bouded subsequece, the t would follow from Clam 3 the proof of Theorem 1 that the operators A, = 1; 2; : : : ; N, dd share a commo zero. I the case of a sgle zero free operator A, we ca prove that such a sequece always exsts. To ths ed, we rst recall the dualty mappg of the space X,.e., the mappg J : X! 2 X whch s de ed by Jx = o 2 X : h; x = kxk 2 = kk 2 : We cotue wth the followg lemma. Lemma 1. If A : X! 2 X s a maxmal mootoe operator wth a bouded e ectve doma, the A 1 (0 ) 6=?. Proof. Let f" g 2N be a sequece of postve umbers whch coverges to zero. The operator A + " J s surjectve for ay 2 N because A s a maxmal mootoe operator (see [19, Theorem 3.11, p. 166]). Therefore, for ay 2 N, there exsts x 2 dom A such that 0 2 (A + " J) x. Cosequetly, for ay 2 N, there are 2 Ax ad 2 Jx such that + " = 0. Therefore we have k k = " k k = " kx k! 0 because fx g 2N s a bouded sequece. Hece there exsts a subsequece fx k g k2n of fx g 2N whch coverges weakly to some x 0 2 X. Sce A s mootoe we have (4.1) h k ; v x k 0; k 2 N for ay (v; ) 2 graph A. Lettg k! +1 (4.1), we obta h; v x 0 0 for all (v; ) 2 graph A ad from the maxmalty of A t follows that x 0 2 A 1 (0 ). Hece A 1 (0 ) 6=?, as clamed.

17 PROXIMAL-TYPE ALGORITHM 17 Theorem 2. Let A : X! 2 X be a maxmal mootoe operator. Let f : X! R be a Legedre fucto whch s bouded, uformly Fréchet d eretable ad totally covex o bouded subsets of X. The, for each x 0 2 X, there are sequeces fx g 2N whch satsfy (3.1). If lm f!+1 > 0, ad the sequece of errors f g 2N X sats es lm!+1 = 0 ad lm sup!+1 h ; y 0, the ether A 1 (0 ) 6=? ad each such sequece fx g 2N coverges strogly to proj f A 1 (0 ) (x 0) or A 1 (0 ) =? ad each such sequece fx g 2N sats es lm!+1 kx k = +1. Proof. I vew of Theorem 1, we oly eed to cosder the case where A 1 (0 ) =?. Frst of all we prove that ths case, for each x 0 2 X, there s a sequece fx g 2N whch sats es (3.1) wth = 0 for all 2 N. We prove ths by ducto. We rst check that the tal step ( = 0) s well de ed. The problem 0 2 Ax (rf(x) rf(x 0 )) always has a soluto (y 0 ; 0 ) because t s equvalet to the problem x =Res f (x 0A 0) ad ths problem does have a soluto sce dom Res f A = X (see Proposto 3 ad [5, Theorem 3.13(v), p. 606]). Now ote that W 0 = X. Sce H 0 caot be empty, the ext terate x 1 ca be geerated; t s the Bregma projecto of x 0 oto H 0 = W 0 \ H 0. Note that wheever x s geerated, y ad ca further be obtaed because the proxmal subproblems always have solutos. Suppose ow that x ad (y ; ) have already bee de ed for = 0; : : : ; ^. We have to prove that x^+1 s also well de ed. To ths ed, take ay z 0 2 dom A ad de e = max fky z 0 k : = 0; : : : ; ^g ad 8 >< 0; kx z 0 k + 1 h(x) = >: +1; otherwse:

18 18 SIMEON REICH AND SHOHAM SABACH The h : X! ( 1; +1] s a proper, covex ad lower semcotuous fucto, ts subd s maxmal mootoe (see [28, Theorem 2.13, p. 124]), ad A 0 = A s also maxmal mootoe (see [33]). Furthermore, A 0 (z) = A (z) for all kz z 0 k < + 1: Therefore 2 A 0 y for = 0; : : : ; ^. We coclude that x ad (y ; ) also satsfy the codtos of Theorem 1 appled to the problem 0 2 A 0 (x). Sce A 0 has a bouded e ectve doma, ths problem has a soluto by Lemma 1. Thus t follows from Clam 1 the proof of Theorem 1 that x^+1 s well de ed. Hece the whole sequece fx g 2N s well de ed, as asserted. If fx g 2N were to have a bouded subsequece, the t would follow from Clam 3 the proof of Theorem 1 that A had a zero. Therefore f A 1 (0 ) =?, the lm!+1 kx k = +1, as asserted. Remark 1. I both Theorems 1 ad 2 we ca replace the assumptos that lm f!+1 > 0 ad f s uformly Fréchet d eretable o bouded subsets of X wth the assumpto that lm!+1 = +1. Ths s because ths case frf(x ) rf(y )g 2N s bouded ad therefore (3.6) cotues to hold. 5. Cosequeces of the Strog Covergece Theorem Algorthm (1.4) s a specal case of algorthm (3.1) whe N = 1 ad = 0 for all 2 N. Hece as a drect cosequece of Theorem 1 we obta the followg result (cf. [21]). Corollary 1. Let A : X! 2 X be a maxmal mootoe operator. Let f : X! R be a Legedre fucto whch s bouded, uformly Fréchet d eretable ad totally covex o bouded subsets of X, ad suppose that lm f!+1 > 0. The for each x 0 2 X, the sequece fx g 2N geerated by (1.4) s well de ed,

19 PROXIMAL-TYPE ALGORITHM 19 ad ether A 1 (0 ) 6=? ad fx g 2N coverges strogly to proj f A 1 (0 ) (x 0) as! +1, or A 1 (0 ) =? ad lm!+1 kx k = +1. Notable corollares of Theorems 1 ad 2 occur whe the space X s both uformly smooth ad uformly covex. I ths case the fucto f(x) = 1 2 kxk2 s Legedre (cf. [3, Lemma 6.2, p.24]) ad uformly Fréchet d eretable o bouded subsets of X. Accordg to [13, Corollary 1(), p. 325], f s sequetally cosstet sce X s uformly covex ad hece f s totally covex o bouded subsets of X. Therefore Theorems 1 ad 2 hold ths cotext ad lead us to the followg two results whch, some sese, complemet Theorem 8 [25] (see also Theorem 1 [37]). Corollary 2. Let X be a uformly smooth ad uformly covex Baach space ad let A : X! 2 X be a maxmal mootoe operator. The, for each x 0 2 X, the sequece fx g 2N geerated by (1.3) s well de ed. If lm f!+1 > 0, the ether A 1 (0 ) 6=? ad fx g 2N coverges strogly to proj f A 1 (0 ) (x 0) as! +1, or A 1 (0 ) =? ad lm!+1 kx k = +1. Corollary 3. Let X be a Hlbert space ad let A : X! 2 X be a maxmal mootoe operator. The, for each x 0 2 X, the sequece fx g 2N geerated by (1.2) s well de ed. If lm f!1 > 0, the ether A 1 (0) 6=? ad fx g 2N coverges strogly to P A 1 (0)(x 0 ) as! +1, or A 1 (0) =? ad lm!+1 kx k = +1. These corollares also hold, of course, the presece of computatoal errors as Theorems 1 ad A Applcato of the Strog Covergece Theorem Let g : X! ( 1; +1] be a proper, covex ad lower semcotuous fucto. Recall that the subd of g s de ed for ay x 2 X (x) := f 2 X : h; y x g (y) g (x) 8y 2 Xg :

20 20 SIMEON REICH AND SHOHAM SABACH Usg Theorem 1 ad the subd eretal of g, we obta a algorthm for dg a mmzer of g. Proposto 7. Let g : X! ( 1; +1] be a proper, covex ad lower semcotuous fucto whch attas ts mmum over X. If f : X! R s a Legedre fucto whch s bouded, uformly Fréchet d eretable, ad totally covex o bouded subsets of X, ad f g 2N s a postve sequece wth lm f!+1 > 0, the, for each x 0 2 X, the sequece fx g 2N geerated by 8 x 0 2 X; >< 0 = + 1 (rf(y ) rf(x )) ; (y ) ; H = fz 2 X : h ; z y 0g ; W = fz 2 X : hrf(x 0 ) rf(x ); z x 0g ; >: x +1 = proj f H \W (x 0 ); = 0; 1; 2; : : : ; coverges strogly to a mmzer of g as! +1. If g does ot atta ts mmum over X, the lm!+1 kx k = +1. Proof. The subd of g s a maxmal mootoe operator sce g s a proper, covex ad lower semcotuous fucto (see [28, Theorem 2.13, p. 124]). Sce the zero set cocdes wth the set of mmzers of g, Proposto 7 follows mmedately from Theorems 1 ad 2. Note that ths case y = arg m g (x) + 1 D f (x; x ) x2x s equvalet to 0 (y ) + 1 (rf(y ) rf(x )) :

21 PROXIMAL-TYPE ALGORITHM Ackowledgemet The rst author was partally supported by the Israel Scece Foudato (Grat 647/07), by the Fud for the Promoto of Research at the Techo, ad by the Techo Presdet s Research Fud. Refereces [1] Ambrosett, A. ad Prod, G.: A prmer of olear aalyss, Cambrdge Uversty Press, Cambrdge, [2] Bauschke, H. H. ad Borwe, J. M.: Legedre fuctos ad the method of radom Bregma projectos, J. Covex Aal. 4 (1997), [3] Bauschke, H. H., Borwe, J. M. ad Combettes, P. L.: Essetal smoothess, essetal strct covexty, ad Legedre fuctos Baach spaces, Commu. Cotemp. Math. 3 (2001), [4] Bauschke, H. H. ad Combettes, P. L.: A weak-to-strog covergece prcple for Fejérmootoe methods Hlbert spaces, Math. Oper. Res. 26 (2001), [5] Bauschke, H. H., Borwe, J. M. ad Combettes, P. L.: Bregma mootoe optmzato algorthms, SIAM J. Cotrol Optm. 42 (2003), [6] Bauschke, H. H., Matoušková, E. ad Rech, S.: Projecto ad proxmal pot methods: covergece results ad couterexamples, Nolear Aal. 56 (2004), [7] Boas, J. F. ad Shapro, A.: Perturbato aalyss of optmzato problems, Sprger Verlag, New York, [8] Bregma, L. M.: A relaxato method for dg the commo pot of covex sets ad ts applcato to the soluto of problems covex programmg, USSR Comput. Math. ad Math. Phys. 7 (1967), [9] Brézs, H. ad Los, P.-L.: Produts s de résolvates, Israel J. Math. 29 (1978), [10] Bruck, R. E. ad Rech, S.: Noexpasve projectos ad resolvets of accretve operators, Housto J. Math. 3 (1977), [11] Butaru, D., Cesor, Y. ad Rech, S.: Iteratve averagg of etropc projectos for solvg stochastc covex feasblty problems, Computatoal Optmzato ad Applcatos 8 (1997), [12] Butaru, D. ad Iusem, A. N.: Totally covex fuctos for xed pots computato ad te dmesoal optmzato, Kluwer Academc Publshers, Dordrecht, 2000.

22 22 SIMEON REICH AND SHOHAM SABACH [13] Butaru, D., Iusem, A. N. ad Resmerta, E.: Total covexty for powers of the orm uformly covex Baach spaces, J. Covex Aal. 7 (2000), [14] Butaru, D., Iusem, A. N. ad Z¼alescu, C.: O uform covexty, total covexty ad covergece of the proxmal pot ad outer Bregma projecto algorthms Baach spaces, J. Covex Aal. 10 (2003), [15] Butaru, D. ad Resmerta, E.: Bregma dstaces, totally covex fuctos ad a method for solvg operator equatos Baach spaces, Abstr. Appl. Aal. 2006, Art. ID 84919, [16] Butaru, D. ad Kassay, G.: A proxmal-projecto method for dg zeroes of set-valued operators, SIAM J. Cotrol Optm. 47 (2008), [17] Cesor, Y. ad Let, A.: A teratve row-acto method for terval covex programmg, J. Optm. Theory Appl. 34 (1981), [18] Cesor, Y. ad Zeos, S. A.: Proxmal mmzato algorthm wth D-fuctos, J. Optm. Theory Appl. 73 (1992), [19] Coraescu, I.: Geometry of Baach spaces, dualty mappgs ad olear problems, Kluwer Academc Publshers, Dordrecht, [20] Eckste, J.: Nolear proxmal pot algorthms usg Bregma fuctos, wth applcato to covex programmg, Math. Oper. Res. 18 (1993), [21] Gárcga Otero, R. ad Svater, B. F.: A strogly coverget hybrd proxmal method Baach spaces, J. Math. Aal. Appl. 289 (2004), [22] Güler, O.: O the covergece of the proxmal pot algorthm for covex mmzato, SIAM J. Cotrol Optm. 29 (1991), [23] Kammura, S. ad Takahash, W.: Approxmatg solutos of maxmal mootoe operators Hlbert spaces, J. Approx. Theory 106 (2000), [24] Kammura, S. ad Takahash, W.: Weak ad strog covergece of solutos to accretve operator clusos ad applcatos, Set-Valued Aal. 8 (2000), [25] Kammura, S. ad Takahash, W.: Strog covergece of a proxmal-type algorthm a Baach space, SIAM J. Optm. 13 (2002), [26] Martet, B.: Régularsato d équatos varatoelles par approxmatos successves, Revue Fraçase d Iformatque et de Recherche Opératoelle 4 (1970), [27] Nevala, O. ad Rech, S.: Strog covergece of cotracto semgroups ad of teratve methods for accretve operators Baach spaces, Israel J. Math. 32 (1979), [28] Pascal, D. ad Sburla, S.: Nolear mappgs of mootoe type, Sjtho & Nordho Iteratoal Publshers, Alphe aa de Rj, 1978.

23 PROXIMAL-TYPE ALGORITHM 23 [29] Rech, S.: Weak covergece theorems for oexpasve mappgs Baach spaces, J. Math. Aal. Appl. 67 (1979), [30] Rech, S.: A weak covergece theorem for the alteratg method wth Bregma dstaces, Theory ad applcatos of olear operators of accretve ad mootoe type, Marcel Dekker, New York, 1996, [31] Rech, S. ad Zaslavsk, A. J.: I te products of resolvets of accretve operator, Topol. Methods Nolear Aal. 15 (2000), [32] Resmerta, E.: O total covexty, Bregma projectos ad stablty Baach spaces, J. Covex Aal. 11 (2004), [33] Rockafellar, R. T.: O the maxmalty of sums of olear mootoe operators, Tras. Amer. Math. Soc. 149 (1970), [34] Rockafellar, R. T.: Mootoe operators ad the proxmal pot algorthm, SIAM J. Cotrol Optm. 14 (1976), [35] Rockafellar, R. T.: Augmeted Lagragas ad applcatos of the proxmal pot algorthm covex programmg, Math. Oper. Res. 1 (1976), [36] Rockafellar, R. T. ad Wets, R. J.-B.: Varatoal aalyss, Sprger Verlag, Berl, [37] Solodov, M. V. ad Svater, B. F.: Forcg strog covergece of proxmal pot teratos a Hlbert space, Math. Program. 87 (2000), [38] Yao, J. C. ad Zeg, L. C.: A exact proxmal-type algorthm Baach spaces, J. Optm. Theory Appl. 135 (2007), [39] Z¼alescu, C.: Covex aalyss geeral vector spaces, World Scet c Publshg, Sgapore, Smeo Rech: Departmet of Mathematcs, The Techo Israel Isttute of Techology, Hafa, Israel E-mal address: srech@tx.techo.ac.l Shoham Sabach: Departmet of Mathematcs, The Techo Israel Isttute of Techology, Hafa, Israel E-mal address: ssabach@tx.techo.ac.l