Operatons Research Letters 26 (2000) 127 136 www.elsever.com/locate/orms On the convergence of the bloc nonlnear Gauss Sedel method under convex constrants L. Grppo a, M. Scandrone b; a Dpartmento d Informatca e Sstemstca, Unversta d Roma La Sapenza, Va Buonarrot 12-00185 Roma, Italy b Isttuto d Anals de Sstem ed Informatca del CNR, Vale Manzon 30-00185 Roma, Italy Receved 1 August 1998; receved n revsed form 1 September 1999 Abstract We gve new convergence results for the bloc Gauss Sedel method for problems where the feasble set s the Cartesan product of m closed convex sets, under the assumpton that the sequence generated by the method has lmt ponts. We show that the method s globally convergent for m = 2 and that for m 2 convergence can be establshed both when the objectve functon f s componentwse strctly quasconvex wth respect to m 2 components and when f s pseudoconvex. Fnally, we consder a proxmal pont modcaton of the method and we state convergence results wthout any convexty assumpton on the objectve functon. c 2000 Elsever Scence B.V. All rghts reserved. Keywords: Nonlnear programmng; Algorthms; Decomposton methods; Gauss Sedel method 1. Introducton Consder the problem mnmze f(x) (1) subject to x X = X 1 X 2 X m R n ; where f : R n R s a contnuously derentable functon and the feasble set X s the Cartesan product of closed, nonempty and convex subsets X R n, for =1;:::;m, wth m =1 n = n. If the vector x R n s parttoned nto m component vectors x R n, Ths research was partally supported by Agenza Spazale Italana, Roma, Italy. Correspondng author. Fax: +39-06 7716461. E-mal address: scandro@as.rm.cnr.t (M. Scandrone) then the mnmzaton verson of the bloc-nonlnear Gauss Sedel (GS) method for the soluton of (1) s dened by the teraton: x +1 = arg mn f(x1 +1 ;:::;x 1 +1 ;y ;x+1;:::;x m); y X whch updates n turn the components of x, startng from a gven ntal pont x 0 X and generates a sequence {x } wth x =(x1 ;:::;x m). It s nown that, n general, the GS method may not converge, n the sense that t may produce a sequence wth lmt ponts that are not crtcal ponts of the problem. Some well-nown examples of ths behavor have been descrbed by Powell [12], wth reference to the coordnate method for unconstraned problems, that s to the case m = n and X = R n. 0167-6377/00/$ - see front matter c 2000 Elsever Scence B.V. All rghts reserved. PII: S 0167-6377(99)00074-7
128 L. Grppo, M. Scandrone / Operatons Research Letters 26 (2000) 127 136 Convergence results for the bloc GS method have been gven under sutable convexty assumptons, both n the unconstraned and n the constraned case, n a number of wors (see e.g. [1,3,5,6,9 11,13,15]). In the present paper, by extendng some of the results establshed n the unconstraned case, we prove new convergence results of the GS method when appled to constraned problems, under the assumptons that the GS method s well dened (n the sense that every subproblem has an optmal soluton) and that the sequence {x } admts lmt ponts. More speccally, rst we derve some general propertes of the lmt ponts of the partal updates generated by the GS method and we show that each of these ponts s a crtcal pont at least wth respect to two consecutve components n the gven orderng. Ths s shown by provng that the global mnmum value n a component subspace s lower than the functon value obtanable through a convergent Armjo-type lne search along a sutably dened feasble drecton. As a consequence of these results, we get a smple proof of the fact that n case of a two bloc decomposton every lmt pont of {x } s a crtcal pont of problem (1), even n the absence of any convexty assumpton on f. As an example, we llustrate an applcaton of the two-bloc GS method to the computaton of crtcal ponts of nonconvex quadratc programmng problems va the soluton of a sequence of convex programmng subproblems. Then we consder the convergence propertes of the GS method for the general case of a m-bloc decomposton under generalzed convexty assumptons on the objectve functon. We show that the lmt ponts of the sequence generated by the GS method are crtcal ponts of the constraned problem both when () f s componentwse strctly quasconvex wth respect to m 2 blocs and when () f s pseudoconvex and has bounded level sets n the feasble regon. In case () we get a generalzaton of well nown convergence results [10,5]; n case () we extend to the constraned case the results gven n [14] for the cyclc coordnate method and n [6] for the unconstraned bloc GS method. Usng a constraned verson of a Powell s counterexample, we show also that nonconvergence of the GS method can be demonstrated for nonconvex functons, when m 3 and the precedng assumptons are not satsed. Fnally, n the general case of arbtrary decomposton, we extend a result of [1], by showng that the lmt ponts of the sequence generated by a proxmal pont modcaton of the GS method are crtcal ponts of the constraned problem, wthout any convexty assumpton on the objectve functon. Notaton. We suppose that the vector x R n s parttoned nto component vectors x R n, as x =(x 1 ;x 2 ;:::;x m ). In correspondence to ths partton, the functon value f(x) s also ndcated by f(x 1 ;x 2 ;:::;x m ) and, for =1;2;:::;m the partal gradent of f wth respect to x, evaluated at x, s ndcated by f(x)= f(x 1 ;x 2 ;:::;x m ) R n. A crtcal pont for Problem (1) s a pont x X such that f(x) T (y x) 0; for every y X, where f(x) R n denotes the gradent of f at x. If both x and y are parttoned nto component vectors, t s easly seen that x X s a crtcal pont for Problem (1) f and only f for all =1;:::;mwe have: f(x) T (y x ) 0 for every y X. We denote by L 0 X the level set of f relatve to X, correspondng to a gven pont x 0 X, that s L 0 X :={x X : f(x)6f(x0 )}. Fnally, we ndcate by the Eucldean norm (on the approprate space). 2. A lne search algorthm In ths secton we recall some well-nown propertes of an Armjo-type lne search algorthm along a feasble drecton, whch wll be used n the sequel n our convergence proofs. Let {z } be a gven sequence n X, and suppose that X for =1;:::;m. Let us choose an ndex {1;:::;m}and assume that for all we can compute search drectons z s parttoned as z =(z 1 ;:::;z m), wth z d = w z wth w X ; (2) such that the followng assumpton holds. Assumpton 1. Let {d } be the sequence of search drectons dened by (2). Then: () there exsts a number M 0such that d 6M for all ; () we have f(z ) T d 0 for all.
L. Grppo, M. Scandrone / Operatons Research Letters 26 (2000) 127 136 129 An Armjo-type lne search algorthm can be descrbed as follows. Lne search algorthm (LS) Data: (0; 1), (0; 1). Compute = max {( ) j : f(z1 ;:::;z +( ) j d ;:::;zm) j=0;1;::: 6f(z )+ ( ) j f(z ) T d }: (3) In the next proposton we state some well-nown propertes of Algorthm LS. It s mportant to observe that, n what follows, we assume that {z } s a gven sequence that may not depend on Algorthm LS, n the may not be the result of a lne search along d. However, ths has no substantal eect n the convergence proof, whch can be deduced easly from the nown results (see e.g. [3]). sense that z +1 Proposton 1. Let {z } be a sequence of ponts n X and let {d } be a sequence of drectons such that Assumpton 1 s satsed. Let be computed by means of Algorthm LS. Then: () there exsts a nte nteger j such that =( ) j satses the acceptablty condton (3); () f {z } converges to z and: lm f(z ) f(z1 ;:::;z + d ;:::;zm) =0; (4) then we have lm f(z ) T d =0: (5) 3. Prelmnary results In ths secton we derve some propertes of the GS method that are at the bass of some of our convergence results. Frst, we state the m-bloc GS method n the followng form: 3.1. GS Method Step 0: Gven x 0 X, set =0. Step 1: For = 1;:::;m compute x +1 = arg mn y X f(x +1 1 ;:::;y ;:::;x m): (6) Step 2: Set x +1 =(x1 +1 ;:::;xm +1 ), = + 1 and go to Step 1. Unless otherwse speced, we assume n the sequel that the updatng rule (6) s well dened, and hence that every subproblem has solutons. We consder, for all, the partal updates ntroduced by the GS method by denng the followng vectors belongng to X : w(; 0) = x ; w(; )=(x1 +1 ;:::;x 1 +1 ;x+1 ;x+1;:::;x m) =1;:::;m 1; w(; m)=x +1 : For convenence we set also w(; m +1)=w(+1;1): By constructon, for each {1;:::;m}, t follows from (6) that w(; ) s the constraned global mnmzer of f n the th component subspace, and therefore t satses the necessary optmalty condton: f(w(; )) T (y x +1 ) 0 for every y X : (7) We can state the followng propostons. Proposton 2. Suppose that for some {0;:::;m} the sequence {w(; )} admts a lmt pont w. Then; for every j {0;:::;m} we have lm f(w(; j)) = f( w): Proof. Let us consder an nnte subset K {0;1; :::;} and an ndex {0;:::;m} such that the subsequence {w(; )} K converges to a pont w. Bythe nstructons of the algorthm we have f(w( +1;))6f(w(; )): (8) Then, the contnuty of f and the convergence of {w(; )} K mply that the sequence {f(w(; ))} has a subsequence convergng to f( w). As {f(w(; ))} s nonncreasng, ths, n turn, mples that {f(w(; ))} s bounded from below and converges to f( w). Then, the asserton follows mmedately from the fact that f(w( +1; ))6f(w( +1; j))6f(w(; )) for 06j6;
130 L. Grppo, M. Scandrone / Operatons Research Letters 26 (2000) 127 136 and f(w( +2;))6f(w( +1;j))6f(w( +1;)) for j6m: Proposton 3. Suppose that for some {1;:::;m} the sequence {w(; )} admts a lmt pont w. Then we have f(w) T (y w ) 0 for every y X (9) and moreover f( w) T (y w ) 0 for every y X ; (10) where = (mod m)+1. Proof. Let {w(; )} K be a subsequence convergng to w. From (7), tang nto account the contnuty assumpton on f, we get mmedately (9). In order to prove (10), suppose rst {1;:::;m 1}, so that = + 1. Reasonng by contradcton, let us assume that there exsts a vector ỹ +1 X +1 such that +1 f(w) T (ỹ +1 w +1 ) 0: (11) Then, lettng d+1 =ỹ +1 w(; ) +1 =ỹ +1 x+1 as {w(; )} K s convergent, we have that the sequence {d+1 } K s bounded. Recallng (11) and tang nto account the contnuty assumpton on f t follows that there exsts a subset K 1 K such that +1 f(w(; )) T d+1 0 for all K 1 ; and therefore the sequences {w(; )} K1 and {d+1 } K 1 are such that Assumpton 1 holds, provded that we dentfy {z } wth {w(; )} K1. Now, for all K 1 suppose that we compute +1 by means of Algorthm LS; then we have f(x1 +1 ;:::;x +1 ;x+1+ +1d +1;:::;x m)6f(w(; )): Moreover, as x+1 X +1, x+1 + d +1 X +1, +1 (0; 1], and X +1 s convex, t follows that x+1 + +1d +1 X +1 : Therefore, recallng that f(w(; + 1)) = mn f(x1 +1 ;:::;x +1 ;y +1 ;:::;x y +1 X +1 m); we can wrte f(w(; + 1)) 6f(x1 +1 ;:::;x +1 ;x+1 + +1d +1;:::;x m) 6f(w(; )): (12) By Proposton 2 we have that the sequences {f(w(; j))} are convergent to a unque lmt for all j {0::::;m}, and hence we obtan lm f(w(; )) f( x1 +1 ;:::;x +1 ;x+1 ; K 1 ++1d +1;:::;x m)=0: Then, nvong Proposton 1, where we dentfy {z } wth {w(; )} K1, t follows that +1 f(w) T (ỹ +1 w +1 )=0; whch contradcts (11), so that we have proved that (10) holds when {1;:::;m 1}. When = m, so that = 1, we can repeat the same reasonngs notng that w(; m +1)=w(+1;1). The precedng result mples, n partcular, that every lmt pont of the sequence {x } generated by the GS method s a crtcal pont wth respect to the components x 1 and x m n the prexed orderng. Ths s formally stated below. Corollary 1. Let {x } be the sequence generated by the GS method and suppose that there exsts a lmt pont x. Then we have 1 f(x) T (y 1 x 1 ) 0 for every y 1 X 1 (13) and m f(x) T (y m x m ) 0: for every y m X m : (14) 4. The two-bloc GS method Let us consder the problem: mnmze f(x)=f(x 1 ;x 2 ); x X 1 X 2 (15) under the assumptons stated n Secton 1. We note that n many cases a two-bloc decomposton can be useful snce t may allow us to employ parallel technques for solvng one subproblem. As an example,
L. Grppo, M. Scandrone / Operatons Research Letters 26 (2000) 127 136 131 gven a functon of the form N f(x)= 1 (x 1 )+ 1(x 1 ) (x ) =2 f we decompose the problem varables nto the two blocs x 1 and (x 2 ;:::;x N ), then once x 1 s xed, the objectve functon can be mnmzed n parallel wth respect to the components x for =2;:::;N. When m=2, a convergence proof for the GS method (2Bloc GS method) n the unconstraned case was gven n [6]. Here the extenson to the constraned case s an mmedate consequence of Corollary 1. Corollary 2. Suppose that the sequence {x } generated by the 2Bloc GS method has lmt ponts. Then; every lmt pont x of {x } s a crtcal pont of Problem (15). As an applcaton of the precedng result we consder the problem of determnng a crtcal pont of a nonlnear programmng problem where the objectve functon s a nonconvex quadratc functon and we have dsjont constrants on two derent blocs of varables. In some of these cases the use of the two-bloc GS method may allow us to determne a crtcal pont va the soluton of a sequence of convex programmng problems of a specal structure and ths may consttute a basc step n the context of cuttng plane or branch and bound technques for the computaton of a global optmum. As a rst example, we consder a blnear programmng problem wth dsjont constrants and we reobtan a slghtly mproved verson of a result already establshed n [7] usng derent reasonngs. Consder a blnear programmng problem of the form: mnmze f(x 1 ;x 2 )=x1qx T 2 + c1 T x 1 + c2 T x 2 subject to A 1 x 1 = b 1 ;x 1 0; (16) A 2 x 2 =b 2 ;x 2 0; where x 1 R n1 and x 2 R n2. As shown n [8], problems of ths form can be obtaned, for nstance, as an equvalent reformulaton on an extended space of concave quadratc programmng problems. Suppose that the followng assumptons are sats- ed: () the sets X 1 = {x 1 R n1 : A 1 x 1 = b 1 ;x 1 0}and X 2 ={x 2 R n2 : A 2 x 2 =b 2 ;x 2 0}are non empty; () f(x 1 ;x 2 ) s bounded below on X = X 1 X 2. Note that we do not assume, as n [7] that X 1 and X 2 are bounded. Startng from a gven pont (x1 0;x0 2 ) X, the two-bloc GS method conssts n solvng a sequence of two lnear programmng subproblems. In fact, gven (x1 ;x +1 2 ), we rst obtan a soluton x1 of the problem mnmze (Qx2 + c 1 ) T x 1 (17) subject to A 1 x 1 = b 1 ;x 1 0 and then we solve for x2 +1 the problem mnmze (Q T x1 +1 + c 2 ) T x 2 (18) subject to A 2 x 2 = b 2 ;x 2 0: Under the assumpton stated, t s easly seen that problems (17) and (18) have optmal solutons and hence that the two bloc GS method s well dened. In fact, reasonng by contradcton, assume that one subproblem, say (17), does not admt an optmal soluton. As the feasble set X 1 s nonempty, ths mples that the objectve functon s unbounded from below on X 1. Thus there exsts a sequence of ponts z j X 1 such that lm (Qx 2 + c 1 ) T z j = j and therefore, as x2 s xed, we have also lm f(z j ;x2) = lm (Qx 2 + c 1 ) T z j + c2 T x2 = : j j But ths would contradct assumpton (), snce (z j ;x2 ) s feasble for all j. We can also assume that x1 and x 2 are vertex solutons, so that the sequence {(x1 ;x 2 )} remans n a - nte set. Then, t follows from Corollary 2 that the two bloc-gs method must determne n a nte number of steps a crtcal pont of problem (16). As a second example, let us consder a (possbly nonconvex) problem of the form mnmze f(x 1 ;x 2 )= 1 2 xt 1 Ax 1 + 1 2 xt 2 Bx 2 + x1 T Qx 2 subject to x 1 X 1 ; x 2 X 2 ; +c T 1 x 1 + c T 2 x 2 (19) where X 1 R n1 ;X 2 R n2 are nonempty closed convex sets, and the matrces A; B are symmetrc and semdefnte postve.
132 L. Grppo, M. Scandrone / Operatons Research Letters 26 (2000) 127 136 Suppose that one of the followng assumptons s vered: () X 1 and X 2 are bounded; () A s postve dente and X 2 s bounded. Under ether one of these assumptons, t s easly seen that the level set L 0 X s compact for every (x0 1 ;x0 2 ) X 1 X 2. Ths mples that the two-bloc GS method s well dened and that the sequence {x } has lmt ponts. Then, agan by Corollary 2, we have that every lmt pont of ths sequence s a crtcal pont of problem (19). 5. The bloc GS method under generalzed convexty assumptons In ths secton we analyze the convergence propertes of the bloc nonlnear Gauss Sedel method n the case of arbtrary decomposton. In partcular, we show that n ths case the global convergence of the method can be ensured assumng the strct componentwse quasconvexty of the objectve functon wth respect to m 2 components, or assumng that the objectve functon s pseudoconvex and has bounded level sets. We state formally the noton of strct componentwse quasconvexty that follows mmedately from a nown denton of strct quasconvexty [10], whch sometmes s called also strong quasconvexty [2]. Denton 1. Let {1;:::;m}; we say that f s strctly quasconvex wth respect to x X on X f for every x X and y X wth y x we have f(x 1 ;:::;tx +(1 t)y ;:::;x m ) max{f(x);f(x 1 ;:::;y ;:::;x m )} for all t (0; 1): We can establsh the followng proposton, whose proof requres only mnor adaptatons of the arguments used, for nstance, n [10,5]. Proposton 4. Suppose that f s a strctly quasconvex functon wth respect to x X on X n the sense of Denton 1. Let {y } be a sequence of ponts n X convergng to some y X and let {v } be a sequence of vectors whose components are dened as follows: v j = { y j f j ; arg mn X f(y1 ;:::;;:::;ym) f j = : Then; f lm f(y ) f(v )=0; we have lm v y =0. Then, we can state the followng proposton. Proposton 5. Suppose that the functon f s strctly quasconvex wth respect to x on X; for each = 1;:::;m 2 n the sense of Denton 1 and that the sequence {x } generated by the GS method has lmt ponts. Then; every lmt pont x of {x } s a crtcal pont of Problem (1). Proof. Let us assume that there exsts a subsequence {x } K convergng to a pont x X. From Corollary 1weget m f(x) T (y m x m ) 0 for every y m X m : (20) Recallng Proposton 2 we can wrte lm f(w(; )) f(x )=0 for =1;:::;m: Usng the strct quasconvexty assumpton on f and nvong Proposton 4, where we dentfy {y } wth {x } K and {v } wth {w(; 1)} K, we obtan lm ; K w(; 1) = x. By repeated applcaton of Proposton 4 to the sequences {w(; 1)} K and {w(; )} K, for =1;:::;m 2, we obtan lm w(; )= x ; K for =1;:::;m 2: Then, Proposton 3 mples f(x) T (y x ) 0 for every y X ; =1;:::;m 1: (21) Hence, the asserton follows from (20) and (21). In the next proposton we consder the case of a pseudoconvex objectve functon. Proposton 6. Suppose that f s pseudoconvex on X and that L 0 X s compact. Then; the sequence {x } generated by the GS method has lmt ponts and every lmt pont x of {x } s a global mnmzer of f. Proof. Consder the partal updates w(; ), wth = 0;:::;m, dened n Secton 3. By denton of w(; )
L. Grppo, M. Scandrone / Operatons Research Letters 26 (2000) 127 136 133 we have f(x +1 )6f(w(; ))6f(x ) for =0;:::;m. Then, the ponts of the sequences {w(; )}, wth = 0;:::;m, belong to the compact set L 0 X. Therefore, f x X s a lmt pont of {x } we can construct a subsequence {x } K such that lm ; K x =x=w 0 ; (22) lm w(; )= ; K w =1;:::;m; (23) where w X for =1;:::;m. We can wrte w(; )=w(; 1) + d(; ) for =1;:::;m; (24) where the bloc components d h (; ) R n h of the vector d(; ), wth h {1;:::;m}, are such that d h (; )= 0fh. Therefore, for =1;:::;m, from (22) (24) we get w =w 1 +d ; (25) where d = lm d(; ) (26) ; K and d h =0; h : (27) By Proposton 2 we have f(x)=f(w ) for =1;:::;m: (28) From Proposton 3 t follows, for =1;:::;m, f(w ) T (y w ) 0 for all y X ; (29) and f( w ) T (y w ) 0 for all y X ; (30) where = (mod m) + 1. Now we prove that, gven j; {1;:::;m} such that f(w j ) T (y w j ) 0 for all y X ; (31) then t follows f(w j 1 ) T (y w j 1 ) 0 for all y X : (32) Obvously, (32) holds f = j (see (30)). Therefore, let us assume j. By (25) (27) we have w j =w j 1 +d j ; (33) where d j h = 0 for h j. For any gven vector R n such that w j 1 + X ; let us consder the feasble pont z()= w j 1 +d(); where d h ()=0 for h and d ()= R n. Then, from (29) and (31), observng that (33) and the fact that j mply = z () w j 1 = z () w j ; we obtan f( w j ) T (z() w j ) = f(w j ) T (w j 1 +d() w j ) = j f(w j ) T (w j 1 j w j j )+ f(w j ) T = j f(w j ) T (w j 1 j w j j )+ f(w j ) T (z () w j ) 0: It follows by the pseudoconvexty of f that f(z()) f( w j ) for all R n such that w j 1 + X : On the other hand, f( w j )=f(w j 1 ), and therefore we have: f(z()) f( w j 1 ) for all R n such that w j 1 + X ; whch, recallng the denton of z(), mples (32). Fnally, tang nto account (30), and usng the fact that (31) mples (32), by nducton we obtan j f(w 0 ) T (y j w 0 j)= j f(x) T (y j x j ) 0 for all y j X j : Snce ths s true for every j {1;:::;m}, the thess s proved. As an example, let us consder a quadratc programmng problem wth dsjont constrants, of the form mnmze f(x)= m =1 m x T Q j x j + j=1 subject to A x b ; =1;:::;m; m c T x (34) =1 where we assume that: () the sets X = {x R n : A x b }, for =1;:::;m are non empty and bounded;
134 L. Grppo, M. Scandrone / Operatons Research Letters 26 (2000) 127 136 () the matrx Q = Q 11 ::: Q 1m ::: ::: Q m1 ::: Q mm s symmetrc and semdente postve. In ths case the functon f s convex and the level set L 0 X s compact for every feasble x0, but the objectve functon may be not componentwse strctly quasconvex. In spte of ths, the m-bloc GS method s well dened and, as a result of Proposton 6, we can assert that t converges to an optmal soluton of the constraned problem, through the soluton of a sequence of convex quadratc programmng subproblems of smaller dmenson. 6. A counterexample In ths secton we consder a constraned verson of a well-nown counterexample due to Powell [12] whch ndcates that the results gven n the precedng sectons are tght n some sense. In fact, ths example shows that the GS method may cycle ndentely wthout convergng to a crtcal pont f the number m of blocs s equal to 3 and the objectve f s a nonconvex functon, whch s componentwse convex but not strctly quasconvex wth respect to each component. The orgnal counterexample of Powell conssts n the unconstraned mnmzaton of the functon f : R 3 R, dened by f(x)= x 1 x 2 x 2 x 3 x 1 x 3 +(x 1 1) 2 + where (t c) 2 + = +( x 1 1) 2 + +(x 2 1) 2 + +( x 2 1) 2 + +(x 3 1) 2 + +( x 3 1) 2 +; (35) { 0 f t6c; (t c) 2 f t c: Powell showed that, f the startng pont x 0 s the pont ( 1 ; 1+ 1 2 ; 1 1 4) the steps of the GS method tend to cycle round sx edges of the cube whose vertces are (±1; ±1; ±1), whch are not statonary ponts of f. It can be easly vered that the level sets of the objectve functon (35) are not compact; n fact, settng x 2 = x 3 = x 1 we have that f(x) as x. However, the same behavor evdenced by Powell s obtaned f we consder a constraned problem wth the same objectve functon (35) and a compact feasble set, dened by the box constrants M6x 6M =1;:::;3 wth M 0 sucently large. In accordance wth the results of Secton 3, we may note that the lmt ponts of the partal updates generated by the GS method are such that two gradent components are zero. Nonconvergence s due to the fact that the lmt ponts assocated to consecutve partal updates are dstnct because of the fact that the functon s not componentwse strctly quasconvex; on the other hand, as the functon s not pseudoconvex, the lmt ponts of the sequence {x } are not crtcal ponts. Note that n the partcular case of m = 3, by Proposton 5 we can ensure convergence by requrng only the strct quasconvexty of f wth respect to one component. 7. A proxmal pont modcaton of the GS method In the precedng sectons we have shown that the global convergence of the GS method can be ensured ether under sutable convexty assumptons on the objectve functon or n the specal case of a two-bloc decomposton. Now, for the general case of nonconvex objectve functon and arbtrary decomposton, we consder a proxmal pont modcaton of the Gauss Sedel method. Proxmal pont versons of the GS method have been already consdered n the lterature (see e.g. [1,3,4]), but only under convexty assumptons on f. Here we show that these assumptons are not requred f we are nterested only n crtcal ponts. The algorthm can be descrbed as follows. PGS method Step 0: Set =0;x 0 X; 0 for =1;:::;m.. Step 1: For =1;:::;m set: x +1 = arg mn y X { f(x +1 1 ;:::;y ;:::;x m) + 1 2 y x 2 }: (36) Step 2: Set x +1 =(x1 +1 ;:::;xm +1 ); =+ 1 and go to Step 1.
L. Grppo, M. Scandrone / Operatons Research Letters 26 (2000) 127 136 135 The convergence propertes of the method can be establshed by employng essentally the same arguments used n [1] n the convex cases and we can state the followng proposton, whose proof s ncluded here for completeness. Proposton 7. Suppose that the PGS method s well dened and that the sequence {x } has lmt ponts. Then every lmt pont x of {x } s a crtcal pont of Problem (1). Proof. Let us assume that there exsts a subsequence {x } K convergng to a pont x X. Dene the vectors w(; 0) = x ; w(; )=(x1 +1 ;:::;x +1 ;x+1;:::;x m) for =1;:::;m: Then we have f( w(; )) 6f( w(; 1)) 1 2 w(; ) w(; 1) 2 ; (37) from whch t follows f(x +1 ) 6f( w(; ))6f( w(; 1))6f(x ) for =1;:::;m: (38) Reasonng as n Proposton 2 we obtan lm f(x +1 ) f(x )=0; and hence, tang lmts n (37) for we have lm w(; ) w(; 1) =0; =1;:::;m; (39) whch mples lm ; K w(; )= x; =0;:::;m: (40) Now, for every j {1;:::;m},asxj +1 accordng to rule (36), the pont w(; j)=(x1 +1 ;:::;xj +1 ;:::;xm) satses the optmalty condton [ j f( w(; j)) + j ( w j (; j) w j (; j 1))] T (y j w j (; j)) 0 for all y j X j : s generated Then, tang the lmt for ; K, recallng (39), (40) and the contnuty assumpton on f, for every j {1;:::;m} we obtan j f(x) T (y j x j ) 0 for all y j X j ; whch proves our asserton. Tang nto account the results of Secton 5, t follows that f the objectve functon f s strctly quasconvex wth respect to some component x, wth {1;:::;m}, then we can set = 0. Moreover, reasonng as n the proof of Proposton 5, we can obtan the same convergence results f we set m 1 = m =0. As an applcaton of Proposton 7, let us consder the quadratc problem (34) of Secton 5. Suppose agan that the sets X are nonempty and bounded, but now assume that Q s an arbtrary symmetrc matrx. Then the objectve functon wll be not pseudoconvex, n general, and possbly not componentwse strctly quasconvex. In ths case, the bloc-gs method may not converge. However, the PGS method s well de- ned and, moreover, f we set 2 mn (Q ), then the subproblems are strctly convex and the sequence generated by the PGS method has lmt ponts that are crtcal ponts of the orgnal problem. Acnowledgements The authors are ndebted to the anonymous revewer for the useful suggestons and for havng drawn ther attenton to some relevant references. References [1] A. Auslender, Asymptotc propertes of the Fenchel dual functonal and applcatons to decomposton problems, J. Optm. Theory Appl. 73 (3) (1992) 427 449. [2] M.S. Bazaraa, H.D. Sheral, C.M. Shetty, n: Nonlnear Programmng, Wley, New Yor, 1993. [3] D.P. Bertseas, n: Nonlnear Programmng, Athena Scentc, Belmont, MA, 1995. [4] D.P. Bertseas, P. Tseng, Partal proxmal mnmzaton algorthms for convex programmng, SIAM J. Optm. 4 (3) (1994) 551 572. [5] D.P. Bertseas, J.N. Tstsls, n: Parallel and Dstrbuted Computaton, Prentce-Hall, Englewood Cls, NJ, 1989. [6] L. Grppo, M. Scandrone, Globally convergent bloc-coordnate technques for unconstraned optmzaton, Optm. Methods Software 10 (1999) 587 637.
136 L. Grppo, M. Scandrone / Operatons Research Letters 26 (2000) 127 136 [7] H. Konno, A cuttng plane algorthm for solvng blnear programs, Math. Programmng 11 (1976) 14 27. [8] H. Konno, Maxmzaton of a convex quadratc functon under lnear constrants, Math. Programmng 11 (1976) 117 127. [9] Z.Q. Luo, P. Tseng, On the convergence of the coordnate descent method for convex derentable mnmzaton, J. Optm. Theory Appl. 72 (1992) 7 35. [10] J.M. Ortega, W.C. Rhenboldt, n: Iteratve Soluton of Nonlnear Equatons n Several Varables, Academc Press, New Yor, 1970. [11] M. Patrsson, Decomposton methods for derentable optmzaton problems over Cartesan product sets, Comput. Optm. Appl. 9 (1) (1998) 5 42. [12] M.J.D. Powell, On search drectons for mnmzaton algorthms, Math. Programmng 4 (1973) 193 201. [13] P. Tseng, Decomposton algorthm for convex derentable mnmzaton, J. Optm. Theory Appl. 70 (1991) 109 135. [14] N. Zadeh, A note on the cyclc coordnate ascent method, Management Sc. 16 (1970) 642 644. [15] W.I. Zangwll, n: Nonlnear Programmng: A Uned Approach, Prentce-Hall, Englewood Cls, NJ, 1969.