496 B.S. HE, S.L. WANG AND H. YANG where w = x y 0 A ; Q(w) f(x) AT g(y) B T Ax + By b A ; W = X Y R r : (5) Problem (4)-(5) is denoted as MVI

Transcription

1 Journal of Computational Mathematics, Vol., No.4, 003, A MODIFIED VARIABLE-PENALTY ALTERNATING DIRECTIONS METHOD FOR MONOTONE VARIATIONAL INEQUALITIES Λ) Bing-sheng He Sheng-li Wang (Department of Mathematics, Nanjing University, Nanjing 0093, China) Hai Yang (Department of Civil Engineering, The Hong Kong University of Science & Technology, Clear Water Bay, Kowloon, Hong Kong) Abstract Alternating directions method is one of the approaches for solving linearly constrained separate monotone variational inequalities. Experience on applications has shown that the number of iteration significantly depends on the penalty for the system of linearly constrained equations and therefore the method with variable penalties is advantageous in practice. In this paper, we extend the Kontogiorgis and Meyer method [] by removing the monotonicity assumption on the variable penalty matrices. Moreover, we introduce a self-adaptive rule that leads the method to be more efficient and insensitive for various initial penalties. Numerical results for a class of Fermat-Weber problems show that the modified method and its self-adaptive technique are proper and necessary in practice. Key words: Monotone variational inequalities, Alternating directions method, Fermat- Weber problem.. Introduction The mathematical form of variational inequalities consists of finding a vector u Λ Ω such that VI(Ω;F) (u u Λ ) T F (u Λ ) 0; 8 u Ω; () where Ω is a nonempty, closed convex subset of R l, F is a continuous mapping from R l to itself. In practice, many VI problems have the following separable structure, namely (e.g., [4]), x f(x) u = ; F (u) = ; () y g(y) Ω=f(x; y)jx X;y Y;Ax+ By = bg; (3) where X ρr n and YρR m are given closed convex sets, f : X!R n, g : Y!R m are given monotone operators, A R r n ;B R r m are given matrices, and b R r is a given vector. By attaching a Lagrange multiplier vector R r to the linear constraints Ax + By = b, the problem under consideration can be explained as a mixed variational inequality (VI with equality restriction Ax + By = b and unrestricted variable ): Find w Λ W; such that (w w Λ ) T Q(w Λ ) 0; 8 w W; (4) Λ Received March 5, 00. ) The first author was supported by the NSFC grant 07054, the third author was supported inpart by the Hong Kong Research Grants Council through a RGC-CERG Grant (HKUST603/99E).

2 496 B.S. HE, S.L. WANG AND H. YANG where w = x y 0 A ; Q(w) f(x) AT g(y) B T Ax + By b A ; W = X Y R r : (5) Problem (4)-(5) is denoted as MVI(W;Q) and will be concerned in this paper. It has been well nown (e.g., see [4]) that solving MVI(W;Q) is equivalent to finding a zero point of e(w) :=w P W [w Q(w)]; (6) where P W ( ) denotes the projection on W. e(w) can be viewed as a `error bound' that measures how much w fails to be a solution of MVI(W;Q). As a tool for solving MVI(W;Q) problems, the alternating directions method was originally proposed by Gabay [5] and Gabay and Mercier [4]. At each iteration of this method, the new iterate w + =(x + ;y + ; + ) X Y R r is generated from a given triple w = (x ;y ; ) X Y R r by the following procedure: First, x + is obtained (with y and held fixed) by solving (x 0 x + ) T f(x + ) A T [ fi(ax + + By b)] and then y + is produced (with x + and held fixed) by solving (y 0 y + ) T g(y + ) B T [ fi(ax + + By + b)] Finally, the multipliers are updated by 0; 8 x 0 X; (7) 0; 8 y 0 Y: (8) + = flfi(ax + + By + b); (9) where fl (0; +p 5 ) and fi > 0 are given constants. This method is referred to as a method of multiplier in the literature [5], and the convergence proof can be found in [4, 6] (for B = I) and [3, 5] (for general B). Further studies and applications of such methods can be found in Glowinsi [6], Glowinsi and Le Tallec [7], Ecstein and Fuushima [] and He and Yang [9]. Experience on applications [, 3, ] has shown that if the fixed penalty fi is chosen too small or too large the solution time can significantly increase. In order to improve such methods, recently, Kontogiorgis and Meyer [] presented a more general alternating directions method, in which they too a sequence of symmetric positive definite (spd) penalty matrices fh g instead of the constant penalty fi. The convergence of their method was proved under the assumption that the eigenvalues of fh g are uniformly bounded from below away from zero, and, with finitely many exceptions, the eigenvalues of H H + are nonnegative. In this paper, we continue the Kontogiorgis and Meyer's research[] and present a modified variable-penalty alternating directions method that allows the eigenvalues of fh g either to increase or to decrease in each iteration. This can be beneficial in applications. In addition, similarly as in [0], we propose a self-adaptive adjusting rule that leads the method to be more advantageous in practice. The following notation is used in this paper. We denote by I n n the identity matrix in R n n. For any real matrix M and vector v, we denote the transposition by M T and v T, respectively. The notation M ν 0 means that M is a positive semi-definite matrix, and M χ 0 means that M is a positive definite matrix. Superscripts such asinv refer to specific vectors and are usually iteration indices. The Euclidean norm of vector z will be denoted by z, i.e., z = p z T z.. The General Structure of the Modified Method Throughout this paper, we call the method bykontogiorgis and Meyer [] and our modified method ADM method and MADM method, respectively. To describe the MADM method, we need a non-negative sequence f g that satisfies P =0 <.

3 A Modified Variable-penalty Alternating Directions Method for VI 497 Modified Variable-penalty Alternating Directions Method (short MADM) P Step 0. Given ">0, fl (0; +p 5 ), a non-negative sequence f g satisfying =0 <, a symmetric positive definite matrix (spd) H 0, y 0 Y and 0 R r. Set =0. Step. Convergence verification ( ) If e(w ) <", stop; Step. Find x + X(with fixed y and ), such that where (x 0 x + ) T f (x + ) 0; 8 x 0 X: (0) f (x) =f(x) A T [ H (Ax + By b)]: () Step 3. Find y + Y(with fixed x + and ), such that where Step 4. Update (y 0 y + ) T g (y + ) 0; 8 y 0 Y: () g (y) =g(y) B T [ H (Ax + + By b)]: (3) + = flh (Ax + + By + b): (4) Step 5. Adjust the penalty matrix H ( ) such that, Set := +, and go to Step. + H μ H + μ ( + )H : (5) Remar. In the ADM method [], the restriction on matrix sequence fh g can be equivalently described as + H μ H + μ H ; 8 0 (with 0 and X = < ): (6) In comparison with (5) and (6), the MADM method does not need the monotonicity assumption and allows the sequence fh g be more flexible. Furthermore, instead of fl in[], we relax fl (0; +p 5 ). Remar. There are various ways to construct such spd matrix P H satisfying Q (5) and we will show some of them in Section 5. Since i is non-negative and i=0 i <, i= ( + i)is convergent and greater than zero. It follows form (5) that the matrix sequence fh g is both upper and below (from away from zero) bounded. Also from (5) we have + H» H +» ( + ) H ;» + H H +» ( + ) : H Remar 3. For bounded fh g,itcanbeshown there is a constant c 0 > 0, such that e(w + )» c 0 Ax + + By + b + B(y y + ) : (7) Since solving MVI(W;Q) is equivalent to finding a zero point ofe(w), we need only to prove that lim! Ax + + By + b + B(y y + ) =0: In fact, it is easy from (0) -(4) to chec, if Ax + + By + b =0andB(y y + )=0, then w + =(x + ;y + ; + ) is a solution of MVI(W;Q).

4 498 B.S. HE, S.L. WANG AND H. YANG For convenience, we mae some basic assumptions to guarantee that the problem under consideration is solvable and the MADM method is well defined. Assumption A. The solution set of MVI(W;Q), denoted by W Λ, is nonempty. Assumption B. Problems (0) and () are solvable. 3. Some Preparations for the Convergence Analysis In this section, we do some preparations for the convergence analysis. We will investigate the difference between Λ H + flb(y y Λ ) H and + Λ H Now, let us first observe the difference of Λ H the identity Λ H we get + Λ H Λ H + H = + Λ H + flb(y + y Λ ) H : and + Λ. Using (4) and H +( Λ ) T H ( + ); fl Ax + + By + b H +fl( Λ ) T (Ax + + By + b): (8) The following lemma provides a desirable property of the last term of (8). Lemma. For any w Λ =(x Λ ;y Λ ; Λ ) W Λ, we have and ( Λ ) T (Ax + + By + b) Ax + + By + b H +(Ax + Ax Λ ) T H (By By + ): (9) Proof. Since w Λ W Λ ; x + X and y + Y,wehave On the other hand, from (0) and (), it follows that and (x + x Λ ) T (f(x Λ ) A T Λ ) 0 (0) (y + y Λ ) T (g(y Λ ) B T Λ ) 0: () (x Λ x + ) T f(x + ) A T [ H (Ax + + By b)] 0; () (y Λ y + ) T g(y + ) B T [ H (Ax + + By + b)] 0: (3) Adding (0) and (), and using the monotonicity of operator f, we get (x + x Λ ) T A T [( Λ ) H (Ax + + By b)] 0: (4) Similarly, adding () and (3), and using the monotonicity of operator g, it follows that (y + y Λ ) T B T [( Λ ) H (Ax + + By + b)] 0: (5) Combining (4) and (5) and using Ax Λ + By Λ = b, we get the assertion of this lemma. From Lemma, we have Λ H = + Λ H + fl( fl)ax + + By + b H +fl(ax + Ax Λ ) T H (By By + ): (6)

5 A Modified Variable-penalty Alternating Directions Method for VI 499 In addition, we have the identity flb(y y Λ ) H flb(y + y Λ ) H + flby By + H Thus, combining (6) and (7) and using Ax Λ + By Λ = b we get +fl(by + By Λ ) T H (By By + ): (7) Λ + flb(y y Λ ) H H = + Λ + flb(y + y Λ ) H H +fl( fl)ax + + By + b H + flb(y y + ) H +fl(ax + + By + b) T H (By By + ): (8) In the following lemma, we observe the last term in (8). Lemma. For we have (Ax + + By + b) T H (By By + ) ( fl)(ax + By b) T H (By By + ): (9) Proof. By setting y 0 = y in () we get (y y + ) T g(y + ) B T [ H (Ax + + By + b)] Similarly, taing := and y 0 = y + in () we have (y + y ) T g(y ) B T [ H (Ax + By b)] By adding (30) and (3) and using the monotonicity of operator g, we obtain (y + y ) T B T [ H (Ax + + By + b)] [ H (Ax + By b)] 0: (30) 0: (3) 0: (3) Substituting = flh (Ax + By b) in (3), the assertion of this lemma follows immediately. 4. The Main Theorem and the Convergence Proof Remember that we restrict fl (0; +p 5 ) and hence + fl fl > 0. Let and then we have T = 3 ( + fl fl ); (33) T = 3 ( + fl fl ) > 0 and T fl = 3 (fl 4fl +5)= 3 [(fl ) +]> 3 : Now we are in the stage to prove the main theorem of this paper. Theorem. Let w Λ = (x Λ ;y Λ ; Λ ) W Λ be a solution point of MVI(W;Q). Let fw g = f(x ;y ; )g be the sequence generated by the MADM method, then there is a 0 0, such that + Λ + flb(y + y Λ ) H H + + fl(t fl)ax + + By + b H +» ( + ) Λ + flb(y y Λ ) H H + fl(t fl)ax + By b H fl( T ) Ax + + By + b + B(y y + ) ; 8 H H 0 : (34)

6 500 B.S. HE, S.L. WANG AND H. YANG Proof. It follows from (8) and (9) that Λ + flb(y y Λ ) H H + Λ + flb(y + y Λ ) H H +fl( fl)ax + + By + b H + flb(y y + ) H +fl( fl)(ax + By b) T H (By By + ): (35) Using Cauchy-Schwarz inequality, we have fl( fl)(ax + By b) T H (By By + ) fl(t fl)ax + By b H fl(t fl)ax + By b H Substituting (36) in (35), we derive where Note that + Λ H» Λ H fl( fl) T fl B(y y + ) H fl( fl) T fl ( + )B(y y + ) H : (36) + flb(y + y Λ ) H + fl(t fl)ax + + By + b H + flb(y y Λ ) H + fl(t fl)ax + By b H fl( T )Ax + + By + b H flffi B(y y + ) H ; (37) ffi := ( fl) (T fl) ( + ): ( fl) ffi = (T fl) ( fl) = ( + fl fl ) (T fl) fl 4fl +5 ( fl) ; (38) (T fl) the second equality of (38) is obtained by using (33). From 0and P =0 <, wehave lim! =0. Then there exists a positive constant 0 such that 0» ( fl) (T fl) < 5 ( + fl fl ); 8 0 : Since sup p fl(0; + 5 ) ffl 4fl +5g =5,itfollows from (38) that for 0 ffi 5 5 ( + fl fl )= 3 ( + fl fl )=( T ): (39) In addition, we have + Λ H + + flb(y + y Λ ) H + + fl(t fl)ax + + By + b H» ( + ) + Λ H + flb(y + y Λ ) H + fl(t fl)ax + + By + b H (40) Substituting (39) and (40) in (37), we obtain the conclusion of the theorem immediately. Using Theorem, we can prove the convergence of our method as follows. Theorem. Let f(x ;y ; )g be the sequence generated by the modified variable-penalty alternating directions method for MVI(W;Q). We have lim! Ax + + By + b H + B(y y + ) H =0: (4)

7 A Modified Variable-penalty Alternating Directions Method for VI 50 P Q Proof. Since f g is nonnegative and i= i <, it follows that i= ( + i) is bounded. Denote C s := X i= From Theorem, we have for all 0 that i ; C p := Y i= ( + i ): (4) + Λ H + + flb(y + y Λ ) H + + fl(t fl)ax + + By + b H» C p 0 Λ H 0 Therefore, there exists a constant C>0, such that Λ H + flb(y 0 y Λ ) H 0 + fl(t fl)ax0 + By 0 b H 0 + flb(y y Λ ) H + fl(t fl)ax + By b H» C; 8 0: (43) From Theorem and (43) we get X and hence i= 0 fl( T ) Ax i+ + By i+ b H i + B(y i y i+ ) H i» ( + Cs )C (44) lim! fl( T ) Ax + + By + b H + B(y y + ) H =0: Since fl (0; +p 5 ) and T = 3 ( + fl fl ) > 0, it follows that lim! Ax + + By + b H + B(y y + ) H =0 and thus Theorem is proved. Remember that the sequence fh g is bounded, it follows from Theorem that lim! and the MADM method is convergent. Ax + + By + b + B(y y + ) =0 5. Numerical Experiments Let w + = (x + ;y + ; + ) X Y R r be generated from a given triple w = (x ;y ; ) X Y R r by (0)-(4) with fl =. In this case, it follows that e y (w + )=0 and thus e(w + ) = e x (w + ) + e (w + ) ; where e x (w) =x P X fx [f(x) A T ]g and e (w) =Ax + By b: For the sae of balance, in [0], the authors adjusted the penalty parameter fi such that e x (w) ße (w). For VI problem ()-(3) has bloc form and x = x ;x ; ;x l ) T ; y =(y ;y ; ;y l T ; (45) f(x)) = f (x );f (x ); ;f l (x l )) T ; g(y) = g (y );g (y ); ;g l (y l ) T : (46) Ω=fu =(x; y)jx i X i ;y i Y i ;A i x i + B i y i = b i ; 8i =; ;l:g (47) we suggest the similar strategy as in [0] for adjusting the penalty matries: A simple strategy for Adjusting penalty matrix H in MADM method.

8 50 B.S. HE, S.L. WANG AND H. YANG H (i) + = 8 >< >: ( + fi )H (i) ; if x i P X i [x i (f i(x i ) AT i i )] <μa ix i + B iyi b i, H (i) +fi ; if μx i P X i [x i (f i(x i ) A T i i )] > A i x i + B i yi b i, H (i) ; otherwise. with a symmetric matrix H 0 = diagfh () 0 ;H() 0 ; ;H(s) 0 g and H(i) 0 χ 0; i=; ;l: In the numerical tests, we consider the Fermat-Weber problem [] min yr n lx i= a i y b (48) in which the vectors b and the weights a i > 0 are given. For n = the problem has a single-facility location interpretation: b are shipment centers, represented as points in the plane; the sought minimizer is the location of the facility to be built, such that the sum of the transportation costs between the centers and the facility is minimized, where each cost is proportional to the Euclidean distance. Introducing auxiliary vectors of x [] ; ;x [l], Problem (48) can be rewritten as min f lx i= Then it can be formulated into the form (45) (47) with a i x jx = y b ; i =; ;lg: (49) x i = x ; y i = y; f i (x i )=a i x i x i ; g i(y i )=0; A i = I n n ; B i = I n n ; b i = b ; X i = Y i = R n ; i =; ;l: Thus, under the non-degeneracy assumption, we can apply the alternating directions method with self-adaptive bloc diagonal penalty matrixtosolve Problem (49), in which wetae fl =, H 0 = diagffi[] 0 I n;fi([] 0 I n; ;fi(l) 0 I ng with fi 0 > 0;i=; ;l and =minf; (maxf; 00g) g = f; ;:::;; 4 ; 9 ; 6 ;:::g: In particular implementation, we have and fi + = 8 >< >: ( + )fi if a i x x < 0:x y + b, fi =( + x ) if 0:a i x > x y + b, otherwise fi x + = a i fi y + = lx i= fi i lx i= ; = + fi y fi b ; i =; ;l; fi x+ + fi b + = fi i (x + y + + b ); i =; ;l: In [], the author used ADM method for Problem (49) by taing H 0 =diagn a b [] I n; a b [] I n; ; ; a l b [l] I n o; L = 0:075 nl lx i= a i ;

9 A Modified Variable-penalty Alternating Directions Method for VI 503 which are denoted by H 0 [] andl[] respectively in the sequel, and H = diagffi[] I n;fi[] I n; ;fi[l] I ng; ( fi = :05fi T ; if fi T <L; maxf0:98fi T ;Lg; otherwise; for i =; ;l; where the penalties were updated every T (= 0) iterations. To assess the impact of the penalty value on performance, we generated classes of data, with the number of points l in f5; 50; 75g and the dimension n in f; 4; 8; 6g. For each class the problem is generated randomly. The weights a i were uniformly distributed in [; 0], while the components of b are uniformly distributed in [0; 00]. The stopping test was e(w )» 0 6. Table reports the computational results by applying the proposed MADM method and the ADM method in [] for solving Problem (49), respectively. Table. Number of iterations for Fermat-Weber problems n l H 0 =0 I H 0 =0 I H 0 = I H 0 =0I H 0 =0 I H 0 = H 0[] ADM MADM ADM MADM ADM MADM ADM MADM ADM MADM ADM MADM > > > > > > > > > > > > It seems that the solution time of the proposed MADM method is much less than that of ADM method. We note that, for the same problem, the iteration numbers of ADM method are significantly depends on the initial penalty. Although the ADM method performs as efficiently as (or somewhat more efficiently than) MADM method when the initial penalty matrix H 0 is selected carefully as in [], it is inconvenienttochoose a proper initial penalty in ADM method for individual problems. Another difficulty encountered by the PADM method is how tochoose aproperlower boundary L. As we have seen that L = 0:075 l nl i= a i in [] isnotanobvious one. The iteration numbers of the proposed MADM method are insensitive to the initial penalty. To demonstrate it more clearly, we test the Fermat-Weber problem with n = 6 and l = 75 (the largest problem in Table and ). The initial penalty matrixh 0 is a positive diagonal matrix whose elements are fi 0 I 6;i=; ; 75. We letfi 0 be uniformly distributed in (0p ; 0 p ) and list the iteration numbers in Table. Table. Number of iterations of the proposed method for Fermat-Weber problems fi 0 (0p ; 0 p ),p = Number of iterations Conclusion. In this paper, we proposed a modified variable-penalty alternating directions method. The presented MADM method extends the ADM method by allowing the penalty matrix to vary more flexible. The preliminary numerical tests show that the proposed method with self-adaptive technique is more preferable in practice. References [] J. Ecstein and M. Fuushima, Some reformulation and applications of the alternating direction method of multipliers, Large Scale Optimization: State of the Art, W.W.Hager et al eds., Kluwer Academic Publishers, (994), 5-34.

10 504 B.S. HE, S.L. WANG AND H. YANG [] M. Fortin and R. Glowinsi, Eds., Augmented Lagrangian Methods: Applications to the solution of Boundary-Valued Problems, North-Holland, Amsterdam, (983). [3] M. Fuushima, Application of the alternating direction method of multipliers to separable convex programming problems, Computational Optimization and Applications, (99), 93-. [4] D. Gabay and B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite-element approximations, Computer and Mathematics with Applications, (976), [5] D. Gabay, Applications of the method of multipliers to variational inequalities, Augmented Lagrange Methods: Applications to the Solution of Boundary-valued Problems, M. Fortin and R. Glowinsi, eds., North Holland, Amsterdam, The Netherlands, (983), [6] R. Glowinsi, Numerical Methods for Nonlinear Variational Problems, Springer-Verlag, New Yor, Berlin, Heidelberg, Toyo, (984). [7] R. Glowinsi and P. Le Tallec, Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics, SIAM Studies in Applied Mathematics, Philadelphia, PA, 989. [8] P.T. Harer and J.S. Pang, Finite-dimensional variational inequality and nonlinear complementarity problems: A Survey of theory, algorithms and applications, Mathematical Programming, 48 (990), 6-0. [9] B.S. He and H. Yang, Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities, Operations Research letters, 3 (998), 5-6. [0] B.S. He, H. Yang and S.L. Wang, Alternating directions method with self-adaptive parameter for monotone variational inequalities, JOTA, 06 (000), [] D. Kinderlehhrer and G. Stampacchia, An Introduction to Variational Inequalities and Their Applications, Academic Press, New Yor, (980). [] S. Kontogiorgis and R.R. Meyer, A variable-penalty alternating directions method for convex optimization, Mathematical Programming, 83 (998), [3] P.L. Lions and B. Mercier, Splitting algorithms for the sum of two nonlinear operators, SIAM J. Numerical Analysis, 6 (979), [4] A. Nagurney, Networ Economics, A Variational Inequality Approach, Kluwer Academic Publishers, Dordrecht, Boston, London, (993). [5] P. Tseng, Applications of splitting algorithm to decomposition in convex programming and variational inequalities, SIAM J. Control Optim., 9 (99), 9-38.