Research Article Stochastic Methods Based on VU-Decomposition Methods for Stochastic Convex Minimax Problems

Transcription

1 Mathematcal Problems n Engneerng, Artcle ID , 5 pages Research Artcle Stochastc Methods Based on VU-Decomposton Methods for Stochastc Convex Mnmax Problems Yuan Lu, 1 We Wang, 2 Shuang Chen, 3 and Mng Huang 3 1 ormal College, Shenyang Unversty, Shenyang , Chna 2 School of Mathematcs, Laonng ormal Unversty, Dalan , Chna 3 School of Mathematcal Scences, Dalan Unversty of Technology, Dalan , Chna Correspondence should be addressed to We Wang; we 0713@sna.com Receved 6 August 2014; Revsed 29 ovember 2014; Accepted 29 ovember 2014; Publshed 4 December 2014 Academc Edtor: Hamd R. Karm Copyrght 2014 Yuan Lu et al. Ths s an open access artcle dstrbuted under the Creatve Commons Attrbuton Lcense, whch permts unrestrcted use, dstrbuton, and reproducton n any medum, provded the orgnal work s properly cted. Ths paper apples sample average approxmaton (SAA) method based on VU-space decomposton theory to solve stochastc convex mnmax problems. Under some moderate condtons, the SAA soluton converges to ts true counterpart wth probablty approachng one and convergence s exponentally fast wth the ncrease of sample sze. Based on the VU-theory, a superlnear convergent VU-algorthm frame s desgned to solve the SAA problem. 1. Introducton In ths paper, the followng stochastc convex mnmax problem (SCMP) s consdered: mn x R nf (x), (1) f (x) = max E [f (x, ξ)]:=0,...,m, (2) and the functons f (x, ξ) : R n R, =0,...,m,areconvex and C 2, ξ : Ω Ξ R n s a random vector defned on probablty space (Ω, Υ, P); E denotes the mathematcal expectaton wth respect to the dstrbuton of ξ. SCMP s a natural extenson of determnstc convex mnmax problems (CMP for short). The CMP has a number of mportant applcatons n operatons research, engneerng problems, and economc problems. Whle many practcal problems only nvolve determnstc data, there are some mportant nstances problems data contans some uncertantes and consequently SCMP models are proposed to reflect the uncertantes. A blanket assumpton s made that, for every x R n, E[f (x, ξ)], = 0,...,m, are well defned. Let ξ 1,...,ξ be a samplng of ξ. A well-known approach based on the samplng s the so-called SAA method, that s, usng sample average value of f (x, ξ) to approxmate ts expected value because the classcal law of large number for random functons ensures that the sample average value of f (x, ξ) converges wth probablty 1 to E[f (x, ξ)] when the samplng s ndependent and dentcally dstrbuted (dd for short). Specfcally, we can wrte down the SAA of our SCMP (1) as follows: (x) = max mn x R n (x), (3) (x) :=0,...,m, (x) := 1 f (x, ξ j ). The problem (3) s called the SAA problem and (1) the true problem. The SAA method has been a hot topc of research n stochastc optmzaton. Pagnoncell et al. [1] present the SAA method for chance constraned programmng. Shapro et al. [2] consder the stochastc generalzed equaton by usng the SAA method. Xu [3] rases the SAA method for a class of stochastc varatonal nequalty problems. Lu et al. [4] (4)

2 2 Mathematcal Problems n Engneerng gve the penalzed SAA methods for stochastc mathematcal programs wth complementarty constrants. Chen et al. [5] dscuss the SAA methods based on ewton method to the stochastc varatonal nequalty problem wth constrant condtons. Snce the objectve functons of the SAA problems n the references talkng above are smooth, then they can be solved by usng ewton method. More recently, new conceptual schemes have been developed, whch are based on the VU-theory ntroduced n [6];seeelse[7 11]. The dea s to decompose R n nto two orthogonal subspaces V and U at a pont x, the nonsmoothness of f s concentrated essentally on V and the smoothness of f appears on the U subspace. More precsely, for a gven g f(x), f(x) denotes the subdfferental of f at x n the sense of convex analyss, then R n can be decomposed nto drect sum of two orthogonal subspaces, that s, R n = U V,V = ln( f(x) g),andu = V. As a result an algorthm frame can be desgned for the SAA problem that makes a step n the V space, followed by a U- ewton step n order to obtan superlnear convergence. A VU-space decomposton method for solvng a constraned nonsmooth convex program s presented n [12]. A decomposton algorthm based on proxmal bundle-type method wth nexact data s presented for mnmzng an unconstraned nonsmooth convex functon n [13]. In ths paper, the objectve functon n (1) s nonsmooth, butthasthestructurewhchhastheconnectonwthvuspace decomposton. Based on the VU-theory, a superlnear convergent VU-algorthm frame s desgned to solve the SAAproblem.Therestofthepapersorganzedasfollows.In the next secton, the SCMP s transformed to the nonsmooth problem and the proof of the approxmaton soluton set converges to the true soluton set n the sense that Hausdorff dstance s obtaned. In Secton 3, the VU-theory of the SAA problem s gven. In the fnal secton, the VU-decomposton algorthm frame of the SAA problem s desgned. 2. Convergence Analyss of SAA Problem In ths secton, we dscuss the convergence of (3) to (1) as ncreases. Specfcally, we nvestgate the fact that the soluton of the SAA problem (3) converges to ts true counterpart as. Frstly, we make the basc assumptons for SAA method. In the followng, we gve the basc assumptons for SAA method. Assumpton 1. (a) Lettng X be a set, for =1,...,n,thelmts M F (t) := lm M F (t) (5) exst for every x X. (b) For every s X, the moment-generatng functon M s (t) s fnte-valued for all t n a neghborhood of zero. (c) There exsts a measurable functon κ:ω R + such that g(s,ξ) g(s, ξ) κ(ξ) s s for all ξ Ωand all s,s X. (6) (d) The moment-generatng functon M κ (t) = E[e tκ(ξ) ] of κ(ξ) s fnte-valued for all t n a neghborhood of zero, M s (t) = E[e t(g(s,ξ) G(s)) ] s the moment-generatng functon of the random varable g(s, ξ) G(s). Theorem 2. Let S and S denote the soluton sets of (1) and (3). Assumng that both S and S are nonempty, then, for any ε > 0,onehasD(S,S ) < ε,d(s,s ) = sup x S d(x, S ). Proof. For any ponts x S and x R n,wehave ( x) = max ( x), =0,...,m = max 1 f ( x, ξ j ), =0,...,m max 1 f (x, ξ j ), =0,...,m. From Assumpton 1, weknowthat,foranyε>0, there exst M;f>M, =0,...,m,then (7) 1 f (x, ξ j ) E[f (x, ξ j )] <ε. (8) By lettng >M,weobtan ( x) max 1 f (x, ξ j ), =0,...,m max E [f (x, ξ j )]+ε, =0,...,m =f(x) +ε. Ths shows that d( x,s ) < ε, whch mples D( x,s ) < ε. Wenowmoveontodscusstheexponentalrateof convergence of SAA problem (3) to the true problem(1) as sample ncreases. Theorem 3. Let x be a soluton to the SAA problem (3) and S sthesolutonsetofthetrueproblem(1). Suppose Assumpton 1 holds. Then, for every ε>0, there exst postve constants c(ε) and d(ε),suchthat for suffcently large. (9) Prob d (x,s ) ε c(ε) exp d(ε) (10) Proof. Letε >0beanysmallpostvenumber. ByTheorem 2 and 1 f (x, ξ j ) E[f (x, ξ j )] <ε, (11)

3 Mathematcal Problems n Engneerng 3 we have d(x,s)<ε. Therefore, by Assumpton 1,wehave Prob d (x,s ) ε Prob 1 f (x, ξ j ) E[f (x, ξ j )] c(ε) exp d(ε). The proof s complete. 3. The VU-Theory of the SAA Problem δ (12) In the followng sectons, we gve the VU-theory, VUdecomposton algorthm frame, and convergence analyss of the SAA problem. The subdfferental of at a pont x R n can be computed n terms of the gradents of the functon that are actve at x.moreprecsely, (x) Theorem 5. Suppose Assumpton 4 holds. Then R n can be decomposton at x:r n = U V, V = ln (x) 0 (x), 0= I(x) U =d R n d, (x) 0 (x) = 0. 0= I(x) (19) Proof. The proof can be drectly obtaned by usng Assumpton 4 and the defnton of the spaces V and U. Gven a subgradent g wth V-component g V = V T g,theu-lagrangan of, dependng on g V, s defned by R dm U u L u (u; g V ) := mn (x+uu + VV) g V, V V. V R dm V (20) The assocated set of V-space mnmzers s defned by = Conv g R n g= I(x) α f (x, ξ j ): (13) W(u;g V ) := V :L u (u; g V )= (x+uu + VV) g V, V V. (21) α=(α ) I(x) Δ I(x), Theorem 6. Suppose Assumpton 4 holds. Let χ(u) = x+u V(u) be a trajectory leadng to x and let H:= 2 L u (0, 0).Then for all u suffcently small the followng hold: I (x) = I (x) = s the set of actve ndces at x,and Δ s =α R s α 0, s =1 (x) (14) α =1. (15) Let x R n be a soluton of (3).Bycontnutyofthestructure functons, there exsts a ball B ε (x) R n such that x B ε (x), I(x) I(x). (16) For convenence, we assume that the cardnalty of I(x) s m 1 +1 (m 1 m)and reorder the structure functons, so that I(x)=0,...,m 1. From now on, we consder that x B ε (x), (x) = (x), I(x). (17) The followng assumpton wll be used n the rest of ths paper. Assumpton 4. The set s lnearly ndependent. (x) 0 (x) 0= I(x) (18) () the nonlnear system, wth varable V and the parameter u, (x+uu + VV) 0 (x+uu + VV) =0, 0 = I(x) (22) has a unque soluton V = V(u) and V : R dm U R dm V s a C 2 functon; () χ(u) s a C 2 -functon wth Jχ(u) = U+VJV(u); () L u (u; 0) = (x +u V(u)) = (x+u V(u)) = (x) + (1/2)u T Hu + o( u 2 ); (v) L u (u; 0) = Hu + o( u ); (v) (χ(u)) = (χ(u)), I(x). Proof. Item () follows from the assumpton that f are C 2 and applyng a Second-Order Implct Functon Theorem (see [14], Theorem 2.1). Snce V(u) s C 2, χ(u) s C 2 and the Jacobans JV(u) exst and are contnuous. Dfferentatng the prmal track wth respect to u, weobtantheexpressonof Jχ(u) and tem () follows. () By the defnton of L u (u; g V ) and W(u; g V ),wehave L u (u; 0) = (x+u V (u)) = (x+u V (u)). (23)

4 4 Mathematcal Problems n Engneerng Accordng to the second-order expanson of L u, we obtan L u (u; 0) =L u (0; 0) α (u) (x (k) )= g (k) (x (k) ) I(x) (31) Snce L u (0; 0) = 2 L u (0; 0), + L u (0; 0),u ut 2 L u (0; 0) u+o( u ). (24) L u (u; 0) = Smlar to (), we get (v): (x), I(x), L u (0; 0) = 0,andH= (x) ut Hu + o ( u 2 ). (25) L u (u; 0) = L u (0; 0) + u T 2 L u (0; 0),u +o( u 2 ) = H+o( u ). (26) The concluson of (v) can be obtaned n terms of () and the defnton of χ(u). 4. Algorthm and Convergence Analyss Supposng 0 (x), we gve an algorthm frame whch can solve (3). Ths algorthm makes a step n the V-subspace, followed by a U-ewton step n order to obtan superlnear convergence rate. Algorthm 7 (algorthm frame). Step 0. Intalzaton: gven ε>0,chooseastartngpontx (0) close to x enough and a subgradent g (0) (x (0) ) and set k=0. Step 1.Stopf g(k) ε. (27) s such that V T g (k) =0.Computex (k+1) = x (k) +δ (k) U 0= x (k) +δ (k) U δ(k) V. Step 6.Update:setk=k+1andreturntoStep1. Theorem 8. Suppose the startng pont x (0) s close to x enough and 0 r (x), 2 L u (0; 0) 0. Then the teraton ponts x (k) k=1 generated by Algorthm 7 converge and satsfy x(k+1) x =o( x(k) x ). (32) Proof. Let u (k) =(x (k) x) U and V (k) =(x (k) x) V +δ (k) V.It follows from Theorem 6() that (x(k+1) x) V = (x(k) x) V =o (x(k) x) U =o (x(k) x). (33) Snce 2 L u (0; 0) exsts and L u (0; 0) = 0, wehavefromthe defnton of U-Hessan matrx that L u (u (k) ;0)=U T g (k) =0+ 2 L u (0; 0) u (k) +o( u(k) U ). (34) By vrtue of (30),wehave 2 L u (0; 0)(u (k) +δ (k) U ) = o( u(k) U ). It follows from the hypothess 2 L u (0; 0) 0 that 2 L u (0; 0) s nvertble and hence u (k) + δ (k) U = o( u(k) U ). In consequence, one has (x (k+1) x) U =(x (k+1) x (k) ) U Step 2. Fnd the actve ndex set I(x). Step 3.ConstructVU-decomposton at x;thats,r n = V U.Compute +(x (k) x (k) ) U +(x (k) x) U =u (k) +δ (k) U =o( u(k) ) U =o( x(k) x ). (35) 2 L u (0; 0) = U T M (0) U, (28) M (0) = α 2 (x). I(x) Step 4.PerformV-step. Compute δ (k) V (22) and set x (k) =x (k) +0 δ (k) V. Step 5.PerformU-step. Compute δ (k) U (29) whch denotes V(u) n from the system U T M (0) Uδ U + U T g (k) =0, (30) The proof s completed by combnng (33) and (35). Conflct of Interests The authors declare that there s no conflct of nterests regardng the publcaton of ths paper. Acknowledgments The research s supported by the atonal atural Scence Foundaton of Chna under Project nos , , and and General Project of the Educaton Department of Laonng Provnce no. L

5 Mathematcal Problems n Engneerng 5 References [1] B. K. Pagnoncell, S. Ahmed, and A. Shapro, Sample average approxmaton method for chance constraned programmng: theory and applcatons, JournalofOptmzatonTheoryand Applcatons,vol.142,no.2,pp ,2009. [2] A. Shapro, D. Dentcheva, and A. Ruszczynsk, Lecture on Stochastc Programmng: Modellng and Theory,SIAM,Phladelpha, Pa, USA, [3] H. Xu, Sample average approxmaton methods for a class of stochastc varatonal nequalty problems, Asa-Pacfc Journal of Operatonal Research,vol.27,no.1,pp ,2010. [4] Y.Lu,H.Xu,andJ.J.Ye, Penalzedsampleaverageapproxmaton methods for stochastc mathematcal programs wth complementarty constrants, Mathematcs of Operatons Research, vol. 36, no. 4, pp , [5] S. Chen, L.-P. Pang, F.-F. Guo, and Z.-Q. Xa, Stochastc methods based on ewton method to the stochastc varatonal nequalty problem wth constrant condtons, Mathematcal and Computer Modellng,vol.55,no.3-4,pp ,2012. [6] C. Lemarechal, F. Oustry, and C. Sagastzabal, The U- Lagrangan of a convex functon, Transactons of the Amercan Mathematcal Socety,vol.352,no.2,pp ,2000. [7] R. Mffln and C. Sagastzábal, VU-decomposton dervatves for convex max-functons, n Ill-Posed Varatonal Problems and Regularzaton Technques, M. Théra and R. Tchatschke, Eds., vol. 477 of Lecture otes n Economcs and Mathematcal Systems, pp , Sprnger, Berln, Germany, [8] C. Lemaréchal and C. Sagastzábel, More than frst-order developments of convex functons: prmal-dual relatons, Journal of Convex Analyss,vol.3,no.2,pp ,1996. [9] R. Mffln and C. Sagastzabal, On VU-theory for functons wth prmal-dual gradent strcture, SIAM Journal on Optmzaton, vol. 11, no. 2, pp , [10] R. Mffln and C. Sagastzábal, Functons wth prmal-dual gradent structure and U-Hessans, n onlnear Optmzaton and Related Topcs, G.PlloandF.Ganness,Eds.,vol.36of Appled Optmzaton, pp , Kluwer Academc, [11] R. Mffln and C. Sagastzábal, Prmal-dual gradent structured functons: second-order results; lnks to EPI-dervatves and partly smooth functons, SIAM Journal on Optmzaton, vol. 13,no.4,pp ,2003. [12] Y.Lu,L.P.Pang,F.F.Guo,andZ.Q.Xa, Asuperlnearspace decomposton algorthm for constraned nonsmooth convex program, Computatonal and Appled Mathematcs, vol.234,no.1,pp ,2010. [13] Y. Lu, L.-P. Pang, J. Shen, and X.-J. Lang, A decomposton algorthm for convex nondfferentable mnmzaton wth errors, Appled Mathematcs, vol. 2012, ArtcleID ,15pages,2012. [14] S. Lang, Real and Functonal Analyss, vol. 142ofGraduate Texts n Mathematcs, Sprnger, ew York, Y, USA, 3rd edton, 1993.

6 Advances n Operatons Research Advances n Decson Scences Appled Mathematcs Algebra Probablty and Statstcs The Scentfc World Journal Internatonal Dfferental Equatons Submt your manuscrpts at Internatonal Advances n Combnatorcs Mathematcal Physcs Complex Analyss Internatonal Mathematcs and Mathematcal Scences Mathematcal Problems n Engneerng Mathematcs Dscrete Mathematcs Dscrete Dynamcs n ature and Socety Functon Spaces Abstract and Appled Analyss Internatonal Stochastc Analyss Optmzaton