ON THE CONVERGENCE OF DECOMPOSITION METHODS FOR MULTISTAGE STOCHASTIC CONVEX PROGRAMS

Transcription

1 ON THE CONVERGENCE OF DECOMPOSITION METHODS FOR MULTISTAGE STOCHASTIC CONVEX PROGRAMS P. GIRARDEAU, V. LECLERE, AND A. B. PHILPOTT Absrac. We prove he alos-sure covergece of a class of sapligbased esed decoposiio algorihs for ulisage sochasic covex progras i which he sage coss are geeral covex fucios of he decisios, ad uceraiy is odelled by a sceario ree. As special cases, our resuls iply he alos-sure covergece of SDDP, CUPPS ad DOASA whe applied o probles wih geeral covex cos fucios. 1. Iroducio Mulisage sochasic progras wih recourse are well ow i he sochasic prograig couiy, ad are becoig ore coo i applicaios. We are oivaed i his paper by applicaios i which he sage coss are oliear covex fucios of he decisios. Producio fucios are ofe odelled as oliear cocave fucios of allocaed resources. For exaple Fiardi ad da Silva [4] use his approach o odel hydro elecriciy producio as a cocave fucio of waer flow. Sooh oliear value fucios also arise whe oe axiizes profi wih liear dead fucios see e.g. [11] givig a cocave quadraic objecive or whe cohere ris easures are defied by coiuous disribuios i ulisage probles [13]. Havig geeral covex sage coss does o preclude he use of cuig plae algorihs for aacig hese probles. The os well-ow of hese is he sochasic dual dyaic prograig SDDP algorih of Pereira ad Pio [9]. This algorih cosrucs feasible dyaic prograig policies usig a ouer approxiaio of a covex fuure cos fucio ha is copued usig Beders cus. The policies defied by hese cus ca be evaluaed usig siulaio, ad heir perforace easured agais a lower boud o heir expeced cos. This provides a covergece crierio ha ay be applied o eriae he algorih whe he esiaed cos of he cadidae policy is close eough o is lower boud. The SDDP algorih has led o a uber of relaed ehods [1, 2, 3, 6, 10] ha are based o he sae esseial idea, bu see o iprove he ehod by exploiig Dae: May 18, Maheaics Subjec Classificaio. 90C14,90C39. Key words ad phrases. Sochasic Prograig, Dyaic Prograig, Sochasic Dual Dyaic Prograig algorih, Moe-Carlo saplig, Beders decoposiio. Acowledgees: The firs auhor was suppored i his research by a gra fro he OSIRIS Depare a EDF R&D, Frace. The auhors also wish o ha Kegy Bary for useful discussios o earlier versios of his paper. 1

2 2 P. GIRARDEAU, V. LECLERE, AND A. B. PHILPOTT he srucure of paricular applicaios. We call hese ehods DOASA for Dyaic Ouer-Approxiaio Saplig Algorihs bu hey are ow geerically aed SDDP ehods. SDDP ehods are ow o coverge alos surely o a fiie sceario ree whe he sage probles are liear progras. The firs foral proof of such a resul was published by Che ad Powell [1] who derived his for heir CUPPS algorih. This proof was exeded by Liowsy ad Philpo [8] o cover oher SDDP algorihs. The covergece proofs i [1] ad [8] ae use of a usaed assupio regardig he idepedece of sapled rado variables ad coverge subsequeces of algorih ieraes. This assupio was ideified by Philpo ad Gua [10], who gave a sipler proof of alos sure covergece of SDDP ehods based o he fiie covergece of he esed decoposiio algorih see [2]. This does o require he usaed assupio, bu explois he fac ha he collecio of subprobles o be solved has a fiie uber of dual exree pois. This begs he quesio of wheher SDDP ehods will coverge alos surely for geeral covex sage probles, where he value fucios ay adi a ifiie uber of subgradies. I his paper we propose a differe approach fro he oe i [1] ad [8] ad show how a proof of covergece for saplig-based esed decoposiio algorihs o fiie sceario rees ca be esablished for odels wih covex subprobles which ay o have polyhedral value fucios. Our resul is proved for a geeral class of ehods icludig all he variaios discussed i he lieraure [1, 2, 3, 6, 9, 10]. The proof esablishes covergece wih probabiliy 1 as log as he saplig i he forward pass is idepede of previous realisaios. Our proof relies heavily o he idepedece assupio ad aes use of he Srog Law of Large Nubers. I coras o [10] we have o show ha covergece is guaraeed i all procedures for cosrucig a forward pass ha visi every ode of he sceario ree a ifiie uber of ies. The resul we prove wors i he space of sae variables expressed as rado variables adaped o he filraio defied by he sceario ree. Because his ree has a fiie uber of odes, his space is copac, ad so we ay exrac coverge subsequeces for ay ifiie sequece of saes. Ulie he argues i [1] ad [8], hese subsequeces are o explicily cosruced, ad so we ca escape he eed o assue properies of he ha we wish o be iheried fro idepede saplig. Alhough he value fucios we cosruc adi a ifiie uber of subgradies, our resuls do require a assupio ha serves o boud he ors of hese. This assupio is a exesio of relaively coplee recourse ha esures ha soe ifeasible cadidae soluios o ay sage proble ca be forced o be feasible by a suiable corol. Sice we are worig i he real of oliear prograig, soe cosrai qualificaio of his for will be eeded o esure ha we ca exrac subgradies. I pracice, SDDP odels use pealies o cosrai violaios o esure feasibiliy, which iplicily bouds he subgradies of he Bella fucios. Our recourse assupios are arguably weaer, sice we do o have

3 DECOMPOSITION OF MULTI-STAGE STOCHASTIC CONVEX PROGRAMS 3 a resul ha shows ha hey eable a equivale forulaio wih a exac pealizaio of ifeasibiliy. The paper is laid ou as follows. We firs cosider a deeriisic ulisage proble, i which he proof is easily udersadable. This is he exeded i 3 o a sochasic proble forulaed i a sceario ree. We close wih soe rears abou he covergece of saplig algorihs. 2. The deeriisic case Our covergece proofs are based aroud showig ha a sequece of ouer approxiaios fored by cuig plaes coverges o he rue Bella fucio i he eighbourhood of he opial sae rajecories. We begi by providig a proof ha Kelley s cuig plae ehod [7] coverges whe applied o he opiizaio proble: W := i W u, u U where U is a covex subse of R, ad W is a covex fucio wih uiforly bouded subgradies. The resul we prove is o direcly used i he ore coplex resuls ha follow, bu he ai ideas o which he proofs rely are he sae. We believe he reader will fid i coveie o already have he schee of he proof i id whe sudyig he ore ipora resuls laer o. Kelley s ehod geeraes a sequece of ieraes u j by solvig, a j N each ieraio, a piecewise liear odel of he origial proble. The odel is he ehaced by addig a cuig plae based o he value W u j ad subgradie g j of W a u j. The odel a ieraio is deoed by W u := ax 1 j W u j + g j, u u j, ad θ := i u U W u = W u +1. We have he followig resul. Lea 2.1. If W is covex wih uiforly bouded subgradies o U ad U is copac he li W u = W. + Proof. This proof is ae fro Ruszczysi [12, Theore 7.7]. Le K ε be he se of idices such ha W + ε < W u < +. The proof cosiss i showig ha K ε is fiie. Suppose 1, 2 K ε ad 1 is sricly saller ha 2. We have ha W u 1 > W + ε ad ha W θ 1. Sice a ew cu will be geeraed a u 1, we will have W u 1 + g 1, u u 1 W 1 u W 2 1 u, u U, where g 1 is a elee of W u 1. I paricular, choosig u = u 2 gives W u 1 + g 1, u 2 u 1 W 1 u 2 W 2 1 u 2 = θ 2 1 W. Bu ε < W u 2 W, so ε < W u 2 W u 1 g 1, u 2 u 1,

4 4 P. GIRARDEAU, V. LECLERE, AND A. B. PHILPOTT ad as g 2 W u 2, he subgradie iequaliy for u = u 1 W u 2 W u 1 g 2, u 2 u 1. yields Therefore, sice W has uiforly bouded subgradies, here exiss κ > 0 such ha ε < 2κ u 2 u 1, 1, 2 K ε, 1 2. Because U is copac, K ε has o be fiie. Oherwise here would exis a coverge subsequece of { u } K ε ad his las iequaliy could o hold for sufficiely large idices wihi K ε. This proves he lea. Noe ha Lea 2.1 does o iply ha he sequece of ieraes u N coverges 1. For isace, if he iiu of W is aaied o a fla par if W is o sricly covex, he he sequece of ieraes ay o coverge. However, he lea shows ha he sequece of W values a hese ieraes will coverge. We ow cosider he ulisage case. Le T be a posiive ieger. We firs cosider he followig deeriisic opial corol proble. 1a 1b 1c 1d 1e i x,u T 1 C x, u + V T x T =0 s.. x +1 = f x, u, = 0,..., T 1, x 0 is give, x X, = 0,..., T, u U x, = 0,..., T 1. I wha follows we le AffX deoe he affie hull of X, ad defie B δ = {y AffX y < δ}. We ow ae he followig assupios H 1, for = 0,..., T 1: 1 X R, X T R, 2 ulifucios U : R R are assued o be covex 2 ad covex copac valued, 3 fucios C ad V T are assued o be covex lower seicoiuous proper fucios, 4 fucios f are liear, 5 fial cos fucio V T is fiie valued ad Lipschiz-coiuous o X T, 6 here exiss δ > 0, defiig X :=X + B δ such ha : a x X, u U x, C x, u <, b for every x X, f x, U x X eve hough because U is copac, here exiss a coverge subsequece. 2 Recall ha a ulifucio U o covex se X is called covex if 1 λux+λuy U1 λx + λy for every x, y X ad λ 0, 1.

5 DECOMPOSITION OF MULTI-STAGE STOCHASTIC CONVEX PROGRAMS 5 Assupios H are ade o guaraee ha proble 1 is a covex opiizaio proble. Sice his proble is i geeral oliear, i also requires a cosrai qualificaio o esure he exisece of subgradies. This is he purpose of Assupio H 1 6. This assupio eas ha we ca always ove fro X a disace of δ 2 i ay direcio ad say i X, which is a for of recourse assupio ha we call exeded relaively coplee recourse ERCR. We oe ha his is less srige ha iposig coplee recourse, which would require X = R. Fially we oe ha we ever eed o evaluae C x, u wih x X \X, so we ay assue ha here exiss a covex fucio, fiie o X, ha coicides wih C o X. Of course o all covex cos fucios saisfy such a propery e.g. x x logx cao be exeded below x = 0 while aiaiig covexiy. We are ow i a posiio o describe a algorih for he deeriisic corol proble 1. The Dyaic Prograig DP equaio associaed wih 1 is as follows. For all = 0,..., T 1, le i u Ux C x, u + V +1 f x, u, x }{{} X 2 V x = :=W x,u +, oherwise. Here he quaiy W x, u is he fuure opial cos sarig a ie fro sae x ad choosig decisio u, so ha V x = i u Ux W x, u. The cuig plae ehod wors as follows. A ieraio 0, defie fucios V 0, = 0,..., T 1, o be ideically equal o. A ie T, sice we ow exacly he ed value fucio, we ipose VT = V T for all ieraios N. A each ieraio, he process is he followig. Sarig wih x 0 = x 0, a ay ie sage, solve 3a 3b 3c 3d θ = i u R x Aff X C x, u + V 1 +1 f x, u, s.. x = x [β ] f x, u X +1 u U x Here β AffX is a vecor of Lagrage ulipliers for he cosrai x = x. We deoe a iiizer of 3 by u. Is exisece is guaraeed by ERCR. Noe ha cosrai 3a ca be see as a iduced cosrai o u. Thus we ca defie he ulifucios Ũ : R R by, for all x R, 4 Ũ x := {u U x f x, u X +1 }. We ca easily chec ha Ũ is covex by lieariy of f ad covexiy of U ad covex copac valued as he iersecio of a copac covex se ad a covex se. Thus 3 ca be wrie as 5a 5b θ = i u Ũx x Aff X C x, u + V 1 +1 f x, u, s.. x = x. [β ]

6 6 P. GIRARDEAU, V. LECLERE, AND A. B. PHILPOTT Now defie, for ay x R : 6 V x := ax V 1 x, θ + β, x x, ad ove o o he ex ie sage + 1 by defiig x +1 = f x, u. Rear 2.1. The assupio ha β is i AffX is ade for echical reasos, ad loses lile geeraliy. Ideed if β R is a opial Lagrage uliplier, he so is is projecio o AffX. I pracice we would expec AffX o be he sae diesio for every. If his diesio happeed o be d sricly less ha, he we igh chage he forulaio by a rasforaio of variables so ha AffX = R d. Rear 2.2. Observe ha our algorih uses V+1 1 whe solvig he wosage proble 3 a sage, alhough os ipleeaios of SDDP ad relaed algorihs proceed bacwards ad are hus able o use he freshly updaed V+1 alhough see e.g. [1] for a siilar approach o he oe proposed here. I he sochasic case we prese a geeral fraewor ha ecopasses bacward passes. Noe ha oly he las fuure cos fucio V T is ow exacly a ay ieraio. All he oher oes are lower approxiaios cosisig of he axiu of affie fucios a ieraio. We aurally have he sae lower approxiaio for fucio W. Thus we defie for ay x, u i R + 7 W x, u := C x, u + V +1 f x, u, ad recall 8 W x, u := C x, u + V +1 f x, u. Usig his oaio we have 9 θ = i W 1 u Ũx x, u Because for ay x X +1 ad ay V 1 +1 x, i follows ha θ = i W 1 u Ũx x, u = W 1 x, u we have ha V 1 +1 x i W 1 u Ũx x, u,, = i W 1 u Ũx x +, x x, u, which, usig covexiy of he opial value fucio, ad he defiiios of θ ad β, gives θ θ + β, x x,. Sice by 6 V 1 x { } = ax θ + β, x x <

7 DECOMPOSITION OF MULTI-STAGE STOCHASTIC CONVEX PROGRAMS 7 i follows ha ad so 6 iplies 10 V x = ax V 1 θ V 1 x x, θ = θ = W 1 x, u. Figure 1 gives a view of he relaios bewee all hese values a a give ieraio. value V x θ + β, x x W x, u V x V 1 x θ = W 1 x, u x x Figure 1. Relaios bewee oaios i he uli-sage case 2.1. Proof of covergece i he deeriisic case. We begi by showig soe regulariy ad oooiciy resuls for he value fucios ad heir approxiaios. Uder assupios H 1, we defie for = 0,..., T 1, ad for all x R, he exeded value fucio 11 Ṽ x = if u U x C x, u + V +1 f x, u. Noe ha he ifiu could be ae o Ũx U x as V +1 = whe f x, u / X +1. I is coveie o exed he defiiio o = T by defiig Ṽ T = V T. We also observe ha Ṽ V as hese are ideical o he doai of V. Lea 2.2. For = 0,..., T 1, i he value fucio V is covex ad Lipschiz coiuous o X ; ii V Ṽ V, ad β is defied;

8 8 P. GIRARDEAU, V. LECLERE, AND A. B. PHILPOTT iii he sequeces β N are bouded. Proof. i We firs show he covexiy ad Lipschiz coiuiy of V o X. We proceed by iducio bacward i ie. By assupio V T is covex ad Lipschiz coiuous o X T. Assue he resul is rue for V +1. The fucio Ṽ x is covex by lea 5.1. Now by ERCR, for ay x X, Ũ x. This iplies ha, for x X, for u Ũx, Ṽ x C x, u + V +1 f x, u < +. By H 1 3 ad he iducio hypohesis, for ay x X, u C x, u + V +1 f x, u is lower sei-coiuous, ad so he copacess of U x esures ha he ifiu i he defiiio of Ṽx is aaied, ad herefore Ṽx >. Ṽ is Lipschiz coiuous o X as X is a copac subse of he relaive ierior of is doai. Fially rearig ha V x = Ṽx if x X gives he coclusio. ii As observed above he iequaliy Ṽ V is iediae as he wo fucios are ideical o he doai of V. To show V for all = 0,..., T 1, β 1 Ṽ le us proceed by iducio forward i. Assue ha is defied ad V 1 Ṽ. Noe ha = V 0 Ṽ, so his is rue for = 1 β 0 is ever used. We ow defie, for all = 0,..., T 1 ad all x R, ˆV x = i C x, u + V 1 u Ũx +1 f x, u. By hypohesis o Ũ, ˆV is covex ad fiie o X which sricly coais X. Thus ˆV resriced o AffX is subdiffereiable a ay poi of X. Moreover by defiiio of β i 3 12 β ˆV AffX. Thus β is defied. By he iducio hypohesis ad iequaliy Ṽ+1 V +1 we have ha V+1 1 f V +1 f. Thus he defiiios of ˆV ad Ṽ yield 13 ˆV Ṽ. we have by 12 ha 14 θ + by 13. The defiiio of V V x = ax β, x x ˆV x Ṽx i 6 gives V 1 x, θ + β, x x which shows V x Ṽx by 14 ad he iducio hypohesis. Thus ii follows for all by iducio. iii Fially we show he boudedess of β N. By defiiio of β we have for all y R,

9 DECOMPOSITION OF MULTI-STAGE STOCHASTIC CONVEX PROGRAMS 9 15 V y V x + β, y x. Recall ha X = X +B δ, so subsiuig y = x + δβ i 15 wheever 2 β β 0 yields β 2 [ V x + δ β ] δ 2 β V x. We defie he copac subse X of do Ṽ as X x X we have ha x + δ V 2 x + δ 2 β β X β β. Cosequely, by ii, ax Ṽ < +. x X := X +B δ 2. Now as Moreover by cosrucio he sequece of fucios V N is icreasig, hus V x V 1 x i V 1 x >. x X Thus we have ha, for all N ad = 0,..., T 1, 16 β 2 ] [ax Ṽ i V 1 x. δ x X This coplees he proof. x X Corollary 2.1. Uder assupios H 1, he fucios V, = 0, 1,..., T 1, are α Lipschiz for soe cosa α for all N. Proof. By 6 ad 16 he subgradies of V are bouded by [ ] 2 α = ax ax Ṽ i V 1 x. =0,1,...,T 1 δ x X x X We ow prove ha boh he upper ad lower esiaes of V coverge o he exac value fucio uder assupios H 1. Theore 2.1. Cosider he sequece of decisios u geeraed by N 3 ad 6, where each u is iself a sequece of decisios i ie u = u 0,..., u T 1, ad cosider he correspodig sequece x of sae values. Uder assupios H 1, for ay = 0,..., T 1 we have ha: N li W x, u V x = 0 ad li V x V x = Proof. The deosraio proceeds by iducio bacwards i ie. A ie + 1, he iducio hypohesis is he secod saee of he heore. Tha is, li V +1 x +1 V+1 x +1 = 0. + I oher words he cus for he fuure cos fucio ed o be exac a x +1 as eds o. The iducio hypohesis is clearly rue a he las ie sage T for which we defied he approxiae value fucio VT o be equal o he ow ed value fucio V T.

10 10 P. GIRARDEAU, V. LECLERE, AND A. B. PHILPOTT Now le us cosider ie ad choose K ε o be he se of idices such ha: 1 V x + ε < W x, u < +, 2 V +1 x +1 θ +1 < ε 2. Noe ha he iducio hypohesis saes li + V +1 x +1 V +1 x +1 = 0 ad 10 gives V+1 x +1 = θ+1 so li V +1 x +1 θ+1 = 0. + Hece codiio 2 is saisfied for every large eough. Thus K ε is fiie if ad oly if he uber of idices saisfyig codiio 1 is fiie. Le 1, 2 be wo idices i K ε, 1 beig saller ha 2. Suppose ha a ew cu has jus bee added o he approxiae fuure cos fucio a ie + 1. Because of 6, we have ha for ay N ad ay x X +1 V+1 x θ+1 + β+1, x x +1. I follows ha he approxiae fuure cos evaluaed a x 2 C x 2, u 2 + θ β 1 +1, x 2 +1 x 1 +1 C = W 1 by defiiio 7 of W 1. Now, because 1 < 2, W 1 x 2, u 2 W 2 1 so 3 ad 10 give C x 2, u 2 + θ β 1 +1, x 2 +1 x 1 +1 Now, usig he fac ha 2 K ε, we have V x 2 x 2, u 2 x 2, u 2 x 2, u 2,, u 2 + V 1 +1, W 2 1 x 2, u 2 = i u Ũ x 2 i u x Ũ 2 = V x 2 + ε < W x 2, u 2 W 2 1 W saisfies x 2 which iplies C x 2, u 2 + θ β 1 +1, x 2 +1 x 1 +1 < W x 2, u 2 ε. Now, he defiiio 8 of W iplies W x 2, u 2 = C x 2, u 2. + V +1 x 2 +1 x 2 +1, x 2, u, u

11 so or 17 DECOMPOSITION OF MULTI-STAGE STOCHASTIC CONVEX PROGRAMS 11 ε < V +1 x 1 +1 Now, 1 K ε iplies ε < V +1 x 2 +1 θ 1 +1 β 1 +1, x 2 +1 x 1 +1, θ V +1 x 2 +1 V +1 x 1 +1 β 1 +1, x 2 +1 x 1 +1 V +1 x 1 +1 θ 1 +1 < ε 2, ad by i of lea 2.2 V is Lipschiz coiuous ad hus here is soe γ 1 such ha: V +1 x 2 +1 V +1 x 1 x +1 γ x Furherore iii of lea 2.2 esures ha β N is bouded. So here exiss a cosa γ 2 > 0 such ha: β 1 +1, x 2 +1 x 1 x +1 γ x Fially, aig γ = γ 1 + γ 2 we obai ε < ε 2 + γ x 2 +1 x 1 givig ε x 18 2 < γ 2 +1 x I follows ha K ε has o be fiie. Oherwise, sice he sae space X +1 is copac, we could fid a coverge subsequece of saes saisfyig 18, ad passig o he lii would lead o ε 2 0. This proves he firs par of he heore a ie. We ow have o show he iducio hypohesis, aely li V x V x = 0 + for ie. Recall 10 gives V x = θ = W 1 = C x, u +1, x, u + V 1 +1, x +1 Usig he defiiio 8 of W, we ca replace C x, u o ge V x = W x, u + x +1 V +1 x +1 We have jus deosraed above ha so V η = W x, u x = V x + η + V 1 +1 V x V 1 +1 The iducio hypohesis a ie + 1 gives V+1 x +1 V +1 x x +1 V +1 x +1. 0,

12 12 P. GIRARDEAU, V. LECLERE, AND A. B. PHILPOTT which by virue of Lea 5.2 wih V +1 replacig f iplies 3 li V +1 x +1 V+1 1 x +1 = 0 + so V x V x 0. which gives he resul. Theore 2.1 idicaes ha he lower approxiaio a each ieraio eds o be exac o he sequece of sae rajecories geeraed hroughou he algorih. This does o ea ha he fuure cos fucio will be approxiaed well everywhere i he sae space. I oly eas ha he approxiaio ges beer ad beer i he eighbourhood of a opial sae rajecory. 3. The sochasic case wih a fiie disribuio 3.1. Sochasic ulisage proble forulaio. Le us ow cosider ha he cos fucio ad dyaics a each ie are iflueced by a rado oucoe ha has a discree ad fiie disribuio. We wrie he proble o he coplee ree iduced by his disribuio. The se of all odes is deoed by N ad {0} is he roo ode. We deoe odes by ad. We rus ha he coex will dispel ay cofusio fro he use of ad as diesios of variables u ad x. A ode here represes a ie ierval ad a sae of he world which has probabiliy ha perais over his ie ierval. We say ha a ode is a ascede of if i is o he pah fro he roo ode o ode icludig. We will deoe a he se of all ascedes of, ad he deph of ode is oe less ha he uber of is ascedes. For sipliciy we ideify his wih a ie idex, alhough he resuls hold rue for sceario rees for which his is o he case. For every ode N \{0}, p represe is pare, ad r is se of childre odes. Fially L is he se of leaf odes of he ree. This gives he followig sochasic progra: 19a 19b 19c 19d 19e i x,u N \{L} r C x, u + L s.. x = f xp, u, N \{0}, x 0 is give, x X, N, u U x p, N \{0}. V x The reader should oe ha radoess ha appears i he cos ad i he dyaics is realized before he decisio is ae i his odel. Hece he corol affecig he soc 4 x is acually idexed by, a child ode 3 Corollary 2.1 esures he α Lipschiz assupio o V +1, ad he oher assupios are obviously verified. 4 We do o ae ay sagewise idepedece assupios o he rado variables ha affec he syse. Hece here is o reaso why variable x should be called a sae variable ad we prefer callig i a soc.

13 DECOMPOSITION OF MULTI-STAGE STOCHASTIC CONVEX PROGRAMS 13 of. Pu differely, he corol adaps o radoess: here are as ay corols as here are childre odes of. We have a fuure cos fucio for each ode N which is defied by 20 V x = r i C x, u + V f x, u. u U x }{{} W x,u We ae he followig assupios H 2 : 1 for all N, X is covex copac; 2 for all N \{0}, he ulifucio U is covex ad covex copac valued; 3 all fucios C, N \L, V, L, are covex lower seicoiuous proper fucios; 4 for all N \{0}, he fucios f are liear; 5 for all L, V is Lipschiz-coiuous o X ; 6 There exiss δ > 0 such ha for all odes N \L, a x X + Bδ, r, f x, U x X, b x X + Bδ, u U x, C x, u <. I geeral he fuure cos fucio a each ode ca be differe fro hose a oher odes a he sae sage. I he special case where he sochasic process defied by he sceario ree is sagewise idepede, he fuure cos fucio is ideical a every ode a sage. Soe for of sagewise idepedece is ypically assued i applicaios as i eables cus o be shared across odes a he sae sage, however we do o require his for our proof. The algorih ha we cosider is a exesio of he deeriisic algorih of he previous secio applied, a each ieraio, o a se of odes chose radoly i he ree a which we updae esiaes of he fuure cos fucio. We assue ha all oher odes have ull updaes, i he sese ha hey jus iheri he fuure cos fucio fro he previous ieraio. We ow describe he algorih forally. We sar he process wih ˆθ 0 =, ˆβ 0 = 0, for each N, ad ipose V = V for all odes L ad all N. We he carry ou a sequece of siulaios ad updaes of he fuure cos fucios as follows. Siulaio: Sarig a he roo ode, geerae socs ad decisios for all possible successors i oher words, visi he whole ree forward by solvig 20 wih V 1 isead of V. Deoe he obaied soc variables by x N ad he corol variables by u N \{0}. Also, for each ode N, ipose θ = V 1 x ad β x. Updae: Selec o-leaf odes 1, 2,..., I i he ree. For each i, x i is a rado variable which is equal o oe of he x. For each V 1

14 14 P. GIRARDEAU, V. LECLERE, AND A. B. PHILPOTT 21a 21b 21c 21d seleced ode i, ad for every child ode of ode i, solve: ˆθ = i C 1 x, u + V f x, u, X i u R x Aff s.. x = x i [ ˆβ ] u U x f x, u X As before ˆβ is a Lagrage uliplier a opialiy. We also defie he ulifucios Ũ : x {u U x f x, u X }. For each seleced ode i, replace he values θ i ad β i obaied durig he siulaio wih: θ i = ˆθ ad β i = ˆβ. i i r i r i Fially, we updae all fuure cos fucios. For every ode 22 V x := ax V 1 x, θ + β, x x, x X. Noe ha we acually oly updae fuure cos fucios o he seleced odes. Sice he cus we add a all oher odes are bidig o he curre odel by cosrucio i he siulaio, here is o poi i sorig he. Therefore, i pracice, oe does o eed o saple he whole sceario ree bu jus eough o aai all seleced odes. I our proof, we eed o loo a wha happes eve o he odes ha are o seleced. The way we selec odes a which o copue cus varies wih he paricular algorih ipleeaio. For exaple DOASA uses a sigle forward pass o selec odes, ad he copues cus i a bacward pass. We represe hese selecios of odes usig a selecio rado variable y = y N ha is equal o 1 if ode is seleced a ieraio ad 0 oherwise. This gives a selecio sochasic process y N, aig values i {0, 1} N \L, ha describes a se of odes i he ree a which we will copue ew cus i ieraio. We le F N deoe he filraio geeraed by y N. To ecopass algorihs such as DOASA ad SDDP he selecio sochasic process ca be viewed as cosisig of ifiiely ay fiie subsequeces, each cosisig of τ > 0 selecios cosisig for exaple of a sequece of selecios of odes i a bacward pass. This cao be doe arbirarily, ad he way ha y N is cosruced us saisfy soe idepedece codiios fro oe ieraio o he ex. Defiiio 1. Le τ be a posiive ieger. The process y N is called a τ-adissible selecio process if i N \L, N, κ {0,..., τ 1}, y τ+κ = 1 = a, y τ = y τ+1 = = y τ+κ 1 = 0; ad he process defied by 23 ỹ := ax{y τ, y τ+1, y τ+2,..., y τ+τ 1 }

15 DECOMPOSITION OF MULTI-STAGE STOCHASTIC CONVEX PROGRAMS 15 saisfies ii for all N \L, ỹ N is i.i.d. ad for all N, ad all N \L, ỹ is idepede of F τ 1 ; iii N \L, Pỹ = 1 > 0. Propery i guaraees ha whe τ > 1, he updaig of cuig plaes is doe bacwards bewee seps τ ad + 1τ. This eas ha if he liear approxiaio of he value fucio V is updaed a sep τ +κ he eiher i or ay approxiaio a ay asceda ode has bee updaed sice sep τ 1. This iplies, as show i lea 5.3, ha x τ+κ has o chaged sice he sep τ, i.e., if y τ+κ = 1 he x τ+κ = x τ. Propery ii provides he idepedece of he selecios ha we will use o prove covergece ad propery iii guaraees ha all odes are seleced wih posiive probabiliy. Wihou ay idepedece assupio i would be easy o creae a case i which he fuure cos fucio a a give ode is updaed oly whe he curre soc variable o his ode is i a give regio, for isace. I such a case he fuure cos fucio could o gaher ay iforaio abou he oher pars of he space ha he soc variable igh visi. I oher words, his idepedece assupio esures ha he values ha are opial ca be aaied a ifiie uber of ies. We rear ha here is o idepedece assupio over he odes for y N \L a fixed. Thus he selecio process could be forced o selec whole braches of he ree for exaple, as i would for he CUPPS algorih. More geerally, we have idepedece whe for fixed τ, y τ N is i.i.d ad he ex τ 1 selecio values are deeried deeriisically fro y τ, ore precisely if for all κ {0,..., τ 1}, here is a deeriisic fucio φ κ such ha y τ+κ = φ κ y τ. O he oher had we have idepedece whe he selecio subsequece y τ, y τ+1,..., y τ+τ 1 N is i.i.d. We will ae use of he followig defiiios, where r: 24 W x, u := C x, u + V f x, u 25 W x, u := C x, u + V f x, u I he case where ode N is seleced a ieraio, i oher words = i, hese defiiios he give ˆθ = i W 1 u Ũx x, u = W 1 x, u. Because for ay x X ad ay we have ha V 1 i follows ha ˆθ = i W 1 x, u u Ũx W 1 x + x V 1 x, i W 1 u Ũx x, u,, = i u Ũx, x x, u, which, usig covexiy of he value fucio of his laer proble ad he defiiios of ˆθ ad ˆβ, gives ˆθ ˆθ + ˆβ, x x,.

16 16 P. GIRARDEAU, V. LECLERE, AND A. B. PHILPOTT ad aig codiioal expecaios wih probabiliies, we obai θ θ + β, x x,. Sice by 22 i follows ha V 1 x { } = ax θ + β, x x < θ V 1 x, ad so 22 iplies oce agai, oly i he case whe = i 26 V x = ax V 1 x, θ = θ = W 1 x, u. r Noe ha because of he way x ad θ are defied durig siulaio for he o-seleced odes, we have V x = ax V 1 x, θ = θ, N Proof of covergece i he sochasic case. For every N \ L we ca defie uder assupios H 2 he exeded value fucio Ṽ x = if C x, u + V f x, u, u Ũx r ad we oe ha Ṽ is fiie o X. We ow sae a lea aalogous o lea 2.2. Lea 3.1. For every N, i he value fucio V is covex ad Lipschiz-coiuous o X ; ii V Ṽ V, ad β is defied; iii he sequeces β N are bouded, hus here is α such ha V α Lipschiz. Proof. We give oly a sech of he proof as i follows exacly he proof of is deeriisic couerpar lea 2.2. i By iducio bacward o he ree Ṽ, is covex ad fiie valued o X as he posiive su of covex fiie valued fucios, ad hus Lipschiz coiuous o X leadig o he resul as Ṽ = V o X. ii Assue ha for all N \L we have V 1 Ṽ. We defie, for a ode N \L x R, ˆV x = r i C x, u + V 1 u Ũx f x, u. By hypohesis o Ũ, ˆV is covex ad fiie o X hus is resricio o AffX is subdiffereiable o X. By defiiio ˆβ ˆV x, ad hus ˆβ is defied. By he iducio hypohesis ad iequaliy Ṽ V we have ha r, ˆV 1 f V f. is

17 DECOMPOSITION OF MULTI-STAGE STOCHASTIC CONVEX PROGRAMS 17 Thus defiiios of ˆV ad Ṽ yield ˆV Ṽ. By defiiio of β ad cosrucio of V we have ha V Ṽ. Iducio leads o iequaliy ii. iii Fially we show he boudedess of β N. As β is a elee of V x, we have V y V x + β, y x. 27 so subsiuig, if β 0, y = x + δβ 2 β β 2 δ [ V β x + δ 2 β V i 15 yields x ]. Thus we have ha, for all N ad N, β 2 [ ] ax Ṽ i V 1 x. δ x X +Bδ/2 x X Which eds he proof. We also ae use of he followig lea. Lea 3.2. Le be a ode i N \L. Suppose ha here is soe ieger τ > 0 such ha for all r we have li V x τ V τ x τ = 0. + The li + r W x τ, u τ V x τ = 0. Proof. Choose K ε o be he se of idices, uliples of τ, such ha: 1 V x + ε < r W x, u < +, 2 r V x θ < ε 2. Noe ha because of he hypohesis ad 26 he quaiy o he lef-had side of codiio 2 coverges o 0 as goes o ifiiy. Hece codiio 2 is saisfied for all large eough ad if K ε is fiie, he i is he uber of idices saisfyig codiio 1 ha has o be fiie. Because of 22, we have ha for ay N, ay child ode of, ad ay x X, 28 V x θ + β, x x. Le 1, 2 be wo idices i K ε, 1 beig saller ha 2. I follows fro 28 applied a = 1, ha he approxiae fuure cos evaluaed a x 2, u 2 saisfies C x 2, u 2 + θ 1 + β 1, x 2 x 1 C x 2, u 2 + V 1 x 2, = W 1 x 2, u 2.

18 18 P. GIRARDEAU, V. LECLERE, AND A. B. PHILPOTT ad 1 < 2 iplies Thus r W 1 C x 2, u 2 +θ 1 + x 2, u 2 W 2 1 x 2, u 2. β 1, x 2 x 1 W 2 1 x 2, u 2, r = i Now, sice 2 K ε, which gives r r r = V x 2. u Ũ x 2 i u Ũ x 2 V x 2 + ε < W x 2, u 2 r C x 2, u 2 + θ 1 + β 1, x 2 x 1 < r Now recall he defiiio 24 of W which gives r W 2 1 x 2, u, W x 2, u, W x 2, u 2 ε. W x 2, u 2 = C x 2, u 2 r + V f x 2, u 2. Subsiuig ad observig r x 2 = f x 2, u 2 gives ε < r V x 2 θ 1 β 1, x 2 x 1,

19 yieldig 29 ε < DECOMPOSITION OF MULTI-STAGE STOCHASTIC CONVEX PROGRAMS 19 r V x 1 Now, 1 K ε iplies r θ 1 + r V x 1 r θ 1 < ε 2. V x 2 V x 1 β 1, x 2 x 1. Furherore by i of lea 3.1 here exiss a cosa γ 1 > 0 such ha V x 2 V x 1 x γ 1 1 x 2, ad by iii of lea 3.1, here exiss a cosa γ 2 > 0 such ha β 1, x 2 x 1 x γ 1 2 x 2. Collecig ers ad aig γ = γ 1 + γ 2 we obai ε < ε 2 + γ x 2 x 1, givig 30 r ε 2 < γ r x 2 x 1. Hece K ε has o be fiie. Oherwise, because he space X is copac, we could fid a coverge subsequece of saes saisfyig 30, ad passig o he lii would lead o ε 2 0. Theore 3.1. Cosider he sequece of decisios u geeraed by N he above described procedure uder assupios H 2, where each u is iself a se of decisios o he coplee ree, ad cosider he correspodig sequece of sae values x. Assue ha he selecio process is τ- N adissible for soe ieger τ > 0. The we have ha, P-alos surely: li W x τ, u τ V x τ = 0. ad + r li V + x τ V τ x τ = 0. Proof. Because he selecio process for odes i he updae sep is sochasic, decisio variables as well as approxiae fuure cos fucios are sochasic hroughou he course of he algorih. Thus, durig he whole proof, all equaliies or iequaliies are ae P-alos surely.

20 20 P. GIRARDEAU, V. LECLERE, AND A. B. PHILPOTT The deosraio follows he sae approach as he proof of Theore 2.1. Le T be he axiu deph of he ree. We procede by bacward iducio o odes of fixed deph. The iducio hypohesis is li + V x τ V τ x τ = 0 for each ode of deph + 1. Sice for every leaf of he ree hose wo quaiies are equal, by defiiio, he iducio hypohesis is rue for every ode L. For ay arbirary ode of deph ha is o a leaf, lea 3.2 gives he firs saee of he heore. We ow have o show he iducio hypohesis, aely li + V x τ V τ x = 0 for every ode of deph. We sar by provig he resul for ieraios τ such ha is seleced i he ex τ 1 seps, i.e. such ha ỹ = 1. Defie κ {0,..., τ 1} such ha y τ+κ = 1. Recall ha by lea 5.3 we have x τ+κ = x τ. We have by 26 V τ+κ x τ+κ = V τ+κ = r r = r x τ i u Ũxτ i u Ũxτ W τ 1 { { W τ+κ 1 W τ 1 x τ, u τ x τ, u } x τ, u } which iplies V τ+κ x τ r = r [ C x τ, u τ [ W x τ, u τ + We have jus deosraed above ha η τ = W x τ, u τ Thus, V τ+κ x τ r V x τ + η τ + r V τ 1 + V τ 1 V x τ x τ V τ 1 x τ V ], x τ 0. x τ V ]. x τ.

21 DECOMPOSITION OF MULTI-STAGE STOCHASTIC CONVEX PROGRAMS 21 However he iducio hypohesis gives li V x τ V τ + x τ = 0 ad by virue of lea 5.2 wih V replacig f 5 iplies li V x τ V τ 1 x τ = 0. + ad hus Bu which gives η τ := η τ + r 31 V V τ 1 x τ V x τ V x τ V τ+κ x τ V x τ + η τ x τ V τ+κ Thus lea 5.2 applied wih κ = τ gives V x τ V τ+κ τ x τ x τ 0. ỹ=1 0, ỹ=1 0. ỹ=1 ad by oooiciy we have V τ+κ τ V τ V, which fially yields 32 V x τ V τ x τ 0. ỹ=1 Now we prove ha he values also coverge for he ieraios such ha ỹ = 0, i.e. he ieraios for which ode is o seleced bewee sep τ ad sep + 1τ 1. By coradicio, suppose he values do o coverge. The by lea 5.2 we have ha V x τ V τ 1 x τ does o coverge o 0. I follows ha here is soe ε > 0 such ha K ε is ifiie where 33 K ɛ := { N V x τ V τ 1 x τ ε}. Le z j deoe he j-h elee of he se {y τ K ε }. Noe ha he rado variables V τ 1 ad x τ are easurable wih respec o F τ 1 := σ y <τ, ad hus so is 1 Kε fro which ỹ is idepede. Moreover he σ algebra geeraed by he pas realisaios of ỹ is icluded i F τ 1. This iplies by lea 5.4 ha rado variables z j j N are i.i.d. ad share he sae probabiliy law as ỹ. 0 Accordig o he Srog Law of Large Nubers [5, page 294] applied o he rado sequece z j j N, we should he have 1 N N j=1 z j N + Ez 1 = Eỹ 0 = P ỹ 0 = 1 > 0. 5 Lea 3.1 iii provides a Lipschiz codiio o V.

22 22 P. GIRARDEAU, V. LECLERE, AND A. B. PHILPOTT However, K ε {ỹ = 1} is fiie because of 31 hus we ow ha here is oly a fiie uber of idices j such ha z j = 1, he res beig equal o 0. So 1 N N j=1 z j N + 0, which is a coradicio. This shows ha V x τ V τ 1 ad oooiciy shows ha, V x τ V τ which coplees he iducio. x τ x τ ỹ = ỹ= Applicaio o ow algorihs. I order o illusrae o our resul we will apply i o wo well ow algorihs. For sipliciy we will assue ha he ree represes a T -sep sochasic decisio proble i which every leaf of he ree is of deph T. We firs defie he CUPPS algorih [1] i his seig. Here a each ajor ieraio we choose a T 1-sep sceario ad copue he opial rajecory while a he sae ie updaig he value fucio for each ode of he brach. I our seig, his uses a 1-adissible selecio process y N defied by a i.i.d. sequece of rado variables, wih y 0 selecig a sigle brach of he ree. Theore 3.1 shows ha for every ode he upper ad lower boud coverges, ha is li W x +, u V x = 0 r ad li V x V x = 0. + We ow place he SDDP algorih [9] ad DOASA algorih [10] i our fraewor. There are wo phases i each ajor ieraio of he SDDP algorih, aely a forward pass, ad a bacward pass of T 1 seps. Give a curre polyhedral ouer approxiaio of he Bella fucio N \L, a ajor ieraio of he SDDP algorih cosiss i: V 1 selecig uiforly a uber N of scearios N = 1 for DOASA; siulaig he opial sraegy for he proble, ha is solvig proble 21 o deerie a rajecory for each sceario x {0,...,T 1} where {0,...,T 1} defies oe of he seleced scearios; For = T 1 dow o = 0 for each sceario solvig proble 21 wih V isead of V 1, ad defiig V x = ax{v 1 x, θ + β, x x }. SDDP fis io our fraewor as follows. Give N, we defie he T 1- adissible selecio process, y T 1 N by a i.i.d. sequece of rado

23 DECOMPOSITION OF MULTI-STAGE STOCHASTIC CONVEX PROGRAMS 23 variables wih y 0 selecig uiforly a se of N pre-leaves i.e. odes whose childre are leaves of he ree. The for κ {1,..., T 2}, N, N \L, we defie T 1+κ y := { 1 if here exis r such ha y T 1+κ 1 = 1 0 oherwise. This algorih defies a SDDP algorih wih N radoly sapled forward passes per sage. The cu sharig feaure used i SDDP whe rado variables are sagewise idepede ca be easily icorporaed. Sice for every ode of he ree excepig he leaves here is a κ such ha T 1+κ Py = 1 > 0, heore 3.1 guaraees he covergece of he lower boud of he SDDP algorih for every ode. 4. Discussio The covergece resul we have proved assues ha we copue ew cus a sceario-ree odes ha are seleced idepedely fro he hisory of he algorih. This eables us o use he Srog Law of Large Nubers i he proof. Previous resuls for ulisage sochasic liear prograig [10] require a selecio process ha visis each ode i he ree ifiiely ofe, which is a weaer codiio ha idepedece, sice i follows by he Borel-Caelli Lea [5, page 288]. A exaple would be a deeriisic roud-robi selecio. We do o have a proof of covergece for such a process i he oliear case. I is ipora o observe ha he polyhedral for of V ha was exploied i he proof [10] is abse i our proble, ad his differece could prove o be criical. The covergece resul is proved for a geeral sceario ree. I SDDP algorihs, he rado variables are usually assued o be sagewise idepede or ade so by addig sae variables. This eas ha he fuure cos fucios V x are he sae a each ode a deph. This allows cuig plaes i he approxiaios o be shared across hese odes. The covergece resul we have show here applies o his siuaio as a special case. Observe ha he class of algorihs covered by our resul is larger ha he exaples preseed i he lieraure. For exaple a algorih where we selec radoly a ode o he whole ree, ad he updae bacwards fro here is prove o coverge. Oe could also hi of cobiig SDDP ad CUPPS algorihs. I he case where oe would wa o add cus a differe odes i he ree i he updae sep of our procedure, he solvig of he subprobles ca be doe i parallel. This is he case i CUPPS, where a whole brach of he ree is seleced a each ieraio. I also allows us o cosider differe selecio sraegies, where odes a a give ieraio could be seleced hroughou he ree depedig o soe crieria defied by he user. I he firs few ieraios, his could highly icrease efficiecy of he approxiaio ad, because he solvig of he subprobles ca be parallelized, would o be very ie-cosuig. Oe should bear i id however ha, a soe poi, he algorih has o coe bac o a appropriae selecio procedure, i.e. oe

24 24 P. GIRARDEAU, V. LECLERE, AND A. B. PHILPOTT ha saisfies he idepedece assupio, i order o esure covergece of he algorih. 5. Appedix: Techical leas Lea 5.1. If J : R R { } is covex, U : R R is covex he φx := i u Ux Ju is covex. Moreover if J is lower-seicoiuous, ad U copac o-epy valued, he he ifiu exiss ad is aaied. Proof. We defie Iu, x := { 0 if u Ux + oherwise The φx = i u R Ju + Iu, x. Fix u 1 Ux 1 ad u 2 Ux 2, he for every λ [0, 1] λu λu 2 Uλx λx 2 by covexiy of U. This shows ha Iu, x is covex, whereby φ is covex as he argial fucio of a joily covex fucio. The secod par of he lea follows iediaely fro he copacess U ad lower-seicoiuiy of J. Lea 5.2. Suppose f is covex ad X is copac, ad suppose for ay ieger κ, he sequece of α-lipschiz covex fucios f, N saisfies f κ x f x f x, for all x X. The for ay ifiie sequece x X li f x f x = 0 li f x f κ x = Proof. If li + f x f κ x = 0 he poiwise oooiciy of f shows ha li + f x f x = 0. For he coverse, suppose ha he resul is o rue. The here is soe subsequece f l l N ad x l X wih 34 li f x l f l x l = 0 + ad ε > 0, L N wih f x l f l κ x l > ε for every l > L. Sice X is copac, we ay assue by aig a furher subsequece ha x l l N coverges o x X. For sufficiely large l, he Lipschiz coiuiy of f l ad f l κ gives f l x f l x l α x l x < ε 4, f l κ x l f l κ x α x l x < ε 4, ad 34 iplies ha for sufficiely large l f x l f l x l < ε 4.

25 DECOMPOSITION OF MULTI-STAGE STOCHASTIC CONVEX PROGRAMS 25 I follows ha f l x f l κ x = f l x f l x l > ε 4, x l +f l x l f +f x l f l κ x l +f l κ x l f l κ x sice f x l f l κ x l is greaer ha ε, ad he oher hree ers each have a absolue value saller ha ε/4. Cosequely f l x > f l κ x + ε 4, for ifiiely ay l which coradics he fac ha f x is bouded above by fx. Lea 5.3. If y N is a τ-adissible selecio process he for all N, κ {0,..., τ 1}, ad all N \L we have { x τ+κ = x τ, y τ+κ = 1 = V τ+κ 1 = V τ 1 if 1. Proof. Le, ad κ be such ha y τ+κ = 1. Le a := 0, 1,..., be he sequece of ascedes of :=, i.e. 0 is he roo ode, ad for all <, = p +1. Defie he hypohesis H, κ : a = x τ x τ+κ, b V τ+κ 1 = V τ 1, if 1. Le κ < κ ad assue ha for κ ad all, H, κ holds rue. This is saisfied for κ = 0. Le < ad assue H, κ + 1 is rue. Sice x 0 is fixed, his is saisfied for = 0. By defiiio of u τ+κ we have { u τ+κ +1 C +1 x τ+κ +1, u +1 arg i u Ũ x τ+κ +1 hus by H, κ + 1 a we have u τ+κ arg i u Ũ x τ + V τ+κ +1 f +1 x τ+κ +1, u } { C +1 x τ, u + V τ+κ +1 f +1 x τ, u}. Now as +1 is a ascede of ad κ < κ by propery i of defiiio 1, we have ha he represeaio of V +1 is o updaed a ieraio κ, i.e. V τ+κ +1 = V τ+κ Ad hus H + 1, κ b gives H + 1, κ + 1 b, i.e. herefore u τ+κ arg i u Ũ x τ V τ+κ +1 = V τ 1 +1, { C +1 x τ, u + V τ 1 +1 f +1 x τ, u},

26 26 P. GIRARDEAU, V. LECLERE, AND A. B. PHILPOTT ad cosequely 6 u τ+κ = u τ +1, which gives by defiiio H +1, κ +1 a. Iducio o gives H, κ +1 for all, ad iducio o κ esablishes H, κ for all κ {0, 1,..., τ}. Lea 5.4. Le w N be a sochasic process wih value i {0, 1} adaped o a filraio F N, such ha he uber of ers ha are o-zero is alos surely ifiie. Le y N be a sequece of i.i.d discree rado variables. Defie he filraio B := σ F σy 1,..., y 1 ad assue ha for all N, y is idepede of B. Le j deoe he j h ieger such ha w = 1, i.e. 0 = 0 ad for all j > 0, j := i{l > j 1 w l = 1}. Fially we defie for all j > 0, he j h value of y such ha w = 1, i.e. z j := y j. The z N is a sequece of i.i.d. rado variables equal i law o y 0. Proof. Le Y deoe he suppor of y 0. We sar wih z 1. For i Y, Pz 1 = i = P { l < l, w l = 0} {w l = 1} {y l = i} l=1 because of he {0,1}defiiio, ad hece = P {y l = i} P { l < l, w l = 0} {w l = 1} l=1 by idepedece, so ha as y l is i.i.d., ad so = P {y 0 = i} P { l < l, w l = 0} {w l = 1} = P {y 0 = i} l=1 as he sequece w N us coai a 1 alos surely. Thus z 1 is equal i law o y 0. Now suppose ha z = z 1,..., z is a sequece of i.i.d. rado variables. Le 1,, be ordered iegers, ad fix b {0, 1} ad i Y. 6 This requires ha he choice of opial corol aog he se of iiizers is deeriisic say ha wih iiu or.

27 DECOMPOSITION OF MULTI-STAGE STOCHASTIC CONVEX PROGRAMS 27 We have P {z = b} {z +1 = i} {1 = 1,..., = } = = ν=0 ν=0 P {z = b} {1 = 1,..., = } {y ν = i} {ν = + 1} P y ν = i P {z = b} {1 = 1,..., = } {ν = + 1} = P y 0 = i P {z = b} {1 = 1,..., = }. For he las equaliy we have used he fac ha y is i.i.d. ad he fac ha + 1 is alos surely fiie ad hus {ν = + 1} ν N is a pariio of he se of eves. Suig over he possible realisaios of 1,...,, we obai P {z = b} {z +1 = i} = P z = b Py 0 = i. Now suig over he possible realisaios of b shows ha z +1 is equal i law o y 0. Thus P {z = b} {z +1 = i} = P {z = b} {y 0 = i} = P z = b Py 0 = i = P z = b Pz +1 = i which shows ha z +1 is idepede of z ad equal i law o y 0. Iducio over coplees he proof. Acowledges. The firs auhor was suppored i his research by a gra fro he OSIRIS Depare a EDF R&D, Frace. The auhors also wish o ha Kegy Bary for useful discussios o earlier versios of his paper. Refereces [1] Z. L. Che ad W. B. Powell. A coverge cuig-plae ad parialsaplig algorih for ulisage liear progras wih recourse. Joural of Opiizaio Theory ad Applicaios, pages , [2] C. J. Doohue. Sochasic Newor Prograig ad he Dyaic Vehicle Allocaio Proble. PhD disseraio, Uiversiy of Michiga, A Arbor, Michiga, [3] C. J. Doohue ad J. R. Birge. The abridged esed decoposiio ehod for ulisage sochasic liear progras wih relaively coplee recourse. Algorihic Operaios Research, 1:20 30, [4] E.C. Fiardi ad E.L. da Silva. Solvig he hydro ui coie proble via dual decoposiio ad sequeial quadraic prograig. IEE Trasacios o Power Syses, 212: , 2006.

28 28 P. GIRARDEAU, V. LECLERE, AND A. B. PHILPOTT [5] G.R. Grie ad D.R. Sirzaer. Probabiliy ad Rado Processes: Secod Ediio. Oxford Uiversiy Press, [6] M. Hidsberger ad A. B. Philpo. ReSa: A ehod for solvig ulisage sochasic liear progras. SPIX Sochasic Prograig Syposiu, Berli, [7] J.E. Kelley Jr. The cuig-plae ehod for solvig covex progras. Joural of he Sociey for Idusrial & Applied Maheaics, 84: , [8] K. Liowsy ad A. B. Philpo. O he covergece of saplig-based decoposiio algorihs for ulisage sochasic progras. Joural of Opiizaio Theory ad Applicaios, 125: , [9] M. V. F. Pereira ad L. M. V. G. Pio. Muli-sage sochasic opiizaio applied o eergy plaig. Maheaical Prograig, 522: , [10] A. B. Philpo ad Z. Gua. O he covergece of sochasic dual dyaic prograig ad relaed ehods. Operaios Research Leers, 364: , [11] A. B. Philpo ad Z. Gua. A ulisage sochasic prograig odel for he New Zealad dairy idusry. Ieraioal Joural of Producio Ecooics, 1342: , [12] A. Ruszczyńsi. Noliear Opiizaio. Priceo Uiversiy Press, [13] A. Shapiro. Aalysis of sochasic dual dyaic prograig ehod. Europea Joural of Operaioal Research, 2091:63 72, P. Girardeau, Uiversiy of Auclad, Dep. of Egieerig Sciece, New Zealad E-ail address: pierre.girardeau@esa.org V. Leclere, CERMICS, École des Pos ParisTech, Frace E-ail address: vice.leclere@cerics.epc.fr A. B. Philpo, Uiversiy of Auclad, Dep. of Egieerig Sciece, New Zealad E-ail address: a.philpo@auclad.ac.z