INEXACT CUTS FOR DETERMINISTIC AND STOCHASTIC DUAL DYNAMIC PROGRAMMING APPLIED TO CONVEX NONLINEAR OPTIMIZATION PROBLEMS

Transcription

1 INEXACT CUTS FOR DETERMINISTIC AND STOCHASTIC DUAL DYNAMIC PROGRAMMING APPLIED TO CONVEX NONLINEAR OPTIMIZATION PROBLEMS Vincen Guigues School of Applied Mahemaics, FGV Praia de Boafogo, Rio de Janeiro, Brazil Absrac. We inroduce an exension of Dual Dynamic Programming (DDP) o solve convex nonlinear dynamic programming equaions. We call his exension Inexac DDP (IDDP) which applies o siuaions where some or all primal and dual subproblems o be solved along he ieraions of he mehod are solved wih a bounded error (inexacly). We show ha any accumulaion poin of he sequence of decisions is an approximae soluion o he dynamic programming equaions. When hese errors end o zero as he number of ieraions goes o infiniy, we show ha IDDP solves he dynamic programming equaions. We exend he analysis o sochasic convex nonlinear dynamic programming equaions, inroducing Inexac Sochasic Dual Dynamic Programming (ISDDP), an inexac varian of SDDP corresponding o he siuaion where some or all problems o be solved in he forward and backward passes of SDDP are solved approximaely. We also show he almos sure convergence of ISDDP for vanishing errors. Keywords: Sochasic programming; Inexac cus for value funcions; Bounding ε-opimal dual soluions; SDDP; Inexac SDDP. AMS subjec classificaions: 90C15, 90C Inroducion Sochasic Dual Dynamic Programming (SDDP) is a sampling-based exension of he nesed decomposiion mehod [2] o solve some T -sage sochasic programs, pioneered by [13]. Originally, in [13], i was presened o solve Mulisage Sochasic Linear Programs (MSLPs). Since many real-life applicaions in, e.g., finance and engineering, can be modelled by such problems, unil recenly mos papers on SDDP and relaed decomposiion mehods, especially heory papers, focused on enhancemens of he mehod for MSLPs. These enhancemens include risk-averse SDDP [16], [8] [7], [14], [11], [17] and a convergence proof in [15]. However, SDDP can be applied o solve nonlinear sochasic convex dynamic programming equaions. For such problems, he convergence of he mehod was proved recenly in [3] for risk-neural problems, in [4] for risk-averse problems, and in [9] for a regularized varian implemened on a nonlinear dynamic porfolio model wih marke impac coss. To he bes of our knowledge, all sudies on SDDP rely on he assumpion ha all primal and dual subproblems solved in he forward and backward passes of he mehod are solved exacly. However, when hese mehods are applied o nonlinear problems, only approximae soluions are available for he subproblems solved in he forward and backward passes of he algorihm. In his conex, he objecive of his paper is o design varians of DDP (he deerminisic counerpar of SDDP) and SDDP o solve nonlinear convex dynamic programming equaions ha ake his fac ino accoun. We call he corresponding varians of DDP and SDDP Inexac DDP (IDDP) and Inexac SDDP (ISDDP). I should be menioned, however, ha here is anoher moivaion for considering inexac varians of DDP and SDDP. Indeed, i is known (see for insance he numerical experimens in [6], [5]) ha for he firs ieraions of he mehod and especially for he firs sages, he cus compued can be quie disan from he corresponding recourse funcion in he neighborhood of he rial poin a which he cu was compued, so his cu is quickly dominaed by oher more relevan cus in his neighborhood. Therefore, i makes sense o ry and solve more quickly and less accuraely (inexacly) all subproblems of he forward and backward passes corresponding o he 1

2 2 firs ieraions, especially for he firs sages, and o increase he precision of he compued soluions as he algorihm progresses. While he idea behind IDDP and ISDDP is simple and he moivaions clear, he descripion and convergence analysis of IDDP and ISDDP require solving he following problems of convex analysis, ineresing per se, and which, o he bes of our knowledge, had no been discussed so far in he lieraure: SDDP for nonlinear programs relies on a formula for he subdifferenial of he value funcion Q(x) of a convex opimizaion problem of form: infy R n f(y, x) (1.1) Q(x) = y Y : Ay + Bx = b, g(y, x) 0, where Y R n is nonempy and convex, f : R n R m R + } is convex, lower semiconinuous, and proper, and he componens of g are convex lower semiconinuous funcions. Formulas for he subdifferenial Q(x) are given in [4]. These formulas are based on he assumpion ha primal and dual soluions o (1.1) are available. When only approximae ε-opimal primal and dual soluions are available for (1.1) wrien wih x = x, we derive formulas for affine lower bounding funcions C for Q, ha we call inexac cus, such ha he disance Q( x) C( x) beween he values of Q and of he cu a x is bounded from above by a known funcion ε 0 of he problem parameers. Of course, we would like ɛ 0 o be as small as possible and ε 0 = 0 when ε = 0. Two cases are considered: (i) he case when he feasible se of (1.1) is Y, i.e., when he argumen x of Q appears only in he objecive funcion of (1.1). In his siuaion, formulas for inexac cus are given in Proposiion 2.2, wih a refined bound on ε 0 given in Proposiions 2.3 and 2.5 under an addiional assumpion. (ii) he general case of a value funcion of form (1.1). The corresponding inexac cus are given in Proposiions 2.7 and 2.8. We provide condiions ensuring ha ε-opimal dual soluions o a convex nonlinear opimizaion problem are bounded. Proposiion 3.1 gives an analyic formula for an upper bound on he norm of hese ε-opimal dual soluions. We show in Proposiions 4.5 and 4.6 ha if we compue inexac cus for a sequence (Q k ) of value funcions of he form (1.1) (wih objecive funcions f k of special srucure) a a sequence of poins (x k ) on he basis of ε k -opimal primal and dual soluions wih lim k + ε k = 0, hen he disance beween he inexac cus and he value funcions a hese poins x k converges o 0 oo. This resul is very naural (see Proposiions 4.5 and 4.6) bu some consrain qualificaion condiions are needed. When opimizaion problem (1.1) is linear, i.e., when Q is he value funcion of a linear program, inexac cus can easily be obained from approximae dual soluions since he dual objecive is linear in his case. This observaion was used in [18] where inexac cus are combined wih Benders Decomposiion [1] o solve wo-sage sochasic linear programs. In his sense, our work can be seen as an exension of [18] where wo-sage sochasic linear problems are considered whereas ISDDP applies o mulisage sochasic nonlinear problems. In ineger programming, inexac maser soluions are also commonly used in Benderslike mehods [12], including in SDDiP, a varian of SDDP o solve mulisage sochasic linear programs wih ineger variables inroduced in [19]. The ouline of he sudy is as follows. Secion 2 provides analyic formulas for compuing inexac cus for a value funcion of an opimizaion problem of he form (1.1). In Secion 3, we provide an explici bound for he norm of ε-opimal dual soluions. Secion 4 inroduces and sudies he IDDP mehod. The class of problems o which his mehod applies is described in Subsecion 4.1. The deailed IDDP algorihm is given in Subsecions while Subsecion 4.5 sudies he convergence of IDDP. For a problem wih T periods, when noises (error erms quanifying he inexacness) are bounded, by, say, ε, we show in Theorem T (T +1) 4.7 and Corollary 4.8 ha any accumulaion poin of he sequence of decisions is a 2 ( δ + ε)-opimal soluion o he problem where δ is an upper bound on he disance beween he value of (heoreical) exac cus and he value of our inexac cus a he rial poins compued by he algorihm. I is ineresing o see he quadraic dependence of he global error wih respec o he number of periods and he linear dependence wih respec o noises. When noises are vanishing we prove ha IDDP solves he nonlinear dynamic programming equaions (see Theorem 4.7). Secion 5 inroduces and sudies ISDDP. The class of problems o which ISDDP applies is given in Subsecion 5.1. A deailed descripion of ISDDP is given

3 3 in Subsecion 5.2 and is convergence is sudied in Subsecion 5.3. More precisely, Theorem 5.3 shows he convergence of he mehod when he noises vanish. We use he following noaion and erminology: - The usual scalar produc in R n is denoed by x, y = x T y for x, y R n. The corresponding norm is x = x 2 = x, x. - ri(a) is he relaive inerior of se A. - B n (x 0, r) = x R n : x x 0 r} for x 0 R n, r 0. - dom(f) is he domain of funcion f. - Diam(X) = max x,y X x y is he diameer of X. - N A (x) is he normal cone o A a x. - X ε := X + εb n (0, 1) is he ε-faening of he se X R n. - C(X ) is he se of coninuous real-valued funcions on X, equipped wih he norm f X = sup x X f(x). - C 1 (X ) is he se of real-valued coninuously differeniable funcions on X. - span(x) is he linear span of se of vecors X and Aff(X) is he affine span of X. 2. Compuing inexac cus for he value funcion of a convex opimizaion problem Le Q : X R be he value funcion given by infy R n f(y, x) (2.2) Q(x) = y S(x) := y Y : Ay + Bx = b, g(y, x) 0}. Here, X R m and Y R n are nonempy, compac, and convex ses, and A and B are respecively q n and q m real marices. We will make he following assumpions which imply, in paricular, he convexiy of Q given by (2.2): (H1) f : R n R m R + } is lower semiconinuous, proper, and convex. (H2) For i = 1,..., p, he i-h componen of funcion g(y, x) is a convex lower semiconinuous funcion g i : R n R m R + }. In wha follows, we say ha C is a cu for Q if C is an affine funcion of x such ha Q(x) C(x) for all x X. We say ha he cu is exac a x X if Q( x) = C( x). Oherwise, he cu is said o be inexac. In his secion, our basic goal is, given x X and ε-opimal primal and dual soluions of (2.2) wrien for x = x, o derive an inexac cu C(x) for Q a x, i.e., an affine lower bounding funcion for Q such ha he disance Q( x) C( x) beween he values of Q and of he cu a x is bounded from above by a known funcion of he problem parameers. Of course, when ε = 0, we will check ha Q( x) = C( x). We firs recall from [4] how o compue exac cus for Q when opimal primal and dual soluions of (2.2) are available Formula for he subdifferenial of he value funcion of a convex opimizaion problem. Consider for (2.2) he dual problem (2.3) sup (λ,µ) R q R p + for he dual funcion (2.4) θ x (λ, µ) = inf y Y θ x (λ, µ) f(y, x) + λt (Ay + Bx b) + µ T g(y, x). We denoe by Λ(x) he se of opimal soluions of he dual problem (2.3) and we use he noaion Sol(x) := y S(x) : f(y, x) = Q(x)} o indicae he soluion se o (2.2). The descripion of he subdifferenial of Q is given in he following lemma: Lemma 2.1. Consider he value funcion Q given by (2.2) and ake x 0 X such ha S(x 0 ). Le Assumpions (H1) and (H2) hold and assume he Slaer-ype consrain qualificaion condiion: here exiss ( x, ȳ) X ri(y ) such ha Aȳ + B x = b and (ȳ, x) ri(g 0}).

4 4 We also assume ha here exiss ε > 0 such ha Y X ε dom(f). Then s Q(x 0 ) if and only if } (0, s) f(y 0, x 0 ) + [A T ; B T ]λ : λ R q (2.5) } + µ i g i (y 0, x 0 ) : µ i 0 + N Y (y 0 ) 0}, i I(y 0,x 0) where y 0 is any elemen in he soluion se Sol(x 0 ) and wih } I(y 0, x 0 ) = i 1,..., p} : g i (y 0, x 0 ) = 0. Moreover, he se x X Q(x) is bounded. In paricular, if f and g are differeniable, hen Q(x 0 ) = x f(y 0, x 0 ) + B T λ + } µ i x g i (y 0, x 0 ) : (λ, µ) Λ(x 0 ). i I(y 0,x 0) Proof. See he proofs of Lemma 2.1 and Proposiion 2.1 in [4]. Le us now discuss he compuaion of inexac cus for Q given by (2.2). We sar wih he case where he argumen x of he value funcion appears only in he objecive funcion of (2.2) Fixed feasible se. As a special case of problem (2.2), le Q : X R be he value funcion given by infy R n f(y, x) (2.6) Q(x) = y Y where X, Y are convex, compac, and nonempy ses. We pick x X and denoe by ȳ Y an opimal soluion of (2.6) wrien for x = x: (2.7) Q( x) = f(ȳ, x). Using Lemma 2.1, if f is differeniable, we have ha x f(ȳ, x) Q( x). If insead of an opimal soluion ȳ of (2.6) we only have a hand an approximae ε-opimal soluion ŷ(ε) i is naural o replace x f(ȳ, x) by x f(ŷ(ε), x). The inexac cu from Proposiion 2.2 below will be expressed in erms of he funcion l 1 : Y X R + given by (2.8) l 1 (ŷ, x) = min y Y yf(ŷ, x), y ŷ = max y Y yf(ŷ, x), ŷ y. Proposiion 2.2. Le x X and le ŷ(ε) Y be an ɛ-opimal soluion for problem (2.6) wrien for x = x wih opimal value Q( x), i.e., Q( x) f(ŷ(ε), x) ε. Assume ha f is differeniable and convex on Y X. Then seing η(ε) = l 1 (ŷ(ε), x), he affine funcion (2.9) C(x) := f(ŷ(ε), x) η(ε) + x f(ŷ(ε), x), x x is a cu for Q a x, i.e., for every x X we have Q(x) C(x) and he quaniy η(ε) is an upper bound for he disance Q( x) C( x) beween he values of Q and of he cu a x. Proof. For every (x, y) X Y using he convexiy of f we have f(y, x) f(ŷ(ε), x) + x f(ŷ(ε), x), x x + y f(ŷ(ε), x), y ŷ(ε). Minimizing over y in Y on each side of he above inequaliy we ge for every x X (2.10) Q(x) C(x) = f(ŷ(ε), x) l 1 (ŷ(ε), x) + x f(ŷ(ε), x), x x which shows ha C is a valid cu for Q. Finally, since ŷ(ε) Y, we have f(ŷ(ε), x) Q( x) and (2.11) C( x) Q( x) = f(ŷ(ε), x) l 1 (ŷ(ε), x) Q( x) l 1 (ŷ(ε), x). We now refine he bound l 1 (ŷ(ε), x) on Q( x) C( x) given by Proposiion 2.2 making he following assumpion: (H3) f is differeniable on Y X and here exiss M 1 > 0 such ha for every x X, y 1, y 2 Y, we have y f(y 2, x) y f(y 1, x) M 1 y 2 y 1.

5 5 Proposiion 2.3. Le x X and le ŷ(ε) Y be an ɛ-opimal soluion for problem (2.6) wrien for x = x wih opimal value Q( x), i.e., Q( x) f(ŷ(ε), x) ε. Then seing η(ε) = l 1 (ŷ(ε), x), if f is differeniable and convex on Y X he affine funcion C(x) given by (2.9) is a cu for Q a x. Moreover, if Assumpion (H3) holds, hen seing (2.12) ε 0 = l1(ŷ(ε), x) 2M 1Diam(Y ) (2M 2 1 Diam(Y ) 2 l 1 (ŷ(ε), x)) if l 1 (ŷ(ε), x) M 1 Diam(Y ) 2, 1 2 l 1(ŷ(ε), x) oherwise, he disance Q( x) C( x) beween he values of Q and of he cu a x is a mos ε 0. Proof. We already know from Proposiion 2.2 ha C is an inexac cu for Q. I remains o show ha if Assumpion (H3) holds hen (2.13) C( x) Q( x) = f(ŷ(ε), x) l 1 (ŷ(ε), x) Q( x) ε 0. Le y Y be such ha l 1 (ŷ(ε), x) = y f(ŷ(ε), x), ŷ(ε) y. Using (H3), for every 0 1, we have f(ŷ(ε) + (y ŷ(ε)), x) f(ŷ(ε), x) + y ŷ(ε), y f(ŷ(ε), x) M 1 2 ŷ(ε) y 2 f(ŷ(ε), x) l 1 (ŷ(ε), x) M 1 2 ŷ(ε) y 2. By convexiy of Y, since ŷ(ε), y Y, for every 0 1 we have ha ŷ(ε) + (y ŷ(ε)) Y and he above relaion yields [ Q( x) f(ŷ(ε), x) max l 1 (ŷ(ε), x) M 1Diam(Y ) 2 2]. [ ] If l 1 (ŷ(ε), x) M 1 Diam(Y ) 2 hen max 0 1 l 1 (ŷ(ε), x) 1 2 M 1Diam(Y ) 2 2 = 1 l 1(ŷ(ε), x) 2 2 and M 1Diam(Y ) 2 (2.14) Q( x) f(ŷ(ε), x) 1 l 1 (ŷ(ε), x) 2 2 M 1 Diam(Y ) 2. [ ] If l 1 (ŷ(ε), x) M 1 Diam(Y ) 2 hen max 0 1 l 1 (ŷ(ε), x) 1 2 M 1Diam(Y ) 2 2 = l 1 (ŷ(ε), x) 1 2 M 1Diam(Y ) 2 and (2.15) Q( x) f(ŷ(ε), x) 1 2 l 1(ŷ(ε), x). Combining (2.14) and (2.15) wih (2.12) gives (2.13) and complees he proof. Remark 2.4. As expeced, if ε = 0 hen ŷ(ε) is an opimal soluion of problem (2.6) wrien for x = x and he firs order opimaliy condiions ensure ha l 1 (ŷ(ε), x) = 0, meaning ha he cu given by Proposiion 2.2 is exac. Oherwise i is inexac. Since l 1 (ŷ(ε), x) 0 we also observe ha ε 0 given in Proposiion 2.3 is nonnegaive and smaller han l 1 (ŷ(ε), x), which shows ha Proposiion 2.3 improves he bound from Proposiion 2.2 for Q( x) C( x). In Proposiions 2.2 and 2.3, if he opimizaion problem max y Y y f(ŷ(ε), x), ŷ(ε) y wih opimal value l 1 (ŷ(ε), x) is solved approximaely, we obain he cus given by Proposiion 2.5. Proposiion 2.5. Le x X and le ŷ(ε 1 ) Y be an ɛ 1 -opimal soluion for problem (2.6) wrien for x = x wih opimal value Q( x), i.e., Q( x) f(ŷ(ε 1 ), x) ε 1. Le also ỹ(ŷ(ε 1 ), x) Y be an approximae ɛ 2 -opimal soluion for he problem max y Y y f(ŷ(ε 1 ), x), ŷ(ε 1 ) y wih opimal value l 1 (ŷ(ε 1 ), x), i.e., l 1 (ŷ(ε 1 ), x) ε 2 y f(ŷ(ε 1 ), x), ŷ(ε 1 ) ỹ(ŷ(ε 1 ), x). Assume ha f is convex and differeniable on Y X. Then seing η(ε 1, ε 2 ) = ε 2 ỹ(ŷ(ε 1 ), x) ŷ(ε 1 ), y f(ŷ(ε 1 ), x) and ˆl 1 (ŷ(ε 1 ), x) = ŷ(ε 1 ) ỹ(ŷ(ε 1 ), x), y f(ŷ(ε 1 ), x), he affine funcion C(x) := f(ŷ(ε 1 ), x) η(ɛ 1, ɛ 2 ) + x f(ŷ(ε 1 ), x), x x is a cu for Q a x, i.e., for every x X we have Q(x) C(x) and he disance Q( x) C( x) beween he values of Q and of he cu a x is a mos ε 2 + ˆl 1 (ŷ(ε 1 ), x). Moreover, if Assumpion (H3) holds, seing ε 2 + ˆl 1 (ŷ(ε 1 ), x) if ˆl 1 (ŷ(ε 1 ), x) 0, (2.16) ε 0 = ε 2 + ˆl 1(ŷ(ε 1), x) 2M 1Diam(Y ) (2M 2 1 Diam(Y ) 2 ˆl 1 (ŷ(ε 1 ), x)) if 0 < ˆl 1 (ŷ(ε 1 ), x) M 1 Diam(Y ) 2, ε ˆl 2 1 (ŷ(ε 1 ), x) oherwise,

6 6 he disance Q( x) C( x) beween he values of Q and of he cu a x is a mos ε 0. Proof. We will use he shor noaion ŷ for ŷ(ε 1 ), ỹ for ỹ(ŷ(ε 1 ), x), and ˆl 1 for ˆl 1 (ŷ(ε 1 ), x). Proceeding as in he proof of Proposiion 2.2, we ge for every x X (2.17) Q(x) f(ŷ, x) l 1 (ŷ, x) + x f(ŷ, x), x x C(x) = f(ŷ, x) + ỹ ŷ, y f(ŷ, x) ε 2 + x f(ŷ, x), x x which shows ha C is a valid cu for Q. Now observe ha C( x) Q( x) = f(ŷ, x) + ỹ ŷ, y f(ŷ, x) ε 2 Q( x) ε 2 ˆl 1. I remains o show ha if Assumpion (H3) holds hen (2.18) f(ŷ, x) + ỹ ŷ, y f(ŷ, x) ε 2 Q( x) ε 0. Using assumpion (H3) we have for every 0 1, This yields f(ŷ + (ỹ ŷ), x) f(ŷ, x) + ỹ ŷ, y f(ŷ, x) M 1 2 ỹ ŷ 2. [ Q( x) f(ŷ, x) + min ˆl M 1Diam(Y ) 2 2]. Three cases are possible: ˆl1 0 (Case A), 0 < ˆl 1 M 1 Diam(Y ) 2 (Case B), ˆl 1 > M 1 Diam(Y ) 2 (Case C). Case A. We have f(ŷ, x) + ỹ ŷ, y f(ŷ, x) ε 2 Q( x) ˆl 1 ε 2 = ε 0 and (2.18) holds. [ Case B. We have min 0 1 ˆl ] M 1Diam(Y ) 2 2 = 1 2 ˆl 1 2 and M 1Diam(Y ) 2 (2.19) Q( x) f(ŷ, x) 1 2 M 1 Diam(Y ) 2. [ Case C. We have min 0 1 ˆl ] M 1Diam(Y ) 2 2 = ˆl M 1Diam(Y ) 2 1 ˆl 2 1 which gives (2.20) Q( x) f(ŷ, x) 1 2 ˆl 1. Combining (2.19) and (2.20) wih (2.16) gives (2.18) for Cases B-C and complees he proof. Remark 2.6. If ε 1 = ε 2 = 0 hen ŷ is an opimal soluion of problem (2.6) wrien for x = x and ε 0 = ε 1 = ε 2 = l 1 (ŷ, x) = ˆl 1 (ŷ(ε 1 ), x) = 0, meaning ha he cu given by Proposiion 2.5 is exac. Also if ε 2 = 0 hen ˆl 1 (ŷ(ε 1 ), x) = l 1 (ŷ(ε 1 ), x) 0. Therefore when ε 2 = 0 and 0 < ˆl 1 (ŷ(ε 1 ), x) M 1 Diam(Y ) 2 or ˆl 1 (ŷ(ε 1 ), x) > M 1 Diam(Y ) 2 he inexac cus from Proposiion 2.5 correspond o he inexac cus given in Proposiion 2.3. For he case ˆl 1 (ŷ(ε 1 ), x) 0 in Proposiion 2.5, if ε 2 = 0 we ge ˆl 1 (ŷ(ε 1 ), x) = 0 which implies η(ε 1, ε 2 ) = 0 and he cu is exac, which is in accordance wih ε 0 = ε 2 = Variable feasible se. Le us now discuss he compuaion of inexac cus for Q given by (2.2). For x X, le us inroduce for problem (2.2) he Lagrangian funcion and he funcion l 2 : Y X R q R p + R + given by L x (y, λ, µ) = f(y, x) + λ T (Bx + Ay b) + µ T g(y, x) (2.21) l 2 (ŷ, x, ˆλ, ˆµ) = min y Y yl x (ŷ, ˆλ, ˆµ), y ŷ = max y Y yl x (ŷ, ˆλ, ˆµ), ŷ y. Wih his noaion he dual funcion (2.4) for problem (2.2) can be wrien ˆl 1 2 θ x (λ, µ) = inf y Y L x(y, λ, µ). We make he following assumpion which ensures no dualiy gap for (2.2) for any x X: (H4) for every x X here exiss y x ri(y ) such ha Bx + Ay x = b and g(y x, x) < 0. The following proposiion provides an inexac cu for Q given by (2.2):

7 7 Proposiion 2.7. Le x X, le ŷ(ɛ) be an ɛ-opimal feasible primal soluion for problem (2.2) wrien for x = x and le (ˆλ(ɛ), ˆµ(ɛ)) be an ɛ-opimal feasible soluion of he corresponding dual problem, i.e., of problem (2.3) wrien for x = x. Le Assumpions (H1), (H2), and (H4) hold. If addiionally f and g are differeniable on Y X hen seing η(ε) = l 2 (ŷ(ɛ), x, ˆλ(ɛ), ˆµ(ɛ)), he affine funcion (2.22) C(x) := L x (ŷ(ɛ), ˆλ(ɛ), ˆµ(ɛ)) η(ε) + x L x (ŷ(ɛ), ˆλ(ɛ), ˆµ(ɛ)), x x is a cu for Q a x and he disance Q( x) C( x) beween he values of Q and of he cu a x is a mos ε + l 2 (ŷ(ɛ), x, ˆλ(ɛ), ˆµ(ɛ)). Proof. To simplify noaion, we use ŷ, ˆλ, ˆµ, for respecively ŷ(ɛ), ˆλ(ɛ), ˆµ(ɛ). Consider primal problem (2.2) wrien for x = x. Due o Assumpion (H4) he opimal value Q( x) of his problem is he opimal value of he corresponding dual problem, i.e., of problem (2.3) wrien for x = x. Using he fac ha ŷ and (ˆλ, ˆµ) are respecively ε-opimal primal and dual soluions i follows ha (2.23) f(ŷ, x) Q( x) + ε and θ x (ˆλ, ˆµ) Q( x) ε. Moreover, since he approximae primal and dual soluions are feasible, we have ha (2.24) ŷ Y, B x + Aŷ = b, g(ŷ, x) 0, ˆµ 0. Using Relaion (2.23), he definiion of dual funcion θ x, and he fac ha ŷ Y, we ge (2.25) L x (ŷ, ˆλ, ˆµ) θ x (ˆλ, ˆµ) Q( x) ε. Due o Assumpions (H1) and (H2), for any λ and µ 0 he funcion L (, λ, µ) which associaes he value L x (y, λ, µ) o (x, y) is convex. I follows ha for every x X, y Y, we have ha L x (y, ˆλ, ˆµ) L x (ŷ, ˆλ, ˆµ) + x L x (ŷ, ˆλ, ˆµ), x x + y L x (ŷ, ˆλ, ˆµ), y ŷ. Since (ˆλ, ˆµ) is dual feasible for dual problem (2.3), he Weak Dualiy Theorem gives Q(x) θ x (ˆλ, ˆµ) = inf y Y L x (y, ˆλ, ˆµ) for every x X and minimizing over y Y on each side of he above inequaliy we obain Finally, using relaion (2.25), we ge Q(x) C(x) = L x (ŷ, ˆλ, ˆµ) l 2 (ŷ, x, ˆλ, ˆµ) + x L x (ŷ, ˆλ, ˆµ), x x. Q( x) C( x) = Q( x) L x (ŷ, ˆλ, ˆµ) + l 2 (ŷ, x, ˆλ, ˆµ) ε + l 2 (ŷ, x, ˆλ, ˆµ). We now refine he bound ε + l 2 (ŷ(ɛ), x, ˆλ(ɛ), ˆµ(ɛ)) on Q( x) C( x) given by Proposiion 2.7 making he following assumpion: (H5) g is differeniable on Y X and here exiss M 2 > 0 such ha for every i = 1,..., p, x X, y 1, y 2 Y, we have y g i (y 2, x) y g i (y 1, x) M 2 y 2 y 1. Proposiion 2.8. Le x X, le ŷ(ɛ) be an ɛ-opimal feasible primal soluion for problem (2.2) wrien for x = x and le (ˆλ(ɛ), ˆµ(ɛ)) be an ɛ-opimal feasible soluion of he corresponding dual problem, i.e., of problem (2.3) wrien for x = x. Le also L x be any lower bound on Q( x). Le Assumpions (H1), (H2), (H3), (H4), and (H5) hold. Then C(x) given by (2.22) is a cu for Q a x and seing M 3 = M 1 + U x M 2 wih U x = f(y x, x) L x + ε min( g i (y x, x), i = 1,..., p), he disance Q( x) C( x) beween he values of Q and of he cu a x is a mos ε + l ε 0 = 2 (ŷ(ɛ), x, ˆλ(ɛ), ˆµ(ɛ)) l2(ŷ(ɛ), x,ˆλ(ɛ),ˆµ(ɛ)) 2 2M 3Diam(Y ) if l 2 2 (ŷ(ɛ), x, ˆλ(ɛ), ˆµ(ɛ)) M 3 Diam(Y ) 2, ε l 2(ŷ(ɛ), x, ˆλ(ɛ), ˆµ(ɛ)) oherwise.

8 8 Proof. As before we use he shor noaion ŷ, ˆλ, ˆµ, for respecively ŷ(ɛ), ˆλ(ɛ), ˆµ(ɛ). We already know from Proposiion 2.7 ha C is a cu for Q. Le us show ha ε 0 is an upper bound for Q( x) C( x). We compue p y L x (y, λ, µ) = y f(y, x) + A T λ + µ i y g i (y, x). Therefore for every y 1, y 2 Y, using Assumpions (H3) and (H5), we have (2.26) y L x (y 2, ˆλ, ˆµ) y L x (y 1, ˆλ, ˆµ) (M 1 + ˆµ 1 M 2 ) y 2 y 1. Nex observe ha L x ε Q( x) ε θ x (ˆλ, ˆµ) f(y x, x) + ˆλ T (Ay x + B x b) + ˆµ T g(y x, x) f(y x, x) + ˆµ 1 max i=1,...,p g i (y x, x). From he above relaion, we ge ˆµ 1 U x, which, plugged ino (2.26) gives (2.27) y L x (y 2, ˆλ, ˆµ) y L x (y 1, ˆλ, ˆµ) M 3 y 2 y 1. The compuaions are now similar o he proof of Proposiion 2.3. More precisely, le y Y such ha Using relaion (2.27), for every 0 1, we ge i=1 l 2 (ŷ, x, ˆλ, ˆµ) = y L x (ŷ, ˆλ, ˆµ), ŷ y. L x (ŷ + (y ŷ), ˆλ, ˆµ) L x (ŷ, ˆλ, ˆµ) + y L x (ŷ, ˆλ, ˆµ), y ŷ M 3 2 y ŷ 2. Since ŷ + (y ŷ) Y, using he above relaion and he definiion of θ x, we obain Q( x) ε θ x (ˆλ, ˆµ) L x (ŷ, ˆλ, ˆµ) l 2 (ŷ, x, ˆλ, ˆµ) M 3 2 y ŷ 2. Therefore Q( x) C( x) = Q( x) L x (ŷ, ˆλ, ˆµ)+l 2 (ŷ, x, ˆλ, ˆµ) ε+l 2 (ŷ, x, ˆλ, ˆµ)+ min ( l 2 (ŷ, x, ˆλ, ˆµ) M 3 2 Diam(Y ) 2) ( and we easily conclude compuing min 0 1 l 2 (ŷ, x, ˆλ, ˆµ) M 3 2 Diam(Y ) ). 2 Remark 2.9. As was done for he exension of Proposiion 2.2 corresponding o Proposiion 2.5, we can exend Proposiion 2.8 o he case when he opimizaion problem max y Y y L x (ŷ, ˆλ, ˆµ), ŷ y wih opimal value l 2 (ŷ, x, ˆλ, ˆµ) is solved approximaely. 3. Bounding he norm of ε-opimal soluions o he dual of a convex opimizaion problem Consider he following convex opimizaion problem: min f(y) (3.28) f = Ay = b, g(y) 0, y Y where (i) Y R n is a closed convex se and A is a q n marix; (ii) f : Y R is convex Lipschiz coninuous wih Lipschiz consan L(f); (iii) g : Y R p where all componens of g are convex Lipschiz coninuous funcions wih Lipschiz consan L(g); (iv) f is bounded from below on he feasible se. We also assume he following Slaer ype consrain qualificaion condiion: (3.29) SL: There exis κ > 0 and y 0 ri(y ) such ha g(y 0 ) κe and Ay 0 = b where e is a vecor of ones in R p. Since SL holds, he opimal value f of (3.28) can be wrien as he opimal value of he dual problem: } (3.30) f = max θ(λ, µ) := minf(y) + λ, Ay b + µ, g(y) }. λ,µ 0 y Y

9 9 Consider he vecor space F = AAff(Y ) b (recall ha 0 F ). Clearly for any y Y and every λ F we have λ T (Ay b) = 0 and herefore for every λ R q, θ(λ, µ) = θ(π F (λ), µ) where Π F (λ) is he orhogonal projecion of λ ono F. I follows ha if F 0}, he se of ɛ-opimal dual soluions of dual problem (3.30) is no bounded because from any ɛ-opimal dual soluion (λ(ε), µ(ε)) we can build an ɛ-opimal dual soluion (λ(ε) + λ, µ(ε)) wih he same value of he dual funcion of norm arbirarily large aking λ in F wih norm sufficienly large. However, he opimal value of he dual (and primal) problem can be wrien equivalenly as (3.31) f = max θ(λ, µ) : µ 0, λ = Ay b, y Aff(Y )}. λ,µ In his secion, our goal is o derive bounds on he norm of ɛ-opimal soluions o he dual of (3.28) wrien in he form (3.31). From Assumpion SL, we deduce ha here exiss r > 0 such ha B n (y 0, r) Aff(Y ) Y and ha here is some ball B q (0, ρ ) of ( posiive radius ρ such ) ha he inersecion of his ball and of he se AAff(Y ) b is conained in he se A B n (y 0, r) Aff(Y ) b. To define such ρ, le ρ : AAff(Y ) b R + given by ρ(z) = max z : 0, z A(B n (y 0, r) Aff(Y )) b}. Since y 0 Y, we can wrie Aff(Y ) = y 0 + V Y where V Y is he vecor space V Y = x y, x, y Aff(Y )}. Therefore A(B n (y 0, r) Aff(Y )) b = A(B n (0, r) V Y ) and ρ can be reformulaed as (3.32) ρ(z) = max z : 0, z A(B n (0, r) V Y )}. Noe ha ρ is well defined and finie valued (we have 0 ρ(z) A r). Also, clearly ρ(0) = 0 and ρ(z) = ρ(λz) for every λ > 0 and z 0. Therefore if A = 0 hen ρ can be any posiive real, for insance ρ = 1, and if A 0 we define (3.33) ρ = minρ(z) : z 0, z AAff(Y ) b} = minρ(z) : z = 1, z AAff(Y ) b}, = minρ(z) : z = 1, z AV Y }, which is well defined and posiive since ρ(z) > 0 for every z such ha z = 1, z AAff(Y ) b (indeed if z AAff(Y ) b wih z = 1 hen z = Ay b for some y Aff(Y ), y y 0, and since r ( y y 0 z = A y 0 + r y y ) ( ) 0 b A B n (y 0, r) Aff(Y ) b, y y 0 we have ρ(z) y y z = 0 our requiremen namely (3.34) B q (0, ρ ) r r y y 0 > 0). We now claim ha parameer ρ we have jus defined saisfies ( ) ( ) AAff(Y ) b A B n (y 0, r) Aff(Y ) b. This can be rewrien as ) (3.35) B q (0, ρ ) AV Y A (B n (0, r) V Y. ( ) ( ) Indeed, le z B q (0, ρ ) AAff(Y ) b. If A = 0 or z = 0 hen z A B n (y 0, r) Aff(Y ) b. Oherwise, by definiion of ρ, we have ρ(z) ρ z. Le 0 be such ha z A(B n (y 0, r) Aff(Y )) b and ρ(z) = z. The relaions ( 1) z 0 and z 0 imply 1. By definiion of, we can wrie z = Ay b where y B n (y 0, r) Aff(Y ). I follows ha z can be wrien ( z = A y 0 + y y 0 Aff(Y ) and ȳ y 0 = y y 0 ) b = Aȳ b where ȳ = y 0 + y y 0 y y 0 r (because 1 and y B n (y 0, r)). ( ) This means ha z A B n (y 0, r) Aff(Y ) b, which proves inclusion (3.34). We are now in a posiion o sae he main resul of his secion:

10 10 Proposiion 3.1. Consider he opimizaion problem (3.28) wih opimal value f. Le Assumpions (i)-(iv) and SL hold and le (λ(ε), µ(ε)) be an ε-opimal soluion o he dual problem (3.31) wih opimal value f. Le (3.36) 0 < r κ 2L(g), be such ha he inersecion of he ball B n (y 0, r) and of Aff(Y ) is conained in Y (his r exiss because y 0 ri(y )). If A = 0 le ρ = 1. Oherwise, le ρ given by (3.33) wih ρ as in (3.32). Le L be any lower bound on he opimal value f of (3.28). Then we have Proof. By definiion of (λ(ε), µ(ε)) and of L, we have (λ(ε), µ(ε)) f(y 0) L + ε + L(f)r. min(ρ, κ/2) (3.37) L ε f ε θ(λ(ε), µ(ε)). Now define z(ε) = 0 if λ(ε) = 0 and z(ε) = ρ λ(ε) λ(ε) oherwise. Observing ha z(ε) B q(0, ρ ) ( ) ( ) AAff(Y ) b and using relaion (3.34) we deduce ha z(ε) A B n (y 0, r) Aff(Y ) b AY b. Therefore, we can wrie z(ε) = Aȳ b for some ȳ B n (y 0, r) Aff(Y ) Y. Nex, using he definiion of θ, we ge θ(λ(ε), µ(ε)) f(ȳ) + λ(ε) T (Aȳ b) + µ(ε) T g(ȳ) since ȳ Y, f(y 0 ) + L(f)r + z(ε) T λ(ε) + µ(ε) T g(y 0 ) + L(g)r µ(ε) 1 using (ii), (iii), ȳ B n (y 0, r), f(y 0 ) + L(f)r ρ λ(ε) κ 2 µ(ε) 1 using SL and (3.36), which can be rewrien as (3.38) (λ(ε), µ(ε)) = λ(ε) 2 + µ(ε) 2 λ(ε) + µ(ε) λ(ε) + µ(ε) 1 f(y0)+l(f)r θ(λ(ε),µ(ε)) min(ρ,κ/2). Combining (3.37) wih (3.38), we obain he desired bound. Recalling ha Aff(Y ) = ỹ +span(y ỹ) for any ỹ Y, he consrains y Aff(Y ) in (3.31) can be wrien y = ỹ + k i=1 α ie i in variables (α i ) k i=1 where (e 1,..., e k ) is a basis of span(y ỹ) and ỹ is an arbirary poin chosen in Y. For insance, if Y ỹ is a box, i.e., Y ỹ = y R n : l y u} wih l < 0 < u hen span(y ỹ) = R n and if Y ỹ = y R n : l i y i u i, i = 1,..., n 0, y i = 0, i > n 0 } wih l i < 0 < u i hen he firs n 0 vecors of he canonical basis of R n form a basis of span(y ỹ)=r n0 0}... 0}. }} n n 0 imes We also have he following immediae corollary of Proposiion 3.1: Corollary 3.2. Under he assumpions of Proposiion 3.1, le f be an upper bound on f on he feasibiliy se of (3.28) and assume ha f is convex and Lipschiz coninuous on R n wih Lipschiz consan L( f). Then we have for (λ(ε), µ(ε)) he bound (λ(ε), µ(ε)) f(y 0) L+ε+L( f)r min(ρ,κ/2). 4. Inexac Dual Dynamic Programming (IDDP) 4.1. Problem formulaion and assumpions. Consider he opimizaion problem T inf f (4.39) (x, x 1 ) x 1,...,x T =1 x X (x 1 ), = 1,..., T, for x 0 given wih he corresponding dynamic programming equaions inf F (x, x 1 ) := f (x, x 1 ) + Q +1 (x ) Q (x 1 ) = x x X (x 1 ), for = 1,..., T, wih Q T Observe ha Q 1 (x 0 ) is he opimal value of (4.39). We will consider wo srucures for ses X (x 1 ), = 1,..., T : (S1) X (x 1 ) = X R n (in his case, for shor, we say ha X is of ype S1);

11 (S2) X (x 1 ) = x R n : x X, g (x, x 1 ) 0, say ha X is of ype S2). A x + B x 1 = b } (in his case, for shor, we Noe ha a mix of hese ypes of consrains is allowed: for insance we can have X 1 of ype S1 and X 2 of ype S2. Seing X 0 = x 0 }, we make he following assumpions (H1): for = 1,..., T, (H1)-(a) X is nonempy, convex, and compac. (H1)-(b) The funcion f (, ) is convex on X X 1 and belongs o C 1 (X X 1 ). For = 1,..., T, if X is of ype S2 we addiionally assume ha: here exiss ε > 0 such ha (wihou loss of generaliy, we will assume in he sequel ha ε = ε) (H1)-(c) each componen g i (, ), i = 1,..., p, of he funcion g (, ) is convex on X X ε 1 and belongs o C 1 (X X 1 ). (H1)-(d) For every x 1 X ε 1, he se X (x 1 ) ri(x ) is nonempy. (H1)-(e) If 2, here exiss x = ( x, x 1 ) ri(x ) X 1 such ha A x + B x 1 = b, and g ( x, x 1 ) < 0. Assumpions (H1)-(a), (b), (c) ensure ha funcions Q are convex. Assumpion (H1)-(d) is used o bound he cu coefficiens (see Proposiion 4.4) and show ha funcions Q are Lipschiz coninuous on X 1. Differeniabiliy and Assumpion (H1)-(e) are useful o derive inexac cus, see Secions , in paricular Lemma 4.1. The Inexac Dual Dynamic Programming (IDDP) algorihm o be presened in he nex secion is a soluion mehod for problem (4.39) ha explois he convexiy of Q, = 2,..., T Inexac Dual Dynamic Programming: overview. Similarly o DDP, o solve problem (4.39), he Inexac Dual Dynamic Programming algorihm approximaes for each = 2,..., T + 1, he funcion Q by a polyhedral lower approximaion Q k a ieraion k. We sar a he firs ieraion wih he lower approximaion Q 0 = for Q, = 2,..., T. A he beginning of ieraion k, we have he lower polyhedral approximaions (compued a previous ieraions) for Q, whose compuaions are deailed below. For convenience, for = 1,..., T, and k 0, le F k (y, x) = f (y, x) + Q k +1(y) and le Q k : X 1 R given by Q k 1 (4.40) Q k (x) = inf y R n F k (y, x) y X (x). Ieraion k sars wih a forward pass: for = 1,..., T, we compue an ε k -opimal soluion x k of inf (4.41) Q k 1 F (x k k 1 (y, x k 1) 1) = y y X (x k 1), saring from x k 0 = x 0 where F k 1 (y, x k 1) = f (y, x k 1) + Q k 1 +1 (y) and knowing ha Qk 1 T +1 = Q T Therefore, we have (4.42) Q k 1 (x k 1) F k 1 (x k, x k 1) Q k 1 (x k 1) + ε k. A ieraion k, a backward pass hen compues a cu C k for Q a x k 1 for = T + 1 down o = 2. For = T +1, he cu is exac: CT k For sep < T +1, we compue an εk -opimal soluion x Bk X (x k 1) of inf (4.43) Q k F (xk k (y, x k 1) 1) = y y X (x k 1), knowing Q k +1. I follows ha (4.44) x Bk X (x k 1) and Q k (xk 1) F k (x Bk, x k 1) Q k (xk 1) + ε k. 11

12 12 If X is of ype S2 we also compue an ε k -opimal soluion (λ k, µ k ) of he dual problem (4.45) sup h k,x k 1(λ, µ) λ = A y + B x k 1 b, y Aff(X ), µ R p + for he dual funcion (4.46) h k inf F k,x µ) = (y, x 1(λ, k 1) + λ T (A y + B x k 1 b ) + µ T g (y, x k 1) k y X. We now check ha Assumpion (H1) implies ha he following Slaer ype consrain qualificaion condiion holds for problem (4.43) (i.e. for all problems solved in he backward passes): (4.47) here exiss x k ri(x ) such ha A x k + B x k 1 = b and g ( x k, x k 1) < 0. The above consrain qualificaion condiion is he analogue of (3.29) for problem (4.43). Lemma 4.1. Le Assumpion (H1) hold. Then for every k N, (4.47) holds. Proof. If x k 1 = x 1 hen recalling (H1)-(e), (4.47) holds wih x k = x. Oherwise, we define x kε 1 = x k 1 + ε xk 1 x 1 x k 1 x 1. Observe ha since x k 1 X 1, we have x kε 1 X ε 1. Seing X = (x, x 1 ) ri(x ) X ε 1 : A x + B x 1 = b, g (x, x 1 ) 0}, since x kε 1 X 1, ε using (H1)-(d), here exiss x kε ri(x ) such ha (x kε, x kε 1) X. Now clearly, since X and X 1 are convex, he se ri(x ) X 1 ε is convex oo and using (H1)-(c), we obain ha X is convex. Since ( x, x 1 ) X (due o Assumpion (H1)-(e)) and recalling ha (x kε, x kε 1) X, we obain ha for every 0 < θ < 1, he poin (4.48) (x (θ), x 1 (θ)) = (1 θ)( x, x 1 ) + θ(x kε, x kε 1) X. For 1 (4.49) 0 < θ = θ 0 = ε < 1, 2 x k 1 x 1 we ge x 1 (θ 0 ) = x k 1, x (θ 0 ) ri(x ), A x (θ 0 ) + B x 1 (θ 0 ) = A x (θ 0 ) + B x k 1 = b, and since g i, i = 1,..., p, are convex on X X ε 1 (see Assumpion (H1)-(c)) and herefore on X, we ge g (x (θ 0 ), x 1 (θ 0 )) = g (x (θ 0 ), x k 1) (1 θ 0 ) g ( x, x 1 ) + θ 0 g }}}}}} (x kε, x k,ε 1 }} ) >0 <0 >0 0 < 0. We have jusified ha (4.47) holds wih x k = x (θ 0 ). From (4.47), we deduce ha he opimal value Q k (xk 1) of primal problem (4.43) is he opimal value of dual problem (4.45) and herefore ε k -opimal dual soluion (λ k, µ k ) saisfies: (4.50) Q k (xk 1) ε k h k,x k 1(λ k, µ k ) Q k (xk 1). We now inend o use he resuls of Secion 2 o derive an inexac cu C k for Q a x k 1. Since for all ieraion k he relaion Q Q k is preserved, Ck will in fac be an inexac cu for Q k and herefore for Q. To proceed, le us wrie funcion Q k +1, which is a maximum of k affine funcions, in he form Q k +1(x ) = max 1 j k ( C j +1 (x ) := θ j +1 ηj +1 (εj +1 ) + βj +1, x x j )

13 for some coefficiens θ j +1, ηj +1 (εj +1 ), and βj +1 whose ieraive compuaion is deailed below wih he convenion ha for = T coefficiens θ j +1, ηj +1 (εj +1 ), βj +1 are all zero. Plugging his represenaion ino (4.43), we ge inf f (x, x k (4.51) Q k x,y 1) + y (xk 1) = x X (x k 1), y θ j +1 ηj +1 (εj +1 ) + βj +1, x x j, j = 1,..., k, which is of form (2.2) wih y = (x, y ), x = x k 1, f(y, x) = f (x, x) + y, Y = y = [x ; y ] : x X, B k +1y b k +1}, and for consrains of ype S2 A = [A 0 q 1 ], B = B, b = b, g(y, x) = g (x, x), where he j-h line of marix B k +1 is [(β j +1 )T, 1] and he j-h componen of b k +1 is θ j +1 + ηj +1 (εj +1 ) β j +1 xj. We can now use he resuls of Secion 2 and consider several cases depending on he problem srucure Compuaion of inexac cus in he backward pass for consrains of ype S1. Le us firs consider he case where X is of ype S1. Le (x Bk, y Bk ) be an ε k -opimal soluion of inf f (x, x k (4.52) Q k x,y 1) + y [ ] (xk 1) = x X, B+1 k x b k +1. We compue θ k = f (x Bk, x k 1) + y Bk where (4.53) l k 1(x Bk, y Bk, x k 1) =, η k (ε k ) = l k 1(x Bk, y Bk y max x,y x f[ (x Bk ], x k 1), x Bk x X, B+1 k x b k +1. y, x k 1), β k = x 1 f (x Bk, x k 1), x + y Bk Using Proposiion 2.2 we have ha C k (x 1 ) = θ k η k (ε k ) + β k, x 1 x k 1 is an inexac cu for Q k and herefore for Q. Moreover, he disance beween Q k (xk 1) and C k (x k 1) is a mos η k (ε k ) = l k 1(x Bk, y Bk, x k 1) Compuaion of inexac cus in he backward pass for consrains of ype S2. We now consider he case where X is of ype S2. Le (x Bk, y Bk ) be an ε k -opimal soluion of inf f (x, x k (4.54) Q k x,y 1) + y [ ] (xk 1) = x X (x k 1), B+1 k x b y k +1. Define for problem (4.54) he Lagrangian y L x k 1 (x, y, λ, µ) = f (x, x k 1) + y + λ T (A x + B x k 1 b ) + µ T g (x, x k 1) and (4.55) l k 2(x Bk, y Bk, x k 1, λ, µ) = max x,y x L x k 1 (x [ ] Bk x X, B+1 k x b y k +1., y Bk, λ, µ), x Bk x + y Bk y Wih his noaion and recalling ha (λ k, µ k ) is an ε k -opimal soluion of (4.45) we pu θ k = L (4.56) x k 1 (x Bk, y Bk, λ k, µ k ), η k (ε k ) = l k 2(x Bk, y Bk, x k 1, λ k, µ k ), β k = x 1 f (x Bk, x k 1) + B T λ k + p i=1 µk (i) x 1 g i (x Bk, x k 1). Using Proposiion 2.7, he affine funcion C k (x 1 ) = θ k η k (ε k ) + β k, x 1 x k 1 13

14 14 defines an inexac cu for Q. Moreover, he disance beween Q k (xk 1) and C k (x k 1) is a mos ε k + l k 2(x Bk, y Bk, x k 1, λ k, µ k ) = ε k + η k (ε k ). For IDDP, we assume ha nonlinear opimizaion problems (such as primal problems (4.52), (4.54) or dual problem (4.45)) are solved approximaely whereas linear opimizaion problems are solved exacly. Noice ha we assumed ha we can compue he opimal value l k 1(x Bk, y Bk, x k 1) of opimizaion problem (4.53) and he opimal value l k 2(x Bk, y Bk, x k 1, λ k, µ k ) of opimizaion problem (4.55) wrien for (λ, µ) = (λ k, µ k ). Since hese opimizaion problems have a linear objecive funcion, hey are linear programs if and only if X is polyhedral. If his is no he case hen a) eiher we add componens o g pushing he nonlinear consrains in he represenaion of X in g or b) we also solve approximaely (4.53) and (4.55). In Case b), we can sill build an inexac cu C k and sudy he convergence of he corresponding varian of IDDP along he lines of Secion 4.5. More precisely, in his siuaion, we obain cu C k using Proposiion 2.5 insead of Proposiion 2.2 if X is of ype S1. If X is of ype S2 we can use he exension of Proposiion 2.7 obained when (2.21) is solved approximaely, exacly as was done for he exension of Proposiion 2.2 corresponding o Proposiion Convergence analysis. The main resul of his secion is Theorem 4.7, a convergence analysis of IDDP. We will use he following immediae observaion: Lemma 4.2. For = 2,..., T + 1, funcion Q is convex and Lipschiz coninuous on X 1. Proof. The proof is by backward inducion on. The resul holds for = T + 1 by definiion of Q T +1. Le us now assume ha Q +1 is convex and Lipschiz coninuous on X for some 2,..., T }. We consider wo cases: X is of ype S1 (Case A) and X is of ype S2 (Case B). Case A. Convexiy of Q immediaely follows from (H1)-(a),(b). (H1)-(b) implies ha f is coninuous on he compac se X X 1 and herefore akes finie values on X X 1 bu also on some neighborhood X X ε0 1 of X X 1 wih ε 0 > 0. Therefore, for every x 1 X ε0 1, we have ha x f (x, x 1 ) + Q +1 (x ) is finie-valued on X, and Q (x 1 ) is finie. Case B. Convexiy of Q immediaely follows from (H1)-(a),(b), (c). As in Case A, f is finie valued on X X ε0 1 for some ε 0 > 0. Combining his observaion wih (H1)-(d), for every x 1 X min(ε0,ε) he funcion x f (x, x 1 ) + Q +1 (x ) is finie-valued on he nonempy se X (x 1 ) and herefore Q (x 1 ) is finie. In boh Cases (A) and (B) we checked ha X 1 is conained in he inerior of he domain of Q which implies ha convex funcion Q is Lipschiz coninuous on X 1. In view of Lemma 4.2, we will denoe by L(Q ) a Lipschiz consan for Q for = 2,..., T + 1. A useful ingredien for he convergence analysis of IDDP is he boundedness of he sequences of approximae dual soluions (λ k, µ k ). Recall ha if X is of ype S2 hen Slaer consrain qualificaion (4.47) holds. From Theorem 2.3.2, p.312 in [10], we deduce ha if he rows of A are independen hen he se of opimal dual soluions of problem (4.45) is bounded. Therefore, he level se of h k associaed o is minimal,x k 1 value is bounded implying ha he level se associaed o his minimal value plus ε k is bounded oo (since for a convex funcion if a level se is bounded hen all level ses are bounded). I follows ha if he rows of A are independen, hen for every k N he norm (λ k, µ k ) is finie. To obain an upper bound on he sequence ( (λ k, µ k ) ) k we will use a slighly sronger assumpion han (H1)-(e), namely we will assume: (H2) For = 2,..., T, here exiss κ > 0, r > 0 such ha for every x 1 X 1, here exiss x X such ha B(x, r ) Aff(X ), A x + B x 1 = b, and for every i = 1,..., p, g i (x, x 1 ) κ. Remark 4.3. Of course, by definiion of he relaive inerior, he condiion B(x, r ) Aff(X ) implies ha x ri(x ).

15 15 However, we do no assume ha he rows of A are independen. Using (H2) and Secion 3 we can now show ha he sequences of cu coefficiens and approximae dual soluions belong o a compac se: Proposiion 4.4. Assume ha noises (ε k ) k 1 are bounded: for = 2,..., T, we have 0 ε k ε < +. If Assumpions (H1) and (H2) hold hen he sequences (θ k ),k, (η k (ε k )),k, (β k ),k, (λ k ),k, (µ k ),k generaed by he IDDP algorihm are bounded: for = 2,..., T +1, here exiss a compac se C such ha he sequence (θ k, η k (ε k ), β k ) k 1 belongs o C and for = 2,..., T, if X is of ype S2 hen here exiss a compac se D such ha he sequence (λ k, µ k ) k 1 belongs o D. Proof. The proof is by backward inducion on. Our inducion hypohesis H() for 2,..., T + 1} is ha he sequence (θ k, η k (ε k ), β k ) k 1 belongs o a compac se C. We have ha H(T + 1) holds because for = T +1 he corresponding coefficiens are all zero. Now assume ha H(T +1) holds for some 2,..., T +1}. We wan o show ha H() holds and if X is of ype S2 ha he sequence (λ k, µ k ) k 1 belongs o some compac se D. Since f and g belong o C 1 (X X 1 ) we can find finie m, M 1, M 2, M 3, M 4 such ha for every x X, x 1 X 1, for every i = 1,..., p, we have m f (x, x 1 ) M 1, f (x, x 1 ) M 2, g i (x, x 1 ) M 3, g (x, x 1 ) M 4. Also since H(+1) holds, he sequence ( β+1 ) k k 1 is bounded from above by, say, L +1, which is a Lipschiz consan for all funcions (Q k +1) k 1. We now consider wo cases: X is of ype S1 (Case A) and X is of ype S2 (Case B). Case A. We have θ k = f (x Bk, x k 1) + Q k +1(x Bk ) which gives he bound m + min x X Q 1 +1(x ) θ k M 1 + max x X Q +1 (x ), k 1, (recall ha due o H( + 1) and Lemma 4.2, he minimum and maximum in he relaion above are well defined because funcions Q 1 +1 and Q +1 are coninuous on he compac X ). Now for η k (ε k ) = l k 1(x Bk, y Bk, x k 1) and recalling definiion (4.53) of l k 1(x Bk, y Bk, x k 1), we see ha (4.57) 0 η k (ε k ) η := (M 2 + L +1 )D(X ), k 1, and of course he norm of β k = x 1 f (x Bk, x k 1) for all k 1 is bounded from above by M 2. This shows H() for Case A. Case B. We firs obain a bound on (λ k, µ k ) using Proposiion 3.1 and Corollary 3.2. Le us check ha he Assumpions of his corollary are saisfied for problem (4.54): (i) X is a closed convex se; (ii) he objecive funcion F k (, x k 1) is bounded from above by f( ) = f (, x k 1) + Q +1 ( ). Since f is convex and finie in a neighborhood of X X 1, i is Lipschiz coninuous on X X 1 wih Lipschiz consan, say, L(f ). Therefore f is Lipschiz coninuous wih Lipschiz consan L(f ) + L(Q +1 ) on X. (iii) Since all componens of g are convex and finie in a neighborhood of X X 1, hey are Lipschiz coninuous on X X 1. (iv) The objecive funcion is bounded on he feasible se by L = min x 1 X 1 Q 1 (x 1) (he minimum is well defined due o Assumpion (H1)). Due o Assumpion (H2) we can find ˆx k ri(x ) such ha ˆx k X (x k 1) and B n (ˆx k, r ) Aff(X ). Therefore, reproducing he reasoning of Secion 3, we can find ρ such ha ( ) B q (0, ρ ) A V X A B n (0, r ) V X where V X is he vecor space V X = x y, x, y Aff(X )} (his is relaion (3.35) for problem (4.54)). Applying Corollary 3.2 o problem (4.54) we deduce ha (λ k, µ k ) U where (L(f ) + L(Q +1 ))r + ε + max (f (x, x 1 ) + Q +1 (x )) min Q 1 x X,x 1 X 1 x (x 1) 1 X 1 U = min(ρ, κ 2 ). For θ k = f (x Bk, x k 1) + Q k +1(x Bk ) + µ k, g (x Bk, x k 1) we ge he bound m U M 4 + min x X Q 1 +1(x ) θ k M 1 + max x X Q +1 (x ).

16 16 Noe ha η k (ε k ) 0 and he objecive funcion of problem (4.55) wrien for (λ, µ) = (λ k, µ k ) wih opimal value η k (ε k ) is bounded from above on he feasible se by (4.58) η = (M max( A T ), M 3 p)u + L +1 D(X ) and herefore he same upper bound holds for η k (ε k ). Finally, recalling definiion (4.56) of β k we have: [ (4.59) β k M 2 + B T λ k ] + M 3 p µ k L := M max( B T, M 3 p)u, which complees he proof and provides a Lipschiz consan L valid for funcions (Q k ) k. To show ha he sequence of error erms (η k (ε k )) k converges o 0 when lim k + ε k = 0, we will make use of Proposiions 4.5 and 4.6 which follow: Proposiion 4.5. Le X R m, Y R n, be wo nonempy compac convex ses. Le f C 1 (Y X) be convex on Y X. Le (Q k ) k 1 be a sequence of convex L-Lipschiz coninuous funcions on Y saisfying Q Q k Q on Y where Q, Q are coninuous on Y. Le (x k ) k 1 be a sequence in X, (ε k ) k 1 be a sequence of nonnegaive real numbers, and le y k (ε k ) Y be an ε k -opimal soluion o (4.60) inf f(y, x k ) + Q k (y) : y Y }. Define (4.61) η k (ε k max y f(y ) = k (ε k ), x k ), y k (ε k ) y + Q k (y k (ε k )) Q k (y) y Y. Then if lim k + ε k = 0 we have (4.62) lim k + ηk (ε k ) = 0. Proof. In wha follows, o simplify noaion, we wrie y k insead of y k (ε k ). We show (4.62) by conradicion. Denoing by y k Y an opimal soluion of (4.60), we have for every k 1 ha (4.63) f(y k, x k ) + Q k (y k ) f(y k, x k ) + Q k (y k ) f(y k, x k ) + Q k (y k ) + ε k. Denoing by ỹ k Y an opimal soluion of opimizaion problem (4.61) we ge (4.64) η k (ε k ) = y f(y k, x k ), y k ỹ k + Q k (y k ) Q k (ỹ k ). Assume ha (4.62) does no hold. Then since η k (ε k ) 0 here exiss ε 0 > 0 and σ 1 : N N increasing such ha for every k N we have (4.65) η σ1(k) (ε σ1(k) ) = y f(y σ1(k), x σ1(k) ), ỹ σ1(k) + y σ1(k) + Q σ1(k) (y σ1(k) ) Q σ1(k) (ỹ σ1(k) ) ε 0. Now observe ha he sequence (Q σ1(k) ) k in C(Y ) (i) is bounded: for every k 1, for every y Y, we have < min Q(y) y Y Qσ1(k) (y) max Q(y) < + ; y Y (ii) is equiconinuous since funcions (Q σ1(k) ) k are Lipschiz coninous wih Lipschiz consan L. Therefore using he Arzelà-Ascoli heorem, his sequence has a uniformly convergen subsequence: here exiss Q C(Y ) and σ 2 : N N increasing such ha seing σ = σ 1 σ 2, we have lim k + Q σ(k) Q Y = 0. Since (y σ(k), y σ(k), ỹ σ(k), x σ(k) ) k 1 is a sequence of he compac se Y Y Y X, aking furher a subsequence if needed, we can assume ha (y σ(k), y σ(k), ỹ σ(k), x σ(k) ) converges o some (ȳ, y, ỹ, x ) Y Y Y X. By coninuiy argumens, for k sufficienly large, say k k 0, we have ha (4.66) y f(y σ(k), x σ(k) ), ỹ σ(k) + y σ(k) y f(ȳ, x ), ỹ σ(k) + ȳ ε 0 /4, y σ(k) ȳ ε0 8L, Qσ(k) Q Y ε 0 /16.

17 17 I follows ha (4.67) y f(ȳ, x ), ỹ σ(k0) + ȳ + Q (ȳ) Q (ỹ σ(k0) ) = y f(y σ(k0), x σ(k0) ), ỹ σ(k0) + y σ(k0) + Q σ(k0) (y σ(k0) ) Q σ(k0) (ỹ σ(k0) ) + y f(ȳ, x ), ỹ σ(k0) + ȳ y f(y σ(k0), x σ(k0) ), ỹ σ(k0) + y σ(k0) +[Q (ȳ) Q σ(k0) (ȳ) + Q σ(k0) (ȳ) Q σ(k0) (y σ(k0) )] [Q (ỹ σ(k0) ) Q σ(k0) (ỹ σ(k0) )], ε 0 ε0 4 2 Q Q σ(k0) Y L ȳ y σ(k0) ε0 2 > 0, where for he las wo inequaliies we have used (4.65) and (4.66). Recalling he definiion of y, k for every k 1 we have ha y σ(k) Y and f(y σ(k), x σ(k) ) + Q σ(k) (y σ(k) ) f(y, x σ(k) ) + Q σ(k) (y), y Y. Taking he limi as k + in he above inequaliy we ge (using he coninuiy of f) f := f(y, x ) + Q (y ) f(y, x ) + Q (y), y Y. Since y Y,we have shown ha y is an opimal soluion for he opimizaion problem min f(y, x ) + Q (4.68) f = (y) y Y. Replacing k by σ(k) in (4.63) and aking he limi as k +, we obain f = f(y, x ) + Q (y ) = f(ȳ, x ) + Q (ȳ). Combining his observaion wih he fac ha ȳ Y, we deduce ha ȳ is also an opimal soluion of (4.68). Nex, since all funcions (Q σ(k) ) k are convex on Y, he funcion Q is convex on Y oo. Recalling Lemma 6.1, he opimaliy condiions for ȳ read Since ỹ σ(k0) Y, we have in paricular y f(ȳ, x ), y ȳ + Q (y) Q (ȳ) 0, y Y. y f(ȳ, x ), ỹ σ(k0) ȳ + Q (ỹ σ(k0) ) Q (ȳ) 0. However, from (4.67), he lef-hand side of he above inequaliy is ε0 2 conradicion. < 0 which yields he desired Proposiion 4.6. Le Y R n, X R m, be wo nonempy compac convex ses. Le f C 1 (Y X) be convex on Y X. Le (Q k ) k 1 be a sequence of convex L-Lipschiz coninuous funcions on Y saisfying Q Q k Q on Y where Q, Q are coninuous on Y. Le g C 1 (Y X) wih componens g i, i = 1,..., p, convex on Y X ε for some ε > 0. We also assume (H) : κ > 0, r > 0, such ha x X y Y : B n (y, r) Aff(Y ), Ay + Bx = b, g(y, x) < κe, where e is a vecor of ones of size p. Le (x k ) k 1 be a sequence in X, le (ε k ) k 1 be a sequence of nonnegaive real numbers, and le y k (ε k ) be an ε k -opimal and feasible soluion o (4.69) inf f(y, x k ) + Q k (y) : y Y, Ay + Bx k = b, g(y, x k ) 0}. Le (λ k (ε k ), µ k (ε k )) be an ε k -opimal soluion o he dual problem (4.70) where sup λ,µ h k x k (λ, µ) λ = Ay + Bx k b, y Aff(Y ), µ 0, h k xk(λ, µ) = inf f(y, y Y xk ) + Q k (y) + λ, Ay + Bx k b + µ, g(y, x k ) }. Define η k (ε k ) as he opimal value of he following opimizaion problem: (4.71) p max y f(y k (ε k ), x k ) + A T λ k (ε k ) + µ k (ε k )(i) y g i (y k (ε k ), x k ), y k (ε k ) y + Q k (y k (ε k )) Q k (y) y Y. i=1

18 18 Then if lim k + ε k = 0 we have (4.72) lim k + ηk (ε k ) = 0. Proof. For simpliciy, we wrie λ k, µ k, y k insead of λ k (ε k ), µ k (ε k )), y k (ε k ), and pu Y(x) = y Y : Ay + Bx = b, g(y, x) 0}. Denoing by y k Y(x k ) an opimal soluion of (4.69), we ge (4.73) f(y k, x k ) + Q k (y k ) f(y k, x k ) + Q k (y k ) f(y k, x k ) + Q k (y k ) + ε k. We prove (4.72) by conradicion. Le ỹ k be an opimal soluion of (4.71): p η k (ε k ) = y f(y k, x k ) + A T λ k + µ k (i) y g i (y k, x k ), y k ỹ k Q k (ỹ k ) + Q k (y k ). i=1 Assume ha (4.72) does no hold. Then here exiss ε 0 > 0 and σ 1 : N N increasing such ha for every k N we have y f(y (4.74) σ1(k), x σ1(k) ) + A T λ σ1(k) + p i=1 µσ1(k) (i) y g i (y σ1(k), x σ1(k) ), ỹ σ1(k) + y σ1(k) +Q σ1(k) (y σ1(k) ) Q σ1(k) (ỹ σ1(k) ) ε 0. Using Assumpion (H) and Proposiion 3.1, we obain ha he sequence (λ σ1(k), µ σ1(k) ) k is a sequence of a compac se, say D. Therefore, same as in he proof of Proposiion 4.5, we can find Q C(Y ) and σ 2 : N N increasing such ha seing σ = σ 1 σ 2, we have lim k + Q σ(k) Q Y = 0, and (y σ(k), y σ(k), ỹ σ(k), x σ(k), λ σ(k), µ σ(k) ) converges o some (ȳ, y, ỹ, x, λ, µ ) Y Y Y X D. I follows ha here is k 0 N such ha for every k k 0 : y f(y σ(k), x σ(k) ) + A T λ σ(k) + p i=1 µσ(k) (i) y g i (y σ(k), x σ(k) ), ỹ σ(k) + y σ(k) (4.75) y f(ȳ, x ) + A T λ + p i=1 µ (i) y g i (ȳ, x ), ỹ σ(k) + ȳ ε0 /4, y σ(k) ȳ ε0 8L, Qσ(k) Q Y ε 0 /16. Same as in he proof of Lemma 4.4, we deduce from (4.74), (4.75) ha p (4.76) y f(ȳ, x ) + A T λ + µ (i) y g i (ȳ, x ), ỹ σ(k0) + ȳ + Q (ȳ) Q (ỹ σ(k0) ) ε 0 /2 > 0. i=1 Due o Assumpion (H), primal problem (4.69) and dual problem (4.70) have he same opimal value and for every y Y and k 1 we have: f(y σ(k), x σ(k) ) + Q σ(k) (y σ(k) ) + Ay σ(k) + Bx σ(k) b, λ σ(k) + µ σ(k), g(y σ(k), x σ(k) ) f(y σ(k), x σ(k) ) + Q σ(k) (y σ(k) ) + ε σ(k) by definiion of y σ(k), y σ(k) and since µ σ(k) 0, y σ(k) Y(x σ(k) ), h σ(k) (λ σ(k), µ σ(k) ) + 2ε σ(k), [(λ σ(k), µ σ(k) ) is an ɛ σ(k) -opimal dual soluion and here is no dualiy gap], x σ(k) f(y, x σ(k) ) + Ay + Bx σ(k) b, λ σ(k) + µ σ(k), g(y, x σ(k) ) + Q σ(k) (y) + 2ε σ(k) by definiion of h σ(k). x σ(k) Taking he limi in he above relaion as k +, we ge for every y Y : f(ȳ, x ) + Aȳ + Bx b, λ + µ, g(ȳ, x ) + Q (ȳ) f(y, x ) + Ay + Bx b, λ + µ, g(y, x ) + Q (y). Recalling ha ȳ Y his shows ha ȳ is an opimal soluion of min f(y, x ) + Q (4.77) (y) + Ay + Bx b, λ + µ, g(y, x ) y Y. Now recall ha all funcions (Q σ(k) ) k are convex on Y and herefore he funcion Q is convex on Y oo. Using Lemma 6.1, he firs order opimaliy condiions for ȳ can be wrien p (4.78) y f(ȳ, x ) + A T λ + µ (i) y g i (ȳ, x ), y ȳ + Q (y) Q (ȳ) 0 i=1 for all y Y. Specializing he above relaion for y = ỹ σ(k0), we ge p y f(ȳ, x ) + A T λ + µ (i) y g i (ȳ, x ), ỹ σ(k0) ȳ + Q (ỹ σ(k0) ) Q (ȳ) 0, i=1