Supplementary Material

Transcription

1 ec1 Supplementary Material EC.1. Measures in D(S, µ, Ψ) Are Less Dispersed Lemma EC.1. Given a distributional set D(S, µ, Ψ) with S convex and a random vector ξ such that its distribution F D(S, µ, Ψ), forall0 α 1 the random vector ζ := α(ξ µ)+µ also has adistributionthatliesind(s, µ, Ψ). Proof: We simply need to verify systematically the conditions imposed on the members of D(S, µ, Ψ). First, P F (ζ S)=P F (αξ +(1 α)µ S)=1, since P F (ξ S)=1, µ S,andthefactthatS is a convex set. Second, it is easy to see that E F [ζ=e F [αξ +(1 α)µ=µ. Finally, for any ψ( ) Ψ, we have that E F [ψ(αξ +(1 α)µ) E F [αψ(ξ)+(1 α)µ = αe F [ψ(ξ) + (1 α)ψ(µ) E F [ψ(ξ) 0, where we applied Jensen s inequality for the two first upper bounding steps. EC.2. Concavity of h(x,ξ) in ξ Consider the second stage stochastic minimization problem h(x,ξ):= minimize y f(x, y,ξ) s.t. y Y(x), where Y(x) isanygivenclosedandboundedsetfunctionofx. Herewemaenoassumptionthat set Y(x) isapolyhedralorevenaconvexset. We assume that f(x, y, ξ) is a concave function in the uncertain parameter-vector ξ. Note that this assumption has been made in most previous stochastic and robust optimization models (see Ben-Tal and Nemirovsi (1998)). Lemma EC.2. The minimum value function h(x,ξ) is a concave function in ξ. This lemma implies that our Proposition 1 is also applicable to most linear and nonlinear twostage optimization problems. We omit including a detailed proof of this lemma as it is well nown that the pointwise infimum of a set of affine functions is a concave function (see Boyd and Vandenberghe (2004) for more details).

2 ec2 e-companion to Delage et al.: The Value of Stochastic Modeling EC.3. Proof of Proposition 2 We follow similar steps as followed in the proof of Proposition 1. We first underline the fact that implementing the MVP solution, a policy that does not adapt to thesequenceofobservableξ, leads to a worst-case expected cost that is equal to the optimal value of the MVP problem, which we refer to as V MVP. [ T T E F ξ T t C t x t = E F [ξ T t C t x t F D(µ [1:T ) F D(µ [1:T ) T = µ T t C t x t, where D(µ [1:T )isshortford(s, [µ 1, µ 2,...,µ T, Ψ). Here, we used the fact that x t does not adapt to the observed uncertain parameters and the fact that all the distributionsinthesetthatis considered lead to the same expected value for the random vectors. We can therefore say that the optimal value of the distributionally robust multi-stage stochastic program must be smaller than V MVP. Secondly, after verifying that the Dirac measure δ µ[1:t lies in the set D(µ [1:T )(theargument being the same as in the proof of Proposition 1), one can show that V MVP is actually also a lower bound for the same distributionally robust problem. min (x 1,x 2,...,x T ) X a.s. [ T E F ξ T t C tx t (ξ [1:t ) F D(µ [1:T ) [ T min E δµ[1:t (x 1,x 2,...,x T ) X a.s. ξ T t C tx t (ξ [1:t ) T = min µ T t C t x t (µ [1:t ) (x 1,x 2,...,x T ) X a.s. = V MVP. Hence, we conclude that the MVP solution is an optimal solution for the distributionally robust multi-stage stochastic program under D(µ [1:T ). EC.4. Proof of Proposition 4 The proof consists of showing that the sequence of distributions F 1,F 2,... with F (ξ)=(1 (1 + β ) 1 )δ µ (ξ)+(1+β ) 1 G (ξ), satisfies E F [ψ(ξ) = 0, ψ Ψ, and that given any ϵ>0, one can choose large enough so that F satisfies: P F (ξ S) 1 ϵ E F [ξ µ ϵ.

3 ec3 Based on Condition 1, we can easily show the first part: E F [ψ(ξ) = (1 (1 + β ) 1 )E δµ [ψ(ξ) + (1 + β ) 1 E G [ψ(ξ) =(1 (1 + β ) 1 )ψ(µ)+(1+β ) 1 β =0. Based on Condition 2, we also have the property that E G [ ξ =O(β 1/γ ), for some γ>1. This is due to the fact that there exists an a>0, b R, andγ>1: E G [ ξ γ E G [ ξ γ =(1/a)(E G [b + a ξ γ b) (1/a)(E G [ψ 0 (ξ) b) =(1/a)(β b), where we used Jensen s inequality and the fact that ψ 0 (ξ)= i θ iψ i (ξ)forsomeconicalcombination of ψ i Ψallowingustoderive E G [ψ 0 (ξ) = E G [ i θ i ψ i (ξ) = i θ i β = β. Hence, we have that E G [ ξ =E G [ ξ γ/γ ((1/a)(β b)) 1/γ = O(β 1/γ ). Thus, we can demonstrate that both P F (ξ S)andE F [ξ willbecomefeasiblefor large enough: P F (ξ S)=(1 (1 + β ) 1 )P δµ (ξ S)+(1+β ) 1 )P G (ξ S) (1 (1 + β ) 1 )P δµ (ξ S)=1 (1 + β ) E F [ξ µ (1 (1 + β ) 1 ) E δµ [ξ µ +(1+β ) 1 E G [ξ µ (1 + β ) 1 ( E G [ξ + µ ) (1 + β ) 1 (E G [ ξ + µ )=O(β (1 1/γ) ). that Ultimately, based on the Lipschitz property of h(x, ) demonstrated in Lemma 1, we can verify E F [h(x, ξ) F D(S,µ,Ψ,ϵ) { F D(S,µ,Ψ,ϵ)} E F [h(x, ξ) = (1 (1 + β ) 1 )h(x, µ)+(1+β ) 1 E G [h(x, ξ) { F D(S,µ,Ψ,ϵ)} (1 (1 + β ) 1 )h(x, µ)+(1+β ) 1 E G [h(x, µ) R C 2 ξ µ { F D(S,µ,Ψ,ϵ)} { F D(S,µ,Ψ,ϵ)} = { F D(S,µ,Ψ,ϵ)} = h(x, µ). h(x, µ) (1 + β ) 1 R C 2 E G [ ξ + µ h(x, µ) (1 + β ) 1 R C 2 µ (1 + β ) 1 R C 2 O(β 1/γ )

4 ec4 e-companion to Delage et al.: The Value of Stochastic Modeling We can then resolve an approximate upper bound using Proposition 1 and the Lipschitz property of h(x, ): E F [h(x, ξ) F D(S,µ,Ψ,ϵ) {z z µ ϵ} F D(R d,z,ψ) {z z µ ϵ} E F [h(x, ξ) h(x, z) h(x, µ)+o(ϵ). We are left with demonstrating near-optimality of the MVP solution. This is easily done using the following argument, where for conciseness we let x MVP refer to the optimal solution of the MVP problem, g(x) beequalto F D(S,µ,Ψ,ϵ) c T 1x+E F [h(x, ξ), and x g be the member of X that minimizes g(x). g(x MVP ) g(x g ) c T 1x MVP + h(x MVP, µ)+o(ϵ) c T 1x g h(x g, µ) =min x X ct 1 x + h(x, µ) (ct 1 x g + h(x g, µ)) + O(ϵ) O(ϵ), so that g(x MVP ) g(x g )+O(ϵ). EC.5. Proof of Lemma 2 We first re-parametrize as z(,δ):=2µ + δ. We therefore need to show that for any R d g(δ):= µ + δ γ α 1 2 γ 1 2µ + δ γ α + µ γ α 0, δ R. This is done by showing that g(δ) isdecreasingfornegativeδ s, achieves the value of zero at δ =0 and is increasing for positive δ s. The function g(δ) therefore achieves its minimum value of zero at δ =0 for any. The simplest step is to show thatg(0) = 0. g(0) = µ γ α 1 2 γ 1 2µ γ α + µ γ α =0. The second step requires us to realize that g(δ) canberepresentedintermsoftheconvexfunction g 2 (δ)= µ + δ γ α: g(δ)=g 2 (δ) 2g 2 (δ/2) + µ γ α. One can verify that g 2 (δ) isconvexusingthefactthatitisthecompositionofafunction y γ,which is convex and increasing over y 0forγ 1, and of a convex function µ + δ α.theconvexityof g 2 (δ) tellsusthatg 2 (δ) g 2 (δ/2) for δ 0andthatg 2 (δ) g 2 (δ/2) for δ 0. Thus, we can easily verify the properties of the derivative of g(δ). While for δ 0, we have g (δ)=g 2(δ) g 2(δ/2) 0, we can also easily show that, for δ 0, we have g (δ)=g 2 (δ) g 2 (δ/2) 0. This completes our proof.

5 ec5 EC.6. Proof of Lemma 3 To mae this demonstration, we use the fact that h(x 2, ξ)=x 2 h(1, ξ). First, in the case where x 2 = 0, we already verified that h(0, ξ)=0. When x 2 > 0, one can easily show that h(x 2, ξ)=x 2 h(1, ξ) for all ξ R d by replacing the inner variable y by z = y/x 2.Thus,forallx 2 0, the objective of problem (6) reduces to F D(R d,0 d,i) {E F [ cx 2 h(x 2, ξ)} = F D(R d,0 d,i) = F D(R d,0 d,i) EC.7. Proof of Lemma 4 {E F [( c h(1, ξ))x 2 } {E F [( c h(1, ξ))} x 2. Based on Lemma 1 of Delage and Ye (2010), considering that h(1, ξ) isf -integrable for all F D(R d, 0 d, I) since E F [h(1, ξ) E F [ ξ 1 de F [ ξ 2 d E F [ ξ 2 2=d, by duality theory we can say that evaluating the remum of such an expression is equivalent to finding the optimal value of minimize Q,q,t t + I Q subject to t c ξ T y ξ T Qξ q T ξ, ξ R d, y Y(1) where Y(1) = {y R d a T y =0& 1 y i 1, i}. Thisisnecessarilyaconvexoptimization problem with linear objective function since each constraint is jointly convex in Q, q, andt. We now show that the separation problem associated with the only constraintofthisproblemisnphard. Thus, by the equivalence of optimization and separation (see Grötschel et al. (1981)) finding the optimal value of this problem can be shown to be NP-hard. Consider separating the solution Q =(1/4)I, q = 0 m,andt = d c from the feasible set. In this case, we must be able to verify its feasibility with respect to theonlyconstraint.thiscanbeshown to reduce to verifying if ξ R d, y Y(1) ξ T y (1/4)ξ T ξ d. After solving in terms of ξ, wegetthatweneedtoverifyiftheoptimalvalueoftheproblem imize y y T y subject to 1 y i 1, i {1, 2,...,d} a T y =0,

6 ec6 e-companion to Delage et al.: The Value of Stochastic Modeling is greater or equal to d or not. This is equivalent to showing if the set defined by Υ(a)={ y { 1, 1} d a T y =0}, for any a R d,isemptyornotsincetheextremepointoftheunitboxaretheonly ones for which y 2 d. Finally,onecaneasilyconfirmthattheNP-completePartition problem can be reduced to verifying whether some Υ(a) isemptyornot. Partition Problem: Givenasetofpositiveintegers{s i } d i=1 with index set A = {1, 2,...,d} is there a partition (B 1, B 2 )ofasuch that i B 1 s i = i B 2 s i? The reduction is obtained by casting a i := s i for all i, andconsideringthatafeasiblesolution y Υ(a) onlyexistsifsuchapartitionexistsandidentifiesthepartition through B 1 = {i A y i = 1}. Thiscompletesourproof. EC.8. Proof of Proposition 6 We first represent D(S, µ, I) intheformproposedinexample1,i.e.,using Ψ={ψ : R d R Q 0,ψ(ξ)=Q ((ξ µ)(ξ µ) T I)} It is indeed the case that this set Ψ satisfies Assumption 3 since the positive semi-definite cone is one for which it is relatively easy to verify feasibility using singular value decomposition. In formulating problem (8), we get that constraint (8b) taes the shape: ξ [ν i τ i,ν i + τ i t w i + ξv i α i β i (ξ ξ i ) q i(ξ µ i ) Q ((ξ µ i ) 2 e i e T i I), i {1, 2,...,d} {1, 2,...,K} Asimplereplacementoft for t + I Q leads to the equivalent formulation of problem (8) minimize t + I Q t,q,q,{w,v } K =1 ξ [ν i τ i,ν i + τ i subject to t w i + ξv i α i β i (ξ ξ i ) q i (ξ µ i ) (ξ µ i ) 2 Q i,i, i {1, 2,...,d} {1, 2,...,K} { i {1, 2,...,d} w i + v i ξm i c T 1 x 1 + h(x 1, µ +(ξ m i µ i )e i ), m, {1, 2,...,K} Q 0. Since only the diagonal terms of Q are involved in both the objective function and the constraints, one can arbitrarily set all off diagonal terms of Q to zero. Thus, we are left with minimize t,q,r,{w,v } K =1 t + e T r

7 ξ [ν i τ i,ν i + τ i subject to t w i + ξv i α i β i (ξ ξ i ) q i (ξ µ i ) (ξ i µ i ) 2 r i, i {1, 2,...,d} {1, 2,...,K} { i {1, 2,...,d} w i + v i ξm i c T 1 x 1 + h(x 1, µ +(ξ m i µ i )e i ), m, {1, 2,...,K} r 0. ec7 We finally apply the S-Lemma (cf., Theorem 2.2 in Póli and Terlay (2007)) to replace, for each fixed i and, thesetofconstraintsindexedovertheinterval[ν i τ i,ν i + τ i byanequivalent linear matrix inequality: [ r i β i v i +q i 2µ i r i 2 β i v i +q i 2µ i r i 2 t w i + α i β i ξ i µ i q i + µ 2 i r i s i [ 1 νi ν i τ 2 i, s i 0. Now regarding the complexity of solving this semi-definite programming problem, it is well nown that in the standard form ( ( K the problem can be solved in O m i=1 i minimize c T x x Rñ subject to A i (x) 0 i =1, 2,..., K ) 0.5 ( ñ 2 K ) ) K i=1 m2 i +ñ i=1 m3 i, where m i stands for the dimension of the positive semi-definite cone (i.e., A i (x) R m i m i ) (see Nesterov and Nemirovsi (1994)). In the SDP that interest us, one can show that ñ =1+(3K +2)d and that both problems can be solved in O(K 5 d 3.5 )operations,withk being the number of scenarios for each random variable that composes ξ.incalculatingthetotalcomplexityofevaluatinglb(x 1, X 2, {ξ i }) under such a Ψ, we also need to account for O(KdT MVP )operationsforevaluatingtherequired α i and β i.thiscompletesourproof. EC.9. Proof of Proposition 7 We first present without proof a Lemma that describes how to approximate a concave function on the real line to any level of accuracy by containing it between an outer and an inner piecewise linear concave functions. Lemma EC.3. Given any set of points {z i } K =1,aconcavefunctiong : R R is contained between the two piecewise linear concave functions. Specifically, g inner (z) g(z) g outer (z), where g outer (z):= min {1,2,...,K} with g (z ) R as any er-gradient of g(z) at z,and g(z )+(z z )g (z ), g inner (z):= min w,v w + zv subject to w + z v g(z ), {1, 2,...,K}.

8 ec8 e-companion to Delage et al.: The Value of Stochastic Modeling We then reduce the intractable tas of evaluating MVSM(x 1 )tothesearchforadistribution in the more manageable set D(S 0, µ, Ψ). The analysis is also limited to the potential regret of having committed to x 1 instead of ˆx 2.ThisnaturallyleadstoalowerboundfortheMVSM which considers distributions in the larger set D(S, µ, Ψ) and any alternative decision in X 2 MVSM(x 1 ) F D(S 0,µ,Ψ) E F [c T 1(x 1 ˆx 2 )+h(x 1, ξ) h(ˆx 2, ξ) We then apply duality theory to the F D(S0,µ,Ψ) operation. Considering the primal problem as a semi-infinite linear conic problem F M E F [c T 1(x 1 ˆx 2 )+h(x 1, ξ) h(ˆx 2, ξ) subject to E F [1{ξ S 0 }=1 E F [ξ=µ E F [r T ψ(ξ) 0, r K, (EC.1a) (EC.1b) (EC.1c) (EC.1d) the dual form taes the shape minimize t,q,r t subject to t c T 1(x 1 ˆx 2 )+h(x 1, ξ) h(ˆx 2, ξ) (ξ µ) T q r T ψ(ξ), ξ S 0 r K, (EC.2a) (EC.2b) (EC.2c) where t, q, andr are the dual variables associated respectively with constraints (EC.1b), (EC.1c), and (EC.1d). One can verify that there is no duality gap between primal and dual problems using the weaer version of Proposition 3.4 in Shapiro (2001) and the fact that the Dirac measure δ µ lies in the relative interior of D(S 0, µ, Ψ). Exploiting the structure of S 0 d i=1 {ξ ξ [ν i τ i,ν i + τ i, ξ = µ +(ξ µ i )e i },wecandecompose constraint (EC.2b) into d simpler constraints t g i (x 1,ξ) g i (ˆx 2,ξ) q i (ξ µ i ) r T ψ(µ +(ξ µ i )e i ) i {1, 2,...,d}(EC.3) where g i (x,ξ)=c T 1 x + h(x, µ +(ξ µ i)e i ). Note that if S 0 = S 0 B (ν, τ ), then this set of constraints is entirely equivalent to the original one. For any fixed i, we now go through a sequence of relaxation steps for constraint (EC.3). t = = min (w,v) W i g i (x 1,ξ) g i (ˆx 2,ξ) q i (ξ µ i ) r T ψ(µ +(ξ µ i )e i ) g i (x 1,ξ) min {α i +(ξ ξ i )β i } q i (ξ µ i ) r T ψ(µ +(ξ µ i )e i ) g i (x 1,ξ) α i (ξ ξ i )β i q i (ξ µ i ) r T ψ(µ +(ξ µ i )e i ) min {w + ξv} α i β i (ξ ξ i ) q i(ξ µ i ) r T ψ(µ +(ξ µ i )e i ) (w,v) W i w + ξv α i β i (ξ ξ i ) q i(ξ µ i ) r T ψ(µ +(ξ µ i )e i ),

9 ec9 where W i is short for {(w,v) R 2 w + ξ m v g i (x 1,ξ m ) m {1, 2,...,K}}. Thetworelaxation steps are obtained through the application of Lemma EC.3 to first replace g(x 2, ξ) byitsouter approximation, and then g(x 1, ξ) byitsinnerapproximation.thelastequalityisobtainedby inversing the order of ξ [νi τ i,ν i +τ i and min (w,v) Wi using Sion s mini theorem (see Sion (1958)). Thus, we obtain the optimization problem presented indefinition1. References Ben-Tal, A., A. Nemirovsi Robust convex optimization. Mathematics of Operations Research 23(4) Boyd, S., L. Vandenberghe Convex Optimization. 1st ed. Cambridge University Press. Delage, E., Y. Ye Distributionally robust optimization under moment uncertainty with application to data-driven problems. Operations Research 58(3) Grötschel, M., L. Lovász, A. Schrijver The ellipsoid method and its consequences in combinatorial optimization. Combinatorica Nesterov, Y., A. Nemirovsi Interior-point polynomial methods in convex programming, vol. 13. Studies in Applied Mathematics, Philadelphia. Póli, I., T. Terlay A survey of the S-Lemma. SIAM Review 49(3) Shapiro, A On duality theory of conic linear problems. M. A. Goberna, M. A. López, eds., Semi-Infinite Programming: Recent Advances. Kluwer Academic Publishers, Dordrecht, Sion, M On general mini theorems. Pacific Journal of Mathematics 8(1)