Complexity of two and multi-stage stochastic programming problems

Transcription

1 Complexity of two and multi-stage stochastic programming problems A. Shapiro School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia , USA

2 The concept of two-stage (linear) stochastic programming problem with recourse Min x X c x + E[Q(x, ξ)], (1) where X = {x : Ax = b, x 0}, c x denotes the scalar product of vectors c, x R n, and Q(x, ξ) is the optimal value of the second stage problem Min y q T y s.t. T x + W y = h, y 0, (2) with ξ = (q, T, W, h). (By bold script like ξ we sometimes denote random variables to distinguish them from a particular realization ξ.) The feasible set X can be finite, i.e., integer first stage problem. Both stages can be integer (mixed integer) problems. Suppose that the probability distribution P of ξ has a finite support, i.e., ξ can take values ξ 1,..., ξ K (called scenarios) with respective probabilities p 1,..., p K. In that case 1

3 where E P [Q(x, ξ)] = K k=1 p k Q(x, ξ k ), Q(x, ξ k ) = inf {q k y k : T k x + W k y k = h k, y k 0}. It follows that we can write problem (1)-(2) as one large linear program: Min x,y 1,...,y K c x + K k=1 p k (q k y k ) subject to Ax = b, T k x + W k y k = h k, k = 1,..., K, x 0, y k 0, k = 1,..., K. Multistage models Let ξ t be a random process. Denote ξ [1,t] = (ξ 1,.., ξ t ) the history of the process ξ t up to time t. The values of the decision vector x t, chosen at stage t, may depend on the information ξ [1,t] available up to time t, but not on the future observations. The decision process has the form decision(x 0 ) observation(ξ 1 ) decision(x 1 )... observation(ξ T ) decision(x T ). 2

4 There are several ways how this decision process can be made precise. Nested formulation of a linear multistage stochastic programming problem with recourse: Min A 1 x 1 =b 1 x 1 0 c 1 x 1 + E [ + E { Min B 2 x 1 +A 2 x 2 =b 2 x 2 0 Min B T x T 1 +A T x T =b T x T 0 c 2 x 2 + c T x T ]}. Here ξ t = (c t, B t, A t, b t ), t = 2,..., T, is considered as a random process, ξ 1 = (c 1, A 1, b 1 ) is supposed to be known, and x t = x t (ξ [1,t] ), t = 2,..., T, are supposed to be functions of the history of the process up to time t. If the number of realizations (scenarios) of the process ξ t is finite, then the above problem can be written as one large linear programming problem. In that respect it is convenient to represent the random process in a form of scenario tree. 3

5 Dynamic programming equations. Consider the last stage linear program of the multistage problem: Min x T c T x T s.t. B T x T 1 + A T x T = b T, x T 0. The optimal value of this problem depends on x T 1 and data ξ T = (c T, B T, A T, b T ), and is denoted Q T (x T 1, ξ T ). At stage t = T 1,..., 2 we consider the optimal value Q t (x t 1, ξ [1,t] ) of the problem Min x t c t x t + E [ Q t+1 (x t, ξ [1,t+1] ) s.t. B t x t 1 + A t x t = b t, x t 0. ] ξ [1,t] At the first stage we obtain the problem: Min x 1 c 1 x 1 + E [Q 2 (x 1, ξ 2 )] s.t. A 1 x 1 = b 1, x 1 0. (3) (4) The expectation in (3) is conditional on the history of the process ξ t up to time t. If the process is Markovian, then this expectation depends only on ξ t, and hence the cost-to-go function Q t (x t 1, ξ t ) is a function of x t 1 and ξ t. 4

6 How difficult is to solve two-stage problems? What about multistage problems? Consider stochastic optimization problem: { } f(x) := E P [F (x, ξ)], Min x X where ξ is a random vector having probability distribution P, F (x, ξ) is a real valued function and X R n. If P has a finite support (i.e., finite number ξ 1,..., ξ K of scenarios), then E P [F (x, ξ)] = K k=1 p k F (x, ξ k ). However, the number K of scenarios grows exponentially with dimension of the data ξ. For example, if d random components of ξ are independent, each having just 3 possible realizations, then the total number of scenarios K = 3 d. No computer in a foreseeable future will be able to handle calculations involving scenarios. 5

7 Monte Carlo sampling approach Let ξ 1,..., ξ N be a generated (iid) random sample drawn from P and ˆf N (x) := N 1 N j=1 F (x, ξ j ) be the corresponding sample average function. By the Law of Large Numbers, for a given x X, we have ˆf N (x) f(x) = E P [F (x, ξ)] w.p.1 as N. Notoriously slow convergence of order O p (N 1/2 ). By the Central Limit Theorem N 1/2 [ ˆf N (x) f(x) ] N(0, σ 2 (x)), where σ 2 (x) := Var[F (x, ξ)]. In order to improve the accuracy by one digit the sample size should be increased 100 times. Good news: rate of convergence does not depend on the number of scenarios, only on the variance σ 2 (x). The accuracy can be improved by variance reduction techniques. However, the rate of the square root of N (of Monte Carlo sampling estimation) cannot be changed. 6

8 Monte Carlo sampling optimization Two basic philosophies: interior and exterior Monte Carlo sampling. In interior sampling methods, sampling is performed inside a chosen algorithm with new (independent) samples generated in the process of iterations (e.g., Higle and Sen (stochastic decomposition), Infanger (statistical L-shape method), Norkin, Pflug and Ruszczynski (stochastic branch and bound method)). In the exterior sampling approach the true problem is approximated by the sample average approximation problem: (SAA) Min x X ˆf N (x) := N 1 N j=1 F (x, ξ j ). Once the sample ξ 1,..., ξ N P is generated, the SAA problem becomes a deterministic optimization and can be solved by an appropriate algorithm. Difficult to point out an exact origin of this method. Variants of this approach were suggested by a number of authors under different names. 7

9 Advantages of the SAA method: Ease of numerical implementation. one can use existing software. Often Good convergence properties. Well developed statistical inference: validation and error analysis, stopping rules. Easily amendable to variance reduction techniques. Ideal for parallel computations. The idea of common random numbers generation. Suppose that X = {x 1, x 2 }. Then the variance of N 1/2 [ ˆf N (x 1 ) ˆf N (x 2 ) ] is: Var[F (x 1, ξ)]+var[f (x 2, ξ)] 2 Cov[F (x 1, ξ), F (x 2, ξ)]. It can be much smaller than Var[F (x 1, ξ)] + Var[F (x 2, ξ)] when the samples are independent. 8

10 Notation v 0 is the optimal value of the true problem S 0 is the optimal solutions set of the true problem S ε is the set of ε-optimal solutions of the true prob ˆv N is the optimal value of the SAA problem ŜN ε is the set of ε-optimal solutions of the SAA pro ˆx N is an optimal solution of the SAA problem Convergence properties Vast literature on statistical properties of the SAA estimators ˆv N and ˆx N : Consistency. By the Law of Large Numbers, ˆf N (x) converge (pointwise) to f(x) w.p.1. Under mild additional conditions, this implies that ˆv N v 0 and dist(ˆx N, S 0 ) 0 w.p.1 as N. In particular, ˆx N x 0 w.p.1 if S 0 = {x 0 }. (Consistency of Maximum Likelihood estimators, Wald (1949)). Central Limit Theorem type results. ˆv N = min x S 0 ˆf N (x) + o p (N 1/2 ). In particular, if S 0 = {x 0 }, then N 1/2 [ˆv N v 0 ] N(0, σ 2 (x 0 )). 9

11 These results suggest that the optimal value of the SAA problem converges at a rate of N. In particular, if S 0 = {x 0 }, then ˆv N converges to v 0 at the same rate as ˆf N (x 0 ) converges to f(x 0 ). If S 0 = {x 0 }, then under certain regularity conditions, N 1/2 (ˆx N x 0 ) converges in distribution. (Asymptotic normality of M-estimators, Huber (1967)). The required regularity conditions are that the expected value function f(x) is smooth (twice differentiable) at x 0 and the Hessian matrix 2 f(x 0 ) is positive definite. This typically happens if the probability distribution P is continuous. In such cases ˆx N converges to x 0 at the same rate as the stochastic approximation iterates calculated with the optimal step sizes (Shapiro, 1996). 10

12 Large Deviations type bounds. Let Y 1,..., Y N be an iid sequence of random variables, and Z N := N 1 Nj=1 Y j. Then (the upper bound of Cramér s LD theorem) P(Z N z) exp { NI(z) } for z > µ, where µ := E[Y ] and { } I(z) := sup tz Λ(t), t R Λ(t) := log M(t) and M(t) := E[e ty ] is the moment generating function. Assume that M(t) is finite valued for all t in a neighborhood of zero (if M(t) = + for any t 0, then I(z) 0 and hence the above upper bound trivially holds). Note that Λ(t) is convex, Λ (0) = µ and Λ (0) = σ 2, where σ 2 := Var[Y ]. The rate function I(z) is convex, I (µ) = 0, I (µ) = σ 2, and I(z) attains its minimum at z = µ. Consequently, (z µ)2 I(z) = 2σ 2 + o( z µ 2 ), and I(z) > 0 for any z µ. Note that if Y N(µ, σ 2 ), then M(t) = exp{µt + σ 2 t 2 /2} and hence I(z) = (z µ)2 2σ 2. 11

13 If M(t) exp{µt + σ 2 t 2 /2}, then I(z) (z µ)2 2σ 2 and hence for z µ, { } N(z µ) 2 P(Z N z) exp 2σ 2. If distribution of Y is supported on a bounded interval [a, b], then for z µ (Hoeffding s inequality): { } N(z µ) 2 P(Z N z) exp (b a) 2. /2 Uniform LD bounds. Let X be a subset of R n of finite diameter D := sup x,x X x x, ξ 1,..., ξ N be an iid sample from a distribution supported on Ξ R d. Suppose that: (i) there is a constant σ > 0 such that M x (t) {σ 2 t 2 /2}, t R, x X, where M x (t) is the moment generating function of the random variable F (x, ξ) f(x), (ii) there is a constant L > 0 such that F (x, ξ) F (x, ξ) L x x, ξ Ξ, x, x X. 12

14 Then for any γ > 0, P { sup x X ˆf N (x) f(x) γ } ( ) } O(1)DL n γ exp { Nγ2 16σ 2. Note that if ˆx N is a δ-optimal solution of the SAA problem and sup x X ˆf N (x) f(x) < γ, then ˆx N is an ε-optimal solution of the true problem with ε = γ + δ. Now choose ε > 0, δ [0, ε) and α (0, 1). Then, for γ := ε δ, by solving ( ) n { } O(1)DL exp Nγ2 γ 16σ 2 α, we obtain that for sample size N O(1)σ2 [ ( O(1)DL (ε δ) 2 n log (ε δ) 2 ) + log ( 1 α) ] we are guaranteed that with probability at least 1 α any δ-optimal solution of the SAA problem is an ε-optimal solution of the true problem. 13

15 If the set X is finite, then the corresponding estimate of the sample size becomes (under the assumption (i)): N 2σ2 (ε δ) 2 log ( X α ). Example Let F (x, ξ) := x 2k 2k ( ξ T x ), where k N and X := {x R n : x 1}. Suppose, that ξ N(0, σ 2 I n ). Then f(x) = x 2k, and for ε [0, 1], the set of ε-optimal solutions of the true problem is {x : x 2k ε}. Let ξ N := (ξ ξ N )/N. The corresponding sample average function is ˆf N (x) = x 2k 2k ( ξ T N x), and ˆx N = ξ N γ ξ N, where γ := 2k 2 2k 1 if ξ N 1, and γ = 1 if ξ N > 1. Therefore, for ε (0, 1), the optimal solution of the SAA problem is an ε-optimal solution of the true problem iff ξ N ν ε, where ν := 2k 2k 1. 14

16 We have that ξ N N(0, σ 2 N 1 I n ), and hence N ξ N 2 /σ 2 has the chi-square distribution with n degrees of freedom. Consequently, the probability that ξ N ν > ε is equal to the probability P ( χ 2 n > Nε 2/ν /σ 2). Moreover, E[χ 2 n] = n and the probability P(χ 2 n > n) increases and tends to 1/2 as n increases. Consequently, for α (0, 0.3) and ε (0, 1), for example, the sample size N should satisfy N > nσ2 ε 2/ν (5) in order to have the property: with probability 1 α an (exact) optimal solution of the SAA problem is an ε-optimal solution of the true problem. Note that ν 1 as k. 15

17 Suppose that the true problem is convex piecewise linear. That is: (i) the distribution P has a finite support, (ii) for almost every ξ the function F (, ξ) is piecewise linear and convex, (iii) the feasible set X is polyhedral (i.e., is defined by a finite number of linear constraints). Suppose also that the optimal solutions set S 0, of the true problem, is nonempty and bounded. Then : (1) W.p.1 for N large enough, ˆx N is an exact optimal solution of the true problem. More precisely, w.p.1 for N large enough, the set Ŝ N of optimal solutions of the SAA problem is nonempty and forms a face of the (polyhedral) set S 0. (2) Probability of the event {Ŝ N S 0 } tends to one exponentially fast. That is, there exists a constant γ > 0 such that 1 lim N N log [ 1 P (Ŝ N S 0 ) ] = γ. (Shapiro & Homem-de-Mello, 2000) 16

18 The idea of repeated solutions. Solve the SAA problem M times using M independent samples each of size N. Let ˆv (1) N,..., ˆv(M) N be the optimal values and ˆx (1) N,..., ˆx(M) N be optimal solutions of the corresponding SAA problems. Probability that at least one of ˆx (i) N, i = 1,..., M is an optimal solution of the true problem is 1 p M N where and hence p N := P (ˆx N x 0 ) CN 1/2 e Nγ. p M N (CN 1/2 ) M e NMγ. Cutting plane (Benders cuts, L-shape) type algorithms. Empirical observation: on average the number of iterations (cuts) does not grow, or grows slowly, with increase of the sample size N. From theoretical point of view it converges to the respective number of the true problem. 17

19 Validation analysis How one can evaluate quality of a given solution ˆx S? Two basic approaches: (1) Evaluate the gap f(ˆx) v 0. (2) Verify the KKT optimality conditions at ˆx. Statistical test based on estimation of f(ˆx) v 0 (Norkin, Pflug & Ruszczynski 98, Mak, Morton & Wood 99): (i) Estimate f(ˆx) by the sample average ˆf N (ˆx), using sample of a large size N. (ii) Solve the SAA problem M times using M independent samples each of size N. Let ˆv (1) N,..., ˆv(M) N be the optimal values of the corresponding SAA problems. Estimate E[ˆv N ] by the average M 1 M j=1 ˆv (j). Note that [ E ˆf N (ˆx) M 1 M j=1 ˆv (j) N = ( f(ˆx) v 0 ) + ( v 0 E[ˆv N ] ), N ] and that v 0 E[ˆv N ] > 0. For ill-conditioned problems the bias v 0 E[ˆv N ] can be large. 18

20 The bias v 0 E[ˆv N ] is positive and (under mild regularity conditions) lim N 1/2 ( v 0 E[ˆv N ] ) = E N [ max Y (x) x S 0 where (Y (x 1 ),..., Y (x k )) has a multivariate normal distribution with zero mean vector and covariance matrix given by the covariance matrix of the random vector (F (x 1, ξ),..., F (x k, ξ)). For ill-conditioned problems this bias is of order O(N 1/2 ) and can be large if the ε-optimal solution set S ε is large for some small ε 0. ], Common random numbers variant: generate a sample (of size N) and calculate the gap ˆf N (ˆx) inf x X ˆf N (x). Repeat this procedure M times (with independent samples), and calculate the average of the above gaps. This procedure works well for well conditioned problems, does not improve the bias problem. 19

21 KKT statistical test Let X := {x R n : c i (x) = 0, i I, c i (x) 0, i J}. Suppose that the probability distribution is continuous. Then F (, ξ) is differentiable at ˆx w.p.1 and f(ˆx) = E P [ x F (ˆx, ξ)]. KKT-optimality conditions at an optimal solution x 0 S 0 can be written as follows: where C(x) := y = f(x 0 ) C(x 0 ), i I J(x) λ i c i (x), λ i 0, i J(x) and J(x) := {i : c i (x) = 0, i J}. The idea of the KKT test is to estimate the distance δ(ˆx) := dist ( f(ˆx), C(ˆx)), by using the sample estimator ˆδ N (ˆx) := dist ( ˆf N (ˆx), C(ˆx) ). The covariance matrix of ˆf N (ˆx) can be estimated (from the same sample), and hence a confidence region for f(ˆx) can be constructed. This allows a statistical validation of the KKT conditions. (Shapiro & Homem-de-Mello 98). 20,

22 Complexity of multistage stochastic programming Consider the following T stage stochastic programming problem [ Min F 1 (x 1 ) + E inf F 2(x 2, ξ 2 )+ x 1 G [ 1 x 2 G 2 (x 1,ξ 2 ) E + E [ inf F T (x T, ξ T ) ]]] x T G T (x T 1,ξ T ) driven by the random data process ξ 2,..., ξ T. Here x t R n t, t = 1,..., T, are decision variables, F t : R n t R d t R are continuous functions and G t : R n t 1 R d t R n t, t = 2,..., T, are measurable multifunctions, the function F 1 : R n 1 R and the set G 1 R n 1 are deterministic. We assume that the set G 1 is nonempty. For example, in linear case F t (x t, ξ t ) := c t, x t, G 1 := {x 1 : A 1 x 1 = b 1, x 1 0}, G t (x t 1, ξ t ) := {x t : B t x t 1 + A t x t = b t, x t 0}, ξ 1 := (c 1, A 1, b 1 ) is known at the first stage (and hence is nonrandom), and ξ t := (c t, B t, A t, b t ), t = 2,..., T, are data vectors some (all) elements of which can be random. 21

23 Conditional sampling: let ξ2 i, i = 1,..., N 1, be an iid random sample of ξ 2. Conditional on ξ 2 = ξ2 i, a random sample ξij 3, j = 1,..., N 2, is generated and etc. The obtained scenario tree is considered as a sample average approximation of the true problem. Note that the total number of scenarios N = T 1 t=1 N t. For T = 3, under certain regularity conditions, for ε > 0 and α (0, 1), and the sample sizes N 1 and N 2 satisfying ) n1 { D1 L O(1)[( 1 ε exp ) n2 { exp ( D1 L 3 ε ) n1 ( D2 L 2 ε O(1)N 1ε 2 σ 2 1 } + O(1)N 2ε 2 σ 2 2 } ] α, we have that any ε/2-optimal solution of the SAA problem is an ε-optimal solution of the true problem with probability at least 1 α. In particular, suppose that N 1 = N 2. Then the required sample size N 1 = N 2 : N 1 O(1)σ2 ε 2 [ (n 1 + n 2 ) log ( DL ε ) + log ( O(1) α )]. 22

24 Example of financial planning. Suppose that we can rebalance our portfolio at several, say T, periods of time. That is, at the beginning we choose values x i0 of our assets subject to the budget constraint ni=1 x i0 = W 0. At the period t = 1,..., T, our wealth is W t = n i=1 ξ it x i,t 1, where ξ it = (1 + R it ) and R it is the return of the i-th asset at the period t. Our objective is to maximize the expected utility Max E [U(W T )] at the end of the considered period, subject to the balance constraints ni=1 x it = W t and x t 0, t = 0,..., T 1. The values of the decision vector x t = (x 1t,..., x nt ), chosen at stage t, may depend on the information ξ [1,t] available up to time t, but not on the future observations. 23

25 The decision process has the form: decision(x 0 ) observation(ξ 1 ) decision(x 1 )... observation(ξ T ) decision(x T ). Dynamic programming equations for this problem can be derived as follows. At the last stage t = T 1 we have to solve the problem Max x T 1 0 s.t. E [ n i=1 U ( n i=1 ξ i,t x i,t 1 ) ξ [1,T 1] ] x i,t 1 = W T 1. ( ) Its optimal value Q T 1 WT 1, ξ [1,T 1] is a function of W T 1 and ξ [1,T 1]. At stage t = T 2,..., 0 we solve Max x t 0 s.t. E [ n i=1 Q t+1 ( n i=1 x it = W t. ξ i,t+1 x it, ξ [1,t+1] ) ξ [1,t] ] Its optimal value is Q t ( Wt, ξ [1,t] ). Suppose now that the random process ξ t is between stages independent. Then the costto-go function Q t (W t ) does not depend on ξ t. 24

26 Consider power utility function U(z) := z γ, with γ 1. Then Q T 1 ( WT 1 ) = W γ T 1 Q T 1 (1), and by induction Q 1 (W 1 ) = W γ 1 T 1 t=1 Q t (1). Consequently, the first stage optimal solution is obtained in a myopic way by solving the problem: Max E x 0 0 [( n i=1 ξ i1 x i0 ) γ ] s.t. n i=1 x i0 = W 0. The optimal value of the corresponding multistage problem is v = W γ 0 T 1 t=0 Q t (1). (6) Suppose that we apply the SAA method to this problem using conditional sampling of size N t at stage t = 1,..., T. In accordance with (6) we have that the optimal value ˆv N of the SAA 25

27 problem can be written in the form ˆv N = W γ 0 T 1 t=0 ˆQ Nt+1 (1), (7) where ˆQ Nt+1 (W t ) is the optimal value of the SAA counterpart of the (true) problem. Note that since the conditional sample is generated here in the between stages independent way, the random variables ˆQ Nt+1 (1) are mutually independent. Therefore E [ˆv N ] = W γ 0 where T 1 t=0 E [ ˆQ Nt+1 (1) ] = v T 1 (1 + β t ), t=0 β t := E [ ˆQ Nt+1 (1) ] Q t (1) Q t (1) is the relative bias of the optimal value of the corresponding t-th stage SAA problem. This indicates that the bias of the optimal value of the SAA problem growth exponentially with the number of stages.