Monte Carlo Methods for Stochastic Programming

Transcription

1 IE 495 Lecture 16 Monte Carlo Methods for Stochastic Programming Prof. Jeff Linderoth March 17, 2003 March 17, 2003 Stochastic Programming Lecture 16 Slide 1

2 Outline Review Jensen's Inequality Edmundson-Madansky Inequality Partititioning Using Bounds in Algorithms Monte Carlo Methods Estimating the optimal objective function value Lower Bounds Upper Bounds Examples March 17, 2003 Stochastic Programming Lecture 16 Slide 2

3 Review Why do we care about bounds in stochastic programming? What is Jensen's inequality? How is Jensen's Inequality used in stochastic programming? What is the Edmundson-Madansky Inequality? How is the Edmundson-Madansky Inequality used in stochastic programming? March 17, 2003 Stochastic Programming Lecture 16 Slide 3

4 Jensen's Inequality in Stochastic Programming If φ is a convex function ω of a random variable over its support Ω, then E ω φ(ω) φ(e ω (ω)) E ω [Q(ˆx, ω)] Q(ˆx, E ω [ω]) Let S = {Ω l, l = 1, 2,... v} be some partition of Ω: E ω [Q(ˆx, ω)] v P (ω Ω l )Q(ˆx, E ω (ω ω Ω l )) l=1 March 17, 2003 Stochastic Programming Lecture 16 Slide 4

5 Edmundson-Madansky Inequality If Ω has a (nite) set of extreme points ext(ω), then there is some probability measure p i (convex multipliers) so that e ext(ω) p ei Q(ˆx, e) Q(ˆx, ω). If Ω = [a 1, b 1 ] [a 2, b 2 ]... [a d, b d ] and if the random variables in the d dimensions are independent... U(ˆx, ω) = d ( ) ωi e i Q(ˆx, e) E ω Q(w, ω) = Q(x) b i a i e ext(ω) i=1 March 17, 2003 Stochastic Programming Lecture 16 Slide 5

6 Bounds in LShaped Method To use bounds, use the θ to underestimate a lower bounding function Q L (x). (We called this E ω L(x, ω) on Monday). Use the LShaped method to optimize the problem using Q L (x). Only include ω scenarios. When optimized with respect to Q L (x), compare to Q U (x) (what we called E ω U(x, ω)). If Q U (x) Q L (x) is sufciently small. Stop. Otherwise, rene the partition (which improves the bounds), and repeat. March 17, 2003 Stochastic Programming Lecture 16 Slide 6

7 Monte Carlo Methods ( ) min {f(x) E P g(x; ξ) x S Ω g(x; ξ)dp (ξ)} Most of the theory presented holds for (*) A very general SP problem Naturally it holds for our favorite SP problem: S {x Ax = b, x 0} f(x) c T x + Q(x) Q(x) E{Q(x, ω)} Q(x, ω) min y 0 {q(ω) T y W y = h(ω) T (ω)x} March 17, 2003 Stochastic Programming Lecture 16 Slide 7

8 Sampling Instead of solving (*), we solve an approximating problem. Let ξ 1,..., ξ N be N realizations of the random variable ξ: min { f N (x) N 1 x S N g(x, ξ j )}. j=1 f N (x) is just the sample average function Since ξ j drawn from P, f N (x) is an unbiased estimator of f(x) E[ f N (x)] = f(x) March 17, 2003 Stochastic Programming Lecture 16 Slide 8

9 Sample Variance Since ξ j are independent, we can estimate Var( f N (x)) : This is known as the sample variance: ˆσ 2 (x) = 1 N(N 1) N [(g(x, ξ j ) f N (x)] 2 j=1 March 17, 2003 Stochastic Programming Lecture 16 Slide 9

10 Statistics Break Let χ 1, χ 2,..., χ n be independent, identically distributed (iid) random variables. Let S n = n i=1 χ i Assume µ E χ i <. Weak Law of Large Numbers ( ) lim P ( S n n n µ δ = 0 δ > 0 March 17, 2003 Stochastic Programming Lecture 16 Slide 10

11 Strong Law of Large Numbers Almost surely. lim n S n n µ Almost surely Equivalent to with probability 1, or.. P ( lim n S n n µ) = 0 March 17, 2003 Stochastic Programming Lecture 16 Slide 11

12 Central Limit Theorem Further assume that χ 1, χ 2,..., χ n have nite nonzero variance σ 2 : lim P n ( Sn nµ σ n ) x = N (0, 1) N (µ, σ 2 ) : Normally distributed random variable with mean µ, variance σ 2. This is an amazing theorem. March 17, 2003 Stochastic Programming Lecture 16 Slide 12

13 A More Convenient Form of the CLT S n nµ σ N (0, 1) n ( ) χ µ n N (0, 1) σ n( χ µ) N (0, σ 2 ) March 17, 2003 Stochastic Programming Lecture 16 Slide 13

14 Sampling Methods Interior sampling methods. Sample during the course of the algorithm LShaped Method (Dantzig and Infanger) Stochastic Decomposition (Higle and Sen) Stochastic Quasi-gradient methods (Ermoliev) Exterior sampling methods Sample. Then solve problem approximating problem. Can we get (statistical) bounds on key solution quantities? March 17, 2003 Stochastic Programming Lecture 16 Slide 14

15 Lower Bound on the Optimal Objective Function Value v = min x S {f(x) E P g(x, ξ) For some sample ξ 1,..., ξ N, let Ω g(x; ξ)p(ξ)dξ} Thm: ˆv N = min x S { f N (x) N 1 N g(x, ξ j )}. j=1 Eˆv N v March 17, 2003 Stochastic Programming Lecture 16 Slide 15

16 min N 1 x X E " min x X N 1 Proof v = min x X E P g(x; ξ) = min x X E" N 1 NX j=1 g(x, ξ j ) N 1 N X j=1 NX j=1 g(x, ξ j ) # NX j=1 g(x, ξ j ) E " X N N 1 g(x, ξ j ) j=1 # E [ˆv N ] E " N 1 N X j=1 g(x, ξ j ) min E" N 1 x X # g(x, ξ j ) x X NX j=1 # g(x, ξ j ) x X x X # = v March 17, 2003 Stochastic Programming Lecture 16 Slide 16

17 Next? Now we need to somehow estimate E[ˆv n ] The expected value E[ v N ] can be estimated as follows. Generate M independent samples, ξ 1,j,..., ξ N,j, j = 1,..., M, each of size N, and solve the corresponding SAA problems min x X { f j N (x) := N 1 N } g(x, ξ i,j ) i=1, (1) for each j = 1,..., M. Let v j N be the optimal value of problem (1), and compute M L N,M 1 M j=1 v j N March 17, 2003 Stochastic Programming Lecture 16 Slide 17

18 Lower Bounds The estimate L N,M is an unbiased estimate of E[ v N ]. By our last theorem, it provides a statistical lower bound for the true optimal value v. When the M batches ξ 1,j, ξ 2,j,..., ξ N,j, j = 1,..., M, are i.i.d. (although the elements within each batch do not need to be i.i.d.) have by the Central Limit Theorem that M [LN,M E( v N )] N (0, σ 2 L) March 17, 2003 Stochastic Programming Lecture 16 Slide 18

19 Condence Intervals The sample variance estimator of σ 2 L is s 2 L(M) 1 M 1 M j=1 ( v j N L N,M ) 2. Dening z α to satisfy P {N(0, 1) z α } = 1 α, and replacing σ L by s L (M), we can obtain an approximate (1 α)-condence interval for E[ v N ] to be [ L N,M z αs L (M) M, L N,M + z ] αs L (M) M March 17, 2003 Stochastic Programming Lecture 16 Slide 19

20 Let's do an example Time Permitting minimize Q(x 1, x 2 ) = x 1 + x subject to 4 ω 1 =1 2/3 ω 2 =1/3 y 1 (ω 1, ω 2 ) + y 2 (ω 1, ω 2 )dω 1 dω 2 ω 1 x 1 + x 2 + y 1 (ω 1, ω 2 ) 7 ω 2 x 1 + x 2 + y 2 (ω 1, ω 2 ) 4 x 1 0 x 2 0 y 1 (ω 1, ω 2 ) 0 y 2 (ω 1, ω 2 ) 0 March 17, 2003 Stochastic Programming Lecture 16 Slide 20

21 Upper Bounds v = min x S {f(x) E P g(x; ξ) Quick, Someone prove... Ω g(x; ξ)p(ξ)dξ} f(ˆx) v How can we estimate f(ˆx)? x X March 17, 2003 Stochastic Programming Lecture 16 Slide 21

22 Estimating f(ˆx) Generate T independent batches of samples of size N, denoted by ξ 1,j, ξ 2,j,..., ξ N,j, j = 1, 2,..., T, where each batch has the unbiased property, namely E f j N(x) := N 1 N i=1 F (x, ξ i,j ) = f(x), for all x X. We can then use the average value dened by as an estimate of f(ˆx). U N,T (ˆx) := T 1 T j=1 f j N(ˆx) March 17, 2003 Stochastic Programming Lecture 16 Slide 22

23 More Condence Intervals By applying the Central Limit Theorem again, we have that T [ U N,T (ˆx) f(ˆx) ] N(0, σ 2 U (ˆx)), as T, where σu 2 (ˆx) := Var [ f N (ˆx) ]. We can estimate σu 2 (ˆx) by the sample variance estimator s 2 U (ˆx; T ) dened by s 2 U (ˆx; T ) := 1 T 1 T j=1 [ ] 2 f j N (ˆx) U N,T (ˆx). By replacing σu 2 (ˆx) with s2 U (ˆx; T ), we can proceed as above to obtain a (1 α)-condence interval for f(ˆx): [ U N,T (ˆx) z αs U (ˆx; T ), U N,T (ˆx) + z ] αs U (ˆx; T ). T T March 17, 2003 Stochastic Programming Lecture 16 Slide 23

24 Putting it all together f N (x) is the sample average function Draw ω 1,... ω N from P f N (x) N 1 N j=1 g(x, ωj ) For Stochastic LP w/recourse solve N LP's. v N is the optimal solution value for the sample average function: v N min x S { fn (x) := N 1 N j=1 g(x, ωj ) Estimate E( v N ) as Ê( v N ) = L N,M = M 1 M j=1 vj N Solve M stochastic LP's, each of sampled size N. } March 17, 2003 Stochastic Programming Lecture 16 Slide 24

25 Recapping Theorems Thm. Thm. E( v N ) v f(x) x fn (ˆx) E( v N ) f(ˆx) v, as N, M, N We are mostly interested in estimating the quality of a given solution ˆx. This is f(ˆx) v. f N (ˆx) computed by solving N independent LPs. Ê( v N) computed by solving M independent stochastic LPs. Independent no synchronization good for the Grid Independent can construct condence intervals around the estimates March 17, 2003 Stochastic Programming Lecture 16 Slide 25

26 An experiment M times Solve a stochastic sampled approximation of size N. (Thus obtaining an estimate of E( v N )). For each of the M solutions x 1,... x M, estimate f(ˆx) by solving N LP's. Test Instances Name Application Ω (m 1, n 1 ) (m 2, n 2 ) LandS HydroPower Planning 10 6 (2,4) (7,12) gbd? (?,?) (?,?) storm Cargo Flight Scheduling (185, 121) (?,1291) 20term Vehicle Assignment (1,5) (71,102) ssn Telecom. Network Design (1,89) (175,706) March 17, 2003 Stochastic Programming Lecture 16 Slide 26

27 Convergence of Optimal Solution Value 9 M 12, N = 10 6 Monte Carlo Sampling Instance N = 50 N = 100 N = 500 N = 1000 N = term gbd LandS storm ssn Latin Hypercube Sampling Instance N = 50 N = 100 N = 500 N = 1000 N = term gbd LandS storm ssn March 17, 2003 Stochastic Programming Lecture 16 Slide 27

28 20term Convergence. Monte Carlo Sampling Lower Bound Upper Bound Value N March 17, 2003 Stochastic Programming Lecture 16 Slide 28

29 20term Convergence. Latin Hypercube Sampling Lower Bound Upper Bound Value N March 17, 2003 Stochastic Programming Lecture 16 Slide 29

30 ssn Convergence. Monte Carlo Sampling Lower Bound Upper Bound Value N March 17, 2003 Stochastic Programming Lecture 16 Slide 30

31 ssn Convergence. Latin Hypercube Sampling Lower Bound Upper Bound Value N March 17, 2003 Stochastic Programming Lecture 16 Slide 31

32 storm Convergence. Monte Carlo Sampling 1.555e e+06 Lower Bound Upper Bound 1.553e e e+06 Value 1.55e e e e e e e N March 17, 2003 Stochastic Programming Lecture 16 Slide 32

33 storm Convergence. Latin Hypercube Sampling e e+06 Lower Bound Upper Bound 1.55e e e+06 Value e e e e e e e N March 17, 2003 Stochastic Programming Lecture 16 Slide 33

34 gbd Convergence. Monte Carlo Sampling 1800 Lower Bound Upper Bound Value N March 17, 2003 Stochastic Programming Lecture 16 Slide 34

35 gbd Convergence. Latin Hypercube Sampling Lower Bound Upper Bound Value N March 17, 2003 Stochastic Programming Lecture 16 Slide 35

36 The Gap f(ˆx) f N (ˆx) v Ê v N E v N Of most concern is the bias v E v N. How fast can we make this go down in N? March 17, 2003 Stochastic Programming Lecture 16 Slide 36

37 A Biased Discussion Some problems are ill-conditioned It takes a large sample to get an accurate estimate of the solution Variance reduction can help reduce the bias You get the right small sample March 17, 2003 Stochastic Programming Lecture 16 Slide 37

38 Next Time Go over homeworks Convergence of Optimal Solution Values Stochastic Decomposition March 17, 2003 Stochastic Programming Lecture 16 Slide 38