1 / 44 The L-Shaped Method Operations Research Anthony Papavasiliou
Contents 2 / 44 1 The L-Shaped Method [ 5.1 of BL] 2 Optimality Cuts [ 5.1a of BL] 3 Feasibility Cuts [ 5.1b of BL] 4 Proof of Convergence [ 5.1c of BL] 5 Example: Capacity Expansion Planning
Table of Contents 3 / 44 1 The L-Shaped Method [ 5.1 of BL] 2 Optimality Cuts [ 5.1a of BL] 3 Feasibility Cuts [ 5.1b of BL] 4 Proof of Convergence [ 5.1c of BL] 5 Example: Capacity Expansion Planning
4 / 44 Extensive Form 2-Stage Stochastic Linear Program K (EF ) : min c T x + p k qk T y k s.t. Ax = b k=1 T k x + Wy k = h k, k = 1,..., K x 0, y k 0, k = 1,..., K K realizations of random vector ξ, with probabilities p k, k = 1,..., K Randomness: q k, h k, T k y k : second-stage decision given realization k
Block Structure 5 / 44 Multi-stage decision making process A T 1 W T 2 W. T K W Idea: ignore constraints of future stages
6 / 44 Master Problem We know that K V (x) = {min p k qk T y k Wy k = h k T k x, y k 0} k=1 is a piecewise linear function of x Define master problem as (M) : min z = c T x + θ (1) s.t. Ax = b D l x d l, l = 1,..., r (2) E l x + θ e l, l = 1,..., s (3) x 0, θ R Feasibility cuts: equation 2 Optimality cuts: equation 3
Optimality Cuts 7 / 44 Consider a trial first-stage decision x v Let πk v be simplex multipliers of second-stage problem: min w = qk T y s.t. Wy = h k T k x v y 0 e s+1 E s+1 x supports V (x) at x v, where E s+1 = e s+1 = K p k (πk v )T T k (4) k=1 K p k (πk v )T h k (5) k=1
8 / 44 V (x) e s+1 E s+1 x x v x
9 / 44 The L-Shaped Algorithm Step 0. Set r = s = v = 0 Step 1. Solve master problem (M). Let (x v, θ v ) be an optimal solution. If s = 0 (no optimality cuts), remove θ from (M) and set θ 0 = Step 2. If x / K 2, add feasibility cut (equation 2) and return to Step 1. Otherwise, go to Step 3. K 2 = {x y : Wy = h k T k x, y 0, k = 1,..., K } Step 3. Compute E s+1, e s+1. Let w v = e s+1 E s+1 x v. If θ v w v, stop with x v an optimal solution. Otherwise, set s = s + 1, add optimality cut to equation 3 and return to Step 1.
Table of Contents 10 / 44 1 The L-Shaped Method [ 5.1 of BL] 2 Optimality Cuts [ 5.1a of BL] 3 Feasibility Cuts [ 5.1b of BL] 4 Proof of Convergence [ 5.1c of BL] 5 Example: Capacity Expansion Planning
11 / 44 Example 1 z = min 100x 1 + 150x 2 + E ξ (q 1 y 1 + q 2 y 2 ) s.t. x 1 + x 2 120 6y 1 + 10y 2 60x 1 8y 1 + 5y 2 80x 2 y 1 d 1, y 2 d 2 x 1 40, x 2 20, y 1, y 2 0 ξ = (d 1, d 2, q 1, q 2 ) = { (500, 100, 24, 28), p 1 = 0.4 (300, 300, 28, 32), p 2 = 0.6
Iteration 1 Step 1. min{100x 1 + 150x 2 x 1 + x 2 120, x 1 40, x 2 20} x 1 = (40, 20) T, θ 1 = Step 3. For ξ = ξ 1 solve min{ 24y 1 28y 2 6y 1 + 10y 2 2400, 8y 1 + 5y 2 1600 0 y 1 500, 0 y 2 100} w 1 = 6100, y T = (137.5, 100), π T 1 = (0, 3, 0, 13) For ξ = ξ 2 solve min{ 28y 1 32y 2 6y 1 + 10y 2 2400, 8y 1 + 5y 2 1600 0 y 1 300, 0 y 2 300} w 2 = 8384, y T = (80, 192), π2 T = ( 2.32, 1.76, 0, 0) 12 / 44
Iteration 1: Optimality Cut h 1 = (0, 0, 500, 100) T, h 2 = (0, 0, 300, 300) T T,1 = ( 60, 0, 0, 0) T, T,2 = (0, 80, 0, 0) T e 1 = 0.4 π1 T h 1+0.6 π2 T h 2 = 0.4 ( 1300)+0.6 (0) = 520 E 1 = 0.4 π1 T T + 0.6 πt 2 T = 0.4(0, 240) + 0.6(139.2, 140.8) = (83.52, 180.48) w 1 = 520 (83.52, 180.48) x 1 = 7470.4 w 1 = 7470.4 > θ 1 =, therefore add the cut 83.52x 1 + 180.48x 2 + θ 520 13 / 44
Iteration 2 14 / 44 Step 1. Solve master min{100x 1 + 150x 2 + θ x 1 + x 2 120, x 1 40, x 2 20, 83.52x 1 + 180.48x 2 + θ 520} z = 2299.2, x 2 = (40, 80) T, θ 2 = 18299.2 Step 3. Add the cut 211.2x 1 + θ 1584
Iteration 3 15 / 44 Step 1. Solve master. z = 1039.375, x 3 = (66.828, 53.172) T, θ 3 = 15697.994 Step 3. Add the cut 115.2x 1 + 96x 2 + θ 2104
16 / 44 Iteration 4 Step 1. Solve master. z = 889.5, x 4 = (40, 33.75) T, θ 4 = 9952 Step 3. There are multiple solutions for ξ = ξ 2. Select one, add the cut 133.44x 1 + 130.56x 2 + θ 0
17 / 44 Iteration 5 Step 1. Solve master min{100x 1 + 150x 2 + θ x 1 + x 2 120, x 1 40, x 2 20, 83.52x 1 + 180.48x 2 + θ 520, 211.2x 1 + θ 1584 115.2x 1 + 96x 2 + θ 2104, 133.44x 1 + 130.56x 2 + θ 0} z = 855.833, x 5 = (46.667, 36.25) T, θ 5 = 10960 Step 3. w 5 = 520 (83.52, 180.48) x 5 = 10960 = θ 5, stop. x = (46.667, 36.25) T is the optimal solution.
18 / 44 Example 2 z = min E ξ (y 1 + y 2 ) s.t. 0 x 10 y 1 y 2 = ξ x y 1, y 2 0 1 p 1 = 1/3 ξ = 2 p 2 = 1/3 4 p 3 = 1/3 K 2 = R
L-Shaped Method in Example 2 19 / 44 Iteration 1, Step 1: x 1 = 0 Iteration 1, Step 3: x 1 not optimal, add cut: θ 7/3 x Iteration 2, Step 1: x 2 = 10 Iteration 2, Step 3: x 2 not optimal, add cut: θ x 7/3 Iteration 3, Step 1: x 3 = 7/3 Iteration 3, Step 3: x 3 not optimal, add cut: θ (x + 1)/3 Iteration 4, Step 1: x 4 = 1.5 Iteration 4, Step 3: x 4 not optimal, add cut: θ (5 x)/3 Iteration 3, Step 1: x 5 = 2 Iteration 3, Step 3: x 5 is optimal
20 / 44 V (x) 1 x 1 1 x 3 4 V (x 1 ) = 7/3 and V (x) = 7/3 x around x 1 (7 x)/3 is the optimality cut at x 1
Geometric Interpretation of Optimality Cuts Q(x, ξ) = min y {q(ω) T y W (ω)y = h(ω) T (ω)x, y 0} yields dual optimal multipliers π T = q B (ω) T B(ω) 1 From linear programming duality Q(x, ξ) = q B (ω) T B(ω) 1 (h(ω) T (ω)x) B(ω) is optimal basis as long as y B = B(ω) 1 (h(ω) T (ω)x) 0, y N = 0 and q B (ω) T B(ω) 1 W q(ω) T Taking expectation, V (x) = E ξ {q B (ω) T B(ω) 1 (h(ω) T (ω)x)} in the set ξ Ξ {x B(ω) 1 (h(ω) T (ω)x) 0} This is exactly the expression for optimality cuts at x v 21 / 44
Table of Contents 22 / 44 1 The L-Shaped Method [ 5.1 of BL] 2 Optimality Cuts [ 5.1a of BL] 3 Feasibility Cuts [ 5.1b of BL] 4 Proof of Convergence [ 5.1c of BL] 5 Example: Capacity Expansion Planning
Feasibility Cuts 23 / 44 Consider the following problem: (F) : min w = e T v + + e T v s.t. Wy + Iv + Iv = h k T k x v y 0, v + 0, v 0 with dual multipliers σ v. Define D r+1 = (σ v ) T T k d r+1 = (σ v ) T h k Step 2 of L-shaped method: For k = 1,..., K solve (F). If w = 0 for all k, go to Step 3. Else, add D r+1 x d r+1, set r = r + 1 and go to Step 1.
24 / 44 Example min 3x 1 + 2x 2 E ξ (15y 1 + 12y 2 ) s.t. 3y 1 + 2y 2 x 1, 2y 1 + 5y 2 x 2 0.8ξ 1 y 1 ξ 1, 0.8ξ 2 y 2 ξ 2 x, y 0 ξ = (4, 4), p 1 = 0.25 (4, 8), p 2 = 0.25 (6, 4), p 3 = 0.25 (6, 8), p 4 = 0.25
Generating a Feasibility Cut For x 1 = (0, 0) T, ξ = (6, 8) T, solve min v + v +,v 1 + v 1 + v + 2 + v 2 + v + 3 + v 3 +,y v + 4 + v 4 + v + 5 + v 5 + v + 6 + v 6 s.t. v + 1 v 1 + 3y 1 + 2y 2 0, v + 2 v 2 + 2y 1 + 5y 2 0 v + 3 v 3 + y 1 4.8, v + 4 v 4 + y 2 6.4 v + 5 v 5 + y 1 6, v + 6 v 6 + y 2 8 We get w = 11.2, σ 1 = ( 3/11, 1/11, 1, 1, 0, 0) h = (0, 0, 4.8, 6.4, 6, 8) T, T,1 = ( 1, 0, 0, 0, 0, 0) T, T,2 = (0, 1, 0, 0, 0, 0) T D 1 = ( 3/11, 1/11, 1, 1, 0, 0) T = (3/11, 1/11), d 1 = ( 3/11, 1/11, 1, 1, 0, 0) h = 11.2 3/11x 1 + 1/11x 2 11.2 25 / 44
26 / 44 Induced Constraints Going by the book: Iteration 2 master problem: x 2 = (41.067, 0) T Iteration 2 feasibility cut: x 2 22.4 Iteration 3 master problem: x 3 = (33.6, 22.4) T Iteration 3 feasibility cut: x 2 41.6 Iteration 4 master problem: x 4 = (27.2, 41.6) T is feasible Induced Constraints: Observe that for ξ = (6, 8) T, y 1 4.8, y 2 6.4 This implies x 1 27.2, x 2 41.6, which should be added directly to the master Personal experience: feasibility cuts are impractical
27 / 44 Complete Recourse and Relative Complete Recourse pos W = {z z = Wy, y 0} K 1 = {x Ax = b}, K 2 = {x y : Wy = h k T k x, k = 1,..., K } Feasibility cuts are not necessary when we have: Complete recourse: pos W = R m 2 Relative complete recourse: K 2 K 1
Table of Contents 28 / 44 1 The L-Shaped Method [ 5.1 of BL] 2 Optimality Cuts [ 5.1a of BL] 3 Feasibility Cuts [ 5.1b of BL] 4 Proof of Convergence [ 5.1c of BL] 5 Example: Capacity Expansion Planning
29 / 44 Proof of Convergence Key observation: when we solve the master problem (M) subject to x K 1 K 2 either θ v V (x v ) or θ v < V (x v ) The first case is our termination criterion and implies optimality. The second case can occur finitely many times. K 1 = {x Ax = b, x 0}, K 2 = {x y : Wy = h k T k x, y 0, k = 1,..., K }
30 / 44 θ v V (x v ) Is Equivalent to the Termination Criterion Q(x, ξ(ω)) = min y {q(ω) T y Wy = h(ω) T (ω)x, y 0} From linear programming duality, Q(x v, ξ k ) = (π v k )T (h k T k x v ) Taking expectation, V (x) = E ξ Q(x v, ξ) = K k=1 p k (π v k )T (h k T k x v ) Our termination criterion is θ v w v, where K K w v = e s+1 E s+1 x v = ( p k (πk v )T h k ) ( p k (πk v )T T k )x v k=1 k=1
31 / 44 θ v V (x v ) Implies Optimality Fact: Q(x, ξ) is convex in x, and π v k is a subgradient at x v : Q(x, ξ k ) (π v k )T h k (π v k )T T k x Taking expectation: V (x) K k=1 p k (π v k )T (h k T k )x Full problem can be written as min c T x + θ s.t. V (x) θ x K 1 K 2 Therefore master (M) is a relaxation and gives lower bound If there were x K 1 K 2 with c T x + V ( x) < c T x v + V (x v ) then x v would not be optimal for (M) since (M) would be at most c T x + V ( x)
θ v < V (x v ) Occurs Finitely Many Times 32 / 44 We have seen that each optimality cut is equivalent to θ V (x l ) + V (x l )(x x l ), l = 1,..., s Every time θ v < V (x v ) occurs, the set of new multipliers (πk v ), k = 1,..., K must be different from those generated previously Optimal multipliers πk v correspond to optimal bases of min y {qk T y Wy = h k T k x, y 0} There are finitely many bases, therefore finitely many combinations of optimal multipliers (πk v ), k = 1,..., K
33 / 44 Finite Termination of Feasibility Cuts Key observations: 1 Only a finite number of feasibility cuts can be generated 2 The feasibility cuts of equation 2 are valid Valid feasibility cuts: remove current candidate x v from consideration do not remove any other candidates x K 2 from consideration
Finite Number of Possible Cuts 34 / 44 Recall feasibility cuts: D l x d l, l = 1,..., r + 1, where D r+1 = (σ v ) T T k d r+1 = (σ v ) T h k and σ v the dual multipliers of (F) : min w = e T v + + e T v s.t. Wy + Iv + Iv = h k T k x v y 0, v + 0, v 0 σ v corresponds to a basis B of W : σ v = c T B B 1 W has a finite number of bases
35 / 44 Valid Feasibility Cuts: x v Is Cut Off If w > 0 then by linear programming duality (σ v ) T (h k T k x v ) > 0, therefore D r+1 x d r+1 removes x v from consideration
Valid Feasibility Cuts: No Other x K 2 Is Cut Off Consider the following subset of R m 2: {z z = h k T k x, x K 2 } This set is contained in pos W = {z z = Wy, y 0} by definition of K 2 The half-space defined by (σ v ) T z 0 contains pos W To see this, note that the reduced cost of y must be 0: c T N c T B B 1 N = (σ v ) T W 0 where c T N = 0, (σv ) T = c T B B 1, N = W Any element of pos W can be expressed as z = Wy, y 0, hence from the previous inequality (σ v ) T z 0 Therefore, (σ v ) T (h Tx) 0 will be a valid feasibility cut 36 / 44
Feasibility Cuts 37 / 44 h k T k x v R m 2 σ r+1 pos W
Table of Contents 38 / 44 1 The L-Shaped Method [ 5.1 of BL] 2 Optimality Cuts [ 5.1a of BL] 3 Feasibility Cuts [ 5.1b of BL] 4 Proof of Convergence [ 5.1c of BL] 5 Example: Capacity Expansion Planning
Example: Capacity Expansion Planning min x,y 0 s.t. n m (I i x i + E ξ C i T j y ij (ω)) i=1 j=1 n y ij (ω) = D j (ω), j = 1,..., m i=1 m y ij (ω) x i, i = 1,... n j=1 I i, C i : fixed/variable cost of technology i D j (ω), T j : height/width of load block j y ij (ω): capacity of i allocated to j x i : capacity of i Note: D j is not dependent on ω 39 / 44
Problem Data Two possible realizations of load duration curve: Reference scenario: 10% 10x wind scenario: 90% Duration (hours) Level (MW) Level (MW) Reference scenario 10x wind scenario Base load 8760 0-7086 0-3919 Medium load 7000 7086-9004 3919-7329 Peak load 1500 9004-11169 7329-10315 40 / 44
Slave Problem 41 / 44 (S ω ) : min y 0 (λ j (ω)) : (ρ i (ω)) : n i=1 j=1 m C i T j y ij n y ij = D j (ω), j = 1,..., m i=1 m y ij x i, i = 1,... n j=1 where x has been fixed from the master problem
Sequence of Investment Decisions Iteration Coal (MW) Gas (MW) Nuclear (MW) Oil (MW) 1 0 0 0 0 2 0 0 0 8736 3 0 0 0 15999.6 4 0 14675.5 0 0 5 10673.8 0 0 0 6 10673.8 0 0 13331.8 7 0 163.8 7174.5 3830.8 8 0 3300.6 7868.4 0 9 0 5143.4 7303.9 1679.4 10 3123.9 1948.1 4953.7 1143.3 11 1680 4322.4 6625 0 12 8747.6 1652.8 0 768.6 13 5701.9 464.9 4233.6 768.6 14 4935.9 1405 3994.7 0 15 6552.6 386.3 3173.7 882.9 16 5085 1311 3919 854 42 / 44
43 / 44 Observations Investment candidate in each iteration necessarily different from all past iterations Greedy behavior: low capital cost in early iterations