INEXACT CUTS IN BENDERS' DECOMPOSITION GOLBON ZAKERI, ANDREW B. PHILPOTT AND DAVID M. RYAN y Abstract. Benders' decomposition is a well-known techniqu

Transcription

1 INEXACT CUTS IN BENDERS' DECOMPOSITION GOLBON ZAKERI, ANDREW B. PHILPOTT AND DAVID M. RYAN y Abstract. Benders' decomposition is a well-known technique for solving large linear programs with a special structure. In particular it is a popular technique for solving multi-stage stochastic linear programming problems. Early termination in the subproblems generated during Benders' decomposition (assuming dual feasibility) produces valid cuts which are inexact in the sense that they are not as constraining as cuts derived from an exact solution. We describe an inexact cut algorithm, prove its convergence under easily veriable assumptions, and discuss a corresponding Dantzig-Wolfe decomposition algorithm. The paper is concluded with some computational results from applying the algorithm to a class of stochastic programming problems which arise in hydroelectric scheduling. Key words. stochastic programming, Benders' decomposition, inexact cuts AMS subject classications. 90C15, 90C05, 90C06, 90C90 1. Introduction. Many large linear programming problems have a block diagonal structure which makes them amenable to decomposition techniques such as Dantzig-Wolfe decomposition ([4], [5]), or its dual, Benders' decomposition [2]. The latter technique has become increasingly popular in stochastic linear programming starting with the independent publication of the L-Shaped method by VanSlyke and Wets [15] for two-stage stochastic linear programming. (The L-shaped method is often referred to as stochastic Benders' decomposition). In this paper we shall be concerned with Benders' decomposition applied to linear programs of the form: P: minimize c T x + q T y subject to Ax = b; T x + W y = h; If we dene x 0; y 0: then P can be written: P: minimize c T x + Q(x) subject to Ax = b; Q(x) = minfq T y j W y = h? T x; y 0g; x 0: Using this notation we can now write down the following algorithm. This research has been supported by the New Zealand Public Good Science Fund, FRST contract 403. y Operations Research Group, Department of Engineering Science, University of Auckland, Private Bag 92019, Auckland, New Zealand (g.zakeri@auckland.ac.nz). 1

2 2 ZAKERI, PHILPOTT, RYAN Benders' Decomposition Algorithm Set i := 0, U 0 := 1, L 0 :=?1, F := IR n fl 0 g. While U i? L i > 0 1. Set i := i Solve the master problem MP: minimize c T x + subject to Ax = b; (x; ) 2 F; x 0; to give optimal primal variables (x i ; i ). 3. Set L i := c > x i + i. 4. Solve the subproblem SP(x i ): minimize q T y subject to W y = h? T x i ; y 0; to give optimal primal variables y i and dual variables i. 5. Set U i := minfu i?1 ; c > x i + q > y i g. 6. F := F \ f(x; ) j i > (h? T x) g. In the classical case the cut dened by Step 6 comes from an optimal basic feasible solution to the subproblem. Since there are a nite number of basis matrices for this problem, nite termination of the algorithm at the optimal solution can be guaranteed (see e.g. [11]). In this paper we explore the Benders' decomposition algorithm in the case where the cuts are not computed from an optimal extreme-point solution to a linear programming subproblem. For example, when the subproblems are very large, it makes sense to determine the cuts by applying a primal-dual interior-point method to the subproblem. Terminating this procedure when it yields a feasible dual solution will still dene a valid cut. We call this an inexact cut. If the dual solution is close to optimal then an inexact cut will also separate the optimal solution from the current iterate (except when this is optimal). As observed by a number of authors (see e.g. [1]), inexact cuts may be less eort to compute than the exact cuts, especially for linear programming algorithms which yield an approximately optimal dual feasible solution before termination. In theoretical terms, Benders' decomposition is a special case of a more general class of convex cutting plane algorithms rst introduced by Kelley [12]. Cutting plane algorithms construct a sequence of hyperplanes which separate the current iterate from the optimal solution. In the case where the cutting planes are computed inexactly, the asymptotic convergence of this process to the optimal solution has been investigated by a number of authors [6], [7], [9], [12]. In the context of Benders' decomposition applied to linear programs of the form P, all of the convergence results in these papers assume that the sets containing x and > T are both bounded. In the convergence theorem we prove for inexact cuts we require that X = fx 0jAx = bg is bounded

3 INEXACT CUTS IN BENDERS' DECOMPOSITION 3 and that X domq. The latter assumption, which is known as relatively complete recourse in stochastic programming, is weaker than requiring that > T is bounded. In the next section we describe a Benders' decomposition algorithm which terminates the solution of the subproblem before optimality to produce an inexact cut. The steps of the algorithm ensure that this cut separates the optimal solution from the current iterate. In section 3 we consider the convergence of the inexact cut algorithm under the above assumptions, and in section 4 we discuss the implications of our results for Dantzig-Wolfe decomposition. In section 5 we give some computational results. 2. The Algorithm. We start the algorithm by choosing a convergence tolerance, setting an iteration counter i := 0, and choosing some decreasing sequence f i g which converges to 0. We also set U 0 := 1, and L 0 :=?1. The remaining steps of the algorithm are as follows. Inexact Cut Algorithm While U i? L i > 1. Set i := i Solve MP to obtain (x i ; i ). 3. Set L i := c > x i + i. 4. Perform an inexact optimization to generate a vector i feasible for the dual of SP(x i ) such that (1) > i (h? T x i ) + i > Q(x i ): 5. Set U i := minfu i?1 ; c > x i + i > (h? T x i) + i g. 6. If i > (h? T x i) > i then add the cut i > (h? T x) to MP else set i := i + 1, x i+1 := x i, i+1 := i, L i+1 := L i, U i+1 := U i and go to Step 4 1. We denote by v i the value of the inexact optimization in step 4. Thus v i = > i (h?t x i). In step 6 of each iteration of this method we check to see if v i > i, which ensures that the hyperplane > i (h? T x) = will strictly separate the current iterate (x i; i ) from any optimal solution of P. If this check fails then we decrease the duality gap tolerance and continue with the solution of SP(x i ), until either i! 0 with no change in (x i ; i ), or (x i ; i ) is separated from an optimal solution of P by a cut. In order to show that this algorithm converges we make use of the following simple results. Lemma 2.1.? > i T is an i-subgradient of Q at x i. Proof. Since i is dual feasible for SP(x), for every x and by (1) Thus Q(x) > i (h? T x); Q(x i ) v i + i ; Q(x)? Q(x i ) > i (h? T x)? > i (h? T x i )? i ; 1 Note that in this case x and remain xed and only (possibly) changes

4 4 ZAKERI, PHILPOTT, RYAN giving Q(x) Q(x i )? i > T (x? x i)? i ; which gives the result. Lemma 2.2. Let U i, L i, x i and i be generated by applying the inexact cut algorithm with i. Then 0 U i? L i v i + i? i : Proof. Since U i is an upper bound on the value of P and L i a lower bound U i? L i 0. Moreover, since U i c > x i + v i + i ; and L i = c > x i + i, we have 0 U i? L i c > x i + v i + i? c > x i? i : Hence 0 U i? L i v i + i? i : 3. Convergence of the Algorithm. In this section we prove that the sequence f(x i ; i )g generated by the inexact cut algorithm converges to an optimal solution to P. As alluded to above, abstract proofs of convergence for cutting plane methods (see [12] ) typically invoke a compactness argument, which in our context relies on an assumption that the sets containing x and > T are both bounded. Since this might not always be the case for P, it is instructive to prove a convergence result directly to see to what extent these boundedness assumptions might be relaxed. We begin by showing that the sequence fi T T g generated by the inexact cut algorithm is bounded provided that the set X = fx 0jAx = bg is bounded, and domq is IR n. (In stochastic programming the latter is known as complete recourse.) We make use of the following technical result. Lemma 3.1. If for some pair (b; ) the epigraph of f(x) = max 1kN fb> k x + kg lies in the half space H = f(x; )j b > x + g then jjbjj max 1kN jjb k jj. Proof. Suppose jjbjj > max 1kN jjb k jj = M and let ~ = max 1kN j k j. Let j?j n > ~ jjbjj 2?Mjjbjj and dene z := nb. We will show that f(z) < b> z + contradicting the hypothesis. Formally: b > z + = njjbjj 2 + > njjbjjm + + j ~? j nmjjbjj + ~ max 1kN [(nb> )b k ] + ~ = max 1kN [(nb> )b k ] + max 1kN j kj max 1kN [(nb> )b k + k ] = f(z):

5 INEXACT CUTS IN BENDERS' DECOMPOSITION 5 This contradicts the assumption that the epigraph of max 1kN fb > k x + kg lies in H which implies that jjbjj M thereby completing the proof. Lemma 3.2. If domq = IR n then the sequence f? i > T g is bounded. Proof. Let f^ k j 1 k Ng be the set of basic feasible solutions of W > q. Since i is dual feasible for SP(x) for every x we have > i (h? T x) max 1kN ^> k (h? T x) = Q(x); where the equation follows by virtue of domq = IR n. Therefore the epigraph of Q lies in the half space: H = f(x; )j b > x + g where b =? i > T and = > i h. The conclusion is then immediate by applying Lemma 3.1. Next we will show that the inexact cut algorithm terminates in a nite number of iterations with a?optimal solution. If the inexact cut algorithm does not terminate in a nite number of iterations, then it will produce an innite sequence f(x i ; i )g, which satises one of the following conditions: 1. There exists m such that for all i m, i v i : 2. There exists a subsequence f(x (i) ; (i) )g such that (i) < v (i). Lemma 3.3. If there exists m such that for all i m; i v i then U i? L i # 0. Proof. Since i v i, Lemma 2.2 implies 0 U i? L i v i + i? i i : The result follows since i! 0. Lemma 3.4. If there exists a convergent subsequence f(x (i) ; (i) )g such that (i) < v (i) then 1. 0 < v (i)? (i) v (i)? v (i?1) + > T (x (i?1) (i)? x (i?1) ); 2. lim v (i)? v (i?1) = 0; 3. lim inf > T (x (i?1) (i)? x (i?1) ) 0: Proof. It is clear that 0 < v (i)? (i) from the assumption. To obtain the second inequality, observe that (x (i) ; (i) ) is constrained to satisfy the cut we added at iteration (i? 1). Therefore which implies (i) > (i?1) (h? T x (i)); (2) v (i)? (i) v (i)? > (i?1) (h? T x (i)) = v (i)? v (i?1) + > (i?1) T (x (i)? x (i?1) ): Now (x (i) ; (i) )! (x ; ) by assumption. Furthermore, from the algorithm, we have therefore Q(x (i) )? (i) v (i) Q(x (i) ); (3) v (i)! Q(x );

6 6 ZAKERI, PHILPOTT, RYAN which implies lim v (i)? v (i?1) = 0: Furthermore, (2) and (3) imply lim inf > (i?1) T (x (i)? x (i?1) ) 0: Lemma 3.5. Suppose X = fx 0jAx = bg is bounded and domq is IR n. If there exists a subsequence f(x (i) ; (i) )g such that (i) < v (i) then U i? L i # 0. Proof. The subsequence f(x (i) ; (i) )g is bounded since X is bounded. Thus we may assume, by extracting a further subsequence if necessary, that f(x (i) ; (i) )g is convergent to f(x ; )g, say. We proceed to show that U (i)?l (i) converges to zero, which implies the result. By Lemma 2.2 we have that so if we let then by Lemma U (i)? L (i) v (i) + (i)? (i) ; V (i) = v (i) + (i)? (i) ; (4) 0 < V (i) v (i)? v (i?1) + > (i?1) T (x (i)? x (i?1) ) + (i) ; and (5) lim v (i)? v (i?1) = 0: Furthermore, since domq = IR n ; by Lemma 3.2, > T is bounded, and so (i?1) > (i?1) T (x (i)? x (i?1) )! 0: Substituting into (4) and taking the limit as (i)! 1 yields V (i)! 0. Since V (i) is an upper bound on U (i)?l (i) and this is bounded below by 0 then it converges to 0. Now by their denitions, fu i g is decreasing and fl i g is increasing. Hence fu i? L i g is decreasing and since a subsequence of this sequence converges, it follows that the whole sequence converges, which completes the proof. Theorem 3.6. If fx 0 : Ax = bg is bounded and domq = IR n then the inexact cut algorithm terminates in a nite number of iterations with a?optimal solution of P. Proof. From Lemma 3.3 and Lemma 3.5 we have that U i? L i # 0. Therefore there exists some I such that U I? L I <, so the algorithm terminates in at most I iterations. Let x k be such that U I = c > x k + v k + k. Then c > x k + Q(x k ) c > x k + v k + k < L I + ; and so c > x k + Q(x k ) is within of the optimum. We shall now consider relaxing the assumption that domq = IR n to X domq. (We retain the assumption that X is bounded.) In this case we are no longer guaranteed that f? > i T g is a bounded sequence, since fx ig could lie on the boundary of the domain of Q. At such points it is possible to have unbounded i -subgradients. Since Lemma 3.4 remains valid without our assumption, in what follows we conne attention to the term > (i?1) T (x (i)? x (i?1) )

7 INEXACT CUTS IN BENDERS' DECOMPOSITION 7 and demonstrate that for some subsequence fx (i) g of fx (i) g (6) lim i!1?> (i?1) T (x (i)? x (i?1) ) = 0: We do this by showing in Lemma 3.9 that for some subsequence fx (i) g of fx (i) g (7) lim inf? > (i?1) T (x (i)? x (i?1) ) 0: Since by virtue of Lemma 3.4 we get lim inf > (i?1) T (x (i)? x (i?1) ) 0; lim sup? > (i?1) T (x (i)? x (i?1) ) 0; which with (7) yields (6). The proof of Lemma 3.9 relies on the fact that some subsequence of fx i g must lie in a bounded polyhedral set. The inequality (7) is then proven by appealing to the following two lemmas for polyhedral sets. Lemma 3.7. Let A = fxj b > j x j; 1 j mg; and suppose b > j x = j ; 1 j m: Then x + y 2 A implies that y is in the recession cone of A. Proof. Since x + y 2 A it follows that for every j = 1; 2; : : : ; m, b > j y 0, and so for any x 2 A, 0 and any j, (8) b > j (x + y) = b > j x + b> j y j + b > j y j ; which shows that y is in the recession cone of A. Lemma 3.8. Suppose fx i g is a sequence of points in G = fxj b > j x j; 1 j mg; converging to x, with b > j x = j if 1 j k b > j x < j otherwise. Then there is some > 0 and N such that for every y in the recession cone of fxj b > j x j; 1 j kg Proof. Let i > N ) x i + y kyk 2 G: A = fxj b > j x j; 1 j kg; and dene C to be the recession cone of A. Since G = A \ fxj b > j x j; k < j mg

8 8 ZAKERI, PHILPOTT, RYAN every member of fx i g lies in A, and satises (9) x i + y 2 A; 0; y 2 C: Now choose = 1 2 min k<jm inffkz? x k : b > j z jg > 0 if this is nite, (or set = 1, otherwise). If N is chosen suciently large so that then for every y 2 C (10) i > N =) kx i? x k < ; x i + y kyk? x kx i? x k + < 2: If b > j z j for any j = k +1; : : : ; m, then kz? x k 2, and so setting z = x i + y kyk it follows from (10) that Furthermore, by (9) b > j (x i + y kyk ) j; k < j m: b > j (x i + y kyk ) j; 1 j k; and so x i + y kyk 2 G. We now apply the above lemmas to prove Lemma 3.9. The proof proceeds by showing that for an appropriately chosen convergent subsequence fx (i) g, the projection of > (i) T in the direction of x (i+1)? x (i) is uniformly bounded. Once this is established the conclusion of Lemma 3.9 is immediate. Lemma 3.9. Suppose f(x (i) ; (i) )g is a subsequence of the sequence of solutions generated by the inexact cut algorithm and let f (i) g be the corresponding approximately optimal solutions to the dual of SP(x (i) ). Then there exists a subsequence of fx (i) g, (indexed by (i)) such that x (i)! x and lim inf? > (i) T (x (i+1)? x (i) ) 0: Proof. Since X is bounded, convex, and polyhedral, the (nite) collection of all relative interiors of the faces of X partition it ([14, Theorem 18.2]). Hence there is a subsequence of fx (i) g; indexed by (i); such that fx (i) g lies in the relative interior of a face G of X, and converges to a point x 2 G. Since G is polyhedral we may represent it by G = fxjb > i x i; 1 i mg: If x is in the interior of G then dene C to be IR n. In this case there is clearly some > 0 such that for every y 2 C, and i suciently large, x (i) + y kyk 2 G. Otherwise, without loss of generality dene k to be such that b > i x = i ; 1 i k; b > i x < i ; k < i m;

9 INEXACT CUTS IN BENDERS' DECOMPOSITION 9 and dene C to be the recession cone of fxj b > i x i; 1 i kg: By Lemma 3.8 there is some > 0 such that for every y 2 C, and i suciently large (11) x (i) + y kyk 2 G: Since we are concerned here with the limiting behaviour of fx (i) g we shall henceforth assume that (11) holds for all members of fx (i) g. We now show that we can choose a subsequence fx (i) g of fx (i) g such that x (i?1)? x (i) 2 C. When C = IR n this is trivial. Otherwise we construct the subsequence by choosing x (k) given x (k?1) in the following manner. Since x (k?1) 2 ri (G) there exists > 0 such that (x (k?1) + B) \ a (G) G; where B is the open unit ball and a (G) is the ane hull of G. Now for (i) large enough we have that x? x (i) 2 B, and so if we choose (k) = (i) then x + (x (k?1)? x (k) ) = x (k?1) + x? x (i) 2 G; since x (k?1) + x? x (i) is also in a (G). Therefore x + (x (k?1)? x (k) ) 2 fxj b > i x i; 1 i kg; and by Lemma 3.7 we deduce that (12) x (k?1)? x (k) 2 C: Since x (i) 2 ri (G) this construction may be repeated to yield an innite sequence. Applying Lemma 2.1 to members of fx (i) g; we have for any x that: If we choose Q(x) Q(x (i?1) )? > (i?1) T (x? x (i?1))? (i?1) : x = x (i?1) + x (i?1)? x (i) x(i?1)? x (i) then by Lemma 3.8, (12) and (11) yield x 2 G and give? > (i?1) T x (i?1)? x (i) x(i?1)? x (i) Q(x)? Q(x (i?1) ) + (i?1) sup Q(x)? inf Q(x) + (i?1) x2g x2g If we set M = sup x2g Q(x)? inf x2g Q(x) + 1 then since f i g is decreasing we obtain? > (i?1) T x (i?1)? x (i) x(i?1)? x (i) M : Therefore? > (i?1) T (x (i)? x (i?1) )? M x(i?1)? x (i)

10 10 ZAKERI, PHILPOTT, RYAN which implies lim inf? > (i?1) T (x (i)? x (i?1) ) 0: Theorem If X = fx 0 : Ax = bg is bounded and X domq then the inexact cut algorithm terminates in a nite number of iterations with a -optimal solution of P. Proof. The proof is similar to Theorem 3.6. We will start by showing U i?l i # 0. If there exists m such that for all i m; i v i then Lemma 3.3 delivers the conclusion. Otherwise, there exists a subsequence f(x (i) ; (i) )g such that (i) < v (i) and since X is bounded, without loss of generality we may assume that f(x (i) ; (i) )g converges, to (x ; ), say. Then by Lemma 3.4 Thus lim inf > (i?1) T (x (i)? x (i?1) ) 0: (13) lim sup? > (i?1) T (x (i)? x (i?1) ) 0: Now we can apply Lemma 3.9 to extract a subsequence fx (i) g of fx (i) g such that (14) lim inf? > (i) T (x (i+1)? x (i) ) 0: From (13) and (14) we have? > (i) T (x (i+1)? x (i) )! 0: This yields U (i)? L (i)! 0 implying that the decreasing sequence fu i? L i g tends to 0 which then gives the result as in the proof of Theorem Dantzig-Wolfe Decomposition. It is well known that Benders' decomposition is dual to Dantzig-Wolfe decomposition. Therefore some form of inexact optimization procedure should apply to the latter algorithm, which mirrors the steps of the inexact cut algorithm described in section 2. In fact such a scheme has already been outlined in the literature by Kim and Nazareth [13] who discuss the computational advantages of using interior-point methods in such an approach. We digress briey in this section to explore the asymptotic convergence properties of such an algorithm. The dual problem of P can be formulated as D: maximize b > u + h > v subject to A > u + T > v c; W > v q: Suppose for the moment that the set fv j W > v qg is bounded with extreme points fv i j i = 1; 2; : : : ; Ng. Then Dantzig-Wolfe decomposition solves a restricted master problem MD: maximize b > u + P i ih > v i subject to A > u + P i it > v i c;

11 INEXACT CUTS IN BENDERS' DECOMPOSITION 11 P i i = 1; 0: The columns given by v i are generated iteratively by solving MD, obtaining optimal dual variables (x; ), and then solving the subproblem: SD(x): maximize (h >? x > T > )v subject to W > v q; T > vi to give a new column to be added to the restricted master problem, in the 1 event that this column has a positive reduced cost dened by (h >? x > T > )v i? : In our inexact Dantzig-Wolfe decomposition algorithm we rst choose a convergence tolerance, set an iteration counter i := 0, and choose some decreasing sequence f i g which converges to 0. We do not require that V = fv j W > v qg is bounded but following [13] we require an initial set of points fv 1 ; v 2 ; : : : ; v N g V so that MD has a feasible solution. The remaining steps of the algorithm are as follows. Inexact Dantzig-Wolfe Decomposition Algorithm While U i? L i > 1. Set i := i Solve MD to obtain (u i ; ) and dual variables x i and i. 3. Set L i := b > u i + P i ih > v i. 4. Perform an inexact optimization to generate a vector v i feasible for SD(x i ) such that (15) v > i (h? T x i ) + i > V (SD(x i )): 5. Set U i := minfu i?1 ; c > x i + v i > (h? T x i ) + i g. 6. If v i > T > vi (h? T x i ) > i then add the column to MD. 1 else set i := i + 1, x i+1 := x i, i+1 := i, L i+1 := L i, U i+1 := U i and go to Step 4. Here V (SD(x i )) is the optimal value of SD(x i ). Since the dual of SD(x i ) is easily seen to be SP(x i ), V (SD(x i )) = Q(x i ); and so Step 4 of this algorithm is identical to the same step of the inexact cut algorithm of section 2. In classical Dantzig-Wolfe decomposition each solution v i obtained for SD is an extreme point, of which there are a nite number, thus guaranteeing nite termination. In the inexact algorithm, this is no longer true. However, the theorem of the previous section may be invoked to yield the following corollary. Corollary 4.1. If X = fx 0 : Ax = bg is a bounded set and for every x 2 X; the problem SD(x) is bounded, then the inexact Dantzig-Wolfe algorithm terminates in a nite number of iterations with a -optimal solution of D. Since SD(x) will always have a feasible solution (if D does) the boundedness condition on SD(x) is equivalent to SP(x) being feasible, which is the relatively complete recourse assumption of the previous section. Although it seems natural in the context of Benders' decomposition, the boundedness condition on X is rather restrictive in the current context, and fails to hold in the case when A and b are both identically zero,

12 12 ZAKERI, PHILPOTT, RYAN a situation which is typical in most applications of Dantzig-Wolfe decomposition. We require X to be bounded to enable the extraction of convergent subsequences. When A and b are both identically zero, this can be guaranteed by imposing a condition that the restricted master problems solved in the course of the algorithm produce a sequence of optimal dual variables which lies in some compact set. This might prove to be dicult to verify a priori, but its counterpart in Benders' decomposition seems a natural assumption that can be imposed if necessary by placing a priori bounds on the components of x. 5. Computational Results. We conclude this paper by presenting some computational results of applying the inexact cut algorithm to a set of problems which arise in the planning of hydro-electric power generation. The problems are all based on a multi-stage stochastic programming model developed by Broad [3], in which the New Zealand electricity system is represented as a side-constrained network model with nodes representing hydro-electric reservoirs, hydroelectric generation facilities, thermal generation facilities, and demand points, and arcs with constant losses representing the transmission network. The model consists of 6 reservoirs, 6 thermal stations and 22 hydro stations. Each stage is a week long, and demand in each week is represented by a piecewise linear load duration curve with three linear sections. At each stage a number of random outcomes are possible for the inows into the reservoirs in the current week. We impose a lower bound on the nal level of the reservoirs at the end of the nal stage. This lower bound is a xed fraction of the original initial level of the reservoirs in the very rst stage. Additional side constraints include DC load ow constraints which govern the transmission ows and conservation of water ow equations in hydro-electric systems. The linear program for each stage has 273 variables and 120 constraints. The objective in each stage is to minimize the cost of thermal electricity generation over the current week plus the expected future cost of thermal generation. The multi-stage models described above were converted into two-stage and threestage problems by aggregating consecutive stages into large problems. For example, to obtain a two stage problem from a multi-stage problem we aggregate each second stage problem and its descendants into a single deterministic equivalent linear program. Table 1 presents the sizes and characteristics of the resulting problems. Although the problems in each pair have the same sizes they dier in the lower bounds imposed on the nal levels of the reservoirs. Column 1 of Table 1 gives the problem identiers, column 2 presents the number of stages in the problem (after aggregation), and column 3 contains the size of the deterministic equivalent problem. Column 4 contains the size of each subproblem after aggregation. Column 5 contains the number of stages in the problem before aggregation. For example, problem P5 is a 6 stage problem, in which we have aggregated the last 5 stages to produce a 2 stage problem. The last column contains the number of random inows at each stage. the When applied to stochastic programs, Benders' decomposition and the inexact cut algorithm must solve a number of subproblems in each iteration. The resulting cut has as coecients the expectation of the subproblem coecients. In the case of three-stage problems we traverse the scenario tree depth rst using the fast pass procedure (see [10], and [16]). Benders' decomposition and the inexact cut algorithm were both implemented using CPLEX 4.0's primal-dual interior point solver, baropt, to solve the subproblems

13 INEXACT CUTS IN BENDERS' DECOMPOSITION 13 Problem # agg stg P Subproblem # stg # scen. per stg P1 2 10,920x24,843 1,200x2, P2 2 10,920x24,843 1,200x2, P3 2 14,520x33,033 4,800x10, P4 2 14,520x33,033 4,800x10, P5 2 43,680x99,372 14,520x33, P6 2 43,680x99,372 14,520x33, P7 3 14,520x33, x P8 3 14,520x33, x P9 3 43,680x99,372 4,800x10, P ,680x99,372 4,800x10, P ,154x42,966 1,404x1, Table 1 Problem sizes Problem # BD cuts # inex cuts BD time inex time % improvement P % P % P % P % P % P % P % P % P % P % P % Table 2 Performance comparison and the simplex solver, optimize, to solve the rst stage problems. We do not apply the crossover operation (hybbaropt) in solving the subproblems. For the inexact cut algorithm we start with = 10; 000 and reduce it by a factor of 10 at each iteration, and terminate baropt when both primal and dual feasibility is attained in the subproblem and the dual objective is at most away from the primal objective. The termination criteria for both algorithms requires a relative gap of 10?5 between the upper and the lower bounds (i.e. we stop when U?L U < 10?5 ). Table 2 contains a comparison of the computational results for the two methods. All times are reported on a SGI Power Challenge. Column 1 contains the problem identiers. Columns 2 and 3 contain the number of cuts under the exact and inexact cut algorithms respectively. Columns 4 and 5 contain the timing in seconds for the exact and inexact methods respectively. The last column contains the percentage of improvement of the inexact cut algorithm over the exact Benders' decomposition algorithm. The entries in this column are calculated as ( exact time?inexact time exact time ) 100%. Note that traditionally the subproblems are not aggregated and they are solved using the (dual) simplex method with warm starting. For some problems this is more

14 14 ZAKERI, PHILPOTT, RYAN ecient than using an interior point method on an aggregated subproblem although in other cases (e.g. P3, P7, and P11) we experienced signicant speed up by aggregating and using the interior point method versus Benders' decomposition with warm starting simplex. It may be possible to warm start the interior point method eectively, when solving the subproblems using recent research developed to this end (see for example [17, 8]). 6. Conclusions. In every one of our problems the inexact cut algorithm improved the time to obtain a solution with the same accuracy as that of the Benders' decomposition algorithm. In our experiments, the choice of f i g is made independently of the problem. Further improvements in speed can be achieved by making a problem dependent choice of f i g. In Table 2 the greatest improvements were obtained in cases where the Benders' decomposition requires a large number of cuts. In these cases we observed that often during the course of the exact algorithm the lower bounds did not change over the course of several iterations. The inexact cut algorithm does not display this behaviour, and reaches an approximately optimal solution with fewer cuts. This suggests that computing cuts inexactly is a promising and simple improvement strategy for operations research practitioners who observe similar behaviour in Benders' decomposition applied to their stochastic linear programming models. REFERENCES [1] O. Bahn, O. Du Merle, J.-L. Goffin, and J.-P. Vial, A cutting plane method from analytic centers for stochastic programming, Mathematical Programming, Series B, 69 (1995), pp. 45{73. [2] J. F. Benders, Partitioning procedures for solving mixed-variables programming problems, Numerische Mathematik, 4 (1962), pp. 238{252. [3] K. P. Broad, Power generation planning using scenario aggregation, master's thesis, University of Auckland, Auckland, New Zealand, [4] G. B. Dantzig and P. Wolfe, Decomposition principle for linear programs, Operations Research, 8 (1960), pp. 101{111. [5], The decomposition algorithm for linear programs, Econometrica, 29 (1961), pp. 767{ 778. [6] E. Flippo and A. Rinnooy Kan, Decompositon in general mathematical programming, Mathematical Programming, 60 (1993), pp. 361{382. [7] A. M. Geoffrion, Generalized benders' decomposition, Journal of Optimization Theory and Applications, 10 (1972), pp. 237{260. [8] J. Gondzio, Warm start of the primal-dual method applied in the cutting plane scheme, Mathematical Programming, (1997), p. to appear. [9] W. Hogan, Application of general convergence theory for outer approximation algorithms, Mathematical Programming, 5 (1973), pp. 151{168. [10] J. Jacobs, G. Freeman, J. Grygier, D. Morton, G. Schultz, K. Staschus, and J. Stedinger, SOCRATES: a system for scheduling hydroelectric generation under uncertainty, Annals of Operation Research, 59 (1995), pp. 99{133. [11] P. Kall and S. W. Wallace, Stochastic Programming, John Wiley & Sons, [12] J. E. Kelley Jr., The cutting-plane method for convex programs, Journal of the SIAM, 8 (1960), pp. 703{712. [13] K. Kim and J. L. Nazareth, The decomposition principle and algorithm for linear programming, Linear Algebra and its Applications, 152 (1991), pp. 119{133. [14] R. T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, NJ, [15] R. VanSlyke and R. J.-B. Wets, L-shaped linear programs with applications to optimal control and stochastic linear programs, SIAM Journal of Applied Mathematics, 17 (1969), pp. 638{ 663. [16] R. J. Wittrock, Advances in a nested decomposition algorithm for solving staircase linear programs, Tech. Report SOL 83-2, Systems Optimization Laboratory, Department of Operations Research, Stanford University, 1983.

15 INEXACT CUTS IN BENDERS' DECOMPOSITION 15 [17] G. Zakeri, D. Ryan, and A. Philpott, Techniques for solving large scale set partitioning problems, Computational Optimization and Applications, (1997), p. submitted.