A robust approach to the chance-constrained knapsack problem

Transcription

1 A robust approach to the chance-constrained knapsack problem Olivier Klopfenstein 1,2, Dritan Nace 2 1 France Télécom R&D, rue du gl Leclerc, Issy-les-Moux cedex 9, France 2 Université de Technologie de Compiègne, Laboratoire Heudiasyc UMR CNRS 6599, Compiègne Cedex, France. olivier.klopfenstein@francetelecom.com, nace@utc.fr June 6, 2006 Abstract Chance-constrained programming is a relevant model for many concrete problems. However, it is known to be very hard to tackle directly. In this paper, the chance-constrained knapsack problem (CKP) is addressed. Relying on the recent advances in robust optimization, a tractable combinatorial algorithm is proposed to solve CKP. It always provides feasible solutions for CKP. Moreover, for two specific classes of uncertain knapsack problems, it is proved to solve CKP at optimality. 1 Introduction. Many concrete problems require to take into account uncertainty on input data. Several frameworks have been proposed these last fifty years to deal with uncertainty in optimization problems. Among them, three main families of approaches can be distinguished: sensitivity analysis, stochastic optimization and robust optimization. As already remarked by many authors, sensitivity analysis is an approach deeply different from the two others, since it does not impact the optimization process: it only tries to say how good is a given solution with respect to uncertainty (post-optimization analysis). In other words, sensitivity analysis does not propose any solution. By contrast, stochastic and robust optimization aim at providing a solution adapted to the uncertain framework considered. The first one relies on probabilistic information, supposed to be available, and tries to find a solution which is good in a probabilistic sense. For instance, the expected value of the objective function can be optimized with respect to the considered possible events. Robust optimization aims generally at providing solutions feasible for all the uncertain events considered. Chance-constrained programming is a very attractive part of stochastic optimization (see for instance [9, 13]). It is devoted to finding the best solution which remains feasible with probability at least 1 ε, for a given ε > 0. Such a model is relevant for many problems, where it is acceptable that a given solution is not feasible for all the events taken into account, as soon as the unfeasibility probability is controlled. For instance, this approach becomes particularly interesting when dealing with problems for which uncertainty assumptions remain approximative. In this case, there is often no sense for looking for a solution always feasible, since the worst case (if there is one) is very likely to never occur. While the direct resolution of chance-constrained programs is often very difficult, some recent robust optimization frameworks bring a new way to deal with these problems [4, 8]. More precisely, given an initial set of uncertain events, the authors propose a way to relax it into a subset of events of probability not less than 1 ε, so that the associated robust optimization problem is tractable. As a result, solving the robust problem provides a feasible solution for the chance-constrained problem. The links between chance-constrained programming and robust optimization have been underlined only recently. [10] compares both approaches and proposes an intermediate resolution framework. The authors study the randomized construction of a set of events with the following property: a solution robust to this set of events is a solution of a chance-constrained program. [12] follows the same line, while dealing with uncertainty on data probability distributions themselves. [15] shows how the robust optimization framework developed in [4, 5] provides an approximation of chance-constrained programming. [11] also proposes robust optimization as a technique to obtain feasible solutions to chance-constrained programs. 1

2 In the same spirit, we propose to use robust optimization techniques to provide a solution to chanceconstrained problems. This work focuses on the knapsack problem, because of its importance for integer linear programming. When dealing with uncertain linear programs, the approach of Bertsimas and Sim [8] appears as particularly appropriate, especially because it preserves the linearity of the basis problem. Furthermore, it can be readily used for integer linear programs. Note that this approach can be seen as a particular case of the robustness model of [2, 3], extended with a probabilistic analysis. In the present paper, the links between this robustness model and chance-constrained programming are investigated for the specific case of the knapsack problem. A pseudo-polynomial time resolution algorithm is designed for the chance-constrained knapsack problem. It is proved to provide always feasible solutions, and even optimal solutions for two specific classes of uncertain knapsack problems. To the best of our knowledge, this is the first time an optimal resolution process is designed for a chance-constrained integer linear program where constraint coefficients are uncertain. 2 Problems and formulations. 2.1 The knapsack problem. Let I = {1,..., n}. Given a profit vector p IR n +, a weight vector w IR n + and a knapsack capacity c > 0, the classical knapsack problem is: i I p ix i i I w ix i c (1) x {0, 1} n For an extensive study of the knapsack problem, we refer for instance to [14]. It is known to be NP-hard, even though there exists a pseudo-polynomial dynamic programming algorithm to solve it. In the rest of this paper, the scalar product of two vectors a and b in IR n will sometimes be denoted by: a.b = i I a ib i. 2.2 The Chance-constrained Knapsack Problem. Let us suppose that the weight vector w is in fact not known with accuracy, that is, w can take values in a set W IR n +. Consider a probability measure P on W (P(w W) = P(W) = 1). The Chanceconstrained Knapsack Problem (CKP) is: i I p ix i P( i I w ix i c) 1 ε x {0, 1} n (2) where ε 0. That is, we look for the best solution x such that x remains feasible with probability at least 1 ε. As noted in most of the introductions to chance-constrained programming, the set of fractional points {x [0, 1] n P(w.x c) 1 ε} is often intractable because of multivariate integral calculations (see e.g. [9, 13] for more details). Example: Let us present a very simple example in IR 2 by describing the set: X = {x [0, 1] 2 P(w 1 x 1 + w 2 x 2 1) 3/4} (ε = 1/4). Suppose that both random weights w 1 and w 2 are uniformly distributed on [0,1]. Let us denote d their density function: d = 1I [0,1] (characteristic function of [0,1]). Let (x 1, x 2 ) [0, 1] 2. We denote by W 1,2 the random variable x 1,2 w 1,2. Each W i has a density function d i = 1/x i.1i [0,xi]. Then P(w 1 x 1 + w 2 x 2 1) = P(W 1 + W 2 1) = 1 t=0 { Easy calculations lead to: P(W 2 1 t) = d 1 (t).p(w 2 1 t)dt = x1 t=0 1 if t 1 x 2 (1 t)/x 2 if t 1 x 2 d 1 (t).p(w 2 1 t)dt If x 1 + x x 2 x 1, it follows that: P(w 1 x 1 + w 2 x 2 1) = 1. If x 1 + x 2 > 1 1 x 2 < x 1, we obtain: P(w 1 x 1 + w 2 x 2 1) = 1 x 2 t=0 d 1 (t)dt + x 1 t=1 x 2 d 1 (t).(1 t)/x 2 dt = 1/x 1 + 1/x 2 1/2.(x 2 /x 1 + x 1 /x 2 ) 1/(2x 1 x 2 ) 2

3 The set X is represented on Figure proba x_ x_ Figure 1: Probability of solutions x [0, 1] 2 for the example Hence, this very basic example shows that chance-constrained problems are quite hard to tackle in a direct way. But observe that CKP can be equivalently written: { } {p.x w V, w.x c} (3) V W:P(V ) 1 ε x {0,1} n (V, x) is said to be a feasible solution of (3) if V W, P(V ) 1 ε and: w V, w.x c. It is optimal if the profit p.x dominates this associated to any other feasible solution. Proposition 1 x is an optimal solution of CKP if, and only if, there exists V W of probability P(V ) 1 ε such that (V, x) is an optimal solution of (3). Proof: It is sufficient to observe that x is a feasible solution of CKP if, and only if, there exists V W of probability P(V ) 1 ε such that (V, x) is a feasible solution of (3). For the direct sense, consider V = {w W w.x c}. For the reverse one, observe that P(w.x c) P(V ). This formulation relies on robust optimization subproblems defined for subsets V of possible weights. For any subset V of W of probability measure P(V ) 1 ε, the resolution of x {0,1} n{p.x w V, w.x c} provides a feasible solution to CKP. This motivates the recourse to robust optimization models for solving CKP. More specifically, the aim of this study is to rely on an adaptation of the appealing framework proposed in [8] to provide good feasible solutions to CKP, and even optimal ones for some specific cases. 2.3 Solving CKP with robust models. From now on, we assume that the coefficients w i lie in intervals [w i, w i ]: w i 0 is the lowest possible weight value for element i, while w i w i is the largest one. Thus, W is included in the cartesian product of intervals [w i, w i ]. To each weight is associated a random variable (r.v.); for the sake of simplicity the random variables and their realizations are denoted by the same symbol w i. Let us also introduce, for all i I, the random variable η i : η i = (w i w i )/(w i w i ) [0, 1]. Let Γ {0,...,n}, the idea of the robust formulation is to find a solution feasible even though up to Γ coefficients of w take their largest values. Thus, in particular, if Γ = 0, we consider only the best scenario, where all weights take their lowest values; if Γ = n, the worst case is taken into account. Note that [8] presents a very similar model where Γ can be non-integral: Γ [0, n]. Although this could directly be adapted here, for the sake of simplicity, our models will assume the integrality of this robustness coefficient. 3

4 Within this framework, let us introduce the Robust Knapsack Problem (RKP), parameterized by Γ: i I p ix i For any x {0, 1} n, we denote: I(x) = {i I x i = 1}. i I\S w i x i + i S w ix i c, S I S = Γ x {0, 1} n (4) Lemma 1 For any feasible solution x of RKP: P(w.x c) P ( i I η i Γ ). Proof: Let x be a feasible solution of RKP. If I(x) Γ, there exists S I of cardinality Γ such that I(x) S. Then: i I w ix i = i I(x) w ix i = i S w ix i + i/ S w i x i c (since x is feasible for RKP). This implies that: P(w.x c) = 1, and thus: P(w.x c) P( i I η i Γ). Suppose now that I(x) Γ+1. For the sake of clarity, let us denote: δ i = w i w i. For any subset S I(x) of cardinality Γ: P(w.x > c) P ( w.x i S w ix i + i/ S w i x ) ( i = P i I(x) w i i S w i + i I(x)\S ( i) w = P i I(x) η iδ i i S i) ( δ = P i I(x)\S η iδ i i S δ i(1 η i )) ( P i I(x)\S η iδ i min j S δ j. i S i)) ( (1 η = P i I(x)\S η i[δ i / min j S δ j ] + i S η i Γ) The first inequality comes from the feasibility of x for RKP. Now, we can ( choose S I(x) so that: ) i I(x) \ S, δ i min j S δ j. In this case, we obtain: P(w.x > c) P i I(x) η i Γ, and thus, clearly: P(w.x > c) P ( i I η i Γ ). Note that this result is general, since it requires no assumption on probability distributions of weights. The robust model proposed, directly inspired by this of [8], appears as particularly relevant for approximating our chance-constrained program. First, the probability results of [8] can be adapted to our framework, for instance: Proposition 2 Assume that ( the random variables w i are independent and symmetrically distributed. Let ε (0, 1). If Γ 1/2. n + ) 2n ln(ε), a feasible solution of RKP will be feasible for CKP. Proof: Let us introduce η i = 2η i 1. The r.v. { η i } i I are independent and symmetrically distributed on [-1,1]. Suppose that Γ n/2. From the probability results of [8], we know that: P( i I η i 2Γ n) exp ( (2Γ n) 2 /(2n) ). On the other hand: P( i I η i 2Γ n) = P( i I η i Γ), and thus: P( i I η i Γ) exp ( (2Γ n) 2 /(2n) ). We conclude thanks to Lemma 1. Better, but also more complex, probability bounds are proved to exist and can lead to a better choice of Γ. Secondly, the robust set of events w associated to the set of feasible solutions of RKP (characterized in Lemma 1) seems appropriate, since in particular it discards the worst-case, where all weights take their imal values w i. Finally, as stated in [8], the multi-knapsack formulation (4) can also be equivalently written: x {0,1} n{p.x z 0, y 0 i I w i x i + Γz + i I y i c } and i I, z + y i (w i w i )x i This formulation makes the computational resolution tractable when using a branch-and-bound algorithm. One of the greatest advantages of the robust framework proposed in [8] is to preserve the linearity of the initial problem. Thus, as clearly expressed by the authors in their initial paper, it is directly usable for integer programming. However, this aspect has not been extensively investigated so far. [8, 7] provide some numerical tests on the robust formulation of some classical combinatorial problems. In particular, a robust knapsack problem very close to ours is introduced, and the authors show numerically 4

5 the impact of protection level Γ on the optimal profit. But to the best of our knowledge, the previous theoretical studies on applications of [8] for robust integer programming have focused on cases where only objective coefficients are uncertain. While [7] provides some complexity results, [16] proves a probability bound easier to compute than this proposed by [8]. [1] investigates some reformulations and tight linear programming formulations for this particular case of uncertain cost coefficients. By contrast, the present paper deals with robust integer programming applied to the knapsack problem when constraint coefficients are uncertain. Finally, with respect to classical stochastic programming approaches, the present study is somewhat original since it is fully combinatorial. That is, no convex optimization tools are needed. 3 On the theoretical link between CKP and RKP. Let I I and Γ {0,..., I }, we introduce the problem RKP(I,Γ): p.x i S w ix i + i I \S w ix i + i/ I w ix i c, S I S = Γ (5) x {0, 1} n RKP(I,Γ) can be seen as a robust knapsack problem (4) with all weights of elements in I \ I set to their worst possible value, with no uncertainty for these values. From now on, to be more precise, RKP will be denoted by RKP(I,Γ). Observe that for any two subsets I and I of I, I I implies that any feasible solution of RKP(I,Γ) is also feasible for RKP(I,Γ). On the other hand, considering Γ Γ, any feasible solution of RKP(I,Γ) is also feasible for RKP(I,Γ ). In particular, any feasible solution of RKP(I,Γ) is also feasible for RKP(I,Γ). Lemma 2 Let Γ {0,...,n}. Suppose that x is an optimal solution of RKP(I,Γ): (i) if Γ I(x ), x is an optimal solution of RKP(I(x ),Γ). (ii) If Γ I(x ), x is an optimal solution of RKP(I,n). Proof: Suppose that Γ I(x ). Since I(x ) I, any feasible solution of RKP(I(x ),Γ) is feasible for RKP(I,Γ). But it can be seen that x is a feasible solution of RKP(I(x ),Γ). Then, as it is optimal for RKP(I,Γ), it is optimal also for RKP(I(x ),Γ). If now Γ I(x ), there exists S I of cardinality Γ such that I(x ) S. Then: i I w ix i = i I(x ) w ix i = i S w ix i + i/ S w i x i c. The last inequality comes from the fact that x is feasible for RKP(I,Γ). Thus, x is shown to be a feasible solution of RKP(I,n). But since Γ n, RKP(I,Γ) is a relaxation of RKP(I,n): x is also optimal for RKP(I,n). Finally, observe that in the case where Γ = I(x ), both statements (i) and (ii) hold. Then, RKP(I(x ),Γ) and RKP(I,n) both admit x as an optimal solution. The following result shows the relevance of the robust setting proposed to deal with CKP when all weight variations (w i w i ) are identical: Theorem 1 Suppose that: i I, w i w i = δ > 0, and that {w i /δ} i I and c/δ are integers. Then, there exist I I and Γ {0,..., I } such that any optimal solution of RKP(I,Γ ) is an optimal solution of CKP. Proof: Let x be an optimal solution of CKP, we define: I = I(x ). Since x is feasible with probability at least 1 ε, we have: P(w.x c) = P ( i I w i c ) 1 ε. Let us denote V = {w W i I w i c}. Let consider now the following problem: (6) p.x w.x c, w V x {0, 1} n Let us prove first that any optimal solution of (6) is an optimal solution of CKP. Any feasible solution x of (6) is feasible for CKP, since: P(w.x c) P(w V ) 1 ε. Thus, we just have to show that 5

6 (6) has the same optimal value as CKP. This is clear, since by construction, the optimal solution x of CKP is a feasible solution of (6). The goal now is to find Γ {0,..., I } such that (6) is equivalent to RKP(I,Γ ). Observe that: w V i I η i ( c i I w i) /δ, and let us denote: Γ = min { I, ( c i I w } i) /δ. We have: w V i I η i Γ. Note that V φ, which implies that ( c i I w i) /δ 0. On the other hand, from the hypothesis, Γ is integral. Let us show that (6) is in fact equivalent to RKP(I,Γ ). Let us prove that x is a feasible solution of (6) if, and only if, x is a feasible solution of RKP(I,Γ ). Let x be feasible for (6). For a subset S I with S = Γ, let us prove that i S w ix i + i I \S w i x i + i/ I w ix i c. Let w(s) denote the weight vector whose components are w i if i I \ S, and w i otherwise. We have: i I w(s) i = i S w i + i I \S w i = Γ δ + i I w i c, and consequently: w(s) V. Then, since x is feasible for (6), we have: w(s).x = i S w ix i + i I \S w ix i + i/ I w ix i c. Hence, x is also feasible for RKP(I,Γ ). Suppose now that x is not feasible for (6), that is, there exists w V such that w.x > c, and let us prove that there exists S I with S = Γ such that: i S w ix i + i I \S w i x i + i/ I w ix i = i I w i x i + i S δx i + i/ I w ix i > c. Let look at I(x) I : if I(x) I Γ (a), we construct S with S = Γ such that S I(x) I. When I(x) I < Γ (b), let S be any extension of I(x) I in I of cardinality Γ : I(x) I S I and S = Γ (recall that, by construction, Γ I ). Then, if (a) is satisfied we obtain: i I w ix i + i S δx i + i/ I w ix i = i I w ix i + δγ + i/ I w i x i i I w i x i + i I η i δ+ i/ I w i x i i I w i x i + i I η i δx i + i/ I w i x i = w.x > c. The first inequality comes from w V ; for the last equality, recall that w i = w i + η i δ. Similarly if (b) is satisfied we obtain: i I w ix i + i S δx i + i/ I w ix i = i I w ix i + δ I(x) I + i/ I w ix i i I w i x i + i I(x) I η i δx i + i/ I w i x i = i I w i x i + i I η i δx i + i/ I w i x i = w.x > c. By contraposition, this proves that any feasible solution of RKP (I,Γ ) is also feasible for (6). Hence, it is shown that x is a feasible solution of (6) if, and only if, x is a feasible solution of RKP(I,Γ ). As a consequence, since both objective functions are identical, both problems are equivalent, and in particular, they admit the same optimal solutions. Since any optimal solution of (6) is an optimal solution of CKP, it is proved that any optimal solution of RKP(I,Γ ) is also optimal for CKP. One of the strengths of the above theorem is that it requires absolutely no assumption on probability distributions for weights. The conditions on integrality of {w i /δ} i I and c/δ are necessary to ensure that Γ can be chosen integral. Nevertheless, recall that in a more general setting, Γ may also be chosen non-integral (cf Section 2.3): in this case, the theorem could be easily generalized without the integrality conditions on {w i /δ} i I and c/δ. However, these assumptions do not seem very constraining for many practical applications, where δ will in fact be small. In the rest of this section, two particular cases are more specifically addressed, when weights or profits are all identical. In each case, the above theorem is specified, since we show that there exists a coefficient Γ such that all optimal solutions of RKP(I,Γ) are optimal also for CKP. Lemma 3 Suppose that: weights and profits can be sorted so that: i < j { wi w j p i p j, for all i I, w i w i = δ > 0, the r.v. {η i } i I are independent and identically distributed. There exists an optimal solution x of CKP such that I(x ) = {1,..., I(x ) }. Proof: Let x be an optimal solution of CKP, and let us build x such that I(x ) = {1,..., I(x) }. As I(x ) = I(x) and profits p i are sorted in non-increasing order: p.x p.x. Moreover, observe that by construction: i I(x ) w i i I(x) w i. Then: P(w.x c) = P( i I(x ) w i c) = P(δ. i I(x ) η i c i I(x ) w i ) = P(δ. i I(x) η i c i I(x ) w i ) (a) P(δ. i I(x) η i c i I(x) w i) (b) = P( i I(x) w i c) = P(w.x c) 6

7 As the r.v. {η i } i I are supposed independent and identically distributed, and since I(x ) = I(x), the r.v. i I(x) η i and i I(x ) η i are identically distributed: this ensures (a). (b) is a direct consequence of i I(x ) w i i I(x) w i. Thus, as x is feasible for CKP, x is also feasible for this problem. Hence, x is an optimal solution of CKP. Theorem 2 Suppose that: weights and profits can be sorted so that: i < j { wi w j p i p j, for all i I, w i w i = δ > 0, {w i /δ} i I and c/δ are integers, the r.v. {η i } i I are independent and identically distributed. Then, there exists Γ {0,...,n} such that any optimal solution of RKP(I,Γ ) is an optimal solution of CKP. Proof: From Theorem 1, we know that there exist I I and Γ {0,..., I } such that an optimal solution of RKP(I,Γ ) is an optimal solution of CKP. Moreover, from the proof, we know that I can be chosen so that I = I(x ) for some optimal solution x of CKP, and that Γ can be chosen so that: Γ = min { I, ( c i I w } i) /δ. Without loss of generality, from Lemma 3, we consider: I(x ) = {1,..., I }. Suppose first that Γ = I. In this case, problems RKP(I,Γ ) and RKP(I,n) are equivalent. Suppose now that Γ = ( c i I w i) /δ < I. Let us prove that any optimal solution of RKP(I,Γ ) is optimal also for CKP. Let x be an optimal solution of RKP(I,Γ ). If I(x ) Γ, x is feasible for RKP(I,n) and consequently: P(w.x c) = 1. Thus, in this case, x is feasible for CKP. Consider now that I(x ) Γ + 1, and let us prove that x is a feasible solution of CKP. Suppose that: i I(x ) w i > i I w i, then for any subset S I(x ) of cardinality Γ : w i x i + w i x i = w i + Γ δ > w i + Γ δ = c Γ δ + Γ δ = c i I i S i I\S i I(x ) This is a contradiction with the feasibility of x for RKP(I,Γ ). As a consequence: i I(x ) w i i I w i. Furthermore, as I is the collection of the I lowest values w i, this implies that I(x ) I. On the other hand, as I I, x is a feasible solution of RKP(I,Γ ), and we have: p.x p.x, i.e.: i I p i i I(x ) p i. As I is the collection of the I largest profit values, this implies that I(x ) I. As I is also the collection of the I lowest values w i, this leads to: i I(x ) w i i I w i. As a result: I(x ) = I and i I(x ) w i = i I w i. Now, let us see that x is feasible for CKP: P(w.x c) = P(δ. i I(x ) η i c i I(x ) w i ) = P(δ. i I(x ) η i c i I(x ) w i ) (a) = P(δ. i I(x ) η i c i I(x ) w i ) (b) = P(w.x c) (a) comes from i I(x ) w i = i I w i. (b) comes from I(x ) = I and probability assumptions on r.v. η i. This shows that x is a feasible solution of CKP. Finally, we have already seen that p.x p.x, which ensures that x is an optimal solution of CKP. As previously for Theorem 1, the integrality of c/δ and of w i /δ for all i I is not really a requirement, since Γ may also be chosen non-integral (cf Section 2.3). As particular cases of the above theorem, the two following results hold: Corollary 1 Suppose that: for all i I, w i = ω > 0 and w i w i = δ > 0, 7

8 ω/δ and c/δ are integers, the r.v. {η i } i I are independent and identically distributed. Then, there exists Γ {0,...,n} such that any optimal solution of RKP(I,Γ ) is an optimal solution of CKP. Corollary 2 Suppose that: for all i I, p i = ρ > 0 and w i w i = δ > 0, for all i I, w i /δ and c/δ are integers, the r.v. {η i } i I are independent and identically distributed. Then, there exists Γ {0,...,n} such that any optimal solution of RKP(I,Γ ) is an optimal solution of CKP. 4 Solving CKP. 4.1 Complexity of RKP and resolution algorithm. Within this section, RKP(I,Γ) is denoted simply RKP. Moreover, all data {w i } i I, {w i } i I and c are suppose to be non-negative integers. It is shown that the classical dynamic programming algorithm can be adapted, and that the pseudo-polynomiality of the classical knapsack problem is preserved: Theorem 3 RKP is weakly NP-hard: there exists a pseudo-polynomial time algorithm to solve it. Proof: That RKP is NP-hard comes immediately, since the particular case Γ = 0 is in fact the classical knapsack problem with weights {w i }. On the other hand, the classical dynamic programming algorithm for knapsack problems can be adapted to solve RKP. Let us denote by RKP k (Γ, b) the problem RKP for robust parameter Γ {0,..., n} and capacity b {1,...,c}, considering only elements in I k = {1,..., k} for k I. Let F k (Γ, b) denote its optimal solution value. We suppose without loss of generality that the elements of I are sorted so that: i < j (w i w i ) (w j w j ). For any k, we assume that: b < 0 F k (., b) =. For k = 1, F 1 is defined as follows: { 0 if b < w1 if Γ = 0: F 1 (0, b) = p 1 if b w 1 { 0 if b < w1 if Γ 1: F 1 (Γ, b) = p 1 if b w 1 Consider b {1,..., c} and Γ {1,...,n}. Then the optimal value of RKP k (Γ, b) can be computed by the recurrence formula: F k (Γ, b) = {F k 1 (Γ, b), p k + F k 1 (Γ 1, b w k )} Let us prove that F k (Γ, b) is the optimal value of RKP k (Γ, b). The result is obvious for k = 1. Let k 2. Let us show first that any feasible solution x k 1 {0, 1} k 1 of RKP k 1 (Γ, b) or RKP k 1 (Γ 1, b w k ) can be extended into a feasible solution x k {0, 1} k of RKP k (Γ, b) by adding a k th element to x k 1. If x k 1 is feasible for RKP k 1 (Γ, b), then considering x k k = 0 ensures that xk is feasible for RKP k (Γ, b). If x k 1 is feasible for RKP k 1 (Γ 1, b w k ), let us consider the vector x k with x k k = 1. Then, we have: i I k w i x k i + { i S (w i w i )x k i : S I k, S Γ} = i I k 1 w i xi k 1 + w k + { i S (w i w i )x k 1 i : S I k 1, S Γ 1}. This comes from the order assumed on weight variations. Combining this with: i I k 1 w i x k 1 i + { i S (w i w i )x k 1 i : S I k 1, S Γ 1} b w k, we obtain that x k is also feasible for RKP k (Γ, b). The reverse, that is for any feasible solution of RKP k (Γ, b), the first k 1 components give a feasible solution either for RKP k 1 (Γ, b) or for RKP k 1 (Γ 1, b w k ), can be shown as follows. Let x k denote 8

9 a feasible solution of RKP k (Γ, b). Let x k 1 {0, 1} k 1 denote the vector composed of the first k 1 elements in x k. Two cases can be distinguished: (a) x k = 0: then xk 1 is feasible for RKP k 1 (Γ, b). (b) x k k = 1: then we can prove by contraposition that xk 1 is a feasible solution of RKP k 1 (Γ 1, b w k ). Suppose that there exists S I k 1 with S = Γ 1, such that: i I k 1 w i x k 1 i + i S (w i w i )xi k 1 > b w k. Then, x k cannot be feasible for RKP k (Γ, b) since for S = S {k} we obtain: i I k w i x k i + i S (w i w i )x k i = i I k w i x k i + i S (w i w i )x k i + (w k w k ) = i I k 1 w i x k 1 i + i S (w i w i )x k 1 i + w k > b. Thus, we have shown that the set of vectors composed of the k 1 first components of any feasible solution x k of RKP k (Γ, b) is equal to the union of all feasible solutions of respectively RKP k 1 (Γ, b) and RKP k 1 (Γ 1, b w k ). Hence we have: F k (Γ, b) = {F k 1 (Γ, b), p k + F k 1 (Γ 1, b w k )}. Finally, observe that the required value F n (Γ, c) can be computed in pseudo-polynomial time O(nΓc). 4.2 An iterative algorithm. In the following algorithm, the value of Γ is progressively increased until the optimal solution of RKP(I,Γ) is a feasible solution of CKP. Algorithm 1: Step 1: Let k = 0. Step 2: Solve the problem RKP(I,k). Let x (k) denote its optimal solution. Step 3: Set I = I(x (k) ). Compute Γ, the largest value of {0,..., I } such that Step 4: x (k) is feasible for RKP(I,Γ ). Compute a bound B P( i I w i c). If B 1 ε, STOP. Step 5: Set k Γ + 1 and go to Step 2. At Step 4, observe that: P( i I w i c) = P(w.x (k) c). Thus, we know that the algorithm stops only with a solution feasible for CKP. Moreover, as x (k) is a feasible solution of RKP(I,k) (cf Lemma 2), we have Γ k. Hence, the increase of index k at each loop ensures the convergence: indeed, when k = n = I, the feasibility probability of x (n) is equal to 1 and the algorithm stops at Step 4. Thus, k can not exceed this value. As a result: Lemma 4 Algorithm 1 provides a feasible solution to CKP by solving at most n + 1 robust knapsack problems. Some further observations have to be made. First, from a given iteration k, it is unnecessary to consider the problems RKP(I,l), with k + 1 l Γ. Indeed, x (k) is already feasible for RKP(I,Γ ), since it is feasible for RKP(I,Γ ). That is the reason for considering directly Γ + 1 at Step 5. On the other hand, the value Γ at Step 3 can be computed in linear time. Indeed, it is the optimal solution of: Γ i I w i + S I : S =Γ i S (w i w i ) c Γ {0,...,n} It is sufficient to consider the empty set S = φ and to fill it progressively with the indices of the largest weight variations w i w i. As soon as i S w i w i > c i I w i, we have: Γ = S 1. To compute B at Step 4, we refer for instance to the probability bounds proposed in [8]. The bound already used in the proof of Proposition 2 under specific probability assumptions may be used. It is clear that the quality of the bound used will directly impact the number of iterations of the algorithm and the quality of the final solution. It has to be noted that if I = Γ, the feasibility probability is 1 and the algorithm stops (cf Lemma 2). As a consequence of Lemma 4, we have: Proposition 3 Algorithm 1 runs in pseudo-polynomial time by using the dynamic programming algorithm provided by the proof of Theorem 3. 9

10 4.3 Two particular cases. Let us finally consider the case of uniform weights, and this of uniform profit values. Lemma 5 Suppose that for all i I, w i = ω > 0 and w i w i = δ > 0. Then an optimal solution x of RKP(I,Γ) satisfies: { x min {n, (c Γδ)/ω } if Γ(ω + δ) c i = c/(ω + δ) otherwise i I Proof: Observe first that any optimal solution of the uniform RKP(I,Γ) will imize the sum of its components ( i I x i). Suppose that Γ(ω+δ) c. This means that an optimal solution x of RKP(I,Γ) has at least Γ elements: i I x i Γ. Then, x is in fact an optimal solution of the problem with the only constraint: i I ωx i c Γδ. The result follows. Now, if Γ(ω+δ) > c, a feasible solution of RKP(I,Γ) can not have more than Γ 1 elements. Then, an optimal solution x of RKP(I,Γ) is in fact an optimal solution of the problem with the only constraint: i I (ω + δ)x i c. This ends the proof. From this lemma, an optimal solution of RKP(I,Γ) can be built in linear time by considering successively the elements of largest profit values. Theorem 4 Suppose that: for all i I, w i = ω > 0 and w i w i = δ > 0, ω/δ and c/δ are integers, the r.v. {w i } i I are independent and identically distributed, P( i I w i c) is known for any subset I I. Then Algorithm 1 provides an optimal solution to CKP in polynomial time O(n 3 ). Proof: It has just been seen that for any value of Γ, an optimal solution of RKP(I,Γ) can be computed in linear time O(n). From Lemma 4, Algorithm 1 stops in at most n + 1 resolutions of robust problems, and since Step 3 is solved in time O(n), it runs in time O(n 3 ). From corollary 3.1, we know that one of the solutions x (k) explored with Algorithm 1 is optimal for CKP. Moreover, the sequence of objective values p.x (k) is non-increasing. As P( i I w i c) is supposed to be exactly known for any I I, the algorithm stops at Step 4 with the first solution feasible for CKP, which then is optimal. Similar results hold for uniform profits. As the proofs use exactly the same ideas as previously, they are not detailed. Lemma 6 Suppose that, for all i I, p i = ρ > 0 and w i w i = δ > 0. If c Γδ, let x be an optimal solution of: i I x i i I w i x i c Γδ (7) x {0, 1} n and let x be an optimal solution of: i I x i i I w ix i c x {0, 1} n (8) If c Γδ and i I x i Γ, x is optimal for RKP(I,Γ). Otherwise, x is optimal for RKP(I,Γ). Note that both problems (7) and (8) are solved in linear time. On the other hand, observe that (7) has no feasible solution if c Γδ < 0. Theorem 5 Suppose that: 10

11 for all i I, p i = ρ > 0 and w i w i = δ > 0, {w i /δ} i I and c/δ are integers, the r.v. {η i } i I are independent and identically distributed, P( i I w i c) is known for any subset I I. Then Algorithm 1 provides an optimal solution to CKP in polynomial time O(n 3 ). 4.4 An example. In this last section, the algorithm 1 is used to solve a simple uniform chance-constrained knapsack problem to the optimum. Consider the following problem: 10 i=1 ix i 10 i=1 w ix i 80 x {0, 1} 10 where all weights w i are uniformly distributed on [8, 12], i.e. w i = 8 and w i = 12 for all i I = {1,...,10}, and δ = 4. Let us consider ε = 0.1, that is, we look for a filling x of the knapsack which will be feasible with probability at least 90%. Note that under the above probability assumptions, the probabilities P( i I w i c) are known analytically for all subsets I I. In the following, let e i denote the 0-1 vector such that e i i = 1 and ei j = 0 for i j. k=0: k=1: From Lemma 5, we know that: i I x(0) i = 80/8 = 10, and thus: x (0) = 10 We have: Γ = ( )/4 = 0, and we compute: P( 10 i=1 w i 80) = 0 (the only event possible is w = w). From Lemma 5: i I x(1) i = (80 4)/8 = 9, and thus: x (1) = 10 i=2 ei. We have: Γ = (80 9 8)/4 = 2, and we compute: P( 10 i=1 ei. i=2 w i 80) = < 0.9. k=3: From Lemma 5: i I x(3) i = (80 12)/8 = 8, and thus: x (3) = 10 i=3 ei. We Γ = (80 8 8)/4 = 4, and we compute: P( 10 i=3 w i 80) = 0.5 < 0.9. k=5: From Lemma 5: i I x(5) i = (80 20)/8 = 7, and thus: x (5) = 10 i=4 ei. We compute: P( 10 i=4 w i 80) = > 0.9. STOP: x (5) is feasible, and thus optimal. Note that considering a feasibility probability of 100% would require to put less elements in the knapsack. Indeed, assuming that all weights take their worst values leads to put at most 80/12 = 6 items in the knapsack. 5 Conclusion. The chance-constrained knapsack problem has been addressed. A relevant robust optimization problem has been proposed. Its theoretical links with the chance-constrained problem have been investigated. Then, a tractable combinatorial algorithm has been designed to obtain good solutions to the chanceconstrained problem, by running a sequence of robust problems. When profits are identical, or when uncertain weights present all the same characteristics, this algorithm is proved to provide an optimal solution. References [1] A. Atamtürk, Strong formulations of robust mixed 0-1 programming, submitted to Math. Program.. Available at FILE/2004/01/816.pdf [2] W. Ben-Ameur and H. Kerivin, Offres de Réseaux Privés Virtuels Flexibles, Tech. Report NT/FTRD/7358, France Télécom R&D (2001). [3] W. Ben-Ameur and H. Kerivin, Routing of uncertain demands, Optim. Eng., Vol. 6, no. 3 (2005), pp

12 [4] A. Ben-Tal and A. Nemirovski, Robust solutions of Linear Programming problems contamined with uncertain data, Math. Program., A88 (2000), pp [5] A. Ben-Tal and A. Nemirovski, Robust optimization - methodology and applications, Math. Program., B92 (2002), pp [6] D. Bertsimas, D. Pachamanova and M. Sim, Robust Linear Optimization under General Norms, Oper. Res. Lett., Vol. 32, Issue 6 (2004), pp [7] D. Bertsimas and M. Sim, Robust discrete optimization and network flows, Math. Program., B98 (2003), pp [8] D. Bertsimas and M. Sim, The Price of Robustness, Oper. Res., Vol. 52, no. 1 (2004), pp [9] J.R. Birge and F. Louveaux, Introduction to Stochastic Programming, Springer-Verlag, [10] G. Calafiore and M.C. Campi, Uncertain convex programs: randomized solutions and confidence levels, Math. Program., A102 (2005), pp [11] X. Chen, M. Sim and P. Sun, A Robust Optimization Perspective of Stochastic Programming, submitted to Oper. Res. (2005). [12] E. Erdoğan and G. Iyengar, Ambiguous chance constrained problems and robust optimization, Math. Program. (online) DOI: /s (2005). [13] P. Kall and S.W. Wallace, Stochastic Programming, Wiley, Chichester, [14] S. Martello and P. Toth, Knapsack Problems: Algorithms and Computer Implementations, Wiley, [15] A. Nemirovski and A. Shapiro, Convex Approximations of Chance Constrained Programs, submitted to SIAM J. Optim.. [16] M.C. Pinar, A Note on Robust 0-1 Optimization with Uncertain Cost Coefficients, 4OR, Vol. 2 (2004), pp [17] N.V. Sahinidis, Optimization under uncertainty: state-of-the-art and opportunities, Computers & Chemical Engineering, 28 (2004), pp