Linear and Integer Programming - ideas

Transcription

1 Linear and Integer Programming - ideas Paweł Zieliński Institute of Mathematics and Computer Science, Wrocław University of Technology, Poland pziel/ Toulouse, France 2012

2 Literature [1] S.P. Bradley, A.C. Hax, T. L. Magnanti Applied Mathematical Programming Addison-Wesley Pub. Co. (Reading, Mass.), [2] G.L. Nemhauser and L.A. Wolsey. Integer and Combinatorial Optimization, John Wiley & Sons, [3] C.H. Papadimitriou, K. Steiglitz, Combinatorial Optimization. Algorithms and Complexity, Dover Publication, Inc,Mineola, A large part of the lecture has been prepared on the basis of the book [1]. pziel/lectures/toulouse/html

3 The general LP Linear Programming problem (LP) z = n c j x j max(min) j=1 X = in a matrix-vector form { n j=1 a ijx j ( ) b i i = 1,..., m, x j 0 j = 1,..., n, z = c T x max(min) { Ax ( ) b, X = x 0, where A R m n, c R n, b R m, and x R n is the vector of variables.

4 A geometrical interpretation LP Example: x Optimal solution 40 x 1 4x 1 + 5x 2 max x 1 + 2x x 1 + 3x x 1 0, x 2 0

5 A geometrical interpretation LP Example: x B C Infinite number of optimal solutions 40 x 1 4x 1 + 3x 2 max x 1 + 2x x 1 + 3x x 1 0, x 2 0

6 A geometrical interpretation LP Example: x Optimal solution x 1 x x 2 min 2x 1 + 5x x 1 + x 2 20 x 1 5 x 1 0, x 2 0

7 Properties of LP x x 2 x x x x 1 Optimal solutions Fact The set of feasible solutions X is a convex set. Fact If an objective function of the LP is bounded in the set of feasible solutions X, then its optimal value is attained at least at one of vertices of set X.

8 Properties of LP The canonical LP z = n j=1 c jx j max { n X = j=1 a ijx j b i i = 1,..., m, x j 0 j = 1,..., n, The standard LP z = n j=1 c jx j max n X j=1 a ijx j + x n+i = b i i = 1,..., m, = x j 0 j = 1,..., n, x n+i 0 i = 1,..., m, where x n+i are slack variables. For inequality, we subtract variable x n+i called surplus variable. Fact There is a one to one correspondence between the set X (the canonical LP) and the set X (the standard LP).

9 Properties of LP z = c T x max(min) Ax = b, x 0, rank(a) = m, B R m m is a nonsingular submatrix of A, and P R m n m. Wlog B is consisted of the first m columns of A. [ ] x Ax = b [B, P] B = b. If x P = 0 then the solution [x B, 0] T of Ax = b is called basic solution and x B = B 1 b. x P The nonsingular matrix B is called basis. If x 0, then x 0 is called basic feasible solution. Theorem Solution x X is a vertex of X if and only if x is a basic feasible solution of Ax = b.

10 Properties of LP 4x 1 + 5x 2 max x 1 + 2x 2 40 X = 4x 1 + 3x x 1, x 2 0 4x 1 + 5x 2 max x 1 + 2x 2 + x 3 = 40 X = 4x 1 + 3x 2 +x 4 = 120 x 1, x 2, x 3, x 4 0 x A B D C x 1 x A = [0, 20, 0, 60 ] x B = [ 24, 8, 0, 0 ] x C = [ 30, 0, 10, 0 ] x D = [ 0, 0, 40, 120 ] x = [40, 0, 0, -60 ] x = [0, 40, -40, 0 ]

11 An idea of an algorithm for LP Idea: In order to find an optimal solution of LP it suffices to enumerate systematically vertices of the set of feasible solutions (or equivalently basic feasible solutions) and choose a vertex in which the optimal value of the objective function is attained. x 2 40 The second vertex Optimal solution 40 x 1 Initial vertex

12 Simplex Algorithm z = c T x min Ax = b, x 0 Assume that we have a basic feasible solution x T = [x B, x P ] T (x B 0, x P = 0) [ ] x Ax = b [B, P] B = b x P Bx B = b, x B = B 1 b and x B 0

13 Simplex Alg. - choosing a column entering the basis Let us partition the cost vector c: Since x B = B 1 b B 1 Px P, z = c T B x B + c T P x P z = c T B B 1 b + (c T P ct B B 1 P)x P = z 0 + p T x P, where z 0 = c T B B 1 b, p T = c T P ct B B 1 P. Vector p, called relative cost vector, can be computed in two steps: B T π = c B and p T := c T P πt P Entering criterion: An index k, k = m + 1,..., n, of column of P = [P m+1,..., P n ] entering to the basis is such that: p k = c k π T P k < 0

14 Simplex Alg. - choosing a column leaving the basis Bx B P k Θ k + P k Θ k = b B(x B B 1 P k Θ k ) + P k Θ k = b B(x B y k Θ k ) + P k Θ k = b Bx(Θ k ) + P k Θ k = b We claim: { x B y k Θ k 0 x(θ k ) = Θ k 0 x(θ k ) = { x B i y k i Θ k 0 1 i m Θ k 0 Leaving criterion: A new basic feasible solution x(θ k ), where Θ k = x { } i B x B yi k = min i y k : yi k > 0, 1 i m i Column B i leaves the basis B = [B 1,..., B m ].

15 Simplex Algorithm Step 1. Construct initial basis B (a basic feasible solution x T = [x B, x P ] T x B 0, x P = 0, x B = B 1 b). Step 2. Check the optimality criterion If p T = c T P πt P 0, then x is optimal - STOP. Stop 3. Choose column P k (variable x k ) to pivot in (i.e., variable x k to introduce into the basis) with respect to criterion: p k = c k π T P k < 0. Step 4. Choose column B i to drive out (variable x i to drive out the basis). If for all i = 1,..., m, yi k 0, then STOP (the problem is unbounded). Step 5. Construct a new basis feasible solution x(θ k ) Go to Step 2.

16 Methods for solving Integer Programming problems (IP) We can distinguish three following approaches for solving integer programming problems: enumeration techniques, including the branch-and-bound technique, cutting-plane techniques, methods based on decomposition.

17 Solving IP by means of LP Example: z = max z = 5x 1 + 8x 2 x 1 + x 2 6 5x 1 + 9x 2 45 x 1, x 2 0 and integer

18 Solving IP by means of LP x 2 6 x 1 + x 2 = L 0 Optimal continuous solution z = 5x 1 + 8x 2 = x 1 + 9x 2 = x 1

19 Solving IP by means of LP Optimal Round Nearest Integer continuous off integer optimal solution solution solution 9 x 1 4 = x 2 4 = z infeasible

20 Branch-and-bound method for IP Branch-and-bound method is based on divide and conquer strategy. Key facts: IP= LP + integrality restrictions Fact 1. The optimal value of the objective function of LP is always an upper bound (a maximization problem) on the optimal value of IP. Fact 2. The value of the objective function of IP for any integer feasible solution is always a lower bound (a maximization problem) on the optimal objective value of IP.

21 Branch-and-bound method for IP Let us drop integrality restrictions and solve the following LP problem. z = max z = 5x 1 + 8x 2 L 0 = x 1 + x 2 6 5x 1 + 9x 2 45 x 1, x 2 0 We get: x 1 = 2 1 4, x 2 = 3 3 4, z0 = and an upper bound z Since the objective function coefficients are integral, we can improve the upper bound z 41.

22 Branch-and-bound method for IP We select a fractional decision variable, for instance x 2 (we may select x 1 as well), however, in practice, a number of useful heuristics are applied to make this choice. Then we add restriction either x 2 3 or x 2 4, excluding the interval (3, 4). x L 1 Optimal continuous solution L x 1

23 Branch-and-bound method for IP L 0 x 1 = 2.25 x 2 = 3.75 z = z 41 x 2 4 x 2 3 L 1 L 2 L 1 = max z = 5x 1 + 8x 2 max z = 5x 1 + 8x 2 x 1 + x 2 6 5x 1 + 9x 2 45 L x = x 1, x 2 0 x 1 + x 2 6 5x 1 + 9x 2 45 x 2 3 x 1, x 2 0 A selection of node L 1 or L 2 is heuristic. Let us consider L 1. We get: x 1 = 1.8, x 2 = 4, z = 41.

24 Branch-and-bound method for IP We select a fractional decision variable, for instance x 1. Then we add restriction either x 1 1 or x 1 2, excluding the interval (1, 2). x L 4 Optimal continuous solution L x 1

25 Branch-and-bound method for IP L 0 x 1 = 2.25 x 2 = 3.75 z = z 41 x 2 4 x 2 3 x 1 = 1.8 x 2 = 4 z = 41 L 1 L 2 x 1 2 x 1 1 L 3 = L 3 L 4 Infeasible problem max z = 5x 1 + 8x 2 max z = 5x 1 + 8x 2 x 1 + x 2 6 5x 1 + 9x 2 45 x 2 4 x 1 2 x 1, x 2 0 L 4 = x 1 + x 2 6 5x 1 + 9x 2 45 x 2 4 x 1 1 x 1, x 2 0 infeasible x 1 = 1, x 2 = 4 4 9, z =

26 Branch-and-bound method for IP We select a fractional decision variable, for instance x 2. Then we add restriction either x 2 4 or x 2 5, excluding the interval (4, 5). x 2 6 L L 4 L 5 Optimal continuous solution L x 1 Consider L 5.

27 Branch-and-bound method for IP L 0 x 1 = 2.25 x 2 = 3.75 z = z 41 x 2 4 x 2 3 x 1 = 1.8 x 2 = 4 z = 41 L 1 L 2 x 1 2 x 1 1 L 3 L 4 Infeasible problem x 1 = 1 x 2 = z = at most 10% from optimum x 2 4 lower bound x 2 5 x 1 = 1 L 5 L 6 x 2 = 4 z = 37 z 37

28 Branch-and-bound method for IP L 0 x 1 = 2.25 x 2 = 3.75 z = z 41 x 2 4 x 2 3 x 1 = 1.8 x 2 = 4 z = 41 L 1 L 2 x 1 2 x 1 1 L 3 L 4 infeasible problem x 1 = 1 x 2 = z = x 2 4 x 2 5 at most 2.5% from optimum! lower bound L 5 L 6 x 1 = 1 x 2 = 4 z = 37 z 37 x 1 = 0 x 2 = 5 z = 40 z 40

29 Branch-and-bound method for IP L 0 x 1 = 2.25 x 2 = 3.75 z = z 41 x 2 4 x 2 3 L 1 L 2 x 1 = 1.8 x 2 = 4 z = 41 x 1 = 3 x 2 = 3 z = 39 x 1 2 x 1 1 L 3 L 4 Infeasible problem x 1 = 1 x 2 = z = x 2 4 optimum! x 2 5 x 1 = 1 L 5 L 6 x 2 = 4 z = 37 z 37 x 1 = 0 x 2 = 5 z = 40 z 40

30 Branch-and-bound method for IP x 2 6 L L 4 L 5 Optimal continuous solution Optimal integer solution L x 1

31 Can the upper bound be improved? L 0 x 1 = 2.25 x 2 = 3.75 z = z 41 x 2 4 x 2 3 x 1 = 1.8 x 2 = 4 z = 41 L 1 L 2 x 1 = 3 x 2 = 3 z = 39 x 1 2 x 1 1 L 3 L 4 Infeasible problem x 1 = 1 x 2 = z = Assume that in our example we consider node L 2 prior L 5 or L 6. Thus an optimal solution is in either L 2 or L 4. Hence z is bounded by Since the 9 objective function coefficients are integral, we can improve upper bound to 40.

32 Summary Assume that we have already considered node L j. Then L j need not be subdivided if: LP over L j is infeasible, an optimal solution of LP in L j is integer, the value of the objective function of LP over L j is not greater than an actual lower bound. Remarks: Relaxations of LP are solved effectively. There are no general methods of selecting decision variable. There are no general methods of selecting node after branching.

33 When can we solve IP by solving LP? IP problem LP problem max c T x Ax = b x 0 and integer max c T x Ax = b x 0 Assume that A and b are integer. When has an optimal solution to LP integer components?

34 When has an optimal solution to LP integer components? Basic solution is [ x Ax = b [B, P] B x P x B = B 1 b, x P = 0. ] = b. If B 1 is an integer matrix, then then x B is integer.

35 Unimodularity A square integer matrix B is called unimodular if its determinant det(b) = ±1 An integer matrix A is called totally unimodular if every square, nonsingular submatrix of A is unimodular. x B = B 1 b = 1 det(b) Badj b, where B adj is the adjoint of B. Hence if B is unimodular and b is integer, x B is integer. Theorem If A is called totally unimodular, then all the vertices of {x : Ax = b, x 0} are integer for any integer vector b.

36 Unimodularity Theorem Let A be a matrix whose elements a ij are equal to 0, 1 or -1. Then A is totally unimodular if Each column contains at most two nonzero elements. The rows can be partitioned into two subsets Q 1 i Q 2 such that: if column contains two nonzero elements with the same signs, then the rows corresponding to these elements belong to the different subsets, if column contains two nonzero elements with the different signs, then the rows corresponding to these elements belong to the same subset. Corollary The node-arc incidence matrix of a directed network is totally unimodular.

37 The minimum cost flow problem Consider the problem of shipment of a commodity through a network in order to satisfy demands at certain nodes V 3 from available supplies at other nodes V 1. For each i V 1 supply a i is given, for each i V 3 demand b i is given. Each arc (i, j) has an associated cost c ij that denotes the cost per unit flow on that arc. A capacity u ij is also associated with each arc (i, j) that denotes the maximum amount that can flow on the arc. The problem consists in finding a least cost flow. Given a network G = (V, A), V = {1,..., n}. The set of nodes V is partitioned into V 1 (sources - supply nodes), V 2 (transshipment nodes), V 3 (sinks - demand nodes). For every i V the following sets are defined S(i) = {j (i, j) A} and P(i) = {j (j, i) A} c ij x ij min i V j S(i) x ij j S(i) j P(i) x ji 0 x ij u ij, (i, j) A. a i i V 1, = 0 i V 2, b i i V 3, the constraints is totally unimodular

38 The transportation problem (with capacities) It is sufficient to set V 2 =, P(i) = for i V 1, S(i) = for i V 3. c ij x ij min i V 1 j S(i) x ij a i, i V 1, j S(i) j P(i) x ji b i, i V 3, 0 x ij u ij, (i, j) A.

39 The assignment problem It is sufficient to set V 2 =, P(i) = for i V 1, S(i) = for i V 3. It is a particular case of the transportation problem: u ij = for (i, j) E, a i = 1 for i V 1, b i = 1 for i V 3 and V 1 = V 3. j S(i) j P(i) min i V 1 j S(i) c ij x ij x ij = 1, i V 1, x ji = 1, i V 3, x ij 0, (i, j) A.

40 The shortest path problem It is sufficient to set V 1 = {1}, V 3 = {m}, a 1 = 1, b m = 1, u ij = 1 for (i, j) A. j S(i) min i V x ij j P(i) x ji j S(i) c ij x ij = 1 i = 1, = 0 i V 2, 1 i i = m, 0 x ij 1. (i, j) A.