MATH 4211/6211 Optimization Linear Programming

Transcription

1 MATH 4211/6211 Optimization Linear Programming Xiaojing Ye Department of Mathematics & Statistics Georgia State University Xiaojing Ye, Math & Stat, Georgia State University 0

2 The standard form of a Linear Program (LP) minimize subject to c T x Ax = b x 0 where x R n is the unknown variable; A R m n is given, where m < n and rank(a) = m; b R m is given. Xiaojing Ye, Math & Stat, Georgia State University 1

3 Other forms of LP can be converted to the standard form. For example, suppose an LP is given as maximize subject to c T x Ax b x 0 Then we can rewrite it as an equivalent standard form minimize subject to ( c) T x Ax + y = b x, y 0 where [x; y] R n+m is the variable to solve for now, and y is called the slack variable. If [x ; y ] T is a solution of the new problem, then x is a solution of the original problem. Xiaojing Ye, Math & Stat, Georgia State University 2

4 Example. Convert the LP below into the standard form maximize x 2 x 1 subject to 3x 1 = x 2 5 x 2 2 x 1 0 x 1 := x 1, then x 1 0 x 1 0 x 2 = u v where u, v 0 x u v 2 Introduce slack variables x 3, x 4 0 such that u v + x 3 = 2 and u v x 4 = 2 Xiaojing Ye, Math & Stat, Georgia State University 3

5 Example (cont). Now we obtain an equivalent problem: minimize x 1 u + v subject to 3x 1 + u v = 5 u v + x 3 = 2 u v x 4 = 2 x 1, u, v, x 3, x 4 0 Note that there are 5 variables now. Xiaojing Ye, Math & Stat, Georgia State University 4

6 Example. Suppose a factory wants to manufacture 4 products with the cost and budget (availability), as well as their profits, given in the table below. The goal is to maximize total profit. P1 P2 P3 P4 availability man weeks kg of material A boxes of material B profit Solution. Let x i be the quantity to manufacture product i, then the LP is maximize 6x 1 + 4x 2 + 7x 3 + 5x 4 subject to x 1 + 2x 2 + x 3 + 2x x 1 + 5x 2 + 3x 3 + 2x x 1 + 4x 2 + 9x x 3 75 x 1, x 2, x 3, x 4 0 Exercise: convert this into the standard form of LP. Xiaojing Ye, Math & Stat, Georgia State University 5

7 Geometric interpretation of the feasible set Ω = {x R n : Ax b, x 0}: Let a T i be the ith row of A for i = 1,..., m. Then a T i x b i is a half space. Hence Ax b is the intersection of m half spaces, which is a polyhedra. So is Ω. The objective c T x is a plane with slope defined on Ω, and the optimal point x is the point in Ω that has the minimum value c T x. Xiaojing Ye, Math & Stat, Georgia State University 6

8 Basic solutions of linear system Ax = b. Suppose A = [B D] where B R m m has rank(b) = m. Denote x B = B 1 b R m. Then x = is called a basic solution of Ax = b. [ ] xb 0 R n Xiaojing Ye, Math & Stat, Georgia State University 7

9 Definition. We introduce the following terms: x B R m : basic variables If x B has zero component(s), then x is called a degenerate basic solution B: basic columns of A. Note that in practice there are up to ( ) n m ways to choose the basic columns. x is called a feasible point if x Ω := {x R n : Ax = b, x 0}. x Ω is called an optimal (feasible) solution if c T x c T x for all x Ω. Xiaojing Ye, Math & Stat, Georgia State University 8

10 Properties of basic solutions Theorem (Fundamental Theorem of LP). For an LP with nonempty feasible set Ω, the following statements hold: (a) A basic feasible point exists. (b) If LP has solution, then there exists an optimal basic feasible solution. Xiaojing Ye, Math & Stat, Georgia State University 9

11 Proof. Part (a). Let x Ω (since Ω ), i.e., Ax = b and x 0. WLOG, assume x 1,..., x p > 0 and x p+1,..., x n = 0. Then where A = [a 1,..., a p,..., a n ]. Ax = x 1 a x p a p = b If a 1,..., a p are linearly independent, then p m and x is a basic feasible point. If a 1,..., a p are linearly dependent, then y 1,..., y p R not all zero such that Ay = y 1 a y p a p = 0 where y = [y 1,..., y p, 0,..., 0] T R n (WLOG we assume at least one y i > 0 otherwise take y = y), then we know A(x εy) = b for all ε. Xiaojing Ye, Math & Stat, Georgia State University 10

12 Proof (cont). Choose ε = min{x i /y i : y i > 0}, then x εy Ω and has only p 1 nonzero components. Repeat this until we find the basic columns of A and its corresponding basic feasible point. Xiaojing Ye, Math & Stat, Georgia State University 11

13 Proof (cont). Part (b). Let x Ω be optimal, and again WLOG assume x 1,..., x p > 0. If a 1,..., a p are linearly independent, then p m and x is a basic feasible solution. If a 1,..., a p are linearly dependent, then with the same argument as in (a), y 1,..., y p R not all zero such that Ay = y 1 a y p a p = 0 If c T y 0 (say > 0), then by choosing ε min{ x i /y i : y i 0}, we have x εy Ω and c T (x εy) = c T x εc T y < c T x, which contradicts to the optimality of x. Hence c T y = 0. So we can choose ε in the same way as in (a) to get x εy which has p 1 nonzeros. Repeat to obtain a basic feasible solution. Xiaojing Ye, Math & Stat, Georgia State University 12

14 Theorem. Consider an LP with nonempty Ω, then x is an extreme point of Ω iff x is a basic feasible point. In theory, we only need to examine the basic feasible points of Ω and find the optimal basic feasible solution of the LP. However, there could be a large amount of extreme points to examine. Xiaojing Ye, Math & Stat, Georgia State University 13

15 Now we study the Simplex Method designed for LP. Recall basic row operations to linear system Ax = b: interchange two rows multiply one row by a real nonzero scalar Adding a scalar multiple of one row to another row Each of these operations corresponds to an invertible matrix E R m premultiplying to A. Xiaojing Ye, Math & Stat, Georgia State University 14

16 Also recall that, to solve a linear system Ax = b, we first form the augmented matrix [A, b], then apply a series of row operations, say E 1,..., E t, to the matrix to obtain E t E 1 [A, b] = [I, D, B 1 b] where D is such that E t... E 1 A = [I, D] and B = (E t E 1 ) 1 R m m is invertible. Then a particular solution of Ax = b is x = [B 1 b; 0] R n. In addition, note that [ Dx D ; x D ] R n is a solution of [I, D]x = 0 for any x D R n m. Therefore, any solution of Ax = b can be written for some x D R n m as [ B x = 1 ] [ ] b DxD + 0 x D Xiaojing Ye, Math & Stat, Georgia State University 15

17 Suppose we have applied basic row operations and converted Ax = b to [I, Y ]x = y 0 (called the canonical form) where Y = y 1 m+1 y 1 n..... y m m+1 y m n R m (n m) y 0 = y 10. y m0 R m Note that the column y q = [y 1q ;... ; y mq ] R m gives the coefficients to represent a q using a 1,..., a m, where a i is the ith column of A: a q = y 1q a y mq a m = [a 1,..., a m ]y q Now we are using a 1,..., a m as basic columns, and x = [B 1 b; 0] is the corresponding basic solution to Ax = b. If B 1 b 0, then x is a basic feasible point of the LP. Xiaojing Ye, Math & Stat, Georgia State University 16

18 Now we want to move to another basic feasible point. To this end, we need to exchange one of a 1,..., a m with another a q (q > m) to form the new basic columns, find the corresponding basic solution x of Ax = b, making sure that x 0. Exchanging basic columns p and q where p m < q is simple: just apply basic row operations [I, Y ] so that the qth column becomes e p. Here e p R n is the vector with 1 as the pth component and 0 elsewhere. However, which a q we should choose to enter the basic columns, and which a p to leave? Xiaojing Ye, Math & Stat, Georgia State University 17

19 Since x = [y 10 ;... ; y m0 ; 0;... ; 0] R n + to Ax = b, we know is a basic solution corresponding y 10 a y m0 a m = b Suppose we decide to let a q enter the basic columns, then due to a q = y 1q a y mq a m = [a 1,..., a m ]y q we know for any ε 0 there is (y 10 εy 1q )a (y m0 εy mq )a m + εa q = b Xiaojing Ye, Math & Stat, Georgia State University 18

20 Note that y q have positive and nonpositive components (if all are nonpositive then the problem is unbounded), then for ε > 0 gradually increases from 0, one of the coefficients (say p) first becomes 0. We will choose a p to leave the basic columns. More precisely, we choose ε = min i {y i0 /y iq : y iq > 0}, and obtain the basic columns a 1,..., â p,..., a m, a q Xiaojing Ye, Math & Stat, Georgia State University 19

21 Now we consider which a q to enter the basic column. Ideally, we should choose a q such that c T x decreases the most. Recall that the current point is x = [y 0 ; 0] R n. If we choose a q to enter, then the objective function becomes c T [ y0 εy q 0 where z i = c T [1:m] y i for i = 0, 1,..., m. ] + εe q = z 0 + ε[c q (c 1 y 1q + + c m y mq )] = z 0 + (c q z q )ε Xiaojing Ye, Math & Stat, Georgia State University 20

22 If c q z q < 0 for some q, then choosing a q to enter can further reduce the objective function since ε > 0. If there are more than one such q, then choose the one with smallest index or the one with smallest c q z q. If c q z q 0 for all q, then the optimal basic point is reached. These answered which a q to enter, and when the simplex method should stop. Xiaojing Ye, Math & Stat, Georgia State University 21

23 To simplify the process, we consider the matrix form of the simplex method. Given the standard form of an LP minimize subject to c T x Ax = b x 0 we form the tableau of the problem as the matrix [ ] [ ] A b B D b c T = 0 c T B ct D 0 R (m+1) (n+1) Note that the tableau contains all information of the LP. Xiaojing Ye, Math & Stat, Georgia State University 22

24 Premultiplying the matrix [ B T 1 [ B 1 ] 0 0 T 1 ] [ B D b c T B ct D 0 Further premultiplying the matrix [ ] [ I 0 I B 1 D B 1 b c T B 1 c T B c T D 0 to the tableau, we obtain ] = [ I 0 c T B 1 ] = [ I B 1 D B 1 b c T B c T D 0 ] yields [ I B 1 D B 1 ] b 0 T c T D ct B B 1 D c T B B 1 b ] Xiaojing Ye, Math & Stat, Georgia State University 23

25 It is easy to check that [B 1 b; 0] R n is the basic solution c T D ct B B 1 D R n m contains the reduced cost coefficients, i.e., z q c q for q = m + 1,..., n c T B B 1 b R is the objective function Xiaojing Ye, Math & Stat, Georgia State University 24

26 According to the discussion above, the simplex method executes the following actions in order: find the index, say q, of the most negative component among c T D ct B B 1 D R n m ; find p = arg min i {y i0 /y iq : y iq > 0}; pivot the tableau about the (p, q) entry by basic row operations so that the column becomes 0 except the (p, q) entry which is 1. [1 : m, n + 1] gives the current basic solution (basic feasible point), the nonzeros in [m + 1, 1 : n] are the reduced cost coefficients, and the (m + 1, n + 1) entry is the negative of objective function. Xiaojing Ye, Math & Stat, Georgia State University 25

27 Example. Consider the following linear programming problem: maximize 7x 1 + 6x 2 subject to 2x 1 + x 2 3 Solve this LP using the simplex method. x 1 + 4x 2 4 x 1, x 2 0 Solution. We first convert this LP to the standard form: minimize 7x 1 6x 2 subject to 2x 1 + x 2 + x 3 = 3 x 1 + 4x 2 + x 4 = 4 x 1, x 2, x 3, x 4 0 Xiaojing Ye, Math & Stat, Georgia State University 26

28 Solution (cont). The tableau of this LP is where the upper left 2 4 submatrix corresponds to A = [a 1,..., a 4 ], c = [ 7; 6, 0; 0], and b = [3; 4]. Note that A is already in the canonical form: a 3, a 4 are the basic columns, and x = [0; 0; 3; 4] is the basic solution of Ax = b. Since 7 is the most negative term, we will let a 1 enter the basic column. Comparing 3/2 and 4/1, we see the former is smaller and hence decide to let a 3 leave the basic column. Xiaojing Ye, Math & Stat, Georgia State University 27

29 Now pivoting about the (1, 1) entry, we obtain Now we see 5/2 is the most negative term, so we let a 2 enter the basic columns. Comparing 3/2 5/2 = 3 and 1/2 7/2 = 5/7, we let a 4 leave the basic column. Then pivoting about the (2, 2) entry, we obtain There are no negative entries in the last row, so we stop. The optimal solution is x = [ 8 7, 5 7, 0, 0] and optimal value is 86 7 of the LP in its standard form. These correspond to the optimal solution x 1 = 8/7, x 2 = 5/7, and optimal value 86/7 of the original problem. 8 Xiaojing Ye, Math & Stat, Georgia State University 28

30 Starting the Simplex Method The simplex method discussed above is referred to the Phase II which requires an initial point to be a basic feasible point. Now we consider the Phase I which finds one of such basic feasible points. To this end, we introduce artificial variables y R m and consider the following artificial problem: minimize subject to 1 T y Ax + Ey = b x 0, y 0 where 1 = [1,..., 1] T R m, E is diagonal with E ii = sign(b i ). Xiaojing Ye, Math & Stat, Georgia State University 29

31 The artificial problem minimize subject to 1 T y Ax + Ey = b x 0, y 0 has an obvious basic feasible point [x; y] = [0;... ; 0; b 1 ;... ; b m ] with which we can directly apply the Phase II simplex method. Xiaojing Ye, Math & Stat, Georgia State University 30

32 Moreover, the artificial problem in Phase I has a solution iff the original problem has nonempty feasible set Ω. In this case, the artificial problem above have an optimal objective value 0 and optimal solution [x ; y ], such that y = 0 and x is a basic feasible point of the original LP problem: If Ω is nonempty, then there exists x Ω, i.e., Ax = b and x 0. So [x; 0] solves the artificial problem in Phase I. If the artificial problem has a solution such that 0 T y = 0, then y = 0, and x satisfies Ax = b and x 0, which means x Ω. Xiaojing Ye, Math & Stat, Georgia State University 31