Linear programming. Saad Mneimneh. maximize x 1 + x 2 subject to 4x 1 x 2 8 2x 1 + x x 1 2x 2 2

Size: px

Start display at page:

Download "Linear programming. Saad Mneimneh. maximize x 1 + x 2 subject to 4x 1 x 2 8 2x 1 + x x 1 2x 2 2"

Sheryl Felicia Harris
5 years ago
Views:

1 Linear programming Saad Mneimneh 1 Introduction Consider the following problem: x 1 + x x 1 x 8 x 1 + x 10 5x 1 x x 1, x 0 The feasible solution is a point (x 1, x ) that lies within the region defined by the lines x 1 x = 8, x 1 + x = 10, 5x 1 x =, x 1 = 0, and x = 0. We need such a point that will x 1 + x = k. Note that x = k x 1 is a line with slope 1 that intersects x 1 = 0 at k. Therefore, as illustrated below, the point (, ) is the optimal solution. In general, it is not possible to solve geometrically, especially for higher dimensions. Instead, we will develop a systematic way for finding the optimal solution algebraically. Assume, in general, we are given the following Linear Program. c T x Ax b

2 where A is an m n matrix, and b and c are both vectors of size m and n respectively. The matrix notation represents m constraints, for i = 1... m, of the form: n a ij x j b i j=1 The goal is to find a vector x of size n that satisfies the constraints and s the objective function c T x. Note that it is always possible to convert a minimization to a maximization, and a constraint to a constraint. Furthermore, if for some x i, the constraint x i 0 is not present, we can replace x i by the difference x i x i, where x i The slack form > 0 and x i > 0. We will transform the above linear program into a more useful form, known as the slack form. We replace the i th constraint: a ij x j b i by j a ij x j + s i = b i j s i 0 where s i 0 is a newly introduced slack variable. After introducing all m slack variables, we make sure that b 0, i.e. b i 0 for all i. This can be achieved, if b i < 0, by multiplying the i th constraint by 1 on both sides. a ij x j s i = b i Our slack form now becomes: j c T x Ax = b (b > 0) where A is an m n matrix (n m), b 0 is a vector of size m, and c is a vector of size n. We seek a solution x. Assume that A can be divided into two parts, a square m m matrix B, and another (n m) m matrix N. Therefore, A = [B, N]. Assume further that B has an inverse B 1. Accordingly, let x T be [x T B, xt N ] with m basic variable x B and n m non-basic variables x N. Set x N = 0 and x B = B 1 b. Therefore, x is a feasible solution because Ax = [B, N]x = Bx B + Nx N = BB 1 b + 0 = b. Using this initial feasible (but not necessarily optimal) solution, we introduce the simplex algorithm below.

3 3 The simplex algorithm Let A j be the j th column of A (the column corresponding to variable x j ). Simplex while x j x N s.t. c j c T B B 1 A j > 0 increase x j until some x i x B is zero switch the roles of x i and x j update B (N, x B, and x N too) Note that increasing x j x N changes x B too. The new objective function becomes: Bx B + A j x j = b x B = B 1 (b A j x j ) c T x = c T Bx B + c j x j = c T BB 1 (b A j x j ) + c j x j = c T BB 1 b + (c j c T BB 1 A j )x j Now observe that c j c T B B 1 A j is the coefficient of x j in the objective function, and c T B B 1 b is the value of the objective function prior to increasing x j. Therefore, since c j c T B B 1 A j > 0, increasing x j increases the objective. But how much can we increase x j? We still require x B 0; therefore, B 1 (b A j x j ) 0 B 1 b B 1 A j x j 0. If B 1 A j 0 (all components of B 1 A j are negative or zero), then we can increase x j as much as we want. In this case, the linear program in unbounded. Otherwise, x j can increase until some element in x B becomes 0. When the condition of the while loop is false, i.e. c j c T B B 1 A j 0 for all x j x N, the solution is optimal. We will later prove this fact using the concept of duality. Example of simplex We will show an example of using simplex. While matrix notation is concise for the purpose of illustration, we will not follow it here. Equivalently, we will choose a non-basic variable with positive coefficient in the objective function, and increase it. We obtain the slack form: 3x 1 + x + x 3 x 1 + x + 3x 3 30 x 1 + x + 5x 3 x 1 + x + x 3 3 x 1, x, x 3 0

4 3x 1 + x + x 3 x 1 + x + 3x 3 + x = 30 x 1 + x + 5x 3 + x 5 = x 1 + x + x 3 + x = 3 x 1, x, x 3, x, x 5, x 0 Consider the initial solution given by x T B = [x, x 5, x ] (the slack variables). 3x 1 + x + x 3 x = 30 x 1 x 3x 3 x 5 = x 1 x 5x 3 x = 3 x 1 x x 3 x 1, x, x 3, x, x 5, x 0 Therefore, x = [0, 0, 0, 30,, 3] is our initial feasible solution (Section 5 deals with the problem when such an initial solution cannot be found). We now repeatedly identify a variable x j x N with a positive coefficient in the objective function and increase it until an x i x B becomes zero. We will always rewrite the objective function in terms of non-basic variables so that this identification is easy to make. We can increase x 1 up to 9 which will make x = 0. We exchange x 1 and x. 7 + x + x 3 3 x x = 1 3 x 5 x 3 + x x 5 = 3 x x 3 + x x 1 = 9 x x3 x x 1, x, x 3, x, x 5, x 0 Next, we can choose, say, x 3. We can increase x 3 up to 3/ which will make x 5 = 0. Updating, we get: x 1 x5 x x 1 = 33 x 1 + x5 8 5 x 1 x 3 = 3 3 x 8 x5 + x 8 x = x x5 x 1, x, x 3, x, x 5, x 0 8 x 1 The only choice we have now is x. Increasing x to makes x 3 = 0. 8 x 3 x 5 x 3 x 1 = 8 + x 3 + x 5 x 3 x = 8 x3 3 x5 x = 18 x3 + x5 x 1, x, x 3, x, x 5, x x 3 We stop with a value of 8 for the objective function. This is the optimal solution for this linear program since all non-basic variables have negative coefficients in the objective function. Therefore, the solution is x T = [x 1, x, x 3, x, x 5, x ] = [8,, 0, 18, 0, 0].

5 5 Initial feasible solution The slack variables may not offer a feasible solution as illustrated in the above example. For instance, consider the following linear program: x 1 x x 1 x x 1 5x x 1, x 0 Converting to slack form (and making b 0), we have: x 1 x x 1 x + x 3 = x 1 + 5x x = x 1, x, x 3, x 0 The initial solution is then given by x T = [x 1, x, x 3, x ] = [0, 0,, ] which is not feasible (because x < 0). Therefore, we need a general approach by which we either find an initial feasible solution, or determine that the linear program is not feasible. Consider the following auxiliary linear program: 1 T y Ax + Iy = b x, y 0 where 1 T = [1,..., 1] and I is the identity matrix. Observation: the original linear program is feasible if and only if the auxiliary linear program has optimal solution y = 0. Proof: on one hand, if the original linear program is feasible, then there exists an x such that Ax = b. Therefore, y = 0 is feasible for the auxiliary linear program, and since the objective is to 1 T y and y 0, y = 0 is optimal. On the other hand, if the y = 0 is the optimal solution for the auxiliary linear program, then we have found an x such that Ax = b, which is feasible for the original linear program. Therefore, we solve for the auxiliary linear program. If the optimal solution is y = 0, then we use x as a feasible solution for the original linear program. Otherwise, the original linear program is not feasible. We still need an initial feasible solution for the auxiliary linear program: x = 0 and y = b is one (since b 0). Running time of simplex If we think of the basic variables ( x B as ) the state of the simplex algorithm, m + n then simplex can be in at most states. This is because we are not m changing the equations, but simply deciding which variables (the basic ones)

6 appear ( on) the left hand side. Therefore, simplex either terminates after at most m + n iterations, or it cycles. This, however, does not mean that the m objective function will increase indefinitely, as it is possible for an iteration not to increase the objective. Here s an example. x 1 + x + x 3 x = 8 x 1 x x 5 = x x 3 Suppose we choose x 1 and increase it to 8 to make x = 0, we get: 8 + x 3 x x 1 = 8 x x x 5 = x x 3 The only choice now is x 3, but we can only increase it up to 0; otherwise, x 5 becomes negative. Doing so will not increase the objective function, but will change the state. 8 + x x x 5 x 1 = 8 x x x 3 = x x 5 We can continue by choosing x now. Cycling is possible but rare. It is avoidable by making more careful choices about which non-basic variable becomes basic. We are not going to explore this any further. 7 Duality and optimality We now prove that when simplex terminates we have the optimal solution. Given the linear program (in slack form here): c T x Ax = b construct its dual: minimize y T b y T A c T y 0 Note that y T A c T and, since, y T Ax c T x. But y T Ax = y T (Ax) = y T b. Therefore, c T x y T b. This means that whenever c T x = y T b, both x and

7 y are optimal solutions for their respective programs. We will prove that this is in deed the case. When simplex terminates, the solution is given by x N = 0, x B = B 1 b, and c T x = c T B B 1 b. Moreover, c j c T B B 1 A j 0 for all x j x N. Let y T = c T B B 1. It is obvious that y T b = c T x. We will show that y is feasible for the dual linear program. y T B = c T BB 1 B = c T B Therefore, y T A c T. y T A j = c T BB 1 A j c j for all x j x N

3 The Simplex Method. 3.1 Basic Solutions

3 The Simplex Method. 3.1 Basic Solutions 3 The Simplex Method 3.1 Basic Solutions In the LP of Example 2.3, the optimal solution happened to lie at an extreme point of the feasible set. This was not a coincidence. Consider an LP in general form,