Linear Programming. (Com S 477/577 Notes) Yan-Bin Jia. Nov 28, 2017

Size: px

Start display at page:

Download "Linear Programming. (Com S 477/577 Notes) Yan-Bin Jia. Nov 28, 2017"

Asher Greene
6 years ago
Views:

1 Linear Programming (Com S 4/ Notes) Yan-Bin Jia Nov 8, Introduction Many problems can be formulated as maximizing or minimizing an objective in the form of a linear function given a set of linear constraints on the resources Linear programming has long proved its merit as a significant model of numerous problems in allocation, transportation, scheduling, and economics It has also found applications in many engineering fields from manufacturing to robotics The continuing growing applications demonstrate the importance of linear programming as a general framework for problem formulation Let us begin with some classic examples that have natural linear programming formulations Example (The diet problem) Polly wonders how much money she must spend on food in order to get all the energy (, kcal), protein ( g), and calcium (8 mg) that she needs every day She chooses six foods that seem to be cheap sources of the nutrients and collects her data in the following table of nutritive value per serving Energy Protein Calcium Price per serving Food Serving size (kcal) (g) (mg) (cents) Oat meal 8 g 4 3 Chicken g 3 4 Eggs large Whole milk 3 cc Cherry pie g 4 4 Pork with beans 6 g She also decides to impose servings-per-day limits on all foods Servings at most per day Oatmeal 4 Chicken 3 Eggs Milk 8 Cherry pie Pork with beans from [][pp 3 ]

2 To design the most economical menu, she speculates about some as yet unspecified menu, consisting of x servings of oatmeal, x servings of chicken, and so on She wants to find numbers x,x,,x 6 that satisfy her self-imposed servings-per-day limits, meet the requirements for energy, protein, calcium, and yet minimize the cost This diet problem can be nicely formulated as follows min 3x +4x +3x 3 +9x 4 +x +9x 6 subject to x 4 x 3 x 3 x 4 8 x x 6 and x +x +6x 3 +6x 4 +4x +6x 6 4x +3x +3x 3 +8x 4 +4x +4x 6 x +x +4x 3 +8x 4 +x +8x 6 8 Example (The transportation problem) Quantities a,a,,a m, respectively, of a certain product are to be shipped from each of m locations and received in amounts b,b,,b n, respectively, at each of n destinations We assume, of course, the amount shipped is equal to the amount received, ie, m i= a i = n j= b j Associated with the shipping of a unit of product from origin i to destination j is a unit shipping cost c ij It is desired to determine the amounts x ij to be shipped between each origin-destination pair so as to satisfy the shipping requirements and minimize the total cost of transportation We can also formulate this problem as a linear program: min i,j c ijx ij subject to n j= x ij = a i m i= x ij = b j x ij for i =,,,m for j =,,,n for i =,,,m and j =,,,n Standard Form of Linear Programming Any linear program can be transformed into the following standard form min c x +c x + +c n x n subject to a x +a x + +a n x n = b a m x +a m x + +a mn x n = b m and x,x,,x n or, more compactly, min cx subject to Ax = b and x

3 In the above a i s, b i s, and c i s are real constants and x i s are real numbers such that x ) b x =, A = (a ij, b =, c = (c,,c n ) m n x n Without loss of generality, wealways assumethat b i for all i We also assumethat rank(a) = m from now on; otherwise rank(a) < m, either some equations in Ax = b are redundant and can be eliminated or the system has no solution Below we use three examples to describe how to transform other forms of linear programming into the standard form Example 3 (slack variables) Consider the problem b m min c x +c x + +c n x n subject to a x +a x + +a n x n b a m x +a m x + +a mn x n b m and x,x,,x n Introducing new positive variables y,,y m, called slack variables, we can convert the inequalities to equations: min c x +c x + +c n x n subject to a x +a x + +a n x n +y = b a m x +a m x + +a mn x n and x,x,,x n y,y,,y m +y m = b m The corresponding matrix becomes (A,I m ), where I m is the m m identity matrix Example 4 (surplus variables) If the linear inequalities are in the form of a i x +a i x + +a in x n b i We can introduce surplus variables y i to change the inequalities to their equivalent forms a i x +a i x + +a in x n y i = b i Example (free variables) When one or more variables can taken on any real values, the problem can be transformed to the standard form by applying either of two additional simple techniques described below Suppose x is free to take on any real value The first technique writes x = u v and requires u and v Then substitutes u v everywhere in the linear program The problem is then expressed in terms of n+ variables u,v,x,,x n Note that a redundancy is introduced since a constant added to both u and v does not change x Nevertheless, this will not hinder the simplex method introduced later 3

4 The second technique eliminates x together with one of the constraint equations More specifically, if there is a constant which has nonzero coefficient for x, say, a i x +a i x + +a in x n = b i, where a i Then this expression is substituted for x everywhere in the linear program As a result, there remains m constraints and n variables 3 Fundamental Theorem of Linear Programming Consider the system of equations Ax = b, where A is an m n matrix, x is n, and b is m As before, we can assume rank(a) = m For notational simplicity, we also assume that the first m columns of A are linearly independent Denote by B the m m matrix formed by these columns Then B is nonsingular and the equation Bx B = b has a unique solution x B, an m-vector Thus x = (x B,) T is a solution to Ax = b This solution is called a basic solution with respect to B The components of x associated with columns of B are called basic variables, which constitute a basis Note that B can be made up of any m independent columns, say, columns k,,k m of A The basic variables are accordingly x k,,x km A vector x satisfying Ax = b x is said to be feasible for these constraints A feasible solution that is also basic is said to be a basic feasible solution Theorem Given a linear program in the standard form min cx subject to Ax = b x where A is an m n matrix of rank m, the following statements hold: if there is a feasible solution, there is a basic feasible solution if there is an optimal feasible solution, there is an optimal basic feasible solution For a proof of the above theorem, we refer to [, pp 9 ] The theorem reduces the task of solving ( an LP problem to that of searching over basic feasible solutions Since there are at most n ) m basic solutions, the fundamental theorem yields an obvious but terribly inefficient finite search technique But there is a much more efficient procedure called the simplex method 4

5 4 The Simplex Method The idea of the simplex method, developed by G B Dantzig in 94, is to proceed from one basic feasible solution of the constraint set of a problem in standard form to another, in such a way as to continually decrease the value of the objective function until a minimum is reached It is efficient in moving basic solutions to construct the minimum To obtain a firm grasp of how simplex procedure works may be a little too involved at this moment So let us work through a simple example first and get a basic picture of what is going on inside the procedure Then we will see how to deal with issues such as pivoting, moving among basic feasible solutions, determining a minimum feasible solution for a general problem Example 6 max 3x +x +3x 3 subject to x +x +x 3 x +x +3x 3 x +x +x 3 6 x,x,x 3 First, we transform the problem into standard form by introducing three non-negative slack variables x 4,x,x 6 and changing the maximization problem to a minimization problem min 3x x 3x 3 subject to x +x +x 3 +x 4 = x +x +3x 3 +x = x +x +x 3 +x 6 = 6 x,x,,x 6 It is customary to represent the system by its corresponding array of coefficients, or tableau, a a a 3 a 4 a a 6 b 3 6 c 3 3 where a,,a 6 are columns of the coefficient matrix The cost function has its coefficients in the last row of the tableau The tableau corresponds to the basic feasible solution x x x 3 x 4 = x x 6 6 of the transformed problem, with cost and variables x 4,x,x 6 in the basis Now we can increase either of x,x,x 3 to decrease the cost since their corresponding entries in the last row of the tableau, which corresponds to the coefficients of the cost function, are all negative: 3,, 3 But how much can we increase, say, x, without violating the constraint that all x i must be nonnegative? This is determined by { b min, b, b } { 3 = min a a a 3,, 6 } =

6 The corresponding entry a ( below) in the tableau is called a pivot Similarly, we find two otherpivots and 3 that determine the maximum possible increase on x and x 3, respectively We can choose any of these three pivots above to bring x,x or x 3 into the basis to replace x 4 or x (Note that their corresponding columns have s appearing in the same rows with the pivots) Suppose we choose a as the pivot to bring x into the basis and x 4 out We do the following two steps to complete the pivoting: Divide the first row by a (which happens to be ) Subtract multiples of the (normalized) first row from other rows in the tableau to make all entries in the second column but a zero So we have Essentially, we substituted 3 x = x x 3 x 4 into every other constraint equation as well as into the cost function, which now becomes x x 3 +x 4 New form of the cost function is read out from the last row in the tableau above, except the last entry is negated We have thus moved to a new basic solution and decreased the cost from to Next we pivot on in the last tableau and obtain The basic solution becomes 3 6

7 and the new cost function is x 3x 4 +x 4 with the new cost 4 At this point we may pivot in either the first or the fourth column Let us select in previous tableau as the pivot, which leads to a new tableau So the basic solution becomes And the new cost function is x + 6 x x with cost In performing this pivoting step, the first row is divided by first before its multiples get subtracted from other rows The last row of the tableau now has no negative elements We can no longer decrease the objective function by increasingthe valueofx,x 4,x Thus(,, 8,,,4)T is the optimalsolution ofthe transformed problem with the objective value of The original maximization problem has optimal value achieved at x =, x =, and x 3 = 8 From Example 6 we can see that by continual pivoting, the simplex algorithm moves among a sequence of equivalent forms(under the equality constraints after the introduction of slack variables) of cost functions: () 3x x 3x 3, () x x 3 +x 4, (3) x 3x 4 +x 4, (4) x + 6 x x Each move decreases the cost The algorithm stops when all coefficients in the cost function are nonnegative, in which case the optimal basic feasible solution has been found References [] D G Luenberger Linear and Nonlinear Programming Addison-Wesley, nd edition, 984 [] V Chvátal Linear Programming W H Freeman and Company, 983

Generality of Languages

Increasing generality Decreasing efficiency Generality of Languages Linear Programs Reduction Algorithm Simplex Algorithm, Interior Point Algorithms Min Cost Flow Algorithm Klein s algorithm Reduction