Chapter 5 Linear Programming (LP)

Chapter 5 Linear Programming (LP) General constrained optimization problem: minimize f(x) subject to x R n is called the constraint set or feasible set. any point x is called a feasible point We consider the case when f (x) is a linear function f (x) = c T x; x = (x 1 ; x ; :::; x n ) R n where c R n is a given constant vector, called cost coe cient. Example: R f (x) = x 1 4x + x = c T x; c = x 1 4 45 ; x = 4x 5 1 x Consider standard convex constraint set = fx : Ax = b; x 0g The notation x 0 means that each component of x = [x 1 ; :::; x n ] T must be 0 : x i 0 for all i Example: = fx : 4x 1 + x = 5; x 1 0; x 0g y 5 4 1 0 0.0 0. 0.4 0.6 0.8 1.0 1. 1.4 x 1

We can write the problem as minimize subject to c T x Ax = b x 0 This problem is called Linear programming problem, or linear programming (LP). In general, constraint sets involve equations and inequalities. Example: Example Production scheduling Woodworking shop data: minimize x 1 + x subject to x 1 x 6 x 1 0 Inputs Products input availability Table Chair labor 5 6 0 Materials 1 Production Levels x 1 x Unit price 1 5 Goal: schedule production to maximize revenue Total revenue: x 1 + 5x Production constraints Labor constraint: 5x 1 + 6x 0 Materials constraint: x 1 + x 1 Physical constraints: x 1 0; x 0

Optimization problem: (LP) in matrix form Equivalent form maximize x 1 + 5x subject to 5x 1 + 6x 0 A = maximize subject to 5 6 ; b = x 1 + x 1 x 1 0; x 0 c T x Ax b x 1 0; x 0 0 ; c = 1 1 5 minimize subject to c T x Ax b x 1 0; x 0 Example: Optimal Diet Nutrition table: Unit Price Vitamin Milk Eggs Daily Requirement 1 F 45 G 4 60 Intake x 1 x Unit cost 5 For consumers: Total cost: x 1 + 5x Dietary Constraints Vitamin F: x 1 + x 45

Vitamin G: 4x 1 + x 60 Physical constraints: x 1 0; x 0 Optimization problem: (LP) in matrix form minimize x 1 + 5x subject to x 1 + x 45 A = minimize subject to This is called Primal LP For sellers: ; b = 4 4x 1 + x 60 x 1 0; x 0 c T x Ax b x 1 0; x 0 45 ; c = 60 5 total revenue (per customer) 45 1 + 60 to compete well with milk: 1 + 4 to compete well with egg: 1 + 5 LP for seller: This is called dual LP. Maximize 45 1 + 60 Subject to 1 + 4 1 + 5 1 ; 0 Geometric method for LP problems in -D 4

Consider the production scheduling example given before: maximize x 1 + 5x subject to 5x 1 + 6x 0 x 1 + x 1 x 1 0; x 0 Level sets of the objective function: x 1 + 5x = c Level sets are parallel lines, moving up as c increases The maximum value 5 is reached when the level set "exiting" the constraint set at the corner point [0; 5] T : x 1 + 5x = 5 The minimum value 0 is reached when the level set "exiting" the constraint set at the corner point [0; 0] T : x 1 + 5x = 0 In fact, unless the level sets happen to be parallel to one of the edges of the constraint set, the solution will lie on a corner point (or vertex). Even if the level sets happen to be parallel to one of the edges of the constraint set, a corner point will be an optimal solution. It turns out that solution of an LP problem (if it exists) always lies on a vertex of the constraint set. 5

Therefore, instead of looking for candidate solutions everywhere in the constraint set, we need only focus on the vertices. Standard form LP problems LP problem in standard form: minimize subject to c T x Ax = b x 0 where A is a m n matrix, m < n; rank (A) = m b 0 Example: In standard form: NOT in standard forms: minimize x 1 + 5x x subject to x 1 + x + 4x = 4 5x 1 x + x = 15 x 1 0; x 0; x 0 maximize x 1 + 5x x subject to x 1 + x + 4x 4 5x 1 x + x 15 x 1 0; x 0 minimize x 1 + 5x x subject to x 1 + x + 4x = 4 5x 1 x + x = 15 x 1 0; x 0; x 0 6

The above is not standard because b 1 = 4 < 0: To be standard form, we require b i 0: minimize x 1 + 5x x subject to x 1 + x + 4x = 4 x 1 + 4x + 8x = 8 x 1 0; x 0; x 0 The above is not standard because the matrix A 4 has a rank of 1; not "full rank". All our analyses and algorithms will apply only to standard form LP problems. What about other variations of LP problems? Any LP problem can be converted into an equivalent standard form LP problem. How to convert from given LP problem to an equivalent problem in standard form? If problem is a maximization, simply multiply the objective function by 1 to get minimization. If A not of full rank, can remove one or more rows. If a component of b is negative, say the ith component, multiply the ith constraint by 1 to obtain a positive right-hand side. What about inequality constraints? add slack variables for " " constraints add surplus variables for " " constraint split variables for free variables Converting to standard form: Slack variables Suppose we have a " " constraint of the form: a 1 x 1 + a x + + a n x n b

We can convert the above inequality constraint into the standard equality constraint by introducing a slack variable x n+1 The above constraint is equivalent to: a 1 x 1 + a x + a n x n + x n+1 = b x n+1 0 Converting to standard form: Surplus variables Suppose we have a constraint of the form: a 1 x 1 + a x + + a n x n b We can convert the above inequality constraint into the standard equality constraint by introducing a surplus variable x n+1 The above constraint is equivalent to: a 1 x 1 + a x + + a n x n x n+1 = b x n+1 0 Converting to standard form: Nonpositive variable Suppose one of the variables (say, x 1 ) has the constraint x 1 0 We can convert the variable into the usual nonnegative variable by changing every occurrence of x 1 by its negative x 0 1 = x 1 Example: The LP problem minimize c 1 x 1 + c x + + c n x n subject to a 1 x 1 + a x + + a n x n a n+1 x n+1 = b x 1 0 is converted standard form minimize c 1 x 0 1 + c x + + c n x n subject to a 1 x 0 1 + a x + + a n x n a n+1 x n+1 = b x 0 1 0 8

Converting to standard form: Free variables Suppose one of the variables (say, x 1 ) does not have any constraint We can split x 1 into its positive part and negative part by introducing two variables u 1 0 and v 1 0 : x 1 = u 1 v 1 Example: The constraint is equivalent to a 1 x 1 + a x + + a n x n = b x 0; x 0; ; x n 0 a 1 (u 1 v 1 ) + a x + + a n x n = b u 1 0; v 1 0; x 0; x 0; ; x n 0 Converting to standard form: An example Consider the LP problem (not in standard form) maximize x 1 + 5x x subject to x 1 + x + 4x 4 (1) 5x 1 x + x 15 () x 0 x 0 We rst convert the maximization problem into minimization problem by multiplying the objective function by ( 1) Next, we introduce a slack variable x 4 in the rst constraint to convert constraint (1) into x 1 + x + 4x + x 4 = 4; x 4 0 and then multiply the equation by ( 1) to make the right-hand side positive: x 1 x 4x x 4 = 4; x 4 0 9

Next, we introduce a surplus variable x 5 to convert the second constraint () to convert it to 5x 1 x + x x 5 = 15; x 5 0 Next, we substitute x 1 with x 6 Finally, we substitute x with x in every expression x 0 in every expression The resulting standard form or minimize (x 6 x ) + 5x 0 + x subject to (x 6 x ) + x 0 4x x 4 = 4 5 (x 6 x ) + x 0 + x x 5 = 15 x 0 ; x ; x 4 ; x 5 ; x 6 ; x 0 minimize 5x 0 + x x 6 + x subject to x 0 4x x 4 + 0x 5 x 6 + x = 4 The matrix form is x 0 + x + 0x 4 x 5 5x 6 + 5x = 15 x 0 ; x ; x 4 ; x 5 ; x 6 ; x 0 minimize c T x subject to Ax = b ; x = x 0 T x x 4 x 5 x 6 x x 0 4 1 0 1 1 4 A = ; b = 1 0 1 5 5 15 Basic solutions Consider LP problem in standard form: minimize subject to c T x Ax = b x 0 where A is a m n matrix, m < n; rank (A) = m; b 0 10

The feasible points are those x that satisfy Ax = b and x 0 (i.e., have nonnegative components). For m < n, there are in nitely many points x satisfying Ax = b REVIEW linear algebra: Recall there are three types of elementary row operations 1. Interchanging two rows of the matrix;. Multiplying one row of the matrix by a (nonzero) constant;. Adding a constant multiple of one row to another row. Pivoting about (i,j) position: using above three types of row operations to make the entry at (i,j) position 1, and the rest entries in the jth column zero To solve Ax = b using elementary row operations: First, we form the augmented matrix [A; b] Using Gauss-Jordan Algorithm [A; b] can be reduced by a sequence of elementary row operations to a unique reduced Echelon form. Assume that the rst m columns of A are linearly independent pivoting about (1; 1) ; (; ) ; :::; (m; m) position consecutively this will reduce [A; b] to the reduced Echelon 1 0 0 y 1 y 1 0 1 0 y 6 4.......... 5 = [I m ; Y; y ] ; y y = 6 4. 5 0 0 1 y m Name: canonical augmented matrix y m 11

In particular, Ax = b has a solution y 1 y y x. = = y 0 m 0 6 4. 5 0 where y is the last column in the above canonical augmented matrix Basic solution: This solution x = y T ; 0 T is called a basic solution to Ax = b The rst m columns of A are called basic columns The rst m variables x 1 ; ; x m are called basic variables The last (n m) variables x m+1 ; ; x n are called nonbasic variables In general, for any solution x of Ax = b x is called a basic solution if x has at most m non-zero components the corresponding variables (of non-zero components) are called basic variables the corresponding columns in A are called basic columns that form a basis For any m basic columns of A : a k1 ; a k ; :::; a km are linearly independent, where 1 k 1 k ::: k m : Canonical augmented matrix: k 1 -th column is e 1 = [1; 0; :::; 0] T ; k -th column is e ; ::; k m -th column is e m 1 0 0 0 1 0 [A; b] = [a 1 ; a ; :::; a n ; b]! 6 4 0. 0 1 5.... 1

In this case, a k1 ; a k ; :::; a km are basic columns, variables associated with basic columns, x k1 ; x k ; :::; x km ; are basic variables. Example: A 4 = [a 1 ; a ; a ; a 4 ] = 1 4 ; b = 5 6 8 6 canonical augmented matrix is 1 4 6 1 1 0 5! 1 5 6 8 0 1 1 4 x and x 4 are basic variables a ; a 4 are basic columns a solution x x = 6 4 0 5 0 5 4 this x = [0; 5; 0; 4] T is a basic solution ( m = ) There are at most n = m n! m! (n m)! basic solutions Example Consider the equation Ax = b with 1 1 1 4 A = [a 1 ; a ; a ; a 4 ] = ; b = 1 1 1 1 1 1 4 8 A b = 1 1 1 8 #1 basic solution: x = [6; ; 0; 0] T ; basic variables x 1 ; x ; basis B = [a 1 ; a ] # basic solution: x = [0; ; 6; 0] T ; basic variables x ; x ; basis B = [a ; a ] 1

# basic solution: x = [0; 0; 0; ] T ; basic variables x ; x 4 ; basis B = [a ; a 4 ] #4 basic solution: x = [0; 0; 0; ] T ; basic variables x 1 ; x 4 ; basis B = [a 1 ; a 4 ] #5 basic solution: x = [0; 0; 0; ] T ; basic variables x ; x 4 ; basis B = [a ; a 4 ] Note that x = [; 1; 0; 1] T is a solution, but it is not basic To nd the last basic solution, we set x = x 4 = 0 the solution of form x = [x 1 ; 0; x ; 0] T for Ax = b is equivalent to 1 1 x1 8 = 1 1 x its augmented matrix 1 1 8 1 1 is inconsistent. So there are only ve basic solutions Basic Feasible Solutions (BFS) Consider the constraints of the LP problem: Ax = b; x 0 Recall de nition of basic solution If x is a basic solution to Ax = b and it also satis es x 0, we call it a basic feasible solution (BFS) Note that a basic solution x is feasible i every basic variable is 0. If at least one of the basic variables is = 0, we say that the BFS is degenerate We henceforth assume nondegeneracy (i.e., all the BFSs we consider are not degenerate) 14

This makes the derivations simpler. The degenerate case can be handled similarly. In Example above: x = [6; ; 0; 0] T is a BFS x = [0; 0; 0; ] T is a degenerate BFS x = [0; ; 6; 0] T is a basic solution, but not feasible x = [; 1; 0; 1] T is feasible, but not basic Geometric view Geometrically, a BFS corresponds to a corner point (vertex) of the constraint set Example: Consider the constraint Ax = b; x 0, where A = 1; ; ; b = [6]: Feasible set the portion of the plane x 1 + x + x = 6 in the rst octant The BFSs are: [6; 0; 0] T ; [0; ; 0] T, and [0; 0; ] T Note that the BFSs are just the vertices of the constraint set. In general, we can think (geometrically) of a BFS as a vertex of the constraint set. Optimal solutions A feasible point that minimizes the objective function is called an optimal feasible solution A BFS that is also optimal is called an optimal basic feasible solution 15

Fundamental Theorem of LP Consider an LP problem in standard form. 1. If there exists a feasible solution, then there exists a BFS;. If there exists an optimal feasible solution, then there exists an optimal BFS. Or, more informally,... If the feasible set is nonempty, then it has a vertex If the problem has a minimizer (optimal solution), then one of the vertices is a minimizer Consequences of FTLP Suppose we are given an LP problem. Assume that a solution exists. To nd the solution, it su ces to look among the set of BFSs. 16

There are only a nite number of BFSs; in fact, at most n n! = basic solutions m m! (n m)! Therefore, the FTLP allows us to transform the original problem (in- nite number of feasible points) to a problem over a nite number of points However, the total number of BFSs may of course be very large. 50 For example, if m = 5 and n = 50, = 118 60 5 The brute force approach of exhaustively comparing all the BFSs is impractical Therefore, we need a more computationally e cient method to nd the minimizer among the possible BFSs Simplex method: an organized way of going from one BFS to another to search for the global minimizer. Simplex Algorithms The FTLP allows us to limit search of optimal solutions to a nite number of points (BFSs). Simplex method: an organized way of going from one BFS to another to search for the global minimizer. The method uses numerical linear algebraic techniques. Sometimes, to avoid confusion, we use boldface letters for vectors Simplex method Basic idea Start with an initial basis corresponding to a BFS 1

We then move to an adjacent BFS (adjacent vertex) in such a way that the objective function decreases If the stopping criterion is satis ed, we stop; otherwise, we repeat the process. We move from one basic solution to an adjacent one: if a p leaves the basis and a q enters the basis, we pivot about the (r; q) position of the canonical augmented matrix p-th column q th column.. 0 0 r-th row 1! 1 0 0 6 6 4. 5 4. 5 0 0 x p is D.V.(departing variable) x q is E.V. (entering variable) Example: a ; a 4 form the original basis. To replace a by a 1 ; we need to pivot about (1; 1) (p = ; r = 1; q = 1): The new basis is fa 1 ; a 4 g 1 1 0 8 1 1 0 8! 0 1 1 0 51 1 18 x is D.V.; x 1 is E.V. Two questions: 1. How do we ensure that the adjacent basic solution we move to is feasible? In other words if we are given x q E.V., how do we choose a D.V. x p such that if we pivot about the (r; q) element ((r; p) is a pivot), the resulting rightmost column of the canonical augmented matrix will have all nonnegative elements? 18

. How do we choose E.V. x q to ensue the objective function at the new basic solution is smaller the previous value? Maintain feasibility (optional): Choosing leaving variable x p Feasibility: the last column 0 Given: A and b. Suppose we have a set of basic variables corresponding to a BFS. For simplicity, assume it is fx 1 ; :::; x m g, the corresponding basis is fa 1 ; :::; a m g Suppose we now want to choose x q as E:V:, where q > m. One of the vectors a p, 1 p m, must leave the basis. Suppose the canonical augmented matrix for the original basis is: 1 0 0 y 1;m+1 y 1;n y 1;0 0 1 0 y ;m+1 y ;n y ;0 [A; b]! 6 4......... 5 0 0 1 y m;m+1 y m;n y m;0 Note y i0 0 due to feasible assumption If, for example, a m+1 replaces a 1 by pivoting bout (1; m + 1) the resulting canonical augmented matrix 1 y 0 0 1 1;n y 1;0 y 1;m+1 y 1;m+1 y 1;m+1 0 1 0 y [A; b]! ;m+1 y ;n y ;0 6 4......... 5 0 0 1 y m;m+1 y m;n y m;0 1 y 0 0 1 1;n y 1;0 y 1;m+1 y 1;m+1 y 1;m+1 y ;m+1 y 1 0 0 y 1;n y y ;n y! 1;m+1 y ;m+1 y 1;0 ;0 y 1;m+1 y ;m+1 1;m+1 6. 4........ 5 y m;m+1 y 0 1 0 y 1;n y y m;n y 1;m+1 y m;m+1 y 1;0 m;0 y 1;m+1 y m;m+1 1;m+1 19

the last column is 6 4 y 10 y 1m+1 y 0 y 10 y 1m+1 y m+1. y y 10 m0 y y mm+1 1m+1 In general, if a q replaces a p by pivoting about (r; q) ((r; p) is a pivot), the last column of new canonical augmented matrix will be i.e., r th element is y r;0 y r;q 6 4. y r;0 y r;q. y y r;0 i;0 y r;q y i;q. 5 r-th i-th 5 all other i th element is y i;0 y r;0 y r;q y i;q for i 6= p If y r;q < 0; then the feasible set is unbounded Hence, to ensue feasibility, we have to choose r such that y r;q > 0; and for all i = 1; ; :::; m; y r;0 yi;0 y r;0 y i0 y i;q = y i;q 0 =) y r;q y i;q y r;q yi;0 r = arg min : i = 1; ; :::; m y i;q We may divide i-th component in q-th column into i-th component in the last column to create "division" y 1;0 y 1;q y 1;0 =y 1;q y ;0 6 4. 5 y ;q 6 4. 5 = y ;0 =y ;q 6 4. 5 y m;0 y m;0 =y m;q y m;q 0

In summary, to maintain feasibility, the leaving variable x p must be the position in the above "quotation" vector with the smallest value, i.e., y r;0 y r;q y i;0 y i;q x q Maintain optimality (optional): Choosing entering variable Finding a better adjacent BFS Once we select a q ; we know how to pick a p to leave the basis The question is how to choose column a q to enter the basis? The goal is that the new (adjacent) BFS should have lower objective function value Suppose the canonical augmented matrix for the original basis is: 1 0 0 y 1;m+1 y 1;n y 1;0 0 1 0 y ;m+1 y ;n y ;0 [A; b] = 6 4......... 5 0 0 1 y m;m+1 y m;n y m;0 basic cost function coe cients are those associated with the basic variables c 0 = [c 1 ; c ; ; c m ] The objective function for the BFS x = [y 10 ; ; y m0 ; 0; 0] T is z 0 = c T x = c T 0 x = mx c i y i0 i=1 Now, consider a new basis where a q enters the basis and a p leaves the basis. The new BFS is (with r = p in this case) x y r;0 y r;0 = y 1;0 y 1;q ; 0; ; y y m;0 y m;q ; 0; 0; y r;0 ; 0 r;q y r;q y r;q 1

The objective function for the new BFS is mx z = c T x y r0 = c i y i0 y iq y rq = = mx i=1 c i y i0 mx c i y i0 i=1 i=1;i6=r y r0 y iq y rq m X i=1 = z 0 + c q m X i=1 Note that the q-th column of A is + c q y r0 y rq y r0 y r0 c i y iq + c q y rq y rq! y r;0 c i y i;q y r;q 6 4 y 1;q y ;q. y m;q 5 + c q y r0 y rq Hence q-th component in c T 0 A is z q = mx c i y i;q i=1 we have z = z 0 + (c q z q ) y r;0 y r;q Hence if (c q z q ) < 0, we conclude z < z 0 ; i.e., the objective function for the new BFS is smaller De ne r i = 0 for i = 1; :::; m (basic) c i z i for i = m + 1; :::; n (nonbasic) we call r i Relative cost coe cients (RCC) for x i z = z 0 + r q y r;0 y r;q

Note: the RCC for a basic variable is always 0. If r q < 0, then the new BFS is better. The smaller r q, the better the new BFS In general, if x is any feasible solution, we can derive the equation nx z = c T x = z 0 + r i x i i=m+1 RCC (Relative cost coe cients) vector r is r = [r 0 ; r ; :::; r n ] T = c T c T 0 A Optimality conditions: r = c T c T 0 A 0: This leads to Theorem A BFS is optimal if and only if the corresponding RCC values (components of c T c T 0 A ) are all 0: In summary, we choose a variable x q that has the most negative RCC value as the entering variable. The Simplex Algorithm 1. Initialization: Given an initial BFS and its canonical augmented matrix.. Calculate the RCCs ( c T c T 0 A ) corresponding to the nonbasic variables.. If r j > 0 all j, then STOP the current BFS is optimal. 4. Otherwise, select a q such that r q < 0 is the most negative. 5. If in the qth column a q ; no element y iq > 0, then STOP the problem is unbounded; else, calculate r = argminfy i0 =y iq : y iq > 0g.

6. Update canonical augmented matrix by pivoting about the (r; q) element.. Go to step. Simplex tableau We design a speci c algorithm to compute optimal solutions We focus on the standard form: Simplex tableau minimize: z = c T x subject to: Ax = b; x 0; rank (A) = m; b > 0 Suppose that we already know one set of basic variables x 0 : Let c 0 be the basic cost coe cient in c associated with basic variable x 0 : For instance, if x 0 = (x ; x 5 ; x 8 ) be the set of basic variables, then c T 0 = [c ; c 5 ; c 8 ] : simplex tableau x T c T x 0 c 0 A b r = c T c T 0 A c T 0 b where the second column from the left (c 0 ) and the second row from the top (c T ) are for convenience in calculating the last row, and can be deleted from the nal tableau: x T x 0 A b c T c T 0 A c T 0 b 4

Example: Consider with the standard form minimize: f (x 1 ; x ) = 10x 1 80x subject to: x 1 + x 6 x 1 + 8x 8: minimize: s = 10 (y 1 z 1 ) 80 (y z ) subject to: (y 1 z 1 ) + (y z ) + w 1 = 6 the initial feasible solution x = 6 4 (y 1 z 1 ) + 8 (y z ) + w = 8 y 1 ; y ; z 1 ; z ; w 1 ; w 0 y 1 z 1 y z w 1 w = 6 5 4 0 0 0 0 6 8 : 5 Thus, basic variables and basic cost coe cients: w1 0 x 0 = ; c 0 = 0 w simplex tableau is c T c T 0 A = c T w 1 0 w 0 y 1 z 1 y z w 1 w 10 10 80 80 0 0 1 1 1 0 6 8 8 0 1 8 10 10 80 80 0 0 0 5

the nal simplex tableau is Simplex method y 1 z 1 y z w 1 w w 1 1 1 1 0 6 w 8 8 0 1 8 10 10 80 80 0 0 0 The simplex algorithm moves the initial feasible but non-optimal solution to an optimal solution while maintaining feasibility through the iterative procedure. The goal is to obtain optimality condition, i.e., the last row (except for the very last element) are all non-negative. step 1. Locate the most negative number in the bottom row of the simplex tableau, excluding the last column. The variable associated with this column is E.V. We also call it the work column. If more than one candidate for the most negative numbers exist, pick any one. step. Form ratios by dividing each positive number in the working column, excluding the last row, into the element in the same row and last column. Designate the element in the work column that yields the smallest ration as the pivot element. The basic variable associated with this pivot element is D.V. If no element in the work column is positive, stop. The problem has no solution. step. Pivoting about the pivot element: Use elementary row operations to convert the pivot element to 1 and then to reduce all other elements in the work column to 0: Step 4. Swap E.V. with D.V. Replace the variable in the pivot row and the rst column by the variable in the rst row and work column. The new rst column is the current set of basic variables. Step 5. Repeat Step 1 through Step 4 until there are no negative numbers in the last row, excluding the last column. The basic feasible solution derived from the very last set of variable is a minimal point: assign to each basic variable (variables in the rst column) the number in 6

the same row and last column, and all others are zero. The negative number of the number in the last row and last column is the minimum value. Example: Consider the example above with the standard form copied below: y 1 z 1 y z w 1 w Solution: w 1 1 1 1 0 w 8 8 0 1 6 8 10 10 80 80 0 0 0 Step 1. The work column is the rst in the middle (-10 is the most negative). Step. The ratios are 6 8 = 4 : So the element is the pivot. w 1 is D.V. and y 1 is E.V. Step. We perform row operations to reduce to the following updated simplex tableau: y 1 z 1 y z w 1 w y 1 1 1 1= 1= 1= 0 w 0 0 9= 9= = 1 0 0 0 0 60 0 60 The basic variable w 1 is replaced by y 1 : Now, in the updated tableau, the basic variables are y 1 and w ; and the updated basic feasible solution is y 1 = ; w = ; all others are zero. We next repeat the same steps: since there is one negative number 0 in the last row, the work column is 4 1= 9= 0 5

and the ratios are 1= 9= = 6 14=9 = 6 1:56 Therefore, the pivot is the one with the largest ratio: 9= : w is D.V. and y is E.V. We now perform row operations to get updated simplex tableau: pivoting by rst divided by 9= y 1 z 1 y z w 1 w y 1 1 1 1= 1= 1= 0 w 0 0 1 1 =9 =9 14=9 0 0 0 0 60 0 60 and then reduce all other elements in the work column to zero y 1 z 1 y z w 1 w y 1 1 1 0 0 1= + (=9) (1=) (=9) (1=) (14=9) (1=) w 0 0 1 1 =9 =9 14=9 0 0 0 0 60 140=9 40=9 60 + 14 0=9 simplify and replace w by y, we obtain the updated simplex tableau: y 1 z 1 y z w 1 w 8 1 y 1 1 1 0 0 y 9 9 0 0 1 1 =9 =9 400 0 0 0 0 40=9 9 0 9 14=9 50 9 Since there is no negative number in the last row, we are done. Optimal solution: z 1 y1 0=9 = ; 6 z 50 y 14=9 4 w 1 5 = 0; minimal value = 91: 11 9 w 8

Now, let go back to the original optimization problem: minimize: f (x 1 ; x ) = 10x 1 80x subject to: x 1 + x 6 x 1 + 8x 8: Recall that x 1 = y 1 z 1 ; x = y z : Thus, the nal solution: Arti cial LP f (x 1 ; x ) attains the minimal value of when x 1 = 0 9 ; x = 14 9 : The simplex method requires an initial basis. How to choose an initial basis? 50 9 91: 11 Brute force: arbitrarily choose m basic columns and transform the augmented matrix for the problem into canonical form. If rightmost column is positive, then we have a legitimate (initial) BFS. Otherwise, try again. Potentially requires Consider the standard form LP n m tries, and tries, and is therefore not practical minimize: z = c T x subject to: Ax = b; x 0; rank (A) = m; A mn ; b 0 De ne the associated Arti cial LP: minimize: y 1 + y + + y m x subject to: [A; I m ] = b y x 0 y 9

where y = [y 1 ; y ; ; y m ] T is called the vector of arti cial variables. Note that the Arti cial LP has an obvious initial BFS 0 b we can proceed with the simplex algorithm to solve the Arti cial LP Note that for y 0 y 1 + y + + y m 0 and the strict inequality holds if one of y i > 0 So if the minimum value of the Arti cial LP = 0; then y i must be non-basic, i.e., y i = 0 Proposition 16.1: The original LP problem has a BFS if and only if the associated arti cial problem has an optimal feasible solution with objective function value zero. In this case, the optimal solution for the arti cial LP serves as a BFS for the original LP Two-phase simplex method: Phase I: solve the arti cial problem using simplex method Phase II: use the BFS resulting from phase I to initialize the simplex algorithm to solve the original LP problem. 0

Example 16.4 Consider Solution: Standard form minimize: x 1 + x subject to: 4x 1 + x 1 x 1 ; x 0 x 1 + 4x 6 minimize: x 1 + x subject to: 4x 1 + x x = 1 There is no obvious feasible solution. Phase I: Solve x 1 + 4x x 4 = 6 x 1 ; :::; x 4 0 minimize: x 5 + x 6 subject to: 4x 1 + x x + x 5 = 1 x 1 + 4x x 4 + x 6 = 6 x 1 ; :::; x 6 0 x = [0; 0; 0; 0; 1; 6] is an obvious BFS simples tableau x 1 x x x 4 x 5 x 6 x 5 4 1 0 1 0 1 x 6 1 4 0 1 0 1 6 5 6 1 1 0 0 18 pivoting position is (; ) : x 6 is D.V., x is E.V. 1

Updated tableau: x 5 x 1 4 x 1 x x x 4 x 5 x 6 1 0 1 1 1 1 0 0 4 1 0 1 0 1 1 4 9 9 Next, pivoting about (1; 1) : x 5 is D.V., x 1 is E.V. updating x 1 x x x 4 x 5 x 6 1 1 18 x 1 1 0 x 1 1 6 0 1 14 14 0 0 0 0 1 1 0 Done: minimum value = 0. It produced a BFS [x 1 ; x ; x ; x 4 ] = 18 ; 6 ; 0; 0 Phase II: we can obtain the simplex tableau for the original LP immediately from above: Deleting two arti cial variable columns, then replace the last row by c T c T 0 A : c T = [; ; 0; 0] ; c T 0 = [; ] 1 6 1 0 A = 4 1 5 0 1 14 c T c T 0 A = 0 0 5 14 4

So the simplex tableau for the original LP : x 1 x x x 4 1 x 1 1 0 x 1 0 1 14 5 4 0 0 14 18 6 54 It is already optimal. The solution is x = 18 6 0 0 T ; f = 54 Homework: 1. A cereal manufacturer wishes to produce 1000 pounds of a cereal that contains exactly 10% ber, % fat, and 5% sugar (by weight). The cereal is to be produced by combining four items of raw food material in appropriate proportions. These four items have certain combinations of ber, fat, and sugar content, and are available at various prices per pound, as shown below: The manufacturer wishes to nd the amounts of each of the above items to be used to produce the cereal in the least expensive way. Formulate the problem as a linear programming problem. What can you say about the existence of a solution to this problem?. Solve the following linear program graphically:

. Use the simplex method to solve the following linear program: 4. Convert the following problem to standard form and then solve it using the simplex method: 5. From textbook: #5.8, 5.1, 5.1, 5.16 (you may combine these four problems into one with several parts) 4