CO 602/CM 740: Fundamentals of Optimization Problem Set 4 H. Wolkowicz Fall 2014. Handed out: Wednesday 2014-Oct-15. Due: Wednesday 2014-Oct-22 in class before lecture starts. Contents 1 Unique Optimum 1 1.1 Solutions Problem 1......................................... 2 1.1.1 Solution Part 1........................................ 2 1.1.2 Solution Part 2........................................ 2 2 The simplex method with upper bound constraints 2 2.1 Solutions Problem 2......................................... 3 2.1.1 Solution Part 1........................................ 3 2.1.2 Solution Part 2........................................ 3 2.1.3 Solution Part 3........................................ 4 2.1.4 Solution Part 4........................................ 4 3 Degeneracy 4 3.1 Solution Question 3......................................... 4 3.1.1 Solution Part 1........................................ 4 3.1.2 Solution Part 2........................................ 5 3.1.3 Solution Part 3........................................ 6 1 Unique Optimum Consider a linear programming problem in standard form and suppose that x is an optimal basic feasible solution. Consider an optimal basis associated with x. Let B and N be the set of basic and nonbasic indices, respectively. Let I be the set of nonbasic indices i for which the corresponding reduced costs are zero. 1. Show that if I is empty, then x is the only optimal solution. 2. Show that x is the unique optimal solution if, and only if, the following linear problem has an optimal value of zero: max i I x i s.t. Ax = b (1) x i = 0, i N\I, x i 0, i B I. 1
1.1 Solutions Problem 1 1.1.1 Solution Part 1 Proof. Suppose x is an optimal basic feasible solution and I =, i.e., c N = c N c T B A B A N > 0. Wlog we can reorder the columns and assume that B = {1,...,m},N = {1,...,n}\B. Then ([ (A null(a) = range B A N) I n m The feasible directions d from x are therefore defined by ( ) ( ) ( ) [ 0 x x +αd = B db x x +α = B (A N d N x + B A ] N) w, w R+ n m, (2) N I n m where w 0 necessarily since x N = 0. By optimality, if w 0 and the corresponding 0 x = x +αd in (2), then we have further by feasibility of x that [ c T x c T x = c T (x +αd) = c T x +c T (A B A ] N) w = c T x + c T Nw, (3) i.e., we conclude that x feasible and x x implies that 0 w 0 and c T x = c T x + c T N w > ct x which means that x is not optimal. 1.1.2 Solution Part 2 Proof. : (Necessity) Suppose x is the unique optimal solution of (1), an LP. Therefore x is a BFS. Let N be the set of nonbasic indices for x. Then N and I are properly and uniquely defined. And, for all i N we have x i = 0. Since I N, it follows that x i = 0 i I. Then max i I x i = 0 over the feasible set. : (Sufficiency) Let x be an optimal BFS solution to the original LP problem. If the corresponding set I =, then x is unique (by part 1). Now suppose i 0 I N for the optimal solution x for the original program. Suppose that the optimal value for (1) is positive, 0 < i I x i, where x is an optimal BFS for (1) and therefore also feasible for the original LP. It is clear that x x. It is enough to show that x is also optimal for the original LP and so uniqueness of x fails. From (2) and feasibility we can write x = x +αd = ( x B x N ) + [ (A B A N) I n m ] I n m ]). w, 0 w R n m +,w i = 0, i N\I. (4) We can use (3) and see that x x and the objective value is c T x = c T x + c T N w = ct x, where we note that c T N w = 0 since w i = 0, i / I. Therefore, x is also optimal. 2 The simplex method with upper bound constraints Consider the LP in the form min c T x s.t. Ax = b 0 x u, where A is m n and has linearly independent rows, and u > 0. 1. Let A B(1),...,A B(m) be m linearly independent columns of A (the basic columns). { We partition the 0 if i L set of all i / {B(1),...,B(m)} into two distinct subsets L and U, where x i = u i if i U. We then solve the equation Ax = b for the basic variables x B(1),...,x B(m). Show that the resulting vector x is a basic solution. Also, show that it is a nondegenerate solution if, and only if, x i 0 and x i u i, for every basic variable x i. 2
2. For this part and the next, assume that the basic solution constructed in part 1 is feasible. We form the simplex tableau and compute the reduced costs as usual. Let x j be some nonbasic variable such that x j = 0 and c j < 0. As usual, choose the variable that leaves the basis by finding the first basic variable to violate that constraints, i.e., increase x j by θ and adjust the basic variables using x B x B θa B A j. What is the largest value for θ? How are the new basic columns determined, i.e., is there a ratio test available? 3. Let x j be some nonbasic variable such that x j = u j and c j > 0. We decrease x j by θ, and adjust the basic variables from x B to x B +θa B A j. Given that we wish to preserve feasibility, what is the largest possible value of θ? Is there a ratio test? How are the new basic columns determined? 4. Assuming that every basic feasible solution is nondegenerate, show that the cost strictly decreases with each iteration and the method terminates. 2.1 Solutions Problem 2 2.1.1 Solution Part 1 By definition a solution x is basic ifall the equalityconstraintsareactive and amongall the activeconstraints there are n of them that are linearly independent. Wolg after reordering the columns, we assume that B = {1,..., m} = {B(1),..., B(m)} are the indices for the basic (linearly independent) columns, and L = {m+1,...,m+l}, U = {m+l+1,...,n} are the indices for the lower and upper bounds. We can now set x i appropriately for i L U and solve for x B, i.e. we solve the m m linear system A B x B = b i U u i A i. This yields the unique solution x = x B x L = x B 0 (5) u x U for the linear system defined by the three index sets. By the uniqueness, this means that the active constraints defined by the three index sets is a linearly independent set of active constraints, i.e., the solution is a basic solution. By definition, a basic solution is degenerate if there are more than n active constraints. From the above we see that the equality constraints together with the constraints corresponding to the indices L U define n active constraints with L + U = n m. Therefore, the basic columns correspond to basic variables and we get a degenerate basic solution if, and only if, we have > n active constraints, i.e., the desired result follows. 2.1.2 Solution Part 2 We assume that 0 x B = A B (b i U u ia i ) u and we have a BFS as in (5). In addition, we can find the current shadow price estimates y T = c T B A B and the reduced cost vectors ct L = ct L yt A L, c T U = ct U yt A U. For j L the simplex method is unchanged, i.e., we can use Bland s rule for the entering variable since this means that x j increasing improves the objective value. However, for the leaving variable we have to ensure that no basic variable goes below zero as well as it does not go above u i. Suppose x j = 0 is a nonbasic variable that enters the basis. Then we increase x j by θ 0 and move to a new point until a constraint is violated. That is, we calculate the maximum θ 0 such that x B θa B A j is feasible: max{θ 0 0 x B θa B A j u B } 1. 0 x B θa B A j: θ min (A B Aj)i>0 2. x B θa B A j u B : θ min (A x Bi (A B Aj)i u Bi x Bi B Aj)i<0 (A B Aj)i 3
The basic variable entering the basis is determined using the Bland rule and this determines the column that enters the basis. The column that leaves the basis is determined using the above ratio test, i.e., by the first variable that reaches an upper or lower bound. 2.1.3 Solution Part 3 The argument is the same as above in Part 2 except that now the variable x j that enters the basis decreases. This can be done by changing the appropriate signs. 2.1.4 Solution Part 4 The algorithm progresses strictly towards an optimum using the same argument as for the simplex method and using the reduced costs. If the basic feasible solution is nondegenerate, then x Bi > 0 and x Bi < u Bi. So, θ > 0. At each iteration, we move from a current point a nonzero distance θ in a direction that strictly improves the cost function, so old basic feasible solutions are not revisited or no cycling. There is only a finite number of ways to choose nonbasic variables from L and U, and basic variables from B, therefore, the procedure will terminate when no choice of basic and nonbasic variables will improve the cost function. 3 Degeneracy Consider the maximization LP max [ 2.3x 1 +2.15x 2 3.55x 3 ] 0.4x( 4 0.4 0.2.4 0.2 0 s.t. x 7.8.4 7.8 0.4 0) x 0 1. Show that the simplex method cycles for this LP, i.e., it does not terminate in a finite number of iterations. 1 2. can you perturb the right hand side by a small amount and stop in a finite number of steps? 3. Do you get a solution (does the algorithm stop) using linprog in MATLAB? (Show your output.) 3.1 Solution Question 3 Solution using the tableau Iteration. 3.1.1 Solution Part 1 ( ) 0.4 0.2.4 0.2 1 0 c t =[-2.3-2.15 13.55 0.4 0 0], A=. 7.8.4 7.8 0.4 0 1 ( ) 1 0 Choose B={5,6},N={1,2,3,4}. Then A B =, c 0 1 t B =[0 0], x B = A B b=b=[0 0]t. 0-2.3-2.15 13.55 0.4 0 0 x 5 = 0 0.4 0.2-1.4-0.2 1 0 x 6 = 0-7.8-1.4 7.8 0.4 0 1 So,x 1 enters and x 5 leaves the basis.. T1 1 Hint: Let x 1 enter the basis for the first iteration; and, let x 2 enter the basis for the second iteration; break the tie by choosing the larger pivot element, i.e., the usual choice for stability of pivots in Gaussian elimination. 4
Updates: B= {1,6}, N={5,2,3,4}. 0 0-1 5.5-0.75 5.75 0 x 1 = 0 1 0.5-3.5-0.5 2.5 0 x 6 = 0 0 2.5-19.5-3.5 19.50 1 x 2 enters and x 6 leaves the basis.. T2 Updates: B= {1,2}, N={5,2,3,6}. 0 0 0-2.3-2.15 13.55 0.4 x 1 = 0 1 0 0.4 0.2-1.4-0.2 x 2 = 0 0 1-7.8-1.4 7.8 0.4 x 3 enters and x 1 leaves the basis.. T3 Updates: B= {3,2}, N={5,2,1,6}. 0 5.75 0 0-1 5.5-0.75 x 3 = 0 2.5 0 1 0.5-3.5-0.5 x 2 = 0 19.5 1 0 2.5-19.5-3.5 x 4 enters and x 2 leaves the basis.. T4 The tableau denoted T1,T3 and T2,T4 are the same up to permutation of columns. So, this process cycles. 3.1.2 Solution Part 2 Perturbing the right hand side. Let b = [0.004,0.001] t. Then repeating the tableau iteration shows that the problem is unbounded. 0-2.3-2.15 13.55 0.4 0 0 x 5 = 0.006 0.4 0.2-1.4-0.2 1 0 x 6 = 0.02-7.8-1.4 7.8 0.4 0 1 So,x 1 enters and x 5 leaves the basis.. T1 Updates: B= {1,6}, N={5,2,3,4}. 0.0345 0-1 5.5-0.75 5.75 0 x 1 = 0.015 1 0.5-3.5-0.5 2.5 0 x 6 = 0.117 0 2.5-19.5-3.5 19.5 1 x 2 enters and x 1 leaves the basis.. T2 Updates: B= {2,6}, N={5,1,3,6}. 0.0645 2 0-1.5-1.75 10.5 0 x 2 = 0.03 2 1-7 -1 5 0 x 6 = 0.042-5 0-2 -1 7 1 Since A 4 < 0, the optimal cost is, hence unbounded.. T2 5
3.1.3 Solution Part 3 When using MATLAB and linprog we get the output: Exiting: The problem is unbounded; the constraints are not restrictive enough. The MATLAB code is f= [ 2.3 2.15-13.55-0.4 ] ; A= [0.4 0.2-1.4-0.2-7.8-1.4 7.8 0.4 ]; b=[.001.00001 ] ; options=optimoptions(@linprog, Algorithm, simplex ); x=linprog(-f,a,b,[],[],zeros(4,1),[],[],options); Therefore MATLAB does not cycle. 6