Midterm Review Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapter 1-4, Appendices) 1
Separating hyperplane theorem The most important theorem about the convex set is the following separating theorem. Theorem 1 (Separating hyperplane theorem) Let C E, where E is either R n or M n, be a closed convex set and let y be a point exterior to C. Then there is a vector a E such that a y < inf x C a x. 2
Farkas Lemma The following results are Farkas lemma and its variants. Theorem 2 Let A R m n and b R m. Then, the system {x : Ax = b, x 0} has a feasible solution x if and only if that A T y 0 implies b T y 0. A vector y, with A T y 0 and b T y = 1, is called a (primal) infeasibility certificate for the system {x : Ax = b, x 0}. Geometrically, Farkas lemma means that if a vector b R m does not belong to the cone generated by a.1,..., a.n, then there is a hyperplane separating b from cone(a.1,..., a.n ). Example Let A = (1, 1) and b = 1. Then, y = 1 is an infeasibility certificate for {x : Ax = b, x 0}. 3
Theorem 3 Let A R m n and c R n. Then, the system {y : A T y c} has a solution y if and only if that Ax = 0 and x 0 imply c T x 0. Again, a vector x 0, with Ax = 0 and c T x = 1, is called a (dual) infeasibility certificate for the system {y : A T y c}. example Let A = (1; 1) and c = (1; 2). Then, x = (1; 1) is an infeasibility certificate for {y : A T y c}. 4
Duality Theory Consider the linear program in standard form, called the primal problem, (LP ) minimize c T x subject to Ax = b, x 0, where x R n. The dual problem can be written as: (LD) maximize b T y subject to A T y + s = c, s 0, where y R m and s R n. The components of s are called dual slacks. 5
Duality Theory Theorem 4 (Weak duality theorem) Let F p and F d be non-empty. Then, c T x b T y where x F p, (y, s) F d. This theorem shows that a feasible solution to either problem yields a bound on the value of the other problem. We call c T x b T y the duality gap. 6
Theorem 5 (Strong duality theorem) Let F p and F d be non-empty. Then, x is optimal for (LP) if and only if the following conditions hold: i) x F p ; ii) there is (y, s ) F d ; iii) c T x = b T y. 7
Theorem 6 (LP duality theorem) If (LP) and (LD) both have feasible solutions then both problems have optimal solutions and the optimal objective values of the objective functions are equal. If one of (LP) or (LD) has no feasible solution, then the other is either unbounded or has no feasible solution. If one of (LP) or (LD) is unbounded then the other has no feasible solution. The above theorems show that if a pair of feasible solutions can be found to the primal and dual problems with equal objective values, then these are both optimal. The converse is also true; there is no gap. 8
For feasible x and (y, s), x T s = x T (c A T y) = c T x b T y is called the complementarity gap. If x T s = 0, then we say x and s are complementary to each other. Since both x and s are nonnegative, x T s = 0 implies that x j s j = 0 for all j = 1,..., n. Xs = 0 Ax = b A T y s = c. This system has total 2n + m unknowns and 2n + m equations including n nonlinear equations. 9
Rules to construct the dual obj. coef. vector right-hand-side A right-hand-side obj. coef. vector A T Max model Min model x j 0 x j 0 jth constraint jth constraint x j free jth constraint = ith constraint y i 0 ith constraint y i 0 ith constraint = y i free 10
Basic Feasible Solution In the LP standard form, select m linearly independent columns, denoted by the index set B, from A. A B x B = b for the m-vector x B. By setting the variables, x N, of x corresponding to the remaining columns of A equal to zero, we obtain a solution x such that Ax = b. Then, x is said to be a (primal) basic solution to (LP) with respect to the basis A B. The components of x B are called basic variables. If a basic solution x 0, then x is called a basic feasible solution. If one or more components in x B has value zero, that basic feasible solution x is said to be (primal) degenerate. 11
A dual vector y satisfying is said to be the corresponding dual basic solution. If the dual basic solution is also feasible, that is A T By = c B s = c A T y 0. If one or more slacks in c N A T N y has value zero, that dual basic feasible solution y is said to be (primal) degenerate. 12
Theorem 7 (LP fundamental theorem) Given (LP) and (LD) where A has full row rank m, i) if there is a feasible solution, there is a basic feasible solution; ii) if there is an optimal solution, there is an optimal basic solution. 13
Sample Problem 1 Let A 1 R m n, A 2 R m p be two given matrices, and let c 1 R n, c 2 R p be two given non-negative vectors. Consider the problem min c T 1 x 1 + c T 2 x 2 s.t. A 1 x 1 + A 2 x 2 = b x 1, x 2 0, and assume it is feasible. (a) The problem has an optimal solution. Why? (b) Let (x 1, x 2 ) be a feasible solution to the problem and its objective value equals b T y where y satisfies A T 1 y α 1 c 1 A T 2 y α 2 c 2, 14
where α 1 and α 2 are two scalars greater than or equal to 1, then b T y α 1 c T 1 x 1 + α 2 c T 2 x 2. (α 1, α 2 ) is usually called the bi-factor approximation ratio and used in approximating algorithms for combinatorial optimization. 15
(a) Consider the dual problem: max b T y subject to A T 1 y c 1, A T 2 y c 2. Since c 1 0 and c 2 0, y = 0 is a feasible point for the dual. By LP duality, since the primal and dual problems are feasible, both must have optimal solutions. 16
(b) Let (x 1, x 2) be a primal optimal solution, and let (x 1, x 2 ) be a primal solution with value c T 1 x 1 + c T 2 x 2 = b T y, where y satisfies A T 1 y α 1 c 1, A T 2 y α 2 c 2. Since (x 1, x 2) is primal feasible, we must have A 1 x 1 + A 2 x 2 = b, and x 1, x 2 0. Then, (A T 1 y) T x 1 α 1 c T 1 x 1, (A T 2 y) T x 2 α 2 c T 2 x 2. Finally, b T y = (A 1 x 1 + A 2 x 2) T y = (A T 1 y) T x 1 + (A T 2 y) T x 2 α 1 c T 1 x 1 + α 2 c T 2 x 2. 17
MS&E, Stanford University MS&E310 Midterm Review Note 18-1 Sample Problem 2 l max 8x 1 +14x 2 +30x 3 +50x 4 subject to x 1 +2x 2 +10x 3 +16x 4 +x 5 = 800 1.5x 1 +2x 2 +4x 3 +5x 4 +x 6 = 1000 0.5x 1 +0.6x 2 +x 3 +2x 4 +x 7 = 340 x 0.
x 1 x 2 x 3 x 4 x 5 x 6 x 7 1 B -8-14 -30-50 0 0 0 0 5 1 2 10 16 1 0 0 800 6 1.5 2 4 5 0 1 0 1000 7 0.5 0.6 1 2 0 0 1 340 18
x 1 x 2 x 3 x 4 x 5 x 6 x 7 1 B 0 0 28 40 5 2 0 6000 2 0 1 11 19 1.5-1 0 200 1 1 0-12 -22-2 2 0 400 7 0 0-4 1.6 0.1-0.4 1 20 19
Questions 1. By how much must the unit profit on variable 3 be increased before it would be profitable to manufacture it? :This can be answered by simply checking the reduced cost of the final tableau for x 3, which is 28. The same question for x 4 is 40. 2. What must the minimum unit profit on variable 2 be so that the optimal basis remains the same? : Take the reduced cost row and the row with x 2 as the basic variable for non-basic variable columns (28, 40, 5, 2) λ(11, 19, 1.5, 1) 0 makes 2 λ 2.1. The answer is 11.9. 3. If the 800 units of Resource 1 is uncertain, for what range of the units will the optimal basis remains the same? 20
200 + 1.5λ 0, 400 2λ 0, 20 + 0.1λ 0 makes 133 λ 200. The answer is [667, 1000]. 4. A competitor located next door has offered the manager additional Resource 1 at a rate of $4.50 per unit. Should he accept his offer? : Easy, take it since the shadow price for this resource is $5. 5. Suppose instead that the competitor offers the manager 250 units of Resource 1 for a total of $1, 100, Should he accept his offer? (The manager can only accept or reject the extra 250 units.) :We know that the first 200 units worth 5 200 = 1000 but don t know how much the next 50 worth for. Sorry, we need to resolve the LP problem to answer this question. 6. The owner has approached the manager with a thought about producing a new type of product that would require 4 units of Resource 1, 4 units of Resource 2 and 1 unit of Resource 3. What should be the minimum unit profit of the new product such that it is to be manufactured? 21
: The answer is $28, why? 22