Note 3: LP Duality If the primal problem (P) in the canonical form is min Z = n j=1 c j x j s.t. nj=1 a ij x j b i i = 1, 2,..., m (1) x j 0 j = 1, 2,..., n, then the dual problem (D) in the canonical form is max W = m i=1 b i y i s.t. mi=1 a ij y i c j j = 1, 2,..., n (2) y i 0 i = 1, 2,..., m. We may think of j as a food type, i as a nutrition type, c j as the per-unit price of food type j, b i as the required quantity of nutrition type i, a ij as the quantity of nutrition type i contained in each unit of food type j, x j as the quantity of food type j to purchase, and y i as the per-unit price to be charged for nutrition type i. The primal problem may be considered as finding the least costing quantities of foods to buy to satisfy nutritional needs, and the dual problem can be considered as finding the most profitable pricing scheme for pure nutritions that is competitive in the face of existing prices of food types. In essence, (P) s variable corresponds to (D) s constraint, and (D) s variable corresponds to (P) s constraint; If (P) is a minimization (maximization) problem, then (D) is a maximization (minimization) problem; (P) s free variable corresponds to (D) s = constraint and its nonnegative variable corresponds to (D) s max / constraint (meaning that (D) is a maximization problem and the constraint is of the type) or min / constraint; on the other hand, (D) s free variable corresponds to (P) s = constraint, its nonnegative variable corresponds to (P) s max / or min / constraint. The dual of dual is the primal itself. In lieu of the above, if the primal is our standard form min Z = n j=1 c j x j s.t. nj=1 a ij x j = b i i = 1, 2,..., m x j 0 j = 1, 2,..., n, 1
with n m, then its dual is max s.t. W = m i=1 b i y i mi=1 a ij y i c i y i is free i = 1, 2,..., m. By introducing the slack variables v = (v 1, v 2,..., v n ), the dual LP can be written as max W = b T y s.t. A T y + v T = c y is free and v T 0. For any m-element subset B of {1, 2,..., n}, the dual is max s.t. W = b T y B T y + v T B = c B N T y + v T N = c N y is free, v B 0, v N 0. Now, once v B is determined, we have y = (c T BB 1 v B B 1 ) T, v N = (c T N c T BB 1 N)+ v B B 1 N, and W = c T BB 1 b v B B 1 b. While solving our standard-form LP (primal) using the simplex method, suppose we are at a BS corresponding to B, we don t have c T N c T BB 1 N until B is the optimal basis. That is, the v N corresponding to such B and to v B = 0 is not nonnegative until B is the optimal basis for the primal. Also, for every unit increase in v i for i belonging to B, the dual objective W decreases by (B 1 b) i. Therefore, the dual LP is optimal if and only if B 1 b 0. So the dual BS v corresponding to B is optimal when the primal BS x corresponding to B is feasible. To summarize, we have that primal optimality corresponds to dual feasibility, and dual optimality corresponds to primal feasibility. Weak duality: If x is a feasible solution for the primal problem and y is a feasible solution for the dual problem, then c T x b T y. Reason: Because x 0 and A T y c, we have c T x y T Ax. Since Ax = b, we have y T Ax = y T b = b T y. 2
Strong duality: If x is an optimal solution for the primal problem and y is an optimal solution for the dual problem, then c T x = b T y. Reason: Let B be the optimal basis for the primal problem, then it satisfies both B 1 b 0 and c T N c T BB 1 N. Let x = (x T B, x T N) T and y = B T c B. It is obvious that x is primal feasible (feasible for the primal problem) and y is dual feasible. But it is obvious that c T x = b T y. From the weak duality, y must also be dual optimal. Complementary slackness: We always have x j (c j m i=1 a ij y i ) 0 for j = 1, 2,..., n for any primal feasible x and dual feasible y. Also, x j(c j m i=1 a ij yj ) = 0 for j = 1, 2,..., n if and only if x is primal optimal and y is dual optimal. Reason: The one for the feasible x and y can be derived from the weak duality property. The one for the optimal x and y can be derived from the strong duality property. In plain English, if the primal problem is of the minimization type and the dual problem is of the maximization type, then the primal objective value is always above the dual objective value. But the minimum that the primal can achieve is exactly the maximum that the dual can achieve. That is why a basis being primal/dual optimal leads to it being dual/primal feasible, and why a basis being primal and dual feasible leads to it being optimal, in both primal and dual senses. Also, an optimal primal/dual variable must be 0 when its corresponding dual/primal constraint is not tight at optimality, and a primal/dual constraint must be tight at optimality when its corresponding dual/primal variable is not 0 at optimality. Duality Theorem: The following are the only possible relationships between the primal and dual problems: 1. If one problem has feasible solutions and a bounded objective function (and so has an optimal solution), then so does the other problem, so both the weak and strong duality properties are applicable. 2. If one problem has feasible solutions and an unbounded objective function (and so no optimal solution), then the other problem has no feasible solutions. 3. If one problem has no feasible solutions, then the other problem has either no feasible solutions or an unbounded objective function. From the above observations stems the dual simplex method. This method in effect applies the simplex method to the dual of a given problem to solve the problem. 3
However, the dual simplex method can use the same tableau as the (primal) simplex method. While this method is solving an LP, at any moment, we have c T N c T BB 1 N 0 T. If B 1 b 0 already, then we have both primal and dual feasibilities and therefore primal and dual optimalities; and W will not increase if any of the v i s for i B is raised from 0, and so the current B is the optimal basis. If not, then we first select any i in B with (B 1 b) i < 0 to be ready for leaving B and entering N. Next, we examine the corresponding row of B 1 N, say (B 1 N) i. If (B 1 N) i 0, then v i can be increased indefinitely without making v N negative, and so the dual problem is unbounded and the original primal problem is infeasible. Otherwise, we let j be the one in N that achieves the minimum in = min{(c j c T BB 1 A j )/( B 1 A j ) i i B and (B 1 A j ) i < 0}. Then j will move from N to B. Just like in the simplex method, we need only to do a pivot on the i-j element in the tableau to transform all necessary information into the one corresponding to the new basis B = B {j} \ {i} and new nonbasis N = N \ {j} {i}. Farkas Lemma: Alternatively, we may use Farkas Lemma to prove the strong duality. The lemma says that, for vectors a 1,..., a m, c R n, the following (A) and (B) are equivalent: (A) for any x R n, as long as a T 1 x 0,..., a T mx 0, it will follow that c T x 0; (B) there exists y 1,..., y m 0 such that y 1 a 1 + + y m a m = c. The derivation from (B) to (A) is obvious. For the other direction, we may use proof by contradiction. Suppose (B) is not true. Then the set S = {y 1 a 1 + + y m a m y 1,..., y m 0}, as a nonempty, closed, and convex set in R n, does not contain the point c. Using the separating plane theorem, we can show the existence of an x R n that violates (A). To show the strong duality for the canonical form, we suppose that some x is an optimal solution of (P). Without loss of generality, we suppose that the primal feasible x satisfies a T 1 x = b 1,..., a T p x = b p, a T p+1x > b p+1,..., a T mx > b m, e T 1 x = 0,..., e T q x = 0, e T q+1x > 0,..., e T nx > 0, 4
where each e k is the vector whose only nonzero element is the 1 at the kth position. Since x is primal optimal, for any x R n, as long as a T 1 x 0,..., a T p x 0, e T 1 x 0,..., e T q x 0, it will follow that c T x 0. By Farkas lemma, we know the existence of y 1,..., y p, z 1,..., z q 0, so that c = y 1 a 1 + + y p a p + z 1 e 1 + + z q e q. Consider y = (y 1,..., y p, 0,..., 0) T R m. We have A T y = (a 1,..., a m )y = y 1 a 1 + + y p a p = c z 1 e 1 z q e q c, and y 0. Thus, y is dual feasible. Also, b T y = b 1 y 1 + + b p y p = a T 1 x y 1 + + a T p x y p = (y 1 a 1 + + y p a p ) T x = (c z 1 e 1 z q e q ) T x = c T x z 1 e T 1 x z q e T q x = c T x. By the weak duality, y must be dual optimal as well. Hence, (P) and (D) share the same objective value. 5