Chapter 1 Linear Programming Paragraph 5 Duality
What we did so far We developed the 2-Phase Simplex Algorithm: Hop (reasonably) from basic solution (bs) to bs until you find a basic feasible solution (bfs) or until it becomes clear that none exists. Hop from bfs to bfs until you cannot find an improving solution or the improving trajectory is unbounded. We have seen that we can guarantee termination but that the worst-case runtime is exponential. Again: Note that LP is polynomial time solvable! CS 149 - Intro to CO 2
Lower Bounds on the Objective Let π Ñ m, π. Consider two sets: P 1 = {x Ñ n Ax b, x } and P 2 = {x Ñ n π T Ax π T b, x }. Since π, it is clear that P 1 Œ P 2, in other words: For all x P 1 we have that π T Ax π T b. Now assume that π and also π T A c T. Then, for all x P 1, we have that π T b π T Ax c T x. Therefore, every such π provides us with a lower bound on the objective! CS 149 - Intro to CO 3
Duality The problem of finding the best such lower bound π T b with π Ñ such that π T A c T, π is called the Dual Problem of (P 1,c). Formally, let P 3 = {π Ñ m A T π c, π }. Then, (P 3,b) is called the dual of the primal (P 1,c). Theorem The dual of the dual is the primal. For all primal feasible x and dual feasible π it holds that π T b c T x [weak duality]. CS 149 - Intro to CO 4
Example Minimize x 1 +3x 2 +x 3 such that 2x 1 +x 2 3 -x 1 +x 3 4 (P) x 1, x 2, x 3 π 1T = (1/2,1) Relax P 1 to.5x 2 +x 3 5.5 x 1, x 2, x 3 Maximize 3π 1 +4π 2 such that 2π 1 - π 2 1 π 1 3 (D) π 2 1 π 1 is dual feasible and shows: every feasible x (if any exists!) has an objective value of at least 5.5! CS 149 - Intro to CO 5
Example Minimize x 1 +3x 2 +x 3 such that 2x 1 +x 2 3 -x 1 +x 3 4 (P) x 1, x 2, x 3 π T = (1,1) Relax P 1 to x1+x2+x3 7 x 1, x 2, x 3 Maximize 3π 1 +4π 2 such that 2π 1 - π 2 1 π 1 3 (D) π 2 1 π is dual optimal and shows: every feasible x (if any exists!) has an objective value of at least 7! CS 149 - Intro to CO 6
The Dual of the Standard Form P 1 = {x Ñ n Ax = b, x } (LP) S = (P 1,c) P 2 = {x Ñ n Ax b, -Ax -b, x } (LP) C = (P 2,c) P 3 = {(η,μ) Ñ m A T η -A T μ c, η,μ } @ P 4 = {π Ñ m A T π c} [π not non-negative anymore!] The dual of (LP) S is therefore (P 4,b). CS 149 - Intro to CO 7
The Dual of Unspecified Forms Min c T x A M+ x b + A M x = b A M- x b - x N+ x N unrestricted x N- Max b T π π M+ π M unrestricted π M- (A T ) N+ π c N+ (A T ) N π = c N (A T ) N- π c N- M+ M M- = {1,..,m} N+ N N- = {1,..,n} CS 149 - Intro to CO 8
Strong Duality c T = c T c BT A B -1 A -c T x A B -1 A A B -1 b Given is an LP in standard form. We know: The current bfs x in the tableau is optimal when c T c BT A B -1 A ñ (c BT A B -1 ) A c. Therefore, π T := c BT A B -1 is dual feasible! But: π T b = c BT A B -1 b= c BT x B = c T x! CS 149 - Intro to CO 9
Duality The problem of finding the best such lower bound π T b with π Ñ such that π T A c T, π is called the Dual Problem of (P 1,c). Formally, let P 3 = {π Ñ m A T π c, π }. Then, (P 3,b) is called the dual of the primal (P 1,c). Theorem The dual of the dual is the primal. For all primal feasible x and dual feasible π it holds that π T b c T x [weak duality]. For primal optimal x and dual optimal π it holds that π T b = c T x [strong duality]. CS 149 - Intro to CO 1
Complementary Slackness Consider an LP in canonical form with x primal optimal, and π dual optimal. c T x = π T b π T Ax c T x π T (Ax -b) =. π T b = c T x π T Ax π T b (c T - π T A) x =. Consequently, since π, Ax -b, π can only have non-zero components where there is no slack in the corresponding primal constraint. Analogously, since x, c T - π T A, x can only have non-zero components where there is no slack in the corresponding dual constraint. CS 149 - Intro to CO 11
Relation of Primal and Dual Primal Dual Finite Optimum Unbounded Infeasible Finite Optimum [sd] [wd] [sd] Unbounded [wd] [wd] [wd] Infeasible [sd] [wd] wd weak duality sd strong duality CS 149 - Intro to CO 12
Farkas Lemma Given are a i Ñ n,1 i m, and a vector c Ñ n. The cone of the a i is defined as the set C(a 1,..,a m ) := {x Ñ n x = Σπ i a i, π i }. Farkas Lemma (For all y with y T a i for all i it holds y T c ) c C(a 1,..,a m ). Proof: Exercise! CS 149 - Intro to CO 13
The Dual Simplex Algorithm We have seen how we compute a dual optimal solution after having solved the primal with the simplex algorithm. Consequently, we can solve the primal by solving the dual! In case that c, π = is dual feasible so we may save the effort for solving phase I when focusing on the dual right away. Example: Transportation Problem, Diet Problem, Shortest Path, CS 149 - Intro to CO 14
The Dual Simplex Algorithm Recall that the dual works by looking at A T instead of A in the tableau. Consequently, we can simply exchange the roles of rows and columns and consider matrix entries with opposite sign. Assume we are dual feasible. Then: Search for a row with negative b i. (no such i dual optimal primal feasible) Find a column j such that a ij < and c j / -a ij is minimal! (no such j dual unbounded primal infeasible) Pivot around a ij. CS 149 - Intro to CO 15
The Dual Simplex Algorithm Consider our instance of the diet problem: CS 149 - Intro to CO 16
Examples Diet Problem M B Amount needed [g] Cost/kg [$] 2 15 Minimize 25M + 15B Carbs [%] 1 25 5 1M + 25B 5 Fat [%] 1 5 25 1M + 5B 25 Protein [%] 15 2 6 15M + 2B 6 CS 149 - Intro to CO 17
The Dual Simplex Algorithm Consider our instance of the diet problem: B 5 4 $ 1 25-1 15-25 -5 2 1-1 -5-25 2 4 5 M 1-15 -2-6 CS 149 - Intro to CO 18
The Dual Simplex Algorithm Consider our instance of the diet problem: B 5 4 $ 1 3/4-5/4 55/4 35/4-45 25 2 1-1/4-25/4-1 2 4 5 M -1/2 3/4 1 3 CS 149 - Intro to CO 19
The Dual Simplex Algorithm Consider our instance of the diet problem: B 5 4 $ 1 11/5 7/5 1/5-8/5-67 11 2-4/25 1/25 1 8/5 2 4 5 M 3/25-4/25 1 9/5 CS 149 - Intro to CO 2
Shadow Prices Consider production planning. In the optimal tableau, the relative costs of the slack variables are exactly -c BT A B -1. Note that the objective in the simplex was to minimize. Thus, the relative costs of slack variables give us the dual solution. If a slack variable is not in the basis, it is in the optimal solution. I.e. the corresponding constraint is tight for optimality. Now, the relative costs tell us, how much we could gain if the constraint was not tight. Consequently, the optimal duals are called shadow prices of their corresponding constraints. CS 149 - Intro to CO 21
Thank you!