CSCI5654 (Linear Programming, Fall 2013) Lecture-8 Lecture 8 Slide# 1
Today s Lecture 1. Recap of dual variables and strong duality. 2. Complementary Slackness Theorem. 3. Interpretation of dual variables. 4. Primal and Dual Simplex. Lecture 8 Slide# 2
Primal-Dual correspondences Variable to constraint correspondence: Dual variable (y j ) Primal constraint A j x b j Primal variable (x i ) Dual constraint A t i y c i Correspondence in dictionary: Dual decision variable y j Primal slack variable x n+j. Primal decision variable (x i ) Dual dictionary slack variable y m+i. Illustrate in class with an example. Lecture 8 Slide# 3
Strong Duality Theorem: If x 1,..., x m is primal optimal then there exists a dual optimal solution y 1,..., y m that satisifies c i xi i = j b j y j. ( if primal has optimal solution then the dual has optimum with the same value as primal). Proof: This is theorem 5.1 in Chvátal (pages 58-59). Lecture 8 Slide# 4
Example maximize x 1 + 2x 2 s.t. x 1 3 x 2 3 x 1 + x 2 1 x 1 + x 2 5 x 1, x 2 0 Lecture 8 Slide# 5
Example (final dictionary) x 1 = 2 + x 5 2 x 6 2 x 2 = 3 x 5 2 + x 6 2 x 3 = x 4 = z = 8 1 2 x 5 3 2 x 6 Insight: Dual variables correspond to primal slack variables. x 3 y 1, x 4 y 2, x 5 y 3, x 6 y 4. Read off dual solution by from objective row of final dictionary: y 1 : 0, y 2 : 0, y 3 : 1 2, y 4 : 3 2. Lecture 8 Slide# 6
Proof of Strong Duality Final Dictionary: x i = b i + j Independent a ijx j. z = z + c1 x 1 + + cn+mx n+m Note: ci = 0 if i Basis ci 0 if i Independent. Claim: Set variable y i = (cn+i ). This is a dual feasible solution with optimal value z. Lecture 8 Slide# 7
Relationship Between Primal/Dual Primal Optimal Infeasible Unbounded Optimal Possible Impossible Impossible Dual Infeasible Impossible Possible Possible Unbounded Impossible Possible Impossible Lecture 8 Slide# 8
Primal and Dual Infeasible Primal: Dual: max. x 2 s.t. x 1 1 x 2 1 x 1, x 2 0 min. y 1 y 2 s.t. y 1 0 y 2 1 y 1, y 2 0 Lecture 8 Slide# 9
Complementary Slackness Theorem I Let x 1,..., x n, x n+1,..., x n+m }{{} slack Let y 1,..., y m, y m+1,..., y m+n }{{} Slack Variables: slack be a primal feasible. be dual feasible. x n+i = b i n k=1 a ikx j y m+j = c k m k=1 a kjy j The following conditions are necessary and sufficient for x and y to be optimal solutions to the primal and dual: x i y m+i = 0, j = 1,..., n y j x n+j = 0, i = 1,..., m Proof: x is a primal feasible solution. Therefore, A x b, x 0. Lecture 8 Slide# 10
Complementary Slackness Theorem II Similarly, y is dual feasible. Therefore, Applying these facts, we obtain A t y c, y 0. c t x (A t y) t x( y t A x) y t b x and y can be optimal solutions for primal/dual resp. iff the inequalities above all hold with equalities. Therefore, y t b = y ta x (1) c t x = y t A x (2) Lecture 8 Slide# 11
Complementary Slackness Theorem III From the equation (1), we obtain y t ( b Ax) = 0, or m y i (b i A i x) = 0. i=1 Note that y i 0 and b i A i x 0. Sum of non-negative terms is zero iff each of the individual terms are zero. Therefore y i (b i A i x) = 0 Note that b i A i x is just the slack variable x n+i. Considering the first equation will derive the remainder of the required complementary slackness conditions. Lecture 8 Slide# 12
Duality: Interpretation Diet problem: Minimize cost, while satisfying dietary constraints. FX ($2) FY ($5) FZ ($ 15) Carbs 20 1 1 Protein 1 30 40 Vitamins 1 10 5 1. At least 200units of carbs. 2. At least 50 units of protien. 3. At least 40 units of vitamins. 4. No more than 20 units of FX. 5. No more than 10 units of FY. 6. No more than 5 units of FZ. Lecture 8 Slide# 13
Diet Problem max. 2x 1 5x 2 15x 3 20x 1 x 2 x 3 200 CARBS x 1 30x 2 40x 3 50 PROT x 1 10x 2 5x 3 40 VIT x 1 20 LIM FX x 2 10 LIM FY x 3 5 LIM FZ x 1, x 2, x 3 0 Optimal: x 1 9.84, x 2 3, x 3 : 0. Dual: y 1 : 0.07, y 2 : 0, y 3 : 0.5, y 4,..., y 6 : 0. Lecture 8 Slide# 14
Shadow costs max. 2x 1 5x 2 15x 3 20x 1 x 2 x 3 200 CARBS(0.07) x 1 30x 2 40x 3 50 PROT(0) x 1 10x 2 5x 3 40 VIT(0.5) x 1 20 LIM FX 0 x 2 10 LIM FY 0 x 3 5 LIM FZ 0 x 1, x 2, x 3 0 Experiment: Play around with RHS of inequalities and observe solution. Lecture 8 Slide# 15
Shadow Cost (Margin Cost) max. 2x 1 5x 2 15x 3 20x 1 x 2 x 3 200 CARBS(0.07) x 1 30x 2 40x 3 50 PROT(0) x 1 10x 2 5x 3 40 VIT(0.5) x 1 20 LIM FX 0 x 2 10 LIM FY 0 x 3 5 LIM FZ 0 x 1, x 2, x 3 0 CARBS: $0.07, PROT: free, VIT: $0.5 Q1: If dual var. corresponding to a constraint is 0 does that mean that we can drop the constraint and still get the same answer? A1: YES. Q2: Can we, then, change the RHS of the inequality to arbitrary values? A2: NO. Lecture 8 Slide# 16
Sensitivity Theorem Consider Primal Problem: decision variables are x 1,..., x n. maximize c 1 x 1 + + c n x n s.t. a j1 x 1 + + a jn x n b j j {1, 2,..., m} x 1,..., x n 0 Optimal Value: z. Primal Optimal: x 1,..., x n. Dual Optimal: y 1,..., y n. Final simplex dictionary is non-degenerate. Modified LP: maximize c 1 x 1 + + c n x n s.t. a j1 x 1 + + a jn x n b j +t j j {1, 2,..., m} x 1,..., x n 0 Optimal Value: There is ɛ > 0 s.t., if t i ɛ, z = z + i t iy i. Shadow Cost: The dual decision variables are interpreted as shadow cost. Lecture 8 Slide# 17
Primal-Dual Simplex Lecture 8 Slide# 18
Primal and Dual Consider the following LP: max. x 1 x 2 2x 1 x 2 4 2x 1 +4x 2 8 x 1 +3x 2 7 x 1, x 2 0 Lecture 8 Slide# 19
Dual LP The dual LP for our example: max. 4y 1 +8y 2 +7y 3 2y 1 +2y 2 +y 3 1 y 1 4y 2 3y 3 1 y 1, y 2, y 3 0 Lecture 8 Slide# 20
Primal-Dual Dictionaries x 3 4 +2x 1 +x 2 x 4 8 +2x 1 4x 2 x 5 7 +x 1 3x 2 z x 1 x 2 y 4 1 2y 1 2y 2 y 3 y 5 1 y 1 +4y 2 +3y 3 w 4y 1 +8y 2 +7y 3 Note: Dual dictionary y 2 enters and y 4 leaves. Lecture 8 Slide# 21
Primal-Dual Dictionaries Complementary pairs: Rule: x 3 y 1, x 4 y 2, x 5 y 3 x 1 y 4, x 2 y 5 v 1 enters and v 2 leaves dual dictionary. v1 c leaves and v 2 c enters primal dictionary. v1 c enters and v 2 c leaves primal dictionary. v 1 leaves and v 2 enters dual dictionary. Lecture 8 Slide# 22
Primal Dual Example y 2.5 y 1.5y 3.5y 4 y 5 3 5y 1 +y 3 2y 4 w 4 12y 1 +3y 3 4y 4 x 3 12 +x 4 +5x 2 x 5 3 +.5x 4 x 2 x 1 4 +.5x 4 +2x 2 z 4.5x 4 3x 2 Lecture 8 Slide# 23
Primal Dual Examples y 3 1 2y 1 2y 2 y 4 y 5 4 7y 1 2y 2 3y 4 w 7 18y 1 6y 2 7y 4 Can you write down the primal dictionary?? Lecture 8 Slide# 24
Dual Simplex Easy transformation of primal to dual and vice versa. Entering variable for dual complementary variable leaves primal dictionary. Leaving variable complementary variable enters primal. Unbounded dual infeasible primal (and vice versa). Lecture 8 Slide# 25
Why Dual Simplex? 1. When primal objective coefficients are all non-positive. Avoid need for an initialization phase. 2. When primal has more variables than constraints. Simplex usually observed to be sensitive to number of variables. 3. As an alternative to the initialization phase! Lecture 8 Slide# 26
Initialization using Dual Simplex Lecture 8 Slide# 27
Example Dual: maximize x 1 +4x 2 subjectto 2x 1 x 2 4 2x 1 +4x 2 8 x 1 +3x 2 7 x1, x2 0 max 4y 1 +8y 2 +7y 3 2y 1 +2y 2 +y 3 1 y 1 4y 2 3y 3 4 y 1, y 2, y 3 0 Problem: Neither primal nor dual initial dictionary is feasible. Lecture 8 Slide# 28
Initialization using Dual Idea: Change of primal objective function can make dual initial dictionary feasible!! Dual: x 1 + 4x 2 x 1 x 2 max 4y 1 +8y 2 +7y 3 2y 1 +2y 2 +y 3 1 y 1 4y 2 3y 3 1 y 1, y 2, y 3 0 The change ensures that dual starts at a feasible dictionary. Lecture 8 Slide# 29
Initialization Using Dual Dualize given problem. If dual starting dictionary is infeasible, then change primal objective to make initial dual dictionary feasible. Perform dual simplex until a dual optimal dictionary is reached. Change final dual dictionary into corresponding primal dictionary. Replace the objective function, taking into account new basis variables. Lecture 8 Slide# 30