CSCI5654 (Linear Programming, Fall 013) Lecture-7 Duality Lecture 7 Slide# 1 Lecture 7 Slide# Linear Program (standard form) Example Problem maximize c 1 x 1 + + c n x n s.t. a j1 x 1 + + a jn x n b j j {1,,..., m} x 1,..., x n 0 Problem has n variables and m constraints. maximize x 1 + x s.t. x 1 3 x 3 x 1 + x 1 x 1 + x 5 Lecture 7 Slide# 3 Lecture 7 Slide# 4
Bounds on Optimal Value of LP Bounds maximize x 1 + x s.t. x 1 3 x 3 x 1 + x 1 x 1 + x 5 Goal: Find lower and upper bounds on the solution to the LP. Lower Bound: Find a feasible solution x. Then x is a lower bound on optimum. (why?). Eg. (x 1 : 3, x : ) is feasible. Therefore, z = x 1 + x 7. Upper Bounds: Can we find upper bounds to optimum? Lecture 7 Slide# 5 Lecture 7 Slide# 6 Example (continued) Example (continued) Note: maximize x 1 + x z s.t. x 1 3 C1 x 3 C x 1 + x 1 C3 x 1 + x 5 C4 x 1 3 C1 x 3 x 1 + x???. C z Q: Using the inequality above what can you say about the solution to the LP? Note: maximize x 1 + x z s.t. x 1 3 C1 x 3 C x 1 + x 1 C3 x 1 + x 5 C4 x 1 3 C1 x 3 x 1 + x 3 + 3 9. C z Q: Using the inequality above what can you say about the solution to the LP? A: 9 is an upper bound to the solution (i.e., z 9). Lecture 7 Slide# 7 Lecture 7 Slide# 8
Finding Upper Bounds Strategy: Multiply constraint rows by positive quantities. maximize x 1 + x z s.t. x 1 3 C1 x 3 C x 1 + x 1 C3 x 1 + x 5 C4 C1 + C z 9. C4 z 10. Question #1: 1 C3 + 3 C z? Question #: C3 + C1 z? Question #3: Can you think of other combinations? Upper Bounds to LP Solution maximize c 1 x 1 + + c n x n z s.t. a 11 x 1 + + a 1n x n b 1 C1 a 1 x 1 + + a n x n b C.... a m1 x 1 + a mn x n b m Cm x 1,..., x n 0 Consider y 1 C1 + y C + + y m Cm. Q: What are the conditions on y 1,..., y m so that this combination upper bounds z? Lecture 7 Slide# 9 Lecture 7 Slide# 10 Upper Bounds to LP Solution Best upper bound on LP Solution Conditions: Upper Bound: b 1 y 1 +... + b m y m a 11 y 1 + a 1 y + + a m1 y m c 1 a 1 y 1 + a y + + a m y m c a 1n y 1 + a 1n y + + a mn y m c n y 1, y,..., y m 0 Q: What values of y 1,..., y m can yield best bound on optimum??. minimize b 1 y 1 + b y + + b m y m s.t. a 11 y 1 + a 1 y + + a m1 y m c 1 a 1 y 1 + a y + + a m y m c Note: This is called the dual problem. a 1n y 1 + a 1n y + + a mn y m c n y 1, y,..., y m 0. Lecture 7 Slide# 11 Lecture 7 Slide# 1
Dual Problem Primal Problem: decision variables are x 1,..., x n. maximize c 1 x 1 + + c n x n s.t. a j1 x 1 + + a jn x n b j j {1,,..., m} x 1,..., x n 0 Dual Problem: decision variables are y 1,..., y m. minimize b 1 y 1 + + b m y m s.t. a 1j y 1 + + a mj y m c j j {1,,..., n} y 1,..., y n 0 Dual Problem (matrix) Primal LP: Dual LP: (dual in standard form) c x s.t. A x b x 0 max. min. max b y A t y c y 0 b y A t y c y 0 Exercise: Use the information in this frame to prove: The dual of dual problem is the same as the original primal problem. Lecture 7 Slide# 13 Lecture 7 Slide# 14 Example Example (dual) Q: Write down the dual problem. maximize x 1 + x s.t. x 1 3 y 1 x 3 y x 1 + x 1 y 3 x 1 + x 5 y 4 min. 3y 1 + 3y + y 3 + 5y 4 s.t. y 1 y 3 + y 4 1 y + y 3 + y 4 y 1, y, y 3, y 4 0 Dual optimal: y 1 = 0, y = 0, y 3 = 1, y 4 = 3 yields optimal value 8. Lecture 7 Slide# 15 Lecture 7 Slide# 16
Example (dictionary) Example (final dictionary) x 3 = 3 x 1 x 4 = 3 x x 5 = 1 + x 1 x x 6 = 5 x 1 x z = 0 + x 1 + x x 1 = + x 5 x 6 x = 3 x 5 + x 6 x 3 = x 4 = z = 8 1 x 5 3 x 6 Lecture 7 Slide# 17 Lecture 7 Slide# 18 Dual Certificate Dual Certificate Situation: I ask X to solve a large LP for me. After sometime X presents a solution x = (x 1,..., x n) that is claimed to be optimal. Q: How do I verify that X s solution is indeed optimal? A1: OK solve the LP yourself!! (not acceptable) Situation: I ask X to solve a large LP for me. After sometime X presents a solution x = (x 1,..., x n) that is claimed to be optimal. Q: How do I verify that X s solution is indeed optimal? A: Provide dual variables y : (y 1,..., y n ). 1. Check primal feasibility of x.. Check dual feasibility of y. 3. Check that the objective values are the same. Primal-Dual Certificate: In the presence of both, each solution can serve as a certificate to the optimality of the other. Lecture 7 Slide# 19 Lecture 7 Slide# 0
Primal dual correspondences Variable to constraint correspondence: Dual variable (y j ) Primal variable (x i ) Correspondence in dictionary: Primal constraint A j x b j Dual constraint A i y c i Dual variable y j Primal slack variable x n+j. Primal variable (x i ) Dual dictionary slack variable y m+i. Strong Duality Theorem: If x 1,..., x m is primal optimal then there exists a dual optimal solution y 1,..., y m that satisifies c i xi i = j b j y j. ( if primal has optimal solution then the dual has optimum with the same value as primal). Proof: This is theorem 5.1 in Chvátal (pages 58-59). Lecture 7 Slide# 1 Lecture 7 Slide# Example Example (final dictionary) maximize x 1 + x s.t. x 1 3 x 3 x 1 + x 1 x 1 + x 5 x 1 = + x 5 x 6 x = 3 x 5 + x 6 x 3 = x 4 = z = 8 1 x 5 3 x 6 Insight: Dual variables correspond to primal slack variables. x 3 y 1, x 4 y, x 5 y 3, x 6 y 4. Read off dual solution by from objective row of final dictionary: y 1 : 0, y : 0, y 3 : 1, y 4 : 3. Lecture 7 Slide# 3 Lecture 7 Slide# 4
Proof of Strong Duality Relationship Between Primal/Dual Final Dictionary: x i = b i + j Independent a ijx j. z = z + c1 x 1 + + cn+mx n+m Note: ci = 0 if i Basis ci 0 if i Independent. Claim: Set variable y i = (cn+i ). This is a dual feasible solution with optimal value z. Primal Optimal Infeasible Unbounded Optimal Possible Impossible Impossible Dual Infeasible Impossible Possible Possible Unbounded Impossible Possible Impossible Lecture 7 Slide# 5 Lecture 7 Slide# 6 Primal and Dual Infeasible Primal: Dual: max. x s.t. x 1 1 x 1 min. y 1 y s.t. y 1 0 y 1 y 1, y 0 Lecture 7 Slide# 7