Algorithms and Theory of Computation Lecture 13: Linear Programming (2) Xiaohui Bei MAS 714 September 25, 2018 Nanyang Technological University MAS 714 September 25, 2018 1 / 15
LP Duality Primal problem (P) max 13A + 23B s.t. 5A + 15B 480 4A + 4B 160 35A + 20B 1190 A, B 0 Goal: find a lower bound on optimal value. Easy. Any feasible solution provides one. Ex 1. (A, B) = (34, 0) = z 442 Ex 2. (A, B) = (12, 28) = z 800 Nanyang Technological University MAS 714 September 25, 2018 2 / 15
LP Duality Primal problem (P) max 13A + 23B s.t. 5A + 15B 480 4A + 4B 160 35A + 20B 1190 Goal: find a upper bound on optimal value. A, B 0 Ex 1. Multiply 2nd inequality by 6: 24A + 24B 960 = z = 13A + 23B 24A + 24B 960 Ex 2. Add 1 times 1st inequality to 2 times 2nd inequality = z = 13A + 23B 13A + 23B 800 Recall lower bound: (A, B) = (12, 28) = z 800 Combine upper and lower bounds: z = 800 Nanyang Technological University MAS 714 September 25, 2018 3 / 15
LP Duality Idea: Add nonnegative combination (C, H, M) of the constraints s.t. 13A + 23B (5C + 4H + 35M)A + (15C + 4H + 20M)B 480C + 160H + 1190M Dual problem. Find best such upper bound. Dual problem (D) min 480C + 160H + 1190M s.t. 5C + 4H + 35M 13 15C + 4H + 20M 23 C, H, M 0 Nanyang Technological University MAS 714 September 25, 2018 4 / 15
LP Duality: Economic Interpretation Brewer: find optimal mix of beer and ale to maximize profits. (P) max 13A + 23B s.t. 5A + 15B 480 4A + 4B 160 35A + 20B 1190 A, B 0 Entrepreneur: buy individual resources from brewer at min cost. C, H, M = unit price for corn, hops, malt. Brewer won t agree to sell resources if 5C + 4H + 35M < 13. (D) min 480C + 160H + 1190M s.t. 5C + 4H + 35M 13 15C + 4H + 20M 23 C, H, M 0 Nanyang Technological University MAS 714 September 25, 2018 5 / 15
LP Duals Canonical form. Primal (P) max c T x s.t. Ax b x 0 Dual (D) min y T b s.t. A T y c y 0 Property The dual of the dual is the primal. Nanyang Technological University MAS 714 September 25, 2018 6 / 15
Taking Duals LP dual recipe. Primal (P) maximize minimize Dual (D) a i x = b i unrestricted constraints a i x b i y i 0 variables a i x b i y i 0 x j 0 a T y c j variables x j 0 a T y c j constraints unrestricted a T y = c j Nanyang Technological University MAS 714 September 25, 2018 7 / 15
Weak Duality Theorem Theorem If x is feasible for the primal and y is feasible for the dual, then c j x j b i y i. j i Proof. c j x j j j ( i y i a ij )x j = i,j y i a ij x j = i ( j a ij x j )y i i b i y i Nanyang Technological University MAS 714 September 25, 2018 8 / 15
Gap or No Gap? An important question Is there a gap between the largest primal value and the smallest dual value? Answer is provided by the Strong Duality Theorem Nanyang Technological University MAS 714 September 25, 2018 9 / 15
Strong Duality Theorem Theorem If the primal problem has an optimal solution x, then the dual also has an optimal solution y, and c j x j = b i y i. j i Paraphrase. If primal has an optimal solution, then there is no duality gap. Nanyang Technological University MAS 714 September 25, 2018 10 / 15
Duality Gap There are four possibilities. 1 Primal optimal, dual optimal (no gap) 2 Primal unbounded, dual infeasible (no gap) 3 Primal infeasible, dual unbounded (no gap) 4 Primal infeasible, dual infeasible (infinite gap) Nanyang Technological University MAS 714 September 25, 2018 11 / 15
Applications: Max-Flow Formulate as linear programming. s 3 2 a 1 b 1 3 t (P) max f sa + f sb s.t. f sa 3 f sb 2 f ab 1 f at 1 f bt 3 f sa f ab f at = 0 f sb + f ab f bt = 0 Nanyang Technological University MAS 714 September 25, 2018 12 / 15
Applications: Max-Flow Formulate as linear programming. s 3 2 a 1 b 1 3 t (D) min 3y sa + 2y sb + y ab + y at + 3y bt s.t. y sa + u a 1 y sb + u b 1 y ab u a + u b 0 y at u a 0 y bt u b 0 y sa, y sb, y ab, y at, y bt 0 Dual describes the min-cut problem! Nanyang Technological University MAS 714 September 25, 2018 13 / 15
History of LP 1947. Simplex algorithm. (Dantzig) 1950. Applications in many fields. military logistics operation research control theory filter design structural optimization 1979. Ellipsoid algorithm. (Khachian) first polynomial time algorithm for LP geometric divide-and-conquer solvable in O(mn 3 L) arithmetic operations L = number of bits to encode input theoretical result, not remotely practical 1984. Projective scaling algorithm. (Karmarkar) O(n 3.5 L) running time efficient implementations possible 1990. Interior point methods. O(n 3 L) running time and practical Nanyang Technological University MAS 714 September 25, 2018 14 / 15
Part I Summary Algorithm Design Techniques greedy divide and conquer dynamic programming Problems Search, Sorting, LCS Graphs traversing graphs (BFS, DFS) minimum spanning trees shortest paths network flows Linear Programming Data Structures lists, stacks, queues, union find, priority queues Nanyang Technological University MAS 714 September 25, 2018 15 / 15