4.6 Linear Programming duality

Size: px

Start display at page:

Download "4.6 Linear Programming duality"

Domenic Watts
5 years ago
Views:

1 4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP Different spaces and objective functions but in general same optimal value! Example: the value of a maximum feasible flow is equal to the capacity of a cut (separating the source s and the sink t) of minimum capacity. E. Amaldi Fondamenti di R.O. Politecnico di Milano 1

2 Motivation: estimate of the optimal value Given max z = 4x 1 + x 2 + 5x 3 + 3x 4 x 1 x 2 x 3 + 3x 4 1 5x 1 + x 2 + 3x 3 + 8x x 1 + 2x 2 + 3x 3 5x 4 3 x i 0 i = 1,, 4 find an estimate of the optimal value z * Lower bound: (0,0,1,0) z * 5 (2,1,1,1/3) z * 15 (3,0,2,0) z * 22 Even if we are lucky, we are not sure it is the optimal solution! E. Amaldi Fondamenti di R.O. Politecnico di Milano 2

3 Upper bound: By multiplying the 2 constraint by 5/3 we obtain an inequality that dominates the objective function: 4x 1 + x 2 + 5x 3 + 3x 4 25/3x 1 + 5/3x 2 + 5x /3x 4 275/3 z * 275/3 Adding 2 and 3 we obtain: 4x 1 + 3x 2 + 6x 3 + 3x 4 58 feasible solution z * 58 better upper bound E. Amaldi Fondamenti di R.O. Politecnico di Milano 3

4 General strategy: linearly combine the constraints with nonnegative multiplicative factors ( i-th constraint multiplied by y i 0 ) first case: y 1 =0, y 2 =5/3, y 3 =0 second case: y 1 =0, y 2 =1, y 3 =1 In general y 1 (x 1 x 2 x 3 +3x 4 ) + y 2 (5x 1 + x 2 + 3x 3 + 8x 4 ) + y 3 (-x 1 + 2x 2 + 3x 3 5x 4 ) y y 2 + 3y 3 E. Amaldi Fondamenti di R.O. Politecnico di Milano 4

5 which is equivalent to: (y 1 + 5y 2 y 3 ) x 1 + (-y 1 + y 2 + 2y 3 ) x 2 + (-y 1 + 3y 2 + 3y 3 ) x 3 + (3y 1 + 8y 2 5y 3 ) x 4 y y 2 + 3y 3 (*) NB: y i 0 so that the inequality direction is unchanged In order to use the left hand side of (*) as upper bound on z = 4x 1 + x 2 + 5x 3 + 3x 4 E. Amaldi Fondamenti di R.O. Politecnico di Milano 5

6 We must have z = 4x 1 + x 2 + 5x 3 + 3x 4 y 1 + 5y 2 y 3 4 -y 1 + y 2 + 2y 3 1 -y 1 + 3y 2 + 3y 3 5 3y 1 + 8y 2 5y 3 3 y i 0, i = 1, 2, 3 In such a case, any feasible solution x satisfies 4x 1 + x 2 + 5x 3 + 3x 4 y y 2 + 3y 3 in particular: z * y y 2 + 3y 3 E. Amaldi Fondamenti di R.O. Politecnico di Milano 6

7 Since we look for the best possible upper bound on z * : (D) min y y 2 + 3y 3 y 1 + 5y 2 y 3 4 -y 1 + y 2 + 2y 3 1 -y 1 +3y 2 + 3y 3 5 3y 1 +8y 2 5y 3 3 y i 0 i = 1, 2, 3 The problem (D) is the dual problem, while the original problem is the primal problem E. Amaldi Fondamenti di R.O. Politecnico di Milano 7

8 In matrix form: max z = c T x Primal (P) Ax b x 0 min w = b T y Dual (D) A T y c y 0 o y T A c T E. Amaldi Fondamenti di R.O. Politecnico di Milano 8

9 Dual problem max z = c T x min w = b T y (P) Ax b x 0 (D) A T y c y 0 Dual of an LP in standard form? min z = c T x Ax = b x 0 E. Amaldi Fondamenti di R.O. Politecnico di Milano 9

10 Standard form: min z = c T x max -c T x (P) Ax = b x 0 with A an m n matrix A x b -A -b x 0 dual A x b A T min (b T b T ) y 1 y 2 y 1 y 2 (A T A T ) -c y 1 0, y 2 0 y = y 1 y 2 E. Amaldi Fondamenti di R.O. Politecnico di Milano 10

11 min (b T b T ) y 1 y 2 y 1 y 2 (A T A T ) -c y 1 0, y 2 0 (D) max w = b T y A T y c y R m min -b T (y 2 y 1 ) y y 2 y 1 unrestricted in sign! -A T (y 2 y 1 ) -c y 1 0, y 2 0 E. Amaldi Fondamenti di R.O. Politecnico di Milano 11

12 Example: max x 1 + x 2 x 1 x 2 2 (P) 3x 1 + 2x 2 12 x 1, x 2 0 max x 1 + x 2 x 1 - x 2 2-3x 1-2x 2-12 x 1, x 2 0 dual min 2y 1-12y 2 y 1-3y 2 1 -y 1-2y 2 1 y 1, y 2 0 E. Amaldi Fondamenti di R.O. Politecnico di Milano 12

13 General transformation rules Primal (min) Dual (max) m constraints n variables coefficients obj. fct right hand side A equality constraints unrestriced variables inequality constraints ( ) variables 0 ( 0) m variables n constraints right hand side coefficients obj. fct E. Amaldi Fondamenti di R.O. Politecnico di Milano 13 A T unrestriced variables equality constraints variables 0 ( 0) inequality constraints ( )

14 Example: using the above rules max x 1 + x 2 x 1 x 2 2 (P) 3x 1 + 2x 2 12 x 1, x 2 0 dual min 2y y 2 y 1 + 3y 2 1 -y 1 + 2y 2 1 y 1 0, y 2 0 y~ 1 -y 1 min 2y 1-12y 2 y 1-3y ~ ~ y 1-2y ~ 2 1 y 1 0, ~ y 2 0 E. Amaldi Fondamenti di R.O. Politecnico di Milano 14

15 Exercize: (P) min 10x x x 3 2x 1 x 2 1 x 2 + x 3 2 x 1 x 3 = 3 x 1 0, x 2 0, x 3 unrestricted Dual? E. Amaldi Fondamenti di R.O. Politecnico di Milano 15

16 Property: the dual of the dual problem coincides with the primal problem. NB: it doesn't matter which one is a max or min problem (P) min z = c T x Ax b x 0 (D) max w = b T y A T y c y 0 E. Amaldi Fondamenti di R.O. Politecnico di Milano 16

17 Weak duality theorem Given X {x : Ax b, x 0} ø and Y {y : A T y c, y 0} ø, for each feasible solution x X of (P) and each feasible solution y Y of (D) we have b T y c T x E. Amaldi Fondamenti di R.O. Politecnico di Milano 17

18 Proof Given x X and y Y, we have Ax b, x 0 and A T y c, y 0which imply that b T y x T A T y x T c = c T x x T A T c E. Amaldi Fondamenti di R.O. Politecnico di Milano 18

19 Consequence: If x * is a feasible solution for the primal problem (P) ( x * X ), y * is a feasible solution (D) ( y * Y ), and the values of the respective objective functions coincide then c T x * = b T y *, x * is optimal for (P) and y * is optimal for (D). E. Amaldi Fondamenti di R.O. Politecnico di Milano 19

20 Strong duality theorem If X = {x : Ax b, x 0} ø and min{c T x : x X} is finite, there exist x * X and y * Y such that c T x * = b T y *. min{ c T x : x X } = max{ b T y : y Y } optimum z = c T x x X feasible for (P) z * = w * w = b T y y Y feasible for (D) E. Amaldi Fondamenti di R.O. Politecnico di Milano 20

21 Proof Derive an optimal solution of (D) from one of (P) Given min c T x max y T b (P) Ax = b x 0 (D) y T A c T y R m and x* is an optimal feasible solution of (P) x x* = * B x * con N x * B = B -1 b x * N = 0 provided (after a finite # of iterations) by the Simplex algorithm with Bland's rule. E. Amaldi Fondamenti di R.O. Politecnico di Milano 21

22 Let us consider y T c T B B -1 y is a feasible solution of (D): c T N = c T N (c T B B -1 )N = c T N y T N 0 T reduced costs of the nonbasic variables optimality of x * y T N c T N E. Amaldi Fondamenti di R.O. Politecnico di Milano 22

23 c T B = c T B (c T B B -1 )B = c T B y T B = 0 T y T B c T B I reduced costs of the basic variables y is an optimal solution of (D): y T b = (c T B B -1 )b = c T B (B -1 b) = c T B x * B = c T x * hence y = y * E. Amaldi Fondamenti di R.O. Politecnico di Milano 23

24 Corollary For any pair of primal-dual problems (P) and (D), only four cases can arise: P D optimal solution unbounded LP Infeasible LP optimal solution 1) unbounded LP infeasible LP 2) 3) 4) E. Amaldi Fondamenti di R.O. Politecnico di Milano 24

25 Strong duality theorem 1) Weak duality theorem 2) and 3) 4) can arise: min -4x 1 2x 2 -x 1 + x 2 2 (P) x 1 x 2 1 x 1, x 2 0 max 2y 1 + y 2 -y 1 + y 2-4 (D) y 1 y 2-2 y 1, y 2 0 empty feasible regions E. Amaldi Fondamenti di R.O. Politecnico di Milano 25

26 Economic interpretation The primal and dual problems correspond to two complementary point of views on the same market Diet problem: n aliments j=1,, n m nutrients i=1,, m (vitamines, ) a ij b i c j quantity of i-th nutrient in one unit of j-th aliment requirement of i-th nutrient cost of one unit of j-th aliment E. Amaldi Fondamenti di R.O. Politecnico di Milano 26

27 (P) min n j=1 n j=1 c j x j a ij x j b i i= 1,..., m x j 0 j= 1,...,n (D) max m i=1 m i=1 b i y i a ij y i c j j= 1,..., n y i 0 i= 1,...,m E. Amaldi Fondamenti di R.O. Politecnico di Milano 27

28 Interpretation of the dual problem: A company that produces pills of the m nutrients needs to decide the nutrient unit prices y i so as to maximize income. If the costumer buys nutrient pills he will buy b i units for each i, 1 i m The price of the nutrient pills must be competitive: m i=1 a ij y i c j j= 1,...,n cost of the pills that are equivalent to 1 unit of j-th aliment E. Amaldi Fondamenti di R.O. Politecnico di Milano 28

29 If both linear programs (P) and (D) admit a feasible solution, the strong duality theorem implies that z * = w * An equilibrium exists (two alternatives with the same cost) NB: Strong connection with Game theory (zero-sum games) E. Amaldi Fondamenti di R.O. Politecnico di Milano 29

30 Optimality conditions Given min z = c T x max w = b T y (P) X Ax b x 0 (D) Y y T A c T y 0 two feasible solutions x * X and y * Y are optimal y * T b = c T x * If x j and y i are unknown, it is a single equation in n+m unknowns! E. Amaldi Fondamenti di R.O. Politecnico di Milano 30

31 Since y *T b y *T Ax * c T x *, we have Ax * c T y *T b = y *T Ax * and y *T Ax * = c T x * therefore y *T (Ax * - b) = 0 and (c T - y *T A) x * = 0 0 T 0 0 T 0 m+n equations in n+m unknowns necessary and sufficient optimality conditions! E. Amaldi Fondamenti di R.O. Politecnico di Milano 31

32 Complementary slackness conditions x * X and y * Y are optimal solutions of, respectively, (P) and (D) if and only if i-th row of A slack s i of i-th constraint of (P) y * i (a T i x * - b i ) = 0 i = 1,, m (c T j y *T A j ) x * j = 0 j = 1,, n slack of j-th constraint of (D) j-th column of A At optimality, the product of each variable with the corresponding slack variable of the relative dual is = 0. E. Amaldi Fondamenti di R.O. Politecnico di Milano 32

33 Economic interpretation for the diet problem n a j= 1 ij x * j > b i y * i = 0 If the optimal diet includes an excess of i-th nutrient, the costumer is not willing to pay y * i > 0 * i n j=1 ij * j y > 0 a x = b i If the company selects a price y * i > 0, the costumer must not have an excess of i-th nutrient E. Amaldi Fondamenti di R.O. Politecnico di Milano 33

34 m y i= 1 * i a ij < c j x * j = 0 it is not convenient for the costumer to buy aliment j price of the pills equivalent to the nutrients contained in one unit of j-th aliment is lower than the price of the aliment * j m i=1 * i x > 0 y a = ij c j If costumer includes the j-th aliment in optimal diet, the company must have selected competitive prices y * i (the price of the nutrients in pills contained in a unit of j-th aliment is not lower than c j ) E. Amaldi Fondamenti di R.O. Politecnico di Milano 34

35 Example: min 13x x 2 + 6x 3 s.v. 5x 1 + x 2 + 3x 3 = 8 (P) 3x 1 + x 2 = 3 x 1, x 2, x 3 0 max 8y 1 + 3y 2 s.v. 5y 1 + 3y 2 13 (D) y 1 + y y 1 6 Verify that the feasible x * = (1, 0, 1) is an optimal, non degenerate, solution of (P). Suppose it is true and derives, via the complementary slackness conditions, the corresponding optimal solution of (D). E. Amaldi Fondamenti di R.O. Politecnico di Milano 35

36 Since (P) is in standard form, the conditions y i* (a T i x * - b i ) = 0 are automatically satisfied i, 1 i 2. The condition (c T j y *T A j ) x * j = 0 is satisfied for j = 2 because x * 2= 0. Since x * 1 > 0 and x * 3 > 0, we obtain the conditions: 5y 1 + 3y 2 = 13 3y 1 = 6 and hence the optimal solution y * 1 = 2 and y * 2 = 1 of (D) with b T y * =19 = c T x *. E. Amaldi Fondamenti di R.O. Politecnico di Milano 36

Introduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs

Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.1-4.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following