ECE 307- Techniques for Engineering Decisions Lecture 4. Dualit Concepts in Linear Programming George Gross Department of Electrical and Computer Engineering Universit of Illinois at Urbana-Champaign
DUALITY Definition: A LP is in smmetric form if all the variables are restricted to be nonnegative and all the constraints are inequalities of the tpe: objective tpe max corresponding inequalit tpe min
DUALITY DEFINITIONS We first define the primal and dual problems max Z = c x st.. min W = b st.. Ax T x A b 0 T c 0 T ( P) ( D)
DUALITY DEFINITIONS The problems (P) and (D) are called the smmetric dual LP problems; we restate them as ( P )
DUALITY DEFINITIONS
EXAMPLE 1: MANUFACTURER TRANSPORTATION PROBLEM shipment cost coefficients R 1 retail stores warehouses R 1 R 2 R 3 W 1 W 1 2 4 3 R 2 W 2 W 2 5 3 4 R 3
EXAMPLE 1: MANUFACTURER TRANSPORTATION PROBLEM We are given that the supplies stored in warehouses W and W 1 2 satisf We are also given the demands needed to be met at the retail stores R, R, and R : 1 2 3
EXAMPLE 1: MANUFACTURER TRANSPORTATION PROBLEM The problem is to determine the least-cost shipping schedule We define the decision variable The shipping costs ma be viewed as
FORMULATION STATEMENT min Z = c x = 2x + 4x + 3x + 5x + 3x + 4x s.t. 2 3 i= 1 j= 1 ij ij x + x + x 11 12 13 11 12 13 21 22 23 300 x + x + x 21 22 23 600 x + 11 21 x 200 x + 12 22 x 300 x 13 23 x 0 i = 1, 2, j = 1, 2, 3 ij + x 400
DUAL PROBLEM SETUP USING SYMMETRIC FORM 2 3 i= 1 j= 1 min Z = c x ij ij s.t. 1 2 3 4 5 x x x x 11 12 13 + 11 21 x x x x 21 22 23 + 12 22 x x + 13 23 300 600 200 300 400 x 0 i = 1,2 j= 1,2,3 ij x x
DUAL PROBLEM SETUP maxw = 300 600 + 200 + 300 + 400 s.t. 1 2 3 4 5 + c = 1 3 11 + c = 1 4 12 + c = 1 5 13 + c = 2 3 21 + c = 2 4 22 + c = 2 5 23 0 i i = 1, 2,...,5 2 4 3 5 3 4
THE DUAL PROBLEM INTERPRETATION The moving compan proposes to the manufacturer to: bu all the 300 units at W 1 at 1 / unit bu all the 600 units at W 2 at 2 / unit sell all the 200 units at R 1 at 3 / unit sell all the 300 units at R 2 at 4 / unit sell all the 400 units at R 3 at 5 / unit To convince the manufacturer to get the business, the mover ensures that the deliver fees cannot exceed the transportation costs the manufacturer would incur (the dual constraints)
THE DUAL PROBLEM INTERPRETATION + c = 1 3 11 + c = 1 4 12 + c = 1 5 13 + c = 2 3 21 + c = 2 4 22 + c = 2 5 23 2 4 3 5 3 4 The mover wishes to maximize profits, i.e., revenues costs dual cost objective function maxw = 300 600 + 200 + 300 + 400 1 2 3 4 5
EXAMPLE 2: FURNITURE PRODUCTS Resource requirements item sales price ($) desks 60 tables 30 chairs 20 lumber board requirements finishing labor carpentr
EXAMPLE 2: FURNITURE PRODUCTS The Dakota Furniture Compan manufacturing: resource desk table chair available lumber board (ft ) 8 6 1 48 finishing (h) 4 2 1.5 20 carpentr (h) 2 1.5 0.5 8 We assume that the demand for desks, tables and chairs is unlimited and the available resources are alread purchased The decision problem is to maximize total revenues
PRIMAL AND DUAL PROBLEM FORMULATION We define decision variables x = number of desks produced x 1 2 = number of tables produced x 3 = number of chairs produced The Dakota problem is max Z = 60x1 + 30x2 + 20x3 s.t. 1 8 x1 + 6 x2 + x3 48 lumber 2 4 x 1 + 2 x 2 + 1.5 x 3 20 finishing 3 2 x1 + 1.5 x2 + 0.5 x3 8 carpentr x,x,x 0 1 2 3
PRIMAL AND DUAL PROBLEM FORMULATION The dual problem is min W = 48 + 20 + 8 s.t. 1 2 3 8 + 4 + 2 60 desk 1 2 3 6 + 2 + 1.5 30 table 1 2 3 + 1.5 + 0.5 20 chair 1 2 3,, 0 1 2 3
PRIMAL AND DUAL PROBLEM FORMULATION max Z = 60 x 1+ 30 x 2 + 20 x 3 1 8 x 1+ 6 x 2 + x 3 48 lumber 2 4 x 1+ 2 x 2 + 1.5x 3 20 finishing 3 2 x 1+ 1.5 x 2 + 0.5x 3 8 carpentr x, x,x 0 1 max W = 48 1+ 20 2 + 8 3 48 1+ 20 2 + 8 3 60 desk 6 1+ 2 2 + 1.5 3 30 table 1+ 1.5 2 + 0. 5 3 20 chair,, 0 1 2 2 3 3
INTERPRETATION OF THE DUAL PROBLEM An entrepreneur wishes to purchase all of Dakota s resources He needs, therefore, to determine the prices to pa for each unit of each resource = price paid for 1 lumber board ft 1 = price paid for 1 h of finishing labor 2 = price paid for 1 h of carpentr labor 3 We solve the Dakota dual problem to determine 1, 2 and 3
INTERPRETATION OF THE DUAL PROBLEM To induce Dakota to sell the raw resources, the resource prices must be set sufficientl high For example, the entrepreneur must offer Dakota at least $ 60 for a combination of resources that consists of 8 ft of lumber board, 4 h of finishing and 2 h of carpentr, since Dakota could use this combination to sell a desk for $ 60: this requirement implies the following dual constraint: 8 + 4 + 2 60 1 2 3
INTERPRETATION OF DUAL PROBLEM In the same wa, we obtain the two additional constraints for a table and for a chair The i th primal variable is associated with the i th constraint in the dual problem statement The j th dual variable is associated with the j th constraint in the primal problem statement
EXAMPLE 3: DIET PROBLEM A new diet requires that all food eaten come from one of the four basic food groups : chocolate cake ice cream soda cheesecake The four foods available for consumption are brownie chocolate ice cream cola pineapple cheesecake
EXAMPLE 3: DIET PROBLEM Minimum requirements for each da are: 500 cal 6 oz chocolate 10 oz sugar 8 oz fat The objective is to minimize the diet costs
EXAMPLE 3: DIET PROBLEM food calories chocolate (oz) sugar (oz) fat (oz) costs (cents) brownie 400 3 2 2 50 chocolate ice cream (scoop) cola (bottle) pineapple cheesecake (piece) 200 2 2 4 20 150 0 4 1 30 500 0 4 5 80
PROBLEM FORMULATION Objective of the problem is to minimize the total costs of the diet Decision variables are defined for each da s purchases x x x x 1 2 3 4 = number of brownies = number of chocolate ice cream scoops = number of bottles of soda = number of pineapple cheesecake pieces
PROBLEM FORMULATION The problem statement is min Z = 50 x + 20 x + 30 x + 80 x s.t. 1 2 3 4 400 x + 200 x + 150 x + 500 x 500 cal 1 2 3 4 3x + 2x 6 oz 1 2 2 x + 2 x + 4 x + 4 x 10 oz 1 2 3 4 2x + 4 x + x + 5x 8 oz 1 2 3 4 x 0 i = i 1,4
EXAMPLE 3: DIET PROBLEM The dual problem is max W = 500 + 6 + 10 + 8 s.t. 1 2 3 4 400 + 3 + 2 + 2 50 1 2 3 4 200 + 2 + 2 + 4 20 1 2 3 4 150 + 4 + 30 1 3 4 500 + 4 + 5 80 1 3 4,,, 0 1 2 3 4 brownie ice - cream soda cheesecake
We consider a salesperson of nutrients who is interested in assuming that each dieter meets dail requirements b purchasing calories, sugar, fat and chocolate as goods The decision is to determine the prices charged i INTERPRETATION OF THE DUAL = price per unit of required nutrient to sell to dieters Objective of the salesperson is to set the prices i so as to maximize revenues from selling to the dieter the dail ration of required nutrients
INTERPRETATION OF DUAL Now, the dieter can purchase a brownie for 50 and have 400 cal, 3 oz of chocolate, 2 oz of sugar and 2 oz of fat The sales price i must be set sufficientl low to entice the buer to get the required nutrients from the brownie: brownie 400 1 + 3 2 + 2 3 + 2 4 50 constraint We derive similar constraints for the ice cream, the soda and the cheesecake
DUAL PROBLEMS T max Z = c x s.t. Ax b x 0 ( P) T min W = b s.t. T A c 0 ( D)
For an feasible for (P ) and an feasible for ( D ) WEAK DUALITY THEOREM x Proof: T T c x b T T T T T A c c A c x Ax T T T T c x Ax b = b
COROLLARY 1 OF THE WEAK DUALITY THEOREM T T x is feasible for P c x b ( ) for an feasible for D ( ) T T c x b = min W for an feasible x for P, T c x min W ( )
COROLLARY 2 OF THE WEAK DUALITY THEOREM T T is feasible for D c x b ( ) for ever feasible x for P ( ) T T T max Z max c x c x b = = for an feasible of D, T b max Z ( )
COROLLARIES 3 AND 4 OF THE WEAK DUALITY THEOREM If ( P ) is feasible and max Z is unbounded, i.e., Z +, then, ( D ) has no feasible solution. If ( D ) is feasible and min Z is unbounded, i.e., Z, then, ( P ) is infeasible.
DUALITY THEOREM APPLICATION Consider the maximization problem ( P)
DUALITY THEOREM APPLICATION The corresponding dual is given b min W = b T s.t. T A c ( D) 0 With the appropriate substitutions, we obtain
DUALITY THEOREM APPLICATION min W = 20 + 20 s.t. 1 2 + 2 1 1 2 2 + 2 1 2 2 + 3 3 1 2 3 + 2 4 1 2 0, 0 1 2
GENERALIZED FORM OF THE DUAL Consider the primal decision x = 1, i = 1, 2, 3,4 ; decision is feasible for (P) with i T Z = c x = 10 The dual decision i = 1, i = 1, 2 is feasible for (D) with T W = b = 40
DUALITY THEOREM APPLICATION Clearl, ( ) ( ) 1 2 3 4 10 40 = 1 2 Z x,x,x,x = W, and so clearl, the feasible decision for (P) and (D) satisf the Weak Dualit Theorem Moreover, we have ( ) corollar 1 10 min W = W 1, 2 ( ) T 1 2 3 4 corollar 2 max Z = Z x, x, x, x b = 40
COROLLARIES 5 AND 6 (P) is feasible and (D) is infeasible, then, (P) is unbounded (D) is feasible and (P) is infeasible, then, (D) is unbounded
EXAMPLE Consider the primal dual problems: st.. Now 1 2 x x x 2 ( P) max Z = x + x 1 + 2 + 3 2x 1+ x 2 x 3 1 x, x, x 0 1 2 3 st.. x = 0 is feasible for ( P ) = 2 1 + 2 min W 1 2 2 1 1 + 2 1 1 2 0 1, 2 0 ( D)
EXAMPLE but is impossible for (D) since it is inconsistent with 1, 2 0 Since (D) is infeasible, it follows from Corollar 5 that Z You are able to show this result b solving (P) using the simplex scheme
OPTIMALITY CRITERION THEOREM We consider the primal-dual problems (P) and (D) with x 0 is feasible for ( P) 0 is feasible for ( D) T 0 T 0 c x = b We next provide the proof: x 0 0 is feasible for ( P) is feasible for ( D) Weak Dualit Theorem T 0 T 0 c x b
OPTIMALITY CRITERION THEOREM but we are given that T 0 T 0 c x = b and so it follows that feasible x with 0 feasible 0 and so x is optimal ; T T 0 T 0 c x b = c x similarl, 0 feasible with x feasible and so it follows that T T 0 T 0 b c x = b 0 is optimal
MAIN DUALITY THEOREM (P) is feasible and (D) is feasible; then, x feasible for ( P) which is optimal and feasible for ( D) which is optimal such that T T c x = b
COMPLEMENTARY SLACKNESS CONDITIONS * * x and are optimal for (P) and (D), respectivel, if and onl if We prove this equivalence result b defining the slack variables and such that and are feasible; at the optimum, x
COMPLEMENTARY SLACKNESS CONDITIONS where the optimal values of the slack variables u and v depend on the optimal values x and Now,
COMPLEMENTARY SLACKNESS CONDITIONS This implies that We need to prove optimalit which is true if and onl if
COMPLEMENTARY SLACKNESS CONDITIONS However, x, are optimal Main Also, Dualit Theorem T T T T c x = b u + v x = 0 T T T T u v x 0 b c x + = = Optimalit Criterion Theorem x is optimal for ( P) and is optimal for ( D)
COMPLEMENTARY SLACKNESS CONDITIONS Note that x,, u, v > 0 component - wise each element 0 + = = = T u v x 0 i u i 0 i m At the optimum, and v x = 0 j = 1,..., n j j 1,..., and
COMPLEMENTARY SLACKNESS CONDITIONS Hence, for i = 1, 2,, m and Similarl for j = 1, 2,, n and m i = 1 n i i ij j j=1 > 0 b = a x m i ij i i j = 1 b a x > 0 = 0 m i ji i j i=1 x > 0 a = c a c > 0 x = 0 ji i j j
EXAMPLE max Z = x + 2x + 3x + 4x 1 2 3 4 st.. x + 2 x + 2 x + 3 x 20 1 2 3 4 ( P ) 2 x + x + 3 x + 2 x 20 1 2 3 4 x 0 i = i 1,...,4
EXAMPLE min W = 20 + 20 st.. 1 2 + 2 1 1 2 2 + 2 1 2 ( D) 2 + 3 3 1 2 3 + 2 4 1 2, 0 1 2
EXAMPLE x, * * optimal ( 20 2 2 3 ) * * * * * x x x x = 0 1 1 2 3 4 ( 20 2 3 2 ) * * * * * x x x x = 0 2 1 2 3 4 * 1.2 = 0.2 is given as an optimal solution with min W = 28
EXAMPLE so that x + 2 x + 2 x + 3 x = 20 2 x + x + 3 x + 2 x = 20 + 2 = 1.2 + 0.4 > 1 x = 0 2 + = 2.4 + 0.2 > 2 x = 0 2 + 3 = 2.4 + 0.6 = 3 3 + 2 = 3.6 + 0.4 = 4 2 x + 3 x = 20 x = 4 3 4 3 3 x + 2 x = 20 x = 4 3 4 4 1 2 3 4 1 2 3 4 1 2 1 1 2 2 1 2 1 2
COMPLEMENTARY SLACKNESS CONDITION APPLICATIONS Ke uses of c.s. conditions are finding optimal (P) solution given optimal (D) solution and vice versa verification of optimalit of solution (whether a feasible solution is optimal) We can start with a feasible solution and attempt to construct an optimal dual solution; if we succeed, then the feasible primal solution is optimal
DUALITY T max Z = c x s.t. Ax b x 0 ( P) T min W = b s.t. T A c 0 ( D)
DUALITY Suppose the primal problem is minimization, then, T min Z = c x (P) s.t. Ax b x 0 T max W b (D) s.t. A T = c 0
INTERPRETATION The economic interpretation is * T * T * * Z max Z c x b W minw b i * i = = = = = constrained resource quantities, i = optimal dual variables Suppose, we change * bi bi + bi Z = i bi In words, the optimal dual variable for each primal constraint gives the net change in the optimal value of the objective function Z for a one unit change in the constraint on resources 1, 2,..., m
INTERPRETATION Economists refer to the dual variable as the shadow price on the constraint resource The shadow price determines the value/worth of having an additional quantit of a resource In the previous example, the optimal dual variables indicate that the worth of another unit of resource 1 is 1.2 while that of another unit of resource 2 is 0.2
GENERALIZED FORM OF THE DUAL We start out with T max Z = c x s.t. Ax = b ( P) x 0
GENERALIZED FORM OF THE DUAL To find ( D), we first put (P) in smmetric form Ax b Ax b x 0 A b x A b smmetric form
GENERALIZED FORM OF THE DUAL Let We rewrite the problem as st.. T min W b T A c is unsigned The c.s. conditions appl =
EXAMPLE 5: THE PRIMAL max Z = x x + x x st.. x + x + x + x = x x x 4 ( P) 1 2 3 4 1 1 2 3 4 2 1 3 2 4 2 x x 5 3 6 3 7 4 x x, x 0 1 4 x, x unsigned 2 3 8 8 4 4 2 10
EXAMPLE 5: THE DUAL minw = 8 + 8 + 4 + 4 + 4 + 2 + 10 st.. 1 2 3 4 5 6 7 π x + 1 1 2 1 x + = 1 ( D) 2 1 3 4 x + = 3 1 5 6 1 x + 4 7,..., 0 2 7 1 1 unsigned
EXAMPLE 5: c.s. conditions We are given that is optimal for (P) x 8 4 = 4 0 Then the c.s. conditions obtain ( ) x + = 0 * * * 1 1 2 1
so that EXAMPLE 5: c.s. conditions x = 8 > 0 + = 1 1 1 2 The other c.s. conditions obtain 4 i a ij x j b i = 0 j = 1 Now, x 4 = 0 implies x 4 10 < 0 and so 7 = 0
EXAMPLE 5: c.s. conditions Also, * x 3 = 4 implies * 6 = 0 Similarl, the c.s. conditions 7 * * x j a ji i c j = 0 i= 1 have implications on the * i variable
EXAMPLE 5: c.s. conditions Since x * 2 = 4, then we have * 3 = 0 Now, with * 7 = 0 we have * 1 > 1 Since, W = T b we have = 1 * * 2 1
EXAMPLE 5 Suppose * 1 = 1 and so, * 2 = 0 Furthermore, implies + = 1 = 1 * * * * 1 3 4 4 * 4 = 2
EXAMPLE 5 Also * * * 1 + 5 6 = 1 implies 1 + = 1 * 5 and so * 5 = 0
EXAMPLE 5 Therefore ( *) ( 8)( 1) ( 8)( ) ( 4)( ) ( 4)( 2) W = + 0 + 0 + + ( 4)( 0) + ( 2)( 0) + ( 10)( 0) = 16 and so ( ) ( ) * * W = 16 = Z optimalit of P and D
PRIMAL DUAL TABLE primal (maximize) dual (minimize) A ( coefficient matrix ) A T ( transpose of the coefficient matrix ) b ( right-hand side vector ) b ( cost vector ) c ( price vector ) c ( right hand side vector ) i th constraint is = tpe the dual variable i is unrestricted in sign i th constraint is tpe the dual variable i 0 i th constraint is tpe the dual variable i 0 x j is unrestricted j th dual constraint is = tpe x j 0 j th dual constraint is tpe x j 0 j th dual constraint is tpe