Sensitivity Analysis and Duality in LP

Sensitivity Analysis and Duality in LP Xiaoxi Li EMS & IAS, Wuhan University Oct. 13th, 2016 (week vi) Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 1 / 35

Organization of this lecture Contents: Sensitivity Analysis Duality Shadow price and reduced cost Change in the RHS constants (resource s capacity) Change in objective coefficients (unit revenue) Formulation of the Primal-Dual problems Duality properties: weak, strong, and the complementary slackness Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 2 / 35

Sensitivity Analysis Question after solving a LP problem: How does the optimal solution / value change in response to modification of LP model data? In what range of the LP model data is an optimal solution robust? Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 3 / 35

Motivation for these questions: allowance for certain error in modelling phase; management decisions in response to market or technology adjustment: i.e. whether to purchase/rent additional resource(s) if available; open a new production line if possible; modify the production plan facing changing in demand (unit revenue). Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 4 / 35

An example under consideration: Maximize z = 5x 1 + 4.5x 2 + 6x 3 6x 1 + 5x 2 + 8x 3 60 1 10x 1 + 20x 2 + 10x 3 150 2 s.t. x 1 8 3 x 1, x 2, x 3 0 + (0.1) After applying the simplex method, we obtain the final simplex tableau: Eq. Basic variables z x 1 x 2 x 3 s 1 s 2 s 3 RHS 4 11 1 (0) z 1 0 0 14 35 0 51 3 (1) x 2 0 0 1 2 1 3 35 0 4 2 (2) s 3 0 0 0 11 2 1 11 14 1 11 2 (3) x 1 0 1 0-1 14 0 6 3 Basic variables: {x 2,s 3,x 1 }; Nonbasic variables: {x 3,s 1,s 2 }. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 5 / 35

Shadow price Question: how would the optimal value change if we increase one unit of Resource (1), (2) or (3)? At the optimal solution, s 1 and s 2 are nonbasic variables, equal to zero: both Constraint (1) and Constraint (2) bind and both resources are exhausted. s 3 is basic variable, equal to 11/: Resource (3), which is the limit production for Activity 1, has 11/ units un-used. Intuitively, increasing or decreasing the capacity of Resource (3) by a little bit will have no effect on the optimal solution/value; while increasing or decreasing the capacity of Resource (1) or Resource (2) even by a little bit will change the optimal solution/value. The Shadow price for a resource/constraint is the changing rate of the optimal value in the resource capacity when the modification is small. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 6 / 35

Computation of Shadow price A nice property: there is no need to re-do the simplex method so as to obtain the shadow prices; rather, they can be red directly from the final (optimal) simplex tableau. Claim. The shadow price for a resource/functional constraint is the associated slack variable s coefficient in Eq (0) of the final (optimal) tableau. for a resource un-exhausted at optimum (basic variable), it is zero; for a resource exhausted at optimum (nonbasic variable), it is positive. For the example. y 1 = 11 14, y 2 = 1 35, y 3 = 0. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) / 35

Shadow price Claim. The shadow price for a resource/constraint is the associated slack variable s coefficient in Eq (0) of the final (optimal) tableau. Proof. Consider an increase of small in Resource (i) s capacity, i.e. the LP is the same as before except that b i is replaced by b i +. w.l.o.g, we assume no tie during the application of the simplex method to the original LP problem. For small, the same operations of the simplex method applies for the perturbed problem also (why?). The obtained final (optimal) simplex tableau has the same entries as the the unperturbed one except for the RHS vector, which will be the sum of the unperturbed RHS vector and an "additional vector corresponding the coefficients for ". The "additional vector corresponding the coefficients for " is the same as the column for s i. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 8 / 35

To illustrate, consider in the example that b 1 = 60 is replaced by 60 +. Eq. Basic variables z x 1 x 2 x 3 s 1 s 2 s 3 RHS (0) z 1-5 -4.5-6 0 0 0 0 + 0 (1) s 1 0 6 5 8 1 0 0 60 + 1 (2) s 2 0 10 20 10 0 1 0 150 + 0 (3) s 3 0 1 0 0 0 0 1 8 + 0 Eq. Basic variables z x 1 x 2 x 3 s 1 s 2 s 3 RHS 4 11 1 (0) z 1 0 0 14 35 0 51 3 + 11 14 (1) x 2 0 0 1 2 1 3 35 0 4 2 + 1 (2) s 3 0 0 0 11 2 1 11 14 1 + 2 (3) x 1 0 1 0 11 The initial and the final (optimal) simplex tableaux. 2-1 14 0 6 3 + 2 s 1 and have the same column vector along the computation. The final vector defines the changing rates of optimal value/solution. The same operations work for s 3 but the changing rates for optimal value/solution are all zero (check). Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 9 / 35

Interpretations of the shadow price: Marginal worth to the LP problem of each resource. For an example, if some additional capacity of Resource (i) is available for purchasing with a unit price u i. Then it will not be profitable if this price is beyond its shadow price, i.e. u i > y i. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 10 / 35

Reduced cost Above we have defined and analysed the shadow price for each functional constraint. The shadow price for a nonnegative constraint has an analogous meaning. It is termed the "reduced cost" due to the fact: it is connected by " rather than " ", so an increase on the RHS entry (which is zero) means a restriction on x j, thus incurring a cost. Claim. The reduced cost for an activity/nonnegative constraint is the negative of the associated decision variable s coefficient in Eq (0) of the final (optimal) tableau. for an activity of positive production at optimum (basic), it is zero; for a resource of zero production at optimum (nonbasic), it is positive. Proof. The similar computation as that for the shadow price, except for an attention paid on the opposite sign. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 11 / 35

Eq. Basic variables z x 1 x 2 x 3 s 1 s 2 s 3 RHS 4 11 1 (0) z 1 0 0 14 35 0 51 3 (1) x 2 0 0 1 2 1 3 35 0 4 2 (2) s 3 0 0 0 11 2 1 11 14 1 11 2 (3) x 1 0 1 0-1 14 0 6 3 For the example. c 1 = 0, c 2 = 0, c 3 = 4. Illustrations. At optimum, x 3 = 0 thus Activity (3) is not produced. By forcing a production of > 0 unit of Activity (3), a loss of 4 in the optimal value is induced. At optimum, x 1 = 6 3 > 0 and x 2 = 4 2 > 0 thus both Activity (1) and Activity (2) are produced. By forcing a production of (small) > 0 unit of either Activity (1) or Activity (2), a loss of zero in the optimal value is induced. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 12 / 35

A different way of computing Activity (j) s reduced cost c j involving the shadow prices {y i } and its unit revenue c j ; (pricing out an activity) reduced cost (net unit revenue) = unit revenue "marginal cost". c j = c j i a ij y i. Interpretations of the reduced cost. The reduced cost measures the unit worth (cost if it is negative) of opening a new production line. Consider a LP without Activity (3): it has the same optimal solution structure (basis, shadow prices...) as when Activity (3) is allowed. A negative reduced cost for Activity (3) implies that there is no need to consider open this production line; On the other hand, if c j is large enough so c j = c j i a ij y i > 0, then it should consider open this line. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 13 / 35

Change in the RHS constants: the optimum range The shadow price/reduced cost measures the changing rate of the optimal value in the RHS constant when the change is small. When the change of the RHS constant could be significant, we consider the optimum range within which the optimal basis remains the same. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 14 / 35

For small, the obtained final simplex tableau after adding unit of Resource (1) is: Eq. Basic variables z x 1 x 2 x 3 s 1 s 2 s 3 RHS 4 11 1 (0) z 1 0 0 14 35 0 51 3 + 11 14 (1) x 2 0 0 1 2 1 3 35 0 4 2 1 (2) s 3 0 0 0 11 2 1 11 14 1 2 11 2 (3) x 1 0 1 0-1 14 0 6 3 + 2 To have the above tableau still optimal thus the optimal basis {x 2,s 3,x 1 } remains the same, we should have: x 2 = 4 2 1 0 30. s 3 = 11 2 0 11 2. x 1 = 6 3 + 2 45 0 2. Thus the optimum range is: 45 2 11 2 and 5 2 bnew 1 131 2. Question: compute the optimum range for Resource (2) & (3), Activity 1, 2, 3. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 15 / 35

Change in the objective coefficients: the optimum range Generally, change in the objective coefficients is less sensitive to the optimal solution/value than the change in the RHS constants. Consider no "shadow price" here, but only the "optimum range". The optimum range for Activity j is the range of such that the optimal solution remains the same after c j is replaced by c j +. Consider a change in Activity j s coefficient from c j to c j +. In the perturbed LP problem, applying the same simplex-method operations as unperturbed will give a "final" simplex tableau almost the same as before, except for x j s coefficient in Eq. (0). Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 16 / 35

To illustrate, we take the example and distinguish two types of activities at optimum of the unperturbed problem. x 3 is a nonbasic variable. The final tableau after operations (assure yourself that it is like this). Eq. Basic variables z x 1 x 2 x 3 s 1 s 2 s 3 RHS 4 (0) z 1 0 0 11 1 14 35 0 51 3 (1) x 2 0 0 1 2 1 3 35 0 4 2 (2) s 3 0 0 0 11 2 1 11 14 1 11 2 (3) x 1 0 1 0-1 14 0 6 3 To keep the same optimal solution unchanged, we should have: 4 0 4, thus cnew 1 6 + 4 = 64. Question. Show that Activity 3 s optimum range is such that the new reduced cost for it remains nonnegative. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 1 / 35

x 1 or x 2 is a basic variable. The final tableau after operations (assure yourself that it is like this). Eq. Basic variables z x 1 x 2 x 3 s 1 s 2 s 3 RHS 4 11 1 (0) z 1 0 14 35 0 51 3 (1) x 2 0 0 1 2 1 3 35 0 4 2 (2) s 3 0 0 0 11 2 1 11 14 1 11 2 (3) x 1 0 1 0-1 14 0 6 3 The above tableau is not optimal for 0 (since x 1 should be a basic variable!), a further operation "Eq.(0) =Eq.(0)+3 Eq.(3)" is needed. Eq. Basic z x 1 x 2 x 3 s 1 s 2 s 3 RHS 4 (0) z 1 0 0 + 11 11 14 + 2 1 35 1 14 0 51 3 +6 3 (1) x 2 0 0 1 2 1 3 35 0 4 2 (2) s 3 0 0 0 11 2 1 11 14 1 11 2 (3) x 1 0 1 0-1 14 0 6 3 Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 18 / 35

Now to have the optimal solution remain the same, we should have: 4 + 11 0 4 11. 11 14 + 2 11 0 4. 1 35 1 14 0 + 2 5. Thus and 4 11 2 5, 4 11 cnew 1 = c 1 + 5 2 5. Ex. Compute the optimum range for Activity 2. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 19 / 35

Duality in LP optimal dual variable y i = primal shadow price of Constraint (i); optimal primal variable x j = dual shadow price of Constraint (j). Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 20 / 35

The shadow price and the duality The shadow prices y i = optimal multipliers of constraints, which measure the marginal worth of an additional unit of resource; can be used for pricing out an activity: Activity i s net marginal revenue (reduced cost) is computed as c j = c j j a ij y j. can be seen as the internal pricing mechanism within the firm for the allocation of scared resources to competing activities. Duality is a unifying theory that help develops the relationship between a given LP problem (the primal) and another one (the dual) stated in terms of variables with a shadow-price interpretation. This theory helps further understanding of the optimal multipliers in terms of shadow-price interpretation; helps take the advantage of the computational efficiency of the dual problem (over the primal one). Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 21 / 35

Back to this example we have studied before: s.t. Maximize z = 5x 1 + 4.5x 2 + 6x 3 6x 1 + 5x 2 + 8x 3 60 1 10x 1 + 20x 2 + 10x 3 150 2 x 1 8 3 x 1, x 2, x 3 0 + After applying the simplex method, we obtain the final simplex tableau: Eq. Basic z x 1 x 2 x 3 s 1 s 2 s 3 RHS 4 11 1 (0) z 1 0 0 14 35 0 51 3 (1) x 2 0 0 1 2 1 3 35 0 4 2 (2) s 3 0 0 0 11 2 1 11 14 1 11 2 (3) x 1 0 1 0-1 14 0 6 3 Basic variables: {x 2,s 3,x 1 }; Nonbasic variables: {x 3,s 1,s 2 }. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 22 / 35

Observations/properties of the shadow prices y i. Value and interpretation. Positive shadow price Zero shadow price y 1 = 14 11, y 2 = 35 1 y 3 = 0 s 1 and s 2 nonbasic s 3 basic Constraint (1) & (2) bind Constraint (3) does not bind Resource (1) & (2) exhausted Resource (3) has unused part Pricing out the activities ( c j = c j i a ij y i 0). Zero reduced cost Negative reduced cost c 1 = c 2 = 0 c 3 = 4 x 1, x 2 basic x 3 nonbasic Activity (1) & (2) produced Activity (3) not produced (1),(2): unit revenue = marginal cost (3): unit revenue < marginal cost Opportunity cost for consuming firm s resources (v = i b i y i ). v = 60 11 14 + 150 35 1 + 8 0 = 360 = 51 3 v = z = j c j xj the optimal value of the (primal) LP problem. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 23 / 35

Imagine that the firm does not own these capacities of resources but has to rent them for the production (= minimize the rents). Then a dual LP problem, characterized by the above summarized properties of shadow prices, can be formulated as follows. s.t. Minimize v = 60y 1 + 150y 2 + 8y 3 6y 1 + 10y 2 + y 3 5 (1) 5y 1 + 20y 2 4.5 (2) 8y 1 + 10y 2 6 (3) y 1, y 2, y 3 0 (+) Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 24 / 35

Formulation of the primal-dual problem Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 25 / 35

Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 26 / 35

Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 2 / 35

Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 28 / 35

Properties of LP duality We consider the (P)-(D) formulation in standard form and write them in compact forms. Let c,x R n, A R m n, b,y R m. (P) Max x z = c T x s.t. Ax b, x 0. (D) Min y v = b T y s.t. A T y c, y 0. Verify. The dual of the dual problem (D) is the primal problem (P). Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 29 / 35

Weak Duality Property Theorem (Weak Duality Property) Let x R n be a feasible solution to (P) and let ȳ R m be a feasible solution to (D). Then c T x ȳ T A x b T ȳ. Moreover, if c T x = b T ȳ, then x solves (P) and ȳ solves (D). Corollary If (P) is unbounded, then (D) is infeasible; if (D) is unbounded, then (P) is infeasible. Proof. in (P): A x b, ȳ 0 = ȳ T A x b T ȳ; in (D): A T ȳ c, x 0 = x T A T ȳ c T x. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 30 / 35

Strong Duality Property Theorem (Strong Duality Property) If either (P) or (D) has a finite optimal solution, then so does the other, and the optimal values are equal. Proof. Assume that (P) has a finite optimal solution x and let z = c T x be its optimal value. Let y R m be the associated shadow price vector for (P) s functional constraints. Properties of the shadow prices: reduced costs are nonpositive: c = c A T y 0; shadow prices are nonnegative: y 0; resources opportunity cost is equal to value of (P): b T y = z. The Weak Duality Property implies that b T y = z = c T x b T ȳ for any feasible ȳ to (D). Now we have shown that y is feasible, so it is optimal to (D). The other direction is the same since the dual of the dual is the primal. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 31 / 35

The Complementary Slackness Conditions Corollary Let (x,y) be a pair of feasible solutions to (P)-(D). Then (x,y) solves (P)-(D) iff. c T x = y T Ax = b T y ( ). Proof. Weak Duality Property ("if") + Strong Duality Property ("only if"). Theorem (Complementary Slackness Conditions) Let (x,y) be a pair of feasible solutions to (P)-(D). Then (x,y) solves (P)-(D) iff. Proof. either x j = 0 or i a ij y i = c j for j = 1,...,n; either y j = 0 or j a ij x i = b i for i = 1,...,m. ( ): x T (A T y c) = j x j ( i a ij y i c i ) = 0; feasibility: x j 0 and i a ij y i c i 0 for j = 1,...,n. Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 32 / 35

The Complementary Slackness Conditions The following corollary can be used to check whether a given feasible solution solves the LP problem or not. Corollary (Complementary Slackness Conditions) Let x R n be a feasible solution to (P). Then x solves (P) iff. there exists a vector y R m feasible for D) such that the following holds: Proof. (i)for each i = 1,...,m, if j a ij x j < b i, then y i = 0, and (ii) for each i = 1,...,n, if x j > 0 then i a ij y i = c j. The "only if " part is implied by the Theorem (Complementary Slackness Conditions). To see the the "if " part, observe in the Theorem (Weak Duality Property) the inequality "c T x y T Ax b T y". Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 33 / 35

The Complementary Slackness Conditions: an example We use the Corollary (Complementary Slackness Conditions) to check a particular point x = (4,3) is optimal to the LP problem: Maximize z = 3x 1 + 5x 2 x 1 4 1 2x 2 12 2 s.t. 3x 1 + 2x 2 18 3 x 1, x 2 0 + From item (i) in Corollary (Complementary Slackness Conditions): Constraint 1 : 4 = 4 = no condition on y 1 Constraint 2 : 2 3 = 6 < 12 = y 2 = 0 Constraint 3 : 3 4 + 2 3 = 18 = no condition on y 3 (0.2) Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 34 / 35

The Complementary Slackness Conditions: an example From item (ii) in Corollary (Complementary Slackness Conditions): { Activity 1 : x 1 = 4 > 0 = y 1 + 0y 2 + 3y 3 = 3 Activity 2 : x 2 = 3 > 0 = 0y 1 + 2y 2 + 2y 3 = 5 Putting the above three equations of (y 1,y 2,y 3 ) together: Constraint 2 : y 2 = 0 Activity 1 : y 1 + 3y 3 = 3 Activity 2 : 2y 2 + 2y 3 = 5 We solved that (y 1,y 2,y 3 ) = ( 9 2,0, 5 2 ), which is not feasible for dual problem of LP (0.2) since y 1 = 9 2 < 0. This implies that (4,3) is not optimal for the primal LP (0.2). Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 35 / 35