Lecture Notes 3: Duality

Transcription

1 Algorithmic Methods 1/11/21 Professor: Yossi Azar Lecture Notes 3: Duality Scribe:Moran Bar-Gat 1 Introduction In this lecture we will present the dual concept, Farkas s Lema and their relation to the field of game theory 2 Duality The general case: primal dual a i x = b i y i < => a i x b i < x j => x j min c t x (A j describs column j, a j describes row j) Claim: If x Primal and y Dual than c t x y t b y i y t A j = c j y t A j c j max y t b Proof: min c t x, Ax = b, A x b where A is of size lxn and A is of size (m l)xn In addition x 1,, x k and x k+1,, x n < => We have: c t x = c j x j + = y t Ax = y i a i x c j x j y t A j x j + l y i b i + i=l+1 y i b i = y t b y t A j x j Claim: If x is an optimal solution to the primal system then there exists an optimal solution to the dual system y st c t x = y t b Proof: The proof is given only for LPS (Ax=b, x, min c t x) We solve using the Simplex algorithm and find an optimal solution For this solution: Let B denote the base, x = A 1 B b, particularly ct c t B A 1 B A and equals ct B A 1 B b (Actually, we showed that c t N ct B A 1 B A N but this immediately implies that c t c t B A 1 B A ) We claim that y t = c t B A 1 B is an optimal solution to the dual system and ct y t A, 3-1

2 y t b = c t x In addition the dual system of LPS is valid meaning: max y t b, y t A c j, y < = > This optimal solution is also equal to the optimal solution to the primal system and therefore it is optimal and feasible Remarks: (1) - Proof for non LPS is done using reduction (2) - dual of dual is primal Possible Cases: (1) - Both dual and primal are feasible, both have an optimum and therefore the optimum values are equal (2) - One is empty and the other is unbounded (3) - Neither is feasible Claim: Complementary Slackness For optimal solutions: y i (a i x b i ) = ; (c j y t A j )x j = Proof: Recall the proof for the weaker claim For optimal solutions the inequalities becomes equalities: c t x = c j x j + = y t Ax = y i a i x= and the claim is immediately derived c j x j = y t A j x j + l y i b i + i=l+1 y i b i = y t y t A j x j 3 Financial Meaning for Dual Variables (Production Problems) Assuming all constraints are positive A factory which manufactures furniture from base materials we have the following data: table chair couch wood metal fabric

3 The Dual Case: Objective function: max 8x 1 + 3x 2 + 2x 3 c T = (1 9 84) x 1, x 2, x 3 The Primal Case: G u 1 (8x 1 + 3x 2 + x 3 ) + u 2 (2x 1 + x 2 + 5x 3 ) + u 3 (2x 2 + 8x 3 ) 1u 1 + 9u u 3 Question: Would purchasing more from a certain material significantly increase profits? We look for changes in the objective function We increase the quantities of products u 1, u 2, u 3 by ɛ 1, ɛ 2, ɛ 3 respectively We have: G = ɛ 1 u 1 + ɛ 2 u 2 + ɛ 3 u 3 4 Farkas s Lemma: Let b, a 1,, a n R m therefore exists λ i st: b = Σλ i a i ( yif i ya i yb ) Proof: ( ) Assuming that there exist λ i st b = Σλ i a i We have yb = y λ i a i = λ i ya i ( ) Assuming that y if ya i yb Define a matrix A=(a 1,, a n ) We solve: Aλ = b λ max λ Now, looking at the dua systeml and checking what is the primal system: miny t b y t A y < = > According to the assumption, the objective function The solution y = is feasible and equals and therefore is optimal We conclude that there exists a feasible solution for the dual system A reduction from the problem of finding an optimal solution to the problem of finding a feasible solution: We show that if there exists a black box which solves the feasibility problem for a linear programming system then we can also find an optimum for ther system Given a linear programming system and objective function min c t x wel add the dual y t A c and the constraint c t x = y t b and get a new system 3-3

4 Claim: There is a feasible solution to the new system There is an optimal solution the the original system Moreover, each solution to the new system gives an optimal solution to the original system 5 Application: Game Theory, Zero-Sum Games: Two players are playing a zero-sum game Player I chooses a column and player II chooses a row (In the general form the size of the matrix can be any m n) a ij represents the amount that player II pays player I if player I chooses j and player II chooses i For Example: The table below describes the chances of each player choosing their respective column/row Each element a ij equals the amount player II has to pay player I II/I q 1-q p p 1 2 For the above example Player I can guarentee making a profit of at least 1 by choosing the column 1 Let α = max j min i a ij Player II can guarantee losing less than 2 by choosing the second row Let β = min i max j a ij Another strategy that each player can take is choosing i (player I) and j (player II) with different probabilities, to ensure maximum (player I) or minimum (player II) payment Let q be the probability player I chooses the first column and 1-q the probability she chooses the second column Accordingly, let p be the probability player II chooses the first row and 1-p the probability she chooses the second row The expected payoff for player I will be: Player II chooses first row : Player II chooses second row : We choose q by the equation: 4q-(1-q) = 5q-1 q+ 2(1-q) = 2-q 5q-1=2-q q=5 If player I chooses a column with probability 5 he guarantees himself at least 15 (the expectation value) regardless of player II s choice We calculate Player II s strategy in the same way: Player I chooses first column : Player I chooses second Column : 4p+1-p = 1+3p 3-4

5 We choose p by the equation: -p+2(1-p) = 2-3p 1+3p = 2-3p p= 1 6 If player II choosea rows 1,2 with probability 1 6, 5 6 respectfuly, he guarantees himself maximum pay of 15 (the expectation value) regardless of player I s choice Consider a general case where we have a matrix A player I s variables are x 1,, x n where x i is the probability of choosing column i Player II s variables are y 1,, y n where y j is the probability of choosing row j For player I: For player II: We show that one is dual to the other: max α Ax α e m x e t n x = 1 min β y t A β e t n y e t m y = 1 β < = > min ( α) Ax α e m e t n x = 1 x max ( β) y t A βe t n e t m y = 1 y This can be easily seen by looking at the Matrix: (c/b) ( ) 1 A 1 1 (-1) ( ) 3-5