Thursday, May 24, Linear Programming

Size: px

Start display at page:

Download "Thursday, May 24, Linear Programming"

Milo Mathews
5 years ago
Views:

1 Linear Programming

2 Linear optimization problems max f(x) g i (x) b i x j R i =1,...,m j =1,...,n Optimization problem g i (x) f(x) When and are linear functions Linear Programming Problem 1 n max c x n 1 m n A x b m 1

3 Doors and windows A small firm produces: Aluminum door (D) Wooden windows (W) Benefit! (per item) c Resources (hours per item) Availability Smith Carpenter 0 A 2 12 b Assembler Plan the production so as to maximize the return Variables: x D x W x

4 x W Geometric representation A 2 f(x) =360 (2, 6) xd =4 2x W =12 Vertices are particularly important A 1 f(x) =160 A 3 A 4 xd =0 (2, 2) c (1, 1) 3x D +2x W =18 How to characterize them? A 5 x W =0 f(x) =80 x D

5 Basic concepts Convex set S R n convex combination For each x, y S λ [0, 1] λx +(1 λ)y S Convex polyhedron P R n P = {x : A i x b i,i=1,...,m} C R n If Cone x C λx C, λ 0 then Convex cone C R n non negative linear combination x, y C λ,µ 0 λx + µy C Polyhedral cone C R n C = {x : A i x 0,i=1,...,m} Finitely generated cone C R n Given x 1,...,x r R n rays C = {x : x = λ 1 x 1 =...,λ r x r, λ i 0,i=1,...,r}

6 Extreme point - Vertex Consider P = {x : A i x b i,i=1,...,m} x P is an extreme point of P if there are no y, z P y, z x x = λy +(1 λ)z for some λ [0, 1] x P is a vertex of P if c R n such that cx > cy, y P, y x The two definitions are equivalent

7 Basic solution Consider P = {x : A i x b i,i=1,...,m} x P I(x )={i : A i x = b i } Set of active constraints x is a basic solution if among all A i : i I(x ) there are n of them linearly independent extreme point vertex basic solution

8 Some properties Consider P = {x : A i x b i,i=1,...,m} P = Q + C Q = conv{x 1,...,x s } C = cone{y 1,...,y t } polytope cone If max{cx : x P } has finite optimum then one of the extreme points of Q is optimum Given a convex set S R n and a point c / S there is a separating hyperplane x R n : cx > zx z S

9 Two problems GDA feed A feed B carbohydrates proteins vitamins cost Farmer: minimize the cost of the feeds Chicken pills seller: maximize the profit

10 Pair of dual problems (symmetric) P: min cx Ax b x 0 Farmer D: max yb ya c y 0 Chicken pill seller Let x and ȳ be feasible solutions for P and D then c x ȳb Important consequence if = holds

11 Other two problems Factories Stores n i=1 a i = production m j=1 d j a 1 a d 1 request 2 2 d 2 transportation cost a i i b ij j d j a n n m d m Owner of factories and stores: transport the production minimize the cost Transport company: buy from the factories and resell to the stores maximize the profit

12 Pair of dual problems (asymmetric) D: P: min yb ya = c y 0 max cx Ax b Owner of factories and stores Transporter Let x and ȳ be feasible solutions for P and D then c x ȳb Important consequence if = holds

13 Problem A supermarket manager has to hire clerks b d = minimum number of clerks required in day d d {M, T, W, T h, F, S, Su} Shifts: 5 consecutive working days + 2 rest days c d = cost of the salary for the shift beginning in day d Minimize the cost Interpretation of the dual

14 Writing the dual min variables constraints cost vector right hand side A i x b i A i x b i A i x = b i x i 0 x i 0 x i unrestricted max constraints variables right hand side cost vector y i 0 y i 0 y i unrestricted ya i c i ya i c i ya i = c i

15 x W Geometric interpretation A 2 A 2 c A 3 A 4 c A 1 A 5 A 1 A 3 write c as linear non negative combination of Ai A 4 x D A 5 P: max cx Ax b min yb D: ya = c y 0

16 Can we improve a feasible solution? P: max cx Ax b Let x be a feasible solution Consider the new point x = x + λξ λ > 0 x ξ R n direction x ξ λξ moving step x must be feasible Ax b must improve the objective function value cx >c x

17 Growing feasible direction What are the characteristics of the direction ξ? Improve the objective function cξ > 0 c Allow to move (of a small step) without violating any constraint polyhedral cone A I ξ 0 any vector in this cone is a feasible growing direction

18 Optimality conditions If a growing feasible direction ξ does not exist Existence of a growing feasible direction cξ > 0 x is optimum i) RP: A I ξ 0 has a solution c belongs to the cone generated by AI ii) ηa I = c η 0 RD: has no solution max cξ LP interpretation A I ξ 0 is unbounded min 0η ηa I = c η 0 Restricted Primal Restricted Dual

19 Optimality conditions If a growing feasible direction ξ does not exist Existence of a growing feasible direction cξ > 0 x is optimum i) RP: A I ξ 0 LP interpretation max cξ A I ξ 0 Restricted Primal has no solution 0 is an optimal solution c belongs to the cone generated by AI ii) ηa I = c η 0 RD: has a solution min 0η ηa I = c η 0 Restricted Dual Either i) or ii) holds true but not both [Farkas Lemma]

20 Strong duality theorem Consider P: max cx Ax b D: min yb ya = c y 0 x and ȳ are optimal solutions for P and D if and only if c x = ȳb Given a solution of ii) η construct a feasible solution for D ȳ i = { ηi i I 0 i Ī

21 Determining the moving step Assuming that a growing feasible direction ξ exists What is the maximum value of such that λ x is feasible? Consider the non active constraints only Ī = {i : A i x <b i } distance from the constraint border approaching speed λ = min{ b i A i x A i ξ : A i ξ > 0,i Ī} λ = if A i ξ 0, i Ī the problem is unbounded

22 Sketch of a solution algorithm Consider a feasible solution x; while a growing feasible direction determine! move of! along Choice of Among all possible directions select one on the border of the polyhedron, if possible

23 Primal-Dual algorithm(a,b,c,x,unbounded,y) optimal false; unbounded false; I {i: A i x=b i }; if I= then grow_along(c,x,i,unbounded); while not optimal and not unbounded do if {"AI=c} has no solution then compute : AI 0, c =1; grow_along(,x,i,unbounded); else if {"AI=c} has a solution and h: "h<0 then compute : AI = -uh; grow_along(,x,i,unbounded); else optimal true; {the dual system has a solution: "AI=c, " 0} yi "i: i I, yi 0 otherwise.

24 grow_along(,x, I, unbounded) λ min{ b i A i x : A i ξ > 0,i A i ξ Ī, } if λ = then unbounded true else x x + λξ I {i: A i x=b i }

25 Searching for a feasible solution P: max cx Ax b If b i 0 i =1,...,n then x =0 is feasible AP: x =0,u I = b I is feasible I + = {i : b i 0} I = {i : b i < 0} Construct an auxiliary problem max i I u i A x b I+ I+ A x u b I I I u 0 I discourage the use of artificial variables If the optimal solution ( x, ū I ) has ū I =0 then x is feasible otherwise P has no solution

26 Complementary slackness conditions P: max cx Ax b ȳ min yb D: ya = c y 0 Given x and feasible for P and D the following three statements are equivalent i) x and ȳ are optimal ii) iii) c x = ȳb ȳ(b A x) =0 if ȳ i 0 A i x = b i if A i x b i ȳ i =0

27 Complementary slackness conditions (2) min cx P: D: Ax b x 0 ȳ(b A x) =0 (c ȳa) x =0 max yb ya c y 0 if ȳ i 0 A i x = b i if if A i x b i x j 0 ȳ i =0 (c j =ȳa j ) if (c j ȳa j ) x j =0

28 Economic interpretation of dual variables x W c A 2 = = max 30x D +50x W x D 4 2x W 12 3x D +2x W 18 x D 0 [ ] x W 0 2 x = 6 A 1 A 3 min 4y 1 +12y 2 +18y 3 y 1 +3y 3 y 4 =30 2y 2 +2y 3 y 5 =50 y 1,y 2,y 3,y 4,y 5 0 ȳ =[0, 15, 10, 0, 0] A 4 x D A 5 Are you willing to pay for one more unit of resource 1? How much would you pay for one more unit of resource 2?

29 Economic interpretation of dual variables (2) x W x(ε) c A 2 P( P: ε ): ε i,i=1,...,m parametric perturbations max cx A 1 x b 1 A 2 x b 2... A m x b m +ε 1 +ε 2 +ε m optimal solution x x(ε) A 1 D( D: ε ): min yb 1 (b y 1 + bɛ 21 y) yb m (by m + ɛ m ) y 1 A 1 + y 2 A y m A m = c optimal solution y 1,y 2,...,y m 0 ȳ is still feasible A 3 A 4 by complementary slackness is also optimal is x D A 5 ε i if are small enough x(ε) defines the same active constraints as x

30 Shadow prices c x(ε) = ȳ(b + ε) m = c x + (ȳ i ε i ) i=1 impact on the objective function for each unit of perturbation

31 Changing cost coefficients max 30x D +50x W x D 4 2x W 12 3x D +2x W 18 x D 0 x W 0 For which values of remains optimal? x = [ 2 6 ] The active constraints do not change x min 4y 1 +12y 2 +18y 3 y 1 +3y 3 y 4 =30 2y 2 +2y 3 y 5 =50 y 1,y 2,y 3,y 4,y 5 0 c remains within the cone generated by the gradients of the active constraints c A2 A3

Midterm Review. Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.

Midterm Review. Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. Midterm Review Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapter 1-4, Appendices) 1 Separating hyperplane