Linear Programming
Linear optimization problems max f(x) g i (x) b i x j R i =1,...,m j =1,...,n Optimization problem g i (x) f(x) When and are linear functions Linear Programming Problem 1 n max c x n 1 m n A x b m 1
Doors and windows A small firm produces: Aluminum door (D) Wooden windows (W) Benefit! (per item) 30 50 c Resources (hours per item) Availability Smith 1 0 4 Carpenter 0 A 2 12 b Assembler 3 2 18 Plan the production so as to maximize the return Variables: x D x W x
x W Geometric representation A 2 f(x) =360 (2, 6) xd =4 2x W =12 Vertices are particularly important A 1 f(x) =160 A 3 A 4 xd =0 (2, 2) c (1, 1) 3x D +2x W =18 How to characterize them? A 5 x W =0 f(x) =80 x D
Basic concepts Convex set S R n convex combination For each x, y S λ [0, 1] λx +(1 λ)y S Convex polyhedron P R n P = {x : A i x b i,i=1,...,m} C R n If Cone x C λx C, λ 0 then Convex cone C R n non negative linear combination x, y C λ,µ 0 λx + µy C Polyhedral cone C R n C = {x : A i x 0,i=1,...,m} Finitely generated cone C R n Given x 1,...,x r R n rays C = {x : x = λ 1 x 1 =...,λ r x r, λ i 0,i=1,...,r}
Extreme point - Vertex Consider P = {x : A i x b i,i=1,...,m} x P is an extreme point of P if there are no y, z P y, z x x = λy +(1 λ)z for some λ [0, 1] x P is a vertex of P if c R n such that cx > cy, y P, y x The two definitions are equivalent
Basic solution Consider P = {x : A i x b i,i=1,...,m} x P I(x )={i : A i x = b i } Set of active constraints x is a basic solution if among all A i : i I(x ) there are n of them linearly independent extreme point vertex basic solution
Some properties Consider P = {x : A i x b i,i=1,...,m} P = Q + C Q = conv{x 1,...,x s } C = cone{y 1,...,y t } polytope cone If max{cx : x P } has finite optimum then one of the extreme points of Q is optimum Given a convex set S R n and a point c / S there is a separating hyperplane x R n : cx > zx z S
Two problems GDA feed A feed B carbohydrates 14 2 2 proteins 20 4 2 vitamins 9 1 3 cost 12 16 Farmer: minimize the cost of the feeds Chicken pills seller: maximize the profit
Pair of dual problems (symmetric) P: min cx Ax b x 0 Farmer D: max yb ya c y 0 Chicken pill seller Let x and ȳ be feasible solutions for P and D then c x ȳb Important consequence if = holds
Other two problems Factories Stores n i=1 a i = production m j=1 d j a 1 a 2 1 1 d 1 request 2 2 d 2 transportation cost a i i b ij j d j a n n m d m Owner of factories and stores: transport the production minimize the cost Transport company: buy from the factories and resell to the stores maximize the profit
Pair of dual problems (asymmetric) D: P: min yb ya = c y 0 max cx Ax b Owner of factories and stores Transporter Let x and ȳ be feasible solutions for P and D then c x ȳb Important consequence if = holds
Problem A supermarket manager has to hire clerks b d = minimum number of clerks required in day d d {M, T, W, T h, F, S, Su} Shifts: 5 consecutive working days + 2 rest days c d = cost of the salary for the shift beginning in day d Minimize the cost Interpretation of the dual
Writing the dual min variables constraints cost vector right hand side A i x b i A i x b i A i x = b i x i 0 x i 0 x i unrestricted max constraints variables right hand side cost vector y i 0 y i 0 y i unrestricted ya i c i ya i c i ya i = c i
x W Geometric interpretation A 2 A 2 c A 3 A 4 c A 1 A 5 A 1 A 3 write c as linear non negative combination of Ai A 4 x D A 5 P: max cx Ax b min yb D: ya = c y 0
Can we improve a feasible solution? P: max cx Ax b Let x be a feasible solution Consider the new point x = x + λξ λ > 0 x ξ R n direction x ξ λξ moving step x must be feasible Ax b must improve the objective function value cx >c x
Growing feasible direction What are the characteristics of the direction ξ? Improve the objective function cξ > 0 c Allow to move (of a small step) without violating any constraint polyhedral cone A I ξ 0 any vector in this cone is a feasible growing direction
Optimality conditions If a growing feasible direction ξ does not exist Existence of a growing feasible direction cξ > 0 x is optimum i) RP: A I ξ 0 has a solution c belongs to the cone generated by AI ii) ηa I = c η 0 RD: has no solution max cξ LP interpretation A I ξ 0 is unbounded min 0η ηa I = c η 0 Restricted Primal Restricted Dual
Optimality conditions If a growing feasible direction ξ does not exist Existence of a growing feasible direction cξ > 0 x is optimum i) RP: A I ξ 0 LP interpretation max cξ A I ξ 0 Restricted Primal has no solution 0 is an optimal solution c belongs to the cone generated by AI ii) ηa I = c η 0 RD: has a solution min 0η ηa I = c η 0 Restricted Dual Either i) or ii) holds true but not both [Farkas Lemma]
Strong duality theorem Consider P: max cx Ax b D: min yb ya = c y 0 x and ȳ are optimal solutions for P and D if and only if c x = ȳb Given a solution of ii) η construct a feasible solution for D ȳ i = { ηi i I 0 i Ī
Determining the moving step Assuming that a growing feasible direction ξ exists What is the maximum value of such that λ x is feasible? Consider the non active constraints only Ī = {i : A i x <b i } distance from the constraint border approaching speed λ = min{ b i A i x A i ξ : A i ξ > 0,i Ī} λ = if A i ξ 0, i Ī the problem is unbounded
Sketch of a solution algorithm Consider a feasible solution x; while a growing feasible direction determine! move of! along Choice of Among all possible directions select one on the border of the polyhedron, if possible
Primal-Dual algorithm(a,b,c,x,unbounded,y) optimal false; unbounded false; I {i: A i x=b i }; if I= then grow_along(c,x,i,unbounded); while not optimal and not unbounded do if {"AI=c} has no solution then compute : AI 0, c =1; grow_along(,x,i,unbounded); else if {"AI=c} has a solution and h: "h<0 then compute : AI = -uh; grow_along(,x,i,unbounded); else optimal true; {the dual system has a solution: "AI=c, " 0} yi "i: i I, yi 0 otherwise.
grow_along(,x, I, unbounded) λ min{ b i A i x : A i ξ > 0,i A i ξ Ī, } if λ = then unbounded true else x x + λξ I {i: A i x=b i }
Searching for a feasible solution P: max cx Ax b If b i 0 i =1,...,n then x =0 is feasible AP: x =0,u I = b I is feasible I + = {i : b i 0} I = {i : b i < 0} Construct an auxiliary problem max i I u i A x b I+ I+ A x u b I I I u 0 I discourage the use of artificial variables If the optimal solution ( x, ū I ) has ū I =0 then x is feasible otherwise P has no solution
Complementary slackness conditions P: max cx Ax b ȳ min yb D: ya = c y 0 Given x and feasible for P and D the following three statements are equivalent i) x and ȳ are optimal ii) iii) c x = ȳb ȳ(b A x) =0 if ȳ i 0 A i x = b i if A i x b i ȳ i =0
Complementary slackness conditions (2) min cx P: D: Ax b x 0 ȳ(b A x) =0 (c ȳa) x =0 max yb ya c y 0 if ȳ i 0 A i x = b i if if A i x b i x j 0 ȳ i =0 (c j =ȳa j ) if (c j ȳa j ) x j =0
Economic interpretation of dual variables x W c A 2 = = max 30x D +50x W x D 4 2x W 12 3x D +2x W 18 x D 0 [ ] x W 0 2 x = 6 A 1 A 3 min 4y 1 +12y 2 +18y 3 y 1 +3y 3 y 4 =30 2y 2 +2y 3 y 5 =50 y 1,y 2,y 3,y 4,y 5 0 ȳ =[0, 15, 10, 0, 0] A 4 x D A 5 Are you willing to pay for one more unit of resource 1? How much would you pay for one more unit of resource 2?
Economic interpretation of dual variables (2) x W x(ε) c A 2 P( P: ε ): ε i,i=1,...,m parametric perturbations max cx A 1 x b 1 A 2 x b 2... A m x b m +ε 1 +ε 2 +ε m optimal solution x x(ε) A 1 D( D: ε ): min yb 1 (b y 1 + bɛ 21 y)+...+ 2 + yb m (by m + ɛ m ) y 1 A 1 + y 2 A 2 +...+ y m A m = c optimal solution y 1,y 2,...,y m 0 ȳ is still feasible A 3 A 4 by complementary slackness is also optimal is x D A 5 ε i if are small enough x(ε) defines the same active constraints as x
Shadow prices c x(ε) = ȳ(b + ε) m = c x + (ȳ i ε i ) i=1 impact on the objective function for each unit of perturbation
Changing cost coefficients max 30x D +50x W x D 4 2x W 12 3x D +2x W 18 x D 0 x W 0 For which values of remains optimal? x = [ 2 6 ] The active constraints do not change x min 4y 1 +12y 2 +18y 3 y 1 +3y 3 y 4 =30 2y 2 +2y 3 y 5 =50 y 1,y 2,y 3,y 4,y 5 0 c remains within the cone generated by the gradients of the active constraints c A2 A3