1 / 21 Linear Programming Operations Research Anthony Papavasiliou
Contents 2 / 21 1 Primal Linear Program 2 Dual Linear Program
Table of Contents 3 / 21 1 Primal Linear Program 2 Dual Linear Program
Linear Programs in Standard Form 4 / 21 Linear program (LP) in standard form: (P) : min z = c T x s.t. Ax = b x 0 where x R n, c R n, A R m n, b R m Any LP can be expressed in standard form
5 / 21 Solution of an LP Solution: a vector x such that Ax = b Feasible solution: a solution with x 0 Optimal solution: a feasible solution x such that c T x c T x for all feasible solutions x
Basis and Basic Solution 6 / 21 Basis: a choice of n linearly independent columns of A Denote A = [B, N] where N are non-basic columns [ ] Each basis corresponds to a basic solution x B = B 1 b and x N = 0 x B x N with Geometric property: Basic feasible solutions correspond to extreme points of the feasible region {x Ax = b, x 0} A basis is feasible if B 1 b 0 optimal if feasible and cn T ct B B 1 N 0
7 / 21 Optimal Basis [ Claim: [ Proof: x B x N x B x N ] is optimal if B 1 b and cn T ct B B 1 N 0 ] is obviously feasible, can we improve objective function by moving away from it? Idea: substitute basic variables for non-basic variables in objective function c T x: [ ] [ B N ] x B x N = b Bx B + Nx N = b x B = B 1 (b Nx N ) (1)
8 / 21 Substituting equation (1) into the objective function, c T x = c T B x B + c T N x N = c T B B 1 b + (c T N ct B B 1 N)x N (2) Non-basic variables can only increase when moving away from the current solution while remaining feasible Since cn T ct B B 1 N 0, second term of equation (2) can only increase when moving in the neighborhood of the current solution current solution is locally optimal In linear programming, locally optimal globally optimal
9 / 21 Our Focus: z(b) We care about how the optimal value of a linear program depends on the right-hand side parameters b: z(b) = min c T x s.t. Ax = b x 0
10 / 21 Example: The Diet Problem Problem: Choose 3 dishes (x 1, x 2, x 3 ) so as to satisfy nutrient requirements b 1 and b 2, while minimizing cost (dishes cost 1 $, 2 $, and 3 $ respectively) Table: The unit of nutrients in each dish. Dish 1 Dish 2 Dish 3 Nutrient 1 0.5 4 1 Nutrient 2 2 1 2 min x 1 + 2x 2 + 3x 3 s.t. 0.5x 1 + 4x 2 + x 3 = b 1 2x 1 + x 2 + 2x 3 = b 2 x 1, x 2 0
Example: The Diet Problem - Basic Solutions Three possible bases: [ ] [ ] [ 0.5 4 0.5 1 4 1 B 1 =, B 2 =, B 3 = 2 1 2 2 1 2 ] Three candidate basic solutions, (parametrized on (b 1, b 2 )): [ ] x B1 0.1333b 1 + 0.5333b 2 = 0.2667b 1 0.0667b 2 [ ] x B2 2b 1 + b 2 = 2b 1 0.5b 2 [ ] x B3 = 0.2857b 1 0.1429b 2 0.1429b 1 + 0.5714b 2 We want to understand how the objective function behaves as we change b 11 / 21
12 / 21 Example: The Diet Problem - Basic Feasible Solutions In order for a basic solution to be feasible, it is necessary that b is such that x B 0 Denote R i = {(b 1, b 2 ) : x Bi 0}, then: R 1 = {0.25b 1 b 2 4b 1 } R 2 = {2b 1 b 2 4b 1 } R 3 = {0.25b 1 b 2 2b 1 }
Example: The Diet Problem - Basic Optimal Solutions 13 / 21 Cost of each basic solution, parametric on (b 1, b 2 ): c T B 1 x B1 = 0.4b 1 + 0.4b 2 c T B 2 x B2 = 4b 1 0.5b 2 c T B 3 x B3 = 0.1429b 1 + 1.4286b 2 Denote S i as region for which B i is optimal: S 1 = {0.25b 1 b 2 4b 1 } S 2 = {b 2 4b 1 } S 3 = {b 2 0.25b 1 }
Example: The Diet Problem - z(b) z(b) is piecewise linear convex function of b Pop quiz: for any b / S 1 the basic solutions are not feasible, what can you conclude about dish 3? 14 / 21
Table of Contents 15 / 21 1 Primal Linear Program 2 Dual Linear Program
Dual LP 16 / 21 The dual of problem (P) is the following linear program: (D) : max π T b s.t. π T A c T If primal problem is not in standard form, use the following rules Primal Minimize Maximize Dual Constraints b i 0 Variables b i 0 = b i Free Variables 0 c j Constraints 0 c j Free = c j
Example: Dual of the Diet Problem 17 / 21 Recall the diet problem: min x 1 + 2x 2 + 3x 3 s.t. 0.5x 1 + 4x 2 + x 3 = b 1 2x 1 + x 2 + 2x 3 = b 2 x 1, x 2 0 Using the table in the previous slide, the dual is: max b 1 π 1 + b 2 π 2 s.t. 0.5π 1 + 2π 2 1 4π 1 + π 2 2 π 1 + 2π 2 3
Example: Dual of the Diet Problem - Feasible Region 18 / 21 Figure: The dual feasible region of the diet problem. Each point A, B and C is a basic solution of the dual feasible region and corresponds to a basis of the primal problem in standard form.
19 / 21 Revisiting z(b) Recall that we care about how the optimal value of a linear program depends on the right-hand side parameters b: z(b) = min c T x s.t. Ax = b x 0
Rewriting the Dual Problem Claim: z(b) is a piecewise linear convex function Proof: If a dual optimal solution exists, then one dual basic solution 1 must be optimal Reformulation of the dual problem: max i=1,...,r πt i b where r indexes finitely many basic feasible solutions 1 General definition of basic solution for a polyhedron P R n (not necessarily in standard form) that is defined by linear equalities and inequalities: a vector x such that (i) all equality constraints are active and (ii) out of the constraints that are active at x, n are linearly independent 20 / 21
21 / 21 Computing Dual Basic Solutions For linear programs in standard form, each basis B of the primal coefficient matrix A corresponds to a basic solution of the dual feasible set, according to the relationship and vice versa π T = c T B B 1
Example: Dual of the Diet Problem - Basic Solutions 22 / 21 Recall the three possible bases of the diet problem: [ ] [ ] [ ] 0.5 4 0.5 1 4 1 B 1 =, B 2 =, B 3 = 2 1 2 2 1 2 Basic solutions of the dual feasible region can be computed according to π = cb T B 1 : A = (0.4, 0.4), B = (4, 0.5), C = (0.1429, 1.4286)