Chapter 2: Linear Programming Basics (Bertsimas & Tsitsiklis, Chapter 1) 33
Example of a Linear Program Remarks. minimize 2x 1 x 2 + 4x 3 subject to x 1 + x 2 + x 4 2 3x 2 x 3 = 5 x 3 + x 4 3 x 1 0 x 3 0 objective function is linear in vector of variables x = (x 1, x 2, x 3, x 4 ) T constraints are linear inequalities and linear equations last two constraints are special (non-negativity and non-positivity constraint, respectively) 34
General Linear Program minimize c T x subject to a i T x b i for i M 1, (2.1) a i T x = b i for i M 2, (2.2) a i T x b i for i M 3, (2.3) x j 0 for j N 1, (2.4) x j 0 for j N 2, (2.5) with c R n, a i R n and b i R for i M 1 M 2 M 3 (finite index sets), and N 1, N 2 {1,..., n} given. x R n satisfying constraints (2.1) (2.5) is a feasible solution. feasible solution x is optimal solution if c T x c T x for all feasible solutions x. linear program is unbounded if, for all k R, there is a feasible solution x R n with c T x k. 35
Special Forms of Linear Programs maximizing c T x is equivalent to minimizing ( c) T x. any linear program can be written in the form minimize subject to c T x A x b for some A R m n and b R m : rewrite ai T x = b i as: a i T x b i a i T x b i, rewrite ai T x b i as: ( a i ) T x b i. Linear program in standard form: min s.t. c T x A x = b x 0 with A R m n, b R m, and c R n. 36
Example: Diet Problem Given: n different foods, m different nutrients a ij := amount of nutrient i in one unit of food j b i := requirement of nutrient i in some ideal diet c j := cost of one unit of food j Task: find a cheapest ideal diet consisting of foods 1,..., n. LP formulation: Let x j := number of units of food j in the diet: min c T x min c T x s.t. A x = b or s.t. A x b x 0 x 0 with A = (a ij ) R m n, b = (b i ) R m, c = (c j ) R n. 37
Reduction to Standard Form Any linear program can be brought into standard form: elimination of free (unbounded) variables x j : replace x j with x + j, x j 0: x j = x + j x j elimination of non-positive variables x j : replace x j 0 with ( x j ) 0. elimination of inequality constraint a T i x b i : introduce slack variable s 0 and rewrite: a i T x + s = b i elimination of inequality constraint a i T x b i : introduce slack variable s 0 and rewrite: a i T x s = b i 38
Example The linear program min 2 x 1 + 4 x 2 s.t. x 1 + x 2 3 3 x 1 + 2 x 2 = 14 x 1 0 is equivalent to the standard form problem min 2 x 1 + 4 x 2 + 4 x2 s.t. x 1 + x 2 + x2 x 3 = 3 3 x 1 + 2 x 2 + 2 x2 = 14 x 1, x 2 +, x 2, x 3 0 39
Affine Linear and Convex Functions Lemma 2.1. a b An affine linear function f : R n R given by f (x) = c T x + d with c R n, d R, is both convex and concave. If f 1,..., f k : R n R are convex functions, then f : R n R defined by f (x) := max i=1,...,k f i (x) is also convex. 40
Piecewise Linear Convex Objective Functions Let c 1,..., c k R n and d 1,..., d k R. Consider piecewise linear convex function: x max i=1,...,k c i T x + d i : min max i=1,...,k c i T x + d i min z s.t. A x b s.t. z c i T x + d i for all i A x b Example: let c 1,..., c n 0 min n c i x i i=1 min n c i z i i=1 s.t. A x b s.t. z i x i z i x i A x b 41
Graphical Representation and Solution 2D example: min x 1 x 2 s.t. x 1 + 2 x 2 3 2 x 1 + x 2 3 x 1, x 2 0 x 2 3 2.5 2 1.5 1 0.5 0 0 0.5 1 1.5 2 2.5 3 x 1 42
Graphical Representation and Solution (cont.) 3D example: min x 1 x 2 x 3 x 2 s.t. x 1 1 x 2 1 x 3 1 x 1, x 2, x 3 0 1 x 3 1 0 0 1 x 1 43
Graphical Representation and Solution (cont.) another 2D example: 3 x 2 min c 1 x 1 + c 2 x 2 s.t. x 1 + x 2 1 x 1, x 2 0 2 1 0 x 1 0 1 2 3 for c = (1, 1) T, the unique optimal solution is x = (0, 0) T for c = (1, 0) T, the optimal solutions are exactly the points x = (0, x 2 ) T with 0 x 2 1 for c = (0, 1) T, the optimal solutions are exactly the points x = (x 1, 0) T with x 1 0 for c = ( 1, 1) T, the problem is unbounded, optimal cost is if we add the constraint x 1 + x 2 1, the problem is infeasible 44
Properties of the Set of Optimal Solutions In the last example, the following 5 cases occurred: i there is a unique optimal solution ii iii iv v there exist infinitely many optimal solutions, but the set of optimal solutions is bounded there exist infinitely many optimal solutions and the set of optimal solutions is unbounded the problem is unbounded, i. e., the optimal cost is and no feasible solution is optimal the problem is infeasible, i. e., the set of feasible solutions is empty These are indeed all cases that can occur in general (see later). 45
Visualizing LPs in Standard Form Example: Let A = (1, 1, 1) R 1 3, b = (1) R 1 and consider the set of feasible solutions P = {x R 3 A x = b, x 0}. x 2 1 x 3 1 0 x 1 0 1 46
Visualizing LPs in Standard Form More general: if A R m n with m n and the rows of A are linearly independent, then {x R n A x = b} is an (n m)-dimensional affine subspace in R n. set of feasible solutions lies in this affine subspace and is only constrained by non-negativity constraints x 0. 47