LINEAR PROGRAMMING Relation to Material in Text After a brief introduction to linear programming on p. 3, Cornuejols and Tϋtϋncϋ give a theoretical discussion including duality, and the simplex solution method in Chapter 2. Another popular type of solution methods, the interior point methods, are described in Chapter 7. Chapters 3 and 4 discuss applications of linear programming to asset/liability cash-flow matching (Ch. 3), and asset pricing and arbitrage (Ch. 4). Relation to the Text My treatment and the treatment in text are complementary, and give you two views of the subject that are similar. There are, nevertheless, some minor differences. One trivial one, is that in their generic description of an optimization problem they use S for the set of feasible solutions, while I generally use D. We both use the term standard form for the same thing. Relation to the Text (cont.) I use the term canonical form for the result of the discussion in Section 2.4.1 of the text (eq. 2.8 is an example). Relation to the Text (cont.) In Section 2.4.3, the text uses a tableau representation of linear programs. I do not like tableaus, and prefer dictionaries (due to Chvátal and Strum). I will not discuss them in this course, because of time constraints, but I do cover them in the supplementary note: The Canonical and Dictionary Forms of LP. Notation Linear programming is: The minimization (maximization) of a linear objective, say c 1 x 1 + c 2 x 2 +... + c n x n, Subject to linear constraints. The constraints can be of two types, equations or inequalities, a 1 x 1 + a 2 x 2 +... + a n x n { =}b. Although a non-negativity constraint on a variable, x j 0, is a special case of the inequality constraint, it is usually treated separately. 1
Example: LP for the Aunt s Loan Maximize ER = 0.07x T + 0.082x G + 0.08x I + 0.0775xU Subject to x T + x G + x I + x U 1 0.07x T - 0.10x G - 0.04x I - 0.02x U 0 x T 0, x G 0, x I 0, x U 0 Modeling Transformations It would seem that there is a combinatorially large number of different types of linear programs depending on whether the objective is to minimize or to maximize the objective, whether the constraints are individually equality or inequality, or if the variables are sign restricted or unrestricted in sign. Fortunately, there are simple transformations which can convert the general case into simple classical forms Modeling Transformations We ve already seen how to transform a: Maximization problem into a minimization problem and conversely, and A greater than or equal inequality into a greater than or equal inequality, and conversely. Let s now look at some more transforms: Transformations: An equality constraint, a 1 x 1 + a 2 x 2 +... + a n x n =b, can be replaced by two inequality constraints, a 1 x 1 + a 2 x 2 +... + a n x n b and -a 1 x 1 -a 2 x 2 -... - a n x n -b. An inequality constraint, a 1 x 1 + a 2 x 2 +... + a n x n b can be replaced by an equality constraint, a 1 x 1 + a 2 x 2 +... + a n x n + s = b by adding a non-negative variable s (s 0) called the slack variable. A variable x, unrestricted in sign, can be replaced using two non-negative variables, x + and x -, by x = x + -x -. Using these devices any linear program can be converted into any one of several classical formats. We will first look at the standard form. Later we will look at the canonical, comprehensive and, symmetric forms. The Standard Form: minimize c 1 x 1 + c 2 x 2 +... + c n x n in x 1, x 2,..., x n subject to a 11 x 1 + a 12 x 2 +... + a 1n x n = b 1 a 21 x 1 + a 22 x 2 +... + a 2n x n = b 2... a m1 x 1 + a m2 x 2 +... + a mn x n = b m and x 1 0, x 2 0,... x n 0. 2
Standard Form Without loss of generality we can (and do) assume the b i s are non-negative. Any linear program can be put in standard form. The standard form can also be written in matrix notation. Matrix Standard Form Minimize cx Subject to Ax = b x 0 Matrix Standard Form (cont.) Example Where: c = ( c, c,..., c ) n a11 a12 a1 n a21 a22 a 2n A = am1 am2 amn x 1 b 1 x 2 b 2 b = x = b m x n Let s put the following linear program in standard form: Maximize 3x 1-2x 2 + x 4 Subject to 6x 1 +x 2 +x 3 2 x 2 + x 4 4 x 1 +x 2 + x 3 + x 4 and x 1, x 2, x 3. Example in Standard Form -Minimize -3x 1 +2 x 2 -x + 4 +x - 4 s.t. 6x 1 +x 2 +x 3 +x - 4 -s 1 = 2 x 2 +x + 4 -x - 4 +s 2 x 1 +x 2 +x 3 +x + 4 -x - 4 x 1, x 2, x 3, x 4+, -x 4-, s 1, s 2 Standard Form Convert the objective to minimization Convert the two inequality constraints into equalities. Convert the unconstrained x 4 into the difference of two sign constrained variables, x 4+ and x 4-. 3
Now let s connect linear programming with our earlier results! The Standard Form as a Special Case of (P) f ~ cx { = } D ~ x Ax b, x 0 Two simple but useful ideas The D for a LP in standard form is not changed If: We multiply an equation by a positive constant. We add one equation to another and replace one of the equations with the sum. Lines, half lines, and segments: 0 1 x = (1 λ) x + λx 0 1 0 = x + λ( x x ) D is Convex 0 1 0 1 A[(1 λ) x + λx ] = (1 λ) Ax + λax = (1 λ) b + λb = b 0 1 (1 λ) x + λx 0 for 0 λ 1 But is D Non-Empty? x + 2x 3x + 6x 0 x 0, x 0 4
Is D Bounded? In fact, in this case D is a cone: x x = 2 x 0, x 0 consider x = 2 + t, and x = t, for any t 0 It is a very simple cone consisting of one ray: 2 1 x x = + λ, λ 0 0 1 Can D be affine? It is difficult if all the decision variables x j, j=1,,n are restricted to be non-negative. There are only trivial cases. If you have variables that are not sign constrained it is certainly possible. How about the objective? We have already observed that the linear objective is both a concave function and a convex function. The first implies that a local minimum is a global one, and the second implies that if there is a minimum there is a minimum at an extreme point. In fact, there are 3 possibilities: There exists a minimum, in which case there is a minimum at an extreme point. The problem is infeasible, or The objective can be made arbitrarily small (there exists y, such that: Ay=0, cy<0, y, and, of course, the problem is feasible) 5
A Comprehensive Format Terminology for Variables Minimize x Subject to z=cx l u b Ax b l x u j,..., n j j j l j finite, less than u j finite - - equals u j u j finite, greater than l j + finite + equals l j Variable j Type bounded lower bounded upper bounded free fixed Terminology for Constraints b l b u i i Constraint i type finite, less than b u i finite, greater than b l i range finite + lower bounded - Finite upper bounded - + relaxed The next linear programming format, the canonical form, is not universal in the sense that not all linear programs can be put in this format. First, some examples: equals b i u equals b i l equality Does this LP have a feasible Does this LP have a feasible P1: minimize z x1 + x2 + x4 x + x + x + x 3 5 x2 x3 + x + x 4 6 x 3x + x + x + x 3 4 7 x1 x2 x3 x x 4 5 x6 x7 P2: Minimize z x1 + x2 + x4 x1 + x2 x3 x2 x3 + x4 + x6 x1 3x2 x3 + x4 x1 x2 x3 x4 x5 x6 x7 6
Does this LP have a feasible Does this LP have a feasible P3: Minimize z x1 x2 + x4 x1 x2 + x3 x2 x3 + x4 + x6 x1 3x2 + x3 + x4 x1 x2 x3 x4 x5 x6 x7 P4: Minimize z x1 x2 + x4 x1 + x2 + x3 x2 x3 + x4 + x6 x1 + 3x2 + x3 + x4 x1 x2 x3 x4 x5 x6 x7 Does this LP have a feasible P5': Minimize z = x1 2x2 + x4 x1 + x2 + x3 2x4 = 0 x2 x3 + x4 + x6 x1 + 3x2 + x3 + x4 x1 x2 x3 x4 x5 x6 x7 These examples pretty much define Dantzig s Simplex Method! Note: These examples are given in the tableaux/dictionary format in the supplementary note: In general, the canonical form is: Min s.t. z= cnxn AxN + IxB = b x x N B Where: The parameters are: c N an n- dimensional row vector, A an mxn matrix, and b a non-negative m dimensional column vector. The decision variables x N and x B are n-dimensional and m- dimensional column vectors respectively. 7
Where: The feasible solution obtained by setting the non-basic variables, x N, to 0, and the basic variables, x B,to b, is called the basic solution. Note: The canonical form is a special case of the standard form. In particular, we can multiply the equations by constants, and add an equations to another without changing D. This basic solution can be improved by increasing x 2. We can then get an improved canonical form. How? P4: Minimize z x1 x2 + x4 =z x1 + x2 + x3 x2 x3 + x4 + x6 x1 + 3x2 + x3 + x4 x1 x2 x3 x4 x5 x6 x7 How: 1.We add the top equation to the objective. 2.We add the top equation to the second. 3.We subtract three times the top equation from the last. We obtain the new canonical form (up to ordering the columns). This basic solution is optimal! P4': Minimize z 2x1 + x3 + x4 + 1 x1 + x2 + x3 x1 + x4 + x6 = 6 2x1-2x3 + x4 3x5 x1 x2 x3 x4 x5 x6 x7 The Simplex Method: Given a canonical form, the simplex method proceeds by going from one canonical form to the next, by replacing one basic variable in the basis by one that was non-basic until either an optimal canonical form is found, or it can be seen that the objective is unbounded below. 8
The Simplex Method: That leaves the question of how to get the initial canonical form. It turns out that an auxiliary problem can be solved that ends up with a initial canonical form for the original problem or shows that the original problem was infeasible. The auxiliary problem is in canonical form and can be solved as described previously. 9