MVE165/MMG631 Extreme points of convex polyhedra; reformulations; basic feasible solutions; the simplex method Ann-Brith Strömberg 2015 03 27
Linear programs, convex polyhedra, extreme points A linear optimization model a linear program n minimize z = c j x j subject to n a ij x j b i, x j 0, c j, a ij, b i : constant parameters i = 1,..., m j = 1,..., n Expressed in vector notation min s.t. z = c T x Ax b x 0 n c, x R n, b R m, A R m n The feasible region is a polyhedron, X R n + X := { x 0 n n } a ij x j b i, i = 1,..., m = { x 0 n Ax b }
Linear programs, convex polyhedra and extreme points (Ch. 4.1) Definition (Convex combination) A convex combination of the points x p, p = 1,..., P, is a point x that can be expressed as P x = λ p x p ; p=1 P λ p = 1; p=1 λ p 0, p = 1,..., P [Draw on the board]
Linear programs, convex polyhedra and extreme points (Ch. 4.1) Linear constraints form a convex set The feasible region of a linear program is a convex set, since for any two feasible points x 1 and x 2 and any λ [0, 1] it holds that and n ) a ij (λxj 1 + (1 λ)xj 2 = λ n a ij xj 1 + (1 λ) n a ij xj 2 λb i + (1 λ)b i = b i, i = 1,..., m λx 1 j + (1 λ)x 2 j 0, j = 1,..., n [Draw on the board]
Linear programs, convex polyhedra and extreme points (Ch. 4.1) Definition (Extreme point (Def. 4.2)) The point x k is an extreme point of the polyhedron X if x k X and it is not possible to express x k as a strict convex combination of two distinct points in X. I.e: Given x 1 X, x 2 X, and 0 < λ < 1, it holds that x k = λx 1 + (1 λ)x 2 only if x k = x 1 = x 2. [Draw on the board] Theorem (Optimal solution in an extreme point (Th. 4.2)) Assume that the feasible region X = {x 0 n Ax b} is non-empty and bounded. Then, the minimum value of the objective c T x is attained at (at least) one extreme point x k of X. [Proof on the board]
A general linear program notation Definition (Notation of linear programs) minimize or maximize subject to a i1 x 1 +... + a in x n x j c 1 x 1 +... + c n x n = 0 unrestricted in sign 0 b i, i = 1,..., m, j = 1,..., n The blue notation refers to the standard form
The standard form and the simplex method for linear programs (Ch. 4.2) Every linear program can be reformulated such that: all constraints are expressed as equalities with non-negative right hand sides all variables involved are restricted to be non-negative Referred to as the standard form These requirements streamline the calculations of the simplex method Software solvers (e.g., Cplex, GLPK, Clp, Gurobi) can handle also inequality constraints and unrestricted variables the reformulations are made automatically
The simplex method standard form reformulations Slack variables: n a ij x j b i, i x j 0, j n a ij x j +s i = b i, i x j 0, j s i 0, i The lego example: 2x 1 +x 2 6 2x 1 +2x 2 8 x 1, x 2 0 2x 1 +x 2 +s 1 = 6 2x 1 +2x 2 +s 2 = 8 x 1, x 2, s 1, s 2 0 s 1 and s 2 are called slack variables they fill out the (positive) distances between the left and right hand sides
The simplex method standard form reformulations Surplus variables: n a ij x j b i, i x j 0, j n a ij x j s i = b i, i x j 0, j s i 0, i Surplus variable s 3 (a different example): [ ] [ x1 + x 2 800 x1 + x 2 s 3 = 800 x 1, x 2 0 x 1, x 2, s 3 0 ]
The simplex method standard form reformulations Suppose that b < 0: n n a j x j b ( a j )x j b x j 0, j x j 0, j n a j x j s = b x j 0, j s 0 Non-negative right hand side: [ ] [ ] [ ] x1 x 2 23 x1 + x 2 23 x1 + x 2 s 4 = 23 x 1, x 2 0 x 1, x 2 0 x 1, x 2, s 4 0
The simplex method standard form reformulations Suppose that some of the variables are unconstrained (here: k < n). Replace x j with x 1 j x 2 j for the corresponding indices: n a j x j b x j 0, j = 1,..., k k a j x j + n j=k+1 a j (xj 1 xj 2 ) + s = b x j 0, j = 1,..., k, xj 1 0, xj 2 0, j = k + 1,..., n s 0 Sign-restricted (non-negative) variables: [ ] [ x1 + x 2 10 x1 + x 2 1 x 2 2 10 ] [ x1 + x x 1 0 x 1, x2 1, x 2 2 0 2 1 x 2 2 + s ] 5 = 10 x 1, x2 1, x 2 2, s 5 0
Basic feasible solutions (Ch. 4.3) Consider m equations with n variables, where m n Set n m variables to zero and solve (if possible) the remaining (m m) system of equations If the solution is unique, it is called a basic solution A basic solution corresponds to an intersection (feasible (x 0) or infeasible (x 0)) of m hyperplanes in R m Each extreme point of the feasible set is an intersection of m hyperplanes such that all variable values are 0 Basic feasible solution extreme point of the feasible set a 11 x 1 +... + a 1n x n = b 1 x 1 0 a 21 x 1 +... + a 2n x n = b 2 x 2 0 a m1 x 1 +... + a mn x n = b m x n 0
Basic feasible solutions Assume that m < n and that b i 0, i = 1,..., m, and let c 1 a 11... a 1n b 1 x 1 c =., A =....., b =., x =.. c n a m1... a mn x n Consider the linear program to b m minimize x subject to z = c T x Ax = b x 0 Partition x into m basic variables x B and n m non-basic variables x N, such that x = (x B, x N ). Analogously, let c = (c B, c N ) and A = (A B, A N ) (B, N) The matrix B R m m with inverse B 1 (if it exists)
Basic feasible solutions (Ch. 4.8) Rewrite the linear program as minimize z = c T B x B + c T N x N (1a) subject to Bx B + Nx N = b (1b) x B 0 m, x N 0 n m (1c) Multiply the equation (1b) with B 1 from the left: B 1 Bx B + B 1 Nx N = x B + B 1 Nx N = B 1 b x B = B 1 b B 1 Nx N (2) Replace x B in (1) by the expression (2): c T B x B+c T N x N = c T B B 1 (b Nx N )+c T N x N = c T B B 1 b+(c T N ct B B 1 N)x N minimize z = c T B B 1 b + (c T N ct B B 1 N)x N subject to B 1 b B 1 Nx N 0 m x N 0 n m
Basic feasible solutions The rewritten program minimize z = c T B B 1 b + (c T N ct B B 1 N)x N (3a) subject to B 1 b B 1 Nx N 0 m (3b) x N 0 n m (3c) At the basic solution defined by B {1,..., n}: Each non-basic variable takes the value 0, i.e., x N = 0 The basic variables take the values x B = B 1 b B 1 Nx N = B 1 b The value of the objective function is z = c T B B 1 b The basic solution is feasible if B 1 b 0 m
The simplex method: Optimality and feasibility and change of basis (Ch. 4.4) Optimality condition (for minimization) The basis B is optimal if c T N ct B B 1 N 0 n m (marginal values = reduced costs 0) If not, choose as entering variable j N the one with the lowest (negative) value of the reduced cost c j c T B B 1 A j Feasibility condition For all i B it holds that x i = (B 1 b) i (B 1 A j ) i x j Choose the leaving variable i B according to { (B i 1 } b) i = arg min i B (B 1 A j ) i (B 1 A j ) i > 0
Basic feasible solutions, example Constraints: x 1 23 (1) 0.067x 1 + x 2 6 (2) 3x 1 + 8x 2 85 (3) x 1, x 2 0 Add slack variables: x 1 +s 1 = 23 (1) 0.067x 1 +x 2 +s 2 = 6 (2) 3x 1 +8x 2 +s 3 = 85 (3) x 1, x 2, s 1, s 2, s 3 0 y 10 C (3) (1) B D 5 E (2) F I 1 A G H x 1 5 10 15 20 25
Basic and non-basic variables and solutions basic basic solution non-basic point feasible? variables variables (0, 0) s 1, s 2, s 3 23 6 85 x 1, x 2 A yes s 1, s 2, x 1 5 1 4 1 28 1 s 3 9 3 3, x 2 H no s 1, s 2, x 2 23 4 5 10 5 x 8 8 1, s 3 C no s 1, x 1, s 3 67 90 185 s 2, x 2 I no s 1, x 2, s 3 23 6 37 s 2, x 1 B yes x 1, s 2, s 3 23 4 7 16 s 15 1, x 2 G yes x 2, s 2, s 3 - - - s 1, x 1 - - x 1, x 2, s 1 15 5 8 s 2, s 3 D yes x 1, x 2, s 2 23 2 2 7 s 15 1, s 3 F yes x 1, x 2, s 3 23 4 7 19 11 s 15 15 1, s 2 E no y 10 C (3) (1) 5 B D E (2) F 1 I A G H x 1 5 10 15 20 25
10 5 1 y C B A (3) 1 5 10 15 20 D F G (1) E 25 (2) H x I Basic feasible solutions correspond to solutions to the system of equations that fulfil non-negativity A: x 1 = x 2 = 0 B: x 1 = s 2 = 0 D: s 3 = s 2 = 0 F: s 3 = s 1 = 0 G: x 2 = s 1 = 0 s 1 = 23 s 2 = 6 s 3 = 85 x 1 +s 1 = 23 0.067x 1 +x 2 +s 2 = 6 3x 1 +8x 2 +s 3 = 85 s 1 = 23 x 2 = 6 8x 2 +s 3 = 85 x 1 +s 1 = 23 0.067x 1 +x 2 = 6 3x 1 +8x 2 = 85 x 1 = 23 0.067x 1 +x 2 +s 2 = 6 3x 1 +8x 2 = 85 x 1 = 23 0.067x 1 +s 2 = 6 3x 1 +s 3 = 85
10 5 1 y C B A (3) 1 5 10 15 20 D F G (1) E 25 (2) H x I Basic infeasible solutions corresp. to solutions to the system of equations with one or more variables < 0 H: x 2 = s 3 = 0 C: x 1 = s 3 = 0 I: s 2 = x 2 = 0 -: s 1 = x 1 = 0 E: s 1 = s 2 = 0 x 1 +s 1 = 23 0.067x 1 +x 2 +s 2 = 6 3x 1 +8x 2 +s 3 = 85 x 1 +s 1 = 23 0.067x 1 +s 2 = 6 3x 1 = 85 s 1 = 23 x 2 +s 2 = 6 8x 2 = 85 x 1 +s 1 = 23 0.067x 1 = 6 3x 1 +s 3 = 85 0 = 23 x 2 +s 2 = 6 8x 2 +s 3 = 85 x 1 = 23 0.067x 1 +x 2 = 6 3x 1 +8x 2 +s 3 = 85
Basic feasible solutions and the simplex method Express the m basic variables in terms of the n m non-basic variables Example: Start at x 1 = x 2 = 0 s 1, s 2, s 3 are basic x 1 +s 1 = 23 1 15 x 1 +x 2 +s 2 = 6 3x 1 +8x 2 +s 3 = 85 Express s 1, s 2, and s 3 in terms of x 1 and x 2 (non-basic): s 1 = 23 x 1 s 2 = 6 1 15 x 1 x 2 s 3 = 85 3x 1 8x 2 We wish to maximize the objective function 2x 1 + 3x 2 Express the objective in terms of the non-basic variables: (maximize) z = 2x 1 + 3x 2 z 2x 1 3x 2 = 0
Basic feasible solutions and the simplex method The first basic solution can be represented as z +2x 1 +3x 2 = 0 (0) x 1 +s 1 = 23 (1) 1 15 x 1 + x 2 + s 2 = 6 (2) 3x 1 + 8x 2 + s 3 = 85 (3) Marginal values for increasing the non-basic variables x 1 and x 2 from zero: 2 and 3, resp. Choose x 2 let x 2 enter the basis Draw graph!! One basic variable (s 1, s 2, or s 3 ) must leave the basis. Which? The value of x 2 can increase until some basic variable reaches the value 0: (2) : s 2 = 6 x 2 0 x 2 6 (3) : s 3 = 85 8x 2 0 x 2 10 5 8 s 2 will leave the basis } s 2 = 0 when x 2 = 6 (and s 3 = 37)
Change basis through row operations Eliminate s 2 from the basis, let x 2 enter the basis using row operations: z +2x 1 +3x 2 = 0 (0) x 1 +s 1 = 23 (1) 1 15 x 1 +x 2 +s 2 = 6 (2) 3x 1 +8x 2 +s 3 = 85 (3) z + 9 5 x 1 3s 2 = 18 (0) 3 (2) x 1 +s 1 = 23 (1) 0 (2) 1 15 x 1 +x 2 +s 2 = 6 (2) 37 15 x 1 8s 2 +s 3 = 37 (3) 8 (2) Corresponding basic solution: s 1 = 23, x 2 = 6, s 3 = 37. Nonbasic variables: x 1 = s 2 = 0 The marginal value of x 1 is 9 5 > 0. Let x 1 enter the basis Which one should leave? s 1, x 2, or s 3?
Change basis... z + 9 5 x 1 3s 2 = 18 (0) x 1 +s 1 = 23 (1) 1 15 x 1 +x 2 +s 2 = 6 (2) 37 15 x 1 8s 2 +s 3 = 37 (3) The value of x 1 can increase until some basic variable reaches the value 0: (1) : s 1 = 23 x 1 0 x 1 23 (2) : x 2 = 6 1 15 x 1 0 x 1 90 (3) : s 3 = 37 37 15 x 1 0 x 1 15 x 1 enters the basis and s 3 leaves the basis s 3 = 0 when x 1 = 15 Perform row operations: z +2.84s 2 0.73s 3 = 45 (0) (3) 15 s 1 +3.24s 2 0.41s 3 = 8 (1) (3) 15 37 x 2 +1.22s 2 0.03s 3 = 5 (2) (3) 15 x 1 3.24s 2 +0.41s 3 = 15 (3) 15 37 37 9 5 37 1 15
Change basis... z +2.84s 2 0.73s 3 = 45 (0) s 1 +3.24s 2 0.41s 3 = 8 (1) x 2 +1.22s 2 0.03s 3 = 5 (2) x 1 3.24s 2 +0.41s 3 = 15 (3) Let s 2 enter the basis (marginal value > 0) The value of s 2 can increase until some basic variable = 0: (1) : s 1 = 8 3.24s 2 0 s 2 2.47 (2) : x 2 = 5 1.22s 2 0 s 2 4.10 s 1 = 0 when s (3) : x 1 = 15 + 3.24s 2 0 s 2 4.63 2 = 2.47 s 2 enters the basis and s 1 will leave the basis Perform row operations: z 0.87s 1 0.37s 3 = 52 (0) (1) 2.84 3.24 0.31s 1 +s 2 0.12s 3 = 2.47 (1) 1 3.24 x 2 0.37s 1 +0.12s 3 = 2 (2) (1) 1.22 3.24 x 1 +s 1 = 23 (3)+(1)
Optimal basic solution z 0.87s 1 0.37s 3 = 52 0.31s 1 +s 2 0.12s 3 = 2.47 x 2 0.37s 1 +0.12s 3 = 2 x 1 +s 1 = 23 No marginal value is positive. No improvement can be made The optimal basis is given by s 2 = 2.47, x 2 = 2, and x 1 = 23 Non-basic variables: s 1 = s 3 = 0 Optimal value: z = 52 y 5 1 1 5 10 15 20 25 x
Summary of the solution course basis z x 1 x 2 s 1 s 2 s 3 RHS z 1 2 3 0 0 0 0 s 1 0 1 0 1 0 0 23 s 2 0 0.067 1 0 1 0 6 s 3 0 3 8 0 0 1 85 z 1 1.80 0 0-3 0-18 s 1 0 1 0 1 0 0 23 x 2 0 0.07 1 0 1 0 6 s 3 0 2.47 0 0-8 1 37 z 1 0 0 0 2.84-0.73-45 s 1 0 0 0 1 3.24-0.41 8 x 2 0 0 1 0 1.22-0.03 5 x 1 0 1 0 0-3.24 0.41 15 z 1 0 0-0.87 0-0.37-52 s 2 0 0 0 0.31 1-0.12 2.47 x 2 0 0 1-0.37 0 0.12 2 x 1 0 1 0 1 0 0 23
Solve the lego problem using the simplex method! maximize z = 1600x 1 + 1000x 2 subject to 2x 1 + x 2 6 2x 1 + 2x 2 8 x 1, x 2 0 Homework!!