Math Models of OR: Extreme Points and Basic Feasible Solutions

Math Models of OR: Extreme Points and Basic Feasible Solutions John E. Mitchell Department of Mathematical Sciences RPI, Troy, NY 1180 USA September 018 Mitchell Extreme Points and Basic Feasible Solutions 1 / 6

Introduction Outline 1 Introduction Let x be a basic feasible solution, show it is an extreme point 3 Let x be an extreme point, show it is a basic feasible solution The general case Mitchell Extreme Points and Basic Feasible Solutions / 6

Introduction Extreme points Recall that if a standard form linear optimization problem min xir n c T x subject to Ax = b x 0 has an optimal solution then it has an optimal solution that is an extreme point. Definition Let P be the feasible set of a linear program. A point x P is an extreme point of P if it is not on the line segment joining two other points in P. Mitchell Extreme Points and Basic Feasible Solutions 3 / 6

Basic feasible solutions Introduction The simplex algorithm works with canonical form tableaus and moves from basic feasible solution to adjacent basic feasible solution. The basic feasible solution corresponding to a canonical form tableau is obtained by setting the nonbasic variables equal to zero and then finding the unique solution for the basic variables. Mitchell Extreme Points and Basic Feasible Solutions / 6

Introduction The relationship We ve seen graphically that basic feasible solutions correspond to extreme points. We prove the following theorem to make this formal: Theorem Let x be a feasible solution to a standard form linear optimization problem. The point x is a basic feasible solution if and only if it is an extreme point of the feasible region of the linear optimization problem. The theorem is an if and only if theorem, so we need to show two directions. Mitchell Extreme Points and Basic Feasible Solutions 5 / 6

Introduction For example x 1 x1 + 3x = 6, x 3 = 0 x 1 = 3, x = 0 x =, x 5 = 0 min xir x 1 x subject to x 1 + 3x apple 6 x 1 apple 3 x apple x 1, x 0 optimal contour x 1 x = 5 0 6 x 1 Mitchell Extreme Points and Basic Feasible Solutions 6 / 6

Let x be a basic feasible solution, show it is an extreme point Outline 1 Introduction Let x be a basic feasible solution, show it is an extreme point 3 Let x be an extreme point, show it is a basic feasible solution The general case Mitchell Extreme Points and Basic Feasible Solutions 7 / 6

Let x be a basic feasible solution, show it is an extreme point Assume x is not an extreme point and derive a contradiction Since x is assumed to not be extreme, it must be the midpoint between two other distinct feasible points x (1) and x (), so x = 1 x (1) + 1 x (). This means that for each component j = 1,...,n, we have x j = 1 x (1) j + 1 x () j. x (1) =(1, ) x =(6, 3) line segment x () =(11, ) Mitchell Extreme Points and Basic Feasible Solutions 8 / 6

Let x be a basic feasible solution, show it is an extreme point Nonbasic components of x We look in particular at nonbasic components of x, so x j = 0. For these components, we have x j = 1 x (1) 1 j + x () j = 0 for 0 since 0 since nonbasic x (1) feasible x () feasible components It follows that we must have x (1) j = x () j = 0 for all the nonbasic components of x. But once the nonbasic components are fixed, the values of the basic variables are then uniquely determined: they are equal to the corresponding entries in b. So we must have x (1) = x () = x. So the points x (1) and x () are not distinct, so x is actually an extreme point. Mitchell Extreme Points and Basic Feasible Solutions 9 / 6

Outline 1 Introduction Let x be a basic feasible solution, show it is an extreme point 3 Let x be an extreme point, show it is a basic feasible solution The general case Mitchell Extreme Points and Basic Feasible Solutions 10 / 6

Introduction to proof This direction is harder to prove, and the proof needs some linear algebra. We prove it by contradiction, so we assume x is not a BFS and show it is not an extreme point. To make the proof a little simpler, we assume x has exactly m positive components; the proof can be extended to handle the general case. Mitchell Extreme Points and Basic Feasible Solutions 11 / 6

Outline 1 Introduction Let x be a basic feasible solution, show it is an extreme point 3 Let x be an extreme point, show it is a basic feasible solution The general case Mitchell Extreme Points and Basic Feasible Solutions 1 / 6

An example We first work an example before abstracting the proof. min xir 6 3x 1 + x + x 3 subject to x 1 + x + x 3 + x = 6 x 1 + x 3 + x 5 = x 1 + x + x 6 = x 1,...,x 6 0 The point x =(1, 3,, 0, 0, 0) is feasible. Is it a BFS? The tableau is x 1 x x 3 x x 5 x 6 0 3 1 0 0 0 6 1 1 1 1 0 0 0 1 0 1 0 1 1 0 0 0 1 Mitchell Extreme Points and Basic Feasible Solutions 13 / 6

Two pivots x 1 x x 3 x x 5 x 6 0 3 1 0 0 0 6 1 1 1 1 0 0 0 1 0 1 0 1 1 0 0 0 1 R 0 R 3, R 1 R 3! x 1 x x 3 x x 5 x 6 5 0 1 0 0 0 1 1 0 1 0 1 0 1 0 1 1 0 0 0 1 R 0 R 1, R R 1! x 1 x x 3 x x 5 x 6 6 3 0 0 1 0 3 0 1 1 0 1 0 0 0 0 1 1 1 1 1 0 0 0 1 Mitchell Extreme Points and Basic Feasible Solutions 1 / 6

We don t have a BFS x 1 x x 3 x x 5 x 6 6 3 0 0 1 0 3 0 1 1 0 1 0 0 0 0 1 1 1 1 1 0 0 0 1 The second constraint does not involve x 1, x, or x 3. So our given feasible solution is not a bfs. Mitchell Extreme Points and Basic Feasible Solutions 15 / 6

A little linear algebra Equivalently, we can say that this means the original first three columns of A are linearly dependent. In fact, notice that 1 1 1 0 1 1 1 0 3 5 1 1 3 5 = Because of this equality, we have a direction we can move in and still maintain feasibility. 0 0 0 3 5. Mitchell Extreme Points and Basic Feasible Solutions 16 / 6

The direction Let d =(1, 1,, 0, 0, 0) T IR 6. Then ˆx := x + td satisfies Aˆx = b for any t, positive or negative, since A x = b and Ad = 0. Further, provided 1 apple t apple 1, we get ˆx 0. Mitchell Extreme Points and Basic Feasible Solutions 17 / 6

Two new feasible points Want to show x is not extreme. Let s take the two new feasible points x (1) and x () with t = 1 and t = 1, respectively (the x, x 5, and x 6 components of d, x (1), x () and x are all zero, so we don t write them out explicitly): 3 3 3 1 1 x (1) = x + d = 3 5 + 1 5 = 5 0 x () = x d = 1 3 3 5 1 1 3 5 = 0 3 5 Mitchell Extreme Points and Basic Feasible Solutions 18 / 6

x is not extreme Notice that 1 x (1) + 1 x () = 1 0 3 5 + 1 0 3 5 = 1 3 3 5 = x. So x is not an extreme point. Mitchell Extreme Points and Basic Feasible Solutions 19 / 6

Graph this example x 3 6 0 6 x 6 x 1 + x + x 3 = 6, x = 0 x 1 Mitchell Extreme Points and Basic Feasible Solutions 0 / 6

Graph this example x 3 6 x 1 + x =, x 6 = 0 0 6 x 6 x 1 + x + x 3 = 6, x = 0 x 1 Mitchell Extreme Points and Basic Feasible Solutions 0 / 6

Graph this example x 3 6 x 1 + x =, x 6 = 0 0 6 x 6 x 1 + x + x 3 = 6, x = 0 x 1 Mitchell Extreme Points and Basic Feasible Solutions 0 / 6

Graph this example x 3 6 x 1 + x =, x 6 = 0 x 1 + x 3 =, x 5 = 0 0 6 x 6 x 1 + x + x 3 = 6, x = 0 x 1 Mitchell Extreme Points and Basic Feasible Solutions 0 / 6

Graph this example x 3 6 x 1 + x =, x 6 = 0 (0,, ) =x () x 1 + x 3 =, x 5 = 0 (1, 3, ) = x 0 6 (,, 0) =x (1) x portion of feasible region with x = x 5 = x 6 = 0 6 x 1 + x + x 3 = 6, x = 0 x 1 Mitchell Extreme Points and Basic Feasible Solutions 0 / 6

Linear algebra characterization of BFS The example illustrates the following result: Let x be a feasible solution to a standard form linear program with m n constraint matrix A with rank equal to m. The point x is a basic feasible solution if and only if the columns of A corresponding to the positive components of x are linearly independent. Mitchell Extreme Points and Basic Feasible Solutions 1 / 6

The general case Outline 1 Introduction Let x be a basic feasible solution, show it is an extreme point 3 Let x be an extreme point, show it is a basic feasible solution The general case Mitchell Extreme Points and Basic Feasible Solutions / 6

The general case Return to the general case Recall, we assume x is not a basic feasible solution and try to derive a contradiction. Without loss of generality, we can assume the m positive components of x are the first m components, by renumbering the components if necessary. So we have x j > 0 for j = 1,...,m = 0 for j = m + 1,...,n Mitchell Extreme Points and Basic Feasible Solutions 3 / 6

The general case Row reducing A We can write the constraint matrix A as A = 6 B {z} m columns 3 7 {z} N 5 (n m) columns If x was a basic feasible solution, we would be able to use elementary row operations to turn this into a canonical form, so h i A =[B N]! I ˆN for some m (n m) matrix ˆN, where I is the m m identity matrix. Mitchell Extreme Points and Basic Feasible Solutions / 6

The general case But we don t have a BFS Since x is not a basic feasible solution, this row reduction cannot be possible. That means the columns of B are linearly dependent, which is equivalent to stating that there exists a vector d B IR m with Bd B = 0, where d B has at least one nonzero component. We can extend d B out to a vector d IR n by appending zeroes: let d j = d B j for j = 1,...,m 0 for j = m + 1,...,n so d = apple d B 0 m components (n m) components Mitchell Extreme Points and Basic Feasible Solutions 5 / 6

The general case Moving in our direction Notice that so for any scalar t we have apple d B Ad =[B N] 0 = Bd B = 0, A( x + td) =A x + tad = b + 0 = b. Further, since d j = 0 if x j = 0, we have x + td close to zero (positive or negative). Thus, choose t > 0 so that both x + td and x 0 for t sufficiently td are feasible. Then notice that x is the midpoint of these two feasible points: x = 1 x + td + 1 x td. Thus, x is not an extreme point. Thus, we ve proved the contrapositive, so we do indeed have the result that if x is an extreme point then it is a basic feasible solution. Mitchell Extreme Points and Basic Feasible Solutions 6 / 6