Math Models of OR: Some Definitions John E. Mitchell Department of Mathematical Sciences RPI, Troy, NY 12180 USA September 2018 Mitchell Some Definitions 1 / 20
Active constraints Outline 1 Active constraints 2 Basic feasible solutions 3 Simplex directions 4 Optimal solutions Mitchell Some Definitions 2 / 20
Active constraints Active constraints Definition Let x be a feasible solution to a linear optimization problem. A constraint of the LOP is active at x if it is satisfied at equality by the point x. Note that equality constraints are active at any feasible point. Inequality constraints may be active at some feasible points and inactive at other feasible points. Mitchell Some Definitions 3 / 20
Active constraints Example For example, consider the linear optimization problem min x IR 2 2x 1 x 2 subject to x 1 x 2 2 3x 1 + x 2 10 x j 0, j = 1, 2 (1) We add slack variables to get an equivalent problem in standard form: min x IR 2 2x 1 x 2 subject to x 1 x 2 + x 3 = 2 3x 1 + x 2 + x 4 = 10 x j 0, j = 1,..., 4 (2) Mitchell Some Definitions 4 / 20
Active constraints Example Graphing the problem in the original (x 1, x 2 )-space gives: x 2 10 3x 1 + x 2 = 10 5 x 1 x 2 = 2 0 (3, 1) 1 2 3 4 x 1 point x active inequalities inactive inequalities (3,1) x 1 x 2 2, 3x 1 + x 2 10 x 1 0, x 2 0 (1,2) none all (0,7) x 1 0 x 1 x 2 2, 3x 1 + x 2 10, x 2 0 (2,4) 3x 1 + x 2 10 x 1 x 2 2, x 1 0, x 2 0 Mitchell Some Definitions 5 / 20
Active constraints Example Graphing the problem in the original (x 1, x 2 )-space gives: x 2 10 3x 1 + x 2 = 10 5 (1, 2) 0 1 2 3 4 x 1 x 2 = 2 x 1 point x active inequalities inactive inequalities (3,1) x 1 x 2 2, 3x 1 + x 2 10 x 1 0, x 2 0 (1,2) none all (0,7) x 1 0 x 1 x 2 2, 3x 1 + x 2 10, x 2 0 (2,4) 3x 1 + x 2 10 x 1 x 2 2, x 1 0, x 2 0 Mitchell Some Definitions 5 / 20
Example Active constraints Graphing the problem in the original (x 1, x 2 )-space gives: x 2 10 (0, 7) 5 3x 1 + x 2 = 10 x 1 x 2 = 2 0 1 2 3 4 x 1 point x active inequalities inactive inequalities (3,1) x 1 x 2 2, 3x 1 + x 2 10 x 1 0, x 2 0 (1,2) none all (0,7) x 1 0 x 1 x 2 2, 3x 1 + x 2 10, x 2 0 (2,4) 3x 1 + x 2 10 x 1 x 2 2, x 1 0, x 2 0 Mitchell Some Definitions 5 / 20
Active constraints Example Graphing the problem in the original (x 1, x 2 )-space gives: x 2 10 3x 1 + x 2 = 10 5 (2, 4) x 1 x 2 = 2 0 1 2 3 4 x 1 point x active inequalities inactive inequalities (3,1) x 1 x 2 2, 3x 1 + x 2 10 x 1 0, x 2 0 (1,2) none all (0,7) x 1 0 x 1 x 2 2, 3x 1 + x 2 10, x 2 0 (2,4) 3x 1 + x 2 10 x 1 x 2 2, x 1 0, x 2 0 Mitchell Some Definitions 5 / 20
Active constraints Example Now consider the formulation with explicit slacks in standard form. The equality constraints are both active at any feasible point. A (possibly empty) subset of the nonnegativity constraints are active. x 2 10 x 4 = 0 5 x 3 = 0 0 (3, 1) 1 2 3 4 x 1 point x active inequalities inactive inequalities (3,1,0,0) x 3 0, x 4 0 x 1 0, x 2 0 (1,2,3,5) none all (0,7,9,3) x 1 0 x 2 0, x 3 0, x 4 0 (2,4,4,0) x 4 0 x 1 0, x 2 0, x 3 0 Mitchell Some Definitions 6 / 20
Active constraints Example Now consider the formulation with explicit slacks in standard form. The equality constraints are both active at any feasible point. A (possibly empty) subset of the nonnegativity constraints are active. x 2 10 x 4 = 0 5 (1, 2) 0 1 2 3 4 x 3 = 0 x 1 point x active inequalities inactive inequalities (3,1,0,0) x 3 0, x 4 0 x 1 0, x 2 0 (1,2,3,5) none all (0,7,9,3) x 1 0 x 2 0, x 3 0, x 4 0 (2,4,4,0) x 4 0 x 1 0, x 2 0, x 3 0 Mitchell Some Definitions 6 / 20
Active constraints Example Now consider the formulation with explicit slacks in standard form. The equality constraints are both active at any feasible point. A (possibly empty) subset of the nonnegativity constraints are active. x 2 10 (0, 7) 5 x 4 = 0 x 3 = 0 0 1 2 3 4 point x active inequalities inactive inequalities (3,1,0,0) x 3 0, x 4 0 x 1 0, x 2 0 (1,2,3,5) none all (0,7,9,3) x 1 0 x 2 0, x 3 0, x 4 0 (2,4,4,0) x 4 0 x 1 0, x 2 0, x 3 0 Mitchell Some Definitions 6 / 20 x 1
Active constraints Example Now consider the formulation with explicit slacks in standard form. The equality constraints are both active at any feasible point. A (possibly empty) subset of the nonnegativity constraints are active. x 2 10 x 4 = 0 5 (2, 4) x 3 = 0 0 1 2 3 4 x 1 point x active inequalities inactive inequalities (3,1,0,0) x 3 0, x 4 0 x 1 0, x 2 0 (1,2,3,5) none all (0,7,9,3) x 1 0 x 2 0, x 3 0, x 4 0 (2,4,4,0) x 4 0 x 1 0, x 2 0, x 3 0 Mitchell Some Definitions 6 / 20
Basic feasible solutions Outline 1 Active constraints 2 Basic feasible solutions 3 Simplex directions 4 Optimal solutions Mitchell Some Definitions 7 / 20
Basic feasible solutions Example Our standard form linear optimization problem has the form min x IR n c T x subject to Ax = b x 0 where A is an m n matrix, b IR m, and x, c IR n. Mitchell Some Definitions 8 / 20
Basic feasible solutions Basic sequence A canonical form tableau for a linear optimization problem in standard form contains the m columns of the identity matrix, as in, for example: x 1 x 2 x 3 x 4 x 5 2 0 1 0 2 0 3 0 2 0 1 1 1 1 5 0 2 0 13 0 10 1 3 0 The first column of the identity appears in the x 5 -column, the second in the x 1 -column, and the third in the x 3 -column. This gives us a basic sequence S = (5, 1, 3) associated with the canonical form tableau. The tableau is in canonical form with respect to the basic sequence S. Mitchell Some Definitions 9 / 20
Basic feasible solutions Basic and nonbasic variables x 1 x 2 x 3 x 4 x 5 2 0 1 0 2 0 3 0 2 0 1 1 1 1 5 0 2 0 13 0 10 1 3 0 The variables corresponding to the columns of the identity are the basic variables, so x 5, x 1, and x 3 for the example. The remaining variables are the nonbasic variables, so x 2 and x 4 for the example. The basic feasible solution corresponding to the canonical form tableau is obtained by setting the nonbasic variables equal to zero and then finding the unique solution for the basic variables. In the example, once we set x 2 = x 4 = 0, the only solution to the system Ax = b is x 5 = 3, x 1 = 1, and x 3 = 13. In general, a basic feasible solution has m basic variables (one for each row) and n m nonbasic variables. Mitchell Some Definitions 10 / 20
Basic feasible solutions Basic and nonbasic variables x 1 x 2 x 3 x 4 x 5 2 0 1 0 2 0 3 0 2 0 1 1 1 1 5 0 2 0 13 0 10 1 3 0 The variables corresponding to the columns of the identity are the basic variables, so x 5, x 1, and x 3 for the example. The remaining variables are the nonbasic variables, so x 2 and x 4 for the example. The basic feasible solution corresponding to the canonical form tableau is obtained by setting the nonbasic variables equal to zero and then finding the unique solution for the basic variables. In the example, once we set x 2 = x 4 = 0, the only solution to the system Ax = b is x 5 = 3, x 1 = 1, and x 3 = 13. In general, a basic feasible solution has m basic variables (one for each row) and n m nonbasic variables. Mitchell Some Definitions 10 / 20
Basic feasible solutions Basic and nonbasic variables x 1 x 2 x 3 x 4 x 5 2 0 1 0 2 0 3 0 2 0 1 1 1 1 5 0 2 0 13 0 10 1 3 0 The variables corresponding to the columns of the identity are the basic variables, so x 5, x 1, and x 3 for the example. The remaining variables are the nonbasic variables, so x 2 and x 4 for the example. The basic feasible solution corresponding to the canonical form tableau is obtained by setting the nonbasic variables equal to zero and then finding the unique solution for the basic variables. In the example, once we set x 2 = x 4 = 0, the only solution to the system Ax = b is x 5 = 3, x 1 = 1, and x 3 = 13. In general, a basic feasible solution has m basic variables (one for each row) and n m nonbasic variables. Mitchell Some Definitions 10 / 20
Basic feasible solutions Extreme points Each basic feasible solution is an extreme point or corner point of the set of feasible solutions to a standard form LOP. P = {x IR n : Ax = b, x 0} Each extreme point corresponds to at least one basic sequence. It may correspond to more than one basic sequence if one of the basic variables is equal to zero; we will return to this point later, and the complications it may cause. We will also define extreme points more formally later. Mitchell Some Definitions 11 / 20
Simplex directions Outline 1 Active constraints 2 Basic feasible solutions 3 Simplex directions 4 Optimal solutions Mitchell Some Definitions 12 / 20
Simplex directions A pivot In a simplex pivot, one of the nonbasic variables becomes a basic variable and one of the basic variables becomes nonbasic. For example, pivoting on entry a 14 in this canonical form tableau: x 1 x 2 x 3 x 4 x 5 2 0 1 0 2 0 3 0 2 0 1 1 1 1 5 0 2 0 13 0 10 1 3 0 leads to the canonical form tableau R 0 + 2R 1, R 2 + 2R 1, R 3 3R 1 x 1 x 2 x 3 x 4 x 5 4 0 3 0 0 2 3 0 2 0 1 1 7 1 1 0 0 2 4 0 4 1 0 3 The basic sequence has been updated from (5, 1, 3) to (4, 1, 3). The variable x 4 has replaced the variable x 5 in the basis. Mitchell Some Definitions 13 / 20
Simplex directions Change in x If we let x 0 IR 5 denote the old BFS and x 1 IR 5 denote the new one, we have 1 7 6 2 x 0 0 = 13 0, x 1 0 = 4 3, x 1 x 0 0 = 9 3 = 3 0 3 1 3 0 3 1 Notice that the change x 1 x 0 is expressed as a scale factor multiplied by a vector. The scale factor is equal to the minimum ratio calculated when determining the pivot element, 3 for the example. Mitchell Some Definitions 14 / 20
Simplex directions Change in x If we let x 0 IR 5 denote the old BFS and x 1 IR 5 denote the new one, we have 1 7 6 2 x 0 0 = 13 0, x 1 0 = 4 3, x 1 x 0 0 = 9 3 = 3 0 3 1 3 0 3 1 Notice that the change x 1 x 0 is expressed as a scale factor multiplied by a vector. The scale factor is equal to the minimum ratio calculated when determining the pivot element, 3 for the example. Mitchell Some Definitions 14 / 20
Simplex directions Change in x If we let x 0 IR 5 denote the old BFS and x 1 IR 5 denote the new one, we have 1 7 6 2 x 0 0 = 13 0, x 1 0 = 4 3, x 1 x 0 0 = 9 3 = 3 0 3 1 3 0 3 1 Notice that the change x 1 x 0 is expressed as a scale factor multiplied by a vector. The scale factor is equal to the minimum ratio calculated when determining the pivot element, 3 for the example. Mitchell Some Definitions 14 / 20
Simplex directions Simplex direction The direction is called the simplex direction and can be derived from the pivot column in the original tableau: d = 1 3 (x 1 x 0 ) = 2 0 3 1 1 negative of a 24, row corresponding to x 1 0, for a variable staying nonbasic negative of a 34, row corresponding to x 3 1, for the variable entering the basis negative of a 14, row corresponding to x 5 Simplex directions always have this structure: 1 for the variable entering the basis 0 for the other nonbasic variables negative of the appropriate entry in the pivot column for the old basic variables Mitchell Some Definitions 15 / 20
Simplex directions Edges If a basic feasible solution can be obtained from another one via a single simplex pivot, we say the basic feasible solutions are adjacent. Geometrically, this corresponds to moving along an edge of the feasible region. We will define edges more formally later. Mitchell Some Definitions 16 / 20
Optimal solutions Outline 1 Active constraints 2 Basic feasible solutions 3 Simplex directions 4 Optimal solutions Mitchell Some Definitions 17 / 20
Optimal solutions Global and local optimality For a general optimization problem: Want to find a globally optimal solution: the best possible feasible solution. May have to be satisfied with finding a locally optimal solution: there is no better point in a neighborhood of the current point. For a linear optimization problem, any locally optimal solution is globally optimal. Mitchell Some Definitions 18 / 20
Optimal solutions Edges If a standard form LOP has an optimal solution then it has an optimal solution that is a BFS. The simplex algorithm will find such an optimal solution. It may have multiple optimal solutions, corresponding to a face of the set of feasible solutions. Each optimal BFS can be reached from any other optimal BFS by a sequence of pivots that only visits optimal basic feasible solutions. Mitchell Some Definitions 19 / 20
Optimal solutions An example where every feasible solution is optimal Let x be a feasible point where none of the inequality constraints are active. The only way the point x can be optimal is if every feasible solution is optimal. For example, the point x = (2, 1, 2) is feasible for the LOP: min x IR 3 3x 1 + 8x 2 + 11x 3 subject to 3x 1 2x 2 + 4x 3 = 12 x 1 + 4x 2 + x 3 = 4 x j 0 For this problem, the objective function is a linear combination of the constraints. In particular, we have 3x 1 + 8x 2 + 11x 3 = 2 (3x 1 2x 2 + 4x 3 ) + 3 ( x 1 + 4x 2 + x 3 ) = 2(12) + 3(4) = 36 for any feasible point. So every feasible point is optimal. Mitchell Some Definitions 20 / 20