1 Review Session. 1.1 Lecture 2

Transcription

1 1 Review Session Note: The following lists give an overview of the material that was covered in the lectures and sections. Your TF will go through these lists. If anything is unclear or you have questions regarding that topic, interrupt your TF to clarify that issue. The more detailed explanations of some of the subjects have been chosen mostly arbitrary to highlight some aspects of the material of the first 10 lectures. It should NOT be understood as a hint what questions will be part of the midterms! Proficiency in all of the topics given below is required! 1.1 Lecture 2 What is a linear program A linear program (LP) is an optimization problem that involves maximizing or minimizing of a linear objective function by choosing values for decision variables subject to a finite number of linear equality and inequality constraints. Standard inequality form Matrix notation Graphical view optimality multiple optimal solutions unbounded objective infeasibility Algebra proof of optimality proof of infeasibility Putting things into canonical form maximization positive RHS equality constraints non-negative variables isolated variables Basic Feasible solutions An LP in canonical form has an associated BFS where the isolated decision variables are non-zero and all other variables are zero. 1

2 1.2 Graphical Interpretation of LPs Consider the following LP (cf. Task 2.1): maximize c 1 x 1 + c 2 x 2 subject to 3x 1 + 4x 2 5 x 1 + 2x 2 2 with different coefficients c 1 and c 2. Figs. 1 2 show the graphical interpretation of the four cases that can be encountered in a linear program. When the slope of the objective function is between the slopes of the constraints, the problem has an optimal solution (be careful with the orientation of the feasible region and the objective function!). Once the isoline of the objective functions are parallel with one of the constraints, multiple optimal solutions can usually be expected unless another constraint interferes. If the slope of the objective function lies outside of the constraints, the linear program becomes unbounded as shown in Fig. 2. The last case is an infeasible problem, where the feasible region is just an empty set (cf. Fig. 2). 4 4 y y x x Figure 1: Feasible region for c 1 = 1,c 2 = 3 (Left) and c 1 = 1,c 2 = 2. The first set of parameters leads to a unique solution, while the second set has multiple optima. This can be seen from the slope of the isoobjective lines. When it is parallel to one of the constraints, multiple optimal solutions exist. 1.3 Lecture 3 Modeling 2

3 4 4 y y x x Figure 2: Unbounded (left, c 1 = 1,c 2 = 3) and infeasible solution (right, additional constraint x 2 2. Using objective to push constraints Friendly Non-linearities Minimize the maximum Absolute value Positive and negative parts (SailCo) Specifying decision variables (Save-It) Ratio constraints 1.4 Handling Special Nonlinearities In the third lecture three special cases were shown, where nonlinear parts in the optimization problem can be replaced by a clever combination of linear constraints. The first case is the situation where we would like to minimize the maximum of an arbitrary subset I max I of the decision variables x i,i I (cf. Task 2.5.2). This can be achieved by first defining an additional decision variable y that is guaranteed to be larger than all of the decision variables in the subset by adding the following linear constraints: y x i, i I max So far y could be almost arbitrary as the constraints only impose a lower limit on it. By putting it into the objective function it will be forced down right to the smallest value still satisfying the constraints, which is exactly the maximum of x i,i I max. 3

4 Another common problem is a constraint, where some of the decision variables should be set into relation to other decision variables. An typical example is a balanced product portfolio, where in addition to the regular production constraints, the output of product x n should not exceed r 100 percent of the total production i I x i with I being the set of products (cf. Task 2.4, SAVE-IT example). Intuitively this constraint would become the following nonlinear constraint x n i I x i r. Fortunately, a simple multiplication resolves this nonlinearity x n r i I x i. Finally we will consider the case, where the absolute value of a decision variable appears in the constraint (examples include SailCo, where positive as well as negative change was accompanied by additional costs, and Task 2.5.5). Assuming that we have the following nonlinear objective: min x 1 with x 1 being a free variable then the following problem is equivalent: 1.5 Lecture 4 Convexity Polyhedron Hyperplane Halfspace Theorem: Halfspace is convex min x 2 x 2 x 1 x 2 x 1 x 1 free, x 2 0. Theorem: Intersection of convex sets is convex Theorem: Every polyhedron is a convex set Theorem: If c T x c T x x Adj(x ), then x is optimal Extreme point Let S be the set of points in the feasible region of an LP. A point y in S is called an extreme point of S if y cannot be written as y = λw + (1 λ)x for two distinct point w and x in S and 0 < λ < 1. That is, y does not lie on the line segment joining any two points of S. 4

5 Theorem: Optimality of Extreme Points Suppose P has at least 1 extreme point and there exists an optimal solution, then there exists an optimal extreme solution. Linearly independent Basis Span Columns of matrix A are linearly independent if the only solution of Ax = 0 is x = 0. A basis of R m is a set of linearly independent m-dimensional vectors A 1,...,A m with the property that every vector of R m can be written as a linear combination of the vectors A 1,...,A m. Note that the vectors A 1,...,A m form a square matrix that is invertible. These vectors A 1,...,A m span the vector space R m. Invertible == columns span == columns are LI == Ax = b has unique solution Basis of a matrix A basis B for an arbitrary m-by-n matrix A can also be seen as a list of m numbers chosen from {1, 2,..., n} such that the square matrix A B with m columns from A indexed by this list is a basis for R m, i.e. the column vectors span the space R m. Again, A B will be invertible. Basic solution The basic solution x of the system Ax = b is the unique solution of this system satisfying x j = 0 for all indices j B. Main LP Assumptions Matrix A has full row rank. Matrix A has less rows than columns. Columns of A span. Existence of a basis Extension rule Theorem: A matrix has a basis if and only if its columns span Basic feasible solution Basic solutions for LPs in canonical form are solutions for Ax = b but they might be infeasible if the non-negativity requirements for the decision variables are not satisfied. If we have a basic solution that is also feasible (i.e. all constraints satisfied), we have a basic feasible solution. Theorem: Extreme points of P are exactly BFS of LP. 5

6 1.6 Lecture 5 Tableau Theorem: if B is a basis for A, there is tableau corresponding to B Degeneracy definition At least one basic variable has value zero. Adjacent bases 2 bases for matrix A are adjacent if they share all but one basic column. Number of possible BFS Speed of Simplex algorithm (worst case vs. in practice) Simplex method Initialization (Phase I) Check optimality Choose entering index Check unboundedness Choose leaving index Pivot to a new tableau, go to check optimality step Optimality condition Reduced costs are all non-negative. Entering index (most negative reduced cost, smallest subscript, etc) Ratio test Leaving index Pivot Theorem: finite termination without degeneracy 1.7 Lecture 6 Phase I Artificial variables Find initial BFS Remove artificial variables Feasibility check Difference between slack and artifical variables 6

7 Slack variables transform inequalities into equalities. May take strictly positive or zero values at feasible solutions to original LP. Artificial variables are tools to find an initial feasible tableau. They must be zero at such a feasible solution. Degeneracy Degenerate basic solution Degenerate tableau When degeneracy occurs Cycling Smallest subscript rule Bland s theorem General termination Fundamental theorems of LP Thoerem: LP is optimal, infeasible, or bounded Lemma: If columns span, LP is feasible and not unbounded, then has optimal solution. Degeneracy in Phase I 1.8 Full Simplex Algorithm In the following the essential steps of the Simplex algorithm will be repeated. For brevity only the first pivot step of each iteration will be shown. Consider the following linear program (cf. Task ) Exercise 1 maximize 3x 1 + 3x 2 x 3 subject to 2x 1 + 2x 2 3x 3 3 7x 1 + 2x 2 2x 3 14 x 1,x 2 0;x 3 free Transform the LP problem to standard equality form. Then, formulate the initial Phase I tableau. End Exercise 1 7

8 We trust that you can take it from here :). See the solution to assignment 3 for the rest of the steps if you need a refresher. 1.9 Lecture 7 Optimality by multipliers Certificate of optimality Primal and Dual form Dual for standard equality form Weak duality theorem Primal Optimal Value Dual Optimal Value Corollary - If LP in standard equality form is unbounded then dual infeasible. Certificate of optimality Given primal feasible solution x and dual feasible solution y where the primal and dual objectives are equal, then x is optimal for the primal problem and y is optimal for the dual problem. Strong duality theorem (no gap) If an LP has an optimal solution then so does its dual and the two optimal values are equal. Lemma: optimal dual solution from primal Duality for general LPs Dual for standard inequality form (special case) 8

9 Auction example (dual interpretation) Summary of duality table 1.10 Lecture 8 Sensitivity Analysis - robustness of solution Considering changes graphically Shadow price Shadow price == dual value == reduced cost on associated slack variable. Generating AMPL sensitivity output Generating a final tableau from a basis Kinds of changes Changing objective coefficient of non-basic variable-check reduced cost on changed coefficient Changing objective coefficient of basic variable-check reduced cost on every nonbasic variable Changing RHS-check b 0 Changing entries in column of nonbasic variable-check reduced cost of affected nonbasic variable Introducing a new activity-price our activity, check reduced cost Summary: Sensitivity Analysis Changing multiple parameters at once Interpretation of Shadow prices 1.11 Lecture 9 Sensitivity Analysis Questions Complementary Slackness Theorem: need corresponding constraint to be tight whenever variable is strictly positive. Proof Checking optimality Other uses of complementary slackness 9

10 1.12 Lecture 10 Naming convention (for understanding dual simplex) What is a Dictionary? Comparing Primal and Dual Simplex Formalizing Dual Simplex Definition Dual feasible Assume have dual feasible tableau Step 1: Pick basic variable, x r, to leave with strictly negative RHS Step 2: Pick nonbasic variable, x k to enter by considering row r and strictly non-negative (nonbasic) entries. Ratio test Step 3: Pivot on (r,k) and go to step 1 Continue until have dual optimal tableau Why use dual simplex? Adding a new constraint 2 Solutions 10

11 Solution 1 We first introduce variables u 0 and v 0 to replace the free variable x 3 : maximize 3x 1 + 3x 2 (u v) subject to 2x 1 + 2x 2 3(u v) 3 7x 1 + 2x 2 2(u v) 14 x 1,x 2,u,v 0 We can then introduce objective variable z and slack variables s 1 and s 2 to convert to equality form: z + 3x 1 3x 2 + u v = 0 2x 1 + 2x 2 3u + 3v s 1 = 3 7x 1 + 2x 2 2u + 2v + s 2 = 14 x 1,x 2,u,v,s 1,s 2 0 To find an initial feasible tableau, we add an artifical variable t, and aim to solve the following problem: minimize subject to 2x 1 + 2x 2 3u + 3v s 1 + t = 3 t 7x 1 + 2x 2 2u + 2v + s 2 = 14 x 1,x 2,u,v,s 1,s 2,t 0 We only need one artificial variable because we can use the slack variable s 2 as a member of the initial basis. We multiple the objective by -1 to convert the problem into a maximization problem (using a variable w to capture the objective value): x 1 x 2 u v s 1 s 2 t w t s We can then add Equation 2 from Equation 1 to remove t from the objective: x 1 x 2 u v s 1 s 2 t w t s End Solution 1 11