APPLIED OPTIMIZATION

Transcription

1 Systems Analysis Group, 1995 Uppsala University APPLIED OPTIMIZATION optimal point, f = 12 Starting solution: f=o 10 Thomas Persson January 1995 Revised in October 1998, Håkan Lanshammar

2

3 Contents 1 INTRODUCTION... l 1.1 Examples of optimization problems General formulation, mathematical program LINEAR PROGRANIMING Examples of typical problems solved by linear programming Efficient use of limited resources The alloy problem General forms of the LP problem Graphical solution to a simple LP problem Standard form of an LP problem Basic feasible solution THE SIMPLEX METHOD The idea of the simplex method Setting up a starting solution The simplex method in tabular form Other mode1 forms Multiple optima, degeneracy and cycling...i SENSITMTY ANALYSIS AND DUALITY The simplex tableau in matrix form Sensitivity analysis The dual simplex method Duality Duality and the simplex method Duality and sensitivity analysis... 30

4 5 INTEGER LINEAR PROGRCWINIING General form Some considerations on solving integer linear programming problems The branch-and-bound technique for solving ILP problems DYNAMIC PROGRAMMING NONLINEAR PROGRCWINIING Analytical conditions for minimum Some considerations on numerical algorithms Minimization of one-dimensional functions Minimization of multi-dimensional functions REFERENCES

5 1 Introduction Systems analysis is a scientific method for describing, analyzing and controlling complex systems. There are different techniques used in systems analysis. Some of them are simulation, optimization and operations research. Optimization techniques can be viewed as being a part of operations research, or as a branch of mathematics. This material is intended to be directly applicable for solving different types of optimization problems, and to cover the optimization part of the course Simulation and Operations Research given in the Master of Science program at Uppsala University. The material gives an overview of different types of optimization problems, with the emphasis on linear programming problems. The problem types are presented both from a theoretical and an applicative point of view. The topics of the course material are linear programming (LP), integer linear programming (ILP), dynamic programming (DP) and nonlinear programming (NLP). 1.1 Examples of optimization problems In this section we will give some examples of optimization problems. Ex l.l)cancan, a can manufacturing company, wants to construct a can with the largest possible volume from a specified amount (area) of sheet. We know that the area, A, and the volume, V, of a can is defined by: A A=2nr2 +2nrh V=nr2h h The problem therefore becomes v maximize V =nr h 2nr -I- 2nrh = a (a is the area) subject to r,h>o

6 Ex 1.2) Cancan decides to manufacture two different types of cans. Two different machines have to be used in the process. Each can has its specific material cost and sale price. All data is summarized in the following table: Can 1 Can 2 time in machine s 1.0 s time in machine s 1.5 s material cost 0.20 : :- sale price 0.40 : :- How many of each type of can should Cancan produce per hour in order to maximize their profit? To formulate the problem we introduce the following variables: This gives the problem x, : the number of Can 1 produced per hour x, : the number of Can 2 produced per hour f = f (x,, x,) : the profit per hour when producing and maximize f = 0.20~~ ~~ selling x, and x, cans, respectively 2x, + 1.0~~ ~~ + 1.5~~ X,,%, General formulation, mathematical program A general optimization problem can be written in the form l max f (x> subject to x E F where X= (x1 X, x ~ is ) a column ~ vector of n decision variables, and F is the feasible set. f is called the objective function or the cost function. The problem can be formulated in a more "mathematical" form:

7 This is called a mathematical program. (The term mathematical program should not be compared to computer program, but rather to mathematical model). The feasible set F in (1.1) has in (1.2) been divided into two parts: the constraints g, (x) $ O and hj (x) = 0, and the feasible set X. The feasible set X can be nonnegative numbers, integers, binary numbers, etc. Note: A constraint with "2" instead of j'<" constraint 2x1-3x O is equal to -2x1 + 3x2-6 <O. can easily be transformed, e.g. the There are some important special cases of mathematical programs. Linear programming, LP - f, g, h are linear - X is the set of nonnegative numbers Integer linear programming, ILP - f, g, h are linear - X is the set of nonnegative integers. If X={O,l} then the problem is called binary programming or zero-one programming. Nonlinear programming, NLP - f andlor g, h are nonlinear LP is described in chapter 2-4, ILP in chapter 5 and a short introduction to NLP is given in chapter 7.

8 2 Linear programming 2.1 Examples of typical problems solved by linear programming Efficient use of limited resources Consider m resources of some sort, e.g. raw material, work force, machines, financial resources, time, which are available in limited amounts denoted by bi. These resources can be used for performing n activities (production processes). Let j be one of these activities and let xj be the (unknown) leve1 of the activity j. If the production process consists of producing a specified product, then xj represents the produced quantity. We denote the quantity of the resource i (i = 1, m) necessary to produce a unit of the product j (j = 1, n) by aij. The total quantity of the resource i used for producing the n products is then determined by: Because xj represents the quantity that must be produced, it cannot be a negative number: The decision-making process implies the consideration of an economic criterion, e.g. to maximize the profit. Let pj - the unit sale price of product j and cj - the unit production cost for product j. Then the total profit is given by: The problem is to find those solutions of the inequation system (2.1) and (2.2) which maximize the profit (2.3).

9 Ex 2,l)Consider a retired joiner who makes two different types of furniture. Each piece of furniture is manufactured using three types of material. Furniture of type 1 requires 1.0 units of material 1, 2.5 units of material 2 and 1.5 units of material 3. Furniture of type 2 requires 1.5 units of material 1, 2.5 units of material 2 and 2.5 units of material 3. The joiner only have 90, 120, and 100 units of the these three materials, respectively, so he must be careful when he decides how many of each type of furniture he is going to manufacture. The material costs for the two types of furniture are 100 US$ and 90 US$, respectively. The joiner can sell furniture of type 1 for 145 US$, and furniture of type 2 for 140 US$. Assume that there is no limit of how many pieces of furniture he can sell and that he wants to maximize the profit. Introduce the following variables: x, : the quantity produced of furnitures of type 1 x, : the quantity produced of furnitures of type 2 We can now formulate the problem as max f = &x, + 50x2 ( 1.0~~ + 15x ~~ + 2.5~~ I 120 subject to 1.5~~ + 2.5~~ I 100 I x,, x, 2 o (The solution is given by x, = 20, x, = 28, f,,, = 2300 ) The alloy problem In a laboratory of a metallurgical company an alloy is produced using n raw materials Mj (j= l, n). Each raw material Mj contains the same m different substances Si (i=l, m). The contents per unit of substance Si in raw material Mj is a,. The alloy has to contain at least Li (i=l, m) units of substance Si, and at most ui (i= l, m) units of the substance Si. The unit cost of the raw material Mj is Cj. The problem is to determine the quantities of each raw material Mj in the alloy, so that its cost is minimized. The problem can be formulated as

10 n max f(x)= CC,M, j= l subject to li <CauMj <ui, i=l, m j=i Mj20, j=l,n 2.2 General forms of the LP problem A general LP problem with n variables and m constraints can always be formulated as (see the general mathematical program in (1.2)) max f =clxl +c2x c,x, allxl + a12x2 +...S a21x1 alnxn <bl + a2,x a,,x, 5 b, allxl+ a12x2 subject to a,x, <bl al+l,lxl +a1+1,2'2 +.q.+al+i,inxn =b~+i amlxl + a,,,x, a,,x, =b, X1,X2,..., X, ro Note: A "2 - inequality" can be transformed into a "2 - inequality" by multiplying both sides of the inequality by -1. For example, the inequality x, -x2 +2x3 2-4 is equivalent to -xl+x2-2x3 54. We ca11 the form (2.5) the general form of a linear programming problem. It is common, though, in real problems to have only inequality constraints, i.e. max f = clxl + c2x C,X, allxl + a12x aln%, 5 bl a2,x1 + a2,x a2,xn <b, subject to < am1'1 +a1n,x a1nnxn X1,X2,..., X, ro We ca11 the form (2.6) the canonical form of a linear programming problem

11 The LP problem (2.6) in canonical form can be formulated in matrix form: (2.7) where the vectors x, b and c, and the matrix A are defined as 2.3 Graphical solution to a simple LP problem Consider this simple LP problem: max f = 2x, +3x2 x,,x, 20 The problem can easily be solved graphically, see Fig lx2 7 feasible region Figure 2.1. Graphical solution of an LP problem The dotted line represents the objective function f and is drawn for the case when it has the value of 6. When maximizing f, we move the dotted line in the direction of its gradient, so that the value of f grows. We see that the optimal point is the

12 intersection between the two lines x, + 3x2 = 9 and x, +x2 = 5. The solution to the problem is therefore given by X, =3 x, =2 f,,, = 12 Some observations can be made from this example: The solution is never inside the feasible region but always at the boundary. We can either have no solution, one solution, or infinite number of solutions. - No solution. This is the case when the feasible set is empty - One solution. The solution is then in a corner, i.e. in an intersection between two or more constraints. - Infinite number of solutions. This can occur when the objective function is parallel to a constraint. 2.4 Standard form of an LP problem For algorithms solving LP problems, an LP problem is usually formulated in the standard form: subject to Notice the two differences from (2.7): The equality constraint, and the demand that all coefficients in the b-vector are nonnegative. Any LP problem can be represented in the standard form using the following transformations: T1) A "< - inequality" is transformed into an equality by adding a nonnegative slack variable to the left hand side of the inequation. For example, x,+x2 +2x3 14 x,+x2 +2x, +x4 =4, x, 20, where x, is a slack variable. T2) A "2 - inequality" is transformed into an equality by adding a nonnegative surplus variable to the left hand side of the inequation. For example, x,+x2+2x324 --> x,+x2+2x3-x,=4, x,>o, where x, is a surplus variable.

13 T3) Any non-positive variable xj, xj l O is transformed into a nonnegative variable by the substitution x; = - xj, x; >O. T4) A non-restricted variable xj, - l xj 5 w can be replaced by two non- f I, negative variables x; and x, as follows: x j = x; - x,, where x j, x j > O T5) Any minimization problem can be transformed into a maximization problem and vice-versa, by changing the signs of the coefficients in the objective function: min f (x) = - max[-f (x)]. For example, xep xaf minimizing fl = 5x1 + 7x2 is mathematically equivalent to maximizing f, = - fl = - 5x1-7x, in that the optimal values of x, and x, will be the same. The only difference is that the objective functions fl and f, will have opposite signs: min fl = - max f2 = - max [-f,] Ex 2.2)The problem max f = 50x1 + 40x, 2x1 +3x, l 9 x, >0,x2 I0 formulated in standard form is max f = 50x1 + 40x, 2x1-3%; + X, =9 -x4 =l2 2.5 Basic feasible solution PI Consider an LP problem in standard form: subject to

14 A is an mx n matrix, i.e. there are m constraints and n variables. In real situation problems we have more variables than constraints, so m c n. To satis@ the m constraints in the equation system Ax = b we only need m variables. The remaining n - m variables can be set equal to zero. In fact, the following statement holds: This corresponds to the formation of an m x m submatrix B of A. Provided that B is non-singular, the values of the remaining variables can be found, as there are now m equations with m unknown variables. Such a solution is called a basic solution and the m variables are called basic variables. A basic solution which is feasible (Le. all variables are nonnegative) is called a basic feasible solution. Let a j,(j= l, n) be a column vector corresponding to the constraint coefficients of x j. Then the constraint set Ax = b can be written The vectors a,, a,,,..,an are all m-vectors and they belong to a vector space of m dimensions. Any set of m vectors which are linearly independent and span the space form a basis. Any other of the n - m remaining vectors in the vector space can be expressed as a linear combination of these basic vectors. The submatrix B which corresponds to the basic vectors is called the basis matrix. We introduce the following notation: x, : column vector of the m basic variables x, : column vector of the m - n nonbasic variables equal to zero B N : m x m submatrix of A, called the basis matrix : (n- m) x m submatrix of A c, : column vector (size m) of the coefficients to the basic variables x, in the objective function c, : column vector (size m - n) of the coefficients to the nonbasic variables x, in the objective function If we know which variables that are zero, we can easily solve for x, and f! This can be seen for x, with the following computation:

15 The objective function f can be written To summarize, we have The problem is thus to determine which variables should be basic (i.e. 2 0) and which should be nonbasic (i.e. = 0).

16 3 The simplex method 3.1 The idea of the simplex method In chapter 2 it was found that if a linear programming problem has a finite optimal solution, then it has at least one optimal solution at an extreme point (a vertex) of the feasible region. The general idea of the simplex method, is to move from one vertex of the feasible region to an adjacent vertex which has a better value of the objective function, until no further improvement in the objective can be obtained. At this point, the optimal solution has been found. Ex 3.l)The simplex method applied to the example problem given in chapter 2.3 will follow this path: Iteration 2: optimal point, f = 12 Starting solution: f=o A family of LP algorithms, known as interior point methods has been developed starting in the 1980's, that can be faster for many (but so far not all) problems. These methods are characterized by constructing a sequence of trial solutions that go through the interior of the solution space, in contrast to the simplex method which stays on the boundary and examines only the corners (vertices). The vertices of the feasible region correspond to basic feasible solutions of the constraint equations associated with particular basis for the column space of the constraints. Mathematically, the simplex method moves from one vertex of the feasible region to an adjacent vertex by moving from one basis of the column space to an 'adjacent' basis. An 'adjacent' basis has one element changed: the new basis has one of the old basic vectors removed and replaced by another vector which was not part of the original basis. The vector to enter the basis is selected so that the value of the objective function will be improved at the new basic feasible solution. The vector to leave the basis is chosen to ensure that the new solution will be feasible.

17 3.2 Setting up a starting solution Consider the linear program subject to Observe the condition that b 2 O! By adding nonnegative slack variables sl, S,,..., S,, one to each constraint, we get the problem in standard form and the constraints become where I, is the m X m identity matrix and s is a column vector of the slack variables. The vectors corresponding to the m slack variables are linearly independent, and they span the space. Thus s form a basis for the space. Expressing b in terms of this basis, the basic solution is: Since b 2 0, this solution is also feasible, and hence it is a basic feasible solution. The original decision variables x all have the value zero at this solution, since the corresponding column vectors a,, a,,..., a, are not in the basis. Referring to the notation introduced in chapter 2.5, the matrix B, the basis rnatrix, is in this case I,. The basic vector x, is s, and the nonbasic vector x, is x. We can reformulate the original problem (3.1) to standard form using slack variables as follows: max f=ctx Ax =b subject to #ro (b2o) where A is an appended matrix: A= (A I,) and X is an appended vector:

18 Then a basic solution of (3.2) is x, =x = O and x, =s = b. This basic solution represents the starting solution of the simplex method. We notice that a basic solution contains at most m positive (non-zero) values, and that the remaining values are equal to zero. We also notice that in the starting solution f = 0, since the slack variables have coefficients of zero in the objective function. If the LP problem satisfies the following conditions: 1. it is a maximization problem, 2. it is in standard form (with slack variables added) and has an initial basic feasible solution x,, 3. the slack variables have coefficients of zero in the objective function, then we can apply the simplex method. 3.3 The simplex method in tabular form To keep track of the coefficients and to organize the computations, the calculations can be performed in the form of a simplex tableau. Basicvariable f S1 S2 f xl x2 a... Coefficient of..... Xn SI Sa S,,... -C, -C, -cj -'n O O O a11 a21 a12 a22. * %j aln 1 O a2 j a2n O 1 m.. a.. m.. s.. s.. %n O am1 am2 amj. ann O O *.S Table 3.1. The starting solution in the simplex tableau... s ' O s s O 1 Right side O bl b2 b, The tableau (see Table 3.1) has one column for each of the problem variables (decision and slack): xl,x,,..., xn,sl,s,,..., s,, and one column for the current basic feasible solution (initially equal to the right-hand side vector of the constraints). For completeness we also have a column for the objective function f, since we can consider f to be a permanent basic variable. The top row is the objective row with the objective coefficients with reversed sign, and the current value of f on the right-hand side. We will ca11 this row the "f-row". The other rows are just the constraint coefficients. The simplex method is iterative. It starts with the starting solution seen in Table 3.1. This tableau is called the initial tableau. During the iterations we will create new tableaux until we reach the final tableau, which gives us the optimal solution to our problem. Basically the method works at follows:

19 Step O: Formulate the problem in standard form with nonnegative right-hand sides. This gives us the starting basic feasible solution. Step 1: Select an entering variable from the current nonbasic variables in such a way that when the entering variable increases the objective function f increases. Step 2: Select the leaving variable from the current basic variables so that the new basic solution will be feasible. Step 3: Determine the values of the (new) basic variables. Step 4: Check for optimality. If the solution is not optimal, go to step 1. The steps 1-4 are done according to this: Step l: Entry criterion. Select the nonbasic variable xw with the largest negative coefficient in the current "f-row" to enter the basis. The objective function f will then, when xnj increases, increase at the fastest rate. The column corresponding to xw is marked in the simplex tableau. Step 2: Exit criterion. Select the basic variable xbi that reaches zero first when the entering variable is increased. Let tji = min is the current coefficient of the entering variable xw in constraint i, and b, is the current right-hand side in constraint i. The i which gives us the minimum ratio tji determines which basic variable to leave the basis. This row is also marked in the simplex tableau. Step 3: Pivoting. This is an adjustment process by which the column vector corresponding to xnj in the tableau becomes an identity column, with a coefficient of 1 in row i, zero elsewhere. This is achieved with row operations. Row i is the pivot row and äg is the pivot element. The process is the same as the one used in solving linear equations, i.e. Gaussian elimination. Step 4: Optimality test. If all the coefficients of the nonbasic variables in the "f-row" are nonnegative then the solution is optimal. If not, go to step 1.

20 The best way to illustrate the simplex method is by an example: Ex 3.2) The example problem given in chapter 2.3 max f = 2x1 + 3x2 is in standard form max f = 2x1 + 3x2 x, + x, +x3 =5 x, + 3x2 + x, = 9 X1>X2>X,>x4 ro where x,, x, are slack variables. The simplex method will yield these iterations: Iteration o 1 Basic variable f x3 x4 f x3 x2 f P- Coefficient of f x1 x2 x3 x4 1-2 o 1 o 1 O 1 o o o 1 o o 1 o o 1 o /3 1 o o o Righthand side 1 O In the initial tableau, iteration 0, we see that the starting solution is This solution is feasible but not optimal. In iteration 1 we have that

21 In the tableau we see that we still have negative coefficients of the nonbasic variables in the "f-row" (the coefficient of x, is -1). The solution in iteration 1 is therefore not optimal either. In iteration 2 we have the solution Now there are no negative coefficients of the nonbasic variables in the "f-row", and the solution is optimal. Since all variables are nonnegative, the solution is also feasible. In iteration 2 we thus have the final tableau. Note 1: If we rewrite the "f-row" for iteration 1 we get If we increase x, with 1 unit, f will increase with 1 since 6, the current coefficient of x,, is 1. The coefficients of the nonbasic variables in the "f-row" are usually called reduced costs. The current coefficients of the basic variables are always zero. Note 2: If we rewrite the "f-row" for the final tableau we get It is obvious that the maximum value for f is 12 for x, =x4 =O. Note 3: Compare the solution in iteration 0, 1 and 2 with the path given in example Other mode1 forms Thus far we have described how to set up the starting solution, i.e. the initial tableau, when we have all constraints in I form and all bi 20. When we have constraints in 2 form and the corresponding bi TO, or when some constraints are in = form it is not straightforward how to find an initial basic feasible solution. The standard approach is to use artificial variables for the just mentioned cases. An artificial variable is a dummy variable introduced in each constraint that

22 needs one, only for the purpose of being the initial basic variable for these constraints. As for all variables, a nonnegative constraint is placed on artificial variables. Since the value of an artificial variable must be zero in the real problem's solution, a penalty is introduced in the objective function, forcing the artificial variable to zero. The iterations of the simplex method then automatically make the artificial variables disappear, i.e. become zero, one at a time. Then the real problem is solved. The penalty in the objective function is done using the Big M method. We introduce a huge, unspecified positive number M as the coefficient of the artificial variables in the objective function. For example, if the objective function is f =2x1 +3x2 and ii?, is an artificial variable, then the objective function is modified to f =2x1 +3x2 -q Because of this penalty we thus introduce artificial variables in the objective function, which makes the columns corresponding to artificial variables not being identity columns. Therefore we can't use the artificial variables directly as starting basic variables. But as we will see in example 3.3 we can solve this with some pre-processing before we put the problem into the initial simplex tableau. 2 inequalitv constraints If constraint i is in the form 2 and bi 20, then we will add a surplus variable to get an equality constraint. For example, the constraint x, +3x2 28 will be transformed to x1 +3x2 -x, =8, x, 20 where x, is a surplus variable. But since a surplus variable comes with a negative sign, it can't be used as an initial basic variable since it will not be feasible (x, =- 8 in this case). Therefore we must add an artificial variable to the constraint. In this case the constraint will be x1 +3x2 -x3 +K =8, x3,x4 20 where F4 is an artificial variable. To ensure that F, is forced to zero, we must also modify the objective function. Equality constraints If constraint i is in the form = and bi 20, then we already have that constraint in standard form. But probably it's not easy to find an initial basic variable for that constraint, and therefore we add an artificial variable to the constraint. Of course we must also modify the objective function to force the artificial variable to zero.

23 Ex 3.3) Consider the problem (P) and (P) transformed to standard form: (p) (P) in standard form max f = 2x, +x2 max f =2xl+x2 -Mx, x,+ x2r6 =6 x,,%2 ro x1,x2,x3>z4,x5 +x5=8 where x, is a surplus variable, Z, is an artificial variable and x, is a slack variable. If we put the problem in a simplex tableau we see that we need to do one iteration to make E, a basic variable before we can use the simplex method. Basic variable Coefficient of - f %l %z %Q x4 '6 I Righthand side Now we can set up the initial tableau and apply the simplex method. Iteration O Basic variable f I Coefficient of - f xl x2 X3 %q '6 1-1-M M O O Righthand side -6M d l2 1 The solution to the problem is, according to the final tableau, f,, = 16 for x, = 8, x, = O. The slack variables are x, = 2, x, = O.

24 Note: When an artificial variable has left the basis it will never enter the basis again, so it will remain zero. Therefore it is not necessary to calculate a column corresponding to an artificial variable after it has left the basis. 3.5 Multiple optima, degeneracy and cycling Multiple optima. If a coefficient for a non-basic variable in the "f-row" is zero in an optimal simplex tableau, the variable could be increased and brought into the basis, with no effect on the objective value. A pivot operation would yield a new tableau with a different basis, but the same value of the objective function f: The indifference lines of the objective lie parallel to an edge of the feasible region, and there are multiple optimal solutions, at the vertices at either end of the edge. The pivot moves the solution from one such vertex to the other. Because the move is along the indifference line, f does not change. In general, if x* and #* are both optimal solutions, then so is any linear combination of the form hx* + (1 - h)#* with O 5 h I 1. However, only the vertex points are basic solutions. Degeneracy and cycling. If one or more variables in the basis x, are zero, the solution is degenerate. This can happen when it is ambiguous which of two (or more) basic variables are to be the leaving variable, i.e. the minimum ratio is the same for both variables. Then, in the next iteration, the variable not chosen to be the leaving variable will be zero since the two variables reach zero at the same time. This situation is common in practical problems. The effect of degeneracy is that the objective function will not increase in the next iteration since b; =O and therefore ai = min =O. If degeneracy does not occur, the objective function will increase in each iteration with the quantity Ej Oj, where Fj is the current coefficient of the entering variable and 6j is the minimum ratio for the leaving variable. Since 5 z0 in a non optimal solution, we will always have that Fj 6 j >O. Degeneracy can imply that the simplex procedure would repeat the same sequence of iterations without improving the objective function. Therefore the method would not converge to an optimal solution. This phenomenon is called cycling, and is mainly of theoretical interest. It is very seldom that cycling occurs in practice.

25 4 Sensitivity analysis and duality 4.1 The simplex tableau in matrix form An LP-problem in canonical form, i.e. max f =ctx Axlb subject to { x20 (b>o) can be transformed into the standard form max f=ctx Ax+I,x, =b subject to x,x, 2.0 (b2.0) where Im is the m x m identity matrix and x, is a column vector of the slack variables. This is basically the same standard form as presented in (3.2). We can put (4.1) directly into an initial simplex tableau in matrix form, since we can use the slack variables as basic variables: Basic variables f x, Table 4.1. The initial simplex tableau in matrix form. In chapter 2.5 we derived that f =csb and x, =B-'b Coefficient of f X X, 1 -C T o o A Im Right-hand side o b If we know which variables are basic in any iteration we can from the standard formulation of a problem get B and c;. From these we can thus calculate the value of f and x,. The simplex tableau at any iteration therefore is Basic variables f XB Coefficient of f X X, 1 c;b-~a-c~ c;b-~ O B-IA B-I Right-hand side c;~-lb ~-lb Table 4.2. The simplex tableau in matrix form at any iteration

26 We get this tableau by using the initial tableau and adding the second row multiplied from the left with c;bml to the first row, and then multiplying the second row from the left with Bml. The same reasoning can be done for problems with surplus and artificial problems. Ex 4.1) Consider the problem max f = 2x1 +x2 subject to x, +2x, 18 2x1- x,<4 Let's assume that we know that the basic variables in the optimal solution are We can now determine the solution to the problem (or the final tableau if we would like). The problem formulated in standard form is max f = 2x, +x2 subject to x1+2x2 +x, =8 2x1 - x, +x5 =4 where x,, x,, x, are slack variables. We identify B and c; as Now it is easy to calculate x, and f:

27 Hence the solution to the problem is f,, = 8 for x, = 3, x, = 2. The slack variables are x, = O, x, = 1, x, =O. 4.2 Sensitivity analysis Often it is of interest to see how a change in one or more parameters in an LP problem influences the optimal solution. It can for example be interesting to examine how a change in a resource bi affects the solution. Instead of applying the simplex method all over again to the revised problem, it is possible to use the solution (final tableau) of the original problem. This is done using the matrix form of the final simplex tableau as was described in chapter 4.1. Ex 4.2) The problem max f = 10xl + 4x, 13x1 +x, 130 subject to 2x1 + X, 125 x1,x2 ro has the final tableau Basic variables f x~ x1 Coefficient of f x1 x2 x3 x4 1 o o 2 2 O O o 1 o 1-1 Right- hand side where x,, x, are slack variables to the constraints, respectively, Assume that we want to change the problem to maxf =llxl+4x, 13x1 +x, 135 subject to 2x, + x, 125 %,,X2 ro What is the new final tableau? We know that the simplex tableau for the problem

28 max f =ctx max f =ctx subject to {%:o u Ax+I,x, =b subject to x, x, ro at any iteration is (Table 4.2) Basic variables f X B Coefficient of f X X, 1 c~b-~a-c~ c;b-~ O BmlA Bml After the changes we have that c; = (4 11) and b = (2;). To carry out the calculations we also need Bml. We can either set up the standard form of the problem and get B from there, or we can identify B-I from the final tableau since B has not changed. We choose to identify B-I from the final -2 3 tableau: B-l = ( [lo). This gives us that B1b =, c;bdlb= 130 and CEB-' =(3 1). The rest of the tableau does not change. x, and x, are basic variables, so c;b-'a- c' =(O O) as before. (Control that!) 3 The new final tableau is Right-hand side c;~-lb Bqlb Basic variable f x2 x1 Coefficient of f x1 x2 x3 x4 1 O O 3 1 o o o 1 O 1-1 Righthand side Since we have only positive coefficients in the "f-row" and the basic variables are nonnegative, the solution is optimal and feasible. We can thus be sure that this really is the final tableau. Note 1: If the revised final tableau becomes nonoptimal, i.e. it does not pass the optimality test because there are negative coefficients in the "f-row", apply the simplex method to the revised tableau until optimality is reached. Note 2: If the revised final tableau becomes infeasible, i.e. there are negative basic variables, apply the dual simplex method to the tableau until feasibility is reached. The dual simplex method is described below.

29 Sensitivity analysis can also be done using parametric programming in which we introduce some parameters 0,, 0,)... into the problem. It can then be of interest to know for which values of these parameters the former basic solution still is optimal (i.e. the new optimal solution has the same basic variables as the former), and what the new solution is. Ex 4.3) The linear programming problem (P) maxf =-5xl+5x2 +13x, has the optimal final tableau -x,+ x, + 3x, x1 + 4 x, + 10x3 < 90 X3 20 Basic variables f x2 Xfi Coefficient of f x1 x2 x3 x4 X6 1 o o 2 5 o o o O 16 O Right- hand side [~~-~). Assume that the right-hand side of the constraints to (P) is changed to For which 0 is the basic solution x. = ) still feasible? We have a feasible solution if x, = B-'b 2 0. In this case we have that 1 O O B = [ ) [ ) = [ ). 3 Peasible solution if The dual simplex method The simplex method starts with a feasible but not optimal solution, a so called suboptimal solution, and moves towards an optimal solution by striving to satisfy the optimality test. The dual simplex method, on the other hand, deals with an optimal but infeasible solution, a so called superoptimal solution, and strives to achieve feasibility. We will use this property in sensitivity analysis. Suppose that an optimal solution has been obtained by the simplex method, and it becomes of interest to make some minor changes in the problem. We then calculate the solution to the revised problem using the method described in chapter 4.2. If the former basic solution no longer is feasible, but the optimality

30 We have that B-'b= [ :)[:3=(28) and o:b-'b=(6 0) test still is satisfied, we can immediately apply the dual simplex method to this superoptimal solution. The dual simplex method is described in these five steps: Step O: Start with a superoptimal solution. Step 1: Exit criterion. Select the basic variable xbi with the most negative value to leave the basis. The row corresponding to xbi is marked in the simplex tableau. Step 2: Entry criterion. Select the nonbasic variable xw whose current coefficient Fj in the "f-row" will reach zero first when the leaving variable is increased towards zero. Let o j = { min I ä, <O}. Where ä, is the current coefficient of the leaving variable xbi in column j. The j which gives us the minimum ratio oj determines which nonbasic variable to enter the basis. This column is marked in the simplex tableau. Step 3: Pivoting. This is the same adjustment process as in the original simplex method by where the column vector corresponding to xw in the tableau becomes an identity column, with a coefficient of 1 in row i, zero elsewhere. This is achieved with row operations. Ex 4.4) Consider the LP problem given in example 4.3, Determine the solution to the problem when 8 = 2! We notice that x, = - 8 < 0 so the solution is infeasible. The change in b does not affect any of the coefficients in the "f-row" (see Table 4.2). Thus the solution still passes the optimality test, and we have a superoptimal solution! Use the dual simplex method.

31 Iter. o initial 1 final Basic variables f x 2 f X 2 Coefficient of f %I %z x, X4 '6 1 o o o -1 1 O 1 16 O 1 16 O O 1 1 O 23 1 O O -8 O o 1 o -4 1 Righthand side The solution to the problem when 0 = 2 is thus f,,, = 112 for x, = O, x, = 12, x, = 4. The slack variables are x, =x, =O. 4.4 Duality Each LP-problem has a dual problem. The original problem is called the primal problem. A problem (P) in canonical form, has the dual problem (D) min g = bty subject to ATY 2 C where y is a column vector with m elements. This means that a primal problem with the variables x,, x,,..., xn is associated with a problem with the variables y,, y,,..., y,, where n is the number of variables and m is the number of constraints in the primal problem. A problem where constraint i is an equality constraint will have the condition > O deleted, i.e. yi is unconstrained. Yi -

32 Ex 4.5) The primal problem max f = 4x1 + x, subject to I x, + 2x2 I 8 2x1 - x, = 4 x,, x, 2 o has the dual problem min g = 5y1 + By2 + 4y3 Yl+ y2 4-2~3 24 y, + 2y2 - Y, 2 1 Y,, Y, 2 0, y3 unconstrained Each constraint in the primal problem is thus associated with a variable in the dual problem. Note that if the problem (D) in (4.3) is our original problem, then its dual is the problem (P) in (4.2). There are some important properties of the primal-dual relationship which are described below, Weak duality: If x is a feasible solution for the primal problem and y is a feasible solution to the dual problem, then Proof: With the conditions given in (4.2) and (4.3) we have that ctx I (A'Y)~X = ytax 2 ytb = bty Strong duality: If x* is an optimal solution to the primal problem, then there exists an optimal solution y* to the dual problem, where Duality can be useful in such cases when it is easier to solve the dual problem than the primal problem. For example, if there are more constraints than variables, i.e. m > n, we have to introduce fewer slack and surplus variables if we solve the dual problem and the computing effort is less. Also it is more convenient to choose the form which has constraints of I-type since we then will have surplus variables instead of slack variables, which often requires the use of artificial variables. Duality can also be used in sensitivity analysis which will be described in chapter

33 4.4.1 Duality and the simplex method A useful property of the simplex method is that it not only gives the optimal solution to the primal problem, but also to the dual problem. The solution to the dual problem is found in the "f-row" as the coefficients of the slack and surplus variables. At each iteration the simplex method simultaneously gives a feasible solution x for the primal problem and a solution y for the dual problem, where ctx= bty. If X is not optimal then y is not feasible. At the final iteration the simplex method gives an optimal solution x* to the primal problem, and an optimal solution y* for the dual problem, where ctx* = bty*. The relationships between the complementary basic solutions are given in Table 4.3, and a summary of the classification of the basic solutions in Table 4.4. Primal basic solution Suboptimal Optimal Superoptimal Infeasible and non-optimal Dual basic solution I Superoptimal l Optimal Suboptimal Infeasible and non-optimal Table 4.A. Relationships between complementary basic solutions. Feasible? Yes No Satisfies the optimality test? Yes Optimal Superoptimal No Suboptimal Infeasible and non-optimal Table 4.B. Classification of basic solutions. We know that for the simplex method ctx = bty at any iteration, and that we can express the objective function f as f = ctx = CBX, = c;~-'b. An identification of terms then gives us that yt = c:b-'. Hence, if we know c: and B*' at the primal problem's optimal solution we can solve for the dual problem's optimal solution using this expression: If we recall Table 4.2, we see that the term c;bql is found in the "f-row" as the coefficients for the slack variables. Therefore, when using the simplex method it is always possible to identify the solution for the dual problem. Ex 4.6) Solve the following problem (P)! min f = 10xl + 6x, + 8x3 x,+ x, +2x3 22 5x1 + 3x, + 2x

34 In this case it is easier to solve the dual problem to (P). If we solve (P) directly we have to convert min f to max g, and use surplus variables and therefore also artificial variables. The dual problem (D) to (P) is max g = 2y1 + y, subject to ~1+5~2510 Y,+~Y, 56 2y1 + 2y258 Applying the simplex method to (D) gives the following tableau: Iter. o initial Basic variables g Y3 y, Coefficient of g Yl Yz Y3 Y4 y6 1 o O o o o 5 1 O o 3 O 1 O I Righthand side 1 final y6 g Y 3 Y4 Yl 0 2 l O 1 O O 1 O O 4 1 O -112 O O 2 O o 1 1 o o From the final tableau (in the "f-row") we see that the solution to (P) is fmin = 8 for x1 = O, x2 = O, x3 = Duality and sensitivity analysis Since linear programming is often used in economics, it is common to give an economic interpretation of duality. Consider the primal problem (4.2) and its dual (4.3). For the optimal solution max f =ming=blyl+b2y,+...+ b,,y,,, whereb, is the amount available of resource i. Each biyi can therefore be interpreted as the contribution of resource i to the profit. This gives the following interpretation of the dual variable yi : yi is the contribution to the profit per unit of resource i (i = 1, m). It measures the rate at which f could be increased by increasing the amount of resource i, i.e. bi (the increase of bi must be sufficiently small so that the set of basic variables for the optimal solution does not change).

35 The dual variable yi is often referred to as the shadow price for resource i. If a shadow pricey, is zero, then resource i is not a limitation. Constraint i is therefore not binding on the optimal solution. Such a constraint is said to be nonbinding, while a constraint corresponding to a resource with a shadow price greater than zero is a binding constraint. Ex 4.6)Recall example 2.1 about the joiner who makes two products from three types of materials. The amount available of each type is 90, 120 and 100 units. The objective function of the dual problem is therefore min g = 90y, + 120y, + 100y, The optimal solution to the dual problem is g, =2300 US$ for yl = O, y, = 15, y, = 5. This means that if we had one unit more of material 2, the profit would increase with 15 US$. If we could get one unit more of material 3, then the profit would increase with 5 US$. The shadow price for material 1 is O. This means that this resource is not limited, and an increase of the amount available of it would not give an increased profit.

36 5 Integer linear programming A special type of linear programming is when some or all decision variables are integer valued. It is often the case that the decision variables only make sense if they have integer values, for example when assigning machines or people to some activity. A shipping company that wants to know how many cargo ships they should buy would not be pleased with the answer 2.25, only an integer number would do. 5.1 General form A general integer linear programming problem can be written n max f = C cjxj j=l f Ca,xj <bi, j=l subject to < xj 20, i=1,2,..., m j= 1,2,,,.,n xj is integer, j = 1,2,..., p (p I n) \ Here we have ordered the variables so that the first n variables are integer valued. If p = n then we have an integer linear programming (ILP) problem. If p < n then the mode1 is referred to as mixed integer linear programming (MILP). men distinguishing the two types from each other, ILP is called pure integer linear programming. Sometimes it is of interest to consider problems with decisions of the type yes or no. For example, should we build a new factory here? Should we start the production of a new product? Such decisions can be represented with decision variables restricted to only two values, zero and one. Let xj represent the jth yes- or-no decision. Then, xj = 1 if decision j is yes, xj = O if decision j is no. Variables of this kind are called binary variables, or 0-1 variables. ILP problems that contain only binary variables are called binary programming, or zero-one programming. (The adjective linear is normally dropped here.)

37 5.2 Some considerations on solving integer linear programming problems One may believe that ILP problems should be relatively easy to solve: LP problems can be solved very efficiently, and the only difference between LP and ILP problems is that ILP problems has far fewer feasible solutions. Actually, ILP problems with a bounded feasible region always have a finite number of feasible solutions. Unfortunately, this is not true. ILP problems are in general much more difficult to solve than LP problems. The efficiency of the simplex method is guaranteed by the fact that if there exists an optimal solution for an LP problem, it is always a corner-point solution. This is not the case for ILP problems because the optimal solution most often is an interior point in the feasible region. Could not we then solve an ILP problem without the integer restriction, and just round off the solution? The answer is no, which this example shows. Solve the following ILP problem: max f = sxl + 4x2 I 2x1 + 2x217 Sxl + 3x2120 subject to X,, X2 2 o (x,, x2 integers The problem is shown graphically in Fig The feasible points filled black circles. x2 are shown with o 1 2 s.. ' 3 4 Figure 5.1. Graphical view of an ILP problem

38 A graphical solution shows that the optimal solution is x, = 2, x, = 1 and f = 14. If we skip the integer restriction we will have the solution x, = 1.9, x, = 1.6, giving us f = This is not an integer solution. If we round off this solution we will get x, = 2, x, = 2 which is an infeasible solution. In this simple example it is easy to test all feasible points to see which one is the optimal point. Checking each point for feasibility and, if it is feasible, calculating the value of the objective function may work for very small problems only. Consider for example a BIP problem with 30 variables, which is a fairly small problem. This gives us z3o solutions to consider is slightly more than a billion, i.e. lo9). Finding the solution to this problem will take a lot of time even on fast computers. Hence, we must find other methods. Another difficulty with ILP problems is that is there is no easy way of telling if a solution is optimal or not. This is because a point can be a local optimum. In the example given above the point x, = O, x, = 3 is a better point than all its feasible neighbours, but it is not an optimal point. Since LP problems in general are much easier to solve than ILP problems, many algorithms for ILP use an LP-solver (e.g. the simplex method) on portions of the ILP problem where the integer restriction is deleted. The corresponding LP problem to a given ILP problem is referred to as its LP-relaxation. 5.3 The branch-and-bound technique for solving ILP problems The branch-and-bound technique uses three basic steps - branching, bounding and fathorning. The basic idea is to divide the ILP problem into small subproblems. This is called branching, and is done by partitioning the entire set of feasible solutions into smaller and smaller subsets. We then need a bound on each of these subproblems of how its best feasible solution can be. The easiest way of doing so is to solve the subproblem's LP-relaxation, i.e. the subproblem without the integer restrictions. The subproblem can be fathomed and dismissed from further consideration depending on the result of the bounding. The branch-and-bound algorithm is given here: Step O: Initialization. Let f be the value of the objective function for the best h integer solution so far. Set f = - w. Solve the LP-relaxation of the whole problem. If not integer valued or infeasible, go to step 1. Step 1: Branching. b ong the remaining subproblems, select the one created most recently. Of two subproblems created at the same time (subproblems are always created in pairs), select the one with the largest bound. From the optimal solution for the LP-relaxation of that subproblem, choose a variable with a noninteger value to be the

39 branching variable. Let xj be that variable, and Xj its value in this solution. Create two new subproblems from the subproblem by adding the constraints xj 5 [ Xj] and xj 2 [ Xj] + l, respectively. ( [ Xj] is the greatest integer less than or equal to xj.) Step 2: Bounding. For the two new subproblems, obtain their upper bound f of the objective function by applying the simplex method to their LPrelaxations. Step 3: Fathoming. Apply the following three tests to the new subproblems, and discard those subproblems that are fathomed by any of the tests. Test 1. If the upper bound is not better than the best integer solution so far, i.e. f 2 j', discard that subproblem. Test 2. If its LP-relaxation has no feasible solutions, discard that subproblem. Test 3. If the optimal solution for its LP-relaxation has integer values, discard that subproblem. If the upper bound is better than the current best integer solution, i.e. f >j', then store this solution and let j' =f. Step 4: Test for optimality. If there are no remaining subproblems, then the solution which gave us j' is the optimal solution. Otherwise, return to step 1. The branch-and-bound technique will be illustrated with the following example. Ex 5.1) Solve the following ILP problem (P): max f =3xl +3x2 +9x3 subject to I 6x1-3x2 + 6x3 18 X1>X2>X3r0 x,, x2, x3 integers We begin by solving the LP-relaxation to (P). This is problem a. PI: max f = 3x, + 3x2 + 9x3-3x1 + 6x2 + 6x x1-3x2 + 6x3 1 8

40 The solution to P1 is x, = x, = 2 8, x, = O and f = 16. Since none of x, and x, are integer valued, we divide P1 into two new subproblems with x, as the branching variable. The additional constraints to the new subproblems are x, < 2 and x, 2 3. Hence, we now have the problems P2 and P3. P2: max f =3x1 +3x, +9x3 { 1-32, + 6x, + 6x3 < 8 6;-3x,+6x3<8 subject to <2 P3: max f = 3x1 + 3x2 + 9x3 subject to { 6;-3x,+6x3<8 23 (-3x1 + 6x2 + 6x3 5 8 The solution to problem P2 is x, = x, = 2, x, = $ and f = 15. P3 has no feasible solution, and is thus discarded from further consideration. The only subproblem is problem P2. Since x, is the only real valued variable, it becomes the branching variable. Problem P2 is now divided into two new subproblems with the additional constraints x, 10 and x, 2 1. This gives us the problems P4 and E. P4: Additional constraints in problem P4 compared to problem P1 are: x1<2, x,io. P5: Additional constraints in problem P5 compared to problem P1 are: x,<2, x321. The solution to problem P4 is x, = 2, x, = 2 $, x, = O and f = 13. P5 has the solution x, = x, = $, x, = l and f = 13. Neither of P4 and P5 can be discarded, and they have the same upper bound. Therefore it is arbitrarily which one them are chosen for branching. In this example we choose P4, which is divided into the subproblems and using x, as the branching variable.