Notes: Deterministic Models in Operations Research

Transcription

1 Notes: Deterministic Models in Operations Research J.C. Chrispell Department of Mathematics Indiana University of Pennsylvania Indiana, PA, 15705, USA May 5, 2012

2 ii

3 Preface These notes will serve as an introduction to the basics of solving deterministic models in operations research. Topics discussed will included optimization techniques and applications in linear programming. Specifically a discussion of sensitivity analysis, duality, and the simplex method will be given. Additional topics to be considered include non-linear and dynamic programming, transportation models, and network models. The majority of this course will follow the presentation given in the Operations Research: Applications and Algorithms text by Winston [8]. I will supplement the Winston text with additional material from other popular books on operations research. For further reading you may wish to consult: Introduction to Operations Research by Hillier and Lieberman [2] Operations Research: An Introduction by Taha [7] Linear Programming and its Applications by Eiselt and Sandblom [1] Linear and Nonlinear Programming by Luenberger and Ye [4] Linear and Nonlinear Programming by Nash and Sofer [5] My Apologies in advance for any typographical errors or mistakes that are present in this document. That said, I will do my very best to update and correct the document if I am made aware of these inaccuracies. -John Chrispell iii

4 iv

5 Contents 1 Introduction Tabel and Chair Example Basic Linear Algebra The Gauss-Jordan Method for Solving Linear Systems Linear Programming Basics Parts of a Linear Program Linear Programming Assumptions Feasible Region and Optimal Solution Special Cases: Example: Multiple Optimal Solutions Example: Infeasible Linear Program Example: Unbounded Optimal Solution Setting up Linear Programs Work-Scheduling Problem: Examples of Linear Programs Diet Problem Solution Diet Problem Scheduling Problem: Solution Scheduling Problem: A Budgeting Problem Solution Budgeting Problem v

6 5 The Simplex Algorithm Standard Form Example: Basic and Nonbasic Variables Directions of Unboundedness Simplex Method Using Matrix-Vector Formulas Problem Using Tableaus Matrix Format of Linear Program Initial Tableau (Matrix Form): Duality A Motivating Example: My Diet Problem: Cononical Form Basic Duality Theory Relationship Between Primal and Dual Weak Duality Strong Duality The Dual Simplex Method Sensitivity Analysis Sensitivity Analysis Verify this graphically: Sensitivity Analysis Using Matrices Illustrating Example Non Linear Programming Data Fitting Linear Problems Taylor series Newton s Method Non-linear Least Squares vi

7 11 Network Flows Dijkstra s Algorithm Appendices Homework Homework Homework Homework Homework Bibliography 113 vii

8 Chapter 1 Introduction If you choose not to decide you still have made a choice. Neil Peart Operations Research, at its heart, uses modeling to solve applied mathematical problems in the hopes of making optimal decisions efficiently. The term Operations Research can trace its roots back to World War II when a scientific approach was adopted to distribute resources between various military operations (scientists doing research on military operations)[2, 8]. In order to do any scientific research on a problem a mathematical model is needed. The WInston text states: Mathematical Model: - is a mathematical representation of an actual situation that may be used to make a better decision or simply to understand the actual situation better. The goal with most models considered in this course will be to optimization some quantity (using an objective function based on decision variables) subject to problem constraints. Several examples where Operations Research techniques were implemented in order for different organizations to obtain optimal solutions and distribution of resources (CITGO Petroleum manufacturing, The San Francisco Police Department Scheduling, GE Capital credit card bill repayment). 1.1 Tabel and Chair Example Consider the following modification of an example given in [6]: A furniture manufacturer produces two products: wooden tables and chairs. The unit profit for the tables is $6, and the unit profit for the chairs is $8. In order to simplify the problem assume that only resources used in the production of the tables and chairs are wood (in board feet, bf) and labor (in hours, h). It takes 6 bf and 1 hour labor to make a table, and 4 bf and 2 hours labor to make a 1

9 chair. There are 20 bf of wood available and 6 hours of labor available. If you wish to maximize profit what is the optimal distribution of these resources? Formulate a Mathematical Model of the Problem The idea now becomes to solve the given problem using a linear model or linear programming. Thus, we need to translate the real world problem into a format with mathematical equations that represent: An objective function: In this case maximize the profit. Decision variables: Number of Tables and Chairs to produce. Constraint set: We only have a limited number of resources available to use when manufacturing the tabels and chairs. For this example the objective function and the constraints will all be linear functions. We should also impose non-negativity conditions on the resources and decision variables in this instance. If we do not enforce these conditions it may be possible to achieve non-physical solutions to the given problem. The problem definition, and formulation are often the most difficult and import step in finding an optimal solution. Lets define the decision variables x 1 and x 2 as the number of tables and the number of chairs to produce respectively. It is sometimes useful to place all the information into a table: Resource Table x 1 Chair x 2 Available Wood (bf) Labor (hr) Unit Profit $6 $8 Thus, the goal is to obtain the largest profit from the objective function by: subject to: the problem constraints. Maximize 6x 1 + 8x 2 6x 1 + 4x 2 20 x 1 + 2x 2 6 x 1 0 x 2 0 2

10 Solving the problem using a Graphical Approach Lets assume that the number of chairs to be produced, x 1 is on the x-axis and the number of tables to be produced x 2 is on the y axis. First write the constraints as equalities, and then finding the intercepts of the feasible region. Thus, 6x 1 + 4x 2 20 = x 2 = 3 2 x (1.1.1) x 1 + 2x 2 6 = x 2 = 1 2 x (1.1.2) x 1 0 x 1 = 0 (1.1.3) x 2 0 x 2 = 0 (1.1.4) (1.1.5) Figure 1.1.1: Plot of the constraints for the table and chair example. If all the wood is used to produce chairs (set x 1 = 0 in 1.1.1): x 2 = 5 this yields point C. If all the wood is used to produce tables (set x 2 = 0 in 1.1.1): x 1 = 10 3 this yields point E. If all the labor is used to produce chairs (set x 1 = 0 in 1.1.2): x 2 = 3 this yields point B. 3

11 If all the labor is used to produce tables (set x 2 = 0 in 1.1.2): x 1 = 6 this yields point D. Note we obtain a feasible region given by the polygon FEAB. Plot instances of the objective function If we now pick two values for the objective function we can determine its slope and a direction of improvement. Let z be the value of the objective function: z = 6x 1 + 8x 2 Thus, if z = 0 then and if z = 8 we see 6x 1 + 8x 2 = 0 = x 2 = 3 4 x 1, 6x 1 + 8x 2 = 8 = x 2 = 3 4 x The direction of improvement is d = [4, 3] T, and is illustrated in Figure The optimal solution will be obtained at the corner or edge of the feasible region that is farthest away from the origin on a line parallel to the objective function contours plotted for z = 0, and z = 8. Specifically from the figure the objective function is maximized at point A when 2 tables and 2 chairs are manufactured using the resources available, and yields a profit of $28. Figure 1.1.2: Illustration of the direction of improvement. 4

12 The optimal solution could have also been found by testing the objective function at each corner of the feasible region. Note if the same value of the objective function has been obtained at multiple corners the optimal solution of would have been any convex combination of the two corners. In this problem we are manufacturing discrete items (building a fraction of a table or chair doesn t make since) so we are lucky that we obtained an integer solution to the mathematical model. Here we will solved this simple Linear Program graphically. In the future we will use more advanced algebraic techniques (such as the simplex method ) to obtain the optimal solution. For more complicated problems it will become necessary to use software packages such as LINGO. In the future we will also look to see how sensitive our optimal solution is to small changes. 5

13 6

14 Chapter 2 Basic Linear Algebra Our weary eyes still stray to the horizon. Though down this road we ve been so many times. Pink Floyd Before looking into more complicated methods of solving linear programs we need to recall some of the basics of linear algebra. We will use the following notation. A matrix is a rectangular array of numbers, with a typical m n matrix A written in the form: a 11 a a 1n a 21 a a 2n A =..... (2.0.1). a m1 a m2... a mn The element of a matrix in the i th row and j th column of matrix A will be denoted by a ij. Thus, if the matrix A = then a 23 = 13. We will think of a matrix with only one column as vectors (or column vectors). Similarly a matrix with only one row will be a row vector. The number of rows in a vector will define its dimension, and the number of columns in a row vector will define its dimension. An m-dimensional vector (row or column) where all elements are zero will be called a zerovector and denoted by 0. In two dimensions [ ] 0 0 = [0 0] and 0 = 0 are zero vectors. 7

15 Vectors correspond to a directed line segment from the origin in the m-dimensional plane. Thus the vectors: [ ] [ ] 1 3 u =, and w = 2 4 are illustrated in Figure Readers should reacquaint themselves with matrix multipli- Figure 2.0.1: Illustration of two vectors in two-dimensional plane. cation, matrix transposition and inner products. The use of matrices and vectors will allow for mathematical models of linear systems to be developed and solved using linear algebra The Gauss-Jordan Method for Solving Linear Systems The three basic types of elementary row operations (ERO) are used when solving systems of linear equation using the Gauss-Jordan Method. 1. Multiply any row of a matrix system by a nonzero scalar. 2. Multiply any row of a matrix system by a nonzero scalar and then add that row to a different row of the matrix. 3. Interchange any two rows of a matrix system. 8

16 Consider the following example: x 1 2x 2 + 3x 3 = 9 x 1 + 3x 2 = 4 2x 1 5x 2 + 5x 3 = 17 We may write this linear system in the augmented matrix system notation as A b =

17 10

18 Chapter 3 Linear Programming Basics Remember to remember me Standing still in your past Floating fast like a hummingbird Wilco 3.1 Parts of a Linear Program We detailed earlier that the major components of a linear program: Decision variables. The objective function. The problem constraints. It is also worthy to note that the coefficient of a variable in the objective function is referred to as an objective function coefficient, and less obviously the coefficients in the constraint functions are sometimes referred to as technological coefficients. It will often be useful to denote the coefficients on the right-hand-side of the constraints with rhs (representing the amount of a distinct resource that is available). We will separate non-negativity of decision variables as a separate type of constraint, and make note that decision variables may be unrestricted in sign. Definitions: A function f(x 1, x 2,..., x n ) of x 1, x 2,..., x n is a linear function if and only if for some set of constants c 1, c 2,..., c n, f(x 1, x 2,..., x n ) = c 1 x 1 + c 2 x c n x n. 11

19 For any linear function f(x 1, x 2,..., x n ) and any number b, the inequalities f(x 1, x 2,..., x n ) b and f(x 1, x 2,..., x n ) b are linear inequalities. With these definitions we can define a linear programming problem or LP as an optimization problem for which we do the following: Maximize or minimize a linear function of some decision variables (our objective function). The decision variables are subject to satisfying a set of constraints each of which is a linear inequality. There is also a set of sign restrictions that will be satisfied. variables may be non-negative or unrestricted in sign. For example decision Linear Programming Assumptions In all linear programs the objective function and constraints are effected proportionally by the decision variables (a constant coefficient multiplies the variable in each). Linear programming problems also have the benefit of each decision variable being independent of others in its contribution to the objective function. Similarly linear constraints have the benefit of independence. For example in the table and chair example we considered the number of hours of labor required to make a chair did not effect the number of hours labor required to manufacture a table. Thus, the left hand side of the labor constraint was the additive sum of constants multiples of the decision variables. In addition to the proportionality and additivity assumptions in order for LP s to represent real situations, the divisibility assumption. That is the decision variables must be allowed to take on fractional values (we can not sell a fraction of a chair). In the future we may look at integer programming as a way of handling problems that require a discrete valued solution. We are also going to make a certainty assumption. That is that the model parameters objective function coefficients, the right hand side of constraints and technological coefficients are known with certainty Feasible Region and Optimal Solution Assuming that a point is a set of distinct values for each decision variable in an LP. The feasible region is the collection of all possible points that satisfy the constraints and sign restrictions on the decision variables. Points not within the feasible for a linear program are considered infeasible. The optimal solution for a maximization problem then becomes the point in the LP s feasible region that obtains the largest value of the objective function. Similarly for a minimization problem the point inside the LP s feasible region that obtains the smallest value of the objective function is considered the optimal solution. 12

20 A second graphical example Consider the Linear Program: max z = x 1 + x 2 s.t. x 1 + 5x x 1 + 5x 2 40 x 1 5 with x 1, x 2 0. The graphical solution to the linear program is shown in Figure Note the value of the objective function is is 7.4, and that only two of the constraints are binding at the optimal solution. We consider constraints binding if the left hand side of the constraint is equal to the right hand side when an optimal solution is achieved. The constraint x 1 5 is an example of a nonbinding constraint. Figure 3.1.1: Graphical solution to LP with bounded Feasible Region. The feasible region for this LP is a convex set. A set of points is aconvex set provided a line segment between any pair of points in the set is wholly contained in the set. 13

21 For any convex set S, a point P is an extreme point if each line segment that lies completely in S and contains the point P has P as an endpoint of the line segment. Extreme points may also be called corner points. We make note that: Any LP that has an optimal solution has an extreme point that is optimal. See the Winston text for a proof of this result. It is an important result because it narrows our search for an optimal solution down from the whole feasible region to just the extreme points of the set. A Minimization Problem Consider the following minimization linear programming problem: min z = x 1 + x 2 s.t. x 1 + 4x 2 8 4x 1 + x 2 6 with x 1, x 2 0. Here ( we again solve the LP graphically. Note that the optimal solution is found at point 16, ) giving the objective function a value of 14. The solution is illustrated in Figure Note that we could have used matrix notation to represent the previous problems. It will sometimes be useful to write our linear programs in this form. Figure 3.1.2: Graphical solution to minimization LP with bounded Feasible Region. 14

22 3.2 Special Cases: Linear programs are not always this nice! Sometimes the linear program has an infinite number of solutions. Sometimes the linear program has no feasible solution, and at other times the solution to the linear program is unbounded Example: Multiple Optimal Solutions Consider the following minimization linear programming problem: max z = 3x 1 + 2x 2 s.t. 6x 1 + 4x 2 32 x 1 + 2x 2 10 x 1 4 with x 1, x 2 0. When this LP is solved graphically as is done in Figure it can be seen that there exist multiple optimal solutions. Thus, the solution to the LP becomes the convex combination of the two extreme points represented in the graph by C and D. We can write the optimal solution to the LP by letting α [0, 1], and stating it as a convex set: optimal set = { (x, y) α (4, 2) + (1 α) (1.5, 5.75) }. (3.2.1) We can note that any values in the optimal set will yield a value of 16 in the objective function. Use the following link to explore this problem using an interactive java applet: Figure 3.2.3: Graphical solution to maximization LP with bounded Feasible Region and multiple optimal solutions. 15

23 3.2.2 Example: Infeasible Linear Program Consider the following minimization linear programming problem: max z = 3x 1 + 2x 2 s.t. x 1 + x 2 10 x x 1 4 with x 1, x 2 0. Figure shows the binding constraint contours and a plot of the objective functions Figure 3.2.4: Plot of binding constraint contours and objective function contour when z = 10. contour for a value z = 10. Note that there is no region that will satisfy all of the linear programs constraints making this an infeasible LP Example: Unbounded Optimal Solution Consider one last example (straight out of the Winston text). linear program graphically. Solve again the following max z = 2x 1 x 2 s.t. x 1 x 2 1 2x 1 + x 2 6 with x 1, x 2 0. In Figure it can be seen that the feasible region is unbounded in the direction of increasing z contours. Thus the optimal solution is unbounded. 16

24 Figure 3.2.5: Plot of the feasible region for the maximization problem. Note that the feasible region is unbounded in the direction of increasing z contours. 3.3 Setting up Linear Programs In many cases the difficult part of linear programming is translating a physical situation into a mathematical model. The following are a collection of word problems that may be formulated as a linear program Work-Scheduling Problem: This example is from the Winston text: Consider a chain of computer stores. The number of skilled repair time the company requires during the next five months is given in the following table: Month 1 (January): 6,000 hours Month 2 (February): 7,000 hours Month 3 (March): 8,000 hours Month 4 (April): 9,500 hours Month 5 (May): 11,000 hours At the beginning of January, 50 skilled technicians work for the company. Each skilled technician can work up to 160 hours per month. To meet future demands, new technicians must be trained. It takes one month to train a new technician. During the month of training the trainee must be supervised for 50 hours by an experienced technician. Each experienced technician is paid $2000 a month regardless of how much he works. During the month of training the trainee receives $1000 for the month. At the end of each month 5% of the companies experienced technicians quit to find other employment. Formulate an LP for the company that will minimize the labor costs incurred and meet the service demands for the next five months. 17

25 18

26 Chapter 4 Examples of Linear Programs The tall one wants white toast, dry, with nothin on it... And the short one wants four whole fried chickens, and a Coke. Mrs. Murphy (The Blues Brothers) In this section our goal will be to consider setting up some common types of linear programs. The list of examples given here is by no means exhaustive, and we should not for get about the work scheduling problem stated in the previous section. 4.1 Diet Problem My diet may not be nearly as poor as that of the blues brothers; however, it is probably still pretty bad. Lets assume that my diet requires that all of my food come from my favourite food groups: pez candy, pop tarts, Cap N crunch, and cookies. I have the following foods available for my consumption: red pez, strawberry pop-tarts, peanut butter crunch, and chocolate chip cookies. The red pez costs $0.20 per (per package), strawberry pop-tarts cost $0.50 (per package), peanut butter crunch costs $0.70 per bowl, and chocolate chip cookies cost $0.65 cents each. I must ingest at least 2500 calories a day in order to maintain my sugary lifestyle. I must also meet the following requirements I need 8 oz. of sugar, 10 oz. of fat, and 50 mg of yellow-5 food coloring. Each of my chosen foods has these required nutrients in the following quantities: Food Calories Sugar (oz.) Fat (oz.) yellow-5 (mg) red pez candy (per package) strawberry pop-tarts (per package) peanut butter Cap N crunch (per bowl) chocolate chip cookies (1 cookie) How can I achieve my dietary constraints at a minimum cost? 19

27 4.1.1 Solution Diet Problem The first step will be do define some decision variables. Here we will define x 1 as the number of pez packages to include in my diet per day x 2 as the number strawberry pop-tart packages to consume per day x 3 as the number of bowls of peanut butter crunch to consume per day x 4 as the number of chocolate chip cookies to consume each day Our objective function can then be defined as: min z = 0.20x x x x 4 which will minimize the cost of our diet. The next aim is to meet the special dietary needs. Here a constraint is set up for each of the nutrients (calories, sugar, fat, and yellow 5). 50x x x x (calories required) 0.1x x 2 + x x 4 8 (sugar required) 0.01x x 2 + x x 4 10 (fat required) 5x 1 + 0x 2 + 4x 3 + x 4 50 (yellow-5 required) It can also be noted that the decision variables will be non-negative in sign giving the sign restriction. x i 0 for all i 1, 2, 3, 4 This problem can be set up in vector notation using the above variable definitions and defining x x = x 2 x 3, c = , b = 8 10, and x A = The optimal diet can now be found by considering: min z = c T x s.t. Ax b with x 0.. The optimal solution can be quickly found using the LINGO software package and the old LINDO syntax. Open LINGO and type: 20

28 min 0. 2 x x x x 4 s. t. 50 x x x x 4 >= x x 2 + x x 4 >= x x 2 + x x 4 >= 10 5 x x x 3 + x 4 >= 50 This yields the optimal objective function value of $7.39 when x 1 = , x 2 = , x 3 = , and x 4 = (4.1.1) So the minimum cost diet has me eating nothing but approximately 2 packages of pez candy and 10 bowls peanut butter crunch to meet my dietary constraints. 4.2 Scheduling Problem: Suppose that the math department wants to schedule tutors for a cram-day before final exams. The number of tutors needed for each four hour shift on this cram-day are: Shift Number Time Number of Tutors Needed 1 12:00 am - 4:00 am 3 2 4:00 am - 8:00 am 4 3 8:00 am - 12:00 pm :00 pm - 4:00 pm 5 5 4:00 pm - 8:00 pm 8 6 8:00 pm - 12:00 am 4 Each tutor will work two consequtive 4 hour shifts. Formulate a linear program that can be used to minimize the number of tutors needed to meet the cram-day demands Solution Scheduling Problem: Its always best to think about the decision variable first! starting shift for the tutors. Here we can think about the Let x i be the number of tutors that start working on shift i. The objective of the linear program is to minimize the number of tutors needed to meet the demands, and we make note that summing the decision variables will count the total number of tutors used. We should also note that since we are only concerning ourselves with 8 hour shifts for a single day, we only need to start tutors for the first five shifts. This give the objective function: 5 minimizez = 21 i=1 x i

29 Then the constraints become: x 1 3 x 1 + x 2 4 x 2 + x 3 6 x 3 + x 4 5 x 4 + x 5 8 x 5 4 where x i 0 for i {1, 2,..., 5}. (4.2.2) Note we also added the non-negativity constraint for all the decision variables. It may have also been nice to consider making the schedule for several cram-days in a row that again all had the same demands for the number of tutors needed during each shift. 4.3 A Budgeting Problem Two investments with varying cash flows (in thousands of dollars) are available. Investment Cash Flow Year 0 Cash Flow Year 1 Cash Flow Year 2 Cash Flow Year Assuming at time 0 that $10,000 is available to invest, and after one year there will be $7,000 available to invest. We will also assume an annual interest rate of 10% is available. We shall also assume that any fraction of an investment may be purchased. Lets find a linear program that will determine the maximum net present value obtained from the two investments Solution Budgeting Problem We set up the linear program as follows. First the decision variables: Let x i be the amount of investment i to purchase (i = 1, 2). Our goal is to maximize the net present value of the investments purchased so we should find the net present value of each investment for use in our objective functions. Net present value: We need to discount each of the cash flows back to the present using a discount factor based on the interest rate. Note that $1.00 Today $ ($1.00) = $1.00(1.10) One Year From Now 22

30 Thus, $1.00 one year from now ( ) 1 $1.00 $ today We can use ( ) n 1 as a discount factor for rate r in year n. 1 + r Let NP V 1 denote the net present value of investment 1. ( ) 1 ( ) 2 ( ) NP V 1 = ( 5000) Let NP V 2 denote the net present value of investment 2. ( ) 1 ( ) 2 ( ) NP V 2 = ( 3000) The objective function for the linear program is then: maximize z = x x 2. The constraints on the linear program are based on the amount of cash we have on hand for each of the first two years, and the additional constraint that we may only purchase upto 100% of each of the two investments. This gives 6000x x x x x 1 1 x 2 1 as our constraints, and we have the sign restriction that x 1 and x 2 are non-negative: x 1 0, and x 2 0. Note we can solve this linear program graphically and Figure illustrates the feasible solution to the problem. We can see by looking for the optimal objective function contour that the optimal investment involves 100% of investment 1 and 50% of investment 2 giving the optimal net present value of the cash flow as: z = (1) (0.5) = Note that there are an infinite number of different problems that may be formulated as linear programs. It is strongly recommended that the interested reader look over more examples presented in [8] and other texts. 23

31 Figure 4.3.1: Feasible region for the budgeting problem with an illustration of a z contour. 24

32 Chapter 5 The Simplex Algorithm Don t wanna wait til tomorrow Why put it off another day? One by one, little problems Build up and stand in our way, oh Van Halen The linear programs that we have been able to solve with out the assistance of a software package so far have all been in two variables. In general linear programs have many variables and the goals of this chapter will be: Transform a linear programming problem into a standard form. Look at basic and nonbasic variables. Introduce the Simplex Algorithm for solving linear programming problems. 5.1 Standard Form As we have seen there are several different ways of writing the objective function for a linear program (maximize profit, minimize cost). Additionally the constraints come in all different forms too (,, =) so when discussing a systematic method for solving linear programs it will be advantages to having written or converted the linear program to a standard form. We will consider the following linear programs to be in standard form: minimize z = c T x, maximize z = c T x, subject to Ax = b, subject to Ax = b, x 0. x 0. where b, c, and x represent vectors of length n, and A is an m n constraint matrix, and b 0. 25

33 Notice that: The decision variables are constrained to be non-negative. All of the constraints are set up to be equalities. The components of the right hand side vector b are non-negative. The observation that a maximization problem could be converted to a minimization problem by multiplying every coefficient in the objective function by a -1 can be made. After the problem is solved the objective value would then need to be multiplied by -1 again in order to obtain the solution to the desired problem. The decision variable values obtained would be the same for both objective functions. Lets consider transforming the following linear program into a standard form minimization problem: maximize z = 3x 1 +4x 2 7x 3 subject to 7x 1 +x 2 +4x x 1 +4x 2 +6x 3 2 6x 1 +3x 2 4x 3 4 x 3 5 with x 1 0 and x 2 free. Objective function: The problem could be written using the equivalent objective function: minimize ẑ = 3x 1 4x 2 + 7x 3 where z = ẑ once the optimal solution has been found. Non-negative right had side: In order to continue converting the given LP to standard form we next consider the constraints. We would like to have all the right hand side coefficients be non-negative. Thus, the third constraint in the given linear program should be multiplied by a -1 yielding: 2x 1 + 4x 2 + 6x 3 2 2x 1 4x 2 6x 3 2 Non-zero lower bounds: In the original statement of the problem we note that the 4th constraint has: x 3 5. To convert this to a standard form constraint we make a change of variables and define: ˆx 3 = x 3 5 This allows us to replace x 3 with ˆx 3 throughout and changes the constraint x 3 5 ˆx ˆx

34 Note we need to modify the other constraints and objective function as well. For our example problem we may redefine ẑ = z 35 z = ẑ 35 for the modified objective function. For the moment we will leave upper bounds on variables right in the coefficient matrix. Free variables: In the given problem we make note that x 2 is a free variable. Free variables may be converted to non-negative constraints by defining x 2 = x 2 x 2 with x 2, x 2 0. and we will have x 2 to handle the positive values of x 2 and the new variable x 2 will take care of the negative pieces. There are other ways to handel free variables, and one is to use a free variable as a way of eliminating a constraint (think about this for a future homework assignment). After applying all of the discussed techniques to our linear programming problem we are left with the following LP: minimize ẑ = 3x 1 4x 2 + 4x 2 +7ˆx 3 subject to 7x 1 +x 2 x 2 +4ˆx 3 6 2x 1 4x 2 + 4x 2 6ˆx x 1 +3x 2 3x 2 4ˆx 3 24 with x 1, x 2, x 2, ˆx 3 0. Equality constraints: In order for the constraints we consider adding slack and excess variables to enforce the equality constraint. For the constraint: 7x 1 + x 2 x 2 + 4ˆx 3 6 let s 1 be a slack variable such that s 1 0 and 7x 1 + x 2 x 2 + 4ˆx 3 + s 1 = 6. For the other two constraints we use excess variables e 1 ande 2. Thus, 2x 1 4x 2 + 4x 2 6ˆx 3 32 = 2x 1 4x 2 + 4x 2 6ˆx 3 e 1 = 32 and 6x 1 + 3x 2 3x 2 4ˆx 3 24 = 6x 1 + 3x 2 3x 2 4ˆx 3 e 2 = 24 with e 1, e 2 0. We can now restate the full linear program in standard form: minimize ẑ = 3x 1 4x 2 + 4x 2 +7ˆx 3 subject to 7x 1 +x 2 x 2 +4ˆx 3 +s 1 = 6 2x 1 4x 2 + 4x 2 6ˆx 3 e 1 = 32 6x 1 +3x 2 3x 2 4ˆx 3 e 2 = 24 with x 1, x 2, x 2, ˆx 3, s 1, e 1, e

35 5.1.1 Example: Try writing the following linear programs in standard form: After manipulation we obtain: maximize z = 3x 1 +5x 2 4x 3 subject to 7x 1 2x 2 3x 3 4 2x 1 +4x 2 +8x 3 = 3 5x 1 3x 2 2x 3 9 with x 1 1, x 2 7, and x 3 0. maximize ẑ = 3ˆx 1 +5x 2 5x 2 4x 3 subject to 7ˆx 1 +2x 2 2x 2 +3x 3 3 2ˆx 1 4x 2 + 4x 2 8x 3 = 1 5ˆx 1 3x 2 + 3x 2 2x 3 4 x 2 x 2 7 with ˆx 1, x 2, x 2, x 3 0 and z = ẑ + 3. We need to take this one more step and include slack variables. Thus, maximize ẑ = 3ˆx 1 +5x 2 5x 2 4x 3 subject to 7ˆx 1 +2x 2 2x 2 +3x 3 +s 1 = 3 2ˆx 1 4x 2 + 4x 2 8x 3 = 1 5ˆx 1 3x 2 + 3x 2 2x 3 +s 2 = 4 x 2 x 2 +s 3 = 7 with ˆx 1, x 2, x 2, x 3, s 1, s 2, s 3 0 and z = ẑ Basic and Nonbasic Variables Lets assume that we have a standard form linear program. minimize z = c T x, subject to Ax = b, x 0. (5.2.1) where b, c, and x represent vectors of length n, and A is an m n constraint matrix (with n m), and b 0. The constraints to the linear program are given by: Ax = b. 28

36 Definition A basic solution to Ax = b is obtained by setting n m variables equal to 0 and solving for the values of the remaining m variables. This assumes that the columns remaining m variables are linearly independent. To find a basic solution to Ax = b, we simply choose a set of n m variables (and set these to zero) to be the nonbasic variables or NBV. The remaining variables will be our basic variables or BV that are used to satisfy the constraints. Consider the following example: x 1 + x 2 = 6 x 2 + x 3 = 4 Here we get to pick one nonbasic variable as we have two equations and three unknowns. Thus, NBV = {x 3 }, then BV = {x 1, x 2 }. The values of the basic variables x 1, and x 2 are found by solving the two equations with x 3 set to zero. Thus, x 2 = 4 and x 1 = 10 If we choose x 1 to be the nonbasic variable then the basic variables x 2 = 6 and x 3 = 10. We should note that not all sets of basic variables will yield a feasible solution to a set of constraints. Consider the constraints x 1 + 2x 2 + 4x 3 = 4 x 1 + 4x 2 + 8x 3 = 6. If x 1 is taken to be the nonbasic variable then the system becomes: 2x 2 + 4x 3 = 4 4x 2 + 8x 3 = 6 a system with no feasible solution and yielding no basic solution to BV = {x 2, x 3 }. Definition Any basic solution to (5.2.1) in which all variables are nonnegative is a basic feasible solution. When solving linear programs we are interested in sets of variables that will satisfy all of the constraints given by Ax = b as well as also satisfying the nonnegativity constraint for the linear program stated in standard form. We also know that 29

37 Any LP that has an optimal solution has an extreme point that is optimal. So a good place to start looking for these optimal extreme points is at basic feasible solutions. It can be shown that: Theorem A point in the feasible region of a linear program is an extreme point if and only if it is a basic feasible solution to the LP. A proof can be found in [3]. Example: The goal of this example is to show the relationship between extreme points and basic feasible solutions. Given maximize z = 4x 1 + 3x 2 subject to x 1 + x 2 4 2x 1 + x 2 6 with x 1, x 2 0, write the LP in standard form by adding two slack variables s 1 and s 2. Standard form: maximize z = 4x 1 + 3x 2 subject to x 1 + x 2 + s 1 = 4 2x 1 + x 2 + s 2 = 6 with x 1, x 2, s 1, s 2 0. The feasible region for the linear program is shown if Figure Note that all of the corner extreme points correspond to a basic feasible solution to the linear program. We note that the intersection at points E, and F are not basic feasible solutions because not all of the basic variables satisfy the nonnegativity constraint for the linear program in standard form Directions of Unboundedness Assume that we have a standard form linear program. minimize z = c T x, subject to Ax = b, x 0. where b, c, and x represent vectors of length n, and A is an m n constraint matrix (with n m), and b 0. The feasible solutions for this linear program will be denoted by S. 30

38 Figure 5.2.1: Definition An n by 1 vector d is a direction of unboundedness if for all x in S, and c 0 then x + cd S. This means we can move as far as we desire in the direction of d and still have a feasible solution to the linear program. Consider the following example minimize z x 1 + x 2, subject to 7x 1 + 2x x x 2 24 x 1, x 2 0. Written in standard form by adding excess variables we obtain, minimize z x 1 + x 2, subject to 7x 1 + 2x 2 e 1 = 28 2x x 2 e 2 = 24 x 1, x 2, e 1, e 2 0. What are the basic feasible solutions for the LP. 31

39 BV NBV bfs (Basic Feasible Solution) x 1, x 2 e 1, e 2 YES, x = (3.6, 1.4, 0, 0) T x 1, e 1 x 2, e 2 YES, x = (12, 0, 56, 0) T x 1, e 2 x 2, e 1 NO, x = (4, 0, 0, 16) T x 2, e 1 x 1, e 2 NO, x = (0, 2, 24, 0) T x 2, e 2 x 1, e 1 YES, x = (0, 14, 0, 144) T e 1, e 2 x 1, x 2 NO, x = (0, 0, 28, 24) T Looking at the problem s feasible region graphically we have the illustration in Figure Specifically the feasible region for the linear program can be made up of a convex combination of the basic feasible solution points plus a positve constant multiple of a direction of unboundedness. Note that moving from the point basic feasible solution at point and moving toward point yields a direction of unboundedness given by: d 1 = Note that the direction of unboundedness is not unique, as 2 d 2 = is a direction of unboundedness found by heading from the basic feasible solution toward interior point Specifically we could write the feasible set by letting α 1, α 2, α 3 [0, 1] such that 3 i=1 α i = 1, and c a nonnegative constant then See S = α α α c

40 Figure 5.2.2: The feasible region is shown. Note that the region may be made up of a convex combination of basic feasible solutions plus a direction of unboundedness. for an interactive version of the solution. 33

41 34

42 Chapter 6 Simplex Method Using Matrix-Vector Formulas Learn to be positive, it s your only chance. The Kinks Lets consider looking at the simplex algorithm using Matrices and compare that with the tableau format we have been using. 6.1 Problem Using Tableaus Consider the following linear program: maximize z = 3x 1 + 5x 2 subject to x 1 4 2x x 1 + 2x 2 18 x 1, x 2 0. Solution full Tableau Form: Adding slack variables in the constraints the initial tableau for the given linear program is written as: Initial Tableau: Row z x 1 x 2 s 1 s 2 s 3 RHS Basis z s s s 3 In the first pivot x 2 enters the basis replacing s 2. Here we have not explicitly shown the ratio test. Thus, 35

43 Pivot 1 Tableau: Row z x 1 x 2 s 1 s 2 s 3 RHS Basis z s x s 3 The next iteration of the simplex method shows that it is desirable to have x 1 enter the basis and replace s 3. (Note again we are not showing the ratio text.) Doing this pivot yields: Pivot 2 Tableau: Row z x 1 x 2 s 1 s 2 s 3 RHS Basis z s x x Here we have obtained an optimal tableau. We can see here that z is maximized at $36,000 when x 1 = 2 and x 2 = Matrix Format of Linear Program A standard form linear program in matrix form is written as: maximize z = c T x subject to Ax = b x 0. Thus, for the example problem we make the following definitions: x 1 3 x 2 x = s 1 s 2, c = , b = s 3 0 and A = In order to implement the simplex algorithm in matrix format we break the given coefficient matrix A into to parts: a basic part and a non-basic part. Thus, A = [N B] and similarly we separate the vectors x, and c into their basic and non-basic parts: ( ) ( ) s 1 xn x1 x =, where x x N = and x B x B = s 2 2 s 3 36.

44 c = ( cn c B ) ( 3, where c N = 5 ) and c B = We can now make note of the correspondence between the simplex method as we know it in tableau format, and the defined matrices. Any current basis may be written as: T x N T x B RHS Basis c T N + c T B B 1 N 0 c T B B 1 b z B 1 N I B 1 b x B Initial Tableau (Matrix Form): For the starting basis x B = (s 1 s 2 s 3 ) T, and x N = (x 1 x 2 ) T we have: N = 0 2, B = = B 1 = with c N T c B T = ( 3 5 ) = ( ) Evaluating each of the expressions in the tableau we have: c T N + c T B B 1 N = ( 3 5 ) c T B B 1 b = ( 0 ) 4 B 1 b = B 1 N = Ratio Test: Note that both non-basic variables are attractive, and we choose x 2 to enter the basis. Doing the ratio test using the second column of B 1 N and the current right hand side vector B 1 b: { } min = 6, 2 2 = 9 and we choose s 2 to leave the basis. Note we have ignored the comparison of 4 over 0 in the ratio test. Compare the above values with the tableau: 37

45 Initial Tableau: Row z x 1 x 2 s 1 s 2 s 3 RHS Basis z s s s 3 Pivot 1 Tableau (Matrix Form): Note we have now replaced s 2 in the basis with x 2. Thus, x B = (s 1 x 2 s 3 ) T, and x N = (x 1 s 2 ) T we have: N = , B = = B 1 = with c N T c B T = ( 3 0 ) = ( ) Evaluating each of the expressions in the tableau we have: c T N + c T B B 1 N = ( c T B B 1 b = ( 30 ) 4 B 1 b = B 1 N = ) Ratio Test: Note that the non-basic variable x 1 should be picked to enter the basis. Doing the ratio test using the first column of B 1 N and the current right hand side vector B 1 b: { 4 min 1 = 4, 6 } 3 = 2 and we choose s 3 to leave the basis. Note we have ignored the comparison of 6 over 0 where x 2 would potentially leave the basis in the ratio test. Compare values with the Piviot 1 Tableau: 38

46 Pivot 1 Tableau: Row z x 1 x 2 s 1 s 2 s 3 RHS Basis z s x s 3 Pivot 2 Tableau (Matrix Form): Note we have now replaced s 3 in the basis with x 1. Thus, x B = (s 1 x 2 x 1 ) T, and x N = (s 3 s 2 ) T we have: N = 0 1, B = = B = with c N T c B T = ( 0 0 ) = ( ) Evaluating each of the expressions in the tableau we have: c T N + c T B B 1 N = ( ) c T B B 1 b = ( 36 ) 2 B 1 b = 6 2 B 1 N = Note that the non-basic variables are nolonger attractive and we have reached an optimal solution to the problem with z = 36 when, x 1 = 2 and x 2 = 6. Compare with the Pivot 2 Tableau: Pivot 2 Tableau: Row z x 1 x 2 s 1 s 2 s 3 RHS Basis z s x x

47 40

48 Chapter 7 Duality I d like to change your mind by hitting it with a rock. They Might Be Giants As is often the case you may look at problems form many different perspectives. This is the case with linear programs. Lets consider a problem form our past in two different perspectives. 7.1 A Motivating Example: My Diet Problem: Recall from earlier this semester that I have a really bad diet that requires that all of my food come from my favourite food groups: pez candy, pop tarts, Cap N crunch, and cookies. I have the following foods available for my consumption: red pez, strawberry pop-tarts, peanut butter crunch, and chocolate chip cookies. The red pez costs $0.20 per (per package), strawberry pop-tarts cost $0.50 (per package), peanut butter crunch costs $0.70 per bowl, and chocolate chip cookies cost $0.65 cents each. I must ingest at least 2500 calories a day in order to maintain my sugary lifestyle. I must also meet the following requirements I need 8 oz. of sugar, 10 oz. of fat, and 50 mg of yellow-5 food coloring. Each of my chosen foods has these required nutrients in the following quantities: Food Calories Sugar (oz.) Fat (oz.) yellow-5 (mg) Price ($) pez candy (per package) pop-tarts (per package) Cap N crunch (per bowl) cookies (1 cookie) How can I achieve my dietary constraints at a minimum cost? Primal Solution To Diet Problem: The first step will be do define some decision variables. Here we will define 41

49 x 1 as the number of pez packages to include in my diet per day x 2 as the number strawberry pop-tart packages to consume per day x 3 as the number of bowls of peanut butter crunch to consume per day x 4 as the number of chocolate chip cookies to consume each day Our objective function can then be defined as: min z = 0.20x x x x 4 which will minimize the cost of our diet. The next aim is to meet the special dietary needs. Here a constraint is set up for each of the nutrients (calories, sugar, fat, and yellow 5). 50x x x x (calories required) 0.1x x 2 + x x 4 8 (sugar required) 0.01x x 2 + x x 4 10 (fat required) 5x 1 + 0x 2 + 4x 3 + x 4 50 (yellow-5 required) It can also be noted that the decision variables will be non-negative in sign giving the sign restriction. x i 0 for all i {1, 2, 3, 4} This problem can be set up in vector notation using the above variable definitions and defining x x = x 2 x 3, c = , b = 8 10, and x A = The optimal diet can now be found by considering: min z = c T x s.t. Ax b with x 0.. Dual of Diet Problem A second way to consider the problem. Suppose that I know a nutrient salesman (Willy Loman) who will sell me supplements that taste just as good as the items in my diet. Specifically Mr. Loman sells calories, sugar, Fat, and yellow-5 and wants to sell me these items in order to meet my daily needs at the maximum price I m willing to pay. Then defining the decision variables: 42

50 y 1 is the price to charge me per calorie. y 2 is the price to charge me per ounce of sugar. y 3 is the price to charge me per ounce of fat. y 4 is the price to charge me per mg of yellow-5. Loman s objective function for my diet would look like: max w = 2500y 1 + 8y y y 4 The constraints for the salesman are found using the available foods. Mr. Loman being a great salesman needs to set his prices low enough that I will purchase his nutrients rather than my regular diet of pez and cookies. Thus, he is subject to the constraints: 50y y y 3 + 5y ( the price of pez) 250y y y 3 + 0y ( the price of pop-tarts) 350y 1 + y 2 + y 3 + 4y ( the price of Cap N crunch) 150y y y y + y ( the price of cookies) where the sign restriction on the decision variables is: y i 0 for all i {1, 2, 3, 4} Writing this in matrix form in terms of the values defined for the primal problem we have: max w = b T y s.t. A T y c with y 0 where y = (y 1 y 2 y 3 y 4 ) T. These two linear programs have the same optimal solution values Cononical Form The cononical form of a primal and its corresponding dual linear program are given as follows: min z = c T x s.t. Ax b with x 0. Primal max w = b T y s.t. A T y c with y 0 Dual 43

51 Taking Dual of General Linear Programs What if we don t start in canonical form? Lets consider the constraints first: min z = c T x s.t. A 1 x b 1 A 2 x b 2 A 3 x = b 3 with x 0. Transform the above problem into standard canonical form (ignoring that the RHS may be negative for a moment). Taking the dual we have: Making the change of variables: and we have: min z = c T x s.t. A 1 x b 1 A 2 x b 2 A 3 x b 3 A 3 x b 3 with x 0. max w = b T 1 y 1 b T 2 y 2 + b T 3 y 3 b T 3 y 3 s.t. A T 1 y 1 A T 2 y 2 + A T 3 y 3 A T 3 y 3 c with y 1, y 2, y 3, y 3 0. y 2 = y 2 and y 3 = y 3 y 3 max w = b T 1 y 1 + b T 2 y 2 + b T 3 y 3 s.t. A T 1 y 1 + A T 2 y 2 + A T 3 y 3 c with y 1 0, y 2 0, y 3 is free What if the variables are not in standard form? min z = c T 1 x 1 + c T 2 x 2 + c T 3 x 3 s.t. A 1 x 1 + A 2 x 2 + A 3 x 3 b with x 1 0, x 2 0, x 3 is free Here we make a change of variables and place the problem back into a canonical form. Let x 2 = x 2 and x 3 = x 3 x 3 44