LINEAR PROGRAMMING. Introduction

Size: px

Start display at page:

Download "LINEAR PROGRAMMING. Introduction"

Tiffany Paul
5 years ago
Views:

1 LINEAR PROGRAMMING Introduction Development of linear programming was among the most important scientific advances of mid-20th cent. Most common type of applications: allocate limited resources to competing activities in an optimal way. Linear programming uses a mathematical model. Linear because it requires linear functions. Programming as synonymous of planning. Besides allocate resources to activities, anyproblem described by a mathematical model can be solved. Very efficient solution procedure: simplex method. 25 1

2 Prototype example Wyndor Glass Co. produces glass products, including windows and glass doors. Plant 1 produces aluminum frames, Plant 2 produces wood frames and Plant 3 produces glass and assembles. Two new products: Glass door with aluminum framing (Plant 1 and 3) New wood-framed glass window (Plant 2 and 3) As the products compete for Plant 3, the most profitable mix of the two products is needed. 26 Problem definition OR team discusses with management to identify the objectives, developing the definition of the problem: Determine production rates for the two products to maximize their total profit, subject to the restrictions imposed by the limited production capacities available in the three plants. (Each product will be produced in batches of 20, so the production rate is defined as the number of batches produced per week.) Any combination of production rates that satisfies these restrictions is permitted, including producing none of one product and as much as possible of the other. 27 2

3 Data for Wyndor Glass Co. OR team also identified data to be gathered: 1. Number of hours of production time available per week in each plant for these new products. 2. Number of hours of production time used in each plant for each batch produced of each new product. 3. Profit per batch produced of each new product. Wyndor Glass Co. Product-Mix Problem Doors Windows Profit Per Batch $3.000 $5.000 (Batches of 20) Hours available Hours Used Per Batch Produced per week Plant Plant Plant Formulation of the LP problem x 1 = number of batches of product 1 produced per week x 2 = number of batches of product 2 produced per week Z= total profit per week (in thousands) maximize Z = 3x + 5x 1 2 subject to x x 12 3x + 2x 18 and x 0, x

Graphical solution Trial-and-error: Z = 10, 20, 36 3 1 Slope of the line is 3/5: x2 = x1 + Z 5 5 Feasible

1 2 1 2 subject to x 4 2 1 2 1 2x 12 3x + 2x 18 30 Solution(Excel Solver) see p. 65-69 Wyndor Glass Co.

000 BatchesProduced C12:D12 Hours Hours HoursAvailable G7:G9 Hours Used Per Batch Produced Used Available

4 Graphical solution Trial-and-error: Z = 10, 20, Slope of the line is 3/5: x2 = x1 + Z 5 5 Feasible region: satisfies all constraints maximize Z = 3x + 5x and x 0, x subject to x x 12 3x + 2x Solution(Excel Solver) see p Wyndor Glass Co. Product-Mix Problem Doors Windows Range Name Cells Profit Per Batch $3.000 $5.000 BatchesProduced C12:D12 Hours Hours HoursAvailable G7:G9 Hours Used Per Batch Produced Used Available HoursUsed E7:E9 Plant <= 4 HoursUsedPerBatchProduceC7:D9 Plant <= 12 ProfitPerBatch C4:D4 Plant <= 18 TotalProfit G12 Doors Windows Total Profit Batches Produced 2 6 $

5 Discussion Solution indicates WyndorGlass Co. should produce products 1 and 2 at the rate of 2 and 6 batches per week, respectively, with a resulting total profit of $36,000 per week, according to the model. However, well-conducted OR studies do not simply find one solution for the initial model formulated and then stop. All six phases described previously are important, including thorough testing of the model and postoptimality analysis. 32 Generalizing example Prototype example Production capacities of plants General problem Resources 3 plants m resources Production of products Activities 2 products n activities Production rate of product j, x j Profit Z Level of activity j, x j Overall measure of performance Z The most common type of LP application involves allocating resources to activities. 33 5

6 Linear programming model Z= value of overall performance measure x j = level of activity j, for j = 1, 2,, n. c j = parameters of Zrelated to x j. b j = amount of resource ithat is available for allocation of activities, for i = 1, 2,, m. a ij = amount of resource iconsumed by each unity of activity j. 34 Standard form maximize Z = c1x1 + c2x2 + + cnxn subject to a x + a x + + a x b n n 1 a x + a x + + a x b n n 2 a x + a x + + a x b m1 1 m2 2 mn n m and 0, 0,, 0. x1 x2 x n 35 6

7 Other possible forms Minimizing rather then maximizing minimize Z = c1x1 + c2x2 + + cnxn Some functional constraints are greater-than-or-equal ai1x1 + ai2x2 + + ainxn bi, for some values of i Some functional constraints in equation form ai1x1 + ai2x2 + + ainxn = bi, for some values of i Deleting the non-negativity constraints for some vars x j unrestricted in sign for some values of i 36 Allocation of resources to activities Resource Resource Usage per Unit of Activity Activity 1 2 n Amount of Resource available 1 a 11 a 12 a 1n b 1 2 a 21 a 22 a 2n b 2 m a m1 a m2 a mn b m Contribution to Z per unit of activity c 1 c 2 c n 37 7

8 Terminology for solutions Solution: any specification of values for the decision variables (x 1, x 2,, x n ). Feasible solution: solution for which allthe constraints are satisfied. Infeasible solution: solution for which at least one constraint is violated. Feasible region: collection of all feasible solutions. No feasible solution: no solution satisfies all constraints. 38 Example: no feasible solution If in the Wyndor Glass Co. problem the profit should be above $50,000 per week: 39 8

Terminology for solutions Optimal solution: solution with the most favorable value of the objective function. Most favorable value: largest valueif maximizingor smallest value if minimizing.

9 Terminology for solutions Optimal solution: solution with the most favorable value of the objective function. Most favorable value: largest valueif maximizingor smallest value if minimizing. No optimal solutions: (1) no feasible solutions or, (2) unbounded Z (see next slide). More than one solution: see next slide. 40 Examples of solutions More than one solution Unbounded cost function 41 9

10 Corner-point feasible solution Solution that lies at a corner of feasible region. Any LP problem with feasible solutions and a bounded feasible region must possess CPFsolutions and at least one optimal solution (the best CPF solution). If a problem has exactly one optimal solution, it must be a CPF solution. 42 Assumptions of linear programming Proportionality: value of objective function Zis proportional to level of activity x j. Violations: 1) Setup costs 43 10

11 Violations of proportionality 2) Increase of marginal returndue to economy of scale (longer production runs, quantity discounts, learning-curve effect, etc.) 44 Violations of proportionality 3) Decrease of marginal returndue to e.g. marketing costs: to sell more the advertisement grows exponentially

12 Assumptions of linear programming Additivity: everyfunction in a linear programming model is the sum of the individual contributions. Divisibility: decision variables can have any values. If values are limited to integer values, the problem needs an integer programming model. Certainty: parameters of the model c j, a ij and b i are assumed to be known constants. In real applications parameters have always some degree of uncertainty, and sensitivity analysismust be conducted. 46 Conclusions Linear programming models can be solved using: Spreadsheet as Excel Solverfor relatively simple problems Problems with thousands (or even millions!) of decision variables and/or functional constraints must use modeling languages such as: CPLEX/MDL or LINGO/LINDO Matlab Optimization Toolbox Linear programming solves many problems. However, when one or more assumptions are violated seriously, other programming techniques are needed

13 SIMPLEX METHOD Simplex method Algebraic procedure with underlying geometric concepts. Definitions: Constraint boundary: line of boundary of the constraint. Corner-point solution Corner-point feasible solution(cfp solution) CFP solutions are adjacentif they share n 1 constraint boundaries. The line segment between two CFP solutions is an edge

14 Example revisited Wyndor Glass Co. 50 Optimality test Optimality test: consider LP with at least one optimal solution. If a CFP solution has no adjacent CFP solutions that are better, it must be an optimal solution. Adjacent CFP solutions for each CFP solution of Wyndor Glass Co. problem: CFP solution Its adjacent CFP solutions (0, 0) (0, 6) and (4, 0) (0, 6) (2, 6) and (0, 0) (2, 6) (4, 3) and (0, 6) (4, 3) (4, 0) and (2, 6) (4, 0) (0, 0) and (4, 3) 51 14

Solving the example Initialization: at CPF solution (0, 0). Optimality test: conclude that (0, 0) is not optimal Iteration 1: 1. Choose to move along the edge with faster rate (x 2 ) 2.

15 Solving the example Initialization: at CPF solution (0, 0). Optimality test: conclude that (0, 0) is not optimal Iteration 1: 1. Choose to move along the edge with faster rate (x 2 ) 2. Stop at new constraint boundary (2x 2 = 12) 3. Determine point of intersection of new set boundaries (0,6) Optimality test: conclude that (0, 6) is not optimal Iteration 2: repeat steps of iteration 1. CFP sol. (2,6) Optimality test: conclude that (2, 6) is optimal. Stop. 52 Steps of simplex algorithm Sequence of CFP solutions: 53 15

16 Setting up the simplex method Original form maximize Z = 3x + 5x subject to x 4 1 2x 12 3x + 2x 18 and x 0, x Augmented form maximize Z = 3x + 5x subject to (1) x + x = 4 (2) 2x + x = 12 (3) 3x + 2x + x = 18 and x 0, j = 1,2,3,4,5. j Convert functional inequalityconstraintsto equivalent equality constraints using slack variables. 54 Setting up the simplex method Slack variable is: Zero: solution lies on constraint boundary; Positive: solution lies on feasible side of constraint; Negative: solution lies on infeasible side of constraint. Augmented solution: solution with original variables (decision variables) and slack variables. Example: (3, 2, 1, 8, 5) Basic solution: augmented corner-point solution. Example(infeasible solution): (4, 6, 0, 0, 6) Basic feasible (BF) solution: augmented corner-point feasible solution

17 Setting up the simplex method Example problem has 5 variables and 3 non redundant equations, and 2 (5 3) degrees of freedom. Two variables have an arbitrary value. Simplex uses the zero value for these nonbasic variables. The solution for the other variables (basic variables) is a basic solution. Basic solutions have thus a number of properties. 56 Properties of a basic solution Each variable is basic or nonbasic. Number of basic variables is equal to number of functional constraints (equations). The nonbasic variables are set to zero. Values of basic variablesare solving system of equations (functional constraints in augmented form). Set of basic variables are called the basis. If basic variables satisfy the nonnegativity constraints basic solution is a BF solution

18 Final form maximize Z subject to (0) Z 3x 5x = (1) x + x = (2) 2x + x = (3) 3x + 2x + x = and x 0, j = 1,,5. j 58 Tabular form (simplex tableau) (a) Algebraic form (b) Tabular Form Basic variable Eq. Coefficient of: Z x 1 x 2 x 3 x 4 x 5 Right side (0) Z 3x 1 5x 2 = 0 Z (0) (1) x 1 + x 3 = 4 x 3 (1) (2) 2x 2 + x 4 = 12 x 4 (2) (3) 3x 1 + 2x 2 + x 5 = 18 x 5 (3)

19 Simplex method in tabular form Initialization. Introduce slack variables. Select decision variablesto be initial nonbasic variablesand slack variables to be initial basic variables. Initial solution of example: (0, 0, 4, 12, 18) Optimality test. Current BF solution is optimal iffevery coefficient in row zero is nonnegative. If yes, stop. Example: coefficients of x 1 and x 2 are 3 and Simplex method in tabular form Iteration. Step 1: Determine entering basic variable, which is the nonbasic variable with the largest negative coefficient. pivot column entering basic variable Basic variable Eq. Coefficient of: Z x 1 x 2 x 3 x 4 x 5 Right side Z (0) x 3 (1) x 4 (2) x 5 (3)

20 Iteration Step 2: Determine leaving basic variable, by applying minimum ratio test: Basic variable Eq. Coefficient of: Z x 1 x 2 x 3 x 4 x 5 Right side Ratio Z (0) pivot row x 3 (1) x 4 (2) /2 = 6 minimum x 5 (3) /2 = 9 pivot number 62 Iteration Step 3: solve for the new BF solutionby using elementary row operations. Basic Coefficient of: Right Iteration Eq. variable Z x side 1 x 2 x 3 x 4 x 5 Z (0) x 3 (1) x 4 (2) x 5 (3) Z (0) x 3 (1) x 2 (2) x 5 (3)

21 Iteration 2: Steps 1 and 2 Iteration Basic variable Eq. Coefficient of: Z x 1 x 2 x 3 x 4 x 5 Right side Ratio Z (0) x 3 (1) /1 = 4 x 2 (2) x 5 (3) /3 = 2 minimum 64 Complete set of simplex tableau Iteration Basic variable Eq. Coefficient of: Z x 1 x 2 x 3 x 4 x 5 Right side Z (0) x 3 (1) x 4 (2) x 5 (3) Z (0) x 3 (1) x 2 (2) x 5 (3) Z (0) x 3 (1) /3-1/3 2 x 2 (2) x 1 (3) /3 1/

22 Adapting to other model forms Equality constraints There is no obvious initial BF solution, so artificial variablesare the Big Mmethodare used. Functional constraints in form Surplus variablesare used. Minimization Maximize Z. Big M method can have two phases: 1) all artificial variables driven to zero; 2) compute BF solutions. 66 Big Mmethod The real problem Maximize Z = 3x + 5x subject to 1 2 x x 12 3x + 2x = 18 and x 0, x The artificial problem Define x = 18 3x 2 x Maximize Z = 3x + 5 x Mx, subject to (1) x 4 (2) 2x 12 (3) 3x + 2x (so 3x + 2x + x = 18) and x 0, x 0, x

23 Postoptimality analysis Reoptimization: when the problem changes slightly, new solution is derived from the final simplex tableau. Shadow pricefor resource i(denoted y i* ) is the marginal value of i. i.e. the rate at which Zcan be increased by slightly increasing the amount of this resource b i. Example: recall b i values of Wyndor Glass problem. The final tableau gives: y 1* = 0, y 2* = 1.5, y 3* = Graphical check Increasing b 2 in 1 unit, the new profit is

24 Shadow prices Increasing b 2 in 1 unit increases the profit in $1500. Should this be done? Depends on the marginal profitability of other products using that hour time. Shadow prices are part of the sensitivity analysis. Constraint on resource 1 is not bindingthe optimal solution. Resources as this are free goods. Constraints on resources 2 and 3 are binding constraints. Resources are scarce goods. 70 Sensitivity analysis Identify sensitive parameters(those that cannot be changed without changing optimal solution). Parameters b i can be analyzed using shadow prices. Parameters c i for Wyndor example (see figure in next slide). Parameters a ij can also be analyzed graphically. For problems with a large number of variables usually only b i and c i are analyzed, because a ij are determined by the technology being used

25 Sensitivity analysis of parameters c i Graphical analysis of Wyndor example: c 1 = 3 can have values in [0, 7.5] without changing optimal c 2 = 5 can have any value greater than 2 without changing optimal 72 Interior-point approach Discovery made in 1984 by Narendra Karmarkar. Used in hugelinear programming problems, beyond the scope of the simplex method. Starts by identifying a feasible trial solution, moving to a better one until it reaches the optimal trial solution. Trial solutions are interior points. Interior-pointor barrier algorithms(each constraint boundary is treated as a barrier)

Example The Wyndor Glass Co. example reaches the solution in 15 iterations for changes smaller than 0.

26 Example The Wyndor Glass Co. example reaches the solution in 15 iterations for changes smaller than Theory of simplex method Recall the standard form: maximize Z = c1x1 + c2x2 + + cnxn subject to a x + a x + + a x b n n 1 a x + a x + + a x b n n 2 a x + a x + + a x b m1 1 m2 2 mn n m and x 0, x 0,, x n

27 Augmented form maximize Z = c1x1 + + cnxn + cn+ 1xn cn+ mxn+ m subject to a x + a x + + a x + x = b n n n+ 1 1 a x + a x + + a x + x = b n n n+ 2 2 a x + a x + + a x + x = b m1 1 m2 2 mn n n+ m m and x 0, j = 1,2,, n + m. j 76 Matrix form (standard form) maximize Z = cx subject to Ax b and x 0 c = [,,, ], c1 c2 c n x1 b1 x = 2 b 2 x, b, = xn bm =, 0 A a a a a a a n n = a a a m1 m2 mn

28 Matrix augmented form Introduce the column vector of slack variables: x s xn+ 1 x = n+ 2 x n+ m The constraints are (with I(m m) and 0(1 (n+m)) ): x x [ A, I] = b and 0 xs xs 78 Solving for a BF solution One iteration of simplex: The nnonbasic variables are set to zero. The mbasic variables is a column vector denoted as x B. The basic matrix Bis obtained from [A, I],by eliminating the columns with coefficients of nonbasic variables. The problem becomes: Bx B = b. The solution for basic variables and the value of the objective function for this basic solution are: x = B b and Z = c x = c B b 1 1 B B B B 79 28

29 Revised simplex method 1. Initialization: (as original) Introduce slack variables x s. Decision variables are initial nonbasic variables. 2. Iteration: Step 1: (as original) Determine entering basic variable (nonbasic variable with the largest negative coefficient). Step 2: Determine leaving basic variable. As original, but computations are simplified. Step 3: Determine new BF solution: set x B = B 1 b. 3. Optimality test: solution is optimal if every coefficient in Z is 0. Computations are simplified. 80 DUALITY THEORY AND SENSITIVITY ANALYSIS 29

30 Duality theory Every linear programming problem (primal) has a dual problem. Most problem parameters are estimates, others (as resource amounts) represent managerial decisions. The choice of parameter values is made based on a sensitivity analysis. Interpretation and implementation of sensitivity analysis are key uses of duality theory. 82 Primal and dual problems Primal problem Dual problem maximize Z subject to n j= 1 n = j= 1 c x a x b, for i = 1,2,, m ij j i and x 0 for j = 1,2,, n. j j j minimize W subject to m i= 1 m = i= 1 b y a y c, for j = 1, 2,, n ij i j and y 0 for i = 1,2,, m. i i i 83 30

31 Duality theory Relations between parameters: The coefficients of the objective functions of the primal are the right-hand sides of the functional constraints in the dual. The right-hand side of the functional constraints in the primal are coefficients of the objective functions in the dual. The coefficients of a variable in the functional constraints of the primal are the coefficients in the functional constraints of the dual. 84 Primal-dual table Primal problem Coefficient of: x 1 x 2 x n Right side Coefficient of: y 1 a 11 a 12 a 1n b 1 y 2 a 21 a 22 a 2n b 2 y m a n1 a n2 a mn b m Right side Coefficients of objective function (minimize) Dual problem c 1 c 2 c n Coefficients of objective function (maximize) 85 31

32 Example: Wyndor Glass Co. Maximize subject to Primal problem x Z = [ ] 1 3 5, x x x x 0 1 and. x 2 0 Dual problem 4 Minimize Z = [ y1 y2 y 3] 12, 18 subject to [ y y y ] 0 2 [ 3 5] [ y y y ] [ ] and Example: Wyndor Glass Co. Tableau representation x 1 x 2 y y y

33 Primal-dual relationships Weak duality property: If xis a feasible solution for the primal problem and yis a feasible solution for the dual problem, then cx yb Strong duality property: If x * is an optimal solution for the primal problem and y * is an optimal solution for the dual problem, then cx * = y * b 88 Primal-dual relationships Complementary solutions property:at each iteration, the simplex method identifies a CFP solution xfor the primal problem and a complementary solutionfor the dual problem, where cx = yb If xis not optimalfor the primal problem, then yis not feasible for the dual problem

34 Primal-dual relationships Complementary optimal solutions property: At the final iteration, the simplex method identifies an optimal solution x*for the primal problem and a complementary optimal solution y*for the dual problem, where cx* = y*b The y i *are the shadow prices for the primal problem. 90 Primal-dual relationships Symmetry property: For anyprimal problem and its dual problem, all relationships between them must be symmetricbecause the dual of this dual problem is this primal problem. Duality theorem: The following are the only possible relationships between the primal and dual problems: 91 34

35 Primal-dual relationships 1. If one problem has feasible solutionsand a bounded objective function, then so does the other problem, so both the weak and strong duality properties are applicable. 2. If one problem has feasible solutionsand an unboundedobjective function, then the other problem has no feasible solution. 3. If one problem has no feasible solutions, then the other problem has either no feasible solutionsor an unbounded objective function. 92 Primal-dual relationships Relationships between complementary basic solutions Primal basic Complementary Both basic solutions solution dual basic solution Primal feasible? Dual feasible? Suboptimal Superoptimal Yes No Optimal Optimal Yes Yes Superoptimal Suboptimal No Yes Neither feasible nor superoptimal Neither feasible nor superoptimal No No 93 35

36 Primal-dual relationships 94 Applications of duality theory If the number of functional constraints mis bigger than the number of variables n, applying simplex directly to the dual problem will achieve a substantial reduction in computational effort. Evaluating a proposed solution for the primal problem. i)if cx = yb, xand ymust be optimal even without applying simplex; ii) if cx < yb, ybprovides an upper bound on the optimal value of Z. Economic interpretation as before 95 36

37 Duality theory in sensitivity analysis Sensitivity analysis investigate the effect of changing parameters a ij, b i, c j in the optimal solution. Can be easier to study these effects in the dual problem Optimal solution of the dual problem are the shadow prices of the primal problem. Changes in coefficients of a nonbasic variable. Only changes one constraint in the dual problem. If this constraint is satisfied the solution is still optimal. Introducing a new variable in the objective function. Only introduces a new constraintin the dual problem. If the dual problem is still feasible, solution is still optimal. 96 Essence of sensitivity analysis LP assumes that all the parameters of the model (a ij, b i and c j ) are known constants. Actually: The parameter values used are just estimates based on a prediction of future conditions; The data may represent deliberate overestimates or underestimates to protect the interests of the estimators. An optimal solution is optimal only with respect to the specific model being used to represent the real problem sensitivity analysis is crucial! 97 37

38 Simplex tableau with parameter changes What changes in the simplex tableau if changes are made in the model parameters, namely b b, c c, A A? New initial tableau Coefficient of: Eq. Z Original variables Slack variables Right side (0) 1 c 0 0 (1,2,,m) 0 A I b Revised final tableau (0) 1 z * c = y * A c y* (1,2,,m) 0 A* = S* A S* Z * = y * b b* = S* b 98 Applying sensitivity analysis Changes in b: Allowable range: range of values for which the current optimal BF solution remains feasible (find range of b i such that b* = S* b 0, assuming this is the only change in the model). The shadow price for b i remains valid if b i stays within this interval. The 100% rule for simultaneous changes in right-hand side: the shadow prices remain valid as long as the changes are not too large. If the sumof the % changes does not exceed 100%, the shadow prices will still be valid

39 Applying sensitivity analysis Changes in coefficients of nonbasic variable: Allowable range to stay optimal: range of values over which the current optimal solution remains optimal (find c j y * A j, assuming that this is the only change in the model). The 100% rule for simultaneous changes in objective function coefficients: if the sumof the % changes does notexceed 100%, the original optimal solution will still be valid. Introduction of a new variable: Same as above. 100 Other simplex algorithms Dual simplex method Modification useful for sensitivity analysis Parametric linear programming Extension for systematic sensitivity analysis Upper bound technique A simplex version for dealing with decision variables having upper bounds: x j u j, where u j is the maximum feasible value of x j

Introduction. Very efficient solution procedure: simplex method.

Introduction. Very efficient solution procedure: simplex method. LINEAR PROGRAMMING Introduction Development of linear programming was among the most important scientific advances of mid 20th cent. Most common type of applications: allocate limited resources to competing