Applications. Stephen J. Stoyan, Maged M. Dessouky*, and Xiaoqing Wang
|
|
- Leo Anderson
- 5 years ago
- Views:
Transcription
1 Introduction to Large-Scale Linear Programming and Applications Stephen J. Stoyan, Maged M. Dessouky*, and Xiaoqing Wang Daniel J. Epstein Department of Industrial and Systems Engineering, University of Southern California, Los Angeles, CA, *Corresponding author ( 1 Introduction Linear Programming has been used as an effective tool to solve a number of applications in manufacturing, transportation, military, healthcare, and finance. In order to accurately model practical systems, however, there may be many variables and constraints necessary to capture all of the attributes in the problem. In many applications, the size of the formulation in terms of the number of variables and constraints can be quite large. For example, Gondzio and Grothey (2006) solve problems with variable sizes up to 10 9 and Leachman (1993) formulates a linear programming model of over 10 5 constraints to solve a production planning problem for semiconductors at Harris Corporation. With such large-size problems, specialized procedures such as column generation and cutting-plane methods may need to be employed to solve them. In some cases, the problem may have a special structure, where decomposition methods can also be used. The purpose of this chapter is to provide an overview of these specialized procedures, and we conclude with an example of a large-scale 1
2 formulation for solving a route balancing problem. For general discussions on the progress made with linear programming, problem size, and algorithmic developments one may refer to [3]. In this chapter we describe some of the fundamental ideas used in large-scale linear programming with an emphasis on the simplex method. However, the reader may also refer to the literature on Interior Point Methods [7, 11]. 2 Linear Programming Problem Consider the following standard Linear Programming (LP) problem: min c x (1) s.t. Ax = b (2) x 0, (3) where x R n, A is an m n matrix, and c and b are vectors of length n and m, respectively. Now suppose the problem is so large, e.g. the A matrix cannot be stored in memory, that the LP cannot be solved. Using the simplex method, large-scale problems can be solved by adding a few extra steps to the algorithm. Depending on the structure of the A matrix it may be possible to decompose the problem such that solving smaller subproblems provide sufficient optimality conditions. Recall from the simplex method, an LP is partitioned into Basic (B) and Nonbasic (N) elements. Such a partitioning on (1) (3) gives: min c Bx B + c Nx N (4) s.t. Bx B + Nx N = b (5) x B, x N 0, (6) where B is an m m matrix composed of the set of linearly independent columns A i i B (A i refers to column i in the A matrix), N is a matrix composed of the remaining set 2
3 of columns A j j N, c B refers to the vector in the objective function containing the elements in B, and c N refers to the elements in N. Also, x B and x N refer to the basic and nonbasic decision variables, respectively. Finally, B 1 exists since the columns of B are linearly independent. Then, solving for x B in equation (5) and making the substitution in (1) provides min c BB 1 b + (c N c BB 1 N)x N. (7) A basis matrix B is then said to be optimal if c N c B B 1 N 0 and B 1 b 0. The first argument is provided in equation (7), if the value in parenthesis is nonnegative then introducing a nonbasic element to B offers no further reduction to the current objective value, which proves the optimality of the solution. Although we will not go through all the details of the algorithm, the simplex method tries to find a set of basic variables that satisfy the optimality criteria. To do so, given an initial basis, reduced costs r j are defined j N where r j = c j c BB 1 A j. (8) The primal simplex method then allows variables to enter and leave the basis while upholding B 1 b 0, and thus, for a given iteration if r j 0 j N the current basis is optimal. This observation is a critical step in many large-scale applications. Given the current basis, one method is to search for a new variable x j j N to enter the basis if its reduced cost is not nonnegative; otherwise, the current solution is optimal. In the next sections we investigate such applications. 3 Column Generation Many practical problems involve large-scale models where there are many more variables than constraints; hence, n >> m. As mentioned in the last section, these instances may 3
4 be quite difficult to solve. For such problems, column generation is an effective solution method. Column generation can also be used for problems where columns may be dynamically generated to form possible solutions. The general idea behind the algorithm is to find the subset of m basic variables that needs to be considered in the solution to the problem since nonbasic variables are equal to zero. To do so, the method forms a subset of the columns in (1) (3) called the master problem, which contains the columns associated with a basic feasible solution to the original problem and possibly a few additional columns. The method then searches for variables that have the potential to reduce the current objective value, which is computed in a subproblem that identifies an entering variable. This implies finding a variable x j j N with a reduced cost r j < 0. The column generation algorithm proceeds as follows: first a master problem is generated involving a subset of the original variables, which contains an initial basis. Then a subproblem is solved to find an entering variable and the corresponding column is added to the master problem. The master problem is then re-solved and this process continues until the subproblem cannot identify an entering variable (i.e. r j 0 j N), in which case, the current solution is optimal. The master problem is defined as: min s.t. c ω x ω (9) ω Ψ A ω x ω = b (10) ω Ψ x 0, (11) where for example Ψ = {i B, j}, which contains all of the current basic columns and a nonbasic column A j. Then a subproblem is introduced that is used to seek a new column with r j < 0 to add to the master problem. Re-solving the master problem will generate a new solution and the process is continued until r j 0 for all variables in the subproblem. We provide an example of a possible subproblem on page 6. One may also note that there are different techniques used to store the number of elements in set Ψ of the master problem. 4
5 In the example described above, the size of Ψ is kept to a minimum of m+1 variables. Other methods never remove any variables that have entered the basis, and some allow Ψ to grow to a certain size before removing variables. Using any of these sizes of Ψ and solving (9) (11) by adding columns in an iterative manner as described above is guaranteed to terminate in the absence of degeneracy since it is simply a special implementation of the revised simplex method. If degeneracy is present in the problem, one can use a method to prevent cycling such as the lexicographic rule. Column generation is also an effective tool at solving problems where instead of defining a large set of decisions, the problem may be more efficiently approached if it is structured to generate candidate solutions (columns) until a better solution cannot be found. Such problems exist in inventory, transportation, and scheduling. In Section 6 we provide a route balancing example where road crews are assigned to clean various areas of Los Angeles based on distance. As will be presented, the problem can be expressed as min e x (12) s.t. Ax = b (13) x 0, (14) where e is a vector of all ones. As mentioned above, matrix A may not involve the set of all possible combinations to solve the initial problem and candidate solutions will be generated to do so. In this case, candidate solutions are equivalent to adding columns A j to the problem. In the route balancing example, finding a candidate solution (column A j ) simply requires finding an area that a crew may be allocated to. Thus, providing an initial basis to (12) (14) and possibly a few additional columns forms the master problem. Suppose that we have an initial basis, then the subproblem will generate an additional candidate solution j with a negative reduced cost, i.e. r j = 1 e B 1 A j < 0. (15) 5
6 Now say that the subproblem generated columns associated with the most negative reduced cost. This is equivalent to taking the column associated with min{r j } = min{1 e B 1 A j }. (16) If the solution to the reduced cost found in (16) is nonnegative, then the current solution is optimal. Notice that solving equation (16) is equivalent to the following inequality: max{e B 1 A j } 1. (17) Thus, the master problem is optimal if the solution to the subproblem satisfies (17). In addition to equation (17), the subproblem also involves constraints that satisfy the original problem such that only feasible solutions are considered. In some cases, the subproblem may simply require solving the knapsack problem. For more information and examples of such issues the reader may refer to [1, 2, 9]. 4 Cutting-Plane Methods In Section 3 we discussed large-scale problems where n >> m. We now examine the case in which the problem involves a large number of constraints. This is equivalent to investigating the dual of the model presented in Section 3. In this case, we define problems where Cutting- Plane methods provide a valid solution approach. Considering the dual of (1) (3): max b y (18) s.t. A y c, (19) where the resulting problem above has many rows or constraints. In order to solve the problem in (18) (19), the cutting-plane method defines a less-constrained problem max b y (20) s.t. A ω y c ω, ω Ω, (21) 6
7 where Ω is a subset of the constraints in the initial problem (i.e. A ω corresponds to a row in (19)). The less-constrained problem in (20) (21) is referred to as the relaxed problem. The cutting-plane algorithm begins by first solving the relaxed problem, then uses a subproblem to check the feasibility of the solution with that of the initial problem in (18) (19). If a constraint is violated in the initial problem, then the violated constraint is added to subset Ω in the relaxed problem and it is re-solved. This continues until the optimal solution of the relaxed problem produces a feasible solution with respect to the initial problem, at which point we have found an optimal solution. Thus, the algorithm continuously solves the relaxed problem with at least one additional constraint being added to Ω after each iteration, until an optimal solution is found that does not violate the initial problem. Given that y Ω is an optimal solution to (20) (21), using the cutting-plane method there are two issues to consider with respect to the optimality of (18) (19), namely (i) If y Ω is a feasible solution to (18) (19), then y D = y Ω ; where y D refers to an optimal solution of (18) (19); (ii) If y Ω is infeasible with respect to (18) (19), then we find the violated constraint(s) and add it to the relaxed problem (20) (21). Thus, given that row A j provides the violation, then set Ω is updated to include row j and the problem is re-solved. Figure 1 is a graphical illustration of cases (i) and (ii) above. Given that the feasible region of the initial problem is A 1, if constraint L 4 is not included in the relaxed problem (20) (21), then the feasible region expands to contain regions A 1 A2. In this case, solving the relaxed problem produces an optimal solution to (18) (19) and thus, y Ω = y D as depicted in the figure. However, if constraint L 2 is not included in the relaxed problem (20) (21), then the feasible region contains A 1 A3. Depending on the slope of the objective function and the rest of the region, the optimal solution may be at vertex yω, as depicted in the figure. In this case, y Ω y D, and violated inequality L 2 would be generated as a cutting plane. 7
8 L 1 y D * A 1 L 4 A 2 L 2 L 3 A 3 * y Figure 1: Illustrative example of issues related to Cutting-Plane methods. The last issue to consider with respect to the cutting-plane algorithm is how to check that the solution is feasible with respect to the initial problem. Also, if the solution is not feasible, an efficient method to identify the violated constraint(s) is necessary. Given that y k is the current solution to the relaxed problem, one method to identify a violated constraint(s) involves solving min l / Ω c l A l y k (22) from the variables in the initial problem (18) (19). This forms the subproblem in the cuttingplane method, and if (22) is nonnegative for all elements l then we have a feasible and therefore, optimal solution. Otherwise, we add a constraint that satisfies the inequality A l y k > c l to the relaxed problem (20) (21). Depending on the problem, solving equation (22) may be challenging. There are techniques that may be employed for such instances; the reader may refer to [1, 2] for more information on this topic. One may also note that the cutting-plane method we describe in this section is different than the method used in integer programming, where constraints (cuts) are added to the problem to form the convex hull of 8
9 the integer feasible set; please refer to [10] for more on this topic. 5 Special Decompositions In addition to the methods mentioned in Sections 3 and 4, many large-scale problems may have special structures that can be further exploited. For example, some problems may contain a sparse A matrix and/or have a unique block-matrix structure. In some cases it is possible to decompose the large-scale system into smaller, more tractable problems. In a simple example, consider the following problem with a block diagonal matrix structure min c 1 x 1 + c 2 x c n x n (23) s.t. D1 x 1 = b 1 (24) D 2 x 2 = b 2 (25)... D n x n = b n (26) x 1, x 2,..., x n 0, (27) where D 1,..., D n represent block matrices, and b 1,..., b n, c 1,..., c n, and x 1,..., x n represent vectors of appropriate dimension. Equations (23) (27) can be easily decomposed into n subproblems involving x 1,x 2,..., x n, and solved independently. The sum of the subproblem solutions will give the optimal value to (23). Now consider problems that have the following 9
10 structure min c 1 x 1 + c 2 x c n x n (28) s.t.  1 x 1 + Â2x Ânx n = b 0 (29) D 1 x 1 = b 1 (30) D 2 x 2 = b 2 (31)... D n x n = b n (32) x 1, x 2,..., x n 0, (33) where Â1,..., Ân also represent block matrices. The constraint structure presented in (28) (33) is quite common for many practical problems. For example, problems in production, finance, and transportation often involve models where some subset of constraints only pertain to particular elements/areas of the problem (e.g. time periods) and others span the whole problem domain (e.g. all time periods). For such instances, Dantzig-Wolfe decomposition is an efficient solution approach [1, 2, 4]. Similar to the methods described earlier, Dantzig-Wolfe decomposition forms a master problem of the system in (28) (33), which contains a basis. Then, the method uses a subproblem to generate admissible columns to add to the master problem, which is re-solved until no further improvement can be made with respect to the objective function. First, the master problem is formed by defining χ i = { D i x i = b i, x i 0} i = 1,..., n, (34) and assuming χ i i = 1,..., n, (28) (33) becomes: n min c i x i (35) s.t. i=1 n  i x i = b 0 (36) i=1 x i χ i, i = 1,..., n. (37) 10
11 Then, using the Representation Theorem [1, 2, 4] we have that any element x i χ i can be expressed as x i = q Q i λ q i vq i + k K i µ k i d k i, (38) where v q i q Q i consist of the set of extreme points in χ i and d k i k K i is the set of extreme rays in χ i ; also note that λ q i 0, µk i 0, and q Q i λ q i = 1 i = 1,..., n. Therefore, from the representation theorem (35) (37) is equivalent to ( ) n min c i (λ q i vq i ) + c i (µ k i d k i ) (39) i=1 q Q i k K ( i ) n s.t. Â i (λ q i vq i ) + Â i (µ k i d k i ) = b 0 (40) i=1 q Q i k K i = 1, i = 1,..., n, (41) q Q i λ q i λ q i 0, i = 1,..., n, q Q i, (42) µ k i 0, i = 1,..., n, k K i, (43) which is the master problem of Dantzig-Wolfe decomposition. One of the key issues with the master problem is that (39) (43) typically contain more variables than the original problem in (28) (33), since a variable is created for every extreme point and extreme ray of each polyhedron χ i i = 1,..., n. This issue can be resolved by using column generation methods (described in Section 3) to facilitate the increased number of variables. For further discussions on the Dantzig-Wolfe algorithm the reader may refer to [1, 2, 4, 9]. Taking the dual of (28) (33) forms a problem that can be solved using Benders decomposition. Such problems describe the structure of a two-stage stochastic linear program and can be found in many practical models where uncertainty is present. Benders decomposition is a Dantzig-Wolfe decomposition algorithm applied to the dual, and hence, the master problem contains many constraints. The key idea in Benders decomposition is to solve a relaxed 11
12 master problem similar to what was done in cutting-plane methods, described in Section 4. For more information on Benders decomposition one may refer to [1, 2, 4, 9]. 6 Route Balancing Example In this section, we present an example of a linear programming formulation to balance the routes for road crews used by street sweepers. Several counties across California have begun to switch from diesel-powered street sweepers to Compressed Natural Gas (CNG) street sweepers in order to comply with Federal and State air quality regulations. However, due to a number reasons, which include slow refueling time at CNG fueling stations and some design characteristics of the CNG sweepers, the productivity of the sweeping operations have decreased with these new sweepers. The goal of the study is to identify new operating policies to improve the productivity of the CNG street sweepers. From a recent analysis of the assigned road miles, it was evident that the workload varied significantly from crew to crew. Hence, a better balance of the assigned miles-to-crews could significantly improve the productivity of the crews. In this section, we present a mathematical model that optimizes the assignment of road miles to crews. Input Parameters: Let, 1, 2,..., n: be the possible road crews with a total of n; r ij : be the maximum possible road miles that crew i can shift to crew j, i = 1, 2,..., n, j = 1, 2,..., n, where r ii = 0; a i : be the original assigned miles of crew i = 1, 2,..., n; u: be the average miles of the region. 12
13 Decision Variables: Let, x ij : be the road miles that crew i shifts to crew j, i = 1, 2,..., n, j = 1, 2,..., n, where x ii = 0; z i : be the deviation between the workload of crew i = 1, 2,..., n and u. Using the variable and parameter definitions above, the road crew linear program is as follows: min s.t. n i=1 z i n x ji + a i j=1 n x ij a i (44) n x ij u z i i = 1, 2,..., n, (45) j=1 j=1 j=1 n x ji + u z i i = 1, 2,..., n, (46) x ij r ij i = 1, 2,..., n, j = 1, 2,..., n, (47) x ij 0 i = 1, 2,..., n, j = 1, 2,..., n. (48) The objective in problem (44) (48) is to minimize the deviation between the assigned miles of each crew and the average miles of the region, which is equivalent to n n n x ji + a i x ij u. (49) i=1 j=1 j=1 Here, variable z i is used to represent the deviation and linear transformations represented by constraints (45) and (46). The objective function is simply the sum of the deviation of all crews. The smaller the objective value, the more balanced the routes are and an objective value of zero means that each route in the region has exactly the same number of miles. The third constraint (47) limits the miles that crew i can shift to another crew. The data establishment of r ij is based on the physical connection between two adjacent crews, since we only allow the reassignment of miles if the two crews are next to each other on the freeway. 13
14 The last constraint (48) is simply a sign restriction on the nonnegativity of variable x ij. The nonnegativity of variable z i has already been ensured by the first and second constraints. Based on the LP in (44) (48), we constructed an example that consists of 37 crews (n = 37) and has miles average for the region (u = miles). The resulting LP model contains 1406 variables and 2812 constraints, which can be easily increased in size. Thus, (44) (48) may be solved using cutting-plane methods or taking the dual and applying column generation. In addition, the problem used pre-assigned miles for road crews, however, using column generation alternative road-mile combinations may be generated; as presented in Section 3. For the given example, Table 1 lists the assigned miles to each crew for the current and optimized solutions, which was solved using CPLEX 9.0. The optimal solution is miles, which is a 29.39% improvement over the current actual practice of miles. 14
15 Current Optimized Current Optimized Crew Solution (miles) Solution (miles) Crew Solution (miles) Solution (miles) Table 1: Crew road miles. 15
16 References [1] Bazaraa, MS, Jarvis, JJ, and Sherali, HD. Linear Programming and Network Flows John Wiley and Sons, Hoboken, NJ. [2] Bertsimas, D and Tsitsiklis, JN. Introduction to Linear Optimization Athena Scientific, Belmont, MA. [3] Bixby, RE. Solving Real-World Linear Programs: A Decade and More of Progress Operations Research, 50(1), [4] Dantzig, GB and Thapa, MN. Linear Programming 2: Theory and Extensions Springer, New York, NY. [5] Gondzio, J and Grothey, A. Direct Solution of Linear Systems of Size 10 9 Arising in Optimization with Interior Point Methods Lecture notes in computer science, [6] Leachman, RC. Modeling Techniques for Automated Production Planning in the Semiconductor Industry, in Optimization in Industry. TA Ciriani and RC Leachman, eds John Wiley and Sons, New York, NY. [7] Roos, C, Terlaky, T and Vial J-P. Interior Point Methods for Linear Optimization Springer, New York, NY. [8] Vanderbei, RJ. Linear Programming: Foundations and Extensions Springer, New York, NY. [9] Winston, WL. Operations Research: Applications and Algorithms Thomson Brooks/Cole, Toronto, ON. [10] Wolsey, LA. Integer Programming John Wiley and Sons, Toronto, ON. 16
17 [11] Ye, Y. Interior Point Algorithms: Theory and Analysis John Wiley and Sons, Toronto, ON. 17
Operations Research Lecture 6: Integer Programming
Operations Research Lecture 6: Integer Programming Notes taken by Kaiquan Xu@Business School, Nanjing University May 12th 2016 1 Integer programming (IP) formulations The integer programming (IP) is the
More informationLinear Programming: Simplex
Linear Programming: Simplex Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. IMA, August 2016 Stephen Wright (UW-Madison) Linear Programming: Simplex IMA, August 2016
More informationBenders Decomposition Methods for Structured Optimization, including Stochastic Optimization
Benders Decomposition Methods for Structured Optimization, including Stochastic Optimization Robert M. Freund April 29, 2004 c 2004 Massachusetts Institute of echnology. 1 1 Block Ladder Structure We consider
More informationIntroduction to optimization
Introduction to optimization Geir Dahl CMA, Dept. of Mathematics and Dept. of Informatics University of Oslo 1 / 24 The plan 1. The basic concepts 2. Some useful tools (linear programming = linear optimization)
More informationYinyu Ye, MS&E, Stanford MS&E310 Lecture Note #06. The Simplex Method
The Simplex Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapters 2.3-2.5, 3.1-3.4) 1 Geometry of Linear
More informationNew Artificial-Free Phase 1 Simplex Method
International Journal of Basic & Applied Sciences IJBAS-IJENS Vol:09 No:10 69 New Artificial-Free Phase 1 Simplex Method Nasiruddin Khan, Syed Inayatullah*, Muhammad Imtiaz and Fozia Hanif Khan Department
More informationIntroduction to Integer Linear Programming
Lecture 7/12/2006 p. 1/30 Introduction to Integer Linear Programming Leo Liberti, Ruslan Sadykov LIX, École Polytechnique liberti@lix.polytechnique.fr sadykov@lix.polytechnique.fr Lecture 7/12/2006 p.
More informationPart 1. The Review of Linear Programming
In the name of God Part 1. The Review of Linear Programming 1.2. Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Basic Feasible Solutions Key to the Algebra of the The Simplex Algorithm
More informationOutline. Relaxation. Outline DMP204 SCHEDULING, TIMETABLING AND ROUTING. 1. Lagrangian Relaxation. Lecture 12 Single Machine Models, Column Generation
Outline DMP204 SCHEDULING, TIMETABLING AND ROUTING 1. Lagrangian Relaxation Lecture 12 Single Machine Models, Column Generation 2. Dantzig-Wolfe Decomposition Dantzig-Wolfe Decomposition Delayed Column
More informationA Review of Linear Programming
A Review of Linear Programming Instructor: Farid Alizadeh IEOR 4600y Spring 2001 February 14, 2001 1 Overview In this note we review the basic properties of linear programming including the primal simplex
More informationAn Integer Cutting-Plane Procedure for the Dantzig-Wolfe Decomposition: Theory
An Integer Cutting-Plane Procedure for the Dantzig-Wolfe Decomposition: Theory by Troels Martin Range Discussion Papers on Business and Economics No. 10/2006 FURTHER INFORMATION Department of Business
More informationSection Notes 9. Midterm 2 Review. Applied Math / Engineering Sciences 121. Week of December 3, 2018
Section Notes 9 Midterm 2 Review Applied Math / Engineering Sciences 121 Week of December 3, 2018 The following list of topics is an overview of the material that was covered in the lectures and sections
More information21. Solve the LP given in Exercise 19 using the big-m method discussed in Exercise 20.
Extra Problems for Chapter 3. Linear Programming Methods 20. (Big-M Method) An alternative to the two-phase method of finding an initial basic feasible solution by minimizing the sum of the artificial
More information56:270 Final Exam - May
@ @ 56:270 Linear Programming @ @ Final Exam - May 4, 1989 @ @ @ @ @ @ @ @ @ @ @ @ @ @ Select any 7 of the 9 problems below: (1.) ANALYSIS OF MPSX OUTPUT: Please refer to the attached materials on the
More information15-780: LinearProgramming
15-780: LinearProgramming J. Zico Kolter February 1-3, 2016 1 Outline Introduction Some linear algebra review Linear programming Simplex algorithm Duality and dual simplex 2 Outline Introduction Some linear
More information1 Review Session. 1.1 Lecture 2
1 Review Session Note: The following lists give an overview of the material that was covered in the lectures and sections. Your TF will go through these lists. If anything is unclear or you have questions
More informationPart 1. The Review of Linear Programming
In the name of God Part 1. The Review of Linear Programming 1.5. Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Formulation of the Dual Problem Primal-Dual Relationship Economic Interpretation
More informationNotes on Dantzig-Wolfe decomposition and column generation
Notes on Dantzig-Wolfe decomposition and column generation Mette Gamst November 11, 2010 1 Introduction This note introduces an exact solution method for mathematical programming problems. The method is
More informationOperations Research Lecture 2: Linear Programming Simplex Method
Operations Research Lecture 2: Linear Programming Simplex Method Notes taken by Kaiquan Xu@Business School, Nanjing University Mar 10th 2016 1 Geometry of LP 1.1 Graphical Representation and Solution Example
More informationSection Notes 9. IP: Cutting Planes. Applied Math 121. Week of April 12, 2010
Section Notes 9 IP: Cutting Planes Applied Math 121 Week of April 12, 2010 Goals for the week understand what a strong formulations is. be familiar with the cutting planes algorithm and the types of cuts
More informationF 1 F 2 Daily Requirement Cost N N N
Chapter 5 DUALITY 5. The Dual Problems Every linear programming problem has associated with it another linear programming problem and that the two problems have such a close relationship that whenever
More informationPart 4. Decomposition Algorithms
In the name of God Part 4. 4.4. Column Generation for the Constrained Shortest Path Problem Spring 2010 Instructor: Dr. Masoud Yaghini Constrained Shortest Path Problem Constrained Shortest Path Problem
More informationA Parametric Simplex Algorithm for Linear Vector Optimization Problems
A Parametric Simplex Algorithm for Linear Vector Optimization Problems Birgit Rudloff Firdevs Ulus Robert Vanderbei July 9, 2015 Abstract In this paper, a parametric simplex algorithm for solving linear
More informationA primal-simplex based Tardos algorithm
A primal-simplex based Tardos algorithm Shinji Mizuno a, Noriyoshi Sukegawa a, and Antoine Deza b a Graduate School of Decision Science and Technology, Tokyo Institute of Technology, 2-12-1-W9-58, Oo-Okayama,
More informationLinear Programming Redux
Linear Programming Redux Jim Bremer May 12, 2008 The purpose of these notes is to review the basics of linear programming and the simplex method in a clear, concise, and comprehensive way. The book contains
More informationBenders Decomposition Methods for Structured Optimization, including Stochastic Optimization
Benders Decomposition Methods for Structured Optimization, including Stochastic Optimization Robert M. Freund May 2, 2001 Block Ladder Structure Basic Model minimize x;y c T x + f T y s:t: Ax = b Bx +
More information1. Algebraic and geometric treatments Consider an LP problem in the standard form. x 0. Solutions to the system of linear equations
The Simplex Method Most textbooks in mathematical optimization, especially linear programming, deal with the simplex method. In this note we study the simplex method. It requires basically elementary linear
More informationLecture 9: Dantzig-Wolfe Decomposition
Lecture 9: Dantzig-Wolfe Decomposition (3 units) Outline Dantzig-Wolfe decomposition Column generation algorithm Relation to Lagrangian dual Branch-and-price method Generated assignment problem and multi-commodity
More informationSensitivity Analysis
Dr. Maddah ENMG 500 /9/07 Sensitivity Analysis Changes in the RHS (b) Consider an optimal LP solution. Suppose that the original RHS (b) is changed from b 0 to b new. In the following, we study the affect
More informationInteger Programming ISE 418. Lecture 16. Dr. Ted Ralphs
Integer Programming ISE 418 Lecture 16 Dr. Ted Ralphs ISE 418 Lecture 16 1 Reading for This Lecture Wolsey, Chapters 10 and 11 Nemhauser and Wolsey Sections II.3.1, II.3.6, II.3.7, II.5.4 CCZ Chapter 8
More informationInteger Programming ISE 418. Lecture 8. Dr. Ted Ralphs
Integer Programming ISE 418 Lecture 8 Dr. Ted Ralphs ISE 418 Lecture 8 1 Reading for This Lecture Wolsey Chapter 2 Nemhauser and Wolsey Sections II.3.1, II.3.6, II.4.1, II.4.2, II.5.4 Duality for Mixed-Integer
More informationMulticommodity Flows and Column Generation
Lecture Notes Multicommodity Flows and Column Generation Marc Pfetsch Zuse Institute Berlin pfetsch@zib.de last change: 2/8/2006 Technische Universität Berlin Fakultät II, Institut für Mathematik WS 2006/07
More informationProjection in Logic, CP, and Optimization
Projection in Logic, CP, and Optimization John Hooker Carnegie Mellon University Workshop on Logic and Search Melbourne, 2017 Projection as a Unifying Concept Projection is a fundamental concept in logic,
More informationRelation of Pure Minimum Cost Flow Model to Linear Programming
Appendix A Page 1 Relation of Pure Minimum Cost Flow Model to Linear Programming The Network Model The network pure minimum cost flow model has m nodes. The external flows given by the vector b with m
More informationLagrangian Relaxation in MIP
Lagrangian Relaxation in MIP Bernard Gendron May 28, 2016 Master Class on Decomposition, CPAIOR2016, Banff, Canada CIRRELT and Département d informatique et de recherche opérationnelle, Université de Montréal,
More information4.5 Simplex method. LP in standard form: min z = c T x s.t. Ax = b
4.5 Simplex method LP in standard form: min z = c T x s.t. Ax = b x 0 George Dantzig (1914-2005) Examine a sequence of basic feasible solutions with non increasing objective function values until an optimal
More informationExtended Formulations, Lagrangian Relaxation, & Column Generation: tackling large scale applications
Extended Formulations, Lagrangian Relaxation, & Column Generation: tackling large scale applications François Vanderbeck University of Bordeaux INRIA Bordeaux-Sud-Ouest part : Defining Extended Formulations
More informationThe dual simplex method with bounds
The dual simplex method with bounds Linear programming basis. Let a linear programming problem be given by min s.t. c T x Ax = b x R n, (P) where we assume A R m n to be full row rank (we will see in the
More information3.10 Lagrangian relaxation
3.10 Lagrangian relaxation Consider a generic ILP problem min {c t x : Ax b, Dx d, x Z n } with integer coefficients. Suppose Dx d are the complicating constraints. Often the linear relaxation and the
More informationMAT016: Optimization
MAT016: Optimization M.El Ghami e-mail: melghami@ii.uib.no URL: http://www.ii.uib.no/ melghami/ March 29, 2011 Outline for today The Simplex method in matrix notation Managing a production facility The
More informationLift-and-Project Inequalities
Lift-and-Project Inequalities Q. Louveaux Abstract The lift-and-project technique is a systematic way to generate valid inequalities for a mixed binary program. The technique is interesting both on the
More informationMATH2070 Optimisation
MATH2070 Optimisation Linear Programming Semester 2, 2012 Lecturer: I.W. Guo Lecture slides courtesy of J.R. Wishart Review The standard Linear Programming (LP) Problem Graphical method of solving LP problem
More informationSection Notes 8. Integer Programming II. Applied Math 121. Week of April 5, expand your knowledge of big M s and logical constraints.
Section Notes 8 Integer Programming II Applied Math 121 Week of April 5, 2010 Goals for the week understand IP relaxations be able to determine the relative strength of formulations understand the branch
More informationAnn-Brith Strömberg. Lecture 4 Linear and Integer Optimization with Applications 1/10
MVE165/MMG631 Linear and Integer Optimization with Applications Lecture 4 Linear programming: degeneracy; unbounded solution; infeasibility; starting solutions Ann-Brith Strömberg 2017 03 28 Lecture 4
More informationInteger programming: an introduction. Alessandro Astolfi
Integer programming: an introduction Alessandro Astolfi Outline Introduction Examples Methods for solving ILP Optimization on graphs LP problems with integer solutions Summary Introduction Integer programming
More informationChap6 Duality Theory and Sensitivity Analysis
Chap6 Duality Theory and Sensitivity Analysis The rationale of duality theory Max 4x 1 + x 2 + 5x 3 + 3x 4 S.T. x 1 x 2 x 3 + 3x 4 1 5x 1 + x 2 + 3x 3 + 8x 4 55 x 1 + 2x 2 + 3x 3 5x 4 3 x 1 ~x 4 0 If we
More informationConvex Analysis 2013 Let f : Q R be a strongly convex function with convexity parameter µ>0, where Q R n is a bounded, closed, convex set, which contains the origin. Let Q =conv(q, Q) andconsiderthefunction
More informationTIM 206 Lecture 3: The Simplex Method
TIM 206 Lecture 3: The Simplex Method Kevin Ross. Scribe: Shane Brennan (2006) September 29, 2011 1 Basic Feasible Solutions Have equation Ax = b contain more columns (variables) than rows (constraints),
More informationIntroduction to Mathematical Programming IE406. Lecture 21. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 21 Dr. Ted Ralphs IE406 Lecture 21 1 Reading for This Lecture Bertsimas Sections 10.2, 10.3, 11.1, 11.2 IE406 Lecture 21 2 Branch and Bound Branch
More informationCO 250 Final Exam Guide
Spring 2017 CO 250 Final Exam Guide TABLE OF CONTENTS richardwu.ca CO 250 Final Exam Guide Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4,
More informationIntroduction to Integer Programming
Lecture 3/3/2006 p. /27 Introduction to Integer Programming Leo Liberti LIX, École Polytechnique liberti@lix.polytechnique.fr Lecture 3/3/2006 p. 2/27 Contents IP formulations and examples Total unimodularity
More informationLecture Notes on Solving Moral-Hazard Problems Using the Dantzig-Wolfe Algorithm
Lecture Notes on Solving Moral-Hazard Problems Using the Dantzig-Wolfe Algorithm Edward Simpson Prescott Prepared for ICE 05, July 2005 1 Outline 1. Why compute? Answer quantitative questions Analyze difficult
More informationOptimization (168) Lecture 7-8-9
Optimization (168) Lecture 7-8-9 Jesús De Loera UC Davis, Mathematics Wednesday, April 2, 2012 1 DEGENERACY IN THE SIMPLEX METHOD 2 DEGENERACY z =2x 1 x 2 + 8x 3 x 4 =1 2x 3 x 5 =3 2x 1 + 4x 2 6x 3 x 6
More informationCOMPUTATIONAL COMPLEXITY OF PARAMETRIC LINEAR PROGRAMMING +
Mathematical Programming 19 (1980) 213-219. North-Holland Publishing Company COMPUTATIONAL COMPLEXITY OF PARAMETRIC LINEAR PROGRAMMING + Katta G. MURTY The University of Michigan, Ann Arbor, MI, U.S.A.
More informationNetwork Flows. 6. Lagrangian Relaxation. Programming. Fall 2010 Instructor: Dr. Masoud Yaghini
In the name of God Network Flows 6. Lagrangian Relaxation 6.3 Lagrangian Relaxation and Integer Programming Fall 2010 Instructor: Dr. Masoud Yaghini Integer Programming Outline Branch-and-Bound Technique
More informationDecomposition methods in optimization
Decomposition methods in optimization I Approach I: I Partition problem constraints into two groups: explicit and implicit I Approach II: I Partition decision variables into two groups: primary and secondary
More informationPart 1. The Review of Linear Programming Introduction
In the name of God Part 1. The Review of Linear Programming 1.1. Spring 2010 Instructor: Dr. Masoud Yaghini Outline The Linear Programming Problem Geometric Solution References The Linear Programming Problem
More informationDM545 Linear and Integer Programming. Lecture 7 Revised Simplex Method. Marco Chiarandini
DM545 Linear and Integer Programming Lecture 7 Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1. 2. 2 Motivation Complexity of single pivot operation
More informationCSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming
CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming January 26, 2018 1 / 38 Liability/asset cash-flow matching problem Recall the formulation of the problem: max w c 1 + p 1 e 1 = 150
More informationAn Adaptive Partition-based Approach for Solving Two-stage Stochastic Programs with Fixed Recourse
An Adaptive Partition-based Approach for Solving Two-stage Stochastic Programs with Fixed Recourse Yongjia Song, James Luedtke Virginia Commonwealth University, Richmond, VA, ysong3@vcu.edu University
More informationLecture 8: Column Generation
Lecture 8: Column Generation (3 units) Outline Cutting stock problem Classical IP formulation Set covering formulation Column generation A dual perspective 1 / 24 Cutting stock problem 2 / 24 Problem description
More informationMath 273a: Optimization The Simplex method
Math 273a: Optimization The Simplex method Instructor: Wotao Yin Department of Mathematics, UCLA Fall 2015 material taken from the textbook Chong-Zak, 4th Ed. Overview: idea and approach If a standard-form
More informationLecture 23 Branch-and-Bound Algorithm. November 3, 2009
Branch-and-Bound Algorithm November 3, 2009 Outline Lecture 23 Modeling aspect: Either-Or requirement Special ILPs: Totally unimodular matrices Branch-and-Bound Algorithm Underlying idea Terminology Formal
More informationRevised Simplex Method
DM545 Linear and Integer Programming Lecture 7 Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1. 2. 2 Motivation Complexity of single pivot operation
More informationInteger Programming ISE 418. Lecture 12. Dr. Ted Ralphs
Integer Programming ISE 418 Lecture 12 Dr. Ted Ralphs ISE 418 Lecture 12 1 Reading for This Lecture Nemhauser and Wolsey Sections II.2.1 Wolsey Chapter 9 ISE 418 Lecture 12 2 Generating Stronger Valid
More information9.1 Linear Programs in canonical form
9.1 Linear Programs in canonical form LP in standard form: max (LP) s.t. where b i R, i = 1,..., m z = j c jx j j a ijx j b i i = 1,..., m x j 0 j = 1,..., n But the Simplex method works only on systems
More informationSimplex method(s) for solving LPs in standard form
Simplex method: outline I The Simplex Method is a family of algorithms for solving LPs in standard form (and their duals) I Goal: identify an optimal basis, as in Definition 3.3 I Versions we will consider:
More informationLP. Lecture 3. Chapter 3: degeneracy. degeneracy example cycling the lexicographic method other pivot rules the fundamental theorem of LP
LP. Lecture 3. Chapter 3: degeneracy. degeneracy example cycling the lexicographic method other pivot rules the fundamental theorem of LP 1 / 23 Repetition the simplex algorithm: sequence of pivots starting
More informationIntroduction to Operations Research
Introduction to Operations Research (Week 4: Linear Programming: More on Simplex and Post-Optimality) José Rui Figueira Instituto Superior Técnico Universidade de Lisboa (figueira@tecnico.ulisboa.pt) March
More informationOverview of course. Introduction to Optimization, DIKU Monday 12 November David Pisinger
Introduction to Optimization, DIKU 007-08 Monday November David Pisinger Lecture What is OR, linear models, standard form, slack form, simplex repetition, graphical interpretation, extreme points, basic
More informationIntroduction to Mathematical Programming IE406. Lecture 13. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 13 Dr. Ted Ralphs IE406 Lecture 13 1 Reading for This Lecture Bertsimas Chapter 5 IE406 Lecture 13 2 Sensitivity Analysis In many real-world problems,
More informationLOWER BOUNDS FOR THE MAXIMUM NUMBER OF SOLUTIONS GENERATED BY THE SIMPLEX METHOD
Journal of the Operations Research Society of Japan Vol 54, No 4, December 2011, pp 191 200 c The Operations Research Society of Japan LOWER BOUNDS FOR THE MAXIMUM NUMBER OF SOLUTIONS GENERATED BY THE
More informationFarkas Lemma, Dual Simplex and Sensitivity Analysis
Summer 2011 Optimization I Lecture 10 Farkas Lemma, Dual Simplex and Sensitivity Analysis 1 Farkas Lemma Theorem 1. Let A R m n, b R m. Then exactly one of the following two alternatives is true: (i) x
More information3.7 Cutting plane methods
3.7 Cutting plane methods Generic ILP problem min{ c t x : x X = {x Z n + : Ax b} } with m n matrix A and n 1 vector b of rationals. According to Meyer s theorem: There exists an ideal formulation: conv(x
More informationDeveloping an Algorithm for LP Preamble to Section 3 (Simplex Method)
Moving from BFS to BFS Developing an Algorithm for LP Preamble to Section (Simplex Method) We consider LP given in standard form and let x 0 be a BFS. Let B ; B ; :::; B m be the columns of A corresponding
More informationLecture 8: Column Generation
Lecture 8: Column Generation (3 units) Outline Cutting stock problem Classical IP formulation Set covering formulation Column generation A dual perspective Vehicle routing problem 1 / 33 Cutting stock
More informationAM 121: Intro to Optimization Models and Methods
AM 121: Intro to Optimization Models and Methods Fall 2017 Lecture 2: Intro to LP, Linear algebra review. Yiling Chen SEAS Lecture 2: Lesson Plan What is an LP? Graphical and algebraic correspondence Problems
More informationOPERATIONS RESEARCH. Linear Programming Problem
OPERATIONS RESEARCH Chapter 1 Linear Programming Problem Prof. Bibhas C. Giri Department of Mathematics Jadavpur University Kolkata, India Email: bcgiri.jumath@gmail.com MODULE - 2: Simplex Method for
More information3. THE SIMPLEX ALGORITHM
Optimization. THE SIMPLEX ALGORITHM DPK Easter Term. Introduction We know that, if a linear programming problem has a finite optimal solution, it has an optimal solution at a basic feasible solution (b.f.s.).
More informationMATHEMATICAL PROGRAMMING I
MATHEMATICAL PROGRAMMING I Books There is no single course text, but there are many useful books, some more mathematical, others written at a more applied level. A selection is as follows: Bazaraa, Jarvis
More informationCritical Reading of Optimization Methods for Logical Inference [1]
Critical Reading of Optimization Methods for Logical Inference [1] Undergraduate Research Internship Department of Management Sciences Fall 2007 Supervisor: Dr. Miguel Anjos UNIVERSITY OF WATERLOO Rajesh
More informationLinear Programming. Murti V. Salapaka Electrical Engineering Department University Of Minnesota, Twin Cities
Linear Programming Murti V Salapaka Electrical Engineering Department University Of Minnesota, Twin Cities murtis@umnedu September 4, 2012 Linear Programming 1 The standard Linear Programming (SLP) problem:
More informationLecture: Algorithms for LP, SOCP and SDP
1/53 Lecture: Algorithms for LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationA DECOMPOSITION PROCEDURE BASED ON APPROXIMATE NEWTON DIRECTIONS
Working Paper 01 09 Departamento de Estadística y Econometría Statistics and Econometrics Series 06 Universidad Carlos III de Madrid January 2001 Calle Madrid, 126 28903 Getafe (Spain) Fax (34) 91 624
More information3. Vector spaces 3.1 Linear dependence and independence 3.2 Basis and dimension. 5. Extreme points and basic feasible solutions
A. LINEAR ALGEBRA. CONVEX SETS 1. Matrices and vectors 1.1 Matrix operations 1.2 The rank of a matrix 2. Systems of linear equations 2.1 Basic solutions 3. Vector spaces 3.1 Linear dependence and independence
More informationStochastic Integer Programming
IE 495 Lecture 20 Stochastic Integer Programming Prof. Jeff Linderoth April 14, 2003 April 14, 2002 Stochastic Programming Lecture 20 Slide 1 Outline Stochastic Integer Programming Integer LShaped Method
More informationDecomposition-based Methods for Large-scale Discrete Optimization p.1
Decomposition-based Methods for Large-scale Discrete Optimization Matthew V Galati Ted K Ralphs Department of Industrial and Systems Engineering Lehigh University, Bethlehem, PA, USA Départment de Mathématiques
More informationDual Basic Solutions. Observation 5.7. Consider LP in standard form with A 2 R m n,rank(a) =m, and dual LP:
Dual Basic Solutions Consider LP in standard form with A 2 R m n,rank(a) =m, and dual LP: Observation 5.7. AbasisB yields min c T x max p T b s.t. A x = b s.t. p T A apple c T x 0 aprimalbasicsolutiongivenbyx
More informationSpring 2017 CO 250 Course Notes TABLE OF CONTENTS. richardwu.ca. CO 250 Course Notes. Introduction to Optimization
Spring 2017 CO 250 Course Notes TABLE OF CONTENTS richardwu.ca CO 250 Course Notes Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4, 2018 Table
More informationSimplex Method for LP (II)
Simplex Method for LP (II) Xiaoxi Li Wuhan University Sept. 27, 2017 (week 4) Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 1 / 31 Organization of this lecture Contents:
More informationMS&E 318 (CME 338) Large-Scale Numerical Optimization
Stanford University, Management Science & Engineering (and ICME) MS&E 318 (CME 338) Large-Scale Numerical Optimization Instructor: Michael Saunders Spring 2015 Notes 4: The Primal Simplex Method 1 Linear
More informationSIMPLEX LIKE (aka REDUCED GRADIENT) METHODS. REDUCED GRADIENT METHOD (Wolfe)
19 SIMPLEX LIKE (aka REDUCED GRADIENT) METHODS The REDUCED GRADIENT algorithm and its variants such as the CONVEX SIMPLEX METHOD (CSM) and the GENERALIZED REDUCED GRADIENT (GRG) algorithm are approximation
More informationΩ R n is called the constraint set or feasible set. x 1
1 Chapter 5 Linear Programming (LP) General constrained optimization problem: minimize subject to f(x) x Ω Ω R n is called the constraint set or feasible set. any point x Ω is called a feasible point We
More informationLinear Programming. Linear Programming I. Lecture 1. Linear Programming. Linear Programming
Linear Programming Linear Programming Lecture Linear programming. Optimize a linear function subject to linear inequalities. (P) max " c j x j n j= n s. t. " a ij x j = b i # i # m j= x j 0 # j # n (P)
More information2.098/6.255/ Optimization Methods Practice True/False Questions
2.098/6.255/15.093 Optimization Methods Practice True/False Questions December 11, 2009 Part I For each one of the statements below, state whether it is true or false. Include a 1-3 line supporting sentence
More informationFoundations of Operations Research
Solved exercises for the course of Foundations of Operations Research Roberto Cordone The dual simplex method Given the following LP problem: maxz = 5x 1 +8x 2 x 1 +x 2 6 5x 1 +9x 2 45 x 1,x 2 0 1. solve
More informationDistributed Real-Time Control Systems. Lecture Distributed Control Linear Programming
Distributed Real-Time Control Systems Lecture 13-14 Distributed Control Linear Programming 1 Linear Programs Optimize a linear function subject to a set of linear (affine) constraints. Many problems can
More informationComplexity of linear programming: outline
Complexity of linear programming: outline I Assessing computational e ciency of algorithms I Computational e ciency of the Simplex method I Ellipsoid algorithm for LP and its computational e ciency IOE
More informationColumn Generation. ORLAB - Operations Research Laboratory. Stefano Gualandi. June 14, Politecnico di Milano, Italy
ORLAB - Operations Research Laboratory Politecnico di Milano, Italy June 14, 2011 Cutting Stock Problem (from wikipedia) Imagine that you work in a paper mill and you have a number of rolls of paper of
More informationAdvanced linear programming
Advanced linear programming http://www.staff.science.uu.nl/~akker103/alp/ Chapter 10: Integer linear programming models Marjan van den Akker 1 Intro. Marjan van den Akker Master Mathematics TU/e PhD Mathematics
More information