Introduction to integer programming II

Size: px
Start display at page:

Download "Introduction to integer programming II"

Transcription

1 Introduction to integer programming II Martin Branda Charles University in Prague Faculty of Mathematics and Physics Department of Probability and Mathematical Statistics Computational Aspects of Optimization Martin Branda (KPMS MFF UK) / 39

2 Obsah Introduction to computational complexity 1 Introduction to computational complexity 2 3 Duality 4 Dynamic programming Martin Branda (KPMS MFF UK) / 39

3 Introduction to computational complexity Introduction to complexity theory Wolsey (1998): Consider decision problems having YES NO answers. Optimization problem max x M ct x can be replaced by (for some k integral) Is there an x M with value c T x k? For a problem instance X, the length of the input L(X ) is the length of the binary representation of a standard representation of the instance. Instance X = {c, M}, X = {c, M, k}. Martin Branda (KPMS MFF UK) / 39

4 Introduction to computational complexity Example: Knapsack decision problem For an instance { n } n X = c i x i k, a i x i b, x {0, 1} n, the length of the input is i=1 i=1 L(X ) = n n log c i + log a i + log b + log k i=1 i=1 Martin Branda (KPMS MFF UK) / 39

5 Introduction to computational complexity Running time Definition f A (X ) is the number of elementary calculations required to run the algorithm A on the instance X P. Running time of the algorithm A f A (l) = sup{f A (X ) : L(X ) = l}. X An algorithm A is polynomial for a problem P if f A (l) = O(l p ) for some p N. Martin Branda (KPMS MFF UK) / 39

6 Introduction to computational complexity Classes N P and P Definition N P (Nondeterministic Polynomial) is the class of decision problems with the property that: for any instance for which the answer is YES, there is a polynomial proof of the YES. P is the class of decision problems in N P for which there exists a polynomial algorithm. N P may be equivalently defined as the set of decision problems that can be solved in polynomial time on a non-deterministic Turing machine 1. 1 NTM writes symbols one at a time on an endless tape by strictly following a set of rules. It determines what action it should perform next according to its internal state and what symbol it currently sees. It may have a set of rules that prescribes more than one action for a given situation. The machine branches into many copies, each of which follows one of the possible transitions leading to a computation tree. Martin Branda (KPMS MFF UK) / 39

7 Introduction to computational complexity Alan Turing The Imitation Game (2014) Martin Branda (KPMS MFF UK) / 39

8 Introduction to computational complexity Polynomial reduction and the class N PC Definition If problems P, Q N P, and if an instance of P can be converted in polynomial time to an instance of Q, then P is polynomially reducible to Q. N PC, the class of N P-complete problems, is the subset of problems P N P such that for all Q N P, Q is polynomially reducible to P. Proposition: Suppose that problems P, Q N P. If Q P and P is polynomially reducible to Q, then P P. If P N PC and P is polynomially reducible to Q, then Q N PC. Proposition: If P N PC, then P = N PC. Martin Branda (KPMS MFF UK) / 39

9 Introduction to computational complexity Open question & Euler diagram Is P = N P? Martin Branda (KPMS MFF UK) / 39

10 Introduction to computational complexity N P-hard optimization problems Definition An optimization problem for which the decision problem lies in N PC is called N P-hard. Martin Branda (KPMS MFF UK) / 39

11 Introduction to computational complexity Simplex algorithm Klee Minty (1972) example: n max 10 n j x j j=1 i 1 s.t i j x j + x i 100 i 1, i = 1,..., n, j=1 x j 0, j = 1,..., n. (1) Can be easily reformulated in the standard form. The Simplex algorithm takes 2 n 1 pivot steps, i.e. it is not polynomial in the worst case. Martin Branda (KPMS MFF UK) / 39

12 Obsah 1 Introduction to computational complexity 2 3 Duality 4 Dynamic programming Martin Branda (KPMS MFF UK) / 39

13 Basic idea: DIVIDE AND RULE Let M = M 1 M 2 M r be a partitioning of thefeasibility set and let f j = min x M j f (x). Then min f (x) = min f j. x M j=1,...,r Martin Branda (KPMS MFF UK) / 39

14 General principles: Solve LP problem without integrality only. Branch using additional constraints on integrality: x i x i, x i x i + 1. Cut inperspective branches before solving (using bounds on the optimal value). Martin Branda (KPMS MFF UK) / 39

15 General principles: Solve only LP problems with relaxed integrality. Branching: if an optimal solution is not integral, e.g. ˆx i, create and save two new problems with constraints x i ˆx i, x i ˆx i. Bounding ( different cutting): save the objective value of the best integral solution and cut all problems in the queue created from the problems with higher optimal values 2. Exact algorithm.. 2 Branching cannot improve it. Martin Branda (KPMS MFF UK) / 39

16 P. Pedegral (2004). Introduction to optimization, Springer-Verlag, New York. Martin Branda (KPMS MFF UK) / 39

17 0. f min =, x min =, list of problems P = Solve LP-relaxed problem and obtain f, x. If the solution is integral, STOP. If the problem is infeasible or unbounded, STOP. 1. BRANCHING: There is xi for some k N: basic non-integral variable such that k < x i < k + 1 Add constraint x i k to previous problem and put it into list P. Add constraint x i k + 1 to previous problem and put it into list P. 2. Take problem from P and solve it: f, x. 3. If f < f min and x is non-integral, GO TO 1. BOUNDING: If f < f min a x is integral, set f min = f a x min = x, GO TO 4. BOUNDING: If f f min, GO TO 4. Problem is infeasible, GO TO If P, GO TO 2. If P = a f min =, integral solution does not exist. If P = a f min <, optimal value and solution are f min, x min. Martin Branda (KPMS MFF UK) / 39

18 Better... 2./3. Take problem from list P and solve it: f, x. If for the optimal value of the current problem holds f f min, then the branching is not necessary, since by solving the problems with added branching constraints we can only increase the optimal value and obtain the same f min. Martin Branda (KPMS MFF UK) / 39

19 Algorithmic issues: Problem selection from the list P: FIFO, LIFO (depth-first search), problem with the smallest f. Selection of the branching variable xi : the highest/smallest violation of integrality OR the highest/smallest coefficient in the objective function. Martin Branda (KPMS MFF UK) / 39

20 B&B Example I min 4x 1 + 5x 2 x 1 + 4x 2 5, 3x 1 + 2x 2 7, x 1, x 2 Z +. After two iterations of the dual SIMPLEX algorithm x 1 x 2 x 3 x 4 5 x 2 8/ /10 1/10 4 x 1 18/ /10-4/10 112/ /10-11/10 Martin Branda (KPMS MFF UK) / 39

21 B&B Example I Branching means adding a cut of the form x 1 1, i.e. (α = (1, 0, 0, 0, 1), α B = (1, 0)) x 1 + x 5 = 1, x x 1 x 2 x 3 x 4 x 5 5 x 2 8/ /10 1/ x 1 18/ /10-4/ x 5-8/ /10 4/ / /10-11/10 0 Dual feasible, primal infeasible run the dual simplex... Martin Branda (KPMS MFF UK) / 39

22 x 1 1 node 1 x 1 = 1 x 2 = 2 ˆf = 14 node 0 x 1 = 1.8 x 2 = 0.8 ˆf = 11.2 x 1 2 x 2 0 node 3 x 1 = 5 x 2 = 0 ˆf = 20 node 2 x 1 = 2 x 2 = 0.75 ˆf = x 2 1 node 4 x 1 = 2 x 2 = 1 ˆf = 13 Martin Branda (KPMS MFF UK) / 39

23 B&B Example II max 23x x x x x 5 s.t. 8x 1 + 7x x 3 + 6x x 5 25, x 1, x 2, x 3, x 4, x 5 {0, 1}. Martin Branda (KPMS MFF UK) / 39

24 B&B Example II x 3 = 0 node 1 ˆf = branching continues... node 0 ˆf = x 3 = 1 x 2 = 0 node 3 ˆf = 65 integral node 2 ˆf = x 2 = 1 x 1 = 0 node 5 ˆf = pruned node 4 ˆf = x 1 = 1 node 6 ˆf = Martin Branda (KPMS MFF UK) / 39

25 remarks If you are able to get a feasible solution quickly, deliver it to the software (solver). Branch-and-Cut: add cuts at the beginning/during B&B. Algorithm termination: (Relative) difference between a lower and an upper bound construct the upper bound (for minimization) using a feasible solution, lower bound... Martin Branda (KPMS MFF UK) / 39

26 Obsah Duality 1 Introduction to computational complexity 2 3 Duality 4 Dynamic programming Martin Branda (KPMS MFF UK) / 39

27 Duality Duality Set S(b) = {x Z n + : Ax = b} and define the value function A dual function F : R m R A general form of dual problem z(b) = min x S(b) ct x. (2) F (b) z(b), b R m. (3) max F {F (b) : s.t. F (b) z(b), b Rm, F : R m R}. (4) We call F a weak dual function if it is feasible, and strong dual if moreover F (b) = z(b). Martin Branda (KPMS MFF UK) / 39

28 Duality Duality A function F is subadditive over a domain Θ if for all θ 1 + θ 2, θ 1, θ 2 Θ. F (θ 1 + θ 2 ) F (θ 1 ) + F (θ 2 ) The value function z is subadditive over {b : S(b) }, since the sum of optimal x s is feasible for the problem with b 1 + b 2 r.h.s., i.e. ˆx 1 + ˆx 2 S(b 1 + b 2 ). Martin Branda (KPMS MFF UK) / 39

29 Duality Duality If F is subadditive, then condition F (Ax) c T x for x Z n + is equivalent to F (a j ) c j, j = 1,..., m. This is true since F (Ae j ) c T e j is the same as F (a j ) c j. On the other hand, if F is subadditive and F (a j ) c j, j = 1,..., m imply F (Ax) m F (a j )x j j=1 m c j x j = c T x. j=1 Martin Branda (KPMS MFF UK) / 39

30 Duality Duality If we set Γ m = {F : R m R, F (0) = 0, F subadditive}, then we can write a subadditive dual independent of x: max F {F (b) : s.t. F (a j) c j, F Γ m }. (5) Weak and strong duality holds. An easy feasible solution based on LP duality (= weak dual) F LP (b) = max b T y s.t. A T y c. (6) y Martin Branda (KPMS MFF UK) / 39

31 Duality Duality Complementary slackness condition: if ˆx is an optimal solution for IP, and ˆF is an optimal subadditive dual solution, then ( ˆF (a j ) c j )ˆx j = 0, j = 1,..., m. Martin Branda (KPMS MFF UK) / 39

32 Obsah Dynamic programming 1 Introduction to computational complexity 2 3 Duality 4 Dynamic programming Martin Branda (KPMS MFF UK) / 39

33 Dynamic programming Knapsack problem max s.t. n c i x i i=1 n a i x i b, i=1 x i {0, 1}. Martin Branda (KPMS MFF UK) / 39

34 Dynamic programming Dynamic programming Let a i, b be positive integers. r f r (λ) = max c i x i i=1 r s.t. a i x i λ, i=1 x i {0, 1}. If ˆx r = 0, then f r (λ) = f r 1 (λ), ˆx r = 1 then f r (λ) = c r + f r 1 (λ a r ). Thus we arrive at the recursion f r (λ) = max {f r 1 (λ), c r + f r 1 (λ a r )}. Martin Branda (KPMS MFF UK) / 39

35 Dynamic programming Dynamic programming 0. Start with f 1 (λ) = 0 for 0 λ < a 1 and f 1 (λ) = max{0, c 1 } for λ a Use the forward recursion f r (λ) = max {f r 1 (λ), c r + f r 1 (λ a r )}. to successively calculate f 2,..., f n for all λ {0, 1,..., b}; f n (b) is the optimal value. 2. Keep indicator p r (λ) = 0 if f r (λ) = f r 1 (λ), and p r (λ) = 1 otherwise. 3. Obtain the optimal solution by a backward recursion: if p n (b) = 0 then set ˆx n = 0 and continue with f n 1 (b), else (if p n (b) = 1) set ˆx n = 1 and continue with f n 1 (b a n )... Martin Branda (KPMS MFF UK) / 39

36 Dynamic programming Knapsack problem Values a 1 = 4, a 2 = 6, a 3 = 7, costs c 1 = 4, c 2 = 5, c 3 = 11, budget b = 10: max s.t. 3 c i x i i=1 3 a i x i 10, i=1 x i {0, 1}. Martin Branda (KPMS MFF UK) / 39

37 Dynamic programming Knapsack problem Dynamic programming a 1 = 4, a 2 = 6, a 3 = 7, c 1 = 4, c 2 = 5, c 3 = 11 r/ λ f r p r Martin Branda (KPMS MFF UK) / 39

38 Dynamic programming Dynamic programming Other successful applications Uncapacitated lot-sizing problem Shortest path problem... Martin Branda (KPMS MFF UK) / 39

39 Literature Dynamic programming V. Klee, G.J. Minty, (1972). How good is the simplex algorithm?. In Shisha, Oved. Inequalities III (Proceedings of the Third Symposium on Inequalities held at the University of California, Los Angeles, Calif., September 1 9, 1969). New York-London: Academic Press, G.L. Nemhauser, L.A. Wolsey (1989). Integer Programming. Chapter VI in Handbooks in OR & MS, Vol. 1, G.L. Nemhauser et al. Eds. L.A. Wolsey (1998). Integer Programming. Wiley, New York. L.A. Wolsey, G.L. Nemhauser (1999). Integer and Combinatorial Optimization. Wiley, New York. Northwestern University Open Text Book on Process Optimization, available online: [ ] Martin Branda (KPMS MFF UK) / 39

Introduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs

Introduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.1-4.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following

More information

Section Notes 9. IP: Cutting Planes. Applied Math 121. Week of April 12, 2010

Section Notes 9. IP: Cutting Planes. Applied Math 121. Week of April 12, 2010 Section Notes 9 IP: Cutting Planes Applied Math 121 Week of April 12, 2010 Goals for the week understand what a strong formulations is. be familiar with the cutting planes algorithm and the types of cuts

More information

Introduction to integer programming III:

Introduction to integer programming III: Introduction to integer programming III: Network Flow, Interval Scheduling, and Vehicle Routing Problems Martin Branda Charles University in Prague Faculty of Mathematics and Physics Department of Probability

More information

Totally unimodular matrices. Introduction to integer programming III: Network Flow, Interval Scheduling, and Vehicle Routing Problems

Totally unimodular matrices. Introduction to integer programming III: Network Flow, Interval Scheduling, and Vehicle Routing Problems Totally unimodular matrices Introduction to integer programming III: Network Flow, Interval Scheduling, and Vehicle Routing Problems Martin Branda Charles University in Prague Faculty of Mathematics and

More information

Review Solutions, Exam 2, Operations Research

Review Solutions, Exam 2, Operations Research Review Solutions, Exam 2, Operations Research 1. Prove the weak duality theorem: For any x feasible for the primal and y feasible for the dual, then... HINT: Consider the quantity y T Ax. SOLUTION: To

More information

Linear and Integer Programming - ideas

Linear and Integer Programming - ideas Linear and Integer Programming - ideas Paweł Zieliński Institute of Mathematics and Computer Science, Wrocław University of Technology, Poland http://www.im.pwr.wroc.pl/ pziel/ Toulouse, France 2012 Literature

More information

(P ) Minimize 4x 1 + 6x 2 + 5x 3 s.t. 2x 1 3x 3 3 3x 2 2x 3 6

(P ) Minimize 4x 1 + 6x 2 + 5x 3 s.t. 2x 1 3x 3 3 3x 2 2x 3 6 The exam is three hours long and consists of 4 exercises. The exam is graded on a scale 0-25 points, and the points assigned to each question are indicated in parenthesis within the text. Problem 1 Consider

More information

3.10 Lagrangian relaxation

3.10 Lagrangian relaxation 3.10 Lagrangian relaxation Consider a generic ILP problem min {c t x : Ax b, Dx d, x Z n } with integer coefficients. Suppose Dx d are the complicating constraints. Often the linear relaxation and the

More information

min 4x 1 5x 2 + 3x 3 s.t. x 1 + 2x 2 + x 3 = 10 x 1 x 2 6 x 1 + 3x 2 + x 3 14

min 4x 1 5x 2 + 3x 3 s.t. x 1 + 2x 2 + x 3 = 10 x 1 x 2 6 x 1 + 3x 2 + x 3 14 The exam is three hours long and consists of 4 exercises. The exam is graded on a scale 0-25 points, and the points assigned to each question are indicated in parenthesis within the text. If necessary,

More information

Integer Programming Duality

Integer Programming Duality Integer Programming Duality M. Guzelsoy T. K. Ralphs July, 2010 1 Introduction This article describes what is known about duality for integer programs. It is perhaps surprising that many of the results

More information

Numerical Optimization

Numerical Optimization Linear Programming - Interior Point Methods Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on Example 1 Computational Complexity of Simplex Algorithm

More information

Lecture 5: Computational Complexity

Lecture 5: Computational Complexity Lecture 5: Computational Complexity (3 units) Outline Computational complexity Decision problem, Classes N P and P. Polynomial reduction and Class N PC P = N P or P = N P? 1 / 22 The Goal of Computational

More information

to work with) can be solved by solving their LP relaxations with the Simplex method I Cutting plane algorithms, e.g., Gomory s fractional cutting

to work with) can be solved by solving their LP relaxations with the Simplex method I Cutting plane algorithms, e.g., Gomory s fractional cutting Summary so far z =max{c T x : Ax apple b, x 2 Z n +} I Modeling with IP (and MIP, and BIP) problems I Formulation for a discrete set that is a feasible region of an IP I Alternative formulations for the

More information

The Primal-Dual Algorithm P&S Chapter 5 Last Revised October 30, 2006

The Primal-Dual Algorithm P&S Chapter 5 Last Revised October 30, 2006 The Primal-Dual Algorithm P&S Chapter 5 Last Revised October 30, 2006 1 Simplex solves LP by starting at a Basic Feasible Solution (BFS) and moving from BFS to BFS, always improving the objective function,

More information

Computational Complexity. IE 496 Lecture 6. Dr. Ted Ralphs

Computational Complexity. IE 496 Lecture 6. Dr. Ted Ralphs Computational Complexity IE 496 Lecture 6 Dr. Ted Ralphs IE496 Lecture 6 1 Reading for This Lecture N&W Sections I.5.1 and I.5.2 Wolsey Chapter 6 Kozen Lectures 21-25 IE496 Lecture 6 2 Introduction to

More information

LOWER BOUNDS FOR THE MAXIMUM NUMBER OF SOLUTIONS GENERATED BY THE SIMPLEX METHOD

LOWER BOUNDS FOR THE MAXIMUM NUMBER OF SOLUTIONS GENERATED BY THE SIMPLEX METHOD Journal of the Operations Research Society of Japan Vol 54, No 4, December 2011, pp 191 200 c The Operations Research Society of Japan LOWER BOUNDS FOR THE MAXIMUM NUMBER OF SOLUTIONS GENERATED BY THE

More information

Integer Programming ISE 418. Lecture 8. Dr. Ted Ralphs

Integer Programming ISE 418. Lecture 8. Dr. Ted Ralphs Integer Programming ISE 418 Lecture 8 Dr. Ted Ralphs ISE 418 Lecture 8 1 Reading for This Lecture Wolsey Chapter 2 Nemhauser and Wolsey Sections II.3.1, II.3.6, II.4.1, II.4.2, II.5.4 Duality for Mixed-Integer

More information

Introduction into Vehicle Routing Problems and other basic mixed-integer problems

Introduction into Vehicle Routing Problems and other basic mixed-integer problems Introduction into Vehicle Routing Problems and other basic mixed-integer problems Martin Branda Charles University in Prague Faculty of Mathematics and Physics Department of Probability and Mathematical

More information

Algorithms for nonlinear programming problems II

Algorithms for nonlinear programming problems II Algorithms for nonlinear programming problems II Martin Branda Charles University in Prague Faculty of Mathematics and Physics Department of Probability and Mathematical Statistics Computational Aspects

More information

On the Number of Solutions Generated by the Simplex Method for LP

On the Number of Solutions Generated by the Simplex Method for LP Workshop 1 on Large Scale Conic Optimization IMS (NUS) On the Number of Solutions Generated by the Simplex Method for LP Tomonari Kitahara and Shinji Mizuno Tokyo Institute of Technology November 19 23,

More information

Section Notes 9. Midterm 2 Review. Applied Math / Engineering Sciences 121. Week of December 3, 2018

Section Notes 9. Midterm 2 Review. Applied Math / Engineering Sciences 121. Week of December 3, 2018 Section Notes 9 Midterm 2 Review Applied Math / Engineering Sciences 121 Week of December 3, 2018 The following list of topics is an overview of the material that was covered in the lectures and sections

More information

IP Duality. Menal Guzelsoy. Seminar Series, /21-07/28-08/04-08/11. Department of Industrial and Systems Engineering Lehigh University

IP Duality. Menal Guzelsoy. Seminar Series, /21-07/28-08/04-08/11. Department of Industrial and Systems Engineering Lehigh University IP Duality Department of Industrial and Systems Engineering Lehigh University COR@L Seminar Series, 2005 07/21-07/28-08/04-08/11 Outline Duality Theorem 1 Duality Theorem Introduction Optimality Conditions

More information

Chapter 1 Linear Programming. Paragraph 5 Duality

Chapter 1 Linear Programming. Paragraph 5 Duality Chapter 1 Linear Programming Paragraph 5 Duality What we did so far We developed the 2-Phase Simplex Algorithm: Hop (reasonably) from basic solution (bs) to bs until you find a basic feasible solution

More information

Lecture 23 Branch-and-Bound Algorithm. November 3, 2009

Lecture 23 Branch-and-Bound Algorithm. November 3, 2009 Branch-and-Bound Algorithm November 3, 2009 Outline Lecture 23 Modeling aspect: Either-Or requirement Special ILPs: Totally unimodular matrices Branch-and-Bound Algorithm Underlying idea Terminology Formal

More information

Relation of Pure Minimum Cost Flow Model to Linear Programming

Relation of Pure Minimum Cost Flow Model to Linear Programming Appendix A Page 1 Relation of Pure Minimum Cost Flow Model to Linear Programming The Network Model The network pure minimum cost flow model has m nodes. The external flows given by the vector b with m

More information

Network Flows. 6. Lagrangian Relaxation. Programming. Fall 2010 Instructor: Dr. Masoud Yaghini

Network Flows. 6. Lagrangian Relaxation. Programming. Fall 2010 Instructor: Dr. Masoud Yaghini In the name of God Network Flows 6. Lagrangian Relaxation 6.3 Lagrangian Relaxation and Integer Programming Fall 2010 Instructor: Dr. Masoud Yaghini Integer Programming Outline Branch-and-Bound Technique

More information

is called an integer programming (IP) problem. model is called a mixed integer programming (MIP)

is called an integer programming (IP) problem. model is called a mixed integer programming (MIP) INTEGER PROGRAMMING Integer Programming g In many problems the decision variables must have integer values. Example: assign people, machines, and vehicles to activities in integer quantities. If this is

More information

On the number of distinct solutions generated by the simplex method for LP

On the number of distinct solutions generated by the simplex method for LP Retrospective Workshop Fields Institute Toronto, Ontario, Canada On the number of distinct solutions generated by the simplex method for LP Tomonari Kitahara and Shinji Mizuno Tokyo Institute of Technology

More information

Section Notes 8. Integer Programming II. Applied Math 121. Week of April 5, expand your knowledge of big M s and logical constraints.

Section Notes 8. Integer Programming II. Applied Math 121. Week of April 5, expand your knowledge of big M s and logical constraints. Section Notes 8 Integer Programming II Applied Math 121 Week of April 5, 2010 Goals for the week understand IP relaxations be able to determine the relative strength of formulations understand the branch

More information

Lecture 11: Post-Optimal Analysis. September 23, 2009

Lecture 11: Post-Optimal Analysis. September 23, 2009 Lecture : Post-Optimal Analysis September 23, 2009 Today Lecture Dual-Simplex Algorithm Post-Optimal Analysis Chapters 4.4 and 4.5. IE 30/GE 330 Lecture Dual Simplex Method The dual simplex method will

More information

An upper bound for the number of different solutions generated by the primal simplex method with any selection rule of entering variables

An upper bound for the number of different solutions generated by the primal simplex method with any selection rule of entering variables An upper bound for the number of different solutions generated by the primal simplex method with any selection rule of entering variables Tomonari Kitahara and Shinji Mizuno February 2012 Abstract Kitahara

More information

Part 4. Decomposition Algorithms

Part 4. Decomposition Algorithms In the name of God Part 4. 4.4. Column Generation for the Constrained Shortest Path Problem Spring 2010 Instructor: Dr. Masoud Yaghini Constrained Shortest Path Problem Constrained Shortest Path Problem

More information

Properties of a Simple Variant of Klee-Minty s LP and Their Proof

Properties of a Simple Variant of Klee-Minty s LP and Their Proof Properties of a Simple Variant of Klee-Minty s LP and Their Proof Tomonari Kitahara and Shinji Mizuno December 28, 2 Abstract Kitahara and Mizuno (2) presents a simple variant of Klee- Minty s LP, which

More information

Advanced Linear Programming: The Exercises

Advanced Linear Programming: The Exercises Advanced Linear Programming: The Exercises The answers are sometimes not written out completely. 1.5 a) min c T x + d T y Ax + By b y = x (1) First reformulation, using z smallest number satisfying x z

More information

On the Polyhedral Structure of a Multi Item Production Planning Model with Setup Times

On the Polyhedral Structure of a Multi Item Production Planning Model with Setup Times CORE DISCUSSION PAPER 2000/52 On the Polyhedral Structure of a Multi Item Production Planning Model with Setup Times Andrew J. Miller 1, George L. Nemhauser 2, and Martin W.P. Savelsbergh 2 November 2000

More information

Math Models of OR: Branch-and-Bound

Math Models of OR: Branch-and-Bound Math Models of OR: Branch-and-Bound John E. Mitchell Department of Mathematical Sciences RPI, Troy, NY 12180 USA November 2018 Mitchell Branch-and-Bound 1 / 15 Branch-and-Bound Outline 1 Branch-and-Bound

More information

The Simplex and Policy Iteration Methods are Strongly Polynomial for the Markov Decision Problem with Fixed Discount

The Simplex and Policy Iteration Methods are Strongly Polynomial for the Markov Decision Problem with Fixed Discount The Simplex and Policy Iteration Methods are Strongly Polynomial for the Markov Decision Problem with Fixed Discount Yinyu Ye Department of Management Science and Engineering and Institute of Computational

More information

BBM402-Lecture 20: LP Duality

BBM402-Lecture 20: LP Duality BBM402-Lecture 20: LP Duality Lecturer: Lale Özkahya Resources for the presentation: https://courses.engr.illinois.edu/cs473/fa2016/lectures.html An easy LP? which is compact form for max cx subject to

More information

Algorithms for nonlinear programming problems II

Algorithms for nonlinear programming problems II Algorithms for nonlinear programming problems II Martin Branda Charles University Faculty of Mathematics and Physics Department of Probability and Mathematical Statistics Computational Aspects of Optimization

More information

Chapter 7 Network Flow Problems, I

Chapter 7 Network Flow Problems, I Chapter 7 Network Flow Problems, I Network flow problems are the most frequently solved linear programming problems. They include as special cases, the assignment, transportation, maximum flow, and shortest

More information

Week Cuts, Branch & Bound, and Lagrangean Relaxation

Week Cuts, Branch & Bound, and Lagrangean Relaxation Week 11 1 Integer Linear Programming This week we will discuss solution methods for solving integer linear programming problems. I will skip the part on complexity theory, Section 11.8, although this is

More information

In the original knapsack problem, the value of the contents of the knapsack is maximized subject to a single capacity constraint, for example weight.

In the original knapsack problem, the value of the contents of the knapsack is maximized subject to a single capacity constraint, for example weight. In the original knapsack problem, the value of the contents of the knapsack is maximized subject to a single capacity constraint, for example weight. In the multi-dimensional knapsack problem, additional

More information

3.7 Cutting plane methods

3.7 Cutting plane methods 3.7 Cutting plane methods Generic ILP problem min{ c t x : x X = {x Z n + : Ax b} } with m n matrix A and n 1 vector b of rationals. According to Meyer s theorem: There exists an ideal formulation: conv(x

More information

maxz = 3x 1 +4x 2 2x 1 +x 2 6 2x 1 +3x 2 9 x 1,x 2

maxz = 3x 1 +4x 2 2x 1 +x 2 6 2x 1 +3x 2 9 x 1,x 2 ex-5.-5. Foundations of Operations Research Prof. E. Amaldi 5. Branch-and-Bound Given the integer linear program maxz = x +x x +x 6 x +x 9 x,x integer solve it via the Branch-and-Bound method (solving

More information

Discrete (and Continuous) Optimization WI4 131

Discrete (and Continuous) Optimization WI4 131 Discrete (and Continuous) Optimization WI4 131 Kees Roos Technische Universiteit Delft Faculteit Electrotechniek, Wiskunde en Informatica Afdeling Informatie, Systemen en Algoritmiek e-mail: C.Roos@ewi.tudelft.nl

More information

21. Solve the LP given in Exercise 19 using the big-m method discussed in Exercise 20.

21. Solve the LP given in Exercise 19 using the big-m method discussed in Exercise 20. Extra Problems for Chapter 3. Linear Programming Methods 20. (Big-M Method) An alternative to the two-phase method of finding an initial basic feasible solution by minimizing the sum of the artificial

More information

Notes on Dantzig-Wolfe decomposition and column generation

Notes on Dantzig-Wolfe decomposition and column generation Notes on Dantzig-Wolfe decomposition and column generation Mette Gamst November 11, 2010 1 Introduction This note introduces an exact solution method for mathematical programming problems. The method is

More information

SOLVING INTEGER LINEAR PROGRAMS. 1. Solving the LP relaxation. 2. How to deal with fractional solutions?

SOLVING INTEGER LINEAR PROGRAMS. 1. Solving the LP relaxation. 2. How to deal with fractional solutions? SOLVING INTEGER LINEAR PROGRAMS 1. Solving the LP relaxation. 2. How to deal with fractional solutions? Integer Linear Program: Example max x 1 2x 2 0.5x 3 0.2x 4 x 5 +0.6x 6 s.t. x 1 +2x 2 1 x 1 + x 2

More information

Answer the following questions: Q1: Choose the correct answer ( 20 Points ):

Answer the following questions: Q1: Choose the correct answer ( 20 Points ): Benha University Final Exam. (ختلفات) Class: 2 rd Year Students Subject: Operations Research Faculty of Computers & Informatics Date: - / 5 / 2017 Time: 3 hours Examiner: Dr. El-Sayed Badr Answer the following

More information

Decision Procedures An Algorithmic Point of View

Decision Procedures An Algorithmic Point of View An Algorithmic Point of View ILP References: Integer Programming / Laurence Wolsey Deciding ILPs with Branch & Bound Intro. To mathematical programming / Hillier, Lieberman Daniel Kroening and Ofer Strichman

More information

A BRANCH&BOUND ALGORITHM FOR SOLVING ONE-DIMENSIONAL CUTTING STOCK PROBLEMS EXACTLY

A BRANCH&BOUND ALGORITHM FOR SOLVING ONE-DIMENSIONAL CUTTING STOCK PROBLEMS EXACTLY APPLICATIONES MATHEMATICAE 23,2 (1995), pp. 151 167 G. SCHEITHAUER and J. TERNO (Dresden) A BRANCH&BOUND ALGORITHM FOR SOLVING ONE-DIMENSIONAL CUTTING STOCK PROBLEMS EXACTLY Abstract. Many numerical computations

More information

A Bound for the Number of Different Basic Solutions Generated by the Simplex Method

A Bound for the Number of Different Basic Solutions Generated by the Simplex Method ICOTA8, SHANGHAI, CHINA A Bound for the Number of Different Basic Solutions Generated by the Simplex Method Tomonari Kitahara and Shinji Mizuno Tokyo Institute of Technology December 12th, 2010 Contents

More information

Written Exam Linear and Integer Programming (DM545)

Written Exam Linear and Integer Programming (DM545) Written Exam Linear and Integer Programming (DM545) Department of Mathematics and Computer Science University of Southern Denmark Monday, June 22, 2015, 10:00 14:00, Festsalen, Niels Bohr Allé 1 The exam

More information

Lecture 9 Tuesday, 4/20/10. Linear Programming

Lecture 9 Tuesday, 4/20/10. Linear Programming UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Spring, 2010 Lecture 9 Tuesday, 4/20/10 Linear Programming 1 Overview Motivation & Basics Standard & Slack Forms Formulating

More information

Part 1. The Review of Linear Programming

Part 1. The Review of Linear Programming In the name of God Part 1. The Review of Linear Programming 1.5. Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Formulation of the Dual Problem Primal-Dual Relationship Economic Interpretation

More information

- Well-characterized problems, min-max relations, approximate certificates. - LP problems in the standard form, primal and dual linear programs

- Well-characterized problems, min-max relations, approximate certificates. - LP problems in the standard form, primal and dual linear programs LP-Duality ( Approximation Algorithms by V. Vazirani, Chapter 12) - Well-characterized problems, min-max relations, approximate certificates - LP problems in the standard form, primal and dual linear programs

More information

KNAPSACK PROBLEMS WITH SETUPS

KNAPSACK PROBLEMS WITH SETUPS 7 e Conférence Francophone de MOdélisation et SIMulation - MOSIM 08 - du 31 mars au 2 avril 2008 - Paris - France Modélisation, Optimisation et Simulation des Systèmes : Communication, Coopération et Coordination

More information

IE 5531: Engineering Optimization I

IE 5531: Engineering Optimization I IE 5531: Engineering Optimization I Lecture 7: Duality and applications Prof. John Gunnar Carlsson September 29, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 29, 2010 1

More information

4. Duality Duality 4.1 Duality of LPs and the duality theorem. min c T x x R n, c R n. s.t. ai Tx = b i i M a i R n

4. Duality Duality 4.1 Duality of LPs and the duality theorem. min c T x x R n, c R n. s.t. ai Tx = b i i M a i R n 2 4. Duality of LPs and the duality theorem... 22 4.2 Complementary slackness... 23 4.3 The shortest path problem and its dual... 24 4.4 Farkas' Lemma... 25 4.5 Dual information in the tableau... 26 4.6

More information

Linear Programming Duality

Linear Programming Duality Summer 2011 Optimization I Lecture 8 1 Duality recap Linear Programming Duality We motivated the dual of a linear program by thinking about the best possible lower bound on the optimal value we can achieve

More information

A primal-simplex based Tardos algorithm

A primal-simplex based Tardos algorithm A primal-simplex based Tardos algorithm Shinji Mizuno a, Noriyoshi Sukegawa a, and Antoine Deza b a Graduate School of Decision Science and Technology, Tokyo Institute of Technology, 2-12-1-W9-58, Oo-Okayama,

More information

Lecture #21. c T x Ax b. maximize subject to

Lecture #21. c T x Ax b. maximize subject to COMPSCI 330: Design and Analysis of Algorithms 11/11/2014 Lecture #21 Lecturer: Debmalya Panigrahi Scribe: Samuel Haney 1 Overview In this lecture, we discuss linear programming. We first show that the

More information

Bounds on the Traveling Salesman Problem

Bounds on the Traveling Salesman Problem Bounds on the Traveling Salesman Problem Sean Zachary Roberson Texas A&M University MATH 613, Graph Theory A common routing problem is as follows: given a collection of stops (for example, towns, stations,

More information

Lecture 9: Dantzig-Wolfe Decomposition

Lecture 9: Dantzig-Wolfe Decomposition Lecture 9: Dantzig-Wolfe Decomposition (3 units) Outline Dantzig-Wolfe decomposition Column generation algorithm Relation to Lagrangian dual Branch-and-price method Generated assignment problem and multi-commodity

More information

LOWER BOUNDS FOR INTEGER PROGRAMMING PROBLEMS

LOWER BOUNDS FOR INTEGER PROGRAMMING PROBLEMS LOWER BOUNDS FOR INTEGER PROGRAMMING PROBLEMS A Thesis Presented to The Academic Faculty by Yaxian Li In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the School of Industrial

More information

1 Review Session. 1.1 Lecture 2

1 Review Session. 1.1 Lecture 2 1 Review Session Note: The following lists give an overview of the material that was covered in the lectures and sections. Your TF will go through these lists. If anything is unclear or you have questions

More information

Spring 2017 CO 250 Course Notes TABLE OF CONTENTS. richardwu.ca. CO 250 Course Notes. Introduction to Optimization

Spring 2017 CO 250 Course Notes TABLE OF CONTENTS. richardwu.ca. CO 250 Course Notes. Introduction to Optimization Spring 2017 CO 250 Course Notes TABLE OF CONTENTS richardwu.ca CO 250 Course Notes Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4, 2018 Table

More information

Farkas Lemma, Dual Simplex and Sensitivity Analysis

Farkas Lemma, Dual Simplex and Sensitivity Analysis Summer 2011 Optimization I Lecture 10 Farkas Lemma, Dual Simplex and Sensitivity Analysis 1 Farkas Lemma Theorem 1. Let A R m n, b R m. Then exactly one of the following two alternatives is true: (i) x

More information

Linear Programming: Simplex

Linear Programming: Simplex Linear Programming: Simplex Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. IMA, August 2016 Stephen Wright (UW-Madison) Linear Programming: Simplex IMA, August 2016

More information

Summary of the simplex method

Summary of the simplex method MVE165/MMG630, The simplex method; degeneracy; unbounded solutions; infeasibility; starting solutions; duality; interpretation Ann-Brith Strömberg 2012 03 16 Summary of the simplex method Optimality condition:

More information

1. Algebraic and geometric treatments Consider an LP problem in the standard form. x 0. Solutions to the system of linear equations

1. Algebraic and geometric treatments Consider an LP problem in the standard form. x 0. Solutions to the system of linear equations The Simplex Method Most textbooks in mathematical optimization, especially linear programming, deal with the simplex method. In this note we study the simplex method. It requires basically elementary linear

More information

IE418 Integer Programming

IE418 Integer Programming IE418: Integer Programming Department of Industrial and Systems Engineering Lehigh University 2nd February 2005 Boring Stuff Extra Linux Class: 8AM 11AM, Wednesday February 9. Room??? Accounts and Passwords

More information

Introduction to Integer Linear Programming

Introduction to Integer Linear Programming Lecture 7/12/2006 p. 1/30 Introduction to Integer Linear Programming Leo Liberti, Ruslan Sadykov LIX, École Polytechnique liberti@lix.polytechnique.fr sadykov@lix.polytechnique.fr Lecture 7/12/2006 p.

More information

A Strongly Polynomial Simplex Method for Totally Unimodular LP

A Strongly Polynomial Simplex Method for Totally Unimodular LP A Strongly Polynomial Simplex Method for Totally Unimodular LP Shinji Mizuno July 19, 2014 Abstract Kitahara and Mizuno get new bounds for the number of distinct solutions generated by the simplex method

More information

Stochastic Integer Programming

Stochastic Integer Programming IE 495 Lecture 20 Stochastic Integer Programming Prof. Jeff Linderoth April 14, 2003 April 14, 2002 Stochastic Programming Lecture 20 Slide 1 Outline Stochastic Integer Programming Integer LShaped Method

More information

Algorithms and Theory of Computation. Lecture 13: Linear Programming (2)

Algorithms and Theory of Computation. Lecture 13: Linear Programming (2) Algorithms and Theory of Computation Lecture 13: Linear Programming (2) Xiaohui Bei MAS 714 September 25, 2018 Nanyang Technological University MAS 714 September 25, 2018 1 / 15 LP Duality Primal problem

More information

(tree searching technique) (Boolean formulas) satisfying assignment: (X 1, X 2 )

(tree searching technique) (Boolean formulas) satisfying assignment: (X 1, X 2 ) Algorithms Chapter 5: The Tree Searching Strategy - Examples 1 / 11 Chapter 5: The Tree Searching Strategy 1. Ex 5.1Determine the satisfiability of the following Boolean formulas by depth-first search

More information

Linear and Combinatorial Optimization

Linear and Combinatorial Optimization Linear and Combinatorial Optimization The dual of an LP-problem. Connections between primal and dual. Duality theorems and complementary slack. Philipp Birken (Ctr. for the Math. Sc.) Lecture 3: Duality

More information

The simplex algorithm

The simplex algorithm The simplex algorithm The simplex algorithm is the classical method for solving linear programs. Its running time is not polynomial in the worst case. It does yield insight into linear programs, however,

More information

Computational Integer Programming Universidad de los Andes. Lecture 1. Dr. Ted Ralphs

Computational Integer Programming Universidad de los Andes. Lecture 1. Dr. Ted Ralphs Computational Integer Programming Universidad de los Andes Lecture 1 Dr. Ted Ralphs MIP Lecture 1 1 Quick Introduction Bio Course web site Course structure http://coral.ie.lehigh.edu/ ted/teaching/mip

More information

Integer Programming Part II

Integer Programming Part II Be the first in your neighborhood to master this delightful little algorithm. Integer Programming Part II The Branch and Bound Algorithm learn about fathoming, bounding, branching, pruning, and much more!

More information

min3x 1 + 4x 2 + 5x 3 2x 1 + 2x 2 + x 3 6 x 1 + 2x 2 + 3x 3 5 x 1, x 2, x 3 0.

min3x 1 + 4x 2 + 5x 3 2x 1 + 2x 2 + x 3 6 x 1 + 2x 2 + 3x 3 5 x 1, x 2, x 3 0. ex-.-. Foundations of Operations Research Prof. E. Amaldi. Dual simplex algorithm Given the linear program minx + x + x x + x + x 6 x + x + x x, x, x. solve it via the dual simplex algorithm. Describe

More information

Sensitivity Analysis

Sensitivity Analysis Dr. Maddah ENMG 500 /9/07 Sensitivity Analysis Changes in the RHS (b) Consider an optimal LP solution. Suppose that the original RHS (b) is changed from b 0 to b new. In the following, we study the affect

More information

MVE165/MMG631 Linear and integer optimization with applications Lecture 5 Linear programming duality and sensitivity analysis

MVE165/MMG631 Linear and integer optimization with applications Lecture 5 Linear programming duality and sensitivity analysis MVE165/MMG631 Linear and integer optimization with applications Lecture 5 Linear programming duality and sensitivity analysis Ann-Brith Strömberg 2017 03 29 Lecture 4 Linear and integer optimization with

More information

Discrete Optimization

Discrete Optimization Prof. Friedrich Eisenbrand Martin Niemeier Due Date: April 15, 2010 Discussions: March 25, April 01 Discrete Optimization Spring 2010 s 3 You can hand in written solutions for up to two of the exercises

More information

Combinatorial Optimization

Combinatorial Optimization Combinatorial Optimization 2017-2018 1 Maximum matching on bipartite graphs Given a graph G = (V, E), find a maximum cardinal matching. 1.1 Direct algorithms Theorem 1.1 (Petersen, 1891) A matching M is

More information

Technische Universität München, Zentrum Mathematik Lehrstuhl für Angewandte Geometrie und Diskrete Mathematik. Combinatorial Optimization (MA 4502)

Technische Universität München, Zentrum Mathematik Lehrstuhl für Angewandte Geometrie und Diskrete Mathematik. Combinatorial Optimization (MA 4502) Technische Universität München, Zentrum Mathematik Lehrstuhl für Angewandte Geometrie und Diskrete Mathematik Combinatorial Optimization (MA 4502) Dr. Michael Ritter Problem Sheet 1 Homework Problems Exercise

More information

Note 3: LP Duality. If the primal problem (P) in the canonical form is min Z = n (1) then the dual problem (D) in the canonical form is max W = m (2)

Note 3: LP Duality. If the primal problem (P) in the canonical form is min Z = n (1) then the dual problem (D) in the canonical form is max W = m (2) Note 3: LP Duality If the primal problem (P) in the canonical form is min Z = n j=1 c j x j s.t. nj=1 a ij x j b i i = 1, 2,..., m (1) x j 0 j = 1, 2,..., n, then the dual problem (D) in the canonical

More information

Appendix A Taylor Approximations and Definite Matrices

Appendix A Taylor Approximations and Definite Matrices Appendix A Taylor Approximations and Definite Matrices Taylor approximations provide an easy way to approximate a function as a polynomial, using the derivatives of the function. We know, from elementary

More information

Integer Programming ISE 418. Lecture 12. Dr. Ted Ralphs

Integer Programming ISE 418. Lecture 12. Dr. Ted Ralphs Integer Programming ISE 418 Lecture 12 Dr. Ted Ralphs ISE 418 Lecture 12 1 Reading for This Lecture Nemhauser and Wolsey Sections II.2.1 Wolsey Chapter 9 ISE 418 Lecture 12 2 Generating Stronger Valid

More information

Integer and Combinatorial Optimization: Introduction

Integer and Combinatorial Optimization: Introduction Integer and Combinatorial Optimization: Introduction John E. Mitchell Department of Mathematical Sciences RPI, Troy, NY 12180 USA November 2018 Mitchell Introduction 1 / 18 Integer and Combinatorial Optimization

More information

Chapter 1. Preliminaries

Chapter 1. Preliminaries Introduction This dissertation is a reading of chapter 4 in part I of the book : Integer and Combinatorial Optimization by George L. Nemhauser & Laurence A. Wolsey. The chapter elaborates links between

More information

Column Generation in Integer Programming with Applications in Multicriteria Optimization

Column Generation in Integer Programming with Applications in Multicriteria Optimization Column Generation in Integer Programming with Applications in Multicriteria Optimization Matthias Ehrgott Department of Engineering Science The University of Auckland, New Zealand email: m.ehrgott@auckland.ac.nz

More information

15.083J/6.859J Integer Optimization. Lecture 10: Solving Relaxations

15.083J/6.859J Integer Optimization. Lecture 10: Solving Relaxations 15.083J/6.859J Integer Optimization Lecture 10: Solving Relaxations 1 Outline The key geometric result behind the ellipsoid method Slide 1 The ellipsoid method for the feasibility problem The ellipsoid

More information

Outline. Relaxation. Outline DMP204 SCHEDULING, TIMETABLING AND ROUTING. 1. Lagrangian Relaxation. Lecture 12 Single Machine Models, Column Generation

Outline. Relaxation. Outline DMP204 SCHEDULING, TIMETABLING AND ROUTING. 1. Lagrangian Relaxation. Lecture 12 Single Machine Models, Column Generation Outline DMP204 SCHEDULING, TIMETABLING AND ROUTING 1. Lagrangian Relaxation Lecture 12 Single Machine Models, Column Generation 2. Dantzig-Wolfe Decomposition Dantzig-Wolfe Decomposition Delayed Column

More information

Linear Programming. Chapter Introduction

Linear Programming. Chapter Introduction Chapter 3 Linear Programming Linear programs (LP) play an important role in the theory and practice of optimization problems. Many COPs can directly be formulated as LPs. Furthermore, LPs are invaluable

More information

Optimisation and Operations Research

Optimisation and Operations Research Optimisation and Operations Research Lecture 15: The Greedy Heuristic Matthew Roughan http://www.maths.adelaide.edu.au/matthew.roughan/ Lecture_notes/OORII/ School of

More information

Integer Programming. The focus of this chapter is on solution techniques for integer programming models.

Integer Programming. The focus of this chapter is on solution techniques for integer programming models. Integer Programming Introduction The general linear programming model depends on the assumption of divisibility. In other words, the decision variables are allowed to take non-negative integer as well

More information

The Strong Duality Theorem 1

The Strong Duality Theorem 1 1/39 The Strong Duality Theorem 1 Adrian Vetta 1 This presentation is based upon the book Linear Programming by Vasek Chvatal 2/39 Part I Weak Duality 3/39 Primal and Dual Recall we have a primal linear

More information

Written Exam Linear and Integer Programming (DM554)

Written Exam Linear and Integer Programming (DM554) Written Exam Linear and Integer Programming (DM554) Department of Mathematics and Computer Science University of Southern Denmark Monday, June 22, 2015, 10:00 14:00, Festsalen, Niels Bohr Allé 1 The exam

More information