Complexity of linear programming: outline
|
|
- Osborne Parker
- 6 years ago
- Views:
Transcription
1 Complexity of linear programming: outline I Assessing computational e ciency of algorithms I Computational e ciency of the Simplex method I Ellipsoid algorithm for LP and its computational e ciency IOE 610: LP II, Fall 2013 Complexity of linear programming Page 139 Problem, instance, algorithm I Linear Programming in inequality form is a problem; I An instance of this problem is given by d =(A, b, c) Definition 8.1 An instance of an optimization problem consists of a feasible set F and a cost function c : F!<. An optimization problem is defined as a collection of instances. I An algorithm is a finite set of instructions, such as arithmetic operators, conditional statements, read and write statements, etc., and the running time of the algorithm is the total number of steps involved in carrying out these instructions until termination is reached I Algorithms are described for problems, but then applied to individual instances. I An algorithm solves the problem if it terminates in finite time and produces a correct answer for all instances of the problem. IOE 610: LP II, Fall 2013 Complexity of linear programming Page 140
2 Algorithms for Solving (Optimization) Problems I It is reasonable to expect the running time of an algorithm to depend on the size of the instance to which it is applied. Definition 8.2 The size of an instance is defined as the number of bits used to describe the instance, according to a prespecified format.... geared towards binary computation and integer input data I r 2 Z, r apple U: needblog 2 Uc + 2 bits (binary representation) r = a k 2 k +a k 1 2 k 1 + +a 1 w 1 +a 0 2 0, a i 2 {0, 1} 8i, k appleblog 2 Uc I An LP instance with d 2 Z mn+m+n,max{ a ij, b i, c j } = U: the size of d is O(mn log 2 U) IOE 610: LP II, Fall 2013 Complexity of linear programming Page 141 Computational e ciency analysis of algorithms Question: Given an algorithm that solves our problem and an instance of our problem of a particular size, how long will the algorithm take to solve it? Two types of answers: I Average performance on a typical problem instance I Mathematically di cult to define or analyze rigorously I Observations from empirical experience give some hints I Worst case performance: how long for the most di cult (for this algorithm) instance of the given size? I Often too pessimistic to predict practical behavior I Relatively easy to define and analyze rigorously I Provides a worst-case guarantee I Important for the theory of complexity: Suggests where the limits of our computational abilities are IOE 610: LP II, Fall 2013 Complexity of linear programming Page 142
3 Worst-case analysis and e cient algorithms I We will be looking at T (S): the worst-case running time of the algorithm on instances of the size (at most) S. I As before, we will use the arithmetic model of computation, and count the number of arithmetic operations to estimate T (S) I The algorithm is usually considered e cient if T (S) canbe bounded above by a polynomial function Definition 8.3 An algorithm runs in polynomial time (poly-time) if 9k 2 Z such that T (S) =O(S k ). IOE 610: LP II, Fall 2013 Complexity of linear programming Page 143 How can we measure the running time of an algorithm? I Arithmetic model of computation: each operation (including arithmetic operations) takes unit time. (Easy to analyze.) I Bit model of computation: each operation is decomposed into bit operations, and it s these elementary operations that are assumed to take unit time (for example, adding two numbers takes longer if the numbers are large). (Better estimate of the true running time) I Fact Suppose I an algorithm takes poly-time under the arithmetic model, and I on instances of size S, any integer produced in the course of the algorithm execution has size bounded by a polynomial in S. Then the algorithm runs in polynomial time under the bit model. IOE 610: LP II, Fall 2013 Complexity of linear programming Page 144
4 Computational e ciency of simplex method I An LP instance in n variables with m constraints min c T x s.t. Ax b I We have shown that each iteration takes polynomial time (O(mn) with revised implementation) I Total number of iterations? I On problems arising in practice simplex method is usually extremely fast I Conventional wisdom suggests that number of iterations in practice is about 3m I Worst case number of iterations exponential in n (Klee and Minty, 1972) IOE 610: LP II, Fall 2013 Complexity of linear programming Page 145 Klee-Minty example For n = 3: max nx 2 n j x j j=1 i 1 X s.t. 2 2 i j x j + x i apple 100 i 1, i =1,...,n x j j=1 0, j =1,...,n max 4x 1 + 2x 2 + x 3 s.t. x 1 apple 1 4x 1 + x 2 apple 100 8x 1 + 4x 2 + x 3 apple x 1, x 2, x 3 0 IOE 610: LP II, Fall 2013 Complexity of linear programming Page 146
5 Klee-Minty cube 5 Case n =3: Feasible region is a distortion to a stretched n-dimensional cube 0 apple x 1 apple 1 0 apple x 2 apple apple x n apple 100 n 1 Starting at x = 0, using steepest pivot rule, simplex method will visit all 2 n vertices before finding the optimal solution... what if we use a di erent pivoting rule? 5 Thanks to R. Vanderbei for the illustration IOE 610: LP II, Fall 2013 Complexity of linear programming Page 147 Simplex method and diameter of polyhedra Examples of bad polyhedra have been given for all known pivoting rules. Is there an exponential example for any pivoting rule, perhaps one not yet invented? I If x and y are two vertices of a polyhedron, consider traveling from x to y via adjacent vertices; I d(x, y) # of steps in the shortest such path I Diameter of a polyhedron: max x,y d(x, y) I (n, m) - maximum diameter of all bounded polyhedra in < n that can be represented by m inequalities I With a perfect pivoting rule, simplex shouldn t need more than (n, m) iterations I Hirsch Conjecture proposed in 1957: (n, m) apple m n I True for n apple 3 and other special cases I Disproven by Francisco Santos, 2010: a polyhedron with n = 86 and m = 43 with diameter bigger than 43 I Still open: (n, m) apple m? Known: (n, m) apple m 1+log 2 n Even if we can bound (n, m), still need the right pivoting rule... IOE 610: LP II, Fall 2013 Complexity of linear programming Page 148
6 Some LP history I 1930 s 1940 s I (Specialized) LP models and solution approaches developed independently in the Soviet Union and the West for a variety of optimal resource allocation and planning applications I Late 1940 s I General LP theory (John Von Neumann) and solution method (Simplex algorithm, George Dantzig) developed in US I Simplex runs quite fast in practice; LP used for military operations and gains widespread use after the war I 1972 I Klee and Minty show that the simplex algorithm is not e cient, i.e., it does not run in polynomial time I 1975: Nobel Prize in Economics is awarded for for their contributions to the theory of optimum allocation of resources via LP to Leonid V. Kantorovich and Tjalling C. Koopmans IOE 610: LP II, Fall 2013 Complexity of linear programming Page 149 LP history, continued I 1970 s I THE BIG QUESTION: Does there exist an algorithm for solving LPs that s poly-time in the worst case? I 1979, in the Soviet Union... I Leonid G. Khachiyan: YES, THERE IS the Ellipsoid algorithm! I NY Times publishes several articles about this discovery; makes some mathematical blunders about the implications of the result and has to print retractions IOE 610: LP II, Fall 2013 Complexity of linear programming Page 150
7 Ellipsoid Algorithm for LP: outline I Develop general ideas for the Ellipsoid algorithm I Specify the algorithm for finding a point in P = {x 2< n : Ax b} I Modify the algorithm for solving min c T x s.t. Ax b IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 151 Volumes and A ne Transformations Definition If L < n, the volume of L is Z Vol(L) = Definition 8.6 x2l If D 2< n n is nonsingular and b 2< n,themapping S(x) = Dx + b is an a ne transformation. dx I Note: by definition, a ne transformation is invertible Lemma 8.1 Let L < n.ifs(x) =Dx + b then Vol(S(L)) = det(d) Vol(L). IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 152
8 Assumptions for Ellipsoid Algorithm Goal of the algorithm: find a point in a suitably described convex set P < n. Assumptions: I P is bounded: 9B(x 0, r) P. LetV Vol(B(x 0, r)). I P is full-dimensional, i.e., has positive volume (Definition 8.7) I Let Vol(P) > v > 0. I P can be described via a separation oracle: given a vector y 2< n,theoracleeitherreportsthaty 2 P, orfindsa separating hyperplane: a 2< n such that a T y < a T x 8x 2 P. IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 153 Positive definite matrices and Ellipsoids Definition 8.4 Let Q be an n n symmetric matrix, i.e., Q T = Q. Q is called positive definite (notation: Q 0) if x T Qx > 0 8x 6= 0. I Symmetric: Q 2 S n ;symmetricpd:q 2 S n + or Q 0 I Q 2 S n has n eigenvalues 1,..., n; det(q) = Q n i=1 i I Q 2 S n + i > 0 Definition 8.5 AsetE < n E = E(z, Q) ={x :(x z) T Q 1 (x z) apple 1} where Q 0isanellipsoid with center z 2< n. IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 154
9 Ellipsoids as a ne transformations of balls I Aballcenteredatzwith radius r > 0: E(z, r 2 I)={x :(x z) T (x z) apple r 2 } = {x : kx zk appler} I Unit ball: ball centered at 0 with radius 1: B(0, 1) = E(0, I) I If Q 2 S+, n 9Q S+: n Q 1 2 Q 1 2 = Q and det(q 1 2 )= p det(q) I Note: an ellipsoid is an a ne transformation of the unit ball: E(z, Q) =Q 1 2 E(0, I)+z Corollary: Let Q 0. Then Vol(E(z, Q)) = p det(q) Vol(B(0, 1)). IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 155 Central-cut ellipsoid algorithm: general idea Input: Ellipsoid E 0, constants V and v as above. Initialization: E 0 = E(x 0, Q 0 ) P, t = 0. Iteration t: (E t = E(x t, Q t ) P) I Call the separation oracle with x t as input. I If x t 2 P terminate. I Otherwise, oracle returns a such that P E t \ {x : a T x a T x t }. Construct ellipsoid E t+1 = E(x t+1, Q t+1 ) of smaller volume containing the set on the right. I Set t t + 1 and continue. IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 156
10 Central-cut ellipsoid algorithm Iteration details How to construct E t+1 E t \ {x : a T x a T x t } Theorem 8.1 Let E = E(z, Q) be an ellipsoid in < n,andlet06= a 2< n. Consider the halfspace H = {x : a T x a T z} and let Q = z = z + 1 Qa p n +1 a T Qa, n2 n 2 Q 1 2 n +1 Qaa T Q a T. Qa Then Q 0, thus E 0 = E( z, Q) is an ellipsoid. Moreover, (a) E \ H E 0 (b) Vol(E 0 ) < e 1 2(n+1) Vol(E). E 0 is the smallest-volume ellipsoid that contains E \ H IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 157 Central-cut ellipsoid algorithm Termination and running time I At iteration t, Vol(E t ) < e 1 2(n+1) t Vol(E0 ) apple e t 2(n+1) I P E t,sov apple Vol(P) apple Vol(E t ) I v < e t 2(n+1) V,hence I Since t is an integer, we have V t < 2(n + 1) ln(v /v) t appled2(n + 1) ln(v /v)e 1 t? 1, i.e., we must have x t? 1 2 P. I So, the ellipsoid algorithm will terminate in at most t? iterations, each iteration consisting of a call to the separation oracle and some arithmetics IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 158
11 Ellipsoid algorithm for linear inequalities Goal: find out whether P = ;. P = {x 2< n : Ax b} I Assume P is bounded and either empty or full-dimensional. Also, presume E 0, v and V are known. I Separation oracle: for given y 2< n,checkif a T i y b i, i =1,...,m I If all constraints satisfied, y 2 P I If ith constraint is violated, a T i x a T i y is a separating hyperplane The oracle requires O(mn) arithmetic operations I We assume for now calculations are made in infinite precision and taking a square root takes the same time as any other arithmetic operation IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 159 Central-cut ellipsoid algorithm for linear inequalities Input: Ellipsoid E 0, constants V and v as above. Initialization: E 0 = E(x 0, Q 0 ) P, t = 0. Let t? = d2(n + 1) ln(v /v)e. Iteration t: (E t = E(x t, Q t ) P) I If t = t?, stop; P = ;. I If x t 2 P, stop; P 6= ;. I Otherwise, find a violated constraint: a T i x t < b i. Construct ellipsoid E t+1 = E(x t+1, Q t+1 ) E t \ {x : a T i x a T i x t } (use Thm. 8.1). I Set t t + 1 and continue. IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 160
12 Assumptions revisited: V Lemma 8.2 (modified) Let A 2 Z m n,andb2< m.letu =max{ a ij, b i }. (a) Every basic solution of {x : Ax b} satisfies (nu) n apple x j apple (nu) n, j =1,...,n (b) Every basic solution of P = {x : Ax = b, x 0} satisfies Proof of (a): (mu) m apple x j apple (mu) m, j =1,...,n I Basic solution is  1ˆb, whereâ 2< n n is a submatrix of A and ˆb 2< n is a subvector of b. I Cramer s rule gives a formula of each component x j If rows of A span < n, P 6= ; i it has extreme points, all of which are contained in E 0 = E(0, n(nu) 2n I ), Vol(E 0 ) < V =(2n) n (nu) n2 IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 161 Assumptions revisited: v Lemma 8.4 Let P = {x : Ax b} be full-dimensional and bounded. Assume A and b have integer entries of magnitude at most U. Then Vol(P) n n (nu) n2 (n+1) Idea of the proof: I P is the convex hull of its extreme points I If P 2< n is full-dimensional, it has n + 1 extreme points that do not belong to a common hyperplane I Vol(P) volume of convex hull of these extreme points I The latter can be bounded by bounding components of extreme points IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 162
13 Assumptions revisited: P full-dimensional Lemma 8.3 Let P = {x : Ax b}. AssumeA and b have integer entries of magnitude at most U. Let = 1 2(n + 1) ((n + 1))U) (n+1). Let P = {x : Ax b e}. (a) If P is empty, then P is empty (b) If P is nonempty, then P is full-dimensional. IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 163 Run-time of the ellipsoid algorithm: number of iterations P = {x : Ax b} Assume A, b have integer components with magnitude bounded by U, rows of A span < n. Case 1: If we know that P is bounded, and either empty or full-dimensional, take E 0 = E(0, n(nu) 2n I ), v = n n (nu) n2 (n+1), V =(2n) n (nu)n 2, and the algorithm with these inputs will terminate in O(n 4 log(nu)) iterations (Recall: instance size is O(mn log U)) Case 2: If P is arbitrary, I Construct P B by adding bounds on variables as in Lemma 8.2 I PB contains all extreme points of P I Construct P B, as in Lemma 8.3 I To decide if P = ;, applyeatop B, ;itwillterminatein O(n 6 log(nu)) iterations. IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 164
14 Run-time of the ellipsoid algorithm: overall running time I We showed that number of iterations is polynomial in the size of the problem I To show the algorithm takes polynomial time, need to show that the number of operations/run-time per iteration is also polynomial I Issues: I Need to take square roots cannot be done accurately in finite-precision calculations I Need to make sure the numbers generated have polynomial size I These problems can be resolved by showing that computations need only be carried out with finite (polynomial) precision. The result: Theorem 8.3 The LP feasibility problem with integer data can be solved in polynomial time. IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 165 Ellipsoid algorithm for LP optimization (P) min c T x (D) max p T b s.t. Ax b s.t. p T A = c T Option 1: Apply ellipsoid algorithm to p 0 Q = {(x, p) :b T p = c T x, Ax b, A T p = c, p 0}. Option 2: Sliding objective: start by finding x 0 2 P; foreacht apply ellipsoid algorithm to P \ {x : c T x < c T x t }. If the set is empty, x t is a solution. Otherwise, we find a point x t+1 2 P that s better than x t, and continue. IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 166
15 Practical implications? I Although a great theoretical accomplishment, Ellipsoid algorithm never became a method of choice for solving LPs in practice I Its observed running time its worst-case running time... I...unlike the simplex method I...or the Interior Point (Barrier) Methods IOE 610: LP II, Fall 2013 Ellipsoid algorithm Page 167
15.081J/6.251J Introduction to Mathematical Programming. Lecture 18: The Ellipsoid method
15.081J/6.251J Introduction to Mathematical Programming Lecture 18: The Ellipsoid method 1 Outline Efficient algorithms and computational complexity Slide 1 The key geometric result behind the ellipsoid
More information15.083J/6.859J Integer Optimization. Lecture 10: Solving Relaxations
15.083J/6.859J Integer Optimization Lecture 10: Solving Relaxations 1 Outline The key geometric result behind the ellipsoid method Slide 1 The ellipsoid method for the feasibility problem The ellipsoid
More informationThe Ellipsoid Algorithm
The Ellipsoid Algorithm John E. Mitchell Department of Mathematical Sciences RPI, Troy, NY 12180 USA 9 February 2018 Mitchell The Ellipsoid Algorithm 1 / 28 Introduction Outline 1 Introduction 2 Assumptions
More informationLINEAR PROGRAMMING III
LINEAR PROGRAMMING III ellipsoid algorithm combinatorial optimization matrix games open problems Lecture slides by Kevin Wayne Last updated on 7/25/17 11:09 AM LINEAR PROGRAMMING III ellipsoid algorithm
More informationThe Ellipsoid (Kachiyan) Method
Yinyu Ye, MS&E, Stanford MS&E310 Lecture Note: Ellipsoid Method 1 The Ellipsoid (Kachiyan) Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
More informationTopics in Theoretical Computer Science April 08, Lecture 8
Topics in Theoretical Computer Science April 08, 204 Lecture 8 Lecturer: Ola Svensson Scribes: David Leydier and Samuel Grütter Introduction In this lecture we will introduce Linear Programming. It was
More informationWeek 8. 1 LP is easy: the Ellipsoid Method
Week 8 1 LP is easy: the Ellipsoid Method In 1979 Khachyan proved that LP is solvable in polynomial time by a method of shrinking ellipsoids. The running time is polynomial in the number of variables n,
More informationLecture Simplex Issues: Number of Pivots. ORIE 6300 Mathematical Programming I October 9, 2014
ORIE 6300 Mathematical Programming I October 9, 2014 Lecturer: David P. Williamson Lecture 14 Scribe: Calvin Wylie 1 Simplex Issues: Number of Pivots Question: How many pivots does the simplex algorithm
More informationOptimization WS 13/14:, by Y. Goldstein/K. Reinert, 9. Dezember 2013, 16: Linear programming. Optimization Problems
Optimization WS 13/14:, by Y. Goldstein/K. Reinert, 9. Dezember 2013, 16:38 2001 Linear programming Optimization Problems General optimization problem max{z(x) f j (x) 0,x D} or min{z(x) f j (x) 0,x D}
More informationCSC Linear Programming and Combinatorial Optimization Lecture 8: Ellipsoid Algorithm
CSC2411 - Linear Programming and Combinatorial Optimization Lecture 8: Ellipsoid Algorithm Notes taken by Shizhong Li March 15, 2005 Summary: In the spring of 1979, the Soviet mathematician L.G.Khachian
More informationPolynomiality of Linear Programming
Chapter 10 Polynomiality of Linear Programming In the previous section, we presented the Simplex Method. This method turns out to be very efficient for solving linear programmes in practice. While it is
More informationOptimization (168) Lecture 7-8-9
Optimization (168) Lecture 7-8-9 Jesús De Loera UC Davis, Mathematics Wednesday, April 2, 2012 1 DEGENERACY IN THE SIMPLEX METHOD 2 DEGENERACY z =2x 1 x 2 + 8x 3 x 4 =1 2x 3 x 5 =3 2x 1 + 4x 2 6x 3 x 6
More informationLecture 5: Computational Complexity
Lecture 5: Computational Complexity (3 units) Outline Computational complexity Decision problem, Classes N P and P. Polynomial reduction and Class N PC P = N P or P = N P? 1 / 22 The Goal of Computational
More informationLecture 1 Introduction
L. Vandenberghe EE236A (Fall 2013-14) Lecture 1 Introduction course overview linear optimization examples history approximate syllabus basic definitions linear optimization in vector and matrix notation
More information7. Lecture notes on the ellipsoid algorithm
Massachusetts Institute of Technology Michel X. Goemans 18.433: Combinatorial Optimization 7. Lecture notes on the ellipsoid algorithm The simplex algorithm was the first algorithm proposed for linear
More informationCS675: Convex and Combinatorial Optimization Spring 2018 The Ellipsoid Algorithm. Instructor: Shaddin Dughmi
CS675: Convex and Combinatorial Optimization Spring 2018 The Ellipsoid Algorithm Instructor: Shaddin Dughmi History and Basics Originally developed in the mid 70s by Iudin, Nemirovski, and Shor for use
More informationCS 6820 Fall 2014 Lectures, October 3-20, 2014
Analysis of Algorithms Linear Programming Notes CS 6820 Fall 2014 Lectures, October 3-20, 2014 1 Linear programming The linear programming (LP) problem is the following optimization problem. We are given
More informationCS711008Z Algorithm Design and Analysis
CS711008Z Algorithm Design and Analysis Lecture 8 Linear programming: interior point method Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China 1 / 31 Outline Brief
More informationMotivating examples Introduction to algorithms Simplex algorithm. On a particular example General algorithm. Duality An application to game theory
Instructor: Shengyu Zhang 1 LP Motivating examples Introduction to algorithms Simplex algorithm On a particular example General algorithm Duality An application to game theory 2 Example 1: profit maximization
More informationWeek 2. The Simplex method was developed by Dantzig in the late 40-ties.
1 The Simplex method Week 2 The Simplex method was developed by Dantzig in the late 40-ties. 1.1 The standard form The simplex method is a general description algorithm that solves any LPproblem instance.
More informationLINEAR PROGRAMMING I. a refreshing example standard form fundamental questions geometry linear algebra simplex algorithm
Linear programming Linear programming. Optimize a linear function subject to linear inequalities. (P) max c j x j n j= n s. t. a ij x j = b i i m j= x j 0 j n (P) max c T x s. t. Ax = b Lecture slides
More informationA Redundant Klee-Minty Construction with All the Redundant Constraints Touching the Feasible Region
A Redundant Klee-Minty Construction with All the Redundant Constraints Touching the Feasible Region Eissa Nematollahi Tamás Terlaky January 5, 2008 Abstract By introducing some redundant Klee-Minty constructions,
More informationLecture 15: October 15
10-725: Optimization Fall 2012 Lecturer: Barnabas Poczos Lecture 15: October 15 Scribes: Christian Kroer, Fanyi Xiao Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have
More informationResource Constrained Project Scheduling Linear and Integer Programming (1)
DM204, 2010 SCHEDULING, TIMETABLING AND ROUTING Lecture 3 Resource Constrained Project Linear and Integer Programming (1) Marco Chiarandini Department of Mathematics & Computer Science University of Southern
More informationLecture slides by Kevin Wayne
LINEAR PROGRAMMING I a refreshing example standard form fundamental questions geometry linear algebra simplex algorithm Lecture slides by Kevin Wayne Last updated on 7/25/17 11:09 AM Linear programming
More informationLinear Programming. Linear Programming I. Lecture 1. Linear Programming. Linear Programming
Linear Programming Linear Programming Lecture Linear programming. Optimize a linear function subject to linear inequalities. (P) max " c j x j n j= n s. t. " a ij x j = b i # i # m j= x j 0 # j # n (P)
More informationLOWER BOUNDS FOR THE MAXIMUM NUMBER OF SOLUTIONS GENERATED BY THE SIMPLEX METHOD
Journal of the Operations Research Society of Japan Vol 54, No 4, December 2011, pp 191 200 c The Operations Research Society of Japan LOWER BOUNDS FOR THE MAXIMUM NUMBER OF SOLUTIONS GENERATED BY THE
More informationTwo Lectures on the Ellipsoid Method for Solving Linear Programs
Two Lectures on the Ellipsoid Method for Solving Linear Programs Lecture : A polynomial-time algorithm for LP Consider the general linear programming problem: δ = max cx S.T. Ax b, x R n, () where A =
More informationTheory and Internet Protocols
Game Lecture 2: Linear Programming and Zero Sum Nash Equilibrium Xiaotie Deng AIMS Lab Department of Computer Science Shanghai Jiaotong University September 26, 2016 1 2 3 4 Standard Form (P) Outline
More informationLinear Programming: Simplex
Linear Programming: Simplex Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. IMA, August 2016 Stephen Wright (UW-Madison) Linear Programming: Simplex IMA, August 2016
More informationLecture notes on the ellipsoid algorithm
Massachusetts Institute of Technology Handout 1 18.433: Combinatorial Optimization May 14th, 007 Michel X. Goemans Lecture notes on the ellipsoid algorithm The simplex algorithm was the first algorithm
More informationCSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming
CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming January 26, 2018 1 / 38 Liability/asset cash-flow matching problem Recall the formulation of the problem: max w c 1 + p 1 e 1 = 150
More informationA Review of Linear Programming
A Review of Linear Programming Instructor: Farid Alizadeh IEOR 4600y Spring 2001 February 14, 2001 1 Overview In this note we review the basic properties of linear programming including the primal simplex
More informationLP Duality: outline. Duality theory for Linear Programming. alternatives. optimization I Idea: polyhedra
LP Duality: outline I Motivation and definition of a dual LP I Weak duality I Separating hyperplane theorem and theorems of the alternatives I Strong duality and complementary slackness I Using duality
More informationLecture 2: The Simplex method
Lecture 2 1 Linear and Combinatorial Optimization Lecture 2: The Simplex method Basic solution. The Simplex method (standardform, b>0). 1. Repetition of basic solution. 2. One step in the Simplex algorithm.
More informationA strongly polynomial algorithm for linear systems having a binary solution
A strongly polynomial algorithm for linear systems having a binary solution Sergei Chubanov Institute of Information Systems at the University of Siegen, Germany e-mail: sergei.chubanov@uni-siegen.de 7th
More informationA Strongly Polynomial Simplex Method for Totally Unimodular LP
A Strongly Polynomial Simplex Method for Totally Unimodular LP Shinji Mizuno July 19, 2014 Abstract Kitahara and Mizuno get new bounds for the number of distinct solutions generated by the simplex method
More information4.5 Simplex method. LP in standard form: min z = c T x s.t. Ax = b
4.5 Simplex method LP in standard form: min z = c T x s.t. Ax = b x 0 George Dantzig (1914-2005) Examine a sequence of basic feasible solutions with non increasing objective function values until an optimal
More informationMulticommodity Flows and Column Generation
Lecture Notes Multicommodity Flows and Column Generation Marc Pfetsch Zuse Institute Berlin pfetsch@zib.de last change: 2/8/2006 Technische Universität Berlin Fakultät II, Institut für Mathematik WS 2006/07
More informationLMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009
LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix
More informationThe Simplex and Policy Iteration Methods are Strongly Polynomial for the Markov Decision Problem with Fixed Discount
The Simplex and Policy Iteration Methods are Strongly Polynomial for the Markov Decision Problem with Fixed Discount Yinyu Ye Department of Management Science and Engineering and Institute of Computational
More informationAlgorithms and Theory of Computation. Lecture 13: Linear Programming (2)
Algorithms and Theory of Computation Lecture 13: Linear Programming (2) Xiaohui Bei MAS 714 September 25, 2018 Nanyang Technological University MAS 714 September 25, 2018 1 / 15 LP Duality Primal problem
More informationMath 5593 Linear Programming Week 1
University of Colorado Denver, Fall 2013, Prof. Engau 1 Problem-Solving in Operations Research 2 Brief History of Linear Programming 3 Review of Basic Linear Algebra Linear Programming - The Story About
More information1. Algebraic and geometric treatments Consider an LP problem in the standard form. x 0. Solutions to the system of linear equations
The Simplex Method Most textbooks in mathematical optimization, especially linear programming, deal with the simplex method. In this note we study the simplex method. It requires basically elementary linear
More information3. Linear Programming and Polyhedral Combinatorics
Massachusetts Institute of Technology 18.433: Combinatorial Optimization Michel X. Goemans February 28th, 2013 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory
More informationMS-E2140. Lecture 1. (course book chapters )
Linear Programming MS-E2140 Motivations and background Lecture 1 (course book chapters 1.1-1.4) Linear programming problems and examples Problem manipulations and standard form problems Graphical representation
More informationInitial feasible origin: 1. Set values of original variables to zero. 2. Set values of slack variables according to the dictionary.
Initial feasible origin: 1. Set values of original variables to zero. 2. Set values of slack variables according to the dictionary. The problems we have solved so far always had an initial feasible origin.
More informationLinear Programming. Chapter Introduction
Chapter 3 Linear Programming Linear programs (LP) play an important role in the theory and practice of optimization problems. Many COPs can directly be formulated as LPs. Furthermore, LPs are invaluable
More informationYinyu Ye, MS&E, Stanford MS&E310 Lecture Note #06. The Simplex Method
The Simplex Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapters 2.3-2.5, 3.1-3.4) 1 Geometry of Linear
More informationTopics in Theoretical Computer Science: An Algorithmist's Toolkit Fall 2007
MIT OpenCourseWare http://ocw.mit.edu 18.409 Topics in Theoretical Computer Science: An Algorithmist's Toolkit Fall 2007 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
More informationOptimization methods NOPT048
Optimization methods NOPT048 Jirka Fink https://ktiml.mff.cuni.cz/ fink/ Department of Theoretical Computer Science and Mathematical Logic Faculty of Mathematics and Physics Charles University in Prague
More informationA Simpler and Tighter Redundant Klee-Minty Construction
A Simpler and Tighter Redundant Klee-Minty Construction Eissa Nematollahi Tamás Terlaky October 19, 2006 Abstract By introducing redundant Klee-Minty examples, we have previously shown that the central
More informationCS Algorithms and Complexity
CS 50 - Algorithms and Complexity Linear Programming, the Simplex Method, and Hard Problems Sean Anderson 2/15/18 Portland State University Table of contents 1. The Simplex Method 2. The Graph Problem
More informationLinear Thinking. Chapter Simplest example: Solving systems of linear equations
Chapter 6 Linear Thinking According to conventional wisdom, linear thinking describes thought process that is logical or step-by-step (i.e., each step must be completed before the next one is undertaken).
More informationCSC373: Algorithm Design, Analysis and Complexity Fall 2017 DENIS PANKRATOV NOVEMBER 1, 2017
CSC373: Algorithm Design, Analysis and Complexity Fall 2017 DENIS PANKRATOV NOVEMBER 1, 2017 Linear Function f: R n R is linear if it can be written as f x = a T x for some a R n Example: f x 1, x 2 =
More information15-850: Advanced Algorithms CMU, Spring 2017 Lecture #17: The Ellipsoid Algorithm March 3, 2017
15-850: Advanced Algorithms CMU, Spring 2017 Lecture #17: The Ellipsoid Algorithm March 3, 2017 Lecturer: Anupam Gupta Scribe: Benjamin Berg, David K. Isenberg In this lecture, we discuss some polynomial-time
More informationAn upper bound for the number of different solutions generated by the primal simplex method with any selection rule of entering variables
An upper bound for the number of different solutions generated by the primal simplex method with any selection rule of entering variables Tomonari Kitahara and Shinji Mizuno February 2012 Abstract Kitahara
More informationCS675: Convex and Combinatorial Optimization Fall 2016 Combinatorial Problems as Linear and Convex Programs. Instructor: Shaddin Dughmi
CS675: Convex and Combinatorial Optimization Fall 2016 Combinatorial Problems as Linear and Convex Programs Instructor: Shaddin Dughmi Outline 1 Introduction 2 Shortest Path 3 Algorithms for Single-Source
More informationOptimisation and Operations Research
Optimisation and Operations Research Lecture 22: Linear Programming Revisited Matthew Roughan http://www.maths.adelaide.edu.au/matthew.roughan/ Lecture_notes/OORII/ School
More informationCOT 6936: Topics in Algorithms! Giri Narasimhan. ECS 254A / EC 2443; Phone: x3748
COT 6936: Topics in Algorithms! Giri Narasimhan ECS 254A / EC 2443; Phone: x3748 giri@cs.fiu.edu https://moodle.cis.fiu.edu/v2.1/course/view.php?id=612 Gaussian Elimination! Solving a system of simultaneous
More informationCombinatorial Optimization Spring Term 2015 Rico Zenklusen. 2 a = ( 3 2 ) 1 E(a, A) = E(( 3 2 ), ( 4 0
3 2 a = ( 3 2 ) 1 E(a, A) = E(( 3 2 ), ( 4 0 0 1 )) 0 0 1 2 3 4 5 Figure 9: An example of an axis parallel ellipsoid E(a, A) in two dimensions. Notice that the eigenvectors of A correspond to the axes
More informationOutline. Outline. Outline DMP204 SCHEDULING, TIMETABLING AND ROUTING. 1. Scheduling CPM/PERT Resource Constrained Project Scheduling Model
Outline DMP204 SCHEDULING, TIMETABLING AND ROUTING Lecture 3 and Mixed Integer Programg Marco Chiarandini 1. Resource Constrained Project Model 2. Mathematical Programg 2 Outline Outline 1. Resource Constrained
More informationCS675: Convex and Combinatorial Optimization Fall 2014 Combinatorial Problems as Linear Programs. Instructor: Shaddin Dughmi
CS675: Convex and Combinatorial Optimization Fall 2014 Combinatorial Problems as Linear Programs Instructor: Shaddin Dughmi Outline 1 Introduction 2 Shortest Path 3 Algorithms for Single-Source Shortest
More informationOn the Number of Solutions Generated by the Simplex Method for LP
Workshop 1 on Large Scale Conic Optimization IMS (NUS) On the Number of Solutions Generated by the Simplex Method for LP Tomonari Kitahara and Shinji Mizuno Tokyo Institute of Technology November 19 23,
More informationInteger Programming ISE 418. Lecture 12. Dr. Ted Ralphs
Integer Programming ISE 418 Lecture 12 Dr. Ted Ralphs ISE 418 Lecture 12 1 Reading for This Lecture Nemhauser and Wolsey Sections II.2.1 Wolsey Chapter 9 ISE 418 Lecture 12 2 Generating Stronger Valid
More informationOptimization: Then and Now
Optimization: Then and Now Optimization: Then and Now Optimization: Then and Now Why would a dynamicist be interested in linear programming? Linear Programming (LP) max c T x s.t. Ax b αi T x b i for i
More informationLectures 6, 7 and part of 8
Lectures 6, 7 and part of 8 Uriel Feige April 26, May 3, May 10, 2015 1 Linear programming duality 1.1 The diet problem revisited Recall the diet problem from Lecture 1. There are n foods, m nutrients,
More informationConvex optimization. Javier Peña Carnegie Mellon University. Universidad de los Andes Bogotá, Colombia September 2014
Convex optimization Javier Peña Carnegie Mellon University Universidad de los Andes Bogotá, Colombia September 2014 1 / 41 Convex optimization Problem of the form where Q R n convex set: min x f(x) x Q,
More information3. Vector spaces 3.1 Linear dependence and independence 3.2 Basis and dimension. 5. Extreme points and basic feasible solutions
A. LINEAR ALGEBRA. CONVEX SETS 1. Matrices and vectors 1.1 Matrix operations 1.2 The rank of a matrix 2. Systems of linear equations 2.1 Basic solutions 3. Vector spaces 3.1 Linear dependence and independence
More information1 Review Session. 1.1 Lecture 2
1 Review Session Note: The following lists give an overview of the material that was covered in the lectures and sections. Your TF will go through these lists. If anything is unclear or you have questions
More informationChap6 Duality Theory and Sensitivity Analysis
Chap6 Duality Theory and Sensitivity Analysis The rationale of duality theory Max 4x 1 + x 2 + 5x 3 + 3x 4 S.T. x 1 x 2 x 3 + 3x 4 1 5x 1 + x 2 + 3x 3 + 8x 4 55 x 1 + 2x 2 + 3x 3 5x 4 3 x 1 ~x 4 0 If we
More information3. Linear Programming and Polyhedral Combinatorics
Massachusetts Institute of Technology 18.453: Combinatorial Optimization Michel X. Goemans April 5, 2017 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory
More informationIntroduction to Linear and Combinatorial Optimization (ADM I)
Introduction to Linear and Combinatorial Optimization (ADM I) Rolf Möhring based on the 20011/12 course by Martin Skutella TU Berlin WS 2013/14 1 General Remarks new flavor of ADM I introduce linear and
More informationLecture 6 Simplex method for linear programming
Lecture 6 Simplex method for linear programming Weinan E 1,2 and Tiejun Li 2 1 Department of Mathematics, Princeton University, weinan@princeton.edu 2 School of Mathematical Sciences, Peking University,
More informationAppendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS
Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS Here we consider systems of linear constraints, consisting of equations or inequalities or both. A feasible solution
More informationApplications. Stephen J. Stoyan, Maged M. Dessouky*, and Xiaoqing Wang
Introduction to Large-Scale Linear Programming and Applications Stephen J. Stoyan, Maged M. Dessouky*, and Xiaoqing Wang Daniel J. Epstein Department of Industrial and Systems Engineering, University of
More informationA Parametric Simplex Algorithm for Linear Vector Optimization Problems
A Parametric Simplex Algorithm for Linear Vector Optimization Problems Birgit Rudloff Firdevs Ulus Robert Vanderbei July 9, 2015 Abstract In this paper, a parametric simplex algorithm for solving linear
More informationOperations Research Lecture 2: Linear Programming Simplex Method
Operations Research Lecture 2: Linear Programming Simplex Method Notes taken by Kaiquan Xu@Business School, Nanjing University Mar 10th 2016 1 Geometry of LP 1.1 Graphical Representation and Solution Example
More informationPreliminaries and Complexity Theory
Preliminaries and Complexity Theory Oleksandr Romanko CAS 746 - Advanced Topics in Combinatorial Optimization McMaster University, January 16, 2006 Introduction Book structure: 2 Part I Linear Algebra
More informationMS-E2140. Lecture 1. (course book chapters )
Linear Programming MS-E2140 Motivations and background Lecture 1 (course book chapters 1.1-1.4) Linear programming problems and examples Problem manipulations and standard form Graphical representation
More informationProperties of a Simple Variant of Klee-Minty s LP and Their Proof
Properties of a Simple Variant of Klee-Minty s LP and Their Proof Tomonari Kitahara and Shinji Mizuno December 28, 2 Abstract Kitahara and Mizuno (2) presents a simple variant of Klee- Minty s LP, which
More informationLinear Algebra Review: Linear Independence. IE418 Integer Programming. Linear Algebra Review: Subspaces. Linear Algebra Review: Affine Independence
Linear Algebra Review: Linear Independence IE418: Integer Programming Department of Industrial and Systems Engineering Lehigh University 21st March 2005 A finite collection of vectors x 1,..., x k R n
More informationAssignment 1: From the Definition of Convexity to Helley Theorem
Assignment 1: From the Definition of Convexity to Helley Theorem Exercise 1 Mark in the following list the sets which are convex: 1. {x R 2 : x 1 + i 2 x 2 1, i = 1,..., 10} 2. {x R 2 : x 2 1 + 2ix 1x
More informationSimplex method(s) for solving LPs in standard form
Simplex method: outline I The Simplex Method is a family of algorithms for solving LPs in standard form (and their duals) I Goal: identify an optimal basis, as in Definition 3.3 I Versions we will consider:
More informationThe Simplex Algorithm: Technicalities 1
1/45 The Simplex Algorithm: Technicalities 1 Adrian Vetta 1 This presentation is based upon the book Linear Programming by Vasek Chvatal 2/45 Two Issues Here we discuss two potential problems with the
More informationLP. Lecture 3. Chapter 3: degeneracy. degeneracy example cycling the lexicographic method other pivot rules the fundamental theorem of LP
LP. Lecture 3. Chapter 3: degeneracy. degeneracy example cycling the lexicographic method other pivot rules the fundamental theorem of LP 1 / 23 Repetition the simplex algorithm: sequence of pivots starting
More informationLecture #21. c T x Ax b. maximize subject to
COMPSCI 330: Design and Analysis of Algorithms 11/11/2014 Lecture #21 Lecturer: Debmalya Panigrahi Scribe: Samuel Haney 1 Overview In this lecture, we discuss linear programming. We first show that the
More informationAlgorithmic Convex Geometry
Algorithmic Convex Geometry August 2011 2 Contents 1 Overview 5 1.1 Learning by random sampling.................. 5 2 The Brunn-Minkowski Inequality 7 2.1 The inequality.......................... 8 2.1.1
More informationLinear Programming. 1 An Introduction to Linear Programming
18.415/6.854 Advanced Algorithms October 1994 Lecturer: Michel X. Goemans Linear Programming 1 An Introduction to Linear Programming Linear programming is a very important class of problems, both algorithmically
More informationExample. 1 Rows 1,..., m of the simplex tableau remain lexicographically positive
3.4 Anticycling Lexicographic order In this section we discuss two pivoting rules that are guaranteed to avoid cycling. These are the lexicographic rule and Bland s rule. Definition A vector u R n is lexicographically
More informationLecture: Algorithms for LP, SOCP and SDP
1/53 Lecture: Algorithms for LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationMATH 4211/6211 Optimization Linear Programming
MATH 4211/6211 Optimization Linear Programming Xiaojing Ye Department of Mathematics & Statistics Georgia State University Xiaojing Ye, Math & Stat, Georgia State University 0 The standard form of a Linear
More informationOptimisation and Operations Research
Optimisation and Operations Research Lecture 5: The Simplex Algorithm Matthew Roughan http://www.maths.adelaide.edu.au/matthew.roughan/ Lecture_notes/OORII/ School of
More information15-850: Advanced Algorithms CMU, Fall 2018 HW #4 (out October 17, 2018) Due: October 28, 2018
15-850: Advanced Algorithms CMU, Fall 2018 HW #4 (out October 17, 2018) Due: October 28, 2018 Usual rules. :) Exercises 1. Lots of Flows. Suppose you wanted to find an approximate solution to the following
More information15.083J/6.859J Integer Optimization. Lecture 2: Efficient Algorithms and Computational Complexity
15.083J/6.859J Integer Optimization Lecture 2: Efficient Algorithms and Computational Complexity 1 Outline Efficient algorithms Slide 1 Complexity The classes P and N P The classes N P-complete and N P-hard
More informationLecture 6 - Convex Sets
Lecture 6 - Convex Sets Definition A set C R n is called convex if for any x, y C and λ [0, 1], the point λx + (1 λ)y belongs to C. The above definition is equivalent to saying that for any x, y C, the
More informationIntroduction to Operations Research
Introduction to Operations Research (Week 4: Linear Programming: More on Simplex and Post-Optimality) José Rui Figueira Instituto Superior Técnico Universidade de Lisboa (figueira@tecnico.ulisboa.pt) March
More informationInteger Programming ISE 418. Lecture 13. Dr. Ted Ralphs
Integer Programming ISE 418 Lecture 13 Dr. Ted Ralphs ISE 418 Lecture 13 1 Reading for This Lecture Nemhauser and Wolsey Sections II.1.1-II.1.3, II.1.6 Wolsey Chapter 8 CCZ Chapters 5 and 6 Valid Inequalities
More informationThe Master Equality Polyhedron: Two-Slope Facets and Separation Algorithm
The Master Equality Polyhedron: Two-Slope Facets and Separation Algorithm by Xiaojing Wang A thesis presented to the University of Waterloo in fulfillment of the thesis requirement for the degree of Master
More information1 Column Generation and the Cutting Stock Problem
1 Column Generation and the Cutting Stock Problem In the linear programming approach to the traveling salesman problem we used the cutting plane approach. The cutting plane approach is appropriate when
More information