Lecture slides by Kevin Wayne
|
|
- Franklin James
- 6 years ago
- Views:
Transcription
1 LINEAR PROGRAMMING I a refreshing example standard form fundamental questions geometry linear algebra simplex algorithm Lecture slides by Kevin Wayne Last updated on 7/25/17 11:09 AM
2 Linear programming Linear programming. Optimize a linear function subject to linear inequalities. (P) max c j x j n j=1 n s. t. a ij x j = b i 1 i m j=1 x j 0 1 j n (P) max c T x s. t. Ax = b x 0 2
3 Linear programming Linear programming. Optimize a linear function subject to linear inequalities. Generalizes: Ax = b, 2-person zero-sum games, shortest path, max flow, assignment problem, matching, multicommodity flow, MST, min weighted arborescence, Why significant? Design poly-time algorithms. Design approximation algorithms. Solve NP-hard problems using branch-and-cut. Ranked among most important scientific advances of 20th century. 3
4 LINEAR PROGRAMMING I a refreshing example standard form fundamental questions geometry linear algebra simplex algorithm Reference: The Allocation of Resources by Linear Programming, Scientific American, by Bob Bland
5 Brewery problem Small brewery produces ale and beer. Production limited by scarce resources: corn, hops, barley malt. Recipes for ale and beer require different proportions of resources. Beverage Corn (pounds) Hops (ounces) Malt (pounds) Profit () Ale (barrel) Beer (barrel) constraint How can brewer maximize profits? Devote all resources to ale: 34 barrels of ale 442 Devote all resources to beer: 32 barrels of beer barrels of ale, 29.5 barrels of beer barrels of ale, 28 barrels of beer 800 5
6 Brewery problem objective function Ale Beer max 13A + 23B s. t. 5A + 15B 480 4A + 4B A + 20B 1190 A, B 0 Profit Corn Hops Malt constraint decision variable 6
7 Reference 7
8 LINEAR PROGRAMMING I a refreshing example standard form fundamental questions geometry linear algebra simplex algorithm
9 Standard form of a linear program Standard form of an LP. Input: real numbers a ij, c j, b i. Output: real numbers x j. n = # decision variables, m = # constraints. Maximize linear objective function subject to linear inequalities. (P) max c j x j n j=1 n s. t. a ij x j = b i 1 i m j=1 x j 0 1 j n (P) max c T x s. t. Ax = b x 0 Linear. No x 2, x y, arccos(x), etc. Programming. Planning (term predates computer programming). 9
10 Brewery problem: converting to standard form Original input. max 13A + 23B s. t. 5A + 15B 480 4A + 4B A + 20B 1190 A, B 0 Standard form. Add slack variable for each inequality. Now a 5-dimensional problem. max 13A + 23B s. t. 5A + 15B + S C = 480 4A + 4B + S H = A + 20B + S M = 1190 A, B, S C, S H, S M 0 10
11 Equivalent forms Easy to convert variants to standard form. (P) max c T x s. t. Ax = b x 0 Less than to equality. x + 2y 3z 17 x + 2y 3z + s = 17, s 0 Greater than to equality. x + 2y 3z 17 x + 2y 3z s = 17, s 0 Min to max. min x + 2y 3z max x 2y + 3z Unrestricted to nonnegative. x unrestricted x = x + x, x + 0, x 0 11
12 LINEAR PROGRAMMING I a refreshing example standard form fundamental questions geometry linear algebra simplex algorithm
13 Fundamental questions LP. For A R m n, b R m, c R n, α R, does there exist x R n such that: Ax = b, x 0, c T x α? Q. Is LP in NP? Q. Is LP in co-np? Q. Is LP in P? Q. Is LP in P R? Blum Shub Smale model Input size. n = number of variables. m = number of constraints. L = number of bits to encode input. 13
14 LINEAR PROGRAMMING I a refreshing example standard form fundamental questions geometry linear algebra simplex algorithm
15 Brewery problem: feasible region Hops 4A + 4B 160 Malt 35A + 20B 1190 (0, 32) (12, 28) (26, 14) Corn 5A + 15B 480 Beer (0, 0) Ale (34, 0) 15
16 Brewery problem: objective function Profit (0, 32) (12, 28) 13A + 23B = 1600 (26, 14) Beer 13A + 23B = 800 (0, 0) Ale (34, 0) 13A + 23B =
17 Brewery problem: geometry Brewery problem observation. Regardless of objective function coefficients, an optimal solution occurs at a vertex. (0, 32) (12, 28) (26, 14) vertex Beer (0, 0) Ale (34, 0) 17
18 Convexity Convex set. If two points x and y are in the set, then so is λ x + (1 λ) y for 0 λ 1. convex combination not a vertex iff d 0 s.t. x ± d in set Vertex. A point x in the set that can t be written as a strict convex combination of two distinct points in the set. vertex x y convex not convex Observation. LP feasible region is a convex set. 18
19 Purificaiton Theorem. If there exists an optimal solution to (P), then there exists one that is a vertex. (P) max c T x s. t. Ax = b x 0 Intuition. If x is not a vertex, move in a non-decreasing direction until you reach a boundary. Repeat. xʹ = x + α * d x + d x x - d 19
20 Purificaiton Theorem. If there exists an optimal solution to (P), then there exists one that is a vertex. Pf. Suppose x is an optimal solution that is not a vertex. There exist direction d 0 such that x ± d P. A d = 0 because A(x ± d) = b. Assume c T d 0 (by taking either d or d). Consider x + λ d, λ > 0 : Case 1. [ there exists j such that d j < 0 ] Increase λ to λ * until first new component of x + λ d hits 0. x + λ * d is feasible since A(x + λ * d) = Ax = b and x + λ * y 0. x + λ * d has one more zero component than x. c T xʹ = c T (x + λ * d) = c T x + λ * c T d c T x. d k = 0 whenever x k = 0 because x ± d P 20
21 Purificaiton Theorem. If there exists an optimal solution to (P), then there exists one that is a vertex. Pf. Suppose x is an optimal solution that is not a vertex. There exist direction d 0 such that x ± d P. A d = 0 because A(x ± d) = b. Assume c T d 0 (by taking either d or d). Consider x + λ d, λ > 0 : Case 2. [d j 0 for all j ] x + λd is feasible for all λ 0 since A(x + λd) = b and x + λd x 0. As λ, c T (x + λd) because c T d < 0. if c T d = 0, choose d so that case 1 applies 21
22 LINEAR PROGRAMMING I a refreshing example standard form fundamental questions geometry linear algebra simplex algorithm
23 Intuition Intuition. A vertex in R m is uniquely specified by m linearly independent equations. 4A + 4B A + 20B 1190 (26, 14) 4A + 4B = A + 20B =
24 Basic feasible solution Theorem. Let P = { x : Ax = b, x 0 }. For x P, define B = { j : x j > 0 }. Then, x is a vertex iff A B has linearly independent columns. Notation. Let B = set of column indices. Define A B to be the subset of columns of A indexed by B. Ex. A = " % # &, b = " 7 % 16 # 0 & x = " 2 % 0 1 # 0 &, B = {1, 3}, A B = " 2 3% 7 2 # 0 0& 24
25 Basic feasible solution Theorem. Let P = { x : Ax = b, x 0 }. For x P, define B = { j : x j > 0 }. Then, x is a vertex iff A B has linearly independent columns. Pf. Assume x is not a vertex. There exist direction d 0 such that x ± d P. A d = 0 because A(x ± d) = b. Define Bʹ = { j : d j 0 }. A Bʹ has linearly dependent columns since d 0. Moreover, d j = 0 whenever x j = 0 because x ± d 0. Thus Bʹ B, so A Bʹ is a submatrix of A B. Therefore, A B has linearly dependent columns. 25
26 Basic feasible solution Theorem. Let P = { x : Ax = b, x 0 }. For x P, define B = { j : x j > 0 }. Then, x is a vertex iff A B has linearly independent columns. Pf. Assume A B has linearly dependent columns. There exist d 0 such that A B d = 0. Extend d to R n by adding 0 components. Now, A d = 0 and d j = 0 whenever x j = 0. For sufficiently small λ, x ± λ d P x is not a vertex. 26
27 Basic feasible solution Theorem. Given P = { x : Ax = b, x 0 }, x is a vertex iff there exists B { 1,, n } such B = m and: A B is nonsingular. x B = A B -1 b 0. x N = 0. basic feasible solution Pf. Augment A B with linearly independent columns (if needed). x = " # " % " 7 % A = , b = 16 # & # 0 & % " 2 3 0, B = { 1, 3, 4 }, A B = # & % & Assumption. A R m n has full row rank. 27
28 Basic feasible solution: example Basic feasible solutions. {B, S H, S M } (0, 32) Basis {A, B, S M } (12, 28) Infeasible {A, B, S H } (19.41, 25.53) max 13A + 23B s. t. 5A + 15B + S C = 480 4A + 4B + S H = A + 20B + S M = 1190 A, B, S C, S H, S M 0 Beer {A, B, S C } (26, 14) {S H, S M, S C } (0, 0) Ale {A, S H, S C } (34, 0) 28
29 Fundamental questions LP. For A R m n, b R m, c R n, α R, does there exist x R n such that: Ax = b, x 0, c T x α? Q. Is LP in NP? A. Yes. Number of vertices ( ) nm. C(n, m) = n m Cramer s rule can check a vertex in poly-time. Cramer s rule. For B R n n invertible, b R n, the solution to Bx = b is given by: x i = det(b i ) det(b) replace ith column of B with b 29
30 LINEAR PROGRAMMING I a refreshing example standard form fundamental questions geometry linear algebra simplex algorithm
31 Simplex algorithm: intuition Simplex algorithm. [George Dantzig 1947] Move from BFS to adjacent BFS, without decreasing objective function. replace one basic variable with another edge Greedy property. BFS optimal iff no adjacent BFS is better. Challenge. Number of BFS can be exponential! 31
32 Simplex algorithm: initialization max Z subject to 13A + 23B Z = 0 5A + 15B + S C = 480 4A + 4B + S H = A + 20B + S M = 1190 A, B, S C, S H, S M 0 Basis = {S C, S H, S M } A = B = 0 Z = 0 S C = 480 S H = 160 S M =
33 Simplex algorithm: pivot 1 max Z subject to 13A + 23B Z = 0 5A + 15B + S C = 480 4A + 4B + S H = A + 20B + S M = 1190 A, B, S C, S H, S M 0 Basis = {S C, S H, S M } A = B = 0 Z = 0 S C = 480 S H = 160 S M = 1190 Substitute: B = 1/15 (480 5A S C ) max Z subject to 16 3 A S C Z = A + B + 1 S 3 15 C = A 4 15 S C + S H = A 4 3 S C + S M = 550 A, B, S C, S H, S M 0 Basis = {B, S H, S M } A = S C = 0 Z = 736 B = 32 S H = 32 S M =
34 Simplex algorithm: pivot 1 max Z subject to 13A + 23B Z = 0 5A + 15B + S C = 480 4A + 4B + S H = A + 20B + S M = 1190 A, B, S C, S H, S M 0 Basis = {S C, S H, S M } A = B = 0 Z = 0 S C = 480 S H = 160 S M = 1190 Q. Why pivot on column 2 (or 1)? A. Each unit increase in B increases objective value by 23. Q. Why pivot on row 2? A. Preserves feasibility by ensuring RHS 0. min ratio rule: min { 480/15, 160/4, 1190/20 } 34
35 Simplex algorithm: pivot 2 max Z subject to 16 3 A S C Z = A + B + 1 S 3 15 C = A 4 15 S C + S H = A 4 3 S C + S M = 550 A, B, S C, S H, S M 0 Basis = {B, S H, S M } A = S C = 0 Z = 736 B = 32 S H = 32 S M = 550 Substitute: A = 3/8 (32 + 4/15 S C S H ) max Z subject to S C 2 S H Z = 800 B S C S H = 28 A 1 10 C S H = C 85 8 S H + S M = 110 A, B, S C, S H, S M 0 Basis = {A, B, S M } S C = S H = 0 Z = 800 B = 28 A = 12 S M =
36 Simplex algorithm: optimality Q. When to stop pivoting? A. When all coefficients in top row are nonpositive. Q. Why is resulting solution optimal? A. Any feasible solution satisfies system of equations in tableaux. In particular: Z = 800 S C 2 S H, S C 0, S H 0. Thus, optimal objective value Z* 800. Current BFS has value 800 optimal. max Z subject to S C 2 S H Z = 800 B S C S H = 28 A 1 10 C S H = C 85 8 S H + S M = 110 A, B, S C, S H, S M 0 Basis = {A, B, S M } S C = S H = 0 Z = 800 B = 28 A = 12 S M =
37 Simplex tableaux: matrix form Initial simplex tableaux. c B T x B + c N T x N = Z A B x B + A N x N = b x B, x N 0 Simplex tableaux corresponding to basis B. (c N T c B T A B 1 A N ) x N = Z c B T A B 1 b I x B + A 1 B A N x N = A 1 B b x B, x N 0 subtract c B T A B -1 times constraints multiply by A B -1 x B = A B -1 b 0 x N = 0 basic feasible solution c N T c B T A B -1 A N 0 optimal basis 37
38 Simplex algorithm: corner cases Simplex algorithm. Missing details for corner cases. Q. What if min ratio test fails? Q. How to find initial basis? Q. How to guarantee termination? 38
39 Unboundedness Q. What happens if min ratio test fails? all coefficients in entering column are nonpositive max Z subject to + 2x x 5 Z = 2 x 1 4x 4 8x 5 = 3 x 2 + 5x 4 12x 5 = 4 x 3 = 5 x 1, x 2, x 3, x 4, x 5 0 A. Unbounded objective function. Z = x 5 " # x 1 x 2 x 3 x 4 x 5 % & = " # 3 + 8x x % & 39
40 Phase I simplex method Q. How to find initial basis? (P) max c T x s. t. Ax = b x 0 A. Solve (P ), starting from basis consisting of all the z i variables. m ( P ") max z i i=1 s. t. A x + I z = b x, z 0 Case 1: min > 0 (P) is infeasible. Case 2: min = 0, basis has no z i variables okay to start Phase II. Case 3a: min = 0, basis has z i variables. Pivot z i variables out of basis. If successful, start Phase II; else remove linear dependent rows. 40
41 Simplex algorithm: degeneracy Degeneracy. New basis, same vertex. Degenerate pivot. Min ratio = 0. max Z subject to 3 x x x 2 6 6x 7 Z = 0 x x 4 4 8x 5 x 6 + 9x 7 = 0 x x 4 12x x 6 + 3x 7 = 0 x 3 + x 6 = 1 x 1, x 2, x 3, x 4, x 5, x 6, x
42 Simplex algorithm: degeneracy Degeneracy. New basis, same vertex. Cycling. Infinite loop by cycling through different bases that all correspond to same vertex. Anti-cycling rules. Bland s rule: choose eligible variable with smallest index. Random rule: choose eligible variable uniformly at random. Lexicographic rule: perturb constraints so nondegenerate. 42
43 Lexicographic rule Intuition. No degeneracy no cycling. Perturbed problem. much much greater, say ε i = δ i for small δ ( P ") max c T x s. t. Ax = b + ε x 0 where ε = # ε 1 % % ε 2 %! % ε n & ( ( ( (, such that ε 1 ε 2 # ε n Lexicographic rule. Apply perturbation virtually by manipulating ε symbolically: ε ε 2 + 8ε ε ε 2 + 3ε 3 43
44 Lexicographic rule Intuition. No degeneracy no cycling. Perturbed problem. much much greater, say ε i = δ i for small δ ( P ") max c T x s. t. Ax = b + ε x 0 where ε = # ε 1 % % ε 2 %! % ε n & ( ( ( (, such that ε 1 ε 2 # ε n Claim. In perturbed problem, x B = A B -1 (b + ε) is always nonzero. Pf. The j th component of x B is a (nonzero) linear combination of the components of b + ε contains at least one of the ε i terms. Corollary. No cycling. which can t cancel 44
45 Simplex algorithm: practice Remarkable property. In practice, simplex algorithm typically terminates after at most 2(m + n) pivots. but no polynomial pivot rule known Issues. Avoid stalling. Choose the pivot. Maintain sparsity. Ensure numerical stability. Preprocess to eliminate variables and constraints. Commercial solvers can solve LPs with millions of variables and tens of thousands of constraints. 45
LINEAR PROGRAMMING I. a refreshing example standard form fundamental questions geometry linear algebra simplex algorithm
Linear programming Linear programming. Optimize a linear function subject to linear inequalities. (P) max c j x j n j= n s. t. a ij x j = b i i m j= x j 0 j n (P) max c T x s. t. Ax = b Lecture slides
More informationLinear Programming. Linear Programming I. Lecture 1. Linear Programming. Linear Programming
Linear Programming Linear Programming Lecture Linear programming. Optimize a linear function subject to linear inequalities. (P) max " c j x j n j= n s. t. " a ij x j = b i # i # m j= x j 0 # j # n (P)
More informationLINEAR PROGRAMMING II
LINEAR PROGRAMMING II LP duality strong duality theorem bonus proof of LP duality applications Lecture slides by Kevin Wayne Last updated on 7/25/17 11:09 AM LINEAR PROGRAMMING II LP duality Strong duality
More information! Fast commercial solvers: CPLEX, OSL. ! Powerful modeling languages: AMPL, GAMS. ! Ranked among most important scientific advances of 20 th century.
Linear Programming Linear Programming see ORF 07 What is it?! Quintessential tool for optimal allocation of scarce resources, among a number of competing activities.! Powerful and general problem-solving
More informationAlgorithms and Theory of Computation. Lecture 13: Linear Programming (2)
Algorithms and Theory of Computation Lecture 13: Linear Programming (2) Xiaohui Bei MAS 714 September 25, 2018 Nanyang Technological University MAS 714 September 25, 2018 1 / 15 LP Duality Primal problem
More informationPart 1. The Review of Linear Programming
In the name of God Part 1. The Review of Linear Programming 1.2. Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Basic Feasible Solutions Key to the Algebra of the The Simplex Algorithm
More informationYinyu Ye, MS&E, Stanford MS&E310 Lecture Note #06. The Simplex Method
The Simplex Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapters 2.3-2.5, 3.1-3.4) 1 Geometry of Linear
More informationClass Meeting #20 COS 226 Spring 2018
Class Meeting #20 COS 226 Spring 2018 Mark Braverman (based on slides by Robert Sedgewick and Kevin Wayne) Linear programming A Swiss army knife for optimization algorithms. Can solve a large fraction
More informationORF 522. Linear Programming and Convex Analysis
ORF 5 Linear Programming and Convex Analysis Initial solution and particular cases Marco Cuturi Princeton ORF-5 Reminder: Tableaux At each iteration, a tableau for an LP in standard form keeps track of....................
More informationLinear Programming. Linear Programming. Brewery Problem: A Toy LP Example. Applications
Linear Programming Linear Programming Linear programming implex method LP duality What is it? Quintessential tool for optimal allocation of scarce resources among a number of competing activities. (e.g.
More informationTIM 206 Lecture 3: The Simplex Method
TIM 206 Lecture 3: The Simplex Method Kevin Ross. Scribe: Shane Brennan (2006) September 29, 2011 1 Basic Feasible Solutions Have equation Ax = b contain more columns (variables) than rows (constraints),
More informationOptimization (168) Lecture 7-8-9
Optimization (168) Lecture 7-8-9 Jesús De Loera UC Davis, Mathematics Wednesday, April 2, 2012 1 DEGENERACY IN THE SIMPLEX METHOD 2 DEGENERACY z =2x 1 x 2 + 8x 3 x 4 =1 2x 3 x 5 =3 2x 1 + 4x 2 6x 3 x 6
More informationStandard Form An LP is in standard form when: All variables are non-negativenegative All constraints are equalities Putting an LP formulation into sta
Chapter 4 Linear Programming: The Simplex Method An Overview of the Simplex Method Standard Form Tableau Form Setting Up the Initial Simplex Tableau Improving the Solution Calculating the Next Tableau
More information3 The Simplex Method. 3.1 Basic Solutions
3 The Simplex Method 3.1 Basic Solutions In the LP of Example 2.3, the optimal solution happened to lie at an extreme point of the feasible set. This was not a coincidence. Consider an LP in general form,
More information1 Review Session. 1.1 Lecture 2
1 Review Session Note: The following lists give an overview of the material that was covered in the lectures and sections. Your TF will go through these lists. If anything is unclear or you have questions
More informationSimplex Algorithm Using Canonical Tableaus
41 Simplex Algorithm Using Canonical Tableaus Consider LP in standard form: Min z = cx + α subject to Ax = b where A m n has rank m and α is a constant In tableau form we record it as below Original Tableau
More informationFebruary 17, Simplex Method Continued
15.053 February 17, 2005 Simplex Method Continued 1 Today s Lecture Review of the simplex algorithm. Formalizing the approach Alternative Optimal Solutions Obtaining an initial bfs Is the simplex algorithm
More informationDr. Maddah ENMG 500 Engineering Management I 10/21/07
Dr. Maddah ENMG 500 Engineering Management I 10/21/07 Computational Procedure of the Simplex Method The optimal solution of a general LP problem is obtained in the following steps: Step 1. Express the
More informationLINEAR PROGRAMMING III
LINEAR PROGRAMMING III ellipsoid algorithm combinatorial optimization matrix games open problems Lecture slides by Kevin Wayne Last updated on 7/25/17 11:09 AM LINEAR PROGRAMMING III ellipsoid algorithm
More informationThe simplex algorithm
The simplex algorithm The simplex algorithm is the classical method for solving linear programs. Its running time is not polynomial in the worst case. It does yield insight into linear programs, however,
More informationLP. Lecture 3. Chapter 3: degeneracy. degeneracy example cycling the lexicographic method other pivot rules the fundamental theorem of LP
LP. Lecture 3. Chapter 3: degeneracy. degeneracy example cycling the lexicographic method other pivot rules the fundamental theorem of LP 1 / 23 Repetition the simplex algorithm: sequence of pivots starting
More informationCSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming
CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming January 26, 2018 1 / 38 Liability/asset cash-flow matching problem Recall the formulation of the problem: max w c 1 + p 1 e 1 = 150
More information4.5 Simplex method. LP in standard form: min z = c T x s.t. Ax = b
4.5 Simplex method LP in standard form: min z = c T x s.t. Ax = b x 0 George Dantzig (1914-2005) Examine a sequence of basic feasible solutions with non increasing objective function values until an optimal
More informationCS 580: Algorithm Design and Analysis. Jeremiah Blocki Purdue University Spring 2018
CS 580: Algorithm Design and Analysis Jeremiah Blocki Purdue University Spring 2018 Recap Network Flow Problems Max-Flow Min Cut Theorem Ford Fulkerson Augmenting Paths Residual Flow Graph Integral Solutions
More informationSimplex method(s) for solving LPs in standard form
Simplex method: outline I The Simplex Method is a family of algorithms for solving LPs in standard form (and their duals) I Goal: identify an optimal basis, as in Definition 3.3 I Versions we will consider:
More informationNotes taken by Graham Taylor. January 22, 2005
CSC4 - Linear Programming and Combinatorial Optimization Lecture : Different forms of LP. The algebraic objects behind LP. Basic Feasible Solutions Notes taken by Graham Taylor January, 5 Summary: We first
More informationMotivating examples Introduction to algorithms Simplex algorithm. On a particular example General algorithm. Duality An application to game theory
Instructor: Shengyu Zhang 1 LP Motivating examples Introduction to algorithms Simplex algorithm On a particular example General algorithm Duality An application to game theory 2 Example 1: profit maximization
More informationCSC373: Algorithm Design, Analysis and Complexity Fall 2017 DENIS PANKRATOV NOVEMBER 1, 2017
CSC373: Algorithm Design, Analysis and Complexity Fall 2017 DENIS PANKRATOV NOVEMBER 1, 2017 Linear Function f: R n R is linear if it can be written as f x = a T x for some a R n Example: f x 1, x 2 =
More informationDistributed Real-Time Control Systems. Lecture Distributed Control Linear Programming
Distributed Real-Time Control Systems Lecture 13-14 Distributed Control Linear Programming 1 Linear Programs Optimize a linear function subject to a set of linear (affine) constraints. Many problems can
More informationIE 400: Principles of Engineering Management. Simplex Method Continued
IE 400: Principles of Engineering Management Simplex Method Continued 1 Agenda Simplex for min problems Alternative optimal solutions Unboundedness Degeneracy Big M method Two phase method 2 Simplex for
More informationΩ R n is called the constraint set or feasible set. x 1
1 Chapter 5 Linear Programming (LP) General constrained optimization problem: minimize subject to f(x) x Ω Ω R n is called the constraint set or feasible set. any point x Ω is called a feasible point We
More informationOPERATIONS RESEARCH. Linear Programming Problem
OPERATIONS RESEARCH Chapter 1 Linear Programming Problem Prof. Bibhas C. Giri Department of Mathematics Jadavpur University Kolkata, India Email: bcgiri.jumath@gmail.com MODULE - 2: Simplex Method for
More informationChap6 Duality Theory and Sensitivity Analysis
Chap6 Duality Theory and Sensitivity Analysis The rationale of duality theory Max 4x 1 + x 2 + 5x 3 + 3x 4 S.T. x 1 x 2 x 3 + 3x 4 1 5x 1 + x 2 + 3x 3 + 8x 4 55 x 1 + 2x 2 + 3x 3 5x 4 3 x 1 ~x 4 0 If we
More informationCS Algorithms and Complexity
CS 50 - Algorithms and Complexity Linear Programming, the Simplex Method, and Hard Problems Sean Anderson 2/15/18 Portland State University Table of contents 1. The Simplex Method 2. The Graph Problem
More informationIII. Linear Programming
III. Linear Programming Thomas Sauerwald Easter 2017 Outline Introduction Standard and Slack Forms Formulating Problems as Linear Programs Simplex Algorithm Finding an Initial Solution III. Linear Programming
More informationLecture: Algorithms for LP, SOCP and SDP
1/53 Lecture: Algorithms for LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationMath 273a: Optimization The Simplex method
Math 273a: Optimization The Simplex method Instructor: Wotao Yin Department of Mathematics, UCLA Fall 2015 material taken from the textbook Chong-Zak, 4th Ed. Overview: idea and approach If a standard-form
More informationCO350 Linear Programming Chapter 8: Degeneracy and Finite Termination
CO350 Linear Programming Chapter 8: Degeneracy and Finite Termination 27th June 2005 Chapter 8: Finite Termination 1 The perturbation method Recap max c T x (P ) s.t. Ax = b x 0 Assumption: B is a feasible
More informationSupplementary lecture notes on linear programming. We will present an algorithm to solve linear programs of the form. maximize.
Cornell University, Fall 2016 Supplementary lecture notes on linear programming CS 6820: Algorithms 26 Sep 28 Sep 1 The Simplex Method We will present an algorithm to solve linear programs of the form
More informationChapter 3, Operations Research (OR)
Chapter 3, Operations Research (OR) Kent Andersen February 7, 2007 1 Linear Programs (continued) In the last chapter, we introduced the general form of a linear program, which we denote (P) Minimize Z
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 5: The Simplex method, continued Prof. John Gunnar Carlsson September 22, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 22, 2010
More informationLinear and Integer Programming - ideas
Linear and Integer Programming - ideas Paweł Zieliński Institute of Mathematics and Computer Science, Wrocław University of Technology, Poland http://www.im.pwr.wroc.pl/ pziel/ Toulouse, France 2012 Literature
More informationContents. 4.5 The(Primal)SimplexMethod NumericalExamplesoftheSimplexMethod
Contents 4 The Simplex Method for Solving LPs 149 4.1 Transformations to be Carried Out On an LP Model Before Applying the Simplex Method On It... 151 4.2 Definitions of Various Types of Basic Vectors
More information3. THE SIMPLEX ALGORITHM
Optimization. THE SIMPLEX ALGORITHM DPK Easter Term. Introduction We know that, if a linear programming problem has a finite optimal solution, it has an optimal solution at a basic feasible solution (b.f.s.).
More informationThe Simplex Method. Standard form (max) z c T x = 0 such that Ax = b.
The Simplex Method Standard form (max) z c T x = 0 such that Ax = b. The Simplex Method Standard form (max) z c T x = 0 such that Ax = b. Build initial tableau. z c T 0 0 A b The Simplex Method Standard
More information15-780: LinearProgramming
15-780: LinearProgramming J. Zico Kolter February 1-3, 2016 1 Outline Introduction Some linear algebra review Linear programming Simplex algorithm Duality and dual simplex 2 Outline Introduction Some linear
More informationMATH 445/545 Homework 2: Due March 3rd, 2016
MATH 445/545 Homework 2: Due March 3rd, 216 Answer the following questions. Please include the question with the solution (write or type them out doing this will help you digest the problem). I do not
More informationThe Simplex Method for Solving a Linear Program Prof. Stephen Graves
The Simplex Method for Solving a Linear Program Prof. Stephen Graves Observations from Geometry feasible region is a convex polyhedron an optimum occurs at a corner point possible algorithm - search over
More informationExample. 1 Rows 1,..., m of the simplex tableau remain lexicographically positive
3.4 Anticycling Lexicographic order In this section we discuss two pivoting rules that are guaranteed to avoid cycling. These are the lexicographic rule and Bland s rule. Definition A vector u R n is lexicographically
More informationLinear programs Optimization Geoff Gordon Ryan Tibshirani
Linear programs 10-725 Optimization Geoff Gordon Ryan Tibshirani Review: LPs LPs: m constraints, n vars A: R m n b: R m c: R n x: R n ineq form [min or max] c T x s.t. Ax b m n std form [min or max] c
More information3 Development of the Simplex Method Constructing Basic Solution Optimality Conditions The Simplex Method...
Contents Introduction to Linear Programming Problem. 2. General Linear Programming problems.............. 2.2 Formulation of LP problems.................... 8.3 Compact form and Standard form of a general
More informationECE 307 Techniques for Engineering Decisions
ECE 7 Techniques for Engineering Decisions Introduction to the Simple Algorithm George Gross Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign ECE 7 5 9 George
More informationGauss-Jordan Elimination for Solving Linear Equations Example: 1. Solve the following equations: (3)
The Simple Method Gauss-Jordan Elimination for Solving Linear Equations Eample: Gauss-Jordan Elimination Solve the following equations: + + + + = 4 = = () () () - In the first step of the procedure, we
More informationAM 121: Intro to Optimization Models and Methods Fall 2018
AM 121: Intro to Optimization Models and Methods Fall 2018 Lecture 5: The Simplex Method Yiling Chen Harvard SEAS Lesson Plan This lecture: Moving towards an algorithm for solving LPs Tableau. Adjacent
More informationThe Avis-Kalunzy Algorithm is designed to find a basic feasible solution (BFS) of a given set of constraints. Its input: A R m n and b R m such that
Lecture 4 Avis-Kaluzny and the Simplex Method Last time, we discussed some applications of Linear Programming, such as Max-Flow, Matching, and Vertex-Cover. The broad range of applications to Linear Programming
More informationMATH 4211/6211 Optimization Linear Programming
MATH 4211/6211 Optimization Linear Programming Xiaojing Ye Department of Mathematics & Statistics Georgia State University Xiaojing Ye, Math & Stat, Georgia State University 0 The standard form of a Linear
More information6. Linear Programming
6. Linear Programming brewer s problem simplex algorithm implementations duality modeling Algorithms, 4 th Edition Robert Sedgewick and Kevin Wayne Copyright 2002 2010 April 20, 2011 10:36:00 AM Overview:
More informationmin 4x 1 5x 2 + 3x 3 s.t. x 1 + 2x 2 + x 3 = 10 x 1 x 2 6 x 1 + 3x 2 + x 3 14
The exam is three hours long and consists of 4 exercises. The exam is graded on a scale 0-25 points, and the points assigned to each question are indicated in parenthesis within the text. If necessary,
More informationMATH2070 Optimisation
MATH2070 Optimisation Linear Programming Semester 2, 2012 Lecturer: I.W. Guo Lecture slides courtesy of J.R. Wishart Review The standard Linear Programming (LP) Problem Graphical method of solving LP problem
More informationAM 121: Intro to Optimization
AM 121: Intro to Optimization Models and Methods Lecture 6: Phase I, degeneracy, smallest subscript rule. Yiling Chen SEAS Lesson Plan Phase 1 (initialization) Degeneracy and cycling Smallest subscript
More informationCOT 6936: Topics in Algorithms! Giri Narasimhan. ECS 254A / EC 2443; Phone: x3748
COT 6936: Topics in Algorithms! Giri Narasimhan ECS 254A / EC 2443; Phone: x3748 giri@cs.fiu.edu https://moodle.cis.fiu.edu/v2.1/course/view.php?id=612 Gaussian Elimination! Solving a system of simultaneous
More informationChapter 4 The Simplex Algorithm Part I
Chapter 4 The Simplex Algorithm Part I Based on Introduction to Mathematical Programming: Operations Research, Volume 1 4th edition, by Wayne L. Winston and Munirpallam Venkataramanan Lewis Ntaimo 1 Modeling
More informationDiscrete Optimization. Guyslain Naves
Discrete Optimization Guyslain Naves Fall 2010 Contents 1 The simplex method 5 1.1 The simplex method....................... 5 1.1.1 Standard linear program................. 9 1.1.2 Dictionaries........................
More informationSimplex Method for LP (II)
Simplex Method for LP (II) Xiaoxi Li Wuhan University Sept. 27, 2017 (week 4) Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 1 / 31 Organization of this lecture Contents:
More information1 The linear algebra of linear programs (March 15 and 22, 2015)
1 The linear algebra of linear programs (March 15 and 22, 2015) Many optimization problems can be formulated as linear programs. The main features of a linear program are the following: Variables are real
More informationAlgorithms. Linear Programming. brewer s problem simplex algorithm implementations duality modeling
Linear Programming Algorithms F O U R T H E D I T I O N brewer s problem simplex algorithm implementations duality modeling R O B E R T S E D G E W I C K K E V I N W A Y N E Algorithms, 4 th Edition Robert
More informationThe augmented form of this LP is the following linear system of equations:
1 Consider the following LP given in standard form: max z = 5 x_1 + 2 x_2 Subject to 3 x_1 + 2 x_2 2400 x_2 800 2 x_1 1200 x_1, x_2 >= 0 The augmented form of this LP is the following linear system of
More informationCO 602/CM 740: Fundamentals of Optimization Problem Set 4
CO 602/CM 740: Fundamentals of Optimization Problem Set 4 H. Wolkowicz Fall 2014. Handed out: Wednesday 2014-Oct-15. Due: Wednesday 2014-Oct-22 in class before lecture starts. Contents 1 Unique Optimum
More informationLinear Programming: Simplex
Linear Programming: Simplex Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. IMA, August 2016 Stephen Wright (UW-Madison) Linear Programming: Simplex IMA, August 2016
More informationChapter 5 Linear Programming (LP)
Chapter 5 Linear Programming (LP) General constrained optimization problem: minimize f(x) subject to x R n is called the constraint set or feasible set. any point x is called a feasible point We consider
More informationLecture 2: The Simplex method
Lecture 2 1 Linear and Combinatorial Optimization Lecture 2: The Simplex method Basic solution. The Simplex method (standardform, b>0). 1. Repetition of basic solution. 2. One step in the Simplex algorithm.
More informationPrelude to the Simplex Algorithm. The Algebraic Approach The search for extreme point solutions.
Prelude to the Simplex Algorithm The Algebraic Approach The search for extreme point solutions. 1 Linear Programming-1 x 2 12 8 (4,8) Max z = 6x 1 + 4x 2 Subj. to: x 1 + x 2
More informationOverview of course. Introduction to Optimization, DIKU Monday 12 November David Pisinger
Introduction to Optimization, DIKU 007-08 Monday November David Pisinger Lecture What is OR, linear models, standard form, slack form, simplex repetition, graphical interpretation, extreme points, basic
More informationLecture 4: Algebra, Geometry, and Complexity of the Simplex Method. Reading: Sections 2.6.4, 3.5,
Lecture 4: Algebra, Geometry, and Complexity of the Simplex Method Reading: Sections 2.6.4, 3.5, 10.2 10.5 1 Summary of the Phase I/Phase II Simplex Method We write a typical simplex tableau as z x 1 x
More informationLectures 6, 7 and part of 8
Lectures 6, 7 and part of 8 Uriel Feige April 26, May 3, May 10, 2015 1 Linear programming duality 1.1 The diet problem revisited Recall the diet problem from Lecture 1. There are n foods, m nutrients,
More informationOptimization WS 13/14:, by Y. Goldstein/K. Reinert, 9. Dezember 2013, 16: Linear programming. Optimization Problems
Optimization WS 13/14:, by Y. Goldstein/K. Reinert, 9. Dezember 2013, 16:38 2001 Linear programming Optimization Problems General optimization problem max{z(x) f j (x) 0,x D} or min{z(x) f j (x) 0,x D}
More informationSystems Analysis in Construction
Systems Analysis in Construction CB312 Construction & Building Engineering Department- AASTMT by A h m e d E l h a k e e m & M o h a m e d S a i e d 3. Linear Programming Optimization Simplex Method 135
More informationIntroduction to optimization
Introduction to optimization Geir Dahl CMA, Dept. of Mathematics and Dept. of Informatics University of Oslo 1 / 24 The plan 1. The basic concepts 2. Some useful tools (linear programming = linear optimization)
More informationSensitivity Analysis and Duality in LP
Sensitivity Analysis and Duality in LP Xiaoxi Li EMS & IAS, Wuhan University Oct. 13th, 2016 (week vi) Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 1 /
More informationCO350 Linear Programming Chapter 8: Degeneracy and Finite Termination
CO350 Linear Programming Chapter 8: Degeneracy and Finite Termination 22th June 2005 Chapter 8: Finite Termination Recap On Monday, we established In the absence of degeneracy, the simplex method will
More informationThe Primal-Dual Algorithm P&S Chapter 5 Last Revised October 30, 2006
The Primal-Dual Algorithm P&S Chapter 5 Last Revised October 30, 2006 1 Simplex solves LP by starting at a Basic Feasible Solution (BFS) and moving from BFS to BFS, always improving the objective function,
More informationLesson 27 Linear Programming; The Simplex Method
Lesson Linear Programming; The Simplex Method Math 0 April 9, 006 Setup A standard linear programming problem is to maximize the quantity c x + c x +... c n x n = c T x subject to constraints a x + a x
More informationSummary of the simplex method
MVE165/MMG631,Linear and integer optimization with applications The simplex method: degeneracy; unbounded solutions; starting solutions; infeasibility; alternative optimal solutions Ann-Brith Strömberg
More informationLinear Programming Redux
Linear Programming Redux Jim Bremer May 12, 2008 The purpose of these notes is to review the basics of linear programming and the simplex method in a clear, concise, and comprehensive way. The book contains
More informationIn Chapters 3 and 4 we introduced linear programming
SUPPLEMENT The Simplex Method CD3 In Chapters 3 and 4 we introduced linear programming and showed how models with two variables can be solved graphically. We relied on computer programs (WINQSB, Excel,
More informationDeveloping an Algorithm for LP Preamble to Section 3 (Simplex Method)
Moving from BFS to BFS Developing an Algorithm for LP Preamble to Section (Simplex Method) We consider LP given in standard form and let x 0 be a BFS. Let B ; B ; :::; B m be the columns of A corresponding
More informationExample Bases and Basic Feasible Solutions 63 Let q = >: ; > and M = >: ;2 > and consider the LCP (q M). The class of ; ;2 complementary cones
Chapter 2 THE COMPLEMENTARY PIVOT ALGORITHM AND ITS EXTENSION TO FIXED POINT COMPUTING LCPs of order 2 can be solved by drawing all the complementary cones in the q q 2 - plane as discussed in Chapter.
More information3. Linear Programming and Polyhedral Combinatorics
Massachusetts Institute of Technology 18.433: Combinatorial Optimization Michel X. Goemans February 28th, 2013 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory
More informationLinear Programming, Lecture 4
Linear Programming, Lecture 4 Corbett Redden October 3, 2016 Simplex Form Conventions Examples Simplex Method To run the simplex method, we start from a Linear Program (LP) in the following standard simplex
More informationFebruary 22, Introduction to the Simplex Algorithm
15.53 February 22, 27 Introduction to the Simplex Algorithm 1 Quotes for today Give a man a fish and you feed him for a day. Teach him how to fish and you feed him for a lifetime. -- Lao Tzu Give a man
More information3. Vector spaces 3.1 Linear dependence and independence 3.2 Basis and dimension. 5. Extreme points and basic feasible solutions
A. LINEAR ALGEBRA. CONVEX SETS 1. Matrices and vectors 1.1 Matrix operations 1.2 The rank of a matrix 2. Systems of linear equations 2.1 Basic solutions 3. Vector spaces 3.1 Linear dependence and independence
More informationLecture 10: Linear programming duality and sensitivity 0-0
Lecture 10: Linear programming duality and sensitivity 0-0 The canonical primal dual pair 1 A R m n, b R m, and c R n maximize z = c T x (1) subject to Ax b, x 0 n and minimize w = b T y (2) subject to
More informationThe Simplex Algorithm
The Simplex Algorithm How to Convert an LP to Standard Form Before the simplex algorithm can be used to solve an LP, the LP must be converted into a problem where all the constraints are equations and
More information4.5 Simplex method. min z = c T x s.v. Ax = b. LP in standard form
4.5 Simplex method min z = c T x s.v. Ax = b x 0 LP in standard form Examine a sequence of basic feasible solutions with non increasing objective function value until an optimal solution is reached or
More information::::: OFTECHY. .0D 0 ::: ::_ I;. :.!:: t;0i f::t l. :- - :.. :?:: : ;. :--- :-.-i. .. r : : a o er -,:I :,--:-':: : :.:
,-..., -. :', ; -:._.'...,..-.-'3.-..,....; i b... {'.'',,,.!.C.,..'":',-...,'. ''.>.. r : : a o er.;,,~~~~~~~~~~~~~~~~~~~~~~~~~.'. -...~..........".: ~ WS~ "'.; :0:_: :"_::.:.0D 0 ::: ::_ I;. :.!:: t;0i
More informationThe Simplex Algorithm and Goal Programming
The Simplex Algorithm and Goal Programming In Chapter 3, we saw how to solve two-variable linear programming problems graphically. Unfortunately, most real-life LPs have many variables, so a method is
More informationSection Notes 9. IP: Cutting Planes. Applied Math 121. Week of April 12, 2010
Section Notes 9 IP: Cutting Planes Applied Math 121 Week of April 12, 2010 Goals for the week understand what a strong formulations is. be familiar with the cutting planes algorithm and the types of cuts
More information1. Algebraic and geometric treatments Consider an LP problem in the standard form. x 0. Solutions to the system of linear equations
The Simplex Method Most textbooks in mathematical optimization, especially linear programming, deal with the simplex method. In this note we study the simplex method. It requires basically elementary linear
More informationA primal-simplex based Tardos algorithm
A primal-simplex based Tardos algorithm Shinji Mizuno a, Noriyoshi Sukegawa a, and Antoine Deza b a Graduate School of Decision Science and Technology, Tokyo Institute of Technology, 2-12-1-W9-58, Oo-Okayama,
More informationTHE UNIVERSITY OF HONG KONG DEPARTMENT OF MATHEMATICS. Operations Research I
LN/MATH2901/CKC/MS/2008-09 THE UNIVERSITY OF HONG KONG DEPARTMENT OF MATHEMATICS Operations Research I Definition (Linear Programming) A linear programming (LP) problem is characterized by linear functions
More information