Lecture 2: The Simplex method. Repetition of the geometrical simplex method. 2. Linear programming problems on standard form. 3. The Simplex algorithm. 4. How to find an initial basic solution. Lecture 2 The Simplex method
Repetition of the geometrical Simplex method The product planning problem maximize 200x + 400x 2 s.t. 40 x + 60 x 2 x + x 2 x k 0, k =, 2 60 x 2 40 c 30 20 0 x 2 = 0.5x + 25 0 20 30 40 x Lecture 2 2 The Simplex method
The idéa of the Simplex method is to iteratively search along edges to other corner points for a better objective function value. Determine initial cornerpoint Is the corner point optimal? Yes Finished No Find a better adjacent cornerpoint Lecture 2 3 The Simplex method
Searching with the geometrical simplex method c x2 60 x2 60 c x2 60 40 30 20 0 d d 2 c 40 30 20 0 d 2 40 30 d d 20 0 d 2 0 20 30 40 x 0 20 30 40 x 0 20 30 40 x c T d < 0 c T d < 0 c T d 2 > 0 c T d 2 > 0 choose direction d 2 choose direction d 2 c T d < 0 c T d 2 < 0 optimal CP Lecture 2 4 The Simplex method
The Product planning example The Standard form for the product planning example. maximize 200x + 400x 2 s.t. 40 x + 60 x 2 x + x 2 x k 0, k =, 2 = minimize 200x 400x 2 s.t. 40 x + 60 x 2 + x 3 = x + x 2 + x 4 = x k 0, k =, 2, 3, 4. How are corner points represented in the standard form? Lecture 2 5 The Simplex method
Geometric interpretation for LP on standard form: 2D x 2 x 2 α α x β x Constraint: a x + a 2 x 2 = b. F is a semi-infinite line segment (left), or finite line segment (right). The optimal ˆx = (0,a), (β, 0) or the problem has no finite solution. Lecture 2 6 The Simplex method
Geometric interpretation for LP on standard form: 3D γ x 3 x α β x 2 Constraint: a x + a 2 x 2 + a 3 x 3 = b. F is the intersection of a plane and the first quadrant. (green) There are three corner points x () = (α, 0, 0), x (2) = (0,β, 0), x (3) = (0, 0,γ), Lecture 2 7 The Simplex method
Geometric interpretation for LP on standard form: 3D x 3 γ β x α α2 x 2 Constraints: a x + a 2 x 2 + a 3 x 3 = b. a 2 x + a 22 x 2 + a 23 x 3 = b 2. F is the intersection of a line and the first quadrant. (green) There are two corner points x () = (α,β, 0), x (2) = (α 2, 0,γ), Note: # nonzero elements in x (k) = # constraints, in these examples. Lecture 2 8 The Simplex method
The Product planning example The constraint in the standard form is Ax = b where A = [a a 2 a 3 a 4 = 0 40 60 b = 0 It can be written as 4 a k x k = a x + a 2 x 2 + a 3 x 3 + a 4 x 4 = b k=0 If we let, e.g., x 2 = x 3 = 0 then a x + a 4 x 4 = A β x β = b, where A β = [a a 4 = 0 40, x β = x x 4 Lecture 2 9 The Simplex method
Then we determine x β = x x 4 = A β b = 40 5 These values of x and x 4, together with x 2 = x 3 = 0, gives a feasible solution, i.e., Ax = b, where (x,x 2 ) = (40, 0). This is a corner-point. The other solutions corresponding to combinations of two columns in A are depicted in the table on the next slide. Lecture 2 0 The Simplex method
Geometrical illustration of the basic solutions 60 40 30 20 0 x 2 (2,4) (2,3) (3,4) 0 (,2) (,4) (,3) 20 30 40 x β A 2 β 3 x 2 β 3 (x, x 2 ) 2 3 (3,4) 4 0 5 4 5 4 0 5 0 0 2 3 2 3 2 3 (2,4) 4 0 60 5 4 60 5 4 0 5 60 2 3 2 53 2 3 (,4) 4 0 40 5 4 40 5 4 40 5 0 2 3 2 5 3 2 3 (2,3) (,3) (,2) 4 60 0 2 4 40 0 2 4 40 5 3 5 60 60 3 5 4 5 2 6 3 4 5 4 2 3 4 20 5 30 4 0 5 2 3 4 5 0 2 3 4 20 5 30 In general, it holds that cornerpoints corresponds to so called basic solutions. Lecture 2 The Simplex method
The LP-problem in standard form minimize s.t. n c j x j j= n a ij x j = b i, i =,...,m j= x j 0, j =,...,n = minimize c T x s.t. Ax = b x 0 where A = a.... a n. [ = a... a n,b = b.,c = c.,x = x. a m... a mn b m c n x n Lecture 2 2 The Simplex method
The Simplex algorithm The constraint Ax = b can be written: n k= a kx k = b. For given vectors of indices of basic variables and non-basic variables β = (β,...,β m ) ν = (ν,...,ν l ), l = n m we define A β = c β = [a β... a βm, A ν = [a ν... a νl, c β., x β = x β. c ν = c ν., x ν = x ν. c βm x βm c νl x νl Then Ax = A β x β + A ν x ν = b and c T x = c T β x β + c T ν x ν. Lecture 2 3 The Simplex method
Definition 4.4: Basic solutions Suppose β is a basic index tuple.. A basic solution corresponding to β is a solution to Ax = b such that A β x β = b and x ν = 0. 2. A basic feasible solution corresponding to β is a basic solution x such that x β 0. 3. A BFS is called non-degenerated if x β > 0 4. A BFS is called degenerated if x βk = 0 for some index β k. The basic solution corresponding to β is given by A β x β = b x β = A β b Lecture 2 4 The Simplex method
The Product planning example 60 40 30 20 0 x 2 (2,4) (2,3) (3,4) 0 (,2) (,4) (,3) 20 30 40 x β A β x β ν x 2 3 2 3 ν (3,4) 4 0 5 4 5 (, 2) 0 0 2 3 2 3 (2,4) 4 0 60 5 4 60 5 (, 3) 0 (,4) (2,3) (,3) (,2) 2 3 4 0 40 5 2 3 4 60 5 0 2 3 4 40 5 2 4 40 0 60 60 3 5 5 2 3 4 40 5 (2, 3) 0 5 2 3 4 5 (, 4) 0 6 2 3 4 5 (2, 4) 0 4 2 3 4 20 5 (3, 4) 0 30 Which basic solutions are feasible? Compare with the geometrical interpretation. Lecture 2 5 The Simplex method
Theorem 4.8 If a linear program in standard form has a finite optimal solution, then it has an optimal basic feasible solution. The Theorem motivates the following Simplex algorithm Determine initial BFS BFS optimal? Yes Finished No Change basic variables The different blocks can be implemented using linear algebra. Lecture 2 6 The Simplex method
Initial BFS β = (β,..., β m ) ν = (ν,..., ν l ) Calculate (y,r ν, b) A T β y = c β r ν = c ν A T ν y A β b = b r ν 0 No Yes Optimal solution x β = b, x ν = 0 ẑ = c T β b = y T b Take ν q so that r ν q < 0 Solve A β ā k = a k ; k := ν q x ν q enters the basis ā k 0 Yes Unbounded problem Solution does not exist t max = min ( bi No ) ā ik ā ik > 0 = ν old := ν q, ν q := β p, β p := ν old b p ā pk x p exits the basis Lecture 2 7 The Simplex method
Simplex applied on the product planning example The standard form for the product planning example. minimize c T x s.t. Ax = b x 0 where A = c T = 40 [ 200 400 0 0, 0 60 0 b = We want to solve this using simplex. When we introduce the slack variables it can be shown that if we take these as starting basic variables they form a BFS. (Given that b 0) Lecture 2 8 The Simplex method
Simplex Let β = {3, 4} and ν = {, 2}, A β = 0 /40 /60, A ν = 0 / / [ [ c T β = 0 0, c T ν = 200 400 The basic solution x β = b = A β b = [ is feasible. The simplex multiplicators are given by the equation A T β y = c β and reduced costs are given by r T ν = c T ν y T A ν, i.e. [ [ T [ 0 y =, and r T ν = [ 0 200 400 0 0 /40 /60 / / = [ 200 400. Lecture 2 9 The Simplex method
Simplex Since the reduced costs are negative, the current BFS can not be optimal. r ν2 is the smallest, so let x ν2 = x 2 be a new basic variable. Which variable should exit the base? [ b is determined from A β b = b, i.e. b = A β b = ā 2 is determined from A β ā 2 = a 2, i.e. ā 2 = A β a 2 = Now x 2 should be made as large as possible while x β is non-negative: [ /60 / x β = b ā 2 x 2 = /60 / x 2 0 The largest possible value of x 2 is, whereby x β2 = x 4 = 0. Therefore, x 4 should be replaced by x 2 in the next basic solution. Lecture 2 20 The Simplex method
Simplex: Iteration 2 Let β = {3, 2} and ν = {, 4}, A β = /60, A ν = /40 0 0 / / [ [ c T β = 0 400, c T ν = 200 0 [ The basic solution x β = b = A β b = feasible. 5/6 0 [ [ = /6 is then The simplex multiplicators are given by the equation A T β y = c β y = A T β c β = 5/6 0 T 0 400 = 0 20000. Lecture 2 2 The Simplex method
Simplex: Iteration 2 Reduced costs are given by r T ν = c T ν y T A ν, i.e. T [ 0 r T ν = 200 0 /40 0 20000 / = [ 200 20000. Since the reduced costs are positive the current BFS is optimal. x 2 60 (2,4) (2,3) 40 30 (,2) 20 0 (3,4) (,4) (,3) 0 20 30 40 x We started at the origin with basic variables (3,4) and then switched to the basic variables (3,2), i.e. the point (0,), which we argued was optimal previously using geometric methods. Lecture 2 22 The Simplex method
Initial BFS Often it is non-trivial to find an initial BFS. Then it is possible to solve the problem with a two-phase method. Here we assume that b 0. hase : Solve the LP-problem minimize e T v where I is the unit matrix and e = s.t. Ax + Iv = b x 0, v 0 [... T. Let the initial BFS have a basis that correspond to v. If the optimal solution (ˆv, ˆx) is such that ˆv = 0, then ˆx is a BFS to the original problem. hase 2: Solve the original problem using the BFS ˆx found in phase. Lecture 2 23 The Simplex method
Reading instructions Chapters 4 and 5 in the book. Lecture 2 24 The Simplex method