Discrete Mathematics for Bioiformatics WS 07/08, G. W. Klau, 23. Oktober 2007, 12:21 1 The Simplex algorithm: Itroductory example The followig itroductio to the Simplex algorithm is from the book Liear Programmig by V. Chvátal. Example: max 5x 1 + 4x 2 + 3x 3 subject to 2x 1 + 3x 2 + x 3 5 4x 1 + x 2 + 2x 3 11 3x 1 + 4x 2 + 2x 3 8 x 1,x 2,x 3 0 The Simplex algorithm: Itroductory example (2) Itroduce slack variables ad obtai stadard form 1 : x 4 = 5 2x 1 3x 2 x 3 x 5 = 11 4x 1 x 2 2x 3 x 6 = 8 3x 1 4x 2 2x 3 z = 5x 1 + 4x 2 + 3x 3 maxz subject to x 1,...,x 6 0 Slack variables: x 4,x 5,x 6 Decisio variables: x 1,x 2,x 3 The Simplex algorithm: Itroductory example (3) Idea of the Simplex algorithm: Start with feasible solutio x 1,...,x 6 ad try to fid aother feasible solutio x 1,..., x 6 with 5 x 1 + 4 x 2 + 3 x 3 > 5x 1 + 4x 2 + 3x 3. Iterate util optimal solutio is foud. Problem 1: Fid first feasible solutio. I our example it is easy: will do ad yields z = 0. x = (0,0,0,5,11,8) The Simplex algorithm: Itroductory example (4) Problem 2: Fidig a better feasible solutio. Lookig at z, we have to icrease x 1, x 2, or x 3. 1 Note: Chvátal calls the form o the previous slide stadard form.
Liear Programmig, Alexader Bockmayr, 23. Oktober 2007, 12:21 2 Blackboard calculatios tell us: x 1 5 2,x 1 11 4,x 1 8 3, so x = ( 5 2,0,0,0,1, 1 25 ) yieldig z =. 2 2 Fidig a eve better solutio? Not so easy. We rewrite our system of equatios such that, agai, ozero variables appear at the left-had side ad zero variables o the right-had side (thus, x 1 ad x 4 swap their positios): x 1 = 5 2 1 2 x 4 3 2 x 2 1 2 x 3 The Simplex algorithm: Itroductory example (5) New system: x 1 = 5 2 1 2 x 4 3 2 x 2 1 2 x 3 x 5 = 1 + 5x 2 + 2x 4 x 6 = 1 2 + 1 2 x 2 1 2 x 3 + 3 2 x 4 z = 25 2 7 2 x 2 + 1 2 x 3 5 2 x 4 Next step: icrease x 3. Blackboard calculatios: x 3 = 1, yieldig (2,0,1,0,1,0) ad z = 13. The Simplex algorithm: Itroductory example (6) New system: x 3 = 1 + x 2 + 3x 4 2x 6 x 1 = 2 2x 2 2x 4 + x 6 x 5 = 1 + 5x 2 + 2x 4 z = 13 3x 2 x 4 x 6 We are doe! Why? We have foud a solutio with z = 13 ad every feasible solutio must satisfy z = 13 3x 2 x 4 x 6 ad x 2,x 4,x 6 0. The Simplex algorithm Give a problem max subject to c j x j x j 0 a ij x j b i i = 1,2,...,m j = 1,2,...,
Liear Programmig, Alexader Bockmayr, 23. Oktober 2007, 12:21 3 we itroduce slack variables ad arrive at our first dictioary x +i = b i z = c j x j, a ij x j i = 1,2,...,m which characterizes each feasible solutio as + m o-egative umbers x 1,...,x,x +1,...,x +m, where x +1,...,x +m deped o x 1,...,x. The Simplex algorithm (2) A iteratio of the Simplex algorithm cosists of replacig a feasible solutio with x 1,...,x +m by x 1,..., x +m, c j x j > c j x j. 2 We do so by choosig a (o-basic) variable x N from the right-had side with positive objective fuctio coefficiet ad icreasig its value maximally, thereby settig a left-had side (basic) variable x B to zero. We compute a ew dictioary where x N is left (basic) ad x B is right (o-basic). The Simplex algorithm (3) Defiitio: (dictioary) The equatios i a dictioary express m of the variables x 1,...,x m+ ad the objective fuctio z i terms of the remaiig variables. Every solutio of a dictioary must be also a solutio of the first dictioary. A dictioary is feasible if settig the o-basic variables to zero ad evaluatig the basic variables yields a feasible solutio. The Simplex algorithm moves from feasible dictioary to feasible dictioary. The Simplex algorithm (4) Are we doe? No! How do we fid the first feasible dictioary? (Iitializatio) Ca we always fid a eterig ad leavig variable to costruct the ext dictioary? (Iteratio) Does the process termiate? (Termiatio) 2 As we will later see, the iequality is ot always strict. The Simplex algorithm (5)
Liear Programmig, Alexader Bockmayr, 23. Oktober 2007, 12:21 4 We start with the secod issue (Iteratio): Cosider the last row of a dictioary z = z + c j x j, j N where N cotais the idices of o-basic variables. If c j 0 for all j N, we are doe. Otherwise, choose ay variable with positive coefficiet for eterig the basis, e. g., the oe with the largest coefficiet c j. Now the leavig variable is that basic variable whose o-egativity imposes the most striget upper boud o the icrease of the eterig variable. Problems: 1. o cadidate 2. multiple cadidates The Simplex algorithm (6) Problem 1 (o cadidate). Cosider x 2 = 5 + 2x 3 x 4 3x 1 x 5 = 7 3x 4 4x 1 z = 5 + x 3 x 4 x 1 No restrictios o x 3. Settig x 3 = t yields x = (0,5 + 2t,t,0,7) with z = 5 + t, that is, the problem is ubouded. Of course, this holds i geeral. Problem 2 (multiple cadidates). I this case, we may choose ay cadidate. This leads, however, to degeerate solutios. The Simplex algorithm (7) A basic solutio is degeerate if oe or more basic variables are at zero. This might have a aoyig side effect: o-icreasig iteratios. Example: x 4 = 1 2x 3 x 5 = 3 2x 1 + 4x 2 6x 3 x 6 = 2 + x 1 3x 2 4x 3 z = 2x 1 x 2 + 8x 3 If, as i this example, oly a few iteratios are degeerate, this is ot a big problem. Otherwise, we may cycle, which is a serious problem! The Simplex algorithm (8)
Liear Programmig, Alexader Bockmayr, 23. Oktober 2007, 12:21 5 Cyclig. Ca the Simplex go through a edless sequece of iteratios without ever fidig a optimal solutio? Yes: x 5 = 1 2 x 1 + 11 2 x 2 + 5 2 x 3 9x 4 x 6 = 1 2 x 1 + 3 2 x 2 + 1 2 x 3 x 4 x 7 = 1 x 1 z = 10x 1 57x 2 9x 3 24x 4 Now, choosig the eterig variable accordig to the largest coefficiet rule ad the leavig variable accordig to the lexicographic rule, we will see the same dictioary agai after six iteratios!