AM 121: Intro to Optimization Models and Methods Lecture 6: Phase I, degeneracy, smallest subscript rule. Yiling Chen SEAS Lesson Plan Phase 1 (initialization) Degeneracy and cycling Smallest subscript rule and Bland s theorem The fundamental theorem of LP Jensen & Bard: 3.7 and 3.8
Making the simplex method whole Need to address two remaining issues How to find a first BFS to initiate the method? How can we be sure the method terminates even if it encounters degeneracy? Finding an initial tableau Easy case: If our initial LP is max c T x s.t. Ax!!b x"0" and b"0, then can always transform into canonical form and thus find an initial BFS and an initial iti tableau (introduce slack vars) Example: max x 1 + x 2 s.t. x 1! 2 x 1 + 2x 2! 4 x 1, x 2 " 0
Initialization: The general case Consider LP with +ve RHS. May have " and = constraints max 2x 1 + x 2 s.t. x 1 + x 2! 3 -x 1 + x 2 " 1 x 1, x 2 " 0 Don t have a basis. Not even sure if feasible! max 2x 1 + x 2 (1) s.t. x 1 + x 2 + x 3 = 3 - x 1 +x 2 x 4 =1 x 1, x 2, x 3, x 4 " 0 Introduce artificial variable x 5. Measures error in the equality constraint. -x 1 +x 2 x 4 +x 5 =1 Auxiliary LP: min x 5 (2) s.t. x 1 + x 2 + x 3 = 3 -x 1 + x 2 x 4 + x 5 = 1 x 1, x 2, x 3, x 4, x 5 " 0 Lemma. (1) feasible # (2) has optimal soln with value zero ($) set x 5 =0 (%) if v * =0 then x 5 =0, x 1 x 4 feasible for (1). Phase 1 of the simplex method Introduce artificial variables in " " and = rows. Solve auxiliary problem to check feasibility max w s.t. w + x 5 = 0 (a) x 1 + x 2 + x 3 = 3 (b) -x 1 + x 2 x 4 + x 5 = 1 (c) x 1,, x 5 " 0 Why did this help? Easy BFS for auxiliary problem! Only objective constraint needs fixing, use (a) - (c). Obtain an initial tableau for basis (x 3,x 5 ): w + x 1 -x 2 + x 4 = -1 x 1 + x 2 + x 3 = 3 -x 1 + x 2 x 4 + x 5 = 1 Apply simplex method to auxiliary problem If obtain w=0 Apply simplex method to auxiliary problem. If obtain w=0 then can find an initial BFS for original problem.
Phase 1-Phase 2 Example (1/2) w + x 1 - x 2 + x 4 = -1 (a) Basic x 3 x 5 x 1 + x 2 + x 3 = 3 (b) Ratio 3/1 1/1 -x 1 + x 2 - x 4 +x 5 = 1 (c) B={3,5}. Pick x 2 to enter. Pick x 5 to leave. w +x 5 = 0 (*) () 2x 1 +x 3 +x 4 -x 5 = 2 -x 1 + x 2 -x 4 +x 5 = 1 B={2,3}. Optimal. x=(0,1,2,0,0). 0 0) w=0. Have x 5 =0, so have solution (x 1,,x 4 )=(0,1,2,0) to original LP. Find initial BFS. Delete (*) and x 5, obtain system: 2x 1 +x 3 + x 4 = 2 -x 1 + x 2 x 4 = 1 x 5 had become nonbasic (since nondegenerate) and so the final basis B={2,3} includes 2 variables in original problem. Phase 1-Phase 2 Example (2/2) z - 2x 1 - x 2 = 0 (a) 2x 1 + x 3 + x 4 = 2 (b) -x 1 + x 2 -x 4 = 1 (c) Need to isolate {x 2,x 3 }. Do (a) + (c) z - 3x 1 - x 4 = 1 Basic x 2 x 3 2x 1 +x 3 +x 4 = 2 Ratio 2/2 -x 1 + x 2 -x 4 = 1 Now begin Phase 2. B={2,3}. Pick x 1 to enter. x 3 leaves. 3 z +! x 3 +! x 4 = 4 x 1 +! x 3 +! x 4 = 1 x 2 +! x 3! x 4 = 2 B={1,2}. Optimal. x=(1,2,0,0). 0) z=4
Summary: Phase 1 Slack variables: transform inequalities into equalities. May take strictly positive or zero values at feasible solutions to original LP Artificial variables: tool to find an initial BFS. At any feasible solution of original LP, any artificial variable must take value zero Key properties of Phase 1 problem: (a) it is easy to initialize, (b) use simplex to prove feasibility, and also generate an initial iti BFS for original i LP. Degeneracy Definition. A tableau is degenerate when one or more RHS values, b i have value zero (implies x i =0 for i&b) If have to choose between several possible leaving indices then next tableau is degenerate. max 2x 1 + x 2 s.t. x 1 x 2! 1 x 1! 1 x 2! 1 x 1, x 2 " 0 Initial tableau (nondegenerate): z 2x 1 x 2 = 0 x 1 x 2 + x 3 = 1 x 1 + x 4 = 1 x 2 + x 5 = 1 Basis B={3,4,5}, initial BFS x=(0,0,1,1,1)
Example (Degeneracy) z 2x 1 x 2 = 0 Basic x 3, x 4, x 5 x 1 x 2 +x 3 = 1 Ratio 1/1 1/1 x 1 + x 4 = 1 x 2 + x 5 = 1 B={3,4,5}. Pick x 1 to enter, x 3 to leave (breaking ties at random) z 3x 2 +2x 3 = 2 Basic x 1, x 4, x 5 x 1 x 2 + x 3 = 1 Ratio 0/1 1/1 x 2 x 3 + x 4 = 0 x 2 +x 5 =1 B={1,4,5}. Note x 4 =0. x=(1,0,0,0,1). Degenerate! x 2 to enter, x 4 to leave z x 3 + 3x 4 = 2 Basic x 1, x 2, x 5 x 1 +x 4 =1 Ratio 1/1 x 2 x 3 + x 4 = 0 x 3 x 4 + x 5 = 1 B={1,2,5}, x=(1,0,0,0,1). 0 0 1) Basis and tableau has changed but BFS and objective value unchanged. x 3 enter, x 5 leave. z +2x 4 + x 5 = 3 x 1 + x 4 = 1 x 2 + x 5 = 1 x 3 x 4 + x 5 = 1 B={1,2,3}, x=(1,1,1,0,0), z=3. Optimal solution! Was OK here! x x 1!1 2 x 2!1 1 3 Non basic Binding x 1, x 2 x 1 =0, x 2 =0 x 1 -x 2!1 x 2, x 3 x 2 =0, x 1 -x 2 =1 x 3, x 4 x 1 -x 2 =1, x 1 =1 x 4, x 5 x 1 =1, x 2 =1 2 0 1 0 5 vars, 3 equations. Define n-m=2 additional binding constraints, implies unique solution. Simplex method picks (n-m) additional binding constraints, implied by (n-m) nonbasic vars Degeneracy e occurs when more than (n-m) constraints ts are binding $ can anticipate degeneracy when have more than (n-m) constraints intersect at an extreme point 1 x 1
Degeneracy and Cycling Will simplex method still terminate in finite time? Can no longer be sure that t objective value will strictly increase at each iteration (earlier termination theorem doesn t apply) Might encounter same basis twice, at which point simplex method might cycle in an infinite loop Cycling: Example (1/2) (Chvatal 83) max 10x 1 57x 2 9x 3-24 x 4 optimal solution s.t. 0.5x 1 5.5x 2 2.5x 3 + 9x 4! 0 x=(1,0,1,0), value 1 0.5x 1 1.5x 2 0.5x 3 + x 4! 0 x 1! 1 x 1, x 2, x 3, x 4 " 0 Put in canonical form: z 10x 1 + 57x 2 + 9x 3 + 24x 4 = 0 x 5 x 6 x 7 0.5x 1 5.5x 2 2.5x 3 + 9x 4 + x 5 = 0 0 0 1 0.5x 1 1.5x 2 0.5x 3 + x 4 + x 6 = 0 0.5 0.5 1 x 1 + x 7 = 1 Pivot rule: For entering, choose most negative reduced cost. For leaving, choose smallest index if we have a tie. z 53x 2 41x 3 +204x 4 + 20x 5 = 0 x 1 x 6 x 7 x 1 11x 2 5x 3 +18x 4 + 2x 5 = 0 0/4 1/11 4x 2 + 2x 3 8x 4 x 5 +x 6 = 0 11x 2 + 5x 3 18x 4 2x 5 +x 7 = 1
x 1 x 2 x 7 0/.5 0/.5 x 3 x 2 x 7 0/2 x 3 x 4 x 7 0/.5 0/.25 x 5 x 4 x 7 0/1 x 5 x 6 x 7 where we started!! How can we prevent cycling?
Smallest subscript rule Entering: amongst those with strictly negative reduced cost, pick variable with the smallest index Leaving: amongst those with minimal ratio, pick variable with the smallest index Definition. The simplex method cycles when it returns to the same tableau E.g., T 0 ' T 1 ' ' T p-1 ' T 0 Bland s Theorem. The simplex method using the smallest subscript rule cannot cycle Proof: see Chvatal Linear programming (Freeman, 1983) Termination Theorem. If the simplex method begins with an initial feasible tableau and uses the smallest subscript rule, it must terminate. Proof. (Exercise in HW #3)
Fundamental theorem of LP Theorem. Any LP has either an optimal solution, or is infeasible or unbounded. Some optimization problems do not have this property, e.g. min 1/x s.t. x"1 The simplex lemma Consider an LP in equality form, i.e. max c T x s.t. Ax=b, x"0 (where columns of A span, but don t necessarily have isolated variables.) Lemma. If columns of A span, and LP is feasible and not unbounded, then it has an optimal solution. Proof. (sketch) If feasible and not unbounded then LP has a BFS by Phase 1 with smallest subscript rule. Phase 1 must terminate, and with optimal value zero. Can obtain a BFS for original LP from final tableau of Phase 1 (need columns of A to span for this.) Now apply simplex method with smallest subscript rule for Phase 2, must terminate and since not unbounded must get an optimal solution.
The fundamental theorem Theorem. Any LP has either an optimal solution, or is infeasible or unbounded. Proof. Consider an LP that is feasible and not unbounded. Introduce slack and surplus variables to convert to equality form (not necessarily with isolated variables) and a constraint matrix A whose columns span. This (equivalent) LP has an optimal solution by the simplex lemma. Fun use an algorithm to generate a theorem!! Closing comment: Handling degeneracy in phase 1 Phase 1 may terminate with an artificial variable u i basic (but with u i =0). Suppose equation is! j=1 a ij x j + u i =0 n If a ij =0 for all j then can delete entire equation and recover a non-degenerate basic solution (this row was a redundant constraint) Else, some a ij >0. Can pivot on entry (i,j), cause variable x j to become basic and variable u i to become nonbasic. Repeat this process until all artificial variables are driven out of the (phase 1) basis.