Solving Linear and Integer Programs

Size: px

Start display at page:

Download "Solving Linear and Integer Programs"

Ronald Page
6 years ago
Views:

1 Solving Linear and Integer Programs Robert E. Bixby ILOG, Inc. and Rice University Ed Rothberg ILOG, Inc. DAY 2 2 INFORMS Practice

2 Dual Simplex Algorithm 3 Some Motivation Dual simplex vs. primal (2002): Dual > 2x faster CPLEX Primal 1.2x improvement Dual 1.7x improvement Best algorithm of MIP There isn t much in books about implementing the dual. 4 INFORMS Practice

3 Dual Simplex Algorithm (Lemke, 1954: Commercial codes ~1990) Input: A dual feasible basis B and vectors X B = A -1 B b and D N = c N A T N B -T c B. Step 1: (Pricing) If X B 0, stop, B is optimal; else let i = argmin{x Bk : k {1,,m}}. Step 2: (BTRAN) Solve B T z= e i. Compute α N =-A NT z. Step 3: (Ratio test) If α N 0, stop, (D) is unbounded; else, let j = argmin{d k /α k : α k > 0}. Step 4: (FTRAN) Solve A B y= A j. Step 5: (Update) Set B i =j. Update X B (using y) and D N (using α N ) 5 Dual Simplex Algorithm (Lemke, 1954: Commercial codes ~1990) Input: A dual feasible basis B and vectors X B = A -1 B b and D N = c N A T N B -T c B. Step 1: (Pricing) If X B 0, stop, B is optimal; else let i = argmin{x Bk : k {1,,m}}. Step 2: (BTRAN) Solve B T z= e i. Compute α N =-A NT z. Step 3: (Ratio test) If α N 0, stop, (D) is unbounded; else, let j = argmin{d k /α k : α k > 0}. Step 4: (FTRAN) Solve A B y= A j. Step 5: (Update) Set B i =j. Update X B (using y) and D N (using α N ) 6 INFORMS Practice

4 Implementing the Dual Simplex Algorithm 7 Implementation Issues for Dual Simplex 1. Finding an initial feasible basis, or the concluding that there is none 2. Pricing: Dual steepest edge 3. Solving the linear systems LU factorization and factorization update BTRAN and FTRAN exploiting sparsity 4. Numerically stable ratio test: Bound shifting and perturbation 5. Bound flipping: Exploiting boxed variables to combine many iterations into one. 8 INFORMS Practice

5 Issue 0 Preparation: Bounds on Variables In practice, simplex algorithms need to accept LPs in the following form: Minimize c T x Subject to Ax = b l x u (P BD ) where l is an n-vector of lower bounds and u an n-vector of upper bounds. l is allowed to have - entries and u is allowed to have + entries. (Note that (P BD ) is in standard form if l j = 0, u j = + j.) 9 (Issue 0 Bounds on variables) Basic Solution A basis for (P BD ) is a triple (B,L,U) where B is an ordered m- element subset of {1,,n} (just as before), (B,L,U) is a partition of {1,,n}, l j > - j L, and u j < + j U. N = L U is the set of nonbasic variables. The associated (primal) basic solution X is given by X L = l L, X U = u U and X B = A -1 B (b A L l L A U u U ). This solution is feasible if l B X B u B. The associated dual basic solution is defined exactly as before: D B =0, Π T A B = c T B, D N = c N A T N Π. It is dual feasible if D L 0 and D U INFORMS Practice

6 (Issue 0 Bounds on variables) The Full Story Modify simplex algorithm Only the Pricing and Ratio Test steps much be changed substantially. The complicated part is the ratio test Reference: See Chvátal for the primal 11 Issue 1 The Initial Feasible Basis Phase I Two parts to the solution 1. Finding some initial basis (probably not feasible) 2. Modified simplex algorithm to find a feasible basis Reference for Primal: R.E. Bixby (1992). Implementing the simplex method: the initial basis, ORSA Journal on Computing 4, INFORMS Practice

7 (Issue 1 Initial feasible basis) Initial Basis Primal and dual bases are the same. We begin in the context of the primal. Consider Minimize c T x Subject to Ax = b (P BD ) l x u Assumption: Every variable has some finite bound. Trick: Add artificial variables x n+1,,x n+m : Ax + I x n+1.. x n+m = b where l j = u j = 0 for j = n+1,,n+m. Initial basis: B = (n+1,,n+m) and for each j B, pick some finite bound and place j in L or U, as appropriate. 13 (Issue 1 Initial feasible basis) Solving the Phase I If the initial basis is not dual feasible, we consider the problem: Maximize Σ (d j : d j < 0) Subject to A T π + d = c This problem is locally linear : Define κ R n by κ j =1if D j < 0, and 0 otherwise. Let K = {j: D j < 0} and K = {j: D j 0} Then our problem becomes Maximize κ T d Subject to A T π + d = c d K 0, d K 0 Apply dual simplex, and whenever d j for j K becomes 0, move it to K. 14 INFORMS Practice

8 Issue 2 Pricing The texbook rule: Choose the largest primal violation is TERRIBLE: For a problem in standard form j = argmin{x Bi : i = 1,,m} Geometry is wrong: Maximizes rate of change relative to axis; better to do relative to edge. Goldfard and Forrest 1992 suggested the following steepest-edge alternative j = argmin{x Bi /η i : i = 1,,m} where η i = e it A -1 B 2, and gave an efficient update. 15 Example: Pricing Model: dfl001 Pricing: Greatest infeasibility Dual simplex - Optimal: Objective = e+07 Solution time = sec. Iterations = (0) Pricing: Goldfarb-Forrest steepest-edge Dual simplex - Optimal: Objective = e+07 Solution time = sec. Iterations = (0) 16 INFORMS Practice

9 Issue 3 Solving FTRAN, BTRAN Computing LU factorization: See Suhl & Suhl (1990). Computing sparse LU factorization for largescale linear programming basis, ORSA Journal on Computing 2, Updating the Factorization: Forrest-Tomlin update is the method of choice. See Chvátal Chapter 24. Exploiting sparsity: This is the main recent development. 17 (Issue 3 Solving FTRAN & BTRAN) We must solve two linear systems per iterstion: FTRAN BTRAN A B y= A j A T B z= e i where A B = basis matrix (very sparse) A j = entering column (very sparse) e i = unit vector (very sparse) y an z are typically very sparse Example: Model pla85900 (from TSP) Constraints Variables Average y INFORMS Practice

10 19 A B = U L Triangular solve: Lw=A j (A B y= L(Uy) = A j ) update update = L w A j Graph structure: Define an acyclic digraph D = ({1,,m}, E) where (i,j) E l ij 0 and i j. Solving using D: Let X = {i V: A ij 0}. Compute X = {j V: a directed path from j to X}. X can be computed in time linear in E(X) + X. w Known in advance Need to find w/o searching PDS Models Patient Distribution System : Carolan, Hill, Kennington, Niemi, Wichmann, An empirical evaluation of the KORBX algorithms for military airlift applications, Operations Research 38 (1990), pp MODEL ROWS pds pds pds pds pds pds pds pds pds CPLEX CPLEX CPLEX SPEEDUP Primal Simplex Dual Simplex Dual Simplex 20 INFORMS Practice

11 CPLEX 1.0 seconds Not just faster -- Growth with size: Quadratic then & Linear now! CPLEX 8.0 seconds # Time Periods: PDS02 -- PDS70 21 Issue 4 Ratio Test and Finiteness The standard form dual problem is 22 Maximize b T π Subject to A T π + d = c d 0 Feasibility means d 0 However, in practice this condition is replaced by d - ε e where e T =(1,,1) and ε =10-6. Reason: Degeneracy. In 1972 Paula Harris proposed suggested exploiting this fact to improve numerical stability. INFORMS Practice

12 (Issue 4 Ratio test & finiteness) STD. RATIO TEST j enter = argmin{d j /α j : α j > 0} Motivation: Feasibility step length θ satisfies D N θα N 0 However, the bigger the step length, the bigger the change in the objective. So, we choose Using ε, we have θ max = min{d j /α j : α j > 0} θ ε max = min{(d j +ε)/α j : α j > 0} > θ max 23 HARRIS RATIO TEST j enter = argmax{α j : D j /α j θ ε max } Advantages (Issue 4 Ratio test & finiteness) Numerical stability α jenter = pivot element Degeneracy Reduces # of 0-length steps Disadvantage D jenter < 0 objective goes in wrong direction Solution: BOUND SHIFTING If D jenter < 0, we replace the lower bound on d jenter by something less than its current value. Note that this shift changes the problem and must be removed: 5% of cases, this produces dual infeasibility process is iterated. 24 INFORMS Practice

13 Example: Bound-Shifting Removal Problem 'pilot87.sav.gz' read. Reduced LP has 1809 rows, 4414 columns, and nonzeros. Iteration log... Iteration: 1 Scaled dual infeas = Iteration: 733 Scaled dual infeas = Iteration: 790 Dual objective = Iteration: Dual objective = Removing shift (3452). Iteration: Scaled dual infeas = Iteration: Scaled dual infeas = Iteration: Dual objective = Elapsed time = sec. (17000 iterations). Iteration: Dual objective = Iteration: Dual objective = Removing shift (76). Iteration: Scaled dual infeas = Iteration: Dual objective = Elapsed time = sec. (18000 iterations). Removing shift (10). Iteration: Scaled dual infeas = Iteration: Dual objective = Removing shift (1). Shift 1: ε = 10-7 Shift 2: ε = 10-8 Shift 3: ε = Dual simplex - Optimal: Objective = e+002 Solution time = sec. Iterations = (1137) (Issue 4 Ratio test & finiteness) Finiteness: Bound shifting is closely related to the perturbation method employed in CPLEX if no progress is being made in the objective. No progress is replaced by d j -ε j = 1,,n d j -ε ε j j = 1,,n, where ε j is random uniform on [0,ε]. 26 INFORMS Practice

14 Issue 5 Bound Flipping A basis is given by a triple (B,L,U) L = non-basics at lower bound: Feasibility D L 0 U = non-basics at upper bound: Feasibility D U 0 Ratio test: Suppose X Bi is the leaving variable, and the step length is blocked by some variable d j, j L, that is about to become negative and such that u j <+ : Flipping means: Move j from L to U. Check: Do an update to see if X Bi is still favorable Can combine many iterations into a single iteration. 27 Example: Bound Flipping Problem 'fit2d.sav.gz' read. Initializing dual steep norms... Iteration log... Iteration: 1 Dual objective = Perturbation started. Iteration: 203 Dual objective = Iteration: 1313 Dual objective = Iteration: 2372 Dual objective = Iteration: 3413 Dual objective = Iteration: 4316 Dual objective = Iteration: 5151 Dual objective = Iteration: 5820 Dual objective = Removing perturbation. w/o flipping Dual simplex - Optimal: Objective = e+004 Solution time = sec. Iterations = 5932 (0) Problem 'fit2d.sav.gz' read. Initializing dual steep norms... Iteration log... Iteration: 1 Dual objective = w/ flipping Dual simplex - Optimal: Objective = e+004 Solution time = 1.88 sec. Iterations = 201 (0) 28 INFORMS Practice

Novel update techniques for the revised simplex method (and their application)

Novel update techniques for the revised simplex method (and their application) Qi Huangfu 1 Julian Hall 2 Others 1 FICO 2 School of Mathematics, University of Edinburgh ERGO 30 November 2016 Overview Background