LUSOL: A basis package for constrained optimization

Transcription

1 LUSOL: A basis package for constrained optimization IFORS Triennial Conference on OR/MS Honolulu, HI, July 11 15, 2005 Michael O Sullivan and Michael Saunders Dept of Engineering Science Dept of Management Sci & Eng University of Auckland Stanford University Auckland, New Zealand Stanford, CA michael.osullivan@auckland.ac.nz saunders@stanford.edu LUSOL p.1/34

2 Abstract LUSOL is currently the BFP (basis factorization package) for several optimization packages, including MINOS, SQOPT, SNOPT, ZIP, PATH, and lp solve. Threshold Rook Pivoting is an important feature for basis repair (recovery from unexpected singularity). We review the open source Fortran and C implementations of LUSOL. LUSOL p.2/34

3 LUSOL Maintains LU factors of a general sparse matrix A Gill, Murray, Saunders, and Wright (1987) Code contributors F77 MATLAB Cmex C (for lp solve) Saunders (1986 present) following Duff, Reid, Zlatev, Suhl and Suhl Michael O Sullivan (1999 present) Kjell Eikland (2004 present) Features Square or rectangular A Rank-revealing LU for basis repair Stable updates Bartels-Golub style LUSOL p.3/34

4 LUSOL A = or or = LU FACTOR [L,U,p,q] = lusol(a) SOLVE Lx = y, L T x = y, Ux = y, U T x = y, Ax = y, A T x = y UPDATE Add, replace, delete a column Add, replace, delete a row Add a rank-one matrix MULTIPLY x = Ly, x = L T y, x = Uy, x = U T y, x = Ay, x = A T y LUSOL p.4/34

5 RANK-REVEALING LU LUSOL p.5/34

6 FACTOR [L,U,p,q] = lusol(a) L(p,p) = U(p,q) = Well defined for any square or rectangular A Permutations p, q balance stability and sparsity Markowitz strategy for suggesting sparse pivots Stability options: TPP Threshold Partial Pivoting TRP Threshold Rook Pivoting TCP Threshold Complete Pivoting LUSOL p.6/34

7 Partial Pivoting LUSOL p.7/34

8 Rook Pivoting LUSOL p.8/34

9 Complete Pivoting LUSOL p.9/34

10 TPP: Threshold Partial Pivoting LUSOL p.10/34

11 TRP: Threshold Rook Pivoting LUSOL p.11/34

12 TCP: Threshold Complete Pivoting LUSOL p.12/34

13 Rank-Revealing Factors A = XDY T = X, Y well conditioned, D diagonal cond(a) cond(d) SVD QR with column interchanges LU with Rook Pivoting LU with Complete Pivoting UDV T QDR LDU LDU LUSOL p.13/34

14 L = ( ) cond(l) = 100? LUSOL p.14/34

15 L = ( ) cond(l) = LUSOL p.15/34

16 Stability tolerance τ P AQ = LDU Threshold pivoting bounds elements of L and/or U: TPP TRP TCP } } L ij τ 100 or 10 or 5 L ij, U ij τ 3 or 2 or 1.1 TRP, TCP are more Rank-Revealing with low τ: cond(l), cond(u) < (1 + τ) n cond(d) cond(a) LUSOL p.16/34

17 The need for rank-revealing LU A = δ δ 1 1 δ 1 = LDU δ δ small TPP would give L = I, D = δi, rank(a) = 4 or 0 (!) TRP or TCP would give δ δ 1 1 δ 1 L δ δ 1 1 δ 2 1 δ 3 δ 4 rank(a) 3 LUSOL p.17/34

18 Implementing TRP At each stage of Gaussian elimination: A A lu T α j = biggest element in col j β i = biggest element in row i LUSOL p.18/34

19 Implementing TCP At each stage of Gaussian elimination: A A lu T α j = biggest element in col j Amax = biggest element in A (= max α j = max β i ) LUSOL p.19/34

20 TCP: Store α j in a Heap Thanks to John Gilbert Amax Ha(k) α j Hj(k) j Hk(j) k (location of j in heap) LUSOL p.20/34

21 RESULTS LUSOL p.21/34

22 Problem memplus from Harwell-Boeing collection A = , nonzeros, Scaled τ nnz(l+u) Time TRP RR RR RR RR RR TCP RR RR RR PP (SuperLU, colamd) LUSOL p.22/34

23 TRP Profile CUTE Problem BRATU2D A = , nonzeros TRP, τ = 1.26, LU = nonzeros Update β i for modified rows 57.7% Markowitz Find stable pivot 31.4% Elimination The algebra 4.0% Dense CP % Update α j for modified cols 1.6% LUSOL p.23/34

24 TCP Profile Harwell-Boeing Problem memplus A = , nonzeros TCP, τ = 10.0, LU = nonzeros Markowitz Find stable pivot 65.0% Dense CP % Elimination The algebra 7.4% Update α j for modified cols 4.7% Update heap 0.1% (!) LUSOL p.24/34

25 SOLVE LUSOL p.25/34

26 SOLVE Dense rhs Currently, Lx = y, L T x = y,... assume rhs y is dense Sparse rhs (future) Gilbert and Peierls (1988), CPLEX 7.1 (2001) Lx = y requires L column-wise L T x = y needs second copy of L (row-wise) Similarly for U, U T (not good when updates modify U) Product-form update would be ok: B k = L 0 U 0 E 1 E 2... E k Points to Schur-complement updates LUSOL p.26/34

27 APPLICATIONS LUSOL p.27/34

28 LUSOL in MINOS and SQOPT BR factorization rank detection for square B B = = LU, P LP T = L 1, P UQ = L 2 L 3 U 1 U 2... TRP or TCP, τ 2.5, discard factors BS factorization basis detection for rectangular W = (B S) W T = = LU, P LP T = L 1 L 2 TPP, TRP, or TCP τ 2.5, discard factors, P UQ = U 1 I 0 New B = first m columns of W P T LUSOL p.28/34

29 Ping-Qi Pan: Deficient-basis simplex methods A revised dual projective pivot algorithm for linear programming, SIOPT, to appear 2005 A revised primal deficient-basis simplex algorithm for linear programming, SIOPT, submitted June 2005 Take advantage of degeneracy: A = B N, Bx B = b Thm: cond(b) cond(b a) Apply LUSOL to rectangular B LUSOL p.29/34

30 lp solve An open source Mixed Integer Programming solver solve/ GNU LGPL, implemented in C, runs on most platforms Repository for a C implementation of LUSOL created by Kjell Eikland (F77 Pascal C) Main LU includes dynamic reallocation of storage solve/files/lusol/ Choice of BFPs LUSOL is now the default LUSOL p.30/34

31 SUMMARY LUSOL p.31/34

32 LUSOL Features Square or rectangular A Rank-revealing LU for basis repair Stable updates Bartels-Golub style LUSOL p.32/34

33 Future Tasks FACTOR SOLVE UPDATE Language Improve β i = max element in each row Special handling of dense columns Sparse rhs s Schur-complement (F90, Hanh Huynh s thesis) F77 C always possible via f2c F77 Pascal C done for lp solve F90 C? (NAG F95 compiler?) COIN-OR project LUSOL p.33/34

34 Future Tasks FACTOR SOLVE UPDATE Language Improve β i = max element in each row Special handling of dense columns Sparse rhs s Schur-complement (F90, Hanh Huynh s thesis) F77 C always possible via f2c F77 Pascal C done for lp solve F90 C? (NAG F95 compiler?) COIN-OR project Documentation LUSOL p.34/34