A convex QP solver based on block-lu updates

Size: px

Start display at page:

Download "A convex QP solver based on block-lu updates"

Brittney Hines
6 years ago
Views:

1 Block-LU updates p. 1/24 A convex QP solver based on block-lu updates PP06 SIAM Conference on Parallel Processing for Scientific Computing San Francisco, CA, Feb 22 24, 2006 Hanh Huynh and Michael Saunders SCCM Program Dept of Management Sci & Eng Stanford University Stanford University Stanford, CA Stanford, CA hhuynh@stanford.edu saunders@stanford.edu

2 Block-LU updates p. 2/24 Abstract Active-set methods have an advantage over interior methods in permitting warm starts. We describe a convex QP method intended for use within SNOPT (a sparse SQP package for constrained optimization). An initial KKT system is factorized by any available method (LUSOL, MA57, PARDISO, SuperLU,... ). Active-set changes are implemented by block-lu updates that retain sparsity while leaving the original KKT factors intact. Acknowledgements: Philip Gill, UC San Diego Comsol, Inc

3 Why a new QP solver? Block-LU updates p. 3/24

4 Block-LU updates p. 4/24 SNOPT Sparse SQP solver for NLP (Gill, Murray & Saunders, 2005) Sequence of QP subproblems currently solved by SQOPT: QP k minimize x subject to g T k x xt H k x linearized constraints and bounds Limited-memory quasi-newton Hessian H 1 = (I + vu T )H 0 (I + uv T ), etc Warm start, few iterations active-set method SQOPT s reduced Hessian Z T H k Z can be large Need a QP solver that works with KKT systems (like QPA in GALAHAD)

5 Block-LU updates p. 5/24 KKT systems Active-set QP solvers start with systems of the form ( ) H A T T K 0 y = d, K 0 = = L 0 D 0 L 0 A 0 and then add/delete rows and cols of H, A Later systems are equivalent to ( ) ( K 0 V V T D y 1 y 2 ) = ( d 1 d 2 ) where we want to treat K 0 as a black box

6 Block-LU updates p. 6/24 QPA Active-set QP solver in GALAHAD (Gould and Toint, 2004) Uses SCU to solve sequence of updated KKT systems ( ) ( ) ( ) K 0 V y 1 d 1 V T = D SCU maintains factors of Schur complement C, where y 2 K 0 T = V C = D V T T K 0 is a black box (QPA uses L 0 D 0 L 0 T via MA27 or MA57) T is not stored (may be fairly dense) One solve with dense C, but two solves with K 0 d 2

7 Symmetric Block-LU updates ( K 0 V T ) V D = ( L 0 Y T D 1 0 I ) ( D 0 L 0 T ) Y C K 0 = L 0 D 0 L 0 T L 0 Y = V C = D Y T D 0 1 Y Solves with L 0, D 0, C, D 0, L 0 T, and products with Y T, Y L 0, D 0 are two black boxes MA57 has separate solves with L 0, D 0, L 0 T Y is likely to be sparse (Thanks Iain!) Small dense LC = U with L square Block-LU updates p. 7/24

8 Same approach for updating simplex-type basis B 0 = L 0 U 0 Block-LU updates p. 8/24 Unsymmetric Block-LU updates ( K 0 W T ) V D = ( L 0 Z T I ) ( U 0 ) Y C K 0 = L 0 U 0, L 0 Y = V, U T 0 Z = W, C = D Z T Y Solves with L 0, C, U 0, and products with Z T, Y L 0, U 0 are two black boxes Y and Z are likely to be sparse Small dense LC = U with L square

9 Block-LU updates p. 9/24 Special request to PARDISO, SuperLU,... developers: Allow separate solves with L, D, U factors (MA57 already does)

10 Rank-Revealing Factors Block-LU updates p. 10/24

11 Block-LU updates p. 11/24 Partial Pivoting

12 Block-LU updates p. 12/24 Rook Pivoting

13 Block-LU updates p. 13/24 Complete Pivoting

14 Block-LU updates p. 14/24 LUSOL pivoting options TPP TRP TCP Threshold Partial Pivoting Threshold Rook Pivoting Threshold Complete Pivoting

15 Block-LU updates p. 15/24 TPP: Threshold Partial Pivoting Require L ij 2.0 (say)

16 Block-LU updates p. 16/24 TRP: Threshold Rook Pivoting A = LDU Require L ij and U ij 2.0 (say)

17 Block-LU updates p. 17/24 TCP: Threshold Complete Pivoting A = LDU Require L ij and U ij 2.0 (say)

18 Block-LU updates p. 18/24 Rank-Revealing Factors A = XDY T = Demmel et al. (1999): X, Y well-conditioned, D diagonal cond(a) cond(d) SVD QR with column interchanges LU with Rook Pivoting LU with Complete Pivoting UDV T QDR LDU LDU

19 Block-LU updates p. 19/24 Rank-Revealing Factors A = XDY T = Demmel et al. (1999): X, Y well-conditioned, D diagonal cond(a) cond(d) SVD QR with column interchanges LU with Rook Pivoting LU with Complete Pivoting MA27, MA57 (L ij bounded, D block-diag) UDV T QDR LDU LDU LDL T

20 Block-LU updates p. 20/24 Special request to PARDISO, SuperLU,... developers: Provide options for computing rank-revealing factors (LUSOL, MA57 already do) Need off-diags 2.5 say (MA57 s pivot tol 0.4)

21 Block-LU updates p. 21/24 Numerical results QP minimize c T x + 1 x 2 xt x subject to Ax = b, l x u QP problems with H = I A, b, c, l, u come from LPnetlib collection (Tim Davis) Matlab implementation of proposed QP solver using block-lu updates of KKT system [L0,U0,P,Q] = lu(k0) via UMFPACK (Tim Davis) Update 20 times then refactorize current KKT matrix

22 Block-LU updates p. 22/24 Block-LU updates ( K 0 W T ) V D = ( L 0 Z T I ) ( U 0 ) Y C Compare nonzeros in Y and Z: L 0 = I, U 0 = L 0 U 0 Separate L 0 and U 0 Y = V L 0 Y = V U T 0 L T 0 Z = W U T 0 Z = W

23 Black-box K 0 vs separate L 0, U 0 Data Source: capri, K0 = I*K0 Number of Nonzeros Iterations Data Source: capri, K0 = L0*U0 Y Z Number of Nonzeros Iterations Y Z Block-LU updates p. 23/24

24 Block-LU updates p. 24/24 Summary Block-LU updates: ( K 0 W T ) V D = ( L 0 Z T I ) ( U 0 ) Y C Black-box K 0 = L 0 U 0 or L 0 D 0 L T 0 Our main hope for parallelism in LP/QP/NLP solvers Need separate solves with L 0, D 0, U 0 Need rank-revealing factors of K 0 Ideally parallel products Y v, Y T w, Zv, Z T w (OBLAS?)

Implementation of a KKT-based active-set QP solver

Block-LU updates p. 1/27 Implementation of a KKT-based active-set QP solver ISMP 2006 19th International Symposium on Mathematical Programming Rio de Janeiro, Brazil, July 30 August 4, 2006 Hanh Huynh