Sparse Rank-Revealing LU Factorization

Transcription

1 Sparse Rank-Revealing LU Factorization SIAM Conference on Optimization Toronto, May 2002 Michael O Sullivan and Michael Saunders Dept of Engineering Science Dept of Management Sci & Eng University of Auckland Stanford University Auckland, New Zealand Stanford, CA michael.osullivan@auckland.ac.nz saunders@stanford.edu Sparse Rank-Revealing LU Factorization p.1/26

2 GOALS Sparse Rank-Revealing LU Factorization p.2/26

3 Goals Given a sparse matrix A (m n, usually m n), Determine if A is ill-conditioned? Determine which columns to delete? (or replace) Maintain sparsity P 1 AP 2 = LU, U = Sparse Rank-Revealing LU Factorization p.3/26

4 Motivation Dynamic Programming (Mike O Sullivan s thesis) P substochastic A = P I, at most one singularity Optimization (MINOS and SNOPT) Basis Repair I A = B, square basis, perhaps ill-conditioned Basis Repair II A = (B S) T, look for better B Sparse Rank-Revealing LU Factorization p.4/26

5 RANK-REVEALING FACTORS Sparse Rank-Revealing LU Factorization p.5/26

6 Rank-Revealing Factors A = XDY T = Demmel et al. (1999) X, Y full column rank, well conditioned D diagonal SVD QR with column interchanges LU with Rook Pivoting (Foster 1997) LU with Complete Pivoting Sparse Rank-Revealing LU Factorization p.6/26

7 Sparse Rank-Revealing Factors QR multifrontal Pierce and Lewis (1997) TPP (Threshold Partial Pivoting) TRP (Threshold Rook Pivoting) TCP (Threshold Complete Pivoting) Not RR Perhaps effective? This talk Sparse Rank-Revealing LU Factorization p.7/26

8 PIVOTING NEEDS Amax j = biggest element in col j Amax = biggest element in A Sparse Rank-Revealing LU Factorization p.8/26

9 Partial Pivoting vs. Complete Pivoting Dense Computing L and U O(n 3 ) PP Finding Amax 1 O(n) CP Finding Amax O(n 3 ) Not so bad! Sparse Computing L and U O(nnz(L + U)) TPP Finding Amax j O(nnz(L + U)) TCP Finding Amax O(n 2 )? Too much Sparse Rank-Revealing LU Factorization p.9/26

10 LUSOL Sparse Rank-Revealing LU Factorization p.10/26

11 LUSOL A = LU + updates, L well-conditioned Gill, Murray, Saunders and Wright (1987) Revised , Markowitz strategy for sparse pivots (cf. MA28, Y12M, LA05, MOPS, MA48) TPP (Threshold Partial Pivoting) Search only a few sparse cols and rows Zlatev 1981 Store Amax j at top of col j Suhl & Suhl 1990 TCP (Threshold Complete Pivoting) New Sparse Rank-Revealing LU Factorization p.11/26

12 Stability Tolerance Lmax P 1 AP 2 = LDU, unit diags on L, U Threshold pivoting bounds the off-diags of L and perhaps U: TPP TRP TCP } } L ij Lmax 10.0 L ij, U ij Lmax 5.0 TRP, TCP are more Rank-Revealing with low Lmax: cond(l), cond(u) < (1 + Lmax) n Sparse Rank-Revealing LU Factorization p.12/26

13 Elimination Step Allowable pivots with Lmax = 3 TPP 1 TRP 4 TCP (Markowitz) Just a few columns and rows change Sparse Rank-Revealing LU Factorization p.13/26

14 Finding Amax from Amax j Naive Method Amax j = biggest element in column j Amax = biggest element in A jmax = column containing Amax Find Amax by searching all Amax j O(n 2 ) Theorem Need to search all Amax j only if Amax decreases Sparse Rank-Revealing LU Factorization p.14/26

15 Maintaining Amax j Amax j New Amax j??? Easy to update modified Amax j Sparse Rank-Revealing LU Factorization p.15/26

16 Maintaining Amax Amax is in column jmax, which may be modified: Amax j Amax Amax j Must search Amax + 5 +? for Amax If col jmax is not modified: Amax j Amax Amax j Amax = Sparse Rank-Revealing LU Factorization p.16/26

17 HEAPS Sparse Rank-Revealing LU Factorization p.17/26

18 Storing Amax j in a Heap Thanks to John Gilbert Amax Ha(k) Amax j Hj(k) j Hk(j) location of j in heap Sparse Rank-Revealing LU Factorization p.18/26

19 Calls to Heap Functions build heap from all Amax j for k = 1 : min(m, n) Choose pivot, do elimination Find Amax j for modified cols delete entry for pivot column for l = 2 : lenpivrow change entry for each modified column end end Remarkably little work Sparse Rank-Revealing LU Factorization p.19/26

20 NUMERICAL RESULTS Sparse Rank-Revealing LU Factorization p.20/26

21 Cost of RR Feature Problem memplus from Harwell-Boeing collection A = , nonzeros Lmax nnz(l+u) Time (secs, 350MHz P3) LUSOL (TCP) RR RR RR RR SuperLU (colamd PP) Sparse Rank-Revealing LU Factorization p.21/26

22 Markowitz CP, then Dense CP Modified Zlatev strategy Markowitz1 until 30% dense: Search at least 5 cols, 4 rows 2 Markowitz2 until 50% dense: Search at least 5 cols, 0 rows 3 Dense Complete Pivoting Sparse Rank-Revealing LU Factorization p.22/26

23 Profile Problem memplus A = , nonzeros TCP, Lmax= 10.0, LU = nonzeros Markowitz Find stable pivot 65.0% Dense CP % Elimination The algebra 7.4% Amax j For modified cols 4.7% change Heap update 0.1% (!) Sparse Rank-Revealing LU Factorization p.23/26

24 CONCLUSIONS Sparse Rank-Revealing LU Factorization p.24/26

25 Conclusions Threshold Complete Pivoting is usually Rank-Revealing with Lmax 4.0 (but sometimes needs 2.5) Heap structure allows Amax to be maintained cheaply RRLUs can be rather dense TPP: Work-horse, usually reliable TCP: Markowitz search dominates cost TRP: Worth investigating Sparse Rank-Revealing LU Factorization p.25/26

26 HEAPS of THANKS John Gilbert Philip Gill The CUTE and Harwell-Boeing Teams Maureen Doyle R. Sedgewick Cormen, Leiserson and Rivest Michael Friedlander Sparse Rank-Revealing LU Factorization p.26/26