A review of sparsity vs stability in LU updates

Transcription

1 A review of sparsity vs stability in LU updates Michael Saunders Dept of Management Science and Engineering (MS&E) Systems Optimization Laboratory (SOL) Institute for Computational Mathematics and Engineering (ICME) Stanford University, California Session: 40 Years of Forrest and Tomlin INFORMS 2012 Annual Meeting Phoenix AZ, Oct 14 18, 2012 Michael Saunders Sparsity vs stability in LU updates 1/31

2 Abstract The Forrest-Tomlin update has stood the test of time within many generations of commercial mathematical programming systems. Its ease of implementation leads to high efficiency and evidently acceptable reliability. We review its relation to Reid s version of the Bartels-Golub update as implemented in LA05, LA15, and LUSOL. In particular, we examine the extent to which FT implementations must live dangerously in order to achieve the desired efficiency. Michael Saunders Sparsity vs stability in LU updates 2/31

3 Outline 1 The Bartels-Golub update 2 The Forrest-Tomlin update 3 Sparse Bartels-Golub 4 Two tolerances 5 A numerical experiment 6 Numerical results 7 Conclusion Michael Saunders Sparsity vs stability in LU updates 3/31

4 The Bartels-Golub update Michael Saunders Sparsity vs stability in LU updates 4/31

5 Dense Bartels-Golub p ū Michael Saunders Sparsity vs stability in LU updates 5/31

6 Dense Bartels-Golub X X X X X X Michael Saunders Sparsity vs stability in LU updates 5/31

7 Dense Bartels-Golub X X X X X X Forget the Hessenberg Michael Saunders Sparsity vs stability in LU updates 5/31

8 The Forrest-Tomlin update Michael Saunders Sparsity vs stability in LU updates 6/31

9 Forrest-Tomlin (conceptually) X X Few nonzeros to eliminate U stored by columns Michael Saunders Sparsity vs stability in LU updates 7/31

10 Sparse Bartels-Golub Michael Saunders Sparsity vs stability in LU updates 8/31

11 Sparse Bartels-Golub (Reid, Saunders) X p X X ū l Michael Saunders Sparsity vs stability in LU updates 9/31

12 Sparse Bartels-Golub (Reid, Saunders) X X l LA05, LA15, LUSOL U stored by rows Michael Saunders Sparsity vs stability in LU updates 9/31

13 The connection X X FT: Eliminate X s with diags as pivots BG: Allow row interchanges ( ) 1 Use a product of and permutations µ 1 How big can µ safely be?? Michael Saunders Sparsity vs stability in LU updates 10/31

14 A Tale of Two Tolerances LU factor tol α LU update tol β Control and/or monitor stability Michael Saunders Sparsity vs stability in LU updates 11/31

15 LU factor tol α Threshold partial pivoting controls cond(l) B = LU L ii = 1 L ij α α = 10.0 or usually sparse and stable Michael Saunders Sparsity vs stability in LU updates 12/31

16 LU factor tol α Threshold partial pivoting controls cond(l) B = LU L ii = 1 L ij α α = 10.0 or usually sparse and stable Threshold rook pivoting controls cond(l), cond(u) B = LDU L ii = 1 L ij α and U ii = 1 U ij α 1 < α 2.0 rank-revealing (for basis repair) Michael Saunders Sparsity vs stability in LU updates 12/31

17 Factor and Update involve triangular matrices M = ( ) cond(m) =?? Michael Saunders Sparsity vs stability in LU updates 13/31

18 Factor and Update involve triangular matrices M = ( ) cond(m) 10 4 Michael Saunders Sparsity vs stability in LU updates 13/31

19 Factor and Update involve triangular matrices M = ( ) 1 µ 1 cond(m) µ 2 µ > 1 Michael Saunders Sparsity vs stability in LU updates 13/31

20 FT update involves triangular matrices 1 R = r p+1... r m 1 cond(r) r 2 max Michael Saunders Sparsity vs stability in LU updates 14/31

21 Product-form update involves triangular matrices 1 v 1 1. T = v p. v m... I cond(t ) v 2 max/ v p Michael Saunders Sparsity vs stability in LU updates 15/31

22 LU update tol β B = B(I + (v e p )e T p ) Use β to control or monitor basis updates Product-form B = BT monitor cond(t ) Bartels-Golub L = LM 1 M 2... control cond(m j ) Forrest-Tomlin L = LR monitor cond(r) Michael Saunders Sparsity vs stability in LU updates 16/31

23 Product-form update LU update tol β B = BT T = I + (v e p )ep T monitor cond(t ) Update if vmax/ v 2 p β else refactor How big should β be?? 10 6? Michael Saunders Sparsity vs stability in LU updates 17/31

24 Product-form update LU update tol β B = BT T = I + (v e p )e T p monitor cond(t ) LU updates Update if vmax/ v 2 p β else refactor How big should β be?? 10 6? BG L = LM 1 M 2... control M ij β FT L = LR monitor R ij β? For BG we can enforce β = 10 say For FT how big should β be?? 10 6? Michael Saunders Sparsity vs stability in LU updates 17/31

25 A numerical experiment Michael Saunders Sparsity vs stability in LU updates 18/31

26 Simulate FT via BG Run MINOS on a numerically challenging LP (e.g. pilot87) Set LUSOL s LU update tol β = 10, 10 2, 10 3, 10 4, 10 5, 10 6,... Count number of times something broke Michael Saunders Sparsity vs stability in LU updates 19/31

27 Simulate FT via BG Run MINOS on a numerically challenging LP (e.g. pilot87) Set LUSOL s LU update tol β = 10, 10 2, 10 3, 10 4, 10 5, 10 6,... Count number of times something broke Normally Factorization freq = 100 LU factor tol α = LU update tol β = 10.0 Michael Saunders Sparsity vs stability in LU updates 19/31

28 Simulate FT via BG Run MINOS on a numerically challenging LP (e.g. pilot87) Set LUSOL s LU update tol β = 10, 10 2, 10 3, 10 4, 10 5, 10 6,... Count number of times something broke Normally Factorization freq = 100 LU factor tol α = LU update tol β = 10.0 How would MINOS measure broke? Michael Saunders Sparsity vs stability in LU updates 19/31

29 Simulate FT via BG Run MINOS on a numerically challenging LP (e.g. pilot87) Set LUSOL s LU update tol β = 10, 10 2, 10 3, 10 4, 10 5, 10 6,... Count number of times something broke Normally Factorization freq = 100 LU factor tol α = LU update tol β = 10.0 How would MINOS measure broke? Row check failed? Never happened Every 60 itns, see if Ax b is too big Michael Saunders Sparsity vs stability in LU updates 19/31

30 Simulate FT via BG Run MINOS on a numerically challenging LP (e.g. pilot87) Set LUSOL s LU update tol β = 10, 10 2, 10 3, 10 4, 10 5, 10 6,... Count number of times something broke Normally Factorization freq = 100 LU factor tol α = LU update tol β = 10.0 How would MINOS measure broke? Row check failed? Never happened Every 60 itns, see if Ax b is too big LU became singular? Every update, check if U has small diag Increasingly often Michael Saunders Sparsity vs stability in LU updates 19/31

31 Tests for accepting LU update ( ) I L = L r T 1 Ū = ( ) U1 u δ Tomlin 1975 Our experiment r max u max δ 16 7 ( ) r max β and u 1 δ ɛ 2/3 ( ) Michael Saunders Sparsity vs stability in LU updates 20/31

32 Tests for accepting LU update ( ) I L = L r T 1 Ū = ( ) U1 u δ Tomlin 1975 Our experiment r max u max δ r max β and u 1 δ ɛ 2/3 ( ) Roughly equivalent to r max u max δ β10 10 = 10 11, 10 12,... Michael Saunders Sparsity vs stability in LU updates 20/31

33 Numerical results Michael Saunders Sparsity vs stability in LU updates 21/31

34 MINOS simulation of FT updates LPnetlib problem pilot ɛ = Featol = Opttol = 10 6 LU update tol β = 10, 10 2, 10 3,... singular average beta itns time LUfacs warnings updates 1e e e e e e e e Michael Saunders Sparsity vs stability in LU updates 22/31

35 MINOS simulation of FT updates 120 pilots: LU update tol = 1e+8 Average 31 updates 100 Updates before new LU LU number Michael Saunders Sparsity vs stability in LU updates 23/31

36 MINOS simulation of FT updates LPnetlib problem pilot87 ɛ = Featol = Opttol = 10 6 LU update tol β = 10, 10 2, 10 3,... singular average beta itns time LUfacs warnings updates 1e e e e e e e e Michael Saunders Sparsity vs stability in LU updates 24/31

37 MINOS simulation of FT updates LPnetlib problem mod2pre ɛ = Featol = Opttol = 10 6 LU update tol β = 10, 10 2, 10 3,... singular beta itns time LUfacs warnings 1e e e e * 1e * 1e infeas 1e * 1e infeas Refactorization gave singular LU Rook pivoting invoked, β reduced to 10.0 Michael Saunders Sparsity vs stability in LU updates 25/31

38 Quad SQOPT simulation of FT updates LPnetlib problem pilot ɛ = Featol = Opttol = LU update tol β = 10 4, 10 6, 10 8,... singular average dual beta itns time LUfacs warnings updates infeas 1e E-25 1e E-27 1e E-21 1e E-24 1e E-27 1e E-29 1e E-12 1e E-19 1e * 1e * 1e * *Refac singular, beta reduced to 50.0 Michael Saunders Sparsity vs stability in LU updates 26/31

39 Conclusion Michael Saunders Sparsity vs stability in LU updates 27/31

40 Conclusion Test for accepting an FT update r max u max WHIZARD 10 8 δ CLP, CPLEX, Gurobi, Xpress, Mozek,... r max u max δ? or? Probably safer r max β and u max δ Average updates per LU β = β = β = β = Michael Saunders Sparsity vs stability in LU updates 28/31

41 References (in chronological order) R. H. Bartels (1971). A stabilization of the simplex method, Numerische Mathematik 16, Eliminated subdiag of dense Hessenberg H = ŪP Michael Saunders Sparsity vs stability in LU updates 29/31

42 References (in chronological order) R. H. Bartels (1971). A stabilization of the simplex method, Numerische Mathematik 16, Eliminated subdiag of dense Hessenberg H = ŪP J. J. H. Forrest and J. A. Tomlin (1972). Updating triangular factors of the basis to maintain sparsity in the product form simplex method, Mathematical Programming 2, Eliminated pth row of sparse H (not subdiag) Michael Saunders Sparsity vs stability in LU updates 29/31

43 References (in chronological order) R. H. Bartels (1971). A stabilization of the simplex method, Numerische Mathematik 16, Eliminated subdiag of dense Hessenberg H = ŪP J. J. H. Forrest and J. A. Tomlin (1972). Updating triangular factors of the basis to maintain sparsity in the product form simplex method, Mathematical Programming 2, Eliminated pth row of sparse H (not subdiag) J. A. Tomlin (1975). An accuracy test for updating triangular factors, Math. Prog. Study 4, Test for an unstable update Michael Saunders Sparsity vs stability in LU updates 29/31

44 References (in chronological order) R. H. Bartels (1971). A stabilization of the simplex method, Numerische Mathematik 16, Eliminated subdiag of dense Hessenberg H = ŪP J. J. H. Forrest and J. A. Tomlin (1972). Updating triangular factors of the basis to maintain sparsity in the product form simplex method, Mathematical Programming 2, Eliminated pth row of sparse H (not subdiag) J. A. Tomlin (1975). An accuracy test for updating triangular factors, Math. Prog. Study 4, Test for an unstable update M. A. Saunders (1976). The complexity of LU updating, in Anderssen and Brent (eds.), The Complexity of Computational Problem Solving, Picture of symmetrically permuted P T ŪP with row spike to be eliminated Michael Saunders Sparsity vs stability in LU updates 29/31

45 References (contd) J. K. Reid (1982). A sparsity-exploiting variant of the Bartels-Golub decomposition for linear programming bases, Mathematical Programming 24, U stored by rows to facilitate BG update Michael Saunders Sparsity vs stability in LU updates 30/31

46 References (contd) J. K. Reid (1982). A sparsity-exploiting variant of the Bartels-Golub decomposition for linear programming bases, Mathematical Programming 24, U stored by rows to facilitate BG update P. E. Gill, W. Murray, M. A. Saunders, and M. H. Wright (1987). Maintaining LU factors of a general sparse matrix, Linear Algebra and its Applics. 88/89, Rectangular LU, more general updates, cond(l) controlled throughout Michael Saunders Sparsity vs stability in LU updates 30/31

47 References (contd) J. K. Reid (1982). A sparsity-exploiting variant of the Bartels-Golub decomposition for linear programming bases, Mathematical Programming 24, U stored by rows to facilitate BG update P. E. Gill, W. Murray, M. A. Saunders, and M. H. Wright (1987). Maintaining LU factors of a general sparse matrix, Linear Algebra and its Applics. 88/89, Rectangular LU, more general updates, cond(l) controlled throughout P. E. Gill, W. Murray, and M. A. Saunders (2005). SNOPT: An SQP algorithm for large-scale constrained optimization, SIAM Review 47:1, LU with Threshold Rook Pivoting, same general updates Michael Saunders Sparsity vs stability in LU updates 30/31

48 Forrest-Tomlin Michael Saunders Sparsity vs stability in LU updates 31/31