McMaster University. Advanced Optimization Laboratory. Title: Computational Experience with Self-Regular Based Interior Point Methods

Transcription

1 McMaster University Advanced Optimization Laboratory Title: Computational Experience with Self-Regular Based Interior Point Methods Authors: Guoqing Zhang, Jiming Peng, Tamás Terlaky, Lois Zhu AdvOl-Report No. 2002/1 June 2002, Hamilton, Ontario, Canada

2 CORS 2002 Toronto Computational Experience with Self-Regular Based Interior Point Methods Guoqing Zhang Coauthors: J. Peng, T. Terlaky, Lois Zhu Department of Computing and Software McMaster University Toronto, Ontario June 3, 2002

3 Outline Introduction Interior Point Methods for LO The Infeasible SR IPM Algorithms Implementation Issues Numerical Results Comparison Conclusions, Ongoing Works 1

4 Introduction Linear Optimization (LO) problems: min{c T x : Ax = b, x 0}, x R n, A R m n 1947: (Dantzig) Simplex method: the first practical method for LO. 1956: (Tucker) Homogeneous Self-dual model. 1972: (Klee, Minty) Exponential example for the Simplex method: O(2 n ) pivots in the worst case. 1978: (Khachiyan) Ellipsoid method: the first polynomial algorithm for LO. Complexity: O(n 2 L) iterations, O(n 4 L) bits operations, where L = the input size of the problem. 1984: (Karmarkar) Interior Point Methods (IPMs): O(nL) iterations, O(n 3.5 L) bit operations. 2

5 Interior Point Methods for LO 1950s-60s: (Frisch) Logarithmic barrier method, (Huard ) Method of centers, (Dikin) Affine scaling methods: first IPMs. 1984: (Kamarkar) Projective methods. 1989: (Kojima-Mizuno-Yoshise) Primal-dual path following methods. 1989, 1992: (Mehrotra) Predictor-corrector. 2000: (Peng, Roos, Terlaky) Self-Regular IPMs: Best known worst-case complexity for large-update IPMs: O ( n log n log n ε ). 3

6 Self-Regular Function ψ(t) is Self-Regular (SR) if 3 SR1: ψ(t) is strongly convex, global minimum: ψ(1) = 0,, ν 1, ν 2 > 0 and p, q 1, such that for t (0, + ) (p=1,q=3) (p=1, q=1) (p=1, q=3) ν 1 (t p 1 + t 1 q ) ψ (t) ν 2 (t p 1 + t 1 q ), 1.5 SR2: For any t 1, t 2 > 0, r [0, 1]. 1 ψ(t r 1 t1 r 2 ) rψ(t 1 ) + (1 r)ψ(t 2 ), 0.5 (p=1,q=1) q : barrier degree; p : growth degree

7 Examples of Self-Regular Functions We have two sets of SR functions in R n ++ R +: and Υ p,q (t) = t p+1 1 p(p + 1) + p 1 (t 1) log t, q = 1 p t p+1 1 p(p + 1) + t1 q 1 q(q 1) + p q (t 1), q > 1. pq (1) Γ p,q (t) = tp+1 1 p t1 q 1, p 1, q > 1. (2) q 1 Υ 1,1 (t) is a well-known logarithmic barrier function. Ψ is a proximity measure, distance of v to e = (1, 1,..., 1) T. Let Ψ(v) = n i=1 ψ(v i ) where ψ(t) is a SR function. 5

8 The Linear Optimization Problem (I) Consider the following LO problem: min c T x s.t Ax = b, 0 x i u i, i I, 0 x j, j J, where A R m n, rank(a) = m, I J = {1, 2,..., n} and I J =. The last two constraints can be written as F x + s = u, x 0, s 0, F R m f n, the rows of F are unit vectors, s, u R m f, m f = I. 6

9 The Linear Optimization Problem (II) The primal LO problem can be rewritten in the form: (LP) min c T x s.t Ax = b, F x +s = u, x, s 0, where c, x R n, b R m, A R m n, F R m f n. The dual problem is: (LD) max b T y u T w s.t A T y F T w +z = c, w, z 0, where y R m, w R m f and z R n. 7

10 The Linear Optimization Problem (III) Optimality Conditions Ax = b, F x + s = u, A T y F T w + z = c, (3) W s = 0, Xz = 0, where X = diag(x), W = diag(w). 8

11 The Linear Optimization Problem (IV) The primal-dual central path is defined as the set of solutions (x(µ), s(µ)) and (y(µ), w(µ), z(µ)) for µ > 0 of the system where e = [1, 1,..., 1] T. Ax = b, F x + s = u, A T y F T w + z = c, (4) W s = µe, Xz = µe, SR proximity: Ψ( x z, µ) = n i=1 ψ ( ) xi z i µ + m f i=1 ψ ( ) si w i, x z = µ ( ) xz. sw 9

12 SR-Infeasible IPM: Newton equation Self-regular proximity based Newton equation is given as: where A F I A T I F T Z 0 0 X 0 0 W 0 0 S x s y z w = r b r u r c µv 1 Ψ(v 1 ) µv 2 Ψ(v 2 ) r b = Ax b, v 1 = xz µ, r u = F x + s u, v 2 = sw µ, r c = A T y F T w + z c, v = [v 1 v 2 ]., (5) 10

13 Predictor-Corrector Search Directions The affine-scaling direction: A F I A T I F T Z 0 0 X 0 0 W 0 0 S x a s a y a z a w a The Self-Regular based corrector direction: A F I A T I F T Z 0 0 X 0 0 W 0 0 S For µ + see the next page. x s y z w = = r b r u r c XZ W S. (6) r b r u r c µ + v 1 + Ψ(v+ 1 ) xa z a e µ + v 2 + Ψ(v+ 2 ) wa s a e (7). 11

14 Normal Equation for SR-Infeasible IPM After simplifying, normal equation is given as: AD 2 A T y = AD 2 r h r b. (8) Where D 2 = (X 1 Z + S 1 W ) 1 r h = r c + S 1 W r u + S 1 µv 1 Ψ(v 1 ) + X 1 µv 2 Ψ(v 2 ) (9) Then, compute z = X 1 ( µv 1 Ψ(v 1 ) Z x), s = r u x, w = S 1 (w(r u + x) µv 2 Ψ(v 2 )), x = D 2 (A T y r h ). (10) 12

15 SR-Infeasible IPMs Input: Given initial point (x 0, s 0, y 0, z 0, w 0 ), such that (x 0, s 0, z 0, w 0 ) > 0 an accuracy parameter ɛ > 0 begin while x T z + s T w ɛ do predictor solve system (6) and compute the max. feas. step size let µ a = (x+αa p x a ) T (z+α a d za )+(s+α a p s a ) T (w+α a d wa ) n+m f corrector end end compute target µ + = (µ a /µ) 3 µ solve (7) for x, s, y, z, w Compute the max. feas. step size α p, α d Update the iterate (x, s) = (x, s) + α p ( x, s) Update the iterate (y, z, w) = (y, z, w) + α d ( y, z, w) 13

16 Sparse Cholesky Solver If the matrix AD 2 A T is sparse, we can use WSMP directly to solve the system AD 2 A T y = ξ by using the following steps: 0. Ordering and Symbolic factorization for the structure of AD 2 A T, just once in the whole algorithm 1. Numerical factorization of AD 2 A T 2. Back solve to get the solution 3. Iterative refinement if necessary If matrix A has one or more dense columns, then AD 2 A T will be dense. 14

17 Numerical Problems with Cholesky Handle Bad and Small Pivots by WSMP: Bad pivot threshold Small pivot threshold 10-7 Small pivot subst. Bad pivot subst. If the pivot value Bad pivot threshold, it will be substituted by Bad pivot subst. If the pivot value Small pivot threshold, it will be substituted by Small pivot subst. Singularity: [ART] check and handle by K. Anderson s algorithm 15

18 Choice of Self-Regular Function We choose SR functions as Υ 1,q (t) or Γ 1,q (t) Fixed settings: p = 1, and q = 1, 2 or 3 Dynamic update of q: Set q = 1 as default, steptol = 10 2 calculate search direction while α steptol, then let q = q + 2 change µ to x T z x T z 1 calculate search direction if q 5, then break end make step reset q = 1 16

19 Numerical Results: compare with OSL,LIPSOL System environment: IBM RS/ P Model 270 Workstation with AIX 4.3 Coding with C, WSMP, OSL, and ESSL. Benchmark problems in Netlib. 17

20 Testing Results From SR-Infeasible IPM and Comparisons (1) SR-IIPM Problem Iter Flag-q Digits 25fv bau3b adlittle afiro agg agg agg bandm beaconfd blend bnl bnl boeing boeing bore3d Iter OSL Digits Iter LIPSOL Digits

21 Testing Results From SR-Infeasible IPM and Comparisons (2) SR-IIPM Problem Iter Flag-q Digits brandy capri cycle czprob d2q06c d6cube degen degen dfl * 1 10 e etamacro fffff finnis fit1d fit1p Iter OSL Digits Iter LIPSOL Digits

22 Testing Results From SR-Infeasible IPM and Comparisons (3) SR-IIPM Problem Iter Flag-q Digits fit2d forplan ganges gfrd-pnc greenbea 38* 1 7 greenbeb grow grow grow israel kb lotfi maros maros-r Iter OSL Digits Iter LIPSOL Digits

23 Testing Results From SR-Infeasible IPM and Comparisons (4) SR-IIPM Problem Iter Flag-q Digits modszk nesm pilot pilot pilotnov recipe sc sc sc50a sc50b scagr scagr scfxm scfxm2 23 * 0 10 scfxm3 23 * 0 8 Iter OSL Digits Iter LIPSOL Digits

24 Testing Results From SR-Infeasible IPM and Comparisons (5) SR-IIPM Problem Iter Flag-q Digits scorpion scrs scsd scsd scsd sctap sctap sctap seba share1b share2b shell ship04l ship04s ship08l Iter OSL Digits Iter LIPSOL Digits

25 Testing Results From SR-Infeasible IPM and Comparisons(7) SR-IIPM Problem Iter Flag-q Digits ship08s ship12l ship12s sierra stair standata standgub standmps stocfor stocfor stocfor truss tuff Iter OSL Digits Iter LIPSOL Digits

26 Testing Results From SR-Infeasible IPM and Comparisons(8) SR-IIPM Problem Iter Flag-q Digits vtpbase wood1p woodw Total Iter OSL Digits Iter LIPSOL Digits

27 Comparisons of SR with Different q Values q=1 q=2 Dynamic Problem Iter Digits Iter Digits Iter Digits degen degen dfl forplan ? 25 6 greenbea wood1p

28 Conclusions: Conclusions, Ongoing Work Our infeasible algorithms promise high precision with reasonable iteration number, though several problems still are in testing. The average iteration number is less than that of LIPSOL, OSL while the solution has a little higher precision. Encouraging computational results, still space to improve. Ongoing work: Embedding IPM based on MATLAB, LIPSOL and WSMP numerical stability of search direction handle numerical singularity of the matrix AD 2 A T adaptive choice of Self-Regular proximity preprocessing and postprocessing Embedding IPM based on OSL, ESSL and WSMP 27

29 Reference Peng J.Peng, C. Roos, and T. Terlaky. Self-Regular proximities and new search directions for linear and semidefinite optimization, Mathematical Programming (2002). ART: E.D.Anderson, C.Roos, T.Terlaky, T.Traflis and J.P.Warners. The use of low-rank updates in interior-point methods. Technical Report, Delft University of Technology, The Netherlands, RTV: C.Roos, T.Terlaky and J.-Ph. Vial. Theory and Algorithms for Linear Optimization. An Interior Approach. John Wiley and Sons, Chichester, UK, W: S. J. Wright. Primal-Dual Interior-Point Methods. SIAM, Philadelphia, Ye: Y. Ye. Interior-Point Algorithms, Theory and Analysis. John Wiley & Sons, Chichester, UK,

30 Appendix Computational Results From SR-Infeasible IPM (1) Problems iter Flag-q Residual Object-value Digits 25fv E E bau3b E E adlittle E E afiro E E agg E E agg E E agg E E bandm E E beaconfd E E blend E E bnl E E+03 7 bnl E E boeing E E boeing E E+02 10

31 Computational Results From SR-Infeasible IPM (2) Problems iter Flag-q Residual Object-value Digits bore3d E E brandy E E capri E E cycle E E czprob E E d2q06c E E+05 7 d6cube E E degen E E degen E E dfl E E e E E etamacro E E+02 7 fffff E E+05 7

32 Computational Results From SR-Infeasible IPM (3) Problems iter Flag-q Residual Object-value Digits finnis E E fit1d E E fit1p E E+03 9 fit2d E E forplan E E+02 6 ganges E E+05 6 gfrd-pnc E E greenbea E E+07 7 greenbeb E E+06 4 grow E E grow E E grow E E israel E E kb E E lotfi E E maros E E+04 11

33 Computational Results From SR-Infeasible IPM (4) Problems iter Flag-q Residual Object-value Digits maros-r E E modszk E E nesm E E+07 6 pilot E E+02 4 pilot E E+02 6 pilotnov E E recipe E E sc E E sc E E sc50a E E sc50b E E scagr E E scagr E E+06 7 scfxm E E+04 11

34 Computational Results From SR-Infeasible IPM (5) Problems iter Flag-q Residual Object-value Digits scfxm E E scfxm E E+04 8 scorpion E E scrs E E+02 5 scsd E E scsd E E scsd E E sctap E E sctap E E sctap E E seba E E share1b E E share2b E E shell E E ship04l E E ship04s E E+06 10

35 Computational Results From SR-Infeasible IPM (6) Problems iter Flag-q Residual Object-value Digits ship08l E E ship08s E E ship12l E E ship12s E E sierra E E stair E E+02 3 standata E E stangub E E standmps E E stocfor E E+04 9 stocfor E E stocfor E E+04 5 truss E E tuff E E vtpbase E E wood1p E E woodw E E+00 11