AN EFFICIENT APPROACH TO UPDATING SIMPLEX MULTIPLIERS IN THE SIMPLEX ALGORITHM

Transcription

1 AN EFFICIENT APPROACH TO UPDATING SIMPLEX MULTIPLIERS IN THE SIMPLEX ALGORITHM JIAN-FENG HU AND PING-QI PAN Abstract. The simplex algorithm computes the simplex multipliers by solving a system (or two triangular systems) at each iteration. This note offers an efficient approach to updating the simplex multipliers in conjunction with the Bartels-Golub and Forrest-Tomlin updates for LU factors of the basis. It only solves one triangular system. The approach was implemented within and tested against MINOS 5.51 on 129 problems from Netlib, Kennington and BPMPD. Computational results show that the new approach improves simplex implementations. Key words. linear programming, simplex algorithm, simplex multipliers, LU factorization, recurrence approach, Bartels-Golub update, Forrest-Tomlin update AMS subject classifications. 65K05, 90C05 1. Introduction. Consider the following linear programming (LP) problem minimize c T x subject to Ax = b, l x u. (1.1) where A R m n (m < n), rank(a) = m, c, l, u R n, b R m. Let B be the current basis and let N be the associated nonbasis. Without confusion, we will henceforth denote both the basic (nonbasic) index set and the basis (nonbasis) by the same notation. For instance, c B R m is the vector consisting of basic components of c, and c N R n m consisting of its nonbasic components, and so on. The solution of the two m m systems at each iteration dominates computations of the simplex algorithm (see, e.g., [5, 14]). One of them defines the simplex multipliers π, i.e., B T π = c B. (1.2) Once π is available, the reduced costs are obtained by so-called pricing, i.e., Let the LU factorization of B be known, say, z N = c N N T π. (1.3) B = LU, (1.4) where L is unit lower-triangular, and U is upper-triangular with nonzero diagonals. Then, the simplex multipliers defined by (1.2) can be obtained by solving the following two triangular systems, successively, U T v = c B, (1.5) L T π = v. (1.6) Department of Mathematics, Southeast University, Nanjing, , People s Republic of China (lakerhjf@163.com). Department of Mathematics, Southeast University, Nanjing, , People s Republic of China (panpq@seu.edu.cn). Project supported by National Natural Science Foundation of China. 1

2 2 Jian-Feng Hu and Ping-Qi Pan 1.1. Standard approaches. Practical approaches were proposed in the past for updating reduced costs or simplex multipliers. Instead of (1.2), as will be decoded, these approaches share a feature of handling a system of the form B T π p = e p, (1.7) where e p denotes the p-th coordinate vector. System (1.7) is practically more advantageous than (1.2), since π p is usually much sparser than π. To simplify our exposition, we assume that the first m indices are basic at current iteration, i.e., B = {1,..., m}. Assume now that an entering index q N = {m + 1,..., n} and a leaving index p B have been determined. Thus, the new basic index set results from the old with p replaced by q. We use a prime to denote the new basis (nonbasis) and its associated vectors (π, π p, z ), and so on; e.g, we have where a. denotes the column of A indexed by.. B = B + (a q a p )e T p, (1.8) 1.2. Zoutendijk s scheme. By (1.8) and the Sheman-Morrison formula [8], it holds that e T p B 1 a j = e T p B 1 a j /e T p B 1 a q, j {1,..., n}. (1.9) Therefore, it can further be shown that z j = z j z q (a T j π p), j {1,..., n}, (1.10) where z q is the old reduced cost associate with the entering index q, and π p is the solution to B T π p = e p. (1.11) Ignoring the basic part of computations, (1.10) can be written z N = z N z qn T π p, (1.12) which is Zoutendijk s formula for updating nonbasic reduced costs [15]. Note that z p = 0 for the leaving index p N Bixby s scheme. Let y be the solution to By = a q. (1.13) Using π p instead of π p, a sightly different formula results from (1.10) and (1.9), that is, or z j = z j (z q /y p )a T j π p, j {1,..., n}, (1.14) z N = z N (z q/y p )N T π p. (1.15) Bixby used the preceding formula [2]. It should be attractive compared with (1.12), as (1.13) is solved for y at each iteration independent of pricing, and the computation of π p is cheaper than that of π p. Moreover, Bixby additionally stored the nonbasic columns by row (provided that computer memory is relatively cheap); therefore, with a single test for each zero component of π p, he avoided all the arithmetic for the corresponding row of N in the computation of N T π p, and consequently achieved substantial speedup, especially in case of m n.

3 An Efficient Approach to Updating Simplex Multipliers in the Simplex Algorithm Tomlin s scheme. A disadvantage of the preceding schemes is that updating the (n m)-vector of nonbasic reduced costs makes partial pricing (computing only a part of the costs) virtually impossible; such a would-be-very-long vector has to be kept in core or buffered in and out. This disadvantage can be removed by updating the simplex multipliers insdead. To this end, substituting into (1.10) gives z j = c j a T j π, j {1,..., n}, (1.16) z j = c j a T j π z q a T j π p = c j a T j (π + z q π p), j {1,..., n}. (1.17) Therefore, it holds that a T j (π + z q π p) = a T j π, j {1,..., n}, (1.18) which along with Rank(A) = m implies that π = π + z q π p. (1.19) Thus, we have proved the following result, forming a base for Tomlin s scheme [10]. Theorem 1.1. Let π be the solution to B T π = c B. If π p solves B T π p = e p (1.11), then π defined by (1.19) solves B T π = c B. The preceding says that (1.19) together with (1.11) can serve as a formula for updating π. According to Tomlin, such doing is favorable in the contexts of using Harris Devex column selection method [9] and/or maintaining LU factors of the basis with Forrest-Tomlin update. It is noted that all the preceding shemes still solve two triangular systems at each iteration. In the next section, we offer a new approach for updating the simplex multipliers, which involves one triangular system only. In section 3, we report computational experiments with a sparse code, implemented within and tested against MINOS 5.51 (the latest version of MINOS) [12], showing the new approach s considerable efficiency. 2. The new approach. In this section, we derive the recurrence approach first, and then discuss the computational complexity of the simplex algorithm using it Derivation. With section 1.4, it is seen that the new simplex multipliers π can be computed by (1.19) along with (1.11). A key point of the proposed approach is that (1.11) can be simplified in conjunction with maintaining LU factors of the basis. We derive the approach in the context of using the Bartels-Golub update [1]. From (1.4) and (1.8), it follows that where B = LR, (2.1) R = U + (h q Ue p )e T p, (2.2) Lh q = a q. (2.3) Note that R is just the upper triangular matrix U, except for column p replaced by h q and that (2.3) is solved for h q at each iteration independent of pricing. Perform a

4 4 Jian-Feng Hu and Ping-Qi Pan cyclic permutation that moves column p of R to the end position m, and apply the same permutation to rows p through m. Then the resulting is an upper triangular matrix, except for the m p possible nonzeros in row m (in columns p through m 1). Thus, if Q denotes the permutation, then the resulting matrix can be written Q T RQ. These entries can be eliminated by a series of Gaussian transformations with some row exchanges [8]. That is to say, permutations P i and unit lower triangular matrices L i, i = 1,..., s (1 s m p), can be determined such that L 1 s P s L 1 1 P 1Q T RQ = U (2.4) is upper triangular with nonzero diagonals. Thus, the LU factorization of B Q follows from (2.1) and (2.4), i.e., B Q = L U, L = LQP T 1 L 1 P T s L s. (2.5) Fortunately, the LU factorization (2.5) is also useful for updating the simplex multipliers, as is claimed in the following theorem. Theorem 2.1. Let π be the solution to B T π = c B and let an LU factorization of B Q be given by (2.5). If π p solves L T π p = (1/u mm)e m, (2.6) where u mm is the m-th diagonal of U, then π defined by π = π + z q π p (1.19) is the solution to B T π = c B. Proof. According to Theorem 1.1, it suffices to show that π p defined by (2.6) solves (1.11). Premultiplying by U T the two sides of (2.6) yields which along with (2.5) and U T e m = u mme m gives U T L T π p = (1/u mm)u T e m, (2.7) Q T B T π p = e m. (2.8) By the definition of the cyclic permutation Q, on the other hand, we have e T p Q = e T m and hence e p = Qe m, (2.9) combining which and (2.8) leads to (1.11). According to Theorem 2.1, (1.19) together with (2.6) can serve as a formula for updating π. This approach involves only one triangular system. It is clear that although the (2.5) was derived from the Bartels-Golub updating process, any LU factorization of B Q (with or without row exchanges) applies. With respect to this, further remarks are in order: 2.2. Bartels-Golub update. LA05 [13] and LUSOL [11] are the best known Bartels-Golub implementations (LUSOL is employed within MINOS [12]). In both cases, the p-th column and row of U are permuted to position l rather than to the end (position m), where l marks the position of the last nonzero in h q. The aim is to reduce the number of transformations L i in (2.4). In practice it is often true that l = m, and there is little loss of efficiency in setting l = m always. This is what we did in our use of LUSOL.

5 An Efficient Approach to Updating Simplex Multipliers in the Simplex Algorithm 5 Table 2.1 The flops ratios for a single iteration Case n=2m n=3m n=4m p=1 p=m p=1 p=m p=1 p=m Classical 4m 2 3m 2 5m 2 4m 2 6m 2 5m 2 Recurrence (7/2)m 2 (5/2)m 2 (9/2)m 2 (7/2)m 2 (11/2)m 2 (9/2)m 2 Ratio(m,n,p) Forrest-Tomlin update. The proposed approach also can be employed in conjuction with the Forrest-Tomlin update (which is used in CPLEX [3]). In fact, this update results in an LU factorization of B Q, in which all permutations in (2.4) are P i = I [6]. (In addition, the matrices L i are combined into a single row transformation.) The update can be implemented more efficiently, at the expense of guaranteed stability Computational complexity. The proposed approach for updating π saves a unit lower triangular system solve. But, how much does it improve the efficiency of the simplex algorithm as a whole? With infinite precision, the simplex variant using the new approach requires the same iterations as its simplex counterpart, and hence what we gain depends on how much it reduces the computational complexity in a single iteration. For dense computations, a simple counting indicates that a single iteration of the standard simplex algorithm requires while the variant requires m 2 + mn + (m p) 2 + 2(m p) + 4m 1 (flops) (1/2)m 2 + mn + (m p) 2 + 2(m p) + (7/2)m 1 (flops). Ignoring the terms of lower order, we obtain the flops ratio of the standard to the variant below: ratio(m, n, p) = 2m2 + 2mn + 2(m p) 2 m 2 > 1. (2.10) + 2mn + 2(m p) 2 It is clear that ratio(m, n, p) increases with p (1 p m) and that 4m + 2n 2m + 2n ratio(m, n, p) 3m + 2n m + 2n. (2.11) In particular, we list in Table 2.1 the cases for n = 2m, 3m and 4m. In summary, the computational complexity of the simplex algorithm is reduced, and hence the proposed approach is favorable for dense computations. 3. Computational experiments. To see how it performs in large-scale sparse computations, the proposed approach was implemented within and compared with MINOS 5.51 [12]. The resulting code, named NEW, was a slight modification of MINOS 5.51, using the proposed approach in both Phases 1 and 2. In Phase 1, it was utilized to update π until a piecewice linear objective renewed. The implementation of the proposed approach is a simple matter. Compiled using the Visual Fortran 5.0, code NEW and MINOS 5.51 were both run under a Windows 2000 system on a PentiumIV 1.7GHz personal computer with

6 6 Jian-Feng Hu and Ping-Qi Pan 256MB of memory. The machine precision was about 16 decimal digits. The CPU time was measured in seconds with utility routine CP U T IME. Tested problems were classified into three sets. Set 1 included 96 problems from Netlib 1. In fact, these are all of the Netlib problems available, except for QAP15, as it is too time-consuming to solve for both MINOS 5.51 and NEW. Set 2 included all the 16 problems from Kennington 2, and set 3 included all 17 problems from BPMPD 3 that were of sizes of no less than 500KB, in compressed form. Numerical results obtained with set 1 3 are displayed in Tables respectively, in the order of increasing sum m + n before slack variables are added. In these tables, statistics obtained with MINOS 5.51 and NEW are listed under columns labeled MINOS 5.51 and NEW and the total iterations and run time required for solving each problem are listed in columns labeled Itns and Time.Final objective values reached are not listed. Tables compare the performance of the two codes by giving iterations and time ratios of MINOS 5.51 to NEW for each problem. These results are summarized in Table 3.7, where the 96 Netlib problems are categorized into three groups: group Small includes the first 38 problems (AFIRO through DEGEN2), Medium includes the next 41 problems (FIT1D through CYCLE), and Large the last 17 problems (SHIP08L through STOCFOR3). Serving as an overall comparison between the two codes, the bottom six lines of Table 3.7 indicate that iteration ratios are higher that one, with the total iteration ratio The NEW s gain should be due to less accumulated roundoff errors. Moreover, time ratios are even much higher. It appears that time ratios increase with sizes of the tested problems. The bottom line labeled Total gives the total time ratio Thus, code NEW outperformed MINOS 5.51 in terms of both iterations and run time. This credit was entirely due to the use of the new recurrence approach. Finally, we conclude that the proposed approach for updating the simplex multipliers should be utilized in simplex implementations. Acknowledgment. The authors are grateful to Professor Michael A. Saunders for providing them the MINOS 5.51 package. They are also grateful to an anonymous referee for his very valuable comments and assistance, which improved this article considerably. In particular, we thank the referee for his idea of using the proposed approach in both Phases 1 and 2. REFERENCES [1] R. H. Bartels and G. H. Golub, The simpex method of linear programming using LU decomposition, Communication ACM, 12 (1969), [2] R.E. Bixby, Progress in linear programming, ORSA J. on Computing, 6(1994), [3] CPLEX, CPLEX Callable Library, CPLEX Optimization, Inc. [4] G. B. Dantzig, Maximization of a linear function of variables subject to linear inequalities, in: Activity Analysis of Production and Allocation(Tj.C. Koopmans, ed.), Wiley, New York, 1951, [5] G. B. Dantzig, Linear Programming and Extensions, Princeton University Press, Princeton, NJ, [6] J.J.H. Forrest and J.A. Tomlin, Updating triangular factors of the basis to maintain sparsity in the product form simplex method, Mathematical Programming, 2(1972), [7] D. M. Gay, Electronic mail distribution of linear programming test problems, Mathematical Programming Society COAL Newsletter, 13 (1985), meszaros/bpmpd/

7 An Efficient Approach to Updating Simplex Multipliers in the Simplex Algorithm 7 Table 3.1 Statistics for set 1 of 96 Netlib problems Problem m n m + n MINOS 5.51 NEW Itns Time Itns Time AFIRO KB SC50B SC50A ADLITTLE BLEND SHARE2B SC STOCFOR SCAGR RECIPE BOEING ISRAEL SHARE1B VTP.BASE SC BEACONFD GROW LOTFI BRANDY E BORE3D FORPLAN CAPRI AGG BOEING SCORPION BANDM SCTAP SCFXM AGG AGG STAIR SCSD TUFF GROW SCAGR DEGEN FIT1D ETAMACRO FINNIS FFFFF GROW PILOT STANDATA SCSD STANDMPS SEBA

8 8 Jian-Feng Hu and Ping-Qi Pan Problem m n m + n MINOS 5.51 NEW Itns Time Itns Time STANDGUB SCFXM SCRS GFRD-PNC BNL SHIP04S PEROLD MAROS FIT1P MODSZK SHELL SCFXM FV SHIP04L QAP WOOD1P PILOT.JA SCTAP GANGES SCSD PILOTNOV SHIP08S SIERRA DEGEN PILOT.WE NESM SHIP12S SCTAP STOCFOR CZPROB CYCLE SHIP08L PILOT BNL SHIP12L D6CUBE D6CUBE PILOT D2Q06C GREENBEA WOODW FIT2D QAP BAU3B MAROS-R FIT2P DFL STOCFOR

9 An Efficient Approach to Updating Simplex Multipliers in the Simplex Algorithm 9 Table 3.2 Statistics for set 2 of 16 Kennington problems Problem m n m + n MINOS 5.51 NEW Itns Time Itns Time KEN CRE-C CRE-A PDS OSA KEN PDS OSA PDS KEN CRE-D CRE-B OSA PDS OSA KEN Table 3.3 Statistics for set 3 of 17 BPMPD problems Problem m n m + n MINOS 5.51 NEW Itns Time Itns Time RAT7A NSCT NSCT ROUTING DBIR DBIR T0331-4L NEMSEMM SOUTHERN RADIO.PRIM WORLD.MOD WORLD RADIO.DUAL NEMSEMM NW LPL DBIC [8] G.H. Golub and C.F. Van Loan, Matrix Computations, The Johns Hopkins University Press, Baltimore, [9] P.M.J. Harris, Pivot selection methods of the devex LP code, Mathematical Programming, 5(1973), [10] J.A. Tomlin, On pricing and backward trasformation in linear programming, Mathematical Programming, 6(1974), [11] P. E. Gill, W. Murray, M. A. Saunders and M. H. Wright, Maintaining LU factors of a general sparse matrix, Linear Algebra and Its Applications, 88/89 (1987), [12] B.A. Murtagh and M. A. Saunders, MINOS 5.5 User s Guide, Technical Report SOL 83-20R, Dept. of Operations Research, Stanford University, Stanford, [13] J. K. Reid, A sparsity-exploiting variant of the Bartels-Golub decomposition for linear programming bases, Mathematical Programming, 24 (1982), [14] R. J. Vanderbei, Linear Programming: Foundations and Extensions, Kluwer Academic Pub-

10 10 Jian-Feng Hu and Ping-Qi Pan Table 3.4 Ratios of MINOS 5.51 to NEW for set 1 Problem Itns Time Problem Itns Time AFIRO STANDGUB KB SCFXM SC50B SCRS SC50A GFRD-PNC ADLITTLE BNL BLEND SHIP04S SHARE2B PEROLD SC MAROS STOCFOR FIT1P SCAGR MODSZK RECIPE SHELL BOEING SCFXM ISRAEL FV SHARE1B SHIP04L VTP.BASE QAP SC WOOD1P BEACONFD PILOT.JA GROW SCTAP LOTFI GANGES BRANDY SCSD E PILOTNOV BORE3D SHIP08S FORPLAN SIERRA CAPRI DEGEN AGG PILOT.WE BOEING NESM SCORPION SHIP12S BANDM SCTAP SCTAP STOCFOR SCFXM CZPROB AGG CYCLE AGG SHIP08L STAIR PILOT SCSD BNL TUFF SHIP12L GROW D6CUBE SCAGR D6CUBE DEGEN PILOT FIT1D D2Q06C ETAMACRO GREENBEA FINNIS WOODW FFFFF FIT2D GROW QAP PILOT BAU3B STANDATA MAROS-R SCSD FIT2P STANDMPS DFL SEBA STOCFOR

11 An Efficient Approach to Updating Simplex Multipliers in the Simplex Algorithm 11 lishers, Boston, [15] G. Zoutendijk, Methods of feasible directions, Elsevier, Amsterdam, 1960.

12 12 Jian-Feng Hu and Ping-Qi Pan Table 3.5 Ratios of MINOS 5.51 to NEW for set 2 Problem Itns Time KEN CRE-C CRE-A PDS OSA KEN PDS OSA PDS KEN CRE-D CRE-B OSA PDS OSA KEN Table 3.6 Ratios of MINOS 5.51 to NEW for set 3 Problem Itns Time RAT7A NSCT NSCT ROUTING DBIR DBIR T0331-4L NEMSEMM SOUTHERN RADIO.PRIM WORLD.MOD WORLD RADIO.DUAL NEMSEMM NW LPL DBIC

13 An Efficient Approach to Updating Simplex Multipliers in the Simplex Algorithm 13 Table 3.7 Summary for all test problems Problem Itns Time Small(38) Medium(41) For MINOS 5.51 Large(17) Kennington(16) BPMPD(17) Total Small(38) Medium(41) For NEW Large(17) Kennington(16) BPMPD(17) Total Small(38) Medium(41) Ratios of Large(17) M.5.51 to NEW Kennington(16) BPMPD(17) Total