ECONOMETRIC INSTITUTE THE COMPLEXITY OF THE CONSTRAINED GRADIENT METHOD FOR LINEAR PROGRAMMING J. TELGEN REPORT 8005/0

Transcription

1 ECONOMETRIC INSTITUTE THE COMPLEXITY OF THE CONSTRAINED GRADIENT METHOD FOR LINEAR PROGRAMMING ffaciundation OF AGRICULTIlt.E,C"..'NOMICE; 1LT- S E P 3 1/880 J. TELGEN REPORT 8005/0 ERASMUS UNIVERSITY ROTTERDAM P.O. BOX 173a 3000 DR ROTTERDAM. ME NETHERLANDS

2 THE COMPLEXITY OF THE CONSTRAINED GRADIENT METHOD FOR LINEAR PW)GRAMMING* Jan Telgen Econometric Institute Erasmus University Rotterdam and Management Science Program University of Tennessee Abstract It. is proved that the computation of, the constrained gradient is equivalent (in a complexity, theoretic sense), to quadratic programming. Since the *latter is an NP hard problem, the constrained gradient method is not a 'good' method for linear programming unless P = AT. Keywords: Linear programming, Complexity, Gradient methods. The author gratefully acknowledges stimulating comments by J. K. Lens tra and A. H. G. Rinnooy Kan. Visiting on a NATO Science Fellowship awarded by the Netherlands Organization for the Advancement of Pure Research (7,140).

3 1. Introduction It has recently been established that LINEAR PROGRAMMING (LP) is solvable in polynomial time, i.e. LP E P (Chacijan [1979]). The method used to achieve this result is quite different from the simplex method. It was well-known that the standard simplex method is not a formally 'goof' (polynomial time) method for LINEAR PROGRAMMING (Klee and Minty [1972]). Similar results were obtained for those variants of the simplex method in which the pivot column is selected to maximize criterion improvement (Jeroslow [1973]) or to maximize the gradient in the space of all variables (Goldfarb and Sit [1978]). But what about the many other methods that have been proposed especially in the early days of the rise of linear programming? The constrained gradient method is such a method: although it could not compete successfully with the simplex method in solving LP problems in practice, its theoretical quality might still be good. Here we show that this is not true unless P = NP. 2. The Constrained Gradient Method The constrained gradient method is an iterative scheme to solve the linear programming problem T minimize c x { subject to Ax < b where A is mxn, b is mxl and c and x are nxl vectors. Given a feasible solution x we partition A and b as: vi wj and

4 such that 1. Vx = V 1 Wx k < w feasible direction d in x k is defined by or equivalently k k V(x +d ) < v V d <0. The constrained gradient in the point x k Is defined as the feasible direction d along which the objective function value achieves its maximal improvement per unit step length. Assume that a feasible solution x 0 is known. The constrained gradient method consists of the following steps: step 0 step 1 Compute the constrained gradient g k in the point x k step 2 If c T gk > 0, then stop : x k is optimal; step 3 Determine the maximum step size t and compute k+1 x : = x k + t.g set k: = k + 1 and continue with step 1. Several authors have modified this scheme in order to bypass the computation of the constrained gradient from scratch at every iteration (e.g., Rosen [1960], Lemke (1961]). They propose some rules by which the constrained gradient used in the preceding iteration can be adapted t yield a 'constrained gradient direction' for the next iteration. However, these rules do not always yield the correct constrained gradient direction. Therefore we shall confine ourselves here to the basic scheme given above.

5 The constrained gradient method may require more than a polynomially bounded number of computations either because of the number of iterations, or because of the number of computations involved in one iteration. We shall consider the latter question in more detail here; the former type of question was used to show that the simplex method is not a formally 'good' method for linear programming. 3. The Computation Of The Constrained Gradient From its definition we see that the constrained gradient g in the point x k can be computed as the optimal solution for d in min c T d vtd (1) s.t. Vd < 0 If the objective function value in the optimal solution of 1 positive we may solve, instead of (1) max d T d s.t. Vd < 0 c T d = 1 If the objective function value in the optimal solution for 1) is negative we may solve min S. t. dt Vd < 0 ct d = (3) In both case, the optimal solution to the quadratic programming problem (2) or (3) is the constrained gradient g.

6 4 Formally we define the problem: CONSTRAINED GRADIENT COMPUTATION (CGC: Given an integer matix V and an integer vector c determine a rational vector g such that Vg < c T g = 1 and g g is maximal. 4. The main result We shall prove that CONSTRAINED GRADIENT COMPUTATION is NP-hard, by showing that a specific quadratic programming problem can be transformed- to the form (2), which implies that it can be solved by a constrained gradient computation. Since the specific quadratic programming problem we use has been proved to be NP-hard in Sahni [1974] (reduction from PARTITION; see also Carey and Johnson [1979]), this argument is sufficient. The specific quadratic programming problem (SQP) is: max s.t. (4) = 1,...n where s is a given nxl vector, r a given scalar and i an nxl vector of ones. (n+1) (2n+1) x n+1) (n+1) Define y c, P cir and q c IR with

7 5 j 2 j =...,n and r -r Is" 2 and = (0, 0, Now we can reformulate (4) in the form (2) as max Y Y Py < 0 T y (5) Thus, by computing the constrained gradient in the point x = (0...,0) for the objective function min q x with respect to the constraints Px < 0 we can solve the quadratic programming problem (5) and thus (4). Since all transformations are of polynomially bounded size we have shown that SQP is reducible to CGC. Sahni [1974] showed that SQP is NP hard. It follows immediately that CONSTRAINED GRADIENT COMPUTATION is NP hard. Therefore we have proved:

8 Theorem: The constrained gradient method does not yield a polynomial time method for UNEAR PROGRAMMING, unless P = NP. Note that this result is obtained without taking into account the number of iterations the constrained gradient method requires, although this number might very well be superpolynomial as in the case of the simplex method.

9 References Chacijan, L. G., Polynomial algorithm for linear programming, Doklady Akademika USSR, Mathematika. vol. 244, no. 5, 1979, pp Galey, M. R. and D. S. Johnson, Computers and intractability: guide to the theory of NP-completeness, Freeman, Goldfarb, D. and W. Y. Sit, Worst case behaviour of the steepest edge simplex method, submitted to Discrete Mathematics, Jeroslow, R. G., The simplex algorithm with the pivot rule of maximizing criterion improvement, Discrete Mathematics, vol. 4, 1973, pp Klee, V. and G. J. Minty, How good is the simplex algorithm? in: 0. Shisha (ed.) Inequalities III, Academic Press, 1972, pp Lemke, C. E., The constrained gradient method of linear programming, J. Soc. Indust. AppZ. Math., vol. 9, no. 1, 1961, pp Rosen, J. B., The gradient projection method for nonlinear programming, J. Soc. Indust. Appl. Math., vol. 8, no. 1, 1960, pp Sahni, S., Computationally related problems, SIAMejr. Comput., vol. 3, no. 4, 1974, pp

10 LIST OF REPORTS "List of Reprints, nos , Abstracts of Reports Second Half 1979". 8001/0 "A Stochastic Method for Global Optimization", by C.G.E. Boender, A.H.G. Rinnooy Kan, L. Stougie and G.T. Timmer. 8002/M "The General Linear Group of Polynomial Rings over Regular Rings", by A.C.F. Vorst. 8003/0 "A Recursive Approach to the Implementation of Enumerative Methods", by J.K. Lenstra and A.H.G. Rinnooy Kan. 8004/E "Linearization and Estimation of the Addi-Log Budget Allocation Model", by P.M.C. de Boer and J. van Daal /0 "The Complexity of the Constrained Gradient Method for Linear Programming", by J. Telgen.

11