Mathematical Developments Arising from Linear Programming

Transcription

1 Mathematical Developments Arising from Linear Programming American Mathematical Society

2 Titles in This Series Volume 1 Markov random fields and their applications, Ross Kindermann and J. Laurie Snell 2 Proceedings of the conference on integration, topology, and geometry in linear spaces, William H. Graves, Editor 3 The closed graph and P-closed graph properties in general topology, T. R. Hamlett and L. L. Herrington 4 Problems of elastic stability and vibrations, Vadim Komkov, Editor 5 Rational constructions of modules for simple Lie algebras, George B. Seligman 6 Umbral calculus and Hopf algebras, Robert Morris, Editor 7 Complex contour integral representation of cardinal spline functions, Walter Schempp 8 Ordered fields and real algebraic geometry, D. W. Dubois and T. Recio, 9 Papers in algebra, analysis and statistics, R. Lidl, Editor 10 Operator algebres and K-theory, Ronald G. Douglas and Claude Schochet, 11 Plane ellipticity and related problems, Robert P. Gilbert, Editor Proceedings of the Northwestern homotopy theory conference, Haynes R. Miller and Stewart B. Priddy, Low dimensional topology, Samuel J. Lomonaco, Jr., Editor Topological methods in nonlinear functional analysis, S. P. Singh, S. Thomeier, and B. Watson, Factorizations of bn & 1, b = 2, 3,5,6,7,10,11,12 up to high powers, John Brillhart, D. H. Lehmer, J. L. Selfridge, Bryant Tuckerman, and S. S. Wagstaff, Jr. Chapter 9 of Ramanujan's second notebook-infinite series identities, transformations, and evaluations, Bruce C. Berndt and Padmini T. Joshi Central extensions, Galois groups, and ideal class groups of number fields, A. Frohlich Value distribution theory and its applications, Chung-Chun Yang, Editor Conference in modern analysis and probability, Richard Beals, Anatole Beck, Alexandra Bellow, and Arshag Hajian, Microlocal analysis, M. Salah Baouendi, Richard Beals, and Linda Preiss Rothschild, 12 Symposium on algebraic topology in Fluids and plasmas: geometry and honor of Jose Adem, Samuel Gitler, dynamics, Jerrold E. Marsden, Editor Editor Automated theorem proving, W. W. 13 Algebraists' homage: Papers in ring theory and related topics, S. A. Amitsur, D. J. Saltman, and G. B. Seligman, 14 Lectures on Nielsen fixed point theory, Boju Jiang 15 Advanced analytic number theory. Part I: Ramification theoretic methods, Carlos J. Moreno 16 Complex representations of GL(2, K) for finite fields K, llya Piatetski-Shapiro 17 Nonlinear partial differential equations, Joel A. Smoller, Editor 18 Fixed points and nonexpansive mappings, Robert C. Sine, Editor Bledsoe and Donald Loveland, Mathematical applications of category theory, J. W. Gray, Editor Axiomatic set theory, James E. Baumgartner, Donald A. Martin, and Saharon Shelah, Proceedings of the conference on Banach algebras and several complex variables, F. Greenleaf and D. Gulick, Contributions to group theory, Kenneth I. Appel, John G. Ratcliffe, and Paul E. Schupp, Combinatorics and algebra, Curtis Greene, Editor

3 Titles in This Series Volume 35 Four-manifold theory, Cameron Gordon and Robion Kirby, 36 Group actions on manifolds, Reinhard Schultz, Editor 37 Conference on algebraic topology in honor of Peter Hilton, Renzo Piccinini and Denis Sjerve, 38 Topics in complex analysis, Dorothy Browne Shaffer, Editor 39 Errett Bishop: Reflections on him and his research, Murray Rosenblatt, Editor 40 Integral bases for affine Lie algebras and their universal enveloping algebras, David Mitzman 41 Particle systems, random media and large deviations, Richard Durrett, Editor 42 Classical real analysis, Daniel Waterman, Editor 43 Group actions on rings, Susan Montgomery, Editor 44 Combinatorial methods in topology and algebraic geometry, John R. Harper and Richard Mandelbaum, 45 Finite groups-coming of age, John McKay, Editor 46 Structure of the standard modules for the affine Lie algebra A?), James Lepowsky and Mirko Primc 47 Linear algebra and its role in systems theory, Richard A. Brualdi, David H. Carlson, Biswa Nath Datta, Charles R. Johnson, and Robert J. Plemmons, 48 Analytic functions of one complex variable, Chung-chun Yang and Chi-tai Chuang, 49 Complex differential geometry and nonlinear differential equations, Yum-Tong Siu, Editor 50 Random matrices and their applications, Joel E. Cohen, Harry Kesten, and Charles M. Newman, 51 Nonlinear problems in geometry, Dennis M. DeTurck, Editor 52 Geometry of normed linear spaces, R. G. Bartle, N. T. Peck, A. L. Peressini, and J. J. Uhl, 53 The Selberg trace formula and related topics, Dennis A. Hejhal, Peter Sarnak, and Audrey Anne Terras, 54 Differential analysis and infinite dimensional spaces, Kondagunta Sundaresan and Srinivasa Swaminathan, 55 Applications of algebraic K-theory to algebraic geometry and number theory, Spencer J. Bloch, R. Keith Dennis, Eric M. Friedlander, and Michael R. Stein, 56 MuFparameter bifurcation theory, Martin Golubitsky and John Guckenheimer, 57 Combinatorics and ordered sets, Ivan Rival, Editor 58.1 The Lefschetz centennial conference. Proceedings on algebraic geometry, D. Sundararaman, Editor The Lefschetz centennial conference. Proceedings on algebraic topology, S. Gitler, Editor The Lefschetz centennial conference. Proceedings on differential equations, A. Verjovsky, Editor 59 Function estimates, J. S. Marron, Editor 60 Nonstrictly hyperbolic conservation laws, Barbara Lee Keyfitz and Herbert C. Kranzer, 61 Residues and traces of differential forms via Hochschild homology, Joseph Lipman 62 Operator algebras and mathematical physics, Palle E. T. Jorgensen and Paul S. Muhly, 63 Integral geometry, Robert L. Bryant, Victor Guillemin, Sigurdur Helgason, and R. 0. Wells, Jr., 64 The legacy of Sonya Kovalevskaya, Linda Keen, Editor 65 Logic and combinatorics, Stephen G. Simpson, Editor 66 Free group rings, Narian Gupta 67 Current trends in arithmetical algebraic geometry, Kenneth A. Ribet, Editor

4 Titles in This Series Volume 68 Differential geometry: The interface between pure and applied mathematics, Mladen Luksic, Clyde Martin, and William Shadwick, 69 Methods and applications of mathematical logic, Walter A. Carnielli and Luiz Paulo de Alcantara, 70 Index theory of elliptic operators, foliations, and operator algebras, Jerome Kaminker, Kenneth C. Millett, and Claude Schochet, 71 Mathematics and general relativity, James A. Isenberg, Editor 72 Fixed point theory and its applications, R. F. Brown, Editor 73 Geometry of random motion, Rick Durrett and Mark A. Pinsky, 74 Geometry of group representations, William M. Goldman and Andy R. Magid, 75 The finite calculus associated with Bessel functions, Frank M. Cholewinski 76 The structure of finite algebras, David C. Hobby and Ralph Mckenzie 77 Number theory and its applications in China, Wang Yuan, Yang Chung-chun, and Pan Chengbiao, 78 Braids, Joan S. Birman and Anatoly Libgober, 79 Regular differential forms, Ernst Kunz and Rolf Waldi 80 Statistical inference from stochastic processes, N. U. Prabhu, Editor 81 Hamiltonian dynamical systems, Kenneth R. Meyer and Donald G. Saari, 82 Classical groups and related topics, Alexander J. Hahn, Donald G. James, and Zhe-xian Wan, 83 Algebraic K-theory and algebraic number theory, Michael R. Stein and R. Keith Dennis, 84 Partition problems in topology, Stevo Todorcevic 85 Banach space theory, Bor-Luh Lin, Editor 86 Representation theory and number theory in connection with the local Langlands conjecture, J. Ritter, Editor 87 Abelian group theory, Laszlo Fuchs, Rudiger Gobel, and Phillip Schultz, 88 Invariant theory, R. Fossum, W. Haboush, M. Hochster, and V. Lakshmibai, 89 Graphs and algorithms, R. Bruce Richter, Editor 90 Singularities, Richard Randell, Editor 91 Commutative harmonic analysis, David Colella, Editor 92 Categories in computer science and logic, John W. Gray and Andre Scedrov, 93 Representation theory, group rings, and coding theory, M. Isaacs, A. Lichtman, D. Passman, S. Sehgal, N. J. A. Sloane, and H. Zassenhaus, 94 Measure and measurable dynamics, R. Daniel Mauldin, R. M. Shortt, and Cesar E. Silva, 95 Infinite algebraic extensions of finite fields, Joel V. Brawley and George E. Schnibben 96 Algebraic topology, Mark Mahowald and Stewart Priddy, 97 Dynamics and control of multibody systems, J. E. Marsden, P. S. Krishnaprasad, and J. C. Simo, 98 Every planar map is four colorable, Kenneth Appel and Wolfgang Haken 99 The connection between infinite dimensional and finite dimensional dynamical systems, Basil Nicolaenko, Ciprian Foias, and Roger Temam, 100 Current progress in hyperbolic systems: Riemann problems and computations, W. Brent Lindquist, Editor 101 Recent developments in geometry, S.-Y. Cheng, H. Choi, and Robert E. Greene, 102 Primes associated to an ideal, Stephen McAdam 103 Coloring theories, Steve Fisk

5 Titles in This Series Volume 104 Accessible categories: The foundations of categorical model theory, Michael Makkai and Robert Pare 105 Geometric and topological invariants of elliptic operators, Jerome Kaminker, Editor 106 Logic and computation, Wilfried Sieg, Editor 107 Harmonic analysis and partial differential equations, Mario Milman and Tomas Schonbek, 108 Mathematics of nonlinear science, Melvyn S. Berger, Editor 109 Combinatorial group theory, Benjamin Fine, Anthony Gaglione, and Francis C. Y. Tang, 110 Lie algebras and related topics, Georgia Benkart and J. Marshall Osborn, 111 Finite geometries and combinatorial designs, Earl S. Kramer and Spyros S. Magliveras, 112 Statistical analysis of measurement error models and applications, Philip J. Brown and Wayne A. Fuller, 113 Integral geometry and tomography, Eric Grinberg and Eric Todd Quinto, 114 Mathematical developments arising from linear programming, Jeffrey C. Lagarias and Michael J. Todd,

6 Mathematical Developments Arising from Linear Programming

7 ATHEMATICS Mathematical Developments Arising from Linear Programming Proceedings of a Joint Summer Research Conference held at Bowdoin College, June 25-July 1, 1988 Jeffrey C. Lagarias and Michael J. Todd, AMERICAN MATHEMATICAL SOCIETY PROVIDENCE, RHODE ISLAND

8 EDITORIAL BOARD Richard W. Beals, managing editor Sylvain E. Cappell Jan Mycielski Jonathan Goodman Linda Preiss Rothschild Craig Huneke Michael E. Taylor The AMS-IMS-SIAM Joint Summer Research Conference on Mathematical Developments Arising from Linear Programming was held at Bowdoin College, Brunswick, Maine, on June 25-July 1, 1988, with support from the National Science Foundation, Grant DMS , the U.S. Army Research Office, Grant DMS , and the Office of Naval Research, Grant /87/A0544. This work relates to Department of Navy, Grant N J-1019 issued by the Office of Naval Research. The United States Government has a royalty-free license throughout the world in all copyrightable material contained herein Mathematics Subject Classification (1985 Revision). Primary 90C, 65K, 49D; Secondary 52A25,58F07. Library of Congress Cataloging-in-Publication Data AMS-IMS-SIAM Joint Summer Research Conference on Mathematical Developments Arising from Linear Programming (1988: Bowdoin College) Mathematical developments arising from linear programming: proceedings of the AMS-IMS-SIAM joint summer research conference held June 25-July 1, 1988, with support from the National Science Foundation, the U.S. Army Research Office, and the Office of Naval ResearchiJeffrey C. Lagarias and Michael J. Todd, editors. p. cm.-(contemporary mathematics, ISSN ; 11 4) Includes bibliographical references. ISBN Mathematical programming-congresses. 2. Linear programming-congresses. I. Lagarias, Jeffrey C., II. Todd, Michael J., Ill. Title. IV. Series: Contemporary mathematics (American Mathematical Society); v QA402.5.A '2-dc20 CIP 1990, American Mathematical Society. All rights reserved. The American Mathematical Society retains all rights except those granted to the United States Government. Printed in the United States of America. Information on copying and reprinting can be found at the back of this volume. The paper used in this book is acid-free and falls within the guidelines established to ensure permanence and This publication was typeset using AMS-TEX, the American Mathematical Society's T$ macro system.

9 Contents 1. Recent Progress and New Directions Some Recent Results on Convex Polytopes CARL W. LEE Probabilistic Analysis of the Simplex Method KARL HEINZ BORGWARDT On Solving the Linear Programming Problem Approximately NIMROD MEGIDDO Riemannian Geometry Underlying Interior-Point Methods for Linear Programming NARENDRA KARMARKAR Steepest Descent, Linear Programming, and Hamiltonian Flows A. M. BLOCH 2. Interior-Point Methods for Linear Programming An O(n3L) Potential Reduction Algorithm for Linear Programming YINYU YE I. I. Dikin's Convergence Result for the Affine-Scaling Algorithm R. J. VANDERBEI AND J. C. LAGARIAS Phase 1 Search Directions for a Primal-Dual Interior Point Method for Linear Programming IRVIN J. LUSTIG Some Results Concerning Convergence of the Affine Scaling Algorithm EARL R. BARNES Dual Ellipsoids and Degeneracy in the Projective Algorithm for Linear Programming KURT M. ANSTREICHER

10 x CONTENTS A Note on Limiting Behavior of the Projective and the Affine Rescaling Algorithms MIROSLAV D. ASIC, VERA V. KOVA~EVIC-VUJ~IC, AND MIRJANA D. RADOSAVLJEVIC-NIKOLIC Trajectories of Interior-Point Methods On the Convergence Behavior of Trajectories for Linear Programming CHRISTOPH WITZGALL, PAUL T. BOGGS, AND PAUL D. DOMICH 161 Limiting Behavior of the Affine Scaling Continuous Trajectories for Linear Programming Problems ILAN ADLER AND RENATO D. C. MONTEIRO 189 Convergence and Boundary Behavior of the Projective Scaling Trajectories for Linear Programming RENATO D. C. MONTEIRO Nonlinear Optimization On the Complexity of a Numerical Algorithm for Solving Generalized Convex Quadratic Programs by Following a Central Path F. JARRE, G. SONNEVEND, AND J. STOER 233 Canonical Problems for Quadratic Programming and Projective Methods for Their Solution BAHMAN KALANTARI 243 An Interior Point Algorithm for Solving Smooth Convex Programs Based on Newton's Method SANJAY MEHROTRA AND JIE SUN 265 A Modified Kantorovich Inequality for the Convergence of Newton's Method A. A. GOLDSTEIN Integer Programming and Multi-Objective Programming An Interior-Point Approach to NP-complete Problems-Part I NARENDRA KARMARKAR 297 Solving Matching Problems Using Karmarkar's Algorithm JOHN E. MITCHELL AND MICHAEL J. TODD Efficient Faces of Polytopes: Interior Point Algorithms, Parameterization of Algebraic Varieties, and Multiple Objective Optimization S. S. ABHYANKAR, T. L. MORIN, AND T. TRAFALIS 319

11 Preface This volume contains the proceedings of the AMS Summer Research Conference on Mathematical Developments Arising from Linear Programming held at Bowdoin College, June 25-July 1, This conference presented current research in linear and nonlinear programming and related areas of mathematics. There has been intense work in this area, much of it in extending and understanding the ideas underlying N. Karmarkar's interior-point linear programming algorithm, which was proposed in This research effort is interdisciplinary, and the conference brought together mathematicians, computer scientists, and operations researchers. The state of the field in 1987 is illustrated in Progress in Mathematical Programming, Interior-Point and Related Methods (N. Megiddo, ed.), Springer- Verlag, Berlin and New York, To put the results presented in this volume in perspective, we first review the state of knowledge at that time. Karmarkar's algorithm is an interior-point method for solving linear programs. It requires as input a linear program provided with a special initial starting point in the interior of the polytope of feasible solutions, called the center. There are two related types of algorithms, the projective scaling algorithm, which uses projective transformations, that Karmarkar proved to be a polynomial time algorithm, and the afine scaling algorithm, which uses affine transformations, and which has not, in general, been proved to converge in polynomial time (and probably does not). The affine scaling algorithm, however, has computational advantages in practice, and many of the computer implementations of "Karmarkar's algorithm" actually use affine scaling ideas. For both algorithms there is a vector field on the polytope of feasible solutions, which yields differential equations giving trajectories of feasible solutions inside the polytope all of which go to an optimal solution. The affine and projective scaling methods have different trajectories in general, but they have one trajectory in common, the central trajectory or central path, which turns out also to be a logarithmic barrier function trajectory previously studied in connection with nonlinear programming algorithms. Karmarkar's method,

12 xii PREFACE and many subsequent algorithms, approximately follow the central trajectory. Karmarkar originally proved that a linear program (in equality form) in n dimensions and with input size L takes O(nL) iterations to converge and o(~~.~l) arithmetic operations in total. J. Renegar, using path-following ideas, found an algorithm requiring at most O(fiL) iterations. By early 1987 P. Vaidya and, independently, C. Gonzaga had obtained methods that followed the central trajectory requiring 0(n3~) arithmetic operations in total. Karmarkar's algorithm also used nonlinear programming ideas involving minimizing a "potential function." Such ideas carry over to give a polynomial time algorithm for convex quadratic programming as was shown by S. Kapoor and P. Vaidya in 1987, and for a class of linear complementarity problems by M. Kojima, S. Mizuno, and A. Yoshise. Finally, much work was done on developing computationally efficient versions of interior-point methods, including variants of the projective scaling algorithm that can handle problems in standard form and which do not require advance knowledge of the optimal objective function value. Now we describe the results presented in this volume. The conference had ten invited talks which were intended to provide broad views of recent work in various areas related to linear programming. Section 1 presents papers based on such talks. The remaining sections present contributed papers classified by subject area, most of which use interior-point ideas. The papers in Section 1 reflect the wide range of areas of mathematics on which linear programming impinges. Convex polytopes are the basic mathematical objects underlying linear programming problems and the simplex method. The paper of C. L. Lee describes recent results on the combinatorial structure of convex polytopes. K. H. Borgwardt surveys "average-case" polynomial running time bounds obtained for variants of the simplex method under various probability models. The paper of N. Megiddo analyzes several different notions of "approximate solution" of a linear programming problem. N. Karmarkar presents new results that use Riemannian geometric methods to study the behavior of trajectories underlying interior-point methods. His results yield insights concerning the running time of such algorithms. Finally the paper of A. Bloch surveys gradient-like flows arising from several algorithms, including interior-point linear programming algorithms, the QRalgorithm for diagonalizing symmetric matrices, and methods for the total least squares problem, and matching problems. It shows that these flows are related to completely integrable Hamiltonian systems by suitable nonlinear transformations. Section 2 presents results on interior-point methods for linear programming. The paper of Y. Ye gives an O(JiiL) iteration algorithm using a new class of potential functions that apparently does not require staying close to the central trajectory. This is a significant advance, because the previously known O(+L) algorithms take "small" steps, so that such algorithms must take on the order of fil iterations to get close to an optimal solution. The

13 PREFACE xiii idea of Ye allows algorithms that can greedily take bigger steps and still the worst-case analysis applies. (In practical implementations one takes bigger steps than the complexity analyses allow.) Ye has since shown that those ideas extend to "projective" algorithms. In another direction, in early 1988 the linear programming community in the West discovered that the affine scaling algorithm was proposed in 1967 by the Soviet mathematician I. I. Dikin, and that he published a proof of convergence for it in R. J. Vanderbei and J. C. Lagarias give an expose of Dikin's proof of convergence, which applies under the assumption of primal nondegeneracy. One of the difficulties of interior-point methods is that they require linear programs to be transformed to a form that comes with an initial interior feasible solution. The paper of I. Lustig presents a new method for obtaining an initial feasible point for a linear program and its dual. E. Barnes discusses another method for obtaining a feasible starting point for the affine scaling algorithm and gives a convergence result for this algorithm. The paper of K. Anstreicher considers ellipsoids containing dual optimal solutions for a projective scaling interior point method in the case of primal degeneracy. M. ASiC, V. KovaEeviC-VujEic, and M. RadosavljeviC-Nikoli analyze the asymptotic behavior of Karmarkar's algorithm, obtaining results valid for degenerate linear programs. They also propose a rounding method to go from an interior feasible point to the exact optimal solution. Section 3 presents results on the trajetories determined by "infinitesimal" versions of the affine scaling and projective scaling algorithms. The paper of C. Witzgall, P. Boggs, and P. Domich and the paper of I. Adler and R. Monteiro both prove that limiting behavior exists for affine scaling trajectories. Their results apply even to degenerate linear programs, including cases where the convergence of the affine scaling algorithm has not yet been proved. The paper of R. C. Monteiro analyzes the boundary behavior of projective scaling trajectories. Section 4 presents results for nonlinear programming problems. F. Jarre, G. Sonnevend, and J. Stoer give a polynomial-time interior point method for solving convex quadratic programming problems having convex quadratic constraints. B. Kalantari studies the problem of finding a zero of a quadratic form over a simplex, a problem which is NP-complete in general. He gives a "projective" algorithm for finding a local minimum of a certain potential function, which gives a polynomial-time algorithm for solving certain convex quadratic programs including linear programming. S. Mehrotra and J. Sun present an interior-point method for smooth convex programming. A. Goldstein gives a criterion specifying a quadratic convergence region when using Newton's method to find a zero of a nonlinear function. Section 5 presents results for integer programming and multi-objective programming. The paper of N. Karmarkar gives an interior-point approach to solving 0-1 integer programming problems. Such problems, which are NPcomplete in general, are converted to nonconvex quadratic programs on a

14 xiv PREFACE hypercube. Karmarkar isolates a subclass of such problems for which the set of optimal solutions is connected. He suggests that this approach will solve a large class of 0-1 integer programs not previously considered tractable, including many set covering problems. J. Mitchell and M. Todd study perfect matching problems, which are a class of integer programs known to be solvable in polynomial time. They describe a cutting plane method for such problems that uses interior-point methods to solve linear programming relaxations of the problem and present computational data. Finally, the paper of S. Abhyankar, T. Morin, and T. Trafalis outlines two methods for solving multi-objective linear programs, including an interior-point method to find a single efficient solution, and a method of circumscribed algebraic sets to find the entire set of efficient solutions. We would like to take this opportunity to thank the anonymous referees. We also thank the National Science Foundation and Office of Naval Research for their support of the conference. The breadth of topics covered owed much to the valuable advice of the organizing committee, consisting of Victor Klee and Steve Smale. Finally, the success of the meeting owed much to the excellent local arrangements and support of the AMS staff, in particular Ms. Carole Kohanski. The papers in this volume are in final form and no version will be submitted for publication elsewhere, except for the paper by I ~in J. Lustig and the paper by John E. Mitchell and Michael J. Todd. Jeffrey C. Lagarias Michael J. Todd

15 Copying and reprinting. Individual readers of this publication, and nonprofit libraries acting for them, are permitted to make fair use of the material, such as to copy an article for use in teaching or research. Permission is granted to quote brief passages from this publication in reviews, provided the customary acknowledgement of the source is given. Republication, systematic copying, or multiple reproduction of any material in this publication (including abstracts) is permitted only under license from the American Mathematical Society. Requests for such permission should be addressed to the Manager of Editorial Services, American Mathematical Society, P.O. Box 6248, Providence, Rhode Island The appearance of the code on the first page of an article in this book indicates the copyright owner's consent for copying beyond that permitted by Sections 107 or 108 of the U.S. Copyright Law, provided that the fee of $1.OO plus $.25 per page for each copy be paid directly to the Copyright Clearance Center, Inc., 27 Congress Street, Salem, Massachusetts This consent does not extend to other kinds of copying, such as copying for general distribution, for advertising or promotional purposes, for creating new collective works, or for resale.

16 ISBN ISBN