LARGE SCALE LINEAR AND INTEGER OPTIMIZATION: A UNIFIED APPROACH

Transcription

1 LARGE SCALE LINEAR AND INTEGER OPTIMIZATION: A UNIFIED APPROACH

2 LARGE SCALE LINEAR AND INTEGER OPTIMIZATION: A UNIFIED APPROACH Richard Kipp Martin Graduate School of Business Universify of Chicago ~. " Springer Science+Business Media, LLC

3 Library of Congress Cataloging-in-Publication Data Martin, Richard Kipp. Large scale linear and integer optimization : a united approach / Richard Kipp Martin. p. cm. Includes bibliographical references and index. ISBN l ISBN (ebook) DOI / Linear programming. 2. Mathematical optimization. 1. Title. T57.75.M '2--dc CIP Copyright 1999 Springer Science+Business Media New York Originally published by Kluwer Academic Publishers in 1999 Softcover reprint of the hardcover 1 st edition 1999 AII rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, mechanical, photocopying, record ing, or otherwise, without the prior written permis sion of the publisher, Springer Science+Business Media, LLC. Printed an acid-free paper.

4 This book is dedicated to my parents, Bruce and Phyllis Martin.

5 CONTENTS Preface xv Part I MOTIVATION 1 1 LINEAR AND INTEGER LINEAR OPTIMIZATION Introduction Linear and Integer Linear Optimization A Guided Tour of Applications Special Structure Linear and Integer Linear Programming Codes Other Directions Exercises 29 Part II THEORY 33 2 LINEAR SYSTEMS AND PROJECTION Introduction Projection for Equality Systems: Gaussian Elimination Projection for Inequality Systems: Fourier-Motzkin Elimination Applications of Projection Theorems of the Alternative Duality Theory Complementary Slackness Sensitivity Analysis Conclusion Exercises 75

6 Vlll LARGE SCALE LINEAR AND INTEGER OPTIMIZATION 3 LINEAR SYSTEMS AND INVERSE PROJECTION Introduction Deleting Constraints by Adding Variables Dual Relationships Sensitivity Analysis Conclusion Homework Exercises INTEGER LINEAR SYSTEMS: PROJECTION AND INVERSE PROJECTION ] Introduction Background Material Solving A System of Congruence Equations Integer Linear Equalities Integer Linear Inequalities: Projection Integer Linear Inequalities: Inverse Projection Conclusion Exercises 137 Part III ALGORITHMS THE SIMPLEX ALGORITHM Introduction Motivation Pivoting Revised Simplex Product Form of the Inverse Degeneracy and Cycling Complexity of the Simplex Algorithm Conclusion Exercises MORE ON SIMPLEX Introduction Sensitivity Analysis The Dual Simplex Algorithm 191

7 Contents ix 6.4 Simple Upper Bounds and Special Structure Finding a Starting Basis Pivot Column Selection Other Computational Issues Conclusion Exercises INTERIOR POINT ALGORITHMS: POLYHEDRAL TRANSFORMATIONS Introduction Projective Transformations Karmarkar's Algorithm Polynomial Termination Purification, Standard Form and Sliding Objective Affine Polyhedral Transformations Geometry of the Least Squares Problem Conclusion Exercises INTERIOR POINT ALGORITHMS: BARRIER METHODS Introduction Primal Path Following Dual Path Following Primal-Dual Path Following Polynomial Termination of Path Following Algorithms Relation to Polyhedral Transformation Algorithms Predictor-Corrector Algorithms Other Issues Conclusion Exercises INTEGER PROGRAMMING Introduction Modeling with Integer Variables Branch-and-Bound Node and Variable Selection 324

8 x LARGE SCALE LINEAR AND INTEGER OPTIMIZATION 9.5 More General Branching Conclusion Exercises 341 Part IV SOLVING LARGE SCALE PROBLEMS: DECOMPOSITION METHODS PROJECTION: BENDERS' DECOMPOSITION Introduction The Benders' Algorithm A Location Application Dual Variable Selection Conclusion Exercises INVERSE PROJECTION: DANTZIG-WOLFE DECOMPOSITION Introduction Dantzig-Wolfe Decomposition A Location Application Taking Advantage of Block Angular Structure Computational Issues Conclusion Exercises LAGRANGIAN METHODS Introduction The Lagrangian Dual Extension to Integer Programming Properties of the Lagrangian Dual Optimizing the Lagrangian Dual Computational Issues A Decomposition Algorithm for Integer Programming Conclusion Exercises 435

9 Contents xi Part V SOLVING LARGE SCALE PROBLEMS: USING SPECIAL STRUCTURE SPARSE METHODS Introduction LU Decomposition Sparse LU Update Numeric Cholesky Factorization Symbolic Cholesky Factorization Storing Sparse Matrices Programming Issues Computational Results: Barrier versus Simplex Conclusion Exercises NETWORK FLOW LINEAR PROGRAMS Introduction Totally Unimodular Linear Programs Network Simplex Algorithm Important Network Flow Problems Almost Network Problems Integer Polyhedra Conclusion Exercises LARGE INTEGER PROGRAMS: PREPROCESSING AND CUTTING PLANES Formulation Principles and Techniques Preprocessing Cutting Planes Branch-and-Cut Lifting Lagrangian Cuts Integer Programming Test Problems Conclusion Exercises 563

10 xii LARGE SCALE LINEAR AND INTEGER OPTIMIZATION 16 LARGE INTEGER PROGRAMS: PROJECTION AND INVERSE PROJECTION Introduction Auxiliary Variable Methods A Projection Theorem Branch-and-Price Projection of Extended Formulations: Benders' Decomposition Revisited Conclusion Exercises 630 Part VI APPENDIX 633 A POLYHEDRAL THEORY 635 A.1 Introduction 635 A.2 Concepts and Definitions 635 A.3 Faces of Polyhedra 640 A.4 Finite Basis Theorems 645 A.5 Inner Products, Subspaces and Orthogonal Subspaces 651 A.6 Exercises 653 B COMPLEXITY THEORY 657 B.1 Introduction 657 B.2 Solution Sizes 660 B.3 The Turing Machine 661 B.4 Complexity Classes 663 B.5 Satisfiability 667 B.6 NP-Completeness 669 B.7 Complexity of Gaussian Elimination 670 B.8 Exercises 674 C BASIC GRAPH THEORY 677 D SOFTWARE AND TEST PROBLEMS 681 E NOTATION 683

11 Contents XUI References AUTHOR INDEX TOPIC INDEX

12 PREFACE This is a textbook about linear and integer linear optimization. There is a growing need in industries such as airline, trucking, and financial engineering to solve very large linear and integer linear optimization problems. Building these models requires uniquely trained individuals. Not only must they have a thorough understanding of the theory behind mathematical programming, they must have substantial knowledge of how to solve very large models in today's computing environment. The major goal of the book is to develop the theory of linear and integer linear optimization in a unified manner and then demonstrate how to use this theory in a modern computing environment to solve very large real world problems. After presenting introductory material in Part I, Part II of this book is devoted to the theory of linear and integer linear optimization. This theory is developed using two simple, but unifying ideas: projection and inverse projection. Through projection we take a system of linear inequalities and replace some of the variables with additional linear inequalities. Inverse projection, the dual of this process, involves replacing linear inequalities with additional variables. Fundamental results such as weak and strong duality, theorems of the alternative, complementary slackness, sensitivity analysis, finite basis theorems, etc. are all explained using projection or inverse projection. Indeed, a unique feature of this book is that these fundamental results are developed and explained before the simplex and interior point algorithms are presented. We feel that projection and inverse projection are the very essence of these fundamental results and proofs based upon simplex or interior point algorithms are not insightful. The ideas of projection and inverse projection are also extended to integer linear optimization. With the projection-inverse projection approach theoretical results in integer linear optimization become much more analogous to their linear optimization counterparts. Thus, once armed with these two concepts the reader is equipped to understand fundamental theorems in an intuitive way. In Part III of the book we present the most important algorithms that are used in commercial software for solving real world problems. Even though the algorithms developed in Part III are appropriate for solving large realistic

13 xvi LARGE SCALE LINEAR AND INTEGER OPTIMIZATION problems, they will often fail for very large scale applications unless advantage is taken of the special structure present in the problem. In Part IV we show how to take advantage of special structure through decomposition. We explain these decomposition algorithms as extensions of the projection and inverse projection concepts developed in Part II. In Part V we show how to take advantage of special structure by modifying and enhancing the algorithms developed in Part III. This section contains a discussion of some of the most current research in linear and integer programming. We also show in Part V how to take different problem formulations and appropriately "modify" them so that the algorithms from Part III are much more efficient. Once again, the projection and inverse projection concepts are used in Part V to present the current research in linear and integer linear optimization in a very unified way. An ambitious one quarter or semester course in linear optimization would include Chapters 1-3 and Chapters 5-8. This set of chapters gives the student exposure to the most important aspects of linear optimization theory as well as the simplex and barrier algorithms. If two quarters are available, then the instructor has several options. The second quarter can be devoted to a brief introduction to integer linear optimization and then move on to large scale linear optimization. This material is in Chapters Or, if the instructor wishes to cover integer linear optimization in more depth, then Chapters 4, 9, 12, and provide a solid background. Chapters 4, 9, 12, provide a nice two quarter, or intense semester course in discrete optimization. No prior knowledge of linear or integer linear optimization is assumed, although this text is written for a mathematically mature audience. Our target audience is upper level undergraduate students and graduate students in computer science, applied mathematics, industrial engineering and operations research/management science. Coursework in linear algebra and analysis is sufficient background. Researchers wishing to brush up on recent developments in large scale linear and integer linear programming will also find this text useful. Readers not familiar with any polyhedral theory should read the first two sections of Appendix A. The necessary background on complexity theory is provided in Appendix B. A brief tutorial on graph theory is provided in Appendix C. This book has benefited from the help of numerous people. Former students Mohan Bala, John Borse, Peter Cacioppi, Kevin Cunningham, Zeger Degraeve, Syam Menon, Santosh Nabar, Fritz Raffensperger, Larry Robinson, Jayaram Sankaran, Jack Shi, Sri Sinnathamby, Kamaryn Tanner, Jurgen Tistaert, Mala Viswanath, and Hongsuk Yang suffered through constant revisions, found numerous errors, and provided useful suggestions for improvement. I thank James

14 Preface xvii Evans, William Lippert, Richard Murphy, James Powell, and the late Richard DeMar for contributing greatly to my mathematical education. I have also benefited from my colleagues Gary Eppen, Ronald Rardin, and the late Robert Jeroslow. Luis Gouveia has been particularly helpful in reading much of the manuscript and making suggestions for improvement. Most of all, I thank Dennis Sweeney for teaching me so much mathematical optimization over the years and Linus Schrage for so graciously enduring a constant barrage of questions related to topics in this book. I thank Gary Folven at Kluwer for being a patient editor and enthusiastically supporting this book. Finally, I thank my parents, Bruce and Phyllis Martin, and Gail Honda for their constant encouragement and support throughout this long project.