Algebraic and Geometric ideas in the theory of Discrete Optimization

Algebraic and Geometric ideas in the theory of Discrete Optimization Jesús A. De Loera, UC Davis Three Lectures based on the book: Algebraic & Geometric Ideas in the Theory of Discrete Optimization (SIAM-MOS 2013) By J. De Loera, R. Hemmecke & M. Köppe July 15, 2013 () July 15, 2013 1 / 25

Algebraic and Geometric Ideas in the Theory of Discrete Optimization offers several research technologies not yet well known among practitioners of discrete optimization, minimizes prerequisites for learning these methods, and provides a transition from linear discrete optimization to nonlinear discrete optimization. This book can be used as a textbook for advanced undergraduates or beginning graduate students in mathematics, computer science, or operations research or as a tutorial for mathematicians, engineers, and scientists engaged in computation who wish to delve more deeply into how and why algorithms do or do not work. Jesús A. De Loera is a professor of mathematics and a member of the Graduate Groups in Computer Science and Applied Mathematics at University of California, Davis. His research has been recognized by an Alexander von Humboldt Fellowship, the UC Davis Chancellor Fellow award, and the 2010 INFORMS Computing Society Prize. He is an associate editor of SIAM Journal on Discrete Mathematics and Discrete Optimization. Raymond Hemmecke is a professor of combinatorial optimization at Technische Universität München. His research interests include algebraic statistics, computer algebra, and bioinformatics. J. A. De Loera, R. Hemmecke, M. Köppe Society for Industrial and Applied Mathematics 3600 Market Street, 6th Floor Philadelphia, PA 19104-2688 USA +1-215-382-9800 Fax +1-215-386-7999 siam@siam.org www.siam.org Mathematical Optimization Society 3600 Market Street, 6th Floor Philadelphia, PA 19104-2688 USA +1-215-382-9800 x319 Fax +1-215-386-7999 service@mathopt.org www.mathopt.org Algebraic and Geometric Ideas in the Theory of Discrete Optimization Jesús A. De Loera Raymond Hemmecke Matthias Köppe Matthias Köppe is a professor of mathematics and a member of the Graduate Groups in Computer Science and Applied Mathematics at University of California, Davis. He is an associate editor of Mathematical Programming, Series A and Asia-Pacific Journal of Operational Research. Algebraic and Geometric Ideas in the Theory of Discrete Optimization This book presents recent advances in the mathematical theory of discrete optimization, particularly those supported by methods from algebraic geometry, commutative algebra, convex and discrete geometry, generating functions, and other tools normally considered outside the standard curriculum in optimization. MO14 MOS -SIA MS Jesús A. De Loera Raymond Hemmecke Matthias Köppe eries ISBN 978-1-611972-43-6 on O ptim izatio n MO14 () July 15, 2013 2 / 25

() July 15, 2013 3 / 25

Menu for Lectures For Lecture ONE: Motivation and Main Statements Non-Linear Polynomials and Discrete Optimization. Some Theorems on Non-Linear Discrete Optimization For Lecture TWO: A taste of the Math Generating Function Methods Graver Bases Methods For Lecture THREE: Using Polynomials when one does not expect them! Hilbert Nullstellensatz and Colorability problems. Central Path of Interior Point Methods. () July 15, 2013 4 / 25

Polynomials and Discrete Optimization Historical Transitioning from linear to non-linear constraints in models () July 15, 2013 5 / 25

Now we have INTEGER or BINARY variables max f (x) subject to g i (x) 0, i = 1, 2,..., k, h j (x) = 0, j = 1, 2,..., m, x R n1 Z n2. Here the objective function f and the constraint functions g i, h j are assumed to be arbitrary real-valued functions. KEY POINT: The study of these problems requires more ideas from Algebra, Geometry and Topology. (1) LET US GO BACK IN TIME... () July 15, 2013 6 / 25

A Classical Example from the beginning of Discrete Optimization Initial work by Kantorovich (1939), T.C Koopmans (1941), von Neumann (1947). The Transportation problem: A company builds laptops in four factories, each with certain supply power. Four cities have laptop demands. There is a cost c i,j for transporting a laptop from factory i to city j. What is the best assignment of transport in order to minimize the cost? SUPPLIES BY FACTORIES 108 286 71 127 220 215 93 64 DEMANDS ON FOUR CITIES A silly way to solve this: run through all possibilities! Well how do I do this?? Not so easy... If supply and demand are all ONE and if number of cities and () 9 July 15, 2013 7 / 25

Modeling with LINEAR equations and inequalities Let x i,j be a variable indicating number of laptops factory i provides to city j. x i,j can only take non-negative integer values, x i,j 0. Then Since factory i produces a i laptops we have n x i,j = a i, for all i = 1,..., n. j=1 and since city j needs b j laptops n x i,j = b j, for all j = 1,..., n. i=1 Now we minimize c i,j x i,j. () July 15, 2013 8 / 25

Overview LINEAR Discrete Optimization (circa 1990) Efficient computation with Convex Sets & Lattices Efficient Optimization () July 15, 2013 9 / 25

At the beginning there was... Linear programs Special integer programs Integer programs max c x max c x max c x s.t. Ax b s.t. Ax b s.t. Ax b all x i integer all x i integer max c Matrix A is SPECIAL! () July 15, 2013 10 / 25

At the beginning there was... Linear programs Special integer programs Integer programs max c x max c x max c x s.t. Ax b s.t. Ax b s.t. Ax b all x i integer all x i integer max c Easy (polynomial-time solvable) Matrix A is SPECIAL! Medium (can be easy or hard) Network problems Fixed dimension knapsacks 0-1 matrices max c Hard (NP-hard) () July 15, 2013 10 / 25

Integer Linear Programming: The state of the art Traditional Algorithms Dual (polyhedral) techniques Enumeration Adhoc methods Cutting plane algorithms based on polyhedral theory Branch-and-bound special structure (e.g. network, matroids, etc.) Mathematical modelling Strong initial IP formulation () July 15, 2013 11 / 25

Integer Linear Programming: The state of the art Traditional Algorithms Dual (polyhedral) techniques Enumeration Adhoc methods max c x 2 x 0 x 1 Cutting plane algorithms based on polyhedral theory Branch-and-bound special structure (e.g. network, matroids, etc.) Mathematical modelling Strong initial IP formulation () July 15, 2013 11 / 25

Integer Linear Programming: The state of the art Traditional Algorithms Dual (polyhedral) techniques Enumeration Adhoc methods max c x 2 x 0 x 1 max c x 0 Cutting plane algorithms based on polyhedral theory Branch-and-bound special structure (e.g. network, matroids, etc.) Mathematical modelling Strong initial IP formulation () July 15, 2013 11 / 25

MANY CHALLENGES!! LIFE IS NON-LINEAR!! () July 15, 2013 12 / 25

Example: Non-linear transportation polytopes 1 In the traditional transportation problem cost at an edge is a constant. Thus we optimize a linear function. 2 but due to congestion or heavy traffic or heavy communication load the transportation cost on an edge could be a non-linear function of the flow at each edge. 3 For example cost at each edge is f ij (x ij ) = c ij x ij a ij for suitable constant a ij. This results on a non-linear function f ij which is much harder to minimize. () July 15, 2013 13 / 25

Reality is NON-LINEAR and worse!! Non-linear Discrete Optimization max/min f (x 1,..., x d ) subject to g j (x 1,..., x d ) 0, for j = 1... s, and with with x i integer with f, g j Non-Linear BAD NEWS: The problem is INCREDIBLY HARD Theorem It is UNDECIDABLE already when f,g i s are arbitrary polynomials (Jeroslow, 1979). EVEN WORSE Theorem: It undecidable even with number of variables=10. (Matiyasevich and Davis 1982). WHAT CAN BE DONE IN THIS GENERAL CONTEXT?? Prove good theorems? Are there efficient algorithms? No theorem or algorithm performance can be proved without ASSUMPTIONS Let us see two nice theorems that were proved in the last years () July 15, 2013 14 / 25

How about polyhedral constraints non-linear objective?? Let f be a multivariate polynomial function, max s.t. f(x) Ax b Special programs max f(x) s.t. Ax b all x i integer max s.t. f(x) Ax b all x i integer Matrix A is SPECIAL! () July 15, 2013 15 / 25

How about polyhedral constraints non-linear objective?? Let f be a multivariate polynomial function, max s.t. f(x) Ax b Hard (NP-hard) Special programs max f(x) s.t. Ax b all x i integer max s.t. f(x) Ax b all x i integer Hard (NP-hard) Matrix A is SPECIAL! () July 15, 2013 15 / 25

How about polyhedral constraints non-linear objective?? Let f be a multivariate polynomial function, max s.t. f(x) Ax b Hard (NP-hard) Special programs max s.t. f(x) Ax b all x i integer max s.t. f(x) Ax b all x i integer Hard (NP-hard) Matrix A is SPECIAL!??? We study TWO special cases () July 15, 2013 15 / 25

MAIN DISH TWO EXAMPLES OF ALGEBRAIC-GEOMETRIC IDEAS FOR DISCRETE OPTIMIZATION () July 15, 2013 16 / 25

Problem type max f (x 1,..., x d ) subject to (x 1,..., x d ) P Z d, where P is a polytope (bounded polyhedron) given by linear constraints, f is a (multivariate) polynomial function non-negative over P Z d, the dimension d is fixed. Prior Work Integer Linear Programming can be solved in polynomial time (H. W. Lenstra Jr, 1983) Convex polynomials f can be minimized in polynomial time (Khachiyan and Porkolab, 2000) WHAT CAN BE PROVED IN THIS CASE?? Lemma Optimizing an arbitrary degree-4 polynomial f over the lattice points of a polygon is NP-hard NP-complete to decide whether, given three positive integers a, b, c, there exists a positive integer x < c such that x 2 is congruent with a modulo b. () July 15, 2013 17 / 25

Theorem (FPTAS for Integer Polynomial Maximization) JDL, Hemmecke, Köppe, Weismantel, 2006 Let the dimension d be fixed. There exists an algorithm whose input data are a polytope P R d, given by rational linear inequalities, and a polynomial f Z[x 1,..., x d ] with integer coefficients and maximum total degree D that is non-negative on P Z d with the following properties. 1 For a given k, it computes in running time polynomial in k, the encoding size of P and f, and D lower and upper bounds L k f (x max ) U k satisfying ( ) k U k L k P Z d 1 f (x max ). 2 For k = (1 + 1/ɛ) log( P Z d ), the bounds satisfy U k L k ɛ f (x max ), and they can be computed in time polynomial in the input size, the total degree D, and 1/ɛ. 3 By iterated bisection of P Z d, it constructs a feasible solution x ɛ P Z d with f (xɛ ) f (x max ) ɛf (x max ). () July 15, 2013 18 / 25

Example : Multiobjective Transportation Problems 1 In the traditional transportation problem one cost per edge. Thus we optimize ONE linear function. 2 but the cost of an edge for the company may not be the same as for an environmentalist or the government!! We face many cost functions at the same time. 3 So we get three or more costs per edge and we are looking to find points where three or more linear functionals are minimized. 4 The three objective functions induce a partial order over the lattice points in the feasible region. 5 The multiobjective optimization approach is to find the minimal elements of a partially ordered set, the Pareto Optima. () July 15, 2013 19 / 25

Main results JDL, Hemmecke, Köppe, 2010 Theorem (Counting and enumeration theorem) Let the dimension n and the number k of objective functions be fixed. Using the input data A Z m n, an m-vector b, and linear functions f 1,..., f k Z n, (i) there exists a polynomial-time algorithm to exactly count the Pareto optima and the Pareto strategies; (ii) there exists a polynomial-space polynomial-delay prescribed-order enumeration algorithm to generate the full sequence of Pareto optima ordered lexicographically. (iii) There exists a polynomial-time algorithm to find a Pareto optimum v that minimizes the distance v ˆv from a prescribed point ˆv Z k for an arbitrary polyhedral norm. (Again in the spirit of Lenstra s 1983 polynomial-time algorithm for ILP in fixed dimension.) () July 15, 2013 20 / 25

Theorem: Convex Integer Optimization on Transportation Polytopes JDL, Hemmecke, Onn, Rothblum, Weismantel, 2009 Problem: Convex function c : R d R, find a nonnegative integer vector x N n maximizing max {c(w 1 x,..., w d x) : Ax = b, x N n }. INTERPRETATION: Given d linear objective functions w 1,..., w d, want to maximize their convex balancing c(w 1 x,..., w d x) over feasible lattice integer points. Theorem (convex balancing on transportation problems) For any fixed d, p there is a polynomial oracle-time algorithm that, given n, arrays w 1,..., w d Z p n, and convex c : R d R given by comparison oracle, solves the convex integer transportation problem with p many suppliers. max{ c(w 1 x,..., w d x) : x N p n, p x i,j = z j, i n x i,j = v i } j () July 15, 2013 21 / 25

Integer Optimization over N-fold Systems Fix any pair of integer matrices A and B with the same number of columns, of dimensions r q and s q, respectively. The n-fold matrix of the ordered pair A, B is the following (s + nr) nq matrix, [A, B] (n) := (1 n B) (I n A) = B B B B A 0 0 0 0 A 0 0....... 0 0 0 A N-fold systems DO appear in applications! Transportation problems with fixed number of suppliers are examples! Example: Consider the matrices A = [1 1] and B = I 2. [A, B] (4) = 1 0 1 0 1 0 1 0 0 1 0 1 0 1 0 1 1 1 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 1 1.. () July 15, 2013 22 / 25

Theorem Fix two integer matrices A, B of sizes r q and s q, respectively. Then there is a polynomial time algorithm that, given any n, an integer vectors b, a cost vector c, solves the corresponding n-fold integer programming problem. max{cx : [A, B] (n) x = b, x N nq }. Similarly, Given a constant number of cost vectors c 1,..., c k, a convex function f, then there is a polynomial time algorithm that given any n, an integer vectors b, a cost vector c, max{f (c 1 x, c 2 x,..., c k x) : [A, B] (n) x = b, x N nq }. Recent Advances: Extensions to convex minimization, stochastic integer optimization, by Hemmecke, J. Lee, S. Onn, M. Köppe, Note: Compare to polynomial-time of network flow integer programs (E. Tardos )!! () July 15, 2013 23 / 25

WHAT IS THE MATH INSIDE? Two clever algebraic encodings of the lattice points in Polyhedra!! Stay tuned for the second lecture! () July 15, 2013 24 / 25

Thank you Danke Merci Gracias () July 15, 2013 25 / 25