Inverse Integer Linear Programs: Complexity and Computation

Transcription

1 Inverse Integer Linear Programs: Complexity and Computation Aykut Bulut 1 Ted Ralphs 1 1 COR@L Lab, Department of Industrial and Systems Engineering, Lehigh University INFORMS Computing Society Conference, 6 January, 2013 Bulut, Ralphs (COR@L Lab) Inverse IP 1 / 37

2 Outline 1 Introduction Motivation Formulation 2 Complexity 3 Algorithm 4 Computational Results Bulut, Ralphs (COR@L Lab) Inverse IP 2 / 37

3 Outline 1 Introduction Motivation Formulation 2 Complexity 3 Algorithm 4 Computational Results Bulut, Ralphs (COR@L Lab) Inverse IP 3 / 37

4 What is an Inverse Problem? We consider a mixed integer linear program (MILP) z IP = min x P c x, (MILP) where c R n and P = { x R n + Ax = b } (Z r R n r ). Roughly speaking, a parametric MILP is the problem of finding the sequence of solutions that arise as we vary the input sytematically. The associated inverse problem reverses this process. We are given a solution from the sequence and we want to determine what input would yield it. Typically, we start with a target and try to find the closest input (by some norm) that yields the desirted output. Note that these concepts apply to optimization problems in general, but we only consider MILPs here. Bulut, Ralphs (COR@L Lab) Inverse IP 4 / 37

5 Our Setting In this talk, we consider problems in which the objective function is parameterized. We refer to the original problem (MILP) as the forward problem and consider the original objective vector c to be the target objective. We denote the parametric version of (MILP) by z IP (d) = min x P d x The function z IP here is the value function of (MILP). The associated inverse problem is further specificed by the target solution x 0 P. The inverse problem is then to find argmin { c d z IP (d) = d x 0 } Bulut, Ralphs (COR@L Lab) Inverse IP 5 / 37

6 A Small Example Define forward problem feasible set P as follows and let c = ( 3, 5), x 0 = (1, 1). Figure : P, its convex hull and feasible d cone Bulut, Ralphs (COR@L Lab) Inverse IP 6 / 37

7 Applications Inverse shortest path problems, used in predicting the movement of earthquakes, arising in geophysical sciences [1]. Inverse programs naturally arise in situations when we want to infer a person s utility function from the person s actions. In numerical analysis, errors for solving the system Ax = b are measured in terms of inverse optimization. Any kind of system that we can observe the solutions but can not measure all the parameters that results in these solutions. Bulut, Ralphs (COR@L Lab) Inverse IP 7 / 37

8 Formulating the Inverse Problem For a given c R n, x 0 P, the inverse problem can be stated as the following mathematical program. min c d s.t. d x 0 d x x P. We assume here that P is bounded. Model can be linearized for l 1 and l norms. The convex hull of P is a polytope. Let E be the set of extreme points of convex hull of P. The constraints set can be replaced by the finite set involving members of E. Bulut, Ralphs (COR@L Lab) Inverse IP 8 / 37

9 Inverse IP with l 1 norm z 1 IP = min n i=1 θ i s.t. c i d i θ i i {1, 2,..., n} d i c i θ i i {1, 2,..., n} d x 0 d x x E. Bulut, Ralphs (COR@L Lab) Inverse IP 9 / 37

10 Inverse IP with l norm zip = min y s.t. c i d i y i {1, 2,..., n} d i c i y i {1, 2,..., n} d x 0 d x x E. For the remainder of the presentation, we deal with the case of l norm. Let S represent the feasible set of the inverse IP, defined as S = { (y, d) R R n y c d, d (x 0 x) 0 x E }. Note that S is a polyhedron. Bulut, Ralphs (COR@L Lab) Inverse IP 10 / 37

11 Outline 1 Introduction Motivation Formulation 2 Complexity 3 Algorithm 4 Computational Results Bulut, Ralphs (COR@L Lab) Inverse IP 11 / 37

12 Polynomially Solvable Forward Problems Ahuja and Orlin [1] determined the complexity of the inverse problem when the forward problem is polynomially solvable. The analysis can be easily extended to accomodate forward problems in other classes. Theorem (Ahuja and Orlin [1]) If a forward problem is polynomially solvable for each linear cost function, then corresponding inverse problems under l 1 and l are polynomially solvable. Bulut, Ralphs (COR@L Lab) Inverse IP 12 / 37

13 Forward Problems Define the following problems related to IP and inverse IP. Definition IP decision problem: Given γ Q decide whether d x γ holds for some x P. Definition IP optimization problem: Find solution vector x such that x argmin x P d x or decide the problem is unbounded or decide the problem is infeasible. Bulut, Ralphs (COR@L Lab) Inverse IP 13 / 37

14 Inverse Problems Definition Inverse IP decision problem: Given γ Q decide whether y γ holds for some (y, d) S. Definition Inverse IP optimization problem: Find solution vector (y, d ), such that (y, d ) argmin (y,d) S y. Definition Inverse IP separation problem: Given a vector (y, d) Q Q n, decide whether (y, d) S, and if not, find [ a hyperplane ] { that [ separates ] (y, d) } from S, i.e., find y y π Q n+1 such that π < min π d (y, d) S. d Bulut, Ralphs (COR@L Lab) Inverse IP 14 / 37

15 Equivalence of Optimization and Separation (GLS) The following theorem by Grötschel et al. indicates the relationship between separation and optimization problems. Theorem (Grötschel et al. [2]) Given an oracle for the separation problem, the optimization problem over a given polyhedron with linear objective can be solved in time polynomial in ϕ, n, and the encoding length of objective coefficient vector, where facet complexity of polyhedron is at most ϕ. Bulut, Ralphs (COR@L Lab) Inverse IP 15 / 37

16 Facet/Vertex Complexity Definition Facet complexity: A polyhedron Q has facet-complexity at most ϕ if there exists a system of inequalities that has solution set P and such that encoding length of each inequality of the system is at most ϕ. Definition Vertex complexity: A polyhedron Q has vertex-complexity at most ν if there exist finite sets V and E of rational vectors such that S = conv(v ) + cone(e) and such that each of the vectors in V and E has encoding length at most ν. In case S =, ν n. If S has facet-complexity at most ϕ, then S has vertex-complexity at most 4n 2 ϕ If S has vertex-complexity at most ν, then S has facet-complexity at most 3n 2 ν Bulut, Ralphs (COR@L Lab) Inverse IP 16 / 37

17 Facet/Vertex Complexity Recall that S = { (y, d) R R n y c d, d (x 0 x) 0 x E }. The facet complexity of S is the encoding length of the inequalities d T (x 0 x) 0 x E. This is the facet complexity of the convex hull of P. It is closely related to the vertex complexity of convex hull of P (the encoding length of members of E). For combinatorial problems, this encoding length would generally be n and would be modest in most practical cases. Bulut, Ralphs (COR@L Lab) Inverse IP 17 / 37

18 Complexity of IP optimization/decision problems Recall that S = { (y, d) R R n y c d, d (x 0 x) 0 x E }. It is easy to see that the separation problem for S is an instance of the forward problem. Thus, by GLS, we see that the decision version of the forward problem and the inverse problem are polynomially equivalent. Similarly, the decision versions of both the forward and inverse problems are polynomially equivalent. But are they in the same complexity class? Bulut, Ralphs (COR@L Lab) Inverse IP 18 / 37

19 The Polynomial Hierarchy The polynomial hierarchy is a scheme for classifying multi-level and multi-stage decision problems. We have P 0 := Σ P 0 := Π P 0 := P, where P is the set of decision problems that can be solved in polynomial time. Higher levels are defined recursively as: P k+1 := P ΣP k, Σ P k+1 := NP ΣP k, and Π P k+1 := conp ΣP k. P H is the union of all levels of the hierarchy. Bulut, Ralphs (COR@L Lab) Inverse IP 19 / 37

20 P 2 and NP Where does inverse integer programming fall in the hierarchy? The decision version of inverse integer programming is an example of a problem in P 2. Such problem are polynomially equivalent to problems in NP, but are not themselves in NP. Intuitively, problems in NP are existentially quantified, whereas problems in P 2 are universally quantified. The decision version of the inverse problem requires proving a lower bound on the optimal value of the forward problem (all solutions have cost greater than or equal to a given target. The decision version of the forward problem requires proving an redupper bound (at least one solution has cost less than or equal to a given target). Optimization requires providing an upper and a lower bound. Higher levels of the hierarchy alternate between embedded universal and existentially quantified problems. Bulut, Ralphs (COR@L Lab) Inverse IP 20 / 37

21 Complexity of Inverse Integer Programming Theorem Inverse IP optimization problem under l /l 1 norm is solvable in time polynomial in ϕ, n + 1/2n, and encoding length of (1, 0,..., 0)/(1,..., 1, 0,..., 0), given an oracle for the IP decision problem. Theorem The inverse IP decision problem is in P 2. Conjecture The inverse IP decision problem is complete for P 2. Bulut, Ralphs (COR@L Lab) Inverse IP 21 / 37

22 Outline 1 Introduction Motivation Formulation 2 Complexity 3 Algorithm 4 Computational Results Bulut, Ralphs (COR@L Lab) Inverse IP 22 / 37

23 Proposed Algorithm First, we define two parametric problems named P k and InvP k as follows min x P dkt x (P k ) min y s.t. c i d i y i {1, 2,..., n} (InvP k ) d i c i y i {1, 2,..., n} d x 0 d x where E k 1 is the set of extreme points found so far. E k 1 Bulut, Ralphs (COR@L Lab) Inverse IP 23 / 37

24 Proposed Algorithm Algorithm 1 k 1 d k c Solve P k, x k x while d kt (x 0 x k ) > 0 do k k + 1 Solve InvP k, d k d Solve P k, x k x end while x k is an optimal solution to P k, whereas d k is an optimal solution to invp k. The algorithm stops when a generated cut is not violated by the current solution. Bulut, Ralphs (COR@L Lab) Inverse IP 24 / 37

25 A Small Example: Iteration 1 Let P and convex hull of P given as in Figure. Let c = ( 3, 5) and x 0 = (1, 1). We know d 1 = c. When objective coefficient of forward problem is d 1, optimal solution is x 1 = (8, 2). Figure : Iteration 1, d 1 and x 1 Using x 1, we generate cut d (x 0 x 1 ) 0, i.e. 7d 1 d 2 0. Bulut, Ralphs (COR@L Lab) Inverse IP 25 / 37

26 A Small Example: Iteration 2 Following figure shows feasible cone for d. d 2 is the inverse optimal with the current cut. x 2 is the forward optimal solution when d 2 is objective coefficient vector. Figure : Iteration 2, feasible d cone ((3, 0) is apex), d 2 and x 2 Using x 2, we generate cut d (x 0 x 2 ) 0, i.e. 2d 1 + d 2 0. Bulut, Ralphs (COR@L Lab) Inverse IP 26 / 37

27 A Small Example: Iteration 3 Figure : Iteration 3, feasible d cone, d 3 and x 3 Using x 3, we generate cut d (x 0 x 3 ) 0, i.e. d 1 + d 2 0. Bulut, Ralphs (COR@L Lab) Inverse IP 27 / 37

28 A Small Example: Iteration 4 Figure : Iteration 4, feasible d cone, d 4 and x 4 Using x 4, we generate cut d (x 0 x 4 ) 0, i.e. d 1 + d 2 0. Note that current d 4 does not violate this cut, then d 4 is the optimal solution of inverse problem. Bulut, Ralphs (COR@L Lab) Inverse IP 28 / 37

29 Outline 1 Introduction Motivation Formulation 2 Complexity 3 Algorithm 4 Computational Results Bulut, Ralphs (COR@L Lab) Inverse IP 29 / 37

30 Inverse Problems in Practice The experiments are designed to test the practical difficulties in solving inverse problems. We conjecture that in practice, the difficulties arise from The practical difficulty of solving the forward problem (for a range of objective values) The number of iterations required. The number of iterations required is a function of the structure of the forward problem (number of extreme points) and the distance of the optimal objective from the target. We therefore conjecture that the practical difficulty of solving these problems will vary substantially betwen problem classes. Bulut, Ralphs (COR@L Lab) Inverse IP 30 / 37

31 Computational Results We used MIPLIB3 benchmark problems to create our inverse problems. We perturbed objective function coefficients, solved the problems and used the resulting solutions as x 0 s. Using generated x 0 s we created inverse IP problems. Experiments are conducted using COIN-OR Branch and Cut (Coin-Cbc) and COIN-OR Open Solver Interface (Coin-Osi) tools with Condor on a cluster running Debian operating system. Bulut, Ralphs (COR@L Lab) Inverse IP 31 / 37

32 Table : MIPLIB Iteration number l 1 p. name pert. 0.1 pert. 0.2 pert. 0.3 pert. 0.4 pert. 0.5 pert. 0.6 bell3a blend dcmulti egout enigma flugpl gen gt l152lav lseu mas misc misc mod mod modglob p p pk pp08acuts qiu rgn rout stein vpm Bulut, Ralphs (COR@L Lab) Inverse IP 32 / 37

33 Table : MIPLIB Iteration number l inf p. name pert. 0.1 pert. 0.2 pert. 0.3 pert. 0.4 pert. 0.5 pert. 0.6 air blend cap dcmulti egout enigma flugpl gen gt harp l152lav lseu mas misc misc mod mod modglob p p pk pp08acuts qnet rgn rout stein vpm Bulut, Ralphs (COR@L Lab) Inverse IP 33 / 37

34 Table : TSPLIB Iteration number l 1 p. name pert. 0.1 pert. 0.2 pert. 0.3 pert. 0.4 pert. 0.5 pert. 0.6 att berlin bier burma ch ch eil eil eil kroa kroc krod kroe lin pr pr pr rat rd st u ulysses ulysses Bulut, Ralphs (COR@L Lab) Inverse IP 34 / 37

35 Table : TSPLIB Iteration number l inf p. name pert. 0.1 pert. 0.2 pert. 0.3 pert. 0.4 pert. 0.5 pert. 0.6 att berlin bier burma ch ch eil eil eil kroa kroa krob kroc krod kroe lin pr pr pr pr pr pr rat rd st u ulysses ulysses Bulut, Ralphs (COR@L Lab) Inverse IP 35 / 37

36 References Ravindra K. Ahuja and James B. Orlin. Inverse optimization. Operations Research, 49(5): , September/October Martin Grötschel, Lászlo Lovász, and Alexander Schrijver. Geometric Algorithms and Combinatorial Optimization, volume 2 of Algorithms and Combinatorics. Springer, second corrected edition edition, Larry J. Stockmeyer. The polynomial-time hierarchy. Theor. Comput. Sci., 3(1):1 22, Bulut, Ralphs (COR@L Lab) Inverse IP 36 / 37

37 End of presentation This is end of presentation! Thank you for listening! Bulut, Ralphs Lab) Inverse IP 37 / 37