The Value Function of a Mixed Integer Linear Programs with a Single Constraint MENAL GUZELSOY TED RALPHS ISE Department COR@L Lab Lehigh University tkralphs@lehigh.edu OPT 008, Atlanta, March 4, 008 Thanks: Work supported in part by the National Science Foundation
Outline Definitions Linear Approximation Properties 4
Motivation The goal of this work is to study the structure of the value function of a general MILP. Eventually, we hope this will lead to methods for approximation useful for sensitivity analysis warm starting other methods that requires dual information Computing the value function (or even an approximation) is difficult even in a small neighborhood Our approach is to begin by considering the value functions of various single-row relaxations.
Previous Work Johnson [97,974,979], Jeroslow [978] : the theory of subadditive duality for integer linear programs. Pure Integer Programs: Jeroslow [98] : Gomory functions - maximum of finitely many subadditive functions. Lasserre [004] : generating functions, two-sided Z transformation. Loera et al. [004] : generating functions, global test set. Mixed Integer Programs: Jeroslow [98] : minimum of finitely many Gomory functions. Blair [995] : Jeroslow formula - consisting of a Gomory function and a correction term.
Definitions Definitions Linear Approximation Properties We consider the primal problem min cx, x S (P) c R n, S = {x Z r + R n r + a x = b} with a Q n, b R. The value function of (P) is z(d) = min x S(d) cx, where for a given d R, S(d) = {x Z r + R n r + a x = d}. Assumptions: Let I = {,...,r}, C = {r +,...,n}, N = I C. z(0) = 0 = z : R R {+ }, N + = {i N a i > 0} and N = {i N a i < 0}, r < n, that is, C = z : R R.
Example Definitions Linear Approximation Properties min x + x + x 4 + x 5 + 4 x 6 s.t x x + x + x 4 x 5 + x 6 = b and x, x, x Z +, x 4, x 5, x 6 R +. z F 5-4 7 - - - 0 4 5 5 7 d
Lower Bound (LP Relaxation) Definitions Linear Approximation Properties The value function of the LP relaxation yields a lower bound. In this case, it has a convenient closed form: ηd if d > 0, F L (d) = max{ud ζ u η, u R} = 0 if d = 0, ζd if d < 0. where F L z. η = min{ c i a i i N + } and ζ = max{ c i a i i N }.
Definitions Linear Approximation Properties Upper Bound (Continuous Relaxation) To get an upper bound, we consider only the continuous variables: F U (d) = min{ c i x i η C d if d > 0 a i x i = d, x i 0 i C} = 0 if d = 0 i C i C ζ C d if d < 0 where η C = min{ c i a i i C + = {i C a i > 0}} and ζ C = max{ c i a i i C = {i C a i < 0}} By convention: C + η C =. C ζ C =. F U z
Example (cont d) Definitions Linear Approximation Properties We have η =, ζ = 0, ηc = and ζ C =. Consequently, { F L (d) = d if d 0 { d if d 0 and F 0 if d < 0 U (d) = d if d < 0 z(d) F U(d) F L(d) ζ C 5 η C ζ -4 7 - - - 0 4 5 5 7 η d
Observations Definitions Linear Approximation Properties {η = η C } {z(d) = F U (d) = F L (d) d R + } {ζ = ζ C } {z(d) = F U (d) = F L (d) d R } z(d) F U(d) F L(d) ζ C 5 η C ζ -4 7 - - - 0 4 5 5 7 η d
Observations Definitions Linear Approximation Properties Let d + U = sup{d 0 z(d) = F U (d)} d U = inf{d 0 z(d) = F U (d)}, d + L = inf{d > 0 z(d) = F L (d)} d L = sup{d < 0 z(d) = F L (d)}. z(d) F U(d) F L(d) 5 d d L U d + d + U L -4 7-5 - - 0 4 5 7 d
Observations Definitions Linear Approximation Properties z(d) = F U (d) d (d U, d+ U ) d + L d+ U if d+ L > 0 and d L d U if d L < 0 if b {d R z(d) = F L (d)}, then, z(kb) = kf L (b), k Z + z(d) F U(d) F L(d) 5 d d d L L U d + d + d + U L L -4 7-5 - - 0 4 5 7 d
Observations Definitions Linear Approximation Properties Notice the relation between F U and the linear segments of z: {η C,ζ C } z(d) F U(d) F L(d) 5 d d d L L U d + d + d + U L L -4 7-5 - - 0 4 5 7 d
Redundant Variables Definitions Linear Approximation Properties Let T C be such that t + T if and only if η C < and η C = c t + a t + t T if and only if ζ C > and ζ C = c t a t. and define and similarly, ν(d) = min s.t. c I x I + c T x T a I x I + a T x T = d x I Z I +, x T R T + Then ν(d) = z(d) for all d R. The variables in C\T are redundant. z can be represented with at most continuous variables.
Jeroslow Formula Definitions Linear Approximation Properties Let M Z + be such that for any t T, Maj a t Z for all j I. Then there is a Gomory function g such that z(d) = min {g( d t T t ) + c t (d d a t )} d R t where d t = at Md M a t. The Gomory function above is the value function of a related PILP: g(q) = min c I x I + M c Tx T + z(ϕ)v s.t a I x I + M a Tx T + ϕv = q x I Z I +, x T Z+, T v Z + for all q R, where ϕ = M t T a t.
Definitions Linear Approximation Properties Piecewise-Linearity of the Value Function For t T, setting we can write ω t (d) = g( d t ) + c t a t (d d t ) d R, z(d) = min t T ω t(d) d R For t T, ω t is piecewise linear with finitely many linear segments on any closed interval and each of those linear segments has a slope of η C if t = t + or ζ C if t = t. Thus, z is also piecewise-linear with finitely many linear segments on any closed interval. Furthermore, each of those linear segments has a slope of η C or ζ C.
Structure of Linear Pieces Definitions Linear Approximation Properties Theorem If the value function z is linear on an interval U R, then there exists a ȳ Z I + such that ȳ is the integral part of an optimal solution for any d U. Consequently, for some t T, z can be written as z(d) = c I ȳ + c t a t (d a I ȳ) d U. Furthermore, for any d U, we have d a I ȳ 0 if t = t + and d a I ȳ 0 if t = t.
Example (cont d) Definitions Linear Approximation Properties T = {4, 5} and hence, x 6 is redundant. η C = and ζ C =, z 5-4 7 - - - 0 4 5 5 7 d U = [0, /], U = [/, ], U = [, 7/6], U 4 = [7/6, /],... y = (0 0 0), y = ( 0 0), y = ( 0 0), y 4 = (0 0 ),... d if d U d + / if d U z(d) = d / if d U d + if d U 4...
Continuity Definitions Linear Approximation Properties ω t + is continuous from the right and ω t is continuous from the left. ω t + and ω t are both lower-semicontinuous. Theorem If z is discontinuous at a right-hand-side b R, then there exists a ȳ Z I + such that b a I ȳ = 0. z is lower-semicontinuous. η C < if and only if z is continuous from the right. ζ C > if and only if z is continuous from the left. Both η C and ζ C are finite if and only if z is continuous everywhere.
Example Definitions Linear Approximation Properties η c = /, ζ C =. min x /4x + /4x s.t 5/4x x + /x = b, x, x Z +, x R +. 7-5 - - 5 - - 5 5 7 For each discontinuous point d i, we have d i (5/4y i y i ) = 0 and each linear segment has the slope of η C = /.
Let f : [0, h] R, h > 0 be subadditive and f(0) = 0. The maximal subadditive extension of f from [0, h] to R + is f(d) if d [0, h] f S (d) = inf f(ρ) if d > h, C C(d) ρ C C(d) is the set of all finite collections {ρ,..., ρ R} such that ρ i [0, h], i =,..., R and P R i= ρi = d. Each collection {ρ,..., ρ R} is called an h-partition of d. We can also extend a subadditive function f : [h, 0] R, h < 0 to R similarly. f S is subadditive and if g is any other subadditive extension of f from [0, h] to R +, then g f S (maximality).
Lemma What if we use z as the seed function? We can change the inf to min : Let the function f : [0, h] R be defined by f(d) = z(d) d [0, h]. Then, z(d) if d [0, h] f S (d) = min z(ρ) if d > h. C C(d) ρ C For any h > 0, z(d) f S (d) d R +. Observe that for d R +, f S (d) z(d) while h. Is there an h < such that f S (d) = z(d) d R +?
We can get the value function by extending it from a specific neighborhood. Theorem Let d r = max{a i i N} and d l = min{a i i N} and let the functions f r and f l be the maximal subadditive extensions of z from the intervals [0, d r ] and [d l, 0] to R + and R, respectively. Let { fr (d) d R F(d) = + f l (d) d R then, z = F. Outline of the Proof. z F : By construction. z F : Using MILP duality, F is dual feasible. In other words, the value function is completely encoded by the breakpoints in [d l, d r ] and slopes.
Constructing The Value Function Two questions Can we obtain the list of breakpoints efficiently? Can we obtain z(d) for some d [d l, d r] from the resulting encoding? We address the second question first. Consider evaluating z(d) = min C C(d) ρ C Can we limit C, C C(d)? Yes! Can we limit C(d)? Yes! z(ρ) for d [d l, d r ].
Theorem Let d > d r and let k d be the integer such that d ( k d d r, k d+ d r ]. Then z(d) = min{ k d i= z(ρ i ) k d i= ρ i = d,ρ i [0, d r ], i =,..., k d }. Therefore, C k d for any C C(d). How about C(d)?
Lower Break Points Theorem Set Ψ be the lower break points of z in [0, d r ]. For any d R + \[0, d r ] there is an optimal d r -partition C C(d) such that C\Ψ. In particular, we only need to consider the collection Λ(d) {H {µ} H C(d µ) Ψ k d, ρ Hρ + µ = d, µ [0, d r ]} In other words, z(d) = min C Λ(d) ρ C Observe that the set Λ(d) is finite. z(ρ) d R + \[0, d r ]
Example (cont d) For the interval [ 0, ], we have Ψ = {0,, }. For b = 5 8, C = { 8,, } is an optimal d r -partition with C\Ψ =. z(d) U-bp L-bp 5-4 7 - - - 0 4 5 5 7 d
Combinatorial Approach We can formulate the problem of evaluating z(d) for d R + \[0, d r ] as a Constrained Shortest Path Problem. Among the paths (feasible partitions) of size k b with exactly k b edges (members of each partition) of each path chosen from Ψ, we need to find the minimum-cost path with a total length of d.
Recursive Construction Set Ψ(p) to the set of the lower break points of z in the interval (0, p] p R +. Let p := d r. For any d `p, p + p, let z(d) = min{z(ρ ) + z(ρ ) ρ + ρ = d, ρ Ψ(p), ρ (0, p]} Let p := p + p and repeat this step.
We can also do the following: z(d) = min g j (d) d j ( p, p + p ] where, for each d j Ψ(p), the functions g j : [ 0, p + p ] R { } are defined as z(d) if d d j, g j (d) = z(d j ) + z(d d j ) if d j < d p + d j, otherwise. Because of subadditivity, we can then write z(d) = min g j (d) d j ( 0, p + p ].
Example (cont d) Extending the value function of () from [ [ ] 0, ] to 0, 9 4 z(d) F U(d) F L(d) 5-4 7 - - - 0 4 5 5 7 d
Example (cont d) Extending the value function of () from [ [ ] 0, ] to 0, 9 4 z(d) F U(d) F L(d) 5 g g -4 7 - - - 0 4 5 5 7 d
Example (cont d) Extending the value function of () from [ [ ] 0, ] to 0, 9 4 z(d) F U(d) F L(d) 5-4 7 - - - 0 4 5 5 7 d
Example (cont d) Extending the value function of () from [ [ ] 0, 4] 9 to 0, 7 8 z(d) F U(d) F L(d) 5-4 7 - - - 0 4 5 5 7 d
Example (cont d) Extending the value function of () from [ [ ] 0, 4] 9 to 0, 7 8 z(d) F U(d) F L(d) 5 g g g g 4-4 7 - - - 0 4 5 5 7 d
Example (cont d) Extending the value function of () from [ [ ] 0, 4] 9 to 0, 7 8 z(d) F U(d) F L(d) 5-4 7 - - - 0 4 5 5 7 d
Finding the Breakpoints Note that z overlaps with F U = η C d in a right-neighborhood of origin and with F U = ζ C d in a left-neighborhood of origin. The slope of each linear segment is either η C or ζ C. Furthermore, if both η C and ζ C are finite, then the slopes of linear segments alternate between η C and ζ C (continuity). For d, d [0, d r] (or [d l, 0]), if z(d ) and z(d ) are on the line with the slope of η C (or ζ C ), then z is linear over [d, d ] with the same slope (subadditivity). With these observations, we can formulate a finite algorithm to evaluate z in [d l, d r ].
Example (cont d) η C ζ C 0 Figure: Evaluating z in [0, ]
Example (cont d) η C ζ C 0 Figure: Evaluating z in [0, ]
Example (cont d) η C ζ C 0 Figure: Evaluating z in [0, ]
Example (cont d) η C ζ C 0 Figure: Evaluating z in [0, ]
Computational experiments Extending our results to bounded case. Extending results to multiple rows. Approximating the value function of a general MILP using the value functions of single constraint relaxations. Applying results to bilevel programming.
Jeroslow Formula - General MILP Let the set E consist of the index sets of dual feasible bases of the linear program min{ M c Cx C : M A Cx C = b, x 0} where M Z + such that for any E E, MA E aj Z m for all j I. Theorem (Jeroslow Formular) There is a g G m such that z(d) = min E E g( d E ) + v E(d d E ) d R m with S(d), where for E E, d E = A E A E d and v E is the corresponding basic feasible solution.