On the Value Function of a Mixed Integer Linear Program

On the Value Function of a Mixed Integer Linear Program MENAL GUZELSOY TED RALPHS ISE Department COR@L Lab Lehigh University ted@lehigh.edu AIRO, Ischia, Italy, 11 September 008 Thanks: Work supported in part by the National Science Foundation

Outline 1 Definitions Linear Approximations Properties 3 4

Motivation The goal of this work is to study the structure of the value function of a general mixed integer linear program (MILP). We hope this will lead to methods for approximation useful for Sensitivity analysis Warm starting Multi-level/hierarchical mathematical programming Other methods that require dual information Constructing the value function (or even an approximation to it) is difficult, even in a small neighborhood. We begin by considering the value functions of single-row relaxations.

Definitions Definitions Linear Approximations Properties We consider the MILP min cx, x S (P) c R n, S = {x Z r + R n r + a x = b} with a Q n, b R. The value function of (P) is z(d) = min x S(d) cx, where for a given d R, S(d) = {x Z r + R n r + a x = d}. Assumptions: Let I = {1,...,r}, C = {r + 1,...,n}, N = I C. z(0) = 0 = z : R R {+ }, N + = {i N a i > 0} and N = {i N a i < 0}, r < n, that is, C 1 = z : R R.

Example Definitions Linear Approximations Properties min 3x 1 + 7 x + 3x 3 + 6x 4 + 7x 5 + 5x 6 s.t 6x 1 + 5x 4x 3 + x 4 7x 5 + x 6 = b and x 1, x, x 3 Z +, x 4, x 5, x 6 R +. (SP) z 18 16 14 1 10 8 6 4-16 -14-1 -10-8 -6-4 - 0 4 6 8 10 1 14 16 18 0 4 6 8 d

Simple Bounding Functions Definitions Linear Approximations Properties F L : LP Relaxation Lower Bounding function F U : Continuous Relaxation Upper Bounding function ηd if d > 0, η C d if d > 0 F L (d) = 0 if d = 0, F U (d) = 0 if d = 0 ζd if d < 0. ζ C d if d < 0 where, setting C + = {i C a i > 0} and C = {i C a i < 0}, η = min{ ci a i i N + } and ζ = max{ ci a i i N } η C = min{ ci a i i C + } and ζ C = max{ ci a i i C }. By convention: C + η C = and C ζ C =. F U z F L

Example (cont d) Definitions Linear Approximations Properties η = 1, ζ = 3 4, ηc = 3 and ζ C = 1: 18 z F U F L ζ C 16 14 1 10 η C ζ 8 6 4 η -16-14 -1-10 -8-6 -4-0 4 6 8 10 1 14 16 18 0 4 6 8 d {η = η C } {z(d) = F U (d) = F L (d) d R + } {ζ = ζ C } {z(d) = F U (d) = F L (d) d R }

Observations Definitions Linear Approximations Properties Consider d + U, d U, d+ L, d L : 18 16 14 1 10 8 6 4-16 d L -14-1 d L -10-8 d L d U -6-4 - d + d + d + 3d + 4d + U L L L L 0 4 6 8 10 1 14 16 18 0 4 6 8 d The relation between F U and the linear segments of z: {η C,ζ C }

Redundant Variables Definitions Linear Approximations Properties Let T C be such that t + T if and only if η C < and η C = c t + a t + t T if and only if ζ C > and ζ C = c t a t. and define and similarly, ν(d) = min s.t. c I x I + c T x T a I x I + a T x T = d x I Z I +, x T R T + Then ν(d) = z(d) for all d R. The variables in C\T are redundant. z can be represented with at most continuous variables.

Example Definitions Linear Approximations Properties min x 1 3/4x + 3/4x 3 s.t 5/4x 1 x + 1/x 3 = b, x 1, x Z +, x 3 R +. η c = 3/, ζ C =. 7-3 5-3 -1 1 5 3 1 1 1-1 3-5 1 1 3 5 3 7 For each discontinuous point d i, we have d i (5/4y i 1 y i ) = 0 and each linear segment has the slope of η C = 3/.

Jeroslow Formula Definitions Linear Approximations Properties Let M Z + be such that for any t T, Maj a t Z for all j I. Then there is a Gomory function g such that z(d) = min t T {g( d t ) + c t a t (d d t )}, d t = a t M Md a t, d R Such a Gomory function can be obtained from the value function of a related PILP. For t T, setting ω t (d) = g( d t ) + c t a t (d d t ) d R, we can write z(d) = min t T ω t(d) d R

Definitions Linear Approximations Properties Piecewise Linearity and Continuity For t T, ω t is piecewise linear with finitely many linear segments on any closed interval and each of those linear segments has a slope of η C if t = t + or ζ C if t = t. ω t + is continuous from the right, ω t is continuous from the left. ω t + and ω t are both lower-semicontinuous. Theorem z is piecewise-linear with finitely many linear segments on any closed interval and each of those linear segments has a slope of η C or ζ C. (Meyer 1975) z is lower-semicontinuous. η C < if and only if z is continuous from the right. ζ C > if and only if z is continuous from the left. Both η C and ζ C are finite if and only if z is continuous everywhere.

Maximal Let f : [0, h] R, h > 0 be subadditive and f(0) = 0. The maximal subadditive extension of f from [0, h] to R + is f(d) if d [0, h] f S (d) = inf f(ρ) if d > h, C C(d) ρ C C(d) is the set of all finite collections {ρ 1,..., ρ R} such that ρ i [0, h], i = 1,..., R and P R i=1 ρi = d. Each collection {ρ 1,..., ρ R} is called an h-partition of d. We can also extend a subadditive function f : [h, 0] R, h < 0 to R similarly. (Bruckner 1960) f S is subadditive and if g is any other subadditive extension of f from [0, h] to R +, then g f S (maximality).

Extending the Value Function Lemma Suppose we use z itself as the seed function. Observe that we can change the inf to min : Let the function f : [0, h] R be defined by f(d) = z(d) d [0, h]. Then, z(d) if d [0, h] f S (d) = min z(ρ) if d > h. C C(d) ρ C For any h > 0, z(d) f S (d) d R +. Observe that for d R +, f S (d) z(d) while h. Is there an h < such that f S (d) = z(d) d R +?

Yes! For large enough h, maximal extension produces the value function itself. Theorem Let d r = max{a i i N} and d l = min{a i i N} and let the functions f r and f l be the maximal subadditive extensions of z from the intervals [0, d r ] and [d l, 0] to R + and R, respectively. Let { fr (d) d R F(d) = + f l (d) d R then, z = F. Outline of the Proof. z F : By construction. z F : Using MILP duality, F is dual feasible. In other words, the value function is completely encoded by the breakpoints in [d l, d r ] and slopes.

General Procedure We will construct the value function in two steps Construct the value function on [d l, d r]. Extend the value function to the entire real line from [d l, d r]. For the rest of the talk We assume η C < and ζ C <. We construct the value function over R + only. These assmuptions are only needed to simplify the presentation.

on [0, d r ] If both η C and ζ C are finite, the value function is continuous and the slopes of the linear segments alternate between η C and ζ C. For d 1, d [0, d r ], if z(d 1 ) and z(d ) are connected by a line with slope η C or ζ C, then z is linear over [d 1, d ] with the respective slope (subadditivity). With these observations, we can formulate a finite algorithm to evaluate z in [d l, d r ].

Example (cont d) d r = 6: η C ζ C 8 6 4 0 4 6 Figure: Evaluating z in [0, 6]

Example (cont d) d r = 6: η C ζ C 8 6 4 0 4 6 Figure: Evaluating z in [0, 6]

Example (cont d) d r = 6: η C ζ C 8 6 4 0 4 6 Figure: Evaluating z in [0, 6]

Example (cont d) d r = 6: η C ζ C 8 6 4 0 4 6 Figure: Evaluating z in [0, 6]

Example (cont d) d r = 6: η C ζ C 8 6 4 0 4 6 Figure: Evaluating z in [0, 6]

Extending the Value Function Consider evaluating z(d) = min C C(d) ρ C Can we limit C, C C(d)? Yes! Can we limit C(d)? Yes! Theorem z(ρ) for d [0, d r ]. Let d > d r and let k d be the integer such that d ( k d d r, k d+1 d r ]. Then z(d) = min{ k d i=1 z(ρ i ) k d i=1 ρ i = d,ρ i [0, d r ], i = 1,..., k d }. Therefore, C k d for any C C(d). How about C(d)?

Lower Break Points Let Ψ be the lower break points of z in [0, d r ]. Theorem For any d R + \[0, d r ] there is an optimal d r -partition C C(d) such that C\Ψ 1. In particular, we only need to consider the collection Λ(d) {H {µ} H C(d µ) Ψ k d 1, ρ Hρ + µ = d, µ [0, d r ]} In other words, z(d) = min C Λ(d) ρ C Observe that the set Λ(d) is finite. z(ρ) d R + \[0, d r ]

Example (cont d) For the interval [0, 6], we have Ψ = {0, 5, 6}. For b = 31, C = {5, 5, 11 } is an optimal d r -partition with C\Ψ = 1. 18 z U-bp L-bp 16 14 1 10 8 6 4-16 -14-1 -10-8 -6-4 - 0 4 6 8 10 1 14 16 18 0 4 6 8 d

Getting z over R + Recursive Construction: Let Ψ((0, p]) to the set of the lower break points of z in the interval (0, p] p R +. 1 Let p := d r. For any d `p, p + p, let z(d) = min{z(ρ 1) + z(ρ ) ρ 1 + ρ = d, ρ 1 Ψ((0, p]), ρ (0, p]} Let p := p + p and repeat this step.

In other words, we do the following at each iteration: ( z(d) = min g j (d) d p, p + p ] j where, for each d j Ψ((0, p]), the functions g j : [ 0, p + p ] R { } are defined as z(d) if d d j, g j (d) = z(d j ) + z(d d j ) if d j < d p + d j, otherwise. Because of subadditivity, we can then write z(d) = min g j (d) d j ( 0, p + p ].

Example (cont d) Extending the value function of (SP) from [0, 6] to [0, 9] 18 z F U F L 16 14 1 10 8 6 4 g 1 g -16-14 -1-10 -8-6 -4-0 4 6 8 10 1 14 16 18 0 4 6 8 d

Example (cont d) Extending the value function of (SP) from [0, 6] to [0, 9] 18 z F U F L 16 14 1 10 8 6 4-16 -14-1 -10-8 -6-4 - 0 4 6 8 10 1 14 16 18 0 4 6 8 d

Example (cont d) Extending the value function of (SP) from [0, 9] to [ 0, 7 ] 18 z F U F L 16 14 g 3 1 10 8 g 1 6 g 4-16 -14-1 -10-8 -6-4 - 0 4 6 8 10 1 14 16 18 0 4 6 8 d

Example (cont d) Extending the value function of (SP) from [0, 9] to [ 0, 7 ] 18 z F U F L 16 14 1 10 8 6 4-16 -14-1 -10-8 -6-4 - 0 4 6 8 10 1 14 16 18 0 4 6 8 d

A Combinatorial Procedure Observe that it is enough to get the lower break points and this can be done more easily. Theorem If d is a lower break-point of z on (p, p + p ] then there exist ρ 1,ρ Ψ((0, p]) such that z(d) = z(ρ 1 ) + z(ρ ) and d = ρ 1 + ρ. Set Υ(p) {z(ρ 1 ) + z(ρ ) p < ρ 1 + ρ p + p,ρ 1,ρ Ψ((0, p])}. Then, z is obtained by connecting the points on the lower envelope of Υ(p). Can we make the procedure finite?

Termination Yes! Periodicity Let D = {d z(d) = F L (d)}. Note that D =. Furthermore, let λ = min{d d d r, d D}. Define the functions f j : R + R, j Z + \{0} as follows { z(d), d jλ f j (d) = kz(λ) + z(d kλ), d ((k + j 1)λ,(k + j)λ], k Z + \{0}. Theorem 1 f j(d) f j+1(d) z(d) for all d R +, j Z +\{0}. There exists q Z +\{0} such that z(d) = f q(d) d R +. 3 In addition, z(d) = f q(d) d R + if and only if f q(d) = f q+1(d) d R +. Therefore, we can extend over the intervals of size λ and stop when we reach the 3. condition above.

Example (cont d) λ = 6, f 1 (d) = { z(d), d 6 kz(6) + z(d 6k), d (6k, 6(k + 1)], k Z + \{0}. 18 16 z F U F L f 1 14 1 10 8 6 4-16 -14-1 -10-8 -6-4 - 0 4 6 8 10 1 14 16 18 0 4 6 8 d

Example (cont d) f (d) = { z(d), d 1 kz(6) + z(d 6k), d (6(k + 1), 6(k + )], k Z + \{0}. 18 16 z F U F L f 14 1 10 8 6 4-16 -14-1 -10-8 -6-4 - 0 4 6 8 10 1 14 16 18 0 4 6 8 d

Example (cont d) f 3 (d) = { z(d), d 18 kz(6) + z(d 6k), d (6(k + ), 6(k + 3)], k Z + \{0}. 18 16 z F U F L f 3 14 1 10 8 6 4-16 -14-1 -10-8 -6-4 - 0 4 6 8 10 1 14 16 18 0 4 6 8 d

Example (cont d) f 4 (d) = { z(d), d 4 kz(6) + z(d 6k), d (6(k + 3), 6(k + 4)], k Z + \{0}. 18 16 z F U F L f 4 14 1 10 8 6 4-16 -14-1 -10-8 -6-4 - 0 4 6 8 10 1 14 16 18 0 4 6 8 d Note that f 4 (d) = f 5 (d) d R +. Therefore, z(d) = f 4 (d) d R +.

A Finite Procedure We can further restrict the search space by again using maximal extension and the fact that z(kλ) = kz(λ) and λ d r. Theorem For a given k, k Z +, z(d) = min{z(ρ 1 ) + z(ρ ) ρ 1 + ρ = d,ρ 1 (0, λ],ρ ((k 1)λ, kλ]} d (kλ,(k + 1)λ]. Revised Recursive Construction: 1 Let p := λ. Set Υ(p) {z(ρ 1) + z(ρ ) p < ρ 1 + ρ p + λ, ρ 1 Ψ((0, λ]), ρ Ψ((p λ, p])} and obtain z over [p, p + λ] by considering the lower subadditive envelope of Υ(p). 3 If z(d) = z(d λ) + z(λ) d Ψ((p, p + λ)), then stop. Otherwise, let p := p + λ and repeat the last step.

Example (cont d) Extending the value function of (SP) from [0, 1] to [0, 18] 18 16 z F U F L Υ 14 1 10 8 6 4-16 -14-1 -10-8 -6-4 - 0 4 6 8 10 1 14 16 18 0 4 6 8 d

Example (cont d) Extending the value function of (SP) from [0, 1] to [0, 18] 18 16 z F U F L Υ 14 1 10 8 6 4-16 -14-1 -10-8 -6-4 - 0 4 6 8 10 1 14 16 18 0 4 6 8 d

Example (cont d) Extending the value function of (SP) from [0, 18] to [0, 4] 18 16 z F U F L Υ 14 1 10 8 6 4-16 -14-1 -10-8 -6-4 - 0 4 6 8 10 1 14 16 18 0 4 6 8 d

Example (cont d) Extending the value function of (SP) from [0, 18] to [0, 4] 18 16 z F U F L Υ 14 1 10 8 6 4-16 -14-1 -10-8 -6-4 - 0 4 6 8 10 1 14 16 18 0 4 6 8 d

Consider a general mixed integer linear program (MILP) z P = min x S cx, (P) c R n, S = {x Z r + R n r + Ax = b} with A Q m n, b R m. The value function of the primal problem (P) is z(d) = min x S(d) cx, where for a given d R m, S(d) = {x Z r + R n r + Ax = d}.

Jeroslow Formula for General MILP Let the set E consist of the index sets of dual feasible bases of the linear program min{ 1 M c Cx C : 1 M A Cx C = b, x 0} where M Z + such that for any E E, MA 1 E aj Z m for all j I. Theorem (Jeroslow Formula) There is a g G m such that z(d) = min E E g( d E ) + v E(d d E ) d R m with S(d), where for E E, d E = A E A 1 E d and v E is the corresponding basic feasible solution.

For E E, setting ω E (d) = g( d E ) + v E (d d E ) d R m with S(d), we can write z(d) = min E E ω E(d) d R m with S(d). Many of our previous results can be extended to general case in the obvious way. Similarly, we can use maximal subadditive extensions to construct the value function.. However, an obvious combinatorial explosion occurs. Therefore, we consider using single row relaxations to get a subadditive approximation.

Basic Idea Consider the value functions of each single row relaxation: z i (q) = min{cx a i x = q, x Z r + R n r + } q R, i M {1,...,m} where a i is the i th row of A. Theorem Let F(d) = max i M {z i(d i )}, d = (d 1,...,d m ), d R m. Then F is subadditive and z(d) F(d) d R m.

Maximal Assume that A Q m +. Let S M and q r Q S + be the vector of the maximum of the coefficients of rows a i, i S. Define max {z i(q i )} q i [0, q r i i S ] i M G S (q) = max max{z q i [0, q r i ] i K i(q i )}, inf G S (ρ) q i > q r i i S\K i K C C(q S\K ) ρ C K S inf G S (ρ) q i R + \[0, q r i ] i M C C(q) ρ C for all q R S where for T S, C(q T ) is the set of all finite collections {ρ 1,...,ρ R },ρ j R T such that ρ j i T [0, q r i ], j = 1,..., R and R j=1 ρ j = q T.

Maximal G S is simply the maximal subadditive extension of the function max {z i(q i )} from the box i S [0, q r i ] to R S +. i S Theorem { } Let F S (d) = max G S (d S ), max {z i (d i )}. F S max {z i(d i )}, is i M\S i M subadditive and z(d) F S (d) for all d R m +.

Aggregation For S M, ω R S, set G S (q,ω) = min{cx ωa S x = ωq, x Z r + R n r + } q R S Theorem Let { } F S (ω, d) = max G S (d S,ω), max {z i (d i )} i M\S, d R m. F S is subadditive and z(d) F S (ω, d) for any ω R S, d R m. As with cutting planes, different aggregation procedures are possible.

Using Cuts Assume that S(d) = {x Z n + Ax d}. Consider the set of Gomory cuts Πx Π 0, Π Q k n,π 0 Q k defined by the sets of multipliers Ω = {ω 1,...ω k 1 }, ω i Q+ m+i 1 as follows m i 1 Π ij = ωla i lj + ωm+lπ i lj i = 1,...,k, j = 1,...,n Theorem Π 0 i = l=1 l=1 m i 1 ωld i l + l=1 l=1 ω i m+lπ 0 l i = 1,...,k For Ω = {ω 1,...ω k 1 },ω i Q+ m+i 1, k Z +, let z m+i (ω i, d) denote the value function of row m + { i, i = 1,...,k 1 and } F(Ω, d) = max max z(d i), max z m+i (ω i, d). i M i=1,...,k 1,ω i Ω Then, F is subadditive and z(d) F(Ω, d) for any d R m.

Extending the theory and algorithms to the general case. Developing upper bounding approximations. Integrating these procedures in with applications Bilevel programming Combinatorial auctions Answering the question Can we do anything practical with any of this?