Integer Programming and Branch and Bound

Transcription

1 Courtesy of Sommer Gentry. Used with permission. Integer Programming and Branch and Bound Sommer Gentry November 4 th, 003 Adapted from slides by Eric Feron and Brian Williams, 6.40, 00. Integer Programming (IP) What is it? Epressing decisions with IP Eclusion between choices Eclusion between constraints Solutions through branch and bound Characteristics Solving Binary IPs Solving Mied IPs and LPs

2 Integer Programs LP: Maimize , 0 b) IP: Maimize , 0, integers a) c) d) e) Integer Programming Integer programs are LPs where some variables are integers Why Integer programming?. Some variables are not real-valued: Boeing only sells complete planes, not fractions.. Fractional LP solutions poorly approimate integer solutions: For Boeing Aircraft Co., producing 4 versus 4.5 airplanes results in radically different profits. Often a mi is desired of integer and non-integer variables (MILP).

3 Graphical representation of IP Integer Programming (IP) What is it? Making decisions with IP Eclusion between choices Eclusion between constraints Solutions through branch and bound Characteristics Solving Binary IPs Solving Mied IPs and LPs

4 Integer Programming for Decision Making Yes or no decisions encoded with binary variables: if decision is yes j 0 if decision is no. Binary Integer Programming (BIP): Binary variables + linear constraints. Binary Integer Programming Eample: Cal Aircraft Manufacturing Company Problem:. Cal wants to epand: Build new factory in Los Angeles, San Francisco, or both. Build new warehouse (only one). Warehouse must be built close to city of a new factory.. Available capital: $0,000, Cal wants to maimize total net present value (profitability vs. time value of money) NPV Price Build a factory in L.A.? $9m $6m Build a factory in S.F.? $5m $3m 3 Build a warehouse in L.A.? $6m $5m 4 Build a warehouse in S.F.? $4m $m

5 Binary Integer Programming Eample: Cal Aircraft Manufacturing Company What decisions are to be made?. Build factory in LA. Build factory in SFO 3. Build warehouse in LA 4. Build warehouse in SFO Introduce 4 binary variables i = if decision i is yes 0 if decision i is no Binary Integer Programming Eample: Cal Aircraft Manufacturing Company. Cal wants to epand (TBD). Available capital: $0,000, Cal wants to maimize total net present value (profitability vs. time value of money) NPV Price Build a factory in L.A.? $9m $6m Build a factory in S.F.? $5m $3m 3 Build a warehouse in L.A.? $6m $5m 4 Build a warehouse in S.F.? $4m $m What is the objective? Maimize NPV: Z = Remaining constraints? Don t go beyond means: <0

6 Binary Integer Programming Eample: Cal Aircraft Manufacturing Company LA factory( ), SFO factory( ), LA warehouse( 3 ),SFO warehouse ( 4 ) Build new factory in Los Angeles, San Francisco, or both. Build new warehouse (only one). Warehouse must be built close to city of a new factory. What are the constraints between decisions?. Only one warehouse: 3 eclusive-or <. Warehouse in LA only if Factory is in LA: 3 implies 3 < 0 3. Warehouse in SFO only if Factory is in SFO: 4 implies 4 - < 0 Encoding Choices: An Eample Mutually eclusive choices Eample: at most decisions in a group can be yes: LP Encoding: + + k <.

7 Binary Integer Programming Eample: Cal Aircraft Manufacturing Company LA factory(), SFO factory(), LA warehouse(3),sfo warehouse (4) Build new factory in Los Angeles, San Francisco, or both. Build new warehouse (only one). Warehouse must be built close to city of a new factory. What are the constraints between decisions?. Only one warehouse: 3 eclusive-or <. Warehouse in LA only if Factory is in LA: 3 implies 3 < 0 3. Warehouse in SFO only if Factory is in SFO: 4 implies 4 - < 0 Binary Integer Programming Eample: Cal Aircraft Manufacturing Company Complete binary integer program: Maimize Z = < < 3 - < < 0 j < j > 0 j = {0,}, j=,,3,4

8 Integer Programming (IP) What is it? Making decisions with IP Eclusion between choices Eclusion between constraints Solutions through branch and bound Characteristics Solving Binary IPs Solving Mied IPs and LPs Cooperative Path Planning MILP Encoding: Constraints Min J T Receding Horizon Fuel Cost Fn s ij w ij, etc. State Space Constraints s i + = As i + Bu i State Evolution Equation i min + My i - i - ma + My i y i y min + My i3 Obstacle Avoidance -y i -y ma + My i4 y ik 3 Similar constraints for Collision Avoidance (for all pairs of vehicles)

9 Obstacles How do we encode? Each obstacle-vehicle pair represents a disjunctive constraint: Red Vehicle is above obstacle OR Red Vehicle is below obstacle OR Red Vehicle is left of obstacle OR Red Vehicle is right of obstacle Each disjunct is an inequality if R, yr are co-ordinates of red vehicle, inequalities above might be: R< 3 yr > 4, etc. Constraints are not limited to rectangular obstacles or obstacles oriented a particular way (inequalities might include both co-ordinates) Encoding Eclusion Constraints Mutually eclusive constraints Eample: Either [ 3 + < 8 (, real) ] Or [ + 4 < 6]. LP Encoding: Use Big M to turn-off constraint: Either: 3 + < 8 and + 4 < 6 + M (and M is very BIG) Or: 3 + < 8 + M and + 6 < 6 Use binary y to decide which constraint to turn off: 3 + < 8 + y M + < 6 + (-y)m y {0,}

10 Cooperative Path Planning MILP Encoding: Constraints Min J T Receding Horizon Fuel Cost Fn s ij w ij, etc. State Space Constraints s i + = As i + Bu i State Evolution Equation i min + My i - i - ma + My i y i y min + My i3 Obstacle Avoidance -y i -y ma + My i4 y ik 3 At least one enabled Similar constraints for Collision Avoidance (for all pairs of vehicles) Encoding Eclusion Constraints K out of N constraints hold: f (,, n ) < d OR : f N (,,, n ) < d N where f i are linear epressions At most K of N hold: LP Encoding: Introduce y i to deselect each constraint: Constrain K of y i to select constraints: N i y i N K N i y i N K Use Big M to turn-off constraint: f(, ---, n ) < d + My : f N (, ---, n ) < d N + My N

11 Encoding Finite Domain Functions Function has one out of n possible values: a +... a n n = [d or d or d p ] LP Encoding: y i {0,} i=,, p ( y i ) = a +... a n n = ( d i y i ) Fied charge problem: Encoding Constraints f i ( j ) = k j + c j j if j >0 0 if j =0 Minimizing costs: Minimizing z=f ( ) f n ( n ) Yes or no decisions: should each of the activities be undertaken? Introduce auiliary variables: y,, y n = 0, y = if > 0 Z n i 0 if = 0 Which can be written as a linear constraint using big M: <ym c k y i i i i

12 Integer Programming (IP) What is it? Making decisions with IP Eclusion between choices Eclusion between constraints Solutions through Branch and Bound IP Characteristics Solving Binary IPs Solving Mied IPs and LPs Naïve approach: rounding A simple way to solve an IP might be to round the non-integer solution to the nearest feasible integer. In practice, try it! In theory, this idea makes no sense at all. Consider Maimize Z Subject to 7 4 0, 3 0, Optimal solution * is (3/7,0), integer

13 Naïve approach: rounding The rounded solution is *=(,0). But this is not even feasible! By enumeration, feasible are (0,0) (0,) (0,) (0,3) (,) (,0) The optimal integer solution * is (0,3) [value = 33] which is nowhere near (3/7,0). In general, just finding a nearby feasible point is computationally challenging Solving Integer Programs: Characteristics Fewer feasible solutions than LPs. Worst-case eponential in # of variables. Commercial software: Cple

14 Solving Integer Programs Branch and Bound Binary Integer Programs Integer Programs Mied Integer (Real) Programs Cutting Planes Branch and Bound Problem: Optimize f() subject to A() 0, D B & B - an instance of Divide & Conquer: I. Bound D s solution and compare to alternatives. ) Bound solution to D quickly. II. Perform quick check by relaing hard part of problem. Rela integer constraints. Relaation is LP. ) Use bound to fathom D if possible. If relaed solution is integer, Then keep soln if best found to date ( incumbent ), delete D i If relaed solution is worse than incumbent, Then delete D i. If no feasible solution, Then delete D i. Otherwise Branch to smaller subproblems ) Partition D into subproblems D D n ) Apply B&B to all subproblems.

15 B&B for Binary Integer Programs (BIPs) Problem: Optimize f() st A() 0, i {0,}, D Domain D i encoding: partial assignment to, { =, = 0, } Branch Step:. Find variable j that is unassigned in D i. Create two subproblems by splitting D i : D i D i { j D i0 D i { j Eample: B&B for BIPs Ma Z = i, i 0, i integer Queue: Incumbent: none Best cost Z*: - inf Initialize

16 Eample: B&B for BIPs Ma Z = i, i 0, i integer Queue: Incumbent: none Best cost Z*: - inf Dequeue Eample: B&B for BIPs Ma Z = i, i 0, i integer Z = 6.5, = <0.8333,,0,> Queue: Incumbent: none Best cost Z*: - inf Bound Constrain i by Rela to LP Solve

17 Eample: B&B for BIPs Ma Z = i, i 0, i integer Z = 6.5, = <0.8333,,0,> Queue: Incumbent: none Best cost Z*: - inf Try to fathom: infeasible? worse than incumbent? integer solution? Eample: B&B for BIPs { { Ma Z = i, i 0, i integer Z = 6.5, = <0.8333,,0,> Queue: { { Incumbent: none Best cost Z*: - inf Branch: select unassigned i pick non-integer Split on i

18 Eample: B&B for BIPs { { Ma Z = i, i 0, i integer Z = 6.5, = <0.8333,,0,> Queue: { { Incumbent: none Best cost Z*: - inf dequeue: depth first or best first Eample: B&B for BIPs { { Ma Z = i, i 0, i integer Z = 6.5, = <0.8333,,0,> Queue: { Incumbent: none Best cost Z*: - inf Bound { constrain by { rela to LP solve

19 Eample: B&B for BIPs { { Ma Z = i, i 0, i integer Z = 6.5, = <0.8333,,0,> Queue: { Incumbent: none Best cost Z*: - inf Bound { constrain by { rela to LP solve Eample: B&B for BIPs { { Ma Z = i, i 0, i integer Z = 9,5, = <0,,0,> Queue: { Incumbent: none Best cost Z*: - inf Bound { constrain by { rela to LP solve

20 Eample: B&B for BIPs { { Ma Z = i, i 0, i integer Z = 9,5, = <0,,0,> Queue: { Incumbent: none Best cost Z*: - inf Try to fathom: infeasible? worse than incumbent? integer solution? Eample: B&B for BIPs { { Ma Z = i, i 0, i integer Z = 9,5, = <0,,0,> Queue: { Incumbent: = <0,,0,> Best cost Z*: 9 Try to fathom: infeasible? worse than incumbent? integer solution?

21 Eample: B&B for BIPs { { Ma Z = i, i 0, i integer Z = 9,5, = <0,,0,> Queue: { Incumbent: = <0,,0,> Best cost Z*: 9 Dequeue and bound Eample: B&B for BIPs { { Ma Z = i, i 0, i integer Z = 6,, = <,.8,0,.8> Queue: Incumbent: = <0,,0,> Best cost Z*: 9 Dequeue and bound

22 Eample: B&B for BIPs { { Ma Z = i, i 0, i integer Z = 6., = <,.8,0,.8> Queue: { Incumbent: = <0,,0,> Best cost Z*: 9 Try to fathom: infeasible? worse than incumbent? integer solution? Eample: B&B for BIPs { { { { Ma Z = i, i 0, i integer Z = 6., = <,.8,0,.8> Queue: { { Incumbent: = <0,,0,> Best cost Z*: 9 Branch Dequeue

23 Eample: B&B for BIPs { { { { Ma Z = i, i 0, i integer Z = 6., = <,.8,0,.8> Queue: { Incumbent: = <0,,0,> Best cost Z*: 9 Bound Eample: B&B for BIPs { { { { Ma Z = i, i 0, i integer Z = 6, = <,,0,.5> Queue: { Incumbent: = <0,,0,> Best cost Z*: 9 Bound Try to fathom: infeasible? worse than incumbent? integer solution?

24 Eample: B&B for BIPs { { { { { 3 { 3 Ma Z = i, i 0, i integer Z = 6, = <,,0,.5> Queue: { 3 } { 3 { Incumbent: = <0,,0,> Best cost Z*: 9 Branch Eample: B&B for BIPs { { { { { 3 { 3 Ma Z = i, i 0, i integer Z = 6, = <,,0,.5> Queue: { 3 { 3 { Incumbent: = <0,,0,> Best cost Z*: 9 Bound

25 Eample: B&B for BIPs { { { { { 3 { 3 Ma Z = i, i 0, i integer Z = 6, = <,,0,.5> Queue: { 3 { Incumbent: = <0,,0,> Best cost Z*: 9 Bound Eample: B&B for BIPs { { { { { 3 { 3 Ma Z = i, i 0, i integer Z = 6, = <,,0,.5> Queue: { 3 { Incumbent: = <0,,0,> Best cost Z*: 9 Bound Try to fathom: infeasible?

26 Eample: B&B for BIPs { { { { { 3 { 3 Ma Z = i, i 0, i integer Z = 6, = <,,0,.5> Queue: { Incumbent: = <0,,0,> Best cost Z*: 9 Dequeue & bound Eample: B&B for BIPs { { { { { 3 { 3 Ma Z = i, i 0, i integer Z = 6, = <,,0,.5> Queue: { Incumbent: = <0,,0,> Best cost Z*: 9 dequeue Try to fathom: & bound infeasible? worse than incumbent? integer solution?

27 Eample: B&B for BIPs { { { 3 { { 4 { 3 { { 4 Queue: { 4 { 4 { Incumbent: = <0,,0,> Best cost Z*: 9 Ma Z = i, i 0, i integer Z = 4, = <,,0,0> Try dequeue to fathom: & bound infeasible? worse than incumbent? integer solution? Eample: B&B for BIPs { { { { { 3 { 3 { 4 { 4 Queue: { 4 { Incumbent: = <0,,0,> <,,0,0> Best cost Z*: 94 Ma Z = i, i 0, i integer Z = 4, = <,,0,0> dequeue Try to fathom: & bound infeasible? worse than incumbent? integer solution?

28 Eample: B&B for BIPs { { { { { 3 { 3 { 4 { 4 Queue: { 4 { Incumbent: = <0,,0,> <,,0,0> Best cost Z*: 94 Ma Z = i, i 0, i integer Z = 8, = <,,0,> dequeue Try to fathom: & bound infeasible? worse than incumbent? integer solution? Eample: B&B for BIPs { { { { { 3 { 3 { 4 { 4 Queue: { Incumbent: = <0,,0,> <,,0,0> Best cost Z*: 94 Ma Z = i, i 0, i integer Z = 3.8, = <,0,.8,0> dequeue Try to fathom: & bound infeasible? worse than incumbent? integer solution?

29 Integer Programming (IP) What is it? Making decisions with IP Eclusion between choices Eclusion between constraints Solutions through branch and bound Characteristics Solving Binary IPs Solving Mied IPs and LPs { { } Eample: B&B for MIPs { } { } { Incumbent: = <0,0,,.5> Best cost Z*: 3.5 { } Ma Z = i 0, i integer,, 3, Z = 4.5, = <.5,.5,.75,0> Z = 4., = <,.,.8,0> Z = 4 /6, = <5/6,,/6,0> Z = 3.5, = <0,0,,.5> Infeasible, = <,,?,?> Z = /6, = <5/6,,/6,0> Infeasible, = <,?,?,?>

30 Cutting planes methods Cutting planes are an alternative to Branch and Bound search, though the methods do have similarities Gomory invented the first finitely terminating cutting plane method in 958. Cutting planes cut off non-integer solutions

31 Cutting planes algorithm Solve the linear programming relaation of the integer program. If *, the optimal solution, is integer, then * is optimal for the integer program. If not, add a linear inequality constraint (cutting plane) that all integer solutions to the original problem satisfy but that is violated by *. Repeat all steps. Gomory cutting planes Given a problem with all variables integer constrained, solve the relaed LP and eamine the final simple tableau. Take one equality with a fractional right-handside b i. This is an equality that forces an integer constrained variable to be fractional. i a ij j a ik Non-basic variables and coefficients k b i

32 Gomory cutting planes Since at this tableau all of j through k are 0, and in all feasible solutions they are >0, it must be that So add the cutting plane constraint that * does not satisfy but all integer solns do: i k ik j ij i i k ik j ij i k ik j ij i b a a b a a a a Eample: Gomory cutting plane integer, 0, 0, Subject to Minimize Z integer,,, 0, 0, 0, 0, Subject to Minimize Z

33 Eample: Gomory continued Solve the LP relaation to find *=(3/,5/) One of the equations in the final simple tableau is: / Taking the floor of all non-basic variable coefficients and the right-hand-side, get and add this constraint to the LP Eample: Gomory continued Solve Minimize Z Subject to , 0, 0, 0 And find *=(3/4,). One of the equations in the optimal tableau is

34 Eample: Gomory continued Add the new Gomory cut: 3 5 Notice that in terms of the original variables, the new cut is: Add this constraint and solve the relaed LP Minimize Z Subject to , 3 0, , , Eample: Gomory continued The new *=(,), which is integer. Why is this * definitely the optimal solution to the original integer program? *=(,) is integer so it is feasible for the integer program No better integer-valued eisted at the start, because none of the Gomory cuts can eliminate any integer solution, regardless of its value.

35 Problems with cutting planes Each cutting plane usually eliminates a very small part of the feasible region. Although finite, no cutting plane method is polynomial-time. The more one knows about the specific structure of the problem, the better one can do with cutting planes. Specialized cutting plane algorithms sometimes work remarkably well, and CPLEX allows you to turn specific types on and off.