If the above problem is hard to solve, we might try to break it into smaller problems which are easier to solve.

Transcription

1 8 Branch and Bound 8.1 Divide and Conquer uppose that we would like to solve the problem { z = max c T x : x. If the above problem is hard to solve, we might try to break it into smaller problems which are easier to solve. Observation 8.1 If = 1 k and z i = max { c T x : x i for i = 1,...,k then z = max { z i : i = 1,...,k. Although Observation 8.1 is (almost) trivial, it is the key to branch and bound methods which turn out to be effective for many integer programming problems. The recursive decomposition of into smaller problems can be represented by an enumeration tree. upposethat B 3.Wefirstdivide into 0 and 1,where 0 = {x : x 1 = 0 1 = {x : x 1 = 1 The sets 0 and 1 will be recursively divided into smaller sets as shown in the enumeration tree in Figure 8.1. As a second example consider the tours of a TP on four cities. Let denote the set of all tours. We first divide into three pieces, 12, 13 and 14,where 1i is the set of all tours which from city 1 go directly to city i. et 1i in turn is divided into two pieces 1i,ij, j 1,i, where 1i,ij is the set of all tours in i1 which leave city i for city j. The recursive division is depicted in the enumeration tree in Figure 8.2. The leaves of the tree correspond to the (4 1)! = 3! = 6possiblepermutationsof{1,...,4 which have 1 at the first place. 8.2 Pruning Enumeration Trees It is clear that a complete enumeration soon becomes hopeless evenforproblemsof medium size. For instance, in the TP the complete enumeration tree would have (n 1)!

2 106 Branch and Bound x 1 = 0 x 1 = x 2 = 0 x 2 = 1 x 2 = 0 x 2 = x 3 = 0 x 3 = 1 x 3 = 0 x 3 = 1 x 3 = 0 x 3 = 1 x 3 = 0 x 3 = Figure 8.1: Enumeration tree for a set B ,23 12,24 13,32 13,34 14,42 14,43 Figure 8.2: Enumeration tree for the TP on four cities.

3 8.2 Pruning Enumeration Trees 107 leaves and thus for n = 60 would be larger than the estimated number of atoms in the universe 1. The key to efficient algorithms is to cut off useless parts of anenumerationtree.thisis where bounds (cf. Chapter 6) come in. The overall scheme obtained is also referred to as implicit enumeration, sincemostofthetimenotalloftheenumerationtreeiscompletely unfolded. Observation 8.2 Let = 1 k and z i = max { c T x : x i for i = 1,...,k then z = max { z i : i = 1,...,k.upposethatwearegivenz i and z i for i = 1,...,k with z i z i z i for i = 1,...,k. Then for z = max { c T x : x it is true that: { { max z i : i = 1,...,k z max z i : i = 1,...,k The above observation enables us to deduce rules how to prune the enumeration tree. Observation 8.3 (Pruning by optimality) If z i = z i for some i,thentheoptimalsolution value for the ith subproblem z i = max { c T x : x i is known and there is no need to subdivide i into smaller pieces. As an example consider the situation depicted in Figure 8.3. The set is divided into 1 and 2 with the upper and lower bounds as shown. z 1 = 20 z 1 = 20 z = 30 z = z 2 = 15 Figure 8.3: Pruning by optimality. z = 25 z = z 2 = 15 ince z 1 = z 1,thereisnoneedtofurtherexplorethebranchrootedat 1.o,thisbranch can be pruned by optimality. Moreover, Observation 8.2 allows us to get new lower and upper bounds for z as shown in the figure. Observation 8.4 (Pruning by bound) If z i < z j for some i j, thentheoptimalsolution for the ith subproblem z i = max { c T x : x i can never be an optimal solution of the whole problem. Thus, there is no need to subdivide i into smaller pieces. Consider the example in Figure 8.4. ince z 1 = 20 < 21 = z 2,wecanconcludethatthe branch rooted at 1 does not contain an optimal solution: any solution in this branch has objective function value at most 20, whereas the branch rooted at 2 contains solutions of value at least 21. Again, we can stop to explore the branch rooted at 1. We list one more generic reason which makes the exploration ofabranchunnecessary. Observation 8.5 (Pruning by infeasibility) If i =, thenthereisnoneedtosubdivide i into smaller pieces, either. The next section will make clear how the case of infeasibility can occur in a branch and bound system. We close this section by showing an example where no pruning is possible.

4 108 Branch and Bound z = 30 z = 10 z 1 = z 1 = 15 z 2 = 21 Figure 8.4: Pruning by bound. z = 25 z = z 2 = 21 z = 30 z = 25 z = 10 z = 17 z 1 = z 1 = z 1 = 15 z 2 = 17 z 1 = 15 z 2 = 17 Figure 8.5: No pruning possible. Consider the partial enumeration tree in Figure 8.5, where again we have divided the set into the sets 1 and 2 with the bounds as shown. Although the lower and upper bounds for the subproblems allow us to get better bounds for the whole problem, we still have to explore both subtrees. 8.3 LP-Based Branch and Bound: An Example The most common way to solve integer programs is to use an implicit enumeration scheme based on Linear Programming. Upper bounds are provided by LP-relaxations and branching is done by adding new constraints. We illustrate this procedure for a small example. Figure 8.6 shows the evolution of the corresponding branch-and-bound-tree. Consider the integer program z = max4x 1 x 2 3x 1 2x 2 + x 3 = 14 x 2 + x 4 = 3 2x 1 2x 2 + x 5 = 3 x Z 5 + (8.1a) (8.1b) (8.1c) (8.1d) (8.1e) Bounding The first step is to get upper and lower bounds for the optimal value z. An upper bound is obtained by solving the LP-relaxation of (8.1). This LP has the optimal solution x =(4.5,3,6.5,0,0). Hence,wegettheupperbound z = 15. ince we do not have any feasible solution yet, by convention we use z = as a lower bound. Branching In the current situation (see Figure 8.6(a)) we have z < z, sowehaveto branch, that is, we have to split the feasible region. Acommonwaytoachievethisisto take an integer variable x j that is basic but not integral and set: 1 = { x : x j x j 2 = { x : x j x j 1 The estimated number of atoms in the universe is

5 8.3 LP-Based Branch and Bound: An Example 109 z = 15 z = z = 15 z = (a) Initial tree 1 2 Pruned by infeasibility (b) Node 2 is pruned by infeasibility. z = 13.5 z = z = 13.5 z = z 1 = Pruned by infeasibility z 1 = (c) After the LP-relaxation of 2 is solved, we get a new upper bound. z 1 = 13.5 z 1 = x x Pruned by infeasibility z = 13.5 z = 13 (d) Branching on variable x 2 leads to two new subproblems. z = 13.5 z = 13 z 1 = 13.5 z 1 = Pruned by infeasibility z 1 = 13.5 z 1 = Pruned by infeasibility x 2 2 x 2 3 x 2 2 x z 12 = 13 Pruned by optimality z 12 = 13 (e) Problem 12 has an integral optimal solution, so we can prune it by optimality. This also gives the first finite lower bound. z 11 = 12 Pruned by bound z 11 = z 12 = 13 Pruned by optimality z 12 = 13 (f) Problem 11 can be pruned by bound, since z 11 = 12 < 13 = z. Figure 8.6: Evolution of the branch-and-bound-tree for the example. The active nodes are shown in white.

6 110 Branch and Bound Clearly, = 1 2 and 1 2 =. Observethatduetotheabovechoiceofbranching, the current optimal basic solution x is not feasible for either subproblem. If there is no degeneracy (i.e., multiple optimal LP solutions), then we get max { z 1, z 2 < z, whichmeans that our upper bound decreases (improves). In the concrete example we choose variable x 1 where x 1 = 4.5 and set 1 = {x : and 2 = {x :. Choosing an active node The list of active problems (nodes) to be examined is now 1, 2. uppose we choose 2 to be explored. Again, the first step is to obtain an upper bound by solving the LP-relaxation: z 2 = max4x 1 x 2 3x 1 2x 2 + x 3 = 14 x 2 + x 4 = 3 2x 1 2x 2 + x 5 = 3 x 0 (8.2a) (8.2b) (8.2c) (8.2d) (8.2e) (8.2f) It turns out that (8.2) is infeasible, so 2 can be pruned by infeasibility (see Figure 8.6(b)). Choosing an active node The only active node at the moment is 1.o,wechoose 1 to be explored. We obtain an upper bound z 1 by solving the LP-relaxation z 1 = max4x 1 x 2 3x 1 2x 2 + x 3 = 14 x 2 + x 4 = 3 2x 1 2x 2 + x 5 = 3 x 0. (8.3a) (8.3b) (8.3c) (8.3d) (8.3e) (8.3f) An optimal solution for (8.3) is x 2 =(4,2.5,7,4,0) with objective function value This allows us to update our bounds as shown in Figure 8.6(c). This time we choose to branch on variable x 2,since x 2 2 = 2.5. The two subproblems are defined on the sets 11 = 1 {x : x 2 2 and 12 = 1 {x : x 2 3, seefigure8.6(d). Choosing an active node solve the LP-relaxation The list of active nodes is 11 and 12. We choose 12 and z 12 = max4x 1 x 2 3x 1 2x 2 + x 3 = 14 x 2 + x 4 = 3 2x 1 2x 2 + x 5 = 3 x 2 3 x 0. (8.4a) (8.4b) (8.4c) (8.4d) (8.4e) (8.4f) (8.4g) An optimal solution of (8.4) is x 12 =(4,3,8,1,0) with objective function value 13. As the solution is integer, we can prune 12 by optimality. Moreover, we can also update our bounds as shown in Figure 8.6(e).

7 8.4 Techniques for LP-based Branch and Bound 111 At the moment we only have one active node: 11.Thecorre- Choosing an active node sponding LP-relaxation is z 11 = max4x 1 x 2 3x 1 2x 2 + x 3 = 14 x 2 + x 4 = 3 2x 1 2x 2 + x 5 = 3 x 2 2 x 0, (8.5a) (8.5b) (8.5c) (8.5d) (8.5e) (8.5f) (8.5g) which has x 11 =(3.5,2,7.5,1,0) as an optimal solution. Hence, z 11 = 12 which is strictly smaller than z = 13. This means that 11 can be pruned by bound. 8.4 Techniques for LP-based Branch and Bound The previous section contained an example for an execution of an LP-based branch and bound algorithm. While the basic outline of such an algorithm should be clear, there are a few issues that need to be taken into account in order to obtainthebestpossibleefficiency Reoptimization Consider the situation in the example from the previous section when we wanted to explore node 1.WehadalreadysolvedtheinitialLPfor. TheonlydifferencebetweenLP() and LP( 1 ) is the presence of the constraint inlp( 1 ).HowcanwesolveLP( 1 ) without starting from scratch? The problem LP() is of the form max { c T x : Ax = b,x 0. An optimal basis for this LP is B = {x 1,x 2,x 3 with solution x B =(4.5,3,6.5) and x N =(0,0). Wecanparametrize any solution x =(x B,x N ) for LP() by the nonbasic variables: x B := A 1 B b A 1 B A Nx N = x B A 1 B A Nx N Let Ā N := A 1 B A N, c T N := ct B A 1 B A N c T N and b := A 1 B b.multiplyingtheconstraints Ax = b by A 1 B gives the optimal basis representation of the LP: maxc T B x B ct N x N x B + Ā N x N = b x 0 (8.6a) (8.6b) (8.6c) Recall that a basis B is called dual feasible, if c N 0. The optimal primal basis is also dual feasible. We refer to [ch86, NW99] for details on Linear Programming. In the branch and bound scheme, we add a new constraint x i t (or x i t) forsome basic variable x i B. Letusmakethisconstraintanequalityconstraintbyaddingaslack variable s 0. The new constraints are x i + s = t, s 0. By (8.6) we can express x i in terms of the nonbasic variables: x i =( b Ā N x N ) i.hencex i + s = t becomes ( Ā N x N ) i + s = t b i.

8 112 Branch and Bound Adding this constraint to (8.6) gives us the following new LP that we must solve: maxc T B x B ct N x N x B + Ā N x N = b ( Ā N x N ) i + s = t b i x 0,s 0 (8.7a) (8.7b) (8.7c) (8.7d) The set B {s forms a dual feasible basis for (8.7), so we can start to solve (8.7) by the dual implex method starting with the situation as given. Let us illustrate the method for the concrete sitation of our example. We have A B = /2 A 1 B = /2 and thus (8.6) amounts to max15 3x 4 2x 5 x 1 +x x 5 = 9 2 x 2 +x 4 = 3 x 3 x x 5 = 13 2 x 1,x 2,x 3,x 4,x 5 0 If we add a slack variable s, thenthenewconstraintbecomesx 1 + s = 4, s 0. ince x 1 = 9 2 x x 5,wecanrewritethisconstraintintermsofthenonbasicvariables as: This gives us the dual feasible representation of LP( 1 ): x x 5 + s = 1 2. (8.8) max15 3x 4 2x 5 x 1 +x x 9 5 = 2 x 2 +x 4 = 3 x 3 x x 5 = 13 2 x x 5 +s = 2 1 x 1,x 2,x 3,x 4,x 5,s 0 We can now do dual implex pivots to find the optimal solution oflp( 1 ) Other Issues toring the tree In practice we do not need to store the whole branch and bound tree. It suffices to store the active nodes. Bounding Upper bounds are obtained by means of the LP-relaxations. Cutting-planes (see the following chapter) can be used to strengthen bounds. Lowerboundscanbe obtained by means of heuristics and approximation algorithms.

9 8.4 Techniques for LP-based Branch and Bound 113 Branching In our example we branched on a variable that was fractional in theoptimal solution for the LP-relaxation. In general, there will be more than one such variable. Achoicethathasprovedtobeefficientinpracticeistobranchonthemost fractional variable,thatis,tochooseavariablewhichmaximizesmax j min { f j,1 f j,where f j = x j x j. There are other rules based on estimating the cost of a variable to become integer. We refer to [NW99] for more details. Choosing an active node In our example we just chose an arbitrary active node to be explored. In practice there are two antagonistic arguments how to choose an active node: The tree can only be pruned significantly if there is a good primal feasible solution available which gives a good lower bound. This suggests to apply a depth-first search strategy. Doing so, in each step a single new constraint is added and we can reoptimize easily as shown in ection On the other hand, one would like to minimize the total number of nodes evaluated in the tree. This suggests to choose an active node with the best upper bound. uch a strategy is known as best-first search. In practice, one usually employs a mixture of depth-first search and best-first search. After an initial depth-first search which leads to a feasible solution one switches to a combined best-first and depth-first strategy. Preprocessing An important technique for obtaining efficient algorithms istousepreprocessing which includes tightening of bounds (e.g. by cutting-planes, see the following chapter) removing of redundant variables variable fixing We demonstrate preprocessing techniques for a small example. Consider the LP max 2x 1 + x 2 x 3 5x 1 2x 2 + 8x x 1 + 3x 2 x 3 9 x 1 + x 2 + x x x x 3 Tightening of constraints uppose we take the first constraint and isolate variable x 1.Thenweget 5x x 2 8x = 9, where we have used the bounds on the variables x 2 and x 3.Thisgivesusthebound x

10 114 Branch and Bound imilarly, taking the first constraint and isolating variable x 3 results in: 8x x 2 5x = 17. Our new bound for x 3 is: x By similar operations we get the new bound x and now using all the new bounds for x 3 in the first inequality gives: x Plugging the new bounds into the third constraint gives: x 1 + x 2 + x < 6. o, the third constraint is superfluous and can be dropped. Our LP has reduced to the following problem: Variable Fixing max 2x 1 + x 2 x 3 5x 1 2x 2 + 8x x 1 + 3x 2 x x x x Consider variable x 2.Increasingvariablex 2 makes all constraints less tight. ince x 2 has apositivecoefficientintheobjectivefunctionitwillbeaslarge as possible in an optimal solution, that is, it will be equal to its upper bound of 1: x 2 = 1. The same conclusion could be obtained by considering the LP dual. One can see that the dual variable corresponding to the constraint x 2 1mustbepositiveinordertoachieve feasibility. By complementary slackness this means that x 2 = 1inanyoptimalsolution. Decreasing the value of x 3 makes all constraints less tight, too. The coefficient of x 3 in the objective is negative, so x 3 can be set to its lower bound. After all the preprocessing, our initial problem has reduced tothefollowingsimplelp: max2x x We formalize the ideas from our example above:

11 8.4 Techniques for LP-based Branch and Bound 115 Observation 8.6 Consider the set n = x : a 0x 0 + a j x j b,l j x j u j, for j = 1,...,n. j=1 The following statements hold: Bounds on variables If a 0 > 0,then x 0 b a j l j a j u j /a 0. j:a j >0 j:a j <0 and, if a 0 < 0, then x 0 b a j l j a j u j /a 0. j:a j >0 j:a j <0 Redundancy The constraint a 0 x 0 + n j=1 a jx j b is redundant, if j:a j >0 Infeasibility The set is empty, if j:a j >0 a j u j + j:a j <0 a j l j + j:a j <0 a j l j b. a j u j >b. Variable Fixing For a maximization problem of the form max { c T x : Ax b,l x u, if a ij 0 for all i = 1,...,m and c j < 0, thenx j = l j in an optimal solution. Conversely, if a ij 0 for all i = 1,...,m and c j > 0,thenx j = u j in an optimal solution. For integer programming problems, the preprocessing can go further. If x j Z and the bounds l j x j u j are not integer, then we can tighten them to l j x j u j. We will explore this fact in greater depth in the following chapter on cutting-planes.