3.5 Branch and Bound Consider a generic Discrete Optimization problem (P) z = max{c(x) : x X }, where X is the feasible region, i.e., the set of the feasible solutions. Branch and Bound is a general semi-enumerative approach (Land and Doig 1960) to explore the feasible region X. By exploiting bounds on the optimal objective function value - it avoids explicitly exploring certain parts of the feasible region X, - it is guaranteed to find an optimal solution. Two main components: Divide and conquer strategy (branching) Implicit enumeration exploiting bounds on the optimal objective function value (bounding) Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 1 / 11
1) Divide and conquer strategy Idea: Partition in a recursive way the feasible region X so as to reduce the solution of (P) to the solution of a sequence of smaller and easier subproblems. Observation: Let X = X 1... X k be a partition of X in k subsets (X i X j = for each pair of indices i j) and z i = max{c(x) : x X i } for 1 i k. Then obviously z = max 1 i k z i. Recursive partition of the feasible region branching operation The procedure can be represented by a enumeration tree whose root node is associated to X and the other nodes to the subsets X i. Examples: - X {0, 1} 3 binary branching - X set of all the Hamiltonian circuits of a given digraph G = (V, A) multiway branching Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 2 / 11
2) Implicit enumeration Explicit enumeration is too heavy computationally even for small instances, it is not enough to recursively subdivide the feasible region. Idea: Exploit upper and lower bounds (primal and dual bounds) on z i, with 1 i k, in order to avoid to explicitly explore some parts of the feasible region X. Observation: Let X = X 1... X k be a partition of X and z i = max{c(x) : x X i } for 1 i k. Moreover, let l i be a lower bound and u i an upper bound on z i, namely l i z i u i. Then l = max 1 i k l i is a lower bound and u = max 1 i k u i is an upper bound on z, that is l z u. Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 3 / 11
Pruning criteria Cases in which the primal and dual bounds for the i-th subproblem can be exploited to avoid exploring (discard) the subregion X i (to prune the corresponding node of the B&B tree): Optimality criterion: If u i = l i, it is not necessary to further explore the subregion X i since we have found an optimal solution in X i of value z i = u i = l i. Bounding criterion: If the upper bound u i is lower than - the objective function value LB of the best solution x LB found so far or - any lower bound l j for j i, it is not necessary to explore the subregion X i because it cannot contain any better feasible solution. Feasibility criterion: X i = Four examples of subproblems (node) configurations, including one where the feasible regions of the subproblems must be further explored. If a subproblem is not solved, we proceed recursively and generate subproblems (branching step). Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 4 / 11
Main ingredients of Branch and Bound method (for max problem) Upper bounds: Efficient method to determine a good quality dual bound u on z. Branching rule: Procedure to (recursively) partition the feasible region X into smaller subregions. Lower bounds: Efficient heuristic to look for a feasible solution x with a value c( x), which provides a good lower bound c( x) on z. To be stored and updated: - a list L of active subproblems with lower and upper bounds on z i : l i z i u i, - a global upper bound UB on z, - a global lower bound LB on z provided by the best feasible solution x LB found so far. Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 5 / 11
General method, we just need to specify: 1 how to choose the next subproblem (active node) to be processed 2 how to generate the subproblems of a given subproblem (the children nodes) 3 how to efficiently compute the primal and dual bounds The performance of a Branch-and-Bound algorithm strongly depends on the efficiency of the branching rule and the quality of primal and dual bounds. N.B.: A Branch-and-Bound approach is applicable to MILP problems as well as to Nonlinear Optimization problems. Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 6 / 11
3.5.1 Branch and Bound for ILP problems Consider an ILP problem: z ILP = max{c t x : Ax = b, x 0 integer} (1) and suppose we look for an optimal solution x ILP. Solve the linear relaxation of (1) and let x LP be an optimal solution of value z LP. Obviously z ILP = c t x ILP z LP = c t x LP. If x LP is integer, then it is also optimal for (1). Otherwise x LP component. has at least one fractional Branching If x LP is not integer, choose a fractional component x h z 1 ILP = max{c t x : Ax = b, x h x h, x 0 integer} and generate the two suproblems: z 2 ILP = max{c t x : Ax = b, x h x h + 1, x 0 integer} with the corresponding subregions X 1 and X 2 of X, which are exhaustive and mutually exclusive. Clearly z ILP = max{z 1 ILP, z 2 ILP}. Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 7 / 11
Recursive process: solve the linear relaxation of each subproblem of the ILP and, if necessary, carry out a branching step. Bounding Consider the i-th subproblem with feasible subregion X i. Solve its linear relaxation, let x LP be the optimal solution and z i LP its value. Clearly, if all the coefficients c i are integer, every feasible solution of the ILP in X i has value z i LP. In Branch and Bound, branching and bounding operations are alternated, while storing and updating the best feasible solution found. We need to decide: 1 Criterion to select the next subproblem (node) to explore. 2 How to generate the children nodes for the node under consideration (choice of the branching variable). 3 Heuristic to determine the lower bounds on the optimal objective function value. Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 8 / 11
1. Choice of the subproblem (node) to be processed Depth first search strategy ( deepest node first): easy to implement but costly if wrong choice. Best bound first strategy (most promising node first): tend to generate less nodes but the subproblems are less constrained (we rarely update the best solution found so far). 2. Choice of the fractional variable for branching Branching first on a fractional variable whose fractional part is closest to 0.5 (in an attempt to generate two subproblems that are equally constrained) is often not the best choice. Strong branching ( estimate the bound improvement if branching on several candidate fractional variables, and branch w.r.t. the best one) is costly but effective for some hard instances. Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 9 / 11
Exponential example for Branch and Bound: Let n be an odd positive integer and consider the ILP problem: max s.t. x n x 0 + 2 n j=1 x j = n 0 x j 1 j {0, 1, 2,..., n} x j Z + j {0, 1, 2,..., n}. It can be verified that, when Branch and Bound is applied to this ILP instance, at least 2 n 1 2 ILP subproblems are inserted in the list L. Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 10 / 11
Example 1: Find an optimal solution of the ILP problem max 4x 1 x 2 s.t. 4x 1 + 2x 2 19 10x 1 4x 2 25 x 2 9 2 x 1, x 2 Z + with the Branch and Bound method by solving graphically the linear relaxation of the subproblems. Branch first with respect to x 1. Example 2: Solve the binary knapsack problem max 10x 1 + 12x 2 + 5x 3 + 7x 4 + 9x 5 s.t. 5x 1 + 8x 2 + 6x 3 + 2x 4 + 7x 5 14 x 1,..., x 5 {0, 1} with the Branch and Bound method. Use a simple greedy heuristic to determine the optimal solutions of the linear relaxations. Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 11 / 11