A Branch-and-Bound Algorithm for Unconstrained Global Optimization

Size: px

Start display at page:

Download "A Branch-and-Bound Algorithm for Unconstrained Global Optimization"

Barbara Cobb
5 years ago
Views:

1 SCAN 2010, Lyon, September 27 30, / 18 A Branch-and-Bound Algorithm for Unconstrained Global Optimization Laurent Granvilliers and Alexandre Goldsztejn Université de Nantes LINA CNRS Interval-based BB framework New hybrid algorithm Experiments using Realpaver

2 SCAN 2010, Lyon, September 27 30, / 18 Unconstrained Global Optimization We consider a class C 2 function f : R n R to be minimized a box B R n (bound constraints, search space) We define a global minimizer as an element x B such that x B : f(x ) f(x) the global minimum f = f(x ) for every global minimizer x Our goal is to compute within given tolerances a set of boxes S enclosing the set of all global minimizers an interval [l, u] enclosing f

3 SCAN 2010, Lyon, September 27 30, / 18 Branch-and-Bound Basic principles the initial box B is recursively split into sub-boxes, forming a tree, until given tolerances are reached the best upper bound u of f is maintained the lower bound of f is calculated on every sub-box; this box can be rejected if this lower bound is strictly greater than u Complexity: worst case exponential in the depth of the tree An efficient algorithm requires many other techniques constraint-based reduction methods graph decomposition techniques use of optimality conditions local optimization adaptive splitting techniques

4 SCAN 2010, Lyon, September 27 30, / 18 Interval Enclosures Definition An interval extension of f is an interval function f such that x B : {f(x) : x x} f(x).

5 SCAN 2010, Lyon, September 27 30, / 18 Interval Enclosures Definition An interval extension of f is an interval function f such that x B : {f(x) : x x} f(x). Consider a subdivision {x k } k K of B. f(x k ) We use: natural form mean value form x k

6 Interval Enclosures Definition An interval extension of f is an interval function f such that x B : {f(x) : x x} f(x). Consider a subdivision {x k } k K of B. We have f [l,u] such that: u l min k K f(xk ) u min k K f(xk ) l SCAN 2010, Lyon, September 27 30, / 18 f

7 Interval Enclosures Definition An interval extension of f is an interval function f such that x B : {f(x) : x x} f(x). Consider a subdivision {x k } k K of B. For all x B, f(x) is an upper bound of f. x Improving u leads to reject sub-boxes (basic principle of BB). u f SCAN 2010, Lyon, September 27 30, / 18 l

8 SCAN 2010, Lyon, September 27 30, / 18 Algorithm 1: S % output boxes 2: L { B,f(B) } % boxes to be considered 3: u + % upper bound 4: 5: while L is not empty do 6: x,f(x) pop (L) % box x s.t. f(x) is minimum 7: a Minimize (f, x) % minimization of f within x 8: u min(u,f(a)) % update the upper bound 9: x Contract (x, B, f, u) % use constraints to contract x 10: Branch (x,ε, L, S) % split x if it is large enough 11: update L and S according to u 12: end while 13: 14: l min{f(x) : x,f(x) S} 15: return S,[l,u]

9 SCAN 2010, Lyon, September 27 30, / 18 Branching Step Branch (x,ε, L, S) 1: if x is not empty then 2: if wid(x) ε then 3: S S { x,f(x) } 4: else 5: {x k } k K Split(x,ε) 6: L L { x k,f(x k ) : k K} 7: end if 8: end if Splitting heuristics round robin, largest domain, smear function bisection, trisection

10 SCAN 2010, Lyon, September 27 30, / 18 Minimization Step The goal is to find a minimizer of f within the given box x in order to update the upper bound of f Some techniques local optimization grid-based exploration metaheuristics hybrid methods Our algorithm line search based Newton s method: x 0, x 1, x 2,... such that x k+1 x k α k H(x k ) 1 f(x k ), k 0 greedy strategy with a grid of n points (linear) stopping when the upper bound is not improved enough hybrid algorithm

11 SCAN 2010, Lyon, September 27 30, / 18 Contraction Step Constraints that must be verified by every global minimizer x i. f(x) u ii. x B = f(x) = 0

12 SCAN 2010, Lyon, September 27 30, / 18 Contraction Step Constraints that must be verified by every global minimizer x i. f(x) u ii. x B = f(x) = 0 Contracting operator for a constraint c: x,x B i. θ(x) x ii. x x \ θ(x) : c(x) iii. x x = θ(x) θ(x )

13 SCAN 2010, Lyon, September 27 30, / 18 Contraction Step Constraints that must be verified by every global minimizer x i. f(x) u ii. x B = f(x) = 0 Contracting operator for a constraint c: x,x B i. θ(x) x ii. x x \ θ(x) : c(x) iii. x x = θ(x) θ(x ) Constraint propagation given θ 1,...,θ k : x B ( k ) ω x θ i (x) i=1 Note: greatest common fixed-point of the θ i included in x

14 SCAN 2010, Lyon, September 27 30, / 18 Inequality Constraint f(x) u s.t. f u f(x) = xcos(x) x = [ 5, 5] u = 2 x u f

15 SCAN 2010, Lyon, September 27 30, / 18 Inequality Constraint f(x) u s.t. f u f(x) = xcos(x) θ(x) = [2.4, 4.3] θ(x) Projection onto x 2B consistency box consistency u f

16 SCAN 2010, Lyon, September 27 30, / 18 System of Equations x B = f(x) = 0 f(x) = xcos(x) x = [ 2.7, 4.5] x

17 SCAN 2010, Lyon, September 27 30, / 18 System of Equations x B = f(x) = 0 f(x) = xcos(x) θ(x) = [ 1.2, 3.6] Solving methods interval Newton θ(x) consistency techniques f(x ) = 0

18 SCAN 2010, Lyon, September 27 30, / 18 Experiments Implementation in Realpaver 1.1 (2010) constraint solving and optimization library high-level modeling language interval arithmetic: gaol (F. Goualard) Initial box: [ 10000,1000] n (not centered around 0) Tolerance ε = Time Out (TO) : 600 seconds Processor: Intel Core2 Duo T GHz

SCAN 2010, Lyon, September 27 30, 2010 12 / 18 Problems (1)

19 SCAN 2010, Lyon, September 27 30, / 18 Problems (1) Schwefel function 2 n i i=1 j=1 x i Dixon-Price function n (x 1 1) 2 ( + i (2x 2 i x i 1 ) 2) i=2 polynomial, long narrow valley unimodal polynomial, large valley two global minimizers

Problems (2) 1 + 1 4000 Griewank function n x 2 i i=1 n ( ) xi cos i

trigonometric, multimodal multimodal, high degree regular distribution

20 Problems (2) Griewank function n x 2 i i=1 n ( ) xi cos i i=1 Michalewicz function n ( sin(x i )sin i x2 i π i=1 ) 20 trigonometric, multimodal multimodal, high degree regular distribution of narrow peaks, large valleys minimizers SCAN 2010, Lyon, September 27 30, / 18

21 SCAN 2010, Lyon, September 27 30, / 18 General Results Problem n occ sol u l split time Dixon Price Rosenbrock Schwefel Griewank Michalewicz Rastrigin

22 SCAN 2010, Lyon, September 27 30, / 18 Contraction Step We compare two algorithms for solving f(x) = 0 Full: best algorithm interval Newton method consistency techniques and constraint propagation Newton: only the interval Newton method Problem n Full Newton u l time u l time Dixon Price TO Rosenbrock Schwefel Griewank Michalewicz TO Rastrigin TO

23 SCAN 2010, Lyon, September 27 30, / 18 Minimization Step We compare three algorithms for improving the upper bound Newton descent method (local optimization) Grid-based algorithm Hybrid Grid+Newton Problem n Descent Grid Hybrid Dixon Price Rosenbrock Schwefel TO 1.90 Griewank Michalewicz Rastrigin

24 SCAN 2010, Lyon, September 27 30, / 18 Conclusions Constraint-based contracting techniques are used to enclose precisely and rigorously the global minimizers, resulting in a precise enclosure of the global minimum. Our algorithm is generic, described by an object model, and the components can vary independently from each others, allowing to test easily many combinations interval extension minimization step contraction step branching step Realpaver has been compared with two rigorous solvers: GlobSol (R.B. Kearfott) and icos (Y. Lebbah) the first results show good performances it should also be compared with other solvers

25 SCAN 2010, Lyon, September 27 30, / 18 A Branch-and-Bound Algorithm for Unconstrained Global Optimization Laurent Granvilliers and Alexandre Goldsztejn Université de Nantes LINA CNRS

Deterministic Global Optimization for Dynamic Systems Using Interval Analysis

Deterministic Global Optimization for Dynamic Systems Using Interval Analysis Youdong Lin and Mark A. Stadtherr Department of Chemical and Biomolecular Engineering University of Notre Dame, Notre Dame,