Derivative-Free Optimization of Noisy Functions via Quasi-Newton Methods. Jorge Nocedal

Transcription

1 Derivative-Free Optimization of Noisy Functions via Quasi-Newton Methods Jorge Nocedal Northwestern University Huatulco, Jan

2 Collaborators Albert Berahas Northwestern University Richard Byrd University of Colorado 2

3 Discussion 1. The BFGS method continues to surprise 2. One of the best methods for nonsmooth optimization (Lewis-Overton) 3. Leading approach for (deterministic) DFO derivative-free optimization 4. This talk: Very good method for the minimization of noisy functions We had not fully recognized the power and generality of quasi-newton updating until we tried to find alternatives! Subject of this talk: 1. Black-box noisy functions 2. No known structure 3. Not the finite sum loss functions arising in machine learning, where cheap approximate gradients are available 3

4 Outline Problem 1: min f (x) 1. f contains no noise 2. Scalability, Parallelism 3. Robustness f smooth but derivatives not available: DFO Problem 2: min f (x;ξ) min f (x) =φ(x) + ε(x) f ( ;ξ) smooth f (x) = φ(x)(1+ ε(x)) Propose method build upon classical quasi-newton updating using finitedifference gradients Estimate good finite-difference interval h Use noise estimation techniques (More -Wild) Deal with noise adaptively Can solve problems with thousands of variables Novel convergence results to neighborhood of solution (Richard Byrd) 4

5 DFO: Derivative free deterministic optimization (no noise) min f (x) f is smooth Direct search/pattern search methods: not scalable Much better idea: Interpolation based models with trust regions (Powell, Conn, Scheinberg, ) min m(x) = xt Bx + g T x s.t. x 2 Δ 1. Need (n+1)(n+2)/2 function values to define quadratic model by pure interpolation 2. Can use O(n) points and assume minimum norm change in the Hessian 3. Arithmetic costs high: n 4 ß scalability 4. Placement of interpolation points is important 5. Correcting the model may require many function evaluations 6. Parallelizable? ß 5

6 Why not simply BFGS with finite difference gradients? x k+1 = x k α k H k f (x k ) f (x) x i f (x + he i ) f (x) h Invest significant effort in estimation of gradient Delegate construction of model to BFGS Interpolating gradients Modest linear algebra costs O(n) for L-BFGS Placement of sample points on an orthogonal set BFGS is an overwriting process: no inconsistencies or ill conditioning with Armijo-Wolfe line search Gradient evaluation parallelizes easily Why now? Perception that n function evaluations per step is too high Derivative-free literature rarely compares with FD quasi-newton Already used extensively: fminunc MATLAB Black-box competition and KNITRO 6

7 Some numerical results Compare: Model based trust region code DFOtr by Conn, Scheinberg, Vicente vs FD-L-BFGS with forward and central differences Plot function decrease vs total number of function evaluations 7

8 Comparison: function decrease vs total # of function evaluations quadratic s271 Smooth Deterministic DFOtr LBFGS FD (FD) LBFGS FD (CD) s334 Smooth Deterministic DFOtr LBFGS FD (FD) LBFGS FD (CD) 10-5 F(x)-F * F(x)-F * Number of function evaluations Number of function evaluations s293 Smooth Deterministic DFOtr LBFGS FD (FD) LBFGS FD (CD) s289 Smooth Deterministic DFOtr LBFGS FD (FD) LBFGS FD (CD) F(x)-F * F(x)-F * Number of function evaluations Number of function evaluations 8

9 Conclusion: DFO without noise Finite difference BFGS is a real competitor of DFO method based on function evaluations Can solve problems with thousands of variables but really nothing new. 9

10 Optimization of Noisy Functions min f (x;ξ) min f (x) =φ(x) + ε(x) where f ( ;ξ) is smooth f (x) = φ(x)(1+ ε(x)) Focus on additive noise Finite-difference BFGS should not work! 1. Difference of noisy functions dangerous 2. Just one bad update once in a while: disastrous 3. Not done to the best of our knowledge f(x) f(x) = sin(x) + cos(x) U(0,2sqrt(3)) Smooth Noisy x 10

11 Finite Differences Noisy Functions (x) f (x) = (x)+ (x) h too small True x = 2.5: 1.6 Finite Di erence x = 2.5: 1.33 h too big True x = 3.5: 0.5 Finite Di erence x = 3.5: 0.5 (x) f (x) = (x)+ (x) h h x h h x 11

12 A Practible Algorithm min f (x) =φ(x) + ε(x) f (x) = φ(x)(1+ ε(x)) Outline of adaptive finite-difference BFGS method 1. Estimate noise ε(x) at every iteration -- More -Wild 2. Estimate h 3. Compute finite difference gradient 4. Perform line search (?!) 5. Corrective Procedure when case line search fails (need to modify line search) Re-estimate noise level Will require very few extra f evaluations/iteration even none 12

13 Noise estimation More -Wild (2011) Noise level: σ = [var(ε(x))] 1/2 Noise estimate: ε f min f (x) =φ(x) + ε(x) At x choose a random direction v evaluate f at q +1 equally spaced points x + iβv, i = 0,...,q Compute function differences: Δ 0 f (x) = f (x) Δ j+1 f (x) = Δ j [Δf (x)] = Δ j [ f (x + β)] Δ j [ f (x)]] Compute finite diverence table: T ij = Δ j f (x + iβv) 1 < j < q 0 < i < j q σ j = γ j q 1 j q j i=0 2 T i, j γ j = ( j!)2 (2 j)! 13

14 X Noise estimation More -Wild (2011) min f (x) =sin(x) + cos(x) U(0,2 3 ) q = 6 β = 10 2 x f f 2 f 3 f 4 f 5 f 6 f e e e e e e e e e e e e e e e e e e e e e k 6.65e e e e e e 4 High order differences of a smooth function tend to zero rapidly, while differences in noise are bounded away from zero. Changes in sign, useful. Procedure is scale invariant! 14

15 Finite difference itervals Once noise estimate ε f has been chosen: Forward difference: h = 8 1/4 ( ε f µ 2 ) 1/2 Central difference: h = 3 1/3 ( ε f µ 3 ) 1/3 µ 2 = max x I f (x) µ 3 f (x) Bad estimates of second and third derivatives can cause problems (not often) 15

16 Adaptive Finite Difference L-BFGS Method Estimate noise ε f Compute h by for forward or central differences [(4-8) function evaluations] Compute g k While convergence test not satisfied: d = H k g k [L-BFGS procedure] (x +, f +, flag) = LineSearch(x k, f k,g k,d k, f s ) IF flag=1 endif [line search failed] (x +, f +,h) = Recovery(x k, f k,g k,d k,max iter ) x k+1 = x +, f k+1 = f + Compute g k+1 [finite differences using h] s k = x k+1 x k, y k = g k+1 g k Discard (s k, y k ) if s k T y k 0 k = k +1 endwhile 16

17 Line Search BFGS method requires Armijo-Wolfe line search f (x k + αd) f (x k ) + αc 1 f (x k )d f (x k + αd) T d c 2 f (x k ) T d Wolfe Armijo Deterministic case: always possible if f is bounded below Can be problematic in the noisy case. Strategy: try to satisfy both but limit the number of attempts If first trial point (unit steplength) is not acceptable relax: f (x k + αd) f (x k ) + αc 1 f (x k )d + 2ε f relaxed Armijo Three outcomes: a) both satisfied; b) only Armijo; c) none 17

18 Corrective Procedure Compute a new noise estimate ε f along search direction d k Compute corresponding h If ĥ / h use new estimat h h; return w.o. changing x k Else compute new iterate (various options): small perturbation; stencil point Finite difference Stencil (Kelley) x s 18

19 Some quotes I believe that eventually the better methods will not use derivative approximations [Powell, 1972] f is somewhat noisy, which renders most methods based on finite differences of little or no use [X,X,X]. [Rios & Sahinidis, 2013] 19

20 END 20