Nonlinear programming

Transcription

1 htt://staff.chemeng.lth.se/~berntn/courses/otimps.htm Otimization of Process Systems Nonlinear rogramming PhD course 08 Bernt Nilsson, Det of Chemical Engineering, Lund University Content Unconstraint Nonlinear Programming Problem formulation Conve roblems Methods Direct methods Search methods, Nelder-Mead and Pattern search Indirect methods and Newton based methods Line search acobian calculations and Hessian udates

2 Unconstraint minimization min X Minimize objective function, by changing the decision variables Maimize => Minimize Does a minima eist? -* = htt://en.wikiedia.org/wiki/nonlinear_rogramming Unconstraint minimization min X Elicit model based otimization min, y X objective dy dt f y, rocess model Parameterized otimization min, y, X objective dy dt f y,, rocess model

3 Unconstraint minimization - conve Does a minima eist? => Is the roblem conve? Yes, if f >0 from one-dim math Multi dimension second order derivative = Hessian matri, H i Hij here eist a minima if H is ositive definite => H has ositive eigenvalues j min X Other roblems Multile otima Local and global solution Noisy objective Robust solver 3

4 Methods overview Direct methods single-oint, derivative-free Fundamental aroach: Probing of the objective function Eamle: Nelder-Mead method Indirect methods single-oint, derivative-based Fundamental aroach: Derivative-based iteration Eamle: quasi-newton metods Evolutionary methods multi-oint, derivative-free Fundamental aroach: Survival of the fittest Eamle: Genetic algorithm Direct methods Fundamental aroach: Probing of the objective function Uses only direct function calls Characteristics: Easy to use Can handle comle object functions discontinuous, noisy, Slow convergence Eamle: Nelder-Mead Pattern search 4

5 Direct method: Nelder-Mead method fminsearch Nelder-Mead simle method Based on a regular figure a simle riangle in -D etrahedron in 3-D Function calls in all corners Steing Reflect = ste away Eand = far away Contract = otima inside 3 4 Reflection = steing away Contraction = steing inside htt://en.wikiedia.org/wiki/nelder%e%80%93mead_method Direct method: Nelder-Mead method. Order all values at the vertices Calculate 0, the center of gravity. Reflection Comute reflected oint, r. If better than second worst, then accet new oint 3. Eansion If reflected oint is best then comute eanded oint. If eanded oint is better than reflected, then accet eanded oint. 4. Contraction If reflected oint is worst then comute contraction oint. If contraction oint is better than worst oint, then accet contraction oint. 5. Reduction For all but the best oint, relace with a reduced oint. f f... f n f r 0 0 n f r f n n r f r f e 0 0 n f f e r n e f n f r c 0 0 n f c f n n c i i 5

6 Direct method: Nelder-Mead method. Order all values at the vertices Calculate 0, the center of gravity r 0 0 n f f. Reflection f r n f r f e 0 0 n f f n e 3. Eansion e r 4. Contraction c 0 0 n f c f n f n r f f f... f n n r n c 5. Reduction i i Direct method: Pattern search method atternsearch Multile function calls in one ste Better descrition of objective function Parallell comuting Polling, new oints in mesh Better oint => eand No better oints => refine htt://en.wikiedia.org/wiki/pattern_search_otimization 6

7 Indirect methods Derivative-based search Fundamental aroach: Newton s method min ' 0 '' ' Newton method I one variable Fundamental aroach: Newton s method Newton in one variable c 0 c c c' c' c Minimization using Newton 0 min ' 0 ' ' '' '' ' 0 Note! Newton s method search for etreem oint => can t see the difference between minima or maima 7

8 Newton method II multile variables acobian gradient vector and Hessian matri second order derivative Newton minimization H H H H 0 n n n n n H Ste: called a Newton ste quasi-newton method Newton ste Problems handled in quasi-newton:. acobian Finite difference Sensitivity functions. Hessian is comutational eensive and have low accuracy Hessian aroimations is the search direction 3. uasi-newton ste length Line search methods garantee decent H B B H, htt://en.wikiedia.org/wiki/uasi-newton_method

9 quasi-newton acobian B acobian, = Partial derivative of the objective function,, with resect to the decision variables, n Numerical aroimation finite difference using a small disturbance - disturbance size have to be bigger then the error in -calculation = +ε ε htt://en.wikiedia.org/wiki/bfgs quasi-newton Hessian B Secant here in D Broyden udate BFGS udate Broyden Fletcher Goldfarb - Shannon k B k k B B k B B B B B becomes not ositive definite after 5 udates => restart B DFP udate of the B-invers Davidson Fletcher Powell B B B B B start B 0 = I => steeest decent = htt://en.wikiedia.org/wiki/bfgs 9

10 dogleg quasi-newton 3 ste length B Near the solution the Newton method is very good! Far from solution B is wrong, making the Newton ste bad large Line search methods Use, and + to make a quadratic estimate of the minimum, find that fullfill, Goldstein-Armijo 0 ' ' where 0 4 & 0.9 rust-region where quadratic aroimation is good Ratio actual/redicted H 0 reduction of, ar/r H I 0 => small, decrease tr => medium, unchange tr H I => large, increase tr Summary Nonlinear rogramming of unconstraint roblem Conve roblem => a solution eists! Method classes: Direct methods E: Nelder-Mead and attern search Indirect methods E: uasi-newton with i Numerical acobian, ii BFGS udate and iii line search ste size method Evolutionary methods GA/DE 0