General Nonlinear Programming (NLP) Software

Transcription

1 General Nonlnear Programmng NLP Software CAS 737 / CES 735 Krstn Daves Hamd Ghaffar Alberto Olvera-Salazar Vou Chs January 2 26

2 Outlne Intro to NLP Eamnaton of: IPOPT PENNON CONOPT LOQO KNITRO Comparson of Computatonal Results Conlusons

3 Intro to NLP The general problem: mn f s. t. h I g j C where R n j J C R {... p} {... m} are funtons defned on C. n s a ertan set and NLP f h g j Ether the objetve funton or some of the onstrants may be nonlnear

4 Intro to NLP ont d Reall: The feasble regon of any LP s a onve set f the LP has an optmal soluton there s an etreme pont of the feasble set that s optmal However: even f the feasble regon of an NLP s a onve set the optmal soluton mght not be an etreme pont of the feasble regon

5 Intro to NLP ont d Some Major approahes for NLP Interor Pont Methods Use a log-barrer funton Penalty and Augmented Lagrange Methods Use the dea of penalty to transform a onstraned problem nto a sequene of unonstraned problems. Generalzed redued gradent GRG Use a bas Desent algorthm. Suessve quadrat programmng SQP Solves a quadrat appromaton at every teraton.

6 Summary of NLP Solvers NLP Augmented Lagrangan Methods PENNON Interor Pont Methods KNITRO TR IPOPT LOQO lne searh Redued Gradent Methods CONOPT

7 IPOPT SOLVER Interor Pont OPTmzer Creators Andreas Wahter and L.T. Begler at CMU ~22 Ams Solver for Large-Sale Nonlnear Optmzaton problems Applatons General Nonlnear optmzaton Proess Engneerng DAE/PDE Systems Proess Desgn and Operatons Nonlnear Model Predtve ontrol Desgn Under Unertanty

8 IPOPT SOLVER Interor Pont OPTmzer Input Format Can be lned to Fortran and C ode MATLAB and AMPL. Language / OS Fortran 77 C++ Reent Verson IPOPT 3. Lnu/UNIX platforms and Wndows Commeral/Free Released as open soure ode under the Common Publ Lense CPL. It s avalable from the COIN-OR repostory

9 IPOPT SOLVER Interor Pont OPTmzer Key Clams Global Convergene by usng a Lne Searh. Fnd a KKT pont Pont that Mnmzes Infeasblty loally Eplots Eat Seond Dervatves AMPL automat dfferentaton If not Avalable use QN appro BFGS Sparsty of the KKT matr. IPOPT has a verson to solve problems wth MPEC Constrants. IPOPT-C

10 IPOPT SOLVER Interor Pont OPTmzer Algorthm Interor Pont method wth a novel lne searh flter. Optmzaton Problem mn R s. t. n f mn R s. t. f μ l log The bounds are replaed by a logarthm Barrer term. The method solves a sequene of barrer problems for dereasng values of μ l n ϕ μ l

11 IPOPT SOLVER Interor Pont OPTmzer Algorthm μ l For a fed value of Solve the Barrer Problem Searh Dreton Prmal-Dual IP Use a Newton method to solve the prmal dual equatons. Hessan Appromaton BFGS update Lne Searh Flter Method Feasblty Restoraton Phase

12 Outer Loop IPOPT SOLVER Interor Pont OPTmzer Optmzaton Problem mn f mn R s. t. R s. t. n The bounds are replaed by a logarthm Barrer term. n ϕ μ l μ l The method solves a sequene of barrer problems for dereasng values of f μ l log

13 IPOPT SOLVER Interor Pont OPTmzer Algorthm For a fed value of Solve the Barrer Problem μ l Searh Dreton Prmal-Dual IP Use a Newton method to solve the prmal dual equatons Hessan Appromaton BFGS update

14 IPOPT SOLVER Interor Pont OPTmzer Inner Loop Inner Loop + e XVe v f μ λ Optmalty ondtons Optmalty ondtons e X v Varables Dual μ.. log mn R s t f l l n μ ϕ μ Barrer Barrer NLP NLP + e e V X v f d d d V X I H T v T μ λ λ 2 L H λ At a Newton's teraton At a Newton's teraton λ v v Σ T T H v d d I I H λ ϕ δ δ μ λ At a Newton's teraton At a Newton's teraton λ v v Algorthm Core: Soluton of ths Lnear system Algorthm Core: Soluton of ths Lnear system

15 IPOPT SOLVER Interor Pont OPTmzer μ l Algorthm For a fed value of Lne Searh Flter Method v + + A tral pont s aepted f mproves feasblty or f mproves the barrer funton v + α d + α d v If or ϕ [ α ] [ ] [ α ] ϕ[ ] Assumes Newton dretons are Good espeally when usng Eat 2 nd Dervatves

16 IPOPT SOLVER Interor Pont OPTmzer Lne Searh - Feasblty Restoraton Phase When a new tral pont does not provdes suffent mprovement. Restore Feasblty Mnmze onstrant volaton mn R s. t. n 2 2 Fore Unque Soluton Fnd losest feasble pont. Add Penalty funton mn R n s. t. 2 2

17 The omplety of the problem nreases when omplementarty ondtons are ntrodued from: m y w y w y w st y w f y w m m n K. mn R R R m y w y w st y w y w f m m n y w m m n K. ln ln ln mn + + R R R μ δμ y w The nteror Pont method for NLPs has been etended to handle omplementarty problems. Raghunathan et al. 23. y w s relaed as + s s y w δμ IPOPT SOLVER Interor Pont OPTmzer

18 IPOPT SOLVER Interor Pont OPTmzer Addtonal IPOPT 3. Is now programmed n C++. Is the prmary NLP Solver n an undergong projet for MINLP wth IBM. Referenes Ipopt homepage: A. Wähter and L. T. Begler On the Implementaton of a Prmal-Dual Interor Pont Flter Lne Searh Algorthm for Large-Sale Nonlnear Programmng Researh Report IBM T. J. Watson Researh Center Yortown USA Marh 24 - aepted for publaton n Mathematal Programmng

19 PENNON PENalty method for NONlnear & semdefnte programmng Creators Mhal Kovara & Mhael Stngl ~2 Ams NLP Semdefnte Programmng SDP Lnear & Blnear Matr Inequaltes LMI & BMI Seond Order Con Programmng SOCP Applatons General purpose nonlnear optmzaton systems of equatons ontrol theory eonoms & fnane strutural optmzaton engneerng

20 SDP SemDefnte Programmng Mnmzaton of a lnear funton subjet to the onstrant that an affne ombnaton of symmetr matres s postve semdefnte mn s. t. T where F F Lnear Matr Inequalty LMI defnes a onve onstrant on F + m F m + symmetr matres F... Fm

21 SDP SemDefnte Programmng -always an optmal pont on the boundary -boundary onssts of peewse algebra surfaes

22 SOCP Seond-Order Con Programmng Mnmzaton of a lnear funton subjet to a seond-order one onstrant T mn s. t. A + b T + d u C u R t R t Called a seond-order one onstrant sne the unt seond-order one of dmenson s defned as: u t Whh s alled the quadrat e-ream or Lorentz one

23 PENNON PENalty method for NONlnear & semdefnte programmng Input Format MATLAB funton routne alled from C or Fortran stand-alone program wth AMPL Language Fortran 77 Commeral/Free Varety of lenses rangng from Aadem sngle user $46 CDN to Commeral ompany $45 CDN

24 PENNON PENalty method for NONlnear & semdefnte programmng Key Clams st avalable ode for ombo NLP LMI & BMI onstrants Amed at very large-sale problems Effent treatment of dfferent sparsty patterns n problem data Robust wth respet to feasblty of ntal guess Partularly effent for large onve problems

25 PENNON PENalty method for NONlnear & semdefnte programmng Algorthm Generalzed verson of the Augmented Langrangan method orgnally by Ben-Tal & Zbulevsy Augmented Problem mn f mg s. t. p ϕ p g g / p Augmented Lagrangan m g F u p f + u p ϕ g g / p > penalty parameter ϕ penalty funton u g... m # of g Lagrange nequalty multpler onstrants

26 PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm Consder only nequalty onstrants from NLP Based on hoe of a penalty funton φ g that penalzes the nequalty onstrants Penalty funton must satsfy multple propertes suh that the orgnal NLP has the same soluton as the followng augmented problem: mn f R s. t. pϕ g g / p... mg NLP φ wth p > [3] Kovara & Stngl

27 [3] Kovara & Stngl PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm Cont d The Lagrangan of NLP φ an be vewed as a generalzed augmented Lagrangan of NLP: m g F u p f + u p ϕ Lagrange multpler g Penalty parameter g / p Inequalty onstrant Penalty funton

28 PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm STEPS Let and u be gven. Let p >... m g. For 2... repeat untl a stoppng rterum s satsfed. Fnd u + p + + < u p suh that ϕ ' g + g / p... m g F + u p... m g K [3] Kovara & Stngl

29 [3] Kovara & Stngl PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm STEPS Let and u be gven. Let p >... m g. Intalzaton Can start wth an arbtrary prmal varable therefore hoose Calulate ntal multpler values u Intal p typally between - π

30 PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm STEPS K p u F that suh Fnd + + Appromate Unonstraned Mnmzaton Performed ether by Newton wth Lne Searh or by Trust Regon Stoppng rtera: [3] Kovara & Stngl /. 2 ' H H g p u F p u F or p g u u p u F or p u F α ψ α α α ' ' ϕ ψ arg mn p u F +

31 PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm STEPS u + u ϕ ' g + g / p... mg Update of Multplers Restrted n order to satsfy: + u μ < < wth a postve u μ μ typally.5 If left-sde volated let If rght sde volate let new u new u μ / μ [3] Kovara & Stngl

32 PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm STEPS p < p... m + g Update of Penalty Parameter No update durng frst 3 teratons Afterwards updated by a onstant fator dependent on ntal penalty parameter Penalty update s stopped f p eps -6 s reahed [3] Kovara & Stngl

33 PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm Choe of Penalty Funton Most effent penalty funton for onve NLP s the quadrat-logarthm funton: ϕ t g 2 t t + 3 t r where r and 4 log t t < r...6 so that propertes hold [4] Ben-Tal & Zbulevsy

34 PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm Overall Stoppng Crtera [3] Kovara & Stngl 7 < + < + ε ε ε where f f f or f p u F f

35 PENNON PENalty method for NONlnear & semdefnte programmng Assumptons / Warnngs More tunng for nononve problems s stll requred Slower at solvng lnear SDP problems sne algorthm s generalzed

36 PENNON PENalty method for NONlnear & semdefnte programmng Referenes Kovara Mhal & Mhael Stngl. PENNON: A Code for Conve and Semdefnte Programmng. Optmzaton Methods and Software 83: Kovara Mhal & Mhael Stngl. PENNON-AMPL User s Gude. August 23. Ben-Tal Aharon & Mhael Zbulevsy. Penalty/Barrer Multpler Methods for Conve Programmng Problems. Sam J. Optm. 72: Pennon Homepage. Avalable onlne January 27.