General Nonlinear Programming (NLP) Software
|
|
- Ami McDowell
- 5 years ago
- Views:
Transcription
1 General Nonlnear Programmng NLP Software CAS 737 / CES 735 Krstn Daves Hamd Ghaffar Alberto Olvera-Salazar Vou Chs January 2 26
2 Outlne Intro to NLP Eamnaton of: IPOPT PENNON CONOPT LOQO KNITRO Comparson of Computatonal Results Conlusons
3 Intro to NLP The general problem: mn f s. t. h I g j C where R n j J C R {... p} {... m} are funtons defned on C. n s a ertan set and NLP f h g j Ether the objetve funton or some of the onstrants may be nonlnear
4 Intro to NLP ont d Reall: The feasble regon of any LP s a onve set f the LP has an optmal soluton there s an etreme pont of the feasble set that s optmal However: even f the feasble regon of an NLP s a onve set the optmal soluton mght not be an etreme pont of the feasble regon
5 Intro to NLP ont d Some Major approahes for NLP Interor Pont Methods Use a log-barrer funton Penalty and Augmented Lagrange Methods Use the dea of penalty to transform a onstraned problem nto a sequene of unonstraned problems. Generalzed redued gradent GRG Use a bas Desent algorthm. Suessve quadrat programmng SQP Solves a quadrat appromaton at every teraton.
6 Summary of NLP Solvers NLP Augmented Lagrangan Methods PENNON Interor Pont Methods KNITRO TR IPOPT LOQO lne searh Redued Gradent Methods CONOPT
7 IPOPT SOLVER Interor Pont OPTmzer Creators Andreas Wahter and L.T. Begler at CMU ~22 Ams Solver for Large-Sale Nonlnear Optmzaton problems Applatons General Nonlnear optmzaton Proess Engneerng DAE/PDE Systems Proess Desgn and Operatons Nonlnear Model Predtve ontrol Desgn Under Unertanty
8 IPOPT SOLVER Interor Pont OPTmzer Input Format Can be lned to Fortran and C ode MATLAB and AMPL. Language / OS Fortran 77 C++ Reent Verson IPOPT 3. Lnu/UNIX platforms and Wndows Commeral/Free Released as open soure ode under the Common Publ Lense CPL. It s avalable from the COIN-OR repostory
9 IPOPT SOLVER Interor Pont OPTmzer Key Clams Global Convergene by usng a Lne Searh. Fnd a KKT pont Pont that Mnmzes Infeasblty loally Eplots Eat Seond Dervatves AMPL automat dfferentaton If not Avalable use QN appro BFGS Sparsty of the KKT matr. IPOPT has a verson to solve problems wth MPEC Constrants. IPOPT-C
10 IPOPT SOLVER Interor Pont OPTmzer Algorthm Interor Pont method wth a novel lne searh flter. Optmzaton Problem mn R s. t. n f mn R s. t. f μ l log The bounds are replaed by a logarthm Barrer term. The method solves a sequene of barrer problems for dereasng values of μ l n ϕ μ l
11 IPOPT SOLVER Interor Pont OPTmzer Algorthm μ l For a fed value of Solve the Barrer Problem Searh Dreton Prmal-Dual IP Use a Newton method to solve the prmal dual equatons. Hessan Appromaton BFGS update Lne Searh Flter Method Feasblty Restoraton Phase
12 Outer Loop IPOPT SOLVER Interor Pont OPTmzer Optmzaton Problem mn f mn R s. t. R s. t. n The bounds are replaed by a logarthm Barrer term. n ϕ μ l μ l The method solves a sequene of barrer problems for dereasng values of f μ l log
13 IPOPT SOLVER Interor Pont OPTmzer Algorthm For a fed value of Solve the Barrer Problem μ l Searh Dreton Prmal-Dual IP Use a Newton method to solve the prmal dual equatons Hessan Appromaton BFGS update
14 IPOPT SOLVER Interor Pont OPTmzer Inner Loop Inner Loop + e XVe v f μ λ Optmalty ondtons Optmalty ondtons e X v Varables Dual μ.. log mn R s t f l l n μ ϕ μ Barrer Barrer NLP NLP + e e V X v f d d d V X I H T v T μ λ λ 2 L H λ At a Newton's teraton At a Newton's teraton λ v v Σ T T H v d d I I H λ ϕ δ δ μ λ At a Newton's teraton At a Newton's teraton λ v v Algorthm Core: Soluton of ths Lnear system Algorthm Core: Soluton of ths Lnear system
15 IPOPT SOLVER Interor Pont OPTmzer μ l Algorthm For a fed value of Lne Searh Flter Method v + + A tral pont s aepted f mproves feasblty or f mproves the barrer funton v + α d + α d v If or ϕ [ α ] [ ] [ α ] ϕ[ ] Assumes Newton dretons are Good espeally when usng Eat 2 nd Dervatves
16 IPOPT SOLVER Interor Pont OPTmzer Lne Searh - Feasblty Restoraton Phase When a new tral pont does not provdes suffent mprovement. Restore Feasblty Mnmze onstrant volaton mn R s. t. n 2 2 Fore Unque Soluton Fnd losest feasble pont. Add Penalty funton mn R n s. t. 2 2
17 The omplety of the problem nreases when omplementarty ondtons are ntrodued from: m y w y w y w st y w f y w m m n K. mn R R R m y w y w st y w y w f m m n y w m m n K. ln ln ln mn + + R R R μ δμ y w The nteror Pont method for NLPs has been etended to handle omplementarty problems. Raghunathan et al. 23. y w s relaed as + s s y w δμ IPOPT SOLVER Interor Pont OPTmzer
18 IPOPT SOLVER Interor Pont OPTmzer Addtonal IPOPT 3. Is now programmed n C++. Is the prmary NLP Solver n an undergong projet for MINLP wth IBM. Referenes Ipopt homepage: A. Wähter and L. T. Begler On the Implementaton of a Prmal-Dual Interor Pont Flter Lne Searh Algorthm for Large-Sale Nonlnear Programmng Researh Report IBM T. J. Watson Researh Center Yortown USA Marh 24 - aepted for publaton n Mathematal Programmng
19 PENNON PENalty method for NONlnear & semdefnte programmng Creators Mhal Kovara & Mhael Stngl ~2 Ams NLP Semdefnte Programmng SDP Lnear & Blnear Matr Inequaltes LMI & BMI Seond Order Con Programmng SOCP Applatons General purpose nonlnear optmzaton systems of equatons ontrol theory eonoms & fnane strutural optmzaton engneerng
20 SDP SemDefnte Programmng Mnmzaton of a lnear funton subjet to the onstrant that an affne ombnaton of symmetr matres s postve semdefnte mn s. t. T where F F Lnear Matr Inequalty LMI defnes a onve onstrant on F + m F m + symmetr matres F... Fm
21 SDP SemDefnte Programmng -always an optmal pont on the boundary -boundary onssts of peewse algebra surfaes
22 SOCP Seond-Order Con Programmng Mnmzaton of a lnear funton subjet to a seond-order one onstrant T mn s. t. A + b T + d u C u R t R t Called a seond-order one onstrant sne the unt seond-order one of dmenson s defned as: u t Whh s alled the quadrat e-ream or Lorentz one
23 PENNON PENalty method for NONlnear & semdefnte programmng Input Format MATLAB funton routne alled from C or Fortran stand-alone program wth AMPL Language Fortran 77 Commeral/Free Varety of lenses rangng from Aadem sngle user $46 CDN to Commeral ompany $45 CDN
24 PENNON PENalty method for NONlnear & semdefnte programmng Key Clams st avalable ode for ombo NLP LMI & BMI onstrants Amed at very large-sale problems Effent treatment of dfferent sparsty patterns n problem data Robust wth respet to feasblty of ntal guess Partularly effent for large onve problems
25 PENNON PENalty method for NONlnear & semdefnte programmng Algorthm Generalzed verson of the Augmented Langrangan method orgnally by Ben-Tal & Zbulevsy Augmented Problem mn f mg s. t. p ϕ p g g / p Augmented Lagrangan m g F u p f + u p ϕ g g / p > penalty parameter ϕ penalty funton u g... m # of g Lagrange nequalty multpler onstrants
26 PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm Consder only nequalty onstrants from NLP Based on hoe of a penalty funton φ g that penalzes the nequalty onstrants Penalty funton must satsfy multple propertes suh that the orgnal NLP has the same soluton as the followng augmented problem: mn f R s. t. pϕ g g / p... mg NLP φ wth p > [3] Kovara & Stngl
27 [3] Kovara & Stngl PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm Cont d The Lagrangan of NLP φ an be vewed as a generalzed augmented Lagrangan of NLP: m g F u p f + u p ϕ Lagrange multpler g Penalty parameter g / p Inequalty onstrant Penalty funton
28 PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm STEPS Let and u be gven. Let p >... m g. For 2... repeat untl a stoppng rterum s satsfed. Fnd u + p + + < u p suh that ϕ ' g + g / p... m g F + u p... m g K [3] Kovara & Stngl
29 [3] Kovara & Stngl PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm STEPS Let and u be gven. Let p >... m g. Intalzaton Can start wth an arbtrary prmal varable therefore hoose Calulate ntal multpler values u Intal p typally between - π
30 PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm STEPS K p u F that suh Fnd + + Appromate Unonstraned Mnmzaton Performed ether by Newton wth Lne Searh or by Trust Regon Stoppng rtera: [3] Kovara & Stngl /. 2 ' H H g p u F p u F or p g u u p u F or p u F α ψ α α α ' ' ϕ ψ arg mn p u F +
31 PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm STEPS u + u ϕ ' g + g / p... mg Update of Multplers Restrted n order to satsfy: + u μ < < wth a postve u μ μ typally.5 If left-sde volated let If rght sde volate let new u new u μ / μ [3] Kovara & Stngl
32 PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm STEPS p < p... m + g Update of Penalty Parameter No update durng frst 3 teratons Afterwards updated by a onstant fator dependent on ntal penalty parameter Penalty update s stopped f p eps -6 s reahed [3] Kovara & Stngl
33 PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm Choe of Penalty Funton Most effent penalty funton for onve NLP s the quadrat-logarthm funton: ϕ t g 2 t t + 3 t r where r and 4 log t t < r...6 so that propertes hold [4] Ben-Tal & Zbulevsy
34 PENNON PENalty method for NONlnear & semdefnte programmng The Algorthm Overall Stoppng Crtera [3] Kovara & Stngl 7 < + < + ε ε ε where f f f or f p u F f
35 PENNON PENalty method for NONlnear & semdefnte programmng Assumptons / Warnngs More tunng for nononve problems s stll requred Slower at solvng lnear SDP problems sne algorthm s generalzed
36 PENNON PENalty method for NONlnear & semdefnte programmng Referenes Kovara Mhal & Mhael Stngl. PENNON: A Code for Conve and Semdefnte Programmng. Optmzaton Methods and Software 83: Kovara Mhal & Mhael Stngl. PENNON-AMPL User s Gude. August 23. Ben-Tal Aharon & Mhael Zbulevsy. Penalty/Barrer Multpler Methods for Conve Programmng Problems. Sam J. Optm. 72: Pennon Homepage. Avalable onlne January 27.
MMA and GCMMA two methods for nonlinear optimization
MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons
More informationCHAPTER 7 CONSTRAINED OPTIMIZATION 2: SQP AND GRG
Chapter 7: Constraned Optmzaton CHAPER 7 CONSRAINED OPIMIZAION : SQP AND GRG Introducton In the prevous chapter we eamned the necessary and suffcent condtons for a constraned optmum. We dd not, however,
More informationSolutions to exam in SF1811 Optimization, Jan 14, 2015
Solutons to exam n SF8 Optmzaton, Jan 4, 25 3 3 O------O -4 \ / \ / The network: \/ where all lnks go from left to rght. /\ / \ / \ 6 O------O -5 2 4.(a) Let x = ( x 3, x 4, x 23, x 24 ) T, where the varable
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationConic Programming in GAMS
Conc Programmng n GAMS Armn Pruessner, Mchael Busseck, Steven Drkse, Ale Meeraus GAMS Development Corporaton INFORMS 003, Atlanta October 19- Drecton What ths talk s about Overvew: the class of conc programs
More information6.854J / J Advanced Algorithms Fall 2008
MIT OpenCourseWare http://ocw.mt.edu 6.854J / 18.415J Advanced Algorthms Fall 2008 For nformaton about ctng these materals or our Terms of Use, vst: http://ocw.mt.edu/terms. 18.415/6.854 Advanced Algorthms
More informationConvex Optimization. Optimality conditions. (EE227BT: UC Berkeley) Lecture 9 (Optimality; Conic duality) 9/25/14. Laurent El Ghaoui.
Convex Optmzaton (EE227BT: UC Berkeley) Lecture 9 (Optmalty; Conc dualty) 9/25/14 Laurent El Ghaou Organsatonal Mdterm: 10/7/14 (1.5 hours, n class, double-sded cheat sheet allowed) Project: Intal proposal
More informationSome modelling aspects for the Matlab implementation of MMA
Some modellng aspects for the Matlab mplementaton of MMA Krster Svanberg krlle@math.kth.se Optmzaton and Systems Theory Department of Mathematcs KTH, SE 10044 Stockholm September 2004 1. Consdered optmzaton
More informationCollege of Computer & Information Science Fall 2009 Northeastern University 20 October 2009
College of Computer & Informaton Scence Fall 2009 Northeastern Unversty 20 October 2009 CS7880: Algorthmc Power Tools Scrbe: Jan Wen and Laura Poplawsk Lecture Outlne: Prmal-dual schema Network Desgn:
More informationChapter Newton s Method
Chapter 9. Newton s Method After readng ths chapter, you should be able to:. Understand how Newton s method s dfferent from the Golden Secton Search method. Understand how Newton s method works 3. Solve
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationLecture-7. Homework (Due 2/13/03)
Leture-7 Ste Length Seleton Homewor Due /3/3 3. 3. 3.5 3.6 3.7 3.9 3. Show equaton 3.44 he last ste n the roo o heorem 3.6. see sldes Show that >.5, the lne searh would exlude the mnmzer o a quadrat, and
More informationMaximal Margin Classifier
CS81B/Stat41B: Advanced Topcs n Learnng & Decson Makng Mamal Margn Classfer Lecturer: Mchael Jordan Scrbes: Jana van Greunen Corrected verson - /1/004 1 References/Recommended Readng 1.1 Webstes www.kernel-machnes.org
More information1 Matrix representations of canonical matrices
1 Matrx representatons of canoncal matrces 2-d rotaton around the orgn: ( ) cos θ sn θ R 0 = sn θ cos θ 3-d rotaton around the x-axs: R x = 1 0 0 0 cos θ sn θ 0 sn θ cos θ 3-d rotaton around the y-axs:
More informationprinceton univ. F 17 cos 521: Advanced Algorithm Design Lecture 7: LP Duality Lecturer: Matt Weinberg
prnceton unv. F 17 cos 521: Advanced Algorthm Desgn Lecture 7: LP Dualty Lecturer: Matt Wenberg Scrbe: LP Dualty s an extremely useful tool for analyzng structural propertes of lnear programs. Whle there
More informationLecture 21: Numerical methods for pricing American type derivatives
Lecture 21: Numercal methods for prcng Amercan type dervatves Xaoguang Wang STAT 598W Aprl 10th, 2014 (STAT 598W) Lecture 21 1 / 26 Outlne 1 Fnte Dfference Method Explct Method Penalty Method (STAT 598W)
More informationController Design for Networked Control Systems in Multiple-packet Transmission with Random Delays
Appled Mehans and Materals Onlne: 03-0- ISSN: 66-748, Vols. 78-80, pp 60-604 do:0.408/www.sentf.net/amm.78-80.60 03 rans eh Publatons, Swtzerland H Controller Desgn for Networed Control Systems n Multple-paet
More informationFeature Selection: Part 1
CSE 546: Machne Learnng Lecture 5 Feature Selecton: Part 1 Instructor: Sham Kakade 1 Regresson n the hgh dmensonal settng How do we learn when the number of features d s greater than the sample sze n?
More informationKernel Methods and SVMs Extension
Kernel Methods and SVMs Extenson The purpose of ths document s to revew materal covered n Machne Learnng 1 Supervsed Learnng regardng support vector machnes (SVMs). Ths document also provdes a general
More information: Numerical Analysis Topic 2: Solution of Nonlinear Equations Lectures 5-11:
764: Numercal Analyss Topc : Soluton o Nonlnear Equatons Lectures 5-: UIN Malang Read Chapters 5 and 6 o the tetbook 764_Topc Lecture 5 Soluton o Nonlnear Equatons Root Fndng Problems Dentons Classcaton
More informationLecture 2 Solution of Nonlinear Equations ( Root Finding Problems )
Lecture Soluton o Nonlnear Equatons Root Fndng Problems Dentons Classcaton o Methods Analytcal Solutons Graphcal Methods Numercal Methods Bracketng Methods Open Methods Convergence Notatons Root Fndng
More informationProseminar Optimierung II. Victor A. Kovtunenko SS 2012/2013: LV
Prosemnar Optmerung II Vctor A. Kovtunenko Insttute for Mathematcs and Scentfc Computng, Karl-Franzens Unversty of Graz, Henrchstr. 36, 8010 Graz, Austra; Lavrent ev Insttute of Hydrodynamcs, Sberan Dvson
More informationMULTICRITERION OPTIMIZATION OF LAMINATE STACKING SEQUENCE FOR MAXIMUM FAILURE MARGINS
MLTICRITERION OPTIMIZATION OF LAMINATE STACKING SEENCE FOR MAXIMM FAILRE MARGINS Petr Kere and Juhan Kos Shool of Engneerng, Natonal nversty of ruguay J. Herrera y Ressg 565, Montevdeo, ruguay Appled Mehans,
More informationMean Field / Variational Approximations
Mean Feld / Varatonal Appromatons resented by Jose Nuñez 0/24/05 Outlne Introducton Mean Feld Appromaton Structured Mean Feld Weghted Mean Feld Varatonal Methods Introducton roblem: We have dstrbuton but
More informationPortfolios with Trading Constraints and Payout Restrictions
Portfolos wth Tradng Constrants and Payout Restrctons John R. Brge Northwestern Unversty (ont wor wth Chrs Donohue Xaodong Xu and Gongyun Zhao) 1 General Problem (Very) long-term nvestor (eample: unversty
More informationME 501A Seminar in Engineering Analysis Page 1
umercal Solutons of oundary-value Problems n Os ovember 7, 7 umercal Solutons of oundary- Value Problems n Os Larry aretto Mechancal ngneerng 5 Semnar n ngneerng nalyss ovember 7, 7 Outlne Revew stff equaton
More informationOPTIMISATION. Introduction Single Variable Unconstrained Optimisation Multivariable Unconstrained Optimisation Linear Programming
OPTIMIATION Introducton ngle Varable Unconstraned Optmsaton Multvarable Unconstraned Optmsaton Lnear Programmng Chapter Optmsaton /. Introducton In an engneerng analss, sometmes etremtes, ether mnmum or
More informationECE559VV Project Report
ECE559VV Project Report (Supplementary Notes Loc Xuan Bu I. MAX SUM-RATE SCHEDULING: THE UPLINK CASE We have seen (n the presentaton that, for downlnk (broadcast channels, the strategy maxmzng the sum-rate
More informationSingle-Facility Scheduling over Long Time Horizons by Logic-based Benders Decomposition
Sngle-Faclty Schedulng over Long Tme Horzons by Logc-based Benders Decomposton Elvn Coban and J. N. Hooker Tepper School of Busness, Carnege Mellon Unversty ecoban@andrew.cmu.edu, john@hooker.tepper.cmu.edu
More informationLagrange Multipliers Kernel Trick
Lagrange Multplers Kernel Trck Ncholas Ruozz Unversty of Texas at Dallas Based roughly on the sldes of Davd Sontag General Optmzaton A mathematcal detour, we ll come back to SVMs soon! subject to: f x
More informationOn a Solution of Fractional Programs via D.C. Optimization Theory
On a Soluton of Fractonal Programs va D.C. Optmzaton Theory Tatana V. Gruzdeva Aleander S. Strekalovsky Matrosov Insttute for System Dynamcs and Control Theory of SB RAS Lermontov str., 134 664033, Irkutsk,
More informationWhich Separator? Spring 1
Whch Separator? 6.034 - Sprng 1 Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng 3 Margn of a pont " # y (w $ + b) proportonal
More informationLecture 20: November 7
0-725/36-725: Convex Optmzaton Fall 205 Lecturer: Ryan Tbshran Lecture 20: November 7 Scrbes: Varsha Chnnaobreddy, Joon Sk Km, Lngyao Zhang Note: LaTeX template courtesy of UC Berkeley EECS dept. Dsclamer:
More information36.1 Why is it important to be able to find roots to systems of equations? Up to this point, we have discussed how to find the solution to
ChE Lecture Notes - D. Keer, 5/9/98 Lecture 6,7,8 - Rootndng n systems o equatons (A) Theory (B) Problems (C) MATLAB Applcatons Tet: Supplementary notes rom Instructor 6. Why s t mportant to be able to
More informationNatural Language Processing and Information Retrieval
Natural Language Processng and Informaton Retreval Support Vector Machnes Alessandro Moschtt Department of nformaton and communcaton technology Unversty of Trento Emal: moschtt@ds.untn.t Summary Support
More informationCISE301: Numerical Methods Topic 2: Solution of Nonlinear Equations
CISE3: Numercal Methods Topc : Soluton o Nonlnear Equatons Dr. Amar Khoukh Term Read Chapters 5 and 6 o the tetbook CISE3_Topc c Khoukh_ Lecture 5 Soluton o Nonlnear Equatons Root ndng Problems Dentons
More informationInteractive Bi-Level Multi-Objective Integer. Non-linear Programming Problem
Appled Mathematcal Scences Vol 5 0 no 65 3 33 Interactve B-Level Mult-Objectve Integer Non-lnear Programmng Problem O E Emam Department of Informaton Systems aculty of Computer Scence and nformaton Helwan
More informationGeneralized Linear Methods
Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set
More informationSolutions HW #2. minimize. Ax = b. Give the dual problem, and make the implicit equality constraints explicit. Solution.
Solutons HW #2 Dual of general LP. Fnd the dual functon of the LP mnmze subject to c T x Gx h Ax = b. Gve the dual problem, and make the mplct equalty constrants explct. Soluton. 1. The Lagrangan s L(x,
More informationSupport Vector Machines. Jie Tang Knowledge Engineering Group Department of Computer Science and Technology Tsinghua University 2012
Support Vector Machnes Je Tang Knowledge Engneerng Group Department of Computer Scence and Technology Tsnghua Unversty 2012 1 Outlne What s a Support Vector Machne? Solvng SVMs Kernel Trcks 2 What s a
More informationAPPENDIX A Some Linear Algebra
APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,
More informationcan be decomposed into r augmenting cycles and the sum of the costs of these cycles equals, c . But since is optimum, we must have c
Ameran Internatonal Journal of Researh n Sene, Tehnology, Engneerng & Mathemats Avalable onlne at http://www.asr.net ISSN (Prnt): 8-9, ISSN (Onlne): 8-8, ISSN (CD-ROM): 8-69 AIJRSTEM s a refereed, ndexed,
More informationA MODIFIED METHOD FOR SOLVING SYSTEM OF NONLINEAR EQUATIONS
Journal of Mathematcs and Statstcs 9 (1): 4-8, 1 ISSN 1549-644 1 Scence Publcatons do:1.844/jmssp.1.4.8 Publshed Onlne 9 (1) 1 (http://www.thescpub.com/jmss.toc) A MODIFIED METHOD FOR SOLVING SYSTEM OF
More informationOptimal Power Flow by a Primal-Dual Interior Point Method
Optmal ower Flow y a rmal-dual Interor ont Method Amnuay augnm and Somporn Srsumrannuul Astrat One of the most mportant requrements n power system operaton, ontrol and plannng n energy management system
More informationSupport Vector Machines
CS 2750: Machne Learnng Support Vector Machnes Prof. Adrana Kovashka Unversty of Pttsburgh February 17, 2016 Announcement Homework 2 deadlne s now 2/29 We ll have covered everythng you need today or at
More informationOutline. Clustering: Similarity-Based Clustering. Supervised Learning vs. Unsupervised Learning. Clustering. Applications of Clustering
Clusterng: Smlarty-Based Clusterng CS4780/5780 Mahne Learnng Fall 2013 Thorsten Joahms Cornell Unversty Supervsed vs. Unsupervsed Learnng Herarhal Clusterng Herarhal Agglomeratve Clusterng (HAC) Non-Herarhal
More informationIV. Performance Optimization
IV. Performance Optmzaton A. Steepest descent algorthm defnton how to set up bounds on learnng rate mnmzaton n a lne (varyng learnng rate) momentum learnng examples B. Newton s method defnton Gauss-Newton
More informationSummary with Examples for Root finding Methods -Bisection -Newton Raphson -Secant
Summary wth Eamples or Root ndng Methods -Bsecton -Newton Raphson -Secant Nonlnear Equaton Solvers Bracketng Graphcal Open Methods Bsecton False Poston (Regula-Fals) Newton Raphson Secant All Iteratve
More informationADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING
1 ADVANCED ACHINE LEARNING ADVANCED ACHINE LEARNING Non-lnear regresson technques 2 ADVANCED ACHINE LEARNING Regresson: Prncple N ap N-dm. nput x to a contnuous output y. Learn a functon of the type: N
More informationSolution of Linear System of Equations and Matrix Inversion Gauss Seidel Iteration Method
Soluton of Lnear System of Equatons and Matr Inverson Gauss Sedel Iteraton Method It s another well-known teratve method for solvng a system of lnear equatons of the form a + a22 + + ann = b a2 + a222
More informationSome Comments on Accelerating Convergence of Iterative Sequences Using Direct Inversion of the Iterative Subspace (DIIS)
Some Comments on Acceleratng Convergence of Iteratve Sequences Usng Drect Inverson of the Iteratve Subspace (DIIS) C. Davd Sherrll School of Chemstry and Bochemstry Georga Insttute of Technology May 1998
More informationSequential Convex Programming for Free Material Optimization with Displacement and Stress Constraints
FP 6 STREP 30717 PLATO N (Aeronautcs and Space) PLATO N A PLAtform for Topology Optmsaton ncorporatng Novel, Large-Scale, Free-Materal Optmsaton and Mxed Integer Programmng Methods Sequental Convex Programmng
More informationLinear Feature Engineering 11
Lnear Feature Engneerng 11 2 Least-Squares 2.1 Smple least-squares Consder the followng dataset. We have a bunch of nputs x and correspondng outputs y. The partcular values n ths dataset are x y 0.23 0.19
More informationSELECTED SOLUTIONS, SECTION (Weak duality) Prove that the primal and dual values p and d defined by equations (4.3.2) and (4.3.3) satisfy p d.
SELECTED SOLUTIONS, SECTION 4.3 1. Weak dualty Prove that the prmal and dual values p and d defned by equatons 4.3. and 4.3.3 satsfy p d. We consder an optmzaton problem of the form The Lagrangan for ths
More informationHomotopy Continuation Solution Method in Nonlinear Model Predictive Control Applications
Ian Davd Lockhart Bogle and Mchael Farweather (Edtors Proceedngs of the nd European Symposum on Computer Aded Process Engneerng 7 - June London. Elsever B.V. All rghts reserved Homotopy Contnuaton Soluton
More informationSimultaneous Optimization of Berth Allocation, Quay Crane Assignment and Quay Crane Scheduling Problems in Container Terminals
Smultaneous Optmzaton of Berth Allocaton, Quay Crane Assgnment and Quay Crane Schedulng Problems n Contaner Termnals Necat Aras, Yavuz Türkoğulları, Z. Caner Taşkın, Kuban Altınel Abstract In ths work,
More informationA nonlinear programming model with implicit variables for packing ellipsoids
A nonlnear programmng model wth mplct varables for packng ellpsods E. G. Brgn R. D. Lobato J. M. Martínez March 23, 2016 Abstract The problem of packng ellpsods s consdered n the present work. Usually,
More informationOn a direct solver for linear least squares problems
ISSN 2066-6594 Ann. Acad. Rom. Sc. Ser. Math. Appl. Vol. 8, No. 2/2016 On a drect solver for lnear least squares problems Constantn Popa Abstract The Null Space (NS) algorthm s a drect solver for lnear
More informationLecture 26 Finite Differences and Boundary Value Problems
4//3 Leture 6 Fnte erenes and Boundar Value Problems Numeral derentaton A nte derene s an appromaton o a dervatve - eample erved rom Talor seres 3 O! Negletng all terms ger tan rst order O O Tat s te orward
More informationMACHINE APPLIED MACHINE LEARNING LEARNING. Gaussian Mixture Regression
11 MACHINE APPLIED MACHINE LEARNING LEARNING MACHINE LEARNING Gaussan Mture Regresson 22 MACHINE APPLIED MACHINE LEARNING LEARNING Bref summary of last week s lecture 33 MACHINE APPLIED MACHINE LEARNING
More informationVector Norms. Chapter 7 Iterative Techniques in Matrix Algebra. Cauchy-Bunyakovsky-Schwarz Inequality for Sums. Distances. Convergence.
Vector Norms Chapter 7 Iteratve Technques n Matrx Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematcs Unversty of Calforna, Berkeley Math 128B Numercal Analyss Defnton A vector norm
More informationLecture 14: Bandits with Budget Constraints
IEOR 8100-001: Learnng and Optmzaton for Sequental Decson Makng 03/07/16 Lecture 14: andts wth udget Constrants Instructor: Shpra Agrawal Scrbed by: Zhpeng Lu 1 Problem defnton In the regular Mult-armed
More informationThe Minimum Universal Cost Flow in an Infeasible Flow Network
Journal of Scences, Islamc Republc of Iran 17(2): 175-180 (2006) Unversty of Tehran, ISSN 1016-1104 http://jscencesutacr The Mnmum Unversal Cost Flow n an Infeasble Flow Network H Saleh Fathabad * M Bagheran
More informationNewton-KKT interior-point methods for indefinite quadratic programming
Comput Optm Applc (2007) 36:5 41 DOI 10.1007/s10589-006-8717-1 Newton-KKT nteror-pont methods for ndefnte quadratc programmng P.-A. Absl AndréL.Tts Receved: 29 September 2004 / Revsed: 5 August 2005 /
More informationTopic 5: Non-Linear Regression
Topc 5: Non-Lnear Regresson The models we ve worked wth so far have been lnear n the parameters. They ve been of the form: y = Xβ + ε Many models based on economc theory are actually non-lnear n the parameters.
More informationSupport Vector Machines CS434
Support Vector Machnes CS434 Lnear Separators Many lnear separators exst that perfectly classfy all tranng examples Whch of the lnear separators s the best? + + + + + + + + + Intuton of Margn Consder ponts
More informationPattern Classification
Pattern Classfcaton All materals n these sldes ere taken from Pattern Classfcaton (nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wley & Sons, 000 th the permsson of the authors and the publsher
More informationOutline and Reading. Dynamic Programming. Dynamic Programming revealed. Computing Fibonacci. The General Dynamic Programming Technique
Outlne and Readng Dynamc Programmng The General Technque ( 5.3.2) -1 Knapsac Problem ( 5.3.3) Matrx Chan-Product ( 5.3.1) Dynamc Programmng verson 1.4 1 Dynamc Programmng verson 1.4 2 Dynamc Programmng
More informationSupport Vector Machines. Vibhav Gogate The University of Texas at dallas
Support Vector Machnes Vbhav Gogate he Unversty of exas at dallas What We have Learned So Far? 1. Decson rees. Naïve Bayes 3. Lnear Regresson 4. Logstc Regresson 5. Perceptron 6. Neural networks 7. K-Nearest
More informationSupplement: Proofs and Technical Details for The Solution Path of the Generalized Lasso
Supplement: Proofs and Techncal Detals for The Soluton Path of the Generalzed Lasso Ryan J. Tbshran Jonathan Taylor In ths document we gve supplementary detals to the paper The Soluton Path of the Generalzed
More information14 Lagrange Multipliers
Lagrange Multplers 14 Lagrange Multplers The Method of Lagrange Multplers s a powerful technque for constraned optmzaton. Whle t has applcatons far beyond machne learnng t was orgnally developed to solve
More informationCalculation of time complexity (3%)
Problem 1. (30%) Calculaton of tme complexty (3%) Gven n ctes, usng exhaust search to see every result takes O(n!). Calculaton of tme needed to solve the problem (2%) 40 ctes:40! dfferent tours 40 add
More informationLecture 20: Lift and Project, SDP Duality. Today we will study the Lift and Project method. Then we will prove the SDP duality theorem.
prnceton u. sp 02 cos 598B: algorthms and complexty Lecture 20: Lft and Project, SDP Dualty Lecturer: Sanjeev Arora Scrbe:Yury Makarychev Today we wll study the Lft and Project method. Then we wll prove
More informationAn Interactive Optimisation Tool for Allocation Problems
An Interactve Optmsaton ool for Allocaton Problems Fredr Bonäs, Joam Westerlund and apo Westerlund Process Desgn Laboratory, Faculty of echnology, Åbo Aadem Unversty, uru 20500, Fnland hs paper presents
More informationOn the Multicriteria Integer Network Flow Problem
BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 5, No 2 Sofa 2005 On the Multcrtera Integer Network Flow Problem Vassl Vasslev, Marana Nkolova, Maryana Vassleva Insttute of
More informationPHYS 705: Classical Mechanics. Calculus of Variations II
1 PHYS 705: Classcal Mechancs Calculus of Varatons II 2 Calculus of Varatons: Generalzaton (no constrant yet) Suppose now that F depends on several dependent varables : We need to fnd such that has a statonary
More informationDistributed and Stochastic Machine Learning on Big Data
Dstrbuted and Stochastc Machne Learnng on Bg Data Department of Computer Scence and Engneerng Hong Kong Unversty of Scence and Technology Hong Kong Introducton Synchronous ADMM Asynchronous ADMM Stochastc
More informationEEL 6266 Power System Operation and Control. Chapter 3 Economic Dispatch Using Dynamic Programming
EEL 6266 Power System Operaton and Control Chapter 3 Economc Dspatch Usng Dynamc Programmng Pecewse Lnear Cost Functons Common practce many utltes prefer to represent ther generator cost functons as sngle-
More informationCIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M
CIS56: achne Learnng Lecture 3 (Sept 6, 003) Preparaton help: Xaoyng Huang Lnear Regresson Lnear regresson can be represented by a functonal form: f(; θ) = θ 0 0 +θ + + θ = θ = 0 ote: 0 s a dummy attrbute
More information3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X
Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number
More informationMLE and Bayesian Estimation. Jie Tang Department of Computer Science & Technology Tsinghua University 2012
MLE and Bayesan Estmaton Je Tang Department of Computer Scence & Technology Tsnghua Unversty 01 1 Lnear Regresson? As the frst step, we need to decde how we re gong to represent the functon f. One example:
More informationEn Route Traffic Optimization to Reduce Environmental Impact
En Route Traffc Optmzaton to Reduce Envronmental Impact John-Paul Clarke Assocate Professor of Aerospace Engneerng Drector of the Ar Transportaton Laboratory Georga Insttute of Technology Outlne 1. Introducton
More information( ) [ ( k) ( k) ( x) ( ) ( ) ( ) [ ] ξ [ ] [ ] [ ] ( )( ) i ( ) ( )( ) 2! ( ) = ( ) 3 Interpolation. Polynomial Approximation.
3 Interpolaton {( y } Gven:,,,,,, [ ] Fnd: y for some Mn, Ma Polynomal Appromaton Theorem (Weerstrass Appromaton Theorem --- estence ε [ ab] f( P( , then there ests a polynomal
More informationVery Large Scale Continuous and Discrete Variable. Woptimization,
Very Large Scale Contnuous and Dscrete Varable Optmzaton Garret N. Vanderplaats * Vanderplaats Research & Development, Inc. 1767 S. 8 th Street Colorado Sprngs, CO 80906 An optmzaton algorthm s presented
More informationReview of Taylor Series. Read Section 1.2
Revew of Taylor Seres Read Secton 1.2 1 Power Seres A power seres about c s an nfnte seres of the form k = 0 k a ( x c) = a + a ( x c) + a ( x c) + a ( x c) k 2 3 0 1 2 3 + In many cases, c = 0, and the
More informationGlobal Optimization of Truss. Structure Design INFORMS J. N. Hooker. Tallys Yunes. Slide 1
Slde 1 Global Optmzaton of Truss Structure Desgn J. N. Hooker Tallys Yunes INFORMS 2010 Truss Structure Desgn Select sze of each bar (possbly zero) to support the load whle mnmzng weght. Bar szes are dscrete.
More informationFIRST AND SECOND ORDER NECESSARY OPTIMALITY CONDITIONS FOR DISCRETE OPTIMAL CONTROL PROBLEMS
Yugoslav Journal of Operatons Research 6 (6), umber, 53-6 FIRST D SECOD ORDER ECESSRY OPTIMLITY CODITIOS FOR DISCRETE OPTIML COTROL PROBLEMS Boban MRIKOVIĆ Faculty of Mnng and Geology, Unversty of Belgrade
More informationMultilayer Perceptron (MLP)
Multlayer Perceptron (MLP) Seungjn Cho Department of Computer Scence and Engneerng Pohang Unversty of Scence and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjn@postech.ac.kr 1 / 20 Outlne
More informationON DISTRIBUTED OPTIMIZATION METHODS FOR SOLVING OPTIMAL POWER FLOW PROBLEM OVER ELECTRICITY GRIDS
The Pennsylvana State Unversty The Graduate School Department of Industral and Manufacturng Engneerng ON DISTRIBUTED OPTIMIZATION METHODS FOR SOLVING OPTIMAL POWER FLOW PROBLEM OVER ELECTRICITY GRIDS A
More informationClustering. CS4780/5780 Machine Learning Fall Thorsten Joachims Cornell University
Clusterng CS4780/5780 Mahne Learnng Fall 2012 Thorsten Joahms Cornell Unversty Readng: Mannng/Raghavan/Shuetze, Chapters 16 (not 16.3) and 17 (http://nlp.stanford.edu/ir-book/) Outlne Supervsed vs. Unsupervsed
More informationOuter Trust-Region method for Constrained Optimization
Outer Trust-Regon method for Constraned Optmzaton Ernesto G. Brgn Emerson V. Castelan André L. M. Martnez J. M. Martínez May 18, 2009 Abstract Gven an algorthm A for solvng some mathematcal problem based
More informationAdditional Codes using Finite Difference Method. 1 HJB Equation for Consumption-Saving Problem Without Uncertainty
Addtonal Codes usng Fnte Dfference Method Benamn Moll 1 HJB Equaton for Consumpton-Savng Problem Wthout Uncertanty Before consderng the case wth stochastc ncome n http://www.prnceton.edu/~moll/ HACTproect/HACT_Numercal_Appendx.pdf,
More informationDynamic Programming. Preview. Dynamic Programming. Dynamic Programming. Dynamic Programming (Example: Fibonacci Sequence)
/24/27 Prevew Fbonacc Sequence Longest Common Subsequence Dynamc programmng s a method for solvng complex problems by breakng them down nto smpler sub-problems. It s applcable to problems exhbtng the propertes
More informationVariance of Estimates in Dynamic Data Reconciliation
9 th Eropean Sypos on Copter Aded Proess Engneerng ESCAPE9 J. Jeżowsk and J. hlle (Edtors) 2009 Elsever B.V./Ltd. All rghts reserved. Varane of Estates n Dyna Data Reonlaton Chrstophe Ullrh, Georges Heyen
More informationVoltammetry. Bulk electrolysis: relatively large electrodes (on the order of cm 2 ) Voltammetry:
Voltammetry varety of eletroanalytal methods rely on the applaton of a potental funton to an eletrode wth the measurement of the resultng urrent n the ell. In ontrast wth bul eletrolyss methods, the objetve
More informationInterconnect Modeling
Interconnect Modelng Modelng of Interconnects Interconnect R, C and computaton Interconnect models umped RC model Dstrbuted crcut models Hgher-order waveform n dstrbuted RC trees Accuracy and fdelty Prepared
More informationOptimality properties of an Augmented Lagrangian method on infeasible problems
Optmalty propertes of an Augmented Lagrangan method on nfeasble problems E. G. Brgn J. M. Martínez L. F. Prudente March 7, 014 Abstract Sometmes, the feasble set of an optmzaton problem that one ams to
More informatione a = 12.4 i a = 13.5i h a = xi + yj 3 a Let r a = 25cos(20) i + 25sin(20) j b = 15cos(55) i + 15sin(55) j
Vetors MC Qld-3 49 Chapter 3 Vetors Exerse 3A Revew of vetors a d e f e a x + y omponent: x a os(θ 6 os(80 + 39 6 os(9.4 omponent: y a sn(θ 6 sn(9 0. a.4 0. f a x + y omponent: x a os(θ 5 os( 5 3.6 omponent:
More informationAugmented Lagrangians with possible infeasibility and finite termination for global nonlinear programming
Augmented Lagrangans wth possble nfeasblty and fnte termnaton for global nonlnear programmng E. G. Brgn J. M. Martínez L. F. Prudente June 5, 01 Abstract In a recent paper, Brgn, Floudas and Martínez ntroduced
More informationEconomics 101. Lecture 4 - Equilibrium and Efficiency
Economcs 0 Lecture 4 - Equlbrum and Effcency Intro As dscussed n the prevous lecture, we wll now move from an envronment where we looed at consumers mang decsons n solaton to analyzng economes full of
More information