Perturbation Analysis of Optimization Problems

Size: px
Start display at page:

Download "Perturbation Analysis of Optimization Problems"

Transcription

1 Perturbation Analysis of Optimization Problems J. Frédéric Bonnans 1 and Alexander Shapiro 2 1 INRIA-Rocquencourt, Domaine de Voluceau, B.P. 105, Rocquencourt, France, and Ecole Polytechnique, France 2 School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia , USA

2 2

3 Contents 1 Introduction 3 2 Background Material Basic Functional Analysis Topological Vector Spaces The Hahn-Banach Theorem Banach Spaces Cones, Duality and Recession Cones Directional Differentiability and Tangent Cones First Order Directional Derivatives Second Order Derivatives Directional Epiderivatives of Extended Real Valued Functions Tangent Cones Elements of Multifunctions Theory The Generalized Open Mapping Theorem Openness, Stability and Metric Regularity Stability of Nonlinear Constrained Systems Constraint Qualification Conditions Convex Mappings Convex Functions Continuity Conjugacy Subdifferentiability Chain Rules Duality Theory Conjugate Duality Lagrangian duality Examples and Applications of Duality Schemes Applications to Subdifferential Calculus Minimization of a Maximum over a Compact Set Conic Linear Problems Generalized Linear Programming and Polyhedral Multifunctions

4 4 CONTENTS 3 Optimality Conditions First Order Optimality Conditions Lagrange Multipliers Generalized Lagrange Multipliers Ekeland s Variational Principle First Order Sufficient Conditions Second Order Necessary Conditions Second Order Tangent Sets General Form of Second Order Necessary Conditions Extended Polyhedricity Second Order Sufficient Conditions General Form of Second Order Sufficient Conditions Quadratic and Extended Legendre Forms Second Order Regularity of Sets and No Gap Second Order Optimality Conditions Second Order Regularity of Functions Second Order Subderivatives Specific Structures Composite Optimization Exact Penalty Functions and Augmented Duality Linear Constraints and Quadratic Programming A Reduction Approach Nonisolated Minima Necessary Conditions for Quadratic Growth Sufficient Conditions Sufficient Conditions Based on General Critical Directions Stability and Sensitivity Analysis Stability of the Optimal Value and Optimal Solutions Directional Regularity First Order Differentiability Analysis of the Optimal Value Function The Case of Fixed Feasible Set Directional Differentiability of the Optimal Value Function Under Abstract Constraints Quantitative Stability of Optimal Solutions and Lagrange Multipliers Lipschitzian Stability in the Case of a Fixed Feasible Set Hölder Stability Under Abstract Constraints Quantitative Stability of Lagrange Multipliers Lipschitzian Stability of Optimal Solutions and Lagrange Multipliers Directional Stability of Optimal Solutions Hölder Directional Stability Lipschitzian Directional Stability

5 CONTENTS Quantitative Stability Analysis by a Reduction Approach Nondegeneracy and Strict Complementarity Stability Analysis Second Order Analysis in Lipschitz Stable Cases Upper Second Order Estimates of the Optimal Value Function Lower Estimates Without the Sigma Term The Second Order Regular Case Composite Optimization Problems Second Order Analysis in Hölder Stable Cases Upper Second Order Estimates of the Optimal Value Function Lower Estimates and Expansions of Optimal Solutions Empty Sets of Lagrange Multipliers Hölder Expansions for Second Order Regular Problems Additional Results Equality Constrained Problems Uniform Approximations of the Optimal Value and Optimal Solutions Second Order Analysis for Nonisolated Optima Second Order Analysis in Functional Spaces Second Order Tangent Sets in Functional Spaces of Continuous Functions Second Order Derivatives of Optimal Value Functions Second Order Expansions in Functional Spaces Additional Material and Applications Variational Inequalities Standard Variational Inequalities Generalized Equations Strong Regularity Strong Regularity and Second Order Optimality Conditions Strong Stability Some Examples and Applications Nonlinear Programming Finite Dimensional Linear Programs Optimality Conditions for Nonlinear Programs Lipschitz Expansions of Optimal Solutions Hölder Expansion of Optimal Solutions High Order Expansions of Optimal Solutions and Lagrange Multipliers Electrical Networks The Chain Problem Semi-definite Programming Geometry of the Cones of Negative Semidefinite Matrices. 497

6 CONTENTS Matrix Convexity Duality First Order Optimality Conditions Second Order Optimality Conditions Stability and Sensitivity Analysis Semi-infinite Programming Duality First Order Optimality Conditions Second Order Optimality Conditions Perturbation Analysis Optimal Control Introduction Linear and Semilinear Elliptic Equations The Dirichlet Problem Semilinear Elliptic Equations Strong Solutions Optimal Control of a Semilinear Elliptic Equation Existence of Solutions, First Order Optimality System Second Order Necessary or Sufficient Conditions Some Specific Control Constraints Sensitivity Analysis State Constrained Optimal Control Problem Optimal Control of an Ill-Posed System The Obstacle Problem Presentation of the Problem Polyhedricity Basic Capacity Theory Sensitivity Analysis and Optimal Control Bibliographical Notes Background Material Optimality Conditions Stability and Sensitivity Analysis Applications Variational Inequalities Nonlinear Programming Semi-definite Programming Semi-infinite Programming Optimal Control

Some Properties of the Augmented Lagrangian in Cone Constrained Optimization

Some Properties of the Augmented Lagrangian in Cone Constrained Optimization MATHEMATICS OF OPERATIONS RESEARCH Vol. 29, No. 3, August 2004, pp. 479 491 issn 0364-765X eissn 1526-5471 04 2903 0479 informs doi 10.1287/moor.1040.0103 2004 INFORMS Some Properties of the Augmented

More information

1. Introduction. Consider the following parameterized optimization problem:

1. Introduction. Consider the following parameterized optimization problem: SIAM J. OPTIM. c 1998 Society for Industrial and Applied Mathematics Vol. 8, No. 4, pp. 940 946, November 1998 004 NONDEGENERACY AND QUANTITATIVE STABILITY OF PARAMETERIZED OPTIMIZATION PROBLEMS WITH MULTIPLE

More information

ON A CLASS OF NONSMOOTH COMPOSITE FUNCTIONS

ON A CLASS OF NONSMOOTH COMPOSITE FUNCTIONS MATHEMATICS OF OPERATIONS RESEARCH Vol. 28, No. 4, November 2003, pp. 677 692 Printed in U.S.A. ON A CLASS OF NONSMOOTH COMPOSITE FUNCTIONS ALEXANDER SHAPIRO We discuss in this paper a class of nonsmooth

More information

DUALITY, OPTIMALITY CONDITIONS AND PERTURBATION ANALYSIS

DUALITY, OPTIMALITY CONDITIONS AND PERTURBATION ANALYSIS 1 DUALITY, OPTIMALITY CONDITIONS AND PERTURBATION ANALYSIS Alexander Shapiro 1 School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332-0205, USA, E-mail: ashapiro@isye.gatech.edu

More information

Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem

Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 0-0 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R

More information

On duality theory of conic linear problems

On duality theory of conic linear problems On duality theory of conic linear problems Alexander Shapiro School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 3332-25, USA e-mail: ashapiro@isye.gatech.edu

More information

Semi-infinite programming, duality, discretization and optimality conditions

Semi-infinite programming, duality, discretization and optimality conditions Semi-infinite programming, duality, discretization and optimality conditions Alexander Shapiro School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332-0205,

More information

In view of (31), the second of these is equal to the identity I on E m, while this, in view of (30), implies that the first can be written

In view of (31), the second of these is equal to the identity I on E m, while this, in view of (30), implies that the first can be written 11.8 Inequality Constraints 341 Because by assumption x is a regular point and L x is positive definite on M, it follows that this matrix is nonsingular (see Exercise 11). Thus, by the Implicit Function

More information

NONSMOOTH ANALYSIS AND PARAMETRIC OPTIMIZATION. R. T. Rockafellar*

NONSMOOTH ANALYSIS AND PARAMETRIC OPTIMIZATION. R. T. Rockafellar* NONSMOOTH ANALYSIS AND PARAMETRIC OPTIMIZATION R. T. Rockafellar* Abstract. In an optimization problem that depends on parameters, an important issue is the effect that perturbations of the parameters

More information

Additional Homework Problems

Additional Homework Problems Additional Homework Problems Robert M. Freund April, 2004 2004 Massachusetts Institute of Technology. 1 2 1 Exercises 1. Let IR n + denote the nonnegative orthant, namely IR + n = {x IR n x j ( ) 0,j =1,...,n}.

More information

II KLUWER ACADEMIC PUBLISHERS. Abstract Convexity and Global Optimization. Alexander Rubinov

II KLUWER ACADEMIC PUBLISHERS. Abstract Convexity and Global Optimization. Alexander Rubinov Abstract Convexity and Global Optimization by Alexander Rubinov School of Information Technology and Mathematical Sciences, University of Ballarat, Victoria, Australia II KLUWER ACADEMIC PUBLISHERS DORDRECHT

More information

Nonlinear Functional Analysis and its Applications

Nonlinear Functional Analysis and its Applications Eberhard Zeidler Nonlinear Functional Analysis and its Applications III: Variational Methods and Optimization Translated by Leo F. Boron With 111 Illustrations Ш Springer-Verlag New York Berlin Heidelberg

More information

EE 227A: Convex Optimization and Applications October 14, 2008

EE 227A: Convex Optimization and Applications October 14, 2008 EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider

More information

Constrained Optimization Theory

Constrained Optimization Theory Constrained Optimization Theory Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. IMA, August 2016 Stephen Wright (UW-Madison) Constrained Optimization Theory IMA, August

More information

LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE

LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization

More information

CONVEX OPTIMIZATION VIA LINEARIZATION. Miguel A. Goberna. Universidad de Alicante. Iberian Conference on Optimization Coimbra, November, 2006

CONVEX OPTIMIZATION VIA LINEARIZATION. Miguel A. Goberna. Universidad de Alicante. Iberian Conference on Optimization Coimbra, November, 2006 CONVEX OPTIMIZATION VIA LINEARIZATION Miguel A. Goberna Universidad de Alicante Iberian Conference on Optimization Coimbra, 16-18 November, 2006 Notation X denotes a l.c. Hausdorff t.v.s and X its topological

More information

ANALYSIS IN SOBOLEV AND BV SPACES

ANALYSIS IN SOBOLEV AND BV SPACES ж ш Л Fi I /V T I Jf"\ Ik 1 Л 1 ANALYSIS IN SOBOLEV AND BV SPACES APPLICATIONS TO PDES AND OPTIMIZATION Hedy Attouch Universite Montpellier I! Montpellier, France Giuseppe Buttazzo Universitä di Pisa Pisa,

More information

Mathematical programs with complementarity constraints in Banach spaces

Mathematical programs with complementarity constraints in Banach spaces Mathematical programs with complementarity constraints in Banach spaces Gerd Wachsmuth July 21, 2014 We consider optimization problems in Banach spaces involving a complementarity constraint defined by

More information

TMA 4180 Optimeringsteori KARUSH-KUHN-TUCKER THEOREM

TMA 4180 Optimeringsteori KARUSH-KUHN-TUCKER THEOREM TMA 4180 Optimeringsteori KARUSH-KUHN-TUCKER THEOREM H. E. Krogstad, IMF, Spring 2012 Karush-Kuhn-Tucker (KKT) Theorem is the most central theorem in constrained optimization, and since the proof is scattered

More information

Convex Optimization Theory. Chapter 5 Exercises and Solutions: Extended Version

Convex Optimization Theory. Chapter 5 Exercises and Solutions: Extended Version Convex Optimization Theory Chapter 5 Exercises and Solutions: Extended Version Dimitri P. Bertsekas Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts http://www.athenasc.com

More information

STUDY PLAN MASTER IN (MATHEMATICS) (Thesis Track)

STUDY PLAN MASTER IN (MATHEMATICS) (Thesis Track) STUDY PLAN MASTER IN (MATHEMATICS) (Thesis Track) I. GENERAL RULES AND CONDITIONS: 1- This plan conforms to the regulations of the general frame of the Master programs. 2- Areas of specialty of admission

More information

GEORGIA INSTITUTE OF TECHNOLOGY H. MILTON STEWART SCHOOL OF INDUSTRIAL AND SYSTEMS ENGINEERING LECTURE NOTES OPTIMIZATION III

GEORGIA INSTITUTE OF TECHNOLOGY H. MILTON STEWART SCHOOL OF INDUSTRIAL AND SYSTEMS ENGINEERING LECTURE NOTES OPTIMIZATION III GEORGIA INSTITUTE OF TECHNOLOGY H. MILTON STEWART SCHOOL OF INDUSTRIAL AND SYSTEMS ENGINEERING LECTURE NOTES OPTIMIZATION III CONVEX ANALYSIS NONLINEAR PROGRAMMING THEORY NONLINEAR PROGRAMMING ALGORITHMS

More information

CHARACTERIZATIONS OF LIPSCHITZIAN STABILITY

CHARACTERIZATIONS OF LIPSCHITZIAN STABILITY CHARACTERIZATIONS OF LIPSCHITZIAN STABILITY IN NONLINEAR PROGRAMMING 1 A. L. DONTCHEV Mathematical Reviews, Ann Arbor, MI 48107 and R. T. ROCKAFELLAR Dept. of Math., Univ. of Washington, Seattle, WA 98195

More information

Reading group: Calculus of Variations and Optimal Control Theory by Daniel Liberzon

Reading group: Calculus of Variations and Optimal Control Theory by Daniel Liberzon : Calculus of Variations and Optimal Control Theory by Daniel Liberzon 16th March 2017 1 / 30 Content 1 2 Recall on finite-dimensional of a global minimum 3 Infinite-dimensional 4 2 / 30 Content 1 2 Recall

More information

Second-order optimality conditions for state-constrained optimal control problems

Second-order optimality conditions for state-constrained optimal control problems Second-order optimality conditions for state-constrained optimal control problems J. Frédéric Bonnans INRIA-Saclay & CMAP, Ecole Polytechnique, France Joint work with Audrey Hermant, INRIA-Saclay & CMAP

More information

ON LICQ AND THE UNIQUENESS OF LAGRANGE MULTIPLIERS

ON LICQ AND THE UNIQUENESS OF LAGRANGE MULTIPLIERS ON LICQ AND THE UNIQUENESS OF LAGRANGE MULTIPLIERS GERD WACHSMUTH Abstract. Kyparisis proved in 1985 that a strict version of the Mangasarian- Fromovitz constraint qualification (MFCQ) is equivalent to

More information

I.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010

I.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010 I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0

More information

M E M O R A N D U M. Faculty Senate approved November 1, 2018

M E M O R A N D U M. Faculty Senate approved November 1, 2018 M E M O R A N D U M Faculty Senate approved November 1, 2018 TO: FROM: Deans and Chairs Becky Bitter, Sr. Assistant Registrar DATE: October 23, 2018 SUBJECT: Minor Change Bulletin No. 5 The courses listed

More information

Advanced Continuous Optimization

Advanced Continuous Optimization University Paris-Saclay Master Program in Optimization Advanced Continuous Optimization J. Ch. Gilbert (INRIA Paris-Rocquencourt) September 26, 2017 Lectures: September 18, 2017 November 6, 2017 Examination:

More information

Generalization to inequality constrained problem. Maximize

Generalization to inequality constrained problem. Maximize Lecture 11. 26 September 2006 Review of Lecture #10: Second order optimality conditions necessary condition, sufficient condition. If the necessary condition is violated the point cannot be a local minimum

More information

Characterizing Robust Solution Sets of Convex Programs under Data Uncertainty

Characterizing Robust Solution Sets of Convex Programs under Data Uncertainty Characterizing Robust Solution Sets of Convex Programs under Data Uncertainty V. Jeyakumar, G. M. Lee and G. Li Communicated by Sándor Zoltán Németh Abstract This paper deals with convex optimization problems

More information

Structural and Multidisciplinary Optimization. P. Duysinx and P. Tossings

Structural and Multidisciplinary Optimization. P. Duysinx and P. Tossings Structural and Multidisciplinary Optimization P. Duysinx and P. Tossings 2018-2019 CONTACTS Pierre Duysinx Institut de Mécanique et du Génie Civil (B52/3) Phone number: 04/366.91.94 Email: P.Duysinx@uliege.be

More information

MATHEMATICS FOR ECONOMISTS. An Introductory Textbook. Third Edition. Malcolm Pemberton and Nicholas Rau. UNIVERSITY OF TORONTO PRESS Toronto Buffalo

MATHEMATICS FOR ECONOMISTS. An Introductory Textbook. Third Edition. Malcolm Pemberton and Nicholas Rau. UNIVERSITY OF TORONTO PRESS Toronto Buffalo MATHEMATICS FOR ECONOMISTS An Introductory Textbook Third Edition Malcolm Pemberton and Nicholas Rau UNIVERSITY OF TORONTO PRESS Toronto Buffalo Contents Preface Dependence of Chapters Answers and Solutions

More information

Conditioning of linear-quadratic two-stage stochastic programming problems

Conditioning of linear-quadratic two-stage stochastic programming problems Conditioning of linear-quadratic two-stage stochastic programming problems W. Römisch Humboldt-University Berlin Institute of Mathematics http://www.math.hu-berlin.de/~romisch (K. Emich, R. Henrion) (WIAS

More information

The Relation Between Pseudonormality and Quasiregularity in Constrained Optimization 1

The Relation Between Pseudonormality and Quasiregularity in Constrained Optimization 1 October 2003 The Relation Between Pseudonormality and Quasiregularity in Constrained Optimization 1 by Asuman E. Ozdaglar and Dimitri P. Bertsekas 2 Abstract We consider optimization problems with equality,

More information

A Dual Condition for the Convex Subdifferential Sum Formula with Applications

A Dual Condition for the Convex Subdifferential Sum Formula with Applications Journal of Convex Analysis Volume 12 (2005), No. 2, 279 290 A Dual Condition for the Convex Subdifferential Sum Formula with Applications R. S. Burachik Engenharia de Sistemas e Computacao, COPPE-UFRJ

More information

Quadratic Growth and Stability in Convex Programming Problems with Multiple Solutions

Quadratic Growth and Stability in Convex Programming Problems with Multiple Solutions Journal of Convex Analysis Volume 2 (1995), No.1/2, 41 57 Quadratic Growth and Stability in Convex Programming Problems with Multiple Solutions J. Frédéric Bonnans INRIA-Rocquencourt, Domaine de Voluceau,

More information

Optimality Conditions for Constrained Optimization

Optimality Conditions for Constrained Optimization 72 CHAPTER 7 Optimality Conditions for Constrained Optimization 1. First Order Conditions In this section we consider first order optimality conditions for the constrained problem P : minimize f 0 (x)

More information

w Kluwer Academic Publishers Boston/Dordrecht/London HANDBOOK OF SEMIDEFINITE PROGRAMMING Theory, Algorithms, and Applications

w Kluwer Academic Publishers Boston/Dordrecht/London HANDBOOK OF SEMIDEFINITE PROGRAMMING Theory, Algorithms, and Applications HANDBOOK OF SEMIDEFINITE PROGRAMMING Theory, Algorithms, and Applications Edited by Henry Wolkowicz Department of Combinatorics and Optimization Faculty of Mathematics University of Waterloo Waterloo,

More information

Some Contributions to Convex Infinite-Dimensional Optimization Duality

Some Contributions to Convex Infinite-Dimensional Optimization Duality Some Contributions to Convex Infinite-Dimensional Optimization Duality Marco A. López Alicante University King s College London Strand Campus June 2014 Introduction Consider the convex infinite programming

More information

Duality Uses and Correspondences. Ryan Tibshirani Convex Optimization

Duality Uses and Correspondences. Ryan Tibshirani Convex Optimization Duality Uses and Correspondences Ryan Tibshirani Conve Optimization 10-725 Recall that for the problem Last time: KKT conditions subject to f() h i () 0, i = 1,... m l j () = 0, j = 1,... r the KKT conditions

More information

Corrections and additions for the book Perturbation Analysis of Optimization Problems by J.F. Bonnans and A. Shapiro. Version of March 28, 2013

Corrections and additions for the book Perturbation Analysis of Optimization Problems by J.F. Bonnans and A. Shapiro. Version of March 28, 2013 Corrections and additions for the book Perturbation Analysis of Optimization Problems by J.F. Bonnans and A. Shapiro Version of March 28, 2013 Some typos in the book that we noticed are of trivial nature

More information

Enhanced Fritz John Optimality Conditions and Sensitivity Analysis

Enhanced Fritz John Optimality Conditions and Sensitivity Analysis Enhanced Fritz John Optimality Conditions and Sensitivity Analysis Dimitri P. Bertsekas Laboratory for Information and Decision Systems Massachusetts Institute of Technology March 2016 1 / 27 Constrained

More information

Convex Optimization & Lagrange Duality

Convex Optimization & Lagrange Duality Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT

More information

Support Vector Machines: Maximum Margin Classifiers

Support Vector Machines: Maximum Margin Classifiers Support Vector Machines: Maximum Margin Classifiers Machine Learning and Pattern Recognition: September 16, 2008 Piotr Mirowski Based on slides by Sumit Chopra and Fu-Jie Huang 1 Outline What is behind

More information

NUMERICAL OPTIMIZATION. J. Ch. Gilbert

NUMERICAL OPTIMIZATION. J. Ch. Gilbert NUMERICAL OPTIMIZATION J. Ch. Gilbert Numerical optimization (past) The discipline deals with the classical smooth (nonconvex) problem min {f(x) : c E (x) = 0, c I (x) 0}. Applications: variable added

More information

Sequential Quadratic Programming Method for Nonlinear Second-Order Cone Programming Problems. Hirokazu KATO

Sequential Quadratic Programming Method for Nonlinear Second-Order Cone Programming Problems. Hirokazu KATO Sequential Quadratic Programming Method for Nonlinear Second-Order Cone Programming Problems Guidance Professor Masao FUKUSHIMA Hirokazu KATO 2004 Graduate Course in Department of Applied Mathematics and

More information

Optimality conditions for Equality Constrained Optimization Problems

Optimality conditions for Equality Constrained Optimization Problems International Journal of Mathematics and Statistics Invention (IJMSI) E-ISSN: 2321 4767 P-ISSN: 2321-4759 Volume 4 Issue 4 April. 2016 PP-28-33 Optimality conditions for Equality Constrained Optimization

More information

LECTURE 12 LECTURE OUTLINE. Subgradients Fenchel inequality Sensitivity in constrained optimization Subdifferential calculus Optimality conditions

LECTURE 12 LECTURE OUTLINE. Subgradients Fenchel inequality Sensitivity in constrained optimization Subdifferential calculus Optimality conditions LECTURE 12 LECTURE OUTLINE Subgradients Fenchel inequality Sensitivity in constrained optimization Subdifferential calculus Optimality conditions Reading: Section 5.4 All figures are courtesy of Athena

More information

Convexity, Duality, and Lagrange Multipliers

Convexity, Duality, and Lagrange Multipliers LECTURE NOTES Convexity, Duality, and Lagrange Multipliers Dimitri P. Bertsekas with assistance from Angelia Geary-Nedic and Asuman Koksal Massachusetts Institute of Technology Spring 2001 These notes

More information

CONTENTS. Preface Preliminaries 1

CONTENTS. Preface Preliminaries 1 Preface xi Preliminaries 1 1 TOOLS FOR ANALYSIS 5 1.1 The Completeness Axiom and Some of Its Consequences 5 1.2 The Distribution of the Integers and the Rational Numbers 12 1.3 Inequalities and Identities

More information

Appendix A Taylor Approximations and Definite Matrices

Appendix A Taylor Approximations and Definite Matrices Appendix A Taylor Approximations and Definite Matrices Taylor approximations provide an easy way to approximate a function as a polynomial, using the derivatives of the function. We know, from elementary

More information

On a class of nonsmooth composite functions

On a class of nonsmooth composite functions On a class of nonsmooth composite functions Alexander Shapiro School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332-0205, USA e-mail: ashapiro@isye.gatech.edu

More information

Optimization and Optimal Control in Banach Spaces

Optimization and Optimal Control in Banach Spaces Optimization and Optimal Control in Banach Spaces Bernhard Schmitzer October 19, 2017 1 Convex non-smooth optimization with proximal operators Remark 1.1 (Motivation). Convex optimization: easier to solve,

More information

Lagrangian Duality Theory

Lagrangian Duality Theory Lagrangian Duality Theory Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapter 14.1-4 1 Recall Primal and Dual

More information

Lectures 9 and 10: Constrained optimization problems and their optimality conditions

Lectures 9 and 10: Constrained optimization problems and their optimality conditions Lectures 9 and 10: Constrained optimization problems and their optimality conditions Coralia Cartis, Mathematical Institute, University of Oxford C6.2/B2: Continuous Optimization Lectures 9 and 10: Constrained

More information

Robust Duality in Parametric Convex Optimization

Robust Duality in Parametric Convex Optimization Robust Duality in Parametric Convex Optimization R.I. Boţ V. Jeyakumar G.Y. Li Revised Version: June 20, 2012 Abstract Modelling of convex optimization in the face of data uncertainty often gives rise

More information

Selected Examples of CONIC DUALITY AT WORK Robust Linear Optimization Synthesis of Linear Controllers Matrix Cube Theorem A.

Selected Examples of CONIC DUALITY AT WORK Robust Linear Optimization Synthesis of Linear Controllers Matrix Cube Theorem A. . Selected Examples of CONIC DUALITY AT WORK Robust Linear Optimization Synthesis of Linear Controllers Matrix Cube Theorem A. Nemirovski Arkadi.Nemirovski@isye.gatech.edu Linear Optimization Problem,

More information

Convex Optimization M2

Convex Optimization M2 Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization

More information

Sequential Pareto Subdifferential Sum Rule And Sequential Effi ciency

Sequential Pareto Subdifferential Sum Rule And Sequential Effi ciency Applied Mathematics E-Notes, 16(2016), 133-143 c ISSN 1607-2510 Available free at mirror sites of http://www.math.nthu.edu.tw/ amen/ Sequential Pareto Subdifferential Sum Rule And Sequential Effi ciency

More information

ON THE ESSENTIAL BOUNDEDNESS OF SOLUTIONS TO PROBLEMS IN PIECEWISE LINEAR-QUADRATIC OPTIMAL CONTROL. R.T. Rockafellar*

ON THE ESSENTIAL BOUNDEDNESS OF SOLUTIONS TO PROBLEMS IN PIECEWISE LINEAR-QUADRATIC OPTIMAL CONTROL. R.T. Rockafellar* ON THE ESSENTIAL BOUNDEDNESS OF SOLUTIONS TO PROBLEMS IN PIECEWISE LINEAR-QUADRATIC OPTIMAL CONTROL R.T. Rockafellar* Dedicated to J-L. Lions on his 60 th birthday Abstract. Primal and dual problems of

More information

Numerisches Rechnen. (für Informatiker) M. Grepl P. Esser & G. Welper & L. Zhang. Institut für Geometrie und Praktische Mathematik RWTH Aachen

Numerisches Rechnen. (für Informatiker) M. Grepl P. Esser & G. Welper & L. Zhang. Institut für Geometrie und Praktische Mathematik RWTH Aachen Numerisches Rechnen (für Informatiker) M. Grepl P. Esser & G. Welper & L. Zhang Institut für Geometrie und Praktische Mathematik RWTH Aachen Wintersemester 2011/12 IGPM, RWTH Aachen Numerisches Rechnen

More information

Duality. Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities

Duality. Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities Duality Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities Lagrangian Consider the optimization problem in standard form

More information

Perturbation analysis of a class of conic programming problems under Jacobian uniqueness conditions 1

Perturbation analysis of a class of conic programming problems under Jacobian uniqueness conditions 1 Perturbation analysis of a class of conic programming problems under Jacobian uniqueness conditions 1 Ziran Yin 2 Liwei Zhang 3 Abstract. We consider the stability of a class of parameterized conic programming

More information

1 Computing with constraints

1 Computing with constraints Notes for 2017-04-26 1 Computing with constraints Recall that our basic problem is minimize φ(x) s.t. x Ω where the feasible set Ω is defined by equality and inequality conditions Ω = {x R n : c i (x)

More information

Lecture 3. Optimization Problems and Iterative Algorithms

Lecture 3. Optimization Problems and Iterative Algorithms Lecture 3 Optimization Problems and Iterative Algorithms January 13, 2016 This material was jointly developed with Angelia Nedić at UIUC for IE 598ns Outline Special Functions: Linear, Quadratic, Convex

More information

MATH2070 Optimisation

MATH2070 Optimisation MATH2070 Optimisation Nonlinear optimisation with constraints Semester 2, 2012 Lecturer: I.W. Guo Lecture slides courtesy of J.R. Wishart Review The full nonlinear optimisation problem with equality constraints

More information

Robustness in Stochastic Programs with Risk Constraints

Robustness in Stochastic Programs with Risk Constraints Robustness in Stochastic Programs with Risk Constraints Dept. of Probability and Mathematical Statistics, Faculty of Mathematics and Physics Charles University, Prague, Czech Republic www.karlin.mff.cuni.cz/~kopa

More information

Optimality, identifiability, and sensitivity

Optimality, identifiability, and sensitivity Noname manuscript No. (will be inserted by the editor) Optimality, identifiability, and sensitivity D. Drusvyatskiy A. S. Lewis Received: date / Accepted: date Abstract Around a solution of an optimization

More information

Lecture 5. The Dual Cone and Dual Problem

Lecture 5. The Dual Cone and Dual Problem IE 8534 1 Lecture 5. The Dual Cone and Dual Problem IE 8534 2 For a convex cone K, its dual cone is defined as K = {y x, y 0, x K}. The inner-product can be replaced by x T y if the coordinates of the

More information

Optimization for Communications and Networks. Poompat Saengudomlert. Session 4 Duality and Lagrange Multipliers

Optimization for Communications and Networks. Poompat Saengudomlert. Session 4 Duality and Lagrange Multipliers Optimization for Communications and Networks Poompat Saengudomlert Session 4 Duality and Lagrange Multipliers P Saengudomlert (2015) Optimization Session 4 1 / 14 24 Dual Problems Consider a primal convex

More information

Optimization problems with complementarity constraints in infinite-dimensional spaces

Optimization problems with complementarity constraints in infinite-dimensional spaces Optimization problems with complementarity constraints in infinite-dimensional spaces Cumulative habilitation thesis Gerd Wachsmuth November 29, 2016 TU Chemnitz, Faculty of Mathematics, Research Group

More information

Active sets, steepest descent, and smooth approximation of functions

Active sets, steepest descent, and smooth approximation of functions Active sets, steepest descent, and smooth approximation of functions Dmitriy Drusvyatskiy School of ORIE, Cornell University Joint work with Alex D. Ioffe (Technion), Martin Larsson (EPFL), and Adrian

More information

Constrained Optimization

Constrained Optimization 1 / 22 Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University March 30, 2015 2 / 22 1. Equality constraints only 1.1 Reduced gradient 1.2 Lagrange

More information

Convex Optimization and Modeling

Convex Optimization and Modeling Convex Optimization and Modeling Duality Theory and Optimality Conditions 5th lecture, 12.05.2010 Jun.-Prof. Matthias Hein Program of today/next lecture Lagrangian and duality: the Lagrangian the dual

More information

SOME STABILITY RESULTS FOR THE SEMI-AFFINE VARIATIONAL INEQUALITY PROBLEM. 1. Introduction

SOME STABILITY RESULTS FOR THE SEMI-AFFINE VARIATIONAL INEQUALITY PROBLEM. 1. Introduction ACTA MATHEMATICA VIETNAMICA 271 Volume 29, Number 3, 2004, pp. 271-280 SOME STABILITY RESULTS FOR THE SEMI-AFFINE VARIATIONAL INEQUALITY PROBLEM NGUYEN NANG TAM Abstract. This paper establishes two theorems

More information

Lecture: Introduction to LP, SDP and SOCP

Lecture: Introduction to LP, SDP and SOCP Lecture: Introduction to LP, SDP and SOCP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2015.html wenzw@pku.edu.cn Acknowledgement:

More information

INVEX FUNCTIONS AND CONSTRAINED LOCAL MINIMA

INVEX FUNCTIONS AND CONSTRAINED LOCAL MINIMA BULL. AUSRAL. MAH. SOC. VOL. 24 (1981), 357-366. 9C3 INVEX FUNCIONS AND CONSRAINED LOCAL MINIMA B.D. CRAVEN If a certain weakening of convexity holds for the objective and all constraint functions in a

More information

Weak sharp minima on Riemannian manifolds 1

Weak sharp minima on Riemannian manifolds 1 1 Chong Li Department of Mathematics Zhejiang University Hangzhou, 310027, P R China cli@zju.edu.cn April. 2010 Outline 1 2 Extensions of some results for optimization problems on Banach spaces 3 4 Some

More information

Lecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.

Lecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min. MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.

More information

arxiv: v1 [math.oc] 13 Mar 2019

arxiv: v1 [math.oc] 13 Mar 2019 SECOND ORDER OPTIMALITY CONDITIONS FOR STRONG LOCAL MINIMIZERS VIA SUBGRADIENT GRAPHICAL DERIVATIVE Nguyen Huy Chieu, Le Van Hien, Tran T. A. Nghia, Ha Anh Tuan arxiv:903.05746v [math.oc] 3 Mar 209 Abstract

More information

CS-E4830 Kernel Methods in Machine Learning

CS-E4830 Kernel Methods in Machine Learning CS-E4830 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 27. September, 2017 Juho Rousu 27. September, 2017 1 / 45 Convex optimization Convex optimisation This

More information

Numerical Optimization

Numerical Optimization Constrained Optimization Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on Constrained Optimization Constrained Optimization Problem: min h j (x) 0,

More information

A sensitivity result for quadratic semidefinite programs with an application to a sequential quadratic semidefinite programming algorithm

A sensitivity result for quadratic semidefinite programs with an application to a sequential quadratic semidefinite programming algorithm Volume 31, N. 1, pp. 205 218, 2012 Copyright 2012 SBMAC ISSN 0101-8205 / ISSN 1807-0302 (Online) www.scielo.br/cam A sensitivity result for quadratic semidefinite programs with an application to a sequential

More information

Curriculum vitae of Giandomenico Mastroeni

Curriculum vitae of Giandomenico Mastroeni Curriculum vitae of Giandomenico Mastroeni General Informations Born in Ancona on 4/12/1964 High school degree (Livorno, 7/26/1982) Graduate in Mathematics at the University of Pisa (3/3/1988) Military

More information

Aubin Property and Uniqueness of Solutions in Cone Constrained Optimization

Aubin Property and Uniqueness of Solutions in Cone Constrained Optimization Preprint manuscript No. (will be inserted by the editor) Aubin Property and Uniqueness of Solutions in Cone Constrained Optimization Diethard Klatte Bernd Kummer submitted 6 January 2012, revised 11 June

More information

A ten page introduction to conic optimization

A ten page introduction to conic optimization CHAPTER 1 A ten page introduction to conic optimization This background chapter gives an introduction to conic optimization. We do not give proofs, but focus on important (for this thesis) tools and concepts.

More information

Optimality, identifiability, and sensitivity

Optimality, identifiability, and sensitivity Noname manuscript No. (will be inserted by the editor) Optimality, identifiability, and sensitivity D. Drusvyatskiy A. S. Lewis Received: date / Accepted: date Abstract Around a solution of an optimization

More information

Constrained optimization: direct methods (cont.)

Constrained optimization: direct methods (cont.) Constrained optimization: direct methods (cont.) Jussi Hakanen Post-doctoral researcher jussi.hakanen@jyu.fi Direct methods Also known as methods of feasible directions Idea in a point x h, generate a

More information

Sparse Optimization Lecture: Dual Certificate in l 1 Minimization

Sparse Optimization Lecture: Dual Certificate in l 1 Minimization Sparse Optimization Lecture: Dual Certificate in l 1 Minimization Instructor: Wotao Yin July 2013 Note scriber: Zheng Sun Those who complete this lecture will know what is a dual certificate for l 1 minimization

More information

ZERO DUALITY GAP FOR CONVEX PROGRAMS: A GENERAL RESULT

ZERO DUALITY GAP FOR CONVEX PROGRAMS: A GENERAL RESULT ZERO DUALITY GAP FOR CONVEX PROGRAMS: A GENERAL RESULT EMIL ERNST AND MICHEL VOLLE Abstract. This article addresses a general criterion providing a zero duality gap for convex programs in the setting of

More information

SENSITIVITY ANALYSIS OF GENERALIZED EQUATIONS

SENSITIVITY ANALYSIS OF GENERALIZED EQUATIONS Journal of Mathematical Sciences, Vol. 115, No. 4, 2003 SENSITIVITY ANALYSIS OF GENERALIZED EQUATIONS A. Shapiro UDC 517.977 In this paper, we study sensitivity analysis of generalized equations (variational

More information

Contents. 2 Sequences and Series Approximation by Rational Numbers Sequences Basics on Sequences...

Contents. 2 Sequences and Series Approximation by Rational Numbers Sequences Basics on Sequences... Contents 1 Real Numbers: The Basics... 1 1.1 Notation... 1 1.2 Natural Numbers... 4 1.3 Integers... 5 1.4 Fractions and Rational Numbers... 10 1.4.1 Introduction... 10 1.4.2 Powers and Radicals of Rational

More information

FULL STABILITY IN FINITE-DIMENSIONAL OPTIMIZATION

FULL STABILITY IN FINITE-DIMENSIONAL OPTIMIZATION FULL STABILITY IN FINITE-DIMENSIONAL OPTIMIZATION B. S. MORDUKHOVICH 1, T. T. A. NGHIA 2 and R. T. ROCKAFELLAR 3 Abstract. The paper is devoted to full stability of optimal solutions in general settings

More information

E5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming

E5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming E5295/5B5749 Convex optimization with engineering applications Lecture 5 Convex programming and semidefinite programming A. Forsgren, KTH 1 Lecture 5 Convex optimization 2006/2007 Convex quadratic program

More information

Numerical optimization

Numerical optimization Numerical optimization Lecture 4 Alexander & Michael Bronstein tosca.cs.technion.ac.il/book Numerical geometry of non-rigid shapes Stanford University, Winter 2009 2 Longest Slowest Shortest Minimal Maximal

More information

Strong duality in Lasserre s hierarchy for polynomial optimization

Strong duality in Lasserre s hierarchy for polynomial optimization Strong duality in Lasserre s hierarchy for polynomial optimization arxiv:1405.7334v1 [math.oc] 28 May 2014 Cédric Josz 1,2, Didier Henrion 3,4,5 Draft of January 24, 2018 Abstract A polynomial optimization

More information

A Geometric Framework for Nonconvex Optimization Duality using Augmented Lagrangian Functions

A Geometric Framework for Nonconvex Optimization Duality using Augmented Lagrangian Functions A Geometric Framework for Nonconvex Optimization Duality using Augmented Lagrangian Functions Angelia Nedić and Asuman Ozdaglar April 15, 2006 Abstract We provide a unifying geometric framework for the

More information

[1] Aubin, J.-P.: Optima and equilibria: An introduction to nonlinear analysis. Berlin Heidelberg, 1993.

[1] Aubin, J.-P.: Optima and equilibria: An introduction to nonlinear analysis. Berlin Heidelberg, 1993. Literaturverzeichnis [1] Aubin, J.-P.: Optima and equilibria: An introduction to nonlinear analysis. Berlin Heidelberg, 1993. [2] Aubin, J.-P.; Ekeland, I.: Applied nonlinear analysis. New York, 1984.

More information

Workshop on Nonlinear Optimization

Workshop on Nonlinear Optimization Workshop on Nonlinear Optimization 5-6 June 2015 The aim of the Workshop is to bring optimization experts in vector optimization and nonlinear optimization together to exchange their recent research findings

More information