Perturbation Analysis of Optimization Problems

Similar documents
Some Properties of the Augmented Lagrangian in Cone Constrained Optimization

1. Introduction. Consider the following parameterized optimization problem:

ON A CLASS OF NONSMOOTH COMPOSITE FUNCTIONS

DUALITY, OPTIMALITY CONDITIONS AND PERTURBATION ANALYSIS

Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem

On duality theory of conic linear problems

Semi-infinite programming, duality, discretization and optimality conditions

In view of (31), the second of these is equal to the identity I on E m, while this, in view of (30), implies that the first can be written

NONSMOOTH ANALYSIS AND PARAMETRIC OPTIMIZATION. R. T. Rockafellar*

Additional Homework Problems

II KLUWER ACADEMIC PUBLISHERS. Abstract Convexity and Global Optimization. Alexander Rubinov

Nonlinear Functional Analysis and its Applications

EE 227A: Convex Optimization and Applications October 14, 2008

Constrained Optimization Theory

LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE

CONVEX OPTIMIZATION VIA LINEARIZATION. Miguel A. Goberna. Universidad de Alicante. Iberian Conference on Optimization Coimbra, November, 2006

ANALYSIS IN SOBOLEV AND BV SPACES

Mathematical programs with complementarity constraints in Banach spaces

TMA 4180 Optimeringsteori KARUSH-KUHN-TUCKER THEOREM

Convex Optimization Theory. Chapter 5 Exercises and Solutions: Extended Version

STUDY PLAN MASTER IN (MATHEMATICS) (Thesis Track)

GEORGIA INSTITUTE OF TECHNOLOGY H. MILTON STEWART SCHOOL OF INDUSTRIAL AND SYSTEMS ENGINEERING LECTURE NOTES OPTIMIZATION III

CHARACTERIZATIONS OF LIPSCHITZIAN STABILITY

Reading group: Calculus of Variations and Optimal Control Theory by Daniel Liberzon

Second-order optimality conditions for state-constrained optimal control problems

ON LICQ AND THE UNIQUENESS OF LAGRANGE MULTIPLIERS

I.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010

M E M O R A N D U M. Faculty Senate approved November 1, 2018

Advanced Continuous Optimization

Generalization to inequality constrained problem. Maximize

Characterizing Robust Solution Sets of Convex Programs under Data Uncertainty

Structural and Multidisciplinary Optimization. P. Duysinx and P. Tossings

MATHEMATICS FOR ECONOMISTS. An Introductory Textbook. Third Edition. Malcolm Pemberton and Nicholas Rau. UNIVERSITY OF TORONTO PRESS Toronto Buffalo

Conditioning of linear-quadratic two-stage stochastic programming problems

The Relation Between Pseudonormality and Quasiregularity in Constrained Optimization 1

A Dual Condition for the Convex Subdifferential Sum Formula with Applications

Quadratic Growth and Stability in Convex Programming Problems with Multiple Solutions

Optimality Conditions for Constrained Optimization

w Kluwer Academic Publishers Boston/Dordrecht/London HANDBOOK OF SEMIDEFINITE PROGRAMMING Theory, Algorithms, and Applications

Some Contributions to Convex Infinite-Dimensional Optimization Duality

Duality Uses and Correspondences. Ryan Tibshirani Convex Optimization

Corrections and additions for the book Perturbation Analysis of Optimization Problems by J.F. Bonnans and A. Shapiro. Version of March 28, 2013

Enhanced Fritz John Optimality Conditions and Sensitivity Analysis

Convex Optimization & Lagrange Duality

Support Vector Machines: Maximum Margin Classifiers

NUMERICAL OPTIMIZATION. J. Ch. Gilbert

Sequential Quadratic Programming Method for Nonlinear Second-Order Cone Programming Problems. Hirokazu KATO

Optimality conditions for Equality Constrained Optimization Problems

LECTURE 12 LECTURE OUTLINE. Subgradients Fenchel inequality Sensitivity in constrained optimization Subdifferential calculus Optimality conditions

Convexity, Duality, and Lagrange Multipliers

CONTENTS. Preface Preliminaries 1

Appendix A Taylor Approximations and Definite Matrices

On a class of nonsmooth composite functions

Optimization and Optimal Control in Banach Spaces

Lagrangian Duality Theory

Lectures 9 and 10: Constrained optimization problems and their optimality conditions

Robust Duality in Parametric Convex Optimization

Selected Examples of CONIC DUALITY AT WORK Robust Linear Optimization Synthesis of Linear Controllers Matrix Cube Theorem A.

Convex Optimization M2

Sequential Pareto Subdifferential Sum Rule And Sequential Effi ciency

ON THE ESSENTIAL BOUNDEDNESS OF SOLUTIONS TO PROBLEMS IN PIECEWISE LINEAR-QUADRATIC OPTIMAL CONTROL. R.T. Rockafellar*

Numerisches Rechnen. (für Informatiker) M. Grepl P. Esser & G. Welper & L. Zhang. Institut für Geometrie und Praktische Mathematik RWTH Aachen

Duality. Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities

Perturbation analysis of a class of conic programming problems under Jacobian uniqueness conditions 1

1 Computing with constraints

Lecture 3. Optimization Problems and Iterative Algorithms

MATH2070 Optimisation

Robustness in Stochastic Programs with Risk Constraints

Optimality, identifiability, and sensitivity

Lecture 5. The Dual Cone and Dual Problem

Optimization for Communications and Networks. Poompat Saengudomlert. Session 4 Duality and Lagrange Multipliers

Optimization problems with complementarity constraints in infinite-dimensional spaces

Active sets, steepest descent, and smooth approximation of functions

Constrained Optimization

Convex Optimization and Modeling

SOME STABILITY RESULTS FOR THE SEMI-AFFINE VARIATIONAL INEQUALITY PROBLEM. 1. Introduction

Lecture: Introduction to LP, SDP and SOCP

INVEX FUNCTIONS AND CONSTRAINED LOCAL MINIMA

Weak sharp minima on Riemannian manifolds 1

Lecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.

arxiv: v1 [math.oc] 13 Mar 2019

CS-E4830 Kernel Methods in Machine Learning

Numerical Optimization

A sensitivity result for quadratic semidefinite programs with an application to a sequential quadratic semidefinite programming algorithm

Curriculum vitae of Giandomenico Mastroeni

Aubin Property and Uniqueness of Solutions in Cone Constrained Optimization

A ten page introduction to conic optimization

Optimality, identifiability, and sensitivity

Constrained optimization: direct methods (cont.)

Sparse Optimization Lecture: Dual Certificate in l 1 Minimization

ZERO DUALITY GAP FOR CONVEX PROGRAMS: A GENERAL RESULT

SENSITIVITY ANALYSIS OF GENERALIZED EQUATIONS

Contents. 2 Sequences and Series Approximation by Rational Numbers Sequences Basics on Sequences...

FULL STABILITY IN FINITE-DIMENSIONAL OPTIMIZATION

E5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming

Numerical optimization

Strong duality in Lasserre s hierarchy for polynomial optimization

A Geometric Framework for Nonconvex Optimization Duality using Augmented Lagrangian Functions

[1] Aubin, J.-P.: Optima and equilibria: An introduction to nonlinear analysis. Berlin Heidelberg, 1993.

Workshop on Nonlinear Optimization

Transcription:

Perturbation Analysis of Optimization Problems J. Frédéric Bonnans 1 and Alexander Shapiro 2 1 INRIA-Rocquencourt, Domaine de Voluceau, B.P. 105, 78153 Rocquencourt, France, and Ecole Polytechnique, France 2 School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332-0205, USA

2

Contents 1 Introduction 3 2 Background Material 11 2.1 Basic Functional Analysis... 11 2.1.1 Topological Vector Spaces... 11 2.1.2 The Hahn-Banach Theorem... 20 2.1.3 Banach Spaces... 25 2.1.4 Cones, Duality and Recession Cones... 34 2.2 Directional Differentiability and Tangent Cones... 37 2.2.1 First Order Directional Derivatives... 37 2.2.2 Second Order Derivatives... 41 2.2.3 Directional Epiderivatives of Extended Real Valued Functions 43 2.2.4 Tangent Cones... 48 2.3 Elements of Multifunctions Theory... 59 2.3.1 The Generalized Open Mapping Theorem... 59 2.3.2 Openness, Stability and Metric Regularity... 62 2.3.3 Stability of Nonlinear Constrained Systems... 65 2.3.4 Constraint Qualification Conditions... 72 2.3.5 Convex Mappings... 77 2.4 Convex Functions... 79 2.4.1 Continuity... 79 2.4.2 Conjugacy... 83 2.4.3 Subdifferentiability... 87 2.4.4 Chain Rules... 98 2.5 Duality Theory... 101 2.5.1 Conjugate Duality... 101 2.5.2 Lagrangian duality... 111 2.5.3 Examples and Applications of Duality Schemes... 114 2.5.4 Applications to Subdifferential Calculus... 121 2.5.5 Minimization of a Maximum over a Compact Set... 125 2.5.6 Conic Linear Problems... 133 2.5.7 Generalized Linear Programming and Polyhedral Multifunctions... 141 3

4 CONTENTS 3 Optimality Conditions 155 3.1 First Order Optimality Conditions... 156 3.1.1 Lagrange Multipliers... 156 3.1.2 Generalized Lagrange Multipliers... 162 3.1.3 Ekeland s Variational Principle... 166 3.1.4 First Order Sufficient Conditions... 168 3.2 Second Order Necessary Conditions... 172 3.2.1 Second Order Tangent Sets... 172 3.2.2 General Form of Second Order Necessary Conditions... 183 3.2.3 Extended Polyhedricity... 190 3.3 Second Order Sufficient Conditions... 196 3.3.1 General Form of Second Order Sufficient Conditions... 196 3.3.2 Quadratic and Extended Legendre Forms... 203 3.3.3 Second Order Regularity of Sets and No Gap Second Order Optimality Conditions... 209 3.3.4 Second Order Regularity of Functions... 219 3.3.5 Second Order Subderivatives... 223 3.4 Specific Structures... 229 3.4.1 Composite Optimization... 229 3.4.2 Exact Penalty Functions and Augmented Duality... 234 3.4.3 Linear Constraints and Quadratic Programming... 241 3.4.4 A Reduction Approach... 253 3.5 Nonisolated Minima... 258 3.5.1 Necessary Conditions for Quadratic Growth... 258 3.5.2 Sufficient Conditions... 262 3.5.3 Sufficient Conditions Based on General Critical Directions 270 4 Stability and Sensitivity Analysis 275 4.1 Stability of the Optimal Value and Optimal Solutions... 276 4.2 Directional Regularity... 281 4.3 First Order Differentiability Analysis of the Optimal Value Function287 4.3.1 The Case of Fixed Feasible Set... 287 4.3.2 Directional Differentiability of the Optimal Value Function Under Abstract Constraints... 293 4.4 Quantitative Stability of Optimal Solutions and Lagrange Multipliers303 4.4.1 Lipschitzian Stability in the Case of a Fixed Feasible Set. 303 4.4.2 Hölder Stability Under Abstract Constraints... 307 4.4.3 Quantitative Stability of Lagrange Multipliers... 311 4.4.4 Lipschitzian Stability of Optimal Solutions and Lagrange Multipliers... 316 4.5 Directional Stability of Optimal Solutions... 319 4.5.1 Hölder Directional Stability... 320 4.5.2 Lipschitzian Directional Stability... 321

CONTENTS 5 4.6 Quantitative Stability Analysis by a Reduction Approach... 331 4.6.1 Nondegeneracy and Strict Complementarity... 332 4.6.2 Stability Analysis... 337 4.7 Second Order Analysis in Lipschitz Stable Cases... 340 4.7.1 Upper Second Order Estimates of the Optimal Value Function341 4.7.2 Lower Estimates Without the Sigma Term... 350 4.7.3 The Second Order Regular Case... 355 4.7.4 Composite Optimization Problems... 359 4.8 Second Order Analysis in Hölder Stable Cases... 366 4.8.1 Upper Second Order Estimates of the Optimal Value Function366 4.8.2 Lower Estimates and Expansions of Optimal Solutions.. 375 4.8.3 Empty Sets of Lagrange Multipliers... 377 4.8.4 Hölder Expansions for Second Order Regular Problems.. 383 4.9 Additional Results... 385 4.9.1 Equality Constrained Problems... 385 4.9.2 Uniform Approximations of the Optimal Value and Optimal Solutions... 389 4.9.3 Second Order Analysis for Nonisolated Optima... 399 4.10 Second Order Analysis in Functional Spaces... 406 4.10.1 Second Order Tangent Sets in Functional Spaces of Continuous Functions... 407 4.10.2 Second Order Derivatives of Optimal Value Functions... 412 4.10.3 Second Order Expansions in Functional Spaces... 415 5 Additional Material and Applications 423 5.1 Variational Inequalities... 423 5.1.1 Standard Variational Inequalities... 423 5.1.2 Generalized Equations... 430 5.1.3 Strong Regularity... 435 5.1.4 Strong Regularity and Second Order Optimality Conditions 446 5.1.5 Strong Stability... 451 5.1.6 Some Examples and Applications... 453 5.2 Nonlinear Programming... 460 5.2.1 Finite Dimensional Linear Programs... 460 5.2.2 Optimality Conditions for Nonlinear Programs... 465 5.2.3 Lipschitz Expansions of Optimal Solutions... 470 5.2.4 Hölder Expansion of Optimal Solutions... 478 5.2.5 High Order Expansions of Optimal Solutions and Lagrange Multipliers... 484 5.2.6 Electrical Networks... 487 5.2.7 The Chain Problem... 491 5.3 Semi-definite Programming... 496 5.3.1 Geometry of the Cones of Negative Semidefinite Matrices. 497

CONTENTS 1 5.3.2 Matrix Convexity... 503 5.3.3 Duality... 505 5.3.4 First Order Optimality Conditions... 509 5.3.5 Second Order Optimality Conditions... 513 5.3.6 Stability and Sensitivity Analysis... 518 5.4 Semi-infinite Programming... 522 5.4.1 Duality... 524 5.4.2 First Order Optimality Conditions... 533 5.4.3 Second Order Optimality Conditions... 542 5.4.4 Perturbation Analysis... 549 6 Optimal Control 555 6.1 Introduction... 555 6.2 Linear and Semilinear Elliptic Equations... 555 6.2.1 The Dirichlet Problem... 556 6.2.2 Semilinear Elliptic Equations... 562 6.2.3 Strong Solutions... 565 6.3 Optimal Control of a Semilinear Elliptic Equation... 567 6.3.1 Existence of Solutions, First Order Optimality System.. 567 6.3.2 Second Order Necessary or Sufficient Conditions... 571 6.3.3 Some Specific Control Constraints... 576 6.3.4 Sensitivity Analysis... 578 6.3.5 State Constrained Optimal Control Problem... 581 6.3.6 Optimal Control of an Ill-Posed System... 583 6.4 The Obstacle Problem... 587 6.4.1 Presentation of the Problem... 587 6.4.2 Polyhedricity... 588 6.4.3 Basic Capacity Theory... 589 6.4.4 Sensitivity Analysis and Optimal Control... 595 7 Bibliographical Notes 599 7.1 Background Material... 599 7.2 Optimality Conditions... 601 7.3 Stability and Sensitivity Analysis... 603 7.4 Applications... 607 7.4.1 Variational Inequalities... 607 7.4.2 Nonlinear Programming... 609 7.4.3 Semi-definite Programming... 609 7.4.4 Semi-infinite Programming... 610 7.5 Optimal Control... 610

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback