University of California, Davis Department of Agricultural and Resource Economics ARE 252 Lecture Notes 2 Quirino Paris
|
|
- Sophie Ella Booth
- 5 years ago
- Views:
Transcription
1 University of California, Davis Department of Agricultural and Resource Economics ARE 5 Lecture Notes Quirino Paris Karush-Kuhn-Tucker conditions page Specification of a nonlinear programming problem NPP Review of the traditional Lagrange method Conversion of the NPP into a traditional Lagrange problem KKT conditions in vector notation Geometric interpretation of KKT conditions Constraint qualification Matching the KKT conditions with the structure of the Economic Equilibrium Karush-Kuhn-Tucker conditions The formulation of mathematical programming problems began in 939 with the publication of the Master of Science thesis by William Karush (W. Karush (939. "Minima of Functions of Several Variables with Inequalities as Side Constraints". M.Sc. Dissertation. Dept. of Mathematics, Univ. of Chicago, Chicago, Illinois.. This work was ignored for a long time to the point that H.W. Kuhn and A.W. Tucker published their 95 article without citing the seminal work of Karush (H. W. Kuhn and A. W. Tucker, Nonlinear Programming, Proc. Second Berkeley Symp. on Math. Statist. and Prob. (Univ. of Calif. Press, 95, Many authors still cite only Kuhn and Tucker when discussing Kuhn-Tucker conditions. The omission of Karush s name is unacceptable. His name belongs to the Karush-Kuhn-Tucker theory of nonlinear programming KKT conditions, for short. Specification of a nonlinear programming problem Most of the time, in this course, we will be dealing with a primal nonlinear programming problem (NPP like NPP max f (x x,x,...,x n subject to g (x b g (x b... g m (x b m x j, j =,...,n f (x is a differentiable concave (quasiconcave function called the objective function (to be optimized. The other relations are differentiable convex (quasiconvex functions called constraints (including the non-negativities of the endogenous variables x j, j =,...,n. As we will see soon, to solve the nonlinear problem stated above we will have to state also a constraint qualification to rule out strange behavior of the constraints.
2 Question: How should the first-order necessary conditions (FONC of the NPP be stated? The answer constitutes the KKT conditions. To understand the FONC of the NPP we will transform the NPP into a problem subject to equations and unrestricted variables in order to be able to apply the traditional Lagrange method. Review of the traditional Lagrange method (See also chapter, Symmetric Programming textbook Only one constraint is sufficient. For differentiable functions, max f (x x,x,...,x n subject to g(x = b. The Lagrange function is stated as L = f (x + λ[b g(x ] f g with FONC = λ, j =,...,n x j x j x j = b g(x,..., x n. λ The traditional Lagrange problem involves only equations and has no restricted variables. The solution of the FONC will provide the equilibrium values of the endogenous variables, x * j, j =,...,n and λ *. *,λ* f ( x * Recall that, at equilibrium, L(x = = λ * and λ * b b may be either a positive or a negative value f / x j depending on the derivatives λ * = g/ x j. Lagrange multipliers are interpreted (always as shadow prices of the corresponding constraint. Figure illustrates the shape of a Lagrange function. Figure. The graph of Lagrange function. y is the Lagrange multiplier Conversion of the NPP into a traditional Lagrange problem (See also chapter 3, Symmetric Programming textbook To make it simple, we will deal with only two constraints and two endogenous variables. NPP, max f (x x,x subject to
3 g (x b g (x b x x Let us hypothesize that the above NPP, represents the problem of maximizing total revenue, TR = f (x, subject to the technology, g (x b, g (x b, of transforming the supply of two inputs, b, b, into two outputs, x. To convert the NPP, into a traditional Lagrange problem (with only equations and unrestricted variables, add slack variables, s h, h =,...,4, to the inequalities to transform them into equations and unrestricted variables: NPP, max f (x x,x subject to g (x + s = b λ g (x + s = b λ x s 3 b 3 λ 3 b 4 λ 4 x s 4 In principle, in this NPP,, there are no inequalities and restricted variables. Therefore, we can apply the traditional Lagrange method in a straightforward way: the Lagrange function is L = f (x + λ [b s g (x ]+ λ [b s g (x ] + λ 3 (b 3 + s 3 x + λ 4 (b 4 + s 4 x with FONC f g g. = λ λ λ 3 g g. = f λ λ λ 4 x x x x 3. = λ s s 4. = λ s s 5. = λ 3 s 3 s 3 6. = λ 4 s 4 s 4 3
4 7. = b s g (x λ 8. = b s g (x λ 9. = b 3 + s 3 x λ 3 0. = b 4 + s 4 x λ 4 In the programming literature, the KKT conditions are presented as a set of inequalities and a set complementary slackness relations. To achieve that specification it is sufficient to eliminate the slack variables introduced above and discuss the remaining relations using the structure of the Economic Equilibrium problem discussed in Lecture Notes. Beginning with FONC 7 (and similarly for 8 l = b g (x = s λ Note that FONC 7 is interpreted as S D using the Economic Equilibrium structure. From FONC 9 (and similarly for 0 l = s3 = x + (0 b 3 x λ 3 Note that FONC 9 is interpreted as Q using the Economic Equilibrium structure. Furthermore, multiplying FONC 7 by λ (and similarly for FONC 8 λ = λ [b s g (x ] = λ [b g (x ] λ because of FONC 3 ( λ s (and similarly FONC 4 for FONC 8. Note that this result is interpreted as the complementary slackness condition P(S D of the Economic Equilibrium structure. Consider now FONC and rewrite it as (similarly for FONC l f g g λ = = λ 3 " x λ #%%%$%%% & MR 0 MR MC = λ 3 Therefore, the Lagrange multiplier λ 3 is interpreted as the (negative Opportunity Cost of producing (or not producing output x. Since λ 3 is non-positive according to the Economic Equilibrium structure the elimination of ( λ 3 from FONC (and similarly for FONC results in the inequality f g g λ = 0 x λ Furthermore, note that from FONC 9 ( b 3 0 and FONC 5 (and similarly from FONC 0 and 6 MC x = s 3 λ 3 x = λ 3 s 3 4
5 and, therefore, by multiplying FONC by x (and similarly FONC by x and using FONC 5 f g g f g g x = x λ λ λ 3 = x λ λ which corresponds to the complementary slackness condition Q(MR MC of the Economic Equilibrium structure. There remains to sign the Lagrange multipliers λ, λ. Since, by hypothesis, f ( is the total revenue function and b, b are supplies of inputs, an increase of input supply will not decrease the total revenue and, therefore, f f = λ, = λ. b b We have derived all the six (6 sets of KKT conditions which match the six (6 sets of relations of the Economic Equilibrium structure. Except that it is a little difficult to list them all clearly as they were developed in this simplified problem. For this reason, we restate now the KKT conditions using vector notation. KKT condition in vector notation Let the problem to solve be stated as the maximization of a differentiable and concave (quasi-concave function f (x subject to a differentiable and convex (quasi-convex vector function g(x such as max f (x = f (x,..., x n x subject to g (x,..., x n b g(x b!!! g m (x,..., x n b m x The dimension of the vector x is (n while the dimension of the vector b is (m with m < n simply as an arbitrary choice. The Lagrange function is and KKT conditions are L = f (x + λ [b g(x] f g. = λ 0 x x x f g Dual relations. x = x λ x x x 3. x 5
6 4. = b g(x λ Primal relations 5. λ = λ [b g(x] λ 6. λ Geometric interpretation of KKT conditions In order to be able to graph the relevant KKT conditions we return to the example with constraints and endogenous variables. The relevant KKT conditions are assumed to be binding f g g λ = x λ f g g λ = x x x x λ It is convenient to rewrite these KKT conditions in gradient notation. Let x * be the optimal point. Then * λ L x (x * = f x (x * g x (x * g x (x * * λ which corresponds to * f x (x * = g x (x * * λ + g x (x * λ * * with λ and λ. This means that the gradient of the objective function evaluated at the optimal point x * represents the main diagonal of the parallelogram constructed using the gradients of the constraints as a basis (mathematical ruler. Figure illustrates this fact. Figure. NPP and KKT conditions 6
7 In figure, D. The parallelogram expresses the KKT conditions with the gradient of the objective function being the main diagonal of the parallelogram constructed on the gradients of the constraints multiplied by positive scalars λ and λ. If the gradient of the objective function, f x (x *, were to fall outside the cone generated by the two gradients of the binding constraints, g x (x *, g x (x *, KKT conditions would be violated. (In a space of two dimensions, a cone is an arrangement of two vectors that form an angle less than 80 degrees. Constraint Qualification In solving a nonlinear programming problem by KKT theory there is the necessity of ruling out a peculiar arrangement of the constraints graph in the neighborhood of the optimal point. This assumption is called constraint qualification. It says that, if the optimal point happens to be in the neighborhood of a cusp, the KKT conditions break down. The Webster dictionary says that a cusp is a fixed point on a mathematical curve at which a point tracing the curve would exactly reverse its direction of motion. Hence, it is wise to stay away from this event by invoking the constraint qualification. A simple numerical example is sufficient to illustrate this unfavorable mathematical event. max f (x = x subject to x ( x 3 0, x Figure 3 illustrates the problem s graph. Figure 3. Violation of KKT conditions This problem fails to satisfy the associated KKT conditions. In other words, it is not possible to express the gradient of the objective function as a linear combination (let alone a positive linear combination of the gradient of the constraint. To verify this assertion we specify the Lagrange function: L = x + λ[( x 3 x ] with KKT conditions for x = 3λ( x 0 x = x [ 3λ( x ] 7
8 Looking at figure 3, the objective function is maximized at x * =. But at this optimal value of the endogenous variable the KKT conditions result in a contradiction because 0. Matching the KKT conditions with the structure of the Economic Equilibrium f g. = λ 0 x x x MR MC 0 f g. x = x λ x x x Q(MR MC 3. x Q 4. = b g(x S D λ 5. λ = λ [b g(x] P(S D λ 6. λ P As stated in Lecture Notes, the Economic Equilibrium structure is more general than the KKT conditions since it does not imply differentiability, integrability and optimization. This matching of the two structures is very important for all the material that we will discuss in this course. I suggest (recommend to learn all these relationships by heart for how they look like and for what they mean geometrically and economically. 8
Nonlinear Programming and the Kuhn-Tucker Conditions
Nonlinear Programming and the Kuhn-Tucker Conditions The Kuhn-Tucker (KT) conditions are first-order conditions for constrained optimization problems, a generalization of the first-order conditions we
More informationMathematical Foundations -1- Constrained Optimization. Constrained Optimization. An intuitive approach 2. First Order Conditions (FOC) 7
Mathematical Foundations -- Constrained Optimization Constrained Optimization An intuitive approach First Order Conditions (FOC) 7 Constraint qualifications 9 Formal statement of the FOC for a maximum
More informationQuiz Discussion. IE417: Nonlinear Programming: Lecture 12. Motivation. Why do we care? Jeff Linderoth. 16th March 2006
Quiz Discussion IE417: Nonlinear Programming: Lecture 12 Jeff Linderoth Department of Industrial and Systems Engineering Lehigh University 16th March 2006 Motivation Why do we care? We are interested in
More informationSTATIC LECTURE 4: CONSTRAINED OPTIMIZATION II - KUHN TUCKER THEORY
STATIC LECTURE 4: CONSTRAINED OPTIMIZATION II - KUHN TUCKER THEORY UNIVERSITY OF MARYLAND: ECON 600 1. Some Eamples 1 A general problem that arises countless times in economics takes the form: (Verbally):
More informationSeminars on Mathematics for Economics and Finance Topic 5: Optimization Kuhn-Tucker conditions for problems with inequality constraints 1
Seminars on Mathematics for Economics and Finance Topic 5: Optimization Kuhn-Tucker conditions for problems with inequality constraints 1 Session: 15 Aug 2015 (Mon), 10:00am 1:00pm I. Optimization with
More informationConstrained Optimization
1 / 22 Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University March 30, 2015 2 / 22 1. Equality constraints only 1.1 Reduced gradient 1.2 Lagrange
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More informationUNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems
UNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems Robert M. Freund February 2016 c 2016 Massachusetts Institute of Technology. All rights reserved. 1 1 Introduction
More informationSummary Notes on Maximization
Division of the Humanities and Social Sciences Summary Notes on Maximization KC Border Fall 2005 1 Classical Lagrange Multiplier Theorem 1 Definition A point x is a constrained local maximizer of f subject
More informationThe Monopolist. The Pure Monopolist with symmetric D matrix
University of California, Davis Department of Agricultural and Resource Economics ARE 252 Optimization with Economic Applications Lecture Notes 5 Quirino Paris The Monopolist.................................................................
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationMore on Lagrange multipliers
More on Lagrange multipliers CE 377K April 21, 2015 REVIEW The standard form for a nonlinear optimization problem is min x f (x) s.t. g 1 (x) 0. g l (x) 0 h 1 (x) = 0. h m (x) = 0 The objective function
More informationEC /11. Math for Microeconomics September Course, Part II Problem Set 1 with Solutions. a11 a 12. x 2
LONDON SCHOOL OF ECONOMICS Professor Leonardo Felli Department of Economics S.478; x7525 EC400 2010/11 Math for Microeconomics September Course, Part II Problem Set 1 with Solutions 1. Show that the general
More informationGeneralization to inequality constrained problem. Maximize
Lecture 11. 26 September 2006 Review of Lecture #10: Second order optimality conditions necessary condition, sufficient condition. If the necessary condition is violated the point cannot be a local minimum
More informationOptimization. A first course on mathematics for economists
Optimization. A first course on mathematics for economists Xavier Martinez-Giralt Universitat Autònoma de Barcelona xavier.martinez.giralt@uab.eu II.3 Static optimization - Non-Linear programming OPT p.1/45
More informationEC /11. Math for Microeconomics September Course, Part II Lecture Notes. Course Outline
LONDON SCHOOL OF ECONOMICS Professor Leonardo Felli Department of Economics S.478; x7525 EC400 20010/11 Math for Microeconomics September Course, Part II Lecture Notes Course Outline Lecture 1: Tools for
More informationNumerical Optimization
Constrained Optimization Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on Constrained Optimization Constrained Optimization Problem: min h j (x) 0,
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More informationOutline. Roadmap for the NPP segment: 1 Preliminaries: role of convexity. 2 Existence of a solution
Outline Roadmap for the NPP segment: 1 Preliminaries: role of convexity 2 Existence of a solution 3 Necessary conditions for a solution: inequality constraints 4 The constraint qualification 5 The Lagrangian
More informationCONSTRAINED OPTIMALITY CRITERIA
5 CONSTRAINED OPTIMALITY CRITERIA In Chapters 2 and 3, we discussed the necessary and sufficient optimality criteria for unconstrained optimization problems. But most engineering problems involve optimization
More informationLectures 9 and 10: Constrained optimization problems and their optimality conditions
Lectures 9 and 10: Constrained optimization problems and their optimality conditions Coralia Cartis, Mathematical Institute, University of Oxford C6.2/B2: Continuous Optimization Lectures 9 and 10: Constrained
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
More information+ 5x 2. = x x. + x 2. Transform the original system into a system x 2 = x x 1. = x 1
University of California, Davis Department of Agricultural and Resource Economics ARE 5 Optimization with Economic Applications Lecture Notes Quirino Paris The Pivot Method for Solving Systems of Equations...................................
More informationIn view of (31), the second of these is equal to the identity I on E m, while this, in view of (30), implies that the first can be written
11.8 Inequality Constraints 341 Because by assumption x is a regular point and L x is positive definite on M, it follows that this matrix is nonsingular (see Exercise 11). Thus, by the Implicit Function
More informationConstrained maxima and Lagrangean saddlepoints
Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analysis and Economic Theory Winter 2018 Topic 10: Constrained maxima and Lagrangean saddlepoints 10.1 An alternative As an application
More informationSymmetric Lagrange function
University of California, Davis Department of Agricultural and Resource Economics ARE 5 Optimization with Economic Applications Lecture Notes 7 Quirino Paris Symmetry.......................................................................page
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationEconomic Foundations of Symmetric Programming
Economic Foundations of Symmetric Programming QUIRINO PARIS University of California, Davis B 374309 CAMBRIDGE UNIVERSITY PRESS Foreword by Michael R. Caputo Preface page xv xvii 1 Introduction 1 Duality,
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationICS-E4030 Kernel Methods in Machine Learning
ICS-E4030 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 28. September, 2016 Juho Rousu 28. September, 2016 1 / 38 Convex optimization Convex optimisation This
More informationKarush-Kuhn-Tucker Conditions. Lecturer: Ryan Tibshirani Convex Optimization /36-725
Karush-Kuhn-Tucker Conditions Lecturer: Ryan Tibshirani Convex Optimization 10-725/36-725 1 Given a minimization problem Last time: duality min x subject to f(x) h i (x) 0, i = 1,... m l j (x) = 0, j =
More informationIntroduction to Machine Learning Prof. Sudeshna Sarkar Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur
Introduction to Machine Learning Prof. Sudeshna Sarkar Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Module - 5 Lecture - 22 SVM: The Dual Formulation Good morning.
More informationCHAPTER 1-2: SHADOW PRICES
Essential Microeconomics -- CHAPTER -: SHADOW PRICES An intuitive approach: profit maimizing firm with a fied supply of an input Shadow prices 5 Concave maimization problem 7 Constraint qualifications
More information1. f(β) 0 (that is, β is a feasible point for the constraints)
xvi 2. The lasso for linear models 2.10 Bibliographic notes Appendix Convex optimization with constraints In this Appendix we present an overview of convex optimization concepts that are particularly useful
More informationIntroduction to Optimization Techniques. Nonlinear Optimization in Function Spaces
Introduction to Optimization Techniques Nonlinear Optimization in Function Spaces X : T : Gateaux and Fréchet Differentials Gateaux and Fréchet Differentials a vector space, Y : a normed space transformation
More informationISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints
ISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints Instructor: Prof. Kevin Ross Scribe: Nitish John October 18, 2011 1 The Basic Goal The main idea is to transform a given constrained
More informationFIN 550 Practice Exam Answers. A. Linear programs typically have interior solutions.
FIN 550 Practice Exam Answers Phil Dybvig. True-False 25 points A. Linear programs typically have interior solutions. False. Unless the objective is zero, all solutions are at the boundary. B. A local
More informationFinite Dimensional Optimization Part I: The KKT Theorem 1
John Nachbar Washington University March 26, 2018 1 Introduction Finite Dimensional Optimization Part I: The KKT Theorem 1 These notes characterize maxima and minima in terms of first derivatives. I focus
More informationMotivation. Lecture 2 Topics from Optimization and Duality. network utility maximization (NUM) problem:
CDS270 Maryam Fazel Lecture 2 Topics from Optimization and Duality Motivation network utility maximization (NUM) problem: consider a network with S sources (users), each sending one flow at rate x s, through
More informationThe Dual of the Maximum Likelihood Method
Open Journal of Statistics, 06, 6, 86-93 Published Online February 06 in SciRes. http://www.scirp.org/journal/ojs http://dx.doi.org/0.436/ojs.06.606 The Dual of the Maximum Likelihood Method Quirino Paris
More information2.098/6.255/ Optimization Methods Practice True/False Questions
2.098/6.255/15.093 Optimization Methods Practice True/False Questions December 11, 2009 Part I For each one of the statements below, state whether it is true or false. Include a 1-3 line supporting sentence
More informationLecture 4: Optimization. Maximizing a function of a single variable
Lecture 4: Optimization Maximizing or Minimizing a Function of a Single Variable Maximizing or Minimizing a Function of Many Variables Constrained Optimization Maximizing a function of a single variable
More informationMicroeconomics I. September, c Leopold Sögner
Microeconomics I c Leopold Sögner Department of Economics and Finance Institute for Advanced Studies Stumpergasse 56 1060 Wien Tel: +43-1-59991 182 soegner@ihs.ac.at http://www.ihs.ac.at/ soegner September,
More informationThe general programming problem is the nonlinear programming problem where a given function is maximized subject to a set of inequality constraints.
1 Optimization Mathematical programming refers to the basic mathematical problem of finding a maximum to a function, f, subject to some constraints. 1 In other words, the objective is to find a point,
More informationStochastic Programming Math Review and MultiPeriod Models
IE 495 Lecture 5 Stochastic Programming Math Review and MultiPeriod Models Prof. Jeff Linderoth January 27, 2003 January 27, 2003 Stochastic Programming Lecture 5 Slide 1 Outline Homework questions? I
More informationSharpening the Karush-John optimality conditions
Sharpening the Karush-John optimality conditions Arnold Neumaier and Hermann Schichl Institut für Mathematik, Universität Wien Strudlhofgasse 4, A-1090 Wien, Austria email: Arnold.Neumaier@univie.ac.at,
More informationOPTIMISATION /09 EXAM PREPARATION GUIDELINES
General: OPTIMISATION 2 2008/09 EXAM PREPARATION GUIDELINES This points out some important directions for your revision. The exam is fully based on what was taught in class: lecture notes, handouts and
More informationDuality Theory of Constrained Optimization
Duality Theory of Constrained Optimization Robert M. Freund April, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 The Practical Importance of Duality Duality is pervasive
More informationIntroduction to Machine Learning Lecture 7. Mehryar Mohri Courant Institute and Google Research
Introduction to Machine Learning Lecture 7 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Convex Optimization Differentiation Definition: let f : X R N R be a differentiable function,
More informationMathematical Economics. Lecture Notes (in extracts)
Prof. Dr. Frank Werner Faculty of Mathematics Institute of Mathematical Optimization (IMO) http://math.uni-magdeburg.de/ werner/math-ec-new.html Mathematical Economics Lecture Notes (in extracts) Winter
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationLECTURE 7 Support vector machines
LECTURE 7 Support vector machines SVMs have been used in a multitude of applications and are one of the most popular machine learning algorithms. We will derive the SVM algorithm from two perspectives:
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Duality in Nonlinear Optimization ) Tamás TERLAKY Computing and Software McMaster University Hamilton, January 2004 terlaky@mcmaster.ca Tel: 27780 Optimality
More informationEcon 508-A FINITE DIMENSIONAL OPTIMIZATION - NECESSARY CONDITIONS. Carmen Astorne-Figari Washington University in St. Louis.
Econ 508-A FINITE DIMENSIONAL OPTIMIZATION - NECESSARY CONDITIONS Carmen Astorne-Figari Washington University in St. Louis August 12, 2010 INTRODUCTION General form of an optimization problem: max x f
More informationLagrange Relaxation and Duality
Lagrange Relaxation and Duality As we have already known, constrained optimization problems are harder to solve than unconstrained problems. By relaxation we can solve a more difficult problem by a simpler
More informationCONSTRAINT QUALIFICATIONS, LAGRANGIAN DUALITY & SADDLE POINT OPTIMALITY CONDITIONS
CONSTRAINT QUALIFICATIONS, LAGRANGIAN DUALITY & SADDLE POINT OPTIMALITY CONDITIONS A Dissertation Submitted For The Award of the Degree of Master of Philosophy in Mathematics Neelam Patel School of Mathematics
More informationRoles of Convexity in Optimization Theory. Efor, T. E and Nshi C. E
IDOSR PUBLICATIONS International Digital Organization for Scientific Research ISSN: 2550-7931 Roles of Convexity in Optimization Theory Efor T E and Nshi C E Department of Mathematics and Computer Science
More informationOptimality Conditions for Constrained Optimization
72 CHAPTER 7 Optimality Conditions for Constrained Optimization 1. First Order Conditions In this section we consider first order optimality conditions for the constrained problem P : minimize f 0 (x)
More informationNonlinear Programming (Hillier, Lieberman Chapter 13) CHEM-E7155 Production Planning and Control
Nonlinear Programming (Hillier, Lieberman Chapter 13) CHEM-E7155 Production Planning and Control 19/4/2012 Lecture content Problem formulation and sample examples (ch 13.1) Theoretical background Graphical
More informationThe Karush-Kuhn-Tucker (KKT) conditions
The Karush-Kuhn-Tucker (KKT) conditions In this section, we will give a set of sufficient (and at most times necessary) conditions for a x to be the solution of a given convex optimization problem. These
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationConvex Optimization. Dani Yogatama. School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA. February 12, 2014
Convex Optimization Dani Yogatama School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA February 12, 2014 Dani Yogatama (Carnegie Mellon University) Convex Optimization February 12,
More informationOPTIMISATION 2007/8 EXAM PREPARATION GUIDELINES
General: OPTIMISATION 2007/8 EXAM PREPARATION GUIDELINES This points out some important directions for your revision. The exam is fully based on what was taught in class: lecture notes, handouts and homework.
More informationCONVEX FUNCTIONS AND OPTIMIZATION TECHINIQUES A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF
CONVEX FUNCTIONS AND OPTIMIZATION TECHINIQUES A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE IN MATHEMATICS SUBMITTED TO NATIONAL INSTITUTE OF TECHNOLOGY,
More informationJørgen Tind, Department of Statistics and Operations Research, University of Copenhagen, Universitetsparken 5, 2100 Copenhagen O, Denmark.
DUALITY THEORY Jørgen Tind, Department of Statistics and Operations Research, University of Copenhagen, Universitetsparken 5, 2100 Copenhagen O, Denmark. Keywords: Duality, Saddle point, Complementary
More informationThe Fundamental Welfare Theorems
The Fundamental Welfare Theorems The so-called Fundamental Welfare Theorems of Economics tell us about the relation between market equilibrium and Pareto efficiency. The First Welfare Theorem: Every Walrasian
More informationLinear and Combinatorial Optimization
Linear and Combinatorial Optimization The dual of an LP-problem. Connections between primal and dual. Duality theorems and complementary slack. Philipp Birken (Ctr. for the Math. Sc.) Lecture 3: Duality
More informationLecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem
Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 0-0 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R
More informationConstrained Optimization
Constrained Optimization Joshua Wilde, revised by Isabel Tecu, Takeshi Suzuki and María José Boccardi August 13, 2013 1 General Problem Consider the following general constrained optimization problem:
More informationIntroduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.1-4.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following
More informationChap 2. Optimality conditions
Chap 2. Optimality conditions Version: 29-09-2012 2.1 Optimality conditions in unconstrained optimization Recall the definitions of global, local minimizer. Geometry of minimization Consider for f C 1
More informationLecture 7: Convex Optimizations
Lecture 7: Convex Optimizations Radu Balan, David Levermore March 29, 2018 Convex Sets. Convex Functions A set S R n is called a convex set if for any points x, y S the line segment [x, y] := {tx + (1
More informationOptimization Theory. Lectures 4-6
Optimization Theory Lectures 4-6 Unconstrained Maximization Problem: Maximize a function f:ú n 6 ú within a set A f ú n. Typically, A is ú n, or the non-negative orthant {x0ú n x$0} Existence of a maximum:
More informationKKT Examples. Stanley B. Gershwin Massachusetts Institute of Technology
Stanley B. Gershwin Massachusetts Institute of Technology The purpose of this note is to supplement the slides that describe the Karush-Kuhn-Tucker conditions. Neither these notes nor the slides are a
More informationLinear programming: Theory
Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analsis and Economic Theor Winter 2018 Topic 28: Linear programming: Theor 28.1 The saddlepoint theorem for linear programming The
More informationLecture Notes on Support Vector Machine
Lecture Notes on Support Vector Machine Feng Li fli@sdu.edu.cn Shandong University, China 1 Hyperplane and Margin In a n-dimensional space, a hyper plane is defined by ω T x + b = 0 (1) where ω R n is
More informationConvex Optimization Lecture 6: KKT Conditions, and applications
Convex Optimization Lecture 6: KKT Conditions, and applications Dr. Michel Baes, IFOR / ETH Zürich Quick recall of last week s lecture Various aspects of convexity: The set of minimizers is convex. Convex
More informationCE 191: Civil & Environmental Engineering Systems Analysis. LEC 17 : Final Review
CE 191: Civil & Environmental Engineering Systems Analysis LEC 17 : Final Review Professor Scott Moura Civil & Environmental Engineering University of California, Berkeley Fall 2014 Prof. Moura UC Berkeley
More informationTMA 4180 Optimeringsteori KARUSH-KUHN-TUCKER THEOREM
TMA 4180 Optimeringsteori KARUSH-KUHN-TUCKER THEOREM H. E. Krogstad, IMF, Spring 2012 Karush-Kuhn-Tucker (KKT) Theorem is the most central theorem in constrained optimization, and since the proof is scattered
More information1.3 The Indirect Utility Function
1.2 Utility Maximization Problem (UMP) (MWG 2.D, 2.E; Kreps 2.2) max u (x) s.t. p.x w and x 0 hx Xi For a cts preference relation represented by a cts utility fn, u ( ): 1. The UMP has at least one solution
More informationGENERALIZED CONVEXITY AND OPTIMALITY CONDITIONS IN SCALAR AND VECTOR OPTIMIZATION
Chapter 4 GENERALIZED CONVEXITY AND OPTIMALITY CONDITIONS IN SCALAR AND VECTOR OPTIMIZATION Alberto Cambini Department of Statistics and Applied Mathematics University of Pisa, Via Cosmo Ridolfi 10 56124
More informationConvex Optimization Overview (cnt d)
Conve Optimization Overview (cnt d) Chuong B. Do November 29, 2009 During last week s section, we began our study of conve optimization, the study of mathematical optimization problems of the form, minimize
More informationTutorial on Convex Optimization: Part II
Tutorial on Convex Optimization: Part II Dr. Khaled Ardah Communications Research Laboratory TU Ilmenau Dec. 18, 2018 Outline Convex Optimization Review Lagrangian Duality Applications Optimal Power Allocation
More informationEE/AA 578, Univ of Washington, Fall Duality
7. Duality EE/AA 578, Univ of Washington, Fall 2016 Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationSupport Vector Machines
Support Vector Machines Ryan M. Rifkin Google, Inc. 2008 Plan Regularization derivation of SVMs Geometric derivation of SVMs Optimality, Duality and Large Scale SVMs The Regularization Setting (Again)
More informationDEPARTMENT OF STATISTICS AND OPERATIONS RESEARCH OPERATIONS RESEARCH DETERMINISTIC QUALIFYING EXAMINATION. Part I: Short Questions
DEPARTMENT OF STATISTICS AND OPERATIONS RESEARCH OPERATIONS RESEARCH DETERMINISTIC QUALIFYING EXAMINATION Part I: Short Questions August 12, 2008 9:00 am - 12 pm General Instructions This examination is
More informationThe Kuhn-Tucker Problem
Natalia Lazzati Mathematics for Economics (Part I) Note 8: Nonlinear Programming - The Kuhn-Tucker Problem Note 8 is based on de la Fuente (2000, Ch. 7) and Simon and Blume (1994, Ch. 18 and 19). The Kuhn-Tucker
More information6.254 : Game Theory with Engineering Applications Lecture 7: Supermodular Games
6.254 : Game Theory with Engineering Applications Lecture 7: Asu Ozdaglar MIT February 25, 2010 1 Introduction Outline Uniqueness of a Pure Nash Equilibrium for Continuous Games Reading: Rosen J.B., Existence
More informationConvex Programs. Carlo Tomasi. December 4, 2018
Convex Programs Carlo Tomasi December 4, 2018 1 Introduction In an earlier note, we found methods for finding a local minimum of some differentiable function f(u) : R m R. If f(u) is at least weakly convex,
More informationFinite Dimensional Optimization Part III: Convex Optimization 1
John Nachbar Washington University March 21, 2017 Finite Dimensional Optimization Part III: Convex Optimization 1 1 Saddle points and KKT. These notes cover another important approach to optimization,
More informationLinear & nonlinear classifiers
Linear & nonlinear classifiers Machine Learning Hamid Beigy Sharif University of Technology Fall 1396 Hamid Beigy (Sharif University of Technology) Linear & nonlinear classifiers Fall 1396 1 / 44 Table
More informationMathematical Economics: Lecture 16
Mathematical Economics: Lecture 16 Yu Ren WISE, Xiamen University November 26, 2012 Outline 1 Chapter 21: Concave and Quasiconcave Functions New Section Chapter 21: Concave and Quasiconcave Functions Concave
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationCS-E4830 Kernel Methods in Machine Learning
CS-E4830 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 27. September, 2017 Juho Rousu 27. September, 2017 1 / 45 Convex optimization Convex optimisation This
More informationLecture: Duality.
Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong
More informationEcon Slides from Lecture 14
Econ 205 Sobel Econ 205 - Slides from Lecture 14 Joel Sobel September 10, 2010 Theorem ( Lagrange Multipliers ) Theorem If x solves max f (x) subject to G(x) = 0 then there exists λ such that Df (x ) =
More informationCO 250 Final Exam Guide
Spring 2017 CO 250 Final Exam Guide TABLE OF CONTENTS richardwu.ca CO 250 Final Exam Guide Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4,
More informationIntroduction to Support Vector Machines
Introduction to Support Vector Machines Shivani Agarwal Support Vector Machines (SVMs) Algorithm for learning linear classifiers Motivated by idea of maximizing margin Efficient extension to non-linear
More informationLagrangian Duality. Richard Lusby. Department of Management Engineering Technical University of Denmark
Lagrangian Duality Richard Lusby Department of Management Engineering Technical University of Denmark Today s Topics (jg Lagrange Multipliers Lagrangian Relaxation Lagrangian Duality R Lusby (42111) Lagrangian
More informationOptimality, Duality, Complementarity for Constrained Optimization
Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of Wisconsin-Madison May 2014 Wright (UW-Madison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear
More information