Constrained Optimization. Unconstrained Optimization (1)
|
|
- Allan Merritt
- 5 years ago
- Views:
Transcription
1 Constrained Optimization Unconstrained Optimization (Review) Constrained Optimization Approach Equality constraints * Lagrangeans * Shadow prices Inequality constraints * Kuhn-Tucker conditions * Complementary slackness Massachusetts Institute of Technology Constrained Optimization Slide 1 of 23 Unconstrained Optimization (1) Definitions: Optimization = Maximum of desired quantity = Minimum of undesired quantity Objective Function = Expression to be optimized = Z(X) Decision Variables = Variables about which we can make decisions = X = (X 1.X n ) Massachusetts Institute of Technology Constrained Optimization Slide 2 of 23 Page 1
2 Unconstrained Optimization (2) F(X) B D A By calculus: C E X If F(X) continuous, analytic: Condition for maxima and minima F(X) / X i = 0 i Massachusetts Institute of Technology Constrained Optimization Slide 3 of 23 Unconstrained Optimization (3) Secondary conditions: 2 F(X) / X 2 i < 0 = > Max (B,D) 2 F(X) / X 2 i > 0 = > Min (A,C,E) These define whether point of no change in Z is a maximum or a minimum Massachusetts Institute of Technology Constrained Optimization Slide 4 of 23 Page 2
3 Unconstrained Optimization (4) Example: Housing insulation F(x) = K 1 / x + K 2 x Total Cost = Fuel cost + Insulation cost x = Thickness of insulation F(x) / x = 0 = -K 1 / x 2 + K 2 => x* = {K 1 / K 2 } 1/2 (starred quantities are optimal) Massachusetts Institute of Technology Constrained Optimization Slide 5 of 23 Unconstrained Optimization (5) K 1 = 500 K 2 = 24 X* = 4.56 Optimizing Cost Example Cost Inches of Insulation Fuel Insulation Total Massachusetts Institute of Technology Constrained Optimization Slide 6 of 23 Page 3
4 Constained Optimization Constrained Optimization involves the optimization of a process subject to constraints Constraints have two basic types Equality Constraints -- some factors have to equal constraints Inequality Constraints -- some factors have to be less less than or greater than the constraints (these are upper and lower bounds Massachusetts Institute of Technology Constrained Optimization Slide 7 of 23 Equality Constraints Example: Best use of budget Maximize: Output = Z(X) = a o x 1 a1 x 2 a2 Subject to (s.t.): Total costs = Budget = p 1 x 1 + p 2 x 2 Z(x) Z* Budget Note: Z(X) / X 0 at optimum X Massachusetts Institute of Technology Constrained Optimization Slide 8 of 23 Page 4
5 Approach Constrained Optimization To solve situations of increasing complexity, (for example, those with equality, inequality constraints)... Transform more difficult situation into one we know how to deal with In this case, transform optimization of a constrained situation to optimization of unconstrained situation Massachusetts Institute of Technology Constrained Optimization Slide 9 of 23 Lagrangean Method (1) Transforms equality constraints into unconstrained problem Start with: Opt: F(x) s.t.: g j (x) = b j => g j (x) - b j = 0 Get to: L = F(x) - Σ j λ j [g j (x) - b j ] λj = Lagrangean multipliers (lambdas) -- these are unknown quantities for which we must solve Note: [g j (x) - b j ] = 0 by definition, thus optimum for F(x) = optimum for L Massachusetts Institute of Technology Constrained Optimization Slide 10 of 23 Page 5
6 Lagrangean Method (2) To optimize L: L / x i = 0 i L / λ j = 0 i Example: Opt: F(x) = 6x 1 x 2 s.t.: g(x) = 3x 1 + 4x 2 = 18 L = 6x 1 x 2 - λ(3x 1 + 4x 2-18) L / x 1 = 6x 2-3λ = 0 L / x 2 = 6x 1-4λ = 0 L / λ i = 3x 1 + 4x 2-18 = 0 Massachusetts Institute of Technology Constrained Optimization Slide 11 of 23 Lagrangean Method (3) Solving as unconstrained problem: L / x 1 = 6x 2-3λ = 0 L / x 2 = 6x 1-4λ = 0 L / λ i = 3x 1 + 4x 2-18 = 0 so that: λ = 2x 2 = 1.5x 1 x 2 = 0.75x 1 3x 1 + 3x 1-18 = 0 x 1 * = 18/3 = 6 x 2 * = 18/8 = 2.25 λ* = 4.5 F(x)* = 40.5 Massachusetts Institute of Technology Constrained Optimization Slide 12 of 23 Page 6
7 Shadow Prices Shadow Price is the Rate of change of objective function per unit change of constraint = F(x) / b j This is meaning of Lagrangean multiplier SP j = F(x)*/ b j = λ j Naturally, this is an instantaneous rate The shadow price is extremely important for system design It defines value of changing constraints Massachusetts Institute of Technology Constrained Optimization Slide 13 of 23 Shadow Prices (2) Let s see how this works in example, by changing constraint by 0.1 units: Opt: F(x) = 6x 1 x 2 s.t.: g(x) = 3x 1 + 4x 2 = 18.1 The optimum values of the variables are x 1 * = (18.1)/6 x 2 * = (18.1)/8 Thus F(x)* = 6(18.1/6)(18.1/8) = F(x) = = 0.45 = λ* (0.1) Massachusetts Institute of Technology Constrained Optimization Slide 14 of 23 Page 7
8 Inequality Constraints Example: Housing insulation Min: Costs = K 1 / x + K 2 x s.t.: x 8 (minimum thickness) Optimizing Cost Example Cost Inches of Insulation Fuel Insulation Total Massachusetts Institute of Technology Constrained Optimization Slide 15 of 23 Inequality Constraints (2) Approach: Transform inequalities into equalities, then proceed as before Again, introduce new variable -- the Slack variable that defines slack or distance between constraint and amount used The resulting equations are known as the Kuhn-Tucker conditions Massachusetts Institute of Technology Constrained Optimization Slide 16 of 23 Page 8
9 Inequality Constraints -- insertion of slack variables in Lagrangean A slack variable, s j, for each inequality g j (x) b j => g j (x) + s j 2 = b j g j (x) b j => g j (x) - s j 2 = b j These are squared to be positive start from: opt: F(x) s.t.: g j (x) b j get to: L = F(x) - Σ j λ j [g j (x) + s j 2 - b j ] Massachusetts Institute of Technology Constrained Optimization Slide 17 of 23 Inequality Constraints -- Complementary Slackness Conditions The optimality conditions are: L / x i = 0 L / λ j = 0 plus: L / s j = 2s j λ j = 0 These new equations imply: s j = 0 or λ j 0 s j 0 λ j = 0 They are the complementary slackness conditions. Either slack or lambda =0 i Massachusetts Institute of Technology Constrained Optimization Slide 18 of 23 Page 9
10 Interpretation of Complementary Slackness Conditions Interpretation: If there is slack on b j, (i.e. more than enough of it) => No value to objective function to having more: λ j = F(x) / b j = 0 If λ j 0, then all available b j used => s j = 0 Massachusetts Institute of Technology Constrained Optimization Slide 19 of 23 Application to Example Min: Costs = K 1 / x + K 2 x s.t.: x 8 (minimum thickness) L = K 1 /x + K 2 x - λ[x - s 2 - b] L = 500/x + 24x - λ[x - s 2-8] 500/x λ = 0 2 λs = 0 x - s 2 = 8 If s = 0, x = 8, λ = 31.8 (at that point) Max = Therefore, worth relaxing (in this case, lowering) constraint to get maximum x * = 4.56 Optimum = 221 Massachusetts Institute of Technology Constrained Optimization Slide 20 of 23 Page 10
11 Unconstrained Optimization (5) K 1 = 500 K 2 = 24 X* = 4.56 Optimizing Cost Example Cost Inches of Insulation Fuel Insulation Total Massachusetts Institute of Technology Constrained Optimization Slide 21 of 23 Another application to Example Min: Costs = K 1 / x + K 2 x s.t.: x 4 (NEW MINIMUM) L = K 1 /x + K 2 x - λ[x + s 2 - b] L = 500/x + 24x - λ[x + s 2-4] 500/x λ = 0 2 λs = 0 x - s 2 = 4 If λ = 0, x = 4.56, slack, s 2 = 1.56 Optimum = 221 not worth changing constraint Massachusetts Institute of Technology Constrained Optimization Slide 22 of 23 Page 11
12 Summary of Presentation Important mathematical approaches Lagreangeans Kuhn- Tucker Conditions Important Concept: Shadow Prices THESE ANALYSES GUIDE DESIGNERS TO CHALLENGE CONSTRAINTS Massachusetts Institute of Technology Constrained Optimization Slide 23 of 23 Page 12
Econ Slides from Lecture 14
Econ 205 Sobel Econ 205 - Slides from Lecture 14 Joel Sobel September 10, 2010 Theorem ( Lagrange Multipliers ) Theorem If x solves max f (x) subject to G(x) = 0 then there exists λ such that Df (x ) =
More informationSeminars on Mathematics for Economics and Finance Topic 5: Optimization Kuhn-Tucker conditions for problems with inequality constraints 1
Seminars on Mathematics for Economics and Finance Topic 5: Optimization Kuhn-Tucker conditions for problems with inequality constraints 1 Session: 15 Aug 2015 (Mon), 10:00am 1:00pm I. Optimization with
More informationLakehead University ECON 4117/5111 Mathematical Economics Fall 2002
Test 1 September 20, 2002 1. Determine whether each of the following is a statement or not (answer yes or no): (a) Some sentences can be labelled true and false. (b) All students should study mathematics.
More informationConstrained maxima and Lagrangean saddlepoints
Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analysis and Economic Theory Winter 2018 Topic 10: Constrained maxima and Lagrangean saddlepoints 10.1 An alternative As an application
More informationMATH2070 Optimisation
MATH2070 Optimisation Nonlinear optimisation with constraints Semester 2, 2012 Lecturer: I.W. Guo Lecture slides courtesy of J.R. Wishart Review The full nonlinear optimisation problem with equality constraints
More informationNonlinear Programming and the Kuhn-Tucker Conditions
Nonlinear Programming and the Kuhn-Tucker Conditions The Kuhn-Tucker (KT) conditions are first-order conditions for constrained optimization problems, a generalization of the first-order conditions we
More informationg(x,y) = c. For instance (see Figure 1 on the right), consider the optimization problem maximize subject to
1 of 11 11/29/2010 10:39 AM From Wikipedia, the free encyclopedia In mathematical optimization, the method of Lagrange multipliers (named after Joseph Louis Lagrange) provides a strategy for finding the
More informationThe Kuhn-Tucker Problem
Natalia Lazzati Mathematics for Economics (Part I) Note 8: Nonlinear Programming - The Kuhn-Tucker Problem Note 8 is based on de la Fuente (2000, Ch. 7) and Simon and Blume (1994, Ch. 18 and 19). The Kuhn-Tucker
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
More informationConstrained Optimization
Constrained Optimization Joshua Wilde, revised by Isabel Tecu, Takeshi Suzuki and María José Boccardi August 13, 2013 1 General Problem Consider the following general constrained optimization problem:
More informationISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints
ISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints Instructor: Prof. Kevin Ross Scribe: Nitish John October 18, 2011 1 The Basic Goal The main idea is to transform a given constrained
More informationConcave programming. Concave programming is another special case of the general constrained optimization. subject to g(x) 0
1 Introduction Concave programming Concave programming is another special case of the general constrained optimization problem max f(x) subject to g(x) 0 in which the objective function f is concave and
More informationLecture 4: Optimization. Maximizing a function of a single variable
Lecture 4: Optimization Maximizing or Minimizing a Function of a Single Variable Maximizing or Minimizing a Function of Many Variables Constrained Optimization Maximizing a function of a single variable
More informationDYNAMIC LECTURE 5: DISCRETE TIME INTERTEMPORAL OPTIMIZATION
DYNAMIC LECTURE 5: DISCRETE TIME INTERTEMPORAL OPTIMIZATION UNIVERSITY OF MARYLAND: ECON 600. Alternative Methods of Discrete Time Intertemporal Optimization We will start by solving a discrete time intertemporal
More informationMathematical Foundations -1- Constrained Optimization. Constrained Optimization. An intuitive approach 2. First Order Conditions (FOC) 7
Mathematical Foundations -- Constrained Optimization Constrained Optimization An intuitive approach First Order Conditions (FOC) 7 Constraint qualifications 9 Formal statement of the FOC for a maximum
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More informationMicroeconomic Theory. Microeconomic Theory. Everyday Economics. The Course:
The Course: Microeconomic Theory This is the first rigorous course in microeconomic theory This is a course on economic methodology. The main goal is to teach analytical tools that will be useful in other
More informationOPTIMIZATION THEORY IN A NUTSHELL Daniel McFadden, 1990, 2003
OPTIMIZATION THEORY IN A NUTSHELL Daniel McFadden, 1990, 2003 UNCONSTRAINED OPTIMIZATION 1. Consider the problem of maximizing a function f:ú n 6 ú within a set A f ú n. Typically, A might be all of ú
More informationLINEAR PROGRAMMING. Relation to the Text (cont.) Relation to Material in Text. Relation to the Text. Relation to the Text (cont.
LINEAR PROGRAMMING Relation to Material in Text After a brief introduction to linear programming on p. 3, Cornuejols and Tϋtϋncϋ give a theoretical discussion including duality, and the simplex solution
More informationTutorial 3: Optimisation
Tutorial : Optimisation ECO411F 011 1. Find and classify the extrema of the cubic cost function C = C (Q) = Q 5Q +.. Find and classify the extreme values of the following functions (a) y = x 1 + x x 1x
More informationCHAPTER 1-2: SHADOW PRICES
Essential Microeconomics -- CHAPTER -: SHADOW PRICES An intuitive approach: profit maimizing firm with a fied supply of an input Shadow prices 5 Concave maimization problem 7 Constraint qualifications
More informationOptimization. A first course on mathematics for economists
Optimization. A first course on mathematics for economists Xavier Martinez-Giralt Universitat Autònoma de Barcelona xavier.martinez.giralt@uab.eu II.3 Static optimization - Non-Linear programming OPT p.1/45
More informationMVE165/MMG631 Linear and integer optimization with applications Lecture 13 Overview of nonlinear programming. Ann-Brith Strömberg
MVE165/MMG631 Overview of nonlinear programming Ann-Brith Strömberg 2015 05 21 Areas of applications, examples (Ch. 9.1) Structural optimization Design of aircraft, ships, bridges, etc Decide on the material
More informationEconomics 101A (Lecture 3) Stefano DellaVigna
Economics 101A (Lecture 3) Stefano DellaVigna January 24, 2017 Outline 1. Implicit Function Theorem 2. Envelope Theorem 3. Convexity and concavity 4. Constrained Maximization 1 Implicit function theorem
More informationHandout 1: Introduction to Dynamic Programming. 1 Dynamic Programming: Introduction and Examples
SEEM 3470: Dynamic Optimization and Applications 2013 14 Second Term Handout 1: Introduction to Dynamic Programming Instructor: Shiqian Ma January 6, 2014 Suggested Reading: Sections 1.1 1.5 of Chapter
More informationNonlinear Programming (Hillier, Lieberman Chapter 13) CHEM-E7155 Production Planning and Control
Nonlinear Programming (Hillier, Lieberman Chapter 13) CHEM-E7155 Production Planning and Control 19/4/2012 Lecture content Problem formulation and sample examples (ch 13.1) Theoretical background Graphical
More informationLESSON 25: LAGRANGE MULTIPLIERS OCTOBER 30, 2017
LESSON 5: LAGRANGE MULTIPLIERS OCTOBER 30, 017 Lagrange multipliers is another method of finding minima and maxima of functions of more than one variable. In fact, many of the problems from the last homework
More informationIntroduction to Machine Learning Prof. Sudeshna Sarkar Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur
Introduction to Machine Learning Prof. Sudeshna Sarkar Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Module - 5 Lecture - 22 SVM: The Dual Formulation Good morning.
More informationIntroduction to Linear Programming (LP) Mathematical Programming (MP) Concept (1)
Introduction to Linear Programming (LP) Mathematical Programming Concept LP Concept Standard Form Assumptions Consequences of Assumptions Solution Approach Solution Methods Typical Formulations Massachusetts
More informationEcon 508-A FINITE DIMENSIONAL OPTIMIZATION - NECESSARY CONDITIONS. Carmen Astorne-Figari Washington University in St. Louis.
Econ 508-A FINITE DIMENSIONAL OPTIMIZATION - NECESSARY CONDITIONS Carmen Astorne-Figari Washington University in St. Louis August 12, 2010 INTRODUCTION General form of an optimization problem: max x f
More informationThe general programming problem is the nonlinear programming problem where a given function is maximized subject to a set of inequality constraints.
1 Optimization Mathematical programming refers to the basic mathematical problem of finding a maximum to a function, f, subject to some constraints. 1 In other words, the objective is to find a point,
More information2001 Dennis L. Bricker Dept. of Industrial Engineering The University of Iowa. Reducing dimensionality of DP page 1
2001 Dennis L. Bricker Dept. of Industrial Engineering The University of Iowa Reducing dimensionality of DP page 1 Consider a knapsack with a weight capacity of 15 and a volume capacity of 12. Item # Value
More informationHow to Characterize Solutions to Constrained Optimization Problems
How to Characterize Solutions to Constrained Optimization Problems Michael Peters September 25, 2005 1 Introduction A common technique for characterizing maximum and minimum points in math is to use first
More informationEC /11. Math for Microeconomics September Course, Part II Problem Set 1 with Solutions. a11 a 12. x 2
LONDON SCHOOL OF ECONOMICS Professor Leonardo Felli Department of Economics S.478; x7525 EC400 2010/11 Math for Microeconomics September Course, Part II Problem Set 1 with Solutions 1. Show that the general
More informationLecture Notes: Math Refresher 1
Lecture Notes: Math Refresher 1 Math Facts The following results from calculus will be used over and over throughout the course. A more complete list of useful results from calculus is posted on the course
More informationOptimality Conditions
Chapter 2 Optimality Conditions 2.1 Global and Local Minima for Unconstrained Problems When a minimization problem does not have any constraints, the problem is to find the minimum of the objective function.
More informationGradient Descent. Dr. Xiaowei Huang
Gradient Descent Dr. Xiaowei Huang https://cgi.csc.liv.ac.uk/~xiaowei/ Up to now, Three machine learning algorithms: decision tree learning k-nn linear regression only optimization objectives are discussed,
More informationMarch 13, Duality 3
15.53 March 13, 27 Duality 3 There are concepts much more difficult to grasp than duality in linear programming. -- Jim Orlin The concept [of nonduality], often described in English as "nondualism," is
More informationChap 2. Optimality conditions
Chap 2. Optimality conditions Version: 29-09-2012 2.1 Optimality conditions in unconstrained optimization Recall the definitions of global, local minimizer. Geometry of minimization Consider for f C 1
More informationIn view of (31), the second of these is equal to the identity I on E m, while this, in view of (30), implies that the first can be written
11.8 Inequality Constraints 341 Because by assumption x is a regular point and L x is positive definite on M, it follows that this matrix is nonsingular (see Exercise 11). Thus, by the Implicit Function
More informationModern Optimization Theory: Concave Programming
Modern Optimization Theory: Concave Programming 1. Preliminaries 1 We will present below the elements of modern optimization theory as formulated by Kuhn and Tucker, and a number of authors who have followed
More informationGeneralization to inequality constrained problem. Maximize
Lecture 11. 26 September 2006 Review of Lecture #10: Second order optimality conditions necessary condition, sufficient condition. If the necessary condition is violated the point cannot be a local minimum
More informationConstrained optimization.
ams/econ 11b supplementary notes ucsc Constrained optimization. c 2016, Yonatan Katznelson 1. Constraints In many of the optimization problems that arise in economics, there are restrictions on the values
More informationOutline. Roadmap for the NPP segment: 1 Preliminaries: role of convexity. 2 Existence of a solution
Outline Roadmap for the NPP segment: 1 Preliminaries: role of convexity 2 Existence of a solution 3 Necessary conditions for a solution: inequality constraints 4 The constraint qualification 5 The Lagrangian
More informationCalculus and optimization
Calculus an optimization These notes essentially correspon to mathematical appenix 2 in the text. 1 Functions of a single variable Now that we have e ne functions we turn our attention to calculus. A function
More informationFinal Exam Advanced Mathematics for Economics and Finance
Final Exam Advanced Mathematics for Economics and Finance Dr. Stefanie Flotho Winter Term /5 March 5 General Remarks: ˆ There are four questions in total. ˆ All problems are equally weighed. ˆ This is
More informationA Project Report Submitted by. Devendar Mittal 410MA5096. A thesis presented for the degree of Master of Science
PARTICULARS OF NON-LINEAR OPTIMIZATION A Project Report Submitted by Devendar Mittal 410MA5096 A thesis presented for the degree of Master of Science Department of Mathematics National Institute of Technology,
More informationConstrained optimization
Constrained optimization In general, the formulation of constrained optimization is as follows minj(w), subject to H i (w) = 0, i = 1,..., k. where J is the cost function and H i are the constraints. Lagrange
More informationMATH529 Fundamentals of Optimization Constrained Optimization I
MATH529 Fundamentals of Optimization Constrained Optimization I Marco A. Montes de Oca Mathematical Sciences, University of Delaware, USA 1 / 26 Motivating Example 2 / 26 Motivating Example min cost(b)
More information1. f(β) 0 (that is, β is a feasible point for the constraints)
xvi 2. The lasso for linear models 2.10 Bibliographic notes Appendix Convex optimization with constraints In this Appendix we present an overview of convex optimization concepts that are particularly useful
More informationCE 191: Civil & Environmental Engineering Systems Analysis. LEC 17 : Final Review
CE 191: Civil & Environmental Engineering Systems Analysis LEC 17 : Final Review Professor Scott Moura Civil & Environmental Engineering University of California, Berkeley Fall 2014 Prof. Moura UC Berkeley
More informationReview of Optimization Methods
Review of Optimization Methods Prof. Manuela Pedio 20550 Quantitative Methods for Finance August 2018 Outline of the Course Lectures 1 and 2 (3 hours, in class): Linear and non-linear functions on Limits,
More informationUNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems
UNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems Robert M. Freund February 2016 c 2016 Massachusetts Institute of Technology. All rights reserved. 1 1 Introduction
More informationFinite Dimensional Optimization Part I: The KKT Theorem 1
John Nachbar Washington University March 26, 2018 1 Introduction Finite Dimensional Optimization Part I: The KKT Theorem 1 These notes characterize maxima and minima in terms of first derivatives. I focus
More informationFINANCIAL OPTIMIZATION
FINANCIAL OPTIMIZATION Lecture 1: General Principles and Analytic Optimization Philip H. Dybvig Washington University Saint Louis, Missouri Copyright c Philip H. Dybvig 2008 Choose x R N to minimize f(x)
More informationConstrained Optimization
1 / 22 Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University March 30, 2015 2 / 22 1. Equality constraints only 1.1 Reduced gradient 1.2 Lagrange
More informationOptimization. Thomas Dangl. March Thomas Dangl Optimization 1
Optimization Thomas Dangl March 2010 Thomas Dangl Optimization 1 Introduction Course Preparation We use the program package R throughout the course. Please download the package from http://www.r-project.org
More informationLagrange Relaxation and Duality
Lagrange Relaxation and Duality As we have already known, constrained optimization problems are harder to solve than unconstrained problems. By relaxation we can solve a more difficult problem by a simpler
More informationLagrangian Duality. Evelien van der Hurk. DTU Management Engineering
Lagrangian Duality Evelien van der Hurk DTU Management Engineering Topics Lagrange Multipliers Lagrangian Relaxation Lagrangian Duality 2 DTU Management Engineering 42111: Static and Dynamic Optimization
More informationIntroduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.1-4.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following
More informationSummary Notes on Maximization
Division of the Humanities and Social Sciences Summary Notes on Maximization KC Border Fall 2005 1 Classical Lagrange Multiplier Theorem 1 Definition A point x is a constrained local maximizer of f subject
More informationConstrained and Unconstrained Optimization Prof. Adrijit Goswami Department of Mathematics Indian Institute of Technology, Kharagpur
Constrained and Unconstrained Optimization Prof. Adrijit Goswami Department of Mathematics Indian Institute of Technology, Kharagpur Lecture - 01 Introduction to Optimization Today, we will start the constrained
More informationCS711008Z Algorithm Design and Analysis
CS711008Z Algorithm Design and Analysis Lecture 8 Linear programming: interior point method Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China 1 / 31 Outline Brief
More informationLagrangian Duality. Richard Lusby. Department of Management Engineering Technical University of Denmark
Lagrangian Duality Richard Lusby Department of Management Engineering Technical University of Denmark Today s Topics (jg Lagrange Multipliers Lagrangian Relaxation Lagrangian Duality R Lusby (42111) Lagrangian
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationLagrangian Methods for Constrained Optimization
Pricing Communication Networks: Economics, Technology and Modelling. Costas Courcoubetis and Richard Weber Copyright 2003 John Wiley & Sons, Ltd. ISBN: 0-470-85130-9 Appendix A Lagrangian Methods for Constrained
More informationFIN 550 Practice Exam Answers. A. Linear programs typically have interior solutions.
FIN 550 Practice Exam Answers Phil Dybvig. True-False 25 points A. Linear programs typically have interior solutions. False. Unless the objective is zero, all solutions are at the boundary. B. A local
More informationOptimization Theory. Lectures 4-6
Optimization Theory Lectures 4-6 Unconstrained Maximization Problem: Maximize a function f:ú n 6 ú within a set A f ú n. Typically, A is ú n, or the non-negative orthant {x0ú n x$0} Existence of a maximum:
More informationCO 250 Final Exam Guide
Spring 2017 CO 250 Final Exam Guide TABLE OF CONTENTS richardwu.ca CO 250 Final Exam Guide Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4,
More informationSupport Vector Machines: Maximum Margin Classifiers
Support Vector Machines: Maximum Margin Classifiers Machine Learning and Pattern Recognition: September 16, 2008 Piotr Mirowski Based on slides by Sumit Chopra and Fu-Jie Huang 1 Outline What is behind
More informationCS711008Z Algorithm Design and Analysis
.. CS711008Z Algorithm Design and Analysis Lecture 9. Algorithm design technique: Linear programming and duality Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China
More informationIn the original knapsack problem, the value of the contents of the knapsack is maximized subject to a single capacity constraint, for example weight.
In the original knapsack problem, the value of the contents of the knapsack is maximized subject to a single capacity constraint, for example weight. In the multi-dimensional knapsack problem, additional
More informationNumerical Optimization
Constrained Optimization Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on Constrained Optimization Constrained Optimization Problem: min h j (x) 0,
More informationMathematical Foundations -1- Supporting hyperplanes. SUPPORTING HYPERPLANES Key Ideas: Bounding hyperplane for a convex set, supporting hyperplane
Mathematical Foundations -1- Supporting hyperplanes SUPPORTING HYPERPLANES Key Ideas: Bounding hyperplane for a convex set, supporting hyperplane Supporting Prices 2 Production efficient plans and transfer
More informationMore on Lagrange multipliers
More on Lagrange multipliers CE 377K April 21, 2015 REVIEW The standard form for a nonlinear optimization problem is min x f (x) s.t. g 1 (x) 0. g l (x) 0 h 1 (x) = 0. h m (x) = 0 The objective function
More informationReview of Optimization Basics
Review of Optimization Basics. Introduction Electricity markets throughout the US are said to have a two-settlement structure. The reason for this is that the structure includes two different markets:
More informationUniversity of California, Davis Department of Agricultural and Resource Economics ARE 252 Lecture Notes 2 Quirino Paris
University of California, Davis Department of Agricultural and Resource Economics ARE 5 Lecture Notes Quirino Paris Karush-Kuhn-Tucker conditions................................................. page Specification
More informationThe Kuhn-Tucker and Envelope Theorems
The Kuhn-Tucker and Envelope Theorems Peter Ireland EC720.01 - Math for Economists Boston College, Department of Economics Fall 2010 The Kuhn-Tucker and envelope theorems can be used to characterize the
More informationOptimization using Calculus. Optimization of Functions of Multiple Variables subject to Equality Constraints
Optimization using Calculus Optimization of Functions of Multiple Variables subject to Equality Constraints 1 Objectives Optimization of functions of multiple variables subjected to equality constraints
More informationSTATIC LECTURE 4: CONSTRAINED OPTIMIZATION II - KUHN TUCKER THEORY
STATIC LECTURE 4: CONSTRAINED OPTIMIZATION II - KUHN TUCKER THEORY UNIVERSITY OF MARYLAND: ECON 600 1. Some Eamples 1 A general problem that arises countless times in economics takes the form: (Verbally):
More informationUniqueness of Generalized Equilibrium for Box Constrained Problems and Applications
Uniqueness of Generalized Equilibrium for Box Constrained Problems and Applications Alp Simsek Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology Asuman E.
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Duality in Nonlinear Optimization ) Tamás TERLAKY Computing and Software McMaster University Hamilton, January 2004 terlaky@mcmaster.ca Tel: 27780 Optimality
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More informationThe Fundamental Welfare Theorems
The Fundamental Welfare Theorems The so-called Fundamental Welfare Theorems of Economics tell us about the relation between market equilibrium and Pareto efficiency. The First Welfare Theorem: Every Walrasian
More informationStructural and Multidisciplinary Optimization. P. Duysinx and P. Tossings
Structural and Multidisciplinary Optimization P. Duysinx and P. Tossings 2018-2019 CONTACTS Pierre Duysinx Institut de Mécanique et du Génie Civil (B52/3) Phone number: 04/366.91.94 Email: P.Duysinx@uliege.be
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationCE 191: Civil and Environmental Engineering Systems Analysis. LEC 05 : Optimality Conditions
CE 191: Civil and Environmental Engineering Systems Analysis LEC : Optimality Conditions Professor Scott Moura Civil & Environmental Engineering University of California, Berkeley Fall 214 Prof. Moura
More informationMacroeconomic Theory and Analysis Suggested Solution for Midterm 1
Macroeconomic Theory and Analysis Suggested Solution for Midterm February 25, 2007 Problem : Pareto Optimality The planner solves the following problem: u(c ) + u(c 2 ) + v(l ) + v(l 2 ) () {c,c 2,l,l
More informationNonlinear Programming (NLP)
Natalia Lazzati Mathematics for Economics (Part I) Note 6: Nonlinear Programming - Unconstrained Optimization Note 6 is based on de la Fuente (2000, Ch. 7), Madden (1986, Ch. 3 and 5) and Simon and Blume
More informationMidterm Exam - Solutions
EC 70 - Math for Economists Samson Alva Department of Economics, Boston College October 13, 011 Midterm Exam - Solutions 1 Quasilinear Preferences (a) There are a number of ways to define the Lagrangian
More informationExample Problem. Linear Program (standard form) CSCI5654 (Linear Programming, Fall 2013) Lecture-7. Duality
CSCI5654 (Linear Programming, Fall 013) Lecture-7 Duality Lecture 7 Slide# 1 Lecture 7 Slide# Linear Program (standard form) Example Problem maximize c 1 x 1 + + c n x n s.t. a j1 x 1 + + a jn x n b j
More informationOptimization. Charles J. Geyer School of Statistics University of Minnesota. Stat 8054 Lecture Notes
Optimization Charles J. Geyer School of Statistics University of Minnesota Stat 8054 Lecture Notes 1 One-Dimensional Optimization Look at a graph. Grid search. 2 One-Dimensional Zero Finding Zero finding
More informationComputational Finance
Department of Mathematics at University of California, San Diego Computational Finance Optimization Techniques [Lecture 2] Michael Holst January 9, 2017 Contents 1 Optimization Techniques 3 1.1 Examples
More informationPaul Schrimpf. October 17, UBC Economics 526. Constrained optimization. Paul Schrimpf. First order conditions. Second order conditions
UBC Economics 526 October 17, 2012 .1.2.3.4 Section 1 . max f (x) s.t. h(x) = c f : R n R, h : R n R m Draw picture of n = 2 and m = 1 At optimum, constraint tangent to level curve of function Rewrite
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationIE 5531 Midterm #2 Solutions
IE 5531 Midterm #2 s Prof. John Gunnar Carlsson November 9, 2011 Before you begin: This exam has 9 pages and a total of 5 problems. Make sure that all pages are present. To obtain credit for a problem,
More informationCSCI5654 (Linear Programming, Fall 2013) Lecture-8. Lecture 8 Slide# 1
CSCI5654 (Linear Programming, Fall 2013) Lecture-8 Lecture 8 Slide# 1 Today s Lecture 1. Recap of dual variables and strong duality. 2. Complementary Slackness Theorem. 3. Interpretation of dual variables.
More informationOptimality conditions for Equality Constrained Optimization Problems
International Journal of Mathematics and Statistics Invention (IJMSI) E-ISSN: 2321 4767 P-ISSN: 2321-4759 Volume 4 Issue 4 April. 2016 PP-28-33 Optimality conditions for Equality Constrained Optimization
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationIntroduction to Machine Learning Lecture 7. Mehryar Mohri Courant Institute and Google Research
Introduction to Machine Learning Lecture 7 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Convex Optimization Differentiation Definition: let f : X R N R be a differentiable function,
More information