# Constrained Optimization

Save this PDF as:

Size: px
Start display at page: ## Transcription

1 Constrained Optimization Joshua Wilde, revised by Isabel Tecu, Takeshi Suzuki and María José Boccardi August 13, General Problem Consider the following general constrained optimization problem: g 1 (x 1,..., x n ) b 1,..., g k (x 1,..., x n ) b k, h 1 (x 1,..., x n ) = c 1,..., h m (x 1,..., x n ) = c m. The function f(x) is called the objective function, g(x) is called an inequality constraint, and h(x) is called an equality constraint. In the above problem there are k inequality constraints and m equality constraints. In the following we will always assume that f, g and h are C 1 functions, i.e. that they are dierentiable and their derivatives are continuous. Notice that this problem diers from the regular unconstrained optimization problem in that instead of nding the maximum of f(x), we are nding the maximum of f(x) only over the points which satisfy the constraints. Example: Maximize f(x) = x 2 subject to 0 x 1. Solution: We know that f(x) is strictly monotonically increasing over the domain, therefore the maximum (if it exists) must lie at the largest number in the domain. Since we are optimizing over a compact set, the point x = 1 is the maximal number in the domain, and therefore it is the maximum. This problem was easy because we could visualize the graph of f(x) in our minds and see that it was strictly monotonically increasing over the domain. However, we see a method to nd constrained maxima of functions even when we can't picture them in our minds. 2 Equality Constraints 2.1 One Constraint Consider a simple optimization problem with only one constraint: x R h(x 1,..., x n ) = c. Now draw level sets of the function f(x 1,..., x n ). Since we might not be able to achieve the unconstrained maxima of the function due to our constraint, we seek to nd the value of x which gets 1

2 2 Constrained Optimization us onto the highest level curve of f(x) while remaining on the function h(x). Notice also that the function h(x) will be just tangent to the level curve of f(x). Call the point which maximizes the optimization problem x, (also referred to as the maximizer). Since at x the level curve of f(x) is tangent to the curve g(x), it must also be the case that the gradient of f(x ) must be in the same direction as the gradient of h(x ), or where λ is merely a constant. Now dene the Lagrangian the following way: f(x ) = λ h(x ), L(x 1,..., x n, λ) = f(x 1,..., x n ) λ [h(x 1,..., x n ) c] Then setting the partial derivatives of this function with respect to x equal to zero will yield the rst order conditions for a constrained maximum: f(x ) λ h(x ) = 0. Setting the partial derivative with respect to λ equal to zero gives us our original constraint back: h(x ) c = 0. So the rst order conditions for this problem are simply L(x, λ) = 0 Example Maximize f(x 1, x 2 ) = x 1 x 2 subject to h(x 1, x 2 ) x 1 + 4x 2 = 16. Solution: Form the Lagrangian The rst order conditions are From the rst two equations we have L(x 1, x 2 ) = x 1 x 2 λ (x 1 + 4x 2 16) dx 1 = x 2 λ = 0 dx 2 = x 1 4λ = 0 dλ = x 1 + 4x 2 16 = 0 x 1 = 4x 2. Plugging this into the last equation we have that x 2 = 2, which in turn implies that x 1 = 8, and that λ = 2. This states that the only candidate for the solution is (x 1, x 2, λ) = (8, 2, 2). Remember that points obtained using this formula may or may not be a maximum or minimum, since the rst order conditions are only necessary conditions. They only give us candidate solutions. There is another more subtle way that this process may fail, however. Consider the case where h(x ) = 0, or in other words, the point which maximizes f(x) is also a critical point of h(x). Remember our necessary condition for a maximum f(x ) = λ h(x ), Since h(x ) = 0, this implies that f(x ) = 0. However, this the necessary condition for an unconstrained optimization problem, not a constrained one! In eect, when h(x ) = 0, the constraint is no longer taken into account in the problem, and therefore we arrive at the wrong solution.

3 Math Camp Many Constraints Consider the problem h 1 (x 1,..., x n ) = c 1,..., h m (x 1,..., x n ) = c m. Let's rst talk about how the lagrangian approach might fail. As we saw for one constraint, if h(x ) = 0, then the constraint drops out of the equation. Now consider the Jacobian matrix, or a vector of the gradients of the dierent h i (x ). Dh(x ) = h 1 (x ). h m (x ) = dh 1(x ) dh 1(x ) dx n dx dh m(x ) dx 1... dh m(x ) dx n Notice that if any of the h i (x )'s is zero, then that constraint will not be taken into account in the analysis. Also, there will be a row of zeros in the Jacobian, and therefore the Jacobian will not be full rank. The generalization of the condition that h(x ) 0 for the case when m = 1 is that the Jacobian matrix must be of rank m. Otherwise, one of the constraints is not being taken into account, and the analysis fails. We call this condition the non-degenerate constraint qualication (NDCQ). Note that we only have to check whether the NDCQ holds at points in the constraint set, since points outside the constraint set are not solution candidates anyways. If we test for NDCQ and nd that the constraint is violated for some point within our constraint set, we have to add this point to our candidate solution set. The Lagrangian technique simply does not give us any information about this point. The Lagrangian for the multi-constraint optimization problem is L(x 1,..., x n, λ) = f(x 1,..., x n ) λ i [h i (x 1,..., x n ) c i ] And therefore the necessary conditions for a maximum are dx 1 = 0,..., dx n = 0. = 0,..., = 0. dλ 1 dλ n Example Maximize f(x, y, z) = xyz subject to h 1 (x, y, z) x 2 + y 2 = 1 and h 2 (x, y, z) x + z = 1. First let us form the Jacobian matrix Dh(x, y, z) = ( 2x 2y Notice that the rank is 1 if and only if both x = y = 0. However, if this is the case, then our rst constraint fails to hold. Therefore, the rank is 2 for all points in the constraint set, and so we don't )

4 4 Constrained Optimization need to worry about the NDCQ. Form the Lagrangean The rst order conditions are L(x, y, z) = xyz λ 1 ( x 2 + y 2 1 ) λ 2 (x + z 1). x = yz 2λ 1x λ 2 = 0 y = xz 2λ 1y = 0 z = xy λ 2 = 0 λ 1 = x 2 + y 2 1 = 0 λ 2 = x + z 1 = 0 In order to solve for λ 1 and λ 2 we divide by y, so we have to assume y 0. We will treat the case y = 0 later. Assuming y 0, λ 1 = xz 2y, λ 2 = xy. Plug these into the rst equation to get Multiply both sides by y 0 yz 2 x2 z 2y xy = 0 y 2 z x 2 z xy 2 = 0 Now we want to solve the two constraints for y and z in terms of x, plug them into this equation, and get one equation in terms of x (1 x 2 )(1 x) x 2 (1 x) x(1 x 2 ) = 0 (1 x) [ (1 x 2 ) x 2 x(1 + x) ] = 0 (1 x) [ 1 3x 2 x ] = 0 { } This yields x = 1, , Let's analyze x = 1 rst. From the second constraint we have that z = 0, and from the rst constraint we have that y = 0. That contradicts our assumption that y 0, so this cannot be a solution. Plugging in the other values, we get four candidate solutions x =.4343, y = ±0.9008, z = x =.7676, y = ±0.6409, z = Finally, let's look at the case y = 0. Then either x = 1 and z = 0 or x = 1 and z = 2, from the equality constraints. In the rst case, all the other rst order conditions hold as well, so (1, 0, 0) is another candidate solution. In the second case, we get 2 = 0 in the second FOC and therefore this point cannot be a candidate solution.

5 Math Camp 5 3 Inequality Constraints 3.1 One Constraint Consider the problem g(x 1,..., x n ) b. In this case the solution is not constrained to the curve g(x), but merely bounded by it. In order to understand the new conditions, imagine the graph of the level sets which we talked about before. Instead of being constrained to the function g(x), the domain is now bounded by it instead. However, the boundary of the function is still the same as before. Notice that there is still a point where the boundary is tangent to some level set of g(x). The question now is whether the boundary is binding or not binding. Remember that we are trying to nd candidate points for a global maximum. Restricting our original domain X to the set of points where g(x) b gives us two types of candidate points: Points on the boundary g(x) = b where g(x) = b is tangent to the level curves of f(x), and local maxima of f(x) for which g(x) b. The rst type we can nd by the constrained FOC f(x) = λ g(x), and the second type we can nd by the unconstrained FOC f(x) = 0. Let's look at each of these in turn. Case 1: Candidates along the boundary (constraint binding) This is the case where an unconstrained maximum lies outside of the constraint set. In other words, the inequality constrains us from reaching a maximum of f. In this case, the gradient of f(x) is going to point in the steepest direction up the graph. The gradient of g(x) points to the set g(x) b (since it points in the direction of increasing g(x)). Therefore g(x) is pointing in the same direction as f(x), which implies that λ 0. So necessary conditions for a solution on the boundary are f(x) λ g(x) = 0 and λ 0. Case 2: Candidates not on the boundary (constraint not binding) This is the case where an unconstrained maximum lies inside the constraint set. In other words, the inequality does not constrain us from reaching this maximum of f. The rst order condition is simply f(x) = 0, which we can rewrite (to take the same shape as above) as f(x) λ g(x) = 0 and λ = 0. In summary, either the constraint is binding (tight), that is g(x) b = 0 and λ 0, or the constraint is not-binding (slack), and then λ = 0. We can summarize this new set of conditions as what is called the complementary slackness conditions [g(x) b] λ = 0. This works because if the constraint is binding, then g(x) b = 0, and if the constraint is not binding, then we want to ignore it by having λ = 0. As we can see, it does not matter whether the constraint binds or does not bind, the Lagrangian multiplier must always be greater than or equal to 0. Therefore, another new set of conditions are λ i 0 i. Finally, we have include our original inequality constraint g(x) b.

6 6 Constrained Optimization We can summarize all these FOC in terms of the Lagrangian, which we dene as before to be L(x, λ) = f(x) λ(g(x) b): (x, λ) x i = f(x) x i λ g(x) x i = 0 for alli = 1,..., n (x, λ) λ = λ [g(x) b] = 0 complementary slackness λ (x, λ) λ = [g(x) b] 0 original constraint λ Many Constraints Consider the problem: g 1 (x 1,..., x n ) b 1,..., g k (x 1,..., x n ) b k. In order to understand the new NDCQ, we must realize that if a constraint does not bind, we don't care if it drops out of the equation. The point of the NDCQ was to ensure that binding constraints do not drop out. Therefore, the NDCQ with inequality constraints is the same as the equality constraints, except for the fact that we only care about the Jacobian matrix of the binding constraints, or the Jacobian for the constraints with λ i > 0. (Notice that we cannot tell in advance which constraints will be binding, so we need to check all of them when we check the NDCQ before computing the solution candidates.) The following rst order conditions will tell us the candidate points for a maximum x 1 = 0,..., x n = 0 [g 1 (x 1,..., x n ) b 1 ] λ 1 = 0,..., [g k (x 1,..., x n ) b k ] λ k = 0 g 1 (x 1,..., x n ) b 1,..., g k (x 1,..., x n ) b k λ 1 0,..., λ k 0 4 Mixed constraints Now consider the general problem, in which we have equality and inequality constraints: g 1 (x 1,..., x n ) b 1,..., g k (x 1,..., x n ) b k, h 1 (x 1,..., x n ) = c 1,..., h m (x 1,..., x n ) = c m. Summarizing the conditions for equality and inequality constraints that we found above we can formulate the following theorem:

7 Math Camp 7 Theorem: Suppose that x is a local maximizer of f on the constraint set given by the k inequality and m equalities above. Without loss of generality, assume that the rst k 0 inequality constraints are binding and that the other k k 0 inequality constraints are not binding. Further suppose that the Jacobian of the equality constraints and the binding inequality constraints at x has full rank. Form the Lagrangian L(x, λ 1,..., λ k, µ 1,..., µ m ) = f(x) k λ i [g i (x) b i ] µ i [h i (x) c i ] Then there exist multipliers λ 1,..., λ k and µ 1,..., µ m such that 1. x i (x, λ, µ ) = 0 for all i {1,..., n} 2. λ i [g i(x) b i ] = 0 for all i {1,..., k} 3. h i (x ) = c i for all i {1,..., m} 4. g i (x ) b i for all i {1,..., k} 5. λ i 0 for all i {1,..., k} Note again that this theorem gives us merely rst order, necessary conditions: If x is a maximum and if the NDCQ holds at x, then there exist Lagrange multipliers for which the conditions hold true. Finding all tuples (x, λ, µ) for which the conditions hold will therefore gives us a set of candidate solutions. We still need to check whether these candidates are actually maximizers. (Conditions that can be used to do so will be taught in your rst semester math class, or you can read Simon & Blume, chapter 19). Notice also that we may not nd all candidate solutions using the Lagrangian method if the NDCQ is violated at some points in the constraint set. Meaning of the multiplier: See Simon & Blume, Chapter Minimization Problems So far we have only discussed maximization problems. What happens if we are instead looking to minimize a function given certain constraints? There are dierent ways in which we can change the above technique of using the Lagrangian to apply it to minimization problems. The easiest is probably the following: Suppose we are given a minimization problem min f(x 1,..., x n ) subject to : g 1 (x 1,..., x n ) b 1,..., g k (x 1,..., x n ) b k, h 1 (x 1,..., x n ) = c 1,..., h k (x 1,..., x n ) = c k. Finding the minimum of f on a certain domain is really the same as nding the maximum of f on that domain. So we can transform the above problem into the maximization problem max f(x 1,..., x n ) subject to : g 1 (x 1,..., x n ) b 1,..., g k (x 1,..., x n ) b k, h 1 (x 1,..., x n ) = c 1,..., h k (x 1,..., x n ) = c k. We can nd candidate solutions for this problem as discussed above. They will also be candidate solutions of the original minimization problem.

8 8 Constrained Optimization 6 Kuhn-Tucker Notation The most common problems in economics are maximization problems dealing with only inequality constraints. Many of these constraints come in the form of non-negativity constraints, such as requiring consumption to be weakly positive. Consider the following problem: max x R n f(x 1,..., x n ) subject to g 1 (x 1 b 1,..., x n ),..., g m (x 1,..., x n ) b m and x 1 0,..., x n 0 The Lagrangian function we would write would take the form L(x, λ, v) = f(x) λ i (g i (x) b i ) + which would lead to the following rst order conditions: = f x 1 x 1 λ i g i x 1 + v 1 = 0,..., x n = f x n n v i x i, λ 1 [g 1 (x) b 1 ] = 0,..., λ m [g m (x) b m ] = 0 g 1 (x 1,..., x n ) b 1,..., g m (x 1,..., x n ) b m λ i g i x n + v n = 0 v 1 x 1 = 0,..., v n x n = 0, and λ i, x j 0 i = 1,..., m and j = 1,..., n That's a lot of conditions! (3m + 3n) Now consider the Lagrangian without the nonnegativity constraints, and call it the Kuhn-Tucker Lagrangian: L(x, λ, v) = L(x, n λ) + v i x i The rst n rst order conditions can be rewritten as which implies = + v j = 0 j = 1,..., n, x j x j x j = v j j = 1,..., n. Plugging those into the nonnegativity constraints we have that x j x j = 0 and x j 0. Also notice that which implies = = b j g j (x) 0 j = 1,..., m, λ j λ j λ j 0 and λ j λ j = 0 j = 1,..., m.

9 Math Camp 9 In summary, the following are the rst order conditions for the Kuhn-Tucker Lagrangian: 0 and x L j = 0 j = 1,..., n x j x j 0 and λ L j = 0 j = 1,..., m λ j λ j λ j 0 j = 1,... m This is 2n + 3m constraints, n less than before. If we are dealing with a lot of non-negativity constraints it is therefore faster to use the Kuhn-Tucker Lagrangian. Exercises 1. There are two commodities: x and y. Let the consumer's consumption set be R 2 + and his preference relation on his consumption set be represented by u(x, y) = (x 4) 2 y 2. When his initial wealth is 2 and the relative price is 1, solve his utility maximization problem if it is well dened. 2. Let f : R + R and f(x) = (x + 1) Solve the maximization problem if it is well dened. 3. Let f : R 2 + R and f(x, y) = 2y x 2. When (x, y) must be on the unit disc, i.e., x 2 + y 2 1, solve the minimization problem if it is well dened. 1 7 Homework 1. In chapter 18 of Simon and Blume, numbers 3, 6, 7, 10, 11, and This is the same problem as in Example of Simon and Blume (1994).

### Seminars on Mathematics for Economics and Finance Topic 5: Optimization Kuhn-Tucker conditions for problems with inequality constraints 1 Seminars on Mathematics for Economics and Finance Topic 5: Optimization Kuhn-Tucker conditions for problems with inequality constraints 1 Session: 15 Aug 2015 (Mon), 10:00am 1:00pm I. Optimization with

### Multi Variable Calculus Multi Variable Calculus Joshua Wilde, revised by Isabel Tecu, Takeshi Suzuki and María José Boccardi August 3, 03 Functions from R n to R m So far we have looked at functions that map one number to another

### The Kuhn-Tucker Problem Natalia Lazzati Mathematics for Economics (Part I) Note 8: Nonlinear Programming - The Kuhn-Tucker Problem Note 8 is based on de la Fuente (2000, Ch. 7) and Simon and Blume (1994, Ch. 18 and 19). The Kuhn-Tucker

### Dynamical Systems. August 13, 2013 Dynamical Systems Joshua Wilde, revised by Isabel Tecu, Takeshi Suzuki and María José Boccardi August 13, 2013 Dynamical Systems are systems, described by one or more equations, that evolve over time.

### Basic Proof Techniques Basic Proof Techniques Joshua Wilde, revised by Isabel Tecu, Takeshi Suzuki and María José Boccardi August 13, 2013 1 Basic Notation The following is standard notation for proofs: A B. A implies B. A B.

### Paul Schrimpf. October 17, UBC Economics 526. Constrained optimization. Paul Schrimpf. First order conditions. Second order conditions UBC Economics 526 October 17, 2012 .1.2.3.4 Section 1 . max f (x) s.t. h(x) = c f : R n R, h : R n R m Draw picture of n = 2 and m = 1 At optimum, constraint tangent to level curve of function Rewrite

### Optimization. A first course on mathematics for economists Optimization. A first course on mathematics for economists Xavier Martinez-Giralt Universitat Autònoma de Barcelona xavier.martinez.giralt@uab.eu II.3 Static optimization - Non-Linear programming OPT p.1/45

### Modern Optimization Theory: Concave Programming Modern Optimization Theory: Concave Programming 1. Preliminaries 1 We will present below the elements of modern optimization theory as formulated by Kuhn and Tucker, and a number of authors who have followed

### Introduction to Real Analysis Introduction to Real Analysis Joshua Wilde, revised by Isabel Tecu, Takeshi Suzuki and María José Boccardi August 13, 2013 1 Sets Sets are the basic objects of mathematics. In fact, they are so basic that

### MATH2070 Optimisation MATH2070 Optimisation Nonlinear optimisation with constraints Semester 2, 2012 Lecturer: I.W. Guo Lecture slides courtesy of J.R. Wishart Review The full nonlinear optimisation problem with equality constraints

### Nonlinear Programming and the Kuhn-Tucker Conditions Nonlinear Programming and the Kuhn-Tucker Conditions The Kuhn-Tucker (KT) conditions are first-order conditions for constrained optimization problems, a generalization of the first-order conditions we

### The Kuhn-Tucker and Envelope Theorems The Kuhn-Tucker and Envelope Theorems Peter Ireland EC720.01 - Math for Economists Boston College, Department of Economics Fall 2010 The Kuhn-Tucker and envelope theorems can be used to characterize the

### Mathematical Foundations -1- Constrained Optimization. Constrained Optimization. An intuitive approach 2. First Order Conditions (FOC) 7 Mathematical Foundations -- Constrained Optimization Constrained Optimization An intuitive approach First Order Conditions (FOC) 7 Constraint qualifications 9 Formal statement of the FOC for a maximum

### EC /11. Math for Microeconomics September Course, Part II Problem Set 1 with Solutions. a11 a 12. x 2 LONDON SCHOOL OF ECONOMICS Professor Leonardo Felli Department of Economics S.478; x7525 EC400 2010/11 Math for Microeconomics September Course, Part II Problem Set 1 with Solutions 1. Show that the general

### Math 1270 Honors ODE I Fall, 2008 Class notes # 14. x 0 = F (x; y) y 0 = G (x; y) u 0 = au + bv = cu + dv Math 1270 Honors ODE I Fall, 2008 Class notes # 1 We have learned how to study nonlinear systems x 0 = F (x; y) y 0 = G (x; y) (1) by linearizing around equilibrium points. If (x 0 ; y 0 ) is an equilibrium

### Math Camp Notes: Everything Else Math Camp Notes: Everything Else Systems of Dierential Equations Consider the general two-equation system of dierential equations: Steady States ẋ = f(x, y ẏ = g(x, y Just as before, we can nd the steady

### Econ 508-A FINITE DIMENSIONAL OPTIMIZATION - NECESSARY CONDITIONS. Carmen Astorne-Figari Washington University in St. Louis. Econ 508-A FINITE DIMENSIONAL OPTIMIZATION - NECESSARY CONDITIONS Carmen Astorne-Figari Washington University in St. Louis August 12, 2010 INTRODUCTION General form of an optimization problem: max x f

### EC /11. Math for Microeconomics September Course, Part II Lecture Notes. Course Outline LONDON SCHOOL OF ECONOMICS Professor Leonardo Felli Department of Economics S.478; x7525 EC400 20010/11 Math for Microeconomics September Course, Part II Lecture Notes Course Outline Lecture 1: Tools for

### STATIC LECTURE 4: CONSTRAINED OPTIMIZATION II - KUHN TUCKER THEORY STATIC LECTURE 4: CONSTRAINED OPTIMIZATION II - KUHN TUCKER THEORY UNIVERSITY OF MARYLAND: ECON 600 1. Some Eamples 1 A general problem that arises countless times in economics takes the form: (Verbally):

### E 600 Chapter 4: Optimization E 600 Chapter 4: Optimization Simona Helmsmueller August 8, 2018 Goals of this lecture: Every theorem in these slides is important! You should understand, remember and be able to apply each and every one

### In view of (31), the second of these is equal to the identity I on E m, while this, in view of (30), implies that the first can be written 11.8 Inequality Constraints 341 Because by assumption x is a regular point and L x is positive definite on M, it follows that this matrix is nonsingular (see Exercise 11). Thus, by the Implicit Function

### 8. Constrained Optimization 8. Constrained Optimization Daisuke Oyama Mathematics II May 11, 2018 Unconstrained Maximization Problem Let X R N be a nonempty set. Definition 8.1 For a function f : X R, x X is a (strict) local maximizer

### MATHEMATICAL ECONOMICS: OPTIMIZATION. Contents MATHEMATICAL ECONOMICS: OPTIMIZATION JOÃO LOPES DIAS Contents 1. Introduction 2 1.1. Preliminaries 2 1.2. Optimal points and values 2 1.3. The optimization problems 3 1.4. Existence of optimal points 4

### IE 5531: Engineering Optimization I IE 5531: Engineering Optimization I Lecture 12: Nonlinear optimization, continued Prof. John Gunnar Carlsson October 20, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I October 20,

### Optimization. A first course on mathematics for economists Problem set 4: Classical programming Optimization. A first course on mathematics for economists Problem set 4: Classical programming Xavier Martinez-Giralt Academic Year 2015-2016 4.1 Let f(x 1, x 2 ) = 2x 2 1 + x2 2. Solve the following

### Economics 101. Lecture 2 - The Walrasian Model and Consumer Choice Economics 101 Lecture 2 - The Walrasian Model and Consumer Choice 1 Uncle Léon The canonical model of exchange in economics is sometimes referred to as the Walrasian Model, after the early economist Léon

### Constrained maxima and Lagrangean saddlepoints Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analysis and Economic Theory Winter 2018 Topic 10: Constrained maxima and Lagrangean saddlepoints 10.1 An alternative As an application

### How to Characterize Solutions to Constrained Optimization Problems How to Characterize Solutions to Constrained Optimization Problems Michael Peters September 25, 2005 1 Introduction A common technique for characterizing maximum and minimum points in math is to use first

### Review of Optimization Methods Review of Optimization Methods Prof. Manuela Pedio 20550 Quantitative Methods for Finance August 2018 Outline of the Course Lectures 1 and 2 (3 hours, in class): Linear and non-linear functions on Limits,

### 2015 Math Camp Calculus Exam Solution 015 Math Camp Calculus Exam Solution Problem 1: x = x x +5 4+5 = 9 = 3 1. lim We also accepted ±3, even though it is not according to the prevailing convention 1. x x 4 x+4 =. lim 4 4+4 = 4 0 = 4 0 = We

### Constrained optimization Constrained optimization In general, the formulation of constrained optimization is as follows minj(w), subject to H i (w) = 0, i = 1,..., k. where J is the cost function and H i are the constraints. Lagrange

### Econ Slides from Lecture 14 Econ 205 Sobel Econ 205 - Slides from Lecture 14 Joel Sobel September 10, 2010 Theorem ( Lagrange Multipliers ) Theorem If x solves max f (x) subject to G(x) = 0 then there exists λ such that Df (x ) =

### Microeconomics I. September, c Leopold Sögner Microeconomics I c Leopold Sögner Department of Economics and Finance Institute for Advanced Studies Stumpergasse 56 1060 Wien Tel: +43-1-59991 182 soegner@ihs.ac.at http://www.ihs.ac.at/ soegner September,

### Constrained Optimization 1 / 22 Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University March 30, 2015 2 / 22 1. Equality constraints only 1.1 Reduced gradient 1.2 Lagrange

### Lecture Notes: Math Refresher 1 Lecture Notes: Math Refresher 1 Math Facts The following results from calculus will be used over and over throughout the course. A more complete list of useful results from calculus is posted on the course

### Final Exam - Math Camp August 27, 2014 Final Exam - Math Camp August 27, 2014 You will have three hours to complete this exam. Please write your solution to question one in blue book 1 and your solutions to the subsequent questions in blue

### MATH529 Fundamentals of Optimization Constrained Optimization I MATH529 Fundamentals of Optimization Constrained Optimization I Marco A. Montes de Oca Mathematical Sciences, University of Delaware, USA 1 / 26 Motivating Example 2 / 26 Motivating Example min cost(b)

### Optimization Over Time Optimization Over Time Joshua Wilde, revised by Isabel Tecu and Takeshi Suzuki August 26, 21 Up to this point, we have only considered constrained optimization problems at a single point in time. However,

### Nonlinear Programming (NLP) Natalia Lazzati Mathematics for Economics (Part I) Note 6: Nonlinear Programming - Unconstrained Optimization Note 6 is based on de la Fuente (2000, Ch. 7), Madden (1986, Ch. 3 and 5) and Simon and Blume

### Generalization to inequality constrained problem. Maximize Lecture 11. 26 September 2006 Review of Lecture #10: Second order optimality conditions necessary condition, sufficient condition. If the necessary condition is violated the point cannot be a local minimum

### Problem Set 2 Solutions EC 720 - Math for Economists Samson Alva Department of Economics Boston College October 4 2011 1. Profit Maximization Problem Set 2 Solutions (a) The Lagrangian for this problem is L(y k l λ) = py rk wl

### Linear Algebra: Linear Systems and Matrices - Quadratic Forms and Deniteness - Eigenvalues and Markov Chains Linear Algebra: Linear Systems and Matrices - Quadratic Forms and Deniteness - Eigenvalues and Markov Chains Joshua Wilde, revised by Isabel Tecu, Takeshi Suzuki and María José Boccardi August 3, 3 Systems

### Constrained Optimization. Unconstrained Optimization (1) Constrained Optimization Unconstrained Optimization (Review) Constrained Optimization Approach Equality constraints * Lagrangeans * Shadow prices Inequality constraints * Kuhn-Tucker conditions * Complementary

### September Math Course: First Order Derivative September Math Course: First Order Derivative Arina Nikandrova Functions Function y = f (x), where x is either be a scalar or a vector of several variables (x,..., x n ), can be thought of as a rule which

### UNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems UNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems Robert M. Freund February 2016 c 2016 Massachusetts Institute of Technology. All rights reserved. 1 1 Introduction

### Outline. Roadmap for the NPP segment: 1 Preliminaries: role of convexity. 2 Existence of a solution Outline Roadmap for the NPP segment: 1 Preliminaries: role of convexity 2 Existence of a solution 3 Necessary conditions for a solution: inequality constraints 4 The constraint qualification 5 The Lagrangian

### Recitation 2-09/01/2017 (Solution) Recitation 2-09/01/2017 (Solution) 1. Checking properties of the Cobb-Douglas utility function. Consider the utility function u(x) Y n i1 x i i ; where x denotes a vector of n di erent goods x 2 R n +,

### . This matrix is not symmetric. Example. Suppose A = Notes for Econ. 7001 by Gabriel A. ozada The equation numbers and page numbers refer to Knut Sydsæter and Peter J. Hammond s textbook Mathematics for Economic Analysis (ISBN 0-13- 583600-X, 1995). 1. Convexity,

### Final Exam Advanced Mathematics for Economics and Finance Final Exam Advanced Mathematics for Economics and Finance Dr. Stefanie Flotho Winter Term /5 March 5 General Remarks: ˆ There are four questions in total. ˆ All problems are equally weighed. ˆ This is

### 3.7 Constrained Optimization and Lagrange Multipliers 3.7 Constrained Optimization and Lagrange Multipliers 71 3.7 Constrained Optimization and Lagrange Multipliers Overview: Constrained optimization problems can sometimes be solved using the methods of the

### Recitation #2 (August 31st, 2018) Recitation #2 (August 1st, 2018) 1. [Checking properties of the Cobb-Douglas utility function.] Consider the utility function u(x) = n i=1 xα i i, where x denotes a vector of n different goods x R n +,

### ARE202A, Fall Contents ARE202A, Fall 2005 LECTURE #2: WED, NOV 6, 2005 PRINT DATE: NOVEMBER 2, 2005 (NPP2) Contents 5. Nonlinear Programming Problems and the Kuhn Tucker conditions (cont) 5.2. Necessary and sucient conditions

### OPTIMISATION /09 EXAM PREPARATION GUIDELINES General: OPTIMISATION 2 2008/09 EXAM PREPARATION GUIDELINES This points out some important directions for your revision. The exam is fully based on what was taught in class: lecture notes, handouts and

### OPTIMISATION 2007/8 EXAM PREPARATION GUIDELINES General: OPTIMISATION 2007/8 EXAM PREPARATION GUIDELINES This points out some important directions for your revision. The exam is fully based on what was taught in class: lecture notes, handouts and homework.

### Let f(x) = x, but the domain of f is the interval 0 x 1. Note I.g Maximum and Minimum. Lagrange Multipliers Recall: Suppose we are given y = f(x). We recall that the maximum/minimum points occur at the following points: (1) where f = 0; (2) where f does not exist;

### Optimality Conditions Chapter 2 Optimality Conditions 2.1 Global and Local Minima for Unconstrained Problems When a minimization problem does not have any constraints, the problem is to find the minimum of the objective function.

### Lagrange Multipliers Lagrange Multipliers 3-15-2018 The method of Lagrange multipliers deals with the problem of finding the maxima and minima of a function subject to a side condition, or constraint. Example. Find graphically

### MATH 4211/6211 Optimization Constrained Optimization MATH 4211/6211 Optimization Constrained Optimization Xiaojing Ye Department of Mathematics & Statistics Georgia State University Xiaojing Ye, Math & Stat, Georgia State University 0 Constrained optimization

### Economics 101A (Lecture 3) Stefano DellaVigna Economics 101A (Lecture 3) Stefano DellaVigna January 24, 2017 Outline 1. Implicit Function Theorem 2. Envelope Theorem 3. Convexity and concavity 4. Constrained Maximization 1 Implicit function theorem

### CHAPTER 1-2: SHADOW PRICES Essential Microeconomics -- CHAPTER -: SHADOW PRICES An intuitive approach: profit maimizing firm with a fied supply of an input Shadow prices 5 Concave maimization problem 7 Constraint qualifications

### Math 10C Practice Final Solutions Math 1C Practice Final Solutions March 9, 216 1. (6 points) Let f(x, y) x 3 y + 12x 2 8y. (a) Find all critical points of f. SOLUTION: f x 3x 2 y + 24x 3x(xy + 8) x,xy 8 y 8 x f y x 3 8 x 3 8 x 3 8 2 So

### Tutorial 3: Optimisation Tutorial : Optimisation ECO411F 011 1. Find and classify the extrema of the cubic cost function C = C (Q) = Q 5Q +.. Find and classify the extreme values of the following functions (a) y = x 1 + x x 1x

### EconS 301. Math Review. Math Concepts EconS 301 Math Review Math Concepts Functions: Functions describe the relationship between input variables and outputs y f x where x is some input and y is some output. Example: x could number of Bananas

### Optimality Conditions for Constrained Optimization 72 CHAPTER 7 Optimality Conditions for Constrained Optimization 1. First Order Conditions In this section we consider first order optimality conditions for the constrained problem P : minimize f 0 (x)

### Constrained optimization. ams/econ 11b supplementary notes ucsc Constrained optimization. c 2016, Yonatan Katznelson 1. Constraints In many of the optimization problems that arise in economics, there are restrictions on the values

### Dynamic Macroeconomic Theory Notes. David L. Kelly. Department of Economics University of Miami Box Coral Gables, FL Dynamic Macroeconomic Theory Notes David L. Kelly Department of Economics University of Miami Box 248126 Coral Gables, FL 33134 dkelly@miami.edu Current Version: Fall 2013/Spring 2013 I Introduction A

### Review for the First Midterm Exam Review for the First Midterm Exam Thomas Morrell 5 pm, Sunday, 4 April 9 B9 Van Vleck Hall For the purpose of creating questions for this review session, I did not make an effort to make any of the numbers

### The Fundamental Welfare Theorems The Fundamental Welfare Theorems The so-called Fundamental Welfare Theorems of Economics tell us about the relation between market equilibrium and Pareto efficiency. The First Welfare Theorem: Every Walrasian

### Lakehead University ECON 4117/5111 Mathematical Economics Fall 2002 Test 1 September 20, 2002 1. Determine whether each of the following is a statement or not (answer yes or no): (a) Some sentences can be labelled true and false. (b) All students should study mathematics.

### 1 Objective. 2 Constrained optimization. 2.1 Utility maximization. Dieter Balkenborg Department of Economics BEE020 { Basic Mathematical Economics Week 2, Lecture Thursday 2.0.0 Constrained optimization Dieter Balkenborg Department of Economics University of Exeter Objective We give the \ rst order conditions"

### Chapter 4. Maximum Theorem, Implicit Function Theorem and Envelope Theorem Chapter 4. Maximum Theorem, Implicit Function Theorem and Envelope Theorem This chapter will cover three key theorems: the maximum theorem (or the theorem of maximum), the implicit function theorem, and

### Lecture 4: Optimization. Maximizing a function of a single variable Lecture 4: Optimization Maximizing or Minimizing a Function of a Single Variable Maximizing or Minimizing a Function of Many Variables Constrained Optimization Maximizing a function of a single variable

### Optimization Theory. Lectures 4-6 Optimization Theory Lectures 4-6 Unconstrained Maximization Problem: Maximize a function f:ú n 6 ú within a set A f ú n. Typically, A is ú n, or the non-negative orthant {x0ú n x\$0} Existence of a maximum:

### Contents. 2 Partial Derivatives. 2.1 Limits and Continuity. Calculus III (part 2): Partial Derivatives (by Evan Dummit, 2017, v. 2. Calculus III (part 2): Partial Derivatives (by Evan Dummit, 2017, v 260) Contents 2 Partial Derivatives 1 21 Limits and Continuity 1 22 Partial Derivatives 5 23 Directional Derivatives and the Gradient

### OPTIMIZATION THEORY IN A NUTSHELL Daniel McFadden, 1990, 2003 OPTIMIZATION THEORY IN A NUTSHELL Daniel McFadden, 1990, 2003 UNCONSTRAINED OPTIMIZATION 1. Consider the problem of maximizing a function f:ú n 6 ú within a set A f ú n. Typically, A might be all of ú

### The Necessity of the Transversality Condition at Infinity: A (Very) Special Case The Necessity of the Transversality Condition at Infinity: A (Very) Special Case Peter Ireland ECON 772001 - Math for Economists Boston College, Department of Economics Fall 2017 Consider a discrete-time,

### ISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints ISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints Instructor: Prof. Kevin Ross Scribe: Nitish John October 18, 2011 1 The Basic Goal The main idea is to transform a given constrained

### ARE211, Fall 2005 CONTENTS. 5. Characteristics of Functions Surjective, Injective and Bijective functions. 5.2. ARE211, Fall 2005 LECTURE #18: THU, NOV 3, 2005 PRINT DATE: NOVEMBER 22, 2005 (COMPSTAT2) CONTENTS 5. Characteristics of Functions. 1 5.1. Surjective, Injective and Bijective functions 1 5.2. Homotheticity

### x 1 1 and p 1 1 Two points if you just talk about monotonicity (u (c) > 0). . (a) (8 points) What does it mean for observations x and p... x T and p T to be rationalized by a monotone utility function? Notice that this is a one good economy. For all t, p t x t function. p t x

### Week 4: Calculus and Optimization (Jehle and Reny, Chapter A2) Week 4: Calculus and Optimization (Jehle and Reny, Chapter A2) Tsun-Feng Chiang *School of Economics, Henan University, Kaifeng, China September 27, 2015 Microeconomic Theory Week 4: Calculus and Optimization

### f( x) f( y). Functions which are not one-to-one are often called many-to-one. Take the domain and the range to both be all the real numbers: I. UNIVARIATE CALCULUS Given two sets X and Y, a function is a rule that associates each member of X with exactly one member of Y. That is, some x goes in, and some y comes out. These notations are used

### Support Vector Machines Support Vector Machines Support vector machines (SVMs) are one of the central concepts in all of machine learning. They are simply a combination of two ideas: linear classification via maximum (or optimal

### ECON2285: Mathematical Economics ECON2285: Mathematical Economics Yulei Luo SEF of HKU September 9, 2017 Luo, Y. (SEF of HKU) ME September 9, 2017 1 / 81 Constrained Static Optimization So far we have focused on nding the maximum or minimum

### Lagrange Multipliers Lagrange Multipliers (Com S 477/577 Notes) Yan-Bin Jia Nov 9, 2017 1 Introduction We turn now to the study of minimization with constraints. More specifically, we will tackle the following problem: minimize

### Bob Brown Math 251 Calculus 1 Chapter 4, Section 4 1 CCBC Dundalk Bob Brown Math 251 Calculus 1 Chapter 4, Section 4 1 A Function and its Second Derivative Recall page 4 of Handout 3.1 where we encountered the third degree polynomial f(x) = x 3 5x 2 4x + 20. Its derivative

### The Kuhn-Tucker and Envelope Theorems The Kuhn-Tucker and Envelope Theorems Peter Ireland ECON 77200 - Math for Economists Boston College, Department of Economics Fall 207 The Kuhn-Tucker and envelope theorems can be used to characterize the

### BEEM103 UNIVERSITY OF EXETER. BUSINESS School. January 2009 Mock Exam, Part A. OPTIMIZATION TECHNIQUES FOR ECONOMISTS solutions BEEM03 UNIVERSITY OF EXETER BUSINESS School January 009 Mock Exam, Part A OPTIMIZATION TECHNIQUES FOR ECONOMISTS solutions Duration : TWO HOURS The paper has 3 parts. Your marks on the rst part will be

### Differentiation. 1. What is a Derivative? CHAPTER 5 CHAPTER 5 Differentiation Differentiation is a technique that enables us to find out how a function changes when its argument changes It is an essential tool in economics If you have done A-level maths,

### Problem Set 3 4.02 Problem Set 3 Due Thursday, October 2, 2004, in class Starred (*) problems will not count for the grade on this problem set; they are based on material from lectures on 0/2 and 0/26, and provide practice

### MATH 255 Applied Honors Calculus III Winter Homework 5 Solutions MATH 255 Applied Honors Calculus III Winter 2011 Homework 5 Solutions Note: In what follows, numbers in parentheses indicate the problem numbers for users of the sixth edition. A * indicates that this

### University of California, Davis Department of Agricultural and Resource Economics ARE 252 Lecture Notes 2 Quirino Paris University of California, Davis Department of Agricultural and Resource Economics ARE 5 Lecture Notes Quirino Paris Karush-Kuhn-Tucker conditions................................................. page Specification

### Method of Lagrange Multipliers Method of Lagrange Multipliers A. Salih Department of Aerospace Engineering Indian Institute of Space Science and Technology, Thiruvananthapuram September 2013 Lagrange multiplier method is a technique

### TMA 4180 Optimeringsteori KARUSH-KUHN-TUCKER THEOREM TMA 4180 Optimeringsteori KARUSH-KUHN-TUCKER THEOREM H. E. Krogstad, IMF, Spring 2012 Karush-Kuhn-Tucker (KKT) Theorem is the most central theorem in constrained optimization, and since the proof is scattered

### Economics 121b: Intermediate Microeconomics Midterm Suggested Solutions 2/8/ (a) The equation of the indifference curve is given by, Dirk Bergemann Department of Economics Yale University Economics 121b: Intermediate Microeconomics Midterm Suggested Solutions 2/8/12 1. (a) The equation of the indifference curve is given by, (x 1 + 2)

### MATH The Chain Rule Fall 2016 A vector function of a vector variable is a function F: R n R m. In practice, if x 1, x n is the input, MATH 20550 The Chain Rule Fall 2016 A vector function of a vector variable is a function F: R n R m. In practice, if x 1, x n is the input, F(x 1,, x n ) F 1 (x 1,, x n ),, F m (x 1,, x n ) where each

### Constrained Optimization and Lagrangian Duality CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may DEPARTMENT OF MATHEMATICS AND STATISTICS UNIVERSITY OF MASSACHUSETTS MATH 233 SOME SOLUTIONS TO EXAM 2 Fall 208 Version A refers to the regular exam and Version B to the make-up. Version A. A particle Andrew McLennan February 9, 1999 Economics 5113 Introduction to Mathematical Economics Winter 1999 Lecture 10 Convexity and Concavity I. Introduction A. We now consider convexity, concavity, and the general