This document was generated at 4:25 PM, 03/29/18 Copyright 2018 Richard T. Woodward

Similar documents
This document was generated at 7:34 PM, 07/27/09 Copyright 2009 Richard T. Woodward

This document was generated at 1:04 PM, 09/10/13 Copyright 2013 Richard T. Woodward. 4. End points and transversality conditions AGEC

Seminar 4: Hotelling 2

1. An introduction to dynamic optimization -- Optimal Control and Dynamic Programming AGEC

ANSWERS TO EVEN NUMBERED EXERCISES IN CHAPTER 6 SECTION 6.1: LIFE CYCLE CONSUMPTION AND WEALTH T 1. . Let ct. ) is a strictly concave function of c

Solutions Problem Set 3 Macro II (14.452)

Essential Microeconomics : OPTIMAL CONTROL 1. Consider the following class of optimization problems

1. An introduction to dynamic optimization -- Optimal Control and Dynamic Programming AGEC

3.1.3 INTRODUCTION TO DYNAMIC OPTIMIZATION: DISCRETE TIME PROBLEMS. A. The Hamiltonian and First-Order Conditions in a Finite Time Horizon

E β t log (C t ) + M t M t 1. = Y t + B t 1 P t. B t 0 (3) v t = P tc t M t Question 1. Find the FOC s for an optimum in the agent s problem.

Macroeconomic Theory Ph.D. Qualifying Examination Fall 2005 ANSWER EACH PART IN A SEPARATE BLUE BOOK. PART ONE: ANSWER IN BOOK 1 WEIGHT 1/3

Appendix 14.1 The optimal control problem and its solution using

Lecture Notes 5: Investment

Reserves measures have an economic component eg. what could be extracted at current prices?

SZG Macro 2011 Lecture 3: Dynamic Programming. SZG macro 2011 lecture 3 1

Economics 8105 Macroeconomic Theory Recitation 6

BU Macro BU Macro Fall 2008, Lecture 4

This document was generated at 5:08 PM, 09/24/13 Copyright 2013 Richard T. Woodward

Lecture 2-1 Kinematics in One Dimension Displacement, Velocity and Acceleration Everything in the world is moving. Nothing stays still.

2.7. Some common engineering functions. Introduction. Prerequisites. Learning Outcomes

RANDOM LAGRANGE MULTIPLIERS AND TRANSVERSALITY

Inventory Analysis and Management. Multi-Period Stochastic Models: Optimality of (s, S) Policy for K-Convex Objective Functions

IMPLICIT AND INVERSE FUNCTION THEOREMS PAUL SCHRIMPF 1 OCTOBER 25, 2013

Problem Set 5. Graduate Macro II, Spring 2017 The University of Notre Dame Professor Sims

Cooperative Ph.D. Program in School of Economic Sciences and Finance QUALIFYING EXAMINATION IN MACROECONOMICS. August 8, :45 a.m. to 1:00 p.m.

The Brock-Mirman Stochastic Growth Model

A Dynamic Model of Economic Fluctuations

Graduate Macro Theory II: Notes on Neoclassical Growth Model

KINEMATICS IN ONE DIMENSION

Problem Set #3: AK models

( ) a system of differential equations with continuous parametrization ( T = R + These look like, respectively:

= ( ) ) or a system of differential equations with continuous parametrization (T = R

Lecture Notes 3: Quantitative Analysis in DSGE Models: New Keynesian Model

THE BELLMAN PRINCIPLE OF OPTIMALITY

1 Answers to Final Exam, ECN 200E, Spring

15. Vector Valued Functions

Midterm Exam. Macroeconomic Theory (ECON 8105) Larry Jones. Fall September 27th, Question 1: (55 points)

1 Consumption and Risky Assets

Math 2142 Exam 1 Review Problems. x 2 + f (0) 3! for the 3rd Taylor polynomial at x = 0. To calculate the various quantities:

!!"#"$%&#'()!"#&'(*%)+,&',-)./0)1-*23)

Full file at

FINM 6900 Finance Theory

Seminar 5 Sustainability

Physics 235 Chapter 2. Chapter 2 Newtonian Mechanics Single Particle

Final Spring 2007

Unit Root Time Series. Univariate random walk

Explaining Total Factor Productivity. Ulrich Kohli University of Geneva December 2015

Online Appendix to Solution Methods for Models with Rare Disasters

Two Coupled Oscillators / Normal Modes

Lecture 3: Solow Model II Handout

Problem 1 / 25 Problem 2 / 20 Problem 3 / 10 Problem 4 / 15 Problem 5 / 30 TOTAL / 100

Expert Advice for Amateurs

Predator - Prey Model Trajectories and the nonlinear conservation law

Macroeconomics I, UPF Professor Antonio Ciccone SOLUTIONS PROBLEM SET 1

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS

A First Course on Kinetics and Reaction Engineering. Class 19 on Unit 18

23.2. Representing Periodic Functions by Fourier Series. Introduction. Prerequisites. Learning Outcomes

O Q L N. Discrete-Time Stochastic Dynamic Programming. I. Notation and basic assumptions. ε t : a px1 random vector of disturbances at time t.

Chapter 7: Solving Trig Equations

1. Consider a pure-exchange economy with stochastic endowments. The state of the economy

T. J. HOLMES AND T. J. KEHOE INTERNATIONAL TRADE AND PAYMENTS THEORY FALL 2011 EXAMINATION

( ) (, ) F K L = F, Y K N N N N. 8. Economic growth 8.1. Production function: Capital as production factor

Problem Set on Differential Equations

Final Exam Advanced Macroeconomics I

Suggested Solutions to Assignment 4 (REQUIRED) Submisson Deadline and Location: March 27 in Class

t is a basis for the solution space to this system, then the matrix having these solutions as columns, t x 1 t, x 2 t,... x n t x 2 t...

2. The basics of differential equations 1 AGEC 642 Fall 2018

Biol. 356 Lab 8. Mortality, Recruitment, and Migration Rates

Solutions to Odd Number Exercises in Chapter 6

Reading from Young & Freedman: For this topic, read sections 25.4 & 25.5, the introduction to chapter 26 and sections 26.1 to 26.2 & 26.4.

Cash Flow Valuation Mode Lin Discrete Time

Policy regimes Theory

Let us start with a two dimensional case. We consider a vector ( x,

Math 333 Problem Set #2 Solution 14 February 2003

Math 116 Second Midterm March 21, 2016

3.6 Derivatives as Rates of Change

3.1 More on model selection

Introduction D P. r = constant discount rate, g = Gordon Model (1962): constant dividend growth rate.

Matlab and Python programming: how to get started

Linear Response Theory: The connection between QFT and experiments

Chapter 15: Phenomena. Chapter 15 Chemical Kinetics. Reaction Rates. Reaction Rates R P. Reaction Rates. Rate Laws

Problem set 3: Endogenous Innovation - Solutions

Fishing limits and the Logistic Equation. 1

18 Biological models with discrete time

Solutions to Assignment 1

Guest Lectures for Dr. MacFarlane s EE3350 Part Deux

13.3 Term structure models

) were both constant and we brought them from under the integral.

Licenciatura de ADE y Licenciatura conjunta Derecho y ADE. Hoja de ejercicios 2 PARTE A

Lecture 20: Riccati Equations and Least Squares Feedback Control

The Brock-Mirman Stochastic Growth Model

On Measuring Pro-Poor Growth. 1. On Various Ways of Measuring Pro-Poor Growth: A Short Review of the Literature

Econ107 Applied Econometrics Topic 7: Multicollinearity (Studenmund, Chapter 8)

T L. t=1. Proof of Lemma 1. Using the marginal cost accounting in Equation(4) and standard arguments. t )+Π RB. t )+K 1(Q RB

The general Solow model

A Note on Raising the Mandatory Retirement Age and. Its Effect on Long-run Income and Pay As You Go (PAYG) Pensions

Announcements: Warm-up Exercise:

Some Basic Information about M-S-D Systems

Designing Information Devices and Systems I Spring 2019 Lecture Notes Note 17

SOLUTIONS TO ECE 3084

Transcription:

his documen was generaed a 4:25 PM, 3/29/8 Copyrigh 28 Richard. Woodward 4. Opimal conrol wih consrains and MRAP problems AGEC 642-28 We now reurn o an opimal conrol approach o dynamic opimizaion. his means ha our problem will be characerized by coninuous ime and will be deerminisic. I is usually he case ha we are no Free o Choose. he choice se faced by decision makers is almos always consrained in some way and he naure of he consrain frequenly changes over ime. For example, a binding budge consrain or producion funcion migh deermine he opions ha are available o he decision maker a any poin in ime. When his is rue we will need o reformulae he simple Hamilonian problem o ake accoun of he consrains. Forunaely, in many cases economic inuiion will ell us ha he consrain will no bind (excep, for example, a =), in which case our life is much simplified. We consider here cases where we're no so lucky, where he consrains canno be ruled ou ex ane. We will assume hroughou ha a feasible soluion exiss o he problem. Obviously, his is somehing ha needs o be confirmed before proceeding o wase a lo of ime rying o solve an infeasible problem. In his lecure we cover consrained opimal conrol problems raher quickly looking a he imporan concepual issues. For echnical deails, I refer you o Kamien & Schwarz, which has chapers on consrained opimal conrol problems. We hen go on o consider a class of problems where he consrains play a paricularly cenral role in he soluion. I. Opimal conrol wih equaliy consrains A. heory Consider a simple dynamic opimizaion problem = g ( z, x, ) (,, ) = c ( ) = x r max e u z, x, d s.. xɺ z h z x x In his case we canno use he Hamilonian alone, because his would no ake accoun of he consrain, h(z,x,)=c. Raher, we need o maximize he Hamilonian subjec o a consrain so we use a Lagrangian 2 in which H c is he objecive funcion, i.e., c φ ( (,, ) ) µ φ L = H + h z x c = u z, x, + g z, x, + c h z, x,. Equivalenly, you can hink abou embedding a Lagrangian, wihin a Hamilonian, i.e. his is an obuse reference o he firs popular book on economics I ever read, Free o Choose by Milon and Rose Friedman. 2 his Lagrangian is given a variey of names in he lieraure. Some call i an augmened Hamilonian, some a Lagrangian, some jus a Hamilonian. As long as you know wha you re alking abou, you can prey much call i whaever you like.

4-2 H = u z, x, + φ c h z, x, + µ g z, x,. We ll use he firs noaion here. c Assuming ha everyhing is coninuously differeniable and ha concaviy assumpions hold, he FOC's of his problem, hen, are: L. = z L 2. = rµ ɺ µ x and, of course, he consrains mus be saisfied: L = xɺ µ L = c h( z, x, ) =. φ Le's look a hese in more deail. he FOC w.r.. z is '. L u g = + µ φ h =, z z z z which can be rewrien u h g ''. φ = µ z z z. As Dorfman showed us, he FOC w.r.. he conrol variable ells us ha a he opimum we balance off he marginal curren benefi and marginal fuure coss. In his case he RHS is he cos o fuure benefis of a marginal increase in z. he LHS, herefore, mus indicae he benefi o curren uiliy from marginal incremens o z. If u/ z>rhs, hen his implies ha here is a cos o he consrain and φ h is he cos o curren uiliy z of he inraemporal consrain, h. If h( ) were marginally relaxed, hen z could be changed o push i closer o balancing wih he conribuion of a marginal uni of z in he fuure. --µdg/dz λ φdh/dz du/dz wih consrain wihou consrain z

4-3 In principle, he problem can hen be solved based on hese equaions. I is imporan o noe ha φ will be a funcion of ime and will ypically change over ime. Wha is he economic significance of φ? B. Opimal conrol wih muliple equaliy consrains he exension o he case of muliple equaliy consrains, is easy; wih n consrains he Lagrangian will ake he form L = u ( z, x, ) + g( z, x, ) + φ ( c h ( z, x, ) ) n λ. i= i i i Obviously, here may no be a feasible soluion unless some of he consrains do no bind or are redundan, especially if n is greaer han he cardinaliy of z. C. Example: he poliical business cycle model (Chiang s (Elemens of Dynamic Opimizaion) presenaion of Nordhaus 975) his model looks a macroeconomic policy. wo policy variables are available, U, he rae of unemploymen, and p, he rae of inflaion. I is assumed ha here is a rade-off beween hese wo so ha suppor for he curren adminisraion can be defined by he equaion v = v U, p so ha he relaionship beween he wo policies can be described by he iso-voe curves in he figure below. p More voes Following sandard Phillips-curve logic, here is an assumed rade-off beween hese wo objecives so ha he inflaion rae goes down if he unemploymen rae goes up, p = γ U + απ, where π is he expeced rae of inflaion. Expecaions evolve according o he differenial equaion ɺ π = b p π. U

4-4 We assume ha he voes obained a ime are a weighed sum of he suppor ha is obained from o, wih suppor nearer o he voing dae being more imporan. Voes r v U p e d. obained a are equal o (, ) he opimizaion problem hen is U, p r max v U, p e d s.. ɺ π p = b( p π ) = γ ( U ) + απ = ( ) π π, and π free. Now clearly he equaliy consrain could be used o subsiue ou for p and conver he problem o a single-variable conrol problem, bu le s consider he alernaive, explicily including he consrain. he Lagrangian for his opimal conrol problem would be r L = v( U, p) e + λ ( b( p π )) + φ γ ( U ) + απ p he opimum condiions would hen be L v e r = + λ b φ = p p L v e r = + φγ ' = U U L = γ ( U ) + απ p = φ ɺ λ = λb φα ( π ) ɺ π = b p If we specify a funcional form (see Chiang chaper 7) we can find ha he opimal pah for policy, which shows ha he poliical process creaes a business cycle. o ge he soluion, i is ofen easier o find he soluion by using equaliy consrains o eliminae variables before geing sared. However, i is also rue ha here is economic meaning in he shadow prices, so your analysis can be enriched by solving he problem wih he consrains saed explicily. II. Opimal conrol wih inequaliy consrains A. heory Suppose now ha he problem we face is one in which we have inequaliy consrains, h i (, x, z) ci, wih i=,, n, for n consrains and x and z are assumed o be vecors of he sae and conrol variables respecively. For each xj x, he sae equaion akes he form xɺ = g, x z. j j, As wih sandard consrained opimizaion problems, he Kuhn-ucker condiions will yield a global maximum if any one of he Arrow-Hurwicz-Uzawa consrain

4-5 qualificaions is me (see Chiang p. 278). he way his is ypically saisfied in mos economic problems is for he h i o be concave or linear in he conrol variables. Assuming ha he consrain qualificaion is me, we can hen proceed o use he Lagrangian specificaion using a Hamilonian which akes he form (,,, λ) (,, ) λ (,, ) H x z u x z g x z m = + j= j which we hen plug ino he Lagrangian wih he consrains, n (,,, λ) ϕi ( i i (,, )) L = H x z + c h x z m i= (,, ) λj j (,, ) ϕi ( i i (,, )) L = u x z + g x z + c h x z j= i= j n Noe: For maximizaion problems I always wrie he consrain erm of he Lagrangian so ha he argumen inside he parenheses is consrained o be greaer han zero, or for minimizaion problems you wrie i so ha he argumen is less han zero. If you follow his rule, your Lagrange muliplier will always be posiive. he FOC's for his problem are: n m L u g j hi = + λj φi = zk zk j= zk i= zk L = λɺ j for all j x j L λ j = xɺ j for all z k z and, for he consrains L hi ( x, z ) ci φi wih he complemenary slackness condiions: L φ i and φi = for all i. φ i As wih all such problems, he appropriae ransversaliy condiions mus be used and, if you choose o use a curren-value Hamilonian, he necessary adjusmens mus be made. Noe ha in he curren value specificaion, he inerpreaion of boh he co-sae variable and he shadow price on he inraemporal consrain would be alered. Chiang solves he problem for a specific funcional form and finds ha if poliicians solve an opimal policy pah like his, unemploymen will end o fall as an elecion approaches and hen rise again immediaely afer he elecion..

4-6 B. Example: Hoelling s opimal exracion problem We reurn o Hoelling s problem from Lecure 6. he planner s problem is o maximize z r max e p( z) dz s.. z d xɺ = z x( ) = x, x. Economic inuiion ells us ha x =. However, we found in lecure 6 ha i is possible o find a soluion in which x becomes negaive and hen, z is negaive for a period o resore x so ha x =. However, by consraining x = and z for all, we can indirecly ensure ha x for all. he associaed Lagrangian would hen be L=e -r u( ) +λ(-z)+φ z. We cover consrains on he sae variable below he associaed maximizaion crieria are: 3. Lz=: e -r u'( ) -λ+φ = e -r p(z) λ+φ= 4. Lx= ɺ λ : ɺ λ = 5. Lλ= xɺ : xɺ = z 6. Lφ : z 7. φ 8. φ z= Kuhn ucker Condiions he ransversaliy condiion is x =. From 4 i sill holds ha λ is consan as we found in Lecure 6. However, 3 can be rewrien p(z) =(λ φ)e r. Using he assumed funcional form for inverse demand curve, p(z)=e -γz, we obain γ z = ln λ φ + r, or e -γz =(λ φ)e r. aking logs we ge ( ) ln ( λ φ ) + r 9. z =. γ Now, using he complemenary slackness condiions, we know ha if z> hen φ= and if z=, φ>. he pah can, herefore, be broken ino wo pars, he firs par from o during which z> and he second par, from o, where z= and φ>. From o, ln ( λ ) + r ln ( λ) + r z = = > γ γ so, from he complemenary slackness condiion, 8, φ=. ln ( λ φ ) + r And from o, = ln ( λ φ ) = r, so ha γ. φ r = λ e.

4-7 r Hence, φ is increasing since he second erm, e, ges smaller as increases. Now, we can speculae abou he soluion. A criical quesion is wheher z and φ will be disconinuous over he planning horizon; i.e. will z approach a gradually, reaching only a, or will i jump from a posiive level o a he insan ha i reaches? Imagine wo possible pahs. In he firs pah he level of consumpion is a a posiive level, 2 ε for he period prior o and zero from onward. In he second pah consumpion is a ε for prior and afer. he oal amoun of consumpion is he same, 2 ε, bu, by Jensen s inequaliy we know ha he uiliy over he shor incremen of ime, 2, will be greaer in he second case. Hence, any ime here is a disconinuous jump in uiliy, we know ha here is a preferred pah wih half he jump. Only a pah in which consumpion decreases coninuously o zero does no violae his principal; a disconinuous jump will never be opimal. So we can sar by assuming ha z approaches coninuously as. Under his assumpion, φ = for so ha, from, r. λ = e. Furhermore, we know ha since z = from onward, we mus exhaus he resource by : 2. ln λ + = = or r z d x d x γ Which we solved in lecure 6 o obain γ r x 2 3. λ = e. Combining and 3, we obain γ r x r 2 r γ 2 2γ 2 r e = e = x = x, which can be simplified o 2γ 4. = x. r Hence, he resource will be exhaused by and he consrain on z is binding from γ r x 2 r r onwards. Finally, for >, recall ha φ = λ e, so φ = e e. his is he shadow price of he consrain on z, which ges larger for >. III. Consrains on he sae space A. heory Suppose now ha we have consrains on he sae variables which define a feasible range. his is common in economic problems. You may, for example, have limied sorage space so ha you canno accumulae your invenory forever. Or, if you were dealing wih a biological problem, you migh be consrained o keep your sock of a

4-8 species above a lower bound where reproducion begins o fail, and an upper bound where epidemics are common. he approach o such problems is similar o ha of he conrol problems. Suppose we have an objecive funcion u(, x, z) d = g(, x, z), x( ) (, x). max xɺ s.. = x and h he augmened Hamilonian for his problem is L = u, x, z + λg, x, z + φh, x and he necessary condiions for opimaliy include, he consrains plus L = z ɺ L λ = x φ and φh = and he ransversaliy condiion. Solving problems like his by hand can be quie difficul, even for very simple problems. (See K&S p.232 if you wan o convince yourself). (An alernaive approach presened in Chiang (p. 3) is ofen easier and we follow his approach below). For much applied analysis, however, here may be no alernaive o seing a compuer o he problem o find a numerical soluion. B. Example: Hoelling s opimal exracion problem Clearly, Hoelling s problem can also be modeled as a resricion ha x. In his case our Lagrangian would ake he form L=e -r u( ) +λ(-z)+φ x. And he associaed maximizaion crieria are: 5. Lz=: e -r u'( ) -λ = e -r p(z) λ= 6. Lx= ɺ λ : λ = φ 7. Lλ= xɺ : xɺ = z 8. Lφ : x 9. φ Kuhn ucker Condiions 2. φx= We won solve his problem in all is deail, bu he soluion mehod would follow a similar pah o ha used above. We divide ime ino wo porions, from o where φ = and λ is consan, and from o, where x = and λ falls wih and φ increases. o solve he problem, we use he same logic as above o deermine ha φ =. We can hen solve o obain 4.

4-9 One hing ha is ineresing in his specificaion is ha he co-sae variable is no longer consan over ime. his makes sense since beween and we are indifferen abou when we ge he exra uni of he resource. Bu afer, i clearly makes a difference he sooner we could obain a marginal uni he more valuable (in PV erms) i will be. When >, we know ha z= p= (he choke price) and λ=e r. A marginal increase in he sock over his range would allow he immediae sale of ha sock a a price of and he presen value of his marginal change in sock would, herefore, be e r. he economic meaning of φ is also of ineres. From 6, ɺ λ = φ. his means ha he shadow price on he inequaliy consrain is inversely relaed o he rae of change in he shadow price in he co-sae variable. For he period when λ is consan, here s no shadow price on he sae variable here s no shorage ye. As λ sars o decline, φ becomes posiive, and ɺ λd = λ λ = φ d So φ can be hough of as he insananeous cos of he inequaliy consrain, while λ is he accumulaion of hose insananeous coss. IV. Bang-bang OC problems We now consider problems for which he opimal pah does no involve a smooh approach o he seady sae or gradual changes over ime. wo imporan classes of such problems are known as "bang-bang" problems and mos rapid approach problems. In such problems he consrains play a cenral role in he soluion. A. Bang-bang example #: A sae variable consrain Consider he following problem in which we seek o maximize discouned linear uiliy obained from a nonrenewable sock (someimes referred o as a cake-eaing problem): r max e z d s.. z ( ) xɺ = z x x = x Wha does inuiion sugges abou he soluion o he problem? Will we wan o consume he resource sock x gradually? Why or why no? Le's check our inuiion. Following he framework from above, we se up he Lagrangian by adding he consrain on he sae variable o he Hamilonian, i.e., L=H+φ(consrain). Using he curren-value specificaion, his give us L = z µ z + φ x he FOCs for he problem are: L (i) = : µ = z

4- (ii) L = r µ ɺ µ : φ = r µ ɺ µ x Because of he consrain, he complemenary slackness condiion mus also hold: (iii) φ x =. Equaion i implies ha µ=. Since his holds no maer he value of, we know ha µɺ = for all. Condiions i and ii ogeher indicae ha µ= and φ=r. he second of hese is mos ineresing. I shows us ha φ, he Lagrange muliplier, is always posiive. From he complemenary slackness condiion, i follows ha x mus equal always. Bu wai! We know his isn' acually rue a =; bu, a =, x is no variable i is parameric o our problem. Since x canno be chosen, his condiion only applies for >; a every insan excep he immediae saring value, x=. So how big is z a zero? he firs hough is ha i mus equal x bu his isn' quie righ. o see his, suppose ha we found ha he consrain sared o bind, no immediaely, bu afer seconds. o ge he x o zero in seconds, z per second would have o equal x/. Now ake he limi of his a he denominaor goes o zero z goes o infiniy. Hence, wha happens is ha for one insan here is a spike of z of infinie heigh and zero lengh ha pushes x exacly o zero. his ype of soluion is known as a bang-bang problem because he sae variable jumps disconinuously a a single poin BANG- BANG! Since, in he real world i's prey difficul o push anyhing o infiniy, we would ypically inerpre his soluion as consume i as fas as you can. his is formalized in he framework of mos-rapid-approach pah problems below. B. Bang-Bang Example #2 (based on Kamien and Schwarz p. 25) A conrol variable consrain Le x be a producive asse ha generaes oupu a he rae rx. his oupu can eiher be consumed or reinvesed. he porion ha is reinvesed will be called z so [-z] is he porion ha is consumed. We assume ha he ineres can be consumed, bu he principal canno be ouched. 3 Our quesion is, Wha porion of he ineres should be invesed and wha porion should be consumed over he inerval [,]? Formally, he problem is: max xɺ z x z [ z ] = z rx ( ) = x rx d s.. his ime we have wo consrains: z and z. Hence, our Lagrangian is 3 his problem is very similar o one looked a in Lecure 3. Comparing he wo you ll see one key difference is ha here uiliy is linear, while in lecure 3 uiliy was logarihmic.

[ ] ( ) where [ z ] rx λz rx L = z rx + λz rx + φ z + φ z, 2 + is he Hamilonian par of he problem and he las wo erms are he consrains. he necessary condiions for an opimum are 2. L = rx + λrx ϕ + ϕ = 2, and z L 22. = ɺ λ ɺ λ = [ z ] r + λzr. x he ransversaliy condiion in his problem is λ= since x is unconsrained wih he Kuhn-ucker condiions, K: φ & φ( z)=, and K2: φ2 & φ2z=. 4- From he K, we know ha if φ >, hen he firs consrain binds and z=. Similarly, from K 2, if φ 2 >, hen he second consrain binds and z=. i.e. φ> z = φ2> z =. φ= z < φ2= z >. Clearly, i is no possible for boh φ and φ 2 o be posiive a he same ime. he firs FOC can be rewrien ( λ ) rx φ + φ. 2 = We know ha rx will always be posiive since consumpion of he capial sock is no allowed. Hence, we can see ha hree cases are possible: ) if λ= φ= φ2= no consrain binds 2) if λ> φ> φ2= z= 3) if λ< φ= φ2> z=. From he second FOC, ɺ λ = z r + λ z r. {[ ] } Since everyhing in he brackes is posiive, he RHS of he equaion is negaive λ is always falling. By he ransversaliy condiion we know ha evenually λ mus hi λ=. Hence, evenually we'll reach case 3 where, λ< and z= and we consume all of our oupu. Bu when do we sar consuming, righ away or afer x has grown for a while? We know from equaion 2 ha a λ= neiher consrain binds. Suppose ha a =n λ=. For <n λ> and z=. For >n λ< and z=.

4-2 An imporan quesion hen is when is n? We can figure his ou by working backwards from λ=. From he second FOC, we know ha in he final period, (when λ<) z=, in which case ɺ λ = r. Solving his differenial equaion yields λ = r + A. Using he ransversaliy condiion, λ = r + A = A = r λ = r + r = r Hence, λn= if r n = = ( ) n r r Hence, we find ha he opimal sraegy is o inves everyhing from = unil = n = r r. Afer =n consume all of he ineres. If ( r ) r < hen i would be opimal o consume everyhing from he very ouse. For ( r ) r >, we can graph he soluion: Wha would be he soluion as? Does his make inuiive sense? Wha is i abou he specificaion of he problem ha makes i inconsisen wih our economic inuiion? V. Mos Rapid Approach Pah problems Bang-bang problems fi ino a general class of problems ha are commonly found in economics: mos-rapid-approach pah problems (MRAP). 4 Here, he opimal policy is o ge as quickly as possible o seady sae where benefis are maximized. Consider he firs example bang-bang example above. Wouldn a soluion in which we move oward he equilibrium as fas as possible raher han impossibly fas be more inuiively appealing? 4 Someimes he erm bang-bang is also used o describe MRAP problems.

4-3 A. MRAP example (Kamien & Schwarz p. 2) A very simple firm generaes oupu from is capial sock wih he funcion f(x) wih he lim f ' x =. he profi rae, herefore, is propery ha x π = p f x c z where x is he firm's capial sock and z is invesmen, p and c are exogenously evolving uni price and uni cos respecively. he capial sock ha sars wih x()=x, depreciaes a he rae b so ha xɺ = z bx. he firm's problem, herefore, is o maximize he presen value of is profis, r e p f ( x ) c z d subjec o xɺ = z bx, wih hree addiional consrains: i) x() ii) z iii) p f ( x ) c z Le's use economic inuiion o help us decide if we need o explicily include all he consrains in solving he problem? he consrain on x almos cerainly does no need o be imposed because as long as f' ges big as x, he opimal soluion will always avoid zero. he consrains on z, on he oher hand migh be relevan. Bu, we'll sar by assuming ha neiher consrain binds, and hen see if we can figure ou acual he soluion based on he assumed inerior soluion or, if no, we'll need o use he Kuhn- ucker specificaion. Noe ha if here does exis a seady sae in x, hen, as long as b>, z mus be greaer han zero. Hence, we anicipae ha much migh be learned from he inerior soluion. Similarly, he profi consrain migh also bind, bu we would expec ha in he long run, profis would be posiive. So again, we sar by solving for an inerior soluion, π = p f x c z. assuming π> where B. he inerior soluion he curren value Hamilonian of he problem (assuming an inerior soluion w.r.. z and x wih π>) is H = p f x c z + µ z bx c he necessary condiions for an inerior soluion are: H c = c + µ = z ( x ) H f c = rµ ɺ µ p µ b = rµ ɺ µ x x Over any range where he consrains on z do no bind, herefore, we have c=µ and, herefore, i mus also hold ha

4-4 ɺ µ = cɺ =. Subsiuing c for µ and rearranging, he second FOC becomes f ( x ) 23. p = ( r + b) c x over any inerval where z>. We see, herefore, ha he opimum condiions ell us abou he opimal level of x, say x *. We can hen use he sae equaion o find he value of z ha mainains his relaion. Since c and p are consan, his means ha he capial sock will be held a a consan pf '( x) level and 23 reduces o r + b = c. his is known as he modified golden rule. Le's hink abou his condiion for a momen. In a saic economy, he opimal choice would be o choose x where he marginal produc of increasing x is equal o he marginal cos, i.e., where pf ' = c. In an infinie-horizon economy, if we could increase x a all poins in ime his would pf ' have a discouned presen value of. However, since he capial sock depreciaes r over ime, his depreciaion rae diminishes he presen value of he gains ha can be obained from an increase in x oday, hence he presen value of he benefi of a pf ' marginal increase in x is. r + b If p and c are no consan, bu grow in a deerminisic way (e.g., consan and equal inflaion) hen we could de-rend he values and find a real seady sae. If p and c boh grow a a consan rae, say w, hen here will be a unique and seady opimal value of x for all z>. C. Corner soluions All of he discussion above assumed ha we are a an inerior soluion, where < z < p f x c. Bu, we ended up finding ha he inerior soluion only holds when he sae variable x is a he poin defined by equaion 23. Hence, if we're no a x * a =, hen i mus be ha we're a a corner soluion, eiher z= or p f ( x ) c z =. If x>x * hen i will follow ha z will equal zero unil x depreciaes o x *. If x< x * hen z p will be as large as possible f ( x ) = z unil x * is reached. c Hence, economic inuiion and a good undersanding of he seady sae can ell us where we wan o ge and how we're going o ge here in he mos rapid approach possible.

4-5 D. Some heory and generaliies regarding MRAP problems A general saemen of he condiions required for a MRAP resul is presened by Wilen (985, p. 64): Spence and Sarre show ha for any problem whose augmened inegrand (derived by subsiuing he dynamic consrain ino he original inegrand) can be wrien as A ( K Kɺ ) = M ( K ) + N( K )Kɺ Π, he opimal soluion reduces o one of simply reaching a seady sae level K=K * as quickly as possible. Where K is he sae variable and by "inegrand" hey mean he objecive funcion, profis in he case considered here. How does his rule apply here? he inegrand is ( ) bx +ɺ x = z, he inegrand can be wrien ( x ) c ( bx + xɺ ) = p f ( x ) cbx c xɺ p f. Convering his o he noaion used by Wilen, M K = p f x c bx and N K K ɺ = c x ɺ. Hence his problem fis ino he general class of MRAP problems. p f x c z. Using he sae equaion For a more inuiive undersanding of why bang-bang and MRAP soluions arise, consider he a general problem of he form s.. = r max e f x zd xɺ g x z z so ha boh he benefi funcion and he sae equaion are linear in z. In his case, he Hamilonian would be wrien H = f x z + µ g x z. c he opimizaion crierion remains: Maximize Hc wih respec o z for all. If f x + µ g x >, hen o maximize H c we should se z a +. If ( ) ( ) µ f x + g x <, hen z should be se a. 5 Hence, boh he benefi funcion and he sae equaion are linear in z a bang-bang or MRAP soluion will be obained. One lesson ha can be obained from his is ha you need o be careful when specifying your model. While linear funcions are nice o work wih and frequenly offer nice inuiion, hey can also lead o corner soluions ha are no inuiive, may no be easy o 5 I found his simple presenaion in Rodriguez e al. (2), hough I imagine ha he presenaion has been presened by ohers previously.

4-6 work wih and may lack he inuiive economic meaning ha he model is se up o deliver. VI. References Chiang, Alpha C. 99. Elemens of Dynamic Opimizaion. McGraw Hill Hoelling, Harold. 93. he Economics of Exhausible Resources. he Journal of Poliical Economy 39(2):37-75. Kamien, Moron I. and Schwarz, Nancy Lou. 99. Dynamic Opimizaion : he Calculus of Variaions and Opimal Conrol in Economics and Managemen. New York, N.Y. : Elsevier. Spence, Michael and David Sarre. 975. Mos Rapid Approach Pahs in Accumulaion Problems. Inernaional Economic Review 6(2):388-43. Wilen, James E. 985. Bioeconomics of Renewable Resource Use, In A.V. Kneese and J.L. Sweeney (eds.) Handbook of Naural Resource and Energy Economics, vol. I. New York: Elsevier Science Publishers B.V.