5. An economic understanding of optimal control as explained by Dorfman (1969) AGEC

Transcription

1 This doumen was generaed a 1:27 PM, 09/17/15 Copyrigh 2015 Rihard T Woodward 5 An eonomi undersanding of opimal onrol as explained by Dorfman (1969) AGEC The purpose of his leure and he nex is o help us undersand he inuiion behind he opimal onrol framework We draw firs on Dorfman's seminal arile in whih he explained OC o eonomiss (For his leure, I will use Dorfman's noaion so k is he sae variable and x is he hoie variable) A The problem Dorfman s problem is o maximize T (1) W, x u, d where x is he sream of all hoies made beween and T The sae equaion is k f, B Sep 1 Divide ime ino wo piees In order o help us undersand his problem, Dorfman divides he ime from o T ino wo piees, from o + and from + o T If is small, hen here is lile loss of auray if we linearize uiliy over he inerval from o +, ie, assume ha u(k,) is onsan over his inerval Tehnially, all he = signs below should be replaed by signs, bu we will assume he approximaion error is rivial Hene, we rewrie,,, T,, W k x u k x u k x d Le's look jus a his seond erm If we assume ha we maximize over he seond inerval from + o T, hen we an eliminae he onrol variable, x, from he seond erm o obain T * * * (2), max,,,, V k W k x u k x d, x where k * and x * are he opimal pahs of he sae and onrol variables Using his value funion, (2), we an wrie he value of he sream of welfare saring a ime wih asses k and hoosing a onsan value x for he period from o + as follows: * V k, x, u k, x, V k, (3) Noe ha he V( ) here is differen from ha in (2) I does no have a *, sine i is no neessarily a he opimum, and i inludes x as an argumen This is differen from V * ( ) in whih x is evaluaed a he opimum, making V * ( ) a funion of only k and

2 5-2 C Sep 2 Evaluae he FOC wr he onrol variable, x Problem (3) an be solved by applying sandard ools of alulus Dorfman akes he FOC, direly wih respe o he hoie variable x * (4) u, x, V, 0 x x We an hen rewrie he seond erm * * V V (5) x x Sine we assume ha he inerval is quie shor, we an approximae he sae equaion k k k k f, x, so ha f (6) 0 x x Dorfman hen subsiues (6) ino (5), and also wries V'=, so ha (4) an be rewrien u f (7) 0 x x Noe ha (7) and he relaionship beween V and λ an also be derived if we sar wih a Lagrangian, *,,,, L u k x V k k k f k x The FOCs would be, u f 0, and x x V * This onfirms wha we know, ha is he value of marginally relaxing he onsrain, ie, he hange in V *, ha would be ahieved by a marginal hange in k Hene, V' and are equivalen Canelling in (7) and hen aking he limi as 0 so ha u f (8) x x, we obain This is he firs of he opimaliy ondiions of he maximum priniple, (ie, H 0 ) z Dorfman (822-23) provides a lear and suin eonomi inerpreaion of his erm: [Equaion (8)] says ha he hoie variable a every insan should be seleed so ha he marginal immediae gains are in balane wih he value of he marginal onribuion o he aumulaion of apial Pu anoher way, he hoie variable should be inreased as long as he marginal immediae benefi is greaer han he marginal fuure oss In problems where he hoie

3 5-3 variable is disree or onsrained, i may no be possible o aually ahieve he equimarginal ondiion, bu he inuiion remains he same So now we've go a nie inuiive explanaion for he firs of he maximum ondiions: The enral priniple of dynami opimizaion is ha opimal hoies are made when a balane is sruk beween he immediae and fuure marginal onsequenes of our hoies D Sep 3 Look a he value of by aking V*/ We now assume ha he opimal hoie of x has been made over our shor inerval, o * * * V k, u k, x, V k, Differeniaing his expression wr k and subsiuing for V ', we ge u * V, * u V, u Sine his is over a shor period, we an approximae f and k k k, so ha 1 Hene, u f 1 u f 2 f 0 u f 2 f or, u f f k Taking he limi a 0, he las erm falls ou and we're lef wih u f (9) whih is he seond maximum ondiion, H k Dorfman (p 821) offers 3 ways o hink abou he eonomi inuiion behind his equaion

4 5-4 To an eonomis, i is he rae a whih he apial is appreiaing is herefore he rae a whih a uni of apial depreiaes a ime In oher words, [1] a uni of apial loses value or depreiaes as ime passes a he rae a whih is poenial onribuion o profis beomes is pas onribuion [or] [2] Eah uni of he apial good is gradually dereasing in value a preisely he same rae a whih i is giving rise o valuable oupus [3] We an also inerpre as he loss ha would be inurred if he aquisiion of a uni of apial were posponed for a shor ime [whih a he opimum mus be equal o he insananeous marginal value of ha uni of apial] So we see ha sine he value of he apial sok a he beginning of he problem is equal o he sum of he onribuions of he apial sok aross ime As we move aross ime, herefore, he apial sok s abiliy o onribue o V is used up To be hones, none of hese really gives me very srong inuiion However, don despair We will reurn o his ondiion when we use he urren-value Hamilonian below, and ha is muh easier o undersand E Sep 4 Summing up Hene, eah of he opimaliy ondiions assoiaed wih he Hamilonian has a lear eonomi inerpreaion H u k, f k, Le FOC Equaion Inerpreaion H Finds he opimal balane beween urren welfare and fuure Choie 0 x onsequenes The marginal value of he sae variable is dereasing a he same rae a whih i is generaing benefis H or Sae Along he opimal pah, he loss ha would be suffered if we delayed aquisiion of a marginal uni of apial for an insan mus equal he insananeous marginal value of ha uni of apial Cosae H k The sae equaion mus hold II A word abou disouning Disouning: Reall ha if r is he annual rae of disoun, hen 1 r T is he disoun faor applied o benefis or oss T years in he fuure If we break eah year ino n periods, hen he periodi disoun rae beomes rn so over n

5 5-5 periods (ie, a year) he one-year disoun faor beomes 1 n, his onverges o r e, he oninuous-ime disoun faor rn n As Consider a modifiaion of Dorfman's problem wih he assumpion ha we will maximize he presen value of u(k,)=e -r w(k,x) over he inerval 0 o T, ie, T r W e w, xd 0 u k, e r w k, x, so he This is a resriive speifiaion of (1) in whih opimaliy ondiions mus sill hold The Hamilonian now is r H e w k, x f k, (10) The inerpreaion of is he same: i is a measure of he onribuion o W of an addiional uni of k in period However, beause of disouning we know ha here is a leas some pressure for o fall over ime If W is he presen value (bak o year zero) of all he benefis from o T, hen beause of disouning W will end o be muh smaller far in he fuure han i is for lose o zero Correspondingly, W will also end o fall over ime Hene, he value of is influened by wo effes: he urren (in period ) marginal value of k, whih ould eiher be inreasing or dereasing, and he disouning effe, whih is always pushing λ oward zero Hene, even if he marginal value of apial is inreasing over ime (in urren dollars), migh be falling Beause of hese wo faors, i ofen happens ha he eonomi meaning of is no easily seen is i falling beause k is beoming less valuable or simply beause of disouning Hene, i is ofen helpful o use an alernaive speifiaion for disouned opimal onrol problems alled he urren value Hamilonian A The Curren Value Hamilonian We begin by defining an alernaive shadow prie variable,, whih is equal o he value of an addiional uni of k o he benefi sream, valued in period unis, ie, =e r Tha is, o ge we have o inflae o onver i from period 0 values o period (urren) values The urren value Hamilonian is obained by inflaing (10) o obain r (11) H w, x f, H e There are wo differenes beween (11) and he sandard Hamilonian (10) Firs, we use r μ insead of λ Seond, he disoun faor e, whih appears before w( ) in (10), anels ou sine we ve muliplied he enire funion by r e As a simple maer of algebra, we an derive he maximum ondiions orresponding o H and insead of H and The firs FOC an be rewrien,

6 5-6 H r H e x x H H so, 0 if and only if 0 x x Hene he analogous priniple holds wr he onrol variable, ie, H 1') 0 x or, more generally, maximize H wih respe o x Now look a he FOC wr he sae variable: The sandard formulaion is H Looking a he LHS of his equaion, we see ha for he urren value Hamilonian, H, H r H e and, on he RHS, sine =e -r re e re e r r r r Puing he LHS and RHS ogeher, we ge H r H r r e re e Canelling e -r gives us he seond opimum ondiion: H 2') r The hird ondiion, ha he sae equaion mus hold, remains unhanged The ransversaliy ondiion migh hange by a disoun faor, bu in many ases analogous ondiions hold For example, if he TC is T =0, and T = T e -rt hen i mus also hold ha T =0 (Noe ha if T, hen for r>0, a suffiien (hough no neessary) ondiion for he ransversaliy ondiion o be saisfied is ha does no go o infiniy as ) Hene, we an use he urren value Hamilonian, bu i is imporan o use he orre opimaliy ondiions

7 5-7 In summary, we seek o maximize T r W e w, xd subje o he sae equaion k f 0, We an do his using he vehile of he urren value Hamilonian, H w, x f, where he maximum rieria are: H 1') 0 x H 2') r H 3') k For reasons ha we will disuss below, eonomiss end o use he urren-value Hamilonians for disouned opimizaion problems, someimes wihou even menioning i I reommend ha if you have disouned problem, use H B An eonomi inerpreaion of he urren-value Hamilonian As in he sandard ase, he ondiion ha H be maximized over ime requires ha we srike a balane a every poin in ime The only differene is ha now we re onsidering his radeoff in erms of he values a fuure poins in ime, raher han all rade-offs in presen value erms A good way o hink abou his is by wriing i as H w, x f, w, x f, 0 or The LHS of his x x x x x expression is he marginal immediae value of a uni of x On he RHS, - is he marginal f, os of reduing he apial sok and ells us how big of an effe on k ha a x marginal hange in x has So you an hink of he RHS as he value marginal fuure os of an inrease in he onrol variable, x A he opimum, hese mus be in balane, oherwise he resoure manager would be advised o inrease or derease x The seond ondiion is a bi rikier, hough sill easier here han in he presen-value speifiaion 1 Reall ha 2' requires H w f r whih we will rewrie w f (12) r The hree erms of he LHS of his equaion an be hough of as a deomposiion of he benefis of holding a marginal uni of he apial sok for an insan longer: 1 Manseung Han, who ook my lass in 2002, grealy helped me in figuring ou a lear presenaion of his par of he problem

8 5-8 u indiaes he marginal immediae benefi of he apial sok f f is he apial sok s marginal value produ Tha is, ells us how he marginal uni of k onribues o he reaion of more k, and his is muliplied by μ he value of ha marginal uni of k Finally, indiaes how he marginal value of he apial is hanging over ime If you hold ha marginal uni for an insan longer, is value will have hanged by The RHS of (12), r, an be hough of as he opporuniy os of holding apial As an example, suppose ha our apial good an be easily ransformed ino dollars and we disoun a he rae r beause i is he marke ineres rae Then r is he immediae opporuniy os of holding apial, sine we ould sell i and earn ineres a he rae r Hene, a he opimum, we will hold our sae variable up o he poin where is marginal value is equal o he marginal opporuniy os Tha sounds familiar, and very eonomially reasonable Why his makes eonomi sense is mos easily seen when refleing on he firs FOC Reall ha in ha ase we had marginal urren benefi equal o marginal fuure oss, whih makes perfe sense Bu wha are hose marginal fuure oss along he opimal pah? The seond FOC helps answer ha quesion C Summary The urren value formulaion is very araive for eonomi analysis beause urren values are usually more ineresing han disouned values For example, in a simple eonomy, he marke prie of a apial sok will equal he urren-value o-sae variable As eonomiss we are usually more ineresed in suh aual pries han we are in heir disouned presen value Hene, very ofen he urren-value Hamilonian is more helpful han he presen-value variey Also, as a praial maer, for analysis i is ofen he ase ha he differenial equaion for will be auonomous (independen of ) while ha for will no be Hene, he dynamis of a sysem involving an be inerpreed using phase-diagram and seady-sae analysis, while his does no hold for One noe of auion: we have saed and derived many of he basi resuls for he presenvalue formulaion (eg, ransversaliy ondiions) When you are using he urren-value formulaion, you need o be areful o ensure ha everyhing is modified onsisenly III Referenes Dorfman, Rober 1969 An Eonomi Inerpreaion of Opimal Conrol Theory Amerian Eonomi Review 59(5): IV Readings for nex lass Harwik, John M "Naural Resoures, Naional Aouning and Eonomi Depreiaion" Journal of Publi Eonomis 43(Deember 1990):