3.1.3 INTRODUCTION TO DYNAMIC OPTIMIZATION: DISCRETE TIME PROBLEMS. A. The Hamiltonian and First-Order Conditions in a Finite Time Horizon

Transcription

1 3..3 INRODUCION O DYNAMIC OPIMIZAION: DISCREE IME PROBLEMS A. he Hamilonian and Firs-Order Condiions in a Finie ime Horizon Define a new funcion, he Hamilonian funcion, H. H he change in he oal value of a sysem a any ime period in an ineremporal opimizaion problem. Change in oal value of he sysem is characerized in wo pars. Par I: value ha arises as a direc resul of consumpion of y in ime. his is given by V (,y ). corn consumpion Par II: he induced ne change in he value of fuure consumpion, as a resul of having consumed y in ime. his is given, for any ime, by λ + f (, y ), where λ + indicaes he opporuniy cos of having consumed y, insead of using i o reproduce for laer consumpion. Suppose ha we have a finie horizon dynamic problem such as: = { MAX V y, } 0 s.. = f = a ( y, ) + F( ) ( y ) +, 0 (3.) Hamilonian funcion H o behave as a "performance indicaor" for each ime : if H is maimized a each, he Lagrangian will also be maimized. H (y,, λ + ) = V (, y ) + λ + f (, y ) (3.2) where V = objecive funcion; λ + = Lagrangian muliplier (marginal value of a uni change in ) f = dynamic (sock) equaion, as defined by Equaion (2.22), i.e. f = (c-) - cy In secion 3..2, we defined he Lagrangian funcion L of (3.) as follows: L= - { V (, y ) - ( - - f (, y )} + F( λ + + ) (3.3) =0 Rewriing he Lagrangian, equaion (3.3), o incorporae H resuls in: Ec 655, 3..3, Dynamic opimizaion, Discree ime

2 L= - =0 {V (, y ) - λ+ = - =0 + + λ {H + λ λ ( f(, y )}+ F( ) )}+ F( ) (3.4) Similar o (2.27) o (2.30), we can rewrie he firs-order condiions for he Lagrangian in erms of he Hamilonian: L H(.) = = 0 ; = 0,..., - y y (3.5) L H(.) = + λ+ - λ = 0 ; =,...,- (3.6) L F = - λ + = 0 (3.7) L λ H(.) = = 0 ; = 0,...,- (3.8) λ + + Modified se of firs-order condiions in erms of he Hamilonian, H (he ime subscrips for equaions (3.9) o (3.3) are indeed as above in (3.5) o (3.8)): H( ) = 0 y H(.) λ+ - λ = - H(.) + - = λ + F λ = 0 = a Maimum Principle cosae (adjoin) equaion sae equaion boundary (ransversaliy) condiion boundary (ransversaliy) condiion (3.9) (3.0) (3.) (3.2) (3.3) Ec 655, 3..3, Dynamic opimizaion, Discree ime 2

3 Equaion (3.9) is similar o he sandard marginal condiions of saic opimizaion. Equaion (3.0) represens he difference equaion for he adjoin (or cosae) variable λ. Equaion (3.) represens he difference equaion for he sae variables. Equaions (3.2) and (3.3) define saring and ending condiions and consrains. B. he Soluion Algorihm o a Finie Horizon Discree ime Dynamic Problem: develop an ieraive process ha leads o he soluion of equaions (3.9) o (3.3). Bu Firs, define some conceps ha furher characerize he ype of dynamic problem. Open Loop and Closed Loop Problems Open loop: y = y * () - he conrol variable, y, is no an eplici funcion of he sae variable,. Closed loop: y = y * ( (),) - he conrol variable is an eplici funcion of and iself. Coupled: he difference equaion for λ (or ) is also a funcion of he oher variable, (or λ ). For eample λ + - λ = f (, λ, y ). Uncoupled: λ ). he difference equaion for λ (or ) is no also a funcion of he oher variable, (or For eample λ + - λ = f (λ, y ) he series of seps ha will solve equaions (3.9) o (3.3) depend in par on wheher he problem is an open or closed loop and wheher he difference equaions (3.0) and (3.) are coupled or uncoupled. he four seps are oulined below. A. () Eliminae y from equaion (3.9) (Maimum principle) (2) here are wo difference equaions, one for and he oher for λ. If one of he difference equaions is "uncoupled", solve he sysem in sequence of ime, subsiuing in he saring and ending values of 0 and, and working forward and backward. B. () If he difference equaions are "coupled", hen eliminae λ +, o obain a second order difference equaion for. his will resul in he sysem of equaions having 3 ime periods, -, and +. Ec 655, 3..3, Dynamic opimizaion, Discree ime 3

4 (2) Deermine any unknowns by imposing given beginning and/or endpoin values ( o, ), and solve sequenially hrough ime. Eample: A discree ime problem wih fied erminal ime and fied erminal sae Suppose ha you wish o solve he following dynamic problem: 2 2 ( + y ) 3 Min y = 0 2 s.. + y = + 0 = 3 = 0 4 he Lagrangian for his problem is: he Hamilonian for his problem is: he necessary condiions for he Hamilonian are: hese condiions give us wha is necessary for a soluion. Now we can apply he 4 seps in he algorihm given above o solve for {, 2, 3, y 0, y, y 2, y 3, λ, λ 2, λ 3, λ 4 }. C. Discouning: he Presen-Valued Hamilonian and he Curren-Valued Hamilonian Ec 655, 3..3, Dynamic opimizaion, Discree ime 4

5 We can use he algorihm above o solve problems like he Robinson Crusoe problem. ha problem involves discouning, however, which we have no ye discussed, as i affecs he Hamilonian and dynamic opimizaion. In mos Resource Economics Applicaions of dynamic opimizaion, we maimize he presen value of discouned ne benefis. For his reason, le us consider incorporaing a discoun rae ino he opimizaion problem: Ma - =0 ρ V(, y )+ ρ F( ) s = f (, y ) 0 = a where ρ = ; + r r = ineres rae ρ = discoun facor (3.4) he Lagrangian associaed wih (3.4) is: L= - ρ =0 { V(, y ) - ρ λ+ ( f (, y )} + ρ F( ) (3.5) where ρ is a one period discoun facor. Noe ha λ + is he value of one more uni of + in +, bu V = V(,y ) is a value in. We shall now consider seing up he Hamilonian, H, for his problem. So, how do we define H? Recall, H is defined as a oal change in value in ime, as a resul of, y. H has wo pars, defined as () he direc benefi of in ime, and (2) he opporuniy cos of having consumed y, insead of using i o produce more + in ime +. In order o arrive a H, we mus make V and λ + comparable by discouning λ + by one period. hen, we can add his o he discouned values from he oher periods o ge he presen value of he Langrangian. We can define he Presen- Valued Hamilonian as: H( ) = ρ V(, y) + µ f(, y) where = ρ + µ λ + (3.6) Ec 655, 3..3, Dynamic opimizaion, Discree ime 5

6 Noes: µ = Presen Value Muliplier, he perspecive is from he poin of view of he ime period = 0. λ + = Curren Value Muliplier, he perspecive is from he poin of view of he specific ime period,. herefore, we can also define a Curren Value Hamilonian as: ~ H =V(, y )+ ρ λ + f (, y ) (3.7) he firs-order condiions of Curren-Valued Hamilonian become: ~ H y ρ λ = + 0 H ~ - λ = - H ~ (.) + - = ρ λ F(.) λ = 0 = a (3.8) (3.9) (3.20) (3.2) (3.22) Noe he relaionship beween he presen and curren valued Hamilonians: + H = ρ H Ec 655, 3..3, Dynamic opimizaion, Discree ime 6

7 D. Reurn o Robinson Crusoe s Problem Formulae as a Hamilonian, solving i wih he soluion algorihm given above. o simplify hings, assume ha here is a erminal condiion ha specifies a beques of he given amoun b, so ha (+) = b. We ll also add a discoun facor: ρ = /(+r) for 0 < ρ <. Using (+) = b, we can ignore he earlier salvage value given by F( ). he maimizaion problem he pioneer faces is as follows: [] Ma ρ =0 y α y Subjec o: = c( = a = b y + ) 0 Now, formulae his as a dynamic opimizaion problem using Opimal Conrol. Les firs rewrie he consrain as a difference equaion: [2] + = c( y ) [3] f(, y) = + = ( c ) cy We can now se up he Hamilonian: [4] H = ρ y α + µ f + Recall ha µ = ρ λ+, he discouned curren-period Lagrangian muliplier. Subsiuing f ino he Hamilonian we have: α [5] H = ρ y + µ {( c ) cy } Applying he Maimum Principle we have: H [6] = αρ y α cµ = 0 y Ec 655, 3..3, Dynamic opimizaion, Discree ime 7

8 H [7] µ µ = = [ µ ( c )] Adjoin equaion H [8] + = = ( c ) cy µ 0 = ; + [9] a = b Sock equaion From Equaion [6] we can solve for y : [0] y cµ = αρ α Simplifying for more ease laer: [] y cµ = αρ β where β = α Equaion [7] can be rewrien as: [2] µ µ = c µ + µ his is uncoupled - why? Muliply boh sides by and we have: c [3] µ = c µ Now saring wih he firs period saring condiions, we subsiue and have for period : [4] µ = µ 0 = µ 0 c c and for periods 2, 3,, k we have: µ 0 2 µ [5] µ 2 = = c = µ 0 c c c Ec 655, 3..3, Dynamic opimizaion, Discree ime 8

9 µ 0 3 [6] µ 3 = c = µ 0 c c c [7] µ k µ 0 c = k Now we ake he epressions for y and µ, as given in equaions [] and [7], and subsiue hese ino equaion [8], he sock equaion: [8] = ( c cy + ) [8] [9] β µ c + = αρ ( c ) c + = ( c ) ( ) c µ c 0 c αρ β Simplifying equaion [9] we ge: [20] + = c cθ v µ where, θ = 0c α β and v = ( ρc) β A = 0 we have: [2] = c cθ a Noe ha v 0 =, 0 = a and c = a consan given in he growh funcion. We sill do no know θ, which we will need o solve for. We can ge an epression for all by ieraively subsiuing, period by period: [22] = c0 cθ [23] = c cθ 2 v Subsiuing he epression for from equaion [22] ino equaion [23]: Ec 655, 3..3, Dynamic opimizaion, Discree ime 9

10 2 2 [24] = c c ( + v ) 2 0 θ c ec so ha if we ierae, subsiuing 2 ino he generalized epression for 3 and so on, we can derive a general epression for : [25] ( v c) ( ) v c = c 0 c θ where v c So now we have epressions for each. µ,, y ha we can use o characerize he opimal pahs for Firs, consider an absolue maimum harves over all ime. As here is no consumpion, and all harvess are replaned. his gives an upper bound for = +. Using equaion [8] and seing y = 0, we have: [26] c = We ge he above because we do no have a y o subsiue ino equaion [8]. So he final harves mus be smaller han his because y > 0. So really: + [27] + = γ c 0 where 0< γ < Noe: he soluion will depend on he value of b, he remaining sock size a = + relaive o + he values of oher consans. If b > c 0, hen here is no soluion because corn canno grow fas enough o saisfy a subsisence consrain on y. Le s go back o our general epression for. We impose he erminal sae condiion, and solve for an epression for θ ha is free of µ 0. [28] ( vc) ( vc) b a = θ c + + Now we have an epression for θ ha is made of all consans, and we have he resricion ha + b < c a. We can now characerize he opimal consumpion of y. [29] y = θ v Ec 655, 3..3, Dynamic opimizaion, Discree ime 0

11 where: v = ( ρ c) β and θ is as deermined in equaion [28]. Noe ha his epression for y is all consans, and so values for y can be deermined: β [ ] [30] y = θ v = θ ( ρ c) We can characerize he ime pah of y graphically: y Consumpion y decreases and increases wih ime depending on wheher v > or v < as described by equaion [30]. We have already assumed ha c <. So given ha ρ <, we know ha v < and y mus be decreasing over ime. Ec 655, 3..3, Dynamic opimizaion, Discree ime