Essenial Microeconomics -- 6.5: OPIMAL CONROL Consider he following class of opimizaion problems Max{ U( k, x) + U+ ( k+ ) k+ k F( k, x)}. { x, k+ } = In he language of conrol heory, he vecor k is he vecor of sae variables in period. he decision-maker conrols he growh rae of his vecor by deciding he level of a vecor of conrol variables x. * his secion was inspired by Lawrence Evans discussion of opimal conrol. For a more comprehensive inroducion go on he web o hp://mah.berkeley.edu/~evans/conrol.course.pdf.
Essenial Microeconomics -2-6.5: OPIMAL CONROL Consider he following class of opimizaion problems Max{ U( k, x) + U+ ( k+ ) k+ k F( k, x)}. { x, k+ } = In he language of conrol heory, he vecor k is he vecor of sae variables in period. he decision-maker conrols he growh rae of his vecor by deciding he level of a vecor of conrol variables x. As long as U, =,..., + and F, =,..., are all concave funcions, he FOC are boh necessary and sufficien condiions for a maximum. he FOC are obained in he usual way by wriing down he Lagrangian. As we shall see, here is a simpler way o proceed. We only need o wrie down par of he Lagrangian. his is called he Hamilonian. Having done so we hen le he inerval beween periods approach zero and obain he limiing FOC. hese are he FOC for he coninuous version of he model.
Essenial Microeconomics -3- he choice of wheher o use a discree or a coninuous model is simply a modeling choice. I is ofen easier o work wih a coninuous model so undersanding he basic ideas of conrol heory is valuable. he goal here is o develop an undersanding of conrol heory and how o use i. For proofs see EM.
Essenial Microeconomics -4- Example: Life-cycle capial accumulaion An individual wih iniial capial k has a discouned life-ime uiliy of = δ U( x) + δ v( k+ ) where he final erm is his uiliy from her beques k +. Financial capial k has a gross reurn of + r. he individual s period wage is w. herefore if she consumes x her capial nex period is k = ( + r) k + + w x. Equivalenly, k k = + rk + w x Coninuous version 0 δ δ e u( x( )) + e v( k( )) where rk w() x() d = +
Essenial Microeconomics -5- he Lagrangian for he discree version of he model is L = U ( k, x ) + U ( k ) + λ ( F( k, x ) k + k ). = + + + Assuming an inerior soluion we have he following firs-order condiions: L U F = + λ = 0, =,...,, (6.5-) x x x L U F = + λ( + ) λ = 0, =,...,, and (6.5-2) k k k L k U λ + = = + k+ 0. (6.5-3) Noe ha if here is no value of he sae variable a ime, he final period value of he shadow price vecor, λ = 0. Rearranging he second condiion, we obain he following expression for he evoluion of he shadow prices U F λ λ = ( + λ ), =,...,. (6.5-4) k k
Essenial Microeconomics -6- Reducing he ime beween periods Firs we modify he discree ime model by inroducing an inerval Δ beween decision poins. We hen consider he necessary condiions as Δ approaches zero. o modify he discree model, suppose ha we have chosen he opimal conrols unil ime s. We divide he remaining ime ino sub-inervals of lengh Δ. We le x be he level of he conrol variable beween ime = s+ ( n ) Δ and s+ Δ. he rae a which he sae variable grows over his inerval is f( k, x ). hen he change in he sae variable beween decision poins is k k = F( k, x) + f( k, x) Δ. he funcion u( k, x) is now he flow of uiliy o he consumer. oal uiliy is herefore U ( k, x ) = u ( k, x ) Δ + U ( k ). + + = s = s
Essenial Microeconomics -7- Appealing o he firs-order condiions (6.5-) and (6.5-4) and and L u f = ( + λ ) Δ = 0, =,..., x x x u f λ λ = Δ ( + λ ), =,...,. k k U k + + ( k ) λ = 0 + Dividing he firs wo condiions by Δ and defining Δ λ = λ λ, he firs-order condiions become u x f + λ = 0 x Δ λ u f U and = ( + + λ ), =,...,, ( k+ ) = λ Δ k k k + (6.5-5)
Essenial Microeconomics -8- here is an easy way o wrie down hese condiions. We define he Hamilonian of he problem as H ( k, x ) u ( k, x ) + λ f ( k, x ) he necessary condiions (6.5-5) can hen be wrien succincly as follows: H x = 0 Δ λ H = Δ k, U k +, + ( k ) = λ. (6.5-6) + Finally, aking he limi as Δ 0, hese argumens srongly sugges he following necessary condiions for he coninuous ime version of he model H x = 0, d λ = H, U ( k( )) = λ( ) (6.5-7) d k hese are indeed he necessary condiions for an inerior soluion (see he ex.)
Essenial Microeconomics -9- Coninuous model Max{ u( k( ), x( ), ) d + v( k( ), )} x () X 0 d = f ( k ( ), x ( ), ) Paralleling he discree ime model, i is convenien o inroduce a vecor, λ (), of shadow prices for he n sae variables and define he Hamilonian where H( kx,, ) = ukx (,, ) + λ f ( kx,, ), dλ H U = and λ( ) = ( k( ), ). (6.5-8) d k k
Essenial Microeconomics -0- We have he following elegan heorem. Proposiion 6.5.: Ponryagin s Maximum Principle If x * () = arg Max{ uxk (,, ( )) d+ vk ( ( ), )} x X, where f ( k ( ), x ( ), ) d = 0 and he funcionsu and f are boh coninuously differeniable, hen here exiss a shadow price funcion n λ():[0, ] saisfying (6.5-8) such ha * x * MaxH k x x X () = arg ( (), (),). Noe ha hese are sronger and more general condiions han he condiions derived as he limi of he discree ime model. here we implicily assumed an inerior soluion for he conrol variable and obained he necessary condiion H ( k ( ), x ( ), ) = 0. x
Essenial Microeconomics -- Applicaion : Lifeime consumpion wih a saving rae r exceeding he discoun rae δ. A consumer earns income a a fixed rae y. Her iniial financial capial k (0) = 0 and consumpion pah x () has lifeime uiliy e δ u ( x ( )) d. 0 We assume ha u > 0, u < 0 and u (0) =. he las assumpion ensures ha opimal consumpion is sricly posiive. If a ime k, () her financial capial is posiive she can inves i a a rae r. he growh rae of her asses hus saisfies rk() y x () d = +. he consumer will never leave a sricly posiive capial sock because she could always do beer by spending more. hus hus k * ( ) 0. Moreover, lenders will never lend money ha canno be paid back k * ( ) 0. herefore his is a fixed endpoin problem.
Essenial Microeconomics -2- he profile * * ( k ( ), x ( )) mus solve he following fixed endpoin problem 2 δ Max{ e u( x()) d rk() w x(), k( ) 0 x () 0 = + =. d We wrie down he Hamilonian δ H = e u( x) + λ( rk + w x). hen dλ H = = rλ and (6.5-9) d k and H e δ = u ( x ) λ = 0. (6.5-0) x
Essenial Microeconomics -3- Since d λ = H = rλ i follows ha λ() = λ(0) e d k Subsiuing ino (6.5-2), r ( r) ( ( )) = (0). (*) u x e δ λ hus marginal uiliy increases over ime. Because u ( x) < 0 i follows ha consumpion increases wih. Phase diagram Financial capial increases if rk() w x() 0 d = + >. hus he phase diagram is as depiced in he Figure. In he inerior of phase II x () is increasing. herefore phase II is absorbing. A profile enering phase I remains in i. Also k is decreasing. Bu k(0) = k( ) = 0. hus he opimal pah mus begin in phase I. k () is increasing in phase I so here mus be some ime ˆ a which he opimal pah exis phase I and eners phase II.
Essenial Microeconomics -4- We have shown ha ( r) ( ( )) = (0) (*) u x e δ λ hen ( r δ ) u ( x(0)) = λ(0) = e u ( x( )) (**) Consider wo pahs wih iniial consumpion x (0) and x(0) > x(0). I follows from (**) ha x () > x () for all. hen k() < k() for all > 0. I follows ha here is a unique x (0) such ha k ( ) = 0.
Essenial Microeconomics -5- Exercise: oal wealh accumulaion Consider an income profile y(). oal capial (human plus financial) is r( s ) () () ( ) W k e ysds hen = + r( s ) () ( ) () ( () () dw = y r e ysds y rw k d d = d Also rk() y() x() d = =. herefore dw = rw () x where d rs W(0) = e y( s) ds. 0 Arguing as above, for any income profile y(), i is opimal o accumulae oal wealh ill some ime ˆ 0and hen o decrease oal wealh.
Essenial Microeconomics -6- Applicaion 2: Lifeime consumpion wih differen lending and borrowing raes A consumer earns income a a fixed rae y has iniial financial capial k (0) = 0 and consumpion pah x (). Her lifeime uiliy is e δ u ( x ( )) d. 0 We assume ha u > 0, u < 0 and u (0) =. he las assumpion ensures ha opimal consumpion is sricly posiive. If a ime k, () her financial capial is posiive she can inves i a a rae r. he growh rae of her asses hus saisfies rk() w x() d = +. If she is in deb she can borrow a a rae ρ and so her asse growh rae is d = ρk () + w x ().
Essenial Microeconomics -7- We consider he case where r < δ < ρ ; ha is, he consumer s discouns rae lies beween her reurn on invesing and her cos of borrowing. Le he lifeime uiliy maximizing profile be * * ( k ( ), x ( )). Consider any ime inerval [, 2] over which he following fixed endpoin problem k * () > 0. hen * * ( k ( ), x ( )) mus solve 2 δ * Max{ e u( x()) d = rk() + w x(), k( i) = k ( i), i =,2} x () 0 d. he consumer will never leave a sricly posiive capial sock because she could always do beer by spending more. hus hus k * ( ) 0. Moreover, lenders will never lend money ha canno be paid back k * ( ) 0. herefore his is a fixed endpoin problem.
Essenial Microeconomics -8- We wrie down he Hamilonian δ H = e u( x) + λ( rk + w x). hen dλ H = = rλ and (6.5-) d k H e δ = u ( x ) λ = 0. (6.5-2) x Figure 6.5.2: Phase diagram (lending) r From (6.5-), λ() = λ(0) e. Subsiuing ino (6.5-2), ( r) ( ( )) = (0). u x e δ λ hus marginal uiliy increases over ime. Because u ( x) < 0 i follows ha consumpion increases wih. Financial capial increases if rk() w x() 0 d = + >. hus he phase diagram is as depiced in Figure 6.5-2.
Essenial Microeconomics -9- o be in he inerior of phase II a ime, he capial sock mus be posiive. However iniial capial is zero. hus o ge o phase II he consumer mus firs have been in phase I. Bu noe ha if he consumer is ever in phase I, capial mus grow hereafer. hen, he erminal condiion canno be saisfied. herefore he Figure 6.5.2: Phase diagram (lending) consumer can never be in he inerior of phase I or phase II.
Essenial Microeconomics -20- We can similarly sudy sub-inervals [, 2] over which he consumer is in deb. Exercise: Confirm ha he new phase diagram is as shown Arguing as earlier, o be in he inerior of phase IV, he consumer mus have firs been in phase III. Bu consumpion and deb increase in phase III so any such pah canno ener phase IV. We can herefore conclude Figure 6.5.3: Phase diagram (borrowing) ha here is no such ime inerval [, 2]. I follows ha capial can never be posiive or negaive. hus he opimal consumpion profile is x () = w.