his documen was generaed a 1:4 PM, 9/1/13 Copyrigh 213 Richard. Woodward 4. End poins and ransversaliy condiions AGEC 637-213 F z d Recall from Lecure 3 ha a ypical opimal conrol problem is o maimize (,, ) subjec o ɺ = f (,, z), he sae equaion. In ha lecure we saed ha he condiions ha characerize he soluion are 1. ma H,, z, λ for all, z ( ) [ ] H λ 2. = ɺ λ = H 3. = ɺ λ 4. ransversaliy condiion (such as, λ = ) In his lecure we focus primarily on condiion 4, he ransversaliy condiion. On our way o deriving he ransversaliy condiion, we also provide a somewha more formal derivaion of condiions 1-3. A ransversaliy condiion describes wha mus be saisfied a he end of he ime horizon. i.e. as we ransverse o he world beyond he planning horizon. he naure of he ransversaliy condiion depends grealy on he saemen of he problem. For eample, i migh be ha he sae variable,, mus equal zero a he erminal ime, i.e., =, or i migh be ha i mus be less han some funcion of, φ ( ) ime is fleible or. ( ) I. ransversaliy condiions for a variey of ending poins (Based on Chiang pp. 181-184). We also consider problems where he ending A. Verical or free-endpoin problems In a verical end poin ype problem, is fied and can ake on any value. his would be appropriae, for eample, if you are managing an asse or se of asses over a fied horizon and i you have no resricions on he condiion of he asses when you reach. We have considered his case previously. When looked a from he perspecive of he beginning of he planning horizon, he value ha akes on a is free and, moreover, i has no effec on wha happens in he fuure. So i is a fully free variable so we would maimize V over. Hence, i follows ha he shadow price of mus equal zero, giving us our ransversaliy condiion, λ =.
4-2 We will now confirm his inuiion by deriving he ransversaliy condiion for his paricular problem and a he same ime giving a more formal presenaion of Ponryagin s maimum principle. he objecive funcion is (,, ) V F z d now, seing up an equaion as a Lagrangian wih he sae-equaion consrain, we have L = F (,, z) + λ ( f (,, z) ) d ɺ. We pu he consrain inside he inegral because i mus hold a every poin in ime. Noe ha he shadow price variable, λ, is acually no a single variable, bu is insead defined a every poin in ime in he inerval o. Since he sae equaion mus be saisfied a each poin in λ f,, z ɺ = a each insan, so ha he value ( ) ime, a he opimum, i follows ha ( ) of L mus equal he value of V. Hence, we migh wrie insead ( ɺ ) (,, ) λ (,, ) V = F z + f z d or { (,, ) λ (,, )} V = F z f z λ + ɺ d. (,,, λ ) V = H z λ ɺ d I will be useful o reformulae he las erm in he inegrand, λɺ, using inegraion by pars: udv = vu vdu wih λ = u and = v, so ha dv = ɺ, we ge [ ] λ ɺ d = λ + ɺ λ d = ɺ λ d + λ λ so, we can rewrie V as
4-3 [, + ɺ λ ] d + λ λ = 1. V H (, z, λ) Derivaion of he maimum condiions (Based on Chiang chaper 7) A his poin we can easily derive he firs hree condiions of he maimum principal which mus hold over he inerval (, ). Assuming an inerior soluion and wice-differeniabiliy, a necessary condiion for an opimum is ha he firs derivaives of choice variables are equal o zero. Firs consider our choice variable, z. A each poin in ime i mus be ha V z =. his reduces o H z =, which is he firs of he condiions saed wihou proof in lecure 3. Ne, for all (,), is also a choice variable in 1 (hough clearly a consrained one), so i mus also hold ha V =. his reduces o if + ɺ λ = or = ɺ λ, he second of he condiions saed in lecure 3. H H Finally, he FOC wih respec o λ is more direcly derived from he Lagrangian above. L λ = f (,, z) ɺ, so his implies ha ( ) L λ = ɺ = f,, z be saisfied., which simply means ha he sae equaion mus Now consider he value of a he erminal ime,. If he erminal condiion is ha can ake on any value, hen i mus be ha he marginal value of a change in mus equal o zero, i.e., V/ =. Hence, he firs-order condiion of 1 wih respec o is V = λ =. he minus sign on he LHS is here because i reflecs he marginal cos of leaving a marginal uni of he sock a ime. In general, we can show ha λ is he value of an addiional uni of he sock a ime. Seing his FOC equal o zero, we obain he ransversaliy condiion, λ =. his confirms our inuiion ha since we're aemping o maimize V over our planning horizon, from he perspecive of he beginning of ha horizon is a variable o be chosen, i mus hold ha λ, he marginal value of an addiional uni of, mus equal zero. Noe ha his is he marginal value o V, i.e., o he sum of all benefis over ime for o, no he value o he benefi funcion, F( ). Alhough an addiional uni may add value if i arrived a ime, i.e., F ( ) >, he coss ha are necessary for ha marginal uni of o arrive a mus eacly balance he marginal benefi.
4-4 B. Horizonal erminal line or fied-endpoin problem In his case, here is no fied endpoin (e.g. =5), bu he ending sae variables mus have a given level. For eample, you can keep an asse as long as you wish, bu a he end of your use i mus be in a cerain sae. Again, we will use equaion 1: V [ H (, z, λ) + λ ] d + λ λ =, ɺ. In his case, however, we are no only choosing he pah of z, bu also he erminal ime,. So if our benefi funcion is F (,, z ), he opimizaion problem would be wrien, z, ( ) ma F,, z d. Hence, a he opimal erminal ime, i mus be he case ha V/ =; oherwise a change in would increase V; if V/ > we would wan o increase he ime horizon, and if V/ < i should be shorened. (Noe ha his is a necessary bu no sufficien condiion we will address he sufficien condiion when we inroduce he infinie horizon framework below). Evaluaing his derivaive using Leibniz s rule for he inegral and he produc rule for he las erm, we ge V = H (,, z, λ ) + ɺ λ ( ɺ λ + λ ) = ɺ. he second and hird erms in his equaion cancel and, since we are resriced o have equal o a specific value, i follows ha ɺ =. Hence, he only hing ha remains is he firs erm: H (,, z, λ ) =, or, epanding his, F ( z ) λ f ( z ) imporanly ha his holds only a =.,, +,, =. Noe C. Fied erminal Poin In his case boh and are fied. Such would be he case if you're managing he asse and, a he end of a fied amoun of ime you have o have he asse in a specified condiion. A simple case: you ren a car for 3 days and a he end of ha ime he gas ank has o have 5 gallons in i. here's nohing complicaed abou he ransversaliy condiion here, i is saisfied by he consrains on and, i.e. 3 =5.
4-5 When added o he oher opimum crieria, his ransversaliy equaion gives you enough equaions o solve he sysem and idenify he opimal pah. D. erminal Curve = ϕ ( ) In his case he erminal condiion is a funcion, ϕ ( ) ɺ. =. Again, we use 1 V = H (,, z, λ ) + λ d + λ λ aking he derivaive wih respec o and subsiuing in ɺ = φ' ( ) V = H (,, z, λ ) + ɺ λ ɺ λ λφ '( ) = which can be simplified o he ransversaliy condiion, V = H (,, z, λ ) λφ' ( ) = E. runcaed Verical erminal Line In his case he erminal ime is fied, bu can only ake on a se of values, e.g.. his would hold, for eample, in a siuaion where you are using a sock of inpus ha mus be used before you reach ime and. You can use he inpu from o, bu can never be negaive.
4-6 For such problems here are wo possible ransversaliy condiions. If >, hen he ransversaliy condiion λ = applies. On he oher hand, if he opimal pah is o reach he consrain on, hen he erminal condiion would be =. In general, he Kuhn-ucker specificaion is wha we wan. ha is, our maimizaion objecive is he same, bu we now have an inequaliy consrain, i.e., we're seeking o maimize V [ H (, z, λ) + λ ] d + λ λ =, ɺ s... he Kuhn-ucker condiions for he opimum hen are: λ,, and ( )λ = where he las of hese is he complemenary slackness condiion of he Kuhn-ucker condiions. As a pracical maer, raher han burying he problem in calculus and algebra, I sugges ha you would ypically ake an educaed guess. Is going o be greaer han? If you hink i is, hen solve, he problem firs using λ =. If your soluion leads o, you're done. If no, subsiue in = and solve again. his will usually work. F. runcaed Horizonal erminal Line In his case he ime is fleible up o a poin, e.g., ma, bu he sae is fied a a given level, say is fied. Again here are wo possibiliies, = ma or < ma. Using he horizonal erminal line resuls from above, he ransversaliy condiion akes on a form similar o he Kuhn-ucker condiions above, ma, H(,, z,λ ), and ( ma )H =. II. A word on salvage value he problems above have assumed ha all benefis and coss accrue during he planning horizon. However, for finie horizon problems i is ofen he case ha here are benefis or coss ha are funcions of a. For eample, owning and operaing a car is cerainly a dynamic problem and here is ypically some value (perhaps negaive) o your vehicle when you're finally finished wih i. Similarly, farm producion problems migh be hough of as a dynamic opimizaion problem in which here are coss during he growing season, followed by a salvage value a harves ime. Values ha accrue o he planner ouside of he planning horizon are referred o as salvage values. he general opimizaion problem wih salvage value becomes
4-7 ( ) + ( ) ma F,, z d S, s.. z ɺ = = (,, ) f z Rewriing equaion 1 wih he salvage value, we obain: [ + λ ] d + λ + S( ) = λ 1' H (, z, ) V λ ɺ,., For he verical end-poin problem we again wan o rea as a choice variable and ake he derivaive wih respec o ha variable, in his case yielding S (, ) S (, ) λ + = λ =. ( ) S, Inuiively, his makes sense: λ is he marginal value of he sock and is he marginal value of he sock ouside he planning horizon. When hese are equal, i means ha he marginal value of is only equal o he marginal value o he salvage value. Noe ha he addiion of he salvage value does no affec he Hamilonian, nor will i affec he firs 3 of he crieria ha mus be saisfied. Wha would be he ransversaliy condiion for a horizonal end-poin problem wih a salvage value? III. An imporan cavea Mos of he resuls above will no hold eacly if here are addiional consrains on he problem or if here is a salvage value. However, you should be able o derive similar ransversaliy condiions equaion 1 and similar logic. IV. Infinie horizon problems I is frequenly he case (I would argue, usually he case) ha he rue problem of ineres has an infinie horizon. he opimaliy condiions for an infinie horizon problem are idenical o hose of a finie horizon problem wih he ecepion of he ransversaliy condiion. Hence, in solving he problem he mos imporan change is how we deal wih he need for he ransversaliy condiions. [Obviously, in infinie horizon problems he mnemonic of ransversing o he oher side doesn' really work because here is no "oher side" o which we migh ransverse.] A. Fied and finie arge value for If we have a value of o which we mus arrive, i.e., lim =k, hen he problem is idenical o he horizonal erminal line case considered above. B. Fleible Recall from above ha for he finie horizon problem we used equaion 1: V = H (,, z, λ ) + ɺ λ d + λ λ. In he infinie horizon case his equaion is rewrien:
4-8 ( ) ɺ V = H,, z, λ λ d λ lim λ + + and, for problem in which is free, he condiion analogous o he ransversaliy condiion in he finie horizon case is lim λ =. Noe ha if our objecive is o maimize he presenvalue of benefis, his means ha he presen value of he marginal value of an addiional uni of mus go o zero as goes o infiniy. Hence, he curren value (a ime ) of an addiional uni of mus eiher be finie or grow a a rae slower han r so ha he discoun facor, e -r, pushes he presen value o zero. One way ha we frequenly presen he resuls of infinie horizon problems is o evaluae he equilibrium where ɺλ = ɺ =. Using hese equaions (and evaluaing convergence and sabiliy via a phase diagram) we can hen solve he problem. See he fishery problem in Lecure 3. V. Summary he cenral idea behind all ransversaliy condiions is ha if here is any fleibiliy a he end of he ime horizon, hen he marginal benefi from aking advanage of ha fleibiliy mus be zero a he opimum. You can apply his general principal o problems wih more han one variable, o problems wih consrains and, as we have seen, o problems wih a salvage value. VI. Reading for ne class Dorfman, Rober. 1969. An Economic Inerpreaion of Opimal Conrol heory. American Economic Review 59(5):817-31.