ECO 504 Spring 2006 Chris Sims RANDOM LAGRANGE MULTIPLIERS AND TRANSVERSALITY 1. INTRODUCTION Lagrange muliplier mehods are sandard fare in elemenary calculus courses, and hey play a cenral role in economic applicaions of calculus because hey ofen urn ou o have inerpreaions as prices or shadow prices. You have seen hem generalized o cover dynamic, non-sochasic models as Hamilonian mehods, or as byproducs of using Ponryagin s maximum principle. In saic models Lagrangian mehods reduce a consrained maximizaion problem o an equaion-solving problem. In dynamic models hey resul in an ordinary differenial equaion problem. In he sochasic models we are abou o consider hey resul in, for discree ime, an inegral equaion problem or, in coninuous ime, a parial differenial equaion problem. Inegral equaions and parial differenial equaions are harder o solve han ordinary equaions or differenial equaions hey are boh less likely o have an analyical soluion and more difficul o handle numerically. The applicaion of Lagrangian mehods o sochasic dynamic models herefore appears o be of less help in solving he opimizaion problem han is heir applicaion o non-sochasic problems. Consequenly many references on dynamic sochasic opimizaion give lile aenion o Lagrange mulipliers, insead emphasizing more direc mehods for obaining soluions. The economic lieraure has o some exen been guided by his paern of emphasis. This is unforunae, because Lagrangian mehods are as helpful in economic inerpreaion of models in sochasic as in non-sochasic models. Also, in general equilibrium models, use of Lagrangian mehods urns ou someimes o simplify he compuaional problem, in comparison o approaches ha ry o solve by more direc mehods all he separae opimizaions embedded in he general equilibrium. 2. STATEMENT OF THE PROBLEM AND THE EULER EQUATION FIRST ORDER CONDITIONS Since in his course we are more ineresed in using hese resuls han in proving hem, we presen hem backwards. Tha is, we begin by wriing down he resul we are aiming a, hen prove ha i is par of a se of sufficien condiions for an opimum. The firs-order condiions we display are in fac also necessary condiions for an opimum under regulariy condiions ha ofen apply in economic models, bu we do no in his se of noes prove ha. A more complee presenaion, ha however gives less aenion o infinie-horizon problems, is in Kushner 1965b and Kushner, 1965a. c 2006 by Chrisopher A. Sims. This documen may be reproduced for educaional and research purposes, so long as he copies conain his noice and are reained for personal use or disribued free.
Noe ha in his course you will be responsible for knowing how o use he condiions displayed in hese noes o analyze and solve economic models, no for reproducing proofs of necessiy or sufficiency. We consider a problem of he form [ maxe β U C C,Z ] 1 0 subjec o =0 g C,Z 0, = 0,...,, 2 where we are using he noaion C n m = {C s,s = m,...,n}. We assume ha he vecor Z is an exogenous sochasic process, ha is, ha i canno be influenced by he vecor of variables ha we can choose, C. For a dynamic, sochasic seing, he informaion srucure is an essenial aspec of any problem saemen. Informaion is revealed over ime, and decisions made a a ime can depend only on he informaion ha has been revealed by ime. Here, we assume ha wha is known a is Z, i.e. curren and pas values of he exogenous variables in he model. 1 The class of sochasic processes C ha have his propery are said o be adaped o he informaion srucure. We can generae firs order condiions for his problem by firs wriing down a Lagrangian expression, [ E β U C,Z β λ g C,Z ], 3 =0 =0 and hen differeniaing i o form he FOC s: Noice ha: [ β E β s U +s s=0 C β s g +s s=0 C λ +s ] = 0, = 0,..., 4 In conras o he deerminisic case, he Lagrangian in 3 and he FOC s in 4 involve expecaion operaors. The expecaion operaor in he FOC is E, condiional expecaion given he informaion se available a, he dae of he choice variable vecor C wih respec o which he FOC is aken. Because U and g each depend only on C s daed and earlier, he infinie sums in 4 involve only U s and g s daed and laer. 2 1 I may seem ha i would be naural o include also pas C s in he informaion se. Bu i is our assumpion ha his would be redundan. Of course a decision maker could make C depend on some exraneous" random elemen like a coin flip. Our assumpion is simply ha if his can occur, he coin flip is par of Z
3 3. REVIEW OF FINITE-DIMENSIONAL, NON-STOCHASTIC KUHN-TUCKER CONDITIONS In finie-dimensional problems, firs order condiions are necessary and sufficien condiions for an opimum in a problem wih concave objecive funcions and convex consrain ses. The condiions in 4 are no as powerful, because his is an infinie-horizon problem. Firs order condiions here, as in simpler problems, are applicaions of he: Separaing Hyperplane Theorem: If x maximizes he coninuous, concave funcion V over a convex consrain se Γ in some linear space, and if here is an infeasible x wih V x >V x, hen here is a coninuous linear funcion f such ha f x > f x implies ha x lies ouside Γ and f x < f x implies V x < V x. In a finie-dimensional problem wih x n 1, we can always wrie any such f as f x = n f i x i 5 i=1 where he f i are all real numbers. If he problem has differeniable V and differeniable consrains of he form g i x 0, hen i will also be rue ha we can always pick and nearly always wrie f i = V x i x 6 g j x f x = λ j x 7 j x wih λ i 0, all i. The nearly" is necessary because of wha is known as he consrain qualificaion". I is possible ha he firs-order properies of he consrains near he opimum do no give a good local characerizaion of he consrain se Γ. However, if we can find an x vecor and a se of non-negaive λ i s ha saisfy he consrains and 6 and 7, we have found he separaing hyperplane and hence he opimum. The sandard Lagrange muliplier equaions are herefore sufficien condiions for an opimum, and hey are nearly" necessary: We know here will always be a separaing hyperplane, and usually we will be able o wrie i in he form 7, bu here are some knife-edge i.e., rare special cases in which his will no be rue. This jusifies he common sraegy of rying o solve such problems by looking for soluions o 6 and 7. The sufficiency par of hese resuls can be summarized as: Kuhn-Tucker Theorem: 2 If V is a coninuous, concave funcion on a finie-dimensional linear space, V is differeniable a x, g i, i = 1,...,k are convex funcions, each differeniable a x, 2 This version of he Kuhn-Tucker heorem is no he mos general possible, even for finie-dimensional spaces.
4 here is a se of non-negaive numbers λ i,i = 1,...,k such ha V x x g i x = λ i, and i x g i x 0, λ i g i x = 0, i = 1,...,k, hen x maximizes V over he se of x s saisfying g i x 0, i = 1,...,k. 4. COMPLICATIONS FROM AN INFINITE HORIZON Bu in an infinie dimensional space i may no be rue ha we can wrie every coninuous linear funcion as an infinie sum analogous o 5, and he poenially infinie sums in 7 and in 5 wih f defined by 6 migh no converge. These complicaions provide addiional reasons ha here can be models in which he Lagrange muliplier equaions are no necessary condiions for an opimum, bu more imporanly hey mean ha hey are no longer sufficien condiions, even for problems wih concave objecive funcions and convex consrain ses. I is o handle hese problems ha we impose on infinie horizon problems wha are called ransversaliy condiions. To apply he Lagrange muliplier idea o our curren problem, inerpre V as given by he maximand in 1, x as being C, he opimal C sequence, and x as being a generic C sequence. In our sochasic problem, 5-7 become C 0,Z0 =0 s=0 E β U C s = f C 0 = E =0 β λ s=0 g C 0,Z 0 C s 8 In order o ge from 8 wha are given as FOC s in 4 above, we inerchange he order of summaion in he expressions on he lef and righ sides of 8, hen equae coefficiens of correspondingly subscriped C s. The version of 8 wih orders of summaion inerchanged is E U β C 0,Z0 C s = E g C 0,Z0 β λ C s, 9 s=0 =s s=0=s from which i is easy o see ha 4 follows, if we equae he coefficiens on C s erms on he wo sides of he equaion. Bu o jusify hese manipulaions, we mus be careful abou issues of convergence. Dealing wih convergence of hese sums is checking ransversaliy. Noe ha simply equaing coefficiens" on he lef and righ of 9 migh seem o imply 4 eiher wihou he E " operaor or wih an unsubscriped E" operaor. To undersand why he E appears, remember ha C is a random variable, a rule for choosing a numerical value for C as a funcion of informaion available a. Is coefficien" in 9 is herefore he sum of all he erms ha muliply i, over boh daes and possible saes of he world given informaion a. I is he sum over saes consisen wih informaion available a ha resuls in he E operaor in he FOC s. The need for he E is explained more precisely in foonoe 3 below, during he formal argumen for sufficiency.
5. SUFFICIENT CONDITIONS FOR THE FINITE LAG, STOCHASTIC, INFINITE HORIZON CASE In mos economic models, here are only finiely many lags as argumens o g and U, which makes many of he infinie sums in 8 and 9 become finie. In fac mos commonly U has no lags in is argumens. To ge versions of ransversaliy ha are closer o wha is commonly discussed in economic models and allow us o prove resuls, we now specialize o he case where U = U C,C 1,Z and g = gc,c 1,Z. This allows us o wrie a version of he Kuhn-Tucker heorem for infinie-dimensional spaces as: Infinie-Dimensional Kuhn-Tucker: Suppose i V [ ] T C,Z = liminf E 0 β U C,C 1,Z ; T =0 ii U is concave and each elemen of gc,c 1 is convex in C and C 1 for each Z, and all ineger 0; iii here is a sequence of random variables C 0 such ha each C is a funcion only of informaion available a, V C,Z is finie wih he parial sums defining i on he righ hand side of 5 converging o a limi, and, for each 0, gc,c 1,Z 0; iv U and g are boh differeniable in C and C 1 for each Z and he derivaives have finie expecaion; v There is a sequence of non-negaive random vecors λ 0, wih each λ in he corresponding informaion se a, and saisfying λ g C, C 1,Z = 0 wih probabiliy one for all ; vi U [ ] C, C 1,Z U C +1, C,Z +1 + βe C C g C, C 1,Z g = λ + βe [λ ] C +1, C,Z +1 10 C C for all i.e., he Euler equaions hold; vii ransversaliy for every feasible C sequence Ĉ 0, eiher V Ĉ 0 < V C 0 or [ U C lim supβ, C 1,Z g C, C 1,Z E λ Ĉ ] C 0 11 C C Then C 0 maximizes V subjec o gc,c 1,Z 0 for all 0 and o he given non-random value of C 1. Proof: Suppose Ĉ 0 is a feasible sequence of consumpion choice rules ha achieves a higher value of V han does C 0, despie C 0 s saisfying he condiions of he heorem. We simplify noaion from his poin on by using U for U C, C 1,Z 5
lim E T and using g for g C, C 1,Z. By differeniabiliy and by concaviy of U and convexiy of g, we know ha for each D 1 U Ĉ C + D2 U Ĉ 1 C 1 U Ĉ,Ĉ 1,Z U 12 and similarly D 1 g Ĉ C + D2 g Ĉ 1 C 1 g Ĉ,Ĉ 1,Z g 13 Using 12, he definiion of V, and our working hypohesis ha Ĉ gives a higher value of V han does C, we conclude ha lim E T [ T β D 1 U Ĉ C + D2 U Ĉ ] 1 C 1 > 0 14 =0 where, because C 1 is given exogenously, Ĉ 1 C 1 = 0 for any feasible Ĉ sequence. The law of ieraed expecaions allows us o rewrie 14 as [ T β D 1 U Ĉ C + E 1 D 2 U Ĉ ] 1 C 1 = =0 lim E T [ T 1 { β D 1 U + E [D 2 U +1 ] Ĉ C } ] + β T D 1 U T Ĉ T C T =0 > 0. 15 Then he Euler equaions as given in vi assure us ha 15 equaes erm by erm, excep for a lefover erm on he end, o he expeced sum of he gradiens of g, weighed by he he λ sequence. 3 In paricular, 15 is exacly [ T E β lim λ D 1 g Ĉ C + D2 g Ĉ ] 1 C 1 =0 T +E [ β T D 1 U T λ T D 1 g T Ĉ ] 16 C Since he Ĉ sequence is by hypohesis feasible, since λ 0, and since λ g = 0 wih probabiliy one, λ g Ĉ,Ĉ 1,Z g 0. The firs expecaion wihin curly brackes in 16 is herefore less han or equal o zero for every T, by convexiy of g. Thus he firs erm has a lim sup less han or equal o zero. The non-posiiviy of he lim sup of he second erm in he curly 3 Noe ha i is in his las sep ha we use he fac ha he FOC wih respec o C has E " in fron of i. If we had only an E" in fron of i, we would no be able o apply he law of ieraed expecaions here. The argumen we are making would go hrough if he Euler equaions were wrien wihou expecaions, since hese much sronger condiions would imply he Euler equaions wih E in fron. Bu of course weaker sufficien condiions are more useful han sronger ones. 6
brackes is exacly wha we assumed in our ransversaliy condiion 11. This complees he proof by conradicion: while 14 has o exceed zero if Ĉ improves on C, he condiions of he heorem guaranee ha i is equal o 16, which has o be non-posiive. Necessiy of Transversaliy: While here are necessiy resuls for ransversaliy condiions in some conexs, i appears difficul o obain one for he seup here. When we specialize o he case of dynamic programming, we will ge necessary condiions. Noe ha he ransversaliy condiion 11 is no in quie he usual form. The usual form would simply asser g [λ ] C, C 1,Z β E C C 7 0 17 Ofen in economic models he U erms in he rue ransversaliy condiion as given in 11 drop ou or converge o zero auomaically. 17 hen guaranees ransversaliy a one paricular poin, Ĉ 0 = 0 which, hough i is feasible in mos economic models, need no always be feasible. The convenional ransversaliy condiion is also oo srong in ha i requires acual convergence, raher han only ha he lim inf be non-negaive. I is oo weak in ha i checks only one poin in he feasible se. This means ha here are models in which, if we replaced our condiion 11 by 17, here would be C sequences ha saisfy all he condiions of he modified heorem ha are no in fac opima. A leading example of such a model is he linear-quadraic permanen income model wih a borrowing consrain replacing he usual bound on he rae of growh of wealh. The sandard linear decision rule is no opimal in such a case, bu i saisfies he sandard ransversaliy condiion 17, while failing our condiion 11. There are also models in which here is an opimum, saisfying he Euler equaions, bu he sandard ransversaliy condiion does no hold a he opimum. An example is he original Ramsey growh model, wihou discouning. This example is explained in Barro and Sala- I-Marin, 1995, p.507-8. 4 REFERENCES BARRO, R. J., AND X. SALA-I-MARTIN 1995: Economic Growh. McGraw-Hill, New York. KUSHNER, H. J. 1965a: On Sochasic Exremum Problems: Calculus, Journal of Mahemaical Analysis and Applicaions, 10, 354 367. 1965b: On he Sochasic Maximum Principle: Fixed Time of Conrol, Journal of Mahemaical Analysis and Applicaions, 11, 78 92. 4 Noe, hough, ha Barro and Sala-I-Marin s claim ha in wo references hey cie he sandard ransversaliy condiion has been shown o be necessary when here is ime discouning and he objecive funcion converges" is misleading. The references hey cie assume regulariy condiions ha rule ou uiliy funcions and echnologies ha appear in sandard macroeconomic growh models.