M.I.T. 5.450-Fall 00 Sloan School of Managemen Professor Leonid Kogan Examples of Dynamic Programming Problems Problem A given quaniy X of a single resource is o be allocaed opimally among N producion processes. Each process produces an oupu of he same good in he amoun x, where x is he amoun of inpu (x has o be nonnegaive). Use dynamic programming o deermine he allocaion of he resource x n, n =,..., N, among he producion processes ha maximizes he aggregae oupu.. Enumerae he producion processes o n. Suppose ha cerain amoun of he resource has been already allocaed among he firs n processes. Le he remaining amoun be X n. Le J n (X n ) be he aggregae oupu of he remaining N n producion processes, given ha he inpu X n is disribued opimally among hem. Show by inducion ha he value funcion J n (X n ) has he form J n (X n ) = c n Xn. Calculae he consans c n for n =,..., N.. Use he fac ha X = X o find x n, n =,..., N, and he opimal aggregae oupu J. Problem Consider he following dynamic invesmen problem. The mare consiss of wo asses: he risless asse and he risy asse. Boh asses are raded periodically a ime periods = 0,,..., T. The ne simple reurn over a single holding period on he risless asse is denoed by R f (e.g., R f = 5%), while he ne simple reurn on he risy asse is denoed by R. I is assumed ha he disribuion of he reurns on he risy asse is given by R = µ + σɛ, where ɛ, = 0,,..., T are independenly and idenically disribued sandard normal random variables, i.e., ɛ N (0, ). Invesor sees o maximize he expeced uiliy of wealh a ime T. Her uiliy funcion is exponenial: U(x) = e γx. The iniial wealh is denoed by W 0. There are no consrains on shor-sales and borrowing and here are no ransacions coss.. Le he conrol variable be x he amoun of wealh invesed in he risy asse a ime. Express he wealh W + a ime + as a funcion of he wealh W a ime and x.. Le J (W ) denoe he value funcion (he indirec uiliy funcion) of wealh W a ime. Show using inducion ha he value funcion has he funcional form J (x) = a e bx.
Show ha he opimal amoun of wealh allocaed ino he risy asse, depend on he curren level of wealh W. x, does no 3. Find a recursive relaion for a and b. Wha is he opimal invesmen sraegy x, = 0,,..., T? How does i depend on he ris-aversion parameer γ, he mean and he variance of he reurns on he risy asse and on he invesmen horizon T? Soluion of Problem. When n = N, all he remaining resource X N should be allocaed ino a single remaining producion process N, i.e., J N (X N ) = X N. Thus, c N =. Le s assume ha for n +, J n (X n ) = c n Xn. Noe ha X + = X x. Therefore, according o he Bellman opimaliy principle, J (X ) = max ( x + J + (X x )), s.. x X. () x The soluion of his problem x From () we find ha can be found from he firs-order condiion c+ + = 0. () X x Then, according o (), J (X ) = + c + X. Thus, c = + c + and we conclude he inducion argumen. x x (X ) = X. + c + (3) In order o find all consans cn explicily, noe ha = and c = + c Therefore, c = N n + and cn = N n +. n. Using he relaion X n+ = X n xn(x n) and 3, we conclude ha c n+ N n = X n+ X n = X. n + c N n + n+ c N n n+. Since X = X, N N N n + N n + X n = X = X. N N N n + N
We use (3) again o conclude ha x = n N + X X =. + N N N Thus, he resource has o be allocaed evenly among all N producion processes. Also = c X = N X. J Soluion of Problem Le Z denoe he excess reurn on he risy asse, i.e., Z = R R f.. If x is he amoun of wealh allocaed ino he risy asse, W x mus be allocaed ino he risless asse. Then W + = x ( + R ) + (W x )( + R f ) = W ( + R f ) + x Z. (4). When = T, J T = e γw T. Thus, a T = and b = γ. Le s assume ha for n +, J n (W n ) = a n e bnwn. Then, according o he Bellman opimaliy principle and (4), J (W ) = max E [ a exp( b W ( + R f ) b x Z )]. x Nex, noe ha E [exp( b x Z )] = E [exp( b x (µ R f + σɛ ))] = exp b x (µ R f ) + σ b x. Thus, [ ] J (W ) = max E a exp b W ( + R f ) b x (µ R f ) + σ b x. x The firs-order condiion for he maximizaion problem is equivalen o which implies ha b (µ R f ) + σ b x = 0, (W ) = µ Rf σ. (5) b x Thus, he opimal amoun of wealh allocaed ino he risy asse x does no depend on he curren level of wealh W. We find ha J (W ) = a exp b W ( + R f ) (µ R f), (6) σ which concludes he sep of inducion. 3
3. From (6) we observe ha (µ R f ) a = a + exp, b = b + ( + R f ). σ We can solve hese recursive equaions using he erminal condiions a T =, b T = γ o obain T (µ R f ) a = γ exp σ, b = ( + R f ) T. We now combine his wih (5) o obain x = µ R f ( + R σ f ) (T ). γ From his we conclude ha x is higher when he ris-aversion parameer γ is lower; i increases linearly wih he mean excess reurn on he risless asse and i is inversely proporional o he variance of he reurns; i is lower for longer invesmen horizons T. 4
MIT OpenCourseWare hp://ocw.mi.edu 5.450 Analyics of Finance Fall 00 For informaion abou ciing hese maerials or our Terms of Use, visi: hp://ocw.mi.edu/erms.