Portfolio Optimization in discrete time

Portfolio Optimization in discrete time Wolfgang J. Runggaldier Dipartimento di Matematica Pura ed Applicata Universitá di Padova, Padova http://www.math.unipd.it/runggaldier/index.html Abstract he paper is intended as a survey of some of the main aspects of portfolio optimization in discrete time. We consider three of the major criteria and discuss the method of Dynamic programming as well as the so-called martingale method as possible solution methods. We work out explicit examples for a logarithmic utility function and for the case of the complete binomial as well as the incomplete trinomial market models. Introduction he paper is intended as a survey of some of the main aspects concerning portfolio optimization in discrete time. We consider the three major types of problems : Maximization of expected utility from terminal wealth Maximization of expected utility from consumption Maximization of expected utility from consumption and terminal wealth and discuss in more detail the case of a logarithmic utility function. For what concerns the solution methodologies we discuss the method of Dynamic programming (DP) as well as the so-called martingale method. he latter method varies according to whether the market is complete or not, i.e. whether there exists or not an unique equivalent martingale measure. As example of a complete market in discrete time we consider the classical binomial market model and as an instance of an incomplete model we consider the trinomial model when there is only one risky underlying asset. In section 2 we describe the problem setup and introduce the main definitions. he two main market models, the complete binomial model and the incomplete trinomial model, are then discussed in section 3 where we discuss also the equivalent martingale measures. A general description of the two main solution methodologies, namely DP and the martingale method, are then given in section 4 and their application to the solution of the three types of problems for the specific case of

a logarithmic utility function are described in section 5. o better illustrate the computations described in section 5, some specific examples are explicitly solved in section 6. In the text we shall not make specific references to the literature, but we limit ourselves to mention here that the main source of reference is the book [] with additional references being [2],[3],[4]. 2 Setting and problem definition 2. Assets and their dynamics We consider a financial market, in which the prices of the assets evolve in discrete time. i.e. for t =,, on an underlying probability space (Ω, F, F t, P ). here is a (locally) non risky asset, the price of which evolves as B t+ = ( + r t )B t where r t is the so-called short rate of interest, and a certain number K of risky assets with price vector S t = (S t,, S K t ). he generic S i t evolves according to S i t+ = S i tξ i t+ () with (ξ i t) i.i.d. sequences of random variables so that, as processes, S i t are Markov. Note that we could equivalently write S i t+ = S i t( + ξ i t+) where ( ξ t) i can be interpreted as random returns; in this case one has in fact Si t+ Si t St i ξ t+. i Notice also that as filtration we may take F t = σ{s u, u t. = 2.2 Portfolio strategies and self-financing portfolios A portfolio strategy is a predictable process α t = (α 0 t, α t,, α K t ), i.e. with α i t F t where α i t denotes the number of units of the i th risky asset held in the portfolio in period t (i =,, K) while α 0 t denotes the number of units of the non-risky asset. he value of the portfolio corresponding to α is thus V t = α 0 t B t + αts i t i (2) and we shall use the symbol α to denote both the strategy and the corresponding portfolio. Ignoring for the moment the possibility of consumption, we shall give the following 2 i=

Definition. he portfolio α is said to be self-financing if V t+ V t = α 0 t+ B t + ( S i t = S i t+ S i t) and this can equivalently be expressed as i= α i t+ S i t (3) α 0 t B t + αts i t i = αt+b 0 t + i= αt+s i t i (4) i= Furthermore, we give Definition 2. We call gains process of a self-financing strategy α the process t G t = αs+ B 0 s + s=0 t s=0 i= α i s+ S i s he value in t of a self-financing portfolio with initial wealth V 0 can then be expressed as V t = V 0 + G t Later on it will be convenient to consider discounted values expressed in units of the non-risky asset B t. In this case B t = 0 and, denoting discounted values with a tilde, the discounted gains process G t for a self-financing strategy α becomes G t = with S i t = S t+ S t and we have t (it is customary to normalize B 0 to so that Ṽ0 = V 0 ). If we allow for the possibility of consumption, we have s=0 αs+ i S s i (5) i= Ṽ t = V 0 + G t (6) Definition 3. A consumption process C is a non-negative, adapted process C = (C t ) where C t denotes the amount of funds consumed in period t. Definition 4. A Consumption-investment plan/strategy is a pair (C, α) with α a portfolio strategy. On the basis of (2) and (4) the self-financing condition then becomes V t = C t + α 0 t+b t + i= α i t+s i t 3

implying and respectively. t V t = V 0 + G t C s (7) Ṽ t = V 0 + G t s=0 t Definition 5. he plan (C, α) is said to be admissible if C V. 2.3 Investment criteria s=0 C s (8) We first recall the standard notion of a utility function that, as a function of the wealth process V = (V t ) is such that strictly increasing u(v ) = strictly concave, differentiable u ( ) := lim V + u (V ) = 0 lim u (V ) = + V 0 + It will be convenient to extend this definition also to V < 0 by putting u(v ) = there. Examples of utility functions (in the range V > 0) are u(v ) = log V the log-utility function and u(v ) = V α (0 < α < )the power utility function for a risk-averse investor α We may now consider three investment criteria : Maximization of expected utility from terminal wealth max E{u(V α ) α with V 0 = v α : self-financing (predictable) Maximization of expected utility from consumption { max E β t u(c t ) (C,α) t=0 with V 0 = v and β (0, ) a discount factor (C, α) : self-financing and admissible 4

Maximization of expected utility from consumption and terminal wealth { max E β t u c (C t ) + β u p (V α C ) (C,α) A v with A v : t=0 α : self-financing and predictable C : nonnegative, adapted, C V 3 Market models and equivalent martingale measures Before coming to possible market models, let us recall the notion of an equivalent martingale measure. Given the real world probability measure P on (Ω, F), a probability measure Q P, i.e. a measure Q that has the same null-sets as P, is called equivalent martingale measure if all the assets in the market, expressed in units of a same asset, for which one usually takes the locally non-risky asset B t, are (Q, F t ) martingales. More precisely if, for i =,, K, we have { S E Q i t+ F t = Si t (9) B t+ B t which is equivalent to requiring E Q {S i t+ F t = B t+ B t S i t = ( + r t ) S i t (0) Recalling the definition of Gt in (5), from (9) it is easily seen that E Q { G t = E Q {E Q { G t F t = 0. From (6) it then follows that, under Q, not only the discounted price processes, but also the discounted value processes of a self financing strategy are all martingales (In the case when there is also consumption, from the positivity of C t and (8) it follows that Ṽt is a supermartingale). In what follows we shall distinguish the case when there is a unique equivalent martingale measure and when there are more. he first case corresponds to what is called a complete market, a notion that comes from the problem of hedging a contingent claim that we do not consider here and so we shall simply identify a complete market as one where there is a unique equivalent martingale measure and incomplete market as one where there are more. In our incomplete markets below the set of equivalent martingale measures will be a convex polyhedron with a finite number of vertices (extreme points) that we shall call the extremal martingale measures. 3. A complete market model : the binomial or Cox-Ingersoll- Ross model We shall begin by recalling a Bernoulli process that is a process {X t, t [0, ] N having the property that X t i.i.d. P {X t = = P {X t = 0 = p (0, ) 5

We may consider it as defined on the following sample space (space of elementary events) Ω = {ω with ω of the form ω = (0,, 0, 0,,, {{ ) times he number of elements in Ω is finite and given by 2 and the probabilities in the measure P are given by P {ω = p n ( p) n (if ω t = n) t= Letting N t := X + + X t, t [0, ] N then ( ) t P {N t = n = p n ( p) t n n 3.. Binomial market model We shall consider here a market with one locally riskless asset and a single risky asset (K = ) evolving according to (), i.e. as S t+ = S t ξ t+ where ξ t are i.i.d. and such that u with probab. p ξ t = d with probab. p It follows that we have the representation ξ t = ux t + d( X t ) with X t a Bernoulli process with parameter p. If N t corresponds to this Bernoulli process, then S t = S 0 u Nt d t Nt and. P {S t = S 0 u n d t n = ( t n ) p n ( p) t n 3..2 Martingale measure in the binomial market model We had seen at the beginning of this section 3 that an equivalent martingale measure is a measure Q P on Ω such that (9) holds. Putting q := Q{X t+ = F t 6

and letting for simplicity r t r, relation (9) becomes in the present case namely (qu + ( q)d) S t = ( + r) S t q = + r d u d his q is unique and (0, ) provided d < + r < u. Recalling the sample space Ω of the binomial model, given ω Ω with i ω i = n we can write P (ω) = p n ( p) n ; Q(ω) = q n ( q) n so that for the Radon-Nikodym derivative on Ω we obtain L(ω) = Q(ω) P (ω) = ( q p ) n ( ) n q () p 3.2 An incomplete market model : the trinomial model Corresponding to the Bernoulli process for the binomial model here we consider a process {X t, t [0, ] N characterized by P {X t = = p X t i.i.d. P {X t = 2 = p 2 t P {X t = 3 = p 3 p + p 2 + p 3 = he sample space can here be represented as Ω = {ω with ω of the form ω = (, 3, 2, 2,, 3, {{ ) times and #Ω = 3. he probabilities in the measure P are given by P (ω) = p n p n 2 2 p n 3 3 if t= {ω t = = n t= {ω t =2 = n 2 t= {ω t =3 = n 3 Letting this time N i t := t {Xs=i, i =, 2, 3 ; t [0, ] N (2) s= one has P {Nt = n, Nt 2 = n 2, Nt 3 t! = n 3 = n!n 2!n 3! pn p n 2 2 p n 3 3 (n + n 2 + n 3 = t) 7

3.2. rinomial market model We shall consider here too a market with one locally riskless asset and a single risky asset (K = ) evolving according to (), i.e. as S t+ = S t ξ t+ where ξ t are i.i.d. and such that u with probab. p ξ t = m with probab. p 2 It follows that we have the representation d with probab. p 3 ξ t = u {Xt= + m {Xt=2 + d {Xt=3 and, if N i t, i =, 2, 3, is as in (2), then we can write S t = S 0 u N t m N 2 t d N 3 t and P {S t = S 0 u n m n 2 d n 3 t! = n!n 2!n 3! pn p n 2 2 p n 3 3 (n + n 2 + n 3 = t) 3.2.2 Martingale measures in the trinomial market model As for the binomial market model, an equivalent martingale measure is also here a measure Q P on Ω such that (9) holds. Notice first that, if X t are i.i.d. under P, this does not necessarily imply that they are independent also under Q. Put therefore q i (t + ) := Q{X t+ = i F t ; i =, 2, 3 ; t = 0, and let also here for simplicity r t r. Relation (9) then becomes in the present case { (q (t + )u + q 2 (t + )m + q 3 (t + )d) S t = ( + r) S t q (t + ) + q 2 (t + ) + q 3 (t + ) = he latter, which is a condition on the conditional probabilities in the generic period t, admits infinitely many solutions (q (t), q 2 (t), q 3 (t)) Σ 3 (simplex in R 3 ) and notice that, since the coefficients in the above system are independent of t and F t, the solutions are also independent of t and F t, i.e. (q (t), q 2 (t), q 3 (t)) (q, q 2, q 3 ). hese solutions form a convex set (segment) characterized by vertices (extremal points) that are independent of t and differ according to whether m + r or m < + r. he st vertex is given by (q, 0 q2, 0 q3) 0 = ( ) 0, +r d, m (+r) m d m d ( ) (+r) m, u (+r), 0 u m u m 8 if if m + r m < + r

and the 2nd is characterized by ( ( + r) d (q, q2, q3) =, 0, u d ) u ( + r) u d hese extremal probability measures are not equivalent to (p, p 2, p 3 ) (all p i are > 0). he equivalent measures are obtained as the convex combinations (q γ, q γ 2, q γ 3 ) = γ (q 0, q 0 2, q 0 3) + ( γ) (q, q 2, q 3) for γ (0, ) and notice that they concern the set of conditional measures in the generic period t which, as remarked previously, are here independent of the conditioning σ-algebra F t. It turns out that also the set of martingale measures on all of Ω is a bounded convex set. However, determining its vertices is a more complex issue and we limit ourselves to illustrate it for a time horizon of = 2. With S 0 fixed and recalling that ω t corresponds to ξ t (t =, 2), we have where, for the trinomial case, Ω = {ω with ω = (ω, ω 2 ) ω, ω 2 {u, m, d and we put ω t = u, ω 2 t = m, ω 3 t = d, (t =, 2). For a generic martingale measure Q on Ω we now have Q(ω i, ω j 2) = Q (ω i ) Q 2 (ω j 2 ω i ), i, j {, 2, 3 with Q the marginal measure in the st period and Q 2 the conditional measure for the 2nd period, given the st. Due to the previous remarks concerning the independence of (q (t), q 2 (t), q 3 (t)) from t and F t, this conditional measure Q 2 reduces here to a marginal measure as well, namely Q 2 (ω2 j ω) i = Q 2 (ω2). j For the sake of specificity assume now the case m < + r and let Q,0 be the extremal measure, among the measures Q, that corresponds to ( ) ( + r) m (q, 0 q2, 0 q3) 0 u ( + r) =, u m u m, 0 and Q, the one corresponding to ( ( + r) d (q, q2, q3) =, 0, u d ) u ( + r). u d Analogously for Q 2,0 and Q 2,. We know now that Q and Q 2 are convex combinations γ t Q t,0 + ( γ t )Q t, of Q t,0 and Q t, for t =, 2 respectively. In general, the coefficients γ t in these convex combinations may depend not only on time, but in a predictive way also on ω itself (i.e. γ t may depend on (ω,, ω t )) thereby inducing a dependence structure over 9

time. Since in our case we have seen in the remarks above that (q (t), q 2 (t), q 3 (t)) are independent of t and F t, we shall let these coefficients depend only on time. We shall thus write Q t (ω i t) = γ t Q t,0 (ω i t) + ( γ t ) Q t, (ω i t) for suitable γ t (0, ). One thus obtains (recall that ω = (ω i, ω j 2)) Q(ω) = γ γ 2 Q,0 (ω)q i 2,0 (ω2) j + ( γ )γ 2 Q, (ω)q i 2,0 (ω2) j + γ ( γ 2 ) Q,0 (ω)q i 2, (ω2) j + ( γ )( γ 2 ) Q, (ω)q i 2, (ω2) j ending up with 4 extremal measures where, e.g. Q = Q,0 Q 2,0, Q2 = Q, Q 2,0, Q3 = Q,0 Q 2,, Q4 = Q, Q 2, Q (u, u) = ( (+r) m u m ) 2, Q (u, d) = 0, Q (u, m) = ((+r) m)(u (+r)) (u m) 2 Q 4 (d, u) = ((+r) d)(u (+r)) (u d) 2, Q4 (d, m) = 0, Q4 (d, d) = Finally, for the Radon-Nikodym derivatives we obtain with P (ω i, ω j 2) = p i p j. L j (ω) = Q j (ω) P (ω), j =,, 4 ( ) 2 u (+r) u d 4 Solution methods (general description) In this section we recall the basic aspects of the Dynamic Programming approach and of the so-called martingale method. 4. Dynamic programming 4.. General context Let V t, t = 0,,,, be a given process (as e.g. the portfolio value process), of which the evolution depends on the choice of a control sequence (equivalent to a strategy) π t, t = 0,,, adapted to a given observed filtration. A typical example for such a situation is V t+ = G t (V t, π t, ξ t+ ), ξ t i.i.d. and notice that, if π t = π t (V t ), (Markov controls), then V t is a Markov process. he control sequence π t is chosen so as to maximize an (additive over time) criterion, i.e. { max E u(v t, π t ) (3) π 0,,π t=0 with u( ) a utility function (in traditional stochastic control applications the last term for t =, i.e. the terminal utility, does not depend on the control π). 0

4..2 he Dynamic programming principle he method of Dynamic programming (DP) to obtain the control maximizing (3) is based on the so-called Dynamic Programming Principle, which states that : if a process is optimal over an entire sequence of periods, then it has to be optimal over each single period. In the general context described above, the DP principle allows to determine the optimal sequence π 0,, π by a sequence of minimizations over the individual controls π t (scalar minimizations). In fact, due to the Markovianity of π t, V t and the additivity over time of the criterion, max E π 0,,π +E + E { u(v t, π t ) t=0 { { + E max [u(v, π ) π { max π u(v, π ) V, π [ { = max u(v 0, π 0 ) + E max [u(v, π ) π 0 π ] 4..3 Implementation of the DP principle V 2, π 2 V, π ] V 0, π 0 ] he way the DP principle is used to determine the control maximizing (3) is as follows. Let { U t (v) := max E u(v s, π s ) V t = v (4) π t,,π then the DP principle leads to the following DP-algorithm U (v) = max π u(v, π ) and, for t <, s=t U t (v) = max [u(v, π t ) + E {U t+ (G t (V t, π t, ξ t+ )) V t = v] π t which gives the optimal value and optimal control (obtained by backwards induction with scalar maximization). Notice that one automatically has πt max as a function of only V t. 4..4 Specific context (expected utility from terminal wealth) Here we mention how the general dynamic programming algorithm, described above, can be applied to the specific case of maximization of expected utility from terminal wealth. We consider as V t the portfolio value. As control/strategy we take π t = (πt 0, πt,, πt K ) with π 0 t = α0 t+b t V t, π i t = αi t+s i t V t (i =,, K) (5)

i.e. the fractions of wealth invested in the individual assets so that π 0 t = πt i (πt i (0, )) i= Recall also that, by the self-financing property, V t = α 0 t B t + αts i t i = αt+b 0 t + i= i= α i t+s i t and that we had supposed α to be predictable with S i t+ = S i t ξ i t+ (ξ i t i.i.d. sequences) By the self-financing property and the previous definitions we obtain V t+ = V t + α 0 t+ B t + = V t + α 0 t+b t r t + = V t + V t [π 0 t r t + i= i= πt i i= α i t+ S i t α i t+s i t ( ξ i t+ ) ( ξ i t+ )] (6) i.e. we have with V t+ = G t (V t, π t, ξ t+ ) G t (V t, π t, ξ t+ ) = V t [π 0 t ( + r t ) + and where now π t = (π t,, π K t ). In the present case we have only a terminal utility, i.e. ] πtξ i t+ i i= (7) u(v t, π t ) = 0 for t <, and u(v, π ) = u(v ) which implies (there is no π over which to maximize) U t (v) := max E {u(v ) V t = v π t,,π and the DP algorithm becomes U (v) = u(v) and, for t <, U t (v) = max π t E {U t+ (G t (V t, π t, ξ t+ )) V t = v (8) 2

4.2 he martingale method 4.2. Introductory remarks o introduce this method, recall from the beginning of section 3 that, in the case when there is no consumption, the discounted value process of any self financing strategy is a martingale under any martingale measure (we implicitly assume the required integrability conditions on the strategy). he martingale method can be used for the solution of general stochastic optimization problems, but it draws its origin from the financial problem of hedging a contingent claim. Although, as already mentioned in the beginning of section 3, the hedging problem as such is not in the scope of this survey, we recall here just what is useful for the understanding of the martingale method. Given is a horizon and a claim H, namely a random variable, which is F measurable (recall from subsection 2. that we may assume F t = σ{s u, u t). he hedging problem consists in finding an initial wealth V 0 = v and a self financing strategy α t such that V = H, P and therefore also Q a.s., which is equivalent to requiring Ṽ = H, P (Q) a.s. (Recall that the quantities with a tilde represent values that are discounted with respect to B t ). Since, as recalled above, the discounted value process of a self financing strategy is a martingale under any martingale measure, this will only be possible if the initial wealth v is such that v = E Q { H for any martingale measure Q. he problem of finding the self financing hedging strategy α can then be seen as equivalent to a martingale representation problem in the following sense. Fix for the moment a MM Q and define the martingale M t := E Q { H F t. If we now determine the self financing strategy α t = (αt 0,, αt K ) in such a way that, for Ṽt from (6) with V 0 = v and G t according to (5), we have Ṽt = M t then, starting from the initial wealth V 0 = v and following this strategy α t, we have Ṽ = H and therefore also V = H. Finding the strategy α t is therefore equivalent to finding the representation of the martingale Ṽt = M t in the form of (6) with G t as in (5) and where the unknown is the strategy α. Notice that, since for the same v we have v = E Q { H for any MM Q, the strategy α t will also be the same under any MM Q (to this effect see also subsection 5.2 below). he martingale method itself consists now of three steps : Determine the set of reachable values for the wealth V in period ; determine the optimal reachable wealth V ; determine a self financing strategy α such that V α = V α where in V make explicit the dependence of the terminal wealth on the strategy α. we Notice that the last step corresponds indeed to a hedging problem, where the claim to be hedged is the optimal reachable wealth. Notice also that this method decomposes the originally dynamic problem of portfolio optimization problem (or any other dynamic optimization problem) into a static problem (determination of the optimal reachable wealth) and a hedging problem (that corresponds to a martingale representation problem ). 3

We proceed now with a description of the first two steps; the third step will be illustrated below when solving specific portfolio optimization problems. More specifically, we shall illustrate the first two steps for the problem of maximization of expected utility from terminal wealth; for the other two investment criteria these steps will be discussed in the next section 5. 4.2.2 First step : set of reachable portfolio values On a more formal basis the problem consists in determining the set V v = {V : V = V α for some self-financing and predictable α with V 0 = v of reachable values for the wealth in period. If the market is complete with unique equivalent martingale measure (EMM) Q, then V v = {V : E Q {B V = v If the market is not complete and the set of all martingale measures forms a bounded convex set (convex polyhedron) with a finite number of extremal values (vertices) Q j (j =,, J) as was the case for the trinomial market model that was described in section 3.2 as an example of an incomplete market model in discrete time, then V v = {V : E Qj {B V = v for j =,, J. 4.2.3 Second step : optimal reachable portfolio value In formal terms the problem is : determine V such that E{u(V ) E{u(V ), V V v o solve this 2nd step problem we shall mention here essentially the method based on the Lagrange multiplier technique. We shall briefly recall also a technique based on convex duality to show that a certain V is indeed optimal. Lagrange multiplier technique a) Case of a complete market - unique martingale measure Q Let L := dq and denote by λ the Lagrange multiplier. Recalling that the dp requirement V 0 = v can be translated into E Q {B V = v the problem in this second step becomes max V [E{u(V ) λ E Q {B V ] = max V E{u(V ) λ B LV (9) By the properties of u( ) as utility function we have that the inverse of the derivative exists and we shall denote it by I( ) = (u ( )). On the other hand, maximizing the expectation on the right hand side of (9) is equivalent to 4

maximizing its argument for each possible scenario/state of nature ω Ω. A necessary condition for this is that u (V ) = λb L implying V = I(λB L) o this we have to add that the Lagrange multiplier λ has to satisfy the following budget equation E Q [B I(λB L)] = v v = E{LB I(λB L) := V (λ) his implies that, whenever V ( ) is invertible, λ = V (v) and, consequently, we obtain V = I(V (v)b L) o illustrate the feasibility of this procedure we mention the case of a log-utility function, i.e. u(v ) = log V, for which the inverse of the derivative is I(y) =. y he Budget Equation then becomes { v = E{LB I(λB L) = E LB B = λl λ = V (λ) implying that λ = v b) Case of an incomplete market and so the optimal wealth becomes V = I(λB L) = vl B (20) We consider here the case when, as in the trinomial model, the set of all martingale measures is bounded and has a finite number J of vertices/extremal points Q j. Let L j := dqj, j =,, J and notice that dp E Q {B V = v Q EQj {B V = v j =,, J Having now J constraints we shall consider a vector λ = (λ,, λ J ) of Lagrange multipliers and the problem of the second step becomes then { J max E u(v ) λ j B V Lj V As in the case of the complete market, we impose the necessary condition for V to maximize, for each scenario, the argument of the expectation namely ( J J ) u (V ) = λ j B Lj implying V = I λ j B Lj j= his time the Lagrange multiplier vector λ has to satisfy the following system of budget equations { E j= j= ( J ) L j B I λ j B Lj = v ; j =,, J j= 5

Convex duality - only case of a complete market We shall now additionally use also the convex dual to verify that the V determined above with the Lagrange multiplier technique is indeed optimal. For this purpose let us first recall the Legendre-Fenchel transform of a function U(x) that leads to the so-called convex dual Ũ(y) of U(x), which is indeed a convex function and is given by It follows that Ũ(y) := max {U(x) xy = U(I(y)) yi(y) x U(I(y)) yi(y) U(x) xy, x (2) Recall now from the Lagrange multiplier approach that V = I(λB L) with λ such that E { LB V = E{LB I(λB {{ L) = v V (λ) Put now x = V, y = λb L for some positive λ. hen, by (2) and (22), U(V ) λb (22) LV U(V ) λb LV (23) aking expectations in (23) and using (22), one has that, for all V satisfying the budget equation i.e. such that E{LB V = v, namely E{U(V ) λv E{U(V ) λv E{U(V ) E{U(V ) and this V satisfying the budget equation. 5 he case of logarithmic utility, explicit computations In this section we shall describe more explicitly the individual steps required both for the method of Dynamic programming and the martingale method in the case of a log-utility function and for each of the three investment criteria. In particular we shall show how to perform the third step in the martingale approach, namely determining the optimal investment strategy. o simplify the presentation we shall assume without loss of generality that the price of the non-risky asset, which is used to discount the various other prices, is B t ; since we implicitly take B 0 =, this is equivalent to assuming that the short rate of interest is r t 0. While the possibility of explicitly computing the various steps in the martingale method depends rather crucially on the choice of the utility function, the calculations by the method of 6

Dynamic programming (DP) are less dependent on the choice of the utility function. Still, in this section we shall make a bit more explicit also the steps required by DP as this is not immediately obvious for the second and third investment criteria. Numerical calculations for specific examples are then reported in the next section 6 for the maximization of utility from terminal wealth. 5. Maximizing expected utility from terminal wealth (binomial market model) After briefly recalling the particular form that the Dynamic programming approach takes in this case (as mentioned above, for this investment criterion this is almost immediate), we shall describe the particular form that the martingale approach takes in this case and illustrate the computation of the optimal strategy. 5.. Dynamic programming Recall from (8) the Dynamic programming algorithm, namely U (v) = u(v) and, for t <, U t (v) = max π t E {U t+ (G t (V t, π t, ξ t+ )) V t = v (24) where the dynamics G t (V t, π t, ξ t+ ) were specified in (7), namely ] G t (V t, π t, ξ t+ ) = V t [πt 0 ( + r t ) + πtξ i t+ i he only particular aspect here is that K = and, letting π t = πt = α t+ St V t, one has πt 0 = ( π t ). With prohibition of short selling, i.e. requiring αt > 0, we obtain π t (0, ). Since r t = 0, we may then write 5..2 Martingale method i= G t (V t, π t, ξ t+ ) = V t [ + π t (ξ t+ )] Recalling from section 3. the process N t, we have that N b(, p) and it represents the r.v. that counts the total number of up-movements of the price process S t. With u(x) = log x one has by (20) and () and recalling that we take B t, ( ) N ( ) N p p V = v (25) q q and the optimal value of expected utility from terminal wealth is then ( ) ( ) E{u(V q q ) = log v log E(N p ) log ( E{N p ) ( ) ( ) (26) = log v p log ( p) log q p 7 q p

Computation of the optimal strategy - Backwards recursion In line with section 2.2 an investment/portfolio strategy is here given by the two-dimensional vector α t = (α 0 t, α t ) where α 0 t and α t denote the number of units of the non-risky and the risky assets respectively held in the portfolio in period t. Recall also that the process α t is supposed to be predictable, i.e. α t is taken to be F t measurable. We now describe the backwards recursion, from period to 0, to determine the optimal investment strategy (α t ) t=0,,. ake period ; we have to decide for α 0, α. Let N = n < (and therefore S = S 0 u n d n ). Recalling that we had assumed r = 0, we have to impose that for all states of nature, i.e. independently on whether prices go up or down, α S + α 0 = V = V (27) his implies that the following system of equations in α, α0 has to be satisfied, namely ( ) n+ ( ) n α S u + α 0 = v p p (prices go up) q q ( ) n ( ) n α S d + α 0 = v p p (prices go down) q q from which we obtain and, consequently, α S (u d) = v( + r) ( p q α = v ( p q ) n ( p q ) n (p q) S (u d)q( q) ) n ( ) n [ p p q q p ] q α 0 = v ( p q ) n ( p q ) n [u( p)q d( q)p] (u d)q( q) Recalling the self-financing condition, the fact that q = +r d, and putting C(p, q) := u d (p q) + (uq dp) + (dpq upq), for the optimal value induced in period we thus obtain Notice now that V V = α S + α 0 = α S + α 0 = v ( p q ) n ( p q ) n C(p,q) (u d)q( q) ( = v has the same structure as V p q ) n ( ) n p q so that the calculations for the following period 2 proceed in exactly the same way as for and so forth, which allows to straightforwardly complete the backwards recursion. In fact, in the generic period t, with N t = n t, the condition (27) becomes α t S t + α 0 t = V t (28) 8

and one obtains α t = v ( p q ) n ( p q ) t n (p q) S 0 u n d t n (u d)q( q) with the optimal wealth α 0 t = v ( p q ) n ( p q ) t n [u( p)q d( q)p] (u d)q( q) V t = v ( ) n ( ) t n p p q q Notice also that the ratio of the wealth invested in the risky asset is π t = α t+s t V t = (p q) (u d)q( q) and it is independent of t and S t. It follows that the optimal strategy requires investing in the risky asset, in each period and in every state, the same fraction of wealth. Notice finally that the fact that the ratio πt does not depend neither on the period nor on the state of the underlying risky asset does not imply that the portfolio itself remains constant; in fact, since the price of the underlying is changing, to keep the ratio constant, one still has to continuously change the number of units invested in the underlying risky asset. 5.2 Maximizing expected utility from terminal wealth (trinomial market model) Here we proceed analogously to the previous subsection for the binomial model noticing that nothing changes concerning the Dynamic programming approach. We shall thus consider here only the martingale method; in fact, while the Dynamic programming approach is independent on whether the market is complete or not (unique or more martingale measures), this is not the case with the martingale method. 5.2. Martingale method Consider the situation described in section 3.2, in particular the subsubsection concerning martingale measures in the trinomial market model, where we had taken = 2 and for which one has four extremal MM s, namely Q, Q2, Q3, Q4. Recall also from section 4.2, in particular point b) in the subsubsection concerning the optimal reachable portfolio value that, for the log-utility function and B t, the optimal terminal wealth is V = λ L + λ 2 L 2 + λ 3 L 3 + λ 4 L 4 (29) with λ,, λ 4 determined according to the system of equations (j =,, 4) { { L j Q j v = E = E (30) λ L + + λ 4 L 4 λ Q + + λ 4 Q4 9

For the explicit computation of the values of the λ is we refer to the numerical calculations in the next section 6.2. here remains to compute the optimal strategy. For this purpose notice that the market is incomplete and so not all future amounts can be replicated by a selffinancing strategy. However, given the constraint E Qj {B V = v for j =,, J, which implies that E Q {B V = v for all MM Q, one has that V is indeed replicable with a self-financing portfolio (consisting of the risky and non-risky assets) starting from an initial wealth of V 0 = v. o determine e.g. α2, α2 0 (in the case of = 2) it thus suffices to impose that α 2S 2 + α 0 2 = V holds in only two of the three states of nature; the condition is then automatically satisfied in the third one. At this point we are back to the same situation as for the binomial model and the calculations that lead to the optimal strategy are thus the same as they were there (an example showing the calculations is given in the next section 6.2). We close this subsection 5.2 by mentioning the following alternative way to compute the optimal investment strategy in the trinomial model and, in general, in an incomplete market model. It consists in completing the market by adding fictitious assets and imposing on the optimal strategy that the investment in these assets is null. 5.3 Maximizing expected utility from consumption As mentioned in the previous subsection, the method of Dynamic programming does not depend on whether the market is complete or not, only the martingale method does. For the present case of expected utility from consumption we therefore do not anymore consider two separate subsection, one for the complete binomial market model and one for the incomplete trinomial model as we did for expected utility from terminal wealth and where it was motivated by the desire to better illustrate the possible differences. Here we simply mention the differences that arise when we discuss the martingale method itself. 5.3. Dynamic programming In the present context, considering again K underlying risky assets with prices S i t+ = S i t ξ i t+ (i =,, K) 20

we have, by the self-financing condition for the case with consumption as described in section 2.2 and in complete analogy with (6) and (7), V t+ = V t + α 0 t+ B t + = V t + V t [π 0 t r t + αt+ S i t i C t i= πt i i= ( ξ i t+ )] C t i.e. we have with = V t [( + r t ) + i= G t (V t, π t, ξ t+ ) = V t [( + r t ) + and with the control sequence/strategy π i t ( ξ i t+ ( + r t ) )] C t ( + r t ) V t+ = G t (V t, π t, ξ t+ ) i= π i t π t = (π t,, π K t, C t ) ( ξ i t+ ( + r t ) )] C t ( + r t ) where, as before, π i t = αi t+ Si t V t. With respect to the general description of the method of Dynamic programming in section 4. in this case we have u(v t, π t ) = u(c t ), t = 0,, with the constraint C V. o implement the Dynamic programming principle, corresponding to (4) here we put { U t (v) := max E C t,,c (C V ) s=t β s t u(c s ) V t = v where the condition C V can be accounted for by setting u(c ) = for C > V he DP algorithm (see (5) now becomes { u(v) if C = v U (v) = if C > v U t (v) = max [u(c t ) + β E {U t+ (G t (V t, π t, ξ t+ )) V t = v] π t where nonnegativity of C is guaranteed by u(c) = for C < 0. 2

5.3.2 Martingale method Recall that for the maximization of utility from terminal wealth we required : determining the set of reachable/attainable values of wealth in ; determining the optimal such wealth; determining a self-financing strategy that replicates the optimal wealth. Here the terminal wealth is replaced by the adapted consumption process C t and so we change the above requirements into the following steps : determine the set of attainable consumption processes; determine the optimal attainable consumption process; determine an investment strategy allowing to consume according to the optimal consumption process. o implement these steps we start from the following lemma. Lemma. Given an initial wealth v 0, a consumption process C t, and a selffinancing strategy α, one has where G t is the discounted gains process V t = v + B G t C s t, t =,, (3) t B s=0 s G t = t s=0 αs+ i S s i i= Proof : By the definition (2) of the portfolio value and the self financing property (7) when there is consumption one has, respectively, α 0 t = V t B t K i= αi St i t B t and α 0 t = V t B t C t B t K i= αi St i t B t so that, since V 0 = v, B 0 =, relation (3) follows immediately for t = and, for t >, by induction. First step : We start from the following Definition 6. A consumption process C t is said to be attainable if α with (C, α) admissible (see Definition 5) s.t. C = V. (α replicates or generates C) 22

For any MM Q, we now have that the process v + G t is a Q martingale with value v at t = 0. Putting V = C, from (3) one immediately has: Proposition 2. Given v, a consumption process C is attainable if a) (complete market; unique MM Q) v = E Q { C0 B 0 +, + C B b) (incomplete market; extremal MM s Q i for j =,, J) { C0 v = E Qj +, + C ; j =,, J B 0 B Second step : We shall discuss this step directly for the more general case of an incomplete market with a finite number J of extremal martingale measures. In the present case of maximizing expected utility from consumption, this second step consists in obtaining the optimal attainable consumption process by solving { max C E t=0 β t u(c t ) subject to E Qj { t=0 C t B t = v ; j =,, J Notice that the only decision variable here is C t (t = 0,, ); α t does not appear. Notice also that nonnegativity of C t is guaranteed by u(c) = ( for C ) < 0. In order to solve problem (32) define N j t := Bt E{L j F t L j = dqj so that dp E Qj { t=0 = E { { C t B t = E L j C t t=0 B t t=0 E { Bt C t L j F t = E { t=0 C tn j t (32) then (32) becomes max C E subject to E { β t u(c t ) t=0 { t=0 C tn j t = v ; j =,, J (33) 23

Using, as before, the Lagrange multiplier technique to solve (33), we have to compute { J max E β t u(c t ) λ j C t N j t C t=0 A necessary condition for C t to maximize, for each scenario, the argument of the expectation is J β t u (C t ) = λ j N j t ; t = 0,, which implies C t = I j= ( J j= λ jn j t β t ) j= t=0 ; t = 0,, with λ j satisfying the system of budget equations { ( J E N j j= t I λ ) jn j t = v ; j =,, J t=0 and the optimal value becomes { ( ( J J(v) = E β t j= u I λ )) jn j t t=0 β t β t hird step : In complete analogy with the previous case of expected utility from terminal wealth, it consists here in determining the investment strategy of a self-financing portfolio having terminal value V = C and the procedure itself can be carried over directly from there. 5.4 Maximizing expected utility from consumption and terminal wealth In the previous case of expected utility only from consumption, all of V was consumed at the terminal time t =. Here we generalize this problem by leaving V C for future investment and maximizing expected utility from consumption of C t for t = 0,,, as well as from the terminal wealth V C. Letting u c ( ) and u p ( ) denote the utility functions for consumption and terminal wealth respectively, in accordance with section 2.3 our problem here is the following Problem : (max utility from consumption and terminal wealth) { J(v) = max E β t u c (C t ) + β u p (V C ) (C,α) A v t=0 24

α : self-financing and predictable with A v : C : nonnegative, adapted, C V We can now easily adapt all previous solution methods. 5.4. Dynamic programming As in the previous pure consumption-investment problem here too we have with V t+ = G t (V t, π t, ξ t+ ) G t (V t, π t, ξ t+ ) = V t [( + r t ) + πt i i= ( ξ i t+ ( + r t ) )] C t ( + r t ) and with the control sequence/strategy π t = (πt,, πt K, C t ) where, again as before, πt i = αi t+ Si t V t. In this case, however, in addition to a running utility up to the terminal time, i.e. u(v t, π t ) = u c (C t ), t = 0,, we also have a terminal utility he DP algorithm then becomes u(v, π ) = u p (V C ) U (v) = max 0 C v [u c(c) + u p (v C)] U t (v) = max [u c (C t ) + β E {U t+ (G t (V t, π t, ξ t+ )) V t = v] π t and notice that the only difference with the simple consumption-investment problem is the form of U (v) (in the previous case it is essentially given by U (v) = u(v) = u c (v)). 5.4.2 Martingale method he only difference with the previous case is that now we shall not require C = V (see Definition 6 of attainability). First step : 25

Instead of the notion of attainability for the previous pure consumption-investment problem given in Definition 6, in the present case we shall require for attainability that { C0 v = E Qj + + C + V, j =,, J B 0 B B (again we consider directly the more general case of an incomplete market with the finite number J of extremal martingale measures). Second step : Corresponding to (33) we now have { max E β t u c (C t ) + β u p (V C ) C,V t=0 { subject to E t=0 C tn j t + V N j = v ; j =,, J (34) with C adapted and V F Notice that the fact that u c (C) = for C < 0, and the same for u p ( ), implies that the solution of (34) automatically satisfies C t > 0 and C < V. Using, as before, the Lagrange multiplier technique to solve (34), we have to compute { [ J ] max E β t u c (C t ) + β u p (V C ) λ j C t N j t + V N j t=0 A necessary condition for C t and V to maximize, for each scenario, the argument of the expectation is (differentiate w.r.to each C t and V ) β t u c(c t ) = J j= λ jn j t, t = 0,, β u c(c ) = β u p(v C ) j= β u p(v C ) = J j= λ jn j Denoting by I c ( ) and I p ( ) the inverses of u c( ) and u p( ) respectively, one has ( J ) j= C t = I c λ jn j t ; t = 0,, β t t=0 ( J ) ( j= V = I λ jn j J ) j= p + I λ jn j β c β with λ j satisfying the system of budget equations { ( J E N j j= t I λ ) ( jn j J t c + N j β t I j= λ ) jn j p = v ; j =,, J β t=0 26

and the optimal value becomes { ( ( J J(v) = E β t j= u c I λ )) ( ( jn j J t c + β j= u β t p I λ )) jn j p β t=0 hird step : In complete analogy to the previous cases, it consists in determining the investment strategy of a self-financing portfolio having terminal value V. 6 Numerical examples (logarithmic utility) o illustrate the computations described in the previous section for the case of maximization of expected log-utility from terminal wealth (subsections 5. and 5.2), we present here numerical examples for the binomial and trinomial market models (complete and incomplete markets) using both the method of dynamic programming as well as the martingale method. 6. Binomial market model We consider the following model : let the horizon be = 2 and S 0 =. Furthermore, let r = 0 (equivalent to taking B t ), u = 2, d = which implies q = and let 2 3 p = 4. 9 max E{log V 2 Objective E Q {V 2 = v, Q : MM i.e. maximize the expected log-utility of the terminal value of a self financing portfolio, starting from an initial capital of V 0 = v. 6.. Dynamic programming Recalling from section 5. the Dynamic programming algorithm (24) and prohibiting here too short selling (which amounts to π t (0, )), for the present example we obtain 27

i) U 2 (v) = log v [ 4 ii) U (v) = log v + max π (0,) 9 log( + π ) + 5 ( 9 log π ) ] 2 = log v + max π (0,) G(π ) = log v + G(π ) iii) U 0 (v) = log v + G(π ) + max π 0 (0,) G(π 0) = log v + G(π ) + G(π 0) thereby obtaining the maximizing values π = π0 =, i.e. (see also end of section 3 5.) the optimal strategy requires investing in the risky asset in each period and in every state simply the same fraction of the wealth. Furthermore, 3 G(π ) = 4 9 log 4 3 + 5 9 log 5 6 = 3 log 2 log 3 + 5 9 log 5 and the optimal value of expected utility is log v + G(π 0) + G(π ) = log v + 2 3 6..2 Martingale method Approach according to section 5.. From (25) we have that the optimal terminal wealth is log 2 2 log 3 + 0 9 log 5 V = v ( ) N2 ( ) 2 N2 4 5 3 6 6 9 v if N 2 = 2 0 namely v if N 9 2 = 25 v if N 36 2 = 0 and, according to (26), the optimal value of expected terminal utility is ( E{u(V ) = log v 2p log q p = log v 8 log ( ) 3 9 4 0 log ( ) 6 9 5 ) ( 2( p) log q p ) (35) = log v 8 9 = log v 2 log 3 + 2 3 0 [log 3 2 log 2] [log 2 + log 3 log 5] 9 log 2 + 0 9 log 5 28

which coincides with that obtained by the DP approach. Alternative elementary approach It may be instructive to consider also the following direct elementary approach (taken from [3]). Let a, b, c, d be the four possible values of V 2. he problem then becomes (recall that the objective function is an expectation under the real-world probability measure, while the constraint is an expectation under the martingale measure) max { 6 20 20 25 log a + log b + log c + log d 8 8 8 8 a + 2b + 2c + 4d = v 9 9 9 9 a, b, c, d 0 Using the technique of Lagrange multipliers to deal with the constraint, we have to maximize ( ) ( 6 25 log a + + 8 8 log c λ 9 a +, 4 ) 9 d and the necessary conditions lead to 6 λ = 0 8 a 9 a = 6 9 λ 20 2λ = 0 b = 0 8 b 9 20 2λ = 0 c = 0λ 8 c 9 9 25 8 9 λ d 4λ 9 = 0 d = 25 he budget equation reads as ( 6 8 + 20 8 + 20 8 + 25 ) λ = v 8 and implies λ = v he optimal expected value of terminal wealth is then 36 λ E{log V2 = log v + ( 6 log 6 + 2 20 0 log + 25 8 9 8 9 8 = log v + 6 8 ) 25 log 36 40 [4 log 2 2 log 3] + [log 2 + log 5 2 log 3] 8 [2 log 5 2 log 2 2 log 3] + 25 8 = log v + 54 62 90 log 2 log 3 + log 5 8 8 8 = log v + 2 3 log 2 2 log 3 + 0 9 log 5 29

Optimal investment strategy From section 5. we already know that the optimal investment strategy requires investing in the risky asset in each period and for every value of the underlying the constant fraction of wealth π t = α t+s t V t = (p q) (u d)q( q) i.e. πt = in line with the results of the Dynamic programming approach. 3 Optimal strategy : independent direct approach In view of the approach that we shall use in the next subsection concerning the trinomial model, we present here an independent direct approach to determine the optimal investment strategy. his will also allow for better insight and for possible comparison with the method of Dynamic programming. Let (α t, α 0 t ) be as before and denote by (φ t, ψ t ) the amounts invested in period t in the risky and non-risky assets respectively, i.e. φ t = α t S t, ψ t = α 0 t. Notice that, while the optimal investment ratio does not depend on the value of the underlying, the number of units and the amounts that are invested do and so below we make explicit this dependence. We now perform a backwards recursion for each period and for each possible value of the underlying risky asset. Recalling that we had supposed S 0 = and u = 2, we start from i) t =, S = 2 aking into account the requirement imposed on the strategy in the next-tolast period and expressed in (27), as well as the optimal portfolio values in the last period as expressed in (35), we obtain the system of equations (in (α t, α 0 t ) and (φ t, ψ t ) respectively) having as solutions 4α 2(2) + α 0 2(2) = 6 9 v α 2(2) + α 0 2(2) = 0 9 v α 2(2) = 2 9 v α 0 2(2) = 8 9 v 2φ 2 (2) + ψ 2 (2) = 6 9 v φ 2 2(2) + ψ 2 (2) = 0v 9 φ 2 (2) = 4 9 v ψ 2 (2) = 8 9 v and implying that the optimal portfolio value in period t = and with a price of the underlying S = 2 is V (2) = 2α 2(2) + α 0 2(2) = φ 2 (2) + ψ 2 (2) = 2 9 v Denoting again by π t the fraction invested in the risky asset in period t, one has for its optimal value in t = 2 π (2) = α 2(2)S V (2) 30 = φ 2(2) V (2) = 3

independently of the choice of t and S and this coincides with the value that we had already determined previously. Next, recalling that we had assumed d =, we perform the computations for 2 ii) t =, S = 2 Proceeding analogously to the previous point i), we obtain the systems α2( ) + 2 α0 2( ) = 0v 2 9 2φ 2 ( ) + ψ 2 2( ) = 0v 2 9 4 α 2( ) + 2 α0 2( ) = 25v 2 36 φ 2 2( ) + ψ 2 2( ) = 25v 2 36 Notice now that the system in (φ, ψ) has the same left hand side as before in i), while the right hand side is multiplied by 5. It follows that 8 (φ 2 ( 2 ), ψ 2( 2 )) = 5 ( 5 8 (φ 2(2), ψ 2 (2)) = 8 v, 5 ) 9 v implying that the optimal portfolio value in period t = and with a price of the underlying S = is 2 V ( 2 ) = φ 2( 2 ) + ψ 2( 2 ) = 5 8 v Again, if we compute the fraction invested in the risky asset we obtain as before for its optimal value Finally, we come to π( 2 ) = 5 8 v 8 5 v = 3 iii) t = 0, S 0 = aking into account the requirement imposed on the strategy in the generic period t and expressed in (28), as well as the optimal portfolio values computed for t = in the previous two cases i) and ii), we obtain the system of equations 2α () + α 0 () = 2 9 v 2 α () + α 0 () = 5 8 v 2φ () + ψ () = 2 9 v φ 2 () + ψ () = 5v 8 and notice, again, that the system in (φ, ψ) has the left hand side the same as before in ii), while the right hand side is multiplied by 6. It follows that 5 (φ (), ψ ()) = 6 5 (φ 2( 2 ), ψ 2( ( 2 )) = 3 v, 2 ) 3 v and, as expected, the optimal portfolio value in period t = 0 results in V 0 = v while the optimal fraction invested in the risky asset is π 0() = 3 o conclude this direct approach notice that only one system of linear equations had actually to be solved. 3

6.2 rinomial market model We consider the following model with data analogous to the binomial model : let the horizon be = 2 and S 0 =. Furthermore, r = 0, u = 2, m =, d = 2, p = p 2 = p 3 = 3 and we have Ω = {ω,, ω 9 with ω = (u, u), ω 2 = (u, m), ω 3 = (u, d) ω 4 = (m, u),, ω 6 = (m, d) ω 7 = (d, u),, ω 9 = (d, d) and P {ω i =, i =,, 9. 9 max E{log V 2 Objective E Qj {V 2 = v, Q j : extremal MM namely it consists, again, in maximizing the expected log-utility of the terminal value of a self financing portfolio strategy starting from an initial capital of V 0 = v. 6.2. Dynamic programming he procedure is completely analogous to that for the Binomial model, except that here we have three possible states of nature. More precisely, we have i) U 2 (v) = log v [ ii) U (v) = log v + max π 3 log( + π ) + 3 log + ( 3 log π ) ] 2 [ ( = log v + max log( + π 3 ) + log π )] π 2 = log v + 3 max π Γ(π ) = log v + 3 Γ(π ) iii) U 0 (v) = log v + 3 Γ(π ) [ + max π 0 3 log( + π 0) + 3 log + ( 3 log π ) ] 0 2 = log v + 3 Γ(π ) + 3 Γ(π 0) thereby obtaining the maximizing values π = π0 = so that, again, the optimal 2 strategy requires investing in the risky asset in each period and in every state the same fraction of the wealth, this time equal to. Furthermore, 2 (36) Γ(π ) = log 3 2 + log 3 4 32