Optimal production management when demand depends on the business cycle

Transcription

1 OPERATIONS RESEARCH Vol., No., Xxxxx, pp. issn 3-364X eissn INFORMS doi.287/xxxx.. c INFORMS Optimal production management when demand depends on the business cycle Abel Cadenillas Department of Mathematical and Statistical Sciences, University of Alberta, Edmonton, AB, T6G 2G, Canada, acadenil@math.ualberta.ca Peter Lakner Department of Information, Operations and Management Sciences, Stern School of Business, New York, NY 2, plakner@stern.nyu.edu Michael Pinedo Department of Information, Operations and Management Sciences, Stern School of Business, New York, NY 2, mpinedo@stern.nyu.edu We assume that consumer demand for an item follows a Brownian motion with a drift that is modulated by a continuous-time Markov chain that represents the regime of the economy. The economy may be in either one of two regimes. The economy remains in one regime for a random amount of time that is exponentially distributed with rate λ, and then moves to the other regime and remains there an exponentially distributed amount of time with rate λ 2. Management of the company would like to maintain the inventory level of the item as close as possible to a target inventory level and would also like to produce the items at a rate that is as close as possible to a target production rate. The company is penalized by the deviations from the target levels and the objective is to minimize the total discounted penalty costs over the long term. We consider two models. In the first model the management of the company observes the regime of the economy at all times, whereas in the second model the management does not observe the regime of the economy. We solve both problems by applying the technique of completing squares and obtain the optimal production policy as well as the minimal total expected discounted cost explicitly. Our analytical results show, among various other results, that in both models the optimal production policy depends on factors that are based on short term concerns as well as factors that are based on long term concerns. We analyze how the impact of these factors depend on the values of the parameters in the model. We furthermore compare the total expected discounted costs of the two models with one another and determine the value of knowing the current regime of the economy. Subject classifications: inventory/production: uncertainty, stochastic; probability: stochastic model applications. Area of review: Stochastic Models.. Introduction Assume that the demand for an item follows a Brownian motion with a drift that is modulated by a continuous-time Markov chain that alternates between two regimes. This continuous-time Markov chain represents the regime or the state) of the economy. One regime may represent a recessionary period with a low demand rate and the other may represent an expansionary period with a high demand rate. The objective of management is to maintain the inventory level as close as possible to a fixed target level; there is a running cost associated with the difference between the inventory level and its target. In addition to that, the manager also wants to maintain a production rate that is as close as possible to a fixed target rate. Here also, there is a running cost associated with the difference between the actual production rate and its target. In this paper we consider two basic models. In the first model, management knows at any time the actual regime of the economy, whereas in the second model we assume that management does not know in which state the economy is.

2 2 Operations Research ), pp., c INFORMS We determine for both models explicitly the optimal production policies by applying the completing squares technique. The explicit solutions allow us to make interesting comparative statics analysis. We can also determine the long term value of the information concerning the exact state of the economy. There is an extensive literature on the theory of optimal production control. In the continuoustime case, Bensoussan, Sethi, Vickson and Derzko 984), Sethi and Thompson 2), and Khmelnitsky, Presman and Sethi 2) consider the case when the demand is constant. Fleming, Sethi and Soner 987) allow the demand to be stochastic, and model it as a continuous-time Markov chain with a finite state space. While a continuous-time Markov chain model for the demand is more realistic than a deterministic model, it still is not ideal. A continuous-time Markov chain model assumes that the demand can take only a finite number of values, and that it remains constant for long periods of time. It does not appear to be suitable for modeling the demand for commodity items such as wine, oil, and electricity. This motivates us to study a production problem in which the demand process changes continuously over time, and is allowed to take a continuum of values. In the present model we do allow the demand to depend on macroeconomic conditions that remain in effect for extended periods of time. This paper is organized as follows. In Section 2 we introduce notation and describe the production model and management s objectives. In Section 3 we consider the case in which management knows the actual regime of the economy, and determine the optimal production policy. In Section 4 we consider the case in which management does not know the regime of the economy, and determine the optimal production policy then. In Section 5 we compare the results of the two cases and determine the value of having full information with regard to the regime of the economy. In Section 6 we provide some numerical results and in Section 7 we present our conclusions. 2. Preliminaries: Notation and Model Formulation Formally, we consider a continuous-time Markov chain with two regimes ɛ = {ɛ t,t }. We assume that for every t : ɛ t = ɛt) S = {,2}. We use the notation ɛ t except when it appears in a subscript in which case we write ɛt) in order to avoid double subscripts.) The Markov chain ɛ has a strongly irreducible generator Q = ϑ ij 2 2 where ϑ ii = λ i < and j S ϑ ij = for every i S. The amount of time the economy remains in regime 2) is therefore exponentially distributed with rate λ λ 2 ). While the economy is in regime 2), the cumulative consumer demand follows a Brownian motion w with a drift µ µ 2 ) and a variance σ 2 σ 2 2). We also assume that ɛ and w are independent. We use the following notation: X t := inventory level at time t, ) D t := cumulative demand up to time t, 2) p t := production rate at time t 3) ɛ t := regime of the economy at time t 4) The production rate p t is actually allowed to be negative. A negative production rate would correspond to a write-off or disposal of inventory for example, due to obsolescence or perishability). In practice, this would not happen very often. In Appendix A we elaborate on the mathematical conditions that ensure that it hardly ever happens in our model. In a specific application, ɛ t = could represent a regime of economic growth while ɛ t = 2 could represent a regime of economic recession. In another application, ɛ t = could represent a regime in which consumer demand is high while ɛ t = 2 could represent a regime in which consumer demand is low.

3 Operations Research ), pp., c INFORMS 3 We assume that the cumulative demand satisfies the dynamics dd t = µ ɛt) dt+ σ ɛt) dw t. 5) Here, at every time t, the demand rate µ ɛt) and the volatility σ ɛt) depend on the regime ɛ t. We allow the demand to take negative values, which represent returns of items. The case in which σ ɛt) = for every t has been studied by Fleming, Sethi and Soner 987). We assume that X and D are adapted stochastic processes. The inventory level X satisfies X t = x + p s ds D t = x + p s µ ɛs) )ds σ ɛs) dw s 6) Our problem can now be formulated as follows. Management wants to select a production rate p :,T Ω, ) that minimizes the functional J defined by { ) 2 ) 2 Jp) := E R t α ɛt) Xt I ɛt) + β ɛt) pt P ɛt) }dt. 7) Here, { R t = exp } r ɛu) du and r i, ), α i, ), β i, ), I i, ), P i, ) are constants for each i S = {, 2}. The integrand in 7) represents the running cost incurred by deviating from the inventory target level I i and from the production target rate P i. We allow disposal of inventory, which occurs when p takes negative values. We assume that ɛ has a known initial distribution: P {ɛ = } = q and P {ɛ = 2} = q 2, where q,, q 2,, and q +q 2 =. One may select the stationary distribution for the initial distribution. The initial value X is assumed to be a known constant. The set of admissible production processes will be defined separately for the full and the limited information cases in the following two sections. 3. The case in which the regime is known We say that a production process p is admissible if it is adapted and the corresponding inventory process satisfies E R t X 2 t <. 8) We denote the class of admissible production processes by A f f for full information). Note that 8) implies lim inf E R t X 2 t = 9) t and by Jensen s inequality E R t X t ) 2 E R 2 tx 2 t E R t X 2 t, 9) implies lim inf t E R tx t =. ) The minimal cost is V f = inf p A f Jp).

4 4 Operations Research ), pp., c INFORMS Let a and a 2 be the positive solutions of = r i a i + α i β i a 2 i + λ i a 3 i λ i a i. ) It can be shown easily that there exist two unique numbers a > and a 2 > satisfying ); we omit the details of this proof. Let f and f 2 be the solutions of ) ai + r i + λ i f i λ i f 3 i + 2a i µ i P i )+ 2α i I i =, 2) β i which is where f i = 2 γ { } a3 i a i µ i P i ) α i I i + r 3 i + λ 3 i λ i a 3 i µ 3 i P 3 i ) λ i α 3 i I 3 i, 3) β 3 i We also define g and g 2 by In addition, let h and h 2 be the solution of that is, where Theorem 3.. The optimal production rate p f ) ) a a2 γ = + r + λ + r 2 + λ 2 λ λ 2. 4) β β 2 g i = α i I 2 i f iµ i + a i σ 2 i 4β i f 2 i + f ip i, i =,2. 5) h 2 h )λ r h = g 6) h h 2 )λ 2 r 2 h 2 = g 2, 7) ) gi r 3 i + λ 3 i ) + λ i g 3 i h i = ), = r + λ )r 2 + λ 2 ) λ λ 2. p fs) = a ɛs) X f) s β ɛs) is given in feedback form by + P ɛs) f ɛs) 2β ɛs), 8) where X f) is the inventory process corresponding to the above production process. The minimal cost is V f = a X 2 + f X h ) q + a 2 X 2 + f 2 X h 2 ) q2. 9) Proof. The proof is based on the completing squares method. By Ito s rule a ɛt) R t X 2 t a ɛ)x 2 = X 2 s da ɛs) + + a ɛs) d X 2 s). 2) Let νω,dt,dx) where the ω will be suppressed as usual) be the jump measure of the process ɛ; that is, for A B, ) R) we have νa) = #{t : t, ɛ t ) A}. The first integral on the right-hand side of 2) can be written as t X 2 s aɛs) a ɛs ) = X 2 s aɛs )+x a ɛs ) νds,dx). 2) <s t R

5 Operations Research ), pp., c INFORMS 5 By another application of Ito s rule 2) can be written as a ɛt) R t X 2 t = a ɛ) X 2 + X 2 s aɛs )+x a ɛs ) νds,dx) + The compensator measure of νdt,dx) is given by R a ɛs) rɛs) X 2 s + 2X s ps µ ɛs) ) + σ 2 ɛs) 2a ɛs) X s σ ɛs) dw s. 22) κdt,dx) = λ ɛt ) dtδ 3 2ɛt ) dx), 23) where δ y dx) represents the Dirac measure centered on y He, Wang, and Yan 992), Theorem.49). Notice that by 8) the expected value of the integral with respect to the Brownian motion on the right-hand side of 22) is zero. On the other hand, by Theorem.8 in Jacod and Shiryaev 23), Chapter II E Xsa 2 ɛs )+x νds,dx) = E Xsa 2 ɛs )+x κds,dx) R R = E Xsa 2 ɛs )+x λ ɛs ) dsδ 3 2ɛs ) dx) R = E Xsa 2 3 ɛs) λ ɛs) ds constant E X 2 s ds <, and one can show similarly that E Xsa 2 ɛs ) νds,dx) = E Xsa 2 ɛs) λ ɛs) ds <. R It follows that after taking expectations in 22) we get E a ɛt) R t X 2 t = E a ɛ) X 2 + {X 2 s a3 ɛs) a ɛs) λɛs) + a ɛs) rɛs) X 2 s + 2X ) s ps µ ɛs) + σ 2 ɛs)) } ds. We now do a similar calculation starting with an application of Ito s rule to f ɛt) R t X t. f ɛt) R t X t = f ɛ) X + f ɛs) ps µ ɛs) r ɛs) X s ds + f ɛs) σ ɛs) dw s. <s t Using the jump measure νdt,dx) of the process ɛ, this can be cast in the form X s fɛs) f ɛs ) ) f ɛt) R t X t = f ɛ) X + f ɛs) ps µ ɛs) r ɛs) X s ds ) t + X s fɛs )+x f ɛs ) νds,dx) f ɛs) σ ɛs) dw s. 25) R 24)

6 6 Operations Research ), pp., c INFORMS In order to deal with the integral with respect to the jump measure we note that by 8) E R ) t X s fɛs )+x f ɛs ) κds,dx) constant E X s ds ) 2 constant te R 2 s X2 s ds <, and the expected value of the integral with respect to the Brownian motion is zero. Hence after taking expectations in 25) and taking 23) into consideration we get E f ɛt) R t X t = E f ɛ) X + We add up equations 24) and 26), and by adding to both sides, we obtain E a ɛt) R t X 2 t + f ɛt) R t X t + = E a ɛ) X 2 + f ɛ)x + ) {f } ɛs) ps µ ɛs) r ɛs) X s + Xs λ ɛs) f3 ɛs) f ɛs) ds. 26) { ) 2 E αɛs) Xs I ɛs) + βɛ s) ) 2 } p s P ɛs) { αɛs) Xs I ɛs) ) 2 + βɛs) ps P ɛs) ) 2 } { α ɛs) Xs I ɛs) ) 2 + β ɛs) ps P ɛs) ) 2 ) +f ɛs) ps µ ɛs) r ɛs) X s + Xs λ ɛs) f3 ɛs) f ɛs) +X 2 s a3 ɛs) a ɛs) λɛs) + a ɛs) rɛs) X 2 s + 2X s ps µ ɛs) ) + σ 2 ɛs)) } ds By completing squares and using the definitions of a i, f i and g i this can be cast in the form { ) E a ɛt) R t X 2 2 ) 2 } t + f ɛt) R t X t + αɛs) Xs I ɛs) + β ɛs) ps P ɛs) = E a ɛ) X 2 + f ɛ)x + β ɛs) p s + a ɛs) X s + f ) 2 ɛs) P ɛs) ds β ɛs) 2β ɛs) +E g ɛs) ds. 27) Notice that the last term involves neither X s nor p s. Taking liminf of both sides as t, and using 9)-), we get Jp) = E a ɛ) X 2 + f ɛ) X + E g ɛs) ds. β ɛs) p s + a ɛs) X s + f ) 2 ɛs) P ɛs) ds + β ɛs) 2β ɛs). 28) From this it follows that Jp) E a ɛ) X 2 + f ɛ) X + E g ɛs) ds 29)

7 Operations Research ), pp., c INFORMS 7 and equality is achieved by p f t) specified in the statement of the theorem. The admissibility of p f follows from Lemma 9. in Appendix B. Next we derive 9). Clearly V f is equal to the right-hand side of 29) so we only need to compute the last term. From straightforward calculus follows that R t h ɛt) = h ɛ) r ɛs) h ɛs) ds + s t hɛs) h ɛs ). Using the jump measure νds,dx) of the process ɛ, this can be written as R t h ɛt) = h ɛ) r ɛs) h ɛs) ds + R hɛs )+x h ɛs ) νds,dx). We take expectations, use again the the compensator κds,dx) of νds,dx) and get E R t h ɛt) = E h ɛ) = E h ɛ) + By 6) and 7) this can be written as Taking limits as t we get which implies 9). r ɛs) h ɛs) ds + R hɛs )+x h ɛs ) κds,dx) { rɛs) h ɛs) + h 3 ɛs) h ɛs) λɛs) } ds. E R t h ɛt) = E h ɛ) + g ɛs) ds. E g ɛs) ds = E h ɛ), 3) To the best of our knowledge, the only other papers that have found an explicit solution for a classical stochastic control problem with regime switching over an infinite horizon are by Sotomayor and Cadenillas 29, 2). Those authors apply the method of dynamic programming. In the present paper we have solved our problem applying the completing squares method. In the case of full information dynamic programming would yield the same solution, and in the full information case the two methods are roughly equally effective. However, we shall see in the next section that in the case of limited information the completing squares method gives results very fast, and is therefore more effective than dynamic programming. Remark 3.. One can write the optimal production rate 8) in the form where p fs) = a ɛs) ) X f) s I ɛs) + Pɛs) β ɛs) +A ɛs) µɛs) P ɛs) ) + Bɛs) µ3 ɛs) P 3 ɛs) ) + Cɛs) I3 ɛs) I ɛs) ), 3) A i = a i β i γa 3 i α 3 i + λ 3 i a i ), B i = λ ia 3 i β i γ, C i = λ iα 3 i β i γ, and γ is given by 4). These formulas can be derived by substituting 3) into 8). We omit the algebraic details, just point out that in the derivation the identity a i β i +r i +λ i = a i α i + λ i a 3 i ) is useful this identity follows from )).

8 8 Operations Research ), pp., c INFORMS Formula 3) has a) simple intuitive interpretation. For brevity, assume that ɛ s = i. The first two terms a i β i X f) s I i and Pi represent short term concerns of the manager. He or she wants to keep the production rate at the target level P i and simultaneously drive the inventory level to its target level I i. The remaining three terms in the formula, which are linear combinations of µ i P i, µ 3 i P 3 i and I 3 i I i, represent long term concerns of the manager. Indeed, if the average demand rate µ i or µ 3 i ) is higher than the target production rate P i or P 3 i ), then the manager expects that in the future the high demand will drive down the inventory level; so he or she compensates for this effect by increasing the production rate in advance. Of course the opposite effect takes place if the average demand rate is lower than the target production rate. If I 3 i > I i than, keeping an eye on the future, the manager increases the production rate in order to get closer to the future inventory target level. The opposite effect takes place if I 3 i < I i. Remark 3.2. It would be of interest to look at the limit of the optimal production process 3) in the full information case when λ and λ λ 2. In this case the economy will spend very little time in regime compared to the amount of time it spends in regime 2. In order to simplify the algebra we assume that r = r 2 = r, α = α 2 = α, and β = β 2 = β, in which case a = a 2 = a and a does not depend on λ i. One can easily see that in this case γ, hence we have A, B 2, and C 2. These results can be understood easily on an intuitive level. The term A µ P ) is a term that is included in the optimal production policy when the regime is equal to. It represents a correction term for the different values of the demand rate and the production target level in regime. But under our assumption the economy is rarely in regime, so one expects this term to be small. Similarly, the term B 2 µ P ) is a term that is included in the optimal production policy when the regime is equal to 2. It represents a correction term for the fact that in the future, when the regime will be, the demand rate and the production target level will be different. Again, it is reasonable that this correction term is small for the same reason as before, i.e., there will be very little time spent in regime. Finally C 2 I I 2 ) is a term that is included in the optimal production policy when the regime is equal to 2. It represents a correction term for the fact that in the future when the regime will be the inventory target rate will be different than it is now. This should be small for the same reason. On the other hand A 2, B and C converge to positive values. This can be explained as follows. The term A 2 µ 2 P 2 ) is a term that is included in the optimal production policy when the regime is equal to 2. It represents a correction term for the different values of the demand rate and the production target level at regime 2. We expect the economy to be in regime 2 most of the time, so we may expect this term to be significant. Similarly, the term B µ 2 P 2 ) represents a correction term, when the actual regime of the economy is, for the different values of the demand rate and the production target level in regime 2. This should not be small for the same reason. The term C I 2 I ) is a correction term, when the actual regime of the economy is, for the different future inventory target level when the regime of the economy will be 2. Again, this correction term should not be small since we do expect to be in regime 2 most of the time. Remark 3.3. Combining equations 6) and 8), we have dx f) t = a ɛt) X f) t β ɛt) P ɛt) 2 ) β ɛt) a f ɛt) µ ɛt) β ɛt) dt σ ɛt) dwt), 32) ɛt) and observe that conditioned on the process ɛ, the optimal inventory level is an Ornstein-Uhlenbeck process with mean-reverting drift. This is consistent with the model proposed by Cadenillas, Lakner and Pinedo 2) for the dynamics of the uncontrolled inventory. It is also worth mentioning that the speed of mean-reversion a ɛt) and the long term mean β ɛt) β ɛt) P ɛt) 2 ) f ɛt) µ ɛt) β ɛt) a ɛt)

9 Operations Research ), pp., c INFORMS 9 do not depend on σ,σ 2 conditioned on ɛ both of these expressions are constants). Remark 3.4. The optimal production rate in 8) is clearly a linear function of the corresponding inventory process. It is worth mentioning that the random) coefficients of this linear function do not depend on σ,σ 2. On the other hand the minimal cost V f given by 9) is an increasing function of both σ and σ 2 ; one can easily verify this using 5), 6), and 7). 4. The case in which the regime is not known Let {F D t, t < } be the filtration generated by the demand process {D t, t < }. We assume that {D t, t < } is observed and {ɛ t, t < } is unobserved. We now require that the production process p be adapted to the filtration {F D t, t < }. We shall assume that r = r 2 = r, σ = σ 2 = σ, α = α 2 = α, and β = β 2 = β. 33) Under this assumption a = a 2 = a and a is the positive root of the quadratic equation ra + α β a2 =. 34) However, f and f 2 may still be different. Since the regime is now not known at any point in time, one may surmise that the target inventory levels and the target production rates should not be regime dependent such targets may most likely be set by management). In practice, one may then expect a single inventory level target and a single production rate target, independent of the regime. However, it turns out that the problem is tractable even if the targets are still regime dependent. In what follows, we first analyze the problem assuming targets that are regime dependent and afterwards consider the special case with a single inventory level target and a single production rate target that are regime independent. We call a production process p admissible if it is F D -adapted and the corresponding inventory process satisfies 8). Let A l be the class of admissible production processes the subscript l refers to the limited information). The cost associated with a production process p becomes Jp) = E R t {α ) 2 X t I ɛt) + β ) } 2 p t P ɛt), where now R t = e rt. The minimal cost is V l = inf p A l Jp). Theorem 4.. The optimal admissible production process under limited information is given by p l s) = a β Xp) s + E P ɛs) 2β f ɛs) F D s, 35) which can be written as p l s) = a β Xl) s + P ) 2β f P { } ɛ s = F D s + P 2 ) 2β f 2 P { } ɛ s = 2 F D s. 36) Here, X l) is the inventory process corresponding to the above production process. The minimal cost is V l = ax 2 + X f h )q + X f 2 h 2 )q 2 + 4β E fɛs) 2βP ɛs) E f ɛs) 2βP ɛs) F D s ) 2 ds. 37)

10 Operations Research ), pp., c INFORMS Proof: Let p be an F D -adapted production process. In the same way that we derived 28), we obtain Jp) = E ax 2 + X f ɛ) + β p s + a β X s + f ) 2 ɛs) 2β P ɛs) ds + g ɛs) ds. 38) Taking 3) in consideration, it follows that Jp) = E ax 2 + X f ɛ) + β p s + a β X s + 2β E ) 2 f ɛs) 2βP ɛs) F D s ds + 4β E fɛs) 2βP ɛs) E ) f ɛs) 2βP ɛs) F D 2 s ds + E g ɛs) ds E ax 2 + X f ɛ) h ɛ) + 4β E fɛs) 2βP ɛs) E f ɛs) 2βP ɛs) F D s ) 2 ds. The last inequality becomes an equality if the production rate is given by 35). The admissibility of this production process follows from Lemma 9.2 in Appendix B. Formula 37) follows from the above calculation. QED To the best of our knowledge, Theorem 4. presents the first analytical solution to a problem of classical stochastic control with regime switching over an infinite horizon with an unobserved regime. In many similar cases of stochastic optimization that have been considered in the literature the optimal policy has been presented in terms of the solution of the Hamilton-Jacobi-Bellman HJB) equation. Here we used the completing squares method instead for the following reason. In the limited information case the optimal policy at a given time t is not a feedback control, i.e., it is not just the function of the inventory level X t. Indeed, the solution involves the conditional probabilities P ɛ s = i Fs D = ˆη i s) see 36) or 55)), and these will depend on the entire path of the observed demand process D up to time t see 48)-49) or Proposition 4.). However, a method based on the HJB equation cannot deal with such situation. Another advantage of our method is that we avoid the complication of solving a nonlinear ordinary differential equation. In order to make the formulas in the statement of Theorem 4. computable we need to compute the conditional probabilities P {ɛ s = i Fs D } for i =,2. In order to follow the standard notations in the theory of hidden Markov processes we introduce the two dimensional process η t = η t),η 2 t)) T as η t = e {ɛt)=} + e 2 {ɛt)=2}, where e and e 2 are the unit vectors in R 2, i.e., e =,) T and e 2 =,) T. Obviously the twodimensional process η carries exactly the same information as the original ɛ and has the same generator ) λ λ Q =. 39) λ 2 λ 2 We introduce the conditional expectation ˆη t = ˆη t), ˆη 2 t)) T = E η t F D t, and notice that P {ɛ s = F D s } = ˆη s) and P {ɛ s = 2 F D s } = ˆη 2 s). 4) Hence we need to compute ˆη t. In this calculation we follow Elliott, Aggoun, & Moore 995). The process η has the semimartingale decomposition η t = η + Q T η s ds + V t, 4)

11 Operations Research ), pp., c INFORMS where V is a martingale Chapter 7, formula 2.9) in Elliott, Aggoun, & Moore 995)). The observable filtration is also generated by σ D t which has the decomposition We define and { Λ t = exp σ σ D t = σ D + σ µ η s) + µ 2 η 2 s))ds + w t. 42) Λ t = { = exp Λ t σ 2 µ η s) + µ 2 η 2 s))dw s 2σ 2 The probability measure P is given by µ η s) + µ 2 η 2 s))dd s 2σ 2 } µ η s) + µ 2 η 2 s)) 2 ds } µ η s) + µ 2 η 2 s)) 2 ds. 43) d P = Λ t dp, dp = Λ t d P, and note that by Girsanov s theorem the process { D σ s D ); s t } is a standard Brownian motion under the probability measure P. By Bayes theorem ˆη t = E Ē Λ η t F D t η F D t = t Ē Λt. 44) Ft D Let ζs) = ζ s),ζ 2 s)) T and ζ i s) = Ē Λs η i s) F D s ; s t, i =,2. 45) By formula 4.) in Chapter 8 of Elliott, Aggoun, & Moore 995), we have the following linear Stochastic Differential Equation SDE) for ζ: ζ t = ζ + Q T ζ s ds + where G is a 2 2 matrix given by G = σ 2 diagµ,µ 2 ) and Equation 46) can be written component-wise as ζ t) = ζ ) + ζ 2 t) = ζ 2 ) + From the relation η s) + η 2 s) = follows that Gζ s dd s, 46) ζ = Eη = q,q 2 ) T. 47) λ ζ s) + λ 2 ζ 2 s)ds + λ ζ s) λ 2 ζ 2 s)ds + σ 2µ ζ s)dd s 48) σ 2µ 2ζ 2 s)dd s. 49) Ē Λt F D t = ζ t) + ζ 2 t). 5) To summarize these results, we have ˆη i t) = ζ i t), i =,2, 5) ζ t) + ζ 2 t)

12 2 Operations Research ), pp., c INFORMS where ζ i t) is calculated by 46) or equivalently by 48)-49)) and by 47). Estimating the solution of the system of SDE s 46) is not a trivial matter. However, we can replace this system of SDE s by a system of linear homogeneous Ordinary Differential Equations ODE) of the first order, which is easier to solve numerically. Let { ) } µi µ 2 Z i u) = exp σ 2D u i 2σ + λ 2 i u, i =,2, and consider the following system of linear first order homogeneous ODE s: ψ u) = λ Z 2 u) 2 Z u) ψ 2u), ψ ) = q 52) ψ 2u) = λ Z u) Z 2 u) ψ u), ψ 2 ) = q 2. 53) Of course Z u) and Z 2 u) are random, but 52) and 53) still represent a system of ODE s for every fixed ω. At time t we observe {D u, u t} hence we can compute {Z i u), u t}. Therefore {Z i u), u t} can be considered known functions for i =,2. Proposition 4.. If ψ u),ψ 2 u)) is a solution of the system of first order linear homogeneous ODE s 52)-53), then ζ i u) = Z i u)ψ i u). Proof. Let ζ i u) = Z i u)ψ i u). Select i =. By 52) we can write ζ s) = Z s) q + s ζ2 u) λ 2 Z u) du. By Karatzas and Shreve 998), Problem or by applying Ito s rule) we have and similarly ζ t) = ζ ) + ζ 2 t) = ζ 2 ) + λ ζ s) + λ 2 ζ2 s) ds + σ 2µ ζ s)dd s λ ζ s) λ 2 ζ2 s) ds + σ 2µ 2 ζ 2 s)dd s in the derivation of the above two identities we work under the probability measure P, and recall that { D σ s D ); s t } is a standard Brownian motion under P). However, by the same reference, the solution to 48)-49) is unique. The statement of the proposition follows. We do not have an explicit solution to the system of ODE s 52)-53). In order to compute ˆη i t) one needs to find a numerical solution to 52)-53), which will determine ˆη i t) via Proposition 4. and 5). Remark 4.. Using basically the same algebra that yielded 3) we can cast 36) in the form p l s) = a β + βγ X l) s E ) I ɛs) F D s + E Pɛs) F D s { λ2 aµ P ) + λ aµ 2 P 2 ) + αe ) } µ ɛs) P ɛs) + λ ɛs) I3 ɛs) I ɛs) F D s, 54)

13 Operations Research ), pp., c INFORMS 3 which can also be written as p l s) = a β Xp s I ˆη s) I 2ˆη 2 s)) + P ˆη s) + P 2ˆη 2 s) + { λ 2 a + αˆη s))µ P )+ λ a + αˆη 2 s))µ 2 P 2 ) βγ } +αλ ˆη s)i 2 I )+ αλ 2ˆη 2 s)i I 2 ). 55) Here, γ is given by 4) which under assumption 33) becomes ) 2 ) a a γ = β + r + λ + λ 2 ) β + r. 56) Formula 54) has the same natural explanation as 3) in the full information case. The first two terms in 54) represent the short term concerns of the manager since he/she wants to select a production rate close to its target level, and drive the inventory close to its target at the same time. However, compared to 3), these target levels are here replaced by their conditional expectation given the available information. The other terms in 54) represent the long term concerns of the manager. Again, since the regime of the economy is not observable, all these terms are replaced by their conditional expectation given the available information, but their interpretation is the same as already described in Remark 3.. Remark 4.2. In the limited information case it may be more reasonable to assume that in addition to 33) we also have I = I 2 = I and P = P 2 = P. This makes sense when the inventory and production rate targets are set by management and not by some exogenous forces); if management sets the targets but does not know the regime of the economy, then it makes sense to have fixed targets that do not depend on the regime. In this case the optimal production rate is p l s) = a β X l) s I ) + P + βγ {λ 2a + αˆη s))µ P)+ λ a + αˆη 2 s))µ 2 P)}. 57) The expression representing the long term concerns of the manager is now a linear combination of µ P and µ 2 P. Remark 4.3. It is again of some interest to look at the limit of the optimal production process 57) in the limited information case when λ and λ λ 2. We make the same assumptions here as in Remark 4.2, that is, 33) and I = I 2 = I, P = P 2 = P. In this case, as already discussed in Remark 3.2, the economy spends very little time in regime compared to the amount of time it spends in regime 2. Recall that both ˆη s) and ˆη s) are bounded: indeed ˆη i s) for each i {, 2}. One can easily see that γ as already pointed out in Remark 3.2), ) hence the coefficient of µ P) converges to while the coefficient of µ 2 P) converges to a >. These results are consistent with our intuition. The term containing µ P) represents a correction term for the different values of the demand rate and the production target level in regime. But under our assumption the economy is rarely in regime, so one expects this term to be irrelevant. Similarly, the term containing µ 2 P) represents a correction term for the different values of the demand rate and the production target level in regime 2. Since the economy spends most of the time in regime 2, this term is very important. a β β +r )

14 4 Operations Research ), pp., c INFORMS 5. Comparisons of optimal costs in the full and limited information cases In order to facilitate a comparison between the full and limited information cases, we shall now assume that 33) holds in the full information case as well. Our objective in this section is to analyze the difference V l V f. It is clear from 9) and 37) that V l V f = 4β E fɛs) 2βP ɛs) E ) f ɛs) 2βP ɛs) F D 2 s ds. 58) Proposition 5.. Under assumption 33) the difference between the optimal costs under full and limited information is where and γ is given by 56). V l V f = 4β f f 2 2β P P 2 ) 2 f f 2 = 2α γ Proof. From 58), it follows that V l V f = µ µ 2 ) + P P 2 ) α a I I 2 ) E η s) ˆη s)) 2 ds, 59) 6) 4β E f 2βP )η s) ˆη s))+ f 2 2βP 2 )η 2 s) ˆη 2 s))) 2 6) and η t) + η 2 t) = ˆη t) + ˆη 2 t) = implies 59). Formula 6) follows from 3) and 34). Remark 5.. If P = P 2 and I = I 2 then we can write the difference between the optimal costs under full and limited information as V l V f = α2 βγ µ 2 µ 2 ) 2 E η s) ˆη s)) 2 ds, which is linear in the term µ µ 2 ) 2. The above difference does not depend on the target values P and I. Notice that the term E η s) ˆη s)) 2 measures the precision of the estimate of the regime when the management has only limited information. So V l V f is proportional to the total discounted expected squared error of the estimate for the regime. Next we shall show that in the limiting case when at least one of the λ s converges to infinity, the advantage of having full information converges to zero. Proposition 5.2. In addition to 33) assume that P = P 2. Let λ n) and λ n) 2 be sequences of persistence rates. If λ n) + λ n) 2 as n, then V n) l V n) f as n. Proof. This follows from 59)-6), since under our assumptions γ converges to infinity, hence f f converges to zero observe that by 34) a does not depend on λ i ). On the other hand the integral term on the right-hand side of 59) is bounded by 4 ds <. Proposition 5.3. If, in addition to 33), ) P n) P n) 2 then lim n =, lim I n) I n) 2 n lim n V n) l ) ) =, and lim µ n) µ n) 2 =, n ) V n) f =.

15 Operations Research ), pp., c INFORMS 5 Proof. Just like in the proof of the previous proposition, one can immediately see from 6) that f f 2 converges to zero, and the statement follows from 59). Proposition 5.4. Assume 33). If σ n)) 2 as n, then V n) l V n) f 4β f f 2 2βP P 2 ) 2 as n. The difference f f 2 is according to 6). Proof. It is sufficient to show that for every t we have Varη s)ds ˆη t) E η t) 62) in probability as σ 2, because then 59) and the Bounded Convergence Theorem imply the statement of the proposition. In order to show 62) it is sufficient to prove that ζ t) E η t) and ζ 2 t) E η 2 t) 63) in L 2 P). Indeed, 63) implies that ζ t)+ζ 2 t) E η t) + η 2 t) = in L 2 P), and by 5) formula 62) follows. Observe that E η t) = Ē Λt η t) = Ē ζ t) so we need to show that ζ i t) Ē ζ it) in L 2 P) for i =,2. Fix a time point t and define w s = σ D s D ), s t. 64) By Girsanov s theorem w is a Brownian motion under P up to time t and we can cast 48) and 49) in the form ζ u) = ζ ) + ζ 2 u) = ζ ) + u u λ ζ s) + λ 2 ζ 2 s)ds + µ σ λ ζ s) λ 2 ζ 2 s)ds + µ 2 σ u u ζ s)d w s 65) ζ 2 s)d w s 66) for all u t. We note that in the two equations above the stochastic integrals have zero mean because by Lemma 9.3 in Appendix B we have, for i =,2, Ē ζ 2 i s)ds = Ē Ē ) t Λs η i s) F D 2 s ds Ē Λ2 sη 2 i s) ds Ē Λ t 2 s ds = E Λ { } K s texp σ 2t. 67) Let m i u) = Ēζ i u). Taking expectations in 65)-66) we get the following system of linear differential equations for m and m 2 : m u) = m ) + m 2 u) = m 2 ) + u u λ m s) + λ 2 m 2 s)ds 68) λ m s) λ 2 m 2 s)ds. 69) D is assumed to be a constant thus ζ i ) = m i ), hence 65) and 68) imply ζ u) m u) = u λ ζ s) m s))+ λ 2 ζ 2 s) m 2 s))ds + µ σ u ζ s)d w s 7)

16 6 Operations Research ), pp., c INFORMS and similarly ζ 2 u) m 2 u) = From 7) and 67) we get 2Ē u u 4tĒ u for u t. Similarly, we can derive u 4tĒ λ ζ s) m s)) λ 2 ζ 2 s) m 2 s))ds + µ 2 σ Ē ζ u) m u)) 2 ) 2 { } λ ζ s) m s)) λ 2 ζ 2 s) m 2 s))ds + 2 µ2 K σ 2texp σ 2t u λ 2 ζ s) m s)) 2 + λ 2 2 ζ 2s) m 2 s)) 2 { } ds + 2 µ2 K σ 2texp σ 2t Ē ζ 2 u) m 2 u)) 2 λ 2 ζ s) m s)) 2 + λ 2 2 ζ 2s) m 2 s)) 2 { } ds + 2 µ2 2 K σ 2texp σ 2t. Adding up these two inequalities we obtain Ē ζ u) m u)) 2 + ζ 2 u) m 2 u)) 2 ζ 2 s)d w s. 7) where θ 2 u Ē ζ u) m u)) 2 + ζ 2 u) m 2 u)) 2 ds + θ, θ = 2µ 2 + µ 2 2) σ 2texp { K σ 2t } and θ 2 = 8tλ 2 + λ 2 2). From Gronwall s inequality Karatzas & Shreve 998), Chapter 5, Problem 2.7), it follows that Ē ζ t) m t)) 2 + ζ 2 t) m 2 t)) 2 which indeed converges to zero as σ 2. θ + θ 2 θ e θ2t s) ds, Remark 5.2. Since η s) = 2 ɛ s so Varη s) = Varɛ s. Also, if in addition to the assumptions in Proposition 5.4 we also assume that P = P 2 = P and I = I 2 = I, then V n) l V n) f α2 βγ µ 2 µ 2 ) 2 Varη s)ds. This limit is linear in µ µ 2 ) 2, and does not depend on the target levels P and I. The integral in the limit represents the total discounted uncertainty of management concerning the unobserved regime of the economy.

17 Operations Research ), pp., c INFORMS 7 6. Computational Results In what follows we analyze the formula 3) numerically. The production rate can be decomposed like ps) = p S s) + p L s), where p S s) is the production rate due to the short-term concerns, and p L s) is the production rate due to the long-term concerns. All of these depend on the current regime ɛ s. Example 6.. Consider demand drifts All other data are the same for the two regimes: µ =., µ 2 =.2. σ = σ 2 =.2, r = r 2 =.2, λ = λ 2 =.2, I = I 2 = 2., P = P 2 =.6, α = α 2 =., β = β 2 = 2. Note that this case is very close to symmetric. The two regimes are identical with the exception of the demand drifts. Regime represents a regime with high demand drift, whereas regime 2 represents a regime with low demand drift. Note that the target production rate is equal to the long term average demand. We obtain and Furthermore, a = a 2 =.2283 a β = a 2 β 2 =.64. A = A 2 =.63, B = B 2 =.243, C = C 2 =.2. These constants are equal to one another since they do not depend on µ i. We now consider three current inventory levels X s. First, let the inventory level at time s be X s =.5. The production rates at time s, assuming we are in state, are p S s) =.97, p L s) =.223, p s) =.94. Similarly, assuming we are in regime 2, p S 2s) =.97, p L 2 s) =.223, p 2 s) =.747. Note that the impact of the short term concerns on the production rates do not depend on the µ i. However, the long term concerns are affected by the µ i. The production rate in regime is clearly higher than in regime 2. Second, let the inventory level at time s be X s = 2. If X s = 2 = I = I 2, then the production rates at time s, assuming we are in state, are p S s) =.6, p L s) =.223, p s) =.823.

18 8 Operations Research ), pp., c INFORMS Similarly, assuming we are in regime 2, p S 2s) =.6, p L 2 s) =.223, p 2 s) = Third, let the inventory level at time s be X s = 2.5. The production rates at time s, assuming we are in state, are Similarly, assuming we are in regime 2, p S s) =.2929, p L s) =.223, p s) = p S 2s) =.2929, p L 2 s) =.223, p 2 s) =.96. In the example above the expected amount of time in regime is equal to the expected amount of time in regime 2. In the next example, the expected amount of time in regime 2 is ten times longer than the expected amount of time in regime. Example 6.2. Consider now the parameter values µ =., µ 2 =.2, λ = 2., λ 2 =.2. This implies that the system remains only a very short time in regime. All other data with regard to the two regimes are identical to those in Example : σ = σ 2 =.2, r = r 2 =.2, I = I 2 = 2., P = P 2 =.6, α = α 2 =., β = β 2 = 2.. Here again, regime represents a regime with high demand drift, while regime 2 represents a regime with low demand drift. The following constants assume the same values as in the previous example: a = a 2 =.2283 and However, If X s =.5, then Similarly, If X s = 2., then Similarly, a β = a 2 β 2 =.64. A =.2538, A 2 =.743, B =.55, B 2 =.5, C =.475, C 2 =.48. p S s) =.97, p L s) =.987, p s) =.884. p S 2s) =.97, p L 2 s) =.267, p 2 s) = p S s) =.6, pl s) =.987, p s) =.53. p S 2s) =.6, p L 2 s) =.267, p 2 s) =.3383.

19 Operations Research ), pp., c INFORMS 9 If X s = 2.5, then Similarly, p S s) =.2929, p L s) =.987, p s) =.942. p S 2s) =.2929, p L 2 s) =.267, p 2 s) =.32. Note that in all three cases the long term component of the production rate is negative. This makes perfect sense, because µ 2 < P and most of the time the economy is in regime 2. Hence a manager with some foresight will reduce the production anticipating a mostly lower demand rate. This effect is, of course, a bit weaker when the economy is in regime ; the long term part of the production rate is still negative. Note that the impact of the short term concerns on the production rates again do not depend on the current regime, whereas the impact of the long term concerns do depend on the current regime. The production rate in regime is always higher than in regime 2. This makes sense, because µ > µ 2. If we compare the above results with those of the baseline case, then we observe that, in regime, the impact of short-term concerns increases when λ increases. This makes sense, because as the economy stays less time in expected value) in regime, the short-term concerns become more important. 7. Conclusions We have obtained, as far as we know for the first time in the literature, an analytical solution for the optimal production management problem when demand depends on the business cycle. The uncertainty is modeled using a Brownian motion as well as a continuous-time Markov chain. We obtained a closed form solution for the optimal production management problem when the manager knows the regime of the economy, and an analytical solution when the manager does not observe the regime of the economy. From a mathematical point of view, we have presented one of the first explicit solutions for a classical stochastic control problem subject to regime switching with full information, and the first analytical solution for a classical stochastic control problem subject to regime switching with limited information. We have applied a method that has allowed us to avoid the difficulties of the standard Hamilton-Jacobi-Bellman method. In addition, our solution in the limited information case is easily computable. The only part that is not explicit in our formula see 36) or 55)) is the vector process ζ t),ζ 2 t)). However, as shown in Proposition 4., this can be calculated numerically by solving a system of linear, non-homogeneous, ODE s. This system of ODE s is stochastic in the sense that the coefficient functions depend on ω, but for a fixed ω it is just a system of deterministic, linear ODE s, and numerical solutions of such systems are quite standard. One of the more important conclusions of this paper is that the factors that determine the optimal production policy can be partitioned into factors that represent short term concerns and factors that represent long term concerns. The explicit formula for the optimal production rate is relatively simple and lends itself easily to algebraic analysis. Another important conclusion is that the optimal production rate at any time t, given the inventory level at that time, does not depend on the volatility in the demand rate. In addition, the difference of the total discounted cost in the full and the limited information cases is proportional to the total discounted expected squared error of the estimate of the state the economy see Proposition 5. and Remark 5.). Our formula for the optimal production rate occasionally requires negative production, that is, disposal of part of the inventory. However, if the parameters satisfy condition 78)-79), then such disposal happens only with a very small probability once the system is in operation for a sufficiently long time.

20 2 Operations Research ), pp., c INFORMS 8. Appendix A Here we shall address the question when the optimal production process under full information given by 3) will be positive, with probability close to. We shall assume that 33) holds. We can write 3) is a short form as p fs) = a β Xf) s + H ɛs) 72) where H i = a β I i + P i + A i µ i P i )+ B i µ 3 i P 3 i )+ C i I 3 i I i ) 73) and A i,b i,c i are given under formula 3). By 6) the corresponding inventory process satisfies X f) t = x + a β Xf) s + H ɛs) µ ɛs) )ds + σw t and it follows that where X f) t = φt) x + ) t Hɛs) µ ɛs) ds φs) φt) = exp { aβ } t. φs) σdw s, Let G be the σ-field generated by {ɛ t,t < }. Conditioned on G, X f) is an Ornstein-Uhlenbeck process, and X f) t follows normal distribution with conditional mean and variance E X f) ) ) t G = φt) x + Hɛs) µ ɛs) ds VAR X f) t G = σ 2 β { exp 2a } φs) 2a β t. Observe that E X f) t φt) x + H H 2 µ µ 2 ) φs) ds xφt) + H H 2 µ µ 2 ) β a, hence the limiting distribution of X f) still conditioned on G) is normal with mean H H 2 µ µ 2 ) β a and SD = σ β 2a, where σ = σ 2 >. It follows that once X f) follows its limiting distribution which will happen for sufficiently large t) then P X f) t H H 2 µ µ 2 ) β β a + z ησ 2a G η, 74) where η is a small probability say.5 or. as customary in statistics), and z η is a number such that PZ > z η = η where Z is a standard normal random variable. For example, if η =.5 then z η =.65. Since the right-hand side of 74) is non-random, the unconditional probability is bounded by the same constant, that is, P X f) t H H 2 µ µ 2 ) β β a + z ησ 2a η. 75)

21 Operations Research ), pp., c INFORMS 2 From 72) follows that p f t) holds if and only if Xf) t β H a ɛt), thus from 75) we conclude that P p f t) η 76) whenever H H 2 µ µ 2 ) β β a + z ησ 2a β a H H 2 ). 77) Using straightforward algebra 77) becomes β z η σ a µ µ 2 H H 2 ), 78) and one can also derive from 73) that H H 2 = α βγ µ P ) µ 2 P 2 )+ a β λ λ 2 ) I I 2 + P P 2. 79) To summarize our results, we derived that if X f) reached its stationary distribution then 76) holds whenever the parameters satisfy condition 78), where H H 2 is given by 79). The parameters will satisfy this numerical condition if σ and µ P ) µ 2 P 2 ) are sufficiently small, and µ,µ 2 are sufficiently large. 9. Appendix B Lemma 9.. If the production process is given by 8) then the corresponding inventory level process X f) satisfies 8). Proof. We can write 6) in the form X f) t = x + where ρ i = P i µ i f i 2β i. Ito s rule yields F t X f) t = x + a ɛs) β ɛs) X f) s + ρ ɛs) ds F s ρ ɛs) ds where { } a ɛs) F t = exp ds. β ɛs) From 8) follows by straightforward algebra and Jensen s inequality that ) 2 3 x 2 + t F 2 t X f) t σ ɛs) dw s, 8) F s σ ɛs) dw s, 8) ) 2 F 2 s ρ 2 ɛs)ds + F s σ ɛs) dw s. 82) Let G = σ {ɛ t ;t < } be the sigma field generated by ɛ already introduced in the Conclusions), and {Q,ω),ω Ω} be the class of regular conditional probabilities given the sigma field G. We note that if A is a zero event under P, that is, PA) =, then QA,ω) = for P-almost all ω Ω. One can show easily that the independence of w and ɛ implies that w is a Brownian motion under Q, ω) for P-almost all ω Ω. Next we observe that the stochastic integral on the right-hand side of 82) can be defined pathwise by F s σ ɛs) dw s = F t σ ɛt) w t w s d F s σ ɛs) ), 83)

22 22 Operations Research ), pp., c INFORMS due to the fact that the integrand in the stochastic integral has finite variation. Thus the right-hand side of 83) represents a version of the stochastic integral on the left-hand side, even under the conditional probability measure Q,ω), for P-almost all ω Ω. This means that 82) holds Q,ω)- almost surely when the stochastic integral is computed under Q, ω) for P-almost all ω Ω). We take expectations on both sides of 82) with respect to the conditional probability measure Q, ω): ) 2 F 2 t E X f) t G ω) 3 x 2 + t F 2 s ρ 2 ɛs)ds + F 2 s σɛs)ds 2 3 x 2 + K t 2 F 2 t + K 2 tf 2 t, where K = max {ρ 2,ρ2 2 } and K 2 = max {σ 2,σ2 2 }. Since F t 2, we get from this the inequality ) 2 E X f) t G ω) 3 x 2 + K t 2 + K 2 t, After taking expectations on both sides now under P), we derive ) 2 E X f) t 3 x 2 + K t 2 + K 2 t, thus E R t X f) t ) 2 3e Kt x 2 + K t 2 + K 2 t where K = min {r,r 2 }, and now 8) follows. Lemma 9.2. If the production process is given by 35) then the corresponding inventory level process X l) satisfies 8). Proof. The proof is quite similar to that of Lemma 9., except that in this case is much simpler since there is no need to use the conditional probability measures. We omit the details. Lemma 9.3. Let Λ s be as in 43). Then where K depends only on µ and µ 2. E Λs exp { K σ 2s } Proof. By the Cauchy-Schwarz inequality { E Λs s = E exp µ η u) + µ 2 η 2 u))dw u + s } µ σ 2σ 2 η u) + µ 2 η 2 u)) 2 du { s = E exp µ η u) + µ 2 η 2 u))dw u s } µ σ σ 2 η u) + µ 2 η 2 u)) 2 du { 3 s } exp µ 2σ 2 η u) + µ 2 η 2 u)) 2 du { 2 s E exp µ η u) + µ 2 η 2 u))dw u 2 s }) 2 µ σ σ 2 η u) + µ 2 η 2 u)) 2 du { 3 s }) 2 E exp µ σ 2 η u) + µ 2 η 2 u)) 2 du { } 3 exp 2σ µ + µ 2 2 ) 2 s.

23 Operations Research ), pp., c INFORMS 23 Acknowledgments The research of A. Cadenillas was supported by the Social Sciences and Humanities Research Council of Canada grant , and by the WCU World Class University) program through the Korea Science and Engineering Foundation funded by the Ministry of Education, Science and Technology R ). References Andijani, A., M. Al-Dajani Analysis of deteriorating inventory/production systems using a linear quadratic regulator. European Journal of Operational Research Bensoussan, A., S.P. Sethi, R. Vickson, N. Derzko Stochastic Production Planning with Production Constraints. SIAM Journal on Control and Optimization Cadenillas, A., P. Lakner, M. Pinedo. 2. Optimal Control of a Mean-Reverting Inventory. Operations Research Elliott, R.J., L. Aggoun, J.B. Moore Hidden Markov Models: Estimation and Control. Springer, New York. Fleming, W.H., S.P. Sethi, H.M. Soner An Optimal Stochastic Production Planning Problem with Randomly Fluctuating Demand. SIAM Journal on Control and Optimization W. Fleming, H.M. Soner Controlled Markov Processes and Viscosity Solutions. Springer, New York. S.W. He, J.G. Wang, J.A. Yan Semimartingale Theory and Stochastic Calculus. CRC Press, Inc., Boca Raton Ann Arbor London Tokyo. Jacod, J., A.N. Shiryaev. 23. Limit Theorems for Stochastic Processes, 2nd edition. Springer, Berlin Heidelberg New York. Karatzas, I., S.E. Shreve Brownian Motion and Stochastic Calculus, 2nd edition. Springer, New York. Khmelnitsky, E., E. Presman, S.P. Sethi. 2. Optimal production control of a failure-prone machine. Annals of Operations Research Sethi, S.P., G.L. Thompson. 2. Optimal Control Theory, Applications to Management Science and Economics. Kluwer Academic Publishers, Norwell. Sotomayor, L.R., A. Cadenillas. 29. Explicit Solutions of Consumption-Investment Problems in Financial Markets with Regime Switching. Mathematical Finance Sotomayor, L.R., A. Cadenillas. 2. Classical and singular stochastic control for the optimal dividend policy when there is regime switching. Insurance: Mathematics and Economics