ICIC Express Letters Part B: Applications ICIC International c 2012 ISSN 2185-2766 Volume 3, Number 2, April 2012 pp. 19 2 UNCERTAIN OPTIMAL CONTROL WITH JUMP Liubao Deng and Yuanguo Zhu Department of Applied Mathematics Nanjing University of Science and Technology Nanjing 21009, Jiangsu, China dengliubao@163.com and ygzhu@mail.njust.edu.cn Received December 2011; accepted March 2012 Abstract. A kind of uncertain jump variable is introduced to describe uncertain events with jump behavior. Then a class of uncertain jump process is defined by uncertain jump variables. An optimal control problem is established for an uncertain dynamical system with jump, which is driven by an uncertain canonical process along with an uncertain jump process. The principle of optimality is given and then the equation of optimality is obtained by applying the method of dynamic programming to solve the proposed problem. As one of applications of the equation, a pension fund control problem is discussed and then the optimal strategies of this problem are presented. Keywords: optimal control, uncertainty, jump process, equation of optimality, pension fund 1. Introduction. Optimal control theory is an important branch of modern control theory. Along with the development of the mathematics and computer science, optimal control theory has a great development not only in theory but also in applications. The study of stochastic optimal control problem was initiated in the late of 1960s and attracted significant interest from many researchers (see, for example, [1-] and the references therein). Since the fuzzy set theory was introduced by scientist on cybernetics Zadeh in 1965, several types of fuzzy optimal control problems were studied in some literature such as [5-12]. However, the complexity of the world makes the events we face be uncertain in various forms. A lot of surveys shows that in many cases, the uncertainty behaves neither like randomness nor like fuzziness. In order to deal with this type of uncertainty, an uncertainty theory was established by Liu [13] in 2007 and refined in 2010 [1] based on normality, self-duality, countable subadditivity, and product measure axioms. Based on uncertain canonical process in uncertainty theory, Zhu [15] introduced and dealt with an uncertain optimal control problem by using the method of dynamic programming in 2010. It is a reasonable model for an uncertain system with the uncertain canonical process. Nevertheless, in real world, some external extreme events or noises have a great influence on uncertain dynamical systems. For example, if an uncertain system jumps from a normal state to a bad state suddenly, the system may be characterized by an uncertain differential equation with jump. In this paper, we will consider an optimal control problem for this kind of system. 2. Preliminary and V -Jump Process. For the convenience, some concepts in uncertainty theory established by Liu [13] will be reviewed first. Let Γ be a nonempty set, and L a σ-algebra over Γ. Each element Λ L is called an event. If the set function M{ } on L satisfies that M{Γ} = 1; M{Λ} + M{Λ c } = 1 for any Λ L; M{ i=1λ i } i=1 M{Λ i} for every countable sequence of events {Λ i }, then M is called an uncertain measure, and the triplet (Γ, L, M) is said to be an uncertainty space. Note that an uncertain measure M has the property: M{Λ 1 } M{Λ 2 } whenever Λ 1 Λ 2 for Λ 1, Λ 2 L. Moreover, an uncertain variable is defined as a measurable function 19
20 L. DENG AND Y. ZHU from an uncertainty space to the set R of real numbers, i.e., for any Borel set B in R, {ξ B} = {γ Γ ξ(γ) B} L. The uncertainty distribution Φ: R [0, 1] of an uncertain variable ξ is defined by Φ(x) = M{ξ x}. The expected value of an uncertain variable ξ is defined as E[ξ] = + M{ξ r} dr 0 M{ξ r} dr provided that at 0 least one of the two integrals is finite. For independent uncertain variables ξ and η, we have E[ξ + η] = E[ξ] + E[η]. Based on normally distributed uncertain variable, Liu [16] introduced a kind of uncertain canonical process. An uncertain process C t is called a canonical process if (i) C 0 = 0 and almost all sample paths are Lipschitz continuous, (ii) C t has stationary and independent increments, (iii) every increment C s+t C s is a normally distributed uncertain variable with expected value 0 and variance t 2, whose uncertainty distribution is Φ(x) = ( 1 + exp ( )) πx 1. 3t Theorem 2.1. (Liu [1]) Let f be a convex function on [a, b], and ξ an uncertain variable that takes values in [a, b] and has expected value e. Then E[f(ξ)] b e b a f(a) + e a b a f(b). Theorem 2.2. (Zhu [15]) Let ξ be a normally distributed uncertain variable with the ( ( )) πx 1. uncertainty distribution Φ(x) = 1 + exp 3σ Then for any real number a, σ 2 2 E[aξ + ξ2 ] σ 2. In stochastic dynamical systems, a jump is generally modeled as a Poisson process. In this paper, we will model the discontinuous jump part of uncertain dynamical systems by introducing a so-called V -jump process, which is associated with an uncertain Z-jump variable Z(r 1, r 2, t) defined by a jump uncertainty distribution. Definition 2.1. An uncertain variable Z(r 1, r 2, t) is said to be an uncertain Z-jump variable with parameters r 1 and r 2 (0 < r 1 < r 2 < 1) for t > 0 if it has a jump uncertainty distribution 0, if x < 0 2r 1 Φ(x) = t x, if 0 x < t 2 r 2 + 2(1 r ( 2) x t ), if t t 2 2 x < t 1, if x t. Theorem 2.3. Assume ξ 1 and ξ 2 are independent uncertain Z-jump variables Z(r 1, r 2, t 1 ) and Z(r 1, r 2, t 2 ), respectively. Then the sum ξ 1 + ξ 2 is also an uncertain Z-jump variable Z(r 1, r 2, t 1 + t 2 ). The product of an uncertain Z-jump variable Z(r 1, r 2, t) and a scalar number k > 0 is also an uncertain Z-jump variable Z(r 1, r 2, kt). Proof: Assume that the uncertain variables ξ 1 and ξ 2 have uncertainty distributions Φ 1 (x) and Φ 2 (x), respectively. Then for α (0, r 1 ) [r 2, 1), we have t 1 + t 2 α, if 0 < α < r 1 2r 1 Φ 1 1 (α) + Φ 1 2 (α) = (α r 2 )(t 1 + t 2 ) 2(1 r 2 ) + t 1 + t 2, if r 2 α < 1. 2 Let Φ : R [0, 1] be an increasing function satisfying that Φ 1 (α) = Φ 1 1 (α) + Φ 1 2 (α) for α (0, r 1 ) [r 2, 1). Then for any x (0, t 1 + t 2 ), there is an α (0, r 1 ) [r 2, 1) such
ICIC EXPRESS LETTERS, PART B: APPLICATIONS, VOL.3, NO.2, 2012 21 that x = Φ 1 (α). On the one hand, M{ξ 1 + ξ 2 Φ 1 (α)} = M{ξ 1 + ξ 2 Φ 1 1 (α) + Φ 1 2 (α)} M{(ξ 1 Φ 1 1 (α)) (ξ 2 Φ 1 2 (α))} = M{ξ 1 Φ 1 1 (α)} M{ξ 2 Φ 1 2 (α)}} = α α = α. On the other hand, there exists an index i = 1 or i = 2 such that {ξ 1 + ξ 2 Φ 1 1 (α) + Φ 1 2 (α)} {ξ i Φ 1 i (α)}. Thus M{ξ 1 + ξ 2 Φ 1 (α)} M{ξ i Φ 1 i (α)} = α. It follows that M{ξ 1 + ξ 2 x} = M{ξ 1 + ξ 2 Φ 1 (α)} = α. In other words, Φ is just the uncertainty distribution of ξ 1 + ξ 2. Hence the sum ξ 1 + ξ 2 is also an uncertain Z-jump variable Z(r 1, r 2, t 1 + t 2 ). The first part is verified. The second part of the theorem can be proved similarly. Definition 2.2. An uncertain process V t is said to be a V -jump process with parameters r 1 and r 2 (0 < r 1 < r 2 < 1) for t 0 if (i) V 0 = 0, (ii) V t has stationary and independent increments, (iii) every increment V s+t V s is an uncertain Z-jump variable Z(r 1, r 2, t). Let V t be an uncertain V -jump process, and V t = V t+ t V t. Then E[ V t ] = + 0 (1 Φ(x)) dx = t 0 (1 Φ(x)) dx = 3 r 1 r 2 Theorem 2.. (Existence Theorem) There is an uncertain V -jump process. Proof: Without loss of generality, we only prove that there is an uncertain V -jump process on the range of t [0, 1]. Let {ξ(r) r represents rational numbers in [0, 1]} be a countable sequence of independent uncertain Z-jump variables Z(r 1, r 2, 1). For each positive integer n, we define an uncertain process as follows: V n (0) = 0, and V n (t) = 1 k ξ n i=1 linear, ( i n ), if t = k n otherwise. (k = 1, 2,, n) By Theorem 2.3, we may verify that the limit lim n V n (t) exists almost surely, and the limitation meets the conditions of V -jump process. Hence there is an uncertain V -jump process. Theorem 2.5. Let C t be an uncertain canonical process, and V t an uncertain V -jump process. Denote ζ = bξ + dη, where ξ = C t, η = V t, b, d R. Let ξ and η be independent. Then for any real number a, E[aζ + ζ 2 ] = ad(3 r 1 r 2 ) t + o( t). (1) Proof: To begin with we have E[aζ + ζ 2 ] E[aζ] = ae[bξ + dη] = a(be[ξ] + de[η]) = ad(3 r 1 r 2 ) t. (2) On the other hand, since aζ +ζ 2 = a(bξ +dη)+b 2 ξ 2 +d 2 η 2 +2bdξη a(bξ +dη)+2(b 2 ξ 2 + d 2 η 2 ) = (abξ +2b 2 ξ 2 )+(adη +2d 2 η 2 ), we have E[aζ +ζ 2 ] E[abξ +2b 2 ξ 2 ]+E[adη +2d 2 η 2 ] because abξ + 2b 2 ξ 2 and ad 2 η + 2d 2 η 2 are independent. It follows from Theorem 2.2 that E[abξ + 2b 2 ξ 2 ] = o( t). Moreover we can get E[adη + 2d 2 η 2 ] E[η] t (ad t + 2d2 ( t) 2 ) = 3 r 1 r 2 (ad t + 2d 2 ( t) 2 ) = ad(3 r 1 r 2 ) t + o( t) t.
22 L. DENG AND Y. ZHU by Theorem 2.1. Thus E[aζ + ζ 2 ] ad(3 r 1 r 2 ) t + o( t). (3) Combining inequalities (2) and (3), we can obtain the result (1). 3. Uncertain Optimal Control Problem with Jump. Now we present the following uncertain optimal control model with jump. [ T ] J(t, x) sup E f(s, X s, u s ) ds + G(T, X T ) u s t subject to () dx s = ν(s, X s, u s ) ds + γ(s, X s, u s ) dc s + χ(s, X s, u s ) dv s, X t = x. where X s denotes the state variable, u s the decision (control) variable, f the objective function, and G the function of terminal reward, ν, γ and χ are three functions of time s, state X s and control u s, C s is an uncertain canonical process and V s an uncertain V -jump process with parameters r 1, r 2 for s > 0, and C s and V s are independent. The J(t, x) is the expected optimal reward obtainable in [t, T ] with the initial condition that at time t we are in state x. To obtain a solution of the model () we present the following principle of optimality and equation of optimality. Theorem 3.1 (Principle of optimality). For any (t, x) [0, T ) R, and t > 0 with t + t < T, we have [ t+ t ] J(t, x) = sup E f(s, X s, u s ) ds + J(t + t, x + X t ), (5) u s t where x + X t = X t+ t. The proof of the theorem is parallel to the corresponding result in [15]. Theorem 3.2 (Equation of optimality). Let J(t, x) be twice differentiable on [0, T ] R. Then we have { J t (t, x) = sup f(t, x, ) + ν(t, x, )J x (t, x) + 3 r } 1 r 2 χ(t, x, )J x (t, x) (6) where J t (t, x) and J x (t, x) are the partial derivatives of the function J(t, x) in t and x, respectively. Proof: For any t > 0, we have t+ t By using Taylor series expansion, we get t f(s, X s, u s ) ds = f(t, x, u(t, x)) t + o( t) (7) J(t + t, x + X t ) =J(t, x) + J t (t, x) t + J x (t, x) X t + 1 2 J tt(t, x) t 2 Substituting equations (7) and (8) into equation (5) yields { f(t, x, ) t + J t (t, x) t + E 0 =sup + 1 2 J xx(t, x) X t 2 + J tx (t, x) t X t + o( t). (8) [ J x (t, x) X t + 1 2 J tt(t, x) t 2 + 1 ] } 2 J xx(t, x) X 2 t + J tx (t, x) t X t + o( t). (9)
ICIC EXPRESS LETTERS, PART B: APPLICATIONS, VOL.3, NO.2, 2012 23 Let η be an uncertain variable such that X t = η + ν(t, x, ) t. It follows from (9) that 0 = sup { f(t, x, ut ) t + J t (t, x) t + ν(t, x, )J x (t, x) t + E[aη + bη 2 ] + o( t) }, (10) where a J x (t, x)+j xx (t, x)ν(t, x, ) t+j tx (t, x) t, and b 1 2 J xx(t, x). It follows from the uncertain differential equation, the constraint in (), that η = X t ν(t, x, ) t = γ(t, x, ) C t + χ(t, x, ) V t. Theorem 2.5 implies that E[aη + bη 2 ] = 3 r 1 r 2 χ(t, x, )J x (t, x) t + o( t). (11) Substituting equation (11) into equation (10) yields J t (t, x) t =sup {f(t, x, ) t + ν(t, x, )J x (t, x) t + 3 r 1 r 2 χ(t, x, )J x (t, x) t + o( t) }. (12) Dividing equation (12) by t, and letting t 0, we can obtain the result (6). The theorem is proved. Remark 3.1. The equation of optimality (6) gives a necessary condition for an extremum. When the equation has solutions, we may derive the optimal control strategy and optimal expected value of objective function. When f is concave in its arguments, then the equation will produce a maximum, and when function f is convex in its arguments, then it will produce a minimum. In practice, the conditions of the Theorem 3.2 may be not difficultly satisfied because the objective function f is generally an elementary one. Remark 3.2. Note that there is also an equation of optimality for uncertain optimal control problem [15]. Comparing with that equation, there is an extra term 3 r 1 r 2 χ(t, x, u t ) J x (t, x) in the equation of optimality (6), which calls us that uncertain optimal control problem with jump is different from one without jump.. Optimal Control of Pension Fund. In this section, we will employ the equation of optimality obtained in the previous section to solve a pension fund problem. We consider the following optimal control problem of pension fund: [ ] J(t, x) min e βs {α 1 (u s c m ) 2 + α 2 (wx s x p ) 2 } ds,w E t subject to dx t = {[b + (µ b)w]x t + B} dt + σ 1 wx t dc t + σ 2 wx t dv t, X t = x where α 1 > 0 and α 2 > 0, e βs is a discount function, b is the return rate on the risky-free asset, and c m, x p, X t,, B denote the constant target contribution rate, the constant target funding level, the fund amount at time t, the contribution rate at time t, and the pension scheme benefit outgo, respectively. The µ, σ 1, σ 2 denote the mean rate of return and the variances of per unit time of risky asset, respectively. The w and 1 w are the proportion of the funds allocated in the risky asset and the risky-free asset, respectively. And C t is an uncertain canonical process and V t an uncertain V -jump process with parameters r 1, r 2 for t > 0. This type of problem was considered by Cairns [17] under stochastic environments. Therefore the selection about some of above parameters may be referred to corresponding literature. By the equation of optimality (6), we have J t = min,w { [ e βt α 1 ( c m ) 2 + α 2 (wx x p ) 2] + (bx + B)J x ( + µ b + 3 r ) } 1 r 2 σ 2 wxj x. (13)
2 L. DENG AND Y. ZHU Conjecture that J(t, x) = e βt (px 2 + qx + r). Denote k = µ b + (3 r 1 r 2 )σ 2 /, a = c m B + kx p, and h = (1/α 1 + k 2 /α 2 )/. Solving (13) gets the optimal contribution rate and the asset allocation proportion by = c m 2b β α 1 h ( x 2 + a b ), w = 1 x And the optimal value is J(t, x) = e βt (px 2 + qx + r), where p = 2b β h, q = (2b β)a 2bh [ k(2b β) ( x x p α 2 h 2 + a ) ]. b, r = a β a2 (2b β) 2 ). βb 2 h Remark.1. The conclusions in the above example are similar to those in the case of randomness (Cairns [17]), but with a modification due to the effect of uncertain jumps. 5. Conclusion. By introducing an uncertain V -jump process, we established an uncertain optimal control model with jump. The principle of optimality and the equation of optimality for uncertain optimal control with jump were obtained. The equation of optimality was efficiently employed to deal with an uncertain pension fund control model with jump. In future work, we will focus on further study such as linear-quadratic model of uncertain optimal control problem with jump. Acknowledgment. This work is supported by the National Natural Science Foundation of China (No.6087038) and HSSF of Anhui(China) (Grant No.2010sk210, No.2011sk177). REFERENCES [1] R. C. Merton, Optimal consumption and portfolio rules in a continuous time model, Journal of Economic Theory, vol.3, no., pp.373-13, 1971. [2] A. K. Dixit and R. S. Pindyck, Investment under Uncertainty, Princeton University Press, Princeton, 199. [3] I. V. Kolmanovsky and T. L. Maizenberg, Optimal control of continuous-time linear systems with a time-varying, random delay, Systems & Control Letters, vol., no.2, pp.119-126, 2001. [] W. Zhang, Finite-time control of linear stochastic systems, International Journal of Innovative Computing, Information and Control, vol., no.3, pp.689-696, 2008. [5] D. Filev and P. Angelov, Fuzzy optimal control. Fuzzy Sets and Systems, vol.7, no.2, pp.151-156, 1992. [6] L. M. Jia, and X. D. Zhang, The fuzzy multiobjective optimal control problem, Engineering Applications of Artificial Intelligence, vol.6, no.2, pp.153-16, 1993. [7] R. Ostermark, A fuzzy control model for dynamic portfolio management, Fuzzy Sets and Systems, vol.78, no.3, pp.23-25, 1996. [8] J. Yao and J. Su, Fuzzy total demand and maximum inventory with backorder based on signed distance method, International Journal of Innovative Computing, Information and Control, vol., no.9, pp.229-2261, 2008. [9] Y. Zhu, A fuzzy optimal control model, Journal of Uncertain Systems, vol.3, no., pp.270-279, 2009. [10] Y. Zhu, Fuzzy optimal control for multi-stage fuzzy systems, IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics, vol.1, no., pp.96-975, 2011. [11] Y. Zhao and Y. Zhu, Fuzzy optimal control of linear quadratic models, Computers and Mathematics with Applications, vol.60, no.1, pp.67-73, 2010. [12] Z. Qin, M. Bai, and D. Ralescu, A fuzzy control system with application to production planning problems, Information Sciences, vol.181, no.5, pp.1018-1027, 2011. [13] B. Liu, Uncertainty Theory, 2nd ed., Springer-Verlag, Berlin, 2007. [1] B. Liu, Uncertainty Theory: A Branch of Mathematics for Modeling Human Uncertainty, Springer- Verlag, Berlin, 2010. [15] Y. Zhu, Uncertain optimal control with application to a portfolio selection model, Cybernetics and Systems, vol.1, no.7, pp.535-57, 2010. [16] B. Liu, Some research problems in uncertainty theory, Journal of Uncertain Systems, vol.3, no.1, pp.3-10, 2009. [17] A. Cairns, Some notes on the dynamics and optimal control of stochastic pension fund models in continuous time, ASTIN Bulletin, vol.30, no.1, pp.19-55, 2000.