Yuefen Chen & Yuanguo Zhu

Indefinite LQ optimal control with equality constraint for discrete-time uncertain systems Yuefen Chen & Yuanguo Zhu Japan Journal of Industrial and Applied Mathematics ISSN 0916-7005 Volume 33 Number 2 Japan J. Indust. Appl. Math. 2016) 33:361-378 DOI 10.1007/s13160-016-0217-9 1 23

Your article is protected by copyright and all rights are held exclusively by The JJIAM Publishing Committee and Springer Japan. This e-offprint is for personal use only and shall not be self-archived in electronic repositories. If you wish to self-archive your article, please use the accepted manuscript version for posting on your own website. You may further deposit the accepted manuscript version in any repository, provided it is only made publicly available 12 months after official publication or later and provided acknowledgement is given to the original source of publication and a link is inserted to the published article on Springer's website. The link must be accompanied by the following text: "The final publication is available at link.springer.com. 1 23

Japan J. Indust. Appl. Math. 2016) 33:361 378 DOI 10.1007/s13160-016-0217-9 ORIGINAL PAPER Area 4 Indefinite LQ optimal control with equality constraint for discrete-time uncertain systems Yuefen Chen 1,2 Yuanguo Zhu 1 Received: 1 November 2015 / Revised: 27 March 2016 / Published online: 17 May 2016 The JJIAM Publishing Committee and Springer Japan 2016 Abstract Based on uncertainty theory, this paper studies a kind of discrete-time uncertain linear quadratic LQ) optimal control with equality constraint for the terminal state, allowing the state and control weighting matrices in the cost function to be indefinite. First, we transform the uncertain LQ optimal control problem into an equivalent deterministic optimal control problem. Then, a necessary condition for the existence of optimal linear state feedback control is presented by means of matrix minimum principle. Moreover, the well-posedness of the uncertain LQ problem is proved by applying the technique of completing squares. Finally, an example is provided to demonstrate the effectiveness of our theoretical results. Keywords Indefinite LQ optimal control Equality constraint Discrete-time uncertain systems Constrained difference equation Mathematics Subject Classification 49J21 49K21 93B52 93C55 1 Introduction Up to now, optimal control theory has been extensively developed and found many interesting applications in finance 1, production planning 2, optimal consumption 3 and so on. LQ optimal control which is initiated by Kalman 4 and extended to stochastic systems by Wonham 5, is one of the most fundamental and widely used B Yuanguo Zhu ygzhu@njust.edu.cn 1 School of Science, Nanjing University of Science and Technology, Nanjing 210094, China 2 College of Mathematics and Information Science, Xinyang Normal University, Xinyang 464000, China

362 Y. Chen, Y. Zhu tools in modern engineering. Until very recently, the main focus has been on the case when the control weight is positive definite. In fact, for deterministic systems, if the control weight is not positive definite, the problem is not well-posed. It has been shown for the first time 6 that for stochastic LQ optimal control, the control weights which can be indefinite or even negative definite, the problem is still well-posed. After that, several papers have been published on this subject, such as continuous time case 7 9 and discrete time case 10,11, as well as with Markovian jumps 11,12 and infinite horizon case 9,13. Regarding the applications, the indefinite LQ optimal control problems arise naturally in many practical situations especially in finance 6,8. In practical engineering, the system state and control input are always subject to various constraints, so the constrained stochastic LQ problem is a more attractive topic and has a concrete application background. Reference 1 was devoted to a stochastic LQ optimal control and an application to portfolio selection, where the control variable is confined to a cone, and all the coefficients of the state equation are random processes. Reference 14 presented a tractable approach to solve the constrained stochastic LQ optimal control problem. Reference 15 dealt with the optimal control problem for deterministic and stochastic linear systems with linear state equality constraints. Reference 16 considered an indefinite LQ optimal control with constraint for discrete-time stochastic systems with state and disturbance dependent noise, and presented a necessary condition for the existence of an optimal control. Reference 17 studied an indefinite discrete-time stochastic LQ problem with a second moment constraint on the terminal state, shown that the well-posedness and the attainability of the LQ problem, the solvability of the generalized difference Riccati equations are equivalent to each other. As we know, the stochastic LQ optimal control problems have been well studied by probability theory which is based on a large number of sample size. A fundamental premise of applying probability theory is that the estimated probability distribution is close enough to the long-run cumulative frequency. Otherwise, the law of large numbers is no longer valid and probability theory is no longer applicable. When the sample size is large enough, it is possible for us to believe the estimated probability distribution is close enough to the long-run cumulative frequency. In this case, there is no doubt that probability theory is the only legitimate approach to deal with the problems on the basis of the estimated probability distributions. However, the sample size in practice is often too small or there is even no-sample to estimate a probability distribution. Thus, we have to invite some domain experts to evaluate their belief degree that each event will occur. In order to model such uncertain phenomena, uncertainty theory was founded by Liu 18 in 2007, refined by Liu 19 in 2010, and became a branch of mathematics. As an application of uncertainty theory, Zhu 20 proposed an uncertain optimal control in 2010, and gave an optimality equation as a counterpart of Hamilton- Jacobi-Bellman equation. Since then, some uncertain optimal control problems have been solved. For example, Xu and Zhu 21 investigated an uncertain bang-bang control problem for a continuous time model, Deng and Zhu 22 dealt with an uncertain LQ optimal control problem with jump, Sheng and Zhu 23 studied an optimistic value model of uncertain optimal control problem and proposed an equation of optimality to solve the model, Yan and Zhu 24 established an uncertain optimal control model for switched systems and introduced a two-stage algorithm to handle such model.

Indefinite LQ Optimal Control for Discrete-Time Uncertain Systems 363 Inspired by the stochastic indefinite LQ optimal control and uncertain optimal control problems, this paper studies an indefinite LQ optiaml control with equality constraint for discrete-time uncertain systems, in which the state and control in dynamics depend on linear uncertain noises. This problem comes from the practical problems where the noise disturbances are lack of observation data to get probability distribution, we have to choose uncertain variables as the noises in the control model. The outline of this paper is organized as follows. In Sect. 2, we give some definitions about uncertainty theory. Section 3 presents an indefinite LQ optimal control with equality constraints and gives a necessary condition for the existence of optimal control. In Sect. 4, we give a sufficient condition for the well-posedness of the constrained LQ optimal control. A numerical example is given in Sect. 5. Finally, we conclude the paper in Sect. 6. For convenience, throughout the paper, we adopt the following notations: R n is the real n-dimensional Euclidean space; R m n the set of all m n matrices; M the transpose of a matrix and trm) the trace of a square matrix M. Moreover, M > 0 resp. M 0) means that M = M and M is positive resp. positive semi-) definite. 2 Some definitions about uncertainty theory In this section, we introduce some useful definitions about uncertainty theory, such as uncertain measures, uncertain variables and uncertainty distributions. Definition 1 Liu 18) Let L be a σ -algebra on a nonempty set Ɣ. Asetfunction M : L 0, 1 is called an uncertain measure if it satisfies the following axioms. Axiom 1 Normality Axiom) M{Ɣ} =1 for the universal set Ɣ. Axiom 2 Duality Axiom) M{ }+M{ c }=1 for any event. Axiom 3 Subadditivity Axiom) For every countable sequence of events 1, 2,..., we have { } M i M{ i }. i=1 The triplet Ɣ, L, M) is called as an uncertainty space Liu 18). Besides, the product uncertain measure was defined by Liu 25 in 2009, thus producing the fourth axiom of uncertainty theory. Axiom 4 Product Axiom) Let Ɣ k, L k, M k ) be uncertainty spaces for k = 1, 2,... Then, the product uncertain measure M on the product σ -algebra satisfies { } M k = M k { k }, k=1 where k are arbitrarily chosen events from L k for k = 1, 2,..., respectively. i=1 k=1

364 Y. Chen, Y. Zhu An uncertain variable is essential a measurable function on an uncertainty space. Definition 2 Liu 18) An uncertain variable is a function ξ from an uncertainty space Ɣ, L, M) to the set of real numbers such that {ξ B} is an event for any Borel set B of real numbers. Definition 3 Liu 19) The uncertain variables ξ 1,ξ 2,...,ξ m are said to be independent if { m } M ξ i B i ) = min M{ξ i B i } 1 i m i=1 for any Borel sets B 1, B 2,...,B n of real numbers. Definition 4 Liu 18) The uncertainty distribution of an uncertain variable ξ is defined by x) = M{γ Ɣ ξγ) x}, for any real number x. Definition 5 Liu 19) An uncertainty distribution x) is said to be regular if it is a continuous and strictly increasing function with respect to x at which 0 < x) <1, and lim x x) = 0, lim x + x) = 1. Example 1 Liu 18) A linear uncertain variable ξ has a linear uncertainty distribution 0, if x a x) = x a)/b a), if a x b 1, if x b where a and b are real numbers with a < b, denoted by ξ La, b). Remark 1 According to Definition 5, it is easy to prove that linear uncertainty distribution is regular. Definition 6 Liu 19) Let ξ be an uncertain variable with regular uncertainty distribution x). Then the inverse function 1 x) is called the inverse uncertainty distribution of x). Expected value is the average value of uncertain variable in the sense of uncertain measure. Definition 7 Liu 18) Let ξ be an uncertain variable. Then the expected value of ξ is defined by + 0 Eξ = M{ξ r}dr M{ξ r}dr 0 provided that at least one of the two integrals is finite.

Indefinite LQ Optimal Control for Discrete-Time Uncertain Systems 365 Theorem 1 Liu 19) Let ξ and η be independent uncertain variables with finite expected values. Then for any real numbers a and b, we have Eaξ + bη =aeξ+beη. Remark 2 Generally speaking, the expected value operator is not necessarily linear if the independence is not assumed. Definition 8 Let ξ 11 ξ 12 ξ 1q ξ = ξ 21 ξ 22 ξ 2q, ξ p1 ξ p2 ξ pq where ξ ij are uncertain variables for i = 1, 2,...,p, j = 1, 2,...,q. The expected value of ξ is provided by Eξ 11 Eξ 12 Eξ 1q Eξ = Eξ 21 Eξ 22 Eξ 2q. Eξ p1 Eξ p2 Eξ pq Example 2 Let ξ be an uncertain variable with regular uncertainty distribution, and let f x) be a strictly monotone increasing or decreasing) function. Then E f ξ) = 1 0 f 1 α))dα. Example 3 Let ξ La, b) with 0 a < b. Then Eξ 2 = 1 0 1 α) 2 dα = 1 0 a + b a)α 2 dα = 4 3 b a)2 + ab. 3 Indefinite LQ optimal control with equality constraint 3.1 Problem setting Consider the indefinite LQ optimal control with equality constraint for discrete-time uncertain systems as follows. inf Jx 0, u) = E x u k Q k x k + u k R ku k + E x N Q N x N k0 k subject to x k+1 =A k x k +B k u k + λ k A k x k + B k u k )ξ k, k = 0, 1,...,, x0) = x 0 F x N = η, 1)

366 Y. Chen, Y. Zhu where λ k R and 0 λ k 1. The vector x k XƔ, R n ) is a state vector with the initial state x 0 R n and u k UR n, U k ) is a control vector subject to a constraint set U k R m, where XƔ, R n ) denotes the function space of all uncertain vectors measurable functions from Ɣ to R n ), and UR n, U k ) denotes the function space of all functions from R n to U k. In this paper we assume U k = R m. Denote u = u 0, u 1,...,u ). Moreover, Q 0, Q 1,...,Q N and R 0, R 1,...,R are real symmetric matrices with appropriate dimensions. In addition, the coefficients A 0, A 1,...,A and B 0, B 1,...,B are assumed to be crisp matrices with appropriate dimensions. Let F R r n, η = η 1,η 2,...,η r ), where η i i = 1, 2,...,r) are uncertain variables. Besides, the noises ξ 0,ξ 1,...,ξ are independent linear uncertain variables L 1, 1) with the distribution 0, if x 1 x) = x + 1)/2, if 1 x 1 1, if x 1. Note that we allow the cost matrices to be singular or indefinite. We need to give the following definitions. Definition 9 The uncertain LQ problem 1) is called well-posed if V x 0 ) = inf Jx 0, u) >, x 0 R n. u k0 k Definition 10 A well-posed problem is called solvable, if for x 0 R n, there is a control sequence u 0, u 1,...,u ) that achieves V x 0). In this case, the control sequence u 0, u 1,...,u ) is called an optimal control sequence. 3.2 An equivalent deterministic optimal control In this subsection, we transform the uncertain LQ problem 1) into an equivalent deterministic optimal control problem. Let X k = Ex k x k. Since state x k R n, we know that x k x k is a n n matrix which elements are uncertain variables, and X k is a symmetric crisp matrix k = 0, 1,...,N). Denote K = K 0, K 1,...,K ), where K i are matrices for i = 0, 1,...,N 1. Theorem 2 If the uncertain LQ problem 1) is solvable by a feedback control sequence u k = K k x k, for k = 0, 1,...,N 1, where K 0, K 1,...,K are constant crisp matrices, then the uncertain LQ problem 1) is equivalent to the following deterministic optimal control problem

Indefinite LQ Optimal Control for Discrete-Time Uncertain Systems 367 min JX 0, K) = K k 0 k subject to X k+1 = 1 + 1 3 λ2 k k = 0, 1,...,N 1, X 0 = x 0 x 0, FX N F = G, G = Eηη. tr Q k + K k R k K k )X k + tr Q N X N ) Ak X k A k + A k X k Kk B k + B k K k X k A k + B k K k X k Kk ) B k Proof Assume that the uncertain LQ problem 1) is solvable by a feedback control sequence u k = K k x k, for k = 0, 1,...,N 1. Considering the dynamical equation of the uncertain LQ problem 1), we have X k+1 = E x k+1 x k+1 { = E A k + B k K k + λ k A k + B k K k )ξ k x k x k A k + K k B k + λ ka k + K k B k )ξ } k = A k X k A k + A k X k Kk B k + B k K k X k A k + B k K k X k Kk B k + EU k ξ k + V k ξk 2, 3) where U k = 2λ k A k X k A k + A k X k Kk B k + B k K k X k A k + B k K k X k Kk B k ), V k = λ 2 k A k X k A k + A k X k Kk B k + B k K k X k A k + B k K k X k Kk B k ). It is easily found that λ k U k = 2V k. Now we compute EU k ξ k + V k ξk 2 as follows. i) If V k = 0, we obtain EU k ξ k + V k ξ 2 k =EU kξ k =U k Eξ k =0. ii) If V k = 0, we know that λ k = 0 and 2 λ k 2. According to Example 2 in 28, we have E 2 2 U k ξ k + V k ξk 2 = E V k ξ k + V k ξk 2 = V k E ξ k + ξk 2 = 1 λ k λ k 3 V k. Based on above analysis, we conclude that E 2) U k ξ k + V k ξk 2 = 1 3 V k. 4)

368 Y. Chen, Y. Zhu Substituting 4)into3), we know that 3) can be written as X k+1 = 1 + 1 ) 3 λ2 k A k X k A k + A k X k Kk B k + B k K k X k A k + B k K k X k Kk B k ). 5) Moreover, the associated cost function is expressed equivalently as min K k 0 k JX 0, K) = min K k 0 k tr Q k + K k R k K k )X k + tr Q N X N. Note that F x N x N F = ηη. 6) Taking expectations in 6), we have FX N F = G, G = Eηη. Therefore, the uncertain LQ problem 1) is equivalent to the deterministic optimal control problem 2). Remark 3 Obviously, if the uncertain LQ problem 1) has a linear feedback optimal control solution u k = K k x k k = 0, 1,...,N 1), then Kk k = 0, 1,...,N 1) is the optimal solution of the deterministic LQ problem 2). 3.3 Definition and lemmas about Moore-Penrose inverse For later use, we first give the following definition and lemmas about Moore-Penrose inverse which will be used to prove our main conclusion. Definition 11 Penrose 26) For any matrix M R m n, a matrix M + R n m is called the Moore-Penrose inverse of M if it satisfies MM + M = M, M + MM + = M +, MM + ) = MM +, M + M) = M + M. 7) The Moore-Penrose inverse M + that satisfies 7) exists and is unique for any M. Lemma 1 Rami 7) For a symmetric matrix S, one has i) S + = S + ), ii) S 0 if and only if S + 0, iii) SS + = S + S. Lemma 2 Penrose 26) Let matrices L, M and N be given with appropriate sizes. Then the matrix equation LXM = N 8)

Indefinite LQ Optimal Control for Discrete-Time Uncertain Systems 369 has a solution X if and only if LL + NMM + = N. Moreover, any solution to 8) is represented by X = L + NM + + Y L + LY MM + where Y is a matrix with an appropriate size. 3.4 A necessary condition for state feedback control In this subsection, we apply the deterministic matrix minimum principle 27 to get a necessary condition for the optimal linear state feedback control with deterministic gains to the uncertain LQ optimal control problem 1). Theorem 3 If the uncertain LQ problem 1) is solvable by a feedback control u k = K k x k, for k = 0, 1,...,N 1, 9) where K 0, K 1,...,K are constant crisp matrices, then there exist symmetric matrices H k, and a matrix ρ R r r solving the following constrained difference equation H k = Q k + 1 + 1 3 k) λ2 A k H k+1 A k Mk L+ k M k L k L + k M k M k = 0, and L k 0 L k = R k + 1 + 1 3 k) λ2 B k H k+1 B k M k = 10) 1 + 1 3 k) λ2 B k H k+1 A k H N = Q N + F ρ F for k = 0, 1,...,N 1. Moreover K k = L + k M k + Y k L + k L ky k 11) with Y k R m n,k= 0, 1,...,N 1, being any given crisp matrices. Proof Assume the uncertain LQ problem 1) is solvable by u k = K k x k, for k = 0, 1,...,N 1, where the matrices K 0,...,K are viewed as the control to be determined. It is obvious that the problem 2) is a matrix dynamical optimization problem. Next, we will deal with this class of problems by minimum principle. Introduce the Lagrangian function associated with problem 2) as follows L = JX 0, K) + trh k+1 g k+1 X k, K k )+trρgx N ),

370 Y. Chen, Y. Zhu where JX 0, K) = tr Q k + Kk R k K k )X k + tr Q N X N, g k+1 X k, K k ) = 1 + 1 3 λ2 k )A k X k A k + A k X k Kk B k + B k K k X k A k +B k K k X k K k B k ) X k+1 gx N ) = FX N F G, and the matrices H 0,...,H k+1 as well as ρ R r r are the Lagrangian multipliers. By the matrix minimum principle 27, the optimal feedback gains and Lagrangian multipliers satisfy the following first-order necessary conditions L = 0 k = 0, 1,...,N 1), K k 12) H k = L k = 0, 1,...,N). X k 13) Based on the partial rule of gradient matrices, 12) can be transformed into R k + 1 + 1 ) 3 λ2 k B k H k+1 B k K k + 1 + 1 ) 3 λ2 k Bk H k+1 A k = 0. 14) Let L k = R k + M k = 1 + 1 3 λ2 k 1 + 1 3 λ2 k ) B k H k+1 B k ) B k H k+1 A k. 15) Then 14) can be rewritten as L k K k + M k = 0. Applying Lemma 2, the solution of 14) is given by K k = L + k M k + Y k L + k L ky k, Y k R m n. 16) if and only if L k L + k M k = M k.by13), we first have that is H N = L X N, 17) H N = Q N + F ρ F. Secondly, we have H k = L X k k = 0, 1,...,N 1),

Indefinite LQ Optimal Control for Discrete-Time Uncertain Systems 371 which is H k = Q k + 1 + 1 ) 3 λ2 k A k H k+1 A k + Kk + 1 + 1 ) 3 λ2 k A k H k+1 B k K k + R k + 1 + 1 3 λ2 k ) B k H k+1 B k K k 1 + 1 3 λ2 k ) Kk B k H k+1 A k. 18) Substituting 16) into18), by using Definition 11 and Lemma 1, it follows H k = Q k + 1 + 1 ) 3 λ2 k A k H k+1 A k Mk L+ k M k. 19) The objective functional Jx 0, u) = = = E x k Q k x k + u k R ku k + E x N Q N x N E { x k Q k x k + u k R ku k + E x k+1 H k+1 x k+1 E x k H k x k } + E x N Q N x N E x N H N x N + x 0 H 0 x 0 { tr Qk + K k R k K k )X k + tr Hk+1 X k+1 tr Hk X k } + tr Q N H N )X N + x 0 H 0x 0. 20) Substituting 5)into20), we can rewrite the cost function as follows JX 0, K) = { tr Q k +Kk R k K k ) + 1 + 1 ) 3 λ2 k A k H k+1 A k +Bk H k+1 A k K k + A k H k+1 B k K k + Kk B k H } k+1 B k K k ) H k Xk + tr Q N H N )X N + x 0 H 0x 0 { = tr Q k + 1 + 1 ) 3 λ2 k A k H k+1 A k H k + 2 1 + 1 ) 3 λ2 k Bk H k+1 A k K k + Kk R k + 1 + 1 ) } 3 λ2 k B k H k+1 B k K k X k + tr Q N H N )X N + x 0 H 0x 0. 21)

372 Y. Chen, Y. Zhu Substituting 15) and 19) into21), a completion of square implies JX 0, K) = tr K k + L + k M k) L k K k + L + k M k)x k + tr Q N H N )X N + x 0 H 0x 0. 22) Next, We will prove that L k k = 0, 1,...,N 1) must satisfy L k = R k + 1 + 1 ) 3 λ2 k Bk H k+1 B k 0. 23) If it is not so, there is an L p for p {0, 1,...,N 1} with a negative eigenvalue λ. Denote the unitary eigenvector with respect to λ as v λ i.e., v λ v λ = 1 and L p v λ = λv λ ). Let δ = 0 be an arbitrary scalar, we construct a control sequence ũ = ũ 1, ũ 2,...,ũ ) as follows ũ k = { L + k M k x k, k = p δ λ 2 1 v λ L + k M k x k, k = p. 24) By 22), the associated cost functional becomes Jx 0, ũ) = = tr K k + L + k M k) L k K k + L + k M k)x k + tr Q N H N )X N + x 0 H 0x 0 E ũ k + L + k M k x k ) L k ũ k + L + k M k x k ) + tr Q N H N )X N + x 0 H 0x 0 δ δ = v λ 2 1 λ L p v λ 2 1 λ + tr Q N H N )X N + x 0 H 0x 0 = δ 2 + tr Q N H N )X N + x 0 H 0x 0. Letting δ, it yields Jx 0, ũ), which contradicts the solvability of the uncertain LQ problem 1). 4 Well-posedness of the uncertain LQ problem In this section, we will show that the solvability of the Eq. 10) is sufficient for the well-posedness of the uncertain LQ problem 1). Moreover, any optimal control can be obtained via the solution to the Eq. 10).

Indefinite LQ Optimal Control for Discrete-Time Uncertain Systems 373 Theorem 4 The uncertain LQ problem 1) is well-posed if there exist symmetric matrices H k solving the constrained difference equation 10). Moreover, the uncertain LQ problem 1) is solvable by u k = R k + 1 + 1 ) + 3 λ2 k B k H k+1 B k 1 + 1 ) 3 λ2 k B k H k+1 A k x k, k = 0, 1,...,N 1. 25) Furthermore, the optimal cost of the uncertain LQ problem 1) is V x 0 ) = x 0 H 0x 0 trρg). Proof Let H k solve the Eq. 10). Then we have Jx 0, u) = = = E x k Q k x k + u k R ku k + E x N Q N x N { E x k Q k x k + u k R ku k + E x k+1 H k+1 x k+1 E x k H k x k } +E x N Q N x N E x N H N x N + x 0 H 0 x 0 { tr Qk + K k R k K k )X k + tr Hk+1 X k+1 tr Hk X k } +tr Q N H N )X N + x 0 H 0x 0 { = tr Q k + 1 + 1 ) 3 λ2 k A k H k+1 A k H k = +2 1 + 1 ) 3 λ2 k Bk H k+1 A k K k R k + 1 + 1 ) } 3 λ2 k B k H k+1 B k K k X k +K k +tr Q N H N )X N + x 0 H 0x 0 tr M k L+ k M k + 2M k K k + K k L k K k Xk +tr Q N H N )X N + x 0 H 0x 0 By Definition 11 and Lemma 1, a completion of square implies JX 0, K) = tr K k + L + k M k) L k K k + L + k M k)x k +tr Q N H N )X N + x 0 H 0x 0. 26)

374 Y. Chen, Y. Zhu Because of L k 0, we know that the cost function of problem 1) is bounded from below by V x 0 ) tr Q N H N )X N + x 0 H 0x 0 >, x 0 R n. Hence the uncertain LQ problem 1) is well-posed. By 26) it is clear that the uncertain LQ problem 1) is solvable by the feedback control u k = K k x k = L + k M k x k, k = 0, 1,...,N 1. Furthermore, 26) indicates that the optimal value equals V x 0 ) = tr Q N H N )X N + x 0 H 0x 0. Since { H N = Q N + F ρ F FX N F = G, and X N = E x N x N, we obtain V x 0 ) = x 0 H 0x 0 trρg). Remark 4 We have shown that the solvability of the constrained difference equation 10) is sufficient for the existence of an optimal linear state feedback control. As a special case, we consider the following indefinite LQ optimal control without constraint for the discrete-time uncertain systems. inf Jx 0, u) = u k0 k E x k Q k x k + u k R ku k +E x N Q N x N subject to x k+1 = A k x k + B k u k + λ k A k x k + B k u k )ξ k, k = 0, 1,...,, x0) = x 0. 27) Corollary 1 If the uncertain LQ problem 27) is solvable by a feedback control u k = K k x k, for k = 0, 1,...,N 1, 28)

Indefinite LQ Optimal Control for Discrete-Time Uncertain Systems 375 where K 0, K 1,...,K are constant crisp matrices, then there exist symmetric matrices H k that solve the following constrained difference equation H k = Q k + 1 + 1 ) 3 λ2 k A k H k+1 A k Mk L+ k M k L k L + k M k M k = 0, and L k 0 L k = R k + 1 + 1 ) 3 λ2 k Bk H k+1 B k M k = 1 + 1 ) 3 λ2 k Bk H k+1 A k H N = Q N 29) for k = 0, 1,...,N 1. Moreover K k = L + k M k + Y k L + k L ky k 30) with Y k R m n,k= 0, 1,...,N 1, being any given crisp matrices. Furthermore, the uncertain LQ problem 1) is solvable by u k = R k + 1 + 1 ) + 3 λ2 k B k H k+1 B k 1 + 1 ) 3 λ2 k B k H k+1 A k x k, k = 0, 1,...,N 1, the optimal cost of the uncertain LQ problem 27) is given by V x 0 ) = x 0 H 0x 0. Proof Let F = 0 and η = 0 in the constrained uncertain LQ problem 1). Then the constrained uncertain LQ problem 1) becomes the unconstrained uncertain LQ problem 27). The conclusions in the corollary directly follow by similar approach as in the Theorems 3 and 4. 5 Numerical example Next we present a two-dimensional indefinite LQ optimal control with equality constraint for discrete-time uncertain systems to illustrate the effectiveness of our result. In the constrained discrete-time uncertain LQ control problem 1), we give out a set of specific parameters of the coefficients: x 0 = 0 1, F = 1) 2, 1 ), N = 2, η L0, 5 3/4), 2

376 Y. Chen, Y. Zhu and A 0 = ) 10, A 00 1 = ) 10 1 1, B 10 0 =, B 0) 1 =, λ 1) 0 = 0.2, λ 1 = 0.1. The state weights and the control weights are as follows Q 0 = ) 1 0, Q 0 1 1 = ) 10, Q 0 0 2 = ) 00, R 00 0 = 1, R 1 = 4. Note that in this example, the state weight Q 0 is negative definite, Q 1 is negative semidefinite, Q 2 is positive semidefinite, and the control weight R 0 is negative definite. The constraint is given as follows FX 2 F = FEx 2 x 2 F = G = Eη 2 = 25 4. Firstly, it follows from H k = Q k + 1 + 1 ) 3 λ2 k A k H k+1 A k Mk L+ k M k L k L + k M k M k = 0 L k = R k + 1 + 1 3 λ2 k M k = 1 + 1 3 λ2 k H 2 = Q 2 + F ρ F X k+1 = 1 + 1 3 λ2 k k = 0, 1, X 0 = x 0 x 0, FX 2 F = 25 4, that ρ = 8. Then we have ) B k H k+1 B k 0 ) B k H k+1 A k, k = 0, 1. ) A k X k A k + A k X k K k B k + B k K k X k A k + B k K k X k K k B k ), H 2 = Q 2 + F ρ F = ) 2 2. 2 2 Secondly, applying Theorem 3, we obtain the optimal controls and optimal cost value as follows. For k = 1, we obtain L 1 = R 1 + M 1 = 1 + 1 3 λ2 1 ) B 1 H 2 B 1 = 12.1067, 1 + 1 ) 3 λ2 1 B1 H 2 A 1 = 8.1067, 0),

Indefinite LQ Optimal Control for Discrete-Time Uncertain Systems 377 H 1 = Q 1 + 1 + 1 ) 3 λ2 1 A 1 H 2 A 1 M1 L+ 1 M 1 = The optimal feedback control is u 1 = K 1 x 1 where K 1 = L + 1 M 1 = 0.6696, 0). For k = 0, we obtain L 0 = R 0 + 1 + 1 ) 3 λ2 0 B0 H 1 B 0 = 0.6840, M 0 = 1 + 1 ) 3 λ2 0 B0 H 1 A 0 = 1.6840, 0), H 0 = Q 0 + 1 + 1 ) 3 λ2 0 A 0 H 1 A 0 M0 L+ 0 M 0 = The optimal feedback control is u 0 = K 0 x 0 where Finally, the optimal cost value is K 0 = L + 0 M 0 = 2.4620, 0). V x 0 ) = x 0 H 0x 0 trρg) = 53.9460. ) 1.6784 5.4283. 5.4283 5.4283 ) 2.9620 4.1460. 4.1460 3.9460 6 Conclusion We have studied an indefinite LQ optimal control with an equality constraint for discrete-time uncertain systems, which can be transformed into an equivalent deterministic constrained matrix dynamical optimization problem. By applying matrix minimum principle, we have presented a necessary condition for the existence of optimal linear state feedback control. Besides, we have proved the well-posedness of the constrained indefinite LQ optimal control problem by applying the technique of completing squares. For further work, we will consider an indefinite LQ optimal control with inequality constraint for discrete-time uncertain systems. Acknowledgments This work is supported by the National Natural Science Foundation of China No. 61273009). References 1. Hu, Y., Zhou, X.Y.: Constrained stochastic LQ control with random coefficients, and application to portfolio selection. SIAM J. Control Optim. 442), 444 466 2005) 2. Bensoussan, A., Vickson, R., et al.: Stochastic production planning with production constraints. SIAM J. Control Optim. 226), 920 935 2000) 3. Watanable, Y.: Study of a degenerate elliptic equation in an optimal consumption problem under partial information. Jpn. J. Ind. Appl. Math. 321), 157 185 2015)

378 Y. Chen, Y. Zhu 4. Kalman, R.E.: Contributions to the theory of optimal control. Boletin de la Sociedad Matematica Mexicana 52), 102 119 1960) 5. Wonham, W.M.: On a matrix Riccati equation of stochastic control. SIAM J. Control Optim. 64), 681 697 1968) 6. Chen, S.P., Li, X.J., Zhou, X.Y.: Stochastic linear quadratic regulators with indefinite control weight costs. SIAM J. Control Optim. 365), 1685 1702 1998) 7. Rami, M.A., Zhou, X.Y.: Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls. IEEE Trans. Autom. Control 456), 1131 1143 2000) 8. Zhou, X.Y., Li, D.: Continuous-time mean-variance portfolio selection: a stochastic LQ framework. Appl. Math. Optim. 421), 19 33 2000) 9. Rami, M.A., Zhou, X.Y., Moore, J.B.: Well-posedness and attainability of indefinite stochastic linear quadratic control in infinite time horizon. Syst. Control Lett. 412), 133 2000) 10. Rami, M.A., Chen, X., Zhou, X.Y.: Discrete-time indefinite LQ control with state and control dependent noises. J. Global Optim. 233), 245 265 2002) 11. Costa, O.L.V., Oliveira, A.D.: Optimal mean-variance control for discrete-time linear systems with Markovian jumps and multiplicative noises. Automatica 482), 304 315 2012) 12. Li, X., Zhou, X.Y.: Indefinite stochastic LQ controls with Markovian jumps in a finite time horizon. Commun. Inf. Syst. 2, 265 282 2002) 13. Huang, J., Li, X., Yong, J.M.: A linear quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Math. Control Related Fields 51), 97 139 2015) 14. Bertsimas, D., Brown, D.B.: Constrained stochastic LQC: a tractable approach. IEEE Trans. Autom. Control 5210), 1826 1841 2007) 15. Ko, S., Bitmead, R.R.: Optimal control for linear systems with state equality constraints. Automatica 439), 1573 1582 2007) 16. Liu, X., Li, Y., Zhang, W.: Stochastic linear quadratic optimal control with constraint for discrete-time systems. J. Appl. Math. 2289), 264 270 2014) 17. Zhang, W., Li, G.: Discrete-time indefinite stochastic linear quadratic optimal control with second moment constraints. Math. Probl. Eng. 111), 809 812 2014) 18. Liu, B.: Uncertainty theory, 2nd edn. Springer, Berlin 2007) 19. Liu, B.: Uncertainty theory: a branch of mathematics for modeling human uncertainty. Springer, Berlin 2010) 20. Zhu, Y.: Uncertain optimal control with application to a portfolio selection model. Cybern. Syst. 417), 535 547 2010) 21. Xu, X., Zhu, Y.: Uncertain bang-bang control for continuous time model. Cybern. Syst. 43, 515 527 2012) 22. Deng, L., Zhu, Y.: Uncertain optimal control with jump. ICIC Express Lett. Part B Appl. 32), 419 424 2012) 23. Sheng, L., Zhu, Y.: Optimistic value model of uncertain optimal control. Int. J. Uncertain. Fuzziness Knowl. Based Syst. 21Suppl. 1), 75 87 2013) 24. Yan, H., Zhu, Y.: Bang-bang control model for uncertain switched systems. Appl. Math. Model. 3910-11), 2994 3002 2015) 25. Liu, B.: Some research problems in uncertainty theory. J. Uncertain Syst. 31), 3 10 2009) 26. Penrose, R.: A generalized inverse of matrices. Math. Proc. Camb. Philos. Soc. 513),406 413 1955) 27. Athans, M.: The matrix minimum principle. Inf. Control 11, 592 606 1968) 28. Zhu, Y.: Functions of uncertain variables and uncertain programming. J. Uncertain Syst. 64), 278 288 2012)