On Irregular Linear Quadratic Control: Deterministic Case

Transcription

1 1 On Irregular Linear Quadratic Control: Deterministic Case Huanshui Zhang and Juanjuan Xu arxiv: v2 math.oc 1 Mar 218 Abstract This paper is concerned with fundamental optimal linear quadratic (LQ control with irregular Riccati equation where the controller could not be solved from the equilibrium condition like the classical LQ control. The problem remains challenging because it is not clear what tool should be applied instead to design the controller although the problem has been investigated since 7 s last century Different from the standard regular Riccati equation based LQ-control theory, a new approach of two-layer optimization is proposed. With the approach, we show the controller entries of irregular-lq controller are derived from two equilibrium conditions in two different layers, which is different from the classical regular LQ control where all the controller entries were obtained from equilibrium condition in one layer based on regular Riccati equation. It is shown that the irregular LQ control is totally different from the regular one due to the irregular controller must guarantee the state terminal constraint ofp 1(Tx(T. The presented results also clarify the differences of open-loop control from closed-loop control in the aspects of solvability and controller design. Examples are presented to show the effectiveness of the proposed approach. The presented results contain fundamental contribution to the LQ-control theory. The stochastic case will be reported in another paper. Index Terms Irregular LQ control; Riccati equation; Openloop solvability; Closed-loop solvability. I. INTRODUCTION Linear quadratic (LQ control is one of the most fundamental and widely used tools in modern engineering. Two approaches including maximum principle 6 and dynamic programming 7 have been extensively studied since 195 s. See 8, 9 and references therein. In the literature it is typically assumed that the cost function has a positive definite weighting matrix for the control term, and a positive semidefinite weighting matrix for the state term 14. In this case, the Riccati equation is used to characterize the solvability of the underlying control problem considered a special case of singular LQ control problem where the weighting matrix of control is zero. In 21, the LQ control with semi-positive weighting matrix of control term was studied. It is noted that only the sub-optimal controller is given although it shows that a constructed sequence of cost function converges to the optimal one. 12, 16 found the well-posedness of the stochastic LQ problem where the cost weighting matrices for the state and the control are allowed to be indefinite. A new type of Riccati equation, called the general Riccati equation, is proposed to explicitly construct the optimal feedback control. In very recently, 19 studied irregular LQ problem with indefinite weighting matrices, focusing on the closed-loop and open-loop solvability for the stochastic control. 19 showed that the solvability obtained in previous work 16 is only applicable to the case of closed-loop control, and the openloop solvability given in 16 is incorrect by presenting a counterexample. In particular, the open-loop control was obtained by constructing a minimizing sequence and the closed-loop solution was given in the feedback form of the state where the feedback gain is required to be in the space L 2 (t,t;r m n. In this paper, following-up the previous works 16, 19, we will study the irregular LQ control problem and and try to clarify the following questions: Question 1: The solvability conditions for the irregular- LQ problem? Question 2: What is the difference of irregular-lq from the regular-lq? Question 3: How to design the irregular controller in open-loop and closed-loop form? By proposing so-called two-layer optimization approach, Question 1 and 2 are to be answered in Section IV, while Question 3 will be answered in Section V. The remainder of the paper is organized as follows. Section II presents the studied problem and some preliminaries. The optimal solution is shown in Section III when Riccati equation is regular. The results for irregular LQ control problem are given in Section IV and V respectively. Two numerical examples are given in Section VI. Some concluding remarks are given in Section VII. The following notations will be used throughout this paper: R n denotes the family of n dimensional vectors. x means the transpose of x. A symmetric matrix M > ( means strictly positive definite (positive semi-definite. Range(M represents the range of the matrix M. M means the Moore- Penrose inverse of the matrix M. We also introduce the following set: This work is supported by the National Natural Science Foundation of China ( , This version has been submitted to IEEE Trans. on Automatic Control for publication H. Zhang and J. Xu are with School of Control Science and Engineering, Shandong University, Jinan, Shandong, P.R.China hszhang@sdu.edu.cn, jnxujuanjuan@163.com L 2 (t,t;r m n {K(t R m n, K(t 2 dt < }, t Ut,T {u(t R m, t u(t 2 dt < }.

2 2 II. DEFINITIONS AND PRELIMINARIES Consider the continuous-time system: ẋ(t A(tx(t+B(tu(t, x(t x, (1 where x R n is the state, u R m is the control input. The matrices A(t, B(t are time-varying matrices with appropriate dimension. x is the given initial value. The cost function is given by J(t,x ;u t x (tq(tx(t+u (tr(tu(tdt +x (THx(T. (2 where Q(t, R(t, H are symmetric matrices with appropriate dimension. Problem (LQ: For any given initial pair (t,x, find a u such that J (t,x ;u min u( J(t,x ;u. V(t,x. To guarantee the convexity of Problem (LQ, it is assumed that R(t. We now introduce the definitions of solvability. Definition 1. Problem (LQ is said to be finite at initial pair (t,x,t R n, if V(t,x >. (3 Problem is said to be finite at t,t if (3 holds for all x R n. Problem is said to be finite if (3 holds for all (t,x,t R n. Definition 2. An element u is called an open-loop optimal control of Problem (LQ for the initial pair (t,x,t R n, if J (t,x ;u J(t,x ;u, u(. (4 If an open-loop optimal control (uniquely exists for (t,x,t R n, Problem (LQ is said to be (uniquely open-loop solvable at (t,x,t R n. Problem (LQ is said to be (uniquely open-loop solvable at t,t if for the given t, (4 holds for all x R n. Problem (LQ is said to be (uniquely open-loop solvable if it is open-loop solvable at all (t,x,t R n. Definition 3. An element u ( K( x ( Ut,T is called a closed-loop optimal control of Problem (LQ on t,t if J (t,x ;K( x ( J(t,x ;u, x R n, u( Ut,T. (5 wherex ( is the solution to the following closed-loop system: ẋ (t A(t+B(tK(tx (t, x(t x. (6 If a closed-loop optimal controller exists on t,t, Problem (LQ is said to be (uniquely closed-loop solvable on t,t. Problem (LQ is said to be (uniquely closed-loop solvable if it is (uniquely closed-loop solvable on any t,t. Remark 1. Noting that the closed-loop solvability is only assumed that K( x ( Ut,T which allows the case that K( L 2 (t,t;r m n. We firstly recall the Moore-Penrose inverse of a matrix which will be used this paper repeatedly. From 23, for a given matrix M R n m, there exists a unique matrix in R m n denoted by M such that MM M M, M MM M, (MM MM, (M M M M. The matrix M is called the Moore-Penrose inverse of M. The following lemma is from 22. Lemma 1. Let matrices L, M and N be given with appropriate size. Then, LXM N has a solution X if and only if LL NMM N. Moreover, the solution of LXM N can be expressed as X L NM +Y L LYMM, where Y is a matrix with appropriate size. In particular, let M I, we have LX N has a solution if and only if LL N N. This is also equivalent to Range(N Range(L where Range(N is the range of N. We now present the the maximum principle. Lemma 2. Problem (LQ is solvable if and only if the following forward and backward differential equations (FBDEs admit a solution: ẋ(t A(tx(t + B(tu(t, (7 ṗ(t A (tp(t+q(tx(t, (8 R(tu(t+B (tp(t. (9 with x(t x and p(t Hx(T. Proof. The result follows from the maximum principle 11. So we omit it. From Lemma 2, we note that the key to solve Problem (LQ is to obtain the solution to (7-(9. To this end, without loss of generality, we assume that p(t P(tx(t+Θ(t (1 where P, Θ are to be determined with terminal values P(T H and Θ(T. Taking derivative to (1 yields ṗ(t P(tx(t+P(t A(tx(t+B(tu(t + Θ(t. (11 From (8 and (1, it is obtained that ṗ(t A (tp(tx(t+a (tθ(t+q(tx(t. (12 With a comparison of (11 and (12, it follows that P(tx(t+P(tA(tx(t +P(tB(tu(t + Θ(t+A (tp(tx(t+a (tθ(t +Q(tx(t. (13 Using (1, the equilibrium condition (9 is reformulated as R(tu(t+B (tp(t Υ (tu(t+γ (tx(t+b (tθ(t, (14

3 3 where Υ (t R(t and Γ (t B (tp(t. There exists two cases: Range Γ (t Range Υ (t and Range Γ (t Range Υ (t. III. OPTIMAL CONTROL WITH REGULAR RICCATI EQUATION In this section, we consider the case: Range Γ (t Range Υ (t, (15 In this case, the following results are obtained directly from 16. Lemma 3. If (15 holds in (14 where P(t is given by the following Riccati equation: P(t+A (tp(t+p(ta(t +Q(t Γ (tυ (tγ (t, (16 with P(T H, then Problem (LQ is solvable and the optimal solution is given by u(t Υ (tγ (tx(t+ I Υ (tυ (t z(t, (17 where R (t represents the Moore-Penrose inverse ofr(t and z(t is an arbitrary vector with compatible dimension. In this case, Θ(t holds in (1. In the case of Range Γ (t Range Υ (t, the Riccati equation (16 is called regular. IV. SOLUTION TO IRREGULAR LQ CONTROL This section focuses on the LQ control problem when (15 does not hold, i.e., Range Γ (t Range Υ (t. (18 In this case the problem is termed as the irregular LQ control problem (IR-LQ, and the controller u(t can not be solved from the equilibrium condition (14. From (18, it is clear that Υ (t is not invertible. ( We assume that rank(υ (t m (t < m. Thus rank I Υ (tυ (t m m (t >. It is not difficult to know that there is an elementary row transformation matrix T (t such that T (t I Υ (tυ (t Υ T (t, (19 where Υ T (t R m m(t m is full row rank. Further denote C (t Γ (t I Υ (tυ (t T 1 (t, B (t B(t( I Υ (tυ (t T 1 (t, (2 and A (t A(t B(tΥ (tγ (t, D (t B(tΥ (tb (t. Lemma 4. Under the condition (18, Problem (IR-LQ is solvable if and only if there exists u 1 (t R m m(t such that C (tx(t+b (tθ(t,θ(t (21 holds, where u 1 (t, x(t,θ(t satisfy the FBSDEs ẋ(t A (tx(t+d (tθ(t+b (tu 1 (t, (22 Θ(t A (tθ(t+c (tu 1(t. (23 Proof. Necessity. If the problem is solvable, then from Lemma 2, (14 must hold. Thus, (14 can be equivalently written as u(t Υ (Γ (t (tx(t+b (tθ(t + I Υ (tυ (t z(t, (24 where z(t is an arbitrary vector with compatible dimension such that following equality hold Denote (I Υ (tυ (t Γ (tx(t+b (tθ(t.(25 T (t I Υ (tυ (t z(t u 1 (t, (26 where u 1 (t Υ T (tz(t R m m(t. Now we rewrite (25 as (21. First, note that I Υ (tυ (I (t Υ (tυ (I (t Υ (tυ (t (I Υ (tυ T (t (t T 1 (t (I Υ (tυ (t Υ T (t ( T 1 (t (I Υ (tυ (t, (27 where (19 has been used in the derivation of the last equality. By using (2, (25 can be written as Υ T (t C (tx(t+b (tθ(t. (28 Note that Υ T (t is full column rank, (28 is rewritten as (21 directly. By substituting (24 into (13 and using (16, it yields that P(tx(t+P(tA(tx(t+A (tp(tx(t +A (tθ(t+q(tx(t Γ (tυ (Γ (t (tx(t +B (tθ(t +Γ (t I Υ (tυ (t z(t Θ(t+ ( A (t Γ (tυ (tb (t Θ(t +Γ (t I Υ (tυ (t z(t. (29

4 4 2 In view of the fact that I Υ (tυ (t I Υ (tυ (t, it is obtained that Γ (t I Υ (tυ (t z(t Γ (I (t Υ (tυ (t T 1 (tt (t I Υ (tυ (t z(t Γ (t I Υ (tυ (t T 1 (t u 1 (t C (t u 1 (t C (tu 1(t. (3 Thus, from (29 we have Θ(t A (tθ(t+c (tu 1(t, (31 this implies that the dynamic of Θ is as (23. By substituting ( (24 into (7 and combining with the fact that B(t I Υ (tυ (t z(t B (tu 1 (t which can be obtained similar to (3, one has the dynamic (22 of the state. Sufficiency We now show Problem (IR-LQ is solvable if there exists u 1 (t to achieve (21. In fact, if (21 is true then (24 and (25 can be rewritten as (14. Further, by taking reverse procedures to (1-(14, it is easily verified that p(t P(tx(t+Θ(t, where x, Θ satisfy (21-(23, are the solutions to (7-(9. Thus, Problem (IR-LQ is solvable according to Lemma 2. The proof is now completed. Remark 2. Applying (26 we have I Υ (tυ (t z(t T 1 (t u 1 (t G (tu 1 (t, (32 where G (t T 1 (t. Thus (24 can be rewritten as u(t Υ (Γ (t (tx(t+b (tθ(t +G (tu 1 (t. (33 Observation 1. The solution to FBSDEs (7-(9 is homogeneous, i.e., Θ(t, if and only if the Riccati equation (16 is regular. Proof. Necessity Assume that the Riccati equation (16 is not regular, then (I Υ (tυ (t Γ (t, (34 which implies C (t using (3. Thus according to Lemma 4, Θ(t satisfies (23 which is not equal to zero. This is a contradiction. Sufficiency The necessity follows from Theorem 3. The proof is now completed. To solve forward and backward differential equations (FB- DEs (21-(23, Denote that Θ 1 (t Θ(t P 1 (tx(t, (35 where P 1 (t obeys a Riccati-like equation associated with (21-(23 as P 1 (t+p 1 (ta (t+a (tp 1 (t+p 1 (td (tp 1 (t while Γ 1(tΥ 1 (tγ 1(t, P 1 (T, (36 Υ 1 (t, Γ 1 (t C (t+b (tp 1(t, From (35 it is clear that Θ 1 (T P 1 (Tx(T. Remark 3. It is interesting to know that P(t+P 1 (t obeys the same Riccati equation as P(t P(t+ P 1 (t+ P(t+P 1 (t A(t+A (t P(t B(tR +P 1 (t P(t+P 1 (t (tb (t P(t +P 1 (t +Q(t, with terminal value P(T+P 1 (T. Theorem 1. Problem (Deterministic IR-LQ is solvable if and only if there exists P 1 (t of (36 with terminal value P 1 (T such that Γ 1 (t, i.e., C (t+b (tp 1(t, t T, (37 and there exists u 1 (t to achieve Θ(T P 1 (Tx(T, (38 where x(t obeys ẋ(t A (t+d (tp 1 (t x(t+b (tu 1 (t, (39 with initial value x(t x. In this case, the controller u(t is given by (33. Proof. Sufficiency By taking derivative to P 1 (tx(t, it yields that dp 1 (tx(t dt P 1 (tx(t+p 1 (ta (t+d (tp 1 (t x(t+p 1 (tb (tu 1 (t A (tp 1 (tx(t Γ 1(tΥ 1 (tγ 1(tx(t +P 1 (tb (tu 1 (t A (tp 1(tx(t C (tu 1(t, (4 where (36 with Υ 1 (t and (37 has been used in the derivation of the last equality. On the other hand, using again (37, we have C (tx(t+b (tp 1 (tx(t. (41 Making comparison between (21, (22, (23 and (39, (4, (41, it is obtained that (21-(23 is solvable with Θ(t P 1 (tx(t if P 1 (Tx(T. Following Lemma 4, Problem (LQ is solvable. Necessity The proof will be completed by applying Reduction to Absurdity. Now we assume that (37 does not hold

5 5 for any terminal value P 1 (T. With similar lines to Θ(t, we have ( Θ 1 (t A (t+p 1(tD (t Θ 1 (t+ C (t +P 1 (tb (t u 1 (t, (42 By using (35, (21 and (22 are respectively rewritten as C (t+b 1P 1 (t x(t+b Θ 1 (t, (43 ẋ(t A (t+d (tp 1 (t x(t+d (tθ 1 (t +B (tu 1 (t, (44 Since (37 does not hold, i.e., C (t+b 1 P 1(t, it is easy to know that Θ 1 (t from (43. So, the control u 1 (t can not be determined at this stage. In order to solve u 1 (t, it is necessary to solve FBDEs (42-(44, we further denote that Θ 2 (t Θ 1 (t P 2 (tx(t, (45 where P 2 (t, similar to P 1 (t, obeys a Riccati-like equation associated with (42-(44 as P ( 2 (t+p 2 (t A (t+d (tp 1 (t + A (t +P 1 (td (t P 2 (t+p 2 (td (tp 2 (t, P 2 (T. (46 Now we will show that Θ 2 (t. In fact, by combining (36 and (46 we know that P 1 (t + P 2 (t satisfies the following Riccati-like equation. P 1 (t+ P 2 (t+a (tp 1 (t+p 2 (t+p 1 (t +P 2 (ta (t+p 1 (t+p 2 (td (tp 1 (t+p 2 (t. (47 It is noted that (47 is the same as (36. On the other hand, substituting (45 into (43 yields C (t+b 1 P 1 (t+p 2 (t x(t+b Θ 2(t, (48 Recall the assumption that (37 does not hold for any terminal value P 1 (T( and (47 is the same as (36, we know that C (t + B 1 P 1 (t + P 2 (t for any P 1 (T + P 2 (T. Thus Θ 2 (t from (48. Similarly, we can prove Θ i+1 (t Θ i (t P i+1 (tx(t for i 2,3, if (37 does not hold for any terminal value P 1 (T. This implies that FBDEs (21-(23 are unsolvable, and thus the Problem (Deterministic IR-LQ is unsolvable which leads to contradiction. In other words, if the problem is solvable, there will exist P 1 (t of (36 with terminal value P 1 (T such that (37 is satisfied, i.e., Θ 1 (t for t T. Now we show that there exists u 1 (t to achieve (38. Otherwise, Θ(T P 1 (Tx(T+Θ 1 (T P 1 (Tx(T, thus the problem is unsolvable from Lemma 4. The proof is now completed. Remark 4. Now the Question 1 in introduction has been answered in Theorem 1. Also, the Question 2 is answered at the same time, i.e., a irregular controller must guarantee the state terminal constraint of P 1 (Tx(T which is the essential difference from the regular LQ. Remark 5. Using Theorem 1, it is easy to know that the terminal value P 1 (T of (36 should be chosen such that C (T+B (TP 1(T. V. OPEN-LOOP AND CLOSED-LOOP SOLUTION Theorem 2. Problem (Deterministic IR-LQ is open-loop solvable if and only if there exists P 1 (t of (36 such that Γ 1 (t, and Range P 1 (t Range G 1 t,t, (49 where the Gramian matrix G 1 t,t is defined by G 1 t,t while P 2 (t,s satisfies t P 2 (t,sc (sc (sp 2 (t,sds, (5 P 2 (t,s A (tp 2 (t,s P 2 (t,t I. In this case, the open-loop solution can be given by (33 while u 1 (t is given by u 1 (t C (tp 2 (t,tg 1 t,tp 1 (t x. (51 Proof. Sufficiency : If (49 holds, i.e., Range P 1 (t Range G 1 t,t, then from the proof of Theorem 1 we have that Θ(t P 1 (tx(t ( t T. Also, if (49 holds, then for any x, there exists ζ such that where P 1 (t x G 1 t,tζ, Then it is easy to verify that ζ G 1 t,tp 1 (t x. u 1 (t C (tp 2(t,tζ C (tp 2(t,tG 1 t,tp 1 (t x, is the open-loop controller to achieve Θ(T P 1 (Tx(T. In fact, from (23, we have Θ(t t P 2 (t,sc (su 1 (sds+c, (52 where C is arbitrary constant. It is clear that Θ(T C from (52. On the other hand, by substituting u 1 (s of (51 into the above equation, we have Θ(t P 2 (t,sc (sc (sp 2(t,sds ζ +C t P 1 (t x +C, it follows that C from the fact Θ(t P 1 (t x(t, which implies that Θ(T. Thus the proof of sufficiency is now completed based on Theorem 1. Necessity If the control problem is solvable, it follows from Theorem 1 that there exists P 1 (t such that Γ 1 (t.

6 6 We now prove that (49 holds. Otherwise, Range P 1 (t Range G 1 t,t. Then there exists a nonzero vector ρ such that It is further obtained that ρ P 1 (t ρ, ρ G 1 t,tρ. ρ G 1 t,tρ ρ P 2 (t,sc (sc (sp 2(t,sds ρ t This implies that Thus, we further have ρ t C (sp 2(t,sρ 2 ds. C (sp 2(t,sρ. t P 2 (t,sc (su 1(sds. Let x ρ. From Θ(t P 1 (t x, one has Θ(t P 1 (t ρ. Combining with (52, it gives that ρ P 1 (t ρ ρ P 2 (t,sc (su 1 (sds ρ. t This is a contradiction. Thus (49 holds. The proof is now completed. We further consider the closed-loop solution. To this end, denote P 1 P 1 (t 11 (t P12 1 (t P12 1 (t P 1, 22 (t Assume that P 1 (t is singular, then there exists T 1 (t such that ˆP(t T 1(tP 1 (tt 1 (t, where ˆP(t is invertible. Let and T 1(tT 1 (t T 1(tB (tt 1 (t T1 (t T 2 (t B1 (t B 2 (t T 1(t Â1 (t A (t+d (tp 1 (t T 1 (t  2 (t. (53 Theorem 3. Assume that Γ 1 (t holds and there exists K(t such that T 1 (t+â1(t+b 1 (tt 1 (tk(tt 1(t I t T, (54 then Problem (Deterministic IR-LQ is closed-loop solvable. In this case, the closed-loop solution is given by (4 where where K(t satisfies that (54. u 1 (t K(tx(t, Proof. Denote y(t T 1(tx(t y1 (t y 2 (t, (55 then using (22 and the feedback control z(t K(tx(t, we have ẏ(t T 1 (tx(t+t 1 (A (t (tx(t+d (tθ(t +B (tk(tx(t ( T 1(tT 1 (t+t 1(t A (t+d (tp 1 (t +B (tk(t T 1 (t 1 (tx(t ( T1 (t Â1 (t B1 (t + + T 2 (t  2 (t B 2 (t K(tT 1 (t y(t T1 (t+â1(t+b 1 (tt 1 (tk(tt 1(t T 2 (t+â2(t+b 2 (tt 1(tK(tT 1 (t T By applying (54, we obtain that I ẏ1 (t t T ẏ 2 (t This yields that ẏ 1 (t I t T y 1(t. y1 (t y 2 (t By solving the above equation, it holds that This further implies that Since y 1 (t T t T t y 1 (t,. T 1 (t y(t. y 1 (T. (56 T 1 (TP 1(TT 1 (TT 1 (Tx(T ˆP(T y(t ˆP(Ty 1 (T, combining with the invertibility of ˆP(t, we have thaty 1 (T. On the other hand, note (55, y(t T 1 (Tx(T, then y 2 (T P 1 (Tx(T P 1 (TT 1 (T y 2 (T T 1 (TT 1 (TP 1(TT 1 (T ˆP(T T 1 (T The proof is now completed. y 2 (T y 2 (T

7 7 Remark 6. When P 1 (t is nonsingular, we have the following results. Problem (IR-LQ is closed-loop solvable if and only if Γ 1 (t holds and there exists K(t such that A (t+d (tp 1 (t+b (tk(t 1 I. (57 t T In this case, the closed-loop solution is given by (4 where where K(t satisfies that (57. u 1 (t K(tx(t, VI. NUMERICAL EXAMPLES Example 1: Consider the system and the cost function J(t,x ;u ẋ(t 1 1 u(t, x(t x, t ( 1 u (t u(t dt+x (Tx(T. The solution to Riccati equation (16 with P(T 1 is 1 P(t 1 t T 1. Noting that Υ (t and 1 Γ (t P(t, the LQ problem is irregular. Following 1 Theorem 1, the solvability of the irregular problem is equivalent to that (37 and (38 hold. By solving equation (36 with P 1 (T 1, it yields that P 1 (t 1 t T 1. This further implies that Γ 1 (t C (t+b (tp 1 (t P(t+P 1 (t, that is (37 holds. We then apply Theorem 2 and 3 to find the open-loop and closed-loop controller to achieve (38. Firstly, following Theorem 2 and using (4 and (51, the open-loop solution is u(t where u 1 (t u 1 (t C (tp 2 (t,tg 1 t,tp 1 (t x x(t T t. Based on Theorem 3, the closed-loop solution is u(t where u u 1 (t 1 (t x(t t T. Example 2: Consider the system and the cost function J(t,x ;u ẋ(t 1 1 u(t, x(t x, t ( 1 x (tx(t+u (t +2x (Tx(T. The Riccati equation (16 is in fact u(t dt P(t P 2 (t+1, (58 with terminal value P(T 2. The solution is P(t 3+e 2(t T 3 e 2(t T. Thus A (t P(t,B (t 1,D (t 1,C (t P(t. The Riccati equation of P 1 (t is as P 1 (t 2P 1 (tp(t P1 2 (t. (59 Based on Theorem 1, Problem (IR-LQ is solvable only if there exists an terminal value P 1 (T such P(t+P 1 (t. However, this is impossible. In fact, by substituting P(t P 1 (t into (59, it yields that P 1 (t+p 2 1 (t. Together with (58, it is obtained that P(t+ P 1 (t+p 2 1(t P 2 (t+1. It is obvious that P(t+P 1 (t 1 e2(t T 1+e 2(t T which is not zero for t < T. Thus, following Theorem 1, the problem (IR-LQ is unsolvable. VII. CONCLUSIONS By proposing the so-called two-layer optimization approach, i.e., solving the FBDEs, the irregular LQ control problem has been converted into the controllability of a specified system. Then we have presented the solution to the open-loop and closed-loop control in this paper. In more specified, the closed-loop controller is designed by solving two Riccati equations and one linear equation, while the openloop solvability condition (necessary and sufficient is checked with the Range of two matrices and the open-loop controller is designed with a Gramian matrix and two Riccati equations. The presented results in this paper have clarified the essential difference of a irregular LQ control from a regular one, i.e., a irregular controller must guarantee the state terminal constraint of P 1 (Tx(T. We have also shown that the difference of a closed-loop control from a open-loop control, i.e., a closed-loop control is to chose the controller in the current state to make P 1 (Tx(T, while the open-loop control is to seek the controller in the form of initial state to reach of P 1 (Tx(T. REFERENCES 1 W. Wonham. On a matrix Riccati equation of stochastic control. SIAM Journal on Control and Optimization, 6(4, , D. L. Kleinman, Optimal stationary contorl of linear systems with control dependent noise. IEEE Trans. Autom. Control, 14(6, pp , W. L. Levison, D. L. Kleinman and S. Baron. A model for human controller remnant. Bolt, Beranek, and Newman, Inc., Cambridge, Mass., Tech. Reppt. pp. 1731, J. Baillieul and P. Antsaklis, Control and communication challenges in networked real-time systems. Proceeding of the IEEE. vol. 95, no. 1, pp. 9-28, J. M. Bismut. Linear quadratic optimal stochastic control with random coefficient. SIAM Journal on Control and Optimization, 14(3, pp , L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, E. F. Mishchenko. The mathematical theory of optimal process. English translation. Interscience, Bellman, Richard. The theory of dynamic programming. Bulletin of the American Mathematical Society, 6(6, , B. D. O. Anderson, J. B. Moore. Optimal control: linear quadratic methods. Englewood Cliffs, NJ: Prentice Hall, F. L. Lewis, D. L. Vrabie, V. L. Syrmos Optimal control. John Wiley & Sons, Inc., B. Oksendal. Stochastic Differential Equations: An Introduction with Applications. Springer, Berlin, H. Zhang, L. Lin, J. Xu, M. Fu, Linear quadratic regulation and stabilization of discrete-time Systems with delay and multiplicative noise, IEEE Transactions on Automatic Control, 6(1, , S. Chen, X. Li, X. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs, SIAM Journal on Control and Optimization, 36(5, , 1998.

8 13 M. Ait Rami, J. B. Moore, and X. Y. Zhou, Indefinite stochastic linear quadratic control and generalized differential Riccati equation, SIAM Journal of Control and Optimization, 4, , M. Davis, Linear Estimation and Stochastic Control. Chapman and Hall, London, W. Wonham, On the separation theorem of stochastic control, SIAM Journal on Control, 6, , A. E. B. Lim and X. Y. Zhou, Stochastic optimal LQR control with integral quadratic constraints and indefinite control weights, IEEE Transactions on Automatic Control, 44, , X. Y. Zhou, D. Li, Continuous-time mean-variance portfoli selection: A stochastic LQ framework, Appl. Math. Optim., 42, 19-33, M. Kohlmann, X. Y. Zhou, Relationship between backward stochastic differential equations and stochastic controls: A linear-quadratic approach, SIAM Journal of Control and Optimization, 38, 1392C147, J. Sun, X. Li, J. Yong, Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM Journal of Control and Optimization, 54(5, , Y-C Ho, Linear stochastic singular control problems, Journal of Optimization Theory and Applications, 9(1, 24-31, H-F. Chen, Unified controls applicable to general case under quadratic index, Acta Mathematicae Applicatae Sinica, 5(1, 45-52, M. Ait Rami, X. Chen, X. Y. Zhou, Discrete-time indefinite LQ control with state and control dependent noise, Journal of Global Optimization, 23, , R. Penrose, A generalized inverse of matrices, Mathematical Proceedings of the Cambridge Philosophical Society, 52, 17-19,