2 F. BUCCI AND L. PANDOLFI to study the regularity properties of the map! V (; x ) on the interval [; T ], when x is xed. We shall consider also the m

Transcription

1 THE VALUE FUNCTION OF THE SINGULAR QUADRATIC REGULATOR PROBLEM WITH DISTRIBUTED CONTROL ACTION FRANCESCA BUCCI y AND LUCIANO PANDOLFI z Abstract. We study the regularity properties of the value function of a quadratic regulator problem for a linear distributed parameter system with distributed control action. No deniteness assumption on the cost functional is assumed. We study the regularity in time of the value function and also the space regularity in the case of a holomorphic semigroup system. Key words. value function, quadratic regulator, distributed systems AMS subject classication. 49J2. Introduction. In this paper we are concerned with a general class of nite horizon linear quadratic optimal control problems for evolution equations with distributed control and non{denite cost. More precisely, we consider the following abstract dierential equation over a nite interval [; T ], < T < +, (.) _x = Ax + Bu; x() = x 2 X; where A is the innitesimal generator of a strongly continuous semigroup e At on a Hilbert space X, B is a linear bounded operator from the control space U to X. With the dynamics (.), we associate the cost functional (.2) J (x ; u) = F (x(t); u(t))dt + hx(t ); P x(t )i; where x() = x(; ; x ; u) is the mild solution to equation (.) and F is the quadratic form (.3) F (x; u) = hx; Qxi + hx; Sui + hsu; xi + hu; Rui (we denoted by h; i inner products in both the spaces X and U). All the operators Q, S, R and P contained in the functional (.2) are linear bounded operators in the proper spaces, with Q = Q, R = R, P = P. We dene as usual the value function of the problem: V (; x ) := The goal of the present work is to characterize the property inf J (x ; u): u2l 2 (;T ;U) (.4) V (; x ) >?; 8x 2 X; 8 2 [; T ]: This research was supported by the Italian Ministero dell'universita e della Ricerca Scientica e Tecnologica within the program of GNAFA{CNR. y Dipartimento di Matematica Applicata, Universita di Firenze, Via S. Marta 3, 539 Firenze, Italy (fbucci@dma.unifi.it). z Dipartimento di Matematica, Politecnico di Torino, Corso Duca degli Abruzzi 24, 29 Torino, Italy (lucipan@polito.it), partially supported also by HCM network CEC n. ERB{CHRX{CT93{ 42.

2 2 F. BUCCI AND L. PANDOLFI to study the regularity properties of the map! V (; x ) on the interval [; T ], when x is xed. We shall consider also the map x! V (; x ) in a special case, see x6. It is well known that if the regulator problem is standard, i.e. (.5) Q ; S = ; R > ; P ; then the solution to the operator Riccati equation corresponding to problem (.){ (.2) provides the synthesis of the unique optimal control. This problem is well understood, both in nite and innite dimensions, over a nite or innite time horizon (compare [], [2, 3]). The purpose of this paper is to examine the case when (.5) fails, with special interest in non{coercive R. We shall see that in this case the function! V (; x ) has some mild regularity properties, see x4. More regularity is obtained in the coercive case, see x5. The study of LQR problems with non{denite cost is related to a large variety of problems. Among them, we recall the study of dissipative systems (see [2]), the analysis of the stability of feedback systems ([4]), the analysis of second variations of non{linear optimization problems (see [5], [5]). When game theory is studied for linear systems then the quadratic form (.3) is non{positive. In particular, the suboptimal H {problem can be recast in this setting ([]). Finally, very recently singular control theory has been used to obtain new results on regular control problems for some class of boundary control systems: systems with input delays rst [6], and later systems described by wave{ or plate{like equations with high internal damping [9]. We recall that the existing results for nite dimensional systems over an innite time interval ([9, 2], see also [4]) were extended to distributed systems in [22, 23, 2, 3, 8]. If T < + the only work we know in an innite dimensional context, in which a nonpositive cost functional is studied, is [6]. This paper considers even time{varying systems, but under the restriction R = I. 2. A simple example. The interest of the results presented in this paper is justied by the possible applications that we already quoted, for instance to H { control theory over a nite time interval, or to the analysis of the second variation of general cost functionals. However the following example may help the reader to understand our problem. The example is a bit articial, since we want to present a very simple one. Nevertheless it is suggested by non trivial problems in network theory. A delay line in its simpler form is described by an input-output relation (2.) v(t; x) = v(t + x) = Z? u(t + s + x) d(s); where t >, x 2 (?; ) and the integral is a Stieltjes integral. For simplicity we assume that the input u() is continuous, a condition that can be very much relaxed. The simplest case described by (2.) is (2.2) v(t; x) = u(t + x? )

3 THE VALUE FUNCTION OF THE GENERAL LQR PROBLEM 3 and corresponds to a jump function, with jump at?. If the system is started at t = then the input (2.) is read only for t >, so that the output v() from (2.) is given by ( (x + t)? < x + t < v(t; x) = R u(t + s + x) d(s) otherwise.? The function describes the \initial state" of the system (quite often it will be = ). In the case of equation (2.2) we have in particular ( u(t + x? ) t + x > v(t; x) = (t + x? ) t + x < : Notice that if () and u() are regular then v(t; x) = v(t + x) solves the rst order hyperbolic equation v t = v x ; v(; x) = (x); v(t; ) = u(t? ): The function v can be interpreted as a delayed potential at the output of the network produced by the potential u() at the input. If the delay line is connected to a resistive load, it produces a current i(t) = v(t? ) and the energy disspated by the load in R time T is given by Since then? i(t)v(t? ) dt =? i(t)v(t? ) dt =? v(t;?) = 8 < :? R T? R R jv(t? )j2 dt: ( (t? ) < t < u(t? ) t > ; j(t? R )j2 dt if T < j(t? R )j2 dt? R T The energy that the load can dissipate is at most inf? u() i(t)v(t? ) dt: ju(t? R )j2 dt if T > : We see from this that the load dissipates a nite amount of energy V () if T <, described by the quadratic functional (2.3) V () =? R j(t? )j2 dt: Otherwise, the load can dissipate as much energy as we want. Hence it makes sense to study the energy function E(T ) E(T ) = inf? u2l 2 (;T ) i(t)v(t? ) dt: In this example the function E(T ) is nite only if T <, and in this case E(T ) is the quadratic functional (2.3). In this paper we consider an analogous problem in more generality: we study the dependence on the interval [; T ] of the \energy" dissipated by a certain linear time invariant system.

4 4 F. BUCCI AND L. PANDOLFI 3. Preliminary results. We recall that the solution to (.) is (3.) x(t) = e A(t? ) x + (L u)(t); with (L u)(t) = e A(t?s) Bu(s) ds; (3.2) : continuous L 2 (; T ; U)! L 2 (; T ; X): Note that t! (L u)(t) is a X{valued continuous function. The adjoint L of L : hl u; fi L 2 (;T ;X) = hu; L fi L 2 (;T ;U) is given by (L f)(t) = B t e A (s?t) f(s) ds; : continuous L 2 (; T ; X)! L 2 (; T ; U): Introduce also the bounded operator from U to X L ;T u = e A(T?s) Bu(s) ds (which describes the map (3.) from the input u to the solution of (.) at time t = T, with initial time and x = ). The adjoint of L ;T is the map given by (L ;T y)(t) = B e A (T?t) y: Using (3.), one can easily show the following Lemma 3.. The cost functional (.2) can be rewritten as (3.3) J (x ; u) = hm x ; x i + 2 Re hn x ; ui + hr u; ui; with M 2 L(X), N 2 L(X; L 2 (; T ; U)) and R 2 L(L 2 (; T ; U)), M and R selfadjoint, dened as follows (3.4) M x = e A (T? ) P e A(T? ) x + e A (t? ) Q e A(t? ) x dt; (3.5) (N x)(t) = (L Q ea(? ) x)(t) + S e A(t? ) x + (L ;T P e A(T? ) x)(t); (R u)(t) = (L Q (L u))(t) + S (L u)(t) + (L Su)(t) (3.6) + Ru(t) + (L ;T P L ;T u)(t): We rst state a Lemma, which will be useful later. Lemma 3.2. If there exists and a constant such that (3.7) R I; then R I for any >.

5 THE VALUE FUNCTION OF THE GENERAL LQR PROBLEM 5 Proof. It is sucient to notice that if > we can write hr u; ui L 2 (;T ;U) = hr v; vi L 2 ( ;T ;U); where v() is given by v(t) = if t < and u(t) when t. Hence, from (3.7) it follows that R I for any 2 [ ; T ]. We shall use the following general result pertaining continuous quadratic forms in Hilbert spaces whose proof is given for the sake of completeness. Lemma 3.3. Let X and U be two Hilbert spaces, and consider with N 2 L(X; U), R 2 L(U), R = R.. If there exists x 2 X such that f(x; u) = 2RehN x; ui + hru; ui V (x) := inf f(x; u) >?; u2u then R. 2. The inmum of f(x; ) is attained if and only if the equation (3.8) Ru =?N x is solvable and in this case any solution u of (3.8) gives a minimum. 3. If for each x 2 X there exists a unique u x such that f(x; u x ) = min u f(x; u) then R is invertible (the inverse R? may not be bounded) and u x =?R? N x so that the transformation x! u x is linear and continuous from X to U. 4. Let us assume that V (x) >? for each x 2 X. Then there exists a linear bounded operator P 2 L(X) such that (3.9) V (x) = hx; P xi 8x 2 X : Proof. If there exists v such that hrv; vi < then f(x; v)!? as! +. This proves the rst item of the Lemma. The second item is well known ([23, Lemma 2.3]). To prove the third item we use item 2: the minimum u x is characterized by (3.8). This equation is uniquely solvable for every x by assumption. Hence, ker R = fg and im N im R. Consequently, u x =?R? N x where R? acts from the closure of the image of R. Hence, R? N is bounded since R? is closed and N is bounded. The proof of the fourth item follows an approach in [7]. If R is coercive, then it is boundedly invertible, so that f(x; ) admits a unique minimum, namely u + =?R? N x, and V (x) = f(x; u + ) =?hx; N R? N xi: Hence, (3.9) holds true and we have obtained an explicit expression for P, i.e. P =?N R? N : If we simply have R, we consider the function f n (x; u) = f(x; u) + n juj2 :

6 6 F. BUCCI AND L. PANDOLFI Now R n = R + I is coercive, hence n with P n 2 L(X). By construction V n (x) = min f n (x; u) = hx; P n xi; u n! hx; P n xi is a decreasing numerical sequence for any x 2 X, and (3.) V (x) hx; P n xi hx; P xi; hence there exists P 2 L(X) such that hx; P xi = lim hx; P nxi = inf hx; P nxi V (x): 8x 2 X: n!+ n To conclude, it remains to show that V (x) coincides with hx; P xi for any x 2 X. Assume by contradiction that V (x) < hx; P xi for a given x 2 X, and let > such that hx; P xi = V (x) + : Since V (x) = inf u f(x; u) there exists u 2 U such that (3.) f(x; u) < V (x) + 2 : Correspondingly, there exists an integer n 2 IN such that (3.2) f n (x; u)? f(x; u) = n juj 2 < 2 : From (3.) and (3.2) it follows V (x) hx; P n xi f n (x; u) < V (x) + ; which is a contradiction, compare (3.). The above lemma and (3.3) imply a rst necessary condition for niteness of the value function. Lemma 3.4. If there exists x such that V (; x ) >?, then (3.3) J (; u) = hr u; ui 8u 2 L 2 (; T ; U): This observation is now used to obtain a necessary condition of more practical interest, which is well known in the nite dimensional case. The symbol I denotes the identity operator acting on a space which will be clear from the contest. Proposition 3.5. If there exists 2 [; T ) and a constant such that R I, then R I. Consequently, (3.4) if there exists x and such that V ( ; x ) >? then R.

7 THE VALUE FUNCTION OF THE GENERAL LQR PROBLEM 7 Proof. We rst consider the case =, hence by assumption R. By contradiction, suppose that there exists a control u 2 U and a constant > such that hru ; u i =?. Given a small >, choose a control u as follows: u(t) = u t < T? T? t T and compute hr u; ui = T? hq T? + hru ; u i + 2 Re + T? hp T? e A(t?s) Bu ds; e A(t?s) Bu dsi dt T? T? h T? e A(t?s) Bu ds; Su i dt e A(T?s) Bu ds; e A(T?s) Bu dsi dt T? (3.5) =? + o() + o( 2 ) as tends to : Since can be taken arbitrarily small, (3.5) yields hr u; ui <, and this contradicts the assumption. Assume instead R I >. By choosing u(t) = for t 2 [ ; T? [, u(t) = u 2 U arbitrary when t 2 [T? ; T ], a direct computation yields ku k 2 hru ; u i + o( 2 )ku k 2 ; which implies hru ; u i ku k 2 for any u 2 U. Finally, if V ( ; x ) >? for some 2 [; T ) and x 2 X, then from Lemma 3.4 it follows that R is a non-negative operator for. Therefore from the previous part of the proof, R. We now show that the value function satises the Bellmann's optimality principle which is known, in the context of linear{quadratic problems, as \Linear Operator Inequality" (LOI) or \Dissipation Inequality" (DI). We begin with the following Lemma 3.6. If for some number and some x 2 X we have V (; x ) >?, then we have also V (t; x(t)) >? for each t 2 [; T ]. Here, x(t) denotes the value at time t of the function given by (3.), for any xed control u() on [; t]. Proof. Let t 2 (; T ). Then J (x ; u) = F (x(s); u(s)) ds + F (x(s); u(s)) ds t + hx(t ); P x(t )i = F (x(s); u(s))ds + J t (x(t); u); where x() = x(; ; x ; u) for any u 2 L 2 (; T ; U). Take now a control v on [; t): then J (x ; u + v) = F (x(s); u(s)) ds + J t (x(t); u + v)

8 8 F. BUCCI AND L. PANDOLFI and (3.6) inf J (x ; u + v) = F (x(s); u(s)) ds + inf J t(x(t); u + v): v v Conclusion immediately follows since in fact inf v J t (x(t); u + v) = V (t; x(t)). Theorem 3.7. Let 2 [; T ] and x 2 X be given. Let V be the value function of problem (.), (.2) and assume that V (; x ) >?. Then (3.7) F (x(s); u(s)) ds + V (t; x(t))? V (; x ) ; for any u() 2 L 2 (; T ; U) and any t 2 (; T ), with x() = x(; ; x ; u). Moreover, the equality holds true if and only if the control u in(3.7) is optimal. Proof. We return to the conclusion of the preceding Lemma, and observe again that (3.8) while (3.9) inf v J t(x(t); u + v) = inf u J t(x(t); u) = V (t; x(t)); inf v J (x ; u + v) V (; x ); hence plugging (3.8) into (3.6) and taking into account (3.9), we get (3.2) V (; x ) F (x(s); u(s))ds + V (t; x(t)); which is nothing but (3.7). Thus, if for a given initial datum x there exists an optimal control u + (; ; x ) minimizing J (x ; u), then we can rewrite (3.6) and (3.9) with u = u + (; ; x ), and (3.9) is in fact an equality. Therefore (3.2) becomes an equality as well. For these arguments compare also []. Viceversa, assume that (3.7) is satised for any control u 2 L 2 (; T ; U) and it is an equality for a given u. Then, passing to the limit, as t! T?, in (3.7) with u = u and x = x = x( ; ; x ; u ) and assuming for the moment that (3.2) we readily get that is V (; x ) = lim t!t? V (t; x (t)) = hx (T ); P x (T )i; F (x (s); u (s)) ds + hx (T ); P x (T )i; V (; x ) = J (x ; u ); hence by denition u is optimal. To conclude, it remains to show that if (x ; u ) satises (3.22) F (x (s); u (s)) ds + V (t; x (t))? V (; x ) = ;

9 THE VALUE FUNCTION OF THE GENERAL LQR PROBLEM 9 then (3.2) holds true. From (3.22) it follows that there exists lim V (t; x (t)) = V (; x )? F (x (s); u (s)) ds; t!t? and by the very denition of the value function it follows To see this rewrite the above limit as lim V (t; x (t)) = V (; x )? t!t? lim t!t? V (t; x (t)) hx (T ); P x (T )i: + hx (T ); P x (T )i: By contradiction, assume now that F (x (s); u (s)) ds? hx (T ); P x (T )i lim t!t? V (t; x (t)) = hx (T ); P x (T )i? ; where is a suitable positive constant. Then, there exists > such that (3.23) V (t; x (t)) < hx (T ); P x (T )i? 2 ; for any t 2 (T? ; T ). Recall now that hence we can rewrite x (T ) = x(t; ; x ; u ) = x(t; t; x (t); u jst ); hx (T ); P x (T )i = he A(T?t) x (t); P e A(T?t) x (t)i {z } A + 2 Re + t hu (s); B e A (T?s) P e A(T?t) x (t)i ds {z } A 2 Take a possibly smaller, in order to get t hu (s); B e (T?s)A P L t;t u i ds : {z } A 3 (3.24) A 2 + A 3 < 4 ; so that (3.23) yields (3.25) V (t; x (t)) < A? 4 : Finally, let such that (3.26) t hqe A(s?t) x (t); e A(s?t) x (t)ids < 8 :

10 F. BUCCI AND L. PANDOLFI Fix now t 2 (T? ; T ), so that (3.24) and (3.26) hold true. From (3.25) it follows that there exists a control v = v t 2 L 2 (t; T ; U) such that that is, by means of (3.3), J t (x (t); v t ) < A? 4 ; hm t x (t); x (t)i + 2 Re hn t x (t); v t i + hr t v t ; v t i < A? 4 ; with M t, N t, R t dened in (3.4), (3.5) and (3.6), respectively. We know that hm t x (t); x (t)i is A + R T h e A (s?t) Q e A(s?t) x (t); x (t)i ds. Thus we cancel the t term A, we take into account (3.26) and we obtain (3.27) 2 Re hn t x (t); v t i + hr t v t ; v t i <? 8 : In particular this implies that v t 6=. Notice now that jhn t x (t); v t ij (T? t) jv t ()j L 2 (t;t ;U); jhr t;t v t ; v t ij const jv t ()j 2 L 2 (t;t ;U) ; and therefore lim inf t!t? jv t()j L 2 (t;t ;U) = > (possibly = +). Hence there exists a sequence t n such that so that we see from (3.27) for n large jv tn ()j 2 hn t n x (t); v tn i! : jv tn ()j 2 hr t n v tn ; v tn i < : In other words J tn (; v tn ) < and this is a contradiction since by assumption J (; u) is non-negative for any u 2 L 2 (; T ; U). The next Proposition is an immediate consequence of Lemma 3. and of Lemma 3.3. We omit the proof. Proposition 3.8. Let 2 [; T ]. If (3.28) V (; x ) >? 8x 2 X; there exists a selfadjoint operator W () 2 L(X) such that W (T ) = P and (3.29) V (; x ) = hx ; W ()x i: 4. Time regularity of the value function: the non{coercive case. In this section we investigate the regularity properties of V (; x ) with respect to the initial time. We note that several regularity results are known for the value function even of non{linear systems, and with more general cost but under special boundedness properties, which are not satised in the present case, compare [, Ch. 6].

11 THE VALUE FUNCTION OF THE GENERAL LQR PROBLEM Our rst result is: Lemma 4.. Let 2 [; T ] and x be such that V ( ; x ) is nite. Then! V (; x ) is upper semicontinuous at. Proof. Fix x 2 X, and let 2 [; T ]. In order to show that lim sup! V (; x ) V ( ; x ); we shall show that for any real number > V ( ; x ) we have > V (; x ), if j? j is taken small enough. We rst consider the case when >. Let u be an admissible control such that (4.) J (x ; u) < ; and dene It is readily veried that x (t) = e A(t? ) x + e A(t?s) Bu(s) ds: : lim x (T ) = x (T );! 2: lim! so that if j? j is small enough, F (x (s); u(s)) ds = V (; x ) J (x ; u) < : F (x (s); u(s)) ds Finally, if <, choose once more u 2 L 2 ( ; T ; U) in such a way that (4.) holds true. It is now sucient to repeat tha same arguments used before, after replacing u with ^u dened as follows: ( < ^u(t) := u(t) t : The proof is complete. As to lower semicontinuity, the following result holds true. Lemma 4.2. Let x be such that! V (; x ) is nite on [; T ]. Then the map! V (; x ) is lower semicontinuous at provided that for each element n of a sequence f n g which tends monotonically to there exists a control u n 2 L 2 ( n ; T ; U) such that (4.2) i) V ( n ; x ) J n (x ; u n ) V ( n ; x ) + n ; ii) there exists > for which ju n ()j L 2 ( n;t ;U) : Proof. Let 2 [; T ] be given, and consider a sequence f n g n2in such that n #. Introduce the inputs ^u n (t) := ( < t < n u n (t) t n :

12 2 F. BUCCI AND L. PANDOLFI and dene x n (t) := x(t; n ; x ; u n ); ^x n (t) := x(t; ; x ; ^u n ): Notice that x n (t)? ^x n (t)!, as n!, for any t, and that its norm is uniformly bounded in L 2 ( ; T ; U), hence Therefore lim n! [J (x ; ^u n )? J n (x ; u n )] = : lim inf n! J (x ; ^u n ) = lim inf n! J n (x ; u n ) = lim inf n! V ( n; x ); where the last equality is due to i). On the other hand ii) implies the existence of an admissible control v 2 L 2 ( ; T ; U) such that as n!. Now the map ^u n * v u()! J (x ; u) is convex continuous, hence weakly lower semi-continuous, so that (4.3) V ( ; x ) J (x ; v) lim inf n! J (x ; ^u n ) = lim inf n! V ( n; x ): To conclude the proof, we need to consider a sequence fr n g n2in such that r n ". In this case, we introduce urn (t) t ~u rn (t) := otherwise. Again from ii) it follows that there exists an input v 2 L 2 ( ; T ; U) such that ~u rn * v in L 2 ( ; T ; U). A similar argument gives which nally yields V ( ; x ) lim inf n! V (r n; x ); V ( ; x ) lim inf!? V (; x ): Consequently, we can conclude Theorem 4.3. Under the same assumptions as Lemma 4.2, the map! V (; x ) is continuous for any 2 [ ; T ]. In the case that an optimal control exists for each near, Lemma 4.2 takes a simpler form. We state this form, under the assumption that an optimal control exists for each. Corollary 4.4. Let x 2 X be xed. Assume that i) for any 2 [ ; T ] there exists an optimal control u + 2 L 2 (; T ; U); ii) there exists a constant >, independent of, such that (4.4) ju + j L 2 (;T :U) 8 2 [ ; T ]:

13 THE VALUE FUNCTION OF THE GENERAL LQR PROBLEM 3 Under these conditions, the map! V (; x ) is continuous. We note explicitly that if there exists an optimal control u for J (x ; v) then for each > there exists an optimal control for J (x( ; ; x ; u ); v). It has some interest to see that if the operator A generates a strongly continuous group then we can prove more: Theorem 4.5. Let us assume that for each 2 [; T ) and each x 2 X there exists a unique optimal control u + (; ; x ) which minimizes J (x ; u). If e At is a strongly continuous group then the value function is continuous from the right. Proof. We prove continuity from the right at a xed 2 [; T ). We know from Lemma 3.3 item 2 that x! u + (; x ) is linear and continuous from X to L 2 (; T ; U), for each 2 [; T ). Now we consider points >. We show that for each xed > there exists x = x (x ) such that (4.5) u + (; ; x (x )) j[;t ] = u+ (; ; x ) : It is sucient to see for this that there exists a solution x of (4.6) x + (; ; x ) = e A(?) x + Z e A(?s) Bu + (s; ; x ) ds = x : If this is true, unicity of the optimal control shows that (4.5) holds. We noted above that ku + (; ; x )k L 2 ( ;T ) Mkx k so that the norm of the operator T x = Z e A(?s) Bu + (s; ; x ) ds can be estimated as follows: kt x k (? )Mkkx k. We write Eq. (4.6) in the form (4.7) x + e?a(?) T x = e?a(?) x : If? is suciently small, ke?a(?) T k is less then hence Eq. (4.7) can be continuously solved for x and gives a linear continuous transformation x = x (x ) which, of course, depends upon. The vector x, x = x (x ) = [I + e?a(?) T ]? e?a(?) x is continuous with respect to x and also with respect to if is close to. In particular,! x (x ) is bounded in a neighborhood of. Therefore, ku + (; x )k = ku + (; x ) j[;t ] k L 2 (;T ) Right continuity follows from Lemma 4.2. ku + (; x )k L 2 ( ;T ) c kx (x )k kx k : The previous theorem presents a case in which the quite involved condition of Lemma 4.2 is satised. The next example shows that the condition in that lemma cannot be avoided if we are to obtain continuity of the value function. We note rst that the value function is not continuous in general, even for nite dimensional systems: if the cost is jx(t )j 2 and the system is controllable then the

14 4 F. BUCCI AND L. PANDOLFI value function has a jump at T. The following example shows that the value function may be discontinuous even at points < T. Example 4.6. Consider the delay system given by (4.8) ( _x = y(t? ) _y = u(t) with initial datum =col[x ; y ()] 2 IR L 2 (?; ). The quadratic functional is J ( ; u) = Z 2 jx(t)j 2 dt + 3jx(2)j 2 : Take =col(; ). When 2 [; 2], then y(t? ) =, hence x(t) = on [; 2]. Consequently In particular J ( ; u) = 4 and J ( ; u) = (2? ) + 3 8u ; 8 2 [; 2] : V (; ) = 4: On the other hand, if 2 [; [, then y(t? ) 6= when t > +, and it can be arbitrarily xed, by means of suitable choices of the control u, within the class of W ;2 functions which are zero at t = +. This set is dense in L 2 ( + ; 2), hence suitable functions y can be found in order to drive x(t) to zero in time >, namely from + to + +, while remaining uniformly bounded. Therefore we have that x(t) = in (; + ), and Z x 2 (t)dt! as!. In conclusion, if <, V (; ) = and the value function is not continuous at =. Remark 4.7. The previous example shows that in the statement of Lemma 4.2 which concerns lower semicontinuity of V (; x ) assumption ii) cannot be dispensed with. In fact that assumption holds in the previous example for! +, but not for!?. 5. Time regularity of the value function: the coercive case. Let ^ 2 [; T ] be given, and consider the operator R^ as dened in (3.6). Throughout this section we shall assume that R^ is coercive, i.e. (5.) 9 > : R^ : Our present goal is to show that under assumption (5.) the value function V (; x ) displays better regularity properties with respect to. We start by showing that the map! V (; x ) is continuous for any 2 [^; T ], with x 2 X xed.

15 THE VALUE FUNCTION OF THE GENERAL LQR PROBLEM 5 We recall that from (5.), by virtue of Lemma 3.2, it follows that R for any 2 [^; T ], and by continuity also on an interval ( ; T ] [^; T ]. Hence there exists a constant such that (5.2) kr? k 8 2 ( ; T ] [^; T ]: Moreover (5.) implies that for any initial time 2 [^; T ] there exists a unique optimal control u + (t) = u + ( ; ; x ) ( u + () for short ), explicitly given in terms of the initial state by (5.3) u + (t; ; x ) =?? R? (N x )() (t); (compare item 3 of Lemma 3.3); and from (5.2) it follows (5.4) ju + ()j L 2 (;T ;U) k^ jx j; with k^ independent of : The following Theorem provides a simple explicit expression of the value function in terms of the optimal pair which will be useful in the next section. Theorem 5.. Let R be coercive, and let (x + (; ; x ); u + (; ; x )) the optimal pair for problem (.)-(.2). Then W ()x = e A (T? ) P x + (T; ; x ) (5.5) + e A (t? )? Qx + (t; ; x ) + Su + (t; ; x ) dt: Proof. Since the inmum of the cost is attained at u + (; ; x ) (u + plugging (5.3) into (3.3) we easily obtain for short), (5.6) The adjoint operator N W ()x = M x + N u+ : : L 2 (; T ; U)! X maps any L 2 (; T ){function v in N v = e A (T? ) P L ;T v + e A (t? ) ((Q L + S)v) (t) dt; hence (5.5) follows from (5.6) by a direct computation. As a consequence of Corollary 4.4, we rst have Theorem 5.2. Let x 2 X be given. Assume that(5.) is satised. Then! V (; x) is continuous on [^; T ]. Actually we are able to show that the value function satises a further regularity property. Before we state a preliminary result. is coercive. If w() is a continuous function, then Lemma 5.3. Assume that R^ the function (5.7) s! (s) := (R? w)(s) is continuous for any ^. In particular, if R is coercive then the optimal control is continuous. Proof. Since R^ is coercive, R is coercive, so that we can assume that R = I. Moreover, for any > ^, R is coercive, hence invertible.

16 6 F. BUCCI AND L. PANDOLFI Let (t) := (R? w)(t), with w() continuous: we know that () is at least an U{ valued L 2 function. But (t) = w(t)? B t Z s e A (s?t) Q e A(s?r) B(r) dr ds? S e A(t?s) B(s) ds? B t e A (s?t) S(s) ds? B e A (T?t) P e A(T?s) B(s) ds; and the second hand side is apparently an U{valued continuos function. The second statement follows from (5.3) since (N x )() is a continuous function, compare (3.5). Theorem 5.4. Let x 2 D(A) be given. Assume that (5.) is satised. Then the map! V (; x) is dierentiable in [^; T ]. Proof. Let x 2 D(A) and let u + = u + (; ; x ) the unique optimal control of problem (.)-(.2), ^. As in (5.6) (5.8) V (; x ) = hx ; W ()x i = hm x ; x i + hn x ; u + i ; with M and N given by (3.4), (3.5), respectively. From the very denition of M it readily follows that the x ; x i exists for any x 2 D(A). In order to show that the the second summand in (5.8), namely (5.9) (N x )(t) u + (t) dt; is dierentiable with respect to, we rst observe that the factor (N x )() is dierentiable, (N x )(t) =?(N (Ax ))(t): Moreover, again from (3.5) it follows that (5.) is a continuos function. We next want to show that for each t > the U{valued function! u + (t) admits rst derivative with respect to and that this is continuous. Fix and consider rst the case >. Introduce the operator ^N 2 L(X; L 2 ( ; T ; U)) dened as follows: By construction and for instance ( ^N x )(t) = ( (N x )(t) t 2 [; T ] (N x )(t) t 2 [ ; [ ^N x jt N x ; R? (N x ) = R? ( ^N x ):

17 THE VALUE FUNCTION OF THE GENERAL LQR PROBLEM 7 Moreover, we take into account (5.) and we see that (5.) lim! + ( ^N x )(t)? (N x )(t)? =?N Ax 8x 2 D(A): In fact it is sucient to observe that ( t 2 [ ; [ ( ^N x )(t)? (N x )(t) = (N x )(t)? (N x )(t) t 2 [; T ]: Now we compute, via (5.3),? (u +? u + )(t) = (5.2) =? R?? (N x )() (t)?? R? (N x )() (t) =?R? " ( ^N x? N x )()? # [R (t) + R?? R ] R?? ( ^N x )(t): The rst summand in (5.2) tends (5.3)? R? (N x )() (t) =? (N Ax )() (t) =?u + (t; Ax ); when! +, due to (5.). As to the second summand, it can be rewritten in the following way: [R R?? R ] R?? ( ^N x )(t) = = R? B t e A (s?t) Q Z e? A(s?r) B(R? ^N x )(r)dr ds {z } a(;s) Z +R? S e? A(t?s) B(R? ^N x )(s)ds Z +R? B e A (T?t) P e? A(T?s) B(R? ^N x )(s)ds = : We rewrite, in turn, Z a(; s) = e? A(s?r) B(R? N x )(r)dr {z } + Z b(;s) e A(s?r) B R ^N? x? N x ( )?! (r)dr : {z } c(;s)

18 8 F. BUCCI AND L. PANDOLFI Observe now that as a consequence of Lemma 5.3 we have lim! + while lim! + b(; s) = e A(s?) B(R? N x )( ) =?e A(s?) Bu + ( ; x ); c(; s) =, hence a(s) := lim! + a(; s) =?e A(s?) Bu + ( ; x ): Finally, since (; s)! a(; s) is bounded, we can conclude that converges to?r? B t e A (s?t) Qe A(s?) B u + ( ; x )ds as tends to +. The convergence of the terms 2 and 3 can be proved even more easily. If < we dene instead ( ^N x )(t) = and rewrite the term R? N? R? N as ( (N x )(t) t 2 [ ; T ] (N x )(t) t 2 [; [ R? (N? ^N ) + (R?? R? ) ^N = = R? (N? ^N ) + R? (R? R )R? ^N : The rest of the proof is completely similar. Therefore we have proved that for each u+ (t) u+ (t) = u + (t; Ax )? R? B t e A (s?t) Qe A(s? ) Bu + (; x ) ds?r? S e A(t? ) Bu + (; x )? R? B e A (T?t) P e A(T? ) Bu + (; x ): In conclusion we saw that the function (N x )(t) u + (t) is dierentiable with respect to, and moreover its derivative is a continuous function in [^; T ] [^; T ]. Therefore (5.9) is dierentiable, and T (N x )(t) u + (t) dt x )()u + ()? (N Ax )(t)u + (t) dt + (N u+ (t) dt: We are now able to deduce a dierential form of the Dissipation Inequality. Proposition 5.5. Assume that (5.) holds true. Then there exists a selfadjoint operator W () 2 L(X) such that i) W (T ) = P ;

19 ii) W () satises (5.4) THE VALUE FUNCTION OF THE GENERAL LQR PROBLEM 9 d ha; W () ai + 2RehAa + Bv; W ()ai + F (a; v) d for any (a; v) 2 D(A) U, for any 2 [^; T ]. Proof. We x a 2 D(A), v 2 U, and take a control u() 2 C ([; T ]; U) such that u() = v. We dene x(t) = x(t; ; a; u). It is well known (see for instance [2]) that in this case x is a strict solution to (.), that is x 2 C ([; T ]; X) \ C([; T ]; D(A)) and it satises (.) on [; T ]. We write the dissipation inequality (3.7) for (x(t); u(t)), t 2 [; T ], namely (5.5) F (x(s); u(s))ds + hx(t); W (t)x(t)i? hx(); W ()x()i : If we divide in (5.5) by t? and let t!, we have d ds hx(); W (s)x()i + 2RehAx() + Bu(); W ()x()i + F (x(); u()) : js= To conclude, substitute x() = a and u() = v. We proved that if we replace an optimal pair in the left hand side of the dissipation inequality in integral form, then we get an equality. Hence we get an equality also in the dierential form (5.4). In particular, we x a 2 dom A and we see that u + (; ; a) is a minimum of the left hand side of inequality (5.4). Hence we nd that u = u + (; ; a) satises Ru + S a + B W ()a = : Since R is coercive then R is coercive too and we see that the optimal control has the well known feedback form u = u + (; ; a) =?R? [S + B W ()] a (if a 2 D(A) and, by continuity, for each a 2 X, see item 3 of Lemma 3.3). Moreover, as u + (t; ; a) = u + (t; t; x + (t; ; a)), the previous equality gives the feedback form of the optimal control on the interval [; T ]. We replace this expression for the unique optimal control in the left hand side of (5.4) and we nd a quadratic dierential equation for W () which is the usual Riccati equation. Of course, the Riccati equation can be written provided that R? is a bounded operator. But, an example in [6] shows that if R is not coercive then the minimum of the cost may exist and be unique, in spite of the fact that the corresponding Riccati equation is not solvable on [; T ]. 6. Space regularity of the value function. This section is devoted to the study of some space regularity properties of the value function in the case that the optimal control problem is driven by an abstract equation of parabolic type. See [7] for analogous arguments. More precisely, we shall make the following assumption: H: A is the generator of an analytic semigroup e ta on X. It is well known (see for instance [8]) that in this case there exists a! 2 IR such that the fractional powers (!I? A) are well dened for any 2 (; ), and moreover there exist constants M, such that the following estimates hold true (6.) kt (!I? A) e At k L(X) M e t ; t > :

20 2 F. BUCCI AND L. PANDOLFI For the sake of simplicity we assume that the semigroup is exponentially stable, i.e. that we can choose! =. We associate the following output to system (.): y = Cx + Du where y belongs to a third Hilbert space Y and C 2 L(X; Y ), D 2 L(U; Y ). We assume that the cost penalizes the output y i.e. that the quadratic functional F in (.3) is given by F (x; u) = kyk 2 Y + hu; R ui so that Q = C C, S = C D, R = R + D D. (A special and important case is D = ). We assume: H2: R ; P. We now use similar arguments as in Lemma 3.3. Introduce a regularized optimal control problem with cost given by (6.2) J ;n (x ; u) = J (x ; u) + n kuk2 L 2 (;T ;U) ; n 2 IN and observe that since the operator R n = R + I is coercive for each n, then there n exists a unique optimal control u + n and V n (; x ) := inf u J ;n(x ; u) = J ;n (x ; u + n ) = hx ; W n ()x i: Arguing as in the proof of statement 4 in Lemma 3.3 we know that W n ()x! W ()x 8x 2 X: Let x + n () = x n (; ; x ; u + n ) and y n + () = Cx + n ()+Du + n (). Then we have the following Lemma 6.. Let such that (?A ) C 2 L(X) and assume that there exists a number 2 (; + ) such that 2 (6.3) Then there exists a constant c such that (?A ) P =2 2 L(X): (6.4) ky + n ()k 2 L 2 (;T ) + kp =2 x + n (T )k 2 X c k(?a)? x k 2 8n 2 IN: Proof. The estimate is easily obtained as follows (note that 2 (A) since we assumed! = ): ky + n (t)k 2 dt + kp =2 x + n (T )k 2 J ;n (x ; u + n ) J (x ; u + ) J (x ; ) = kc(?a) k 2 M 2 + kp =2 (?A) k 2 k(?a)? x k 2 : kce A(t? ) x k 2 dt + kp =2 e A(T? ) x k 2 T?2(?)? 2(? ) k(?a)? x k 2

21 THE VALUE FUNCTION OF THE GENERAL LQR PROBLEM 2 Remark 6.2. We stress that since T c = max kc(?a) k 2?2(?) M 2 =2 ; kp (?A)? 2(? k 2 ; ) the estimate (6.4) is uniform with respect to n and. Lemma 6.3. Under the same assumptions of Lemma 6. there exists a constant k such that (6.5) k(?a ) W n ()(?A) k k 8n 2 IN: Proof. Let 2 X. We recall that since by construction the operator R ;n relative to J n ( ; u) is coercive for each xed n, then the regularized control problem admits a unique optimal pair (x + n (; ); u + n (; )), and Theorem 5. yields W n () = e A (T? ) P x + n (T; ) + e A (t? ) C y + n (t; ) dt: The regularity assumptions on C and P imply that W n () 2 D((?A ) ) and (?A ) W n () = e A (T? ) [(?A ) P =2 ][P =2 x + n (T; )] +(?A )? e A (s? ) [(?A ) C ][y + n (s; )] ds: Now, as a consequence of (6.4) there exists k such that k(?a ) W n () k k k(?a)? k uniformly in n. Conclusion follows immediately by choosing = (?A) x with x 2 D((?A) ). Consequently we have the following Theorem 6.4. Under the same assumptions of Lemma 6. the operator (?A ) W ()(?A) admits a bounded extension to X for any < + 2. REFERENCES [] T. Basar and P. Bernhard, H {Optimal Control and Related Minimax Design Problems. A Dynamic Game Approach, Birkhauser, Boston, 99. [2] A. Bensoussan, G. Da Prato, M. C. Delfour and S. K. Mitter, Representation and Control of Innite Dimensional Systems, vol. I, Birkhauser, Boston, 992. [3], Representation and Control of Innite Dimensional Systems, vol. II, Birkhauser, Boston, 993. [4] S. Bittanti, A. J. Laub and J. C. Willems, eds., The Riccati equation, Springer{Verlag, Berlin, New York, 99. [5] D. J. Clements and B. D. O. Anderson, Singular Optimal Control: the Linear{Quadratic Problem, Lecture Notes in Control and Information Sci. 5, Springer Verlag, Berlin, 978. [6] B. Jacob, Linear quadratic optimal control of time{varying systems with indenite costs on Hilbert spaces: the nite horizon problem, J. Math. Systems, Estimation and Control, 5 (995), pp. {28.

22 22 F. BUCCI AND L. PANDOLFI [7] E. A. Jonckheere and L. M. Silverman, Spectral thoery of the linear quadratic optimal control problem: discrete-time single-input case, IEEE Trans. Circuits and Systems CAS{ 25 (978), pp. 8{825. [8] B. van Keulen, Equivalent conditions for the solvability of the nonstandard LQ{Problem for Pritchard{Salamon systems, SIAM J. Control Optim., 33 (995), pp. 326{356. [9] I. Lasiecka, L. Pandolfi and R. Triggiani, A singular control approach to highly damped second-order abstract equations and applications, in Control of Partial Dierential Equations, E. Casas ed., M. Dekker, New York, 995, pp. 57{69. [] I. Lasiecka and R. Triggiani, Dierential and Algebraic Riccati Equations with Application to Boundary/Point Control Problems: Continuous Theory and Approximation Theory, Lecture Notes in Control and Information Sci. 64, Springer Verlag, Berlin, 99. [] X. Li and J. Yong, Optimal Control Theory for Innite Dimensional Systems, Birkhauser, Basel, 995. [2] A. L. Likhtarnikov and V. A. Yakubovich, The frequency theorem for equations of evolutionary type, Siberian Math. J., 7 (976), pp. 79{83. [3] J.-Cl. Louis and D. Wexler, The Hilbert space regulator problem and operator Riccati equation under stabilizability, Ann. Soc. Sci. Bruxelles, Ser. I, 5 (99), pp. 37{65. [4] A. I. Lur'e, Some nonlinear problems in the theory of automatic control, H.M. Stationery Oce, London, 957. [5] B. P. Molinari, Nonnegativity of a quadratic functional, SIAM J. Control Optim., 3 (975), pp. 792{86. [6] L. Pandolfi, The standard regulator problem for systems with input delays: an approach through singular control theory, Appl. Math. Optim., 3 (995), pp. 9{36. [7], Singular regulator problem for holomorphic semigroup systems, Politecnico di Torino, Dipartimento di Matematica, Rapporto interno n. 28, 994. [8] A. Pazy, Semigroups of Linear Operators and Applications to Partial Dierential Equations, Springer Verlag, Berlin, 983. [9] J. C. Willems, Least squares stationary optimal control and the algebraic Riccati Equation, IEEE Trans. Automat. Contr., AC{6, (97), pp. 62{634; a correction in On the existence of a nonpositive solution to the Riccati Equation, ibidem, AC{9, (974), pp. 592{593. [2], Dissipative dynamical systems. Part I: General theory, Arch. Rational Mechanics and Analysis, 45 (972), pp. 32{35; Part II: Linear systems with quadratic supply rates, Arch. Rational Mechanics and Analysis, 45 (972), pp. 352{392. [2] V. A. Yakubovich, The frequency theorem in control theory, Siberian Math. J., 4 (973), pp. 384{49. [22], A frequency theorem for the case in which the state and control spaces are Hilbert spaces with an application to some problems in the synthesis of optimal controls, I, Siberian Math J., 5 (974), pp. 457{476. [23], A frequency theorem for the case in which the state and control spaces are Hilbert spaces with an application to some problems in the synthesis of optimal controls, II, Siberian Math J., 6 (975), pp. 828{845.