Linear-quadratic control problem with a linear term on semiinfinite interval: theory and applications

Linear-quadratic control problem with a linear term on semiinfinite interval: theory and applications L. Faybusovich T. Mouktonglang Department of Mathematics, University of Notre Dame, Notre Dame, IN 46556 USA 15 December 3 Abstract We describe a complete solution of the linear-quadratic control problem on a semiinfinite interval with the linear term in the objective function. Some applications are considered. 1 Introduction In this paper we consider a standard linear-quadratic control problem on the semiinfinite interval. The only difference is that the problem has an extra linear term in the cost function which makes it time-dependent. The major motivation for this extension comes from [FT] where we consider applications of very efficient primal-dual interior-point algorithms to the computational analysis of multi-criteria linear-quadratic control problems in minimax form. To compute a primal-dual direction it is necessary to solve several linear-quadratic control problems with the same quadratic and different linear parts in the performance index. Another natural motivation comes from the multi-target linear-quadratic control problem (which is, of course, a particular case of multi-criteria LQ problem but admits a much simpler solution than the general problem [FM]). Surprisingly, the solution of LQ problem with an extra term is quite simple and described here in the standard L setting in full generality. Main Result Denote by L n [, ) the Hilbert space of square integrable functions f : [, + ) R n. This paper is based upon work supported in part by National Science Foundation Grant No. 168 1

Let Y = L n [, ) L m [, ), X = {(x, u) Y : ẋ = Ax + Bu, x is absolutely continuous,x() = }, A is an n by n matrix and B is an n by m matrix; Let Σ11 Σ Σ = 1 Σ 1 Σ be a symmetric (m + n) (m + n) matrix. Σ 11 in an n by n matrix and Σ is an m by m matrix; (y, v) Y. Consider the following linear-quadratic control problem: J(x, u) = 1 x x <, Σ > dt (1) u u + [< y, x > + < v, u >] dt min () ẋ = Ax + Bu, x() = x (3) Here x R n is a fixed vector and <, > is the standard scalar product in a finite-dimensional Euclidean space. We assume that (x, u) is in an affine subspace with the linear part equal to X. To describe the solution of (1)-(3) we need the following result. Theorem 1 Let A be an antistable n by n matrix ( i.e. real parts of all eigenvalues of A are positive). Consider the following system of linear differential equations: ẋ = Ax + f, (4) where f L n [, ). There exists a unique solution L(f) of (4) such that L(f) L n [, ). Moreover the map f L(f) is linear and bounded. Explicitly: L(f)(t) = e Aτ f(τ + t)dτ (5) Proof Since A is antistable, there exist a positive definite symmetric matrix H such that A T H + HA = I, (6) where I is the identity matrix (see e.g. [Leonov]). Let x be any solution to (4). Consider W (t) =< x(t), Hx(t) >. Then, using (6), we easily obtain: dw dt = x(t) + < x(t), Hf(t) >,

where x(t) =< x(t), x(t) >. Using Cauchy-Schwarz inequality, we obtain dw dt x(t) x(t) Hf(t) x(t) x(t) H f(t), where H is the Matrix norm induced by the Euclidean norm on R n. It is quite obvious that and hence, x(t) H f(t) 1 x(t) + H f(t) dw dt x(t) Integrating (7) from to t, we obtain: H f(t). (7) Consequently, W (t) W () x(τ) dτ H f(τ) dτ. Let now x(τ) dτ 4 H f(τ) dτ + [W (t) W ()] (8) x(t) = e Aτ f(τ + t)dτ = One can easily see that x is the solution to (4) such that x() = One can easily see from (9) that x(t) e Aτ dτ t e Aτ f(τ)dτ e A(τ t) f(τ)dτ (9) f(τ) dτ for any t. Hence x and consequently W are bounded on [, + ). Using (8), we conclude that x(t) dt < i.e. x(t) given by (9) is in L n [, + ). Since x is a solution to (4), we conclude that ẋ L n [, + ) and hence x(t), t + (see appendix for the proof). But then using (8) again, we conclude that x(τ) dτ 4 H f(τ) dτ W () 3

(4 H + H )( f(τ) dτ + C e Aτ dτ f(τ) dτ) f(τ) dτ (1) for some constant C. Observe now that (4) may have only one solution in L n [, + ). Indeed, if x 1 and x are two such solutions and x = x 1 x, then x L n [, + ) and ẋ = Ax. Since A is antistable and x L n [, + ), we immediately conclude that x. We can conclude now by (1) that the linear map f L(f) is bounded. Consider the following linear-quadratic control problem: 1 x x <, Σ > dt min (11) u u x X (1) u As is well-known, the following algebraic Riccati equation plays the crucial role in the description of optimal solution to (11),(1): Here KLK + Kà + ÃT K Q = (13) à = A BΣ 1 Σ 1, Q = Σ 11 Σ 1 Σ 1 Σ 1, L = BΣ 1 BT. (14) Observe that (13) is defined under the assumption that Σ is invertible. Recall, that a symmetric solution K = K T to (13) is called stabilizing if the matrix à + LK is stable. the following result is a well-known (for a concise proof see e.g [Lancaster],[Faybusovich]). Theorem The following conditions are equivalent: i) Σ is a positive definite (symmetric) matrix and (13) has a stabilizing solution. ii) The pair (A, B) is stabilizable and there exists ɛ > such that Γ(x, u) = for all (x, u) X. [ x < u ] [ x, Σ u ] > dt ɛ ( x + u )dt Remark. The condition ii) means that the quadratic functional Γ(x, u) is strictly convex on X. 4

Theorem 3 Suppose that the pair (A, B) is stabilizable. Then X is a closed vector subspace in Y. Let ṗ + A Z = { T p B T : p L n p [, + ), p is absolutely continuous, ṗ L n [, + )} Then Z is an orthogonal complement of X in Y. (i.e. Z = X ) Proof Let (x, u) X, and (ṗ + A T p, B T p) Z. We are going to show that x ṗ + A α = <, T p u B T > dt = p Indeed, α = = = [< x, ṗ + A T p > + < u, B T p >]dt [< Ax + Bu, p > + < x, ṗ >]dt d dt < x, p > dt = lim < x(t ), p(t ) >. T + Observe that x() =, since (x, u) X. But lim T x(t ) = lim T p(t ) =, since both x and p are in L n [, + ), absolutely continuous and are such that ẋ L n [, + ), ṗ L n [, + ) (see Appendix). Hence, α =. We next show that any (φ, ψ) Y admits the following representation: [ φ ψ ] [ x = u ] [ ṗ + A T p B T p ] (15) with (x, u) X and (ṗ+a T p, B T p) Z. This will easily imply that Z = X and both X and Z are closed. If Σ is the identity matrix, Theorem easily implies that the corresponding algebraic Riccati equation (13) has a stabilizing solution K. Observe that in this case à = A, Q = I n, L = BB T. We can rewrite (15) in the form: ṗ = x A T p φ (16) We also have Substituting (17) into (18), we obtain u = B T p + ψ (17) ẋ = Ax + Bu, x() = (18) ẋ = Ax + BB T p + Bψ. (19) We are looking for the solution to (16),(19)in the form. p = K x + ρ, () 5

where K is the stabilizing solution to the algebraic Riccati equation (13). Substituting () into (16),(19), we obtain: ẋ = Ax + BB T K x + BB T ρ + Bψ, (1) K ẋ + ρ = x A T K x A T ρ φ. () Finally, substituting (1) into (), we obtain i.e. (K A + A T K + K BB T K I)x + ρ = A T ρ φ K Bψ K BB T ρ, ρ = (A T + K BB T )ρ φ K Bψ. (3) Since the matrix A + BB T K is stable, the matrix (A + BB T K ) is antistable hence we can apply Theorem 4. (Observe that φ K Bψ L n [, + )) Thus (3) possesses a unique solution ρ L n [, + ), ρ = L(φ + K Bψ). (4) Reversing our reasoning, we see that if ρ is defined as in (4), x is defined as (1) with x() =, and p is defined as in (), we then obtain the representation (15). We are now in position to describe a solution of the LQ-problem on a semiinfinite interval. Theorem 4 Suppose that the conditions of theorem are satisfied. Then the problem (1)-(3) has a unique solution which can be described as follows. There exists a stabilizing solution K to the Riccati equation (13). Then the matrix C = (Ã + LK ) is antistable, (K B Σ 1 )Σ 1 v + y Ln [, + ). Let ρ be a unique solution from L n [, + ) of the system of differential equations ρ = C T ρ + (K B Σ 1 )Σ 1 v + y (which exists according to Theorem 1); x is the solution to the system of differential equations ẋ = (Ã + LK )x + Lρ BΣ 1 v, x() = x, p = K x + ρ, u = Σ 1 (BT p v Σ 1 x). Remark. Observe that by (5) we have the following explicit description of ρ: ρ(t) = e CT τ ((K B Σ 1 )Σ 1 v + y)(t + τ)dτ Sketch of the proof Since conditions of Theorem are satisfied, we know that the functional in (1)-(3) is strictly convex, the matrix Σ is positive definite and the algebraic Riccati equation (13) has (a unique) stabilizing solution K. 6

The necessary and sufficient optimality condition for (1)-(3) obviously takes the form: x y Σ + X, (x, u) statifies (3). u v Or using the description of X from Theorem 3: Σ 11 x + Σ 1 u + y = ṗ + A T p, Σ 1 x + Σ u + v = B T p for some p satisfying condition of Theorem 3. We are looking for p in the form: p = K x + ρ, where K is the stabilizing solution to the algebraic Riccati equation (13). We finish the proof exactly as in Theorem 3. Remark. The extension of Theorem 4 to the discrete time case is pretty straightforward. 3 Some Applications Consider, first, the tracking problem: [< x φ, x φ > + < u ψ, u ψ >]dt min, (5) ẋ = Ax + Bu, x() = x. (6) Here (φ, ψ) Y. It is quite clear that (5),(6) is equivalent to 1 [< x, x > + < u, u >]dt [< x, φ > + < u, ψ >]dt min, ẋ = Ax + Bu, x() = x and hence can be solve using Theorem 4. Observe that for x =,(5),(6) is the problem of finding the orthogonal projection of (φ, ψ) Y onto the closed subspace X. In [FM] we have considered the following minimax problem. Let (V, <, > V ) be a Hilbert space, T its closed vector subspace, v, v 1,..., v l be vectors in V. Consider: max v v i min, (7) 1 i l v v + T. (8) 7

In case, V = Y, T = X, we arrive at multi-target linear-quadratic control problem on a semiinfinite interval. We have shown in [FM] that the optimal solution of this problem is contained in the convex hull W = conv IR (π T v 1,, π T v m ). Hence, it has the form: v opt = m i=1 µ opt i π T (v i ), where µ opt m i, i=1 µopt = 1. Finding µ opt i can be reduced to solving a finite-dimensional second-order cone programming. Indeed, let m v(µ) = µ i π T v i We have v(µ) v i = i=1 v(µ) π T v i + ν i, where ν i = π T v i is the norm of the orthogonal projection of the vector v i onto the orthogonal complement T of T in Y. Furthermore, v(µ) π T v i = µ T (i)γ µ(i), where µ j (i) = µ j forj i, µ i (i) = µ i 1 and Γ = ( π T v i, π T v j ). Let Γ = B T B be the Cholesky decomposition of Γ. Then v(µ) π T v i = B µ = Bµ b i, where b i = Be i the i-th column of B. Hence, Bµ bi v(µ) v i = ν i and we can rewrite the original problem (7),(8) in the following equivalent form: t min, (9) Bµ bi ν t, i = 1,,..., m, (3) i µ IR m. (31) (See [FM] for details.) The problem (9)- (31) is the second order cone programming problem which can be easily solved using the standard interiorpoint software. Here π T stands for the orthogonal projector of V onto T. In [FM], we have considered linear-quadratic control problem on a finite interval. In the present paper we described the procedure of calculating π X on a semiinfinite interval. Thus we can solve a multi-target LQ problem on semiinfinite interval. Using Theorem 4, one can easily see that for finding π T v i, i = 1,,..., m, it suffices to solve an algebraic Riccati equation K A + A T K + K BB T K I = only once. 8

4 Appendix Let H n 1 ([, + )) = {x L n ([, + )), is absolutely continuous and ẋ L n ([, + ))}. This is one of the standard Sobolev spaces. Then the proof of the following Lemma is obtained e.g. from a proof of a similar result in [Leonov]. Lemma 1 Let x H n 1 ([, + )). Then Proof We have: < x(τ), ẋ(τ) > dτ lim x(t) = t + Since, x(t), ẋ(t) L n ([, + )), we have: is finite. But have x(τ) dτ + lim < x(τ), ẋ(τ) > dτ t + ẋ(τ) dτ < x(τ), ẋ(τ) > dτ = 1 x(t) 1 x(). Hence, the limit lim t + x(t) exists, and since x(t) dt <, we References lim x(t) =. t + [Leonov] G.A. Leonov, Mathematical Problems of Control Theory, World Scientific, 1. [Lancaster] P. Lancaster and L. Rodman, Algebraic Riccati Equations, Oxford Science Publications, 1995. [Faybusovich] L. Faybusovich, Algebraic Riccati Equation and Symplectic Algbra, International Journal of Control, vol.43, pp. 781-79 (1986). [FM] L. Faybusovich and T. Mouktonglang, Multi-target linear quadratic control problem and second-order cone programming (to appear in System and Control letters) [FT] L.Faybusovich and T.Tsuchiya, Primal-dual Algorithms and Infinitedimensional Jordan Algebras of finite rank, Math. programming, Ser. B, vol. 97(3), pp. 471-493. 9