Linear Riccati Dynamics, Constant Feedback, and Controllability in Linear Quadratic Control Problems

Size: px

Start display at page:

Download "Linear Riccati Dynamics, Constant Feedback, and Controllability in Linear Quadratic Control Problems"

Mildred Higgins
5 years ago
Views:

1 Linear Riccati Dynamics, Constant Feedback, and Controllability in Linear Quadratic Control Problems July 2001 Ronald J. Balvers Douglas W. Mitchell Department of Economics Department of Economics P.O. Box 6025 P.O. Box 6025 West Virginia University West Virginia University Morgantown, WV Morgantown, WV Phone: (304) Phone: (304) ABSTRACT Conditions are derived for linear-quadratic control (LQC) problems to exhibit linear evolution of the Riccati matrix and constancy of the control feedback matrix. One of these conditions involves a matrix upon whose rank a necessary condition and a sufficient condition for controllability are based. Linearity of Riccati evolution allows for rapid iterative calculation, and constancy of the control feedback matrix allows for time-invariant comparative static analysis of policy reactions. JEL classification: C61 Keywords: Controllability, Riccati Equation, Linear Quadratic Control.

2 Linear Riccati Dynamics, Constant Feedback, and Controllability in Linear Quadratic Control Problems 1. Introduction This paper presents some conditions for linear evolution of the Riccati matrix, for constancy of the control-feedback matrix, and for controllability or lack thereof, in the standard linear quadratic control (LQC) problem. Computational aspects and simplification of the LQC problem have figured prominently in the recent literature on dynamic linear economies [see Amman (1997), Anderson et al. (1997), Anderson and Moore (1985), Binder and Pesaran (2000), Blanchard and Kahn (1980), Ehlgen (1999), Klein (2000), Ljungqvist and Sargent (2000), and Sims (2000)]. Linearity of Riccati evolution allows for rapid iterative calculation, and constancy of the control feedback matrix allows for time-invariant comparative static analysis of policy reactions. 2. The control problem A typical statement of the finite horizon linear quadratic control problem is: Min T (1a) V(y s, s) ', {u t } T ' y ) t K y t s%1 t's%1 (1b) subject to y t ' A y t&1 % C u t, t ' s%1,..., T, y s given, where K and A are n x n, C is n x k, y t is n x 1, and u t is k x 1. The cost matrix K is positive definite, and C has full column rank; the transition matrix A need not have full rank. If the original problem statement has control costs, one can augment the state vector with the costed

3 controls [see Chow (1975)], putting all costs on the state vector and thus giving the problem formulation in equations (1). It is well known [Chow (1975)] that the optimal controls are given by: (2) t ' & (C ) H t C) &1 C ) H t A y t&1 / &F t y t&1, t # T, u opt (3) H t&1 ' K % A ) H t A & A ) H t C (C ) H t C) &1 C ) H t A, H T ' K, t # T, where the symmetric n x n matrix H t is positive definite. The nonlinear dynamic matrix equation (3) is the Riccati equation. The system is said to be controllable if and only if the state vector can be driven from any value to any value within n periods. The well-known necessary and sufficient condition for controllability [e.g., Anderson and Moore (1979), (1990), and Söderström (1994)] is that rank ( C AC A 2 C... A n&1 C ) ' n. 3. A Preliminary Lemma Write C ' C 1 C 2 with C 1 being q x k where q / n - k, and with C 2 being k x k. Since C was assumed to be of full column rank, there is at least one k x k sub-matrix of C that is invertible. Proper prior arrangement of the y t vector (and concomitant arrangement of C, A, and K ) is thus sufficient to guarantee that C 2 is invertible. Define the n x q matrix M as M / I n&k, so &(C ) 2 )&1 C ) 1 2

4 that M ) C ' 0. We can now prove the following identity: LEMMA (IDENTITY RE-EXPRESSING A FUNCTION OF C IN TERMS OF M). For C (which is n x k with full column rank) and M (which is n x q (q = n - k) with full column rank) such that M ) C ' 0, and D any (symmetric) positive definite n x n matrix, the following identity holds: (4) D & DC(C ) DC) &1 C ) D ' M(M ) D &1 M) &1 M ). Proof. Consider a Cholesky decomposition such that D ' Q ) Q, with Q full rank (such a decomposition exists for any positive definite matrix). Then we can premultiply equation (4) by (Q ) ) &1, postmultiply by Q &1, and rearrange to obtain (5) I & (Q ) ) &1 M(M ) D &1 M) &1 M ) Q &1 ' QC(C ) DC) &1 C ) Q ). Note that both sides of equation (5) express symmetric idempotent matrices. Define the n x n matrix that Z / I & (Q ) ) &1 M(M ) D &1 M) &1 M ) Q &1 & QC(C ) DC) &1 C ) Q ). It is easy to confirm ZQC ' 0, and Z(Q ) ) &1 M ' 0. Hence, defining the n x n matrix Y / Y 1! Y 2 / QC! Q )&1 M, we have: (6) Z Y ' 0 n, n. 3

5 Next show that Y has full rank: (i) Q has full rank n and C has full column rank k, so Y 1 = QC has rank k; (ii) Q has full rank n and M has full column rank q, so Y 2 = (Q ) ) &1 M has rank q; (iii), so all the k columns of Y 1 are Y ) 2 Y 1 / [(Q) ) &1 M] ) QC ' M ) Q &1 QC ' M ) C ' 0 q, k independent of all the q columns of Y 2. Given (i), (ii), and (iii), Y has full rank. Thus, from equation (6), Z = 0 implying that by the definition of Z equation (5) must hold; but equation (5) is equivalent to equation (4) in the statement of the Lemma. 4. Conditions for Linear Riccati Evolution, Constant Feedback, and Controllability Apply the Lemma to the Riccati equation (3), setting D ' H t which is in fact positive definite. This yields the following alternative statement of the Riccati equation: (7) H t&1 ' K % A ) M (M ) H &1 t M) &1 M ) A, H T ' K, t # T. This allows us to obtain Theorem 1: THEOREM 1 (LINEAR DYNAMICS OF H t ). Linear evolution of H t, and constancy of the control feedback matrix F t for all t # T&1, are both implied by each of (a) M ) AC ' 0, and (b) M ) AK &1 M ' 0. More strongly, (b) implies constancy of H t. Proof. (a) Post-multiply equation (7) by C. With M ) AC ' 0 this gives H t&1 C ' KC for all t # 4

6 T. Using this equation and its transpose in equations (2) and (3) gives F t ' (C ) K C) &1 C ) K A, and (8) H t&1 ' K % A ) H t A & A ) K C (C ) KC) &1 C ) K A, H T ' K, t # T. (b) Use a standard matrix inversion identity [e.g. Söderström (1994), pp ] on equation (7) to obtain (9) H &1 t&1 ' K&1 & K &1 A ) M (M ) H &1 t M % M ) A K &1 A ) M) &1 M ) A K &1, H T ' K, t # T, and post-multiply this by M to obtain H &1 t&1 M ' K&1 M when M ) AK &1 M ' 0. Using this in equation (7) gives H t&1 ' K % A ) M (M ) K &1 M) &1 M ) A which is constant for t # T. Equation (2) with constant H t&1 gives constant F t&1 for t # T. Linear evolution of H t in equation (8) gives obvious computational advantages. Constancy of the control feedback matrix F t implies that comparative static analysis of policy reactions can be conducted in straightforward fashion, with the results not dependent on time to horizon. This Theorem also implies a result for stabilizability [Ljungqvist and Sargent (2000), p.61]: 5

7 COROLLARY (SUFFICIENT CONDITION FOR STABILIZABILITY). M ) AK &1 M ' 0 is sufficient for stabilizability, and moreover, under this condition optimal stabilization drives the state variables to their target values of 0 in two periods. The Corollary follows trivially from the immediate constancy of matrix is. For period T the cost H t&1 H T ' K; for period T-1 it equals its steady state value; and with more than two periods to the horizon, the cost is unchanged; thus y t is driven to its desired value 0 in two periods. For the latter to occur, two things are necessary: A and C, and hence M, must be such that it is feasible to do so; and K must be such that it is desirable to do so. The corollary shows that the condition M ) AK &1 M ' 0 meets both of these criteria. THEOREM 2 (RELATION OF (a) Full row rank of M ) AC TO CONTROLLABILITY). M ) AC implies controllability; (b) M ) AC ' 0 implies, but is not implied by, lack of controllability. Proof. (a) Consider a canonical form of the decision problem in equations (1) with y ( t ' A ( y ( t&1 % C( u t such that C () ' (0 I k ) and M () ' (I q 0). Any possible form of the problem in equations (1) can be put in this canonical form by setting A ( ' T &1 AT, I C ( ' T &1 C, and y ( q C 1 I q &C 1 C &1 t ' T &1 y t, where T ' and T &1 2 '. It is easy 0 k,q C 2 0 k,q C &1 2 6

8 to check that rank ( C ( A ( C ( A (2 C (... A (n&1 C ( ) ' rank ( C AC A 2 C... A n&1 C ), full row rank (n) of which is equivalent to controllability (so controllability of the canonical problem is equivalent to controllability of the original problem). Now M ) AC ' M () T &1 T A ( T &1 TC ( A ( ' M () A ( C ( ' A (, where we partition A * 1 A ( 2 2 as with A ( being q x k. Thus we A ( 3 A ( 2 4 need to show that, if rank(a ( 2 ) ' q, the system is controllable. (Of course this requires q # k -- 0 A ( equivalently, n # 2k.) Let W ( / [ C ( A ( C ( A (2 C (... A (n&1 C ( 2... ] =. I k A ( 4... Clearly full row rank of A ( 2 (' M ) AC) implies full row rank of W *. So the canonical problem and hence the original problem are controllable. (b) We show that M ) AC ' 0 implies rank ( C AC A 2 C... A n&1 C ) < n, where the latter defines noncontrollability. Consider again the canonical form referred to above. Since A ( 2 = M ) AC which now equals zero, this directly implies lack of controllability for the canonical problem (sub-vector y ( 1t cannot be controlled directly since C ( 1 ' 0, nor indirectly by controlling y ( 2t since A ( 2 ' 0 ). As shown above, lack of controllability of the canonical problem implies lack of controllability of the original problem. Part (b) further asserts that lack of controllability need not imply that M ) AC ' 0. We 7

9 show this by example: Let 2k < n and take A to be idempotent and such that M ) AC ' A ( 2 0. Then the system is noncontrollable since rank ( C AC A 2 C... A n&1 C ) ' rank ( C AC AC... AC ) # 2k < n, since C and AC each have k columns. 1 Note that M ) AC ' 0 does not imply lack of stabilizability. We can show this by counterexample: let A ' ai n, in which case M ) AC ' a M ) C ' 0 since M ) C ' 0 by construction of M. If a < 1the state equation is stable in the absence of time-varying control, and hence trivially is stabilizable A numerical example is: A ' 0 0 &1, C ' 0, M '

10 References Amman, H. M., 1997, Numerical methods for linear-quadratic models, in H.M. Amman, D.A. Kendrick, and J. Rust, eds., Handbook of Computational Economics (North-Holland, Amsterdam). Anderson, B. D. A. and J. B. Moore, 1979.Optimal filtering (Prentice Hall, Englewood Cliffs). Anderson, B. D. A. and J. B. Moore, Optimal control: linear quadratic methods (Prentice Hall, Englewood Cliffs). Anderson, E., L. P. Hansen, E. R. McGrattan and T. Sargent, Mechanics of forming and estimating dynamical linear economies, in H.M. Amman, D.A. Kendrick, and J. Rust, eds., Handbook of Computational Economics (North-Holland, Amsterdam). Anderson, G. and G. Moore, A linear algebraic procedure for solving linear perfect foresight models, Economics Letters 17, Binder, M. and M. H. Pesaran, Solution of finite-horizon multivariate linear rational expectations models and sparse linear systems, Journal of Economic Dynamics and Control 24, Blanchard, O. J. and Kahn, C. M., The solution of linear difference models under rational expectations, Econometrica 48, Chow, G. C., Analysis and control of dynamic economic systems (Wiley, New York). Ehlgen, J., A nonrecursive solution method for the linear-quadratic optimal control problem with a singular transition matrix, Computational Economics 13, Klein, P., Using the generalized Schur form to solve a multivariate linear rational expectations model, Journal of Economic Dynamics and Control 24, Ljungqvist, L. and T. J. Sargent, Recursive macroeconomic theory (MIT Press, Cambridge). Sims, C. S., 2000, Solving linear rational expectations models, Working paper, Yale University. Söderström, T., Discrete-time stochastic systems: estimation and control (Prentice-Hall, New York). 9

Linear Riccati Dynamics, Constant Feedback, and Controllability in Linear Quadratic Control Problems

Linear Riccati Dynamics, Constant Feedback, and Controllability in Linear Quadratic Control Problems July 2001 Revised: December 2005 Ronald J. Balvers Douglas W. Mitchell Department of Economics Department