Online Solution of State Dependent Riccati Equation for Nonlinear System Stabilization

Transcription

1 > REPLACE American HIS Control LINE Conference WIH YOUR PAPER IDENIFICAION NUMBER (DOUBLE-CLICK HERE O EDI) FrC3. Marriott Waterfront, Baltimore, MD, USA Jne 3-Jly, Online Soltion of State Dependent Riccati Eqation for Nonlinear System Stabilization ansel Ycelen, Stdent Member, IEEE, Arn Shekar Sadahalli, Stdent Member, IEEE, and Farzad Porboghrat, Senior Member, IEEE Abstract A nmber of comptational methods have been proposed in the literatre for synthesizing nonlinear control based on state-dependent Riccati eqation (SDRE). Most of these methods are nmerically comple or depend on correct initial conditions. his paper presents a new and comptationally efficient online method for the design of stabilizing control for a class of nonlinear systems based on state-dependent Riccati eqation sing a gradient-type neral network. Moreover, the proposed network is proven to be stable. he efficacy of this approach is demonstrated throgh illstrative eamples for the proof of concept. I. INRODUCION State-dependent Riccati eqation (SDRE) has appeared in many techniqes for stabilization of nonlinear systems []- [3]. Althogh there eist a nmber of other methods for stabilization of nonlinear systems, SDRE based techniqes are among the few sccessfl approaches that have important properties, sch as applicability to a large class of nonlinear systems, allowing the control designer to make tradeoff between control effort and state errors, and its systematic formlation. It is well known that the soltion of the SDRE cannot be fond analytically, ecept for a very limited nmber of nonlinear systems. In [3], it is given that aylor series and interpolation methods can be sed to approimate the offline soltion of the SDRE. However, it is hard to find the soltion of the SDRE with these methods when the dynamics of the nonlinear system become comple and/or of high-order. herefore, a fast online comptation method for the SDRE soltion is clearly reqired. One proposed method for online soltion of the SDRE is given in [4] sing Hamiltonian and Kleinman algorithm. As it is known, Hamiltonian algorithm reqires Schrdecomposition, which is not always possible to perform when the nonlinear system dynamics are comple. On the other hand, Kleinman algorithm is a reliable approach for SDRE soltion. However, this method is given for linearqadratic type SDRE, and needs to be reformlated for other types of SDRE soltions (i.e. for H type SDRE and/or. Ycelen is with the School of Aerospace Engineering, Georgia Institte of echnology, Atlanta, GA 333 ( tansel@gatech.ed). A. S. Sadahalli is with the Department of Electrical and Compter Engineering, Sothern Illinois University at Carbondale (SIUC), Carbondale, IL USA ( arnsid@si.ed). F. Porboghrat is with the Department of Electrical and Compter Engineering, Sothern Illinois University at Carbondale (SIUC), Carbondale, IL USA ( por@si.ed). linear-qadratic type SDRE that is robst to parameter ncertainties). he more recent online soltion to SDRE is proposed by Imae et al. [5] and [6]. Qasi-Newton method in [5] depends on correct selection of the initial conditions, which is generally not a trivial task. In [6], an iterative algorithm is sed for SDRE soltion. However, this relatively comple approach especially reqires a fastsampling period to avoid system instability dring SDRE iteration. Varios theoretical developments have been made in recent years regarding the SDRE-based nonlinear statefeedback control and otpt-feedback control with asymptotic stability and convergence properties [3]. More importantly, a variety of sccessfl implementations of the SDRE-based control approach have been reported in [5]-[8]. It is indeed significant that the SDRE approach is applicable to aviation systems [8], despite their nstable dynamics. In this paper, the soltion of the state-dependent Riccati eqation (SDRE)-based nonlinear control design problem is given sing gradient-type neral networks. his techniqe is an etension of a method for the soltion of algebraic Riccati eqations (ARE) introdced in [9] and [3]. he proposed approach is relatively fast, comptationally simple, and is applicable to all SDRE-based control methods, inclding H type SDRE control. he proposed approach is demonstrated throgh illstrative eamples, one with eact soltion, for the prpose of comparison and the proof of concept. he paper is organized as follows. Section II presents an overview of the SDRE approach, while section III presents a gradient-type neral network for on-line soltion of SDRE. Simlation stdies are given in section IV. Finally, conclsions are smmarized in section V. II. OVERVIEW OF SAE-DEPENDEN RICCAI EQUAION APPROACH In this section, we consider the problem of otpt feedback stabilizing control design for nonlinear systems, sing state-dependent Riccati eqation. For this prpose, consider a smooth nonlinear system of the form [], [] = f( ) + g ( ) = y = h ( ), =,..., p m where =[,, n ] R n is the vector of state variables in a n smooth state-space manifold denoted by M R. Also = [,..., ] m m R is the inpt vector and () //$6. AACC 6336

2 > REPLACE HIS LINE WIH YOUR PAPER IDENIFICAION NUMBER (DOUBLE-CLICK HERE O EDI) y y y p = [,..., ] R p is the otpt vector. It is assmed that there eists an eqilibrim state M. Moreover, it is assmed that the system is both controllable and observable [3]. For simplicity of notation, eqation () is written as = f( ) + g( ) () y = h( ) where f( ) =. he idea behind the method is to etend the applicability of the algebraic Riccati eqation (ARE) for the control design of linear systems to a class of nonlinear systems () that can be epressed in a state-dependent linear form, as = A ( ) + B ( ) (3) y = C( ) he following sbsections will define the nonlinear otpt feedback control problem in two parts; state feedback control and state estimation; de to the fact that the separation principle is demonstrated (see [3]). A. State Feedback Control Assme that all the state variables are available for feedback. he goal is to find a state feedback control law of the form =-K() that minimizes a cost fnction given by (, ) ( J = Q+ R ) (4) t where n n Q R is a symmetric positive semi-definite mm matri, and R R is a symmetric positive definite matri. Moreover, Q is a measre of control accracy and R is a measre of control effort []. It shold be noted that the SDRE formlation allows one to tradeoff between the control accracy and control effort, which is a property not generally fond in other nonlinear control design methods [7]. o minimize the above cost fnction, a statefeedback control law can be given as ( ) = K ( ) = R B( P ) ( ) (5) where P() is the niqe, symmetric, positive-definite soltion of the state-dependent Riccati eqation of the form A() P () + PA () () + Q PBR () () B() P () = (6) However, the selection of A() and B() is not niqe [3]. o choose the one of the correct parameterizations for A() and B(), one shold consider the following remark. Remark. From many possible choices, matrices A() and B() mst be chosen in sch a way that system (3) is controllable or at least stabilizable. Now consider the state-dependent controllability matri ψ ( ) ( ) ( ) ( ) ( ) n C = B AB A ( B ) ( ) (7) he nonlinear system in (3) is said to be controllable if n and only if rank[ ψ C ( )] = n, for all M R, [3]. he system is said to be stabilizable if its ncontrollable modes are stable. Once a sitable choice for A() and B() is fond, there always eists a control law (5) that makes the closedloop system asymptotically stable. B. State Estimation Assme that not all the state variables are available for feedback. he goal is to design a state estimator that minimizes a cost fnction given by J ( ˆ) = lim Ε ( ˆ) ( ˆ) (8) t { } where ˆ is the estimated state vector that can be fond from ˆ = A ( ˆ) ˆ BK ( ˆ) ( ˆ) ˆ+ L ( ˆ) y C ( ˆ) ˆ (9) ( ) so that the estimation error is asymptotically stable at the zero eqilibrim [3]. Minimizing (8), L() can be given as L ( ˆ) = SC ( ˆ) ( ˆ) Θ () where S() is the niqe, symmetric, positive-definite soltion of the state-dependent Riccati eqation of the form AS ()() ˆ ˆ + SA () ˆ () ˆ +Ξ SC () ˆ () ˆ Θ CS ()() ˆ ˆ = () where Ξ=E(ww ) and Θ=E(vv )> are symmetric covariance matrices, corresponding to the white noise vectors w and v affecting the state and otpt eqations, respectively, whose vales are to be presmed. he selection of Ξ and Θ matrices determines a tradeoff between the estimation accracy and correction effort for state estimation problem similar to the state feedback control problem. It is worth mentioning that finding the optimal estimator gain () is eqivalent to finding the optimal state feedback gain for a dal state-dependent linear system of the form (3) where matrices A(), B(), Q, and R are replaced by matrices A (), C (), Ξ, and Θ, respectively. Similar to reqiring stabilizability condition in remark, the selection of matrices A() and C() mst satisfy the detectability conditions stated in Remark for the state estimation problem. Remark. Matrices A() and C() mst be chosen so that system (3) is observable or at least detectable. Now consider the state-dependent observability matri ( n ) ψ O ( ) = C ( ) C ( ) A ( ) C ( ) A ( ) () he nonlinear system (3) is observable if and only if n rank[ ψ O( )] = n, for all M R, [3]. he system is said to be detectable if its nobservable modes are stable. Once, a sitable choice for A() and C() is fond, there always eists an asymptotically stable state estimation law (9)-(). C. Otpt Feedback Control he otpt feedback control law can be achieved by combining the state feedback control law and the state estimation process. In Banks et al. [3], it is shown that the compensated system is locally asymptotically stable, as depicted in the following theorem. heorem. Consider the system in (), sch that f() and f(x)/ i (i=,,n) are continos in for all ˆr, ˆr >, and assme that one can write f()=a(), g()=b(), and h()=c(). Assme frther that A(), B(), and C() are continos. If A(), B(), and C() are chosen sch that the 6337

3 > REPLACE HIS LINE WIH YOUR PAPER IDENIFICAION NUMBER (DOUBLE-CLICK HERE O EDI) 3 pair (A(),C()) is detectable and (A(),B()) is stabilizable for all M, then ( ˆ,e)=(,) for the system (3) is locally asymptotically stable, where e= - ˆ. Proof. See the proof of heorem 5. in [3]. herefore, there is a need to solve two independent statedependent Riccati eqations to realize the nonlinear otpt feedback stabilization problem. he online comptation method for these SDREs is given in the following section. III. ONLINE GRADIEN-YPE NEURAL NEWORK SOLUION OF SAE-DEPENDEN RICCAI EQUAION In this paper, the soltion of state-dependent Riccati eqation is given sing gradient-type neral network for nonlinear controller realization. he proposed approach is the nonlinear etension of a method developed in [9] and [3] for the soltion of algebraic Riccati eqation (ARE) for the LI systems. he following sbsections describe the proposed online comptation algorithm. A. Online Comptation Algorithm Here, the aim is to find the soltions of the SDREs (6) and (), reqired for state feedback control synthesis and state estimation design, respectively. For this prpose, consider the following generalized SDRE that one needs to solve M ( ) V( ) + V( ) M( ) V( ) N( ) V( ) + O( ) = (3) where V(), M(), N() and O() respectively correspond to P(), A(), B()R - B () and Q for feedback control synthesis, and S(), A (), C ()Θ - C(), Ξ for state estimation problem. We know that V() mst be positive definite and symmetric. However, it is known that eqation (3) has a niqe soltion which is positive definite and symmetic if it has a Cholesky factorization [3]. ha is, to reqire GV ( ( ), L ( )) = [ g ] = LL ( ) ( ) V ( ) =, k, =,..., n (4), k where g,k is the k th element of the obective fnction G, and L() is a Cholesky factor for V(). In order to solve (3) for a stabilizing control law, let s define the cost fnction G( V( )) = [ g, k ] = M ( ) V( ) + V( ) M( ) + O( ) V( ) N( ) V( ) =,, k =,..., n (5) where g,k is the k th element of the obective fnction G. o solve for V() from (4) and (5), the following Lyapnov energy fnction is first derived [9], n n E[ G ( V( ), L( )), G( V( )) ] = g, k + g, k (6) i= = hen, a matri-oriented gradient algorithm is developed to find the pdate rle for V() by changing the variables in the direction of the negative gradient of the energy fnction E to minimize (6), as dv ( ) E = n (7) V V ( ) dl( ) E = n (8) L L( ) herefore, the pdate law can be given as [9], [3], dv ( ) = nv[ M( ) Ψ +Ψ M ( ) +Ψ N( ) V( ) Ψ ΨV( ) N ( )] (9) dl( ) = nl[ Ψ L( ) ] () where n V and n L are positive scalar learning factors, and Ψ ( V ()) =I M () V () + VM () () + O () VNV () () ()] () Ψ ( V( ), L ( )) = I LL ( ) ( ) V( ) () where I is a symmetric non-decreasing activation fnction. ypical eamples of I are thoroghly eamined in [3]. Here, for simplicity, the activation fnction is selected as I ( f ( )) = f( ) (3) It is given in [9] that to ensre positive definiteness of V() in steady state, one generally reqires the matri L() to converge faster than matri V(). In other words, the learning factor n L shold be greater than the learning factor n V. he architectre of the gradient-type neral network consists of two bidirectionally connected layers, where (), () act as hidden layers, and (9), () act as otpt layers [9], [3]. he proof of stability and convergence properties of this method are given in the following sbsections, which etend the proofs given in [9], [3], [4] for LI systems to the case of state-dependent psedo-linear systems. B. Stability and Solvability Parallel to [4], we applied vec operation to compte E/ t. It is important to note here that (6) is a valid positive definite Lyapnov fnction if and only if E. E = de / is given as [ ] de G ( V( ), L( )), G( V( )) E dvecv ( ) E dvecl( ) = + vecv ( ) vecl( ) Using (7) and (8), (4) can be rewritten as [ ] (4) de G( V( ), L( )), G( V( )) dvecv() dvecv() dvecl() dvecl() = (5) nv nl n n dv i dli = + i= = n V nl where (5) shows that E. herefore, the gradient type neral network algorithm is asymptotically stable. Solvability property of the proposed algorithm can be given similar to [3] as follows. heorem. he steady state matri of the gradient type neral network algorithm always gives the symmetric and positive definite soltion to the SDRE from any symmetric initial state V(), and any non-zero initial state L(), if and only if rank(v()n()-m())=n and all diagonal elements of 6338

4 > REPLACE HIS LINE WIH YOUR PAPER IDENIFICAION NUMBER (DOUBLE-CLICK HERE O EDI) 4 L() are non-zero. Proof. See the proof of heorem in [3]. IV. SIMULAION SUDIES In this section, one illstrative eample, which has analytical soltion for SDRE state feedback optimal control design, is given first for comparison prposes. hen, a second eample is given for SDRE otpt feedback control design sing the proposed online comptation algorithm. A. Eample he first eample is a scalar system from [5] that has an eact soltion for the SDRE state feedback control problem. For this problem, we compare or reslts not only with st eact soltion, bt also with Kleinman algorithm [4] and Qasi-Netwon algorithm [5]. For this prpose, consider the following cost fnction to be minimized (, ) = ( + ) t J (6) where Q=, R=, and the associated nonlinear dynamics is 3 = + (7) A direct state-dependent parameterization of this nonlinear state dynamics can be given as = ( ) + () (8) A ( ) B ( ) where the eact SDRE soltion for (8), that is P() in (6) or eqivalently V() in (3), is given by 4 V( ) = ( ) + + (9) For the proposed algorithm, and for the other algorithms as well, the initial SDRE soltion is set to V()=.. Initial state vector is selected as ()=4. Figres -3 and able I present the reslts. From Figre, all three algorithms are able to solve the SDRE, online, with some errors. However, the proposed method gives the best reslt, which is almost the same as the eact soltion. Figre. Soltions of SDRE Figres and 3 show that the proposed control drives the states to zero, and that the control signal and the reslting states are almost the same as the eact soltion. he other two simlated methods are also able to drive the states to zero. However, the control signals for these methods are notably different than the eact SDRE control soltion. Figre. Comparison of state signals Figre 3. Comparison of control signals In able I, the costs for the closed loop controlled systems are given. Again, nlike the other algorithms, the proposed method controls the nonlinear scalar system with a cost comparable to that for the eact optimal soltion. Even for a scalar system, this reslt demonstrates the efficiency of the gradient-type neral network algorithm. B. Eample ABLE I COMPARISON OF COSS FOR EXAMPLE Methods Costs Eact Soltion Proposed Method Kleinman Method 3.73 Qasi-Newton Method his eample considers an otpt feedback control design for a magnetic levitation system [7]. he design reqires the soltion of two SDREs, one for state feedback control synthesis, P(), and one for state estimation, S(). Here, the cost fnction for feedback control synthesis is.5 3 J (, ) = ( + (.5 ).5 ) (3) he design matrices, Ξ and Θ, for state estimation are. Ξ=, Θ=.. 3 (3) Also, the associated nonlinear dynamics are given as = (α + r + ) e β g + (3) ( α + re + ) ( α + re + ) y = 6339

5 > REPLACE HIS LINE WIH YOUR PAPER IDENIFICAION NUMBER (DOUBLE-CLICK HERE O EDI) 5 where the constant gains g, α, β, and the eqilibrim reference position r e (t) are selected near their eact vales, [7], as 9.8 m/s,.5 m,.6, and -., respectively. It shold be noted that in (3) the system eqations are rewritten so that the eqilibrim states are at zero [7]. Since the SDRE approach considers a stabilization problem, this formlation allows the system to track constant reference signals. A stabilizable and detectable state-dependent parameterization of (3) can be given as ( + + ) = α r + β (33) e g ( α + re + ) ( α + re + ) A ( ) B ( ) y = [ ] C( ) he initial states were chosen as ()=[-.,.5]. he initial vales and the parameters for the proposed gradienttype neral network algorithm were chosen as, P()=I, L P ()=I, n P =, n LP = for state feedback synthesis, and S()=5I, L S ()=I, n S =5, n LS = for state estimation. Figres 7 present the reslts for the nonlinear stabilization problem. Figre 4. System response and reference position Figre 5. Control signal Figre 7. Soltion of P() for state feedback control Figre 8. Soltion of S() for state estimation From Figres 4 and 5, it is obvios that the control methodology stabilizes the system and drives the system otpt to zero eqilibrim reference position. Figre 6 shows the actal and the estimated states, while Figres 7 and 8 give the soltion of the two SDREs for state feedback control and state estimation, P() and S(), respectively. Now, for tracking control of the magnetic levitation system, consider the cost fnction for nonlinear control synthesis, as. J (, ) = ( +.. ) (34) and let the design matrices, Ξ and Θ, for state estimation prpose be given as. Ξ=, Θ=.5.5 (35) We chose the initial state vales as ()=[-.,.]. Also we chose, P()=4I, L P ()=I, n P =, n LP = for state feedback synthesis, and S()=5I, L S ()=I, n S =5, n LS = for state estimation. For tracking problem, r e (t) was selected as.5+.5sqare(πt). he tracking reslts and the SDRE soltions are shown in Figres 9 to 3, which clearly display the efficacy of the proposed method. Figre 6. Actal states and estimated States Figre 9. System response and reference position 634

6 > REPLACE HIS LINE WIH YOUR PAPER IDENIFICAION NUMBER (DOUBLE-CLICK HERE O EDI) 6 Figre. Control signal gradient-type neral network algorithm is shown to be effective, fast, and relatively simple. Frther, it is shown via simlation eamples that this techniqe does not depend on initial conditions and reslts in a soltion that is very close to the eact soltion. In particlar, the performance of the proposed method was investigated for nonlinear qadratic Gassian (NQG) control design problem. It was shown that the proposed method can provide fast and accrate soltion to the SDRE-based NQG control design problem. In addition, the proposed methodology can be easily etended to adaptive and robst control of nonlinear systems via SDRE based NQG methods. Figre. Actal States and estimated states Figre. Soltion of P() for State Feedback Control Figre 3. Soltion of S() for State Estimation As it is well known from the linear qadratic Gassian (LQG) theory [], a desired performance can be achieved by adsting the penalty matrices (Q, R for state feedback control and Ξ, Θ for state estimation). he proposed method is a nonlinear etension of the LQG, i.e., a nonlinear qadratic Gassian (NQG) method. Similar to the LQG, the proposed NQG allows for the adstments of penalty matrices to achieve a desired performance. From the soltions of two eamples, it is evident that the proposed online gradient-type neral network algorithm is fast and accrate enogh to realize and solve an NQG problem. REFERENCES [] J. R. Clotier, State-Dependent Riccati Eqation echniqes: An Overview, IEEE Proc. of American Control Conference, pp , Albqerqe, NM, Jne, 997. [] J. S. Shamma, J. R. Clotier, Eistence of SDRE Stabilizing Feedback, IEEE rans. on Atomatic Control, Vol. 48, No. 3, pp.53-57, 3. [3] H.. Banks, B. M. Lewis, H.. ran, Nonlinear Feedback Controllers and Compensators: A State-Dependent Riccati Eqation Approach, J. Comptational Optimization and Applications, Vol. 37, No., pp.77-8, 7. [4] P. K. Menon,. Lam, L. S. Crawford, V. H. L. Cheng, Real-ime Comptational Methods for SDRE Nonlinear Control of Missiles, IEEE Prof. of American Control Conference, pp.3-37, Jne,. [5] J. Imae, H. Sagami,. Kobayashi, G. Zhai, Nonlinear Control Design Method Based on State-Dependent Riccati Eqation (SDRE) via Qasi-Newton Method, IEEE Proc. of Conf. on Decision and Control, pp.74-74, Paradise Island, Bahamas, December, 4. [6] J. Imae, K. Yoshimiz,. Kobayashi, G. Zhai, Algorithmic Control for Real-ime Optimization of Constrained Nonlinear Systems: Swing-p Problems of Inverted Pendlms, IEEE Proc. of Conf. on Control Applications, pp , oronto, Canada, Agst, 5. [7] E. B. Erdem, A. G. Alleyne, Design of a Class of Nonlinear Controllers via State-Dependent Riccati Eqations, IEEE rans. on Control Systems echnology, Vol., No., pp.33-37, 4. [8] A. Bogdanov, E. Wan, State-Dependent Riccati Eqation Control for Small Atonomos Helicopters, J. Gidance, Control, and Dynamics, Vol.3, No., pp.47-6, 7. [9] C.-L. Lin, C.-C. Lai,.-H. Hang, A Neral Network for Linear Matri Ineqality Problems, IEEE rans. on Neral Networks, Vol., No.5, pp.78-9,. [] A. Isidori, Nonlinear Control Systems, nd ed. Berlin: Springer-Verlag, 989. [] A. J. van der Schaft, L -Gain Analysis of Nonlinear Systems and Nonlinear State Feedback H Control, IEEE rans. on Atomatic Control, Vol.37, No.6, pp , 99. [] P. Dorato, C.. Abdallah, V. Cerone, Linear Qadratic Control: An Introdction, st ed. Florida: Kreiger, 995. [3] J. Wang, G. W, A Mltilayer Neral Network for Solving Continos-ime Algebraic Riccati Eqations, Neral Networks, Vol., No.5, pp , 998. [4] C.-L. Lin, C.-L. Chen, Realization of a Riccati Eqation Based Controller Using Gradient-ype Neral Networks, Control Engineering Practice, Vol.9, pp.39-34,. [5] R. A. Freeman, P. V. Kokotovic, Optimal Nonlinear Controllers for Feedback Linearizable Systems, Workshop on Robst Control via Variable Strctre and Lyapnov echniqes, Italy, September, 994. V. CONCLUSION A new method is reported for the online soltion of the state-dependent Riccati eqations (SDRE). he proposed 634