Approximate optimal solution of the DTHJB equation for a class of nonlinear affine systems with unknown dead-zone constraints

Transcription

1 Soft Compt (214) DOI 1.17/s METHODOLOGIES AND APPLICATION Approximate optimal soltion of the DTHJB eqation for a class of nonlinear affine systems with nnown dead-zone constraints Deha Zhang Derong Li Ding Wang Pblished online 11 Jne 213 Ó Springer-Verlag Berlin Heidelberg 213 Abstract In this paper, an optimal control scheme of a class of nnown discrete-time nonlinear systems with dead-zone control constraints is developed sing adaptive dynamic programming (ADP). First, the discrete-time Hamilton Jacobi Bellman (DTHJB) eqation is derived. Then, an improved iterative ADP algorithm is constrcted which can solve the DTHJB eqation approximately. Combining with Riemann integral, detailed proofs of existence and niqeness of the soltion are also presented. It is emphasized that this algorithm allows the implementation of optimal control withot nowing internal system dynamics. Moreover, the approach removes the reqirements of precise parameters of the dead-zone. Finally, simlation stdies are given to demonstrate the performance of the present approach sing neral networs. Keywords Nonlinear affine system Dead-zone DTHJB Neral networs Riemann integral 1 Introdction Dead-zone, baclash, satration, hysteresis and actator nonlinearities are very common in most practical indstrial Commnicated by G. Acampora. D. Zhang D. Li (&) D. Wang State Key Laboratory of Management and Control for Complex Systems, Institte of Atomation, Chinese Academy of Sciences, Beijing 11, China derong.li@ia.ac.cn D. Zhang dhazhang@163.com D. Wang ding.wang@ia.ac.cn control systems. Becase of the nonanalytical natre of these actator nonlinearities and the fact that their accrate nonlinear mathematical fnctions are difficlt to now, sch systems present a challenge for practitioners. Ths, there have been many discssions on this sbject. For example, baclash, hysteresis and satration nonlinearities were considered in Tao and Kootovic (15a, b, 16), Zhang et al. (2), Bernstein (15), Saberi et al. (16) and Sssmann et al. (14), respectively. Dead-zone is one of the most important nonlinearities in many indstrial processes, which can severely affect the system s performance. The stdy of control systems with dead-zone nonlinearities has been the focs of researchers for many years. Several soltions for deriving control laws considering the dead-zone phenomena can be fond in Tao and Kootovic (14, 15c), Gao and Rasto (26), Recer et al. (11), Selmic and Lewis (2), Lewis et al. (1), X et al. (25), Wang et al. (24), Ma and Yang (21) and Zhang and Ge (27, 2). In X et al. (25), affine nonlinear systems with dead-zone inpt were first investigated. The stdy of adaptive control for systems with nnown dead-zone was initiated by Recer et al. (11), where an adaptive scheme was proposed for the case of fll state measrement. In recent years, there are some wors of handling dead-zone nonlinearities from the perspective of adaptive control (Tao and Kootovic 14, 15c Wang et al. 24 Ma and Yang 21 Zhang and Ge 27, 2). In fact the exact soltion of the Hamilton Jacobi Bellman (HJB) eqation is generally impossible to obtain for nonlinear systems especially with dead-zone constraints. To overcome the difficlty, recrsive methods are employed to obtain the soltion of HJB eqation indirectly, sch as the iterative adaptive dynamic programming (ADP). ADP is a very sefl tool in solving optimization and optimal control problems by employing the principle of

2 35 D. Zhang et al. optimality, which is expressed as An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions mst constitte an optimal policy with regard to the state reslting from the first decision (Bellman 157). In recent years, ADP sing niversal fnction approximators has received mch attention from many researchers in order to obtain approximate soltions of the HJB eqation (Ab-Khalaf and Lewis 25 Balarishnan et al. 2 Al-Tamimi et al. 27 Al-Tamimi and Lewis 2 Li et al. 21, 25 Wei et al. 2). Typically, these methods se fzzy logic systems or neral networs (NNs) to parameterize the nnown nonlinearities. According to Werbos (12) and Prohorov and Wnsch (17), ADP approaches were classified into several main schemes heristic dynamic programming (HDP), action-dependent HDP (ADHDP), dal heristic dynamic programming (DHP), ADDHP, globalized DHP (GDHP), and ADGDHP. In Seiffertt et al. (21), HJB eqations were motivated and proven on time scales. The athors copled the calcls of time scales with stochastic control via ADP algorithm and pointed ot three significant directions for the investigation of ADP on time scales. In the present paper, we stdy this problem throgh ADP in order to solve the discrete-time HJB (DTHJB) eqations and then the optimal control problems for general affine nonlinear discrete-time systems with dead-zone control constraints. In smmary, the main contribtions of this paper are as follows. 1. Present a framewor for optimal control of nonlinear systems with nnown dead-zone constraints. 2. Develop a novel nonqadratic fnctional to deal with dead-zone control constraints of nonlinear discretetime systems and derive the corresponding DTHJB eqation. 3. Utilize two NNs to approximate the dead-zone and the corresponding nonlinear control system, respectively, and ths solve the corresponding nnown model problems. 4. The two simlation examples demonstrate that the iterative vale fnction seqence converges to the optimal vale fnction, and show the validity of the proposed algorithm. Specifically, we se a model networ to approximate the nonlinear system dynamics, which renders the iterative ADP algorithm sitable to nnown plants. In addition, the dead-zone nonlinearity will be approximated sing NN. It is the first time to se HDP techniqe to solve the optimal control problem with dead-zone constraints. 2 Problem statement and preliminaries Consider a class of affine nonlinear discrete-time system Rðf gþ with dead-zone control inpts described by x þ1 ¼ f ðx Þþgðx Þ ð1þ U r ðv i Þ v i > b r i ¼ Uðv i Þ¼ b l \v i \b r ð2þ U l ðv i Þ v i 6 b l where x ¼ xðþ 2R n is the state vector, and f R n! R n g R n! R nm are differentiable with respect to their argments with f(). In addition, i ¼ Uðv i Þ is the otpt of dead-zone where UðÞ is an abstract mathematical description of the dead-zone, v i v i (x ) is the inpt vector to dead-zone, U r ðv i Þ and U l ðv i Þ are nnown nonlinear fnctions for v i [ [b r,??) and for v i [ (-?, b l ], respectively. We denote X ¼f j ¼½ m Š T 2 R m i ¼ Uðv i Þ i ¼ 1... mg As stated in Tao and Kootovic (14), the simplest linear symmetric dead-zone model is only a static simplification of diverse physical phenomena with negligible fast dynamics. Actally, in many indstrial processes the deadzone parameters are nnown, bt its model is monotonic. Ths the dead-zone inverse fnction exists. The ey featres of the control problems investigated in this paper are 1. We have the inpt/otpt data of the dead-zone, namely prior nowledge of the dead-zone control. 2. Assme that f? g is Lipschitz continos on a set X R n containing the origin, and that the system Rðf gþ is controllable in the sense that there exists at least a continos control law on X that asymptotically stabilizes the system. This assres not only the existence of soltion of the DTHJB eqation bt also the corresponding optimal control. 3. The dead-zone and the system models cannot be acqired exactly. 4. The dead-zone is continos monotonic. Definition 1 (cf. Zhang et al. 2) Stabilizable system a nonlinear dynamical system is said to be stabilizable on a compact set X R n if for all initial conditions x 2 X there is a control inpt 2 R m sch that the state x? as???. It is desired to find the optimal control action for the system Rðf gþ which minimizes the infinite-horizon vale fnction given by Vðx Þ¼ X1 c i Uðx i i Þ ð3þ i¼

3 Approximate optimal soltion of the DTHJB eqation 351 where U(x i, i ) is the tility fnction and is positive definite, i.e., U(, ) for x i, i, and U(x i, i ) [ for Vx i, i. c is the discont factor with \ c 6 1. The tility fnction sally can be expressed as Uðx i i Þ¼x T i Qx i þ Wð i Þ ð4þ where Q and W( i ) are positive definite as above, and for nconstrained control inpts, a common choice for W( i )is W( i ) i T R i, where R [ R 2 R mm Bt for constrained control inpts, inspired by Ab- Khalaf and Lewis (25) and Lyshevsi (1) who introdced a generalized nonqadratic fnction when dealing with bonded controls, we can define Wð i Þ¼2 Z i U T ðv i ÞRdv i Uðv i Þ¼½/ðv 1i Þ /ðv 2i Þ... /ðv mi ÞŠ T ð5þ where UðÞ is a constrained and continos monotonic fnction, and R is positive definite and assmed to be symmetric for simplicity. Since /() is a monotonic odd fnction and R is positive definite, W() is positive definite. For the general optimal control problems, the controllers need to be stable and also to garantee that (3) is finite, i.e., the control mst be adminissible (Ab-Khalaf and Lewis 25). Definition 2 Admissible control a control is said to be admissible with respect to (3) onx if is continos on a compact set X R n ¼ stabilizes (3) onx and x 2 X Vðx Þ is finite. Eqation (3) can be rewritten as follows Vðx Þ¼ X1 c i x T i Qx i þ Wð i Þ i¼ ¼ x T Qx þ Wð Þ þ c X1 c i 1 x T i Qx i þ Wð i Þ i¼þ1 ¼ x T Qx þ 2 Z U T ðv ÞRdv þ cvðx þ1 Þ ð6þ According to Bellman s optimality principle, the optimal vale fnction V * (x ) satisfies the following DTHJB eqation Z V ðx Þ¼min x T Qx þ 2 U T ðv ÞRdv þ cv ðx þ1 Þ ð7þ The optimal control * at time is the which achieves the aforementioned minimm, i.e., ¼ arg min x T Qx þ 2 Z U T ðv ÞRdv þ cv ðx þ1 Þ ðþ Note that the DTHJB eqation develops bacward in time, and * satisfies the first-order necessary condition, which is given by the gradient of the right-hand side of (7) with respect to as 1 Z x T o Qx þ 2 U T ðv ÞRdv þ cv ðx þ1 ÞA ¼ ðþ Therefore, we have ¼ U c! ox T þ1 ov ðx þ1 Þ 2 R 1 ð1þ o ox þ1 By sbstitting (1) into (7), the DTHJB can be expressed as V ðx Þ¼x T Qx þ 2 Z U T ðv ÞRdv þ cv ðx þ1 Þ ð11þ which is the optimal vale fnction corresponding to the optimal control policy *. Eqations (1) and (11) are called the best optimized implementation of dynamic programming problems. In the general nonlinear dynamic case, the HJB eqation cannot be solved analytically de to the well-nown crses of the dimensionality. Therefore, in the following sections, we will present how the HDP algorithm wors with the definition of Riemann integral to solve approximately the optimal control problem with dead-zone constraints. 3 The approximate soltion of the DTHJB eqation for general nonlinear systems As said above, the DTHJB eqation is generally nonanalitical for nonlinear systems and the iterative ADP algorithm is a good method to solve it. Therefore, in order to overcome the difficlty in solving the DTHJB eqation, we employ recrsive ADP method combined with Riemann integral to obtain its approximate soltion. Also we demonstrate the existence and niqeness proofs of the soltion in this section. 3.1 The iterative ADP algorithm In this sbsection, we develop the iterative ADP algorithm, based on the Bellman s principle of optimality, the Riemann integral, and the greedy HDP algorithm. This

4 352 D. Zhang et al. improved algorithm can avoid sing the complex mathematical analysis method and ensre the calclating accracy of the DTHJB Eq. (7). The algorithm can be performed as follows. For any x 2 R n at time, the first step is to assign an arbitrary nonnegative constant to the initial vale fnction V (x ), e.g., V (), then we can search the corresponding control vector, which optimizes the vale fnction V 1 (x ) ¼ arg minfv 1 ðx Þg ¼ arg min x T Qx þ 2 Z U T ðv ÞRdv þ cv ðx þ1 Þ ð12þ Once we find the corresponding optimal control vector, the vale fnction in the next step V 1 (x ) can be pdated as follows Z V 1 ðx Þ¼min x T Qx þ 2 U T ðv ÞRdv þ cv ðx þ1 Þ ¼ x T Qx þ 2 Z U T ðv ÞRdv ð13þ In this way, for the iteration index i ¼ the algorithm can be exected between i ¼ arg minfv iþ1 ðx Þg ¼ arg min x T Qx þ 2 Z and V iþ1 ðx Þ¼min x T Qx þ 2 ¼ x T Qx þ 2 Z i U T ðv ÞRdv þ cv i ðx þ1 Þ Z ð14þ U T ðv ÞRdv þ cv i ðx þ1 Þ U T ðv ÞRdv þ cv i ðx þ1 Þ ð15þ ntil both the vale fnction and the corresponding control law converge to their respective optimal vales. Note that the integral term in the iterative algorithm cannot be solved for the nnown dead-zone nonlinear model. To get arond sch difficlty, we introdce the Riemann integral. Based on the Jordan measre, the Riemann integral is defined by taing the limit of the Riemann sm. Ths, the integral term in the above DTHJB eqation can easily be solved when expressed as the following form Z U T ðv ÞRdv ¼ X n lim ðu T ðv ÞRMv Þ maxðmv Þ! ¼1 ð16þ where v is an arbitrary point in the interval Dv, and the sbinterval Dv /n *.Thevalemax(Dv ) is called the maximm of a partition of the interval [, ] into sbintervals Dv. So far, the DTHJB eqation has been solved throgh the improved iterative ADP algorithm and the Riemann integral. In the next sbsection, we will prove the soltion s existence and niqeness 3.2 Existence and niqeness of the soltion of the DTHJB eqation Theorem 1 Consider the iterative vale fnction seqence {V i (x )} in (15) and its corresponding controls { i } in (14), respectively. If the nonlinear system is stabilizable on a compact set X and there is an admissible control seqence { i }, then the derived DTHJB Eq. (7) has a niqe soltion and it can be obtained throgh the iteration between (14) and (15), i.e., V 1 ðx Þ¼V ðx Þ 1 ¼ Proof 1. Existence For any admissible control inpt n i, the corresponding vale fnction C i ðx Þ is pdated by C iþ1 ðx Þ¼x T Qx þ Wðn i ÞþcCi ðx þ1 Þ ð17þ Let C ðþ ¼ V ðþ be any nonnegative constant. For convenience, let the constant be zero. So C iþ1 ðx Þ C i ðx Þ¼c C i ðx þ1 Þ C i 1 ðx þ1 Þ ¼ c 2 C i 1 ðx þ2 Þ C i 2 ðx þ2 Þ ¼ c 3 C i 2 ðx þ3 Þ C i 3 ðx þ3 Þ.. ¼ c i C 1 ðx þi Þ C ðx þi Þ ¼ c i C 1 ðx þi Þ ð1þ Ths, C iþ1 ðx Þ can be rewritten as the following form C iþ1 ðx Þ¼C i ðx Þþc i C 1 ðx þi Þ ¼ C i 1 ðx Þþc i 1 C 1 ðx þi 1 Þþc i C 1 ðx þi Þ ¼ C 1 ðx ÞþcC 1 ðx þ1 Þþ þ c i 1 C 1 ðx þi 1 Þþc i C 1 ðx þi Þ ¼ Xi j¼ c j C 1 ðx þj Þ ð1þ

5 Approximate optimal soltion of the DTHJB eqation 353 Combining with C i ðþ > we can frther get C iþ1 ðx Þ¼ Xi c j j¼ 6 X1 c j j¼ x T þj Qx þj þ Wðn þj Þ x T þj Qx þj þ Wðn þj Þ ð2þ Since the nonlinear system is stabilizable, that is, the state x? as!1 we have C iþ1 ðx Þ 6 X1 c j j¼ x T þj Qx þj þ Wðn þj Þ 6 C i ð21þ where C is a nonnegative constant. On the other hand, V i? 1 (x ) is the minimm of the right hand side of (15) with respect to the corresponding control inpt. Ths, V iþ1 ðx Þ 6 C iþ1 ðx Þ 6 C i ð22þ In conclsion, the limit of the vale fnction seqence {V i } exists. 2. Uniqeness First, we will prove that {V i (x )} is a nondecreasing i? 1 seqence. Define a new vale fnction at W iþ1 ðx Þ¼x T Qx þ Wð iþ1 ÞþcW i ðx þ1 Þ ð23þ with the initial vale W ðþ ¼ V ðþ For i, V 1 ðx Þ¼x T Qx þ Wð ÞþW ðx Þ > W ðx Þ We assme that V i ðx Þ > W i 1 ðx Þ for i - 1. Then for i, we have V iþ1 ðx Þ¼x T Qx þ Wð i ÞþcVi ðx þ1 Þ > x T Qx þ Wð i ÞþcWi 1 ðx þ1 Þ¼W i ðx Þ Combining with (22), we get V iþ1 ðx Þ > W i ðx Þ > V i ðx Þ ð24þ Next we will present how to prove the niqeness of the soltion sing the above property. According to (15), V i ðx Þ 6 x T Qx þ Wð ÞþcV i 1 ðx þ1 Þ 6 x T Qx þ Wð ÞþcV 1 ðx þ1 Þ When i!1 V 1 ðx Þ 6 x T Qx þ Wð ÞþcV 1 ðx þ1 Þ which sggests that V 1 ðx Þ 6 minfx T Qx þ Wð ÞþcV 1 ðx þ1 Þg ð25þ On the other hand, V 1 ðx Þ > V i ðx Þ ¼ minfx T Qx þ Wð ÞþcV i 1 ðx þ1 Þg i Similarly, when i!1 V 1 ðx Þ > minfx T Qx þ Wð ÞþcV 1 ðx þ1 Þg ð26þ Combining (25) with (26), we can conclde that V 1 ðx Þ¼minfx T Qx þ Wð ÞþcV 1 ðx þ1 Þg ð27þ Given the above, we have proved the existence and niqeness of the soltion of the DTHJB eqation. The soltion can be obtained throgh the iterative algorithm. Remar 1 As we all now, the optimal control of nonlinear discrete time systems is often redced to solving the nonlinear DTHJB eqation. From above, we can see that V 1 ðx Þ is the niqe soltion of the DTHJB eqation, which indicates that both the vale fnction and the corresponding control law seqence converge to the optimal vales, respectively, i.e. V 1 ðx Þ¼V ðx Þ 1 ¼ 4 NN implementation It is well nown that NNs can be employed to approximate any continos fnction on prescribed compact sets (Li et al. 21 Zhang et al. 21), so it is natral to se NNs to approximate the nonlinear system and dead-zone model. Here, we se the mlti-layered bac-propagation (BP) NNs. Figre 1 shows the strctral diagram of the iterative algorithm. As for (1), we cold bild an inverse dead-zone NNs sing the prior nowledge of the dead-zone control. Then, for i ¼ the iterative algorithm can be implemented between Z i ¼ argmin x T Qx þ 2 U T ðvþrdv þ cv i ðx þ1 Þ ¼ U c ox þ1 2 R 1 o i and V iþ1 ðx Þ¼min x T Qx þ 2 ¼ x T Qx þ 2 where D v /n *.! T ov i ðx þ1 Þ ð2þ lim Z ox þ1 X n maxðmv Þ! ¼1 U T ðv ÞRdv þ cv i ðx þ1 Þ U T ðv ÞRMv þ cv i ðx þ1 Þ ð2þ

6 354 D. Zhang et al. + γ + Fig. 1 Flowchart of the proposed iterative algorithm with dead-zone In the above iterative algorithm, i is the iterative index of the vale fnction and the control law, while is the time index. The vale fnction and the control law are pdated ntil they both converge to their respective optimal vales. In the above flowchart, the critic networ estimates the optimal vale fnction V ðx Þ and is trained forward in time. While the action networ is sed to approximate the optimal control inpt. To approximate the nonlinear vale fnction and nonlinear dynamic system, we choose bacpropagation algorithm to train artificial neral networs and mean sqare error is sed as a measre of how well the neral networ has learnt. The nonlinear activation fnction in the critic networ is chosen as logistic fnction which can mae its otpt a positive nmber, while in the model networ and the action networ we adopt tansig fnction. For the strctre and other parameters of or NNs, we will present them in the next two specific examples. 5 Simlation stdies In this section, the effectiveness of the proposed iterative algorithm is demonstrated for nonlinear affine systems with nnown dead-zone control inpt by sing two examples. Example 1 (Nonlinear affine system) First of all the present algorithm is implemented for the example identical to the one in Chen and Jagannathan (2) except the deadzone control. We consider the following system x 1 þ1 5x ¼ 2 þ x 1 x 2 þ1 5x 1 335x 3 1 þ x 2 þ 5 ð3þ v 1 v > 1 ¼ Uðv Þ¼ 1\v \1 ð31þ v þ 1 v 6 1 We choose three-layer three-layer BP NNs as model networ, dead-zone networ, critic networ, and action networ with the strctres , , 2 1, and 2 1, respectively. The initial weights of the for networs are all set to be random in [ -1,1]. It shold be noted that the model identification of the dead-zone and the dynamic system is implemented first nder the learning rate l m.5, and then their weights are ept nchanged. After that, we se the trained model networ and the deadzone networ to train the critic networ and the action networ with the learning rates l c l a.5, for i 5 iteration steps, where n * 1,. The discont factor is chosen as c.35. Enogh iteration steps shold be implemented to garantee the soltion accracy of the DTHJB eqation. The vale fnction is defined as (4), where Q ¼ 1 and R.5. 1 Then for the given initial state x [1, - 1] T,we apply the iterative algorithm to the controlled nonlinear system and obtain the iterative vale fnction as in Fig. 2 which verified the presented theory. From Figs. 3 and 4,we can see that the system states, the vale fnction and the corresponding control seqence converge to their respective optimal vales qicly. Example 2 (The pendlm) The second example stdied here is the pendlm swinging p and balancing control problem with dead-zone constraints. This example is chosen from Si and Wang (21) with some modifications. Here we consider the following nonlinear affine discretetime system x 1 þ1 1x ¼ 2 þ x 1 þ x 2 þ1 4 sinðx 1 Þþx 21 ð32þ The dead-zone nonlinear model is as follows ¼ Uðv Þ ð1 3 sinðv ÞÞðv 2Þ v > 2 ¼ 2\v \2 ð 2 cosðv ÞÞðv þ 2Þ v 6 2 ð33þ We choose NNs as above and apply the algorithm to the plant with the initial state [1-1]. The simlation parameters and vale fnction are defined the same as in

7 Approximate optimal soltion of the DTHJB eqation x 1 x 2 The iterative vale fnction Iteration step i The system states Fig. 3 The system state vector Fig. 2 The vale fnction with iteration step i Example 1. Then we can obtain the following simlation reslts. Figre 5 shows the vale fnction with sccessive approximation whereas Fig. 6 demonstrates the vale fnction with the pdating time step. From Fig. 7 we can see that the state trajectories converge to the eqilibrim point qicly. Figres and display the dead-zone inpt and its otpt with the time step iterations. From the reslts, we can see that the algorithm has been proven to be effective. The vale fnction and the control converge to their optimal vales and the system states also achieve the static eqilibrim qicly. Note that the method is independent on the mathematical models of nonlinear system and dead-zone. The models in the paper act as the role of providing data of the plant. 6 Conclsions Based on ADP techniqe and Rimann integral, we derived the DTHJB eqation and solved approximately the eqation throgh the improved iterative algorithm. We also demonstrated proofs of the existence and the niqeness of the soltion. Inspired by the fact that NNs can approximate arbitrary nonlinear fnctions, we established dead-zone networ and the model networ, and ths obtained the approximate soltion for the optimal control problem of Fig. 4 The optimal vale fnction and the corresponding optimal control with time step The optimal l vale fnction The optimal control

8 356 D. Zhang et al. The iterative vale fnction Iteration step i Fig. 5 The vale fnction with iteration step i The dead zone inpt Fig. The dead-zone inpt The optimal vale fnction The dead zone otpt Fig. 6 The optimal vale fnction with time step Fig. The dead-zone otpt 1 x 1 x 2 nonlinear affine system with dead-zone constraints. The correctness and validity of the theoretical analysis and the algorithm were demonstrated by simlation reslts. The system states Fig. 7 The system state vector Acnowledgments This wor was spported in part by the National Natral Science Fondation of China nder Grants 61342, 631, and References Ab-Khalaf M, Lewis FL (25) Nearly optimal control laws for nonlinear systems with satrating actators sing a neral networ HJB approach. Atomatica 41(5)77 71 Al-Tamimi A, Ab-Khalaf M, Lewis FL (27) Adaptive critic designs for discrete-time zero-sm games with application to H 1 control. IEEE Trans Syst Man Cybern B 37(1) Al-Tamimi A, Lewis FL (2) Discrete-time nonlinear HJB soltion sing approximate dynamic programming convergence proof. IEEE Trans Syst Man Cybern B 3(4)43 4

9 Approximate optimal soltion of the DTHJB eqation 357 Balarishnan SN, Ding J, Lewis FL (2) Isses on stability of ADP feedbac controllers for dynamical systems. IEEE Trans Syst Man Cybern B 3(4)13 17 Bellman RE (157) Dynamic programming. Princeton University Press, Princeton Bernstein DS (15) Optimal nonlinear, bt continos, feedbac control of systems with satrating actators. Int J Control 62(5) Chen Z, Jagannathan S (2) Generalized Hamilton Jacobi Bellman formlation-based neral networ control of affine nonlinear discrete-time systems. IEEE Trans Neral Netw 1(1) 16 Gao W, Rasto RS (26) Neral networ control of a class of nonlinear systems with actator satration. IEEE Trans Neral Netw 17(1) Lewis FL, Tim WK, Wang LZ, Li ZX (1) Deadzone compensation in motion control systems sing adaptive fzzy logic control. IEEE Trans Control Syst Technol 7(6) Li DR, Xiong XX, Zhang Y (21) Action-dependent adaptive critic designs. In Proceeding of the International Joint Conference on Neral Networs, Washington, pp 5 Li DR, Zhang Y, Zhang HG (25) A self-learning call admission control scheme for CDMA celllar networs. IEEE Trans Neral Netw 16(5) Li ZW, Zhang HG, Zhang QL (21) Novel stability analysis for recrrent neral networs with mltiple delays via line integraltype L-K fnctional. IEEE Trans Neral Netw 21(11) Lyshevsi SE (1) Optimal control of nonlinear continos-time systems design of bonded controllers via generalized nonqadratic fnctional. In Proceedings of American control conference, Philadelphia, pp 25 2 Ma HJ, Yang GH (21) Adaptive otpt control of ncertain nonlinear systems with non-symmetric dead-zone inpt. Atomatica 46(2) Prohorov DV, Wnsch DC (17) Adaptive critic designs. IEEE Trans Neral Netw (5)7 17 Recer D, Kootovic P, Rhode D, Winelman J (11) Adaptive nonlinear control of systems containing a dead-zone. In Proceedings of the 3th IEEE conference on decision and control, Brighton, pp Saberi A, Lin Z, Teel A (16) Control of linear systems with satrating actators. IEEE Trans Atom Control 41(3)36 37 Seiffertt J, Sanyal S, Wnsch DC (21) Hamilton Jacobi Bellman eqations and approximate dynamic programming on time scales. IEEE Trans Syst Man Cybern B 3(4)1 23 Selmic RR, Lewis FL (2) Deadzone compensation in motion control systems sing neral networs. IEEE Trans Atom Control 45(4) Si J, Wang Y (21) Online learning control by association and reinforcement. IEEE Trans Neral Netw 12(2) Sssmann H, Sontag ED, Yang Y (14) A general reslt on the stabilization of linear systems sing bonded controls. IEEE Trans Atom Control 3(12) Tao G, Kootovic PV (14) Adaptive control of plants with nnown dead-zones. IEEE Trans Atom Control 3(1)5 6 Tao G, Kootovic PV (15) Adaptive control of systems with nnown otpt baclash. IEEE Trans Atom Control 4(2) Tao G, Kootovic PV (15) Continos-time adaptive control of systems with nnown baclash. IEEE Trans Atom Control 4(6)13 17 Tao G, Kootovic PV (15) Discrete-time adaptive control of systems with nnown deadzones. Int J Control 61(1)1 17 Tao G, Kootovic PV (16) Adaptive control of systems with actator and sensor nonlinearities. John Wiley, New Yor Wang XS, S CY, Hong H (24) Robst adaptive control of a class of nonlinear systems with nnown dead-zone. Atomatica 4(3) Wei QL, Zhang HG, Dai J (2) Model-free mltiobjective approximate dynamic programming for discrete-time nonlinear systems with general performance index fnctions. Nerocompting 72(7 )13 14 Werbos PJ (12) Approximate dynamic programming for real-time control and neral modeling. In White DA, Sofge DA (eds) Handboo of intelligent control neral, fzzy, and adaptive approaches, New Yor X JX, X J, Lee TH (25) Iterative learning control for systems with inpt deadzone. IEEE Trans Atom Control 5() Zhang TP, Ge SS (27) Adaptive neral control of MIMO nonlinear state time-varying delay systems with nnown dead-zones and gain signs. Atomatica 43(6) Zhang TP, Ge SS (2) Adaptive dynamic srface control of nonlinear systems with nnown deadzone in pre feedbac form. Atomatica 44(7)15 13 Zhang H, Lo Y, Li D (2) Neral-networ-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans Neral Netw 2() Zhang HG, Li ZW, Hang GB, Wang ZS (21) Novel weightingdelay-based stability criteria for recrrent neral networs with time-varying delay. IEEE Trans Neral Netw 21(1)1 16