Backstepping controller design for a class of stochastic nonlinear systems with Markovian switching

Transcription

1 Backstepping controller design for a class of stochastic nonlinear systems with Markovian switching Zhao-Jing WU a, Xue-Jun XIE b, Peng SHI c,yuan-qing XIA d a School of Mathematics and Information Science, Yantai University, Yantai, Shandong Province, 645, China b Institute of Automation, Qufu Normal University, Qufu, Shandong Province, 73165, China c Department of Computing and Mathematical Sciences, University of Glamorgan, Pontypridd, CF371DL, United Kingdom d Department of Automatic Control, Beijing Institute of Technology, Beijing 181, China. Abstract A more general class of stochastic nonlinear systems with irreducible homogenous Markovian switching are considered in this paper. As preliminaries, the stability criteria and the existence theorem of strong solution are firstly presented by using the inequality of mathematic expectation of Lyapunov function. The state-feedback controller is designed by regarding Markovian switching as constant such that the closed-loop system has a unique solution, and the equilibrium is asymptotically stable in probability in the large. The output-feedback controller is designed based on a quadratic-plus-quartic-form Lyapunov function such that the closed-loop system has a unique solution with the equilibrium being asymptotically stable in probability in the large in unbiased case and has a unique bounded-in-probability solution in biased case. Key words: Nonlinear stochastic systems, backstepping, Markovian switching. 1 Introduction Stability of stochastic differential equationssde) has been one of the most important research topics not only for pure mathematics but also for other subjects such as cybernetics. For some representative work on this general topic, to name a few, we refer readers to Arnold 197, Friedman 1976, Khas minskii 198, Skorohod 1989), and the references therein. Recently, stability of SDE with Markovian switching has received a lot of attention. Ji & Chizeck 199) studied the stability of a jump linear equation; Basak, Bisi & Ghosh 1996) discussed the stability of a semi-linear SDE with Markovian switching; Mao 1999) discussed the exponential stability of general nonlinear differential equations with This paper was not presented at any IFAC meeting. Xue-Jun XIE is also with School of Electrical Engineering and Automation, Xuzhou Normal University, Xuzhou, Jiangsu Province, 1116, China. Peng Shi is also with ILSCM, School of Science and Engineering, Victoria University, Melbourne, VIC 81, Australia, and School of Mathematics and Statistics, University of South Australia, Mawson Lakes 573, SA, Australia. address: wuzhaojing@188.com Zhao-Jing WU). Markovian switching. As a direct application and also an origination, controller design for this type of hybrid SDE has been an important issue Yuan & Mao 4, Shi, Xia, Liu & Rees 6, Shi, Boukas & Agarwal 1999, Shi & Boukas 1997). It should be noted that these design methods are in linear case, which means much strict conditions imposed on the practical controlled system such as global Lipschitz condition or linear growth condition. For nonlinear control a breakthrough came in 199s: backstepping, a recursive design for systems with nonlinearities non constrained by linear bound Krstić, Kanellakopoulos & Kokotović 1995, Marino & Tomei 1995, Tong & Li 7). Backstepping designs for systems with stochastic disturbance were firstly proposed by Krstić & Deng 1998, Pan & Başar 1999) and were further developed by the recent work of Liu & Zhang 6, Liu, Zhang & Jiang 7, Tian & Xie 6, Xie & Tian 7, Wu, Xie & Zhang 7). The purpose of this paper is to design nonlinear controllers for stochastic strict-feedback systems with Markovian switching. The main work consists of the following aspects: By adopting the backstepping control design used in Preprint submitted to Automatica 3 November 8

2 Krstić & Deng 1998, Wu et al. 7), Wiener process can be well dealt with. While in the infinitesimal generator of Lyapunov function there appear the interconnected items caused by Markovian switching see )). They can not be suppressed by summing up all the single Lyapunov functions as in Wu, Xie & Zhang 4) because the infinitesimal generator is not a linear operator about Lyapunov function. Under the standard assumption that the finite homogeneous Markov process is irreducible, we can suppress the interconnections in the expectation of the infinitesimal generator and then obtain the result that the expectation of Lyapunov function can be bounded by exponential functions of time see 5) and 8)). This leads to two new problems how to guarantee the existence and uniqueness of strong solution to closedloop system and how to prove the stochastic stability of this solution based on these inequalities. As mathematic preliminaries, Lemma 1 and Theorems 1, are presented, which are different from the corresponding results in Yuan & Mao 4). State-feedback and output-feedback backstepping controllers are designed for stochastic nonlinear systems 17) and 3), respectively, which are proved to be robust against the irreducible homogenous Markovian switchings. The paper is organized as follows: Section begins with the mathematical preliminaries. In section 3 and 4 the state-feedback and the output-feedback backstepping controller are designed, respectively. Finally, the paper is concluded in Section 5. Notations: The following notations are used throughout the paper. For a vector x, x denotes its usual Euclidean norm, x T denotes its transpose and x i = x 1,, x i ) T. R + denotes the set of all nonnegative real numbers; R n denotes the real n-dimensional space; R n r denotes the real n r matrix space. C i denotes the set of all functions with continuous ith partial derivative; C,1 R n R + S; R + ) denotes the family of all nonnegative functions V xt), t, rt))) on R n R + S which are C in x and C 1 in t. Mathematical preliminaries In this section, we will extend the existence and uniqueness theorem of strong solution, the criteria on asymptotic stability in probability and boundedness in probability to hybrid stochastic nonlinear systems, which is required for backstepping control design. Consider the following stochastic nonlinear system with Markovian switching dxt) = fxt), t, rt))dt + gxt), t, rt))dw t), 1) where xt) R n is the state of system. W t) is an m- dimensional independent standard Wiener process or Brownian motion). The underlying complete probability space is taken to be the quartet Ω, F, F t, P ) with a filtration F t satisfying the usual conditions i.e., it is increasing and right continuous while F contains all P - null sets). Let rt) be a right-continuous homogeneous Markov process on the probability space taking values in a finite state space S = {1,,..., N} with generator Γ = γ pq ) N N given by P pq t) = P {rt + s) = q rs) = p} { γpq t + ot) if p q = 1 + γ pp t + ot) if p = q ) for any s, t. Here γ pq > is the transition rate from p to q if p q while γ pp = N q=1,q p γ pq. We assume that the Markov process rt) is independent of the Brownian motion W t). For nonlinear controller design, other standard assumptions on rt) will be given in the end of this section. The following hypothesis is imposed on the the Borel measurable functions f : R n R + S R n and g : R n R + S R n p. H: Both f and g are locally Lipschitz in x R n for all t, namely, for any R >, there exists a constant C R such that fx 1, t, p) fx, t, p) + gx 1, t, p) gx, t, p) C R x 1 x for any t, p) R + S and x 1, x ) U R = {ξ : ξ R}. Moreover, f, t, p) = g, t, p) =. For V x, t, rt)) C,1 R n R + S; R + ), introduce the infinitesimal generator by LV x, t, p) = V t x, t, p) + V x x, t, p)fx, t, p) + 1 Tr [ g T x, t, p)v xx x, t, p)gx, t, p) ] + N q=1 γ pqv x, t, q), where V t x, t, p) = V x,t,p) x n 3) V x,t,p) V x,t,p) t, V x x, t, p) = x 1,...,. Just for the convenience of the reader we cite the following useful property proposed by Mao & Yuan 6, Lemma 1.9): Let V x, t, rt)) C,1 R n R + S; R + ) and τ 1, τ be bounded stopping times such that τ 1 τ a.s. If V x, t, rt)) and LV x, t, rt)) are bounded on ), V xx x, t, p) = V x,t,p) x p x q )n n

3 t [τ 1, τ ] a.s., then E[V x, τ, rτ )) V x, τ 1, rτ 1 ))] = E τ τ 1 LV x, t, rt))dt. 4) Definition A stochastic process xt) is said to be bounded in probability if the random variables xt) are bounded in probability uniformly in t, i.e., lim sup P { xt) > R} =. 9) R t>t The following statement about the existence and uniqueness of strong solution to a jump stochastic differential equation and the line of its proof are originated from Mao & Yuan 6, Theorem 3.19). Lemma 1 Let H holds for system 1). For any l >, define the first exit time η l as The asymptotic stability criterion is given as follows. Theorem 1 Assume that system 1) has a uniqueness solution in almost surely sense in t [t, ) and that there exist a positive function V C,1 R n R + S; R + ) and parameters D > and c > such that η l = inf{t : t t, xt) l}. EV x, t, rt)) De ct t), 1a) Assume that there exist a positive function V x, t, rt)) C,1 R n R + S; R + ) and parameters d and D such that EV x, η l t, rη l t)) De dη l t t ), 5a) R = V R = inf V x, t, rt)). 5b) t t, x >R Then for every xt ) = x R n and rt ) = i S, there exists a solution xt) = xx, i ; t, rt)), unique up to equivalence, of system 1). Proof. By Mao & Yuan 6, Theorem 3.15), the locally Lipschitz condition guarantees that there exists a unique maximal solution xt) on [t, η ), where η is the explosion time. Following the same line as in Mao & Yuan 6, Theorem 3.19), from 5a) and 5b) we can obtain η =, a.s., 6) which implies the result of this theorem. The following definitions about stability that were proposed by Khas minskii 198), are represented now for the research on stochastic systems with Markovian switching. Definition 1 The equilibrium xt) = of 1) is said to be weakly) stable in probability if, for every ε > and δ >, there exists an r such that if t > t, x < r and i S, then P { xt) > ε} < δ. 7) asymptotically stable in probability in the large if it is stable in probability and, for each ε >, x R n and i S there is lim P { xt) > ε} =. 8) t V R = sup V x, t, rt)) R. t t, x <R 1b) Then for any x R n and i S, the equilibrium xt) = is asymptotically stable in probability in the large. Proof. From 1b), it can be inferred that for each ε > there exists a ε = εε) such that xt) ε when V xt), t, rt)) ε, while for any ε, δ > there exists an r = rε, δ) such that V x, t, i ) < 1 D εδ when x < r. Then by Chebyshev s inequality, from 1a), it follows that P { xt) > ε} P {V x, t, rt)) > ε} DEV x, t, i )e ct t) 11) ε < δ, which means that xt) = is stable in probability. By 11), it is obvious that 8) is valid for each ε >, x R n and i S, which implies that the equilibrium is asymptotically stable in probability in the large. The criterion for boundedness in probability is given as follows. Theorem Assume that system 1) has a uniqueness solution in almost surely sense in t [t, ) and that there exist a positive function V C,1 R n R + S; R + ) and parameters d c > such that EV x, t, rt)) d c 1) and 5b) hold. Then for any x R n and i S, the solution of system 1) is bounded in probability. Proof. By Khas minskii 198, Lemma 1.4.1), from 1), it follows that P { xx, i ; t, rt)) > R} EV x,t,rt)) inf x >R,t t V x,t,rt)) dc V R, 13) 3

4 which, together with 5b), means that 9) holds. For controller design by using backstepping techniques, we further assume, as a standard hypothesis, that rt) is irreducible Please see Mao 1999, P.183)). The algebraic interpretation of irreducibility is rank Γ) = N 1. This means that the Markov process admits a unique stationary distribution π = π 1,, π N ), which is also a limit distribution and can be obtained by solving πγ =, N π p = 1, and π p >, p S. 14) p=1 Consider how to generate a stationary Markov process based on stationary limit) distribution π Ross 1996, P.59). Since rt) is a homogenous finite irreducible Markov process, which means that it is ergodic, then one can suppose that it started at the moment t =. Such a process will be stationary, i.e, it satisfies P rt) = p) = N π l P lp t) = π p, t, p S. 15) l=1 Another method is to choose the initial state according to stationary distribution, that is, if r) satisfies π, then 15) holds, which is exactly what we pursue in this paper. In fact, these two approaches are consistent with each other from the viewpoint that the initial state of a new Markov process is the final state of an old one with the same distribution. For V x, t, rt)) C,1 R n R + S; R + ), according to Ross 1996, P.1), we have EV x, t, rt)) = N p=1 EV x, t, p)π p, ELV x, t, rt))) = N p=1 ELV x, t, p))π p, 16) as long as the expectations involved exist and are finite. Remark 1 For the same Markov process, transition probability ) and unconditional probability 15) are introduced. The former is used to define infinitesimal generator 3), and the latter is used in the calculus of expatiations such as 16). 3 State-feedback backstepping controller design Consider the stochastic nonlinear systems with Markovian switching as follows dx i = x i+1 dt + ϕ i x i, rt))dw t), i = 1,, n 1, dx n = udt + ϕ n x n, rt))dw t). 17) where x = x n = x 1,, x n ) T R n, u R are the state, the input of system, respectively. The known function ϕ i is smooth and ϕ i, rt)) =. Let rt) be a rightcontinuous homogeneous irreducible Markov process in a state space S = {1,,..., N} with generator Γ = γ pq ) N N and assume that initial state r) = i satisfying stationary distribution π given by 14). The explanations about Wiener process and probability space are as same as those for system 1). The objective of this section is to design a statefeedback controller such that the equilibrium of the closed-loop system is asymptotically stable in probability in the large. Remark When rt) equals to a constant, system 17) will reduce to system 3.8),3.9) of Krstić & Deng 1998). For a smooth controller u = ux, rt)) with u, rt)) =, the right side of system 1) satisfies local Lipschitz condition H. 3.1 Controller design Introducing the following transformation z 1 = x 1, z i x i, rt)) = x i α i 1 x i 1, rt)), i =,..., n, where smooth function α i 1 will be designed later. Choose the Lyapunov function candidate V x, rt)) = n 18) 1 4 z4 i x i, rt)). 19) For the simplicity, we introduce the notions ϕ ip = ϕ i x i, p), z ip = z i x i, p), α ip = α i x i, p), u p = ux, p), p S. According to 3), we have LV x, p) = n 1 +z 3 npu p n 1 1 n 1 j,k=1 ϕt jp z3 ip z i+1,p α n 1,p j=1 x j x j+1 α n 1,p x j x k ϕ kp ) + n 1 z3 ip α ip i 1 j=1 1 i 1 j,k=1 ϕt α i 1,p jp + 3 x j x k ϕ kp ) n z ip ϕ ip i 1 ϕ ip i 1 j=1 j=1 α i 1,p x j x j+1 α i 1,p x j x j+1 ) T α i 1,p x j x j+1 ) + N q=1 γ pqv x, q). ) Compared ) with 3.4) of Krstić & Deng 1998), the only difference is N q=1 γ pqv x, q) caused by Markovian 4

5 switching. By ignoring this interconnected item and following the same line of this work by M. Krstić et al., we obtain the controller α 1 = c 1 z d z 1 3 z 1β T 11β 11 3m 4 z n 1 k= α i = c i z i + i i 1 j,k=1 k 1 l=1 d k1l, α i 1 l=1 x l x l+1 α i 1 x j x k ϕ T j ϕ k 3 4 d 4 3 i z i 4d 4 i 1 z i 3 z iβ T ii β ii 3β T ii i 1 i 1 l=1 3 4 z r i j=1 k=1 n k 1 k=i+1 3m 4 u = c n z n + n n 1 j,k=1 l=1 α n 1 3 z nβ T nnβ nn 3β T nn 3 4 z n r j=1 l=1 d kil, α n 1 x l x l+1 i 1 k=1 z kβ ik 1 d ikl β ikj β ilj x j x k ϕ T j ϕ k 1 n 1 k=1 z 4d 4 n n 1 i 1 n 1 l=1 k=1 z kβ nk 1 β d nkj β nlj, nkl 1) where c i, d i, d ikj > are design parameters with d ikj = d kij ; smooth function β ik x i, rt)) = ψ ik x i, rt)) i 1 α i 1 l=k x j ψ lk x k, rt)), k = 1,, i, with αi 1,p x i = and β ikj is the j-th component of the vector β ik ; and ψ ik is smooth function defined by ϕ i x i, rt)) = i z k ψ ik x i, rt)). k=1 Then the infinitesimal generator of the system satisfies: LV x, p) n c iz 4 ip + N q=1 γ pqv x, q) cv x, p) c 4 n i= z4 ip N q=1 γ pq n i= z4 iq. ) where c = min{4c 1, c,..., c n } and c = 3 min n i={c i }. 3. Stability analysis Theorem 3 By choosing the design parameters c,, c n appropriately, the closed-loop system consist of 17) and 1) has a unique solution, and xt) = is asymptotically stable in probability in the large for any i S satisfying distribution π and every x R n. Proof. From 18) and 1), step by step, we can conclude that bounded z i implies bounded x i, and vice versa, which results in that V R = inf V x, rt)) R. 3) t t, x >R For any l >, define the first exit time η l = inf{t : t t, xt) l}. Let t l = η l t for any t t. Since x ) < l in the interval [t, t l ] a.s., which, together with 3), implies that V x, r )) is bounded on [t, t l ] a.s. From ), it can be obtained that LV x, r )) is also bounded on [t, t l ] a.s. It comes from 16) and ) that ELV x, rt l ))) = N p=1 ELV x, p))π p c N p=1 π pev x, p) E N p=1 + N p=1 1 4 π p N q=1 γ pq n i= z4 iq ) cev x, rt l )), cπ p 4 n i= z4 ip 4) where c,..., c n are chosen such that c c = max N p=1 {πp} min N p=1 {πp} maxn q=1 { N p=1 γ pq}. According to formula 4), it is followed from 4) that EV x, rt l )) EV x, i ). 5) By Lemma 1, from 5) and 3), for every xt ) = x R n and rt ) = i S, there exists a solution xt) = xx, i ; t, rt)), unique up to equivalence, of the closedloop system consist of 17) and 1). The subsequent part is originated from line of Mao 1997, Theorem 4.4). From 6), one has η l a.s.) when l. Again from 4) and 4), we obtain that Ee ct l V x, rt l ))) e ct EV x, i )) + E t l t e cs LV x, rs)))ds +ce t l t e cs V x, rs))ds, which together with 4) implies that 6) EV x, rt l )) e ct l t ) EV x, i ). 7) Letting l gives EV x, rt)) e ct t) EV x, i ). 8) From 18) and 1), step by step, we can obtain R V R = sup V x, rt)). 9) t t, x <R From 8), 9) and Theorem 1, we can conclude that xt) = is asymptotically stable in probability in the large for any x R n and i S, which completes the proof. Remark 3 By comparison, our controller is in the same form as that in Krstić & Deng 1998) except for that the design parameters c,, c n should be large enough. In other words, we can design controller for system 17) by regarding rt) as constant and tuning our parameter c i appropriately. 5

6 3.3 A simulation example Consider system 17) n = ) with ϕ 1 = x 1t) + x 1 t)rt), ϕ = x 1t) + x t) + x t)rt) where W t) being a scaler Wiener process. The state-feedback control law is given by 18) and 1) with β 11 = x 1 + rt), β 1 = x 1 α1p x 1 and β = x + rt). rt) is a homogenous irreducible Markov process which belongs to the space S = {1, } with generater Γ = γ pq ) given by γ 11 = 4, γ 1 = 4, γ 1 = 3 and γ = 3, which means that π 1 = 3 7, π = 4 7. Choose the initial values x 1 ) = 1, x ) = 1, r) = 1, the design parameters c 1 = 1.1, c = 1.1 satisfying 3.3 = c > c = 4 3 ), d 1 =.4 and d = 1. Figure 1 demonstrates that the state of State A1: For each 1 i n, there exist unknown constants ďi, ˆd i such that i x, rt)) φ i y, rt)) + ďi, Ω i x, rt)) ϕ i y, rt)) + ˆd i, where φ i, ϕ i are known nonnegative smooth functions with φ i, rt)) = ϕ i, rt)) =, which means that there exist smooth functions φ i, ϕ i such that φ i y, rt)) = y φ i y, rt)), ϕ i y, rt)) = y ϕ i y, rt)). Remark 4 When i and Ω i are known and all the states x i are measurable, system 3) is reduced to 17). From A1 the static uncertainties are nonlinear parameterized, which is considered by Jiang & Praly 1998, Jiang 1999) in the deterministic case and by Wu et al. 7) in the stochastic case without Markovian switching. A detailed example satisfying A1 can be found in Subsection Controller design Control Since x i i =,, n) is unavailable, as in Jiang 1999), the following reduced-order observer is introduced Fig.1. The responses of closed-loop system with Markovian switching. nonlinear system with Markovian switching can be regulated to the origin asymptotically by the same controller designed for non switching case. 4 Output feedback backstepping controller design Consider the stochastic nonlinear systems with Markovian switching as follows dx i = x i+1 dt + i x, rt))dt + Ω i x, rt))dw i t), i = 1,, n 1, dx n = udt + n x, rt))dt + Ω n x, rt))dw n t), y = x 1, 3) where y R is the output; the uncertain functions i and Ω i are locally Lipschitz; other explications are as same as that for system 17). The objective of this section is to design an outputfeedback controller such that the solution of the closedloop system is bounded in probability in biased case and the equilibrium is asymptotically stable in probability in the large in unbiased case. The following assumption is made on system 3). dˆx i = ˆx i+1 + k i+1 y k i ˆx 1 + k 1 y))dt, 1 i n, 31) dˆx n 1 = u k n 1 ˆx 1 + k 1 y))dt, where k = k 1,, k n 1 ) T is chosen such that A = ) I n k is asymptotically stable. By 3) and 31), the observer error ε defined by ε = ε 1, ε,, ε n 1 ) T, 3) ε i = x i+1 ˆx i k i x 1, satisfies dε = A εdt + x, rt))dt + ĀΩx, rt))dw t), 33) where = 1,, n 1 ) T, W t) = W 1 t),, W n t)) T, Ā = A e n 1 ), e n 1 =,,, 1) T R n 1, i = i+1 k i 1, Ω = diagω 1,, Ω n ). From assumption A1, we have i x, rt)) ψ i y, rt)) + d i, 34) where ψ i = φ i+1 + k i φ 1, and d i = ďi+1 + k i ď 1. From 3) and 3), one gets dy = ˆx 1 + ε 1 + k 1 y)dt + 1 dt + Ω 1 dw 1 t), 35) 6

7 which together with 3), 33) and 35) consist of the following interconnected system dε = A εdt + dt + ĀΩdW t), dy = ˆx 1 + ε 1 + k 1 y)dt + 1 dt + Ω 1 dw 1 t), dˆx i = ˆx i+1 + k i+1 y k i ˆx 1 + k 1 y))dt, 1 i n, dˆx n 1 = u k n 1 ˆx 1 + k 1 y))dt, 36) Next, we will develop an output-feedback controller along the y, ˆx 1,, ˆx n 1 )-system of 36) step by step using backstepping techniques. From assumption A1, one has r ε T P + yε ) + 1 Ω 1 + Tr[ĀΩ) T P ĀΩ] 1 4 r ε + 8r P n 1 +8r P n 1 ψ i y d i r ε + ψ1 + 1 r ) y +d 1 y + 1 4d 1 ď 1 + ϕ 1 + P Ā + ˆd 1 + P Ā n = 1 r ε + z 1 Ψ 1 + d 1 y + λ 1, ˆd i n ϕ i )y 4) Let us introduce the following notions for the use of the recursive design Ξ 1 = y, ε) T, Ξ i = y, ˆx 1,..., ˆx i 1, ε) T, X 1 = y, X i = y, ˆx 1,..., ˆx i 1 ) T, i = 1,, n with Ξ = Ξ n, X = X n ). Introducing the following transformations where d 1 > is any design parameter, λ 1 = ˆd d 1 ď 1 + P Ā n ˆd i + 8r P n 1 d i and Ψ 1y, rt)) is given by Ψ 1 = 8r P n 1 ψ i y + ψ1 + 1 r ) y + ϕ 1y + P Ā n ϕ i y, where ψ i = φ i+1 +k i φ1. Substituting z 1 z 3 4 z 1 3 z z4 and 4) into 39) gives z 1 = y, z i X i, rt)) = ˆx i 1 α i 1 X i 1, rt)), i =,..., n, 37) LV 1 Ξ 1, p) 1 4 z4 p + y 3 4 z 1 3 z α 1p + k 1 y + Ψ 1p ) + d 1 y 1 r ε + λ 1. 41) where the smooth function α i will be designed later. Just for the simplify, let us further introduce the notions 1p = 1 x, p), Ω 1p = Ω 1 x, p), α ip = α i X i, p), z ip = z i X i, p). where Ψ 1p = Ψ 1 y, p). The stabilizing function α 1 X 1, rt)) is designed as α 1 = 3 4 z 1 3 z k 1y c 1 y Ψ 1. 4) From 41) and 4), it follows that Step 1. Let us consider the Lyapunov function candidate V 1 Ξ 1, rt)) = 1 y + r ε T P ε, 38) where P is the solution of P A + A T P = I n 1, r > is design parameter. In view of 36) and 37), the infinitesimal generator of V 1 satisfies LV 1 Ξ 1, p) = yz p + α 1p + k 1 y + ε 1 + 1p ) + 1 Ω 1p r ε + r ε T P x, p) +Tr[ĀΩx, p)) T P ĀΩx, p)]. 39) LV 1 c 1 z z4 1 4 r ε + d 1 y + λ 1. Step i =,, n. Assume that one has designed smooth function α j j i 1) such that the infinitesimal generator of V i 1 = V i z4 i 1 satisfies LV i 1 Ξ i 1, p) c 1 z1 i 1 j= c jzjp 4 1 r i 1 ε z4 ip + i 1 j=1 d jy + i 1 j=1 λ j N q=1 i 1 j= γ pqz 4 jq, p S, 43) where d i, c i > are design parameters. In the sequel, we will prove that 43) holds for the i-th Lyapunov function 7

8 candidate V i = V i z4 i. The infinitesimal generator of V i satisfies LV i Ξ i, p) LV i 1 Ξ i 1, p) α i 1,p Ω 1p + 3 z ip αi 1,p ) Ω 1p 1 z3 ip ) + zip 3 z i+1,p + α ip + η ip αi 1,p ε 1 + 1p ) 44) N q=1 γ pqziq 4, p S, where η ip = η i X i, p), η i X i, rt)) = k i y k i 1 ˆx 1 + k 1 y) αi 1,p ˆx 1 + k 1 y) i α i 1,p j=1 ˆx j ˆx j+1 + k j+1 y k j ˆx 1 + k 1 y)). Following the similar procedure as in the initial step, one has zi 3 αi + 3 z αi 1 i ε ) ) Ω i r ε + z 6 i αi 1 z3 α i 1 i Ω 1 ) ) ψ 1 d i1 + 1 d i + i r + di1 y + d i ď 1 + di3 y + 1 d i3 9zi 4 αi 1 )4 +zi 6 α i 1 ) ) ϕ 1 ) 4 y + 1 d i4 9zi 4 αi 1 )4 +zi 6 α i 1 ) ) + 1 d ˆd i4 4 1 = 1 r i ε + Ψ i zi 3 + d iy + λ i, 45) where d i = 1 d i1 + d i3 ), λ i = 1 d i ˆd d i ď 1 and Ψ i X i, rt)) is defined as Ψ i = z 3 i + 1 αi 1 ) ) ψ 1 d i1 + 1 d i + i r d i3 9z i αi 1 )4 + zi 3 α i 1 ) ) ϕ 1 ) 4 y d i4 9z i αi 1 )4 + zi 3 α i 1 ) ). + 1 From 43)-45) and the fact zi 3z i z4 i z4 i+1, it is easy to show that LV i Ξ i, p) c 1 z1 i 1 c j z 4 jp 1 i r ε j= ) z ip + α ip + η ip + Ψ ip z4 i+1,p + 46) z3 ip +Σ i j=1 d jy + Σ i j=1 λ j + 1 N i 4 q=1 j= γ pqzjq 4, where Ψ ip = Ψ i X i, p). By choosing the virtue control α i X i, rt)) as α i = z i c i z i η i Ψ i, 47) it follows from 46) that 43) holds for V i. At the end of the recursive procedure, we obtain the control law u = α n X, rt)). 48) From 43) and 48), by selecting c 1 n j=1 d j, one gets LV n Ξ, p) cv n Ξ, p) c n 4 i= z4 ip + 1 N n 4 q=1 j= γ pqzjq 4 + d 49) a, p S, where d a = Σ n d iď 1+Σ n λ i, c = min{c 1, c,..., c n, 1 n } and c = 3 min n i={c i }. 4. Stability analysis Theorem 4 For any i S satisfying distribution π and every x R n, by choosing the design parameters c 1,, c n such that c 1 n j=1 d j and c c = maxn p=1 {πp} min N p=1 {πp} maxn q=1{ N p=1 γ pq}, the closed-loop system consist of 3) and 48) has a unique solution, which is bounded in probability. Furthermore, if ď i = ˆd i = is given, the zero solution of the closed-loop system is asymptotically stable in probability in the large. Proof. From 37) and 48), step by step, we can conclude that R V R = inf V nξ, rt)). 5) t t, Ξ >R As in the proof of Theorem 3, from 16) and 49), one has ELV n Ξ, rt l ))) cev n Ξ, rt l )) + Nd a, 51) then the existence and uniqueness of solution to the closed-loop system consist of 3) and 48) can be obtained. Continuing following the line of Theorem 3, it comes from 51) that EV n Ξ, rt)) EV n Ξ, i )e ct t) + Nd a /c, 5) which implies that EV n Ξ, rt)) d c 53) where d c = EV n Ξ, i ) + Nd a /c. From 53), 5) and Theorem, we can prove that the solution of the closedloop system is bounded in probability for any initial conditions, which implies the first result of this theorem. When ďi = ˆd i =, 5) reduces to EV n Ξ, rt)) EV n Ξ, i )e ct t). 54) From 37) and 48), step by step, we can obtain 5) and V R = sup V n Ξ, rt)) R. 55) t t, Ξ <R 8

9 From 54), 55) and Theorem 1, we obtain the second result of this theorem. Remark 5 To overcome the observation errors and adaptive errors, the quadratic form Lyapunov functions are used in the initial step of backstepping design. To deal with the effect of Hessian items, the quartic Lyapunov functions are remained in the subsequent steps. 4.3 A simulation example Let n = in system 3). Chose 1 = 1 + rt))x 1 sin x + ď1, Ω 1 = x 1 sin x + ˆd 1, = x 1 cos x + ď, Ω = rt) 1)x 1 cos x + ˆd with ď1, ˆd 1, ď and ˆd being unknown parameters. The following observer is needed ˆx = u kˆx + ky), where k >. Perform the transformation 37) and output feedback control 48). By verifying assumption A1, it is easy to obtain that φ 1 = 1 + rt))x 1, φ = x 1, ϕ 1 = x 1, ϕ = rt) 1)x 1. To check the main results in Theorem 4, the following two cases are considered. Biased case: ď 1 = ď = ˆd 1 = ˆd = 1. Unbiased case: ď 1 = ď = ˆd 1 = ˆd =. The homogenous irreducible Markov process rt) belongs to the space S = {1, } with generater with generater Γ = γ pq ) given by γ 11 = 4, γ 1 = 4, γ 1 = 3, γ = 3. Choose the initial values x 1 ) =.5, x ) = 4, r) = 1, the design parameters k = 1.6, d 1 =.1, d 1 = d 3 = 1, d = d 4 = 1, c 1 =.5, c = satisfying c 1 > d 1 + d ) and 6 = c > c = 4 3 ), r =.5 and ˆx) = 3.3. Figures shows the responses of the closed-loop system in biased cases; Figures 3 shows the responses of the closed-loop system in unbiased cases. From Figures we can see that the residual errors remain in all the solutions of closed-loop system because the non-vanishing biased parameters prevent the equilibrium at zero. Figures 3 demonstrates that the solutions of the closed-loop system can be regulated to the origin asymptotically in unbiased case. 5 Conclusion Two important issues: backstepping and hybrid diffusion are well combined so that the stochastic nonlinear control has been improved to a new level, in which the mode studied has the capability to describe complex systems not only with exterior random perturbations but also with inner jumping parameters. Meanwhile, the novel existence theorem and stability criterion of solution are proposed to SDE with Markovian switching for nonlinear controller design without linear growth condition. States Control Fig.. The responses of closed-loop system in biased case. States Control Fig.3. The responses of closed-loop system in unbiased case. Acknowledgements This work is supported by Program for the New Century Excellent Talents in University of China No: NCET-5-67) and the National Natural Science Foundation of China 67741)., Program for Summit of Six Types of Talents of Jiangsu Province 7-A-), and Program for Fundamental Research of Natural Sciences in Universities of Jiangsu Province 7KJB51114) and Engineering and Physical Sciences Research Council, UK EP/F9195). We would like to thank Dr. Yu Xin for helpful comments and thank the reviewers and the editors for suggestions in improving the paper. References Arnold, L. 197). Stochastic Differential Equations: Theory and Applications, Wiley, New York. Basak, G. K., Bisi, A. & Ghosh, M. K. 1996). Stability of a random diffusion with linear drift, Journal of Mathematical Analysis and Applications, Friedman, A. 1976). Stochastic Differential Equations and Their applications, Academic Press, New York. Ji, Y. & Chizeck, H. J. 199). Controllability, stabilizability and continuous-time markovian jump linear quadratic control, IEEE Transactions on automatic control 35, Jiang, Z. P. 1999). A combined backstepping and small-gain approach to adaptive output feedback control, Automatica 35, Jiang, Z. P. & Praly, L. 1998). Design of robust adaptive controllers for nonlinear systems with dynamic uncertainties, Automatica 34, Khas minskii, R. Z. 198). Stochastic Stability of Differential Equations, S & N International publisher, Rockville, Maryland. 9

10 Krstić, M. & Deng, H. 1998). Stability of Nonlinear Uncertain Systems, Springer, New York. Krstić, M., Kanellakopoulos, I. & Kokotović, P. 1995). Nonlinear and Adaptive Control Design, John Wiley, New York. Liu, S. J., Zhang, J. F. & Jiang, Z. P. 7). Decentralized adaptive output-feedback stabilization for large-scale stochastic nonlinear systems, Automatica 43, Liu, Y. G. & Zhang, J. F. 6). Practical output-feedback risksensitive control for stochastic nonlinear systems with stable zero-dynamics, SIAM Journal of Control and Optimization 45, Mao, X. 1997). Stochastic Differential Equations and Applications, Horwood, New York. Mao, X. 1999). Stability of stochastic differential equations with markovian switching, Stochastic Processes and Their Applications 79, Mao, X. & Yuan, C. 6). Stochastic Differential Equations with Markovian switching, Imperial College Press, London. Marino, R. & Tomei, P. 1995). Nonlinear Control Design: Geometric, Adaptive and Robust, Prentice Hall, New Jersey. Pan, Z. G. & Başar, T. 1999). Backstepping controller design for nonlinear stochastic systems under a risk-sensitive cost criterion, SIAM Journal of Control and Optimization 37, Ross, S. M. 1996). Stochasic processes, John Wiley & Sons, Inc., New York. Shi, P. & Boukas, E. K. 1997). H-infinity control for markovian jumping linear systems with parametric uncertainties, Journal of Optimization Theory and Applications 95, Shi, P., Boukas, E. K. & Agarwal, R. 1999). Control of markovian jump discrete-time systems with norm bounded uncertainty and unknown delay, IEEE Transactions on automatic control 44, Shi, P., Xia, Y. Q., Liu, G. P. & Rees, D. 6). On designing of sliding-mode control for stochastic jump systems, IEEE Transactions on automatic control 51, Skorohod, A. V. 1989). Asymptotic Methods In The Theory Of Stochastic Differential Equations, American Mathematical Society, Providence RI. Tian, J. & Xie, X. J. 6). Adaptive state-feedback stabilization for high-order stochastic non-linear systems with uncertain control coefficients, International Journal of Control 79, Tong, S. & Li, Y. 7). Direct adaptive fuzzy backstepping control for a class of nonlinear systems, International Journal of Innovative Computing, Information and Control 3, Wu, Z. J., Xie, X. J. & Zhang, S. Y. 4). Robust decentralized adaptive stabilization for a class of interconnected systems with unmodeled dynamics, International Journal of Systems Science 35, Wu, Z. J., Xie, X. J. & Zhang, S. Y. 7). Adaptive backstepping controller design using stochastic small-gain theorem, Automatica 43, Xie, X. J. & Tian, J. 7). State-feedback stabilization for highorder stochastic nonlinear systems with stochastuic inverse dynamics, International Journal of Robust and Nonlinear Control 17, Yuan, C. & Mao, X. 4). Robust stability and controllability of stochastic differential delay equations with markovian switching, Automatica 4, Zhaojing WU received the M.S. and Ph.D. degrees from Qufu Normal University and Northeastern University, China, in 3 and 5, respectively. He is currently with the School of Mathematics and Information Science, Yantai University, China. His research interests include nonlinear, adaptive and stochastic control theory. Xuejun XIE received his Ph.D. degree from the Institute of Systems Science, Chinese Academy of Sciences in Now he is a professor of Qufu Normal University and Xuzhou Normal University, China. He received the Program for New Century Excellent Talents from Ministry of Education of China in 5. His current research interests include stochastic nonlinear control systems and adaptive control. Peng SHI received the ME degree in control theory from Harbin Engineering University, China in 1985; the PhD degree in electrical engineering from the University of Newcastle, Australia in 1994; and the PhD degree in mathematics from the University of South Australia in He was awarded the degree of Doctor of Science by the University of Glamorgan, UK, in 6. He joined in the University of Glamorgan, UK, in 4 as a professor. Dr Shis research interests include robust control and filtering, fault detection techniques, Markov decision processes, and optimization techniques. Dr Shi currently serves as Editor-in-Chief of Int. J. of Innovative Computing, Information and Control, and as Regional Editor of Int.J. of Nonlinear Dynamics and Systems Theory. He is also an Associate Editor/Editorial Member for a number of other journals, such as IEEE Transactions on Automatic Control, IEEE Transactions on Systems, Man and Cybernetics-B, and IEEE Transactions on Fuzzy Systems. Dr Shi is a Fellow of Institute of Mathematics and its Applications UK), and a Senior Member of IEEE. Yuanqing XIA received the M.S. and Ph.D. degrees from Anhui University and Beijing University of Aeronautics and Astronautics, China, in 1998 and 1, respectively. He is a professor of the Department of Automatic Control, Beijing Institute of Technology. His current research interests include networked control systems, active disturbance rejection control and biomedical signal processing. 1