The LaSalle Stability Theorem of General Stochastic Hybrid Systems

Joint 48th IEEE Conference on Decision and Control and 8th Chinese Control Conference Shanghai, PR China, December 16-18, 009 ThAIn38 The LaSalle Stability Theorem of General Stochastic Hybrid Systems Haijun Liu and iaowu Mu Abstract This paper deals with the LaSalle stability theorem of general stochastic hybrid systems In general the hybrid systems have discontinuous trajectories And such property brings some difficulty in stability analysis To overcome this difficulty a new concept named by finite-extraction systems is proposed For finite-extraction systems their trajectories vary little in a very short time Depending on this concept and the extended generator formula for GSHS, the LaSalle stability Theorem is extended to general stochastic hybrid systems I INTRODUCTION Hybrid systems have been studied extensively in the past decades [1-10] However, the field of stochastic hybrid systems (SHS) is rather young In [1], David introduced Piecewise-deterministic Markov Processes, which is such a hybrid system that the subsystems are deterministic, but the transitions are triggered by stochastic processes In [], Hu, Lygeros and Sastry proposed stochastic hybrid systems (SHS) The systems involve a hybrid state space, with both continuous and discrete states The continuous state obeys a SDE that depends on the hybrid state Transitions occur when the continuous state hits the boundary of the state space Whenever a transition occurs the hybrid state is reset instantly to a new value The value of the discrete state after transition is determined deterministically by the hybrid state before transition The new value of the continuous state, on the other hand, is governed by a probability law which depends on the last hybrid state Another important family of stochastic hybrid systems is switching diffusion process or Markovian jump stochastic process [3,4] The discrete state of switching diffusion process submits to a Markovian chain And the continuous state, on other hand, flows along the solution of an ordinary or stochastic differential equation; the dynamics of this differential equation may depend on the value of the mode at the given time Fortunately Bujorianu and Lygeros have developed a model named by General Stochastic Hybrid Systems [5](denoted by GSHS) which allows: 1 Diffusion processes in the continuous evolution; Spontaneous discrete transitions (according to a transition rate); 3 Forced transitions (driven by a boundary hitting time); 4 Probabilistic reset of the hybrid state as a result of discrete transitions Now this system have becomes a uniformly framework for stochastic hybrid systems From then many problems have been discussed under this new This work is supported by the National Natural Science Foundation of China under Grant 60874006 and the China Postdoctoral Science Foundation under Grant 007040813 H Liu is with the Department of Mathematics, University of Zhengzhou, 45005 Zhengzhou, China liuhaijun05@163com Mu is with the Department of Mathematics, University of Zhengzhou, 45005 Zhengzhou, China muxiaowu@zzueducn frame, such as reachability, optimal control and so on In this paper, the stability of GSHS (with its continuous state) is discussed On the stability of stochastic systems defined by Ito s processes, many authors have done prominent works Such as Has minskiǐ and Kushner extended the Lyapunov asymptotic stability theorem in [14-17] Florchinger discussed the stochastic Artstein theorem in [18,19] Deng and Krstic studied the backstepping design with stochastic systems in [0-] One of the important developments in stability theory is the LaSalle stability theorem [13], which establishes the relation between the asymptotic behavior of systems and their ω-it sets For the stability of Markovian jump processes, Mao, Yuan and Lygeros [11,4,4] investigated the exponential stability, asymptotic stability and boundedness Costa and Boukas [3] discussed the stability and stabilizability Khas minskii, Zhu and Yin discussed the asymptotic properties of switching diffusion systems in [17] Inspired by Mao[11] and Kushner[14,15], this note discusses the LaSalle s stability theorems for GSHS In stochastic stability theory the continuity of systems plays an important role But for GSHS their trajectories may be discontinuous So a new condition is needed to place the continuity Here a new concept named by finite-extraction systems is proposed It means that within small time the systems change very little Depending on this concept and the extended generator formula for GSHS, the LaSalle stability Theorem is extended to general stochastic hybrid systems II PRELIMINARIES ON STOCHASTIC STABILITY Firstly let us introduce GSHS given by Bujorianu and Lygeros in [5] General Stochastic Hybrid Systems (GSHS) are a collection of non-linear stochastic continuous-time hybrid dynamical systems They are characterized by a hybrid state defined by two components: a continuous state (denoted by x) and a discrete state (denoted by q) The hybrid state is (q, x) When the value of the discrete state is q, the domain of the continuous state is i Meanwhile, the continuous state is governed by a stochastic differential equation (SDE) that depends on the hybrid state The discrete state dynamics produces transitions in both (continuous and discrete) state variables Whenever a transition occurs, the hybrid state is reset instantaneously to a new value, according to a probability law depending on the pre-jump location A sample trajectory has the form (q t, x t, t 0), where (x t, t 0) is piecewise continuous and q t Q is piecewise constant We denote the domain of the stochastic hybrid system by = Dom H = q Q {q} q The boundary of the hybrid state space will 978-1-444-387-3/09/$500 009 IEEE 3969

ThAIn38 be = Dom H = q Q {q} q The closure of the hybrid state space will be Dom H = q Q {q} q The hybrid state will be x = (q, x) Definition 1: A general stochastic hybrid system is a collection H = ((Q, d, m, χ), b, σ, Init, λ, R), is given as follows: Q : {q 1, q,, q n } is a finite set of discrete states; d : Q N gives the dimensions of the modes; m: Q N gives the dimensions of the Wiener processes that govern the continuous state evolution; χ: Q R d( ) maps each q Q into an open subset q of R d(q) ; b : (Q, d, χ) R d( ) is a vector field; σ: (Q, d, χ) R d( ) m( ) is a ( )-valued matrix; Init : B() [0, 1] is an initial probability on ; λ: (Q, d, χ) R + is a transition rate function; R: B( ) [0, 1] is a transition measure Now we define the GSHS execution: Definition : A stochastic process x t = (q(t), x(t)) is called a GSHS execution if there exists a sequence of stopping times T 0 = 0 < T 1 < such that k N, x 0 = (q 0, x q0 0 ) is a Q -valued random variable extracted according to the probability measure Init; For t [T k, T k+1 ), q t = q Tt is constant and x(t) is a (continuous) solution of the SDE dx(t) = b(q Tk, x(t))dt + σ(q Tk, x(t))dw(t) (1) where w(t) is the m-dimensional standard Wiener processes; T k+1 = T k + S i k, where S i k is chosen according with the survivor function (); The probability distribution of x(t k+1 ) is governed by the law R((q Tk, x(t k+1 )), ); The executions of the GSHS can be thought of as being generated by the following algorithm Algorithm (GSHS Executions): set T = 0 select -valued random variable ˆx according to Init repeat set i = χ 1 (ˆx) select R + -valued random variable Ŝ such that Ŝ = inf{t > 0 F(t, ) e cit } () set x t as solution of (1) with initial condition equal to ˆx, for all t [T, T + Ŝ) select -valued random variable ˆx according to R(, xŝ) set T = T + Ŝ until true All random extractions in Algorithm 1 are assumed to be independent In [5], the authors give the regular conditions for GSHS as follows (i) For all i the functions b(i, ) : i R d(i) and σ(i, ) : i R d(i) m(i) are bounded and Lipschitz continuous (ii) λ : R + is a measurable function such that t λ(x i t (ω i)) is integrable on [0, ε(x i )), for some ε(x i ) > 0 for each x i i and each ω i starting at x i (iii) For each i Q the restriction of λ to i is bounded Let c i = sup λ(x i ) x i i (iv) For all A B(), R(, A) is measurable and for all x the function R(, A) is a probability measure (v) Denote the number of jump times in the interval [0, t] by N t (ω) For every starting point x, EN t <, for all t R + Under conditions as above, any GSHS defines: 1 a Borel right process; a cádlág process, ie for all ω the trajectories t x t (ω) are right continuous on [0, ) with left its on (0, ) [5]; 3 it is a Markov process and has strong Markov property The Process Generator We denote by B b () the set of all bounded measurable functions f: R This is a Banach space under the norm f = sup f(x) Let (P t ) be the semigroup of x the whole Markov process (x t ), P t f(x) = E x f(x t ) = E x {f(x t ) t < ς}, where f is bounded B-measurable function and ς is the lifetime when the process retires to, ie, ς := inf{t x t = } Associated with the semigroup (P t ) is its strong generator which, loosely speaking, is the derivative of P t at t = 0 Let D(L) B b () be the set of functions f for which the following it exists 1 tց0 t (P tf f) (3) and denote this it Lf The it refers to convergence in the norm, i e for f D(L) we have tց0 1 t (P tf f) Lf = 0 (4) Specifying the domain D(L) is an essential part of specifying the operator L Lemma 1[5] Let H be an GSHS as in definition 1 Then the domain D(L) of the extended generator L of H, as a Markov process, consists of those measurable functions f on satisfying: 1 f : R, B-measurable; t f(x i t (ω i )) should have second order derivatives on [0, S i (ω i )), for all ω i Ω i ; Boundary condition f(x) = f(y)r(x, dy)), x ; (5) 3 Bf L loc 1 (p), where Bf(x, s, ω) : +f(x) f(x s (ω)) : (6) For f D(L), Lf is given by Lf(x) = L cont f(x) + λ(x) (f(y) f(x))r(x, dy) (7) where L cont f(x) = L b f(x) + 1 Tr(σ(x)σ(x)T H f (x)) (8) 3970

ThAIn38 Following [6], for A B( ) define p, p and p as follows: p(t, A) = I (t Tk )I (xtk A); p (t) = I (xt ); t T k k p(t, A) = t 0 R(x s, A)λ(x s )ds + t 0 R(x s, A)dp (s) = R(x T, A) k T k t Note that p, p are counting processes, p (t) is counting the number of jumps from the boundary of the process (x t ) p(t, A) is the compensator of p(t, A) The process q(t, A) = p(t, A) p(t, A) is a local martingale Lemma (GSHS Differential Formula)[6]: If f satisfies the conditions 1 and 3 of the Th Then t 0 f(x t ) f(x) = t 0 Lf(x s)ds + t 0 g(x s) f(x s )dw + (s) + t 0 Bfq(s, du)ds + t 0 Cf(x s )dp (s) where Cf(z) := f(y)r(z, dy) f(z), z (9) To discuss the stability of a GSHS, we assume d(i) = d, i Q and x(t; 0) 0 is its equilibrium point Definition III1 (Finite Contraction): The GSHS H is called finite contraction to a nonnegative function W( x) : x R, if for every positive ρ, there exists constants µ(ρ) > 0, k(ρ) > 0 and p 0 (ρ) > 0, depending only on ρ such that inf P t >0 0, x{ sup W( x(s)) k(ρ)} p 0 (10) +µ s when W( x 0 ) > ρ In following, when the GSHS H is called finite contraction to a nonnegative function W( x) : x R, we also say the function W( x) is finite-contraction to system GSHS H III STABILITY IN PROBABILITY A solution x(t, ω) of equation (1,) is said to be stable in probability for t if for any s >, i 0 {1,, N}and ε > 0 P {sup x(t,, x 0, i 0 ) > ε} = 0 x 0 t>s Theorem 31: For GSHS H, assume there exists a function V (x, i, t), i = 1,, N, which is positive definite and twice differentiable in x, proper and once differentiable in t, satisfies LV (x, i, t) 0 and CV ( x) 0 for x Then the continuous state of the solution of equation (1,) is stable in probability Proof: Let ρ be any positive number, and let Q ρ be the neighborhood {x : x ρ} Set = inf V (x, i, t) Let τ Qρ = inf{t : x(t) > ρ} By x Q ρ,i Q,t> Lemma 1 EV (x(τ t), r(τ t), τ t) V (x 0, i 0 ), for x 0 < ρ Using this result and Chebyshev inequality, we get P { sup x(u,, x 0, i 0 ) > ρ} u t Letting t, we finally have P {sup u x(u,, x 0, i 0 ) > ρ} EV (x(τ t),r(τ t),τ t) V (x0,r0) V (x0,r0) Since V (0, i, t) = 0 and V (x, i, t), i = 1,, N are continuous, this implies the desired assertion Corollary 31: Under the conditions of Theorem 31, the solutions of (1) are bounded in the sense that P {sup x(t) < ρ} = 1 ρ t IV THE LASALLE STABILITY THEOREM In this section we discuss the asymptotic behavior in the sense of almost surely convergence Theorem 41: For GSHS H, let (i)-(v) in Section II hold Assume it is finite contract If there exists a function V (x, q, t), which is continuously twice differentiable in x and once in t, satisfying V (0, q, t) = 0, V (x, q, t) > 0, for x 0, and a nonnegative continuous function W : R n S R + which is finite-contraction with system H, such that and x inf V (x, q, t) = (11) q Q,t R + LV (x, q; t) W(x, q) < 0, (x, q; t) ( /{0} [, + )), (1) and CV (z) := V (y)r(z, dy) V (z) 0, z, (13) then for x 0 R n, r 0 S, W( x(t; x 0)) = 0, as (14) Proof of Theorem 41: Fix any initial value x 0 = φ 0 and for simplicity write x(t;, x 0 ) = x(t) By Lemma and condition (1) and (13), it follows that V ( x(t), t) = V ( x 0, ) + t LV ( x(s), s)ds + t 0 CV (x s )dp (s) + t V x ( x(s), s)g( x(s), s)dw(s) + t 0 BV q(s, du)ds V ( x 0, ) t w( x(s))ds + t + t V x ( x(s), s)g( x(s), s)dw(s) + t 0 BV q(s, du)ds 0 CV (x s )dp (s) Since w( x(s)) 0 and CV (z) 0, taking expectation and noticing the last two terms in the left are martingales, we obtain, t E W( x(s))ds V ( x 0, ) (15) 3971

ThAIn38 for every t Hence for almost every ω Ω, W( x(t, ω))dt < (16) For every positive number ε, let τ(t, φ) denote the total time such that W( x(t)) > ε in [t, ] Then by (15), τ(t, φ) is almost surely finite and τ(t, φ) = 0 for any ε > 0 We prove by contradiction Assume that Theorem 41 is not true Because S is finite set and function W(x, q) is continuous, there exist an event A and positive numbers ε 0 and p 0 such that P(A) = p 0 > 0 and for every ω A, W( x(t)) > ε 0 (17) holds By (1) and (13), the system x(t) is ultimate uniformly bounded Hence there is a positive number ρ (related to x 0 ) such that P t0, x 0 { sup s<+ x(t) ρ} > 1 p 0 (18) Let Q ǫ = {x : x ǫ} for some ǫ > 0 Let B 1 = {ω : x(t) ρ} From (16) and (17), let B = A B 1 sup s + Then P(B) > p0 and for every ω B, W( x(t)) > ε 0, and x(t) Q ρ, for t (19) hold For a positive number ε, let E ε = { x : W( x) ε} Q ρ By the finite contract of H, choose k(ε) > 0, µ(ε) > 0 and p(ε) > o as defined in Definition III1 Here one can choose k(ε) < ε Then, by uniform continuity of W( x) on Q ρ, for ε 0 defined in(16), the distance between Q ρ Eε c and E k(ε) ( Eε c is the complement of E ε ) is positive (say δ(ε)) for 0 ε ε 0, and Q ρ Eε c is not empty Define a sequence of Markov times τ n, τ n as follows (if τ n or τ n is not defined at a sample ω, set it equal to there) τ 0 =, τ 0 = inf{t : x(t) E k(ε 0), t τ 0 }, τ 1 = inf{t : x(t) Q ρ E c ε 0, t τ 0 }, τ n = inf{t : x(t) E k(ε0), t τ n 1 }, τ n = inf{t : x(t) Q ρ E c ε 0, t τ n}, etc It is obvious that for every sample ω B, τ n, τ n, n = 1,, are existing and finite By the finite-contract of H to function W( ), for δ(ε0) > 0 there exists some µ so that inf P, x(t W( x(){ sup W( x(s)) δ(ε S = {1, } with generator 0) 0)) Q ρ,>0 +µ s } p 0 Γ = (γ ij ) = (0) Define A n = {ω : x(τ n + s) E c k(ε 0), 0 s µ, τ n < + } (1) If ω A n infinite often, then the total time out of E k(ε0) is infinite for the corresponding path x t (ω) That is impossible So I An (ω) <, as Since F τn measures x s, s τ n, 397 thus all A i, i = 1,,,n 1, are all in F τn, this is equivalent to (please refer to [1],p398-399) n P x {A n F τn } () By the strong Markov property of system (1,) and the inequality (0), we get P {A n F τn } n P τn, x(τ n){ sup 1 I {τn< } µ s 0 x(τ n + s) x(τ n) δ(ε 0) }I {τn< } (3) Since for every n, B {τ n < } holds, that comprises that for each ω B, I An (ω) = We have a contradiction Therefore W( x(t, ω)) = 0, for all ω Ω Theorem 41 is followed Corollary 41: In Theorem 41, if w( x) > 0 for x > 0, then the continuous states of the system (1,) are asymptotically stable in probability, ie, for any ε > 0, P { x(t) > ε} = 0 V ASYMPTOTIC STABILITY OF SWITCHING DIFFUSION SYSTEMS In this section, we use the result above to study the asymptotical stability of the diffusion processes with Markovian jump, which is one of general stochastic hybrid systems About these systems, please refer to [11] From above discussion, it is followed that if there exists a twice differentiable function V ( x in x such that its infinitesimal operator LV w( x) < 0 for every x R n /{0}, then its equilibrium point x = 0 is asymptotically stable The infinitesimal operator LV is defined by LV (x, i, t) := V t + V x f(x, i, t) + 1 tr(gt V x g) + j γ ij V (x, j, t) and µ(dl, ds) = v(dl, ds) m(dl)ds is a martingale measure Note that in this case = and condition (13) is trivial Example 51: Let B(t) be a scalar Brownian motion Let r(t) be a right-continuous Markovian chain taking values in ( ) γ11 γ 1 γ 1 γ Assume that B(t) and r(t) are independent Consider a onedimensional stochastic differential equation with Markovian switching of the form dx(t) = f(x(t), r(t))dt + g(x(t), r(t))db(t) on t 0, where

ThAIn38 f(x, 1) = x, g(x, 1) = sinx f(x, ) = sinx, g(x, ) = x, To examine the asymptotical stability, we select a function V : R S R + by V (x, i) = β i x with β 1 = 1 and β = β a constant to be determined It is easy to show that LV (x, 1) x ( 1 + γ 1 (β 1)) and LV (x, ) x (3β + γ 1 (1 β)) Hence if choose β satisfying β > γ 1 γ 1 3, and β < 1 + 1/γ 1, then by Corollary 41, the system is asymptotically stable And for this the parameters γ ij should meet γ 1 > 3(1+γ 1 ) [16] RZ Khas minskii, Stochastic Stability of Differential Equations, S & N International Publishers, Rockville, MD, 1980 [17] RZ Khas minskii, C Zhu, G Yin, Stability of regime-switching diffusion, Stochastic Anal Appl, 117 (007) 1037-1051 [18] P Florchinger, A universal formula for the stabilization of control stochastic differential equation, Stochastic Anal Appl, 1993, 11:155-16 [19] P Florchinger, Lyapunov-like techniques for stochastic stability, SIAM J Control Optim, 1995, 33:1151-1169 [0] H Deng, M Krstic, Stabilization of stochastic nonlinear systems driven by noise of unknown covariance, Systems & Control letters, 000, 39:173-18 [1] H Deng, M Krstic, Stochastic nonlinear stabilization-part I:A backstepping design, Systems & Control letters, 1997 3:143 150 [] H Deng, M Krstic, Output-feedback stochastic nonlinear stabilization, IEEE Trans Automat Control, 1999, 44:38-333 [3] Mao, Stability of stochastic differential equations with Markovian switching, Stoch Process Appl, 79(1999) 45-67 [4] C Yuan and J Lygeros, On Exponential Stability of Switching Diffusion Process, IEEE Trans Automat Control, 50(005) 1 181-185 VI CONCLUSION By introducing a concept of finite-contraction system, the LaSalle stability theorem is given for general stochastic hybrid systems According to this result the stability of switching diffusion process is discussed In the future we may consider the input-to-state stability of GSHS REFERENCES [1] M H A Davis, Markov Modek and Optimization, London: Chapman & Hall, 1993 [] J Hu, J Lygeros, S Sastry, Towards a Theory of Stochastic Hybrid Systems, In NLynch, BKrogh Eds Hybrid Systems : Computation and Control, LNCS 1790, pp160-173, 000 [3] O L Costa and K Boukas, Necessary and sufficient condition for robust stability and stabilizability of continuous-time linear systems with Markovian jumps[j] J Optimization Theory Appl, 1998, 99(11), 1155-1167 [4] C Yuan and J Lygeros, Asymptotic Stability and Boundedness of Delay Switching Diffusions, IEEE Trans Automat Control, 006, 51(1), 171-175 [5] M L Bujorian and J Lygeros, General stochastic hybrid systems IEEE Mediterranean Conference on Control and Automation, MED 04, Kusadasi,Turkey, June 004 [6] L Shi, A Abate and S Sastry, Optimal control for a class of stochastic hybrid systems, Preceeding of 43rd IEEE Conference on Decision and Control, December, 14-17, 004, WeB011, Atlantis, Paradise Island, Bahamas [7] J Lygeros, K H Johansson, S N Simić, J Zhang, S S Sastry, Dynamical Properties of Hybrid Automata, IEEE Transactions on Automatic Control, 003, 48(1) -17 [8] H Liu, Mu, Exponential stability of general stochastic hybrid systems and its application; Proceeding of 9th International Conference On Control, Automation, Robotics and Vision, Singapore, 5-8th December 008 195-199 [9] AV Savkin and RJ Evans, Hybrid dynamical systems, Boston Basel Berlin: Birkäuser, 00 [10] H Liu, Mu, An asymptotic stability theorem of Peuteman-Aeyels for Itô processes and its applications in synchronous switching systems, Systems & Control letters, 006, 55(1), 963-970 [11] Mao, stability of stochastic differential equations with Markovian switching, Stoch Process Appl, 1999, 79(1), 45-67 [1] E G Dynkin, Markov Processes, Berlin:Springer-Verlag, 1965 [13] JP LaSalle, Stability Theory for Ordinary Differential Equations, J of Differential Equations, Vol4 (1968), 57-65 [14] HJ Kushner, Converse theorems for stochastic Liapunov functions, SIAM J Control 5 (1967) 8-33 [15] HJ Kushner, The concept of invariant set for stochastic dynamocal systems and applications to stochastic stability, Stochastic Optimization and Control H Karreman, Editer, John Wiley and Sons, New York, 1968 3973