Why Should I Care About. Stochastic Hybrid Systems?

Transcription

1 Research Supported by NSF & ARO Why Should I Care About Stochastic Hybrid Systems? João Hespanha 1 Why Care Should I Care About SHSs? Eamples Feedback over shared networks Estimation using remote sensors Biology / Gene regulation Modeling/Analysis tools Time-triggered SHSs Lyapunov-based analysis Moments dynamics (e) students: D. Antunes (IST), A. Mesquita (UCSB), Y. Xu (Advertising.com), A. Singh (UCSD) collaborators: M. Khammash (UCSB), C. Silvestre (IST) acknowledgements: NSF, Institute for Collaborative bio-technologies (ARO), AFOSR (STTR program) disclaimer:! This is an overview, technical details in papers referenced in bottom right corner 2

2 Talk outline Eamples Feedback over shared networks Estimation using remote sensors Biology / Gene regulation Modeling/Analysis tools Time-triggered SHSs Lyapunov-based analysis Moments dynamics Disclaimer: Several other important applications/researchers not mentioned in this talk. E.g., (e) students: D. Antunes (IST), Y. Xu (Advertising.com), air traffic control [Bujorianu, Lygeros, Prandini, Hu, Tomlin,...] A. Singh (UCSD) network traffic modeling [Bohacek, Lee, Yin,...] collaborators: M. Khammash (UCSB), C. Silvestre (IST) queuing systems [Cassandras,...] acknowledgements: NSF, Institute for Collaborative bio-technologies (ARO), economics [Davis, Yin, ] AFOSR (STTR program) biology [Hu, disclaimer: Julius, Lygeros,! This Pappas,...] is an overview, technical details in papers referenced in bottom right corner 3 Deterministic Hybrid Systems continuous dynamics q 1 f 1 q 2 f 2 q 3 f 3 reset-maps guard conditions q(t)! Q={1,2, }! " discrete state (t)! R n! " continuous state 4

3 Deterministic Hybrid Systems continuous dynamics q 1 f 1 q 2 f 2 q 3 f 3 reset-maps guard conditions q(t)! Q={1,2, }! " discrete state (t)! R n! " continuous state g() 0? ẋ = f() impulsive system (single discrete mode) φ() 5 Stochastic Hybrid Systems (time-triggered) continuous dynamics t 1,t 4,t 7,... q 1 f 1 q 2 f 2 t 2,t 5,t 8,... t 3,t 6,... q 3 f 3 reset-maps transition times t k+1 t k i.i.d. with given distribution Also known as SHSs driven by renewal processes: N(t) " # of transitions before time t t k renewal process (iid inter-increment times) stochastic impulsive system (SIS) (single discrete mode) ẋ = f() φ() 6

4 Stochastic Hybrid Systems (time-triggered) continuous dynamics t 1,t 4,t 7,... q 1 f 1 q 2 f 2 t 2,t 5,t 8,... t 3,t 6,... q 3 f 3 reset-maps transition times t k+1 t k i.i.d. with given distribution Also known as SHSs driven by renewal processes: Special case: when tk+1! tk i.i.d. eponentially distributed N(t) " # of called transitions Markovian before time Jump t Systems renewal in process this case (t) is a Markov Process (iid inter-increment times) ẋ = f() well developed theory (analysis stochastic & design) impulsive [Costa, Fragoso, system Boukas, (SIS) Loparo, Lee, Dullerud] (single discrete mode) tk φ() 7 Eample #1: Networked Control System controller process hold 1 hold 2 sampling times sensor 1 shared network sensor 2 process: controller: round-robin network access: hold 8

5 Eample #1: Networked Control System controller process hold 1 hold 2 sampling times sensor 1 shared network sensor 2 process: controller: What if the network is not available at a sample time tk? round-robin network access: 1 st wait until network becomes available 2 nd send (old) data from original sampling of continuous-time output or hold 2 nd send (latest) data from current sampling of continuous-time output! intersampling times tk+1 tk typically become random variables 9 Eample #1: Networked Control System controller process hold 1 hold 2 sampling times sensor 1 shared network sensor 2 t k+1 t k # time-interval between successive transmissions ŷ1 (t k ) = ŷ 2 (t k ) t 1,t 3,... y1 (t k ) ŷ 2 (t k ) ŷ1 (t k ) = ŷ 2 (t k ) ŷ1 (t k ) y 2 (t k ) t 2,t 4,... 10

6 Stability of Linear Time-triggered SIS t k ẋ = A J stochastic impulsive system (single discrete mode) t k+1 t k # i.i.d., with cumulative distribution function F( ) Defining k := (t k ) state at jump times k+1 = Je A(t k+1 t k ) k reset continuous evolution 11 Stability of Linear Time-triggered SIS t k ẋ = A J stochastic impulsive system (single discrete mode) t k+1 t k # i.i.d., with cumulative distribution function F( ) Defining k := (t k ) state at jump times k+1 = Je A(t k+1 t k ) k reset continuous evolution For a given P = P > 0 E[ k+1p k+1 k ]= k E F ( ) e A J PJe A k epectation w.r.t.! = t k+1 t k (cumulative distribution F ) 12

7 f\delta Stability of Linear Time-triggered SIS t k J If there eists ẋ = A stochastic impulsive system (single discrete mode) LMI on P n$ n P>0, E F ( ) e A J PJe A <P then Defining k := (t k ) limstate E[ at jump k+1 = Je A(t k+1 t k 2 times ] = 0 (ep. fast in inde k) k ) k k For a given P = P > 0 What about (t) between resetjumps? continuous evolution E[ k+1p k+1 k ]= k E F ( ) e A J PJe A k epectation w.r.t.! = t k+1 t k (cumulative distribution F ) 13 Stability of Linear Time-triggered SIS All stability notions require lim k = 0 eponentially fast k t k J the nec. & suff. conditions only differ on the requirements on the tail of distribution t k+1 t k # i.i.d., with cumulative ẋ = A 1 F (s) =P(t k+1 distribution t k >s) function F( ) Mean-square eponential stability, i.e., lim t E[(t)2 ep. fast ] = 0 Mean-square asymptotic stability, i.e., lim t E[(t)2 ]=0 Mean-square stochastic stability, i.e., 0 E[(t) 2 ]dt < 14

8 Stability of Linear Time-triggered SIS All stability notions require lim k = 0 eponentially fast k t k J the nec. & suff. conditions only differ on the requirements on the tail of distribution t k+1 t k # i.i.d., with cumulative ẋ = A 1 F (s) =P(t k+1 distribution t k >s) function F( ) (versions of these results for multiple discrete modes are available) Theorem: Mean-square eponential stability, i.e., lim t E[(t)2 ep. fast ] = 0 P 0, E F e A J PJe A P and lim s e A s e As 1 F s ep. fast 0 Mean-square asymptotic stability, i.e., Mean-square stochastic stability, i.e., lim t E[(t)2 ]=0 P >0, E F ( ) e A J PJe A ] <P and lim s eas e As 1 F (s) =0 P >0, E F ( ) e A J PJe A <P and 0 E[(t) 2 ]dt < 0 e As e As F (ds) < [Antunes et al, 2009] 15 Talk outline Eamples Feedback over shared networks Estimation using remote sensors Biology / Gene regulation Modeling/Analysis tools Time-triggered SHSs Lyapunov-based analysis Moments dynamics (e) students: D. Antunes (IST), A. Mesquita (UCSB), Y. Xu (Advertising.com), A. Singh (UCSD) collaborators: M. Khammash (UCSB), C. Silvestre (IST) acknowledgements: NSF, Institute for Collaborative bio-technologies (ARO), AFOSR (STTR program) disclaimer:! This is an overview, technical details in papers referenced in bottom right corner 16

9 So far time-driven SHSs... continuous dynamics t 1,t 4,t 7,... q 1 f 1 q 2 f 2 t 2,t 5,t 8,... t 3,t 6,... q 3 f 3 reset-maps transition times t k+1 t k i.i.d. with given distribution q(t)! Q={1,2, }! " discrete state (t)! R n! " continuous state 17 Stochastic Hybrid Systems continuous dynamics q 1 f 1 q 2 f 2 q 3 f 3 reset-maps transition intensities (probability of transition in small interval (t, t+dt]) q(t)! Q={1,2, }! " discrete state (t)! R n! " continuous state λ ()dt probability of transition in an elementary interval (t, t+dt] Special case: Markovian jump system " When all # λ ()! are constant time triggered SIS with eponential t k+1 " t k instantaneous rate of transitions per unit of time 18

10 Stochastic Hybrid Systems with Diffusion stochastic differential eq. f 1 g 1 w f 2 g 2 w f 3 g 3 w reset-maps transition intensities (probability of transition in small interval (t, t+dt]) q(t)! Q={1,2, }! " discrete state (t)! R n! " continuous state 19 Eample #2: Estimation through network process state-estimator white noise disturbance encoder (t 1 ) (t 2 ) packet-switched network decoder for simplicity: full-state available no measurement noise no quantization no transmission delays encoder logic " determines when to send measurements to the network decoder logic " determines how to incorporate received measurements 20

11 Stochastic communication logic process state-estimator white noise disturbance encoder (t 1 ) (t 2 ) packet-switched network decoder for simplicity: full-state available no measurement noise no quantization no transmission delays decoder logic " determines how to incorporate received measurements 1. upon reception of t k,resetˆ t k to t k encoder logic " determines when to send measurements to the network [related ideas pursued by Astrom, Basar, Hristu, Kumar, Tilbury] 21 Error Dynamics process state-estimator white noise disturbance encoder (t 1 ) (t 2 ) packet-switched network decoder for simplicity: full-state available no measurement noise no quantization no transmission delays Error dynamics: prob. of sending data in (t,t+dt] depends on current error e reset error to zero 22

12 ODE! Lie Derivative Given scalar-valued function V : R n R dv (t) dt = V (t) f (t) derivative along solution to ODE L f V Lie derivative of V Basis of Lyapunov formal arguments to establish boundedness and stability E.g., picking V () := 2 dv t dt V f 0 V t t remains bounded along trajectories! 23 Generator of a Stochastic Hybrid System f q g q w λ q, dt (q, ) φ (q, ) Given scalar-valued function V : Q R n R where d dt E V q(t),(t) = E (LV ) q(t),(t) & q are discontinuous, but the epected value is differentiable Dynkin s formula (in differential form) LV (etended) generator of the SHS q, V q, f q m 1 intensity instantaneous variation λ q, V φ q, V q, Lie derivative Reset term 1 2 trace g q 2 V 2 g q Diffusion term 24

13 Lyapunov Analysis! SHSs f g w λ()dt φ() d dt E V (t) = E (LV ) (t) sample-path notions class-k functions: (zero at zero & mon. increasing) α 1 () V () α 2 () LV () α 3 ()! probability of (t) eceeding any given bound M, can be made arbitrarily small by making 0 small P t : (t) M α 2 ( 0 ) α 1 (M) P (t) 0 =1 almost sure (a.s.) asymptotic stability epected-value notions V () 0 LV () W () V () W () 0 LV () µv + c!! 0 E W (t) dt < E W (t) e µt V ( 0 )+ c µ stochastic stability (mean square when W () = 2 ) eponential stability (mean square when W () = 2 ) 25 Eample #2: Remote estimation error dynamics in remote estimation ė = Ae + Bẇ λ(e)dt Dynkin s formula d dt E V e(t) = E (LV ) e(t) (LV )(e) := V e Ae + λ(e) V (0) V (e) trace B 2 V e 2 B e 0 For constant rate: #(e) = $ (ep. distributed inter-jump times) 1. E[ e ] % 0! if and only if 2. E[ e m ] bounded! if and only if γ λ i A, i γ m λ i A, i using V (e) =e Pe getting more moments bounded requires higher comm. rates For radially unbounded rate: #(e) (reactive transmissions) 3. E[ e ] % 0! (always) 4. E[ e m ] bounded! & m using V (e) =e 2 Moreover, one can achieve the same E[ e 2 ] with less communication than with a constant rate or periodic transmissions [Xu et al, 2006] 26

14 Back to Time-triggered SIS... t k φ() λ()dt φ() ẋ = f() t k+1 t k # F( ) ẋ = f() Can we pick an intensity #( ) to obtain the desired distribution for the tk? YES 27 Back to Time-triggered SIS... t k ẋ = f() φ() This representation allows one to combine in the same SHS time- and -triggered transitions! t k+1 t k # F( ) Can we pick an intensity #( ) to obtain the desired distribution for the tk? YES F (τ) 1 F (τ) dt ẋ = f() τ =1 φ() τ 0 time since last reset τ(t) =t t k t 1 t 2 t 3 t the aggregate state (,!) is a Markov process 28

15 Converse Lyapunov Stability Theorem: t k ẋ = f() φ() t k+1 t k # F( ) System is mean-square eponentially stable, i.e., " F (τ) 1 F (τ) dt ẋ = f() τ =1 lim t E[(t)2 ep. fast ] = 0 φ() τ 0 P (τ) such that defining V (, τ) = P (τ) Lyapunov-like function quadratic on for fied! (motivates choices for Lyapunov function for nonlinear systems) 29 Talk outline Eamples Feedback over shared networks Estimation using remote sensors Biology / Gene regulation Modeling/Analysis tools Time-triggered SHSs Lyapunov-based analysis Moments dynamics (e) students: D. Antunes (IST), A. Mesquita (UCSB), Y. Xu (Advertising.com), A. Singh (UCSD) collaborators: M. Khammash (UCSB), C. Silvestre (IST) acknowledgements: NSF, Institute for Collaborative bio-technologies (ARO), AFOSR (STTR program) disclaimer:! This is an overview, technical details in papers referenced in bottom right corner 30

16 Eample III:(Unregulated) Gene Epression Gene epression " process by which a gene (encoded in the DNA) produces proteins: * %& mrna (constant rate) mrna %& X + mrna translation mrna %& * mrna decay X %& * protein decay 31 Eample III:(Unregulated) Gene Epression * %& mrna mrna %& X + mrna mrna %& * (constant rate) translation mrna decay one X %& * protein decay (t) " number of proteins at time t prob. of one in (t, t+dt] Kdt ddt prob. of one decay in (t, t+dt] # of proteins produced per + N 1 equivalent to Gillespie s stochastic simulation algorithm (SSA) 32

17 Eample III:(Unregulated) Gene Epression * %& mrna mrna %& X + mrna mrna %& * (constant rate) translation mrna decay one X %& * How to go beyond stability/bounds and (t) study " number the dynamics of proteins of means, at time variances, t co-variances, etc.? protein decay prob. of one in (t, t+dt] Kdt ddt prob. of one decay in (t, t+dt] # of proteins produced per + N 1 equivalent to Gillespie s stochastic simulation algorithm (SSA) 33 Moment Dynamics Kdt ddt decay + N 1 d dt E V () = E (LV )() (LV )() =K V ( + N) V () + d V ( 1) V () de[] = K E[N] d E[] dt de[ 2 ] = K E[N 2 ]+(2K E[N]+d)E[] 2d E[ 2 ] dt 34

18 Moment Dynamics Kdt ddt decay + N 1 d dt E V () = E (LV )() (LV )() =K V ( + N) V () + d V ( 1) V () de[] = K E[N] d E[] dt de[ 2 ] = K E[N 2 ]+(2K E[N]+d)E[] 2d E[ 2 ] dt One can show that 35 (Unregulated) Gene Epression Kdt ddt decay + N 1 Thus, at steady-state, measure of stochastic fluctuations in protein level (normalized by mean population) intrinsic noise (solely due to random protein epression/degradation) 36

19 Auto-Regulated Gene Epression * %& mrna (non-constant rate) mrna %& X + mrna translation mrna %& * mrna decay X %& * protein decay Protein production rate is a function of the current protein molecule count through regulation: al response (stochastic rate at which g() dt ddt s occur) Altering the RNA polymerase specificity for a given promoter or set of promoters Binding to non-coding sequences on the DNA to impede RNA polymerase's progress + N 1 37 Auto-Regulatory Negative Feedback g() dt ddt decay + N 1 negative feedback ' protein production rate is a decreasing function of the protein molecule count al response g() Common form of auto regulation (e.g., half of the repressors in E. Coli) Eperimentally shown to ehibit noise reduction ability 38

20 Moment Dynamics g() dt ddt decay + N 1 d dt E V () = E (LV )() (LV )() =g() V ( + N) V () + d V ( 1) V () de[] =E[N]E[g()] d E[] dt de[ 2 ] =E[N 2 ]E[g()] +2E[N]E[g()] + de[] 2d E[ 2 ] dt!when g() is an affine function we still get a finite system of linear equations!when g() is a polynomial, we get a closed but infinite system of linear equation (general property of polynomial SHSs)!For other g(), one generally does not get a closed system of equations 39 Auto-Regulated Gene Epression g() dt ddt decay + N 1 Approimate Analysis Methods Distribution-based: assume a specific type of distribution (Normal, LogNormal, Poisson, etc.) and force dynamics to be compatible with this type of distribution Large numbers/large volume: take the limit as volume & ( and assume concentrations do not & 0 Derivative matching: force solutions of approimate dynamics to match eact equation locally in time Linearization: Linearize al response around steady-state value of the mean 40

21 Auto-Regulatory Negative Feedback g() dt ddt decay + N 1 For a al response approimately linear around steady-state mean protein s response-time (with feedback) CV [] = StdDev[] E[] protein s half-live (response time without feedback) Negative feedback reduces T r with respect to Tp! decreases noise = T r N T p E[] unregulated intrinsic noise steady-state population mean T r 1 T p 2.25 eperimental study using a synthetic gene circuit in which the repressor TetR fused to GFP represses its own promoter Rosenfeld et al, J. Molecular Biology, Eogenous Noise g(,z) dt d dt decay + N 1 In practice, rate also depends on eogenous species (e.g., RNA polymerase and other enzymes) g(, z) ' al response (stochastic rate at which s occur) eogenous species (with stochastic fluctuations) 42

22 Eogenous Noise g(,z) dt d dt decay + N 1 CV [] 2 T r N T p E[] + Tr 2CV [z] 2 T p intrinsic noise (as before) etrinsic noise CV of etrinsic species T r ' protein s response-time (with feedback) T p ' protein s half-live (response time without feedback) Negative feedback reduces T r with respect to T p! attenuates both intrinsic and etrinsic noise! more efficient at reducing etrinsic noise! surprisingly good matches with eperimental results T r T p ! offers a new technique to discover sources of etrinsic noise (solve for CV[z]!) [Singh et al, 2009; related results by Paulsson 2004] 43 Why Should I Care About SHS? 1. SHS models that find use in several areas (network traffic modeling, networked control systems, distributed estimation, biochemistry, population dynamics in ecosystems) 2. The analysis of SHSs is challenging but there are tools available (stability conditions for linear time-triggered SHS, Lyapunov methods, moment dynamics, linearization, truncations) 3. Lots of work to be done: theory stability/robustness/performance of SHSs networked control systems protocol design to optimize performance & minimize communication resources biology quantitative study of common motifs (modules) in systems biology study of spatial processes other applications 44