Stochastic Hybrid Systems: Applications to Communication Networks

Transcription

1 research supported by NSF Stochastic Hybrid Systems: Applications to Communication Networks João P. Hespanha Center for Control Engineering and Computation University of California at Santa Barbara Talk outline 1. A (simple) model for stochastic hybrid systems (SHSs) 2. SHSs models for network traffic under TCP 3. Analysis tools for SHSs 4. Dynamics of TCP Collaborators: Stephan Bohacek, Junsoo Lee, Katia Obraczka, Abhyudai Singh, Yonggang Xu Acknowledgements: Mustafa Khammash, John Lygeros 1

2 Deterministic Hybrid Systems continuous dynamics guard conditions reset-maps q(t) Q={1,2, } discrete state x(t) R n continuous state right-continuous by convention we assume here a deterministic system so the invariant sets would be the exact complements of the guards Stochastic Hybrid Systems continuous dynamics transition intensities (instantaneous rates at which transitions occur) reset-maps N l (t) N transition counter, which is incremented by one each time the lth reset-map ϕ l (x) is activated right-continuous by convention at least one transition on (t, t+dt] proportional to elementary interval length dt and transition intensity λ l (x) 2

3 Stochastic Hybrid Systems continuous dynamics transition intensities (instantaneous rates at which transitions occur) reset-maps N l (t) N transition counter, which is incremented by one each time the lth reset-map ϕ l (x) is activated right-continuous by convention number of transitions on (t 1, t 2 ] equal to integral of transition intensity λ l (x) on (t 1, t 2 ] Stochastic Hybrid Systems continuous dynamics transition intensities (instantaneous rates at which transitions occur) reset-maps Special case: When all λ l are constant, transitions are controlled by a continuous-time Markov process q = 1 q = 2 specifies q (independently of x) q = 3 3

4 Formal model Summary State space: q(t) Q={1,2, } discrete state x(t) R n continuous state Continuous dynamics: Transition intensities: Reset-maps (one per transition intensity): # of transitions Formal model Summary State space: q(t) Q={1,2, } discrete state x(t) R n continuous state Continuous dynamics: Transition intensities: Reset-maps (one per transition intensity): # of transitions Results: 1. [existence] Under appropriate regularity (Lipschitz) assumptions, there exists a measure consistent with the desired SHS behavior 2. [simulation] The procedure used to construct the measure is constructive and allows for efficient generation of Monte Carlo sample paths 3. [Markov] The pair ( q(t), x(t) ) Q R n is a (Piecewise-deterministic) Markov Process (in the sense of M. Davis, 1993) Hespanha. Stochastic Hybrid Systems: Applications to Communication Networks. HSCC 4 4

5 Transmission Control Protocol transmits data packets receives data packets server r network client packets dropped with probability p drop congestion control selection of the rate r at which the server transmits packets feedback mechanism packets are dropped by the network to indicate congestion w (window size) TCP window-based control number of packets that can remain unacknowledged for by the destination e.g., w = 3 source f destination f t 1 st packet sent 2 nd packet sent t 1 3 rd packet sent t 2 τ 1 st packet received & ack. sent τ 1 2 nd packet received & ack. sent 1 st ack. received t 3 τ 2 4 th packet can be sent 3 rd packet received & ack. sent w effectively determines the sending rate r: t t Built-in negative feedback: r increases congestion round-trip time r decreases RTT increases (queuing delays) 5

6 TCP Reno congestion avoidance 1. While there are no drops, increase w by 1 on each RTT (additive increase) 2. When a drop occurs, divide w by 2 (multiplicative decrease) (congestion controller constantly probes the network for more bandwidth) per-packet drop prob. pckts sent per sec = pckts dropped per sec additive increase total # of packets already sent multiplicative decrease Three feedback mechanisms: As rate r increases a) congestion RTT increases (queuing delays) r decreases b) congestion p drop increases (active queuing) r decreases c) multiplicative decrease more likely r decreases Delay in drop detection 3. In general there is some delay between drop occurrence and detection (assumed exponentially distributed with mean τ delay ) drop occurred multiplicative decrease drop detected 6

7 TCP Reno slow start 3. Until a drop occurs double w on each RTT 4. When a drop occurs divide w by 2 and transition to congestion avoidance (get to full bandwidth utilization fast) # of packets already sent for simplicity this diagram ignores delay TCP has several other modes that can be modeled by hybrid systems [Bohacek, Hespanha, Lee, Obraczka. A Hybrid Systems Modeling Framework for Fast and Accurate Simulation of Data Communication Networks. In SIGMETRICS 3] On-off TCP model Between transfers, server remains off for an exponentially distributed time with mean τ off Transfer-size is a random variable with cumulative distribution F(s) determines duration of on-periods 7

8 On-off TCP model Between transfers, server remains off for an exponentially distributed time with mean τ off Transfer-size is exponentially distributed with average k (packets) simple but not very realistic SHS models for TCP long-lived TCP flows (no delays) on-off TCP flows (no delays) on-off TCP flows with delay long-lived TCP flows with delay 8

9 Analysis Lie Derivative Given function ψ : R n [, ) R derivative along solution to ODE L f ψ Lie derivative of ψ One can view L f as an operator space of scalar functions on R n [, ) space of scalar functions on R n [, ) ψ (x, t) a L f ψ(x, t) L f completely defines the system dynamics Analysis Generator of the SHS continuous dynamics transition intensities reset-maps Given function ψ : Q R n [, ) R generator for the SHS where Dynkin s formula (in differential form) L f ψ Lie derivative of ψ reset instantaneous variation intensity L completely defines the SHS dynamics Disclaimer: see following paper for technical assumptions Hespanha. Stochastic Hybrid Systems: Applications to Communication Networks. HSCC 4 9

10 Long-lived TCP flows long-lived TCP flows long-lived TCP flows (with slow start) Long-lived TCP flows slow-start congestion avoidance 1

11 Analysis PDF for SHS state continuous dynamics transition intensities reset-maps Let ρ( ; t) be the probability density function (PDF) for the state (x,q) at time t : When ρ( ; t) is smooth, one can deduce from the generator that inverse of functional PDE very difficult to solve Analysis Moments for SHS state continuous dynamics transition intensities reset-maps often a few low-order moments suffice to study a SHS z (scalar) random variable with mean µ and variance σ 2 Markov inequality ( e > ) Bienaymé inequality ( e >, a R, n N) Tchebychev inequality 11

12 Polynomial SHSs continuous dynamics transition intensities reset-maps Given function ψ : Q R n [, ) R where generator for the SHS Dynkin s formula (in differential form) A SHS is called a polynomial SHS (pshs) if its generator maps finite-order polynomial on x into finite-order polynomials on x Typically, when are all polynomials q, t Moment dynamics for pshs x(t) R n continuous state q(t) Q={1,2, } discrete state Test function: Given for short x (m) Uncentered moment: E.g, 12

13 Moment dynamics for pshs x(t) R n continuous state q(t) Q={1,2, } discrete state Test function: Given for short x (m) Uncentered moment: For polynomial SHS monomial on x polynomial on x linear comb. of test functions linear moment dynamics long-lived TCP flows Moment dynamics for TCP 13

14 long-lived TCP flows Moment dynamics for TCP on-off TCP flows (exp. distr. files) Truncated moment dynamics Experimental evidence indicates that the sending rate is well approximated by a Log-Normal distribution PDF of the congestion window size w w z Log-Normal r Log-Normal (on each mode) Data from: Bohacek, A stochastic model for TCP and fair video transmission. INFOCOM 3 14

15 long-lived TCP flows Moment dynamics for TCP on-off TCP flows (exp. distr. files) finite-dimensional nonlinear ODEs long-lived TCP flows with delay (one RTT average) Long-lived TCP flows Monte-Carlo & theoretical steady-state distribution (vs. drop probability) 15

16 Long-lived TCP flows moment dynamics in response to abrupt change in drop probability sending rate mean sending rate std. deviation Monte Carlo (with 99% conf. int.) reduced model On-off TCP flows SHS for on-off TCP flows Transfer-sizes exponentially distributed with mean k = 3.6KB (Unix 93 files) Off-time exponentially distributed with mean τ off = 1 sec Round-trip-time RTT = 5msec 16

17 On-off TCP flows probabilities ss ca sending rate mean sending rate std. dev. total ss ca total ss ca Monte Carlo (solid) reduced model (dashed) On-off TCP flows SHS for on-off TCP flows Mixture of exponential distribution for transfer-sizes: m exp. distr. with mean { k 1, k 2,, k m } m probabilities { p 1, p 2,, p m } transfer-sizes are extracted from an exponential distribution with mean k i w.p. p i can approximate well distributions found in the literature 17

18 On-off TCP flows: Case I Transfer-size approximating the distribution of file sizes in the UNIX file system p 1 = 88.87% k 1 = 3.5KB ( mice files) p 2 = 11.13% k 2 = 246KB ( elephant files, but not heavy tail) Probability.9.8 any active mode slow-start congestion-avoidance.7 Rate Standard Deviation total slow-start congestion-avoidance small "mice" transfers mid-size "elephant" transfers p drop 6 5 Rate Mean total slow-start congestion-avoidance small "mice" transfers mid-size "elephant" transfers p drop Monte-Carlo & theoretical steady-state distribution (vs. drop probability) p On-off TCP flows: Case II Transfer-size distribution at an ISP s www proxy [Arlitt et al.] p 1 = 98% k 1 = 6KB ( mice files) p 2 = 1.7% k 2 = 4KB ( elephant files) p 3 =.2% k 3 = 1MB ( mammoth files) Rate Mean Rate Standard Deviation total slow-start congestion-avoidance small "mice" transfers mid-size "elephant" transfers large "mammoth" transfers total slow-start congestion-avoidance small "mice" transfers mid-size "elephant" transfers large "mammoth" transfers p drop p drop Theoretical steady-state distribution (vs. drop probability) 18

19 Truncated moment dynamics (revisited) For polynomial SHS Stacking all moments into an (infinite) vector µ linear moment dynamics In TCP analysis infinite-dimensional linear ODE lower order moments of interest moments of interest that affect µ dynamics approximated by nonlinear function of µ Truncation by derivative matching infinite-dimensional linear ODE truncated linear ODE (nonautonomous, not nec. stable) nonlinear approximate moment dynamics Assumption:1)µ and ν remain bounded along solutions to and 2) is asymptotically stable Theorem: δ > N s.t. if then class KL function Hespanha. Polynomial Stochastic Hybrid Systems. HSCC'5 19

20 Truncation by derivative matching infinite-dimensional linear ODE truncated linear ODE (nonautonomous, not nec. stable) nonlinear approximate moment dynamics Assumption:1)µ and ν remain bounded along solutions to and 2) is asymptotically stable Theorem: δ > N s.t. if then Proof idea: 1) N derivative matches µ & ν match on compact interval of length T 2) stability of A matching can be extended to [, ) Truncation by derivative matching infinite-dimensional linear ODE Given δ finding N is very difficult In practice, small values of N (e.g., N=2) already yield good results truncated linear ODE nonlinear approximate Can use (nonautonomous, d k not µ nec. stable) moment dynamics dt k = dk ν dt k, k {1,...,N} Assumption:1)µ to determine and νϕ( ): remain k = bounded 1 boundary along solutions condition to on ϕ k = 2 and PDE on ϕ 2) is asymptotically stable Theorem: δ > N s.t. if then Proof idea: 1) N derivative matches µ & ν match on compact interval of length T 2) stability of A matching can be extended to [, ) 2

21 and now something completely different Decaying-dimerizing molecular reactions (DDR): c 1 c 2 c 3 S 1 S 2 2 S 1 S 2 c 4 pshs model population of species S 1 reaction rates population of species S 2 Moment dynamics for DDR Decaying-dimerizing molecular reactions (DDR): c 1 c 2 c 3 S 1 S 2 2 S 1 S 2 c 4 21

22 Truncated DDR model Decaying-dimerizing molecular reactions (DDR): c 1 c 2 c 3 S 1 S 2 2 S 1 S 2 c 3 c 4 by matching d k µ dt k = dk ν, k {1,2} dt k for deterministic distributions Hespanha. Polynomial Stochastic Hybrid Systems. HSCC'5 Monte Carlo vs. truncated model 4 populations means x E[x 1 ] E[x 2 ] populations standard deviations Std[x 1 ] Fast time-scale transient 2 1 Std[x 2 ] x populations correlation coefficient ρ[x 2,x 2 ] x 1-3 (lines essentially undistinguishable at this scale) 22

23 Monte Carlo vs. truncated model 4 populations means 3 2 E[x 1 ] E[x 2 ] populations standard deviations Slow time-scale evolution Std[x 2 ] Std[x 1 ] populations correlation coefficient ρ[x 2,x 2 ] only noticeable error when populations become very small (a couple of molecules) Conclusions 1. A simple SHS model (inspired by piecewise deterministic Markov Processes) can go a long way in modeling network traffic 2. The analysis of SHSs is generally difficult but there are tools available (generator, Dynkin s equation, moment dynamics, truncations) 3. This type of SHSs (and tools) finds use in several areas (traffic modeling, networked control systems, molecular biology) 23

24 Bibliography M. Davis. Markov Models & Optimization. Chapman & Hall/CRC, 1993 S. Bohacek, J. Hespanha, J. Lee, K. Obraczka. Analysis of a TCP hybrid model. In Proc. of the 39th Annual Allerton Conf. on Comm., Contr., and Computing, Oct. 21. S. Bohacek, J. Hespanha, J. Lee, K. Obraczka. A Hybrid Systems Modeling Framework for Fast and Accurate Simulation of Data Communication Networks. In Proc. of the ACM Int. Conf. on Measurements and Modeling of Computer Systems (SIGMETRICS), June 23. J. Hespanha. Stochastic Hybrid Systems: Applications to Communication Networks. In Rajeev Alur, George J. Pappas, Hybrid Systems: Computation and Control, number 2993 in Lect. Notes in Comput. Science, pages , Mar. 24. Hespanha. Polynomial Stochastic Hybrid Systems. To be presented at the HSCC'5. J. Hespanha, A. Singh. Stochastic Models for Chemically Reacting Systems Using Polynomial Stochastic Hybrid Systems. Submitted to the Int. J. on Robust Control Special Issue on Control at Small Scales, Nov. 24. All papers (and some ppt presentations) available at 24