research supported by NSF Stochastic Hybrid Systems: Applications to Communication Networks João P. Hespanha Center for Control Engineering and Computation University of California at Santa Barbara Deterministic Hybrid Systems continuous dynamics guard conditions reset-maps q(t) Q={1,2, } discrete state x(t) R n continuous state right-continuous by convention we assume here a deterministic system so the invariant sets would be the exact complements of the guards
Stochastic Hybrid Systems continuous dynamics transition intensities (instantaneous rates at which transitions occur) reset-maps N (t) N transition counter, which is incremented by one each time the th reset-map ϕ (x) is activated right-continuous by convention number of transitions on (t 1, t 2 ] equal to integral of transition intensity λ (x) on (t 1, t 2 ] Stochastic Hybrid Systems continuous dynamics transition intensities (instantaneous rates at which transitions occur) reset-maps Special case: When all λ are constant, transitions are controlled by a continuous-time Markov process q = 1 q = 2 specifies q (independently of x) q = 3
Formal model Summary State space: q(t) Q={1,2, } discrete state x(t) R n continuous state Continuous dynamics: Transition intensities: Reset-maps (one per transition intensity): # of transitions Results: 1. [existence] Under appropriate regularity (Lipschitz) assumptions, there exists a measure consistent with the desired SHS behavior 2. [simulation] The procedure used to construct the measure is constructive and allows for efficient generation of Monte Carlo sample paths 3. [Markov] The pair ( q(t), x(t) ) Q R n is a (Piecewise-deterministic) Markov Process (in the sense of M. Davis, 1983) Example I: TCP congestion control transmits data packets receives data packets server r network client packets dropped with probability p drop congestion control selection of the rate r at which the server transmits packets feedback mechanism packets are dropped by the network to indicate congestion
Example I: TCP congestion control transmits data packets receives data packets server r network client packets dropped with probability p drop congestion control selection of the rate r at which the server transmits packets feedback mechanism packets are dropped by the network to indicate congestion TCP (Reno) congestion control: packet sending rate given by congestion window (internal state of controller) round-trip-time (from server to client and back) initially w is set to 1 until first packet is dropped, w increases exponentially fast () after first packet is dropped, w increases linearly (congestion-avoidance) each time a drop occurs, w is divided by 2 (multiplicative decrease) Example I: TCP congestion control # of packets already sent per-packet pckts sent drop prob. = pckts dropped per sec per sec TCP (Reno) congestion control: packet sending rate given by congestion window (internal state of controller) round-trip-time (from server to client and back) initially w is set to 1 until first packet is dropped, w increases exponentially fast () after first packet is dropped, w increases linearly (congestion-avoidance) each time a drop occurs, w is divided by 2 (multiplicative decrease)
Analysis Lie Derivative Given function ψ : R n [, ) R derivative along solution to ODE L f ψ Lie derivative of ψ One can view L f as an operator space of scalar functions on R n [, ) space of scalar functions on R n [, ) ψ(x, t) L f ψ(x, t) L f completely defines the system dynamics Analysis Generator of the SHS continuous dynamics transition intensities reset-maps Given function ψ : Q R n [, ) R generator for the SHS where Dynkin s formula (in differential form) L f ψ Lie derivative of ψ reset instantaneous variation intensity L completely defines the SHS dynamics disclaimer: see paper for technical assumptions
Example I: TCP congestion control congestion avoidance This TCP model is always-on Example II: On-off TCP model Between transfers, server remains off for an exponentially distributed time with mean τ off Transfer-size is a random variable with cumulative distribution F(s) determines duration of on-periods
Using the SHS generator Mixture of exponential distribution for transfer-sizes: M exp. distr. with mean { k 1, k 2,, k M } M probabilities { p 1, p 2,, p M } transfer-sizes are extracted from an exponential distribution with mean k i w.p. p i can approximate well distributions found in the literature µ q,n expected value of r n when SHS is in mode q probability of being in off mode with file-size from ith exponential distribution cong.-avoidance with file-size from ith exponential distribution Using the SHS generator Mixture of exponential distribution for transfer-sizes: M exp. distr. with mean { k 1, k 2,, k M } M probabilities { p 1, p 2,, p M } transfer-sizes are extracted from an exponential distribution with mean k i w.p. p i can approximate well infinite distributions system of ODEs found in the literature But: distributions are well approximated by log-normal µ q,n expected value of r n when so one SHS can is truncate in mode the q infinite system of ODEs (keeping only first 2 moments) off mode with file-size from ith exponential distribution cong.-avoidance with file-size from ith exponential distribution
Case I Transfer-size approximating the distribution of file sizes in the UNIX file system p 1 = 88.87% k 1 = 3.5KB ( mice files) p 2 = 11.13% k 2 = 246KB ( elephant files, but not heavy tail) Probability.9.8 any active mode congestion-avoidance.7 Rate Standard Deviation.6.5.4 9 8 7 total congestion-avoidance small "mice" transfers mid-size "elephant" transfers.3 6.2 5.1 4 1-3 1-2 1-1 Rate Mean 3 p drop 6 5 total congestion-avoidance small "mice" transfers mid-size "elephant" transfers 2 1 1-3 1-2 1-1 4 p drop 3 2 1 1-3 1-2 1-1 p Monte-Carlo & theoretical steady-state values (vs. drop probability) Case II Transfer-size distribution at an ISP s www proxy [Arlitt et al.] p 1 = 98% k 1 = 6KB ( mice files) p 2 = 1.7% k 2 = 4KB ( elephant files) p 3 =.2% k 3 = 1MB ( mammoth files) Rate Mean Rate Standard Deviation 2.5 2 total congestion-avoidance small "mice" transfers mid-size "elephant" transfers mid-size "mammoth" transfers 9 8 7 total congestion-avoidance small "mice" transfers mid-size "elephant" transfers mid-size "mammoth" transfers 6 1.5 5 4 1 3.5 2 1 1-3 1-2 1-1 p drop 1-3 1-2 1-1 p drop Steady-state values vs. drop probability
Conclusions 1. Presented a new model for SHSs with transitions triggered by stochastic events (inspired by piecewise deterministic Markov Processes) 2. Provided analysis tool SHS s generator 3. Illustrated use by modeling and analyzing on-off TCP flows Current/future work 1. Investigate the use of SHS to extend solutions of deterministic hybrid systems 2. Develop other tools of analyze SHS 3. Investigate the implications of the TCP-Reno results on active queuing mechanism 4. Construct SHS models for other congestion control algorithms 5. Use SHS models to study problems of distributed control with limited communication bandwidth Generalizations 1. Stochastic resets can be obtained by considering multiple intensities/reset-maps One can further generalize this to resets governed by to a continuous distribution [see paper]
Generalizations Gaussian white noise 2. Stochastic differential equations (SDE) for the continuous state can be emulated by taking limits of SHSs The solution to the SDE is obtained as e + more details on paper Generalizations 3. Deterministic guards can also be emulated by taking limits of SHSs barrier function -1 1 e + g(x) The solution to the hybrid system with a deterministic guard is obtained as e + This provides a mechanism to regularize systems with chattering and/or Zeno phenomena
Example III: Bouncing-ball g y y c (,1) energy absorbed at impact The solution of this deterministic hybrid system is only defined up to the Zeno-time t Zeno-time Example III: Bouncing-ball g y c (,1) energy absorbed at impact 1.2 1.2 1.2 1.8 1 e = 1 2 e = 1 3 e = 1 4.8.8 1.6.6.6.4.4.4.2.2.2 -.2 5 1 15 -.2 5 1 15 -.2 5 1 15 mean (blue) 95% confidence intervals (red and green)