Polynomial Stochastic Hybrid Systems

Transcription

1 Polynomial Stochastic Hybrid Systems João Pedro Hespanha Center for Control Engineering and Computation University of California, Santa Barbara, CA 93 Abstract. This paper deals with polynomial stochastic hybrid systems (pshss), which generally correspond to stochastic hybrid systems with polynomial continuous vector fields, reset maps, and transition intensities. For pshss, the dynamics of the statistil moments of the continuous states evolve according to infinite-dimensional linear ordinary differential equations (ODEs). We show that these ODEs n be approximated by finite-dimensional nonlinear ODEs with arbitrary precision. Based on this result, we provide a procedure to build this type of approximations for certain claes of pshss. We apply this procedure for several examples of pshss and evaluate the accuracy of the results obtained through comparisons with Monte Carlo simulations. These examples include: the modeling of TCP congestion control both for long-lived and on-off flows; state-estimation for networked control systems; and the stochastic modeling of chemil reactions. Introduction Hybrid systems are characterized by a state-space that n be partitioned into a continuous domain (typilly R n ) and a discrete set (typilly finite). For the stochastic hybrid systems considered here, both the continuous and the discrete components of the state are stochastic procees. The evolution of the continuous-state is determined by a stochastic differential equation and the evolution of the discrete-state by a transition or reset map. The discrete transitions are triggered by stochastic events much like transitions between states of a continuous-time Markov chains. However, the rate at which these transitions occur may depend on the continuous-state. The model used here for SHSs, whose formal definition n be found in Sec., was introduced in [ and is heavily inspired by the Piecewise-Deterministic Markov Procees (PDPs) in [. Alternative models n be found in [3,4,5. The extended generator of a stochastic hybrid system allows one to compute the time-derivative of a test function of the state of the SHS along solutions to the system, and n be viewed as a generalization of the Lie derivative for deterministic systems [,. Polynomial stochastic hybrid systems (pshss) are characterized by extended generators that map polynomial test functions into Supported by the National Science Foundation under grants CCR-384, ANI

2 polynomials. This happens, e.g., when the continuous vector fields, the reset maps, and the transition intensities are all polynomial functions of the continuous state. An important property of pshss is that if one creates an infinite vector containing the probabilities of all discrete modes, as well as all the multi-variable statistil moments of the continuous state, the dynamics of this vector are governed by an infinite-dimensional linear ordinary differential equation (ODE), which we ll the infinite-dimensional moment dynamics (cf. Sec. 3). SHSs n model large claes of systems but their formal analysis presents signifint challenges. Although it is straightforward to write partial differential equations (PDEs) that expre the evolution of the probability distribution function for their states, generally these PDEs do not admit analytil solutions. The infinite-dimensional moment dynamics provides an alternative characterization for the distribution of the state of a pshs. Although generally statistil moments do not provide a description of a stochastic proce as accurate as the probability distribution, results such as Tchebycheff, Markoff, or Bienaymé inequalities n be used to infer properties of the distribution from its moments. In general, the infinite-dimensional linear ODEs that describe the moment dynamics for pshss are still not easy to solve analytilly. However, sometimes they n be accurately approximated by a finite-dimensional nonlinear ODE, which we ll the trunted moment dynamics. We show in Sec. 3 that, under suitable stability aumptions, it is in principle poible for a finite-dimensional nonlinear ODE to approximate the infinite-dimensional moment dynamics, up to an error that n be made arbitrarily small. Aside from its theoretil interest, this result motivates a procedure to actually construct these finite-dimensional approximations for certain claes of pshss. This procedure, which is described in Sec. 4, is applible to pshss for which the (infinite) matrix that characterizes the moment dynamics exhibits a certain diagonal-band structure and appropriate decoupling between certain moments of distinct discrete modes. The details of this structure n be found in Lemma. To illustrate the applibility of the results, we consider several systems that appeared in the literature and that n be modeled by pshss. For each example, we construct in Sec. 5 trunted moment dynamics and evaluate how they compare with estimates for the moments obtained from a large number of Monte Carlo simulations. The examples considered include:. The modeling of network traffic under TCP congestion control. We consider two distinct ses: long-lived traffic corresponding to the transfer of files with infinite length; and on-off traffic consisting of file transfers with exponentially distributed lengths, alternated by times of inactivity (also exponentially distributed). These examples are motivated by [,6.. The modeling of the state-estimation error in a networked control system that ocsionally receives state measurements over a communition network. The rate at which the measurements are transmitted depends on the current estimation error. This type of scheme was shown to out-perform periodic transmiion and n actually be used to approximate an optimal transmiion scheme [7,8.

3 3. Gillespie s stochastic modeling for chemil reactions [9, which describes the evolution of the number of particles involved in a set of reactions. The reactions analyzed were taken from [, and are particularly difficult to simulate due to the existence of two very distinct time sles. Polynomial Stochastic Hybrid Systems A SHS is defined by a stochastic differential equation (SDE) ẋ = f(q, x, t) + g(q, x, t)ṅ, f : Q R n [, ) R n, () a family of m discrete transition/reset maps g : Q R n [, ) R n k, (q, x) = φ l (q, x, t), φ l : Q R n [, ) Q R n, () l {,..., m}, and a family of m transition intensities λ l (q, x, t), λ l : Q R n [, ) [, ), (3) l {,..., m}, where n denotes a k-vector of independent Brownian motion procees and Q a (typilly finite) discrete set. A SHS characterizes a jump proce q : [, ) Q lled the discrete state; a stochastic proce x : [, ) R n with piecewise continuous sample paths lled the continuous state; and m stochastic counters N l : [, ) N lled the transition counters. In eence, between transition counter increments the discrete state remains constant whereas the continuous state flows according to (). At transition times, the continuous and discrete states are reset according to (). Each transition counter N l counts the number of times that the corresponding discrete transition/reset map φ l is activated. The frequency at which this occurs is determined by the transition intensities (3). In particular, the probability that the counter N l will increment in an elementary interval (t, t + dt, and therefore that the corresponding transition takes place, is given by λ l (q(t), x(t), t)dt. In practice, one n think of the intensity of a transition as the instantaneous rate at which that transition occurs. The reader is referred to [ for a mathematilly precise characterization of this SHS. The following result n be used to compute expectations on the state of a SHS. For briefne, we omit a few technil aumptions that are straightforward to verify for the SHSs considered here: Theorem ([). Given a function ψ : Q R n [, ) R that is twice continuously differentiable with respect to its second argument and once continuously differentiable with respect to the third one, we have that d E[ψ(q(t), x(t), t) dt = E[(Lψ)(q(t), x(t), t), (4)

4 where (q, x, t) Q R n [, ) (Lψ)(q, x, t) := and ψ(q,x,t) ψ(q,x,t) t, x ψ(q, x, t) ψ(q, x, t) f(q, x, t) + + x t + trace ψ(q, x) g(q, x, t)g(q, x, t) + x mx + ψ`φ l (q, x, t), t ψ(q, x, t) λ l (q, x, t), (5) l=, and ψ(q,x) x denote the partial derivative of ψ(q, x, t) with respect to t, the gradient of ψ(q, x, t) with respect to x, and the Heian matrix of ψ with respect to x, respectively. The operator ψ Lψ defined by (5) is lled the extended generator of the SHS. We say that a SHS is polynomial (pshs) if its extended generator L is closed on the set of finite-polynomials in x, i.e., (Lψ)(q, x, t) is a finite-polynomial in x for every finite-polynomial ψ(q, x, t) in x. By a finite-polynomials in x we mean a function ψ(q, x, t) such that x ψ(q, x, t) is a (multi-variable) polynomial of finite degree for each fixed q Q, t [, ). A pshs is obtained, e.g., when the vector fields f and g, the reset maps φ l, and the transition intensities λ l are all finite-polynomials in x. Examples of Polynomial Stochastic Hybrid Systems Example (TCP long-lived [). The congestion window size w [, ) of a long-lived TCP flow n be generated by a SHS with continuous dynamics ẇ = and a reset map w w p w, with intensity λ(w) :=. The roundtrip-time and the drop-rate p are parameters that we aume constant. This SHS has a single discrete mode that we omit for simplicity and its generator is given by (Lψ)(w) = ψ(w) w + pw `ψ(w/) ψ(w), which is closed on the set of finite-polynomials in w. Example (TCP on-off [). The congestion window size w [, ) for a stream of TCP flows separated by inactivity periods n be generated by a SHS with three discrete modes Q := {,, off}, one corresponding to slowstart, another to congestion avoidance, and the final one to flow inactivity. Its continuous dynamics are defined by 8 (log )w >< q = ẇ = q = >: q = off; the reset maps aociated with packet drops, end of flows, and start of flows are given by φ (q, w) := ( ), w, φ (q, w) := (off, ), and φ 3 (q, w) := (, w ), respectively; and the corresponding intensities are λ (q, w) := ( p w q {, } q = off λ (q, w) := ( w k q {, } q = off

5 λ 3(q, w) := ( τ off q = off q {, }. The round-trip-time, the drop-rate p, the average file size k (exponentially distributed), the average off-time τ off (also exponentially distributed), and the initial window size w are parameters that we aume constant. The generator for this SHS is given by 8 >< (Lψ)(q, w) = >: (log )w ψ(,w) w pw` ψ(,w/) ψ(,w) + pw` ψ(,w) ψ(,w/) ψ(,w) + w ψ(,w ) ψ(off,w) τ off w` ψ(off,) ψ(,w) + k + w` which is closed on the set of finite-polynomials in w. ψ(off,) ψ(,w) k q = q = q = off, Example 3 (Networked control system [7). Suppose that the state of a stochastic slar linear system ẋ = a x + b ṅ is estimated based on state-measurements received through a network. For simplicity we aume that state measurements are noisele and delay free. The corresponding state estimation error e R n be generated by a SHS with continuous dynamics ė = a e + b ṅ and one reset map e that is activated whenever a state measurement is received. It was conjectured in [7 and later shown in [8 that transmitting measurements at a rate that depends on the state-estimation error is optimal when one wants to minimize the variance of the estimation error, while penalizing the average rate at which meages are transmitted. This motivates considering the following reset intensity λ(e) := e ρ, ρ N. This SHS has a single discrete mode that we omitted for simplicity and its generator is given by (Lψ)(e) := a e ψ(e) + b ψ(e) + `ψ() ψ(e) e ρ, e e which is closed on the set of finite-polynomials in e. Example 4 (Deying-dimerizing reaction set [,). The number of particles x := (x, x, x 3 ) of three species involved in the following set of deyingdimerizing reactions c S c, c S S, S 3 c S, S 4 S3 (6) n be generated by a SHS with continuous dynamics ẋ = and four reset maps i i i h x i φ (x) := φ (x) := φ 3(x) := φ 4(x) := h x x x 3 h x x + x 3 h x + x x 3 x x 3 + with intensities λ (x) := c x, λ (x) := c x (x ), λ 3 (x) := c 3 x, and λ 4 (x) := c 4 x, respectively. Since the numbers of particles take values in the discrete set of integers, we n regard the x i as either part of the discrete or continuous state. We choose to regard them as continuous variables beuse we are interested in studying their statistil moments. In this se, the SHS has a single discrete mode that we omit for simplicity and its generator is given by (Lψ)(x) = c x `ψ(x, x, x 3) ψ(x) + c x(x )`ψ(x, x +, x 3) ψ(x) + c 3x `ψ(x +, x, x 3) ψ(x) + c 4x `ψ(x, x, x 3 + ) ψ(x), which is closed on the set of finite-polynomials in x.

6 3 Moment Dynamics To fully characterize the dynamics of a SHS one would like to determine the evolution of the probability distribution for its state (q, x). In general, this is difficult so a more reasonable goal is to determine the evolution of (i) the probability of q(t) being on each mode and (ii) the moments of x(t) conditioned to q(t). In fact, often one n even get away with only determining a few loworder moments and then using results such as Tchebycheff, Markoff, or Bienaymé inequalities to infer properties of the overall distribution. Given a discrete state q Q and a vector of n integers m = (m, m,..., m n ), we define the test-function aociated with q and m to be N n ψ (m) q (q, x) := ( x (m) q = q q q, q Q, x R n and the (uncentered) moment aociated with q and m to be µ (m) q (t) := E [ ψ (m) ( ) q(t), x(t) q t. (7) Here and in the sequel, given a vector x = (x, x,..., x n ), we use x (m) to denote the monomial x m xm x mn n. PSHSs have the property that if one stacks all moments in an infinite vector µ, its dynamics n be written as µ = A (t)µ t, (8) for some appropriately defined infinite matrix A (t). This is beuse q Q, m = (m,..., m n ) N n, the expreion (Lψ(m) q )(q, x, t) is a finite-polynomial in x and therefore n be written as a finite linear combination of test-functions (poibly with time-varying coefficients). Taking expectations on this linear combination and using (4), (7), we conclude that µ (m) q n be written as linear combination of uncentered moments in µ, leading to (8). In the sequel, we refer to (8) as the infinite-dimensional moment dynamics. Analyzing (and even simulating) (8) is generally difficult. However, as mentioned above one n often get away with just computing a few low-order moments. One would therefore like to determine a finite-dimensional system of ODEs that describes the evolution of a few low-order models, perhaps only approximately. When the matrix A is lower triangular (e.g., as in Example 3 with ρ = ), one n simply trunte the vector µ by dropping all but its first k elements and obtain a finite-dimensional system that exactly describes the evolution of the moments. However, in general A has nonzero elements above the main diagonal and therefore if one defines µ R k to contain the top k elements of µ, one obtains from (8) that µ = I k A (t)µ = A(t)µ + B(t) µ, µ = Cµ, (9)

7 where I k denotes a matrix composed of the first k rows of the infinite identity matrix, µ R r contains all the moments that appear in the first k elements of A (t)µ but that do not appear in µ, and C is the projection matrix that extracts µ from µ. Our goal is to approximate the infinite dimensional system (8) by a finite-dimensional nonlinear ODE of the form ν = A(t)ν + B(t) ν(t), ν = ϕ(ν, t), () where the map ϕ : R k [, ) R r should be chosen so as to keep ν(t) close to µ(t). We ll () the trunted moment dynamics and ϕ the truntion function. We need the following two stability aumptions to establish sufficient conditions for the approximation to be valid. Aumption (Boundedne). There exist sets Ω µ and Ω ν such that all solutions to (8) and () starting at some time t in Ω µ and Ω ν, respectively, exist and are smooth on [t, ) with all derivatives of their first k elements uniformly bounded. The set Ω ν is aumed to be forward invariant. Aumption (Incremental Stability). There exists a function β KL such that, for every solution µ to (8) starting in Ω µ at some time t, and every t t, ν Ω ν there exists some ˆµ (t ) Ω µ whose first k elements match ν and µ(t) ˆµ(t) β( µ(t ) ˆµ(t ), t t ), t t, () where µ(t) and ˆµ(t) denote the first k elements of the solutions to (8) starting at µ (t ) and ˆµ (t ), respectively. Remark. Aumption was purposely formulated without requiring Ω µ to be a subset of a normed space to avoid having to choose a norm under which the (infinite) vectors of moments are bounded. The result that follows establishes that the difference between solutions to (8) and () converges to an arbitrarily small ball, provided that a sufficiently large but finite number of derivatives of these signals match point-wise. To state this result, the following notation is needed: We define the matrices C i (t), i N recursively by C (t) = C, C i+ (t) = C i (t)a (t) + Ċi (t), t, i N, and the family of functions L i ϕ : R k [, ) R r, i N recursively by (L ϕ)(ν, t) = ϕ(ν, t), (L i+ ϕ)(ν, t) = (Li ϕ)(ν, t) ν (L `A(t)ν + B(t)ϕ(ν, t) i ϕ)(ν, t) +, t t, ν R k, i N. These definitions allow us to compute time derivatives of µ(τ) and ν(τ) along solutions to (8) and (), respectively, beuse d i µ(t) dt i = C i (t)µ (t), d i ν(t) dt i = (L i ϕ)(ν(t), t), t, i N. () A function β : [, ) [, ) [, ) is of cla KL if β(, t) =, t ; β(s, t) is continuous and strictly increasing on s, t ; and lim t β(s, t) =, s.

8 Theorem ([3). For every δ >, there exists an integer N sufficiently large for which the following result holds: Auming that for every τ, µ Ω µ C i (τ)µ = (L i ϕ)(µ, τ), i {,,..., N}, (3) where µ denotes the first k elements of µ, then µ(t) ν(t) β( µ(t ) ν(t ), t t ) + δ, t t, (4) along all solutions to (8) and () with initial conditions µ (t ) Ω µ ν(t ) Ω ν, respectively, where µ(t) denotes the first k elements of µ (t). and 4 Construction of Approximate Truntions Given a constant δ > and sets Ω µ, Ω ν, it may be very difficult to determine the integer N for which the approximation bound (4) holds. This is beuse, although the proof of Theorem is constructive, the computation of N requires explicit knowledge of the function β KL in Aumption and, at least for most of the examples considered here, this aumption is difficult to verify. Neverthele, Theorem is still useful beuse it provides the explicit conditions (3) that the truntion function ϕ should satisfy for the solution to the trunted system to approximate the one of the original system. For the problems considered here we require (3) to hold for N =, for which (3) simply becomes Cµ = ϕ(µ, τ), CA (τ)µ = ϕ(µ, τ) I k A (τ)µ + µ ϕ(µ, τ), (5) t µ Ω µ, τ. Lacking knowledge of β, we will not be able to explicitly compute for which values of δ (4) will hold, but we will show by simulation that the truntion obtained provides a very accurate approximation to the infinite-dimensional system (8), even for such a small choice of N. We restrict our attention to functions ϕ and sets Ω µ for which it is simple to use (5) to explicitly compute trunted systems. Separable truntion functions: For all the examples considered, we consider functions ϕ of the form ϕ(ν, t) = Λν (Γ ) := Λ ν γ ν γ ν γ k k. ν γ r ν γ r ν γ rk k, (6) for appropriately chosen constant matrices Γ := [γ ij R r k and Λ R r r, with Λ diagonal. In this se, (5) becomes Cµ = Λ µ (Γ ), (7a) CA (τ)µ = Λ diag[cµ Γ diag[µ, µ,..., µ k I k A (τ)µ. (7b)

9 Deterministic distributions: A set Ω µ that is particularly tractable corresponds to deterministic distributions F det := {P ( ; q, x) : x Ω x, q Q}, where P ( ; q, x) denotes the distribution of (q, x) for which q = q and x = x with probability one; and Ω x a subset of the continuous state space R n. For a particular distribution P ( ; q, x), the (uncentered) moment aociated with q and m N n is given by µ (m) q := ψ (m) q ( q, x)p (d q d x; q, x) = ψ (m) q (q, x) := { x (m) q = q q q, and therefore the vectors µ in Ω µ have this form. Although this family of distributions may seem very restrictive, it will provide us with truntions that are accurate even when the pshss evolve towards very nondeterministic distributions, i.e., with signifint variance. For this set Ω µ, (7) takes a particularly simple form and the following result provides a simple set of conditions to test if a truntion is poible. Lemma ([3). Let Ω µ be the set of deterministic distributions F det with Ω x containing some open ball in R n and consider truntion functions ϕ of the form (6). The following conditions are neceary for the existence of a function ϕ of the form (6) that satisfies (5):. For every moment µ (m l) q l in µ the polynomial i= q i=q l a l,i x (mi) must belong to the linear subspace generated by the polynomials. For every moment µ (m l) q l n X o (m a j,i x l m j +m i ) : j k, q j = q l. i= q i =q l in µ and every moment µ (mi) q i in µ with q i q l, we must have a l,i =. Here we are denoting by a j,i the jth row, ith column entry of A. Condition imposes a diagonal-band-like structure on the submatrices of A consisting of the rows/columns that correspond to each moment that appears in µ. This condition holds for Examples,, and 3, but not for Example 4. However, we will see that the moment dynamics of this example n be simplified so as to satisfy this condition without introducing a signifint error. Condition imposes a form of decoupling between different modes in the equations for µ. This condition holds trivially for all examples that have a single discrete mode. It does not hold for Example, but also here it is poible to simplify the moment dynamics to satisfy this condition without introducing a signifint error. We are considering polynomials with integer (both positive and negative) powers.

10 probability of drop p rate mean E[r M. Carlo red. model rate standard deviation Std[r M. Carlo red. model Fig.. Comparison between Monte Carlo simulations and the trunted model (8), (9) for Example, with = 5ms and a step in the drop-rate p from % to %. 5 Examples of Truntions We now present trunted systems for the several examples considered before and discu how the trunted models compare to estimates of the moments obtained from Monte Carlo simulations. All Monte Carlo simulations were rried out using the algorithm described in [. Estimates of the moments were obtained by averaging a large number of Monte Carlo simulations. In most plots, we used a sufficiently large number of simulations so that the 99% confidence intervals for the mean nnot be distinguished from the point estimates at the resolution used for the plots. Ir is worth to emphasize that the results obtained through Monte Carlo simulations required computational efforts orders of magnitude higher than those obtained using the trunted systems. Example (TCP long-lived). Since for this system it is particularly meaningful to consider moments of the packet sending rate r := w, we choose ψ(m) (w) = w m, m N m. We consider a truntion whose state contains the first and second moments of the sending rate. In this se, (9) n be written as follows: [ µ () µ () [ = p [ µ () µ () [ [ + + 3p 4 µ, (8) where µ := µ (3) evolves according to µ (3) = 3µ() 7pµ(4) 8. In this se, (7) has a unique solution ϕ, resulting in a trunted system given by (8) and µ = ϕ(µ (), µ () ) := (µ() ) 5 (µ () ). (9) Figure shows a comparison between Monte Carlo simulations and this trunted model. A step in the drop probability was introduced at time t = 5sec to show that the trunted model also ptures well transient behavior.

11 Example (TCP on-off). For this system we also consider moments of the sending rate r := on the and modes, and therefore we use w ( off (q, w) = ψ () ψ (m) (q, w) = q = off q {, } ( w m m q = q {, off} ψ (m) (q, w) = ( w m m q = q {, off} We consider a truntion whose state contains the zeroth, first, and second moments of the sending rate. In this se, (9) n be written as follows: 6 4 µ () off µ () µ () µ () µ () µ () µ () 3 = where µ := [ µ (3) τ off τ off k k k p p k τ off w log k p p k p τ off w log 4 µ (3) evolves according to µ (3) µ () off µ () µ () µ () µ () µ () µ () = τ off w3 3 µ() off + log 8 µ(3) ( k + p)µ(4), µ (3) = 3 µ() + p 8 µ(4) ( k + 7p 8 )µ(4). k p p 4 k 3p µ, () (a) (b) However, () does not satisfy condition in Lemma beuse the different discrete modes do not appear decoupled: µ (3) depends on µ () off, and µ(3) depends on µ (4). For the purpose of determining ϕ, we ignore the cro coupling terms and approximate () by µ (3) log 8 µ(3) ( k + p)µ(4), µ (3) 3 µ() ( k + 7p 8 )µ(4). () The validity of these approximations generally depends on the network parameters. When () is used, it is straightforward to verify that (7) has a unique solution ϕ, resulting in a trunted system given by () and [ µ = ϕ(µ) = µ () (µ() )3 (µ () )3 (µ () ) () 5 (µ ). (3) (µ () ) Figure shows a comparison between Monte Carlo simulations and the trunted model (), (3). The dynamics of the first and second order moments are accurately predicted by the trunted model. In preparing this paper, several simulation were executed for different network parameters and initial conditions. Figure shows typil best-se (before t = ) and worst-se (after t = ) results.

12 Fig.. (left) Comparison between Monte Carlo simulations (solid lines) and the trunted model (), (3) (dashed lines) for Example, with = 5ms, τ off = sec, k =.39 packets (corresponding to 3.58KB files broken into 5bytes packets, which is consistent with the file-size distribution of the UNIX file system reported in [4), and a step in the drop-rate p from % to % at t = sec. probability of drop p b probability on each mode mean rate E[r rate standard deviation Std[r total total E[e M. Carlo red. model E[e 4 M. Carlo red. model E[e 6 M. Carlo red. model Fig. 3. (right) Comparison between Monte Carlo simulations (solid lines) and the trunted model (4), (5) (dashed lines) for Example 3, with a =, q =, and a step in the parameter b from to at time t =.5sec. Example 3 (Networked control system). Now ψ (m) (e) = e m, m N and (Lψ (m) )(e) = a m ψ (m) (e) + m(m )b ψ (m ) (e) ψ (m+ρ) (e). For ρ =, the infinite-dimensional dynamics have a lower-triangular structure and therefore an exact truntion is poible. However, this se is le interesting beuse it corresponds to a reset-rate that does not depend on the continuous state and is therefore farther from the optimal [7,8. We consider here ρ =. In this se, the odd and even moments are decoupled and n be studied independently. It is straightforward to check that if the initial distribution of e is symmetric around zero, it will remain so for all times and therefore all odd moments are constant and equal to zero. Regarding the even moments, the smallest truntion for which condition in Lemma holds is a third order one, for which (9) n be written as follows: [ µ () µ (4) µ (6) = [ a 6b 4a 5b 6a [ µ () + µ (4) µ (6) [ [ b + µ, (4) where µ := µ (8) evolves according to µ (8) = 8b µ (6) + 8aµ (8) µ (). It is straightforward to verify that (7) has a unique solution ϕ, resulting in a trunted system given by(4) and µ = ϕ(µ (), µ (4), µ (6) ) = µ () µ (6) µ (4) 3. (5)

13 Figure 3 shows a comparison between Monte Carlo simulations and the trunted model (4), (5). The dynamics of the all the moments are accurately predicted by the trunted model. The nonlinearity of the underlying model is apparent by the fact that halving b at time t =.5sec, which corresponds to dividing the variance of the noise by 4, only results in approximately dividing the variance of the estimation error by. Example 4 (Deying-dimerizing reaction set). For this system the test functions are of the form ψ (m,m,m) (x) = x m xm xm3 3, m, m, m 3 N and m X (Lψ (m,m,m ) )(x) = c + c m,m X i,j= (i,j) (m,m ) m,m X + c 3 i= i,j= (i,j) (m,m ) ( m i ) ( ) m i ψ (i+,m,m 3 ) (x) ( m i ) ` m j ( ) m i`ψ (i+,j,m3) (x) ψ (i+,j,m3) (x) ( m i ) ` m i j m ( ) m j ψ (i,j+,m3) (x) m,m X 3 + c 4 i,j= (i,j) (m,m 3 ) ( m i ) ` m 3 j ( ) m i ψ (m,i+,j) (x), (6) where the summations result from the power expansions of the terms (x i c) mi. For this example we consider a truntion whose state contains all the first and second order moments for the number of particles of the first and second species. To keep the formulas small, we omit from the truntion the second moments of the third species, which does not appear as a reactant in any reaction and therefore its higher order statistics do not affect the first two. In this se, (9) n be written as follows: µ (,,) µ (,,) µ (,,) µ (,,) µ (,,) µ (,,) = c +c c 3 c c c c 3 c 4 c 4 c c 4c 3 4c c 4c 3 c c c 3+c 4 c 3 c 4 c c c 3 3c c 3 c c c 3 c 4 where µ := [ µ (3,,) µ (,,) evolves according to µ (,,) µ (,,) µ (,,) µ (,,) µ (,,) µ (,,) µ (3,,) = ( c + 4c )µ (,,) + 8c 3µ (,,) + (3c c )µ (,,) + c 3µ (,,) + c c c c (7) + ( 3c + 9c )µ (3,,) + 6c 3µ (,,) 3c µ (4,,) (8a) µ (,,) = c µ (,,) 4c 3µ (,,) + 4c µ (,,) + 4c 3µ (,,) + (c c 4c 3)µ (,,) 5cµ(3,,) + (4c c c 3 c 4)µ (,,) + 4c 3µ (,,) + cµ(4,,) c µ (3,,). (8b) µ,

14 Table. Comparison between estimates obtained from Monte Carlo simulations and the trunted model for Example 4. The Monte Carlo data was taken from [. Source for the estimates E[x (.) E[x (.) StdDev[x (.) StdDev[x (.), MC. simul model (7), (9) This system does not satisfy condition in Lemma beuse the µ (,,), µ (,,) terms in the right-hand sides of (8) lead to monomials in x and x that do not exist in any of the polynomials { i= a j,i x (m l m j+m i) : j 6 }. These terms n be traced back to the lowest-order terms in power expansions in (6) and disappear if we disrd them. This leads to a simplified version of (7) for which condition in Lemma does hold, allowing us to find a unique solution ϕ to (7), resulting in a trunted system given (7) and µ = ϕ(µ) = [ ( µ (,,) µ (,,) ) 3 µ (,,) µ (,,) ( µ (,,) µ (,,) ). (9) Ignoring the lowest-order powers of x and x in the power expansions is valid when the populations of these species are high. In practice, the approximation still seems to yield good results even when the populations are fairly small. Figure 4 shows a comparison between Monte Carlo simulations and the trunted model (7), (9). The coefficients used were taken from [, Example : c =, c =, c 3 =, c 4 =. In Fig. 4(a), we used the same initial conditions as in [, Example : x () = 4, x () = 798, x 3 () =. The match is very accurate, as n be confirmed from Table. The values of the parameters chosen result in a pshs with two distinct time sles, which makes this pshs computationally difficult to simulate ( stiff in the terminology of [). Fig. 4(a) shows the evolution of the system on the slow manifold, whereas Fig. 4(b) zooms in on the interval [, 5 4 and shows the evolution of the system towards this manifold when it starts away from it at x () = 8, x () =, x 3 () =. Figures 4(c) 4(d) shows another simulation of the same reactions but for much smaller initial populations: x () =, x () =, x 3 () = 5. The trunted model still provides an extremely good approximation, with signifint error only in the covariance between x and x when the averages and standard deviation of these variables get below one. 6 Conclusions and Future Work In this paper, we showed that the infinite-dimensional linear moment dynamics of a pshs n be approximated by a finite-dimensional nonlinear ODE with arbitrary precision. Moreover, we provided a procedure to build this type of approximation. The methodology was illustrated using a varied pool of examples, demonstrating its wide applibility. Several observations arise from these examples, which point to directions for future research:

15 5 5 5 E[x E[x population means E[x STD[x STD[x population standard deviations Corr[x,x populations correlation coefficient population means 8 6 E[x 4 E[x E[x x 4 population standard deviations STD[x 5 STD[x x 4 populations correlation coefficient.9.95 Corr[x,x x 4 (a) Large population over a long time sle (b) Large population over a short time sle E[x E[x population means STD[x STD[x population standard deviations Corr[x,x populations correlation coefficient E[x 3 population means 3 E[x E[x E[x x 3 population standard deviations 4 STD[x 3 STD[x x 3 populations correlation coefficient.9.95 Corr[x,x x 3 (c) Small population over a long time (d) Small population over short time sle Fig. 4. Comparison between Monte Carlo simulations (solid lines) and the trunted model (7), (9) (dashed lines) for Example 4.. In all the examples presented, we restricted our attention to truntion functions ϕ of the form (6) and we only used deterministic distributions to compute ϕ. Mostly likely, better results could be obtained by considering more general distributions, which may require more general forms for ϕ.. The truntion of pshss that model chemil reactions proved especially accurate. This motivates the search for systematic procedures to automatilly construct a trunted system from chemil equations such as (6). Another direction for future research consists of comparing the trunted models obtains here with those in [5.

16 An additional direction for future research consists of establishing computable bounds on the error between solutions to the infinite-dimensional moments dynamics and to its finite-dimensional truntions. References. Hespanha, J.: Stochastic hybrid systems: Applitions to communition networks. In Alur, R., Pappas, G., eds.: Hybrid Systems: Computation and Control. Number 993 in Lect. Notes in Comput. Science. Springer-Verlag, Berlin (4) Davis, M.H.A.: Markov models and optimization. Monographs on statistics and applied probability. Chapman & Hall, London, UK (993) 3. Hu, J., Lygeros, J., Sastry, S.: Towards a theory of stochastic hybrid systems. In Lynch, N.A., Krogh, B.H., eds.: Hybrid Systems: Computation and Control. Volume 79 of Lect. Notes in Comput. Science., Springer () Pola, G., Bujorianu, M., Lygeros, J., Benedetto, M.D.: Stochastic hybrid models: An overview. In: Proc. of IFAC Conf. on Anal. and Design of Hybrid Syst. (3) 5. Bujorianu, M.: Extended stochastic hybrid systems and their reachability problem. In: Hybrid Systems: Computation and Control. Lect. Notes in Comput. Science. Springer-Verlag, Berlin (4) Bohacek, S., Hespanha, J., Lee, J., Obraczka, K.: A hybrid systems modeling framework for fast and accurate simulation of data communition networks. In: Proc. of ACM SIGMETRICS. (3) 7. Xu, Y., Hespanha, J.: Communition logics for networked control systems. In: Proc. of 4 Amer. Contr. Conf. (4) 8. Xu, Y., Hespanha, J.: Optimal communition logics for networked control systems. In: Proc. of 43rd Conf. on Decision and Contr. (4) 9. Gillespie, D.T.: A general method for numerilly simulating the stochastic time evolution of coupled chemil reactions. J. Comp. Physics (976) Gillespie, D., Petzold, L.: Improved leap-size selection for accelerated stochastic simulation. J. of Chemil Physics 9 (3) Rathinam, M., Petzold, L., Cao, Y., Gillespie, D.: Stiffne in stochastic chemilly reacting systems: The implicit tau-leaping method. J. of Chemil Physics 9 (3) Hespanha, J.: A model for stochastic hybrid systems with applition to communition networks. Submitted to the Int. Journal of Hybrid Systems (4) 3. Hespanha, J.P.: Polynomial stochastic hybrid systems (extended version). Technil report, University of California, Santa Barbara, Santa Barbara (4) Available at 4. Irlam, G.: Unix file size survey 993. Available at (994) 5. Van Kampen, N.: Stochastic Procees in Physics and Chemistry. Elsevier ()