MODELING AND ANALYSIS OF STOCHASTIC HYBRID SYSTEMS

Transcription

1 1 MODELING AND ANALYSIS OF STOCHASTIC HYBRID SYSTEMS João P Hespanha Center for Control, Dynamil Systems, and Computation University of California, Santa Barbara, CA 9311 hespanha@eceucsbedu hespanha Abstract The author describes a model for Stochastic Hybrid Systems (SHSs) where transitions between discrete modes are triggered by stochastic events The rate at which these transitions occur is allowed to depend both on the continuous and the discrete states of the SHS Several examples of SHSs arising from a varied pool of applition areas are discued These include modeling of the Transmiion Control Protocol s (TCP) algorithm for congestion control both for long-lived and on-off flows; state-estimation for networed control systems; and the stochastic modeling of chemil reactions These examples illustrate the use of SHSs as a modeling tool Attention is mostly focused on polynomial stochastic hybrid systems (pshss) that generally correspond to stochastic hybrid systems with polynomial continuous vector fields, reset maps, and transition intensities For pshss, the dynamics of the statistil moments of the continuous states evolve according to infinite-dimensional linear ordinary differential equations (ODEs) We show that these ODEs n be approximated by finite-dimensional nonlinear ODEs with arbitrary precision Based on this result, a procedure to build this type of approximations for certain claes of pshss is provided This procedure is applied to several examples and the accuracy of the results obtained is evaluated through comparisons with Monte Carlo simulations I INTRODUCTION Hybrid systems are characterized by a state-space that n be partitioned into a continuous domain (typilly R n ) and a discrete set (typilly finite) For the stochastic hybrid systems considered here, both the continuous and the discrete components of the state are stochastic procees The evolution of the continuous-state is determined by a stochastic differential equation and the evolution of the discrete-state by a transition or reset map The discrete transitions are triggered by stochastic events much lie transitions between states of a continuous-time Marov chains However, the rate at which these transitions occur may depend on the continuous-state The model used This version of the paper differs from the one that appeared in print in that we corrected a typo in Example 3 The author would lie to than Shauna Bopardiar for maing us aware of it This material is based upon wor supported by the National Science Foundation under grants CCR-31184, ANI-3476

2 here for SHSs, whose formal definition n be found in Sec II, was introduced in [1 and is heavily inspired by the Piecewise-Deterministic Marov Procees (PDPs) in [ Alternative models n be found in [3 5 The extended generator of a stochastic hybrid system allows one to compute the time-derivative of a test function of the state of the SHS along solutions to the system, and n be viewed as a generalization of the Lie derivative for deterministic systems [1, Polynomial stochastic hybrid systems (pshss) are characterized by extended generators that map polynomial test functions into polynomials This happens, eg, when the continuous vector fields, the reset maps, and the transition intensities are all polynomial functions of the continuous state An important property of pshss is that if one creates an infinite vector containing the probabilities of all discrete modes, as well as all the multi-variable statistil moments of the continuous state, the dynamics of this vector are governed by an infinitedimensional linear ordinary differential equation (ODE), which we ll the infinite-dimensional moment dynamics and define formally in Sec IV SHSs n model large claes of systems but their formal analysis presents signifint challenges Although it is straightforward to write partial differential equations (PDEs) that expre the evolution of the probability distribution function for their states, generally these PDEs do not admit analytil solutions The infinite-dimensional moment dynamics provides an alternative characterization for the distribution of the state of a pshs Although generally statistil moments do not provide a description of a stochastic proce as accurate as the probability distribution, results such as Tchebycheff, Maroff, or Bienaymé inequalities [6, pp n be used to infer properties of the distribution from its moments In general, the infinite-dimensional linear ODEs that describe the moment dynamics for pshss are still not easy to solve analytilly However, sometimes they n be accurately approximated by a finite-dimensional nonlinear ODE, which we ll the trunted moment dynamics We show in Sec IV that, under suitable stability aumptions, it is in principle poible for a finite-dimensional nonlinear ODE to approximate the infinite-dimensional moment dynamics, up to an error that n be made arbitrarily small This result is based on a Taylor series expansion that is used to prove that the difference between the solutions to the finite- and infinite-dimensional ODEs n be made arbitrarily small on a compact time interval [, T Subsequently, this is extended to the unbounded time interval [, ) using an argument similar to the one found in [7 to analyze two time sle systems Aside from its theoretil interest, the above mentioned result motivates a procedure to actually construct finitedimensional approximations for certain claes of pshss This procedure, which is described in Sec V, is applible to pshss for which the (infinite) matrix that characterizes the moment dynamics exhibits a certain diagonal-band structure and appropriate decoupling between certain moments of distinct discrete modes The details of this structure n be found in Lemma 1 To illustrate the applibility of the results, we consider several systems that appeared in the literature and that n be modeled by pshss (cf Sec III) For each example, we construct in Sec VI trunted moment dynamics and evaluate how they compare with estimates for the moments obtained from a large number of Monte Carlo simulations The examples considered include:

3 3 1) The modeling of the sending rate for networ traffic under TCP congestion control We consider two distinct ses: long-lived traffic corresponding to the transfer of files with infinite length; and on-off traffic consisting of file transfers with exponentially distributed lengths, alternated by times of inactivity (also exponentially distributed) These examples are motivated by [1, 8 ) The modeling of the state-estimation error in a networed control system that ocsionally receives state measurements over a communition networ The rate at which the measurements are transmitted depends on the current estimation error This type of scheme was shown to out-perform periodic transmiion and n actually be used to approximate an optimal transmiion scheme [9, 1 3) Gillespie s stochastic simulation algorithm (SSA) for chemil reactions [11, which describes the evolution of the number of particles involved in a set of reactions The reactions analyzed were taen from [1, 13 and are particularly difficult to simulate due to the existence of two very distinct time sles A few other ademic examples with tutorial value are also included A subset of the results and examples presented here appeared in [14 II STOCHASTIC HYBRID SYSTEMS MODELING A SHS is defined by a stochastic differential equation (SDE) ẋ = f(q,x,t)+g(q,x,t)ṅ, f : Q R n [, ) R n, (1) g : Q R n [, ) R n, a family of m discrete transition/reset maps (q,x) = φ l (q,x,t), φ l : Q R n [, ) Q R n, l {1,,m}, () and a family of m transition intensities λ l (q,x,t), λ l : Q R n [, ) [, ), l {1,,m}, (3) where n denotes a -vector of independent Brownian motion procees and Q a (typilly finite) discrete set A SHS characterizes a jump proce q : [, ) Q lled the discrete state; a stochastic proce x : [, ) R n with piecewise continuous sample paths lled the continuous state; and m stochastic counters N l : [, ) N lled the transition counters In this paper, R denotes the set of real numbers and N the set of nonnegative integers In eence, between transition counter increments the discrete state remains constant whereas the continuous state flows according to (1) At transition times, the continuous and discrete states are reset according to () Each transition counter N l counts the number of times that the corresponding discrete transition/reset map φ l is activated The frequency at which this occurs is determined by the transition intensities (3) In particular, the probability that the counter N l will increment in an elementary interval (t,t + dt, and therefore that the corresponding transition taes place, is given by λ l (q(t),x(t),t)dt In practice, one n thin of the intensity of a

4 4 transition as the instantaneous rate at which that transition occurs The reader is referred to [1 for a mathematilly precise characterization of this SHS It is often convenient to represent SHSs by a directed graph as in Figure 1, where each node corresponds to a discrete mode and each edge to a transition between discrete modes The nodes are labeled with the corresponding discrete mode and the vector fields that determines the evolution of the continuous state in that particular mode The start of each edge is labeled with the probability of transition in an elementary interval and the end of each edge is labeled with the corresponding reset-map q = q 3 λ l1 (q 3,x,t)dt ẋ = f(q 3,x,t) (q 3,x) φ l1 (q 3,x,t) q = q 1 ẋ = f(q 1,x,t) q = q ẋ = f(q,x,t) λ l (q 1,x,t)dt (q 1,x) φ l (q 1,x,t) Fig 1 Graphil representation of a stochastic hybrid system The following result n be used to compute expectations on the state of a SHS For briefne, we omit a few technil aumptions that are straightforward to verify for the SHSs considered here: Theorem 1 ([1): Given a function ψ : Q R n [, ) R that is twice continuously differentiable with respect to its second argument and once continuously differentiable with respect to the third one, we have that where (q,x,t) Q R n [, ) (Lψ)(q,x,t) := ψ(q,x,t) x and ψ(q,x,t) t, ψ(q,x,t) x, and ψ(q,x) x de[ψ(q(t),x(t),t) dt f(q,x,t)+ ψ(q,x,t) + t + 1 ( trace ψ(q,x) = E[(Lψ)(q(t),x(t),t), (4) x g(q,x,t)g(q,x,t) ) + m + l=1 ( ψ ( φ l (q,x,t),t ) ψ(q,x,t) ) λ l (q,x,t), (5) denote the partial derivative of ψ(q,x,t) with respect to t, the gradient of ψ(q,x,t) with respect to x, and the Heian matrix of ψ with respect to x, respectively The operator ψ Lψ defined by (5) is lled the extended generator of the SHS We say that a SHS is polynomial (pshs) if its extended generator L is closed on the set of finite-polynomials in x, ie, (Lψ)(q,x,t) is a finite-polynomial in x for every finite-polynomial ψ(q,x,t) in x By a finite-polynomials in x we mean a function ψ(q,x,t) such that x ψ(q,x,t) is a (multi-variable) polynomial of finite degree for each fixed q Q, t [, ) A pshs is obtained, eg, when the vector fields f and g, the reset maps φ l, and the transition intensities λ l are all finite-polynomials in x

5 5 dt ǫ dt ǫ ẋ = ax x x b ǫ x x+b ǫ Fig Random wal SHS in Example 1 III EXAMPLES OF POLYNOMIAL STOCHASTIC HYBRID SYSTEMS Example 1 (Random wal): Let x R denote the velocity of a particle moving in a viscous medium that every so often is instantaneously increased or decrease by a fixed amount b ǫ, with b R, ǫ > The interval of time between increases (as well as between decreases) is exponentially distributed with mean ǫ > The proce x n be generated by a SHS (represented graphilly in Figure ) with continuous dynamics ẋ = ax and two reset maps x φ 1, (x) := x±b ǫ both with constant transitions intensities λ 1, (x) := 1 ǫ The constant a accounts for viscous friction This SHS has a single discrete mode that we omitted for simplicity and its generator is given by (Lψ)(x) = ax ψ(x) x + ψ(x+b ǫ)+ψ(x b ǫ) ψ(x), ǫ which is closed on the set of finite-polynomials in x It is well nown that as ǫ, x approaches the solution to the SDE ẋ = ax+bṅ, where n is a Brownian motion proce This SDE is easy to solve analytilly, at least when the initial distribution of x is Gauian Our (somewhat ademic) goal here would be to study what happens when ǫ is not necearily close to zero and when the distribution of the initial state is arbitrary ẇ = 1 pwdt Fig 3 TCP long-lived SHS in Example w w Example (TCP long-lived [15): The congestion window size w [, ) of a long-lived TCP flow n be generated by a SHS (represented graphilly in Figure 3) with continuous dynamics ẇ = 1 and a reset map w w pw, with intensity λ(w) := The round-trip-time and the drop-ratep are parameters that we aume constant This SHS has a single discrete mode that we omit for simplicity and its generator is given by (Lψ)(w) = 1 ψ(w) w + pw ( ) ψ(w/) ψ(w), which is closed on the set of finite-polynomials in w

6 6 dt τ off q = off w ẇ = w w wdt pwdt q = ẇ = (log)w pwdt w w q = ẇ = 1 w w Fig 4 TCP on-off SHS in Example 3 Example 3 (TCP on-off [15): The congestion window size w [, ) for a stream of TCP flows separated by inactivity periods n be generated by a SHS (represented graphilly in Figure 4) with three discrete modes Q := {,,off}, one corresponding to slow-start, another to congestion avoidance, and the final one to flow inactivity Its continuous dynamics are defined by (log )w q = ẇ = 1 q = q = off; the reset maps aociated with pacet drops, end of flows, and start of flows are given by (q,w) φ 1 (q,w) := (, w ), (q,w) φ (q,w) := (off,), (q,w) φ 3 (q,w) := (,w ) respectively; and the corresponding intensities are pw q {,} w q = λ 1 (q,w) := λ (q,w) := q = off q = {,off} 1 τ λ 3 (q,w) := off q = off q {,} The round-trip-time RT T, the drop-rate p, the average file size (exponentially distributed), the average off-time τ off (also exponentially distributed), and the initial window size w are parameters that we aume constant The generator for this SHS is given by (Lψ)(q,w) = (log )w 1 ψ(,w) ( ) ψ(,w) w + pw ψ(,w/) ψ(,w) ( ) w + pw ψ(,w/) ψ(,w) ψ(,w ) ψ(off,w) τ off which is closed on the set of finite-polynomials in w + w ( ) ψ(off,) ψ(,w) q = q = q = off, Example 4 (Networed control system [9): Suppose that the state of a stochastic slar linear system ẋ = ax+ b ṅ is estimated based on state-measurements received through a networ For simplicity we aume that state

7 7 ė = ae+bṅ e ρ dt e Fig 5 Networed control system SHS in Example 4 measurements are noisele and delay free The corresponding state estimation error e R n be generated by a SHS (represented graphilly in Figure 5) with continuous dynamics ė = ae + bṅ and one reset map e that is activated whenever a state measurement is received It was conjectured in [9 and later shown in [1 that transmitting measurements at a rate that depends on the state-estimation error is optimal when one wants to minimize the variance of the estimation error, while penalizing the average rate at which meages are transmitted This motivates considering a reset intensity of the form λ(e) := e ρ, for some ρ N This SHS has a single discrete mode that we omitted for simplicity and its generator is given by (Lψ)(e) := ae ψ(e) e which is closed on the set of finite-polynomials in e + b ψ(e) e + ( ψ() ψ(e) ) e ρ, ẋ = c x(x 1)dt x x Fig 6 Deying reaction SHS in Example 5 Example 5 (Deying reaction): The number of particles x of a species S 1 involved in a deying reaction S 1 c n be generated by a SHS (represented graphilly in Figure 6) with continuous dynamics ẋ = and reset map x x with intensity λ(x) := c x(x 1) Since the number of particles taes values in the discrete set of integers, we n regard x as either part of the discrete or the continuous state We choose to regard it as a continuous variable beuse we are interested in studying its statistil moments In this se, the SHS has a single discrete mode that we omit for simplicity This model is exactly equivalent to (author?) s [11 stochastic simulation algorithm (SSA) and its generator is given by (Lψ)(x) = c x(x 1)( ψ(x ) ψ(x) ), which is closed on the set of finite-polynomials in x

8 8 c 1 x 1 dt c x 1(x 1 1)dt x 1 x 1 1 x x 1 x 3 x 3 +1 ẋ 1 = ẋ = ẋ 3 = x 1 x 1 x x +1 x 1 x 1 + x x 1 c 4 x dt c 3 x dt Fig 7 Deying reaction SHS in Example 6 Example 6 (Deying-dimerizing reaction set [1, 13): The number of particles x := (x 1,x,x 3 ) of three species involved in the following set of deying-dimerizing reactions c S 1 c 1, c S1 S, S 3 c S1, S 4 S3 (6) n be generated by a SHS (represented graphilly in Figure 7) with continuous dynamics ẋ = and four reset maps x 1 1 x 1 x φ 1 (x) := x x φ (x) := x +1 with intensities x 3 x 3 x 1 + x 1 x φ 3 (x) := x 1 x φ 4(x) := x 1 x 3 +1 x 3 λ 1 (x) := c 1 x 1, λ (x) := c x 1(x 1 1), λ 3 (x) := c 3 x, λ 4 (x) := c 4 x, respectively Also here we choose to regard the x i as continuous variables beuse we are interested in studying their statistil moments The generator for this SHS is given by (Lψ)(x) = c 1 x 1 ( ψ(x1 1,x,x 3 ) ψ(x) ) + c x 1(x 1 1) ( ψ(x 1,x +1,x 3 ) ψ(x) ) +c 3 x ( ψ(x1 +,x 1,x 3 ) ψ(x) ) +c 4 x ( ψ(x1,x 1,x 3 +1) ψ(x) ), which is closed on the set of finite-polynomials in x IV MOMENT DYNAMICS To fully characterize the dynamics of a SHS one would lie to determine the evolution of the probability distribution for its state (q,x) In general, this is difficult so a more reasonable goal is to determine the evolution of (i) the probability of q(t) being on each mode and (ii) the moments of x(t) conditioned to q(t) In fact, often one

9 9 n even get away with only determining a few low-order moments and then using results such as Tchebycheff, Maroff, or Bienaymé inequalities [6, pp to infer properties of the overall distribution Given a discrete state q Q and a vector of n integers m = (m 1,m,,m n ) N n, we define the test-function aociated with q and m to be ψ (m) q (q,x) := x (m) q = q q q, q Q,x R n and the (uncentered) moment aociated with q and m to be q (t) := E [ ψ (m) ( ) q q(t),x(t) µ (m) t (7) Here and in the sequel, given a vector x = (x 1,x,,x n ), we use x (m) to denote the monomial x m1 1 xm x mn n PSHSs have the property that if one stacs all moments in an infinite vector µ, its dynamics n be written as µ = A (t)µ t, (8) for some appropriately defined infinite matrix A (t) This is beuse q Q,m = (m 1,,m n ) N n, the expreion (Lψ (m) q )(q,x,t) is a finite-polynomial in x and therefore n be written as a finite linear combination of test-functions (poibly with time-varying coefficients) Taing expectations on this linear combination and using (4), (7), we conclude that µ (m) q n be written as a linear combination of uncentered moments in µ, leading to (8) In the sequel, we refer to (8) as the infinite-dimensional moment dynamics Analyzing (and even simulating) (8) is generally difficult However, as mentioned above one n often get away with just computing a few low-order moments One would therefore lie to determine a finite-dimensional system of ODEs that describes the evolution of a few low-order models, perhaps only approximately When the matrix A is lower triangular (eg, as in Example 1 and Example 4 with ρ = ), one n simply trunte the vector µ by dropping all but its first elements and obtain a finite-dimensional system that exactly describes the evolution of the moments 1 However, in general A has nonzero elements above the main diagonal and therefore if one defines µ R to contain the top elements of µ, one obtains from (8) that µ = I A (t)µ = A(t)µ+B(t) µ, µ = Cµ, (9) where I denotes a matrix composed of the first rows of the infinite identity matrix, µ R r contains all the moments that appear in the first elements of A (t)µ but that do not appear in µ, and C is the projection matrix that extracts µ from µ Our goal is to approximate the infinite dimensional system (8) by a finite-dimensional nonlinear ODE of the form ν = A(t)ν +B(t) ν(t), ν = ϕ(ν,t), (1) 1 Other truntions are poible, but it is often convenient to eep the low-order moments of x as the states of the trunted system beuse these are physilly meaningful

10 1 where the map ϕ : R [, ) R r should be chosen so as to eep ν(t) close to µ(t) We ll (1) the trunted moment dynamics and ϕ the truntion function We need the following two stability aumptions to establish sufficient conditions for the approximation to be valid Aumption 1 (Boundedne): There exist sets Ω µ and Ω ν such that all solutions to (8) and (1) starting at some time t in Ω µ and Ω ν, respectively, exist and are smooth on [t, ) with all derivatives of their first elements uniformly bounded The set Ω ν is aumed to be forward invariant Aumption (Incremental Stability): There exists a function β KL such that, for every solution µ to (8) starting in Ω µ at some time t, and every t 1 t, ν 1 Ω ν there exists some ˆµ (t 1 ) Ω µ whose first elements match ν 1 and µ(t) ˆµ(t) β( µ(t 1 ) ˆµ(t 1 ),t t 1 ), t t 1, (11) where µ(t) and ˆµ(t) denote the first elements of the solutions to (8) starting at µ (t 1 ) and ˆµ (t 1 ), respectively Remar 1: Aumption is eentially a requirement of uniform stability for (8), but it was purposely formulated without requiring Ω µ to be a subset of a normed space to avoid having to choose a norm under which the (infinite) vectors of moments are bounded It is important to emphasize that a certain richne of Ω µ is implicit in this aumption Indeed, for the existence of ˆµ (t 1 ) Ω µ whose first components match an arbitrary ν 1 Ω ν one needs Ω ν to be contained in the projection of Ω µ into its first components The result that follows establishes that the difference between solutions to (8) and (1) converges to an arbitrarily small ball, provided that a sufficiently large but finite number of derivatives of these signals match point-wise To state this result, the following notation is needed: We define the matrices C i (t), i N recursively by C (t) = C, C i+1 (t) = C i (t)a (t)+ċi (t), t, i N, and the family of functions L i ϕ : R [, ) R r, i N recursively by (L ϕ)(ν,t) = ϕ(ν,t), (L i+1 ϕ)(ν,t) = (Li ϕ)(ν,t) ν ( ) (L i ϕ)(ν,t) A(t)ν +B(t)ϕ(ν,t) +, t t, ν R, i N These definitions allow us to compute time derivatives of µ(t) and ν(t) along solutions to (8) and (1), respectively, beuse d i µ(t) dt i = C i (t)µ (t), d i ν(t) dt i = (L i ϕ)(ν(t),t), t, i N (1) Theorem (cf Appendix): For every δ >, there exists an integer N sufficiently large for which the following result holds: Auming that for every τ, µ Ω µ C i (τ)µ = (L i ϕ)(µ,τ), i {,1,,N}, (13) A function β : [, ) [, ) [, ) is of cla KL if β(,t) =, t ; β(s,t) is continuous and strictly increasing on s, t ; and lim t β(s,t) =, s

11 11 where µ denotes the first elements of µ, then µ(t) ν(t) β( µ(t ) ν(t ),t t )+δ, t t, (14) along all solutions to (8) and (1) with initial conditions µ (t ) Ω µ and ν(t ) Ω ν, respectively, where µ(t) denotes the first elements of µ (t) The proof of this theorem is technil and therefore we leave it for the appendix The idea behind it is that the conditions (13) guarantees that when the solution µ to the infinite-dimensional system (8) starts in Ω µ, the N + 1 derivatives of its first elements µ match those of the solution ν to the trunted system (1) Using a standard argument based on a Taylor series expansion, we thus conclude that these two solutions will remain close in a bounded interval of length T To extend this to the unbounded interval [, ), we use the stability of the infinite-dimensional (8): After the first interval [,T, we n start another solution ˆµ to (8) on Ω µ, whose first elements ˆµ match ν at time T, and conclude from (13) that ˆµ and ν will remain close on [T,T But beuse of stability µ and ˆµ will converge to each other and therefore µ and ν also remain close This argument n be repeated to obtain the bound (14) on the unbounded time interval [, ) A similar argument was used in the proof of [7, Theorem 1 to analyze two time sle systems It is important to emphasize that for this argument to hold, Ω µ need not be forward invariant beuse we only use (13) for the solutions ˆµ that are always re-started in Ω µ This explains why one may get good matches between the original and the trunted systems, even if (13) only holds for fairly small subsets Ω µ of the overall infinite-dimensional state-space V CONSTRUCTION OF APPROXIMATE TRUNCATIONS Given a constant δ > and sets Ω µ, Ω ν, it may be very difficult to determine the integer N for which the approximation bound (14) holds This is beuse, although the proof of Theorem is constructive, the computation of N requires explicit nowledge of the function β KL in Aumption and, at least for most of the examples considered here, this aumption is difficult to verify Neverthele, Theorem is still useful beuse it provides the explicit conditions (13) that the truntion function ϕ should satisfy for the solution to the trunted system to approximate the one of the original system For the problems considered here we require (13) to hold for N = 1, which corresponds to a second Taylor expansions of the solution Note that for N = 1, (13) simply becomes Cµ = ϕ(µ,τ), CA (τ)µ = ϕ(µ,τ) µ I A (τ)µ + ϕ(µ,τ), (15) t µ Ω µ, τ Lacing nowledge of β, we will not be able to explicitly compute for which values of δ (14) will hold, but we will show by simulation that the truntion obtained provides a very accurate approximation to the infinite-dimensional system (8), even for such a small choice of N We restrict our attention to functions ϕ and sets Ω µ for which it is simple to use (15) to explicitly compute trunted systems

12 1 a) Separable truntion functions:: For all the examples considered, we consider functions ϕ of the form ν γ11 1 ν γ1 ν γ 1 ν γ1 ϕ(ν,t) = Λν (Γ) 1 ν γ ν γ := Λ, (16) ν γr1 1 ν γr ν γ r for appropriately chosen constant matrices Γ := [γ ij R r and Λ R r r, with Λ diagonal In this se, ν γ11 1 ν γ1 ν γ 1 γ 11 ν1 1 ν γ11 1 ν γ1 ν γ 1 γ 1 ν 1 ν γ11 1 ν γ1 ν γ 1 γ 1 ν 1 ϕ(ν) ν γ1 1 ν γ ν γ γ 1 ν1 1 ν γ1 1 ν γ ν γ γ ν 1 ν γ1 1 ν γ ν γ γ ν 1 = Λ ν ν γr1 1 ν γr ν γ r γ r1 ν1 1 ν γr1 1 ν γr ν γ r γ r ν 1 ν γr1 1 ν γr ν γ r γ r ν 1 and therefore (15) becomes = Λdiag[ν (Γ) Γdiag[ν1 1,ν 1,,ν 1 Cµ = Λµ (Γ), (17a) CA (τ)µ = Λdiag[Cµ Γdiag[µ 1 1,µ 1,,µ 1 I A (τ)µ (17b) This particular form for ϕ is convenient beuse (17a) often uniquely defines Λ and then (17b) provides a linear system of equations on Γ, for which it is straightforward to determine if a solution exists b) Deterministic distributions:: A set Ω µ that is particularly tractable corresponds to deterministic distributions F det := {P( ; q,x) : x Ω x,q Q}, where P( ; q,x) denotes the distribution of (q,x) for which q = q and x = x with probability one; and Ω x a subset of the continuous state space R n For a particular distribution P( ; q,x), the (uncentered) moment aociated with q and m N n is given by µ (m) q := ψ (m) q ( q, x)p(d qd x;q,x) = ψ (m) x (m) q = q q (q,x) := q q, and therefore the vectors µ in Ω µ have this form Although this family of distributions may seem very restrictive, it will provide us with truntions that are accurate even when the pshss evolve towards very nondeterministic distributions, ie, with signifint variance For this set Ω µ, (17) taes a particularly simple form and the following result provides a simple set of conditions to test if a truntion is poible Lemma 1 (cf Appendix): Let Ω µ be the set of deterministic distributions F det with Ω x containing some open ball in R n and consider truntion functions ϕ of the form (16) The matrices Λ and Γ must satisfy: 1) Λ must be the identity matrix ) For every moment µ (m l) q l in µ and every moment µ (mi) q i in µ for which q i q l, one must have γ l,i = 3) For every moment µ (m l) q l in µ, one must have i=1 γ l,i m i = m l q i=q l Moreover, the following conditions are neceary for the existence of a function ϕ of the form (16) that satisfies (15):

13 13 4) For every moment µ (m l) q l in µ the polynomial 3 i=1 q i=q l a l,i x (mi) must belong to the linear subspace generated by the polynomials 5) For every moment µ (m l) q l { a j,ix i=1 q i=q l in µ and every moment µ (mi) q i (m l m j+m i) : 1 j, q j = q l } we are denoting by a j,i the jth row, ith column entry of A in µ with q i q l, we must have a l,i = Here Condition 4 imposes a diagonal-band-lie structure on the submatrices of A consisting of the rows/columns that correspond to each moment that appears in µ This condition holds for Examples, 3, and 4 It does not quite hold for Examples 5 and 6 but their moment dynamics n be simplified so as to satisfy this condition without introducing a signifint error Condition 5 imposes a form of decoupling between different modes in the equations for µ This condition holds trivially for all examples that have a single discrete mode It does not hold for Example 3, but also here it is poible to simplify the moment dynamics to satisfy this condition without introducing a signifint error VI EXAMPLES OF TRUNCATIONS We now present trunted systems for the several examples considered before and discu how the trunted models compare to estimates of the moments obtained from Monte Carlo simulations All Monte Carlo simulations were rried out using the algorithm described in [1 Estimates of the moments were obtained by averaging a large number of Monte Carlo simulations In most plots, we used a sufficiently large number of simulations so that the 99% confidence intervals for the mean nnot be distinguished from the point estimates at the resolution used for the plots It is worth to emphasize that the results obtained through Monte Carlo simulations required computational efforts orders of magnitude higher than those obtained using the trunted systems Example 1 (Random wal): For this system we consider test functions of the form ψ (m) (x) = x m, m N, for which (Lψ (m) )(x) = max m + (x+b ǫ) m +(x b ǫ) m x m ǫ m ( ) m = max m + b i ǫ i ψ (m i) (x), i i= i even 3 We are considering polynomials with integer (both positive and negative) powers

14 14 yielding a b a 3b 3a = b + ǫ 6b 4a 5b ǫ 1b 5a µ (1) µ () µ (3) µ (4) µ (5) µ (1) µ () µ (3) µ (4) µ (5) Since µ () (t) = 1, t we excluded µ () from the vector µ, resulting in an affine system (as opposed to linear) Beuse the infinite-dimensional moment dynamics are lower-triangular, this system n be trunted without introducing any error For example, a third order truntion yields µ (1) a µ () = a 3b 3a which n be solved explicitly: for a, and µ (3) µ (1) µ () µ (3) + b µ (1) (t) = e at µ (1) (), µ () (t) = b a (1 e at )+e at µ () (), µ (3) (t) = 3b a (e at e 3at )µ (1) ()+e 3at µ (3) (), µ (1) (t) = µ (1) (), µ () (t) = bt+µ () (), µ (3) (t) = 3btµ (1) ()+µ (3) (), for a = It is interesting to note that these equations do not depend on ǫ beuse this parameter only appears in the equation for µ (4) This means that up to the third order moments this pshs is indistinguishable from the SDE ẋ = ax+bṅ Example (TCP long-lived): For this system it is particularly meaningful to consider moments of the pacet sending rate r := w We therefore choose the test functions to be of the form ψ(m) (w) = wm m, m N, for which yielding (Lψ (m) )(w) = mwm 1 m+1 m 1 pw m+1 m m+1 = mψ(m 1) (w) µ (1) µ () µ (3) µ (4) = 1 p m 1 m ψ(m+1) (w), p 3p p p 16 µ (1) µ () µ (3) µ (4)

15 probability of drop p probability of drop p rate mean E[r M Carlo red model 15 1 rate mean E[r M Carlo red model rate standard deviation Std[r M Carlo red model rate standard deviation Std[r M Carlo red model (a) Trunted model (18), (19) (b) Trunted model (18), () Fig 8 Comparison between Monte Carlo simulations and two trunted models for Example, with = 5ms and a step in the drop-rate p from % to 1% at time t = 5sec Since µ () (t) = 1, t we excluded µ () from the vector µ, resulting in an affine system (as opposed to linear) We consider a truntion whose state contains the first and second moments of the sending rate In this se, (9) n be written as follows: µ(1) µ () = p µ(1) µ () p 4 µ, (18) where µ := µ (3) evolves according to µ (3) = 3µ() 7pµ(4) 8 In this se, (17) has a unique solution ϕ, resulting in a trunted system given by (18) and µ = ϕ(µ (1),µ () ) := (µ() ) 5 (µ (1) ) (19) Figure 8(a) shows a comparison between Monte Carlo simulations and this trunted model The dynamics of the first moment of the sending rate is very accurately predicted by the trunted model and there is roughly a 1% error in the steady-state value of the standard deviation In the simulations, a step in the drop probability was introduced at time t = 5sec to show that the trunted model also ptures well transient behavior Figure 8(b) refers to a somewhat more accurate trunted model proposed in [1, which corresponds to ( µ µ = ϕ(µ (1),µ () ()) 3 ) := () µ (1) Its construction relied on the experimental observation that the steady-state distribution of the sending rate appears to be approximately Log-Normal It turns out that this model could also have been obtained using the procedure described in Sec IV by requiring (13) to hold only for N = but for a set Ω µ consisting of all Log-Normal distributions, which is richer than the set F det of deterministic distributions used to construct (19)

16 16 Example 3 (TCP on-off): For this system we consider moments of the sending rate r := w modes, and therefore we use the following test functions ψ () off (q,w) = 1 q = off w m ψ (m) q = (q,w) = m q {,} q {,off} for which yielding (Lψ () off )(q,w) = τ 1 off ψ() off (q,w)+ 1 ψ(1) (q,w)+ 1 ψ(1) (q,w) )(q,w) = τ 1 off wm mlog ( (q,w)+ (q,w) (Lψ (m) (Lψ (m) )(q,w) = p µ () off µ () µ () µ (1) µ (1) µ () µ () µ (3) µ (3) = mψ() off m ψ(m+1) (q,w)+ m ψ(m) ψ(m 1) (q,w) τ 1 off 1 1 τ 1 off 1 p p 1 τ 1 off w log 1 p 1 p 1 p τ 1 off w log 4 ψ (m) (q,w) = ) ψ (m+1) w m m p+ 1 (q, w) ( 1 +p p ) m 1 p p 4 1 3p 4 τ 1 off w3 3 log p p 8 1 7p 8 on the and q = q {,off}, ψ (m+1) (q,w), Since µ () off, µ(), and µ () are the probabilities that q is equal to off,, and, respectively, these quantities must add up to one We could therefore exclude one of them from µ We consider a truntion whose state contains the zeroth, first, and second moments of the sending rate In this se, (9) n be written as follows: µ () off τ off µ () τ 1 off 1 p µ () p 1 µ (1) τ = off w log 1 p µ (1) 1 p µ 1 p τ off w log4 µ () [ where µ := µ (3) µ (3) evolves according to µ (3) = τ 1 off w3 3µ() off + log8 µ(3) µ (3) = 3 µ() + p 8 µ(4) µ () off µ () µ () µ (1) µ (1) µ () µ () ( 1 ) +p ( 1 + 7p 8 µ () off µ () µ () µ (1) µ (1) µ () µ () µ (3) µ (3) µ (4) µ (4) + 1 p p 4 1 3p 4 µ (4), ) µ (4) µ, (1) (a) (b)

17 17 probability of drop p probability on each mode mean rate E[r 1 total rate standard deviation Std[r 5 total Fig 9 (left) Comparison between Monte Carlo simulations (solid lines) and the trunted model (1), (4) (dashed lines) for Example 3, with = 5ms, τ off = 1sec, = 39 pacets (corresponding to 358KB files broen into 15bytes pacets, which is consistent with the file-size distribution of the UNIX file system reported in [16), and a step in the drop-rate p from 1% to % at t = 1sec However, () does not satisfy condition 5 in Lemma 1 beuse the different discrete modes do not appear decoupled: µ (3) depends on µ (), and µ(3) depends on µ (4) For the purpose of determining ϕ, we ignore the cro coupling off terms and approximate () by µ (3) log8 µ(3) ( 1 ) +p µ (4), µ (3) 3 µ() ( 1 + 7p ) µ (4) (3) 8 The validity of these approximations generally depends on the networ parameters When (3) is used, it is straightforward to verify that (17) has a unique solution ϕ, resulting in a trunted system given by (1) and () µ = ϕ(µ) = )1 (µ () )5 (4) [ µ () (µ() )3 (µ (1) ) 3 (µ (µ (1) ) Since µ () off, µ(), and µ () correspond to the probabilities that q is equal to off,, and, respectively, these quantities must add up to one and we n exclude one of them from the state Figure 9 shows a comparison between Monte Carlo simulations and the trunted model (1), (4) The dynamics of the first and second order moments are accurately predicted by the trunted model There is some steady-state error in the probabilities of each mode, especially after the drop-rate p dropped to % This corresponds to a situation in which the sending rate in the mode exceeds that of the mode and therefore the µ (4) term dropped from (3) in the µ (3) equation may actually have a signifint effect Somewhat surprisingly, even in this se the first and second order moments are very well approximated In preparing this paper, several simulation were executed for different networ parameters and initial conditions Figure 9 shows typil best-se (before t = 1) and worst-se (after t = 1) results The average file size used to produce Fig 9 is consistent with the one observed in the UNIX file system reported in [16 It should be emphasized that the overall distribution of the UNIX file system and more importantly of files

18 18 transmitted over the Internet is not well approximated by an exponential distribution More reasonable distributions are Pareto or mixtures of exponentials Both could be modeled by pshss, but for simplicity here we focused our attention on the simplest se of a single exponential distribution Example 4 (Networed control system): For this system we consider test functions of the form ψ (m) (e) = e m, m N, for which (Lψ (m) )(e) = ame m + m(m 1)b e m e m+ρ = amψ (m) (e)+ m(m 1)b ψ (m ) (e) ψ (m+ρ) (e), yielding µ (1) µ () µ (3) µ (4) µ (5) = aµ (1) µ (1+ρ) b +aµ () µ (+ρ) 3b µ (1) +3aµ (3) µ (3+ρ) 6b µ () +4aµ (4) µ (4+ρ) 1b µ (3) +5aµ (5) µ (5+ρ) In particular, for ρ = we get µ (1) µ () µ (3) µ (4) µ (5) = b + a 1 a 1 3b 3a 1 6b 4a 1 1b 5a 1 µ (1) µ () µ (3) µ (4) µ (5) and for ρ = 1 we get µ (1) µ () µ (3) µ (4) µ (5) = b + a 1 a 1 3b 3a 1 6b 4a 1 1b 5a 1 µ (1) µ () µ (3) µ (4) µ (5) Since µ () (t) = 1, t we excluded µ () from the vector µ, resulting in an affine system (as opposed to linear) For ρ =, the infinite-dimensional dynamics have a lower-triangular structure and therefore an exact truntion is poible However, this se is le interesting beuse it corresponds to a reset-rate that does not depend on the continuous state and is therefore farther from the optimal [9, 1 We consider here ρ = 1 In this se, the odd and even moments are decoupled and n be studied independently It is straightforward to chec that if the

19 19 b E[e 5 M Carlo red model E[e 4 1 M Carlo red model E[e 6 M Carlo red model Fig 1 (right) Comparison between Monte Carlo simulations (solid lines) and the trunted model (5), (6) (dashed lines) for Example 4, with a = 1, q = 1, and a step in the parameter b from 1 to at time t = 5sec initial distribution of e is symmetric around zero, it will remain so for all times and therefore all odd moments are constant and equal to zero Regarding the even moments, the smallest truntion for which condition 4 in Lemma 1 holds is a third order one, for which (9) n be written as follows: µ () a 1 µ () b µ (4) = 6b 4a 1µ (4) + + (5) µ, µ (6) 15b 6a µ (6) 1 where µ := µ (8) evolves according to µ (8) = 8b µ (6) +8aµ (8) µ (1) It is straightforward to verify that (17) has a unique solution ϕ, resulting in a trunted system given by (5) and µ = ϕ(µ (),µ (4),µ (6) ) = µ ()( µ (6) µ (4) ) 3 (6) Figure 1 shows a comparison between Monte Carlo simulations and the trunted model (5), (6) The dynamics of the all the moments are accurately predicted by the trunted model The nonlinearity of the underlying model is apparent by the fact that halving b at time t = 5sec, which corresponds to dividing the variance of the noise by 4, only results in approximately dividing the variance of the estimation error by Example 5 (Deying reaction): For this system we consider test functions of the form ψ (m) (x) = x m, m N, for which (Lψ (m) )(x) = c x(x 1)( (x ) m x m) = c ( ψ (m+1) + m = ( ) ) m m+ +1 m +1 ( )m ψ (+1), (7)

20 yielding µ (1) µ () µ (3) µ (4) µ (5) = c For this example it is not poible to find a function ϕ for which (17) holds exactly for any finite truntion However, finding ϕ becomes poible if we ignore some of the lowest-order powers of x in the summation in (7) that defines the generator of the system This is reasonable when the number of particles is signifintly larger than one In the extreme se, for which we only leave the highest order term, a first-order truntion is poible and (9) n be written as follows: µ (1) = c µ, where µ := µ () evolves according to µ () = cµ (3) In this se, (17) has a unique solution ϕ, resulting in the following trunted system ν (1) = c(ν (1) ), which would also be obtained using the deterministic formulation of chemil inetics However, by leaving more terms one n obtain approximations that should be more accurate, especially for smaller number of particles For example, if one would eep the two highest powers of x in the summation in (7), a second-order truntion is poible and (9) n be written as follows: µ(1) = c c µ () 4c µ(1) µ () + µ, c where µ := µ (3) evolves according to µ (3) = 9cµ (3) 3cµ (4) In this se, (17) has a unique solution ϕ, resulting in the following trunted system ν(1) ν () = c c 4c ν(1) ν () + ( ν () ) 3 c ν (1) The relationship between the first three moments expreed by this ϕ is exact for Log-Normal distributions It turns out that if we had ept a larger number of higher-order powers of x in the summation in (7), we would have obtained higher-order truntion for which ϕ that would also be exactly compatible with Log-Normal distributions This observation has been further explored in [17 For concisene, we omit the comparison between Monte Carlo simulations and the trunted model for this very simple reaction Example 6 (Deying-dimerizing reaction set): For this system we consider test functions of the form ψ (m1,m,m) (x) = x m1 1 xm xm3 3, m 1,m,m 3 N, µ (1) µ () µ (3) µ (4) µ (5)

21 1 for which (Lψ (m 1,m,m ) ( )(x) = c 1x 1 (x1 1) m 1 x m ) 1 1 x m xm c x1(x1 1)( (x 1 ) m 1 (x +1) m x m 1 1 xm ) x m 3 3 ( +c 3x (x1 +) m 1 (x 1) m ) x m 1 1 xm x m 3 ( 3 +c 4x (x 1) m (x 3 +1) m 3 ) x m xm 3 3 x m c m 1,m i,j= (i,j) (m 1,m ) m 1,m +c 3 m 1 1 = c 1 i= i,j= (i,j) (m 1,m ) ( m 1 i )( 1) m 1 i ψ (i+1,m,m 3 ) (x) ( m 1 i ) ( m j ) ( ) m 1 i ( ψ (i+,j,m 3) (x) ψ (i+1,j,m 3) (x) ) ( m 1 i ) ( m j ) m 1 i ( 1) m j ψ (i,j+1,m 3) (x) m,m 3 +c 4 i,j= (i,j) (m,m 3 ) where the summations result from the power expansions of the terms (x i c) mi Therefore µ (1,,) µ (,1,) µ (,,1) µ (,,) µ (,,) µ (1,1,) µ (3,,) µ (,1,) = ( m i ) ( m 3 j ) ( 1) m i ψ (m 1,i+1,j) (x), (8) c 1 +c c 3 c c c c 3 c 4 c 4 c 1 c 4c 3 c 1 +4c 4c 3 c c c c 3 +c 4 c 3 c 4 c c c c 3 3c c c 3 c 1 +c c 3 c 4 c c 1 +4c 8c 3 3c 1 1c 1c 3 3c 1 +9c 6c 3 3c c 4c 3 4c 4c 3 c 1 c 4c 3 5c c c 1 +4c c 3 c 4 4c 3 c 3 µ (1,,) µ (,1,) µ (,,1) µ (,,) µ (,,) µ (1,1,) µ (3,,) µ (,1,) µ (1,,) µ (4,,) µ (3,1,) For this example we consider a truntion whose state contains all the first and second order moments for the number of particles of the first and second species To eep the formulas small, we omit from the truntion the second moments of the third species, which does not appear as a reactant in any reaction and therefore its higher order statistics do not affect the first two In this se, (9) n be written as follows: = + where µ := µ (1,,) µ (,1,) µ (,,1) µ (,,) µ (,,) µ (1,1,) c 1+c c 3 c c c c 3 c 4 c 4 c 1 c 4c 3 4c c 1 4c 3 c c c 3+c 4 c 3 c 4 c c c 3 3c c 3 c c 1 c 3 c 4 [ µ (3,,) µ (,1,) evolves according to µ (1,,) µ (,1,) µ (,,1) µ (,,) µ (,,) µ (1,1,) µ (3,,) = ( c 1 +4c )µ (1,,) +8c 3 µ (,1,) +(3c 1 1c )µ (,,) +1c 3 µ (1,1,) c c c c µ, (9) +( 3c 1 +9c )µ (3,,) +6c 3 µ (,1,) 3c µ (4,,) (3a) µ (,1,) = c µ (1,,) 4c 3 µ (,1,) +4c µ (,,) +4c 3 µ (,,) +(c 1 c 4c 3 )µ (1,1,) 5c µ (3,,) +(4c c 1 c 3 c 4 )µ (,1,) +4c 3 µ (1,,) + c µ (4,,) c µ (3,1,) This system does not satisfy condition 4 in Lemma 1 beuse the µ (1,,), µ (,1,) terms in the right-hand sides of (3) lead to monomials in x 1 and x in i=1 a l,ix (mi) that do not exist in any of the polynomials (3b)

22 TABLE I COMPARISON BETWEEN ESTIMATES OBTAINED FROM MONTE CARLO SIMULATIONS AND THE TRUNCATED MODEL FOR EXAMPLE 6 THE MONTE CARLO SIMULATION DATA WAS TAKEN FROM [13 Source for the estimates E[x 1 () E[x () StdDev[x 1 () StdDev[x () 1, MC simul model (9), (3) { i=1 a j,ix (m l m j+m i) : 1 j 6 } These terms n be easily traced bac to the lowest-order terms in power expansions in (8) and disappear if we only eep the three highest powers of x 1 in the expansion of (x 1 1) m1 that corresponds to the first reaction and the two highest powers of x 1 and x in the expansions of (x 1 ± ) m1 and (x ±1) m that correspond to the second and third reactions In practice, this leads to the following simplified version of (9) where now µ := µ (1,,) µ (,1,) µ (,,1) µ (,,) µ (,,) µ (1,1,) = c 1+c c 3 c c c c 3 c 4 c 4 c 1 4c c 1 4c 3 c 4 c 3 c 4 c c c 3 3c c 3 c c 1 c 3 c 4 [ µ (3,,) µ (,1,) evolves according to µ (1,,) µ (,1,) µ (,,1) µ (,,) µ (,,) µ (1,1,) µ (3,,) = 3c 1 µ (,,) +( 3c 1 +3c )µ (3,,) +6c 3 µ (,1,) 3c µ (4,,) µ (,1,) = c µ (,,) +(c 1 4c 3 )µ (1,1,) 5c µ (3,,) + c c c c +( c 1 +c c 3 c 4 )µ (,1,) +4c 3 µ (1,,) + c µ (4,,) c µ (3,1,), µ, (31) for which condition 4 in Lemma 1 does hold, allowing us to find a unique solution ϕ to (17), resulting in a trunted system given by (9) and µ = ϕ(µ) = [ (µ (,,) µ (1,,) ) 3 µ (,,) µ (,1,) ( µ (1,1,) µ (1,,) ) (3) Ignoring the lowest-order powers of x 1 and x in the power expansions is valid when the populations of these species are high In practice, the approximation still seems to yield good results even when the populations are fairly small as shown by the Monte Carlo simulations Figure 11 shows a comparison between Monte Carlo simulations and the trunted model (9), (3) The coefficients used were taen from [13, Example 1: c 1 = 1, c = 1, c 3 = 1, c 4 = 1 1 In Fig 11(a), we used the same initial conditions as in [13, Example 1: x 1 () = 4, x () = 798, x 3 () = (33) The match is very accurate, as n be confirmed from Table I The values of the parameters chosen result in a pshs with two distinct time sles, which maes this pshs computationally difficult to simulate ( stiff in the terminology of [13) The initial conditions (33) start in the slow manifold x = 5 1 x 1(x 1 1) and Fig 11(a)

23 3 eentially shows the evolution of the system on this manifold Figure 11(b) zooms in on the interval [,5 1 4 and shows the evolution of the system towards this manifold when it starts away from it at x 1 () = 8, x () = 1, x 3 () = (34) Figures 11(c) 11(d) shows another simulation of the same reactions but for much smaller initial populations: x 1 () = 1, x () = 1, x 3 () = 5 (35) The trunted model still provides an extremely good approximation, with signifint error only in the covariance between x 1 and x when the averages and standard deviation of these variables get below one This happens in spite of having used (31) instead of (3) to compute ϕ 1 5 E[x E[x 1 population means E[x STD[x 1 STD[x population standard deviations Corr[x 1,x populations correlation coefficient population means 8 6 E[x 1 4 E[x E[x x 1 4 population standard deviations STD[x STD[x x 1 4 populations correlation coefficient 9 95 Corr[x 1,x x 1 4 (a) Large population, over a long time sle with initial condition (b) Large population, over a short time sle with initial condition (33) (34) 5 15 E[x 1 population means 1 E[x 3 5 E[x STD[x 1 STD[x population standard deviations Corr[x 1,x populations correlation coefficient population means 3 E[x 1 1 E[x E[x x 1 3 population standard deviations 4 STD[x 1 3 STD[x x 1 3 populations correlation coefficient 9 95 Corr[x 1,x x 1 3 (c) Small population, over a long time sle with initial condition (d) Small population, over short time sle with initial condition (35) (35) Fig 11 Comparison between Monte Carlo simulations (solid lines) and the trunted model (9), (3) (dashed lines) for Example 6

24 4 VII CONCLUSIONS AND FUTURE WORK We showed that the infinite-dimensional linear moment dynamics of a pshs n be approximated by a finitedimensional nonlinear ODE with arbitrary precision Moreover, we provided a procedure to build this type of approximation The methodology was illustrated using a varied pool of examples, demonstrating its wide applibility Several observations arise from these examples, which point to directions for future research: 1) The algorithm proposed to compute the truntion function does not directly tae into account the steady-state distribution of the pshs This is desirable beuse for all the examples presented it is difficult to compute the steady-state values of the moments However, when steady-state distributions are available, they n improve the steady-state accuracy of the trunted system This was done in Example, for which experimental observations indite that the steady-state distribution of the sending rate appears to be approximately Log- Normal However, it would be desirable to devise constructions that improve the steady-state accuracy even when this type of insight is not readily available ) In all the examples presented, we restricted our attention to truntion functions ϕ of the form (16) and we only used deterministic distributions to compute ϕ Mostly liely, better results could be obtained by considering more general distributions, which may require more general forms for ϕ 3) The truntion of pshss that model chemil reactions proved especially accurate This motivates the search for systematic procedures to automatilly construct a trunted system from chemil equations such as (6) This iue has been pursued in [18 Another direction for future research consists of comparing the trunted models obtains here with those in [19 An additional direction for future research consists of establishing computable bounds on the error between solutions to the infinite-dimensional moments dynamics and to its finite-dimensional truntions APPENDIX Proof of Theorem For a given δ >, let T > and N > be sufficiently large so that for every solution µ and ν to (8) and (1) starting at some t in Ω µ and Ω ν, respectively, we have 4 β( µ(t 1 ) ν(t 1 ),τ) δ, τ T, t 1 t (36) (T) N+ ( sup d N+ µ(τ) (N +)! dt N+ + sup d N+ ) ν(τ) dt N+ δ (37) τ t The existence of T is guaranteed by the fact that β KL and by the boundedne of ν and µ, as per Aumption 1 For the chosen T, the existence of N is guaranteed by Aumption 1 and the fact that lim N (T) N+ (N+)! = The justifition for these definitions will become clear shortly Let µ and ν be solutions to (8) and (1) starting in Ω µ and Ω ν, respectively, at some time t For a given t 1 t, since Ω ν is forward invariant, ν 1 := ν(t 1 ) Ω ν and therefore, beuse of Aumption, there exists some τ t 4 The left-hand-sides of (36) and (37) need to add up to δ but other options are poible