A Model for Stochastic Hybrid Systems with Application to Communication Networks

Transcription

1 A Mode for Stochastic Hybrid Systems with Appication to Communication Networks João P. Hespanha Dept. of Eectrica and Computer Eng., Univ. of Caifornia, Santa Barbara, CA Abstract. We propose a mode for Stochastic Hybrid Systems SHSs) where transitions between discrete modes are triggered by stochastic events much ike transitions between states of a continuoustime Markov chains. However, the rate at which transitions occur is aowed to depend both on the continuous and the discrete states of the SHS. Based on resuts avaiabe for Piecewise-Deterministic Markov Process PDPs), we provide a formua for the extended generator of the SHS, which can be used to compute expectations and the overa distribution of the state. As an appication, we construct a stochastic mode for on-off TCP fows that considers both the congestion-avoidance and sow-start modes and takes directy into account the distribution of the number of bytes transmitted. Using the toos derived for SHSs, we mode the dynamics of the moments of the sending rate by an infinite system of ODEs, which can be truncated to obtain an approximate finite-dimensiona mode. This mode shows that, for transfer-size distributions reported in the iterature, the standard deviation of the sending rate is much arger than its average. Moreover, the ater seems to vary itte with the probabiity of packet drop. This has significant impications for the design of congestion contro mechanisms. keywords: Hybrid systems; Stochastic processes; Congestion contro; TCP 1 Introduction The idea of integrating ogic and continuous dynamics in the contro of compex systems is not new but ony in the ast 15 years has the theoretica anaysis of hybrid systems i.e., systems with both continuous dynamics and discrete ogic) been pursued in a systematic fashion. A fair amount of research has been done on the modeing of hybrid systems [1, 3, 1, 12, 29, 3, 36, 4, 44, 46], but most of this work concentrated on deterministic systems. Instances of stochastic hybrid systems can be found in the work on stochastic impuse contro. For exampe Bensoussan and Lion [6], considered the probem of optimay choosing impuse times and intensities for a given stochastic differentia equation. In more recent hybrid systems terminoogy, this woud amount to optimay seecting reset times and the vaues to which the state is reset. One specific exampe is the cassic optima stopping time probem, where one wants to determine an optima time instant at which the evoution of a differentia equation is to be frozen. Interestingy enough, Bensoussan and Lion [6] aso considered the case of integer-vaued states. One of the more genera forma modes for stochastic hybrid systems was proposed by Hu et a. [2], where the deterministic differentia equations for the continuous fows are repaced by their stochastic counterparts, and the reset maps are generaized to state-dependent) distributions that define the probabiity density of the state after a discrete transition. However, in this mode transitions are aways triggered by deterministic conditions guards) on the state. This materia is based upon research supported by the Nationa Science Foundation under Grants No. CCR , ANI Version dated March 27, 25. To appear in Noninear Anaysis Specia Issue on Hybrid Systems.

2 In the mode proposed in this paper for SHSs, transitions between modes are triggered by stochastic events, much ike the transitions between states of a continuous-time Markov chains. However, the probabiity that a transition occurs in a given interva of time depends both on the continuous and discrete components of the current SHS s state. The state of the SHS so defined is a Piecewise-Deterministic Markov Process PDP) in the sense of [14]. Based on this observation, we provide a formua for the extended generator of the SHS, which can be used to compute expectations and probabiity distributions/densities of its discrete/continuous states. The basic mode proposed is i) imited to a deterministic evoution of the continuous state inside each discrete mode and ii) it does not aow for transitions triggered by deterministic conditions on the state e.g., guards being crossed). The first restriction is ony introduced for simpicity of presentation and it is shown how the mode can be generaized to aow for continuous evoutions driven by stochastic differentia equations. The rationae for disaowing transitions triggered by guard-crossings is that this aows us to excude Zeno phenomena [24, 46]. However, we show how one can approximate systems with guards and potentiay prone to Zeno phenomena) by a sequence of Zeno-free SHSs. The mode proposed is inspired by piecewise deterministic jump systems PDJSs), where the evoution of the continuous state in each mode is modeed by a deterministic differentia equation and transitions between modes are governed by a continuous-time Markov process [5, 13, 15, 16, 45]. In genera, the transitions rates in PDJSs are assumed independent of the continuous state, which is too restrictive for our appications. The work of Fiar et a. [16] is a notabe exception but requires a time-scae separation between the purey deterministic) continuous dynamics and the discrete jump dynamics. In switched diffusion processes SDPs), as defined by Ghosh et a. [17], the evoution of the continuous state in each mode is modeed by a stochastic differentia equation and transitions between modes are controed by a continuous-time Markov process. The transition rates of the Markov process can depend on the state but transitions do not generate jumps on the continuous state i.e., no resets). The reader is referred to [39] for a comparison of the modes in [14, 17, 2]. The SHSs considered here can be viewed as specia cases of genera jump-diffusion processes [23]. In fact, Theorem 1 can be viewed as an Itô rue for SHSs. However, in the appication of interest, we are faced with the technica difficuty that jump-intensities are not bounded and moreover the jump-distributions do not have compact support. Our SHS mode was inspired by the need to obtain accurate modes for TCP congestion contro in communication networks. The use of hybrid modes to characterize the behavior of congestion contro was proposed by Hespanha et a. [19] and further pursued in [8, 9]. In these modes, packet drops trigger transitions between different modes for the evoution of TCP s congestion window size. The drop modes in [8, 19] were competey deterministic but, as noted in [9], their use is imited to simpe network topoogies. There is an extensive iterature on modes that describe the behavior of TCP congestion contro for ongived fows, i.e., fows that have an infinite amount of data to transmit. A great dea of effort has been paced in characterizing the steady-state behavior of these fows [32, 33, 37, 38, 42]. In particuar, in studying the reationship between the average transmission rate µ, the average round-trip time, and the per-packet drop rate p drop for a singe TCP fow. In most of this work, µ and p drop shoud be understood as time-averages for a singe TCP fow. This type of approach was aso pursued in [25 28, 38, 41] to derive dynamic modes for the congestion avoidance stage of ong-ived TCP fows. However, these singe-fow modes are ony vaid over time scaes much onger than the round-trip time for one packet. To avoid averaging over ong time intervas, Misra et a. [34, 35] utiized ensembe averages to construct modes for the dynamics of ong-ived fows. Shakkottai and Srikant [41] aso used stochastic aggregation to reduce the time-scaes over which a singe-fow mode is vaid. They showed that when n singe-fow modes are aggregated, the average mode becomes vaid over time scaes n times smaer than those of the origina singe-fow modes. Interestingy, aside from the different interpretation of the quantities invoved, the ensembe-average modes for ong-ived fows do not differ significanty from the time-average ones. We pursue here a stochastic anaysis of the hybrid modes proposed in [9]. As in [34, 35] time averaging is done over intervas of roughy one round-trip time to obtain continuousy varying sending rates, and we then 2

3 investigate the dynamics of ensembe averages. However, here we consider ensembes of on-off TCP fows. The off-periods are assumed exponentiay distributed whereas the on-periods are determined by the amount of data being transfered. We take as given the probabiity distribution of the transfer-sizes, which impicity determines the distribution for the on-periods. The mode takes into account the correation between the variabe) sending rate and the duration of the aso variabe) on-periods. Not surprisingy, the ensembe behavior of on-off TCP fows varies very significanty with the probabiity distribution of the transfer sizes. By feeding our modes with reaistic distributions reported in the iterature, we concude that the dynamics of the sending rate is dominated by high-order statistics, exhibiting much arger standard deviations than the average vaue. Moreover, the packet drop rate seems to have a surprisingy sma effect on the average drop rate but provides a strong contro on its standard deviation. This has significant impications for the design of congestion contro mechanisms. To reach a wide audience we paced a great dea of effort in keeping technicaities to a minimum and reegated the most technica arguments to Sect. 3.3, which can be skipped without oss of continuity. The remainder of this paper is organized as foows: the basic stochastic hybrid mode is introduced in Sect. 2 and severa possibe generaizations are discussed. Sect. 3 discusses the basic theoretica toos to anayze SHSs. In Sect. 4, we utiize SHSs to mode TCP congestion contro and iustrate how the toos derived in Sect. 3 can be used to anayze the resuting system. Sect. 5 contains some concusions and directions for future research. The proofs of a few technica resuts are incuded in Appendix. A subset of the resuts in this paper were presented at the 24 Workshop on Hybrid Systems : Computation and Contro. Notation. By a piecewise continuous signa it is meant a function x : [, ) R n that is right-continuous and has eft-imit at every point. We denote by x t) the eft-imit of xτ) as τ t. A signa x is caed piecewise constant if it is piece-continuous and it is constant on every interva where it is continuous. Given a measurabe space Ω, F) and probabiity measure P : F [, 1], vector-vaued random variabes α : Ω R n and stochastic processes x : Ω [, ) X R n are denoted in bodface. A stochastic process with piecewise constant sampe paths is caed a jump process. When a jump process takes vaues on the set N of nonnegative integers it is caed a stochastic counter. 2 Stochastic Hybrid Systems A stochastic hybrid system SHS) is defined by a differentia equation ẋ =fq, x, t), f : Q R n [, ) R n, 1) a famiy of m discrete transition/reset maps q, x) =φ q, x, t), φ : Q R n [, ) Q R n, {1,..., m}, 2) and a famiy of m transition intensities λ q, x, t), λ : Q R n [, ) [, ), {1,..., m}, 3) where Q denotes a typicay finite) set with no particuar topoogica structure. A SHS characterizes a jump process q : Ω [, ) Q caed the discrete state; a stochastic process x : Ω [, ) R n with piecewise continuous sampe paths caed the continuous state; and m stochastic counters N : Ω [, ) N caed the transition counters. In essence, between transition counter increments the discrete state remains constant whereas the continuous state fows according to 1); and at transition times the continuous and discrete states are reset according to 2). Each transition counter N counts the number of times that the corresponding discrete transition/reset map φ is activated. The frequency at which this occurs is determined by the transition 3

4 PSfrag repacements intensities 3). In particuar, the probabiity that the counter N wi increment and therefore that the corresponding transition takes pace in an eementary interva t, t + dt] is given by λ qt), xt), t)dt. In practice, one can think of the intensity of a transition as the instantaneous rate at which that transition occurs. We wi shorty make these statements mathematicay precise. It is often convenient to represent SHSs by a directed graph as in Figure 1, where each vertex corresponds to a discrete mode and each edge to a transition between discrete modes. The vertices are abeed with the corresponding discrete mode and the vector fieds that determines the evoution of the continuous state in that particuar mode. The source of each edge is abeed with the probabiity that the transition wi take pace in an eementary interva t, t + dt] and the destination is abeed with the corresponding reset-map. λq 3, x, t)dt q 3, x) φq 3, x, t) q = q 3 ẋ = fq 3, x, t) q = q 1 ẋ = fq 1, x, t) q = q 2 ẋ = fq 2, x, t) λq 1, x, t)dt q 1, x) φq 1, x, t) Fig. 1. Graphica representation of a stochastic hybrid system 2.1 Construction of the stochastic processes We now provide the forma procedure to construct the sampe paths of the discrete state, continuous state, and transition counters and show that it is indeed consistent with the intuition given above. Aside from its theoretica interest, this construction is usefu to run Monte Caro simuations of SHSs. It is inspired by the one used in [14, Chapter 2] to define Piecewise-Deterministic Markov Processes PDPs). The foowing reguarity assumption on the vector fied f is required for the SHS to be we defined: Assumption 1. For every q, x, t ) Q R n [, ) there exists a unique goba soution ϕ ; t, q, x ) : [t, ) R n to 1) with initia condition xt ) = x and qt ) = q. In what foows, the µ k, {1,..., m}, k N denote independent random variabes a uniformy distributed in the interva [, 1]. We wi ca these transition triggers. Consider an initia condition q, x, t ) Q R n [, ). For a given ω Ω, the sampe paths of qω, ) : [t, ) Q, xω, ) : [t, ) R n, and a the N ω, ) : [t, ) N can be constructed as foows: 1. Set t ω) = t, qω, ) = q, xω, ) = x, N ω, ) =,. 2. Let t 1 ω) be the argest time on t ω), ] for which e R t t ω) λ qω,t ω)),ϕs;t ω),qω,t ω)),xω,t ω))),s)ds > µ ω), 4) t [t ω), t 1 ω)), {1,..., m}. 3. On the interva [t ω), t 1 ω)), the sampe paths of qω, ) and a the counters N ω, ) remain constant, whereas the sampe path of xω, ) equas ϕ ; t ω), qω, t ω)), xω, t ω))). 4. Denoting by 1 ω) {1, 2,..., m} the index for which 4) is vioated 1 at time t = t 1 ω), the counter N 1ω)ω) is incremented by one and qω, t1 ω)), xω, t 1 ω)) ) = φ 1 ω) q ω, t 1 ω)), x ω, t 1 ), t 1 ω) ). 1 In principe 4) coud be vioated simutaneousy for more than one {1,..., m}, but since this event has zero probabiity we can ignore it. 4

5 5. In case t 1 ω) <, repeat the construction from the step 2 above with t ω), µ ω), t 1ω), 1 ω) repaced by t k ω), µ k ω), t k+1ω), k+1 ω), respectivey, for k = 1, 2,... The random variabes t k defined by 4) are caed transition times. The stochastic processes so defined depend on the initia condition for the SHS. To emphasize this dependence we sometimes use P z and E z to denote the probabiity measure and expected vaue corresponding to the initia condition z := q, x, t ) Q R n [, ). Step 1 provides the initiaization for a the stochastic processes. Step 3 guarantees that the discrete state remains constant and the continuous state fows according to 1) between transitions. Step 4 enforces that the continuous and discrete states are reset according to 2) at transition times. The frequency at which these occur is determined by Step 2. In fact, one can derive from 4) cf. Appendix) that P N t + dt) > N t) ) [ 1 im = im E dt dt dt dt t+dt t ] λ qt), ϕs; t, qt), xt)), s)ds, 5) which shows that the probabiity P N t + dt) > N t) ) that the transition φ wi occur in an arbitrariy) sma interva t, t + dt] is proportiona to the ength of the interva with the proportionaity constant given by the right-hand-side of 5). This equation specifies the precise meaning of the observation made above to the extent that the probabiity that the counter N wi increment in an eementary interva t, t + dt] is given by λ qt), xt), t)dt. Note that when λ is continuous, the right-hand-side of 5) is precisey equa to E[λ qt), xt), t)]. The above construction guarantees that the sampe-paths are indeed right-continuous and have eft-imits at every point with probabiity one. However, without further assumptions there is no guarantee that the sampe path are defined gobay on [, ). We wi return to this issue ater. 2.2 Generaizations The mode for stochastic hybrid systems presented above is more genera than it may appear at first. We discuss next some of the generaizations possibe. We wi return to these in Sect. 3.1 to study their impication on the resuts in Sect. 3. The mode aows for transitions where the next state is chosen according to a given distribution. For exampe, suppose one woud ike the intensity λq, x, t) to trigger transitions to the discrete-states q 1 or q 2 with probabiities p 1 or 1 p 1, respectivey. This coud be achieved by considering two transitions with intensities p 1 λq, x, t) and 1 p 1 )λq, x, t), respectivey, and reset maps φ 1 q, x, t) = q 1, ϕ 1 x, t), t) and φ 2 q, x, t) = q 2, ϕ 2 x, t), t), respectivey, where the ϕ i denote possiby distinct) continuous-state resets. The above mode does not directy consider differentia equations driven by stochastic processes. However, many important casses of stochastic processes can be obtained as the imit of jump processes that can be modeed by SHSs. For exampe, the stochastic differentia equation ẋ = ax + bẇ 6) where w denotes Brownian motion, can be regarded as the imit as ɛ of the jump system with continuous dynamics ẋ = ax and resets x x + b ɛ and x x b ɛ both triggered with fixed intensity 1 2ɛ. We can therefore mode continuous evoutions of the form 6) as imits to a sequences of SHSs. The basic mode aso does not directy consider discrete transitions triggered by deterministic conditions of the state, e.g., a guard being crossed. However, this behavior can aso be obtained as the imiting soution to a sequence of SHSs. Consider for exampe the we known bouncing-ba singe-mode deterministic hybrid system with dynamics ẍ = g, g > and state reset x, ẋ), cẋ), c, 1) triggered by the condition 5

6 ẋ < and x. We coud approximate this system by a sequence of SHSs for which the resets are triggered with intensities given by a barrier function of the form { ɛ e x/ɛ ẋ < λ ɛ x, ẋ) :=, ɛ >. 7) ẋ >, As ɛ, transitions wi occur in a sma neighborhood of x = with increasingy higher probabiity. Figure 2 shows confidence intervas for the sampe-paths of three SHSs that approximate with increased accuracy the deterministic bouncing-ba. It is important to emphasize that, for any ɛ >, the sampe paths of the SHSs are gobay defined with probabiity one. This approach may in fact be a promising technique to overcome difficuties posed by the Zeno phenomena that occur for the deterministic bouncing-ba system [24, 46] Fig % confidence intervas for the x sampe paths of three SHS that approximate the bouncing-ba deterministic system with Zeno time equa to 1. The transition intensities of the SHSs are given by 7) with ɛ = 1 2, 1 3, 1 4 from eft to right. These resuts were obtained via Monte Caro simuations. 3 Generator for SHSs In this section we provide a resut to compute expectations on the state of a SHS. The foowing assumptions are needed: Assumption 2. i) The transition intensities λ : Q R n [, ) [, ), {1,..., m} are measurabe functions e.g., continuous). ii) For every initia condition z := q, x, t ) Q R n there exists a continuous functions α z : [, ) [, ) such that the sampe-paths are defined gobay and xt) α z t), t t with probabiity one with respect to P z. Assumption 2ii) may be difficut to check, but we wi shorty provide conditions that are more friendy to verify. We are now ready to state the main resut of this section: Theorem 1. Suppose that Assumptions 1 and 2 hod. For every initia condition z := q, x, t ) Q R n [, ) and every function ψ : Q R n [, ) R that is continuousy differentiabe with respect to its second and third arguments, we have that where q, x, t) Q R n [, ) Lψ)q, x, t) := [ t ] E z [ψqt), xt), t)] = ψq, x, t ) + E z Lψ)qs), xs), s)ds, 8) t ψq, x, t) ψq, x, t) fq, x, t) + + t 6 =1 ψ φ q, x, t), t ) ) ψq, x, t) λ q, x, t), 9)

7 and ψq,x,t) and ψq,x,t) t denote the gradient of ψq, x, t) with respect to x and the partia derivative of ψq, x, t) with respect to t, respectivey. Foowing [14], we ca the operator ψ Lψ defined by 9) the extended generator of the SHS. It is often convenient to write 8) in the foowing differentia form 2 d E[ψqt), xt), t)] dt = E[Lψ)qt), xt), t)]. 1) We eave the proof of Theorem 1 to Sect. 3.3 and proceed to contrast it with Itô cacuus [22]. Consider the foowing jump-diffusion process dx = fx, t)dt + φ x, t) x)dn, 11) =1 where the N are Poisson counters independent of each other and of the process state x. Each counter N has constant intensity λ and induces a state jump x φ x, t) at increment-times. The Itô rue for 11) can be written as dψx, t) = ψx, t) fx, t)dt + ψx, t) dt + t ψφ x, t), t) ψx, t))dn. 12) where ψ denotes a scaar-vaued smooth function of x and t cf., e.g., [11]). The equation for the expectation of ψx, t) can be derived from 12) by repacing dn with λ dt, dividing the resut by dt and take expectations, which eads to d E[ψx)] dt [ ψx, t) = E fx, t) + ψx, t) t Theorem 1, thus suggests the foowing Itô rue for SHSs: dψq, x, t) = ψq, x, t) fq, x, t)dt + ψq, x, t) dt + t + =1 =1 ψφ x, t), t) ψx, t))λ ]. =1 ψ φ q, x, t), t ) ) ψq, x, t) dn q, x), from which one woud derive 1) using a procedure simiar to the one described above for jump-diffusion processes. It is interesting to note that the existence of a discrete component q in the state does not significanty change the Itô rue. Assumption 2ii) rues out finite escape time amost surey. Athough this is a mid requirement, it may be difficut to verify directy. The foowing emma proved in the Appendix) provides a condition that is more restrictive but easiy checkabe: Lemma 1. Let φ x : Q R n [, ) R n, {1,..., m} denote the projection of φ into R n [i.e., φ x q, x, t) = x where q, x) = φ q, x, t)]. Assumptions 2ii) hods, if there exists a continuous function γ f : [, ) [, ) and constants c f, c φ such that fq, x, t) max{γ f t) x, c f }, φ x q, x, t) max{ x, c φ }, q Q, x R n, t, {1,..., m}. This emma essentiay requires f and the φ to have inear growth in x over R n. Moreover, the growth constant of the φ must not be arger than one. This is a strong requirement and one might be tempted to think that it coud be repaced by a oca condition if one woud restrict one s attention to a finite time interva [, T ], T <. It turns out that in genera this is not true. Consider for exampe a SHS with a 2 Reca that a signas are right-continuous with probabiity one and derivatives shoud be understood as right-imits. 7

8 singe discrete mode and scaar state x R that evoves according to ẋ =, between transitions, and that undergoes a nonocay Lipschitz reset x x p, p > 1 at transition times. For simpicity, we consider a constant intensity µ for the transition. Given two time instants t τ >, it is straightforward to concude that xt) = x pn t) and therefore E x [ xt) ] = n= x pn µn t n It turns out that when x > 1 the above series does not converge for any t > and a soution to the SHS is not defined on any interva of positive ength. Probems may sti arise when the reset maps have inear growth but with a growth constant arger than one. For exampe, considering instead the gobay Lipschitz reset map x px, p > 1 and transition intensity λx, t) = x, it is possibe to show that E x [ xt) ] woud expode in finite time. n! e µt. 3.1 Generaizations We now return to the generaizations of the basic SHS mode mentioned in Sect. 2.2 and discuss their impact on Theorem 1. Suppose that we woud ike every intensity λ to triggers a transition for which the next state is chosen according to a given distribution ν over Q R n. Conceptuay this can be viewed as expanding each origina intensity λ into a famiy of intensities, each one reseting to one particuar next-state in Q R n, and with intensity proportiona to the vaue of the distribution ν. In this case, the extended generator of the SHS becomes ψq, x, t) ψq, x, t) Lψ)q, x, t) := fq, x, t) + t ) + ψ q, x, t) ψq, x, t) λ q, x, t)ν q, x, t, d q, d x). Q R n =1 Needess to say that appropriate assumptions on the reset distributions ν are required to make sure that Assumption 2 hods. Suppose now that one woud ike to drive the differentia equation on each discrete mode by Gaussian white noise. In particuar, suppose that one woud ike to repace 1) by ẋ = fq, x, t) + k g i q, x, t)ẇ i, 13) where each w i denotes an independent Brownian motion process. The stochastic differentia equation 13) can be regarded as the imit as ɛ of the jump system with continuous dynamics ẋ = fq, x, t) and resets q, x) q, x + ɛ g i q, x, t) ), q, x) q, x ɛ g i q, x, t) ) both triggered with fixed intensities ɛ 2 cf., e.g., [11]). As ɛ, the extended generator of the SHS converges to i=1 Lψ)q, x, t) := ψq, x, t) ψq, x, t) fq, x, t) + + t =1 ψ φ q, x, t), t ) ) ψq, x, t) λ q, x, t) k i=1 g i q, x, t) 2 ψq, x, t) 2 g i q, x, t), 14) denotes the Hessian matrix of ψ with respect to x cf. Appendix). Athough, for each ɛ >, the assumptions of Lemma 1 are not satisfied for the reset maps above, the ess strict assumptions of Lemma 2 are satisfied at east for we-behaved ψ, e.g., bounded) and therefore the concusions of Theorem 1 sti hod cf. Sect. 3.3 beow). where 2 ψq,x) 2 8

9 3.2 Probabiity density Theorem 1 aso aows one to compute the probabiity density function of the SHS s continuous state assuming that one exists). The procedure is standard and is ony incuded here for competeness. Assume that there exists a joint probabiity density function ρx, q; t) such that, given any function ψ : Q R n [, ) R for which Theorem 1 hods, we have E[ψqt), xt), t)] = ψq, x, t)ρq, x; t)dx. q Q R n Then for a continuousy differentiabe function ψ with bounded support, we concude from Theorem 1 that q Q R n ρq, x; t) ψq, x, t) t ψq, x, t) fq, x, t)ρq, x; t) =1 ψφ q, x, t), t) ψq, x, t) ) ) λ q, x, t)ρq, x; t) dx =. Assuming that fq, x, t)ρq, x; t) is differentiabe with respect to x on the support of ψq,, t), by integration by parts we obtain ρq, x; t) ψq, x, t) R t n q Q + fq, x, t)ρq, x; t) ) + =1 = q Q ) λ q, x, t)ρq, x; t) dx R n =1 ψφ q, x, t), t)λ q, x, t)ρq, x; t)dx. In case x φ x q, x, t) is invertibe on Rn with inverse φ x : Q R n [, ) R n,i.e., φ x q, φ x q, x, t), t) = x, q, x, t, we can make the change of integration variabe z = φ x q, x, t) on the right-hand-side and obtain ρq, x; t) ψq, x, t) q Q R t n = ψφ q q Q R n =1 + fq, x, t)ρq, x; t) ) q, φ x + =1 ) λ q, x, t)ρq, x; t) dx q, z, t), t), z)λ q, φ x q, z, t), t)ρq, φ x q, z, t); t) φ x q, z, t) dz. z Choosing ψq, x, t) to be zero for every q other than q and a deta-distribution around x, we obtain ρq, x ; t) t = fq, x, t)ρq, x ; t) ) + + =1 q Q [x,t] λ q, x, t)ρq, x ; t)+ λ q, φ x q, x, t), t)ρq, φ x q, x ); t) φ x q, x, t) ), z where Q [x, t] denotes the set of vaues q Q for which φ q q, φ x q, x, t)) = q. When the previous system of partia differentia equations one for each q ) has a unique soution, it defines the probabiity density of the SHS s state. A simiar derivation coud have been done to obtain a Fokker-Panck-ike equation for a SHS with a differentia equation on each discrete mode driven by Brownian motion as in 13). In this case, using 14) instead of 9) we woud eventuay obtain 9

10 ρq, x ; t) = fq, x, t)ρq, x ; t) ) + 1 k t 2 i=1 + λ q, x, t)ρq, x ; t) + =1 where g j i denotes the jth entry of the vector g i. q Q [x,t] 2 g j i q, x, t)gi q, x, t)ρq, x; t) ) j j λ q, φ x q, x ), t)ρq, φ x q, x ); t) φ x q, x ) ), z 3.3 Proving Theorem 1 Let q, x, N 1, N 2,..., N m be the stochastic processes characterized by a SHS using the construction in Sect. 2.1 with initia condition q, x, t ) Q R n. The process w := {t, q, x, N 1, N 2,..., N m } is then a Piecewise-Deterministic Markov Processes PDPs) as defined in [14, Chapter 2] with initia condition w) = {t, q, x,,..., }. This stems directy from the fact that the construction in Sect. 2.1 mimics the one in [14, Chapter 2] to define PDPs. By coecting resuts from [14] and adapting them to our setting the foowing is straightforward to prove. Theorem 2. Suppose that Assumptions 1 and 2i) hod and that, for every z := q, x, t ) Q R n, [ E z N t) ] <, t t. 15) Then 8) hods for every z := q, x, t ) Q R n and every function ψ : Q R n [, ) R that is continuousy differentiabe with respect to its second argument and [ ] E z ψqt k ), xt k ), t k ) ψqt k ), xt k ), t k) <, t N. 16) t k t Proof of Theorem 2. Assumptions 1, 2i), and 15) guarantee that the standard conditions required by [14, Theorem 26.14] hod for the PDP defined by w := {t, q, x, N 1,..., N m } 3. Moreover, 16) guarantees that ψt, q, x, n 1,..., n m ) := ψq, x, t) beongs to the domain of the extended generator of w and aso that t C ψ t) := ψqt), xt), t) ψq, x, t ) Lψ)qs), xs))ds t is actuay a Martingae rather than just a oca Martingae cf., [14, Remark 26.14]). Therefore E z [C ψ t)] = C ψ t ) =, from which 8) foows. In view of Theorem 2, to prove Theorem 1 it suffices to show that Assumption 2ii) impies that both 15) and 16) hod. Actuay, as shown in the foowing Lemma, something sighty weaker than Assumption 2ii) suffices. Lemma 2. For a given z := q, x, t ) Q R n and function ψ : Q R n R, assume that there exist continuous functions α, β : [, ) [, ) such that λ qt), xt), t) αt), ψqt), xt), t) βt),, t, 17) with probabiity one with respect to P z. Then 15) and 16) hod. 3 At every switching time at east one of the counters N increases by one so assumption 24.8, 3. of [14] hods triviay. 1

11 Before proving Lemma 2 note that Assumption 2ii) does guarantee the existence of the functions α and β for which 17) hods and therefore Theorem 1 foows directy from Theorem 2 and Lemma 2. Proof of Lemma 2. From the definition of the transition times t k, we have that P N t) k) = Pt k t) = P e R t λ t k qs),xs),s)ds k 1 µ k ) k 1 = = P e R t t k 1 λ k qs),xs),s)ds µ k k 1, e R ti t i 1 λ i qs),xs),s)ds µ i i 1, i < k). Note that a the events added in the ast equaity occur with probabiity one actuay with equaity) so they do not change the overa probabiity. Because of 17), with probabiity one, therefore e R b a λ k qs),xs),s)ds e R b a αs)ds, b > a, P N t) k) P e R t R αs)ds t k 1 µ k k 1, ti e t αs)ds i 1 µ i t = P t k 1 αs)ds og µ k k 1, ti t P αs)ds + t k 1 t = P αs)ds t k 1 ti i=1 k i=1 t i 1 αs)ds ) og µ i i 1. i 1, 1 i < k) t i 1 αs)ds og µ i i 1, 1 i < k ) k i=1 ) og µ i i 1 Since a the µ i i 1 are independent and uniformy distributed in the interva [, 1], the og µi i 1 are independent random variabes exponentiay distributed with unit mean. Therefore k is Erang distributed and P k og µ i i=1 ) k 1 i 1 η = 1 where the right-inequaity can be proved by showing that is zero at zero and ϕη) η i= k 1 η ηi ϕη) := 1 e i ηk k i= η ηi e i! ηk k!, η = ηk 1 k 1) e η 1) <, η >. We therefore concude that k= P N t) k) k= t t αs)ds ) k. k! Finay, E [ N t) ] k P t N t t) k) k αs)ds ) k k! from which 15) foows. On the other hand, [ E ψqt k ), xt k ), t k ) ψqt k ), xt k ), t k) ] [ E t k t t k t i=1 og µi i 1 t ) R t t = αs)ds e αs)ds <, t ] [ 2βt) = E 2βt) ] N t), where, without oss of generaity, we assumed that β is monotone increasing. Inequaity 16) foows from this and 15). 11

12 4 A stochastic mode for TCP fows dt τ off w w, s q = off ẇ = ṡ = In this section we present a SHSs mode for a singe-user on-off TCP fow based on the hybrid modeing framework proposed by Bohacek et a. [9]. The mode is represented graphicay in Figure 3. It has two conq = ss og 2)w ẇ = n ack ṡ = w p drop w dt w, s w, s wf s s) dt 1 F s s)) q = ca wf s s) dt 1 F s s)) 1 ẇ = n ack ṡ = w w w w w 2 2 p drop w dt Fig. 3. Stochastic hybrid mode for a TCP fow, where n ack denotes the number of data packets acknowedged per each ACK packet received and w :=.693 when n ack = 1 or w := when n ack = 2. tinuous states TCP s congestion window size w and the cumuative number of packets sent in a particuar connection s and three discrete states {off, ss, ca}. 1. During the off mode the fow is inactive and we simpy have w = s =. 2. The ss mode corresponds to TCP s sow-start. In this mode, the congestion window size w increases by one for each ACK packet received and w packets are sent each round-trip time. This can be modeed by w ẇ = og 2)r og 2)w = n ack n ack, ṡ = r = w where r = denotes the instantaneous average sending rate and n ack the number of data packets acknowedged per each ACK packet received. The og 2) term compensates for the error introduced by approximating the discrete increments by a continuous increase. Indeed, without deayed ACKs we have n ack = 1 and this mode eads to the usua doubing of w every. With deayed ACKs, typicay n ack = 2 and this mode eads to a mutipication of the w by 2 every. This is consistent with the anaysis by Sikdar et a. [43], which shows that for n ack = 2 the number of packets sent in the nth round-trip time of sow-start is approximatey equa to ) 2 n. This formua is matched exacty by the fuid mode in 18) when one sets w = at the beginning of ss. On the other hand, for n ack = 1, the number of packets sent in the nth round-trip time of sow-start shoud be equa to 2 n 1. This matches the fuid mode by making w =.693 at the beginning of ss. 3. The ca mode corresponds to TCP s congestion-avoidance. In this mode w increases by 1/w for each ACK packet received and, as in sow-start, w packets are sent each round-trip time. This can be modeed by ẇ = 1 w The transitions between modes occur as foows: r 1 = n ack n ack, ṡ = r = w 1. Drops occurrences which correspond to transitions from the ss or the ca modes to the ca mode occur at a rate p drop r, where p drop denotes the per-packet drop probabiity and r := the packet sending 12 w 18)

13 rate. The corresponding intensity and reset maps are given by { pdrop t)w q {ss, ca} λ drop q, w, s, t) := otherwise { ca, w φ drop q, w, s, t) := 2, s) q {ss, ca} q, w, s) otherwise. 2. The start of new fows which correspond to the transitions from the off to the ss mode occur at a rate 1 τ off. This is consistent with an exponentiay distributed duration of the off periods with average τ off. The corresponding intensity and reset maps are given by λ start q, w, s, t) := φ start q, w, s, t) := { 1 τ off q = off otherwise { ss, w, ) q, w, s) q = off otherwise, where w := {.693 n ack = n ack = 2, 3. The termination of fows which correspond to transitions from the ss and ca modes to the off mode occur at a rate rf s s) 1 F s s), r := w. 19) This is consistent with a distribution F s : [, ) [, 1] for the number s of packets sent in each TCP session cf. Appendix). The corresponding intensity and reset maps are given by λ end q, w, s, t) := φ end q, w, s, t) := { wf s s) 1 F s s)) q {ss, ca} { otherwise off,, ) q {ss, ca} q, w, s) otherwise. Three main simpifications were considered: we ignored fast-recovery after a drop is detected by three dupicate ACKs, we ignored the deay between the time a drop takes pace and the time it is detected, and we ignored timeouts. Fast-recovery takes reativey itte time and has itte impact on the overa throughput uness the number of drops is very high [8]. Timeouts can have a severe impact on the throughput when drops are highy correated. Here, we are mosty interested in RED for which high correations are unikey. The drop detection deay is mosty important for the stabiity anaysis of congestion contro protocos, which wi not be pursued here. We focus our attention on two specific instances of the genera mode in Figure 3: Exponentia sizes This mode is obtained by assuming that the number of packets to transmit is exponentiay distributed with mean k, i.e., F s s) = 1 e s k, s. In this case, the intensity of the counter N end is given by λ end q, w, s, t) := { k 1 w q {ss, ca} otherwise. 13

14 Mixed-exponentia sizes It has been observed that modeing the distribution of transfer-sizes as an exponentia is an over-simpification. For exampe, it has been observed that heavy-tai modes are more fitting to experimenta data cf., e.g., [2, 4]). An aternative that turns out to be computationay attractive and sti fits reasonaby we with experimenta data is a mixture of exponentias. According to this mode, transfer-sizes are samped from a famiy of M exponentia random variabes s i, i {1, 2,..., M} by seecting a sampe from the ith random variabe s i with probabiity p i. Each s i corresponds to a distinct mean transfer-size k i. To mode this as a SHS, we consider M aternative {ss i, ca i : i = 1, 2,..., M} modes, each corresponding to a specific exponentia distribution for the transfer-sizes. The transition from the inactive mode off to the sow-start mode ss i corresponding to a mean transfer-size of k i occurs with probabiity p i and are associated with intensities and reset maps given by { { pi λ i q, w, s, t) := τ off q = off ss i, w, ) q = off φ i q, w, s, t) := otherwise q, w, s) otherwise, which repace the λ start and φ start in the previous mode. To obtained the desired distribution for the transfersize, the intensity and reset maps of the transitions to the inactive mode off, shoud be repaced by k 1 1 w q {ss 1, ca 1 } λ end q, w, s, t) :=.. k 1 M w q {ss M, ca M } otherwise { off,, ) q {ss i, ca i : i = 1, 2,..., M} φ end q, w, s, t) := q, w, s) otherwise. The intensities and reset maps of the transitions to the congestion avoidance modes must be adapted in the obvious way: { pdrop t)w λ drop q, w, s, t) := q {ss i, ca i : i = 1, 2,..., M} otherwise ca1, w 2, s) q = {ss 1, ca 1 } φ drop q, w, s, t) :=.. cam, w 2, s) q = {ss M, ca M } q, w, s) otherwise The exponentia-sizes mode in Sect. 4 is a specia case of this for M = 1. A simiar technique coud be used to obtain a mixture of exponentias for the distribution of the off periods. 4.1 Anaysis of the TCP SHS modes To investigate the dynamics of the moments of the sending rate rt) = TCP mode, n, q Q we define wt) t) for the mixed-exponentias µ q,nt) := E [ ψ q,nqt), wt), t) ], ψ q,nq, w, t) := { w n t) n q = q otherwise. 2) From these definitions we concude that E[r n t)] = q Q µ q,n t), t 14

15 and that µ q, t) = 1, µ off,n =, n 1, t. q Q The foowing resut can be obtained by directy appying Theorem 1 to the SHS TCP mode. Detais of the computations can be found in the Appendix. Theorem 3 Fu-order modes). For the mixed-exponentias mode in Sect. 4 we have 4 µ off, = µ off, τ off + MX j=1 µ ssi,n = piwn µ off, τ off + n og 2) n ackrt T n n ack µ cai,n = nµca i,n 1 n ack 2 n RT T µ cai,n 4.2 Reduced-order mode k 1 j µ ssj,1 + µ caj,1) 21) µ ssi,n p drop + k 1 i )µ ssi,n+1 22) p drop + k 1 i )µ cai,n+1 + p drop 2 n µss i,n+1 + µ cai,n+1). 23) The system of infinitey many differentia equations that appear in Theorem 3 describes exacty the evoution of the moments of the sending rate r but finding the exact soution to these equations does not appear to be simpe. However, as noted by Bohacek [7], Monte Caro simuations revea that the steady-state distribution of the sending rate is often we approximated by a Log-Norma distribution. Assuming that on each mode the sending rate r approximatey obeys a Log-Norma distribution even during transients, we can truncate the systems of infinitey many differentia equations that appear in Theorem 3. We reca that, if the random variabe x has a Log-Norma distribution with parameters µ and σ, then σ2 µ+ E[x] = e 2, E[x 2 ] = e 2µ+2σ2, E[x 3 9σ2 3µ+ ] = e 2, and therefore E[x 3 ] = E[x2 ] 3 E[x] 3. Therefore if r is approximatey Log-Norma distributed in the mode q Q, we have that where we used the fact that µ q,3 = µ q, E[r 3 E[r 2 q = q] 3 q = q] µ q, E[r q = q] 3 = µ q, µ 3 q,2, 24) µ 3 q,1 µ q,n = Pq = q) E[r n q = q] = µ q, E[r n q = q]. Using 24) in 21) 23), we can eiminate any terms µ q,n, n 3 in the equations for µ q,n, n 2, thus constructing the foowing finite-dimensiona mode to approximatey describe the dynamics of the first two moments of the sending rate: µ ssi, = pi`1 P M j=1 µss i, + µ cai,) p drop + k 1 i ) µ ssi,1 25) τ off µ cai, = p drop µ ssi,1 k 1 i µ cai,1 26) µ ssi,1 = w pi`1 P M j=1 µss i, + µ cai,) + og 2) n ack µ ssi,1 p drop + k 1 i ) µ ssi,2 27) µ cai,1 = µ cai, n ack 2 τ off n ack RT T µ cai,1 + p dropµ ssi,2 ` p drop 2 2 µ ssi,2 = w2 p i`1 P M j=1 µss i, + µ cai,) τ off 2 + og 4) µss i,2 + k 1 i µcai,2 28) n ack 2 RT T µ ssi,2 4 To simpify the notation, we omit the time-dependence of and p drop. p drop + k 1 i ) µss i, µ 3 ss i,2 µ 3 ss i,1 29) 15

16 µ cai,2 = 2 µca i,1 n ack 2 RT T µ cai,2 2 + p drop 4 µ ssi, µ 3 ss i,2 µ 3 ss i,1 3 pdrop 4 + k 1 i µcai, µ 3 ca i,2. 3) µ 3 ca i,1 We present next simuations of this reduced mode for a few representative parameter vaues. Figure 4 corresponds to a transfer-size distribution that resuts from the mixture of two exponentias M = 2) with parameters p 1 = 88.87%, k 1 = 3.5KB, p 2 = 11.13%, k 2 = 246KB. 31) The first exponentia corresponds to sma mice transfers 3.5KB average) and the second to eephant mid-size transfers 246KB average). The sma transfers are assumed more common 88.87%). These parameters resut in a distribution with an average transfer-size of 3.58KB and for which 11.13% of the transfers account for 89.7% of the tota voume transfered. This is consistent with the fie distribution observed in the UNIX fie system [21]. However, it does not accuratey capture the tai of the distribution it acks the mammoth fies that wi be considered ater). The resuts obtained with the reduced mode 25) 3) match reasonaby we those obtained from Monte Caro simuations of the fu SHS mode, especiay taking into account the very arge standard deviations. It is worth it to point out that the simuation of 25) 3) takes just a few seconds, whereas each Monte Caro simuation takes severa hours of CPU. Two somewhat surprising concusions can be drawn from Figure 4 for this distribution of transfer-sizes and off-times: 1. The average tota sending rate varies very itte with the drop rate at east up to the drop rate of 33% shown in the pots), with most of the packets transmitted beonging to eephant mid-size transfers. This is perhaps not surprising when most packets are transmitted in the sow-start mode for drop rates beow.8%) but sti hods when a significant fraction of packets are sent in the congestion avoidance mode. 2. The dynamics of TCP are competey dominated by second order moments. In Figure 4, the standard deviation is 5 to 2 times arger than the average sending rate, which is very accuratey predicted by the reduced mode. As the drop rate increases, the standard deviation decreases but even for p drop = 33% the standard deviation is sti 5 times arger than the average sending rate. This behavior is competey different from the one observed for TCP fows that are aways on, for which it c has been shown that the steady-state average sending rate is approximatey given by p drop, where denotes the average round-trip time, p drop the per-packet drop rate, and c a constant ranging from to 1.31 depending on the method used to derive the equation [8, 19, 32, 33, 37]. This equation is vaid at east for simpe network topoogies, sma vaues of p drop, and one acknowedgment per ACK packet n ack = 1). Generaizations can be found, e.g., in [38, 42]. We considered next a transfer-size distribution that resuts from a mixture of three exponentias M = 3) with parameters p 1 = 98%, p 2 = 1.7%, p 3 =.2%, k 1 = 6KB, k 2 = 4KB, k 3 = 1MB. 32) The first exponentia corresponds to sma mice transfers, the second to mid-size eephant transfers, and the third to arge mammoth transfers. The resuting distribution, shown in Figure 5, approximates the one reported by Aritt et a. [2] obtained from monitoring transfers from a word-wide web proxy within an Internet Service Provider. This distribution has a much heavier tai than the one considered before. Figure 6 contains resuts obtained from the reduced mode. We do not present Monte Caro resuts because the simuation times needed to capture the tais of the transfer-size distribution are prohibitivey arge. In this figure we varied the average off-time τ off from.2 to 5 seconds. This essentiay scaes the sending rate but does not significanty change the way it varies with the drop probabiity. It turns out that the main 16

17 Probabiity any active mode sow start congestion avoidance sma "mice" transfers mid size "eephant" transfers p drop Rate Mean 5 tota sow start congestion avoidance sma "mice" transfers mid size "eephant" transfers p drop Rate Standard Deviation 8 7 tota sow start congestion avoidance sma "mice" transfers mid size "eephant" transfers p drop Fig. 4. Steady-state vaues for the probabiity of a fow being on each mode top) and the average mid) and standard deviation bottom) of the sending rate as a function of the drop probabiity. The soid ines were obtained for the mode 25) 3) with = 5ms, n ack = 1, and a transfer-size distribution that resuts from the mixture of two exponentias with the parameters in 31). The mean off time was set to τ off = 5sec. The arger) symbos were obtained from Monte Caro Simuations. 17

18 1 1 2 Tai ProbX>x)) mixture of exponentias Aritt et a. distribtuion transfer size [Bytes] Fig. 5. Distribution of transfer-sizes resuting from the mixture of three exponentias with the parameters in 32), which were used in the simuations in Figure 6. concusions drawn before sti hod: the average sending rate varies reativey itte with the drop rate and the dynamics of TCP are competey dominated by second order moments. The mid-size eephants sti dominate foowed by the sma mice. The arge mammoth transfers occur at a rate that is not sufficienty arge to have a significant impact on the average sending rate. 5 Concusions This paper presents a new mode for SHSs where transitions between discrete modes are triggered by stochastic events, which occur at rates that are aowed to depend on both the continuous and the discrete states of the SHS. Based on resuts avaiabe for Piecewise-Deterministic Markov Process PDPs), we provide a formua for the extended generator of the SHS. As an iustration, we presented a SHS mode for on-off TCP fows that considers both sow-start and congestion avoidance. One important observation that stems from this work is that for reaistic transfer-size distributions high-order statistica moments seem to dominate the dynamics of TCP. Aso, the probabiity of drop appears to have a much arger effect on the standard deviation of the sending rate than on its mean vaue. We are currenty investigating the impact of this observation on the stabiity and performance of congestion contro mechanisms. This work opens severa avenues for future research. We woud ike to determine conditions under which Assumption 2ii) hods, mider than those required by Lemma 1. It is aso worth to investigate how soutions to deterministic hybrid systems that exist ony in a finite interva due to Zeno phenomena can be gobay extended using SHSs. Finay, one needs to deveop genera toos to approximate the dynamics of SHSs by finite-dimensiona systems of ODEs, as was done for the TCP exampe. Acknowedgments The author woud ike to thank Roger Brockett for providing a preprint of [11]; Martin Aritt for making avaiabe the processed data regarding the transfer-size distribution reported in [2]; Sanjoy Mitter and John Lygeros for provided severa reevant references; and Stephan Bohacek, Yonggang Xu, and Guiaume Bonnet for insightfu comments. 18

19 Rate Mean Rate Standard Deviation tota sow start congestion avoidance sma "mice" transfers mid size "eephant" transfers mid size "mammoth" transfers tota sow start congestion avoidance sma "mice" transfers mid size "eephant" transfers mid size "mammoth" transfers p drop p drop Rate Mean Rate Standard Deviation 12 1 tota sow start congestion avoidance sma "mice" transfers mid size "eephant" transfers mid size "mammoth" transfers tota sow start congestion avoidance sma "mice" transfers mid size "eephant" transfers mid size "mammoth" transfers p drop p drop Rate Mean Rate Standard Deviation tota sow start congestion avoidance sma "mice" transfers mid size "eephant" transfers mid size "mammoth" transfers 35 3 tota sow start congestion avoidance sma "mice" transfers mid size "eephant" transfers mid size "mammoth" transfers p drop p drop Fig. 6. Steady-state vaues for the average eft) and standard deviation right) of the sending rate as a function of the drop probabiity. These resuts were obtained from the reduced mode, with = 5ms, n ack = 1, and a transfer-size distribution resuting from the mixture of three exponentias with the parameters in 32). The mean off time was set to τ off = 5sec top), 1sec mid), and.2sec bottom). 19

20 Appendix Proof of Equation 5). Consider an interva t, t+dt], dt > and et t k be the argest transition time smaer than or equa to t. The probabiity that the counter N wi be incremented in t, t + dt] is given by P N t + dt) > N t) ) = P e R t+dt t λs)ds k µ R k < t e λs)ds ) t k = [ = E P e R t+dt t λs)ds k µ k < e R t λs)ds t k tk, qt k ), xt k ) )], where λs) := λ qt k ), ϕs; t k, qt k ), xt k )), s) and the expectation is taken with respect to the random variabes t k, qt k ), xt k ). Since µ k is uniformy distributed in [, 1], we have that P e R t+dt λs)ds t k µ k < e R t λs)ds t k tk, qt k ), xt k ) ) = e R t λs)ds R t+dt t k e t k with probabiity one, and therefore P N t + dt) > N t) ) = E [e R t λ t qt),ϕs;t,qt),xt)),s)ds k e R ] t+dt λ t qt),ϕs;t,qt),xt)),s)ds k, 33) where we used the facts that ϕs; t k, qt k ), xt k )) = ϕs; t, qt), xt)) and that qt k ) = qt) with probabiity one because there are no resets in t k, t] due to the definition of t k. Equation 5) foows from dividing both sides of 33) by dt and taking the imit as dt. λs)ds Proof of Lemma 1. Let q, x) denote a sampe path of q, x) constructed according to the agorithm in Sect. 2.1 with initia conditions q, x, t ) Q R n [, ) and t i, i the corresponding transition times. On each interva [t i, t i+1 ), x evoves according to 1) and therefore s xs) = xt i ) + fqt i ), xr), r)dr, s [t i, t i+1 ). t i Taking norms we obtain xs) xt i ) + s t i max{γ f r) xr), c f }dr, s [t i, t i+1 ). Since γ f r) xr) is continuous on [t i, t i+1 ), one of the foowing three cases must occur: 1. γ f r) xr) > c f, r [t i, t i+1 ). In this case, s xs) xt i ) + γ f r) xr) dr, s [t i, t i+1 ), t i and we concude using the Beman-Gronwa Lemma that R s γ t xs) e f r)dr i xt i ), s [t i, t i+1 ). 2. γ f r ) xr ) = c f for some r [t i, t i+1 ) and γ f r) xr) c f, r [r, t i+1 ). In this case, we concude from appying the Beman-Gronwa Lemma on [r, t i+1 ) that xs) e R s r γ f r)dr xr ) = er s r γ f r)dr c f γ f r ) s [r, t i+1 ). 3. γ f r) xr) < c f, r [t i, t i+1 ). 2