STEADY STATE AND SCALING LIMIT FOR A TRAFFIC CONGESTION MODEL

Transcription

1 STEADY STATE AND SCALING LIMIT FOR A TRAFFIC CONGESTION MODEL ILIE GRIGORESCU AND MIN KANG Abstract. In a general model used in internet congestion control, the time dependent data flow vector x(t) > undergoes a biased random walk on two distinct scales. The amount of data of each component x(t) goes up to x(t) + 1 with probability p on a unit scale or down to γx(t) with probability 1 p on a logarithmic scale, where p depends on the joint state of the system. We investigate the long time behavior, mean field limit, and the one particle case. A scaling limit is proved, in the form of a continuum model with jump rate α(x). The ergodic properties are established via the local Doeblin condition for the case when the rate α is bounded above an away from zero, and an explicit formula of the invariant measure is provided when α is constant. 1. Introduction In a general model used in internet congestion control [2, 3], related to classical autoregressive models ([8], Chapter 2), the time dependent data flow x(t) undergoes a biased random walk with unit steps in one direction (x moves to x + 1) and on a logarithmic scale in the other (x moves to γx, where < γ < 1). More precisely, assume (Ω, Σ, P ) is a probability space, {F t } t is a filtration. Let ζ i : [, ) (, ) n [, 1] be continuous functions, for each index i, 1 i n. In addition, we consider a family of n independent Poisson processes {π i (t)} 1 i n with rate λ >, adapted to the same filtration. For any starting point x with components x i, 1 i n, n Z +, let x(t) denote the pure jump Markov process on (, ) n with components (x 1 (t), x 2 (t),..., x n (t)), constructed as follows. A Poisson clock π i (t) is attached to each particle x i (t), 1 i n. When the Poisson clock π i, associated to the particle x i (t), rings at time τ, then x i moves to x i (τ ) + 1 with probability 1 ζ i (τ, x(τ )) and to γx i (τ ) with probability ζ i (τ, x(τ )). In this standard construction, there are no simultaneous jumps. Date: October 23, Mathematics Subject Classification. Primary: 6K35; Secondary: 6J5, 35K15. 1

2 The process x(t) can be regarded as a generalization of a continuous time linear state space model LSS(F, G) ([8], page 9), with driving matrices F and G depending on the trajectory of the process. The paper is organized as follows. The multidimensional process and the canonical coupling with a driving family of Poisson processes, together with a general result on the existence of probability invariant measures (Proposition 1) are presented in the current section. The rest of the paper is divided in two parts. First, Section 2 and 3 discuss the mean field limit of the multidimensional process in the case when the jump probabilities ζ depend on the average of the particles at a given time. The components decouple as the size of the system n, reducing the study of all relevant questions to the one particle process case. Existence and uniqueness of the solution to a hybrid linear differencedifferential equation (2.2) is proven, and an explicit formula for the invariant measure when ζ is constant is provided. The second part of the paper looks at a scaling limit (proven in Section 4), leading to a process x(t) in the continuum. This moves along the trajectory of a deterministic solution of an ode, and is interrupted by jumps to γx(t) at random time intervals, as in the discrete version. Section 5 deals with the ergodic properties of the process, via the local Doeblin condition, and again gives an explicit formula for the invariant measure when the jump rate is constant Martingale problem and a class of test functions. Due to the natural bound x i (t) x i + π i (t), for all 1 i n and t, (1.1) E[exp{θx i (t)}] exp{θ[x i + (λt)(e θ 1)]}, θ >. Equivalently, the process {x(t)} can be seen as the solution to a martingale problem. Denote R i x = (x 1, x 2,..., x i + 1,..., x n ) and L i x = (x 1, x 2,..., γx i,..., x n ). For a test function f C b ((, ) n ), let M f (t) be defined by the stochastic differential equation (1.2) λ M f (t) = f(x(t)) f(x()) n [( )( ) 1 ζ i (s, x(s )) f(r i x(s )) f(x(s )) i=1 ( )] ζ i (s, x(s )) f(l i x(s )) f(x(s )) ds. Then, M f (t) is a martingale with quadratic variation 2

3 (1.3) (1.4) M f (t) = λ n i=1 [( )( ) 2 1 ζ i (x(s )) f(r i x(s )) f(x(s )) ( ) 2 ] + ζ i (x(s )) f(l i x(s )) f(x(s )) ds. Definition 1. The solution to the martingale problem given by (1.2) will be called a γ - process. The average of the positions x i of the particles will be denoted by x. In the special case when ζ i (t, x) = p(t, x) for all particles 1 i n, where p(t, x) is a continuous function p : [, ) (, ) [, 1], {x(t)} t shall be called a mean field γ-process. Definition 2. Given θ >, let φ C 1,2 θ ([, ) (, )) be functions in C 1,2 ([, ) (, )) with derivatives a t b xφ, a 1, b 2 uniformly bounded by an exponential e θx. Precisely, there exists a positive constant K φ such that (1.5) sup sup t a xφ(t, b x)e θx = K φ <. a 1, b 2 (t,x) [, ) (, ) 1.2. Existence of invariant measures. A Poisson process escapes towards infinity as t. However, a lower bound on the rate ζ i of going back x γx, ensures that the process has at least one invariant probability measure. Proposition 1. Assuming that there exists p > such that ζ i p, for all 1 i n. Then the process defined by (1.2)-(1.3) has at least one invariant probability measure. Proof. Our goal is to prove the tightness of the family of probability measures on (, ) n (1.6) ν t (dx) = t 1 P x (s, dx) ds, t > where P x (s, dx) = P (x(s) dx x() = x ). Without loss of generality we shall take λ = 1 and also show the result only for n = 1. In this case, we drop the subscripts i. First, we shall give an estimate on the first moment of the process. Namely, if M m (t) = E[x(t) m ], we prove for m = 1 (1.7) M 1 (s)ds x + (1 p )t p (1 γ) The first observation is that moments are finite due to the bound provided by the Poisson processes dominating x(t).. Second, the moments are continuous functions in time, an 3

4 immediate consequence of the differential formula (1.2). We apply the expected value in (1.2) to a time interval t s t in the one-particle case, to obtain M m (t) = M m (t ) + t M m (t ) + (1 p ) t [ E (1 ζ(s, x(s )) (x(s ) + 1) m x(s ) m) m 1 j= for any m 1. Particularizing for m = 1, t = (1 γ m )ζ(s, x(s ))x(s ) m] ds ( ) m M j (s) (1 γ m )p M m (s)ds j M 1 (t) x + (1 p )t (1 γ)p M 1 (s)ds which can be re-written with the integral term on the left hand side where C(x ) depends only on x. M 1 (s)ds x + (1 p )t (1 γ)p C(x )t, Let M > be a large number. Then, for ν t defined in (1.6), (1.8) ν t ((M, )) = 1 t showing that (1.9) lim M P x (x(s) > M)ds 1 t lim sup ν t ([, M] c ) t E[x(s)] M ds C(x ) M C(x ) lim M M = which proves our claim. Any limit point of {ν t (dx)} t> is an invariant measure. 2. The one particle process We want to investigate the dynamics of the process governed by (1.2)-(1.3) in the special case n = 1. In view of Definition 1, without loss of generality, we denote the jump probabilities by p(t, x). Denote by A t the operator (2.1) A t φ(t, x) = (1 p(t, x))(φ(t, x + 1) φ(t, x)) + p(t, x)(φ(t, γx) φ(t, x)), applied to φ C 1,2 θ ([, ) (, ) 2 ). For a probability measure µ (dx) on (, ), we say that the time indexed measures µ(t, dx) are a weak solution to the evolution equation µ t = A t µ with initial condition 4

5 µ(, x) = µ (dx), where the star indicates the formal adjoint of A t in the space variable, if (2.2) φ(t, x), µ(t, dx) φ(, x), µ (dx) = A s φ(s, x), µ(s, dx) ds, with the notation φ, µ designating the space variable integral of the test function φ against µ. When the time dependence is emphasized, we shall write φ(t), µ(t) The forward equation. The next proposition looks at the case when p(t, x) depends only on time. We write p(t, x) = p(t) for simplicity. On one hand, this is a special case of the one particle process, and Proposition 2 applies to the case when p(t, x) is constant. On the other hand, the interest for this setting comes form the fluid limit from Section 3. By a law of large numbers effect, the average x(t) approaches a deterministic limit as in (3.4). In that case, p(t, x(t)) p(t) becomes a function of t only. Uniqueness of the pde (2.2) is essential for closing the argument of the fluid limit from Theorem 2. Proposition 2. The solution in weak sense to equation (2.2), corresponding to the forward equation of the one particle γ - process with jump rates ζ(t, x) = p(t), exists and is unique. Proof. Existence. It is evident that the transition probability µ(t, dx) = P µ (x(t) dx) of the process defined by (1.2)-(1.3) for n = 1 satisfies the desired equation, which is exactly the forward equation of the process. Uniqueness. The forward equation in integral form reads (2.3) φ(t, x), µ(t) φ(, x), µ() = { s φ(s, x) + (1 p(s))(φ(s, Rx) φ(s, x)) } +p(s)(φ(s, Lx) φ(s, x)), µ(s) ds, and should be valid for test functions φ(t, x) C 1,2 θ ([, ) (, )), and in particular φ(x) = e θ x, θ < θ, implying that their moment generating function is nontrivial (exists on an interval including the origin). Next, let φ(t, x) = x m, for integers m. We can see that { x m, µ(t) } m are defined recursively by a system of affine odes with uniqueness and global existence of solutions (see [5]). Two solutions will have equal moments for any fixed t, and will have equal moment generating functions, hence the measures are the same. 5

6 2.2. The invariant measure. The next result describes the invariant measure when the jump rate p is constant. Theorem 1. In the case when the jump probabilities are constant with ζ(t, x) = p > for all (t, x), the invariant measure µ(dx) is unique and has characteristic function ( (2.4) E µ [e iξ ] = Π n= 1 1 p ) 1 p (eiγnξ 1), ξ R. Alternatively, let {W n } n be i.i.d. geometric random variables with parameter p, that is P (W n = k) = (1 p) k p, k. Then, the probability measure µ(dx) on (, ) defined in (2.4) is the distribution of the random variable (2.5) X = γ n W n. Remark. n= Under general conditions on the function p(t, x), the solution to (3.4) has a limit as t. The particles are approaching a steady state corresponding equal to the equilibrium distribution of the process with constant p = lim t p(t, z(t)), whenever the limit exists. Proof. The existence and uniqueness of the invariant measure µ(dx) is proven in Propositions 1 and 2. The functions ξ e iξ are bounded and continuous. The series (2.5) is convergent in distribution (also almost surely) as long as γ < 1, so the distribution of X is well defined. We also notice that the infinite product in (2.4) is convergent due to the inequality e iξ 1 2 sin ξ 2 ξ. Then, it is immediate to verify that the characteristic function (2.4) satisfies the equation (2.6) (, ) A φ(x)µ(dx) =, with A the operator defined at (2.1) for constant p(t, x) = p. 3. The fluid limit for the mean-field model Throughout this section we assume that p(t, x) is differentiable with bounded derivatives in both variables. We start the investigation by considering the empirical measure of the n particle γ-process (3.1) µ n (t, dx) = 1 n n δ xi (t)(dx), i=1 6

7 with initial distribution (3.2) µ n (, dx) = 1 n n δ xi ()(dx). i=1 Assumption (A1). There exists θ > such that (3.3) lim sup E[ e θx, µ n (dx) ] <. n Assumption (A2). The initial distribution is said to have an initial deterministic profile µ (dx) M 1 ((, )) if lim n µ n (, dx) = µ (dx) in probability in the sense of the weak convergence of finite measures. Assumption (A3). We choose a finite collection of particles {x n j ( )}, 1 j l, with l a positive integer fixed for all n. Since the limit is considered as n the condition n l is trivial. We assume that for all 1 j l, the initial point x n j () has a deterministic limit x j. Theorem 2. Under (A1) and (A2), the average process x n ( ) is tight in D([, ), (, )) and any limit point x( ) is the solution of the ordinary differential equation (3.4) dy dt = (1 p(t, y)) (1 γ)p(t, y)y, y(). Additionally, the empirical measure process (3.1) is tight in the Skorohod space of timeindexed measure-valued paths D([, ), M 1 ((, )). Any limit point is the unique weak solution in the sense of (2.2) to the equation (3.5) φ(t), µ(t) φ(), µ() { s φ(s) + (1 p(s, x(s))(φ(s, Rx) φ(s, x)) } +p(s, x(s))(φ(s, Lx) φ(s, x)), µ(s) ds =, where φ C 1,2 θ ([, ) (o, )). Proof. The average process. We recall the coupling between x(t) and a family of Poisson processes, as in the discussion from Subsection 1. We arrange the particles, including the Poisson points, in a set of n pairs (x i (t), π i (t)), 1 i n. The Poisson processes are the clocks that trigger the jumps of the particles x i, and they only move forward. We then 7

8 have inequality x i (t) x i () + π i (t) and (1.1) for all i, and (3.6) x n (t) x() + 1 n n π i (t). i=1 The differential equations (1.2)-(1.3) are valid for f(x) C b ((, )), for any θ >. Here we are interested in f(x) = n 1 n i=1 φ(x i), with φ(x) C 1,2 θ ([, ) (, )). It is easy to see that we can extend (1.2)-(1.3) to this class of test functions, and implicitly to polynomials, due to the exponential bounds (1.1). Let T >, be fixed but arbitrary. Let φ(x) = x. At time t =, the average process x n () is tight (3.3). Then, (1.2)-(1.3) show that for t 1 t 2 T, [ ] E sup x n (t) x n (s) 2 C(t 2 t 1 ), t 1 s t t 2 with C = C(T ) independent of n, showing that { x n ( )} n 1 is tight in D([, T ], (, )), and any limit point x( ) is continuous. Moreover, the tight family of processes { x n ( )} n 1 satisfy, due to Doob s maximal inequality, [ lim E (3.7) sup x n (t) x n () n t T (1 p(s, x n (s ))) (1 γ)p(s, x n (s )) x n (s )ds 2 ] =. Assumption (A2) and (A1) imply that x n () converges in distribution to the deterministic point y = xµ (dx). Since p is continuous and bounded, and using once more the fact that expected values polynomials φ(x) have uniform bounds over n and t T, we have shown that any limit point x( ) solves (3.4). We note that (3.4) has unique local solutions, and since it has an affine bound, it also has global solutions [5]. The fluid limit. The proof of (3.5) follows the same steps as the proof for the average process. One has to prove tightness, which is a consequence of the Doob s maximal inequality applied to the martingale (1.2). The tightness is true on the Skorohod space D([, ), M 1 ((, ))) of time indexed, probability measure-valued paths, continuous to the right and with limit to the left. Moreover, any limit point of the tight family of measure valued processes (indexed by n) satisfies equation (3.5) modulo an error term of order 1/n. To close the argument, we only need the uniqueness of the solution of the pde (2.2), proven in Proposition 2. The details of the proof are standard in any hydrodynamic limit [6], also in a more similar 8

9 context in [4]. In addition, the proof of Theorem 4 outlines the main steps of essentially the same argument. Theorem 3. Under assumptions (A1), (A2), (A3), each particle x n j ( ) is tight and its limit point is equal in law to the one particle γ - process defined by equation (1.2) with n = 1 and a space independent p(t, x) = p(t, x(t)), where x( ) is the solution of (3.4). Moreover, the joint system of l tagged particles converges to a collection of independent one particle processes starting at x j, 1 j l. Proof. Under (A1), (A2), the average process x n ( ) converges to the solution of the ode (3.4). It is easy to see that under (A3), that takes care of the initial point, each individual particle is tight. In the Skorohod space D([, T ], (, ) l ), consider the joint l - dimensional limit point {x j ( )} 1 j l of the tagged particle collection x n j ( ), 1 j l, obtained as n. Taking into account the continuity of the coefficients p(s, ), we see that the l dimensional process {x j ( )} 1 j l solves the martingale problem (1.2)-(1.3) with n = l, i = j, ζ j (s, x) = p(t, x(t)) and x() = (x 1, x 2,..., x l ). Naturally, in this setting the l components are independent since no coefficient of the infinitesimal generator depends on more than one component. 4. Scaling limit We consider the mean field γ process with scaling given by n N, a time speed up t Nt given by λ N, the shrinking of the forward jump size equal to N 1. In addition, p N (t, x) = N 1 α(t, x), with α C,1 b ([, ) (, )), the space of functions with bounded continuous derivatives up to the multiindex (, 1). Finally, the backward jump size obeys the same rule with γ (, 1). The scaled process considered is x N (t) = x(nt). Denote the empirical measure by µ N (t, dx) = N 1 δ xi (Nt). We recall that for φ(x) C 1,2 θ ([, ) (, )) we use the shorthand φ, µ for the integral of φ against the measure µ Initial profile. We shall assume that there exists θ >, such that (4.1) lim sup E[ e θx, µ N (, dx) ] < +. N 9

10 Assume that there exists a deterministic measure µ (dx) M 1 ((, )) having all finite moments such that, for any ɛ >, and any φ C ((, )) we have the limit (4.2) lim N P ( φ(x), µ N (, dx) φ(x), µ (dx) > ɛ ) =. The measure µ (dx) is called the initial profile of the particle system. Notice that, formally, one can write x N () = x, µ N (, dx). Due to the condition (4.1) we are allowed to use polynomial functions as test functions in the definition of weak convergence, even though (, ) is unbounded. Then lim N x, µ N (, dx) = x, µ (dx) which implies that z = xµ (dx) The ode (4.3) satisfied by the average. For a given α(t, x), an initial value z, construct the solution z(t) to the ode (4.3) z (t) = 1 (1 γ)α(t, z(t))z(t), z() = z. Remark. Let λ = 1 and α constant. In this case z(t) = 2 α (1 (1 α 2 )e α 2 t ) remains bounded for all t The pde. Starting with the solution z(t) of equation (4.3), we define the operator B t on the space of test functions φ C 1,2 θ ([, ) (, )) as follows (4.4) B t φ(t, x) = φ(t, x) + α(t, z(t))(φ(t, γx) φ(t, x)). For any t, B t denotes the formal adjoint of B t in the space variable. We shall say that time indexed measures µ(t, dx) satisfy the equation (4.5) t µ = B t µ, µ(, dx) = µ (dx) with initial condition µ (dx) in weak sense, if µ(, dx) = µ (dx), and for any test function φ C 1,2 θ ([, ) (, )), (4.6) φ(t), µ(t) φ(), µ() s φ(s) + B s φ(s), µ(s) ds =. A solution µ(t, dx) to (4.6) can be obtained probabilistically as the transition probabilities of the inhomogeneous Markov process solving the martingale problem associated to (4.4). Since the construction can be done from a scaled pure jump random walk, there is only a question about the uniqueness of the solution, which is done in Step 4 of the proof of Theorem 4. 1

11 Theorem 4. For any initial profile, the average x N ( ) converges in probability to a deterministic continuous function z(t) solving the equation (4.3). In addition, the measurevalued processes µ N (, dx) converge weakly in probability to the unique measure-valued path µ(, dx) C([, ), M F ((, )) solving the equation (4.5). Proof. Without loss of generality, we shall prove the theorem on any time interval [, T ], where T is fixed but arbitrary. Step 1. We shall prove that all moment bounds (4.1) hold for µ N (t, dx), for any t. In fact, we prove [ ] (4.7) lim sup E N sup e θx, µ N (s, dx) s t < +. Also, ( (4.8) P sup s t x m, µ N (s, dx) µ N (, dx) ) > λt e C(t)N. This inequality is based on the martingale inequality applied to a process π N (t) defined by coupling. At time t =, both processes start from the same points x i () = π i (). Whenever the clock associated to, say, particle i rings, the particle π i simply jumps forward by N 1. Naturally π i (t) x i () are independent Poisson processes, and all bounds are easy to calculate. The uniform integrability from (4.7) implies that {µ N (t, dx)} N> is a tight family of measures for any t. Step 2. We prove the limit for the average process. First, assume φ(x) = x. Denote φ, µ(t) = z(t) in this case. Then equation (4.6) is a differential equation in all coordinates of Z d of the form (4.3), where we assume that the initial profile has asymptotic average z. Step 3. Let φ C 1,2 θ ([, ) (, )). For each N, the differential formula corresponding to φ(t), µ N (t) can be obtained directly from (1.2), applied to the function f(t, x) = N 1 φ N (t, x i ), with the substitution t Nt using the scaled test function φ N (t, x) = 11

12 φ(n 1 t, x). The differential formulas (1.2)-(1.3) give (4.9) (4.1) (4.11) (4.12) M N φ (t) = φ(t), µn (t) φ(), µ N () { t n N 1 s φ(s, x N i (s )) + i=1 [( )( N 1 p N (s, x N (s )) φ(s, x N i (s ) + 1 ) N ) φ(s, xn i (s )) ( + p N (s, x N (s )) φ(s, γx N i (s )) φ(s, x N i (s )))] } ds. (1) The integrands on the right hand side of (4.1)-(4.12) have uniformly bounded moments due to the estimates (4.7), which proves that φ( ), µ N (t), indexed by N, are tight processes on D([, T ], M 1 ((, ))). Moreover, any limit point on the Skorohod space is, in fact, continuous in time, more precisely belongs to C([, T ], M 1 ((, ))). (2) Modulo error terms of order N 1, line (4.11) is equal to N 1 φ(x N i (s)). Here we use that α C,1 b ([, ) (, )), the Taylor formula with the remainder in integral form. (3) If φ γ (t, x) = φ(t, γx), and ɛ >, we have (4.13) (4.14) lim P N ( sup t T φ(t), µ N (t) φ(), µ N () ) s φ(s) + φ + α(s, x N (s ))(φ γ φ), µ N (s ) ds > ɛ = as a consequence of the martingale inequality and the fact that the quadratic variation (1.3) of M N (T ) is of order O(N 2 ). (4) Any limit point µ(, dx) of the tight sequence of empirical measures {µ N (, dx)} is continuous in time and satisfies equation (4.5) in weak sense. We only have to prove that the equation has a unique deterministic solution, completed in the next step. Step 4. To prove the uniqueness of the solution to (4.5), we define µ m (t) = x m, µ(t, dx), m and see that (4.6) with φ(t, x) = x m gives the recurrence (4.15) d dt µ m(t) = mµ m 1 (t) + α(t, z(t))(γ m 1)µ m (t), µ (t) = 1. The affine ode have unique global solutions since α is bounded (a general result when the equation has an affine bound [5]). Once again (4.7) imply the existence of the moment 12

13 generating function of the measures µ(t, dx) obtained as limit points. moments shows uniqueness. The equality of Remark. An alternative proof for the uniqueness of the weak solution to (4.5) can be carried out as follows. We solve the forward equation corresponding to the operator (4.4) by construction the Markov process y(t) and calculating explicitly its transition kernel g(s, x; t, dx ). Next, we show that g is absolutely continuous, with Radon-Nikodym derivative a smooth function vanishing at infinity, denoted by g as well. Finally we take φ(, ) = g(, ; t, x ) in (4.6) and obtain uniqueness. 5. The one particle process for the scaled model The one particle case arises naturally. If we isolate a single particle tagged by label i = 1 without loss of generality, equations (1.2)-(1.3) applied to f(t, x) = φ(t, x 1 ) in the same manner as in (4.9), show that {x N 1 (t)} N> is tight and any limit point is a process solving the martingale problem associated to (4.4) Irreducibility and recurrence. It is intuitive that if α(x) approaches zero, the time before a jump backwards becomes very large and the particle escapes, at constant speed, to infinity. The extreme case α(x) is when the particle performs a uniform forward motion. Evidently, there is no equilibrium probability measure. The other enlightening case is when α(x) is constant, when Theorem 6 gives the explicit form of the invariant measure. However, if α is only bounded away from zero, we have the next theorem. Theorem 5. The time homogeneous process with generator (4.4) with α bounded away from zero has a unique invariant measure, absolutely continuous with respect to the Lebesgue measure λ(dx). Remark 1. The invariant measure may not be finite. However, a proof along the same lines as Proposition 1 shows that the invariant measure is a probability measure. Alternatively, Proposition 3 implies the same when < γ < 1/2. Remark 2. Theorem 5 is not true when α(x) converges to zero as x, an example being α(x) = (1 + x) 1. Then z ln(2 + z) = t + C, C = z ln(2 + z ). We note that z > 1/2 hence it is increasing to infinity. Before anything else notice that as t (at equilibrium, if it exists), z(t) and α(z(t)). The steady state equation (4.5) reduces to µ = so the solution is the Lebesgue measure (not a probability distribution). 13

14 Proof. Let l(dx) be the Lebesgue measure on (, ), A B((, )) with l(a) >. Write G(x, A) = E x [ ] 1 A (x(t))dt for the Green function associated to the process, with x (, ). We shall use Theorem 1..1 in [8] to prove the theorem. We have to show that (1) P x (τ A < ), or the process is λ - irreducible, and (2) G(x, A) = for any x (, ) and any A B((, )) with λ(a) >. Without loss of generality, we assume that A is an open interval (a, b). Let < a < a and denote τ = inf{t x(t) < a } and τ A the first hitting time of A. Denote α > the lower bound of α such that α(x) α > and pick an arbitrary ɛ >. Let χ 1, χ 2,... be the i.i.d. holding times of the Poisson process driving x(t), w 1, w 2,... the actual holding times of the process and τ =, τ 1, τ 2,... be the actual jump times of x(t). More precisely, for j 1, (5.1) τ j = inf{t > τ j 1 χ i < τ j 1 α(x(s))ds}, w i = τ i τ i 1. Part (1). We first show that P x (τ < ) = 1 for all x >. By construction, (5.1) implies that w i α 1 χ i with probability one. Right after exactly the n-th jump, a particle that started at x will be at n 1 (5.2) x(τ n) = γ n x + γ k w n k γn x + α 1 γk χ n k. k= It is straightforward to see that unless x < a, we have that τ coincides with the position x(τ n), exactly after a jump, for some n 1. A coupling argument based on a process with constant α = α driven by the holding times {χ i } i, together with (5.2), shows that if the process with constant rate reaches (, a ), then for sure {x(t)} t reaches it even before. Proposition 4 (not dependent on the results in this section) concludes the argument. Part (2). Using once again (5.2), the position at t = τ n, after n consecutive holding times of length less than ɛ is n 1 x(τ n) = γ n x + γ k w n k γn x + ɛα 1 (1 γ) 1. k= 14

15 We choose n and ɛ such that γ n x < a /2 and ɛ < ( a 2 )α (1 γ). Noticing that τ n τ, we have P x (τ A < ) P x (χ 1 ɛ, χ 2 ɛ,... χ n ɛ, w n+1 > a) (1 e ɛ ) n e α a >. Part (3). Starting with τ defined in Part (i), for i = 1, 2,... we set (5.3) σ i = inf{t > τ i x(t) A}, τ i+1 = inf{t > σ i x(t) (, a )}. In view of the results from Part 1, the event that all τ i τ i 1 <, i 1, has probability one. We need to calculate [ ] [ (5.4) G(x, A) E x 1 A (x(t))dt E x τ i= τi+1 τ i ] 1 A (x(t))dt. Applying the strong Markov property to (5.4), it is enough to show that [ τ 1 ] (5.5) inf E y (,a y 1 A (x(t))dt >. ) For χ an exponential random variable with mean value one, independent of the process, define the first jump time of the process w = inf{t > χ < α(x(s))ds}. The rates α(x) are bounded above by < α < and α w χ. Then [ τ 1 ] E y 1 A (x(t))dt ( b a ( 2 )P y w + y > a + b ) 2 ( b a ( 2 )P y w > a + b ) 2 ( b a ( 2 )P y χ > α ( a + b ) 2 ) = exp{ α ( a + b 2 )} > Ergodic properties. Let τ F = inf{t > x(t) F } be the first entry time in the open set F. Both the following definition, which is a variant of the local Doeblin condition, and the next theorem due to Orey are adapted form [1] and [7]. Definition 3. (Local Doeblin condition) A subset F of the state space S of the Markov process will be said Doeblin attractive if (i) P x (τ F < ) = 1, for any x S, and (ii) there exists a probability measure ν (dx) concentrated on F, a constant c (, 1) such that, for all x F and all Borel sets B of S, we have P x (B) cν (B). 15

16 Theorem (Orey). If a Markov process has a Doeblin attractive set, then the process has a unique invariant measure µ and for every x in the state space, we have the limit lim t P x (x(t) ) µ( ) =, where is the total variation norm of a finite measure. Proposition 3. If γ (, 1 2 ), then any set (, a ), a > is an attractive Doeblin subset of the state space S = (, ). Remark. The condition γ < 1 2 can be relaxed by replacing the condition that exactly one jump has happened (as in the proof of the proposition) with the condition that exactly n > 1 jumps have happened, for an appropriate integer n. Proof. We shall use the same notations for the holding times and jump times as in the proof of Theorem 5. In that proof we show that condition (i) in Definition 3 is satisfied for any set (, a ). We proceed to the proof of (ii). Pick a point x (, a ). Denote A(y) = y α(x )dx and notice that A is differentiable with continuous strictly positive derivative, so A is invertible. Then (5.6) (5.7) A(w 1 + x) A(x) = χ 1 A(γ(x + w 1) + w 2) A(γ(x + w 1)) = χ 2. Whenever γ < 1/2, we can fix t such that t/a (γ(1 γ) 1, (1 γ)γ 1 ) and consider y 1 y 2 two numbers in [γa + γt, t]. The condition on t makes sure (i) that t γa + γt, (ii) whenever y γa + γt, then the lower bound for w 1 in (5.1) is nonnegative, and (iii) this interval has a nontrivial intersection with (, a ). (5.8) (5.9) If exactly one jump has been observed, then x(t) = γ(x + w 1 ) + (t w 1 ) and ( ) ) P x y 1 < x(t) y 2 P x (y 1 < x(t) y 2, w 1 t < w 1 + w 2 P x (y 1 < γ(x + w 1) + (t w 1) y 2, w 1 t < w 1 + w 2 Due to the conditions on t, y 1, y 2, the event we calculate the probability of in (5.9) is equal to (5.1) = ). { } (1 γ) 1 (γx + t y 2 ) w 1 < (1 γ) 1 (γx + t y 1 ), t w 1 < w 2. 16

17 From (5.7) we see that t w 1 < w 2 is the same as A(γ(x + w 1) + t w 1) A(γ(x + w 1)) < χ 2. Equation (5.6) shows that w 1 = A 1 (A(x) + χ 1 ) x is a function of χ 1, proving that w 1 and χ 2 are independent. Let ( )} { } ρ(w) = α(x + w) exp{ A(w + x) A(x) α exp α w be the density of the random variable w 1. If y i = (1 γ) 1 (γx + t y i ), the probability from (5.9) is equal to i = 1, 2, then (5.11) y 1 { ( )} exp A(γ(x + w) + t w) A(γ(x + w)) ρ(w) dw y 2 { ( )} (1 γ) 1 α exp α t (y 2 y 1 ). We have shown (ii) from Definition 3 with ν (dy) equal to the uniform probability measure on [γa + γt, t] and c < max{1, α (1 γ) 1 (t(1 γ) γa ) 1 e α t }, after incorporating the normalization constant for the uniform measure on [γa + γt, t] in the lower bound The case α constant. In the present discussion we assume a constant α(t, x) = α > and the process with pre-generator (4.4). Proposition 4. For any a > and any x >, let τ = inf{t > x(t) < a } and u θ (x) = E x [e θτ ], θ. If û(β), β is the Laplace transform of u θ (x) in the variable x >, then (5.12) û(β)(θ + 1 β) = γ 1 û(γ 1 β) + θβ 1 (1 e βa ) 1 which leads to (5.13) û(β) = n= [ ] [ 1 θβ 1 γ n (1 e γ n βa ) 1 Π n k= γ(1 + θ γ β)] k, whenever β γ n (1 + θ), n. Moreover, u (x) = P x (τ < ) = 1. Proof. For any θ, let (5.14) u θ (x) = E x [e θτ ], u θ (x) = 1, x a. 17

18 We are interested mainly in P x (τ < ) = u (x), having to show that u (x) 1. Let χ be the first exponential holding time of intensity one (without loss of generality). Then, if x < a, we have u θ (x) = 1, and if x a we can derive the relation E x [e θτ ] = which reduces to (5.15) u θ (x) = E x [e θτ χ = s] e s ds = After a substitution y = γ(x + s), E γ(x+s) [e θ(s+τ ) χ = s] e s ds u θ (γ(x + s))e (θ+1)s ds, x a. (5.16) u θ (x)e (θ+1)x = γ 1 u θ (y)e γ 1 (θ+1)y dy, x a. γx Due to the integral equation (5.16), u θ (x) is bounded, we derive that it is continuous, and then that it is also differentiable. simplification by a factor e (θ+1)x, we have After differentiating both sides of (5.16) and a (5.17) u θ (x) = (θ + 1)u θ(x) u θ (γx), x a. Naturally, this is (B θ)u θ =, with B = B t in the time homogeneous case and α 1 from (4.4). Keeping in mind that (5.17) is true only for x a, we calculate the Laplace transform û θ (β), β, of u θ (x) (in the space variable). It satisfies [ (5.18) 1 + βû(β) = (θ + 1) û(β) 1 ] [ β (1 e βa ) γ 1 û(γ 1 β) 1 ] β (1 e βa ) which gives (5.12). Solving the recurrence relation for β γ k β, k n 1 and performing the necessary cancellations we obtain (5.13). We want to prove that the solution is unique. The Laplace transform of the difference v θ (x) of two possible solutions of (5.15) satisfies (5.19) γ 1ˆv(γ 1 β) = (1 + θ β)ˆv(β), ˆv( ) =. Iterating once again relation (5.19) for β γ k β, k n 1, we can see that [ 1 (5.2) ˆv(x) = Π n 1 k= γ(1 + θ γ β)] k ˆv(γ n β), β > 1 + θ. Letting n, we have ˆv =. When θ =, the constant solution equal to 1 solves (5.15)-(5.18). By uniqueness, we have proved that P x (τ < ) = 1. 18

19 5.4. The invariant measure. If the scaled process solving the martingale problem associated to the operator (4.4) with α(t, x) = α(x) is ergodic, then x N (t) approaches a deterministic value as t, (5.21) lim t x N (t) = lim t x, µ N eq(dx) and then we look at the scaled limit (5.22) lim N x, µn eq(dx) = z eq. Then the steady state equation derived from (4.5) is (5.23) φ + α(φ(γx) φ(x)), µ =, α = α(z eq ), µ(dx) M 1 ((, )), where φ C 1,2 θ ([, ) (, )). An alternative way to identify the steady state is by setting Bw(x) = 2w(2x) α 1 w(x); then a steady state is a fixed point of the operator B. A heuristic approach, that offers insight in the way the equilibrium is approached by the process, is to look at the position of the Markov process right after each jump. A particle starting at y drifts with constant velocity one in the positive direction. When an exponential clock with intensity α rings, it jumps to a position equal to γ times its current position. Let W n be a sequence of i.i.d. exponentials with intensity α. Let Y n be the position at the n th jump. Then Y 1 = γ(y + W 1 ), Y 2 = γ(y 1 + W 2 ),... Y n = γ(y n 1 + W n ) yielding Y n = γ n Y + n j=1 γn+1 j W j. We calculate the limiting distribution of Y n. The moment generating function is [ ] (5.24) E e ξy n = e ξγn y Π n j=1 (1 ( ξ α )γn+1 j) 1 with limit as n equal to ] (5.25) E [e ξy = [ Π j=1 (1 ξ( γj )] 1 α ). One can see from (5.25) that Y k=1 γk Wk, where W k are i.i.d. exponential with parameter α and indicates equivalence in probability law. Theorem 6. The steady state of the process, and also the stationary solution of the equation (4.5) for constant α(t, x) = α is the measure with moment generating function (5.25). 19

20 Proof. Once the uniqueness of the invariant measure is proven (Theorem 5), the proof reduces to a verification of B µ =, the equation (4.5) when α is constant and the solution is stationary. Using the moment generating function, it is enough to check (4.4) for exponential test functions. The details are identical to the proof of Theorem 1. References [1] Duflo, M. Random iterative models. Applications of Mathematics (New York), 34. Springer-Verlag, Berlin, [2] Eun, D. Y. On the limitation of fluid-based approach for internet congestion control. IEEE International Conference on Computer Communications and Networks (ICCCN), San Diego, CA, Oct. 25 [3] Eun, D. Y. Fluid approximation of a Markov chain for TCP/AQM with many flows. Preprint. [4] Grigorescu, I., Kang, M. (24) Hydrodynamic Limit for a Fleming-Viot Type System. Stochastic Process. Appl. 11, no. 1, [5] Hale, J.K. Ordinary Differential Equations, Wiley-Interscience, [6] Kipnis, C.; Landim, C. (1999) Scaling Limits of Interacting Particle Systems. Springer-Verlag, New York. [7] Lacroix,J. Chaines de Markov et processus de Poisson, [8] Meyn, S. P.; Tweedie, R. L. Markov chains and stochastic stability. Communications and Control Engineering Series. Springer-Verlag London, Ltd., London, Department of Mathematics, University of Miami, Coral Gables, FL address: igrigore@math.miami.edu Department of Mathematics, North Carolina State University, Raleigh, NC address: kang@math.ncsu.edu 2