A Simple Memoryless Proof of the Capacity of the Exponential Server iming Channel odd P. Coleman ECE Department Coordinated Science Laboratory University of Illinois colemant@illinois.edu Abstract his paper provides a conceptually simple, memoryless-style proof to the capacity of the Anantharam and Verdu s exponential server timing channel (ESC). he approach is inspired by Rubin s approach for characterizing the ratedistortion of a Poisson process with structured distortion measures. his approach obviates the need for using the information density to prove achievability, by exploiting: ) the ESC channel on [, ] law is a product of [, ] ESC channel laws given intermediate queue states; 2) for Poisson inputs, the queue states form a Harris-recurrent Markov process. Achievability is subsequently shown by first demonstrating via the law of large numbers for Markov chains that an AEP holds, followed by standard random coding arguments. We extend our methodology to demonstrate achievability with Poisson inputs for any point process channel where the conditional intensity at any time is only a function of the queue state. Lastly, we demonstrate achievability with Poisson inputs for the tandem queue. I. INRODUCION he landmark Bits through Queues (BQ) paper by Anantharam & Verdú [] characterized the fundamental limits of coding through the timing of packets in queuing systems. A special case of FCFS queuing timing channel is the exponential server timing channel (ESC) where service times are i.i.d. and exponentially distributed of rate µ. For an arrival process of rate λ, it was shown in [] that the First- Come First-Serve (FCFS) /M/ continuous-time queue with service rate µ > λ > has a capacity C(λ, µ) given by C(λ, µ) λ log µ, λ < µ nats/s. () λ hen the capacity of the FCFS /M/ continuous-time queue with service rate µ is given by the maximum of C(λ, µ) over all possible arrival rates λ < µ, namely, C(µ) e µ nats/s, (2) where the maximum corresponding to (2) is achieved in () at λ e µ. m Fig.. encoder ESC (µ) Q decoder Conveying information through packet timings in a queueing system. ˆm A. Related Work on Proving C(λ, µ) he original BQ work [] first proved C(λ, µ) by considering the probabilistic dynamics relating n packet arrivals to n packet departures of an ESC. hey describe these probabilistic dynamics and then use a standard Fano s inequality converse by controlling the idling times. Achievability with a Poisson input process was shown using the General formula for capacity [2] information density calculus - because the channel is not memoryless. In [3], Bedekar and Azizoglu considered the discrete-time memoryless queuing system. Using similar techniques to [] involving information density arguments pertaining to the probabilistic dynamics relating n packet arrivals to n departures, they proved the analogous capacity of the discrete-time memoryless queue. In [4], Prabhakar and Gallager discussed the discrete-time memoryless queuing system from the n packet arrivals to n packet departures, but from a different viewpoint. hey upper bounded the limit of maximal normalized inputoutput mutual information by noting that the service times form a bijection with the output times given the input times. hat being said, as they themselves mention (in [4, Sec IV, footnote 2 ]), this technically is not sufficient to prove capacity, because this is not a memoryless channel. In [5], Sundaresan and Verdú re-visited the ESC, this time from a point processes viewpoint, by studying probabilistic dynamics between the arrival and departure point processes on [, ]. hey exploited the intimate relationship between the ESC and the Poisson channel, and were able to use capacity results of Poisson channels to provide a converse for the ESC, but still used information density arguments to illustrate achievability. B. Methodology of Our Approach his paper, similar to [5], considers all input/output processes in continuous time on the [, ] interval. Ours differs because we illustrate that one can reason about this channel coding problem completely from a traditional memoryless channel perspective using random coding and the AEP, by exploiting a few key properties: he ESC channel on [, ] law is a product of [, ] ESC channel laws given intermediate queue states;
For Poisson inputs, the queue states form a Harrisrecurrent Markov process. he channel law can be completely characterized in terms of the output process and the queue states he law of large numbers for Markov chains [4] holds to demonstrate that an AEP holds, Standard random coding arguments apply to arrive at achievability. We extend our methodology to demonstrate (via Burke s theorem) achievability with Poisson inputs for any point process channel where the conditional intensity at any time is only a function of the queue state. Lastly, we demonstrate achievability with Poisson inputs for the tandem queue. Since many converses for the ESC have been established, we focus here completely on proving achievability and showing such channels are all information-stable. he reasoning of the proof techniques for the ESC in this is paper is in some sense dual to that of lossy compression scheme for compression Poisson processes [7] with queuing distortion measures, developed in [8], [9]. II. PRELIMINARIES A. Notation on Point Processes Let (Ω, F) to be a measurable space. Denote ( t ) t [, ] as a stochastic process on [, ], and define σ ( s ) s [,t] F t Define to be the set of functions x : [, ] Z + that are non-decreasing, right-continuous, and x. In other words, is the set of point processes on [, ]. Define the input space as ( ), F and the output space as ( ), F with. With a probability space (Ω, F, P ), a stochastic point process ( t ) t [, ] on the space ( ), F, adapted to (F t ) t [, ] has the conditional intensity process λ (λ t ) t [, ] with respect to (F t ) t [, ] if P ( t+ t F t ) λ t lim. (3) Define P to be the probability measure on ( ), F s.t. is a point process with unit-rate conditional intensity with respect to ( ) Ft t [, ]. B. he Exponential Server iming Channel Let us now consider an ESC on [, ], where we denote the arrival process as on ( ), F, and the departure process as on ( ), F. Define the random variable Q as the queue state at time. Since the service times are i.i.d. and exponentially distributed (and thus memoryless), for a fixed x and q Z +, the probability measure P,Q ( x, q ) on ( ), F for the ESC has the rate process λ t µ Qt > (4) Q t q + x t t. (5) with respect to the filtration (H t ) t [, ] : H t σ ( t ) t [,t]. Define the Radon-Nikodym derivative of P,Q ( x, q ) with respect to P as [], [5]: f,q (y x, q ) dp,q ( x, q ) dp (y) (6a) exp log λ t dy t + [ λ t ]dt (6b) exp log ρ(q t )dy t + [ ρ(q t )]dt µ y e exp ρ(q) µ q> ρ(q t )dt (6c) (6d) Define P ( ) as the probability measure for the stochastic process on ( ), F. Denote the density of P ( ) with respect to P is given by [3], [] f (x) dp dp (x). Given f (x), a marginal distribution on Q, and f,q (y x, q ), the probability measure P ( ) on ( ), F follows. Similarly, denote the density of P ( ) with respect to P as f (y) dp dp (y). he mutual information I (; ) is given as I (; Q ) E log f,q (, Q ) f. Q ( Q ) C. Encoder Constraints, Achievability and Capacity M equally likely messages are mapped to codewords, each of which a member of. For λ < µ, we constrain the encoders to satisfy [ ] E λ. (7) he decoder observes the departure process on [, ], y, and decodes to one of the M messages. Such a code with an average error probability of P e is defined a (, M, ɛ, P e ) code. A rate R is achievable if there exists a sequence of ( k, 2 k R, Pe,k, λ) codes with lim k k and lim k P e,k. he λ-capacity is the supremum of all achievable rates.
III. ACHIEVABILI FOR HE ESC USING HE AEP Consider an arbitrary > and a positive integer n. For x and y, define x x n ( x,..., x n ), x i ( ) x t x (i ) t [(i ),i ) y ỹ n (ỹ,..., ỹ n ), ỹ i ( ) y t y (i ) t [(i ),i ) q n ( q,..., q n ), (8a) (8b) q i (q t ) t [(i ),i ) (8c) Note that x i and ỹ i. See Figure 2 as an example of the relationship between x and x n ( x,..., x n ). 9 8 7 6 5 4 3 2 3 2 x x 2 3 x x 2 x 3 Fig. 2. op: a realization of x, with n 3. Note that x 9 - so at time t, 9 packets have arrived. Bottom: x can be equivalently expressed in terms of n 3 counting functions x ( x, x 2, x 3 ) with each x i, given by (8a). he simple yet crucial observation we make is that For any t [(i ), i ): q t q + x t y t q + x (i ) y (i ) + [x t x (i ) ] [y t y (i ) ] q (i ) + x i,t ỹ i,t. It follows from (6) that the channel law is memoryless conditioned upon intermediate queue states: f,q (y x, q ) exp log ρ(q t )dy t + [ ρ(q t )]dt i exp t(i ) f,q (ỹ i x i, q i, ) where f,q (, ) is given by (6). log ρ(q t )dy t + [ ρ(q t )]dt (9a) (9b) We develop an achievability theorem for the ESC with Poisson inputs by proving the existence of an asymptotic equipartition property (AEP) [2]. Define P ( ) as the probability measure for the stochastic process on ( ), F corresponding to a Poisson process of rate λ. As such, the density of P ( ) with respect to P is given by [3], [] f (x) dp dp (x) λ x e (λ ) () Draw Q according to the steady-state distribution of a (λ, µ) birth-death Markov chain: ( P Q (k) λ ) ( ) k λ, k. µ µ hen from Burke s theorem [3], [], the output will also be a Poisson process of rate λ, and so f (y) dp dp (y) λ y e (λ ). () With these observations, we can develop the following AEP with Poisson inputs: Marginal on : log f () λ n i, log λ a.s. λ( log λ) (2) where (2) follows because of the independent increments property of the Poisson Process. Marginal on : from Burke s theorem, the output is Poisson. So from (2): log f ( ) a.s. λ( log λ) (3) Joint on and : Define: ν ( q i ) ρ( q i,t )dt (4) Note that since the queue states (Q t ) t [, ) form a Harris-recurrent continuous-time Markov process, the ( Q i : i n) form a discrete-time Harris-recurrent Markov chain [4]. Note from (9),(6b), (4) that: log f,q (, Q ) Ỹ i, n log µ + ν ( n Q i ), a.s. λ( log µ) by the strong law of large numbers (SLLN) and the SLLN for Markov chains [4]. As a consequence, the AEP holds, and from standard random coding arguments [2], any rate R < I (; ) λ log ( ) µ λ is achievable.
IV. ACHIEVABILI HEOREM FOR AN CHANNEL WHERE HE CONDIIONAL INENSI ONL DEPENDS ON HE QUEUE SAE We now extend the aforementioned approach to general timing channels where the conditional intensity is only a function of the current queue state. An example of which is the /M/m queue, where an achievabiilty theorem with Poisson inpus was developed in [5]. We illustrate that for such general channels, the mutual information corresponding to Poisson inputs is an achievable rate. Our assumption is that for a fixed x and q Z +, the probability measure P,Q ( x, q ) on ( ), F has the rate process λ t ρ (Q t ) (5) with respect to (H t ) t [, ], with H t σ ( t ) t [,t] and Q t given by (5). hus, by (6b), for x and y : f,q (y x, q ) exp log ρ (q t ) dy t + [ ρ (q t )]dt f ),Q (ỹi x i, q (i ) φ(ỹ i, q i ) (6) ) where (6) is explicitly denoting how f,q (ỹi x i, q (i ) is a measurable function of (ỹ i, q i ). heorem 4.: he rate R I (; Z) pertaining to P ( ) on ( ), F as a Poisson process of rate λ is achievable on any point process channel satisfying the assumption in (5) for which the induced queue process is recurrent. Proof: Define a b (a τ : τ [, b]). Let f (x) be the density of a Poisson process of rate λ on ( ), F. hen the process (Q t ) t [, ] is a birth-death Markov process. Draw Q P Q ( ) with P Q ( ) the steady-state probability distribution on the birth-death chain. Note the joint process (Q t, t ) t [, ] is also a Markov process: In infinitesimal time, the queuing dynamics satisfy Q t Q t ( t t ) ( t t ). So: P ( Q t Q t σ t, Q t ) P ( t t t t σ t, Q t ) P ( t t t t ) (7) λ t t where (7) follows because of the independentincrements nature of the Poisson process. In infinitesimal time (small ), from (5): P ( t+ t σ t, Q t ) ρ (Q t ) (8) Define π as the steady-state distribution on the Harris-recurrent process (Ỹi, Q i ) i n. hus it follows that the AEP holds: Marginal on : from (2) log f () a.s. λ( log λ) (9) Marginal on : via reversibility on the birth-death process, Burke s theorem [3], [] holds and so is also Poisson log f ( ) a.s. λ( log λ) (2) Joint on and : from (6), log f,q (, Q ) n log φ(ỹi, Q i ) a.s. E π log φ(ỹi, Q i ) (2) where (2) follows because of the SLLN for Markov chains applied to the Markov chain (Ỹi, Q i ). V. ACHIEVABILI FOR HE ANDEM QUEUE Now consider communication in tandem over two ESCs, of parameter µ and µ 2 (see Figure 3), with the same constraint (7). Sundaresan and Verdú studied this channel when µ µ 2 and simulated the mutual information rate corresponding to Poisson inputs, but were unable to demonstrate achievability [5, Sec IV.B]. Here, we demonstrate that the mutual information rate corresponding to Poisson inputs is indeed achievable, by again demonstrating the existence of an AEP using the LLN numbers for Markov chains applied to the joint queue states. Define Z and denote Z as the stochastic process on ( ) Z, F Z pertaining to the output of the second queue. Assume the initial states of both queues are and define: Q,t x t t, Q 2,t t Z t (22) Q k (Q k,t ) t [, ], k, 2 (23) From the total probability theorem, for x, the probability measure PZ ( x) on ( ) Z, F Z has Radon-Nikodym derivative with respect to P as: fz (z x) dp Z ( x) dp (z) fz,q 2, (z y, ) f,q, (y x, ) P (dy) µ z 2 e ν (y z) µ y e ν (x y) P (dy) (24) (µ µ 2 ) z µ y z e ν (y z)+ν (x y) P (dy) (25) ( ) with fz,q 2, (, ) f,q, (, ) given by (6) with parameter µ (µ 2 ) respectively. We state the following theorem:
ESC (µ ) ESC (µ 2 ) Q Q 2 Fig. 3. he tandem queue heorem 5.: he rate R I (; Z) pertaining to P ( ) on ( ), F as a Poisson process of rate λ < min(µ, µ 2 ) is achievable on the tandem queue. Proof: Define the abelian group G as the set of all functions g : [, ] Z such that g is right-continuous, with ( t ) t [, ] and g g (g t g t) t [, ] for any g, g G. Define the following two functions β : G R, β 2 : G R as β (g) µ g e ν (g), β 2 (g) e ν (g). (26) Note that G and so we can define P (G \ ) PZ (G \ x) to extend P ( ) and PZ ( ) to probability measures on (G, F G ). hus: fz (z x) (µ µ 2 ) z β (y z)β 2 (x y)p (dy) (27) G (µ µ 2 ) z β β 2 (x z) (28) (µ µ 2 ) z β β 2 (q + q 2 ) (29) where (27) follows from (25) and (26); (28) follows from the definition of convolution on abelian groups; and (29) follows from (22) and (23). So by defining, q i,k (q k,t ) t [(i ),i ], k, 2 (3) ( ) z i zt z (i ), (3) t [(i ),i ) for z Z, x, we have: fz (z x) (µ µ 2 ) z β (y z)β 2 (x y)p (dy) G (µ µ 2 ) z µ y z e ν (y z)+ν (x z) P G (µ µ 2 ) n z i, n µ ỹi, z i, G n exp ν (ỹ i z i ) + ν ( x i z i ) Z (dy) P (dy) (32) (µ µ ) z i, 2 β β 2 ( q,i + q 2,i ) (33) where (32) follows because of (8), (4), and (3); and (33) follows from (3). Let f (x) be the density of a Poisson process of rate λ on ( ), F. Note from Burke s theorem, asymptotically, Z is Poisson. Lastly, note that the queue states (Q,t, Q 2,t ) form a Harris-recurrent Markov process. hus, from (33) and the SLLN for Markov chains, the AEP holds: Marginal on : from (2) log f () a.s. λ( log λ) (34) Marginal on Z: via Burke s theorem holds and so Z is asymptotically Poisson: log fz (Z) a.s. λ( log λ) (35) Joint on and Z: Define π as the steady-state distribution on the Harris-recurrent Markov process ( Q,i, Q 2,i ) i n. From (33), log fz (Z ) n n z i, log(µ µ 2 ) log β β 2 ( Q,i + Q 2,i ) a.s. λ log(µ µ 2 ) E π log β β 2 ( Q,i + Q 2,i ) by the SLLN for Markov chains. hus, a random coding argument along with jointly typical decoding suffices to achieve the rate I (; Z). ACKNOWLEDGEMENS he author would like to thank Negar Kiyavash, Maxim Raginsky, and Vijay Subramanian for useful discussions. REFERENCES [] V. Anantharam and S. Verdú, Bits through queues, IEEE ransactions on Information heory, vol. 42, no., pp. 4 8, 996. [2] S. Verdú, A general formula for channel capacity, Information heory, IEEE ransactions on, vol. 4, no. 4, pp. 47 57, 994. [3] A. S. Bedekar and M. Azizoglu, he information-theoretic capacity of discrete-time queues, IEEE ransactions on Information heory, vol. 44, no. 2, pp. 446 46, 998. [4] B. Prabhakar and R. Gallager, Entropy and the timing capacity of discrete queues, IEEE ransactions on Information heory, vol. 49, no. 2, pp. 357 37, February 23. [5] R. Sundaresan and S. Verdú, Capacity of queues via point-process channels, IEEE ransactions on Information heory, vol. 52, no. 6, pp. 2697 279, June 26. [6] B. Prabhakar and N. Bambos, he entropy and delay of traffic processes in AM networks, in Proceedings of the Conference on Information Science and Systems (CISS), Baltimore, Maryland, 995, pp. 448 453. [7] S. Verdú, he exponential distribution in information theory, Problems of Information ransmission, vol. 32, no., pp. 86 95, Jan-Mar 996. [8] I. Rubin, Information rates and data-compression schemes for Poisson processes, IEEE ransactions on Information heory, vol. 2, no. 2, pp. 2 2, 974. [9]. P. Coleman, N. Kiyavash, and V. Subramanian, he rate-distortion function of a Poisson process with a queuing distortion measure, IEEE ransactions on Information heory, submitted May 28. [] J. McFadden, he entropy of a point process, SIAM Journal of Applied Mathematics, vol. 3, pp. 988 994, 965. [] P. Bremaud, Point Processes and Queues: Martingale Dynamics. New ork: Springer-Verlag, 98. [2]. M. Cover and J. homas, Elements of Information heory, 2nd ed. New ork: Wiley, 26. [3] R. Gallager, Discrete Stochastic Processes. Boston, MA: Kluwer, 996. [4] S. Meyn and R. weedie, Markov chains and stochastic stability. Springer-Verlag, 993. [5] R. Sundaresan and S. Verdú, Sequential decoding for the exponential server timing channel, IEEE ransactions on Information heory, vol. 46, no. 2, pp. 75 79, March 2.