Delay-Constrained Energy Efficient Data Transmission over a Wireless Fading Channel

Transcription

1 Delay-Constrained Energy Efficient Data Transmission over a Wireless Fading Channel Murtaza Zafer and Eytan Modiano Massachusetts Institute of Technology Cambridge, MA 239, USA {murtaza,modiano}@mit.edu Abstract We study optimal rate control for transmitting deadline-constrained data over a time-varying channel. Specifically, we consider a wireless transmitter where the channel gain varies stochastically over time and the packets in the queue have strict delay constraints. The transmitter can adapt the rate over time by varying the power and the goal is to obtain the rate-control policy that minimizes the epected energy ependiture while meeting the deadline constraints. We first consider the case of B bits of data that must be transmitted by a deadline T and using a novel continuous-time stochastic control formulation obtain the optimal policy. Based on a cumulative curves methodology and a decomposition approach, we then obtain the optimal policy when the queue has packets with variable deadline constraints. Finally, we present a heuristic policy for the case of arbitrary packet arrivals to the queue and compare its performance using simulation results with a non-adaptive scheme. Inde Terms Energy efficiency, Delay constraints, Wireless fading channel, Rate control, Quality of Service. I. INTRODUCTION Delay constraints and energy efficiency are important concerns in the design of modern wireless systems. Data services such as video and real-time multimedia streaming, high throughput file transfers and Voice-over-IP require strict delay constraints on data delivery. Similarly, in sensor networks time critical sensing applications impose deadline constraints within which the data must be transmitted back to a central processing entity. Energy consumption is also an important concern and minimizing this cost has numerous advantages in efficient battery utilization of mobile devices, increased lifetime of sensor and ad-hoc networks, and better utilization of limited energy sources in satellites. Furthermore, since transmission energy constitutes the bulk of the total energy ependiture, it is imperative to minimize this cost to achieve significant energy savings. Our focus in this work is to utilize dynamic transmission rate control to address the above concerns. Modern wireless devices are equipped with channel measurement and rate adaptation capabilities []. Channel measurement allows the transmitter-receiver pair to measure the fade state using a pre-determined pilot signal while rate control capability allows the transmitter to adjust the transmission rate over time. Such a control can be achieved in various ways that This work was supported by NSF ITR grant CCR-3254, by DARPA/AFOSR through the University of Illinois grant no. F , by NASA Space Communication Project grant number NAG and by ONR grant number N4664. include adjusting the power level, symbol rate, coding scheme, constellation size or any combination of these approaches; further, in some technologies the receiver can detect these changes directly from the received data without the need for an eplicit rate change control information [2]. The transmission rate can also be adapted very rapidly in time over millisecond duration time-slots []. These capabilities, thus, provide ample opportunity to utilize rate control algorithms to optimize system performance. The power-rate function defines the relationship that governs the amount of transmission power required to reliably transmit at a certain rate. Two fundamental aspects of this function, which are ehibited by most encoding/communication schemes and hence are common assumptions in the literature [4] [6], [9] [], [5], are as follows. First, for a fied bit error probability and channel state, the required transmission power is a conve function of the communication rate as shown in Figure (a). This implies, from Jensen s inequality, that transmitting data at low rates over a longer duration is more energy efficient as compared to high rate transmissions. Second, the wireless channel is time-varying which shifts the conve power-rate curves as a function of the channel state as shown in Figure (b). As good channel conditions require less transmission power, eploiting this variability over time by adapting the rate in response to the channel conditions leads to reduced energy cost. Thus, by adapting the transmission rate intelligently over time we can minimize the energy cost while also ensuring that the delay constraints are met. In this paper, we consider the following setup. The system model consists of a wireless transmitter with packets having strict delay constraints. The channel state varies stochastically over time and is modelled as a general Markov process. The transmitter can control the transmission rate over time and the epended power depends on both the chosen rate and the present channel condition. The objective is to obtain a ratecontrol policy that serves the packets arriving to the queue within the deadline constraints and minimizes the transmission energy ependiture. Towards this end, we first consider a simplified problem where the queue has only B bits of data that must be transmitted by deadline T. Using a novel continuoustime stochastic control formulation we obtain in eplicit/closed form the optimal transmission policy. We then consider a cumulative curves methodology and a novel decomposition approach to obtain the optimal policy with variable deadline

2 Power, P conve increasing rate, r Figure (a) Power, P Figure (b) conve increasing improving channel state rate, r Fig.. Transmission power as a function of the rate and the channel state; (a) fied channel state, (b) variable channel state. modulation bits/symbol SNR/symbol 2 PAM 4 QAM 6 QAM 64 QAM d 2.5 d 2.25 d d 2 SNR/symbol Tr. scheme g(r).43 r rate (bits/symbol) Fig. 2. Modulation scheme considered in [] as given in the table. The corresponding plot shows the least squares monomial fit,.43r 2.67, to the scaled piecewise linear power-rate curve. constraints. Finally, we present a heuristic policy for the case of arbitrary packet arrivals to the queue and compare its performance using simulation results with a non-adaptive scheme. Transmission power/rate control is an active area of research in communication networks in various different contets. Adaptive network control and scheduling has been studied in the contet of network stability [], [3], average throughput [2], [4], average delay [4], [5] and packet drop probability [6]. This literature considers average metrics that are measured over an infinite time horizon and hence do not directly apply for delay constrained/real-time data. Furthermore, for strict deadline constraints rate-adaptation simply based on steady state distributions does not suffice and one needs to take into account the system dynamics over time; thus, introducing new challenges and compleity into the problem. Recent work in this direction includes [5] []. The work in [5] studied offline formulations under complete knowledge of the future and devised heuristic policies using the offline solution. The authors in [6] studied several data transmission problems using Dynamic Programming (DP), however, the specific problem that we consider in this work becomes intractable using this methodology. This is due to the large state space in the DPformulation or the well-known curse of dimensionality. The works in [9], [] studied formulations for energy efficient data transmission over a static channel without fading. Similarly, our earlier work in [7] for static non-fading channels used a calculus approach to obtain minimum energy policies with general QoS constraints. The work in this paper is an etension of [8] where we considered a restricted scenario with only a single deadline constraint. II. SYSTEM MODEL We consider a continuous-time model of the system. Clearly, such a model is an approimation of the actual system but the assumption is justified since in practice, transmission rate can be adapted over a communication-slot duration which is very short, on the order of msec []. In contrast the packet delay requirements can be on the order of msec, thus justifying a continuous-time view of the system. A significant advantage of such a model is that it makes the problem mathematically tractable and yields simple solutions. The alternative discretetime dynamic programming setup is intractable and would only yield numerical solutions without much insights. The results obtained using the continuous-time model can then be applied to the discrete-time system in a very straightforward manner by simply evaluating the solution at discrete times as done for the simulation results in Section V-B. A. Transmission Model Let h t denote the channel gain, P (t) the transmitted signal power and P rcd (t) the received signal power at time t. We make the common assumption [4] [6], [9] [], [5] that the required received signal power for reliable communication, with a certain low bit-error probability, is conve in the rate; i.e. P rcd (t) = g(r(t)), where g(r) is a non-negative conve increasing function for r. Since the received signal power is given as P rcd (t) = h t 2 P (t), the required transmission power to achieve rate r(t) is given by, P (t) = g(r(t)) c(t) where c(t) = h t 2. The quantity c(t) is referred to as the channel state at time t. Its value at time t is assumed known either through prediction or direct channel measurement but evolves stochastically in the future. Some specific eamples of () can be found in [4], [5]. It is worth emphasizing that while we defined c(t) as h t 2 to motivate the relationship in (), more generally, c(t) could include other stochastic variations in the system and (uncontrollable) interference from other transmitter-receiver pairs, as long as the power-rate relationship is given by (). In this work, our primary focus will be on g(r) belonging to the class of Monomial functions, namely, g(r) = kr n, n >, k > (n, k R). While this assumption restricts the generality of the problem, it serves several purposes. First, mathematically it leads to simple closed-form optimal solutions that can be applied in practice. Second, most importantly, for most practical transmission schemes g( ) is described numerically and its eact analytical form is unknown. In such situations, one can obtain the best approimation of that function to the form kr n by choosing the appropriate k, n and then applying the results thus obtained. For eample, consider the modulation scheme considered in [] and reproduced here in Figure 2. The table gives the rate and the normalized signal power per symbol, where d is the minimum distance between signal points and the scheme is designed for error ()

3 queue B Fig. 3. server c(t) data in queue B (t) Schematic description of the system for the BT -problem. T time probabilities less than 6. The plot gives the least squares monomial fit to the transmission scheme and one can see from the plot that for this eample the monomial approimation is fairly close. Third, monomials form the first step towards studying etensions to polynomial functions which would then apply to a general g( ) function using the polynomial epansion. Under a more restrictive setting in Section III- C, we also study the class of Eponential functions, namely, g(r) = k(α r ), α >, k > (α, k R). Finally without loss of generality, throughout the paper we take k =, since any other value of k simply scales the energy cost without affecting the policy results. B. Channel Model We consider a general continuous-time discrete state space Markov model for the channel state process. Markov processes constitute a large class of stochastic processes that ehaustively model a wide set of fading scenarios and there is substantial literature on these models [7], [8] and their applications to communication networks [8], [9]. Denote the channel state process as C(t) and the state space as C. Let c C denote a particular channel state and {c(t), t } denote a sample path. Starting from state c, let J c be the set of all states ( c) to which the channel can transition when the state changes. Let λ c c denote the channel transition rate from state c to c, then, the sum transition rate at which the channel jumps out of state c is, λ c = c J c λ c c. Clearly, the epected time that C(t) spends in state c is /λ c and one can view λ c as the coherence time of the channel in state c. Now, define λ = sup c λ c and a random variable, Z(c), as, { Z(c) = c/c, with prob. λ c c /λ, c J c (2), with prob. λ c /λ With this definition, we obtain a compact and simple description of the process evolution as follows. Given a channel state c, there is an Eponentially distributed time duration with rate λ after which the channel state changes. The new state is a random variable which is given as C = Z(c)c. Clearly, from (2) the transition rate to state c J c is unchanged at λ c c, whereas with rate λ λ c there are indistinguishable selftransitions. This is a standard Uniformization technique and there is no process generality lost with the new description as it yields a stochastically identical scenario. Eample: Consider the standard Gilbert-Elliott channel model [8] that has two states b and g denoting the bad and the good channel conditions respectively. The two states correspond to a two level quantization of the channel gain. If the measured channel gain is below some value, the channel is labelled as bad and c(t) is assigned an average value c b, otherwise c(t) = c g for the good condition. Let the transition rate from the good to the bad state be λ gb and from the bad to the good state be λ bg. Let γ = c b /c g, and using the earlier notation we get, λ = ma(λ bg, λ gb ). For state c g, we obtain Z(c g ) as, { γ, with prob. λ gb /λ Z(c g ) = (3), with prob. λ gb /λ To obtain Z(c b ), replace γ with /γ and λ gb with λ bg in equation (3) above. III. BT - PROBLEM We begin first with the following problem. The queue has B units of data that must be transmitted by deadline T, with minimum energy over a time-varying channel. We refer to this as the BT -problem where the notation implies that the amount of data under consideration is B, and the deadline is T. The case with variable deadline constraints is treated in the net section. We now describe in detail the control formulation and the optimality conditions for the BT -problem. A. Optimal Control Formulation Consider the BT -problem and let (t) denote the amount of data left in the queue at time t. The system state can be described as (, c, t), where the notation means that at the present time t, (t) = and c(t) = c. Let r(, c, t) denote the chosen transmission rate for the corresponding system state (, c, t). Since the underlying process is Markov, it is sufficient to restrict attention to transmission policies that depend only on the present system state [23]. Clearly then, (, c, t) is a Markov process. The system is schematically depicted in Figure 3. Given a policy r(, c, t), the system evolves in time as a Piecewise-Deterministic-Process (PDP) as follows. It starts with () = B and c() = c. Until τ, where τ is the first time instant after t = at which the channel changes, the buffer is reduced at the rate r((t), c, t). Hence, over the interval [, τ ), (t) satisfies the ordinary differential equation, d(t) dt = r((t), c, t) (4) Equivalently, (t) = () t r((s), c, s)ds, t [, τ ]. Then, starting from the new state ((τ ), c, τ ), the above procedure repeats until t = T is reached. A transmission policy, r(, c, t), is admissible, if it satisfies the following, (a) r(, c, t) <, (non-negativity) (b) r(, c, t) =, if = (no data left to transmit) and, (c) (T ) =, a.s. (deadline constraint). An additional technical requirement is that r(, c, t) be continuous and locally lipschitz in (for > ) which ensures that (t) is the unique solution of (4).

4 Consider now an admissible transmission policy r( ) and define a cost-to-go function, J r (, c, t), as the epected energy cost starting at time t < T in state (, c, t). Then, J r (, c, t) = E [ T t g(r((s), c(s), s))ds c(s) where the term within the brackets is the total energy ependiture obtained as the integral of the power cost over time and the epectation is conditioned on the starting state (t) =, c(t) = c. Define a minimum cost function, J(, c, t), as the infimum of J r (, c, t) over the set of all admissible transmission policies. ] (5) J(, c, t) = inf r( ) J r(, c, t), r(, c, t) admissible (6) Now, stated concisely, the optimization problem is to compute the minimum cost function J(, c, t) and obtain the optimal policy r (, c, t) that achieves this minimum cost. B. Optimality Conditions A standard approach towards studying continuous time problems is to investigate their behavior over a small time interval. In the contet of the BT -problem, this methodology applies as follows. Suppose that the system is in state (, c, t). We first apply a transmission policy, r( ), in the small interval [t, t+h] and thereafter, starting from ((t + h), c(t + h), t+h) we assume that the optimal policy is followed. By assumption, the energy cost is optimal over [t + h, T ], hence, investigating the system over [t, t + h] would give conditions for the optimality of the chosen rate at time t. Since t is arbitrary, we obtain formal conditions for an optimal policy. Following the above approach, we now present the details of the analysis. Consider t [, T ) and a small interval [t, t+h], where t + h < T. From Bellman s principle [2] we get, { t+h J(, c, t) = min E g(r((s), c(s), s))ds r( ) t c(s) } +EJ( t+h, c t+h, t + h) (7) where t+h, c t+h is a short-hand notation for (t + h) and c(t + h) respectively. The epression within the minimization bracket in (7) denotes the total cost with policy r( ) being followed over [t, t + h] and the optimal policy thereafter. This cost must be greater than the cost of applying the optimal policy directly from the starting state (, c, t). Thus, we get, t+h J(, c, t) E g(r((s), c(s), s))ds t c(s) + E [J( t+h, c t+h, t + h)] (8) E[J( t+h,c t+h, t + h)] J(, c, t) +E t+h t g(r((s), c(s), s))ds (9) c(s) Divide (9) by h and take the limit h. In the limit we have, E ( ) t+h g(r(s,c s,s)) t c s ds g(r) () h c where r is the transmission rate at time t, i.e. r = r(, c, t). EJ( Define lim t+h,c t+h,t+h) J(,c,t) h h = A r J(, c, t), then in the limit, (9) simplifies to, A r J(, c, t) + g(r) () c The quantity A r J(, c, t) is called the differential generator of the Markov process ((t), c(t)) for policy r( ). Intuitively, it can be viewed as a natural generalization of the ordinary time derivative for a function that depends on a stochastic process. An elaborate discussion on this topic can be found in [2] [23]. For the process ((t), c(t)), using the time evolution in (4), the quantity A r J(, c, t) can be evaluated as [2], A r J(, c, t)= J r(, c, t) J t +λ(e z [J(, Z(c)c, t)] J(, c, t)) (2) where E z is the epectation with respect to the Z variable as defined in (2). Now, in the above steps from (8)-() if policy r( ) is replaced with the optimal policy r ( ) there is equality throughout and we get, A r J(, c, t) + c g(r ) = (3) Thus, for a given system state (, c, t), the optimal transmission rate r is that value of r that minimizes () and the minimum value of this epression equals zero. This gives, ] min r [, ) [ g(r) + A r J(, c, t) c = (4) Substituting A r J from (2), we get a partial differential equation (PDE) in J(, c, t) which is also referred as the Hamilton-Jacobi-Bellman (HJB) equation. This is the Optimality Equation for the BT -problem. min r [, ) { g(r) c + J(, c, t) t +λ(e z [J(, Z(c)c, t)] J(, c, t)) J(, c, t) r } = (5) The boundary conditions for the above PDE are, J(, c, t) =, and J(, c, T ) =, if >. The last condition follows due to the deadline constraint of T on the data. C. Optimal Transmission Policy We have, so far, presented general results on the optimality condition for the BT -problem. We, now, give specific analytical results for the optimal policy and discuss some of the insights that can be drawn from it. However, before proceeding further, a few additional notations regarding the channel process are required. Let there be total m channel states in the Markov model and denote the various states c C as c, c 2,..., c m. Given a channel state c i, the values taken by the random variable Z(c i ) (defined in (2)) are denoted as {z ij }, where z ij = c j /c i. The probability that Z(c i ) = z ij is denoted as p ij. Clearly, if there is no transition from state c i to c j, p ij =. Also, as pointed out earlier,

5 without loss of generality we take the multiplicative constant k = in the power-rate function. Theorem I: Consider the BT -problem with g(r) = r n, n >, n R and a Markov channel model. The optimal policy, r (, c, t), and the function, J(, c, t), are, r (, c i, t) =, i =,..., m (6) f i (T t) J(, c i n, t) = c i, i =,..., m (7) (f i (T t)) n The functions {f i (s)} m i= are the solution of the following ODE with the boundary conditions f i () =, f i () =, i2, f (s) = + λf (s) n λ n. f m(s) = + λf m(s) n λ n m k= m k= p k (f (s)) n (8) z k (f k (s)) n p mk (f m (s)) n (9) z mk (f k (s)) n Proof: It can be shown that the functions in (6)-(7) satisfy the optimality equation in (5). The proof is omitted for brevity and can be found in [3]. The results in the above theorem can be interpreted as follows. From (6), the optimal rate given amounts of data left, channel state c i and time t, is f i (T t), where the function f i (s) is associated with the channel state c i. The corresponding minimum epected cost starting from state (, c i, t) is n c i f i (T t). The boundary condition f n i () = is due to the deadline constraint, since at the deadline, (T t) = and we require J(, c i, T ) =, if. The system of ODE above can be easily solved offline numerically using standard techniques (eg. ODE solvers in MATLAB). Furthermore, this computation needs to be done only once before the system starts operating. In fact, {f i (s)} can be pre-determined and stored in a table in the transmitter memory. Once {f i (s)} are known, the closed form structure of the optimal policy in (6) warrants no further computation. At time t, the transmitter simply looks at the amount of data left in the queue,, the channel state, c i, and then using the appropriate f i ( ) function it computes the transmission rate f i (T t). The solution in (6) provides several interesting observations and insights as follows. At time t, the optimal rate depends on the channel state c i through the function f i (T t) and this rate is linear in with slope f i(t t). We can view the quantity f i(t t) as the urgency of transmission at time t under the channel state c i and with (T t) time left until the deadline. This view gives a nice separation form for the optimal rate: rate = amount of data left * urgency of transmission 2 For numerical evaluation of the ODE solution, the two boundary conditions can be combined by taking a small ɛ >, letting f i (ɛ) = ɛ, i and then using an initial-value ODE solver to obtain {f i (s)}, s ɛ. f b (T t), f g (T t) time, t f b (T t) f g (T t) Fig. 4. f b (T t) and f g(t t) plot for the bad and the good channel respectively. Other parameters include, g(r) = r 2, T =, λ = 5, γ =.3. Due to the boundary condition, as t approaches T, f i (T t) goes to zero; thus, as epected the urgency of transmission, f i (T t), increases as t approaches the deadline. Finally, one last observation is that if we set λ = (no channel variations), f i (T t) = T t, i and r (, c, t) = T t. Thus, with no channel variations the optimal policy is to transmit at a rate that just empties the buffer by the deadline. This observation is consistent with the earlier results in the literature for nonfading/time-invariant channels [5], [7], [9]. We refer to this policy as the Direct Drain (DD) policy. Numerical Eample: Consider the Gilbert-Elliott (GE) channel model with two states bad and good as described in Section II-B. Let g(r) = r 2 and for simplicity take λ bg = λ gb = λ. Denoting γ = c b /c g, we have, Z(c g ) = γ, w.p., and Z(c b ) = /γ, w.p.. Denoting f b (s), f g (s) as the respective functions in the bad and the good states, we have, T t f b(s) = + λf b (s) γλ(f b(s)) 2 f g (s) f g(s) = + λf g (s) λ(f g(s)) 2 γf b (s) (2) (2) Figure 4 plots these functions, evaluated using MATLAB, for T =, λ = 5, γ =.3. First, as epected f g (T t) f b (T t), t, which implies that given units of data in the f g (T t) buffer and time t, the rate is higher under the good state than the bad state. Second, f g (T t) T t f b (T t), where the function, T t, gives the rate, T t, corresponding to the direct drain (DD) policy. Thus, the optimal policy both spreads the data over time and adapts the rate in response to the time-varying channel condition and this adaptation is governed by the respective functions {f i ( )}. The following simulation results compare the performance of the optimal policy with the direct drain (DD) policy. For the simulations, we consider the GE channel model with c g =, c b = γ and take g(r) = r 2. We let, T = and partition the interval [, ] into slots of length dt = 3, thus, having, time slots. The transmission rate chosen in each slot is obtained by evaluating the respective policies at the time corresponding to the start of that slot. A channel sample path is simulated using a Bernoulli process, where in a slot the channel transitions with probability λdt and with probability λdt there is no transition. At each transition, the new state is

6 Epected energy cost DD Optimal 5 5 (a) Lambda Fig. 5. Epected energy cost DD Optimal..2.3 (b) Gamma Epected energy cost for the optimal and the DD policy. c = Z(c)c which for the GE model amounts to jumps between the two states. Epected energy cost is computed by taking an average over 4 sample paths. Figure 5(a) plots the energy costs of the two policies as λ is varied with γ =.3, B =. When λ is small the channel is essentially time-invariant over the deadline interval and the two policies are comparable. As λ increases, the optimal cost substantially decreases due to the channel adaptation. In Figure 5(b), γ is varied with λ = 5, B =. As γ decreases the good and bad channel quality differ significantly and the optimal rate adaptation leads to a much lower energy cost in terms of an order of magnitude as compared to the DD policy. Constant Drift Channel: Theorem I gives the optimal policy for a general Markov channel model. By considering a special structure on the channel model which we refer to as the Constant Drift channel, two specialized results can be obtained. First, we obtain the f( ) function in closed form for the Monomial class (g(r) = r n ), and second, we obtain the optimal policy for the Eponential class (g(r) = α r ). In the constant drift channel model, we assume that the epected value of the random variable /Z(c) is independent of the channel state, i.e. E[/Z(c)] = β, a constant. Thus, starting in state c, if c denotes the net transition state we have E [ ] [ ] c = E Z(c) c = β c. This means that if we look at the process /c(t), the epected value of the net state is a constant multiple of the present state. We refer to β as the drift parameter of the channel process. If β >, the process /c(t) has an upward drift; if β =, there is no drift and if β <, the drift is downwards. There are various situations where the above model is applicable over the time scale of the deadline interval. For eample, when a mobile device is moving in the direction of the base station, the channel has an epected drift towards improving conditions and viceversa. Similarly, in case of satellite channels, changing weather conditions such as cloud cover makes the channel drift towards worsening conditions and vice-versa. In cases when the time scale of these drift changes is longer than the packet deadlines, the constant drift channel serves as an appropriate model. The net theorem, Theorem II, gives the optimal policy result for the monomial class of functions while Theorem III gives the result for the eponential class. Theorem II: Consider the BT -problem with g(r) = r n, n >, n R and a constant drift channel with drift β. The optimal policy, r (, c, t), and the function, J(, c, t), are, r (, c, t) = J(, c, t) = f(t t) (22) n c(f(t t)) n (23) where f(t t) = (n ) λ(β ) λ(β ) ( ep( n (T t))). Proof: For the constant drift channel model, the functions f i (s) are the same for all the channel states. Denoting this common function as f(s) the ODE equation from Theorem I becomes f (s) = λ(β ) n f(s). Evaluating this for the boundary conditions in Theorem I gives the above result. The details of the proof are omitted for brevity and can be found in [3]. The closed-form epression of f( ) above provides an interesting intuitive observation related to the parameter β. Suppose that the present channel state is c, then for a fied r( the [ epected ] power ) cost for the net channel state is E g(r) Z(c)c = g(r) c β which is β times the present cost, g(r) c. This means that for higher values of parameter β, the channel on every transition drifts in an epected sense towards higher epected power cost or worsening conditions and vice-versa as β decreases. Hence, as epected, the urgency of transmission /f(t) is an increasing function with respect to β since for large β values it becomes more energy efficient to utilize the present channel conditions. Interestingly, when β =, the epected future power cost does not change and in this case the optimal policy reduces to the direct drain (DD) policy T t r (, c, t) =. Thus, we see that the direct drain policy is optimal both under no channel variations and under a constant drift channel model with β =. Theorem III: Consider the BT -problem with g(r) = α r, α > and a constant drift channel with drift β. The optimal policy, r (, c, t), is the following, Case : β, 2λ(β ) λ(β )(T t)2 ln α, < r (, c, t) = T t + λ(β )(T t), Case 2: < β <, {, < r (, c, t) = λ( β)(t t) T t, λ(β )(T t)2 (24) λ( β)(t t)2 λ( β)(t t)2 (25) Proof: To obtain the above functions we consider a discrete approimation of the time interval [, T ] with step size dt. Using discrete dynamic programming (DP), we obtain the optimal functions for the discrete system and then take the limit dt ; the limiting functions given above can be shown to satisfy the optimality equation in (5). The details are omitted for brevity and can be found in [3].

7 B Fig. 6. A(t) D(t) (a) T D min (t) time B M B M- B 2 B D(t) A(t) T T 2 T M- T M (b) D min (t) time Cumulative curves for (a) BT -problem, (b) Variable deadlines case. From above, we see that the optimal rate function has all the properties discussed earlier - it is monotonically increasing in, increasing as t approaches the deadline and also increasing in the drift parameter β. IV. VARIABLE DEADLINES SETUP In the last section, we dealt with a specific case of the energy minimization problem involving B bits of data and a single deadline T. We now etend the results to a more general setup where the data in the queue has variable deadlines. We adopt a cumulative curves methodology [2] and using a decomposition of the problem in terms of the BT -problem, obtain the optimal policy for this setup. As will be evident, the novel cumulative curves formulation provides a very appealing and simple visualization of the problem. A. Problem Setup Let us first define the following cumulative curves. Define the Arrival Curve, A(t), as the total number of bits that have arrived to the queue in time [, t]; the Departure Curve, D(t), as the total number of bits that have departed (served) in time interval [, t] and the Minimum Departure Curve, D min (t), as the minimum number of bits that must depart by time t to satisfy the deadline constraints. For eample, in the BT - problem case, we have A(t) = B, t [, T ] since the queue has B bits to begin with at time and no more data is added. We have D min (t) =, t [, T ), D min (T ) = B since until the deadline t < T there is no minimum data transmission requirement while at T the entire B bits must have been transmitted. Finally, the curve D(t) represents the data departure over time which depends on the chosen transmission policy. A schematic diagram of this is given in Figure 6(a). Consider now the variable deadlines problem. Here, the queue has M packets that are arranged and served in the earliest-deadline-first order. Let b j be the number of bits in the j th packet and T j be the deadline for this packet; assume < T < T 2 <... < T M. There are no new arrivals and the objective is to obtain a transmission policy that serves this data over the time-varying channel with minimum epected energy cost while meeting the deadline constraints. In terms of the cumulative curves, the setup can be visualized as depicted in Figure 6(b). Let B j = j l= b l; where B j is the cumulative amount of data of the first j packets. Then, A(t) = B M, t, since a total B M bits are in the queue at time and no more data is added. And, D min (t) is a piecewise-constant curve with jumps at times T j, i.e. at time T j, D min (T j ) = B j since the first B j bits must be transmitted by T j. Finally, we require that for admissibility a transmission policy must be such that the departure curve, D(t), satisfy D min (t) D(t) A(t); in other words, data must be served such that the cumulative amount lies above the minimum departure curve (to satisfy the deadline constraints) and below the arrival curve (to satisfy the causality constraints). B. Optimal Policy It is evident that a direct solution of the variable deadlines problem stated in the previous section is fairly difficult due to the compleity of the multiple deadline constraints involved. Interestingly, however, the cumulative curves formulation with its graphical visualization provides an intuitive and natural decomposition of the problem in terms of multiple BT -problems which then yields the optimal solution. A visual comparison of the two diagrams in Figure 6 suggests the following approach. First, we can visualize the deadline constraints in terms of the cumulative amounts as {B j T j } M j= constraints, that is, a total of B j bits must be transmitted by deadline T j (j =,..., M). Clearly, each B j T j constraint is like a BT -problem ecept that now there are multiple such constraints that all need to be satisfied. For every time t and channel state c, we know the optimal transmission rate to meet each of the B j T j constraint individually (assuming only this constraint eisted), thus, to meet all the constraints a natural solution is to simply choose the maimum value among the transmission rates evaluated for all of the B j T j constraints. More precisely, the transmission policy is described as follows. Let the system state be denoted as (D, c, t), where D is the cumulative data that has been transmitted by time t and c is the corresponding channel state. Using the optimal rate function in (6), the rate for an individual B j T j constraint for channel state c i B, is j D f i (T j t), since (B j D) is the amount of data left and (T j t) is the time left until the deadline T j. Let r (D, c, t) denote the transmission rate, then r ( ) is the maimum value among the rates for all B j T j constraints for which (B j D, T j t). r (D, c i, t) = ma j:(b j D,T j t) r j (B j D, c i, t) (26) = ma j:(b j D,T j t) B j D f i (T j t) (27) Clearly, by construction all the B j T j constraints are satisfied since at all times we choose the maimum rate among those needed to meet each of the remaining constraints. Hence, the policy in (26) is admissible and starting with D() = the departure curve obtained using (26) satisfies D min (t) D(t) A(t), t [, T M ]. Furthermore, the following theorem also shows that the above policy is in fact optimal.

8 Theorem IV: (Variable Deadlines Case) Consider the variable deadlines problem with g(r) = r n, n >, n R and the Markov channel model. The optimal policy, r (D, c, t) for D min (t) D A(t), t [, T M ) is given as, r (D, c i, t) = ma j:(b j D,T j t) B j D, i =,..., m (28) f i (T j t) The functions {f i (s)} m i= are the solution of the ODE system in (8)-(9) with the boundary conditions f i () =, f i () =, i Proof: The proof is based on a direct verification that the above functional form satisfies the HJB equation and the boundary conditions. It is omitted here for brevity and can be found in [3]. The optimal solution in (28) is based on the BT -solution, therefore clearly it inherits all the properties of that solution. As before the functions {f i (s)} m i= can be obtained numerically using a standard ODE solver. Furthermore, this computation needs to be done only once before the system starts operating and {f i (s)} can be pre-determined and stored in a table in the transmitter memory. Once {f i (s)} are known, the online computation is minimal. At time t, the transmitter simply looks at the cumulative amount of data transmitted D, the channel state, c i, and then using the appropriate f i ( ) function it computes the maimum among a set of values as given in (28). V. PACKET ARRIVALS WITH DEADLINES The optimal solution to the variable deadlines problem provides a simple heuristic way to etend the results to a more general setup involving packet arrivals to the queue. Consider now an arbitrary stream of packet arrivals to the queue with each packet having a deadline by which it must depart. Regardless of the underlying stochastic process generating the packets, we net present a heuristic energy-efficient policy based on the variable deadlines solution. We call it the BT- Adaptive (BTA) policy. Later, we present simulation results comparing the BTA policy with a non-adaptive scheme. A. BT-Adaptive (BTA) Policy Consider packet arrivals to the queue and assume that the arrivals occur at discrete times with each packet having a deadline associated with it. Clearly, at the instant immediately following a packet arrival the transmitter queue consists of, (a) earlier remaining packets with their deadlines and (b) the new packet with its own deadline. Re-arranging the data in the queue in the earliest-deadline-first order we can view the queue as consisting of some amount B M of data with variable deadlines, identical to the case considered in the last section. Neglecting the future arrivals and using (28), we have the optimal policy to empty the transmitter buffer. As this policy is followed, at the net packet arrival instance the above procedure is repeated by updating the data amount taking into account the new packet. Summarizing, the BTA policy is as follows: Epected energy cost BTA HLD (a) Packet arrival rate Energy cost BTA HLD 2 4 (b) Sample path inde Fig. 7. Energy cost comparison for Poisson arrival process for (a) different arrival rate, (b) different sample paths. Transmit the data in the queue with the rate as given in (28); at every packet arrival instant re-arrange the data in the earliest-deadline-first order to obtain a new set of B i T i values including the new packet and its deadline; re-initialize D to zero and follow (28) thereafter. Note that since this policy is not based on a specific arrival process, an interesting feature of it is that it is robust to changes in the arrival statistics and can accommodate multiple deadline classes of packet arrivals to the queue. B. Simulation Results In this section, we present simulation results to evaluate the performance of the BT-Adapative policy. For comparison purposes we consider a policy that can be easily implemented in practice and refer to it as the Head-of-Line Drain (HLD) policy. In HLD policy, the data in the queue is arranged in the earliest-deadline-first order and the packets are served in that order. At time t, let H t be the amount of data left in the head-of-the-line packet and T H be its deadline, then the rate chosen is r t = H t T H. Thus, the transmitter serves the first packet in queue at a rate to transmit it out by its deadline, then moves to the net packet in line and so on. At every packet arrival instant, the data in the queue is re-arranged in the earliestdeadline-first order and the above policy is repeated with the new packet taken into account. The setup is as follows. The queue has packet arrivals and each packet has a deadline associated with it. On each simulation run, the total time over which the packets arrive and the system is operated is taken as L = seconds. This interval [, ] is partitioned into, slots, thus each slot is of duration dt = msec. The channel model is the two state model, described in Section II-B, with the parameters, c g =, c b =.2, λ bg = λ gb = λ = 5. Thus, the average time spent in a state before the channel transitions is /5 seconds, or 2 msec. A channel sample path is simulated using a Bernoulli process where in a slot the channel transitions with probability λdt; otherwise there is no transition. For simplicity, the packet arrival and the channel state transitions occur only at the slot boundaries. For both the BTA and the HLD policies, the rate chosen in a slot is obtained by evaluating the respective

9 Epected energy cost BTA HLD Epected energy cost BTA HLD Deadline in msec Packet size Fig. 8. Energy cost versus packet deadline for Poisson arrival process. Fig. 9. Energy cost versus packet size for Uniform arrival process. policies at the time corresponding to the start of that slot. We take g(r) = r 2, with appropriate units that have been omitted. Energy cost per slot is r2 dt c and the total epected energy cost is obtained as an average of the total cost over sample runs. We first consider a Poisson packet arrival process with each packet having unit of data and a deadline of 2 msec. Figure 7(a) is a plot of the epected energy cost, plotted on a log scale, versus the packet arrival rate. Note that a packet arrival rate of implies that the average inter-arrival time of a packet is / sec. or msec. As is evident from the plot, the BTA policy has a much lower energy cost compared to the HLD policy and as the arrival rate increases the two costs are roughly an order of magnitude apart. This can be intuitively eplained as follows. When the arrival rate is low, most of the time the queue has at most a single packet. Hence, both policies choose a rate based on the head-of-line packet with the BTA policy also adapting the rate with the channel state. As the arrival rate increases and due to the bursty nature of the Poisson process, the queue tends to have more packets. The BTA policy then adapts based on the channel and the deadlines of all the packets in the queue, whereas, the HLD policy chooses a rate based solely on the head-of-line packet. The energy efficiency of the BTA policy is not just in an average sense but even on individual sample paths. This is shown in Figure 7(b) for 5 sample paths for arrival rate. In Figure 8, the packet arrival process is Poisson with rate but now the packet deadline is varied. Clearly, as seen in the figure, the energy cost decreases as the packet deadline increases since lower transmission rates are required to meet the deadlines. Also, as the deadline increases the energy cost difference between the BTA and the HLD policy increases. This is because with a larger delay constraint there is more room for the adaptive techniques employed in the BTA policy to have a larger effect. In Figure 9, we consider a Uniform packet arrival process where now the inter-arrival time between packets is uniformly distributed between 5 and 5 msec. The deadline for each packet is taken as 2 msec while the packet size is varied. First, as epected the energy cost for both the policies increases with packet size and second, as before the BTA policy has a much less energy cost compared with the HLD policy even when the arrival process is less bursty as compared to the Poisson process. VI. CONCLUSION We considered transmission of delay-constrained data over time-varying channels with the objective of minimizing the total transmission energy ependiture. We adopted a novel approach based on a continuous-time formulation and stochastic control theory to obtain optimal solutions for an otherwise difficult set of problems. We first considered the problem of transmitting B bits of data by deadline T and obtained the optimal rate adaptation policy. Using a cumulative curves methodology and a decomposition approach, we then obtained the optimal solution when the data in the queue has variable deadline constraints. Finally, we presented an energy-efficient transmission policy for arbitrary packet arrival process and compared its performance through simulations. We believe that the framework of this paper holds promise for various etensions addressing QoS-constrained data transmission in wireless systems. Some of the natural etensions include a network model with multiple transmitter-receiver pairs and multi-hop transmissions with end-to-end delay constraints. REFERENCES [] A. Jalali, R. Padovani and R. Pankaj, Data throughput of CDMA-HDR a high efficiency high data rate personal communication wireless system, IEEE Vehicular Technology Conf., vol. 3, 2. [2] L. Tsaur and D. Lee, Closed-loop architecture and protocols for rapid dynamic spreading gain adaptation in CDMA networks,infocom, 24. [3] M. Zafer and E. Modiano, Optimal rate control for delay-constrained data transmission over a wireless channel, In preparation for submission, [4] R. Berry and R. Gallager, Communication over fading channels with delay constraints, IEEE Tran. on Information Theory, May 22. [5] E. Uysal-Biyikoglu and A. El Gamal, On Adaptive transmission for energy-efficiency in wireless data networks, in IEEE Trans. Information Theory, December 24. [6] A. Fu, E. Modiano and J. Tsitsiklis, Optimal energy allocation for delay constrained data transmission over a time-varying channel, IEEE INFOCOM 23, vol. 2, pp 95-5, April 23. [7] M. Zafer and E. Modiano, A Calculus approach to minimum energy transmission policies with quality of service guarantees, IEEE INFOCOM 25, March 25. [8] M. Zafer and E. Modiano, Continuous-time optimal rate control for delay constrained data transmission, 43rd Allerton conf. on Communication, Control and Computing, Monticello, Sept. 25. [9] M. Khojastepour and A. Sabharwal, Delay-constrained scheduling: power efficiency, filter design and bounds, IEEE INFOCOM, March 24.

10 [] P. Nuggehalli, V. Srinivasan and R. Rao, Delay constrained energy efficient transmission strategies for wireless devices, IEEE INFOCOM 22, vol. 3, pp ,June 22. [] M. Neely, E. Modiano and C. Rohrs, Dynamic power allocation and routing for time varying wireless networks, IEEE INFOCOM, 23. [2] A. Goldsmith, The capacity of downlink fading channels with variable rate and power, IEEE Tran. Vehicular Technology, vol. 45, Oct [3] L. Tassiulas and A. Ephremides, Stability properties of constrained queueing systems and scheduling policies for maimum throughput in multihop radio networks, IEEE Tran. on Automatic Control, Dec [4] X. Liu, E. Chong and N. Shroff, A framework for opportunistic scheduling in wireless networks Computer Networks, vol. 4, 23. [5] B. Collins and R. Cruz, Transmission policies for time varying channels with average delay constraints, Proceedings, 999 Allerton conf. on Communication, Control and Computing, Monticello, IL, 999. [6] B. Ata, Dynamic power control in a wireless static channel subject to a quality of service constraint, Operations Research 53 (25), No 5, [7] F. Babich and G. Lombardi, A Markov model for the mobile propagation channel, IEEE Tran. on Vehicular Technology, vol. 49, no., pp , Jan. 2. [8] E. N. Gilbert, Capacity of burst-noise channel, Bell Syst. Tech. J.,vol. 39, no. 9, pp , Sept. 96. [9] A. Goldsmith and P. Varaiya, Capacity, mutual Information and coding for finite state markov channels, IEEE Trans. Information Theory, 996. [2] J. Le Boudec, P. Thiran, Network calculus, Springer-Verlag, 2. [2] W. Fleming and H. Soner, Controlled Markov Processes and Viscosity Solutions, Springer-Verlag, 993. [22] M. Davis, Markov Models and Optimization, Chapman and Hall, 993. [23] B. Oksendal, Stochastic Differential Equations, Springer, 5 th edn., 2.