Energy-Aware Wireless Scheduling with Near Optimal Backlog and Convergence Time Tradeoffs

Transcription

1 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 24, NO. 4, AUG. 2016, PP Energy-Aware Wireless Scheduling wih Near Opimal Backlog and Convergence Time Tradeoffs Michael J. Neely Universiy of Souhern California Absrac This paper considers a wireless link wih randomly arriving daa ha is queued and served over a ime-varying channel. I is known ha any algorihm ha comes wihin ɛ of he minimum average power required for queue sabiliy mus incur average queue size a leas Ω(log(1/ɛ. However, he opimal convergence ime is unknown. This paper develops a scheduling algorihm ha, for any ɛ > 0, achieves he opimal O(log(1/ɛ average queue size radeoff wih a convergence ime of O(log(1/ɛ/ɛ. An example sysem is presened for which all algorihms require convergence ime a leas Ω(1/ɛ, and so he proposed algorihm is wihin a logarihmic facor of he opimal convergence ime. The mehod uses he simple drif-plus-penaly echnique wih an improved convergence ime analysis. I. INTRODUCTION This paper considers power-aware scheduling in a wireless link wih a ime-varying channel and randomly arriving daa. Arriving daa is queued for evenual ransmission. The ransmission rae ou of he queue is deermined by he curren channel sae and he curren power allocaion decision. Specifically, he conroller can make an opporunisic scheduling decision by observing he channel before allocaing power. For a given ɛ > 0, he goal is o push average power o wihin ɛ of he minimum possible average power required for queue sabiliy while ensuring opimal queue size and convergence ime radeoffs. A major difficuly is ha he daa arrival rae and he channel probabiliies are unknown. Hence, he convergence ime of an algorihm includes he learning ime associaed wih esimaing probabiliy disribuions or sufficien saisics of hese disribuions. The opimal learning ime required o achieve he average power and backlog objecives, as well as he appropriae sufficien saisics o learn, are unknown. This open quesion is imporan because i deermines how fas an algorihm can adap o is environmen. A conribuion of he curren paper is he developmen of an algorihm ha, under suiable assumpions, provides an opimal powerbacklog radeoff while provably coming wihin a logarihmic facor of he opimal convergence ime. This is done via he exising drif-plus-penaly algorihm bu wih an improved convergence ime analysis. Work on opporunisic scheduling was pioneered by Tassiulas and Ephremides in 2], where he Lyapunov mehod and he max-weigh algorihms were inroduced for queue sabiliy. Relaed opporunisic scheduling work ha focuses on uiliy The auhor is wih he Elecrical Engineering deparmen a he Universiy of Souhern California, Los Angeles, CA. This maerial was presened in par a he IEEE INFOCOM conference, Hong Kong, ]. This work was suppored in par by he NSF Career gran CCF opimizaion is given in 3]4]5]6]7]8]9]10] using dual, primal-dual, and sochasic gradien mehods, and in 11] using index policies. The basic drif-plus-penaly algorihm of Lyapunov opimizaion can be viewed as a dual mehod, and is known o provide, for any ɛ > 0, an ɛ-approximaion o minimum average power wih a corresponding O(1/ɛ radeoff in average queue size 10]12]. This radeoff is no opimal. Work by Berry and Gallager in 13] shows ha, for queues wih sricly concave rae-power curves, any algorihm ha achieves an ɛ-approximaion mus incur average backlog of Ω( 1/ɛ, even if ha algorihm knows all sysem probabiliies. Work in 14] shows his radeoff is achievable (o wihin a logarihmic facor using an algorihm ha does no know he sysem probabiliies. The work 14] furher considers he excepional case when rae-power curves are piecewise linear. In ha case, an improved radeoff of O(log(1/ɛ is boh achievable and opimal. This is done using an exponenial Lyapunov funcion ogeher wih a drif-seering argumen. Work in 15]16] shows ha similar logarihmic radeoffs are possible via he basic drif-plus-penaly algorihm wih Lasin-Firs-Ou scheduling. Now consider he quesion of convergence ime, being he ime required for he average queue size and power guaranees o kick in. This convergence ime quesion is unique o problems of sochasic scheduling when sysem probabiliies are unknown. If probabiliies were known, he opimal fracions of ime for making cerain decisions could be compued offline (possibly via a very complex opimizaion, so ha sysem averages would kick in immediaely a ime 0. Thus, convergence ime in he conex of his paper should no be confused wih algorihmic complexiy for non-sochasic opimizaion problems. Unforunaely, prior work ha reas sochasic scheduling wih unknown probabiliies, including he basic drif-pluspenaly algorihm as well as exensions ha achieve square roo and logarihmic radeoffs, give only O(1/ɛ 2 convergence ime guaranees. Recen work in 17] reas convergence ime for a relaed problem of flow rae allocaion and concludes ha consrain violaions decay as c(ɛ/, where c(ɛ is a consan ha depends on ɛ and is he oal ime he algorihm has been in operaion. While 17] does no specify he size of c(ɛ, i can be shown ha c(ɛ = O(1/ɛ. Inuiively, his is because c(ɛ is relaed o an average queue size, which is O(1/ɛ. The ime needed o ensure consrain violaions are a mos ɛ is found by solving c(ɛ/ = ɛ. The simple answer is = O(1/ɛ 2, again exhibiing O(1/ɛ 2 convergence ime! This leads one o suspec ha O(1/ɛ 2 is opimal. However, a recen resul in 18] shows ha a echnique for Lagrange muliplier esimaion can, in some cases, reduce queue backlog o O(1/ɛ 2/3 and

2 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 24, NO. 4, AUG. 2016, PP convergence ime o O(1/ɛ 1+2/3. 1 This paper pushes he radeoff furher o achieve a convergence ime resul ha is near opimal. Specifically, under he same piecewise linear assumpion in 14], and for he special case of a sysem wih jus one queue, i is shown ha he exising drif-plus-penaly algorihm yields an ɛ-approximaion wih boh O(log(1/ɛ average queue size and O(log(1/ɛ/ɛ convergence ime. This is an encouraging resul ha shows learning imes for power-aware scheduling can be pushed much smaller han expeced. Furher, an example sysem is demonsraed for which every algorihm incurs convergence ime a leas Ω(1/ɛ, so ha he proposed algorihm is wihin a logarihmic facor of opimaliy. The echniques in his paper can likely be exended o more general muli-queue siuaions (see recen resuls in his direcion in 19]. The nex secion specifies he problem formulaion. Secion III shows a lower bound on convergence ime of Ω(1/ɛ. Secion IV develops an algorihm ha achieves his bound o wihin a logarihmic facor. II. SYSTEM MODEL Consider a wireless link wih randomly arriving raffic. The sysem operaes in sloed ime wih slos {0, 1, 2,...}. Daa arrives every slo and is queued for ransmission. Define: Q( = queue backlog on slo a( = new arrivals on slo µ( = service offered on slo The values of Q(, a(, µ( are nonnegaive and heir unis depend on he sysem of ineres. For example, hey can ake ineger unis of packes (assuming packes have fixed size, or real unis of bis. Assume he queue is iniially empy, so ha Q(0 = 0. The queue dynamics are: Q( + 1 = maxq( + a( µ(, 0] (1 Assume ha {a(} =0 is an independen and idenically disribued (i.i.d. sequence wih mean λ = E a(]. For simpliciy, assume he amoun of arrivals in one slo is bounded by a consan a max, so ha 0 a( a max for all slos. If he conroller decides o ransmi daa on slo, i uses one uni of power. Le p( {0, 1} be he power used on slo. The amoun of daa ha can be ransmied depends on he curren channel sae. Le ω( be he amoun of daa ha can be ransmied on slo if power is allocaed, so ha: µ( = p(ω( Assume ha ω( is i.i.d. over slos and akes values in a finie se Ω = {ω 0, ω 1, ω 2,..., ω M }, where ω 0 = 0 and ω i is a posiive real number for all i {1,..., M}. Assume hese values are ordered so ha: 0 = ω 0 < ω 1 < ω 2 < < ω M For each ω k Ω, define π(ω k = P rω( = ω k ]. 1 The work 18] shows he ransien ime for backlog o come close o a Lagrange muliplier vecor is O(1/ɛ 2/3. For ransiens o be amorized, he oal ime for averages o be wihin ɛ of opimaliy is O(1/ɛ 1+2/3. Every slo he sysem conroller observes ω( and hen chooses p( {0, 1}. The choice p( = 1 acivaes he link for ransmission of ω( unis of daa. Fewer han ω( unis are ransmied if Q( < µ( (see he queue equaion (1. The larges possible average ransmission rae is E ω(], which is achieved by using p( = 1 for all. I is assumed hroughou ha 0 λ E ω(]. A. Opimizaion goal For a real-valued random process b(τ ha evolves over slos τ {0, 1, 2,...}, define is ime average expecaion over > 0 slos as: b( = 1 1 E b(τ] (2 where = represens defined o be equal o. Wih his noaion, µ(, p(, Q( respecively denoe he ime average expeced ransmission rae, power, and queue size over he firs slos. The basic sochasic opimizaion problem of ineres is: Minimize: lim sup p( (3 Subjec o: lim inf µ( λ (4 p( {0, 1} {0, 1, 2,...} (5 The assumpion λ E ω(] ensures he above problem is always feasible, so ha i is possible o saisfy consrains (4- (5 using p( = 1 for all. Define p as he infimum average power for he above problem. An algorihm is said o produce an ɛ-approximaion a ime if, for a given ɛ 0: p( p + ɛ (6 λ µ( ɛ (7 Given ɛ > 0, an algorihm is said o have a convergence ime of T ɛ if i ensures (6-(7 hold for all T ɛ. An algorihm is said o produce an O(ɛ-approximaion if he ɛ symbols on he righ-hand-side of (6-(7 are replaced by some consan muliples of ɛ. Convergence imes o an O(ɛ approximaion shall also be denoed T ɛ o emphasize dependence on ɛ. Fix ɛ > 0. This paper shows ha a simple drif-plus-penaly algorihm ha akes ɛ as an inpu parameer (and ha has no knowledge of he arrival rae or channel probabiliies can ensure here is a ime T ɛ for which: The algorihm produces an O(ɛ-approximaion for all T ɛ. The algorihm ensures he following for all {0, 1, 2,...}: Q( O(log(1/ɛ (8 T ɛ = O(log(1/ɛ/ɛ. The average queue size bound (8 is known o be opimal, in he sense ha no algorihm can provide a sub-logarihmic guaranee 14]. The nex secion shows ha he convergence ime O(log(1/ɛ/ɛ is wihin a logarihmic facor of he opimal convergence ime.

3 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 24, NO. 4, AUG. 2016, PP B. Discussion of he ime average expecaion Can convergence be defined using he sample pah ime average, raher han is expecaion? This does no work because, over a finie horizon, i is impossible o ensure sample pahs of ransmission rae and power are boh close o heir ergodic opimums (consider he example when all channels are bad for he firs slos. Hence, his paper uses a ime average expecaion. This represens expeced behavior over he firs slos. Forunaely, he sample pah ends o say close o is expecaion, as demonsraed via simulaion in Secion VII and via a deviaion probabiliy calculaion in Secion V-H. III. A LOWER BOUND ON CONVERGENCE TIME A. Expecaions and randomizaion One ype of power allocaion policy is an ω-only policy ha, every slo, observes ω( and independenly chooses p( {0, 1} according o some saionary condiional probabiliies P rp( = 1 ω( = ω] ha are specified for all ω Ω. The resuling average power and ransmission rae is: E p(] = M k=0 π(ω kp rp( = 1 ω( = ω k ] E µ(] = M k=0 π(ω kω k P rp( = 1 ω( = ω k ] I is known ha he problem (3-(5 is solvable over he class of ω-only policies 10]. Specifically, if he arrival rae λ and he channel probabiliies π(ω k were known in advance, one could offline compue an ω-only policy o saisfy: E p(] = p (9 E µ(] = λ (10 This is a 0-approximaion for all 0. In paricular, such a randomized algorihm achieves a convergence ime of 0. However, such an algorihm would ypically incur infinie average queue size (since he service rae equals he arrival rae. Furher, i is no possible o implemen his algorihm wihou perfec knowledge of λ and π(ω k for all ω k Ω. This secion develops a convergence ime lower bound in he case when sysem probabiliies are unknown. A differen ype of lower bound is given in 17]. I shows he sample pah ime average of an ineger sequence ha approaches a non-ineger real number has error magniude ha decays like Ω(1/ (for example, he error migh be 1/ on odd slos and 1/ on even slos. This is a non-probabilisic resul ha holds regardless of wheher or no probabiliies are known. Of course, if probabiliies are known, one can design a randomized algorihm o have opimal expecaions every slo, so he expeced error is zero. This secion shows ha, if probabiliies are unknown, even he expeced error is necessarily Ω(1/. The proof is necessarily differen from 17] and has nohing o do wih averages of ineger sequences. B. Inuiion Suppose one emporarily allows for infinie average queue size. Consider he following hough experimen. Consider an algorihm ha does no know he sysem probabiliies and hence makes a single misake a ime 0, so ha: E p(0] = p + c where c > 0 is some consan gap away from he opimal average power p. However, suppose a genie gives he conroller perfec knowledge of he sysem probabiliies a ime 1, and hen for slos 1 he nework makes decisions o achieve he ideal averages (9-(10. The resuling ime average expeced power over he firs > 1 slos is: p( = p + c ( 1p + = p + c Thus, o reach an ɛ-approximaion, his genie-aided algorihm requires a convergence ime = c/ɛ = Θ(1/ɛ. C. An example wih Ω(1/ɛ convergence ime The above hough experimen does no prove an Ω(1/ɛ bound on convergence ime because i assumes he algorihm makes decisions according o (9-(10 for all slos 1, which may no be he opimal way o compensae for he misake on slo 0. This secion defines a simple sysem for which convergence ime is a leas Ω(1/ɛ under any algorihm. Consider a sysem wih deerminisic arrivals of 1 packe every slo (so λ = 1. There are hree possible channel saes ω( {1, 2, 3}, wih probabiliies: π(3 = y, π(2 = z, π(1 = 1 y z For each slo > 0, define he sysem hisory H( = {(a(0, ω(0, p(0,..., (a( 1, ω( 1, p( 1}. Define H(0 = 0. For each slo, a general algorihm has condiional probabiliies θ i ( defined for i {1, 2, 3} by: θ i ( = P rp( = 1 ω( = i, a(, H(] On a single slo, i is no difficul o show ha he minimum average power E p(] required o achieve a given average service rae µ = E µ(] is characerized by he following funcion h(µ: µ/3 if 0 µ 3y h(µ = y + (µ 3y/2 if 3y µ 3y + 2z µ 2y z if 3y + 2z µ 2y + z + 1 There are wo significan verex poins (µ, h(µ for his funcion. The firs is (3y, y, achieved by allocaing power if and only if ω( = 3. The second is (3y +2z, y +z, achieved by allocaing power if and only if ω( {2, 3}. Define R as he se of poins (µ, p ha lie on or above he curve h(µ: R = {(µ, p R 2 0 µ 2y + z + 1, h(µ p 1} The se R is convex. Under any algorihm one has: (E µ(τ], E p(τ] R τ {0, 1, 2,...} For a given > 1, he following wo vecors mus be in R: (µ 0, p 0 (µ 1, p 1 = (E µ(0], E p(0] 1 1 = (E µ(τ], E p(τ] 1 τ=1 Tha (µ 1, p 1 is in R follows because i is he average of poins in R, and R is convex. By definiion of (µ(, p(: (µ(, p( = 1 (µ 0, p (µ 1, p 1 (11

4 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 24, NO. 4, AUG. 2016, PP Power"p" 1" (μ 0,"p 0 "is"in"" his"region." (1/4,"1/8" 0" 0" Rae"μ" 1" 1.4" A" B" X" h(μ"curve" (1/2,"1/4" (1,"7/8" C" (5/4,"1" Fig. 1. The performance region for case 1. The line segmens beween (1/4, 1/8 and C and beween (1, 7/8 and B inersec a poin A. Fix a small value ɛ > 0. The algorihm mus ensure (µ(, p( is an ɛ-approximaion o he arge poin (1, h(1, so ha: µ( 1 ɛ, p( h(1 + ɛ The algorihm has no knowledge of he probabiliies y and z a ime 0, so θ 1 (0, θ 2 (0, θ 3 (0 are arbirary. Suppose a genie reveals y and z on slo 1, and he nework makes decisions on slos {1,..., 1} ha resul in a (µ 1, p 1 vecor ha opimally compensaes for any misake on slo 0. Thus, (µ 1, p 1 is assumed o be he vecor in R ha ensures (11 produces an ɛ-approximaion in he smalles ime. The following proof considers wo cases: The firs case assumes θ 2 (0 1/2, bu considers probabiliies y and z for which minimizing average power requires always ransmiing when ω( = 2. The second case assumes θ 2 (0 > 1/2, bu hen considers probabiliies y and z for which minimizing average power requires never ransmiing when ω( = 2. In boh cases, he nonlinear srucure of he h(µ curve prevens a fas recovery from he iniial misake. Case 1: Suppose θ 2 (0 1/2. Consider y = 0, z = 1/4. Then ω( {1, 2} for all, π(1 = 3/4, π(2 = 1/4, and ω( = 2 is he mos efficien sae. The h(µ curve is shown in Fig. 1. The minimum average power o suppor λ = 1 is h(1 = 3/4, and so he arge poin is X = (1, 3/4. The poin (µ 0, p 0 = (E µ(0], E p(0] is: ( θ2 (0 (µ 0, p 0 = 2 + 3θ 1(0, θ 2(0 + 3θ 1( The se of possible (µ 0, p 0 is formed by considering all θ 2 (0 0, 1/2], θ 1 (0 0, 1]. This se lies inside he lef (orange shaded region of Fig. 1. To see his, noe ha if θ 2 (0 is fixed a a cerain value, he resuling (µ 0, p 0 poin lies on a line segmen of slope 1 ha is formed by sweeping θ 1 (0 hrough he inerval 0, 1]. If θ 2 (0 = 1/2, ha line segmen is beween poins (1/4, 1/8 and (1, 7/8 in Fig. 1. If θ 2 (0 < 1/2 hen he line segmen is shifed o he lef. The small riangular (green shaded region in Fig. 1, wih one verex a poin A, is he arge region. The vecor (µ(, p( mus be in his region o be an ɛ- approximaion. The poin A is defined: A = X + ( ɛ, ɛ = (1 ɛ, 3/4 + ɛ I suffices o search for an opimal compensaion vecor (µ 1, p 1 on he curve (µ, h(µ. This is because he average power p 1 from a poin (µ 1, p 1 above he curve (µ, h(µ can be reduced, wihou affecing µ 1, by choosing a poin on he curve. By geomery, (µ 1, p 1 mus lie on he line segmen beween poins B and C in Fig. 1. Specifically, he poin C is defined as he poin where he line beween (1/4, 1/8 and poin A pierces he curve (µ, h(µ, and he poin B is defined similarly. To undersand his, noe ha if (µ 1, p 1 were on he (µ, h(µ curve bu no in beween poins B and C, i would be impossible for a convex combinaion of (µ 1, p 1 and (µ 0, p 0 o be in he arge region (which is required by (11. For example, suppose (µ 1, p 1 is on he line segmen beween poins C and (5/4, 1 (no including poin C iself. Define L as he infinie line beween (µ 1, p 1 and he poin (1/4, 1/8. The green arge riangle is sricly below line L, while he orange region conaining (µ 0, p 0 lies on or above line L, as does he line segmen beween (µ 0, p 0 and (µ 1, p 1. By basic algebra for inersecing lines we have: ( ɛ B = X 1 16ɛ, ɛ 1 16ɛ ( 11ɛ C = X ɛ, 11ɛ 1 16ɛ Observe ha: (µ(, p( X ɛ 2 (12 (µ 1, p 1 X O(ɛ (13 (µ 0, p 0 (µ 1, p 1 2/16 (14 where (12 follows by considering he maximum disance beween X and any poin in he arge region, (13 holds because any vecor on he line segmen beween B and C is O(ɛ disance away from X, and (14 holds because he disance beween any poin on he line segmen beween B and C and a poin in he lef (orange shaded region is a leas 2/16 (being he disance beween he wo parallel lines of slope 1. Saring from (12 one has: ɛ 2 (µ(, p( X = (1/(µ 0, p 0 + (1 1/(µ 1, p 1 X = (1/(µ 0, p 0 (µ 1, p 1 ] X (µ 1, p 1 ] (1/ (µ 0, p 0 (µ 1, p 1 X (µ 1, p 1 2/(16 O(ɛ where he firs equaliy holds by (11, he second-olas inequaliy uses he riangle inequaliy W Z W Z for any vecors W, Z, and he final inequaliy uses (13 and (14. So 2/(16 O(ɛ. I follows ha Ω(1/ɛ. Case 2: Suppose θ 2 (0 > 1/2. However, suppose y = z = 1/2. So ω( {2, 3}, π(2 = π(3 = 1/2, and

5 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 24, NO. 4, AUG. 2016, PP % Power%p% 0% 0% (.5,%.25% (μ 0,%p 0 %is%in%% his%region.% A% B% 1% X% C% Fig. 2. The performance region for case 2. (1.5,%.5% Rae%μ% (2.5,%1% (2,%.75% h(μ%curve% ω( = 2 is he leas efficien sae. The h(µ curve is shown in Fig. 2. Noe ha h(1 = 1/3, and so he arge poin is X = (1, 1/3. The poin A = (1 ɛ, 1/3 + ɛ is shown in Fig. 2. Poin A is one verex of he small riangular (green arge region ha defines all poins (µ(, p( ha are ɛ-approximaions. Because θ 2 (0 1/2, he poin (µ 0, p 0 lies somewhere in he (orange shaded region in Fig. 2. Indeed, if θ 2 (0 = 1/2, hen (µ 0, p 0 is on he line segmen beween poins (0.5, 0.25 and (2, I is above his line segmen if θ 2 (0 > 1/2. As before, he geomery of he problem ensures an opimal compensaion vecor (µ 1, p 1 lies somewhere on he line segmen of he h(µ curve beween poins B and C of Fig. 2. As before, i holds ha: and: B = X (O(ɛ, O(ɛ C = X + (O(ɛ, O(ɛ (µ(, p( X O(ɛ (µ 1, p 1 X O(ɛ (µ 0, p 0 (µ 1, p 1 Θ(1 As before, i follows ha Ω(1/ɛ. D. Discussion The above Ω(1/ɛ bound can be inerpreed as a Cramer- Rao ype resul for conrolled queues. Classical Cramer-Rao heory lower-bounds he error of any algorihm for esimaing a mean in a sysem wih unknown probabiliies. Similarly, he above resul lower-bounds he convergence ime for conrolling a queue wih unknown probabiliies. There are many ways of conrolling a queueing sysem. Some ways migh rely on esimaion of cerain quaniies of ineres. This approach is aken in he recen work 18], where a Lagrange muliplier esimae is used o improve convergence ime from O(1/ɛ 2 o O(1/ɛ 1+2/3. The resul in 18] holds for more general muliqueue sysems. The nex secion shows ha, in he special case of one link, convergence ime can be improved furher o wihin a logarihmic facor of he Ω(1/ɛ bound. IV. THE DYNAMIC ALGORITHM This secion shows ha a simple drif-plus-penaly algorihm achieves O(log(1/ɛ/ɛ convergence ime and O(log(1/ɛ average queue size. A. Problem srucure Wihou loss of generaliy, assume ha π(ω k > 0 for all k {1,..., M} (else, remove ω k from he se Ω. The value of π(ω 0 is possibly zero. For each real number µ in he inerval 0, E ω(]], define h(µ as he minimum average power required o achieve an average ransmission rae of µ. I is known ha p = h(λ. Furher, i is no difficul o show ha h(µ is non-decreasing, convex, and piecewise linear wih h(0 = 0 and h(e ω(] = 1 π(ω 0. The poin (0, 0 is a verex poin of he piecewise linear curve h(µ. There are M oher verex poins, achieved by he ω-only policies of he form: { 1 if ω( ωk p( = (15 0 oherwise for k {1,..., M}. This means ha a verex poin is achieved by only using channel saes ω( ha are on or above a cerain hreshold ω k. Lowering he hreshold value by selecing a smaller ω k allows for a larger E µ(] a he expense of someimes using less efficien channel saes. The proof ha his class of policies achieves he verex poins follows by a simple inerchange argumen ha is omied for breviy. For ease of noaion, define ω M+1= and µ M+1= 0. Le {µ 1, µ 2,..., µ M, µ M+1 } be he se of ransmission raes a which here are verex poins. Specifically, for k {1,..., M}, µ k corresponds o he hreshold ω k in he policy (15. Tha is: M µ k = ω i π(ω i (16 Noe ha: i=k 0 = µ M+1 < µ M < µ M 1 < < µ 1 = E ω(] I follows ha h(µ k is he corresponding average power for verex k, so ha (µ k, h(µ k is a verex poin of he curve h(µ: M h(µ k = π(ω i (17 i=k The numbers {µ 1, µ 2,..., µ M, µ M+1 } represen a se of measure 0 in he inerval 0, E ω(]]. I is assumed ha he arrival rae λ is a number in 0, E ω(]] ha lies sricly beween wo poins µ b+1 and µ b for some index b {1,..., M}. Tha is: µ b+1 < λ < µ b Thus, he poin (λ, h(λ can be achieved by imesharing beween he verex poins (µ b+1, h(µ b+1 and (µ b, h(µ b : λ = θµ b+1 + (1 θµ b (18 p = h(λ = θh(µ b+1 + (1 θh(µ b (19 for some probabiliy θ ha saisfies 0 < θ < 1. In paricular: θ = µ b λ µ b µ b+1

6 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 24, NO. 4, AUG. 2016, PP B. The drif-plus-penaly algorihm For each slo {0, 1, 2,...}, define L( = 1 2 Q(2 and ( = L( + 1 L(. Le V be a nonnegaive real number. The drif-plus-penaly algorihm from 10]12] makes a power allocaion decision ha, every slo, minimizes a bound on ( + V p(. The value V can be chosen as desired and affecs a performance radeoff. This echnique is known o yield average queue size of O(V wih deviaion from opimal average power no more han O(1/V 10]12]. This holds for general muli-queue neworks. By defining ɛ = 1/V, his produces an O(ɛ approximaion wih average queue size O(1/ɛ. Furher, i can be shown ha convergence ime is O(1/ɛ 2 (see Appendix D in 20]. In he conex of he simple one-queue sysem of he curren paper, he drif-plus-penaly algorihm reduces o he following: Every slo, observe Q( and ω( and choose p( {0, 1} o minimize: V p( Q(ω(p( Tha is, choose p( according o he following rule: { 1 if Q(ω( V p( = 0 oherwise (20 The curren paper shows ha, for his special case of a sysem wih only one queue, he above algorihm leads o an improved queue size and convergence ime radeoff. C. The induced Markov chain The drif-plus-penaly algorihm induces a Markov srucure on he sysem. The sysem sae is Q( and he sae space is he se of nonnegaive real numbers. Observe from (20 ha he drif-plus-penaly algorihm has he following behavior: Q( V/ω b+1, V/ω b = p( = 1 if and only if ω( ω b+1. In his case one has (from (16 and (17: E µ( Q(] = µ b+1, if Q( V/ω b+1, V/ω b E p( Q(] = h(µ b+1, if Q( V/ω b+1, V/ω b Q( V/ω b, V/ω b 1 = p( = 1 if and only if ω( ω b. In his case one has: E µ( Q(] = µ b, if Q( V/ω b, V/ω b 1 E p( Q(] = h(µ b, if Q( V/ω b, V/ω b 1 where V/0 is defined as (in he case ω b 1 = ω 0 = 0, and ω M+1 = so ha V/ω M+1 = 0. Now define inervals I (1, I (2, I (3, I (4 (see Fig. 3: I (1 = 0, V/ω b+1 I (2 = V/ω b+1, V/ω b I (3 = V/ω b, V/ω b 1 I (4 = V/ω b 1, If V/ω b+1 = 0 hen I (1 is defined as he empy se, and if V/ω b 1 = hen I (4 is defined as he empy se. The above 0 % Posi5ve%dri:% Nega5ve%dri:% I (1% I (2% I V/ω b+1% V/ω (3% I b% V/ω (4% b(1% Fig. 3. An illusraion of he four inervals I i for i {1, 2, 3, 4}. equaliies can be rewrien as: Q( % E µ( Q(] = µ b+1, if Q( I (2 (21 E p( Q(] = h(µ b+1, if Q( I (2 (22 E µ( Q(] = µ b, if Q( I (3 (23 E p( Q(] = h(µ b, if Q( I (3 (24 Recall ha under he drif-plus-penaly algorihm (20, if Q( I (2 hen he se of all ω( ha lead o a ransmission is equal o {ω Ω ω ω b+1 }. If Q( I (1, hen he se of all ω( ha lead o a ransmission depends on he paricular value of Q(. However, since inerval I (1 is o he lef of inerval I (2, he se of all ω( ha lead o a ransmission when Q( I (1 is always a subse of {ω Ω ω ω b+1 }. Similarly, since I (4 is o he righ of I (3, he se of all ω( ha lead o a ransmission when Q( I (4 is a superse of he se of all ω( ha lead o a ransmission when Q( I (3. Therefore, under he drif-plus-penaly algorihm one has: E µ( Q(] µ b+1, if Q( I (1 (25 E p( Q(] h(µ b+1, if Q( I (1 (26 E µ( Q(] µ b, if Q( I (4 (27 E p( Q(] h(µ b, if Q( I (4 (28 For each i {1, 2, 3, 4} define he indicaor funcion: { 1{Q( I (i 1 if Q( I (i } = 0 oherwise For each slo > 0 and each i {1, 2, 3, 4}, define 1 (i ( as he expeced fracion of ime ha Q( I (i : 1 (i ( = 1 1 ] E 1{Q( I (i } I follows ha (using (22, (24, (26: p( 1 (2 (h(µ b (3 (h(µ b +1 (1 (h(µ b (4 ( (29 where he final erm follows because p( 1 for all slos. Similarly (using (21, (23, (25: µ( 1 (2 (µ b (3 (µ b +1 (1 (µ b (4 (E ω(] (30 where he final erm follows because E µ( Q( I (4] E ω(]. Likewise (using (21, (23, (27: µ( 1 (2 (µ b (3 (µ b + 1 (4 (µ b (31 which holds because E µ( Q( I (1] 0.

7 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 24, NO. 4, AUG. 2016, PP In he nex secion i is shown ha: µ( is close o λ when is sufficienly large. 1 (1 ( and 1 (4 ( are close o 0 when and V are sufficienly large. 1 (2 ( and 1 (3 ( are close o θ and 1 θ, respecively, when and V are sufficienly large. p( is close o p when and V are sufficienly large. Furhermore, o address he issue of convergence ime, he noion of sufficienly large mus be made precise. A key sep is esablishing bounds on he average queue size. V. ANALYSIS A. The disance beween µ( and λ Recall ha ω M is he larges possible value of ω(. Assume ha V ωm 2. Lemma 1: If V ωm 2, hen under he drif-plus-penaly algorihm: a One has p( = µ( = 0 whenever Q( < ω M. b The queueing equaion (1 can be replaced by he following for all slos {0, 1, 2,...}: Q( + 1 = Q( + a( µ( Proof: Suppose V ωm 2. To prove (a, suppose ha Q( < ω M. Since ω( ω M for all, one has: Q(ω( Q(ω M < ωm 2 V and so he algorihm (20 chooses p( = 0, so ha µ( is also 0. This proves par (a. To prove (b, noe ha par (a implies Q( µ( for all slos. Indeed, his holds in he case Q( < ω M (since par (a ensures µ( = 0 in his case, and also holds in he case Q( ω M (since ω M µ( always. Thus: Q( + 1 = maxq( + a( µ(, 0] = Q( + a( µ( Lemma 2: If V ω 2 M and Q(0 = q 0 wih probabiliy 1 (for some consan q 0 0, hen for every slo > 0: µ( = λ E Q( q 0 ] / Proof: By Lemma 1 one has for all slos τ {0, 1, 2,...}: Q(τ + 1 Q(τ = a(τ µ(τ Summing he above over τ {0, 1, 2,..., 1} and dividing by gives: Q( q 0 = 1 1 a(τ 1 1 µ(τ (32 Taking expecaions proves he resul. The above lemma implies ha if V ωm 2, hen µ( converges o λ whenever E Q( q 0 ] / converges o 0. B. The disance beween 1 (2 ( and θ The following lemma shows ha if 1 (1 (, 1 (4 (, and E Q( q 0 ] / are close o 0, hen 1 (2 ( is close o θ. Lemma 3: If V ω 2 M and Q(0 = q 0 wih probabiliy 1 (for some consan q 0 0, hen for all slos > 0: θ µ b1 (1 ( ψ(] µ b µ b+1 1 (2 ( where ψ( is defined: ψ( =E Q( q 0 ] / Proof: Fix > 0. Lemma 2 implies: λ = µ( + ψ( θ + 1(4 (E ω(] + ψ( µ b µ b+1 1 (2 (µ b (3 (µ b + 1 (4 (µ b + ψ( = 1 (2 (µ b+1 + (1 1 (2 (µ b 1 (1 (µ b + ψ( where he firs inequaliy holds by (31. Subsiuing he ideniy for λ given in (18 ino he above inequaliy gives: θµ b+1 + (1 θµ b 1 (2 (µ b+1 + (1 1 (2 (µ b 1 (1 (µ b + ψ( Rearranging erms proves ha: θ µ b1 (1 ( ψ(] µ b µ b+1 1 (2 ( To prove he second inequaliy, noe ha: λ = µ( + ψ( (33 1 (2 (µ b (3 (µ b + 1 (1 (µ b+1 +1 (4 (E ω(] + ψ( (34 = 1 (2 (µ b+1 + (1 1 (2 (µ b 1 (1 (µ b 1 (4 (µ b + 1 (1 (µ b+1 +1 (4 (E ω(] + ψ( 1 (2 (µ b+1 + (1 1 (2 (µ b +1 (4 (E ω(] + ψ( (35 where (33 holds by Lemma 2, (34 holds by (30, and (35 holds because µ b+1 < µ b. Subsiuing he ideniy for λ given in (18 gives: θµ b+1 + (1 θµ b 1 (2 (µ b+1 + (1 1 (2 (µ b Rearranging erms proves he resul. C. Posiive and negaive drif +1 (4 (E ω(] + ψ( Define E Q( + 1 Q( Q(] as he condiional drif. Assume ha V ωm 2, so ha Lemma 1 implies Q( + 1 Q( = a( µ( for all slos. Thus: E Q( + 1 Q( Q(] = E a( µ( Q(] = λ E µ( Q(]

8 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 24, NO. 4, AUG. 2016, PP where he final equaliy follows because a( is independen of Q(. From (21 and (25 one has for all slos : E µ( Q(] µ b+1, if Q( < V/ω b Likewise, from (23 and (27 one has: E µ( Q(] µ b, if Q( V/ω b Define posiive consans β L and β R (associaed wih drif when Q( is o he Lef and Righ of he hreshold V/ω b by: I follows ha: β L = λ µ b+1, β R = µ b λ E Q( + 1 Q( Q(] β L if Q( < V/ω b (36 E Q( + 1 Q( Q(] β R if Q( V/ω b (37 Thus, he sysem has posiive drif if Q( < V/ω b, and negaive drif oherwise (see Fig. 3. I is remarkable ha he probabiliy-unaware drif-plus-penaly algorihm yields a drif picure of Fig. 3 ha is qualiaively similar o he algorihm of 13] ha is designed offline using sysem probabiliies. D. A basic drif lemma Consider a real-valued random process Z( over slos {0, 1, 2,...}. The following drif lemma is similar in spiri o resuls in 21]16], bu focuses on a finie ime horizon wih an arbirary iniial condiion Z(0 = z 0 (raher han on seady sae, and on expecaions a a given ime (raher han ime averages. These disincions are crucial o convergence ime analysis. The lemma will be applied using Z( = Q( for bounds on average queue size and on 1 (4 (. I will hen be applied using Z( = V/ω b Q( o bound 1 (1 (. Assume here is a consan δ max > 0 such ha wih probabiliy 1: Z( + 1 Z( δ max {0, 1, 2,...} (38 Suppose here are consans θ R and β > 0 such ha: { δmax if Z( < θ E Z( + 1 Z( Z(] (39 β if Z( θ Noe ha if (38 holds hen (39 auomaically holds for he special case Z( < θ. Thus, he negaive drif case Z( θ is he imporan case for condiion (39. Furher, if (38-(39 boh hold, hen he consan β necessarily saisfies: 0 < β δ max Lemma 4: Suppose Z( is a random process ha saisfies (38-(39 for given consans θ, δ max, β (wih θ R and 0 < β δ max. Suppose Z(0 = z 0 for some z 0 R. Then for every slo 0 he following holds: E e rz(] D + (e rz0 D ρ (40 where consans r, ρ, D are defined: r = β δ 2 max + δ max β/3 (41 ρ = 1 rβ/2 (42 D = ρe rθ (erδmax (43 Noe ha he propery 0 < β δ max can be used o show ha 0 < ρ < 1. Proof: (Lemma 4 The proof is by inducion. The inequaliy (40 rivially holds for = 0. Suppose (40 holds a some slo 0. The goal is o show ha i also holds on slo + 1. Le r be a posiive number ha saisfies 0 < rδ max < 3. I is known from resuls in 21] ha for any real number x ha saisfies x δ max : e rx 1 + rx + (rδ max 2 2(1 rδ max /3 (44 Define δ( = Z( + 1 Z( and noe ha δ( δ max for all. Then: e rz(+1 = e rz( e rδ( e rz( 1 + rδ( + (rδ max 2 ] (45 2(1 rδ max /3 where he final inequaliy holds by (44. Choose r such ha: (rδ max 2 2(1 rδ max /3 = rβ 2 (46 I is no difficul o show ha he value of r given in (41 simulaneously saisfies (46 and 0 < rδ max < 3. For his value of r, subsiuing (46 ino (45 gives: e rz(+1 e rz( 1 + rδ( + rβ ] (47 2 Now consider he following wo cases: Case 1: Suppose Z( θ. Taking condiional expecaions of (47 gives: ] E e rz(+1 Z( E e rz( 1 + rδ( + rβ2 ] ] Z( e rz( 1 rβ + rβ 2 ] (48 = e rz( ρ where (48 follows by (39, and he final equaliy holds by definiion of ρ in (42. Case 2: Suppose Z( < θ. Then: E e rz(+1 Z( ] = E Puing hese wo cases ogeher gives: E e rz(+1] e rz( e rδ( Z( e rz( e rδmax ] ρe e rz( Z( θ P rz( θ] ] +e rδmax E e rz( Z( < θ P rz( < θ] e rz(] = ρe ] +(e rδmax ρe e rz( Z( < θ P rz( < θ] ρe e rz(] + (e rδmax ρe rθ where he final inequaliy uses he fac ha e rδmax > 1 > ρ. By he inducion assumpion i is known ha (40 holds on ]

9 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 24, NO. 4, AUG. 2016, PP slo. Subsiuing (40 ino he righ-hand-side of he above inequaliy gives: E e rz(+1] ρ D + (e rz0 D ρ ] +(e rδmax ρe rθ = D + (e rz0 D ρ +1 where he final equaliy holds by he definiion of D in (43. This complees he inducion sep. Le 1{Z(τ θ + c} be an indicaor funcion ha is 1 if Z(τ θ + c, and 0 else. The nex corollary shows ha he expeced fracion of ime ha his indicaor is 1 decays exponenially in c. Corollary 1: If he assumpions of Lemma 4 hold, hen for any c > 0 and any slos T and ha saisfy 0 T < : 1 1 E 1{Z(τ θ + c}] (erδmax ρe rc T + + er(z0 c θ ρ T ] ( (49 where r and ρ are defined in (41-(42. Furher, if z 0 θ hen for any > 0: 1 1 E 1{Z(τ θ + c}] e rc (e rδmax ρ + 1/ ( (50 The inuiion behind he righ-hand-side of (49 is ha he firs erm represens a seady sae bound as, which decays like e rc. The las wo erms (in brackes are due o he ransien effec of he iniial condiion z 0. This ransien can be significan when z 0 > θ. In ha case, e r(z0 c θ migh be large, and a ime T is required o shrink his erm by muliplicaion wih he facor ρ T. Proof: (Corollary 1 One has for > T : 1 E 1{Z(τ θ + c}] T + 1 τ=t However, for every slo τ 0 one has: e rz(τ e r(θ+c 1{Z(τ θ + c} Taking expecaions of boh sides gives: E 1{Z(τ θ + c}] E e rz(τ] e r(θ+c E 1{Z(τ θ + c}] Rearranging he above shows ha for every slo τ 0: E 1{Z(τ θ + c}] e r(θ+c E e rz(τ] e r(θ+c D + (e rz0 Dρ τ ] (51 where he final inequaliy uses (40. Subsiuing he above inequaliy ino he righ-hand-side of (51 gives: 1 E 1{Z(τ θ + c}] 1 T + e r(θ+c D + (e rz0 Dρ τ ] τ=t = T + e r(θ+c ( T D + (e rz0 Dρ T (1 ] ρ T ( T + e r(θ+c D + erz0 ρ T ] ( Dividing by and subsiuing he definiion of D proves (49. Inequaliy (50 follows immediaely from (49 by choosing T = 0. E. Bounding E Q(] and 1 (4 ( Le Q( be he backlog process under he drif-pluspenaly algorihm. Assume ha V ωm 2 and he iniial condiion is Q(0 = q 0 for some consan q 0. Define δ max= maxω M, a max ] as he larges possible change in Q( over one slo, so ha: Q( + 1 Q( δ max {0, 1, 2,...} From (37 i holds ha: { δmax if Q( < V/ω E Q( + 1 Q( Q(] b β R if Q( V/ω b I follows ha he process Q( saisfies he condiions (38- (39 required for Lemma 4. Specifically, define Z( = Q(, z 0 = q 0, θ = V/ω b, β = β R. Lemma 5: If 0 q 0 V/ω b and V ωm 2, hen for all slos 0 one has: E Q(] V + 1 ( log 1 + er Rδ max ρ R = O(V ω b r R R where consans r R and ρ R are defined: r R = β R δ 2 max + δ max β R /3 (52 ρ R = 1 r R β R /2 (53 The lemma provides a bound on E Q(] ha does no depend on. The bound holds whenever he iniial condiion saisfies 0 q 0 V/ω b. Typically, he iniial condiion is q 0 = 0. However, a place-holder echnique in Secion VI requires a nonzero iniial condiion ha sill saisfies he desired inequaliy 0 q 0 V/ω b. Proof: For ease of noaion, le r and ρ respecively denoe r R and ρ R given in (52 and (53. Define θ = V/ω b and β = β R. By (40 one has for all 0 (using Z(0 = Q(0 = q 0 : E e rq(] D + (e rq0 Dρ D + e rv/ω b (54 where D is given in (43, and where he final inequaliy uses Dρ 0 and q 0 V/ω b. Using Jensen s inequaliy gives: e req(] D + e rv/ω b

10 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 24, NO. 4, AUG. 2016, PP Taking a log of boh sides and dividing by r gives: E Q(] log(d + erv/ω b r = 1 r log ( e rv/ω b + (erδmax ρe rv/ω b = V ω b + 1 r log ( 1 + erδmax ρ Lemma 6: If 0 q 0 V/ω b and V ωm 2, hen for all slos > 0: 1 (4 ( O(e r RV ( 1 ω b 1 1 ω b where r R is given by (52. Proof: For ease of noaion, his proof uses r o denoe r R. If he inerval I (4 does no exis hen 1 (4 ( = 0 and he resul is rivial. Now suppose inerval I (4 exiss (so ha he inerval I (3 is no he final inerval in Fig. 3. Define θ = V/ω b, c = V (1/ω b 1 1/ω b, β = β R, ρ = 1 rβ R /2. Then 1{Q(τ θ + c} = 1 if and only if Q(τ V/ω b 1, which holds if and only if Q(τ I 4. Thus, for all slos > 0: 1 (4 ( = 1 1 E 1{Q(τ θ + c}] e rc (e rδmax ρ + 1/ = e rv ( 1 ω b 1 1 ω b (e rδ max ρ + 1/ (55 where (55 holds by (50 (which applies since z 0 = q 0 θ. The righ-hand-side of he above inequaliy is indeed of he form O(e rv ( 1 ω 1 b 1 ω b. F. Bounding 1 (1 ( One can similarly prove a bound on 1 (1 (. The inuiion is ha he posiive drif in region I (2 of Fig. 3, ogeher wih he fac ha he size of inerval I (2 is Θ(V, makes he fracion of ime he queue is o he lef of V/ω b decay exponenially as we move furher lef. The resul is given below. Recall ha Q(0 = q 0 for some consan q 0 0. Lemma 7: If q 0 0 and V ωm 2, hen for all slos > 0 one has: 1 (1 ( O(V / + O(e r LV ( 1 ω b 1 ω b+1 where r L is defined: r L= δmax 2 + δ max β L /3 Inuiively, he firs erm in he above lemma (ha is, he O(V / erm bounds he conribuion from he ransien ime saring from he iniial sae Q(0 = q 0 and ending when he hreshold V/ω b is crossed. The second erm represens a seady sae probabiliy assuming an iniial condiion V/ω b. The proof defines a new process Z( = V/ω b Q(. I hen applies inequaliy (49 of Corollary 1, wih a suiably large ime T > 0, o handle he iniial condiion z 0 = V/ω b q 0. β L Proof: (Lemma 7 Define Z( = V/ω b Q( and noe ha Z( + 1 Z( δ max sill holds. Furher, from (36 i holds: { δmax if Z( 0 E Z( + 1 Z( Z(] β L if Z( > 0 Now define θ as any posiive value. I follows ha: { δmax if Z( < θ E Z( + 1 Z( Z(] β L if Z( θ Thus, he condiions (38-(39 hold for his Z( process, wih iniial condiion z 0 = V/ω b q 0. Therefore, Corollary 1 can be applied. For ease of noaion le r represen r L, le β represen β L, and le ρ represen ρ L, where ρ L = 1 r L β L /2. Define c = V/ω b V/ω b+1. From (49 of Corollary 1, he following holds for all slos T, such ha 0 T < : 1 1 E 1{Z(τ θ + c}] (erδmax ρe rc T + + er(z0 c θ ρ T ] ( This holds for all θ > 0. Taking a limi as θ 0 + gives: 1 1 E 1{Z(τ > c}] (erδmax ρe rc T + + er(z0 c ρ T ] ( Noice ha he even 1{Z(τ > c} is equivalen o he even {Q(τ < V/ω b+1 }, which is he same as he even Q(τ I 1 (see Fig. 3. Thus, he lef-hand-side of he above inequaliy is he same as 1 (1 (. Hence: 1 (e rc T ( (erδmax + + er(z0 c ρ T ] ( ρe rc T (erδmax + + ρ T ] erv/ωb+1 ( where he final inequaliy uses he fac ha z 0 V/ω b. By definiion of c, he firs erm on he righ-hand-side is O(e rv ( 1 ω 1 b ω b+1. I remains o choose a value T > 0 for which he remaining wo erms (in brackes are O(V /. To his end, define x=r/(ω b+1 log(1/ρ. Choose T as he smalles ineger ha is greaer han or equal o xv. Then T = O(V and: T e rv/ω b+1 ρ T ( = O(V / erv/ω b+1 ρ xv ( 1 ( O(V /

11 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 24, NO. 4, AUG. 2016, PP G. Opimal backlog and near-opimal convergence ime Define: γ = min ( 1 r R 1 ( 1, r L 1 ] ω b 1 ω b ω b ω b+1 Resuls of Lemmas 5-7 imply ha if he drif-plus-penaly algorihm (20 is used wih V ωm 2, and if he iniial queue sae saisfies 0 q 0 V/ω b, hen for all > 0: Q( O(V (56 E Q(] / O(V / (57 1 (4 ( O(e γv (58 1 (1 ( O(e γv + O(V / (59 Indeed, (56-(57 follow from Lemma 5, while (58 and (59 follow from Lemmas 6 and 7, respecively. Fix ɛ > 0 and define: V = max(1/γ log(1/ɛ, ω 2 M ] (60 T ɛ = log(1/ɛ/ɛ Inequaliies (56-(59 can be used o easily derive he following facs: 2 Fac 1: For all slos > 0 one has Q( O(log(1/ɛ. Fac 2: For all slos > T ɛ one has E Q(] / O(ɛ. Fac 3: For all slos > 0 one has 1 (4 ( O(ɛ. Fac 4: For all slos > T ɛ one has 1 (1 ( O(ɛ. Fac 2 and Lemma 2 ensure ha for > T ɛ : µ( λ O(ɛ (61 Facs 2, 3, 4 and Lemma 3 ensure ha for > T ɛ : 1 (2 ( θ O(ɛ, 1 (3 ( (1 θ O(ɛ Subsiuing he above ino (29 proves ha for > T ɛ : p( θh(µ b+1 + (1 θh(µ b + O(ɛ = p + O(ɛ (62 The guaranees (61 and (62 show ha he drif-plus-penaly algorihm gives an O(ɛ-approximaion wih convergence ime T ɛ = O(log(1/ɛ/ɛ. This is wihin a facor log(1/ɛ of he convergence ime lower bound given in Secion III. Hence, he algorihm has near-opimal convergence ime. In 14] i is shown ha, under mild sysem assumpions, any algorihm ha yields an O(ɛ-approximaion mus have average queue size of Q( Ω(log(1/ɛ. Fac 1 shows he drif-plus-penaly algorihm mees his bound wih equaliy. Hence, no only does i provide near opimal convergence ime, i provides an opimal average queue size radeoff. 2 For example, Fac 1 follows from (56 and he fac ha V = O(log(1/ɛ, Fac 2 follows from (57 and he fac ha V/T ɛ O(ɛ, and so on. H. Sample pah consrain violaion probabiliy Does he sample pah ransmission rae usually have behavior similar o is expecaion? Recall from (32 ha if Q(0 = 0, he gap beween 1 1 a(τ and 1 1 µ(τ is Q(/. This gap is probabilisically bounded as follows: P rq(/ > ɛ] = P re rq( > e rɛ ] E e rq(] e rɛ (63 (D + e rv/ω b e rɛ (64 where (63 holds by he Markov inequaliy, and (64 holds from he momen generaing funcion bound (54. Recall ha D + e rv/ω b is polynomial in he 1/ɛ value. 3 Thus, here is a consan c > 0 such ha he righ-hand-side of (64 is O(ɛ whenever c log(1/ɛ/ɛ. A. Place-holders VI. PRACTICAL IMPROVEMENTS The srucure of his problem admis a pracical improvemen in queue size via he place-holder echnique of 10]. This does no change he O(log(1/ɛ average queue size radeoff wih ɛ, bu can reduce he coefficien ha muliplies he log(1/ɛ erm. Assume ha V 0 and define he following nonnegaive parameer: ] V q place= max ω M, 0 (65 ω M The echnique uses a nonzero iniial condiion Q(0 = q place, where he iniial backlog q place is fake daa, also called placeholder backlog. Noe ha q place > 0 if and only if V > ωm 2. The following lemma refines Lemma 1 and shows ha his place-holder backlog is never ransmied. Hence, i acs only o shif he queue size up o a value required o make desirable power allocaion decisions via (20. Lemma 8: If V ωm 2 and Q(0 = q place, hen he drif-plus-penaly algorihm (20 chooses p( = µ( = 0 whenever Q( < V/ω M. Thus, Q( q place for all. Proof: The proof is similar o ha of Lemma 1 and is omied for breviy. Consequenly, a every slo he queue can be decomposed as Q( = q place + Q real (, where Q real ( is he real queue backlog from acual arrivals. The sample pah of Q( and all power decisions p( are he same as when he drif-pluspenaly algorihm is implemened wih he nonzero iniial condiion q place. Of course, every ransmission µ( sends real daa from he queue, raher han fake daa. The resuling algorihm is: Iniialize Q real (0 = 0. Every slo, observe Q real ( and ω( and choose: { 1 if (qplace + Q p( = real (ω( V 0 oherwise Updae Q real ( by: Q real ( + 1 = maxq real ( + a( p(ω(, 0] (66 3 Indeed, recall from (60 and he proof of Lemma 5 ha V = O(log(1/ɛ, D is given in (43, and θ = V/ω b.

12 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 24, NO. 4, AUG. 2016, PP If q place > 0 hen q place = V/ω M ω M V/ω b. Thus, 0 q place V/ω b, and so he iniial condiion Q(0 = q place sill mees he requiremens of he lemmas of he previous secion. Therefore, he same performance bounds hold for he power process p( and he queue size process Q(. However, a every insan of ime, he real queue size Q real ( is reduced by exacly q place in comparison o Q(. B. LIFO scheduling The queue updae equaions (66 and (1 allow for any workconserving scheduling mechanism. The defaul mechanism is Firs-In-Firs-Ou (FIFO. However, Las-In-Firs-Ou (LIFO scheduling can provide significan delay improvemens for 98% of he packes 22]15]. Inuiively, his is because he backlog Q( is almos always o he righ of he V/ω b+1 poin in Fig. 3. Packes ha arrive when Q( V/ω b+1 mus wai for a leas V/ω b+1 unis of daa o be served under FIFO, bu are ransmied more quickly under LIFO. Work in 15] mahemaically formalizes his observaion. Roughly speaking, mos packes have average delay reduced by a leas V/(ω b+1 λ under LIFO (and wihou he place-holder echnique. Wih he place-holder echnique, his reducion is changed o (V/ω b+1 q place /λ (since he place-holder echnique already reduces average delay of all packes by q place /λ. One cavea is ha, under LIFO, a finie amoun of arriving daa migh never be ransmied. Of course, using LIFO as opposed o FIFO does no change he oal queue size or he fundamenal radeoff beween oal average queue size and average power. These issues are explored via simulaion in he nex secion. A. Two channel saes Average power Ep] V=0 δ=.25 VII. SIMULATION Power versus backlog (2 channel saes δ=.1667 ω only algorihm δ=.0833 δ= V=1 δ=.0125 δ=.0063 V=5 DPP V= V=4 DPP place V=10 V=20 V=40 V= Average backlog EQ] (log scale Fig. 4. Average power versus average backlog for he case of 2 channel saes. All daa poins are averages obained afer simulaion over 1 million slos. Three algorihms are shown. The drif-plus-penaly (DPP algorihms use various values of V. The V values are labeled for selec poins on he DPP curve (green. The ω-only algorihm uses various values of δ. Consider he scenario of Case 1 in Secion III. There are wo channel saes ω( {1, 2} wih π(1 = 3/4, π(2 = 1/4. The h(µ curve is shown in Fig. 1. Assume he arrival process a( is i.i.d. over slos wih: P ra( = 0] = 2 5, P ra( = 1] = 1 5, P ra( = 2] = 2 5 The arrival rae is λ = E a(] = 1, and he minimum average power required for sabiliy is p = h(1 = 3/4. Three differen algorihms are considered below: Drif-plus-penaly (DPP wih Q(0 = 0. DPP wih place-holder (DPP-place wih q place = maxv/2 2, 0] (from (65 and Q real (0 = 0. An ω-only policy designed o saisfy E µ(] = λ + δ and E p(] = h(λ + δ. The DPP algorihms operae online wihou knowledge of λ, π(1, π(2, while he ω-only policy is designed offline wih knowledge of hese values. Resuls are ploed in Fig. 4 for various values of V 0 and δ 0. The DPP algorihms significanly ouperform he ω-only algorihm even hough hey do no have knowledge of he sysem probabiliies. The heoreical radeoffs of he previous secion were derived under he assumpion ha V ωm 2 (in his case, ω2 M = 22 = 4. However, he DPP algorihms can be implemened for any value V 0. Observe from he figure ha average power sars approaching opimaliy even for values V < 4, and converges o he opimal p = 3/4 as V is increased beyond 4. I can be shown ha he ω-only algorihm achieves an O(ɛ- approximaion wih average queue size Θ(1/ɛ, whereas resuls in he previous secion prove he DPP algorihms achieve an O(ɛ-approximaion wih average queue size Θ(log(1/ɛ. The simulaions verify hese heoreical resuls. In his example, he DPP place-holder algorihm gives performance very close o sandard DPP, wih only a modes gain in he range V 4, 10]. For values V 4 he DPP and DPP-place algorihms are idenical. Convergence ime o he desired consrain µ( λ is illusraed in Fig. 5 by ploing he empirical value of E µ(] versus ime. The ω-only policy is no ploed because i achieves he consrain immediaely by is offline design. The DPP-place algorihm shows a sligh convergence ime improvemen over DPP. Boh DPP algorihms demonsrae ha µ( λ decays like V/. This is consisen wih he heoreical guaranees derived in he previous secion. Indeed, for an O(ɛ-approximaion, one ses V = Θ(log(1/ɛ, so afer ime Θ(log(1/ɛ/ɛ he deviaion from he consrain is a mos O(V/ O(ɛ. The corresponding average power E p(] is ploed in Fig. 6. B. Nine channel saes Now consider a process ω( wih 9 possible raes {ω 0,..., ω 9 }: The probabiliies are: Ω = {0, 3, 7, 11, 18, 22, 24, 36, 46} π(ω i = 1/15 if i {0, 1, 2} 2/9 if i {3, 4, 5} 2/45 if i {6, 7, 8} The arrival process a( has probabiliies: P ra( = 0] = 0.42, P ra( = 20] = 0.58 wih arrival rae λ = 11.6 packes/slo. The DPP-place algorihm uses q place = maxv/46 46, 0] (as in (65, and

13 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 24, NO. 4, AUG. 2016, PP Eµ(] V=5 V=10 V=20 V=40 Eµ(] versus ime for DPP algorihms ime slo Fig. 5. Average ransmission rae E µ(] versus ime, obained from 10 5 independen simulaion runs over he firs 500 slos. The DPP curves are hick, solid, and labeled wih V {5, 10, 20, 40}. The DPP-place curves are hin dashed curves where V {5, 10, 20, 40} corresponds o red, purple, grey, green, respecively. 0.8 V=5 Ep(] versus ime for DPP algorihms from he previous secion is considered. The simulaion is run over 6000 slos, broken ino hree phases of 2000 slos each. The sysem probabiliies are changed a he beginning of each phase. The algorihm is no aware of he changes and mus adap. Specifically: 1 Firs phase: The same parameers of he previous subsecion are used (so λ = Second phase: Channel probabiliies are he same as phase 1. The arrival rae is increased o λ = 13 by using P ra( = 20] =.65, P ra( = 0] = Third phase: The same arrival rae λ = 13 of phase 2 is used. However, channel probabiliies are changed o: π(ω i = 1/15 if i {0, 1, 2} 1/9 if i {3, 4, 5} 7/45 if i {6, 7, 8} Ep(] V=20 V=40 V= ime slo Ep(] Ep(] versus ime for nonergodic variaion DPP and DPP place (V=1000 DPP place (V=10000 DPP (V=10000 Fig. 6. Average power E p(] versus ime for he same experimens, V parameers, and color scheme as Fig. 5. q place > 0 if and only if V > 46 2 = I can be shown ha p = h(λ = 7/15 for his sysem. Simulaions for DPP and DPP-place are in Fig. 7. As before, he DPP algorihms ouperform he ω-only policy, alhough he improvemens are no as dramaic as hey are in Fig. 4. This is because he arrival rae vecor in his case is close o a verex poin of he h(µ curve. As before, he DPP-place algorihm performance is similar o ha of DPP wih a shifed V parameer. Fig. 7. Avg power Ep] V=0 V=250 δ=4.4 Power versus backlog (9 channel saes ω only δ=2.0 DPP δ=0.3 δ=0.01 V=1000 V=2116 DPP place V=10000 V= Avg backlog EQ] (log scale Simulaion for he ergodic sysem wih 9 channel saes. C. Robusness o non-ergodic changes This subsecion illusraes how he algorihm reacs o nonergodic changes. The sysem wih 9 possible channel saes ime slo Fig. 8. Average power (obained from independen simulaion runs versus ime for a sysem ha changes nonergodically over 3 phases. EQ(] EQ(] versus ime for nonergodic variaion DPP place (V=10000 DPP and DPP place (V=1000 DPP (V= ime slo Fig. 9. Average queue size (obained from independen simulaion runs versus ime for a sysem ha changes nonergodically over 3 phases. The resuling power and queue size averages are ploed in Figs. 8 and 9. The daa is obained by averaging sample pahs over independen runs. Fig. 8 shows ha for large V, average power converges o a value close o he long-erm opimum associaed wih each phase. Thus, he DPP algorihms adap o changing environmens. For each V, average power of DPP-place is roughly he same as DPP (Fig. 8. Average queue size of DPP-place is smaller han ha of DPP when V is large (Fig. 9.