Intelligent Packet Dropping for Optimal Energy-Delay Tradeoffs in Wireless Downlinks

Transcription

1 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 54, NO. 3, PP , MARCH Inelligen Packe Dropping for Opimal Energy-Delay Tradeoffs in Wireless Downlinks Michael J. Neely Universiy of Souhern California hp://www-rcf.usc.edu/ mjneely Absrac We explore he advanages of inelligenly dropping a small fracion of packes ha arrive for ransmission over a ime varying wireless downlink. Wihou packe dropping, he opimal energy-delay radeoff conforms o a square roo radeoff law, as shown by Berry and Gallager (2002). We show ha inelligenly dropping any non-zero fracion of he inpu rae dramaically changes his relaion from a square roo radeoff law o a logarihmic radeoff law. Furher, we demonsrae an innovaive algorihm for achieving his logarihmic radeoff wihou requiring a-priori knowledge of arrival raes or channel probabiliies. The algorihm can be implemened in real ime and easily exends o yield similar performance for muli-user sysems. I. INTRODUCTION Wireless sysems mus offer high hroughpu and low delay while operaing wih very lile power. In order o maximize performance, i is desirable for sysems o reac o curren channel condiions using rae adapive and power adapive ransmission echnology. In his paper, we develop a scheduling algorihm ha uses channel informaion o yield an average power expendiure ha can be pushed arbirarily close o he minimum average power required for sysem sabiliy, wih a corresponding opimal radeoff in average delay. In [2] i was shown ha when all packes mus be ransmied, he opimal energy-delay radeoff is given by a square roo radeoff law, known as he Berry-Gallager bound. In his paper, we consider opimal energy-delay radeoffs under he assumpion ha a small fracion of packes can be dropped. We show ha inelligenly dropping packes can dramaically change he energy-delay relaion from a square roo radeoff law o a logarihmic radeoff law. This resul holds for any non-zero bound on he packe drop rae. Furher, we demonsrae an innovaive algorihm for join power allocaion and packe dropping ha achieves he opimal logarihmic radeoff wihou requiring a-priori knowledge of he inpu rae or he channel sae probabiliies. The algorihm can be implemened in real ime and easily exends o offer provably opimal energy-delay radeoffs for muli-user sysems. This demonsraes ha significan improvemens in average delay are possible if a non-zero packe drop rae can be oleraed. Relaed work in [3] [4] [5] [6] considers energy and delay issues in a single wireless downlink wih a saic channel, and This work was presened in par a he WiOp conference, Boson, April This maerial is suppored in par by one or more of he following: he DARPA IT-MANET program gran W911NF , he NSF gran OCE , he NSF Career gran CCF work in [2] [7] [8] considers downlinks wih fading channels. The fundamenal square roo radeoff for single-user sysems is developed by Berry and Gallager in [2], and his radeoff is exended o muli-user sysems in [9]. The problem of fairness and uiliy opimal flow conrol is invesigaed in [10], where i is shown ha he fundamenal uiliy-delay radeoff law is quie differen and has a logarihmic srucure. The dynamic conrol algorihms of [9] [10] combine he conceps of buffer pariioning developed in [2] and performance opimal Lyapunov neworking from [11]-[14]. Specifically, he work in [11]-[14] presens simple Lyapunov echniques for achieving sabiliy and performance opimizaion simulaneously (exending he Lyapunov resuls developed for queueing sabiliy in works such as [15]-[22]). This paper uses similar echniques o address he problem of inelligen packe dropping for energy efficiency. However, he opimal conrol sraegies in his conex have a differen srucure from hose of [2] [9]. Specifically, he algorihms of [2] [9] pariion he buffer of an infinie queue ino wo halves, where differen drif modes are designed for each pariion. Here, we design a sraegy ha emulaes a finie buffer queue wih sricly posiive drif. We show ha he sraegy yields a logarihmic delay radeoff ha canno be achieved in sysems ha do no allow packe dropping. An ouline of his paper is as follows: In he nex secion we presen he sysem model and problem formulaion. In Secion III he basic posiive drif algorihm is developed, under he assumpion ha all channel sae probabiliies are a-priori known. A more pracical dynamic sraegy ha does no require such a-priori knowledge is developed in Secion IV. The sraegy uses a novel form of Lyapunov heory o make on-line decisions ha are radeoff opimal. Necessiy of he logarihmic radeoff is proven in Secion VI for he special case of sysems wih no channel sae variaion. Exensions o muli-user sysems are briefly considered in Secion VII. A modified adapive hreshold algorihm is presened in Secion VIII o yield a furher consan-facor delay improvemen while mainaining radeoff opimaliy. This adapive hreshold policy exends he conference version of his paper [1] and can be viewed as a new echnique for nework learning. Simulaions are provided in Secion IX. II. SYSTEM MODEL Consider a single wireless ransmier ha operaes in sloed ime wih slos normalized o ineger unis {0, 1, 2,...}.

2 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 54, NO. 3, PP , MARCH The ransmission rae offered by he ransmier on slo depends on a conrollable power variable P () and an unconrollable channel sae S() according o a general rae-power funcion C(P (), S()), aking unis of bis/slo. We assume ha power allocaions are limied by a peak power consrain P max, so ha 0 P () P max for all. Channel saes S() are assumed o be conained wihin some finie bu arbirarily large sae space S. An example rae-power funcion is he logarihmic capaciy curve for an Addiive Whie Gaussian Noise channel: C(P, S) = log(1 + P α S ) (1) where α S is an aenuaion/noise coefficien associaed wih channel sae S. Oher examples include link budge funcions corresponding o a finie number of modulaion and coding schemes designed o achieve a sufficienly low probabiliy of error, in which case he C(P, S) funcion can be a disconinuous sep funcion in P for each channel sae S (see Fig. 1). Such disconinuous seps migh also arise in he special case when all packes have fixed bi lenghs and when C(P, S) akes only a finie se of raes associaed wih ransmiing an inegral number of packes. Our analysis holds for all such C(P, S) funcions, and in paricular we assume only ha C(P, S) saisfies he following srucural properies: 1 1) Boundedness: There is a maximum ransmission rae µ max such ha 0 C(P, S) µ max for all P P max and all S S. 2) Zero Rae for Zero Power: C(0, S) = 0 for all channel saes S S. 3) Upper Semi-Coninuiy: For each channel sae S S, he C(P, S) funcion is upper semi-coninuous in he P variable. Specifically, for any power level P such ha 0 P P max and for any infinie sequence {P n } such ha lim n P n = P, we have: C(P, S) lim sup C(P n, S) n We noe ha he class of upper semi-coninuous funcions includes all coninuous funcions, and also includes all piecewise coninuous funcions such ha he funcion value a any poin of disconinuiy is equal o he larges of is limiing values a ha poin (see Fig. 1). This is a naural propery for any pracical rae-power curve, as a poin of disconinuiy usually represens a hreshold poin a which i is possible o suppor a larger ransmission rae. Channel saes S() are assumed o be independen and idenically disribued (i.i.d.) every slo, wih sae probabiliies π S = P r[s() = S] for all S S. Le A() represen he amoun of new daa ha eners he sysem a ime (in unis of bis). This arrival process A() is assumed o be i.i.d. wih rae λ, so ha E {A()} = λ for all. Furher, we assume he second momen of arrivals is bounded in erms of a finie consan  max, so ha: E { A() 2} Â2 max for all 1 The firs srucural propery (boundedness) is he only one essenial o our analysis. The oher wo properies can be removed wihou affecing he main resuls of his paper. They are used only o simplify exposiion of Lemma 1 in Secion III. C(P, S) (a) Fig. 1. Example C(P, S) funcions: (a) A funcion ha is concave and increasing in P. (b) A piecewise consan funcion wih he upper semiconinuiy propery. Newly arriving daa is eiher admied o he sysem, or dropped. Le Ã() be a conrol variable represening he amoun of new arrivals admied on slo, where 0 Ã() A(). All admied daa is sored in a queue o awai ransmission, and we le U() represen he queue backlog or unfinished work in he sysem a ime. Every imeslo, a downlink conroller observes he curren channel sae S() and he curren queue backlog U() and chooses a power allocaion P () subjec o he consrain 0 P () P max. This yields an offered ransmission rae of µ() = C(P (), S()). The queueing dynamics hus proceed as follows: P (b) U( + 1) = max[u() µ(), 0] + Ã() (2) Noe ha he acual bis ransmied can be differen from µ() if here are no enough bis in he queue o ransmi a he full offered ransmission rae. Le µ() represen he acual amoun of bis ransmied during slo. Noe ha µ() µ(), and sric inequaliy can only occur if U() < µ max. Le ρ < 1 represen a required accepance raio. The goal is o achieve an opimal energy-delay radeoff while mainaining an accepance rae greaer han or equal o ρλ. Tha is, we require he following guaranee on long erm hroughpu: lim inf 1 1 A. The Berry-Gallager Bound E { µ(τ)} ρλ Le µ c represen he downlink capaciy, so ha he sysem can sably suppor any arrival rae λ such ha 0 λ < µ c, and µ c is he larges number wih his propery. Define Φ(λ) as he minimum energy required o sabilize he queue if he inpu rae is λ (assuming ha 0 λ < µ c ). I can be shown ha Φ(λ) indeed depends only on λ (and no on higher order arrival saisics), and ha i is convex over he inerval 0 λ < µ c. In [2], i is shown ha a sequence of sabilizing power allocaion algorihms, indexed by increasing posiive numbers V, can be designed ha push average power expendiure arbirarily close o Φ(λ). Furher, i was shown ha, subjec o some concaviy assumpions on he C(P, S) funcion and some admissibiliy assumpions on he inpu process, any sabilizing power allocaion algorihm ha yields average power wihin O(1/V ) of he minimum power Φ(λ) mus also have average delay greaer han or equal o Ω( V ). 2 We refer o his square 2 Where he noaion f(v ) = Ω( V ) denoes a funcion ha increases a leas as fas as a square roo funcion. P

3 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 54, NO. 3, PP , MARCH roo radeoff law as he Berry-Gallager bound. We noe ha he admissibiliy assumpions required for his bound include he assumpion ha arrivals and channel saes are i.i.d. over imeslos. Furher, he bound is derived under he assumpion ha no daa is allowed o be dropped. As an example, consider a downlink ha saisfies all assumpions required for he Berry-Gallager bound. Assume he arrival process is i.i.d. wih rae λ. However, suppose ha we only need o admi a fracion ρ < 1 of all incoming daa, so ha a drop rae of up o (1 ρ)λ bis/slo can be oleraed. The minimum power required o sabilize such a sysem is hus equal o Φ =Φ(ρλ). Hence, he new goal is o push average power expendiure arbirarily close o Φ. Consider now he naive dropping policy ha makes random and independen admission decisions every imeslo, where all incoming daa A() is admied wih probabiliy ρ, and else i is dropped. The resuling admied rae is exacly equal o ρλ. However, he admied inpu sream is sill i.i.d. from slo o slo, and hence he Berry-Gallager bound sill governs he energy-delay performance associaed wih scheduling his admied daa. Therefore, his naive approach o packe dropping canno overcome he square roo radeoff relaion. However, insead of randomly dropping packes, we consider schemes ha inelligenly drop packes. Remarkably, we find ha for any arbirarily small bu posiive dropping raio (i.e., any (1 ρ) > 0), i is possible o design an inelligen packe dropping scheme (ogeher wih a power allocaion scheme) ha yields an average power expendiure ha differs from Φ by a mos O(1/V ), while yielding average delay ha grows only logarihmically in he conrol parameer V. Hence, he abiliy o drop packes dramaically improves he energy-delay radeoff law. This resul shows ha he square roo curvaure of he Berry-Gallager bound is due only o a very small fracion of packes ha arrive a in-opporune imes. Average delay can be dramaically reduced by idenifying hese packes and dropping hem. III. A DROPPING SCHEME FOR KNOWN SYSTEM STATISTICS In his secion we demonsrae exisence of a scheme ha uses inelligen packe dropping o overcome he Berry- Gallager bound. The policy developed in his secion is no inended as a pracical means of conrol, as i can only be consruced via off-line compuaions based on full knowledge of he arrival rae λ and he channel sae probabiliies π S (for each S S). In Secion IV we consruc an on-line sraegy ha achieves he same performance wihou requiring knowledge of hese parameers. We firs presen he following Lemma from [13]: Lemma 1: If channel saes S() are i.i.d. and if he raepower funcion C(P, S) saisfies he srucural properies of he previous secion, hen for any λ < µ c a saionary power allocaion policy can be designed ha makes randomized power allocaion decisions P () based only on observaions of he curren channel sae S(), yielding: E {P ()} = Φ(λ) for all E {µ ()} = λ for all where µ () = C(P (), S()) is he associaed ransmission rae of he randomized scheme. Noe ha he expecaions of he above lemma are aken wih respec o he random channel sae S() and he poenially random power allocaion ha depends on S(). Such a policy could in principle be consruced wih a-priori knowledge of λ and π S for all S S. I can be shown ha if he srucural properies 2 and 3 for he C(P, S) funcion are removed, hen he above lemma can be modified o sae ha here exiss an infinie sequence of randomized power allocaion policies Pn() such ha lim n E {Pn()} = Φ(λ) and lim n E {µ n()} λ. This modified saemen can also be used o prove our main resuls, alhough i is more convenien o use he simpler saemen given in Lemma 1. A. The Posiive Drif Algorihm The firs sep of our inelligen packe dropping algorihm is o emulae a finie buffer queueing sysem wih buffer size Q, where he consan Q is o be deermined laer. Tha is, we modify he queueing updae equaion as follows: U( + 1) = min [Q, max[u() µ(), 0] + A()] (3) This is he same queue updae equaion as (2), wih he excepion ha any daa exceeding he buffer size Q is necessarily dropped. Specifically, he amoun of arrivals Ã() admied every slo is decided purely in erms of his finie buffer hreshold, so ha Ã() = A() whenever adding all new A() arrivals does no make oal backlog exceed he hreshold, and else Ã() is equal o only ha porion of he new arrivals ha ake backlog up o he Q hreshold. The following policy is defined in erms of a given required accepance raio ρ < 1. Posiive Drif Algorihm for Known Saisics: 1) Emulae he finie buffer sysem (3) using a consan buffer size Q (o be chosen laer). 2) Le P () = P (), where P () is he saionary policy ha observes S() and hen randomly allocaes power o yield E {P ()} = Φ((ρ + ɛ)λ), E {µ ()} = (ρ + ɛ)λ for all (as in Lemma 1), for some small value ɛ such ha 0 < ɛ < (1 ρ), o be deermined laer. For suiable choices of Q and ɛ, he above policy yields a logarihmic energy-delay radeoff relaion. I is perhaps surprising ha he policy is designed o have a posiive drif in he direcion of he finie buffer heshold Q. Indeed, every slo he expeced new arrivals is given by E {A()} = λ, which exceeds he expeced ransmission rae E {µ ()} = (ρ + ɛ)λ (recall ha ɛ is chosen so ha ρ+ɛ < 1). Inuiively, one migh expec an opimal queueing conrol algorihm o have negaive drif owards he empy sae U() = 0. However, his is precisely wha he algorihm is designed o avoid, as fundamenal inefficiencies arise from he edge effecs associaed wih a queue becoming empy. The algorihm is similar in spiri o he buffer pariioning algorihm of [2], which uses a posiive drif whenever queue backlog is below a given hreshold and a negaive drif when backlog is larger han his hreshold. However, in our algorihm above, he hreshold is given by he finie buffer size Q. Any daa ha violaes his hreshold is simply dropped.

4 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 54, NO. 3, PP , MARCH B. Analysis of he Posiive Drif Algorihm To analyze performance of he algorihm, noe ha ime average power expendiure saisfies: P = 1 1 lim E {P ()} = Φ(ρλ + ɛλ) Φ(ρλ) + Φ (λ)ɛλ where Φ (λ) denoes he righ derivaive of he Φ( ) funcion evaluaed a λ (noe ha finie righ derivaives exis for any convex funcion over an open inerval). Thus, because Φ =Φ(ρλ), average power expendiure saisfies: P Φ + Φ (λ)ɛλ (4) For any fixed conrol parameer V 1, he idea is o choose ɛ = (1 ρ)/(2v ). Wih his choice, i follows from (4) ha average power expendiure is wihin O(1/V ) of he minimum average power Φ. Nex, noe ha he ime average ransmission rae is given by: 1 1 lim E {µ ()} = (ρ + ɛ)λ However, his ime average ransmission rae can be larger han he ime average hroughpu, due o he fac ha he acual daa ransmied may be less han µ () if U() < µ max. To ensure ha he hroughpu is greaer han or equal o ρλ, we presen he following lemma concerning edge effecs in any queueing sysem wih a ransmission rae µ(). Recall ha µ() is defined as he acual daa ransmied during slo. Lemma 2: (Edge Effecs) If µ() µ max for all, hen any sochasic queueing sysem ha ransmis a he full rae µ() whenever U() µ max mus saisfy: lim inf 1 1 E { µ(τ)} lim inf 1 1 E {µ(τ)} αµ max where: α= 1 1 lim sup P r[u() < µ max ] (5) Proof: Noe ha we have µ() µ() for all, wih equaliy whenever U() µ max. Hence: µ() µ() µ max 1 [U()<µmax] (6) where 1 X is an indicaor funcion equal o 1 if even X is saisfied, and zero else. Inequaliy (6) can be verified as follows: If U() < µ max, hen he righ hand side of (6) is equal o µ() µ max, which is non-posiive. Hence he inequaliy rivially holds in his case. Oherwise, U() µ max, and he inequaliy (6) holds wih equaliy. Taking expecaions of (6) yields for all : E { µ()} E {µ()} µ max P r[u() < µ max ] Summing over τ {0,..., 1}, dividing by, and aking he lim inf as yields he resul. Inuiively, he above lemma indicaes ha he acual hroughpu of he queueing sysem differs from he ime average ransmission rae by an amoun ha is a mos αµ max, where α represens ime average probabiliy ha he queue backlog drops below µ max. We call α he edge probabiliy. Applying he above lemma o he posiive drif algorihm above (where E {µ ()} = (ρ + ɛ)λ for all ) yields he following guaranee on ime average hroughpu: lim inf 1 1 E { µ (τ)} ρλ + ɛλ αµ max (7) To ensure ha he hroughpu is greaer han or equal o ρλ, from (7) we find i suffices o ensure ha he edge probabiliy α is small enough o saisfy αµ max ɛλ. However, noe ha on every imeslo, he expeced difference beween he arrival rae and he ransmission rae saisfies: E {A() µ ()} = λ (ρ + ɛ)λ = λ(1 ρ ɛ) (8) The above expecaion is defined as he drif of he algorihm. Using he fac ha ɛ = (1 ρ)/(2v ), i follows ha he drif is greaer han or equal o λ(1 ρ)/2 whenever V 1. This posiive drif ends o increase queue backlog, pushing U() away from he edge region U() < µ max. Furher, i can be shown ha he resuling edge probabiliy α decays exponenially in he buffer size Q. Therefore, he edge probabiliy α can be made as small as desired, saisfying αµ max ɛλ, while mainaining a buffer size Q ha is logarihmic in 1/ɛ, and hence logarihmic in V. Formally, he fac ha α decays exponenially in Q is shown by he following lemma. Lemma 3: Given a queueing sysem wih a finie buffer size Q and a posiive drif ha saisfies (8), here exiss a posiive consan θ such ha he edge probabiliy α saisfies: α e θ (Q µ max) The above lemma follows from he Kingman bound [23], which also specifies he consan θ. The proof, ogeher wih a simple lower bound on θ, are given in Appendix B. Hence, if Q is chosen as follows: Q =µ max + 1 θ log ( µmax ɛλ hen he lemma implies α ɛλ/µ max, ensuring from (7) ha hroughpu is greaer han or equal o ρλ. Furher, because 1/ɛ = O(V ) and U() Q for all, i follows from (9) ha average queue backlog is O(log(V )), as is he average delay of admied daa (via Lile s Theorem). This demonsraes feasibiliy of a logarihmic energy-delay radeoff. While he posiive drif algorihm is concepually very simple, i canno be implemened wihou full a-priori knowledge of he arrival rae λ and he channel probabiliies π S (for each S S). Even if all of hese parameers are esimaed, he inrinsic esimaion error migh preclude realizaion of he desired performance, and could lead o significan mismach problems if inpu raes or channel probabiliies change over ime. Furher, he algorihm does no easily exend o muli-user, muli-channel sysems, because he oal number of channel saes in such sysems grows geomerically wih he number of channels. Therefore, i is essenial o consruc a more pracical algorihm o achieve a logarihmic energydelay radeoff. ) (9)

5 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 54, NO. 3, PP , MARCH (ρ + ɛ)a() µ() X() Fig. 2. An illusraion of he virual queueing sysem associaed wih he X() updae equaion (10). IV. AN ON-LINE ALGORITHM FOR INTELLIGENT PACKET DROPPING To consruc an on-line algorihm for achieving a logarihmic energy-delay radeoff, we use he heory of performance opimal Lyapunov neworking [11]-[14]. To his end, suppose he sysem emulaes a finie buffer sysem wih buffer size Q (o be chosen laer), so ha queue backlog U() evolves according o (3). Define he following Lyapunov funcion L(U): L(U) =e ω(q U) where ω > 0 is a parameer o be deermined laer. Because U() Q for all, he Lyapunov funcion L(U()) reaches is minimum value when U() = Q, and increases exponenially when queue backlog deviaes from he buffer hreshold Q. We show ha scheduling o minimize he drif of his Lyapunov funcion from one slo o he nex ensures ha he edge probabiliy α decays exponenially in Q. To mainain high hroughpu, i is desirable o ensure ha he ime average ransmission rae µ() is greaer han or equal o (ρ + ɛ)λ, for some value ɛ such ha 0 < ɛ < (1 ρ), o be deermined laer. To his end, we use he virual queue concep developed in [13]. Le X() represen a virual queue ha is implemened purely in sofware, where X(0) = 0 and where X() follows he following updae equaion every slo: X( + 1) = max[x() µ(), 0] + (ρ + ɛ)a() (10) where A() is he amoun of new arrivals during slo (some of which may no be admied o he acual queue U()), and where µ() is he ransmission rae chosen by he downlink conrol algorihm. Noe ha X() can be viewed as he backlog in a queue wih inpu (ρ+ɛ)a() and ime varying server rae µ() (see Fig. 2). Definiion 1: A queueing sysem wih unfinished work X() is srongly sable if: lim sup 1 1 E {X(τ)} < I is no difficul o show ha a srongly sable queue wih an upper bounded ransmission rae has he propery ha he lim inf of he difference beween he ime average server rae and he ime average arrival rae is non-negaive [13]. Because he ime average arrival rae o he X() queue is given by (ρ + ɛ)λ, we have he following lemma: Lemma 4: If he X() queue is srongly sable and he A() process is i.i.d. wih rae λ, hen: lim inf E {µ(τ)} ρλ + ɛλ (11) 1 lim inf E { µ(τ)} ρλ + ɛλ αµ max (12) Proof: Because he X() queue is srongly sable wih an upper bounded ransmission rae µ max, he lim inf difference beween he ime average server rae and arrival rae saisfies: lim inf 1 1 E {µ(τ) (ρ + ɛ)a(τ)} 0 The inequaliy (11) follows from he above inequaliy ogeher wih he fac ha E {A()} = λ for all. The inequaliy (12) follows from (11) ogeher wih Lemma 2. A. Performance Opimal Lyapunov Neworking Our echnique of sochasic queue opimizaion is based on he heory of performance opimal Lyapunov neworking, which allows sabiliy and performance opimizaion o be achieved via a single drif argumen [11] [13] [14]. This exends he Lyapunov sabiliy resuls of [15]-[22], and is closely relaed o sochasic gradien opimizaion (see, for example, [24] for an applicaion o daa neworks). To demonsrae he echnique, consider a sysem wih a vecor process Z() represening a se of queue saes ha evolve according o some probabiliy law. Le P () represen a non-negaive conrol process ha affecs sysem dynamics, and le P represen a arge upper bound desired for he ime average of P (). Le Ψ(Z) represen any non-negaive funcion of Z (represening a Lyapunov funcion), and le (Z()) represen he condiional Lyapunov drif, defined as follows: 3 (Z()) =E {Ψ(Z( + 1)) Ψ(Z()) Z()} (13) We have he following imporan lemma, which is a modified version of similar resuls developed in [11][13][14]. Lemma 5: (Lyapunov Opimizaion [11][13][14]) If here is a process B() and consans ɛ > 0, V 0, ogeher wih a non-negaive funcion f(z), such ha he queueing sysem saisfies he following drif inequaliy for all and all Z(): (Z())+V E {P () Z()} B() ɛf(z())+v P (14) hen: where lim sup lim sup 1 P inf = lim inf E {P (τ)} P + B/V E {f(z(τ))} B + V (P P inf ) ɛ E {P (τ)}, B = lim sup 1 1 E {B(τ)} 3 Sricly speaking, correc noaion for he condiional Lyapunov drif in (13) is (Z(), ), as he drif may also depend on he imeslo. However, we use he simpler noaion (Z()) as a formal and more concise represenaion of he righ hand side of (13).

6 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 54, NO. 3, PP , MARCH If V is a conrol parameer of he sysem, he above lemma indicaes ha he ime average of he P () process can be bounded by a value ha is arbirarily close o he arge value P, wih a corresponding radeoff in he ime average value of f(z()) ha is a mos linear in V. To apply Lemma 5 o our queueing problem, le Z() = (U(), X()) represen he vecor queue sae of boh he acual and virual queues, and define he following mixed Lyapunov funcion: Ψ(Z) =L(U) X2 = e ω(q U) X2 The condiional drif (Z()) for he above Lyapunov funcion is defined in (13). Moivaed by Lemma 5, he goal of our dynamic conrol sraegy is o choose P () o minimize a bound on he following drif meric every imeslo : Drif Meric: (Z()) + V E {P () Z()} where V 1 is a conrol parameer ha effecs he energydelay performance of he algorihm. Noe ha his drif meric is simply he lef hand side of (14). B. Algorihm Consrucion To compue a bound on he drif meric of he previous subsecion, i is useful o define σ 2 o be any posiive consan ha saisfies he following inequaliy for all and all Z(): σ 2 E { (µ() A()) 2 Z() } (15) Noe ha because µ() µ max and E { A() 2} Â2 max, choosing σ 2 =µ 2 max + Â2 max ensures ha (15) is saisfied for all. A igher bound is given by σ 2 =Â2 max + max[0, µ 2 max 2λµ max ], which is useful in cases when he inpu rae λ is known. Likewise, if here exiss a deerminisic arrival bound A max such ha A() A max for all, hen choosing σ 2 = max[µ 2 max, A 2 max] also ensures ha (15) is saisfied. Lemma 6: If a posiive consan ω is chosen o saisfy he inequaliy: ωe ωµmax λ(1 ρ ɛ)/σ 2 (16) hen for all we have he following bound on he drif meric: (Z()) + V E {P () Z()} B + V E {P () Z()} ωe ω(q U()) [λ E {µ() Z()} λ(1 ρ ɛ)/2] X() [E {µ() Z()} (ρ + ɛ)λ] (17) where µ() = C(P (), S()), and where: B= µ2 max + (ρ + ɛ) 2 Â 2 max + 1 (18) 2 Proof: The proof involves summing he Lyapunov drif expressions associaed wih he acual queue U() and he virual queue X(). These expressions are compued using he dynamic queue updae equaions (10) and (3). The deailed proof is given in Appendix A. I is no difficul o show ha if ω is chosen as follows: ω= λ(1 ρ ɛ) e λµmax(1 ρ ɛ)/σ 2 (19) σ 2 hen he inequaliy consrain (16) is saisfied, and hence he resul of Lemma 6 holds. This follows direcly from he fac ha for any posiive value c, he inequaliy xe x c is always saisfied by he variable x = ce c. If λ is unknown, hen any posiive lower bound λ 0 (such ha 0 < λ 0 λ) can be used in replacemen for λ in (19) while sill ensuring ha (16) is saisfied. We design he dynamic conrol policy o minimize he drif bound given in he righ hand side of (17) every imeslo, considering all possible power allocaion opions P () such ha 0 P () P max. Isolaing he erms on he righ hand side of (17) ha depend on he conrol variable P () (noing ha µ() = C(P (), S())), i is clear ha minimizing he bound in (17) is equivalen o choosing P () in reacion o he curren channel sae and he curren queue backlogs in order o maximize he following expression every imeslo: {( E X() ωe ω(q U())) } C(P (), S()) V P () Z() Maximizing he above condiional expecaion is accomplished by deerminisically maximizing he resuling expression corresponding o he paricular channel sae realizaion S() and he paricular queue sae Z() = (U(), X()) observed on he curren imeslo. This leads o he following dynamic policy, which uses a conrol parameer V 1 and uses fixed parameers Q, ɛ, ω o be deermined laer in erms of V and ρ. Dynamic Packe Dropping Policy: Every imeslo, observe he curren channel sae S() and he curren queue backlogs U() and X(). Then: 1) Allocae power P () = P, where P solves: Maximize: Subjec o: C(P, S())(X() ωe ω(q U()) ) V P 0 P P max 2) Ierae he virual queue X() according o (10), using µ() = C(P (), S()). 3) Emulae he finie buffer queue U() according o (3). Noe ha he power allocaion sep in he above conrol policy involves a simple opimizaion of a funcion of one variable, and can easily be solved in real ime for mos pracical C(P, S) funcions. For example, if C(P, S) is concave and differeniable in P for all channel saes S (as in (1)), hen he opimal P () value can be solved simply by aking a derivaive and seing P () o he local maximum found on he inerval 0 P P max (possibly achieved a he endpoins P = 0 or P = P max ). If C(P, S) is piecewise consan wih a fixed number of ransmission rae opions (and hence a fixed number of power opions), hen he soluion is found simply by comparing each opion. Theorem 1: (Dynamic Packe Dropping Performance) For a given value ρ < 1 and a fixed conrol parameer V 1, if parameers ɛ, ω, and Q are chosen so ha ɛ = (1 ρ)/(2v ), ω is posiive and saisfies (16), and Q = log(xv )/ω, where x is any value ha saisfies: x 4µ max e ωµmax B λ 2 ω(1 ρ ɛ)(1 ρ) hen: 1 1 (a) lim sup E {P (τ)} Φ + O(1/V ) (b) U() Q for all, where Q = O(log(V )) (20)

7 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 54, NO. 3, PP , MARCH (c) lim inf E { µ(τ)} ρλ We prove he above heorem in he nex secion. Noe ha because U() O(log(V )) for all, average delay is also O(log(V )) (by Lile s Theorem). Hence, he algorihm saisfies he required accepance rae and yields a logarihmic energy-delay radeoff. Noe ha he consans can be chosen o saisfy he necessary inequaliies (16) and (20) jus by knowing a lower bound λ 0 on he inpu rae λ, so ha he exac inpu rae λ is no required. Likewise, he channel sae probabiliies π s are no required for implemenaion. Hence, he algorihm can easily adap o he siuaion where he channel sae probabiliies change beween periods of nework operaion. We noe ha he value of Q was chosen only o ensure a sufficienly small analyical bound on he edge probabiliy α. Our analysis was conservaive, and experimenally we find ha (consan facor) improvemens in delay can be achieved by appropriaely reducing he value of Q, wihou affecing hroughpu or average energy expendiure. This is discussed furher in Secions VIII and IX, where simulaion resuls are presened and a modified adapive hreshold policy is inroduced. V. PERFORMANCE ANALYSIS Here we prove Theorem 1. Noe ha he dynamic policy is designed o minimize he righ hand side of he drif bound (17) over all possible power allocaion policies. In paricular, he resuling bound is less han or equal o he corresponding expression associaed wih any alernaive power allocaion policy P (). Thus, for any policy where P () is randomly chosen every slo in reacion o he curren channel sae S() bu independenly of he curren queue sae Z() = (U(), X()), we have: (Z()) + V E {P () Z()} B + V E {P ()} ωe ω(q U()) [λ E {µ ()} λ(1 ρ ɛ)/2] X()[E {µ ()} (ρ + ɛ)λ] (21) where µ () = C(P (), S()). We emphasize ha he P (), U(), X() variables used in he above inequaliy correspond o he dynamic conrol policy under invesigaion (and are he same as hose in (17)), while he new variables P () and µ () are used as replacemens in he righ hand side of (17) and correspond o an alernaive power assignmen. Consider now he paricular randomized policy P () ha allocaes power in reacion o he curren channel sae S() (bu independenly of queue backlog) o yield he following for all : E {µ ()} = (ρ + ɛ)λ (22) E {P ()} = Φ((ρ + ɛ)λ) (23) Such a policy is guaraneed o exis by Lemma 1. Using equaions (22) and (23) in (21) yields: (Z()) + V E {P () Z()} B + V Φ((ρ + ɛ)λ) ωe ω(q U()) λ(1 ρ ɛ)/2 (24) The above drif inequaliy is in he exac form as given in he Lyapunov Opimizaion Lemma (Lemma 5). Direcly applying he lemma yields: 1 1 lim sup lim sup 1 E {P (τ)} Φ((ρ + ɛ)λ) + B/V (25) 1 E { e ω(q U(τ))} B+V [Φ((ρ+ɛ)λ) P inf ] ωλ(1 ρ ɛ)/2 (26) Proof of par (a) of Theorem 1: Recall ha Φ =Φ(ρλ). Thus, from (25) we have: lim sup 1 1 E {P (τ)} Φ(ρλ + ɛλ) + B/V Φ + Φ (λ)ɛλ + B/V Φ + O(1/V ) where he final inequaliy follows because ɛ = (1 ρ)/(2v ). Proof of par (b) of Theorem 1: Noe ha he finie buffer queueing updae equaion in (3) ensures ha U() Q for all. The resul follows by noing ha Q = O(log(V )). Proof of par (c) of Theorem 1: To prove par (c), we firs make he following claims: Claim 1: The X() queue is srongly sable. Claim 2: The edge probabiliy α (defined in (5)) saisfies α λɛ/µ max. The claims are proven a he end of his secion. Because he X() queue is sable, from Lemma 4 of he previous secion we have ha he ime average sysem hroughpu saisfies: lim inf 1 1 E { µ(τ)} ρλ + ɛλ αµ max where he final inequaliy follows from Claim 2. I remains only o prove Claims 1 and 2. Proof: (Claim 1) Consider again he drif bound in (21). However, insead of considering a power allocaion policy P () ha saisfies (22) and (23), we consider an alernaive policy P () ha also makes randomized decisions based only on he curren channel sae S(), bu which yields: ρλ E {µ ()} = λ(1 + ρ + ɛ)/2 E {P ()} = Φ(λ(1 + ρ + ɛ)/2) Again, such a policy is guaraneed o exis by Lemma 1. Using he above expressions in (21) yields: (Z()) + V E {P () Z()} B + V P max X()λ(1 ρ ɛ)/2 Because (1 ρ ɛ) > 0, he above drif expression is in he exac form for applicaion of Lemma 5. We hus have: lim sup 1 1 E {X(τ)} proving ha X() is srongly sable. B + V P max λ(1 ρ ɛ)/2

8 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 54, NO. 3, PP , MARCH Proof: (Claim 2) For any imeslo τ and for any disribuion on he random variable U(τ), we have: E { e ω(q U(τ))} E { e ω(q U(τ)) U(τ) < µ max } P r[u(τ) < µmax ] e ω(q µmax) P r[u(τ) < µ max ] Summing he above inequaliy over τ {0,..., 1} yields: e ω(q µmax) 1 1 P r[u(τ) < µ max ] E {e ω(q U(τ))} Dividing boh sides by, aking he lim sup, and using he inequaliy (26) yields: e ω(q µmax) α B + V [Φ((ρ + ɛ)λ) P inf ] ωλ(1 ρ ɛ)/2 (27) where α is defined in (5). However, from Claim 1 we know ha he X() queue is sable, and hence he lim inf of he ime average energy used o sabilize he queue mus be greaer han or equal o he minimum average energy required for sabiliy [13]. Because he ransmission rae of he X() queue is given by µ() = C(P (), S()) and he average inpu rae is given by (ρ + ɛ)λ (see (10) and Fig. 2), he lim inf of he expended energy is given by P inf, and he minimum energy for sabiliy is given by Φ((ρ + ɛ)λ). Therefore, we have P inf Φ((ρ + ɛ)λ). Using his fac in (27) yields: Therefore: e ω(q µmax) α [ α e ωµmax B ωλ(1 ρ ɛ)/2 B ωλ(1 ρ ɛ)/2 ] e ωq Using he fac ha Q= log(xv )/ω, we have: [ ] e ωµmax B 1 α ωλ(1 ρ ɛ)/2 V x Using (20) o replace he x variable in he righ hand side of he above inequaliy yields: α λ(1 ρ) 2µ max V Using he fac ha ɛ =(1 ρ)/(2v ) in he above inequaliy yields α λɛ/µ max, proving he claim. VI. NECESSITY OF THE LOGARITHMIC TRADEOFF The logarihmic energy-delay radeoff may seem o be an arifac of he exponenial Lyapunov funcion L(U) ha was used, so ha anoher Lyapunov funcion (perhaps doubly exponenial) could perhaps offer sub-logarihmic performance. However, his is no he case. Here we presen a class of sysems for which he opimal energy-delay radeoff is necessarily logarihmic, and hence his radeoff is fundamenal. We consider he special case of a sysem wih no channel variaion, so ha he rae-power funcion is given by C(P ). Furher, we assume he sysem has he following properies: 1) Arrivals A() are i.i.d. over imeslos, and here exiss a probabiliy q > 0 such ha P r[a() = 0] = q. 2) All admission/rejecion decisions are made immediaely upon arrival, so ha admied daa is necessarily served. 3) The minimum average power funcion Φ(x) is non-linear over he inerval ρλ/2 x ρλ, and hence (by convexiy): Φ (ρλ) > Φ(ρλ) Φ(ρλ/2) ρλ/2 (28) where Φ (ρλ) is he righ derivaive of he minimum energy funcion a he poin ρλ. We furher resric aenion o he class of ergodic scheduling policies wih well defined ime averages. Theorem 2: If a conrol policy of he ype described above yields a hroughpu of a leas ρλ and has an average energy expendiure P such ha: P Φ(ρλ) 1/V (29) hen average congesion (and hence average delay) mus be greaer han or equal o Ω(log(V )). Proof: Assume ha ρλ saisfies (28), and define he consan β as follows: β =Φ (ρλ) Φ(ρλ) Φ(ρλ/2) ρλ/2 Noe ha β > 0. Consider a conrol policy as described in he saemen of he heorem. Le U and P represen he ime average queue backlog and he ime average power expendiure, respecively. Assume ha (29) holds. Furher, define δ as he ime average fracion of ime ha µ() < ρλ/2, where µ() is he acual amoun of daa ransmied during slo. Tha is: δ= 1 1 lim 1 [ µ(τ)<ρλ/2] where 1 X is an indicaor funcion ha is 1 whenever condiion X is rue, and is zero oherwise. We make he following claims (proven below): Claim 1: δ 2/(V βρλ) Claim 2: There exis posiive consans C and c (ha do no depend on V or U) such ha: δ Cq cu Combining Claims 1 and 2 yields: Cq cu 2/(V βρλ) Taking he logarihm of boh sides and shifing erms yields: U log(v βρλc/2) c log(1/q) esablishing he resul. To complee he proof, below we prove Claims 1 and 2. Proof: (Claim 1) Define µ 1 as he condiional ime average rae of µ() given ha µ() ρλ/2, and define P 1 as he condiional ime average power expendiure associaed wih such ransmissions. Similarly, define µ 2 and P 2 as he condiional ime averages given ha µ() < ρλ/2.

9 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 54, NO. 3, PP , MARCH Noe by definiion ha 0 µ 2 ρλ/2. Because oal hroughpu is greaer han or equal o ρλ, we have: ρλ (1 δ)µ 1 + δµ 2 Rearranging erms in he above inequaliy yields: Likewise, we have: µ 1 ρλ + δ(ρλ µ 2) (1 δ) (30) P = (1 δ)p 1 + δp 2 (1 δ)φ(µ 1 ) + δφ(µ 2 ) (31) where (31) follows because Φ(x) is defined as he minimum average energy required o suppor an average ransmission rae of x, and hence is less han or equal o he average power of any paricular sraegy ha achieves a ransmission rae of a leas x. Specifically, recall here ha he channel is saic, so ha he power allocaion sraegy used o achieve he condiional ransmission rae µ 1 over he special slos in which µ() ρλ/2 can be used over any slo o achieve an uncondiional ime average ransmission rae of a leas µ 1 wih a ime average power equal o P 1. Using (30) in (31) and noing ha Φ(x) is non-decreasing and convex, we have: ( P (1 δ)φ ρλ + δ(ρλ µ ) 2) + δφ(µ (1 δ) 2 ) [ (1 δ) Φ(ρλ) + Φ (ρλ) δ(ρλ µ ] 2) + δφ(µ (1 δ) 2 ) [ = Φ(ρλ) + δ(ρλ µ 2 ) Φ (ρλ) Φ(ρλ) Φ(µ ] 2) ρλ µ [ 2 ] Φ(ρλ) + δ(ρλ µ 2 ) Φ Φ(ρλ) Φ(ρλ/2) (ρλ) ρλ/2 = Φ(ρλ) + δ(ρλ µ 2 )β where he inequaliies follow by convexiy of Φ(x) ogeher wih he fac ha 0 µ 2 ρλ/2. Therefore, P Φ(ρλ) + δβρλ/2, and hence by (29) we have 1/V δβρλ/2, proving he claim. Proof: (Claim 2) Le be a ime a which he sysem is in seady sae, so ha E {U()} = U. By he Markov inequaliy, we have P r[u() 2U] 1/2. The probabiliy ha U() 2U and ha we hen have k consecuive slos over which here are no arrivals is hus a leas (1/2)q k. Le k = 2U/(ρλ/2) + 1. If here are no arrivals over his se of k slos, we know here mus be a leas one of he k slos in which fewer han ρλ/2 unis of daa were served. Indeed, if all k slos served a leas ρλ/2 unis of daa, hen he oal amoun of ransmied daa over hese slos would be a leas kρλ/2, which is greaer han 2U and hence a conradicion. Wihou loss of generaliy, assume he sysem is in seady sae a ime 0, and divide he imeline ino successive frames of size k slos. The ime average rae δ a which we serve fewer han ρλ/2 bis is greaer han or equal o (1/k) imes he probabiliy ha a paricular frame experiences such an even, and so δ (1/2)q k /k. Thus: δ (1/2)q k /k (1/2)q [4U/(ρλ)+2] /[4U/(ρλ) + 2] = Cq cu U + θ where C = ρλq 2 /8, c = 4/(ρλ), and θ = ρλ/2. Le d = 1/ log(1/q). I is no difficul o show ha q d(u+θ) 1/(U + θ) (using he fac ha e x 1/x for all x > 0), and hence: δ Cq cu q d(u+θ) = Cq cu where we defined c = c+d and C = Cq dθ. This proves Claim 2. While Theorem 2 is presened for he case of saic channels, we conjecure ha he analysis can be exended o prove ha he logarihmic delay radeoff is also necessary in he case of dynamic channels. VII. MULTI-USER SYSTEMS The algorihm can easily be exended o muli-user sysems wih L links wih a vecor link sae process S() = (S 1 (),..., S L ()), a vecor valued rae-power funcion C(P (), S()), and a vecor arrival process A() = (A 1 (),..., A L ()). In his case, we have acual queues U() = (U 1 (),..., U L ()) and virual queues X() = (X 1 (),..., X L ()), each of which is updaed according o queueing equaions similar o (3) and (10). Le λ = (λ 1,..., λ L ) represen he inpu rae vecor, and for simpliciy assume ha λ i > 0 for all i. The minimum energy funcion Φ(λ) for his muli-user problem specifies he minimum average sum power expended by he sysem, minimized over all possible policies ha suppor he inpu raes λ. For a given value ρ < 1, he goal is o design a sraegy ha ensures a hroughpu vecor of a leas ρλ, while pushing average power expendiure arbirarily close o he arge value Φ = Φ(ρλ). Suppose ha he power vecor P () is conained wihin a compac se Π every imeslo, where Π represens he se of accepable power allocaion vecors. The muli-user policy observes he curren channel sae S() and he curren queue backlogs X() and U(), and chooses a power vecor P () = (P 1,..., P L ) every slo o opimize: Maximize: L [ i=1 Ci (P, S()) ( X i () ω i e ωi(qi Ui())) ] V P i subjec o P () Π. The virual queues X i () are hen updaed as follows: X i ( + 1) = max[x i () µ i (), 0] + (ρ + ɛ)a i () where µ i () = C i (P (), S()). Each U i () queue operaes according o he finie buffer queueing equaion (3) wih a buffer size Q i and wih an inpu process A i () and server rae process µ i (). This muli-channel, muli-user algorihm yields a logarihmic energy-delay radeoff for appropriaely chosen {Q i }, {ω i }, ɛ values. Specifically, for each i {1,..., L}, le

10 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 54, NO. 3, PP , MARCH σi 2 be any consan such ha he following holds for all Z() (where Z() =[U(); X()]): σ 2 i E { (µ i () A i ()) 2 Z() } Furher suppose ha for all i, he he consans ω i are chosen o be posiive and o saisfy: ω i e ωiµmax λ i (1 ρ ɛ)/σ 2 i (32) Theorem 3: (Muli-User Packe Dropping) For a given value ρ < 1 and a fixed conrol parameer V 1, if parameers ɛ, {ω i }, and {Q i } are chosen so ha ɛ = (1 ρ)/(2v ), each ω i is posiive and saisfies (32), and Q i = log(x i V )/ω i, where x i is any value ha saisfies: x i 4µ maxlbe ωiµmax λ 2 i ω i(1 ρ ɛ)(1 ρ) where B is defined in (18), hen: 1 1 L (a) lim sup i=1 E {P i(τ)} Φ + O(L/V ) (b) U i () Q i for all, where Q i = O(log(V )) for all i (c) lim inf 1 1 E { µ i(τ)} ρλ i for all i The proof of he above heorem is similar o he proof for he single-user case, and is omied for breviy. VIII. AN ADAPTIVE-THRESHOLD POLICY FOR CONSTANT-FACTOR DELAY IMPROVEMENT Our dynamic packe-dropping policy was proven o achieve an opimal [O(1/V ), O(log(V ))] energy-delay radeoff. Specifically, he policy ensures ha no more han a fracion 1 ρ of packes are dropped, ha ime average power expendiure is wihin O(1/V ) from opimaliy, and ha queue backlog U() saisfies U() Q for all ime, where Q = O(log(V )) (see Theorem 1). However, while he Q parameer is logarihmic in V, i was chosen o have a consan coefficien ha is large enough o analyically ensure ha he edge probabiliy α saisfies α λɛ/µ max. Our analysis was conservaive, and in simulaions i was observed ha edge evens were very rare. Indeed, i was found ha he condiion α λɛ/µ max was ypically saisfied under he much less conservaive hreshold Q = Q/15 (see simulaions in Secion IX). This demonsraes ha, while he policy achieves he opimal [O(1/V ), O(log(V ))] asympoic performance, average delay can be furher reduced by a significan consan facor. In his secion we design a modified policy ha preserves he same analyical delay guaranees, bu ha adapively adjuss he Q parameer o yield significan consan facor improvemens in delay, as observed in he simulaions of Secion IX. The policy uses a novel ype of virual queue ha accumulaes he excess amoun by which he edge probabiliy has violaed is ime average consrain. The Q parameer is adjused in response o his virual queue. For simpliciy, in his secion we consider only he single-queue problem, alhough he echnique readily exends o he muli-user scenario reaed in Secion VII. A. The Threshold-Adapive Packe Dropping Policy Fix he accepance raio ρ < 1. Again le V be a posiive conrol parameer ha affecs he energy-delay radeoff, and define consans ɛ, ω, and x as in Theorem 1, where ɛ = (1 ρ)/(2v ), ω is posiive and saisfies (16), and x is any posiive value ha saisfies (20). The modified algorihm uses a ime varying maximum buffer size Q(), wih queueing updae equaion as follows: U( + 1) = min[q(), max[u() µ(), 0] + A()] (33) The value of Q() is chosen so ha Q min Q() Q max for all, where Q max = log(xv )/ω. Tha is, Q max is he same as he consan value Q used in he original algorihm of Theorem 1. We se he minimum Q value o be Q min = max[q max /f, 10µ max ], where f is a suiably defined consan reducion facor. In our simulaions we choose he reducion facor f o be beween 15 and 40. The Q() value is modified over ime in response o a new virual queue, as follows: Le Y () be a new virual queueing process wih Y (0) = 0, and wih updae equaion: Y ( + 1) = max[y () ɛλ/µ max, 0] + 1 α () (34) where 1 α () is an indicaor funcion ha is 1 if and only if an edge even occurs on slo : { 1 if U() < µmax 1 α () = 0 if U() µ max I is clear ha if he virual queue Y () is sable, hen he lim sup ime average edge probabiliy α is less han or equal o ɛλ/µ max, as desired. Le θ be a posiive value ha defines an accepable number of imeslos by which he ime average edge probabiliy can exceed is desired upper bound ɛλ/µ max (we use θ = 5 hroughou). The hreshold-adapive algorihm is designed so ha Y () consisenly reurns o a value below θ. This is accomplished by ensuring ha Q() increases and remains a Q max whenever Y () is above he θ hreshold for a long duraion of ime. Specifically, le s = (Q max Q min )/100 be he addiive incremen by which Q() can be increased (or decreased), so ha: Q() {Q min, Q min + s, Q min + 2s,..., Q max } Iniialize he adapive Q hreshold o Q(0) = Q min. For each slo {0, 1, 2,...}, le Q cap () be he smalles value of he adapive Q hreshold under which no edge even has occurred (so ha here exiss a ime τ such ha U(τ) < µ max and Q(τ) = Q cap () s). Iniialize Q cap (0) = Q min + s. Every slo, afer he queueing equaion (33) is updaed, Q() is updaed o Q( + 1) as follows: Q( + 1) = min[q() + s, Q max ] if Y () > θ, if U( + 1) = Q(), and Q() < Q cap (). Q( + 1) = max[q() s, Q min ] if Y () θ, if U( + 1) Q() s, and no edge even has occurred wihin he pas 1000 slos.

11 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 54, NO. 3, PP , MARCH Q( + 1) = Q() if neiher of he above evens are saisfied. Inuiively, he above policy decreases Q() when Y () is small, as a small Y () indicaes a low ime average number of edge evens and suggess he curren Q value is unnecessarily large. Noe ha Q() can only decrease if no edge evens have occurred wihin he pas 1000 slos (a heurisic ha enables a quicker response o comba edge evens), and is no decreased unless U( + 1) Q() s (so ha decreasing he Q value would no force any daa o be dropped on slo + 1). Alernaively, Q() is increased when Y () is large in order o reduce he fuure rae of edge evens. The Q cap () parameer is added so ha Q() only increases when an edge even has acually occurred under a Q value ha was greaer han or equal o is curren value. This is useful because edge evens ypically disappear alogeher once Q is large enough, so ha here is no value in making Q larger unless more edge evens are experienced. The requiremen ha Q() canno be increased unless U( + 1) = Q() ensures ha he hreshold is only adjused when queue backlog is a he level of he old hreshold, a requiremen ha plays a crucial role in our Lyapunov analysis of he above sraegy (provided in he nex sub-secion). Threshold-Adapive Packe Dropping Policy: Every imeslo, observe he curren channel sae S() and he curren queue backlogs U(), X(), and Y (). Then: 1) Allocae power P () = P, where P solves: Maximize: Subjec o: C(P, S())(X() ωe ω(q() U()) ) V P 0 P P max 2) Ierae he virual queue X() according o (10), using µ() = C(P (), S()). 3) Ierae he virual queue Y () according o (34). 4) Emulae he finie buffer queue U() according o (33). 5) Updae he Q() value as specified above. Theorem 4: (Threshold-Adapive Algorihm Performance) For any parameer V > 0, for values ɛ, ω, x chosen as above, and for any posiive θ value, he hreshold-adapive packe dropping policy yields he same guaranees specified in pars (a), (b), and (c) of Theorem 1 for he original packe dropping algorihm. Thus, he hreshold-adapive policy preserves he same analyical delay guaranees as he original algorihm, bu experimenally achieves much beer delay performance (see simulaions in Secion IX). Choosing a hreshold value θ = 5 allows he ime average edge rae o be 5 slos over is desired rae before Q() is increased. A larger θ value would make Q() less likely o increase, a he expense of expanding he amoun of ime required for he ime average edge probabiliy o begin averaging ou o a value less han ɛλ/µ max. The adapive Q() hreshold creaes a non-rivial modificaion of he sysem dynamics of he original algorihm, and below we presen a proof of Theorem 4. B. Performance Analysis of he Threshold-Adapive Policy Define he following ime-dependen Lyapunov funcion: L(U(), ) = e ω(q() U()). Again define he vecor of queue backlogs Z() =(U(), X()), and define: Ψ(Z(), ) =L(U(), ) (X())2 Using he same drif analysis as Lemma 6 (in Appendix A), we have: (Z()) + V E {P () Z()} = L (Z()) + J (Z()) +V E {P () Z()} where J (Z()) saisfies he same bound as in par (a) of Lemma 7. The compuaion o bound L (Z()) is similar o ha given in Appendix A: Because U( + 1) min[q(), U() µ() + A()], he inequaliy (37) from Appendix A is modified o: and hence: e ω(q() U(+1)) e ω(q() U()) e ω(µ() A()) + 1 e ω(q(+1) U(+1)) [e ω(q(+1) U(+1)) e ω(q() U(+1)) ] +e ω(q() U()) e ω(µ() A()) + 1 (35) The erm in brackes on he righ hand side of (35) can be bounded by: [e ω(q(+1) U(+1)) e ω(q() U(+1)) ] e ωs 1 (36) Inequaliy (36) is rue because he lef hand side is only posiive if he hreshold is increased on slo (so ha Q() < Q( + 1)), and any increase is by exacly s and occurs only when U( + 1) = Q(). Therefore, using (35) and (36) and following he same seps as in Appendix A, we have he following bound: (Z()) + V E {P () Z()} B + (e ωs 1) + V E {P () Z()} ωe ω(q() U()) [λ E {µ() Z()} λ(1 ρ ɛ)/2] X()[E {µ() Z()} (ρ + ɛ)λ] Because he hreshold-adapive dropping policy makes power allocaion decisions o minimize he righ hand side of he above drif expression, we have: (Z()) + V E {P () Z()} B + (e ωs 1) + V E {P () Z()} ωe ω(q() U()) [λ E {µ () Z()} λ(1 ρ ɛ)/2] X()[E {µ () Z()} (ρ + ɛ)λ] where P () corresponds o any alernaive feasible power allocaion policy, and µ () = C(P (), S()). Consider he queue backlog-independen policy from (22) and (23) ha yields E {µ ()} = (ρ + ɛ)λ and E {P ()} = Φ((ρ + ɛ)λ). We hus have: (Z()) + V E {P () Z()} B + (e ωs 1) + V Φ((ρ + ɛ)λ) ωe ω(q() U()) [λ(1 ρ ɛ)/2]