Opportunism, Backpressure, and Stochastic Optimization with the Wireless Broadcast Advantage

Transcription

1 ASILOMAR CONFERENCE ON SIGNALS, SYSTEMS, AND COMPUTERS, PACIFIC GROVE, CA, OCTOBER Opporunism, Backpressure, and Sochasic Opimizaion wih he Wireless Broadcas Advanage Michael J. Neely, Rahul Urgaonkar Universiy of Souhern California hp://www-rcf.usc.edu/ mjneely Absrac This paper provides a uorial reamen of recen sochasic nework opimizaion echniques, including Lyapunov nework opimizaion, backpressure, and max-weigh decision making. A new echnique of place holder bis ha improves delay for neworking problems wih general coss is also presened. An example applicaion is given for he problem of energy-aware scheduling and rouing in a wireless mobile nework wih channel errors and muli-receiver diversiy. The Diversiy Backpressure Rouing algorihm (DIVBAR, Neely and Urgaonkar 2006, 2008) is illusraed and simulaed in comparison o he Exremely Opporunisic Rouing sraegy (ExOR, Biswas and Morris 2005). I. INTRODUCTION This paper provides a uorial reamen of a recen heory of sochasic nework opimizaion. This heory provides online conrol sraegies for ime varying neworks wih general classes of penalies, rewards, and uiliy funcions. We firs describe he general echnique, which involves conceps of Lyapunov opimizaion, backpressure, and max-weigh decision making. This maerial is aken largely from [1], where a more deailed reamen is given. We hen provide a new exension of he heory ha uses place holder bis o improve congesion and delay in neworks wih general coss [2]. Finally, we presen an example applicaion for a wireless mobile ad-hoc nework wih muli-receiver diversiy. In his nework, a single wireless ransmission can be overheard by muliple receivers, each wih differen success probabiliies. The hroughpu and energy-opimal Diversiy Backpressure Rouing algorihm (DIVBAR) from [3] [4] is presened for his conex. An example simulaion ha compares DIVBAR o he Exremely Opporunisic Rouing algorihm (ExOR) of [5] is given, along wih a simulaion of an enhanced E- DIVBAR algorihm ha combines backpressure and shorespah echniques for furher delay improvemen. II. BACKGROUND The echnique of using Lyapunov drif o sabilize a mulihop packe radio nework was inroduced by Tassiulas and Ephremides in [6], where backpressure rouing and maxweigh scheduling principles are derived. This echnique has had wide success in developing dynamic algorihms for sabiliy in compuer neworks [7] [8] [9] [10], wireless sysems This maerial is suppored in par by one or more of he following: he DARPA IT-MANET program gran W911NF , he NSF gran OCE , he NSF Career gran CCF wih opporunisic scheduling [11] [12] [13], and mobile adhoc neworks [14]. Nework sabiliy can be achieved in hese sysems wihou requiring knowledge of raffic arrival raes or channel probabiliies. An exended Lyapunov drif heorem ha allows join sabiliy and performance opimizaion is developed in [15] [16] [17] o rea uiliy-opimal flow conrol in sochasic neworks. This is exended in [18] for energy opimizaion, and in [1] for a general class of sochasic penaly, reward, and uiliy merics. Performance of hese merics can be pushed o wihin O(1/V ) of opimaliy (where V is a conrol parameer ha can be made arbirarily large), wih a corresponding radeoff in average nework congesion and delay ha is O(V ). This can be applied o rea merics of reliabiliy [20], disorion [2], revenue [21] [22], and o rea neworks wih non-ergodic, non-repeaable mobiliy [23]. This sochasic nework opimizaion echnique is relaed o classical dualiy heory and saic convex programming (see [15] [14] [1] for discussion of he similariies, and [24] [25] for applicaions of backpressure o disribued compuaion). An ineresing observaion is ha sochasic problems are ofen easier o solve han corresponding saic problems, as hey do no require convexiy assumpions (he ime average objecive ends o convexify any non-convexiies of he problem). Relaed alernaive approaches o sochasic nework opimizaion are developed in [26] [27] [28]. Work in [26] considers scheduling in a wireless downlink (similar o [15] [16]), and uses a fluid model ransformaion o relae he sochasic problem o a saic problem. Opimaliy can be approached wih his mehod if he scaled queue backlogs are assumed o converge o appropriae Lagrange mulipliers. Work in [27] reas neworks wih more general penalies and rewards, and uses a relaed fluid model ransformaion ogeher wih a primal-dual algorihm from saic convex opimizaion heory. Work in [28] considers uiliy maximizaion via a sochasic gradien. This paper focuses on he echnique of [1]. This echnique does no require fluid model ransformaions, and direcly leads o he explici performance-delay radeoff [O(1/V ), O(V )]. While his radeoff holds for general neworks and merics, we noe ha i is no necessarily opimal for paricular sysems. Indeed, an improved energy-delay radeoff of [O(1/V ), O( V )] is shown o be opimal for wireless downlinks [29] [30], and a uiliy-delay radeoff of [O(1/V ), O(log(V ))] is shown o be opimal for sysems wih flow conrol [31] [32]. The work in [30] [31] [32] exends he sochasic nework opimizaion

2 ASILOMAR CONFERENCE ON SIGNALS, SYSTEMS, AND COMPUTERS, PACIFIC GROVE, CA, OCTOBER echniques of [1] o achieve hese opimal delay radeoffs, alhough he resuling dynamic algorihms are more complex and, for breviy, shall no be considered here. III. STOCHASTIC NETWORK OPTIMIZATION A. A General Sochasic Nework Consider a sochasic queueing nework ha operaes in sloed ime wih normalized slos {0, 1, 2,...}. Le Q represen he se of all queues in he nework, and for each q Q, le Q q () be he curren backlog in queue q on slo (assumed o be non-negaive). The paricular unis of backlog, such as ineger-valued unis of packes or real-valued unis of bis, can be assigned as appropriae for a given problem. Le Q() = (Q q ()) q Q be he vecor of all queue backlogs. This vecor changes from slo o slo according o a parially conrollable sochasic process, which depends on a nework sae variable S() and a conrol acion I(). The nework sae variable S() represens an unconrollable source of randomness associaed wih he nework condiion on slo (such as node locaions or channel condiions), and is assumed o evolve independenly of any conrol acions aken in he nework. For convenience, hroughou his paper we assume ha S() is i.i.d. over slos and akes values on some absrac (possibly infinie) sae space S, alhough all of our resuls hold also when S() evolves according o a finie sae, irreducible (possibly non-aperiodic) Markov chain [12] [14] [20] [22]. A he beginning of slo, he nework conroller observes he curren S() and he curren Q(), and makes a conrol acion I() I S(), where I S() is an absrac (possibly infinie) se of all feasible conrol opions under nework sae S(). The conrol acion deermines how queue backlog is added, removed, or ransferred from one queue o anoher in he nework, and may involve a collecion of flow conrol, rouing, and ransmission rae scheduling decisions. On a given slo, he ransiion probabiliies associaed wih he queue vecor are deermined only by he curren Q(), S(), and I() I S() : Q() S(),I() Q( + 1) (1) The equaion (1) can be viewed as a Markov relaion. However, radiional Dynamic Programming and Markov Decision Theory approaches o his problem involve offline compuaions wih very high complexiy and require full a-priori knowledge of all sysem saisics. Our approach is quie differen and allows for simple online decision making, ofen wihou requiring he underlying probabiliy disribuions, provided ha he sysem has a paricular general srucure o be made precise in he nex secion. To begin, we assume he queueing dynamics saisfy he following inequaliy for all q Q: 1 Q q (+1) max[q q () Rq ou (I(), S()), 0]+Rq in (I(), S()) (2) 1 The inequaliy in (2) is ypical, as he Rq in ( ) and Rou q ( ) funcions should depend only on I() and S() and no on queue backlog. Hence, hese funcions represen ransmission raes offered by he nework on slo, alhough hese raes may no fully be uilized if here is lile or no backlog o send (see [1]). For example, we migh have Rq in () = A q() + Pa Ω q() Rou a (), where Aq() is a random amoun of exogenous arrivals, and Ω q() is he se of all queues a ha are currenly ransmiing o q. where Rq ou ( ) represens he (poenially random) amoun of backlog ha can be shifed ou of queue q under nework sae S() and conrol acion I(), and Rq in ( ) is he (poenially random) amoun ha can arrive (considering boh exogenous arrivals from he ranspor layer, and possible endogenous arrivals from oher queues). Such a srucure is ypical for single or muli-hop neworks, and in paricular i applies o all he neworks reaed in he references of his paper (see [1] for examples and more deails on he queueing dynamics for neworks). We furher assume ha he Rq ou ( ) and Rq in ( ) values have bounded condiional second momens, regardless of he conrol acion I() aken on slo. B. Sochasic Penalies We say ha he queueing nework is srongly sable if: lim sup 1 1 E {Q q (τ)} < for all q Q We consider he problem of minimizing he ime average of some general nework penaly funcion, subjec o nework sabiliy and o a collecion of addiional ime average penaly consrains. Specifically, le p(i(), S()) be a random penaly process associaed wih he curren nework sae S() and conrol acion I() for slo. Le x k (I(), S()) be addiional random penaly processes for k {1,..., K}. Noe ha p( ) and x k ( ) can be deerminisic funcions, in which case we do no require any convexiy or coninuiy assumpions, or can be random funcions. 2 We assume only ha hey are non-negaive. Define he following ime average expecaions: p 1 1 = lim sup x k = lim sup 1 1 E {p(i(τ), S(τ))} E {x k (I(τ), S(τ))} Le x =(x 1,..., x K ). Our objecive is o design a nework conrol algorihm ha achieves he following: Minimize: p Subjec o: 1) x x av 2) Nework Sabiliy where x av is a vecor of posiive average penaly consrains. For example, he penaly funcion p( ) migh represen he sum power expended in he nework, and he addiional penalies x k ( ) migh represen powers expended by each paricular node of he nework, which enforces average power consrains a each node [18]. Alernaively, some of he penalies migh represen signal processing disorions [2]. This framework can be used o rea rewards g(i(), S()) by defining a penaly process as a negaive reward process: p(i(), S()) = g max g(i(), S()), assuming ha rewards are upper bounded by a finie consan g max. 2 The work in [1] reas all penalies as deerminisic funcions, alhough random funcions p(i(), S()) can be reaed in his way by using a corresponding deerminisic funcion ˆp(I(), S()) =E {p(i(), S()) I(), S()}.

3 ASILOMAR CONFERENCE ON SIGNALS, SYSTEMS, AND COMPUTERS, PACIFIC GROVE, CA, OCTOBER The above problem is in fac a special case of he more general sochasic nework opimizaion framework developed in [1]. Indeed, he work in [1] considers he problem wih K differen penalies x = (x 1,..., x K ) and M differen consrains, where he objecive is o minimize f 0 (x) subjec o consrains f m (x) b m for m {1,..., M}, where f 0 ( ), f 1 ( ),..., f M ( ) are general convex muli-variable funcions ha are non-decreasing in each variable. The objecive of minimizing a nonlinear funcion of a ime average is quie differen han minimizing he ime-average iself, and can be solved by inroducing auxiliary variables and flow sae queues, see [1] and [16] [17]. C. Virual Queues and Lyapunov Drif For each of he penaly consrains x k x k,av (for k {1,..., K}), we creae a virual queue Z k () wih updae equaion as follows: Z k ( + 1) = max[z k () x k,av, 0] + x k (I(), S()) (3) This queue is implemened only in sofware, and has a virual consan service rae of x k,av and a virual inpu process equal o he penaly x k (I(), S()) incurred by our conrol acion on slo. Ensuring ha such a queue is srongly sable implies ha he lim sup ime average expecaion of he inpu process is less han or equal o he virual service rae x k,av, and hence ensures ha he ime average penaly consrain x k x k,av is saisfied [18]. This ransforms he consrained opimizaion problem ino a pure problem of sabilizing (virual and acual) queues while minimizing he ime average objecive p. Such virual queues can be viewed as a sochasic version of a Lagrange muliplier, and were inroduced in [18] [19] o maximize nework hroughpu subjec o average power consrains. Le Z() = (Z 1 (),..., Z K ()) be he vecor of virual queues. Recall ha Q() is he vecor of acual queues, and define Θ() = (Q(), Z()) as he combined queue vecor. Define he following quadraic Lyapunov funcions: L Z (Θ()) = 1 K 2 Z k() 2, L Q (Θ()) = 1 2 q Q Q q() 2 L(Θ()) =L Z (Θ()) + L Q (Θ()) Define he one-sep condiional Lyapunov drif Z (Θ()), Q (Θ()), and (Θ()) as follows: Z (Θ()) Q (Θ()) (Θ()) = E {L Z (Θ( + 1)) L Z (Θ()) Θ()} = E {L Q (Θ( + 1)) L Q (Θ()) Θ()} = Z (Θ()) + Q (Θ()) (4) Squaring he equaion (3) and aking expecaions, i can be shown via a sandard argumen ha Z (Θ()) Z (Θ()), where (see, for example, [1]): Z (Θ()) =B Z K Z k ()E {x k,av x k (I(), S()) Θ()} (5) where B Z is a finie consan ha depends on he maximum second momen of he penaly processes, assumed o be upper bounded regardless of he conrol acion I(). Similarly, squaring (2) yields Q (Θ()) Q (Θ()), where: Q q ()E { Rq ou (I(), S()) Θ() } Q (Θ()) =B Q q Q + q Q Q q ()E { R in q (I(), S()) Θ() } (6) where B Q is a finie consan ha depends on he maximum second momens of he Rq ou ( ) and Rq in ( ) processes. D. Lyapunov Opimizaion and he Max-Weigh Algorihm Here we presen he basic Lyapunov opimizaion heorems required for our sochasic analysis (see proofs in [1]). Le Θ() = (Θ q ()) q D be any sochasic vecor of queue backlogs ha evolves in discree ime, where D is an index se for he queues. Le L(Θ()) be a nonnegaive funcion of he queue backlog vecor, and define (Θ()) =E {L(Θ( + 1)) L(Θ()) Θ()}. Theorem 1: (Lyapunov drif [1]) Le f() and g() be any sochasic processes, possibly relaed o Θ(). If for all and all Θ() we have: hen for all we have: (Θ()) E {g() f() Θ()} 1 1 E {f(τ)} 1 1 E {g(τ)} + E {L(Θ(0))} Now le x() be any (possibly negaive) sochasic penaly process associaed wih he sysem, and le x represen a arge penaly. Theorem 1 implies he following. Theorem 2: (Lyapunov Opimizaion [1] [18] [15]) If here are consans B 0, ɛ 0, V 0 such ha he following holds for all and all Θ(): (Θ()) + V E {x() Θ()} B ɛ q D hen: lim sup 1 1 Θ q () + V x x x + B/V q D E {Θ q(τ)} B+V (x x inf ) ɛ where x is he lim sup ime average expecaion of x() and x inf is he he lim inf ime average expecaion of x(). Thus, if he condiion of Theorem 2 holds, he ime average penaly x is a mos B/V beyond he arge x. If V is a conrol parameer ha can be chosen independenly of B, hen B/V can be made arbirarily small, wih a corresponding linear radeoff in ime average congesion. For our sysem, we can define x() = p(i(), S()). Hence, Theorem 2 suggess he conrol sraegy ha observes S(), Θ() on each slo and chooses I() I S() o minimize (subjec o a given Θ()): (Θ()) + V E {p(i(), S()) Θ()} (7) where he above expecaion includes averaging over all possible S() values. However, given ha we can firs observe S(), he opimal policy is o greedily minimize he inside of he expecaion subjec o S() knowledge (recall ha E {X} =

4 ASILOMAR CONFERENCE ON SIGNALS, SYSTEMS, AND COMPUTERS, PACIFIC GROVE, CA, OCTOBER E Y {E {X Y }} for any random variables X and Y ). 3 Hence, we do no require knowledge of he S() probabiliies. Acually, achieving he minimum of (7) o wihin any addiive consan will suffice. Using (Θ()) as defined in he previous subsecion and noing ha (Θ()) Z (Θ())+ Q (Θ()) leads o he following: The Generalized Max-Weigh Policy: Every imeslo, observe S() and Θ(), and choose I() I S() o greedily minimize (subjec o a given Θ()): Z (Θ()) + Q (Θ()) + V E {p(i(), S()) Θ()} (8) wih Z ( ), Q ( ) defined in (5), (6). Then updae he virual queues Z k () for each k {1,..., K} according o (3) (he acual queue dynamics proceed according o (2)). Using he definiions of Z (Θ()) and Q (Θ()), he minimizaion of (8) is equivalen o choosing I() I S() o minimize: Q q ()[ q Q V ˆp(I(), S()) + K Z k ()ˆx k (I(), S()) ou ˆR q (I(), S()) ˆR in q (I(), S())] (9) where ˆp(I(), S()) =E {p(i(), S()) I(), S()}, and where ˆx k (I(), S()), ˆRou in q (I(), S()), and ˆR q (I(), S()) are similarly defined as he condiional expecaions given I() and S(). These funcions of I() and S() are assumed o be known. Noe ha for a given slo, he observed nework sae S() and queue backlogs Q() and Z() ac as known consans in he minimizaion of (9). Hence, his minimizaion does no require knowledge of he probabiliy disribuion of S(). Furher, oher han compuing expecaions associaed wih he ˆp( ), ˆx( ), ˆRou in q ( ), and ˆR q ( ) funcions, i does no require knowledge of any oher saisics ha affec he nework evoluion, such as raffic arrival raes. 4 IV. ALGORITHM PERFORMANCE Consider he following class of conrol policies ha choose I() I S() as a saionary and poenially randomized funcion only of he curren nework sae S(), and independen of queue backlog. We call such policies S-only policies. Noe ha, by he law of large numbers and he assumpion ha S() is i.i.d. over slos, his class of policies leads o well defined sysem ime averages. Le p be he infimum ime average penaly p incurred over he class of S-only algorihms I () ha saisfy he following: E {p(i (), S())} = p (10) E {x k (I (), S())} x k,av, k {1,..., K} E { R ou q (I (), S()) } E { R in q (I (), S()) }, q Q 3 Specifically, E {X Θ} = E S Θ {E {X Θ, S}} for any random variables X, Θ, and S. If X = X(I, S) (i.e., a random funcion ha depends on S and on an addiional, possibly random, inpu I), hen E {X(I, S) Θ} is minimized by he conrol inpu ha chooses I as he funcion of S and Θ ha minimizes E {X(I, S) Θ, S}. 4 The ˆR in q ( ) funcion may include an unconrollable exogenous arrival rae erm λ q ha is no needed o minimize (9), or may include a flow conrol erm ha can be minimized wih knowledge of he new arrivals A q() observed on slo, wihou requiring he disribuion of A q(), see [1]. In he laer case, A q() can be viewed as a par of he curren nework sae S(). We are implicily assuming he above ime average consrains are feasible, and hence p is he infimum ime average penaly over all feasible S-only algorihms. While our Generalized Max-Weigh Policy is no an S-only policy, we shall measure is opimaliy wih respec o p. For many nework objecives, such as hroughpu, average power, fairness, ec., opimaliy can be aained over he class of S-only algorihms, and hence p is he opimal performance over all conrol algorihms [14] [18]. For simpliciy, we assume ha he infimum p is achievable, in he sense of saisfying he consrains of (10), by a paricular S-only policy I (). 5 We mus furher make he following slackness assumpion: There exiss a consan ɛ > 0 ogeher wih an S-only policy I s () such ha: 6 E {x k (I s (), S())} + ɛ x k,av, k {1,..., K} E { Rq ou (I s (), S()) } ɛ + E { Rq in (I s (), S()) }, q Q Theorem 3: (Performance Theorem [1] [18]) Suppose here is a value p max < so ha 0 p(i(), S()) p max always. Under he Generalized Max-Weigh Policy wih any conrol parameer V 0, he nework is sable, he consrains x x av are saisfied, he lim sup ime average penaly p saisfies: lim sup 1 1 E {p(i(τ), S(τ))} p + (B Z + B Q )/V (11) and he ime average queue backlogs saisfy: 1 1 lim sup K E {Q q (τ)} + E {Z k (τ)} q Q (B Z + B Q + V p max )/ɛ (12) Thus, he V parameer can be chosen as desired o affec he [O(1/V ), O(V )] performance-congesion radeoff. Proof: Le I () be he S-only policy ha achieves all feasibiliy consrains (10) and yields E {p(i (), S())} = p. Le I MW () denoe he Max-Weigh policy. Le MW (Θ()) denoe he Lyapunov drif under he Max-Weigh policy I MW () (as defined in (4)), and le MW Z (Θ()) and MW Q (Θ()) denoe he corresponding drif bounds as defined in Secion III-C (equaions (5) and (6)). Le Z(Θ()) and Q(Θ()) be he corresponding drif bounds for he I () policy. Because I MW () (by definiion) minimizes (8), we have: MW (Θ()) + V E { p(i MW (), S()) Θ() } Z(Θ()) + Q(Θ()) + V E {p(i (), S()) Θ()} (13) Now noe ha because I () is S-only and S() is i.i.d. over slos, I () is independen of queue backlogs Θ(), and so: E {x k (I (), S()) Θ()} = E {x k (I (), S())} x k,av where he final inequaliy above is due o he inequaliy consrains of (10). Thus, plugging he inequaliy consrains 5 Else, we can prove he same performance resuls wih a limiing argumen using a sequence of policies I n() ha converge o he performance p. 6 This single ɛ-slackness assumpion ha holds for all consrains simulaneously can be relaxed by replacing i wih muliple ɛ-slackness assumpions ha hold for each consrain individually while all oher consrains are saisfied wih possible 0-slackness.

5 ASILOMAR CONFERENCE ON SIGNALS, SYSTEMS, AND COMPUTERS, PACIFIC GROVE, CA, OCTOBER of (10) ino he definiions of Z(Θ()) and Q(Θ()) given in (5) and (6), i is a simple maer o verify: Z(Θ()) + Q(Θ()) B Z + B Q Define B =B Z +B Q. Using he above inequaliy in (13) yields: MW (Θ()) + V E { p(i MW (), S()) Θ() } B + V p This is in he exac form for using he Lyapunov Opimizaion Theorem (Theorem 2) wih ɛ = 0, x() = p(i MW (), S()), x = p, which proves he performance bound (11). Similarly, o derive he ime average bound on queue backlog, we can use he S-only policy I s () ha yields he slackness consrains. Noe ha he inequaliy (13) holds equally if we plug he I s () policy ino each of he hree erms on he righ hand side, raher han he I () policy (as I MW () minimizes he righ hand side over any oher conrol acion). I is again a simple maer o verify from he slackness consrains and he definiions of s Z(Θ()) and s Q(Θ()) (represening drif bounds under he I s () policy) ha: K s Z(Θ()) + s Q(Θ()) B ɛ Q q () Z k () + q Q which can be plugged ino he righ hand side of (13) o yield he resuling bound (12) upon applicaion of Theorem 2. A. On Disribued Implemenaion and Queue Groupings We noe ha disribued implemenaion of he Generalized Max-Weigh policy is possible when channels are orhogonal and/or ransmission opions are randomized [14] [18] [23]. Consan-facor hroughpu and performance resuls, where sabiliy regions are reduced by a consan facor, are guaraneed when a disribued algorihm comes wihin a consan facor of maximizing (9). This is shown in [15] for sabiliy, in [1] [17] for join sabiliy and hroughpu-uiliy maximizaion, and in [3] for sabiliy and power minimizaion. A differen approach o sable scheduling via maximal machings can also be shown o yield consan-facor resuls [33] [34] [35] [36]. Maximal scheduling is ofen analyzed using queue-grouped Lyapunov funcions ha can provide igher delay guaranees for reduced hroughpu regions [37] [38]. Queue groupings can also be used o prove order-opimal delay for he full-hroughpu max-weigh scheduling algorihm in wireless downlinks wih ON/OFF channels [39]. B. On Delay Reducion and Wors Case Backlog We noe ha nework congesion and delay can ofen be improved by direcly minimizing he drif expression (7), raher han is upper bound ha uses (Θ()) Z (Θ())+ Q (Θ()). The performance bounds of Theorem 3 hold also in his case (he proof is exacly he same). However, congesion is ypically beer because he drif expression (7) uses he acual amoun of backlog ransferred (ofen being a minimum of he offered ransfer rae and he acual backlog available), while he drif bound uses only he offered ransfer rae (he offered rae and he acual amoun ransferred are he same when queue backlog is sufficienly large). Alernaively, one can minimize any expression ha differs by only an addiive consan from eiher he drif expression (7) or is upper bound. This simply adds an addiive consan o he congesion bound of Theorem 3. Such a scenario arises if backlog updaes are delayed so ha acual queue backlogs differ by a consan from hose used in he conrol decision [1]. Queue backlogs can also be augmened by consans o achieve addiional desired behavior while mainaining sabiliy. For example, delay in muli-hop neworks can ofen be improved by adding a shores pah bias erm o each queue backlog. This is used in he Enhanced Dynamic Rouing and Power Allocaion (EDRPC) algorihm of [14] and he Enhanced Diversiy Backpressure Rouing (E-DIVBAR) algorihm of [3], where simulaions demonsrae considerable delay improvemen (see also Secion VI). We furher noe ha in sysems wih flow conrol, he max-weigh policy (or modificaions ha add a consan o he backpressure value) ofen allow wors case queue backlog guaranees [18] [23] [21]. V. PLACE HOLDER BITS FOR DELAY IMPROVEMENT Noe ha he iniial condiion of he queues does no affec he ime average penaly or queue backlog bounds of Theorem 2 or Theorem 3. Suppose now ha he nework dynamics under he Max-Weigh policy (9) saisfy he following propery: Propery 1: There is a non-negaive queue backlog vecor Θ 0 = (Q 0 ; Z 0 ) such ha if Θ(0) Θ 0, hen Θ() Θ 0 for all 0, where inequaliy is aken enrywise. If Propery 1 holds for a non-zero vecor Θ 0 = (Q 0, Z 0 ), we iniialize he virual queues o Z(0) = Z 0, and we iniialize he acual queues so ha here is no acual daa iniially in hese queues, bu here is an amoun Q 0 of fake daa or place holder daa. Specifically, suppose he acual queue backlog is given by ˆQ(), and define: Q() = ˆQ() + Q 0 The Max-Weigh algorihm uses he Q() values in is decisions (9). However, whenever i makes a ransmission decision, i ransmis only acual daa (no fake daa), whenever possible. The resuling sample pah of Q() is equivalen o ha of a sysem wih iniial condiion Θ(0) = (Q 0 ; Z 0 ), and hence yields he exac same ime average bounds on penaly p and queue backlog Q q as specified in Theorem 3. Furher, because Propery 1 holds, we will have (Q(); Z()) (Q 0 ; Z 0 ) for all, and so he acual queue backlog will never drop low enough o require ransmission of fake daa. I follows ha he acual backlog vecor ˆQ() will be exacly an amoun Q 0 lower han Q() for every insan of ime, which reduces he average queue backlog by exacly his amoun, wihou any change in he ime average penaly p. Thus, his fake daa is simply acing as a place holder o properly affec he performance opimizaion. We recenly developed his concep of place holder bis in [2] for a single-hop sysem (he reader is referred o more specific examples given here), alhough i applies equally well in his poenially muli-hop seing. For cos minimizaion problems, he Max-Weigh algorihm of (9) ypically does saisfy Propery 1 for a non-zero vecor Θ 0. Indeed, from

6 ASILOMAR CONFERENCE ON SIGNALS, SYSTEMS, AND COMPUTERS, PACIFIC GROVE, CA, OCTOBER (9) i can be seen ha a queueing node q will ypically no ransmi any daa ou unless he amoun of backlog i currenly has, muliplied by he amoun i will ransmi ou, exceeds a hreshold equal o he penaly incurred by his ransmission muliplied by V. The backlog reducion of place holder daa can be quie significan, and can be successfully applied o energy minimizaion algorihms [18] [3] and oher relaed cos minimizaion algorihms o improve delay. VI. MULTI-RECEIVER DIVERSITY AND DIVBAR This sochasic opimizaion echnique is applied in our prior work [3] o minimize average power in a muli-hop wireless mobile nework wih muli-receiver diversiy. Specifically, a packe ransmission a a given node can poenially be received by a se of neighbors, wih heerogeneous success probabiliies ha depend on he curren opology sae S(). All successful receivers provide ACK feedback, and he ransmiing node sends a final message o indicae which one (if any) should ake responsibiliy for he packe. The deailed nework conrol variables are given in [3], where he Max-Weigh policy is applied o creae a Diversiy Backpressure Rouing Algorihm (DIVBAR) ha is hroughpu and energy opimal. In he case of a single commodiy, orhogonal channels, and no power opimizaion (i.e., V = 0), DIVBAR reduces o having each node ransmi whenever possible, and shifing forwarding responsibiliy o he successful receiver wih he larges differenial backlog (being he difference in queue backlog a he ransmier and receiver), reaining he packe a he ransmier if no receiver has a posiive differenial backlog. If power minimizaion is included (V > 0), he algorihm is more complex and requires insananeous success probabiliies of neighbor nodes under a given opology sae, bu does no require knowledge of raffic raes or of how he opology changes from one slo o he nex. If a mos one packe can be ransmied on a slo, and each ransmission requires 1 uni of power, i can be shown ha DIVBAR saisfies Propery 1 wih Q 0 = (max[ V 1, 0]) q Q (in unis of packes). Figure 1 illusraes a cell-pariioned mobile ad-hoc nework, from Example 2 of [3]. There are 9 source nodes: 3 saionary, 3 locally mobile, and 3 fully mobile. The locally mobile nodes are resriced o move in he shaded cells while he fully mobile nodes can move anywhere in he nework. There are 2 saionary sinks and packes can be delivered o eiher of hem (hus, his is a single commodiy scenario). The mobile Saionary Node Locally Mobile Node Fully Mobile Node Sink Fig. 1. A mobile nework wih wo sinks and heerogeneous mobiliy [3]. nodes perform a Markovian random walk over heir respecive regions (choosing each slo o eiher remain in heir curren cell or o move o an adjacen cell). We assume a mos one packe can be ransmied per cell, bu ransmissions in differen cells are orhogonal. Figure 2 illusraes he performance of DIVBAR for his example (see he deailed model descripion in [3]). This figure also includes new daa ha demonsraes he benefis of backlog reducion achieved by using place holder packes, a no addiional energy cos (as shown in Figure 3). Average Toal Congesion Average Toal Congesion vs V DIVBAR DIVBAR wih place holder packes V Fig. 2. Average oal congesion (i.e., oal packes summed over all queues in he nework) versus V under DIVBAR wih and wihou place holder packes on he example mobile nework of Fig. 1. Average power (per node) Average Power vs V DIVBAR DIVBAR wih place holder packes V Fig. 3. Average power versus V under DIVBAR wih and wihou place holder packes on he example mobile nework of Fig. 1. The wo curves are almos idenical, demonsraing no loss of energy efficiency wih place holder packes. Figure 4 reas a differen nework: he 2-commodiy saic nework from Example 1 in [3] (figure omied for breviy, see [3]). In he muli-commodiy case, DIVBAR selecs he opimal commodiy o ransmi via a rank-ordering ha depends on he curren backlog and recepion success probabiliies of neighboring nodes. Such a commodiy disincion is required for opimaliy, and is quie differen from oher diversiy algorihms (such as ExOR) ha do no opimize over commodiy selecion. We consider only sabiliy here, wihou power opimizaion (so ha V = 0), and compare o he ExOR algorihm of [5] ha uses he heurisic of shifing packes o he successful receiver ha is closes o he desinaion, where proximiy is measured wih respec o a shores pah meric.

7 ASILOMAR CONFERENCE ON SIGNALS, SYSTEMS, AND COMPUTERS, PACIFIC GROVE, CA, OCTOBER Unlike DIVBAR, he ExOR algorihm is no hroughpuopimal, and in his example i becomes unsable a raes ha are a lile more han half he disance o he capaciy region boundary. However, is shores-pah properies allow ExOR o have lower delay in he low rae region. A combined sraegy E-DIVBAR ha uses backpressure and shores pah in he forwarding decision is also illusraed (his is relaed o he mixed backpressure/shores pah enhancemen in [14]). I is shown in [3] ha E-DIVBAR is also hroughpu opimal, and we see in his example ha i has lower delay han boh DIVBAR and ExOR across all inpu raes. Average Toal Occupancy (log scale) ExOR DIVBAR E DIVBAR Average Toal Occupancy vs. Inpu Rae Inpu Rae (packes/slo) Fig. 4. A comparison of DIVBAR, ExOR, and E-DIVBAR for he 2- commodiy saic nework example of [3]. REFERENCES [1] L. Georgiadis, M. J. Neely, and L. Tassiulas. Resource allocaion and cross-layer conrol in wireless neworks. Foundaions and Trends in Neworking, vol. 1, no. 1, pp , [2] M. J. Neely and A. Sharma. Dynamic daa compression wih disorion consrains for wireless ransmission over a fading channel. arxiv: v1, July 24, [3] M. J. Neely and R. Urgaonkar. Opimal backpressure rouing in wireless neworks wih muli-receiver diversiy. Ad Hoc Neworks (Elsevier), 2008, doi: /j.adhoc [4] M. J. Neely. Opimal backpressure rouing for wireless neworks wih muli-receiver diversiy. Proc. of Conf. on Informaion Sciences and Sysems (CISS), Invied paper on Opimizaion of Communicaion Neworks, March [5] S. Biswas and R. Morris. ȨxO Ŗ: Opporunisic muli-hop rouing for wireless neworks. Proc. of Sigcomm, [6] L. Tassiulas and A. Ephremides. Sabiliy properies of consrained queueing sysems and scheduling policies for maximum hroughpu in mulihop radio neworks. IEEE Transacaions on Auomaic Conrol, vol. 37, no. 12, pp , Dec [7] P. R. Kumar and S. P. Meyn. Sabiliy of queueing neworks and scheduling policies. IEEE Trans. on Auomaic Conrol, vol.40,.n.2, pp , Feb [8] N. Kahale and P. E. Wrigh. Dynamic global packe rouing in wireless neworks. Proc. IEEE INFOCOM, [9] N. McKeown, V. Ananharam, and J. Walrand. Achieving 100% hroughpu in an inpu-queued swich. Proc. IEEE INFOCOM, [10] E. Leonardi, M. Mellia, F. Neri, and M. Ajmone Marsan. Bounds on average delays and queue size averages and variances in inpu-queued cell-based swiches. Proc. IEEE INFOCOM, [11] L. Tassiulas and A. Ephremides. Dynamic server allocaion o parallel queues wih randomly varying conneciviy. IEEE Transacions on Informaion Theory, vol. 39, pp , March [12] L. Tassiulas. Scheduling and performance limis of neworks wih consanly changing opology. IEEE Trans. on Inf. Theory, May [13] M. Andrews, K. Kumaran, K. Ramanan, A. Solyar, and P. Whiing. Providing qualiy of service over a shared wireless link. IEEE Communicaions Magazine, vol. 39, no.2, pp , [14] M. J. Neely, E. Modiano, and C. E Rohrs. Dynamic power allocaion and rouing for ime varying wireless neworks. IEEE Journal on Seleced Areas in Communicaions, vol. 23, no. 1, pp , January [15] M. J. Neely. Dynamic Power Allocaion and Rouing for Saellie and Wireless Neworks wih Time Varying Channels. PhD hesis, Massachuses Insiue of Technology, LIDS, [16] M. J. Neely, E. Modiano, and C. Li. Fairness and opimal sochasic conrol for heerogeneous neworks. Proc. IEEE INFOCOM, March [17] M. J. Neely, E. Modiano, and C. Li. Fairness and opimal sochasic conrol for heerogeneous neworks. IEEE/ACM Transacions on Neworking, vol. 16, no. 2, pp , April [18] M. J. Neely. Energy opimal conrol for ime varying wireless neworks. IEEE Transacions on Informaion Theory, vol. 52, no. 7, pp , July [19] M. J. Neely. Energy opimal conrol for ime varying wireless neworks. Proc. IEEE INFOCOM, March [20] R. Urgaonkar and M. J. Neely. Opporunisic scheduling wih reliabiliy guaranees in cogniive radio neworks. Proc. IEEE INFOCOM, April [21] M. J. Neely. Opimal pricing in a free marke wireless nework. Proc. IEEE INFOCOM, May [22] L. Huang and M. J. Neely. The opimaliy of wo prices: Maximizing revenue in a sochasic nework. Proc. 45h Alleron Conf. on Communicaion, Conrol, and Compuing, Sep [23] M. J. Neely and R. Urgaonkar. Cross layer adapive conrol for wireless mesh neworks. Ad Hoc Neworks (Elsevier), vol. 5, no. 6, pp , Augus [24] M. J. Neely. Disribued and secure compuaion of convex programs over a nework of conneced processors. DCDIS Conf., Guelph, Onario, July [25] M. Andrews, K. Jung, and A. Solyar. Sabiliy of he max-weigh rouing and scheduling proocol in dynamic neworks and a criical loads. Proc. STOC, June [26] A. Eryilmaz and R. Srikan. Fair resource allocaion in wireless neworks using queue-lengh-based scheduling and congesion conrol. Proc. IEEE INFOCOM, March [27] A. Solyar. Maximizing queueing nework uiliy subjec o sabiliy: Greedy primal-dual algorihm. Queueing Sysems, vol. 50, pp , [28] J. W. Lee, R. R. Mazumdar, and N. B. Shroff. Opporunisic power scheduling for dynamic muliserver wireless sysems. IEEE Transacions on Wireless Communicaions, vol. 5, no.6, pp , June [29] R. Berry and R. Gallager. Communicaion over fading channels wih delay consrains. IEEE Transacions on Informaion Theory, vol. 48, no. 5, pp , May [30] M. J. Neely. Opimal energy and delay radeoffs for muli-user wireless downlinks. IEEE Transacions on Informaion Theory, vol. 53, no. 9, pp , Sep [31] M. J. Neely. Super-fas delay radeoffs for uiliy opimal fair scheduling in wireless neworks. IEEE Journal on Seleced Areas in Communicaions, Special Issue on Nonlinear Opimizaion of Communicaion Sysems, vol. 24, no. 8, pp , Aug [32] M. J. Neely. Inelligen packe dropping for opimal energy-delay radeoffs in wireless downlinks. Proc. of WiOp, Boson, April [33] X. Lin and N. B. Shroff. The impac of imperfec scheduling on crosslayer rae conrol in wireless neworks. Proc. IEEE INFOCOM, [34] X. Wu, R. Srikan, and J. R. Perkins. Scheduling efficiency of disribued greedy scheduling algorihms in wireless neworks. IEEE Transacions on Mobile Compuing, June [35] P. Chaporkar, K. Kar, X. Luo, and S. Sarkar. Throughpu and fairness guaranees hrough maximal scheduling in wireless neworks. IEEE Trans. on Informaion Theory, vol. 54, no. 2, pp , Feb [36] L. Lin, X. Lin, and N. B. Shroff. Low-complexiy and disribued energy minimizaion in muli-hop wireless neworks. Proc. IEEE INFOCOM, [37] S. Deb, D. Shah, and S. Shakkoai. Fas maching algorihms for repeiive opimizaion: An applicaion o swich scheduling. Proc. of 40h Annual Conference on Informaion Sciences and Sysems (CISS), Princeon, NJ, March [38] M. J. Neely. Delay analysis for maximal scheduling in wireless neworks wih bursy raffic. Proc. IEEE INFOCOM, April [39] M. J. Neely. Delay analysis for max weigh opporunisic scheduling in wireless sysems. Proc. 46h Annual Alleron Conf. on Communicaion, Conrol, and Compuing, Sep