Delay Reduction via Lagrange Multipliers in Stochastic Network Optimization

Transcription

1 PROC. OF 7TH INTL. SYMPOSIM ON MODELING AND OPTIMIZATION IN MOBILE, AD HOC, AND WIRELESS NETWORKS (WIOPT), JNE Delay Reducion via Lagrange Mulipliern Sochasic Nework Opimizaion Longbo Huang, Michael J. Neely Absrac In his paper, we consider he problem of reducing nework delay in sochasic nework uiliy opimizaion problems. We sar by sudying he recenly proposed quadraic Lyapunov funcion based algorihms (QLA). We show ha for every sochasic problem, here is a corresponding deerminisic problem, whose dual opimal soluion exponenially aracs he nework backlog process under QLA. In paricular, he probabiliy ha he backlog vecor under QLA deviaes from he aracor is exponenially decreasing in heir Euclidean disance. This suggess ha one can roughly subrac ou a Lagrange muliplier from he sysem induced by QLA. We hus develop a family of Fas Quadraic Lyapunov based Algorihms (FQLA) ha achieve an [O(1/ ), O(log ( ))] performance-delay radeoff. These resuls highligh he nework graviy role of Lagrange Mulipliern nework scheduling. This role can be viewed as he counerpar of he shadow price role of Lagrange Mulipliers in flow regulaion for classic flow-based nework problems. Index Terms Queueing, Dynamic Conrol, Lyapunov analysis, Sochasic Opimizaion I. INTRODCTION In his paper, we consider he problem of reducing nework delay in he following general framework of he sochasic nework uiliy opimizaion problem. We are given a ime sloed sochasic nework. The nework sae, such as he nework channel condiion, ime varying according o some probabiliy law. A nework conroller performs some acion based on he observed nework sae a every ime slo. The chosen acion incurs a cos (since cos minimizaion is mahemaically equivalen o uiliy maximizaion, below we will use cos and uiliy inerchangeably), bu also serves some amoun of raffic and possibly generaes new raffic for he nework. This raffic causes congesion, and hus leads o backlogs a noden he nework. The goal of he conroller is o minimize ime average cos subjec o he consrain ha he ime average oal backlog in he nework is finie. This seing is very general, and many exising works fall ino his caegory. Furher, many echniques have been used o sudy his problem (see [1] for a survey). In his paper, we focus on algorihms ha are buil upon quadraic Lyapunov funcions (called QLA in he following), e.g., [], [3], [4], [5], [6], [7]. These QLA algorihms are easy o implemen, greedy in naure, and are parameerized by a scalar conrol variable. I has been shown ha when he nework sae is Longbo Huang ( longbohu@usc.edu) and Michael J. Neely (web: hp://www-rcf.usc.edu/ mjneely) are wih he Deparmen of Elecrical Engineering, niversiy of Souhern California, Los Angeles, CA 90089, SA. This maerial is suppored in par by one or more of he following: he DARPA IT-MANET program gran W911NF , he NSF gran OCE 05034, he NSF Career gran CCF i.i.d., QLA algorihms can achieve a ime average uiliy ha is wihin O(1/ ) o he opimal. Therefore, as grows large, he ime average uiliy can be pushed arbirarily close o he opimal. However, such close-o-opimal uiliy is usually a he expense of large nework delay. In fac, in [3], [4], [7], i is shown ha an O( ) nework delay incurred when an O(1/ ) close-o-opimal uiliy is achieved. Two recen papers [8] and [9], which show ha i is possible o achieve wihin O(1/ ) of opimal uiliy wih only O(log( )) delay, use a more sophisicaed algorihm design approach based on exponenial Lyapunov funcions. Therefore, i seems ha hough being simple in implemenaion, QLA algorihms have undesired delay performance. However, we noe ha he delay resuls of QLA are usually given in erms of long erm upper bounds of he average nework backlog e.g., [7]. Thus hey do no examine he possibiliy ha he acual backlog vecor (or ime average) converges o some fixed value. Work in [10] considers drif properies owards an invarian backlog vecor, derived in he special case when he problem exhibis a unique opimal Lagrange muliplier. An upper bound on he long erm deviaion of he acual backlog and he Lagrange muliplier vecor is obained. While his suggess Lagrange mulipliers are graviaional aracors, he boundn [10] do no show ha he he acual backlog is very unlikely o deviae significanly from he aracor. In his paper, we focus on obaining sronger probabiliy resuls of he seady sae backlog process behavior under QLA. We firs show ha under QLA, even hough he backlog can grow linearly in, i ypically says close o an aracor, which is he dual opimal soluion of a deerminisic opimizaion problem. In paricular, he probabiliy ha he backlog vecor deviaes from he aracor is exponenially decreasing in disance, which significanly ighens he aracor analysis in [10]. Thimplies ha a large amoun of he daa is kep in he nework simply for mainaining he backlog a he righ level. Therefore, even if we replace hese daa wih some fake daa (denoed as place-holder bis [11]), he performance of QLA will no be heavily affeced. Based on his finding, we propose a family of Fas Quadraic Lyapunov based Algorihms (FQLA), which inuiively speaking, can be viewed as subracing ou a Lagrange muliplier from he sysem induced by QLA. We show ha when he nework sae i.i.d., FQLA is able o achieve wihin O(1/ ) of opimal uiliy wih an O(log ( )) delay guaranee. The developmen of FQLA also provides us wih addiional insighno QLA algorihms and he role of Lagrange mulipliern sochasic nework opimizaion. We now summarize he main conribuions of

2 PROC. OF 7TH INTL. SYMPOSIM ON MODELING AND OPTIMIZATION IN MOBILE, AD HOC, AND WIRELESS NETWORKS (WIOPT), JNE 009 his paper in he following: This paper proves ha in seady sae, he backlog process under QLA is exponenially araced o an aracor. This paper proposes a family of Fas Quadraic Lyapunov based Algorihms (FQLA), which are usually easy o implemen, and can achieve an [O(1/ ), O(log ( ))] performance-delay radeoff for general sochasic opimizaion problems. This paper highlighs a new funcionaliy of Lagrange mulipliers: he nework graviy in nework scheduling. The paper is organized as follows: In Secion II, we se up our noaions. In Secion III, we sae our nework model. We hen review he QLA algorihm and define he deerminisic problem in Secion I. In Secion, we show ha he backlog process under QLA always says close o an aracor. In Secion I, we propose he FQLA algorihm. Secion II provides simulaion resuls. We discuss he graviy role of Lagrange mulipliers and relae QLA o he randomized incremenal subgradien mehod (RISM) [1] in Secion III. II. NOTATIONS R: he se of real numbers R + (or R ): he se of nonnegaive (or non-posiive) real numbers R n (or R n +): he se of n dimensional column vecors, wih each elemen being in R (or R + ) bold symbols x and x T : column vecor and is ranspose x y: vecor x is enrywise no less han vecor y 0: column vecor wih all elemens being 0 III. SYSTEM MODEL In his secion, we specify he general nework model we use. We consider a nework conroller ha operaes a nework wih he goal of minimizing he ime average cos, subjec o he queue sabiliy consrain. The nework is assumed o operae in sloed ime, i.e., {0, 1,,...}. We assume here are r 1 queuen he nework. A. Nework Sae We assume here are a oal of M differen random nework saes, and define S = {s 1, s,..., s M } as he se of possible saes. Each paricular sae indicaes he curren nework parameers, such as a vecor of channel condiions for each link, or a collecion of oher relevan informaion abou he curren nework channels and arrivals. Le S() denoe he nework sae a ime. We assume ha S() i.i.d. every ime slo, and le p si denoe is probabiliy of being in sae, i.e., p si = P r{s() = }. We assume he nework conroller can observe S() a he beginning of every slo, bu he p si probabiliies are no necessarily known. B. The Cos, Traffic and Service A each ime, afer observing S() =, he conroller chooses an acion x() from a se X (si), i.e., x() = x (si) for some x (si) X (si). The se X (si) is called he feasible acion se for nework sae and is assumed o be ime-invarian and compac for all S. The cos, raffic and service generaed by he chosen acion x() = x (si) are as follows: (a) The chosen acion has an associaed cos given by he cos funcion f() = f(, x (si) ) : X (si) R + (or X (si) R in he case of reward maximizaion problems); (b) The amoun of raffic generaed by he acion o queue j is deermined by he raffic funcion A j () = g j (, x (si) ) : X (si) R +, in unis of packes; (c) The amoun of service allocaed o queue j is given by he rae funcion µ j () = b j (, x (si) ) : X (si) R +, in unis of packes; Noe ha A j () includes boh he exogenous arrivals from ouside he nework o queue j, and he endogenous arrivals from oher queues, i.e., he ransmied packes from oher queues, o queue j (See Secion III-C and III-D for furher explanaions). We assume he funcions f(, ), g j (, ) and b j (, ) are ime-invarian, heir magniudes are uniformly upper bounded by some consan δ max (0, ) for all, j, and hey are known o he nework operaor. We also assume ha here exiss a se of acions {x (si)k } k=1,...,r+ i=1,...,m wih x(si)k X (si) such ha { p si k ϑ(si) k [g j (, x (si)k ) b j (, x (si)k )] } ɛ for some ɛ > 0 for all j, wih j ϑ(si) k = 1 and ϑ (si) k 0 for all and k. Tha is, he consrains are feasible wih ɛ slackness. Thus, here exiss a saionary randomized policy ha sabilizes all queues (where ϑ (si) k represens he probabiliy of choosing acion x (si)k when S() = ). In he following, we use: A() = (A 1 (), A (),..., A r ()) T, (1) µ() = (µ 1 (), µ (),..., µ r ()) T, () o denoe he arrival and service vecors a ime. I is easy o see from above ha if we define: hen A() µ() B for all. B = rδ max, (3) C. Queueing, Average Cos and he Sochasic Problem Le () = ( 1 (),..., r ()) T R r +, = 0, 1,,... be he queue backlog vecor process of he nework, in unis of packes. We assume he following queueing dynamics: j ( + 1) = max [ j () µ j (), 0 ] + A j () j, (4) and (0) = 0. Noe ha by using (4), we assume ha when a queue does no have enough packes o send, null packes are ransmied. In his paper, we adop he following noion of queue sabiliy: E { r j=1 j } lim sup j=1 r E { j (τ) } <. (5) We also use f π av o denoe he ime average conduced by an acion-seeking policy π, defined as: f π av lim sup E { f π (τ) }, (6) where f π (τ) is he concurred a ime τ by policy π. We call an acion-seeking policy under which (5) holds a sable policy,

3 DRAFT 3 PROC. OF 7TH INTL. SYMPOSIM ON MODELING AND OPTIMIZATION IN MOBILE, AD HOC, AND WIRELESS NETWORKS (WIOPT), JNE where f π TABLE I (τ) is he concurred a ime τ by policy π. We call NETWORK STATE, TRAFFIC AND RATE an acion-seeking policy under which (5) holds a sable policy, and use fav andouse denoe fav ohe denoe opimal he opimal ime average ime average cos cos over over S() R() S 1() S () A 1() A () µ 1() µ () s all sable policies. Every slo, he nework operaor observes 1 0 B B 0 x 1 x 1 x all sable policies. Every slo, he nework conroller observes s 0 B G 0 x 1 x 1 x he curren nework sae and chooses a conrol acion, wih s 3 0 G B 0 x 1 x 1 x he curren he nework goal ofsae minimizing and chooseme average a conrol cos subjec acion, owih nework s 4 0 G G 0 x 1 x 1 x s he goal of sabiliy. minimizing This goal imecan average be mahemaically cos subjec saed o nework as: 5 B B x 1 x 1 x s 6 B G x 1 x 1 x sabiliy. This goal can be mahemaically min : f av saed as: s 7 G B x 1 x 1 x s 8 G G x 1 x 1 x s.. (5). min : f av, s.. (5). In he res of he paper, we will refer o his problem as he Fig.. Nework sae, Traffic and Rae funcions In he ressochasic of he paper, problem. wethis willsochasic refer oproblem his problem framework as can he sochasic be used problem. o model This many sochasic nework uiliy problem problems, framework such as he A. The QLA algorihm can be usedenergy o model minimizaion many problem nework[3] uiliy and he problems, access poin such pricing problem [5]. A. The ToQLA solvealgorihm he sochasic problem using QLA, we firs define as he energy minimizaion problem [3] and he access poin We noe ha a similar nework model wih sochasic penalies is reaed in [1] using a fluid model and a primal-dual ap- We hen define he one-slo condiional Lyapunov drif: a quadraic Lyapunov funcion L(()) = pricing problem [5]. We noe ha a similar nework model 1 r To solve he sochasic problem using QLA, we j=1firs j (). define wih sochasic proach penalies ha achieves reaed opimaliy in [13] in ausing limiing a sense. fluid model The framework is also approach reaed in ha [7] using achieves a quadraic opimaliy Lyapunov in abasedwe we hen obain define he following he one-slo drif expression: condiional Lyapunov drif: a quadraic Lyapunov (()) = E { funcion L(()) = L(( + 1)) L(()) () } 1 r. From j=1(4), j (). and a primal-dual algorihm (QLA) ha provides an explici [O(1/ ),O( )] limiing sense. The framework is also reaed in [7] using a (()) = E { L(( + 1)) L(()) () }. From (4), performance-delay radeoff when he nework sae i.i.d.. quadraic Lyapunov based algorihm (QLA) ha provides an we obain he following explici [O(1/ D. An ), Example O( )] performance-delay of he Model (()) C E { r drif expression: radeoff when he j () [ µ j () A j () ] () }, j=1 nework saehere i.i.d.. we provide an example o illusrae our model. Con- (()sider he -queue nework in Fig.1. Every slo, he nework j=1 C E { r j () [ µ j () A j () ] () }, operaor makes a decision on how o allocae power on where C = rδ D. An Example of he Model max. Now add o boh sides he erm queues, so as o suppor all arriving raffic, i.e., mainain queue where E { C f() = () rδ } max., where Now add 1 is aoscalar bohconrol sidesvariable, he erm Here we sabiliy, provide wih an example minimumoenergy illusrae expendiure. our model. EveryCon- sider he -queue nework in Fig.1. Every slo, he nework slo, he E { we f() obain: () }, where 1 is a scalar conrol variable, number of arrival packes R(), i.i.d., being eiher or 0 wih probabiliies 5/8 and 3/8 respecively. The channel saes we obain: operaor makes S 1 (),S a decision () are also on i.i.d. wheher beingor eiher no G=good o allocae or B=bad one (()) + E { f() () { C E f() (7) uni power o wih serve equal packes probabiliies. a each Onequeue, uni ofso power as o cansuppor serve all packes (()) + E { f() () } { C E f() (7) arriving raffic, in a good i.e., channel mainain buqueue can only sabiliy, serve one wih in aminimum bad channel. r + j () [ µ j () A j () ] } energy expendiure. Every slo, he number of arrival packes r (). j=1 A1()=R()!1()=A()!() + R(), i.i.d., being eiher or 10 wih probabiliies j () [ µ j () A j () ] } (). 5/8 and S1() S() j=1 3/8 respecively. The channel saes S 1 (), S () are also i.i.d. The QLA algorihm is hen obained by choosing an acion being eiher Fig. G=good 1. A -queue or sysem B=bad wih equal probabiliies. The QLA algorihm is hen obained by choosing an acion x a every ime slo o minimize he righ hand side of (7) One uni of power can serve packen a good channel bu x agiven every (). ime Specifically, slo o he minimize QLA algorihm he righ works hand as follows: side of (7) In his example, a nework sae S() is an can only serve (R(),S one 1 (),S in a ()) bad uple channel. and Boh he nework channelssae canisbe given i.i.d.. QLA: (). A Specifically, every ime slo he, QLA observe algorihm he curren works nework as follows: sae acivaed simulaneously There are eighwihou neworkaffecing saes. A each oher. sae, he acion QLA: S() and A every he backlog ime (). slo, Ifobserve S() =she i, choose curren x (si) nework X (si) sae x (si) is a pair (x (si) 1,x (si) ), wih x (si) i represening he S() ha andmaximizes he backlog he following: (). If S() =, choose x (si) X (si) A1()=R() amoun of energy!1()=a() spen a queue i.!() The cos funcion is ha solves he following: always f( 1,x)=x 1 + x for all i. The sae informaion, he r raffic funcion and service S1() rae funcions S() are summarized in max f(,x)+ r j () [ b j (,x) g j (,x) ] (8) he following able. Noe ha in his example A 1 () =R() max f(, x) + j=1 j () [ b j (, x) g j (, x) ] (8) Fig. 1. A -queue is par sysem of he nework sae and hundependen of x (si) ; s.. x X (si). j=1 while A () =µ 1 () hence depends on he acion. s.. x X (si). In his case, a I. nework QLA AND saethe S() DETERMINISTIC is a (R(), S 1 PROBLEM (), S ()) Depending on he problem srucure, (8) can usually be uple and S() Inishi.i.d.. secion, There weare firs eigh reviewpossible he quadraic nework Lyapunov saes. funcionss i based, he algorihms acion x (si) (heis QLA a pair algorihm) (x 1, x[7] ), for wih solving x i he decomposed [5]. Also, i ino canseparae be shown, pars an ha [5] ha, are QLA easier canoachieve solve, ae.g., being he amoun sochasic ofproblem. energy spen Then we a define queuehe i, deerminisic and (x 1, x ) problem [3], ime [5]. average Also, i cos can(defined be shown, in (6)) asha in is wihin O(log( )/ ) Depending decomposed onino heseparae problem parssrucure, ha are easier (8) o solve can e.g., usually [3], be A each sae X (si) and is dual. We hen describe he ordinary subgradien = {0/1, 0/1}. The cos funcion is always f(, x (si) mehod o f ) = av, wih a ime average backlog E { [7] ha, r j=1 j}, or equivalenly, ime average delay, being O( ). Furher, if S() is (OSM) ha can be used o solve he dual. The dual problem x 1 +x for and all sosm i. The will nework also be used saes, laer hefor raffic our analysis funcions of he and fav QLA = fav + O(1/ ), QLA = O( ), (9) seady i.i.d., hen QLA achieves an [O(1/ ),O( )] performancedelay f radeoff. service raesae funcions backlogare behavior summarized under QLA. in Fig.. Noe here where A 1 () = R() is par of S() and hundependen of x (si) av QLA is he average cos under QLA and QLA is he ; ime average nework backlog size under QLA. while A () = µ 1 () hence depends on x (si). Also noe ha A () equals µ 1 () insead of min[µ 1 (), 1 ()] due o our B. The Deerminisic Problem idle fill assumpion in Secion III-C. Consider he deerminisic problem as follows: I. QLA AND THE DETERMINISTIC PROBLEM In his secion, we firs review he quadraic Lyapunov funcions based algorihms (he QLA algorihm) [7] for solving he sochasic problem. Then we define he deerminisic problem and is dual. We hen describe he ordinary subgradien mehod (OSM) ha can be used o solve he dual. The dual problem and OSM will also be used laer for our analysis of he seady sae backlog behavior under QLA. min F(x) p si f(, x (si) ) (10) s.. G j (x) p si g j (, x (si) ) B j (x) p si b j (, x (si) ) j x (si) X (si) i = 1,,..., M,

4 PROC. OF 7TH INTL. SYMPOSIM ON MODELING AND OPTIMIZATION IN MOBILE, AD HOC, AND WIRELESS NETWORKS (WIOPT), JNE where p si corresponds o he probabiliy of S() = and x = (x (s1),..., x (s M ) ) T. The dual problem of (10) can be obained as follows: max q() (11) s.. 0, where q() is called he dual funcion and is defined as: { q() = inf p si f(, x (si) ) (1) x ( ) X ( ) + j j [ p si g j (, x (si) ) p si b j (, x (si) ) ] }. By rearranging he erms, we noe ha q() can also be wrien in he following separable form, which is more useful for our laer analysis. { q() = inf f(, x (si) ) (13) p si x ( ) X ( ) + j j [ gj (, x (si) ) b j (, x (si) ) ]}. Here = ( 1,..., r ) T is he Lagrange muliplier of (10). I is well known ha q() in (1) is concave in he vecor, and hence he problem (11) can usually be solved efficienly, paricularly when cos funcions and rae funcions are separable over differen differen nework componens. I is also well known ha in many siuaions, he opimal value of (11) is he same as he opimal value of (10) and in his case we say ha here is no dualiy gap [1]. We noe ha he deerminisic problem (10) is no necessarily convex as he ses X (si) are no necessarily convex, and he funcions f(, ), g j (, ) and b j (, ) are no necessarily convex. Therefore, here may be a dualiy gap beween he deerminisic problem (10) and is dual (11). Furhermore, solving he deerminisic problem (10) may no solve he sochasic problem. This so since a every nework sae, he sochasic problem may require ime sharing over more han one acion, bu he soluion o he deerminisic problem gives only a fixed operaing poin per nework sae. However, one can show, by using an argumen similar o showing he exisence of an opimal saionary randomized algorihm in [5], ha he dual problem (11) gives he exac value of fav, where fav is he opimal ime average cos, even if (10) is non-convex. Among he many algorihms ha can be used o solve (11), he following algorihm is he mos common one (for performance see [1]), we denoe i as he ordinary subgradien mehod (OSM): OSM: Iniialize (0); a every ieraion, observe (), 1) Find x (si) X (si) for i {1,..., M} ha achieves he infimum of he righ hand side of (1). ) sing he x = (x (s1) j ( + 1) = max, x(s ) [ j () α,..., x(s M ) )T found, updae: p si [ bj (, x (si) ) (14) ] g j (, x (si) )], 0. We use x (si) o highligh is dependency on (). The erm α > 0 is called he sep size a ieraion. In he following, we will always assume α = 1 when referring o OSM. Noe ha if here is only one nework sae, QLA and OSM will choose he same acion given he same, and hey differ only by (4) and (14). The erm G = (G,1, G,,..., G,r ) T, wih: G,j = G j (x ) B j (x ) (15) = [ p si bj (, x (si) ) + g j(, x (si) )], is called he subgradien of q() a (). I is well known ha for any oher Û Rr, we have: (Û ())T G q(û) q(()). (16) sing G B, we noe ha (16) also implies: q(û) q(()) B Û () Û, Rr (17) We are now ready o sudy he seady sae behavior of () under QLA. To simplify noaions and highligh he scaling effec of he scalar in QLA, we use he following noaions: 1) We use q 0 () and 0 o denoe he dual objecive funcion and an opimal soluion of (11) when = 1; and use q() and (also called he opimal Lagrange muliplier) for heir counerpars wih general 1; ) We use x (si) o denoe an acion chosen by QLA for a given () and S() = ; and use x = (x (s1),..., x(s M ) )T o denoe a soluion chosen by OSM for a given (). To simplify analysis, we assume he following hroughou: Assumpion 1: = ( 1,..., r )T is unique for all 1. Noe ha Assumpion 1 is no very resricive. In fac, i holdn many nework uiliy opimizaion problems, e.g., [10]. In many cases, we also have 0. Moreover, for he assumpion o hold for all 1, i suffices o have jus 0 being unique. This shown in he following lemma regarding he scaling effec of he parameer on he opimal Lagrange muliplier. Lemma 1: = 0. Proof: From (13) we see ha: { q()/ = inf f(, x (si) ) p si x ( ) X ( ) + j Û j [ gj (, x (si) ) b j (, x (si) ) ]}, where Ûj = j. However, he righ hand side is exacly q 0 (Û), and hus maximized a Û = 0. Hence q() is maximized a 0.. BACKLOG ECTOR BEHAIOR NDER QLA In his secion we sudy he backlog vecor behavior under QLA of he sochasic problem. The following heorem summarizes he main resuls. Recall ha B is defined in (3) as he upper bound of he magniude change of in a slo.

5 PROC. OF 7TH INTL. SYMPOSIM ON MODELING AND OPTIMIZATION IN MOBILE, AD HOC, AND WIRELESS NETWORKS (WIOPT), JNE Theorem 1: If he dual funcion q 0 () saisfies: q 0 ( 0) q 0 () + L 0 0, (18) for some consan L > 0 independen of, hen under QLA, (a) There exis consans D η > 0, boh independen of, such ha whenever () D, we have: E { ( + 1) () } () η. (19) In paricular, he consans D and η ha saisfy (19) can be chosen as follows: Choose η as any value such ha 0 < η < L, independen of. Then, choose D as: 1 [ B η ] D = max (L η), η. (0) (b) For given consans D, η in (a), here exis some consans c, β > 0, independen of, such ha: P(D, m) c e β m, (1) where P(D, m) is defined as: P(D, m) lim sup P r{ (τ) > D + m}. () Noe ha if m = log( ) β, by (1) we have P(D, m) c. Also if a seady sae disribuion of () exiss under QLA, i.e., he limi of 1 1 P r{ (τ) > D + m} exiss as, hen one can replace P(D, m) wih he seady sae probabiliy ha () deviaes from by an amoun of D + m, i.e., P r{ () > D + m}. Therefore Theorem 1 can be viewed as showing ha when (18) is saisfied, for a large, he backlog () under QLA will mosly be wihin O(log( )) disance from. Thimplies ha he average backlog will roughly be j, which is ypically Θ( ) by Lemma 1. However, his fac will also allow us o build FQLA upon QLA o subrac ou roughly j daa from he nework and reduce nework delay. Theorem 1 also highlighs a deep connecion beween he seady sae behavior of he nework backlog process () and he srucure of he dual funcion q 0 (). We noe ha (18) is no very resricive. In fac, if q 0 () is polyhedral (e.g., X (si) is finie for all ), wih a unique opimal soluion 0 0, hen (18) can be saisfied (see Secion II for an example). To prove he heorem, we need he following lemma. Lemma : nder QLA, we have for all, E { ( + 1) () } () + B (3) (q( ) q(())). Proof: See [14]. We now use Lemma o prove Theorem 1. Proof: (Theorem 1) Par (a): We firs show ha if (18) holds for q 0 () wih L, hen i also holds for q() wih he same L. To his end, suppose (18) holds for q 0 () for all 0. Muliplying boh sides of (18) by, we ge: q 0 ( 0) q 0 () + L 0. 1 I can be seen from (17) ha B L. Thus B > η. Now using = 0 and q() = q 0 (/ ) in he above inequaliy, we have for all 0: q( ) q( ) + L. Thus for any 0, we have: q( ) q() + L. (4) Now for a given η > 0, if: B ( q( ) q(()) ) η η (), (5) hen by (3), we have: E { ( + 1) () } ( () η), which hen by Jensen nequaliy implies: (E { ( + 1) () } ) ( () η). Thus (19) follows whenever (5) holds and () η. I suffices o choose D and η such ha D η and ha (5) holds whenever () D. Now noe ha (5) can be rewrien as he following inequaly: q( ) q(()) + η () + Y (6) where Y = B η. Choose η (0, L) independen of. By (4), if: L () η () + Y (7) hen (6) holds. Now choose D as defined in (0), we see ha if () D, hen (7) holds, which implies (6), and equivalenly (5). We also have D η, hence (19) holds. Par (b): Now we show ha (19) implies (1). Choose consans D and η ha are independen of in (a). Denoe Y () = (), we see hen whenever Y () D, we have E { Y ( + 1) Y () () } η. I is also easy o see ha Y ( + 1) Y () B, as B is defined in (3) as he upper bound of he magniude change of in a slo. Define Ỹ () = max [ Y () D, 0 ]. We see ha whenever Ỹ () B, we have: E { Ỹ ( + 1) Ỹ () ()} (8) = E { Y ( + 1) Y () () } η. Now define a Lyapunov funcion of Ỹ () o be L(Ỹ ()) = e wỹ () wih some w > 0, and define he one-slo condiional drif o be: (Ỹ ()) E{ L(Ỹ ( + 1)) L(Ỹ ()) ()} = E { e wỹ (+1) e wỹ () () }. (9) I is shown in Appendix A ha by choosing w = we have for all Ỹ () 0: η B +Bη/3, (Ỹ ()) ewb wη ewỹ (). (30) Taking expecaion on boh sides and carrying ou a elescoping sum, we obain: 1 wη E{ e wỹ (τ)} e wb + E { e wỹ (0)}. (31)

6 PROC. OF 7TH INTL. SYMPOSIM ON MODELING AND OPTIMIZATION IN MOBILE, AD HOC, AND WIRELESS NETWORKS (WIOPT), JNE Divide boh sides by and ake he limsup as goes o infiniy, we obain: wη lim sup E{ e wỹ (τ)} e wb. (3) sing ha E { e wỹ (τ)} e wm P r{ỹ (τ) > m}, we obain: wη lim sup ewm P r{ỹ (τ) > m} ewb. (33) Plug in w = η B +Bη/3 and use he definiion of Ỹ (), we ge: P(D, m) ewb wη e wm = (B + Bη/3)e η B+η/3 η e ηm B +Bη/3, (34) where P(D, m) is defined in (). Therefore (1) holds wih: η B+η/3 c = (B + Bη/3)e η, β = η B + Bη/3. (35) I is easy o see ha c and β are independen of. Noe from (31) and (3) ha Theorem 1 indeed holds for any finie (0). We will laer use his fac o prove he performance of FQLA. The following heorem is a special case of Theorem 1 and gives a more direc illusraion of Theorem 1. Recall ha P(D, m) is defined in (). Define: P (r) (D, m) (36) lim sup P r{ j, j (τ) j > D + m}. Theorem : If he condiion in Theorem 1 holds, hen under QLA, for any c > 0: P(D 1, ck 1 log( )) c 1 c, (37) P (r) (D 1, ck 1 log( )) c 1 c. (38) where D 1 = B L + L 4, K 1 = B +BL/6 L/ and c 1 = 8(B L +BL/6)e B+L/6 L. Proof: Choose η = L/, hen we see from (0) ha [ B L /4 D = max, L ] B L L + L 4. Now by (35) we see ha (1) holds wih c = c 1 and β = L/ B +BL/6. Thus by aking D 1 = B L + L 4, we have: P(D 1, ck 1 log( )) c e ck1β log( ) = c 1e c log( ), where he las sep follows since β K 1 = 1. Thus (37) follows. Equaion (38) follows from (37) by using he fac ha for any consan ζ, he evens E 1 = { j, j (τ) j > ζ} and E = { (τ) > ζ} saisfy E 1 E. Thus: P r{ j, j (τ) j > ζ} P r{ (τ) > ζ}. Theorem can be viewed as showing ha for a large, he probabiliy for j () o deviae from he j h componen of is exponenially decreasing in he disance. Thu rarely deviaes from j by more han Θ(log( )) disance. Noe ha one can similarly prove he following heorem for OSM: Theorem 3: If he condiion in Theorem 1 holds, hen here exis posiive consans D = Θ(1) and η = Θ(1), i.e, independen of, such ha, under OSM, if () D, ( + 1) () η. (39) Proof: I is easy o show ha under OSM, Lemma holds wihou he expecaion. Thus he heorem follows by he same argumen an Theorem 1. Therefore, when here is a single nework sae, we see ha given (18), he backlog process converges o a ball of size Θ(1) around. I. THE FQLA ALGORITHM In his secion, we propose a family of Fas Quadraic Lyapunov based Algorihms (FQLA) for general sochasic nework opimizaion problems. We firs provide an example o illusrae he idea of FQLA. We hen describe FQLA wih known, called FQLA-Ideal, and sudy is performance. Afer ha, we describe he more general FQLA wihou such knowledge, called FQLA-General. A. FQLA: a Single Queue Example To illusrae he idea of FQLA, we firs look a an example. Figure 3 shows a slo sample backlog process under QLA. We see ha afer roughly 1500 slos, () always says very close o, which is a Θ( ) scalar in his case. To reduce delay, we can firs find W (0, ) such ha: under QLA, here exiss a ime 0 so ha ( 0 ) W and once () W, i remains so for all ime (he solid line in Fig. 3 shows one for hese 10 4 slos). We hen place W fake bis (called place-holder bis [11]) in he queue a ime 0, i.e., iniialize (0) = W, and run QLA. I is easy o show ha he uiliy performance of QLA will remain he same wih his change, and he average backlog is now reduced by W. However, such a W may require W = Θ( ), hus he average backlog may sill be Θ( ). * Fig. 3. (). () Sar here Number of place holder bis !!4 W()!W () max[w()!w, 0] +! max Lef: A sample backlog process; Righ: An Example of W () and FQLA insead finds a W such ha in seady sae, he backlog process under QLA rarely goes below i, and places W place-holder bin he queue a ime 0. FQLA hen uses an auxiliary process W (), called he virual backlog process, o keep rack of he backlog process ha should have This sample backlog process one sample backlog process of queue 1 of he sysem considered in Secion II, under QLA wih = 50.

7 PROC. OF 7TH INTL. SYMPOSIM ON MODELING AND OPTIMIZATION IN MOBILE, AD HOC, AND WIRELESS NETWORKS (WIOPT), JNE been generaed if QLA is used. Specifically, FQLA iniializes W (0) = W. Then a every slo, QLA is run using W () as he queue size, and W () is updaed according o QLA. Wih W () and W, FQLA works as follows: A ime, if W () W, FQLA performs QLA s acion (obained based on S() and W ()); else if W () < W, FQLA carefully modifies QLA s acion so as o mainain () max[w () W, 0] for all (see Fig.3 for an example). Similar as above, his roughly reduces he average backlog by W. The difference is ha now we can show ha W = max[ log ( ), 0] mees he requiremen. Thu is possible o bring he average backlog down o O(log ( )). Also, since W () can be viewed as a backlog process generaed by QLA, i rarely goes below W in seady sae. Hence FQLA is almos always he same as QLA, hus able o achieve an O(1/ ) close-o-opimal uiliy performance. B. The FQLA-Ideal Algorihm In his secion, we presen he FQLA-Ideal algorihm. We assume he value = ( 1,..., r )T is known a-priori. FQLA-Ideal: (I) Deermining place-holder bis: For each j, define: W j = max [ j log ( ), 0 ], (40) as he number of place-holder bis of queue j. (II) Place-holder-bi based acion: Iniialize j (0) = 0, W j (0) = W j, j. For 1, observe he nework sae S(), solve (8) wih W () in place of (). Perform he chosen acion wih he following modificaion: Le A() and µ() be he arrival and service rae vecors generaed by he acion. For each queue j, do (Idle fill whenever needed): a) If W j () W j : admi A j () arrivals, serve µ j () daa, i.e., updae he backlog by: j ( + 1) = max [ j () µ j (), 0 ] + A j (). b) If W j () < W j : admi Ãj() = max [ A j () W j + W j (), 0 ] arrivals, serve µ j () daa, i.e., updae he backlog by: j ( + 1) = max [ j () µ j (), 0 ] + Ãj(). c) pdae W j () by: W j ( + 1) = max [ W j () µ j (), 0 ] + A j (). From above we see ha FQLA-Ideal is he same as QLA based on W () when W j () W j for all j. When W j () < W j for some queue j, FQLA-Ideal admis roughly he excessive packes afer W j () is brough back o be above W j for he queue. Thus for problems where QLA admis an easy implemenaion, e.g., [3], [5], i is also easy o implemen FQLA. However, we also noice wo differen feaures of FQLA: (1) By (40), W j can be 0. However, when is large, his happens only when 0j = j = 0 according o Lemma 1. In his case W j = j = 0, and queue j indeed needs zero place-holder bis. () Packes may be dropped in Sep II-(b) upon heir arrivals, or afer hey are admied ino he nework in a mulihop problem. Such packe dropping is naural in many flow conrol problems and does no change he naure of hese problems. In oher problems where such opion is no available, he packe dropping opion inroduced o achieve desired delay performance, and i can be shown ha he fracion of packes dropped can be made arbirarily small. Noe ha packe dropping here is o compensae for he deviaion from he desired Lagrange muliplier, hus differen from ha in [15], where packe dropping is used for drif seering. C. Performance of FQLA-Ideal We look a he performance of FQLA-Ideal in his secion. We firs have he following lemma ha shows he relaionship beween () and W (). We will use i laer o prove he delay bound of FQLA. Noe ha he lemma also holds for FQLA-General described laer, as FQLA-Ideal/General differ only in he way of deermining W = (W 1,..., W r ) T. Lemma 3: nder FQLA-Ideal/General, we have j, : max [ W j () W j, 0 ] j () max [ W j () W j, 0 ] +δ max (41) where δ max is defined in Secion III-B o be he upper bound of he number of arriving or deparing packes of a queue. Proof: See [14]. The following heorem summarizes he main performance resuls of FQLA-Ideal. Recall ha for a given policy π, f π av denoes average cos defined in (6) and f π () denoes he conduced by π a ime. Theorem 4: If he condiion in Theorem 1 holds and a seady sae disribuion exiss for he backlog process generaed by QLA, hen wih a sufficienly large, we have under FQLA-Ideal ha, = O(log ( )), (4) fav F I = fav + O(1/ ), (43) P drop = O(1/ c0 log( ) ), (44) where c 0 = Θ(1), is he ime average backlog, fav F I is he ime average cos of FQLA-Ideal, fav is he opimal ime average cos and P drop is he ime average fracion of packes ha are dropped in Sep-II (b). Proof: Since a seady sae disribuion exiss for he backlog process generaed by QLA, we see ha P(D, m) in () represens he seady sae probabiliy of he even ha he backlog vecor deviaes from by disance D + m. Now since W () can be viewed as a backlog process generaed by QLA, wih W (0) = W insead of 0, we see from he proof of Theorem 1 ha Theorem 1 and hold for W (), and by [7], QLA based on W () achieves an average cos of fav + O(1/ ). Hence by Theorem, here exis consans D 1, K 1, c 1 = Θ(1) so ha: P (r) (D 1, ck 1 log( )) c 1. By c he definiion of P (r) (D 1, ck 1 log( )), himplies ha in seady sae: P r{w j () > j + D 1 + m} c 1e m K 1, Now le: Q j () = max[w j () j D 1, 0]. We see ha P r{q j () > m} c 1e m K 1, m 0. We hus have Q j = O(1), where Q j is he ime average value of Q j (). Now i is

8 PROC. OF 7TH INTL. SYMPOSIM ON MODELING AND OPTIMIZATION IN MOBILE, AD HOC, AND WIRELESS NETWORKS (WIOPT), JNE easy o see by (40) and (41) ha j () Q j () + log ( ) + D 1 + δ max for all. Thus (4) follows since for a large : j Q j + log ( ) + D 1 + δ max = Θ(log ( )). Now consider he average cos. To save space, we use FI for FQLA-Ideal. From above, we see ha QLA based on W () achieves an average cos of f av + O(1/ ). Thu suffices o show ha FQLA-Ideal performs almos he same as QLA based on W (). Firs we have for all 1 ha: f F I (τ) = f F I (τ)1 E(τ) + f F I (τ)1 E c (τ), where 1 E(τ) is he indicaor funcion of he even E(τ), E(τ) is he even ha FQLA-Ideal performs he same acion as QLA a ime τ, and 1 Ec (τ) = 1 1 E(τ). Taking expecaion on boh sides and using he fac ha when FQLA-Ideal akes he same acion as QLA, f F I (τ) = f QLA (τ), we have: E { f F I (τ) } E { f QLA } (τ)1 E(τ) E { δ max 1 E (τ)} c. Taking he limi as goes o infiniy on boh sides and using f QLA (τ)1 E(τ) f QLA (τ), we ge: fav F I fav QLA + δ max lim = f QLA av 1 + δ max lim 1 E { } 1 Ec (τ) P r{e c (τ)}. (45) However, E c (τ) included in he even ha here exiss a j such ha W j (τ) < W j. Therefore by (38) in Theorem, for a large such ha 1 log ( ) D 1 and log( ) 8K 1, lim P r{e c (τ)} P (r) (D 1, log ( ) D 1 ) sing his fac in (45), we obain: = O(c 1/ 1 K 1 log( ) ) = O(1/ 4 ). (46) f F I av = f QLA av + O(δ max / 4 ) = f av + O(1/ ), where he las equaliy holds since fav QLA = fav + O(1/ ). This proves (43). (44) follows since packes are dropped a ime τ only if E c (τ) happens, hus by (46), he fracion of ime when packe dropping happens O(1/ c0 log( ) ) wih c 0 = 1 K 1 = Θ(1), and each ime no more han rb packes can be dropped. D. The FQLA-General algorihm Now we describe he FQLA algorihm wihou any a-priori knowledge of, called FQLA-General. FQLA-General firs runs he sysem for a long enough ime T, such ha he sysem eners seady sae. Then i chooses a sample of he queue vecor value o esimae and uses ha o decide he number of place holder bis. FQLA-General: (I) Deermining place-holder bis: a) Choose a large ime T (See Secion I-E for he size of T ) and iniialize W (0) = 0. Run he QLA algorihm wih parameer, a every ime slo, updae W () according o he QLA algorihm and obain W (T ). b) For each queue j, define: W j = max [ W j (T ) log ( ), 0 ], (47) as he number of place-holder bis. (II) Place-holder-bi based acion: same as FQLA-Ideal. The performance of FQLA-General is summarized as follows: Theorem 5: Assume he condiionn Theorem 4 hold and he sysem in seady sae a ime T, hen under FQLA-General wih a sufficienly large, wih probabiliy 1 O( 1 ): (a) = O(log ( )), (b) f F G 4 av = fav + O(1/ ), and (c) P drop = O(1/ c0 log( ) ), where c 0 = Θ(1) and fav F G is he ime average cos of FQLA-General. Proof: We will show ha wih probabiliy of 1 O( 1 4 ), W j is close o max[ j log ( ), 0]. The res can hen be proven similarly an he proof of Theorem 4. For each queue j, define: v + j = j + 1 log ( ), v j = max [ j 1 log ( ), 0 ]. Noe ha v j is defined wih a max[ ] operaor. This due o he fac ha j can be zero. An (46), we see ha by Theorem, here exiss D 1 = Θ(1), K 1 = Θ(1) such ha if is such ha 1 4 log ( ) D 1 and log( ) 16K 1, hen: P r { j, W j (T ) / [v j, v+ j ]} P (r) (D 1, 1 log ( ) D 1 ) = O(1/ 4 ) Thus we see ha P r { W j (T ) [v j, v+ j ] j} = 1 O(1/ 4 ), which implies: P r { W j [ˆv j, ˆv+ j ] j} = 1 O(1/ 4 ). where ˆv + j = max [ j 1 log ( ), 0 ] and ˆv j = max [ j 3 log ( ), 0 ]. Hence for a large, wih probabiliy 1 O( 1 ): if 4 j > 0, we have j 3 log ( ) W j j 1 log ( ); else if j = 0, we have W j = j. The res of he proof is similar o he proof of Theorem 4. E. Pracical Issues From Lemma 1 we see ha he magniude of can be Θ( ). This means ha T in FQLA-General may need o be Ω( ), which is no very desirable when is large. We can insead use he following heurisic mehod o accelerae he process of deermining W: For every queue j, guess a very large W j. Then sar wih his W and run he QLA algorihm for some T 1, say slos. Observe he resuling backlog process. Modify he guess for each queue j using a bisecion algorihm unil a proper W is found, i.e. when running QLA from ha value, we observe flucuaions of W j () around W j insead of a nearly consan increase or decrease for all j. Then le W j = max[w j log ( ), 0] be he number of place-holder

9 PROC. OF 7TH INTL. SYMPOSIM ON MODELING AND OPTIMIZATION IN MOBILE, AD HOC, AND WIRELESS NETWORKS (WIOPT), JNE bis of queue j. To furher reduce he error probabiliy, one can repea Sep-I (a) muliple imes and use he average value as W (T ). Noe ha even hough resuln Theorem 4 and 5 assume a large, in pracice, he value may no have o be very large (See Secion II for an example). II. SIMLATION In his secion we provide simulaion resuls for he FQLA algorihms. We consider a five queue sysem ha exends he example in Secion III-D. In his case r = 5. The sysem is shown in Fig. 4. The goal is o perform power allocaion a each node so as o suppor he arrival wih minimum energy expendiure. Fig. 4. R() S1() S() S3() S4() S5() A five queue sysem In his example, he random nework sae S() is he vecor conaining he random arrivals R() and he channel saes S i (), i = 1,..., 5. Similar an Secion III-D, we have: A() = (R(), µ 1 (), µ (), µ 3 (), µ 4 ()) T, µ() = (µ 1 (), µ (), µ 3 (), µ 4 (), µ 5 ()) T, i.e., A 1 () = R(), A i () = µ i 1 () for i, where µ i () is he service rae obained by queue i a ime. R() is 0 or wih probabiliies 3 8 and 5 8, respecively. S i() can be Good or Bad wih equal probabiliies for 1 i 5. When he channel is good, one uni of power can serve wo packes; oherwise one uni of power can serve only one packe. We assume all channels can be acivaed a he same ime wihou affecing ohers. I can be verified ha = (5, 4, 3,, ) T is unique. In his example, he backlog vecor process evolves as a Markov chain wih counably many saes. Thus one can show ha here exiss a saionary disribuion for he backlog vecor under QLA. We simulae FQLA-Ideal and FQLA-General wih = 50, 100, 00, 500, 1000 and 000. We run each case for slos under boh algorihms. For FQLA-General, we use T = 50 in Sep-I and repea Sep-I 100 imes and use heir average as W (T ). I is easy o see from he lef plo in Fig. 5 ha he average queue sizes under boh FQLAs are always close o he value 5 log ( ) (r = 5). From he middle plo we also see ha he percenage of packes dropped decreases rapidly and ges below 10 4 when 500 under boh FQLAs. These plos show ha in pracice, may no have o be very large for Theorem 4 and 5 o hold. The righ plo shows a sample (W 1 (), W ()) process for a slo inerval under FQLA-Ideal wih = 1000, considering only he firs wo queues of Fig. 4 for his example. We see ha during hinerval, (W 1 (), W ()) always remains close o ( 1, ) = (5, 4 ), and W 1() W 1 = 495, W () W = 395. For all values, he average power expendiure is very close o 3.75, which is he opimal energy expendiure, and he average of W j () is very close o 15 (plos omied for breviy) FQLA!I FQLA!G rlog () !1 10! 10!3 10!4 FQLA!I FQLA!G 10! W () (W 1 (), W ()) (5000, 4000) W () 1 Fig. 5. FQLA-Ideal performance: Lef - Average queue size; Middle - Percenage of packes dropped; Righ - Sample (W 1 (), W ()) process for [10000, ] and = 1000 under FQLA-Ideal. III. LAGRANGE MLTIPLIER: SHADOW PRICE AND NETWORK GRAITY I is well known ha Lagrange Mulipliers can play he role of shadow prices o regulae flown many flow-based problems wih differen objecives, e.g., [16]. Thimporan feaure has enabled he developmen of many disribued algorihms in resource allocaion problems, e.g., [17]. However, a problem of his ype ypically requires daa ransmissions o be represened as flows. Thun a nework ha is discree in naure, e.g., ime sloed or packeized ransmission, a rae allocaion soluion obained by solving such a flow-based problem does no immediaely specify a scheduling policy. Recenly, several Lyapunov algorihms have been proposed o solve uiliy opimizaion problems under discree nework seings. In hese algorihms, backlog vecors ac as he graviy of he nework and allow opimal scheduling o be buil upon hem. I is also revealed in [18] ha QLA is closely relaed o he dual subgradien mehod and backlogs play he same role as Lagrange mulipliern a ime invarian nework. Now we see by Theorem 1 ha he backlogndeed play he same role as Lagrange mulipliers even under a more general sochasic nework. In fac, he backlog process under QLA can be closely relaed o a sequence of updaed Lagrange mulipliers under a subgradien mehod. Consider he following imporan varian of OSM, called he randomized incremenal subgradien mehod (RISM) [1], which makes use of he separable naure of (13) and solves he dual problem (11) as follows: RISM: Iniialize (0); a ieraion, observe (), choose a random sae S() S according o some probabiliy law. (1) If S() =, find x (si) X (si) ha solves he following: min f(, x) + j () [ g j (, x) b j (, x) ] j s.. x X (si). (48) () sing he x (si) 3 found, updae () according o: [ ] j ( + 1) = max j () α b j (, x (si) ), 0 + α g j (, x (si) ). As an example, S() can be chosen by independenly choosing S() = wih probabiliy p si every ime slo. In his 3 Noe ha his updae rule is differen from RISM s usual rule, i.e., j (+ 1) = max ˆ j () α b j (, x) + α g j (, x), 0, bu i almos does no affec he performance of RISM.

10 PROC. OF 7TH INTL. SYMPOSIM ON MODELING AND OPTIMIZATION IN MOBILE, AD HOC, AND WIRELESS NETWORKS (WIOPT), JNE case S() will be i.i.d.. Noe ha in he sochasic problem, a nework sae is chosen randomly by naure as he physical sysem sae a ime ; while here a sae is chosen arificially by RISM according some probabiliy law. Now we see from (8) and (48) ha: given he same () and, QLA and RISM choose an acion in he same way. If also α = 1 for all, and ha S() under RISM evolves according o he same probabiliy law as S() of he physical sysem, we see ha applying QLA o he nework indeed equivalen o applying RISM o he dual problem of (10), wih he nework sae being chosen by naure, and he nework backlog being he Lagrange muliplier. Therefore, Lagrange Mulipliers under such sochasic discree nework seings ac as he nework graviy, hus allow scheduling o be done opimally and adapively based on hem. This nework graviy funcionaliy of Lagrange Mulipliern discree nework problems can hus be viewed as he counerpar of heir shadow price funcionaliy in he flow-based problems. Furher more, he nework graviy propery of Lagrange Mulipliers enables he use of place holder bis o reduce nework delay in nework uiliy opimizaion problems. This a unique feaure no possessed by is price counerpar. APPENDIX A PROOF OF (30) Here we prove ha for Ỹ () defined in he proof of par (b) of Theorem 1, we have for all Ỹ () 0: (Ỹ ()) ewb wη ewỹ (). Proof: If Ỹ () > B, denoe δ() = Ỹ ( + 1) Ỹ (). I is easy o see ha δ() B. Rewrie (9) as: (Ỹ ()) = ewỹ () E {( e wδ() 1 ) () }. (49) By a Taylor expansion, we have ha: e wδ() = 1 + wδ() + w δ () g(wδ()), (50) y k k! = (ey 1 y) y where g(y) = k= [19] has he following properies: 1) g(0) = 1; g(y) 1 for y < 0; g(y) is monoone increasing for y 0; ) For y < 3, g(y) = k= Thus by (50) we have: y k k! k= y k 3 k = 1 1 y/3. e wδ() 1 + wδ() + w B g(wb). (51) Plug hino (49) and noe ha Ỹ () > B, so by (8) we have: E { δ() () } η. Hence: (Ỹ ()) ()( ewỹ wη + w B g(wb) ). (5) η Choosing w = B +Bη/3, we see ha wb < 3, hus: w B g(wb) w B 1 1 wb/3 = wη. The las equaliy follows since: η w = B + Bη/3 w(b + Bη/3) = η wb = η wbη/3 wb 1 1 wb/3 = η. Therefore (5) becomes: (Ỹ ()) wη ewỹ () e wb wη ewỹ (). (53) Now if Ỹ () B, i is easy o see ha (Ỹ ()) ewb e wỹ () e wb wη ewỹ (), since Ỹ (+1) B +Ỹ () B 1. Thus for all Ỹ () 0, (30) holds. and wη REFERENCES [1] Y.Yi and M.Chiang. Sochasic nework uiliy maximizaion: A ribue o kelly s paper published in his journal a decade ago. European Transacions on Telecommunicaions, vol. 19, no. 4, pp , June 008. [] L.Tassiulas and A.Ephremides. Sabiliy properies of consrained queueing sysems and scheduling policies for maximum hroughpu in mulihop radio neworks. IEEE Transacions on Auomaic Conrol, vol. 37, no. 1, pp , Dec [3] M.J.Neely. Energy opimal conrol for ime-varying wireless neworks. IEEE Transacions on Informaion Theory 5(7): , July 006. [4] M.J.Neely, E.Modiano, and C.Li. Fairness and opimal sochasic conrol for heerogeneous neworks. IEEE INFOCOM Proceedings, March 005. [5] L.Huang and M.J.Neely. The opimaliy of wo prices: Maximizing revenue in a sochasic nework. Proc. of 45h Annual Alleron Conference on Communicaion, Conrol, and Compuing (invied paper), Sep [6] R.rgaonkar and M.J.Neely. Opporunisic scheduling wih reliabiliy guaraneen cogniive radio neworks. IEEE INFOCOM Proceedings, April 008. [7] L.Georgiadis, M.J.Neely, and L.Tassiulas. Resource Allocaion and Cross-Layer Conrol in Wireless Neworks. Foundaions and Trendn Neworking ol. 1, no. 1, pp , 006. [8] M.J.Neely. Opimal energy and delay radeoffs for muli-user wireless downlinks. IEEE Transacions on Informaion Theory vol. 53, no. 9, pp , Sep [9] M.J.Neely. Super-fas delay radeoffs for uiliy opimal fair scheduling in wireless neworks. IEEE Journal on Seleced Arean Communicaions (JSAC), Special Issue on Nonlinear Opimizaion of Communicaion Sysems, 4(8), Aug [10] A.Eryilmaz and R.Srikan. Fair resource allocaion in wireless neworks using queue-lengh-based scheduling and congesion conrol. IEEE/ACM Trans. New., 15(6): , 007. [11] M.J.Neely and R.rgaonkar. Opporunism, backpressure, and sochasic opimizaion wih he wireless broadcas advanage. Asilomar Conference on Signals, Sysems, and Compuers, Pacific Grove, CA (Invied paper), Oc [1] D.P.Bersekas, A.Nedic, and A.E.Ozdaglar. Convex Analysis and Opimizaion. Boson: Ahena Scienific, 003. [13] A.L.Solyar. Greedy primal-dual algorihm for dynamic resource allocaion in complex neworks. Queueing Sysems, ol. 54, No.3, pp.03-0, 006. [14] L.Huang and M.J.Neely. Delay reducion via lagrange mulipliern sochasic nework opimizaion. arxiv: v1, April 009. [15] M.J.Neely. Inelligen packe dropping for opimal energy-delay radeoffs in wireless downlinks. Proc. of he 4h In. Symposium on Modeling and Opimizaion in Mobile, Ad Hoc, and Wireless Neworks (WiOp), April 006. [16] F.Kelly. Charging and rae conrol for elasic raffic. European Transacions on Telecommunicaions, vol. 8, pp , [17] C.Curescu and S.Nadjm-Tehrani. Price/uiliy-based opimized resource allocaion in wireless ad hoc neworks. IEEE SECON, 85-95, 005. [18] M.J.Neely, E.Modiano, and C.E.Rohrs. Dynamic power allocaion and rouing for ime-varying wireless neworks. IEEE Journal on Seleced Arean Communicaions, ol 3, NO.1, January 005. [19] F.Chung and L.Lu. Concenraion inequaliies and maringale inequaliies a survey. Inerne Mah., 3 ( ),