A Game-Theoretic Framework for Medium Access Control

Transcription

1 IEEE JOURNAL ON SELECTED AREAS OF COMMUNICATIONS, VOL. XX, NO. XX, DECEMBER 28 A Game-Theoretc Framework for Medum Access Control Tao Cu, Ljun Chen, Member, IEEE, and Steven H. Low, Fellow, IEEE Abstract In ths paper, we generalze the random access game model, and show that t provdes a general game-theoretc framework for desgnng contenton based medum access control. We extend the random access game model to the network wth multple contenton measure sgnals, study the desgn of random access games, and analyze dfferent dstrbuted algorthms achevng ther equlbra. As examples, a seres of utlty functons s proposed for games achevng the maxmum throughput n a network of homogeneous nodes. In a network wth n traffc classes, an N-sgnal game model s proposed whch acheves the maxmum throughput under the farness constrant among dfferent traffc classes. In addton, the convergence of dfferent dynamc algorthms such as best response, gradent play and Jacob play under propagaton delay and estmaton error s establshed. Smulaton results show that game model based protocols can acheve superor performance over the standard IEEE 82. DCF, and comparable performance as exstng protocols wth the best performance n lterature. Index Terms Medum access control, Random access game, Nash equlbrum, Dstrbuted strategy update mechansm, Wreless LANs. I. INTRODUCTION Wreless channel s a shared medum that s nterferencelmted. A contenton-based medum access control (contenton control) s a dstrbuted strategy to access and share a wreless channel among competng wreless nodes. It dynamcally adjusts channel access probablty n response to the amount of contenton n the network. Note that the amount of contenton tself depends on the channel access probabltes chosen by the wreless nodes. Hence contenton control s an teratve feedback system descrbed mathematcally as: p (t + ) = F (p (t), q (t)), q (t) = G (p(t)), () where p (t) s the channel access probablty of node, p(t) = {p (t)} s the correspondng vector, q (t) s a vector of certan measures of contenton observed by node that depends on the vector p(t). The channel access probablty p (t) s usually mplemented ether through a backoff algorthm on contenton wndow or as a persstence probablty. For example, the standard IEEE 82. DCF has a backoff algorthm that nduces a channel access probablty and can be modeled by some functon F. The algorthm responds to whether there Manuscrpt receved August 5, 27; revsed March, 28; accepted Aprl, 28. Ths work has been supported n part by NSF grants CNS and CNS-52349, and Caltech s Lee Center for Advanced Networkng. Ths paper has been presented n part at the Internatonal Wreless Internet Conference, October 27, Austn, Texas, USA. T. Cu, L. Chen and S. H. Low are wth the Dvson of Engneerng and Appled Scence, Calforna Insttute of Technology, Pasadena, CA 925, USA (Emal: {taocu@, chen@cds., slow@}caltech.edu). s a collson, and hence the measure of contenton q (t) n DCF s the probablty of collson whose dependence on the channel access probablty vector p(t) can be modeled by some functon G. The performance of a MAC, e.g., the throughput, farness and collson, depends crtcally on the equlbrum and stablty of the dynamcal system defned by (). In [], [2], Chen et al. propose a general game-theoretc model, called random access game, to understand the dynamcal system () for the network where each node observes a sngle contenton measure q, and use t to gude the desgn of new medum access protocols. The key dea of the random access game model s to consder each node to have a utlty functon U (p ) as a functon of ts channel access probablty p. The goal of node s to maxmze ts payoff functon u (p) := U (p ) p q gven the contenton measure q. Hence, the steady state propertes of a MAC can be analyzed or desgned through the specfcaton of the utlty functon U (p ) and the choce of contenton measure q (e.g., collson probablty, or dle tme between channel access, etc). Ther specfcaton defnes the underlyng random access game whose equlbrum determnes the steady state propertes such as throughput, farness and collson of the MAC. The adaptaton of channel access probablty can be specfed through (F, G) and corresponds to dfferent strateges to approach the equlbrum of the game. In ths paper, we extend the random access game model to the network where each node can observe multple contenton measure sgnals q, study the desgn of random access games, analyze dstrbuted algorthms achevng ther equlbra, and show that the random access game model provdes a general framework for desgnng contenton based medum access control. Specfcally, n Secton III, we descrbe the generalzed random access game model, and provde condtons under whch an equlbrum for the game exsts and s unque. Several examples are provded on how to desgn random access games by forward engneerng from desred operatng ponts (e.g., n terms of some target throughput and farness) and based on heurstcs. A seres of utlty functons s proposed for games achevng the maxmum throughput n a network of homogeneous nodes. In a network wth n traffc classes, an N- sgnal game model s proposed whch acheves the maxmum throughput under the farness constrant among dfferent traffc classes. Supermodular game s also consdered, whch guarantees the exstence of Nash equlbrum. Moreover, the best response strategy dscussed n Secton IV always converges to a Pareto domnant equlbrum of supermodular random access game. In Secton IV, we also consder another two dynamc algorthms to acheve the equlbrum: gradent play and Jacob

2 IEEE JOURNAL ON SELECTED AREAS OF COMMUNICATIONS, VOL. XX, NO. XX, DECEMBER 28 2 play. We show that under mld condtons both algorthms converge to the unque equlbrum. We also establsh the convergence of gradent play under propagaton delay and estmaton error. Due to the approxmaton made n utlty functon desgn, the dynamc algorthms may not converge exactly to the desred operatng pont. An equlbrum selecton algorthm s thus proposed to make these algorthms actually ht the desred pont. Smulaton results show that game model based protocols can acheve superor performance over the standard IEEE 82. DCF, and comparable performance as exstng protocols wth the best performance n lterature. II. RELATED WORK There are lots of works on medum access control. Here we only menton a few that are most closely related to ths work. Game-theoretc approach has been appled extensvely to study medum access, see, e.g., [] [8]. Jn et al. [3] study noncooperatve equlbrum of Aloha networks and ther local convergence. Borkar et al. [5] study dstrbuted scheme for adaptng random access. Čagalj et al. [6] study selfsh behavor n CSMA/CA networks and propose a dstrbuted protocol to gude multple selfsh nodes to a Pareto-optmal Nash equlbrum. Lee et al. [7], [8] reverse-engneer exponentalbackoff-based MAC protocols usng a noncooperatve game model. Related work also ncludes [9] that proposes an dle sense access method wthout estmatng the number of nodes, whch compares the mean number of dle slots between transmsson attempts wth the optmal value and adopts an addtve ncrease and multplcatve decrease algorthm to dynamcally control the contenton wndow n order to mprove throughput and short-term farness. However, dle sense method ntends to make contenton wndows equal for all wreless nodes and requres the calculaton of optmal average number of dle slots between transmssons. It s not clear how to acheve ths wth dfferent traffc classes. A prorty-based protocol s proposed n [] to acheve farness among flows of dfferent traffc classes, whch estmates the number of nodes n each class every step and computes the contenton wndow sze usng these estmates. However, as commented n [], protocols lke that n [] based on estmatng the number of nodes do not converge. III. GAME-THEORETIC MODEL OF CONTENTION A. Random Access Game CONTROL Consder a set N of wreless nodes n a wreless LAN wth contenton-based medum access. In ths paper, we consder sngle-cell wreless LANs, where every wreless node can hear every other node n the network. We assume all nodes always have a frame to transmt, and the network s nose free and packet loss s only due to collson. In practce, t s hard for wreless nodes to learn the exact channel access probabltes of others. Each node nfers the contenton of the wreless network through observng certan contenton measure sgnals q (p), whch are functons of the nodes channel access probabltes. Followng [], [2], we model the nteracton among wreless nodes as a random access game. Defnton : A generalzed random access game G s defned as a trple G := {N, (S ) N, (u ) N }, where N s a set of players (wreless nodes), player N strategy S := {p p [ν, ω ]} wth ν < ω, and payoff functon u (p) = U (p ) p C (q ) wth utlty functon U (p ) and prce functon C (q ). Ths defnton of game model s an extenson of the basc random access game model proposed n [], [2], whch was defned for the network where each wreless node observes a sngle contenton measure q, to the network where each node can observe multple contenton measure sgnals q. The dfference s the ntroducton of a prce functon C (q ), nstead of adaptng drectly to a contenton measure q. Although we can reduce the above defnton to the basc random access game model by defnng the contenton measure q = C (q ), the ntroducton of the prce functon enables us to gve physcal nterpretaton to the contenton measure sgnals. As shown n [], [2], random access game s a rather general model for contenton control, as t can be reverseengneered from exstng protocols. To see ths, note that the equlbrum pont of () defnes an mplct relaton between channel access probablty p and contenton measure sgnals q. If ths relaton can be wrtten as C (q ) = F (p ), (2) the utlty functon of each node s defned as U (p ) = F (p )dp. (3) Therefore, we can reverse engneer medum access control protocols and study them n game theoretc framework: medum access control can be nterpreted as a dstrbuted strategy update algorthm to acheve the equlbrum of the random access game. In random access game, one of the most mportant questons s whether a Nash equlbrum exsts or not. Denote the channel access probablty for all nodes but by p := (p,..., p, p +,..., p N ), and wrte (p, p ) := p. We have the followng defnton of Nash equlbrum [2]. Defnton 2: A channel access probablty vector p s sad to be a Nash equlbrum f no node can mprove ts payoff by unlaterally devatng from Nash equlbrum,.e., u (p, p ) u (p, p ), p S. A Nash equlbrum p s a nontrval equlbrum f p satsfes p u (p, p ) =, N. (4) The reason to consder nontrval Nash equlbrum s to avod those equlbra n whch some player takes strategy at the boundary of the strategy space, whch usually results n great unfarness or low payoff. Throughout ths paper, we wll only consder those contenton measure sgnals that can be descrbed by q = G (p ). To facltate analyss n the followng, we lst the assumptons that wll be used n ths paper. A: The utlty functon U ( ) s twce contnuously dfferentable, strctly concave and ncreasng, wth fnte

3 IEEE JOURNAL ON SELECTED AREAS OF COMMUNICATIONS, VOL. XX, NO. XX, DECEMBER 28 3 curvatures bounded away from zero,.e., there exst some postve constants µ and χ such that µ /U (p ) χ. A2: The nverse functon (U ) (C (q )) maps any q nto a pont wthn S for all N. A3: At a nontrval Nash equlbrum p, there exsts a functon Φ (p ) for each node such that Φ (p ) = Φ j(p j ),, j N and Φ (p ) s strctly monotone n S, N. These assumptons are smlar to those specfed n [], [2], and smlarly, the followng results are mmedate. Theorem 3: Under assumpton A, there exsts a Nash equlbrum for any random access game G. Theorem 4: Suppose A2 holds. Random access game G has a nontrval Nash equlbrum. Theorem 5: Suppose that A and A3 hold and random access game G has a nontrval Nash equlbrum. If addtonally for all N, C (q (p)) s strctly ncreasng n p, then G has a unque nontrval Nash equlbrum. Snce the equlbrum determnes the operatng pont of medum access control, t s desred to have a unque nontrval Nash equlbrum. Theorem 5 guarantees the unqueness of nontrval Nash equlbrum. Ths wll facltate the desgn of medum access methods. B. Utlty Functon Desgn As shown n the last subsecton, random access game can be reverse engneered from the extng protocols. An example of reverse-engneerng the IEEE 82. DCF was gven n [], [2]. In the followng, we gve several examples to show how to desgn utlty functons and random access games by forward engneerng from desred operatng ponts and based on heurstcs. ) Forward Engneerng from Desred Operatng Ponts: A System of Homogeneous Nodes: In [9], a medum access method s proposed by usng the mean number of dle slots between transmsson attempts. Let T c denote the average collson duraton and T SLOT denote the slot duraton. It s derved n [9] that when the number of users n the network N, the throughput-optmal number of dle slots between two transmsson attempts s e ξ n opt =, (5) e ξ where ξ satsfes ξ = ηe ξ and η = T SLOT /T c. Note that n opt s completely determned by the protocol parameters but not by the number of nodes n the network. Let q := j N /{} ( p j). The probablty of an dle slot s ( p )( q ) = nopt n opt + = e ξ. (6) Applyng (3) wth C (q ) = q, we obtan the utlty functon as U (p ) = p + e ξ log( p ). (7) Note that U (p ) does not satsfy A2 but clearly the random access game wth utlty (7) has a nontrval Nash equlbrum. Ths also shows the lmtaton of Theorem 4. Utlty (7) does not satsfy Theorem 5. In fact, there exst nfnte number of equlbra for the game wth (7). To desgn a game wth unque equlbrum, we note that when N s large the optmal attempt probablty that maxmzes the throughput s very small as shown n [9]. We thus have ( p ) α ( q ) = ( p ) α e ξ e ξ, (8) where α > and the approxmaton holds when α s not very large. Applyng (3), we obtan the utlty functon as U (p ) = p + e ξ α ( p ) α. (9) Note that (9) stll does not satsfy A2. But at least one nontrval Nash equlbrum exsts,.e., p = e ξ/(α+ N ). Defne Φ (p ) = ( p )( U (p e )) = ξ ( p ), whch s α strctly ncreasng n p when α >. Also q (p) s strctly ncreasng n p. By Theorem 5, the random access game G has a unque nontrval Nash equlbrum. Note that due to the approxmaton made n (8), the equlbrum pont obtaned by (9) may not acheve the optmal number of dle slots n opt. We wll dscuss n Secton IV-D how to desgn equlbrum selecton algorthm such that the equlbrum pont by usng (9) can actually ht n opt. A System of Heterogenous Nodes: We have consdered the network wth a sngle traffc class. We now consder a network wth n > dfferent traffc classes as n []. Let φ denote the weght assocated wth class- traffc and f denote the set of nodes carryng class- traffc, n. Wthout loss of generalty, we assume that = φ > φ 2 > > φ n > and each wreless node carres only one class traffc. At equlbrum, to acheve the desred farness among the nodes carryng the same traffc class, say class, the channel access probabltes of these nodes must be equal, denoted as p. To acheve the desred farness among dfferent traffc classes, we must have [] p ( p j ) = p j( p ),, j n. () φ φ j The probablty of a successful transmsson from any node carryng class- traffc s P t = f p ( p ) f j ( p j ) fj, () and the probablty of an dle slot s P I = ( p ) f. (2) The optmal channel access probablty of each node can be obtaned by maxmzng the throughput subject to farness constrant (). As shown n [], t s dffcult to maxmze the throughput drectly. We nstead consder the case that all f are large. Let f p = ξ and ξ = n = ξ. By followng smlar argument as n [9], we fnd that the throughput s maxmzed when ξ s the soluton to ξ = ηe ξ where η s defned as n (5). Note that p also needs to satsfy (). When f s large, we use the approxmaton p. Fnally, we obtan ξ = f φ n j= f j φ j ξ. (3) In [], n + contenton measurements are used: the average number of consecutve dle slots and the average number of tme slots between two consecutve successful class-j transmssons, whch are equvalent to n+ contenton

4 IEEE JOURNAL ON SELECTED AREAS OF COMMUNICATIONS, VOL. XX, NO. XX, DECEMBER 28 4 measure sgnals q k = P I and q k = P t, =,..., n, at node k N. At equlbrum, we have q k = f p ( p ) f j ( p j ) f j f p e ξ ξ e ξ, and q k = (4) ( p ) f e ξ. (5) From (4), we obtan f e ξ q k /p. By usng (3) and assumng p s small, we obtan q k ( p ) α q k φ /p n j= q q k ξ, (6) kjφ j /p j where α >. Applyng (3), we obtan the utlty functon U k (p ) = φ ξ α + ( p ) α+, =,..., n, (7) and the prce functon n j= C k (q k ) = q kjφ j /p j, =,..., n, (8) q k /p where q k = [q k, q k,..., q kn ] T. Note that there are n utlty and prce functons at each node k. Each node k keeps and updates ts own p, =,..., n, denoted as p k. At node, p n (8) actually means p k. Computng (8) does not requre nformaton exchange between nodes. The game wth utlty (7) and prce (8) does not qute ft nto the random access game model. The Nash equlbrum s not the proper soluton concept ether. Instead, the nontrval equlbrum should be defned as those p that satsfes p U (p ) = C (q ), N. (9) Clearly, there exsts a nontrval equlbrum n the resultng random access game. We can further show that ths nontrval equlbrum s unque provded that p < / N, N [3]. 2) Forward Engneerng Based on Heurstcs: Consder a random access game wth the followng payoff functon u (p) := U (p ) p ( p j ) = U (p ) p q, (2) j where C (q ) = q = j ( p j) s the contenton measure sgnal representng the probablty that all nodes except node do not transmt. Ths payoff functon s motvated by the heurstc that each wreless node should be charged accordng to the throughput t acheves. It turns out that the random access game wth payoff (2) s a supermodular game [4]. Supermodular games have many nce propertes such as the exstence of Nash equlbra and the convergence of the equlbra under dfferent strategy update algorthms. The smplcty of supermodular games makes concavty/convexty and dfferentablty assumptons unnecessary, though we make such assumptons n ths paper. In the settng of random access games, the defnton of supermodularty and supermodular game reduces to the followng. Defnton 6: The payoff functon u (p, p ) has ncreasng dfferences (supermodularty) n (p, p ) f for all p p the quantty u (p, p ) u (p, p ) s ncreasng n p. For twce dfferentable payoffs, supermodularty s equvalent to 2 u (p) p p j for all j. Defnton 7: A random access game G s supermodular f for each node N the payoff functon u (p, p ) has ncreasng dfferences n (p, p ). It s easy to verfy that 2 u (p)/ p p j = j,j j ( p j ). The followng result s mmedate [4]. Theorem 8: A random access game G wth the payoff functon (2) s a supermodular game, and the set of Nash equlbra for G s nonempty. As ndcated by Theorem 8, no concavty/convexty assumpton on utlty functon s requred to guarantee the exstence of Nash equlbra as n non-supermodular games. However, the unqueness of Nash equlbrum may requre stronger condton. Smlarly to Theorem 5, we have the followng corollary on the unqueness of equlbrum for supermodular random access games. Corollary 9: Suppose that utlty functon U ( ) s twce contnuously dfferentable, ncreasng and strctly convex, and the supermodular random access game G wth the payoff (2) has a nontrval Nash equlbrum. If Φ (p ) = ( p )U (p ) s a strctly monotone functon n S, then G has a unque nontrval Nash equlbrum. As an example, we consder the followng utlty functon gven n [], [2] U (p ) := a ( (a )b a ln (a p b ) p ), (2) where < b <, a <, and p ( b b + b a, 2 +a (a b b 2 b ) a ). It s easy to check that U (p ) s strctly convex and Φ (p ) < when p < b + b 2 +a (a b b 2 b ) a. From Corollary 9, the supermodular game wth utlty functon (2) has a unque nontrval Nash equlbrum. There are many ways to desgn utlty functons and random access games. We only show a few examples n ths secton. The key message s that the random access game model s a rather general constructon, as we can derve or desgn the game by reverse engneerng from exstng protocols and by forward engneerng from desred operatng ponts and based on heurstcs. IV. DYNAMICS OF RANDOM ACCESS GAME The dynamc of game studes how players could converge to an equlbrum. It s a dffcult problem n general. In random access games, wreless nodes can observe the outcome of the actons of others, but do not have drect knowledge of other nodes actons and payoffs. We consder repeated play of random access game, and look for dstrbuted strategy update mechansm to acheve the Nash equlbrum. A. Basc Dynamc Algorthms ) Best Response: The smplest update mechansm s the best response strategy: at each stage, every node chooses the best response to the actons of other nodes n the prevous stage. Let p() be the largest vector n the strategy space

5 IEEE JOURNAL ON SELECTED AREAS OF COMMUNICATIONS, VOL. XX, NO. XX, DECEMBER 28 5 (S ) N. At stage t +, node chooses a channel access probablty { } p (t+) = B (p(t)) := max arg max u (p, p (t)) p S. (22) At each stage, f there are more than one best response probabltes, the algorthm (22) always chooses the largest probablty. Clearly, f the above dynamcs reaches a steady state, ths state s a Nash equlbrum. As there s no convergence result for general games usng ths dynamcs, we restrct our dscusson to supermodular games wth payoff (2) n ths subsecton. We have the followng result. Theorem : The best response strategy converges to a Nash equlbrum of random access game G. Moreover, t s the largest equlbrum p n the set of Nash equlbra. The proof bascally follows [4, Lemma 4.] and the detals can be found n [3], [5]. If we set p() to the smallest vector n the strategy space and always choose the smallest probablty, the best response strategy wll converge to the smallest equlbrum p. When there exst multple equlbra, the followng theorem ndcates that the equlbrum attaned by (22) yelds the hghest aggregate payoff. Theorem : The best response strategy converges to a Pareto domnant equlbrum,.e., u (p) u (p) for all p n the strategy space. The followng result guarantees that the best response converges to a nontrval equlbrum. Theorem 2: If the best responses to the smallest and largest vectors n the strategy space are wthn the strategy space, then nontrval Nash equlbrum exsts. Moreover, the best response strategy (22) converges to the largest nontrval Nash equlbrum. The proof of Theorem and Theorem 2 can be found n [3], [5]. By usng Theorem 2, t s easy to obtan condtons on a and b n (2) such that the best response strategy converges to a nontrval equlbrum of the correspondng game. 2) Gradent Play: One other update mechansm s gradent play [6]. In gradent play, every node adjusts ts channel access probablty gradually n a gradent drecton suggested by contenton measure sgnals. At stage t +, node N updates ts strategy accordng to p (t + ) = [p (t) + ɛ (t)(u (p (t)) C (q (p(t))))] S, (23) where the stepsze ɛ ( ) >, [ ] S denotes the projecton onto node s strategy space. In the followng, We assume that all nodes have the same stepsze ɛ (t) = ɛ(t), N. Theorem 3: Let C(p) = (C (q (p))) be a mappng and J C = (Jj C ) be the Jacoban of C(p). Suppose that the smallest egenvalue of J C, λ mn (J C ), satsfes µ + λ mn (J C ) >, max j J C 2 j M, and the random access game has a unque nontrval Nash equlbrum p. The gradent play (23) converges geometrcally to p f the stepsze ɛ(t) < µ+λmn(jc ) χ 2 + N M. The proof of Theorem 3 s gven n Appendx A. Theorem 3 also shows the convergence rate of gradent play. As an example of usng Theorem 3, we consder the utlty functon defned n (9). By assumng that all nodes strategy spaces are dentcal,.e., S = [ν, ω]. In ths case, we have µ = αe ξ ( ν) α+, χ = αe ξ. (24) ( ω) α+ To fnd λ mn (J C ), we note that ( (dag(x) J C (p) = ( p )) 2 xx T ), (25) [ ] T where x = p,..., p N. Note that each entry of x s less than ω. By usng Raylegh quotent [7], t s easy to show that the maxmum egenvalue of dag(x) 2 xx T s less than ( ω). Thus, Theorem 3 requres that 2 λ mn (J C ( υ) N ) + µ ( ω) 2 + αe ξ >. (26) ( ν) α+ Condton (26) s mld. For example, f we take ω = 2/33 and α = 2, all ν [, ] satsfy (26). We see that a larger α ndcates a larger µ, whch means a greater convergence rate by (44). 3) Jacob Play: Fnally, we consder another alternatve strategy update mechansm called Jacob play [8]. In Jacob play, every player adjusts current channel access probablty gradually towards the best response strategy. At stage t +, node N chooses a channel access probablty p (t+) = J (p(t)) := [p (t)+ɛ (t) (B (p(t)) p (t))] S, (27) where the stepsze ɛ (t) > and B (p(t)) s defned n (22). When ɛ (t) =, we recover the best response strategy. In the case of supermodular game, f ɛ (t), t s easy to verfy that {p (t)} s a nonncreasng sequence. Thus, Theorem 2 stll apples to Jacob play. For general random access games, we can also show the convergence of Jacob play n the same way as n gradent play, see [3] for detals. B. Dynamc Algorthms under Propagaton Delay Due to propagaton delay, wreless nodes may use feedback sgnals generated at dfferent tmes. In ths subsecton, we dscuss the convergence of the algorthms n Secton IV- A under propagaton delay. We assume that the contenton measure sgnals that node uses to update ts channel access probablty result from the vector ) p(τ (t)) = (p (τ(t)), p 2 (τ2(t)),..., p N (τ N (t)), (28) where τj (t) t denotes the most recent tme that node j s acton affects node s observaton, and τ (t) = t. ) Best Response: The best response strategy (22) s modfed to { p (t+) = B (p(τ ( (t))) := max arg max u p, p (τ (t)) ) }. p S (29) Parallel to Theorem, we have the followng theorem on the convergence of the best response under propagaton delay for supermodular games. Theorem 4: The best response strategy (29) converges to a Nash equlbrum of the random access game G. Furthermore, t s the largest equlbrum n the set of Nash equlbra.

6 IEEE JOURNAL ON SELECTED AREAS OF COMMUNICATIONS, VOL. XX, NO. XX, DECEMBER 28 6 Proof: We show ths by nducton. Suppose that p(τ + ) p(τ), τ {,..., t }. It s true when t = as p() s the largest vector n the strategy space. As τj (t+) τ j (t), we have p j (τj (t + )) p j(τj (t)). By nducton hypothess, we get p (τj (t + )) p (τj (t)). By supermodularty and [4, Lemma 4.], we can show that p (t + ) p (t). Therefore, the hypothess s also true when τ = t. By nducton, we have p() p() p(t), (3) or {p(t)} s a nonncreasng sequence. The remander of proof follows that of Theorem. All other results n Secton IV-A for best response also hold n the case under delay. 2) Gradent Play: The gradent play (23) s modfed to p (t + ) = [ p (t) + ɛ (t) ( U (p (t)) C (q (p(τ (t)))) )] S. (3) Snce at each step, nodes update channel access probabltes by a small amount, gradent play s expected to converge f τj (t) s not far away from t, j N. The followng result confrms ths ntuton. Theorem 5: Let C(p) = (C (q (p))) be a mappng and J C = (Jj C ) be the Jacoban of C(p). Assume a constant stepsze ɛ (t) = ɛ n (3). Suppose that J C M, max j Jj C 2 M 2, and the random access game has a unque nontrval Nash equlbrum p, and t τj (t) B wth constant B. The gradent play (3) geometrcally converges to p f there exsts ɛ > and < γ < such that N γ = 2ɛ (µ M (χ + ɛ 2 + M 2 N 2 ) ). (32) γ B The proof s gven n Appendx B. We can smlarly establsh the convergence of the Jacob play under propagaton delay. γ B C. Dynamc Algorthms under Estmaton Error In ths secton, we consder dynamc algorthms under estmaton error. The dynamc algorthms requre the nformaton of contenton measure sgnals. In practce, contenton measure sgnals can be estmated va the observaton of the wreless medum. As an example, we consder the contenton measure condtonal collson probablty used n the game for the network of homogeneous nodes studed n Secton III-B.. Let n and n denote the number of consecutve dle slots and ts mean between two transmssons. As proposed n [], [2], we can estmate the condtonal collson probablty by observng the dle perod between transmssons: at every ntrans transmssons, each node updates n accordng to n β n + ( β) sum ntrans, where sum s the total number of dle slots durng ntrans transmssons, and estmates ts condtonal collson probablty accordng to q = ( n+)p ( n+)( p ). Due to the use of estmated contenton measure sgnals, the algorthms n Secton IV-A are n fact stochastc algorthms. In the followng, we only consder gradent play. The results for Jacob play can be obtaned smlarly. We assume that C (q (p(t))) s replaced by Ĉ(q (p(t))) = C (q (p(t))) + w (t) n (23), where w (t) s the estmaton error. Wthout loss of generalty, we wrte w (t) as w (t) = w (t) + w (t), where w (t) = E{w (t)} can be consdered as the determnstc error and w (t) = w (t) w (t) s the stochastc error wth zero mean. We further assume that lm t w (t) = w. The determnstc error may be caused by the bas of sgnal estmaton and carrer sense error due to fadng and background nose. For ease of understandng, n the followng, we dscuss determnstc and stochastc errors separately. The proof of the followng theorems can be found n Appendces C, D and F. Theorem 6: Let λ mn (J C ) denote the smallest egenvalue of J C and max j Jj C 2 M. Let p denote the equlbrum defned by U (p ) = C (q (p )) + w. (33) If p s wthn the strategy space and s the unque equlbrum defned by (33), the gradent play converges to p provded µ + λ mn (J C ) > and ɛ(t) < µ+λmn(jc ) χ 2 +4 N M. The unqueness of p can be obtaned by usng Theorem 5. Note that under certan condtons, by mplct functon theorem [9], (33) defnes an mplct functon p ( w) at the neghborhood of w =. Therefore, for any ɛ >, there exsts a δ > such that f w 2 < δ, p ( w) p () 2 < ɛ. So the gradent play converges to a neghborhood of the equlbrum pont wthout errors. For the stochastc error, we consder gradent play wth varable stepsze and constant stepsze, respectvely. Theorem 7: Let λ mn (J C ) denote the smallest egenvalue of J C. Suppose that E{w (t)} =, E{w 2 (t)} B, and ɛ(t) =, ɛ 2 (t) <, e.g., ɛ(t) = /t. (34) t= t= If p s the unque nontrval Nash equlbrum, the gradent play converges to p wth probablty provded µ + λ mn (J C ) >. Theorem 8: Let λ mn (J C ) denote the smallest egenvalue of J C and max j J C 2 j M. Suppose that E{w (t)} =, E{w 2(t)} B, and ɛ(t) = ɛ, t. If p s the unque nontrval Nash equlbrum, there exsts a constant D(B, ɛ) > such that lm sup p(t) p 2 D(B, ɛ) (35) t provded µ + λ mn (J C ) > and ɛ < µ+λmn(jc ) χ 2 +4 N M. By combnng Theorems 6 and 8, we can conclude that wth constant stepsze, the stochastc gradent play converges to a neghborhood of the equlbrum pont. D. Equlbrum Selecton The equlbrum attaned by usng the dynamc algorthms n Secton IV-A does not necessarly converge to the desred operatng pont when the utlty functons n Secton III-B. are consdered. Ths s because the approxmaton used n (8). One approach of equlbrum selecton s to estmate the number of users va ˆN = log( q )/ log( p ) + at equlbrum and to set the channel access probablty to be the optmal value computed by usng ˆN. However, as commented n [], ths

7 IEEE JOURNAL ON SELECTED AREAS OF COMMUNICATIONS, VOL. XX, NO. XX, DECEMBER 28 7 approach may not converge due to open loop control. The other approach s to use an outer loop teraton and treat the algorthms n Secton IV-A as the nner loop teraton. Take utlty functon (9) for example. Let τ denote the counter of outer loop teraton and defne the utlty functon at the τ-th outer teraton as U (p ) = p + η(τ) α ( p ) α, (36) wth η() = e ξ. Denote the equlbrum of the game wth utlty (36) by p(τ). To cancel the effect of neglectng ( p ) α n (8), we do the outer teraton η(τ + ) = ( p (τ)) α e ξ. (37) At equlbrum, all nodes have the same access probablty, denoted as p(τ). By (37), we obtan p(τ + ) = N +α ( p(τ)) α e ξ. (38) Let M(p) be the mappng defned by (38). By mean value theorem, t s easy to see M(p ) M(p 2 ) e N +α (α )( ω) N + α ξ e ξ α N +α (α )( ω) N +α α N +α p p 2 (39) Thus, f N +α <, M(p) s a contracton mappng [9] and (38) converges to the unque fxed pont of M(p), whch s the desred operatng pont. From (39), we can see that a larger α ndcates a smaller outer loop convergence rate, whle a larger α results a greater nner loop convergence rate as suggested n Theorem 3. Therefore, there exsts an optmal α to acheve the least overall convergence rate. In practce, outer loop teraton can be executed wthout watng for the convergence of the nner loop teraton. V. EXPERIMENTAL RESULTS In ths secton, we run some numercal experments to compare the performance of dfferent medum access protocols. The system parameters are those specfed n the IEEE 82.b standard wth DSSS PHY layer [2], summarzed n Table I. The RTS/CTS mechansm s dsabled. We consder a sngle-cell network wth perfect wreless channel,.e., there s no corrupted frame. In all smulatons, the ntal channel access probablty s set to be 2/33, whch corresponds to CW mn = 32 n 82.b DCF. For our game based protocols, we set ntrans = 5 and β =.8. Throughput and farness are obtaned after 6 transmssons. A. One-sgnal Game We consder the one-sgnal game wth utlty functon (9) derved n Secton III-B., and compare the performance of the MAC based on ths game wth that of dle sense protocol n [9]. In (9), we choose ξ =.622 and α = 2. The parameters n dle sense are set as those n [9]. Fgure compares the evoluton of channel access probablty of dle sense and gradent paly (23) of the one-sgnal game n a network of 2 nodes, where for the gradent play ξ =.622, α = 2 and the stepsze s chosen to be TABLE I PARAMETERS IN SIMULATIONS Slot Tme (T SLOT ) 2 µs SIFS µs DIFS 5 µs Basc Rate Mbps Data Rate Mbps Propagaton Delay µs PHY Header 92 bts MAC Header 272 bts ACK 2 bts Packet Payload (s d ) 2 bts ɛ (t) =. We see even wth perfect knowledge of expected number of dle slots, dle sense oscllates around the optmal value. On the other hand, game model acheves a smoother dynamc n both cases wth perfect sgnal and estmated sgnal. Both algorthms have roughly the same convergence rate. We can clearly see the geometrc convergence rate predcted by Theorem 3. The equlbrum by our method s close to the optmal value but not equal due to the approxmaton n (8). Channel access probablty Perfect Idle Sense Estmated Idle Sense Number of teratons (a) Idle sense Channel access probablty Perfect Gradent Play Estmated Gradent Play Optmal Number of teratons (b) Game model Fg.. The evoluton of channel access probablty of dle sense and gradent play of the one-sgnal game wth utlty functon (9) n a network of 2 nodes. Fgure 2 compares the throughput of dle sense, the game based desgn, and DCF wth the same parameters as used n Fgure. We use estmated sgnals n dle sense and the game based desgn. When the number of nodes n the network s small, dle sense acheves the hghest throughput. Game based desgn performs worse n ths case because the approxmaton used n (8) s not accurate when the number of nodes s small. The performance of the game based desgn can be mproved by usng equlbrum selecton algorthm. As the number of users ncreases, both dle sense and the game based desgn perform farly close to the optmal throughput. They acheve a much hgher throughput than DCF. Ths also ndcates that when the number of users s large, equlbrum selecton s not necessary as the acheved throughput by the game based desgn s already very close to the optmal throughput. Fgure 3 compares the short-term farness of dfferent protocols usng Jan farness ndex [2] for normalzed wndow szes that are multples of the number of wreless nodes. All parameters are the same as those used n Fgure. We see that both dle sense and the game based desgn provde much better short-term farness than 82.b as n both protocols wreless nodes have roughly the same contenton wndow sze.

8 IEEE JOURNAL ON SELECTED AREAS OF COMMUNICATIONS, VOL. XX, NO. XX, DECEMBER 28 8 Throughput (Mbps) Game Model Idle Sense 82.b DCF Optmal Channel access probablty Number of teratons Channel access probablty Perfect Gradent Play, Class Estmated Gradent Play, Class Optmal, Class Perfect Gradent Play, Class 2 Estmated Gradent Play, Class 2 Optmal, Class Number of teratons 5 (a) P-MAC (b) Game model Number of nodes Fg. 2. The throughput comparson between dle sense and the MAC based on one-sgnal game wth utlty functon (9) n a network of 2 nodes. Jan ndex Game Model Idle Sense 82.b DCF.5 Normalzed wndow sze Fg. 3. The farness comparson between dle sense and the MAC based on one-sgnal game wth utlty functon (9) n a network of 2 nodes. B. N-sgnal Game Next, we compare P-MAC protocol proposed n [] wth the MAC based on the N-sgnal game wth utlty functon (7) and prce functon (8) derved n Secton III-B.. In (7), we choose ξ =.622 and α = 2. The parameters n P-MAC are set as those n []. Fgure 4 compares the dynamcs of P-MAC and the MAC based on N-sgnal game n a network of 25 nodes and two traffc classes wth f =, f 2 = 5 and weghts φ =, φ 2 =.5. The N-sgnal game uses gradent play, wth the stepsze ɛ (t) =.5. The ntal value of wndow sze s chosen to be CW mn plus a random number between and. We see P-MAC does not converge because t uses open loop control, whch agrees wth the observaton n []. On the other hand, the game based desgn converges to the equlbrum only after teratons wth perfect contenton measure sgnals. Even wth estmated sgnals, the game based desgn converges to a neghborhood of the equlbrum after 2 teratons. In Fgure 4, we also show the optmal channel access probablty that acheves the maxmum throughput. The equlbrum of class s less than the optmal value, whle the equlbrum of class 2 s greater than the optmal value. Ths Fg. 4. The dynamcs of P-MAC and the MAC based on N-sgnal game wth utlty functon (7) and prce functon (8) n a network of 25 nodes and two traffc classes. We choose ξ =.622 and α = 2 n (7). s because we use the approxmaton (3). As wll be shown later, ths approxmaton does not affect the throughput and farness too much when the number of users s large. Fgure 5 compares the throughput of P-MAC and the MAC based on N-sgnal game. We use estmated sgnals n both protocols. The number of nodes n two traffc classes are K 3 and 2K 3, respectvely, K =,..., 5. When the number of nodes n the network s small, P-MAC acheves the hghest throughput. The game based desgn performs worse n ths case because of the approxmaton used n (3). As the number of users ncreases, both P-MAC and the game based desgn perform farly close to the optmal throughput. They acheve a much hgher throughput than DCF. Throughput (Mbps) Game Model P MAC 82.b DCF Optmal 4 Number of nodes Fg. 5. The throughput comparson between P-MAC and the MAC based on N-sgnal game wth utlty functon (7) and prce functon (8). Fgure 6 compares the short-term farness of dfferent protocols usng Jan farness ndex [2]. All parameters are the same as those used n Fgure 4. We see that both P-MAC and the game based desgn provde much better farness than 82.b as 82.b does not dfferentate dfferent traffc classes. To show that the game based desgn s well-behaved n the presence of traffc fluctuatons, we consder a network wth a varable number of nodes, as shown n Fgure 7. At frst, f = and f 2 = 5. After 3 teratons, 5 class- and 5 class-2 traffc nodes enter the network. After 6 teratons, 5 class- and 5 class-2 traffc nodes leave the network. P-MAC

9 IEEE JOURNAL ON SELECTED AREAS OF COMMUNICATIONS, VOL. XX, NO. XX, DECEMBER α=.5 α=. α=.5 α=2 α=2.5 Desred Operatng Pont Jan ndex Channel access probablty Game Model P MAC 82.b DCF.4 Normalzed wndow sze Fg. 6. The farness comparson between P-MAC and the MAC based on N-sgnal game wth utlty functon (7) and prce functon (8) Number of teratons Fg. 8. The dynamcs of the game wth utlty functon (9) usng equlbrum selecton n a network of 5 nodes. Dfferent α values for (9) are compared. Channel access probablty 5.5. Channel access probablty 5.5. Perfect Gradent Play, Class Estmated Gradent Play, Class Perfect Gradent Play, Class 2 Estmated Gradent Play, Class 2 the correspondng condtons are satsfed, and establshed the convergence of varous dynamc algorthms to the equlbrum. Smulaton results have shown that the game model based protocols acheve superor performance over the standard IEEE 82. DCF, and comparable performance as exstng protocols wth the best performance n lterature Number of teratons Number of teratons REFERENCES (a) P-MAC (b) Game model Fg. 7. The dynamcs of P-MAC and the MAC based on N-sgnal game wth utlty functon (7) and prce functon (8) n the presence of traffc fluctuatons n a network wth two traffc classes. stll does not converge. The game based desgn responses to traffc fluctuaton very fast. Wth estmated contenton measurement sgnals, the game based desgn oscllates around the equlbrum. C. Equlbrum Selecton Fnally, we check the equlbrum selecton algorthm descrbed n Secton IV-D. We consder a network of 5 nodes. The gradent play for the game wth utlty functon (9) s smulated, where ξ =.622 and the stepsze ɛ (t) =. We assume perfect contenton measure sgnals and we decde that the nner loop convergence s attaned f p(t + ) p(t) Fgure 8 compares the evoluton of channel access probablty wth dfferent α values for (9). We see that the nner loop convergence rate ncreases by ncreasng α, whle the outer loop convergence rate decreases by ncreasng α. VI. CONCLUSIONS We have generalzed the random access game model, and shown that t provdes a general framework for desgnng contenton based medum access control. Several examples have been gven on how to desgn random access games from reverse-engneerng and forward-engneerng. We have shown that the gven examples attan a unque equlbrum provded [] L. Chen, S. H. Low, and J. C. Doyle, Random access game and medum access control desgn, Mar chen/ papers/ramac.pdf, Caltech CDS, Tech. Rep. [2], Contenton control: A game-theoretc approach, n Proc. of IEEE CDC, Dec. 27, pp [3] Y. Jn and G. Kesds, Equlbra of a noncooperatve game for heterogeneous users of an Aloha networks, IEEE Commun. Lett., vol. 6, no. 7, pp , July 22. [4] A. B. MacKenze and S. B. Wcker, Stablty of multpacket slotted Aloha wth selfsh users and perfect nformaton, n Proc. of IEEE Infocom, Apr. 23, pp [5] V. S. Borkar and A. A. Kheran, Random access n wreless ad hoc networks as a dstrbuted game, n Proc. of WOpt, Mar. 24. [6] Čagalj, S. Ganerwal, I. Aad, and J. P. Hubaux, On selfsh behavor n CSMA/CA networks, n Proc. of IEEE Infocom, Mar. 25, pp [7] J. W. Lee, M. Chang, and R. A. Calderbank, Utlty-optmal randomaccess protocol, IEEE Trans. Wreless Commun., vol. 6, no. 7, pp , July 27. [8] J. W. Lee, A. Tang, J. Huang, M. Chang, and A. R. Calderbank, Reverse-engneerng MAC: A non-cooperatve game model, IEEE J. Select. Areas Commun., vol. 25, no. 6, pp , Aug. 27. [9] M. Heusse, F. Rousseau, R. Guller, and A. Dula, Idle sense: An optmal access method for hgh throughput and farness n rate dverse wreless LANs, n Proc. of ACM Sgcomm, Aug. 25, pp [] D. Qao and K. G. Shn, Achevng effcent channel utlzaton and weghted farness for data communcatons n IEEE 82. WLAN under the DCF, n Proc. of IWQoS, May 22. [] C. Hu and J. C. Hou, A novel approach to contenton control n IEEE 82.e-operated WLANs, n Proc. of IEEE Infocom, May 27, pp [2] D. Fudenburg and J. Trole, Game Theory. MIT Press, 99. [3] T. Cu, L. Chen, S. H. Low, and J. C. Doyle, Game-theoretc framework for medum access control, Aug taocu/rag.pdf, Caltech CDS, Tech. Rep. [4] D. M. Topks, Equlbrum ponts n nonzero-sum n-person submodular games, SIAM J. of Contr. and Optm., vol. 7, no. 6, pp , 979. [5] L. Chen, T. Cu, S. H. Low, and J. C. Doyle, A game-theoretc model for medum access control, n Proc. of Internatonal Wreless Internet Conference (WICON), Oct. 27.

10 IEEE JOURNAL ON SELECTED AREAS OF COMMUNICATIONS, VOL. XX, NO. XX, DECEMBER 28 [6] S. D. Flam, Equlbrum, evolutonary stablty and gradent dynamcs, Internatonal Game Theory Revew, vol. 4, no. 4, pp , Dec. 22. [7] R. A. Horn and C. R. Johnson, Matrx Analyss. Cambrdge Unversty Press, 985. [8] R. La and V. Anantharam, Utlty based rate control n the nternet for elastc traffc, IEEE/ACM Trans. Networkng, vol., no. 2, pp , Apr 22. [9] R. Abraham, J. E. Marsden, and T. Ratu, Manfolds, Tensors, Analyss, and Applcatons, 2nd ed. Sprnger-Verlag, 988. [2] Wreless LAN meda access control (MAC) and physcal layer (PHY) specfcatons, IEEE Standard 82., June 999. [2] R. Jan, D. Chu, and W. Hawe, A quanttatve measure of farness and dscrmnaton for resource allocaton n shared computer systems, DEC Research Report TR-3, Sept [22] D. P. Bertsekas and J. N. Tstskls, Gradent convergence n gradent methods wth errors, SIAM J. Optm., vol., no. 3, pp , May 999. APPENDIX A: PROOF OF THEOREM 3 Proof: By equaton (23), we have p(t + ) p 2 2 p (t) + ɛ(t)(u (p (t)) C (p(t))) p N p(t) p ɛ(t) + ɛ 2 (t) (a) p(t) p ɛ(t) 2ɛ(t) + ɛ 2 (t) U (p (t)) C (p(t)) 2 (p (t) p ) U (p (t)) C (p(t)) (p (t) p )(U (p (t)) U (p )) (p (t) p ) (C (p(t)) C (p )) U (p (t)) C (p(t)) 2, 2 (4) where we have used C (p(t)) to denote C (q (p(t))). In (a), we use the fact that U (p ) = C (p ) at the nontrval Nash equlbrum. By mean value theorem, we fnd (p (t) p ) U (p (t)) U (p ) = U ( p )(p (t) p ) 2 µ p(t) p 2 2, (4) where p = γp (t) + ( γ)p, γ. Defne a scalar functon f(p) = (p(t) p ) T C(p). By mean value theorem, we have f(p(t)) f(p ) =(p(t) p ) T J C ( p)(p(t) p ) We also have U (p (t)) C (p(t)) 2 = 2 λ mn(j C ) p(t) p 2 2. U (p (t)) U (p ) + C (p ) C (p(t)) 2 (U (p (t)) U (p )) (a) 2χ 2 p(t) p χ 2 p(t) p (χ 2 + N M) p(t) p 2 2, (C (p(t)) C (p )) 2 J C ( p )(p(t) p ) 2 max j J C j ( p ) 2 ) p(t) p 2 2 (42) (43) where (a) comes from mean value theorem. Substtutng (4)- (43) nto (4), we obtan p(t + ) p 2 2 2ɛ(t) µ + λ mn (J C ) ɛ(t) χ 2 + N M p(t) p 2 2. (44) Therefore, f µ + λ mn (J C ) > and ɛ(t) < µ+λ mn(j C ) χ 2 + N M, p(t) converges to p geometrcally. APPENDIX B: PROOF OF THEOREM 5 Proof: We show ths by nducton. The proof bascally follows that of Theorem 3. For brevty, we omt several mmedate steps. Suppose that p(τ +) p 2 2 γ p(τ) p 2 2, τ {,..., t }, (45) where < γ < s a constant. When τ = t, by equaton (3), p(t + ) p 2 2 N p (t) + ɛ U (p (t)) C p(τ (t)) p p(t) p ɛ 2ɛ (p (t) p ) U (p (t)) U (p ) (p (t) p ) C (p(τ (t))) C (p ) + ɛ 2 U (p (t)) C (p(τ (t))) 2. By mean value theorem, we have f(p(τ (t))) f(p ) = (p(t) p ) T J C ( p) ( p(τ (t)) p ) J C ( p) p(t) p 2 p(τ (t)) p 2 M p(t) p 2 p(τ (t)) p 2. Note that p(τ (t)) p 2 2 = p j (τj(t)) p 2 j p(τ j(t)) p 2 j N j N γ τ j (t) t p(t) p 2 N γ B p(t) p 2. j N Smlar to (44), we obtan p(t + ) p 2 2 p(t) p 2 N 2 2ɛ µ M γ + ɛ B χ2 + M 2 N 2 γ B 2 (46) (47) (48) Therefore, f there exsts ɛ > and < γ < such that (32) holds, the nducton hypothess s true for τ = t. APPENDIX C: PROOF OF THEOREM 6 Proof: By followng (4), we obtan p(t + ) p 2 2 p(t) p ɛ 2 (t) + 2ɛ(t) γ p(t) p ɛω U (p (t)) C (p(t)) w (t) 2 (p (t) p ) U (p (t)) C (p(t) w (t)) w (t) w + 2ɛ 2 ( w (t) w ) 2, )) (49).

11 IEEE JOURNAL ON SELECTED AREAS OF COMMUNICATIONS, VOL. XX, NO. XX, DECEMBER 28 where γ = 2ɛ(µ + λ mn (J C ) ɛ(χ 2 + 4M)) and ɛ < µ+λ mn (J C ) χ 2 +4 N M. By assumpton lm t w (t) = w, for any δ >, there exsts a t such that f t t w (t) w < δ,. Applyng (49) recursvely, we obtan p(t + ) p 2 2 t t t t γ t t p(t ) p ɛω N δ γ τ + 2ɛ N δ 2 γ t t p(t ) p ɛω N δ γ τ= + τ= γ 2τ 2ɛ N δ2 γ 2. (5) By takng δ and t, we obtan lm sup t p(t) p 2 2 =. Therefore, p(t) converges to p. APPENDIX D: PROOF OF THEOREM 7 Proof: By followng (4), we obtan E p(t + ) p 2 2 p(t) p ɛ 2 (t) U (p (t)) C (p(t)) w (t) 2 + 2ɛ(t) (p (t) p ) U (p (t)) C (p(t) w (t)) { } p(t) p 2 2 ɛ(t)κ p(t) p ɛ 2 (t)e w 2 (t) { } 2ɛ(t)E (p (t) p ) w (t) p(t) p 2 2 ɛ(t)κ p(t) p ɛ 2 (t) N B, where (5) 2(µ + λ mn (J C ) ɛ(t)(χ N M)) > κ >. (52) From (34), t, κ such that for all t t, (52) holds. Takng expectaton both sdes of (5) over F t and applyng the resultng equaton recursvely, E p(t + ) p 2 2 E p(t ) p 2 2 κ ɛ(t)e p(t) p N B ɛ 2 (t), t=t t=t (53) from whch we get t=t ɛ(t)e{ p(t) p 2 2} <. Snce t= ɛ(t) = and E{ p(t) p 2 2}, p(t) converges to p wth probablty. APPENDIX E: PROOF OF THEOREM 8 Proof: By followng (49), we obtan p(t + ) p 2 2 p(t) p ɛ 2 (t) + 2ɛ(t) γ p(t) p 2 2 2ɛ U (p (t)) C (p(t)) w (t) 2 (p (t) p ) U (p (t)) C (p(t) w (t)) (p (t) p ) w (t) + 2ɛ 2 w 2 (t), (54) where γ s defned after (49). Applyng (54) recursvely, we obtan p(t + ) p 2 2 γ t p() p ɛ 2 w 2 (τ) 2ɛ τ= γ t τ τ= γ t τ (p (τ) p ) w (τ). (55) As E{w (t)} = and E{ w2 (τ)} B, by usng [22, Lemma 2], there exsts a constant D(B, ɛ) > such that lm nf t 2ɛ2 γ t τ τ= 2ɛ Therefore, we get (35). w 2 (τ) τ= γ t τ (p (τ) p ) w (τ) ) D(B, ɛ). (56) Tao Cu (S 4) receved the M.Sc. degree n the Department of Electrcal and Computer Engneerng, Unversty of Alberta, Edmonton, AB, Canada, n 25, and the M.S. degree from the Department of Electrcal Engneerng, Caltech, Pasadena, USA, n 26. He s currently workng toward the Ph.D. degree at the Department of Electrcal Engneerng, Caltech. Hs research nterests are n the nteractons between networkng theory, communcaton theory, and nformaton theory. Mr. Cu receved the Best Paper Award at the IEEE Internatonal Conference on Moble Ad-hoc and Sensor Systems (MASS) n 27 and the Second Place n the ACM Student Research Competton at the 27 Rchard Tapa Celebraton of Dversty n Computng Conference. Mr. Cu was a recpent of postgraduate scholarshps from the Alberta Ingenuty Fund and the Alberta Informatcs Crcle of Research Excellence. Ljun Chen (S 5-M 7) receved hs B.S. from Unversty of Scence and Technology of Chna, M.S. from Insttute of Theoretcal Physcs, Chnese Academy of Scences and from Unversty of Maryland at College Park, and Ph.D. from Caltech. He s currently a Research Scentst n the Control & Dynamcal Systems Department at Caltech. Hs research nterests are n communcaton networks, network economcs and onlne market, and optmzaton, game theory and ther engneerng applcaton. He was a co-recpent of the Best Paper Award at the IEEE Internatonal Conference on Moble Ad-hoc and Sensor Systems (MASS) n 27. Steven H. Low (F 8) receved hs B.S. from Cornell Unversty, and PhD from Berkeley, both n electrcal engneerng. He s a Professor of the Computer Scence and Electrcal Engneerng Departments at Caltech. He was wth AT&T Bell Laboratores, Murray Hll, from 992 to 996, the Unversty of Melbourne, Australa from 996 to 2, and was a Senor Fellow of the Unversty of Melbourne from 2 to 24. He was a co-recpent of the IEEE Wllam R. Bennett Prze Paper Award n 997 and the 996 R&D Award. He was on the edtoral board of IEEE/ACM Transactons on Networkng from 997 to 26 and on that of Computer Networks Journal from 23 to 25. He s on the edtoral boards of ACM Computng Surveys, NOW Foundatons and Trends n Networkng. He s a Senor Edtor of the IEEE Journal on Selected Areas n Communcatons and a Co-Edtor of Sprnger Book Seres on Optmzaton and Control of Communcaton Systems: Theory and Applcatons. He s a member of the Networkng and Informaton Technology Techncal Advsory Group for the US Presdent s Councl of Advsors on Scence and Technology (PCAST).