WITH the proliferation of smart wireless devices and mobile

Transcription

1 Ths arcle has been acceped for publcaon n a fuure ssue of hs journal, bu has no been fully eded Conen may change pror o fnal publcaon Caon nformaon: DOI 1119/TMC , I Transacons on Moble Compung 1 Dynamc Compuaon Offloadng for Moble Cloud Compung: A Sochasc Game-Theorec Approach Janchao Zheng, Member, I, Yuemng Ca, Senor Member, I, Yuan Wu, Senor Member, I, and Xuemn Sherman Shen, Fellow, I Absrac Drven by he growng populary of moble applcaons, moble ud compung has been envsoned as a promsng approach o enhance compuaon capably of moble devces and reduce he energy consumpons In hs paper, we nvesgae he problem of mul-user compuaon offloadng for moble ud compung under dynamc envronmen, wheren moble users become acve or nacve dynamcally, and he wreless channels for moble users o offload compuaon vary randomly As moble users are self-neresed and selfsh n offloadng compuaon asks o he moble ud, we formulae he moble users offloadng decson process under dynamc envronmen as a sochasc game We prove ha he formulaed sochasc game s equvalen o a weghed poenal game whch has a leas one Nash qulbrum N We quanfy he effcency of he N, and furher propose a mul-agen sochasc learnng algorhm o reach he N wh a guaraneed convergence rae whch s also analycally derved Fnally, we conduc smulaons o valdae he effecveness of he proposed algorhm and evaluae s performance under dynamc envronmen Index Terms Moble ud compung, mul-user compuaon offloadng, dynamc envronmen, sochasc game, mul-agen sochasc learnng 1 INTRODUCTION WITH he prolferaon of smar wreless devces and moble nerne servces, more and more moble applcaons such as neracve gamng, face recognon, and augmened realy have emerged and drawn ncreasng neress [] These sophscaed applcaons usually requre sgnfcan amouns of compuaon resources and energy consumpons, whch, however, canno be drecly afforded by mos moble devces due o her lmed compuaon resources and baery capaces [3], [4] Therefore, moble ud compung, whch enables moble devces o offload her compuaon asks o he resource-rch ud nfrasrucures such as Amazon C, Mcrosof Azure, and Google App ngne va wreless lnks, has been envsoned as a promsng approach o address hs challenge ssue [5] In ud nfrasrucure, each moble devce s assocaed wh a sysem-level ud ne runnng a vrual machne ha execues moble applcaons on behalf of he moble devce [6] Offloadng moble users compuaon asks o he moble ud nfrasrucure usually nvolves consderable communcaon burdens beween he ud and moble devces, whch hus necessae a careful desgn of mul-user compuaon offloadng sraegy o mprove wreless access effcency [7] A movang example s as follows When many moble devces aggressvely offload her compuaon asks o a moble ud over he same wreless channel, hey may generae severe co-channel nerference o each oher Such a severe nerference leads o lower offloadng raes e, moble users achevable daa raes for sendng compuaon asks o he moble ud over he wreless lnk and hgher energy consumpons for moble devces, whch consequenly compromse he benef of offloadng compuaon asks Therefore, s very mporan o acheve an effcen compuaon offloadng coordnaon when many moble devces compee for a lmed number of wreless channels o offload compuaon asks o he moble ud nfrasrucure Game heory s a wdely adoped mahemacal ool o model and analyze complcaed decson-makng processes among a group of raonal decson-makers of conflcng objecves [8], [9], [1], [11], [1] Snce dfferen moble devces are usually owned by dfferen users, s naural o adop game heory o analyze he compuaon offloadng process for mulple moble users who explo a common se of wreless channels o offload her compuaon asks Specfcally, each user s modeled as a raonal game player ha observes and reacs o oher users offloadng sraeges n he bes response manner Such an neracve decson process s expeced o reach an equlbrum pon also referred o as Nash qulbrum, N, a whch no ndvdual user wll change s offloadng sraegy unlaerally Moreover, by leveragng he nellgence of moble users, game heory s useful for desgnng decenralzed mechansms wh low complexy, whch help o ease he heavy conrollng and sgnalng overhead of complex cenralzed managemen [1] However, applyng game heory o model he mul-user compuaon offloadng process should carefully deal wh he complex Ths work s suppored by he Jangsu Provncal Naural Scence Foundaon of Chna under Gran BK17755, by he Naonal Posdocoral Program for Innovave Talens of Chna under Gran BX1719, by he Zhejang Provncal Naural Scence Foundaon of Chna under Gran LR17F1, and by he Naural Scence and ngneerng Research Councl NSRC, Canada Ths paper has been presened n par a he I/CIC ICCC Conference [1], July 16, Chengdu, Chna J Zheng s wh he Naonal Innovaon Insue of Defense Technology, Academy of Mlary Scences PLA, Bejng, 11, Chna He s also wh he College of Communcaons ngneerng, PLA Army ngneerng Unversy, Nanjng, 17, Chna emal: longxngrenzjcs@163com Y Ca s wh he College of Communcaons ngneerng, PLA Army ngneerng Unversy, Nanjng, 17, Chna emal: caym@vpsnacom acve or nacve 1 dynamcally, and wreless channels are also real nework envronmen Specfcally, moble users may become Y Wu s wh he College of Informaon ngneerng, Zhejang Unversy of Technology, Hangzhou, 313, Chna emal: ewuy@zjueducn 1 A moble user s acve f has a compuaon ask o be execued, whle X Shen s wh he Deparmen of lecrcal and Compuer ngneerng, Unversy of Waerloo, Waerloo, ON NL 3G1, Canada emal: sshen@uwaerlooca a moble user s nacve or slen f does no have a compuaon ask o be execued c 18 I Personal use s permed, bu republcaon/redsrbuon requres I permsson See hp://wwweeeorg/publcaons_sandards/publcaons/rghs/ndexhml for more nformaon

2 Ths arcle has been acceped for publcaon n a fuure ssue of hs journal, bu has no been fully eded Conen may change pror o fnal publcaon Caon nformaon: DOI 1119/TMC , I Transacons on Moble Compung me-varyng To capure he dynamcs of nework envronmen, n hs paper, we adop a novel sochasc game-heorec approach o analyze he users compuaon offloadng decson-makng process under dynamc condons, and hen propose a mul-agen sochasc learnng algorhm o reach he N of he sochasc game The man conrbuons of hs paper are summarzed as follows: We formulae a sochasc game o model and analyze he mul-user compuaon offloadng problem under dynamc envronmen, wheren boh moble users acveness and wreless channel gans are me-varyng To show he exsence of N n he formulaed sochasc game, we prove s equvalence o a weghed poenal game whch has a leas one N Moreover, we analyze he performance bound of he N n erms of he sysem cos and he number of moble users who can benef from ud compung To reach he N of he formulaed sochasc game, we propose a mul-agen sochasc learnng algorhm for he mul-user compuaon offloadng under dynamc envronmen The proposed algorhm runs n a fully dsrbued manner whou any nformaon exchange, e, each user ndependenly adjuss s offloadng sraegy based on s receved acon-reward nsead of knowng oher users dealed offloadng-sraeges As an mporan echncal conrbuon n hs paper, we heorecally derve he convergence rae of he mul-agen sochasc learnng algorhm I s echncally challengng o prove he convergence propery of he desgned learnng algorhms wh mul-user neracons under dynamc envronmen, and our sudy here s he frs one successfully addressng hs ssue The res of hs paper s organzed as follows In Secon II, we gve a bref revew of he relaed works In Secon III, our sysem model s nroduced In Secon IV, we propose a sochasc game o nvesgae he problem of dynamc compuaon offloadng In Secon V, he performance of he N of he game s analyzed In Secon VI, we propose a mul-agen sochasc learnng algorhm o fnd he N under dynamc envronmen Secon VII presens smulaon resuls and dscussons Conclusons are drawn n Secon VIII RLATD WORK Moble Cloud Fber Lnk Wreless Access Pon Moble Users In he leraure, many exsng works have suded he compuaon offloadng problem from he perspecve of a sngle moble user Rudenko e al [17] used expermenal resuls o show ha compuaon offloadng can save sgnfcan energy In [18], he auhors desgned an adapve meou scheme for compuaon offloadng o mprove he energy savngs on moble devces Wen e al [6] proposed an opmzaon scheme for energy-effcen applcaon execuon on he ud-asssed moble applcaon plaform Hueracanepa and Lee [19] proposed an adapve applcaon offloadng mechansm based on boh he curren sysem condons and he execuon hsory of applcaons By nvokng he Lyapunov opmzaon, [] and [1] suded he dynamc compuaon offloadng polces for mnmzng CPU and nework energy consumpon under real nework envronmen In [], he auhors modeled he unsable nework as an alernang renewal process and proposed an offloadng decson model for moble ud applcaon 3 SYSTM MODL Only a few works have dscussed he compuaon offloadng As shown n Fgure 1, we consder a group of moble users problem n he mul-user case Yang e al [3] proposed a genec N = {1,,, N}, where each user may have a compuaonally algorhm o solve he paron problem of wreless nework nensve ask T o be compleed There exss a wreless access c 18 I Personal use s permed, bu republcaon/redsrbuon requres I permsson See hp://wwweeeorg/publcaons_sandards/publcaons/rghs/ndexhml for more nformaon Inacve Acve Fg 1 An llusraon of he sysem model a group of moble users offload compuaon asks o a moble ud va a se of avalable wreless channels ach moble user could dynamcally change s acveness, and he wreless channels are me-varyng bandwdh among mulple users, whch acheves hgh hroughpu of processng he sreamng daa In [3], he auhors devsed a low-complexy heursc mehod o perform energy-effcen ask offloadng for muluser moble ud compung, whle sasfyng he delay requremens Sardell e al [4] proposed an erave algorhm o perform he jon opmzaon of rado and compuaonal resources for mul-cell moble-edge compung, under laency and power budge consrans The above sudes all belonged o he cenralzed compuaon offloadng mechansms whch dd no consder he neracons among mulple self-organzng users when hey ndependenly chose her compuaon offloadng sraeges [1], [11], [1], [13], [14], [15], [16] modeled moble users as self-neresed game players and proposed decenralzed mechansms o solve he mul-user compuaon offloadng problems These prevous sudes manly focused on he compuaon offloadng problems under relavely sac envronmen However, n real nework envronmen, due o dynamc moble users acveness and me-varyng wreless channels, he uly of each player s dynamcally varyng, and hus he equlbrum soluon of he sac game model may never be reached In hs paper, we sudy he mul-user compuaon offloadng problem wh consderaon of he dynamcs of users behavors and me-varyng channels, and adop he heory of sochasc game ha accouns for all he possble saes of he dynamc process o successfully solve he problem The ask of achevng N soluons of he sochasc game n he dsrbued and dynamc envronmen s challengng Mos exsng algorhms, such as he bes or beer response [4], fcous play [5], spaal adapve play [6], and no-regre learnng [7] requre enormous nformaon exchanges for users sraegy updang and requre he envronmen o be unchanged unl reachng convergence of he algorhms For he dsrbued and dynamc envronmen, some effcen algorhms have been proposed by nvokng he sochasc learnng auomaa SLA [8], [9], [3] Specfcally, he convergence of SLA-based algorhms o N has been esablshed for coordnaon games [8] and exac poenal games [9], [3] In hs paper, we show he formulaed sochasc game s equvalen o a weghed poenal game, and hen prove he convergence of he desgned mul-agen sochasc learnng algorhm o he N of he weghed poenal game Moreover, o he bes of our knowledge, our work s he frs o esablsh he convergence rae of he SLA-based algorhm n he dsrbued and dynamc envronmen

3 Ths arcle has been acceped for publcaon n a fuure ssue of hs journal, bu has no been fully eded Conen may change pror o fnal publcaon Caon nformaon: DOI 1119/TMC , I Transacons on Moble Compung 3 TABL 1 Summaon of Used Noaons Noaons Descrpon Noaons Descrpon N se of moble users u payoff funcon for sac game G,Λ M se of wreless channels u payoff funcon for sac game G 1,Λ s user s offloadng sraegy ū payoff funcon for sochasc game G θ user s acve probably A se of acve users p user s ransm power s A sraegy profle of all acve users g,o nsananeous channel gan s A\{} sraegy profle of all acve users excludng ḡ,o expeced channel gan s sraegy profle of all users R user s daa rae s sraegy profle of all users excludng I user s receved nerference w user s sraegy selecon probably vecor V ud compung cos b learnng sep-sze of he proposed algorhm V loc local compung cos r user s acon-reward a me pon AP hrough whch he moble users can offload her compuaon asks o he ud cener deployed by he elecom operaor Here, he wreless AP could be a WF access pon, 3G/4G macro-cell or small-cell base saon Suppose ha here are M avalable wreless channels denoed as M = {1,,, M} We use s {} M o denoe moble user s compuaon offloadng sraegy Specfcally, s > denoes ha user chooses o offload he compuaon ask o he moble ud va wreless channel s ; n oppose, s = denoes ha user decdes o compue s ask locally whou offloadng o moble ud Noce ha for each user, choosng dfferen channels wll lead o dfferen offloadng raes when sends compuaon ask o he AP, whch consequenly yelds dfferen coss 31 Dynamcs of Moble Users Acveness and Wreless Channels 3 Communcaon Model for Acve Moble Users To make a clear presenaon, we frs consder one realzaon of he sochasc sysem sae, whch s denoed by Λ Gven Λ, we defne he se of acve users as A = { N : a = 1} Suppose ha user chooses o offload s compuaon ask o he ud va wreless channel s > Gven he sraegy profle s A = [s ] A of all acve moble users, he uplnk daa rae of user A can be compued by R s A, Λ = Blog 1 + Communcaon sysems operae n a me-sloed fashon over me slos of equal duraon eg, several mcroseconds or mllseconds [3] A compuaon offloadng perod eg, several 33 Compuaon Model for Acve Moble Users seconds [1] usually consss of mulple me slos In hs paper, We consder ha each acve user A has a compuaon ask T we consder a general and praccal case ha moble users may ha needs o be execued eher locally on he moble devce or remoely on he elecom ud hrough compuaon offloadng The become acve or nacve dynamcally whn dfferen me slos Specfcally, a moble user s acve f has a compuaon ask o be compuaon ask can be expressed as T = C, D loc, D, where execued Oherwse, he moble user s nacve We use he on-off C s he sze of all npu compuaon daa eg, he moble sysem dsrbuon o model moble user s acveness, e, moble user sengs, he program codes, and he npu parameers nvolved n s acve or nacve wh probably θ or 1 θ he ask, D loc and D are he oal number of CPU cycles requred To model he me-varyng wreless channels, we assume ha o complee he compuaon ask on he moble devce and he he channels beween moble users and he AP follow Raylegh elecom ud, respecvely 3 For he sake of clear presenaon, fadng, whch s a realsc and wdely adoped moble channel we use he leers loc and o represen LOCal compung model [33], [34] Specfcally, he nsananeous channel power gan and CLOud compung, respecvely The dealed modelngs of he from user o he AP s gven by g,o = d,o α β,o, where d,o compuaon cos are as follows s he dsance from user o he AP for clear presenaon, we use o o denoe he AP, α s he pah loss exponen, and β,o s 331 he Raylegh fadng facor Noce ha, he nsananeous random pung Compuaon Cos When Choosng o Perform Cloud Com- coeffcen β,o vares from me slo o me slo If acve user chooses o offload s compuaon ask T o he We consder a more praccal model ha all sysem parameers ud, would ncur he cos for ransmng he npu daa o he e, channel power gans and he users acve probables are In hs paper, we focus on he wreless nerference model gven n 1 unknown For convenence of analyss, we defne a probably whch s wdely adoped n he leraure Noe ha, we can also adop some space as Ω, H, P, where Ω s he sample space over all sysem meda access conrol proocols such as CSMA n whch he mulple access saes, H s a mnmal σ-algebra on subses of Ω, and P s a among users for he shared specrum s carred ou over he packe level As shown n [11], he analyss could be very smlar o our adoped wreless probably measure on Ω, H Le Λ denoe an even n he sample space Ω Θ Λ = [a Λ, g Λ] : Ω N R N nerference model s a random vecor, 3 A moble user can oban he nformaon of C, D loc, and D by applyng he mehods eg, call graph analyss n [35], [36] If we consder he relaed where a = [a ] N, a {, 1} denoes he user s sae for nacve, and 1 for acve ha sasfes he on-off dsrbuon wh overhead, would be added jus as a consan n he sysem model, whch would no affec he followng mahemacal analyss Besdes, snce he moble probably θ, and g = [g,o ] N follows Raylegh fadng For devces and he remoe ud compung servers have dfferen nsrucon se beer readng, Table 1 summarzes he manly used noaons n archecures, he numbers of CPU cycles for he wo archecures e, D loc hs paper and D are dfferen c 18 I Personal use s permed, bu republcaon/redsrbuon requres I permsson See hp://wwweeeorg/publcaons_sandards/publcaons/rghs/ndexhml for more nformaon p g,o j A\{}:s j =s p j g j,o + σ, 1 where B s he channel bandwdh, p s he ransm power of user, g,o s he channel gan from user o he AP, and σ denoes he background nose power As shown n 1, f oo many acve users choose he same channel o offload her compuaon asks o he AP, hey may ncur severe nerference o each oher, resulng n low offloadng raes

4 Ths arcle has been acceped for publcaon n a fuure ssue of hs journal, bu has no been fully eded Conen may change pror o fnal publcaon Caon nformaon: DOI 1119/TMC , I Transacons on Moble Compung 4 ud va wreless access Accordng o he communcaon model 1, he ransmsson me and energy consumpon for offloadng he npu daa of sze C can be respecvely compued by T,1 s A, Λ = C, and s A, Λ = R s A, Λ p C R s A, Λ Afer he ransmsson, he ud expends he execuon me T, = D F o fnsh moble user s ask, where F denoes he compuaon capably e, CPU cycles per second assgned o user by he ud 4 Then, consderng boh he processng me 5 and he energy consumpon, he oal cos 6 when moble user chooses o perform ud compung e, s > can be gven by: V s A, Λ = µ T T,1 s A, Λ + T, + µ s A, Λ µ = p + µ T C + µ T T,, A, and s >, R s A, Λ where µ T and µ, 1 denoe he weghs of compuaonal me and energy for moble user s sraegy decson, respecvely Here, he uns of µ T and µ are 1 Second and 1 Joule, respecvely 33 Compuaon Cos When Choosng o Perform Local Compung ach acve moble user can also choose o execue s compuaon ask T locally by self e, whou nvokng any compuaon offloadng o he ud Le F loc be he compuaon capably of moble user The compuaon execuon me of ask T by = Dloc local compung s hen gven by T loc F loc compuaonal energy can be compued by loc The correspondng = η D loc, where η s he coeffcen denong he energy consumpon per CPU cycle Then, by akng no accoun boh he processng me and he energy consumpon, we can compue moble user s oal cos when chooses o perform local compuaon e, s = as follows: V loc = µ T T loc + µ loc, A, and s = 3 Based on he communcaon and compuaon models above, we see ha, when choosng o offload compuaon ask o he ud, each moble user s cos depends no only on s own offloadng sraegy, bu also on all he oher acve peers Specfcally, as shown n he cos funcon, f oo many moble users are acve and usng he same sraegy e, choosng o use he same wreless channel o offload her compuaon asks o he ud, hey may experence low offloadng raes, whch wll ncur more compuaon cos ncludng longer ransmsson me and hgher energy consumpon In hs case, would be more benefcal for he moble user o compue he ask locally by self Due o such an ner-dependence among dfferen moble users, game heory s a suable mahemacal ool o model and analyze users decson makng for compuaon offloadng However, due o dynamcally varyng of moble users acveness, each user may no be able o know oher peers acve/nacve saes Moreover, moble users prefer beer channel condon for offloadng her compuaon asks, bu he wreless channels are me-varyng, whch makes he problem more challengng 4 STOCHASTIC COMPUTATION OFFLOADING GAM In hs secon, we formulae a sochasc game o model he decson process for moble users compuaon offloadng o he ud For he sake of clear presenaon, we frs descrbe he game model n a sac case, and hen llusrae he dynamc case 41 Game Models 411 Sac Case Gven oher users sraegy decsons, each acve user A ndependenly adjuss s compuaon offloadng sraegy o mnmze s own compuaon cos Specfcally, gven a realzaon Λ n he probably space Ω, H, P, he sae-based payoff funcon of each acve user s naurally defned by { V loc, f s = ; u s, s A\{}, Λ = 4 F s consdered o be prese for each user In hs paper, we do no sudy he allocaon/schedulng of compuaon resources o dfferen users from he perspecve of moble ud, snce hs ssue has been wdely nvesgaed n Lemma 1 Gven a compuaon offloadng sraegy profle s A, ud leraure, eg, [4], [] compung s benefcal o an acve user A f s suffered nerference 5 Snce he AP s conneced o he moble ud va he hgh-speed fber I lnk, he ransmsson me cos among hem could be negleced, compared s A, Λ= j A\{}:s j =s p j g j,o on he seleced wreless channel s > wh he much hgher wreless access me cos resuled from he consraned sasfes ha I s A, Λ Q, wh he hreshold Q = p g,o wreless specrum resource Besdes, snce he sze of he compuaon oucome ψ 1 σ, µ s usually much smaller han he sze of npu compuaon daa for many where he parameer ψ = p +µ T C Bµ applcaons eg, face recognon, we neglec he me cos for he ud o T T loc+µ loc µt T, send he compuaon oucome back o he moble user The smlar assumpon Proof: Accordng o Defnon 1, ud compung s benefcal o user only f V also appears n many prevous works [11], [17], [18], [19], [1] 6 I would also be neresng o consder users economc cos Snce F s s A, Λ V loc Based on he compung consdered o be prese for each user n our sysem model, he economc cos s models n and 3, hs condon corresponds o jus a consan added no q, and hus wll no affec he followng gameheorec soluon However, f F s consdered as an opmzaon varable, µ p + µ T C + µ T T, µ T T loc + µ loc, how o opmally decde he prce for he compung resources would also be a R s A, Λ key problem for he servce provders In hs case, he economc game models such as he marke model, barganng model, bddng model, aucon model, whch afer some manpulaons leads o he followng condon: duopoly model, and Sackelberg model could be adoped Snce hs economc consderaon wll lead o sgnfcan change of he curren game model, we µ R s A, Λ p + µ T C consder hs ssue as an mporan fuure drecon o exend our work µ T c 18 I Personal use s permed, bu republcaon/redsrbuon requres I permsson See hp://wwweeeorg/publcaons_sandards/publcaons/rghs/ndexhml T loc + µ loc µ T T for, more nformaon V s, s A\{}, Λ, f s >, where s denoes acve user s sraegy, s A\{} denoes he sraegy profle of all he acve users excludng user, V and V loc are defned n and [ 3, respecvely Then, he game can be formulaed as G,Λ = A, Λ, {S } N, { u } ], where A s N he se of acve users, S denoes acve user s sraegy space, S = {, 1,, M} ach acve user auonomously chooses s sraegy s o mnmze s own payoff, e, G,Λ : mn u s, s A\{}, Λ, A 5 s S Defnon 1: Gven a compuaon offloadng sraegy profle s A = [s ] A, for an acve user A ha chooses he ud compung approach e, s >, f he ud compung does no yeld a hgher cos han he local compung e, V s A, Λ V loc, we say ha he ud compung s benefcal o user In parcular, from 1,, and 3, we observe ha wheher offloadng compuaon ask o ud s benefcal o moble user or no srongly depends on s suffered nerference, e, I s A, Λ = j A\{}:s j =s p j g j,o Referrng o he smlar proof n [11], we can acheve Lemma 1 as follows: 4

5 Ths arcle has been acceped for publcaon n a fuure ssue of hs journal, bu has no been fully eded Conen may change pror o fnal publcaon Caon nformaon: DOI 1119/TMC , I Transacons on Moble Compung 5 Theorem, Sac Gven 1, Sac All possble Dynamc Fg Graphcal represenaon of relaonshps among he hree proposed game models Based on he communcaon model 1, he above condon can be equvalenly ransformed no: p g,o p j g j,o j A\{}:s j=s µ p +µt C B µ T T loc +µ loc µt T, 1 σ Ths complees he proof of Lemma 1 Based on Lemma 1, we desgn he followng game model G 1,Λ whch wll be shown o be equvalen o G,Λ wh proofs n Lemma and Theorem as follows: G 1,Λ : mn s S u s, s A\{}, Λ, A, 6 where each user s payoff funcon s gven by: u s, s A\{}, Λ { Q, f s = ; = I s A, Λ, f s > 41 Dynamc Case We nex exend he sac game G 1,Λ under a gven Λ no a correspondng sochasc game ha experences all possble Λ Specfcally, n he dynamc and sochasc envronmen, we defne an expeced payoff funcon for each user N as follows: { Q, f s = ; ū s, s = Θ [u s, s, Θ]= Θ [I s, s, Θ], f s >, 8 where s = [s j ] j N \{} denoes he sraegy profle of all users excludng user, Q = p ḡ,o ψ 1 σ, and ḡ,o s he expeced channel gan from moble user o he AP Based on 8, we formulae a sochasc game denoed by G = [ N, Θ, {S } N, {ū } N ] ach moble user ndependenly adjuss s sraegy o mnmze s ndvdual expeced payoff funcon, whch can be expressed as: 7 G : mn s S ū s, s, N 9 Fgure llusraes he connecons among he game models G,Λ, G 1,Λ, and G Defnon 3 N of G 1,Λ : For a realzaon Λ Ω, a compuaon offloadng sraegy profle s A = [s ] A s a pure-sraegy N of he game G 1,Λ f and only f no acve moble user can mnmze s payoff funcon u by unlaerally devang, e, u s,s A\{},Λ u s,s A\{},Λ, A, s S 11 Defnon 4 xpeced N of G : A compuaon offloadng sraegy profle s = [s ] N s an expeced pure-sraegy N of he sochasc game G f and only f no moble user can mnmze s expeced payoff funcon ū by devang unlaerally, e, ū s, s ū s, s, N, s S 1 Theorem 1 For an arbrary realzaon Λ Ω, G 1,Λ s a weghed poenal game whch has a leas one N Proof: The key of he proof s o show ha for each user k A, he change of s payoff funcon due o s unlaeral change of sraegy s proporonal o he change n a carefully chosen poenal funcon for he whole sysem The deals are as follows We frs consruc a sae-based poenal funcon as follows: Φ s A, Λ = 1 j A\{k} A\{k} 4 Analyss of Nash qulbrum 15 In game heory, Nash qulbrum N s he mos mporan soluon concep for analyzng he oucome of he sraegc neracon Usng 14 and 15, we can derve he followng resul: of mulple decson-makers In hs subsecon, we frs nvesgae Φ s A, Λ = p k g k,o p j g j,o l {sj =s k }l {sk >} he exsence of N for he sac game models G,Λ and G 1,Λ j A\{k} Moreover, by provng he equvalence beween G,Λ and G 1,Λ, we + p k g k,o Q k l {sk =} + Ξ s A\{k}, Λ 16, llusrae he raonaly of desgnng he game model G 1,Λ Then, based on he analyss n he sac case, we derve he exsence where Ξ s A\{k}, Λ = 1 p g,o p j g j,o l {sj =s of N n he sochasc game G To proceed, we frs nroduce he }l {s >} A\{k} j A\{,k} defnon of N for game models n boh sac and dynamc cases + p g,o Q l {s=} s ndependen of user k s sraegy s k Defnon N of G,Λ : For a realzaon Λ Ω, a compuaon A\{k} offloadng sraegy profle s A = [s ] A s a pure-sraegy N of Besdes, based on 7, we have he followng equaon: he game G,Λ f and only f no acve moble user can mnmze s payoff funcon u by unlaerally devang, e, u k sk, s A\{k}, Λ =I k s A, Λ l {sk >} + Q k l {sk =} u s,s A\{},Λ u s,s A\{},Λ = p j g j,o l {sj =s k }l {sk >} + Q k l {sk =} 17, A, s S 1 j A\{k} c 18 I Personal use s permed, bu republcaon/redsrbuon requres I permsson See hp://wwweeeorg/publcaons_sandards/publcaons/rghs/ndexhml for more nformaon A j A\{} + A p g,o Q l {s =}, p g,o p j g j,o l {sj =s }l {s >} 13 where l {condon} s an ndcaor funcon, and s equal o resp, 1 when he condon s false resp, rue The above equaon 13 can be equvalenly wren as follows: Φ s A, Λ = 1 + j A\{k} A\{k} + p k g k,o p j g j,o l {sj=s k }l {sk >} p g,o p k g k,o l {sk =s }l {s>} A\{k} j A\{,k} + p k g k,o Q k l {sk =} + p g,o p j g j,o l {sj =s }l {s >} A\{k} In parcular, he followng resul always holds: p g,o Q l {s =} 14 p k g k,o p j g j,o l {sj =s k }l {sk >} = p g,o p k g k,o l {sk =s }l {s >}

6 Ths arcle has been acceped for publcaon n a fuure ssue of hs journal, bu has no been fully eded Conen may change pror o fnal publcaon Caon nformaon: DOI 1119/TMC , I Transacons on Moble Compung 6 Therefore, for each user k A and s wo dfferen sraeges s k and s k, we have he followng equaon: Φ s k, s A\{k}, Λ Φ s k, s A\{k}, Λ = p k g k,o p j g j,o l {sj=s k} l {s >}+p kg k,o Q k l k {s k =} j A\{k} p k g k,o p j g j,o l {sj =s k }l {sk >} p k g k,o Q k l {sk =} j A\{k} = p k g k,o u k s k, s A\{k}, Λ u k sk, s A\{k}, Λ 18 As saed before, 18 essenally means ha for each user k A, he change of s payoff funcon due o s unlaeral change of sraegy s proporonal o he change n he poenal funcon 13 for he whole sysem Thus, accordng o he poenal game heory n [4], G 1,Λ s a weghed poenal game wh weghfacor p k g k,o whch has a leas one N Ths concludes he proof Accordng o he proof n [11], G,Λ s an ordnal poenal game whch also has a leas one pure-sraegy N pon In he followng, we wll nvesgae he relaonshp beween G,Λ and G 1,Λ, and llusrae he raonaly of desgnng he game model G 1,Λ Lemma In G,Λ and G 1,Λ, all users sraegy preferences are he same Tha s, for an arbrary realzaon Λ Ω, A, for any s s, u s,s A\{},Λ u s,s A\{},Λ u s,s A\{},Λ u s,s A\{},Λ 19 Proof: The proof s essenally based on our prevous Lemma 1 Specfcally, we consder he followng hree cases: 1 s >, s = : Accordng o he defnon of payoff funcons, u s,s A\{},Λ = V s,s A\{},Λ, u s, s A\{}, Λ = V loc, and u s, s A\{}, Λ = I s, s A\{}, Λ, u s, s A\{}, Λ = Q Based on Lemma 1, V s,s A\{},Λ V loc I s,s A\{},Λ Q Therefore, u s,s A\{},Λ u s, s A\{}, Λ u s, s A\{}, Λ u s, s A\{}, Λ s =, s > : Accordng o he defnon of payoff funcons, u s,s A\{},Λ = V loc, u s,s A\{},Λ = V s A,Λ, and u s,s A\{},Λ = Q, u s,s A\{},Λ = I s A,Λ Based on Lemma 1, V s A,Λ V loc I s A, Λ Q Therefore, u s,s A\{},Λ u s,s A\{},Λ u s,s A\{},Λ u s,s A\{},Λ 3 s >, s > : Accordng o he defnon of payoff funcons, u s,s A\{},Λ = V s,s A\{},Λ, u s, s A\{}, Λ = V s A, Λ, and u s, s A\{}, Λ = I s, s A\{}, Λ, u s,s A\{},Λ = I s A,Λ Snce V µ p+µt C R V = + µ T T,, we can ge he followng resul: s, s A\{}, Λ V s A, Λ R s, s A\{}, Λ R s A, Λ As R s a decreasng funcon of he receved nerference I, R s,s A\{},Λ R s A, Λ I s,s A\{},Λ I s A, Λ Therefore, u s,s A\{},Λ u s,s A\{},Λ u s,s A\{},Λ u s,s A\{},Λ By summarzng he above hree cases, we can oban he resuls n Lemma A, s S, whch mples ha s Ψ 1 Followng he smlar argumen, we also have s Ψ 1 s Ψ Therefore, we can conclude ha Ψ and Ψ 1 are he same Lemma and Theorem ogeher mean ha G,Λ and G 1,Λ are essenally equvalen Thus, by dervng he N of G 1,Λ, we can also ge he N for G,Λ Moreover, G 1,Λ can be used for desgnng he sochasc game model G Based on he propery of G 1,Λ as shown n Theorem 1, we can oban he followng resul Theorem 3 The sochasc game G s a weghed poenal game wh he expeced poenal funcon gven by: Φs= Θ [Φs, Θ] = 1 θ θ j p ḡ,o p j ḡ j,o l {sj=s }l {s>}+ θ p ḡ,o Q l {s=}, N j N \{} N where s = [s ] N denoes he sraegy profle of all moble users, θ s he acve probably of moble user N Proof: By akng he operaon of expecaon for 18, we can oban he followng resul Φs k,s k Φ s k,s k =p k ḡ k,o ū k s k,s k ū k s k,s k, 1 where ū k s k, s k s he payoff funcon defned n 8 Thus, accordng o he poenal game heory n [4], G s a weghed poenal game wh wegh-facor p k ḡ k,o As proved n [4], every weghed poenal game possesses he fne mprovemen propery, and hus G has a leas one pure-sraegy N Tha s, he nvesgaed moble users decsonmakng process for compuaon offloadng s guaraneed o have a pure-sraegy N under he dynamc envronmen However, he behavors of users n he game are selfsh o mnmze s own payoff whou carng abou he oher peers, whch may lead o an neffcen N In he followng secon, we wll analyze he achevable performance of N for he sochasc game G under dynamc envronmen 5 PRFORMANC ANALYSIS OF NASH QUILIBRIUM To evaluae he performance of he N, we frs sudy he merc of sysem-wde compuaon cos, and hen analyze he number of moble users who benef from ud compung 51 Merc I: Sysem-Wde Compuaon Cos In dynamc and sochasc envronmen, canno characerze he cos of ud compung, snce he ransmsson rae R s s dynamcally varyng Therefore, we compue he expeced cos of offloadng compuaon ask o moble ud n dynamc envronmen as follows 7 : V s = µ p + µ T Θ [R s, Θ] Theorem ach N of game G,Λ s an N of game G 1,Λ, and each { N of game G 1,Λ s also an N of game G,Λ V loc, f s = ; Γ s, s = Proof: Denoe he se of N of G,Λ by Ψ, and he se of V 3 s, f s > N of G 1,Λ by Ψ 1 s Ψ, accordng o he defnon of N, u s, s A\{}, Λ u s, s A\{}, Λ 7 An alernave choce of he expeced cos can be V s =, A, s S Then, based on 19, we have u s, c 18 I Personal use s permed, s A\{}, Λ u s, s A\{}, Λ Θ [ µ p +µ T C ] + µ R s,θ T T, In hs paper, we choose o use for he, convenence of analyss bu republcaon/redsrbuon requres I permsson See hp://wwweeeorg/publcaons_sandards/publcaons/rghs/ndexhml for more nformaon C + µ T T, In comparson, each moble user s oal cos for performng local compung s sll gven by 3, snce he dynamc envronmen channel dynamcs, users dynamc acveness does no mpac he local compung In summary, we evaluae he compuaon cos n dynamc envronmen by he followng merc:

7 Ths arcle has been acceped for publcaon n a fuure ssue of hs journal, bu has no been fully eded Conen may change pror o fnal publcaon Caon nformaon: DOI 1119/TMC , I Transacons on Moble Compung 7 Defnon 5 Prce of Anarchy [38]: Prce of Anarchy PoA s he rao of sysem-wde compuaon coss beween he wors expeced N and he globally opmal soluon n cenralzed schemes, e, PoA = max s Ψ N θ Γ s, 4 N θ Γ ŝ where θ s moble user s acve probably, Ψ s he se of expeced N of he sochasc game G, and ŝ denoes he cenralzed opmal soluon ha mnmzes he sysem-wde compuaon cos, e, ŝ = arg mn N θ Γ s, where S denoes he sraegy s S space of all he users Noce ha he PoA provdes a meanngful merc ha ndcaes how good he N s compared o he cenralzed opmal soluon for mnmzng he sysem cos Moreover, accordng o Jensen s nequaly [41] and he upper bound of expeced nerference gven n 3, we have [ Θ Blog 1+ I s ], Θ Blog 1+ Θ [I s, Θ] Lemma 3 For an arbrary expeced N s of he sochasc game G, f s >, user s expeced ransmsson rae Θ [R s, Θ] for Lemma 3 characerzes he lower bound of expeced ransmsson rae of each user for compuaon offloadng a any N pon compuaon offloadng s lower bounded by nf [ R nf = g,o Blog 1+ p g,o j N \{} ] Blog σ 1+ p Accordng o 5, he lower bound R ncreases wh he number jḡ j,o θ j of avalable channels M The reason s ha as he number of Mσ channels ncreases, moble users can avod muual nerference by 5 choosng dfferen channels for compuaon offloadng Secondly, when moble users acve probables are lower, he lower bound Proof: Accordng o Defnon 4 for he expeced Nash qulbrum, N, ū s, s R ū s, s nf becomes larger, whch mples ha hgher offloadng rae can, whch along wh be acheved Wh Lemma 3, he followng resul can be acheved 8 leads o [ Θ I s, s, Θ ] [ Θ I s, s, Θ ] Theorem 4 For he mul-user sochasc compuaon offloadng game, s M S 6 G, he PoA of he sysem-wde compuaon cos sasfes ha { } By summng up he wo sdes of 6, we can derve N [ M Θ I s, s, Θ ] [ Θ I s, s s M, Θ ] θ max V loc, µ p+µt C + µ R T =1 nf T, 7 1 PoA { }, 35 N [ Obvously, Θ I s, s, Θ] = θ mn V j N \{}:s j =s p loc, µ p +µ T C + µ R T jḡ j,o θ sup T, j Thus, =1 Θ[ I s, s s M,Θ ] = sup where R p j ḡ j,o θ = Blog 1 + p ḡ,o σ j s M j N \{}:s j =s = p Proof: 1 Le s Ψ be an arbrary expeced N of he game jḡ j,o θ j l s M j N \{} {s j =s } = G If s loc =, user chooses local compung wh he cos V ; f p jḡ j,o θ j j N \{} s l M {s j =s } s >, user chooses ud compung wh he cos V s Thus, he followng resul always holds: 8 If s j =, s l M {s j =s } = ; f s j M, s l M {s j =s } = 1 { } Γ s max V loc, V s 36 Thus, [ Θ I s, s s M, Θ ] Besdes, accordng o Lemma 3, we have p jḡ j,o θ j, 9 j N \{} µ V s = p +µ T C µ whch along wh 7 yelds Θ [R s, Θ] + µt T, p +µ T C R nf + µ T T, [ Θ I s, s, Θ ] 1 37 M p jḡ j,o θ j 3 Therefore, j N \{} { µ Besdes, Γ s max V loc, p + µ T } C [ ] R Θ [R s p g,o, Θ] = Θ Blog 1 nf + µ T T, 38 + I s, Θ + σ [ For he cenralzed opmal soluon ŝ, he expeced ransmsson rae for compuaon offloadng s = Θ Blog 1+ I s,θ+p g,o ] Θ [Blog σ 1+ I s ],Θ [ ] σ [ = g,o Blog 1 + p ] g,o g,o [Blog σ 1 + p ] p g,o g Θ [R ŝ, Θ]= Θ Blog,o 1+ I ŝ, Θ + σ σ [ [ + Θ Blog 1+ I s,θ+p g,o ] Θ [Blog σ 1+ I s ],Θ g,o Blog 1+ p g,o ] Blog σ 1+ p 39 ḡ,o, σ σ 31 where he las nequaly n 39 s based on Jensen s nequaly Noably, [41] Then, we oban he followng resul: [ g,o Blog 1+ p ] g,o Θ [Blog σ 1+ I s ],Θ+p g,o µ V ŝ= p +µ T C µ + µ T T, p +µ T C σ Θ [R ŝ, Θ] R sup + µ T T, c 18 I Personal use s permed, bu republcaon/redsrbuon requres I permsson See hp://wwweeeorg/publcaons_sandards/publcaons/rghs/ndexhml for more nformaon σ Blog 1+ j N \{} p jḡ j,o θ j Mσ Then, 31, 3 and 33 lead o [ Θ [R s, Θ] g,o Blog 1 + p ] g,o Blog 1+ σ σ j N \{} p jḡ j,o θ j Mσ 33 34

8 Ths arcle has been acceped for publcaon n a fuure ssue of hs journal, bu has no been fully eded Conen may change pror o fnal publcaon Caon nformaon: DOI 1119/TMC , I Transacons on Moble Compung 8 Snce user s compuaon cos s eher V loc have { { } Γ ŝ mn V loc µ, V ŝ mn V loc, p +µ T C R sup or V ŝ, we +µ T T, 41 whch along wh 38 leads o he upper bound of he PoA gven n 35 Besdes, snce he cenralzed opmum ŝ mnmzes he sysemwde compuaon cos, we hence have PoA 1 Therefore, we complee he proof of Theorem 4 Accordng o Theorem 4 and Lemma 3, more avalable channels and lower acve probables help ncrease he lower bound of expeced offloadng rae, whch hus decreases he gap beween he wors N and he cenralzed opmal soluon 5 Merc II: Benefcal Cloud Compung Users For an arbrary N s, le us denoe he number of users who benef from ud compung by N c s Nex, we wll analyze he bound of N c s Leng Z max max N {p ḡ,o }, Z mn mn N {p ḡ,o }, Q max max N { Q }, Qmn mn N { Q }, θ max max N {θ }, and θ mn mn N {θ }, we can derve he followng heorem Theorem 5 Suppose < N c s < N, hen for he mul-user dynamc compuaon offloadng game, he oal number of users who benef from ud compung a any N pon sasfes: M Q mn Z max θ max N c s M Qmax Z mn θ mn + 1 } 4 Proof: 1 Snce N c s < N, here exss a leas one user k ha chooses he local compung manner, e, s k = Snce s s an N, we know ha he user canno reduce s payoff by choosng compuaon offloadng va any channel m M Accordng o 8, we have Θ [I k s, Θ] = j N \{k} p jḡ j,o θ j l {s k =m} Q k, m M Then, le Nm c s = N =1 l {s =m} denoe he number of users on channel m, and we have N c m s Z max θ max j N \{k} p jḡ j,o θ j l {s k =m} Q k Q mn Thus, N c m s Q mn Z maxθ max, m M Then, we can oban, 43 N c s = M N c m=1 m s M Q mn 44 Z max θ max Snce N c s >, here exss a leas one user k ha chooses he ud compung manner, e, s k > Whou loss of generaly, suppose user k s on he channel m, whch s occuped by mos users, e, Nm c s N c m s, m M Snce s s an N, we know ha he user canno reduce s payoff by choosng local compuaon Accordng o 8, we have Θ [I k s, Θ] Q k Tha s, j N \{ k} p jg j,o θ j l {s j =m} Q k Then, he followng Remark 1 The above analyss ndcaes ha he N pons mgh enable moble users o acheve desrable and aracve performance by playng he proposed sochasc game for compuaon offloadng I s very neresng snce moble users selfsh and compeve behavors lead o desrable game oucomes The reasons can be explaned as follows If oo many moble users are usng he same wreless channel o offload her compuaon asks o he ud, hey may experence low offloadng raes In order o reduce he compuaon cos, some moble users wll defnely choose oher wreless channels for offloadng or compue he ask locally by self Consequenly, leads o balanced occupaon of wreless channels for compuaon offloadng, whch s benefcal for he whole sysem 6 MULTI-AGNT STOCHASTIC LARNING UNDR DY- NAMIC NVIRONMNT Alhough he N pons exhb desrable and aracve performance, s challengng for moble users o reach he N n a dsrbued manner and under dynamc envronmen Mos exsng game-heorec algorhms eg, bes or beer response [4], spaal adapve play [6] updae users sraeges based on her receved nsananeous uly/payoff However, due o he dynamcs of users acveness and wreless channels n our compuaon offloadng problem, each user mgh receve dfferen ules/payoffs n dfferen me slos, even f chooses o use he same sraegy Thus, he exsng game-heorec algorhms may never reach N Ths movaes us o ncorporae he dea of sochasc learnng [8], [9], [3] no he desgn of an effcen ye dsrbued algorhm under dynamc envronmen n order o reach he N of our proposed sochasc game G 61 Proposed Mul-Agen Sochasc Learnng Algorhm The deals are shown n he Mul-Agen Sochasc Learnng Algorhm e, referred as MASL-Algorhm Specfcally, each moble user acs as a learnng auomaon ha ndependenly and auomacally selecs s offloadng sraegy accordng o a probably vecor over he sraegy space, and updaes he probably vecor based on he acon-reward receved from he dynamc envronmen For he sake of clear presenaon, we denoe he sraegy selecon probably vecor for an arbrary user as w = w, w 1,, w M, where w denoes he probably o selec he sraegy of local compung, w m m M denoes he probably o selec offloadng he compuaon ask o he moble ud va wreless channel m MASL-Algorhm: To reach he N of he sochasc game G Inalzaon: A he nal me =, each moble user N ses s sraegy selecon probably vecor as a unform dsrbuon, e, w = 1 M+1,, 1 M+1 Loop for =, 1,, resul always holds: N c m s 1 Z mn θ mn 1 Updang compuaon offloadng sraegy: In he -h me j N \{ k} p jg j,o θ j l {s j =m} Q k Q max, slo, each acve user A, selecs an offloadng sraegy s accordng o s curren sraegy selecon probably vecor 45 whch leads o Nm c s Q max w Z mn θ mn +1 Then, we have The nacve users N \A keep slen and ake no acon Measurng nsananeous payoff 8 : ach acve user evaluaes N c s = M N c ms M Qmax s respecve payoff u N c m=1 m=1 ms accordng o 7, namely, acve user M +1 evaluaes s receved nerference I Z mn θ f s > ; oherwse, mn 46 acve user drecly compues Q whch s gven n Lemma Therefore, Theorem 5 s proved 8 As we dscussed before, he receved payoff u Theorem 5 provdes a quanave characerzaon abou how depends no only on oher users acveness, bu also on he curren channel condon Specfcally, oher many users can evenually benef from offloadng compuaon users acveness mpacs s receved nerference I, whle he curren channel asks o he moble ud, by playng he sochasc game G condon mpacs boh I c 18 I Personal use s permed, bu republcaon/redsrbuon requres I permsson See hp://wwweeeorg/publcaons_sandards/publcaons/rghs/ndexhml and Q for more nformaon

9 Ths arcle has been acceped for publcaon n a fuure ssue of hs journal, bu has no been fully eded Conen may change pror o fnal publcaon Caon nformaon: DOI 1119/TMC , I Transacons on Moble Compung 9 1 Besdes, he nacve users N \A keep slen and ake no acon 3 Updang sraegy selecon probably: ach acve user updaes s sraegy selecon probably vecor for he nex me slo accordng o he followng rule: w +1 = w + br e s w, 47 where < b < 1 s he learnng sep-sze, e s s an M + 1- dmensonal un vecor wh he s -h elemen beng one, and r s he receved acon-reward defned by r = 1 γ u The compuaon offloadng sraegy wh less cos s gven larger acon-reward Here γ s a scalng facor, and we requre γ 1 {u } o guaranee he acon-reward r posve Besdes, max he nacve moble users N \A keep her sraegy selecon probably vecors unchanged, e, w +1 = w 48 nd loop unl all users do no adjus her respecve offloadng sraeges As shown above, he proposed MASL-Algorhm s operaed n an erave manner Whn each round of eraon, each acve user ndependenly selecs s offloadng sraegy based on a probably vecor over he sraegy space, and receves an acon-reward from he dynamc envronmen The compuaon offloadng sraegy wh less cos s gven larger acon-reward, and he sraegy wh larger reward value wll be assgned wh larger probably By connuously neracng wh he random envronmen, each moble user wll fnally choose s opmal offloadng sraegy wh probably one We emphasze ha durng he operaon of MASL- Algorhm, each moble user operaes enrely based on s own sraeges and he consequenly receved reward, whou requrng any knowledge from oher users and any pror knowledge of probably space Ω, H, P of he dynamc envronmen Therefore, he proposed MASL-Algorhm s fully dsrbued whch makes self aracve for a praccal mplemenaon Noce ha due o he dynamcs of users acveness and wreless channels, user mgh receve dfferen acon-rewards n dfferen me slos, even f chooses o use he same sraegy Ths mposes he key challenge o esablsh he convergence of MASL-Algorhm Alhough here exs several prevous sudes [8], [9], [3] nvesgang he convergence of some sochasc learnng algorhms, our proposed MASL-Algorhm dffers from hose algorhms n erms of akng no accoun he dynamcs of boh users acveness and wreless channels Moreover, he defnon of payoff funcon s applcaon-dependen, and dfferen payoff funcons adoped by even he same learnng mechansm wll lead o dfferen learnng soluons [31] Therefore, he prevous analyses are no applcable o our case n hs sudy, whch movaes us o perform a deep analyss abou he convergence propery of he MASL-Algorhm n he nex subsecon 6 Convergence Properes of MASL-Algorhm We frs re-wre he updang rule n Sep 3 of he proposed algorhm as follows: = w + ba r e s w, N, 49 w +1 where a = a 1,, a N, s = s 1,, s N, r = r 1,, r N, and f represens he updang rule specfed by 49 Then, accordng o Theorem 31 n [8], we can derve he followng lemma Lemma 4 Wh a suffcenly small sep-sze b, e, b, he sequence {W } wll converge weakly o he soluon of he followng ordnary dfferenal equaon OD: where a denoes he acveness of user n he -h me slo Le W = w1,, w T X m,w X m,w N denoe he sraegy selecon probably vecor of all he users, and hus we can express he evoluon of he sraegy selecon probably vecor of he game G as follows: = θ w,m w,m Y m, W X m, W X m, W W +1 = W + bf W, a, s, r,m,m, c 18 I Personal use s permed, bu republcaon/redsrbuon requres I permsson See hp://wwweeeorg/publcaons_sandards/publcaons/rghs/ndexhml for more nformaon dw d wh he nal sae W = Θ [ f W, a, s, r, Θ W = W ] = hw, 51 [ ] 1 M+1, and h W N M+1 = Lemma 5 Wh a suffcenly small sep-sze b, our proposed MASL- Algorhm converges o a sable saonary pon of he OD gven n 51 Proof: Le r s = Θ [r s, Θ] denoe user s expeced reward funcon under sraegy profle s, and le X m, W denoe user s probablsc reward funcon when adops pure sraegy m and oher users adop probably vecor for sraegy selecon W = w 1, w 1, w +1,, w N Specfcally, defne X m, W = r m, W = r m, s w j,sj, 5 s S ī where S denoes he sraegy space of all he users excludng, w j,sj s he probably of user j o choose pure sraegy s j In addon, defne he probablsc poenal funcon Y W = ΦW = Φ s w,s, 53 s S N and Y W Y m, W = = Φ m, s w j,sj, 54 w,m s S ī where Φ s he expeced poenal funcon defned n, S denoes he sraegy space of all he users Accordng o Lemma 4, we have dw,m =θ w,m 1 w,m r m,w + w,m w,m r m,w d m m = θ w,m r m, W w,m r m, W m S = θ w,m X m, W w,m X m, W m S = θ w,m X m, W X m, W Then, m S w,m dy W = Y W dw,m d w,m,m d = Y m,w θ w,m w,m,m m S j j 55

10 Ths arcle has been acceped for publcaon n a fuure ssue of hs journal, bu has no been fully eded Conen may change pror o fnal publcaon Caon nformaon: DOI 1119/TMC , I Transacons on Moble Compung 1 Noably,,m,m θ w,m w,m Y m, W X m, W X m, W =,m,m θ w,m w,m Y m,w X m,w X m,w Therefore, we can oban he followng resul: dy W d = 1 θ w,m w,m Y m,w Y m,w,m,m X m,w X m,w As gven n Sep 3 of he proposed algorhm,, r = 1 γu, hus he expeced reward r = 1 γū Then, 1 yelds Φ s, s Φ s, s = p ḡ,o γ r s, s r s, s 59 By usng qs 5, 54, and 59, we can derve ha m, m S, Y m,w Y m,w = p ḡ,o γ X m,w X m,w, whch, ogeher wh 58, yelds 6 dyw = 1 p ḡ,o θ w,mw,m Xm,W X m,w d γ,m,m ndcaes ha Y W monoonously decreases when he algorhm eraes Moreover, snce Y W s lower bounded by Y W, we know Y W wll converge o a saonary pon when dy W d dy W d =, and = w,m w,m X m, W X m, W = 6 Then, accordng o 55, we have dw,m d =,, m, and hus dw d = Hence, W converges o a saonary pon of OD 51 Ths complees he proof of Lemma 5 I has been proved by Theorem 3 n [8] ha all pure-sraegy N of G concde wh he sable saonary pons of he OD gven n 51 Thus, based on Lemma 5, we can derve he followng heorem Theorem 6 Wh a suffcenly small sep-sze b, our proposed MASL- Algorhm converges o a pure-sraegy N pon of G Moreover, he convergence rae of he MASL-Algorhm can be characerzed as follows: Proof: As shown n 61, he probablsc poenal funcon Y W = ΦW monoonously converges Specfcally, a eraon ndex, he correspondng convergence rae s gven by: ρ = Φ W +1 Φ W Φ W Φ W = 1 + Y W +1 Y W Y W Y W, 66 where W s a saonary pon of he OD, e, an N As saed n [39], ρ ndcaes how se Φ W +1 s o Φ W, compared wh Φ W Y W +1 Y W Y W w W w,m,m,m = Y m, W θ bw,m w,m X m, W X m, W,m m S = b θ w,mw,m Y m, W X m, W X m, W,m,m = b θ w,mw p ḡ,o,m X m, W X m, W γ,m,m 67 Besdes, [8], [4] show ha only he pure-sraegy N s sable, hus we only sudy he convergence o a pure-sraegy N s Obvously, Y W can be equvalenly wren as Φ s Therefore, ρ =1 b θ w,m w p ḡ,o,m γ,m,m X m, W X m, W Φ s w,s Φ s s S whch along wh 5 yelds ρ =1 = 1 b p θ ḡ,o,m,m s S b,m,m s S N r γ m, s r m, s w,mw,m Φ s w,s Φ s s S N θ γ p ḡ,o ū m, s ū m, s s S Φ s w,s Φ s N, 68 wj,s j j w,s w,s N:s =m N:s =m 69 A =, w,s = 1 M+1, N, s S, hus, we can derve ρ as 64 A =, le he probably n N w,s = 1 ε, he N probably for only one user devang N wj,s j = ε w,s N \{j} NM Theorem 7 The average convergence rae of he proposed MASL-, s j s j, and he probably for more han one user devang N o be As ε, we oban ρ as 65 Then, accordng o he Algorhm s gven by: ρ ave = defnon n [39], he average rae of convergence can be compued ρ ρ, 63 by ρ ave = ρ ρ Noably, he me aken for Φ W Φ W wh o decrease L mes s value s T ave = log L b θ γ p ḡ,o ū m, s ū m, s log ρ ave Remark The mpac of sep-sze b I s noed ha he convergence ρ N m S m =1 S s S M + 1 Φ N s Φ s, rae s hgher when ρ ave s smaller [39] Noably, ρ ave decreases lnearly wh he value of b Thus, when b s larger, he convergence ges faser s S 64 However, he approxmaon of he erave process o he OD requres and a suffcenly small b, as shown n Lemma 4 [4] has characerzed he b θ γ p ḡ,o ū s, s ū s accuracy of approxmaon as [ ρ N s = 1 S Φs, s Φ s, 65 W W b] = O b, 7 N s S where W s he erave process of algorhm, and W b represens he where S denoes he sraegy space of all he users excludng, S value of he rajecory of OD a me b Therefore, here s a radeoff denoes he sraegy space of all he users, and s s a pure-sraegy N beween he convergence rae and he accuracy of approxmaon o N c 18 I Personal use s permed, bu republcaon/redsrbuon requres I permsson See hp://wwweeeorg/publcaons_sandards/publcaons/rghs/ndexhml for more nformaon

11 Ths arcle has been acceped for publcaon n a fuure ssue of hs journal, bu has no been fully eded Conen may change pror o fnal publcaon Caon nformaon: DOI 1119/TMC , I Transacons on Moble Compung 11 TABL Parameers Seng Coverage of AP 5 m Daa sze C 5 KB Number of moble users N [, 45] Number of CPU cycles for local compung D loc 1 Megacycles User s acve probably θ, 1] Number of CPU cycles for ud compung D 1 Megacycles Number of channels M [4, 14] Local compuaonal capably F loc Randomly se from {5, 8, 1} GHz Bandwdh of channel B 5 MHz Cloud compuaonal capably F 1 GHz Transm power p 1 mw Wegh of compuaonal energy µ Randomly se from {, 5, 1} Pass loss exponen α 4 Wegh of compuaonal me µ T 1 µ Background nose σ -1 dbm Compung energy effcency 1/η Randomly se from {4, 5, 6} Megacycles/J Sraegy selecon probables Offloadng va channel 1 Offloadng va channel Offloadng va channel 3 Offloadng va channel 4 Offloadng va channel 5 Local compung Sysem wde compuaon cos MASL Algorhm BR Algorhm Ieraon ndex Ieraon ndex Fg 3 Convergence of sraegy selecon probables Fg 4 Convergence of sysem-wde compuaon cos of he sochasc game, and he parameer seng of b s applcaondependen n pracce Fgure 5 n he nex secon verfes he above dscussons 7 SIMULATION RSULTS In hs secon, we conduc smulaons o valdae he proposed MASL-Algorhm and s performance under dynamc envronmen We se up a scenaro nework where a group of moble users are randomly scaered n he coverage of he AP For each user, we randomly se s acve probably θ accordng o a unform dsrbuon whn,1] The me-varyng channels follow Raylegh fadng, where he random fadng coeffcen β s exponenally dsrbued wh un-mean Table summarzes he key parameers used n smulaons, whch are se smlar o [1], [11] If no oherwse specfed, he defaul seng for he number of users s 3, and ha for he number of channels s 5 Moreover, n he proposed MASL-Algorhm, he defaul value of he sepsze b s 1, and ha of he scalng facor γ s 1 5 he mpac wll be dscussed laer on 71 Convergence Analyss To evaluae convergence of he proposed MASL-Algorhm, we 9 Snce he ypcal lengh of a me slo n wreless sysems s a he me plo he sraegy selecon probables of one arbrarly seleced scale of mcroseconds eg, 7 mcroseconds for a me slo n LT sysem user n Fgure 3 A he very begnnng, hs arge user randomly [3], he me used by he compuaon offloadng decson process s acually selecs s compuaon offloadng sraegy accordng o a unform very shor eg, 1 mllseconds for 3 eraons n LT sysem Such a shor duraon s neglgble, compared wh he compuaon execuon process, dsrbuon As he MASL-Algorhm operaes, hs arge user s whch s ypcally a he scale of seconds eg, he execuon me for moble sraegy selecon probables keep on updang and fnally con- gamng applcaons s ypcally several hundred mllseconds [37] c 18 I Personal use s permed, bu republcaon/redsrbuon requres I permsson See hp://wwweeeorg/publcaons_sandards/publcaons/rghs/ndexhml for more nformaon verge afer around 3 eraons 9 Specfcally, afer convergence, he probably of choosng offloadng compuaon ask va channel 1 s equal o 1, whle he probables for oher sraeges decrease o Ths resul means ha, afer convergence, he arge user wll only choose channel 1 o offload compuaon ask o he moble ud Fgure 4 plos he dynamcs of sysem-wde compuaon cos for our MASL-Algorhm For comparson, he bes-response algorhm referred as BR-Algorhm n [11] s ploed, whch also runs n an erave manner A each eraon, he BR-Algorhm allows he user who receved he updae-permsson message o selec s opmal sraegy based on s nsananeous compuaon cos, whle oher users keep her sraeges unchanged We can see ha he MASL-Algorhm can grealy reduce he compuaon cos o s mnmum e, N afer convergence, whle he BS-Algorhm yelds a flucuang compuaonal cos whch s much greaer han ha of he MASL-Algorhm Ths s because he BR-Algorhm always conducs a myopc play based on he nsananeous compuaon cos, whle he envronmen s dynamcally varyng We hen analyze he compuaonal complexy of he proposed algorhm Snce mos operaons only nvolve some basc arhmecal calculaons, he domnang par s he updang of he sraegy selecon probably n Sep 3, whch nvolves he operaons of vecor-vecor sums, 1 scalar-vecor produc, and 1 scalar-scalar produc Thus, he proposed MASL-Algorhm runs

12 Ths arcle has been acceped for publcaon n a fuure ssue of hs journal, bu has no been fully eded Conen may change pror o fnal publcaon Caon nformaon: DOI 1119/TMC , I Transacons on Moble Compung Sysem wde compuaon cos b=4 b=3 b= b=5 Sysem wde compuaon cos γ =1 3 γ =1 4 γ =1 6 5 b=1 5 γ = Ieraon ndex Ieraon ndex Fg 5 Impac of dfferen values of sep-sze b Fg 6 Impac of dfferen values of scalng facor γ 9 45 Sysem wde compuaon cos MASL Algorhm BR Algorhm RSS Algorhm Sysem wde compuaon cos MASL Algorhm BR Algorhm RSS Algorhm Number of moble users Number of channels Fg 7 Performance comparson n erms of sysem cos for dfferen numbers of moble users Fg 8 Performance comparson n erms of sysem cos for dfferen numbers of channels n a low complexy of O3M + 1 In conras, [11] has shown ha resul, he performance of he algorhm ges worse when γ = 1 6 he compuaonal complexy of he BR-Algorhm s OM log M Fgure 5 shows he mpac of he sep-sze on he convergence speed of he proposed MASL-Algorhm We vary he sep-sze b 7 Performance valuaon o be 5, 1,, 3, and 4, respecvely Fgure 5 shows ha as We furher evaluae he performance of he proposed MASLhe sep-sze ncreases, he algorhm can speed up s convergence, Algorhm, n comparson wh he performance of he BRbu obans an nferor soluon no N As shown n Remark Algorhm [11] and he random sraegy selecon algorhm referred Secon VI, here s a radeoff beween he convergence speed as RSS-Algorhm Specfcally, n he RSS-Algorhm, each and he accuracy of convergence o N, whch are boh mpaced moble user randomly selecs a sraegy n each me slo The by he sep-sze b For our case, s preferable o se he sep-sze as followng presened resuls are obaned by smulang 5 ndependen 1, whch can lead he algorhm converge o an N whn abou rals and hen akng he average value 35 eraons I s noed seng he sep-sze as 5 can also ge Fgure 7 shows he performance of our proposed MASLhe N, bu he convergence speed s very slow more han 5 Algorhm versus dfferen numbers of moble users Fgure 7 eraons shows ha he MASL-Algorhm always consumes a sgnfcanly Moreover, Fgure 6 shows he mpac of he scalng facor γ on less oal cos han he BR-Algorhm and he RSS-Algorhm he performance of he proposed MASL-Algorhm We vary he In addon, he consumed oal cos ncreases as he number of parameer γ o be 1 3, 1 4, 1 5, and 1 6, respecvely Fgure 6 moble users ncreases, whch s conssen wh he nuon shows ha when γ = 1 5, he algorhm acheves he bes performance Fgure 8 furher shows he advanage of usng MASL-Algorhm n erms of reducng he sysem-wde compuaon cos by varyng dfferen numbers of channels Fgure 8 agan shows Larger γ seng could enhance users response o he compuaon ha our proposed MASL-Algorhm always ouperforms he BRcos, and hus lead users sensve sraegy adjusmen owards Algorhm and he RSS-Algorhm he opmal one Tha s why we observe from he fgure ha 1 5 s Fgures 9 and 1 evaluae our proposed MASL-Algorhm n beer han 1 3 and 1 4 for he seng of γ However, γ canno be erms of he number of moble users who benef from performng oo large n order o guaranee he acon-reward r posve As a ud compung noce ha he decmals n he resuls are due c 18 I Personal use s permed, bu republcaon/redsrbuon requres I permsson See hp://wwweeeorg/publcaons_sandards/publcaons/rghs/ndexhml for more nformaon

13 Ths arcle has been acceped for publcaon n a fuure ssue of hs journal, bu has no been fully eded Conen may change pror o fnal publcaon Caon nformaon: DOI 1119/TMC , I Transacons on Moble Compung Number of benefcal ud compung users MASL Algorhm BR Algorhm RSS Algorhm Number of benefcal ud compung users MASL Algorhm BR Algorhm RSS Algorhm Number of moble users Number of channels Fg 9 Performance comparson n erms of benefcal ud compung users for dfferen numbers of users Fg 1 Performance comparson n erms of benefcal ud compung users for dfferen numbers of channels o he average resuls Boh fgures demonsrae ha our proposed MASL-Algorhm acheves beer performances han he BR- Algorhm and he RSS-Algorhm Fgure 9 shows ha he number of users who benef from ud compung ncreases when he number of users becomes larger Inuvely, when he number of oal users ncreases, more users may possbly choose ud compung However, due o lmaon of avalable channels, he number of benefcal ud compung users s also lmed, snce users would generae severe nerference o each oher, leadng o lower offloadng raes Fgure 1 shows ha he number of benefcal ud compung users ncreases when he number of avalable channels ncreases 8 CONCLUSION In hs paper, we have nvesgaed he problem of mul-user compuaon offloadng for moble ud compung under he praccal dynamc envronmen By formulang hs problem as a sochasc game, we proved ha such a dynamc offloadng decson process always leads o a pure-sraegy N Moreover, we have analyzed he performance bounds of he N n erms of boh he sysem cos and he number of users who can benef from ud compung To reach he N, we proposed a fully dsrbued algorhm e, MASL-Algorhm wh a guaraneed convergence rae under dynamc envronmen Smulaon resuls have been presened o valdae he effecveness of our proposed algorhm and show s sgnfcan performance advanage In our fuure work, we wll sudy he jon opmzaon of dynamc offloadng decson-makng and ransm power conrol, whch wll be an mporan and echncal challengng problem Anoher neresng drecon s o nvesgae he moble compuaon offloadng from he perspecve of economcs, and specfcally, o consder moble users economc expenses for offloadng compuaon asks o he moble ud RFRNCS [4] S Sardell, G Scuar, and S Barbarossa, Jon opmzaon of rado and compuaonal resources for mulcell moble-edge compung, I Trans Sgnal and Informaon Process over New, vol 1, no, pp 89-13, June 15 [5] Y Ca, F R Yu, and S Bu, Cloud compung mees moble wreless communcaons n nex generaon cellular neworks, I New, vol 8, no 6, pp 54-59, 14 [6] Y Wen, W Zhang, and H Luo, nergy-opmal moble applcaon execuon: Tamng resource-poor moble devces wh ud nes, n Proc I INFOCOM, 1, pp [7] M V Barbera, S Kosa, A Me, and J Sefa, To offload or no o offload? he bandwdh and energy coss of moble ud compung, n Proc I INFOCOM, 13, pp [8] J Zheng, Y Wu, N Zhang, H Zhou, Y Ca, and X Shen, Opmal power conrol n ulra-dense small cell neworks: A game-heorec approach, I Trans Wreless Commun, vol 16, no 7, pp , Jul 17 [9] D Nyao, Hossan, and Z Han, Dynamcs of mulple-seller and mulple-buyer specrum radng n cognve rado neworks: A gameheorec modelng approach, I Trans Moble Compung, vol 8, no 8, pp 19-1, Aug 9 [1] X Chen, Decenralzed compuaon offloadng game for moble ud compung, I Trans Parallel Dsrb Sys, vol 6, no 4, pp , Apr 15 [11] X Chen, L Jao, W L, and X Fu, ffcen mul-user compuaon offloadng for moble-edge ud compung, I/ACM Trans New, vol 4, no 5, pp , Oc 16 [1] T Ln, T Alpcan, K Hnon, A game-heorec analyss of energy effcency and performance for ud compung n communcaon neworks, I Sys J, vol 11, no, pp , Jun 17 [13] V Cardelln, V D N Personé, V D Valero, F Facchne, V Grass, F L Pres, and V Pccall, A game-heorec approach o compuaon offloadng n moble ud compung, Mah Program, vol 157, no, pp , Jun 16 [14] M Chen, M Dong, and B Lang, Mul-user moble ud offloadng game wh compung access pon, n Proc I Cloudne, 16, pp [15] Y Wang, S Meng, Y Chen, R Sun, X Wang, and K Sun, Mul-leader mul-follower Sackelberg game based dynamc resource allocaon for moble ud compung envronmen, Wreless Pers Commun, vol 93, no, pp , Mar 17 [16] Y Lu, C Xu, Y Zhan, Z Lu, J Guan, and H Zhang, Incenve mechansm for compuaon offloadng usng edge compung: A Sackelberg game approach, Compu New, vol 19, no, pp , Dec 17 [17] A Rudenko, P Reher, G J Popek, and G H Kuennng, Savng porable compuer baery power hrough remoe process execuon, Moble Compu Commun Rev, vol, no 1, pp 19-6, 1998 [18] C Xan, Y Lu, and Z L, Adapve compuaon offloadng for energy conservaon on baery-powered sysems, n Proc I ICDCS, 7, pp 1-8 [19] G Hueracanepa and D Lee, An adapable applcaon offloadng scheme based on applcaon behavor, n Proc nd In Conf Adv Inf New Appl- Workshops, 8, pp [] J Kwak, Y Km, J Lee, and S Chong, DRAM: Dynamc resource and ask allocaon for energy mnmzaon n moble ud sysems, I J [1] J Zheng, Y Ca, Y Wu, and X Shen, Sochasc compuaon offloadng game for moble ud compung, n Proc I/CIC In Conf on Commun n Chna ICCC, 16, pp 1-6 [] Y Xu and S Mao, A survey of moble ud compung for rch meda applcaons, I Wreless Commun, vol, no 3, pp 46-53, June 13 [3] Y Zhao, S Zhou, T Zhao, and Z Nu, nergy-effcen ask offloadng for muluser moble ud compung, n Proc I/CIC In Conf on Commun n Chna ICCC, 15, pp 1-5 Sel Areas Commun, vol 33 no 1, pp 51-53, Dec c 18 I Personal use s permed, bu republcaon/redsrbuon requres I permsson See hp://wwweeeorg/publcaons_sandards/publcaons/rghs/ndexhml for more nformaon

14 Ths arcle has been acceped for publcaon n a fuure ssue of hs journal, bu has no been fully eded Conen may change pror o fnal publcaon Caon nformaon: DOI 1119/TMC , I Transacons on Moble Compung 14 [1] D Huang, P Wang, and D Nyao, A dynamc offloadng algorhm for moble compung, I Trans Wreless Commun, vol 11, no 6, pp , Jun 1 [] H Wu, D Huang, and S Bouzefrane, Makng offloadng decsons ressan o nework unavalably for moble ud collaboraon, n Proc I Collaboraecom, 13, pp [3] L Yang, J Cao, S Tang, T L, and A T S Chan, A framework for paronng and execuon of daa sream applcaons n moble ud compung, n Proc I In Conf Cloud Compung, 1, pp [4] D Monderer and L S Shapley, Poenal games, Games and conomc Behavor, vol 14, pp , 1996 [5] J Marden, G Arslan, and J Shamma, Jon sraegy fcous play wh nera for poenal games, I Trans Auoma Conrol, vol 54, no, pp 8-, Feb 9 [6] J Zheng, Y Ca, Y Lu, Y Xu, B Duan, and X Shen, Opmal power allocaon and user schedulng n mulcell neworks: Base saon cooperaon usng a game-heorec approach, I Trans Wreless Commun, vol 13, no 1, pp , Dec 14 [7] S Har and A Mas-Colell, A smple adapve procedure leadng o correlaed equlbrum, conomerca, vol 68, no 5, pp , [8] P Sasry, V Phansalkar, and M Thahachar, Decenralzed learnng of Nash equlbra n mul-person sochasc games wh ncomplee nformaon, I Trans Sys, Man, Cybern, vol 4, no 5, pp , May 1994 [9] J Zheng, Y Ca, N Lu, Y Xu, and X Shen, Sochasc game-heorec specrum access n dsrbued and dynamc envronmen, I Trans Veh Tech, vol 64, no 1, pp , Oc 15 [3] Y Xu, J Wang, Q Wu, A Anpalagan, and Y Yao, Opporunsc specrum access n unknown dynamc envronmen: A game-heorec sochasc learnng soluon, I Trans Wreless Commun, vol 11, no 4, pp , Apr 1 [31] M Benns, S M Perlaza, P Blasco, Z Han, and H V Poor, Selforganzaon n small cell neworks: A renforcemen learnng approach, I Trans Wreless Commun, vol 1, no 7, pp 3-31, Jul 13 [3] T Innovaons, LT n a nushell, Whe Paper, 1 [33] T S Rappapor, Wreless Communcaons: Prncples and Pracce nd ed Prence Hall PTR, 1 [34] Bernard Sklar, Raylegh fadng channels n moble dgal communcaon sysems Par I: Characerzaon I Commun Mag, vol 35, no 7, pp 91, Jul 1997 [35] Cuervo, A Balasubramanan, D Cho, A Wolman, S Sarou, R Chandra, and P Bahl, MAUI: Makng smarphones las longer wh code offload, n Proc ACM ghh In Conf Moble Sys, Appl, Serv, 1, pp 49-6 [36] B Chun, S Ihm, P Manas, M Nak, and A Pa, Cloneud: lasc execuon beween moble devce and ud, n Proc ACM Sxh Conf Compu Sys, 11, pp [37] S Dey, Y Lu, S Wang, and Y Lu, Addressng response me of udbased moble applcaons, n Proc 1s In Workshop Moble Cloud Compu New, 13, pp 3-1 [38] T Roughgarden, Selfsh Roung and he Prce of Anarchy Cambrdge, MA, USA: MIT Press, 5 [39] K S Narendra and M A L Thahachar, Learnng Auomaa: An Inroducon nglewood Clffs, NJ: Prence-Hall, 1989 [4] M A L Thahachar and P S Sasry, Neworks of Learnng Auomaa: Technques for Onlne Sochasc Opmzaon Sprnger Scence & Busness Meda, 11 [41] O Ayach and R W Heah, Inerference algnmen wh analog channel sae feedback, I Trans Wreless Commun, vol 11, no, pp , Feb 1 Janchao Zheng M 16 receved hs BS degree n Communcaons ngneerng, and PhD degree n Communcaons and Informaon Sysems from College of Communcaons ngneerng, PLA Unversy of Scence and Technology, Nanjng, Chna, n 1 and 16 respecvely From 15 o 16, he was a Vsng Scholar wh he Broadband Communcaons Research Group, Deparmen of lecrcal and Compuer ngneerng, Unversy of Waerloo, Canada He s currenly a Research Assocae wh he Naonal Innovaon Insue of Defense Technology, Academy of Mlary Scences PLA Chna He s also a Pos-Docoral Research Assocae wh he College of Communcaons ngneerng, PLA Army ngneerng Unversy He has publshed several papers n nernaonal conferences and repued journals n hs research area Hs research neress nclude nerference mgaon echnques, green communcaons and compung neworks, game heory, learnng heory, and opmzaon echnques Yuemng Ca M 5-SM 1 receved he BS degree n Physcs from Xamen Unversy, Xamen, Chna n 198, he MS degree n Mcro-elecroncs ngneerng and he PhD degree n Communcaons and Informaon Sysems boh from Souheas Unversy, Nanjng, Chna n 1988 and 1996 respecvely Hs curren research neres ncludes cooperave communcaons, sgnal processng n communcaons, wreless sensor neworks, and physcal layer secury Yuan Wu M 1-SM 16 receved he PhD degree n lecronc and Compuer ngneerng from he Hong Kong Unversy of Scence and Technology, Hong Kong, n 1 He s currenly an Assocae Professor n he College of Informaon ngneerng, Zhejang Unversy of Technology, Hangzhou, Chna Durng 1-11, he was he Posdocoral Research Assocae a he Hong Kong Unversy of Scence and Technology Durng 16-17, he s he vsng scholar a he Broadband Communcaons Research BBCR group, Deparmen of lecrcal and Compuer ngneerng, Unversy of Waerloo, Canada Hs research neress focus on rado resource allocaons for wreless communcaons and neworks, and smar grd He s he recpen of he Bes Paper Award n I Inernaonal Conference on Communcaons ICC n 16 Xuemn Sherman Shen M 97-SM -F 9 receved he BSc198 degree from Dalan Marme Unversy Chna and he MSc 1987 and PhD degrees 199 from Rugers Unversy, New Jersey USA, all n elecrcal engneerng He s a Professor and Unversy Research Char, Deparmen of lecrcal and Compuer ngneerng, Unversy of Waerloo, Canada He s also he Assocae Char for Graduae Sudes Dr Shen s research focuses on resource managemen n nerconneced wreless/wred neworks, wreless nework secury, socal neworks, smar grd, and vehcular ad hoc and sensor neworks He s an eleced member of I ComSoc Board of Governor, and he Char of Dsngushed Lecurers Selecon Commee Dr Shen served as he Techncal Program Commee Char/Co-Char for I Globecom 16, Infocom 14, I VTC 1 Fall, and Globecom 7, he Symposa Char for I ICC 1, he Tuoral Char for I VTC 11 Sprng and I ICC 8, he General Co-Char for ACM Mobhoc 15, Chnacom 7 and QShne 6, he Char for I Communcaons Socey Techncal Commee on Wreless Communcaons, and PP Communcaons and Neworkng He also serves/served as he dor-n-chef for I Nework, Peer-o-Peer Neworkng and Applcaon, and IT Communcaons; an Assocae dor-n-chef for I Inerne of Thngs Journal, a Foundng Area dor for I Transacons on Wreless Communcaons; an Assocae dor for I Transacons on Vehcular Technology, Compuer Neworks, and ACM/Wreless Neworks, ec; and he Gues dor for I JSAC, I Wreless Communcaons, I Communcaons Magazne, and ACM Moble Neworks and Applcaons, ec Dr Shen receved he xcellen Graduae Supervson Award n 6, and he Ousandng Performance Award n 4, 7, 1, and 14 from he Unversy of Waerloo, he Premers Research xcellence Award PRA n 3 from he Provnce of Onaro, Canada, and he Dsngushed Performance Award n and 7 from he Faculy of ngneerng, Unversy of Waerloo Dr Shen s a regsered Professonal ngneer of Onaro, Canada, an I Fellow, an ngneerng Insue of Canada Fellow, a Canadan Academy of ngneerng Fellow, a Royal Socey of Canada Fellow, and a Dsngushed Lecurer of I Vehcular Technology Socey and Communcaons Socey c 18 I Personal use s permed, bu republcaon/redsrbuon requres I permsson See hp://wwweeeorg/publcaons_sandards/publcaons/rghs/ndexhml for more nformaon