Execution Cost and Fairness Optimization for Multi-Server Mobile-Edge Computing Systems with Energy Harvesting Devices

Transcription

1 In. J. xxxxxxxxx xxxxxxxxxxxxxms, Vol. X, No. Y, xxxx 1 Execuion Cos and Fairness Opimizaion for Muli-Server Mobile-Edge Compuing Sysems wih Energy Harvesing Devices Hailiang Zhao, Wei Du, Wei Liu, Tao Lei, Qiwang Lei School of Compuer Science and Technology, Wuhan Universiy of Technology, Wuhan, China Absrac: Mobile edge compuing sysems wih energy harvesing devices pave he way for qualiy of user experience (QoE) improvemen by compuaion offloading and green compuing by uilizing renewable energy. Neverheless, because baery energy is obained free and ime-varying, compuaion offloading sraegies should have differen design objecive and is emporally correlaed. Moreover, considering muli-user and muli-server sysems where mobile devices can move arbirarily, resource compeiion and MEC server selecion should be incorporaed in he sraegies. In his paper, we will develop wo effecive compuaion offloading sraegies. The execuion cos (i.e., execuion laency and penaly for dropped asks) and he fairness (in erms of raio of compuaion asks offloaded) are adoped as he performance merics. Two online algorihms, namely, he LODCO-Based Greedy Algorihm and LODCO-Based ε-greedy Algorihm, are proposed. Boh of hem are based on Lyapunov Opimizaion echnologies and he LODCO Algorihm 1. By choosing he execuion mode among local execuion, offloading execuion and ask dropping for each mobile device, our algorihms can asympoically obain he opimal resuls for he whole sysem. Two algorihms proposed are lowcomplexiy and could be operaed wihou oo much priori knowledge. Moreover, he algorihms will inheri every advanage from LODCO Algorihm and adap perfecly o he more complex environmen. Simulaion resuls illusrae ha compared wih he LODCO Algorihm, our algorihms could improve he raio of compuaion asks offloaded by 5% and 10%, respecively. Keywords: Mobile edge compuing, Energy harvesing, Compuaion offloading, Greedy sraegy, Lyapunov Opimizaion. 1 Inroducion Mobile edge compuing (MEC) is a new paradigm which provides IT environmen and cloud compuing capabiliy wihin radio access neworks [1]. By offloading compuaion inensive asks o MEC servers, users could experience low laency and ulra-high bandwidh services in a MEC sysem [2]. To overcome he limiaions of baery-powered mobile devices, energy harvesing (EH) is inroduced o MEC sysems [3], where mobile devices could be charged by renewable energy sources such as solar radiaion, wind and human moion energy [4]. Recenly, efficien compuaion offloading sraegies have been 1 LODCO is he abbreviaion of he Lyapunov opimizaion-based dynamic compuaion offloading, which is an algorihm designed in Reference [3]. Copyrigh 2018 Inderscience Enerprises Ld.

2 Hailiang Zhao, Wei Du, Wei Liu, Tao Lei, Qiwang Lei developed for single-user and single-server MEC sysems wih EH devices [3]. Unforunaely, hese sraegies are no suiable for muli-user and muli-server sysems, which are more ypical scenarios in a real world [5] because of failing o address resource compeiion and server selecion. To exploi in full he benefis of compuaion offloading in he considered muli-user and muli-server MEC sysems wih EH devices, here are several key challenges ha need o be addressed. Firsly, he compuaional resource and he radio resource are shared by muliple mobile devices. As a resul, inerference and compeiion could no be ignored. How o allocae limied resources fairly among users should be invesigaed. Secondly, a user probably could connec o more han one MEC server. Then, he user could choose one server o offload is ask. So, how o choose a proper server according o he sysem opimizaion merics or he user s preference has o be invesigaed. In his paper, we design wo online compuaion offloading sraegies for muli-user and muli-server MEC sysems wih EH devices. Our major conribuions are summarized as follows: We consider a general MEC sysem wih muliple EH mobile devices and muliple resource-consrained MEC servers where every mobile device can move arbirarily wihin cerain areas. User mobiliy and is effec on resources conenion and server selecion are modelled as one par of a non-convex opimizaion problem. The execuion cos (i.e. execuion laency and penaly for dropped asks) and fairness (in erms of he raio of offloading compuaion asks) are opimized simulaneously. Afer uilizing Lyapunov Opimizaion, he original problem can be convered o a deerminisic opimizaion problem a each ime slo, which is a cornersone of wo algorihms proposed in his paper. Boh wo proposed algorihms can handle he correlaion beween any wo mobile devices when choosing he compuaion modes, especially he offloading compuaion mode, which could no be solved by he LODCO Algorihm. In addiion, boh wo proposed algorihms can obain he opimal resuls afer several ieraions. By comparing wih he LODCO Algorihm, he LODCO-Based Greedy Algorihm and he LODCO-Based ε-greedy Algorihm no only keeps perfecly he advanages of LODCO Algorihm bu also promoes he raio of compuaion asks offloaded noably. The organizaion of his paper is as follows. We survey sae of he ar in Secion 2. In Secion 3, he sysem model is inroduced. In Secion 4, he LODCO-Based Greedy Algorihm and he LODCO-Based ε -Greedy Algorihm are proposed based on he formulaed problem. Simulaion resuls and Conclusions of his paper are demonsraed in Secion 5 and Secion 6, respecively. 2 Relaed works Compuaion offloading for muli-user muli-server mobile sysems is a very challenging problem because of complexiy of he scenario and inerdependence among users and servers. A few sraegies have proposed in recen years. In [6], he power-delay rade-off in he conex of ask offloading was sudied. The problem was formulaed as a compuaion and ransmis power minimizaion subjec o laency and reliabiliy consrains. In [7], he

3 Execuion Cos and Fairness Opimizaion for Muli-Server Mobile-Edge Compuing Sysems wih Energy Devices power minimizaion for he mobile devices by daa offloading was invesigaed. Cenralized and disribued algorihms for join power allocaion and channel assignmen ogeher wih decision making were proposed. In [8], he problem of join ask offloading and resource allocaion was formulaed as a mix ineger non-linear program. The ask offloading decision, uplink ransmission power of mobile users and compuaional resource allocaion a he MEC servers were joinly opimized. The users ask offloading gains, which are measured by he reducion in ask compleion ime and energy consumpion, were maximized. Obviously, energy consumpion is always opimized in all he works menioned above. However, he opimizaion objecive of MEC sysems wih EH devices is shifed from minimizing he baery energy consumpion as he harvesed energy is ample and free. As a resul, hose compuaion offloading sraegies dedicaed o he energy conservaion canno be uilized wihou modificaion. In [9], a device-edge-cloud MEC sysem was invesigaed. A nework aware muli-user muli-edge compuaion pariioning problem was formulaed. Compuaion and radio resources were allocaed such ha he average hroughpu of he users was maximized. The sysem considered in [9] is similar o his paper. Neverheless, a parial offloading model was uilized in [9] while a binary offloading is exploied in his paper. The work mos similar o his paper is [3]. In [3], a green MEC sysem wih EH devices was analysed, and an effecive compuaion offloading sraegy was developed. Moreover, he execuion cos, which includes boh he execuion laency and ask failure, was adoped as he performance meric. Finally, a low-complexiy online algorihm was proposed. The key differences beween our work and [3] are summarized as follows. Firsly, while he sysem discussed in [3] focused on a single user single MEC server, we dedicae on muliuser muli-server scenario. Secondly, user mobiliy, which was ignored in [3], is considered in his paper. Thirdly, he MEC server has limied compuaion capabiliy, which generalizes he work in [3], where he MEC server was assumed o be wih unlimied compuaional resources. Fourhly, compared o he single-objecive opimizaion problem formulaed in [3], we are ineresed in opimizing wo objecives simulaneously. Due o he differences menioned above, our work is more complicaed and difficul han ha in [3]. 3 Sysem model 3.1 Sysem descripion We consider a MEC sysem consising of N mobile devices equipped wih EH componen and M MEC servers, where each mobile device and each MEC server share he same propery, respecively. We assume ha ime is sloed, and denoe he ime slo lengh and he ime slo index se by τ and T {0,1, }, separaely. We use N {1,2,, N} o denoe he se of mobile devices and M {1,2,, M} o denoe he se of MEC servers.

4 Hailiang Zhao, Wei Du, Wei Liu, Tao Lei, Qiwang Lei Fig. 1. A sysem wih muliple mobile device and muliple MEC server. As shown in Fig. 1, all mobile devices and MEC servers are limied o a specific area, where each MEC server is locaed a a paricular posiion wihou moving and each mobile device can move around arbirarily. Each mobile device s locaion is assumed o be independen and idenically disribued (i.i.d.), i.e., ih mobile device s locaion remains saic wihin each ime slo bu varies among differen ime slo. Denoe he locaion of ih mobile device a h ime slo as (x i, y i ), and x i ~U(0, W), y i ~U(0, L), T, i N, where W and L denoe he widh and lengh of he specific space, respecively. Denoe he disance beween ih mobile device and jh MEC server a h ime slo as 2 d i,j, where d i,j (x i x j ) 2 + (y i y j ) 2, i.e., he disance marix a h ime slo D (d i,j ) N M is deermined, while ime-varying in differen ime slos. 3.2 Compuaion asks model We focus on delay-sensiive compuaion asks wih he execuion deadline no greaer han he lengh of ime slo [6]. Denoe he compuaion ask generaed by ih mobile device a h ime slo as CT i, who has he fixed size L (in bis). Meanwhile, we assume he compuaion asks are modeled as an i.i.d. Bernoulli process [3], i.e., a he beginning of each ime slo, for each mobile device one compuaion ask CT i is requesed wih probabiliy ρ where 0 < ρ < 1. Denoe ζ i = 1 if ih mobile device ge compuaion ask reques a h ime slo. In our sysem, here exiss no cache queue eiher in mobile devices or MEC servers. As a resul, compuaion ask CT i can eiher be execued locally a ih mobile device, or be offloaded o jh MEC server and hen be execued, where j is he chosen MEC server for ih mobile device by sysem operaion. Besides, if he energy is insufficien of ih mobile device or ζ i = 0, he compuaion ask a h ime slo will be dropped. Denoe I i,c {0,1} wih c = {l, r, d} as he compuaion mode indicaors [3] for ih mobile device a h ime slo, where I i,l = 1, I i,d = 1 and I i,r = 1 indicae ha he compuaion ask is execued locally, execued remoely by MEC server and dropped, independenly. Because here

5 Execuion Cos and Fairness Opimizaion for Muli-Server Mobile-Edge Compuing Sysems wih Energy Devices exiss only 3 modes for ih mobile device o choose a h ime slo, hence hose indicaors should follow he equaion below: I i,l + I i,r + I i,d = 1, T, i N. (1) 3.3 Offloaded compuaion model Denoe c i,j = 1 as he indicaor of ih mobile device choosing jh MEC server o offload, where c i,j {0,1}. Thus, he connecion marix C (c i,j ) N M is ime-varying, which is he same as he disance marix. We each compuaion ask is assigned o only one server when choosing he offloading compuaion mode [6], i.e., M j=1 c i,j = 1, T, i N, jεm. (2) Denoe γ i,j as he small-scale fading channel power gains, which are assumed o be exponenially disribued wih uni mean. Besides, each mobile device shares he same γ i,j a h ime slo. According o communicaion heory, he channel power gain from ih mobile device o jh MEC server can be expressed by h i,j = γ i,j g 0 ( d 0 d i,j ) θ, where d 0 denoes he reference disance, θ denoes he pass-loss exponen and g 0 denoes he pahloss consan. As a resul, we can obain he achievable rae Γ(h i,j, p i ) by Γ(h i,j, p i ) = ωlog 2 (1 + h i,j p i σ) according o Shannon Theorem, where ω represens he sysem bandwidh, σ is he noise power a each MEC server and p i is he ransmi power who canno exceed he maximum value p max. We assume ha every conneced mobile device of jh MEC server shares he same bandwidh and each MEC server shares he same noise power. Denoe X mobile and X server as he numbers of CPU cycles required o process one bi ask of mobile device and MEC server, respecively. We assume ha he compuaional abiliies of MEC servers are consrained, i.e., i N I( I i,r ) c i,j LX server f max server τ, T, jεm, (3) max where I( ) is he indicaor funcion and f server denoes he upper bound of each MEC server s CPU-cycle frequency. We ake no consideraion of execuion laency consumed by he MEC server s execuion process [3], i.e., for ih mobile device, he oal execuion laency of his mode equals he ransmission delay for he inpu ask. Thus, L D i,remoe =,pi, T, i N, jεm. (4) ) Γ(h i,j Similarly, we ake no consideraion of MEC server s energy consumpion. Thus, he energy consumed by ih mobile device can obained by L E i,remoe = p i,pi, T, i N, jεm. (5) ) 3.4 Local compuaion model Γ(h i,j In order o execue L bis compuaion ask successfully, LX mobile CPU cycles are required. By applying he dynamic volage and frequency scaling echnologies (DVFS) [10], mobile device can conrol he energy consumed and he execuion laency. Thus, he oal execuion laency of his mode can be obained by LX D i,local = mobile v=1 (f i,v ) 1, T, i N. (6) hus, he energy consumed by ih mobile device can obained by [3] E i,local LX mobile = v=1 s(f i,v ) 2, T, i N, (7)

6 Hailiang Zhao, Wei Du, Wei Liu, Tao Lei, Qiwang Lei where s is he effecive capaciance coefficien. Moreover, we denoe he upper bound of CPU-cycle frequency for each mobile device as f max mobile,i.e., v {1,2,, LX mobile }, f i,v f max mobile, T, i N. 3.5 Energy harvesing model In order o embody he sochasic and inermied naure of he renewable energy process [11], we assume ha he harvesable energy E H for each mobile device a he beginning of h ime slo is uniformly disribued wih he maximum value of E max H. The sysem need o decide he size of energy who will be sored in he baery of ih mobile device [3]. Denoe his par of energy as e i, hen we have: 0 e i E H, T, i N. (8) We assume ha oher kinds of energy consumpion besides local-execuion and remoeexecuion is sufficien small. Denoe he energy consumed by i h mobile device as E(I i, f i, p i ), where I i [I i,l, I i,r, I i,d ], f i [f i,1,, f i,lxmobile ]. Then We can obain E(I i, f i, p i ) by he following equaion: E(I i, f i, p i ) = I i,l E i,local Denoe he baery level of ih mobile device a h ime slo as b i. Obviously, he energy consumpion a each ime slo canno surpass he baery level, i.e., E(I i, f i, p i ) b i, T, i N. (10) Besides, b i evolves according o he following equaion: b +1 i = b i E(I i, f i, p i ) + e i, T, i N. (11) 3.6 QoE-Cos funcion + I i,r E i,remoe, T, i N. (9) Users QoE [12] consiss of execuion delay and he penaly for dropping he ask. Denoe he QoE-cos of he whole sysem a h ime slo as cos sum, which is he sum of QoEcos of each mobile device (denoed as cos i ). Therefore, we can obain he following equaion: cos sum iεn cos i = iεn [D(I i, f i, p i ) + I(ζ i I i,d )], T, i N, (12) where denoes as he weigh of ask dropping cos and D(I i, f i, p i ) is given by D(I i, f i, p i ) = I(ζ i = 1) (I i,l D i,local + I i,r D i,remoe ), T, i N. (13) 3.7 Opimizaion model In our sysem environmen, each MEC server has more compuing power han each mobile device, i.e., while he cos of offloading he ask is sill lower han he cos of local execuion, he more compuaion asks execued remoely, he beer user s qualiy of experience. We can obain he number of offloading compuaion asks a h ime slo by i N I(ζ i I i,r ), T, (14) where I( ) is he indicaor funcion. Therefore, we have wo opimizaion goals, i.e., he minimizaion of he average weighed sum QoE-cos and he maximizaion of he number of offloading compuaion asks. Denoe he sysem operaion a he h ime slo as SO [I, f, p, C, e ], T, (15) where I [I 1,, I N ], f [f 1,, f N ], p [p 1,, p N ], e [e 1,, e N ]. Consequenly, he opimizaion problem can be expressed as: 1 P 1 : min lim T 1 cos SO T T =0 sum ψ i N I(ζ i I i,r )

7 Execuion Cos and Fairness Opimizaion for Muli-Server Mobile-Edge Compuing Sysems wih Energy Devices s.. (1), (2), (3), (8), (10) 0 f i,v f max mobile, T, i N, v {1,, LX mobile } (16) E(I i, f i, p i ) E max, T, i N (17) 0 p i p max, T, i N (18) I i,c {0,1}, c {l, r, d}, T, (19) where ψ defined as he weigh of he second opimizaion goal. (16) and (18) incarnae he consrains of mobile devices CPU-cycle frequency and maximum ransmi power, respecively. (17) incarnaes he upper bound of baery discharging for securiy reasons, i.e., he amoun of energy oupu energy canno exceed E max a each ime slo. (19) represens he 0-1 indicaor consrain which has been described in subsecion Online Algorihms for Execuion Cos and Fairness Opimizaion In his secion, we will develop LODCO-Based Greedy Algorihm and LODCO-Based ε- Greedy Algorihm o solve P 1 on accoun of LODCO algorihm [3]. Firs of all, we will conver he original problem which is ime-dependen o a deerminisic problem P 2 by aking advanages of Lyapunov Opimizaion. Then he LODCO algorihm will be upgraded and reconsruced for he muli-user and muli-server MEC sysem by virue of Greedy Sraegy. We will no demonsrae he deails of LODCO algorihm, bu we will explain closely abou why we can apply i o our model. 4.1 Drif plus penaly formula Lyapunov opimizaion echnologies demands ha he allowable acion ses are i.i.d., which canno be saisfied by he ime-dependen baery queues of mobile devices. Therefore, for each mobile device we use he perurbaion parameer θ 1 (which is lower bounded by E max + V in [3]) o define o virual baery queue b i by b i b i θ, T, i N, (20) where E max min {max{kw(f max mobile ) 2, p max τ d }, E max }. Besides, if a ask requesed a he h ime slo is being execued locally, he opimal frequencies of he LX mobile CPU cycles should be he same, i.e., f i,v = f i, i {1,, LX mobile }, which can be obained by Inequaliy of arihmeic and geomeric means. According o he analysis above, we define he Lyapunov funcion as L() 1 2 (b i ) 2 iεn = 1 (b 2 iεn i θ) 2, T. (21) Thus, he condiional Lyapunov drif can be wrien as () E[L( + 1) L() b ], T, (22) where b [b 1,, b N ]. Then he Lyapunov drif-plus-penaly funcion can be wrien as V () () + V E[cos sum b ], T. (23) Because of he energy evoluion equaion (11), we can obain ha (b i+1 ) 2 (b i ) 2 + 2b i (e i E(I i, f i, p i )) + (e i ) 2 + E 2 (I i, f i, p i ), T, i N. (24) 1 he perurbaion parameer θ is proposed in [3] o circumven he above issue, which is ha he vanilla version of Lyapunov Opimizaion echnologies canno be applied direcly. The process o obain he value of θ is an imporan componen of LODCO Algorihm.

8 Hailiang Zhao, Wei Du, Wei Liu, Tao Lei, Qiwang Lei Because E max denoes as he real energy consumpion s upper bound consumed by ih mobile device a h ime slo, we have E(I i, f i, p i ) E max. Since e i E i,h and E i,h s are i.i.d. among differen ime slos wih he maximum value of E max H, so we can ge e i E max H. As a resul, wih some manipulaions, we have () iεn E[b i (e i E(I i, f i, p i )) b ] + C, T, (25) where C = N 2 (E max + E max H ) 2. Since (24), we can obain ha V () b i (e i E(I i, f i, p iεn i )) + V E[cos sum b ] + C, T, (26) which means V () is upper bounded. Obviously, he upper bound of V () has he same srucure wih he problem defined in [3]. Thus, he LODCO Algorihm can hopefully be applied o deicide SO by solving he following deerminisic problem P 2 : min SO b i (e i E(I i, f i, p iεn i )) + V E[cos sum b ], T, i N, which subjecs o every consrain condiions of P Applicaion of LODCO Algorihm P 2 is an opimized forma of P 1 by Lyapunov Opimizaion excep maximizing he number of offloading compuaion asks. According o LODCO Algorihm, he problem can be decomposed o wo sub problems. The firs one is o find he opimal energy harvesing, and he second one is o decide he opimal compuaion modes. Opimal energy harvesing: he opimal amoun of harvesed energy e i for i h mobile device can be obained by solving he following problem: min 0 e i Ei,H iεn b i e i.,iεn, T Because each mobile device s decision on opimal energy harvesing is muually independen, he opimal e i can be obained separaely for each mobile device. Thus, we can uilize he conclusion of [3] direcly, i.e., we obain e i for each mobile device by (21) in [3]. Decide he compuaion modes: we need o obain he mode wih he minimum value of J CO (I i, f i, p i ) for each mobile device, where J CO (I i, f i, p i ) I(I i,l ) J mobile (f i ) + I(I i,r ) J server (p i ) + I(I i,d ) V, T, i N, (27) where J mobile (f i ) and J server (p i ) denoe as he sub problems of local-execuion mode and remoe-execuion mode, respecively. Boh of hem are consiuen pars of he LODCO Algorihm. We have he same Lyapunov srucure wih [3], hus we can obain he opimal SO when each mobile device can make decision separaely. However, here exiss correlaion beween any wo mobile devices when choosing he compuaion modes, especially he offloading compuaion mode. In order o solve he problem, we propose LODCO-Based Greedy Algorihm for he muli-user and muli-server sysem. Beyond ha, we propose he modified version, i.e., LODCO-Based ε-greedy Algorihm o maximize he number of offloading compuaion asks by virue of heoreical knowledge of Reinforcemen Learning. 4.3 LODCO-Based Greedy Algorihm In his secion, we demonsrae he deails of proposed algorihm. Algorihm 1: LODCO-Based Greedy Algorihm

9 Execuion Cos and Fairness Opimizaion for Muli-Server Mobile-Edge Compuing Sysems wih Energy Devices 1: A he beginning of ime slo, iniialize flag[m] wih 0 and esablish a map o sore he indexes of mobile device and corresponding chosen MEC server. 2: for each mobile device i do 3: Obain he ask reques indicaor ζ i, virual energy queue b i and harvesable energy E i,h. 4: Generae he locaion of each mobile device and compue he disance d i,j beween he ih mobile devices and he jh MEC server. 5: Obain he opimal harvesed energy e i by he LODCO Algorihm. 6: Obain he opimal f i for local execuion by he LODCO Algorihm, hen record he opimal value J mobile (f i ). If he baery energy level is insufficien for local execuion, se J mobile (f i ) as inf. 7: Obain he opimal f i for local execuion by he LODCO Algorihm, hen record he opimal value J mobile (f i ) (assume ha mobile devices have no correlaion now). 8: for each MEC server j do 9: Obain he channel power gain h i,j from ih mobile device o jh MEC server by h i,j = γ i,j g 0 ( d 0 d i,j ) θ. 10: Obain he opimal p i,j from i h mobile device o j h MEC server by he LODCO Algorihm, hen record he opimal value J server (p i,j ). If he baery energy level is insufficien for offloaded execuion from ih mobile device o jh MEC server, se J server (p i,j ) as inf. 11: Choose he opimal p i by selecing he one wih minimum J server (p i,j ), denoe as J server (p i ) and hen record j. 12: end for 13: Compare J mobile (f i ), J server (f i ) and V, choose he mode wih he minimum value and se he corresponding indicaor variable I i,c as 1 (wihin he power limi). 14: if I i,r = 1 hen 15: obain he i h mobile device and he corresponding j h MEC server, hen inser hem ino he map wih key i and value j. 16: end if 17: end for 18: Calculae he upper bound of he MEC server S UB by S UB f server τ LX server. 19: while he map is no NULL do 20: Obain he key-value pair i-j wih he minimum J server (p i ). 21: Compare J mobile (f i ), J server (p i ) and V, choose he mode wih he minimum value and se he corresponding indicaor variable I i,c as 1 (we have o do he search again because i is possible ha J server (p i ) has been modified). 22: if I i,r = 1 hen 23: if flag[j] S UB hen 24: Remove he key-value pair i- j from he map and flag[j ]++. Then se ) as inf. J server (p i,j 25: else

10 Hailiang Zhao, Wei Du, Wei Liu, Tao Lei, Qiwang Lei 26: if min{j server (p i,: )}!= inf hen 1 27: Find he opimal j by min{j server (p i,: )} and he inser hem o he map. Then coninue. 28: else 29: Selec he opimal mode from oher 2 modes: local execuion and dropping he ask. Then remove he corresponding key-value pair from map. 30: end if 31: end if 32: else 33: Keep he corresponding I i,c = 1 wihou change and hen remove he corresponding key-value pair from map. 34: end if 35: end while 36: Calculae he value of i N I(ζ i I i,r ) i N I(ζ i ), i N I(ζ i I i,d ) i N I(ζ i ) and i N I(ζ i I i,l ) i N I(ζ i ). 37: Obain each mobile device s execuion cos and energy consumpion. 38: Updae he baery level for each mobile device. 39: Updae o + 1. As shown in Algorihm 1, we can use he LODCO Algorihm o obain he opimal e i and f i because hey are independen o each oher. We can sill use he LODCO Algorihm o obain p i,j for ih mobile device no maer which MEC server is chosen, hen we use he Greedy Policy choose he bes p i among hose opimal p i,j. 4.4 LODCO-Based ε-greedy Algorihm In his secion, we will demonsrae he deails of LODCO-Based ε-greedy Algorihm. There exiss he Exploraion-Exploiaion dilemma in he heoreical model of K-armed bandi [13], which is a ypical Single-Sep Reinforcemen Learning Task. Thus, ε-greedy Sraegy was proposed o achieve a rade-off. In he same way, we view he opimizaion arge of P 2 and he number of compuaion offloading asks as exploiaion and exploraion, respecively. Then, we can obain he following algorihm based on LODCO-Based Greedy Algorihm. Algorihm 2: LODCO-Based ε-greedy Algorihm 1: Run sep. 1 ~ sep. 18 in Algorihm 1. 2: while he map is no NULL do 3: Obain he key-value pair i-j wih he minimum J server (p i ). 4: if rand() < ε hen 5: if flag[j] S UB hen 6: Remove he key-value pair i-j from he map and hen flag[j]++ no maer wheher J server (p i ) is he minimum among J mobile (f i ), J server (p i ) and V. Then se J server (p i,j ) as inf. 7: else 8: if min{j server (p i,: )}!= inf hen 1 J server (p i,: ) is defined as [J server (p i,1 ), J server (p i,2 ),, J server )]. (p i,m

11 Execuion Cos and Fairness Opimizaion for Muli-Server Mobile-Edge Compuing Sysems wih Energy Devices 9: Find he opimal j by min{j server (p i,: )} and he inser hem o he map. Then coninue. 10: else 11: Selec he opimal mode from oher 2 modes: local execuion and dropping he ask. Then remove he corresponding key-value pair from map. 12: end if 13: end if 14: else if rand() ε hen 15: Run sep. 20 ~ sep. 33 in Algorihm 1. 16: end if 17: end while ) 18: Calculae he value of i N I(ζ i I i,r i N I(ζ i ), i N I(ζ i I i,d i N I(ζ i ) and i N I(ζ i I i,l ) i N I(ζ i ). 19: Obain each mobile device s execuion cos and energy consumpion. 20: Updae he baery level for each mobile device. 21: Updae o + 1. ) 5 Simulaion resuls In his secion, we will demonsrae he resuls of he proposed algorihms and verify heir effeciveness. Then, we will show he impacs of he sysem parameers by conrol variable mehod. As menioned in secion 4, we will no elaborae upon he deails abou he verificaion of LODCO Algorihm. The simulaion was run on a machine wih an Inel Core 2.5 GHz i7-4710mq CPU. The algorihm was implemened in MATLAB R2015b and was given up o 8 GB of memory if needed. In our sysem, he harvesable energy E i,h is uniformly disribued wih he maximum value of E max H, where E max H can be obained by average EH power P H, i.e., E max H = P H 2τ, T. (28) We assume ha P H = 12 mw, g 0 = 40 db (pah-loss consan), s = (effecive capaciance coefficien), τ = = 2 ms (ime slo lengh and he weigh of he ask dropping cos, respecively). In addiion, ω = 1 MHz (sysem bandwidh), σ = max max W (noise power a each MEC server), f mobile = f server = 1.5 GHz (upper bounds of each mobile device and each MEC server s CPU-cycle frequency, respecively), E max = 2 mj (he real energy consumpion s upper bound consumed by mobile device a each ime slo), L = 1000 bis (size of each compuaion ask), V = 10 5 (coefficien of he penaly in Lyapunov Opimizaion), and X server = X mobile = 5900 cycles per bye (numbers of CPU cycles required by each mobile device and each MEC server). Besides, we assume here are 10 mobile devices and 5 MEC servers, i.e., N = 10, M = 5, and E min = 0.02 mj 1. According o he above parameer values, he upper bound S UB is 4 by S UB f max server τ LX server, which describes he maximum number of mobile devices ha can be conneced. In he following par, he defaul value of ρ (ask reques probabiliy) and ε in 1 E min is he non-zero lower bound of mobile devices defined in [3].

12 Hailiang Zhao, Wei Du, Wei Liu, Tao Lei, Qiwang Lei Algorihm 2 are 0.6 and 0.25 unless saed, respecively. γ i,j (he small-scale fading channel power gains a h ime slo) is exponenially disribued wih mean 1. Besides, we assume he maximum disance beween random mobile and random MEC server is 100 m unless saed, which is he upper limi for he uniform disribuion. 5.1 Validaion of Effeciveness In his subsecion, we will verify he effeciveness of LODCO-Based Greedy Algorihm and LODCO-Based ε-greedy Algorihm compared wih LODCO Algorihm. As shown in Fig. 2, he baery energy level of each mobile device, which is he mean value of he proposed wo algorihms, demonsrae he feasibiliy of our improving on LODCO Algorihm. The energy level of each mobile device keeps accumulaing a earlier sage, and finally sabilizes around he perurbed energy level a abou 180 h ime slo, which exacly keeps he advanages of LODCO Algorihm. Fig. 2. Baery energy level of each mobile device vs. ime. As depiced in Fig. 3, he Y-axis describes he average value of 1500 ime slos of each mobile device s energy level. As seen, each energy level is confined wihin [0,0.005] 1 J, which conforms o he heoreical resuls derived in LODCO Algorihm. Fig. 3. Average energy level of each mobile device. Fig. 4 demonsraes he raio of each chosen modes in our muli-user and muli-server sysem by LODCO-Based ε -Greedy Algorihm. A he very earlier sage, pleny of is he specific value of θ + E H max.

13 Execuion Cos and Fairness Opimizaion for Muli-Server Mobile-Edge Compuing Sysems wih Energy Devices compuaion asks are dropped due o he insufficien energy level. Then he raio of dropped asks significan decreases o 0 along wih he ascending baery energy level of each mobile device. Meanwhile, he raio of offloading asks clearly greaer han he raio of locally-execued asks and he average raio of offloading asks obained by LODCO Algorihm 1, which means ha LODCO-Based ε-greedy Algorihm is able o obain beer performance han LODCO Algorihm in erms of geing he mos of offloading compuaion number. Fig. 4. The raio of each chosen modes vs. ime. Fig. 5. Comparison beween 2 proposed algorihms. 1 The difference values are approximaely 5% and 10%, respecively.

14 Hailiang Zhao, Wei Du, Wei Liu, Tao Lei, Qiwang Lei We compare he performance of LODCO-Based ε-greedy Algorihm wih ha of LODCO-Based Greedy Algorihm on he second opimizaion goal, i.e., he number of offloading compuaion asks. As depiced in Fig. 5, he average raio of offloading asks obained by LODCO-Based ε-greedy Algorihm (which is %) is greaer han obained by LODCO-Based Greedy Algorihm (which is %), where boh of hem are larger han he average raio of offloading asks obained by LODCO Algorihm (which is %). The resuls verify ha our second algorihm can obain beer offloading compuaion asks number han he firs algorihm by ε-greedy Sraegy can do. 5.2 Effecs of sysem parameers In his subsecion, we will demonsrae he impacs of sysem parameers on he performance of proposed algorihms. (a) (b) Fig. 6. Average raion of offloading asks vs. maximum disance and ε. Fig. 6(a) depics he impac of he maximum disance beween random mobile device and random MEC server. As seen, along wih he increase of he maximum disance, he average raio of asks offloaded gradually decrease. When he disance is arbirarily far, he number of offloading compuaion asks will be zero. The reason is ha he channel power gain grows wih he disance beween each mobile device and each MEC server, which will lead o larger energy consumpion and longer execuion delay. As a resul, more and more mobile devices will choose o execue he compuaion asks locally. As depiced in Fig. 6(b), along wih he increase of ε, which belong o [0,1], he average raio of offloading asks gradually increase wih a slowdown rae and finally converge o he specific value %. The reason is ha LODCO-Based ε-greedy is based on he ε-greedy Sraegy, i.e., a greaer ε will bring a lager probabiliy o choose he offloading mode. 6 Conclusions In his paper, we invesigaed a mobile-edge compuing sysems wih muli-user and muliserver. Then we proposed wo algorihms o obain he lowes execuion cos and larges number of offloading compuaion modes based on LODCO Algorihm, i.e., LODCO- Based Greedy Algorihm and LODCO-Based ε-greedy Algorihm. Those wo algorihms are online algorihms wih low-complexiy. Mos imporanly, hey have no need of oo much priori knowledge. By exensive simulaion and performance analysis, we can see ha hose wo algorihms inheri every advanage from LODCO Algorihm and adap o he more complex environmen perfecly and offer more han 5% and 10% raio of offloading

15 Execuion Cos and Fairness Opimizaion for Muli-Server Mobile-Edge Compuing Sysems wih Energy Devices compuaion asks, respecively. LODCO-Based ε -Greedy Algorihm can choose he offloading mode as far as possible, which can bring resource-limied MEC servers superioriy ino full play. In conclusion, our sudy provides a viable suggesion o design a complex sysem which is much more approachable o realiy. References [1] Y.C. Hu, M. Pael, D. Sabella, e al., Mobile edge compuing A key echnology owards 5G, ETSI Whie Paper, [2] T.X. Tran, H. Abolfazl, P. Pandey, e al., Collaboraive mobile edge compuing in 5G neworks: New paradigms, scenarios, and challenges, IEEE Communicaion Magazine, 2017, 55(4): [3] Y.Y. Mao, J. Zhang, K. B. Leaief, Dynamic compuaion offloading for mobile-edge compuing wih energy harvesing devices, IEEE Journal of Seleced Areas Communicaions, 2016, 34(12): [4] S. Sudevalayam, P. Kulkarni, Energy harvesing sensor nodes: Survey and implicaions, IEEE Communicaions Surveys & Tuorials, 2011, 13(3): [5] X. Ge, S. Tu, G. Mao, e al., 5G ulra-dense celluar neworks, IEEE Wireless Communicaions, 2016, 23(1): [6] C.F. Liu, M. Bennis, H.V. Poor, Laency and reliabiliy-aware ask offloading and resource allocaion for mobile edge compuing, IEEE Global Communicaions Conference Workshops (GLOBECOM Workshops), Singapore, [7] M. Masoudi, B. khamidehi, C. Cavdar, Green cloud compuing for muli cell neworks, IEEE Wireless Communicaions and Neworking Conference (WCNC), San Francisco, USA, [8] T.X. Tran, D. Pompili, Join ask offloading and resource allocaion for muli-server mobile-edge compuing neworks, arxiv: , [9] L. Yang, J.N. Cao, Z.Y. Wang, e al., Nework aware muli-user compuaion pariioning in mobile edge clouds, Inernaional Conference on Parallel Processing (ICPP), Brisol, Unied Kingdom, [10] Y.Y. Mao, J. Zhang, K. B. Leaief, e al., A survey on mobile edge compuing: The communicaion perspecive, IEEE Communicaion Surveys & Tuorials, 2017, 19(4): [11] L.B. Huang, M.J. Neely, Uiliy opimal scheduling in energy-harvesing neworks, IEEE/ACM Transacions on Neworking, 2013, 21(4): [12] W.W. Zhang, Y.G. Wen, K. Guan, e al., Energy-opimal mobile cloud compuing under sochasic wireless channel, IEEE Transacions on Wireless Communicaions, 2013, 12(9): [13] S. Bubeck, N. Cesa-Bianchi, Regre analysis of sochasic and nonsochasic muliarmed bandi problems, Foundaions and Trends in Machine Learning, 2012, 5(1): 1-22.