Mistreatment-Resilient Distributed Caching

Transcription

1 Mitreatment-Reilient Ditributed Caching Georgio Smaragdaki, Nikolao Laoutari,2, Azer Betavro Ibrahim Matta, Ioanni Stavrakaki 2 Abtract The ditributed partitioning of autonomou, elf-aware node into cooperative group, within which carce reource could be effectively hared for the benefit of the group, i increaingly emerging a a hallmark of many newly-propoed overlay and peer-topeer application. Ditributed caching protocol in which group member cooperate to atify local requet for object i a canonical example of uch application. In recent work of our we identified mitreatment a a potentially eriou problem for node participating in uch cooperative caching arrangement. Mitreatment materialize when a node acce cot for fetching object woren a a reult of cooperation. To that end, we outlined an emulation-baed framework for the development of mitreatment-reilient ditributed elfih caching cheme. Under thi framework, a node opt to participate in the group only if it individual acce cot i le than the one achieved while in iolation. In thi paper, we argue againt the ue of uch tatic all or nothing approache which force an individual node to either join or not join a cooperative group. Intead, we advocate the ue of a moother approach, whereby the level of cooperation i tied to the benefit that a node beget from joining a group. To that end, we propoe a ditributed and eaily deployable feedback-control cheme which mitigate mitreatment. Under our propoed adaptive cheme, a node independently emulate it performance a if it were acting in a greedy local manner and then adapt it caching policy in the direction of reducing it meaured acce cot below it emulated greedy local cot. Uing control-theoretic analyi, we how that our propoed cheme converge to the minimal acce cot, and indeed outperform any tatic cheme. We alo how that our cheme reult in inignificant degradation to the performance of the caching group under typical operating cenaria. Key word: Cooperative Caching, Service Overlay Network, Peer-to-Peer Network, Control Theory, Performance Evaluation.

2 Introduction Background and Scope: Network application often rely on ditributed reource available within a cooperative grouping of node to enure calability and efficiency. A typical in many application uch a web erver farm or content ditribution network, the grouping of node i dictated by a common trategic objective, and a uch, the payoff from cooperation i aeed by the overall benefit to the group a oppoed to the benefit reaped by individual node in the group (which in thi cae are not preumed to be elfih). More recently, however, new clae of network application have emerged for which the grouping of node i more ad hoc in the ene that it i not dictated by organizational boundarie or trategic goal. Example include the variou overlay and peer-to-peer (P2P) application [2,3]. For uch application, the grouping of node i not governed by a common objective, but rather by the individual (elfih) objective of the contituent node. Under uch a etting, and a we have hown in prior work of our [4,5], it i poible for a node (or a et of node) to be mitreated in the ene that it participation in the group (while advantageou to the group) would not be advantageou to it own objective. In thi paper we how how to deign mitreatment-reilient cooperative application. Mitreatment in Ditributed Selfih Replication and Caching Sytem: A part of our recent work on mitreatment in ditributed cooperative etting [4,5], we focued on content networking application, whereby the ditributed reource being hared amongt a group of node i torage. In particular, we conidered a group of node that tore information object and make them available to their local uer a well a to remote node. A uer requet i firt received by the local node. If the requeted object i tored locally, it i returned to the requeting uer immediately, thereby incurring a minimal acce cot. Otherwie, the requeted object i earched for, and fetched from other node of the group, at a potentially higher acce cot. If the object cannot be located anywhere in the group, it i retrieved from an origin erver, which i aumed to be outide the group, thu incurring a max- A. Betavro and I. Matta are upported in part by a number of NSF award, including CNS Cybertrut Award # , CNS NeTS Award #052066, CNS ITR Award # , and EIA RI Award I. Stavrakaki i upported in part by EU IST project CASCADAS and E-NEXT. N. Laoutari i upported by a Marie Curie Outgoing International Fellowhip of the EU MOIF-CT A preliminary verion of thi work appeared in the proceeding of 2006 IFIP Networking Conference []. Computer Science Dept, Boton Univerity, Boton, Maachuett, USA. {gmaragd, nlaout, bet, matta}@c.bu.edu 2 Dept of Informatic and Telecommunication, Univerity of Athen, Athen, Greece. itavrak@di.uoa.gr 2

3 imal acce cot. Contrary to mot previou work in the field, we conidered elfih node, i.e., node that cater trictly and only to the minimization of the acce cot for their local client population (diregarding any conequence for the performance of the group a a whole). In [4,5] we etablihed the vulnerability of many ocially optimal (SO) object replication/caching cheme to mitreatment problem. A mitreated node wa defined a a node whoe acce cot under ome cooperative cheme i higher than the correponding minimal acce cot that the node can guarantee for itelf by being uncooperative. Unlike centrally deigned/controlled group where all contituent node have to abide by the ultimate goal of optimizing the ocial utility of the group, an autonomou, elfih node will not tolerate uch a mitreatment. Indeed, the emergence of uch mitreatment may caue elfih node to ecede from the replication group, reulting in evere inefficiencie for both the individual uer a well a the entire group. Proactive replication trategie uch a thoe tudied in [4] are not practical in a highly dynamic content networking etting, which i likely to be the cae for mot of the Internet overlay and P2P application we enviion for a variety of reaon: () Fluid group memberhip make it impractical for node to decide what to replicate baed on what (and where) object are replicated in the group. (2) Acce pattern a well a acce cot may be highly dynamic (due to burty network/erver load), neceitating that the election of replica and their placement be done continuouly, which i not practical. (3) Both the identification of the appropriate re-invocation time [6] and the etimation of the non-tationary demand (or equivalently, the timecale for a tationarity aumption to hold) [7] are non-trivial problem. (4) Content object may be dynamic and/or may expire, neceitating the ue of pull (i.e., on-demand caching) a oppoed to puh (i.e., pro-active replication) approache. Uing on-demand caching i the mot widely acceptable and natural olution to all of thee iue becaue it require no a priori knowledge of local/group demand pattern and, a a conequence, repond dynamically to change in thee pattern over time (e.g., introduction of new object, reduction in the popularity of older one, etc.). Therefore, in [5] we conidered the problem of Ditributed Selfih Caching (DSC), which could be een a the on-line equivalent of the Ditributed Selfih Replication (DSR) problem [4]. In DSC, we adopted an object caching model, whereby a node ued demand-driven temporary torage of object, combined with replacement. Baed on that model, we uncovered the operational characteritic of a DSC group that can give rie to mitreatment problem. And, while we argued that under tationary condition, imple parametric verion of already etablihed protocol and mechanim are capable of mitigating thee problem, we did not precribe an integrated approach for regulating thee parameter in a manner that adapt to the contantly changing con- 3

4 dition of the group (e.g., varying group ize, node capacitie, delay, and demand pattern). Mitreatment-Reilient Ditributed Selfih Caching: In thi paper, we take our work one ignificant, contructive tep further by propoing a general control-theoretic framework, which enable the parametrization of DSC protocol o a to make thee protocol reilient to mitreatment, even under the aforementioned fluid group condition. Through extenive analyi and imulation experiment, we how that our adaptive cheme not only mitigate mitreatment that may evolve in a ditributed caching group, but it alo guarantee an acce cot for each individual node that i lower than the one achieved by any tatic cheme. We alo how that the impact of thi adaptive cheme on the performance of the ditributed caching group i minimal under typical operating cenaria. Organization of the Paper: The ret of the paper i organized a follow. Section 2 ummarize related work in cooperative caching and dynamic cheme ued to optimize caching. In Section 3 we decribe our model of a ditributed caching group. In Section 4 we demontrate the caue of mitreatment in ditributed caching group. The deign of a generic feedback controller for the mitigation of mitreatment i covered in Section 5. In Section 6 we argue that a node equipped with our feedback controller achieve the minimum acce cot (compared to any other tatic cheme). In Section 7, we tudy the impact of uch a controller on the overall performance of the ditributed group. We evaluate the performance of our controller with extenive imulation in Section 8. Section 9 conclude the paper. 2 Related Work Cooperative caching [8,9] allow multiple cache to cooperate in ervicing each other requet. Monitoring and controlling the number of copie for the ame document acro different cache ha been tudied in [0] for the web and [] for wirele ad hoc network. All aforementioned tudie were concerned with the minimization of the aggregated cot of the group. Apart from our previou work [5,], we are aware of only two additional work that deal with dynamic cheme to optimize caching: According to [2] different caching algorithm may be employed dynamically to bet erve the demand; in [3], a control-theoretic approach i propoed for achieving performance differentiation in proxy cache. Our tudy differ from thee two work, by focuing on characterizing the impact of dynamic cheme on cache group, a oppoed to individual cache. Given the multi-faceted nature of the relationhip between our work and thi body of literature, and for the ake of a better expoition of our contribution, rather than enumerating thee tudie here, we dicu how 4

5 we leverage and relate to uch work throughout the paper, a appropriate. 3 Model of a Ditributed Caching Group In thi ection we preent the model of a ditributed caching group that we conider in our tudy. Let o i, i N, and v j, j n, denote the i th unit-ized object and the j th node, and let O = {o,..., o N } and V = {v,..., v n } denote the correponding et. Node v j i aumed to have torage capacity for up to C j unit-ized object, a total requet rate λ j (total number of requet per unit time, acro all object), and a demand decribed by a probability ditribution over O, p j = {p j,..., p Nj }, where p ij denote the probability of object o i being requeted by the local uer of node v j. Succeive requet are aumed to be independent and identically ditributed. 3 For our numerical example in later ection we will aume that the i th mot popular object i requeted according to a generalized power-law ditribution, i.e., with probability p i = Λ/i α (uch ditribution have been oberved in many meaured workload [8,20]). Let t l, t r, t denote the acce cot paid for fetching an object locally, remotely, or from the origin erver, repectively, where t > t r > t l ; 4 thee cot can be interpreted either a delay cot for delivering an object to the requeting uer or a bandwidth conumption cot for bringing the object from it initial location. Uer requet are erviced by the cloet node that tore the requeted object along the following chain: local node, group, origin erver. Each node employ an object admiion algorithm for toring (or not) object retrieved remotely either from the group or from the origin erver. Furthermore, each node employ a replacement algorithm for managing the content of it cache. In thi work we focu on the Leat Recently Ued (LRU) replacement algorithm but we can obtain imilar reult under other replacement algorithm, uch a Leat Frequently Ued (LFU) replacement algorithm (ee alo our previou work in [5]). 3 The Independent Reference Model (IRM) [4] i commonly ued to characterize cache acce pattern [5 8]. The impact of temporal correlation wa hown in [7,9] to be minucule, epecially under typical, Zipf-like object popularity profile. 4 The aumption that the acce cot i the ame acro all node pair in the group i made only for the ake of implifying the preentation (thoe value can alo be aumed a upper bound in our analyi). Our reult can be adapted eaily to accommodate arbitrary inter-node ditance. 5

6 4 Mitreatment in Ditributed Caching Group The examination of the operational characteritic of a group of node involved in a ditributed caching olution enabled u to identify two key culprit for the emergence of mitreatment phenomena [5]: () the ue of a common caching cheme acro all the node of the group, irrepectively of the particular capabilitie and characteritic of each individual node, and (2) the mutual tate interaction between replacement algorithm running on different node. 4. Mitreatment Due to Common Scheme The common caching cheme problem i a very generic vehicle for the manifetation of mitreatment. To undertand it, one ha firt to oberve that mot of the work on cooperative caching ha hinged on the fundamental aumption that all node in a cooperating group adopt a common caching cheme. We ue the word cheme to refer to the combination of: (i) the employed replacement algorithm, (ii) the employed requet redirection algorithm, and (iii) the employed object admiion algorithm. Cae (i) and (ii) are more or le elf-explanatory. Cae (iii) refer to the deciion of whether to cache locally an incoming object after a local mi. The problem here i that the adoption of a common cheme can be beneficial to ome of the node of a group, but harmful to other, particularly to node that have pecial characteritic that make them outlier. A imple cae of an outlier, i a node that i ituated further away from the center of the group, where mot node lie. Here ditance may have a topological/affine meaning (e.g., number of hop, or propagation delay), or it may relate to dynamic performance characteritic (e.g., variable throughput or latencie due to load condition on network link or erver node). Such an outlier node cannot rely on the other node for fetching object at a mall acce cot, and thu prefer to keep local copie of all incoming object. The ret of the node, however, a long a they are cloe enough to each other, prefer not to cache local copie of incoming object that already exit elewhere in the group. Since uch object can be fetched from remote node at a mall acce cot, it i better to preerve the local torage for keeping object that do not exit in the group ince otherwie, they would have to be fetched from the origin erver at a high acce cot. Enforcing a common cheme under uch a etting i bound to mitreat either the outlier node or the ret of the group. Conider the group depicted in Figure in which n node are clutered together, meaning that they are very cloe to each other (t r t l 0), while there alo a ingle outlier node at ditance t r from the cluter. The n node would naturally employ a Single Copy (SC) cheme, i.e., a cheme where there can be at mot one copy 6

7 Single Copy cheme n node t r 0 / t r outlier node local requet cache cluter Fig.. An example of a group compoed of a cluter of n node and a unique outlier. virtual cache Fig. 2. Block diagram of a node equipped with a virtual cache. of each ditinct object in the group (e.g. LRU-SC [0]) in order to capitalize on their mall remote acce cot. From the previou dicuion it hould be clear that the bet cheme for the outlier node would depend on t r. If t r t r, the outlier hould obviouly follow LRU-SC and avoid duplicating object that already exit elewhere in the group. If t r t r, then the outlier hould follow a Multiple Copy (MC) cheme, i.e., a cheme where there can be multiple copie of the ame object at different node an example of an MC cheme i the LRU-MC. Under LRU-MC, if a node retrieve an object from a remote node in the group (or the origin erver), then it tore a copy of it locally replacing an exiting object if the cache i full, according to the LRU policy. 4.2 Mitreatment Due to State Interaction The tate interaction problem take place through the o-called remote hit. Conider node v, u and object o. A requet for object o iued by a uer of v that cannot be erved at v but could be erved at u i aid to have incurred a local mi at v, but a remote hit at u. Conider now the implication of the remote hit at u. If u doe not dicriminate between hit due to local requet and hit due to remote requet 5, then the remote hit for object o will affect the tate of the replacement algorithm in effect at u. If u i employing LRU replacement, then o will be brought to the top of the LRU lit. If u employ LFU replacement, then the frequency of o will be increaed, and o on with other replacement algorithm [22]. If the frequency of remote hit i ufficiently high, e.g., becaue v ha a much higher local requet rate and thu end an intene mi-tream to u, then there could be performance implication for the u: u cache may get invaded by object that follow v demand, thereby depriving the uer of u from valuable torage pace for caching their own object. Thi can lead to the mitreatment of u, whoe cache i effectively hijacked by v. 5 The Internet Cache Protocol (ICP)/Squid web cache [2] and other ytem (e.g., Akamai Content Ditribution Network, IBM Olympic Server Architecture), by default, do not dicriminate between local and remote requet. 7

8 5 Toward Mitreatment-Reilient Caching From the expoition o far, it hould be clear that there exit ituation under which an inappropriate, or enforced, cheme may mitreat ome of the node. While we have focued on detecting and analyzing two caue of mitreatment which appear to be important (namely, due to the adoption of a common cache management cheme and cache tate interaction), it hould be evident that mitreatment may well arie through other caue. For example, we have not invetigated the poibility of mitreatment due to requet re-routing [23], not to mention that there are vatly more parameter et and combination of cheme that cannot all be invetigated exhautively. 5. Deign Dicipline To addre the above challenge, we firt ketch a general framework for deigning mitreatment-reilient cheme. We then apply thi general framework to the two type of mitreatment that we have conidered in thi work. We target open ytem in which group etting (e.g., number of node, ditance, demand pattern) change dynamically. In uch ytem it i not poible to addre the mitreatment iue with predefined, fixed deign. Intead, we believe that node hould adjut their cheme dynamically o a to avoid or repond to mitreatment if and when it emerge. To achieve thi goal we argue that the following three requirement are neceary. Detection Mechanim: Thi requirement i obviou but not trivially achievable when operating in a dynamic environment. How can a node realize that it i being mitreated? In our previou work on replication [4], a node compared it acce cot under a given replication cheme with the guaranteed maximal acce cot obtained through greedy local (GL) replication. Thi gave the node a reference point for a mitreatment tet. In that game theoretic framework, we conidered node that had a priori knowledge of their demand pattern, thu could eaily compute their GL cot threhold. In caching, however, demand pattern (even local one) are not known a priori, nor are they tationary. Thu in our DSC etting, the node have to etimate and update their threhold in an on-line manner. We believe that a promiing approach for thi i emulation. Figure 2 depict a node equipped with an additional virtual cache, alongide it real cache that hold it object. The virtual cache doe not hold actual object, but rather object identifier 6. It i ued 6 Since the virtual cache tore object identifier a oppoed to the actual object, the memory it ue i inignificant (compared to the memory required to tore the actual object). The proceing cot i alo trivial (i.e., incurring O() update cot for each requet under both LRU and LFU replacement policie). 8

9 for emulating the cache content and the acce cot under a cheme different from the one being currently employed by the node to manage it real cache under the ame requet equence (notice that the input local requet tream i copied to both cache). The baic idea i that the virtual cache can be ued for emulating the threhold cot that the node can guarantee for itelf by employing a greedy (non-cooperative) cheme. Mitigation Mechanim: Thi requirement enure that a node ha a mechanim that allow it to react to mitreatment a mechanim via which it i able to repond to the onet of mitreatment. In the context of the common cheme problem, the outlier hould adjut it caching behavior according to it ditance from the group. For thi purpoe, we introduce the LRU(q)-cheme, under which, object that are fetched from the group are cached locally only with probability q; q will hereafter be referred to a the reliance parameter, capturing the amount of reliance that the node put into being able to fetch object efficiently from other node. In the context of the tate interaction problem, one may define an interaction parameter p and the correponding LRU(p ) cheme, in which a remote hit i allowed to affect the local tate with probability p, wherea it i denied uch acce with probability (-p ). A it will be demontrated later on, node may avoid mitreatment by electing appropriate value for thee parameter according to the current operating condition. Control Scheme: In addition to the availability of a mitreatment mitigation mechanim (e.g., LRU(q)), there need to be a programmatic cheme for adapting the control variable() of that mechanim (e.g., how to et the value of q). Since the optimal etting of thee control variable depend heavily on a multitude of other time-varying parameter of the DSC ytem (e.g., group ize, torage capacitie, demand pattern, ditance), it i clear that there cannot be a imple (tatic) rule-of-thumb for optimally etting the control variable of the mitigation mechanim. To that end, dynamic feedback-baed control become an attractive option. To make the previou dicuion more concrete, we now focu on the common cheme problem and demontrate a mitreatment-reilient olution baed on the previou three principle requirement. A imilar olution can be developed for the tate interaction problem. 5.2 Reilience to Common-Scheme-Induced Mitreatment We tart with a imple hard-witch olution that allow a node to change operating parameter by electing between two alternative cheme. Thi can be achieved by uing the virtual cache for emulating the LRU(q =) cheme, 9

10 PSfrag replacement PSfrag replacement n=4, C=250, N=000, a=0.9, tl=0, t=2 n=4, C=250, N=000, a=0.9, tl=0, t= acce cot outlier virtual cache outlier (LRU(0)) t r normalized acce cot outlier (LRU(0)) outlier (LRU(0.)) outlier (LRU(0.5)) outlier (LRU()) t r Fig. 3. Simulation reult on the effect of the remote acce cot t r on the acce cot of the outlier node under the virtual cache and LRU(0) cheme. Fig. 4. Simulation reult on the effect of the remote acce cot t r on the normalized (by the virtual cot) acce cot of the outlier node under different LRU(q) cheme. capturing the cae that the outlier node doe not put any trut on the remote node for fetching object and, thu, keep copie of all incoming object after local mie. Equipped with uch a device, the outlier can calculate a running etimate of it threhold cot baed on the object it emulate a preent in the virtual cache. 7 By comparing the acce cot of ticking to the current cheme to the acce cot obtained through the emulated cheme, the outlier can decide which one of the two cheme i more appropriate. For example, it may tranit between the two extreme LRU(q) cheme the LRU(q = 0) cheme and the LRU(q = ) cheme. Figure 3 how that the relative performance ranking of the two cheme depend on the ditance from the group t r and that there i a value of t r for which the ranking change. A more efficient deign can be obtained by manipulating the reliance parameter q at a finer cale. Indeed, there are ituation in which intermediate value of q, 0 < q <, are better than either q = 0 and q = (ee the LRU(0.) and LRU(0.5) curve in Figure 4). Conider two different value of the reliance parameter q and q 2 uch that q < q 2. Figure 5 illutrate a typical behavior of the average object acce cot under q and q 2 a a function of the ditance t r of the outlier node from it cooperative cluter. A dicued in the previou ection, q (q 2 ) will perform better with mall (large) t r. In the remainder of thi ection, we preent and evaluate a Proportional-Integral-Differential (PID) controller for controlling the value of q. Thi type of controller i known for it good convergence and tability propertie (converge to a target value with zero error) [24,25]. 7 The outlier can include in the emulation the cot of remote fetche that would reult from mie in the emulated cache content; thi would give it the exact acce cot under the emulated cheme. A impler approach would be to replace the acce cot of remote fetche by that from the origin erver and thu reduce the internode query traffic; thi would give it an upper bound on the acce cot under the emulated cheme. 0

11 PSfrag replacement average acce cot Virtual Cache cot q q 2 Fig. 5. Repreentative behavior of average object acce cot a a function of the reliance parameter and ditance of the outlier from the cluter, q < q 2. A node equipped with the PID controller maintain an Exponential Weighted Moving Average (EWMA) of the object acce cot (cot virtual ) for the emulated greedy cheme. The virtual cache emulate an LRU(q = )-cheme in which no remote fetche are conidered, o a to avoid doubling the number of querie ent to remote node. Let cot q denote the EWMA of the object acce cot of the employed LRU(q)-cheme in the actual cache of the node. Let dit denote the difference between the virtual acce cot and the actual acce cot, and let diff be the difference between two conecutive value of dit. The PID controller adapt q proportionally to the magnitude of diff; if the magnitude of diff i mall then conecutive obervation of the performance of the ytem are very cloe, thu the value of q hould be changed moothly until the diff i zeroed. On the other hand, if the magnitude of diff i high, then the value of q ha to be updated ignificantly. A peudo-code for thi proce i provided in Algorithm. In the forthcoming ection (Section 6), we argue that the acce cot of a node equipped with thi controller converge to a value which i lower than that of any cheme that employ a fixed q. We alo provide an etimation of the converged value a a function of controller parameter and other ytem characteritic. Our algorithm ha two parameter. The firt one, denoted by α c, i the gain of the controller, which determine the rate with which the value of q i changed in a ingle control period. The econd parameter, denoted by β c, i the update weight of the difference in the cot that i oberved in the lat (two) update. A methodology on how to tune thee parameter and how enitive i the performance of the controller to thee parameter will be preented in Section 8 (evaluation ection). t r 5.3 Reilience to State-Interaction-Induced Mitreatment Immunizing a node againt mitreatment that emerge from tate interaction could be imilarly achieved. The interaction parameter p can be con-

12 Algorithm : mitigation of mitreatment dit(t) = cot virtual (t) cot q (t) dit(t ) = cot virtual (t ) cot q (t ) diff(t) = dit(t) dit(t ) σ = ign(diff(t)) if q(t) q(t ) then q(t + ) q(t) + σ α c diff(t) + σ β c diff(t) diff(t ) ele q(t + ) q(t) σ α c diff(t) σ β c diff(t) diff(t ) trolled uing cheme imilar to thoe we conidered above for the reliance parameter q. It i important to note that one may argue for iolationim (by permanently etting p = 0) a a imple approach to avoid tate-interactioninduced mitreatment. Thi i not a viable olution. Specifically, by adopting an LRU(p = 0) approach, a node i depriving itelf from the opportunity of uing mi tream from other node to improve the accuracy of LRU-baed cache/no-cache deciion (auming a uniform popularity profile for group member). To conclude thi ection, we note that the approache we preented above for mitreatment reilience may be viewed a paive or end-to-end in the ene that a node infer the onet of mitreatment implicitly by monitoring it utility function. A we alluded at the outet of thi paper, for the emerging cla of network application for which grouping of node i ad hoc (i.e., not dictated by organizational boundarie or trategic goal), thi might be the only realitic olution. In particular, to undertand exactly how and exactly why mitreatment i taking place would require the ue of proactive meaure (e.g., monitoring/policing group member behavior, meauring ditance with ping, etc.), which would require group member to ubcribe to ome common ervice or to trut ome common authority both of which are not conitent with the autonomou nature (and the mutual ditrut) of participating node. 6 Convergence of the Controller In thi ection we argue that the acce cot of a node equipped with our adaptive mechanim converge to a value that i lower than the one under any other tatic cheme, and we analytically etimate thi value. We conider the cenario with the outlier node that wa preented in Section 4.. Claim: When Algorithm i ued for controlling the probability q of caching an incoming object at the outlier node, then it average acce cot will converge to a ingle value which i upper-bounded by the minimum average cot 2

13 Virtual Cache cot q=0 PSfrag replacement average acce cot c deired q= c 0 t l χ ψ θ t t r optimal value 0 for q: Region A Region B Fig. 6. Repreentative behavior of tatic cheme (LRU(q = 0), LRU(q = )) and the deired behavior of the controller. of any tatic cheme (i.e. a cheme employing a fixed q). Jutification: A illutrated in Figure 5, the average acce cot of the outlier increae linearly with it ditance from the group (t r). When t r t l, the optimal value of q 0. When t r t, the optimal value for q. We would like our controller to exhibit thi behavior while tuning the value of q. We can approximate the deired average acce cot of the outlier a a linear function of q. In Figure 6, we illutrate a repreentative behavior of the average acce cot of two tatic cheme, LRU(q = 0) and LRU(q = ). We alo illutrate the deired behavior of the controller a a linear function with lope θ. It i clear that the average acce cot of the controller will be a lower bound on the average acce cot of every tatic cheme, a θ < ψ and although χ < θ the average acce cot of LRU(q = ) will converge to the acce cot of the controller only when t r = t, becaue the initial acce cot for LRU(q = ) i higher than that of LRU(q = 0). In our analyi we aume that the cache capacity C and the kewne of the demand α are contant; their exact value affect only the lope. We define two operating region for the controller: Region A and Region B, a denoted in Figure 6. In Region A, the cot of the outlier under the LRU(q = )- cheme, i alway higher than the one under the LRU(q = 0)-cheme and vice vera for Region B. We now proceed to analyze the behavior of our adaptive cheme in thee two region. We can deign the controller uch that the control update rate i higher that the rate at which t r change. Thu, for the purpoe of our analyi, let u 3

14 Sfrag replacement θ αc + βc - DIF F αc + βc Q µ PSfrag replacement θ + + c 0 COST q αc + βc DIF F αc + βc - Q µ + + c 0 COST q c σ DIST + COST virtual c σ DIST + COST virtual Sfrag replacement θ αc + βc - αc + βc Q + (a) µ PSfrag replacement θ + c COST q αc + βc αc + βc - Q + (b) µ + c COST q c 0 σ DIF F DIST + COST virtual c 0 σ DIF F DIST + COST virtual (c) (d) Fig. 7. Laplace-Tranform of the control loop for cae: (a) A; (b) A2; (c) B; (d) B2. aume that t r i fixed for a hort period that include few control update. In Region A, we conider two cae: cae A: If q(t) q(t ) then cot(t) cot(t ), dit(t) dit(t ) and a a reult diff(t) 0, thu our controller witche coure and decreae the value of q at the adaptation point t +. cae A2: If q(t) < q(t ) then cot(t) < cot(t ), dit(t) > dit(t ) and a a reult diff(t) > 0, thu the controller will keep decreaing the value of q at the adaptation point t +. In both cae, our adaptive cheme examine locally the poible value of q and update it value toward the direction that reduce the average acce cot. In Region B, we conider the ame two cae: cae B: If q(t) q(t ) then cot(t) cot(t ), dit(t) dit(t ) and a a reult diff(t) 0, thu the controller will keep increaing the value of q at the adaptation point t +. 4

15 cae B2: If q(t) < q(t ) then cot(t) > cot(t ), dit(t) < dit(t ) and a a reult diff(t) < 0, thu the controller will change coure and increae the value of q at the adaptation point t +. A in the previou cae, our adaptive cheme update the value of q in the direction of reducing the acce cot of the outlier. We follow a control-theoretic approach to how the convergence propertie of our controller. We tart by providing the proof for cae A: From Algorithm, we derive the following continuou-time equation: dit(t) t = diff(t) () where dit(t) = cot virtual (t) cot q (t) (2) The value of q i updated a follow: q(t) t = α c diff(t) + β c diff(t) t (3) We can approximate the average acce cot under our adaptive cheme a follow: cot q (t) c 0 + µ q(t) (4) where µ = tan(θ) and c 0 i the cot of the outlier for q = 0 and t r = t l. Next, we take the Laplace-tranform of Equation (), (2), (3) and (4), and draw the block diagram that decribe the flow of the ignal (Figure 7(a)). We can now derive the relation between DIF F and COST q (in the -domain): c 0 + DIF F () αc + β c µ = COST q () 5

16 Furthermore, auming that cot virtual i contant 8, we have: ( ) DIF F () = COST virtual COST q () After ome algebraic manipulation we have: COST q () = c 0 + (α c + β c ) µ COST virtual ( + (α c + β c ) µ) From the above equation we can conclude that the ytem i table and overdamped ince the pole = β c ( µ + α c) i negative [24,25]. In order to find the teady-tate value of the average cot, we ue the Final Value Theorem [24,25]: cot q ( ) = lim COST q () = c α c µ COST virtual + α c µ If we can calculate the deired lope µ, we can etimate the optimal value for α c. In principle, for mall c 0, the maller the value of α c < the lower the acce cot i, even when the value of µ i not known in advance. We how that thi condition on α c hold for all other cae a well. In cae A2: We follow the ame analyi that wa provided for the A cae, but with a new expreion in place of Equation (3): q(t) t = α c diff(t) β c diff(t) t The block diagram for thi cae i illutrated in Figure 7(b). It i eay to how that the teady-tate value of the average acce cot i given by: cot q ( ) = c 0 α c µ COST virtual α c µ In cae B: We follow the ame analyi that wa provided for the A cae, but with a new expreion for Equation (4): cot q (t) c µ ( q(t)) 8 Note that the value of the virtual cache cot i independent of t r. 6

17 where c i the cot of the outlier for q = and t r = t. Furthermore, note that c c 0 + µ. The block diagram for thi cae i illutrated in Figure 7(c). After ome algebraic manipulation, we can how that the teady-tate value of the average acce cot i given by: cot q ( ) = c 0 + α c µ COST virtual + α c µ Note that thi teady-tate value i the ame a in cae A. In cae B2: We follow the ame analyi that wa provided for the A2 cae, but with a new expreion for the Equation (4). The block diagram for thi cae i illutrated in Figure 7(d). Following the ame analyi, it i eay to how that the teadytate value of the average acce cot i given by: cot q ( ) = c 0 α c µ COST virtual α c µ Note that thi teady-tate value i the ame a for cae A2. 7 The Effect of Individual Controller on the Overall Performance of the Group In thi ection we turn our attention to the performance implication reulting from the ue of individual-cot minimizing controller at different node. In particular, we look at the impact on the (global) group performance by comparing the aggregated teady-tate acce cot of the ditributed caching group (AAC P ID ) when all contituent node are equipped with the PID controller decribed in Section 5.2 with the correponding cot of the ditributed caching group (AAC) when node are not equipped with the PID controller. We refer to the ratio AAC P ID /AAC a the controller impact ratio (CIR) on group performance. 7. Analyi We conider a etting imilar to the one decribed in Section 4., but with n k node clutered together and k outlier node at ditance t r from the cluter. Node u j, j n k, are cluter node wherea node u j, n k + j n, are outlier node. Each node ha torage capacity up to C 7

18 unit-ized object, and ha the ame requet rate λ a all other node in the caching group. In our analyi, we aume that the i th mot-popular object ha a requet probability drawn from a generalized power-law with kewne a, i.e., p i = Λ/i a, i N. Cloed-form analytic model for prediction of cache performance do not exit (ee [26] for the development of numerical model). A imulation tudy i more tractable, but given the ize of the parameter pace we are conidering, it i not clear that imulation tudie would be ueful. Thu, the alternative we adopt in thi paper relie on the obervation that the content of a cache could be een a the reult of uperimpoing an offline-optimal replication trategy with the noie caued by replacement error. Uing thi approach we are able to draw ome baic qualitative concluion for caching by tudying replication [27]. Under thi framework, the clutered group of node can be abtracted by a ingle cache with torage capacity of (n k)c unit-ized object, wherea each outlier node repreent a ingle cache with torage capacity of C unit-ized object. Auming that the ize of the cluter i larger than the population of the outlier (i.e., n > 2k), the (n k)c mot popular object will be tored in the clutered node, thi would occur under LRU (q = 0) or any other SCcheme (which aume full collaboration in caching deciion, e.g., a with the cae of hah-baed caching). Object that are le popular namely object with id (n k)c +,.., nc would be tored in the outlier node. Without lo of generality, we aume that object (j )C +,.., jc, will be tored in the node u j, n k + j n. The aggregated acce cot of the ditributed caching group i the ummation of the individual cot of the clutered and the outlier node, and i equal to: n k AAC = cot j + = n j= j=n k+ n k (n k)c j= + p i t l + cot j nc p i t r + i= i=(n k)c+ (n k)c n nc j=n k+ i= p i t r + i=(n k)c+ N i=nc+ p i t ij p i t jc i=(j )C+ p i t ij + N i=nc+ p i t where t ij, (n k)c+ i nc, n k+ j n i the cot of fetching object o i by outlier node u j from a different outlier node. In the wort cae cenario, thi object will be tored in another outlier which cannot be within a ditance greater than 2t r. Thu, for the aforementioned value of i, j, t ij min(2t r, t ). For the ret of our analyi, we will aume that t r < 0.5t, in order to tudy a more realitic etting, in which outlier are not extremely far from the cluter. 8

19 Subtituting in the above equation, with t ij 2t r and t l = 0, we get: n k (n k)c nc N AAC p i t l + p i t r + p i t j= i= i=(n k)c+ i=nc+ (n k)c n nc jc N + p i t r + p i 2t r p i 2t r + p i t j=n k+ i= i=(n k)c+ i=(j )C+ i=nc+ nc N = (n k) p i t r + p i t i=(n k)c+ i=nc+ (n k)c nc N + k p i t r + 2 p i t r + p i t i= i=(n k)c+ i=nc+ n jc 2 p i t r j=n k+ i=(j )C+ nc N = (n k) p i t r + p i t i=(n k)c+ i=nc+ (n k)c nc N + k p i t r + 2 p i t r + p i t i= i=(n k)c+ i=nc+ nc 2 p i t r i=(n k)c+ = Λ[(n k) (( H (a) nc H (a) ) (n k)c t r + ( H (a) N H (a) ) ) nc t + k (( H (a) (n k)c ) H(a) 0 t r + 2 ( H (a) nc H (a) ) (n k)c t r + ( H (a) N H (a) ) ) nc t 2 ( H (a) nc H (a) (n k)c) t r ] = Λ [ nt H (a) N (n 2)t rh (a) (n k)c + (n + k 2) t rh (a) nc nt H (a) nc ] The generalized harmonic number H (a) C expreion [28]: can be approximated by it integral H (a) C = C i= C i a l dl = C a a a where H (a) 0 = 0. 9

20 Uing thi approximation we obtain: AAC Λ [( ((n + k 2)t a r nt ) n a (n 2)t r(n k) a) ] C a + nt N a kt r (5) In the preence of a controller at each node, the analyi i more involved. In particular, we note that the outlier node are located at ditance t r from the cluter, and a a conequence, the cot to fetch a popular object from the group become much greater than the correponding cot if the object wa tored locally. Therefore, ome of the object (the mot popular one) may end up being cached locally, which i preciely the goal of the PID controller of Section 5.2. Thu, although the (n k)c mot-popular object would till be tored in the clutered node, the object placement in the outlier node may reult in having duplicate (local) copie of thee object at the outlier a well. The aggregated acce cot for thi etting i equal to: n k n AAC P ID = cot j + cot j j= j=n k+ n k (n k)c N = p i t l + p i t j= i= i=(n k)c+ (n k)c n C N + p i t l + p i t r + p i t j=n k+ i= i=c+ i=(n k)c+ (n k)c N N = (n k) p i t + k p i t r + p i t i=(n k)c+ i=c+ i=(n k)c+ = Λ [ (n k) ( H (a) N H (a) ) (n k)c t + k (( H (a) (n k)c ) H(a) C t r + ( H (a) N = Λ [ nt H (a) N + (kt r nt ) H (a) (n k)c kt rh (a) ] C Λ a [( (kt r nt )(n k) a kt r) C a + nt N a] H (a) ) )] (n k)c t Uing the harmonic approximation given in Equation (5) and ubtituting in the above, we can lower-bound the controller impact ratio a follow. ((kt r nt )(n k) a kt CIR r) C a + nt N a (((n + k 2)t r nt ) n a (n 2)t r(n k) a ) C a + nt N a kt r ( ) ( ) a (kt r nt )( k n ) a k C n t r N + t = (( ( + k ) ) ( ) a 2 n n )t r t n a (n 2)t r( k C n ) a N + t k nn t r 20

21 Figure 8 how how the above lower-bound on CIR relate to the kewne of the power-law demand under a range of value for the relative population of the outlier. 7.2 Obervation The above bound and illutration allow u to make three important obervation. Firt, given ome relative torage capacity C/N, the controller impact ratio i contant, and a one would expect, it i more pronounced for maller value of C/N. Second, under kewed demand (a ), the impact of the controller on the performance of the caching group i mall. Thi could be explained by noting that for workload with le kewed (i.e., more uniform) demand ditribution the marginal utility of the outlier cache to the caching group become higher, reulting in a more pronounced impact ratio. However, for highly-kewed demand, the collective cache of the caching group (excluding thoe at the outlier) would be able to accommodate mot of the working et in the requet tream, rendering the impact ratio inignificant. The aforementioned explanation i alo upported by our numerical reult. Figure 9 depict the cluter node cot increae ratio and the lower bound of outlier node cot reduction ratio (cot of a node equipped with the PID controller compared to it cot when it i not equipped with the PID controller). Under kewed demand, the cluter node cot increae in a linear fahion, but the outlier node cot decreae exponentially fat. Thi i an important obervation becaue mot reference tream characterized in the literature exhibit highly-kewed demand ditribution (ee [8] for example). Third, when the relative population of the outlier i mall (k/n 0), the controller impact on the performance of the caching group i inignificant 9. Thi obervation i important becaue in a typical etting, one would expect that the clutering of node in a caching group would reult in the preence of a mall ratio of outlier node (k/n). Moreover, in a typical etting, a large k/n ratio may ignify an anomalou condition (e.g., the onet of congetion that effectively partition the cluter), which may trigger a regrouping. To ummarize, under typically kewed demand ditribution and for wellformed caching group featuring a mall proportion of outlier, our adaptive cheme guarantee minimal acce cot for each individual elfih node, with inignificant degradation to the group performance. 9 Both cluter and outlier node cot increae in a linear fahion with k/n (ee Figure 9). 2

22 replacement θ n = 00, C = 50, N = 0000, t l = 0, t = 2, t r = 0.3t lower bound of CIR k n a Fig. 8. The behavior of the lower bound of CIR, under power-law demand and different value for the relative population of outlier. θ PSfrag replacement n = 00, C = 50, N = 0000, t l = 0, t = 2, t r = 0.3t cluter node cot increae ratio lower bound of outlier node cot reduction ratio k n a Fig. 9. The behavior of the cluter node cot increae ratio and the lower bound of outlier node cot reduction ratio, under power-law demand and different value for the relative population of the outlier. 8 Evaluation Of the Controller In order to evaluate our adaptive cheme, we compare it teady-tate average acce cot to the correponding cot of one of the two extreme tatic cheme (LRU(q = 0) or LRU(q = )), correponding to full- or no-cooperation, repectively. To that end, we define the following performance metric: minimum cot reduction (%) = 00 cot tatic cot adaptive (6) cot tatic where cot adaptive i the acce cot of our adaptive mechanim, and cot tatic i the minimum cot of the two tatic cheme: cot tatic = min( cot(lru(q = 22

23 0)), cot(lru(q = )) ). Thi metric capture the minimum additional benefit that our adaptive cheme ha over the previou tatic cheme. To capture the maximum additional benefit of our adaptive cheme (the optimitic cae), we imilarly define maximum cot reduction a in Equation (6), where cot tatic = max( cot(lru(q = 0)), cot(lru(q = )) ). 8. Experimental Setting In thi ubection we provide a detailed decription of the deign and parameterization of the experiment ued to evaluate the performance of our adaptive cheme. We conider the outlier cenario decribed in Section 4.. Motivated by a realitic cenario from a wirele caching application [], we capture the dynamic in t r by having the outlier move according to the Modified Random Waypoint Model (modified RWP) [29]. 0 Under the modified RWP model, the outlier (mobile) node pick an initial ditance D 0 according to the ditribution: FD(r) 0 = 3 ( ) r r3 2R max 3R 2 max for the firt time period X 0. R max i the maximum ditance that a mobile node can travel in a given choen direction. Moreover, the node pick a velocity V 0 uniformly from [V min, V max ], where V min and V max denote the minimum and maximum peed of the outlier node, repectively. For the following time period X i, i > 0, the outlier node pick ditance D i according to the ditribution: ( r F D (r) = R max ) 2 and peed according to the ditribution: F V (v) = v2 V 2 min V 2 max V 2 min Upon reaching the randomly choen detination point, the outlier node paue for a time period P, and the proce repeat itelf until the end of the imulation. 0 Thi recent verion fixe the non-tationarity of the original model, and thu provide better tatitical confidence. 23

24 Modified RWP parameter Outlier Speed mean(t r) tdv(t r) low: V max = du/tu 0.67 du 0.49 du moderate: V max = 5 du/tu 0.62 du 0.45 du high: V max = 20 du/tu 0.65 du 0.46 du Table The characteritic of the Modified RWP model ued in our imulation tudy. In order to map the ditance (determined by thi mobility model) to an aociated cot in the mobile environment, we ue the energy cot function which i proportional to the quare of that ditance [30], i.e. the intantaneou outlier cot i given by t r(t) = ( ) r(t) 2 t R max, where r(t) i the current ditance of the outlier from other group member. Unle otherwie pecified, for the modified RWP mobility model, we et V min and P to zero, and the dimenion of the pace inide which the outlier node move are given by a circle of radiu R max =000 ditance unit (du) centered around other (non-mobile) node in the cooperative group. We alo take the time between ucceive requet for object a our baic time unit (tu). The rate at which our feedback controller update q hould depend only on the rate of change of the acce cot of local requet. Given that the latter i determined by the rate of change in ditance traveled by the outlier, the control update period mut be et taking thi ditance change into conideration. The average ditance that the mobile object travel i given by: E[D] = D i D i pmf(d i ) = D i D i (F D (D i+ ) F D (D i )) = 2/3 R max Following the ame analyi it can be hown that E[V ] = 2/3 (V max V min ) and a a reult the average travel time of the outlier i E[X] = E[D] E[V ] = R max V max V min. Under the aumption that each node in the group (of n node) generate on average the ame number of requet, the embedded controller in a node update q at leat every E[X]/n local requet. 8.2 Experimental Reult For the aforementioned etting, we conider a group of n = 4 node compoed of a cluter of 3 node and a unique outlier. The ize of the object univere wa et to N = 000. Each node i aumed to have the ame torage capacity C and the ame requet rate a all the other node in the group. All the 24

25 Controller parameter Outlier Speed update control period α c, C=250/50/50 β c low: V max = du/tu 250 local requet 0./0./ 0.0 moderate: V max = 5 du/tu 50 local requet 0./0./ 0.0 high: V max = 20 du/tu 3 local requet 0./0./ 0.0 Table 2 The characteritic of the PID Controller ued in our imulation tudy. minimum cot reduction (%) θ PSfrag replacement C=250 C=50 C=50 n=4, N=000, tl=0, t=2, Vmax= α (kewne of demand) maximum cot reduction (%) θ PSfrag replacement C=250 C=50 C=50 n=4, N=000, tl=0, t=2, Vmax= α (kewne of demand) Fig. 0. Simulation reult on the cot reduction that i achieved uing our adaptive mechanim, (left): The minimum cot reduction, (right): The maximum cot reduction. node in the caching group have the ame Zipf (with parameter a) object popularity profile. We conider three different value for C uch that the total capacity of the caching group, nc, i maller than or equal to the ize of the object univere. We alo conider three different value of peed for the outlier (mobile) node: low, moderate, and high. We generate 00, 000 requet uniformly initiated from the peer in the group. For our adaptive controller, we et α c = 0. and β c = 0.0. To match the aggreivene of LRU under mall cache ize (C=50), we et α c =. Table and 2 ummarize the parameter of the modified Random Waypoint Model and the controller ued in our experiment repectively. We repeated the experiment 0 time under each etting (of cache ize, acce demand kewne and maximum peed of the outlier) under the adaptive and the tatic cheme uing the ame random eed for a ingle run of the experiment. In all experiment we report the virtual cache cot a well a the actual acce cot averaged uing EWMA where the weight of the hitory w wa et to Figure 0 ummarize reult (along with 95 th -percentile confidence interval) we obtained under different cache ize, demand kewne, and movement peed V max = ditance unit/time unit (imilar reult are oberved under higher peed a well). By employing our adaptive cheme, the outlier achieve a maximum cot reduction that can be up to 60% under kewed demand. The depicted profile 25