Elastic Caching. Anupam Gupta Ravishankar Krishnaswamy Amit Kumar Debmalya Panigrahi. Abstract

Transcription

1 Elastic Caching Anupam Gupta Ravishankar Krishnaswamy Amit Kumar Debmalya Panigrahi Abstract Motivated by applications in cloud computing, we study the classical online caching problem for a cache of variable size, where the algorithm pays a maintenance cost that monotonically increases with cache size. This captures not only the classical setting of a fixed cache size, which corresponds to a maintenance cost of 0 for a cache of size at most k and otherwise, but also other natural settings in the context of cloud computing such as a concave rental cost on cache size. We call this the elastic caching problem. Our results are: (a) a randomized algorithm with a competitive ratio of O(log n) for maintenance cost that is an arbitrary function of cache size, (b) a deterministic algorithm with a competitive ratio of 2 for concave, or more generally submodular maintenance costs, (c) a deterministic n-competitive algorithm when the cost function is any monotone non-negative set function, and (d) a randomized constant-factor approximation algorithm for the offline version of the problem. Our algorithms are based on a configuration LP formulation of the problem, for which our main technical contribution is to maintain online a feasible fractional solution that can be converted to an integer solution using existing rounding techniques. 1 Introduction In the classical caching problem, we are given a cache of fixed size, and the goal is to devise a page eviction strategy such that the number of page faults is minimized. This model is well-suited to traditional models of computation where the fast cache memory resides in the processor or in the RAM, and the size of the cache remains more-orless fixed throughout the lifetime of a program. Managing a cache (i.e., deciding which pages to keep in the cache and which to evict) is a natural online problem. In fact, it was one of the first problems to be studied extensively in the online computation model, and led to the development of the widely popular competitive analysis framework and its extensions and refinements. In this competitive analysis framework, the basic caching problem is very well-understood: given a cache which can hold up This research was done under the auspices of the Indo-US Virtual Networked Joint Center on Algorithms under Uncertainty. Carnegie Mellon University, Pittsburgh PA anupamg@cs.cmu.edu. Supported in part by NSF awards CCF , CCF , and CCF Microsoft Research India. rakri@microsoft.com Department of Computer Science and Engg., IIT Delhi. amitk@cse.iitd.ac.in Duke University. debmalya@cs.duke.edu. Supported in part by NSF grants CCF and CCF , and an NSF CAREER Award CCF to k pages, we know deterministic k-competitive and randomized H k -competitive algorithms, and also that these results are best possible [16, 14]. Subsequently, there has been much work in understanding more general caching problems where pages can have non-uniform eviction costs (called weighted paging), and/or non-uniform sizes (called generalized caching). These problems are also by now well understood, with k-competitive deterministic algorithms and Θ(log k)-competitive randomized algorithms [18, 11, 5, 3] known. The latter results are based on recent developments using the online primaldual framework. In the past few years, however, the model of computation have undergone a paradigm shift. Vast amounts of computational and storage resources are available on the cloud, and one can rent varying amounts of these resources. Some well-known examples include the Amazon Web Service (AWS), Google Cloud and Microsoft Azure services. To accompany this new model of computation, the prevalent model of caching has also changed: essentially unlimited amounts of cache memory are available, and this cache memory has much lower latencies when compared to fetching data from typical data stores located on the cloud. Crucially, one can rent varying amounts of cache memory depending on one s requirements at any point of time. This service has come to be known as elastic caching, and is offered via products like Amazon s Elasticache [1] and Microsoft s Azure Redis Cache [2]. Of course, it would cost more to rent a larger cache, and so a user needs to trade off between the cache rental cost (which we call the maintenance cost ) and the latency cost when the desired page is not in the cache. For example, at the time of writing, Azure Cache offers individual cache units whose sizes range from 250 MB (at a cost of $0.055 per hour) to 53 GB (at a cost of $2.10 per hour). Indeed, finding this balance between minimizing maintenance costs and minimizing latency naturally leads to a new set of algorithmic challenges. In this paper, we initiate a theoretical study of such problems in a setting we call elastic caching. Let us present our formal model. We have a slow memory of n pages, and some subset A t of these pages are in the cache at time t. Requests to some of these n pages arrive over time. If a requested page is not present in the cache, Copyright c 2019 by SIAM Unauthorized reproduction of this article is prohibited

2 we fetch this page into the cache, and may also decide to evict one or more pages. Each page i [n] has a weight w i and a size s i. There are two kinds of costs: Eviction cost: If we move from set A t 1 to A t, we pay some cost for evicting the pages in A t 1 \ A t. In this paper, we consider the case where the eviction cost is d(a t 1, A t ) := i A t 1\A t w i. In other words, we pay w i for evicting page i out of our cache. Maintenance cost: We are given a monotone function c : R 0 R 0, and we pay a maintenance cost of c( i A t s i ) every time step. More generally, we also consider the case when the cost function is a non-negative set function, i.e., C(A t ) is the cost of maintaining the set A t for one unit of time. While in general this could be an arbitrary monotone function C : 2 [n] R 0, we also consider the case when C has some structure, e.g., it is submodular. The goal is to minimize the sum of the eviction and maintenance costs over all time. I.e., min ( ) (1.1) d(a t 1, A t ) + c(a t ). t Note that the regular fixed-cache-size paging problems are a special case of the above problem, where we set the maintenance cost function c(s) = 0 for s k and for s > k. Setting w i = s i = 1 gives the classical paging problem [16], having general weights but unit sizes gives the weighted paging problem [17], and having both being general gives the generalized caching problem [5]. 1.1 Our Results. Our main result is the following: THEOREM 1.1. (MAIN THEOREM) There is an O(log n)-competitive randomized algorithm for elastic caching with general sizes and eviction weights when the maintenance cost only depends on the cache size. We also address the deterministic online complexity of the problem. In fact, we can even deal with arbitrary monotone maintenance cost functions. THEOREM 1.2. (DETERMINISTIC ALGORITHM) There is an n-competitive deterministic algorithm for the elastic caching problem with general eviction weights w i, if the maintenance cost function C : 2 [n] R 0 is an arbitrary monotone function of the set of pages in the cache. In particular, this implies an n-competitive algorithm for the elastic caching with general weights and sizes. Since the paging problem with n pages and a fixed cache of size k = n 1 has lower bounds of n and H n for deterministic and randomized algorithms, these results are tight upto constant factors. We then turn our attention to cost functions with some additional structure: Indeed, if the cost function is concave (i.e., the marginal cost per unit of additional cache decreases as we increase the cache size), or more generally, any monotone submodular function of the pages, we can give substantially improved results: THEOREM 1.3. (SUBMODULAR COSTS) There is a deterministic 2-competitive algorithm for the elastic caching problem with general eviction weights w i, if the maintenance cost C : 2 [n] R 0 is a monotone submodular function of the set of pages in the cache. Finally, we consider the offline problem, and give a constant-factor approximation. THEOREM 1.4. (OFFLINE PROBLEM) There is a polytime O(1)-approximation algorithm for elastic caching with general sizes and eviction weights w i where the maintenance cost only depends on the cache size. 1.2 Our Techniques. Our algorithms follow the online primal-dual paradigm: we write an LP relaxation for the problem, solve it fractionally, and then round the fractional solution in an online fashion. In the fixedcache size setting, the LP relaxation is well-understood: for each page p and time t, we have a variable x p,t which indicates whether p is outside the cache or not. Now the eviction cost at time t is given by p (x p,t x p,t 1 ) + ; since there is a fixed cache size, there is no maintenance cost. This LP relaxation has a constant integrality gap when all pages are unit sized, and it forms the basis of very elegant O(log k)-competitive randomized algorithms based on the primal-dual method for weighted paging [5]. For pages with varying sizes, one needs to augment this LP relaxation with the knapsack-cover inequalities [5], and in this case, both the online fractional algorithm and the online rounding are more involved [3]. Indeed, it is not difficult to extend the LP relaxation to the setting of elastic caching: for each time t and size s, we need to add indicator variables for whether a cache of size s is being used at time t details are in Section 5. Moreover, for generalized caching, we can also strengthen the LP relaxation by adding the knapsack cover inequalities for all possible sizes. However, obtaining a primal-dual algorithm requires some need new ideas. For instance, since the size of the cache is not fixed, when do we stop raising the primal and dual variables when a new request arrives? In the fixed-size caching problems,

3 we can evict the pages other than the currently requested one using a multiplicative weights approach until the total fractional size of the non-evicted parts reaches the cache size k. It is not clear how to generalize this approach to our setting. A related question is how to raise the dual variables to get enough money to pay for the cache size we choose? (In fact, these questions are non-trivial even for the case of unit costs and weights.) Finally, the myriad Knapsack cover inequalities would only add to our woes! A crucial insight to help address these issues (especially the last one) is to actually work with a stronger configuration LP relaxation for this problems. Indeed, the associated dual LP has much more structure that we leverage in our primal-dual updates. Using this much-larger LP, we show in Section 2 how to update the primal/dual variables, and how to get a stopping condition so that we can pay for both the eviction and maintenance costs to within an O(log n) factor of a feasible dual. At a high level, we use the tight constraints of the dual to decide a tentative cache size at each time step, and check if the fractional primal variables satisfy the knapsack cover inequalities w.r.t this tentative size. If not, we identify a violated constraint and monotonically increase the primal/dual variables corresponding to this set of pages. This will then change the tentative cache size, and the process would continue until we converge. Finally, we show how we can adapt (in a straight-forward manner) an elegant rounding scheme of [3] to convert our fractional solution into a randomized integer solution. Using the configuration LP has another advantage: it allows us to cleanly generalize our results to settings when the maintenance cost of a cache is a submodular function of the pages in it. We design a simple primal-dual algorithm which actually recovers an integral solution which is also 2-competitive. The main idea here is that we can express the primal as a purely covering problem (submodularity means that we don t need the usual constraint which enforces that we pick at most one configuration at each time), and this observation greatly simplifies the algorithm. The configuration LP is crucially needed when the maintenance cost is a monotone function of the set of pages in the cache. We show that it has integrality gap of at most n, and in fact, we can even give an on-line algorithm with this competitive ratio. We again rely on the dual of the configuration LP, but the dual growth process is more straightforward. Finally, to get our offline approximations, we show that one can solve the configuration LP up to a constant factor by relating it to a weaker LP relaxation which uses the Knapsack Cover inequalities. Then, given a fractional solution to the configuration LP, we use filtering based techniques to round it. 1.3 Related Work. The classical paging/caching problem with a fixed cache size has been widely studied since the mid-80s. When the cache holds k pages, Belady s offline algorithm (Farthest in the Future) performs an optimal number of evictions [7]. Sleator and Tarjan [16] and Fiat et al. [14] gave deterministic k-competitive and randomized O(ln k)-competitive algorithms respectively, and showed that these were both optimal. In weighted paging, each page has a weight and the goal is to minimize the total weight of evicted pages. This is equivalent to the k-server problem on a star graph, so deterministic k-competitiveness follows from the algorithm of Chrobak et al. [12] for k-server on trees. Bansal et al. [4] settled the randomized complexity of this problem by showing a randomized O(ln k)-competitive algorithm, illustrating the power of the primal-dual technique for these problems. In generalized caching, pages also have sizes, and the total size of the pages in the cache must not exceed the cache capacity k. (All sizes are integers.) The off-line problem is NP-hard and an O(1)-approximation was given by Bar-Noy et al. [6]. Cao and Irani [9] and Young [17] gave deterministic (k + 1)-competitive algorithms. Bansal et al. [5] gave an O(log 2 k)-competitive randomized algorithm using primal-dual techniques, which was subsequently improved to the tight O(log k)-factor by Adamaszek et al. [3]. All these problems have a related (h, k)- counterpart, where the offline adversary is restricted by a cache of size h < k. Deterministic k/(k h + 1)-competitive and randomized O(ln k/(k h + 1)- competitive algorithms are known for all these variants [18, 5, 3]. Recently, there has also been work on generalizing paging by considering settings where the set of pages in cache must be an independent set of a matroid [8], or the setting where there is a matroid where we need to maintain a basis but the maintenance cost function (which is linear over the ground elements) changes over time, and the objective is to minimize the sum of maintenance and eviction costs [15]. Another problem loosely related to ours is called generalized caching with rejections, where we can choose not to serve a page request by incurring a penalty [13]. 1.4 Preliminaries and Notation. For a subset T of pages, we use s(t ) to denote the total size i T s i of pages in T. We also use S to denote the total size i s i of all the n pages. We use the small letter c : R 0 R 0 to denote maintenance costs which depend only on the cache size, and the capital letter C : 2 [n] R 0 for general cost functions. There are page requests at integer time steps, and we use the symbol i t to denote the request at time t. For each 3

4 page i and j 1, let t be the time at which the j th request for page i is made. For notational convenience, we divide the entire time horizon into phases for each page. For j 1, the j th phase for page i comprises of the time slots in the interval [t, t +1 ). Finally, for a page i and time t, we slightly abuse notation and use j(i, t) to denote the current phase of page i at time t, i.e., j(i, t) = arg max j t t. 1.5 Roadmap. In Section 2, we describe the randomized O(log n)-competitive algorithm for elastic caching for size-dependent maintenance cost. In Section 3, we give a 2-competitive algorithm for submodular maintenance costs. Then in Section 4 we consider deterministic algorithms for general monotone maintenance costs and finally, in Section 5, we consider the offline setting. 2 An O(log n)-competitive Algorithm for Elastic Caching In this section, we consider the setting where pages have arbitrary non-negative sizes and weights, and the cost for maintaining a set A of pages in cache is c(s(a)), and devise a randomized O(log n)-competitive online algorithm using the online primal-dual framework. 2.1 The Primal and Dual Linear Programs. We begin by stating the primal and dual formulations. (LP1) (2.2) (2.3) (2.4) (LPD1) (2.5) (2.6) min w i x + c(s())y,t,t x (i,t) + y,t 1 i i t, t :i :i t y,t 1 y,t 1 max i,t x 0 t t i, j y,t 0, t. γ i,t t σ t γ i,t w i t +1 t<t +1 γ i,t σ t c(s()) i γ i,t 0 σ t 0 i, j, t i, t t. For the primal, we use a configuration LP (LP1), with variables y,t for each configuration 1 and time t. Recall that i t is the requested page at time t, t denotes the time at which the j th request for page i is made, and j(i, t) is the current phase for page i at time t. Here x denotes the indicator variable whether the page i is evicted from the cache during phase j or not. For each time slot t, and each configuration [n] of pages, there is an indicator variable y,t for whether the set of pages in the cache at time t is. For any configuration, we use the expression s() to denote i s i, the total size of configuration. For any request sequence, the total cost incurred is the total eviction cost plus the total maintenance cost, which is our LP objective function. Constraint (2.2) ensures that, at each time step, either a page is already evicted in the corresponding phase, or we must select a configuration which contains it; constraint (2.3) enforces that at each time t, the requested page be included in the cache; and constraint (2.4) enforces that we select at most one configuration at each time step. The following lemma is then easy to see. LEMMA 2.1. The optimal solution to (LP1) has objective value at most OP T. 2.2 Online Algorithm for Fractional Solution. At any time step t, the relevant primal variables for our online algorithm are the x (i,t) variables which intuitively correspond to the eviction probabilities of page i in its current phase j(i, t). To make the overall intuition of our algorithm more transparent, the dual constraints (2.5) are called the weight constraints, and the dual constraints (2.6) the size constraints. When a new page request arrives at time t, we run Algorithm 2.1 to update the x (i,t), γ i,t and σ t variables monotonically at suitably chosen rates. We note that we do not directly compute the configuration y variables online, but instead identify a size s t of the cache at each time step such that the maintenance cost c(s t ) can be charged to the increase in dual objective. Then using the x variables and the cache sizes computed, we infer a randomized eviction policy which ensures that the pages which remain in the cache have total size at most s t. The Intuition. At a high level, we raise the x variables for some subset of the pages at an exponential rate: since x is the fraction of the page i that has been evicted in phase j, this increase gradually evicts these pages from the cache. Simultaneously, we raise the corresponding dual variable γ i,t linearly, such that the weight constraint for i becomes tight exactly when the primal variable reaches 1 (i.e., when page i has been fully evicted). But when do we stop raising the x variables? For the classical paging 1 We shall use the term configuration to refer to a subset of pages.

5 problem, this is when the sum of x values equals n k, ensuring that k pages are in cache. But we are in the elastic caching setting, and the cache size is not fixed! Moreover, how do these x ij variables interact with the choice of the configuration at time t, and how do we account for the maintenance cost c()? We need some new ideas for this. This is where the tight size constraints come into play. The intuition is that tight size constraints (2.6) indicate how large a cache we can maintain at various time steps. Indeed, if during the execution of the update algorithm, the size constraint becomes tight for some configuration with s( ) = s, then the dual objective value collected until now at time t is i γ i,t σ t i γ i,t σ t = c(s( )), enough to pay for a cache of size s. So as long we we ensure that the dual objective only increases monotonically during Algorithm 2.1 s execution, we will be able to charge the maintenance cost of c(s ) to the increase in dual objective at time t. This is important to note because during the course of the execution of Algorithm 2.1, a size constraint may become tight at some point, and subsequently become slack because of the different rates in which our algorithm will update the γ i,t and σ t variables. Motivated by the preceding discussion, during the execution of Algorithm 2.1 at time step t, we maintain a critical size, denoted s t. This critical size is equal to the maximum value of s() over all configurations such that the the size constraint corresponding to became tight during the the execution of the update algorithm at time step t. Note that the value of s t increases monotonically over the course of the continuous update algorithm for time step t, and moreover, from the discussion in the preceding paragraph, we can charge the maintenance cost c(s t ) to the increase in dual objective at time t. It remains to figure out the subset of pages for which we increase the x and γ variables, and also when to stop the algorithm. To this end, we work with the goal that if the critical size at any instant of the update algorithm is s t, the set of pages evicted from cache account for a size of at least S s t (meaning that the remaining pages residing in cache can fit in a size of s t ). So one natural stopping point would be when i s ix (i,t) first exceeds S s t. However, the covering knapsack has a large integrality gap, and so we cannot translate this into a convex combination of feasible integer configurations which fit into a s t -sized cache. This leads us to our final idea: we increase the x, γ and σ variables until the x (i,t) variables satisfy the so-called Knapsack Cover (KC) inequalities for a covering requirement of S s t. While these inequalities also appear in previous algorithms for generalized caching [5], it is more subtle in our setting. Indeed, notice the cyclic nature of the process here: the currently violated KC-inequality will dictate how we update the primal and dual variables, which in turn will dictate how the critical size s t changes over time, which in turn dictates the subsequent KC-inequalities we need to satisfy. To this end, we first define the Knapsack Cover inequalities, and then formally present the update algorithm. For clarity in presentation, we present the algorithm as a continuous-time procedure, and we can easily discretize it using standard techniques. DEFINITION 1. Given a set of pages T and a covering requirement of R s(t ), a fractional solution {z i } {i T } over the pages in T is said to satisfy the knapsack cover (KC) inequalities for the requirement of R if for all subsets X T such that s(x) > s(t ) R, we have min(s i, (s(x) (s(t ) R)))z i i X (s(x) (s(t ) R)). Equivalently, we want that for all subsets Y T such that s(y ) < R, we have i T \Y min(s i, (R s(y )))z i R s(y ). So to summarize, we update our x, γ i,t, and σ t variables such that the total eviction cost is comparable to the dual increase, and moreover, the x variables at the end of the continuous process satisfy the KC inequalities for a requirement of S s t. More accurately, we enforce that out of the pages in [n] \ {i t }, we evict a total size of at least (S s it ) (s t s it ), so that even with including s i in the cache, the total size would be at most s t. The Formal Algorithm. The algorithm for maintaining the fractional solution is shown below; we set the parameters α = ln(1 + 1/δ) and δ = 1/n. The set A t denotes the set of pages i for which we can potentially raise the γ i,t variables. In Step 6, we choose a suitable subset of pages, and raise the dual variables corresponding to these in Steps 7 9. We will show that happens to be a subset of A t. In Step 11, we raise σ t to ensure that the dual constraints (2.6) are satisfied. Observe that the actual increase in Steps 7 11 is infinitesimal, and then we go back to the while loop condition in Step 4. A more discrete way of stating this is that we increase these dual variables till (i) the set A t changes, or (ii) the set as defined in Step 6 changes. We now prove that the Algorithm 2.1 maintains a feasible primal-dual pair of low cost. CLAIM 1. Consider the execution of Algorithm 2.1 at time t. The set defined in Step 4 is a subset of A t. 5

6 Algorithm 2.1 ELASTICCACHE(t, i t ) 1: Initialize A t {i : x (i,t) < 1} 2: Initialize γ it,t = c(s i ), γ i,t = 0 for all i i t, σ t = 0, and s t s i 3: Let T [n] \ {i t } 4: while {x (i,t) } {i T } does not satisfy KC inequalities for T and R := S s t do 5: Let T denote the set of all tight size constraints at this moment 6: Let T correspond to a minimally violated KC inequality 7: for all i do 8: Increase γ i,t at rate min(s i, s() s t + s it ) 9: Increase x (i,t) at rate α w i min(s i, s() s t + s it ) (x (i,t) + δ) 10: 11: Increase γ it,t at rate min(s it, s() s t + s it ) Increase σ t at rate max T i ( {i min(s t}) i, s() s t + s it ) 12: Update A t {i : x (i,t) < 1} 13: Update s t max(s t, max T s( )) Proof. Suppose the KC inequality for the requirement R and set T as defined in Steps 3 4 are not satisfied by the subset of pages. In other words: min(s i, (s() (s(t ) R)))x (i,t) i < (s() (s(t ) R)). Further assume that there is a page i for which x i,j(i,t) is 1. If min(s i, (s() (s(t ) R))) = s() (s(t ) R), then the above inequality is no valid. So it must happen that min(s i, (s() (s(t ) R))) is s i. Now, subtracting s i from both sides in the inequality above, we see that the KC inequality for the subset \ {i } is also violated, contradicting the minimality property of. LEMMA 2.2. At all times during the execution of the algorithm, the dual variables remain feasible. Proof. We prove this by induction over time. Indeed, suppose the statement holds for all times up to t 1, and consider the algorithm execution after the arrival of page i t at time t. Firstly, we initialize the algorithm with γ it,t = c(s i ) and σ t = 0. Since the variable γ it,t does not appear in any weight constraint, and c( ) is a monotonically increasing function, all size constraints are feasible after this initialization. Now, consider the weight constraints (2.5) for some i i t. Our update rules for x (i,t) and γ i,t variables imply that x (i,t) = δ [exp(αγ/w i ) 1] where Γ := t +1 t <t +1 γ i,t is the total increase for the dual γ variables contributing to the dual weight constraint during its current phase. Hence, when Γ reaches w i, we have x (i,t) reaches 1 as well, and the page is no longer in the set A t. Claim 1 shows that we will not raise γ i,t any more, and so this ensures feasibility of the weight constraints. The size constraints (2.6) for time t are independent of previous times, so consider the current time step t. By the definition of the rate of increase of σ t, we ensure that if size constraint is tight at any instant of this continuous update, the rate of increase of i γ i,t is at most the rate of increase of σ t, and hence the size constraint is never violated. Next, we show that the first term in the primal objective can be bounded against the dual objective, for which the following claim will serve useful. CLAIM 2. At each instant of the update algorithm, if s t denotes the critical size at this instant, and if denotes the minimally violated KC constraint identified at this instant, then the rate of increase of dual objective at this instant is at least s() s t + s it. Proof. Using the notation in Algorithm 2.1, we see that the rate of increase of dual objective is given by min(s i, s() s t + s it ) i {i t} max T i ( {i t}) min(s i, s() s t + s it ). Indeed the first term is the rate at which i γ i,t increases and the second term is the rate of increase of σ t. Therefore it suffices to show that for any T, i {i t} min(s i, s() s t + s it ) i ( {i t}) s() s t + s it. min(s i, s() s t + s it )

7 This is equivalent to needing: i ( {i t})\ min(s i, s() s t + s it ) s() s t + s it, which would in turn follow from: i ( {i t})\ s i s() s t + s it, which would be implied by: s t i ( {i t}) s i. But this last inequality is true, since s t is the maximum size of a configuration corresponding to a tight constraint and is a tight configuration. LEMMA 2.3. At any instant of the update algorithm, the rate of increase of i w ix is upper bounded by 2α times the the rate of increase of the dual objective. Proof. Let us first calculate the rate of increase of i w ix at any instant of the update algorithm. By definition, this is equal to α i min(s i, s() s t + s it ) (x (i,t) + δ) which is upper bounded by α(t 1 + T 2 ) where T 1 = i min(s i, s() s t + s it )x (i,t) and T 2 = δ i min(s i, s() s t + s it ). We now bound T 1 and T 2 separately by the rate of growth of the dual objective. Note that T 1 is precisely the left hand side of the minimally violated KC inequality. Therefore T 1 s() (s(t ) R) = s() + s it s t, which is at most the rate of growth of dual objective by Claim 2. As for T 2, note that it is at most δn(s() s t + s it ), which is at most the dual growth rate by using δ = 1 n and from Claim 2. Another implication of Claim 2 is the following: COROLLARY 2.1. At any time step, the dual objective is non-decreasing over the execution of the algorithm. Proof. At any instant, the dual growth rate is at least s() s t + s it by Claim 2. But note that for to be a violated KC inequality, it must be the case that s() > s t s it, by plugging in the values of T = [n] \ {i t } and R = S s t in definition 1. LEMMA 2.4. During the course of the execution of Algorithm 2.1, the quantity c(s t ) is at most the increase in dual objective at time step t, i.e., i γ i,t σ t. Proof. If s t is updated at any moment of the algorithm s execution, it means that there exists a tight size constraint such that s t = s( ). In particular, at this moment, we have i γ i,t σ t i γ i,t σ t = c(s( )) = c(s t ). This observation, along with Corollary 2.1 which asserts that the dual is always increasing, completes the proof. LEMMA 2.5. When Algorithm 2.1 terminates for a time step t, it will maintain primal variables x that satisfy the KC inequalities for a requirement of R = S s t. Proof. This follows just from the fact that if there is a violated constraint, then the algorithm updates the primal and dual variables until all KC inequalities are satisfied. We now conclude this section by summarizing the output of our algorithm in the following lemma. THEOREM 2.1. Using Algorithm 2.1, we can compute in an online manner a sequence of fractional vectors x t and a critical size s t for each time t such that: (a) the total fractional eviction cost i,t w i x i,t x i,t 1 is O(α)OP T, (b) at each time t, the x i,t variables satisfy the KC inequalities for the set of pages [n] \ {i t } and requirement S s t, (c) x it,t = 0 for each t and the requested page i t, and (d) the total maintenance cost t c(s t ) is O(α)OP T. Proof. Recall that for every page i, the configuration LP defines variables x, where j corresponds to the phase defined by [t, t +1 ) for page i. We will like to extend this variable to x i,t for all (integer) time t. Again the interpretation should be that x i,t is 1 iff i is not in the cache at time t. During our primal-dual algorithm, for each i and j, the variable x remains 0 till time t reaches t. Then from time t + 1 onwards till time t +1 the algorithm raises x and when the phase ends, it does not change x. Therefore, for a time t [t, t j+1 ), we can simply define x i,t as the value of x at time t during our primal-dual procedure. So we can simply output the vector x t defined by x i,t for all pages i along with the critical sizes s t computed when the Algorithm 2.1 terminates at each time step. Since x i,t+1 x i,t unless a phase for page i ends at time t (in which case x i,t is reset to 0), the following observation is easy to see: (2.7) w i x i,t+1 x i,t 2 w i x. i,t Therefore, property (a) above follows by combining this observation along with Lemma 2.3 and 2.2. Property (b) 7

8 follows by the termination condition of the while loop in Algorithm 2.1. Property (c) follows because we do not increase the x (i,t) at the beginning of a new phase, i.e., at time t = t, and hence x it,t = 0 for all requested pages (since they define the beginning of a new phase). Finally, property (d) follows from Lemma Online Rounding. Finally, we show how to round the fractional solution promised by Theorem 2.1. Indeed, this procedure is in fact a direct adaptation of a very elegant rounding scheme of Adamaszek et al. [3] for generalized caching. Our main observation is that the rounding algorithm of [3] essentially does not use the fact that the cache size is a fixed parameter over all time steps, and constructs the rounded solution only using the knowledge of the fractional solution x t over time. THEOREM 2.2. Suppose we have a sequence of vectors x t and sizes s t satisfying the properties of Theorem 2.1. Then, the online rounding algorithm of [3], which, when given this sequence of vectors x t, efficiently computes a sequence of distributions D t over subsets of evicted pages such that: every set in the support of D t evicts a total size of at least S s t pages, the request page i t is not evicted in any set in the support of D t, and the total earthmover distance EMD(D t 1, D t ) is at most O(1) i w i x i,t x i,t 1. Proof. In what follows, we give a proof sketch of Theorem 2.2. We describe the rounding of Adamaszek et al. at a high-level while deferring the complete details to [3]. Importantly, we will abstract out the features of their rounding which will be most relevant to our setting and use the other ingredients in [3] in a black-box manner. Recall that by running the online primal-dual Algorithm 2.1, we can compute in an online manner, a sequence of fractional vectors x t and a critical size s t for each time t such that: (a) the total fractional eviction cost i,t w i x i,t x i,t 1 is O(α)OP T, (b) at each time t, the x i,t variables satisfy the KC inequalities for the set of pages [n] \ {i t } and requirement S s t, (c) x it,t = 0 for each t and the requested page i t, and (d) the total maintenance cost t c(s t ) is O(α)OP T. We now sketch the rounding procedure, which will convert the x t vector into a distribution of integer cache states, such that each valid cache state evicts at least S s t size, and moreover, i t is never evicted from any of them. Most importantly, successive distributions for time steps t and t + 1 are close to each other. We first introduce some notation before proceeding with the details. Let us partition the pages into different size classes, with size class S l := {i : 2 l 1 < s i 2 l }. Ideally, we want our cache states in our distribution to be close in some manner to the x t vector, and it turns out that the correct characterization of close is the following. DEFINITION 2. Given a fractional vector z [0, 1] n, let L denote the prefix of pages (sorted by non-increasing order of size) such that the total z value of pages in L first exceeds 1 (or L = [n] if no such prefix exists). We say that a subset E [n] of pages mirrors the vector z if: for all pages i, z i = 0 implies that i / E, for all pages i, z i = 1 implies that i E, for each integer l, we have i S l z i E S l, and we have E L 1. Intuitively, the definition requires that within each size class, the total number of pages in E should be at least the total number of fractional pages in z. Moreover, for some technical reasons which will crop up in the proof of Lemma 2.6, we also need that from the prefix of pages with fractional z value adding to 1, the set E must select at least one page. From our perspective, it is crucial that the definition of when a set E mirrors a solution z does not depend on the cache size, and only depends on the the structure of the solution z in different size classes and the prefix of 1 unit of mass. Using this definition, the following structural lemma (a restatement of Lemma 3.3 in [3] to handle time-dependent cache sizes) shows that mirroring a fractional solution is sufficient to be a valid cache state at any time step. LEMMA 2.6. Let x t be a fractional solution which satisfies the KC inequalities for requirement S s t, and suppose x it,t = 0. Define z i,t = min(1, 16 x i,t ). Then if a set E of pages mirrors the vector z t, then E is a feasible integral solution for the covering knapsack problem with requirement S s t, and E does not contain i t. Equivalently the complementary pages [n] \ E describes a valid cache state containing i t, and having pages of total size at most s t. After this point, the algorithm in [3] ignores the cache size altogether, and simply seeks to maintain a distribution D t over pages such that every set with positive probability in the distribution D t mirrors the fractional solution z t where z i,t = min(1, 16 x i,t ). Moreover, D t can be computed by making bounded changes from D t 1. More formally, they show the following result (Section 3.2

9 of [3]) which is also their main contribution, and which we use in a black-box manner. THEOREM 2.3. The online rounding algorithm of [3], which, when given this sequence of vectors z t, efficiently computes a sequence of distributions D t over subsets of evicted pages such that: every set in the support of D t mirrors z t pages, and the total earthmover distance EMD(D t 1, D t ) is at most O(1) i w i z i,t z i,t 1. Combining Lemma 2.6 with Theorem 2.3 completes the proof. Combining this with Theorem 2.1 then proves Theorem Elastic Caching with Submodular Costs We now consider the Submodular Elastic Caching problem, where the maintenance cost for a set [n] of pages in the cache is given by a monotone submodular function C(). We now show that the primal-dual approach gives a deterministic 2-competitive algorithm in this case. The configuration LP is written in a similar manner as LP1, given below. As compared to LP1, the main simplification here is that we do not have any constraint corresponding to (2.4) here, i.e., we do not need to explicitly enforce that we only pick one configuration at any time instant. Indeed, if any solution picks multiple configurations, say, 1, 2,..., l to satisfy the constraint (3.8), then = i will also satisfy all the constraints, with only lesser holding cost as C() C( i ) due to submodularity of C( ). (LP2) (3.8) (3.9) min w i x + C()y,t,t x (i,t) + y,t 1 i i t, t :i :i t max i,t y,t 1 x 0 t i, j y,t 0, t. γ i,t The interpretation of variables x remains as in LP1. The variable y,t is the indicator variable which is 1 iff is the set of pages in the cache at time t. The second term in the objective function captures the maintenance cost. The constraints (3.8) and (3.9) are similar to the constraints (2.2) and (2.3) in LP1. We now describe the update algorithm when a new page is requested. Indeed, at any time t, say that page i is active if the dual constraint (3.10) is not tight for page i and the current phase at time t, i.e., j = j(i, t). We also maintain a set F t of frozen variables, which is initialized to. In the update at time t, the algorithm begins with all dual variables γ i,t = 0 and raises all the dual variables for active, unfrozen pages (i.e., those in A t \ F t ) uniformly at rate 1, until one of the following event happens: (a) There is a page i for which the dual constraint (3.10) becomes tight. This page is then removed from A t, and gets evicted from the cache. (b) One of the dual constraints (3.11) for a set becomes tight. In this case, all pages in gets frozen, i.e., we set F t := F t. This process continues till we reach a stage when i,t γ i,t becomes larger than C(A t ) using submodularity, we will show later that such a stage will always occur. The update at time t then stops, and we keep all of A t in the cache. The algorithm is described formally in Algorithm 3.1. We now analyze this algorithm. LEMMA 3.1. The variables γ i,t form a feasible solution to the dual LP. Proof. Feasibility of (3.11) is easy to see whenever this constraint becomes tight, we add to the set F t, and then we never γ i,t for any i thenceforth. Feasibility of (3.10) also follows from a similar argument. Indeed, if (3.10) becomes tight for any page i and phase j, the page i gets removed from the active set A t, and is also evicted from cache. Hence, the γ i,t variables are no longer updated for all subsequent time steps in this phase for page i. We now show that the algorithm is well-defined, i.e., the while loop terminates. The following structural claim, which is essentially an uncrossing argument, will be useful for this. (3.10) (3.11) γ i,t w i t +1 t<t +1 γ i,t C() i γ i,t 0 i, j i, t, t CLAIM 3. If constraint 3.11 is tight for two sets 1 and 2, then it is tight for 1 2. Proof. Observe that C( 1 2 ) + C( 1 2 ) i 1 2 γ i,t + i 1 2 γ i,t 9

10 Algorithm 3.1 Online Primal-Dual Update at time t 1: Initialize A t as set of all pages i for which (3.10) is not tight for the current phase j(i, t). 2: Initialize γ i,t = 0 for all i, and initialize set F t = 3: while i γ i,t < C(A t ) do 4: for all i A t \ F t do 5: increase γ i,t at rate t γ i,t = 1 6: Update A t as set of all pages i for which (3.10) is not tight for the current phase j(i, t). 7: for all sets for which constraint (3.11) is tight do 8: Update F t := F t = i 1 γ i,t + i 2 γ i,t = C( 1 ) + C( 2 ) C( 1 2 ) + C( 1 2 ). The first inequality follows because of dual feasibility, the last inequality follows from submodularity, and the second equality follows because constraint 3.11 is tight for two sets 1 and 2. So all inequalities have to be satisfied at equality above, and in particular, the constraint (3.11) has to be tight for 1 2 and 1 2. COROLLARY 3.1. At any moment of the update algorithm, we have C(F t ) i F t γ i,t Proof. By nature of when elements get added to F t, we have that at any moment, F t is a union of tight i s. Hence from Claim 3 we get that the dual constraint 3.11 is tight for F t. Using the above corollary we show that the algorithm is well-defined. LEMMA 3.2. For every time t, the while loop in Algorithm 3.1 terminates. Moreover, the requested page i t is in A t when the loop terminates. Proof. Fix some time t. We now show that if i γ i,t < C(A t ), then A t \ F t (hence steps 11 and 12 are executed). Indeed, suppose not, i.e., A t F t. Then Corollary 3.1 says that i γ i,t i F t γ i,t C(F t ) C(A t ), which contradicts our assumption that i γ i,t < C(A t ). As for the current request i t, observe that a new phase begins for this page, moreover, the variable γ it,t does not contribute to its constraint (3.10). Hence it will not get removed from A t in this time step. We can now bound the total cost of the on-line algorithm. LEMMA 3.3. The total eviction cost and the cache maintenance cost of the on-line algorithm is at most twice the optimal cost. Proof. We first bound the eviction cost. Any time we evict a page i in some phase j, it must be that t <t<t +1 γ i,t = w i. So we can simply charge the eviction cost w i of page i in phase j to the term t <t<t +1 γ i,t. Summing over all i and j, we get that the total eviction cost is at most i,t γ i,t. As for the maintenance cost, we are guaranteed by the termination condition of the while loop, that at each time, the total holding cost C(A t ) is at most i γ i,t. Summing over all t, we get that the total holding cost is at most i,t γ i,t. The proof then follows by weak duality and from the fact that γ is a feasible dual solution (Lemma 3.1). 4 Deterministic Algorithms for Elastic Caching with Monotone Maintenance Cost In this section, we consider the Elastic Caching problem where the maintenance cost is an arbitrary non-negative monotone set function C() of the configuration of pages in the cache, i.e., C() 0 for all, and C() C( ) for. We give a simple deterministic n- competitive algorithm for this problem using the primaldual method. Note that this also implies an n-competitive algorithm for the basic Elastic Caching problem studied in Section 2 when the maintenance cost is a function of the cache size. One can write a configuration LP which is identical to (LP1), except that c(s()) is replaced by C() in the objective function. Our algorithm will need to work with the dual LP, which is again very similar to the dual LP (LPD1). The dual LP is as follows: (LPD2) (4.12) (4.13) max i,t γ i,t t σ t γ i,t w i t +1 t<t +1 γ i,t σ t C() i γ i,t 0 σ t 0 i, j, t i, t t. The algorithm is described in Algorithm 4.1. At each time t, the algorithm maintains a set A t of active pages

11 these are the pages i for which the constraint (4.12) for j = j(i, t) is not tight (i.e., satisfied with equality). Any page which becomes inactive gets evicted from the cache (if it was in this cache at this moment). The algorithm uniformly raises all the γ i,t variables for i A t. We add a configuration to a set T if the constraint (4.13) for, t gets satisfied with equality at some point of the execution. To ensure feasibility of (4.13), we raise σ t at a rate equal to the maximum number of active pages in a configuration T. The process continues as long as all the active pages are not contained in a single configuration in T. All active pages when the algorithm terminates remain in cache at time t. Termination of the algorithm follows from the fact that either the set T grows, or the set of active pages A t shrinks over the course of execution. Also note that the page i t will never get removed from A t because γ it,t does not appear in any constraint of the form (4.12), and hence i t A t when the algorithm terminates. Finally, we note that it is possible that constraint (4.13) for some gets slack after becoming tight. However, once enters T, it continues to remain in it. We now analyze the algorithm. CLAIM 4. The dual variables are feasible, and the rate of increase of dual objective function is always at least 1. Further, γ i,t σ t 1 γ i,t. n t i,t Proof. Feasibility of dual variables is easy to see whenever (4.12) gets tight, we stop raising the corresponding γ (i,t) variable, and whenever (4.13) gets tight, we raise σ t so that it never gets violated. The rate of increase of i γ i,t is A t, while that for σ t is A t for some T. By the termination condition of while loop, it must be the case that A t < A t. Therefore the rate of increase of i γ i,t σ t is at least 1. Since the rate of increase of i γ i,t is at most n, the last statement in the claim also follows. We now bound the cost of the algorithm. A page i gets evicted during phase j if the corresponding constraint (4.12) becomes tight. Therefore, the total eviction cost can be bounded by i,t γ i,t t +1 t t +1 i,t n γ i,t i,t t σ t, γ i,t where the last inequality is given by Claim 4. Thus, the total eviction cost is at most n times the cost of a feasible dual solution. Now we account for the maintenance cost. Fix a time t. The cache contains the set A t at the end of the update step in Algorithm 4.1. Note that there is a configuration T containing A t. By monotonicity, it follows that C(A t ) C(). The fact that was added to T implies that there was a moment during the update algorithm when the constraint (4.13) for, t became tight, i.e., when γ i,t σ t = C(). i Since the LHS is at most the total dual raised during time t (using Claim 4), we can pay for the maintenance cost by the total dual objective function. This completes the proof of Theorem Offline Algorithm for Elastic Caching In this section we describe a constant factor polynomialtime approximation algorithm for elastic caching when the maintenance cost is a monotone function of the cache size. Indeed, one approach would be to solve the configuration LP (LP1) in polynomial time, and round it. However, notice that finding a separation oracle for (2.6) in the dual LP is equivalent to solving several knapsack problems, one for each possible size of the cache. While this can be done in time polynomial in the total size of the n pages, we would like to design strongly polynomial-time algorithms. Moreover, since the dual constraint involves difference of dual variables, it is not clear if an approximation algorithm (even an FPTAS) for the knapsack problem is useful as a separation oracle. As a result, we present a different formulation in Section 5.1, based on the Knapsack Cover inequalities. We first show a structural theorem that proves that the new formulation and the configuration LP are within a factor of 2 of each other. Moreover, the proof will be constructive, the meaning of which will become clear soon. So we can use the structure theorem to approximately solve the configuration LP in polynomial time. Once we do this, we do some simple filtering steps to decide the appropriate cache size at each time t, and then simply deploy our online rounding algorithm! 5.1 A Different LP relaxation for Elastic Caching. We now describe the new LP formulation for Elastic Caching, which we call knapsack cover LP, in Figure 5.1. Recall here that S denotes the total size of all pages. For each page i and phase j corresponding to page i, the variable α is an indicator variable for whether page i is evicted during phase j this is same as the variable x 11