Multi-version Coding for Consistent Distributed Storage of Correlated Data Updates

Transcription

1 Multi-version Coding for Consistent Distributed Storage of Correlated Data Updates Ramy E. Ali and Vivek R. Cadambe 1 arxiv: v1 [s.it] 21 Aug 2017 Abstrat Motivated by appliations of distributed storage systems to loud-based key-value stores, the multiversion oding problem has been reently formulated to store frequently updated data in asynhronous distributed storage systems. Inspired by onsisteny requirements in distributed omputing, the main goal in multi-version oding is to ensure that the latest possible version of the data is deodable, even if all the data updates have not reahed all the servers in the system. In this paper, we study the storage ost of ensuring onsisteny for the ase where the various versions of the data are orrelated, in ontrast to previous work where versions were treated as being independent. We provide multi-version ode onstrutions that show that the storage ost an be signifiantly smaller than the previous onstrutions depending on the degree of orrelation between the different versions of the data. Our onstrutions are based on update-effiient odes, Reed-Solomon ode, random binning oding and BCH odes. Speifially, we onsider the multi-version oding setting with n servers and ν versions, with a failure tolerane of n, where the ν message versions form a Markov hain. The message version is distributed uniformly over all binary vetors of length, and given a version, the subsequent version is uniformly distributed in a Hamming ball of radius δ entered around that given version. Previously derived ahievable shemes have a per-server storage ost of at least 1 ν 2 o) when ν, whereas + ν 1 we propose a sheme that has a storage ost of log V olδ, ), where V olr, ) is the volume of a ball of radius r in the dimensional Hamming ube. Through a onverse result, we show that our multi-version ode onstrutions are nearly-optimal in ertain orrelation regimes. I. INTRODUCTION Distributed key-value stores are an important part of modern loud omputing infrastruture 1. eyvalue stores are ommonly used by several appliations inluding reservation systems, transations and multi-player gaming. Owing to their utility, there are numerous ommerial and open-soure loud-based key-value store implementations suh as Amazon Dynamo [2], Apahe Cassandra [3], and CouhDB [4]. In typial distributed key-value stores, the data stored is updated frequently, and the time sales of data updates and aess are often omparable to the time sales of dispersing the data to the servers See [2]). In suh settings, ensuring that a lient gets the latest, most updated, version of the data an be hallenging as the data updates may not have reahed all servers. The notion that the latest version of the data must be aessible to the users despite the frequent updates is known as onsisteny in omputer siene [2], [5], [6]. The design of a onsistent distributed storage system is a topi that has been widely studied in theory of distributed systems See [5], [7] and referenes therein). In addition to loud omputing, maintaining onsisteny in several omputing appliations where there is data redundany and frequent updates, for instane for ahing in edge omputing systems See [8], [9]). Ramy E. Ali ramy.ali@psu.edu) and Vivek R. Cadambe vivek@engr.psu.edu) are with the Shool of Eletrial Engineering and Computer Siene, The Pennsylvania State University, University Park, PA This work is supported by NSF grant No. CCF and is published in part in the Proeedings of the 2016 IEEE Information Theory Workshop, September 2016 [1]. 1 In simple words, a key-value store is a shared database that an be aessed by lients and supports two operations: a get or read operation, and a put or write operation.

2 2 Reently, the multi-version oding problem [7], [10], [11] has been formulated to study the storage ost of ensuring onsisteny in distributed systems from an information theoreti viewpoint. The analysis of [7], [10], [11] was onduted under the modeling assumption that different versions of the data are independent of eah other. In this paper, we study the impat of orrelations among the different data versions in the multi-version oding framework. The shared memory emulation problem is a distributed omputing theoreti abstration, that models distributed key-value store implementations 2. The goal of the shared memory emulation problem is to implement a read-write variable over a distributed system of servers. The multi-version oding problem studies odes for the shared memory emulation problem from an information theoreti perspetive. Inspired by the shared memory emulation model, the multi-version oding problem differs from previous studies of distributed storage in information theory, e.g., [12], [13], in the following aspets: 1) Asynhrony: a new version of the data may not arrive at all servers simultaneously. 2) Deentralized nature: there is no single enoder that has knowledge of all the versions of the data simultaneously, and a server is not aware of whih versions are reeived by other servers; instead, servers store odeword symbols of the old versions, and then update the stored symbols or append to storage based on the newly arrived data version. 3) Consisteny: a lient aessing the storage system must be able to retrieve the latest possible version of the data by onneting to a subset of the servers; the ardinality of the subset of the servers that need to make the data aessible is diretly related to the desired fault tolerane in the system. A lassial erasure oding approah does not diretly apply under the above assumptions. Unlike in repliation where servers store the entire data, in erasure oding, servers store odeword symbols and the deoder has to aess a suffiient number of servers to reover the data. As a onsequene, when a new version arrives in the system, a server annot diretly update the odeword symbol until the version has propagated to a suffiient number of servers. Speifially, in systems with frequently updated data, servers have to store several versions of the data, thereby offsetting the storage ost benefits of erasure oding. The multi-version oding problem analyses the extent of this offset from an information theoreti perspetive. By deriving ahievable shemes and impossibility results for the multi-version oding framework, referene [7] showed that, interestingly, there is an inevitable prie in terms of storage ost for maintaining onsisteny in an asynhronous, deentralized, distributed storage system. Furthermore, the storage ost grows with the degree of asynhrony in the system. In this paper, we show that the storage ost an be signifiantly redued if the degree of orrelation between subsequent updates is signifiant. We onsider a variant of the multi-version oding problem as follows. We onsider a distributed storage system with n servers and the objetive is to store a message with ν versions, whih are totally ordered; messages with higher ordering are referred to as later versions, and lower ordering as earlier versions. Eah server reeives an arbitrary subset of the ν versions, and enodes them. A lient who onnets to any servers an obtain the latest ommon version among them, as per the total ordering, or a later version. The goal of the multi-version oding is to minimize the storage ost. Here, we assume that eah message version is bits long, and model the orrelation between suessive versions in terms of the bit-strings that represent them. More speifially, given a version of the message, we assume that the next version is uniformly distributed in the Hamming ball of radius δ, entered around that given version. The multi-version oding problem studied previously [7] assumed that the versions are independent, i.e., δ = 1. In this lassial erasure oding model, where ν = 1, the Singleton bound ditates that the storage ost per server is at least. Note however that for ν > 1, a server annot simply store the odeword symbol orresponding to one version, sine another server may not have reeived the orresponding version. In the multi-version oding setting, referene [7] showed via an ahievable shemes 2 The shared memory emulation problem, being a theoretial abstration, is atually a muh simplified view of key-value stores. Nonetheless, algorithms and design priniples of key-value implementations have muh in ommon with solutions to the shared memory emulation problem.

3 3 and onverse that, the storage ost per server is at least 1 2 o) when ν. Sine the per-server ost of storing eah version is, we may interpret the result as follows: when the versions are independent, to ompensate for the asynhrony and still maintain onsisteny, the servers have to store - from a ost perspetive - at least half of all the ν versions. In this paper, we derive novel ahievable shemes and an approximately tight onverse for all regimes of δ. Our results signifiantly improve upon the result of [7] when the orrelation is signifiant, speifially, when δ < 1/2. In partiular, for the ase where δ = m - that is, m bits hange in eah suessive version - we propose a sheme that has a per-server storage ost of m log + ν 1) + olog ). For the ase where δ = δ, that is the number of bits that hange in suessive versions is proportional to, our sheme has a per-server storage ost of Hδ) + ν 1) + olog ). We derive a onverse that shows that for the ase of δ = δ, our sheme is approximately optimal. Related Work ν The idea of exploiting the orrelation between the different versions to effiiently update, store or exhange data has a rih history of study in network information theory [14]. In their lassi work, Slepian and Wolf [15] studied the problem of ompressing orrelated distributed soures, where the objetive is deoding the data of all soures. Linear ode onstrutions that approah the Slepian-Wolf limits have been proposed in [16] [20] and referenes therein. The idea of exploiting orrelated information to design storage and ommuniation systems has been studied for several modern and emerging appliations in reent times. Referene [21] onsiders simultaneous and sequential interative data exhange problems between two users. By leveraging Slepian- Wolf and belief propagation tehniques, pratial shemes have been proposed that exploit the orrelation to minimize the number of exhanged bits between the users. The problem of enoding inremental updates effiiently is the motivation of the delta enoding/ompression tehniques used ommonly in data storage. Referenes [22], [23], and referenes therein refine the notion of delta ompression by modeling the data updates using the edit distane; in partiular, the referenes develop oding shemes that synhronize a small number of edits between a lient and a server effiiently. The study of exploiting orrelations between data updates to improve effiieny in a distributed storage setting, where multiple servers store odeword symbols orresponding to the data, has been of signifiant reent interest [24] [30]. Referenes [24], [25] devise oding shemes that use as input, the old and new versions of the data, and outputs an n-length ode that an be used to store both versions of the data effiiently in a distributed storage system. Referenes [26], [27] study apaity-ahieving updateeffiient odes for binary symmetri and erasure hannels, where a small hange in the message leads to a odeword whih is lose to the original odeword. Hene, the onstruted odes lead to effiient updates of data in distributed storage systems. We use the results of [26], [27] in one of the multi-version oding onstrutions in our paper. Referene [30] studies the problem of storing orrelated data for distributed ahing and ontent distribution, and shows the utility of the lassial Gray-Wyner soure oding [31] in the studied appliation. Referene [28] studied the ommuniation ost of updating a stale server that did not get an updated message, by downloading data from already updated servers in suh distributed setting. The referene proposed ode onstrutions and tight bounds for this problem. We note that the problem of [28] has some modeling elements that are ommon with our approah, speifially, there is a limited degree of asynhrony - a single update does not a reah a partiular server. A side information problem is presented in [29], where the goal is to send an updated version to a remote entity that has as side information, an arbitrary linear transform of an old version. The referene shows that the optimal enoding funtion is related to a maximally reoverable subode of the linear transform assoiated with the side information. The problem of [29] is of peripheral interest to some of the solutions of our paper, sine we aim to store use odeword symbols of some versions as side information to store other versions of the data. Although our problem formulation and solutions have some ommon ingredients with previous works, our setting differs from all the previous works beause we are motivated by the shared memory emulation

4 4 problem, where the data update time sales are similar to the message propagation time sales in the network. Speifially, in our setting a) eah server in the storage system reeives an arbitrary set of message versions, and b) no node in the system is aware of the versions reeived by any other node in the system. An important outome of our study is that orrelation between versions an be used to redue storage osts in distributed systems, despite the asynhrony, deentralized nature and onsisteny requirements. Organization of this paper The rest of this paper is organized as follows. Setion II presents the multi-version oding problem, bakground and the results of [7]. We provide a summary of the main results in Setion III. In Setion IV, we develop update-effiient multi-version ode onstrutions. In Setion V, we onstrut multi-version odes that are motivated by Slepian-Wolf oding. Setion VI provides a lower bound on the storage ost. Finally, onlusions are disussed in Setion VII. II. SYSTEM MODEL AND BACGROUND OF MULTI-VERSION CODES We begin with some notation. We use boldfae for vetors and apital letters for random variables/vetors. In the n-dimensional Eulidean spae, the standard basis vetors are denoted by {e 1, e 2,, e n }. For a vetor x, we denote its Hamming weight by w H x). For any two vetors x 1 and x 2, we denote the Hamming distane between these two vetors by d H x 1, x 2 ). For a positive integer i, we denote by [i] the set {1, 2,, i}. For any set of ordered indies S = {s 1, s 2,, s S } Z where s 1 < s 2 < < s S, and for any ensemble of variables {X i : i S}, the tuple X s1, X s2,, X s S ) is denoted by X S. We use BECp) to denote a binary erasure hannel with erasure probability p. We use log.) to denote the logarithm to the base 2 and H.) to denote the binary entropy funtion. We will sometimes use the notation [2 ] to denote the set of -length binary strings, that is, we assume an impliit mapping between [2 ] and {0, 1}. Finally, an ode of length n and dimension k over alphabet A onsists of a mapping C : A k A n. We refer to a ode C of length n and dimension k as an n, k) ode. Sometimes, to keep notation terse, we use the abbreviation MVC to refer to multi-version odes. A. Multi-version Codes MVCs) Fig. 1: A distributed storage system with n = 4 servers storing ν = 2 versions. A user an reover the latest ommon version or a later version from any = 2 servers.

5 5 We now present a variant of the definition of the multi-version ode from [7], where we model orrelations between the various message versions. We onsider a distributed storage system with n servers. The objetive is to store ν possibly orrelated versions of the message where W i [2 ] is the i-th version, i [ν], and is the message length in bits. The versions are assumed to be totally ordered, i.e., if i > j, W i is interpreted as a later version with respet to W j. We assume that W 1 W 2... W ν form a Markov hain. W 1 is uniformly distributed over the set of all length binary vetors. Given W m, W m+1 is in a Hamming ball of radius δ, BW m, δ ), where We denote the volume of the ball by BW m, δ ) = {W : d H W, W m ) δ }. 1) V olδ, ) = BW m, δ ) = δ Remark 1. Note that for δ = δ, where δ < 1/2 is a onstant, we have j=0 ). 2) j Hδ) o) log V olδ, ) Hδ), 3) where the last inequality follows from Stirling s inequality [32]. We use the above approximation for expository purposes in parts of the paper. The i-th server reeives an arbitrary subset of versions Si) [ν]. We refer to the subset of versions available at the i-th server, Si), as the state of that server. We denote the system state by S = {S1), S2),, Sn)} P[ν]) n, where P[ν]) denotes the power set of [ν]. For the i-th server with state S = Si) = {s 1, s 2,, s S }, where s 1 < s 2 < s S, the server stores a odeword symbol whih is generated by the enoding funtion ϕ i) S that takes an input W S and outputs an element in [q]. For any set of servers T [n], u L = max i T Si) denotes the latest ommon version among these servers. The objetive of the multi-version oding is to design enoding funtions that minimizes the storage ost suh that the latest ommon version or a later version of the message is deodable from any set T [n] of servers. That is, the system an tolerate any n server failures erasures). Fig. 1 depits a possible senario with n = 4 servers. We next provide a formal definition for the multi-version oding formulation. Definition 1 ɛ-error Multi-version ode MVC)). An ɛ-error n,, ν, 2, q, δ ) multi-version ode MVC) onsists of the following for ɛ > 0 enoding funtions deoding funtions that satisfy the following [ P S, T ) := Pr ϕ i) S : [2 ] S [q], for every i [n] and every state S [ν], ψ T ) S ϕ t1) St 1),, ϕt) St ) ψ T ) S : [q] [2 ] {NULL}, ) ] = W m, for some m u L, if i T Si) 1 ɛ, for every possible system state S P[ν]) n and every set of servers T [n], where T = {t 1, t 2,, t }, t 1 < t 2 < < t. Definition 2 Storage Cost of a Multi-version Code). The storage ost of an ɛ-error n,, ν, 2, q, δ ) MVC is equal to log q bits. Referene [7] studied 0-error MVCs with independent versions, that is, the speial ase of ɛ = 0, δ = 1.

6 6 Remark 2. In ontrast to the original multi-version oding framework [7], our orretness requirement is probabilisti, that is, we require orret deoding with a probability that is at least 1 ɛ. This relaxation leads to storage ost savings in the setting of orrelated versions. We next desribe the results of [7], and provide new ahievable shemes when the different versions are orrelated. B. Bakground - Repliation and Simple Erasure Coding Repliation and simple maximum distane separable MDS) odes provide two natural MVC onstrutions. Suppose that state of the i-th server is S = {s 1, s 2,..., s S }, where s 1 < s 2 <... < s S. Repliation based MVCs: In this sheme, eah server stores the latest version it reeives. The enoding funtion is ϕ i) S W S) = W s S. Therefore, the storage ost per server is, that is, the number of bits stored by a server is the number of bits in one version. Simple MDS odes based MVCs MDS-MVCs): In this sheme, an n, ) MDS ode is used to enode eah version separately. Speifially, suppose that C : [2 ] [2 / ] n is an n, ) MDS ode over alphabet [2 / ], and suppose that C i) : [2 ] [2 / ] denotes the i-th o-ordinate of the output of C. The MVC is onstruted as ϕ i) S W S) = C i) W s1 ), C i) W s2 ),..., C i) W s S )). That is, eah server stores one odeword symbol for eah version it reeives. In the worst-ase where a server reeives all versions, the storage ost is ν. Beause of the distributed nature of the setting, no server is aware of what versions are present at other servers. Therefore, when MDS odes are used, a server has to store odeword symbols orresponding to multiple versions as the latest version at the server may not have propagated to a suffiient number of servers. In ontrast, with repliation a server stores only the latest version. Referene [7] developed multi-version oding shemes and onverse results. An important outome of the study of [7] is that, when the different versions are independent, i.e., if δ = 1, then the storage ost ν ν+ 1 annot be smaller than o). In partiular, the best possible MVC sheme is, approximately, at most twie as ost-effiient as the better among repliation and simple erasure oding. In this paper, we show that repliation and simple erasure oding are signifiantly ineffiient if the different versions are orrelated, i.e., if δ is smaller than 1. Our shemes resemble simple erasure odes in their onstrution; however, we exploit the orrelation between the versions to store fewer bits per server. III. MAIN RESULTS In this setion, we present the main results of the paper, desribing the storage osts of various multiversion oding shemes and impossibility results. The onstrutions and the proofs are presented in later setions. We begin with some simple shemes in Setion III-A, that set a omparison point for our shemes. We then desribe our ode onstrutions in Setions III-B and III-C. We present our onverse results in Setion III-D. We tabulate all results in Table I. A. Motivating sheme Consider a simple MDS odes based multi-version oding MDS-MVC) sheme of Setion II-B; assume that we use a Reed-Solomon ode over a field F p of binary harateristi. The generator matrix of a Reed-Solomon ode is usually expressed over F p. However, every element in F p is a vetor over F 2, and a multipliation over the extension field F p is a linear transformation over F 2. Therefore, the generator matrix of the Reed-Solomon ode an be equivalently expressed over F 2 as follows G = G 1), G 2),..., G n) ), where G is a n/ binary generator matrix, and G i) has dimension /. Beause Reed-Solomon odes an tolerate n faults, we know that every matrix of the form G t1), G t2),..., G t) ), where

7 7 t 1, t 2,..., t are distint elements of [n], has a full rank of over F 2. We now desribe modifiations of the sheme of Setion II-B that use the orrelation among the versions to redue storage ost. We begin with the following simple, albeit non-ausal, multi-version oding sheme. For server i whih has versions S = {s 1, s 2,, s S } [ν], s 1 < s 2,..., s S, the server stores bits of W s1 using an MDS ode, that is, it stores Ws T 1 G i). For eah subsequent version, the server simply stores the vetor W sm W sm 1. Beause W sm W sm 1 lies in a Hamming ball of volume V ol s m s m 1 δ, ), the server effetively stores log 2 V ol s m s m 1 δ, ) bits for version s m. The deoder that onnets to any servers an deode the latest ommon version u L among these servers. This is beause, a deoder that onnets to server i an ompute W ul W s1 and then evaluate Ws T 1 G i) + Wu T L Ws T 1 )G i) to get Wu T L G i), thereby obtaining suffiient information from servers to reover W ul. The worst-ase storage ost assoiated with the ase where the server reeives all versions - is given by + ν 1) log V olδ, ). The sheme, albeit simple, is quite powerful if δ is small. For instane, in the speial ase where only a single bit hanges in every subsequent version, i.e., δ = 1/, the above sheme stores + ν 1) log + 1) bits, whih is signifiantly smaller than the storage ost of both erasure oding and repliation. A desirable feature of this sheme is that a priori knowledge of the parameter δ is not required. However, the sheme is not ausal; having stored a odeword symbol orresponding to W s1, it is not lear how the server would ompute W s2 W s1. We now explain a sheme that is ausal, whereas, it requires knowledge of the parameter δ. The sheme is also a modifiation of the simple erasure oding based sheme of Setion II-B. Suppose that the i-th server reeives the set of versions S = {s 1, s 2,, s S } [ν]. The server enodes W s1 using the binary ode as W T s 1 G i). For W sm, where m > 1, the server finds a differene vetor y i) s m,s m 1 satisfies the following 1) y i) T s m,s m 1 G i) = W T s m W T s m 1 )G i) and 2) w H y i) s m,s m 1 ) s m s m 1 δ. Note that it is not neessary and even possible to ensure) that y s i) m,s m 1 = W sm W sm 1. Also note that y s i) m,s m 1 an be represented by log V ol s m s m 1 δ, ) bits. It is easy to see that the deoder an obtain Wu T L G i) from the i-th server by applying Ws T 1 G i) + Wu T L Ws T 1 )G i). Therefore, the deoder an obtain the latest ommon version among any servers. The worst-ase storage ost is again, + ν 1) log V olδ, ). Now, the two shemes we desribed above motivate the following two questions. Q1: Can we obtain a ausal MVC onstrution that is oblivious to the parameter δ with a storage ost of + ν 1) log V olδ, )? Q2: Can we obtain a MVC onstrution - possibly even non-ausal and non-oblivious to the parameter δ - with a signifiantly smaller ost as ompared with + ν 1) log V olδ, )? Our results of Setion III-B provide relatively elementary ode onstrutions based that address question Q1. The shemes assoiated with the results of Setion III-B use the idea of update-effiient odes [26], [27]. Our results of Setion III-C address question Q2. The onstrutions assoiated with Setion III-B, are inspired by Cover s random binning proof of the Slepian-Wolf soure oding problem [33]. B. Update-effiient Multi-version Codes In Setion IV, we develop MVC onstrutions that exploit the orrelation to redue the storage ost. Our update-effiient MVCs are essentially modifiations of the MDS-MVCs of Setion II-B, where we apply ideas of delta-ompression [34] to store the odeword symbols of the reeived versions at the servers. Reall that in MDS-MVCs a server stores a odeword symbol for eah version that it reeives. In update-effiient MVCs shemes, a server stores bits for the first version that it reeives, similar to the sheme of Setion II-B. However, unlike the sheme of Setion II-B, a server may not store all odeword symbols for suessive versions; instead, it may only store the symbols that hange from the previous version and their positions. If the versions are enoded using update-effiient odes - where a that

8 8 Sheme Worst-ase Storage Cost Comments. Repliation based MVCs, oblivious to δ Simple Erasure odes ν, outperforms repliation if ν ; oblivious to δ G Update-effiient ode + ν 1)δOlog2 ) + olog 2 ), asymptotially outperforms the above shemes for δ = o1/ log 2 ); oblivious to δ. Reed-Solomon ode Update-effiient + ν 1)δlog + olog )), asymptotially outperforms the above shemes for δ = o1/ log ); oblivious to δ. Motivating sheme of Setion III-A Random Binning BCH Binning assumes ommon baseline version) + ν 1) log V olδ, ), not oblivious to δ. + ν 1 log V olδ, ) + olog ), asymptotially outperforms all the above shemes if δ < 0.5, ν <, not oblivious to δ + νν 1)/2 δ log + 1), asymptotially outperforms updateeffiient odes for ν < 2,not oblivious to δ. Lower Bound + ν 1) log V ol, δ) + O1) +ν 1 +ν 1 TABLE I: Storage ost. appliable for all δ. small number of hanges in message symbols lead to only a few odeword symbols hanging - then we an obtain MVCs that are more storage-effiient as ompared to MDS-MVCs. We desribe the results for two lasses of update-effiient odes next: the olhin-generator G) onstrution of [26], [35], and an interpretation of Reed-Solomon odes. 1) G Update-Effiient MVCs: Referene [26] showed that there exist a sequene of odes with rate arbitrary lose to the apaity of the binary erasure hannel, indexed by the number of bits enoded, with vanishing probability of error suh that their update effiieny - the number of bits in the odeword that hange when a single bit in the message hanges - is Olog ). In our model, eah suessive version differs in at most δ bits as ompared with the previous version of the message. If we use MDS-MVCs with generator matries hosen as per the onstrution of [26], then for eah subsequent version, we only need to store the positions of at most δ Olog ) odeword bits that differ from the odeword of the previous version. For a single server storing approximately bits of the first version it reeives, storing the position of eah hange in the odeword of the subsequent version inurs a ost of log bits. Sine the update-effiient ode onstrution of [26] ensures that the number of bits that hange for eah subsequent version is δ Olog ) bits, and there are ν 1 subsequent versions, the storage ost of the sheme is + ν 1)δ Olog 2 ) + olog 2 ). We state our result formally next. Theorem 1. [G Update-effiient MVC] For every 0 < α < 1, γ > 1, there exists a sequene of n,, ν, 2, q, δ ) multi-version odes, indexed by, whose worst-ase storage ost is α + ν 1 minδ γln α ) log α /), α ), 4) where α = /α, and has a probability of error that goes to 0 as. The proof of the theorem is provided in Setion IV. Note that in the theorem, α and γ are parameters ontrol the trade-off between the storage ost and the probability of error of the sheme see Setion IV

9 9 and the Appendix). The main tehnial hange as ompared with [26] in the above sheme is that the result of [26] is for the random erasure model, and we need to adapt their methods to our setting where the server failures are adversarial. It is also worth noting that the theorem an be relaxed by hoosing α = +log, suh that the storage ost of Theorem 1 an be expressed as + ν 1)δ Olog 2 ) + olog 2 ). 2) Reed-Solomon Update-Effiient MVCs: In Reed-Solomon update-effiient MVCs, we simply use Reed-Solomon ode of length n and dimension, over a field of size n p, where n p is the smallest prime power that is greater than n. That is, we split any message of bits into log n p bloks of log n p bits eah, and eah blok is enoded using this n, ) Reed-Solomon ode. The i-th server then stores the i-th odeword symbol of the Reed-Solomon ode for eah of the log n p bloks. Therefore, for a subsequent version, where at most δ bits hange in the message, at most δ odeword symbols hange per server. We an then store the hanged odeword symbols and their loations. Therefore, the worst-ase storage ost of this sheme is +ν 1)δ log np log n p = +ν 1)δ log +olog )). Observe here that the storage ost is asymptotially superior to G Update-effiient MVC. We state our result formally below. Theorem 2. [Reed-Solomon Update-Effiient MVC] There exists a 0-error n,, ν, 2, q, δ ) updateeffiient multi-version ode whose worst-ase storage is given by ) + ν 1) minδ np log, /), 5) log n p where n p > n is the smallest prime power that is greater than n. Although our disussion here provides suffiient hints for this rather elementary sheme and the proof of the above theorem, for the sake of ompleteness, we formally provide details of our sheme and a proof in Setion IV-B. It is worth noting that the update-effiient MVCs shemes do not require a priori knowledge of the parameter δ. Disussion: For the regime of δ = o1/ log ), there remains the question of optimality of the Reed-Solomon update-effiient MVCs. The Reed-Solomon update-effiient MVCs whose storage ost is +ν 1)δ log +olog )) may be interpreted as follows. For the i-th server, for the first version it reeives, store the i-th symbol of an n, ) MDS erasure ode to store bits; for eah subsequent version, remaining simply store, in its entirety, a representation of the differene with respet to the first version. The fat that the differene with respet to the first version is effetively) stored in its entirety motivates the following question, whih is essentially a variant of question Q2 posed in Setion III-A: Can we use erasure oding based ideas for the subsequent versions as well, to obtain a storage ost of + ν 1 δ log + olog ))? The answer to the above question Q2 is partiularly non-trivial beause of the nature of the multiversion oding problem. For instane, one set of servers may reeive versions W 1 and W 3. A seond set of servers may not reeive W 1, but instead reeive W 2 and W 3. The first set of servers has to enode the differene between W 1 and W 3, whereas the seond set of servers have to enode the differene between W 2 and W 3 in a manner that W 3 is deodable from all the servers. The development or modifiation of update-effiient ode onstrutions to satisfy this deoding onstraint, and still obtain a 1/ erasure-oding gain fator for the differene is non-trivial in suh senarios. Next, we summarize the results of ode onstrutions that answers the question posed above positively. In partiular, we desribe ode onstrutions whose storage ost is approximately + ν 1 log V olδ, ). Unlike the update-effiient shemes, a priori knowledge of δ is required in the ode onstrution. C. Slepian-Wolf inspired Multi-version Codes In Setion V, we develop MVCs that are inspired by Slepian-Wolf oding. In Setion V-A, we develop a MVC that is based on random binning. The following theorem speifies the worst-ase storage ost of this sheme.

10 10 Theorem 3. [Random Binning MVC] There exists an ɛ-error multi-version ode n,, ν, 2, q, δ ) whose worst-ase storage ost is given by + ν 1) log V olδ, ) + νν 1)/2 ν log ɛ. 6) Choosing ɛ = 1 log, the above theorem implies that there exist a sequene of MVCs with vanishing error probability and storage ost of ν 1) log V olδ,) + + olog ). For the ase where δ is a onstant, our sheme speializes to the sheme presented in our earlier work [1] as given by the following orollary. Corollary 3.1. For δ = δ, where δ < 1/2 is a onstant, the worst-ase storage ost is upper-bounded by + ν 1)Hδ) + νν 1)/2 ν log ɛ. 7) We note that this orollary follows from Remark 1. The ode onstrution of Theorem 3 is non-onstrutive. In Setion V-B, we onsider a variant of multiversion oding assuming that eah server has a ommon baseline version W 1, that is, 1 Si), i [n]. Under the assumption of a ommon baseline version, we provide linear MVC ode onstrutions inspired by the ideas of [17]. The storage ost of our onstrutions is given by the following theorem. Theorem 4. [Linear Binning MVC] In a distributed storage system with a ommon baseline version W 1, there exists a 0-error n,, ν, 2, q, δ ) multi-version ode whose worst-ase storage ost is given by ν 1 R opt, 2i 1)δ + 1)) + i=2, 8) where R opt N, d) is the highest possible rate of a linear binary ode with length N and minimum distane d. Using the speial ase of BCH odes, we obtain the following orollary. Corollary 4.1 BCH Binning MVC). In a distributed storage system with a ommon baseline version W 1, there exists a 0-error n,, ν, 2, q, δ ) multi-version ode whose worst-ase storage ost is at most + νν 1)/2 δ log + 1). 9) We notie that this sheme may have a better storage ost as ompared with the sheme of Theorem 2 depending on the values of ν and. The result of Theorem 4 leads to at least two interesting open questions: i) an we design algorithms in distributed systems that ensure that all servers have a ommon baseline version?, and ii) an we design onstrutive MVC shemes orresponding to the existene results of Theorem 3, where we do not assume a ommon baseline version at all servers? The answers to both questions are outside the sope of our work, but we disuss some ideas here. The first question relies on ideas in the field of distributed algorithms, and the idea of hekpointing [36] is used in several distributed systems to take a snapshot of the urrent state of the system at eah server node. This idea may help develop algorithms, where we may be able to ahieve a ommon baseline version at all servers. Furthermore, ertain algorithms suh as [37] already do ensure, via a finalize label, a suffiient number of servers in the system agree that a partiular message version has propagated to

11 11 a suffiient number of servers. It is an open question whether our ode onstrutions an be adapted to suh algorithms. Although there are linear ode onstrutions for the Slepian-Wolf setting [17] [19], the seond question is interesting, at least in part, beause our setting is slightly more ompliated than the usual Slepian-Wolf setting. Speifially, in our setting, the deoder desires to deode only the latest ommon version, or a later version, whereas in the usual Slepian-Wolf setting, the deoder desires to deode all the messages. Furthermore, our setting has a onept of states, similar in spirit to ompound wireless hannels - the state of the system an hange and the version is to be deoded, and the possible ahievable rates, an depend on the state of the system. It is not immediately lear whether the ideas of [17] [19] an be inherited or adapted to our setting. D. Lower Bound on The Storage Cost Lower Bound on the storage ost: In Setion VI, we extend the storage ost lower bound derived in [7] for the ase where we have orrelated versions. We state our onverse results in the following theorem and the subsequent orollary. Theorem 5. [Storage Cost Lower Bound] An ɛ-error n,, ν, 2, q, δ ) multi-version ode with orrelated versions suh that W 1 W 2... W ν form a Markov hain, W m [2 ] and given W m, W m+1 is in a Hamming ball of radius δ entered around W m must satisfy log q + ν 1) log V olδ, ) + ν 1 Corollary 5.1. For δ = δ, where δ < 1/2 is a onstant, we have log q + log1 ɛ2νn ) log +ν 1 ν + ν 1 ) ν!. 10) + ν 1 + ν 1 Hδ) + o) 11) + ν 1 We note that the above orollary follows from Remark 1. The above orollary implies that the storage ost haraterized in Corollary 3.1 is approximately optimal if ν < for δ = δ, that is, δ is a onstant. Speifially, sine 1/ + v 1) 0.5/ if ν <, the storage ost of the onverse is at least half of + ν 1 Hδ) + o), whih is the ahievable storage ost in Corollary 3.1. IV. UPDATE-EFFICIENT MULTI-VERSION CODES In this setion, we develop simple multi-version oding shemes that exploit the orrelation between the different versions and have smaller storage ost as ompared with [7]. In these shemes, the servers do not know the orrelation degree δ in advane. We begin by realling the definition of the update effiieny of a ode from [26]. Definition 3 Update effiieny). For a ode C of length N and dimension with enoder C : F F N, the update effiieny of the ode is the maximum number of odeword symbols that must be updated when a single message symbol is hanged. Formally, the update effiieny is expressed as follows t = max d H CW), CW )). 12) W,W F : d HW,W )=1 We observe that the update effiieny of a linear ode is the maximum row weight of the generator matrix of this ode, hene it is at least the minimum distane of the ode. Definition 4 Update effiieny of a server). Suppose that C i) : F F N/n denotes the i-th o-ordinate of the output of C stored by the i-th server. The update effiieny of the i-th server is the maximum

12 12 number of odeword symbols that must be updated in this server when a single message symbol is hanged. Formally, the update effiieny of the i-th server is given by t i) = max d H C i) W), C i) W )). 13) W,W F : d HW,W )=1 Suppose that G = G 1), G 2),, G n) ) is the generator matrix of a linear ode C, where G i) is of dimension N/n and orresponds to the i-th server. The update effiieny of the i-th server is the maximum row weight of G i). Definition 5 Maximum update effiieny per server). The maximum update effiieny per server is the maximum number of odeword symbols that must be updated in any server when a single message symbol is hanged. Formally, the maximum update effiieny per serve is given by t s = max i [n] t i). 14) An N, ) ode C is referred to as an update-effiient ode if it has an update effiieny of on). Referenes [26] and [27] have onstruted update-effiient odes with update effiieny Olog ) that an tolerate random erasures, ahieve rates arbitrarily lose to the apaity over BECp) and have arbitrary small probability of error. A. G Update-effiient MVC The G ode onstrution in [26] has been proposed for the random failures model, where eah server fails with probability p independently from the other servers. In this Setion, we provide the proof of the simple sheme of Theorem 1, where we show that the G ode also an tolerate any n server failures with high probability. Proof of Theorem 1. We use the n α /, ) binary ode C given by the following onstrution. Constrution 1 G Update-Effiient MVC). Consider the n α /, ) ode C r haraterized by the generator matrix G = [g ij ] whose entries are hosen randomly and independently of eah other based on the distribution Pr[g ij = 1] = ln α + x α i [], j [n α /], 15) where α = /α, x = a log α for some onstants a > 0 and α < 1. There exists a determinsti ode with a generator matrix G = G 1), G 2),, G n) ) in this ensemble that an tolerate any n server failures with probability that goes to 1 as and has a maximum update effiieny per server that is at γ ln α most, where γ > 1. Suppose that the i-th server reeives the set of versions S = {s 1, s 2,..., s S }. For W s1, the server stores Ws T 1 G i). For W sm, where m > 1, the server may store the oordinates that have been updated or store W T s m G i). Therefore, it stores min s m s m 1 δ t i) log α /), α /) bits. We show that this onstrution an tolerate any n failures with probability that goes to 1 as in the Appendix. The worst-ase storage ost orresponds to the ase where a server reeives all versions. The storage ost per server in this ase is at most α mind H W j+1, W j )t s log α /), α /), + ν 1 j=1

13 13 Hene, the worst-ase storage ost is given by α + ν 1 minδ γln α ) log α /), α ). Remark 3. In the ahievable sheme given by Theorem 1, the server stores the index of eah bit that hanges independently. Sine the number of bits that hange from a version to the next version is at most δ t s, we have log δts ) possibilities of the indies of these bits and hene the following j=0 storage ost is also ahievable α/ j α δts + ν 1) log j=0 α / j ), 16) where t s < γ ln α. However, in this sheme the servers need to know δ a priori. B. Reed-Solomon Update-effiient MVC The previous update-effiient multi-version oding sheme has an update effiieny of Olog ) whih grows with the file size. Theorem 2 presents a pratial update-effiient multi-version oding sheme that is based on Reed-Solomon ode and has an update effiieny of n. Hene, the update effiieny grows with the number of servers, but not with the size of the file. Moreover, the maximum update-effiieny per server is 1. We prove the theorem next. Proof of Theorem 2. We start by desribing the ode onstrution. Constrution 2 Reed-Solomon Update-Effiient MVC). Suppose that the i-th server reeives the versions S = {s 1, s 2,..., s S }. We divide a version W sj, j [S], into log n p bloks, eah of length log n p. In eah blok, we represent every onseutive string of log n p bits by a symbol in F np. We denote the representation of W sj over F np by W sj. We enode eah blok by a n, ) Reed-Solomon ode with a generator matrix G whih is given by λ 1 λ 2 λ n G = λ 2 1 λ 2 2 λ 2 n...., 17) λ1 1 λ2 1 λn 1 where Λ = {λ 1, λ 2,, λ n } F np is a set of distint elements. For W s1, the i-th server stores W T s 1 G i), where G i) is given by Ge i G i) 0 Gei 0 0 = ) Gei For W sm, where m > 1, the server may only store the updated symbols from the old version W sm 1 or store all symbols, thus it stores at most min s m s m 1 δ log log n p ) + log n p ), /) bits of W sm.

14 14 The worst-ase storage ost orresponds to the ase where a server reeives all versions. Therefore, the storage ost per server is at most ν 1 + ) np mind H W i+1, W i )log, /). log n p i=1 Hene, the worst-ase storage ost is given by + ν 1) minδ np log log n p ), /). V. SLEPIAN-WOLF INSPIRED MULTI-VERSION CODES In this setion, we introdue a random binning based argument showing the existene of a multi-version oding sheme satisfying the result of Theorem 3. Later on, in Setion V-B, we onstrut a linear oding onstrution, showing Theorem 4. A. Random Binning Multi-version Code. Reall that Slepian-Wolf oding [15], [38] is a distributed data ompression tehnique for orrelated soures that are drawn in independent and idential manner aording to a given distribution. In the Slepian-Wolf setting, the deoder is interested in deoding the data of all soures. In the multi-version oding problem, the deoder is interested in deoding the latest ommon version, or a later version, among a set of servers. Inspired by the random binning proof for the Slepian-Wolf problem [33], [39], we build multi-version oding onstrutions where older versions earlier provide side information to the deoder to reover the latest ommon version. Similarly, in our onstrutions, a version that is a latest ommon version in some state may at as a side information to the deoder in a different state. We note that our model slightly differs from the standard Slepian-Wolf setting. First, in our setting, we do not aspire to deode all the versions, but we only want to ensure that the latest ommon version is deodable for every state. Seond, unlike the standard Slepian-Wolf setting, we are interested in a broader orrelation model, where the bits of subsequent versions are not drawn in an independent and idential manner given the previous versions. For instane, given W 1, in the standard Slepian-wolf setting, W 2 is onentrated in a spherial shell of radius δ ± o) for some onstant δ. However, our setting is more general. Despite these differenes, the idea of random binning naturally leads to a proof in our paper. An interesting aspet of our proof is the argument that a point with the rate presribed in Theorem 3 indeed exists in the feasible rate region for the given error probability. The lossless soure oding problem with a helper [40], [41] may seem to be related to our approah, sine the side information of older versions that is used to deode the latest ommon version in our approah may be interpreted as helpers. In an optimal strategy for the helper setting, the helper side information is enoded via a joint typiality enoding sheme, whereas the random binning is used for the message. However, note that in the multi-version oding setting, the versions that may be a side information for one state may required to be deoded in another state. For this reason, a random binning sheme for all versions - even those that may be helpers in ertain states - leads to shemes with a reasonable storage ost. Before proving Theorem 3, we introdue the following useful definitions.

15 15 Definition 6 δ -possible Set of Tuples). The set A δ of δ -possible set of tuples w u1, w u2,, w ul ) is defined as follows where u 1 < u 2 < < u L. A δ W u1, W u2,, W ul ) = {w u1, w u2,, w ul ) : w u1 [2 ], w u2 Bw u1, δ u 2 u 1 ))}, w u3 Bw u2, δ u 3 u 2 )),, w ul Bw ul 1, δ u ul u ul 1 ))}, 19) We omit the dependeny on the messages and simply write A δ, when it is lear from the ontext. Similiarly, we an also define the set of possible tuples w F1 given a partiular tuple w F2, A δ W F1 w F2 ), where F 1, F 2 be two subsets of {u 1, u 2,, u L }. We next provide a proof of Theorem 3. Proof of Theorem 3. We first desribe the random binning onstrution [15]. Constrution 3 Random binning multi-version ode). Random ode generation: For a version s j the enoder assigns an index at random from {1, 2,, 2 Rs j } uniformly and independently to eah sequene of length bits. The set of sequenes whih have the same index form a bin. Enoding: The server stores the orresponding bin index to eah version that it reeives. The deoder is also aware of the mapping used in the binning sheme. Assume that the i-th server reeives the set of ordered versions S = {s 1, s 2,, s S } [ν], where s 1 < s 2 < < s S. The enoding funtion of the i-th server is defined as follows where ϕ i) S = ϕi) s 1, ϕ i) s 2,, ϕ i) s S ), 20) ϕ i) s j : [2 ] {1, 2, 2 Rs j }, for j {1, 2,, S }, where R sj / is the ompression rate of version s j. In partiular, we hoose the rates as follows R s1 = + s 1 1) log V olδ, ) + s 1 1) log ɛ, 21) R sj = s j s j 1 ) log V olδ, ) + s j 1) log ɛ, j {2, 3,, S }. 22) Deoding: For every set S P[ν]) n and set T = {t 1, t 2,, t } [n] of servers, the deoder employs the possible set deoding strategy that we will explain next to reover the latest ommon version among those servers. Suppose that a version s j, j S, is reeived by a set of servers {i 1, i 2,, i r } [n], then the bin index orresponding to this version is given by ϕ sj = ϕ i1) s j, ϕ i2) s j,, ϕ ir) s j ). Assume that W ul is the latest ommon version among these servers and that the versions W u1, W u2,, W ul 1 have been reeived by some servers out of those servers before reeiving W ul. We define this set of versions formally as follows ) S T = {u 1, u 2,, u L } = St) \ {u L + 1, u L + 2,, ν}, t T where u 1 < u 2 < < u L. Given the bin indies b u1, b u2,, b ul ), the deoder first finds all tuples w u1, w u2,, w ul ) suh that ϕ u1 w u1 ) = b u1, ϕ u2 w u2 ) = b u2,, ϕ ul w ul ) = b ul )

16 16 and w u1, w u2,, w ul ) A δ. If all of these tuples have the same latest ommon version w ul, the deoder delares ŵ ul = w ul. Otherwise, the deoder delares an error. Denoting E as the event that there is an error, we an write E = { w u 1, w u 2,..., w u L ) A δ : w u L W ul and ϕ u w u) = ϕ u W u ) u S T }. The error event in deoding an be equivalently expressed as follows: E = E I, 23) where I S T :u L I E I = { w u W u u I : ϕ u w u) = ϕ u W u ) u I and w I, W ST \I) A δ }, 24) for I S T. By the union bound, the probability of error in deoding the latest ommon version among these servers is upper-bounded as follows P e = P E) = P I S T :u L I I S T :u L I E I P E I ), and we require that P e < ɛ. Therefore, we require the following for every I S T suh that u L I P E I ) < ɛ2 L 1), 25) We now proeed in a ase by ase manner. We first onsider the ase where u L 1 / I, later we will onsider the ase where u L 1 I. For the ase where u L 1 / I, we have Consequently, we have E I Ẽu L 1 := { w u L W ul : ϕ ul w u L ) = ϕ ul W ul ) and we an upper-bound P Ẽu L 1 ) as follows P Ẽu L 1 ) = w ul 1,w ul ) and W ul 1, w u L ) A δ }. 26) pw ul 1, w ul ) P E I ) < P Ẽu L 1 ), 27) P w u L w ul : ϕ ul w u L ) = ϕ ul w ul ) and w ul 1, w u L ) A δ ) pw ul 1, w ul ) P ϕ ul w u L ) = ϕ ul w ul )) w ul 1,w ul ) w ul 1,w ul ) w ul 1,w ul ) w u L w ul w ul 1,w u L ) A δ pw ul 1, w ul )2 Ru L Aδ W ul w ul 1 ) pw ul 1, w ul )2 Ru log V olul ul 1)δ,) L 2 Ru L log V olul ul 1)δ,)). 28)

17 17 Choosing R ul to satisfy R ul log V olu L u L 1 )δ, ) + L 1) log ɛ ensures that P E I ) < ɛ2 L 1). Now, we onsider the ase where u L 1 I. In this ase, we onsider the following two ases. First, we onsider the ase where u L 2 / I, later will onsider the ase where u L 2 I. For the ase where u L 2 / I, we have Therefore, we have E I Ẽu L 2 := { w u L 1 W ul 1, w u L W ul : ϕ ul 1 w u L 1 ) = ϕ ul 1 W ul 1 ), ϕ ul w u L ) = ϕ ul W ul ) and W ul 2, w u L 1, w u L ) A δ }. 29) and we an upper-bound P Ẽu L 2 ) as follows P Ẽu L 2 ) w ul 2,w ul 1,w ul ) w u L 1 w ul 1 w u L w ul w ul 2,w u,w u L 1 L ) A δ w ul 2,w ul 1,w ul ) P E I ) < P Ẽu L 2 ), 30) pw ul 2, w ul 1, w ul ) P ϕw u L 1 ) = ϕw ul 1 ))P ϕ ul w u L ) = ϕw ul )) pw ul 2, w ul 1, w ul )2 Ru L 1 +Ru L ) A δ W ul 1, W ul w ul 2 ) pw ul 2, w ul 1, w ul )2 Ru L 1 +Ru ) L w ul 2,w ul 1,w ul ) 2 log V olul ul 1)δ,)+log V olul 1 ul 2)δ,). 31) Hene, we require the following R ul 1 + R ul ) L j=l 1 log V olu j u j 1 )δ, ) + L 1) log ɛ. We next onsider the other ase where u L 2 I. In this ase, we also have two ases based on whether u L 3 is in I or not. By applying the above argument repeatedly, we obtain the following onditions for the overall probability of error to be upper bounded by ɛ. L L log V olu j u j 1 )δ, ) + L 1) log ɛ, j=i R uj j=i i {2, 3,, L}. L L R uj + log V olu j u j 1 )δ, ) + L 1) log ɛ. 32) j=1 j=2