On the Locality of Codeword Symbols

Transcription

1 On the Locality of Codewod Symbols Paikshit Gopalan Micosoft Reseach Cheng Huang Micosoft Reseach Segey Yekhanin Micosoft Reseach Huseyin Simitci Micosoft Copoation Abstact Conside a linea [n, k, d] q code C. We say that that i-th coodinate of C has locality, if the value at this coodinate can be ecoveed fom accessing some othe coodinates of C. Data stoage applications equie codes with small edundancy, low locality fo infomation coodinates, lage distance, and low locality fo paity coodinates. In this pape we cay out an in-depth study of the elations between these paametes. We establish a tight bound fo the edundancy n k in tems of the message length, the distance, and the locality of infomation coodinates. We efe to codes attaining the bound as optimal. We pove some stuctue theoems about optimal codes, which ae paticulaly stong fo small distances. This gives a faily complete pictue of the tadeoffs between codewods length, wost-case distance and locality of infomation symbols. We then conside the locality of paity check symbols and easue coection beyond wost case distance fo optimal codes. Using ou stuctue theoem, we obtain a tight bound fo the locality of paity symbols possible in such codes fo a boad class of paamete settings. We pove that thee is a tadeoff between having good locality fo paity checks and the ability to coect easues beyond the minimum distance. 1 Intoduction Moden lage scale distibuted stoage systems such as data centes stoe data in a edundant fom to ensue eliability against node (e.g., individual machine) failues. The simplest solution hee is the staightfowad eplication of data packets acoss diffeent nodes. Altenative solution involves easue coding: the data is patitioned into k infomation packets. Subsequently, using an easue code, n k paity packets ae geneated and all n packets ae stoed in diffeent nodes. Using easues codes instead of eplication may lead to damatic impovements both in tems of edundancy and eliability. Howeve to ealize these impovements one has to addess the challenge of maintaining an easue encoded epesentation. In paticula, when a node stoing some packet fails, one has to be able to quickly econstuct the lost packet in ode to keep the data eadily available fo the uses and to maintain the same level of edundancy in the system. We say that a cetain packet has locality if it can be ecoveed fom accessing only othe packets. One way to ensue fast econstuction is to use easue codes whee all packets have low locality k. Having small value of locality is paticulaly impotant fo infomation packets. These consideations lead us to intoduce the concept of an (, d)-code, i.e., a linea code of distance d, whee all infomation symbols have locality at most. Stoage system based on (, d)-codes 1

2 povide fast ecovey of infomation packets fom a single node failue (typical scenaio), and ensue that no data is lost even if up to d 1 nodes fail simultaneously. One specific class of (, d)-codes called Pyamid Codes has been consideed in [5]. Pyamid codes can be obtained fom any systematic Maxmimum Distance Sepeable (MDS) codes of distance d, such as Reed Solomon codes. Assume fo simplicity that the fist paity check symbol is the sum k x i of the infomation symbols. Replace this with k paity checks each of size at most on disjoint infomation symbols. It is not had to see that the esulting code C has infomation locality and distance d, while the edundancy of the code C is given by k n k = + d 2. (1) 1.1 Ou esults In this pape we cay out an in-depth study of the elations between edundancy, easue-coection and symbol locality in linea codes. Ou fist esult is a tight bound fo the edundancy in tems of the message length, the distance, and the infomation locality. We show that in any [n, k, d] q code of infomation locality, k n k + d 2. (2) We efe to codes attaining the bound above as optimal. Pyamid codes ae one such family of codes. The bound (2) is of paticula inteest in the case when k, since othewise one can impove the code by inceasing the dimension while maintaining the (, d)-popety and edundancy intact. A close examination of ou lowe bound gives a stuctue theoem fo optimal codes when k. This theoem is especially stong when d < + 3, it fixes the suppot of the paity check matix, the only feedom is in the choice of coefficients. We also show that the condition < d + 3 is in fact necessay fo such a stong statement to hold. We then tun ou attention to the locality of paity symbols. We pove tight bounds on the locality of paity symbols in optimal codes assuming d < + 3. In paticula we establish the existence of optimal (, d)-codes that ae significantly bette than pyamid codes with espect to locality of paity symbols. Ou codes ae explicit in the case of d = 4, and non-explicit othewise. The lowe bound is poved using the stuctue theoem. Finally, we elax the conditions d < + 3 and k and exhibit one specific family of optimal codes that gives locality fo all symbols. Ou last esult concens easue coection beyond the wost case distance of the code. Assume that we ae given a bipatite gaph which descibes the suppots of the paity check symbols. What choice of coefficients will maximize the set of easue pattens that can be coected by such a code? In [5] the authos gave a necessay condition fo an easue patten to be coectable, and showed that ove sufficiently lage fields, this condition is also sufficient. They called such codes Genealized Pyamid codes. We show that such codes cannot have any non-tivial paity locality; thus establishing a tadeoff between paity locality and easue coection beyond the wost case distance. 1.2 Related wok Thee ae two classes of easue codes poviding fast ecovey pocedues fo individual codewod coodinates (packets) in the liteatue. 2

3 Regeneating codes. These codes wee intoduced in [2] and developed futhe in e.g., [8, 1]. See [3] fo a suvey. One cucial idea behind egeneating codes is that of sub-packetization. Each packet is composed of few sub-packets, and when a node stoing a packet fails all (o most of) othe nodes send in some of thei sub-packets fo ecovey. Efficiency of the ecovey pocedue is measued in tems of the oveall bandwidth consumption, i.e., the total size of sub-packets equied to ecove fom a single failue. Somehow supisingly egeneating codes can in many cases achieve a athe significant eduction in bandwidth, compaed with codes that do not employ sub-packetization. Ou expeience with data centes howeve suggests that in pactice thee is a consideable ovehead elated to accessing exta stoage nodes. Theefoe pue bandwidth consumption is not necessaily the ight single measue of the ecovey time. In paticula, coding solutions that do not ely on sub-packetization and thus access less nodes (but download moe data) ae sometimes moe attactive. Locally decodable codes. These codes wee intoduced in [6] and developed futhe in e.g., [10, 4, 7]. See [11] fo a suvey. An -quey Locally Decodable Code (LDC) encodes messages in such a way that one can ecove any message symbol by accessing only codewod symbols even afte some abitaily chosen (say) 10% of codewod coodinates ae eased. Thus LDCs ae in fact vey simila to (, d)-codes addessed in the cuent pape, with an impotant distinction that LDCs allow fo local ecovey even afte a vey lage numbe of symbols is eased, while (, d)-codes povide locality only afte a single easue. Not supisingly locally decodable codes equie substantially lage codewod lengths then (, d)-codes. 1.3 Oganization In section 3 we establish the lowe bound fo edundancy of (, d)-codes and obtain a stuctual chaacteization of optimal codes, i.e., codes attaining the bound. In section 4 we stengthen the stuctual chaacteization fo optimal codes with d < + 3 and show that any such code has to be a canonical code. In section 5 we pove matching lowe and uppe bounds on the locality of paity symbols in canonical codes. Ou code constuction is not explicit and equies the undelying field to be faily lage. In the special case of codes of distance d = 4, we come up with an explicit family that does not need a lage field. In section 6 we pesent one optimal family of non-canonical codes that gives unifom locality fo all codewod symbols. Finally, in section 7 we study easue coection beyond the wost case distance and pove that systematic codes coecting the maximal numbe of easue pattens (conditioned on the suppot stuctue of the geneato matix) cannot have any non-tivial locality fo paity symbols. 2 Peliminaies We use standad mathematical notation Fo an intege t, [t] = {1,..., t}; Fo a vecto x, Supp(x) denotes the set {i : x i 0}; Fo a vecto x, wt(x) = Supp(x) denotes the Hamming weight; Fo a vecto x and an intege i, x(i) denotes the i-th coodinate of x; Fo sets A and B, A B denotes the disjoint union. 3

4 Let C be an [n, k, d] q linea code. Assume that the encoding of x F k q is by the vecto C(x) = (c 1 x, c 2 x,..., c n x) F n q. (3) Thus the code C is specified by the set of n points C = {c 1,..., c n } F k q. The set of points must have full ank fo C to have k infomation symbols. It is well known that the distance popety is captued by the following condition (e.g., [9, theoem 1.1.6]). Fact 1. The code C has distance d if and only if fo evey S C such that Rank(S) k 1, S n d. (4) In othe wods, evey hypeplane though the oigin misses at least d points fom C. In this wok, we ae inteested in the ecovey cost of each symbol in the code fom a single easue. Definition 2. Fo c i C, we define Loc(c i ) to be the smallest intege fo which thee exists R C of cadinality such that c i = j R λ j c j. We futhe define Loc(C) = max i [n] Loc(c i ). Note that Loc(c i ) k, povided d 2, since this guaantees that C \ {c i } has full dimension. We will be inteested in (systematic) codes which guaantee locality fo the infomation symbols. Definition 3. We say that a code C has infomation locality if thee exists I C of full ank such that Loc(c) fo all c I. Fo such a code we can choose I as ou basis fo F k q and patition C into I = {e 1,..., e k } coesponding to infomation symbols and C \ I = {c k+1,..., c n } coesponding to paity check symbols. Thus the code C can be made systematic. Definition 4. A code C is an (, d)-code if it has infomation locality and distance d. Fo any code C, the set of all linea dependencies of length at most + 1 on points in C defines a natual hypegaph H (V, E) whose vetex set V = [n] is in one-to-one coespondence to points in C. Thee is an edge coesponding to set S V if S + 1 λ i c i = 0, λ i 0. i S Equivalently S [n] is an edge in H if it suppots a codewod in C of weight at most + 1. Since will usually be clea fom the context, we will just say H(V, E). A code C has locality if thee ae no isolated vetices in H. A code C has infomation locality if the set points coesponding to vetices that ae incident to some edge in H has full ank. We conclude this section pesenting one specific class of (, d)-codes has been consideed in [5]: Pyamid codes. In what follows the dot poduct of vectos p and x is denoted by p x. To define an (, d) pyamid code C encoding messages of dimension k we fix an abitay linea systematic [k + d 1, k, d] q code E. Clealy, E is MDS. Let E(x) = (x, p 0 x, p 1 x,..., p d 2 x). 4

5 We patition the set [k] into t = k subsets of size up to, [k] = i [t] S i. Fo a k-dimensional vecto x and a set S [k] let x S denote the S -dimensional estiction of x to coodinates in the set S. We define the systematic code C by C(x) = (x, (p 0 S1 x S1 ),..., (p 0 St x St ), p 1 x,..., p d 2 x). It is not had to see that the code C has distance d. To see that all infomation symbols and the fist k paity symbols of C have locality one needs to obseve that (since E is an MDS code) the vecto p 0 has full Hamming weight. The last d 2 paity symbols of C may have locality as lage as k. 3 Lowe Bound and the Stuctue Theoem We ae inteested in systematic codes with infomation locality. Given k,, d ou goal is to minimize the codewod length n. Since the code is systematic, this amounts to minimizing the edundancy h = n k. Pyamid codes have h = k + d 2. Ou goal is to pove a matching lowe bound. Lowe bounds of k/ and d 1 ae easy to show, just fom the locality and distance constaints espectively. The had pat is to sum them up. Theoem 5. Fo any [n, k, d] q linea code with infomation locality, k n k + d 2. (5) Poof. Ou lowe bound poceeds by constucting a lage set S C whee Rank(S) k 1 and then applying Fact 1. The set S is constucted by the following algoithm: 1. Let i = 1, S 0 = {}. 2. While Rank(S i 1 ) k 2: 3. Pick c i C \ S i 1 such that thee is a hypeedge T i in H containing c i. 4. If Rank(S i 1 T i ) < k, set S i = S i 1 T i. 5. Else pick T T i so that Rank(S i 1 T ) = k 1 and set S i = S i 1 T. 6. Incement i. In Line 3, since Rank(S i 1 ) k 2 and Rank(I) = k, thee exists c i as desied. Let l denote the numbe of times the set S i is gown. Obseve that the final set S l has Rank(S l ) = k 1. We now lowe bound S. We define s i, t i to measue the incease in the size and ank of S i espectively: s i = S i S i 1, S l = l s i, t i = Rank(S i ) Rank(S i 1 ), Rank(S l ) = l t i = k 1. We analyze two cases, depending on whethe the condition Rank(S i 1 T i ) = k is eve eached. Obseve that this condition can only be eached when i = l. Case 1: Assume Rank(S i 1 T i ) k 1 thoughout. In each step we add s i + 1 vectos. Note that these vectos ae always such that some nontivial linea combination of them yields a (possibly zeo) vecto in Span(S i 1 ). Theefoe we have t i s i 1. So thee ae l k 1 steps in all. Thus l l k 1 S = s i (t i + 1) k 1 + (6) 5

6 Note that k 1 + k 1 k + k 2 with equality holding wheneve = 1 o k 1 mod. Case 2: In the last step, we hit the condition Rank(S l 1 T l ) = k. Since the ank only inceases by pe step, l k. Fo i l 1, we add a set Ti of s i + 1 vectos. Again note that these vectos ae always such that some nontivial linea combination of them yields a (possibly zeo) vecto in Span(S i 1 ). Theefoe Rank(S i ) gows by t i whee t i s i 1. In Step l, we add T T l to S. This inceases Rank(S) by t l 1 (since Rank(S) k 2 at the stat) and S by s l t l. Thus S = l l 1 s i (t i + 1) + t l = k + The conclusion now follows fom Fact 1 which implies that S n d. k 2. (7) Definition 6. We say that an (, d)-code C is optimal if its paametes satisfy (5) with equality. Pyamid codes [5] yield optimal (, d)-codes fo all values of, d, and k when the alphabet q is sufficiently lage. The poof of theoem 5 eveals infomation about the stuctue of optimal (, d)-codes. We think of the algoithm as attempting to maximize S l Rank(S) = s i l t. i With this in mind, at step i we can choose c i such that s i t i is maximized. An optimal length code should yield the same value fo S fo this (o any) choice of c i. This obsevation yields an insight into the stuctue of local dependencies in optimal codes, as given by the following stuctue theoem. Theoem 7. Let C be an [n, k, d] q code with infomation locality. Suppose k, < k, and n = k + k + d 2; (8) then hypeedges in the hypegaph H(V, E) ae disjoint and each has size exactly + 1. Poof. We execute the algoithm pesented in the poof of theoem 5 to obtain a set S and sequences {s i } and {t i }. We conside the case of = 1 sepaately. Since all t i 1 we fall into Case 1. Combining fomulas (6), (4) and (8) we get S = l s i = l t i + l = 2k 2. Combining this with l t i = k 1 we conclude that l = k 1, all s i equal 2, and all t i equal 1. The latte two conditions peclude the existence of hypeedges of size 1 o intesecting edges in H. We now poceed to the case of > 1. When k, the bound in equation (6) is lage than that in equation (7). Thus, we must be in Case 2. Combining fomulas (7), (4) and (8) we get S = l s i = l t i + l 1 = k + k 2. 6

7 Obseve that l t i = k 1 and thus l = k. Togethe with the constaint t i, this implies that t j = 1 fo some j [l] and t i = fo i j. We claim that in fact j = l. Indeed, if j < l, we would have i l 1 t i = k 1 and t l =, hence we would be in Case 1. Now assume that thee is an edge T with T. By adding this edge to S at the fist step, we would get t 1 1. Next assume that T 1 T 2 is non-empty. Obseve that this implies Rank(T 1 T 2 ) < 2. So if we add edges T 1 and T 2 to S, we have t 1 + t Clealy these conditions lead to contadiction if l = k 3. In fact, they also give a contadiction fo k = 2, since they put us in Case 1. 4 Canonical Codes The stuctue theoem implies then when d is sufficiently small (which in ou expeience is the setting of inteest in most data stoage applications), optimal (, d)-codes have athe igid stuctue. We fomalize this by defining the notion of a canonical code. Definition 8. Let C be a systematic [n, k, d] q code with infomation locality whee k, < k, and n = k + k + d 2. We say that C is canonical if the set C can patitioned into thee goups C = I C C such that: 1. Points I = {e 1,..., e k }. 2. Points C = {c 1,..., c k/ } whee wt(c i ) =. The suppots of these vectos ae disjoint sets which patition [k]. 3. Points C = {c 1,..., c d 2 } whee wt(c i ) = k. Clealy any canonical code is systematic and has infomation locality. The distance popety equies a suitable choice of vectos {c } and {c }. Pyamid codes [5] ae an example of canonical codes. We note that since < k, thee is always a distinction between symbols {c } and {c }. Theoem 9. Assume that d < + 3, < k, and k. Let n = k + k + d 2. Evey systematic [n, k, d] q code with infomation locality is a canonical code. Poof. Let C be a systematic [n, k, d] code with infomation locality. We stat by showing that the hypegaph H(V, E) has k edges. Since C is systematic, we know that I = {e 1,..., e k } C. By theoem 7, H(V, E) consists of m disjoint, ( + 1)-egula edges and evey vetex in I appeas in some edge. But since the points in I ae linealy independent, evey edges involves at least one vetex fom C \ I and at most fom I. So we have m k. We show that equality holds. Assume fo contadiction that m k + 1. Since the edges ae egula and disjoint, we have n m( + 1) = k + k > k + k + d 2 which contadicts the choice of n. Thus m = k. This means that evey edge T i is incident on exactly vetices e i1..., e i fom I and one vetex c i outside it. Hence c i = λ ij e ij. j=1 7

8 Since the T i s ae disjoint, the vectos c 1,..., c k/ have disjoint suppots which patition [k]. We now show that the emaining vectos c 1,..., c d 2 must all have wt(c ) = k. Fo this, we conside the encoding of e j. We note e i e j 0 iff i = j and c i e j 0 iff j Supp(c i ). Thus only 2 of these inne poducts ae non-zeo. Since the code has distance d, all the d 2 inne poducts c i e j ae non-zeo. This shows that wt(c i ) = k fo all i. The above bound is stong enough to sepaate having locality of fom having just infomation locality of. The following coollay follows fom the obsevation that the hypegaph H(V, E) must contain n k ( + 1) = d 2 isolated vetices, which do not paticipate in any linea elations of size + 1. Coollay 10. Assume that 2 < d < + 3 and k. Let n = k + k + d 2. Thee ae no [n, k, d] q linea codes with locality. 5 Canonical codes: paity locality Theoem 9 gives a vey good undestanding optimal (, d)-codes in the case < d+3 and k. Fo any such code the coodinate set C = {c i } i [n] can be patitioned into sets I, C, C whee fo all c I C, Loc(c) =, and fo all c C, Loc(c ) >. It is natual to ask how low can the locality of symbols c C be. In this section we addess and esolve this question. 5.1 Paity locality lowe bound We begin with a lowe bound. Theoem 11. Let C be a systematic optimal (, d)-code with paametes [n, k, d] q. Suppose d < + 3, < k, and k. Then some k paity symbols of C have locality exactly, and d 2 othe paity symbols of C have locality no less than ( ) k k 1 (d 3). (9) Poof. Theoem 9 implies that C is a canonical code. Let C = I C C be the canonical patition of the coodinates of C. Clealy, fo all k symbols c C we have Loc(c ). We now pove lowe bounds on the locality of symbols in C C. We stat with ( symbols ) c C. Fo evey j [k/] we define a subset R j C that we call a ow. Let S j = Supp c j. The j-th ow contains the vecto c j, all unit vectos in the suppot of c j and the set C. R j = {c j} i S j e i C. Obseve that esticted to I C ows {R j } j [k/] fom a patition. Conside an abitay symbol c C. Let l = Loc(c ). We have c = c i, (10) i L whee L = l. In what follows we show that fo each ow R j, R j L (11) 8

9 needs to hold. It is not had to see that this togethe with the stuctue of the sets {R j } implies inequality (9). To pove (11) we conside the code C j = {C(x) x F k q such that Supp(x) S j }. (12) It is not had to see that Supp(C j ) = R j and dim C j =. Obseving that the distance of the code C j is at least d and R j = + d 1 we conclude that (esticted to its suppot) C j is an MDS code. Thus any symbols of C j ae independent. It emains to note that (10) esticted to coodinates in S j yields a non-tivial dependency of length at most R j L + 1 between the symbols of C j. We poceed to the lowe bound on the locality of symbols in C. Fix an abitay c j C. A easoning simila to the one above implies that if Loc(c j ) < ; then thee is a dependency of length below + 1 between the coodinates of the [ + d 1,, d] q code C j (defined by (12)) esticted to its suppot. Obseve that the bound (9) is close to k only when is lage and d is small. In othe cases theoem 11 does not ule out existence of canonical codes with low locality fo all symbols (including those in C ). In the next section we show that such codes indeed exist. In paticula we show that the bound (9) can be always met with equality. 5.2 Paity locality uppe bounds Ou main esults in this section ae given by theoems 15 and 16. Theoem 15 gives a geneal uppe bound matching the lowe bound of theoem 11. The poof is not explicit. Theoem 16 gives an explicit family of codes in the naow case of d = 4. We stat by intoducing some concepts we need fo the poof of theoem 15. Definition 12. Let L F n q be a linea space and S [n] be a set, S = k. We say that S is a k-coe fo L if fo all vectos v L, Supp(v) S. It is not had to veify that S is a k-coe fo L, if and only if S is a subset of some set of infomation coodinates in the space L. In othe wods S is a k-coe fo L, if and only k columns in the (n dim L)- by-n geneato matix of L that coespond to elements of S ae linealy independent. Definition 13. Let L F n q be a linea space. Let {c 1,..., c n } be a sequence of n vectos in F k q. We say that vectos {c i } ae in geneal position subject to L if the following conditions hold: 1. Fo all vectos v L we have n v(i)c i = 0; 2. Fo all k-coes S of L we have Rank ({c i } i S ) = k. The next lemma assets existence of vectos that ae in geneal position subject to an abitay linea space povided the undelying field is lage enough. Lemma 14. Let L F n q be a linea space and k be a positive intege. Suppose q > kn k ; then thee exists a family of vectos {c i } i [n] in F k q that ae in geneal position subject to L. Poof. We obtain a matix M F k n q picking the ows of M at andom (unifomly and independently) fom the linea space L. We choose vectos {c i } to be the columns of M. Obseve that the fist condition in definition 13 is always satisfied. Futhe obseve that ou choice of M induces a unifom distibution on evey set of k columns of M that fom a k-coe. The second condition in definition 13 9

10 is satisfied as long as all k-by-k minos of M that coespond to k-coes ae invetible. This happens with pobability at least ( ( n ) k ( ) ) ( ( n ) ( k q i ) ) k 1 n k k k q q > 0. This concludes the poof. We poceed to the main esult of this section. Theoem 15. Let 2 < d < + 3, < k, k. Let q > kn k be a pime powe. Let n = k + k + d 2. Thee exists a systematic [n, k, d] q code C of infomation locality, whee k paity symbols have locality, and d 2 othe paity symbols have locality k ( k 1) (d 3). Poof. Let t = k. Fix some t + 1 subsets P 0, P 1,..., P t of [n] subject to the following constaints: 1. P 0 = k (t 1) (d 3) + 1; 2. Fo all i [t], P i = + 1; 3. Fo all i, j [t] such that i j, P i P j = ; 4. Fo all i [t], P 0 P i = d + 3. Fo evey set P i, 0 i t we fix a vecto v i F n q, such that Supp(v i ) = P i. We ensue that non-zeo coodinates of v 0 contain the same value. We also ensue that fo all i [t] non-zeo coodinates of v i contain distinct values. The lowe bound on q implies that these conditions can be met. Fo a finite set A let A denote a set that is obtained fom A by dopping at most one element. Note that fo all i [t] and all non-zeo α, β in F q we have Supp(αv 0 + βv i ) = (P 0 \ P i ) (P 0 P i ) (P i \ P 0 ). (13) Conside the space L = Span ({v i } 0 i t ). Let M = P 0 \ t P i. Obseve that M = k (t 1)(d 3) + 1 t( d + 3) = d 2. By (13) fo any v L we have P i, i T Supp(v) = M i [n]\t fo some T [t] OR (P 0 P i ) (P 0 P i ) (P i \ P 0 ) fo some T [t]. i T Obseve that a set K [n], K = k is a k-coe fo L if and only if fo all i [t], P i K and M K; OR [ Pi P M K and i [t] such that 0 K < d + 2; OR P i P 0 K = d + 2 and P i \ P 0 K. i T (14) (15) Let I [n] be such that M I = and fo all i [t], I P i =. By (15) I is a k-coe fo L. We use lemma 14 to obtain vectos {c i } i [n] F k q that ae in geneal position subject to the space L. We choose vectos {c i } i I as ou basis fo F k q and conside the code C defined as in (3). 10

11 ( In it not had to see that C is a systematic code of infomation locality. All t paity symbols in the set i [t] i) P \ I also have locality. Futhemoe all d 2 paity symbols in the set M have locality k (t 1)(d 3). It emains to pove that the code C has distance d = n k t + 2. (16) Accoding to Fact 1 the distance of C equals n S whee S [n] is the lagest set such that vectos {c i } i S do not have full ank. By definition 13 fo any k-coe K of L we have Rank{c i } i K = k. Thus in ode to establish (16) it suffices to show that evey set S [n] of size k + t 1 contains a k-coe of L. Ou poof involves case analysis. Let S [n], S = k + t 1 be an abitay set. Set b = #{i [t] P i S}. Note that since t( + 1) > S we have b t 1. Case 1: M S. We dop t 1 elements fom S to obtain a set K S, K = k such that fo all i [t], P i K. By (15) K is a k-coe. Case 2: M S and b t 2. We dop t 1 elements fom S to obtain a set K S, K = k such that M K and fo all i [t], P i K. By (15) K is a k-coe. Case 3: M S and b = t 1. Let i [t] be such that P i S. Such i is unique. Obseve that P i S = k + t 1 (d 2) (t 1)( + 1) = d + 2. Also obseve that P i \ P 0 = + 1 ( d + 3) = d 2 1. Combining the last two obsevations we conclude that eithe [ Pi P 0 S < d + 2; OR (17) P i P 0 S = d + 2 and P i \ P 0 S. Finally, we dop t 1 elements fom S to obtain a set K S, K = k such that fo all i [t], P i K. By (15) and (17) K is a k-coe. Theoem 15 gave a geneal constuction of (, d)-codes that ae optimal not only with espect to infomation locality and edundancy but also with espect to locality of paity symbols. That theoem howeve is weak in two espects. Fistly, the constuction is not explicit. Secondly, the constuction equies a lage undelying field. The next theoem gives an explicit constuction that woks even ove small fields in the naow case of codes of distance 4. Theoem 16. Let < k, k be positive integes. Let q + 2 be a pime powe. Let n = k + k + 2. Thee exists a systematic [n, k, 4] q code C of infomation locality, whee k paity symbols have locality, and 2 othe paity symbols have locality k k + 1. Poof. Fix an abitay systematic [ + 3,, 4] q code E. Fo instance, one can choose E to be a Reed Solomon code. Let E(y) = (y, p 0 y, p 1 y, p 2 y). Since E is a MDS code all vectos {p i } have weight. Thus fo some non-zeo {α j } j [] we have 1 p 1 = α j e j + α p 2, (18) j=1 11

12 whee {e j } j [] ae the -dimensional unit vectos. To define a systematic code C we patition the input vecto x F n q into t = k vectos y 1,..., y t F q. We set C(x) = ( y 1,..., y t, p 0 y 1,..., p 0 y t, (p 1 ) y i, (p 2 )) y i, (19) whee the summation is ove all i [t]. It is not had to see that the fist k + t coodinates of C have locality. We ague that the last two coodinates have locality k t + 1. Fom (18) we have (p 1 ) 1 ( y i = α j e j ) ( y i + α p 2 ) y i, j=1 whee the summation is ove all i [t]. Equivalently, (p 1 ) 1 y i = j=1 i [t] α j y i (j) + α ( p 2 y i ). Thus the next-to-last coodinate of C can be ecoveed fom accessing ( 1)t infomation coodinates and the last coodinate. Similaly, the last coodinate can be ecoveed fom k t infomation coodinates and the next-to-last coodinate. To pove that the code C has distance 4 we give an algoithm to coect 3 easues in C. The algoithm has two steps. Step 1: Fo evey i [t], we efe to a subset (y i, p 0 y i ) of + 1 coodinates of C as a block. We go ove all t blocks. If we encounte a block whee one symbol is eased, we ecove this symbol fom othe symbols in the block. Step 2: Obseve that afte the execution of Step 1 thee can be at most one block that has easues. If no such block exists; then on Step 1 we have successfully ecoveed all infomation symbols and thus we ae done. Othewise, let the unique block with easues be (y j, p 0 y j ) fo some j [t]. Since we know all vectos {y i } i j, i [t], fom (p 1 i [t] y i ) and (p 2 i [t] y i ) (if these symbols ae not eased) we ecove symbols p 1 y j and p 2 y j. Finally, we invoke the decoding pocedue fo the code E to ecove y j fom at most 3 easues in E(y j ) = (y j, p 0 y j, p 1 y j, p 2 y j ). 6 Non-Canonical Codes In this section we obseve that canonical codes detailed in sections 4 and 5 ae not the only family of optimal (, d)-codes. If one elaxes conditions of theoem 9 one can get othe families. One such family that yields unifom locality fo all symbols is given below. The (non-explicit) poof esembles the poof of theoem 15 albeit is much simple. Theoem 17. Let n, k,, and d 2 be positive integes. Let q > kn k be a pime powe. Suppose ( + 1) n and k n k = + d 2; then thee exists an [n, k, d] q code whee all symbols have locality. Poof. Let t = n +1. We patition the set [n] into t subsets P 1,..., P t each of size + 1. Fo evey i [t] we fix a vecto v i F n q, such that Supp(v i ) = P i. We set all non-zeo coodinates in vectos {v i } i [t] 12

13 to be equal to 1. We conside the linea space L = Span ( {v i } i [t] ). Fo evey any v L we have Supp(v) = i T P i fo some fo some T [t]. Obseve that a set K [n], K = k is a k-coe fo L if and only if fo all i [t], P i K. Also obseve that conditions of the theoem imply k n t. Theefoe k-coes fo L exist. We use lemma 14 to obtain vectos {c i } i [n] F k q that ae in geneal position subject to the space L. We conside the code C defined as in (3). In it not had to see that C has dimension k and locality fo all symbols. It emains to pove that the code C has distance k d = n k + 2. (20) Ou poof elies on Fact 1. Let S [n] be an abitay subset such that Rank{c i } i S < k. Clealy, no k-coe of L is in S. Let b = #{i [t] P i S}. We have S b k 1 since dopping b elements fom S (one fom each P i S) tuns S into an ( S b)-coe. We also have b k 1 since dopping one element fom each P i S gives us a b-coe in S. Combining the last two inequalities we conclude that k 1 S k + 1. Combining this inequality with the identity k 1 = k 1 and Fact 1 we obtain (20). 7 Beyond Wost-Case Distance In this section, codes ae assumed to be systematic unless othewise stated. They will have k infomation symbols and h = n k paity check symbols. 7.1 Genealized Pyamid Codes The suppots of the paity check symbols in a code can be descibed using a bipatite gaph. Moe geneally, we define the notion of a set of points with suppots matching a gaph G. Definition 18. Let G([k], [h], E) be a bipatite gaph. We say that c 1,..., c h F k q have suppots matching G if Supp(c j ) = Γ(j) fo all j [h] whee Γ(j) denotes the neighbohood of j in G. Given points c 1,..., c h, conside the k h matix C with columns c 1,..., c h. Fo I [k] and J [h], let C I,J denote the sub-matix of C with ows indexed by I and columns indexed by J. Definition 19. Points c 1,..., c h F k q with suppots matching G ae in geneal position if fo evey I [h] and J [k] such that thee is a pefect matching fom I to J in G, the sub-matix C I,J is invetible. Standad aguments show that ove sufficiently lage fields F q, choosing c 1,..., c h andomly fom the set of vectos with suppot matching G gives points in geneal position. Coming back to codes, the suppots of the paity checks define a bipatite gaph which we will call the suppot gaph. This is closely elated to but distinct fom the Tanne gaph. 13

14 Definition 20. Let C be a systematic code with point set C = {e 1,..., e k, c 1,..., c h }. The suppot gaph G([k], [h], E) of C is a bipatite gaph whee (i, j) E if e i Supp(c j ). Fo instance in any canonical (, d)-code, the suppot gaph is specified up to elabeling. Thee ae k vetices in V of degee coesponding to C C, whose neighbohood patitions the set U and d 2 vetices of degee k coesponding to C C. The minimum distance of such a code is exactly d, and hence thee ae some pattens of d easues that the code cannot coect. Howeve it is possible that the code could coect many pattens of easues of weight d and highe, fo a suitable choice of c i s. In geneal one could ask: among all codes with a suppot gaph G, which codes can coect the most easue pattens? A Pioi, it is unclea that thee should be a single code that is optimal in the sense that it coects the maximal possible set of pattens. As shown by [5] such codes do exist ove sufficiently lage fields. Conside a systematic code C with suppot gaph G([k], [h], E). Given I [k] and J [h], let Γ J (I) denote Γ(I) J (define Γ I (J) similaly). Conside a set of easues S T whee S [k] and T [h] ae the sets of infomation and paity check symbols espectively that ae eased. To coect these easues, we need to ecove the symbols coesponding to {e i : i S} fom the paity checks coesponding to {c j : j T = [h] \ T }. Fo this to be possible, a necessay condition is that fo evey S S, Γ T (S ) S. By Hall s theoem, this is equivalent to the existance of a matching in G fom S to T. We say that such a set of easues satisfies Hall s condition. Definition 21. A systematic code C with suppot gaph G is a genealized pyamid code if evey set of easues satisfying Hall s condition can be coected. We can ephase this definition in algebaic tems using the notion of points with specified suppots in geneal position. Theoem 22. [5] Let C be a systematic code with suppot gaph G. C is a genealized pyamid code iff c 1,..., c h ae in geneal position with suppots matching G. 7.2 The Tadeoff between Locality and Easue Coection Fo any paity check symbol c j, it is clea that Loc(c j ) wt(c j ) = deg(j). We will show that no bette locality is possible fo a genealized pyamid code. This esult elies on a chaacteization of the suppot of the vectos in the space V spanned by {c 1,..., c h } in tems of the gaph G. Let V denote the space spanned by {c 1,..., c h } which ae in geneal position with suppots matching G. Let Supp(V) 2 [k] denote the set of suppots of vectos in V. We give a necessay condition fo membeship in Supp(V). Ou condition is in tems of sets of coodinates that can be eliminated by combination of cetain c j s. Definition 23. Let c = j J µ jc j whee µ j 0. Let I = j J Γ(j) \ Supp(c). We say that the set I has been eliminated fom j J Γ(j). Theoem 24. Let {c 1,..., c h } be vectos with suppots matching G in geneal position. The set I can be eliminated fom j J Γ(j) only if Γ J (I ) > I fo evey I I. Poof. Let c = j J µ jc j whee µ j 0. Let I = j J Γ(j) \ Supp(c). Assume fo contadiction that thee exists Ĩ I whee Γ J(Ĩ) Ĩ. We will show that thee exists I Ĩ so that Γ J(I ) = I and that Γ J (I ) > I fo evey non-empty subset I I. 14

15 It suffices to pove the existence of I Ĩ whee Γ J(I ) = I ; the claim about subsets of I will then follow by taking a minimal such I. Obseve that evey i I must have Γ J (i) 2, since if i occus in exactly one S j, then it cannot be eliminated. Hence we must have Ĩ 2. One can constuct the set I stating fom a singleton and adding elements one at a time, giving a sequence I 1,..., I l = I. We claim that fo any l l, Γ J (I l ) I l Γ J (I l 1 ) I l 1 1. This holds since Γ J (I l ) can only incease on adding i to I l 1 while I l inceases by 1. Since Γ J (I 1 ) I 1 1 wheeas Γ(I l ) I l 0, we must have Γ J (I l ) I l = 0 fo some l l. Thus we have a set whee Γ J (I l ) = I l as desied. Since the set I satisfies Hall s matching condition, thee is a pefect matching fom I to J = Γ J (I ) in G. But this means that the sub-matix C I,J has full ank. On the othe hand, c = j µ jc j and I I = j Γ(j) \ Supp(c). Let π(c) denote the estiction of c onto the co-odinates in I. Then we have µ j π(c j ) = µ j π(c j ) = π(c) = 0. j J j J The fist equality holds because π(c j ) = 0 fo j Γ(I ), the second by lineaity of π and the last since π(c) = 0. Hence the vecto µ J = {µ j} j J lies in the kenel of C I,J which contadicts the assumption that it has full ank. This shows that the condition Γ J (I ) > I fo all I I is necessay. Coollay 25. If the set I can be eliminated fom j J Γ(j), then I J 1. If the field size q is sufficiently lage, the necessay condition given by theoem 24 is also sufficient. We defe the poof of this statement to Appendix A and pove ou lowe bound on the locality of genealized pyamid codes. Theoem 26. In a genealized pyamid code, Loc(c j ) = deg(j) fo all j [h]. Poof. Assume fo contadiction that Loc(c t ) deg(t) 1 fo some t [h]. Hence thee exist A [k] and B [h] (not containing t) such that c t = i A λ i e i + j B µ j c j. We have A = a, B = b and a + b deg(t) 1. Hence c t j B µ j c j = i A λ i e i. Thus we have eliminated at least deg(t) a b + 1 indices fom j B {t} Supp(c j ) using a linea combination of b + 1 vectos. By coollay 25 this is not possible fo vectos in geneal position. Refeences [1] Viveck R. Cadambe, Syed A. Jafa, and Hamed Maleki. Distibuted data stoage with minimum stoage egeneating codes - exact and functional epai ae asymptotically equally efficient. Axiv ,

16 [2] Alexandos G. Dimakis, Bighten Godfey, Yunnan Wu, Matin J. Wainwight, and Kannan Ramchandan. Netwok coding fo distibuted stoage systems. IEEE Tansactions on Infomation Theoy, 56: , [3] Alexandos G. Dimakis, Kannan Ramchandan, Yunnan Wu, and Changho Suh. A suvey on netwok codes fo distibuted stoage. Poceedings of the IEEE, 99: , [4] Klim Efemenko. 3-quey locally decodable codes of subexponential length. In 41st ACM Symposium on Theoy of Computing (STOC), pages 39 44, [5] Cheng Huang, Minghua Chen, and Jin Li. Pyamid codes: flexible schemes to tade space fo access efficiency in eliable data stoage systems. In 6th IEEE Intenational Symposium on Netwok Computing and Applications (NCA 2007), pages 79 86, [6] Jonathan Katz and Luca Tevisan. On the efficiency of local decoding pocedues fo eocoecting codes. In 32nd ACM Symposium on Theoy of Computing (STOC), pages 80 86, [7] Swastik Koppaty, Shubhangi Saaf, and Segey Yekhanin. High-ate codes with sublinea-time decoding. In 43nd ACM Symposium on Theoy of Computing (STOC), pages , [8] K. V. Rashmi, Niha B. Shah, and P. Vijay Kuma. Optimal exact-egeneating codes fo distibuted stoage at the MSR and MBR points via a poduct-matix constuction. Axiv , [9] Michael Tsfasman, Sege Vladut, and Dmity Nogin. Algebaic geometic codes: basic notions. Ameican Mathematical Society, Povidence, Rhode Island, USA, [10] Segey Yekhanin. Towads 3-quey locally decodable codes of subexponential length. Jounal of the ACM, 55:1 16, [11] Segey Yekhanin. Locally decodable codes. Foundations and tends in theoetical compute science, To appea. Peliminay vesion available fo download at now.pdf. A Spaces spanned by geneal position vectos Lemma 27. Let q n be a pime powe. The set of suppots of vectos in any linea space V F n q is closed unde union. Poof. Conside two vectos a and b in V with Supp(a) = S and Supp(b) = T. We may assume that S, T n 1 and that one set does not contain the othe. Now conside a + λb fo λ F q. It suffices to find λ such that a(i) + λb(i) 0 fo each i S T. This ules out at most S T n 2 values of λ, so thee is a solution povided q 1 > n 2 o q n. It is easy to see that the condition q n is tight by consideing the length 3 paity check code ove F 2, whee the set of suppots is not closed unde union. Theoem 28. Let q n. Let {c 1,..., c h } be vectos with suppots matching G in geneal position which span a space V. Supp(V) consists of all sets of the fom j J Γ(j) \ I whee I satisfies the condition Γ J (I ) > I fo evey I I. 16

17 Poof. Theoem 24 shows that the condition on I is necessay, we now show that it is sufficient. Fo j Γ(I) the sets Γ(j) and I ae disjoint. Hence we can wite j J Γ(j) \ I = ( j J Γ(I) Γ(j) \ I) ( j J Γ(I) Γ(j)). By the closue unde union, it suffices to pove the statement in the case when J Γ(I). Fix j 0 J and let J = J \ {j 0 }. Since Γ J (I ) > I we have Γ J (I ) I fo evey I I. So thee is a matching fom I to some subset J J whee J = I, and the matix C I,J is of full ank since the c j s ae in geneal position. Let π(c) denote the estiction of a vecto c onto coodinates in I. Since C I,J is invetible, the ow vectos {π(c j )} j J have full ank. Note that π(c j0 ) is not a zeo vecto since j 0 Γ(I). So thee exist {µ j } fo j J which ae not all 0 and π(c j0 ) = j J µ j π(c j ). Now conside the vecto c j 0 = c j0 j J µ jc j. Note that π(c j 0 ) is a zeo vecto, which shows that Supp(c j 0 ) j J {j 0 }Γ(j) \ I. We will show that equality holds by using coollay 25. Since we have eliminated I vectos, the linea combination must involve at least I + 1 vectos, which means that µ j 0 fo all j. Futhe the set of eliminated co-odinates cannot be lage than I, since this would violate coollay 25. Hence we have Supp(c j 0 ) = j J {j 0 }Γ(j) \ I. (21) By epeating this agument fo evey j 0 J, we will be able to find J(j 0 ) J of size I + 1 which contains j 0 and a vecto c j 0 such that Supp(c j 0 ) = j J(j0 )Γ(j) \ I. Using the closue unde union of suppots, we conclude that Supp(V) contains the set j0 J j J(j0 ) Γ(j) \ I = j J Γ(j) \ I. 17