Query Complexity Lower Bounds for Reconstruction of Codes

Transcription

1 Quey Complexity Lowe Bounds fo Reconstuction of Codes Souav Chakaboty Elda Fische Aie Matsliah Abstact We investigate the poblem of local econstuction, as defined by Saks and Seshadhi (2008), in the context of eo coecting codes. The fist poblem we addess is that of message econstuction, whee given an oacle access to a coupted encoding w {0, 1} n of some message x {0, 1} k ou goal is to pobabilistically ecove x (o some potion of it). This should be done by a pocedue (econstucto) that given an index i as input, pobes w at few locations and outputs the value of x i. The econstucto can (and indeed must) be andomized, but all its andomness is specified in advance by a single andom seed, such that with high pobability all k values x i fo 1 i k ae econstucted coectly. Using the econstucto as a filte allows to evaluate cetain classes of algoithms on x efficiently. Fo instance, in case of a paallel algoithm, one can initialize seveal copies of the econstucto with the same andom seed, and they can autonomously handle decoding equests while poducing outputs that ae consistent with the oiginal message x. Anothe example is that of adaptive queying algoithms, that need to know the value of some x i befoe deciding which index should be decoded next. The second poblem that we addess is codewod econstuction, which is similaly defined, but instead of econstucting the message ou goal is to econstuct the codewod itself, given an oacle access to its coupted vesion. Eo coecting codes that admit message and codewod econstuction can be obtained fom Locally Decodable Codes (LDC) and Self Coectible Codes (SCC) espectively. The main contibution of this pape is a poof that in tems of quey complexity, these ae close to be the best possible constuctions, even when we disegad the length of the encoding. 1 Intoduction Conside the following poblem: a lage data x {0, 1} k is stoed on a stoage device in encoded fom. A small faction of the stoed infomation may be coupted. Now suppose that we want to execute an algoithm M on x, but most likely M will need a only a small faction of x fo its execution. This can be the case if M is a single pocess in a lage paallelized system, o if M is a queying algoithm with limited memoy. M can even be adaptive, not knowing which bits of the input it will need in advance. Ideally, in all these cases M should have the ability to efficiently decode any bit of x only when the need aises. In paticula, decoding evey bit should be done by eading only a small faction of the coupted encoding w. To ensue coectness of the execution it is necessay that M succeeds (with high pobability) in coectly Centum Wiskunde & Infomatica, Amstedam. {souav.chakaboty, aiem}@cwi.nl Compute Science Faculty, Isael Institute of Technology (Technion). elda@cs.technion.ac.il. 1

2 decoding all bits that ae equied fo its execution. One way of ensuing this is to decode the whole message x in advance, but in many cases this may be vey inefficient, o even impossible. To pefom the above task, a message econstucto is equied, that can simulate quey access to x in a local manne. Concetely, message econstucto is an algoithm that can ecove the oiginal message x {0, 1} k fom a coupted encoding w {0, 1} n unde two main conditions: (locality) fo evey i [k] econstucting x i equies eading w only at vey few locations; (consistency) with high pobability, all indices i [k] should be econstucted coectly. When the consistency condition is weakened, so that only each index in itself is econstucted coectly with high pobability, the definition becomes equivalent to Local Decoding, as fomally defined in [KT00]. Infomally, a locally decodable code (LDC) is an eo-coecting code which allows to pobabilistically decode any symbol of an encoded message by pobing only few symbols of its coupted encoding. As in the case of LDCs, the main challenge in message econstuction is to find shot codes that allow econstuction of evey x i with as few queies to w as possible. Any q-quey LDC can be used fo O(q log k)-quey message econstuction, by simply epeating the local decoding pocedue O(log k) times (fo evey decoding equest x i ) and outputting the majoity vote. The epetition will educe the pobability of incoectly econstucting x i to 1/(3k), so that with pobability 2/3 all k indices ae econstucted coectly. Thus having O(1)-quey LDCs (e.g. the Hadamad code) immediately implies the existence of codes with an O(log k)-quey message econstucto. The question that immediately aises is whethe one can do bette, specifically in tems of quey complexity. The fist theoem of this pape (see Theoem 3.2) states that fo any encoding of any length, a non-adaptive message econstucto must make Ω(log k) queies pe decoding equest. Anothe family of eo coecting codes elated to LDCs ae self coectable codes (SCC) [BLR93]. In a q-quey self-coectable code the pobabilistic decode is equied to ecove an abitay symbol of the encoding itself. The second poblem that we study hee is of codewod econstuction, which is elated to SCCs in the same manne that message econstuction is elated to LDCs. Concetely, a codewod econstucto is an algoithm that can ecove a codewod y {0, 1} n fom its coupted vesion w {0, 1} n unde two main conditions: (locality) fo evey i [n] econstucting y i equies eading w only at vey few locations; (consistency) with high pobability, all indices i [n] should be econstucted coectly. Hee too, if the consistency condition is weakened, so that only each index in itself is econstucted coectly with high pobability, the definition becomes equivalent to self decoding; and any q-quey SCC can be used fo O(q log n)-quey codewod econstuction. Thus having O(1)-quey SCCs (e.g. the Hadamad code) immediately implies the existence of an O(log n)-quey codewod econstucto. The second theoem of this pape (see Theoem 3.4) gives lowe bounds on the quey complexity of codewod econstuction fo linea codes. Denoting by ˆn the numbe of distinct ows in the geneating matix of a linea code, Theoem 3.4 states that codewod econstuction equies Ω( log ˆn) queies pe decoding equest. Since essentially all known SCCs ae linea codes with ˆn = n, this bound is tight up to the squae oot. Futhemoe, a lowe bound on the quey complexity can neithe be stated fo geneal (non-linea) codes, no stated in tems of n alone. We elaboate on this in Remak 3.5 below. 2

3 1.1 Related wok Local econstuction Initially the model of online popety econstuction was intoduced in [ACCL08]. In this setting a data set f is given (we can think of it as a function f : [n] d N) which should have a specified stuctual popety P, but this popety may not hold due to unavoidable eos. The specific popety studied in [ACCL08] was monotonicity, and the goal of the econstucto was to constuct (online) a new function g that is monotone, but not vey fa fom the oiginal f. The authos developed such a econstucto, which given x [n] d could compute g(x) by queying f at only few locations. Howeve, the econstucto had to emembe its pevious answes in ode to be consistent with some fixed monotone function g, making it not suitable fo paallel o memoy-limited applications. This issue was addessed in [SS08], whee the authos pesented a puely local econstucto fo monotonicity, that could output g(x) based only on few inspections of f. Given a andom sting, the econstucto of [SS08] could locally econstuct g at any point x, such that all answes wee consistent with some function g, which fo most andom stings was monotone. Such a econstucto affods an obvious distibuted implementation: geneate one andom seed, and distibute it to each of the copies of the econstucto. Since they ae all deteministic, thei answes will be consistent. Local econstuction was also studied in the context of gaphs [KPS08, GGR98] and geometic poblems [CS06]. In paticula, [KPS08] studied the poblem of expande econstuction. Given an oacle access to a gaph that is close to being an expande, the algoithm of [KPS08] can simulate an oacle access to a coected gaph, that is an expande. Reconstuction in the context of patition poblems fo dense gaphs was implicitly studied in [GGR98]. Given a dense gaph G that is close to satisfying some patition popety (say k-coloability), the appoximate patitioning algoithm fom [GGR98] can be made into one that efficiently and locally econstucts a gaph G that satisfies the patition popety and is close to G. LDCs and SCCs Locally decodable codes wee explicitly defined in [KT00], but they wee extensively studied peviously in the context of self-coecting computation, wost-case to aveage-case eductions, andomness extaction, pobabilistically checkable poofs and pivate infomation etieval (see [Te04] fo a suvey). The initial constuctions of LDCs (and SCCs) wee based on Reed-Mulle codes and allowed to encode a k-bit message by a poly(log k)-quey LDC of length poly(k) [BFLS91, CKGS98]. Fo a fixed numbe of queies, the complexity of LDCs was fist studied in [KT00], and has since been the subject of a lage body of wok (see [Te04, Gas04] fo suveys). Until this day, thee is a nealy exponential gap between the best known uppe bounds on the length of q-quey LDCs [Yek08, Ef09] and the coesponding lowe bounds [KdW04, WdW05, Woo07], fo any constant q 3. While inteesting on its own, studying message econstuction can help us in undestanding the limitations of LDCs. If we view the local decoding algoithm as a deteministic algoithm that takes as input i [k] and a andom sting, then we equie that fo each i, most choices of lead to the coect decoding of the ith bit of the message. Fo message econstuction we swap the fo all i and fo most quantifies, and equie that fo most choices of, the algoithm coectly decodes all k bits of the encoded message. As we explained ealie, a constant quey LDC of polynomial length would imply the existence of O(log k)-quey 3

4 message econstucto of polynomial ate. Thus constucting a O(log k)-quey message econstuctible code of polynomial ate can be an intemediate step towads constant-quey LDCs of polynomial length. Theoem 3.2 has also an inteesting intepetation in the theoy of Pivate Infomation Retieval (PIR) [CKGS98], following the tight connection between LDCs and PIRs [KT00]. 2 Peliminaies Fo α Σ n we denote by α i the i th symbol of α, and fo a subset I [n] we denote by α I the estiction of α to the indices in I. The Hamming distance, o simply the distance d(α, β) between two stings α, β Σ n is the numbe of indices i [n] such that α i β i. Given a set S Σ n and α Σ n, the distance of α fom S is defined as d(α, S) = min β S {d(α, β)}, whee as expected the minimum ove an empty set is defined to be +. Fo ɛ > 0 we say that α is ɛ-close to S if d(α, S) ɛ w. An [n, k, d] Σ code is a mapping C : Σ k Σ n such that fo evey x x Σ k, d(c(x), C(x )) d. Elements y C ae called codewods. The paamete k is called the message length (o the infomation length) of the code and n is called the block length o simply length of the code. Sometimes we will efe to C also as a subset of Σ n defined as C = {y : x Σ k s.t. y = C(x)}. Usually we will be inteested in some family C of codes C k k N, with functions n = n(k) and d = d(n), in which evey C k is an [n, k, d] Σ code. Wheneve the alphabet Σ is not specified it means that Σ = {0, 1}. With a slight abuse of notation, fo x Σ k we will denote by C(x) the mapping C k (x) given by the ight size code C k fom the family C. Similaly, fo n = n(k) and w Σ n we define the distance of w fom C as d(w, C) = d(w, C k ), which is the minimal distance between w and some same length codewod of C. If d(w, C) < d/2 then the minimum is achieved by a unique codewod y C, and we denote this codewod by D C (w). In this case the oiginal message is also well defined, and it will be denoted simply by C 1 (D C (w)). An [n, k, d] code C is linea if (fo all k) it has a geneating matix G {0, 1} n k such that 1 fo all x {0, 1} k, C(x) = Gx. We denote by R(C) the numbe of distinct non-zeo ows in C s geneating matix. 3 Definition of ou model and statement of main esults Hee we fomally define message and codewod econstuction, and state ou main esults. All ou definitions apply to non-adaptive algoithms, i.e. algoithms that base thei quey stategy solely on thei andom bits, while basing thei output on both andom bits and the answes to the queies. Fo claity of analysis we make the dependence on a andom sting of bits (assumed to be chosen unifomly and independently) explicit, and we estict 2 ou attention to families of [n, k, d] codes, whee Σ = {0, 1}. We disegad computation time consideations because all ou lowe bounds hold egadless of computation time. Definition 3.1 (Message Reconstuction) Let C be a family of [n, k, d] codes with d 2δn fo some fixed δ > 0, let q, ρ : N N and let ɛ be a fixed constant 3 satisfying 0 < ɛ < δ. A (q, ɛ, ρ) message econstucto 1 Hee and in the following we may identify {0, 1} with the field of two elements in the standad way. 2 Nevetheless, the bounds pesented in this pape genealize to lage alphabets as well. 3 Fo claity of pesentation ɛ will be consideed to be an absolute constant. Nevetheless, the bounds pesented hee have only logaithmic dependance on 1/ɛ. 4

5 fo C is a deteministic machine A, taking as inputs k, n = n(k) N, i [k] and a andom sting {0, 1} ρ(k), that fo evey w {0, 1} n satisfies the following: A geneates sets Q i, [n] of q = q(k) indices, and functions C i, : {0, 1} q {0, 1}. Let A w (i) {0, 1} denote C i, (w Qi, ), the value of C i, on w esticted to indices in Q i,, and let A w {0, 1} k denote the concatenation A w (1)A ] w (2) A w (k). If d(w, C) ɛ then P [A w = C 1 (D C (w)) 2/3. When it is not impotant, we will avoid mentioning explicitly the andom sting length ρ, and will simply use the tem (q, ɛ) message econstucto (o even q-quey message econstucto to mean a (q, Ω(1)) message econstucto). Fo abbeviation, we may also use A to denote the algoithm A that opeates with the fixed andom sting {0, 1} ρ. In this teminology, if a code C has a (q, ɛ) message econstucto then it is locally decodable with q queies, up to noise ate ɛ. On the othe hand, a code that is locally decodable with q queies (up to noise ate ɛ) has an (O(q log k), ɛ) message econstucto, and so the existence of constant quey LDCs implies that thee exist codes that have a (O(log k), Ω(1)) message econstucto. Ou fist esult shows that in tems of the numbe of queies this is essentially optimal, even fo abitaily long codes. Theoem 3.2 Thee ae no codes with an (o(log k), Ω(1)) message econstucto. Next we fomally define codewod econstuction. Notice that hee the quey complexity and the andomness of the algoithm ae mentioned in tems of n the block length, athe than k as in the definition of message econstuction. Definition 3.3 (Codewod Reconstuction) Let C, δ, q,ρ and ɛ be as in Definition 3.1. A (q, ɛ) codewod econstucto fo C is a deteministic machine A, taking as inputs k, n = n(k) N, i [n] and a andom sting {0, 1} ρ(n), that fo evey w {0, 1} n satisfies the following conditions: A geneates sets Q i, [n] of q = q(n) indices, and functions C i, : {0, 1} q {0, 1}. Let A w (i) {0, 1} denote C i, (w Qi, ), the value of C i, on w esticted to indices in Q i,, and let A w {0, 1} n denote the concatenation] A w (1)A w (2) A w (n). If d(w, C) ɛ then P [A w = D C (w) 2/3. Hee too, we may avoid mentioning ρ explicitly, and may use A to denote the algoithm A that opeates with the fixed andom sting. If a code C has a (q, ɛ) codewod econstucto then it is self-coectable with q queies, and convesely, any code that is self-coectable with q queies up to noise ate ɛ has an (O(q log n), ɛ) codewod econstucto. As stated ealie, constant quey SCCs exist, hence thee ae codes with an (O(log n), Ω(1)) codewod econstucto. In fact, essentially all known LDCs and SCCs ae linea codes, and futhemoe they satisfy R(C) = n (i.e., all ows in thei geneating matix ae distinct). So we actually have linea codes with an (O(log R(C)), Ω(1)) codewod econstucto. Ou second esult shows that in the case of linea codes one cannot get significantly bette than that. 5

6 Theoem 3.4 Thee ae no linea codes with an (o( log R(C)), Ω(1)) codewod econstucto. Remak 3.5 Fo geneal (non-linea) codes thee is no lowe bound on the quey complexity which is polylogaithmic in n. This follows fom a simple obsevation that if a code has a q-quey message econstucto then it also has a qk-quey codewod econstucto (in qk queies it is possible to decode the whole message and its encoding). So, fo example, Long Codes admit a codewod econstucto that makes only O(k log k) = O(log log n log log log n) queies. Futhemoe, even if we focus on linea codes only, stating the bound in Theoem 3.4 in tems of n (instead of R(C)) is impossible, since thee ae linea [n, k, d] codes that have a q-quey codewod econstucto, with q abitaily smalle than n. Fo example, let C be a linea [n, k, d ] code with a (q, ɛ) codewod econstucto, and define a new linea [n = tn, k, d = td ] code C as C(x) = C (x) C (x), namely, encoding x into t copies of C (x). Now fo any t (and hence abitaily lage n), C(x) has a (q, ɛ/2) codewod econstucto which picks a andom copy of C (x) fom the encoding, and simulates on it the oiginal codewod econstucto fo C. In the following coollay we use the fact that any linea code can be tansfomed into a systematic code 4 and combine Theoem 3.2 with Theoem 3.4. Coollay 3.6 Thee ae no linea codes with an (o(max{log k, log R(C)}), Ω(1)) codewod econstucto. 4 Poof of Theoem 3.2 [TODO: outline] Fix ɛ > 0. Let C be a family of [n, k, d] codes and let A be its (q, ɛ) message econstucto. Ou goal is to pove that q = Ω(log k). Recall that Q i, [n] denotes the set of indices queied by A on input i [k], given the andom sting. Fo j [n] we define I(j, ) = {i [k] : j Q i, }. Namely, I(j, ) is the set of indices whose econstucted value may depend on the j th bit of the eceived wod, given that the andom seed is. Similaly, fo a subset S [n] we define I(S, ) = {i [k] : S Q i, }. We call an index j [n] influential with espect to if I(j, ) > 10q, and non-influential othewise. Lemma 4.1 Fo any, thee ae at most k/10 influential indices in [n]. Poof. If thee ae moe than k/10 influential indices then n j=1 I(j, ) > ( k 10 )(10q) = kq. On the othe hand n j=1 I(j, ) = k i=1 Q i, kq, a contadiction. Lemma 4.2 Let A 0 [n] be the set of influential indices (with espect to ). Thee exists a patition A 1, A 2,..., A T of [n] \ A 0 into into T k pats such that fo all i [T ], I(A i, ) 10q. 4 A code C is systematic if C(x) [k] = x fo all x {0, 1} k. 6

7 Poof. Recall that j I(j, ) kq fom the pevious poof. Let us constuct a patition A1,..., A T of the non-influential indices as follows: A 1 contains the fist l 1 non-influential indices, whee l 1 is the maximal numbe fo which I(A 1 ) 10q (note that in paticula l 1 1); then A 2 contains the next l 2 non-influential indices, whee l 2 is the lagest numbe satisfying I(A 2 ) 10q and so on. We claim that in the end of this pocess T k. Assume that T > k. Conside the patition B 1,..., B T/2 of the non-influential indices whee B i = A 2i 1 A 2i. Notice that since T k + 1, thee ae at least k/2 pats in this patition. By the definition of A i s, all but at most one (the last) of the B i s satisfy I(B i ) > 10q. So we have n I(j, ) j=1 contadicting the fact n j=1 I(j, ) kq. T/2 i=1 I(B i, ) > (k/2)10q > kq Now let us define a distibution D ove eceived wods. We will use the notation x D to mean that x is chosen at andom accoding to D, and wheneve D is a set, we also use x D to mean that x is chosen unifomly at andom fom D. Wheneve the distibution o the set ae clea fom context we may omit thei specification. Recall that we need a distibution ove eceived wods that ae ɛ-close to C, on which evey deteministic message econstucto fails with pobability lage than 1/3. Instead, we define a distibution D that will povide a wod that is ɛ-close to C only with pobability 1 o(1). Howeve, this will be sufficient because we will show that any algoithm will with pobability at least 1 2 o(1) fail to econstuct the coect message, so that also if we condition on the 1 o(1) event we still get the same eo pobability estimate. A andom wod w D is geneated by picking a unifomly andom x {0, 1} k, setting y = C(x) and then obtaining w by flipping each of the bits of y with pobability ɛ/2, independently of the othe bits. In fact this will be a distibution ove w and x, but with some abuse of notation we will geneally omit x as with pobability 1 o(1) it is just the decoding the the codewod closest to w. We need the following fact about D. Lemma 4.3 Fo evey q log n, a subset Q [n] of size Q = q and α {0, 1} q, P [w w D Q = α] (ɛ/2)q. We shall pove that if the quey complexity of A is o(log k) then the pobability that A fails on w D is lage, namely, P w D, [Aw = C 1 (D C (w))] 1/2 + o(1). Fo w {0, 1} n, {0, 1} ρ and i [k] we say that A w (i) is detemined by w A 0 if A w (i) = A y (i) fo evey y {0, 1} n with y A 0 = w A 0. The next lemma says that fo most andom stings, the expected numbe of indices coectly detemined by the influential indices only is not vey lage, whee the expectation is taken ove w D. 7

8 Definition 4.4 Fo a sting w {0, 1} n that is supposed to decode x {0, 1} k and fo evey {0, 1} ρ we set β w,x = 0 if thee is any index i such that A w (i) is detemined by w A 0 but does not match the value x i. If thee is no such index, then we set β w,x = {i [k] : A w (i) is detemined (coectly) by w A 0}. We call {0, 1} ρ typical if E w,x D [β w,x ] 2 3 k. Lemma 4.5 Let A be a message econstucto fo C. Then P [ is typical] > 1/2. The lemma is poved in Section 4.1. Fo evey andom sting {0, 1} ρ we denote by N(), 0 N() k, the expected numbe of coectly econstucted bits by ] A, whee the expectation is taken ove w D. Fomally, N() = E w D [k d(a w, C 1 (D C (w)). Futhemoe, given y D (by y D we mean y {0, 1} n that is in the suppot of D) and S [n] we denote by N(, y, S) the expected numbe of coectly econstucted bits by A, whee the expectation is taken ove w D conditioned on the event w S = y S. Fomally, N(, y, S) = E w D [k d(a w, C 1 (D C (w)) w S = y S ]. Lemma 4.6 Fo any typical, E y D [N(, y, A 0 )] k ) (1 (ɛ/2)q 3. Poof. By the definition of a typical, the expected numbe of bits whose econstucted value eithe depends on non-influential indices o is guaanteed to be always wong is at least k/3. Call these bits fee bits. This means that evey fee bit may be incoectly econstucted by A fo some assignment α to the non-influential indices (if it can be coectly econstucted at all). So, by Lemma 4.3 the pobability that A fails on a specific fee bit, taken ove w D, is at least (ɛ/2) q (We can assume that the econstucted value of each bit depends on at most q = log k log n of the non-influential indices, since othewise the quey complexity is not as advetised and we ae done). Hence by lineaity of expectation the expected numbe of fee bits that ae econstucted coectly by A, taken ove w D, is at most 2 3 k + k ( ( ɛ q ) ) 1 = k (1 3 2) (ɛ/2)q. 3 We now define sets of T + 1 andom vaiables foming Doob Matingales. Fo a fixed {0, 1} ρ, y D and evey t, 0 t T, we define H y, t N(, y, A 0 A t ), that is the expected numbe of bits econstucted coectly by A, whee the expectation is taken ove all w D that agee with y on indices A 0, A 1,..., A t. ) By Lemma 4.6, H (1,y (ɛ/2)q 3 fo all typical. On the othe hand, if A popely econstucts y then H,y T 0 k = k. Thus, fo evey typical we have [ ] [ ] [ P A y = C 1 (D C (y)) = P H,y y D y D T = k P H,y y D T H,y 0 k (ɛ/2)q ]. 3 8

9 By Lemma 4.2, T k and the influence I(A t, ) of each set A t, 1 t T, is bounded by 10q. Thus fo all 1 t < T we have H,y t+1 H,y t 10q. Now we can apply Azuma s Inequality, by which we get [ P H,y y D T H,y 0 k (ɛ/2)q ] ( k 2 (ɛ/2) 2q ) ( ) k(ɛ/2) 2q exp 3 9T (10q) 2 exp 900q 2 whee in the last inequality we used the fact that T k. This means that the pobability that A econstucts all k bits coectly with a typical is o(1), unless q = Ω(log k). Since a andom is typical with pobability at least 1/2, we conclude that unless q = Ω(log k), A fails on w D with oveall pobability 1/2 o(1). 4.1 Poof of Lemma 4.5 We need to pove that most andom stings satisfy E w,x D [β w,x ] 2 3k. To this end, we need the following auxiliay lemma, which is essentially an entopy pesevation agument. Lemma 4.7 Fo any two (possibly pobabilistic) algoithms given by thei functions, an encode E : {0, 1} k {0, 1} k/10 and a decode D : {0, 1} k/10 {0, 1} k, [ ] P x = D(E(x)) 2 9k/10,,x {0,1} k whee denotes the outcome of the andom coin flips of E and D. Futhemoe, the inequality holds even if E and D have shaed andomness. Poof. Let E, D be the two algoithms. Denote by E and D thei deteministic vesions opeating with a fixed andom seed. Fo evey possible, patition the set {0, 1} k into at most m 2 k/10 pats X1,..., X m such fo all i and x, y Xi we have E (x) = E (y). Obseve that fo evey fixed, and conditioned ove the event that a andom x falls in Xi, P x Xi [D (E (x)) = x] 1/ Xi, and clealy the pobability that x falls in Xi is exactly X i /2k. So we have [ ] P x = D (E (x)) P,x [ m i=1 ( X i 2 k )( 1 X i ) ] P [ m i=1 1/2 k] [ P 2 9k/10] 2 9k/10. Now conside the following encode/decode pai E, D coesponding to C and A. We will assume that E and D shae a andom sting, and descibe thei opeation fo evey possible. The encode E on x {0, 1} k computes y = C(x), convets y into w by flipping each bit with pobability ɛ/2 independently of the othe bits, and outputs E (x) w A 0 (the identity of the flipped bits is not pat of the shaed andomness). By definition of A 0, E (x) k/10 fo evey x and (If necessay, E (x) can be padded abitaily up to length k/10). Befoe defining the decode, let us denote by S z [k] the set of bits fo which the value is detemined by A, given that the assignment to the influential indices A 0 of the eceived wod equals z. The decode D on z {0, 1} k/10 constucts D (z) x {0, 1} k by econstucting x i accoding to A fo all i S z, and guessing x i {0, 1} unifomly at andom fo all othe indices i [k] \ Sz. We can now pove Lemma 4.5 by showing that the pai E, D defined above contadicts the statement of Lemma 4.7, unless fo most andom stings, E w,x D [β w,x ] 2 3k. We stat by obseving that the 9

10 distibutions E (x) : x {0, 1} k and w A 0 : w D ae identical fo evey. Theefoe, since fo evey [ ] [ ] the value β w,x depends only on w A 0 and x, we have E w,x D β w,x = E x {0,1} k fo all β ext(e(x)),x as well, whee ext(e (x)) {0, 1} n is any abitay sting (extension) whose estiction to A 0 equals to E (x). Now assume towads a contadiction that fo at least half of the andom stings, E w,x D [β w,x ] = E x {0,1} k[β ext(e(x)),x ] > 2 3 k. Since 0 β w,x k fo all w, x and, this means that fo at least half of the andom stings, P x {0,1} k[βext(e(x)),x k/2] > 1/6. Theefoe (taking into account that β ext(e(x)),x > 0 also implies that each of the bits not coectly detemined by A 0 obtains the coect value with pobability 1 2 independently of the othes), [ ] P x = D (E (x)) ( 1,x {0,1} k 2 )(1 6 )2 k/2 > 2 9k/10 contadicting Lemma Poof of Theoem 3.4 Fo the poof of Theoem 3.4 we use the Yao s Lemma and show that if we take a andom codewod and add andom noise to it then any detemistic codewod econstucto with quey complexity o( R(C)) is coect with pobability o(1). Ou goal is to find a set of indices S [n] with the popety: fo evey i S if E i is the event that the econstucto es on input i then the events E i (i S) ae independent. Since the econstucto is coect only if it does not e on any index so the pobability that the econstucto is coect goes down exponentially with the size of the set S. Using the famous Sunflowe Lemma we can guaantee a lage set S having the above popety, whee the size of S depends on the quey complexity of the econstucto. This intun gives the lowe bound on the quey complexity. Fix ɛ > 0. Let C be a family of linea [n, k, d] codes and let A be its (q, ɛ) codewod econstucto. Let G 1,..., G n {0, 1} k denote the ows of C s geneating matix G, satisfying C(x) i = G i, x (mod 2) fo evey x {0, 1} k and i [n], and let ˆn R(C) denote the numbe of distinct ows in G. We will show a distibution D ove eceived wods w {0, 1} n, such that if q = o( log ˆn) then fo any fixed andom sting, P w D [A w = D C (w)] < 2/3. D is defined similaly to the definition in Section 4, that is, a andom wod w D is geneated by picking a unifomly andom y C and then obtaining w by flipping each of the bits of y with pobability ɛ/2, independently of the othe bits. As we also mentioned in Section 4, w D is ɛ-close to C only with pobability 1 o(1), howeve this will be sufficient because we will show that fo any, A will fail to econstuct the coect codewod with pobability at least 1/2. Lemma 5.1 The following holds fo D and any linea code C: 10

11 1. Let T [n] and i [n] \ T be such that G i (the i th ow of C s geneating matix) is linealy independent of ows {G j : j T }. Then fo any α {0, 1} T, P w D:w T =α[d C(w) i = 1] = P w D:w T =α[d C(w) i = 0] = 1 2, whee w D : w T = α is shothand fo w D conditioned ove the event that w T = α. 2. Let S 1,..., S t [n] be disjoint sets with S i q log n fo all i [t]. Fo any sequence α i {0, 1} S i i [t] of patial assignments, [ ] P w w D Si = α i fo some i [t] 1 (1 (ɛ/2) q ) t. Poof. Fo the fist item, notice that since the codewods of C fom a linea subspace, fo evey i [n] we have P y C [y i = 1] = P y C [y i = 0] = 1/2. (Hee we assume without loss of geneality that C is not edundant, i.e. the geneating matix of C has no all-zeo ows.) Conditioning the above ove some estiction to y T has no affect on indices i that ae linealy independent of the indices in T. (This holds since the subset of C fomed by the estiction is a linea subspace as well.) Finally, in D the i th bit of y can be flipped with some pobability, but this has no affect since the pobability of y i being flipped is independent of the value y i. The second item follows fom definitions. Recall that Q i, [n] is the set of indices queied by A on input i [n] and andom sting. Fom this point on let us fix and pove that unless q max i [n] { Q i, } is of ode Ω( log ˆn), the pobability (ove w D) that A coectly econstucts w (into y = D C (w)) is less than 1/2. Since is fixed, we will make the notation shote by omitting the subscipt fom the sets Q i,. Ou poof has two steps. In the fist step we find a lage subset F [n] of indices, such that Q i Q j = Q i Q j fo all i, j, i, j F, and in addition, fo all i, j F the values D C (w) i, D C (w) j ae independent of w Qi Q. F is constucted by fist finding a lage sunflowe in the sets Q 1,..., Q n and then emoving j fom it the (not too many) bad petals that violate the above popety. In the second pat of the poof we show that given a lage enough set F as above, the pobability that A econstucts at least one of the indices in F incoectly is high. Definition 5.2 A sunflowe with t petals and a coe T is a collection of sets S 1,..., S t such that S i S j = T fo all i j. Note that a family of paiwise disjoint sets is a sunflowe (with an empty coe). Lemma 5.3 (Sunflowe Lemma [ER60]) Let Q be a family of n sets, each set having cadinality at most q. If n > q!(t 1) q then Q contains a sunflowe with t petals. In paticula, Q contains a sunflowe of size at least 1 q n1/q. Let R be a set of ˆn indices coesponding to ˆn distinct ows in G. Let Q = Q i i R be the family of sets queied by A on inputs i R. By definition, Q contains ˆn sets, each of size at most q. Fom Lemma

12 we can obtain a sunflowe S Q, S = Q i1,..., Q it, with t 1 q ˆn1/q petals. Let T [n] denote the coe of S. We define the span of the coe T as { } span(t ) = span{g i : i T } = α i G i (mod 2) : i, α i {0, 1} {0, 1} k. i T Since T q the span of T contains at most 2 q diffeent ows fom G. Next we fom a family S S of sets by emoving fom S evey petal Q ij fo which G ij belongs to the span of T. Namely, we set S {Q ij S : G ij / span(t )}. Notice that the esulting family S is a sunflowe as well, with the same coe T. Futhemoe, the size of S is at least t 1 q ˆn1/q 2 q. Intuitively, the quey sets in S coespond to those indices in R that ae independent of w T. We call these indices fee indices and denote thei set by F. Namely, F = {i : Q i S }. By the fist item of Lemma 5.1, fo evey fee index i F and all α {0, 1} T we have P w D:w T =α[d C(w) i = 1] = P w D:w T =α[d C(w) i = 0] = 1 2. (1) Now we can show that if q = o( log ˆn), then with pobability at least 1/2 one of the fee indices will be econstucted incoectly by A. Assume that the contay is tue, i.e. P w D [β w ] > 1/2, whee β w is the indicato of the event that A econstucts all fee indices coectly. This implies that thee exists an α {0, 1} T such that P [β w] > 1/2. (2) w D:w T =α If fo some i F the value of A w (i) is detemined by the fact that w T = α then by Equation 1 the pobability that A econstucts i incoectly is 1/2. So it must be the case that having w T = α does not detemine A w (i) fo any of the fee indices i, and in paticula, fo evey S i Q i \ T, Q i S thee must be an assignment α i to w Si fo which A econstucts i incoectly. Since the sets S i ae disjoint we can apply the second item of Lemma 5.1 by which the pobability that one of the fee indices is econstucted incoectly is at least 1 2 (1 (ɛ/2)q ) S 1 2 (1 (ɛ/2)q ) 1 q ˆn1/q 2 q 1 2 ( e (ɛ/2)q 1 q ˆn1/q 2 q). Notice that the above pobability is lage than 1/2 (in fact it is almost 1) if (ɛ/2) q 1 q ˆn1/q > 10, which is the case fo q = o( log ˆn). Hence a contadiction. 6 Extensions and open poblems 6.1 Reconstuction against andom noise In the usual definition of locally decodable codes (and self-econstuctible codes) it is assumed that the noise is advesaial, i.e. that the set of coupted locations is chosen in a wost case manne. Ou definition of message and codewod econstuction is against advesaial noise as well. But does econstuction become easie with andom noise? While both local decoding and self coection become tivial in the andom-noise model (via simple epetition codes), this is not the case fo econstuction. To see this conside the following eduction. 12

13 Claim 6.1 Let C be any code that has a q-quey message econstucto against andom noise, and let H be a p-quey LDC. Then the code H(C)(x) H(C(x)) (the LDC H composed with C) has an O(pq)-quey message econstucto (fo advesaial noise). We omit hee the fomal poof (since we did not even define econstuction against andom noise), but the idea is that the additional LDC encoding enables conveting advesaial noise into andom one. This is the case since by the definition of an LDC, evey bit can be decoded coectly with high pobability, whee without loss of geneality the pobability is taken only ove the andom coins of the local decode. Combining Claim 6.1 with Theoem 3.2, and using the fact that thee ae constant quey LDCs we get that message econstuction against andom noise equies Ω(log k) queies. The lowe bound fo codewod econstuction against andom noise can be deduced similaly. Since we do not know of any polynomial-length code with an O(log k)-quey message econstucto, the eal bound in Theoem 3.2 can be much highe fo efficient codes. On the othe hand, the epetition code, whose encoding is obtained by concatenating O(log k) copies of the oiginal message, has a tivial log k-quey message econstucto against andom noise. Thus fo andom noise ou lowe bound is optimal, iespective of the length of the code. 6.2 Patial econstuction In a moe geneal setting, we may equie that a message econstucto should decode coectly any t bits of the message, instead of all k. It is staightfowad to extend the lowe bound in Theoem 3.2 fo this genealization to Ω(log t). Similaly, if we equie that a codewod econstucto should decode coectly any t bits of the codewod instead of all n, then we can extend Theoem 3.4 to povide a lowe bound of Ω( log ˆt), whee ˆt is the numbe of distinct ows in the t ows coesponding to the decoded pat. 6.3 Open poblems While ou esults suggest that one cannot get message econstuctos that ae significantly moe queyefficient than the amplified vesions of constant-quey LDCs, it is inteesting question whethe this connection is bidiectional, i.e. whethe q-quey message econstuction implies O(q/ log k)-quey local decoding. Using the ecent 3-quey locally decodable codes of Yekhanin [Yek08] and Efemenko [Ef09] one obtains a code of sub-exponential length that has an O(log k)-quey message econstucto. Ae thee codes of polynomial length that have O(log k)-quey message econstuctos? In the case of a codewod econstuction, Theoem 3.4 says that fo any linea code the codewod econstucto must have quey complexity Ω( log ˆn). On the othe hand we have linea codes with O(log ˆn)-quey codewod econstucto. We expect the uppe bound to be the coect one, but we ae unable to close this gap. The ecent LDC constuctions of Yekhanin [Yek08] and Efemenko [Ef09] ae not known to be selfcoectable codes. Thus we do not yet know of any (geneal) code of sub-exponential length that has an O(log k)-quey codewod econstucto. 13

14 Acknowledgments Refeences [ACCL08] Ni Ailon, Benad Chazelle, Seshadhi Comandu, and Ding Liu. Popety-peseving data econstuction. Algoithmica, 51(2): , [BFLS91] [BLR93] László Babai, Lance Fotnow, Leonid A. Levin, and Maio Szegedy. Checking computations in polylogaithmic time. In STOC, pages 21 31, Manuel Blum, Michael Luby, and Ronitt Rubinfeld. Self-testing/coecting with applications to numeical poblems. J. Comput. Syst. Sci., 47(3): , [CKGS98] Benny Cho, Eyal Kushilevitz, Oded Goldeich, and Madhu Sudan. Pivate infomation etieval. J. ACM, 45(6): , [CS06] [Ef09] Benad Chazelle and C. Seshadhi. Online geometic econstuction. In Symposium on Computational Geomety, pages , Klim Efemenko. 3-quey locally decodable codes of subexponential length. In STOC, pages 39 44, [ER60] P. Edös and R. Rado. Intesection theoems fo systems of sets. J. London Math. Soc., 35:85 90, [Gas04] [GGR98] [KdW04] [KPS08] [KT00] William I. Gasach. A suvey on pivate infomation etieval (column: Computational complexity). Bulletin of the EATCS, 82:72 107, Oded Goldeich, Shafi Goldwasse, and Dana Ron. Popety testing and its connection to leaning and appoximation. J. ACM, 45(4): , Iodanis Keenidis and Ronald de Wolf. Exponential lowe bound fo 2-quey locally decodable codes via a quantum agument. J. Comput. Syst. Sci., 69(3): , Satyen Kale, Yuval Pees, and C. Seshadhi. Noise toleance of expandes and sublinea expande econstuction. In FOCS, pages , Jonathan Katz and Luca Tevisan. On the efficiency of local decoding pocedues fo eocoecting codes. In STOC, pages 80 86, [SS08] Michael E. Saks and C. Seshadhi. Paallel monotonicity econstuction. In SODA, pages , [Te04] [WdW05] Luca Tevisan. Some applications of coding theoy in computational complexity. Quadeni di Matematica, 13: , Stephanie Wehne and Ronald de Wolf. Impoved lowe bounds fo locally decodable codes and pivate infomation etieval. In ICALP, pages ,

15 [Woo07] [Yek08] David P. Wooduff. New lowe bounds fo geneal locally decodable codes. Electonic Colloquium on Computational Complexity (ECCC), 14(006), Segey Yekhanin. Towads 3-quey locally decodable codes of subexponential length. J. ACM, 55(1),