Locality in Coding Theory

Locality in Coding Theory Madhu Sudan Harvard April 9, 2016 Skoltech: Locality in Coding Theory 1

Error-Correcting Codes (Linear) Code CC FF qq nn. FF qq : Finite field with qq elements. nn block length kk = dim(cc) message length RR(CC) kk/nn: Rate of CC δδ CC min {δδ uu, vv Pr [uu ii vv ii ]}. xx yy CC ii Basic Algorithmic Tasks Encoding: map message in FF qq kk to codeword. Testing: Decide if uu CC Correcting: If uu CC, find nearest vv CC to uu. April 9, 2016 Skoltech: Locality in Coding Theory 2

Locality in Algorithms Sublinear time algorithms: Algorithms that run in time o(input), o(output). Assume random access to input Provide random access to output Typically probabilistic; allowed to compute output on approximation to input. (Property testing Sublinear time for Boolean functions) LTCs: Codes that have sublinear time testers. Decide if uu CC probabilistically. Allowed to accept uu if δδ(uu, CC) small. LCCs: Codes that have sublinear time correctors. If δδ(uu, CC) is small, compute vv ii, for vv CC closest to uu. April 9, 2016 Skoltech: Locality in Coding Theory 3

LTCs and LCCs: Formally CC is a (l, εε)-ltc if there exists a tester that Makes l(nn) queries to uu. Accept uu CC w.p. 1 Reject uu w.p. at least εε δδ(uu, CC). C is a (l, εε)-lcc if exists decoder DD s.t. Given oracle access uu close to vv CC, and ii Decoder makes l(nn) queries to uu. Decoder DD uu (ii) usually outputs vv ii. ii, Pr DD [ DD uu ii vv ii ] δδ(uu, vv)/εε Often: ignore εε (fix to 1 ) and focus on l 3 LDC if only coordinates of message can be recovered. April 9, 2016 Skoltech: Locality in Coding Theory 4

Outline of this talk Part 0: Definitions of LTC, LCC Part 1: Elementary construction Part 2: Motivation (historical, current) Part 3: State-of-the-art constructions of LCCs Part 4: State-of-the-art constructions of LTCs April 9, 2016 Skoltech: Locality in Coding Theory 5

Part 1: Elementary Construction April 9, 2016 Skoltech: Locality in Coding Theory 6

Main Example: Reed-Muller Codes Message = multivariate polynomial; Encoding = evaluations everywhere. RM mm, rr, qq { ff αα αα FFqq mm ff FF q xx 1,, xx mm, deg ff rr} Locality? Say rr < qq Restrictions of low-degree polynomials to lines yield low-degree (univ.) polys. Random lines sample FFmm qq uniformly (pairwise ind ly) April 9, 2016 Skoltech: Locality in Coding Theory 7

LCCs and LTCs from Polynomials Decoding (rr < qq): Problem: Given ff pp, αα FF mm qq, compute pp αα. Pick random ββ and consider ff LL where LL = {αα + tt ββ tt FF qq } is a random line αα. Find univ. poly h ff LL and output h(αα) Testing (rr < qq/2): Verify deg ff LL rr for random line LL Parameters: nn = qq mm Ideas ; l = can qq = be nn 1 mm extended ; RR CC to 1 rr > qq. Locality poly(δδ 1 ) mm! Analysis non-trivial April 9, 2016 Skoltech: Locality in Coding Theory 8

Part 2: Motivations April 9, 2016 Skoltech: Locality in Coding Theory 9

Motivation 1 ( Practical ) How to encode massive data? Solution I Encode all data in one big chunk Pro: Pr[failure] = exp(- big chunk ) Con: Recovery time ~ big chunk Solution II Break data into small pieces; encode separately. Pro: Recovery time ~ small Con: Pr[failure] = #pieces X Pr[failure of a piece] Locality (if possible): Best of both Solutions!! April 9, 2016 Skoltech: Locality in Coding Theory 10

Aside: LCCs vs. other Localities Local Reconstruction Codes (LRC): Recover from few (one? two?) erasures locally. AND Recover from many errors globally. Regenerating Codes (RgC): Restricted access pattern for recovery: Partition coordinates and access few symbols per partition. Main Differences: #errors: LCCs high vs LRC/RgC low Asymptotic (LCC) vs. Concrete parameters (LRC/RgC) April 9, 2016 Skoltech: Locality in Coding Theory 11

Motivation 2 ( Theoretical ) (Many?) mathematical consequences: Probabilistically checkable proofs: Use specific LCCs and LTCs Hardness amplification: Constructing functions that are very hard on average from functions that are hard on worst-case. Any (sufficiently good) LCC Hardness amplification Small set expanders (SSE): Usually have mostly small eigenvalues. LTCs SSEs with many big eigenvalues [Barak et al., Gopalan et al.] April 9, 2016 Skoltech: Locality in Coding Theory 12

Aside: PCPs (1 of 4) Familiar task: Protect massive data mm 0,1 kk mm φφ mm PCP task: Protect mm + analysis φφ mm {0,1}. φφ(mm) is just one bit long would like to read & trust φφ(mm) with few probes. Can we do it? Yes! PCPs! Functional Error-correction EE(mm) EE φφ (mm) April 9, 2016 Skoltech: Locality in Coding Theory 13

PCPs (2 of 4) - Definition 0 W 1 1 HTHHTH V PCP Verifer: Given WW 0,1 NN 1. Tosses random coins 2. Determines query locations 3. Reads locations. Accepts/Rejects WW EE φφ mm with φφ mm = 1 V accepts w.h.p. WW far from every EE φφ mm with φφ mm = 1 rejects w.h.p. Distinguishes φφ 1 1 from φφ 1 1 = April 9, 2016 Skoltech: Locality in Coding Theory 14

PCPs (3 of 4): Polynomial-speak mm MM(xx) low-degree (multiv.) polynomial φφ Φ : local map from poly s to poly s φφ mm = 1 AA, BB, CC s. t. Φ MM, AA, BB, CC 0 EE φφ mm = ( MM, AA, BB, CC ) (evaluations) Local testability of RM codes can verify EE φφ (mm) syntactically correct. ( MM, AA, BB, CC polynomials ) Distance of RM codes + Φ MM, AA, BB, CC aa = 0 for random aa Semantically correct (φφ mm = 1)). April 9, 2016 Skoltech: Locality in Coding Theory 15

PCP (4 of 4) mm EE: VV VV 0,1 (edges of graph). EE : FF 2 qq FF qq with deg EE nn VV φφ EE = 1 graph 3-colorable. χχ: VV { 1,0,1} s.t. yy, zz VV, EE yy, zz = 1 χχ yy χχ zz Φ χχ, AA, AA 1, BB, BB 1, BB 2, tt = AA(yy) χχ yy + 1 χχ (yy) χχ (yy) + 1 + tt AA yy AA 1 yy (yy aa) aa V + tt 2 BB yy, zz EE yy, zz χχ yy χχ zz jj jj { 2, 1,1,2} +tt 3 BB yy, zz BB 1 yy, zz aa VV yy aa BB 2 yy, zz (zz bb) bb VV April 9, 2016 Skoltech: Locality in Coding Theory 16

Part 3: Recent Progress on LCCs + LTCs April 9, 2016 Skoltech: Locality in Coding Theory 17

Summary of Recent Progress Till 2010: locality nn = oo nn RRRRRRRR < 1 2. 2016: LCC locality nn = 2 ~ log nn & RRRRRRRR 1 l nn = 2 ~ log nn meeting Singleton Bound l nn = 2 ~ log nn binary, Zyablov bound. (locally correcting half-the-distance!) LTC locality nn = ~poly log nn & RRRRRRRR 1 Singleton Bound, Zyablov bound April 9, 2016 Skoltech: Locality in Coding Theory 18

Main References Multiplicity codes [KoppartySarafYekhanin 10] See also Lifted Codes [GuoKoppartySudan 13] 1 Expander codes [HemenwayOstrovskyWootters 13] Tensor codes [Viderman 11] (see also [GKS 13] ) Above + Alon-Luby composition: [KoppartyMeirRon-ZewiSaraf 16a] 3 A Zig-Zag construction with above ingredients: [KMR-ZS 16b] 4 2 April 9, 2016 Skoltech: Locality in Coding Theory 19

Lifted Codes Codes obtained by inverting decoder: Recall decoder for RM codes. What code does it decode? CC mm,rr,qq = ff: FF qq mm FF qq deg ff LL rr LL} What we know: RM mm, rr, qq CC mm,rr,qq Theorem [GKS 13]: δδ(cc mm,rr,qq ) δδ RM mm, rr, qq RRRRRRRR CC mm,rr,qq 1 if qq = 2 tt and tt Local decodability by construction. Local testability [KaufmanS 07,GuoHaramatyS 15]. April 9, 2016 Skoltech: Locality in Coding Theory 20

Why does Rate CC mm,rr,qq 1? Example: mm = 2 Some monomials OK! E.g. qq = 2 tt ; rr = qq 2 ; ff xx, yy = xxrr yy rr satisfies deg ff llllllll rr for every line! xx rr aaaa + bb rr = bb rr xx rr + aa rr xx (mod xx qq xx) As rr qq many more monomials Proof: Lucas s lemma + simple combinatorics April 9, 2016 Skoltech: Locality in Coding Theory 21

Multiplicity Codes Basic example Message = (coeffs. of) poly pp FF qq xx, yy. Encoding = Evaluations of pp,, over FF qq 2. Length = nn = qq 2 ; Alphabet = FF qq 3 ; Rate 2 3 Local-decoding via lines. Locality = O More multiplicities Rate 1 More derivatives Locality nn εε nn April 9, 2016 Skoltech: Locality in Coding Theory 22

Multiplicity Codes - 2 Why does Rate 2 3? Every zero of pp,, two zeroes of pp Can afford to use pp of degree 2qq. Dimension 4 ; But encoding length 3 (Same reason that multiplicity improves radius of listdecoding in [Guruswami,S.]) April 9, 2016 Skoltech: Locality in Coding Theory 23

State-of-the-art as of 2014 εε, αα > 0 δδ = δδ εε,αα > 0 s.t. codes w. Rate 1 αα Distance δδ Locality = nn εε Promised: Locality nn oo(1) Singleton bound [What if you need higher distance?] Zyablov bound [What if you want a binary code?] April 9, 2016 Skoltech: Locality in Coding Theory 24

Alon-Luby Transformation Key ingredient in [Meir14], [Kopparty et al. 15] Short MDS Expander-based Interleaving Rate 1 Code Message Encoding April 9, 2016 Skoltech: Locality in Coding Theory 25

Alon-Luby Transformation Key ingredient in [Meir14], [Kopparty et al. 15] Short MDS Expander-based Interleaving Rate 1 Code Message Encoding Rate/Distance of final code ~ Rate of MDS [Meir] Locality ~ Locality of Rate 1 code Proof = Picture April 9, 2016 Skoltech: Locality in Coding Theory 26

Subpolynomial Locality Apply previous transform, with initial code of Rate 1 oo(1) and locality nn oo(1)! [e.g., multiplicity codes with mm = ωω(1)] Singleton bound Zyablov bound? Concatenation [Forney 66] April 9, 2016 Skoltech: Locality in Coding Theory 27

Part 4: Locally Testable Codes (after break) April 9, 2016 Skoltech: Locality in Coding Theory 28