Linear time list recovery via expander codes

Transcription

1 Linear time list recovery via expander codes Brett Hemenway and Mary Wootters June 7 26

2 Outline Introduction List recovery Expander codes List recovery of expander codes Conclusion

3 Our Results One slide version Inner code + expander graph Expander code

4 Our Results One slide version Inner code + expander graph Expander code [SS96, Spi96, Zem, BZ2, BZ5, BZ6] Inner code has decent distance So does the expander code and it s decodable in linear time!

5 Our Results One slide version Inner code + expander graph Expander code [SS96, Spi96, Zem, BZ2, BZ5, BZ6] [HOW3] Inner code has decent distance Inner code has decent locality So does the expander code and it s decodable in linear time! So does the expander code and it s locally decodable in sub-linear time!

6 Our Results One slide version Inner code + expander graph Expander code [SS96, Spi96, Zem, BZ2, BZ5, BZ6] [HOW3] Moral? Inner code has decent distance Inner code has decent locality Inner code has decent So does the expander code and it s decodable in linear time! So does the expander code and it s locally decodable in sub-linear time! So does the expander code and there s an efficient algorithm!

7 Our Results One slide version Inner code + expander graph Expander code [SS96, Spi96, Zem, BZ2, BZ5, BZ6] [HOW3] Moral? Inner code has decent distance Inner code has decent locality Inner code has decent This work: Inner code has decent list-recoverability So does the expander code and it s decodable in linear time! So does the expander code and it s locally decodable in sub-linear time! So does the expander code and there s an efficient algorithm! So does the expander code and it s list-recoverable in linear time!

8 Our Results One Two slide version Inner code has decent list-recoverability So does the expander code and it s list-recoverable in linear time! List-recoverable: decodable from uncertainty (instead of errors).

9 Our Results One Two slide version Inner code has decent list-recoverability So does the expander code and it s list-recoverable in linear time! List-recoverable: decodable from uncertainty (instead of errors). Known linear-time list-recoverable codes have rate < /2.

10 Our Results One Two slide version Inner code has decent list-recoverability So does the expander code and it s list-recoverable in linear time! List-recoverable: decodable from uncertainty (instead of errors). Known linear-time list-recoverable codes have rate < /2. Expander codes can have rate ε!

11 Our Results One Two slide version Inner code has decent list-recoverability So does the expander code and it s list-recoverable in linear time! List-recoverable: decodable from uncertainty (instead of errors). Known linear-time list-recoverable codes have rate < /2. Expander codes can have rate ε! We can plug our codes into [Meir 4] and get the optimal rate/error trade-off, for any rate.

12 Outline Introduction List recovery Expander codes List recovery of expander codes Conclusion

13 List Decoding

14 List Decoding G O O D T I M E F O R P I E

15 List Decoding G O OX DA TC IU MO E FV OR RT P I E

16 List Decoding GOODTIMEFORPIE GOATSATEALLPIE

17 List Recovery G L M Y W O O I V D A E A H T N I T B D M E F T R I T I G O E E V L E T L P I E E H

18 List Recovery G L M Y W O O I V D A E A H T N I T B D M E F T R I T I G O E E V L E T L P I E E H

19 List Recovery GOODTIMEFORPIE MYWHATBIGTEETH LIVEANDLETLIVE G L M Y W O O I V D A E A H T N I T B D M E F T R I T I G O E E V L E T L P I E E H

20 List Recovery From Erasures GOODTIMEFORPIE MYWHATBIGTEETH LIVEANDLETLIVE G L M???? Y W D A N B E F T R I T O O E A I D I G O E E V I V H T T M L E T L P I E E H

21 List Recovery Definition Definition C Σ N is (α, l, L)-list-recoverable (from erasures) if: for any set of lists S,..., S N so that at least αn of them have size l, there are at most L codewords c C so that c i S i for all i.

22 List Recovery Definition Definition C Σ N is (α, l, L)-list-recoverable (from erasures) if: for any set of lists S,..., S N so that at least αn of them have size l, there are at most L codewords c C so that c i S i for all i. α fraction of codeword not erased l number of symbols in each slot L number of codewords that match Note: L l.

23 List decoding concatenated codes An application of list recovery message

24 List decoding concatenated codes An application of list recovery message C out

25 List decoding concatenated codes An application of list recovery message C out C in C in C in C in C in C in C in

26 List decoding concatenated codes An application of list recovery C out C in C in C in C in C in C in C in

27 List decoding concatenated codes An application of list recovery List decode C in C out C in C in C in C in C in C in C in

28 List decoding concatenated codes An application of list recovery List decode C in C out list list list list list list list C in C in C in C in C in C in C in

29 List decoding concatenated codes An application of list recovery List recover C out List decode C in C out list list list list list list list C in C in C in C in C in C in C in

30 List decoding concatenated codes An application of list recovery List recover C out {List of messages} List decode C in C out list list list list list list list C in C in C in C in C in C in C in Concatenated code is list decodable

31 List Recovery Applications Related to list decoding [GI2, GI3, GI4] Compressed sensing [NPR2, GNP + 3] Group testing [INR] Erasure model is weaker than error model (erasure model was studied before in [GI4])

32 Outline Introduction List recovery Expander codes List recovery of expander codes Conclusion

33 Tanner Codes [Tanner 8] Given: A d-regular graph G with n vertices and N = nd 2 An inner code C with block length d over Σ edges We get a Tanner code C. C has block length N and alphabet Σ. Codewords are labelings of edges of G. A labeling is in C if the labels on each vertex form a codeword of C. (We fix an arbitrary ordering of edges at each vertex)

34 Example [Tanner 8] G is K 8, and C is the [7, 4, 3]-Hamming code. N = ( 8 2) = 28 and Σ = {, }

35 Example [Tanner 8] G is K 8, and C is the [7, 4, 3]-Hamming code. A codeword of C is a labeling of edges of G. red blue (,,,,,,,,,,,,,,,,,,,,,,,,,,, ) C {, } 28

36 Example [Tanner 8] G is K 8, and C is the [7, 4, 3]-Hamming code. These edges form a codeword in the Hamming code red blue (,,,,,,,,,,,,,,,,,,,,,,,,,,, ) C {, } 28

37 Encoding Tanner Codes Encoding is Easy!. Generate parity-check matrix Requires: Edge-vertex incidence matrix of graph Parity-check matrix of inner code 2. Calculate a basis for the kernel of the parity-check matrix 3. This basis defines a generator matrix for the linear Tanner Code 4. Encoding is just multiplication by this generator matrix

38 Linearity If the inner code C is linear, so is the Tanner code C C = Ker(H ) for some parity check matrix H. x C H x =

39 Linearity If the inner code C is linear, so is the Tanner code C C = Ker(H ) for some parity check matrix H. x C H x = So codewords of the Tanner code C also are defined by linear constraints: v y y C v G, H = y Γ(v)

40 Example: vertex edge incidence matrix of K 8 row for each vertex column for each edge Columns have weight 2 (Each edge hits two vertices) Rows have weight 7 (Each vertex has degree seven)

41 Example: parity-check matrix of a Tanner code K 8 and the [7, 4, 3]-Hamming code Parity-check of Hamming code Edge-vertex incidence matrix of K 8

42 Example: parity-check matrix of a Tanner code K 8 and the [7, 4, 3]-Hamming code Vertex

43 Example: parity-check matrix of a Tanner code K 8 and the [7, 4, 3]-Hamming code

44 Example: parity-check matrix of a Tanner code K 8 and the [7, 4, 3]-Hamming code

45 Example: parity-check matrix of a Tanner code K 8 and the [7, 4, 3]-Hamming code

46 Example: parity-check matrix of a Tanner code K 8 and the [7, 4, 3]-Hamming code

47 If the inner code has good rate, so does the outer code Say that C is linear If C has rate r, it satisfies ( r )d linear constraints. Each of the n vertices of G must satisfy these constraints.

48 If the inner code has good rate, so does the outer code Say that C is linear If C has rate r, it satisfies ( r )d linear constraints. Each of the n vertices of G must satisfy these constraints. C is defined by at most n ( r )d constraints.

49 If the inner code has good rate, so does the outer code Say that C is linear If C has rate r, it satisfies ( r )d linear constraints. Each of the n vertices of G must satisfy these constraints. C is defined by at most n ( r )d constraints. Length of C = N = # edges = nd/2

50 If the inner code has good rate, so does the outer code Say that C is linear If C has rate r, it satisfies ( r )d linear constraints. Each of the n vertices of G must satisfy these constraints. C is defined by at most n ( r )d constraints. Length of C = N = # edges = nd/2 The rate of C is R = k N N n ( r )d N = 2r.

51 Better rate bounds? The lower bound R > 2r is independent of the ordering of edges around a vertex Tanner already noticed that order matters. Let G be the complete bipartite graph with 7 vertices per side Let C be the [7, 4, 3] hamming code Then different natural orderings achieve a Tanner code with [49, 6, 9] ( ) [49, 2, 6] ( ) [49, 7, 7] ( ) Meets lower bound of 2 7

52 Expander codes When the underlying graph is an expander graph, the Tanner code is a expander code. Expander codes admit very fast decoding algorithms [Sipser and Spielman 996] Further improvements in [Sipser 96, Zemor, Barg and Zemor 2, 5, 6]

53 Outline Introduction List recovery Expander codes List recovery of expander codes Conclusion

54 Linear-time list-recoverable codes Theorem (Main Theorem) Suppose that C is (α, l, L )-list recoverable from erasures. Suppose G is an expander graph of degree d. d needs to be big enough compared to l, L. Then the expander code C is (α, l, L)-list recoverable from erasures. It can be list-recovered in linear time. Above, α, L are constants (independent of n)....

55 Comparison with Previous Work Code Rate Decoding time Random code - ε lots Folded RS (+friends) [GR8, Gur, DL2] - ε poly(n) [GI3, GI4] /poly(l) O(n) This work (Random inner code) All list sizes L are constants but some are huge constants ε O(n)

56 Improving the rate/error tradeoff Theorem ( Dream theorem of Meir 4) Constructing codes of rate with property P and decent distance Constructing codes with P that approach the Singleton bound.

57 Improving the rate/error tradeoff Theorem For any R >, l >, and ε >, and for any large enough L, d, q (depending only on l, ε), there is a family of codes with: rate at least R (R + ε, l, L)-list-recoverable in linear time.

58 Improving the rate/error tradeoff The Or Meir Construction [Mei4] x C (x) z c y y C (y ) c y n C (y n ) c n F R m q F m q ( ) F R d n Redistribute q ( F d ) n according to G ( ) q F d n q

59 The algorithm Suppose that C is list-recoverable. S S 2... S d = {a,b}

60 The algorithm Suppose that C is list-recoverable. S S 2... c. c L a d c d c f b f c d c f a d b c r e t a b c b c d b a b x d e f a b z d e d a S d = {a,b}. List recover locally at this vertex: get {c,..., c L }

61 The algorithm Suppose that C is list-recoverable. S S 2... c. c L a d c d c f b f c d c f a d b c r e t a b c b c d b a b x d e f a b z d e d a S d = {a,b}. List recover locally at this vertex: get {c,..., c L } 2. What if we know this symbol is a?

62 The algorithm Suppose that C is list-recoverable. S S 2... c. c L a d c d c f b f c d c f a d b c r e t a b c b c d b a b x d e f a b z d e d a S d = {a,b}. List recover locally at this vertex: get {c,..., c L } 2. What if we know this symbol is a?

63 The algorithm Suppose that C is list-recoverable. S S 2... c. c L a d c d c f b f c d c f a d b c r e t a b c b c d b a b x d e f a b z d e d a S d = {a,b}. List recover locally at this vertex: get {c,..., c L } 2. What if we know this symbol is a?

64 The algorithm Suppose that C is list-recoverable. S S List recover locally at this vertex: get {c,..., c L } c S d = {a,b} 2. What if we know this symbol is a?. c L a d c d c f b f c d c f a d b c r e t a b c b c d b a b x d e f a b z d e d a 3. Then we know a bunch of other symbols too.

65 The algorithm Suppose that C is list-recoverable. S S List recover locally at this vertex: get {c,..., c L } c S d = {a,b} 2. What if we know this symbol is a?. c L a d c d c f b f c d c f a d b c r e t a b c b c d b a b x d e f a b z d e d a 3. Then we know a bunch of other symbols too. L l possible columns. Each choice fixes about d/l l others

66 The algorithm Suppose that C is list-recoverable Make one decision.

67 The algorithm Suppose that C is list-recoverable Make one decision.

68 The algorithm Suppose that C is list-recoverable Make one decision.

69 The algorithm Suppose that C is list-recoverable Make one decision.

70 The algorithm Suppose that C is list-recoverable Make one decision. Determine a bunch of other edges.

71 The algorithm Suppose that C is list-recoverable Make one decision. Determine a bunch of other edges. Make another decision.

72 The algorithm Suppose that C is list-recoverable Make one decision. Determine a bunch of other edges. Make another decision. Determine some more edges.

73 The algorithm Suppose that C is list-recoverable Make one decision. Determine a bunch of other edges. Make another decision. Determine some more edges....

74 The algorithm Suppose that C is list-recoverable Make one decision. Determine a bunch of other edges. Make another decision. Determine some more edges.... Correct the rest using regular decoding alg.

75 The algorithm Suppose that C is list-recoverable Make one decision. Determine a bunch of other edges. Make another decision. Determine some more edges.... Correct the rest using regular decoding alg. Number of possibilities = l number of decision edges = l O().

76 Why does propagation work? Equivalence classes of edges L > l so choosing single symbol won t determine CW L CWs, l choices per index L l possible columns (For each index i, L l maps from CW to symbol) (L l maps from [L] l) Edges are in same equivalance class (wrt vertex) if knowing one determines all the others Each vertex has d edges At most L l equivalence classes Average equivalence class is of size d/l l

77 Why does propagation work? Large Equivalance Classes Average size of an equivalence class is d/l l What if we get unlucky and pick an edge in a small equivalence class? By the expansion property of the graph there is a large (constant fraction) subset of edges that all have large equivalence class Probability a uniformly chosen edge covers an ε(ε λ) fraction of the graph is at least 3εl L

78 Results (R + η, l, L)-list recoverable codes Any R > Any l > Any η > L depends only on l, η Linear-time recovery algorithm

79 Outline Introduction List recovery Expander codes List recovery of expander codes Conclusion

80 Moral of the story Inner code has decent So does the expander code and there s an efficient algorithm!

81 Moral of the story Inner code has decent list-recoverability So does the expander code and it s list-recoverable in linear time!

82 Moral of the story Inner code has decent list-recoverability So does the expander code and it s list-recoverable in linear time! Open Questions:

83 Moral of the story Inner code has decent list-recoverability So does the expander code and it s list-recoverable in linear time! Open Questions: What other properties can expander codes improve?

84 Moral of the story Inner code has decent list-recoverability So does the expander code and it s list-recoverable in linear time! Open Questions: What other properties can expander codes improve? Handle errors?

85 Moral of the story Inner code has decent list-recoverability So does the expander code and it s list-recoverable in linear time! Open Questions: What other properties can expander codes improve? Handle errors? Other applications with erasures?

86 Moral of the story Inner code has decent list-recoverability So does the expander code and it s list-recoverable in linear time! Open Questions: What other properties can expander codes improve? Handle errors? Other applications with erasures? Better inner code? Better constants?

87 The end Thanks!

88 References I Alexander Barg and Gilles Zemor. Error exponents of expander codes. Information Theory, IEEE Transactions on, 48(6): , June 22. Alexander Barg and Gilles Zemor. Concatenated codes: serial and parallel. Information Theory, IEEE Transactions on, 5(5): , May 25. Alexander Barg and Gilles Zemor. Distance properties of expander codes. Information Theory, IEEE Transactions on, 52():78 9, January 26. Zeev Dvir and Shachar Lovett. Subspace Evasive Sets. In STOC 2, pages , October 22.

89 References II Venkatesan Guruswami and Piotr Indyk. Near-optimal Linear-time Codes for Unique Decoding and New List-decodable Codes over Smaller Alphabets. In Proceedings of the Thiry-fourth Annual ACM Symposium on Theory of Computing, STOC 2, pages 82 82, New York, NY, USA, 22. ACM. Venkatesan Guruswami and Piotr Indyk. Linear time encodable and list decodable codes. In Proceedings of the thirty-fifth annual ACM symposium on Theory of computing, STOC 3, pages 26 35, New York, NY, USA, 23. ACM.

90 References III Venkatesan Guruswami and Piotr Indyk. Efficiently decodable codes meeting Gilbert-Varshamov bound for low rates. In SODA 4: Proceedings of the fifteenth annual ACM-SIAM symposium on Discrete algorithms, pages , Philadelphia, PA, USA, 24. Society for Industrial and Applied Mathematics. Anna C. Gilbert, Hung Q. Ngo, Ely Porat, Atri Rudra, and Martin J. Strauss. 2/2-Foreach Sparse Recovery with Low Risk. In FedorV Fomin, Rsiņš Freivalds, Marta Kwiatkowska, and David Peleg, editors, Automata, Languages, and Programming, volume 7965 of Lecture Notes in Computer Science, pages Springer Berlin Heidelberg, 23.

91 References IV Venkatesan Guruswami and Atri Rudra. Concatenated Codes Can Achieve List-decoding Capacity. In Proceedings of the Nineteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 8, pages , Philadelphia, PA, USA, 28. Society for Industrial and Applied Mathematics. Venkatesan Guruswami. Linear-algebraic list decoding of folded Reed-Solomon codes. In IEEE Conference on Computational Complexity, pages 77 85, 2. Brett Hemenway, Rafail Ostrovsky, and Mary Wootters. Local Correctability of Expander Codes. In ICALP, LNCS. Springer, April 23.

92 References V Piotr Indyk, Hung Q. Ngo, and Atri Rudra. Efficiently decodable non-adaptive group testing. In Proceedings of the Twenty-First Annual ACM-SIAM Symposium on Discrete Algorithms, SODA, pages 26 42, Philadelphia, PA, USA, 2. Society for Industrial and Applied Mathematics. Or Meir. Locally Correctable and Testable Codes Approaching the Singleton Bound. ECCC Report 24-7, 24. Hung Q. Ngo, Ely Porat, and Atri Rudra. Efficiently Decodable Compressed Sensing by List-Recoverable Codes and Recursion. In Christoph Dürr and Thomas Wilke, editors, 29th International Symposium on Theoretical Aspects of Computer Science (STACS 22), volume 4 of Leibniz International

93 References VI Proceedings in Informatics (LIPIcs), pages 23 24, Dagstuhl, Germany, 22. Schloss Dagstuhl Leibniz-Zentrum fuer Informatik. Daniel A. Spielman. Linear-time encodable and decodable error-correcting codes. Information Theory, IEEE Transactions on, 42(6):723 73, November 996. Michael Sipser and Daniel A. Spielman. Expander codes. Information Theory, IEEE Transactions on, 42(6):7 722, November 996. Gilles Zemor. On expander codes. Information Theory, IEEE Transactions on, 47(2): , February 2.