List Decoding of Reed Solomon Codes
|
|
|
- Joel Barker
- 6 years ago
- Views:
Transcription
1 List Decoding of Reed Solomon Codes p. 1/30 List Decoding of Reed Solomon Codes Madhu Sudan MIT CSAIL
2 Background: Reliable Transmission of Information List Decoding of Reed Solomon Codes p. 2/30
3 List Decoding of Reed Solomon Codes p. 3/30 The Problem of Information Transmission Sender Noisy Channel Receiver
4 List Decoding of Reed Solomon Codes p. 4/30 The Problem of Information Transmission We are not ready Sender Noisy Channel Receiver
5 List Decoding of Reed Solomon Codes p. 5/30 The Problem of Information Transmission We are not ready We are now ready Sender Noisy Channel Receiver When information is digital, reliability is critical. Need to understand errors, and correct them.
6 List Decoding of Reed Solomon Codes p. 6/30 Shannon (1948) Model noise by probability distribution. Example: Binary symmetric channel (BSC) Parameter p [0, 1 2 ]. Channel transmits bits. With probability 1 p bit transmitted faithfully, and with probability p bit flipped (independent of all other events). Shannon s architecture Sender encodes k bits into n bits. Transmits n bit string on channel. Receiver decodes n bits into k bits. Rate of channel usage = k/n.
7 List Decoding of Reed Solomon Codes p. 7/30 Shannon s theorem Every channel (in broad class) has a capacity s.t., transmitting at Rate below capacity is feasible and above capacity is infeasible. Example: Binary symmetric channel (p) has capacity 1 H(p), where H(p) is the binary entropy function. p = 0 implies capacity = 1. p = 1 2 p < 1 2 implies capacity = 0. implies capacity > 0. Example: q-ary symmetric channel (p): On input σ F q receiver receives (independently) σ, where σ = σ w.p. 1 p. σ uniform over F q {σ} w.p. p. Capacity positive if p < 1 1/q.
8 List Decoding of Reed Solomon Codes p. 8/30 Constructive versions Shannon s theory was non-constructive. Decoding takes exponential time. [Elias 55] gave polytime algorithms to achieve positive rate on every channel of positive capacity. [Forney 66] achieved any rate < capacity with polynomial time algorithms (and exponentially small error). Modern results (following [Spielman 96]) lead to linear time algorithms.
9 List Decoding of Reed Solomon Codes p. 9/30 Hamming (1950) Modelled errors adversarially. Focussed on image of encoding function (the Code ). Introduced metric (Hamming distance) on range of encoding function. d(x,y) = # coordinates such that x i y i. Noticed that for adversarial error (and guaranteed error recovery), distance of Code is important. (C) = min x,y C {d(x,y)}. Code of distance d corrects (d 1)/2 errors.
10 Contrast between Shannon & Hamming List Decoding of Reed Solomon Codes p. 10/30
11 List Decoding of Reed Solomon Codes p. 10/30 Contrast between Shannon & Hamming [Sha48] : C probabilistic. E.g., flips each bit independently w.p. p. Tightly analyzed for many cases e.g., q-sc(p). Channel may be too weak to capture some scenarios. Need very accurate channel model.
12 List Decoding of Reed Solomon Codes p. 10/30 Contrast between Shannon & Hamming [Sha48] : C probabilistic. Corrects many errors. Channel restricted.
13 List Decoding of Reed Solomon Codes p. 10/30 Contrast between Shannon & Hamming [Sha48] : C probabilistic. Corrects many errors. Channel restricted. [Ham50] : C flips bits adversarially Safer model, good codes known Too pessimistic: Can only decode if p < 1/2 for any alphabet.
14 List Decoding of Reed Solomon Codes p. 10/30 Contrast between Shannon & Hamming [Sha48] : C probabilistic. Corrects many errors. Channel restricted. [Ham50] : C flips bits adversarially Fewer errors. More general errors.
15 List Decoding of Reed Solomon Codes p. 10/30 Contrast between Shannon & Hamming [Sha48] : C probabilistic. Corrects many errors. Channel restricted. [Ham50] : C flips bits adversarially Fewer errors. More general errors. Which model is correct? Depends on application. Crudely: Small q Shannon. Large q Hamming. Today: New Models of error-correction + algorithms. List-decoding: Relaxed notion of decoding.
16 List Decoding of Reed Solomon Codes p. 10/30 Contrast between Shannon & Hamming [Sha48] : C probabilistic. Corrects many errors. Channel restricted. [Ham50] : C flips bits adversarially Fewer errors. More general errors. Which model is correct? Depends on application. Crudely: Small q Shannon. Large q Hamming. Today: New Models of error-correction + algorithms. List-decoding: Relaxed notion of decoding. More errors Strong (enough) errors.
17 Reed-Solomon Codes List Decoding of Reed Solomon Codes p. 11/30
18 List Decoding of Reed Solomon Codes p. 12/30 Motivation: [Singleton] Bound Suppose C F n q has q k codewords. How large can its distance be?
19 List Decoding of Reed Solomon Codes p. 12/30 Motivation: [Singleton] Bound Suppose C F n q has q k codewords. How large can its distance be? Bound: (C) n k + 1.
20 List Decoding of Reed Solomon Codes p. 12/30 Motivation: [Singleton] Bound Suppose C F n q has q k codewords. How large can its distance be? Bound: (C) n k + 1. Proof: Project code to first k 1 coordinates. By Pigeonhole Principle, two codewords collide. These two codewords thus disagree in at most n k + 1 coordinates.
21 List Decoding of Reed Solomon Codes p. 12/30 Motivation: [Singleton] Bound Suppose C F n q has q k codewords. How large can its distance be? Bound: (C) n k + 1. Proof: Project code to first k 1 coordinates. By Pigeonhole Principle, two codewords collide. These two codewords thus disagree in at most n k + 1 coordinates. Surely we can do better?
22 List Decoding of Reed Solomon Codes p. 12/30 Motivation: [Singleton] Bound Suppose C F n q has q k codewords. How large can its distance be? Bound: (C) n k + 1. Proof: Project code to first k 1 coordinates. By Pigeonhole Principle, two codewords collide. These two codewords thus disagree in at most n k + 1 coordinates. Surely we can do better? Actually - No! [Reed-Solomon] Codes match this bound!
23 List Decoding of Reed Solomon Codes p. 13/30 Reed-Solomon Codes Messages Polynomial. Encoding Evaluation at x 1,...,x n. n > Degree: Injective x1 x2 x3 x4 x5 x6 x7 x8 x9 m1 m2 m3 m4 n Degree: Redundant
24 List Decoding of Reed Solomon Codes p. 14/30 Reed-Solomon Codes (formally) Let F q be a finite field. Code specified by k,n,α 1,...,α n F q. Message: c 0,...,c k F k+1 q coefficients of degree k polynomial p(x) = c 0 + c 1 x + c k x k. Encoding: p p(α 1 ),...,p(α n ). (k + 1 letters to n letters.) Degree k poly has at most k roots Distance d = n k. These are the Reed-Solomon codes. Match [Singleton] bound! Commonly used (CDs, DVDs etc.).
25 List-Decoding of Reed-Solomon Codes List Decoding of Reed Solomon Codes p. 15/30
26 List Decoding of Reed Solomon Codes p. 16/30 Reed-Solomon Decoding Restatement of the problem: Input: n points (α i,y i ) F 2 q; agreement parameter t Output: All degree k polynomials p(x) s.t. p(α i ) = y i for at least t values of i. We use k = 1 for illustration. i.e. want all lines (y ax b = 0) that pass through t out of n points.
27 List Decoding of Reed Solomon Codes p. 17/30 Algorithm Description [S. 96] n = 14 points; Want all lines through at least 5 points.
28 List Decoding of Reed Solomon Codes p. 17/30 Algorithm Description [S. 96] n = 14 points; Want all lines through at least 5 points. Find deg. 4 poly. Q(x,y) 0 s.t. Q(α i,y i ) = 0 for all points.
29 List Decoding of Reed Solomon Codes p. 17/30 Algorithm Description [S. 96] n = 14 points; Want all lines through at least 5 points. Find deg. 4 poly. Q(x,y) 0 s.t. Q(α i,y i ) = 0 for all points. Q(x,y) = y 4 x 4 y 2 + x 2 Let us plot all zeroes of Q...
30 List Decoding of Reed Solomon Codes p. 17/30 Algorithm Description [S. 96] n = 14 points; Want all lines through at least 5 points. Find deg. 4 poly. Q(x,y) 0 s.t. Q(α i,y i ) = 0 for all points. Q(x,y) = y 4 x 4 y 2 + x 2 Let us plot all zeroes of Q... Both relevant lines emerge!
31 List Decoding of Reed Solomon Codes p. 17/30 Algorithm Description [S. 96] n = 14 points; Want all lines through at least 5 points. Find deg. 4 poly. Q(x,y) 0 s.t. Q(α i,y i ) = 0 for all points. Q(x,y) = y 4 x 4 y 2 + x 2 Let us plot all zeroes of Q... Both relevant lines emerge! Formally, Q(x, y) factors as: (x 2 + y 2 1)(y + x)(y x).
32 List Decoding of Reed Solomon Codes p. 18/30 What Happened? 1. Why did degree 4 curve exist? Counting argument: degree 4 gives enough degrees of freedom to pass through any 14 points. 2. Why did all the relevant lines emerge/factor out? Line l intersects a deg. 4 curve Q in 5 points = l is a factor of Q
33 List Decoding of Reed Solomon Codes p. 19/30 Generally Lemma 1: Q with deg x (Q),deg y (Q) D = n passing thru any n points. Lemma 2: If Q with deg x (Q),deg y (Q) D intersects y p(x) with deg(p) d intersect in more that (D + 1)d points, then y p(x) divides Q.
34 List Decoding of Reed Solomon Codes p. 20/30 Efficient algorithm? 1. Can find Q by solving system of linear equations
35 List Decoding of Reed Solomon Codes p. 20/30 Efficient algorithm? 1. Can find Q by solving system of linear equations 2. Fast algorithms for factorization of bivariate polynomials exist ( 83-85) [Kaltofen, Chistov & Grigoriev, Lenstra, von zur Gathen & Kaltofen]
36 List Decoding of Reed Solomon Codes p. 20/30 Efficient algorithm? 1. Can find Q by solving system of linear equations 2. Fast algorithms for factorization of bivariate polynomials exist ( 83-85) [Kaltofen, Chistov & Grigoriev, Lenstra, von zur Gathen & Kaltofen] Immediate application:
37 List Decoding of Reed Solomon Codes p. 20/30 Efficient algorithm? 1. Can find Q by solving system of linear equations 2. Fast algorithms for factorization of bivariate polynomials exist ( 83-85) [Kaltofen, Chistov & Grigoriev, Lenstra, von zur Gathen & Kaltofen] Immediate application: Theorem: Can list-decode Reed-Solomon code from n (k + 1) n errors.
38 List Decoding of Reed Solomon Codes p. 20/30 Efficient algorithm? 1. Can find Q by solving system of linear equations 2. Fast algorithms for factorization of bivariate polynomials exist ( 83-85) [Kaltofen, Chistov & Grigoriev, Lenstra, von zur Gathen & Kaltofen] Immediate application: Theorem: Can list-decode Reed-Solomon code from n (k + 1) n errors. With some fine-tuning of parameters:
39 List Decoding of Reed Solomon Codes p. 20/30 Efficient algorithm? 1. Can find Q by solving system of linear equations 2. Fast algorithms for factorization of bivariate polynomials exist ( 83-85) [Kaltofen, Chistov & Grigoriev, Lenstra, von zur Gathen & Kaltofen] Immediate application: Theorem: Can list-decode Reed-Solomon code from n (k + 1) n errors. With some fine-tuning of parameters: Theorem: [S. 96] Can list-decode Reed-Solomon code from 1 2R-fraction errors.
40 List Decoding of Reed Solomon Codes p. 20/30 Efficient algorithm? 1. Can find Q by solving system of linear equations 2. Fast algorithms for factorization of bivariate polynomials exist ( 83-85) [Kaltofen, Chistov & Grigoriev, Lenstra, von zur Gathen & Kaltofen] Immediate application: Theorem: Can list-decode Reed-Solomon code from n (k + 1) n errors. With some fine-tuning of parameters: Theorem: [S. 96] Can list-decode Reed-Solomon code from 1 2R-fraction errors. Does not meet combinatorial bounds though!
41 List Decoding of Reed Solomon Codes p. 20/30 Efficient algorithm? 1. Can find Q by solving system of linear equations 2. Fast algorithms for factorization of bivariate polynomials exist ( 83-85) [Kaltofen, Chistov & Grigoriev, Lenstra, von zur Gathen & Kaltofen] Immediate application: Theorem: Can list-decode Reed-Solomon code from n (k + 1) n errors. With some fine-tuning of parameters: Theorem: [S. 96] Can list-decode Reed-Solomon code from 1 2R-fraction errors. Does not meet combinatorial bounds though!
42 Improved List-Decoding List Decoding of Reed Solomon Codes p. 21/30
43 List Decoding of Reed Solomon Codes p. 22/30 Going Further: Example 2 [Guruswami+S. 98] n = 11 points; Want all lines through 4 pts.
44 List Decoding of Reed Solomon Codes p. 22/30 Going Further: Example 2 [Guruswami+S. 98] n = 11 points; Want all lines through 4 pts. Fitting degree 4 curve Q as earlier doesn t work.
45 List Decoding of Reed Solomon Codes p. 22/30 Going Further: Example 2 [Guruswami+S. 98] n = 11 points; Want all lines through 4 pts. Fitting degree 4 curve Q as earlier doesn t work. Why?
46 List Decoding of Reed Solomon Codes p. 22/30 Going Further: Example 2 [Guruswami+S. 98] n = 11 points; Want all lines through 4 pts. Fitting degree 4 curve Q as earlier doesn t work. Why? Correct answer has 5 lines. Degree 4 curve can t have 5 factors!
47 List Decoding of Reed Solomon Codes p. 23/30 Going Further: Example 2 [Guruswami+S. 98] n = 11 points; Want all lines through 4 pts. Fit degree 7 poly. Q(x,y) passing through each point twice. Q(x,y) = (margin too small) Plot all zeroes...
48 List Decoding of Reed Solomon Codes p. 23/30 Going Further: Example 2 [Guruswami+S. 98] n = 11 points; Want all lines through 4 pts. Fit degree 7 poly. Q(x,y) passing through each point twice. Q(x,y) = (margin too small) Plot all zeroes... All relevant lines emerge!
49 List Decoding of Reed Solomon Codes p. 23/30 Going Further: Example 2 [Guruswami+S. 98] n = 11 points; Want all lines through 4 pts. Fit degree 7 poly. Q(x,y) passing through each point twice. Q(x,y) = (margin too small) Plot all zeroes... All relevant lines emerge!
50 List Decoding of Reed Solomon Codes p. 24/30 Where was the gain? Requiring Q to pass through each point twice, effectively doubles the # intersections between Q and line. So # intersections is now 8. On the other hand # constraints goes up from 11 to 33. Forces degree used to go upto 7 (from 4). But now # intersections is less than degree! Can pass through each point twice with less than twice the degree! Letting intersection multiplicity go to gives decoding algorithm for upto 1 R errors.
51 List Decoding of Reed Solomon Codes p. 25/30 Summary Can correct errors in Reed-Solomon codes well beyond half the distance (Hamming) barrier!
52 List Decoding of Reed Solomon Codes p. 25/30 Summary Can correct errors in Reed-Solomon codes well beyond half the distance (Hamming) barrier! Matches best known combinatorial bounds on list-decodability.
53 List Decoding of Reed Solomon Codes p. 25/30 Summary Can correct errors in Reed-Solomon codes well beyond half the distance (Hamming) barrier! Matches best known combinatorial bounds on list-decodability. Open Question: Correct more errors, or show this leads to exponentially large lists!
54 List Decoding of Reed Solomon Codes p. 25/30 Summary Can correct errors in Reed-Solomon codes well beyond half the distance (Hamming) barrier! Matches best known combinatorial bounds on list-decodability. Open Question: Correct more errors, or show this leads to exponentially large lists! Techniques: The polynomial method, and the method of multiplicities!
55 List Decoding of Reed Solomon Codes p. 26/30 The Polynomial Method Goal: Understand some combinatorial parameters of some algebraically nice set. E.g.,
56 List Decoding of Reed Solomon Codes p. 26/30 The Polynomial Method Goal: Understand some combinatorial parameters of some algebraically nice set. E.g., Minimum number of points in the union of l sets where each set is t points from a degree k polynomial =? Minimum number of points in K F n q such that K contains a line in every direction.
57 List Decoding of Reed Solomon Codes p. 26/30 The Polynomial Method Goal: Understand some combinatorial parameters of some algebraically nice set. E.g., Method: Fit low-degree polynomial Q to the set K. Infer Q is zero on points outside K, due to algebraic niceness. Infer lower bound on degree of Q (due to abundance of zeroes). Transfer to bound on combinatorial parameter of interest.
58 List Decoding of Reed Solomon Codes p. 27/30 Kakeya Sets Definition: K F n q is a Kakeya set if it contains a line in every direction. Question: How small can K be?
59 List Decoding of Reed Solomon Codes p. 27/30 Kakeya Sets Definition: K F n q is a Kakeya set if it contains a line in every direction. Question: How small can K be? Bounds (till 2007): K, K q n/2 K, K q n
60 List Decoding of Reed Solomon Codes p. 27/30 Kakeya Sets Definition: K F n q is a Kakeya set if it contains a line in every direction. Question: How small can K be? Bounds (till 2007): K, K q n/2 K, K (q/2) n [Mockenhaupt & Tao]
61 List Decoding of Reed Solomon Codes p. 27/30 Kakeya Sets Definition: K F n q is a Kakeya set if it contains a line in every direction. Question: How small can K be? Bounds (till 2007): K, K q n/2 K, K (q/2) n [Mockenhaupt & Tao] In particular, even exponent of q unknown!
62 List Decoding of Reed Solomon Codes p. 27/30 Kakeya Sets Definition: K F n q is a Kakeya set if it contains a line in every direction. Question: How small can K be? Bounds (till 2007): K, K q n/2 K, K (q/2) n [Mockenhaupt & Tao] In particular, even exponent of q unknown! [Dvir 08] s breakthrough: K, K q n /n!
63 List Decoding of Reed Solomon Codes p. 27/30 Kakeya Sets Definition: K F n q is a Kakeya set if it contains a line in every direction. Question: How small can K be? Bounds (till 2007): K, K q n/2 K, K (q/2) n [Mockenhaupt & Tao] In particular, even exponent of q unknown! [Dvir 08] s breakthrough: K, K q n /n! Subsequently [Dvir, Kopparty, Saraf, S.] K, K (q/2) n
64 List Decoding of Reed Solomon Codes p. 28/30 Polynomial Method and Kakeya Sets [Dvir 08] s analysis: Fit low-degree polynomial Q to K. (Interpolation Degree not too high if K not large.)
65 List Decoding of Reed Solomon Codes p. 28/30 Polynomial Method and Kakeya Sets [Dvir 08] s analysis: Fit low-degree polynomial Q to K. (Interpolation Degree not too high if K not large.) Show homogenous part of Q zero at y if line in direction y contained in K.
66 List Decoding of Reed Solomon Codes p. 28/30 Polynomial Method and Kakeya Sets [Dvir 08] s analysis: Fit low-degree polynomial Q to K. (Interpolation Degree not too high if K not large.) Show homogenous part of Q zero at y if line in direction y contained in K. Conclude homogenous part is zero too often!
67 List Decoding of Reed Solomon Codes p. 28/30 Polynomial Method and Kakeya Sets [Dvir 08] s analysis: Fit low-degree polynomial Q to K. (Interpolation Degree not too high if K not large.) Show homogenous part of Q zero at y if line in direction y contained in K. Conclude homogenous part is zero too often! [Saraf + S.], [Dvir + Kopparty + Saraf + S.]: Fit Q to vanish many times at each point of K. Yields better bounds!
68 List Decoding of Reed Solomon Codes p. 29/30 Conclusions Importance of model of error.
69 List Decoding of Reed Solomon Codes p. 29/30 Conclusions Importance of model of error. Virtues of relaxing some notions (e.g., list-decoding vs. unique-decoding)
70 List Decoding of Reed Solomon Codes p. 29/30 Conclusions Importance of model of error. Virtues of relaxing some notions (e.g., list-decoding vs. unique-decoding) New algorithmic insights: Can be useful outside the context of list-decoding (e.g., [Koetter-Vardy] Soft-decision decoder).
71 List Decoding of Reed Solomon Codes p. 29/30 Conclusions Importance of model of error. Virtues of relaxing some notions (e.g., list-decoding vs. unique-decoding) New algorithmic insights: Can be useful outside the context of list-decoding (e.g., [Koetter-Vardy] Soft-decision decoder). Central open question:
72 List Decoding of Reed Solomon Codes p. 29/30 Conclusions Importance of model of error. Virtues of relaxing some notions (e.g., list-decoding vs. unique-decoding) New algorithmic insights: Can be useful outside the context of list-decoding (e.g., [Koetter-Vardy] Soft-decision decoder). Central open question: Constructive list-decodable binary codes of rate 1 H(ρ) correcting ρ-fraction errors!! Corresponding question for large alphabets resolved by [ParvareshVardy05, GuruswamiRudra06].
73 List Decoding of Reed Solomon Codes p. 29/30 Conclusions Importance of model of error. Virtues of relaxing some notions (e.g., list-decoding vs. unique-decoding) New algorithmic insights: Can be useful outside the context of list-decoding (e.g., [Koetter-Vardy] Soft-decision decoder). Central open question: Constructive list-decodable binary codes of rate 1 H(ρ) correcting ρ-fraction errors!! Corresponding question for large alphabets resolved by [ParvareshVardy05, GuruswamiRudra06]. New (?) mathematical insights.
74 List Decoding of Reed Solomon Codes p. 29/30 Conclusions Importance of model of error. Virtues of relaxing some notions (e.g., list-decoding vs. unique-decoding) New algorithmic insights: Can be useful outside the context of list-decoding (e.g., [Koetter-Vardy] Soft-decision decoder). Central open question: Constructive list-decodable binary codes of rate 1 H(ρ) correcting ρ-fraction errors!! Corresponding question for large alphabets resolved by [ParvareshVardy05, GuruswamiRudra06]. New (?) mathematical insights. Challenge: Apply existing insights to other practical settings.
75 Thank You!! List Decoding of Reed Solomon Codes p. 30/30
Reed-Solomon Error Correcting Codes
Reed-Solomon Error Correcting Codes Ohad Rodeh Reed-Solomon Error Correcting Codes p.1/22 Introduction Reed-Solomon codes are used for error correction The code was invented in 1960 by Irving S. Reed and
Error-Correcting Codes:
Error-Correcting Codes: Progress & Challenges Madhu Sudan Microsoft/MIT Communication in presence of noise We are not ready Sender Noisy Channel We are now ready Receiver If information is digital, reliability
Notes 10: List Decoding Reed-Solomon Codes and Concatenated codes
Introduction to Coding Theory CMU: Spring 010 Notes 10: List Decoding Reed-Solomon Codes and Concatenated codes April 010 Lecturer: Venkatesan Guruswami Scribe: Venkat Guruswami & Ali Kemal Sinop DRAFT
Bridging Shannon and Hamming: List Error-Correction with Optimal Rate
Proceedings of the International Congress of Mathematicians Hyderabad, India, 2010 Bridging Shannon and Hamming: List Error-Correction with Optimal Rate Venkatesan Guruswami Abstract. Error-correcting
Lecture 4: Codes based on Concatenation
Lecture 4: Codes based on Concatenation Error-Correcting Codes (Spring 206) Rutgers University Swastik Kopparty Scribe: Aditya Potukuchi and Meng-Tsung Tsai Overview In the last lecture, we studied codes
Lecture 8: Shannon s Noise Models
Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 2007) Lecture 8: Shannon s Noise Models September 14, 2007 Lecturer: Atri Rudra Scribe: Sandipan Kundu& Atri Rudra Till now we have
CSCI 2570 Introduction to Nanocomputing
CSCI 2570 Introduction to Nanocomputing Information Theory John E Savage What is Information Theory Introduced by Claude Shannon. See Wikipedia Two foci: a) data compression and b) reliable communication
Decoding Reed-Muller codes over product sets
Rutgers University May 30, 2016 Overview Error-correcting codes 1 Error-correcting codes Motivation 2 Reed-Solomon codes Reed-Muller codes 3 Error-correcting codes Motivation Goal: Send a message Don t
Limits to List Decoding Random Codes
Limits to List Decoding Random Codes Atri Rudra Department of Computer Science and Engineering, University at Buffalo, The State University of New York, Buffalo, NY, 14620. atri@cse.buffalo.edu Abstract
Lecture 3: Error Correcting Codes
CS 880: Pseudorandomness and Derandomization 1/30/2013 Lecture 3: Error Correcting Codes Instructors: Holger Dell and Dieter van Melkebeek Scribe: Xi Wu In this lecture we review some background on error
Error Correcting Codes Questions Pool
Error Correcting Codes Questions Pool Amnon Ta-Shma and Dean Doron January 3, 018 General guidelines The questions fall into several categories: (Know). (Mandatory). (Bonus). Make sure you know how to
Shannon s noisy-channel theorem
Shannon s noisy-channel theorem Information theory Amon Elders Korteweg de Vries Institute for Mathematics University of Amsterdam. Tuesday, 26th of Januari Amon Elders (Korteweg de Vries Institute for
Entropies & Information Theory
Entropies & Information Theory LECTURE I Nilanjana Datta University of Cambridge,U.K. See lecture notes on: http://www.qi.damtp.cam.ac.uk/node/223 Quantum Information Theory Born out of Classical Information
Error-correction up to the information-theoretic limit
Error-correction up to the information-theoretic limit Venkatesan Guruswami Computer Science and Engineering University of Washington Seattle, WA 98105 venkat@cs.washington.edu Atri Rudra Computer Science
Lecture 12: November 6, 2017
Information and Coding Theory Autumn 017 Lecturer: Madhur Tulsiani Lecture 1: November 6, 017 Recall: We were looking at codes of the form C : F k p F n p, where p is prime, k is the message length, and
Lecture 6: Expander Codes
CS369E: Expanders May 2 & 9, 2005 Lecturer: Prahladh Harsha Lecture 6: Expander Codes Scribe: Hovav Shacham In today s lecture, we will discuss the application of expander graphs to error-correcting codes.
Introduction to Low-Density Parity Check Codes. Brian Kurkoski
Introduction to Low-Density Parity Check Codes Brian Kurkoski kurkoski@ice.uec.ac.jp Outline: Low Density Parity Check Codes Review block codes History Low Density Parity Check Codes Gallager s LDPC code
List and local error-correction
List and local error-correction Venkatesan Guruswami Carnegie Mellon University 8th North American Summer School of Information Theory (NASIT) San Diego, CA August 11, 2015 Venkat Guruswami (CMU) List
Lecture 9: List decoding Reed-Solomon and Folded Reed-Solomon codes
Lecture 9: List decoding Reed-Solomon and Folded Reed-Solomon codes Error-Correcting Codes (Spring 2016) Rutgers University Swastik Kopparty Scribes: John Kim and Pat Devlin 1 List decoding review Definition
Lecture 4: Proof of Shannon s theorem and an explicit code
CSE 533: Error-Correcting Codes (Autumn 006 Lecture 4: Proof of Shannon s theorem and an explicit code October 11, 006 Lecturer: Venkatesan Guruswami Scribe: Atri Rudra 1 Overview Last lecture we stated
Exercise 1. = P(y a 1)P(a 1 )
Chapter 7 Channel Capacity Exercise 1 A source produces independent, equally probable symbols from an alphabet {a 1, a 2 } at a rate of one symbol every 3 seconds. These symbols are transmitted over a
for some error exponent E( R) as a function R,
. Capacity-achieving codes via Forney concatenation Shannon s Noisy Channel Theorem assures us the existence of capacity-achieving codes. However, exhaustive search for the code has double-exponential
An Introduction to (Network) Coding Theory
An to (Network) Anna-Lena Horlemann-Trautmann University of St. Gallen, Switzerland April 24th, 2018 Outline 1 Reed-Solomon Codes 2 Network Gabidulin Codes 3 Summary and Outlook A little bit of history
Arrangements, matroids and codes
Arrangements, matroids and codes first lecture Ruud Pellikaan joint work with Relinde Jurrius ACAGM summer school Leuven Belgium, 18 July 2011 References 2/43 1. Codes, arrangements and matroids by Relinde
Lecture Introduction. 2 Formal Definition. CS CTT Current Topics in Theoretical CS Oct 30, 2012
CS 59000 CTT Current Topics in Theoretical CS Oct 30, 0 Lecturer: Elena Grigorescu Lecture 9 Scribe: Vivek Patel Introduction In this lecture we study locally decodable codes. Locally decodable codes are
Coding into a source: an inverse rate-distortion theorem
Coding into a source: an inverse rate-distortion theorem Anant Sahai joint work with: Mukul Agarwal Sanjoy K. Mitter Wireless Foundations Department of Electrical Engineering and Computer Sciences University
Notes 3: Stochastic channels and noisy coding theorem bound. 1 Model of information communication and noisy channel
Introduction to Coding Theory CMU: Spring 2010 Notes 3: Stochastic channels and noisy coding theorem bound January 2010 Lecturer: Venkatesan Guruswami Scribe: Venkatesan Guruswami We now turn to the basic
Algebraic Codes for Error Control
little -at- mathcs -dot- holycross -dot- edu Department of Mathematics and Computer Science College of the Holy Cross SACNAS National Conference An Abstract Look at Algebra October 16, 2009 Outline Coding
Shannon s Noisy-Channel Coding Theorem
Shannon s Noisy-Channel Coding Theorem Lucas Slot Sebastian Zur February 13, 2015 Lucas Slot, Sebastian Zur Shannon s Noisy-Channel Coding Theorem February 13, 2015 1 / 29 Outline 1 Definitions and Terminology
Error Correcting Codes: Combinatorics, Algorithms and Applications Spring Homework Due Monday March 23, 2009 in class
Error Correcting Codes: Combinatorics, Algorithms and Applications Spring 2009 Homework Due Monday March 23, 2009 in class You can collaborate in groups of up to 3. However, the write-ups must be done
Lecture 12. Block Diagram
Lecture 12 Goals Be able to encode using a linear block code Be able to decode a linear block code received over a binary symmetric channel or an additive white Gaussian channel XII-1 Block Diagram Data
Lecture 18: Shanon s Channel Coding Theorem. Lecture 18: Shanon s Channel Coding Theorem
Channel Definition (Channel) A channel is defined by Λ = (X, Y, Π), where X is the set of input alphabets, Y is the set of output alphabets and Π is the transition probability of obtaining a symbol y Y
List Decoding of Binary Goppa Codes up to the Binary Johnson Bound
List Decoding of Binary Goppa Codes up to the Binary Johnson Bound Daniel Augot Morgan Barbier Alain Couvreur École Polytechnique INRIA Saclay - Île de France ITW 2011 - Paraty Augot - Barbier - Couvreur
EE376A: Homework #3 Due by 11:59pm Saturday, February 10th, 2018
Please submit the solutions on Gradescope. EE376A: Homework #3 Due by 11:59pm Saturday, February 10th, 2018 1. Optimal codeword lengths. Although the codeword lengths of an optimal variable length code
6.02 Fall 2011 Lecture #9
6.02 Fall 2011 Lecture #9 Claude E. Shannon Mutual information Channel capacity Transmission at rates up to channel capacity, and with asymptotically zero error 6.02 Fall 2011 Lecture 9, Slide #1 First
An Introduction to (Network) Coding Theory
An Introduction to (Network) Coding Theory Anna-Lena Horlemann-Trautmann University of St. Gallen, Switzerland July 12th, 2018 1 Coding Theory Introduction Reed-Solomon codes 2 Introduction Coherent network
Belief propagation decoding of quantum channels by passing quantum messages
Belief propagation decoding of quantum channels by passing quantum messages arxiv:67.4833 QIP 27 Joseph M. Renes lempelziv@flickr To do research in quantum information theory, pick a favorite text on classical
atri/courses/coding-theory/book/
Foreword This chapter is based on lecture notes from coding theory courses taught by Venkatesan Guruswami at University at Washington and CMU; by Atri Rudra at University at Buffalo, SUNY and by Madhu
Reed-Solomon codes. Chapter Linear codes over finite fields
Chapter 8 Reed-Solomon codes In the previous chapter we discussed the properties of finite fields, and showed that there exists an essentially unique finite field F q with q = p m elements for any prime
Error Detection and Correction: Hamming Code; Reed-Muller Code
Error Detection and Correction: Hamming Code; Reed-Muller Code Greg Plaxton Theory in Programming Practice, Spring 2005 Department of Computer Science University of Texas at Austin Hamming Code: Motivation
MATH3302. Coding and Cryptography. Coding Theory
MATH3302 Coding and Cryptography Coding Theory 2010 Contents 1 Introduction to coding theory 2 1.1 Introduction.......................................... 2 1.2 Basic definitions and assumptions..............................
Lecture 19 : Reed-Muller, Concatenation Codes & Decoding problem
IITM-CS6845: Theory Toolkit February 08, 2012 Lecture 19 : Reed-Muller, Concatenation Codes & Decoding problem Lecturer: Jayalal Sarma Scribe: Dinesh K Theme: Error correcting codes In the previous lecture,
Digital Communications III (ECE 154C) Introduction to Coding and Information Theory
Digital Communications III (ECE 154C) Introduction to Coding and Information Theory Tara Javidi These lecture notes were originally developed by late Prof. J. K. Wolf. UC San Diego Spring 2014 1 / 8 I
Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Discussion 6A Solution
CS 70 Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Discussion 6A Solution 1. Polynomial intersections Find (and prove) an upper-bound on the number of times two distinct degree
Lecture 4 Noisy Channel Coding
Lecture 4 Noisy Channel Coding I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw October 9, 2015 1 / 56 I-Hsiang Wang IT Lecture 4 The Channel Coding Problem
Algebraic Soft-Decision Decoding of Reed Solomon Codes
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 49, NO. 11, NOVEMBER 2003 2809 Algebraic Soft-Decision Decoding of Reed Solomon Codes Ralf Koetter, Member, IEEE, Alexer Vardy, Fellow, IEEE Abstract A polynomial-time
Chapter 7 Reed Solomon Codes and Binary Transmission
Chapter 7 Reed Solomon Codes and Binary Transmission 7.1 Introduction Reed Solomon codes named after Reed and Solomon [9] following their publication in 1960 have been used together with hard decision
Lecture 7. Union bound for reducing M-ary to binary hypothesis testing
Lecture 7 Agenda for the lecture M-ary hypothesis testing and the MAP rule Union bound for reducing M-ary to binary hypothesis testing Introduction of the channel coding problem 7.1 M-ary hypothesis testing
Algebraic Codes and Invariance
Algebraic Codes and Invariance Madhu Sudan Harvard April 30, 2016 AAD3: Algebraic Codes and Invariance 1 of 29 Disclaimer Very little new work in this talk! Mainly: Ex- Coding theorist s perspective on
Algebraic Geometry Codes. Shelly Manber. Linear Codes. Algebraic Geometry Codes. Example: Hermitian. Shelly Manber. Codes. Decoding.
Linear December 2, 2011 References Linear Main Source: Stichtenoth, Henning. Function Fields and. Springer, 2009. Other Sources: Høholdt, Lint and Pellikaan. geometry codes. Handbook of Coding Theory,
Bounding the number of affine roots
with applications in reliable and secure communication Inaugural Lecture, Aalborg University, August 11110, 11111100000 with applications in reliable and secure communication Polynomials: F (X ) = 2X 2
CS6304 / Analog and Digital Communication UNIT IV - SOURCE AND ERROR CONTROL CODING PART A 1. What is the use of error control coding? The main use of error control coding is to reduce the overall probability
Notes for the Hong Kong Lectures on Algorithmic Coding Theory. Luca Trevisan. January 7, 2007
Notes for the Hong Kong Lectures on Algorithmic Coding Theory Luca Trevisan January 7, 2007 These notes are excerpted from the paper Some Applications of Coding Theory in Computational Complexity [Tre04].
Lecture B04 : Linear codes and singleton bound
IITM-CS6845: Theory Toolkit February 1, 2012 Lecture B04 : Linear codes and singleton bound Lecturer: Jayalal Sarma Scribe: T Devanathan We start by proving a generalization of Hamming Bound, which we
Related-Key Statistical Cryptanalysis
Related-Key Statistical Cryptanalysis Darakhshan J. Mir Department of Computer Science, Rutgers, The State University of New Jersey Poorvi L. Vora Department of Computer Science, George Washington University
2. Polynomials. 19 points. 3/3/3/3/3/4 Clearly indicate your correctly formatted answer: this is what is to be graded. No need to justify!
1. Short Modular Arithmetic/RSA. 16 points: 3/3/3/3/4 For each question, please answer in the correct format. When an expression is asked for, it may simply be a number, or an expression involving variables
atri/courses/coding-theory/book/
Foreword This chapter is based on lecture notes from coding theory courses taught by Venkatesan Guruswami at University at Washington and CMU; by Atri Rudra at University at Buffalo, SUNY and by Madhu
Performance-based Security for Encoding of Information Signals. FA ( ) Paul Cuff (Princeton University)
Performance-based Security for Encoding of Information Signals FA9550-15-1-0180 (2015-2018) Paul Cuff (Princeton University) Contributors Two students finished PhD Tiance Wang (Goldman Sachs) Eva Song
Lecture 19: Elias-Bassalygo Bound
Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 2007) Lecturer: Atri Rudra Lecture 19: Elias-Bassalygo Bound October 10, 2007 Scribe: Michael Pfetsch & Atri Rudra In the last lecture,
A Public Key Encryption Scheme Based on the Polynomial Reconstruction Problem
A Public Key Encryption Scheme Based on the Polynomial Reconstruction Problem Daniel Augot and Matthieu Finiasz INRIA, Domaine de Voluceau F-78153 Le Chesnay CEDEX Abstract. The Polynomial Reconstruction
Lecture 7 September 24
EECS 11: Coding for Digital Communication and Beyond Fall 013 Lecture 7 September 4 Lecturer: Anant Sahai Scribe: Ankush Gupta 7.1 Overview This lecture introduces affine and linear codes. Orthogonal signalling
SIPCom8-1: Information Theory and Coding Linear Binary Codes Ingmar Land
SIPCom8-1: Information Theory and Coding Linear Binary Codes Ingmar Land Ingmar Land, SIPCom8-1: Information Theory and Coding (2005 Spring) p.1 Overview Basic Concepts of Channel Coding Block Codes I:
Algebraic Soft-Decision Decoding of Reed-Solomon Codes Using Bit-level Soft Information
1 Algebraic Soft-Decision Decoding of Reed-Solomon Codes Using Bit-level Soft Information arxiv:cs/0611090v [cs.it] 4 Aug 008 Jing Jiang and Krishna R. Narayanan Department of Electrical and Computer Engineering,
Shannon's Theory of Communication
Shannon's Theory of Communication An operational introduction 5 September 2014, Introduction to Information Systems Giovanni Sileno g.sileno@uva.nl Leibniz Center for Law University of Amsterdam Fundamental
Polynomial interpolation over finite fields and applications to list decoding of Reed-Solomon codes
Polynomial interpolation over finite fields and applications to list decoding of Reed-Solomon codes Roberta Barbi December 17, 2015 Roberta Barbi List decoding December 17, 2015 1 / 13 Codes Let F q be
Lecture 21: P vs BPP 2
Advanced Complexity Theory Spring 206 Prof. Dana Moshkovitz Lecture 2: P vs BPP 2 Overview In the previous lecture, we began our discussion of pseudorandomness. We presented the Blum- Micali definition
An Introduction to Algorithmic Coding Theory
An Introduction to Algorithmic Coding Theory M. Amin Shokrollahi Bell Laboratories Part : Codes - A puzzle What do the following problems have in common? 2 Problem : Information Transmission MESSAGE G
Towards Optimal Deterministic Coding for Interactive Communication
Towards Optimal Deterministic Coding for Interactive Communication Ran Gelles Bernhard Haeupler Gillat Kol Noga Ron-Zewi Avi Wigderson Abstract We show an efficient, deterministic interactive coding scheme
The Method of Types and Its Application to Information Hiding
The Method of Types and Its Application to Information Hiding Pierre Moulin University of Illinois at Urbana-Champaign www.ifp.uiuc.edu/ moulin/talks/eusipco05-slides.pdf EUSIPCO Antalya, September 7,
Fast algorithms for multivariate interpolation problems
Fast algorithms for multivariate interpolation problems Vincent Neiger,, Claude-Pierre Jeannerod Éric Schost Gilles Villard AriC, LIP, École Normale Supérieure de Lyon, France ORCCA, Computer Science Department,
Lecture 6 I. CHANNEL CODING. X n (m) P Y X
6- Introduction to Information Theory Lecture 6 Lecturer: Haim Permuter Scribe: Yoav Eisenberg and Yakov Miron I. CHANNEL CODING We consider the following channel coding problem: m = {,2,..,2 nr} Encoder
arxiv: v1 [cs.it] 1 Sep 2015
![arxiv: v1 [cs.it] 1 Sep 2015](/thumbs/92/108111662.jpg "arxiv: v1 [cs.it] 1 Sep 2015")
A Number-Theoretic Error-Correcting Code Eric Brier 1, Jean-Sébastien Coron 2, Rémi Géraud 1,3, Diana Maimuţ 3, and David Naccache 2,3 arxiv:1509.00378v1 [cs.it] 1 Sep 2015 1 Ingenico 28-32 boulevard de
ECEN 655: Advanced Channel Coding
ECEN 655: Advanced Channel Coding Course Introduction Henry D. Pfister Department of Electrical and Computer Engineering Texas A&M University ECEN 655: Advanced Channel Coding 1 / 19 Outline 1 History
Efficiently decodable codes for the binary deletion channel
Efficiently decodable codes for the binary deletion channel Venkatesan Guruswami (venkatg@cs.cmu.edu) Ray Li * (rayyli@stanford.edu) Carnegie Mellon University August 18, 2017 V. Guruswami and R. Li (CMU)
Noisy-Channel Coding
Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/05264298 Part II Noisy-Channel Coding Copyright Cambridge University Press 2003.
MATH 433 Applied Algebra Lecture 21: Linear codes (continued). Classification of groups.
MATH 433 Applied Algebra Lecture 21: Linear codes (continued). Classification of groups. Binary codes Let us assume that a message to be transmitted is in binary form. That is, it is a word in the alphabet
Polynomial Codes over Certain Finite Fields
Polynomial Codes over Certain Finite Fields A paper by: Irving Reed and Gustave Solomon presented by Kim Hamilton March 31, 2000 Significance of this paper: Introduced ideas that form the core of current
Noisy channel communication
Information Theory http://www.inf.ed.ac.uk/teaching/courses/it/ Week 6 Communication channels and Information Some notes on the noisy channel setup: Iain Murray, 2012 School of Informatics, University
: Error Correcting Codes. October 2017 Lecture 1
03683072: Error Correcting Codes. October 2017 Lecture 1 First Definitions and Basic Codes Amnon Ta-Shma and Dean Doron 1 Error Correcting Codes Basics Definition 1. An (n, K, d) q code is a subset of
Context: Theory of variable privacy. The channel coding theorem and the security of binary randomization. Poorvi Vora Hewlett-Packard Co.
Context: Theory of variable privacy The channel coding theorem and the security of binary randomization Poorvi Vora Hewlett-Packard Co. Theory of security allows binary choice for data revelation: Bob
Introduction to Information Theory. Uncertainty. Entropy. Surprisal. Joint entropy. Conditional entropy. Mutual information.
L65 Dept. of Linguistics, Indiana University Fall 205 Information theory answers two fundamental questions in communication theory: What is the ultimate data compression? What is the transmission rate
Dept. of Linguistics, Indiana University Fall 2015
L645 Dept. of Linguistics, Indiana University Fall 2015 1 / 28 Information theory answers two fundamental questions in communication theory: What is the ultimate data compression? What is the transmission
Reed-Solomon Error-correcting Codes
The Deep Hole Problem Matt Keti (Advisor: Professor Daqing Wan) Department of Mathematics University of California, Irvine November 8, 2012 Humble Beginnings Preview of Topics 1 Humble Beginnings Problems
Computing minimal interpolation bases
Computing minimal interpolation bases Vincent Neiger,, Claude-Pierre Jeannerod Éric Schost Gilles Villard AriC, LIP, École Normale Supérieure de Lyon, France University of Waterloo, Ontario, Canada Supported
Example: sending one bit of information across noisy channel. Effects of the noise: flip the bit with probability p.
Lecture 20 Page 1 Lecture 20 Quantum error correction Classical error correction Modern computers: failure rate is below one error in 10 17 operations Data transmission and storage (file transfers, cell
Introduction to Convolutional Codes, Part 1
Introduction to Convolutional Codes, Part 1 Frans M.J. Willems, Eindhoven University of Technology September 29, 2009 Elias, Father of Coding Theory Textbook Encoder Encoder Properties Systematic Codes
Practical Polar Code Construction Using Generalised Generator Matrices
Practical Polar Code Construction Using Generalised Generator Matrices Berksan Serbetci and Ali E. Pusane Department of Electrical and Electronics Engineering Bogazici University Istanbul, Turkey E-mail:
What s New and Exciting in Algebraic and Combinatorial Coding Theory?
What s New and Exciting in Algebraic and Combinatorial Coding Theory? Alexander Vardy University of California San Diego vardy@kilimanjaro.ucsd.edu Notice: Persons attempting to find anything useful in
Lecture 12: Reed-Solomon Codes
Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 007) Lecture 1: Reed-Solomon Codes September 8, 007 Lecturer: Atri Rudra Scribe: Michel Kulhandjian Last lecture we saw the proof
CSCI-B609: A Theorist s Toolkit, Fall 2016 Oct 4. Theorem 1. A non-zero, univariate polynomial with degree d has at most d roots.
CSCI-B609: A Theorist s Toolkit, Fall 2016 Oct 4 Lecture 14: Schwartz-Zippel Lemma and Intro to Coding Theory Lecturer: Yuan Zhou Scribe: Haoyu Zhang 1 Roots of polynomials Theorem 1. A non-zero, univariate
channel of communication noise Each codeword has length 2, and all digits are either 0 or 1. Such codes are called Binary Codes.
5 Binary Codes You have already seen how check digits for bar codes (in Unit 3) and ISBN numbers (Unit 4) are used to detect errors. Here you will look at codes relevant for data transmission, for example,
Physical Layer and Coding
Physical Layer and Coding Muriel Médard Professor EECS Overview A variety of physical media: copper, free space, optical fiber Unified way of addressing signals at the input and the output of these media:
On the NP-Hardness of Bounded Distance Decoding of Reed-Solomon Codes
On the NP-Hardness of Bounded Distance Decoding of Reed-Solomon Codes Venkata Gandikota Purdue University vgandiko@purdue.edu Badih Ghazi MIT badih@mit.edu Elena Grigorescu Purdue University elena-g@purdue.edu
Lecture 11: Polar codes construction
15-859: Information Theory and Applications in TCS CMU: Spring 2013 Lecturer: Venkatesan Guruswami Lecture 11: Polar codes construction February 26, 2013 Scribe: Dan Stahlke 1 Polar codes: recap of last
Soft-decision Decoding of Chinese Remainder Codes
Soft-decision Decoding of Chinese Remainder Codes Venkatesan Guruswami Amit Sahai Ý Madhu Sudan Þ Abstract Given Ò relatively prime integers Ô Ô Ò and an integer Ò, the Chinese Remainder Code, ÊÌ É ÔÔÒ,
Linear Block Codes. Saravanan Vijayakumaran Department of Electrical Engineering Indian Institute of Technology Bombay
1 / 26 Linear Block Codes Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay July 28, 2014 Binary Block Codes 3 / 26 Let F 2 be the set
3F1: Signals and Systems INFORMATION THEORY Examples Paper Solutions
Engineering Tripos Part IIA THIRD YEAR 3F: Signals and Systems INFORMATION THEORY Examples Paper Solutions. Let the joint probability mass function of two binary random variables X and Y be given in the
Linear-algebraic list decoding for variants of Reed-Solomon codes
Electronic Colloquium on Computational Complexity, Report No. 73 (2012) Linear-algebraic list decoding for variants of Reed-Solomon codes VENKATESAN GURUSWAMI CAROL WANG Computer Science Department Carnegie
A Combinatorial Bound on the List Size
1 A Combinatorial Bound on the List Size Yuval Cassuto and Jehoshua Bruck California Institute of Technology Electrical Engineering Department MC 136-93 Pasadena, CA 9115, U.S.A. E-mail: {ycassuto,bruck}@paradise.caltech.edu