Error-Correcting Codes:
|
|
- Chastity Palmer
- 5 years ago
- Views:
Transcription
1 Error-Correcting Codes: Progress & Challenges Madhu Sudan Microsoft/MIT
2 Communication in presence of noise We are not ready Sender Noisy Channel We are now ready Receiver If information is digital, reliability is critical
3 Shannon s Model: Probabilistic Noise Sender Receiver Encode (expand) Noisy Channel Decode (compress?) E:Σ k Σ n D:Σ n Σ k Probabilistic Noise: E.g., every letter flipped to random other letter of Σ w.p. p Focus: Design good Encode/Decode algorithms.
4 Hamming Model: Worst-case error Errors: Upto t worst-case errors Focus: Code: C = Image(E) = {E(x) x Є Σ k } (Note: Not encoding/decoding) Goal: Design code to correct every possible pattern of t errors.
5 Problems in Coding Theory, Broadly Combinatorics: Design best possible errorcorrecting codes. Probability/Algorithms: Design algorithms correcting random/worst-case errors.
6 Part I (of III): Combinatorial Results
7 Hamming Notions Hamming Distance: (x,y) = {i x i y i } Distance of Code: (C) = min x,y 2 C { (x,y)} Code of distance 2t+1 corrects t errors. Main question: Four parameters: Length n, message length k, distance d, alphabet q = Σ. - How do they relate? - Want + n, " k, " d,? q Let: R = k/n; δ= d/n; How do R, δ, q relate?
8 Simple results Ball(x,r) = {y \in Σ n Δ(x,y) r} Volume of Ball:Vol(q,n,r) = Ball(x,r) Entropy function: H q (δ) = c s.t. Vol(q,n, δn) ¼ q cn Hamming (Packing) Bound: Balls of radius δn/2 around codewords are disjoint. q k q H q(δ/2)n q n R + H q (δ/2) 1
9 Simple results (contd.) Gilbert-Varshamov (Greedy) Bound: Let C:Σ k Σ n be maximal code of distance d. Then balls of radius d-1 around codewords cover Σ n So q k q H q(δn) q n Or R 1 Hq(δ)
10 Simple results (Summary) For the best code: 1 H q (δ) R 1 H q (δ/2) Which is right? After fifty years of research We still don t know.
11 Binary case (q =2): Case of large distance: δ = ½ - ², ² 0. Ω(² 2 ) R O * (² 2 ) GV/Cherno LP Bound Case of small (relative) distance: No bound better than R 1 (1-o(1)) H(δ/2) Hamming Case of constant distance d: (d/2) log n n-k (1-o(1)). (d/2) \log n BCH Hamming
12 Binary case (Closer look): For general n,d: # Codewords 2 n / Vol (2,n, d-1) Can we do better? Twice as many codewords? (won t change asymptotics of R, δ ) Recent progress [Jiang-Vardy]: # Codewords d 2 n / Vol(2,n,d-1)
13 Major questions in binary codes: Give explicit construction meeting GV bound. Specifically: Codes with δ = ½ - ² & R = Ω(² 2 ) Is Hamming tight when δ 0? Do there exist codes of distance δ with R = 1 [ c (1 o(1)) δ log 2 (1/δ) ] for c < 1? [Hamming: c > ½ ] Is LP Bound tight?
14 Combinatorics (contd.): q-ary case Fix δ and let q 1 (then fix q and let n 1) 1 δ O(1/log q) R 1 δ 1/q GV bound Plotkin Surprising result ( 80s): Algebraic Geometry yields: R 1 δ 1/( q 1) (Also a negative surprise: BCH codes only yield 1 R (q-1)/q log q n) Not Hamming
15 Major questions: q-ary case Suppose R = 1 δ f(q) What is the fastest decaying function f(.)? (somewhere between 1/ q and 1/q). Give a simple explanation for why f(q) 1/ q Fix d, and let q 1 How does (n-k)/(d log q n) grow in the limit? Is it 1 or ½? Or somewhere in between?
16 Part II (of III): Correcting Random Errors
17 Recall Shannon 1948 Σ-symmetric channel w. error prob. p: Transmits σ 2 Σ as σ w.p. 1-p; Shannon s Coding Theorem: and as 2 Σ- {σ} w.p. p/(q-1). Can transmit at rate R = 1 H q (p) - ², 8 ² > 0 If R = 1 H q (p) - ², then for every n and k = Rn, there exist E:Σ k Σ n and D:Σ n Σ k s.t. Pr Channel,x [D(Channel(E(x)) x] exp(-n). Converse Coding Theorem: Can not transmit at rate R = 1 H q (p) + ² So: No mysteries?
18 Constructive versions Shannon s functions: E random, D brute force search. Can we get poly time E, D? [Forney 66]: Yes! (Using Reed-Solomon codes correcting ²-fraction error + composition.) [Sipser-Spielman 92, Spielman 94, Barg- Zemor 97]: Even in linear time! Still didn t satisfy practical needs. Why? [Berrou et al. 92] Turbo codes + belief propagation: No theorems; Much excitement
19 What is satisfaction? Articulated by [Luby,Mitzenmacher,Shokrollahi,Spielman 96] Practically interesting question: n = 10000; q = 2, p =.1; Desired error prob. = 10-6 ; k =? [Forney 66]: Decoding time: exp(1/(1 H(p) (k/n))); Rate = 90% ) decoding time 2 100; Right question: reduce decoding time to poly(n,1/ ²); where ² = 1 H(p) (k/n)
20 Current state of the art Luby et al.: Propose study of codes based on irregular graphs ( Irregular LDPC Codes ). No theorems so far for erroneous channels. Strong analysis for (much) simpler case of erasure channels (symbols are erased); decoding time = O(n log (1/²)) (Easy to get composition based algorithms with decoding time = O(n poly(1/²)) Do have some proposals for errors as well (with analysis by Luby et al., Richardson & Urbanke), but none known to converge to Shannon limit.
21 Part III: Correcting Adversarial Errors
22 Motivation: As notions of communication/storage get more complex, modeling error as oblivious (to message/encoding/decoding) may be too simplistic. Need more general models of error + encoding/decoding for such models. Most pessimistic model: errors are worst-case.
23 Gap between worst-case & random errors In Shannon model, with binary channel: Can correct upto 50% (random) errors. ( 1-1/q fraction errors, if channel q-ary.) In Hamming model, for binary channel: Code with more than n codewords has distance at most 50%. So it corrects at most 25% worst-case errors. ( ½(1 1/q) errors in q-ary case.) Shannon model corrects twice as many errors: Need new approaches to bridge gap.
24 Approach: List-decoding Main reason for gap between Shannon & Hamming: The insistence on uniquely recovering message. List-decoding: Relaxed notion of recovery from error. Decoder produces small list (of L) codewords, such that it includes message. Code is (p,l) list-decodable if it corrects p fraction error with lists of size L.
25 List-decoding Main reason for gap between Shannon & Hamming: The insistence on uniquely recovering message. List-decoding [Elias 57, Wozencraft 58]: Relaxed notion of recovery from error. Decoder produces small list (of L) codewords, such that it includes message. Code is (p,l) list-decodable if it corrects p fraction error with lists of size L.
26 What to do with list? Probabilistic error: List has size one w.p. nearly 1 General channel: Need side information of only O(log n) bits to disambiguate [Guruswami 03] (Alt ly if sender and receiver share O(log n) bits, then they can disambiguate [Langberg 04]). Computationally bounded error: Model introduced by [Lipton, Ding Gopalan L.] List-decoding results can be extended (assuming PKI and some memory at sender) [Micali et al.]
27 List-decoding: State of the art [Zyablov-Pinsker/Blinovskii late 80s] There exist codes of rate 1 H q (p) - \epsilon that are (p,o(1))-list-decodable. Matches Shannon s converse perfectly! (So can t do better even for random error!) But [ZP/B] non-constructive!
28 Algorithms for List-decoding Not examined till 88. First results: [Goldreich-Levin] for Hadamard codes (non-trivial in their setting). More recent work: [S. 96, Shokrollahi-Wasserman 98, Guruswami-S. 99, Parvaresh-Vardy 05, Guruswami-Rudra 06] Decode algebraic codes. [Guruswami-Indyk 00-02] Decode graphtheoretic codes. 02/17/2010 ECC: Progress/Challenges
29 Results in List-decoding q-ary case: [Guruswami-Rudra 06] Codes of rate R correcting 1 R - ² fraction errors with q = q(²) Matches Shannon bound (except for q(²) ) 9 Codes of rate ²c correcting 1 ² fraction errors. 2 c = 4: Guruswami et al c! 3: Implied by Parvaresh-Vardy 05 c = 3: Guruswami Rudra
30 Major open question ² Construct (p; O(1)) list-decodable binary code of rate 1 H(p) ² with polytime list decoding.. ² Note: If running time is poly(1=²) then this implies a solution to the random error problem as well.
31 Conclusions Coding theory: Very practically motivated problems; solutions influence (if not directly alter) practice. Many mysteries remain in combinatorial setting. Significant progress in algorithmic setting, but many more questions to resolve.
List Decoding of Reed Solomon Codes
List Decoding of Reed Solomon Codes p. 1/30 List Decoding of Reed Solomon Codes Madhu Sudan MIT CSAIL Background: Reliable Transmission of Information List Decoding of Reed Solomon Codes p. 2/30 List Decoding
More informationLecture 8: Shannon s Noise Models
Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 2007) Lecture 8: Shannon s Noise Models September 14, 2007 Lecturer: Atri Rudra Scribe: Sandipan Kundu& Atri Rudra Till now we have
More informationLecture 6: Expander Codes
CS369E: Expanders May 2 & 9, 2005 Lecturer: Prahladh Harsha Lecture 6: Expander Codes Scribe: Hovav Shacham In today s lecture, we will discuss the application of expander graphs to error-correcting codes.
More informationBridging Shannon and Hamming: List Error-Correction with Optimal Rate
Proceedings of the International Congress of Mathematicians Hyderabad, India, 2010 Bridging Shannon and Hamming: List Error-Correction with Optimal Rate Venkatesan Guruswami Abstract. Error-correcting
More informationatri/courses/coding-theory/book/
Foreword This chapter is based on lecture notes from coding theory courses taught by Venkatesan Guruswami at University at Washington and CMU; by Atri Rudra at University at Buffalo, SUNY and by Madhu
More informationAn Introduction to Algorithmic Coding Theory
An Introduction to Algorithmic Coding Theory M. Amin Shokrollahi Bell Laboratories Part : Codes - A puzzle What do the following problems have in common? 2 Problem : Information Transmission MESSAGE G
More informationECEN 655: Advanced Channel Coding
ECEN 655: Advanced Channel Coding Course Introduction Henry D. Pfister Department of Electrical and Computer Engineering Texas A&M University ECEN 655: Advanced Channel Coding 1 / 19 Outline 1 History
More informationLimits to List Decoding Random Codes
Limits to List Decoding Random Codes Atri Rudra Department of Computer Science and Engineering, University at Buffalo, The State University of New York, Buffalo, NY, 14620. atri@cse.buffalo.edu Abstract
More informationLecture 4: Codes based on Concatenation
Lecture 4: Codes based on Concatenation Error-Correcting Codes (Spring 206) Rutgers University Swastik Kopparty Scribe: Aditya Potukuchi and Meng-Tsung Tsai Overview In the last lecture, we studied codes
More informationIntroduction to Low-Density Parity Check Codes. Brian Kurkoski
Introduction to Low-Density Parity Check Codes Brian Kurkoski kurkoski@ice.uec.ac.jp Outline: Low Density Parity Check Codes Review block codes History Low Density Parity Check Codes Gallager s LDPC code
More informationLecture 19: Elias-Bassalygo Bound
Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 2007) Lecturer: Atri Rudra Lecture 19: Elias-Bassalygo Bound October 10, 2007 Scribe: Michael Pfetsch & Atri Rudra In the last lecture,
More informationPhase Transitions of Random Codes and GV-Bounds
Phase Transitions of Random Codes and GV-Bounds Yun Fan Math Dept, CCNU A joint work with Ling, Liu, Xing Oct 2011 Y. Fan (CCNU) Phase Transitions of Random Codes and GV-Bounds Oct 2011 1 / 36 Phase Transitions
More informationNotes for the Hong Kong Lectures on Algorithmic Coding Theory. Luca Trevisan. January 7, 2007
Notes for the Hong Kong Lectures on Algorithmic Coding Theory Luca Trevisan January 7, 2007 These notes are excerpted from the paper Some Applications of Coding Theory in Computational Complexity [Tre04].
More informationList and local error-correction
List and local error-correction Venkatesan Guruswami Carnegie Mellon University 8th North American Summer School of Information Theory (NASIT) San Diego, CA August 11, 2015 Venkat Guruswami (CMU) List
More informationLecture 4: Proof of Shannon s theorem and an explicit code
CSE 533: Error-Correcting Codes (Autumn 006 Lecture 4: Proof of Shannon s theorem and an explicit code October 11, 006 Lecturer: Venkatesan Guruswami Scribe: Atri Rudra 1 Overview Last lecture we stated
More informationLecture 12: Reed-Solomon Codes
Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 007) Lecture 1: Reed-Solomon Codes September 8, 007 Lecturer: Atri Rudra Scribe: Michel Kulhandjian Last lecture we saw the proof
More informationCSCI 2570 Introduction to Nanocomputing
CSCI 2570 Introduction to Nanocomputing Information Theory John E Savage What is Information Theory Introduced by Claude Shannon. See Wikipedia Two foci: a) data compression and b) reliable communication
More informationA list-decodable code with local encoding and decoding
A list-decodable code with local encoding and decoding Marius Zimand Towson University Department of Computer and Information Sciences Baltimore, MD http://triton.towson.edu/ mzimand Abstract For arbitrary
More informationEE229B - Final Project. Capacity-Approaching Low-Density Parity-Check Codes
EE229B - Final Project Capacity-Approaching Low-Density Parity-Check Codes Pierre Garrigues EECS department, UC Berkeley garrigue@eecs.berkeley.edu May 13, 2005 Abstract The class of low-density parity-check
More informationNotes 3: Stochastic channels and noisy coding theorem bound. 1 Model of information communication and noisy channel
Introduction to Coding Theory CMU: Spring 2010 Notes 3: Stochastic channels and noisy coding theorem bound January 2010 Lecturer: Venkatesan Guruswami Scribe: Venkatesan Guruswami We now turn to the basic
More informationList Decoding Algorithms for Certain Concatenated Codes
List Decoding Algorithms for Certain Concatenated Codes Venkatesan Guruswami Madhu Sudan November, 2000 Abstract We give efficient (polynomial-time) list-decoding algorithms for certain families of errorcorrecting
More informationLocality in Coding Theory
Locality in Coding Theory Madhu Sudan Harvard April 9, 2016 Skoltech: Locality in Coding Theory 1 Error-Correcting Codes (Linear) Code CC FF qq nn. FF qq : Finite field with qq elements. nn block length
More informationDecoding Concatenated Codes using Soft Information
Decoding Concatenated Codes using Soft Information Venkatesan Guruswami University of California at Berkeley Computer Science Division Berkeley, CA 94720. venkat@lcs.mit.edu Madhu Sudan MIT Laboratory
More informationNotes 10: List Decoding Reed-Solomon Codes and Concatenated codes
Introduction to Coding Theory CMU: Spring 010 Notes 10: List Decoding Reed-Solomon Codes and Concatenated codes April 010 Lecturer: Venkatesan Guruswami Scribe: Venkat Guruswami & Ali Kemal Sinop DRAFT
More informationCS151 Complexity Theory. Lecture 9 May 1, 2017
CS151 Complexity Theory Lecture 9 Hardness vs. randomness We have shown: If one-way permutations exist then BPP δ>0 TIME(2 nδ ) ( EXP simulation is better than brute force, but just barely stronger assumptions
More informationLecture 3: Error Correcting Codes
CS 880: Pseudorandomness and Derandomization 1/30/2013 Lecture 3: Error Correcting Codes Instructors: Holger Dell and Dieter van Melkebeek Scribe: Xi Wu In this lecture we review some background on error
More informationApproaching Blokh-Zyablov Error Exponent with Linear-Time Encodable/Decodable Codes
Approaching Blokh-Zyablov Error Exponent with Linear-Time Encodable/Decodable Codes 1 Zheng Wang, Student Member, IEEE, Jie Luo, Member, IEEE arxiv:0808.3756v1 [cs.it] 27 Aug 2008 Abstract We show that
More informationRelaxed Locally Correctable Codes in Computationally Bounded Channels
Relaxed Locally Correctable Codes in Computationally Bounded Channels Elena Grigorescu (Purdue) Joint with Jeremiah Blocki (Purdue), Venkata Gandikota (JHU), Samson Zhou (Purdue) Classical Locally Decodable/Correctable
More informationAnd for polynomials with coefficients in F 2 = Z/2 Euclidean algorithm for gcd s Concept of equality mod M(x) Extended Euclid for inverses mod M(x)
Outline Recall: For integers Euclidean algorithm for finding gcd s Extended Euclid for finding multiplicative inverses Extended Euclid for computing Sun-Ze Test for primitive roots And for polynomials
More information6.895 PCP and Hardness of Approximation MIT, Fall Lecture 3: Coding Theory
6895 PCP and Hardness of Approximation MIT, Fall 2010 Lecture 3: Coding Theory Lecturer: Dana Moshkovitz Scribe: Michael Forbes and Dana Moshkovitz 1 Motivation In the course we will make heavy use of
More informationError Correcting Codes: Combinatorics, Algorithms and Applications Spring Homework Due Monday March 23, 2009 in class
Error Correcting Codes: Combinatorics, Algorithms and Applications Spring 2009 Homework Due Monday March 23, 2009 in class You can collaborate in groups of up to 3. However, the write-ups must be done
More informationEfficiently decodable codes for the binary deletion channel
Efficiently decodable codes for the binary deletion channel Venkatesan Guruswami (venkatg@cs.cmu.edu) Ray Li * (rayyli@stanford.edu) Carnegie Mellon University August 18, 2017 V. Guruswami and R. Li (CMU)
More informationTutorial: Locally decodable codes. UT Austin
Tutorial: Locally decodable codes Anna Gál UT Austin Locally decodable codes Error correcting codes with extra property: Recover (any) one message bit, by reading only a small number of codeword bits.
More informationNotes 7: Justesen codes, Reed-Solomon and concatenated codes decoding. 1 Review - Concatenated codes and Zyablov s tradeoff
Introduction to Coding Theory CMU: Spring 2010 Notes 7: Justesen codes, Reed-Solomon and concatenated codes decoding March 2010 Lecturer: V. Guruswami Scribe: Venkat Guruswami & Balakrishnan Narayanaswamy
More informationAn Introduction to Low Density Parity Check (LDPC) Codes
An Introduction to Low Density Parity Check (LDPC) Codes Jian Sun jian@csee.wvu.edu Wireless Communication Research Laboratory Lane Dept. of Comp. Sci. and Elec. Engr. West Virginia University June 3,
More informationLecture 17: Perfect Codes and Gilbert-Varshamov Bound
Lecture 17: Perfect Codes and Gilbert-Varshamov Bound Maximality of Hamming code Lemma Let C be a code with distance 3, then: C 2n n + 1 Codes that meet this bound: Perfect codes Hamming code is a perfect
More informationELEC 519A Selected Topics in Digital Communications: Information Theory. Hamming Codes and Bounds on Codes
ELEC 519A Selected Topics in Digital Communications: Information Theory Hamming Codes and Bounds on Codes Single Error Correcting Codes 2 Hamming Codes (7,4,3) Hamming code 1 0 0 0 0 1 1 0 1 0 0 1 0 1
More informationShannon s noisy-channel theorem
Shannon s noisy-channel theorem Information theory Amon Elders Korteweg de Vries Institute for Mathematics University of Amsterdam. Tuesday, 26th of Januari Amon Elders (Korteweg de Vries Institute for
More informationfor some error exponent E( R) as a function R,
. Capacity-achieving codes via Forney concatenation Shannon s Noisy Channel Theorem assures us the existence of capacity-achieving codes. However, exhaustive search for the code has double-exponential
More informationLecture 7 September 24
EECS 11: Coding for Digital Communication and Beyond Fall 013 Lecture 7 September 4 Lecturer: Anant Sahai Scribe: Ankush Gupta 7.1 Overview This lecture introduces affine and linear codes. Orthogonal signalling
More informationError Correcting Codes Questions Pool
Error Correcting Codes Questions Pool Amnon Ta-Shma and Dean Doron January 3, 018 General guidelines The questions fall into several categories: (Know). (Mandatory). (Bonus). Make sure you know how to
More informationA Combinatorial Bound on the List Size
1 A Combinatorial Bound on the List Size Yuval Cassuto and Jehoshua Bruck California Institute of Technology Electrical Engineering Department MC 136-93 Pasadena, CA 9115, U.S.A. E-mail: {ycassuto,bruck}@paradise.caltech.edu
More informationLocal correctability of expander codes
Local correctability of expander codes Brett Hemenway Rafail Ostrovsky Mary Wootters IAS April 4, 24 The point(s) of this talk Locally decodable codes are codes which admit sublinear time decoding of small
More informationBelief propagation decoding of quantum channels by passing quantum messages
Belief propagation decoding of quantum channels by passing quantum messages arxiv:67.4833 QIP 27 Joseph M. Renes lempelziv@flickr To do research in quantum information theory, pick a favorite text on classical
More informationLP Decoding Corrects a Constant Fraction of Errors
LP Decoding Corrects a Constant Fraction of Errors Jon Feldman Columbia University (CORC Technical Report TR-2003-08) Cliff Stein Columbia University Tal Malkin Columbia University Rocco A. Servedio Columbia
More informationLinear time list recovery via expander codes
Linear time list recovery via expander codes Brett Hemenway and Mary Wootters June 7 26 Outline Introduction List recovery Expander codes List recovery of expander codes Conclusion Our Results One slide
More informationThe Complexity of the Matroid-Greedoid Partition Problem
The Complexity of the Matroid-Greedoid Partition Problem Vera Asodi and Christopher Umans Abstract We show that the maximum matroid-greedoid partition problem is NP-hard to approximate to within 1/2 +
More informationWhat s New and Exciting in Algebraic and Combinatorial Coding Theory?
What s New and Exciting in Algebraic and Combinatorial Coding Theory? Alexander Vardy University of California San Diego vardy@kilimanjaro.ucsd.edu Notice: Persons attempting to find anything useful in
More informationLecture 03: Polynomial Based Codes
Lecture 03: Polynomial Based Codes Error-Correcting Codes (Spring 016) Rutgers University Swastik Kopparty Scribes: Ross Berkowitz & Amey Bhangale 1 Reed-Solomon Codes Reed Solomon codes are large alphabet
More informationDecoding Reed-Muller codes over product sets
Rutgers University May 30, 2016 Overview Error-correcting codes 1 Error-correcting codes Motivation 2 Reed-Solomon codes Reed-Muller codes 3 Error-correcting codes Motivation Goal: Send a message Don t
More informationDigital Communications III (ECE 154C) Introduction to Coding and Information Theory
Digital Communications III (ECE 154C) Introduction to Coding and Information Theory Tara Javidi These lecture notes were originally developed by late Prof. J. K. Wolf. UC San Diego Spring 2014 1 / 8 I
More informationError-correction up to the information-theoretic limit
Error-correction up to the information-theoretic limit Venkatesan Guruswami Computer Science and Engineering University of Washington Seattle, WA 98105 venkat@cs.washington.edu Atri Rudra Computer Science
More informationEntropies & Information Theory
Entropies & Information Theory LECTURE I Nilanjana Datta University of Cambridge,U.K. See lecture notes on: http://www.qi.damtp.cam.ac.uk/node/223 Quantum Information Theory Born out of Classical Information
More informationList-Decodable Codes
5 List-Decodable Codes The field of coding theory is motivated by the problem of communicating reliably over noisy channels where the data sent over the channel may come out corrupted on the other end,
More informationOn the List-Decodability of Random Linear Codes
On the List-Decodability of Random Linear Codes Venkatesan Guruswami Computer Science Dept. Carnegie Mellon University Johan Håstad School of Computer Science and Communication KTH Swastik Kopparty CSAIL
More informationLecture 18: Shanon s Channel Coding Theorem. Lecture 18: Shanon s Channel Coding Theorem
Channel Definition (Channel) A channel is defined by Λ = (X, Y, Π), where X is the set of input alphabets, Y is the set of output alphabets and Π is the transition probability of obtaining a symbol y Y
More informationGeneral Strong Polarization
General Strong Polarization Madhu Sudan Harvard University Joint work with Jaroslaw Blasiok (Harvard), Venkatesan Gurswami (CMU), Preetum Nakkiran (Harvard) and Atri Rudra (Buffalo) December 4, 2017 IAS:
More informationSection 3 Error Correcting Codes (ECC): Fundamentals
Section 3 Error Correcting Codes (ECC): Fundamentals Communication systems and channel models Definition and examples of ECCs Distance For the contents relevant to distance, Lin & Xing s book, Chapter
More informationLow-density parity-check codes
Low-density parity-check codes From principles to practice Dr. Steve Weller steven.weller@newcastle.edu.au School of Electrical Engineering and Computer Science The University of Newcastle, Callaghan,
More informationLecture 12: November 6, 2017
Information and Coding Theory Autumn 017 Lecturer: Madhur Tulsiani Lecture 1: November 6, 017 Recall: We were looking at codes of the form C : F k p F n p, where p is prime, k is the message length, and
More informationSphere Packing and Shannon s Theorem
Chapter 2 Sphere Packing and Shannon s Theorem In the first section we discuss the basics of block coding on the m-ary symmetric channel. In the second section we see how the geometry of the codespace
More informationBelief-Propagation Decoding of LDPC Codes
LDPC Codes: Motivation Belief-Propagation Decoding of LDPC Codes Amir Bennatan, Princeton University Revolution in coding theory Reliable transmission, rates approaching capacity. BIAWGN, Rate =.5, Threshold.45
More informationLinear-algebraic list decoding for variants of Reed-Solomon codes
Electronic Colloquium on Computational Complexity, Report No. 73 (2012) Linear-algebraic list decoding for variants of Reed-Solomon codes VENKATESAN GURUSWAMI CAROL WANG Computer Science Department Carnegie
More informationIMPROVING THE ALPHABET-SIZE IN EXPANDER BASED CODE CONSTRUCTIONS
IMPROVING THE ALPHABET-SIZE IN EXPANDER BASED CODE CONSTRUCTIONS 1 Abstract Various code constructions use expander graphs to improve the error resilience. Often the use of expanding graphs comes at the
More informationBlock Codes :Algorithms in the Real World
Block Codes 5-853:Algorithms in the Real World Error Correcting Codes II Reed-Solomon Codes Concatenated Codes Overview of some topics in coding Low Density Parity Check Codes (aka Expander Codes) -Network
More informationImproved Decoding of Reed Solomon and Algebraic-Geometry Codes. Venkatesan Guruswami and Madhu Sudan /99$ IEEE
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 45, NO 6, SEPTEMBER 1999 1757 Improved Decoding of Reed Solomon and Algebraic-Geometry Codes Venkatesan Guruswami and Madhu Sudan Abstract Given an error-correcting
More informationLecture 4 Noisy Channel Coding
Lecture 4 Noisy Channel Coding I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw October 9, 2015 1 / 56 I-Hsiang Wang IT Lecture 4 The Channel Coding Problem
More informationAlgebraic Geometry Codes. Shelly Manber. Linear Codes. Algebraic Geometry Codes. Example: Hermitian. Shelly Manber. Codes. Decoding.
Linear December 2, 2011 References Linear Main Source: Stichtenoth, Henning. Function Fields and. Springer, 2009. Other Sources: Høholdt, Lint and Pellikaan. geometry codes. Handbook of Coding Theory,
More informationExplicit Capacity-Achieving List-Decodable Codes or Decoding Folded Reed-Solomon Codes up to their Distance
Explicit Capacity-Achieving List-Decodable Codes or Decoding Folded Reed-Solomon Codes up to their Distance Venkatesan Guruswami Atri Rudra Department of Computer Science and Engineering University of
More informationOptimal Rate and Maximum Erasure Probability LDPC Codes in Binary Erasure Channel
Optimal Rate and Maximum Erasure Probability LDPC Codes in Binary Erasure Channel H. Tavakoli Electrical Engineering Department K.N. Toosi University of Technology, Tehran, Iran tavakoli@ee.kntu.ac.ir
More informationDecoding Codes on Graphs
Decoding Codes on Graphs 2. Probabilistic Decoding A S Madhu and Aditya Nori 1.Int roduct ion A S Madhu Aditya Nori A S Madhu and Aditya Nori are graduate students with the Department of Computer Science
More informationLecture 21: P vs BPP 2
Advanced Complexity Theory Spring 206 Prof. Dana Moshkovitz Lecture 2: P vs BPP 2 Overview In the previous lecture, we began our discussion of pseudorandomness. We presented the Blum- Micali definition
More informationSparse Superposition Codes for the Gaussian Channel
Sparse Superposition Codes for the Gaussian Channel Florent Krzakala (LPS, Ecole Normale Supérieure, France) J. Barbier (ENS) arxiv:1403.8024 presented at ISIT 14 Long version in preparation Communication
More informationLower Bounds on the Graphical Complexity of Finite-Length LDPC Codes
Lower Bounds on the Graphical Complexity of Finite-Length LDPC Codes Igal Sason Department of Electrical Engineering Technion - Israel Institute of Technology Haifa 32000, Israel 2009 IEEE International
More information: Error Correcting Codes. October 2017 Lecture 1
03683072: Error Correcting Codes. October 2017 Lecture 1 First Definitions and Basic Codes Amnon Ta-Shma and Dean Doron 1 Error Correcting Codes Basics Definition 1. An (n, K, d) q code is a subset of
More informationCoding problems for memory and storage applications
.. Coding problems for memory and storage applications Alexander Barg University of Maryland January 27, 2015 A. Barg (UMD) Coding for memory and storage January 27, 2015 1 / 73 Codes with locality Introduction:
More informationNew constructions of WOM codes using the Wozencraft ensemble
New constructions of WOM codes using the Wozencraft ensemble Amir Shpilka Abstract In this paper we give several new constructions of WOM codes. The novelty in our constructions is the use of the so called
More informationExplicit List-Decodable Codes with Optimal Rate for Computationally Bounded Channels
Explicit List-Decodable Codes with Optimal Rate for Computationally Bounded Channels Ronen Shaltiel 1 and Jad Silbak 2 1 Department of Computer Science, University of Haifa, Israel ronen@cs.haifa.ac.il
More informationSIGACT News Complexity Theory Column 25
SIGACT News Complexity Theory Column 25 Lane A. Hemaspaandra Dept. of Computer Science, University of Rochester Rochester, NY 14627, USA lane@cs.rochester.edu Introduction to Complexity Theory Column 25
More information(each row defines a probability distribution). Given n-strings x X n, y Y n we can use the absence of memory in the channel to compute
ENEE 739C: Advanced Topics in Signal Processing: Coding Theory Instructor: Alexander Barg Lecture 6 (draft; 9/6/03. Error exponents for Discrete Memoryless Channels http://www.enee.umd.edu/ abarg/enee739c/course.html
More informationBifurcations in iterative decoding and root locus plots
Published in IET Control Theory and Applications Received on 12th March 2008 Revised on 26th August 2008 ISSN 1751-8644 Bifurcations in iterative decoding and root locus plots C.M. Kellett S.R. Weller
More informationLow-complexity error correction in LDPC codes with constituent RS codes 1
Eleventh International Workshop on Algebraic and Combinatorial Coding Theory June 16-22, 2008, Pamporovo, Bulgaria pp. 348-353 Low-complexity error correction in LDPC codes with constituent RS codes 1
More informationImproving the Alphabet Size in Expander Based Code Constructions
Tel Aviv University Raymond and Beverly Sackler Faculty of Exact Sciences School of Computer Sciences Improving the Alphabet Size in Expander Based Code Constructions Submitted as a partial fulfillment
More informationGeneral Strong Polarization
General Strong Polarization Madhu Sudan Harvard University Joint work with Jaroslaw Blasiok (Harvard), Venkatesan Guruswami (CMU), Preetum Nakkiran (Harvard) and Atri Rudra (Buffalo) Oct. 8, 018 Berkeley:
More informationA Public Key Encryption Scheme Based on the Polynomial Reconstruction Problem
A Public Key Encryption Scheme Based on the Polynomial Reconstruction Problem Daniel Augot and Matthieu Finiasz INRIA, Domaine de Voluceau F-78153 Le Chesnay CEDEX Abstract. The Polynomial Reconstruction
More informationTHIS paper is aimed at designing efficient decoding algorithms
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 7, NOVEMBER 1999 2333 Sort-and-Match Algorithm for Soft-Decision Decoding Ilya Dumer, Member, IEEE Abstract Let a q-ary linear (n; k)-code C be used
More informationCOMPSCI 650 Applied Information Theory Apr 5, Lecture 18. Instructor: Arya Mazumdar Scribe: Hamed Zamani, Hadi Zolfaghari, Fatemeh Rezaei
COMPSCI 650 Applied Information Theory Apr 5, 2016 Lecture 18 Instructor: Arya Mazumdar Scribe: Hamed Zamani, Hadi Zolfaghari, Fatemeh Rezaei 1 Correcting Errors in Linear Codes Suppose someone is to send
More informationMATH Examination for the Module MATH-3152 (May 2009) Coding Theory. Time allowed: 2 hours. S = q
MATH-315201 This question paper consists of 6 printed pages, each of which is identified by the reference MATH-3152 Only approved basic scientific calculators may be used. c UNIVERSITY OF LEEDS Examination
More informationTURBO codes [7] and low-density parity-check (LDPC)
82 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 53, NO. 1, JANUARY 2007 LP Decoding Corrects a Constant Fraction of Errors Jon Feldman, Tal Malkin, Rocco A. Servedio, Cliff Stein, and Martin J. Wainwright,
More informationNew Steganographic scheme based of Reed- Solomon codes
New Steganographic scheme based of Reed- Solomon codes I. DIOP; S.M FARSSI ;O. KHOUMA ; H. B DIOUF ; K.TALL ; K.SYLLA Ecole Supérieure Polytechnique de l Université Dakar Sénégal Email: idydiop@yahoo.fr;
More informationExercise 1. = P(y a 1)P(a 1 )
Chapter 7 Channel Capacity Exercise 1 A source produces independent, equally probable symbols from an alphabet {a 1, a 2 } at a rate of one symbol every 3 seconds. These symbols are transmitted over a
More informationList Decoding of Error-Correcting Codes
List Decoding of Error-Correcting Codes by Venkatesan Guruswami B.Tech., Indian Institute of Technology, Madras (1997) S.M., Massachusetts Institute of Technology (1999) Submitted to the Department of
More informationAn introduction to basic information theory. Hampus Wessman
An introduction to basic information theory Hampus Wessman Abstract We give a short and simple introduction to basic information theory, by stripping away all the non-essentials. Theoretical bounds on
More informationList Decoding in Average-Case Complexity and Pseudorandomness
List Decoding in Average-Case Complexity and Pseudorandomness Venkatesan Guruswami Department of Computer Science and Engineering University of Washington Seattle, WA, U.S.A. Email: venkat@cs.washington.edu
More informationLIST decoding was introduced independently by Elias [7] Combinatorial Bounds for List Decoding
GURUSWAMI, HÅSTAD, SUDAN, AND ZUCKERMAN 1 Combinatorial Bounds for List Decoding Venkatesan Guruswami Johan Håstad Madhu Sudan David Zuckerman Abstract Informally, an error-correcting code has nice listdecodability
More informationLecture 11: Polar codes construction
15-859: Information Theory and Applications in TCS CMU: Spring 2013 Lecturer: Venkatesan Guruswami Lecture 11: Polar codes construction February 26, 2013 Scribe: Dan Stahlke 1 Polar codes: recap of last
More informationLecture 9: List decoding Reed-Solomon and Folded Reed-Solomon codes
Lecture 9: List decoding Reed-Solomon and Folded Reed-Solomon codes Error-Correcting Codes (Spring 2016) Rutgers University Swastik Kopparty Scribes: John Kim and Pat Devlin 1 List decoding review Definition
More informationMaking Error Correcting Codes Work for Flash Memory
Making Error Correcting Codes Work for Flash Memory Part I: Primer on ECC, basics of BCH and LDPC codes Lara Dolecek Laboratory for Robust Information Systems (LORIS) Center on Development of Emerging
More informationGuess & Check Codes for Deletions, Insertions, and Synchronization
Guess & Check Codes for Deletions, Insertions, and Synchronization Serge Kas Hanna, Salim El Rouayheb ECE Department, Rutgers University sergekhanna@rutgersedu, salimelrouayheb@rutgersedu arxiv:759569v3
More information