Fountain Codes. Amin Shokrollahi EPFL
|
|
- Hugh Wade
- 6 years ago
- Views:
Transcription
1 Fountain Codes Amin Shokrollahi EPFL
2 Content Fountain Codes LT-Codes Raptor Codes Extensions
3 Transmission Problem Transmit information from one sender to multiple receivers where the channel between sender and each receiver is an erasure channel with unknown erasure probability. Want the transmission to be such that the transmission rate is very close to the capacity on all the transmssion channels, simultaneously. Current protocols do not meet our demands (TCP/IP, UDP,...) Traditional codes are not able to meet our demands. Fountain codes are the right objects!
4 Fountain Codes D k probability distribution on F k 2. A Fountain code with parameters (k, D k ) is a k-dimensional subpsace of F N 2 in which every coordinate function is sampled independently from D k. The coordinates of the input vector in F k 2 are called the input symbols. The coordinates of the code are called the output symbols. Example: (X 4, X 1 + X 2, X 1 + X 6 + X 93, X 7 + X 512,...) Unlike traditional codes, fountain codes are usually constructed on the fly.
5 Fountain codes Input Symbols Output Symbols
6 Encoding Encoding (x 1,..., x k ) F k 2: to compute value of a coordinate, sample from D k to obtain linear from X i1 + + X il, and add x i1 + + x il. Encoding algorithm uses algorithm for sampling from D k. Count number of arithmetic operations over F 2. Cost of encoding: Expected number of arithmetic operations to compute one coordinate. Cost(Encoding) [Cost(D k ) +] E Dk [Hamming weight].
7 Example: Random Fountain Codes D k is the uniform distribution on F k 2. Encoding: For every coordinate position take a random subset of {1,..., k} and add the input elements according to that subset. Cost of encoding is k/2 ( = expected weight of an element in F k 2 under uniform distribution).
8 C fountain code of dimension k. Decoding A decoding algorithm of overhead ε is an algorithm that can recover the k input symbols of the code from any set of k(1 + ε) output symbols and errs with probability at most 1/k c for some c > 0. The cost of the decoding algorithm is the expected number of operations for recovering the k input symbols.
9 Transmission Problem: Solution For sending k symbols to all the receivers use a fountain code of dimension k which has a decoding algorithm of overhead ε, for small ε. On any of the erasure channels the code used is ε away from the capacity of the channel. But do such fountain codes exist?
10 Example: Random Fountain Codes Theorem. Maximum Likelihood is a decoding algorithm of overhead O(log(k)/k) for random fountain codes. Proof. ML decoding is equivalnet to solving system of equations. M has k columns and k(1 + ε) rows, entries binary i.i.d. random with prob 1/2, then probability that rank of M is not full is at most 2 kε. For ε = O(log(k)/k) probability that rank is not full is at most 1/k c for some constant c.
11 Efficiency Want to construct fountain codes that have low cost encoders and decoders. Example: Gaussian elimination for uniform D k : Encoding cost is O(k), decoding cost is O(k 3 ). But want: Constant encoding cost. Linear decoding cost.
12 LT-Codes First class of efficient fountain codes invented by Michael Luby in 1998 and published in Ω probability distribution on {1,..., k} determined by Ω 1, Ω 2,..., Ω k. Distribution D k : x F k 2: Pr[x] = Ω w, where w = wgt(x). ( w) k Encoding cost: Ω (1). Want this to be small. How small can we get it?
13 Lower Bound on the Cost of Encoding/Decoding Theorem. C LT-code of dimension k and distribution Ω(x) with decoding algorithm of overhead ε. for some c > 0. Ω (1) c ln(k)/(1 + ε). Proof. Collect output symbols and consider covered input symbols. Decoding successful only if all input symbols are covered. E = k(1 + ε)ω (1). ( 1 1 ) E 1 k k c e E/k 1 k c Ω (1) c ln(k)/(1 + ε)
14 LT-Codes: Heuristic Analysis Design degree distribution Ω(x). Call output node released at time d if d input symbols recovered and output symbol becomes of degree one when (d + 1)-st input symbol recovered. Pr[Node released at time d weight = w] = ( ) 1 d 1 k 1 k w (w 1) ( ) d w 2 k. E[Nodes released at time d] = ( 1 d 1 ) ( ) d Ω. k k (1 x)ω (x) = 1 Ω(x) = c 1 + c 2 x + x2 2 + x xl l(l 1) +
15 Put Ω 1 = 1/k, Ω i = 1 i(i 1) Soliton Distribution for 1 i k and normalize. Average degree is H(k) ln(k). (Essentially optimal!) In expectation exactly one output node is released at each round of decoding. But: variance!
16 LT-Codes: BP Decoding and Robust Soliton Distribution Luby has found a robust Soliton distribution such that: Corresponding codes have average degree O(ln(k)), They have decoding algorithms of overhead O(log 2 (k)/ k) and cost O(k log(k)) (BP decoding). Absolutely remarkable, since this matches the information theoretic lower bound!
17 Raptor Codes Want linear time encoding and decoding, but how to circumvent lower bound? Idea: Pre-code the input symbols with a traditional code prior to encoding them with an appropriate LT-code. Precode can recover from any (1 δ)- fraction w.h.p. Precode LT-code can recover a (1 δ)-fraction w.h.p. LT-code Raptor codes are fountain codes!
18 Different types of Raptor Codes LT-codes: Raptor codes with trivial pre-code. Pre-code-only (PCO) codes: Trivial decoding algorithm has overhead ln(1 R)/R which approaches 0 only if R approaches 0.
19 Asymptotically Optimal Raptor Codes Precode: Appropriate LDPC-code. LT-code: Use degree distribution 1 µ + 1 (µx + x x D (D 1) D + xd+1 D where µ ε/2 and D 2/ε to obtain BP decoding algorithm of overhead ε and linear time decoding. )
20 Finite Length Analysis of Raptor Codes Finite Length analysis of LT-codes gives recursion for state generating function (joint work with Dick Karp and Michael Luby) P u 1 (x, y) = P ( u x(1 pu ) + yp u, 1 u + y ( ( 1 u)) 1 Pu x(1 pu ), 1 u y Here p u is the probability that an output node of residual degree larger than one is released when u input nodes remain unrecovered. The expression for this probability can be found on the next page. ).
21 p u = 1 1 K u 1 K K u D i 2 Ω i u(u 1) i=1 K u D i 1 Ω i u D K i Ω i K u i, a b := ( ) a b!. b i=1 K i i=1 K i
22 Finite Length Analysis of Raptor Codes For right-poisson LDPC code with n left and r right nodes and left node degree fractions Λ i the block error probability can be estimated (joint work with Rüdiger Urbanke): A 0 (r, 0) := 1, A 0 (z, o) := 0 for (z, o) (r, 0), A n+1 (z, o) := A n (l, k) l,k d for n 0. ( l )( k l z Λ d )( r l k k+l z o d k 2l+2z+o ( r d) ) Block error probability is upper bounded by ( ) r ( n A s (z, 0) 1 ( r z )) n s Λ d d ( s r. z=0 d d)
23 Combination: A Practical Raptor Code Precode 1: Hamming code. Precode 2: Left-regular, right-poisson LDPC code with left degree 4. LT-code: Use degree distribution Ω(x) = 0.008x x x x 4 + with average degree x x x x x x 66, Resulting code has block error probability at most for k
24 Systematic raptor codes. Further Work Raptor codes for error correction on the q-ary symmetric channel, q large (with Dick Karp, Michael Luby, and Avi Wigderson). Raptor codes for general memoryless symmetric channels (with Omid Etesami and Mehdi Molkaraie). Inactivation decoding for raptor codes and its analysis. Raptor codes for data compression (with Guiseppe Caire, Shlomo Shamai, and Sergio Verdu).
25 What Can We Do With Raptor Codes? Scalable Point-to-Multipoint Data Distribution See accompanying ppt slide.
26 What Can We Do With Raptor Codes? Point-to-point transmission: Send encoded data via UDP and not TCP. Transmission speed increases fold. But: need multirate flow control algorithm. Easy with Fountain Codes! Content Different parts of encoding offered at different rates Clients pull content at their own rate
27 What Can We Do With Raptor Codes? Multipoint-to-multipoint transmission: Receive encoded data from several Fountain encoders encoding the same object. See accompanying ppt slide
28 Conclusion and Further Remarks Fountain Codes are a new class of codes designed for robust, unsynchronized, and scalable transmission of information from multiple senders to multiple receivers over computer networks. The theory of Fountain Codes is very interesting, and also provides new insights into the theory of LDPC codes. They can also be used for channels other than the erasure channel. New asynchronous multicast applications can be based on Fountain Codes, such as asynchronous video on demand, or multiple rate congestion control.
Raptor Codes: From a Math Idea to LTE embms. BIRS, October 2015
Raptor Codes: From a Math Idea to LTE embms BIRS, October 2015 The plan is to... 1 introduce LT codes and Raptor codes 2 provide insights into their design 3 address some common misconceptions 2 / 31 The
More informationWindowed Erasure Codes
Windowed Erasure Codes Chris Studholme Department of Computer Science University of Toronto Email: cvs@csutorontoca Ian Blake Department of Electrical and Computer Eng University of Toronto Email: ifblake@commutorontoca
More informationSpatially Coupled LDPC Codes
Spatially Coupled LDPC Codes Kenta Kasai Tokyo Institute of Technology 30 Aug, 2013 We already have very good codes. Efficiently-decodable asymptotically capacity-approaching codes Irregular LDPC Codes
More informationErasure Codes with a Banded Structure for Hybrid Iterative-ML Decoding
Erasure Codes with a Banded Structure for Hybrid Iterative-ML Decoding Alexandre Soro 1, Mathieu Cunche 2, Jérôme Lacan 1 and Vincent Roca 2 1 Univ of Toulouse, ISAE/DMIA, 10 avenue Edouard Belin, BP 54032-31055
More informationUniversal variable-length data compression of binary sources using fountain codes
Universal variable-length data compression of binary sources using fountain codes Giuseppe Caire Shlomo Shamai Amin Shokrollahi Sergio Verdú Institut Eurecom Technion EPFL Princeton University giuseppe.caire@eurecom.fr,
More informationProofs of Retrievability via Fountain Code
Proofs of Retrievability via Fountain Code Sumanta Sarkar and Reihaneh Safavi-Naini Department of Computer Science, University of Calgary, Canada Foundations and Practice of Security October 25, 2012 Outsourcing
More informationPart III Advanced Coding Techniques
Part III Advanced Coding Techniques José Vieira SPL Signal Processing Laboratory Departamento de Electrónica, Telecomunicações e Informática / IEETA Universidade de Aveiro, Portugal 2010 José Vieira (IEETA,
More informationFountain Uncorrectable Sets and Finite-Length Analysis
Fountain Uncorrectable Sets and Finite-Length Analysis Wen Ji 1, Bo-Wei Chen 2, and Yiqiang Chen 1 1 Beijing Key Laboratory of Mobile Computing and Pervasive Device Institute of Computing Technology, Chinese
More informationEE229B - Final Project. Capacity-Approaching Low-Density Parity-Check Codes
EE229B - Final Project Capacity-Approaching Low-Density Parity-Check Codes Pierre Garrigues EECS department, UC Berkeley garrigue@eecs.berkeley.edu May 13, 2005 Abstract The class of low-density parity-check
More informationDesign and Analysis of LT Codes with Decreasing Ripple Size
Design and Analysis of LT Codes with Decreasing Ripple Size 1 Jesper H. Sørensen, Petar Popovski, Jan Østergaard, Aalborg University, Department of Electronic Systems, E-mail: {jhs, petarp, arxiv:1011.2078v2
More informationLecture 14 October 22
EE 2: Coding for Digital Communication & Beyond Fall 203 Lecture 4 October 22 Lecturer: Prof. Anant Sahai Scribe: Jingyan Wang This lecture covers: LT Code Ideal Soliton Distribution 4. Introduction So
More informationRelaying a Fountain code across multiple nodes
Relaying a Fountain code across multiple nodes Ramarishna Gummadi, R.S.Sreenivas Coordinated Science Lab University of Illinois at Urbana-Champaign {gummadi2, rsree} @uiuc.edu Abstract Fountain codes are
More informationDesign of Raptor Codes in the Low SNR Regime with Applications in Quantum Key Distribution
Design of Raptor Codes in the Low SNR Regime with Applications in Quantum Key Distribution Mahyar Shirvamoghaddam, Sarah J. Johnson, and Andrew M. Lance School of Electrical Engineering and Computer Science,
More informationOn the Design of Raptor Codes for Binary-Input Gaussian Channels
1 On the Design of Raptor Codes for Binary-Input Gaussian Channels Zhong Cheng, Jeff Castura, and Yongyi Mao, Member, IEEE Abstract This paper studies the problem of Raptor-code design for binary-input
More informationAn Introduction to Low Density Parity Check (LDPC) Codes
An Introduction to Low Density Parity Check (LDPC) Codes Jian Sun jian@csee.wvu.edu Wireless Communication Research Laboratory Lane Dept. of Comp. Sci. and Elec. Engr. West Virginia University June 3,
More informationCompressed Sensing and Linear Codes over Real Numbers
Compressed Sensing and Linear Codes over Real Numbers Henry D. Pfister (joint with Fan Zhang) Texas A&M University College Station Information Theory and Applications Workshop UC San Diego January 31st,
More informationMethods for Efficient Network Coding
Methods for Efficient Network Coding Petar Maymounkov and Nicholas J. A. Harvey Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology Cambridge, MA 02139, USA E-mail:
More informationLecture 4 : Introduction to Low-density Parity-check Codes
Lecture 4 : Introduction to Low-density Parity-check Codes LDPC codes are a class of linear block codes with implementable decoders, which provide near-capacity performance. History: 1. LDPC codes were
More informationCoding for loss tolerant systems
Coding for loss tolerant systems Workshop APRETAF, 22 janvier 2009 Mathieu Cunche, Vincent Roca INRIA, équipe Planète INRIA Rhône-Alpes Mathieu Cunche, Vincent Roca The erasure channel Erasure codes Reed-Solomon
More informationIterative Decoding for Wireless Networks
Iterative Decoding for Wireless Networks Thesis by Ravi Palanki In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy California Institute of Technology Pasadena, California
More informationJoint Iterative Decoding of LDPC Codes and Channels with Memory
Joint Iterative Decoding of LDPC Codes and Channels with Memory Henry D. Pfister and Paul H. Siegel University of California, San Diego 3 rd Int l Symposium on Turbo Codes September 1, 2003 Outline Channels
More informationLecture 8: Shannon s Noise Models
Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 2007) Lecture 8: Shannon s Noise Models September 14, 2007 Lecturer: Atri Rudra Scribe: Sandipan Kundu& Atri Rudra Till now we have
More informationIterative Quantization. Using Codes On Graphs
Iterative Quantization Using Codes On Graphs Emin Martinian and Jonathan S. Yedidia 2 Massachusetts Institute of Technology 2 Mitsubishi Electric Research Labs Lossy Data Compression: Encoding: Map source
More informationConvergence analysis for a class of LDPC convolutional codes on the erasure channel
Convergence analysis for a class of LDPC convolutional codes on the erasure channel Sridharan, Arvind; Lentmaier, Michael; Costello Jr., Daniel J.; Zigangirov, Kamil Published in: [Host publication title
More informationInformation Theory. Lecture 10. Network Information Theory (CT15); a focus on channel capacity results
Information Theory Lecture 10 Network Information Theory (CT15); a focus on channel capacity results The (two-user) multiple access channel (15.3) The (two-user) broadcast channel (15.6) The relay channel
More informationMATH Examination for the Module MATH-3152 (May 2009) Coding Theory. Time allowed: 2 hours. S = q
MATH-315201 This question paper consists of 6 printed pages, each of which is identified by the reference MATH-3152 Only approved basic scientific calculators may be used. c UNIVERSITY OF LEEDS Examination
More informationLecture 2 Linear Codes
Lecture 2 Linear Codes 2.1. Linear Codes From now on we want to identify the alphabet Σ with a finite field F q. For general codes, introduced in the last section, the description is hard. For a code of
More informationError-Correcting Codes:
Error-Correcting Codes: Progress & Challenges Madhu Sudan Microsoft/MIT Communication in presence of noise We are not ready Sender Noisy Channel We are now ready Receiver If information is digital, reliability
More informationCapacity Region of the Permutation Channel
Capacity Region of the Permutation Channel John MacLaren Walsh and Steven Weber Abstract We discuss the capacity region of a degraded broadcast channel (DBC) formed from a channel that randomly permutes
More informationSerial Concatenation of RS Codes with Kite. Codes: Performance Analysis, Iterative Decoding and Design
Serial Concatenation of RS Codes with Kite 1 Codes: Performance Analysis, Iterative Decoding and Design arxiv:1104.4927v1 [cs.it] 26 Apr 2011 Xiao Ma, Kai Zhang, Baoming Bai and Xiaoyi Zhang Abstract In
More informationAn Introduction to Algorithmic Coding Theory
An Introduction to Algorithmic Coding Theory M. Amin Shokrollahi Bell Laboratories Part : Codes - A puzzle What do the following problems have in common? 2 Problem : Information Transmission MESSAGE G
More informationCapacity-Achieving Ensembles for the Binary Erasure Channel With Bounded Complexity
Capacity-Achieving Ensembles for the Binary Erasure Channel With Bounded Complexity Henry D. Pfister, Member, Igal Sason, Member, and Rüdiger Urbanke Abstract We present two sequences of ensembles of non-systematic
More informationVariable Rate Channel Capacity. Jie Ren 2013/4/26
Variable Rate Channel Capacity Jie Ren 2013/4/26 Reference This is a introduc?on of Sergio Verdu and Shlomo Shamai s paper. Sergio Verdu and Shlomo Shamai, Variable- Rate Channel Capacity, IEEE Transac?ons
More informationAnalysis of Rate-distortion Functions and Congestion Control in Scalable Internet Video Streaming
Analysis of Rate-distortion Functions and Congestion Control in Scalable Internet Video Streaming Min Dai Electrical Engineering, Texas A&M University Dmitri Loguinov Computer Science, Texas A&M University
More informationOn Generalized EXIT Charts of LDPC Code Ensembles over Binary-Input Output-Symmetric Memoryless Channels
2012 IEEE International Symposium on Information Theory Proceedings On Generalied EXIT Charts of LDPC Code Ensembles over Binary-Input Output-Symmetric Memoryless Channels H Mamani 1, H Saeedi 1, A Eslami
More informationLecture 4 Noisy Channel Coding
Lecture 4 Noisy Channel Coding I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw October 9, 2015 1 / 56 I-Hsiang Wang IT Lecture 4 The Channel Coding Problem
More informationIntroduction to Low-Density Parity Check Codes. Brian Kurkoski
Introduction to Low-Density Parity Check Codes Brian Kurkoski kurkoski@ice.uec.ac.jp Outline: Low Density Parity Check Codes Review block codes History Low Density Parity Check Codes Gallager s LDPC code
More informationMa/CS 6b Class 25: Error Correcting Codes 2
Ma/CS 6b Class 25: Error Correcting Codes 2 By Adam Sheffer Recall: Codes V n the set of binary sequences of length n. For example, V 3 = 000,001,010,011,100,101,110,111. Codes of length n are subsets
More informationLecture 7. Union bound for reducing M-ary to binary hypothesis testing
Lecture 7 Agenda for the lecture M-ary hypothesis testing and the MAP rule Union bound for reducing M-ary to binary hypothesis testing Introduction of the channel coding problem 7.1 M-ary hypothesis testing
More informationECEN 655: Advanced Channel Coding
ECEN 655: Advanced Channel Coding Course Introduction Henry D. Pfister Department of Electrical and Computer Engineering Texas A&M University ECEN 655: Advanced Channel Coding 1 / 19 Outline 1 History
More informationInformation Theory with Applications, Math6397 Lecture Notes from September 30, 2014 taken by Ilknur Telkes
Information Theory with Applications, Math6397 Lecture Notes from September 3, 24 taken by Ilknur Telkes Last Time Kraft inequality (sep.or) prefix code Shannon Fano code Bound for average code-word length
More informationSingle-Gaussian Messages and Noise Thresholds for Low-Density Lattice Codes
Single-Gaussian Messages and Noise Thresholds for Low-Density Lattice Codes Brian M. Kurkoski, Kazuhiko Yamaguchi and Kingo Kobayashi kurkoski@ice.uec.ac.jp Dept. of Information and Communications Engineering
More informationRCA Analysis of the Polar Codes and the use of Feedback to aid Polarization at Short Blocklengths
RCA Analysis of the Polar Codes and the use of Feedback to aid Polarization at Short Blocklengths Kasra Vakilinia, Dariush Divsalar*, and Richard D. Wesel Department of Electrical Engineering, University
More informationX 1 : X Table 1: Y = X X 2
ECE 534: Elements of Information Theory, Fall 200 Homework 3 Solutions (ALL DUE to Kenneth S. Palacio Baus) December, 200. Problem 5.20. Multiple access (a) Find the capacity region for the multiple-access
More informationAn Introduction to (Network) Coding Theory
An to (Network) Anna-Lena Horlemann-Trautmann University of St. Gallen, Switzerland April 24th, 2018 Outline 1 Reed-Solomon Codes 2 Network Gabidulin Codes 3 Summary and Outlook A little bit of history
More informationLow-density parity-check codes
Low-density parity-check codes From principles to practice Dr. Steve Weller steven.weller@newcastle.edu.au School of Electrical Engineering and Computer Science The University of Newcastle, Callaghan,
More informationLower Bounds on the Graphical Complexity of Finite-Length LDPC Codes
Lower Bounds on the Graphical Complexity of Finite-Length LDPC Codes Igal Sason Department of Electrical Engineering Technion - Israel Institute of Technology Haifa 32000, Israel 2009 IEEE International
More information16.36 Communication Systems Engineering
MIT OpenCourseWare http://ocw.mit.edu 16.36 Communication Systems Engineering Spring 2009 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 16.36: Communication
More informationAn Introduction to (Network) Coding Theory
An Introduction to (Network) Coding Theory Anna-Lena Horlemann-Trautmann University of St. Gallen, Switzerland July 12th, 2018 1 Coding Theory Introduction Reed-Solomon codes 2 Introduction Coherent network
More informationCapacity of a channel Shannon s second theorem. Information Theory 1/33
Capacity of a channel Shannon s second theorem Information Theory 1/33 Outline 1. Memoryless channels, examples ; 2. Capacity ; 3. Symmetric channels ; 4. Channel Coding ; 5. Shannon s second theorem,
More informationGLDPC-Staircase AL-FEC codes: A Fundamental study and New results
GLDPC-Staircase AL-FEC codes: A Fundamental study and New results Ferdaouss Mattoussi, Vincent Roca, Bessam Sayadi To cite this version: Ferdaouss Mattoussi, Vincent Roca, Bessam Sayadi. GLDPC-Staircase
More informationDigital Communications III (ECE 154C) Introduction to Coding and Information Theory
Digital Communications III (ECE 154C) Introduction to Coding and Information Theory Tara Javidi These lecture notes were originally developed by late Prof. J. K. Wolf. UC San Diego Spring 2014 1 / 8 I
More informationLecture 8: Channel and source-channel coding theorems; BEC & linear codes. 1 Intuitive justification for upper bound on channel capacity
5-859: Information Theory and Applications in TCS CMU: Spring 23 Lecture 8: Channel and source-channel coding theorems; BEC & linear codes February 7, 23 Lecturer: Venkatesan Guruswami Scribe: Dan Stahlke
More informationExercise 1. = P(y a 1)P(a 1 )
Chapter 7 Channel Capacity Exercise 1 A source produces independent, equally probable symbols from an alphabet {a 1, a 2 } at a rate of one symbol every 3 seconds. These symbols are transmitted over a
More informationLDPC Codes. Slides originally from I. Land p.1
Slides originally from I. Land p.1 LDPC Codes Definition of LDPC Codes Factor Graphs to use in decoding Decoding for binary erasure channels EXIT charts Soft-Output Decoding Turbo principle applied to
More informationLecture 7 MIMO Communica2ons
Wireless Communications Lecture 7 MIMO Communica2ons Prof. Chun-Hung Liu Dept. of Electrical and Computer Engineering National Chiao Tung University Fall 2014 1 Outline MIMO Communications (Chapter 10
More informationGuess & Check Codes for Deletions, Insertions, and Synchronization
Guess & Check Codes for Deletions, Insertions, and Synchronization Serge Kas Hanna, Salim El Rouayheb ECE Department, Rutgers University sergekhanna@rutgersedu, salimelrouayheb@rutgersedu arxiv:759569v3
More informationSource Coding. Master Universitario en Ingeniería de Telecomunicación. I. Santamaría Universidad de Cantabria
Source Coding Master Universitario en Ingeniería de Telecomunicación I. Santamaría Universidad de Cantabria Contents Introduction Asymptotic Equipartition Property Optimal Codes (Huffman Coding) Universal
More informationExact Probability of Erasure and a Decoding Algorithm for Convolutional Codes on the Binary Erasure Channel
Exact Probability of Erasure and a Decoding Algorithm for Convolutional Codes on the Binary Erasure Channel Brian M. Kurkoski, Paul H. Siegel, and Jack K. Wolf Department of Electrical and Computer Engineering
More informationShannon s noisy-channel theorem
Shannon s noisy-channel theorem Information theory Amon Elders Korteweg de Vries Institute for Mathematics University of Amsterdam. Tuesday, 26th of Januari Amon Elders (Korteweg de Vries Institute for
More informationAn Introduction to Low-Density Parity-Check Codes
An Introduction to Low-Density Parity-Check Codes Paul H. Siegel Electrical and Computer Engineering University of California, San Diego 5/ 3/ 7 Copyright 27 by Paul H. Siegel Outline Shannon s Channel
More informationLow-Density Parity-Check Codes
Department of Computer Sciences Applied Algorithms Lab. July 24, 2011 Outline 1 Introduction 2 Algorithms for LDPC 3 Properties 4 Iterative Learning in Crowds 5 Algorithm 6 Results 7 Conclusion PART I
More informationCoding Techniques for Data Storage Systems
Coding Techniques for Data Storage Systems Thomas Mittelholzer IBM Zurich Research Laboratory /8 Göttingen Agenda. Channel Coding and Practical Coding Constraints. Linear Codes 3. Weight Enumerators and
More informationA Tight Upper Bound on the Second-Order Coding Rate of Parallel Gaussian Channels with Feedback
A Tight Upper Bound on the Second-Order Coding Rate of Parallel Gaussian Channels with Feedback Vincent Y. F. Tan (NUS) Joint work with Silas L. Fong (Toronto) 2017 Information Theory Workshop, Kaohsiung,
More informationIntegrated Code Design for a Joint Source and Channel LDPC Coding Scheme
Integrated Code Design for a Joint Source and Channel LDPC Coding Scheme Hsien-Ping Lin Shu Lin and Khaled Abdel-Ghaffar Department of Electrical and Computer Engineering University of California Davis
More informationIterative Encoding of Low-Density Parity-Check Codes
Iterative Encoding of Low-Density Parity-Check Codes David Haley, Alex Grant and John Buetefuer Institute for Telecommunications Research University of South Australia Mawson Lakes Blvd Mawson Lakes SA
More informationTHE seminal paper of Gallager [1, p. 48] suggested to evaluate
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 11, NOVEMBER 2004 2657 Extrinsic Information Transfer Functions: Model and Erasure Channel Properties Alexei Ashikhmin, Member, IEEE, Gerhard Kramer,
More informationHow to Achieve the Capacity of Asymmetric Channels
How to Achieve the Capacity of Asymmetric Channels Marco Mondelli, S. Hamed Hassani, and Rüdiger Urbanke Abstract arxiv:406.7373v5 [cs.it] 3 Jan 208 We survey coding techniques that enable reliable transmission
More informationInformation and Entropy
Information and Entropy Shannon s Separation Principle Source Coding Principles Entropy Variable Length Codes Huffman Codes Joint Sources Arithmetic Codes Adaptive Codes Thomas Wiegand: Digital Image Communication
More informationThe PPM Poisson Channel: Finite-Length Bounds and Code Design
August 21, 2014 The PPM Poisson Channel: Finite-Length Bounds and Code Design Flavio Zabini DEI - University of Bologna and Institute for Communications and Navigation German Aerospace Center (DLR) Balazs
More informationLecture 4 : Adaptive source coding algorithms
Lecture 4 : Adaptive source coding algorithms February 2, 28 Information Theory Outline 1. Motivation ; 2. adaptive Huffman encoding ; 3. Gallager and Knuth s method ; 4. Dictionary methods : Lempel-Ziv
More informationCOMPSCI 650 Applied Information Theory Apr 5, Lecture 18. Instructor: Arya Mazumdar Scribe: Hamed Zamani, Hadi Zolfaghari, Fatemeh Rezaei
COMPSCI 650 Applied Information Theory Apr 5, 2016 Lecture 18 Instructor: Arya Mazumdar Scribe: Hamed Zamani, Hadi Zolfaghari, Fatemeh Rezaei 1 Correcting Errors in Linear Codes Suppose someone is to send
More informationECE Information theory Final
ECE 776 - Information theory Final Q1 (1 point) We would like to compress a Gaussian source with zero mean and variance 1 We consider two strategies In the first, we quantize with a step size so that the
More informationComputing the threshold shift for general channels
Computing the threshold shift for general channels Jeremie Ezri and Rüdiger Urbanke LTHC EPFL jeremie.ezri@epfl.ch, ruediger.urbanke@epfl.ch Andrea Montanari and Sewoong Oh Electrical Engineering Department
More informationSingle-letter Characterization of Signal Estimation from Linear Measurements
Single-letter Characterization of Signal Estimation from Linear Measurements Dongning Guo Dror Baron Shlomo Shamai The work has been supported by the European Commission in the framework of the FP7 Network
More information1 1 0, g Exercise 1. Generator polynomials of a convolutional code, given in binary form, are g
Exercise Generator polynomials of a convolutional code, given in binary form, are g 0, g 2 0 ja g 3. a) Sketch the encoding circuit. b) Sketch the state diagram. c) Find the transfer function TD. d) What
More informationECE Information theory Final (Fall 2008)
ECE 776 - Information theory Final (Fall 2008) Q.1. (1 point) Consider the following bursty transmission scheme for a Gaussian channel with noise power N and average power constraint P (i.e., 1/n X n i=1
More informationNetwork Coding and Schubert Varieties over Finite Fields
Network Coding and Schubert Varieties over Finite Fields Anna-Lena Horlemann-Trautmann Algorithmics Laboratory, EPFL, Schweiz October 12th, 2016 University of Kentucky What is this talk about? 1 / 31 Overview
More informationSparse Regression Codes for Multi-terminal Source and Channel Coding
Sparse Regression Codes for Multi-terminal Source and Channel Coding Ramji Venkataramanan Yale University Sekhar Tatikonda Allerton 2012 1 / 20 Compression with Side-Information X Encoder Rate R Decoder
More informationPractical Polar Code Construction Using Generalised Generator Matrices
Practical Polar Code Construction Using Generalised Generator Matrices Berksan Serbetci and Ali E. Pusane Department of Electrical and Electronics Engineering Bogazici University Istanbul, Turkey E-mail:
More informationINSPIRED by Shannon s revolutionary work and his seminal
Raptor Codes in the Low SNR Regime Mahyar Shirvanimoghaddam, Member, IEEE, and Sarah Johnson, Member, IEEE arxiv:5.883v [cs.it] 8 Oct 5 Abstract In this paper, we revisit the design of Raptor codes for
More informationShort Polar Codes. Peihong Yuan. Chair for Communications Engineering. Technische Universität München
Short Polar Codes Peihong Yuan Chair for Communications Engineering July 26, 2016 LNT & DLR Summer Workshop on Coding 1 / 23 Outline 1 Motivation 2 Improve the Distance Property 3 Simulation Results 4
More informationAppendix B Information theory from first principles
Appendix B Information theory from first principles This appendix discusses the information theory behind the capacity expressions used in the book. Section 8.3.4 is the only part of the book that supposes
More informationLecture 11: Polar codes construction
15-859: Information Theory and Applications in TCS CMU: Spring 2013 Lecturer: Venkatesan Guruswami Lecture 11: Polar codes construction February 26, 2013 Scribe: Dan Stahlke 1 Polar codes: recap of last
More informationCommunication by Regression: Sparse Superposition Codes
Communication by Regression: Sparse Superposition Codes Department of Statistics, Yale University Coauthors: Antony Joseph and Sanghee Cho February 21, 2013, University of Texas Channel Communication Set-up
More informationChapter 2. Error Correcting Codes. 2.1 Basic Notions
Chapter 2 Error Correcting Codes The identification number schemes we discussed in the previous chapter give us the ability to determine if an error has been made in recording or transmitting information.
More informationOne-Bit LDPC Message Passing Decoding Based on Maximization of Mutual Information
One-Bit LDPC Message Passing Decoding Based on Maximization of Mutual Information ZOU Sheng and Brian M. Kurkoski kurkoski@ice.uec.ac.jp University of Electro-Communications Tokyo, Japan University of
More informationCompression and Coding
Compression and Coding Theory and Applications Part 1: Fundamentals Gloria Menegaz 1 Transmitter (Encoder) What is the problem? Receiver (Decoder) Transformation information unit Channel Ordering (significance)
More informationCSCI 2570 Introduction to Nanocomputing
CSCI 2570 Introduction to Nanocomputing Information Theory John E Savage What is Information Theory Introduced by Claude Shannon. See Wikipedia Two foci: a) data compression and b) reliable communication
More informationAPPLICATIONS. Quantum Communications
SOFT PROCESSING TECHNIQUES FOR QUANTUM KEY DISTRIBUTION APPLICATIONS Marina Mondin January 27, 2012 Quantum Communications In the past decades, the key to improving computer performance has been the reduction
More informationBelief propagation decoding of quantum channels by passing quantum messages
Belief propagation decoding of quantum channels by passing quantum messages arxiv:67.4833 QIP 27 Joseph M. Renes lempelziv@flickr To do research in quantum information theory, pick a favorite text on classical
More informationMessage Passing Algorithm with MAP Decoding on Zigzag Cycles for Non-binary LDPC Codes
Message Passing Algorithm with MAP Decoding on Zigzag Cycles for Non-binary LDPC Codes Takayuki Nozaki 1, Kenta Kasai 2, Kohichi Sakaniwa 2 1 Kanagawa University 2 Tokyo Institute of Technology July 12th,
More informationDistributed Arithmetic Coding
Distributed Arithmetic Coding Marco Grangetto, Member, IEEE, Enrico Magli, Member, IEEE, Gabriella Olmo, Senior Member, IEEE Abstract We propose a distributed binary arithmetic coder for Slepian-Wolf coding
More informationVariable Length Codes for Degraded Broadcast Channels
Variable Length Codes for Degraded Broadcast Channels Stéphane Musy School of Computer and Communication Sciences, EPFL CH-1015 Lausanne, Switzerland Email: stephane.musy@ep.ch Abstract This paper investigates
More informationNAME... Soc. Sec. #... Remote Location... (if on campus write campus) FINAL EXAM EE568 KUMAR. Sp ' 00
NAME... Soc. Sec. #... Remote Location... (if on campus write campus) FINAL EXAM EE568 KUMAR Sp ' 00 May 3 OPEN BOOK exam (students are permitted to bring in textbooks, handwritten notes, lecture notes
More informationGraph-based codes for flash memory
1/28 Graph-based codes for flash memory Discrete Mathematics Seminar September 3, 2013 Katie Haymaker Joint work with Professor Christine Kelley University of Nebraska-Lincoln 2/28 Outline 1 Background
More informationDigital communication system. Shannon s separation principle
Digital communication system Representation of the source signal by a stream of (binary) symbols Adaptation to the properties of the transmission channel information source source coder channel coder modulation
More information(Classical) Information Theory III: Noisy channel coding
(Classical) Information Theory III: Noisy channel coding Sibasish Ghosh The Institute of Mathematical Sciences CIT Campus, Taramani, Chennai 600 113, India. p. 1 Abstract What is the best possible way
More informationTurbo Compression. Andrej Rikovsky, Advisor: Pavol Hanus
Turbo Compression Andrej Rikovsky, Advisor: Pavol Hanus Abstract Turbo codes which performs very close to channel capacity in channel coding can be also used to obtain very efficient source coding schemes.
More information