Belief-Propagation Decoding of LDPC Codes
|
|
- Alban Glenn
- 5 years ago
- Views:
Transcription
1 LDPC Codes: Motivation Belief-Propagation Decoding of LDPC Codes Amir Bennatan, Princeton University Revolution in coding theory Reliable transmission, rates approaching capacity. BIAWGN, Rate =.5, Threshold.45 db of Shannon limit. BIAWGN, Rate =.88, Threshold.88 db of Shannon limit. BSC, Rate =.5, Threshold.5 of maximum crossover. BEC, any rate: achieve capacity! Low-complexity decoding: belief propagation 2 History Discrete-time, memoryless channel 963: Invented by Gallager 988: Belief-propagation, by Pearl 993: Turbo-codes, (Berrou, Glavieux, Thitimajshima) 996: Rediscovered (MacKay, Neal, Sipser, Spielman) Discrete time instances,,..., N. Output y n dependent only on x n. 3 4
2 Example: Binary Symmertic Channel (BSC) Decoding, no code case Assume y n =. Which x n was transmitted? Maximum likelihood rule: Pr[X n = Y n = ] =? (Assume p = /4) (assume p = /4) Pr[X n = Y n = ] =? 5 6 Decoding, no code case Decoding, no code case Pr[X n = Y n = ] = = Pr[Y n =,X n = ] Pr[Y n = ] = Pr[Y n = X n = ] Pr[X n = ] Pr[Y n = ] = Pr[Y n = X n = ] /2 Pr[Y n = ] Assumption: equal a priori probabilities: Pr[X n = ] = Pr[X n = ] = /2 Assume y n =. Which x n was transmitted? Pr[X n = Y n = ] =.75 (Assume p = /4) Pr[X n = Y n = ] =.25 Decoder decides: ˆx n = 7 8
3 Maximum likelihood rule: Decoding, no code case ˆx n = argmax Pr[X n = d Y n = y n ] d=, Example: C = Let s focus on x 2. Decoding, code case, Which x 2 was transmitted? y = 9 Example: C = Decoding, code case, y = Pr[X 2 = Y 2 = ] =.75 Pr[X 2 = Y 2 = ] =.25 Old decoding rule, Decoding, code case ˆx n = argmax Pr{ X n = d Y n = y n } d=, Better decoding rule, ˆx n = argmax Pr{ X n = d Y = y,..., Y N = y N, d=, [X,..., X N ] is a codeword } Decoder decides: ˆx 2 = Bad!!!!! 2
4 Example: C = Decoding, code case, With new decoding rule, y = Pr[X 2 = Y = y,x is a codeword] = Pr[X 2 = Y = y,x is a codeword] = Word error vs. bit error Possibility : Minimize probability of word error. Pr[error] = Pr[ˆx x] Possibility 2: At each bit n, minimize probability of bit error. Pr[error in bit n] = Pr[ˆx n x n ] Our focus: bit error Decoder decides: ˆx 2 = ˆx 2 = 3 4 Old decoding rule, Decoding, code case ˆx n = argmax Pr{ X n = d Y n = y n } d=, Better decoding rule, ˆx n = argmax Pr{ X n = d Y = y,..., Y N = y N, d=, (R > is rate of code) [X,..., X N ] is a codeword } Complexity Θ(2 RN ) Status: Old decoding rule, Decoding, code case ˆx n = argmax Pr{X n = d Y n = y n } d=, Bad performance, excellent complexity. New decoding rule, ˆx n = argmax Pr{X n = d Y = y,x is a codeword} d=, Excellent performance, terrible complexity. Any compromise? 5 6
5 Linear binary block codes Parity check matrix. A binary matrix H. e.g. H x = Linear binary block codes Parity check matrix. A binary matrix H. e.g. H x = x x 2... x N =... h h 2... h M x x 2... x N =... x is a codeword H x = x is a codeword H x = h m x =, m =,..., M Each equation h m x = called a parity check. 7 8 ˆx n Linear binary block codes X is a codeword H X = h m X =, m =,..., M = argmaxpr{x n = d Y = y, X is a codeword} d=, = argmaxpr{x n = d Y = y, H X = } d=, = argmaxpr{x n = d Y = y,y 2 = y 2,..., Y N = y N d=, h X =,h 2 X =,...,h M X = } Decoding, code case Old decoding rule, ˆx n = argmaxpr{x n = d Y n = y n } d=, New decoding rule, ˆx n = argmaxpr{x n = d Y = y,y 2 = y 2,..., Y N = y N d=, h X =,h 2 X =,...,h M X = } Compromise: Use some {y n }, some parity checks! ˆx n = argmaxpr{x n = d Y l = y l,..., Y ll = y ll d=, h m X =,...,h mk X = } 9 2
6 Compromise: Iterative Decoding. Start with old decoding rule, Some formal stuff... Let w = [w,..., w N ], assume w / C, ˆx n = argmaxpr{x n = d Y n = y n } d=, Pr[X = w] =? 2. Iteratively add more h s and y s, ˆx n = argmaxpr{x n = d Y l = y l,..., Y ll = y ll d=, h m X =,...,h mk X = } How? Belief propagation Answer: Pr[X = w] = ( ) N 2 Pr[X = w X is a codeword] = Formal probability model 2 22 Properties of formal probability model Concepts of belief-propagation. Assumes no code 2. Valid mathematically 3. Non-restrictive 4. We can express other useful values.. Graph based 2. Beliefs 3. Iterative message passing 4. Extrinsic information rule 5. Ignore loops 23 24
7 Graph based Graph based Variable node #n, corresponds to time slot to unknown code bit X n. to received channel output y n. Check node #m, corresponds: to parity-check h m Parity check Concepts of belief-propagation. Graph based 2. Beliefs 3. Iterative message passing 4. Extrinsic information rule 5. Ignore loops h X = X + X 2 + X 3 + X 6 = Check node connected to participating variables
8 Belief Propagation Belief Propagation The knowledge ( beliefs ) we have: Y = y,y 2 = y 2,..., Y N = y N h X =,h 2 X =,...,h M X = Divide it between the nodes. Variable nodes know channel outputs. Variable n knows value of y n Belief Propagation Concepts of belief-propagation. Graph based 2. Beliefs 3. Iterative message passing 4. Extrinsic information rule 5. Ignore loops Check nodes known parity checks. Check m knows that h m X =. 3 32
9 Iterative message passing Iterative message passing Nodes communicate using messages. Messages are sent through edges to neighboring nodes. Each message is a number m [, ]. Message from variable n to check m: V n m = Pr[X n = some h s and some y s] Iterative message passing Iterative message passing Message from check m to check n: C m n = Pr[X n = other h s and other y s] Rightbound and leftbound iterations. Rightbound iteration. Variables send messages to checks. Leftbound iteration. Checks send messages to variables
10 Iterative message passing Iterative message passing At node n,. Collect all incoming messages, previous iteration 2. Add my knowledge Rightbound iteration #: At variable n,. Collect all incoming messages, previous iteration (none) 2. Add my knowledge (channel output y n ) V n m = Pr[X n = Y n = y n ] Iterative message passing Iterative message passing Leftbound iteration #: At check node, to variable 3. Collect all incoming messages (V,V 2,V 3,V 6 ) 2. Add my knowledge (parity check X + X 2 + X 3 + X 6 = ) C 3 = Pr[X 3 = X + X 2 + X 3 + X 6 =, Y = y,y 2 = y 2,Y 3 = y 3,Y 6 = y 6 ] Leftbound iteration #: At check node, to variable 3. Collect all incoming messages (V,V 2,V 3,V 6 ) 2. Add my knowledge (parity check X + X 2 + X 3 + X 6 = ) C 3 = Pr[X 3 = X + X 2 + X 3 + X 6 =, Y = y,y 2 = y 2,Y 3 = y 3,Y 6 = y 6 ] Extrinsic information rule: Message to node never function of message from node. 39 4
11 Iterative message passing Some formal stuff... Leftbound iteration #: At check node, to variable 3. Collect all incoming messages (V,V 2,V 6 ) 2. Add my knowledge (parity check X + X 2 + X 3 + X 6 = ) Pr[X =,X 2 = ] = Pr[X = ] Pr[X 2 = ] Pr[X =,Y 2 =,X 2 = ] = Pr[X = ] Pr[Y 2 =,X 2 = ] Pr[X + X 3 =,Y 3 =,X 2 + X 4 =,Y 4 = ] = Pr[X + X 3 =,Y 3 = ] Pr[X 2 + X 4 =,Y 4 = ] C 3 = Pr[X 3 = X + X 2 + X 3 + X 6 =, Y = y,y 2 = y 2,Y 6 = y 6 ] =? 4 42 Iterative message passing Leftbound iteration #: At check node, to variable 3. Collect all incoming messages (V,V 2,V 6 ) 2. Add my knowledge (parity check X + X 2 + X 3 + X 6 = ) C 3 = Pr[X 3 = X + X 2 + X 3 + X 6 =, = 2 Y = y,y 2 = y 2,Y 6 = y 6 ] ( 2V i ) i=,2,6 Leftbound iteration #: At check node, to variable 3. Collect all incoming messages (V,V 2,V 3,V 6 ) 2. Add my knowledge (parity check X + X 2 + X 3 + X 6 = ) C 3 = Pr[X 3 = X + X 2 + X 3 + X 6 =, = 2 Y = y,y 2 = y 2,Y 6 = y 6 ] ( 2V i ) i=,2,
12 Iterative message passing Rightbound iteration #2: From variable 3, to check node 3. Collect all incoming messages (C 3,C 2 3 ) 2. Add my knowledge (channel output y 3 ) V 3 3 = Pr[X 3 = some knowledge] Leftbound iteration #: At check node, to variable 3. Collect all incoming messages (V 4 2,V 3 2,V 5 2 ) 2. Add my knowledge (parity check X 3 + X 4 + X 5 = ) C 2 3 = Pr[X 3 = X 3 + X 4 + X 5 =, Y 4 = y 4,Y 5 = y 6 ] = 2 ( 2V i 2 ) i=4, Notation: V 3 3 = Pr[X 3 = X + X 2 + X 3 + X 6 =,Y = y,y 2 = y 2,Y 6 = y 6, X 3 + X 4 + X 5 =,Y 4 = y 4,Y 5 = y 5 Y 3 = y 3 ] C 3 = Pr[X 3 = X + X 2 + X 3 + X 6 =,Y = y,y 2 = y 2,Y 6 = y 6 ] C 2 3 = Pr[X 3 = X 3 + X 4 + X 5 =,Y 4 = y 4,Y 5 = y 5 ] V 3 3 = Pr[X 3 = h X =,Y = y,h 2 X =,Y 2 = y 2,Y 3 = y 3 ] Therefore, V 3 3 = Pr[X 3 = X + X 2 + X 3 + X 6 =,Y = y,y 2 = y 2,Y 6 = y 6, X 3 + X 4 + X 5 =,Y 4 = y 4,Y 5 = y 5 Y 3 = y 3 ] 47 48
13 Iterative message passing Pr[X 3 = h X =,Y = y,h 2 X =,Y 2 = y 2,Y 3 = y 3 ] = = Pr[X 3 =,h X =,Y = y,h 2 X =,Y 2 = y 2,Y 3 = y 3 ] Pr[h X =,Y = y,h 2 X =,Y 2 = y 2,Y 3 = y 3 ] Rightbound iteration #2: From variable 3, to check node 3. Collect all incoming messages (C 3,C 2 3 ) 2. Add my knowledge (channel output y 3 ) V 3 3 = Pr[X 3 = h X =,Y = y,h 2 X =,Y 2 = y 2,Y 3 = y 3 ] 49 5 Pr[X 3 =,h X =,Y = y,h 2 X =,Y 2 = y 2,Y 3 = y 3 ] = = Pr[X 3 =,Y 3 = y 3, X + X 2 + X 3 + X 6 =,Y = y,y 2 = y 2,Y 6 = y 6, X 3 + X 4 + X 5 =,Y 4 = y 4,Y 5 = y 5 ] = Pr[X 3 =,Y 3 = y 3, X + X X 6 =,Y = y,y 2 = y 2,Y 6 = y 6, + X 4 + X 5 =,Y 4 = y 4,Y 5 = y 5 ] =? Pr[X 3 =,h X =,Y = y,h 2 X =,Y 2 = y 2,Y 3 = y 3 ] = = Pr[X 3 =,Y 3 = y 3, X + X 2 + X 3 + X 6 =,Y = y,y 2 = y 2,Y 6 = y 6, X 3 + X 4 + X 5 =,Y 4 = y 4,Y 5 = y 5 ] = Pr[X 3 =,Y 3 = y 3, X + X X 6 =,Y = y,y 2 = y 2,Y 6 = y 6, + X 4 + X 5 =,Y 4 = y 4,Y 5 = y 5 ] = Pr[X 3 =,Y 3 = y 3 ] Pr[X + X X 6 =,Y = y,y 2 = y 2,Y 6 = y 6 ] Pr[ + X 4 + X 5 =,Y 4 = y 4,Y 5 = y 5 ] 5 52
14 Pr[X 3 =,h X =,Y = y,h 2 X =,Y 2 = y 2,Y 3 = y 3 ] = = Pr[X 3 =,Y 3 = y 3 ] Pr[X + X X 6 =,Y = y,y 2 = y 2,Y 6 = y 6 ] Pr[ + X 4 + X 5 =,Y 4 = y 4,Y 5 = y 5 ]... = Pr[X 3 = Y 3 = y 3 ] Pr[X 3 = X + X 2 + X 3 + X 6 =,Y = y,y 2 = y 2,Y 6 = y 6 ] Pr[X 3 = X 3 + X 4 + X 5 =,Y 4 = y 4,Y 5 = y 5 ] fun(y,y 2,y 6,y 4,y 5 ) Pr[X 3 =,h X =,Y = y,h 2 X =,Y 2 = y 2,Y 3 = y 3 ] = = Pr[X 3 =,Y 3 = y 3 ] Pr[X + X X 6 =,Y = y,y 2 = y 2,Y 6 = y 6 ] Pr[ + X 4 + X 5 =,Y 4 = y 4,Y 5 = y 5 ]... = P 3 C 3 C 2 3 fun(y,y 2,y 6,y 4,y 5 ) P 3 = Pr[X 3 = Y 3 = y 3 ] Pr[X 3 = h X =,Y = y,h 2 X =,Y 2 = y 2,Y 3 = y 3 ] = = Pr[X 3 =,h X =,Y = y,h 2 X =,Y 2 = y 2,Y 3 = y 3 ] Pr[h X =,Y = y,h 2 X =,Y 2 = y 2,Y 3 = y 3 ] P 3 C 3 C 2 3 fun(y,y 2,y 6,y 4,y 5 ) = Pr[h X =,Y = y,h 2 X =,Y 2 = y 2,Y 3 = y 3 ]... P 3 i=,2 = C i 3 P 3 i=,2 C i 3 + ( P 3 ) i=,2 ( C i 3) Iterative message passing Rightbound iteration #2: From variable 3, to check 3. Collect all incoming messages (C 3,C 2 3 ) 2. Add my knowledge (channel output y 3 ) P 3 i=,2 V 3 3 = C i 3 P 3 i=,2 C i 3 + ( P 3 ) i=,2 ( C i 3) 55 56
15 Let s change the problem... Information flow graph, A loop! Pr[X 3 =,h X =,Y = y,h 2 X =,Y 2 = y 2,Y 3 = y 3 ] = = Pr[X 3 =,Y 3 = y 3, X + X 2 + X 3 + X 6 =,Y = y,y 2 = y 2,Y 6 = y 6, X 3 + X 2 + X 5 =,Y 2 = y 2,Y 5 = y 5 ] = Pr[X 3 =,Y 3 = y 3, X + X X 6 =,Y = y,y 2 = y 2,Y 6 = y 6, + X 2 + X 5 =, Y 2 = y 2,Y 5 = y 5 ] Pr[X 3 =,Y 3 = y 3 ] Pr[X + X X 6 =,Y = y, Y 2 = y 2,Y 6 = y 6 ] Pr[ + X 2 + X 5 =,Y 2 = y 2,Y 5 = y 5 ] Pr[X 3 = h X =,Y = y,h 2 X =,Y 2 = y 2,Y 3 = y 3 ] P 3 i=,2 C i 3 P 3 i=,2 C i 3 + ( P 3 ) i=,2 ( C i 3) What to do? 59 6
16 Iterative message passing Rightbound iteration #2: From variable 3, to check 3. Collect all incoming messages (C 3,C 2 3 ) 2. Add my knowledge (channel output y 3 ) Ignoring loops Why is ignoring loops okay? Number of loops small. Simulation results okay even when some loops. P 3 i=,2 V 3 3 = C i 3 P 3 i=,2 C i 3 + ( P 3 ) i=,2 ( C i 3) Ignore loop: Compute V 3 3 as if no loop! 6 62 Low-density parity checks: d << N, Graph randomly generated. d = average check degree N = block length Belief Propagation Algorithm Rightbound iteration #t: At variable node n, P n i A(n)\{m} V n m = C i n P n i A(n)\{m} C i n + ( P n ) i A(n)\{m} ( C i n) Leftbound iteration #t: At check node m, C m n = 2 ( 2V i m ) i A(m)\{n} where A(n), A(m) are the sets of adjacent nodes to n and m
ECEN 655: Advanced Channel Coding
ECEN 655: Advanced Channel Coding Course Introduction Henry D. Pfister Department of Electrical and Computer Engineering Texas A&M University ECEN 655: Advanced Channel Coding 1 / 19 Outline 1 History
More informationLow-density parity-check codes
Low-density parity-check codes From principles to practice Dr. Steve Weller steven.weller@newcastle.edu.au School of Electrical Engineering and Computer Science The University of Newcastle, Callaghan,
More informationIntroduction to Low-Density Parity Check Codes. Brian Kurkoski
Introduction to Low-Density Parity Check Codes Brian Kurkoski kurkoski@ice.uec.ac.jp Outline: Low Density Parity Check Codes Review block codes History Low Density Parity Check Codes Gallager s LDPC code
More information5. Density evolution. Density evolution 5-1
5. Density evolution Density evolution 5-1 Probabilistic analysis of message passing algorithms variable nodes factor nodes x1 a x i x2 a(x i ; x j ; x k ) x3 b x4 consider factor graph model G = (V ;
More informationLDPC Codes. Slides originally from I. Land p.1
Slides originally from I. Land p.1 LDPC Codes Definition of LDPC Codes Factor Graphs to use in decoding Decoding for binary erasure channels EXIT charts Soft-Output Decoding Turbo principle applied to
More informationBounds on the Performance of Belief Propagation Decoding
Bounds on the Performance of Belief Propagation Decoding David Burshtein and Gadi Miller Dept. of Electrical Engineering Systems Tel-Aviv University Tel-Aviv 69978, Israel Email: burstyn@eng.tau.ac.il,
More informationEE229B - Final Project. Capacity-Approaching Low-Density Parity-Check Codes
EE229B - Final Project Capacity-Approaching Low-Density Parity-Check Codes Pierre Garrigues EECS department, UC Berkeley garrigue@eecs.berkeley.edu May 13, 2005 Abstract The class of low-density parity-check
More informationLow Density Parity Check (LDPC) Codes and the Need for Stronger ECC. August 2011 Ravi Motwani, Zion Kwok, Scott Nelson
Low Density Parity Check (LDPC) Codes and the Need for Stronger ECC August 2011 Ravi Motwani, Zion Kwok, Scott Nelson Agenda NAND ECC History Soft Information What is soft information How do we obtain
More informationLecture 4 : Introduction to Low-density Parity-check Codes
Lecture 4 : Introduction to Low-density Parity-check Codes LDPC codes are a class of linear block codes with implementable decoders, which provide near-capacity performance. History: 1. LDPC codes were
More informationCodes on graphs and iterative decoding
Codes on graphs and iterative decoding Bane Vasić Error Correction Coding Laboratory University of Arizona Prelude Information transmission 0 0 0 0 0 0 Channel Information transmission signal 0 0 threshold
More informationOn the Application of LDPC Codes to Arbitrary Discrete-Memoryless Channels
On the Application of LDPC Codes to Arbitrary Discrete-Memoryless Channels Amir Bennatan and David Burshtein Dept. of Electrical Engineering Systems Tel Aviv University Tel Aviv 69978, Israel Email: abn@eng.tau.ac.il,
More informationGraph-based Codes and Iterative Decoding
Graph-based Codes and Iterative Decoding Thesis by Aamod Khandekar In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy California Institute of Technology Pasadena, California
More informationMessage Passing Algorithm and Linear Programming Decoding for LDPC and Linear Block Codes
Message Passing Algorithm and Linear Programming Decoding for LDPC and Linear Block Codes Institute of Electronic Systems Signal and Information Processing in Communications Nana Traore Shashi Kant Tobias
More informationAnalysis of Sum-Product Decoding of Low-Density Parity-Check Codes Using a Gaussian Approximation
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 47, NO. 2, FEBRUARY 2001 657 Analysis of Sum-Product Decoding of Low-Density Parity-Check Codes Using a Gaussian Approximation Sae-Young Chung, Member, IEEE,
More informationGraph-based codes for flash memory
1/28 Graph-based codes for flash memory Discrete Mathematics Seminar September 3, 2013 Katie Haymaker Joint work with Professor Christine Kelley University of Nebraska-Lincoln 2/28 Outline 1 Background
More informationAn Introduction to Low Density Parity Check (LDPC) Codes
An Introduction to Low Density Parity Check (LDPC) Codes Jian Sun jian@csee.wvu.edu Wireless Communication Research Laboratory Lane Dept. of Comp. Sci. and Elec. Engr. West Virginia University June 3,
More informationDecomposition Methods for Large Scale LP Decoding
Decomposition Methods for Large Scale LP Decoding Siddharth Barman Joint work with Xishuo Liu, Stark Draper, and Ben Recht Outline Background and Problem Setup LP Decoding Formulation Optimization Framework
More informationIntroducing Low-Density Parity-Check Codes
Introducing Low-Density Parity-Check Codes Sarah J. Johnson School of Electrical Engineering and Computer Science The University of Newcastle Australia email: sarah.johnson@newcastle.edu.au Topic 1: Low-Density
More informationLOW-density parity-check (LDPC) codes were invented
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 54, NO 1, JANUARY 2008 51 Extremal Problems of Information Combining Yibo Jiang, Alexei Ashikhmin, Member, IEEE, Ralf Koetter, Senior Member, IEEE, and Andrew
More informationAn Introduction to Low-Density Parity-Check Codes
An Introduction to Low-Density Parity-Check Codes Paul H. Siegel Electrical and Computer Engineering University of California, San Diego 5/ 3/ 7 Copyright 27 by Paul H. Siegel Outline Shannon s Channel
More informationCodes on graphs and iterative decoding
Codes on graphs and iterative decoding Bane Vasić Error Correction Coding Laboratory University of Arizona Funded by: National Science Foundation (NSF) Seagate Technology Defense Advanced Research Projects
More informationLow-Density Parity-Check Codes
Department of Computer Sciences Applied Algorithms Lab. July 24, 2011 Outline 1 Introduction 2 Algorithms for LDPC 3 Properties 4 Iterative Learning in Crowds 5 Algorithm 6 Results 7 Conclusion PART I
More informationTurbo Codes are Low Density Parity Check Codes
Turbo Codes are Low Density Parity Check Codes David J. C. MacKay July 5, 00 Draft 0., not for distribution! (First draft written July 5, 998) Abstract Turbo codes and Gallager codes (also known as low
More informationConnecting Belief Propagation with Maximum Likelihood Detection
Connecting Belief Propagation with Maximum Likelihood Detection John MacLaren Walsh 1, Phillip Allan Regalia 2 1 School of Electrical Computer Engineering, Cornell University, Ithaca, NY. jmw56@cornell.edu
More informationOn Generalized EXIT Charts of LDPC Code Ensembles over Binary-Input Output-Symmetric Memoryless Channels
2012 IEEE International Symposium on Information Theory Proceedings On Generalied EXIT Charts of LDPC Code Ensembles over Binary-Input Output-Symmetric Memoryless Channels H Mamani 1, H Saeedi 1, A Eslami
More informationAn Introduction to Algorithmic Coding Theory
An Introduction to Algorithmic Coding Theory M. Amin Shokrollahi Bell Laboratories Part : Codes - A puzzle What do the following problems have in common? 2 Problem : Information Transmission MESSAGE G
More informationLower Bounds on the Graphical Complexity of Finite-Length LDPC Codes
Lower Bounds on the Graphical Complexity of Finite-Length LDPC Codes Igal Sason Department of Electrical Engineering Technion - Israel Institute of Technology Haifa 32000, Israel 2009 IEEE International
More informationThe Distribution of Cycle Lengths in Graphical Models for Turbo Decoding
The Distribution of Cycle Lengths in Graphical Models for Turbo Decoding Technical Report UCI-ICS 99-10 Department of Information and Computer Science University of California, Irvine Xianping Ge, David
More informationDesign and Analysis of Nonbinary LDPC Codes for Arbitrary Discrete-Memoryless Channels
Design and Analysis of Nonbinary LDPC Codes for Arbitrary Discrete-Memoryless Channels Amir Bennatan, Student Member, IEEE, David Burshtein, Senior Member, IEEE Abstract arxiv:cs/05040v [cs.it] 9 Nov 2005
More informationCOMPSCI 650 Applied Information Theory Apr 5, Lecture 18. Instructor: Arya Mazumdar Scribe: Hamed Zamani, Hadi Zolfaghari, Fatemeh Rezaei
COMPSCI 650 Applied Information Theory Apr 5, 2016 Lecture 18 Instructor: Arya Mazumdar Scribe: Hamed Zamani, Hadi Zolfaghari, Fatemeh Rezaei 1 Correcting Errors in Linear Codes Suppose someone is to send
More informationSpatially Coupled LDPC Codes
Spatially Coupled LDPC Codes Kenta Kasai Tokyo Institute of Technology 30 Aug, 2013 We already have very good codes. Efficiently-decodable asymptotically capacity-approaching codes Irregular LDPC Codes
More informationLDPC Codes. Intracom Telecom, Peania
LDPC Codes Alexios Balatsoukas-Stimming and Athanasios P. Liavas Technical University of Crete Dept. of Electronic and Computer Engineering Telecommunications Laboratory December 16, 2011 Intracom Telecom,
More informationMa/CS 6b Class 25: Error Correcting Codes 2
Ma/CS 6b Class 25: Error Correcting Codes 2 By Adam Sheffer Recall: Codes V n the set of binary sequences of length n. For example, V 3 = 000,001,010,011,100,101,110,111. Codes of length n are subsets
More informationBifurcations in iterative decoding and root locus plots
Published in IET Control Theory and Applications Received on 12th March 2008 Revised on 26th August 2008 ISSN 1751-8644 Bifurcations in iterative decoding and root locus plots C.M. Kellett S.R. Weller
More informationDecoding Codes on Graphs
Decoding Codes on Graphs 2. Probabilistic Decoding A S Madhu and Aditya Nori 1.Int roduct ion A S Madhu Aditya Nori A S Madhu and Aditya Nori are graduate students with the Department of Computer Science
More informationBOUNDS ON THE MAP THRESHOLD OF ITERATIVE DECODING SYSTEMS WITH ERASURE NOISE. A Thesis CHIA-WEN WANG
BOUNDS ON THE MAP THRESHOLD OF ITERATIVE DECODING SYSTEMS WITH ERASURE NOISE A Thesis by CHIA-WEN WANG Submitted to the Office of Graduate Studies of Texas A&M University in partial fulfillment of the
More informationCHAPTER 3 LOW DENSITY PARITY CHECK CODES
62 CHAPTER 3 LOW DENSITY PARITY CHECK CODES 3. INTRODUCTION LDPC codes were first presented by Gallager in 962 [] and in 996, MacKay and Neal re-discovered LDPC codes.they proved that these codes approach
More informationLecture 7. Union bound for reducing M-ary to binary hypothesis testing
Lecture 7 Agenda for the lecture M-ary hypothesis testing and the MAP rule Union bound for reducing M-ary to binary hypothesis testing Introduction of the channel coding problem 7.1 M-ary hypothesis testing
More informationLinear Block Codes. Saravanan Vijayakumaran Department of Electrical Engineering Indian Institute of Technology Bombay
1 / 26 Linear Block Codes Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay July 28, 2014 Binary Block Codes 3 / 26 Let F 2 be the set
More informationPerformance Analysis and Code Optimization of Low Density Parity-Check Codes on Rayleigh Fading Channels
Performance Analysis and Code Optimization of Low Density Parity-Check Codes on Rayleigh Fading Channels Jilei Hou, Paul H. Siegel and Laurence B. Milstein Department of Electrical and Computer Engineering
More informationMaking Error Correcting Codes Work for Flash Memory
Making Error Correcting Codes Work for Flash Memory Part I: Primer on ECC, basics of BCH and LDPC codes Lara Dolecek Laboratory for Robust Information Systems (LORIS) Center on Development of Emerging
More informationIterative Decoding for Wireless Networks
Iterative Decoding for Wireless Networks Thesis by Ravi Palanki In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy California Institute of Technology Pasadena, California
More informationAn algorithm to improve the error rate performance of Accumulate-Repeat-Accumulate codes Tae-Ui Kim
An algorithm to improve the error rate performance of Accumulate-Repeat-Accumulate codes Tae-Ui Kim The Graduate School Yonsei University Department of Electrical and Electronic Engineering An algorithm
More informationA Universal Theory of Pseudocodewords
A Universal Theory of Pseudocodewords Nathan Axvig, Emily Price, Eric Psota, Deanna Turk, Lance C. Pérez, and Judy L. Walker Abstract Three types of pseudocodewords for LDPC codes are found in the literature:
More informationMessage Passing Algorithm with MAP Decoding on Zigzag Cycles for Non-binary LDPC Codes
Message Passing Algorithm with MAP Decoding on Zigzag Cycles for Non-binary LDPC Codes Takayuki Nozaki 1, Kenta Kasai 2, Kohichi Sakaniwa 2 1 Kanagawa University 2 Tokyo Institute of Technology July 12th,
More informationCapacity-approaching codes
Chapter 13 Capacity-approaching codes We have previously discussed codes on graphs and the sum-product decoding algorithm in general terms. In this chapter we will give a brief overview of some particular
More informationFactor Graphs and Message Passing Algorithms Part 1: Introduction
Factor Graphs and Message Passing Algorithms Part 1: Introduction Hans-Andrea Loeliger December 2007 1 The Two Basic Problems 1. Marginalization: Compute f k (x k ) f(x 1,..., x n ) x 1,..., x n except
More informationLow-Density Parity-Check codes An introduction
Low-Density Parity-Check codes An introduction c Tilo Strutz, 2010-2014,2016 June 9, 2016 Abstract Low-density parity-check codes (LDPC codes) are efficient channel coding codes that allow transmission
More informationABSTRACT. The original low-density parity-check (LDPC) codes were developed by Robert
ABSTRACT Title of Thesis: OPTIMIZATION OF PERMUTATION KEY FOR π-rotation LDPC CODES Nasim Vakili Pourtaklo, Master of Science, 2006 Dissertation directed by: Associate Professor Steven Tretter Department
More informationAPPLICATIONS. Quantum Communications
SOFT PROCESSING TECHNIQUES FOR QUANTUM KEY DISTRIBUTION APPLICATIONS Marina Mondin January 27, 2012 Quantum Communications In the past decades, the key to improving computer performance has been the reduction
More information4216 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 51, NO. 12, DECEMBER Density Evolution for Asymmetric Memoryless Channels
4216 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 51, NO. 12, DECEMBER 2005 Density Evolution for Asymmetric Memoryless Channels Chih-Chun Wang, Sanjeev R. Kulkarni, Fellow, IEEE, and H. Vincent Poor,
More informationBounds on Achievable Rates of LDPC Codes Used Over the Binary Erasure Channel
Bounds on Achievable Rates of LDPC Codes Used Over the Binary Erasure Channel Ohad Barak, David Burshtein and Meir Feder School of Electrical Engineering Tel-Aviv University Tel-Aviv 69978, Israel Abstract
More informationPerformance of Low Density Parity Check Codes. as a Function of Actual and Assumed Noise Levels. David J.C. MacKay & Christopher P.
Performance of Low Density Parity Check Codes as a Function of Actual and Assumed Noise Levels David J.C. MacKay & Christopher P. Hesketh Cavendish Laboratory, Cambridge, CB3 HE, United Kingdom. mackay@mrao.cam.ac.uk
More informationInformation Theoretic Imaging
Information Theoretic Imaging WU Faculty: J. A. O Sullivan WU Doctoral Student: Naveen Singla Boeing Engineer: James Meany First Year Focus: Imaging for Data Storage Image Reconstruction Data Retrieval
More informationCodes on Graphs. Telecommunications Laboratory. Alex Balatsoukas-Stimming. Technical University of Crete. November 27th, 2008
Codes on Graphs Telecommunications Laboratory Alex Balatsoukas-Stimming Technical University of Crete November 27th, 2008 Telecommunications Laboratory (TUC) Codes on Graphs November 27th, 2008 1 / 31
More informationDigital Communications III (ECE 154C) Introduction to Coding and Information Theory
Digital Communications III (ECE 154C) Introduction to Coding and Information Theory Tara Javidi These lecture notes were originally developed by late Prof. J. K. Wolf. UC San Diego Spring 2014 1 / 8 I
More informationTurbo Compression. Andrej Rikovsky, Advisor: Pavol Hanus
Turbo Compression Andrej Rikovsky, Advisor: Pavol Hanus Abstract Turbo codes which performs very close to channel capacity in channel coding can be also used to obtain very efficient source coding schemes.
More informationShannon s noisy-channel theorem
Shannon s noisy-channel theorem Information theory Amon Elders Korteweg de Vries Institute for Mathematics University of Amsterdam. Tuesday, 26th of Januari Amon Elders (Korteweg de Vries Institute for
More information2 Information transmission is typically corrupted by noise during transmission. Various strategies have been adopted for reducing or eliminating the n
Finite size eects and error-free communication in Gaussian channels Ido Kanter and David Saad # Minerva Center and Department of Physics, Bar-Ilan University, Ramat-Gan 52900, Israel. # The Neural Computing
More informationConstruction and Performance Evaluation of QC-LDPC Codes over Finite Fields
MEE10:83 Construction and Performance Evaluation of QC-LDPC Codes over Finite Fields Ihsan Ullah Sohail Noor This thesis is presented as part of the Degree of Master of Sciences in Electrical Engineering
More informationAnalysis of a Randomized Local Search Algorithm for LDPCC Decoding Problem
Analysis of a Randomized Local Search Algorithm for LDPCC Decoding Problem Osamu Watanabe, Takeshi Sawai, and Hayato Takahashi Dept. of Mathematical and Computing Sciences, Tokyo Institute of Technology
More informationLow-density parity-check (LDPC) codes
Low-density parity-check (LDPC) codes Performance similar to turbo codes Do not require long interleaver to achieve good performance Better block error performance Error floor occurs at lower BER Decoding
More informationAalborg Universitet. Bounds on information combining for parity-check equations Land, Ingmar Rüdiger; Hoeher, A.; Huber, Johannes
Aalborg Universitet Bounds on information combining for parity-check equations Land, Ingmar Rüdiger; Hoeher, A.; Huber, Johannes Published in: 2004 International Seminar on Communications DOI link to publication
More informationOne-Bit LDPC Message Passing Decoding Based on Maximization of Mutual Information
One-Bit LDPC Message Passing Decoding Based on Maximization of Mutual Information ZOU Sheng and Brian M. Kurkoski kurkoski@ice.uec.ac.jp University of Electro-Communications Tokyo, Japan University of
More informationIntroduction to Convolutional Codes, Part 1
Introduction to Convolutional Codes, Part 1 Frans M.J. Willems, Eindhoven University of Technology September 29, 2009 Elias, Father of Coding Theory Textbook Encoder Encoder Properties Systematic Codes
More informationQuasi-cyclic Low Density Parity Check codes with high girth
Quasi-cyclic Low Density Parity Check codes with high girth, a work with Marta Rossi, Richard Bresnan, Massimilliano Sala Summer Doctoral School 2009 Groebner bases, Geometric codes and Order Domains Dept
More informationSingle-Gaussian Messages and Noise Thresholds for Low-Density Lattice Codes
Single-Gaussian Messages and Noise Thresholds for Low-Density Lattice Codes Brian M. Kurkoski, Kazuhiko Yamaguchi and Kingo Kobayashi kurkoski@ice.uec.ac.jp Dept. of Information and Communications Engineering
More informationLecture 8: Shannon s Noise Models
Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 2007) Lecture 8: Shannon s Noise Models September 14, 2007 Lecturer: Atri Rudra Scribe: Sandipan Kundu& Atri Rudra Till now we have
More informationDesign and Analysis of Graph-based Codes Using Algebraic Lifts and Decoding Networks
University of Nebraska - Lincoln DigitalCommons@University of Nebraska - Lincoln Dissertations, Theses, and Student Research Papers in Mathematics Mathematics, Department of 3-2018 Design and Analysis
More informationMaximum Likelihood Decoding of Codes on the Asymmetric Z-channel
Maximum Likelihood Decoding of Codes on the Asymmetric Z-channel Pål Ellingsen paale@ii.uib.no Susanna Spinsante s.spinsante@univpm.it Angela Barbero angbar@wmatem.eis.uva.es May 31, 2005 Øyvind Ytrehus
More informationIterative Plurality-Logic and Generalized Algorithm B Decoding of q-ary LDPC Codes
1 Iterative Plurality-Logic and Generalized Algorithm B Decoding of q-ary LDPC Codes Kristin Jagiello and William E. Ryan kristin@angeldojo.com and ryan@ece.arizona.edu ECE Dept., University of Arizona,
More informationDensity Evolution and Functional Threshold. for the Noisy Min-Sum Decoder
Density Evolution and Functional Threshold 1 for the Noisy Min-Sum Decoder C. Kameni Ngassa,#, V. Savin, E. Dupraz #, D. Declercq # CEA-LETI, Minatec Campus, Grenoble, France, {christiane.kameningassa,
More informationIterative Encoding of Low-Density Parity-Check Codes
Iterative Encoding of Low-Density Parity-Check Codes David Haley, Alex Grant and John Buetefuer Institute for Telecommunications Research University of South Australia Mawson Lakes Blvd Mawson Lakes SA
More informationLDPC Decoder LLR Stopping Criterion
International Conference on Innovative Trends in Electronics Communication and Applications 1 International Conference on Innovative Trends in Electronics Communication and Applications 2015 [ICIECA 2015]
More informationMessage-Passing Decoding for Low-Density Parity-Check Codes Harish Jethanandani and R. Aravind, IIT Madras
Message-Passing Decoding for Low-Density Parity-Check Codes Harish Jethanandani and R. Aravind, IIT Madras e-mail: hari_jethanandani@yahoo.com Abstract Low-density parity-check (LDPC) codes are discussed
More informationInformation Projection Algorithms and Belief Propagation
χ 1 π(χ 1 ) Information Projection Algorithms and Belief Propagation Phil Regalia Department of Electrical Engineering and Computer Science Catholic University of America Washington, DC 20064 with J. M.
More informationConstruction of low complexity Array based Quasi Cyclic Low density parity check (QC-LDPC) codes with low error floor
Construction of low complexity Array based Quasi Cyclic Low density parity check (QC-LDPC) codes with low error floor Pravin Salunkhe, Prof D.P Rathod Department of Electrical Engineering, Veermata Jijabai
More information9 Forward-backward algorithm, sum-product on factor graphs
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.438 Algorithms For Inference Fall 2014 9 Forward-backward algorithm, sum-product on factor graphs The previous
More informationPractical Polar Code Construction Using Generalised Generator Matrices
Practical Polar Code Construction Using Generalised Generator Matrices Berksan Serbetci and Ali E. Pusane Department of Electrical and Electronics Engineering Bogazici University Istanbul, Turkey E-mail:
More informationDecoding of LDPC codes with binary vector messages and scalable complexity
Downloaded from vbn.aau.dk on: marts 7, 019 Aalborg Universitet Decoding of LDPC codes with binary vector messages and scalable complexity Lechner, Gottfried; Land, Ingmar; Rasmussen, Lars Published in:
More information1e
B. J. Frey and D. J. C. MacKay (1998) In Proceedings of the 35 th Allerton Conference on Communication, Control and Computing 1997, Champaign-Urbana, Illinois. Trellis-Constrained Codes Brendan J. Frey
More informationUTA EE5362 PhD Diagnosis Exam (Spring 2011)
EE5362 Spring 2 PhD Diagnosis Exam ID: UTA EE5362 PhD Diagnosis Exam (Spring 2) Instructions: Verify that your exam contains pages (including the cover shee. Some space is provided for you to show your
More informationRecent results on bit-flipping LDPC decoders
Recent results on bit-flipping LDPC decoders Chris Winstead 1,2, Gopalakrishnan Sundararajan 1, Emmanuel Boutillon 2 1 Department of Electrical and Computer Engineering LE/FT Lab Utah State University
More informationSlepian-Wolf Code Design via Source-Channel Correspondence
Slepian-Wolf Code Design via Source-Channel Correspondence Jun Chen University of Illinois at Urbana-Champaign Urbana, IL 61801, USA Email: junchen@ifpuiucedu Dake He IBM T J Watson Research Center Yorktown
More informationEFFICIENT DECODING ALGORITHMS FOR LOW DENSITY PARITY CHECK CODES
EFFICIENT DECODING ALGORITHMS FOR LOW DENSITY PARITY CHECK CODES Master s thesis in electronics systems by Anton Blad LiTH-ISY-EX--05/3691--SE Linköping 2005 EFFICIENT DECODING ALGORITHMS FOR LOW DENSITY
More informationAN INTRODUCTION TO LOW-DENSITY PARITY-CHECK CODES
AN INTRODUCTION TO LOW-DENSITY PARITY-CHECK CODES Item Type text; Proceedings Authors Moon, Todd K.; Gunther, Jacob H. Publisher International Foundation for Telemetering Journal International Telemetering
More informationIterative Quantization. Using Codes On Graphs
Iterative Quantization Using Codes On Graphs Emin Martinian and Jonathan S. Yedidia 2 Massachusetts Institute of Technology 2 Mitsubishi Electric Research Labs Lossy Data Compression: Encoding: Map source
More informationIn Praise of Bad Codes for Multi-Terminal Communications
SUBMITTED TO THE IEEE TRANSACTIONS ON INFORMATION THEORY In Praise of Bad Codes for Multi-Terminal Communications Amir Bennatan, Shlomo Shamai (Shitz) and A. Robert Calderbank arxiv:008.766v cs.it] 0 Aug
More informationLP Decoding Corrects a Constant Fraction of Errors
LP Decoding Corrects a Constant Fraction of Errors Jon Feldman Columbia University (CORC Technical Report TR-2003-08) Cliff Stein Columbia University Tal Malkin Columbia University Rocco A. Servedio Columbia
More informationAn Efficient Maximum Likelihood Decoding of LDPC Codes Over the Binary Erasure Channel
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 5, NO. 11, NOVEMBER 24 1 An Efficient Maximum Likelihood Decoding of LDPC Codes Over the Binary Erasure Channel David Burshtein and Gadi Miller School of Electrical
More informationHow to Achieve the Capacity of Asymmetric Channels
How to Achieve the Capacity of Asymmetric Channels Marco Mondelli, S. Hamed Hassani, and Rüdiger Urbanke Abstract arxiv:406.7373v5 [cs.it] 3 Jan 208 We survey coding techniques that enable reliable transmission
More informationLecture 3: Channel Capacity
Lecture 3: Channel Capacity 1 Definitions Channel capacity is a measure of maximum information per channel usage one can get through a channel. This one of the fundamental concepts in information theory.
More informationAsynchronous Decoding of LDPC Codes over BEC
Decoding of LDPC Codes over BEC Saeid Haghighatshoar, Amin Karbasi, Amir Hesam Salavati Department of Telecommunication Systems, Technische Universität Berlin, Germany, School of Engineering and Applied
More informationStatistical mechanics and capacity-approaching error-correctingcodes
Physica A 302 (2001) 14 21 www.elsevier.com/locate/physa Statistical mechanics and capacity-approaching error-correctingcodes Nicolas Sourlas Laboratoire de Physique Theorique de l, UMR 8549, Unite Mixte
More informationJoint Iterative Decoding of LDPC Codes and Channels with Memory
Joint Iterative Decoding of LDPC Codes and Channels with Memory Henry D. Pfister and Paul H. Siegel University of California, San Diego 3 rd Int l Symposium on Turbo Codes September 1, 2003 Outline Channels
More information16.36 Communication Systems Engineering
MIT OpenCourseWare http://ocw.mit.edu 16.36 Communication Systems Engineering Spring 2009 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 16.36: Communication
More informationOn Bit Error Rate Performance of Polar Codes in Finite Regime
On Bit Error Rate Performance of Polar Codes in Finite Regime A. Eslami and H. Pishro-Nik Abstract Polar codes have been recently proposed as the first low complexity class of codes that can provably achieve
More informationConvergence analysis for a class of LDPC convolutional codes on the erasure channel
Convergence analysis for a class of LDPC convolutional codes on the erasure channel Sridharan, Arvind; Lentmaier, Michael; Costello Jr., Daniel J.; Zigangirov, Kamil Published in: [Host publication title
More informationBelief Propagation, Information Projections, and Dykstra s Algorithm
Belief Propagation, Information Projections, and Dykstra s Algorithm John MacLaren Walsh, PhD Department of Electrical and Computer Engineering Drexel University Philadelphia, PA jwalsh@ece.drexel.edu
More informationTwo-Bit Message Passing Decoders for LDPC. Codes Over the Binary Symmetric Channel
Two-Bit Message Passing Decoders for LDPC 1 Codes Over the Binary Symmetric Channel Lucile Sassatelli, Member, IEEE, Shashi Kiran Chilappagari, Member, IEEE, Bane Vasic, arxiv:0901.2090v3 [cs.it] 7 Mar
More information