An Introduction to Low-Density Parity-Check Codes
|
|
- Jasmin Craig
- 6 years ago
- Views:
Transcription
1 An Introduction to Low-Density Parity-Check Codes Paul H. Siegel Electrical and Computer Engineering University of California, San Diego 5/ 3/ 7 Copyright 27 by Paul H. Siegel
2 Outline Shannon s Channel Coding Theorem Error-Correcting Codes State-of-the-Art LDPC Code Basics Encoding Decoding LDPC Code Design Asymptotic performance analysis Design optimization 5/ 3/ 7 LDPC Codes 2
3 Outline EXIT Chart Analysis Applications Binary Erasure Channel Binary Symmetric Channel AWGN Channel Rayleigh Fading Channel Partial-Response Channel Basic References 5/ 3/ 7 LDPC Codes 3
4 A Noisy Communication System INFORMATION SOURCE TRANSMITTER RECEIVER DESTINATION CHANNEL SIGNAL RECEIVED SIGNAL MESSAGE MESSAGE NOISE SOURCE 5/ 3/ 7 LDPC Codes 4
5 Channels Binary erasure channel BEC(ε) - ε ε? ε - ε Binary symmetric channel BSC(p) -p p p -p 5/ 3/ 7 LDPC Codes 5
6 More Channels Additive white Gaussian noise channel AWGN f ( y ) f ( y ) P P 5/ 3/ 7 LDPC Codes 6
7 Shannon Capacity Every communication channel is characterized by a single number C, called the channel capacity. It is possible to transmit information over this channel reliably (with probability of error ) if and only if: R def # information bits = < C channel use 5/ 3/ 7 LDPC Codes 7
8 Channels and Capacities Binary erasure channel BEC(ε) - ε ε? ε - ε C = ε Binary symmetric channel BSC(p) C = H 2 ( p) -p p p -p / 3/ 7 LDPC Codes H 2 ( p) = p log 2 p ( p)log2 ( p)
9 More Channels and Capacities Additive white Gaussian noise channel AWGN f (y ) f (y ) C P + σ = log P P / 3/ 7 LDPC Codes 9
10 Coding We use a code to communicate over the noisy channel. Source x = x, x, 2, x k Encoder c = c, c, 2, c n Channel Sink xˆ = xˆ, xˆ 2,, ˆ x k Decoder y = y, y, 2, y n Code rate: R = 5/ 3/ 7 LDPC Codes k n
11 Shannon s Coding Theorems If C is a code with rate R>C, then the probability of error in decoding this code is bounded away from. (In other words, at any rate R>C, reliable communication is not possible.) C = Max (H(x) H y (x)) For any information rate R < C and any δ >, there exists a code C of length n δ and rate R, such that the probability of error in maximum likelihood decoding of this code is at most δ. Proof: Non-constructive! 5/ 3/ 7 LDPC Codes
12 Review of Shannon s Paper A pioneering paper: Shannon, C. E. A mathematical theory of communication. Bell System Tech. J. 27, (948) , A regrettable review: Doob, J.L., Mathematical Reviews, MR26286 (,33e) The discussion is suggestive throughout, rather than mathematical, and it is not always clear that the author s mathematical intentions are honorable. Cover, T. Shannon s Contributions to Shannon Theory, AMS Notices, vol. 49, no., p., January 22 Doob has recanted this remark many times, saying that it and his naming of super martingales (processes that go down instead of up) are his two big regrets. 5/ 3/ 7 LDPC Codes 2
13 Finding Good Codes Ingredients of Shannon s proof: Random code Large block length Optimal decoding Problem Randomness + large block length + optimal decoding = COMPLEXITY! 5/ 3/ 7 LDPC Codes 3
14 Solution State-of-the-Art Long, structured, pseudorandom codes Practical, near-optimal decoding algorithms Examples Turbo codes (993) Low-density parity-check (LDPC) codes (96, 999) State-of-the-art Turbo codes and LDPC codes have brought Shannon limits to within reach on a wide range of channels. 5/ 3/ 7 LDPC Codes 4
15 Evolution of Coding Technology LDPC codes from Trellis and Turbo Coding, Schlegel and Perez, IEEE Press, 24 5/ 3/ 7 LDPC Codes 5
16 Linear Block Codes - Basics Parameters of binary linear block code C k = number of information bits n = number of code bits R = k/n d min = minimum distance There are many ways to describe C Codebook (list) Parity-check matrix / generator matrix Graphical representation ( Tanner graph ) 5/ 3/ 7 LDPC Codes 6
17 Example: (7,4) Hamming Code (n,k) = (7,4),, R = 4/7 d min min = 3 single error error correcting double erasure correcting 5 Encoding rule:.. Insert data bits in in,, 2, 2, 3, 3, Insert parity bits in in 5, 5, 6, 6, 7 to to ensure an an even number of of s s in in each circle / 3/ 7 LDPC Codes 7
18 Example: (7,4) Hamming Code 2 k =6 codewords Systematic encoder places input bits in positions, 2, 3, 4 Parity bits are in positions 5, 6, 7 5/ 3/ 7 LDPC Codes 8
19 Hamming Code Parity Checks / 3/ 7 LDPC Codes 9
20 Hamming Code: Matrix Perspective 5/ 3/ 7 2 LDPC Codes = T H c [ ] ,,,,,, c c c c c c c c = Parity check matrix H = H Generator matrix G [ ] [ ] c G u c c c c c c c c u u u u u = = = ,,,,,,.,, = G
21 Parity-Check Equations 5/ 3/ 7 2 LDPC Codes Parity-check matrix implies system of linear equations = = = c c c c c c c c c c c c = H Parity-check matrix is not unique. Any set of vectors that span the rowspace generated by H can serve as the rows of a parity check matrix (including sets with more than 3 vectors).
22 Hamming Code: Tanner Graph Bi-partite graph representing parity-check equations c c 2 c 3 c 4 c 5 c 6 c 7 c c c + c2 + c3 + c5 = + c3 + c4 + c6 = + c2 + c4 + c7 = 5/ 3/ 7 LDPC Codes 22
23 Tanner Graph Terminology variable nodes (bit, left) check nodes (constraint, right) The degree of of a node is is the the number of of edges connected to to it. it. 5/ 3/ 7 LDPC Codes 23
24 Low-Density Parity-Check Codes Proposed by Gallager (96) Sparseness of matrix and graph descriptions Number of s in H grows linearly with block length Number of edges in Tanner graph grows linearly with block length Randomness of construction in: Placement of s in H Connectivity of variable and check nodes Iterative, message-passing decoder Simple local decoding at nodes Iterative exchange of information (message-passing) 5/ 3/ 7 LDPC Codes 24
25 Another pioneering work: Review of Gallager s Paper Gallager, R. G., Low-Density Parity-Check Codes, M.I.T. Press, Cambridge, Mass: 963. A more enlightened review: Horstein, M., IEEE Trans. Inform. Thoery, vol., no. 2, p. 72, April 964, This book is an extremely lucid and circumspect exposition of an important piece of research. A comparison with other coding and decoding procedures designed for high-reliability transmission... is difficult...furthermore, many hours of computer simulation are needed to evaluate a probabilistic decoding scheme... It appears, however, that LDPC codes have a sufficient number of desirable features to make them highly competitive with... other schemes... 5/ 3/ 7 LDPC Codes 25
26 Gallager s LDPC Codes Now called regular LDPC codes Parameters (n,j,k) n = codeword length j = # of parity-check equations involving each code bit = degree of each variable node k = # code bits involved in each parity-check equation = degree of each check node Locations of s can be chosen randomly, subject to (j,k) constraints. 5/ 3/ 7 LDPC Codes 26
27 Gallager s Construction (n,j,k) =(2,3,4) First n/k =5 rows have k=4 s each, descending. Next j-=2 submatrices of size n/k x n =5 x 2 obtained by applying randomly chosen column permutation to first submatrix. Result: jn/k x n = 5 x 2 parity check matrix for a (n,j,k) =(2,3,4) LDPC code. π π / 3/ 7 LDPC Codes 27
28 Regular LDPC Code Tanner Graph n = 2 variable nodes left degree j = 3 nj = 6 edges nj/k = 5 check right degree k = 4 nj = 6 edges 5/ 3/ 7 LDPC Codes 28
29 Properties of Regular LDPC Codes Design rate: R(j,k) = j/k Linear dependencies can increase rate Design rate achieved with high probability as n increases Example: (n,j,k)=(2,3,4) with R = 3/4 = /4. For j 3, the typical minimum distance of codes in the (j,k) ensemble grows linearly in the codeword length n. Their performance under maximum-likelihood decoding on BSC(p) is at least as good...as the optimum code of a somewhat higher rate. [Gallager, 96] 5/ 3/ 7 LDPC Codes 29
30 Performance of Regular LDPC Codes Gallager, Gallager, / 3/ 7 LDPC Codes 3
31 Performance of Regular LDPC Codes Gallager, Gallager, / 3/ 7 LDPC Codes 3
32 Performance of Regular LDPC Codes Gallager, Gallager, / 3/ 7 LDPC Codes 32
33 Performance of Regular LDPC Codes Irregular LDPC (3,6) (3,6) Richardson, Richardson, Shokrollahi, Shokrollahi, and and Urbanke, Urbanke, 2 2 n= n= 6 6 R=/2 R=/2 5/ 3/ 7 LDPC Codes 33
34 Irregular LDPC Codes Irregular LDPC codes are a natural generalization of Gallager s LDPC codes. The degrees of variable and check nodes need not be constant. Ensemble defined by node degree distribution functions. Λ d v i Λ( Λ x Ρ( ) = Ρ i i x ) = i= = number of variable nodes of degree i x i= 2 Normalize for fraction of nodes of specified degree L( x) = Λ( x) Λ() 5/ 3/ 7 LDPC Codes 34 Ρ i d c = number of nodes of R ( x) = x i i check degree P( x) P() i
35 Irregular LDPC Codes Often, we use the degree distribution from the edge perspective d v λ( λ x ) = i= x i i λi = fraction of edges connected to variable nodes of degree i d c ρ( ρ x ) = i= 2 x i i ρi = fraction of edges connected to check nodes of degree i Conversion rule λ( x) = Λ Λ ( x) () = L L ( x) () ρ( x) = P P ( x) () = R R ( x) () 5/ 3/ 7 LDPC Codes 35
36 Irregular LDPC Codes Design rate R( λ, ρ ) = ρ i i i λi i i = ρ ( λ ( x ) dx x ) dx Under certain conditions related to codewords of weight n/2, the design rate is achieved with high probability as n increases. 5/ 3/ 7 LDPC Codes 36
37 Examples of Degree Distribution Pairs Hamming (7,4) code Λ Ρ λ ρ ( x) = 3x 4 ( x) = 3x ( x) = 4 3 ( x) = x + + 3x 2 2 x x x 3 2 # edges = R( λ, ρ) = = (j,k) regular LDPC code, length-n ( x) Λ j = nx λ ( x ) Ρ jn k k ( x) = x ρ = x j k ( x) = x R( λ, ρ) / k = = / j j k 5/ 3/ 7 LDPC Codes 37
38 Encoding LDPC Codes Convert H into equivalent upper triangular form H H = n-k n-k k (e.g., by Gaussian elimination and column swapping complexity ~ O(n 3 ) ) This is a pre-processing step only. 5/ 3/ 7 LDPC Codes 38
39 Encoding LDPC Codes Set c n-k+,,c n equal to the data bits x,,x k. Solve for parities c l, l=,, n-k, in reverse order; i.e., starting with l=n-k, compute c l = n k j= l + H l, j c j k j= l + H l, j n+ k x j (complexity ~O(n 2 ) ) Another general encoding technique based upon approximate lower triangulation has complexity no more than O(n 2 ), with the constant coefficient small enough to allow practical encoding for block lengths on the order of n= 5. 5/ 3/ 7 LDPC Codes 39
40 Linear Encoding Complexity It has been shown that optimized ensembles of irregular LDPC codes can be encoded with preprocessing complexity at most O(n 3/2 ), and subsequent complexity ~O(n). It has been shown that a necessary condition for the ensemble of (λ, ρ)-irregular LDPC codes to be linear-time encodable is λ ( ) ρ () > Alternatively, LDPC code ensembles with additional structure have linear encoding complexity, such as irregular repeataccumulate (IRA) codes. 5/ 3/ 7 LDPC Codes 4
41 Decoding of LDPC Codes Gallager introduced the idea of iterative, messagepassing decoding of LDPC codes. The idea is to iteratively share the results of local node decoding by passing them along edges of the Tanner graph. We will first demonstrate this decoding method for the binary erasure channel BEC(ε). The performance and optimization of LDPC codes for the BEC will tell us a lot about other channels, too. 5/ 3/ 7 LDPC Codes 4
42 Decoding for the BEC Recall: Binary erasure channel, BEC(ε) x i -ε ε ε -ε? y i x = (x, x 2,, x n ) transmitted codeword y = (y, y 2,, y n ) received word Note: if y i {,}, then x i = y i. 5/ 3/ 7 LDPC Codes 42
43 Optimal Block Decoding - BEC Maximum a posteriori (MAP) block decoding rule minimizes block error probability: MAP xˆ ( y) = arg max P ( x y) x C Assume that codewords are transmitted equiprobably. MAP xˆ ( y) = arg max P ( y x) x C If the (non-empty) set X(y) of codewords compatible with y contains only one codeword x, then xˆ MAP ( y) = If X(y) contains more than one codeword, then declare a block erasure. x X Y Y X 5/ 3/ 7 LDPC Codes 43
44 Optimal Bit Decoding - BEC Maximum a posteriori (MAP) bit decoding rule minimizes bit error probability: MAP xˆ ( y) = arg max P ( b y) i = b b {,} arg max } {, x x = C b Assume that codewords are transmitted equiprobably. X Y If every codeword x X(y) satisfies x i =b, then set i X i Y P ( x y) xˆ MAP ( y) = b Otherwise, declare a bit erasure in position i. 5/ 3/ 7 LDPC Codes 44
45 MAP Decoding Complexity Let E {,,n} denote the positions of erasures in y, and let F denote its complement in {,,n}. Let w E and w F denote the corresponding sub-words of word w. Let H E and H F denote the corresponding submatrices of the parity check matrix H. Then X(y), the set of codewords compatible with y, satisfies { T T } X( y) = x C x = y and H x = H y F F E E F F So, optimal (MAP) decoding can be done by solving a set of linear equations, requiring complexity at most O(n 3 ). For large blocklength n, this can be prohibitive! 5/ 3/ 7 LDPC Codes 45
46 Simpler Decoding We now describe an alternative decoding procedure that can be implemented very simply. It is a local decoding technique that tries to fill in erasures one parity-check equation at a time. We will illustrate it using a very simple and familiar linear code, the (7,4) Hamming code. We ll compare its performance to that of optimal bitwise decoding. Then, we ll reformulate it as a message-passing decoding algorithm and apply it to LDPC codes. 5/ 3/ 7 LDPC Codes 46
47 Local Decoding of Erasures d min = 3, so any two erasures can be uniquely filled to get a codeword. Decoding can be done locally: Given any pattern of one or two erasures, there will always be a parity-check (circle) involving exactly one erasure. The parity-check represented by the circle can be used to fill in the erased bit This leaves at most one more erasure. Any parity-check (circle) involving it can be used to fill it in. 5/ 3/ 7 LDPC Codes 47
48 Local Decoding - Example All- s codeword transmitted. Two erasures as shown. Start with either the red parity or green parity circle. The red parity circle requires that the erased symbol inside it be.?? 5/ 3/ 7 LDPC Codes 48
49 Local Decoding -Example Next, the green parity circle or the blue parity circle can be selected. Either one requires that the remaining erased symbol be.? 5/ 3/ 7 LDPC Codes 49
50 Local Decoding -Example Estimated codeword: [ ] Decoding successful!! This procedure would have worked no matter which codeword was transmitted. 5/ 3/ 7 LDPC Codes 5
51 Initialization: Decoding with the Tanner Graph: an a-peeling Decoder Forward known variable node values along outgoing edges Accumulate forwarded values at check nodes and record the parity Delete known variable nodes and all outgoing edges 5/ 3/ 7 LDPC Codes 5
52 Peeling Decoder Initialization x??? x??? Forward known known values values 5/ 3/ 7 LDPC Codes 52
53 Peeling Decoder - Initialization x Accumulate parity x Delete known variable nodes and edges?????? 5/ 3/ 7 LDPC Codes 53
54 Decoding step: Decoding with the Tanner Graph: an a-peeling Decoder Select, if possible, a check node with one edge remaining; forward its parity, thereby determining the connected variable node Delete the check node and its outgoing edge Follow procedure in the initialization process at the known variable node Termination If remaining graph is empty, the codeword is determined If decoding step gets stuck, declare decoding failure 5/ 3/ 7 LDPC Codes 54
55 Peeling Decoder Step x Find degree- check node; forward accumulated parity; determine variable node value x Delete check node and edge; forward new variable node value???? 5/ 3/ 7 LDPC Codes 55
56 Peeling Decoder Step x?? Accumulate parity LDPC Codes Delete known variable nodes and edges 5/ 3/ 7 56 x??
57 x? Peeling Decoder Step 2 Find degree- check node; forward accumulated parity; determine variable node value LDPC Codes Delete check node and edge; forward new variable node value 5/ 3/ 7 57 x?
58 Peeling Decoder Step 2 x? Accumulate parity LDPC Codes Delete known variable nodes and edges 5/ 3/ 7 58 x?
59 x Peeling Decoder Step 3 Find degree- check node; forward accumulated parity; determine variable node value LDPC Codes Delete check node and edge; decoding complete 5/ 3/ 7 59 x
60 Message-Passing Decoding The local decoding procedure can be described in terms of an iterative, message-passing algorithm in which all variable nodes and all check nodes in parallel iteratively pass messages along their adjacent edges. The values of the code bits are updated accordingly. The algorithm continues until all erasures are filled in, or until the completion of a specified number of iterations. LDPC Codes 5/ 3/ 7 6
61 Variable-to-Check Node Message from channel???? v=? edge e uu? u? v=u edge e Variable-to-check message on edge e If all other incoming messages are?, send message v =? If any other incoming message u is or, send v=u and, if the bit was an erasure, fill it with u, too. (Note that there are no errors on the BEC, so a message that is or must be correct. Messages cannot be inconsistent.) 5/ 3/ 7 LDPC Codes 6
62 Check-to-Variable Node Message u =?? u = v + v 2 + v 3 v edge e edge e v v 2 v 2 v 3 Check-to-variable message on edge e If any other incoming message is?, send u =? If all other incoming messages are in {,}, send the XOR of them, u = v + v 2 + v 3. 5/ 3/ 7 LDPC Codes 62
63 y??? Message-Passing Example Initialization 5/ 3/ 7 LDPC Codes 63 x y?????? Variable-to-Check
64 x y?????? Message-Passing Example Round Check-to-Variable 5/ 3/ 7 LDPC Codes 64 x y????? Variable-to-Check
65 x y????? Message-Passing Example Round 2 Check-to-Variable 5/ 3/ 7 LDPC Codes 65 x y???? Variable-to-Check
66 x y???? Message-Passing Example Round 3 Check-to-Variable 5/ 3/ 7 LDPC Codes 66 x y??? Variable-to-Check Decoding complete
67 Sub-optimality of Message-Passing Decoder Hamming code: decoding of 3 erasures There are 7 patterns of 3 erasures that correspond to the support of a weight-3 codeword. These can not be decoded by any decoder! The other 28 patterns of 3 erasures can be uniquely filled in by the optimal decoder. We just saw a pattern of 3 erasures that was corrected by the local decoder. Are there any that it cannot? Test:?????? 5/ 3/ 7 LDPC Codes 67
68 Sub-optimality of Message-Passing Decoder Test:??? There is a unique way to fill the erasures and get a codeword: The optimal decoder would find it. But every parity-check has at least 2 erasures, so local decoding will not work! 5/ 3/ 7 LDPC Codes 68
69 Stopping Sets A stopping set is a subset S of the variable nodes such that every check node connected to S is connected to S at least twice. The empty set is a stopping set (trivially). The support set (i.e., the positions of s) of any codeword is a stopping set (parity condition). A stopping set need not be the support of a codeword. 5/ 3/ 7 LDPC Codes 69
70 Stopping Sets Example : (7,4) Hamming code Codeword support set S={4,6,7} / 3/ 7 LDPC Codes 7
71 Stopping Sets Example 2: (7,4) Hamming code / 3/ 7 LDPC Codes 7
72 Stopping Sets Example 2: (7,4) Hamming code Not the support set of a codeword S={,2,3} / 3/ 7 LDPC Codes 72
73 Stopping Set Properties Every set of variable nodes contains a largest stopping set (since the union of stopping sets is also a stopping set). The message-passing decoder needs a check node with at most one edge connected to an erasure to proceed. So, if the remaining erasures form a stopping set, the decoder must stop. Let E be the initial set of erasures. When the messagepassing decoder stops, the remaining set of erasures is the largest stopping set S in E. If S is empty, the codeword has been recovered. If not, the decoder has failed. 5/ 3/ 7 LDPC Codes 73
74 Suboptimality of Message-Passing Decoder An optimal (MAP) decoder for a code C on the BEC fails if and only if the set of erased variables includes the support set of a codeword. The message-passing decoder fails if and only the set of erased variables includes a non-empty stopping set. Conclusion: Message-passing may fail where optimal decoding succeeds!! Message-passing is suboptimal!! 5/ 3/ 7 LDPC Codes 74
75 Comments on Message-Passing Decoding Bad news: Message-passing decoding on a Tanner graph is not always optimal... Good news: For any code C, there is a parity-check matrix on whose Tanner graph message-passing is optimal, e.g., the matrix of codewords of the dual code. Bad news: That Tanner graph may be very dense, so even message-passing decoding is too complex. C 5/ 3/ 7 LDPC Codes 75
76 Another (7,4) Code 5/ 3/ 7 76 LDPC Codes = H R=4/7 d min =2 All stopping sets contain codeword supports. Message-passing decoder on this graph is optimal! (Cycle-free Tanner graph implies this.)
77 Comments on Message-Passing Decoding Good news: If a Tanner graph is cycle-free, the messagepassing decoder is optimal! Bad news: Binary linear codes with cycle-free Tanner graphs are necessarily weak... Good news: The Tanner graph of a long LDPC code behaves almost like a cycle-free graph! 5/ 3/ 7 LDPC Codes 77
78 Analysis of LDPC Codes on BEC In the spirit of Shannon, we can analyze the performance of message-passing decoding on ensembles of LDPC codes with specified degree distributions (λ,ρ). The results of the analysis allow us to design LDPC codes that transmit reliably with MP decoding at rates approaching the Shannon capacity of the BEC. In fact, sequences of LDPC codes have been designed that actually achieve the Shannon capacity. The analysis can assume the all- s codeword is sent. 5/ 3/ 7 LDPC Codes 78
79 Key Results - Concentration With high probability, the performance of l rounds of MP decoding on a randomly selected (n, λ, ρ) code converges to the ensemble average performance as the length n. Convergence to cycle-free performance The average performance of l rounds of MP decoding on the (n, λ, ρ) ensemble converges to the performance on a graph with no cycles of length 2l as the length n. 5/ 3/ 7 LDPC Codes 79
80 Key Results - 2 Computing the cycle-free performance The cycle-free performance can be computed by a tractable algorithm density evolution. Threshold calculation There is a threshold probability p*(λ,ρ) such that, for channel erasure probability ε < p*(λ,ρ), the cycle-free error probability approaches as the number of iterations l. 5/ 3/ 7 LDPC Codes 8
81 Asymptotic Performance Analysis We assume a cycle-free (λ,ρ) Tanner graph. Let p = ε, the channel erasure probability. We find a recursion formula for p l, the probability that a randomly chosen edge carries a variable-to-check erasure message in round l. We then find the largest ε such that p l converges to, as l. This value is called the threshold. This procedure is called density evolution analysis. 5/ 3/ 7 LDPC Codes 8
82 Density Evolution- Consider a check node node of of degree d with with independent incoming messages. Pr ( u =?) = Pr( vi =?, for some i =,, d ) = Pr( v?, for all i =,, d ) = ( i p ) d The probability that edge e connects to a check node of degree d is ρ d, so Pr dc d ( u =?) = ρ ( ( p ) ) d = = d d = = ρ c d ρ ( p d ( p ) ) d v uu edge e v d 2 v d 5/ 3/ 7 LDPC Codes 82
83 u from channel u d u d 2 Density Evolution-2 Consider a variable node of degree d with independent incoming messages. edge e vv Pr( v =?) = Pr( u =?) Pr( ui =?, for all i =,, d ) d = p u [ ρ( p ) ] The probability that edge e connects to a variable node of degree d is λ d, so Pr v ( v?) = λ p [ ρ( p )] = ( ρ( p )) 5/ 3/ 7 LDPC Codes 83 = d = p l = p λ ( ρ( p l- )) d p λ d d
84 Threshold Property p l = p λ ( ρ( p l- )) There is a threshold probability p*(λ, ρ) such that if then ( λ, ) p = ε < p * ρ, lim p. 5/ 3/ 7 LDPC Codes 84
85 Threshold Interpretation Operationally, this means that using a code drawn from the ensemble of length-n LDPC codes with degree distribution pair (λ, ρ), we can transmit as reliably as desired over the BEC(ε) channel if ε < p ( λ, ) * ρ, for sufficiently large block length n. 5/ 3/ 7 LDPC Codes 85
86 Computing the Threshold Define f (p,x) = p λ ( ρ( x)) The threshold p*(λ, ρ) is the largest probability p such that f (p,x) x < on the interval x (,]. This leads to a graphical interpretation of the threshold p*(λ, ρ) 5/ 3/ 7 LDPC Codes 86
87 Graphical Determination of the Threshold Example: (j,k)=(3,4) ( 3 x ) 2 ( ) x f ( x, p) x = p p*.6474 p =.7 p =.6474 p=.6 p=.5 5/ 3/ 7 LDPC Codes 87
88 (j,k)-regular LDPC Code Thresholds There is a closed form expression for thresholds of (j,k)-regular LDPC codes. Examples: (j,k) R p Sh p*(j,k) (3,4) /4 ¾= (3,5) 2/5 3/5= (3,6) /2 ½= (4,6) /3 ⅔ p *(3,4) = (4,8) /2 ½= / 3/ 7 LDPC Codes 88
89 Degree Distribution Optimization Two approaches: Fix design rate R(λ,ρ) and find degree distributions λ(x), ρ(x) to maximize the threshold p*(λ,ρ). Fix the threshold p*, and find degree distributions λ(x), ρ(x) to maximize the rate R(λ,ρ). For the latter, we can: start with a specific ρ(x) and optimize λ(x); then, for the optimal λ(x), find the optimal check distribution; ping-pong back and forth until satisfied with the results. LDPC Codes 5/ 3/ 7 89
90 Variable Degree Distribution Optimization Fix a check degree distribution ρ(x) and threshold ε. Fix maximum variable degree l max. Define g( x, λ,, λ 2 lmax ) = ελ( ρ( x)) x = ε i 2 λ ( ρ( x)) Use linear programming to find max λ l max ( λ / i) λ ; max i i i= 2 i= 2 l Since the rate R(λ,ρ) is an increasing function of λ i /i, this maximizes the design rate. i λ = ; g i i x for x [,] 5/ 3/ 7 LDPC Codes 9
91 Practical Optimization In practice, good performance is found for a check degree distribution of the form: ρ( r x ) = ax + ( a) Example : l max = 8, r =6, design rate ½ x r λ( x) =.49x +.22x x x 6 +.5x 7 ρ( x) = x 5 Rate: R(λ,ρ).54 Threshold: p*(λ,ρ).48 5/ 3/ 7 LDPC Codes 9
92 Bound on the Threshold Taylor series analysis yields the general upper bound: p *( λ, ρ) λ () ρ (). For previous example with p*(λ,ρ).48, the upper bound gives: λ () ρ () = (.49) / 3/ 7 LDPC Codes 92
93 EXIT Chart Analysis Extrinsic information transfer (EXIT) charts provide a nice graphical depiction of density evolution and MP decoding [tenbrink,999] Rewrite the density evolution recursion as: f ( x, p) = = pλ( ρ( v p (c( x)) x)) where v p ( x) = pλ( x) c( x) = ρ( x) 5/ 3/ 7 LDPC Codes 93
94 EXIT Chart Analysis Recall that the MP convergence condition was f ( x, p) < x, for all x (,) Since λ(x) is invertible, the condition becomes c( x) < v p ( x), for all x (,) Graphically, this says that the curve for c(x) must lie below the curve for v ( x) for all p < p*. p 5/ 3/ 7 LDPC Codes 94
95 EXIT Chart Example 5/ 3/ 7 95 LDPC Codes Example: (3,4)-regular LDPC code, p*= ) ( ) ( x x x x = = ρ λ ) ( ) ( ) ( ) ( ) ( ) ( = = = = = p x x v x px x x c x p x v p p ρ λ
96 EXIT Chart Example Example: (3,4)-regular LDPC code, p*= p=.5 p=.6 p*.6474 p=.7 p=.8 v p ( x ) x = p 2 for various values of initial erasure probability p.2. c( x) = ( x) / 3/ 7 LDPC Codes 96
97 EXIT Charts and Density Evolution 5/ 3/ 7 97 LDPC Codes EXIT charts can be used to visualize density evolution. Assume initial fraction of erasure messages p =p. The fraction of erasures emitted successively by check node q i and by variable nodes and p i are obtained by successively applying c(x) and v p (x). ] ) ( [note : )) ( ( ) ( ) ( ] ) ( [note : )) ( ( ) ( ) ( q p v p c v q v p p c q q p v p c v q v p p c q p p p p p p = = = = = = = =
98 EXIT Charts and Density Evolution Graphically, this computation describes a staircase function. If p < p*, there is a tunnel between v p - (x) and c(x) through which the staircase descends to ground level, i.e., no erasures. If p > p*, the tunnel closes, stopping the staircase descent at a positive fraction of errors. 5/ 3/ 7 LDPC Codes 98
99 Density Evolution Visualization - Example: (3,4)-regular LDPC code, p= v.6 x ( x) = c( x) = ( x) / 3/ 7 LDPC Codes 99
100 Density Evolution Visualization-2 Example: (3,4)-regular LDPC code q fraction of erasures from check nodes p = fraction of erasures from variable nodes p 5/ 3/ 7 LDPC Codes
101 Density Evolution Visualization Example: (3,4)-regular LDPC code q fraction of erasures from check nodes p =.6 q = fraction of erasures from variable nodes p 5/ 3/ 7 LDPC Codes
102 Density Evolution Visualization Example: (3,4)-regular LDPC code q fraction of erasures from check nodes p.5257 q fraction of erasures from variable nodes p 5/ 3/ 7 LDPC Codes 2
103 Density Evolution Visualization Example: (3,4)-regular LDPC code q fraction of erasures from check nodes p.5257 q fraction of erasures from variable nodes p 5/ 3/ 7 LDPC Codes 3
104 Density Evolution Visualization Example: (3,4)-regular LDPC code q fraction of erasures from check nodes p q fraction of erasures from variable nodes p 5/ 3/ 7 LDPC Codes 4
105 Density Evolution Visualization Example: (3,4)-regular LDPC code q fraction of erasures from check nodes p q fraction of erasures from variable nodes p 5/ 3/ 7 LDPC Codes 5
106 Density Evolution Visualization Example: (3,4)-regular LDPC code.9 q fraction of erasures from check nodes p q fraction of erasures from variable nodes p 5/ 3/ 7 LDPC Codes 6
107 Density Evolution Visualization Example: (3,4)-regular LDPC code.9 q fraction of erasures from check nodes p p l continues through the tunnel to fraction of erasures from variable nodes p 5/ 3/ 7 LDPC Codes 7
108 Density Evolution Visualization Example: (3,4)-regular LDPC code q fraction of erasures from check nodes p l continues through tunnel to fraction of erasures from variable nodes p 5/ 3/ 7 LDPC Codes 8
109 Matching Condition For capacity-achieving sequences of LDPC codes for the BEC, the EXIT chart curves must match. This is called the matching condition. Such sequences have been developed: Tornado codes Right-regular LDPC codes Accumulate-Repeat-Accumulate codes 5/ 3/ 7 LDPC Codes 9
110 Decoding for Other Channels We now consider analysis and design of LDPC codes for BSC(p) and BiAWGN(σ) channels. We call p and σ the channel parameter for these two channels, respectively. Many concepts, results, and design methods have natural (but non-trivial) extensions to these channels. The messages are probability mass functions or loglikelihood ratios. The message-passing paradigm at variable and check nodes will be applied. The decoding method is called belief propagation or BP, for short. 5/ 3/ 7 LDPC Codes
111 Belief Propagation 5/ 3/ 7 LDPC Codes Consider transmission of binary inputs X {±} over a memoryless channel using linear code C. Assume codewords are transmitted equiprobably. Then where is the indicator function for C. { } { } { } { } ) ( ) ( argmax ) ( ) ( argmax ) ( argmax ) ( arg max ) ( ˆ ~ ~ ~ x f x y P x P x y P y x P y x P y x C x n j j j X Y x X x X Y x x Y X x i Y X x MAP i j j i i i i i i i i = = = = = ± ± ± ± f C (x)
112 Belief Propagation For codes with cycle-free Tanner graphs, there is a messagepassing approach to bit-wise MAP decoding. The messages are essentially conditional bit distributions, denoted u = [u(), u(-)]. The initial messages presented by the channel to the variable nodes are of the form u ch, i = [ uch, i (), uch, i ( )] = [ py X ( yi ), py X ( yi )] i i i i The variable-to-check and check-to-variable message updates are determined by the sum-product update rule. The BEC decoder can be formulated as a BP decoder. 5/ 3/ 7 LDPC Codes 2
113 Sum-Product Update Rule Variable-to-check u ch from channel v u u u d 2 v( b) = u d ch k = u k ( b), for b { ± } u d Check-to-variable u( b) = { x, x 2,, x d } f ( b, x, x 2,, x d ) d k = v k ( x where f f is is the the parity-check indicator function. k ), v v d- v v d u 5/ 3/ 7 LDPC Codes 3
114 Variable Node Update - Heuristic Variable-to-check v u ch from channel u d u u u d 2 v( b) = u d ch k = u k ( b), for b { ± } Suppose incoming messages u,, u,,..., u d- d- from from check nodes,,,,...,..., d- d- and and message u ch from ch from the the channel are are independent estimates of of [P(x [P(x = ), ), P(x P(x = -)]. -)]. Then, a reasonable estimate to to send send to to check node node based upon upon the the other other estimates would be be the the product of of those estimates (suitably normalized). Pˆ( x = b) = P ch ( x = b) We We do do not not use use the the intrinsic information u provided by by check node node.. The The estimate v represents extrinsic information. 5/ 3/ 7 LDPC Codes 4 d k = P k ( x = b)
115 Check-Node Update - Heuristic u( b) = { x, x 2,, x d } f ( b, x, x 2,, x d ) d k = v k ( x k ), Parity-check node node equation: r r s s t t = Over Over {-,}, this this translates to: to: r r s s t t = P(r=) = P(s P(s =,, t t = ) ) + P(s P(s = -, -, t t = -) -) r r s s v v u = P(s P(s = )P(t )P(t = ) ) + P(s P(s = -)P(t = -) -) [by [by independence assumption] t t v 2 Similarly P(r P(r = -) -) = P(s P(s =,, t t = -)+P(s = -, -, t t = ) ) = P(s P(s = )P(t )P(t = -)+P(s = -)P(t = ) ) 5/ 3/ 7 LDPC Codes 5
116 Log-Likelihood Formulation The sum-product update is simplified using log-likelihoods For message u, define u() L( u) = log u( ) Note that L( u) ( e u ) = and u( ) = L( u) L( u + e + e ) 5/ 3/ 7 LDPC Codes 6
117 Log-Likelihood Formulation Variable Node The variable-to-check update rule then takes the form: L(v) from channel L ( u ) L( u ) L( u d 2 ) L( v) d = k = L( u k ) L( u ) d 5/ 3/ 7 LDPC Codes 7
118 Log-Likelihood Formulation Check Node The check-to-variable update rule then takes the form: L(u) edge e L( u) d = 2 tanh k = tanh L( v k 2 ) L(v ) L(v d-2 ) L(v d- ) 5/ 3/ 7 LDPC Codes 8
119 Log-Likelihood Formulation Check Node To see this, consider the special case of a degree 3 check node. It is easy to verify that P r Q r = ( Ps Qs )( Pt Qt ) where t t P = P( a = ) and Q = P( a = ),for node a a a This can be generalized to a check node of any degree by a simple inductive argument. r r s s v v v 2 u 5/ 3/ 7 LDPC Codes 9
120 Log-Likelihood Formulation Check Node 5/ 3/ 7 2 LDPC Codes Translating to log-likelihood ratios, this becomes Noting that we conclude u v v 2 v r r s s t t ) ( ) ( ) ( ) ( ) ( ) ( = + v L v L v L v L u L u L e e e e e e ) tanh( 2 ) ( ) ( ) ( 2 ) ( 2 ) ( 2 ) ( 2 ) ( a L a L a L a L a L e e e e e e a L a L = + = + ) ) tanh( tanh( ) tanh( ) ( ) ( ) ( v L v L u L =
121 Key Results - Concentration With high probability, the performance of l rounds of BP decoding on a randomly selected (n, λ, ρ) code converges to the ensemble average performance as the length n. Convergence to cycle-free performance The average performance of l rounds of MP decoding on the (n, λ, ρ) ensemble converges to the performance on a graph with no cycles of length 2l as the length n. 5/ 3/ 7 LDPC Codes 2
122 Key Results -2 Computing the cycle-free performance The cycle-free performance can be computed by a somewhat more complex, but still tractable, algorithm density evolution. Threshold calculation There is a threshold channel parameter p*(λ,ρ) such that, for any better channel parameter p, the cycle-free error probability approaches as the number of iterations l. 5/ 3/ 7 LDPC Codes 22
123 Density Evolution (AWGN) Assume the all- s sequence is transmitted The density evolution algorithm computes the probability distribution or density of LLR messages after each round of BP decoding. Let P denote the initial LLR message density. It depends on the channel parameter σ. Let P l denote the density after l iterations. The density evolution equation for a (λ,ρ) degree distribution pair is: P = P λ( Γ ( ρ( Γ( P )))) 5/ 3/ 7 LDPC Codes 23
124 Density Evolution P = P λ( Γ ( ρ( Γ( P )))) Here denotes convolution of of densities and Γ is is interpreted as as an an invertible operator on on probability densities. We interpret λ(p) and ρ(p) as as operations on on densities: λ( P) = i 2 λ ( P) i ( i ) and ρ( P) = i 2 ρ ( P) i ( i ) The fraction of of incorrect (i.e., negative) messages after l iterations is: is: P ( z) dz LDPC Codes 5/ 3/ 7 24
125 Threshold P = P λ( Γ ( ρ( Γ( P )))) The threshold σ* σ* is is the the maximum σ such that lim P ( z) dz =. Operationally, this represents the the minimum SNR such that a code drawn from the the (λ,ρ) ensemble will ensure reliable transmission as as the the block length approaches infinity. 5/ 3/ 7 LDPC Codes 25
126 Degree Distribution Optimization For a given rate, the objective is is to to optimize λ(x) and ρ(x) for the best threshold p*. The maximum left and right degrees are fixed. For some channels, the optimization procedure is is not trivial, but there are some techniques that can be applied in in practice. 5/ 3/ 7 LDPC Codes 26
127 Thresholds - (j,k)-regular BSC(p) (j,k) R p*(j,k) p Sh (3,4) (4,6) (3,5).4.3 (3,6).5.84 (4,8) BiAWGN(σ) (j,k) R σ* σ Sh (3,4) (4,6).333. (3,5).4. (3,6).5.88 (4,8) / 3/ 7 LDPC Codes 27
128 Thresholds and Optimized Irregular Codes BiAWGN σ Sh Sh =.979 Rate R=½ λ max σ* / 3/ 7 LDPC Codes 28
129 Irregular Code vs. Turbo Codes AWGN R=/2 n = = 3 3,, 4 4,, 5 5,, 6 6 BER Richardson, Richardson, Shokrollahi, Shokrollahi, and and Urbanke, Urbanke, 2 2 5/ 3/ 7 LDPC Codes 29
130 Density Evolution Density evolution must track probability distributions/densities of the log-likelihood ratio messages. A discretized version of the sum-product algorithm, and associated discretized density evolution, speeds code design considerably. This design method has produced rate ½ LDPC ensembles with thresholds within.45db of the Shannon limit on the AWGN channel! A rate /2 code with block length 7 provided BER of -6 within.4 db of the Shannon limit! 5/ 3/ 7 LDPC Codes 3
131 Some Really Good LDPC Codes Chung, Chung, et et al., al., dB from from Shannon limit! 5/ 3/ 7 LDPC Codes 3
132 Good Code Performance Chung, Chung, et et al., al., / 3/ 7 LDPC Codes 32
133 Applications of LDPC Codes The performance benefits that LDPC codes offer on the BEC, BSC, and AWGN channels have been shown empirically (and sometimes analytically) to extend to many other channels, including Fading channels Channels with memory Coded modulation for bandwidth-limited channels MIMO Systems 5/ 3/ 7 LDPC Codes 33
134 Rayleigh Fading Channels Hou, Hou, et et al., al., 2 2 R=/2, (3,6) (3,6) 5/ 3/ 7 LDPC Codes 34
135 Rayleigh Fading Channels 5/ 3/ 7 LDPC Codes 35
136 Rayleigh Fading Channels 5/ 3/ 7 LDPC Codes 36
137 Partial-Response Channels Kurkoski, et et al., al., / 3/ 7 LDPC Codes 37
138 Dicode (-D) Channel Results Rate Rate 7/8 7/8 Regular j=3 j=3 n=495 5/ 3/ 7 LDPC Codes 38
139 EPR4 (+D-D 2 -D 3 ) Channel Results Rate Rate 7/8 7/8 Regular j=3 j=3 n=495 5/ 3/ 7 LDPC Codes 39
140 Optimized Codes for Partial Response Varnica and and Kavcic, / 3/ 7 LDPC Codes 4
141 Optimized Codes for Partial Response 5/ 3/ 7 LDPC Codes 4
142 Optimized Codes for Partial Response R=.7 n= n= 6 6 5/ 3/ 7 LDPC Codes 42
143 Some Basic References R.G. Gallager, Low-Density Parity-Check Codes. Cambridge, MA: MIT Press, 963 (Sc.D. MIT, 96). T. Richardson and R. Urbanke, Modern Coding Theory. Cambridge University Press (Preliminary version, May 2, 27) Special Issue on Codes on Graphs and Iterative Algorithms, IEEE Transactions on Information Theory, February 2. S-Y Chung, et al., On the design of low-density parity-check codes within.45 db of the Shannon limit, IEEE Commun. Letters, vol. 5, no. 2, pp. 58-6, February 2. 5/ 3/ 7 LDPC Codes 43
144 Additional References J. Hou, P.H. Siegel, and L.B. Milstein, Performance analysis and code optimization of low density parity-check codes on Rayleigh fading channels, IEEE J. Select. Areas Commun., Issue on The Turbo Principle: From Theory to Practice I, vol. 9, no. 5, pp , May 2. B.M. Kurkoski, P.H. Siegel, and J.K. Wolf, Joint message passing decoding of LCPC coded partial-response channels, IEEE Trans. Inform. Theory, vol. 48, no. 6, pp , June 22. (See also B.M. Kurkoski, P.H. Siegel, and J.K. Wolf, Correction to Joint message passing decoding of LCPC coded partial-response channels, IEEE Trans. Inform. Theory, vol. 49, no. 8, p. 276, August 23.) N. Varnica and A. Kavcic, Optimized LDPC codes for partial response channels, IEEE Communications Letters, vol. 7, pp. 68-7, April 23. LDPC Codes 5/ 3/ 7 44
145 Concluding Remarks LDPC codes are very powerful codes with enormous practical potential, founded upon deep and rich theory. There continue to be important advances in all of the key aspects of LDPC code design, analysis, and implementation. LDPC codes are now finding their way into many applications: Satellite broadcast Cellular wireless Data storage And many more 5/ 3/ 7 LDPC Codes 45
Performance Analysis and Code Optimization of Low Density Parity-Check Codes on Rayleigh Fading Channels
Performance Analysis and Code Optimization of Low Density Parity-Check Codes on Rayleigh Fading Channels Jilei Hou, Paul H. Siegel and Laurence B. Milstein Department of Electrical and Computer Engineering
More informationEE229B - Final Project. Capacity-Approaching Low-Density Parity-Check Codes
EE229B - Final Project Capacity-Approaching Low-Density Parity-Check Codes Pierre Garrigues EECS department, UC Berkeley garrigue@eecs.berkeley.edu May 13, 2005 Abstract The class of low-density parity-check
More informationIntroduction to Low-Density Parity Check Codes. Brian Kurkoski
Introduction to Low-Density Parity Check Codes Brian Kurkoski kurkoski@ice.uec.ac.jp Outline: Low Density Parity Check Codes Review block codes History Low Density Parity Check Codes Gallager s LDPC code
More informationLDPC Codes. Intracom Telecom, Peania
LDPC Codes Alexios Balatsoukas-Stimming and Athanasios P. Liavas Technical University of Crete Dept. of Electronic and Computer Engineering Telecommunications Laboratory December 16, 2011 Intracom Telecom,
More informationECEN 655: Advanced Channel Coding
ECEN 655: Advanced Channel Coding Course Introduction Henry D. Pfister Department of Electrical and Computer Engineering Texas A&M University ECEN 655: Advanced Channel Coding 1 / 19 Outline 1 History
More informationLecture 4 : Introduction to Low-density Parity-check Codes
Lecture 4 : Introduction to Low-density Parity-check Codes LDPC codes are a class of linear block codes with implementable decoders, which provide near-capacity performance. History: 1. LDPC codes were
More informationLDPC Codes. Slides originally from I. Land p.1
Slides originally from I. Land p.1 LDPC Codes Definition of LDPC Codes Factor Graphs to use in decoding Decoding for binary erasure channels EXIT charts Soft-Output Decoding Turbo principle applied to
More information5. Density evolution. Density evolution 5-1
5. Density evolution Density evolution 5-1 Probabilistic analysis of message passing algorithms variable nodes factor nodes x1 a x i x2 a(x i ; x j ; x k ) x3 b x4 consider factor graph model G = (V ;
More informationLow-density parity-check codes
Low-density parity-check codes From principles to practice Dr. Steve Weller steven.weller@newcastle.edu.au School of Electrical Engineering and Computer Science The University of Newcastle, Callaghan,
More informationBOUNDS ON THE MAP THRESHOLD OF ITERATIVE DECODING SYSTEMS WITH ERASURE NOISE. A Thesis CHIA-WEN WANG
BOUNDS ON THE MAP THRESHOLD OF ITERATIVE DECODING SYSTEMS WITH ERASURE NOISE A Thesis by CHIA-WEN WANG Submitted to the Office of Graduate Studies of Texas A&M University in partial fulfillment of the
More informationAn Introduction to Low Density Parity Check (LDPC) Codes
An Introduction to Low Density Parity Check (LDPC) Codes Jian Sun jian@csee.wvu.edu Wireless Communication Research Laboratory Lane Dept. of Comp. Sci. and Elec. Engr. West Virginia University June 3,
More informationCapacity-approaching codes
Chapter 13 Capacity-approaching codes We have previously discussed codes on graphs and the sum-product decoding algorithm in general terms. In this chapter we will give a brief overview of some particular
More informationOn Generalized EXIT Charts of LDPC Code Ensembles over Binary-Input Output-Symmetric Memoryless Channels
2012 IEEE International Symposium on Information Theory Proceedings On Generalied EXIT Charts of LDPC Code Ensembles over Binary-Input Output-Symmetric Memoryless Channels H Mamani 1, H Saeedi 1, A Eslami
More informationOn the Typicality of the Linear Code Among the LDPC Coset Code Ensemble
5 Conference on Information Sciences and Systems The Johns Hopkins University March 16 18 5 On the Typicality of the Linear Code Among the LDPC Coset Code Ensemble C.-C. Wang S.R. Kulkarni and H.V. Poor
More informationCodes on graphs and iterative decoding
Codes on graphs and iterative decoding Bane Vasić Error Correction Coding Laboratory University of Arizona Funded by: National Science Foundation (NSF) Seagate Technology Defense Advanced Research Projects
More informationCodes on graphs and iterative decoding
Codes on graphs and iterative decoding Bane Vasić Error Correction Coding Laboratory University of Arizona Prelude Information transmission 0 0 0 0 0 0 Channel Information transmission signal 0 0 threshold
More informationMessage-Passing Decoding for Low-Density Parity-Check Codes Harish Jethanandani and R. Aravind, IIT Madras
Message-Passing Decoding for Low-Density Parity-Check Codes Harish Jethanandani and R. Aravind, IIT Madras e-mail: hari_jethanandani@yahoo.com Abstract Low-density parity-check (LDPC) codes are discussed
More informationIterative Encoding of Low-Density Parity-Check Codes
Iterative Encoding of Low-Density Parity-Check Codes David Haley, Alex Grant and John Buetefuer Institute for Telecommunications Research University of South Australia Mawson Lakes Blvd Mawson Lakes SA
More informationPractical Polar Code Construction Using Generalised Generator Matrices
Practical Polar Code Construction Using Generalised Generator Matrices Berksan Serbetci and Ali E. Pusane Department of Electrical and Electronics Engineering Bogazici University Istanbul, Turkey E-mail:
More informationLOW-density parity-check (LDPC) codes were invented
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 54, NO 1, JANUARY 2008 51 Extremal Problems of Information Combining Yibo Jiang, Alexei Ashikhmin, Member, IEEE, Ralf Koetter, Senior Member, IEEE, and Andrew
More informationBelief-Propagation Decoding of LDPC Codes
LDPC Codes: Motivation Belief-Propagation Decoding of LDPC Codes Amir Bennatan, Princeton University Revolution in coding theory Reliable transmission, rates approaching capacity. BIAWGN, Rate =.5, Threshold.45
More informationLow-density parity-check (LDPC) codes
Low-density parity-check (LDPC) codes Performance similar to turbo codes Do not require long interleaver to achieve good performance Better block error performance Error floor occurs at lower BER Decoding
More informationIntroducing Low-Density Parity-Check Codes
Introducing Low-Density Parity-Check Codes Sarah J. Johnson School of Electrical Engineering and Computer Science The University of Newcastle Australia email: sarah.johnson@newcastle.edu.au Topic 1: Low-Density
More informationFountain Uncorrectable Sets and Finite-Length Analysis
Fountain Uncorrectable Sets and Finite-Length Analysis Wen Ji 1, Bo-Wei Chen 2, and Yiqiang Chen 1 1 Beijing Key Laboratory of Mobile Computing and Pervasive Device Institute of Computing Technology, Chinese
More informationDecoding of LDPC codes with binary vector messages and scalable complexity
Downloaded from vbn.aau.dk on: marts 7, 019 Aalborg Universitet Decoding of LDPC codes with binary vector messages and scalable complexity Lechner, Gottfried; Land, Ingmar; Rasmussen, Lars Published in:
More informationRCA Analysis of the Polar Codes and the use of Feedback to aid Polarization at Short Blocklengths
RCA Analysis of the Polar Codes and the use of Feedback to aid Polarization at Short Blocklengths Kasra Vakilinia, Dariush Divsalar*, and Richard D. Wesel Department of Electrical Engineering, University
More informationModern Coding Theory. Daniel J. Costello, Jr School of Information Theory Northwestern University August 10, 2009
Modern Coding Theory Daniel J. Costello, Jr. Coding Research Group Department of Electrical Engineering University of Notre Dame Notre Dame, IN 46556 2009 School of Information Theory Northwestern University
More informationJoint Iterative Decoding of LDPC Codes and Channels with Memory
Joint Iterative Decoding of LDPC Codes and Channels with Memory Henry D. Pfister and Paul H. Siegel University of California, San Diego 3 rd Int l Symposium on Turbo Codes September 1, 2003 Outline Channels
More informationCOMPSCI 650 Applied Information Theory Apr 5, Lecture 18. Instructor: Arya Mazumdar Scribe: Hamed Zamani, Hadi Zolfaghari, Fatemeh Rezaei
COMPSCI 650 Applied Information Theory Apr 5, 2016 Lecture 18 Instructor: Arya Mazumdar Scribe: Hamed Zamani, Hadi Zolfaghari, Fatemeh Rezaei 1 Correcting Errors in Linear Codes Suppose someone is to send
More informationExact Probability of Erasure and a Decoding Algorithm for Convolutional Codes on the Binary Erasure Channel
Exact Probability of Erasure and a Decoding Algorithm for Convolutional Codes on the Binary Erasure Channel Brian M. Kurkoski, Paul H. Siegel, and Jack K. Wolf Department of Electrical and Computer Engineering
More informationLower Bounds on the Graphical Complexity of Finite-Length LDPC Codes
Lower Bounds on the Graphical Complexity of Finite-Length LDPC Codes Igal Sason Department of Electrical Engineering Technion - Israel Institute of Technology Haifa 32000, Israel 2009 IEEE International
More informationLecture 8: Shannon s Noise Models
Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 2007) Lecture 8: Shannon s Noise Models September 14, 2007 Lecturer: Atri Rudra Scribe: Sandipan Kundu& Atri Rudra Till now we have
More informationB I N A R Y E R A S U R E C H A N N E L
Chapter 3 B I N A R Y E R A S U R E C H A N N E L The binary erasure channel (BEC) is perhaps the simplest non-trivial channel model imaginable. It was introduced by Elias as a toy example in 954. The
More informationGraph-based codes for flash memory
1/28 Graph-based codes for flash memory Discrete Mathematics Seminar September 3, 2013 Katie Haymaker Joint work with Professor Christine Kelley University of Nebraska-Lincoln 2/28 Outline 1 Background
More information6.451 Principles of Digital Communication II Wednesday, May 4, 2005 MIT, Spring 2005 Handout #22. Problem Set 9 Solutions
6.45 Principles of Digital Communication II Wednesda, Ma 4, 25 MIT, Spring 25 Hand #22 Problem Set 9 Solutions Problem 8.3 (revised) (BCJR (sum-product) decoding of SPC codes) As shown in Problem 6.4 or
More informationSlepian-Wolf Code Design via Source-Channel Correspondence
Slepian-Wolf Code Design via Source-Channel Correspondence Jun Chen University of Illinois at Urbana-Champaign Urbana, IL 61801, USA Email: junchen@ifpuiucedu Dake He IBM T J Watson Research Center Yorktown
More information16.36 Communication Systems Engineering
MIT OpenCourseWare http://ocw.mit.edu 16.36 Communication Systems Engineering Spring 2009 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 16.36: Communication
More informationSingle-Gaussian Messages and Noise Thresholds for Low-Density Lattice Codes
Single-Gaussian Messages and Noise Thresholds for Low-Density Lattice Codes Brian M. Kurkoski, Kazuhiko Yamaguchi and Kingo Kobayashi kurkoski@ice.uec.ac.jp Dept. of Information and Communications Engineering
More informationAn Efficient Maximum Likelihood Decoding of LDPC Codes Over the Binary Erasure Channel
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 5, NO. 11, NOVEMBER 24 1 An Efficient Maximum Likelihood Decoding of LDPC Codes Over the Binary Erasure Channel David Burshtein and Gadi Miller School of Electrical
More informationMaking Error Correcting Codes Work for Flash Memory
Making Error Correcting Codes Work for Flash Memory Part I: Primer on ECC, basics of BCH and LDPC codes Lara Dolecek Laboratory for Robust Information Systems (LORIS) Center on Development of Emerging
More informationCoding Techniques for Data Storage Systems
Coding Techniques for Data Storage Systems Thomas Mittelholzer IBM Zurich Research Laboratory /8 Göttingen Agenda. Channel Coding and Practical Coding Constraints. Linear Codes 3. Weight Enumerators and
More informationOptimal Rate and Maximum Erasure Probability LDPC Codes in Binary Erasure Channel
Optimal Rate and Maximum Erasure Probability LDPC Codes in Binary Erasure Channel H. Tavakoli Electrical Engineering Department K.N. Toosi University of Technology, Tehran, Iran tavakoli@ee.kntu.ac.ir
More informationTHE seminal paper of Gallager [1, p. 48] suggested to evaluate
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 11, NOVEMBER 2004 2657 Extrinsic Information Transfer Functions: Model and Erasure Channel Properties Alexei Ashikhmin, Member, IEEE, Gerhard Kramer,
More informationMaximum Likelihood Decoding of Codes on the Asymmetric Z-channel
Maximum Likelihood Decoding of Codes on the Asymmetric Z-channel Pål Ellingsen paale@ii.uib.no Susanna Spinsante s.spinsante@univpm.it Angela Barbero angbar@wmatem.eis.uva.es May 31, 2005 Øyvind Ytrehus
More informationAn Introduction to Algorithmic Coding Theory
An Introduction to Algorithmic Coding Theory M. Amin Shokrollahi Bell Laboratories Part : Codes - A puzzle What do the following problems have in common? 2 Problem : Information Transmission MESSAGE G
More informationAnalysis of Sum-Product Decoding of Low-Density Parity-Check Codes Using a Gaussian Approximation
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 47, NO. 2, FEBRUARY 2001 657 Analysis of Sum-Product Decoding of Low-Density Parity-Check Codes Using a Gaussian Approximation Sae-Young Chung, Member, IEEE,
More informationGraph-based Codes for Quantize-Map-and-Forward Relaying
20 IEEE Information Theory Workshop Graph-based Codes for Quantize-Map-and-Forward Relaying Ayan Sengupta, Siddhartha Brahma, Ayfer Özgür, Christina Fragouli and Suhas Diggavi EPFL, Switzerland, UCLA,
More informationEnhancing Binary Images of Non-Binary LDPC Codes
Enhancing Binary Images of Non-Binary LDPC Codes Aman Bhatia, Aravind R Iyengar, and Paul H Siegel University of California, San Diego, La Jolla, CA 92093 0401, USA Email: {a1bhatia, aravind, psiegel}@ucsdedu
More informationConvergence analysis for a class of LDPC convolutional codes on the erasure channel
Convergence analysis for a class of LDPC convolutional codes on the erasure channel Sridharan, Arvind; Lentmaier, Michael; Costello Jr., Daniel J.; Zigangirov, Kamil Published in: [Host publication title
More informationCapacity-Achieving Ensembles for the Binary Erasure Channel With Bounded Complexity
Capacity-Achieving Ensembles for the Binary Erasure Channel With Bounded Complexity Henry D. Pfister, Member, Igal Sason, Member, and Rüdiger Urbanke Abstract We present two sequences of ensembles of non-systematic
More informationSpatially Coupled LDPC Codes
Spatially Coupled LDPC Codes Kenta Kasai Tokyo Institute of Technology 30 Aug, 2013 We already have very good codes. Efficiently-decodable asymptotically capacity-approaching codes Irregular LDPC Codes
More informationMessage Passing Algorithm with MAP Decoding on Zigzag Cycles for Non-binary LDPC Codes
Message Passing Algorithm with MAP Decoding on Zigzag Cycles for Non-binary LDPC Codes Takayuki Nozaki 1, Kenta Kasai 2, Kohichi Sakaniwa 2 1 Kanagawa University 2 Tokyo Institute of Technology July 12th,
More informationAdaptive Cut Generation for Improved Linear Programming Decoding of Binary Linear Codes
Adaptive Cut Generation for Improved Linear Programming Decoding of Binary Linear Codes Xiaojie Zhang and Paul H. Siegel University of California, San Diego, La Jolla, CA 9093, U Email:{ericzhang, psiegel}@ucsd.edu
More informationRate-Compatible Low Density Parity Check Codes for Capacity-Approaching ARQ Schemes in Packet Data Communications
Rate-Compatible Low Density Parity Check Codes for Capacity-Approaching ARQ Schemes in Packet Data Communications Jing Li (Tiffany) Electrical Engineering Dept, Texas A&M University College Station, TX
More informationQuasi-cyclic Low Density Parity Check codes with high girth
Quasi-cyclic Low Density Parity Check codes with high girth, a work with Marta Rossi, Richard Bresnan, Massimilliano Sala Summer Doctoral School 2009 Groebner bases, Geometric codes and Order Domains Dept
More informationCodes on Graphs. Telecommunications Laboratory. Alex Balatsoukas-Stimming. Technical University of Crete. November 27th, 2008
Codes on Graphs Telecommunications Laboratory Alex Balatsoukas-Stimming Technical University of Crete November 27th, 2008 Telecommunications Laboratory (TUC) Codes on Graphs November 27th, 2008 1 / 31
More informationGraph-based Codes and Iterative Decoding
Graph-based Codes and Iterative Decoding Thesis by Aamod Khandekar In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy California Institute of Technology Pasadena, California
More informationAPPLICATIONS. Quantum Communications
SOFT PROCESSING TECHNIQUES FOR QUANTUM KEY DISTRIBUTION APPLICATIONS Marina Mondin January 27, 2012 Quantum Communications In the past decades, the key to improving computer performance has been the reduction
More informationWhy We Can Not Surpass Capacity: The Matching Condition
Why We Can Not Surpass Capacity: The Matching Condition Cyril Méasson EPFL CH-1015 Lausanne cyril.measson@epfl.ch, Andrea Montanari LPT, ENS F-75231 Paris montanar@lpt.ens.fr Rüdiger Urbanke EPFL CH-1015
More informationConstruction of low complexity Array based Quasi Cyclic Low density parity check (QC-LDPC) codes with low error floor
Construction of low complexity Array based Quasi Cyclic Low density parity check (QC-LDPC) codes with low error floor Pravin Salunkhe, Prof D.P Rathod Department of Electrical Engineering, Veermata Jijabai
More informationELEC546 Review of Information Theory
ELEC546 Review of Information Theory Vincent Lau 1/1/004 1 Review of Information Theory Entropy: Measure of uncertainty of a random variable X. The entropy of X, H(X), is given by: If X is a discrete random
More informationAsynchronous Decoding of LDPC Codes over BEC
Decoding of LDPC Codes over BEC Saeid Haghighatshoar, Amin Karbasi, Amir Hesam Salavati Department of Telecommunication Systems, Technische Universität Berlin, Germany, School of Engineering and Applied
More informationLDPC Decoder LLR Stopping Criterion
International Conference on Innovative Trends in Electronics Communication and Applications 1 International Conference on Innovative Trends in Electronics Communication and Applications 2015 [ICIECA 2015]
More informationAn algorithm to improve the error rate performance of Accumulate-Repeat-Accumulate codes Tae-Ui Kim
An algorithm to improve the error rate performance of Accumulate-Repeat-Accumulate codes Tae-Ui Kim The Graduate School Yonsei University Department of Electrical and Electronic Engineering An algorithm
More informationThe Turbo Principle in Wireless Communications
The Turbo Principle in Wireless Communications Joachim Hagenauer Institute for Communications Engineering () Munich University of Technology (TUM) D-80290 München, Germany Nordic Radio Symposium, Oulu,
More informationDistributed Source Coding Using LDPC Codes
Distributed Source Coding Using LDPC Codes Telecommunications Laboratory Alex Balatsoukas-Stimming Technical University of Crete May 29, 2010 Telecommunications Laboratory (TUC) Distributed Source Coding
More informationFactor Graphs and Message Passing Algorithms Part 1: Introduction
Factor Graphs and Message Passing Algorithms Part 1: Introduction Hans-Andrea Loeliger December 2007 1 The Two Basic Problems 1. Marginalization: Compute f k (x k ) f(x 1,..., x n ) x 1,..., x n except
More informationCoding for loss tolerant systems
Coding for loss tolerant systems Workshop APRETAF, 22 janvier 2009 Mathieu Cunche, Vincent Roca INRIA, équipe Planète INRIA Rhône-Alpes Mathieu Cunche, Vincent Roca The erasure channel Erasure codes Reed-Solomon
More informationOn Achievable Rates and Complexity of LDPC Codes over Parallel Channels: Bounds and Applications
On Achievable Rates and Complexity of LDPC Codes over Parallel Channels: Bounds and Applications Igal Sason, Member and Gil Wiechman, Graduate Student Member Abstract A variety of communication scenarios
More informationMessage Passing Algorithm and Linear Programming Decoding for LDPC and Linear Block Codes
Message Passing Algorithm and Linear Programming Decoding for LDPC and Linear Block Codes Institute of Electronic Systems Signal and Information Processing in Communications Nana Traore Shashi Kant Tobias
More informationLow Density Parity Check (LDPC) Codes and the Need for Stronger ECC. August 2011 Ravi Motwani, Zion Kwok, Scott Nelson
Low Density Parity Check (LDPC) Codes and the Need for Stronger ECC August 2011 Ravi Motwani, Zion Kwok, Scott Nelson Agenda NAND ECC History Soft Information What is soft information How do we obtain
More informationTurbo Codes for Deep-Space Communications
TDA Progress Report 42-120 February 15, 1995 Turbo Codes for Deep-Space Communications D. Divsalar and F. Pollara Communications Systems Research Section Turbo codes were recently proposed by Berrou, Glavieux,
More informationA t super-channel. trellis code and the channel. inner X t. Y t. S t-1. S t. S t+1. stages into. group two. one stage P 12 / 0,-2 P 21 / 0,2
Capacity Approaching Signal Constellations for Channels with Memory Λ Aleksandar Kav»cić, Xiao Ma, Michael Mitzenmacher, and Nedeljko Varnica Division of Engineering and Applied Sciences Harvard University
More informationTime-invariant LDPC convolutional codes
Time-invariant LDPC convolutional codes Dimitris Achlioptas, Hamed Hassani, Wei Liu, and Rüdiger Urbanke Department of Computer Science, UC Santa Cruz, USA Email: achlioptas@csucscedu Department of Computer
More informationCommunication by Regression: Achieving Shannon Capacity
Communication by Regression: Practical Achievement of Shannon Capacity Department of Statistics Yale University Workshop Infusing Statistics and Engineering Harvard University, June 5-6, 2011 Practical
More informationOptimizing Flash based Storage Systems
Optimizing Flash based Storage Systems 1 Lifetime Reliability Latency Throughput 2 Projects Reliability/Latency/Throughput: Error Correction Code (ECC) Parallelization and Incremental Redundancy Lifetime:
More informationLecture 4 Noisy Channel Coding
Lecture 4 Noisy Channel Coding I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw October 9, 2015 1 / 56 I-Hsiang Wang IT Lecture 4 The Channel Coding Problem
More informationBounds on Achievable Rates of LDPC Codes Used Over the Binary Erasure Channel
Bounds on Achievable Rates of LDPC Codes Used Over the Binary Erasure Channel Ohad Barak, David Burshtein and Meir Feder School of Electrical Engineering Tel-Aviv University Tel-Aviv 69978, Israel Abstract
More informationECC for NAND Flash. Osso Vahabzadeh. TexasLDPC Inc. Flash Memory Summit 2017 Santa Clara, CA 1
ECC for NAND Flash Osso Vahabzadeh TexasLDPC Inc. 1 Overview Why Is Error Correction Needed in Flash Memories? Error Correction Codes Fundamentals Low-Density Parity-Check (LDPC) Codes LDPC Encoding and
More informationError Floors of LDPC Coded BICM
Electrical and Computer Engineering Conference Papers, Posters and Presentations Electrical and Computer Engineering 2007 Error Floors of LDPC Coded BICM Aditya Ramamoorthy Iowa State University, adityar@iastate.edu
More informationBounds on Mutual Information for Simple Codes Using Information Combining
ACCEPTED FOR PUBLICATION IN ANNALS OF TELECOMM., SPECIAL ISSUE 3RD INT. SYMP. TURBO CODES, 003. FINAL VERSION, AUGUST 004. Bounds on Mutual Information for Simple Codes Using Information Combining Ingmar
More informationLecture 12. Block Diagram
Lecture 12 Goals Be able to encode using a linear block code Be able to decode a linear block code received over a binary symmetric channel or an additive white Gaussian channel XII-1 Block Diagram Data
More informationSIPCom8-1: Information Theory and Coding Linear Binary Codes Ingmar Land
SIPCom8-1: Information Theory and Coding Linear Binary Codes Ingmar Land Ingmar Land, SIPCom8-1: Information Theory and Coding (2005 Spring) p.1 Overview Basic Concepts of Channel Coding Block Codes I:
More informationCompressed Sensing and Linear Codes over Real Numbers
Compressed Sensing and Linear Codes over Real Numbers Henry D. Pfister (joint with Fan Zhang) Texas A&M University College Station Information Theory and Applications Workshop UC San Diego January 31st,
More informationRecent Results on Capacity-Achieving Codes for the Erasure Channel with Bounded Complexity
26 IEEE 24th Convention of Electrical and Electronics Engineers in Israel Recent Results on Capacity-Achieving Codes for the Erasure Channel with Bounded Complexity Igal Sason Technion Israel Institute
More informationA Short Length Low Complexity Low Delay Recursive LDPC Code
A Short Length Low Complexity Low Delay Recursive LDPC Code BASHAR M. MANSOOR, TARIQ Z. ISMAEEL Department of Electrical Engineering College of Engineering, University of Baghdad, IRAQ bmml77@yahoo.com
More informationIntegrated Code Design for a Joint Source and Channel LDPC Coding Scheme
Integrated Code Design for a Joint Source and Channel LDPC Coding Scheme Hsien-Ping Lin Shu Lin and Khaled Abdel-Ghaffar Department of Electrical and Computer Engineering University of California Davis
More informationECE Information theory Final (Fall 2008)
ECE 776 - Information theory Final (Fall 2008) Q.1. (1 point) Consider the following bursty transmission scheme for a Gaussian channel with noise power N and average power constraint P (i.e., 1/n X n i=1
More informationTurbo Codes are Low Density Parity Check Codes
Turbo Codes are Low Density Parity Check Codes David J. C. MacKay July 5, 00 Draft 0., not for distribution! (First draft written July 5, 998) Abstract Turbo codes and Gallager codes (also known as low
More informationStructured Low-Density Parity-Check Codes: Algebraic Constructions
Structured Low-Density Parity-Check Codes: Algebraic Constructions Shu Lin Department of Electrical and Computer Engineering University of California, Davis Davis, California 95616 Email:shulin@ece.ucdavis.edu
More informationThe PPM Poisson Channel: Finite-Length Bounds and Code Design
August 21, 2014 The PPM Poisson Channel: Finite-Length Bounds and Code Design Flavio Zabini DEI - University of Bologna and Institute for Communications and Navigation German Aerospace Center (DLR) Balazs
More informationIterative Decoding for Wireless Networks
Iterative Decoding for Wireless Networks Thesis by Ravi Palanki In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy California Institute of Technology Pasadena, California
More informationAnalysis of Absorbing Sets and Fully Absorbing Sets of Array-Based LDPC Codes
Analysis of Absorbing Sets and Fully Absorbing Sets of Array-Based LDPC Codes Lara Dolecek, Zhengya Zhang, Venkat Anantharam, Martin J. Wainwright, and Borivoje Nikolić dolecek@mit.edu; {zyzhang,ananth,wainwrig,bora}@eecs.berkeley.edu
More informationAdvances in Error Control Strategies for 5G
Advances in Error Control Strategies for 5G Jörg Kliewer The Elisha Yegal Bar-Ness Center For Wireless Communications And Signal Processing Research 5G Requirements [Nokia Networks: Looking ahead to 5G.
More informationTurbo Compression. Andrej Rikovsky, Advisor: Pavol Hanus
Turbo Compression Andrej Rikovsky, Advisor: Pavol Hanus Abstract Turbo codes which performs very close to channel capacity in channel coding can be also used to obtain very efficient source coding schemes.
More informationDecomposition Methods for Large Scale LP Decoding
Decomposition Methods for Large Scale LP Decoding Siddharth Barman Joint work with Xishuo Liu, Stark Draper, and Ben Recht Outline Background and Problem Setup LP Decoding Formulation Optimization Framework
More informationOn Bit Error Rate Performance of Polar Codes in Finite Regime
On Bit Error Rate Performance of Polar Codes in Finite Regime A. Eslami and H. Pishro-Nik Abstract Polar codes have been recently proposed as the first low complexity class of codes that can provably achieve
More informationResearch Letter Design of Short, High-Rate DVB-S2-Like Semi-Regular LDPC Codes
Research Letters in Communications Volume 2008, Article ID 324503, 4 pages doi:0.55/2008/324503 Research Letter Design of Short, High-Rate DVB-S2-Like Semi-Regular LDPC Codes Luca Barletta and Arnaldo
More informationThe Design of Rate-Compatible Structured Low-Density Parity-Check Codes
The Design of Rate-Compatible Structured Low-Density Parity-Check Codes A Thesis Presented to The Academic Faculty by Jaehong Kim In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy
More informationOn Turbo-Schedules for LDPC Decoding
On Turbo-Schedules for LDPC Decoding Alexandre de Baynast, Predrag Radosavljevic, Victor Stolpman, Joseph R. Cavallaro, Ashutosh Sabharwal Department of Electrical and Computer Engineering Rice University,
More information