Quasi-cyclic Low Density Parity Check codes with high girth, a work with Marta Rossi, Richard Bresnan, Massimilliano Sala Summer Doctoral School 2009 Groebner bases, Geometric codes and Order Domains Dept of Mathematics, Univ of Trento Italy 10 June 2009
Outline 1 Introduction Basics of Low Density Parity Check codes 2 Quasi-cyclic codes Girth investigation Quasi-cyclic codes with girth 10, (3,6) regular Quasi-cyclic codes with girth 8 and rate 2/3
Outline 1 Introduction Basics of Low Density Parity Check codes 2
Outline 1 Introduction Basics of Low Density Parity Check codes 2 Quasi-cyclic codes Girth investigation Quasi-cyclic codes with girth 10, (3,6) regular Quasi-cyclic codes with girth 8 and rate 2/3
Intro to Low Density Parity Check codes LDPC codes are linear block codes A code C can be define as a subspace of the vector space GF(2 m ) N of dimension K A basis B = {g 0, g 1,, g K 1 } of C In matrix form G= [g 0,, g K 1 ], [K N] generator matrix By definition of Basis every element of C can be written as a linear combination of the basis elements This gives a method to compute a codeword from an input message u ([1 K] vector): c = ug c is a [1 N] vector that is transmitted through the channel
Intro to Low Density Parity Check codes Consider now the null space of C, the set of vectors x for which xg = 0 A basis B = {h 0, h 1,, h N K 1 } In matrix form H= [h 0,, h N K 1 ], [(N K) N] parity check matrix For each codeword in C is: ch T = 0 this gives an easy way to test if the received message contains errors A Low Density Parity Check Code is a linear block code for which the parity check matrix H has a low density of 1 s (Gallager 63)
Why LDPC codes? The sparseness of the H matrix has several advantages (decoding, minimum distance, ) They perform close to Shannon limit at very long (> 10000) block lengths Superior error correcting capabilities than Turbo codes Implementation complexity is one magnitude order smaller than codes with similar performances Ideal for long distance satellite, mobile communications High rate codes used also in storage
Graphical representation of LDPC Tanner Graph c 0 c 1 c 2 c 3 Check Nodes b 0 b 1 b 2 b 3 b 4 b 5 b 6 b 7 Bit Nodes 2 H = 6 4 1 0 1 0 1 0 0 0 0 1 0 1 0 1 0 0 0 0 0 1 0 0 1 1 0 0 1 0 1 0 0 1 Allow efficient iterative decoding, Belief propagation 3 7 5
Iterative decoding Check nodes q a mn r a mn Variable nodes q a mn= prob bit n=a given all information coming from check node different from m r a mn= prob check m is satisfied given that bit n=a
Issues of LDPC codes Hardware point of view Long randomly constructed LDPC codes have very good performances but are hard to implement Dimension of the code Connections network complexity Encoding complexity Area/Memory consumption Performance point of view Performances for short (<2000) medium (2000-10000)length codes cannot be predicted Iterative decoding is equivalent to ML decoding only if the code graph has no cycles Performance of the codes improve increasing the length of the smallest cycle (girth)
Issues of LDPC codes Hardware point of view Long randomly constructed LDPC codes have very good performances but are hard to implement Dimension of the code Connections network complexity Encoding complexity Area/Memory consumption Performance point of view Performances for short (<2000) medium (2000-10000)length codes cannot be predicted Iterative decoding is equivalent to ML decoding only if the code graph has no cycles Performance of the codes improve increasing the length of the smallest cycle (girth) Performance: probability of error versus Signal to noise ratio
Analytic Codes Systematically designed using algebra, geometry or non-random algorithms Attempts to create codes with good performances and which simplify the decoder implementation Loss of performance for long codes in comparison with random codes Not so marked a difference for short/medium codes They are a good trade-off In this work a class of Analytic Codes, the Quasi-cyclic codes has been investigated
Outline 1 Introduction 2 Quasi-cyclic codes Girth investigation Quasi-cyclic codes with girth 10, (3,6) regular Quasi-cyclic codes with girth 8 and rate 2/3
Outline 1 Introduction Basics of Low Density Parity Check codes 2 Quasi-cyclic codes Girth investigation Quasi-cyclic codes with girth 10, (3,6) regular Quasi-cyclic codes with girth 8 and rate 2/3
Circulant matrices Definition (Circulant) Let C be an [m m] matrix over GF(2) We say that C is circulant if its rows are obtained by successive shifts 2 6 4 The polynomial representation of the first row, p(x), is called the polynomial of C We call exponents of C, ǫ(p) the exponents of its polynomial representation We call separation of C, s(p) = min(b a, m (b a)) circular submatrix 0 1 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 1 1 0 0 0 0 1 0 0 1 0 0 0 0 1 1 0 1 0 0 0 0 3 7 5
Circulant matrices Definition (Circulant) Let C be an [m m] matrix over GF(2) We say that C is circulant if its rows are obtained by successive shifts 2 6 4 The polynomial representation of the first row, p(x), is called the polynomial of C We call exponents of C, ǫ(p) the exponents of its polynomial representation We call separation of C, s(p) = min(b a, m (b a)) circular submatrix 0 1 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 1 1 0 0 0 0 1 0 0 1 0 0 0 0 1 1 0 1 0 0 0 0 3 7 5 Polynomial representation C(x) = x + x 3
Circulant matrices Definition (Circulant) Let C be an [m m] matrix over GF(2) We say that C is circulant if its rows are obtained by successive shifts 2 6 4 The polynomial representation of the first row, p(x), is called the polynomial of C We call exponents of C, ǫ(p) the exponents of its polynomial representation We call separation of C, s(p) = min(b a, m (b a)) circular submatrix 0 1 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 1 1 0 0 0 0 1 0 0 1 0 0 0 0 1 1 0 1 0 0 0 0 3 7 5 Polynomial representation C(x) = x + x 3 Separation & exponents e(c) = {1, 3}; s(c) = 2;
Quasi-cyclic codes Definition (Quasi-cyclic) A quasi-cyclic code of index t is a linear block code C in which a cyclic shift of any codeword in C by t positions is also a codeword The generator matrix G for these codes is a matrix where every row is a t circular shift of the previous row A quasi-cyclic codes can be obtained by building H matrix assembling circulant submatrices H matrix
Quasi-cyclic codes Definition (Quasi-cyclic) A quasi-cyclic code of index t is a linear block code C in which a cyclic shift of any codeword in C by t positions is also a codeword The generator matrix G for these codes is a matrix where every row is a t circular shift of the previous row A quasi-cyclic codes can be obtained by building H matrix assembling circulant submatrices H matrix G matrix
Advantages of quasi-cyclic codes Encoding The complexity of the encoding process scales with O(n) Really efficient hardware implementation No need to store full G matrix (Johnson 02) Decoding Implementation benefits from the rigid structure No need to store full H matrix Others - The structure of the code allow mathematical analysis - Various properties of the codes can be easily studied - Of easy construction
Advantages of quasi-cyclic codes Encoding The complexity of the encoding process scales with O(n) Really efficient hardware implementation No need to store full G matrix (Johnson 02) Decoding Implementation benefits from the rigid structure No need to store full H matrix Others - The structure of the code allow mathematical analysis - Various properties of the codes can be easily studied - Of easy construction
Advantages of quasi-cyclic codes Encoding The complexity of the encoding process scales with O(n) Really efficient hardware implementation No need to store full G matrix (Johnson 02) Decoding Implementation benefits from the rigid structure No need to store full H matrix Others - The structure of the code allow mathematical analysis - Various properties of the codes can be easily studied - Of easy construction
Outline 1 Introduction Basics of Low Density Parity Check codes 2 Quasi-cyclic codes Girth investigation Quasi-cyclic codes with girth 10, (3,6) regular Quasi-cyclic codes with girth 8 and rate 2/3
Literature review Girth is one parameter usually targeted for optimization of performances, in particular error floor Many studies on girth and quasi-cyclic Fundamental result from Fossorier ( 04) proves limitation to girth 12 for a certain class of QC codes Most of previous works present a specific construction that guarantee high girth And many works consider only circulant permutation matrix of weight 1 (permutation matrices) We present the set of conditions that all quasi-cyclic codes have to satisfy to have a certain girth Considering also weight 2 circulant matrices
Cycles and girth Check nodes 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 0 00 11 00 11 1 00 000 11 111 00 11 00 11 00 00 11 11 000 111 000 111 00 00 110 1000 111 00 00 11 11 11 00 00 0 11 11 1 000 111 1100 000 111 000 111 00 00 00 11 11 11 000 111 00 11 00 11 000 111 00 00 00 00 000 11 11 11 11 111 00 00 0 00 0000 11 11 1000 111 00 11 00 11 11 1111 00 11 00 000 11 111 00 000 11 111 00 11 00 11 00 11 000 111 Edges 00 11 000 111 00 11 00 00 0 00 0 11 1 000 111 00 11 000 111 00 11 000 111 00 00 00 11 11 11 000 11100 00 00 00 00 00 000 11 11 1 000 111 00 000 11 111 11 11 11 11 11 111 00 0000 000 11 1111 111 00 11 000 111 00 11 000 111 00 0000 000 11 1111 111 00 11 00 000 11 111 00 11 11000 111 00 11 000 111 00 11 000 111 00 11 00 00 11 11 000 111 00 00 0 00 00 0 11 11 1 000 111 11000 111 000 111 00 11 00 11 000 111 00 00 00 00 00 000 11 11 1 00 11 11 11 11 11 11 111 0 00 0000 0 11 1111 1 00 000 11 111 00 11 1 00 11 000 111 00 0000 00 11 1111 11 000 111 00 000 11 111 000 111 000 111 00 11 000 111 00 11 000 1110 1 000 111 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 Symbol Nodes Figure: Example of a cycle on a Tanner Graph In a graph, a cycle is a path that starts from a node v and ends in v The girth of a graph is the smallest of its cycles
New Cycles definition on matrices Check nodes H = 1 1 1 0 1 0 0 0 1 1 1 0 1 0 1 0 1 1 0 0 1 Symbol Nodes We want to define 2s-cycles in an arbitrary binary matrix Definition (Linked set) Sub-set V of 2s entries of the matrix is called linked if the entries lay in s columns and s rows 1 1 1 1 1 1 1 1 1 1 1 1 1 1 6 entries 8 entries
New Cycles definition on matrices Definition (2s-Cycle) A sub-set V of 2s entries of the matrix is called a 2s-cycle if : V is linked and For any r such that 2 <= t < s there is no linked sub-set W of V of 2t entries 1 1 1 1 1 1 1 1 8 entries,no cycle 1 1 1 1 1 1 1 1 8-cycle
New Cycles definition on matrices Definition (2s-Cycle) A sub-set V of 2s entries of the matrix is called a 2s-cycle if : V is linked and For any r such that 2 <= t < s there is no linked sub-set W of V of 2t entries 1 1 1 1 1 1 1 1 8 entries,no cycle 1 1 1 1 1 1 1 1 8-cycle
Problem and methodology AIM: Find and classify all possible cycles and connected conditions on a generic quasi-cyclic matrix
Problem and methodology AIM: Find and classify all possible cycles and connected conditions on a generic quasi-cyclic matrix Problems: How to prove that all has been found? How to deal with the huge amount of possibilities?
Problem and methodology AIM: Find and classify all possible cycles and connected conditions on a generic quasi-cyclic matrix Problems: How to prove that all has been found? How to deal with the huge amount of possibilities? Methodology: Various mathematical tools have been constructed to deal with the problem - Configuration: all cycles that are obtainable from other under row/column permutation are equivalent - Theorems and lemma useful to trim the research - Numeric representation of the configurations and cycle lying in them
Examples of cycles in generic matrix First part of the study work on finding possible cycles on any generic matrix that can be decomposed in square sub-matrices Cycles present in a configuration are represented in a numerical way by associating any circulant matrix with a number representing the number of point of the cycle that lies in it [ B1 B 2 B 3 B 4 B 5 ] [ 2 1 1 1 1 ]
Examples of cycles in generic matrix First part of the study work on finding possible cycles on any generic matrix that can be decomposed in square sub-matrices Cycles present in a configuration are represented in a numerical way by associating any circulant matrix with a number representing the number of point of the cycle that lies in it [ B1 B 2 B 3 B 4 B 5 2 1 1 0 0 0 1 0 1 0 0 0 1 0 1 0 0 0 1 1 ] [ 2 1 1 1 1 1 1 1 1 0 1 0 0 0 1 0 1 1 0 0 0 0 0 1 1 ]
Examples of cycles in generic matrix First part of the study work on finding possible cycles on any generic matrix that can be decomposed in square sub-matrices Cycles present in a configuration are represented in a numerical way by associating any circulant matrix with a number representing the number of point of the cycle that lies in it [ B1 B 2 B 3 B 4 B 5 2 1 1 0 0 0 1 0 1 0 0 0 1 0 1 0 0 0 1 1 ] [ 2 1 1 1 1 1 1 1 1 0 1 0 0 0 1 0 1 1 0 0 0 0 0 1 1 ]
The quasi-cyclic case Notation : [ C 1 3 J 2 1 J 3 1 4 1 ] First result For any matrix M st at least two weight-2 circulant lay in the same row or column then the girth is less or equal 8 [ C 1 4 C 2 4 ]
Example 0 1 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 1 1 0 0 0 0 1 0 0 1 0 0 0 0 1 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 1 0 0 1 0 0 0 0 1 0 0 1 0 0 0 0 1 0 0 1 0 0 0 0 1 0 0 1
4-cycles 41 42 43 C 4, s(p) = m/2 C 1 2 C 2 2, s(p 1 ) = s(p 2 ) 1 1 2 1 3 1 4 1, ǫ(p1 ) ǫ(p 2 ) ǫ(p 3 ) + ǫ(p 4 ) = 0 1 0 0 1 0 0 0 1 0 0 1 0 41 = 0 0 1 0 0 1 1 0 0 1 0 0 0 1 0 0 1 0 0 0 1 0 0 1
6-cycles 61 C 6, s(p) = m/3 62 C 1 4 C 2 2, s(p 2 ) ±2s(p 1 ) 2 6 4 1 0 1 0 0 0 0 1 0 1 0 0 0 0 1 0 1 0 0 0 0 1 0 1 1 0 0 0 1 0 0 1 0 0 0 1 32 7 5 6 4 0 1 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 1 1 0 0 0 0 1 0 0 1 0 0 0 0 1 1 0 1 0 0 0 0 3 7 5 2 6 4 0 1 0 0 0 1 0 0 0 1 0 0 0 1 1 0 0 1 0 1 0 0 1 0 0 1 0 1 0 0 1 0 0 1 0 0 0 0 1 0 0 1 1 0 0 0 1 0 0 3 7 5
Complete list 63 C 1 2 C 2 2 C 3 2, ± s(p 1 ) ± s(p 2 ) ± s(p 3 ) 0, 85 813 J 1 2 2 1 3 1 1 2 2 2 3 2 4 2, C 4 2 O O O 5 1 6 1, ǫ(p 1 ) + ǫ(p 1 ) ǫ(p 2 ) ǫ(p 2 ) ǫ(p 3 ) ǫ(p 3 ) + ǫ(p 4 ) + ǫ(p 4 ) 0, ǫ(p 1 ) ǫ(p 1 ) + ǫ(p 2 ) ǫ(p 3 ) ǫ(p 5 ) + ǫ(p 6 ) ±s(p 4 ) 64 65 66 67 81 82 83 84 C1 2 2 2 O C 3 2, ǫ(p 2 ) ǫ(p 2 ) ±s(p 1 ) ± s(p 3 ), C1 3 2 1 3 1 4 1 ǫ(p 1 ) ǫ(p 2 ) ǫ(p 3 ) + ǫ(p 4 ) ±s(p 1 ),, C1 2 2 1 3 1 O 4 1 5 1 ǫ(p 2 ) ǫ(p 3 ) ǫ(p 4 ) + ǫ(p 5 ) ±s(p 1 ), 1 1 2 1 O 3 1 O 4 1 O 5 1 6 1, ǫ(p 1 ) ǫ(p 2 ) ǫ(p 3 ) + ǫ(p 4 ) + ǫ(p 5 ) ǫ(p 6 ) 0 C1 5 J 2 1 J 3 1 4 1 C1 4 J 2 2 0 C 3 2, C 8, s(p) = m/4 811, ǫ(p1 ) ǫ(p 2 ) ǫ(p 3 ) + ǫ(p 4 ) ±2s(p 1 ),, ǫ(p2 ) ǫ(p 2 ) ±s(p 3 ) ± 2s(p 1 ) C1 3 J 2 1 J 3 1 C 4 3, ǫ(p 1 ) ǫ(p 2 ) ǫ(p 3 ) + ǫ(p 4 ) ±s(p 1 ) ± s(p 3 ), 86 87 C1 4 J 2 1 J 3 1 O 4 1 5 1 ǫ(p 2 ) ǫ(p 3 ) ǫ(p 4 ) + ǫ(p 5 ) ±2s(p 1 ), 2 C1 3 O J 1 J 1 J 1 C 2 812 C 1 3 J 2 1 O J 3 1 O 4 1 O 5 1 6 1, ǫ(p 1 ) ǫ(p 2 ) ǫ(p 3 ) + ǫ(p 4 ) + ǫ(p 5 ) ǫ(p 6 ) ±s(p 1 ),,, 814 1 2 2 1 3 1 4 1 5 1 O 6 1 O 7 1 ǫ(p 1 ) + ǫ(p 1 ) ǫ(p 2 ) ǫ(p 3 ) ǫ(p 4 ) + ǫ(p 5 ) ǫ(p 6 ) + ǫ(p 7 ) = 0, or ǫ(p 2 ) ǫ(p 3 ) + ǫ(p 4 ) ǫ(p 5 ) ǫ(p 6 ) + ǫ(p 7 ) = ±s(p 1 ), ǫ(p 1 ) ǫ(p 2 ) ǫ(p 3 ) + ǫ(p 5 ) ±s(p 1 ) ± s(p 4 815 ), C 1 2 O O J 2 1 3 1 O 88 J 4 1 J 5 1 C 6 2, 1 2 2 1 3 1 4 2 5 1 6 1, ǫ(p 2 ) ǫ(p 3 ) ǫ(p 4 ) + ǫ(p 5 ) ±s(p 1 ) ± s(p 6 ), ǫ(p 1 ) + ǫ(p 1 ) ǫ(p 2 ) ǫ(p 3 ) ǫ(p 4 ) ǫ(p 4 ) + ǫ(p 5 ) + ǫ(p 6 ) = 0, 816 1 2 2 1 3 1 4 1 89 5 1 6 1 O 0 C1 2 J 2 2 O O J 3 2 C 4 2, O O 7 1 8 1, ǫ(p 2 ) ǫ(p 3 ) ǫ(p 4 ) + ǫ(p 5 ) + ǫ(p 6 ) ǫ(p 7 ) = ±s(p 1 ), ǫ(p 2 ) ǫ(p 2 ) + ǫ(p 3 ) ǫ(p 3 ) ±s(p 1 ) ± s(p 4 ), 817 C 1 2 J 2 1 J 3 1 O 810 C1 2 O J 3 1 J 4 1 O 4 1 O 5 1 O C 2 2 J 5 1 J 6 1, O O 6 1 7 1, ǫ(p 3 ) ǫ(p 4 ) ǫ(p 5 ) + ǫ(p 6 ) ±s(p 1 ) ± s(p 2 ǫ(p 2 ) ǫ(p 3 ) ǫ(p 4 ) + ǫ(p 5 ) + ǫ(p 6 ) ǫ(p 7 ) = ±s(p 1 ), ), 818 1 1 2 1 O O 3 1 O 4 1 O 1 1 2 1 3 1 4 1 5 1 6 1 7 1 8 1, O 5 1 O 6 1, O O 7 1 8 1 ǫ(p 1 ) + ǫ(p 2 ) ǫ(p 3 ) ǫ(p 4 ) ǫ(p 5 ) ǫ(p 6 ) + ǫ(p 7 ) + ǫ(p 8 ) = 0, or ǫ(p 1 ) ǫ(p 2 ) ǫ(p 3 ) + ǫ(p 4 ) + ǫ(p 5 ) ǫ(p 6 ) ǫ(p 7 ) + ǫ(p 8 ) = 0, ǫ(p 1 ) ǫ(p 2 ) + ǫ(p 3 ) ǫ(p 4 ) ǫ(p 5 ) + ǫ(p 6 ) ǫ(p 7 ) + ǫ(p 8 ) = 0, or ǫ(p 1 ) ǫ(p 2 ) ǫ(p 3 ) + ǫ(p 4 ) ǫ(p 5 ) + ǫ(p 6 ) + ǫ(p 7 ) ǫ(p 8 ) = 0,
Important aspects to remember The most general case has been considered The conditions given are not limited to a particular construction There are a lot of possible cycle configurations The analysis gives indication of the shape that the H matrix has to have to avoid cycles During the studies various constructions have been developed and studied The performance of some of these codes are presented next
Outline 1 Introduction Basics of Low Density Parity Check codes 2 Quasi-cyclic codes Girth investigation Quasi-cyclic codes with girth 10, (3,6) regular Quasi-cyclic codes with girth 8 and rate 2/3
H matrix form H 1 = H = [H 1 H 2 ] H1 1 0 0 I I H2 1 0 0 0 I 0 0 0 I Hα 1
A particular case with girth 10 H 2 = J 1 0 0 I 0 0 I 0 0 0 J 2 0 0 I 0 0 I 0 I I 0 0 0 0 0 I I 0 0 0 I I J α
A particular case with girth 10 H 2 = J 1 0 0 I 0 0 I 0 0 0 J 2 0 0 I 0 0 I 0 I I 0 0 0 0 0 I I 0 0 0 I I J α
Performance of (3, 6)-regular, N = 2, 294 girth tent quasi-cyclic code
Outline 1 Introduction Basics of Low Density Parity Check codes 2 Quasi-cyclic codes Girth investigation Quasi-cyclic codes with girth 10, (3,6) regular Quasi-cyclic codes with girth 8 and rate 2/3
H matrix form Extension of the Bresnan codes used for hardware implementation H = [H 1 H 2 H 3 ] 2 H 1 = 6 4 H 1 1 0 0 I I H 1 2 0 0 0 I 0 0 0 I H 1 α 3 2 7 5 H 2 = 6 4 H 2 3 1 I 0 0 0 H 2 2 I 0 0 0 7 2 I 5 I 0 0 H 2 α H 3 = 6 4 H 3 1 0 I 0 0 H 3 2 0 I 0 0 3 7 5 0 I 0 H 3 α
Performance of R = 2/3, N 1, 000 girth eight quasi-cyclic code
Thanks for listening! Any Questions? E-mail: christianspagnol@ieeeorg Acknowledgments: This research was funded by the Embark Initiative Research Scholarship Scheme from the Irish Research Council for Science, Engineering and Technology (IRCSET)