CHAPTER 3 LOW DENSITY PARITY CHECK CODES

Size: px

Start display at page:

Download "CHAPTER 3 LOW DENSITY PARITY CHECK CODES"

Theodora Hawkins
5 years ago
Views:

1 62 CHAPTER 3 LOW DENSITY PARITY CHECK CODES 3. INTRODUCTION LDPC codes were first presented by Gallager in 962 [] and in 996, MacKay and Neal re-discovered LDPC codes.they proved that these codes approach the channel capacity, given by the Shannon limit [],[2]. There are two kinds of LDPC codes, one is called regular code, having parity-check matrix with constant column and row weights (i.e., number of ones in binary LDPC codes) and the other is the irregular codes where the number of ones are not constant in each row or column. Irregular codes [5],[6],[7] can perform better than the regular codes but involves higher decoding complexity. Many researchers proposed decoding methodologies for different variants of LDPC codes as in [8] but at the implementation level, these methodologies does pose different challenges. The biggest advantage of the LDPC codes is that, the required generator matrix and the parity check matrix are of low density of non zeros, resulting in easier encoding. The recent trends and objectives of LDPC decoding algorithms are to exploit the parallelization in decoding process to achieve speed or higher data rate handling capacity and at the same time the complexity is constrained to be simple. The other most important issue is to maintain no error-floors at high SNR s. The wireless

2 63 communication demanded some more pre-requisites and redefined the objectives for the current and the next generation communication standard. They are Requirement of low power in battery powered wireless devices To use lower strength signals (by error coding) Reduction of inter-device interference, allowing more devices to share the same spectrum Supporting High data rates for delivering today s data rich in multi-media content to portable devices Higher Data security and Compression Ability to reconfigure the hardware according to the requirements is extremely desirable 3.2 LINEAR BLOCK CODES LDPC codes belongs to Linear Block Codes. An overview of these codes is discussed here. To encode, one needs to map the information vector i = [a,a2,..ak] into a codeword C=[c,c 2,...,c k,c k+,.,c n] i.e. code word is a function of information vector, C =f(i) and can be expressed as C = i * G, where G is a k x n matrix. All code words of the vector C are distinct when the rank of G is k. The code rate of such a code is given by k/n i.e. there are k message bits in a total of n bits of the codeword. First, write the basis vectors (of size x n) of G i.e., [g,g 2,..,g k] of C as rows of matrix G (k x n). Information vector i = [a, a2,, ak] can be encoded as, C = i * G = [a, a2,., ak] * G where ai Є GF(2) (3.)

3 64 The (k x n ) matrix G over GF(2) has k rows g,g 2 g k- which are linearly independent.there exists an (n-k) x n matrix H over GF(2). h h, h, h,n- h h, h, h,n- H = * = * * * h n-k- hn-k-, hn-k-, hn-k-,n-...(3.2) Where h,h,.hn-k- are (n-k) linearly independent rows of H. The row space of G is a null space of H i.e.,for any row gi in G and hj in H, gi * hj = The Parity Check Criterion The most important linear block codes principle is that a code vector C is a codeword in C provided that CH T =. If C is the code word having G as the generator matrix, then generator matrix should be linearly independent and at the same time the rank of the column should be equal to that of the row rank. G = [P I k]... (3.3) In systematic generator matrix, the k information bits are embedded in a sequence such that they can be identified easily. C = I * G = m * G where (I = m = information /message vector) = [m,m,.mk-] * [P Ik]

4 65 = [c c.c n-k-]...(3.4) When decoding is done, the last information k symbols are extracted. For binary code words P T = P T. Using Gaussian eliminations, the generator matrix can be converted into respective parity check matrix in the systematic form as H = [In-k -P T ]...(3.5) For code words P T = P T. (in binary form) Parity check matrix H(n-k) n in binary form consists of zeros and ones, particularly the number of ones are very few leading to the reduced computations. For k information bits, the set of LDPC code words c C, should satisfy the relation CH T =. 3.3 REPRESENTATION OF LDPC CODES A parity check matrix or alternatively a Bipartite graph using variable and check nodes constructed from the parity check matrix can be used to represent the LDPC codes Matrix Representation The parity check matrix having dimensions n m of (8, 4) code is shown. For LDPC, the wc value (weight of s in columns ) is much less than n for columns, wr should be much less than m for rows. The small matrix A shown below does not satisfy the requirement for LDPC codes to be effective.the number of zeros should be large while number of ones should be small to be called as Low Density ( of ones) Parity Check (LDPC) Matrix. The ones participate in the computation of parity to find the error.hence, if ones are less, computation is less

66 Since the current work involves Projective Geometry (PG) based LDPC codes, the points and lines of PG can create a parity check matrix as outlined above.

5 66 Since the current work involves Projective Geometry (PG) based LDPC codes, the points and lines of PG can create a parity check matrix as outlined above. Inorder to understand PGLDPC, one should understand the bipartite graph Representation by Bipartite Graph A bipartite graph as shown below, is a graph containing nodes or vertices interconnected with the help of the information from the parity check matrix, by the undirected edges Fig.3. Bipartite (Tanner) graph The check node finds the probability that whether a parity check meets the condition or not. In case, a particular bit is set as one (or zero) while the other bits have values with some probability distribution based on the known a priori probabilities. The bit nodes find the probability of a data bit being (or ), while the information from the remaining checks are used to arrive at this calculation. The connection in the Tanner graph between check and variable nodes

6 67 exists for the bits having an entry one at a specific location in its parity check matrix.only these corresponding nodes participate in the calculations. As an example, consider the graph constructed using s parity check matrix A with rows representing check nodes and the columns representing variable nodes A= The connections between nodes, determine the location of the ones and zeros in matrix A. Having a one at row i, column j, simply indicates that check node i is connected to bit node j. Looking at the figure, check node H is having connections to bit nodes X, X2, X4, X6, and X 9.So ones are placed in first, second, fourth, sixth, and ninth locations of the first row. Like wise, bit node X is having connections to check nodes H, H 3 and H 5.So ones are placed in first, third and fifth locations of the first column. The number of ports that the check node has determines the number of ones in a corresponding row and the number of ports that the bit node has, determines the number of ones with respect to a particular column.

7 LDPC Codes Classification LDPC codes are broadly classified as regular and irregular codes. If the code has Wc number of ones constant for every column and wr = wc * (n/m) (where n rows and m columns ) number of ones are constant for every row is called a regular LDPC code, otherwise, it is an irregular code. To construct an irregular LDPC code, the degree of the variable and check nodes in the bipartite graph are chosen according to some distribution. Fig-3.2: Irregular LDPC code. An irregular LDPC code shown above has 4 of the 2 variable nodes have degree 3 and the remaining variable nodes have degree 2. 3 of the 5 check nodes have degree 6 and the remaining check nodes have degree Short cycles in bipartite Graph A cycle is the one wherein, starting from the first vertex traverses through the edges only once each time and returns to the starting

8 69 vertex. The total number of such edges for a given cycle is known as the length of the cycle and the shortest such length of the cycle is well known as girth. The performance is directly affected by the girth because, the message passing gets saturated. LDPC codes do suffer from the short cycles (girth 4) in their bipartite graph and affects the error detection capability. The girth should be increased from 4 to 6 at least. 3.4 Construction Of LDPC Codes Suppose, LDPC codes are specified by a parity-check matrix denoted by A ( which is sparse). LDPC code is denoted by (n,tc,tr), where n denotes the block length, tc indicates the weight of each column of the matrix A, and tr indicates the weight of each row with tr > t c. The structure of these codes is well portrayed by bipartite graphs, which consists of variable and check nodes. Variable nodes indicate the elements of the code word. Check node indicate the set of paritycheck conditions. The -by-n code vector C is first partitioned as in[9] and [2]: Code vector C = bn-kx : mkx where b = parity vector The parity matrix A T = Since C.A T =,which implies b n-kx. A + mkx. A2 =... (3.6)

9 7 and b = m * P, substituting in equation (3.6) PA + A 2 = P= A 2. A -...(3.7). The Generator Matrix is defined as G = P : I K = A 2 * A - : I K... (3.8) After getting generator matrix G, one can find out the codeword C for the given message bits m by C = m * G. For example let the bipartite graph be as shown in figure 3. for (,3,5) LDPC code has the matrix representation A = A The parity matrix A T = A2 Then, A = A2=

10 7 We know that P= A 2 * A - One can obtain from the above relation A 2 * A - = The matrix G = P : I K = A 2. A - : I K = The code word C = m * G For example, If m = [ ] and using the above Generator Matrix G, the code vector is given by C = [ ] 3.5 DECODING OF LDPC CODES LDPC codes can be decoded using message passing algorithm involving updation of check and variable-nodes in sequence. Some of the well known algorithms for this purpose are Sum of Products (SP), the second one is min-sum (MS) etc Belief Propagation Algorithm This algorithm is explained in [23], is also called message passing algorithm and is iterative in nature as the messages are exchanged from variable to check and vice versa. The information from message nodes to check nodes are calculated depending on the observed value

11 72 of the variable node and some of the messages passed from the other nodes to the current message node excluding the message sent in the previous round from check node c to the message (variable) node v. The vice versa approach is similar i.e., the same is true for messages passed from check (parity) nodes to message nodes. This algorithm is irrespective of the channel, even though the messages passed through interconnections are fully dependent on the channel. The iterative decoding is described in [2]: Initialization: The variables x ij and x ij are set to the priori probabilities e j and e j of the symbols and respectively with e j + e j=. Horizontal step: The check (parity) nodes generate y ij values ( where y ijis the probability that check j meets the correct condition, assuming that data bit t i = and while, y ij represent the probability that check j meets the condition, assuming that data bit t i =.Compute these probabilities. The meaning of the notation i row[ j]/{ i} is that the indices i ( <= i <= n) of all bits in row j ( <= j <= m) which have value, not including the current bit index i. yij = / 2 ( xi ' j xi ' j ) ' i i ( j)/ i

12 73 yij = / 2 ( xi ' j xi ' j ) ' i i ( j)/ i...(3.9) Vertical step: The bit nodes(variable nodes) generate the x ij values (where x ij represents the probability that bit t i =, given the values of all checks done other than the current j and x ij represents the probability that the bit t i =, given the values of all checks other than the value of j. The meaning of the notation j col[ i]/{ j} is that the indices j ( <= j <= m) of all checks in the column i ( <= i <= n) having the value, not including the current check index j, α ij is normalizing value chosen such that x ij + x ij =. p i and p i represent the current estimate of the posterior probabilities for every bit. These are two extrinsic values, as discussed below, for all the iterations after the initial one. For the initial iteration, they are initialized to values found by the data received from the channel.for example, if the channel demodulator finds that the signal received is very close to the expected value for a bit one, then, it would assign a high value i.e. logical. The channel can supply binary values for p and p. i i x = α p y i j i j i i j ' j ' j ( i ) / j x = α p y i j i j i i j ' j ' j ( i ) / j...(3.)

13 74 The bit nodes also determines the extrinsic (pseudo) probabilities e i, which are the calculated posterior probabilities that a bit, having a given value i.e., e i is the computed probability that the bit at the i th position t i =. These extrinsic probabilities are used to find the decoded values are for every bit and are useful in the bit node equation.the accuracy of these probabilities improves with each iteration of the algorithm.the extrinsic probability calculations are performed as shown in [2].The notation j ( <= j <= m) of all checks in column i ( <= i <= n) which have value one. α ij is another normalized value chosen satisfying e + e =. i i e = α p y i j j i j i i ( j ) ' e i = α j p j y i j ' Where e + e = i i i i ( j )..(3.) The quantities obtained in the vertical step are used to determine a temporary estimate C^. The computation can be terminated, if the condition of the product of the estimate and the parity check matrix is zero. Otherwise, the algorithm iterates back to the horizontal step.if, after some predecided maximum number of iterations (e.g,2 or 25), there is no valid decoding,meaning a decoding failure has occurred and the same is declared Bit Flipping Algorithm The key behind this algorithm as the name suggests, is to flip the minimum number of bits till all the parity checks are satisfied. Assume that each Bit node begins with a value of either zero or

14 75 one. In every iteration, the Bit node decides based on this computation either to flip its value or to keep it as it is. When a large number of such neighboring check equations are not satisfied, the Bit node takes a decision to flip its value based on the assumption that the Bit node value (which is in error) has the maximum number of unsatisfied check equations. This process is generally easier, when H is of low density, i.e., when only a small number of bits are involved in each check equation and that each bit is involved in only a small number of check equations Majority Logic Decoding Algorithm The regular LDPC code can be one-step majority logic decodable when the concept of the Projective geometry (discussed in the next chapter) is applied to the LDPC codes using points and lines similar to nodes and edges and their interconnection done in the bipartite graph. Since this concept leads to cyclic approach, this can be easily encoded using linear feedback shift register (its corresponding generator polynomial implementation is also simple). For the proposed p_adic extension of PG based LDPC codes, this seems to the best technique as reported by Shu Lin and Costello in [2]. A separate chapter on majority logic decoding is added to discuss this technique.

15 CONCLUSIONS The LDPC codes approach Shannon limit.due to the low density of the non zero entries of parity check matrix, the computational overhead is minimized and hence they are very popular in wireless applications. The LDPC codes construction, decoding by different methods are studied. The girth of the cycles in tanner graph affects the decoding performance of these codes. In order to increase the girth,concept of Projective Geometry is applied and is called Projective Geometry based LDPC (PG-LDPC) codes, discussed in the next chapter.

An Introduction to Low Density Parity Check (LDPC) Codes

An Introduction to Low Density Parity Check (LDPC) Codes Jian Sun jian@csee.wvu.edu Wireless Communication Research Laboratory Lane Dept. of Comp. Sci. and Elec. Engr. West Virginia University June 3,