Iterative Decoding Performance Bounds for LDPC Codes on Noisy Channes arxiv:cs/060700v1 [cs.it] 6 Ju 006 Chun-Hao Hsu and Achieas Anastasopouos Eectrica Engineering and Computer Science Department University of Michigan, Ann Arbor, MI, 48109-1 emai: {chhsu, anastas}@umich.edu Abstract The asymptotic iterative decoding performances of ow-density parity-check (LDPC) codes using min-sum (MS) and sum-product (SP) decoding agorithms on memoryess binary-input output-symmetric (MBIOS) channes are anayzed in this paper. For MS decoding, the anaysis is done by upper bounding the bit error probabiity of the root bit of a tree code by the sequence error probabiity of a subcode of the tree code assuming the transmission of the azero codeword. The resut is a recursive upper bound on the bit error probabiity after each iteration. For SP decoding, we derive a tighter recursivey determined upper bound on the bit error probabiity by tracking the evoution of the Bhattacharryya parameters associated with the decoding messages after each iteration. This recursive upper bound recovers the density evoution equation of LDPC codes on the binary erasure channe (BEC) with inequaities satisfied with equaities. A significant impication of this resut is that the performance of LDPC codes under SP decoding on the BEC is a ower bound of the performance on a MBIOS channes with the same Bhattacharryya parameter. A resuts hod for the more genera muti-edge type LDPC codes. 1 Introduction Low-density parity-check (LDPC) codes, first introduced by Gaager [1], have been shown to be powerfu channe codes under iterative decoding by numerous simuations in the iterature. However, the oopy graphica structure of the LDPC codes generay prohibits their iterative decoding performance from being exacty anayzed. This probem has been partiay soved in [] by considering the asymptotic average performance of an ensembe of LDPC codes when their codeword ength goes to infinity. In particuar, the authors in [] proved that the performance of each code in the ensembe asymptoticay approaches the average performance of the ensembe with oop-free oca structure within finite iterations as the codeword ength goes to infinity. This resut gave birth to the density evoution (DE) method proposed in [], and was used to provide the exact asymptotic performance of LDPC codes after an arbitrary number of iterations on memoryess binary-input output-symmetric (MBIOS) channes. A major drawback of the DE method is that the evoved densities in genera require an infinite dimensiona description. Therefore, DE is not suitabe for the derivation of anaytica performance bounds. To sove this probem, severa approaches have been proposed that track the evoution
of the densities projected to some specific finite dimensiona space. These incude the Gaussian approximation [3], the extrinsic information transfer (EXIT) chart [4], and the generaized EXIT (GEXIT) [5] chart methods. Unfortunatey, since the EXIT and GEXIT chart methods sti require numerica cacuations, and the Gaussian approximation does not impy any upper or ower bounds on the exact performance, these resuts sti can not be used for guaranteeing the code performance. In [6], the authors propose to map the evoved densities to Bhattacharryya parameters, which are further used to bound the bit error probabiity of the LDPC codes under sum-product (SP) decoding [7] on MBIOS channes. It turns out that an upper bound on the Bhattacharryya parameters can be obtained by a recursion invoving ony one-dimensiona rea numbers, so the whoe resut can be used to determine a guaranteed decoding capabiity for the LDPC codes. In this paper, we improve this resut of [6] in two different directions. First, we show that the same recursivey determined upper bound on the bit error probabiity of the LDPC codes not ony hods for the SP decoding, but aso hods for the min-sum (MS) decoding [7]. This resut is attained by upper bounding the probabiity of error of the root bit of a tree code by a sequence error probabiity of a subcode of the tree code, and then using the union bound. Therefore, the whoe proof does not invove the Bhattacharryya parameters. Then, we turn our attention to SP decoding and derive a new recursive upper bound on the Bhattacharryya parameter. Compared with the recursion in [6], our recursion gives significanty tighter upper bounds in the sense that the upper bounds become exact and recover the one-dimensiona density evoution equation on the BEC. Expoiting the resembance of our recursion to the DE equation on the BEC, we prove that LDPC codes under SP decoding on an MBIOS channe with Bhattacharryya parameter D aways have an equa or better performance than that on the BEC with erasure probabiity ǫ = D. Since an abundance of studies have resuted in LDPC codes working reiaby on any given BEC, we can aways find codes working reiaby on any given MBIOS channe according to this resut. Note aso that, due the nature of the proofs of the main emmas, this connection between BEC and MBIOS channes is aso true for the genera famiy of muti-edge type LDPC codes [8], incuding the irreguar repeat-accumuate (IRA) codes [9,10] and the ow-density parity-check and generator matrix (LDPC-GM) codes [11]. The remaining of this paper is structured as foows. In Section, we review the preiminary background on the Bhattacharryya parameters, MBIOS channes, and the asymptotic anaysis of LDPC codes. Then, we present our asymptotic performance anaysis of LDPC codes on MBIOS channes under MS and SP decoding in Section 3 and 4, respectivey. Finay, we concude this paper in Section 5. Preiminaries Let the random variabe Y from the aphabet Y be an observation of the binary variabe X from the aphabet X {0, 1}. Suppose Y is a discrete random variabe, and its statistics are competey characterized by the conditiona probabiity mass function p(y x). In the two-hypothesis testing probem of decoding X given a reaization y of Y, the Bhattacharyya parameter B is defined as B y Y p(y 0)p(y 1), and it is shown in [1, Theorem 7.5] that B is an upper bound on the maximum-ikeihood (ML) decoding error probabiity. Notice that a our resuts hod aso when Y is a continuous random variabe. In which case, we simpy treat p(y x) as a conditiona probabiity density function and the summation sign as an integration sign. However, for convenience, we wi assume Y to be a discrete random variabe whenever we refer to this kind of scenarios throughout
this paper. A property of B foowing from the Cauchy-Schwarz inequaity is that 0 B = ( )( ) p(y 0)p(y 1) p(y 0) p(y 1) = 1 (1) y Y y Y y Y If x and y are the input and output symbos, respectivey, of an MBIOS channe, then we may assume that Y is the set of rea numbers R and further have the symmetry condition: p(y 0) = p( y 1), y R () To anayze the asymptotic average iterative decoding performance of a (λ, ρ) irreguar LDPC ensembe when the codeword ength goes to infinity, where λ(x) λ ix i 1, and ρ(x) ρ ix i 1 are the standard variabe and check node degree distributions, respectivey, from the edge perspective as defined in [13], it is shown in [] that we can as we consider the cyce-free case. In this case, the probabiity of bit error after decoding iterations is the probabiity of decoding error of the root bit on the tree of + 1 (variabe node) eves whose construction is dictated by the degree distributions (λ, ρ) as in [13]. Due to the symmetry condition of MBIOS channes, we can aso assume without oss of generaity that the a-zero codeword is transmitted. Hence, in the foowing two sections, we wi anayze the asymptotic MS and SP iterative decoding performance of the (λ, ρ) LDPC ensembe on MBIOS channes by considering the corresponding tree codes and assuming the transmission of the a-zero codeword. 3 Min-Sum Decoding Performance Anaysis Consider an arbitrary binary tree code whose codebook is C = {c 0 = 0,c 1,...,c M }, where c i = (c i1, c i,..., c in ) is a codeword of ength n for a i, and c 0 is the a-zero codeword. Let c i1 be the root bit of this tree for a i, x = (x 1, x,...,x n ) the transmitted codeword, and y = (y 1, y,...,y n ) the received sequence from an MBIOS channe with conditiona probabiity mass function p(y x). When min-sum (MS) decoding is performed on this tree, it essentiay performs maximum ikeihood sequence detection (MLSqD) on the whoe sequence to produce an estimate ĉ = (ĉ 1, ĉ,..., ĉ n ). Therefore, if we define the decision region for the codeword c i as U i k i U ik, (3) where U ik { y R n p(y j c ij ) } p(y j c kj ), (4) then the probabiity of the root bit being in error under MS decoding assuming x = 0 is ( = Pr(ĉ 1 = 1 x = c 0 ) = Pr y ) U i x = c 0. (5) P MS b i,c i1 =1 We woud ike to find a compact representation of the set U U i, (6) i,c i1 =1
or at east a superset of U so that we can bound Pb MS from above. Note that such an upper bound is aso a vaid upper bound on the probabiity of root bit error Pb SP when the SP decoding is performed on the tree since the SP decoding essentiay performs optima maximum ikeihood decoding for each bit. Define the subcode C r of C as the set of codewords in C such that the root bit is 1, and each check node has exacty one chid variabe node of vaue 1 if it has a parent node of vaue 1, and zero chid nodes of vaue 1 if it has a parent node of vaue 0. We show how we can use this reduced codebook C r to characterize U in the foowing emma. Lemma 1 U i,c i C r U i0. Proof: Given any y U = i,c i1 =1 U i, there exists a k such that y U k and c k1 = 1. To proceed with our proof, we first carry out the foowing abeing procedure. 1. At the initia state, the root bit is abeed as survivor, and a the other variabe nodes are unabeed. Note that in this abeing procedure, the survivor variabe nodes wi aways have vaue 1 in c k. We first consider the check nodes at the topmost eve of the tree.. For every check node h at this eve whose parent node is abeed as survivor, it must have at east one chid variabe node with vaue 1 in c k since survivor nodes aways have vaue 1 in c k. Choose an arbitrary chid variabe node of h with vaue 1 in c k, and abe it as survivor. Then, abe the subtrees emanating from the other unabeed chid variabe nodes of h as dropped. 3. If there are no check nodes at the next ower eve of the tree, stop. Otherwise, move to the check nodes at the next ower eve and go back to. As we can see after this abeing procedure, the check nodes with a survivor parent node a have exacty one survivor chid node, and the ones with a dropped parent node have purey dropped chid nodes. Therefore, if we et { 0 for a dropped bits c m = (7) 1 = c k for a survivor bits then we have c m C r C. In the foowing, we woud ike to prove that y U m0, which competes the proof. Let { c k for a dropped bits c = (8) 0 for a survivor bits i.e., et c be the bitwise XOR of c k and c m. Then, since c k and c m are vaid codewords in C, so is c. Moreover, since y U k impies y U k, we have from (4) that p(y j c kj ) p(y j c j ) p(y j c kj ) p(y j c j ) which proves that y U m0 as desired. a survivor bits j a survivor bits j p(y j c mj ) p(y j c mj ) a survivor bits j a survivor bits j p(y j 0) p(y j 0) (9)
This emma shows that we can upper bound Pb MS by the probabiity of MLSqD error Ps MLSqD on the reduced codebook C r assuming that the a-zero codeword is transmitted. One way to proceed from here is to use union bound and the fact (see [1, Theorem 7.5] for a proof) that Pr(y U i0 x = c 0 ) D w(c i), i (10) where w(c i ) denotes the Hamming weight of c i, and D is the Bhattacharryya parameter associated with the MBIOS channe p(y x), to further upper bound Ps MLSqD. For this purpose, we woud ike to introduce the weight enumerator N (x) of the reduced codebook C of the tree code G of eve +1 associated with a randomy drawn code from the (λ, ρ) LDPC ensembe. Let A i be the number of codewords of weight i in C. We define N (x) by N (x) A ix i. Moreover, et N (x) denote the expected vaue of N (x) averaged over the whoe (λ, ρ) LDPC ensembe. We have the foowing emma. Lemma N 0 (x) = x and where ρ (x) denotes the derivative of ρ(x). N (x) = λ(ρ (1)N 1 (x)), 1 (11) Proof: It is obvious that N 0 (x) = x. To prove the recursion for N (x), first consider the subtree emanating from the ith check node h i immediatey beow the root bit. Let Z (i) (x) denote the weight enumerator of the reduced codebook of this subtree. Since, the root bit is 1 for a codewords in the reduced codebook C, there is exacty one chid subtree with root 1 emanating from c for a codewords in C. Therefore, if c has degree d c, then we have Z (i) (x) = (d c 1)N 1 (x) Z (i) (x) = Simiary, if the root bit has degree d v, then we have N (x) = d c 1 Z (i) (x) N (x) = d c 1 (i 1)N 1 (x)ρ i = ρ (1)N 1 (x). (1) Z (i) (x)λ j = i=j d c 1 Z (i) (x)λ j = λ(z (1) (x)) (13) where the second equaity foows from the fact that the subtrees emanating from different h i s are generated independenty, and the third equaity foows from the fact that Z (i) (x) does not depend on i as shown in (1). Combining (1) and (13), the emma is proved. Using this emma and the union bound on Ps MLSqD, we have the foowing theorem. Theorem 1 Given any (λ, ρ) LDPC ensembe, et P MS and P SP be its asymptotic (as the codeword ength approaches infinity) average bit error probabiity after iterations under MS and SP decoding, respectivey, on an MBIOS channe with Bhattacharryya parameter D. If we define the sequence {z } =0 by z 0 = D, and z = λ(ρ (1)z 1 ), for a 1, then we have P SP P MS z, for a 0. Proof: From Lemma we see that z = N (D) for a. Now, the emma foows from the union bound on Ps MLSqD as discussed above. A simiar resut to Theorem 1 is estabished in [6, Lemma 1]. However, Theorem 1 differs from [6, Lemma 1] in two aspects. First, Theorem 1 hods for both MS and SP decoding whie [6, Lemma 1] hods ony for the SP decoding. Second, we did not keep track of the evoution of the Bhattacharryya parameters, which are used in [6] to bound P SP for a. In the next section when we focus ony on the SP decoding, we wi keep track of the evoution of the Bhattacharryya parameters as in [6, Lemma 1] to estabish a stronger resut than [6, Lemma 1].
4 Sum-Product Decoding Performance Anaysis Let m = og[p(y 0)/p(y 1)] be the og-ikeihood ratio (LLR) of x given the observation y, where the statistics of y are reated to x by the conditiona probabiity mass function p(y x). Assuming x = 0, the Bhattacharyya parameter associated with p(y x) can be expressed as B = y p(y 0) p(y 1) p(y 0) = E [ e Now, consider the SP decoding on a tree code used on an MBIOS channe. As shown in [], a the SP decoding messages can be represented by the LLR s and satisfy the symmetry condition (). Assuming that the a-zero codeword is transmitted, we have the foowing emma describing the reationship between Bhattacharryya parameters associated with the incoming and outgoing messages of a variabe node. Lemma 3 Let v be a variabe node of degree d v. Furthermore, et m 0 be the incoming message from the channe, m v the outgoing message on an edge, and m 1, m,...,m dv 1 the incoming messages from the other edges. Assuming a the incoming messages are independent with each other, we have [ ] d v 1 [ E e mv = E Proof: It foows directy from the fact that under SP decoding [], m v = d v 1 i=0 m i, and the independence of the incoming messages. A simiar reationship for a check node is aso derived. However, since the derivation is more compicated than the one for a variabe node, we first consider the simpe case where the check node is of degree 3. Lemma 4 Let c be a check node of degree 3. Furthermore, et m c be the outgoing message on an edge, and m 1 and m the incoming messages from the other edges. Assuming a the incoming messages are independent with each other, we have [ E e mc ] 1 i=0 e m i ( [ 1 E Proof: Under SP decoding, we have from [] that ( 1 + m c =og tanh ) m i 1 tanh = og m i ] e m i m] ]) ( 1 + e m 1 ) e m e m 1 + e m Let p 1 and p be the conditiona probabiity mass functions associated with m 1 and m, respectivey, such that (14) (15) (16) (17) m i = og p i(y 0) p i (y 1) = og p i(y 0), i = 1, (18) p i ( y 0) Writing p i (y 0) as p i (y) for a i for conciseness, we have on the eft hand side of (16) that [ ] [ ] E e mc e m 1 + e m = E 1 + e m 1 e m = p 1 ( y 1 )p (y ) + p 1 (y 1 )p ( y ) p 1 (y 1 )p (y ) p y 1,y 1 (y 1 )p (y ) + p 1 ( y 1 )p ( y ) (19)
Since for a y i such that p i (y i ) p i ( y i ), there exists a y i = y i such that p i (y i) p i ( y i), and vice versa for a i, it foows that [ ] E e mc = p 1 ( y 1 )p (y ) + p 1 (y 1 )p ( y ) [p 1 (y 1 )p (y ) + p 1 ( y 1 )p ( y )] p 1 (y 1 )p (y ) + p 1 ( y 1 )p ( y ) = y i : p i ( y i ) p i (y i ) 1, + [p 1 (y 1 )p ( y ) + p 1 ( y 1 )p (y )] y i : p i ( y i ) p i (y i ) 1,, p 1 (y 1 )p (y ) p 1 (y 1 )p (y ) + p 1 ( y 1 )p ( y ) p 1 ( y 1 )p (y ) + p 1 (y 1 )p ( y ) ][ p1 ( y 1 ) p 1 (y 1 )p (y ) p 1 (y 1 ) + p ] ( y ) p (y ) [ 1 + p 1( y 1 )p ( y ) Simiary, the right hand side of (16) can be written as foows. ( [ ]) [ ( )( )] 1 1 E e m i =E 1 1 e m 1 1 e m = = [p (y ) + p ( y )] p1 (y 1 )p 1 ( y 1 ) y i : p i ( y i ) p i (y i ) 1,, y i : p i ( y i ) (0) + [p 1 (y 1 ) + p 1 ( y 1 )] p (y )p ( y ) 4 p 1 (y 1 )p 1 ( y 1 )p (y )p ( y ) [ ( 1 + p ) ( y ) p 1 ( y 1 ) p (y ) p 1 (y 1 ) + p 1 (y 1 )p (y ) p i (y i ) 1,, ( 1 + p 1( y 1 ) p 1 (y 1 ) ) p ( y ) p (y ) ] p 1 ( y 1 )p ( y ) p 1 (y 1 )p (y ) where the first equaity foows from the independence of m 1 and m. For any given y i, et a i p i ( y i )/p i (y i ), for i = 1,. Then, we see from (0) and (1) that it is sufficient to prove (1 + a1 a )(a 1 + a ) (1 + a ) a 1 + (1 + a 1 ) a a 1 a () for a 0 a i 1, for a i. Now, since [(1 + a ) a 1 + (1 + a 1 ) a a 1 a ] (1 + a 1 a )(a 1 + a ) = a 1 a [4 a 1 a + (1 + a 1 )(1 + a ) (1 + a 1 ) a (1 + a ) a 1 ] = a 1 a (1 a 1 ) (1 a ) 0 (3) () is true and the emma is proved. Armed with the previous emma, we can proceed to the genera case where the check node has degree d c by an induction argument. Coroary 1 Let c be a check node of degree d c. Furthermore, et m c be the outgoing message on an edge, and m 1, m,...,m dc 1 the incoming messages from the other edges. Assuming a the incoming messages are independent with each other, we have [ E e mc ] 1 d c 1 ( [ 1 E e m i ]) (1) (4)
Proof: We can aways first take m 1 and m to form a temporary outgoing message m of a check node. Then take m and m 3 to form another temporary outgoing message m 3. By repeatedy appying this procedure, we wi get the desired outgoing message in d c steps. Since for each of the above steps, Lemma 4 hods, and 0 E [ ] e m 1, for a LLR m as shown in (1), we have [ ] ( [ ]) ( ]) E e mc 1 1 E e m dc 1 1 E [e m dc ( [ ])( [ ]) ( ]) 1 1 E e m dc 1 1 E e m dc 1 E [e m dc 3 1 ( [ 1 E e m dc 1 ])( [ 1 E e m dc ]) ( [ ]) 1 E e m 1 (5) Hence the coroary is true. For a tree code used on an MBIOS channe under SP decoding, a the incoming messages to the variabe and check nodes are independent with each other. Hence, Lemma 3 and Coroary 1 can be used to characterize the evoution of the Bhattacharryya parameters associated with the messages after processes of the variabe and check nodes on a tree, and impy the foowing theorem. Theorem For the (λ, ρ) LDPC ensembe used on an MBIOS channe with conditiona probabiity mass function q(y x), the Bhattacharyya parameter B associated with the outgoing message of any variabe node after the th decoding iteration asymptoticay satisfies where D y q(y 0)q(y 1), and B0 = D. B Dλ (1 ρ (1 B 1 )) (6) Proof: As discussed in Section, the Bhattacharyya parameter associated with the outgoing message of any variabe node after the th decoding iteration is, asymptoticay as the codeword ength goes to infinity, the one of the root variabe node of a tree of eve +1. Since a the variabe and check nodes in the tree have exacty the same degree distributions λ and ρ, respectivey, and the channe is memoryess, a the incoming messages from chid nodes to a parent node are independent and identicay distributed. Hence, if we et v be a variabe node in the tree, d v a random variabe denoting its degree, m v its outgoing message, and m c one of its incoming message, then we have from Lemma 3 that E [ e mv ] = E [ ] e mv d v = i λ i = [ E e mc ] (i 1) ( [ λi = λ E Simiary, if we et c be a check node in the tree, d c a random variabe denoting its degree, m c its outgoing message, and m v one of its incoming message, then we have from Coroary 1 that [ ] [ ] E e mc = E e mc d c = i =1 ρ ρ i [ ( [ ]) ] (i 1) 1 1 E e mv ρ i ( [ ]) 1 E e mv where the ast equaity foows from the fact that ρ(1) = 1. Combining (7), (8), and the fact that λ is a monotonicay increasing function (since a λ i s are nonnegative), the theorem is proved. e mc ]) (7) (8)
Notice that on the binary erasure channe (BEC) of erasure probabiity ǫ, the Bhattacharryya parameter of this channe is aso ǫ. Hence, (6) is satisfied with equaity, and recovers the weknown DE equation x = ǫλ (1 ρ (1 x 1 )), x 0 = ǫ (9) where x is the bit erasure probabiity after the th iteration on the BEC. Using this fact, we have the foowing coroary. Coroary For any (λ, ρ) LDPC ensembe, its asymptotic SP decoding performance on the MBIOS channe with Bhattacharryya parameter D is aways better than its asymptotic SP decoding performance on the BEC with erasure probabiity ǫ D. Proof: From (6) and (9), we have x B for a 0 when ǫ D. Since B is an upper bound on the bit error probabiity under SP decoding after iterations on the MBIOS channe whose Bhattacharyya parameter is equa to D, the coroary is proved. Notice that, Lemma 3 and Coroary 1 can aso be used in the more genera famiy of mutiedge type LDPC codes [8], incuding the irreguar repeat-accumuate (IRA) codes [9, 10] and the ow-density parity-check and generator matrix (LDPC-GM) codes [11], to produce simiar resuts. Hence, Coroary is not restricted to the irreguar LDPC codes, but aso hods for the genera muti-edge type LDPC codes. 5 Concusion In this paper, we anayze the asymptotic performance of LDPC codes under MS and SP decoding on MBIOS channes. This is done by upper bounding the bit error probabiity of the root bit of the tree code associated with the (λ, ρ) LDPC ensembe assuming the a-zero codeword is transmitted. When MS decoding is performed on this tree code, we upper bound the probabiity of the root bit being in error by the probabiity of sequence error under ML decoding of a subcode of the tree code. A recursive equation describing the evoution of the weight enumerator of this subcode after each iteration is then derived and used in a union bound to bound the ML decoded sequence error of this subcode. As a resut, we obtain a recursive upper bound on the bit error probabiity as a function of the number iterations for the LDPC codes under MS decoding on MBIOS channes. Note that these upper bounds are aso upper bounds for the SP decoding since SP decoding is optima on the bit error probabiity for tree codes. This resut is very simiar to [6, Lemma 1] with the difference being that we estabish it not ony for the SP decoding, but aso for the MS decoding, and that we obtain it via a totay different approach. When SP decoding performance is considered, we derive a recursive upper bound on the Bhattacharryya parameter associated with the outgoing message of the root bit as a function of the number of iterations. More significanty, this recursion recovers the DE equation on the BEC for LDPC codes with the upper bound being an exact equaity. This further impies that the SP decoding performance of LDPC codes on the BEC can serve as a ower bound of the performance on a MBIOS channes with the same Bhattacharryya parameter. This resut is aso true for the more genera muti-edge type LDPC codes, incuding IRA and LDPC-GM codes, since the main ingredient in the proof, i.e., Lemma 3 and Coroary 1, can aso be utiized for these codes.
References [1] R. G. Gaager, Low density parity check codes, IEEE Trans. Information Theory, vo. 8, pp. 1 8, Jan. 196. [] T. J. Richardson and R. L. Urbanke, The capacity of ow-density parity-check codes under message-passing decoding, IEEE Trans. Information Theory, vo. 47, no., pp. 599 618, Feb. 001. [3] T. J. Richardson S.-Y. Chung and R. L. Urbanke, Anaysis of sum-product decoding of ow-density parity-check codes using a Gaussian approximation, IEEE Trans. Information Theory, vo. 47, no., pp. 657 670, Feb. 001. [4] S. ten Brink, Convergence behavior of iterativey decoded parae concatenated codes, IEEE Trans. Communications, vo. 49, no. 10, pp. 177 1737, Oct. 001. [5] C. Measson, A. Montanari, and R. Urbanke, Why we can not surpass capacity: The matching condition, in Proc. Aerton Conf. Commun., Contro, Comp., Monticeo, IL, Sept. 005. [6] M. Lentmaier, D. Truhachev, K. Zigangirov, and D. Costeo, An anaysis of the bock error probabiity performance of iterative decoding, IEEE Trans. Information Theory, vo. 51, no. 11, pp. 3834 3855, Nov. 005. [7] N. Wiberg, Codes and Decoding on Genera Graphs, Ph.D. thesis, Linköping University, Linköping, Sweden, 1996. [8] T. Richardson and R. Urbanke, Muti-edge type dpc codes, 004, [Onine]. Avaiabe: http://thcwww.epf.ch/papers/mutiedge.ps. [9] H. Jin, A. Khandekar, and R. J. McEiece, Irreguar repeat-accumuate codes, in Proc. Internationa Symposium on Turbo Codes and Reated Topics, Brest, France, Sept. 000, pp. 1 8. [10] H. D. Pfister, I. Sason, and R. Urbanke, Capacity-achieving ensembes for the binary erasure channe with bounded compexity, IEEE Trans. Information Theory, vo. 51, no. 7, pp. 35 379, Juy 005. [11] C.-H. Hsu and A. Anastasopouos, Capacity-achieving codes with bounded graphica compexity on noisy channes, in Proc. Aerton Conf. Commun., Contro, Comp., Monticeo, IL, Sept. 005. [1] R. J. McEiece, The Theory of Information and Coding, nd ed., Cambridge University Press, Cambridge, UK, 00. [13] T. J. Richardson, M. A. Shokroahi, and R. L. Urbanke, Design of capacity-approaching irreguar ow-density parity-check codes, IEEE Trans. Information Theory, vo. 47, no., pp. 619 637, Feb. 001.