Forty-Seventh Annual Allerton Conference Allerton House, UIUC, Illinois, USA September 30 - October 2, 2009

Transcription

1 Forty-Seventh Annua Aerton Conference Aerton House, UIUC, Iinois, USA September 30 - October 2, 2009 On systematic design of universay capacity approaching rate-compatibe sequences of LDPC code ensembes over binary-input output-symmetric memoryess channes Hamid Saeedi, Hossein Pishro-Nik, and Amir H. Banihashemi Department of Eectrica and Computer Engineering, University of Massachusetts, Amherst, MA, USA Department of Systems and Computer Engineering, Careton University, Ottawa, ON, Canada {hsaeedi,pishro@ecs.umass.edu; ahashemi@sce.careton.ca} Abstract Despite tremendous amount of research on the design of Low-Density Parity-Check (LDPC) codes with beief propagation decoding over different types of Binary-Input Output-Symmetric Memoryess (BIOSM) channes, most resuts on this topic are based on numerica methods and optimization which do not provide much insight into the design process. In particuar, systematic design of provaby capacity achieving sequences of LDPC code ensembes over the genera cass of BIOSM channes, has remained a fundamenta open probem. For the case of the Binary Erasure channe, expicit construction of capacity achieving sequences have been proposed based on a property caed the fatness condition. In this paper, we propose a systematic method to design universay capacity approaching rate-compatibe LDPC code ensembe sequences over BIOSM channes. This is achieved by interpreting the fatness condition over the BEC, as a Successive Maximization (SM) principe that is generaized to other BIOSM channes to design a sequence of capacity approaching ensembes caed the parent sequence. The SM principe is then appied to each ensembe within the parent sequence, this time to design rate-compatibe puncturing schemes. As part of our resuts, we extend the stabiity condition which was previousy derived for degree-2 variabe nodes to other variabe node degrees as we as to the case of rate-compatibe codes. Consequenty, we rigorousy prove that using the SM principe, one is abe to design universay capacity achieving rate-compatibe LDPC code ensembe sequences over the BEC. Unike the previous resuts on such schemes over the BEC in the iterature, the proposed SM approach is naturay extendabe to other BIOSM channes. The performance of the rate-compatibe schemes designed based on our systematic method is comparabe to those designed by optimization. I. INTRODUCTION Low-Density Parity-Check (LDPC) codes have received much attention in the past decade. During this period there have been great achievements in the area of designing LDPC code ensembes with Beief Propagation (BP) decoding which exhibit an asymptotic performance practicay cose to the capacity over different types of channes, incuding the genera cass of Binary-Input Output-Symmetric Memoryess (BIOSM) channes [1]-[10]. In particuar, for the Binary Erasure Channe (BEC), the performance anaysis and code design This work was supported by Nationa Science Foundation under grants ECS and CCF /09/$ IEEE 400 have been addressed in both the asymptotic regime [3]-[8] and for finite bock engths [1], [2]. In [3], [4], [5], Shokroahi et a. proposed a scheme to design sequences of LDPC code ensembes over the BEC, whose performance is proved to achieve the capacity for sufficienty arge average check and variabe node degrees. A more genera category of capacity achieving sequences over the BEC were proposed in [11], [12], [13]. Construction and anaysis of capacity achieving ensembe sequences of codes defined on graphs has aso been studied in [6], [7], [8] for the BEC. A sequence of degree distributions with rate R is said to be capacity achieving over the BEC if the threshods of the ensembes can be made arbitrariy cose to 1 R, the capacity upper bound over the BEC, as the average check and variabe node degrees tend to infinity. For BIOSM channes, it is easier to consider ensembes for a given channe parameter instead of a given rate. The resuts however are easiy extendabe to the case of fixed rate ensembes. We ca a sequence of degree distributions capacity achieving over a BIOSM channe, if the rate of the ensembes within the sequence can be made arbitrariy cose to the channe capacity whie maintaining the reiabe communication. The design of provaby capacity achieving sequences over genera BIOSM channes is sti an open probem. Another important probem of interest in LDPC codes is to design rate-compatibe LDPC code schemes. In such a scheme, starting from a given primary ensembe caed the parent code, we are interested in obtaining a set of codes with higher transmission rates, which can provide reiabe transmission when the channe condition improves, by puncturing the parent code. For rate-compatibiity, the design shoud be such that for two consecutive rates, the code with the higher rate can be constructed by puncturing the code with the ower rate. Starting from a parent code with performance cose to capacity, the important chaenge in a rate-compatibe design is to aso keep the performance of the punctured codes cose to the capacity. More specificay, if the parent code is chosen from a capacity achieving sequence, a punctured codes have to be capacity achieving as average check node degree increases. To formuate the probem mathematicay, imagine a parent code with rate R n from a capacity achieving sequence which can

2 provide reiabe transmission over a channe with parameter θ 0. Our aim is to provide reiabe transmission over a set of channes with parameters θ j, j = 1,..., J, whie increasing the rate by puncturing the parent code in a rate-compatibe fashion. For each θ j, j = 1,..., J, we need to choose a puncturing pattern that maximizes the corresponding reiabe transmission rate R n,j. Let c(θ j ) denote the capacity of the channe with parameter θ j, and assume that θ i < θ j and c(θ i ) > c(θ j ) for i > j. We ca a rate-compatibe scheme universay capacity achieving, if im n R n,j = c(θ j ) for j = 1,.., J. Anaysis and design of rate-compatibe LDPC codes have been addressed asymptoticay in [14]-[18] and for finite bock engths in [19]-[22]. It is worth mentioning here that Raptor codes [23] can aso achieve the capacity of the BEC at severa rates but in a different framework than puncturing. Unike the BEC for which amost a aspects of conventiona and rate-compatibe codes have been anayticay investigated, for the genera famiy of BIOSM channes, the contributions are mosty based on numerica methods and optimization. This usuay provides itte insight into the design method. In this respect, a fundamenta open probem is to prove the existence of capacity achieving sequences of LDPC codes with BP decoding over BIOSM channes as we as to systematicay construct such sequences. This can be seen as a sub-probem as we as a buiding bock for the more genera probem of designing universay capacity achieving rate-compatibe LDPC coding schemes. The anaytica resuts on this topic are very imited in the iterature. In [9], it has been shown that capacity approaching LDPC codes over BIOSM channes can be designed using optimization 1. Severa important anaytica properties incuding the so-caed stabiity condition have been proven for BIOSM channes in [9], [10]. It has been shown in [24] that for an ensembe with a rate cose to capacity, variabe node and check node Generaized Extrinsic Information Transfer (GEXIT) curves have to satisfy a socaed matching condition. In [25], severa bounds have been derived for LDPC codes that are universay vaid over any BIOSM channe. It has been shown in [14], [16] that there is an upper bound on the puncturing ratio of LDPC codes over BIOSM channes, above which the code can not provide reiabe transmission for any channe parameter. Moreover, it has been shown that over the BEC, the random puncturing maintains the ratio of rate to capacity at the same vaue as that of the parent code. Severa important bounds on the performance of punctured LDPC codes have been derived in [18]. For the case of maximum-ikeihood decoding, capacity achieving codes have been designed based on puncturing in [17]. Among important resuts on the optimization-based design of rate-compatibe codes over BIOSM channes, we can mention [15] for the asymptotic regime and [19], [20], [21] for finite bock engths. In this paper, we systematicay design sequences of uni- 1 We distinguish between capacity approaching and capacity achieving sequences. The former term is used when the performance of the ensembe sequence can be shown (probaby numericay) to approach capacity without any guarantee to achieve it. The atter term is used if the performance provaby tends to capacity as the average node degrees tend to infinity. versay capacity approaching rate-compatibe LDPC code ensembes over BIOSM channes. We then provide some evidence suggesting that the designed sequences coud in fact be universay capacity achieving. Starting from the conventiona (unpunctured) case, we extend some of the properties of capacity achieving sequences over the BEC [11], to BIOSM channes. Among such properties, ony the stabiity condition [9] has been shown to be extendabe to BIOSM channes other than the BEC. We wi anayze the case where the stabiity condition is satisfied with equaity, i.e; the fraction of degree 2 edges (λ 2 ) is set equa to its upper bound, and show that this imposes an upper bound on the fraction of degree 3 edges (λ 3 ). Using a simiar approach for the other degrees, we propose Successive Maximization (SM) of λ i vaues as a systematic approach to design a sequence of LDPC code ensembes with performance approaching the capacity as the average check node degree increases. We then conjecture that such sequences might in fact be capacity achieving over BIOSM channes. For the rate-compatibe LDPC codes on BIOSM channes, we first prove a property simiar to the stabiity condition [9]. We show that for a given parent code (which provides reiabe transmission over a channe with parameter θ), there is an upper bound for the fraction of punctured degree-2 variabe nodes (Π 2 ) above which the probabiity of error of the punctured code is bounded away from zero and beow which the probabiity of error tends to zero if it is made sufficienty sma. We then consider the specia case of the BEC and show that simiar upper bounds can be obtained for variabe nodes of a degrees in addition to degree-2 nodes. Using such upper bounds, we prove that appying the SM principe resuts in a universay capacity achieving rate-compatibe scheme over the BEC. Moreover, for such a scheme, if puncturing fractions are used to puncture the parent sequence (λ n,ρ n ) over the channe with parameter θ j, where i is the variabe node degree, the vaues of Π n,j i are independent of n. This resut is consistent with the one obtained in [16], [14] based on a competey different approach. We then extend a weaker version of the resuts on the BEC to genera BIOSM channes. Assuming that simiar upper bounds on the puncturing ratios of other variabe nodes (in addition to the upper bound on Π 2 ) exist, we show that the SM principe can be appied to puncturing fractions of variabe nodes to systematicay design a coding scheme whose performance universay approaches the capacity in a rate-compatibe fashion. This proposes a significanty different approach than the existing optimizationbased methods in the iterature. Our numerica resuts indicate that if the parent ensembe is chosen from the capacity approaching sequences designed based on the SM principe, the performance of the resuting rate-compatibe schemes is simiar to that of the existing optimization-based resuts in the iterature. Moreover, we show that for a sequence of parent code ensembes (λ n,ρ n ) designed based on the SM principe, Π n,j i for degree 2 variabe nodes (i = 2) are independent from the parent ensembe (n) and ony depend on the origina channe parameter (θ 0 ) and the one for which the puncturing pattern is designed (θ j ). Our numerica resuts suggest that this property may in fact hod for other vaues of i. The importance of this property is that 401 the vaues of puncturing fractions Π n,j i

3 for a given channe parameter θ j, the computed vaues of Π i can universay be appied to any ensembe designed based on the SM method for a given origina channe parameter θ 0 with an arbitrary check node distribution. The paper organization is as foows. The next section is devoted to notations and some definitions. In Section III, after a short review on the construction methods of capacity achieving sequences over the BEC, we expain our approach (Successive Maximization) to devise capacity approaching sequences for other channes. In Section IV, we focuss on the puncturing of a given ensembe within a sequence that is designed based on the methodoogy of previous sections. We aso provide some properties of rate-compatibe codes for the BEC and BIOSM channes. Moreover, we show that a simiar SM principe to that of Sections III can be used to devise a universay capacity approaching rate-compatibe scheme. In Section V, we show exampes of our designs and Section VI concudes the paper. The proof of theorems have not been presented due to ack of space and can be found in [26]. II. DEFINITIONS AND NOTATIONS In this section we present some definitions and properties which wi be frequenty used throughout the paper. We mainy foow the notations and definitions of [10], [16]. As our focus is on symmetric channes and a BP decoder, throughout the paper, without oss of generaity, we assume that the aone code word is transmitted. Moreover, we assume that the messages in the BP agorithm are in the og-ikeihood ratio domain. We represent a (λ, ρ) LDPC code ensembe with its edge-based check and variabe node degree distributions as ρ(x) = D c ρ ix i 1 and λ(x) = D v λ ix i 1, with constraints D c ρ i = 1 and D v λ i = 1, (1) where the coefficient of x i represents the fraction of edges connected to the nodes of degree i + 1, and D v and D c represent the maximum variabe node degree and the maximum check node degree, respectivey. Average check node and variabe node degrees are given by: d c = 1/( D c ρ i/i) and d v = 1/( D v λ i/i), respectivey. The code rate R satisfies R = 1 d v /d c. (2) We aso define node-based degree distributions as ρ(x) = Dc ρ ix i 1 and λ(x) = D v λ ix i 1, with constraints D c ρ i = 1 and D v λ i = 1, (3) where the coefficient of x i represents the fraction of nodes having degree i + 1. We represent a BIOSM channe with parameter θ by C(θ) and define c(θ) as the Shannon capacity of that channe. We aso assume that the channe is physicay degraded when θ increases. For a sequence of degree distributions (λ n (x), ρ n (x)), λ n i and ρn i indicate the ith coefficient of the nth member of the sequence for variabe node and check node degree distributions, respectivey. Simiar to [5], we imit 402 ourseves to check node degree distributions for which T i s, the Tayor series expansion coefficients of 1 ρ 1 (1 x) around x = 0, are positive. For exampe, check reguar ensembes exhibit such a property. Consider now the density evoution in the beief propagation agorithm for the channe C(θ) where we track the evoution of the initia channe density P 0 throughout iterations. Based on [9], [10], Q, the outgoing density from check nodes at iteration can be written as Q = Γ 1 ρ(γ(p 1 )), (4) where P 1 is the density from iteration 1 entering the check nodes and Γ is the check node operator defined in [9], [10]. Aso, P the outgoing density from variabe nodes at iteration can be written as P = P 0 λ(q ), (5) where is the convoution operation, and the power of a density in variabe node and check node degree distributions has been defined in [10]. Note that there is a one-to-one mapping between density P 0 and parameter θ. We now review the foowing important definitions and theorems from [9], [10]. Let P be a symmetric density (as defined in [9]). For such a density we define P(P) and P(P) as: and P(P) = 0.5 P(P) = P(x)e ( x/2 +x/2) dx, (6) P(x)e (x/2) dx. (7) The first integra is the probabiity that the corresponding random variabe is negative. The unusua form of the integra makes it possibe to take care of the impuse densities at zero. The second integra is the Bhattacharyya constant. For any given density P, the Bhattacharyya constant tends to zero if and ony if (iff) P(P) tends to zero. Let p = P(P ) and q = P(Q ). Corresponding to (4) and (5), we then have the foowing reationships [10]: q 1 ρ(1 p 1 ), (8) p = P(P 0 )λ(q ). (9) From (8) and (9) we can see that: p P(P 0 )λ(1 ρ(1 p 1 )). (10) It is important to note that for the BEC, (8) and (10) are satisfied with equaity. Moreover, P(P 0 ) is equa to the average erasure probabiity for the BEC. The stabiity of an ensembe is defined as foows [9]. A given degree distribution (λ, ρ) is stabe iff there exists ξ > 0 such that if P(P ) < ξ then im P(P ) = 0. In that respect, it is proven in [9], [10] that if λ (0)ρ (1) > 1/P(P 0 ) then P(P ) is bounded away from zero for every and if λ (0)ρ (1) < 1/P(P 0 ), then the ensembe is stabe. We ca an ensembe (λ,ρ) convergent over C(θ), if starting from the initia density P 0, im P(P ) = 0. The threshod

4 of an ensembe over C(θ) is the supremum vaue of θ for which the ensembe is convergent. Consider now the (k+2)-tupe (λ 2,λ 3,..., λ k,d v,ρ(x);θ) which corresponds to a degree distribution (λ(x), ρ(x)) = ( k λ ix i 1 +(1 k λ i)x Dv 1,ρ(x)) over C(θ) where D v > k, and 0 λ i < 1, i {2,...,k,D v }. We ca this setting a code-channe pair. With sight negigence, we ca a code-channe pair convergent if the ensembe is convergent over the channe. III. CAPACITY ACHIEVING SEQUENCES For the moment, consider the case of the BEC. We first reca that any ensembe sequence designed in [3], [4], [5] consists of a set of ensembes with a fixed rate R. Then it is proven that the threshods associated to such ensembes can be made arbitrariy cose to θ = c 1 (R). In this paper for the sake of simpicity, we consider a sighty different case where the channe parameter θ is fixed and we design ensembes to have a variabe rate R n which can be made arbitrariy cose to c(θ). Consequenty, the definition of capacity achieving sequences can be extended to the case of fixed θ. More specificay, a sequence of degree distributions (λ n,ρ n ) is caed capacity achieving over a BIOSM channe C(θ), if the corresponding ensembes are convergent over C(θ) and if their rates R n, can be made arbitrariy cose to c(θ) for sufficienty arge average check node degrees as n tends to infinity. Concentrating again on the BEC, note that the channe parameter θ is the same as the channe probabiity of erasure. The derivation of capacity achieving sequences for the BEC proposed in [3], [4], [5] is based on the fatness condition. Based on [3], if a sequence of LDPC code ensembes satisfies the conditions beow, it can be proved to be capacity achieving. d i dx i [θλn (1 ρ n (1 x)) x] x=0 = 0, 1 i D v 2, (11) where D v, the maximum variabe node degree, is determined by θ and ρ(x) such that (see [11], [12]): Dv 1 Dv T i θ < T i. (12) If we appy the fatness condition, we have: where we define 2 λ n i = T i /θ = λ n i, 2 i Dv 1, (13) λ n i T i /P(P 0 ). (14) One can easiy verify that λ n 2 = 1/(P(P 0 )ρ (1)). In other words, the vaue of λ n 2 has been set to its maximum vaue dictated by the stabiity condition. In [11], [12], [13], we presented an aternate approach for the design of capacity achieving sequences over the BEC. In this method, the vaues of λ n i are computed based on the foowing principe: Starting from i = 2, set the vaue of λ n i to a maximum vaue λ n i such that the ensembe remains convergent for sufficienty arge D v. It is shown in [12] that the vaues of λ n i computed based on this principe are the same as those derived based on (11). In other words, for the BEC, we have λ n i = λ n i. To have an intuition of why such a process resuts in good ensembes whose rates achieve the capacity in the imit, note that based 1 on (2), maximizing the rate is equivaent to maximizing d v. 1 Based on the definition of d v, this impies that we shoud assign higher percentages to the ower degree coefficients as far as the constructed ensembe remains convergent. We remind the reader that the structure of the ensembes proposed by Shokroahi is in such a way that a variabe node degrees from 2 to D v have to be present. In [11] a super-set of such sequences has been proposed which incudes Shokroahi s sequences as a specia case. It is shown in [11] that if each ensembe within the sequence ony contains a variabe node degrees from degree 2 to k < D v and degree D v, where k is a stricty increasing function of D v (and utimatey a function of ρ(x) and θ), the sequence comprising such ensembes is aso capacity achieving 3. In this paper, we dea with such sequences which are more genera. For the case of BIOSM channes, the fatness condition can not be defined simiar to the BEC case as the density evoution equation is not in poynomia form anymore. We, however, expect that appying the new interpretation of fatness condition, i.e., obtaining a sequence of upper bounds λ n i and setting λ n i = λ n i, may resut in a systematic approach for devising capacity achieving sequences for other BIOSM channes. Note that such upper bounds have to fufi a threshod property simiar to that of λ n i over the BEC as foows: If we set λ n i = λ n i for i < k, the vaue of λ n k has to be in such a way that whie the probabiity of error is bounded away from zero for any ensembe with λ n k > λ n k, for any ensembe with λn k < λ n k, the probabiity of error has to tend to zero as the number of iterations tends to infinity, if it is made sufficienty sma, regardess of the vaue of other λ n i s (i > k). Unike the BEC case, there is no proof that such upper bounds exist for other BIOSM channes (with the exception of the bound on λ 2 ), and even if they do exist, their vaues may not be easiy obtained anayticay (except for i = 2, where we have λ n 2 = λn 2 ). In the next section, we prove the existence of a positive upper bound on λ 3 that fufis the aforementioned properties and conjecture that simiar upper bounds exist for other λ i vaues. This makes it possibe to appy the SM principe as a design too for ensembe sequences. Numerica evidence presented in Section V confirms that the resuting sequences are at east capacity approaching and may in fact be capacity achieving. Reca that the stabiity condition theorem in [9], [10] remains sient about the case where λ (0)ρ (1) is exacty equa to 1/P(P 0 ). Here, we show that when this is the case, a simiar upper bound exists for λ 3, i.e., if λ 2 = 1/(P(P 0 )ρ (1)), there exists a threshod vaue for λ 3 beow which the ensembe is convergent and above which it is not. Theorem 1: Consider the code-channe pair (λ 2,λ 3,D v,ρ(x);θ) where D v can be arbitrariy arge 2 Note that the definition of (14) wi be used for any type of BIOSM channe throughout the paper Shokroahi s sequences correspond to those of [11] with k(d v) = D v

5 and et λ U 3 = 3/(d c (1 c(θ))) (3/2)λ 2. There exists a threshod vaue λ 3 in the interva [λ 3,λ U 3 ] such that if λ 3 < λ 3, the ensembe is convergent for sufficienty arge vaue of D v and if λ 3 > λ 3, the probabiity of error is bounded away from zero regardess of the vaue of D v. We expect the resut of Theorem 1 to be generaized to λ k,k > 3, if λ i = λ i,2 i k 1. This, however, remains to be proved. IV. UNIVERSALLY CAPACITY APPROACHING RATE-COMPATIBLE LDPC CODES In previous sections, we considered sequences of degree distributions (λ n,ρ n ) and our intention was to design them such that their rates approach the capacity as n tends to infinity. In this section, we consider the probem of puncturing a degree distribution for a given n. For simpicity, we sometime drop the index n and refer to the ensembe as the parent ensembe. We use the notations (λ p,ρ) and R p for the parent ensembe and its rate, respectivey. We show the fraction of the punctured bits (variabe nodes) by Π. The resuting code rate in this case is equa to R p /(1 Π). If the puncturing is performed randomy, we refer to it as random puncturing. Otherwise, the puncturing is caed intentiona [15]. In intentiona puncturing, variabe nodes of degree i can potentiay have different puncturing fractions Π i. The overa puncturing fraction Π can then be expressed as Dv Π = Π i λ p i (15) 2 where λ punc i i = Epunc E punc. Notation. denotes the cardinaity of the set. We aso define ϕ punc as the fraction of punctured edges: ϕ punc = Epunc. E The poynomia λ un (x) and ϕ un can be defined simiary for unpunctured edges. We can now derive the density evoution equations for our setting. Simiar to the previous section, et Q be the probabiity density function of outgoing message of the check nodes at iteration. We define P punc and P un as the density at the output of the punctured and unpunctured variabe nodes, respectivey. We then have P punc P un = P punc o λ punc (Q ), = P un o λ un (Q ), (16) P = ϕ punc P punc + ϕ un P un, Q = Γ 1 (ρ(γ(p 1))), in which P punc 0 = 0 where x is the Dirac deta function at x [10]. A punctured scheme is convergent if the probabiity of error tends to zero as the number of iterations tends to infinity. Consider a sequence of degree distributions (λ n (x),ρ n (x)). Consider aso a set of channes with parameters θ j, j = 0, 1,..., J, ordered increasingy by their quaity. Now assume that the parent ensembe sequence (λ n (x),ρ n (x)) is punctured p where {λ i } is the node-based degree distribution of variabe by the set φ n,j = {Π n,j i, 2 i D v } to create higher rate nodes for the parent ensembe. ensembe sequences that are convergent over the corresponding In many situations, it is necessary to obtain more than one rate by puncturing. In this case, for a simpe impementation, the puncturing pattern shoud be in such a way that for 2 consecutive rates, the punctured code with a higher rate can be constructed by puncturing the code with the ower rate. A puncturing pattern with this property is caed ratecompatibe. Let the set of channe parameters θ j be ordered reversey by channe degradation (i.e., θ 0 is for the worst channe condition which corresponds to the parent code). For channes. This scheme is universay capacity achieving if im n R n,j = c(θ j ) for a vaues of j. A universay capacity achieving scheme is caed rate-compatibe if the puncturing patterns φ n,j are rate-compatibe for every vaue of n. We now prove an important theorem in puncturing a given degree distribution. Consider a parent ensembe (λ p, ρ) with threshod equa to θ 0. The code-channe pair (λ p,ρ; θ) is convergent for any θ θ 0. Let P 0 be the channe density any C(θ j ), consider the set Φ j = {Π j i, 2 i D v}. 4 For associated with θ. We define parameter Π 2, corresponding to a rate-compatibe scheme, we must have Π m i Π n i for any the parent code-channe pair, as: m < n and any i. In the rest of the paper, with sight abuse of anguage, we ca a puncturing scheme rate-compatibe if Π 2 = [1 P(P 0)ρ (1)λ p 2 ] these conditions are satisfied. [1 P(P 0 )]ρ (1)λ p. (17) 2 To anayze the asymptotic behavior of a punctured ensembe, we mode the puncturing of LDPC codes over a channe Note that if the pair is stabe, Π 2 0. C(θ) as the transmission of the unpunctured bits over C(θ) Theorem 2: Let (λ p,ρ) be a parent code convergent whie sending the punctured bits on an erasure channe with over C(θ) with λ p 2 0. Suppose that this code is punctured erasure probabiity of 1. Let E be the set of a edges in the graph. Aso et E punc based on the set Φ = {Π i ; i = 2,..D v } (note that C(θ) i be the set of edges in the graph which is uniquey associated with the channe density P 0 ). There are connected to the variabe nodes of degree i which are punctured. Aso et E punc be the union of sets E punc exists a threshod vaue Π 2, given by (17), such that if i. Simiary Π define E un and Ei un 2 > Π 2, then for any, P(P punc ) and P(P un ) are bounded for unpunctured edges. We define λ punc (x) = away from zero and if Π 2 < Π 2, there exists a stricty λ punc i x i 1, positive constant ξ such that if P(P punc ), P(P un ) < ξ for some, then im P(P punc ) = 0 and im P(P un ) = 0. 4 For λ i = 0, we assume Π i =

6 This property is simiar to the stabiity condition [9] for conventiona LDPC codes which provides an upper bound on the fraction of degree 2 variabe nodes. Coroary 1 (Independency of Π 2 from n for puncturing schemes with Π 2 = Π 2 ): Consider a sequence of ensembes (λ n (x),ρ n (x)) which are convergent over C(θ 0 ) and et P0 0 be the associated channe density. Now consider an improved channe C(θ j ), j > 0 and et P j 0 be the associated channe density. If for any ensembe within the sequence, the vaue of λ 2 satisfies the stabiity condition corresponding to θ 0 with equaity, i.e., if λ n 2 = λ n 2, then the vaue of the upper bound Π 2 corresponding to θ j obtained in Theorem 2, is independent of n (in fact independent from the parent ensembe sequence (λ n,ρ n )). Coroary 2 (Rate-compatibiity of Π 2 for puncturing schemes with Π 2 = Π 2 ): Consider a sequence of ensembes (λ n (x),ρ n (x)) which are convergent over C(θ 0 ) and et P0 0 be the associated channe density. Now consider an improved channe C(θ j ), j 0 and et P j 0 be the associated channe density. If for any ensembe within the sequence, the vaue of λ 2 satisfies the stabiity condition corresponding to θ 0 with equaity, i.e., if λ n 2 = λ n 2, then the vaue of the upper bound Π 2 is a decreasing function of θ j. Now reca from Section III that over the BEC, simiar upper bounds to that of stabiity condition were obtained for other variabe node degrees. In the foowing, we show a simiar behavior for the case of rate-compatibe codes over the BEC. Proposition 1: Let (λ p,ρ) be a convergent parent code over the BEC with channe parameter ɛ 0. Suppose that the parent code is punctured to be used over a channe with parameter ɛ j < ɛ 0. Let p un 0 be the Bhattacharyya constant for this channe, i.e., p un 0 = ɛ j. Aso assume that λ p i 0, 2 i n D v. Define Π i = 1 pun 0 λ p i /T i (1 p un 0 )λp i /T. (18) i Then if Π i = Π i for 2 i < n, there exists an upper bound Π n = 1 pun 0 λp n /T n (1 p un 0 )λp n/t n on Π n above which the resuting punctured ensembe is not convergent over C(ɛ j ) and beow which the ensembe is convergent over C(ɛ j ) if the probabiity of erasure can be made sufficienty sma. We now woud ike to prove that the construction of universay capacity achieving rate-compatibe LDPC codes over the BEC can be achieved by appying the SM principe to the vaues of Π i s, i.e., starting from a parent sequence and for each ensembe member of the sequence, we maximize Π 2 as far as the ensembe remains convergent and continue this procedure successivey for other Π i vaues. This wi be performed for each of the J target channe parameters and we demonstrate that the resuting puncturing patterns are in fact rate-compatibe. We aso show that if the origina parent sequence is capacity achieving, so wi be a the J sequences 405 of punctured ensembes. Theorem 3: Let the parent ensembe sequence (λ n,ρ n ), constructed based on the method of [11], be capacity achieving over the BEC with parameter ɛ 0. For the set of channe erasure vaues ɛ j (ɛ 1 > ɛ 2 >... > ɛ J ), we puncture each ensembe within the parent sequence based on the SM principe. The resuting scheme is then universay capacity achieving rate-compatibe. This resut is consistent with the one obtained in [16] stating that random puncturing of a parent ensembe over the BEC, preserves the distance to capacity. The approach taken in [16] is, however, different and is based on the fact that one can mode the puncturing of an ensembe over the BEC, as the concatenation of the origina BEC channe with another BEC channe with erasure rate equa to puncturing. Simiar to the fatness condition, the approach of [16] is not extendabe to other BIOSM channes. The importance of our approach is that in principe, it may be extendabe to other BIOSM channes where we can expect that appying the SM principe to compute Π i vaues, might aso resut in (a scheme performing cose to) a universay capacity achieving rate-compatibe scheme. Unike the BEC case, however, the upper bounds on Π i have to be estimated numericay (simiar to the procedure we use to compute the upper bounds of λ i ; i > 2, for the unpunctured case) except for Π 2 whose upper bound is given by Theorem 2. Appying this procedure to the capacity approaching ensembes designed based on the method of Section III as parent ensembes, we have in-fact been abe to design universay capacity approaching rate-compatibe ensembes over other BIOSM channes. It is important to note that the vaues of Π j,n i in Theorem 3 do not depend on i and n. Whie independency of i is a specia property for the BEC, based on Coroary 1 these vaues are independent from n for i = 2 over any BIOSM channe. Our numerica resuts show that for a given i > 2,i D v and j, the vaues of Π j,n i are very cose for different vaues of n, suggesting a genera independency from n. V. DESIGN EXAMPLES We consider check reguar sequence with ρ(x) = x Dc 1 for channe parameter σ = Let k be the number of constituent variabe node degrees. We start with D c = 5 and k = 3, and for D c > 5, we set k = 2 Dc This means that the number of constituent variabe node degrees for an ensembe with check node degree D c is roughy twice that of an ensembe with check node degree D c 1. As can be seen in Tabe I, the performance of the ensembes consistenty improves as the average check node degree increases. For the rate-compatibe codes, we consider the sequence of Tabe I and puncture the first three ensembes for a set of four channes with noise powers smaer than that of the parent ensembe. The detais are provided in Tabe II, where we define the puncturing poynomia Π(x) = Dv Π ix i 1 to represent the puncturing fractions. In Fig. 1, we have potted the distance to capacity (R j /c(θ j )) of the ensembes of Tabe II versus

7 TABLE I PERFORMANCE OF A CHECK REGULAR SEQUENCE DESIGNED BASED ON THE SM METHOD OVER A BIAWGN CHANNEL WITH σ = D c R AWGN /c(.9557) ℵ TABLE II THE VALUES OF Π i USED TO PUNCTURE THE FIRST 3 ENSEMBLES OF THE SEQUENCE OF TABLE I. d c Parent λ(x) =.4322x λ(x) =.3457x λ(x) =.2881x ensembe x x x x x x x 9 σ = Π(x) =.2948x Π(x) =.2948x Π(x) =.2948x x x x x x x x 9 σ = Π(x) =.4115x Π(x) =.4115x Π(x) =.4115x x x x x x x x 9 σ = Π(x) =.4703x Π(x) =.4703x Π(x) =.4703x x x x x x x x 9 σ = Π(x) =.5308x Π(x) =.5308x Π(x) =.5308x x x x x x x x 9 R/c Fig. 1. The ratio of rate to capacity (R/C) for the rate-compatibe ensembe sequence of Tabe II. 0.7 σ D c = 7 D c = 6 D c = 5 TABLE III THE VALUES OF Π i S USED TO PUNCTURE THE ENSEMBLE C SM. σ = Π(x) = x x x x 9 σ = Π(x) = x x x x 9 σ = Π(x) = x x x x 9 σ = Π(x) = x x x x the channe noise standard deviations (σ). As can be seen, the performance of the punctured codes for a given parent ensembe is simiar to or better than the parent ensembe. In fact, we expect the punctured ensembe to perform amost the same as the parent ensembe, simiar to the case of the BEC. To justify the improvement resuting from puncturing, we note that athough the parent ensembes have been constructed based on the SM method, for finite vaues of D c, they are not necessariy optima in that they may not provide the best possibe rate for the given channe parameter. This eaves the door open for further improvement with puncturing. From Fig. 1, it is aso observed that for any given channe parameter, the performance of punctured ensembes approaches the capacity as the average check node degree increases. Based on Tabe II, the designed sequence aso fufis the rate-compatibiity property. Note however that unike, for exampe the approach of [15], we did not impose any constraint to guarantee ratecompatibiity and our empirica resuts suggest that this property is inherent in the proposed method. For the case of Π 2, we anayticay proved this fact in Coroary 2. To compare the performance of the schemes designed based on the SM principe and those obtained by optimization, we consider the ensembe used in [15] as a reference. This ensembe (C) has been optimized for the rate one haf and has a threshod of σ = We can assume that ensembe C has been optimized for the highest rate when the channe parameter σ is set to The degree distribution of C is: λ C (x) =.25105x x x x 9 ρ C (x) =.63676x x 7 Keeping the check node degree distribution of ensembe C intact, we design an ensembe C SM with the same number of constituent variabe nodes using the SM method: λ SM (x) =.2717x x x x 9. We then appy the SM method again, this time to puncture C SM. The puncturing poynomias for the same four channes considered in Tabe II are given in Tabe III. The distance to capacity (R/C) for the parent ensembe and its punctured versions is reported in Fig. 2. As can be seen in Fig. 2, the scheme performs very cosey to the scheme obtained by optimization-based puncturing of the ensembe C. In fact, the proposed scheme even sighty outperforms the scheme of [15] on channes with σ =.6300 and σ = The proposed scheme performs inferior ony on the best channe parameter (σ =.4675) and even for this channe parameter, the performance gap is ess than.08 db. 5 We have aso demonstrated the performance of random puncturing of the ensembe C for comparison. Aso note again that unike [15], we did not impose any constraint to guarantee ratecompatibiity. This reduces the design compexity significanty. It is interesting to see that based on Tabes II and III, except for i = D v, the vaues of Π i are amost independent (for Π 2 provaby independent based on Coroary 1) of the parent ensembes and ony depend on the channe parameter for which the puncturing is appied. In other words, for a given channe parameter θ j, the computed vaues of Π i can universay be appied to any ensembe designed based on the SM method for a given origina channe parameter θ 0. 5 Note that our parent code itsef performs cose to.1db worse than C and the gap in performance is aways ess than this gap for different puncturing rates. 406

8 Distance to capacity (db) Optimized puncturing on optimized ensembe C Random puncturing on optmized enesembe C SM puncturing on C SM σ Fig. 2. Performance comparison of schemes constructed based on the proposed SM method and those constructed based on optimization method of [15]. 0.6 VI. CONCLUSION In this paper, we proposed the concept of successive maximization for the systematic design of universay capacity approaching rate-compatibe LDPC code ensembe sequences. This was achieved by interpreting the fatness condition over the BEC as a Successive Maximization principe that was generaized to other BIOSM channes to design a sequence of capacity approaching parent ensembes. The SM principe was then appied to each parent ensembe, this time to design rate-compatibe puncturing schemes. As part of our resuts, we were abe to extend the stabiity condition which was previousy derived for degree-2 variabe nodes to other variabe node degrees as we as to the case of rate-compatibe codes. Consequenty, we rigorousy proved that using the SM principe, one is abe to design universay capacity achieving ratecompatibe LDPC code ensembe sequences over the BEC. Unike the previous resuts on such schemes over the BEC in the iterature, the proposed SM approach can be naturay extended to other BIOSM channes. Using such an extension, we designed rate-compatibe codes over BIAWGN channes whose performance universay approaches the capacity as the average check node degree increases. We demonstrated that for finite vaues of D c, the performance of the ensembes designed by our method is comparabe to those designed based on optimization. One major step in the continuation of this work is to anayticay compute the vaues of λ i. This can pave the road toward the anaytica proof that the proposed sequences can in fact achieve the capacity of BIOSM channes. REFERENCES [1] C. Di, D. Proietti, I. E. Teatar, T. J. Richardson, R. L. Urbanke, Finiteength anaysis of ow-density parity-check codes on the binary erasure channe, IEEE Transactions on Information Theory, vo. 48, no. 6, pp , June [2] M. Luby, M. Mitzenmacher, M. A. Shokroahi, D. Spieman, and V. Stemann, Practica oss-resiient codes, in Proc. 29th annua ACM Symposium on Theory of Computing, pages , [3] A. Shokroahi, Capacity-achieving sequences, in Codes, Systems, and Graphica Modes (IMA Voumes in Mathematics and Its Appications), B. Marcus and J. Rosentha, Eds., New York: Springer-Verag, 2000, vo. 123, pp [4] A. Shokroahi, New sequences of inear time erasure codes approaching the channe capacity, in Proc. 13th Internationa Symposium on Appied Agebra, Agebraic Agorithms, and Error Correcting Codes (M. Fossorier, H. Imai, S. Lin, and A. Poi), no in Lecture Notes in Computer Science, pp , [5] P. Oswad and A. Shokroahi, Capacity-Achieving sequences for the erasure channe, IEEE Transactions on Information Theory, vo. 48, no. 12, pp , Dec [6] I. Sason and R. Urbanke, Parity-check density versus performance of binary inear bock codes over memoryess symmetric channes, IEEE Transctions on Information Theory, vo. 49, no. 7, pp , Juy [7] H. D. Pfister, I. Sason, and R. Urbanke, Capacity-achieving ensembes for the binary erasure channe with bounded compexity, IEEE Transactions on Information Theory, vo. 51, no. 7, pp , Juy [8] H. D. Pfister and I. Sason, Accumuate-Repeat-Accumuate Codes: capacity-achieving ensembes of systematic codes for the erasure channe with bounded compexity, IEEE Transactions on Information Theory, vo. 53, no. 6, pp , June [9] T. J. Richardson, A. Shokroahi, and R. L. Urbanke, Design of capacity approaching irreguar ow density parity check codes, IEEE Transactions on Information Theory, vo. 47, no. 2, pp , Feb [10] T. Richardson and R. Urbanke, Modern Coding Theory, New York:Cambridge University Press, [11] H. Saeedi and A. H. Banihashemi, New sequences of capacity achieving LDPC code ensembes for the binary erasure channe, in Proc. Internationa Symposium on Information Theory, Toronto, Canada, Juy [12] H. Saeedi and A. H. Banihashemi, Deterministic design of LDPC codes for binary erasure channes, in Proc. IEEE Goba Teecommuncatin Confence,Washington, DC, USA, Dec [13] H. Saeedi and A. H. Banihashemi, Deterministic design of LDPC codes for binary erasure channes, accepted for pubication in IEEE Transactions on Communications, May [14] H. Pishro-Nik and F. Fekri, Resuts on punctured ow-density paritycheck codes and improved iterative decoding techniques, IEEE Transactions on Information Theory, vo. 53, no. 2, pp , Feb [15] J. Ha, J. Kim, S. W. McLaughin, Rate-compatibe puncturing of owdensity parity-check codes, IEEE Transactions on Information Theory, vo. 50, no. 11, pp , Nov [16] H. Pishro-Nik, N. Rahnavard, F. Fekri, Nonuniform error correction using ow-density parity-check codes, IEEE Transactions on Information Theory, vo. 51, no. 7, pp , Juy [17] H. Chun-Hao, A. Anastasopouos, Capacity achieving LDPC codes through puncturing, IEEE Transactions on Information Theory, vo. 54,no. 10, Oct [18] I. Sason and G. Wiechman, On achievabe rates and compexity of LDPC codes over parae channes: bounds and appications, IEEE Transactions on Information Theory, vo. 53, no. 2, pp , Feb [19] J. Ha, J. Kim, D. Kinc, S. W. McLaughin, Rate-compatibe punctured ow-density parity-check codes with short bock engths, IEEE Transactions on Information Theory, vo. 52, no. 2, pp , Feb [20] B. N. Veambi, F. Fekri, Finite-ength rate-compatibe LDPC codes: a nove puncturing scheme, IEEE Transactions on Communications, vo. 57, no. 2, pp , Feb [21] M. R. Yazdani and A. H. Banihashemi, On construction of ratecompatibe ow-density Parity-check codes, IEEE Communications Letters, vo. 8, no. 3, pp , March [22] Y. Guosen Yue, W. Xiaodong, and M. Madihian, Design of ratecompatibe irreguar repeat accumuate codes, IEEE Transactions on Communications, vo. 55, no. 6, pp , June [23] A. Shokroahi, Raptor codes, IEEE Transactions on Informatin Theory, vo. 52, no. 6, pp , June [24] C. Measson, A. Montanari, R. Urbanke. Why we can not surpass capacity: the matching condition in Proc. The Aerton Conference on Communication, Contro and Computing, Monticeo, USA, October [25] I. Sason, On universa properties of capacity-approaching LDPC code ensembes, IEEE Transactions on Information Theory, vo. 55, no. 7, pp , Juy [26] H. Saeedi, H. Pishro-Nik, and A. H. Banihashemi, On systematic design of capacity approaching universay rate-compatibe LDPC codes over BIOSM channes, submitted to IEEE Transactions on Information Theory, Sept