Iterative Decoding of LDPC Codes over the q-ary Partial Erasure Channel

Transcription

1 1 Iterative Decoding of LDPC Codes over the q-ary Partial Erasure Channel Rai Cohen, Graduate Student eber, IEEE, and Yuval Cassuto, Senior eber, IEEE arxiv: v2 [cs.it] 24 ay 2016 Abstract In this paper, we develop a new channel odel, which we nae the q-ary partial erasure channel QPEC. The QPEC has a q-ary input, and its output is either the input sybol or a set of 2 q sybols, containing the input sybol. This channel serves as a generalization to the binary erasure channel, and iics situations when a sybol output fro the channel is known only partially; that is, the output sybol contains soe abiguity, but is not fully erased. This type of channel is otivated by non-volatile eory ulti-level read channels. In such channels the readout is obtained by a sequence of current/voltage easureents, which ay terinate with partial knowledge of the stored level. Our investigation is concentrated on the perforance of low-density parity-check LDPC codes when used over this channel, thanks to their low decoding coplexity using belief propagation. We provide the exact QPEC density-evolution equations that govern the decoding process, and suggest a cardinality-based approxiation as a proxy. We then provide several bounds and approxiations on the proxy density evolutions, and verify their tightness through nuerical experients. Finally, we provide tools for the practical design of LDPC codes for use over the QPEC. Index Ters Density evolution, belief propagation, lowdensity parity-check LDPC codes, non-volatile eories, q- ary codes, partial erasure, iterative decoding, decoding threshold, erasure channels. I. INTRODUCTION The rapid developent of eory technologies have introduced challenges to the continued scaling of eory devices in density and access speed. One of the coon coputer eory technologies is non-volatile eory NV. In ulti-level NVs, such as flash eories, an inforation sybol is represented in a eory cell by one of q voltage levels, where inforation is written/read by adding/easuring cell voltage [1], [2]. The read process is usually perfored by coparing the stored voltage level to certain threshold voltage levels. To scale storage density in NVs, the nuber of levels per cell is continuously increased [3], [4]. As the nuber of levels increases, errors becoe ore and ore prevalent The authors are with the Departent of Electrical Engineering, Technion - Israel Institute of Technology, Haifa, Israel eail: rc@capus.technion.ac.il, ycassuto@ee.technion.ac.il This work was supported in part by the Geran-Israel Foundation, by the Israel inistry of Science and Technology, and by the Israel Science Foundation. Parts of this work were presented at the 2014 IEEE International Syposiu on Inforation Theory ISIT, Hawaii, USA, at the 8th International Syposiu on Turbo Codes & Iterative Inforation Processing ISTC, Breen, Gerany, and at the 2015 IEEE International Workshop on Inforation Theory ITW, Jerusale, Israel. Copyright c 2014 IEEE. Personal use of this aterial is peritted. However, perission to use this aterial for any other purposes ust be obtained fro the IEEE by sending a request to pubs-perissions@ieee.org. due to intercell interference [5]. In addition, the use of ultilevel eory cells in the eerging technology of resistive eories introduces significant reliability challenges [6]. Apart fro classical channels and error odels, ulti-level eories otivate coding for a diversity of new channels with rich features. Our work here is otivated by a class of channels we call easureent channels, in which inforation is written/read by adding/easuring electrical charges. These channels encopass a variety of equivocations introduced to the inforation by an iperfect read process, due to either physical liitations or speed constraints. The channel we propose and study here the q-ary partial erasure channel QPEC is a basic and natural odel for a easureent channel in ulti-level eories. The odel coes fro a read process that occasionally fails to read the inforation at its entirety, and provides as decoder inputs q-ary sybols that are partially erased. In the QPEC odel, an output sybol is a set of sybols that includes the correct input sybol. This set can be either of cardinality 1 or a set of sybols is a paraeter, 2 q. In the latter case, we say that a partial erasure event occurred, odeling the uncertainty that ay occur in the read process due to iperfect easureents. Theoretically speaking, the QPEC is a generalization of the binary or q-ary erasure channel BEC or QEC. In the BEC/QEC, sybols are either received perfectly or erased copletely; in the QPEC, partially erased sybol provide inforation in quantity that grows as gets saller. In this work, we suggest the use of GFq low-density parity-check LDPC codes [7] [9] for encoding inforation over the QPEC, due to their low coplexity of ipleentation and good perforance under iterative decoding [10]. These codes were shown to achieve perforance close to the capacity for several iportant channels, using efficient decoding algoriths [10], [11]. Non-binary LDPC codes were considered in several works, such as in [9], [12] [14], and were shown as superior to binary codes in several cases [9], [14]. In our analysis, we propose a essage-passing decoder for decoding GFq-LDPC codes over the QPEC, extending the known iterative decoder for the BEC/QEC to deal with partial erasures. The iterative-decoding perforance evaluation of LDPC codes is usually perfored using the density-evolution ethod [15] that tracks the decoding failure probability. However, this ethod becoes prohibitively coplex in practice as q increases, as it requires an iterative evaluation of ulti-diensional probability distributions [13], [15]. Thus, we provide approxiation schees for tracking the QPEC decoding perforance efficiently and verify their tightness. Finally, we develop tools for the design of good LDPC codes for the QPEC.

2 2 The paper is structured as follows. We begin by introducing the QPEC channel and its capacity in Section II. In Section III, we give a short review of GFq-LDPC codes and propose a essage-passing based decoder for the QPEC. Decoding perforance analysis is provided in Sections IV and V, and code design tools are discussed in Section VI. Finally, conclusions are given in Section VII. II. THE q-ary PARTIAL ERASURE CHANNEL A. Channel odel The q-ary partial erasure channel QPEC is a generalization of the well known binary erasure channel BEC [16] in two ways. First, siilarly to the q-ary erasure channel QEC, its input alphabet is q-ary, with q 2. Second, generalizing the BEC erasure event, a partial erasure occurs when the input sybol is known to belong a set of is a paraeter, 2 q sybols rather than q sybols. The QPEC is defined as follows. Let X be the transitted sybol, taken fro the alphabet X = { 0, α 0 = 1, α 1,..., α q 2} of cardinality q, where α is a priitive eleent of the finite field GFq i.e., the eleents in X are the field eleents. Define the super-sybols? i x for each x X and for i = 1, 2,..., q 1 i 1, such that? x is a set of sybols containing the sybol x and 1 other sybols, taken fro X \ {x}. The output Y given an input sybol x is a set of sybols, which is either the singleton {x}, or one of of cardinality for soe i. Therefore, the output alphabet Y contains all possible sets of cardinality 1 and cardinality taken fro X. The transition probabilities governing the QPEC are as follows: the sets? i x { 1 ε, y = {x} Pr Y = y X = x = ε/ q 1 i 1, y =? x, where 0 ε 1 is the partial erasure probability. That is, with probability 1 ε the input sybol is received with no error, and with probability ε a partial-erasure event occurs, such that the input sybol is known up to sybols. In the latter case, the output sets? i x are equiprobable. This odels a situation of axiu uncertainty at the output, which is uniforly distributed on sets of cardinality containing x. Note that for = q = 2 the QPEC is equivalent to the BEC, where for = q > 2 the QPEC is equivalent to the QEC. The transition probabilities are given explicitly in the following exaple for a particular choice of q, and a transitted sybol x. Exaple 1: Assue that q = 4 and that the sybol 0 was transitted. If = 2, the possible output sets, and their transition probabilities fro 1, are given by: Pr Y = y X = 0 q=4,=2 = 1 1 ε, y = {0} ε/3, y = { 0, α 0} ε/3, y = { 0, α 1} ε/3, y = { 0, α 2}. 2 B. Capacity Denote p x = Pr X = x, for x = 0, α 0, α 1..., α q 2, to be the input distribution to the channel. According to the definition of the channel capacity C, C = ax {p x} I X; Y = ax H Y H Y X, 3 {p x} where I X; Y is the utual inforation between the input X and the output Y, and H Y, H Y X are the entropy of Y and the conditional entropy of Y given X, respectively. The conditional entropy H Y X can be calculated using 1: q 1 H Y X = 1 ε log 1 ε ε log ε/. 1 4 The conditional entropy is independent of {p x } as expected, iplying that it is sufficient to axiize the entropy H Y to find the capacity. The QPEC capacity is provided in the following theore. Theore 1: Capacity The QPEC capacity is: C QPEC = 1 εlog q, 5 easured in q-ary sybols per channel use. The proof of this theore is provided in Appendix A. As one ay expect due to the unifor distribution of the output when a partial-erasure occurs, H Y is axiized under the unifor distribution of the input i.e., for p x = 1/q. Note the agreeent of 5 with the QEC capacity for = q, and in particular with the BEC capacity for = q = 2. C. axiu-likelihood decoding Assue that a codeword c taken fro a codebook C was transitted over the QPEC and that the output y was received. The eleents y i of y should be understood in a generalized sense, as they contain either a set of one sybol or a set of sybols according to the transition probabilities in Equation 1. For = q, in which the QPEC is essentially the QEC, codewords coinciding with y in non-erased positions are said to be copatible with y [10], and they serve as axiu-likelihood L decoding of y. However, when < q, partially-erased codeword sybol positions should be considered for the L decoding of y. To extend the notion of copatibility to QPECs with < q, we define the set: { } Ψ = c C : i, c i yi, 6 which is the set of all codewords that have in each position a sybol that is contained in the corresponding output of y in the sae position. Each codeword in Ψ can serve as an L decoding of c, since c and y ust agree in nonerased positions, and in the reaining positions the correct transitted codeword sybol c i is contained in y i by the QPEC definition. Therefore, y is decoded correctly with probability 1 if and only if Ψ = 1. In a siilar anner, when L sybol decoding is used, y i is decoded correctly with probability 1 if and only if all the codewords in Ψ contain the sae sybol in their i th position. In practice, L decoding coplexity is usually prohibitive. In the next section, we ove

3 3 Fig. 1: An exaple of a Tanner graph over GF4. Circles denote variable nodes codeword sybols, and squares denote check nodes parity-check equations. The sybols on the edges are the labels, leading to the parity-check equations on the right. to specify a low-coplexity iterative essage-passing decoder for GFq LDPC codes used over the QPEC. III. GFq LDPC CODES AND ESSAGE-PASSING A. GFq LDPC codes DECODING Before developing our coding results for the QPEC, we include soe well-known facts on LDPC codes as a necessary background. A GFq [n, k] LDPC code is defined in a siilar way to its binary counterpart, by a sparse parity-check atrix, or equivalently by a Tanner graph [17]. This graph is bipartite, with n variable left nodes, which correspond to codeword sybols, and n k check right nodes, which correspond to parity-check equations. The codeword sybols are taken fro GFq, where the labels on the graph edges are taken fro the non-zero eleents of GFq. In the graph, a check node c is connected by edges to variable nodes v N c, where N c denotes the set of variable nodes adjacent to check node c. The induced parity-check equation is h c,v v = 0, where v Nc h c,v are the labels on the edges connecting variable node v to check node c. Note that the calculations are perfored using GFq arithetic. An exaple of a Tanner graph is given in Figure 1. LDPC codes are usually characterized by the degree distributions of the variable nodes and the check nodes. They are called regular if both variable nodes and check nodes have constant degree. Otherwise, they are called irregular. Denote by d v and d c the axial degree of variable nodes and check nodes, respectively. As is custoary [10], we define the following degree-distribution polynoials: λ x = d v λ i x i 1, 7 d c ρ x = ρ i x i 1, 8 where for each i, a fraction λ i ρ i of the edges is connected to variable check nodes of degree i. These polynoials will be used later for analyzing the iterative-decoding perforance of LDPC codes over the QPEC. The design rate r of an LDPC code with degree-distribution polynoials λx and ρx, easured in q-ary sybols per channel use, is [10]: r = 1 1 ρ xdx 0 = 1 1 λ x dx 0 d c d v ρ i /i. 9 λ i /i The design rate equals to the actual rate if the rows of the LDPC code parity-check atrix are linearly independent. B. essage-passing decoder for the QPEC The following decoder for GFq LDPC codes over the QPEC is a variation of the standard essage passing/belief propagation algorith over a Tanner graph, generalizing the iterative decoding process used over the BEC/QEC. The key change is that in the QPEC setting the exchanged beliefs are sets of sybols, rather than individual sybols and erasure sybols as with the BEC/QEC. We have two types of essages at each decoding iteration l: variable to check VTC essages and check to variable CTV essages, denoted VTC l v c and CTV l c v, respectively. Each outgoing essage fro a variable check node to a check variable node depends on all its incoing essages, except for the incoing essage originated fro the target node. At iteration l = 0, channel inforation is sent fro variable nodes to check nodes: partially-erased nodes send sets of sybols of cardinality, while non-erased ones send sets of cardinality 1 recall that both sets contain the correct sybol. The channel inforation sent fro variable node v will be denoted VTC 0 v. In the subsequent iterations, the operations of the essagepassing decoder translate to the following operations on sets. CTV l c v contains the possible values of v given the incoing essages fro the variable nodes {N c \v}, such that the parity-check equation induced by check node c is satisfied. For later use, we note that the calculation of can be represented copactly, as follows. Define the suset or inkowski su [18] operation between sets S j j = 1, 2,..., J that contain GFq eleents: J J S j = S j : s j S j. 10 CTV l c v j=1 j=1 That is, the suset results in a set containing all sus using GFq arithetic of eleents taken fro S j. Exaple 2: Assue that q = 4 and consider the sets { 0, α 0} and { 0, α 1}. The suset of these sets is { 0, α 0} + { { 0, α 1} = 0 + 0, 0 + α 1, α 0 + 0, α 0 + α 1} = { 0, α 0, α 1, α 2}, i.e., all the field eleents. On the other hand, if both sets are { 0, α 0}, then the suset is { 0, α 0} + { 0, α 0} = { 0, α 0}. For each pair of check node c and variable node v, define the following sets for each v {N c \v} : { } B l v = h c,v g : g VTC l 1, l 1 11 v c which are the VTC essage sets sent fro the variable nodes adjacent to c except v to check node c at iteration l 1, ultiplied by the additive inverses of the edge labels. We can

4 4 a An exaple for a CTV essage over GF4. The outgoing essage is α2 {0, α 0} + α3 {0} = { 0, α 1}. α α b An exaple for a VTC essage. The edge labels are not shown as they are not required for the VTC essage calculation. Fig. 2: Exaples for CTV/VTC essages in the decoding process. Circles denote variable nodes, and squares denote check nodes. Sybols on edges represent edge labels. The two sets on the botto are incoing essages, where the set on the top is the corresponding outgoing essage. then represent the calculation of CTV essages in a copact anner: CTV l c v = 1 B l v, l h c,v v {N c\v} In words, CTV l c v is the set of possible values of v given the incoing VTC essages fro the variable nodes in {N c \v}, where an exaple is given in Figure 2a. Note that a CTV essage can be of cardinality between 1 and q. We now ove to calculate the VTC essages, which are based on the CTV essages. The VTC essages are calculated as follows: VTC l v c = VTC 0 v CTV l c v c {N v\c}, l That is, the VTC essage VTC l v c is the set of sybols containing the intersection of the channel inforation and the incoing CTV essages to variable node v, where an exaple is given in Figure 2b. A VTC essage cardinality can be at ost, as the channel inforation cardinality is at ost. A decoding failure occurs if at the end of the decoding process there is a VTC essage containing ore than one sybol. The decoding process described above reduces to the known iterative decoder proposed for the BEC/QEC [10] when = q. In this case, the passed essages can be either the set containing all q sybols full erasure or a set containing the correct sybol only. This greatly siplifies the asyptotic iterative-decoding perforance analysis of LDPC codes when used over the BEC [10]. However, apart fro erasure/nonerasure essages in the BEC case, there are any other possible essage sets in the QPEC decoding process, aking the analysis prohibitively coplex as q increases. In the following section, we discuss our approach for low-coplexity approxiate asyptotic decoding perforance analysis, which is later shown to capture the exact behaviour quite well. IV. DENSITY EVOLUTION ANALYSIS Density evolution analysis of decoding perforance is carried out by tracking the asyptotic in the codeword length probability of decoding failure at each iteration based on the probabilities of the passed essages [13], [15], [19]. In this section, we use this ethod for asyptotic perforance evaluation of the decoder described in Section III-B. As custoary, we assue a randoly constructed Tanner graph with a degree-distribution pair λ and ρ, and a rando i.i.d. selection of edge labels distributed uniforly on the non-zero eleents of GFq. In addition, a sufficiently large codeword length is assued, such that incoing essages to each node at each iteration of the decoding process are statistically independent with high probability known as the independence assuption [15]. We start with deriving the exact QPEC density-evolution equations, and then ove to propose approxiate density evolution analysis due to coplexity reasons. Let us denote by S t, t = 1, 2,..., 2 q 1, the non-epty subsets of the input alphabet X of q sybols, ordered by cardinality and in lexicographical order. These subsets ay be passed throughout the decoding process as either VTC or CTV essages see Section III-B. Denote by z l t the probability that a VTC essage at iteration l is S t. Siilarly, denote by w l t the probability that a CTV essage at iteration l is S t. I d resp. J d will denote an ordered list containing d = d 1 indices taken with possible repetitions fro the set of essage indices {1, 2,..., 2 q 1}, representing VTC resp. CTV essages to a degree-d check resp. variable node. Enuerating the edges connected to a check variable node 1 to d, an eleent in I d resp. J d is the index of the essage on the corresponding edge. For exaple, there are 15 2 = 225 ordered lists I 2 for a degree-3 check node when q = 4: 1, 1, 1, 2, 2, 1, 2, 2,..., 15, 15, where the eleents of I 2 are the first and second incoing essage indices. χ t I d will denote the probability that the VTC essages indexed in I d lead to the CTV essage S t. Siilarly, η t J d will denote the probability that the CTV essages indexed in J d lead to the VTC essage S t. The distributions χ t and η t are obtained with respect to the unifor edge labels and the channel inforation, as deonstrated in the following exaple. Exaple 3: Assue a degree-3 check node and that q = 4. Consider I 2 = 5, 5 and recall that according to our convention, S 5 = { 0, α 0}. To calculate χ t I 2, we find all possible outcoes of the suset h 1 /h 3 {0, α 0} + h 2 /h 3 {0, α 0} where h 1, h 2 and h 3 are i.i.d. rando variables uniforly distributed on { α 0, α 1, α 2}, representing the edge labels. If h 1 = h 2 = h 3, the suset is { 0, α 0} + { 0, α 0} = { 0, α 0} =

5 5 S 5. On the other hand, if the edge labels are not the sae, the suset is { 0, α 0, α 1, α 2} = S 15. Therefore, the non-zero χ t values are χ 5 = 1/9 and χ 15 = 8/9 in this case. Now consider a degree-3 variable node, where J 2 = 6, 6 and the channel inforation sets are S 5 = { 0, α 0}, S 6 = { 0, α 1} and S 7 = { 0, α 2} i.e, = 2, each with probability 1/3. If the channel inforation is S 5 or S 7, then the intersection between the essages indexed in J 2 and the channel inforation is S 1 = {0}. If the channel inforation is S 6, the intersection results in S 6. Therefore, we get that for J 2 = 2, 2, the nonzero η t values are η 1 = 2/3 and η 6 = 1/3. As GFq LDPC codes are linear codes, the probability of a given codeword sybol taken fro the codebook is 1/q. This eans that a variable node contains a certain set coposed of one sybol i.e., non-erasure with probability 1 ε/q. To incorporate this probability in the density-evolution equations, we define the indicator θ t, which equals 1 if S t = 1 and 0 otherwise. Equipped with the notations above, we get the following copact representation of the QPEC densityevolution equations: d c w l t = ρ i χ t I i 1, 14 z l t = ε d v I i 1 λ i J i 1 z l 1 j j I i 1 w l j j J i 1 η t J i ε q θ t. 15 The suation over I i 1 or J i 1 is understood over all ordered lists containing i 1 eleents where i is the node degree taken fro the set of indices {1, 2,..., 2 q 1}. A decoding failure occurs when a variable node is not resolved, i.e., when it contains a set with ore than one sybol: p l e = t: S t >1 z l t = 1 t: S t =1 z l t. 16 Note that for = q, only z 1, w 1, z 2q 1 and w 2q 1 i.e., probabilities of full-erasure/non-erasure sets ight be positive. In this case, these probabilities can be represented solely by z 2 q 1, as the distributions χ t and η t degenerate due to the siple BEC/QEC decoding rules. Equations can be then readily siplified to obtain the BEC/QEC onediensional density-evolution equations [10]. Calculating χ t I d and η t J d in Equations ight be prohibitive in practice, as the nuber of subsets increases exponentially with q. To get an estiate of the coplexity, consider basic check and variable nodes of degree 3. Given two incoing essage sets to the check, calculating the distribution χ t requires q 1 3 realizations of edge labels. Because there are O 2 q input-set pairs, we get O q 3 2 2q coplexity for calculating χ t. In a siilar anner, O q 2 2q operations are required for calculating η t the first factor now being the nuber of possible channelinforation sets for a degree-3 variable node. As an exaple, about operations are required for the calculation of χ t when q = 16, growing to the order of when q = 32. In addition to prohibitive coplexity, the exhaustive calculation of χ t and η t as deonstrated in Exaple 3 provides no insights on their behaviour. oreover, χ t requires the explicit use of GFq arithetic, aking its analysis difficult. These reasons otivate us to propose a ore efficient way for estiating the QPEC decoding perforance, which we discuss in the following subsection. A. Cardinality-based approxiated density-evolution equations To overcoe the difficulties in evaluating Equations 14-15, we propose to track the probability distribution of the VTC/CTV essage set cardinalities. In our approach, we approxiate essages of the sae cardinality passed in the decoding process as being equiprobable. The intuition behind this approxiation coes fro the randoness of the edge labels and the channel output that soothen ost of the non-unifority that ay occur due to the algebraic structure of GFq. In particular, as the node degrees and the field order grow, the incidence probability of equal-cardinality sets becoes increasingly unifor. The reason is that the entropy of each su in the suset perfored at check nodes increases with the degree. In addition, the nuber of sus within the suset increases with q, increasing the entropy of the suset result. The approxiation was verified epirically as well, where we show in Section V-D that perforance analyzed with this assuption gives a very good approxiation of the true decoding perforance. To distinguish between essage sets and their cardinalities, we use the notation d to denote an ordered list of d = d 1 eleents taken fro {1, 2,..., q}, understood as possible incoing essage-set cardinalities to a degree d check node. W l resp. Z l will denote the probability that a CTV VTC essage at iteration l is of cardinality = 1, 2,..., q. P d resp. Q d will denote the probability that the essage-set cardinalities in d lead to an outgoing CTV VTC essage of cardinality. Note that the distributions P and Q are obtained by suing the probabilities of χ t and η t for all t with S t =, assuing unifor distribution on the input sets with cardinalities in d. Finally, under our approxiation, the following equations are derived: W l d c ρ i Z l ε d v λ i i 1 i 1 Z l 1 i ε δ [ 1], W l i 1 P i 1, Q i where δ [] is the discrete Dirac delta function. The suation over i 1 is understood over the ordered lists of i 1 eleents taken fro the set of possible incoing essage-set cardinalities. This set is {1, 2,..., } for incoing VTC and {1, 2,..., q} for incoing CTV essage-set cardinalities. The initial conditions are Z 0 1 = 1 ε, Z 0 = ε and Z0 = 0 for 1,. The asyptotic probability of decoding failure at

6 6 iteration l is the probability of a VTC essage-set cardinality larger than 1 at iteration l: q p l e = Z l = 1 Z l =2 We note here that in our experients the probability of decoding failure calculated using 19 is virtually the sae as 16 even for sall q and check-node degree values, such that the cardinality-based equations can be safely used for QPEC perforance evaluation. However, though we oved fro O2 q possible essage sets to q possible essageset cardinalities, we still need efficient ways to calculate P and Q. A straightforward calculation enuerates χ t and η t for O2 2q realizations of essage set pairs, which does not quite solve the coplexity proble. Thus we devote the reainder of this section and the next section to efficient calculations, bounding, and approxiations for P and Q. We begin with providing in Section IV-B an exact closedfor expression for Q. In Section V we show that finding a closed-for expression for P is hard. Therefore, we propose coputationally efficient bounds and approxiation odels for P. We later use our odels and bounds to deterine the QPEC decoding threshold and to design good LDPC codes. B. Forula for Q Q d is the probability of an intersection of cardinality between CTV essages with cardinalities taken fro d and a channel inforation set of cardinality, where essage sets of the sae cardinality are equiprobable. Define d to contain the cardinalities in d together with the channel inforation set cardinality and µ to be the sallest cardinality in d, i.e. µ = in d. In the following, we find the nuber of ways to realize the sets in d such that their intersection is of cardinality, and later take into account the presence of the correct sybol in each set. We begin with the following lea. Lea 2: Nuber of ways to get an intersection of cardinality Consider d essage sets whose cardinalities are in d. The nuber of ways to realize the sets such that their intersection is of cardinality = 0, 1,..., µ is: K d ; q = µ s=0 where q υ +s = + s 1 s υ + s +s d, 20 q + s s Proof Consider a fixed subset of µ eleents taken fro a set of q eleents. The nuber of ways to choose d subsets with cardinalities in d such that they all contain the subset of µ eleents is µ, as we are free to choose only d q µ µ eleents for each subset of cardinality. Taking into account the nuber of ways to choose a subset of µ eleents, which is q µ, we have q q µ K µ = µ = υ µ 22 µ d ways to choose the subsets such that their intersection is of cardinality µ. To find K for = µ 1, we proceed as follows. The nuber of ways to choose the subsets such that they contain a fixed subset of µ 1 eleents is q µ 1 µ 1. d However, the subsets ay also contain a subset of cardinality µ such that the fixed subset of cardinality µ 1 is its subset, resulting in overcounting. Since there are µ µ 1 = µ sets of cardinality µ 1 contained in a set of cardinality µ, we correct for overcounting as follows: q K µ 1 = µ 1 d =υ µ 1 µ υ µ. q µ 1 µ υ µ 23 µ 1 oving to µ 2, we first count sets of cardinality µ 2 with υ µ 2 and then subtract µ 1 µ 2 υµ 1 sets to account for sets of cardinality µ 1. However, we now over-correct soe sets of cardinality µ. We account for that by considering the µ µ 2 sets of cardinality µ 2 contained in a set of cardinality µ to obtain: µ 1 µ K µ 2 = υ µ 2 υ µ 1 + υ µ. 24 µ 2 µ 2 Continuing in the sae fashion essentially, we use the inclusion-exclusion principle, we get: t K µ t = 1 i υ µ t + i µ t+i, 25 µ t i=0 for t = 0, 1,..., µ. Index shifting leads to the desired result. We are now ready to provide a forula for Q. Lets us denote by d 1 the ordered list obtained by subtracting 1 fro each nuber set cardinality in d. Theore 3: Forula for Q K 1 d 1;q 1, if µ > 1 q 1 Q d = 1 d 26 δ [ 1], otherwise. Proof We use K 1, d 1 and q 1 as we can choose effectively 1 eleents for each subset of cardinality, as the correct sybol appears in the subsets. We then noralize by the nuber of subsets with cardinalities 1 taken fro a set of q 1 eleents to obtain a probability distribution. Note that when µ = 1 the intersection is necessarily of cardinality 1, such that that Q 1 = 1. V. BOUNDS AND APPROXIATIONS FOR P P d in Equation 17 is the probability that the suset of the sets with cardinalities in d is of cardinality, where sets of the sae cardinality are equiprobable and the edge labels are uniforly distributed. Considering all possible realizations of the essages becoes intractable as the field size or the node degree increase. The ajor reason for the difficulty in calculating P unlike Q is that it involves GFq arithetic. Thus, finding a closed-for expression for P is hard, see e.g. the discussion on susets in [18], [20], [21]. Because of that, we seek instead efficient bounds and

7 7 approxiations for P. Let I d contain indices of arbitrary essage sets whose cardinalities are in d. Denote κ = ax d as the axial nuber set cardinality in d. In addition, denote N = as the nuber of sus in d the calculation of S j. j I d Exaple 4: Assue that q = 4 and d = 2. If d = {2, 3}, then the first eleent in I d can be between 5 and 10, and the second eleent can be between 11 and 14. A. Upper and lower bounds on P using additive cobinatorics In this subsection we derive bounds on the cardinality of the suset S j. These bounds will be a function of the j I d essage-set cardinalities d, such that they are universal for all realizations of sets adhering to the cardinalities in d. We begin with siple lower and upper bounds. Lea 4: Siple bounds on a suset cardinality [18] κ S j in q, N. 27 j I d The following lea provides a sufficient condition for attaining the axial suset cardinality q. Lea 5: Sufficient condition for the suset of cardinality q [18] If there are, d where and are taken fro two different positions in d such that + > q, then S j = q. j I d For later use, we say that the q-condition holds if the condition of Lea 5 is satisfied. Note that this condition can be satisfied only if > q/2. We now proceed to obtain iproved lower bounds on the suset cardinality, using the following two theores. Theore 6: Cauchy-Davenport Theore [18] Consider the finite field GFp, p prie. Let S a and S b be two non-epty subsets of GFp. Then: S a + S b in p, S a + S b The following theore by Károlyi provides an extension of the Cauchy-Davenport theore to finite groups. Theore 7: Károlyi s theore for finite groups [22] Let S a and S b be two non-epty subsets of a finite group G. Denote by p G the sallest prie factor of G. Then: S a + S b in p G, S a + S b This theore can be used for extending the inequality 28 to extension fields, as we have in the following theore. Theore 8: Iproved suset cardinality bounds Denote by p the prie factor of q. Then: ax κ, in p, d + 1 S j 30 d j I d in q, N. Proof This theore is proved by Lea 4 and Theore 7, followed by induction on the nuber of subsets see e.g. [23] for the proof technique when q is prie. The bounds of Theore 8 are sharp i.e., there exist subsets S j with cardinalities in d such that the bounds are attained [18]. We will denote by B L and B U the lower and upper bounds of inequality 30, respectively. We use these bounds to derive two bounding distributions P ax and P in : the forer to bound the output set cardinalities fro above, and the latter fro below. To get P ax, the suset is assued as of cardinality B U with probability 1, unless the q-condition is satisfied. P ax = { δ [ q], if the q-condition holds δ [ B U ], otherwise. 31 In a siilar anner, P in is calculated using the lower bound B L on the suset cardinality: P in = { δ [ q], if the q-condition holds δ [ B L ], otherwise. The iportance of P ax resp. P in 32 is that using the in the density evolution iteration in place of the true P gives a lower resp. upper bound on the asyptotic probability of decoding failure 19 calculated using the cardinality-based density-evolution equations. Going beyond the bounds above to a potentially tighter characterization of P, in the reainder of the section we propose two low-coplexity approxiation odels for P. We begin with a siple balls-and-bins odel, and later refine it with a tighter odel. Finally, we copare the bounds above with the proposed approxiation odels. B. The balls-and-bins odel The ajor difficulty in calculating P exactly is its dependence on the structure of the finite-field arithetic. Going around this difficulty, we propose a pure-probabilistic approxiation of P using the balls-and-bins odel [24]. In this odel, balls are placed independently and uniforly at rando to bins, where we are usually interested in the distribution of the nuber of non-epty bins once all the balls were placed. otivated by the randoness induced by the rando edge labels, we propose to consider the N sus in the calculation of the suset as the balls, and the q eleents of GFq as the bins. This way, P is odeled as the probability of nonepty bins after the N balls were placed. As a consequence, the use of GFq arithetic is not required when the ballsand-bins odel is used. The balls-and-bins odel is an absorbing arkov process with q + 1 possible states, with state = 0, 1,..., q corresponding to non-epty bins out of q. The absorbing state is q, as once q bins are non-epty the nuber of nonepty bins cannot change. The q + 1 q + 1 arkov atrix describing this process takes a siple for, since we can either stay at state or ove to state + 1. Denoting the arkov atrix as Γ balls, its entries are: Γ balls, = q, Γ balls,+1 = 1 q, 33

8 8 where the reaining entries are zeros. That is, if the current state is, then a ball is placed in a one of the non-epty bins with probability /q, and is placed in a different bin with probability 1 /q. Let us denote by g N = g N 0, g N 1,..., g q N the probability distribution on the states defined by Γ balls, where g N is the probability of state after the N balls were placed. According to the arkov property, g N = g 0 Γ N balls where ΓN balls is Γ balls raised to power N. As g 0 = 1, 0,..., 0 i.e., the bins are epty at the beginning, g N is siply the first row of Γ N balls. Finally, using the q-condition Lea 5 and the lower bound B L see Section V-A, we define the following approxiation odel for P : 0, if < B L δ [ q], if the q-condition holds P balls = g N q =B L g N, otherwise. 34 The expected nuber of balls required to get into the absorbing state q when starting at state 0 is q ln q + qγ up to O 1/ 2q ters, where γ is the Euler- ascheroni constant [24]. That is, all the bins will be nonepty on average when N q ln q + q γ, which can be thought as the probabilistic extension of the q-condition to the balls-and-bins odel. For such N values, the suset cardinality approxiated using the balls-and-bins odel is expected to be q and g N degenerates to the absorbing distribution 0,..., 0, 1. Therefore, Γ N balls should be calculated in practice for values of N up to approxiately q ln q + qγ, even for high-degree check nodes. C. The union odel In the previous sub-section, we odeled P using the balls-and-bins odel, where each ball which corresponds to an eleent obtained by a su within the suset is independent of the other balls. In this part, we iprove this approxiation by exploiting an iportant property of the N sus within the suset: they can be divided into N/κ sets of κ distinct eleents. This is proved by viewing the sus as generated by one eleent fro the axial-cardinality subset of cardinality κ and eleents fro the reaining subsets. This observation leads us to suggest a refined version of the balls-and-bins odel, which we ter as the union odel. In this odel, the probability of a suset of cardinality is odeled as the probability that the union of N/κ rando sets with cardinality κ each results in a set of cardinality. In view as balls-and-bins, it is the probability of non-epty bins after κ groups of N/κ balls are placed into the q bins, where the balls in each group are placed uniforly at rando into κ distinct bins. Let us denote by u N/κ = u N/κ 0, u N/κ 1,..., u N/κ q the probability distribution on the q + 1 states after N/κ groups of balls were placed into the bins. That is, u N/κ is the probability of state after N/κ groups of κ balls each were placed in the bins according to the union odel. The transition probability P union fro state to state Fig. 3: The expected nuber of balls required for getting q non-epty bins in the union odel. is equivalent to the probability that the union of a rando set of cardinality with a rando set of cardinality κ is of cardinality given that the set eleents are taken fro q eleents. To calculate this probability, denote by A a set of cardinality and by B a set of cardinality κ. We have: P union = Pr A B = 35 = Pr A B = + κ, where we used the inclusion-exclusion principle. Thus, we can equivalently find the probability that the intersection of the sets A and B is of cardinality + κ. Recall that K +κ, κ see Lea 2, q is oitted for brevity is the nuber of ways to obtain such an intersection cardinality. Dividing K +κ, κ by the nuber of possible realizations of eleents in the sets provides the desired probability P union. Therefore, the entries of the arkov atrix associated with the union odel are: Γ union, = K +κ, κ q. 36 κ It is not hard to check that for κ = 1, Γ union reduces to Γ balls defined in 33. As before, the arkov property iplies that u N/κ is siply the first row of Γ N/κ union. Finally, the following approxiation for P is based on the union odel: 0, if < B L δ [ q], if the q-condition holds P union = u N/κ q, otherwise 37 u N/κ =B L To obtain the expected nuber of balls required to get into the absorbing state q, we use the fundaental atrix [25] associated with an absorbing arkov chain. In our case, this atrix is Φ union = I q Q union 1 where I q is the identity atrix of diensions q q and Q union is the upper-left q q sub-atrix of Γ union. The expected nuber of groups of balls required to get into the absorbing state q when starting with state i 0 is the i th 0 entry of the vector Φ union1, where 1 is a colun vector whose entries are all 1 [25] to get the expected nuber of balls, we ultiply by κ. In Figure 3, the expected q

9 9 nuber of balls required for getting fro state 0 to state q is given as a function of q and κ. As entioned earlier see Section V-B, this expected nuber can be used for extending the q-condition to the union odel. Note that for κ = 1 the union odel is essentially the balls-and-bins odel and we have Φ union 1 i = 0 q ln q + qγ as we saw earlier. D. Coparison of the bounds and approxiations In this part, we verify the tightness of our approxiations by coparing the decoding threshold [15] obtained fro the exact and the approxiate cardinality-based density-evolution equations. The QPEC decoding threshold for a given degreedistribution pair, denoted ε th, is the axial partial-erasure probability ε such that the probability of decoding failure tends to zero. Its operational eaning is the robustness of the iterative decoder to partially-erased codeword sybols, i.e., the fraction of partially-erased codeword sybols that the decoder can tolerate. In Figure 4, we plot the exact and approxiate decoding threshold values using the bounds and odels for P for several values of q and for the regular 3, 6 LDPC code enseble of rate 1/2. We note that the exact threshold for q = 16 is not provided in Figure 4 due to coplexity reasons. When = q, the QPEC density-evolution equations are equivalent to the BEC/QEC density-evolution equation see Section IV. In this case, all the odels and bounds give the exact threshold, which is According to Figure 4, the upper bound on the threshold calculated using the cardinality-based equations becoes loose as q increases. This is due to the dependency of the suset cardinality lower bound see Theore 8 on the sallest prie factor of q, which is 2 for binary fields. This akes the threshold upper bound for such fields less tight copared to prie fields. The lower bound is also soewhat loose, as it corresponds to the upper bound on the suset cardinality that depends on the nuber of sus in the suset. However, the bounds on the suset cardinality are sharp see Section V-A, so it is difficult to iprove the bounds shown in Figure 4. On the other hand, the balls-and-bins odel and the union odel provide good approxiations of the exact threshold, and they are significantly tighter than the bounds. Recall that these approxiations can be calculated efficiently, aking the odels especially attractive for large values of q. We deduce fro Figure 4 an interesting result: not considering the algebraic structure of the field when using the approxiation odels leads on average to saller suset cardinalities copared to the exact calculation of the susset. If, as we conjecture, the approxiation odels give indeed upper bounds on the threshold, the uncertainty interval of the exact threshold is relatively sall, and it becoes saller as approaches q. Figure 4 suggests a potential application of the QPEC to speed up the read process in easureent channels. As an exaple, suppose that q = 8 and = 4. The decoding threshold in this case is approxiately Thus, instead of perforing q 1 = 7 coparative easureents to copletely read the stored sybol, in 59% of the cells we ay perfor only one easureent yielding q/2 = 4 = uncertainty. In ters of read rate, we now need only 3.46 easureents on average, iproving the read rate by ore than 50%. VI. CODE DESIGN USING LINEAR PROGRAING The design of good LDPC codes for the QPEC using the exact density-evolution equations is difficult due to their O2 2q diensionality see Section IV. otivated by the efficiency and the good approxiations obtained using the cardinality-based approach, we propose two ethods for linear prograing LP optiization of degree distributions. In Section VI-A, we present a threshold-oriented iterative optiization process. In Section VI-B, we use the union odel to achieve a target of sall decoding-failure probability. A. Iterative QPEC code design Let us denote by ε c th the approxiate decoding threshold obtained using the cardinality-based equations In this subsection, we propose two LP optiization ethods for obtaining a degree-distribution pair with a desired ε c th value. Recall that W q l denotes the probability of a CTV essage of cardinality q at iteration l of the decoding process when the cardinality-based equations are used see Section IV-A. Assuing a QPEC with > q/2, a sufficient condition for an outgoing CTV essage to be of cardinality q is the q- condition, eaning that there is at least one pair of incoing VTC essages whose su of cardinalities exceeds q see Lea 5. Therefore, W q l is bounded fro below by the probability that at least two incoing VTC essages are of cardinality, since 2 > q when > q/2. As Z l 1 denotes the probability for a VTC essage of cardinality at iteration l 1, we get: d c i 1 i 1 j i 1 j W q l ρ i Z l 1 1 Z l 1 j j=2 = 1 ρ 1 Z l 1 Z l 1 ρ 1 Z l 1, 38 where ρ x denotes the derivative of the polynoial ρx with respect to x. A sufficient condition for obtaining VTC essages of cardinality is that a variable node is a partial erasure and all its incoing CTV essages are of cardinality q. Therefore, Z l 1 ε d v i=1 λ i W q l 1 i 1 = ελ W l 1 q. 39 λx is an increasing function of x for x 0, since λx is a polynoial with non-negative coefficients. Note that both W q l and the right-hand side of 38 are non-negative as probabilities. Thus, according to 38, λ W q l 1 λ 1 ρ 1 Z l 2 Z l 2 ρ 1 Z l Finally, by cobining 39 and 40, we get: Z l 1 ελ ρ 1 Z l 1 Z l 1 ρ 1 Z l 1, 41 with the initial condition Z 0 = ε. The inequality in 41 applies to any > q/2, for all q which is prie or prie power, and depends solely on the

10 10 a q = 3 b q = 4 c q = 5 d q = 7 e q = 8 f q = 16 Fig. 4: The QPEC threshold of the regular 3, 6 LDPC code enseble, as a function of q and. degree distributions λx and ρx. Recall that in the case of the BEC, we have an equality rather than an inequality, and without the additional ter Z l 1 ρ 1 Z l 1. This ter leads to an upper bound on ε c th, as we will see. Define the function h ε x = ελ 1 ρ 1 x xρ 1 x which is the right-hand side of the inequality in 41 with Z l 1 replaced by x, and denote by h l ε x the l th coposition of h ε x with itself. We begin with the following lea. Lea 9: 1 Z l hl ε ε, l 1. 2 li l h l ε ε exists and is an increasing function of ε. The proof of this lea is provided in Appendix B. Observing

11 11 that h l ε ε = 0 for ε = 0 and using the second part of Lea 9, we are able to define the following value: { } ε = sup ε [0, 1] : li h l ε ε = l Note that ε is defined with respect to a certain degreedistribution pair λ and ρ. This definition of ε leads to an upper bound on ε c th. Theore 10: For a QPEC with > q/2, ε c th ε. Proof Z l is bounded fro below by a strictly positive value for all l when ε > ε, according to Lea 9 and the definition of ε in 42. Since the probability of decoding failure according to the cardinality-based approach 19 in this case is necessarily non-zero, ε c th cannot exceed ε. Fig. 5: An exaple of the iterative optiization process q = For the forulation of an LP optiization, we derive an 3, = 2. equivalent definition for ε, by extending the fixed-point characterization of the BEC threshold [19]. Theore 11: For a QPEC with > q/2, { } LP optiization. Because the BEC is a degraded version of ε = sup ε [0, 1] : x = h ε x has no solution x in 0, 1]. 43 The proof of this theore is siilar to the proof of Theore 3.59 in [10], and is oitted. We now forulate an LP optiization for deterining good in ters of code rate variable-node degree distribution λx for given ρx and ε assuing that > q/2. A axiu constraint d v on variable-node degrees is set, as usual [10], to control ipleentation coplexity and convergence speed. According to the ε equivalent definition 43, the condition for degree distributions whose threshold is upper bounded by ε is that h ε x x 0 for x 0, 1]. This leads us to forulate an LP optiization for the QPEC, where axial rate is sought under the constraint that ε c th is upper bounded by ε : { dv λ i ax λ i :λ i 0, d v } λ i = 1, h ε x x 0, x 0, 1]. 44 We ter the LP optiization in 44 as QPEC* LP. Note that the decoding threshold increases as decreases, such that the degree distributions obtained by QPEC* LP provide at least the sae threshold for a QPEC with q/2. The difference between the known BEC or QEC LP [10] and QPEC* LP is in using in 44 the function h ε x specially developed for the QPEC, instead of the function f ε x = ε λ 1 ρ 1 x derived fro the BEC density-evolution equation. The QPEC* LP optiization provides a degree-distribution pair with ε c th upper-bounded by ε. This suggests the following strategy for obtaining degree distributions with a desired value of ε c th. Choose ε that is larger than the desired ε c th, and solve the QPEC* LP optiization. Find ε c th of the optiized degree distributions using the cardinality-based densityevolution equations where the union odel is suggested for large q. If the threshold is saller than ε c th, increase ε and repeat the process. Otherwise, decrease ε and repeat the process. An alternative design ethod using previously known theoretical tools is to seek the desired ε c th using the BEC the QPEC, here we will choose a target BEC threshold ε BEC th saller than the desired ε c th. We then siilarly calculate εc th of the resulting degree distributions, and decrease/increase the BEC threshold as needed note that the BEC LP approach is valid for q/2 as well. It turns out that using the QPEC* LP approach can result in better codes copared to the BEC optiization. In the sequel we show this by nuerical exaples. The intuition behind this iproveent is that the QPEC* LP optiization better captures the decoding perforance for QPECs with < q. We now show the benefit of the new QPEC LP optiization in achieving better code ensebles than those obtained using the BEC LP optiization. As an exaple, assue that ρx = x 5, d v = 5 and the desired ε c th is 0.6. We concentrate here on QPECs with = q/2 +1 for several values of q this value of is the sallest satisfying > q/2. An illustration of the iterative optiization process is provided in Figure 5. The plot shows the sequence of optiization runs of the QPEC* optiizer right, and the sequence of runs for the BEC optiizer left. The QPEC* LP approaches the target of ε c th = 0.6 fro above, and the BEC LP fro below. Note the approxiate linear behaviour of ε c th as a function of ε, rendering the iterated QPEC* LP as a sipler way for code design. As a consequence, reaching the desired QPEC threshold took typically fewer optiization instances with the QPEC* optiizer than with the BEC optiizer. The optiized variable degree distributions and their corresponding rates are listed in Table I, together with the values of ε and ε BEC th resulting in ε c th. When coparing the results, we observe that for all the paraeters checked the rates achieved by the QPEC* optiizer are strictly better than the rates resulting fro the BEC optiizer. Another interesting observation is that in soe cases the BEC optiizer required a λx polynoial with ore non-zero coefficients than the QPEC optiizer. To evaluate the decoding perforance in the practical setting of finite-length codes, we constructed rando parity-check atrices for varying code lengths and perfored 80 iterative decoding iterations using the essage-passing de-