The Stopping Redundancy Hierarchy of Cyclic Codes

Transcription

1 The Stopping Redundancy Hierarchy of Cyclic Codes Thorsten Hehn, Stefan Laendner, Olgica Milenkovic, and Johannes B. Huber Institute for Information Transmission University of Erlangen-Nuremberg Erlangen, Germany {hehn, Electrical and Computer Engineering Dept. University of Colorado at Boulder Boulder, CO, USA {laendner, Abstract We extend the framework for studying the stopping redundancy of a linear block code by introducing and analyzing the stopping redundancy hierarchy. The stopping redundancy hierarchy of a code represents a measure of the trade-off between performance and complexity of iteratively decoding a code used over the binary erasure channel. It is defined as an ordered list of positive integers in which the i- th entry, termed the i-th stopping redundancy, corresponds to the minimum number of rows in any parity-check matrix of the code that has stopping distance at least i. In particular, we derive lower and upper bounds for the i-th stopping redundancy of a code by using probabilistic methods and Bonferroni-type inequalities. Furthermore, we specialize the findings for cyclic codes, and show that parity-check matrices in cyclic form have some desirable redundancy properties. We also propose to investigate the influence of the generator codeword of the cyclic parity-check matrix on its stopping distance properties. I. INTRODUCTION Several parameters of linear block codes are known to exhibit a strong influence on their performance under iterative decoding. These sets include some well studied entities, such as the minimum distance of the code or the number of codewords of minimum weight, as well as some combinatorial sets that are important only if the decoder operates in an iterative fashion. One such class of combinatorial entities includes the so-called stopping sets [1]. When a code is used for signaling over the binary erasure channel (BEC, stopping sets determine the failure modes of iterative decoders. Stopping sets consist of subsets of nodes of the Tanner graph of a code and are characterized in terms of certain properties imposed on the columns of the parity-check matrix. Since a large number of rows in the parity-check matrix of a code ensures more flexibility in terms of meeting predefined constraints on the structure of the columns, several authors [], [3], [4], proposed using redundant parity-check matrices to improve the performance of iterative decoders. Unfortunately, adding redundant rows increases the overall complexity of decoding. It is therefore important to study the trade-off between the number of redundant rows and the size of the smallest stopping set in a parity-check matrix of the code. In this context, Schwartz and This work was in part supported by a German Academic Exchange Service (DAAD fellowship awarded to Thorsten Hehn, NSF Grant CCF and a fellowship by the Institute of Information Transmission (LIT at the University of Erlangen-Nuremberg, Germany, awarded to Stefan Laendner. Vardy [5] introduced the notion of the stopping redundancy of a code as the smallest number of codewords that span the dual of the code and constitute a matrix with no stopping sets of size smaller than the minimum distance of the code. They also provided lower and upper bounds on the stopping redundancy, the latter growing exponentially with the codimension of the code for most examples considered. This finding raised the question if there exist codes for which one could retain a small number of redundant rows in the parity-check matrix and sacrifice very little in terms of the size of the smallest stopping set. The first results in this direction were presented by Weber and Abdel-Ghaffar [6] as well as by Hollman and Tolhuizen [7], who posed the more complicated questions of determining the smallest redundancy of a parity-check matrix of a code that allows for decoding all correctable erasure patterns. One major drawback of the analysis provided in [6] and [7] is that it only shows that Hamming codes represent the hardest case for constructing redundant rows that eliminate a large number of stopping sets. Since Hamming codes are cyclic, and since the class of cyclic codes contains a large number of representatives with large minimum distance, and the codes themselves have a rich mathematical structure as well as simple encoding algorithms, it is of interest to analyze the distribution of stopping set sizes in such codes. We therefore restrict our attention to cyclic codes. The contributions of this work are two-fold. The first consists in introducing the notion of the stopping redundancy hierarchy of a code and in providing several general upper and lower bounds on the elements of this list. This hierarchy provides a new measure for the trade-off between performance and complexity of codes used over the BEC channel. The second contribution is in terms of characterizing the relationship between the stopping redundancy hierarchy of a cyclic code and the particular form of its cyclic paritycheck matrix. We show that it is advantageous to construct the parity-check matrix from a minimum-weight codeword of the dual code and a certain number of its cyclic shifts. Among the codes investigated are the [3, 1, 7] Golay code and several primitive BCH codes [8], [9]. The paper is organized as follows. Section II introduces relevant definitions and the terminology used throughout the paper, as well as the notion of the stopping redundancy hierarchy. The same section contains several upper and lower

2 bounds on the elements of the stopping redundancy hierarchy. Section III describes an extension of the approach used in [7] for computing the stopping redundancy of Hamming codes, applied to BCH codes. The results presented in this section show that using parity-check matrices of BCH codes in cyclic form provides significant gains in the stopping redundancy, when compared to the straightforward extension of the approach in [7]. A comprehensive study of the stopping redundancy hierarchy of cyclic codes is conducted in second part of Section III, while a set of simulation results is presented in Section IV. II. DEFINITION AND TERMINOLOGY In this section we introduce the stopping redundancy hierarchy and provide general upper and lower bounds on the elements in this list. Let H be an arbitrary parity-check m n matrix of a linear [n, k, d] code C, and let the columns of H be indexed by the integers J = {1,...,n}. For a set I J, we define the restriction of H to I as the m I array of elements composed from the columns of H indexed by I. Definition.1: For a given parity-check matrix H of C, a stopping set S(n(H of size σ is a set of σ columns for which the restriction of H does not contain rows of Hamming weight one. The stopping distance of a paritycheck matrix H is the size of the smallest stopping set in H. As described in [5], the stopping distance and the overall number of stopping sets depends on the particular choice of the parity-check matrix of the code. We introduce next the stopping redundancy hierarchy of a code which is independent of the choice of the parity-check matrix representation of C. Definition.: Let C be a linear code with minimum distance d. Forl d, the l-th stopping redundancy of C is the smallest integer ρ l (C such that there exists a paritycheck matrix H of C with ρ l (C rows and stopping distance at least l. The ordered set of integers (ρ (C,ρ 3 (C,...,ρ d (C is called the stopping redundancy hierarchy of C. The stopping redundancy hierarchy generalizes the notion of the stopping redundancy ρ d (C of a code C, first introduced in [5]. For codes with minimum distance at least three, no two columns of the parity-check matrix are identical nor is any of the columns equal to the all-zero vector. Therefore, ρ (C = ρ 3 (C = n k. Consequently, only stopping redundancies of order larger than three are of interest. We present next two results from [5] pertaining to the size of the stopping redundancy of a code. As we will show, these results can be generalized in the setting of the stopping redundancy hierarchy in a straightforward manner. Theorem.1 ([5]: Let C be a binary linear code with parameters [n, k, d], and let ω i =max{ (n +1/i 1,d },,...,d 1, where d denotes the minimum distance of the code dual to C. Then ( n ρ d (C ( i ω n ωi, for all i =1,...,d 1. i i 1 Furthermore, if d =3, ρ d = n k, while for d>3 one has ( n k ρ d (C ( n k. (1 d It is instructive to briefly repeat the arguments leading to the results in Theorem.1. The lower bound is established by a simple counting argument utilizing the fact that every subset of column-indices that does not correspond to a stopping set must have a restriction with at least one row of weight one. The proof of the upper bound is constructive. One starts with an arbitrary parity-check matrix H of the code C and then successively adds all sums of not more than d 1 distinct rows of H. A little technical improvement [5] allows to stop adding these combinations after only d terms are encountered. We present straightforward upper and lower bounds on the l-th stopping redundancy, l =4,...,d, of any binary linear code. The lower bound given below follows directly from Theorem.1, and therefore its proof is omitted. Theorem.: Let C be a binary [n, k, d] linear code, and let ω i =max{ (n +1/i 1,d },,...,l 1, where d denotes the minimum distance of the dual code of C. Then ( n ρ l (C ( i ω n ωi for all i =1,...,l 1. i i 1 Note that this bound tends to be very loose, especially for small values of l. In certain cases, the parity-check matrix of a code is fixed and one is required to estimate the performance of the code represented by the given matrix. In this case, it is important to find the distribution of stopping set sizes of the given matrix. If formulas for counting stopping sets of a given size are available, then the stopping redundancy computation becomes a straightforward problem. We show how bounds on the number of stopping sets of size σ in a parity-check matrix H of dimension m and length n can be derived using Bonferroni-type inequalities [10]. Definition.3: For a given parity-check matrix of a code, we say that the matrix resolves a set of coordinates if those coordinates do not correspond to a stopping set. The number of resolved sets of coordinates in a paritycheck matrix H of dimension m can be estimated from the intersections of zero- and one-entries of its rows: the number of resolved subsets of size σ in a matrix H is the cardinality of the union of all subsets resolved by the individual rows of H. The cardinality of this union can be found in terms of the well known inclusion-exclusion formula.

3 Theorem.3 ([10]: Let χ(a be the indicator function of a set A with respect to a universal set Ω. Furthermore assume that V is a set of indices {1,..., V } and I is a subset of it. Then the principle of inclusion-exclusion (PIE states for a family of sets {A v } v V that ( χ A v = ( ( 1 I 1 χ A i. v V I V i I I The PIE can be used to find the number of stopping sets resolved by a matrix of size m, by replacing the set A i by the set Σ i of stopping sets resolved by the i-th row. Note that Σ i depends on the size σ of the stopping sets. If V denotes the set of row indices of H, V = {1,,...,m}, then the overall number of stopping sets resolved by these rows can be expressed as m Σ i = ( 1 I 1 i I Σ i I V I m = ( 1 j 1 S j ( with j=1 S j = ( m j ι j=1 s ιj, where 1 ι j ( m j enumerates the elements in a set of all possible collections of j rows. The symbol s ιj denotes the number of stopping sets resolved by collection ι j, i.e. the number of stopping sets resolved simultaneously by all j rows in this collection. The number of stopping sets resolved jointly by all collections ι j with 1 ι j ( m j is given by S j. Making use of a bound from [11], we state the following theorem. Theorem.4: The number of stopping sets resolved by a union of parity-checks can be upper bounded by m m Σ i Σ i m i, j =1 i<j Applying a Bonferroni-type inequality to the results of Theorem.3 simplifies the calculation at the cost of the accuracy of the expression. A Bonferroni-type inequality is obtained by neglecting terms with intersections of a size greater than a threshold β, i.e. retaining only terms for which the subset size I fulfills I β [10]. Applying this bounding technique to Eq. ( is conducted by limiting j to 1 j β. Setting β =would lead to m Σ i (1 S 1 +( 1 m S, which leads directly to Eq. (3. Similar results exist for the case that only terms with more than three intersections are neglected, i.e. β =3is set. These m Σ i Σ j. (3 results will not be presented in detail but will be used to produce simulation results presented in Section IV. Lemma.5: The number of stopping sets resolved by one row i is given by Σ i = ( ω 1 ( n ω σ 1 The number of stopping sets resolved simultaneously by two rows i and j is ( ( 1i 1 j 0i 0 j Σ i Σ j = + 1 σ 1 ( ( ( 1i 0 j 0i 1 j 0i 0 j, 1 1 σ where 1 l,l {i, j} denotes the set of positions of ones in row l, and 0 l denotes the set of positions of zeros in row l. Since a stopping set of size σ in a given set of column positions is resolved by a combination of a single one and (σ 1 zeros in these positions, the i-th row of length n and weight ω resolves Σ i stopping sets what proofs the first part of Lemma.5. A pair of rows can simultaneously cover the same stopping set either if both rows have a 1 in the same position, and zero values at all the remaining positions in the stopping set, or if within the support of the stopping set the two rows share all but two positions of 0 symbols, and have two non-overlapping 1 s. This idea leads to the proof of the second part of Lemma.5. Example.6: Consider a parity-check matrix H of the form shown below. ( H = Columns {1,, 3} have two restrictions 100, while column sets {, 4, 5} and {3, 4, 5} both have a 001 in the first and 010 in the second row of the restriction, therefore both alternate a 0 and 1 in columns 4 and 5, simultaneously removing the same stopping set. All three stopping set combinations {1,, 3}, {, 4, 5}, and {3, 4, 5} are resolved by both rows, while other triples of rows are resolved at most once by one of the rows. Since one aims for a minimum number of intersections between the stopping sets resolved by different rows, i.e. a minimum number of common positions in any pair, triplet, etc. of rows, minimum weight codewords of the dual code C are used as rows of the parity-check matrix H of a code C. It is worth pointing out that for many codes, the codewords of a fixed weight follow a design [1]. In those cases, the intersection numbers of their minimum weight codewords can be obtained from designs, and these parameters are known for many classes of codes, e.g. quadratic residue codes or the Golay codes [1]. This idea will be discussed in more details elsewhere. Example.7: Consider the class of [ s 1,s, s 1 ] simplex codes. Since simplex codes are constant-weight codes,.

4 it is straightforward to see that for their dual codes - the class of Hamming codes - one has ( S = s (1 + s s 1 ( s 1. σ 1 σ 1 A similar result can be found for S 3, since all the sub-codes of a simplex code of the same support weight are equivalent. Upper bounds on the stopping redundancy hierarchy can be derived by invoking the techniques developed by one of the authors in [13], based on probabilistic techniques similar to those described in [4]. For this purpose, we need the following definitions and two results regarding Lovász Local Lemma [14], [15]. Lemma.8: Let E 1,E,...,E N be a set of events in an arbitrary probability space. Suppose that each event E i is independent of all other events E j, except for at most τ of them, and that P {E i } p for all 1 i N. If e p (τ +1 1, then P { N E i} > 0. Similarly, let E 1,E,...,E N be a set of events in an arbitrary probability space, and let 0 <ɛ<1. Suppose that each event E i is independent from all other events E j, except for at most τ of them. If P {E i } ɛ ( 1 ɛ τ, 1 i N, l l then P { N E i} > 1 ɛ. Based on Lovász Local Lemma, one can obtain the following bounds on the l-th stopping redundancy of a code, for l < d/. The derivations are straightforward, and based on associating stopping-set properties of restrictions of a parity-check matrix with the events E i described in the above lemma. The proofs are omitted. Theorem.9: Let C be an [n, k, d] code. If l <dand ( l ( 1+log(l 1 + log n 1 j j=1 m log ( 1 l 1 + n k l +1, l 1 then ρ l (C m. The result in Theorem.9 is non-constructive, since it relies on a probabilistic argument. One way to make this finding more useful for practical purposes is to generalize it in the framework of the high probability version of Lovász Local Lemma, as discussed in [13]. This generalization ensures that with probability close to one, any randomly chosen parity-check matrix with m rows chosen uniformly and at random from the set of codewords of the dual code has a stopping redundancy which exceeds some prescribed value. This result is more formally stated in the following theorem. Theorem.10: Let C be an [n, k, d] code, and assume that m is greater than or equal to ( ( ɛ log +(l 1 log 1 ɛ l ( n 1 l 1 ( n l 1 i ( n i i i=0 log ( 1 l 1. l 1 Then the probability that a parity-check matrix consisting of m randomly chosen codewords of the dual code (with possible repetitions of a codeword has stopping distance l d/ is at least 1 ɛ. With at most n k l +1 additional rows added, the matrix also has rank n k and represents a parity-check matrix of the code C. As the application of Theorems.9 and.10 is restricted to l< d/ we also present another method to calculate an upper bound for the l-th stopping redundancy. Assume that one can identify a sub-code S of C with dual distance d S. Since the generators of a subcode S form a subset of the generators of C, one needs to apply the procedure of adding redundant rows leading to Theorem.1 or Theorem.9/.10 only to the basis vectors in S in order to ensure that the redundant matrix has stopping distance at least d S. This argument leads to the following result. Theorem.11: Let Θ be the set of all sub-codes of the dual code C of a linear [n, k, d] code C that have support weight 1 n and dual distance l. Furthermore, let the dimensions of the sub-codes in Θ be K i,,..., Θ and define K =min i K i. Then ρ l (C ( K ( K. (4 l III. A CASE STUDY: CYCLIC CODES Although the results in Section II can be applied to any linear code, they are very loose. We are interested in whether there exist specialized and tighter bounds for the class of cyclic codes, defined below. Definition 3.1: Let C be a [n, k, d] binary linear code of length n, dimension k, and minimum distance d. A code is called cyclic if any cyclic shift of a codeword c C is also a codeword. A parity-check matrix of a cyclic code is said to be in cyclic form if it consists of m cyclic shifts of one codeword of the dual of the code. A standard form for H with m = n k cyclic shifts is shown below H = A cyclic code can be completely described by a normalized generator polynomial g(x =g 0 g 1 x g x g n k 1 x n k 1 x n k F n k [x] of degree n k or a normalized parity-check polynomial h(x = h 0 h 1 x h x h k 1 x k 1 x k F k of degree k. The generator matrix G and parity-check matrix H of a cyclic code can be constructed by using the generator polynomial and parity-check polynomial with g µ,η = g(η µ mod n (x (η µ mod n!, x=0 1 The support weight of a sub-code of a code is defined as the number of positions for which at least one of the codewords of the sub-code has a non-zero value (5

5 h µ,η = h(k η+µ mod n (x (k η + µ mod n! x=0 where f (ζ (x denotes the ζ-th derivative of f for x and 1 µ m, 1 η n. There usually exist several choices for a codeword of the dual code whose n k cyclic shifts constitute a parity-check matrix of the underlying code. In order to efficiently identify code representations with a good stopping redundancy hierarchy, we consider redundant parity-check matrices obtained by including m>n k shifts of a given minimum weight codeword of the dual code. Theorem 3.1: For a parity-check matrix of cyclic form of size m n with m n, the number of stopping sets resolved by a union of parity-checks can be upper bounded by m Σ i m Σ µ (6 m m 1 (m κ Σ µ Σ ((µ+κmod m µ, κ=1 where mod denotes the (somewhat non-standard modulo function which assumes that m mod m = m instead of 0. We tacitly assume that µ and κ are chosen in such a way that row ((µ + κmod m is the κ-th cyclic shift of row µ. Note that Eq. (6 is just a specialization of Eq. (3 for parity-check matrices of cyclic form. Each row of such a parity-check matrix resolves the same number of stopping sets as the positions of the resolved stopping sets are cyclic shifts of each other. Furthermore, the number of stopping sets that are resolved by two rows i and j, i j simultaneously only depends on the number κ of cyclic shifts necessary to transform one of the rows into the other. Definition 3.: The set XY κ contains all the pairs of positions {p, p κ} with an X in position p and an Y in position p κ, 1 p n. Definition 3.3: Let H be a parity-check matrix of cyclic form of a cyclic [n, k, d] code C. The series h[η], 1 η n is an ordered sequence of coefficients of any arbitrary row of H. Lemma 3.: The number of stopping sets that are resolved by two rows of a parity-check matrix of cyclic form simultaneously is Σ µ Σ ((µ+κmod m = ( ( ( ( 11κ 00κ 10κ 01κ + 1 σ with 11 κ = n ω=1 1 ( 00κ σ n h[η](δ[η ω]+δ[η ω κ], η=1 µ, 00 κ = n ω=1 1 n (1 h[η] (δ[η ω]+δ[η ω κ], η=1 01 κ = 10 κ = [ ( n n 1 h[η](δ[η ω]+δ[η ω κ] ω=1 η=1 ] n (h[η](δ[η ω]+δ[η ω κ], and η=1 ( n n η=1 Σ µ = h[η] σ 1 n h[η], η=1 where δ[k] denotes the discrete Dirac impulse with δ[k] = 1 iff k =0. Note that we assume ( i j =0if i<j. The equations presented in Lemma 3. are only specializations of the results from Lemma.5 for the case of parity-check matrices of cyclic form. All equations regarding XY κ are obtained by simple counting rules. For the last equation in Lemma 3. it is furthermore considered that stopping sets are only resolved if the restriction of the paritycheck matrix on the positions of the stopping set contains a row of weight one. From Inequality (6 a lower bound on the number of rows that is needed to remove all (or a certain number of stopping sets of a given size from a parity-check matrix of cyclic form, can be obtained. In order to find tight upper bounds for the case of cyclic codes we do not attempt to derive analytic results but rather use the fact that one can come up with a parity-check matrix representation which constitutes an upper bound on the redundancy hierarchy of the underlying code. To this aim we will use representations in cyclic form and compare their redundancies to general bounds and bounds obtained by approaches known so far. A. Case study: A comparison of non-cyclic and cyclic paritycheck matrix representations Recently there has been a lot of interest in finding bounds on the stopping redundancy of a code. Vardy and Schwartz [5] presented an upper bound on the stopping redundancy of all codes, given in Eq. (1. Weber and Abdel-Ghaffar [6] considered Hamming codes and introduced 3-erasure correcting parity-check collections, resolving all correctable erasure patterns of a Hamming code with d =3. Hollman and Tolhuizen [7] generalized this result to generic ( m, σ erasure correcting sets A m, σ, consisting of all vectors {a = (a 1,a,...,a m a 1 =1, wt(a σ}. IfH is a paritycheck matrix of the dimension m σ, the collection of parity-checks {ah a A m, σ } resolves all correctable erasure patterns up to size σ. As BCH codes are the most widely used class of cyclic codes, we will evaluate the results mentioned above for this class of codes and compare them to results obtained with parity-check matrices of cyclic form.

6 Hamming codes are a subclass BCH codes with minimum distance d = 3 and thus a good starting point for this comparison. For this case we construct parity-check matrices of cyclic form and parity-check matrices that are generic ( m, σ erasure correcting sets according to [7]. As ρ (C = ρ 3 (C = n k and a stopping distance of four can not be reached, we compare the different representations by counting unresolved stopping sets of size σ = 3 when a certain value for the redundancy is given. Assume that the parity-check matrix of a Hamming code is given in terms of the row vector h Hamming = ( α 0 b α 1 b... α (n 1 b where α denotes a primitive element of the Galois field F n+1, n = n 1, for some positive integer n, i.e. α is a root of some primitive polynomial p α (x. The parameter b will be henceforth set to one. In order to form a binary parity-check matrix for the Hamming code, the elements of h Hamming are represented as vectors over the binary field F log (n+1, i.e. each element α x b,x {0,...,n 1}, is described in terms of a binary column vector. We will now modify the parity-check matrices of a Hamming code as described in [7] to eliminate all correctable stopping sets up to size d. The generic erasure correcting set A m, σ is constructed and it is assumed that m =log (n +1 and σ =3as we are interested in resolving stopping sets up to size σ =3only. Multiplication of A m, σ with the paritycheck matrix of the code results in a redundant parity-check matrix for a Hamming code, generally of non-cyclic form. This matrix has σ 1 ( m 1 (7 i i=0 rows and does not contain correctable stopping sets of up to size σ = σ =3, i.e. the remaining stopping sets of size σ =3correspond to Hamming codewords of weight d =3. For comparison we construct a redundant parity-check matrix in cyclic form, obtained by shifting a minimum weight codeword of the dual code m>n k times. To decide which approach produces better results, we set m equal to (7 and determine the total number of unresolved stopping sets of size σ =3. Tables I and II show these values for the chosen code lengths n =63and n = 17. # remaining stopping sets, size σ =3 m non-cyclic representation, [7] cyclic representation 6=n k (Eq. ( TABLE I NUMBER OF ROWS AND NUMBER OF STOPPING SETS OF SIZE σ =3FOR THE [63, 57, 3] HAMMING CODE WHEN USING A PARITY-CHECK MATRIX OF NON-CYCLIC FORM [7] AND CYCLIC FORM, RESPECTIVELY. # remaining stopping sets, size σ =3 m non-cyclic representation, [7] cyclic representation 7=n k (Eq. ( TABLE II NUMBER OF ROWS AND NUMBER OF STOPPING SETS OF SIZE σ =3FOR THE [17, 10, 3] HAMMING CODE WHEN USING A PARITY-CHECK MATRIX OF NON-CYCLIC FORM [7] AND CYCLIC FORM, RESPECTIVELY. It is to be observed that the results do not differ significantly and the performance of the different code representations appears to be almost identical, what is obtained from simulations. The fact that the cyclic representations perform very well compared to [7], although that approach was specifically developed for Hamming codes, implies that the cyclic approach might be well suited for erasure decoding in general. As the approach proposed in [7] was originally designed for the class of Hamming codes, which is the most difficult for constructing redundant rows [7], it will be suboptimal in terms of required redundancy when applied to a parity-check matrix of a BCH code. For this reason we propose a novel scheme to construct a parity-check matrix for BCH codes with low redundancy which uses the approach from [7] only for the part of the parity-check matrix which defines the Hamming code containing the BCH code of interest. We compare the determined upper bounds on the stopping redundancy hierarchy ρ l (C, l 5 for these two approaches to the hierarchy obtained when a parity-check matrix of cyclic form is used. The proposed construction consists of 4 steps. 1 A redundant parity-check matrix for a Hamming code that resolves all correctable stopping sets of size up to σ = d 1=4is created. To this end a generic ( m, σ erasure correcting set with m =log (n +1, σ =4is multiplied with the parity-check matrix of the Hamming code. The resulting matrix contains only unresolved stopping sets of size σ =3and σ =4which correspond to a Hamming codeword or include such a codeword. This matrix will be referred to as the Hamming matrix, H H, g. Figure 1 represents a visualization of this step. The Hamming matrix is transformed into the paritycheck matrix of a BCH-code with minimum distance d =5by appending additional rows to the matrix. This can be accomplished by appending the binary expansion of the row vector h BCH = ( α 0 (b+c α 1 (b+c... α (n 1(b+c to H H, g, where c is chosen according to the standard procedure for generating BCH codes (for example, one common choice for c is two. We will refer to this part of the overall parity-check matrix as H BCH. Figure presents a visualization of this step. We will now append an additional set of redundant rows to the matrix in order to resolve all stopping sets of size σ =3and σ =4. The fact that all unresolved stopping sets correspond to Hamming codewords is used when appending

7 H H, s Step 1 H H, g Fig. 1. Step 1: Transform the standard parity-check matrix for the Hamming code into a generic ( m, σ erasure correcting set according to [7] H H, g H H, g H BCH Step Step 3 Step 4 Fig.. Steps -4: Append H BCH (Step and additional parity-check equations to H H, g to remove stopping sets of size σ =3and σ =4(Step 3 and 4 judiciously chosen linear combinations of rows from either H BCH or both H H, g and H BCH. Let t denote the positions of an unresolved stopping set of size σ =3. The restriction of H H, g to t is of the form [ (α a ( α b( α a α b] with 0 a, b < n, a b. A restriction of H BCH to t is of the form [( α 3a ( α 3b ( (α a α b 3], where we have set c = for h BCH. Consider first all stopping sets for which α 3a = α 3b. Claim 3.3: If α 3a = α 3b, then (α a α b 3 = α 3a. The proof can be found in the appendix of the paper. Assume that n/3 is an integer. Then there exist exactly n/3 stopping sets corresponding to Hamming codewords of weight 3 with α 3a = α 3b as a can only be chosen in the range 0 <a<n/3. If we now consider binary representations of the parity-check matrices, it is easy to see that the restriction of H H, g to t only contains rows of weight 0 or and the same restriction on H BCH only contains rows of weight 0 or 3. These properties are used in step 3 of the proposed construction. 3 Each stopping set corresponding to Hamming codewords of weight 3 with α 3a = α 3b can be resolved by an appropriate linear combination of one row from H H, g and one row of H BCH, provided that n/3 is an integer. We conduct the search for appropriate summands in a greedy fashion, i.e. eliminate as many stopping sets as possible with each appended linear combination. Figure visualizes this step. Step 3 covers most of the stopping sets of size σ =3. In order to remove stopping sets of size σ =3from a code where n/3 is not an integer or those which do not satisfy the property α 3a = α 3b as well as stopping sets of size σ =4, we used a simple computer search strategy. This is due to the fact that these unresolved stopping sets do not have simple structural properties. 4 To the concatenation of H H, g, H BCH and the paritychecks added in step 3, additional rows are added. These rows are chosen in a greedy fashion. Figure provides a visualization for this step. Using this extended method it is possible to remove all stopping sets up to size σ =4from the parity-check matrix of a BCH code of minimum distance d =5. In Table III we compare an upper bound on the stopping redundancy hierarchy ρ l (C, l 5 determined by a cyclic approach and the construction method which is proposed in this section and based on [7]. Furthermore we compare the results to the required redundancy if the approach from [7] was directly applied to the BCH code matrix and to the general upper bound [5] given in Eq. (1. We have used these methods to parity-check matrices for the [63, 51, 5] BCH code as well as the [17, 113, 5] BCH code. U(ρ 4 (C Code Cyclic Construction [7] General [5] [63, 51, 5] [17, 113, 5] U(ρ 5 (C Code Cyclic Construction [7] General [5] [63, 51, 5] [17, 113, 5] TABLE III UPPER BOUND ON ρ l (C, l 5 Table III shows that the general bound [5] is very loose. The approach from [7] is tighter as it is optimized for Hamming codes which contain BCH codes but still suboptimal as the approach has to cover Hamming codes, for which it is most difficult to construct redundant rows. For this reason it can be outperformed by our construction method. Observe that the results for the proposed construction are much better for the [63, 51, 5] code as n/3 is an integer in this case and the structured methods from Step 3 can be applied to remove a significant number of stopping sets with very little redundancy. The bounds obtained with cyclic codes outperform all other bounds significantly. For this reason we will only consider cyclic codes in Section IV, as the results obtained above strongly indicate that such matrices have good stopping distance properties. The reason for this will be given in part in the next section. B. Stopping sets in cyclic parity-check matrices The following result provides an intuitive explanation why redundant parity-check matrices in cyclic form codes have good stopping set properties. Definition 3.4: Let the first row in a cyclic parity-check matrix H have a non-zero symbol in its first position. This can always be accomplished since the matrix is cyclic. The span of the first row, denoted by sp, is defined as the largest value j for which h 1,j =1. The zero-span z is defined as

8 the number of consecutive coordinates at the end of the first row containing zero symbols. Clearly, sp + z = n. In order to properly distinguish between parity-check matrices of cyclic form, we introduce the notion of a cyclic group generator. Definition 3.5: Let C be an [n, k, d] binary linear code. Partition the set of codewords of the dual code C into groups consisting only of cyclic shifts of one codeword. Define one of the codewords as the according representative for that group, denoted as the cyclic group generator (cgg. Consider a code C and one of the parity-check matrices of cyclic form. The latter consists of a cyclic group generator and its first m 1 cyclic shifts, m>0. The resulting matrix of cyclic form has i 1 leading zeros and z i +1 tailing zeros in row i, 1 i m. A stopping set is resolved by such a matrix if the matrix contains a row of Hamming weight 1 in the restriction of the parity-check matrix to the indices of the stopping set. Such a matrix of cyclic form containing m = z +1 rows resolves any stopping set with coordinates confined to the set {1,,...,z+1}, as can be seen from (5. For any stopping set with largest column index r z + 1, the r-th row of the matrix has an entry 1 in column r, and zeros in all positions {1,,...,r 1}. Therefore, the rows in the matrix resolve all ( z+1 σ stopping sets of size σ with support contained in the set {1,,...,z +1}. Equivalently, for all stopping sets with coordinates confined to the set {n z,n z+1,...,n}, let the smallest coordinate in the stopping set be l. Then row l (n z has a 1 in the leftmost position l, and zeros in all remaining positions of the stopping set. For a matrix of cyclic form containing more than z +1 consecutive shifts of this cyclic group generator, the above results can be generalized in a straightforward manner. As can be easily seen, a matrix of m = n cyclic shift rows resolves all stopping sets lying within z +1 consecutive column positions. IV. RESULTS In this section we present lower bounds on the stopping redundancy hierarchy for parity-check matrices of cyclic form, as well as upper bounds on the stopping redundancy hierarchy. Furthermore simulation results for a set of selected cyclic codes including the [3, 1, 7] Golay code and BCH codes of minimum distance five and seven are given. A. Golay Code To obtain a cyclic representation of the parity-check matrix of the Golay code, the 506 codewords of minimum weight eight in the dual of the Golay code with parameters [3, 11, 8] are arranged into groups of codewords which are cyclic shifts of each other. The codewords of weight eight in the dual [3, 11, 8] code are equal to the codewords of weight eight of the [3, 1, 7] binary Golay code and form sets of 3 codewords each. The representations obtained with different cyclic group generators vary with respect to the number of stopping sets and give different bounds on the stopping redundancy hierarchy. While for example representations based on some cyclic group generators do not contain any stopping sets of size σ =3when choosing m =10cyclically shifted rows, others require m =13rows to resolve all these stopping sets. We use the smallest value obtained by search for an upper bound on the stopping redundancy hierarchy of the codes investigated. For the [3, 1, 7] binary Golay code we investigate all parity-check matrix representation of cyclic form, based on different cyclic group generators of weight eight. We will concentrate on the best and worst performing representations and refer to them by letters A to D. Representation A provides the best result in terms of resolving stopping sets and is obtained from the cyclic shifts of the group representative cgg 3,1,A =[113500](octal representation, most significant bit on the left side with polynomial representation g(x =1+x 4 + x 6 + x 10 + x 1 + x 13 + x 14 + x 16. The parity-check matrix representation with the worst stopping set characteristics among the cyclic group generators of weight eight is formed by shifts of cyclic group generator D given by cgg 3,1,D = [ ]. Upper bounds U(ρ j (C on the stopping redundancy hierarchy obtained from cyclic group generators A to D of the paritycheck matrix of the [3, 1, 7] binary Golay code and the general upper bound obtained from Eq. (1 are listed in Table IV. U(ρ 4 (C U(ρ 5 (C U(ρ 6 (C U(ρ 7 (C cgg 3,1,A n k cgg 3,1,B cgg 3,1,C n k cgg 3,1,D n k 16 1 > 3 Overall n k Eq. ( TABLE IV UPPER BOUNDS ON THE STOPPING REDUNDANCY HIERARCHY OF THE [3, 1, 7] BINARY GOLAY CODE. As can be seen in Table IV, the Golay code has a stopping redundancy hierarchy for which ρ 4 (C n k, ρ 5 (C 15,ρ 6 (C 18,ρ 7 (C 3. A comparison of the performance in terms of residual bit error rate after decoding when using the parity-check matrix representation A are shown in Figure 3 for representations with 11, 16, 18, and 3 rows that correspond to the upper bounds on the stopping redundancy hierarchy obtained from this cyclic representation. Lower bounds L(ρ j (C on the redundancy of matrices of cyclic form are obtained from Eq. (3 and (6. For each representation the best bounds obtained this way are shown in Table V. As can be seen from the tables, the lower bounds on the stopping redundancy hierarchy are very loose and depend on the choice of the cyclic group generator.

9 cgg 17,113,C = [ ] BER cgg 17,113,D = [ ]. Upper bounds on stopping redundancy hierarchies obtained from these representations are given in Table VI cgg A, 11 rows cgg A, 16 rows cgg A, 18 rows cgg A, 3 rows ER Fig. 3. Performance of the [3, 1, 7] Golay code using parity-check matrix representation A with 11, 16, 18, and 3 rows. L(ρ 4 (C L(ρ 5 (C L(ρ 6 (C L(ρ 7 (C cgg 3,1,A cgg 3,1,B cgg 3,1,C cgg 3,1,D Overall TABLE V COMPARISON OF LOWER BOUNDS ON THE STOPPING REDUNDANCY HIERARCHY FOR PARITY-CHECK MATRIX REPRESENTATIONS OF CYCLIC FORM, DETERMINED FOR THE [3, 1, 7] BINARY GOLAY CODE B. BCH codes We present upper bounds on the stopping redundancy hierarchies and performance simulation results for the [17, 113, 5] BCH code and the [31, 16, 7] BCH code with minimum distance d =5and d =7, respectively. Similarly as for the Golay code, only minimum weight codewords of the dual code will be used to create the parity-check matrix in cyclic form [3], and four selected cyclic group generators A to D will be considered in detail. For the [17, 113, 5] BCH code, 457 codewords of the dual code have minimum weight 56 and can be separated into 36 cyclic groups. Similarly there exist 15 cyclic groups of codewords in the dual of the [31, 16, 7] BCH code, each with 31 codewords of minimum codeword weight eight. Comparing all possible parity-check matrix representations, representation A performs best in terms of number of resolved stopping sets. This representation is constructed by cgg 17,113,A, denoted in octal representation with most significant bit on the left side: cgg 17,113,A = [ ] A representation with similar properties is based on the cyclic group generator cgg 17,113,B while two cyclic generators cgg 17,113,C and cgg 17,113,D with poor stopping set properties are listed below. cgg 17,113,B = [ ] U(ρ 4 (C U(ρ 5 (C cgg 17,113,A 0 34 cgg 17,113,B 34 cgg 17,113,C 3 46 cgg 17,113,D 45 Overall 0 34 Eq. ( TABLE VI UPPER BOUND ON THE STOPPING REDUNDANCY HIERARCHY FOR THE [17, 113, 5] BCH CODE As can be seen in Table VI, the [17, 113, 5] BCH code has a stopping redundancy hierarchy for which ρ 4 (C 0, ρ 5 (C 34. Results for the [31, 16, 7] BCH code using four out of 15 cyclic group generators are presented in Table VII. All the cyclic group generators have weight eight. The cyclic group generators are given in the following. cgg 31,16,A = [ ] cgg 31,16,B = [ ] cgg 31,16,C = [ ] cgg 31,16,D = [ ]. U(ρ 4 (C U(ρ 5 (C U(ρ 6 (C U(ρ 7 (C cgg 31,16,A n k cgg 31,16,B n k 16 0 cgg 31,16,C n k n k 0 8 cgg 31,16,D n k Overall n k n k 19 1 Eq. (1, ( TABLE VII UPPER BOUND ON THE STOPPING REDUNDANCY HIERARCHY FOR THE [31, 16, 7] BCH CODE As can be seen in Table VII, the [31, 16, 7] BCH code has a stopping redundancy hierarchy for which ρ 4 (C n k, ρ 5 (C n k, ρ 6 (C 19, ρ 7 (C 1. Performance results for the [31, 16, 7] BCH code are presented in Figure 4, for the representations A and C of Table VII with the number of rows set to the smallest listed bound. Taking only the representation A into account, it can be observed that the most significant improvement is achieved when the matrix has m =18instead of m =15

10 rows. These results indicate that all cyclic representations for the short codes we considered perform similarly with respect to the residual bit error rate. Nevertheless, for codes of longer length and larger codimension, the particular choice of the cyclic group generator may have a significant bearing on the stopping set characteristics. ( u 1 ( α in/u u 1 = α in/u i,j i<j α in/u α jn/u. Since the field has characteristic two, the last term in the above expression can be omitted. Finally, BER ER 0.1 cgg A, 15 rows cgg A, 18 rows cgg A, 19 rows cgg A, 1 rows cgg C, 15 rows cgg C, 1 rows Fig. 4. Performance of the [31, 16, 7] BCH code using parity-check matrix representation A with 18, 19, and 1 rows as well as representation C with 15 and 1 rows. Lower bounds on the stopping redundancy hierarchy of the [31, 16, 7] and the [17, 113, 5] BCH code for a paritycheck matrix of cyclic form are derived using Eqs. (3 and (6. For each code the best bound obtained is shown in Table VIII. Unlike for the Golay codes, the choice of the representation for the parity-check matrices does not affect the bounds. Code L(ρ 4 (C L(ρ 5 (C L(ρ 6 (C L(ρ 7 (C [17, 113, 5] BCH code 5 6 [31, 16, 7] BCH code TABLE VIII COMPARISON OF LOWER BOUNDS ON THE STOPPING REDUNDANCY HIERARCHY FOR PARITY-CHECK MATRIX REPRESENTATIONS OF CYCLIC FORM, DETERMINED FOR THE [17, 113, 5] BCH CODE AND THE [31, 16, 7] BCH CODE. APPENDIX 0.08 Proof: (α a α b 3 = α 3a if α 3a = α 3b. (α a α b 3 = α 3a 3α a+b 3α a+b α 3b = α a+b α a+b since the underlying field has characteristic two. Furthermore, 0.06 α a+b α a+b = α 3a ( α b a α a b. As 3a 3b = kn mod n, a b = k n 3 mod n and b a = k n 3 mod n. To complete the proof, we have to show that for each odd u, u {k +1 k N} which divides n, one has ( α in/u =1. Observe that u u 1 α in/u = (u 1/ (u 1/ = (u 1/ = u 1 = α in/u, α in/u + α in/u + α in/u + u 1 i=(u+1/ (u 1/ i =1 (u 1/ i =1 α in/u α (i +u/ 1/n/u α (i 1n/u where we have introduced i = i u/ +1/ and used the fact that α n =1. This completes the proof. REFERENCES [1] C. Di, D. Proletti, I. Telatar, T. Richardson, and R. Urbanke, Finite length analysis of low-density parity-check codes, IEEE Trans. on Inform. Theory, vol. 48, no. 6, pp , June 00. [] R. Koetter, Iterative coding techniques, pseudocodewords, and their relationship, in Workshop on Applications of Statistical Physics to Coding Theory, Santa Fe, New Mexico, January 005. [3] N. Santhi and A. Vardy, On the effect of parity-check weights in iterative decoding, in Proc. of the IEEE Internat. Symp. on Inform. Theory, Chicago, Illinois, July 004, p. 3. [4] J. Han and P. Siegel, Improved upper bounds on stopping redundancy, submitted to IEEE Trans. on Inform. Theory, November 005. [5] M. Schwartz and A. Vardy, On the stopping distance and stopping redundancy of codes, submitted to IEEE Trans. on Inform. Theory, March 005. [6] J. Weber and K. Abdel-Ghaffar, Stopping and dead-end set enumerators for binary Hamming codes, in Proceedings of the Twenty-sixth Symp. on Inform. Theory in the Benelux, Brussels, Belgium, May 005, pp [7] H. Hollman and L. Tolhuizen, On parity check collections for iterative erasure decoding that correct all correctable erasure patterns of a given size, submitted to IEEE Trans. on Inform. Theory, July 005. [8] A. Hocquenghem, Codes correcteurs d erreurs, Chiffres, vol., pp , September [9] R. Bose and D. Ray-Chaudhuri, On a class of error correcting binary group codes, Information and Control, vol. 3, pp , March [10] K. Dohmen, Improved Bonferroni Inequalities via Abstract Tubes. Springer-Verlag, 003. [11] S. Kwerel, Most stringent bounds on aggregated probabilities of partially specified dependent probability systems, J. Amer. Statist. Assoc., vol. 70, pp , [1] F. MacWilliams and N. Sloane, The Theory of Error-Correcting Codes. North-Holland Publishing Company, [13] O. Milenkovic, E. Soljanin, and P. Whiting, Stopping and trapping sets in generalized covering arrays, in Proceedings of the 40th annual Conference on Information Sciences and Systems (CISS, March 006. [14] D. Deng, D. Stinson, and R. Wei, The Lovász local lemma and its applications to some combinatorial arrays, Designs, Codes and Cryptography, vol. 3, no. 1-3, pp , May 004. [15] N. Alon and J. Spencer, The Probabilistic Method, ser. Interscience Series in Discrete Mathematics and Optimization. John Wiley, 000.