On the Stopping Distance and the Stopping Redundancy of Codes

Transcription

1 On the Stoppng Dstance and the Stoppng Redundancy of Codes Moshe Schwartz Unversty of Calforna San Dego La Jolla, CA 92093, U.S.A. Abstract It s now well known that the performance of a lnear code C under teratve decodng on a bnary erasure channel (and other channels) s determned by the sze of the smallest stoppng set n the Tanner graph for C. Several recent papers refer to ths parameter as the stoppng dstance s of C. Ths s somewhat of a msnomer snce the sze of the smallest stoppng set n the Tanner graph for C depends on the correspondng choce of a party-check matrx. It s easy to see that s d, whered s the mnmum Hammng dstance of C, and we show that t s always possble to choose a party-check matrx for C (wth suffcently many dependent rows) such that s = d. We thus ntroduce a new parameter, termed the stoppng redundancy of C, defned as the mnmum number of rows n a party-check matrx H for C such that the correspondng stoppng dstance s(h) attans ts largest possble value, namely s(h) =d. We then derve general bounds on the stoppng redundancy of lnear codes. We also examne several smple ways of constructng codes from other codes, and study the effect of these constructons on the stoppng redundancy. Specfcally, for the famly of bnary Reed-Muller codes (of all orders), we prove that ther stoppng redundancy s at most a constant tmes ther conventonal redundancy. We show that the stoppng redundances of the bnary and ternary extended Golay codes are at most 34 and 22, respectvely. Fnally, we provde upper and lower bounds on the stoppng redundancy of MDS codes. I. INTRODUCTION The recent surge of of renewed nterest n the bnary erasure channel (BEC) s due n large part to the fact that t s the prme example of a channel over whch the performance of teratve decodng algorthms can be analyzed precsely. In partcular, t was shown n [3] that the performance of an LDPC code (and, n fact, any lnear code) under teratve decodng on the BEC s completely determned by certan combnatoral structures called stoppng sets. A stoppng set S n a code C s a subset of the varable nodes n a Tanner graph for C such that all the neghbors of S are connected to S at least twce. The sze s of the smallest stoppng set was termed the stoppng dstance of C n a number of recent papers [5], [7]. The stoppng dstance plays an mportant role n understandng the performance of a code under teratve decodng over the BEC, akn to the role played by the mnmum Hammng dstance d for maxmum-lkelhood decodng. Just as one would lke to maxmze the mnmum dstance d f maxmum-lkelhood or algebrac decodng s to be used, so one should try to maxmze the stoppng dstance s n the case of teratve decodng. There s, however, an mportant dfference between the mnmal dstance d and the stoppng dstance s. Whle the former s a property of a code C, the latter depends on the specfc Alexander Vardy Unversty of Calforna San Dego La Jolla, CA 92093, U.S.A. vardy@klmanjaro.ucsd.edu Tanner graph for C or, equvalently, on the specfc choce of a party-check matrx H for C. In order to emphasze ths, we wll henceforth use s(h) to denote the stoppng dstance and d(c) to denote the mnmum dstance. In algebrac codng theory, a party-check matrx H for a lnear code C usually has n dm(c) lnearly ndependent rows. However, n the context of teratve decodng, t has been already observed n [8], [10] and other papers that addng lnearly dependent rows to H can be advantageous. Certanly, ths can ncrease the stoppng dstance s(h). Thus, throughout ths paper, a party-check matrx for C should be understood as any matrx H whose rows span the dual code C. Then the redundancy r(c) of C may be defned as the mnmum number of rows n a party-check matrx for C. Analogously, we defne the stoppng redundancy ρ(c) of C as the mnmum number of rows n a party-check matrx H for C such that s(h) =d(c). Ths work may be thought of as the frst nvestgaton of the trade-off between the parameters ρ(c), r(c), and d(c). In the next secton, we frst show that the stoppng redundancy ρ(c) s well-defned. That s, gven any lnear code C, t s always possble to fnd a party-check matrx H for C such that s(h) =d(c). In fact, the party-check matrx consstng of all the nonzero codewords of the dual code C has ths property. Hence ρ(c) 2 r(c) 1 for all bnary lnear codes. We then show that f d(c) 3, then any party-check matrx H for C satsfes s(h) =d(c), so ρ(c) =r(c) n ths case. The man result of Secton II s an extenson of ths smple observaton to a general upper bound on the stoppng redundancy of lnear codes (Theorem 4). In Secton III, we study several smple ways of constructng codes from other codes, such as the drect-sum constructon and code extenson by addng an overall party-check. We nvestgate the effect of these constructons on the stoppng redundancy. Note that although we have lmted our dscusson to bnary codes, most of the results n Sectons II and III extend straghtforwardly to lnear codes over an arbtrary fnte feld. We contnue n Secton IV wth an n-depth analyss of the well-known (u, u + v) constructon, and n partcular ts applcaton n the recursve defnton [6, p. 374] of bnary Reed- Muller codes. By slghtly modfyng ths constructon, we establsh a strong upper bound on the stoppng redundancy of Reed-Muller codes of arbtrary orders. Specfcally, we prove that f C s a Reed-Muller code of length 2 m and order r, then ρ(c) d(c)r(c)/2. Thus for any constant d(c), wehave an ncrease n redundancy by only a constant factor.

2 In Secton V, we study the (, 12, 8) extended bnary Golay code G. We prove that ρ(g ) 34 by provdng specfc party-check matrces for ths code. We take G as a test case, and compare the performance of three dfferent decoders: a maxmum-lkelhood decoder, an teratve decoder usng the conventonal 12 double-crculant party-check matrx of [6, p.65], and an teratve decoder usng the 34 partycheck matrx wth maxmum stoppng dstance. In each case, exact analytc expressons for the probablty of decodng falure are derved usng a computer program (see Fgure 1). We conclude n Secton VI wth a bref dscusson and a lst of open problems. Some of our results on stoppng redundancy of lnear codes that are not ncluded here due to space lmtatons are also brefly mentoned n Secton VI. II. GENERAL BOUNDS We begn wth rgorous defntons of the stoppng dstance and the stoppng redundancy. Let C be a bnary lnear code and let H =[h, j ] be a party-check matrx for C. The correspondng Tanner graph T for C s a bpartte graph wth each column of H represented by a varable node and each row of H represented by a check node n such a way that the j-th varable node s connected to the -th check node f and only f h, j 0. As already mentoned, a stoppng set n T s a subset S of the varable nodes such that all the check nodes that are neghbors of a node n S are connected to at least two nodes n S. We dspense wth ths graphcal representaton of stoppng sets n favor of an equvalent defnton drectly n terms of the underlyng party-check matrx H. Thus we say that a stoppng set s a set of columns of H wth the property that the projecton of H onto these columns does not contan a row of weght one. The resultng defnton of the stoppng dstance the smallest sze of a stoppng set bears a strkng resemblance to the defnton of the mnmum Hammng dstance of a lnear code. Recall that the mnmum dstance of a lnear code C can be defned as the largest nteger d(c) such that every d(c) 1 or less columns of H are lnearly ndependent. For bnary codes, ths s equvalent to sayng that d(c) s the largest nteger such that every set of d(c) 1 or less columns of H contans at least one row of odd weght. Defnton 1. Let C be a lnear code and let H be a party-check matrx for C. Then the stoppng dstance of H s defned as the the largest nteger s(h) such that every set of s(h) 1 or less columns of H contans at least one row of weght one. The followng corollary s an mmedate consequence of juxtaposng the defntons of s(h) and d(c) above. Corollary 1. Let C be a lnear code and let H be an arbtrary party-check matrx for C. Then s(h) d(c). Indeed, t s well known [3], [4], [5] that the support of every codeword s a stoppng set, whch s another way to see that s(h) d(c) regardless of the choce of H. Thus gven a lnear code C, the largest stoppng dstance one could hope for s d(c), no matter how cleverly the Tanner graph for C s constructed. The pont s that ths bound can be always acheved by addng dependent rows to H (see Theorem 2). Ths makes the noton of the stoppng dstance, as a property of a code C, somewhat meanngless: wthout restrctng the number of rows n a party-check matrx for C, we cannot dstngush between the stoppng dstance and the conventonal mnmum dstance. Ths observaton, n turn, leads to the followng defnton. Defnton 2. Let C be a lnear code wth mnmum Hammng dstance d(c). Then the stoppng redundancy of C s defned as the the smallest nteger ρ(c) such that there exsts a partycheck matrx H for C wth ρ(c) rows and s(h) =d(c). The followng theorem shows that the stoppng redundancy s, ndeed, well-defned. Theorem 2. Let C be a lnear code, and let H denote the party-check matrx for C consstng of all the nonzero codewords of the dual code C. Then s(h )=d(c). Proof: Let [C ] denote the n C matrx consstng of all the codewords of C. It s well known (cf. [6, p.139]) that [C ] s an orthogonal array of strength d(c) 1. Ths means that any set of t d(c) 1 columns of [C ] contans all the vectors of length t among ts rows, each vector appearng the same number of tmes. In partcular, any set of d(c) 1 or less columns of [C ] contans all the vectors of weght one. Theorem 2 also provdes a trval upper bound on the stoppng redundancy. In partcular, t follows from Theorem 2 that ρ(c) 2 r(c) 1 for any bnary lnear code C. Ths bound holds wth equalty n the degenerate case of the sngle-partycheck code. The next theorem determnes ρ(c) exactly for all bnary lnear codes wth mnmum dstance d(c) 3. Theorem 3. Let C be a bnary lnear code wth mnmum dstance d(c) 3. Then any party-check matrx H for C satsfes s(h) =d(c), and therefore ρ(c) =r(c). Proof: If H contans an all-zero column, then t s obvous that s(h) =d(c) =1. Otherwse s(h) 2, snce then every sngle column of H must contan a row of weght one. Now, f d(c) =3, then every two columns of H are dstnct. Ths mples that these two columns must contan ether the 01 row or the 10 row (or both). Hence s(h) =3. The followng theorem, whch s our man result n ths secton, shows that Theorem 3 s, n fact, a specal case of a general lower bound on the stoppng redundancy of lnear codes. Theorem 4. Let C be a bnary lnear code wth mnmum dstance d(c) 3. Then r(c) r(c) r(c) ρ(c) (1) 1 2 d(c) 2 Proof: We frst prove a slghtly weaker result, whch s conceptually smpler. Namely, let us show that r(c) r(c) r(c) ρ(c) (2) 1 2 d(c) 1 Let H be an arbtrary party-check matrx for C wth r(c) lnearly ndependent rows. Construct another party-check matrx H whose rows are all the lnear combnatons of t rows of H,

3 for all t = 1, 2,..., d(c) 1. Clearly, the number of rows of H s gven by the rght-hand sde of (2). Now let H t, respectvely H t, denote a matrx consstng of some t columns of H, respectvely the correspondng t columns of H. Observe that for all t d(c) 1, thet columns of H t are lnearly ndependent. Ths mples that the row-rank of H t s t, and therefore some t rows of H t must form a bass for F2 t. Hence the 2t 1 nonzero lnear combnatons of these t rows of H t generate all the nonzero vectors n F2 t, ncludng all the vectors of weght one. But for t d(c) 1, the2 t 1 nonzero lnear combnatons of any t rows of H t are among the rows of H t by constructon. Ths proves that s(h )=d(c) and establshes (2). To transton from (2) to (1), observe that we do not need to have all the nonzero vectors of F2 t among the rows of H t ;t would suffce to have at least one vector of weght one. Gven asets F2 t and a postve nteger m, letms denote the set of all vectors obtaned as a lnear combnaton of at most m vectors from S. Defne μ(t) as the smallest nteger wth the property that for any bass B of F2 t, the set μ(t)b contans at least one vector of weght one. Then n the constructon of H, t would suffce to take all the lnear combnatons of at most μ(d(c) 1) rows of H. Clearly μ(t) t 1 for all t (n fact, μ(t) =t 1 for all t), and the theorem follows. The bound of (1), whle much better than ρ(c) 2 r(c) 1, s stll too general to be tght for most codes. Nevertheless, we can conclude from Theorem 4 that when d(c) s a constant, the stoppng redundancy s only polynomal n the (conventonal) redundancy and, hence, n the length of the code. An obvous queston s whether we can do substantally better than Theorem 4. At least n the case of Reed-Muller codes, we shall see n Secton IV that the answer s yes. III. CONSTRUCTIONS OF CODES FROM OTHER CODES In ths secton, we examne several smple ways of constructng codes from other codes. Whle for most such constructons, t s trval to determne the redundancy of the resultng code, we fnd t consderably more dffcult to determne the resultng stoppng redundancy, and resort to boundng t. We start wth two smple examples. The frst example (Theorem 5) s the well-known drect-sum constructon or, equvalently, the (u, v) constructon. The second one (Theorem 6) s the (u, u) constructon, or concatenaton of a code wth tself. Both theorems have smple constructve proofs whch we omt. Theorem 5. Let C 1, C 2 be (n 1, k 1, d 1 ), (n 2, k 2, d 2 ) bnary lnear codes, respectvely. Then C 3 = {(u, v) : u C 1, v C 2 } s an (n 1 + n 2, k 1 + k 2, mn{d 1, d 2 }) code wth ρ(c 3 ) ρ(c 1 )+ρ(c 2 ) (3) Theorem 6. Let C 1 be an (n, k, d) bnary lnear code. Then the code C 2 = {(u, u) : u C 1 } s a (2n, k,2d) code wth ρ(c 2 ) ρ(c 1 )+n (4) Here s an nterestng observaton about Theorems 5 and 6. It follows from (3) and (4) that f the consttuent codes are optmal, n the sense that ther stoppng redundancy s equal to ther redundancy, then the resultng code s also optmal. Ths ndcates that the bounds n (3) and (4) are tght. In contrast, the nnocuous constructon of extendng a lnear code C by addng an overall party-check [6, p.27] appears to be much more dffcult to handle. The next theorem deals only wth the specal case where d(c) =3. Theorem 7. Let C be an (n, k,3) bnary lnear code. Then the extended code C s an (n + 1, k,4) code wth ρ(c ) 2ρ(C) =2r(C ) 2 (5) Proof: Let H be an arbtrary r(c) n party-check matrx for C. We construct a party-check matrx for C as follows H H 0 = (6) H 1 where H s the btwse complement of H, whle 0 and 1 are the all-zero and the all-one column vectors, respectvely. Label the columns n H by 1, 2,..., n + 1, and let I be a subset of {1, 2,..., n + 1} wth I 3. In fact, t would suffce to consder the case where I {1, 2,..., n} and I = 3; all other cases easly follow from the fact that s(h) =3 by Theorem 3. Let H(I) and H(I) denote the projectons of H and H, respectvely, on the three postons n I. IfH(I) contans a row of weght one, we are done. If H(I) contans a row of weght two, we are also done then the correspondng row n H(I) has weght one. But otherwse, the only rows n H(I) are 000 and 111, whch means that the three columns n H(I) are dentcal, a contradcton snce d(c) =3. The constructon n (5) and (6) s not optmal. For example, f C s the (8, 4, 4) extended Hammng code, t produces a party-check matrx for C wth 6 rows. However, C s also the Reed-Muller code R(1, 3) for whch we gve n the next secton a party-check matrx H wth s(h) =4 and only 5 rows. IV. REED-MULLER CODES We now focus on the well-known (u, u + v) constructon, n partcular n connecton wth the recursve defnton of bnary Reed-Muller codes. Our goal s to derve a constructve upper bound on the stoppng redundancy of R(r, m) the bnary Reed-Muller code of order r and length 2 m. We begn by recallng several well-known facts. Frst, for all r = 0, 1,..., m, the dmenson of R(r, m) s k = r (m ) and ts mnmum dstance s d = 2 m r.letg(r, m) be a generator matrx for R(r, m). Then, usng the (u, u + v) constructon, G(r, m) can be defned recursvely, as follows: G(r, m) def G(r, m 1) G(r, m 1) = 0 G(r 1, m 1) wth the recurson n (7) beng bootstrapped by G(m, m)=i 2 m and G(0, m) =(11 1) for all m. By conventon, the code R( 1, m) s the set {0} for all m. Then R(r, m) = R(m r 1, m) (8) for all m and all r = 1, 0, 1,..., m. It follows from (8) that G(r, m) s a party-check matrx for R(m r 1, m), a code wth mnmum dstance 2 r+1. Hence every 2 r+1 1 columns of G(r, m) are lnearly ndependent. (7)

4 Our objectve n what follows s to construct an alternatve party-check matrx H(r, m) for R(m r 1, m) =R(r, m) such that s(h(r, m)) = 2 r+1. Then the number of rows n H(r, m) gves an upper bound on the stoppng redundancy of R(m r 1, m). Here s the recursve constructon that we use. Recursve Constructon A: For all postve ntegers m and for all r = 1,2,...,m 2, we defne: H(r, m) = H H(r, m 1) H(r, m 1) top def = 0 H(r 1, m 1) (9) H bot H(r 1, m 1) 0 wth the recurson n (9) beng bootstrapped as follows: for all m = 0,1,..., the matrces H(0, m), H(m 1, m), H(m, m) are defned by H(0, m) def = G(0, m) =(11 1) (10) H(m 1, m) def = G(m 1, m) (11) H(m, m) def = G(m, m) =I 2 m (12) We omt the proofs of the next two propostons and lemma. Proposton 8. H(r, m) s a generator matrx for R(r, m) and, hence, a party-check matrx for R(m r 1, m). Proposton 9. The stoppng dstance of H(r, m) s 2 r+1 for all postve ntegers m and for all r = 0, 1,..., m 1, The remanng task s to compute the number of rows n the matrx H(r, m). We denote ths number as g(r, m). Lemma 10. For all r = 0, 1,..., m 1, the number of rows n H(r, m) s gven by m r 1 + g(r, m) = 2 r We are now n a poston to summarze the results of ths secton n the followng theorem. Theorem 11. For all m = 1, 2,... and for all r = 0, 1,..., m, the stoppng redundancy of R(r, m) s upper bounded by ρ ( R(r, m) ) ( r + m r 1 ) 2 (13) Proof: Follows mmedately from (8), Proposton 8, Proposton 9, and Lemma 10. To see how far Theorem 11 s from the (conventonal) redundancy of Reed-Muller codes, let us make a smple calculaton. For ths, t wll be more convenent to work wth the dual code C = R(r, m). Recall that the redundancy of C s r (m ). Comparng ths to the bound on ρ(c) n (13), we fnd that ( m r 1 + ρ(c) )2 2 r r m = 2 r r(c) r Therefore, for any fxed order r, the stoppng redundancy of R(r, m) s at most the redundancy of R(r, m) tmes a constant. Alternatvely, f we take C = R(r, m), then Theorem 11 mples that ρ(c) d(c)r(c)/2. Thus for any fxed d(c), the ncrease n redundancy s by a constant factor. V. GOLAY CODES The (, 12, 8) bnary Golay code G s arguably the most remarkable bnary block code. It s often used as a benchmark n studes of code structure and decodng algorthms. There are several canoncal party-check matrces for G, see [1], [2], [9] and other papers. Our startng pont s the systematc double-crculant matrx H gven n MacWllams and Sloane [6,p.65] and shown n Table I. It can be readly verfed that s H = 4, whch means that H acheves only half of the maxmum possble stoppng dstance. Curously, the stoppng dstance of the two trells-orented party-check matrces for G, gven n [9, p. 2060] and [1, p.1441], s also 4. Computng the bound of Theorem 4 for the specal case of G produces the rather weak result: ρ G Havng tred several methods to construct a party-check matrx for G wth stoppng dstance 8, our best result was acheved usng a greedy (lexcographc) computer search. Specfcally, wth the 4095 nonzero vectors of G lsted lexcographcally, we teratvely construct the party-check matrx H, at each teraton adjonng to H the frst vector on the lst wth the hghest score. Each vector receves ponts to ts score for each yet uncovered -set t covers, where {1,2,...,7}.The resultng matrx s gven n Table I. TABLE I TWO PARITY-CHECK MATRICES FOR THE (, 12, 8) GOLAY CODE G H = H = To evaluate the effect of ncreasng the stoppng dstance, t would be nterestng to compare the performance of teratve decoders for G based on H or H, respectvely. As a baselne for such a comparson, t would be also useful to have the performance of a maxmum-lkelhood decoder for G.In what follows, we gve analytc expressons for the performance of the three decoders on the bnary erasure channel (BEC). Clearly, a maxmum-lkelhood decoder fals to decode (recover) a gven erasure pattern f and only f ths pattern contans the support of (at least one) nonzero codeword of G.

5 Decoder-Falure Probablty TABLE II NUMBER OF UNDECODABLE ERASURE PATTERNS BY WEIGHT w IN THREE DECODERS FOR G w Total Patterns Ψ ML (w) Ψ H (w) Ψ H (w) ( w ) ( w ) ( w ) ( w ) maxmum-lkelhood decoder teratve decoder based upon H teratve decoder usng H Erasure Probablty Fg. 1. The decodng falure probablty of three decoders for G :amaxmum-lkelhood decoder and teratve decoders based upon H and H Let Ψ ML (w) denote the number of such erasure patterns as a functon of ther weght w. Then Pr ML {decodng falure} = Ψ ML (w) p w (1 p) w where p s the erasure probablty of the BEC. In contrast, an teratve decoder (based on H or H ) fals f and only f the erasure pattern contans a stoppng set. Thus Pr H {decodng falure} = Ψ H (w) p w (1 p) w 10-4 Pr H {decodng falure} = Ψ H (w) p w (1 p) w where Ψ H (w) and Ψ H (w) denote the number of erasure patterns of weght w that contan a stoppng set of H and H, respectvely. It remans to compute Ψ H, Ψ H, and Ψ ML. Obvously, Ψ ML (w) =0 for w 7 and Ψ ML (w) =( w ) for w 13 (any 13 columns of a party-check matrx for G are lnearly dependent). For the other values of w, wehave w 11 Ψ ML (w) = w ( )+2576 w = 12 where we made use of Table IV of [2]. To fnd Ψ H ( ) and Ψ H ( ), we used exhaustve computer search. These functons are gven n Table II. The resultng probabltes of decodng falure are plotted n Fgure 1. Note that whle we may add rows to H to elmnate more stoppng sets, ths would have neglgble effect snce the slope of the performance curve s domnated by the smallest w for whch Ψ H (w) 0. VI. FURTHER RESULTS AND OPEN PROBLEMS Ths paper only scratches the surface of the many nterestng and mportant problems that arse n the nvestgaton of stoppng redundancy. Here s a representatve sample: Determne the stoppng redundancy of well-known codes wth substantal algebrac and/or combnatoral structure. In partcular, s t true that ρ(g )=34? It appears that provng lower bounds on the stoppng redundancy, even for specfc codes such as G, s qute dffcult. Is the constructon devsed for bnary Reed-Muller codes n Secton IV optmal? More generally, for whch famles of codes can one fnd party-check matrces wth only O ( r(c) ) rows and stoppng dstance equal to d(c)? Are there codes wth non-vanshng rate and normalzed dstance, whose stoppng redundancy s O ( r(c) )?We can answer ths wth a no, n the case where the dual codes also have non-vanshng normalzed dstance. REFERENCES [1] A.R. Calderbank, G.D. Forney, Jr., and A. Vardy, Mnmal tal-btng trellses: the Golay code and more, IEEE Trans. Inform. Theory, vol. 45, pp , July [2] J.H. Conway and N.J.A. Sloane, Orbt and coset analyss of the Golay and related codes, IEEE Trans. Inform. Theory, vol. 36, pp , September [3] C. D, D. Proett, I.E. Telatar, T.J. Rchardson, and R. Urbanke, Fntelength analyss of low-densty party-check codes on the bnary erasure channel, IEEE Trans. Inform. Theory, vol. 48, pp , Jun [4] J. Feldman, Decodng Error-Correctng Codes va Lnear Programmng Ph.D. Thess, Massachusetts Insttute of Technology, September [5] N. Kashyap and A. Vardy, Stoppng sets n codes from desgns, n Proc. IEEE Internat. Symp. Informaton Theory, Yokohama, Japan, July [6] F.J. MacWllams and N.J.A. Sloane, The Theory of Error-Correctng Co-des. Amsterdam: North-Holland, [7] A. Orltsky, K. Vswananathan, and J. Zhang, Stoppng set dstrbuton of LDPC code ensembles, IEEE Trans. Inform. Theory, vol. 51, pp , March [8] N. Santh and A. Vardy, On the effect of party-check weghts n teratve decodng, n Proc. IEEE Internatonal Symposum Informaton Theory, Chcago, IL., July [9] A. Vardy, Trells structure of codes, Chapter n the HANDBOOK OF CODING THEORY, V. Pless and W.C. Huffman (Edtors), Elsever, [10] J.S. Yedda, J. Chen, and M. Fossorer, Generatng code representatons sutable for belef propagaton decodng, n Proc. 40-th Allerton Conference Commun., Control, and Computng, Montcello, IL., October 2002.