Explicit Interleavers for a Repeat Accumulate Accumulate (RAA) code construction

Transcription

1 Eplicit Interleavers for a Repeat Accumulate Accumulate RAA code construction Venkatesan Gurusami Computer Science and Engineering University of Wasington Seattle, WA 98195, USA venkat@csasingtonedu Widad Macmouci Computer Science and Engineering University of Wasington Seattle, WA 98195, USA idad@csasingtonedu Abstract Repeat Accumulate Accumulate RAA codes are turbo-like codes ere te message is first repeated k 2 times, passed troug a first permutation called interleaver, ten an accumulator, ten a second permutation, and finally a second accumulator Bazzi, Madian, and Spielman 2003 prove tat RAA codes are asymptotically good it ig probability en te to permutations are cosen at random RAA codes admit linear-time encoding algoritms, and are peraps te simplest knon family of linear-time encodable asymptotically good codes An eplicit construction of an asymptotically good RAA code is tus a very interesting goal We focus on te case en k = 2 and e consider a variation of RAA codes ere te inner repeat accumulate code is systematic We give an eplicit construction of te first permutation for ic e so tat te resulting code is asymptotically good it ig probability en te second permutation is cosen at random Te eplicit construction uses a cubic Hamiltonian grap it logaritmic girt I INTRODUCTION Repeat Accumulate RA codes [DJM98] are turbo-like codes it te folloing encoding: te message is repeated k times, ere k is called te repetition factor of te code, ten te repeated message is passed troug a first permutation π 1 and fed to an accumulator An accumulator takes a binary string a 1, a 2,, a m and outputs te binary string b 1, b 2,, b m ere b i = i j=1 a j In [BMS03], Bazzi, Madian and Spielman so tat suc a code is asymptotically bad, ie te minimum distance doesn t gro linearly it te bock lengt Repeat Accumulate Accumulate RAA are etensions of RA codes studied, for eample, in [BDMP98], [DJM98], [PS99], [BMS03] To get RAA codes, te output bits from te RA code are passed troug a second permutation π 2 and ten fed to a second accumulator Definition 1: [BMS03] Let k 2 and n > 0 be to integers and let m = kn Let r k : {0, 1} n {0, 1} kn be te encoder of te repetition code it repetition factor k and let A : {0, 1} m {0, 1} m be te encoder of te accumulator code given by: Aa = i j=1 a j m i=1 ere a = a 1a 2 a m {0, 1} m Ten te RAA code it repetition factor k and permutations π 1 and π 2 is te code ose encoder is C k,π1,π 2 : {0, 1} n {0, 1} m Aπ 2 Aπ 1 r k In [BMS03], te autors prove tat en π 1 and π 2 are cosen uniformly at random, C k,π1,π 2 as, it ig probability, a n kn kn kn kn kn r k π 1 A π 2 A Fig 1 Encoding sceme of C k,π1,π 2 minimum distance linear in te block lengt Teorem 1 [BMS03]: Let k 2 and n be integers, and let π 1 and π 2 be to permutations of lengt kn cosen uniformly at random Ten for eac constant δ > 0, tere eists a constant ε > 0, suc tat te RAA code encoded by C k,π1,π 2 as minimum distance at least εn it probability at least 1 δ for large enoug n Etensions of RAA codes are studied also in [CKZ07] ere te autors prove tat te gap to Gilbert-Varsamov bound can be made arbitrarily small by serially concatenating RAA codes it multiple accumulators and random permutations In tis paper, e ill consider a different version of tese RAA codes We use te inner-systematic RAA code, C s k,π 1,π 2, given by te folloing map: C s k,π 1,π 2 : {0, 1} n {0, 1} k+1n Aπ 2, Aπ 1 r k Note tat, altoug te repetition factor is still k, te block n r k kn π 1 kn A n π 2 kn + n A kn Fig 2 Encoding sceme of C s k,π 1,π 2 kn + n lengt and te lengt of π 2 are k + 1n We use systematic RA codes for tecnical convenience A Problem motivation and contet RA codes ave te advantage of a simple structure and etremely simple linear time encoding algoritm Hoever, it is knon tat teir structure is too simple to yield asymptotically good codes Indeed, a direct application of Teore in [BMS03] on RA codes te convolutional encoder described in te teorem is no an accumulator it repetition factor k and message lengt n gives a minimum distance d =

2 On 1 2/k log n, ic is not linear in te block lengt kn Namely, for k = 2, te distance is bounded by d = Olog n One of te motivations beind studying RAA codes as to determine eter, unlike RA codes, tey could include asymptotically good codes, ie, eter teir minimum distance could gro linearly it te block lengt for a suitable coice of te interleavers By Teorem 1, RAA codes can be asymptotically good, ic raise te folloing interesting question problem tat as te motivation for our ork: Can one find to eplicit permutations π 1 and π 2 suc tat te resulting code C k,π1,π 2 as a minimum distance linear in te block lengt? Finding suc permutations ould give us an eplicit asymptotically good code it linear time encoding So far, te construction of Spielman [Spi96] based on a cascade of epander graps is te only knon eplicit construction of linear-time encodable codes tat are asymptotically good Our ope is to investigate if RAA codes, ic ave admit linear time encoding by design, can be made eplicit, ile also being asymptotically good B Summary of results We focus on te case k = 2 We construct an eplicit permutation π 1 for ic e so tat a random permutation π 2 gives, it ig probability, a linear minimum distance for te inner-systematic RAA code C s 2,π 1,π 2 We divide te result into to main parts In te first part, e derive properties of a binary linear code C suc tat te code C tat maps {0, 1} n to Aπ 2 C as, it ig probability, a good minimum distance Specifically, e prove: Lemma 1: Let n be a positive integer and c > 1, d and l be positive constants Let π 2 be a permutation cosen uniformly at random and A te encoder of te accumulator code Let C be a binary linear code it message lengt n and block lengt cn Let C be te code it te folloing encoder: C : {0, 1} n {0, 1} cn, Aπ 2 C If C satisfies te folloing properties: 1 minimum distance property: C as minimum distance at least log dn 2 eponential eigt distribution property: Te number of codeords in C of eigt is at most l Ten, for every constant δ > 0, tere eists a constant ε > 0 dependent only on δ, c, d and l, suc tat for all large enoug n, te code C as minimum distance at least εn it probability 1 δ, ere te probability is taken over te uniform random coice of π 2 Te net result gives an eplicit construction of a code C satisfying bot properties We use an RA code it repetition factor 2, ere te permutation π 1 is constructed from a cubic Hamiltonian grap it logaritmic girt g Proving tat suc systematic RA codes satisfy te conditions relies on te fact tat k = 2 We set C to be te systematic version of C 2,π1, ere e append te original message to te output of te code We prove tat te number of codeords of eigt is at most eponential in Moreover, using tecniques from [BMS03] and [FK04], e prove tat te minimum distance of te systematic version is g In particular, e so tat: Lemma 2: let n be a positive integer and let C 2,π1 be an RA code it permutation π 1 Let C be te block lengt-3n code ose encoder maps {0, 1} n to C 2,π1, Ten, for infinitely many values of n, tere eist an eplicit construction of π 1 from a cubic Hamiltonian grap it logaritmic girt suc tat: 1 C as minimum distance at least log 2n 2 Te number of codeords in C of eigt is at most 16 Combining te to lemmas above and setting c to 3, d to 2 and l to 16 in Lemma 1, e get te main teorem of tis ork: Teore: Let n be a positive integer and π 2 a permutation on 3n elements cosen uniformly at random Let C s 2,π 1,π 2 be te inner-systematic RAA code it k = 2, first permutation π 1 constructed from a cubic Hamiltonian grap it logaritmic girt, as eplained in Section III, and second permutation π 2 Ten for every constant δ > 0, tere eists a constant ε > 0, suc tat, for infinitely many n, C s 2,π 1,π 2 as minimum distance at least εn it probability 1 δ, ere te probability is taken over te random coice of π 2 C Organization of rest of te paper In Section II, e prove Lemma 1 using te probabilistic metod and properties of te accumulator code In Section III, e give an eplicit description of te code C by constructing te permutation π 1 of te RA code from a cubic Hamiltonian grap it logaritmic girt II SERIALLY CONCATENATING A WEAK CODE WITH AN ACCUMULATOR In tis section, e prove Lemma 1 We assume te eistence of te code C it te required properties No, e ill permute te bits of te codeords of C and feed tem to an accumulator We ant to find a permutation π 2 so tat te minimum distance at te output of te accumulator, ie te minimum distance of C, is linear in te block lengt We ill so tat suc permutation eists by te probabilistic metod We follo te same tecnique used in [BMS03] to prove tat RAA codes ave good minimum distance en bot permutations are cosen at random Let C be te code described in Lemma 1: C : {0, 1} n {0, 1} cn, Aπ 2 C We ill calculate te probability tat C as minimum distance less tan εn By Markov s inequality, te probability tat tere eists a nonzero codeord of eigt less tan εn is bounded from above by te epected number of codeords of eigt less tan εn We ill denote tis latter epectation by E εn Let α, denote te probability tat a random cn bit input string of eigt leads to Hamming eigt eigt at te

3 accumulator s output By linearity of epectation, e clearly ave E εn = εn =1 =log dn X α, ere X denotes te number of codeords it input eigt Note tat te upper bound of is set to 2εn: for a binary string of eigt at te input of an accumulator, te output A ill ave eigt at least 2 Hence e get a codeord of eigt εn only if te input eigt of te codeord is at most 2εn In [DJM98], te autors calculate te number of codeords of an accumulator code of input eigt and output eigt, denoted A, If N is te block lengt of te accumulator, ten A N, = N Back to α 1,, e get: α, = Acn, = Using y, y 2 y e get Ten α, < E εn < = < y/2, 4e y y 2 1 4cen εn = εn /2 4e /2 = =1 =log dn =log dn =log dn =log dn y/2 4e y and y 4e 4e X X 4e εn =1 /2 X 4e εn εn /2 X 4e ε Using te eponential eigt property of te code C: X l, e get E εn < εn =log dn For 4le ε < 1 2, ie ε < c 4 3 e 2 l 2, e get E εn < εn =log dn c 4le ε 2 < εn2 log dn+4 = 16ε d To sum up, for ε < 4 3 e 2 l, e ave son tat te probability 2 tat te minimum distance of C εn is at least 1 16ε d Tus c by picking ε < min{ 4 3 e 2 l, δd 2 16 }, e can conclude tat C as minimum distance at least εn it probability at least 1 δ, as desired In te above proof, e assumed te eistence of te code C it te minimum distance and te eponential eigt distribution properties In te folloing section, e construct suc codes from systematic RA codes it repetition factor 2, and Lemma 1 ill apply by setting c = 3, d = 2 and l = 16 III SYSTEMATIC RA CODES FROM CUBIC HAMILTONIAN GRAPHS In tis section, e construct codes satisfying te properties needed in te serial concatenation sceme used in section 2 Tese codes are systematic RA codes ose permutation π 1 is constructed from cubic Hamiltonian graps it logaritmic girt Te repetition factor k is set to 2 Te construction and proof eavily use te fact tat k = 2 We ill so tat te systematic RA code, C s 2,π 1, as te requisite minimum distance and eponential eigt distribution properties: 1 C s 2,π 1 as minimum distance at least log 2n 2 Let X is te number of codeords in C s 2,π 1 of eigt Ten X 16, for all If n is te message lengt of te systematic RA code, te block lengt is 3n: 2n from te output of te accumulator and n from te appended message Te construction of te permutation π 1 is based on te construction presented in [BMS03] and adapted in [FK04] for RA codes A Construction Te construction uses a cubic Hamiltonian undirected grap G = V, E it logaritmic girt Constructions of suc graps ere proposed by Erdös and Säcs [Big98] based on a greedy algoritm For a message lengt n, G as 2n vertices and 3n edges We remove te edge v 1, v 2n for tecnical convenience tat e eplain later Te nodes represent te bits of te message after repetition Let v 1,,v 2n be te nodes of G and let = 1,, 2n be te repeated permuted version of a message m {0, 1} n, ten v i is associated it te bit i Let y be te output of te accumulator en applied to, ie y = C 2,π1 m = A All te edges along te broken Hamiltonian cycle ill be referred to as line edges Te remaining edges are referred to as matcing edges Te nodes at te endpoints of a matcing edge are repeated nodes, so if v i, v j is a matcing edge, ten i = j Te matcing edges are ordered from 1 to n so tat eac matcing edge corresponds to one of te n bits of te original message m To encode, e set to 1 te nodes of eac matcing edge corresponding to 1 in te input message and to 0 te remaining nodes in te grap Te bits are ten entered in te nodes order to te accumulator Figure 3 sos te grap G and o te nodes and edges correspond to te bits of te message and te codeord To summarize, e ave te folloing: m {0, 1} n is te original message, = π 1 r 2 m {0, 1} 2n, y = A {0, 1} 2n and C2,π s 1 m = y, m {0, 1} 3n G = V, E is te grap ere V = 2n and E = 3n If v i, v i+1, v j V i < j and v i, v i+1, v i, v j E,

4 v 2, v j is a matcing edge ten 2 = j v j v 2n v 1 v 2 v 2n 1 m 1 m i matcing edges corresponding to te bits of m Fig 3 Grap G line edges corresponding to te bits of y ten v i is associated it i, te line edge v i, v i+1 is associated it y i and te matcing edge v i, v j is associated it m l, for some l {1,,n} B Minimum Distance To calculate te minimum distance of C s 2,π 1, e ill so te equivalence beteen a codeord and a union of disjoint simple cycles: te eigt of a codeord corresponds to te total lengt of te cycles Tis correspondence is a variation of tat in [BMS03] and [FK04] adapted to RA codes For eac nonzero codeord y, m, construct te grap G y as follos: If m i = 1, add te matcing edge corresponding to m i to G y If y j = 1, add te line edge v j, v j+1 to G y Note tat te line edge v 1, v 2n is never picked since y 2n = 1 2 2n = 0 m 1 v 1 v 2 v 3 v 4 v 5 v m 6 3 v 2 v 3 v 4 v 5 v 6 Fig 4 An eample soing o to construct G y for y, m = , 011 Te top grap is G and te bottom one G y We ill prove tat G y is a union of disjoint cycles and ten deduce te minimum distance from te equivalence of codeords and unions of disjoint cycles Lemma 3: Let y, m be a codeord in C s 2,π 1 and let G y be te subgrap of G corresponding to y, m as eplained above Ten 1 G y is a union of disjoint cycles of lengt equal to te Hamming eigt of y, m, denoted ty, m 2 Eac union of disjoint cycles in G correspond to a codeord of C s 2,π 1 Proof: 1 By te construction of G y, te number of edges in G y equals ty + tm = ty, m Let v i G y v i is connected to 3 edges in G: te to line edges v i 1, v i and v i, v i+1 and te matcing edge v i, v j m 3 Let m l be te bit in m corresponding to te matcing edge v i, v j Note tat i = j = m l We ave to cases for m l : If m l = 1, ten i = 1 and te matcing edge v i, v j is in G y Note tat y i 1 y i since y i = y i 1 i = y i 1 1 Hence only one of te to line edges v i 1, v i and v i, v i+1 appears in G y Terefore v i is connected to eactly to edges in G y If m l = 0, ten i = 0 and te matcing edge v i, v j is not in G y y i 1 = y i since y i = y i 1 i = y i 1 0 Hence bot edges v i 1, v i and v i, v i+1 appear in G y since v i G y Terefore v i is only connected to te to line edges in G y Tus all nodes in G y ave degree 2 Tis implies tat G y as is a disjoint union of cycles 2 Eac cycle in te union sould ave at least one matcing edge since te line edge v 1, v 2n is removed Setting to 1 te bits corresponding to te endpoints of eac matcing edge and to 0 te remaining bits gives us a binary string, ere = π 1 r 2 m for some codeord y, m in C2,π s 1 : te matcing edges correspond to te 1-bits in te message m and te endpoints of te matcing edges ill correspond to te 1-bits in Tese bits come in pair repetition factor 2 since bot bits corresponding to te endpoints are set simultaneously Hence, te codeord y, m ill correspond to te union of cycles considered by te construction of G y eplained above Note tat if e did not remove te edge v 1, v 2n from G, a cycle may contain v 1, v 2n Tis ould imply tat y 2n = 1, ic is not true, and ence te equivalence beteen codeords and union of disjoint simple cycles breaks Combining all te above, e get te folloing variation of te codeords-cycles correspondence in [BMS03], [FK04], adapted to systematic RA codes: Corollary 1: Let G be a cubic Hamiltonian grap it girt g and let C s 2,π 1 be te systematic RA code ose permutation π 1 is constructed from G as eplained above Ten C s 2,π 1 as minimum distance equal to te girt g of G C Number of codeords of eac eigt We no so tat C s 2,π 1 as te eponential eigt distribution property By te equivalence of codeords and cycles, e prove tat te number of unions of disjoint cycles in te cubic Hamiltonian grap is at most eponential in te total lengt of tese cycles In particular, e so tat: Lemma 4: X 16, for all, ere X is te number of codeords in C s 2,π 1 of eigt Proof: Let G be te cubic Hamiltonian grap used as te permutation π 1 G as 2n vertices and a girt g = log 2n Our goal is to bound X Since C s 2,π 1 as minimum distance log 2n, X = 0 for all < log 2n Recall from Lemma 3 tat X is equal to te number of unions of disjoint simple cycles of total lengt To simplify counting, e ill consider cycles

5 it ordered nodes and not necessarily simple and disjoint cycles For log 2n, let Z be te number of unions of ordered cycles not necessarily simple and disjoint it ordered nodes of total lengt Tus, X Z We ill bound Z by induction on Let C be te number of single cycles not necessarily simple it ordered nodes of lengt For = g = log 2n, C = Z since a union of cycles of lengt equal to te girt sould contain one cycle only 1 Bound on C : For a cycle of lengt, e ave at most 2n coices for te first verte, ic as 3 coices for its neigbor Te last verte as one coice only, te first verte Te remaining 2 vertices eac as 2 coices We get: C 2n = 6n log 6n 2 1 since g = log 2n 2 Bound on Z : We ill so by induction on tat Z 4 2 Te base case is en = g and Z g = C g = 4 g 4 2g Assume te ypotesis is true for all l, g l i, e ill prove it true for i + 1 REFERENCES [BMS03] L Bazzi, M Madian, and D Spielman Te Minimum Distance of Turbo-Like Codes, preprint, 2003 To appear in IEEE Transactions on Information teory [BDMP98] S Benedetto, D Divsalar, G Montorsi, and F Pollara Analysis, design, and iterative decoding of double serially concatenated codes it interleavers, IEEE Journal on Selected Areas In Communications, Vol 16, No 2, February 1998 [Big98] NBiggs Construction of Cubic Graps it Large Girt Electronic Journal of Combinatorics, 5A1, 1998 [CKZ07] D J Costello, Jr, J Klieer, and K S Zigangirov Ne results on te minimum distance of repeat multiple accumulate codesproceedings of te Annual Allerton Conference on Communication, Control, and Computing, September 2007 [DJM98] D Divsalar, H Jin, and R McEliece Coding Teorems for Turbo-Like Codes Proceedings of te Annual Allerton Conference on Communication, Control, and Computing, pp , 1998 [FK04] J Feldman, and D Karger Decoding Turbo-Like Codes it Linear Programming Journal of Computer and System Sciences, Volume 68, Issue 4, June 2004 [PS99] H Pfister, and P H Siegel On te serial concatenation of rate-one codes troug uniform random interleavers37t Allerton Conference on Communication, Control, and Computing, September 1999 [Spi96] D Spielman Linear-time encodable and decodable error-correcting codes IEEE Transactions on Information Teory Volume 42, No 6, pp , 1996 Z i C l Z i l 4 l 4 2i l = 4 2i 4 l 2i 4 g = 4 2i g+1 4 2i since 4 g+1 1, g 1 Finally, e get X Z 16 for all Note tat te logaritmic girt becomes an essential condition in proving te upper bound on C IV CONCLUSIONS We gave an eplicit construction of a permutation π 1 suc tat te inner-systematic RAA code it first permutation π 1 is, it ig probability, asymptotically good, ere te probability is taken over te random coice of te second permutation π 2 Tis leads to te folloing questions: 1 Can te properties of te cubic Hamiltonian grap elp construct an eplicit permutation π 2, so tat te resulting inner-systematic RAA code as good minimum distance? 2 Can oter constructions of cubic Hamiltonian graps, eg, algebraic constructions, give more insigt on te construction of π 2 to acieve a good minimum distance?