Ripple Design of LT Codes for AWGN Channel

Transcription

1 MITSUBISHI ELECTRIC RESEARCH LABORATORIES Rppe Desgn of LT Codes for AWGN Channe Sorensen, J.H.; Koke-Akno, T.; Ork, P. TR Juy 212 Abstract In ths paper, we present an anaytca framework for desgnng LT codes n addtve whte Gaussan nose AWGN) channes. We show that some of anaytca resuts from bnary erasure channes BEC) aso hod n AWGN channes wth sght modfcatons. Ths enabes us to appy a rppe-based desgn approach, whch unt now has ony been used n the BEC. LT codes desgned by ths way show promsng performance whch s near the Shannon mt even wth short codewords. IEEE Intternatona Symposum on Informaton Theory Proceedngs ISIT) Ths work may not be coped or reproduced n whoe or n part for any commerca purpose. Permsson to copy n whoe or n part wthout payment of fee s granted for nonproft educatona and research purposes provded that a such whoe or parta copes ncude the foowng: a notce that such copyng s by permsson of Mtsubsh Eectrc Research Laboratores, Inc.; an acknowedgment of the authors and ndvdua contrbutons to the work; and a appcabe portons of the copyrght notce. Copyng, reproducton, or repubshng for any other purpose sha requre a cense wth payment of fee to Mtsubsh Eectrc Research Laboratores, Inc. A rghts reserved. Copyrght c Mtsubsh Eectrc Research Laboratores, Inc., Broadway, Cambrdge, Massachusetts 2139

2 MERLCoverPageSde2

3 Rppe Desgn of LT Codes for AWGN Channe Jesper H. Sørensen, Toshak Koke-Akno, Php Ork, Jan Østergaard, and Petar Popovsk Aaborg Unversty, Department of Eectronc Systems, Fredrk Bajers Vej 7, 922 Aaborg, Denmark Mtsubsh Eectrc Research Laboratores MERL), 21 Broadway, Cambrdge, MA 2139, USA E-ma: {jhs, jo, {koke, Abstract In ths paper, we present an anaytca framework for desgnng LT codes n addtve whte Gaussan nose AWGN) channes. We show that some of anaytca resuts from bnary erasure channes BEC) aso hod n AWGN channes wth sght modfcatons. Ths enabes us to appy a rppe-based desgn approach, whch unt now has ony been used n the BEC. LT codes desgned by ths way show promsng performance whch s near the Shannon mt even wth short codewords. I. INTRODUCTION LT codes [1] were the frst practca exampes of a rateess erasure correctng code whch approaches capacty for ncreasng message ength. Raptor codes [2] were ater deveoped as an extenson of LT codes. Such rateess codes may potentay generate an nfnte amount of encoded symbos from fnte nput symbos. An mportant eement n the desgn of both LT and Raptor codes for the bnary erasure channe BEC) s a parameter caed the rppe. The performance depends sgnfcanty on how ths parameter evoves durng decodng, and thus successfu desgns have mosty focused on ths. Athough LT codes were orgnay deveoped for the BEC, severa works have recenty focused on desgnng such codes for nosy channes. Desgn of Raptor codes for the bnary symmetrc channe BSC) and bnary-nput addtve whte Gaussan nose B- AWGN) channe s treated n [3]. Ths desgn s based on the Gaussan approxmaton method [4], whch s used to derve constrants for the degree dstrbuton of the LT code. In [5, 6], approaches based on EXIT charts are apped to the desgn of LT and Raptor codes, respectvey. Another desgn of Raptor codes for arbtrary symmetrc channes s presented n [7], where an anaogue to the rppe s defned as the ncrease n correct bt estmates n a gven decodng round. The desgn s based on fndng an optma vaue for ths measure. In ths paper, we present an anaytca framework for the desgn of LT codes n the AWGN channe. It has strong anaoges to the framework presented n [1] for the BEC. Interestngy, we show that key anaytca resuts n the BEC aso hod even n the AWGN channe. Ths enabes us to make a rppe-based desgn for the AWGN channe wth sght modfcatons, whch expot characterstcs unque n AWGN. The man contrbuton of ths work s a brdge between the work n the BEC and the AWGN channe, and t can hep further desgn extensons for such nosy channes. II. BACKGROUND OF LT CODES In ths secton, an overvew of LT codes s presented. Assume we wsh to transmt a gven amount of data, whch s dvded nto k nput symbos. From these nput symbos a potentay nfnte amount of encoded symbos, caed output symbos, are generated. Output symbos are excusveor XOR) combnatons of nput symbos. The number of nput symbos used n the XOR s caed the degree of the output symbo, and a nput symbos contaned n an output symbo are caed neghbors of the output symbo. The output symbos foow a certan degree dstrbuton, Ωd). The encodng process can be broken down nto three steps: 1) Randomy choose a degree d by sampng Ωd). 2) Choose unformy at random d of the k nput symbos. 3) Perform XOR of the d chosen nput symbos. The resutng symbo s the output symbo. Ths process can be terated as many tmes as needed, that resuts n a rateess code. A. Decodng n AWGN Channe A wdey used decoder for LT codes s a beef propagaton BP) decoder. For the BP decoder, messages are passed between neghborng symbos,.e. from output symbos to nput symbos or vce versa. Such a message refects the current beef of the sender on the vaue of an nput symbo. The beef s quantfed ) by the og-kehood rato LLR), defned as n PrX =1 Y ) PrX = Y ), where X s the -th bnary nput symbo and Y s the AWGN channe output symbos vector of potentay nfnte ength. The o-th output symbo s denoted as Y o. The -th round of the BP decoder starts wth a output symbos passng a message, m o, to a ther neghborng nput symbos. Based on those messages, the nput symbos pass a message, m,o, back to a ther neghborng output symbos. These rounds contnue unt a specfed stop crteron has been reached, e.g. a certan number of rounds or a target error rate. The beef messages are updated as foows [3]: m,o = o o m 1 o,, m o, = 2 arctanh tanh Zo 2 ) m ) ) tanh,o, 1) 2 where Z o s the LLR of Y o based ony on the channe output. The product sum) s taken over a neghborng nput output) symbos other than the message recpent o) tsef. B. Decodng n BEC In the BEC, the BP decoder can be sgnfcanty smpfed snce a receved symbos n decodng are competey reabe.

4 Ths mpes that the decoder can perform the ogca XOR operatons nversey from the encodng process. Frst, a degree- 1 output symbos are dentfed and moved to a storage referred to as the rppe. Symbos n the rppe are processed one by one, n whch they are XOR ed wth a output symbos who have them as neghbors. Once a symbo has been processed, t s removed from the rppe and consdered decoded. The processng of symbos n the rppe w potentay reduce some of the buffered symbos to degree one, n whch case they are moved to the rppe. Ths s caed a symbo reease. The decoder can then process symbos n a successve fashon. The rppe s an mportant parameter n the BEC. In [1], a trade-off was descrbed. If a new symbo s reeased every tme, there s a rsk that t s aready n the rppe, that causes redundancy. It suggests that the rppe sze shoud be kept ow. However, n order to decrease the rsk of decodng faure, whch occurs when the rppe sze s zero, the rppe sze shoud be kept hgh enough. A good souton to ths tradeoff s the man desgn probem n the BEC. In the foowng anayss, we use nformaton-theoretc toos to present an anaytca framework whch generazes the rppe-based approach towards the AWGN channe. III. RIPPLE-ORIENTED ANALYSIS We consder the BP decoder descrbed n secton II-A, wth a sght modfcaton, n order to factate the anayss. Instead of ettng a nput symbos pass a new message n a round, we aow ony one randomy chosen nput symbo to do so. A other nput symbos pass the message n the prevous round. Ths modfed BP decoder for nput-to-output message updatng s expressed as m 1 m o,, f s aowed to pass,,o = o o 2) m 1,o, f s not aowed to pass. Ths modfcaton s known as a random schedung for BP. It s known that such a sequenta schedung offers better convergence performance than a standard parae schedung. Note that a code desgned by ths anayss can be decoded by any BP decoder schedung. A. Anaytca Framework We frst express the entropy, HZ ), of the -th nput symbo after decodng rounds as foows: HZ ) = PrX = x Y ) og 2 PrX = x Y ) ), x {,1} PrX = 1 Y ) = expz ) 1 + expz 3) ), PrX = Y ) = 1 PrX = 1 Y ), Z = o m o,. When an nput symbo passes a new message to ts neghbors, we refer to the nformaton t hods as processed nformaton. After the -th decodng round, the processed nformaton, IP, s defned as the tota amount of nformaton passed from nput symbos to output symbos. It s gven as k IP = 1 HZ p )), Z p = o m,o d 1, 4) =1 where HZ p ) s nterpreted as the entropy of the -th nput symbo at the pont of ts ast message passng. Drecty foowng from 4), we have the defnton of unprocessed nformaton, IL = k =1 HZp ). When decdng whch nput symbo shoud be aowed to pass a new message, a unform random seecton s performed among the nput symbos, whch hod nformaton not yet passed to ts neghbors. We say that these canddates contrbute to the nformaton rppe. The nformaton rppe, IR, after the -th decodng round, s defned as the tota amount of nformaton, hed by the nput symbos whch have not yet been passed to the output symbos. We have I R = k =1 HZ p ) HZ ) ). 5) After the nput symbo has passed ts message, the output symbos obtan a chance to pass messages back to ther neghborng nput symbos. Ony output symbos, whch are neghbors to the ast message passng nput symbo, have new nformaton to pass. Ths new nformaton s referred to as reeased nformaton, denoted IQ. It s expressed as I Q = Z p+ k =1 = Z p + o HZ p ) HZp+ ) ), m o, m 1 ) o,, 6) where HZ p+ ) s nterpreted as the entropy of the -th nput symbo when processed and newy reeased nformaton s taken nto account. Here, IQ s defned by HZp ) as reference, whch s the nformaton known by the output symbos. Hence, IQ can be regarded as the amount of new nformaton passed to the nput symbos, as seen from an output symbo perspectve. In fact, ths s not the true amount of new nformaton snce t mght be combned wth nformaton n the rppe. For ths case, the actua reference s HZ 1 amount of nformaton added to the rppe, IA I A = = k =1 HZ 1 ) H k H Z 1 =1 ) and we can defne the actua, as foows: m o, m 1 ) )) o, Z 1 + o ) H Z ) ). 7) The quanttes defned n 3) through 7) are ustrated n Fg. 1, where the entropy of a snge nput symbo has been potted as a functon of ts LLR. Due to the convexty of the entropy functon except at very ow LLR), we have I A < I Q, whch means oss of nformaton. Ths s anaogous to the rsk of redundancy for nonzero rppe n the BEC.

5 1 H ) Z P H Z P+ ) H Z 1 ) H ) Z I 1 P = IP I Q I 1 R I A I R I 1 L = IL LLR Fg. 1. Entropy as a functon of LLR for an nput symbo. The contrbuton to the defned quanttes s annotated, assumng that the symbo has not passed a new message, but receved new nformaton from ts neghbors. B. Frst Moment of Informaton Rppe In genera, there s a strong reaton between the quanttes n 3) through 7) and the terms defned n secton II-B for the BEC decoder. They are essentay contnuous entropy counterparts of the dscrete symbo based versons from the BEC. One nterestng quantty n a rppe-based desgn s the expected amount of reeased nformaton from an output symbo of degree d as a functon of the amount of unprocessed nformaton. It expresses the unversa connecton between the degree dstrbuton, whch s the desgn parameter, and the rate of recovery of new nformaton durng the decodng process. It was derved n [1] for the BEC, more specfcay q1, k) = 1, qd, L) = dd 1)L d 3 j= k L + 1) j) d 1 j= k j), for d = 2,..., k, and L = k d + 1,..., 1, qd, L) =, for a other d and L, 8) where L s the amount of unprocessed nput symbos. Dervng the AWGN channe equvaent to 8) s outsde the scope of ths paper. However, we can easy obtan an understandng of ts behavor by smuatng an LT code n an AWGN channe and oggng IQ versus I L for dfferent degrees. In order to compare wth 8), we quantze ths data to nteger vaues of IL and normaze such that t sums up to one. We thus have the fracton of the reeased nformaton, whch s reeased when I L bts reman unprocessed. Ths s denoted as I q d, I L ) for output symbos of degree d and s defned as I q d, I L ) = : I L I L <.5 I Q I Q, 9) where ony nformaton from output symbos of degree d s ncuded. Fg. 2 shows a pot of the resuts at a sgna-to-nose power rato SNR) of 5 db. It reveas a cear correspondence between the theory derved for the BEC and what s observed n the AWGN channe. However, as descrbed n connecton wth 7), not a reeased nformaton s added to the rppe. In order to de- Iq d, IL) IL 3 2 d = 2 d = 3 d = 4 BEC Theory Fg. 2. Comparson of smuated I Q as a functon of d and quantzed I L and correspondng curves from 8). Normazed Frequency I A I Q A Q Fg. 3. Hstogram of the dfference between smuated IA /I Q from AWGN channe and theoretc AL)/QL) from the BEC. termne how much, we must know I A, the rato of reeased IQ nformaton whch s added to the rppe. The BEC counterpart s AL) QL) = L RL) L, where AL), QL) and RL) are symbos added to the rppe, reeased symbos and symbos n the rppe after the k L)-th decodng round, respectvey. For the AWGN channe, I A I Q 1 has been determned for < R 2 and < L 64 through a smuaton smar to the one used for determnng I q d, I L ). Fg. 3 shows a hstogram of I A AL) IQ QL), whch ustrates that the AWGN channe behaves as the BEC wth a sma perturbaton. Based on qd, L) and the fact that AL) QL) = L RL) L hods n the BEC, t s possbe to contro the expected amount of symbos added to the rppe n each decodng round, through the choce of degree dstrbuton. We can now concude that ths theory aso hods n the AWGN channe. Hence, the same rppe-based desgn approach can be used for the AWGN channe, n order to contro the frst moment of the rppe. Ths suggests that degree dstrbutons desgned for the BEC shoud aso perform we n the AWGN channe, whch was aso concuded n [8]. However, t was aso mentoned that there s st a room for mprovement, and thus behavora dfferences between the BEC and the AWGN channe. These dfferences shoud be evdent n the hgher moments of the rppe. We take ths nto account n our desgn, whch s descrbed next. IV. RIPPLE-BASED DESIGN IN AWGN A typca approach to desgn LT codes s to choose desgn crteron for the frst moment of the rppe and meet t through proper choce of the degree dstrbuton. The frst moment crteron s chosen based on heurstc assumptons about the second moment behavor. Ths approach was taken n both

6 [1] and [2] for the BEC. Usng the anaytca framework presented n the prevous secton, we w appy the rppebased approach to desgnng LT codes n the AWGN channe. In [2], the assumptons about the second moment behavor of the rppe n the BEC was based on a random wak mode. It was assumed that the rppe sze ether ncreases or decreases by one, wth equa probabtes, n each decodng round,.e., R +1 = R ± 1. Snce one symbo s processed n each round n the BEC, L rounds w reman, and we can cacuate the expected dstance from the orgn, E [ R +L R ] E[ ] denotes an expectaton), after the random wak as foows: E [ R +L R ) +L 1 2] = j= E [ R j+1 R j) 2] = L, E [ R +L R ] = L. 1) Based on ths t was argued that the expected rppe shoud be kept at c L, for a propery chosen constant c, when L symbos reman unprocessed,.e. n the k L)-th decodng round. We w adopt ths approach, for our choce of expected nformaton rppe for the AWGN channe wth a sght modfcaton whch deas wth the characterstcs of ths channe. Frst, we note that a decodng round does not necessary resut n the processng of 1 bt,.e. HZ p ) HZ ) 1. Hence, we assume that the rppe w ncrease or decrease by α wth equa probabtes. Moreover, t s possbe that HZ ) > HZp ),.e. the messages passed from output symbos to nput symbo, snce ast tme t was aowed to pass, has ncreased the entropy of that nput symbo. In ths case, we actuay have negatve decodng progress. Numerca experments show that ths happens more frequenty ater n the decodng progress, whch resuts n an ncreasng number of decodng rounds per one bt of processed nformaton. We assume a near ncrease, such that the number of steps per processed bt s θ = β1 I L k ) + 1. Hence, we obtan E [ R +L R ) +θi 2] = E [ R +L R ] = α L 1 j= E [ R j+1 R j) 2] = α 2 θi L, β1 I L k ) + 1 I L. 11) It determnes a desred nformaton rppe evouton wth propery chosen constants, α and β. Note that ths mode s based on heurstcs, as n [1] and [2]. It has been confrmed that a hgher-order poynoma mode of θ had amost no gans. The AWGN channe dffers from the BEC n another sgnfcant way n BP decodng. Messages passed from output symbos may be mseadng, n the sense that they contrbute wth an LLR of an opposte sgn of the true vaue of the bt. If one or more bts are n error, more output symbos w need to be coected. In ths case, the errors may propagate and make future output symbos mseadng as we. Consder the exampe where 2 out of k nput symbos are n error and a new output symbo of degree 3 s receved. The three neghbors of the new symbo are the two erroneous Probabty p 1 d) p 2 d) γd) Degree d) Fg. 4. Probabtes of havng 1 and 2 erroneous neghbors, and the dfference between them, as a functon of degree. nput symbos and one correct nput symbo. When ths new output symbo passes a message to an nput symbo, t s based on a product of the messages from the two other neghbors. Hence, for the erroneous nput symbos, yet another mseadng message w be passed, whe for the correct nput symbo, the errors cance out. If ony one erroneous nput symbo had been a neghbor, the output symbo woud have made a hepfu contrbuton. Smar exampes can be made wth hgher numbers of errors. In genera, we can say that f t s more key that an even number of erroneous nput symbos are neghbors to a new output symbo, compared to an odd number, then the errors w be sef-perpetuatng. Ths observaton can be used to create a desgn crteron, where we focus on the cases of 1 and 2 erroneous neghbors, snce these are most key. We defne γd) = p 1 d) p 2 d), where p e d) s the probabty that a new output symbo of degree d has e and ony e erroneous neghbors) for e = 1, 2. They are expressed as p 1 d) = 1) d k d 1 ) and p 2 d) = 2) d k d ) k 2) k 2) and potted as a functon of d n Fg. 4. The pots ustrate that hgh degree symbos shoud be avoded, whereas degrees n the ower haf of the spectrum are more usefu. In our desgn, we seek to maxmze E[γ] = d γd)ωd), under the constrants gven by our choce of rppe. We now summarze the anayss and the rppe-based desgn method for nosy channes. In secton III-B, we showed that 8), whch was orgnay derved for the BEC, aso hods n the AWGN channe. Moreover, we showed that I A I Q = I L I R I L s aso anaogous to the BEC. If we quantze the decodng process to nteger vaues of I L, we can express the expected evouton of the nformaton rppe for each processed bt: I R k) = α I L, I R I L 1) = I R I L ) 1 + I L I R I L ) I L I Q d, I L ) = d d for k > I L, I Q d, I L ), nix; Y )Ωd)I q d, I L ), 12) where IX; Y ) s the mutua nformaton of the AWGN channe and n s the number of symbos coected for decodng. Combnng our rppe constrant from 11) wth 12), we d

7 obtan the system of equatons as foows: q1, k) I R k).... n Ω1) I R k 1) q1, 1) qk, 1). =., 13) n ωγ1) ωγk) Ωk) I R 1) n ωe[γ] where n = nix; Y ) and I R I L 1) = I R I L 1) I R I L )+1)I L I L I R I L ). Here we have aso added E[γ] = d γd)ωd), whch we wsh to maxmze. Ths equaton has a mutper ω 1 snce ths s a frm constrant. Note that n acts as a normazaton factor, whch ensures a vad degree dstrbuton. We can then formaze the desgn probem as foows: max E[γ] s.t. An Ωd) b 2 < t, 14) where A s the matrx n 13), b s the vector on the RHS of 13) and t s an appropratey chosen toerance eve of the devaton from the target nformaton rppe. V. NUMERICAL RESULTS Smuatons have been performed n order to compare the proposed desgn, Ωd), wth exstng degree dstrbutons. A toerance eve of t =.1 has been chosen for the optmzaton probem n 14). A numerca optmzaton of the parameters α and β has been performed, whch reveaed that parameter pars where 1. α 1.4 and β 3 perform we, dependng on the SNR. In genera α shoud ncrease for ncreasng SNR, whe β shoud decrease. References n the frst smuaton are the Robust Soton Dstrbuton RSD), wth parameters c =.1 and δ = 1, and a degree dstrbuton desgned by the proposed approach presented n ths work, but wth the rppe target n 1), whch was suggested n [2]. We refer to ths dstrbuton as Γd). A numerca optmzaton resuted n c = 1.3, whch has been used. A dstrbutons have been evauated at k = 64 for an SNR of, 2,..., 1 db usng BPSK moduatons. They are compared wth respect to average overhead necessary n order to successfuy decode a nput symbos. An nput symbo s consdered successfuy decoded f ts error probabty s beow Fg. 5 shows the resuts, where t s evdent that the approach presented n ths work sgnfcanty outperforms the dstrbutons desgned for the BEC. Especay at ow SNR, where the dfference between the AWGN channe and the BEC s more sgnfcant, the proposed desgn exces. In ths fgure, we aso present the ower bound of the overhead cacuated by the Shannon mt n AWGN for BPSK-constraned codebooks wth an nfnte ength). As we can see, the proposed degree dstrbuton approaches the Shannon mt n the ow SNR regmes wthn 1 db even for such a very short message ength of k = 64. Another smuaton has been performed, where the purpose s to evauate the degree dstrbutons wth respect to bock error rate at specfc overheads. A bock error occurs when at east one symbo has an error probabty above 1 12 after 2 decodng rounds on 1 + ϵ)k output symbos. The Avg. Overhead Fg. 5. Bock Error Rate SNR RSD Γd) Ωd) Shannon Lmt BPSK) Average overhead versus SNR for the smuated degree dstrbutons Overhead 1 + ǫ) RSD Γd) Ωd) Fg. 6. Bock error probabty versus overhead for the smuated degree dstrbutons. smuaton s performed at an SNR of db usng the same parameters as n the frst smuaton. The resuts are shown n Fg. 6 and t s seen that the proposed desgn outperforms the other degree dstrbutons at any overhead. VI. CONCLUSIONS We have presented an anaytca framework for LT codes n AWGN channes. Surprsngy, key anaytca resuts known n the BEC can be apped to AWGN channes wthn ths framework. Ths enabes us to ntroduce a rppe-based desgn scheme n AWGN channes wth a few modfcatons. LT codes based on ths desgn show promsng resuts compared to standard desgns. In partcuar, the desgned codes can approach the capacty n the ow SNR regmes even for a very short message ength. Our anaytca framework s a strong too for the rppe-based desgn n any nosy channes. REFERENCES [1] M. Luby, LT codes, n The 43rd Annua IEEE Symposum on Foundatons of Computer Scence, pp , 22. [2] A. Shokroah, Raptor codes, IEEE Transactons on Informaton Theory, vo. 52, pp , June 26. [3] O. Etesam and A. Shokroah, Raptor codes on bnary memoryess symmetrc channes, IEEE Transactons on Informaton Theory, vo. 52, pp , May 26. [4] S.-Y. Chung, T. Rchardson, and R. Urbanke, Anayss of sum-product decodng of ow-densty party-check codes usng a Gaussan approxmaton, IEEE Transactons on Informaton Theory, vo. 47, pp , Feb. 21. [5] Z. Cheng, J. Castura, and Y. Mao, On the desgn of Raptor codes for bnary-nput Gaussan channes, IEEE Transactons on Communcatons, vo. 57, pp , Nov. 29. [6] I. Hussan, M. Xao, and L. Rasmussen, LT coded MSK over AWGN channes, n The 6th Internatona Symposum on Turbo Codes and Iteratve Informaton Processng ISTC), pp , Sept. 21. [7] P. Pakzad and A. Shokroah, Desgn prncpes for Raptor codes, n IEEE ITW 6 Punta de Este., pp , March 26. [8] R. Paank and J. Yedda, Rateess codes on nosy channes, n Avaabe at 24.