ARTICLE IN PRESS. Murat Hüsnü Sazlı a,,canişık b. Syracuse, NY 13244, USA

Transcription

1 S ) /FLA AID:621 Vol ) [+odel] P1 1-7) YDSPR:3SC+ v 153 Prn:13/02/2006; 15:33 ydspr621 by:laurynas p 1 Digital Signal Processing ) wwwelsevierco/locate/dsp Neural network ipleentation of the BCJR algorith Murat Hüsnü Sazlı a,,canişık b a Faculty of Engineering, Electronics Engineering Departent, Ankara University, Tandogan 06100, Ankara, Turkey b LC Sith College of Engineering and Coputer Science, Departent of Electrical Engineering and Coputer Science, Syracuse University, Syracuse, NY 13244, USA Abstract In this paper, we first show that the BCJR algorith or Bahl algorith) can be ipleented via soe atrix anipulations As a direct result of this, we also show that this algorith is equivalent to a feedforward neural network structure We verified through coputer siulations that this novel neural network ipleentation yields identical results with the BCJR algorith 2006 Elsevier Inc All rights reserved Keywords: Turbo codes; Turbo coding/decoding; Channel coding; Bahl algorith; BCJR algorith; MAP algorith; Neural networks 1 Introduction Artificial neural networks or neural networks in short, have becoe a great focus of interest in the field of counications and especially digital counications aongst the researchers in the past decade There are nuerous applications found in the literature, which applied the neural networks to any different probles of counications Many of those can be found in the survey of Ibnkahla that cited over 200 references [1] Nonlinear channel odeling and identification; channel equalization; coding, decoding, and error correcting codes; spread spectru applications; vector quantization and iage processing; nonlinear signal processing are the categories that are enuerated in [1] Aongst these, channel coding and decoding are entioned as very proising fields of neural network applications Bahl, Cocke, Jelinek, and Raviv BCJR) algorith, also known as MAP algorith axiu a posteriori algorith), is the optiu algorith used in soft input soft output SISO) decoders [2] Therefore, it has becoe of great iportance in iterative decoding schees, and especially turbo decoders Nickl et al used a neurocoputer, which was originally designed to copute large neural networks, to ipleent the MAP algorith using atrix anipulations [3] However, as they pointed out in their paper, their approach required considerable odifications to the original algorith and application of soe nonlinear transforations In this paper, we show that the BCJR algorith can be ipleented using soe atrix anipulations without doing any odifications to the original algorith As a direct result of this, we prove that the BCJR algorith is equivalent to a feedforward neural network structure Berrou et al suggested recursive systeatic convolutional RSC) codes to be used in conjunction with turbo codes and gave the odified version of the BCJR algorith for decoding of RSC codes [4] It is iportant for us to note that * Corresponding author E-ail address: sazli@engankaraedutr MH Sazlı) /$ see front atter 2006 Elsevier Inc All rights reserved doi:101016/jdsp

2 S ) /FLA AID:621 Vol ) [+odel] P2 1-7) YDSPR:3SC+ v 153 Prn:13/02/2006; 15:33 ydspr621 by:laurynas p 2 2 MH Sazlı, C Işık / Digital Signal Processing ) these odifications are only needed in order to take into account the recursive nature of the encoder Therefore we ust ephasize that, regardless of these odifications it is in essence the sae algorith and also optial Since it is this odified version of the BCJR algorith used in turbo decoders, we do our forulation on this odified version We directly give our work in the next section Interested reader is referred to [2,4,5] for a detailed description of the BCJR algorith and the odified BCJR algorith We stick with the notation used in [4,5] Note that this work is a part of a doctoral dissertation and details can be found in [6] 2 Neural network ipleentation of the BCJR algorith In this section, we reforulate the BCJR algorith via soe atrix anipulations Then, we show that those anipulations are equivalent to soe feedforward neural network structures In the following, we consider a recursive convolutional encoder with a constraint length K, and code eory ν = K 1 There are 2 ν states of this encoder We also suppose that BPSK odulation is used, ie, bit one is apped to +1, and bit zero is apped to 1 21 Calculation of α alpha) coefficients forward recursion of the algorith) Let us begin with the recursive equation to obtain the α coefficients of the BCJR algorith: i=+1 i= 1 α k 1 )γ i R k,,) i=+1 i= 1 α k 1 )γ i R k,,), 1) where = 0, 1, 2,,M, is the index of the states with M = 2 ν 1 By expanding the inner suations we get: α k)[γ 1 R k,,)+ γ +1 R k,,)] α k)[γ 1 R k,,)+ γ +1 R k,,)], 2) α k)γ 1 R k,,)+ α k)γ +1 R k,,) α k)γ 1 R k,,)+ α k)γ +1 R k,,)) 3) Let us define: α k ) α k 1 )γ 1 Rk,, ), α + k ) α k 1 )γ +1 Rk,, ), α p k ) α k ) + α+ k ) Using 3) 6) we can rewrite 1) as αp k ) αp k ) 4) 5) 6) 7) Now let us define soe vectors and atrices Fro now on capital letters are used to denote vectors and atrices A k [ αk 0) α k 1) α k M) ], 8) A + k [ α k + 0) α+ k 1) α+ k M) ], 9) A p k A k + A+ k = [ α p k 0) αp k 1) αp k M) ], 10) A k [ α k 0) α k 1) α k M) ], 11) γ 1 R k, 0, 0) γ 1 R k, 0, 1) γ 1 R k, 0,M) Ɣ k γ 1 R k, 1, 0) γ 1 R k, 1, 1) γ 1 R k, 1,M), 12) γ 1 R k,m,0) γ 1 R k,m,1) γ 1 R k,m,m)

3 S ) /FLA AID:621 Vol ) [+odel] P3 1-7) YDSPR:3SC+ v 153 Prn:13/02/2006; 15:33 ydspr621 by:laurynas p 3 MH Sazlı, C Işık / Digital Signal Processing ) 3 γ +1 R k, 0, 0) γ +1 R k, 0, 1) γ +1 R k, 0,M) Ɣ + k γ +1 R k, 1, 0) γ +1 R k, 1, 1) γ +1 R k, 1,M), 13) γ +1 R k,m,0) γ +1 R k,m,1) γ +1 R k,m,m) γr k, 0, 0) γr k, 0, 1) γr k, 0,M) Ɣ k Ɣ k + Ɣ+ k = γr k, 1, 0) γr k, 1, 1) γr k, 1,M), 14) γr k,m,0) γr k,m,1) γr k,m,m) where γ R k,, ) γ 1 Rk,, ) + γ +1 Rk,, ) 14a) Now let us take 4) and expand it: αk ) = α k 10)γ 1 R k, 0,)+ α k 1 1)γ 1 R k, 1,)+ +α k 1 M)γ 1 R k,m,) 15) We can easily show that 15) can be written as a atrix product as follows: γ 1 R k, 0,) αk ) = [ α k 1 0) α k 1 1) α k 1 M) ] γ 1 R k, 1,) 16) γ 1 R k,m,) One can notice that the first ter in 16) is A k, and the second ter corresponds to the th colun of Ɣ k Reebering the definition of A k fro 8), we can prove that A k can be written as A k = A k 1Ɣ k 17) Siilarly, we can also show that A + k can be written as A + k = A k 1Ɣ + k Once we obtained A k and A+ k, fro 10) we can also obtain Ap k Now let us take the denoinator of 1) Fro 7): α p k ) = αp k 0) + αp k 1) + +αp k M) SUMk) 19) This suation yields a constant scalar Let us define this as SUMk) Then finally A k can be written as A k = Ap k SUMk) 20) Assuing that the encoder ust be at zero state at the beginning, initial condition for A k can be written as A 0 = [ ] 21) since α 0 0) = 1, 21a) α 0 ) = 0, 0 We can easily show that 17) 20) can be ipleented as feedforward neural networks by assigning the γ coefficients to the weights of the neural networks In Figs 1 and 2, A k 1 ) T is applied to the input layer, and respectively A k )T, A + k )T are obtained fro the output layers of the neural networks Notice that there is no hidden layer in these networks Also notice that linear neurons are used in the output layers Fro 10), direct suation of the outputs of these networks yields A p k )T Fro 19), we see that SUMk) is defined as the suation of all the ters in A p k A k) T can be obtained using a feedforward neural network structure as shown in Fig 3 At each tie step, this procedure has to be repeated The block diagra of the neural network structure to obtain A k at tie step k is shown in Fig 4 18)

4 S ) /FLA AID:621 Vol ) [+odel] P4 1-7) YDSPR:3SC+ v 153 Prn:13/02/2006; 15:33 ydspr621 by:laurynas p 4 4 MH Sazlı, C Işık / Digital Signal Processing ) Fig 1 Single layer feedforward neural network to obtain A k fro A k 1 Fig 2 Single layer feedforward neural network to obtain A + k fro A k 1 Fig 3 Neural network to obtain A k fro A p k Fig 4 Block diagra of the structure to obtain α coefficients at tie step k 22 Calculation of β beta) coefficients backward recursion of the algorith) Now, we can follow a siilar procedure to forulate the β coefficients of the BCJR algorith in atrix notation, which can also be ipleented as feedforward neural networks i=+1 i= 1 β k ) = β k+1 )γ i R k+1,, ) i=+1 i= 1 α k)γ i R k+1,, ) 22) Reebering 1), 4), 5), 6), and 19), we can show that the denoinator can be written as i=+1 i= 1 α k )γ i Rk+1,, ) = SUMk + 1) 23) Notice that SUMk + 1) is readily available fro the coputation of α s Now let us take the nuerator of 22) and in a siilar anner to 6), let us define it as β p k ):

5 S ) /FLA AID:621 Vol ) [+odel] P5 1-7) YDSPR:3SC+ v 153 Prn:13/02/2006; 15:33 ydspr621 by:laurynas p 5 i=+1 MH Sazlı, C Işık / Digital Signal Processing ) 5 β p k ) β k+1 )γ i Rk+1,, ), 24) i= 1 β p k ) = β k+1 ) [ γ 1 Rk+1,, ) + γ +1 Rk+1,, )], 25) β p k ) = β k+1 )γ R k+1,, ), 26) β p k ) = β k+10)γ R k+1,,0) + β k+1 1)γ R k+1,,1) + +β k+1 M)γ R k+1,,m) 27) Siilar to 15), we can show that 27) can be written as a atrix product as follows: γr k+1,,0) β p k ) = [ β k+1 0) β k+1 1) β k+1 M) ] γr k+1,,1) 28) γr k+1,,m) Let us define soe atrices for β coefficients, siilar to the ones for α coefficients: B p k [ β p k 0) βp k 1) βp k M) ], 29) B k [ β k 0) β k 1) β k M) ] 30) Obviously, the first ter in 28) is B k+1, and the second ter corresponds to the th row of the Ɣ k+1 Instead of repeating 28) for all state values s), we can write it as a atrix product as follows: γr k+1, 0, 0) γr k+1, 1, 0) γr k+1,m,0) B p k = [ β k+1 0) β k+1 1) β k+1 M) ] γr k+1, 0, 1) γr k+1, 1, 1) γr k+1,m,1) 31) γr k+1, 0,M) γr k+1, 1,M) γr k+1,m,m) One can notice that the second atrix is the transpose of Ɣ k+1 Using that we can write: B p k = B k+1ɣ k+1 ) T and, finally, B p k B k = SUMk + 1) 33) Assuing that the encoder ust be at zero state at the end, initial condition for B k can be written as B N =[1 0 0 ] 34) since β N 0) = 1, 34a) β N ) = 0, 0 Siilar to the one for α coefficients, this can also be ipleented as a single layer feedforward neural network Since SUMk + 1) is already obtained fro the neural network for α, we do not need to copute that again to obtain B k fro B p k Single layer feedforward neural network to obtain Bp k fro B k+1 is shown in Fig 5 The block diagra of the neural network structure to obtain B k at tie step k isshowninfig6 23 Calculation of LLRs the logarith of the likelihood ratios) of the inforation bits Now, we show how we can copute the logarith of the likelihood ratios LLR) associated with each bit d k using the previously coputed A, B, and Γ atrices LLR associated with each bit d k is given by Λd k ) = ln γ +1R k,,)α k 1 )β k ) γ 1R k,,)α k 1 )β k ) 35) 32)

6 S ) /FLA AID:621 Vol ) [+odel] P6 1-7) YDSPR:3SC+ v 153 Prn:13/02/2006; 15:33 ydspr621 by:laurynas p 6 6 MH Sazlı, C Işık / Digital Signal Processing ) Fig 5 Single layer feedforward neural network to obtain B p k fro B k+1 Fig 6 Block diagra of the structure to obtain β coefficients at tie step k Fig 7 Block diagra of the structure to obtain LLR at tie k Let us define the nuerator as p k, and the denoinator as n k, where p k and n k are scalars Then, l k Λd k ) = ln p k = lnp k ) lnn k ) 36) n k Since both p k and n k are the sae except where γ, we can work on either one of the to forulate the as atrix anipulations p k = β k )α k 1 )γ +1 Rk,, ), 37) p k = β k ) α k 1 )γ +1 Rk,, ) 38) Using 5), p k = β k )α + k ), p k = [ β k 0) β k 1) β k M) ] α + k 0) α + k 1) α + k M) Notice that the second atrix corresponds to the transpose of A + k Then, 39) 40) p k = B k A + ) T k Siilarly we can show that n k = B k A ) T k Finally, we show that this also can be ipleented as a feedforward neural network in Fig 7 41) 42)

7 S ) /FLA AID:621 Vol ) [+odel] P7 1-7) YDSPR:3SC+ v 153 Prn:13/02/2006; 15:33 ydspr621 by:laurynas p 7 MH Sazlı, C Işık / Digital Signal Processing ) 7 3 Conclusions and future work In this paper, we showed the BCJR algorith to be equivalent to a feedforward neural network by reforulating its recursive equations using atrix anipulations We tested this neural network ipleentation in a turbo decoder with different constraint lengths of code eory and interleaver sizes for various signal-to-noise ratios SNR) in an AWGN channel Our extensive coputer siulations verified that this novel ipleentation, indeed, yields identical results with the original algorith as one can expect BER perforances of both ipleentations are also identical, which has been tested in various channel conditions In short, our work fors a theoretical basis to ipleent the BCJR algorith using neural networks Being shown equivalent to a neural network, the algorith inherits all the advantageous aspects of a neural network Especially, this reforulation of the BCJR algorith using atrix anipulations is a vital step in ipleenting this algorith using parallel structures such as vector processors For instance, a novel parallel ipleentation on a odern FPGA or a DSP chip ay yield soe iproveents for coputational coplexity, and consequently for decoding speed In turn, such ipleentations ay have soe practical use in iterative decoding schees However, investigations on these fields are beyond the topic of this paper and ay be subjects for other researches This ipleentation ay also lead to adaptive features, which ay be utilized in fading channels Besides, the equivalence of the BCJR algorith to a feedforward neural network structure ay shed soe light to a deeper understanding and interpretation of it Acknowledgents The authors wish to express their sincere gratitude to Dr Biao Chen for valuable discussions throughout this work References [1] M Ibnkahla, Applications of neural networks to digital counications: A survey, Signal Process ) [2] LR Bahl, J Cocke, F Jelinek, J Raviv, Optial decoding of linear codes for iniizing sybol error rate, IEEE Trans Infor Theory 20 2) 1974) [3] H Nickl, J Hagenauer, F Burkert, Approaching Shannon s capacity liit by 027 db using Haing codes in a turbo -decoding schee, in: Proc IEEE Syp Info Theory, June 1997, p 12 [4] C Berrou, A Glavieux, P Thitiajshia, Near Shannon liit error-correcting coding and decoding: Turbo-codes, in: Proc IEEE Int Conf Coun, Geneva, Switzerland, May 1993, pp [5] C Berrou, A Glavieux, Near optiu error-correcting coding and decoding: Turbo-codes, IEEE Trans Coun 44 10) 1996) [6] MH Sazli, Neural network applications to turbo decoding, PhD dissertation, Syracuse University, 2003 Dr Murat H Sazlı was born in 1973 in Elazığ, Turkey He received the BSc and MSc degrees in electronics engineering fro Electronics Engineering Departent, Ankara University, with high honors in 1994 and 1997, respectively He received the PhD degree in electrical engineering fro Syracuse University in 2003 He was the recipient of Outstanding Teaching Assistant Award fro Syracuse University in 2002 He is currently a faculty eber in Electronics Engineering Departent of Ankara University His areas of interest include turbo coding and decoding, neural networks and their applications, wireless counications Dr Sazlı is a eber of the Institute of Electrical and Electronics Engineers Dr Can Işık was born in 1955 in Adana, Turkey He received the BSc and MSc degrees fro Departent of Electrical and Electronics Engineering, Middle East Technical University, in 1978 and 1980, respectively He received the PhD degree in electrical engineering fro University of Florida in 1985 He has been a faculty eber in Electrical Engineering and Coputer Science Departent of Syracuse University since 1985 He is currently a Professor of Electrical Engineering and Senior Associate Dean for Acadeic & Student Affairs of the LC Sith College of Engineering and Coputer Science at Syracuse University He is interested and actively involved in research in the areas of neural networks and their applications, fuzzy systes, hybrid systes, and intelligent systes Dr Işık is a senior eber of the Institute of Electrical and Electronics Engineers, a eber of North Aerican Fuzzy Inforation Processing Society, and Eta Kappa Nu, and is listed in Who is Who in Aerican Education He was the general chair of NAFIPS 97, the annual eeting of North Aerican Fuzzy Inforation Processing Society