Jul 4, 2005 turbo_code_primer Revision 0.0. Turbo Code Primer

Transcription

1 Jul 4, 5 turbo_code_primer Reviion. Turbo Code Primer. Introduction Thi document give a quick tutorial on MAP baed turbo coder. Section develop the background theory. Section work through a imple numerical example.. Theory A highlevel view of a rate / turbo code ytem i hown in figure.. We ue upper cae to denote binary number and lower cae to denote ignal/ymbol value. For each ource information bit, a pair of convolutional encoder generate a correponding pair of parity bit. The two parity bit and the original information bit are then mapped to ymbol (ignal value for tranmion through an AWGN channel. ( X k, P k, P k, p k, p k (, p' k, p' k Lx ( k map to X k encoder ymbol decoder lice Xˆ k X k P k P k x k p k p k information data bit encoder # parity bit encoder # parity bit information data ymbol encoder # parity ymbol encoder # parity ymbol N(, σ σ p' k p' k Lx ( k decoder oft deciion ˆ decoder hard deciion X k noie variance noiy information data ymbol noiy encoder # parity ymbol noiy encoder # parity ymbol Figure. Turbo Code Sytem. Turbo Encoder The tructure of a rate / encoder i hown in fig.. Two identical convolutional encoder are ued.the order of the information bit undergo a peudo random permutation P prior to been fed into the econd convolutional encoder. Accordingly a turbo code i a block coding cheme where a block of K information bit are buffered prior to applying the permutation. x k X k P convolutional encoder # convolutional encoder # P k P k Figure. Turbo Encoder Many type of permutation can be ued, provided they are ufficiently random. The permutation make the two contituent encoder appear uncorrelated at the receiver. In the example in ection we will ue a permutation baed on the tate equence of a maximal length hift regiter (PN equence. Author: VA of 4

2 Jul 4, 5 turbo_code_primer Reviion. Each convolutional encoder i baed on a recurive ytematic convolutional code. Fig. how a imple 4 tate example. The operation of the encoder i ummaried by the trelli diagram which how the bit pair output for each poible tranition between uceive tate. x n u u tate u u x n n n+ p n+ x n n,,,,, p n Figure. Recurive Sytematic Code n X n,p n,,, The tate tranition diagram in fig. can be repeatedly drawn to generate a trelli diagram for the encoder. The trelli diagram (labelled with the tranmit ymbol value for a equence of 8 bit i hown in fig.4. x, p x, p x, p x, p x 4, p 4 x 5, p 5 x 6, p 6 -,- -,- -,- -,- -,- -,- -,- +,+ +,+ +,+ +,+ +,+ +,+ +,+ x 7, p 7 -,- +,+ +,+ -,- +,- -,+ +,+ -,- +,- -,+ +,+ -,- +,- -,+ +,+ -,- +,- -,+ +,+ -,- +,- -,+ +,+ -,- +,- -,+ +,+ -,- +,- -,+ +,+ -,- +,- -,+ -,+ -,+ -,+ -,+ -,+ -,+ -,+ -,+ +,- +,- +,- +,- +,- +,- +,- +,- Figure.4 Trelli Diagram. Turbo Decoder The tructure of a turbo decoder i hown in fig.5. It conit of a pair of decoder which work cooperatively in order to refine and improve the etimate of the original information bit. The decoder are baed on the MAP (maximum apoteriori probability algorithm and output oft deciion information learned from the noiy parity bit. Initially decoder # tart without initialiation information (apriori etimate are et to zero. In ubequent iteration, the oft deciion information of one decoder i ued to initialie the other decoder. The decoder information i cycled around the loop until the oft deciion converge on a table et of value. The latter oft deciion are then liced to recover the original binary equence. Author: VA of 4

3 Jul 4, 5 turbo_code_primer Reviion. P - ( xk p' k MAP decoder # MAP decoder # P p' k ( xk Figure.5 P Turbo Decoder.. MAP Decoder The MAP algorithm minimie the probability of bit error by uing the entire received equence to identify the mot probable bit at each tage of the trelli. The MAP algorithm doe not contrain the et of bit etimate to necearily correpond to a valid path through the trelli. So the reult can differ from thoe generated by a Viterbi decoder which identifie the mot probable valid path through the trelli. We will ue the following horthand for the tranmitted/received ymbol pair. y k { x k, p k } y y y' k {, p' k }, K { y, y,, y K } y' y', K { y', y',, y' K } The MAP oft deciion are defined a the log likelihood ratio: Pr + y' log Pr y' For a convolutional code we can expre the likelihood ratio in term of the trelli: Pr( k ', k y' Pr( k ', k, y' Pr + y' ( ', S ( ', S Pr y' Pr( k ', k y' Pr( k ', k, y' ( ', S ( ', S Author: VA of 4

4 Jul 4, 5 turbo_code_primer Reviion. The numerator um i over all poible tate tranition aociated with a data bit, and the denominator over all poible tate tranition aociated with a data bit. Fig.6 how the tate tranition for the cae of our example - tate code. k ' y k, p k k k ' y k, p k k, +, + +, +, +,, + S + +, {( ', x k +} S, + {( ', x k } Figure.6 Tranition Set The probability of a particular tate tranition and the noiy obervation aociated with a trelli tranition can be expreed (uing Baye theorem a: Pr( k ', k, y', K Pr( k ', y', k Pr( k, y' k k ' Pr( y' k +, K k αk ( ' γ k ( ', β k ( Note the probabilitie aociated with the continuou valued received obervation taking a particular value are infiniteimally mall. A the final reult will be a probability ratio (the likelihood ratio we can relax the notation and work with probabilitie to help keep the mathematic imple. In term of the above definition of α k ( ', γ k ( ', and β k (, the a poteriori likelihood ratio can be rewrriten a the ratio: Pr + y' Pr y' ( ', S + α k ( ' γ k ( ', β k ( α k ( ' γ k ( ', β k ( ( ', S The term α k ( ' i the probability of arriving at a branch in a particular tate and the equence of noiy obervation y', k y', y',, y' k which led up to that tate. By umming over all path leading into that tate we get a forward recurion for calculating α k ( ' in term of the value of γ k ( ',. α k ( Pr( k, y', k Pr( k ', y', k Pr( k, y' k k ' ' ' α k ( ' γ k ( ', To begin the forward recurion we need to initialie the forward tate probabilitie. In turbo coder all convolutional encoder are tarted in tate. Thu we can begin the forward recurion with: α ( Pr( Author: VA 4 of 4

5 Jul 4, 5 turbo_code_primer Reviion. The term β k ( i the probability of exiting a branch via a particular tate and the equence of noiy obervation y' k +, K y' k +, y' k +,, y' K which finih off the trelli. By umming over all path exiting that tate we get a backward recurion for calculating β k ( in term of the value of γ k ( ',. β k ( ' Pr( y' k +, K k Pr( k +, y' k + k ' Pr( y' k +, K k ' γ k + ( ', β k + ( To begin the backward recurion we need to initialie the backward tate probabilitie. Convolutional encoder # i uually terminated at tate. However, in general, the final tate for convolutional encoder # i data dependent, and unknown beforehand. We will aume a uniform ditribution for the final tate of encoder #. A uitable initialiation (where the number of tate in the convolutional encoder i ν i a follow: MAP #: β K ( Pr( K MAP #: β K ( Pr( K ---- ν for all We will now derive explicit expreion which can be ued in the receiver to calculate γ k ( ',. For a given tate tranition, the tranmitted ignal i the databit and parity pair y k. Alo, for a given tarting tate, the next tate i completely determined by the value of the databit. Uing the Baye theorem the branch probability can be expreed a: γ k ( ', Pr( k, y' k k ' Pr( y' k k ', k Pr( k k ' Pr( y' k y k Pr The probability of the databit taking a particular value can be expreed in term of the log likeklihood of the apriori probability ratio: Pr exp --L a exp + exp[ ] --x k log Pr Pr B k exp --x k The probability of the oberved noiy databit and parity ymbol taking particular value can be expreed in term of Gauian probability ditribution a: Pr( y k ' y k Pr( x k Pr( p' k p k ( x k ( p' k p k exp πσ σ exp πσ σ x k p' k p k A k exp σ Here the term are infiniteimally mall range about the particular value. Thee term will cancel out in the final expreion for the likelihood ratio. We can now expre the tranition probability γ k ( ', in term of log likelihood ratio and the noiy obervation a: γ k ( ', A k B k exp -- ( x k + x k + p k p' k σ Author: VA 5 of 4

6 Jul 4, 5 turbo_code_primer Reviion. Recalling the definition of the MAP log likelihhood ratio: Pr + y' log Pr y' log Pr( y' k x k Pr( y' k x k + log Pr Pr We now compare thi definition with the verion which we derived baed on the trelli tructure. ( ', S + α k ( ' γ k ( ', β k ( log α k ( ' γ k ( ', β k ( ( ', S exp exp p k p' α ( ' k exp β k k ( ( ', S log exp p exp k p' k α ( ' exp β k ( ( ', S k Noting that the ummation in the numerator i over all tate tranition aociated with a databit ymbol x k equal to +, and that the ummation in the denominator i over all tate tranition aociated with a databit ymbol x k equal to -, we have: + +log p k p' k α k ( ' exp β k ( ( ', S p k p' k α k ( ' exp β k ( ( ', S We can now eperate the MAP log likelihood into three ditinct component. + + The firt term i the apriori information. Thi information i our initial etimate prior to running the MAP algorithm. The econd term i the information provided by that part of the noiy obervation which doe not depend on the convolutional code contraint. The third term i the information which we learn via the parity contraint. Thi information i referred to a extrinic information, and i that information gleaned from the code. In a turbo decoder the extrinic information of one MAP decoder i ued a the apriori input to the other MAP decoder. In the turbo decoder iteration the extrinic information i ping ponged back and forth between MAP decoder.. Worked Example We will now illutrate the implementation of a turbo coder uing a very imple example. We ue the 4 tate encoder hown in fig. and a block length of only 6 data bit with trelli termination bit. Note practical implementation require coniderably much longer block length in order to approach the Shannon limit. Alo in hardware implementation the MAP calculation would be normally performed in the log domain to reduce complexity and improve numerical behavour (when uing finite preciion arithmetic.. Encoder We are given the information databit a: { X, X, X, X 4, X 5, X 6 } {,,,,, } Author: VA 6 of 4

7 Jul 4, 5 turbo_code_primer Reviion. Thee bit are input to a convolutional encoder #. The reultant path through the trelli i hown below in fig..,,,,,,,, Figure. Encoder # Trelli Path (X k,p k The two bit required to park the trelli tate at tate are: { X 7, X 8 } {, } We now feed the block of data through a imple permuter. X X X X X 4 X 5 X 6 X 7 X 5 X 6 X 7 X X X 4 X X Figure. Peudorandom Premutation In our example we ue a 7 element PN equence with a element appended. The block of permuted databit are then fed into convolutional encoder # reulting in the path through the trelli hown below in fig..,,,,,,,, Figure. Encoder # Trelli Path (Perm(X k, P k The databit and the parity bit are mapped to ymbol. X k, P k, P x k k, p k, p k,,,,,,,,,,,,,,,, +, +, - +, -, - -, -, + -, +, + +, -, - -, +, + +, +, + -, -, + Author: VA 7 of 4

8 Jul 4, 5 turbo_code_primer Reviion. In our example we will aume the channel add unity variance Gauian noie with the following value: x k p k p k + AWGN p' k p' k Decoder.. MAP Iteration # In the very firt iteration no extrinic information i available from MAP decoder #, o the apriori information i et to. p' k MAP decoder # ( xk Figure.4 Iteration # The branch probabilitie can be calculated from the noiy obervation according to: γ k ( ', exp -- ( x k + x k + p k p' k Uing the trelli diagram in fig.4 to determine x k, p k for the variou tate tranition we calculate the branch probabilitie for the firt tage: γ (, exp( x' p' exp( γ (, exp( x' + p' exp( γ (, exp( x' p' exp( γ (, exp( x' + p' exp( γ (, exp( x' + p' exp( γ (, exp( x' p' exp( γ (, exp( x' + p' exp( γ (, exp( x' p' exp( σ Author: VA 8 of 4

9 Jul 4, 5 turbo_code_primer Reviion. Repeating for all tage of the trelli we get: ', γ ( ', γ ( ', γ ( ', γ 4 ( ', γ 5 ( ', γ 6 ( ', γ 7 ( ', γ 8 ( ',, , , , , , , , The forward recurion can be calculated according to: In our example the reultant value are: α k ( α k ( ' γ k ( ', ' α ( α ( α ( α ( α 4 ( α 5 ( α 6 ( α 7 ( α 8 ( We then normalie the forward probabilitie by dividing by the um of the forward probabilitie acro all 4 tate at each tage reulting in: α ( α ( α ( α ( α 4 ( α 5 ( α 6 ( α 7 ( α 8 ( The backward recurion can be calculated according to β k ( ' γ ( k ', β k In our example the reultant value are: ' ( β 8 ( β 7 ( β 6 ( β 5 ( β 4 ( β ( β ( β ( β ( Author: VA 9 of 4

10 Jul 4, 5 turbo_code_primer Reviion. We normalie the backward probabilitie by dividing by the um of the backward probabilitie acro all 4 tate at each tage reulting in: β 8 ( β 7 ( β 6 ( β 5 ( β 4 ( β ( β ( β ( β ( Fig ummarie the overall reult for the trelli aociated with encoder #. γ ( ', γ ( ', γ ( ', γ ( ', γ 4 ( ', γ 5 ( ', γ 6 ( ', γ 7 ( ', Figure.5 Iteration # - Annotated trelli for MAP decoder # We now compute the oft deciion of the MAP decoder according to: log ( ', S + α k ( ' γ k ( ', β k ( α k ( ' γ k ( ', β k ( ( ', S α k ( γ k (, β k ( + α k ( γ k (, β k ( + α k ( γ k (, β k ( + α k ( γ k (, β k ( α k ( γ k (, β k ( + α k ( γ k (, β k ( + α k ( γ k (, β k ( + α k ( γ k (, β k ( The oft and hard deciion are: Xˆ k ign[ ] Comparing with the original databit we ee that uing the hard deciion from the firt iteration would reult in bit error. Author: VA of 4

11 Jul 4, 5 turbo_code_primer Reviion. The extrinic information output from the firt iteration can be calculated by ubtracting out the apriori and ignal component. -[ + ] ( x MAP Iteration # In the econd iteration the extrinic information from MAP decoder # i ued a the apriori information for the MAP decoder #. The latter log likelihood ratio are permuted along with the noiy databit to match the order which wa originally ued in encoder #. The effective relabelling of the input to decoder # i illutrated below in fig.6. ( xk MAP # p' k P p' k ( xk Figure.6 Iteration # After the relabelling we have the following input to MAP decoder #. P p' k Author: VA of 4

12 Jul 4, 5 turbo_code_primer Reviion. Uing the trelli diagram in fig.4 to determine x k, p k for the variou tate tranition we calculate the branch probabilitie for the firt tage: γ (, exp( ( ( x x' p' exp( γ (, exp( + ( x + x' + p' exp( γ (, exp( + ( x + x' p' exp( γ (, exp( ( x x' + p' exp( γ (, exp( + ( x + x' + p' exp( γ (, exp( ( x x' p' exp( γ (, exp( ( x x' + p' exp( γ (, exp( + ( x + x' p' exp( Repeating for all tage of the trelli we get: ', γ ( ', γ ( ', γ ( ', γ 4 ( ', γ 5 ( ', γ 6 ( ', γ 7 ( ', γ 8 ( ',, , , , , , , , Performing the forward recurion and normaliing at each tage give: α ( α ( α ( α ( α 4 ( α 5 ( α 6 ( α 7 ( α 8 ( Performing the backward recurion and normaliing at each tage give: β 8 ( β 7 ( β 6 ( β 5 ( β 4 ( β ( β ( β ( β ( Author: VA of 4

13 Jul 4, 5 turbo_code_primer Reviion. We again compute the oft deciion of the MAP decoder and eperate out the extrinic information: -[ + ] If we lice the oft deciion we get the hard deciion. Comparing thi with the encoder # trelli path in fig. we ee that the decoder ha correctly etimated all databit... Further MAP Iteration The following table ummarie the extrinic information generated during the firt 8 MAP iteration. # # # # #4 #5 #6 #7 ( xk ( xk ( xk ( xk ( xk ( xk ( xk ( xk Inthe cae of our example the extrinic information converge to a teady tate olution by the 4th iteration. Author: VA of 4

14 Jul 4, 5 turbo_code_primer Reviion. 4. Reference [] S.A. Barbulecu, "Iterative Decoding of Turbo Code and Other Concatenated Code" [] W.E. Ryan,"A Turbo Code Tutorial" New Mexico State Univerity [] M.C. Reed, S.S. Pietrobon, "Turbo-code termination cheme and a novel alternative for hort frame" IEEE Int. Symp. on Peronal, Indoor and Mobile Radio Commun., Taipei, Taiwan, pp , Oct [4] J. Fei, On a Turbo Coder Deign for Low Power Diipation Author: VA 4 of 4