On the Achievabe Extrinsic Information of Inner Decoders in Seria Concatenation Jörg Kiewer, Axe Huebner, and Danie J. Costeo, Jr. Department of Eectrica Engineering, University of Notre Dame, Notre Dame, IN 46556, U.S.A. e-mai: {jkiewer,huebner.7,costeo.}@nd.edu Abstract In this paper we address the extrinsic information transfer functions of inner decoders for a seriay concatenated coding scheme. For the case of an AWGN channe, we give a universa proof for the fact that ony inner encoders yieding an infinite output weight for a weight-one input sequence, such as recursive convoutiona encoders, ead to perfect extrinsic information at the output of the corresponding SISO decoder. As an exampe we consider bit-intereaved coded moduation with iterative demapping (BICM-ID) and insert an additiona recursive precoder prior to the mapping operation. Simuation resuts show that the proposed system does not suffer from an error foor and thus significanty outperforms BICM-ID systems that soey use mappings as inner encodings, even when they are optimized. I. INTRODUCTION It has been shown in [], for a seriay concatenated coding scheme, that recursive inner encoders are preferabe since they aways ead to an intereaver gain, in contrast to nonrecursive or bock encoders. A simiar observation can be made by considering extrinsic information transfer functions [], [3]. If the sequence of information bits can be described as an independent and identicay distributed (iid) random process, ony recursive inner encoders can achieve an average mutua information of one between the information bits and the extrinsic soft-vaues if perfect a priori information is appied. This fact has been observed by many researchers and, in the case of two constituent encoders, is a necessary condition for the absence of an error foor. As an exampe we consider bit-intereaved coded moduation with iterative demapping (BICM-ID) [4]. BICM [5 7] represents a seria concatenation of an outer encoder, an intereaver, and a mapper or moduator which maps groups of bits to compex waveforms prior to transmission over the channe. Since the inner encoder is a nonrecursive one, perfect extrinsic information can never be achieved, eading to an error foor even for moderatey distorted channes. The achievabe average extrinsic information with perfect a priori information for the demoduator depends on the mapping, where it has been shown, e.g. in [8], that Gray mapping is unsuitabe in order to obtain a significant gain from iterations. However, improved mappings have been given in [7], [9], [0]. In [], a genera mapping optimization agorithm is proposed on the basis of maximizing the achievabe extrinsic information at the SISO demapper output, or, aternativey, This work was party supported by the German Research Foundation (DFG) under grant KL 080/3-, NSF grants CCR0-0530 and EEC-003366, and NASA grant NNG05GH73G. minimizing the pairwise error probabiity. However, generay, a noticeabe error foor is observed with a mappings, even when they are optimized. As a main resut of this paper we present a forma proof that ony inner encoders producing an infinite output weight for a weight-one input sequence, such as recursive inner encoders, ead to perfect extrinsic information at the output of the corresponding SISO decoder. For the sake of simpicity we restrict ourseves to AWGN channes. In contrast to [], where ony rate- convoutiona codes are considered via an anaysis of their state-space representation, our proof uses information theoretic considerations and universay hods for arbitrary code rates and inner encoders (not just convoutiona ones). As an exampe, our findings are then appied to BICM- ID where, in order to eiminate the error foor associated with soft demapping, we propose to add an accumuator as a recursive precoder prior to the mapper. At the decoder, the demapper is then repaced by a SISO decoder working on the joint accumuator / mapper treis. Simuation resuts for Graymapped 6-QAM show the performance gain of the proposed precoded BICM-ID scheme compared to those approaches that soey empoy optimized mappings without precoding. A reated approach has been discussed in [3], where decoding is carried out by iterating between a turbo decoder and the demapper. Whie this approach theoreticay aso eads to perfect overa mutua information after the iterative decoder has converged in practice this depends on the error foor of the outer turbo code. Another drawback compared to our approach is the arger decoding compexity. II. SYSTEM MODEL FOR INNER ENCODING Fig. depicts the underying system mode for inner encoding in a seria concatenation. A binary sequence V = [V,..., V,..., V N ] of ength N, with random variabes (RVs) V and reaizations v {0, }, is appied to a rate-n/q inner encoder C i. The resuting codeword U = [U,..., U,..., U Q ] is then transmitted over the BPSKmoduated AWGN communication channe. At the channe output the sequence Y = [Y,..., Y,..., Y Q ] consisting of soft-bit reaizations y IR with channe probabiity density function (pdf) p(y v ) = πσn exp ( (y ( v )) /(σ n) ), () is observed. Herein, σn = N 0 /(E s ) represents the normaized noise variance on the AWGN channe, E s the
V Fig.. [4]. Encoder C i U Comm. Channe A Priori Channe Y W, A i SISO Decoder System mode for decoding the inner code in a seria concatenation E i for the outer decoder. The mapping between I Ao and I Eo is then given as I Eo = T o (I Ao ), where T o ( ) represents the EXIT function of the outer channe decoder. An EXIT chart can now be obtained by potting the transfer functions T i/o for both constituent decoders on a singe diagram, where the axes must be swapped for one of the constituent decoders. For further detais about EXIT charts we refer to the corresponding iterature (e.g., [], [3], [8], [4]). transmitted energy per code bit, and N 0 the singe-sided noise power spectra density. The a priori AWGN channe modes the a priori information used at the inner constituent decoder in an iterative decoding scheme. At the output we observe the soft-bit sequence W = [W,..., W,..., W N ], where w IR represents a soft-bit reaization. The sequence A i = [A i,,..., A i,,..., A i,n ], with A i, := L (i) a (V ), () contains the corresponding a priori og-ikeihood ratios (LLRs) L a (i) (V ) = n p(w V = 0) p(w V = ) for {,,..., N}, (3) where the a priori channe pdf p(w v ) is anaogous to (). The SISO decoder empoys the outputs of both the communication and the a priori channes for computing extrinsic a posteriori probabiities (APPs) P (V = v y, a i, ), where a i, = [a i,,..., a i,, a i,+,..., a i,n ] describes reaizations of a priori LLRs with the -th eement a i, excuded. The sequence E i contains the extrinsic LLRs and is defined as E i = [E i,,..., E i,,..., E i,n ], with E i, := L (i) e (V ), (4) and L e (i) (V ) = n P (V = 0 y, a i, ) P (V = y, a i, ) for {,,..., N}. (5) Denoting the mutua information between two RVs X and Y as I(X; Y ), we define the quantities I Ai I Ei := N := N I(V ; A i, ), 0 I Ai, (6) = I(V ; E i, ), 0 I Ei. (7) = Herein, I Ai is defined as the average a priori information and denotes the average capacity of the a priori channe, whereas I Ei refers to the average extrinsic information. By defining a continuous mapping between a priori and extrinsic information as I Ei = T i (I Ai, E s /N 0 ), we obtain the inner extrinsic information transfer (EXIT) function or characteristic T i ( ) for the SISO decoder. Here, T i ( ) depends on both the capacity of the a priori and the communication channe, where the atter is characterized by the channe parameter E s /N 0. As in (6), (7), both I Ao and I Eo can aso be determined III. CONDITIONS FOR PERFECT INNER EXTRINSIC INFORMATION In the foowing we give a forma proof for the fact that an inner encoder producing an infinite output weight for a weight-one input sequence (e.g., a recursive convoutiona encoder) is sufficient for achieving I Ei (I Ai = bit) = bit. The average extrinsic information I Ei at the output of a genera inner SISO decoder in a seriay concatenated scheme can be expressed as I Ei = N I(V ; E i, )= N = H(V E i, ), (8) = where H(V E i, ) represents the conditiona entropy of V given E i,. In (8) we assume, without oss of generaity, that H(V ) = bit for a {,,..., N}. In [4], [5] it is shown that, for a SISO decoder emitting a posteriori probabiities (APPs), the mutua information I(V ; E i, ) can aso be written as I(V ; E i, ) = I(V ; A i,, Y ) = H(V ) H(V A i,, Y ). (9) We define the foowing information bit sequence reaizations v := [v,..., v, v +,..., v N ] and v q := [v,..., v, q, v +,..., v N ] with q {0, }. The conditiona entropy H(V A i,, Y ) in (9) may now be further expanded as H(V A i,, Y ) = H(V V, Y ) IA= = y v p(y, v ) P (V = q v, y) q=0 og (P (V = q v, y)) dy. (0) Lemma. Let the a-zero information sequence v (0) of ength N be appied to a recursive convoutiona encoder C of rate R = N/Q. Let the resuting BPSK-moduated codeword c (0) 0 = C{v (0) } be transmitted over an AWGN channe with noise variance σn, eading to the channe observation y with error sequence e 0 = y c (0) 0. Furthermore, et e = y c (0) = y ( C{v (0) }) for the same observation y denote the resuting error sequence when the weight-one information sequence v (0) is transmitted. A information bits except the one at bit position, {,,..., N}, are assumed to be perfecty known at the decoder. Then, for
Q, N, and fixed R for e 0 < e, P (v (0), y) = for e 0 = e, 0 for e 0 > e, where e = P (v (0) = v (0), y) = Q = e. 0 for e 0 < e, for e 0 = e, for e 0 > e. Proof. Ceary, a weight-one information sequence v (0) resuts in an infinite weight at the output of a recursive encoder if the bockength tends to infinity. When the encoded a-zero information sequence C{v (0) } is transmitted over the BPSK-moduated AWGN channe, y = c (0) 0 +n is observed at the channe output. Herein, n = [n,..., n,..., n Q ], where n, {,,..., Q}, represents a zero-mean Gaussian noise sampe reaization with variance σn. Additionay, achieving the same observed sequence y with the BPSK-moduated codeword c (0) eads to the error sequences e 0 = n, e = y c (0) = c (0) + n = d + n () with d = [d,..., d,..., d Q ] := c (0). The conditiona probabiities P (V (0) = q v (0), y), q {0, }, can be described by using Bayes theorem and assuming equiprobabe information sequences as foows: P (V (0) = q v (0), y) = p(y v (0) q ) p(y v (0) ) + p(y v (0) ). () For an AWGN channe the corresponding pdfs in () are given by and p(y v (0) ) = p(y v (0) ) = πσn πσn Q = Q = exp n σn σ n (3) exp ( (d + n ) ). (4) Combining (), (3), and (4) yieds the expression P (V (0), y) = ( e0 e ). (5) + exp σn We now consider the foowing three cases: (i) e 0 < e : The exponentia term in (5) can be rewritten as ( e0 e ) exp σn = ( exp Q ) ( σn d n exp Q ) σ d. (6) = n = }{{}}{{} =:α(q) =:β(q) Herein, α(q) does not necessariy tend to zero. However, due to the fact that d {0, } and im Q w(d/), where w( ) denotes the Hamming weight, it is cear that im Q β(q) = 0 in (6). But since e 0 < e, the overa product α(q) β(q) in (6) tends to zero as we. Thus, from (5) we obtain for N, eading to Q with Q = N/R for fixed R,, y) =, = v (0), y) = 0. (ii) e 0 > e : We exchange v (0) and v (0) derivation, where now e = y c (0) = n, e 0 = d + n. in the above Anaogousy to the previous case, it can be shown from (5) that (iii) e 0 = e : Here, we obtain, y) = 0, = v (0), y) =., y) =, = v (0), y) =. Note that for a given v (0) and v (0), this case occurs ony for specific observations y ying on a (Q )-dimensiona hyperpane characterized by e 0 = e. We can now state the main resut of this section in the foowing theorem. Theorem. Ony an inner recursive convoutiona encoder C i in a seriay concatenated scheme achieves I E (I A = bit) = bit for an information bockength tending to infinity and transmission over a BPSK-moduated AWGN communication channe. Proof. Let c 0 := C i {v 0 } and c := C i {v }, respectivey. Assume that we transmit the BPSK codeword c 0 over an AWGN channe, where the sequence y is observed at the channe output. Due to the inearity of the encoder
we may then express the error sequences e 0/ required for receiving y when the codeword c 0/ is transmitted as e 0 = n, e = c 0 c + n = c (0) + n, which are the same as in (). Using the resut from Lemma, we concude that either, y) =, = v (0), y) = 0, or, y) = 0, = v (0), y) =. (7) for a information sequence reaizations v V and any communication channe observation y. Inserting (7) into (0) and considering that im x 0 x og x = 0 proves the theorem. (Note that the case e 0 = e in Lemma need not be considered in the integration of (0) since the corresponding y are ocated on a singuar hyperpane with respect to the function to be integrated.) It is cear from the proof of Theorem that the condition I Ei (I Ai = bit) = bit cannot be satisfied for inner coding schemes, where an information sequence weight of one generates a finite codeword weight. This hods for a bock codes and non-recursive convoutiona encoders, as we as for moduators or ISI channes. Hence, Theorem provides an aternative expanation of the we-known fact that nonrecursive convoutiona encoders shoud not be used as inner encoders in a seriay concatenated coding scheme []. IV. EXAMPLE: BICM WITH RECURSIVE PRECODING As an exampe we consider a BICM-ID scheme with recursive precoding, where the corresponding encoder is shown in Fig.. The binary input sequence B is appied to a rate-r o outer encoder C o generating the sequence C. After bit-intereaving using the permutation function π, we obtain V = [V,..., V,..., V N ] = π(c ). In order to be abe to achieve an average extrinsic information of I Ei = bit, we insert an accumuator as a recursive precoder with generator poynomia g(d) = /( + D) before the mapping operation. The resuting sequence is denoted V. Aternativey, we may decompose V into M-bit symbos, resuting in the symbo vector I = [I,..., I k,..., I L ], with N = ML, where k denotes the symbo-time instant. Each I k has a reaization i k I in the finite aphabet I = {0,,..., M }. The i k are then mapped to signa points u k in the compex pane using the mapping function M : I C, where u k is a reaization of U k and U = [U,..., U k,..., U L ]. Then U is transmitted over an AWGN channe with the parameter E s /N 0. The received sequence Y at the output of the communication channe is then appied to the iterative decoder, which contains both an outer SISO decoder for encoder C o and an inner symbo-by-symbo SISO decoder working on the joint treis for the accumuator / M -ary mapper as constituent decoders. The treis for the inner decoder has M states, and B Encoder C o Fig.. Intereaver C V V π +D S/P... M M inner encoder Mapping BICM encoder structure with a recursive precoder. U Comm. Channe E s/n 0 each state s k at time instant k has transitions to a states s k at time instant k, where M transitions per state aways correspond to both the same input symbo i k I and output symbo i k I. An exampe is given for QPSK with M = in Fig. 3, where the state transitions are abeed i k /i k to indicate the input and output symbos. Decoding is carried out via a symbo-based BCJR agorithm [6] which computes the APPs P (I k = i k y, P a (I k = i k y)) given the a priori conditiona probabiities P a (I k = i k y). 00 0 0 0/00 00/00 00/00 0/0 /0 /0 0/0 0/00 0/0 /0 /0 00/ 00/ 00/ 00/ 0/0 s k s k s k+ Fig. 3. Treis representation for the joint accumuator / M -ary mapper for QPSK with M =. The abes on the state transitions denote (input symbo) / (output symbo). Numerica resuts are obtained for an inner accumuator combined with Gray-mapped 6-QAM, where the outer encoder is a memory- recursive systematic convoutiona (RSC) encoder with feedforward (g ) and feedback (g r ) poynomias (g r, g ) = (3, ) 8 in octa form and a code rate of R o = /. This system has an effective throughput of η = R o M = bit/s/hz. Fig. 4 shows the corresponding EXIT chart and some snapshot decoding trajectories for E s /N 0 = 7.5 db. We observe that the EXIT characteristic for the combined inner accumuator / mapper achieves I Ei = bit for perfect a priori information, as guaranteed by Theorem. The bit error rate (BER) versus E b /N 0 curve, with E b = E s /η, is shown in Fig. 5. Besides the proposed system, two BICM-ID systems empoying optimized mapping without recursive precoding are aso shown for comparison. The M6a mapping is proposed in [], where it was obtained by an optimization approach minimizing the difference I Ei (I Ai = bit). In [7] the MSP mapping was suggested as a good trade-off between arge I Ei for both zero and perfect a priori information. For both the proposed system and the M6a and MSP mappings, the (g r, g ) = (3, ) 8 outer RSC encoder is used. The BER was averaged over 000 simuated transmissions for a bockength of 0,000 information bits, and 0 iterations were aowed for the iterative decoder. It
IAo, I Ei 0.8 0.6 0.4 0. Gray-mapped 6-QAM + accumuator E s /N 0 =7.5 db, symbo-based BCJR dec. measured decoding trajectories RSC code (g r, g ) 8 = (3, ) 8, BCJR dec. 0 0 0. 0.4 0.6 0.8 I Eo, I Ai Fig. 4. EXIT chart and snapshot decoding trajectories for the accumuator / 6-QAM Gray mapper with a memory- (g r, g ) = (3, ) 8 RSC outer encoder. can be seen from Fig. 5 that, in contrast to both the MSP and M6a mappings, no error foor is observed for the joint accumuator / Gray mapper, where a simuated transmissions were observed as error-free for E b /N 0 > 4 db. This confirms the fact that I Ei (I Ai = bit) = bit is a necessary condition for the absence of an error foor in this case. In particuar, at a BER of 0 6, the joint accumuator / Gray mapper outperforms the M6a mapping by over db. This shows the advantage of the proposed scheme compared to using optimized mappings without recursive precoding. BER 0 0 0 0 4 0 6 Acc.+Gray mapping M6a mapping MSP mapping.5 3 3.5 4 4.5 5 5.5 6 E b /N 0 [db] Fig. 5. BER versus E b /N 0 for the proposed inner joint accumuator / 6- QAM Gray mapper and the M6a [] and MSP [7] mappings for the 6-QAM consteation (outer (g r, g ) = (3, ) 8 RSC encoder, 000 simuated transmissions, bockength 0,000 information bits). V. CONCLUSIONS We have considered inner encoding in a seriay concatenated coding scheme. For the corresponding decoder we have given a universa proof for the fact that an encoder that produces an infinite output weight for a weight-one input sequence (e.g., a recursive convoutiona encoder) eads to perfect extrinsic information when perfect a priori information is avaiabe. As an exampe, we considered a BICM-ID scheme in which the SISO demapper does not achieve perfect extrinsic information even when optimized symbo mappings for the consteation points of the moduation are used. This eads to a noticeabe error foor in the BER performance curve. We proposed adding an accumuator as a recursive precoder before the mapper aong with Gray mapping, and at the receiver the demapper in the iterative decoder is repaced by a symbo-bysymbo SISO decoder on the joint accumuator / mapper treis. Simuation resuts for a arge number of channe reaizations have not shown any error foor for the proposed scheme, in contrast to using optimized mappings without precoding. Finay, we note that the proposed scheme may further be optimized by choosing mappings other than Gray mapping. REFERENCES [] S. Benedetto, D. Divsaar, G. Montorsi, and F. Poara, Seria concatenation of intereaved codes: Performance anaysis, design, and iterative decoding, IEEE Trans. Inf. Theory, vo. 44, no. 3, pp. 909 96, May 998. [] S. ten Brink, Convergence behavior of iterativey decoded parae concatenated codes, IEEE Trans. Comm., vo. 49, no. 0, pp. 77 737, October 00. [3] J. Huber and S. Huettinger, Information processing and combining in channe coding, in Proc. 3rd Internationa Symposium on Turbo Codes & Reated Topics, Brest, France, Sept. 003, pp. 95 0. [4] S. ten Brink, S. Speide, and R.-H. Yan, Iterative demapping and decoding for mutieve moduation, in Proc. IEEE Goba Teecommun. Conf., Sydney, Austraia, Nov. 998, vo., pp. 579 584. [5] E. Zehavi, 8-PSK treis codes for a Rayeigh channe, IEEE Trans. Comm., vo. 40, no. 5, pp. 873 884, May 99. [6] G. Caire, G. Taricco, and E. Bigieri, Bit-intereaved coded moduation, IEEE Trans. Inf. Theory, vo. 44, no. 3, pp. 97 946, May 998. [7] A. Chindapo and J. A. Ritcey, Design, anaysis, and performance evauation for BICM-ID with square QAM consteations in Rayeigh fading channes, IEEE J. Se. Areas in Commun., vo. 9, no. 5, pp. 944 957, May 00. [8] S. ten Brink, Convergence of iterative decoding, Eectron. Lett., vo. 35, no. 0, pp. 806 808, May 999. [9] X. Li, A. Chindapo, and J.A. Ritcey, Bit-intereaved coded moduation with iterative decoding and 8PSK signaing, IEEE Trans. Comm., vo. 50, no. 8, pp. 50 57, Aug. 00. [0] J. Tan and G. L. Stüber, Anaysis and design of symbo mappers for iterativey decoded BICM, IEEE Trans. Wireess Commun., vo. 4, no., pp. 66 67, Mar. 005. [] F. Schreckenbach, N. Goertz, J. Hagenauer, and G. Bauch, Optimization of symbo mappings for bit-intereaved coded moduation with iterative decoding, IEEE Commun. Lett., vo. 7, no., pp. 593 595, Dec. 003. [] M. Tuecher, Design of seriay concatenated codes depending on the bock ength, IEEE Trans. Comm., vo. 5, no., pp. 09 8, Feb. 004. [3] F. Schreckenbach and G. Bauch, Irreguar signa consteations, mappings and precoder, in Proc. Internationa Symposium on Information Theory and its Appications, Parma, Itay, Oct. 004, pp. 33 336. [4] A. Ashikhmin, G. Kramer, and S. ten Brink, Extrinsic information transfer functions: Mode and erasure channe properties, IEEE Trans. Inf. Theory, vo. 50, no., pp. 657 673, Nov. 004. [5] I. Land, P. Hoeher, and S. Gigorević, Computation of symbo-wise mutua information in transmission systems with ogapp decoders and appication to EXIT charts, in Proc. Internationa ITG Conference on Source and Channe Coding (SCC), Erangen, Germany, Jan. 004, pp. 95 0. [6] L. R. Bah, J. Cocke, F. Jeinek, and J. Raviv, Optima decoding of inear codes for minimizing symbo error rate, IEEE Trans. Inf. Theory, pp. 94 87, Mar. 974.