2146 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 61, NO. 6, JUNE 2013

Transcription

1 246 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 6, NO. 6, JUNE 203 On Achieving an Asymptoticay Error-Free Fixed-Point of Iterative Decoding for Perfect APrioriInformation Jörg Kiewer, Senior Member, IEEE, and Danie J. Costeo, Jr., Life Feow, IEEE Abstract In this paper we provide necessary and sufficient conditions for constituent codes in (mutipe) concatenated and graph-based coding schemes to achieve an asymptoticay errorfree iterative decoding fixed-point if the maximum possibe a priori information is avaiabe. At east one constituent code in an iterative decoding scheme must satisfy these conditions in order to ensure an asymptoticay vanishing bit error probabiity at the convergence point of the decoder. Our resuts are proved for arbitrary binary-input symmetric memoryess channes (BISMCs) and thus can be universay appied to many transmission scenarios. Specificay, using a factor graph framework, it is shown that non-inner codes in a seria concatenation or check nodes in generaized LDPC codes achieve perfect extrinsic information if and ony if the minimum Hamming distance between codewords is two or greater. For inner codes in a seria concatenation, constituent codes in a parae concatenation, or variabe nodes in douby-generaized LDPC codes the corresponding encoder condition for acquiring perfect extrinsic information is an infinite codeword weight for a weight-one input sequence. For this case we provide a genera proof which hods for a inear encoders and BISMCs. We aso show that these resuts can improve the performance of concatenated coding schemes. Index Terms Extrinsic information transfer functions, iterative decoding, factor graphs, code concatenation. I. INTRODUCTION METHODS of characterizing the transfer of mutua information through a soft-input soft-output (SISO) decoder or, more generay, through factor nodes in a factor graph representation, such as extrinsic information transfer (EXIT) charts [2], [3], information processing charts (IPCs) [4], density evoution [5], [6], and generaized EXIT (GEXIT) charts [7] have emerged as usefu toos for predicting the convergence properties of iterative decoding schemes for ow density parity check (LDPC) codes or concatenated codes in the asymptotic bock ength regime. In EXIT or GEXIT charts, each constituent decoder or specific type of node in a factor graph is associated with such a transfer function that reates the mutua information of the aprioriinput to the extrinsic mutua Manuscript received May 24, 202; revised November 4, 202 and February 6, 203. The editor coordinating the review of this paper and approving it for pubication was H. Pishro-Nik. J. Kiewer is with the Kipsch Schoo of Eectrica and Computer Engineering, New Mexico State University, Las Cruces, NM 88003, USA (e-mai: jkiewer@nmsu.edu). D. J. Costeo, Jr. is with the Department of Eectrica Engineering, University of Notre Dame, 275 Fitzpatrick Ha, Notre Dame, IN 46556, USA (e-mai: costeo.2@nd.edu). This work was party supported by NSF grants CCF , CCF , CCF , and CCF This paper was presented in part at the IEEE Internationa Symposium on Information Theory, Seatte, WA, Juy 2006 []. Digita Object Identifier 0.09/TCOMM information at the decoder output. By combining the mutua information from a the transfer functions, the transfer of average mutua information between the individua decoders or factor nodes in a factor graph can be evauated and the decoding convergence behavior can be assessed. One important goa in designing a concatenated coding scheme is to arrive at a virtuay error-free reconstruction at the convergence point of the iterative decoder. In fact, achieving perfect extrinsic information, i.e., the maximum mutua information between aprioribits and extrinsic softinformation, at the output of at east one constituent decoder represents a necessary and sufficient condition for obtaining an asymptoticay vanishing bit error probabiity at the convergence point. This was aready observed in [2] for the specia case of a binary input AWGN channe. Conversey, this means that asymptoticay error-free reconstruction cannot be achieved if perfect extrinsic information cannot be obtained for at east one constituent decoder at the point of convergence. In this paper we address the required conditions for the constituent codes of (mutipe) concatenated and graph-based coding schemes to achieve perfect extrinsic information if perfect aprioriinformation is avaiabe. This is equivaent to an anaysis of the fixed-point of iterative decoding for perfect aprioriinformation. Our considerations are carried out for arbitrary binary-input symmetric memoryess channes (BISMCs) and thus can be universay appied to many transmission scenarios. A BISMC randomy maps the input symbos X i {0, } into output symbos Y i taken from a Q-ary set Y = {0,,...,Q } according to the transition probabiities P (Y = y X = x). A channe is said to be symmetric if it can be decomposed into strongy symmetric subchannes [5]. First, we extend an earier resut for non-inner codes in seria concatenation, namey that a minimum Hamming distance of two or greater between codewords is necessary and sufficient in order to achieve perfect extrinsic information [8], to non-inner factor nodes in factor graphs, i.e., to factor nodes without access to channe information. This resut hods for arbitrary inear and non-inear codes and apriorichannes and is, for exampe, usefu for improving the abiity of joint source-channe decoding schemes to expoit residua source redundancy [9] [] or outer entropy encoding [2], [3], which often exhibit a minimum distance of ony one between two source codewords. It aso appies to check nodes in generaized LDPC codes, where generic inear codes can be /3$3.00 c 203 IEEE This aso appies to douby-generaized LDPC codes [4] where both check and variabe nodes can be repaced by inear bock codes.

2 KLIEWER and COSTELLO: ON ACHIEVING AN ASYMPTOTICALLY ERROR-FREE FIXED-POINT OF ITERATIVE DECODING FOR PERFECT A PRIORI used as check nodes, and thus extends the resut of [4] for a binary erasure apriorichanne to arbitrary BISMCs. Second, we address the case of inner codes in a seria concatenation, constituent codes in a parae concatenation, or variabe nodes in douby-generaized LDPC codes [4]. It has been shown in [5] for a seriay concatenated coding scheme that recursive inner encoders are preferabe since they aways ead to an intereaver gain, in contrast to nonrecursive or bock encoders. A simiar observation has been made in [6] for the constituent encoders of a (parae concatenated) turbo code. This can aso be noticed by considering EXIT functions: If the sequence of information bits can be described as a uniformy distributed i.i.d. (independent and identicay distributed) process, ony recursive inner encoders in seria concatenation or recursive constituent encoders in parae concatenation can achieve the maximum average mutua information of one bit between the information bits and the extrinsic soft-vaues if perfect aprioriinformation is avaiabe. The same hods for inner factor nodes, i.e., for factor nodes with direct access to channe information. Note that in contrast to [5], [6], where the use of recursive encoders is motivated by an anaysis of error probabiity for maximum ikeihood (ML) decoding based on the weight distribution of concatenated codes, we show that the findings in [5], [6] aso hod in the context of iterative decoding. In addition, we present a forma proof for seria concatenation and arbitrary BISMCs that, for bockengths tending to infinity, ony inner encoders which generate an infinite output weight for a weight-one input sequence, such as recursive inner encoders, ead to perfect extrinsic information at the output of the corresponding SISO decoder. This fact was observed in [7] but no anaysis was given. In contrast to [8], where ony rate- convoutiona codes are considered via an anaysis of their state-space representation, our proof uses information theoretic considerations and universay hods for arbitrary code rates and inear encoders. This is aso verified by the fact that these resuts can be equivaenty formuated in a factor graph framework. As an exampe for our findings, we address iterativey-decoded bit intereaved coded moduation (BICM-ID), where, in order to eiminate the error foor associated with soft demapping, we consider adding an accumuator as a recursive precoder prior to the mapper. At the decoder, the soft demapper is then repaced by a SISO decoder working on the joint accumuator / mapper treis. By combining the resuts for both inner and non-inner codes, necessary and sufficient conditions for the existence of a fixed-point of iterative decoding for perfect a priori information are provided. The same conditions aso hod for the existence of a fixed-point for density evoution by considering the equivaent proofs in a factor graph framework. The paper is organized as foows. In Section II the underying system mode and notation is introduced. Section III addresses both the case of non-inner codes in a seria concatenation and non-inner factor nodes and proves, by using a factor graph framework, that a minimum distance of two or greater between codewords is necessary and sufficient for achieving perfect extrinsic information. As an exampe we consider iterative joint source-channe decoding of a Markov source. In Section IV the case of inner codes in a seria concatenation, constituent codes in a parae concatenation, and inner factor nodes is addressed. Here we show that for a bockength tending to infinity perfect extrinsic information is achieved if and ony if the encoder provides an infinite output codeword weight for a weight-one information sequence. Furthermore, the finite bockength case is addressed, where it is shown for the binary-input AWGN channe that a moderatey arge output weight for a weight-one input is sufficient to get cose to the idea case. Then our findings are appied to improve the performance of a BICM-ID-based transmission system. Finay, some concusions are given in Section V. II. SYSTEM MODEL In the foowing, random variabes (r.v. s) are denoted with upper case etters, and the corresponding reaizations with ower case etters, such as A or a, respectivey. Sequences are represented with upper or ower case bod etters, such as A or a. Further,weusep( ) for probabiity density functions (pdfs) and P ( ) for probabiities and probabiity mass functions (pmfs). In the foowing we present two different constituent decoder modes which wi be empoyed in this paper, an apriori channe mode and a mode based on factor graphs and the sum-product agorithm (SPA). A. A priori channe mode of the constituent decoder The underying system mode is depicted in Fig. [3, Fig. 2]. For the iterative decoding of a non-inner code in a (mutipe) seria concatenation, the connections marked with are active in Fig.. In this case, X c = X a and the communication channe is not used at a. A other scenarios for iterative decoding (inner seriay concatenated code and (mutipe) parae concatenation) are modeed by Fig. with the ganged switches in position 2. Note that the case where the connection marked with is active aso describes the situation for the check nodes of generaized LDPC codes [4], [9] (see aso the description of the factor graph mode in Section II-B). Assume a binary sequence B =[B,B 2,...,B,...,B K ] of ength K with r.v. s B and corresponding reaizations b {0, } is appied to a code C with code rate R c = K/N. Specificay, for a joint source-channe coding scenario, the sequence B may be interpreted as the binary version of a ength-l index sequence I =[I,I 2,...,I k,...,i L ] with J- bit indices I k = i k I, I = {0,,...,2 J } and K = LJ. In the simpest case such an index may be obtained by J-bit quantization of a waveform source sampe, but generay it represents an arbitrary J-bit source encoder parameter. A binary codeword X c =[X c,,x c,2,...,x c,,...,x c,n ] of ength N is generated by the encoder and transmitted over the communication channe. At the channe output the sequence Y c = [Y c,,y c,2,...,y c,,...,y c,n ] is observed. The a priori channe (or extrinsic channe) modes the a priori information used at each constituent decoder in an iterative decoding scheme. The input sequence X a = [X a,,x a,2,...,x,...,x a,m ] has reaizations x from the binary aphabet X a, where M = N if in Fig. the switch is in position (i.e., in the case of a non-inner decoder in seria concatenation) and M = K otherwise. The a priori channe observation is denoted by Y a = [Y a,,y a,2,...,y,...,y a,m ]. The sequence A contains the

3 248 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 6, NO. 6, JUNE 203 I, B Encoder (Code C) X c 2 Comm. Channe Y c SISO D 2 X a APriori Channe Y a, A Decoder E Fig.. Underying apriorichanne system mode [3, Fig. 2]. corresponding aprioriog-ikeihood ratios (L-vaues) with eements ( ) p(y X =0) A n, for =, 2,...,M, () p(y X =) where the (a priori) channe probabiity density function (pdf) is p(y x ). We aso define the sequences v a,\ [v a,,...,v,v +,...,v a,n ] and v q [v a,,...,v,q,v +,...,v a,n ] for any generic ength N sequence v. The symbo-by-symbo SISO decoder empoys both the outputs of the communication and apriorichannes for computing a posteriori probabiities (APPs) P (X = x y c, y a ) and extrinsic APPs P (X = x y c, y a,\ ), x {0, }, which foows the mode in [3]. The sequences D and E comprise the corresponding conditiona L-vaues and each eement is defined as r.v.s ( ) P (X =0 Y c, Y a ) D n and P (X = Y c, Y a ) ( ) (2) P (X =0 Y c, Y a,\ ) E n, P (X = Y c, Y a,\ ) resp., for =, 2,...,N,whereY c and Y a,\ denote the random vectors corresponding to the reaizations y c and y a,\. Denoting the mutua information between two r.v. s X and Y as I(X; Y ), we define the quantities I A N I E N N I(X ; A ), 0 I A, (3) = N I(X ; E ), 0 I E, (4) = where I A is defined as the average aprioriinformation and I E refers to the average extrinsic information, respectivey. These quantities are reated to the average sequence or word-wise mutua information per channe input bit by the expressions N I(X a; A) I A, N I(X a; E) I E. (5) By defining a continuous mapping between a priori and extrinsic information, we obtain the EXIT function (transfer characteristic) T for the SISO decoder in the mode of Fig. as I E = T (I A,I ch ),wherei ch I(X c ; Y c ). B. Factor graph mode Optima decoding for a channe code or redundant source code incudes computing the marginas of a goba function. The idea of factor graphs [20] [22] is to aeviate the burden of this computation by breaking the goba function up into µ (in) (x ) f(x,x 2,...,x M ) µ (out) m (x m ) µ (in) M (x M) x x 2 x m x M Fig. 2. Underying factor graph mode for a fixed choice of m. smaer oca functions [23]. Thus we obtain bipartite graphs which consist of factor nodes representing the computation of the oca functions [24]. A we-known exampe is the factor graph representation of LDPC codes in conjunction with the SPA [5], [9], [2]. Here the factor nodes associated with the individua parity check equations described by the rows of the parity check matrix H are caed check nodes, and the factor nodes associated with the distribution of the received channe information are caed variabe nodes, respectivey. Factor graph representations are aso known, e.g., for convoutiona and turbo codes [20], for Markov sources [20], [25], and for non-inear codes such as variabe-ength codes [25], to which the SPA can be appied as we. In the foowing, we wi empoy a simpe update rue for the SPA on a factor graph. Consider a factor node of degree M associated with a goba function f(x,x 2,...,x M ),where x, x 2,..., x M are the M independent variabes associated with this function. Note that f(x,x 2,...,x M ) can be factored into a product of oca functions [24]. The SPA update rue states that the outgoing message aong the edge directed towards x m, m =,...,M, is given as m (x m )= f(x,x 2,...,x M ) x,...,x m,x m+,...,x M γ m m μ (in) (x ), where γ m > 0 is a normaization factor. Herein, the messages μ (in) ( ) and m ( ) are defined as μ (in) (x ) p(y x ) and m (x m ) P (x a,m y a,\m, y c ), and with this definition decoding on the factor graph corresponds to decoding on the SISO decoder in Fig.. The update rue in (6) is iustrated in Fig. 2. (6)

4 KLIEWER and COSTELLO: ON ACHIEVING AN ASYMPTOTICALLY ERROR-FREE FIXED-POINT OF ITERATIVE DECODING FOR PERFECT A PRIORI III. NON-INNER CODES IN SERIAL CONCATENATION A. Condition for perfect extrinsic information As a starting point, consider the average extrinsic information I E at the output of the SISO decoder in Fig. given by I E = N N I(X ; E ) = = I E,max N N H(X E ) [0, ], (7) = where I E,max N N = H(X ). It can be seen from (7) that in order to maximize I E we require H(X E )=0for a. In [3], [26] it was shown that for a SISO decoder emitting APPs the mutua information terms I(X ; E ) can aso be written as I(X ; E )=I(X ; Y a,\, Y c )= H(X ) H(X Y a,\, Y c ), (8) where Y a,\ =[Y a,,...,y,y +,...,Y a,n ] describes the received sequence at the output of the apriorichanne. Since for non-inner decoders in a seria concatenation the communication channe is not present in the mode of Fig., the conditiona entropy H(X Y a,\, Y c ) in (8) does not depend on Y c. Without oss of generaity, we consider discrete output channes in the foowing 2. Then, the conditiona entropy H(X Y a,\ ) may be further expanded as H(X Y a,\ )= y a,\ q=0 ( og 2 P (y a,\ ) where P (y a,\ )= x a X M a :x =q x a X M a :x =q [ P (ya,\ x a ) P (x a ) ] ) P (y a,\ x a ) P (x a ), (9) P (y a,\ x a) P (x a)+ x a XM a :x =0 P (y a,\ x a) P (x a). (0) x a XM a :x = From (8) we can see that, for bockengths tending to infinity, X can be estimated from Y a,\ (and Y c for the case of inner decoders), with vanishing (bit) error probabiity P e when I A = I A,max, if and ony if H(X Y a,\ )=0. By appying Fano s inequaity and considering that x {0, }, P e can be ower bounded as P e h ( H(X Y a,\ ) ),whereh(p) = p og(p) ( p)og( p) is the binary entropy function for p [0, 0.5]. This shows that if H(X Y a,\ ) > 0, asymptoticay a non-zero error probabiity remains even when I A = I A,max. For (mutipe) seria concatenation and a (not necessariy symmetric) binary-input memoryess apriorichanne mode 2 For continuous-output channes the probabiity mass functions (pmfs) need to be repaced by probabiity density functions (pdfs) and the sums over the channe outputs by integras, respectivey. (see Fig. ), we restate the foowing theorem from [8], which provides a reationship between the minimum Hamming distance of the codewords X a and the achievabiity of perfect extrinsic information at the output of a non-inner SISO decoder. In contrast to [8], where a proof based on the apriori channe mode is given, we provide a new simper proof which generaizes the vaidity of Theorem to any non-inner factor node in a factor graph mode empoying the SPA, for exampe to the check nodes of generaized LDPC codes. Theorem ( [8]): For any binary-input memoryess apriori channe, et I E = N N = I(X ; E ) denote the average extrinsic information at the output of a non-inner SISO decoder in a (mutipe) seria concatenation. Furthermore, et I A,max = I E,max = N N = H(X ) and et d min denote the minimum Hamming distance between the 2 K N- bit codewords X a = x a X M a of the code C. Then, d min 2 is a necessary and sufficient condition such that I E (I A = I A,max )=I E,max at the decoder output. Proof: Consider a non-inner factor node of degree M = N with associated variabes x a,,x a,2,...,x a,m, a corresponding function defined as { if [xa,,x a,2,...,x a,m] C, f(x a,,x a,2,...,x a,m) = 0 otherwise. () and the SPA update equation in (6), where C is a (not necessariy inear) binary code with bock ength M. Theorem states that if μ (in) (x )= and μ (in) (x ( x ))=0 (2) for a =,...,M, which is equivaent to I A = I A,max,then m (q) = q for q {0, } and a m =,...,M,which is equivaent to H(X a,m Y a,\m )=0, if and ony if d min 2. By using the incoming messages in (2), the SPA update rue in (6) simpifies significanty, and we obtain the foowing outgoing messages: m (0) = f(x a,,...,x a,m, 0,x a,m+,...,x a,m ), m () = f(x a,,...,x a,m,,x a,m+,...,x a,m ). If d min 2 we have m (q) = q for q {0, } and a m =,...,M by considering the function definition in (). However, if d min =there exists at east one m for some codeword [x a,,...,x a,m ] Csuch that m (0) = m () = 0.5 after normaization. Remark: The theorem can aso be shown to hod for nonbinary channe input aphabets in a straightforward way by modifying the incoming and outgoing messages in the SPA appropriatey. B. Impications for joint source-channe coding In genera, d min 2 hods for non-inner encoders in seria concatenation. However, Theorem has important consequences for joint source-channe coding schemes with an outer source code and an inner channe encoder, as discussed in the foowing two cases. The first case considers a variabe-ength code (VLC) as a (noninear) mapping C in Fig.. Here, the free distance of a VLC is defined as the minimum Hamming distance d min between a possibe codeword combinations of any ength [27]. A free distance of d free =occurs, for exampe, if C

5 250 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 6, NO. 6, JUNE 203 I E [bits / channe use] (9, 5) shortened Hamming code no channe code (6, 5) singe parity check code 0.2 BEC AWGN BSC I A [bits / channe use] function compared to the (6, 5) singe parity check code. Note that for a memoryess source, a BEC apriorichanne 3, and H(X )=bit, it is shown in [3, Theorem ] that the area A is connected to the tota code rate R by the expression A = R. In genera, for an L-th order Markov source I, an (n, k) bock code, and a BEC apriorichanne, the area A under the EXIT function can be easiy obtained from [3, Theorem ] as ( ) A = IA,max 2 H(I) n = H(X, (3) ) where H(I) denotes the entropy rate of I. Here, the tota code rate R incudes both the rate contribution of the bock code and the residua redundancy of the Markov source (see, e.g., [30] [32]). Fig. 3. EXIT functions for uniform scaar quantization with J =5bits of a first-order autoregressive source with correation coefficient a = 0.95 and different (n, k) bock codes C. is chosen as a Huffman VLC, and it foows that I E (I A = bit) =bit cannot be achieved by a VLC SISO decoder [3]. A soution in this case is to insert expicit redundancy into the VLC codewords, as in the case of reversibe VLCs [28], eading to d free =2and to I E (I A =bit) =bit. The second case invoves the use of the impicit source redundancy inherent in the sequence I for error correction, e.g., when I is emitted from a Markov source. Transmitting uncoded source indices over the a priori channe in the absence of a code C eads to d min =. Thus, according to Theorem, the maximum possibe average extrinsic information cannot be obtained at the decoder output by soey expoiting residua source redundancy, even if there is a strong correation between adjacent source indices I k and I k. However, a code C can be additionay used to increase the distance between codewords, as can be seen from the foowing exampe. Exampe : Fig. 3 shows EXIT functions for a first-order Markov process I, which is obtained by uniform scaar quantization with J =5bits of a first-order autoregressive process with correation coefficient a =0.95. The code C is either absent or chosen as an (n, k) bock code, where n and k denote the ength of a singe code bock and information bock, respectivey. A natura mapping from quantizer reconstruction eves to J-bit source indices with J = k is used in a experiments, eading to I E,max =bit. The resuting EXIT functions are shown in Fig. 3, where the bock code and the Markov source with known index transition probabiities P (I k = i k I k = i k ) are jointy decoded using an indexbased BCJR agorithm [29]. The overa rate R is defined as R = R s R c, with R s = H(I k I k )/J denoting the rate contribution from the residua source redundancy. We can observe that, despite the strong source correation resuting in a source coding rate of R s =0.47, d min =impies that perfect average extrinsic information cannot be achieved even if perfect aprioriinformation is appied at the decoder input. If d min is increased to two, e.g., by adding a singe parity check to the source indices, we can observe from Fig. 3 that I E (I A =bit) =bit, which is consistent with Theorem. A further reduction of the code rate by using a shortened (9, 5) Hamming code eads to a arger area A under the EXIT IV. PARALLEL CONCATENATION AND INNER CODES IN SERIAL CONCATENATION A. Condition for perfect extrinsic information For the constituent encoders in parae concatenation and for the inner encoder in seria concatenation, the output Y c of the communication channe must be considered. In the mode of Fig., the switch is in position 2 since the constituent decoder has aprioriinformation on the information bits. In the foowing we assume inear encoders C. By considering the bit-wise extrinsic information in (8) for a continuous-output communication channe and assuming perfect aprioriinformation I A = I A,max for a given distribution of the source bits, the conditiona entropy can be written as H(X Y a,\, Y c ) = H(X X a,\, Y c ) (4) IA=I A,max = p(y c, x a,\ ) P (X = q x a,\, y c ) y c x a,\ q=0 og 2 (P (X = q x a,\, y c )). (5) We now show that H(X Y a,\, Y c ) =0can be IA=I A,max obtained for inner recursive convoutiona codes in seria concatenation and for recursive constituent convoutiona encoders in parae concatenation. The foowing emma is an extension of the resut in [] for the binary-input AWGN (BIAWGN) channe to arbitrary BISMCs. Lemma : Let the a-zero information word x a (0) of ength K be appied to a inear encoder C of rate R c = K/N. Let the resuting BPSK-moduated codeword c (0) 0 = 2C (x a (0) ) (an a + sequence), where C (x a ) denotes the mapping induced by code C, be transmitted over a BISMC with capacity C, eading to the channe observation y c. A information bits except the one at bit position, =, 2,...,K,are assumed to be perfecty known at the decoder. Then the fact that C generates an infinite output weight for a weight-one input sequence x (0),forK, N,andfixed R c, represents a necessary and sufficient condition for the 3 An extension of the area theorem to a BISMCs is given in [7].

6 KLIEWER and COSTELLO: ON ACHIEVING AN ASYMPTOTICALLY ERROR-FREE FIXED-POINT OF ITERATIVE DECODING FOR PERFECT A PRIORI foowing APP evauations for N : a,\, y c)=, P(X (0) (0) = x a,\, y c)=0, for 0 <C, a,\, y c)= 2, for C =0. P(X(0) (0) = x a,\, y c)= 2, Proof: In the foowing we state the proof based on the a priori channe mode in Fig.. We need a resut from [33], [34] stating that every BISMC with input x {0, } and a discrete or continuous output y can be decomposed into binary symmetric subchannes (BSCs). This can be obtained by decomposing the output aphabet Y of a BISMC into disjoint subsets Y(z) with one or two eements. The subsets Y(z) represent the output aphabets of the individua BSCs associated with a reaization Z = z Z of a so caed subchanne indicator Z independent of X,andZ denotes the aphabet of z. For channes with discrete outputs, Z is a discrete r.v. and the channe pmf can be decomposed as P (y x) =P (y,z x) =P (z) P (y x, z), z Z, y Y, y Y(z), x {0, }, (6) where P (y x, z) =P (y x) is the channe pmf of the BSC associated with Z = z. We assume non-zero seection probabiities P (z), which means that Z contains ony subchannes with non-zero capacity. For BISMCs with continuous outputs (such as the BIAWGN channe), Z is a continuous r.v. and the pmfs in (6) are repaced by pdfs 4. Eq. (6) hods as we if x, y, andz are repaced by the sequences x =[x,x 2,...,x N ], y =[y,y 2,...,y N ],andz =[z,z 2,...,z N ]. For the sake of simpicity, the foowing proof addresses BISMCs with discrete outputs (BISDMCs). By empoying Bayes theorem and assuming equiprobabe information sequences we can write = q x a,\, y c)= P (y c x (0) q ) P (y c x (0) a )+P (y c x (0) ), (7) q {0, }. By inserting the sequence-based version of (6) in (7) we obtain, for the a-zero input hypothesis, a,\, y c)= + P (y c x (0), z, (8) c) P (y c x a (0), z c ) where z c represents the subchanne indicator sequence for the communication channe in Fig.. First consider an arbitrary channe observation y c of ength N at the output of the BISDMC. Each eement y c,, =,...,N, is associated with a specific BSC subchanne with subchanne indicator z c,λ and transition probabiity ρ(z c,λ ), λ =,..., Z, where Z denotes the cardinaity of the set Z. Thus it suffices to show that im N P (X (0) = 0 x (0) a,\, y c) = hods asymptoticay for a BSCs with arbitrary transition probabiities ρ(z c,λ ) (0, /2). 4 Some exampes are given, e.g., in [35, Chapter 2]. Now, for each BSC subchanne z c,λ of the BISDMC with transition probabiity ρ(z c,λ ), consider the foowing error sequences: e 0,λ = y c,λ x a (0), e,λ = y c,λ x (0) = x a (0) x (0) e 0,λ, (9) where indicates binary addition and where y c,λ contains the part of the observed sequence y c at the output of the BIS- DMC which corresponds to the subchanne λ, 2,..., Z. Let us aso define N λ as the number of symbos in y c,λ,where the bock ength N = Z λ= N λ.byusingd x a (0) x (0) the two channe pmfs for the same observation y c can be written as P (y c x (0) a, z c )= Z λ= P (y c x (0), z c)= ( ρ(z c,λ )) N λ w(e 0,λ ) ρ(z c,λ ) w(e 0,λ), Z λ= ( ρ(z c,λ )) N λ w(d e 0,λ) (20) ρ(z c,λ ) w(d e 0,λ), (2) where w(x ) denotes the Hamming weight of sequence x.by inserting (20) and (2) into (8) we obtain a,\, y c)= + Z λ= ( ρ(zc,λ ) ρ(z c,λ ) ). (22) w(d e0,λ ) w(e 0,λ ) We now show that im Nλ (w(d e 0,λ ) w(e 0,λ )) is unbounded for infinite output weight w(d). The Strong Law of Large Numbers impies that P (im Nλ w(e 0,λ )/N λ = ρ(z c,λ )) = for any λ {, 2,... Z }. Thus, the term w(d e 0,λ ) can asymptoticay be written as im w(d e 0,λ) =w(d) w(d)ρ(z c,λ )+ N λ (N λ w(d))ρ(z c,λ ), (23) =( 2ρ(z c,λ ))w(d)+n λ ρ(z c,λ ), (24) where in (23) the second term on the right hand side indicates the number of bits that are fipped from to 0, and the third term indicates the number of bits which are fipped from 0 to, respectivey. From (24) we can now see that im (w(d e 0,λ) w(e 0,λ )) = ( 2ρ(z c,λ ))w(d), (25) N λ which, for ρ(z c,λ ) (0, /2), tends to infinity since our assumption im N w(d) impies that im Nλ w(d). By combining (22) and (25) and assuming that 0 <C BISDMC, which impies ρ(z c,λ )/( ρ(z c,λ )) < for a λ, we finay obtain im P N (X(0) (0) a,\, y c)= { for 0 <C BISDMC, 2 for C BISDMC =0. (26) This proves the sufficient part of Lemma. The necessary part can easiy be shown by forcing the quotient in the denominator of (8) to tend to zero for K, N, and fixed R c.

7 252 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 6, NO. 6, JUNE 203 For BISMCs with continuous outputs the proof can be carried out in a simiar way by considering the imiting case Z. Remarks: The above proof can be extended to inner factor nodes in factor graphs. The proof is reated to the above proof for the apriorichanne mode, and a sketch for BISDMCs is presented in the Appendix. Lemma says that for any C>0the missing bit x (0) can be perfecty estimated asymptoticay by an ML decoder if a other bits in the word are known at the decoder. Intuitivey, this works since the effective code rate R eff for the singe missing bit x is /N, and therefore there exists an N such that R eff <Ccan aways be guaranteed for any bockength N>N. We can now state the main resut of this section in the foowing theorem for the case where I A,max = I E,max =bit. Theorem 2: For an inner encoder in a seria concatenation or a constituent encoder in a parae concatenation of two or more codes, a recursive convoutiona encoder C is required to achieve I E (I A =bit) =bit for an information bockength tending to infinity and transmission over a BPSK-moduated BISMC with non-zero capacity. Proof: First note that for recursive convoutiona encoders the generator matrix G(D) in the transform domain has entries which are rationa functions in D. Thus, since for weight-one input sequences the rationa poynomia P (D) =/( m k= a kd k ), a k {0, }, with finite degree m in the denominator poynomia has an unbounded degree, the corresponding (time-domain) output sequence p has infinite weight. Let c 0 2 C (x 0 ) and c 2 C (x ), respectivey. Assume that we transmit the BPSK codeword c 0 and the sequence y c is observed at the channe output. Due to the inearity of the encoder and the fact that a weight-one information word x (0) asymptoticay resuts in an infinite codeword weight at the output of a recursive convoutiona encoder, we can directy appy Lemma. With c 0 corresponding to c (0) 0 and c corresponding to c (0),resp., we concude that im P (X a,\, y c )=, N (27) im P (X = x a,\, y c )=0 N for a information word reaizations b and a channe observations y c associated with a non-zero capacity. Inserting (27) into (5) and considering that im x 0 x og x =0proves the theorem. It is cear from the proof of Theorem 2 that the condition I Ei (I Ai = bit) = bit cannot be satisfied by codes for which an information word of weight one generates a finite codeword weight. This hods for a bock codes and nonrecursive convoutiona encoders. Hence this extends the we known resut that, with ML decoding, non-recursive convoutiona encoders shoud not be used as inner encoders in a seriay concatenated coding scheme [5] or as constituent encoders in parae concatenation [6] to the case of iterative decoding. Further, combining the resuts from Theorems and 2 provides necessary and sufficient conditions under which iterative decoding of concatenated inear codes has a fixedpoint if the aprioriinformation is perfect. I E Precod. + Gray map. M6a mapping MSP mapping natura mapping Gray mapping I E I A I A Fig. 4. Comparison of (inner) demapper EXIT functions for different 6- QAM mappings and E s/n 0 =7.5 db on an AWGN channe. The M6a and MSP mappings were proposed in [4] and [36], respectivey. These resuts aso ead to necessary and sufficient conditions for factor nodes in a factor graph framework empoying the iterative SPA to achieve asymptoticay perfect outgoing messages when perfect incoming messages are given. In the context of bipartite graphs for inear codes, where inner factor nodes are the variabe nodes and non-inner (outer) factor nodes are the check nodes, this resut suggests that, if the conditions stated in Theorem and Lemma are satisfied, density evoution has a fixed-point under perfect a priori information. Exampe 2: The resut from Theorem 2 is now appied to iterativey-decoded bit-intereaved coded moduation (BICM- ID) in a seria concatenation. The inner encoder simpy maps a bock of J bits to a compex waveform, and the corresponding constituent decoder is represented by a SISO demapper (see, e.g., [36], [37]). Since the inner encoder is non-recursive, perfect extrinsic information is never achieved for perfect a priori information. In order to be abe to achieve an average extrinsic information of I E =bit, we insert an accumuator with generator poynomia g(d) =/( + D) as a recursive precoder before the mapping operation (see, e.g., [38] [40]). SISO decoding can then be carried out on the joint treis for the recursive precoder and the mapper. Fig. 4 shows some constituent decoder EXIT functions for seected 6- QAM mappings and the recursive precoding foowed by Gray mapping for an AWGN channe with E s /N 0 =7.5 db. The M6a mapping was proposed in [4], where it was obtained by an optimization approach minimizing the difference I E (I A =bit). In [36], the MSP mapping was suggested as a good trade-off to achieve arge I E for both zero and perfect a priori information. We observe from Fig. 4 that ony Gray-mapped recursive precoding is abe to achieve I E =bit, whereas both the M6a and MSP mappings ony get cose to I E = bit. Athough for these mappings the deviation from idea behavior is very sma, this has a strong impact on performance in the error foor region. This can be seen from numerica resuts for a seriay concatenated coding scheme where the outer encoder is a rate R o =/2, memory-, recursive systematic convoutiona (RSC) encoder

8 KLIEWER and COSTELLO: ON ACHIEVING AN ASYMPTOTICALLY ERROR-FREE FIXED-POINT OF ITERATIVE DECODING FOR PERFECT A PRIORI with generator matrix G(D) = [ /( + D)]. The resuting bit error rate (BER) performance is shown in Fig. 5, where η = R o J = 2bit/s/Hz is the effective throughput and E b = E s /η. It is seen from Fig. 5 that, in contrast to both the MSP and M6a mappings, no error foor is observed for the Gray-mapped recursive precoding scheme, where a simuated transmissions were observed as error-free for E b /N 0 > 4 db. BER Precod. + Gray map. M6a mapping MSP mapping B. Achievabe extrinsic information for finite bockength The proof of Theorem 2 ony addresses the case of an infinite output codeword weight for an information word weight of one. In the foowing we address the question of how the achievabe extrinsic information for perfect a priori information depends on a finite output codeword weight w(d c ), with d c C (x a (0) ) C (x (0) ), where, without oss of generaity, we consider the BIAWGN channe. By considering the inearity of the code C, (5) can be rewritten as H(X Y a,\, Y c ) = p(y c x (0) y c = E p(yc) { IA=I A,max a,\ ) q=0 q=0 P (X = q x (0) a,\, y c) og 2 (P (X = q x (0) a,\, y c)) dy c P (X = q x (0) a,\, y c) } og 2 (P (X = q x (0) a,\, y c)), (28) where (28) expoits the fact that the pdf p(y c x (0) a,\ ) is conditioned on the a-zero information word. For the BIAWGN channe the encoded a-zero information word C (x a (0) ) is transmitted, where y c = c (0) 0 + n is observed at the channe output, n =[n,...,n,...,n N ],andn, =, 2,...,N, represents a zero-mean Gaussian noise sampe with variance σn. 2 Additionay, observing the same channe output sequence y c with the BPSK-moduated codeword c (0) = 2 C (x (0) ) eads to the error sequences e 0 = n, e = y c c (0) = c (0) 0 c (0) + n = d + n, (29) where d c (0) 0 c (0) = 2d c and w(d) = w(d c ).By empoying (8) with a pmfs repaced by Gaussian pdfs we obtain a,\, y c)= ( e0 2 e 2 ). (30) +exp 2 σn 2 Inserting this into (28) and considering (7) and (8), we obtain the desired reation between I E (I A = I A,max ) and the corresponding average output weight w(d c ) for a given channe noise variance σn. 2 In Fig. 6 this reation is potted for different E s /N 0 =/(2 σn 2) and I A,max =bit, where the expectation in (28) is taken over 000 channe reaizations. We can see from Fig. 6 that, for exampe, for E s /N 0 db an average codeword weight of w(d c )=6suffices to achieve I E bit. Thus, in practice, for moderate E s /N 0 and bock E /N [db] b 0 Fig. 5. BER versus E b /N 0 for the proposed inner Gray-mapped recursive precoding scheme and the M6a [4] and MSP [36] mappings for the 6- QAM consteation and an outer memory- RSC encoder (000 simuated transmissions, bockength information bits). IE(IA =bit) [bit / channe use] E /N = db s s/n 0 =db E /N = db s s/n 0 = db E /N = 3 db s s/n 0 = 3 db E /N = 5 db s s/n 0 = 5 db w(c (x (0) )) Fig. 6. Reation between I E (I A =bit) and the corresponding average codeword weight w(d c) for a weight-one information word x (0) as a function of E s/n 0 for a BIAWGN channe. or non-recursive convoutiona codes with a modest vaue of w(d c ), the error I E is very sma, so that these codes may be used as inner codes in seria concatenation or constituent codes in parae concatenation with ony a negigibe oss in performance compared to recursive convoutiona codes. V. CONCLUSION We have proved necessary and sufficient conditions for the constituent codes of concatenated coding schemes over BISMCs that aow them to asymptoticay achieve the maximum possibe average extrinsic information at the output of the corresponding SISO decoder if perfect aprioriinformation is avaiabe. By extending these resuts to a factor graph framework and the SPA the same conditions can be estabished for factor nodes, for exampe for check and variabe nodes of douby-generaized LDPC codes. Specificay, we have addressed two cases: For non-inner codes in a (mutipe) seria concatenation and non-inner factor

9 254 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 6, NO. 6, JUNE 203 nodes, a maximum average extrinsic information requires a minimum Hamming distance of two or greater between adjacent codewords. For inear inner codes in a (mutipe) seria concatenation, inear constituent codes in a (mutipe) parae concatenation, or inner factor nodes associated with a inear code, perfect extrinsic information is obtained if and ony if the encoder produces an infinite output weight for a weight-one input sequence (e.g., a recursive convoutiona encoder). Combining these resuts provides necessary and sufficient conditions under which both iterative decoding of concatenated inear codes and density evoution on a factor graph have a fixed-point for perfect aprioriinformation. Exampes for both cases have shown that the resuts can be used to improve overa system performance. In the non-inner code case the first exampe considers joint source-channe decoding schemes with an outer source mapping that produces codewords with a minimum Hamming distance of one (e.g., a Markov source). An additiona high-rate bock encoding after the source encoder resuts in d min 2, so that the joint SISO decoder is abe to achieve perfect extrinsic information. The second exampe for the inner code case considers BICM-ID in which the SISO demapper does not achieve perfect extrinsic information even when optimized mappings are used, eading to a noticeabe error foor in the BER performance curve. By adding an accumuator as a recursive precoder combined with joint SISO mapping / decoding, perfect achievabe extrinsic information is guaranteed for the inner decoder. APPENDIX PROOF OF LEMMA FOR A FACTOR-GRAPH FRAMEWORK For the sake of simpicity we ony consider discrete-output BISMCs. The extension to arbitrary BISMCs is straightforward. Consider a factor node of degree M = K and associated independent variabes x a,,...,x a,m with the function definition N f(x a,,x a,2,...,x a,m )= P (y c,t C t (x a )), (3) t= where x c,t = C t (x a ) is the t-th component of the codeword x c in the inear code C. Lemma states that if the a zero codeword x c (0) is transmitted, then μ (in) (q) = q for q {0, } and a =,...,M, (32) which is equivaent to I A = I A,max. Then, with probabiity as N, the normaized outgoing messages are m (q) = q for q {0, } and a m =,...,M, (33) which impies H(X a,m Y a,\m, Y c )=0. By using the incoming messages in (32), the SPA update rue in (6) simpifies significanty and we obtain the foowing outgoing messages: m (0) = m () = N P (y c,t C t (0, 0,...,0,...,0, 0)), t= N P (y c,t C t (0, 0,...,,...,0, 0)). t= We now have ( ) m (0) N ( ) P (yc,t C t (0, 0,...,0,...,0, 0)) n = n m () P (y t= c,t C t (0, 0,...,,...,0, 0)) ( ) P (yc,t 0) = n. P (y c,t ) t:c t(0,0,...,,...,0,0)= (34) Next we assume that the communication channe has a bit error probabiity of ess than 0.5, i.e., ( ) P (yc,t 0) P (y c,t 0) n > 0, P (y y c,t ) c,t t =,...,N, (35) and that ρ m = im w(c (0, 0,...,,...,0, 0)) > 0, N N m =,...,M. (36) This condition means that we assume the normaized output weight ρ m of C grows ineary in N for an input weight of one, since w(c (0, 0,...,,...,0, 0)) for N. By appying the Strong Law of Large Numbers to the r.h.s. of (34), asymptoticay we obtain with probabiity for a m =,...,M, t =,...,N that im N N n ( m (0) m () ) = ρ m E { n ( )} P (Yc,t 0) P (Y c,t ) = ρ m P (y c,t 0) y c,t ( ) P (yc,t 0) n > 0, (37) P (y c,t ) where Y c,t is distributed according to P (y c,t 0), and the inequaity on the r.h.s. of (37) foows from the assumptions in (35) and (36). From (37) finay the resut in (33) foows. ACKNOWLEDGMENT The authors wish to thank Pasca Vontobe for contributing the idea of appying the resuts for the apriorimode aso to factor graphs. REFERENCES [] J. Kiewer, A. Huebner, and D. J. Costeo, Jr., On the achievabe extrinsic information of inner decoders in seria concatenation, in Proc IEEE Int. Sympos. on Information Theory, pp [2] S. ten Brink, Convergence behavior of iterativey decoded parae concatenated codes, IEEE Trans. Commun., vo. 49, no. 0, pp , Oct [3] A. Ashikhmin, G. Kramer, and S. ten Brink, Extrinsic information transfer functions: mode and erasure channe properties, IEEE Trans. Inf. Theory, vo. 50, no., pp , Nov [4] S. Huettinger and J. Huber, Information processing and combining in channe coding, in Proc Int. Symp. on Turbo Codes & Re. Topics, pp [5] R. G. Gaager, Low-Density Parity-Check Codes. MIT Press, 963. [6] T. J. Richardson and R. L. Urbanke, The capacity of ow-density paritycheck codes under message-passing decoding, IEEE Trans. Inf. Theory, vo. 47, no. 2, pp , Feb [7] C. Méasson, A. Montanari, T. Richardson, and R. Urbanke, Life above threshod: from ist decoding to area theorem and MSE, in Proc IEEE Information Theory Workshop. [8] J. Kiewer, N. Goertz, and A. Mertins, Iterative source-channe decoding with Markov random fied source modes, IEEE Trans. Signa Process., vo. 34, no. 0, pp , Oct

10 KLIEWER and COSTELLO: ON ACHIEVING AN ASYMPTOTICALLY ERROR-FREE FIXED-POINT OF ITERATIVE DECODING FOR PERFECT A PRIORI [9] K. Sayood and J. C. Borkenhagen, Use of residua redundancy in the design of joint source/channe coders, IEEE Trans. Commun., vo. 39, no. 6, pp , June 99. [0] H. H. Otu and K. Sayood, A joint source/channe coder with bock constraints, IEEE Trans. Commun., vo. 47, no. 22, pp , Nov [] N. Görtz, On the iterative approximation of optima joint sourcechanne decoding, IEEE J. Se. Areas Commun., vo. 4, no. 9, pp , Sept [2] J. Kiewer and R. Thobaben, Iterative joint source-channe decoding of variabe-ength codes using residua source redundancy, IEEE Trans. Wireess Commun., vo. 4, no. 3, pp , May [3] R. Thobaben and J. Kiewer, Low compexity iterative joint sourcechanne decoding for variabe-ength encoded Markov sources, IEEE Trans. Commun., vo. 53, no. 2, pp , Dec [4] E. Paoini, M. P. C. Fossorier, and M. Chiani, Generaized and douby generaized LDPC codes with random component codes for the binary erasure channe, IEEE Trans. Inf. Theory, vo. 54, no. 4, pp , Apr [5] S. Benedetto, D. Divsaar, G. Montorsi, and F. Poara, Seria concatenation of intereaved codes: performance anaysis, design, and iterative decoding, IEEE Trans. Inf. Theory, vo. 44, no. 3, pp , May 998. [6] S. Benedetto and G. Montorsi, Unveiing turbo codes: some resuts on parae concatenated coding schemes, IEEE Trans. Inf. Theory, vo. 42, no. 2, pp , Mar [7] D. Divsaar, S. Doinar, and F. Poara, Iterative turbo-decoder anaysis based on density evoution, IEEE J. Se. Areas Commun., vo. 9, no. 5, pp , May 200. [8] M. Tuecher, Design of seriay concatenated codes depending on the bock ength, IEEE Trans. Commun., vo. 52, no. 2, pp , Feb [9] R. M. Tanner, A recursive approach to ow compexity codes, IEEE Trans. Inf. Theory, vo. 27, no. 5, pp , Sept. 98. [20] F. R. Kschischang, B. J. Frey, and H.-A. Loeiger, Factor graphs and the sum-product agorithm, IEEE Trans. Inf. Theory, vo. 2, no. 47, pp , Feb [2] N. Wiberg, Codes and decoding on genera graphs, Ph.D. thesis, Linköping University, Sweden, 996. [22] G. D. Forney, Jr., Codes on graphs: norma reaizations, IEEE Trans. Inf. Theory, vo. 2, no. 47, pp , Feb [23] S. M. Aji and R. J. McEiece, The generaized distributive aw, IEEE Trans. Inf. Theory, vo. 46, no. 2, pp , Feb [24] H.-A. Loeiger, An introduction to factor graphs, IEEE Signa Process. Mag., pp. 28 4, Jan [25] A. Guyader, E. Fabre, C. Guiemot, and M. Robert, Joint sourcechanne turbo decoding of entropy-coded sources, IEEE J. Se. Areas Commun., vo. 9, no. 9, pp , Sept [26] I. Land, P. Hoeher, and S. Gigorević, Computation of symbo-wise mutua information in transmission systems with ogapp decoders and appication to EXIT charts, in Proc Internationa ITG Conference on Source and Channe Coding, pp [27] R. Bauer and J. Hagenauer, On variabe ength codes for iterative source/channe decoding, in Proc. 200 IEEE Data Compression Conference, pp [28] Y. Takishima, M. Wada, and H. Murakami, Reversibe variabe ength codes, IEEE Trans. Commun., vo. 43, no. 2/3/4, pp , Feb./Mar./Apr [29] L. R. Bah, J. Cocke, F. Jeinek, and J. Raviv, Optima decoding of inear codes for minimizing symbo error rate, IEEE Trans. Inf. Theory, pp , Mar [30] F. I. Aajaji, N. C. Phamdo, and T. E. Fuja, Channe codes that expoit the residua redundancy in CELP-encoded speech, IEEE Trans. Speech and Audio Process., vo. 4, no. 5, pp , Sept [3] T. Faze and T. Fuja, Robust transmission of MELP-compressed speech: an iustrative exampe of joint source-channe decoding, IEEE Trans. Commun., vo. 5, no. 6, pp , June [32] T. Fingscheidt and P. Vary, Softbit speech decoding: a new approach to error conceament, IEEE Trans. Speech and Audio Process., vo. 9, no. 3, pp , Mar [33] N. Chayat and S. Shamai, Extension of an entropy property for binary input memoryess symmetric channes, IEEE Trans. Inf. Theory, vo. 35, no. 5, pp , Sept [34] I. Land, S. Huettinger, P. A. Hoeher, and J. B. Huber, Bounds on information combining, IEEE Trans. Inf. Theory, vo. 5, no. 2, pp , Feb [35] I. Land and J. Huber, Information Combining, Foundations and Trends in Communications and Information Theory. NOW Pubishers, [36] A. Chindapo and J. A. Ritchey, Design, anaysis, and performance evauation for BICM-ID with square QAM consteations in Rayeigh fading channes, IEEE J. Se. Areas Commun., vo. 9, no. 5, pp , May 200. [37] X. Li, A. Chindapo, and J. A. Ritcey, Bit-intereaved coded moduation with iterative decoding and 8PSK signaing, IEEE Trans. Commun., vo. 50, no. 8, pp , Aug [38] N. Gresset, J. J. Boutros, and L. Brune, Optima inear precoding for BICM over MIMO channes, in Proc IEEE Int. Sympos. on Information Theory, p. 66. [39] F. Simoens, H. Wymeersch, and M. Moenecaey, Linear precoders for bit-intereaved coded moduation on AWGN channes: anaysis and design criteria, IEEE Trans. Inf. Theory, vo. 54, no., pp , Jan [40] D. Duyck, J. J. Boutros, and M. Moenecaey, Precoding for word error rate minimization of LDPC coded moduation on bock fading channes, IEEE Trans. Wireess Commun., vo., no. 7, pp , Juy 202. [4] F. Schreckenbach, N. Goertz, J. Hagenauer, and G. Bauch, Optimization of symbo mappings for bit-intereaved coded moduation with iterative decoding, IEEE Commun. Lett., vo. 7, no. 2, pp , Dec Jörg Kiewer (S 97-M 99-SM 04) received the Dip.-Ing. (M.Sc.) degree in eectrica engineering from Hamburg University of Technoogy, Hamburg, Germany, in 993 and the Dr.-Ing. degree (Ph.D.) in eectrica engineering from the University of Kie, Germany, in 999, respectivey. From 993 to 998, he was a Research Assistant at the University of Kie, and from 999 to 2004, he was a Senior Researcher and Lecturer with the same institution. In 2004, he visited the University of Southampton, U.K., for one year, and from 2005 unti 2007, he was with the University of Notre Dame, IN, as a Visiting Assistant Professor. In August 2007, he joined New Mexico State University, Las Cruces, NM, as an Assistant Professor. His research interests incude error-correcting codes, network coding, wireess communications, and communication networks. Dr. Kiewer was the recipient of a Leverhume Trust Award and a German Research Foundation Feowship Award in 2003 and 2004, respectivey. He was a Member of the Editoria Board of the EURASIP Journa on Advances in Signa Processing from and is Associate Editor of the IEEE TRANSACTIONS ON COMMUNICATIONS since Danie J. Costeo, Jr. (S 62-M 69-SM 78-F 86- LF 08) received his Ph.D. in Eectrica Engineering from the University of Notre Dame in 969. Since 985, he has been a Professor of Eectrica Engineering at Notre Dame and from 989 to 998 served as Chair of the Department. In 2000, he was named the Leonard Bettex Professor of Eectrica Engineering. Dr. Costeo has been a member of IEEE since 969 and was eected Feow in 985. In 2000, the IEEE Information Theory Society seected him as a recipient of a Third-Miennium Meda, and he was a co-recipient of the 2009 IEEE Donad G. Fink Prize Paper Award and the 202 ComSoc & Information Theory Society Joint Paper Award. Dr. Costeo s research interests are in the area of digita communications, with emphasis on error contro coding and coded moduation. He has numerous technica pubications in his fied, and in 983 he co-authored a textbook entited Error Contro Coding: Fundamentas and Appications, the 2nd edition of which was pubished in 2004.