6.451 Principles of Digital Communication II Wednesday, May 4, 2005 MIT, Spring 2005 Handout #22. Problem Set 9 Solutions

Transcription

1 6.45 Principles of Digital Communication II Wednesda, Ma 4, 25 MIT, Spring 25 Hand #22 Problem Set 9 Solutions Problem 8.3 (revised) (BCJR (sum-product) decoding of SPC codes) As shown in Problem 6.4 or Figure of Chapter, an (n, n, 2) single-parit-check code has a two-state trellis diagram. Consider the (8, 7, 2) code, and let the received sequence from a discrete-time AWGN channel with input alphabet {±} and noise variance 2 σ = be given b r =(.,.,.7,.8,.,.3,.9,.5). Perform BCJR (sumproduct) decoding of this sequence to determine the APP vector for each smbol Y k. [Hint: the special algorithm given in Section (see Problem 9.5) to implement the sumproduct update rule for zero-sum nodes ma be helpful here.] First, let us draw the normal graph of a minimal trellis realization of the (8, 7, 2) code. This graph is shown abstractl below: Y Y Y 2 Y 3 Y 4 Y 5 Y 6 Y 7 B Σ B Σ 2 Σ 3 Σ 4 Σ 5 Σ 6 Σ 7 B 2 B 3 B 4 B 5 B 6 B 7 (2, ) (3, 2) (3, 2) (3, 2) (3, 2) (3, 2) (3, 2) (2, ) Moreover, it is eas to see that the (3, 2) branch constraint codes are all (3, 2, 2) zero-sum codes, and the (2, ) codes are simple repetition codes that need not actuall be shown. Therefore the trellis realization ma be drawn simpl as follows: Y Y Y 2 Y 3 Y 4 Y 5 Y 6 Y Note that this trellis realization of the (8, 7, 2) code is another ccle-free realization that uses 6 (3, 2, 2) zero-sum constraints, as in the reduced HT realization of Problem 8.; however, the diameter of this realization is 5 (which is as large as it could possibl be). For a binar-input Gaussian-noise channel with inputs {±} and Gaussian conditional probabilit densit p(r ) =(2πσ) /2 exp (r ) 2 /2σ 2, the a posteriori probabilit (APP) of {±} given a received value r is, b Baes rule, p(r ) p( r) = p(r ) + p(r ) e r/σ2 =. e r/σ 2 + e r/σ 2 Here σ 2 = (so SNR = ( db); i.e., the channel is ver nois). The two values p(± r k ) form the intrinsic information message k = {p k,p k } derived from each received smbol r k, k 7. These values are computed from the received vector r =(.,.,.7,.8,.,.3,.9,.5) in the following table. (Note that it would have sufficed to use the unnormalized pair {e r k /σ 2,e r k /σ 2 }.) We also compute the Hadamard transform {p k + p k =,p k p k = I k } of each pair of values for later use.

2 r k p k p k I k r = r = r 2 = r 3 = r 4 = r 5 = r 6 = r 7 = Note that I k =tanh r k /σ 2,so I k r k /σ 2 for small r k. The sign of I k represents a hard decision, whereas the magnitude I k represents the reliabilit of that decision. We then propagate the APP messages through the graph below, using the BCJR (sumproduct) algorithm. Note that in the forward direction α =, α k is the sum-product update of α k and k for 2 k 7, and finall ε 7 = α 7. Similarl, in the backward direction, β 7 = 7, β k is the sum-product update of β k+ and k for 6 k, and finall ε = β.then ε k is the sum-product update of α k and β k+ for k 6. Y Y Y 2 Y 3 Y 4 Y 5 Y 6 Y 7 ε ε ε 2 ε 3 ε 4 ε 5 ε 6 ε α 7 α 2 2 α 3 3 α 4 4 α 5 5 α 6 6 α β β 2 β 3 β 4 β 5 β 6 β 7 For a (3, 2, 2) constraint code C k, the set of past 2-tuples consistent with a value for an incident variable is C k () = {, }, and similarl C k () = {, }. Therefore the sum-product update rule is p( r P ) = p( r Pj )p( r Pj )+ p( r Pj )p( r Pj ); p( r P ) = p( r Pj )p( r Pj )+ p( r Pj )p( r Pj ), where P j and P j are the two pasts upstream of P. Alternativel, following the hint, we ma use the special rule for zero-sum nodes to obtain the Hadamard transform of (p( r P ),p( r P )) simpl b componentwise multiplication of the Hadamard transforms of (p( r Pj ),p( r Pj )) and (p( r Pj ),p( r Pj )). B either method, we obtain the following values for the forward messages α k = {α k,α k } and their Hadamard transforms {α k + α k =,α k α k = A k }, the backward messages βk = {β k,β k } and their Hadamard transforms {,B k }, and the extrinsic information messages ε k = {ε k,ε k } and their Hadamard transforms {,E k }. α k A k α = α k α k α α α α α α

3 β k β k β k B k β 7 = β β β β β β ε k ε k ε k E k ε = β ε ε ε ε ε ε ε 7 = α Notice how the reliabilit of the forward and backward APP messages α k and β k degenerates as more and more intrinsic information messages k are incorporated into them. The APP vectors {p(y k = r),p(y k = r)} for the smbol variables Y k are ultimatel obtained b componentwise multiplication of k and ε k, normalized. We note that for all k, since ε k ε k,we have {p(y 2 k = r),p(y k = r)} { k, k }; i.e., the intrinsic information k dominates. Thus if hard decisions are made on each smbol, the result is the same as if hard decisions had been made smbol-b-smbol based solel on the channel puts r k, and the resulting sequence of hard decisions is not a code sequence in the (8, 7, 2) code. In contrast, suppose that we perform the max-product (equivalent to the min-sum) algorithm with this received sequence. We obtain the following values: α k α k α k α = α α α α α α

4 β k β k β k β 7 = β β β β β β ε k ε k ε k ε = β ε ε ε ε ε ε ε 7 = α In this case, the reliabilit of a forward or backward message is the minimum reliabilit of an intrinsic information that is incorporated into it. Eventuall, this means that the extrinsic information ε dominates the intrinsic information for the least reliable smbol Y, so the original hard decision is corrected in this case. The same correction would be performed b the Viterbi algorithm or b Wagner decoding. The max-product (min-sum) algorithm finds the maximum-likelihood code sequence, whereas the BCJR (sum-product) algorithm computes the APP vector of each bit. A bit decision based on the maximum APP minimizes the bit error probabilit, so the bit error probabilit could be (slightl) lower with the BCJR algorithm. ML sequence detection minimizes the probabilit of decoding to the wrong codeword. The sequence of maximum-app bits from the BCJR algorithm ma not be a codeword, as we have just seen. Compare the complexit of BCJR decoding to Viterbi algorithm decoding (Problem 6.6). The BCJR algorithm is considerabl more complex. For each trellis segment, it has to compute the sum-product update rule three times. The straightforward sum-product update rule involves four multiplications and two additions; the simplified rule requires a single multiplication. B contrast, the VA requires four additions and two comparisons per trellis segment. Computation of the intrinsic APP vector is also more complex, and requires estimation of the noise variance σ 2, which could be a problem. Finall, the logic of the BCJR algorithm is more complex, since it is two-wa rather than one-wa. In short, the VA is significantl simpler than the BCJR algorithm. For this reason, the VA was preferred for trellis decoding for man ears. The BCJR algorithm was resurrected with the advent of turbo codes, where the fact that it produces soft (APP) puts is essential to its role as a part of iterative decoding algorithms. (For this scenario, Wagner decoding is even simpler; see the solution to Problem 6.6.) 4

5 Problem 9. (Iterative decoding on the BEC) (a) Using a graph of the (8, 4, 4) code like that of Figure of Chapter 3 for iterative decoding, decode the received sequence (,,,?,,?,?,?). We ma use the equivalent normal graph of Figure 3(b) of Chapter, which ma be converted to a directed normal graph (encoder) as shown in Figure 5 of Chapter, repeated below. - = H HHHH - = XXX H XX H HHHH HHHH XX XXX 2 - = H H jz- + HHH HHH 9 3 = XX H HHHH Hj- : X + X XXX X 4 - = XX Hz- XXX j + XX X XXXX 5 = 9 z- : + 6 = = 7 = The directed input edges show that (,, 2, 4 ) is an information set for this code. Given (,, 2, 4 )=(,,, ), the sum-product algorithm fills in the erasures b following the directed state edges as shown above. On the first iteration, the first check ields 3 =; on the second iteration, the second and third checks ield 5 = 6 = ; and on the third and last iteration, the last check ields 7 =. Then tr to decode the received sequence (,,,,?,?,?,?). Wh does decoding fail in the latter case? Give both a local answer (based on the graph) and a global answer (based on the code). As shown in the graph below, iterative decoding stalls because at the initial iteration, the first check checks, and each of the last three checks has two or more erased inputs, so none of the checks ields an new smbol values. The smbols ( 4, 5, 6, 7 )and thelast three checks are called a stopping set, because at least two of the smbols are involved in each of the three checks; therefore if all are erased, no progress can be made. 5

6 - = H HHHH - = XX H HHHH XXX H HHHH XX XXX 2 - = H H jz- + 3 = 9 H XX : j- HHHHHHHH + XXX X XX 4 - = XX XXH z- j XXX + X XX 5 = 9 XXz- : + 6 = = 7 = In this case, however, even a global (ML) decoder must fail, because there are two codewords with (,, 2, 3 )=(,,, ), namel and. (b) For the received sequence (,,,,?,?,?, ), show that iterative decoding fails but that global (i.e., ML) decoding succeeds. In this case, there is onl one codeword with (,, 2, 3, 7 )=(,,,, ), namel, so global (ML) decoding will succeed. On the other hand, iterative decoding will fail, because the smbols ( 4, 5, 6 )and the last three checks form a stopping set of this graph: : = X - XX + X XX XXXX 5 = 9 z-: + 6 = = Problem 9.2 (Simulation of LDPC decoding on a BEC) (a) Perform a simulation of iterative decoding of a regular (d λ =3,d ρ =6) LDPC code on a BEC with p =.45 (i.e., on Figure 9 of Chapter 3), and show how decoding gets stuck at the first fixed point (q l r.35,q r l.89). Ab how man iterations does it take to get stuck? We start with q r l = and use the sum-product update relationships developed for this case in Chapter 3: q l r =(.45)(q r l ) 2 ; q r l = ( q l r ) 5. 6

7 We obtain the following sequence: q r l q l r The point is that it takes a ver finite number of iterations to get stuck, ab 5 3 full iterations, depending on how we define stuck. (b) B simulation of iterative decoding, compute the coordinates of the fixed point to six significant digits. B the above simulation, the fixed point to six significant digits is at (q r l,q l r ) = ( , ). 7

8 Problem 9.3 (Iterative decoding threshold) B analsis or simulation, show that the smallest p such that the equation x = ( px 2 ) 5 has a solution in the interval < x< is p = Explain the significance of this calculation for iterative decoding of LDPC codes on a BEC. A simulation such as that in Problem 9.2 will succeed for p =.429 and get stuck for p =.43. Therefore p is somewhere between these two values. The significance of this calculation is that a rate-/2 regular (d λ =3,d ρ = 6) LDPC code with iterative decoding can be used on an BEC with p <p = This is not as good as a random linear rate-/2 code with an ML decoder, which should be able to be used on an BEC with p <.5, but it is prett good. It has now been shown that there exist irregular rate-/2 LDPC codes that can be used with iterative decoding on an BEC with p <.5. Problem 9.4 (Stabilit condition) (a) Show that if the minimum left degree of an irregular LDPC code is 3, then the stabilit condition necessaril holds. If λ(x) = d λ d x d = λ 3 x 2 +,then λ () =, so the stabilit condition pλ ()ρ () < necessaril holds. (b) Argue that such a left degree distribution λ(x) cannot be capacit-approaching, in view of Theorem 3.2. If λ(x) = λ 3 x 2 +, then in the neighborhood of the top right point (, ) of the EXIT chart, the curve q l r = pλ(q r l ) is approximatel quadratic, q l r pλ 3 (q r l ) 2,whereas the curve q r l = ρ( q l r ) is approximatel linear, q r l ρ ()q l r. Therefore there must be a gap of nonnegligible area between these two curves in the neighborhood of the (, ) point. B the area theorem, if the gap between the two curves has a nonnegligible area, then capacit cannot be approached arbitraril closel. Problem 9.5 (Sum-product update rule for zero-sum nodes) (a) Prove that the algorithm of Section implements the sum-product update rule for a zero-sum node, up to scale. [Hint: observe that in the product j (w w ), the terms with positive signs sum to w, whereas the terms with negative signs sum to w.] Following the hint, we have W = W = (w + w ); W = W = (w w ). j j j j In the latter equation, we observe that the terms with positive signs sum to w,whereas the terms with negative signs sum to w, so the second product equals w w. The same terms occur in the former equation, but all with positive signs, so the first product equals w + w,w. Therefore the pair (W ) is the Hadamard transform of (w,w ), so we ma obtain (w,w )from(w,w )btaking theinverse Hadamard transform (up to scale): w = W + W ; w = W W. 8

9 (b) Show that if we interchange w and w in an even number of incoming APP vectors, then the going APP vector {w,w } is unchanged. On the other hand, show that if we interchange w and w in an odd number of incoming APP vectors, then the components w and w of the going APP vector are interchanged. Interchanging w and w changes the sign of W, while leaving W unchanged. Therefore if there an even number of such interchanges, W and W are unchanged, and so therefore are w and w. If there an odd number of such interchanges, then the sign of W is changed, which results in an interchange between w and w. (c) Show that if we replace APP weight vectors {w,w } b log likelihood ratios Λ = ln w /w, then the zero-sum sum-product update rule reduces to the tanh rule ( ) + tanh Λ /2 Λ =ln j, tanh Λ /2 where the hperbolic tangent is defined b tanh x = (e x e x )/(e x + e x ). Since Λ /2=ln(w /w ) /2,wehave (w /w /w ) /2 w tanh Λ ) /2 (w /2= w = =. (w /w /w ) /2 + w W ) /2 +(w w W W tanh Λ /2= W = W. j W (Note that if (w,w ) are normalized so that w = W + w =, then tanh Λ /2= W.) Consequentl j j It follows that ( ) + tanh Λ /2 W + W ln j =ln =ln w j tanh Λ /2 W W w =Λ. (d) Show that the tanh rule ma alternativel be written as tanh Λ /2= tanh Λ /2. As with the expression for tanh Λ /2above,wehave W tanh Λ /2=, W whichwehaveshown abovetobeequal to j tanh Λ /2. j 9