Maximum Likelihood Decoding of Codes on the Asymmetric Z-channel

Transcription

1 Maximum Likelihood Decoding of Codes on the Asymmetric Z-channel Pål Ellingsen Susanna Spinsante Angela Barbero May 31, 2005 Øyvind Ytrehus Abstract The aim of this paper is to extend some basic concepts related to the Maximum Likelihood decoding of codes on the Z-channel, which is a particular, but very important, example of an asymmetric channel. We study distance properties of linear codes over the Z-channel, in order to define a suitable metric for the implementation of a Maximum Likelihood decoder on the channel. A combinatorial expression for an upper bound on the probability of incorrect Maximum Likelihood decoding is also provided, and comparisons are given to evaluate the tightness of the bound, when a Hamming code, a Turbo code and an LDPC code are decoded on the asymmetric channel. 1 Introduction Errors induced by communications on a noisy channel or storage medium can be reduced to any desired level by proper encoding of the information, as long as the information rate is less than the capacity of the channel. This fact was shown in a landmark paper by Shannon, in 1948 [1]; since that time, much work has been devoted to the problem of devising efficient encoding and decoding methods for error control in a noisy environment. In a block-coded system, the source output u represents a k-bit message, the encoder output c represents an n-symbol codeword, the demodulator output r represents the corresponding binary received n-tuple and the decoder output û represents the k-bit estimate of the encoded message. The decoder must produce an estimate û of the information sequence u based on the received sequence r. Equivalently, since there is a one-to-one correspondence between the information sequence u and the codeword c, the decoder can produce an estimate ĉ of the codeword c. Obviously, û = u if and only if ĉ = c. The strategy adopted for choosing an Susanna Spinsante completed this work under the FASTSEC Marie Curie Training Site contract HPMT- CT , Selmersenteret, Department of Informatics, University of Bergen. 1

2 estimated codeword ĉ for each possible received sequence r is called a decoding rule. If the codeword c was transmitted, a decoding error occurs if and only if ĉ c. Let E be the event that the decoder makes an error. Then, given that r is received, the conditional error probability of the decoder is defined as P (E r) P (ĉ c r). The error probability of the decoder is then given by P (E) = r P (E r)p (r), where P (r) is the probability of the received sequence r. P (r) is independent of the decoding rule adopted, since r is produced prior to decoding. So, an optimum decoding rule that minimizes P (E), must minimize P (E r) = P (ĉ c r) for all r. Because minimizing P (ĉ c r) is equivalent to maximizing P (ĉ = c r), P (E r) is minimized for a given r by choosing ĉ as the codeword c that maximizes P (r c) P (c) P (c r) = ; (1) P (r) that is, ĉ is chosen as the most likely codeword given that r is received. P (r) is independent of the decoding rule, so if all the information sequences and consequently all the codewords are equally likely, P (c) is the same for all c, then, maximizing the last expression is equivalent to maximizing P (r c). For a DMC (Discrete Memoryless Channel), P (r c) = i P (r i c i ), (2) since for a memoryless channel each received symbol depends only on the corresponding transmitted symbol. If the decoder chooses its estimate to maximize P (r c), it is called Maximum Likelihood (ML) decoder. If the codewords are not equally likely, an ML decoder is not necessarily optimum and the conditional probabilities P (r c) must be weighted by the codeword probabilities P (c) to determine the codeword which maximizes P (c r); the corresponding decoding rule turns into the more general MAP (Maximum A Posteriori probability) decoder. For the purpose of this paper, we have to deal with binary outputs, so that many quantities, including the distance metric, can have a simplified definition. An ML decoder is sometimes called a minimum distance decoder as the ML decoding rule chooses the codeword c which minimizes the distance between the received vector r and the codeword c itself. The literature contains a great amount of papers dealing with ML decoding over different channels, like the AWGN (Additive White Gaussian Noise) channel, the BSC or the BEC (Binary Erasure Channel) and the problems related to the probability of incorrect decoding. As in the case of symmetric codes, when the code dimension is not trivial, the ML rule becomes too complex to be applied, also in the case of an asymmetric channel. In this paper, a performance evaluation based on a union bound approach is proposed, regarding the application of ML decoding to the asymmetric channel. In the next section, we describe the relevant distance properties of linear codes over asymmetric channels. These will be useful to develop our final result in section 3. 2

3 2 Distance properties of linear codes over the Z-channel The BSC can be suitably adopted to model binary communication systems where the crossover probabilities are independent and identical. In other communication systems, however, the 1 0 crossover probability can be much larger than the 0 1 one. If we assume that the 0 1 crossover probability is 0, the communication system can be modeled by the Z-channel. According to Kløve [2], the following definition can be given: Definition The Z-channel is the channel with {0, 1} as input and output alphabets, where the crossover 1 0 occurs with positive probability q, whereas the crossover 0 1 never occurs. Kløve [2] also defines an asymmetric distance metric between two binary vectors in the following way: Definition Let x and y be binary vectors and let N(x, y) be the number of positions i where x i = 1 and y i = 0. The asymmetric distance d A (x, y) between x and y is max{n(x, y), N(y, x)}. By definition, a distance measure d must satisfy these requirements for any pair (x, y) of points: 1. d(x, y) d(x, y) = d(y, x). 3. d(x, y) d(x, a) + d(a, y) for any point a. It can easily be checked that d H satisfies all three requirements, so it is indeed a distance function. For a given codeword x we define the minimum asymmetric distance d min A (x) to be the minimum d A (x, y) for all y C. Similarly, the minimum asymmetric distance d min A (C) of a code C is the minimum d A (x, y) over all pairs of codewords x and y. From Definition we can derive some properties of the asymmetric distance: 1. Since d H (x, y) = N(x, y) + N(y, x), d A (x, y) d H (x, y). 2. From Property 1. follows d min (C) dmin (C). A 3. Since max(n(x, y), N(y, x)) N(x,y)+N(y,x), d min 2 A (C) dmin H (C)/2. H If we assume ML-decoding, correct decoding is guaranteed if less than d A (x, y) asymmetric errors occur. Further, for a given codeword an error will occur if the number of bit errors is greater than d min A (x), so the probability of incorrect decoding is at least qdmin A (x)+1. Even if d A determines the probability of incorrect decoding over the Z-channel, this definition of the asymmetric distance cannot be used to formulate an ML-decoding rule for the Z-channel. To this end, we define the following function: 3

4 Definition The non-symmetric distance d NS (x, y) between the two vectors x = (x 1, x 2,..., x n ) and y = (y 1, y 2,..., y n ) is given by d NS (x, y) = i δ(x i, y i ) where 0 if x i = y i δ(x i, y i ) = 1 if x i > y i if x i < y i Referring to the requirements for distance functions cited above, we see that d NS satisfies conditions 1. and 3., but not 2., i.e. the metric is not symmetric, in the sense that d NS (x, y) = d NS (y, x) may not be true. Therefore we say that the function d NS is a pseudo-metric or a non-symmetric distance measure. This reflects the fact that a transmitted word, in general, can only give rise to a subset of the 2 n possible strings of length n. Using d NS, we can formulate the ML-decoding rule for the Z-channel as: For a received word r, decode to the codeword c which minimizes the distance d NS (c, r). (3) 3 On the probability of incorrect MLD on the Z-channel In Definition the definition of a non-symmetric distance d NS was provided, which can be more suitable than the traditional one for the implementation of an ML decoder on the Z-channel. In this section we show how a bound on the probability of incorrect ML decoding on the Z-channel can be derived, by adopting a general approach. 3.1 An upper bound on ML decoding Assume that a vector r of weight w has been received. If the codewords are chosen with equal probability, the ML decoding strategy given in (3) can be carried out by decoding to a codeword c of weight v w, so that v is minimum. Thus, the ML decoder will determine the codeword c that maximizes P (c transmitted r received) = P (c r). From (1) we know that this is equivalent to minimizing P (r c), so the ML decoder selects the codeword c that maximizes P (r c) = q dns(c,r) as required by the decoding rule. For q < 1, this is equivalent to finding the codeword c that contains r and minimizes w H (c). Let E be the event that a decoding error occurs. Then, the probability of incorrect ML decoding (under the assumption that all codewords are equally likely) is given by: P (E) = c 0 C P (c 0 transmitted)p (E c 0 transmitted) (4) = 2 k 1 P (c 0 transmitted)p ( Decode to c i c 0 transmitted) (5) c 0 C 4 i=1

5 By the Union Bound, the probability of the union of the events is less than or equal to the sum of the individual events, so if we sum over the error events in (5): P (E) 2 k 1 P (c 0 transmitted) P (Decode to c i c 0 transmitted) (6) c 0 C i=1 If we assume the weight distribution within every codeword is random, then for words c 0 of equal weight, the probability P (Decode to c i c 0 transmitted) of a pairwise error event with a codeword c i of weight w is the same, so instead of summing over all words in C, we can rather take the sum over the weights. Further, we express the weight distribution of a code of length n by a weight enumerating function A(X) = n d=0 A dx d where the coordinates A d are the number of codewords of weight d in the code. Then, the first sum in (6) can be written: 2 k 1 P (c 0 transmitted) P (Decode to c i c 0 transmitted) c 0 C = n u=1 i=1 = A u 2 k n A u P (E weight u word transmitted) 2k u P (Decode to weight v word weight u word transmitted) (7) u=1 v=1 Consider the two codewords c 0 and c i. Using the definition of N(, ) in Definition 2.0.2, we let α N(c 0, c i ) and β N(c i, c 0 ) for notational convenience. If we permute the indices of the words so that the 1-bits of c 0 come first and let w H (c 0 ) = u, we can illustrate these quantities as follows: c 0 : u n u {}}{{}}{ α {}}{{}}{ c i : For a general code, we can express the probability P (Decode to c i c 0 transmitted) as a function of α, q and n. In (7), P (Decode to c i c 0 transmitted) is 0 if w H (c i ) > u, so we assume w H (c i u). Then, the probability becomes q α if w H (c i ) < u, and supp(c 0 ) supp(c i ) = u α. Now d H (c 0, c i ) = α + β. Also, since w H (c i ) u and β α, α d H (c 0, c i )/2 d min /2 and β = d H (c 0, c i ) α d min α. The codewords c i that can be mistaken for c 0 must then have weight u α + d min α = u + d min 2α. Thus for a given u and α, the number of possible candidates for decoding is ( u u )( n u ) α v u+α A v ( n (8) v) v=u+d min 2α Substituting into (7), we get an upper bound on the probability of decoding error given by: n A u u ( ) ( u u n u ) v u+α P (E) A 2 k v ( α n q u=d min v) α (9) α= d min /2 5 β v=u+d min 2α

6 For linear codes, each codeword c i that can be mistaken for c 0 determines a unique codeword c 0 c i of weight α + β, so we can simplify the expression to: = n A u 2 k u=d min u α= d min /2 ( ) min(n u,α) u α β=min{n u,max(d min α,0)} ( n u ) β A α+β )q α (10) ( n α+β By applying this formula, we obtain a polynomial expression in the variable q, which represents the crossover probability of the asymmetric channel, for the probability of incorrect decoding. 3.2 Valid range of the bound Because of the weakening of the bound in (9) using the Union Bound theorem, the bound grows quickly with higher values of q because of the overlapping of error events, and rises to values above 1 for larger q. In practice, the problematic terms of the polynomial goes to zero very fast as q decreases, so that the upper bound is tight for any reasonable choice of parameters. We can however limit the impact of the terms with large exponents by splitting the polynomial at some point t. The probability of error then becomes: t n P (E) = P (E i errors) P (i errors) + P (E i errors) P (i errors) (11) i=d min A Now, the last term of the sum is the one that makes the sum grow sharply for higher values of q. We can upper limit it by noticing that n P (E i errors) P (i errors) P (E t + 1 or more errors) P (t + 1 or more errors) i=t+1 = = i=t+1 P (t + 1 or more errors) n j ( ) j P (w H = j) q k (1 q) j k k j=t+1 n j=t+1 ( ) n 1 j 2 Thus, for higher values of q a tighter upper bound is t P (E) min P (E i errors) P (i errors) + t i=d min A 3.3 Examples of Application (7, 4) Hamming code n k=t+1 k=t+1 j ( ) j q k (1 q) j k (12) k n j=t+1 ( ) n 1 j 2 n j k=t+1 ( ) j )q k (1 q) j k k As a first example, we test the tightness of this bound on the Hamming (7,4) code, whose weight spectrum is completely known. For a code of this size, we can compute the exact value 6 (13)

7 of (5) as P (E) = 7 16 (q3 + 2q 2 (1 q)) (q4 + 2q 3 (1 q) + 6q 2 (1 q) 2 ) (1 ((1 q)7 + 7q(1 q) q 2 (1 q) 5 )) (14) The upper bound on performance obtained by applying (10) to the probability of incorrect decoding is compared to the results obtained by application of the ML decoding algorithm to this code, for the Z-channel, by considering a crossover probability q ranging from 10 5 to The results are depicted in Fig. 1, where the dashed line represents the bound computation, while the continuous line refers to the exact error probability for ML decoding on the Z-channel. The graphical comparison in Fig. 1 shows a quite good behavior of the bound expression used to estimate the probability of error, when compared to ML decoding of the Hamming (7,4) code on the Z-channel ML-decoding based on (14) Upper bound based on (10) BER e q Figure 1: Bound performance compared to ML decoding of the Hamming code (150, 48) Turbo code The main problem related to the application of the formula for the upper bound on the probability of incorrect ML decoding, is that it is necessary to know the weight distribution of the code in order to estimate the probability of incorrect decoding for different values of the crossover probability of the Z-channel. Actually, in the case of PCCCs (Parallel Concatenated Convolutional Codes) also known as Turbo codes, we know that as the block length and 7

8 corresponding interleaver size increases the weight spectrum of a PCC code begins to approximate a random-like distribution, that is, the distribution that would result if each bit in every codeword were selected randomly from an independent and identically distributed probability distribution. The exact weight spectrum of a Turbo code depends on the particular interleaver chosen; in order to deal with the weight spectrum of a Turbo code in a general way, the concept of a uniform interleaver [3] has been introduced. This notion allows us to calculate the average weight spectrum of a PCC code which is typical of the weight spectrum obtained for Upper bound, random interleaver Turbo code, random interleaver Upper bound, optimized interleaver Turbo code, optimized interleaver FER Figure 2: Comparison with Turbo code q a randomly chosen interleaver. Recently, Rosnes and Ytrehus [4] presented an algorithm for calculating the weight spectrum of Turbo codes. We have used this algorithm to find the lower part of the weight spectrum of a Turbo code in which both constituent codes are generated by (2,1,2) systematic feedback encoders, for two different interleavers. The first interleaver was a randomly chosen interleaver with minimum distance 9, the second interleaver was a Dithered Relative Prime interleaver optimized with respect to minimum distance, resulting in a code with minimum distance 19. The interleavers and codeword weights up to 25 can be found in Appendix A. Thanks to the uniform interleaver approach, it seems reasonable to approximate the remaining part of the weight spectrum by means of a binomial distribution given by A w = ( n 2 ) k, where w w 2 n is the weight of the codeword, k and n are the code parameters. The results obtained comparing the probability of incorrect decoding on the asymmetric channel, computed by means of the bound, to the same quantity obtained by means of an implementation of a Turbo decoder for the 8

9 previously described Turbo codes, are depicted in Fig.2; the crossover probability q ranges from 0.12 to 0.8. We see that the code with the random interleaver falls below the predicted upper bound, indicating that the decoding performance approaches ML-decoding. The code with the optimized interleaver on the other hand, is well above the predicted bound for the values of q we have used for simulation. A possible explanation for this could be that for a code with large minimum distance, pseudocodewords and stopping sets can deteriorate the performance of the decoding process compared to the performance that could be expected judging from the minimum distance alone. See for example [5] and [6]. However, the impact of these factors in the context of asymmetric channels is not well understood and will be addressed in future work by the authors (495, 433) LDPC code The idea of approximating the WEF of the code by means of a binomial distribution has been also applied to the evaluation of the bound on the probability of incorrect decoding of LDPC (Low Density Parity Check) codes. In this case, nothing similar to the uniform interleaver approach has been developed, in order to represent the code s behavior in a general way, as for the Turbo codes. Actually, many different algorithms and schemes have been proposed in order to get a feasible estimation of the minimum distance of LDPC codes, but the problem is still mostly open. In [7] the authors propose a randomized algorithm, called the Approximately Nearest Codewords (ANC) algorithm, to compute the minimum distance of nontrivial LDPC codes. As an example to support the main idea of the paper, the low-part Hamming weight distribution of MacKay s [8] (495,433) code is computed. The code is a regular rate-433/495 LDPC code, which turns out to have a minimum distance of 4 and a multiplicity of at least 60. The other approximated weight values obtained by means of the ANC algorithm are: A 6 = 256, A 8 = 1024, A 10 = 4096, A 12 = 16384, A 14 = 32768, A 16 = 65536, A 18 = We have used these values and the binomial approximation to compute the WEF of the MacKay s (495,433) code, in order to evaluate the bound on the probability of incorrect decoding and to compare it to the simulation of an LDPC decoder, as shown in Fig Tightness of the bound Previously, the best upper bound known to us on ML decoding on the Z-channel was the bound found in [9]. Also based on the Union Bound, an upper limit on ML decoder error is given by P (E) A(γ) 1 (15) where A( ) is the weight enumerator polynomial of the code and γ is the Battacharya noise parameter, which in the case of the Z-channel is q. Both bounds sum over the error events instead of taking the union, but the bound in [9] is further loosened by multiplying each term of the sum by p(r c i )/p(r c 0 ) which is always greater than or equal to 1. Finally, the bound is weakened by summing over all possible vectors r to avoid finding the subset of vectors that can result from c. Thus, we expect the new bound to be tighter than the bound in [9]. We can verify this for the previous examples by computing the two bounds for each code. As can be seen in Fig. 4(a) and 4(b), the new bound is indeed tighter as asserted. 9

10 10 0 Upper bound LDPC code FER Conclusion Figure 3: Comparison with LDPC code, t = 20 In this paper some concepts related to Maximum Likelihood decoding of codes on the Z-channel have been developed, leading to the definition of an upper bound on the probability of incorrect ML decoding, which relies on the approximate computation of the code weight spectrum. The bound has been evaluated with respect to a Hamming code, a Turbo code and an LDPC code over an asymmetric channel for different values of the error transition probability. Simulations regarding the selected Turbo and LDPC codes have shown that the bound only applies to the decoding results below the waterfall, when the decoder starts performing like an ML decoder. The bound has been proved to be tighter than a previously proposed one and this result could be further improved by the development of new algorithms for a more thorough calculation of the weight spectra, most of all in the case of Turbo and LDPC codes. q 10

11 10 0 Old bound New bound FER q (a) Comparison for the code in section Old bound New bound FER q (b) Comparison for the code in section Figure 4: Performance comparison between the old bound described in [9] and the new bound from (10) 11

12 Appendix A A.1 Random interleaver The following interleaver was chosen at random: Using two constituent encoders with generator matrix ( 1, 1+D2 1+D+D2) and the above interleaver, we obtained the following weight spectrum up to 25: IOWEF of Turbo code: Outputweight 9 : 1 Outputweight 10 : 4 Outputweight 11 : 2 Outputweight 12 : 3 Outputweight 13 : 11 Outputweight 14 : 5 Outputweight 15 : 9 Outputweight 16 : 16 Outputweight 17 : 20 Outputweight 18 : 48 Outputweight 19 : 56 Outputweight 20 : 126 Outputweight 21 : 207 Outputweight 22 : 352 Outputweight 23 : 674 Outputweight 24 : 1065 Outputweight 25 : 1775 A.2 Optimized interleaver The following interleaver was found by exhaustive search of all Dithered Relative Prime interleavers with M = 2 and M = 5, see Crozier and Guinand [10] for details Parallell concatenation of the code in A.1 with this interleaver yields a code with the following weight distribution up to output weight 25: IOWEF of Turbo code: Outputweight 19 : 62 Outputweight 20 : 67 Outputweight 21 : 90 Outputweight 22 :

13 Outputweight 23 : 112 Outputweight 24 : 584 Outputweight 25 : 1218 References [1] C. E. Shannon, A mathematical theory of communication, Bell System Technical Journal, vol. 27, pp and , July and October [2] T. Kløve, Error Correcting Codes for the Asymmetric Channel. HiB, N-5020 Bergen, Norway: Department of Informatics, University of Bergen, [3] S. Benedetto and G. Montorsi, Unveiling turbo codes: Some results on parallel concatenated coding, IEEE Trans. Inform. Theory, vol. 42, pp , March [4] E. Rosnes and Ø. Ytrehus, Improved algorithms for the determination of turbo-code weight distributions, Communications, IEEE Transactions on, vol. 53, no. 1, pp , January [5] P. O. Vontobel and R. Koetter, Graph-covers and iterative decoding of finite length codes, in Proc. 3rd International Symposium on Turbo Codes 2003, September [6] P. Vontobel and R. Koetter, Lower bounds on the minimum pseudoweight of linear codes, in Proceedings. International Symposium on Information Theory 2004, July 2004, p. 70. [7] X.-Y. Hu and M. Fossorier, On the computation of the minimum distance of low-density parity-check codes, [Online]. Available: [8] D. J. Mackay, Encyclopedia of sparse graph codes. [Online]. Available: [9] P. Ellingsen, Iterative coding for the asymmetric channel, University of Bergen, Tech. Rep. 295, April [10] S. Crozier and P. Guinand, Distance upper bounds and true minimum distance results for turbo-codes designed with drp interleavers, in Proceedings 3rd International Symposium on Turbo Codes & Related Topics, 2003, pp