4488 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 10, OCTOBER /$ IEEE

Transcription

1 4488 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 10, OCTOBER 2008 List Decoding of Biorthogonal Codes the Hadamard Transform With Linear Complexity Ilya Dumer, Fellow, IEEE, Grigory Kabatiansky, Cédric Tavernier Abstract Let a biorthogonal Reed Muller code RM (1;m) of length n =2 m be used on a memoryless channel with an input alphabet 61 a real-valued output. Given any nonzero received vector y in the Euclidean space n some parameter 2 (0; 1), our goal is to perform list decoding of the code RM (1;m) retrieve all codewords located within the angle arccos from y. For an arbitrarily small, we design an algorithm that outputs this list of codewords with the linear complexity order of n ln 2 bit operations. Without loss of generality, let vector y be also scaled to the Euclidean length p n of the transmitted vectors. Then an equivalent task is to retrieve all coefficients of the Hadamard transform of vector y whose absolute values exceed n. Thus, this decoding algorithm retrieves all n-signicant coefficients of the Hadamard transform with the linear complexity n ln 2 instead of the complexity n ln 2 n of the full Hadamard transform. Index Terms Biorthogonal codes, Hadamard transform, softdecision list decoding. I. INTRODUCTION B IORTHOGONAL (first-order) Reed Muller codes have been extensively used in communications addressed in many papers since the 1960s. These codes have optimal parameters achieve the maximum possible distance for the given length dimension. One renowned decoding algorithm designed by Green [1] performs maximum-likelihood decoding of codes finds the distances from the received vector to all codewords of with complexity of bit operations. Another algorithm designed by Litsyn Shekhovtsov [2] performs bounded distance decoding corrects up to errors with linear complexity. In the area of probabilistic decoding, a major breakthrough has been achieved by Goldreich Levin [3]. Their algorithm takes any received vector outputs the list of codewords of within a decoding radius performing this task with a high probability a low poly-logarithmic complexity for any. Recently, Manuscript received July 2, Current version published September 17, The work of I. Dumer was supported in part by the National Science Foundation under Grants CCF CCF The work of G. Kabatiansky was supported in part by the Russian Foundation for Fundamental Research under Grants The material in this paper was presented in part at the IEEE International Symposium on Information Theory, Nice, France, June I. Dumer is with the Department of Electrical Engineering, University of Calornia, Riverside, CA USA ( dumer@ee.ucr.edu). G. Kabatiansky is with the Institute for Information Transmission Problems, Moscow , Russia with INRIA, Rocquencourt, France ( kaba@iitp.ru). C. Tavernier is with Communications Systems (CS), Le Plessis Robinson, France; ( tavernier.cedric@gmail.com). Communicated by T. Etzion, Associate Editor for Coding Theory. Digital Object Identier /TIT list decoding of codes has been extended to deterministic algorithms. In particular, the algorithm of [4] performs error-free list decoding within the radius with linear complexity for any received vector. This paper advances the results of [4] in two dferent directions. First, we extend list decoding of codes to an arbitrary memoryless semi-continuous channel. Second, the former complexity of [4] will be reduced to. In doing so, we use the following setup. Let a binary vector be mapped onto the Euclidean vector with symbols. Given two binary vectors, consider the Hamming distance, the Euclidean distance, the inner product of their maps. Then Now any binary code is mapped into the cube, which in turn belongs to the Euclidean sphere of radius in the Euclidean space. Thus, any binary code of Hamming distance becomes a spherical code, where two dferent codewords have the inner product at most the angle at least. Below, we consider a memoryless channel with an input alphabet some larger output alphabet (usually, ). We use a code on this channel replace an output in any position with its log-likelihood ratio We then call a received vector. Note that any codeword has a higher posterior probability than another codeword it also has a larger inner product. Note that all codewords become equiprobable for therefore, we will assume that. Without loss of generality, we can multiply by the scalar, where is the squared Euclidean length of vector Then, all vectors belong to the same sphere We now proceed with the biorthogonal codes. Let be any affine Boolean function defined on all points (1) /$ IEEE

2 DUMER et al.: LIST DECODING OF BIORTHOGONAL CODES AND THE HADAMARD TRANSFORM WITH LINEAR COMPLEXITY 4489 As above, we map all outputs onto the vector with symbols Then, codevectors form the biorthogonal code. Given any received vector any parameter our main goal is to retrieve all codewords such that. In equivalent terms, given any, we seek the codewords within the angle from. To define the list, we will construct the corresponding list of affine functions Here each function is recorded as the vector. Now let be decomposed into the orthogonal Hadamard code, whose codevectors are generated by linear functions with, its coset of complementary vectors. Recall also that codewords of code considered as rows form an Hadamard matrix, which satisfies equality, where is the identity matrix. Then the vector represents the Hadamard transform of vector, whereas the vector gives opposite values. Here positions in both vectors are marked as binary -tuples. Now we see that the list gives all positions in which the coefficients of the Hadamard transform have absolute values. Our main result is as follows. Theorem 1: Let the biorthogonal code of length be used on a general memoryless channel. For any any received vector, the list of affine functions can be retrieved error-free with complexity that has linear order of for any fixed as. Finally, we reformulate Theorem 1 as follows. Corollary 2: For any constant, code requires linear complexity order of to output the list of codewords located within the angle from any received vector (softdecision decoding); the Hamming distance from any received vector (hard-decision decoding). Note that linear decoding complexity is achieved in Corollary 2 even the decoding radius is within an arbitrarily small -margin to the code distance. In particular, the new algorithm removes the performance-complexity gap between the maximum-likelihood decoding of the Green machine [1] bounded-distance decoding of the Litsyn Shekhovtsov algorithm [2]. Finally, note for a high-noise case, with of a vanishing order, the output list can include as many as codewords. Each of these is defined by information bits. Thus, in this high-noise case, the newly presented algorithm has complexity that closely approaches the bit size of its output. In Section II, we consider the Johnson bound for real-valued outputs upper-bound the maximum list size for any code. This also yields a tight bound for any biorthogonal code Then in Sections III IV, we proceed with a new soft-decision list decoding algorithm prove Theorem 1. II. JOHNSON BOUND FOR CODES IN Given a code of length any received vector, decoding within the Hamming radius produces the list. Then the classic Johnson bound reads as follows. The Johnson Bound: Let be a code of the minimum Hamming distance. Then for any any positive, such that, the list has size The following lemma shows that the Johnson bound (2) can be applied to any soft-decision output without any alterations. A similar lemma is given in [5] with a dferent proof. Lemma 3: The Johnson bound (2) holds for any code, any output, any positive, such that. Proof: Consider any list of size let. Then we have inequality We can also use the Cauchy Schwartz inequality By construction of our list Since, we combine all three inequalities as follows: which gives the required bound (2). For a code, Lemma 3 gives the following corollary. Corollary 4: Let be a received vector. Then for any, the list of affine functions has size Remark: The term of (2) is replaced with in the last expression. Here we use the fact that a binary code is linear contains an all-one codeword, in which case at most half the code belongs to for any. III. LIST DECODING FOR CODES In this section, we design the Sums Facet algorithm that performs soft-decision list decoding of a code within a threshold. Given any, we represent any -variate linear Boolean function in the form (2)

3 4490 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 10, OCTOBER 2008 Then we define its -prefix as the -variate linear Boolean function Then in each step, we use the function the list of prefixes that pass the threshold test construct that begins with the same coefficients. Given a channel output, the algorithm performs the following steps. In each step, the algorithm receives some list of prefixes Finally, for each, let. For each facet, consider two -dimensional subfacets defined as (7) derives the subsequent list Also, consider the list (3) that includes all prefixes of required functions. For the given, we will sometimes shorten our notation call the above lists, respectively. In the sequel, we show that for all. First, consider the -dimensional Boolean cube any -dimensional facet Then the inner products used in (5) can be recalculated recursively for each prefix as follows: (8) In summary, performs three subroutines: A of (8), B of (5), C of (7) in each step. This is done as follows: Here the prefixes run through for the suffixes are fixed. Also, for. Let denote the restrictions of some vectors to a given facet, be the inner product of these vectors. Let denote the two vectors in that correspond to the above functions. Note that all facets give the same vector,as the latter does not depend on. Now we can use the equality where Input: numbers vector Set Step Input: the list of prefixes numbers for all For each take prefix calculate numbers For each find Pass into Step : pass into Otherwise, pass (9) is the same multiplicative constant for each, vectors satisfy equalities. Since on each. Now we use the inner products define the Sums Facet function The main idea of the algorithm is to calculate for each cidate employ this function as an upper bound for the unknown product. Namely, (4) (5) show that (4) (5) (6) Now consider the Sums Facet function (5) in more detail. Given any vector, define its Facet Span as the subset of vectors obtained by flipping the vector on dferent facets. In contrast to the linear extensions, most vectors are obtained by nonlinear transformations. Then our function can be considered as the maximum inner product over the entire span. This setting is illustrated as follows. Example: In Fig. 1, we consider the code with decoding threshold. As an example, we analyze three cidates, in step. These vectors along with are shown in the first four lines of Fig. 1. Here symbols are marked by a symbols are marked by a. In step, all vectors form four facets of length. Here we also give the values of inner products,, obtained on each facet. The last three lines of Fig. 1 indicate which facets must be flipped within each span,, to obtain

4 DUMER et al.: LIST DECODING OF BIORTHOGONAL CODES AND THE HADAMARD TRANSFORM WITH LINEAR COMPLEXITY 4491 Fig. 1. Decoding of code RM (1; 5). optimal extensions that give the function.we see that only pass the test, since Similarly, we can consider the following steps use recursion (8) for the two remaining cidates. Then it is easy to very from (8) that the cidate (obtained from by taking in step ) gives inner product passes the test, whereas all other extensions of fail. Finally, we compare the Sums Facet algorithm (9) with the classic Green machine. Similarly to algorithm, the Green machine calculates all inner products using all facets. This calculation is similar to Step A of our algorithm. However, the Green machine skips both Steps B C of our algorithm. Instead, each step outputs all possible prefixes their inner products. More generally, the Green machine performs the complete fast Hadamard transform (FHT) of vector using recursion (8). By contrast, the algorithm represents an expurgated version of FHT, which eventually outputs only those coefficients of the HT vector, whose absolute values exceed the given threshold, such coefficients exist. IV. LIST SIZE AND COMPLEXITY OF THE SUMS FACET ALGORITHM Lemma 5: For any received vector, any, any step, the list includes prefixes (3) of all required functions Proof: By definition of,. Thus, according to (6). Then. In the following, we show that all incorrect cidates are filtered out after steps. Lemma 6: The list obtained after all steps equals the required list. Proof: Indeed, any prefix left in the final step is a full function defined on the single facet. Also,. Therefore the proof is completed. Remark: From (8), we also deduce that is a monotonic function on two consecutive prefixes, that strict inequality holds for at least one extension. This implies that, in general, consecutive steps become more restrictive The next lemma shows that -function is fairly restrictive in a sense that each possible list does not accept too many incorrect cidates. Lemma 7: For any received vector, any, any step, the list has size Proof: The bound is obvious, as there exist prefixes. Note that on each facet, the corresponding vectors form an orthogonal code of length. For each prefix each facet, let be the vector such that (10) Let be the corresponding vector of length that equals on each facet. For each, dferent vectors are orthogonal, so are vectors. Then their extensions to full length are also orthogonal. Next, observe from (6) (10) that for any prefix Finally, recall that any prefix satisfies the Sums Facet criterion. Therefore,. Now we apply the generalized Johnson bound of Lemma 3 to the orthogonal code its sublist. Similarly to Corollary 4, this gives the estimate for the latter list. In combinatorial terms, the above proof shows that any two vectors taken from dferent facet spans are pairwise orthogonal. Fig. 1 also illustrates the same fact. Here the three original prefixes,, are orthogonal on each facet, so are all flipped versions of these prefixes. Now we introduce an important parameter

5 4492 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 10, OCTOBER 2008 Corollary 8: Each step leaves (11) for, complete the proof by calculating the entire complexity of as follows:, (11) prefixes in the list. Proof of Theorem 1: Given two positive integers, below we use a procedure that adds -digital binary numbers. We estimate the complexity of in bit operations, with one operation counted for each addition, inversion, comparison of bits. To perform, we first couple two -digital numbers find -digital sums. Then we proceed in the same manner using pairwise additions. Then requires steps has complexity Now we use to estimate the complexity of step of our algorithm (9), which outputs the list using three subroutines A, B, C. For each cidate, our first subroutine A calculates real numbers. Any such calculation uses one addition, possibly, one inversion of two real numbers in (8). To count these real-valued operations in binary bits, we assume that each symbol of the received vector is formed by bits, where is a fixed parameter for any. Thus, step uses -digital inputs -digital outputs. Consequently, step inputs -digital numbers outputs -digital numbers. This calculation requires Remark: Alternatively, we can also assume that any two real numbers are added or compared in one operation. However, the above analysis is more conservative. In particular, it accounts for the fact that all steps employ real numbers of growing bit length as, even the original channel symbols have fixed length. Note also that we obtain a similar bound on we replace -bit operations with one-byte operation but still use an increasing number of -sized bytes as. Above, our outputs were taken from the entire space, or, equivalently, from the sphere normalized. Obviously, the same results also hold for any quantized channel, with the outputs taken from a discrete space of size, where are some positive constants. Finally, note that Corollary 2 is obtained the inner product is replaced with the Euclidean or the Hamming distance. Note also that the last case decoding in the Hamming metric has been recently considered in [6] using a slightly dferent hard-decision algorithm. It is also proven in [6] that for codes, the classic Johnson bound is tight up to the universal constant, given any Hamming radius with. Namely, there exist outputs that are surrounded by or more codewords in a sphere of radius. binary operations for each prefix. The second subroutine B calculates the function of (5) using numbers. Here we need at most bitwise inversions bitwise additions. The procedure gives an number has complexity The third subroutine retrieves the list (7) requires -digital V. CONCLUDING REMARKS In this paper, a new decoding algorithm has been designed for biorthogonal codes that decodes any output into the list of codewords. For any, the algorithm requires linear complexity order of instead of the order of the Hadamard transform. In the decoding process, the algorithm employs an efficient Sums Facet function removes distant codewords, by applying a threshold test to their intermediate prefixes instead of the codewords. In this way, this algorithm extends the classic Green algorithm. In equivalent terms, the algorithm performs the linear order of bit operations to retrieve all -signicant Hadamard coefficients, whose absolute values exceed the threshold. operations per each cidate. Note that for any integers, any step Given at most cidates processed in any step, the entire complexity is Finally, the last step requires only one comparison at most inversions per each prefix. Now we use bound REFERENCES [1] F. J. MacWilliams N. J. A. Sloane, The Theory of Error-Correcting Codes. Amsterdam, The Netherls: North-Holl, [2] S. Litsyn O. Shekhovtsov, Fast decoding algorithm for first order Reed-Muller codes, Probl. Inf. Transm., vol. 19, pp , [3] O. Goldreich L. A. Levin, A hard-core predicate for all one-way functions, in Proc. 21st ACM Symp. Theory of Computing, Seattle, WA, May 1989, pp [4] G. Kabatiansky C. Tavernier, List decoding of Reed-Muller codes of the first order, in Proc. 9th Int. Workshop Algebraic Combinatorial Coding Theory, Kranevo, Bulgaria, Jun. 2004, pp [5] V. I. Levenstein, Universal bounds for codes designs, in Hbook of Coding Theory, V.S. Pless W.C. Huffman, Eds. Amsterdam, The Netherls: Elsevier, 1998, ch. 6, pp [6] I. Dumer, G. Kabatiansky, C. Tavernier, List decoding of the first-order binary Reed-Muller codes, Probl. Inf. Transm., vol. 43, pp , 2007.