Error Correction in DFT Codes Subject to Low-Level Quantization Noise

Transcription

1 Error Correction in DFT Codes Subject to Low-Level Quantization Noise Georgios Takos, Student Member, IEEE and Christoforos N. Hadjicostis*, Senior Member, IEEE 1 Abstract This paper analyzes the effects of quantization noise on the error correcting capability of a popular class of real-number Bose-Chaudhuri-Hocquenguem (BCH codes known as discrete Fourier transform (DFT codes. Among other uses, DFT codes heve been proposed for joint source-channel coding because their robustness to channel noise is apparently superior to the classical tandem source-channel coding (where the source code is designed without regard to the possibility of channel errors. In order to handle quantization noise, we develop a decoding algorithm that is based on the Peterson-Gorenstein-Zierler (PGZ algorithm and we explore its performance in the presence of quantization noise via analytic means and simulations. As a result of our analysis, we obtain an explicit lower bound on the precision needed to guarantee correct identification of the number of errors; our simulations suggest that this bound can be tight. Finally, we prove that the optimal bit allocation for DFT codes (in terms of minimizing an upper bound used as a threshold in the part of our algorithm that determines the number of errors is the uniform one. Index Terms Real-number codes, Bose-Chaudhuri-Hocquenguem (BCH codes, discrete Fourier transform (DFT codes, quantization noise, Peterson-Gorenstein-Zierler (PGZ algorithm. EDICS NAME: DSP-QUAN (Quantization effects and roundoff analysis This material is based upon work supported in part by the National Science Foundation under NSF Career Award and NSF ITR Award , and in part by the Air Force Office of Scientific Research under URI Award No F URI. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of NSF or AFOSR. G. Takos and C. N. Hadjicostis are with the Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA ( chadjic@uiuc.edu, takos@uiuc.edu.

2 I. INTRODUCTION ERROR-CORRECTING codes in a real-number setting were first introduced in [1] which described block codes based on discrete transforms, including the discrete Fourier transform (DFT code. The author of [1] showed that DFT codes are cyclic and belong to the class of Bose-Chaudhuri-Hocquenguem (BCH codes. Moreover, DFT codes have a minimum Hamming distance that depends on the construction of the code; in particular, for an (N,K DFT code with u N K, the minimum Hamming distance was proven to be u + 1, so that one can correct up to d = u/ errors. During the last few years, discrete transform codes have also been used for joint sourcechannel coding (see, for example, [] and [3]. In these settings, one normally considers two types of errors: channel errors which have large amplitude and quantization errors of smaller amplitude. The authors of [] proposed a joint source-channel encoding algorithm and compared it to the standard tandem source-channel coding (which first designs the source code without any consideration to the channel errors and then obtains the channel code without regard to the nature of the source. They showed through simulations that joint source-channel coding results in about 3dB improvement in signal-to-noise ratio (SNR when compared to tandem source-channel coding. The performance of different decoding algorithms for DFT codes in terms of the ability to accurately localize errors and as a function of the SNR was studied in [3]. In this paper we consider the problem of how quantization noise affects our ability to perform error detection, identification and correction for a DFT code. Essentially, in order to deal with quantization noise we need to develop a decoding algorithm for real BCH codes. We consider a decoding scheme that is a translation of the well-known Peterson-Gorenstein-Zeirler algorithm to a real-number setting and focus on making the first part of the decoding algorithm, which is responsible for determining the number of large errors, robust to quantization noise. As it will become evident from our development, in order to achieve this robustness we need first to understand how finite precision affects the eigenvalues of a matrix whose rank (when no quantization noise is present is exactly equal to the number of the large errors. Assuming no particular distribution on the quantization noise but simply a bound on its maximum magnitude, we are able to obtain upper and lower bounds on each eigenvalue of this associated matrix. Our analysis can be used to handle not only quantization noise (which is bounded in most cases of

3 3 interest, but also any type of noise that takes relatively small values and affects the real-number DFT code-words. Our proposed techniques are reminiscent of the techniques used in [] and [3]; unlike [3], however, we also consider the problem of determining the number of channel errors and, unlike [], we make no assumptions on the type of channel errors. In particular, [3] used the Akaike criterion [4] to determine the number of large errors; whereas [] relied on an empirically computed threshold in order to estimate the number of large errors. In contrast, our analysis provides mathematical insight as to how finite precision affects our ability to determine the number of large errors. Among other things, we use this insight to obtain a lower bound on the required precision (i.e., the required number of fractional bits in the representation of real numbers in fixed point arithmetic in order to guarantee the correct identification of the number of errors. More generally, our approach can be used to obtain an upper bound on the maximum magnitude of the quantization noise, in order to guarantee the correct identification of the number of errors. Moreover, we prove that if we are allowed to assign a different number of bits for the representation of each entry in the code-words, then the optimal bit allocation for DFT codes is the uniform one. This uniform bit allocation is optimal in terms of minimizing the threshold used for determining the number of large errors in our algorithm. This paper is organized as follows. Section II introduces notation and presents the necessary background on DFT codes. Section III develops an algorithm for correcting faults in the presence of quantization noise. In Section IV we obtain the lower bound on the required precision for guaranteed correct identification of the number of errors. Section V proves that the optimal bit allocation for the representation of each codeword is the uniform one. Section VI contains experimental results for this algorithm, an analysis of its performance and comparisons with previous approaches. Finally, in Section VII we conclude our work and give future research directions. II. REVIEW OF DFT CODES In [1] an (N,K DFT code is produced from the N N DFT matrix W N defined as W N (m, n = 1 ( exp i π N N (m 1(n 1,

4 4 x W K X zero Y * W N y (DFT padding (IDFT Fig. 1. DFT encoding scheme. where m and n range from 1 to N. The generator matrix G of a DFT code has as columns any K of the rows of W N, while the parity check matrix H consists of the remaining rows. A codeword y is generated by the data vector x as y = Gx. Clearly, since every row of W N is orthogonal to all other rows, the product Hy (referred to as the syndrome will give zero. If, however, some entries of the code-vector are corrupted by errors, the syndrome will no longer be zero. In fact, the syndrome allows us to not only detect errors but also to correct up to a certain number of errors. In this paper we consider a slightly different DFT code defined by the generator matrix [], [3]: G = WNΣW K, with K odd and N even and Σ is an N K zero-padding matrix (see Fig. 1. In particular, x ǫ R K is the source word and y ǫ R N the code-vector, with y = Gx (N > K. In this case, Σ N K, m = is the N K zero-padding matrix whose only nonzero elements are: Σ(m, m = 1,,..., K+1 and Σ(N m, K m = N K, m = 0, 1,..., K 3. If we let, X and Y be the (K-point and N-point respectively DFTs of x and y, Σ then adds N K consecutive zeros to X so that Hermitian symmetry is preserved (see Chapter 4.5 in [5]. Decoding algorithms when the DFT of the codeword has (N K consecutive zeros were studied in [6]. As an example of a DFT code, let N = 6 and K = 3 so that W K = i i i i

5 5 6/ / Σ = /3 and G = In order to obtain the (N K N parity check matrix H for this code we take the N K columns of W N K+3 that correspond to the zeros of Σ, i.e., the columns with indexes,..., (N K 1 to form the conjugate transpose of H. In particular, for the (6,3 DFT code above H = i i i i i i i i With this choice of parity check matrix the syndrome, defined as s Hy is equal to the zero vector, provided that no errors corrupt the code-vector y (since HG = 0.

6 6 If an error vector e corrupts the code-vector, so that we receive y + e, the syndrome becomes s = H(y + e = He. III. DECODING ALGORITHM The original Peterson-Gorenstein-Zeirler (PGZ decoder was developed for BCH codes over a finite field [5]. In that setting the decoding is a three-step algorithm. First, it determines the number of errors by constructing a matrix whose rank is equal to the number of errors. Then, it determines the locations of the errors through an error locator polynomial whose roots are the inverses of the error locations (the roots are determined by substituting all values that are possible roots into the polynomial, choosing the roots that give zero, and therefore obtaining the error locations. Finally, with the error locations known, the magnitudes of the errors are determined by solving a set of linear equations. Our algorithm follows the same structure, but is modified to function in a real number setting and in the presence of quantization (or finite precision noise. In [7] a similar algorithm was presented for real number DFT codes, but no quantization noise was considered. When finite precision is not an issue, the real number PGZ decoder can find the coefficients of the error locator polynomial Λ = x d + Λ 1 x d Λ d, where d = (N K 1/, by solving the following set of equations s m Λ d + s m+1 Λ d s m+d 1 Λ 1 = s m+d, (1 for m = 1,,..., (d + 1. This is equivalent to solving the matrix equation [5]: s d+1 s d+ s d Λ d s d+1 s d s d+1 s d 1 Λ d 1 s d. =, } s s 3 {{ s d+1 } Λ 1 s d+ Q where s H(y+e [s 1 s... s (d+1 ]. If less than d errors occurred (i.e., if the error vector e has less than d nonzero entries, matrix Q is singular (Q has rank equal to the number of errors but the decoder can solve for a length d 1 polynomial by eliminating the last row and column

7 7 of Q; this process can continue until we have a nonsingular Q. With the (low-level quantization noise that we consider in this paper, the received vector is y +e+q, where q is the quantization noise vector that corrupts the code-vector. The syndrome is affected by both large errors and quantization noise so that it is given by H(e+q. Our analysis does not assume a distribution on the quantization noise because we are primarily interested in bounding the performance of our decoding algorithm. Thus, our analysis considers as given the maximum possible absolute value of the quantization noise, namely q max = q. In the presence of quantization noise, matrix Q needs to be replaced by s r(d+1 s r(d+ s r(d s r s r(d+1 s r(d 1 Q r =.,..... s r s r3 s r(d+1 where s rl, l = 1,,..., (d + 1, are the entries of a new syndrome s r [s r1 s r... s r(d+1 ] T = H(e + q. The main difference is that the above syndrome comes from two error vectors: the channel error vector e which contains only the (presumably large d or less errors and the vector q which consists only of ( presumably smaller quantization noise components. Note that one can also write Q r as the sum of two components, i.e., Q r = Q+Q q, where Q q is the d d matrix produced in the same fashion as Q but using the components of the vector s q [s q1 s q... s q(d+1 ] T = Hq. Note that s q is not available to the decoder (only s r is. Step 1: Determining the Number of Large Errors With quantization noise, matrix Q r is unlikely to be singular, so its rank is not necessarily equal to the number of errors (note that when quantization noise is absent the rank of Q is equal to the number of errors. In the sequel, we study the nature of the eigenvalues of Q r in an attempt to determine the number of errors. Note that we can write the entries of Q r as s rl = 1 N N m=1 ( exp iπ( K 1 + l(m 1/N (e m + q m,

8 8 for l = 1,,..., (d + 1, where e m and q m are the m-th entries of e and q respectively. We see that s r(d+1 ǫ R (recall that d = (N K 1/ and s r(d+1 n = s r(d+1+n, for n = d, d + 1,..., d (recall that N is even. This means that Q r is a Hermitian matrix. ( If we let V be the N d matrix with elements V(m, n = exp iπ(m 1(n 1/N, and W r be the N N diagonal matrix with elements W r (m, m = 1 N (e m + q m ( 1 (m 1, we can see that we can write [7] Q r = V W r V. Clearly, the decomposition of the error vector in its two components e and q, results in a similar decomposition for the W r matrix. In particular, we can decompose Q r as a sum of Q and Q q, and if we write W r = W +W q (where W depends only on elements of e, while W q depends only on elements of q, we see that Q = V WV and Q q = V W q V. Note that V is a d N matrix which can be written in block form as V = [V 1 V V m], ( where m = N d, V j, j = 1,,, m 1 are d d matrices and V m is a d (N d(m 1 matrix. Note that, as long as we rearrange the entries on the diagonals of W and W q appropriately V 1 does not necessarily have to be formed from the first d columns of V, but from any d columns of V. All the columns of Vj, j = 1,,, m, however, have to be distinct. For simplicity, in the sequel we assume N to be a multiple of d. If N is not a multiple of d, in particular if N = md n, for some nonnegative integer n (strictly less than d, then we can add n zero columns to V and get similar results with slightly more complicated analysis. If we apply a matching block decomposition to matrix W q, we can express it in terms of d d matrices as W q = W q W q W qm Thus, we we can write Q q as Q q = VjW qj V j.

9 9 A similar decomposition can be obtained for Q but, since W has rank up to d (there can only be at most d large errors, we can erase the N d zero columns and rows, (whose exact place we do not know only their existence to get a reduced d d matrix W e. By erasing the corresponding columns and rows of matrices V and V, respectively, we can write Q = V ew e V e, where Ve and V e are the reduced V and V matrices, respectively, of size d d. If the actual number of large errors is smaller than d we can pack any columns/rows for Ve and V e and set the corresponding values at the diagonal of W e to zero. Therefore, we can write Q r as a sum of d d matrices Q r = VeW e V e + VjW qj V j. (3 We denote the (real eigenvalues of a d d Hermitian matrix Q by λ 1 (Q λ (Q... λ d (Q. We are now ready to prove the following theorem for the eigenvalues of Q r. Theorem 1: Let the Hermitian matrix Q r be written as in (3. Then, for the kth smallest eigenvalue of Q r, namely λ k (Q r, we have the following upper and lower bounds: l k λ k (Q r u k, where l k = λ k (W e λ 1 (VeV e + λ 1 (W qj λ 1 (VjV j, u k = λ k (W e λ d (VeV e + λ d (W qj λ d (VjV j. Proof: For the proof we make use of the following theorems from [10]. Theorem : Let Q and Q q be n n Hermitian matrices. Let λ i (A denote the ith smallest eigenvalue of an n n Hermitian matrix (i.e., λ 1 (A λ (A... λ n (A. Then, for k = {1,,..., n}, λ k (Q + λ 1 (Q q λ k (Q + Q q λ k (Q + λ n (Q q. Theorem 3: Let W 0 and V 0 be n n matrices with W 0 Hermitian and V 0 nonsingular and let the eigenvalues of W 0 and V 0 V 0 be arranged in increasing order as in the previous theorem.

10 10 Then, for k = {1,,..., n}, λ k (W 0 λ 1 (V0V 0 λ k (V0W 0 V 0 λ k (W 0 λ n (V0V 0. By applying Theorem above to (3, we get λ k (VeW e V e + λ 1 ( VjW qj V j λ k (Q r λ k (VeW e V e + λ d ( VjW qj V j. Moreover, by repeated application of Theorem, we get λ k (VeW e V e + λ 1 (VjW qj V j λ k (Q r λ k (VeW e V e + Finally, by applying Theorem 3, we get λ d (VjW qj V j. λ k (W e λ 1 (VeV e + λ 1 (W qj λ 1 (VjV j λ k (Q r λ k (W e λ d (V ev e + λ d (W qj λ d (VjV j, (4 which completes our proof. Since [5] has shown that Q has rank equal to the number of errors, say d, the d nonzero eigenvalues of Q correspond to the d nonzero values of W e. If the quantization noise is small enough, then we expect the eigenvalues of the full-rank matrix Q r to be close to the eigenvalues of Q and hope that d of the eigenvalues of Q r will be significantly larger in magnitude than the remaining d d eigenvalues. The latter would correspond to the zero eigenvalues of Q, while the former would correspond to the d nonzero eigenvalues of Q. Therefore, we can associate the eigenvalues of Q r that have large magnitude with the nonzero entries in W e (i.e., the errors. We formalize this intuition in the approach that we describe next for determining the number of large errors. From (4 we can see that the eigenvalues of Q r that do not correspond to errors, i.e., the ones that correspond to an eigenvalue of W e in (4 that is zero, can be bounded by

11 11 λ 1 (W qj λ 1 (VjV j λ(q r λ d (W qj λ d (VjV j. (5 This equation (5 leads to the following theorem, based on which we can determine the number of errors in the system. Theorem 4: The magnitude of all eigenvalues of Q r that do not correspond to errors (as explained earlier, satisfy λ(q r q max m λ d (VjV j, (6 N where q max is the maximum absolute value the quantization noise can get and N is the size of the codeword. Proof: We know that if l λ u, then λ max ( l, u. Therefore, from equation (5, since λ d (Vj V j are positive, an upper bound on the magnitude of the eigenvalues of Q r that do not correspond to large errors is λ(q r max λ l(w qj λ d (VjV j. 1 l d Since W qj is a diagonal matrix with elements ( 1 k q k / N for some k = 1,..., N, and q max is the maximum value of the quantization noise, this results in the overall upper bound on the particular eigenvalues as given in (6. The eigenvalues λ d (Vj V j were the subject of study in [11], from where we know that if V j has size d d (and is of the form considered in this paper then d λ d (V jv j N. This inequality suggests that the best possible bound for λ(q r in (6 can be qmax N md = q max N. We now show that this minimal upper bound is indeed attainable. As we mentioned earlier, when we decompose V into square submatrices V j, as shown in (, any reordering of the columns of V before the formation of the submatrices is allowed (as long as we appropriately reorder the elements on the diagonal matrix W. We now prove that there is a grouping of columns such that λ d (V j V j = d, for all j = 1,..., m. Combining the latter with the fact that d λ d (V j V j, we see that there is a grouping of columns that makes m λ d(v j V j =

12 1 md (= N, which by [11] is the minimum value this sum can take. The (x, y element of V is ( V (x, y = exp iπ(x 1(y 1/N. If we choose the l-th column of V j to be the ( (l 1m + j -th column of V then, ( Vj(x, y = exp iπ(x 1{(y 1m + j}/n, ( V j (y, z = exp iπ(z 1{(y 1m + j}/n. Therefore, the (x, z element of V j V j is V jv j (x, z = = d Vj(x, yv j (y, z y=1 d y=1 = exp d y=1 ( exp iπ(x z{(y 1m + j}/n ( iπ(x zj/n ( exp iπ(x z(y 1m/N But N = md, and if we denote z d = exp(iπ/d (the d-th root of unity, we have ( VjV j (x, z = exp iπ(x zj/n d y=1 z (y 1(x z d. We know that the sum of all powers of the roots of unity zd i, i = 1,,..., d is zero, and that ( i, i = 1,,..., d is just a permutation of the roots of unity when x z, as long as z (x z d (x z and d are relative prime numbers. Therefore, 0 if x z, VjV j (x, z = d if x = z. ( If (x z and d have a common divisor, namely f, then z (x z d (7 i, i = 1,,..., d are the d/f roots of unity, with each one appearing f times in the sum. Thus, (7 still holds for any choice

13 13 of (x z and d. This means that by choosing the l-th column of Vj to be the ( (l 1m+j -th column of V then, Vj V j = di d, where I d is the identity matrix of size d, and λ d (Vj V j = d for all j in {1,,..., m}. We can now prove the following theorem. Theorem 5: The magnitude of all eigenvalues of Q r that do not correspond to errors (as explained earlier, satisfy λ(q r q max N, (8 where q max is the maximum absolute value the quantization noise can get and N is the size of the code-vector. Proof: From (6 we know that λ(q r qmax m N λ d(vj V j. But we have also seen that λ d (Vj V j are lower bounded by d and that there exists a choice of V j so that all these eigenvalues are equal to d. Therefore, since N = md, an improved upper bound for the magnitude of the eigenvalues of Q r that do not correspond to errors is given by (8. A popular technique for determining the number of large errors in a DFT code is to find the eigenvalues of Q r and use an (empirical or otherwise obtained threshold to determine how many of them correspond to errors. For example, in [] an empirical threshold was used, while the authors of [3] employed the well-known Akaike Information Criterion (AIC [4]. In particular, the AIC looks at the ordered eigenvalues of S r S r, where s r1 s r s r(d+1 s r s r3 s r(d+ S r = s r(d+1 s r(d+ s r(d+1 ; if these ordered eigenvalues are denoted by λ 1, λ,..., λ d+1, the AIC decides that the number of large errors is ( arg min (d + 1 kln( AM k + k ( (d + 1 k, 1 k d GM k where AM k and GM k are the arithmetic and geometric means of λ k+1,..., λ d+1. Note that the matrix S r is similar to the matrix Q r we have considered in our analysis. In Section VI, we compare the performance our scheme and the AIC through simulations.

14 14 As a result of our analysis above, one way to compute a threshold for the eigenvalues of Q r is the following: compute the eigenvalues of Q r, take their absolute values and sort them. Compare all these values with the threshold given in (8 and choose the number of large errors as the number of eigenvalues of Q r with magnitude greater than the threshold. This is a nonconservative threshold since we know that all eigenvalues that do not correspond to large errors will be below the threshold (therefore, we can never detect more than d errors, where d is the actual number of errors. In Section IV, we discuss conditions under which we can choose the threshold to guarantee that we correctly determine the number of large errors. Step : Determining the Error Locations Once the number of errors is known, the second part of the PGZ algorithm focuses on determining the error locations. In the finite field case, this is accomplished by substituting all possible values into the error locator polynomial and choosing those that give a result of zero. In the real-number setup that we consider here, if there is no quantization noise, then ( Λ = x d + Λ 1 x d Λ d will be zero for exactly d different x k = exp iπk/n, k = 1,,..., N (for the k s that correspond to the error locations. The presence of quantization noise will prevent the error locator polynomial from taking on a value that is exactly zero. Instead, we choose the d x k s that result in the d smallest magnitudes when substituted into the error locator polynomial (d is the number of errors that were determined to have occurred in the first step of the algorithm. These d x k s determine, of course, the error locations. Step 3: Determining the Error Values Once our error locations are known, we proceed to the final step and calculate the error values. This can be formulated as an estimation problem because we can write the syndrome vector as s r = Te 0 + Hq, where T is a d d Vandermonde matrix (a reduced version of the parity check matrix H whose columns correspond to the error locations and e 0 is a reduced version of e where all zeros have been eliminated. If we have knowledge of the distribution of the quantization noise (but no prior knowledge on the statistics of the large errors, then we can use least square estimation [1]. For instance,

15 15 a very common model for the quantization noise has each component drawn from a uniform distribution over ( B, + B ], where B is the number of fractional bits in the representation of real numbers in our scenario [9]. Under this model, the noise added to the codeword y is an N-dimensional random vector q with zero mean and covariance matrix K q = B 1 I N σ I N. Taking all of the above into account, the least squares estimate for the vector e 0 of the error magnitudes is ê 0,LS (s r = (T Q 1 T 1 T Q 1 s r with Q = HK r H and K r = σ I N. If we have more information on the statistics of the large errors, such as σe, i.e., the a priori variance of the errors, we can use linear minimum mean square error (LMMSE estimation [1]. In this case, the corresponding estimator is ê 0,LMMSE (s r = σ et (σ ett + K r 1 s r. IV. CONDITIONS FOR CORRECTLY DETERMINING THE NUMBER OF ERRORS In the absence of quantization noise, the original PGZ algorithm presented in [8] would correctly identify the number of channel errors that occurred in the system as the rank of matrix Q. When quantization noise is considered, then Q becomes Q r, which is generally a full-rank matrix. For small quantization noise, however, we expect that the eigenvalues of Q r which correspond to quantization noise (and that were the zero eigenvalues of Q will still be small, so that their magnitude will be smaller than the magnitude of the other eigenvalues (that correspond to large errors. In this section we find a lower bound on the number of fractional bits (in the representation of real numbers needed to ensure that the magnitude of the eigenvalues corresponding to quantization noise will always be smaller than that of the eigenvalues that correspond to large errors. Under these conditions, we can find a threshold that separates the two groups of eigenvalues, so that the modified PGZ algorithm is guaranteed to determine the correct number of errors. We have seen in the previous section that all the eigenvalues that correspond to quantization

16 16 noise will be smaller than the threshold in (8. From equation (4 and the fact that if l λ u then λ min( l, u, we see that, in our scenario, u = λ k (W e λ d (VeV e + λ d (W qj λ d (VjV j and l = λ k (W e λ 1 (VeV e + λ 1 (W qj λ d (VjV j. Clearly, we can lower bound u and l as u λ k (W e λ d (VeV e λ d (W qj λ d (VjV j and l λ k (W e λ 1 (VeV e λ 1 (W qj λ 1 (VjV j. Note that in this case λ k (W e will be nonzero, since these are the eigenvalues that correspond to large errors. Using the fact that V ev e and V j V j, j = {1,,..., m}, are positive definite matrices, so that λ d (V ev e λ 1 (V ev e 0, then min( l, u λ k (W e λ 1 (VeV e λ d (W qj λ d (VjV j. We have already seen (in the proof of Theorem 5 that λ d (W qj q max / N, and that λ d (VjV j = N (for a certain choice of V j. Therefore, by assuming that ε min is the minimum value a large error can take, we see that λ k (W e ε min / N,

17 17 for all 1 k d. Combining this with the above, we conclude that the eigenvalues that correspond to large errors have magnitude at least as large as ε min N λ 1 (V ev e q max N. Clearly if ε min N λ 1 (V ev e q max N qmax N, which is the bound for all eigenvalues that correspond to quantization noise in (8, then we can always separate the eigenvalues that correspond to large errors from the ones that correspond to quantization noise. We can then solve for q max to get q max λ 1(VeV e ε min. N Note that all the parameters involved in the last inequality are known, except for the matrix V e which can take different values depending on the number and the locations of the large errors. Therefore, we can offline compute the smallest λ 1 (V ev e, for all possible V e, namely λ min, so that q max λ min N ε min. In the computation of λ min, we need to consider all different V e with 1,,..., d nonzero columns, i.e., there are ( N 1 + ( N ( N d possibilities for V e V e. Note that in reality one only has to check ( N d possible matrices Ve, because it can be proved that the minimum eigenvalue of V e V e is larger than the minimum eigenvalue of V ev e when V e consists of a subset of columns from V e [10]. If we have B fractional bits available for the representation of real numbers, as we have seen in Section III, then q max = B. We now only need to find a sufficiently large B so that B λ min N ε min, or, equivalently,

18 18 ( N B log. (9 λ min ε min V. OPTIMAL BIT ALLOCATION FOR DFT CODES In obtaining an upper bound on the eigenvalues corresponding to quantization noise we used equation (5. If we assume that our quantization noise vector has entries uniform in ( B, + B ], where B is the number of fractional bits in the representation of real numbers in our scenario, and that we assign a different number of bits in each entry of the codevector y, then the magnitude of the nonzero entries of W qj (for all j will be upper bounded by W qj (k, k 1 Bjk, (10 N where B jk is the number of fractional bits available for the representation of the entry of y that corresponds to the k-th entry of W qj. Using equation (10 and the fact that λ d (W qj is bounded by ( 1 Bjk max 1 k d N we can rewrite the second part of (5 as λ(q r = 1 (mink Bjk N (mink 1 Bjk λ d (V N jv j. (11 Our goal in this section is to determine which bit allocation will minimize the upper bound in (11. In other words, we need to find B jk, j = 1,,..., m and k = 1,,..., d such that the right part of (11 is minimized 1 given that we have B bits available in total for all N entries of y. The latter is equivalent to d B jk = B. k=1 We now prove that for DFT codes the optimal bit allocation is the uniform one, i.e., B jk = B N, for all j and k. 1 As we will see, the bound in (11 is fairly tight, so minimizing the bound is also expected to minimize the actual value.

19 19 Our minimization problem is formulated as min m 1 N (min k B jk λ d (V j V j s.t. m d k=1 B jk = B Note that for each W qj we are only interested in its largest (in magnitude eigenvalue, or, equivalently, in its entry with the least available fractional bits. Therefore, it will be a waste of bits if we assign more bits to the remaining entries of W qj. Hence, we assume that all the entries in each W qj are assigned the same number of bits, namely B j j = {1,,..., m}. Our minimization problem becomes min s.t. I = m α j B j m B j = B d where α j 1 N λ d (V j V j. Note that there are two parameters that affect the value of the quantity in our minimization problem: a the bits B j assigned in each location and, b the decomposition of V into V j (different decompositions will give different α j s and different overall upper bounds on the small eigenvalues. First, we deal with the optimal bit allocation problem (assuming a specific but not necessarily known decomposition into α j s and, then, we incorporate some of our earlier results to solve the optimal grouping problem. We introduce the Lagrange multiplier L α j Bj ( m λ B j B d and calculate L Bj = α j ln λ. B j L ( λ = 0 B j = log B. (1 j α j ln Since m B j = B d, we have (( λ m 1 log ln m α = B j d.

20 0 If we denote GM the geometric mean of the α j s then (( λ m 1 log ln GM m = B d, or (N = md Hence, and (1 becomes ( λ log = B GM ln N. λ ln = B N GM, B j = log ( B N GM α j = B N log (GM/α j. (13 Notice that the optimal bit allocation for a specific decomposition V j is the uniform one plus a correction term that depends on the decomposition. We know try to solve the second part of the problem, which is to find the optimal decomposition V j in order to minimize I = m α j B j. We substitute the values B j found in (13 and we see that Bj = B/N GM α j or I = B/N GM = B/N mgm. Therefore, the optimal grouping will be the one that minimizes the geometric mean GM of the α j s. We have already seen in Section III that α j d/ N, hence, GM d/ N. However, we have proved that there exists a grouping for which all α j s are equal to d/ N so that GM = d/ N. Therefore, the minimum value for I is obtained when GM = d/ N which means I = md/ N B/N = N B/N (recall N = md. Note that B/N = q max and that, eventually, I is the threshold we used in our analysis earlier. Finally, since GM = α j, then B j = B N, for all j. We have, therefore, proved that the optimal bit allocation for minimizing the upper bound used in the first part of our decoding algorithm is the uniform one.

21 1 VI. EXPERIMENTAL RESULTS AND ANALYSIS We now evaluate the performance of our decoding algorithm in the presence of quantization noise. Note that our problem consists of two subproblems: (i determining the number of channel errors and (ii determining their effects (so as to correct them. For this reason, we run simulations in MATLAB for a (0,9 DFT code. The entries of the input vector x were selected independently from a uniform probability distribution over the range [ 1, 1]. Locations and values of errors are determined randomly as well, with error values being uniformly distributed over [ 1, q max ] [q max, 1], where q max is taken to be B / (if the upper bound on the magnitude of the large noise is larger than 1 things are actually easier. Since our encoding allows us to correct up to 5 errors we add one, two, three, four or five random errors at the encoded vector y. With the above choices, we simulate the encoding and decoding procedure with quantization noise generated due to different finite precisions ranging from 1 to 16 fractional bits. The first part of our algorithm identifies the number of large errors. In Figures and 3 we plot the probability of correctly identifying the number of large errors as a function of precision (measured in the number of bits available. Figure provides plots for one, two and three errors and Figure 3 provides plots for four and five errors. For comparison purposes we also plot the probability of correctly determining the number of errors using the AIC (see our discussion in Section III. We run 10,000 simulations for each precision level. We see that our proposed scheme works better that the AIC for most cases, except for one large error, where the AIC seems to work perfectly well. In Figure 4 we plot the probability of correctly identifying both the number of errors and the error locations for one, two, four and five errors. We run 10,000 simulations for each point. For the same example, we also compute the lower bound on the precision needed to achieve guaranteed correct identification of the number of errors as given in (9 for different levels of error (0.1 < ε min < We then run 10,000 simulations (for one, two, three, four and five errors and for each different large error level and find the actual precision where 100% correct identification of the number of errors is achieved. In Figure 5 we compare these plots for one, two and three large errors and in Figure 6 for four and five large errors. Note that in Fig. 5 negative fractional bits appear, which correspond to the last bits of the integer part of the real number representation. Also note that if we know the exact number of large errors that have

22 1 0.9 probability of correct identification of number of errors large error proposed scheme large errors proposed scheme 3 large errors proposed scheme large error AIC large errors AIC 3 large error AIC Fractional bits available Fig.. Effect of finite precision on ability to determine the number of errors for one, two and three large errors ,4 large errors proposed scheme 5 large errors proposed scheme 4 large errors AIC 5 large errors AIC probability of correctly identifying the number of errors Fractional bits available Fig. 3. Effect of finite precision on ability to determine the number of errors for four and five large errors. corrupted our code-vector, then we can compute more accurate values for λ min. For example, if we know that 3 large errors have occurred, then we calculate all the ( N 3 possible Ve matrices, the corresponding minimum eigenvalues of V ev e and choose as λ min the minimum of all. Notice that our bound is relatively tight (since it is a worst-case scenario, the comparison should really be made against the case when there are exactly 5 errors present. Also notice that the curves shown in the figure are actually conservative estimates (they only actually shift upwards if more than 10,000 simulations are performed. VII. CONCLUSIONS - FUTURE WORK In this paper we have developed a decoding algorithm that allows us to detect, localize and correct channel errors in the presence of quantization noise for a class of real number BCH codes. In addition, we have obtained a lower bound on the precision needed to guarantee correct

23 large error large errors 3 large errors 4 large errors 5 large errors probability of correct localization fractional bits available Fig. 4. Effect of finite precision on ability to determine the number and location of errors Theoretical Bound 1 large error Theoretical Bound large errors Theoretical Bound 3 large errors Actual Bound 1 large error Actual Bound large errors Actual Bound 3 large errors 8 fractional bits log (ε 10 min Fig. 5. Precision needed for guaranteed correct identification of number of errors for one, two and three large errors. determination of the number of channel errors and proved that the optimal bit allocation for DFT codes is the uniform one. Note that our analysis can also be applied to any real-number Bose- Chaudhuri-Hocquengen (BCH code because of the Vandermonde structure of our parity-check matrix. The well-known Akaike Information Criterion (AIC has been used in the signal processing society for identifying the number of signals in array processing, where multiple observations (equivalent to the syndrome in our setup are available. It would be interesting to extend the first part of our decoding algorithm to the case when more than one syndrome is available at the decoder. This setup, of course, is not applicable in coding theory, but any results can be directly applied to the array processing problem. Moreover, in our analysis we assumed an upper bound for quantization noise. One possible

24 4 4 Theoretical Bound 4 large errors Theoretical Bound 5 large errors Actual Bound 4 large errors Actual Bound 5 large errors 0 18 fractional bits log (ε 10 min Fig. 6. Precision needed for guaranteed correct identification of number of errors for four and five large errors. future research direction is to investigate how our decoding algorithm would be affected by assuming a distribution on the quantization noise. Finally, we are looking into developing real BCH encoding algorithms that take advantage of the analysis in this paper and have applications to image processing. REFERENCES [1] T. G. Marshall Jr., Coding of real-number sequences for error-correction: a digital signal processing problem, IEEE Journal on Selected Areas in Communications, vol. SAC-, no., pp , March [] A. Gabay, P. Duhamel and O. Rioul, Spectral interpolation coder for impulse noise cancellation over a binary symmetric channel, European Signal Processing Conference 000, Tampere, Finland, Sept [3] G. Rath and C. Guillemot, Subspace algorithms for error localization with quantized DFT codes, IEEE Transactions on Communications, vol. 5, no. 1, pp , December 004. [4] S. M. Kay, Modern Spectral Estimation Theory and Application. Englewood Cliffs, NJ: Prentice Hall, [5] R. E. Blahut, Algebraic Methods for Signal Processing and Communications Coding. New York, USA: Springer- Verlag, 199. [6] J. K. Wolf, Redundancy, the discrete Fourier transform, and impulse noise cancellation, IEEE Transactions on Communications, vol. COM-31, no. 3, pp , March [7] P. J. S. G. Ferreira and J. M. N. Vieira, Locating and correcting errors in images, Proceedings of the 4th IEEE International Conference on Image Processing, pp , Santa Barbara, CA, USA, October [8] R. E. Blahut, Algebraic Codes for Data Transmission. Cambridge, UK: Cambridge Univ. Press, 00. [9] A. V. Oppenheim, R. W. Schafer, and J. R. Buck, Discrete-Time Signal Processing. Englewood Cliffs, NJ: Prentice- Hall, [10] R. G. Horn and C. R. Johnson, Matrix Analysis. Cambridge, UK:Cambridge University Press, [11] P. J. S. G. Ferreira, The eigenvalues of matrices that occur in certain interpolation problems, IEEE Transactions on Signal Processing, vol. 45, no. 8, pp , August 1997.

25 5 [1] H. Stark and J. W. Woods, Probability and Random Processes with Applications to Signal Processing. Upper Saddle River, NJ: Prentice Hall, 00. Georgios Takos Biography text here. PLACE PHOTO HERE Christoforos N. Hadjicostis Biography text here. PLACE PHOTO HERE