Error Floors in LDPC Codes: Fast Simulation, Bounds and Hardware Emulation

Transcription

1 Error Floors in LDPC Coes: Fast Simulation, Bouns an Harware Emulation Pamela Lee, Lara Dolecek, Zhengya Zhang, Venkat Anantharam, Borivoje Nikolic, an Martin J. Wainwright EECS Department University of California, Berkeley, CA, 9472, USA EECS Department, Massachusetts Institute of Technology, Cambrige, MA, 2139, USA pam Abstract The error-correcting performance of low-ensity parity check (LDPC) coes, when ecoe using practical iterative ecoing algorithms, is known to be very close to Shannon limits in the asymptotic limit of large blocklengths. A substantial limitation to the use of finite-length LDPC coes is the presence of an error floor in the low frame error rate (FER) region. This paper evelops two methos, a stochastic one base on importance sampling an a eterministic one base on high SNR asymptotics, as applie to suitably efine absorbing structures within the LDPC coe, to preict error floors. Our results are in very close agreement with harware-base experimental results, an moreover exten the preiction of the error probability to as low as 1 3. Our eterministic estimates are guarantee to be a lower boun to the error probability in the high SNR regime. I. INTRODUCTION The class of low-ensity parity check (LDPC) coes was first introuce by Gallager [8], an has recently been the focus of intensive stuy [12]. These coes are attractive because they yiel excellent error-correction, getting close to the Shannon limit for large blocklengths, even when ecoe using suboptimal but practical message-passing algorithms. However, for intermeiate block-lengths an low bit error rates that are of primary interest in practical systems, many LDPC coes exhibit an error floor, which correspons to a ecrease in the slope in the plot of bit error rate (BER) versus signal-to-noise ratio (SNR). The error floor is commonly attribute to the suboptimality of the iterative ecoing algorithms on graphs with cycles, an past work has stuie concepts such as near-coewors [1], trapping sets [11], pseuocoewors [7], an elementary trapping sets [9]. In our own previous work, we have introuce the notion of absorbing sets as the main cause of the error floor of structure LDPC coes. These absorbing sets are a specific type of nearcoewors [1] or trapping sets [11] that are stable uner bit-flipping operations. It can be shown [3] that the factor graphs associate with certain structure LDPC coes contain absorbing sets which have strictly fewer bits than the minimum coewor weight. The performance of an iterative ecoing algorithm in the low BER region is preominantly ictate by the number an the structure of the smallest absorbing sets, in contrast to the performance of a maximum-likelihoo ecoer governe by the minimum istance coewors. In early work on error floors, Richarson [11] evelope a fast-simulation metho, base on using simulation traces of a harware emulator to extract trapping set caniates, an then using an approximate integration technique to estimate the associate error probability. This metho involves a simulation over a sequence of channel noise realizations, suitably biase towars a caniate trapping set. Another approach to preict the performance utilizes a variant of importance sampling, where typically the simulations are performe uner a single but carefully chosen biasing ensity. This approach also obviates the nee for analytical approximations. Since the pioneering work by Richarson [11], the problem of efficiently preicting the error-floor behavior of graph-base coes has spurre significant research activity. Other past work has propose various types of importance sampling schemes base on trapping sets [2], [5], [15], [16]. We work irectly with combinatorial structures of factor graphs referre to as absorbing sets. An absorbing set is efine inepenently of the particular ecoing scheme or channel noise moel, an as such can be stuie analytically [3]. Our results also emonstrate a very close agreement with the experimental results obtaine on a harware emulator [17] for low probabilities of error. Our past work [4] introuce a metho base on exact enumeration of absorbing sets an importance sampling. In this work, we both refine this importance-sampling-base approach, an evelop an analytic approach for lower bouning the error probability. We illustrate its behavior for various coes, an quantize forms of sum-prouct ecoing, showing goo agreement with harware-base experimental results, as well as theoretical preictions that exten to probabilities of error as low as 1 3. II. BACKGROUND A low-ensity parity check coe of blocklength n can be represente as the null space of a sparse parity check matrix H {, 1} m n. It is convenient to view the coe in terms of its Tanner graph [14]: a bipartite graph G =(V,F,E) in which V = {1,...,n} inex the coe bits an F = {1,...,m} inex the coe checks, an E = {e(i, j) H(j, i) =1}. To keep the paper self-containe, we briefly efine the notion of an absorbing set; see the papers [3], [4] for more etails. For a subset D of V,letE(D) (resp. O(D)) bethe set of neighboring vertices of D in F in the graph G with even (resp. o) egree with respect to D. Given an integer pair (a, b), an(a, b) fully absorbing set is a subset D of V of size a, with O(D) of size b an with the property that each /8/$ IEEE 444

2 element of V has strictly fewer neighbors in O(D) than in F \O(D). Array-base LDPC coe constructions [6] are a representative of a class of high-performing structure LDPC coes. It is known [3] that these coes have absorbing sets that are strictly smaller than the minimum istance of the coe; moreover, results from harware emulation show that their low BER performance an the error floor are inee ominate by these absorbing sets [17]. These coes will be use in the remainer of the paper to conveniently illustrate the methoology evelope here. We consier the sum-prouct ecoing algorithm, an BPSK signalling (with mapping 1, 1 1) over the aitive white Gaussian noise (AWGN) channel. The AWGN channel moel assumes that the channel input x i { 1, 1}, corresponing to the ith bit in the transmitte coewor, is receive as y i = x i + w i, where w i is a zero-mean Gaussian signal with variance σ 2, itself inepenent of the channel input an Gaussian signals associate with other x j for i j. The exact equations escribing message upates of this algorithm are by now stanar an well-known [12]. The sum-prouct algorithm is typically allowe to run for a fixe number of iterations, both because convergence is not guarantee when cycles are present, an ue to practical (elay) constraints. For practical harware implementations, the real-value messages in the sum-prouct ecoing algorithm are necessarily quantize, an we present results for various quantization schemes. III. IMPORTANCE SAMPLING AND HARDWARE EMULATION In this section, we escribe an importance sampling metho for preicting error probabilities, base on mean-shifting the original Gaussian ensity towars an absorbing set of interest. We also erive approximate confience intervals for these stochastic estimates, an show that harware-base experimental results fall well within 95% confience intervals. Importance sampling (IS) is a particular type of Monte Carlo metho which uses statistical sampling to approximate analytic expressions of probabilities. The basic iea is to perform simulation uner a tilte istribution so as to make the event of interest more likely; the averages are then reweighte to compensate for the tilting. Supposing without loss of generality that the all-zeros wor is transmitte, let Y (1),...,Y (M) be a set of M trials, each Y (i) R n sample in an i.i.. manner from a biase istribution f bias. The associate IS estimate of a particular absorbing probability is given by ˆq IS = 1 M M I error (Y (i) )w(y (i) ), i=1 where I error is a 1-value inicator function for whether the ecoer converges to the given absorbing set on trial Y (i),an w(y (i) )= f(y (i) ) f bias is the appropriate weighting function to (Y (i) ) prouce an unbiase estimate. In the case of the all-zeros coewor being transmitte in a BPSK - moulate Gaussian channel, the original ensity f is an n-variate Gaussian N( 1 n,σ 2 I n n ). A suitable choice [4] of f bias is the mean-shifte Gaussian N(ν(μ),σ 2 I), where ν(μ) i = 1 μ for elements insie the absorbing set, an ν(μ) i =1otherwise. With this choice, the IS weight w is given by 1 w(y (i) μ, σ 2 )= e 2σ 2 [ a (i) k=1 (Y k 1)2 ], e 1 2σ 2 [ a (i) k=1 (Y k (1 μ))2 ] where Y (i) k are the values of the noes in the absorbing set. Due to symmetry of the coe an channel, the probability of error that is, the absorbing probability of any fixe (a, b) absorbing set is equal to that of any other exemplar having the same absorbing set structure. Since the associate events are isjoint, the error probability p associate with all (a, b) absorbing sets is equal to p = N sets q, where N sets is the total number of (a, b) absorbing sets of the same structure. The associate IS estimate of p is given by p ˆp IS = N sets ˆq IS. Note that since we are using ˆq IS as an approximation for q, where q enotes the error probability ue to a fixe representative of the class of isomorphic (a, b) absorbing sets, the quantity ˆp IS is an approximation for p, an not guarantee to either upper or lower boun the true error probability. The total number of (a, b) absorbing sets in a given array-base LDPC coe can be foun using the technique iscusse in [3]. When the error floor is ominate by a particular isomorphic sub-class of (a, b) absorbing sets, the associate absorbing probability can be use to estimate the error floor. In orer to evaluate the accuracy of the importance sampling error curves, we calculate 95% confience intervals for points on the curves. The stanar scale for comparing FER curves is log 1 (ˆp IS ) an log 1 (p) versus SNR; accoringly, we first fin the variance of log 1 (ˆp IS ). The variance of the estimator ˆp IS is given by var(ˆp IS )=E[(ˆp IS ) 2 ] p 2. Letting enote convergence in istribution, the eltametho [13] states that if some sequence of ranom variables {X M } satisfies M( X M μ) N(,σ 2 ), then for any twice-ifferentiable function f, wehave M(f( XM ) f(μ)) N(, [f (μ)] 2 σ 2 ). Note that by the central limit theorem, we have M(ˆpIS p) N(,β 2 ) where β 2 = M var(ˆp IS ). By applying the elta-metho to f(x) =log 1 (x), we obtain ( β 2 ) M(log(ˆpIS ) log(p)) N, p 2 (ln 1) 2. In practice, we cannot make irect use of this asymptotic result, since we know neither β 2 = M var(ˆp IS ) nor p. However, we can estimate these quantities by their sample versions namely, p ˆp IS,an ( ) β 2 Nsets 2 1 M I error (Y (i) )w 2 (Y (i) ) ˆq IS 2. M i=1 thereby obtaining the approximation var(log(ˆp IS )) β 2 ˆp 2 IS M (ln. Finally, using this approximation, we can compute an 1)2 approximate 95% confience interval for log(ˆp IS ) by 445

3 (4,8) abs. set error (IS est.) harware FER curve (6,4) abs. set error (IS est.) harware FER curve 5 log(perror) 5 1 log(perror) SNR (B) SNR (B) Fig. 1. Comparisons of importance sampling estimates of error probabilities with harware-base emulation results. (4, 8) absorbing set of the (229, 1978) array-base LDPC coe (6, 4) absorbing set of the (229, 224) array-base LDPC coe. Dots below an above correspon to approximate 95% confience intervals; see text for etails. applying Chebyshev s inequality: P( log(ˆp IS ) log(p) ɛ) var(log(ˆp IS)) ɛ 2 var(log(ˆp IS)) ɛ 2. A. Absorbing regions for ecoers Thus, the approximate 95% confience interval for log(ˆp IS ) is [log(ˆp IS ) ± ɛ], where ɛ = var(log(ˆp IS ))/.5. Figure 1 provies a comparison of the importance-sampling (IS) preictions to results from harware emulation [17] for two ifferent coes, having ifferent absorbing sets. The ots surrouning the IS curves correspon to the 95% confience interval; note how the IS preiction shows very goo agreement with the harware emulation. IV. DETERMINISTIC LOWER BOUNDS AND CHANNEL INVARIANCE Applying the IS metho requires substantially less computation than irect simulation, but the computation must be re-performe each time that the channel parameters are change. In this section, we escribe a proceure for generating analytical expressions, guarantee to lower boun the error probability in the high SNR regime, an that require substantially less computation. At a high level, our proceure consists of the following three steps. First, we compute an inner boun to the absorbing region associate with an absorbing set from a particular class. This step is a purely eterministic computation, an is inepenent of the unerlying channel moel. Secon, using a generalize Chernoff approximation [1], we compute an analytical approximation to the absorbing probability: i.e., the probability uner the specifie channel moel that the log-likelihoo vector falls into this given absorbing region. This approximation is guarantee to lower boun the true absorbing probability in the high SNR regime. (c) Thir, we combine the absorbing probability with a count (or lower boun) on the carinality of the class of absorbing sets. The combination of these three ingreients yiels eterministic approximations to the error probability, guarantee to be lower bouns in the high SNR regime. We note that this proceure can be carrie out for any channel. Inee, once the absorbing regions an counts in steps one an three have been calculate once, this computation nee not be repeate for aitional channels, since the only channelepenent quantity is the boun calculation in the secon step. Consier some fixe ecoer (e.g., floating-point sumprouct, quantize sum-prouct, or a bit-flipping ecoer) that operates on an LDPC coe of blocklength n. Onany given trial, the ecoer is initialize with a log-likelihoo ratio (LLR) vector z R n of observations at each bit noe. After some fixe number of iterations, the estimate LLRs of the ecoer are threshole, yieling a {, 1} n sequence that is an estimate of the transmitte coewor. The absorbing region R of a given absorbing set is the set of LLR vectors z R n for which the ecoer outputs the inicator vector of the absorbing set as its estimate. Two properties of this absorbing region are important. First, it is a channel-inepenent quantity, since it is only a function of the initializing LLR vector z R n. (Although z is a ranom vector, with istribution epenent on the channel, conitione on a particular initialization, the ecoer s behavior is purely eterministic, an hence channel-inepenent.) Secon, it varies as features of the ecoer (e.g., quantization scheme etc.) are change; inee, the relative size of the absorbing region is a measure of its impact on a particular ecoer. Exact computation of the absorbing region is prohibitively expensive, since it involves testing the ecoer over an n- imensional space. (For instance, iscretizing each imension to l locations, yiels the complexity O(l n ).) In practice, we are force to seek approximations to the absorbing region, base on examining particular low-imensional projections R of the absorbing region. To approximate the exact absorbing region, we first ivie the bit noes of the absorbing set into groups of noes with the same number of satisfie an unsatisfie check noes. For example, the (8, 6) absorbing set in the (229, 1978) array coe is mae up of six variable noes of type (4, 1) an two variable noes of type (5, ), where any (4, 1) noe is connecte to four satisfie check noes an one unsatisfie check noe an any (5, ) noe is connecte to five satisfie checks an zero unsatisfie checks. Each of these groups of noes is associate with an axis. All bit noes are initially assigne value 1 (the all-zeroes coewor uner BPSK). In the two-imensional case, the region is foun by 446

4 shift of pair 2 (3,2) noes (6,2) quantization (8,4) quantization (9,5) quantization shift of pair 1 (3,2) noes shift of (5,) noes (6,2) quantization (8,4) quantization (9,5) quantization shift of (4,1) noes Fig. 2. Subsets of the signal space regions for which the ecoer converges to the absorbing set. These plots show particular projections of these absorbing regions. Absorbing region for a (4, 8) absorbing set in the (229, 1978) array coe. The plot shows three ifferent levels of quantization: (6, 2), (8, 4), an(9, 5) fixe-point quantization [17]. Note how the absorbing region contracts as the quantization scheme improves. Absorbing region for a (8, 6) absorbing set in the (229, 1978) array coe. trying all combinations of shifts for the two groups of noes, where each axis ranges from the absorbing set (centere at a shift of 2) to the all-zeros coewor (centere at a shift of ), separate by increments of size.1. The ecoer is run for 5 iterations, an if the har ecision at the en of these iterations is the absorbing set, then we inclue this combination of shifts of the absorbing set noes in a set S. The approximation to the absorbing region is this set S of points that ecoe to the absorbing set. Figure 2 illustrates some approximations of absorbing regions, for ifferent ecoers an ifferent absorbing sets. Each panel is a two-imensional plot, with the upper right (, ) point corresponing to receiving the all-zeros coewor (without any noise), an the lower left ( 2, 2) centere on the absorbing set. The marke contours correspon to the bounary between not ecoing to an ecoing to the appropriate absorbing set (towars lower left). Panel shows regions for the (4, 8) absorbing set, taken from the Tanner graph of the (229, 1978) array-base LDPC coe, for three ifferent quantize forms of sum-prouct (etails of the quantization choices are in [17]). The four variable noes in the (4, 8) absorbing set are all (3, 2) noes, so the four noes are ranomly ivie into two pairs in orer to show a twoimensional plot, which highlights the symmetry of variable noes of the same type an thus supports the metho of grouping noes of one type into one axis. Note how the size of the absorbing region shrinks as the quantization is improve. The effect of better quantization on absorbing regions is also seen in panel, which shows absorbing regions for the (8, 6) absorbing set of the same coe uner three ifferent quantization schemes. The effect of a finer quantization scheme is more pronounce when the bits in the absorbing set have only marginally more neighboring satisfie versus unsatisfie checks since the aitional bits use to represent the messages can more easily help favorable messages overpower the unfavorable messages, as is the case for the (4, 8) absorbing sets. B. Asymptotics for absorbing probability Whereas the absorbing region itself is channel-inepenent, its effect on ecoing epens heavily on the channel. In particular, if we view the ecoer s input vector Z R n as ranom, rawn from a istribution specifie by the channel moel, then the absorbing set s influence correspons to the probability mass assigne to the absorbing region. Let γ enote the signal-to-noise ratio of the channel, an for a given absorbing region R, consier the absorbing probability P γ [Z R]= R f Z(z; γ)z, where P γ enotes the fact that the channel probability istribution varies as a function of the SNR, an f Z (z; γ) is the ensity of the channel uner consieration. Assuming that Z has a moment generating function E[exp(λ T Z)] for some neighborhoo of zero, we may use the (generalize) Chernoff approximation [1]: { log P γ [Z R] inf SR ( λ)+loge[exp(λ T Z)] },(1) λ> where S R ( λ) := sup u R ( λ T u) is the support function of the set R. As an example, when Z is multivariate Gaussian with mean zero an signal-to-noise parameter γ = 1/σ 2, from (1) we obtain the approximation log P γ [Z R] 1 2σ 2 ( ; R), where ( ; R) = 2 min u R u 2 enotes the minimum Eucliean istance from to R. This approximation becomes exact as the signal-tonoise γ =1/σ 2 tens to infinity. Moreover, this same approach can also be applie to compute analytical approximations for other channels, which coul be use to moel time-varying behavior or moel uncertainty. Consier a mixture channel, in which the noise on per-symbol basis is either N(,σ 2 ) with probability 1/2 ( goo channel state), or N(, 4σ 2 ) with probability 1/2 ( ba channel state). In this case the Chernoff approximation (1) no longer has an explicit solution, but can be compute easily. C. Computing lower bouns on error floors We now put together the pieces to compute approximations to error floors. We also prove that these approximations are guarantee to be lower bouns in the high SNR regime. Overall, given that the all-zeros coewor was sent, the error 447

5 1 1 (8,6) abs. set error (IS est.) Asymptotic approximation 1 1 (8,6) abs. set error (IS est.) Asymptotic approximation Absorbing Probability Absorbing Probability SNR (B) SNR (B) Fig. 3. IS result an the eterministic estimate of abs. probability for the (8, 6) abs. sets of the (229, 1978) array-base LDPC coe. Stanar AWGN channel. Mixture channel: noise given by N(,σ 2 ) with probability 1 2,anN(, 4σ2 ) with probability 1 2. probability can be ecompose into a isjoint union of error events, incluing the error probability associate with some class A of isomorphic absorbing sets, so that p err P[A]. The error event associate with the class A can be ecompose into a isjoint union of error events associate with iniviual absorbing sets. Since each absorbing set belonging to the same class has the same unlabele structure, they all look the same from the point of view of the transmitte coewor, an thus each of these iniviual error probabilities is ientical, so that p err car(a) P[Z R], (2) where car(a) enotes the number of absorbing sets in the given class. In past work [3], we have escribe analytical methos for computing car(a) for structure LDPC coes, so let us focus on the absorbing probability P[Z R]. From equation (1), the absorbing probability for the AWGN channel ecays at a rate specifie by 2 (; R), the minimum Eucliean istance from the all-zeros vector to the absorbing region R. In practice, we cannot compute this minimum istance exactly, since the true absorbing region R is a complicate subset of R n. However, consier the projections R R compute by the proceure previously escribe. Note that by construction, any vector insie R R, if pae with (n ) zeros to exten it to an n-vector, is guarantee to lie within the true absorbing region. Therefore, the probability P[Z inverse image of R uner projection] is guarantee to be a lower boun to the true absorbing probability P[Z R]. We illustrate the boun obtaine by this proceure for (8, 6) absorbing sets of the (229, 1978) array-base LDPC coe in Fig. 3. The boun shows a very close agreement with the IS curve an, as preicte theoretically, becomes a lower boun as the SNR increases. V. CONCLUSION We presente two methos for evaluating the probability of error of iteratively ecoe LDPC coes. The first metho, builing on our previous work, uses importance sampling with appropriate mean shifts with respect to absorbing sets, an prouces stochastic error probabilities, showing very close agreement with experimental results obtaine from a harware emulator. The secon metho prouces eterministic estimates, base on computing projections of absorbing regions an using asymptotics to estimate associate error probabilities. These eterministic estimates are guarantee to lower boun the true error probability in the high SNR regime. An interesting future irection is whether similar techniques can yiel matching upper bouns on error probabilities. Acknowlegements: Work supporte by NSF grant CCF an a grant from Marvell Corp. through the UC Micro program. Computing infrastructure support provie by NSF grant CNS PROJECT. REFERENCES [1] S. Boy an L. Vanenberghe. Convex optimization. Cambrige University Press, Cambrige, UK, 24. [2] E. Cavus, C. Haymes, an B. Daneshra. A highly efficient importance sampling metho for performance evaluation of LDPC coes at very low bit errors rates. Submitte IEEE Trans. on Comm. [3] L. Dolecek, Z. Zhang, V. Anantharam, an M. Wainwright B. Nikolic. Analysis of absorbing sets of array-base LDPC coes. In IEEE Int. Conf. on Comm., June 27. [4] L. Dolecek, Z. Zhang, M. Wainwright, V. Anantharam, an B. Nikolic. Evaluation of the low frame error rate performance of LDPC coes using importance sampling. In IEEE Info. Theory Workshop, Sept. 27. [5] C. A. Cole et al. A general metho for fining low error rates of LDPC coes. Submitte to IEEE Trans. Info. Theory. [6] J. Fan. Array-coes as low-ensity parity-check coes. In 2n Int. Symp. on Turbo Coes, Sept. 2. [7] B. J. Frey, R. Koetter, an A. Vary. Skewness an pseuocoewors in iterative ecoing. In ISIT, [8] R. G. Gallager. Low-ensity parity check coes. MIT Press, Cambrige, MA, [9] S. Laenner an O. Milenkovic. Algorithmic an combinatorial analysis of trapping sets in structure LDPC coes. In WirelessComm, June 25. [1] D. MacKay an M. Postol. Weaknesses of Margulis an Ramanujan- Margulis low-ensity parity-check coes. Elec. Notes in Theo. Comp. Science, 74, 23. [11] T. Richarson. Error floors of LDPC coes. In Allerton, Oct. 23. [12] T. Richarson, A. Shokrollahi, an R. Urbanke. Design of capacityapproaching irregular low-ensity parity check coes. IEEE Trans. Info. Theory, 47: , February 21. [13] R. J. Serfling. Approximation Theorems of Mathematical Statistics. Wiley Series in Probability an Statistics. Wiley, 198. [14] R. M. Tanner. A recursive approach to low complexity coes. IEEE Trans. Info. Theory, 81(5): , September [15] B. Xia an W. E. Ryan. Estimating LDPC coewor error rates via importance sampling. In ISIT, July 24. [16] B. Xia an W. E. Ryan. Importance sampling for tanner trees. IEEE Trans. on Info. Theory, 51: , June 25. [17] Z. Zhang, L. Dolecek, V. Anantharam, an M. Wainwright B. Nikolic. Quantization effects of low-ensity parity-check ecoers. In IEEE Int. Conf. on Comm., June