Improved Min-Sum Decoding of LDPC Codes Using 2-Dimensional Normalization

Transcription

1 Improved Min-Sum Decoding of LDPC Codes sing -Dimensiona Normaization Juntan Zhang and Marc Fossorier Department of Eectrica Engineering niversity of Hawaii at Manoa Honouu, HI 968 Emai: juntan, Daqing Gu and Jinyun Zhang Mitsubishi Eectric Research Laboratories 01 Broadway, Cambridge, MA 0139 Emai: dgu, Abstract A two-dimensiona post normaization scheme is proposed to improve the performance of conventiona min-sum MS and normaized MS decoding of irreguar ow density parity check codes. An iterative procedure based on parae differentia optimization agorithm is presented to obtain the optima two-dimensiona normaization factors. Both density evoution anaysis and specific code simuation show that the proposed method provides a comparabe performance as beief propagation decoding whie requiring ess compexity. Interestingy, the new method exhibits a ower error foor than that of beief propagation decoding in the high SNR region. With respect to standard MS and one-dimensiona normaized MS decodings, the two-dimensiona normaized MS offers a consideraby better performance. I. INTRODCTION Low-density parity-check LDPC codes [1] were first introduced in the 1960 s and rediscovered in the 1990 s. Recenty, it was found that the beief propagation BP agorithm [] provides a powerfu too for iterative decoding of LDPC codes. As shown in [3], LDPC codes with iterative decoding based on BP achieve a remarkabe error performance that is very cose to the Shannon imit [4]. Consequenty, LDPC codes have received significant attention recenty. An LDPC code is specified by a parity-check matrix containing mosty zeros and ony a sma number of ones. In genera, LDPC codes can be categorized into reguar LDPC codes and irreguar LDPC codes. An LDPC code is caed reguar if the weights of rows and cous in its parity check matrix are equa and is caed irreguar if not. It has been shown, both theoreticay and by simuation, that with propery chosen structure, irreguar LDPC codes have better performance than reguar ones [5]. To decode LDPC codes, either soft decision, hard decision or hybrid decision decoding can be used. It has been shown that soft decision decoding based on BP is one of the most powerfu decoding methods for LDPC codes. Athough BP decoding offers good performance, it can become too compex for hardware impementation because of foating point computations. By approximating the cacuation at the check nodes with a simpe minimum operation, the min-sum MS agorithm reduces the compexity of BP [6], [7]. Whie MS is hardware efficient, its utimate performance is often much worse than that of BP. It has been observed that the degradation of MS can be compensated by inear post processing normaization of the messages deivered by check nodes [8], [9]. The optima normaization factors can be determined using density evoution to search for the factors which yied the owest threshod. Simuation resuts and density evoution anaysis show that for decoding reguar LDPC codes, normaized MS with a singe normaization factor is sufficient to achieve good performance which is near that of BP. For decoding many irreguar LDPC codes, however, conventiona normaized MS exhibits a arge performance degradation compared to that of BP. In this paper, we present a two-dimensiona -D normaized MS decoding of irreguar LDPC codes. In -D normaized MS scheme, beief messages outgoing from both check and bit node processors are normaized. The normaization factor of a check bit node processor depends on the degree of that node. To circumvent brute force search, parae differentia optimization and density evoution are used to obtain the -D optima normaization factor pair. This paper is organized as foows. Section briefy reviews the standard BP decoding. Section 3 and Section 4 anayze density evoution of conventiona MS and normaized MS decoding, respectivey. Section 5 presents -D normaized MS decoding and the corresponding density evoution. Section 6 describes the parae differentia optimization procedure to obtain the optima -D normaization factor pairs. Section 7 reports simuation resuts and Section 8 concudes this paper. II. STANDARD BP Suppose a reguar binary N, Kdv, dc LDPC code C is used for error contro over an AWGN channe with zero mean and power spectra density N 0 /. Assume BPSK signaing with unit energy, which maps a codeword w = w 1, w,..., w N into a transmitted sequence q = q 1, q,..., q N, according to q n = 1 w n, for n = 1,,..., N. If w = [w n ] is a codeword in C and q = [q n ] is the corresponding transmitted sequence, then the received sequence is q + g = y = [y n ], with y n = q n + g n, where for 1 n N, g n s are statisticay independent Gaussian random variabes with zero mean and variance N 0 /. Let H = [H ] be the parity check matrix which defines the LDPC code. We denote the set of bits that participate in check m by N m = {n : H = 1} and the set of checks in

2 which bit n participates as Mn = {m : H = 1}. We aso denote N m\n as the set N m with bit n excuded, and Mn\m as the set Mn with check m excuded. We define the foowing notations associated with i th iteration: ch,n : The og-ikeihood ratios LLR of bit n which is derived from the channe output y n. In BP decoding, we initiay set ch,n = 4 N 0 y n. : i The LLR of bit n which is sent from check node m to bit node n. : i The LLR of bit n which is sent from the bit node n to check node m. : The a posteriori LLR of bit n computed at each iteration. i n The standard LLR BP agorithm is carried out as foows [3]: Initiaization: Set i = 1, maximum number of iteration to I Max. For each m,n, set 0 = ch,n. Step 1: i Horizonta Step, for 1 n N and each m Mn, process: Step : Step 3: ii i = tanh 1 n N m\n tanh i 1 ertica Step, for 1 n N and each m Mn, process: i = ch,n + m Mn\m i n = ch,n + m Mn Hard decision and stopping criterion test: Create ŵ i = [ŵ n i ] such that ŵ n i < 0, and ŵ n i = 0 if n i 0. i n = 1 if ii If Hŵ i = 0 or the maximum iteration number I Max is reached, stop the decoding iteration and go to Step 3. Otherwise set i := i + 1 and go to Step 1. Output ŵ i as the decoded codeword. III. MS DECODING The check node processing in the standard BP decoding may require considerabe computationa resource and may cause hardware impementation difficuties as we as high decoding deay. BP decoding can be simpified by approximating the cacuation at the check nodes with a simpe minimum operation, which resuts in MS decoding [6], [7]. A. MS agorithm In MS decoding, the bit node operation is the same as in the standard BP. Taking advantage of the odd property of the 1 function tanh, MS simpifies the updating rue in check nodes by modifying 1 into i = sgn i 1 min i 1 n N 3 m\n n N m\n The MS agorithm is much simper than the BP decoding, since ony comparisons and additions are needed in check and bit node processing. The product of the signs in 3 can be easiy cacuated by moduo addition of the hard decision of a { i 1 : n N m\n}. The minimum magnitude can be found by comparison. B. Density evoution of MS Density evoution can be used to track the process of iterative decoding and determine the threshod of an iterativey decodabe code when decoded by a certain agorithm [4]. The density evoution of MS has been presented in [10], [11]. By appying a simiar anaysis approach as [1, Lemma 4.1], in the foowing we derive it in a different way. Let f i i and f be the pdf s of LLR deivered by bit and check node, respectivey. In MS, the density evoution of bit node is the same as that in BP f i = F 1 Ff ch Ff i dc 1 where F denotes Fourier transform. To derive the density evoution of check node processor in MS, we can first prove a theorem foowing a simiar spirit as the emma 4.1 [1, p.41]. Consider J independent random variabes {X 1, X,..., X J }. Let p Xj x be the pdf of X j, fo = 1,,..., J. Given a nonnegative vaue a, et P Xj,+ and P Xj, be the probabiities of X j has magnitude greater or equa than a, and sign + and, respectivey; i.e, P Xj,+ = + p a Xj xdx and P Xj, = a p Xjxdx. Next consider a new random variabe J J Y = sgnx j min X j. 4 Simiary define P Y,+ and P Y, as the probabiities that Y has magnitude greater or equa than a, and sign + and, respectivey. We have Lemma 1: J P Y,+ = P Xj,+ + P Xj, + J P Xj,+ P Xj, 5 J P Y, = P Xj,+ + P Xj, J P Xj,+ P Xj, 6 Proof: Consider the function J P Xj,+ + t P Xj, 7 Observe that if this function is expanded as a poynomia in t, the coefficient of t k is the probabiity that in

3 {X 1, X,..., X J } there are k negative items and J k positive items, and each of them has a magnitude greater than or equa to a. The expansion of function J P Xj,+ t P Xj, 8 is identica except that a the coefficients of t k with odd power k are the opposite of those in the expansion of 7. Adding 7 and 8, a coefficients of t k with even power k are doubed, whie a the coefficients of t k with odd power k are canceed out. Setting t = 1 and dividing the summation by, the resut is the probabiity that in {X 1, X,..., X J }, there are even number of items with sign and magnitude greater than or equa to a. From 4, it is cear that this probabiity equas the probabiity that Y has sign + and magnitude greater than or equa to a; therefore we obtain 5. Foowing a simiar approach, it is straightforward to prove 6. If the J independent random variabes {X 1, X,..., X J } have identica distribution p X x, we have Coroary : P Y,+ = P X,+ + P X, J + P X,+ P X, J P Y, = P X,+ + P X, J P X,+ P X, J where P X,+ = + p a X xdx and P X, = a p Xxdx. Based on the assumption of infinite codeword ength and tree ike structure of the graph representation of dv, dc reguar LDPC codes, the LLR vaues deivered by check bit nodes have the same distribution. The check node processing 3 can be expressed as = dc 1 sgn i 1 j dc 1 min i 1 j. 9 where 1,,..., dc 1 are i.i.d. Foowing simiar notations as in [10], for a > 0, we define ψ i + a = + f i a vdv and ψ i a = a f i vdv. Hence ψi + a and ψ i a are the probabiities that i has magnitude no ess than a, and sign + and, respectivey. Appying Coroary, for u > 0, the PDF of is F i u = 1 P reven number of negative vaues in { i 1 }, and i 1 > u, j j = ψ i 1 [ ψ i 1 j + u + ψ i 1 u + u ψ i 1 ] dc 1 u dc 1 10 Simiary for u < 0, the PDF of is F i u = P rodd number of negative vaues in { i 1 j }, and i 1 j > u, j = 1 [ ψ i 1 ψ i 1 + u + ψ i 1 u + u ψ i 1 dc 1 ] dc 1 u. 11 Differentiating 10 and 11, we obtain the pdf of [ f i dc 1 u = f i 1 u + f i 1 u dc ψ i 1 + u + ψ i 1 u + f i 1 ψ i 1 u f i 1 u + u ψ i 1 ] dc u I. CONENTIONAL NORMALIZED MS 1 Athough the MS agorithm greaty reduces the decoding compexity for impementation, it may cause a significant performance degradation compared to BP decoding. In [8], [9], normaized MS is proposed to improve the performance of standard MS. A. Normaized MS agorithm Let BP and MS denote the LLR vaues cacuated by BP and MS decoding with 1 and 3, respectivey. It was proved that sgn BP = sgn MS and BP > MS [8], which impies that a inear post processing of MS coud reduce the gap between BP and MS decoding. The normaized MS modifies the check node processing 3 as i α i 13 where α is a normaization factor, 0 < α 1. B. Density evoution of normaized MS By keeping the normaization factor constant, it is straightforward to obtain the density evoution of normaized MS from that of standard MS. The ony change is the density function f i u in 1 as f i u 1 α f i u α and the other procedures are the same as in MS [9].. TWO-DIMENSIONAL NORMALIZED MS 14 Another possibe way to improve the performance of MS is to reprocess the deivered LLR from bit nodes by modifying into i = ch,n + β m Mn\m 15

4 where β is a normaization factor. For reguar LDPC codes, the normaization factor for each check bit node is identica. In this case, bit node normaization processing 15 is equivaent to check node normaization. For irreguar LDPC codes, the conventiona normaized MS ony reprocesses the LLRs outgoing from check nodes whie keeping the bit node processor unchanged. For any given check node, the outgoing LLRs are normaized equay without considering the degree differences among the adjacent bit nodes in the graph representation of the code. However, intuitivey, for bit nodes with different degrees, this same incoming LLR shoud pay a different roe in the outgoing LLRs. This impies it is possibe to appy normaization post processing in both check and bit nodes. Because conventionay, ony outgoing LLR from check nodes horizonta step are normaized, we may refer to it as onedimensiona normaized MS whie we refer to this proposed approach as two-dimensiona both horizonta and vertica step normaized MS decoding. A. -D normaized MS agorithm Consider an irreguar LDPC code with degree distribution dv max dc max λx = λ j x j 1 and ρx = ρ j x j 1, which specify the degree distribution of bit nodes and check nodes, respectivey. The parameters λ j and ρ j are the fractions of edges beonging to degree-j bit and check nodes, respectivey. The imits dv max and dc max denote the maximum degree for bit and check nodes, respectivey. Let α j be the normaization factor for check nodes with degree j, fo = 1,,..., dc max. Let β j be the normaization factor for bit nodes with degree j, fo = 1,,..., dv max. Let dvn be the degree of bit node n, for n = 1,,..., N. Let dcm be the degree of check node m, for m = 1,,..., M. The initiaization and hard decision procedures in -dimensiona normaized MS are the same as in MS decoding. Step 1 of normaized MS decoding is Step 1: i Horizonta Step, for 1 n N and each m Mn, process: = α dcm n N m\n sgn i 1 min i 1 n N 16 m\n ii ertica Step, for 1 n N and each m Mn, process: i = ch,n + β dvn 17 i n = ch,n + β dvn m Mn\m m Mn 18 It is worth mentioning that by using different normaization factors at different iterations especiay the first few iterations, the error performance of -D normaized MS decoding can be improved further. B. Density evoution of -D normaized MS Given the structure of a code and an iterative messagepassing decoding agorithm, if the bit error rate BER after each iteration is independent of the transmitted codeword, the anaysis of density evoution can be greaty simpified by assuming the a-zero codeword was transmitted. It has been shown in [4] that, if the channe and message-passing decoding satisfy three symmetry conditions, the entire behavior of the decoder can be predicted from its behavior assuming transmission of the a-zero codeword. Next we verify that the -D normaized MS decoding satisfies these symmetric conditions and therefore its density evoution can be anayzed based on the a-zero transmitted codeword. Channe symmetry: For an AWGN channe, the output is symmetric, fy q = 1 exp πσ = f y q y q σ where σ = 1/R E b / is the variance of the noise and R is the code rate. Check node symmetry: In check node processing, the sign of an outgoing message can be factored out: = Ψ i c 1,,..., dcm 1 = α dcm sgn = dcm 1 n N m\n i 1 min i 1 n N m\n sgn j Ψ i c 1,,..., dcm 1 Bit node symmetry: In bit node processing, the sign of an outgoing message fips if the signs of a the corresponding incoming messages fip, i = Ψ v ch,n, 1,,..., dvn 1 = n,ch β dvn m Mn\m = Ψ v ch,n, 1,,..., dvn 1 Moreover, at iteration 0, the initia beief message ch,n = og P w n = 0 y n P w n = 1 y n = σ y n has density function which is aso symmetric: f ch,n u = σ π exp σ 4/σ = f ch,n u expu Athough the origina pdf of beief messages in -D normaized MS satisfies symmetry conditions, this symmetry property is not preserved during decoding iterations. Therefore there is no guarantee that the probabiity of error is non-increasing for -D normaized MS decoding [1]. Since the pdf of the

5 outgoing messages of check nodes with different degree are distinct, the expectation of these pdf s is the overa pdf. Simiary to 14, the density evoution of degree-j check processor in -D normaized MS is based on f i,j u 1 f i u 19 α j where f i,j u is the pdf of an outgoing message at a degreej check. The overa pdf for a messages going out of check nodes is dcmax f i u = ρ j f i,j u dc max α j ρ j f i u α j α j Simiary the overa pdf for a messages going out of bit nodes is dvmax f i v = λ j 1 j F Ff ch Ff i v β j β j I. OPTIMAL -D NORMALIZATION FACTORS PAIR Let α = {α 1, α,..., α dcmax } and β = {β 1, β,..., β dvmax } be the normaization factors vectors of check and bit nodes in -D normaization MS decoding, respectivey. Given an irreguar LDPC code with degree distribution pair λ, ρ and a normaization factor pair α,β, we can obtain the threshod of this code with -D normaized MS decoding, by using density evoution. We try to find the optima normaization factor pair α,β which yieds the argest noise threshod. This is a noninear cost function minimization probem with continuous space parameters. The brute force search method becomes intractabe when dv max dc max is arge. However an agorithm caed parae differentia has been shown to be efficient and robust. In [5], this technique has been successfuy appied to construct optima irreguar LDPC codes. In this paper parae differentia optimization is used to generate the optima normaization factor pair of -D normaized MS decoding. A. Parae differentia optimization Parae differentia is a parae direct search technique featured by evoution of variabe vectors. Starting from an initia set of randomy generated vectors, the agorithm iterativey update each vector in the set simutaneousy based on the currenty best vector, which yieds the smaest cost function vaue in the ast iteration. The updating procedure is repeated unti the iterations converge, and the resuting soution is the superior vector which has the best cost function vaue. Because after each iteration of updating, the new generation in genera has a smaer average cost function vaue than that of the ast generation, this agorithm is aso caed differentia evoution. By updating a the vectors of the set in parae, the agorithm can hep the vectors escape oca minima and prevent misconvergence. B. Optima normaization factor pair based on density evoution and parae differentia optimization Given an irreguar dv max, dc max LDPC code, the number of degrees of freedom with normaization factor pair is dv max dc max. In differentia evoution optimization, the popuation of each generation is often seected to be mutipe of the number of degrees of freedom. Let L = T dv max dc max be the popuation in each generation, where T is a constant. Apparenty, the arger T is, the higher the probabiity that a better optima resut coud be found, and correspondingy the higher the needed compexity. Let α g, β g denote the -th member of generation g. The procedure to find the optima factor pair of -D normaized MS based on density evoution and parae differentia optimization is carried out as foows Step 1 Initiaization: Set the maximum number of generations to G max and the generation index g = 0. Randomy generate a set of normaization factor pairs {α 0, β 0 } with cardinaity L. For = 1,,..., L, run density evoution of -D normaized MS based on the factor pair α 0, β 0 and obtain the corresponding threshod E b. Then 0 compare and find the factor pair which yieds E the smaest b. We denote this best pair among the origina generation as α 0 best, β 0 best, where 0 Eb best = arg min L. =1 Step Mutation and test: Set g g +1. pdate each factor pair in the candidate set according to the foowing mutation agorithm. For = 1,,..., L, generate J distinct random numbers { 1 L, }, and generate the test vector α g+1 +γ, β g+1 1 j J j:even 1 j J j:odd t = α g best, β g best α g, β g α g, β g 0 where γ is a pre seected rea constant which contros the ampification of the differentia variation. The use of differences between candidate vectors increases the variation, therefore heping the iteration escaping from a oca minimum. Then for = 1,,..., L, run density evoution of -D normaized MS based on the newy generated test factor pair α g+1 and obtain the corresponding threshod E b Step 3 Compare and update: For = 1,,..., L, set α g+1 = {, β g+1 t g+1,t., β g+1 α g+1, β g+1 t if E b g+1,t E b g α g, β g otherwise

6 and g+1 Eb 10 0 g+1 Eb,t = g+1 g if Eb Eb N,t o g Eb otherwise Then update the best factor pair of generation g + 1 to α g+1 g+1 Eb best, β g+1 best, where best = arg min L. =1 Step 4 Stoping test and output: If G max is reached or the smaest threshod of generations stop decreasing, output α g+1 best, β g+1 best. Otherwise, go to Step. II. SIMLATION RESLTS Figure 1 depicts the word error rate WER performance of the standard BP, -D normaized MS, conventiona normaized MS and MS agorithms for decoding a 1600,700 irreguar LDPC code. The check and bit node distributions of this code are ρx = x x x x x 6 and λx = x x x 7, respectivey. We observe that -D normaized MS provides a comparabe performance as BP and interestingy has a ower error foor than that of BP in the high SNR region. We aso observe that -D normaized MS outperforms conventiona normaized MS and MS by about 0.3dB. For this 1600, 700 irreguar LDPC code, the threshods computed by density evoution for standard BP, MS, conventiona MS and -D normaized MS are 0.77dB, 0.93dB, 0.90dB and 0.85dB, respectivey. The optimum normaization factor for 1-D normaized MS is α = 0.75 and the optimum normaization vectors of -D normaized MS are α = 1.00, 0.94, 0.9, 0.88, 0.86 and β = 1.00, 1.00, 0.91, III. CONCLSION In this paper a -D normaized MS decoding has been presented to improve the performance of standard MS and normaized MS decoding. The proposed method requires consideraby ess compexity than that of BP whie introducing sma performance degradation compared with BP. It is interesting to note that in the high SNR region, -D normaized MS can have a ower error foor than that of BP. With respect to conventiona normaized MS which can be viewed as a simpified version of -D normaized MS decoding, the presented method offers a better performance with negigibe increased compexity. Simuation and density evoution show that -D normaized MS provides a good performance versus compexity tradeoff for decoding irreguar LDPC codes. The method discussed in this paper can be extended to offset MS decoding of irreguar LDPC codes [13]. WER Standard BP D Normaized Min Sum 1 D Normaized Min Sum Min Sum Nb/NodB Fig. 1. WER of standard BP, -D normaized Min-Sum, conventiona normaized Min-Sum and Min-Sum agorithms for decoding a 1600,700 irreguar LDPC code I max=00. REFERENCES [1] R. G. Gaager, Low-Density Parity-Check Codes. Cambridge, MA: M.I.T. Press, [] J. Pear, Probabiistic Reasoning in Inteigent Systems: Networks of Pausibe Inference. San Mateo, CA: Morgan Kaufmann, [3] D. J. C. MacKay, Good error-correcting codes based on very sparse matrices, IEEE Trans. Inform. Theory, vo. 45, pp , Mar [4] T. J. Richardson and R. L. rbanke, The capacity of ow-density paritycheck codes under message-passing decoding, IEEE Trans. Inf Theory, vo.47, pp , Feb [5] T. J. Richardson, M. A. Shokroahi, and R.L. rbanke, Design of capacity-approaching irreguar ow-density parity-check codes, IEEE Trans. Inf Theory, vo.47, pp , Feb [6] J. Hagenauer, E. Offer and L. Papke, Iterative decoding of bock and convoutiona codes, IEEE Trans. on Inform., vo. 4, pp , March [7] M. Fossorier, M. Mihajevic and H. Imai, Reduced compexity iterative decoding of ow-density parity check codes based on beief propagation, IEEE Trans. Commun., vo. 47, pp , May [8] J. Chen and M. Fossorier, Near optimum universa beief propagation based decoding of ow-density parity-check codes, IEEE Trans. Commun., vo. 50, pp , Mar. 00. [9] J. Chen, A. Dhoakia, E. Eeftheriou, M. Fossorier and X.-Y. Hu, Reduced-Compexity Decoding of LDPC Codes, IEEE Trans. Commun., vo. 53, pp , Aug [10] X. Wei and A. N. Akansu, Density evoution for ow-density paritycheck codee under Max-Log-MAP decoding, Eectron. Lett., vo.37, pp , Aug [11] J. Chen and M. Fossorier, Density evoution for two improved BP-sased decoding agorithms of LDPC codes, IEEE Commun. Lett., vo.6, pp , May 00. [1] S. Y. Chung, On the construction of some capacity-approaching coding schemes, Ph.D. dissetation, MIT, Cambridge, MA, 000. [13] J. Zhang, M. Fossorier, D. Gu, and J.Y. Zhang, Two-Dimensiona Correction for Min-Sum Decoding of Irreguar LDPC Codes, submitted to IEEE Commun. Lett. ACKNOWLEDGEMENT The authors woud ike to express their gratitude to Jinghu Chen for many interesting discussions and constructive comments.