A Tutorial on Low Density Parity-Check Codes

Transcription

1 A Tutoal on Low Densty Paty-Check Codes Tuan Ta The Unvesty of Texas at Austn Abstact Low densty paty-check codes ae one of the hottest topcs n codng theoy nowadays. Equpped wth vey fast encodng and decodng algothms (pobablstcally,) LDPC ae vey attactve both theoetcally and pactcally. In ths pape, I pesent a thoughout vew of LDPC. I dscuss n detal LDPC s popula decodng algothm: belef popagaton. I also pesent some notceable encodng algothms ncludng Rchadson and Ubanke s lowe tangula modfcaton that gves se to a lnea encodng tme. I show that egula LDPC pefom bette than egula LDPC thus ae moe desable. Index Tems belef popagaton, densty evoluton, Gallage codes, egula codes, low densty patycheck codes L I. INTRODUCTION OW densty paty-check code (LDPC) s an eo coectng code used n nosy communcaton channel to educe the pobablty of loss of nfomaton. Wth LDPC, ths pobablty can be educed to as small as desed, thus the data tansmsson ate can be as close to Shannon s lmt as desed. LDPC was developed by Robet Gallage n hs doctoal dssetaton at MIT n 96 []. It was publshed 3 yeas latte n MIT Pess. Due to the lmtaton n computatonal effot n mplementng the code and decode fo such codes and the ntoducton of Reed-Solomon codes, LDPC was gnoed fo almost 3 yeas. Dung that long peod, the only notable wok done on the subect was due to R. Mchael Tanne n 98 [] whee he genealzed LDPC codes and ntoduced a gaphcal epesentaton of the codes late called Tanne gaph. Snce 993, wth the nventon of tubo codes, eseaches swtched the focus to fndng low complexty code whch can appoach Shannon channel capacty. LDPC was envented wth the wok of Mackay [3], [4] and Luby [5]. Nowadays, LDPC have made ts way nto some moden applcatons such as GBase-T Ethenet, WF, WMAX, Dgtal Vdeo Boadcastng (DVB). Befoe dscussng LDPC, I pesent a bef evew of eo coectng code n secton II. I pay the most attenton to lnea block code. (7, 4) Hammng code s ntoduced as an example to llustate the behavo of a lnea block code. I also dscuss the use of Tanne gaph as equvalence to a paty-check matx fo lnea block codes. Secton II s concluded by the defnton of LDPC. In secton III, I dscuss how to decode an LDPC usng belef popagaton algothm. Both the had-decson decode and the soft-decson decode ae pesented. An example s gven to llustate the use of the algothm. Secton IV talks about how to encode an LDPC n lnea tme. Two appoaches ae pesented, the accumulate appoach, and Rchadson and Ubanke s lowe tangula modfcaton appoach. Secton V dscusses egula LDPC and shows that egula codes pefom bette than egula codes. Thee desgns ae consdeed: Luby s, Rchadson s and Chung s. Secton V shows that t s possble fo egula LDPC to come as close as.45 db to Shannon capacty. II. ERROR CORRECTING CODES A. Intoducton In communcaton, eos can occu due to a lot of easons: nosy channel, scatches on CD o DVD, powe suge n electonc ccuts, etc. It s often desable to detect and coect those eos. If no addtonal nfomaton s added to the ognal message, eos can tun a legal message (bt patten) nto anothe legal bt patten. Theefoe, edundancy s used n eo-coectng schemes. By

2 addng edundancy, a lot of bt pattens wll become llegal. A good codng scheme wll make an llegal patten caused by eos to be close to one of the legal pattens than othes. A metc used to measue the closeness between two bt pattens s Hammng dstance. The Hammng dstance between two bt pattens s the numbe of bts that ae dffeent. Fo example, bt patten and dffe by one bt (the st bt), thus have Hammng dstance of one. Two dentcal bt pattens have Hammng dstance of zeo. A paty bt s an addtonal bt added to the bt patten to make sue that the total numbe of s s even (even paty) o odd (odd paty). Fo example, the nfomaton message s and even paty s used. Snce the numbe of s n the ognal message s 3, a s added at the end to gve the tansmtted message of. The decode counts the numbe of s (nomally done by exclusve OR the bt steam) to detemne f an eo has occued. A sngle paty bt can detect (but not coect) any odd numbe of eos. In the pevous example, the code ate (numbe of useful nfomaton bts/total numbe of bts) s 8/9. Ths s an effcent code ate, but the effectveness s lmted. Sngle paty bt s often used n scenaos whee the lkelhood of eos s small and the eceve s able to equest etansmsson. Sometmes t s used even when etansmsson s not possble. Ealy IBM Pesonal Computes employ sngle bt paty and smple cash when an eo occus [6]. B. Lnea block code If a code uses n bts to povde eo potecton to k bts of nfomaton, t s called a (n, k) block code. Often tmes, the mnmum Hammng dstance d between any two vald codewods s also ncluded to gve a (n, k, d) block code. An example of block codes s Hammng code. Consde the scenao whee we wsh to coect sngle eo usng the fewest numbe of paty bts (hghest code ate). Each paty bt gves the decode a paty equaton to valdate the eceved code. Wth 3 paty bts, we have 3 paty equatons, whch can dentfy up to 3 8 eo condtons. One condton dentfes no eo, so seven would be left to dentfy up to seven places of sngle eo. Theefoe, we can detect and coect any sngle eo n a 7-bt wod. Wth 3 paty bts, we have 4 bts left fo nfomaton. Thus ths s a (7, 4) block code. In a (7, 4) Hammng code, the paty equatons ae detemned as follow: The fst paty equaton checks bt 4, 5, 6, 7 The second paty equaton checks bt, 3, 6, 7 The thd paty equaton checks bt, 3, 5, 7 Ths ule s easy to emembe. The paty equatons use the bnay epesentaton of the locaton of the eo bt. Fo example, locaton 5 has the bnay epesentaton of, thus appeas n equaton and 3. By applyng ths ule, we can tell whch bt s wong by eadng the value of the bnay combnaton of the esult of the paty equatons, wth beng ncoect and beng coect. Fo example, f equaton and ae ncoect and equaton 3 s coect, we can tell that the bt at locaton 6 () s wong. At the encode, f locaton 3, 5, 6, 7 contan the ognal nfomaton and locaton,, 4 contan the paty bts (locatons whch ae powe of ) then usng the fst paty equaton and bts at locaton 5, 6, 7, we can calculate the value of the paty bt at locaton 4 and so on. (7, 4) Hammng code can be summazed n the followng table [7] Bt numbe Equaton coesponds to p Equaton coesponds to p Equaton coesponds to p 3 TABLE (7, 4) HAMMING CODE FULL TABLE p p d p 3 d d 3 d 4

3 3 In table, p denotes paty bts, d denotes data bts. Removng the paty bts, we have TABLE (7, 4) HAMMING CODE ABBREVIATED TABLE d d d 3 d 4 p p p 3 Fo lage dmenson block codes (lage n and k), a matx epesentaton of the codes s used. Ths epesentaton ncludes a geneato matx, G, and a paty-check matx, H. Gven a message p, the codewod wll be the poduct of G and p wth entes modulo : c Gp () Gven the eceved codewod y, the syndome vecto s z Hy () If z then the eceved codewod s eo-fee, else the value of z s the poston of the flpped bt. Fo the (7, 4) Hammng code, the paty-check matx s H (3) and the geneato matx s G (4) H s dven staght fom Table. G s obtaned by Fo paty bt locatons: use assocated data bts (fom Table ) Fo data bt locatons: put a fo the poston of the data bt, the est ae s Fo the geneato matx of (7, 4) Hammng code above, bt locaton ( st ow) s a paty bt, thus we use ow fom table (). Bt locaton 5 (5 th ow) s a data bt, and bt 5 s data bt numbe, thus we set bt to (). If the message s p then the codewod wll be 3 Gp c If no bt s flpped dung tansmsson, n othe wods, y c. Then the syndome vecto s

4 z Hy 4 shown below, wth check nodes on the left and message nodes on the ght. c +c 3 +c 5 +c 7 c +c 3 +c 6 +c 7 f f c c c 3 c 4 If the 6 th bt s flpped, c 4 +c 5 +c 6 +c 7 f c 5 then z Hy y 3 Readng z fom the bottom up (hghe poston fst), we see the flpped bt s ndeed 6 (). C. Tanne gaph A vey useful way of epesentng lnea block codes s usng Tanne gaph. Tanne gaph s a bpatte gaph, whch means the gaph s sepaated nto two pattons. These pattons ae called by dffeent names: subcode nodes and dgt nodes, vaable nodes and check nodes, message nodes and check nodes. I wll call them message nodes and check nodes fom now on. Tanne gaph maps dectly to the paty-check matx H of a lnea block code, wth check nodes epesent the ows of H. The Tanne s gaph of (7, 4) Hammng code s Fgue : Tanne gaph fo (7, 4) Hammng code As seen above, thee s an edge connects a check node wth a message node f the message node s ncluded n the check node s equaton. Fom a Tanne gaph, we can deduce a paty-check matx by puttng a at poston (, ) f thee s an edge connectng f and c. The code defned by a Tanne gaph (o a paty-check matx) s the set of vectos c (c,,c n ) such that Hc T. In othe wods, the code foms the null space of H. D. Low Densty Paty-check Code (LDPC) Any lnea code has a bpatte gaph and a paty-check matx epesentaton. But not all lnea code has a spase epesentaton. A n m matx s spase f the numbe of s n any ow, the ow weght w, and the numbe of s n any column, the column weght w c, s much less than the dmenson (w << m, w c << n). A code epesented by a spase paty-check matx s called low densty patycheck code (LDPC). The spase popety of LDPC gves se to ts algothmc advantages. An LDPC code s sad to be egula f w c s constant fo evey column, w s constant fo evey ow and n w wc. An LDPC whch s not egula s called m egula. c 6 c 7 4

5 III. DECODING Dffeent authos come up ndependently wth moe o less the same teatve decodng algothm. They call t dffeent names: the sum-poduct algothm, the belef popagaton algothm, and the message passng algothm. Thee ae two devatons of ths algothm: had-decson and soft-decson schemes. A. Had-decson Decode Fgue : Belef popagaton example code In [8], Lene uses a (4, 8) lnea block code to llustate the had-decson decode. The code s epesented n Fgue, ts coespondng patycheck matx s H (5) An eo fee codewod of H s c [ ] T. Suppose we eceve y [ ] T. So c was flpped. The algothm s as follow:. In the fst step, all message nodes send a message to the connected check nodes. In ths case, the message s the bt they beleve 5 to be coect fo them. Fo example, message node c eceves a (y ), so t sends a message contanng to check nodes f and f. Table 3 llustates ths step.. In the second step, evey check nodes calculate a esponse to the connected message nodes usng the messages they eceve fom step. The esponse message n ths case s the value ( o ) that the check node beleves the message node has based on the nfomaton of othe message nodes connected to that check node. Ths esponse s calculated usng the paty-check equatons whch foce all message nodes connect to a patcula check node to sum to (mod ). In Table 3, check node f eceves fom c 4, fom c 5, fom c 8 thus t beleves c has (+++), and sends that nfomaton back to c. Smlaly, t eceves fom c, fom c 4, fom c 8 thus t beleves c 5 has (+++), and sends back to c 5. At ths pont, f all the equatons at all check nodes ae satsfed, meanng the values that the check nodes calculate match the values they eceve, the algothm temnates. If not, we move on to step In ths step, the message nodes use the messages they get fom the check nodes to decde f the bt at the poston s a o a by maoty ule. The message nodes then send ths had-decson to the connected check nodes. Table 4 llustates ths step. To make t clea, let us look at message node c. It eceves s fom check nodes f and f. Togethe wth what t aleady has y, t decdes that ts eal value s. It then sends ths nfomaton back to check nodes f and f. 4. Repeat step untl ethe ext at step o a cetan numbe of teatons has been passed.

6 In ths example, the algothm temnates ght afte the fst teaton as all patycheck equatons have been satsfed. c s coected to. TABLE 3 CHECK NODES ACTIVITIES FOR HARD-DECISION DECODER FOR CODE OF FIGURE check nodes actvtes f eceve c c 4 c 5 c 8 send c c 4 c 5 c 8 f eceve c c c 3 c 6 send c c c 3 c 6 f 3 eceve c 3 c 6 c 7 c 8 send c 3 c 6 c 7 c 8 f 4 eceve c c 4 c 5 c 7 send c c 4 c 5 c 7 TABLE 4 MESSAGE NODES DECISIONS FOR HARD- DECISION DECODER FOR CODE OF FIGURE message nodes y messages fom check nodes decson c f f 4 c f f c 3 f f 3 c 4 f f 4 c 5 f f 4 c 6 f f 3 c 7 f 3 f 4 c 8 f f 3 B. Soft-decson Decode The soft-decson decode opeates wth the same pncple as the had-decson decode, except that the messages ae the condtonal pobablty that the eceved bt s a o a gven the eceved vecto y. P P c y be the condtonal Let [ ] pobablty that c s a gven the value of y. We P c y P. have [ ] Let (l ) q be the message sent by message node c to check node f at ound l. Evey message contans a pa (l ) q () and (l ) q () whch stands fo the amount of belef that y s o, ( ) () + l () q q (). In patcula, q () P q () () P Smlaly, let. and (l) be the message sent by check node f to message node c at ound l. Evey message contans a pa (l) () and (l) () whch stands fo the amount of belef that y s o. We also have ( l ) ( ) () + l (). (l) () s also the pobablty that thee ae an even numbe of s on all othe message nodes athe than c. Fst, let consde the pobablty that thee ae an even numbe of s on message nodes. Let q be the pobablty that thee s a at message node c and q be the pobablty that thee s a at message node c. We have P[ c c ] qq + ( q )( q ) q + q q q ( q q + 4qq ) [+ ( q )( q)] q (6) Now consde the pobablty that thee ae an even numbe of s on 3 message nodes, c, c and c 3. Note that q s the pobablty that thee ae an odd numbe of s on c and c. P[( c c) c3 ] [+ ( ( q))( q 3 )] 6

7 [+ ( q )( q)( q3)] (7) In geneal P[ c (8) n... c n ] + ( q ) Theefoe, the message that f sends to c at ound l s () + ' V ( q ( l ) ' ()) (9) () () () whee V s the set of all message nodes connected to check node f. The message that c sends to f at ound l s q ( l ) ( ) k ( P ) () () ' ' C q ( l ) ' C ( ) k P () () ' whee C s the set of all check nodes connected to message node c. The constant k s chosen so that q () + q () At each message node, the followng calculatons ae made Q () k ( P ) () C C ( l ) Q () k P () (3) (4) (l ) Q s the effectve pobablty of and at ( l ) message node c at ound l. If Q () > Q () then the estmaton at ths pont s c, othewse c. If ths estmaton satsfes the paty-check equatons then the algothm temnates. Else, the algothm uns though a pedetemned numbe of teatons. As seen above, ths algothm uses a lot of multplcatons whch ae costly to mplement. Anothe appoach s to use logathmc lkelhood ato. Let P[ c L P[ c y] P y] P (5) P[ c y] l ln L ln (6) P[ c y] L s the lkelhood ato and l s the log lkelhood ato at message node c. Usng log ato tuns multplcatons nto addtons whch ae much cheape to mplement n hadwae. Wth log lkelhood ato, we have P + L (7) Fom () and (), the message that c sends to f at ound l s m ( l ) ( l ) q () P ln ln q () P ' C l + ' C ( l ) ( l ) ' ( l ) ' () () m ' (8) Fom (9) and (), the message that f sends to c at ound l s m ln ( l ) () ln () + ' V ' V ' V We have (9) because fom (8), e ( q ( q ( l ) ' ( l ) ' ()) ()) ( l ) m ' + tanh( ) ' V ln (9) ( l ) m' tanh( ) m ' q' () () q () ' 7

8 Thus and q ' () () m ' + e m ' e m' q' () tanh( ) () m ' e + Equaton (3), (4) tun nto l Q () () ln l + l m ( ) (3) Q () If l > then c else c. C In pactce, belef popagaton s executed fo a maxmum numbe of teatons o untl the passed lkelhoods ae closed to cetanty, whcheve comes fst. A cetan lkelhood s l ±, whee P fo l and P fo l -. One vey mpotant aspect of belef popagaton s that ts unnng tme s lnea to the code length. Snce the algothm taveses between check nodes and message nodes, and the gaph s spase, the numbe of tavesals s small. Moeove, f the algothm uns a fxed numbe of teatons then each edge s tavesed a fxed numbe of tmes, thus the numbe of opeatons s fxed and only depends on the numbe of edges. If we let the numbe of check nodes and message nodes nceases lnealy wth the code length, the numbe of opeatons pefomed by belef popagaton also nceases lnealy wth the code length. C. Pefomance of Belef Popagaton A paamete to measue the pefomance of belef popagaton s the expected facton of ncoect messages passed at the lth teaton, P n e (l). In [9], Rchadson and Ubanke show that. Fo any δ >, the pobablty that the actual facton of ncoect messages passed among any nstance at ound l that les outsde n n ( Pe δ, Pe + δ ) conveges to zeo exponentally fast wth n. 8 n. lm P P, whee P e (l) n e e s the expected facton of ncoect messages passed at ound l assumng that the gaph does not contan any cycle of length l o less. The assumpton s to ensue that the decodng neghbohoods become tee-lke so that the messages ae ndependent fo l ounds []. The value P e (l) can be calculated by a method called densty evoluton. Fo a message alphabet of sze q, P e (l) can be expessed by means of q coupled ecusve functons. 3. Thee exsts a channel paamete σ* wth the followng popety: f σ < σ* then lm, else f σ > σ* then thee P e l exsts a constant γ(σ) > such that P e (l) > γ(σ) fo all l. Hee σ s the nose vaance n the channel. In othe wods, σ* sets the lmt to whch belef popagaton decodes successfully. IV. ENCODING If the geneato matx G of a lnea block code s known then encodng can be done usng equaton (). The cost (numbe of opeatons) of ths method depends on the Hammng weghts (numbe of s) of the bass vectos of G. If the vectos ae dense, the cost of encodng usng ths method s popotonal to n. Ths cost becomes lnea wth n f G s spase. Howeve, LDPC s gven by the null space of a spase paty-check matx H. It s unlkely that the geneato matx G wll also be spase. Theefoe the staghtfowad method of encodng LDPC would eque numbe of opeatons popotonal to n. Ths s too slow fo most pactcal applcatons. Theefoe t s desable to have encodng algothms that un n lnea tme. Ths secton wll look at two appoaches to acheve that goal. A. Accumulate appoach The fst appoach modfes LDPC code so t has

9 an nheted fast encodng algothm. In ths case, we assgn a value to each check node whch s equal to the sum of all message nodes that ae connected to t. (It would be moe appopate to talk about nfomaton nodes and edundant nodes, but fo consstence of notaton, I wll use message nodes and check nodes.) The numbe of summatons needed to calculate the value of a check node s bounded by the numbe of message nodes connected to a check node. Ths s a constant when the code s spase. The message conssts of the message nodes appended by the values of the check nodes. To llustate the dffeence between ths modfed veson of LDPC and the ognal veson, consde Fgue. If Fgue epesents an ognal LDPC then c, c, c 3, c 4 ae nfomaton bts and c 5, c 6, c 7, c 8 ae paty bts whch have to be calculated fom c, c, c 3, c 4 by solvng the paty-check equatons n f, f, f 3, f 4. The code ate s 4/8 /. Now f Fgue epesents a modfed LDPC, then all of c, c, c 3, c 4,c 5, c 6, c 7, c 8 ae nfomaton bts; whle f, f, f 3, f 4 ae edundant bts calculated fom c,,c 8. f s connected to c, c 4, c 5, c 8 so f c + c 4 + c 5 + c 8 and so on. The codewod n ths case s [c c 8 f f 4 ] T. The code ate s 8/ /3. Although ths appoach gves a lnea encode, t causes a mao poblem at the decode. In case the channel s easue, the value of the check nodes mght be eased. On the contay, the check nodes of the ognal LDPC ae dependences, not values, thus they ae not affected by the channel. In othe wods, a check node defnes a elatonshp of ts connected message nodes. Ths elatonshp comes staght fom the paty-check matx. The fact that n modfed LDPC, the values of check nodes can be eased ceates a lowe bounded fo the convegence of any decodng algothm. In [4], Shokollah poves the exstence of such lowe bound. Suppose that the channel s easue wth the easue pobablty p. Then an expected p-facton of the message nodes and an expected p-facton of the check nodes wll be eased. Let M d be the facton of message nodes of degee d (connected wth d check nodes.) The pobablty of a message node of k degee d havng all ts connected check nodes eased s p d. Ths pobablty s condtoned on the event that the degee of the message node s d. Snce the gaph s ceated andomly, the pobablty that a message node has all ts connected check nodes eased s d M p d d, whch s a constant ndependent of the length of the code. Theefoe, no algothm can ecove the value of that message node. B. Lowe tangula modfcaton appoach In [], Rchadson and Ubanke popose an encodng algothm that has effectvely lnea unnng tme fo any code wth a spase paty-check matx. The algothm conssts of two phases: pepocessng and encodng. In the pepocessng phase, H s conveted nto the fom shown n Fgue 3 by ow and column pemutatons. n - k A Fgue 3: Paty-check matx n appoxmately lowe tangula fom In matx notaton, g B k - g C D E n k - g A B T H (4) C D E whee T has a lowe tangula fom wth all dagonal entes equal to. Snce the opeaton s done by ow and column pemutatons and H s spase, A, B, C, D, E, T ae also spase. g, the gap, measues how close H can be made, by ow and column pemutatons, to a lowe tangula matx. T g 9

10 TABLE 5 COMPUTING p USING RICHARDSON AND URBANKE S ENCODING ALGORITHM Opeaton Comment Complexty As T Multplcaton by spase matx O(n) T [As T ] Back-substtuton, T s lowe tangula O(n) E[T As T ] Multplcaton by spase matx O(n) Cs T Multplcaton by spase matx O(n) [ ET As T ]+ [Cs T ] Addton O(n) Φ ( ET As T + Cs T ) Multplcaton by g g matx O(n+g ) TABLE 6 COMPUTING p USING RICHARDSON AND URBANKE S ENCODING ALGORITHM Opeaton Comment Complexty As T Multplcaton by spase matx O(n) T Bp Multplcaton by spase matx O(n) [As T ] + [Bp T ] Addton O(n) T (As T + Bp T ) Back-substtuton, T s lowe tangula O(n) Multple H fom the left by we get I (5) ET I A B T (6) ET A+ C ET B+ D Let the codewod c (s, p, p ) whee s s the nfomaton bts, p and p ae the paty-check bts, p has length g, p has length k g. By Hc T, we have I ET Hc I T s A B T p ET A C ET B D (7) + + p Theefoe As T + Bp T + Tp T (8) ( ET A + C)s T + ( ET B + D)p T (9) The pocedue to fnd p and p s summazed n Table 5 and 6. Defne: Φ ET B + D and assume fo the moment that Φ s nonsngula. Then p T Φ ( ET A + C)s T Φ ( ET As T + Cs T ) (3) Fst we compute As T. Snce A s spase, ths s done n lnea tme O(n). Then we compute T [As T ] y T. Snce [As T ] Ty T and T s lowe tangula, by back-substtuton we can compute y T n lnea tme. The calculatons Ey T and Cs T ae also done n O(n) as E, C ae spase. Now we have ( ET As T + Cs T ) z T computed n O(n). Snce Φ s g g, p s computed fom (3) n O(n+g ). Fom (8), p T T (As T + Bp T ). The steps to calculate p ae qute smla and ae shown n Table 6. We see that p can be computed n O(n). As seen n Table 5 and Table 6, c can be computed n O(n+g ). Rchadson and Ubanke

11 pove n [] that the gap g concentates aound ts expected value, αn, wth hgh pobablty. α hee s a small constant. Fo a egula LDPC wth ow weght w 6, column weght w c 3, α.7. Theefoe even though mathematcally the encodng algothm un n O(n ) (α O(n ) to be pecse), n pactce the encode stll uns n easonable tme fo n,. In the same pape, Rchadson and Ubanke also show that fo known optmzed codes, the expected g s bounded by O( encode uns n O(n). V. IRREGULAR CODES n ) thus the It has been shown that egula LDPC pefom bette than egula LDPC [], [3], [4]. The dea was poneeed by Luby et al n []. He thnks of fndng coeffcents fo an egula code as a game, wth the message nodes and check nodes as playes. Each playe tes to choose the ght numbe of edges fo them. A constant of the game s that the message nodes and the check nodes must agee on the total numbe of edges. Fom the pont of vew of the message nodes, t s best to have hgh degee snce the moe nfomaton t has fom the check nodes, the moe accuately t can udge what ts coect value should be. On the othe hand, fom the pont of vew of the check nodes, t s best to have low degee, snce the lowe the degee of a check node, the moe valuable the nfomaton t can tansmt back to the message nodes. These two equements must be appopately balanced to have a good code. MacKay shows n [5], [6] that fo egula codes, t s best to have low densty. Howeve, allowng egula codes povdes anothe degee of feedom. In [], Luby shows that havng a wde spead of degee s advantageous, at least fo the message nodes. The eason s message nodes wth hgh degee tend to coect the value faste. These nodes then povde good nfomaton to the check nodes, whch subsequently povde bette nfomaton to the lowe degee message nodes. Theefoe egula gaph has potental to povde a wave effect whee hgh degee message nodes ae coected fst, followed by slghtly smalle degee nodes, and so on. Befoe gettng nto the detals of how to constuct egula codes, let us ntoduce some notatons. Let d l, d be the maxmum degees of message nodes and check nodes. Defne the left (ght) degee of an edge to be the degee of the message node (check node) that s connected to the edge. λ (ρ ) s the facton of edges wth left (ght) degee. Any LDPC gaph s specfed by the sequences λ,..., ) and ρ,..., ). Futhe, defne and ( λ d l x x ) ( ρ d λ ( λ (3) x x ) ρ ( ρ (3) to be the degee dstbuton of message nodes and check nodes. Also, defne p to be the pobablty that an ncoect message s passed n the th teaton. Now consde a pa of message node and check node (m, c) and let c be anothe check node of m dffeent than c. At the end of the th teaton, c wll send m ts coect value f thee ae an even numbe (ncludng ) of message nodes othe than m sendng c the ncoect bt. By an analogous analyss to equaton (8), the pobablty that c eceves an even numbe of eos s d + ( ) fo the case of unvayng degees of the nodes, and p +ρ( p ) (33) (34) fo the case of vayng degees of the nodes, whee ρ(x) s defned n (3). A. Luby s desgn In [], Luby poves the teatve descpton of p.

12 p p d l + λ. p + ρ( p t t b, ρ( + p ) p ) t t ) + ρ( p t ( t b, ρ(. p ) t t ) (35) Note that p s the eo pobablty of the channel. b, s gven by the smallest ntege the satsfes p p + ρ( p ) ρ( p ) b, + (36) The goal of ths desgn s to fnd sequences λ λ,..., ) and ρ ρ,..., ) that yeld the ( λ d l ( ρ d bggest value of p such that the sequence {p } deceases to. Defne So p + f(p ), theefoe we want f(x) < x. Anothe constant to λ and ρ s l λl ρ ( R) (38) l Equaton (38) makes sue that the total numbe of left and ght degees ae the same. Luby s appoach tes to fnd any sequence λ that satsfes (38) and f(x) < x fo x, p ). It accomplshes ( ths task by examnng the condtons at x p / fo some ntege. By pluggng those values of x nto (37), t ceates a system of lnea nequaltes. The algothm fnds any λ that satsfes ths lnea system. As seen, Luby s appoach cannot detemne the best sequence λ and ρ. Instead, t detemnes a good vecto λ gven a vecto ρ and a desed code ate R. Luby shows though smulatons that the best codes have constant ρ, n othe wods, the check nodes have the same degee. Some esults fom [6] s epoduced n Table 7. TABLE 7 LUBY S IRREGULAR CODES d l f ( x) p λ. p + ρ( x) t t b, t ρ( x) t + p ) + ρ( x t ( t b, ρ( x). t t ) (37) p* n Table 7 s the maxmum value of p

13 acheved by each code. All of the code above have code ate R /. Pevously, the best p* fo Gallage s egula codes wth code ate / s.57 []. B. Rchadson s desgn In [3], Rchadson, Shokollah and Ubanke popose a desgn of egula LDPC that can appoach Shannon channel capacty tghte than tubo code. The algothm employs two optmzatons: toleate the eo floo fo pactcal pupose and caefully desgn quantzaton of densty evoluton to match the quantzaton of messages passed. The dea of the fome optmzaton s that n pactce, we always allow a fnte (but small) pobablty of eo ε. If we choose ε small enough then t automatcally mples convegence. The latte optmzaton makes sue that the pefomance loss due to quantzaton eos s mnmzed. Snce belef popagaton s optmal, the quantzed veson s suboptmal, theefoe the smulaton esults can be thought of as lowe bound fo actual values. Rchadson s algothm stats wth an abtay degee dstbuton (λ, ρ). It sets the taget eo pobablty ε and the maxmum numbe of teatons m. The algothm seaches fo the maxmum admssble channel paamete such that belef popagaton etuns a pobablty of eo less than ε afte m teatons. Now slghtly change the degee dstbuton pa and uns the algothm agan and check f a lage admssble channel paamete o a lowe pobablty of eo s found. If yes then set the cuent dstbuton pa to the new dstbuton pa, else keep the ognal pa. Ths pocess s epeated a lage numbe of tmes. The basc of ths algothm s that Rchadson notces the exstence of stable egons whee the pobablty of eo does not decease much wth the ncease numbe of teatons. Ths fact helps lmt the seach space of the degee dstbuton thus shotens the unnng tme. Anothe optmzaton n Rchadson s algothm s the fact that he lets the degee to be a contnuous vaable, and ound t to etun to eal ntege 3 degee. The eason why ths optmzaton woks s because t suts Dffeental Evoluton well. The detal of Dffeental Evoluton s dscussed n [7]. Some esults fom [3] s epoduced n Table 8. TABLE 8 RICHARDSON S IRREGULAR CODES d v s the maxmum message node degee, fo each d v, the ndvdual degee facton s povded. σ* s the channel paamete dscussed n Secton III C. p* s the nput bt eo pobablty of a haddecson decode. All codes have ate /. C. Chung s desgn In hs PhD dssetaton, Chung ntoduces a devaton of densty evoluton called dscetzed densty evoluton. Ths devaton s clamed to model exactly the behavo of dscetzed belef popagaton. In hs lette [4], Chung ntoduces an egula code whch s wthn.45 db of Shannon capacty. Ths code has d v 8, whch s

14 much geate than the maxmum message node degee studed by Luby and Rchadson. Chung s code s the closest code to Shannon capacty that has been smulated. It futhe confms that LDPC ndeed appoaches channel capacty. VI. CONCLUSION Ths pape summaes the mpotant concepts egadng low densty paty-check code (LDPC). It goes though the motvaton of LDPC and how LDPC can be encoded and decoded. Dffeent modfcatons of the codes ae pesented, especally egula codes. I chose to leave out devatons of egula codes such as MacKay codes [8], epeataccumulate codes [9] because they have become less mpotant wth the advance of egula codes. Ths pape howeve has not mentoned how LDPC s mplemented n eal hadwae. Fo ths, I efe the eades to [], whee a decode desgn based on IEEE 8.n standads wth vey hgh thoughput (9Mbps fo FPGA, Gbps fo ASIC desgn) s dscussed. REFERENCES [] R. Gallage, Low densty paty-check codes, IRE Tans, Infomaton Theoy, pp. -8. Januay 96. [] R. M. Tanne, A ecusve appoach to low complexty codes, IEEE Tans. Infomaton Theoy, pp , Septembe 98. [3] D. Mackay and R. Neal, Good codes based on vey spase matces, Cyptogaphy and Codng, 5 th IMA Conf., C. Boyd, Ed., Lectue otes n Compute Scence, pp. -, Beln, Gemany, 995. [4] D. Mackay, Good eo coectng codes based on vey spase matces, IEEE Tans. Infomaton Theoy, pp , Mach 999. [5] N. Alon and M. Luby, A lnea tme easueeslent code wth nealy optmal ecovey, IEEE Tans. Infomaton Theoy, vol. 47, pp , Febuay. [6] Infomaton and Entopy, MIT OpenCouseWae. Spng 8. [7] Wkpeda. Hammng(7,4). Accessed May, 9. [8] B. M. J. Lene, LDPC Codes a bef Tutoal, Apl 5. [9] T. Rchadson and R. Ubanke, The capacty of low-densty paty check codes unde messagepassng decodng, IEEE Tans. Infom. Theoy, vol. 47, pp ,. [] A. Shokollah, LDPC Codes: An Intoducton, Dgtal Fountan, Inc., Apl, 3. [] T. Rchadson and R. Ubanke, Effcent encodng of low-densty paty-check codes," IEEE Tans. Infom. Theoy, vol. 47, pp ,. [] M. Luby, M. Mtzenmache, A. Shokollah, and D. Spelman, Impoved Low-Densty Paty-Check Codes Usng Iegula Gaphs," IEEE Tans. Infom. Theoy, vol. 47, pp ,. [3] T. Rchadson, A. Shokollah, and R. Ubanke, Desgn of capacty-appoachng egula low-densty paty-check codes," IEEE Tans. Infom. Theoy, vol. 47, pp ,. [4] S.-Y. Chung, D. Foney, T. Rchadson, and R. Ubanke, On the desgn of low-densty paty-check codes wthn.45 db of the Shannon lmt," IEEE Communcaton Lettes, vol. 5, pp. 58 6,. [5] D. J. C. MacKay, Good eo coectng codes based on vey spase matces, IEEE Tans. Infom. Theoy, vol. 45, pp , Ma [6] D. J. C. MacKay and R. M. Neal, Nea Shannon lmt pefomance of low-densty paty-check codes, Electon. Lett., vol. 3, pp ,

15 [7] K. Pce and R. Ston, Dffeental evoluton A smple and effcent heustc fo global optmzaton ove contnuous spaces, J. Global Optmz., vol., pp , 997. [8] D. Mackay, Infomaton Theoy, Intefeence, and Leanng Algothms, Cambdge Unvesty Pess 3. [9] D.Dvsala, H. Jn and R. McElece, Codng theoems fo tubo-lke codes, Poc. 36th Annual Alleton Conf. on Comm., Contol and Conputngce, pp. -. Septembe 998. [] Maan Kakoot, Pedag Radosavlevc and Joseph R. Cavallao, Confguable, Hgh Thoughput, Iegula LDPC Decode Achtectue: Tadeoff Analyss and Implementaton, Rce Dgtal Scholashp Achve, Septembe, 6. 5