Joint Decoding of Content-Replication Codes for Flash Memories

Transcription

1 Ffty-thrd Annual Allerton Conference Allerton House, UIUC, Illnos, USA Setember 29 - October 2, 2015 Jont Decodng of Content-Relcaton Codes for Flash Memores Qng L, Huan Chang, Anxao (Andrew) Jang, and Erch F. Haratsch Seagate Technology, San Jose, CA, Comuter Sc. and Eng. Det.,Texas A&M Unversty, College Staton, TX, {qngl, hchang, ajang }@cse.tamu.edu Abstract One serous challenge for flash memores s data relablty. In ths work, we resent the contentrelcaton codeword roblem, and t leads to our roosed jont decoder. We focus on jont decodng algorthms and study ther theoretcal decodng erformances. The roosed scheme s novel for flash memores, and we show ther relablty can be enhanced by ncreasng the dversty of error-correctng codes. I. INTRODUCTION One challenge for flash memores s the data relablty as several tyes of nose [1], [5] exst. Besdes strong error correctng codes, e.g., LDPC codes, another mechansm to rotect flash memores s memory scrubbng [10],.e., whle errors accumulate n a codeword, wth the next block erasure, the codeword s corrected and a new error-free codeword s wrtten back to the memory. However, n flash memory rewrtes are made n an out-of-lace fashon,.e., an udated codeword s stored at a new hyscal address and the orgnal codeword remans n the memory. Those mechansms can lead to multle coes of codewords,.e., the contentrelcated codeword roblem. In addton to memory scrubbng, other factors also may cause the contentrelcaton roblem such as garbage collecton, wearlevelng, etc, and t s estmated that on average 3 13 (.e., the exact number deends on the workload traffc and varous Flash Translaton Layer algorthms [2] used) coes of content-relcated codewords can be generated [3]. In ths work, we enhance flash memory relablty by utlzng the exstence of two content-relcated codewords for decodng, ncludng an old codeword and a new codeword storng the same nformaton. We am at desgnng a jont decodng scheme havng access to both content-relcated codewords, and exlore ts decodng erformance. Ths leads to relablty mrovement n flash memores. We further study a new aradgm where the two content-relcated codewords have dfferent forms for better erformance. The sgnfcance of ths aer s two-fold: on the ractcal sde, the new codng scheme utlzes the unque roertes of flash memores; on the theoretcal sde, we show that ncreasng the dversty of error-correctng codes n the storage system can mrove the relablty of relcated data even f there exst constrants n ther jont decodng algorthms. II. PROBLEM STATEMENT Let D = {0, 1,, M 1} be the message set for M N, and let X and Y be two alhabets of the symbols stored n a cell. Let two encoders be f 1 : D X N and f 2 : D X N, and the desred jont decoder be h : Y N Y N D, where N s the length of codewords. Let P = (X, Y, P Y X ) and Q = (X, Y, Q Y X ) be two ndeendent channels. We llustrate the model n Fg. 1. Here, m s a common message to both encoders, the N-dmensonal vectors x N 1 0 (1), x0 N 1 (2) X N are two codewords obtaned through two encoders (those encoders are not necessarly dentcal), and y0 N 1 (1), y0 N 1 (2) are two nosy codewords through P and Q. The task s to desgn a jont decoder to gve a relable estmaton of the message m, whch s denoted as ˆm, gvng y0 N 1 (1) and y0 N 1 (2). The roblem statement s resented below: Defnton 1. Gven two (N, 2 NR ) error-correctng codes, a message set D = {0, 1,, 2 NR 1}, ther encodng functons f 1 : D X N and f 2 : D X N, and two ndeendent channels P and Q, the task s to desgn a jont decodng scheme h : Y N Y N D such that P r(h(y0 N 1 (1), y0 N 1 (2)) x N 1 0 (1) = f 1 (), x0 N 1 (2) = f 2 ())) 0 for D as N. We ont out two mlct requrements for the jont decoder n the above defnton: the frst s the rate of the gven code should be larger than the caactes of underlyng two channels,.e., R > C(P) and R > C(Q), therefore relable decodng s mossble for searate decoders,.e., g 1 : Y N D and g 2 : Y N D such that P r(g 1 (y0 N 1 (1)) or g 2 (y0 N 1 (2)) x0 N 1 (1) = f 1 (), x N 1 0 (2) = f 2 ())) 0 for D as N. Otherwse, the jont decoder degenerates to the searate decoder n channel codng model; the second requrement s for the gven encoders, when channels are not degradng too much, relable searate decoders exst. More recsely, gven the same arameters N, R, we requre f 1 ( ) and f 2 ( ) meet the condton that when R < C(P 1 ) and R < C(Q 1 ) for some P 1 and Q 1, /15/$ IEEE 712

2 there exst g 1 : Y N D and g 2 : Y N D such that P r(g 1 (y0 N 1 (1)), g 2 (y0 N 1 (2)) x0 N 1 (1) = f 1 (), x N 1 0 (2) = f 2 ())) 0 for D as N. The above requrements are due to the motvatons of jont decoders: the jont decoder s not to relace exstng ndvdual decoders (as t s ossble that ndvdual decoders suffce to relably decode when channels do not degrade too much, and also the content-relcated codewords cannot always be guaranteed to exst) but to relace ndvdual decoders when they fal. It s also those requrements that dfferentate the jont decoder from other codng models lke Multle Access Channels wth correlated sources by Slean and Wolf [11] and Fountan code [6]. In the followng, we assume P and Q are dentcal Bnary Erasure Channels (BEC) n Secton III and dentcal Addtve Whte Gaussan Nose (AWGN) channels n Secton IV, and both encoders are systematc LDPC encoders. The followng notatons wll be used: let the rate of two systematc LDPC codes be K N, let G 1, G 2 be the encodng matrces, and H 1, H 2 denotes ther arty check matrces. Let y0 N 1 (1), y0 N 1 (2) {0, 1,?} N be two codewords receved for BECs and let y0 N 1 (1), y0 N 1 (2) R N also be those for AWGN channels. Fg. 1. Illustraton of jont decodng content-relcated codewords. III. JOINT DECODER FOR BECS In ths secton, we resent several jont decoder desgns when P and Q are Bnary Erasure Channels wth the same arameter. A. Jont decoder for dentcal content-relcated codes The gven codes are dentcal n ths case,.e., G 1 = G 2 and H 1 = H 2. Gven y0 N 1 (1) and y0 N 1 (2), a combned codeword y0 N 1 s obtaned as follows, for = 0, 1,, N 1, y =? f y (1) = y (2) =?, y (1) f y (2) =? and y (1)?, y (2) else The arty check matrx for y0 N 1 s H 1. The decodng result s obtaned by alyng belef roagaton to y0 N 1 wth H 1 and ntal erasure robablty ɛ 2. Let λ(x) and ρ(x) be degree dstrbutons for the LDPC codes used, let ɛ BP (λ, ρ) be ts orgnal threshold as n [8], and ɛ BP den (λ, ρ) be the threshold for our jont decoder. The comarson of ɛ BP den (λ, ρ) and ɛbp (λ, ρ) for some regular LDPC codes s resented n the second and the thrd columns of Table I, and we have ɛ BP den > ɛbp. Note that the above scheme can be generalzed to cases when P and Q are wth dfferent ɛ, and due to sace lmtaton we do not resent that here. TABLE I COMPARISON OF ɛ BP, ɛ BP den AND ɛbp df (d v, d c ) ɛ BP ɛ BP den ɛ BP df (3,4) (3,5) (3,6) (4,6) (4,8) B. Jont decoder of dfferent content-relcated codes In the above subsecton, the two codes are dentcal, whch are effectvely reetton codes, and ths motvates us to exlore another jont decoder desgn when the two encoders are dfferent. 1) Jont decoder desgn: The gven codes are dfferent n ths case,.e., G 1 G 2 and H 1 H 2, but codewords carry dentcal systematc nformaton bts, that s, two encodng functons are x N 1 0 (1) = u0 K 1 G 1 and x0 N 1 (2) = u K 1 0 G 2. Let I 1, I 2 {0, 1,, N 1} be the nformaton bt ndex sets for y0 N 1 (1) and y0 N 1 (2), and let P 1 and P 2 be ther arty check bt ndex sets. Let y0 N 1 (1) I1 = (y (1) : I 1 ),.e., nformaton bts of y0 N 1 (1), and smlar notatons aly to y0 N 1 (2) I2, y0 N 1 (1) P1 and y0 N 1 (2) P2. Let g( ) : I 1 I 2 be a one-to-one mang such that x (1) = x g() (2) for I 1. Smlar to the revous secton, we defne (y0 N 1 ) I1, where y =? f y (1) = y g() (2) =?, y (1) f y g() (2) =? and y (1)?, y g() (2) else Then, a constructed combned codeword s y0 2N K 1 = [(y0 N 1 ) I1, y0 N 1 (1) P1, y0 N 1 (2) P2 ]. That s, y0 2N K 1 s constructed by aendng nformaton bts from y0 N 1 (1) and y0 N 1 (2) after rerocessng, and arty check bts from y0 N 1 (1) and y0 N 1 (2). Let H 1 = [H 1,0, H 1,1,, H 1,N 1 ], let H 1,I1 = [H 1, : I 1 ], and let H 1,P1 = [H 1, : P 1 ]. Smlarly, we dvde H 2 nto H 2,I2 and H 2,P2. Then, the arty check matrx H for y0 2N K 1 s of the form n Fg. 2. An examle for a combned codeword and ts arty check matrx s llustrated n Fg. 3. In Fg. 3, (a) s the Tanner grah and H 1 for y0 N 1 (1), where nformaton bts are black and arty check bts are red; (b) s the Tanner grah and H 2 for y0 N 1 (2), where nformaton bts are black and arty check bts are green; (c) s the 713

3 Fg. 2. Illustraton of the arty check matrx H. Fg. 3. Illustraton of constructed y 2N K 1 0 and H. constructed Tanner grah and H based on (a) and (b), where nformaton bts are black, arty check bts from y0 N 1 (1) are red, and arty check bts from y0 N 1 (2) are green. The decodng result s obtaned by alyng belef roagaton to y0 2N K 1 wth H, the ntal erasure robablty ɛ 2 for (y0 N 1 ) I1, and ɛ for y0 N 1 (1) P1 and y0 N 1 (2) P2. 2) Performance analyss by densty evoluton: a) Notatons: In a Tanner grah of an LDPC code, for an edge f ts one end connects to an nformaton bt of varable nodes, we call t nformaton edge; f t connects to a arty check bt of varable nodes, we call t arty edge. For examle, n Fg. 3 (c), the edges connectng to c 0, c 1, c 2, c 3 are nformaton edges and the remanng edges are arty edges. For nformaton edges (res. arty edges), let λ () ) be the fracton of edges connectng to an (res. λ () varable node wth degree. Let λ () (x) = where d v =1 λ () d v =1 λ () = 1, and λ () (x) = d v =1 d v =1 λ () x 1, λ () x 1, where = 1, be the degree dstrbuton functons from the edge ersectve. For examle, λ () (x) = 6 15 x x x5 and λ () (x) = 1 n Fg. 3 (c). Let ρ be the fracton of edges connectng to a check node wth degree j + k, of whch j edges are nformaton edges and k edges are arty edges. Let ρ(x, y) = ρ x j y k, where ρ = 1, denote the edge degree dstrbuton functons from the check node ersectve. For examle, ρ(x, y) = x3 y x2 y n Fg. 3 (c). Let ρ () ρ = 1 ρ 0,j+k and ρ () ρ = 1 ρ j+k,0, let ρ () (y, x) = ρ () xj 1 y k where ρ () = 1 and j 1, k 0, and ρ () (x, y) = ρ () xj y k 1 = 1 and j 0, k 1. For examle, where ρ () ρ () (x, y) = x2 y xy and ρ() (x, y) = x x n Fg. 3 (c), where ρ () (x, y) haens to be the same as ρ () (x, y) for ths examle. b) Edge degree dstrbutons: Lemma 2. Gven two regular (d v, d c ) LDPC codes (whch are not necessarly the same), the edge degree dstrbutons of constructed the combned LDPC code are: λ () (x) = x 2dv 1, λ () (x) = x dv 1, and ρ = ( j+k j ) ( d c d v d c ) j ( d v d c ) k, where j + k = d c. Proof: Based on the constructon resented, for the Tanner grah of y0 2N K 1, both check nodes of y0 N 1 (1) and y0 N 1 (2) connect to nformaton bts of varable nodes of y0 2N K 1, thus those node degrees are doubled; The degree of arty check bt of varables nodes of y0 2N K 1 remans the same as those of y0 N 1 (1) and y0 N 1 (2). The result for ρ follows from that for a random edge t s an nformaton edge wth robablty d c d v d c, a arty edge wth robablty d v d c, and the robablty dstrbuton that j out of j + k edges are from nformaton edges s a bnomal dstrbuton. c) Densty evoluton: From Lemma 2, we know that λ () (x) and λ () (x) are not dentcal, the ntal effectve erasure robablty s ɛ 2 for nformaton bts of y0 2N K 1 and ɛ for arty bts of y0 2N K 1, thus the robabltes of a arty bt and an nformaton bt beng an erasure at the l-round of belef roagaton decodng are not the same (we show ths ont n Fg.4 through a smulaton wth both (3, 6) LDPC code and ntal erasure robablty 0.6). Let be the average robablty of an nformaton bt of y0 2N K 1 beng an erasure after the l-round of belef roagaton decodng, and smlarly let be that for a arty check bt of y0 2N K 1. Our man result based on densty evoluton [8] s resented below: Theorem 3. For our jont decodng of dfferent contentrelcated codes, the average erasure robabltes after 714

4 Fg. 4. Densty evoluton comarson of nformaton bts and arty bts for jont decoder of dfferent content-relcated (3, 6) LDPC codes wth ntal erasure robablty 0.6. l-round of belef-roagaton decodng are gven by = ɛ 2 λ () (1 ρ () (1 x (l 1), 1 x (l 1) ))), = ɛλ () (1 ρ () (1 x (l 1), 1 x (l 1) ))), where λ () ( ), λ () ( ), ρ () ( ) and ρ () ( ) are for the combned LDPC code. Proof: We break the roof nto two stes. Frst, let y (l) be the average robablty of beng an erasure under belef-roagaton decodng after l rounds for an outut edge from a check node to an nformaton bt of varable node. It s gven by y (l) = ρ () (1 (1 ) j 1 (1 ) k ) = 1 ρ () (1, 1 ). Smlarly, let y (l) be the average robablty of erasure under belef-roagaton decodng after l-round for an outut edge from a check node to a arty check bt of varable node. It s gven by y (l) = 1 ρ () (1, 1 ). Second, the average robablty of erasure for the outut message of an nformaton bt of varable nodes s gven by = ɛ 2 λ () (y (l 1) ) 1 = ɛ 2 λ () (y (l 1) ). Smlarly, the average robablty of erasure for the outut message of an arty check bt of varable nodes s gven by = ɛλ () (y (l 1) ). Combnng the above two stes, we obtan the desred results. The followng theorem resents us the exstence of densty evoluton threshold. Theorem 4. Based on Theorem 3, one sees densty evoluton udates are gven by f (ɛ, x, y) = ɛ 2 λ () (1 ρ () (1 y, 1 x)) and f (ɛ, x, y) = ɛλ () (1 ρ () (1 x, 1 y)). We observe the followng: 1) f (ɛ, x, y) and f (ɛ, x, y) are non-decreasng n all arguments for ɛ, x, y [0, 1] and strctly ncreasng f ɛ, x, y (0, 1). 2) For any x 0, y 0, ɛ [0, 1], the sequence x l+1 = f (ɛ, x l, y l ) and y l+1 = f (ɛ, x l, y l ) are monotonc n l. 3) Let x l+1 (ɛ) and y l+1 (ɛ) be defned recursvely by x l+1 (ɛ) = f (ɛ, x l (ɛ), y l (ɛ)), y l+1 (ɛ) = f (ɛ, x l (ɛ), y l (ɛ)), x 0 (ɛ) = ɛ 2 and y 0 (ɛ) = ɛ. Then, x l+1 (ɛ) and y l+1 (ɛ) are non-decreasng n ɛ. 4) The functon x (ɛ) = lm x l (ɛ) and y (ɛ) = lm (y l(ɛ)) exst and are non-decreasng for all ɛ [0, 1]. d Proof: For 1), we observe that dɛ f (ɛ, x, y) = 2ɛλ () (1 ρ () (1 y, 1 x)) s not negatve for ɛ, x, y [0, 1], and d dɛ f (ɛ, x, y) = λ () (1 ρ () (1 d x, 1 y)) are ostve for x, y [0, 1]. dx f (ɛ, x, y) = ɛ 2 λ () (1 ρ () (1 y, 1 x)))ρ () (1 y, 1 x) s ostve for ɛ, x, y (0, 1) and d dx f (ɛ, x, y) = ɛλ () (1 ρ () (1 x, 1 y))ρ () (1 x, 1 y) s also ostve for ɛ, x, y (0, 1). Smlarly, we can rove d dy f (ɛ, x, y) and d dy f (ɛ, x, y) are also ostve for ɛ, x, y (0, 1). For 2), the monotoncty of f (ɛ, x, y) and f (ɛ, x, y) mles that x l+1 = f (ɛ, x l, y l ) x l and x l+2 = f (ɛ, x l+1, y l+1 ) x l+1. Therefore, monotoncty holds nductvely and the drecton of x l deends only on the frst ste. Smlarly, we can rove y l+1 = f (ɛ, x, y) are monotonc. For 3), we frst observe that x 0 (ɛ) and y 0 (ɛ) are nondecreasng n ɛ. Next, we roceed by nducton, for any ɛ ɛ, to see that x l+1 (ɛ) = f (ɛ, x l (ɛ), y l (ɛ)) f (ɛ, x l (ɛ ), y l (ɛ )) = x l+1 (ɛ ). Smlarly, we can rove that y l+1 (ɛ) s non-decreasng n ɛ. For 4), the lmt exsts because 2) mles the sequence x l (ɛ) s monotonc and bounded for all ɛ [0, 1]. The lmt functon s non-decreasng because 3) mles that, for any ɛ ɛ, we have x (ɛ) = lm x l (ɛ) lm x l(ɛ ) = x (ɛ ). The same rocess ales for the sequence y l (ɛ). Let ɛ BP df (λ(), ρ () ) = su{ɛ [0, 1] : x (ɛ) = 0} (whch s clearly equal to su{ɛ [0, 1] : y (ɛ) = 0}) be the threshold defned by the densty evoluton. We comute ɛ BP, ɛ BP den, ɛbp df, where ɛbp df s based on the recursve functons defned n Theorem 3, for some regular LDPC codes n the fourth column of Table I. Comarng wth revous results, we can see that ɛ BP ɛ BP den s ossble. df > C. Jont decoder of related content-relcated codes 1) Related encoder desgn: Let G 3 be an ntermedate systematc LDPC generator matrx wth rate 1 2. Smlarly, let I and P denote the nformaton bt ndex set and the arty check bt ndex set for codes wth G, = 1, 2, 3. The encodng algorthm s below, where (x N 1 0 ) P3 denotes the subvector (x : P 3 ). 1) f 1 : x0 N 1 (1) = u K 1 0 G

5 Fg. 5. Fg. 6. Illustraton of arty check matrx H Illustraton of constructed y 2N 1 0 and H. 2) v0 K 1 = (u0 K 1 G 3 ) P3. 3) f 2 : x N 1 0 (2) = v0 K 1 G 2, where n the above f 1 and f 2 are the two encoders defned n Defnton 1. That s, (x N 1 0 (1)) I1 and (x N 1 0 (2)) I2 are related through G 3. 2) Jont decoder desgn: A combned codeword s obtaned by assemblng y0 N 1 (1) and y0 N 1 (2) n the followng way, y0 2N 1 = (y0 N 1 (1) P1, y0 N 1 (1) I1, y0 N 1 (2) P2, y0 N 1 (2) I2 ). Let H 3 be the arty check matrx corresondng to G 3. Then, the arty check matrx H for y0 2N 1 s of the form n Fg. 5. An examle for a combned codeword and ts arty check matrx s llustrated n Fg. 6, where (a) s the Tanner grah and H 1 for y0 N 1 (1), where nformaton bts are black and arty check bts are red; (b) s the Tanner grah and H 2 for y0 N 1 (2), where nformaton bts are green and arty check bs are blue; (c) s the Tanner grah and H 3 for v0 K 1, where nformaton bts are black and arty check bts are blue; (d) s the constructed Tanner grah and H for y0 2N 1. The decodng result s obtaned by alyng belef roagaton to y0 2N 1 wth H and ntal erasure robablty ɛ. 3) Performance analyss by densty evoluton: For x N 1 0 (1) and x0 N 1 (2) the two LDPC codes, we use same notatons of revous subsecton, λ () (x), λ () (x), ρ () (x, y) and ρ () (x, y), to denote the edge degree dstrbutons from the varable nodes. For the ntermedate LDPC code, v0 K 1, we use the usual ρ 3 (x) and λ 3 (x) to denote ts edge degree dstrbutons. We have the followng results for densty evoluton, where we use the same notatons,.e., and, as the revous subecton. Theorem 5. For our jont decodng of related contentrelcated codes, the average erasure robabltes after l-round of belef-roagaton decodng are gven by = ɛλ () (1 ρ () (1 x (l 1), 1 x (l 1) ))), = ɛ 2 λ () (1 ρ () (1 x (l 1), 1 x (l 1) )) λ 3 (1 ρ 3 (1 x (l 1) )). where λ () ( ), λ () ( ), ρ () ( ) and ρ () ( ) are for the ndvdual LDPC code, and λ 3 ( ) and ρ 3 ( ) are for the ntermedate LDPC code. Proof: The roof s the smlar to that of the revous theorem, and thus we resent ts sketch as follows. Let y (l) (res. y (l) ) denote the average robablty of beng an erasure after the l th round of belef roagaton decodng for an outut edge from a check node of y0 N 1 (1) or y0 N 1 (2) to an nformaton bt (res. arty check bt) of varable node of y0 2N 1. Clearly, they follow the same formulas as Theorem 3. For ((y0 N 1 (1)) I1, (y0 N 1 (2)) I2 ), let y (l) denote the robablty that the message sent to an varable node s an erasure, and t s easy to know that y (l) = 1 ρ 3 (1 ). Next, we focus on, we know that an nformaton bt of varable nodes receves both messages from arty bts of LDPC codes (ρ 1, λ 1 ) and (ρ 3, λ 3 ), thus ɛ 2 λ () (1 ρ () (1 x (l 1) x (l 1) )). The equaton for, 1 x (l 1) = )) λ 3 (1 ρ 3 (1 remans the same as that of the revous secton, and the followng conclusons hold mmedately. Let us verfy one secal case of Theorem 5: when (d v, d c ) s (1, 2) regular LDPC code, t should degenerate to the dfferent content-relcated code case of Theorem 3, and ths result concdes wth ths ont. Smlarly, we obtan the followng convergence results: Theorem 6. Based on Theorem 5, one sees densty evoluton udates are gven by f (ɛ, x, y) = ɛ 2 λ () (1 ρ () (1 y, 1 x))λ 3 (1 ρ 3 (1 x)) and f (ɛ, x, y) = ɛλ () (1 ρ () (1 x, 1 y)). We observe the followng: 1) f (ɛ, x, y) and f (ɛ, x, y) are non-decreasng n all arguments for ɛ, x, y [0, 1] and strctly ncreasng f ɛ, x, y (0, 1). 2) For any x 0, y 0, ɛ [0, 1], the sequence x l+1 = f (ɛ, x l, y l ) and y l+1 = f (ɛ, x l, y l ) are monotonc n l. 3) Let x l+1 (ɛ) and y l+1 (ɛ) be defned recursvely by x l+1 (ɛ) = f (ɛ, x l (ɛ), y l (ɛ)), y l+1 (ɛ) = 716

6 f (ɛ, x l (ɛ), y l (ɛ)), x 0 (ɛ) = ɛ 2 and y 0 (ɛ) = ɛ. Then, x l+1 (ɛ) and y l+1 (ɛ) are non-decreasng n ɛ. 4) The functon x (ɛ) = lm x l (ɛ) and y (ɛ) = lm (y l(ɛ)) exst and are non-decreasng for all ɛ [0, 1]. Let ɛ BP re (λ (), ρ () ) = su{ɛ [0, 1] : x (ɛ) = 0} be the threshold defned by the densty evoluton. We calculate several ɛ BP re based on the recursve functons defned n Theorem 5 n Table II, where the frst row ndcates the regular LDPC for G 3, and the frst column ndcates the regular LDPC code for G 1 and G 2. For examle, the result s the threshold when LDPC codes for G 1 and G 2 are (4, 6) regular codes, and the ntermedate LDPC code s (2, 4) code n Table II. From ths table, we see that ɛ BP re > ɛ BP df s ossble wth arorate G 3. That s the threshold can be mroved by ncreasng the dversty of the underlyng error-correctng codes. TABLE II CALCULATION OF ɛ BP re (d v, d c ) (1,2) (2,4) (3,6) (4,8) (3,4) (3,5) (3,6) (4,6) (4,8) IV. JOINT DECODERS FOR AWGN CHANNEL In ths secton, we resent the jont decoder desgns for AWGN channel wth the nsght rovded n revous sectons. In the followng, we assume that both P and Q are AWGN channels wth the same arameters, let the rates of two LDPC codes stll be K N, let G 1, G 2 be the encodng matrces, and let H 1, H 2 denote ther arty check matrces. Let x N 1 0 (1) and x N 1 0 (2) be all ones due to the channel symmetry, y0 N 1 (1) and y0 N 1 (2) are nosy codewords through P and Q, resectvely, thus y (1), y (2) N (1, 2 ) for = 0,, N 1. A. Jont decoder of dentcal content-relcated codes We frst resent the jont decoder desgn and ts theoretcal erformance for the case when encoders are dentcal,.e., G 1 = G 2 and H 1 = H 2. Gven nosy codewords y0 N 1 (1), y0 N 1 (2) R N of the same codeword x N 1 0, the log-lkely-rato (LLR) message from channel P, denoted as u P (),, s ln (y (1) x =1) (y (1) x =0) = 2y (1),.e., Gaussan wth mean and varance 4, and so s for the LLR message from channel Q, denoted 2 as u Q (). Therefore, by averagng of LLR messages from P and Q, we can obtan the combned LLR message as u 0 () = u P ()+u Q () 2,.e., Gaussan wth mean 2 and varance The decodng result s obtaned by alyng sumroduct algorthm wth H 1 and ntal LLR messages u 0 () for = 0,, N 1. That s, let v be a LLR message from a varable node (wth ntal LLR u 0 ()) to a check node, then v = u 0 () + d v 1 =1 u, where u, = 1,, d v 1, are the ncomng LLRs from the neghbors of the varable node excet the check node that gets the message v, and u s udated by tanh( u 2 ) = dc 1 =1 tanh( v 2 ), where v, = 1,, d c 1, are the ncomng LLRs from d c 1 neghbors of a check node. Let u(l) be the average of LLRs sent to a varable node at the l-th round of sum-roduct decodng, let λ(x) and ρ(x) be degree dstrbuton functons for the LDPC code used, and let den BP (λ, ρ) = su{ : u(l) as l } be the threshold for the jont decoder. den BP (λ, ρ) can be obtaned through the methods rovded by Fu [4], we comare t wth BP (λ, ρ) n Table III, and we conclude den BP (λ, ρ) > BP (λ, ρ). B. Jont decoder for dfferent content-relcated codes In ths art, we resent the jont decoder desgn for dfferent content-relcated codes. We use two (d v, d c ) regular LDPC codes to smlfy the analyss of decodng algorthm. The two content-relcated codes are dfferent n ths way,.e, G 1 G 2, H 1 H 2. 1) Jont decoder desgn: Let I 1, I 2, P 1, P 2 and g( ) be the same notatons as before, and a combned codeword s obtaned as y0 2N K 1 = (y I1, y0 N 1 (1) P1, y0 N 1 (2) P2 ) wth ntal LLR from channel u I1 N ( 2, 2 2 ) for I 2 1 (that s by combnng LLRs from y0 N 1 (1) I1 and y0 N 1 (2) I2 ), and u P1, u P2 N ( 2, 4 2 ) for P 2 1 P2. The decodng result s obtaned by alyng sum-roduct decodng algorthm on y0 2N K 1 wth H demonstrated n Fg. 2 wth u I1, u P1 and u P2 secfed as above. 2) Theoretcal erformance analyss by densty evoluton: a) Densty Evoluton: For bts of y0 2N K 1, let v (l) be the average LLR from an nformaton bt to arty nodes at the l-th round, and smlarly let v (l) be that from a arty check bt of y0 2N K 1. For a arty node connectng to j nformaton edges and k arty edges, let µ (l) (j, k) and µ (l) (j, k) be ts LLR sent to an nformaton and a arty bt at l-round, resectvely. Thus, we have µ (l) ( (j, k) = 2 tanh 1 (tanh v(l) 2 )j 1 (tanh v(l) 2 )k) ( µ (l) (j, k) = 2 tanh 1 (tanh v(l) 2 )j (tanh v(l) 2 )k 1). (1) Let u (l) be the average LLR from a arty nodes to an nformaton bt at the l-round, and smlarly let u (l) be that from a arty check node to a arty bt. Then, by averagng LLR from a check node to a nformaton 717

7 and arty bt, we have d c u (l) = u (l) = ρ () µ(l) j=1 d c j=1 (j, k), ρ () µ(l) (j, k), (2) where ρ () and ρ() are the same as revous sectons. Thus we obtan our man result of ths subsecton as below: Theorem 7. For jont decodng of dfferent contentrelcated codes, the LLRs after the l-th round of sumroduct decodng at the varable node are gven by v (l) = µ (0) + (2d v 1) u (l 1), v (l) = µ (0) + (d v 1) u (l 1), where µ (0) s the ntal LLR for nformaton bts of y0 2N K 1, and µ (0) s that for arty bts. Check nodes are udated as equaton (2) and equaton (1). b) Aroxmate algorthm for densty evoluton: For the calculaton of densty evoluton of LDPC codes, there are several aers so far, such as [7], [9] and [4]. The method resented n [7] obtans thresholds wth the Fourer transform, whch s comutatonally ntensve. The method resented n [9] obtans aroxmate thresholds for AWGN channels wth sum-roduct decodng based on two assumtons of the LLR assed: one s ther denstes are aroxmately Gaussan when the channel s AWGN, and the other one s the so-called symmetry condton whch requres a densty functon f(x) to satsfy f(x) = f( x)e x (by enforcng ths condton for Gaussan wth mean m and varance 2, ths condton reduces to 2 = 2m). Fu [4] onted out that the Gaussan assumton does not always hold esecally for LLR from check nodes. For our analyss of densty evoluton, we turn to the method resented n [4] to obtan the aroxmate threshold for two reasons: one s the Gaussan assumton s nvald as onted by Fu [4], and the other one s the symmetry condton roerty does not hold for our case, whch s verfed by ntensve numercal calculatons as shown n Fg. 7 (.e., from ths fgure clearly the assumton that 2 = 2m does not hold). Also the udate rules stated by Theorem 7 and the ntal samles of µ (0) and µ (0) are statonary (.e., nvarant wth resect to the teraton number), thus those udate rules reserve ergodcty. Therefore, based on the wellknown roerty of ergodcty,.e., any statstcal arameter of the random rocess can be arbtrarly closely aroxmated by averagng over a suffcent number of samles, we have the followng aroxmate algorthm for densty evoluton. 1) Ste 0: choose a large number n, generate an ntal std of LLR m of LLR (3,4) (3,5) (3,6) (4,6) (4,8) Fg. 7. m and 2 of LLR for jont decodng of dfferent contentrelcated codes. n samles of µ (0) accordng to N (2/ 2, 2/ 2 ), and smlarly generate a n samles of µ (0) accordng to N (2/ 2, 4/ 2 ). 2) Ste 1 (for varable nodes): For teraton 0, coy µ (0) to v (l) and coy µ (0) to v (l) as shown by varable udate formula of Theorem 7. For other teratons, take the n samles of u (l 1) and u (l 1) from the revous teraton, randomly nterleave (d v 1) samles u (l) and (2d v 1) samles u (l), resectvely. Then, udate v (l) and v (l) by varable udate formula Theorem 7. 3) Ste 2 (for check nodes): For each teraton, take the n samles of v (l) and v (l) as calculated above. Randomly nterleave (d c 1) samles of them, and then comute the n samles of u (l) and u (l) based on equaton (1) and check udate formula Theorem 7. c) Numercal results and analyss: Let u(l) be the average of LLRs from a check node to a varable node at the l-th round of sum-roduct decodng, let λ(x) and ρ(x) be degree dstrbuton functons for the LDPC code used, and defne BP (λ, ρ) = su{ : u(l) dff as l } be the threshold for our jont decoder. We calculate dff BP (λ, ρ) based on the method resented above and comare t wth BP (λ, ρ) and den BP (λ, ρ) n Table III. From the table we can see that t s ossble that dff BP (λ, ρ) > BP den (λ, ρ). C. Jont decoder for related content-relcated codes 1) Jont decoder desgn: Smlar to the BEC case, an ntermedate generator matrx G 3 wth rate 1/2 s used to connect two LDPC generator matrces G 1 and G 2, and the encodng rocess s exactly the same as the BEC counterart. The decodng rocess s resented here: gven y0 N 1 (1) and y0 N 1 (2), a combned codeword y0 2N 1 s constructed the same as before,.e., y 2N 1 (y N 1 0 = 0 (1) P1, y0 N 1 (1) I1, y0 N 1 (2) P2, y0 N 1 (2) I2 ). The decodng result s obtaned by alyng sum-roduct decodng algorthm to y0 2N 1 wth the arty check 718

8 matrx H (constructed the same as Fg.5) and the ntal LLR message u 0 N ( 2, 4 2 ). 2) Theoretcal erformance 2 analyss by densty evoluton: a) Densty Evoluton: For densty evoluton, we assume that (d v, d c) regular LDPC code s used to connect two (d v, d c ) regular LDPC codes. For one (d v, d c ) LDPC code, let v (l) be the average LLR from an nformaton bt to ts arty nodes (not the ntermedate ones) at the l-round, and smlarly let v (l) be that from a arty check bt. For a arty node of one (d v, d c ) LDPC code, let u (l) and u (l) be ts LLR sent to an nformaton bt and a arty bt at l-round, resectvely, and smlarly ther values can be exressed the same as equaton (2). For the ntermedate (d v, d c) LDPC code, let be the average LLR sent to ts arty nodes, and let y (l) be the average LLR sent to ts varable nodes at the l-round of sum-roduct decodng. Thus we have = µ (0) + d v u (l 1) + (d v 1) y (l 1), y (l) = 2 tanh 1 (tanh x(l) 2 )d c 1, where µ (0) s the ntal LLR for bts of y0 2N 1. We have the followng result for the densty evoluton of our jont decoder: Theorem 8. For jont decodng of related contentrelcated codes (.e., the two LDPC codes are both (d v, d c ) LDPC codes and the ntermedate LDPC code s an (d v, d c) LDPC code), the LLRs after the l-th round of sum-roduct decodng at the varable node are gven by v (l) = µ (0) + (d v 1) u (l 1) + d v y (l 1), v (l) = µ (0) + (d v 1) u (l 1), and u (l) where u (l) are udated as equaton (2). b) Aroxmate algorthm for densty evoluton: We resent the aroxmate algorthm for densty evoluton based on Theorem 8 below. 1) Ste 0: choose a large number n, generate an ntal n samles of µ (0) accordng to N (2/ 2, 4/ 2 ). 2) Ste 1 (for varable nodes): For teraton 0, coy µ (0) to ν (l), x (0) and ν (0) as shown by varable udate formula of Theorem 8 and equaton (2). For other teratons, take the n samles of u (l 1), u (l 1) and y (l 1) from the revous teraton, randomly nterleave (d v 1) samles u (l 1), u (l 1) and y (l 1), resectvely. Then, udate v (l), v (l) and by varable udate formula of Theorem 8 and equaton (2). 3) Ste 2 (for check nodes): For each teraton, take the n samles of v (l), and v (l) as calculated above. Randomly nterleave the samles of them, and then comute the n samles of u (l), and based on equaton (2) and check udate formula u (l) of Theorem 8 and equaton (2). c) Numercal results and analyss: Let d BP v,d c (λ, ρ) be the threshold for our jont decoder wth (λ, ρ) beng degree dstrbuton functons for our LDPC codes and (d v, d c) as the ntermedate LDPC code, that s (λ, ρ) = su{ : u(l) as l }. We calculate d BP (λ, ρ) based on the method resented v,d c above and comare t wth BP (λ, ρ), den BP (λ, ρ) and dff BP (λ, ρ) n the Table III. From the results, we can see that t s ossble that d BP (λ, ρ) > v,d BP c den (λ, ρ) wth arorate (d v, d c). BP d v,d c TABLE III THRESHOLDS OF AWGN CHANNELS FOR JOINT DECODERS (d v, d c ) BP den BP dff BP 2,4 BP 3,6 BP 4,8 BP (3,4) (3,5) (3,6) (4,6) (4,8) REFERENCES [1] Y. Ca, E. F. Haratsch, O. Mutlu, and K. Ma, Error atterns n MLC NAND flash memory: measurement, characterzaton, and analyss, n Proceedngs of the Conference on Desgn, Automaton and Test n Euroe, ser. DATE 12, Dresden, Germany, 2012, [2] T.-S. Chung, D.-J. Park, S. Park, D.-H. Lee, S.-W. Lee, and H.- J. Song, A Survey of Flash Translaton Layer, J. Syst. Archt., vol. 55, no. 5-6, , may [3] P. Desnoyers, Analytc modelng of ssd wrte erformance, n Internatonal Systems and Storage conference (SYSTOR 2012), June [4] M. Fu, On gaussan aroxmaton for densty evoluton of low-densty arty-check codes, n Int. Conf. Communcatons, Istanbul, May [5] Q. L, A. Jang, and E. F. Haratsch, Nose Modelng and Caacty Analyss for NAND Flash Memores, n Proc. IEEE Internatonal Symosum on Informaton Theory(ISIT), Honolulu, HI, June 2014, [6] D. J. C. MacKay, Fountan codes, IEE Proc.-Commun, vol. 152, , [7] T. J. Rchardson, M. A. Shokrollah, and R. L. Urbanke, Desgn of caacty-aroachng rregular low-densty arty check codes, IEEE Transacton on Informaton Theory, vol. 47, no. 2, , February [8] T. J. Rchardson and R. L. Urbanke, The caacty of Low- Densty Party-Check Codes Under Message-Passng Decodng, IEEE Transacton on Informaton Theory, vol. 47, no. 2, , February [9] T. J. R. Sae-Yong Chung and R. L. Urbanke, Analyss of sum-roduct decodng of low-densty arty-check codes usng a gaussan aroxmaton, IEEE Transcatons on Informaton Theory, vol. 47, no. 2, , Febuary [10] A. M. Saleh, J. J. Serrano, and J. H. Patel, Relablty of scrubbng recover-technques for memory systems, IEEE Transcatons on relablty, vol. 39, no. 3, , Arl [11] D. Slean and J. L. Wolf, A Codng Theorem for Multle Access Channels Wth Correlated Sources, n The bell system techncal journal, Setember 1973,