Content-assisted File Decoding for Nonvolatile Memories

Transcription

1 Content-asssted Fle Decodng for Nonvolatle Memores Yue L, Yue Wang, Anxao (Andrew) Jang Department of Computer Scence and Engneerng Texas A&M Unversty College Staton, TX 7784 {yl, yuewang, Jehoshua Bruck Department of Electrcal Engneerng Calforna Insttute of Technology Pasadena, CA 95 Abstract Nonvolatle memores (NVMs) such as flash memores play a sgnfcant role n meetng the data storage requrements of today s computaton actvtes. The rapd ncrease of storage densty for NVMs however brngs relablty ssues due to closer algnment of adacent cells on chp, and more levels that are programmed nto a cell. We propose a new method for error correcton, whch uses the random access capablty of NVMs and the redundancy that nherently exsts n nformaton content. Although t s theoretcally possble to remove the redundancy va data compresson, exstng source codng algorthms do not remove all of t for effcent computaton. We propose a method that can be combned wth exstng storage solutons for text fles, namely content-asssted decodng. Usng the statstcal propertes of words and phrases n the text of a gven language, our decoder dentfes the locaton of each subcodeword representng some word n a gven nput nosy codeword, and flps the bts to compute a most lkely word sequence. The decoder can be adapted to work together wth tradtonal ECC decoders to keep the number of errors wthn the correcton capablty of tradtonal decoders. The combned decodng framework s evaluated wth a set of benchmark fles. I. INTRODUCTION Nonvolatle memores (NVMs), such as flash memores, have excellent speed and storage capacty. They have emerged as a crucal technology for storage systems. However, accompanyng the mprovement n data densty, the relablty ssue of NVMs are attractng more and more attenton []. In ths paper, we propose a new method for error correcton named content-asssted decodng. Our method uses the fast random access capablty of NVMs and the redundancy that nherently exsts n nformaton content. Although t s theoretcally possble to remove the redundancy va data compresson, exstng source codng algorthms do not remove all of t for effcent computaton. Our method can be combned wth exstng storage solutons for text fles. Wth dctonares storng the statstcal propertes of words and phrases of the same language, our decoder frst breaks the nput nosy codeword nto subcodewords, wth each subcodeword correspondng to a set of possble words. The decoder then flps the bts n each nosy subcodeword to select a most lkely word sequence as the correcton. Consder the example n Fgure. The Ths work was supported n part by the NSF CAREER Award CCF and the NSF grant CCF The authors thank the Texas A&M Unversty Brazos HPC cluster for provdng computng resources to support the research reported here. Codeword Text Huffman encodng (,,,,,,, ) I am ECC encodng (,,,,,,,,,,, ) I am Nose receved (,,,,,,,,,,, ) IIaa ECC decodng falure (,,,,,,,,,,, ) IIaa Content-asssted decodng (,,,,,,,,,,, ) I am ECC decodng success (,,,,,,,,,,, ) I am Fg.. An example on correctng errors n the codeword of a text. Englsh text I am s stored usng the Huffman codng: {I (, ), (, ), a (, ), m (, )}, where denotes a space. The nformaton bts are encoded wth a (, 8)-shortened Hammng code whch corrects sngle bt errors (the bold bts denote the party check bts). Assume that three errors (marked by the underlnes) are receved by the codeword. The number of errors exceeds the code s correcton capablty, and ECC decodng fals. Our decoder takes n the nosy codeword, and corrects the errors n the nformaton symbols by lookng up a dctonary whch contans two words {I, am}. Ths brngs the number of errors down to one. Therefore, the second tral of ECC decodng succeeds, and all the errors are corrected. Our approach s sutable for natural languages, and can potentally be extended to other types of data where the redundancy n nformaton content s not fully removed by data compresson. The scheme takes advantage of the fast random access speed provded by flash memores for fast dctonary look-up and content verfcaton. For performance evaluaton, we have tested a decodng framework that combnes a soft decson decoder of low-densty party-check (LDPC) codes and our scheme wth a set of text fle benchmarks. Expermental results show that our decoder ndeed ncreases the correcton capablty of the LDPC decoder. The rest of the paper s organzed as follows. Secton II presents the prelmnares, and defnes the text fle decodng problem. Secton III specfes the algorthms of the contentasssted fle decoder. Secton IV dscusses mplementaton detals and expermental results. II. THE MODELS OF FILE DECODING We frst defne a few notatons used throughout ths paper. Let x denote a bnary codeword (x, x,, x n ) {, } n, and we use x[ : ] to represent the subcodeword (x, x +,, x ). Let the functon length(x) compute the

2 length of a codeword x, and we use d H (x, x ) for computng the Hammng dstance between two codewords of the same length. Let A be an alphabet set, and let s A be a symbol. We denote a space by A. A word w (s,, s n ) of length n s a fnte sequence of symbols wthout any space. A phrase p (w,, w ) s defned as a combnaton of two words separated by a space. Defne a text t (w,, w,,,, w n ) as a sequence of words separated by. A word dctonary D w {[w : p ], [w : p ], } s a fnte set of records where a record [w : p] has a key w and a value p >. The value p s an average probablty that the word w occurs n a text. Smlarly, a phrase dctonary D p {[p : p ], [p : p ], } stores the probabltes that a set of phrases appear n any gven text. The dctonary look-up operatons denoted by D w [w] and D p [p] return the probabltes of words and phrases, respectvely. We use the notaton w D w (or p D p ) to ndcate that there s a record n D w (or D p ) wth key w (or p). Let π s be a bectve mappng between a symbol and a bnary codeword, and let x s = π s ( ). In ths paper, the mappng π s s used durng data compresson before ECC encodng, and t encodes each symbol separately. In the example of Secton I, π s refers to the Huffman codebook. The bectve mappng between a word w = (s,, s n ) and ts bnary codeword s defned as π w (w) (π s (s ),, π s (s n )), and the bectve mappng from a text to ts bnary representaton s defned as π t (t) (π w (w ), x s,, x s, π w (w n )) where x s = π s ( ). We use πs, πw to denote the correspondng nverse mappngs. The model of the data storage channel s shown n Fgure. Encoder Fg.. and π t Channel Encoder Nose Channel The channel model for data storage. A text t s generated by the source. The text s compressed by the source encoder, producng a bnary codeword y = π t (t) {, } k. The compressed bts are fed to a channel encoder, obtanng an ECC codeword x = ψ(y) {, } n where n > k. Here we assume a systematc ECC s used. The codeword s then stored by memory cells, and receves an addtve error e {, } n. In ths paper, a bnary symmetrc channel (BSC) wth bt-flppng rate f s assumed. When the cells are read, the channel outputs a nosy codeword x = x e where s the bt-wse exclusve-or over codewords. The nosy codeword s frst corrected by a channel decoder, producng an estmated ECC codeword ŷ = ψ (x ). The source decoder decompresses the corrected codeword, and returns an estmated text ˆt = πt (ŷ) upon success. Ths work focuses on desgnng better channel decoders ψ for correctng bt errors n text fles. We propose a new decodng framework whch connects a tradtonal ECC decoder wth a content-asssted decoder (CAD) as shown n Fgure. A nosy codeword s frst passed nto an ECC decoder. If decodng fals, the decodng output s passed to Fg.. ECC Channel Succeeds or Y Reaches Iteraton Lmt? N Content-asssted The work-flow of a channel decoder wth content-asssted decodng. CAD. Wth the statstcal nformaton stored n D w and D p, the CAD selects a word for each subcodeword to form a lkely text as the correcton for the nosy codeword. The corrected text s fed back to the ECC decoder. The teraton contnues untl ether the ECC decoder succeeds or an teraton lmt s reached. The text fle decodng problem for our CAD s defned as follows. Defnton. Let t be some text generated from the source, and let x {, } n be a nosy channel output codeword of t. Gven two dctonares D w and D p, the text fle decodng problem for the CAD s to fnd an estmated text ˆt whch s the most lkely correcton for x,.e. argmax Pr{ˆt x, D p, D w }. ˆt III. THE CONTENT-ASSISTED DECODING ALGORITHMS The CAD approxmates the soluton to the problem n Defnton n the three steps: () estmate space postons n the nosy codeword to dvde the codeword nto subcodewords, wth each subcodeword representng a set of words n D w. () Resolve ambguty by selectng a word for each subcodeword to form a most lkely sequence. () Perform postprocessng to revert the aggressve bt flps done n () and (). We descrbe the algorthm of each step n ths secton. A. Creatng dctonares The dctonares D w and D p are used n our decodng algorthms. To create the dctonares, we smply count the frequences of words and phrases of two words whch appear n a relatvely large set of dfferent texts n the same language as the texts generated by the source. Fast dctonary look-up s acheved by storng the dctonares n a content-addressable way thanks to the random access n flash memores,.e., the probablty n a dctonary record s addressed by the value of the correspondng word or phrase. As we show later n secton IV, the completeness of the dctonares effects the decodng performance. B. Codeword segmentaton The codeword segmentaton functon σ takes n a nosy codeword and a word dctonary, then flps the mnmum number of bts to make the corrected codeword represent a text, e.g., a sequence of vald words separated by spaces. If σ(x, D w ) = ((x, x,, x k ), (,,, k )), where the number of records D w s bounded by some constant K, and N s the ndex of the frst bt of the -th space n x, the subcodeword x = x[ : ], x k = x[ k + length(x s ) : length(x)], and x = x[ + length(x s ) : ] for {,,, k }. The mappng σ s requred to satsfy the followng propertes: () for each subcodeword x, w D w such that length(x ) = length(π w (w)).

3 () d H (x, (x, x s, x, x s,, x s, x k )) s mnmzed. Intutvely, as the bt-flp rate f s very small (whch s common for NVM channels), the segmentaton functon s a maxmum lkelhood decoder whch flps the mnmum number of bts of the codeword. Let the cost functon c(, ) return the mnmum number of flps taken to convert the subcodeword x[ : ] to represent a text. We have the followng recurrence: { mn{g(, ), h(, )} f < c(, ) otherwse, where g(, ) mn w Dw d H (π w (w), x[ : ]), h(, ) mn k [+, length(xs )] c(, k ) + c(k + length(x s ), )+ d H (x[k : k + length(x s ) ], x s ). The functon g(, ) computes the mnmum number of flps taken to turn x[ : ] nto the codeword of a word n D w. The functon h(, ) computes the mnmum flp cost taken to obtan a codeword representng a text wth at least two words. Example. Consder the example n secton I. The nput nosy codeword x = (,,,,,,, ), and the word dctonary D w = {[I :.5], [am :.5]}. We have σ(x, D w ) = (((, ), (,,, )), ()). Startng from c(, 8), we recursvely compute c(, ) for all <. The results are shown n Fgure 4(b). For nstance, to compute c(5, 8), we frst compute g(5, 8) = as the subcodeword can be turned to represent the word I wth bt-flp. We then compute h(5, 8) =. Ths s because length(x s ) = and the mnmum codeword length of a word n D w s, therefore t s mpossble to splt the subcodeword (,,, ) by a space. Fnally, we have c(5, 8) = mn(, ) =. Our obectve s to compute c(, n) gven an nput codeword of length n, and fnd out the space postons whch help acheve the mnmum cost. When c(, ) s computed recursvely startng from c(, n), some entres wll be recomputed unnecessarly. For nstance, n example, the entry c(4, 5) needs to be computed when we compute c(, 7) and c(, 8). A good way for speedng up such computaton s to use dynamc programmng technques shown n Algorthm, whch computes the fnal result teratvely startng from c(, ), an entry computed n the prevous teraton s saved for later teratons. The algorthm treats c(, ) as the entres of a two dmensonal table. Startng from c(, ), the table the algorthm flls each entry dagonally across the table as shown n Fgure 4(a). The correspondng space locatons for breakng the subcodeword x[ : ], or the set of words that x[ : ] can be flpped to represent s recorded usng a two dmensonal table m. In practce, as f s close to, the average number of errors n the codeword of a word s small. Computng the set of possble words S w for a gven nosy codeword can be accelerated by passng an addtonal Hammng dstance lmt d to reduce the search space,.e. nstead of searchng the whole D w as n g(, ), we search the set {w w D w, d H (π w (w), x[ : ]) < d} to skp the words whch are too far from the nosy codeword n terms of d and Hammng dstance metrc. As we are more Algorthm CodewordSegmentaton(x, D w ) n length(x), l length(x s ) Let c and m be two n n tables Let wordsets and spaces be two empty lsts for t from to n do for from to n t + do + t d mn mn w Dw d H (π w (w), x[ : ]) S w {w w D w, d H (π w (w), x[ : ]) = d mn } k for k from + to l do d c(, k) + c(k + l, ) + d H (x s, x[k : k + l ]) f d < d mn then d mn d k k f k = then m(, ).words S w m(, ).words m(, ).space k c(, ) d mn TraceBack(, n, spaces, wordsets, m, l) return wordsets and spaces Algorthm TraceBack(,, spaces, wordsets, m, l) f m(, ).words = then k m(, ).space TraceBack(, k, spaces, m, l) spaces. append(k) TraceBack(k + l,, spaces, m, l) wordsets. append(m(, ).words) nterested n the space locatons than the value of c(, ), after the entres of c and m have been flled, Algorthm s used to recursvely trace back the soluton path recorded n m. The results are the ordered space locatons and the sets of words for the codewords between the spaces. Assume that K s a constant whch s much smaller than N, and that the codeword of each word has lmted length bounded by some constant. The tme complexty of our dynamc programmng algorthm s O(n). Ths s because only O(n) entres need to be computed and each computaton takes O() tme. The algorthm requres O(n ) space for storng the tables c and m. Example. For the example n secton I, the tables c and m computed by Algorthm are shown n Fgure 4(b) and 4(c). The mnmum flppng cost s c(, 8) =, and the ndex of the estmated space s m(, 8).space =. Wth the estmated space, the subcodeword x[ : ] = (, ) can be flpped to denote a word n the set {I}, and the subcodeword x[5 : 8] = (,,, ) can be flpped to denote a word n the set {am}. C. Ambguty resoluton Gven the subcodewords (x, x,, x k ) between the estmated spaces, and a lst of word sets (W, W,, W k ) computed from the codeword segmentaton algorthm, for {,, k} we select a word w from W to form a most probable text ˆt = (w,, w,,,, w k ). The codeword π t (ˆt) s a correcton for the nput nosy codeword. Specfcally,

4 n n X X X X X {am} {am} {am} {am} {am} X X X X X X {I} {I} {I} {I} {I} {I} 4 X 5 7 {I} (a) Iteratve table fllng. (b) Table c. (c) Table m. Fg. 4. The examples of codeword segmentaton. In Fgure (c): A number n the table denotes the ndex of the frst bt of an estmated space; a set of word means the subcodeword can be flpped to any of the word n the set. The cross means a subcodeword can nether be flpped to represent a word nor to a text wth at least two words. ths step s to compute w, w, w, w 4, argmax (w,w,,w k ) W W W k Pr{(w, w,, w k ), (x, x,, x k )}. Let the functon P(w ) compute the maxmal ont probablty when some word w s selected from W and appended to the prevously selected word sequence (w, w,, w ). For [, k], we have P(w ) max (w,,w ) W W Pr{(w,, w ), (x,, x )}. Assume the words n a text form a one-step Markov chan,.e., for, Pr{w (w, w,, w )} = Pr{w w }. Therefore, we rewrte the equaton above as: P(w ) = max (w,,w ) W W Pr{w w } Pr{x w } Pr{w } Pr{w w } Pr{w w } k= Pr{x k w k } = max (w,,w ) W W Pr{w w } Pr{x w } Pr{(w,, w ), (x,, x )} = max w W Pr{x w } Pr{w w } max (w,,w ) W W Pr{(w,, w ), (x,, x )} = max w W Pr{x w } Pr{w w } P(w ). () and P(w ) = Pr{w } Pr{x w }. The condtonal probablty Pr{x k w k } s computed from the channel statstcs by Pr{x k w k } = f d H(π w (w k ),x k ) ( f ) length(x k) d H (π w (w k ),x k ). The probabltes Pr{w } = D w [w ] and Pr{w k w k } = D p [(w k,, w k )] are looked up from the dctonares: The derved recurrence suggests that the optmzaton problem can be mapped to the problem of trells decodng, whch s agan solved by dynamc programmng. The trells for our problem has k tme stages. The observed codeword at the - th stage s x for {,, k}. There are W vertces at stage wth each representng an element w of W and beng assocated wth the condtonal probablty Pr{w x }. The weght of the drected edge from a vertex at stage wth word w x to a vertex of stage + wth word w y s the condtonal probablty Pr{w y w x }. An example of the mappng s shown n Fgure 5. Our target s to compute the sequence whch acheves max wk W k P(w k ), whch leads to the Vterb path n the correspondng trells startng from a vertex n stage and endng at a vertex n stage k. The dynamc programmng algorthm for solvng our trells decodng problem s specfed n Algorthm, whch s w, w, w, w, w, w 4, Fg. 5. Example of the mappng to trells decodng. The word sets W = {w,, w, }, W = {w,, w,, w, }, W = {w,, w,, w, } and W 4 = {w 4,, w 4, } respectvely corresponds to the subcodewords x, x, x and x 4. adapted from the Vterb decodng []. The fnal soluton s computed teratvely, startng from P(w ) accordng to the recurrence. When the last teraton s fnshed, we trace back Algorthm Vterb((W,, W k ), (x,, x k ), f, D w, D p ) n max l [,k] W l Let p and s be two n k tables p max, ndex for t from to k do for from to W t do p f d H(π w (W t []),x t ) ( f ) length(x t) d H (π w (W t []),x t ) f t = then p(, t) p D w [W t []] p max, ndex for from to W t do p p D p [(W t [],, W t [])] p[, t ] f p > p max then p max p ndex p(, t) p max s(, t) ndex words [W k [ndex]] for t from k to do s(ndex, t) words. appendtofront(w t []) ndex return words along the Vterb path recorded n the table s, collectng the selected words to form an estmated text ˆt. The complexty of the Vterb decodng algorthm s O(n k) where k = O(N) s the length of the nput codeword lst, and n = max [,k] W = O(K) s the cardnalty of the bggest nput word set. As K s a constant whch s much smaller than N, the Vterb decodng for our case has tme complexty O(N). The algorthm requres O(nk) = O(N) space for storng the tables p and s.

5 D. Post-processng If unknown words or phrases occur n the nput codeword, addtonal errors wll be ntroduced durng codeword segmentaton and ambguty resoluton. Unknown words (phrase) refers to new or rare words (phrases) whch are not ncluded n D w (D p ). Upon encounterng an unknown word, the codeword segmentaton algorthm tends to splt ts codeword nto subcodewords representng known words wth the space symbol. Such segmentaton ntroduces addtonal bt errors. We use a smple post-processng step whch undoes the btflps ssued by such aggressve segmentaton. The dea s to use the phrase dctonary D p to check whether two adacent words returned by the Vterb decoder s known to D p. If so, the post-processor smply accepts the segmentaton, otherwse the correspondng bts n the ntal nosy codeword are used to replace the codewords for those unknown phrases. The complexty of ths step s O(k) = O(N). A. Implementaton detal IV. EXPERIMENTS Our mplementaton supports the use of basc punctuaton n the nput text fles, ncludng,,.,? and!. Ths s done by addng another functon n the defnton of c(, ) when <. The functon measures the number of flps taken to turn a subcodeword to represent a word followed by a punctuaton. Durng ambguty resoluton, overflow may occur n the multplcatons of probabltes when N s large. We thus use a logarthmc verson of Eq.(). Usng addtons nstead of multplcatons of floatng pont numbers sgnfcantly delays the overflow. A smoothng technque s used for computng Pr{w w }. The probablty Pr{w } wll be used f the phrase (w,, w ) s unknown to D p. The reason s that returnng for unknown phrases suddenly makes the whole ont probablty be n Eq.() and cancels the path. B. Evaluaton We evaluated decodng performance of the channel decoder combnng the LDPC sum-product decoder and the CAD. We compared the bt error rates (BER) of the combned channel decoder wth those of the scheme usng the LDPC sumproduct decodng alone. The test nputs nclude self-collected paragraphs and 8 paragraphs randomly extracted from the Canterbury Corpus, the Calgary Corpus, the Large Corpus [], and the large text compresson benchmark [4] (see Table I). The dctonares are bult usng the books randomly extracted from Proect Gutenberg [5]. The functons π s and πs are mplemented wth Huffman codng. A (584, 4)-random LDPC code s used as the ECC. The teraton lmt of the sum-product decoder s. The teraton threshold for the LDPC-CAD exchange s. The bt-flp rate of the BSC s., whch makes the sum-product decoder fal to converge wth hgh probablty. The decodng BERs for complete and ncomplete dctonares are shown n Table I and Table II, respectvely. The BERs for each benchmark are averaged from experments. In Table I, the combned channel decoder sgnfcantly outperforms the tradtonal decoder thanks to TABLE I THE DECODING BERS WHEN THE DICTIONARIES ARE COMPLETE Name Category From ECC only Combned emal Emal dscusson Calgary lcet Lecture notes Canterbury 8.4. alce Novel Canterbury confntro Call for paper Self-made 8.7. bble The bble Large asyoulke Shakespeare play Canterbury plrabn Poetry Canterbury 8.6. news Web news Self-made enwk Wkpeda texts Large text 8.. world9 The world fact book Large the completeness of the dctonares. The performance for the benchmark world9 s not as good as others. Ths s because world9 has much more punctuaton but fewer words than other benchmarks do, and more errors occur n the punctuatons whch the CAD s not good at correctng. In Table II, to see the effectveness of the post-processor, we also show the performance of the combned decoder wthout the post-processor. The completeness of the dctonares deter- TABLE II THE DECODING BERS WHEN THE DICTIONARIES ARE INCOMPLETE. Name ECC only Combned After PP UW% UP% emal lcet alce confntro bble asyoulke plrabn news enwk world mnes the decodng performance. For nstance, the benchmarks world9 and enwk have consderable number of words and phrases whch are unknown to our dctonares. The combned decoder wthout post-processng ntroduces addtonal errors by aggressvely breakng the codewords of the unknown words nto subcodewords separated wth spaces. In such cases, the post-processor s able to recognze and revert most of the over-aggressve bt-flps. Ths greatly reduces the number of addtonal errors ntroduced due to the gnorance of the CAD. For the benchmark confntro, the performance of the decoder wthout post-processng s much better than that of the decoder usng post-processng. Ths s because confntro has only a few unknown words but many techncal phrases whch are unknown to D p. The unknown phrases makes the postprocessor tend to revert reasonable correctons done n the prevous steps. REFERENCES [] L. M. Grupp, J. D. Davs, and S. Swanson, The bleak future of nand flash memory, n Proceedngs of the th USENIX conference on Fle and Storage Technologes, Berkeley, CA, USA,. [] A. Vterb, Error bounds for convolutonal codes and an asymptotcally optmum decodng algorthm, IEEE Transactons on Informaton Theory, vol., no., pp. 6 69, aprl 967. [] The Canterbury Corpus: [4] Large Text Compresson Benchmark: [5] Proect Gutenberg: