Coding for Noisy Write-Efficient Memories

Transcription

1 Coding for oisy Write-Effiient Memories Qing Li Computer Si. & Eng. Dept. Texas A & M University College Station, TX qingli@se.tamu.edu Anxiao (Andrew) Jiang CSE and ECE Departments Texas A & M University College Station, TX ajiang@se.tamu.edu Abstrat For nonvolatile memories suh as flash memories and phase-hange memories, endurane and reliability are both important hallenges. Write-Effiient Memory (WEM) is an important rewriting model to solve the endurane problem. An optimal rewriting ode has been proposed to approah the rewriting apaity of WEM. Aig at jointly solving the endurane and the data reliability problem, this work fouses on a ombined error orretion and rewriting ode for WEM. To that end, a new oding model, noisy WEM, is proposed here. Its noisy rewriting apaity is explored. An effiient oding sheme is onstruted for a speial ase of noisy WEM. Its deoding and rewriting operations an be done in time O( log ), with as the length of the odeword, and it provides a lower bound to the noisy WEM s apaity. I. ITRODUCTIO onvolatile memories (suh as flash memories and phase-hange memories (PCM)) are beog ubiquitous nowadays. Besides the well-known endurane [8] problem, another serious hallenge is the data reliability issue, e.g., retention error in AD flash memories and resistane drift in PCM. Write-effiient memory (WEM) [] is a oding model that an be used to solve the endurane problem for nonvolatile memories. In WEM, odewords are partitioned into several disjointed sets, where odewords of the same set represent the same data. A ost onstraint has to be satisfied during the rewriting (namely, updating the data stored in WEM). WEM is a natural model for PCM [], and an also be applied to flash memory when data representation sheme suh as rank modulation [3] is used. In WEM, there is a ost assoiated with hanging the level of a ell. For nonvolatile memories suh as PCM, this ost is important beause ells age with programg and have the endurane problem. An optimal ode [4] has been proposed to ahieve the rewriting apaity of WEM. However, rewriting odes ombined with error orretion are still limited [9], espeially for WEM [6]. In this paper, we propose a joint error orretion and rewriting sheme for WEM. While previous results are mainly for Write-One Memories [3], our work fouses on WEM. We propose a new oding model, noisy WEM, and provide a haraterization for its apaity. We present an effiient oding sheme based on polar odes [2] for a speial ase of noisy WEM. The sheme is related to the oding shemes in [], [3] and [9]. We also provide a lower bound to noisy WEM s apaity, and experimentally verify the ode onstrution s performane. The rest of this paper is organized as follows. In Setion II, we introdue noisy write-effiient memory model. In Setion III, we present haraterization of noisy rewriting apaity of noisy WEM. In Setion IV, we present an effiient ode onstrution for a speial ase of binary noisy WEM, and verify its performane experimentally. We onlude this paper in Setion V. II. OISY WRITE-EFFICIET MEMORY MODEL In this setion, we first introdue terms and notations used throughout the paper, and then formally present the definitions of noisy WEM and related parameters. A. Terms and notations Let X = {,,, q } be the alphabet of a symbol stored in a ell. (For example, for PCM, it denotes the q levels of a ell.) x, y X, let the rewriting ost of hanging a ell s level from x to y be ϕ(x, y). Given ells and, y X, let ϕ(x, y ) = i= ϕ(x i, y i ) be the rewriting ost of hanging the ell levels from x to y. Let M and D = {,,, M }. We use D to denote the M possible values of the data stored in the ells. Let the deoding funtion be D : X D, whih maps the ells levels to the data they represent. Let the rewriting funtion be R : X D X, whih hanges the ells levels to represent the new input data. aturally, we require D(R(x, i)) = i for any X and i D. Assume the sequene of data written to the storage medium is {M,, M t }, where we assume M i for i t is uniformly dis-

2 tributed over D, and the average rewriting ost def t is D = lim t t ϕ( (i), D(M i, x (i))), i= where (i) is the urrent ell states before the i th update. By assug the stationary distribution of ell levels (i) is π( ), D = π( ) j D D j ( ), where D j ( ) is the average rewriting ost of updating ell level states x to a odeword representing j D. Let P(X X ) be the set of joint probability distributions over X X. For a pair of random variables (S, X) (X, X ), let P SX, P S, P X S denote the joint probability distribution, the marginal distribution, and the onditional probability distribution, respetively. E( ) denotes the expetation operator. If X is uniformly distributed over {,,, q }, denote it by X U(q). B. oisy WEM with an average rewriting ost onstraint We formally present the definition of noisy WEM with an average rewriting ost onstraint as follows: Definition. An (, M, q, d) ave noisy WEM ode for the storage hannel P = (X, X, P Y X (y x)) onsists of D and C = i D C i, where C i X is the set of odewords representing data i. We require i j, C i Cj =. (Here P Y X (y x) represents the transition probability of the noisy hannel, whih hanges a ell s level from x to y.) It also onsists of a rewriting funtion R(s, i) and a deoding funtion D(y ). Let d R + be an upper bound to the average rewriting ost; namely, we require D d. The noisy WEM model is illustrated in Fig.. Here the -dimensional vetor s X is the urrent ell states, and the message M is the new information to write, whih is independent of s. The rewriter uses both s and M to hoose a new odeword X, whih will be programmed as the ells new states, suh that the average rewriting ost satisfies the predefined ost onstraint. The odeword passes a noisy hannel, and the noisy odeword y X is its output. The deoder an reliably deode y to reover the message M. The ell states s are drawn independently and identially from the probability distribution P S (s). The noisy hannel is memoryless, and is haraterized by the transition probabilities P Y X (y x). Let λ i = P r(d(y ) i = R(s, i)) be the deoding error probability given data i. Let λ () be max λ i. Let R = log M i D be the ode rate, and we say R is ahievable if there exists a (, 2 R, q, d) ave ode suh that λ () as. The noisy rewriting Fig.. The noisy WEM model. M, s,, y and ˆM are, respetively, the message, the urrent ell states, rewritten odeword, the noisy hannel s output, and the estimated message. apaity C(q, d) ave is the supremum of all ahievable rates. The noisy WEM problem is: given the average rewriting ost onstraint d, find the maximal rate R of reliable rewriting supported by the rewriter and the deoder despite the noisy hannel. Let P(q, d) be {P SX P(X X ) : P S = P X, E(ϕ(S, X)) d}. When there is no noise, C(q, d) ave is R(q, d) ave = max P SX P(q,d) []. C. oisy WEM with a maximal rewriting ost onstraint The WEM ode in definition puts a onstraint on the average rewriting ost. We now define a ode with a maximal rewriting ost onstraint. Definition 2. An (, M, q, d) noisy WEM ode for the storage hannel P = (X, X, P Y X (y x)) onsists of D and C = i D C i, where C i X is the set of odewords representing data i. We require i j, C i Cj =. It also onsists of a rewriting funtion R(s, i) and a deoding funtion D(y ). Let d R + be an upper bound to the maximal rewriting ost; namely, we require ϕ(s, R(s, i)) d for any s C and i D. The rate R and noisy rewriting apaity C(q, d) an be defined similarly as before. When there is no noise, C(q, d) is R(q, d) = R(q, d) ave []. III. CHARACTERIZIG OISY REWRITIG CAPACITY In this setion, we present the haraterization of noisy rewriting apaity of C(q, d) and C(q, d) ave, respetively. The haraterization of C(q, d) is presented below. It is effetively the generalization of that of Gel fand and Pinsker [7], whih onsiders the problem without ost onstraint. Due to the spae onstraint, we skip details of its proof and present its sketh below. A omplete proof is provided in the full version of this paper [2]. The diret part proof is based on random oding and typial sequenes; the onverse part is based on tehniques of Fano s inequality [4] and Csiszár sum identity [5], and auxiliary random variables identifiation. Lemma 3. For a given rewriting ost funtion ϕ(, ), C(q, d) = max {I(Y ; U) I(U; S)}, where U is P U S,P X U,S P SX P(q,d) an auxiliary random variable, and U (X, S) Y is a Markov hain.

3 The next lemma presents us the haraterization of C ave (q, d), whih is the same as C(q, d). We omit its proof as any ode for (, M, q, d) is a ode for (, M, q, d) ave, therefore R ave C(q, d). The onverse part is the same as previous one. Lemma 4. For a given rewriting ost funtion ϕ(, ), C ave (q, d) = C(q, d) = max {I(Y ; U) I(U; S)}, P U S,P X U,S P SX P(q,d) where U (X, S) Y is a Markov hain. IV. A CODE COSTRUCTIO FOR BIARY DEGRADED AD SYMMETRIC OISY WEM Let P s (q, d) = {P SX P(X X ) : P S = P X, S U(q), E(ϕ(S, X)) d} be the set of joint probabilities with uniform marginal distributions. Let symmetri rewriting apaity be defined as R s (q, d) = max H(X S). Let W SX be P SX P s (q,d) arg max H(X S). We all W = (X, X, W X S) P XS P s (q,d) the WEM hannel. We say Q = (X, Z, Q Z X ) is degraded with respet to W = (X, Y, W Y X ) (whih we denote by Q W) if there exists a hannel P = (Y, Z, P Y Z ) suh that for all z Z and x X, we have Q Z X (z x) = W Y X (y x) P Z Y (z y). y Y In this setion, we onsider symmetri rewriting apaity, and present a ode onstrution for noisy WEM when the WEM hannel W is degraded with respet to the symmetri storage hannel P. We fous on binary ells, namely, X = 2. We all suh WEM a binary degraded and symmetri noisy WEM. (ote that when the flipping rates meet W X S > P Y X, the degradation ondition is naturally satisfied.) A. A nested polar ode onstrution for binary degraded and symmetri noisy WEM with an average rewriting ost onstraint ) A brief introdution to binary polar odes [2]: Let W : X Y be a binary-input disrete memoryless (DMC) ) hannel. Let G n 2 be n-th Kroneker produt of. Let Z(W ) = WY X (y )W Y X (y ). ( 2 u i+ i= y Y The polar ode C (F, u F ) is { = u G n 2 : u F {, } F }, where F {,,, }, u F is the subvetor u i : i F, and u F {, } F. Let W (i) : {, } Y {, } i be a sub-hannel with W (i) (y, u i def u i ) = W Y X (y i (u G n 2 ) i), and (u G n 2 ) i denotes the i-th element of u G n 2. Fig. 2. Illustration of relationship between F W and F P. The line represents the indexes, and F W and F P are the frozen set for the WEM hannel W and the storage hannel P, respetively. 2) The ode onstrution: We fous on the ode onstrution with symmetri rewriting ost funtion, whih satisfies x, y, z {, }, ϕ(x, y) = ϕ(x + z, y + z), where + is the XOR operation over GF(2). (Many ost funtions, suh as the Hamg-distane based ost funtion satisfies this onstraint.) Algorithm IV. A ode onstrution for binary degraded and symmetri noisy WEM and storage hannel P : Let C (F P, u FP ) be a polar ode [2] designed for the storage hannel P, where F P = {i {,,, } : Z(P (i) ) 2 β } and u FP is set to. 2: Let C (F W, u FW ) be a polar ode designed for the WEM hannel W, where F W = {i {,,, } : Z(W (i) ) 2 β } and F P F W. 3: The (, M, 2, d) ave ode is C = C (F P, u FP ) = {C (F W /F P, u FW/FP (i))}, where u FW/FP (i) is the binary representation form of i {,, M }. The ode onstrution is presented in Algorithm IV., where we use nested polar odes (i.e., the polar ode for hannel oding [2] and the polar ode for WEM [4]) to design noisy WEM odes. The fat that F P F W follows from [, lemma 4.7]. Fig. 2 presents us a pitorial presentation of F W and F P. The rewriting funtion is presented in Algorithm IV.2. It is very similar to that of [4] exept for how to set u F. The deoding funtion is presented in Algorithm IV.3, where we use the SC (Suessive Canellation) deoding [2] to assure λ () as. 3) Theoretial performane analysis: a) Code analysis: Theorem 5. For a binary degraded symmetri noisy WEM, fix d, R R s (2, d) ave H(P Y X ) and any < β < 2, there exists a sequene of nested polar odes of length with rates R R so that under the above rewriting and deoding operations, D d + O(2 β ), λ () O(2 β ), and the rewriting as well as the deoding operation omplexity is O( log ). Proof: Let ɛ and < β < 2 be some onstants.

4 Algorithm R(, i). IV.2 The rewriting operation y = : Let v = + g, where g is a ommon and uniformly distributed message, and + is over GF(2). 2: Apply SC (Suessive Canellation) enoding [] to v, and this results in a vetor u = Û(v, u FW/FP (i)), that is, u j = (u FW/FP (i)) j if j F W/FP if j F P m and ŷ = u G n 3: y = ŷ + g. with probability 2. W(u j,v j m), W(u j m,v m ) Algorithm IV.3 The deoding operation u FW/FP (i) = D( ). : ŷ = + g. 2: Apply SC deoding to ŷ, and this results in y {, i.e., y j = if j F P arg m max P (j) (yj, ŷ m) else 3: u FW/FP (i) = (y (G n 2 ) ) ufw/fp. F P and F W are F W = {i : Z(W (i) ) 2 β }, F P = {i : Z(P (i) ) 2 β }. Based on [, lemma 2.6] lim P n r(z(w(i) ) 2 β ) = I(W) = R s (2, d) ave, thus F W Rs (2, d) ave +ɛ for suffiiently large. Similarly, F P H(P Y X) ɛ for suffiiently large. As mentioned, F P F W, thus R = F W F P R s (2, d) ave H(P Y X ). Sine the rewriting funtion is similar to that of [4], the rewriting ost is guaranteed by [4, theorem 8], i.e., D d + O(2 β ). The error probability λ () is guaranteed by [2, theorem 4], that is λ () Z(P (i) ) O(2 β ). i F P b) Covering radius of polar odes: In the following, we present overing radius to theoretially upper bound the maximal rewriting ost when the ost metri is the Hamg distane between old and new ell levels. Let the polar ode ensemble be C (F ) = {C (F, u F ) : u F {, } F }. Let the overing radius of C (F ) (whih we denote as H (C (F ))) be max d H (, y C (F,u F (i)) y C (F,u F (j)) d H (, y ), where ) is the Hamg distane between and y. Lemma 6. H (C (F )) = l F 2wt(l), where wt(l) is the number of ones in (i.e., Hamg weight of) the binary representation of l. Proof: H (C (F )) = max = max = max C (F,u F (i)) y C (F,u F (j)) C (F,u F (i)) y C (F,u F (j)) z C (F,u F (i)+u F (j)) d H (, y ), wt( y ), wt(z ), = max k l F 2wt(l), () = l F 2wt(l), where eq. () is based on [, lemma 6.2], i.e., the imal distane of polar ode C (F, u F ) is l F 2wt(l). The above results an be generalized to the following polar odes C,M (F ) = M C (F, u F (i)), where i= {u F (i)} forms a group under binary operations in GF(2). 4) Experimental performane: The experimental performane is presented in Fig. 3, where the rewriting ost funtion is the Hamg distane between old and new ell states, the upper bound of C(2, d) is H(d) [], the storage hannel P is the binary symmetri hannel with flipping rate p =., and the lower bound is H(d) H(p). λ () is set to be around 5. We an see that the rates and the average rewriting osts approah those of points of H(d) H(.) as the length of odeword inreases. Longer odewords are needed for further approahing the lower bound. B. A nested polar ode onstrution for binary degraded and symmetri noisy WEM with a maximal rewriting ost onstraint The ode onstrution in Algorithm IV., the rewriting funtion in Algorithm IV.2 and the deoding funtion in Algorithm IV.3 an be applied to noisy WEM odes with a maximal rewriting ost onstraint as well. Similar to the analysis of Theorem 5, we obtain the following result for the theoretial performane of the proposed ode onstrution. Theorem 7. For a binary degraded symmetri noisy WEM, fix d, δ, R R s (2, d) H(P Y X ) and any < β < 2, there exists a sequene of nested polar odes of length with rates R R, so that under the above rewriting operation and deoding operation, the probability that the rewriting ost between a urrent

5 We present our experimental results in Fig. 4. The rewriting ost funtion, storage hannel P, and λ () are the same as those of the previous subsetion. We let δ =.d, and d =.32,.24, and.9, respetively. The empirial probability Q(ϕ(y, ).d) is presented in Fig. 4. As predited by Theorem 7 it dereases (nearly exponentially) as the length of odeword inreases. However, even longer odewords are needed to make the probability to be truly negligible. V. COCLUDIG REMARKS In this paper, we analyze the apaity for noisy WEM, and present a ode onstrution for binary degraded and symmatri noisy WEM. The ode onstrution is both theoretially analyzed and experimentally verified. We are interested in extending the ode onstrution to q-ary ells, and to more general settings regarding hannel degradation. Those remain as our future researh diretions. Fig. 3. Experimental performane for noisy WEM with an average ost onstraint for polar ode with various lengths, where the x-axis is the rewriting rate, the y-axis the average rewriting ost, and the theoretial points are those points (R, d) (R {.2,.3,,.9}) satisfying R = H(d) H(.). Fig. 4. Experimental performane for noisy WEM with a maximal ost onstraint with d =.32,.24 and.9, respetively, where the x-axis is the odeword length, and y-axis is the empirial probability Q(ϕ(y, ).d). odeword y and its updated odeword x larger than d + δ is bounded by Q(ϕ(y, ) d + δ) < 2 β, λ () O(2 β ), and the deoding and rewriting operations omplexity of the ode is O( log ). VI. ACKOWLEDGEMET This work was supported in part by the SF Grant CCF REFERECES [] R. Ahlswede and Z. Zhang, Coding for write-effiient memory, Information and Computation, vol. 83, no., pp. 8 97, Otober 989. [2] E. Arikan, Channel polarization: a method for onstruting apaity-ahieving odes for symmetri binary-input memoryless hannels, IEEE Trans on Inf Theory, vol. 55, no. 7, pp , July 29. [3] D. Burshtein and A. Strugatski, Polar Write One Memory Codes, IEEE Transaation on Information Theory, vol. 59, no. 8, pp , August 23. [4] T. M. Cover and J. A. Thomas, Elements of Information Theory. ew York: Wiley, 99. [5] I. Criszar and J. Korner, Broadast hannels with onfidential messages, IEEE Transation on Information Theory, vol. 24, no. 3, pp , May 978. [6] F. Fu and R. W. Yeung, On the apaity and error-orreting odes of write-effiient memories, IEEE Trans on Inf Theory, vol. 46, no. 7, pp , ov 2. [7] S. I. Gel fand and M. S. Pinsker, Coding for Channel with Random Parameters, Problems of Control Theory, vol. 9, no., pp. 9 3, 98. [8] A. Jiang, V. Bohossian, and J. Bruk, Rewriting odes for joint information storage in flash memories, IEEE Trans on Inf Theory, vol. 56, no., pp , Otober 2. [9] A. Jiang, Y. Li, E. E. Gad, M. Lanberg, and J. Bruk, Joint rewriting and error orretion in write-one memories, in Pro. IEEE International Symposium on Information Theory(ISIT), Istanbul, Turkey, June 23, pp [] S. B. Korada, Polar odes for hannel and soure oding, Ph.D. dissertation, EPFL, 2. [] L. A. Lastras-Montano, M. Franeshini, T. Mittelholzer, J. Karidis, and M. Wegman, On the lifetime of multilevel memories, in Proeedings of the 29 IEEE international onferene on Symposium on Information Theory (ISIT 9), Coex, Seoul, Korea, 29, pp [2] Q. Li and A. Jiang, Coding on oisy Write-effiient Memories, available on faulty.s.tamu.edu/ajiang/publiations/24/wemfull.pdf. [3] Q. Li, Compressed Rank Modulation, in Pro. 5th Annual Allerton Conferene on Communiation, Control and Computing (Allerton), Montiello, IL, Otober 22. [4] Q. Li and A. Jiang, Polar odes are optimal for Write-effiient Memories, in pro 5th Annual Allerton Conferene on Communiation, Control and Computing (Allerton), Montiello, IL, Otober 23.