Sum-Capacity of Multiple-Write Noisy Memory

Size: px
Start display at page:

Download "Sum-Capacity of Multiple-Write Noisy Memory"

Transcription

1 2011 IEEE Iteratioal Symposium o Iformatio Theory Proceedigs Sum-Capacity of Multiple-Write Noisy Memory Lele Wag ad Youg-Ha Kim Departmet of Electrical ad Computer Egieerig Uiversity of Califoria, Sa Diego La Jolla, CA USA {lew001, yhk}@ucsdedu Abstract Motivated by the emergig iterests i o-volatile solid-state computer memories such as flash memories, this paper studies the problem of repeatedly storig iformatio o memory cells with oise ad state The goal is to reliably covey t messages by writig Xj o a -cell oisy memory py j x j,y j 1), which stores Yj at the j-th write We model this problem as a chael with state ad itroduce the multiple-write oisy memory model, which icludes the write-oce memory ad flash memory models as special cases The t-write sum-capacity for the multiple-write oisy memory is established as C sumt) max [ IX 1;Y 1)+ t IU ] j;y j) IU j;y j 1)), j2 where the maximum is over all pmfs px 1) t j2 puj yj 1) ad fuctios x ju j,y j 1),j 2,,t We derive three outer bouds o the capacity regio ad discuss their extesio to other classes of memory models These results exted Wolf, Wyer, Ziv, ad Körer s work o the biary write-oce memory ad Fu ad Vick s work o the geeralized writeoce memory to oisy memories I INTRODUCTION AND MAIN RESULT A write-oce memory WOM) [12] is a biary iformatio storage medium, where each cell of the memory is iitialized at 0-state ad is allowed to chage from 0-state to 1-state, but ot vice versa Suppose we wish to reliably store t idepedet messages by writig t times o a -cell WOM The goal is to maximize the total umber of bits stored per cell Oe way to deal with this problem is to desig codewords such that i ay two cosecutive writes, there is o trasitio from 1 to 0 i the same cell positio The total umber of codewords satisfyig such property characterizes the limit o storig iformatio o the WOM A uderlyig assumptio i this combiatorial approach is that the memory cell always faithfully store the writte codewords A alterative probabilistic) way to attack this problem is through chael codig for commuicatio over a DMC with state Codewords are assumed to be arbitrary without ay costrait, while the memory cell may ot always record the iput faithfully Specifically i the WOM, whe a cell is at 0-state, it records what is writte Whe a cell is at 1-state, it always stores 1 regardless of what the ecoder writes see Figure 1) Clearly, the kowledge of the cell state iformatio help to make the commuicatio more reliable How much do the ecoder ad the decoder kow about the cell state? The cell state i the WOM is the output of the previous write ad before each write, the ecoder has access to the whole output sequece, while the decoder does ot Hece, the problem is readily recogized as a chael with state where the state iformatio is available ocausally oly at the ecoder X S 0 Y X S 1 Y Fig 1 Biary write-oce memory May attempts have bee made to determie the theoretical limit of maximum umber of bits stored per cell Several code costructios for the biary WOM were proposed by Rivest, Shamir [12] ad Cohe, Godlewski, ad Merkx [3] Wolf, Wyer, Ziv, ad Körer [13] established the zero-error) capacity regio of the biary WOM Fiat ad Shamir [5] geeralized the biary WOM by allowig q-ary alphabet ad extedig the legal state trasitios to ay directed acyclic graph This model has bee suggested as a good model for flash memories [9] Practical code costructios based o the geeralized WOM have bee proposed by Zémor, Cohe [15], Jiag, Bruck [9], Yaakobi, Siegel, Vardy, ad Wolf [14] amog others Fu ad Vick [6] established the zero-error) capacity regio of the geeralized WOM Further geeralizig the determiistic models for memories, Heegard [8] cosidered oisy memories ad foud the capacity regio for the WOM with symmetric oise The mai goal of this paper is to establish the capacity regio of oisy memories Toward this goal, we focus o the memories of the form t py j x j,y j 1 ), j1 where y 0 by covetio This model is simple, yet still captures the major challege i updatig iformatio o memory cells such as the issue of spatial depedece of the memory state It is geeral eough to iclude all the aforemetioed models of WOM, geeralized WOM, oisy WOM as special cases More specifically, we cosider the multiple-write oisy memory MWNM) model X 1 X 2 X t, py 1 x 1 )py 2 x 2,y 1 ) py t x t,y t 1 ), Y 1 Y 2 Y t ), as show i Figure 2 A 2 R1,,2 Rt,) code for the MWNM cosists of t message sets [1:2 R1 ],,[1:2 Rt ], t ecoders, where ecoder1 assigs a codewordx 1 m 1) to each message m 1 [1:2 R1 ] at the first write ad US Govermet work ot protected by US copyright 2494

2 M 1 M 2 M t X1 Y1 Ecoder 1 py 1 x 1 ) Decoder 1 X2 Y2 Ecoder 2 py 2 x 2,y 1 ) Decoder 2 Ecoder t X t Fig 2 Y 1 Y t 1 py t x t,y t 1 ) Y t multiple-write oisy memory Decoder t ecoder j [2 : t] assigs a codeword x j m j,yj 1 ) to each message m j [1:2 Rj ] with side iformatio yj 1 at the j-th write, ad t decoders, where decoder j [1: t] assigs a estimate ˆm j or a error e to each received sequece yj at the j-th write The average probability of error is defied as P ) e P{ ˆM 1,, ˆM t ) M 1,,M t )} A rate tuple R 1,,R t ) is said to be achievable for the MWNM if there exists a sequece of 2 R1,,2 Rt,) codes such that lim P e ) 0 The capacity regio C is the closure of all achievable rate tuples R 1,,R t ) The t- write sum-capacity C sum t) of the MWNM is the maximum achievable sum-rate t j1 R j As the mai result of this paper, we will prove the followig theorem i Sectio III Theorem 1: The t-write sum-capacity of the MWNM is C sum t) max [ t IX 1 ;Y 1 )+ IU j ;Y j ) IU j ;Y j 1 )) ], j2 where the maximum is over all pmfs px 1 ) t j2 pu j y j 1 ) ad fuctios x j u j,y j 1 ) for j [2:t] The achievability proof follows directly by usig the Gelfad Pisker codig scheme [7] The mai challeges i establishig the sum-capacity lie i the proof of the coverse Note that here the chael state sequece is the output of the previous write ad hece may ot be iid over space Such spatial correlatio i the state sequece makes this problem differet from the stadard setup of chaels with discrete memoryless state However, i our proof of the coverse, by carefully combiig the stadard techiques such as Fao s iequality, Csiszár sum idetity [4], ad auxiliary radom variables idetificatio, we ca cacel out such correlatio cumulatively I additio to the sum-capacity, three outer bouds o the capacity regio are established i Sectios II ad III We discuss their optimality i special cases The ier ad outer bouds recover all existig capacity results Heegard [8] cosidered a more geeral memory model t j1 py j,s j x j,s j 1 ), where s j, x j, ad y j deote the chael state, iput, ad output at the j-th write, respectively ˆM 1 ˆM 2 ˆM t However, as a cost of geerality, it is extremely hard to characterize the capacity regio or eve the sum-capacity The capacity regio of Heegard s model remais ope after more tha three decades Recetly, the phase-chage memory PCM) has received cosiderable attetio as the ext geeratio o-volatile solid-state memory techology A PCM cell stores data usig its amorphous ad crystallie states The cell chages betwee these two states through high temperature However, due to the sesitivity of the cells to temperature, it is importat to balace the heat i both time ad space whe programmig the cells Several models have bee proposed to study the PCM Jiag, Bruck, ad Li [10] itroduced the α, β, p) space-time costraied codes for the PCM, where each segmet of β cells ca be programmed at most p times i ay α cosecutive writes, which ca be viewed as a special case of Heegard s model As a alterative, Ahlswede ad Zhag s model of write-efficiet memory [2] has bee suggested as a good model for the PCM [11] I their model, cost fuctios are itroduced based o the assumptio that trasitios from 0 to 1 or 1 to 0 are costly, while writig the same symbol is cheap These two models differ from our MWNM model i that the ecoder may have oly partial access to the state iformatio ad that cost fuctios are itroduced Noetheless, we will briefly discuss how our results o the MWNM ca be applied to the models for phase-chage memories i Sectio IV II TWO-WRITE MEMORY We establish three outer bouds o the capacity regio The sum-capacity of the MWNM will be established alog the way For brevity, we first cosider the 2-write case i this sectio ad provide its extesio to t writes i Sectio III We first recall the followig Propositio 21 Special case of [8, Theorem 2]): A rate pair R 1,R 2 ) is achievable for the 2-write MWNM py 1 x 1 )py 2 x 2,y 1 ) if R 1 IX 1 ;Y 1 ), R 2 IU 2 ;Y 2 ) IU 2 ;Y 1 ) for some pmf px 1 )pu 2 y 1 ) ad fuctio x 2 u 2,y 1 ) Remark 21: This ier boud does ot become larger if we evaluate it for geeral pmf px 1 )pu 2,x 2 y 1,x 1 ) To see this, we first ote that the ier boud does ot become larger if evaluated over px 1 )pu 2,x 2 y 1 ), which follows similar argumets as i the Gelfad Pisker codig scheme The, it suffices to show for ay rate pair IX 1 ;Y 1 ),IU 2 ;Y 2 ) IU 2 ;Y 1 )) attaied by p x 1 )p u 2,x 2 y 1,x 1 ), there exist some pmf px 1 ) pu 2,x 2 y 1 ) attaiig the same rate pair For ay p x 1 )p u 2,x 2 y 1,x 1 ), let px 1 ) p x 1 ) ad pu 2,x 2 y 1 ) x 1 p u 2,x 2 y 1,x 1 )px 1 y 1 ) p py 1 x 1 )p x 1 ) u 2,x 2 y 1,x 1 ) x 1 x 1 py 1 x 1 )p x 1 ) 2495

3 The, p x 1,y 1 ) px 1,y 1 ),p u 2,y 2 ) pu 2,y 2 ), ad p u 2,y 1 ) pu 2,y 1 ), which fully determie the achievable rate pair IX 1 ;Y 1 ),IU 2 ;Y 2 ) IU 2 ;Y 1 )) We establish three outer bouds o the capacity regio Propositio 22: If a rate pair R 1,R 2 ) is achievable for the 2-write MWNMpy 1 x 1 )py 2 x 2,y 1 ), the it must satisfy R 1 IX 1 ;Y 1 ), R 1 +R 2 IX 1 ;Y 1 )+IU 2 ;Y 2 ) IU 2 ;Y 1 ) for some pmf px 1 )pu 2,x 2 y 1,x 1 ) Proof: To establish the first iequality, cosider R 1 HM 1 ) IM 1 ;Y1 )+ǫ IM 1 ;Y 1i Y1,i+1)+ǫ a) b) IM1,Y 1,i+1 ;Y 1i) IY 1,i+1 ;Y 1i) ) +ǫ IM1,Y 1,i+1,X 1i;Y 1i ) IY 1,i+1 ;Y 1i) ) +ǫ IX1i ;Y 1i ) IY 1,i+1;Y 1i ) ) +ǫ 1) IX 1i ;Y 1i )+ǫ, where a) follows sice X 1i is a fuctio of M 1 ad b) follows sice M 1,Y1,i+1 ) X 1i Y 1i form a Markov chai For the secod iequality, the key is to idetify U 2i such that U 2i X 2i,Y 1i ) Y 2i form a Markov chai for each i [1:] Let U 2i M 2,Y2 i 1,Y1,i+1 ) The R 2 IM 2 ;Y 2i Y2 i 1 )+ǫ a) IM 2,Y i 1 2 ;Y 2i )+ǫ IM2,Y i 1 2,Y 1,i+1;Y 2i ) IY1,i+1 ;Y 2i M 2,Y2 i 1 ) ) +ǫ IM2,Y2 i 1,Y1,i+1 ;Y 2i) IY 1i ;Y2 i 1 M 2,Y1,i+1) ) +ǫ IM2,Y2 i 1,Y1,i+1;Y 2i ) IY 1i ;Y2 i 1,M 2,Y1,i+1)+IY 1i ;Y1,i+1) ) +ǫ IU2i ;Y 2i ) IU 2i ;Y 1i ) +IY 1i ;Y 1,i+1) ) +ǫ, 2) where a) follows from the Csiszár sum idetity Now combiig 1) ad 2), we have the boud for sum-rate R 1 +R 2 ) IX1i ;Y 1i )+IU 2i ;Y 2i ) IU 2i ;Y 1i ) ) +ǫ Usig the stadard time sharig radom variable ad takig completes the proof of Propositio 22 Remark 22: The sum-rate outer boud i Propositio 2 together with the ier boud i Propositio 21 establishes the 2-write sum-capacity C sum 2) max IX 1 ;Y 1 )+IU 2 ;Y 2 ) IU 2 ;Y 1 ) ), where the maximum is over all pmfs px 1 )pu 2 y 1 ) ad fuctio x 2 u 2,y 1 ) Remark 23: The outer boud i Propositio 22 is tight for ay sum-rate R 1 +λr 2 with λ [0,1] ad maxr 1 +λr 2 ) max [ IX 1 ;Y 1 )+λiu 2 ;Y 2 ) IU 2 ;Y 1 )) ], where the maximum is over all pmfs px 1 )pu 2 y 1 ) ad fuctios x 2 u 2,y 1 ) We ow preset the secod outer boud Propositio 23: If a rate pair R 1,R 2 ) is achievable for the 2-write MWNMpy 1 x 1 )py 2 x 2,y 1 ), the it must satisfy R 1 IX 1 ;Y 1 ), ad R 2 IU 2 ;Y 2 Y 1 ) for some px 1 )pu 2,x 2 y 1,x 1 ) Proof: The first iequality is exactly the same as i Propositio 22 To establish the secod iequality, choose U 2i M 2,Y2 i 1,Y1,i+1 Y 2i, i [1:], as desired Cosider,Y i 1 1 ) AgaiU 2i X 2i,Y 1i ) R 2 IM 2 ;Y2 Y 1 )+ǫ IM 2 ;Y 2i Y2 i 1,Y1 )+ǫ IM 2,Y i 1 2,Y 1,i+1,Y i 1 1 ;Y 2i Y 1i )+ǫ IU 2i ;Y 2i Y 1i )+ǫ Usig stadard time sharig radom variable ad takig completes the proof Remark 24: The auxiliary radom variable U 2 idetified i Propositio 23 is differet from the oe i Propositio 22 As a result, we do ot have a otrivial boud o the sum-rate i Propositio 23 Remark 25: The outer boud i Propositio 23 is tight whe py j x j,y j 1 ) is determiistic, ie, the geeralized WOM Takig U 2 Y 2, we have R 1 IX 1 ;Y 1 ) HY 1 ) ad R 2 IU 2 ;Y 2 Y 1 ) HY 2 Y 1 ) for some px 1 )px 2 y 1 ), which recovers the capacity regio of the geeralized write-oce memories established by Fu ad Vick [6] This outer boud follows essetially from the fact that the capacity whe side iformatio is available oly at the ecoder is upper bouded by the capacity whe side iformatio is available at both the ecoder ad the decoder I the determiistic case, these two capacities coicide We ow preset the third outer boud 2496

4 Propositio 24: If a rate pair R 1,R 2 ) is achievable for the 2-write MWNMpy 1 x 1 )py 2 x 2,y 1 ), the it must satisfy R 1 IU 1 ;Y 1 ), R 2 IU 2 ;Y 2 Y 1,U 1 ), R 1 +R 2 IU 1 ;Y 1 )+IU 2 ;Y 2 ) IU 2 ;Y 1 ) for some pu 1,x 1 )pu 2,x 2 y 1,u 1,x 1 ) Proof: I this outer boud, we itroduce a ew auxiliary radom variable U 1 To establish the first iequality, we expady1 differetly way from the previous proofs Idetify U 1i M 1,Y1 i 1 ) The R 1 a) IM 1 ;Y 1i Y i 1 1 )+ǫ IM1,Y i 1 1 ;Y 1i ) IY i 1 1 ;Y 1i ) ) +ǫ IU1i ;Y 1i ) IY 1,i+1;Y 1i ) ) +ǫ, 3) where a) follows from the fact that i 1 IY1 ;Y 1i ) HY 1i) HY1 ) IY 1,i+1 ;Y 1i) To establish the secod iequality, idetify U 2i M 2,Y2 i 1,Y1,i+1 ) The R 2 HM 2 Y1,M 1 ) IM 2 ;Y2 Y 1,M 1)+ǫ IM 2 ;Y 2i Y i 1 2,Y 1,M 1)+ǫ IM 2,Y i 1 2,Y 1,i+1;Y 2i Y 1i,M 1,Y i 1 1 )+ǫ IU 2i,Y 2i Y 1i,U 1i )+ǫ Sice we use the same U 2i as i Propositio 22, the iequality i 2) cotiues to hold Combiig 3) ad 2) establishes the third iequality R 1 +R 2 ) IU1i ;Y 1i )+IU 2i ;Y 2i ) IU 2i ;Y 1i ) ) Usig the stadard time sharig radom variable completes the proof of Propositio 24 Remark 26: The outer boud i Propositio 24 is tight both for ay sum-rate R 1 + λr 2, where λ [0,1], ad for the determiistic case Takig U 1 X 1, this outer boud recovers the outer boud i Propositio 22 I the determiistic case, this outer boud recovers the outer boud i Propositio 23 However, it is ot clear whether this is true i geeral III EXTENSION TO MORE THAN TWO WRITES The ier boud ad outer bouds for the 2-write MWNM exted aturally to the geeral t-write MWNM Propositio 31 Special case of [8, Theorem 2]): A rate tuple R 1,,R t ) is achievable for the t-write MWNM py 1 x 1 ) t j2 py j x j,y j 1 ) if R 1 IX 1 ;Y 1 ), R 2 IU 2 ;Y 2 ) IU 2 ;Y 1 ), R t IU t ;Y t ) IU t ;Y t 1 ) for some pmf px 1 ) t j2 pu j y j 1 ) ad fuctios x j u j,y j 1 ) for j [2:t] Remark 31: For a similar reaso as i Remark 21, this ier boud does ot become larger whe evaluated for geeral pmfs pu 1,x 1 ) t j2 pu j,x j y j 1,u j 1,x j 1 ) We ow exted Propositio 22 to the t-write MWNM Propositio 32: If a rate tuple R 1,,R t ) is achievable for the t-write MWM py 1 x 1 ) t j2 py j x j,y j 1 ), the it must satisfy the coditios R 1 IU 1 ;Y 1 ), R 1 +R 2 IU 1 ;Y 1 )+IU 2 ;Y 2 ) IU 2 ;Y 1 ), t R 1 + +R t IU 1 ;Y 1 )+ IUj ;Y j ) IU j ;Y j 1 ) ) j2 for some pu 1,x 1 ) t j2 pu j,x j y j 1,u j 1,x j 1 ) Proof: To boud the sum-rate, the key is still to cacel the oegative terms i 1 IYj 1 ;Y j 1,i) We play a similar trick as i the proof of Propositio 24 to cacel them cumulatively For each j [2 : t], idetify U ji M j,y i 1 j,yj 1,i+1 ) ad assume Y0 We have R j IM j ;Y ji Yj i 1 )+ǫ IMj,Y i 1 j,yj 1,i+1;Y ji ) IY i 1 j ;Y ji ) IYj 1,i+1;Y ji M j,y i 1 j ) ) +ǫ IMj,Y i 1 j,yj 1,i+1;Y ji ) IY i 1 j ;Y ji ) IY j 1,i ;Y i 1 j M j,yj 1,i+1) ) +ǫ IMj,Yj i 1,Yj 1,i+1 ;Y ji)+iyj 1,i+1 ;Y j 1,i) IY j 1,i ;Y i 1 j,m j,y IUji ;Y ji ) IU ji ;Y j 1,i ) a) j 1,i+1) IY i 1 j ;Y ji ) ) +ǫ +IYj 1,i+1 ;Y j 1,i) IYj i 1 ;Y ji ) ) +ǫ IUji ;Y ji ) IU ji ;Y j 1,i ) +IY i 1 j 1 ;Y j 1,i) IY i 1 j ;Y ji ) ) +ǫ, where a) follows from the fact IY j 1,i+1 ;Y j 1,i) HY j 1,i) HYj 1 ) i 1 IYj 1 ;Y j 1,i) It ca be readily observed that the oegative terms i 1 IYj 1 ;Y j 1,i),j 2,,t, telescope ad thus will be caceled i the sum-rate Usig the stadard time sharig radom variable ad takig completes the proof 2497

5 Remark 32: The ier boud i Propositio 31 ad the outer boud i Propositio 32 are tight for sum-rate, which is the mai result stated i Theorem 1 We ow exted Propositio 23 to the t-write MWNM Propositio 33: If a rate tuple R 1,,R t ) is achievable for the t-write MWNM py 1 x 1 ) t j2 py j x j,y j 1 ), the it must satisfy the iequalities R 1 IU 1 ;Y 1 ), R 2 IU 2 ;Y 2 Y 1 ), R t IU t ;Y t Y t 1 ) for some pu 1,x 1 ) t j2 pu j,x j y j 1,u j 1,x j 1 ) Proof sketch: The proof follows similar argumets as i Propositio 23 with U ji M j,yj i 1,Yj 1,i+1 i 1,Yj 1 ) for j [1:t] ad Y0 Remark 33: This boud is tight for the determiistic case We ow exted Propositio 24 to the t-write MWNM Propositio 34: If a rate tuple R 1,,R t ) is achievable for the t-write MWNM py 1 x 1 ) t j2 py j x j,y j 1 ), the it must satisfy the iequalities R 1 IU 1 ;Y 1 ), R 2 IU 2 ;Y 2 Y 1,U 1 ), R t IU t ;Y t Y t 1,U t 1 ), R 1 +R 2 IU 1 ;Y 1 )+IU 2 ;Y 2 ) IU 2 ;Y 1 ), t R 1 + +R t IU 1 ;Y 1 )+ IUj ;Y j ) IU j ;Y j 1 ) ) j2 for some pu 1,x 1 ) t j2 pu j,x j y j 1,u j 1,x j 1 ) Proof sketch: The proof follows similar argumets as i Propositio 24 with U ji M j,y i 1 j,yj 1,i+1 ) for j [1 : t] ad Y0 Remark 34: This boud is tight for the determiistic chaels ad for the sum-rate of arbitrary chaels IV EXTENSION TO PHASE-CHANGE MEMORY I [10], Jiag, Bruck, ad Li itroduced the α, β, p) space-time costraied codes for the PCM Ulike i the MWNM, there is a hidde memory state that is ot available at the ecoder Thus, to fid the sum-capacity for rewritig o such memories, we look ito Heegard s model t j1 py j,s j+1 x j,s j ), where S 1 for covetio Throughout our proof of the coverse, the techiques we used are Fao s iequality, Csiszár sum idetity, ad the chai rule for mutual iformatio, which are valid for Heegard s model as well The oly differece that eeds to be checked is whether our idetificatio of auxiliary radom variables still guaratees the right Markov chai coditio U ji X ji,s ji ) Y ji,s j+1,i ) for i [1:],j [2:t] This ca be verified for all our three idetificatios Thus Propositio ca be readily exteded to Heegard s geeral model I particular, the sum-capacity is upper bouded as C sum t) max [ t IX 1 ;Y 1 )+ IU j ;Y j ) IU j ;Y j 1 )) ], j2 where the maximum is over px 1 ) t j2 pu j,x j s j ), ad lower bouded usig the same expressio as C sum t) max [ t IX 1 ;Y 1 )+ IU j ;Y j ) IU j ;Y j 1 )) ], j2 except that the maximum is over px 1 ) t j2 pu j,x j y j 1 ) These two bouds ca be differet i geeral, however I [2], Ahlswede ad Zhag proposed the write-efficiet memory They itroduced a cost fuctio to characterize whether there is a chage i a memory cell Motivated by this result, we ca icorporate a cost fuctio bx,s) i the MWNM model By the mootoicity, cocavity, ad cotiuity of the capacity-cost fuctio C sum t,b) i B ad the typicality average lemma [1], our results o the MWNM model ca be exteded to the sum-capacity with cost costrait B as C sum t,b) max [ IX 1 ;Y 1 )+ t IU j ;Y j ) IU j ;Y j 1 )) ], j2 where the maximum is over px 1 ) t j2 pu j,x j y j 1 ) such that 1/t) E [ t j1 bx j,y j 1 ) ] B REFERENCES [1] A El Gamal ad Y-H Kim 2010) Lecture otes o etwork iformatio theory [Olie] Available: [2] R Ahlswede ad Z Zhag, Codig for write-efficiet memory, Iformatio ad computatio, vol 83, pp 80 97, 1989 [3] G D Cohe, P Godlewski, ad F Merkx, Liear biary code for write-oce memories, IEEE Tras Iform Theory, vol 32, o 5, pp , 1986 [4] I Csiszár ad J Körer, Broadcast chaels with cofidetial messages, IEEE Tras Iform Theory, vol 24, pp , 1978 [5] A Fiat ad A Shamir, Geeralized write-oce memories, IEEE Tras Iform Theory, vol 30, o 1, pp , 1984 [6] F W Fu ad A J H Vick, O the capaicty of geeralized writeoce memory with state trasitios described by a arbitrary directed acyclic graph, IEEE Tras Iform Theory, vol 45, o 1, pp , 1999 [7] S I Gelfad ad M S Pisker, Codig for chaels with radom parameters, Problem of Cotrol ad Iformatio Theory, vol 9, o 1, pp 19 31, 1980 [8] C Heegard, O the capaicty of permaet memory, IEEE Tras Iform Theory, vol IT-31, o 1, pp 34 42, 1985 [9] A Jiag ad J Bruck, Joit codig for flash memory storage, i Proc IEEE Iteratioal Symposium o Iformatio Theory, Toroto, Caada, 2008, pp [10] A Jiag, J Bruck, ad H Li, Costraied codes for phase-chage memories, i Proc IEEE Iformatio Theory Workshop ITW), Dubli, Irelad, 2010 [11] L A Lastras-Motao, M Fraceschii, T Mittelholzer, J Karidis, ad M Wegma, O the lifetime of multilevel memories, i Proc IEEE Iteratioal Symposium o Iformatio Theory, Seoul, Korea, 2009, pp [12] R L Rivest ad A Shamir, How to reuse a write-oce memory, Iformatio ad Cotrol, vol 55, o 1-3, pp 1 19, 1982 [13] J K Wolf, A D Wyer, J Ziv, ad J Körer, Codig for a write-oce memory, AT&T Bell Labs Tech J, vol 63, o 6, pp , 1984 [14] E Yaakobi, P H Siegel, A Vardy, ad J K Wolf, Multiple errorcorrectig WOM-codes, i Proc IEEE Iteratioal Symposium o Iformatio Theory, Austi, TX, 2010, pp [15] G Zémor ad G D Cohe, Error-correctig WOM-codes, IEEE Tras Iform Theory, vol 37, o 3, pp , May

Lecture 11: Channel Coding Theorem: Converse Part

Lecture 11: Channel Coding Theorem: Converse Part EE376A/STATS376A Iformatio Theory Lecture - 02/3/208 Lecture : Chael Codig Theorem: Coverse Part Lecturer: Tsachy Weissma Scribe: Erdem Bıyık I this lecture, we will cotiue our discussio o chael codig

More information

Information Theory Tutorial Communication over Channels with memory. Chi Zhang Department of Electrical Engineering University of Notre Dame

Information Theory Tutorial Communication over Channels with memory. Chi Zhang Department of Electrical Engineering University of Notre Dame Iformatio Theory Tutorial Commuicatio over Chaels with memory Chi Zhag Departmet of Electrical Egieerig Uiversity of Notre Dame Abstract A geeral capacity formula C = sup I(; Y ), which is correct for

More information

Entropies & Information Theory

Entropies & Information Theory Etropies & Iformatio Theory LECTURE I Nilajaa Datta Uiversity of Cambridge,U.K. For more details: see lecture otes (Lecture 1- Lecture 5) o http://www.qi.damtp.cam.ac.uk/ode/223 Quatum Iformatio Theory

More information

Lecture 6: Source coding, Typicality, and Noisy channels and capacity

Lecture 6: Source coding, Typicality, and Noisy channels and capacity 15-859: Iformatio Theory ad Applicatios i TCS CMU: Sprig 2013 Lecture 6: Source codig, Typicality, ad Noisy chaels ad capacity Jauary 31, 2013 Lecturer: Mahdi Cheraghchi Scribe: Togbo Huag 1 Recap Uiversal

More information

Lecture 14: Graph Entropy

Lecture 14: Graph Entropy 15-859: Iformatio Theory ad Applicatios i TCS Sprig 2013 Lecture 14: Graph Etropy March 19, 2013 Lecturer: Mahdi Cheraghchi Scribe: Euiwoog Lee 1 Recap Bergma s boud o the permaet Shearer s Lemma Number

More information

The Maximum-Likelihood Decoding Performance of Error-Correcting Codes

The Maximum-Likelihood Decoding Performance of Error-Correcting Codes The Maximum-Lielihood Decodig Performace of Error-Correctig Codes Hery D. Pfister ECE Departmet Texas A&M Uiversity August 27th, 2007 (rev. 0) November 2st, 203 (rev. ) Performace of Codes. Notatio X,

More information

Lecture 7: Channel coding theorem for discrete-time continuous memoryless channel

Lecture 7: Channel coding theorem for discrete-time continuous memoryless channel Lecture 7: Chael codig theorem for discrete-time cotiuous memoryless chael Lectured by Dr. Saif K. Mohammed Scribed by Mirsad Čirkić Iformatio Theory for Wireless Commuicatio ITWC Sprig 202 Let us first

More information

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1 EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum

More information

Asymptotic Coupling and Its Applications in Information Theory

Asymptotic Coupling and Its Applications in Information Theory Asymptotic Couplig ad Its Applicatios i Iformatio Theory Vicet Y. F. Ta Joit Work with Lei Yu Departmet of Electrical ad Computer Egieerig, Departmet of Mathematics, Natioal Uiversity of Sigapore IMS-APRM

More information

Universal source coding for complementary delivery

Universal source coding for complementary delivery SITA2006 i Hakodate 2005.2. p. Uiversal source codig for complemetary delivery Akisato Kimura, 2, Tomohiko Uyematsu 2, Shigeaki Kuzuoka 2 Media Iformatio Laboratory, NTT Commuicatio Sciece Laboratories,

More information

Lecture 7: October 18, 2017

Lecture 7: October 18, 2017 Iformatio ad Codig Theory Autum 207 Lecturer: Madhur Tulsiai Lecture 7: October 8, 207 Biary hypothesis testig I this lecture, we apply the tools developed i the past few lectures to uderstad the problem

More information

Multiterminal Source Coding with an Entropy-Based Distortion Measure

Multiterminal Source Coding with an Entropy-Based Distortion Measure 20 IEEE Iteratioal Symposium o Iformatio Theory Proceedigs Multitermial Source Codig with a Etropy-Based Distortio Measure Thomas A. Courtade ad Richard D. Wesel Departmet of Electrical Egieerig Uiversity

More information

Lecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound

Lecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound Lecture 7 Ageda for the lecture Gaussia chael with average power costraits Capacity of additive Gaussia oise chael ad the sphere packig boud 7. Additive Gaussia oise chael Up to this poit, we have bee

More information

arxiv: v1 [cs.it] 13 Jul 2012

arxiv: v1 [cs.it] 13 Jul 2012 O the Sum Capacity of the Discrete Memoryless Iterferece Chael with Oe-Sided Weak Iterferece ad Mixed Iterferece Fagfag Zhu ad Biao Che Syracuse Uiversity Departmet of EECS Syracuse, NY 3244 Email: fazhu{biche}@syr.edu

More information

A Partial Decode-Forward Scheme For A Network with N relays

A Partial Decode-Forward Scheme For A Network with N relays A Partial Decode-Forward Scheme For A etwork with relays Yao Tag ECE Departmet, McGill Uiversity Motreal, QC, Caada Email: yaotag2@mailmcgillca Mai Vu ECE Departmet, Tufts Uiversity Medford, MA, USA Email:

More information

Lecture 15: Strong, Conditional, & Joint Typicality

Lecture 15: Strong, Conditional, & Joint Typicality EE376A/STATS376A Iformatio Theory Lecture 15-02/27/2018 Lecture 15: Strog, Coditioal, & Joit Typicality Lecturer: Tsachy Weissma Scribe: Nimit Sohoi, William McCloskey, Halwest Mohammad I this lecture,

More information

SUCCESSIVE INTERFERENCE CANCELLATION DECODING FOR THE K -USER CYCLIC INTERFERENCE CHANNEL

SUCCESSIVE INTERFERENCE CANCELLATION DECODING FOR THE K -USER CYCLIC INTERFERENCE CHANNEL Joural of Theoretical ad Applied Iformatio Techology 31 st December 212 Vol 46 No2 25-212 JATIT & LLS All rights reserved ISSN: 1992-8645 wwwatitorg E-ISSN: 1817-3195 SCCESSIVE INTERFERENCE CANCELLATION

More information

Rank Modulation with Multiplicity

Rank Modulation with Multiplicity Rak Modulatio with Multiplicity Axiao (Adrew) Jiag Computer Sciece ad Eg. Dept. Texas A&M Uiversity College Statio, TX 778 ajiag@cse.tamu.edu Abstract Rak modulatio is a scheme that uses the relative order

More information

A New Achievability Scheme for the Relay Channel

A New Achievability Scheme for the Relay Channel A New Achievability Scheme for the Relay Chael Wei Kag Seur Ulukus Departmet of Electrical ad Computer Egieerig Uiversity of Marylad, College Park, MD 20742 wkag@umd.edu ulukus@umd.edu October 4, 2007

More information

Shannon s noiseless coding theorem

Shannon s noiseless coding theorem 18.310 lecture otes May 4, 2015 Shao s oiseless codig theorem Lecturer: Michel Goemas I these otes we discuss Shao s oiseless codig theorem, which is oe of the foudig results of the field of iformatio

More information

Information Theory and Statistics Lecture 4: Lempel-Ziv code

Information Theory and Statistics Lecture 4: Lempel-Ziv code Iformatio Theory ad Statistics Lecture 4: Lempel-Ziv code Łukasz Dębowski ldebowsk@ipipa.waw.pl Ph. D. Programme 203/204 Etropy rate is the limitig compressio rate Theorem For a statioary process (X i)

More information

IET Commun., 2009, Vol. 3, Iss. 1, pp doi: /iet-com: & The Institution of Engineering and Technology 2009

IET Commun., 2009, Vol. 3, Iss. 1, pp doi: /iet-com: & The Institution of Engineering and Technology 2009 Published i IET Commuicatios Received o 4th Jue 28 Revised o 3th July 28 doi: 1.149/iet-com:28373 ISSN 1751-8628 Symmetric relayig based o partial decodig ad the capacity of a class of relay etworks L.

More information

Information-based Feature Selection

Information-based Feature Selection Iformatio-based Feature Selectio Farza Faria, Abbas Kazeroui, Afshi Babveyh Email: {faria,abbask,afshib}@staford.edu 1 Itroductio Feature selectio is a topic of great iterest i applicatios dealig with

More information

Infinite Sequences and Series

Infinite Sequences and Series Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet

More information

Multiterminal source coding with complementary delivery

Multiterminal source coding with complementary delivery Iteratioal Symposium o Iformatio Theory ad its Applicatios, ISITA2006 Seoul, Korea, October 29 November 1, 2006 Multitermial source codig with complemetary delivery Akisato Kimura ad Tomohiko Uyematsu

More information

Precise Rates in Complete Moment Convergence for Negatively Associated Sequences

Precise Rates in Complete Moment Convergence for Negatively Associated Sequences Commuicatios of the Korea Statistical Society 29, Vol. 16, No. 5, 841 849 Precise Rates i Complete Momet Covergece for Negatively Associated Sequeces Dae-Hee Ryu 1,a a Departmet of Computer Sciece, ChugWoo

More information

Refinement of Two Fundamental Tools in Information Theory

Refinement of Two Fundamental Tools in Information Theory Refiemet of Two Fudametal Tools i Iformatio Theory Raymod W. Yeug Istitute of Network Codig The Chiese Uiversity of Hog Kog Joit work with Siu Wai Ho ad Sergio Verdu Discotiuity of Shao s Iformatio Measures

More information

Information Theory and Coding

Information Theory and Coding Sol. Iformatio Theory ad Codig. The capacity of a bad-limited additive white Gaussia (AWGN) chael is give by C = Wlog 2 ( + σ 2 W ) bits per secod(bps), where W is the chael badwidth, is the average power

More information

Sequences. Notation. Convergence of a Sequence

Sequences. Notation. Convergence of a Sequence Sequeces A sequece is essetially just a list. Defiitio (Sequece of Real Numbers). A sequece of real umbers is a fuctio Z (, ) R for some real umber. Do t let the descriptio of the domai cofuse you; it

More information

Convergence of random variables. (telegram style notes) P.J.C. Spreij

Convergence of random variables. (telegram style notes) P.J.C. Spreij Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space

More information

Symmetric Two-User Gaussian Interference Channel with Common Messages

Symmetric Two-User Gaussian Interference Channel with Common Messages Symmetric Two-User Gaussia Iterferece Chael with Commo Messages Qua Geg CSL ad Dept. of ECE UIUC, IL 680 Email: geg5@illiois.edu Tie Liu Dept. of Electrical ad Computer Egieerig Texas A&M Uiversity, TX

More information

Lesson 10: Limits and Continuity

Lesson 10: Limits and Continuity www.scimsacademy.com Lesso 10: Limits ad Cotiuity SCIMS Academy 1 Limit of a fuctio The cocept of limit of a fuctio is cetral to all other cocepts i calculus (like cotiuity, derivative, defiite itegrals

More information

Inequalities for Entropies of Sets of Subsets of Random Variables

Inequalities for Entropies of Sets of Subsets of Random Variables Iequalities for Etropies of Sets of Subsets of Radom Variables Chao Tia AT&T Labs-Research Florham Par, NJ 0792, USA. tia@research.att.com Abstract Ha s iequality o the etropy rates of subsets of radom

More information

IP Reference guide for integer programming formulations.

IP Reference guide for integer programming formulations. IP Referece guide for iteger programmig formulatios. by James B. Orli for 15.053 ad 15.058 This documet is iteded as a compact (or relatively compact) guide to the formulatio of iteger programs. For more

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak

More information

w (1) ˆx w (1) x (1) /ρ and w (2) ˆx w (2) x (2) /ρ.

w (1) ˆx w (1) x (1) /ρ and w (2) ˆx w (2) x (2) /ρ. 2 5. Weighted umber of late jobs 5.1. Release dates ad due dates: maximimizig the weight of o-time jobs Oce we add release dates, miimizig the umber of late jobs becomes a sigificatly harder problem. For

More information

Optimally Sparse SVMs

Optimally Sparse SVMs A. Proof of Lemma 3. We here prove a lower boud o the umber of support vectors to achieve geeralizatio bouds of the form which we cosider. Importatly, this result holds ot oly for liear classifiers, but

More information

Entropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP

Entropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP Etropy ad Ergodic Theory Lecture 5: Joit typicality ad coditioal AEP 1 Notatio: from RVs back to distributios Let (Ω, F, P) be a probability space, ad let X ad Y be A- ad B-valued discrete RVs, respectively.

More information

Lecture 9: Hierarchy Theorems

Lecture 9: Hierarchy Theorems IAS/PCMI Summer Sessio 2000 Clay Mathematics Udergraduate Program Basic Course o Computatioal Complexity Lecture 9: Hierarchy Theorems David Mix Barrigto ad Alexis Maciel July 27, 2000 Most of this lecture

More information

STAT Homework 1 - Solutions

STAT Homework 1 - Solutions STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better

More information

The source coding game with a cheating switcher

The source coding game with a cheating switcher The source codig game with a cheatig switcher Hari Palaiyaur hpalaiya@eecsberkeleyedu Cheg Chag cchag@eecsberkeleyedu Departmet of Electrical Egieerig ad Computer Scieces Uiversity of Califoria at Berkeley

More information

TEACHER CERTIFICATION STUDY GUIDE

TEACHER CERTIFICATION STUDY GUIDE COMPETENCY 1. ALGEBRA SKILL 1.1 1.1a. ALGEBRAIC STRUCTURES Kow why the real ad complex umbers are each a field, ad that particular rigs are ot fields (e.g., itegers, polyomial rigs, matrix rigs) Algebra

More information

CSE 1400 Applied Discrete Mathematics Number Theory and Proofs

CSE 1400 Applied Discrete Mathematics Number Theory and Proofs CSE 1400 Applied Discrete Mathematics Number Theory ad Proofs Departmet of Computer Scieces College of Egieerig Florida Tech Sprig 01 Problems for Number Theory Backgroud Number theory is the brach of

More information

A 2nTH ORDER LINEAR DIFFERENCE EQUATION

A 2nTH ORDER LINEAR DIFFERENCE EQUATION A 2TH ORDER LINEAR DIFFERENCE EQUATION Doug Aderso Departmet of Mathematics ad Computer Sciece, Cocordia College Moorhead, MN 56562, USA ABSTRACT: We give a formulatio of geeralized zeros ad (, )-discojugacy

More information

ECE 564/645 - Digital Communication Systems (Spring 2014) Final Exam Friday, May 2nd, 8:00-10:00am, Marston 220

ECE 564/645 - Digital Communication Systems (Spring 2014) Final Exam Friday, May 2nd, 8:00-10:00am, Marston 220 ECE 564/645 - Digital Commuicatio Systems (Sprig 014) Fial Exam Friday, May d, 8:00-10:00am, Marsto 0 Overview The exam cosists of four (or five) problems for 100 (or 10) poits. The poits for each part

More information

An Introduction to Randomized Algorithms

An Introduction to Randomized Algorithms A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis

More information

Hybrid Coding for Gaussian Broadcast Channels with Gaussian Sources

Hybrid Coding for Gaussian Broadcast Channels with Gaussian Sources Hybrid Codig for Gaussia Broadcast Chaels with Gaussia Sources Rajiv Soudararaja Departmet of Electrical & Computer Egieerig Uiversity of Texas at Austi Austi, TX 7871, USA Email: rajivs@mailutexasedu

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.

More information

On Random Line Segments in the Unit Square

On Random Line Segments in the Unit Square O Radom Lie Segmets i the Uit Square Thomas A. Courtade Departmet of Electrical Egieerig Uiversity of Califoria Los Ageles, Califoria 90095 Email: tacourta@ee.ucla.edu I. INTRODUCTION Let Q = [0, 1] [0,

More information

Definitions and Theorems. where x are the decision variables. c, b, and a are constant coefficients.

Definitions and Theorems. where x are the decision variables. c, b, and a are constant coefficients. Defiitios ad Theorems Remember the scalar form of the liear programmig problem, Miimize, Subject to, f(x) = c i x i a 1i x i = b 1 a mi x i = b m x i 0 i = 1,2,, where x are the decisio variables. c, b,

More information

Product measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.

Product measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014. Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the

More information

Lecture 19: Convergence

Lecture 19: Convergence Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may

More information

7 Sequences of real numbers

7 Sequences of real numbers 40 7 Sequeces of real umbers 7. Defiitios ad examples Defiitio 7... A sequece of real umbers is a real fuctio whose domai is the set N of atural umbers. Let s : N R be a sequece. The the values of s are

More information

A Block Cipher Using Linear Congruences

A Block Cipher Using Linear Congruences Joural of Computer Sciece 3 (7): 556-560, 2007 ISSN 1549-3636 2007 Sciece Publicatios A Block Cipher Usig Liear Cogrueces 1 V.U.K. Sastry ad 2 V. Jaaki 1 Academic Affairs, Sreeidhi Istitute of Sciece &

More information

Fixed-Threshold Polar Codes

Fixed-Threshold Polar Codes Fixed-Threshold Polar Codes Jig Guo Uiversity of Cambridge jg582@cam.ac.uk Albert Guillé i Fàbregas ICREA & Uiversitat Pompeu Fabra Uiversity of Cambridge guille@ieee.org Jossy Sayir Uiversity of Cambridge

More information

On Routing-Optimal Network for Multiple Unicasts

On Routing-Optimal Network for Multiple Unicasts O Routig-Optimal Network for Multiple Uicasts Chu Meg, Athia Markopoulou Abstract I this paper, we cosider etworks with multiple uicast sessios. Geerally, o-liear etwork codig is eeded to achieve the whole

More information

Capacity Theorems for the Finite-State Broadcast Channel with Feedback

Capacity Theorems for the Finite-State Broadcast Channel with Feedback Capacity Theorems for the Fiite-State Broadcast Chael with Feedback Ro abora ad Adrea Goldsmith ept of Electrical Egieerig, Staford Uiversity Abstract We cosider the discrete, time-varyig broadcast chael

More information

4.3 Growth Rates of Solutions to Recurrences

4.3 Growth Rates of Solutions to Recurrences 4.3. GROWTH RATES OF SOLUTIONS TO RECURRENCES 81 4.3 Growth Rates of Solutios to Recurreces 4.3.1 Divide ad Coquer Algorithms Oe of the most basic ad powerful algorithmic techiques is divide ad coquer.

More information

Coding for Noisy Write-Efficient Memories

Coding for Noisy Write-Efficient Memories Coding for oisy Write-Efficient Memories Qing Li Computer Sci. & Eng. Dept. Texas A & M University College Station, TX 77843 qingli@cse.tamu.edu Anxiao (Andrew) Jiang CSE and ECE Departments Texas A &

More information

6.3 Testing Series With Positive Terms

6.3 Testing Series With Positive Terms 6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial

More information

Last time, we talked about how Equation (1) can simulate Equation (2). We asserted that Equation (2) can also simulate Equation (1).

Last time, we talked about how Equation (1) can simulate Equation (2). We asserted that Equation (2) can also simulate Equation (1). 6896 Quatum Complexity Theory Sept 23, 2008 Lecturer: Scott Aaroso Lecture 6 Last Time: Quatum Error-Correctio Quatum Query Model Deutsch-Jozsa Algorithm (Computes x y i oe query) Today: Berstei-Vazirii

More information

If a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero?

If a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero? 2 Lebesgue Measure I Chapter 1 we defied the cocept of a set of measure zero, ad we have observed that every coutable set is of measure zero. Here are some atural questios: If a subset E of R cotais a

More information

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i

More information

1 Convergence in Probability and the Weak Law of Large Numbers

1 Convergence in Probability and the Weak Law of Large Numbers 36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec

More information

Increasing timing capacity using packet coloring

Increasing timing capacity using packet coloring 003 Coferece o Iformatio Scieces ad Systems, The Johs Hopkis Uiversity, March 4, 003 Icreasig timig capacity usig packet colorig Xi Liu ad R Srikat[] Coordiated Sciece Laboratory Uiversity of Illiois e-mail:

More information

A Hadamard-type lower bound for symmetric diagonally dominant positive matrices

A Hadamard-type lower bound for symmetric diagonally dominant positive matrices A Hadamard-type lower boud for symmetric diagoally domiat positive matrices Christopher J. Hillar, Adre Wibisoo Uiversity of Califoria, Berkeley Jauary 7, 205 Abstract We prove a ew lower-boud form of

More information

CS284A: Representations and Algorithms in Molecular Biology

CS284A: Representations and Algorithms in Molecular Biology CS284A: Represetatios ad Algorithms i Molecular Biology Scribe Notes o Lectures 3 & 4: Motif Discovery via Eumeratio & Motif Represetatio Usig Positio Weight Matrix Joshua Gervi Based o presetatios by

More information

Supplementary Material for Fast Stochastic AUC Maximization with O(1/n)-Convergence Rate

Supplementary Material for Fast Stochastic AUC Maximization with O(1/n)-Convergence Rate Supplemetary Material for Fast Stochastic AUC Maximizatio with O/-Covergece Rate Migrui Liu Xiaoxua Zhag Zaiyi Che Xiaoyu Wag 3 iabao Yag echical Lemmas ized versio of Hoeffdig s iequality, ote that We

More information

Beurling Integers: Part 2

Beurling Integers: Part 2 Beurlig Itegers: Part 2 Isomorphisms Devi Platt July 11, 2015 1 Prime Factorizatio Sequeces I the last article we itroduced the Beurlig geeralized itegers, which ca be represeted as a sequece of real umbers

More information

EE 4TM4: Digital Communications II Information Measures

EE 4TM4: Digital Communications II Information Measures EE 4TM4: Digital Commuicatios II Iformatio Measures Defiitio : The etropy H(X) of a discrete radom variable X is defied by We also write H(p) for the above quatity. Lemma : H(X) 0. H(X) = x X Proof: 0

More information

Math 525: Lecture 5. January 18, 2018

Math 525: Lecture 5. January 18, 2018 Math 525: Lecture 5 Jauary 18, 2018 1 Series (review) Defiitio 1.1. A sequece (a ) R coverges to a poit L R (writte a L or lim a = L) if for each ǫ > 0, we ca fid N such that a L < ǫ for all N. If the

More information

Covert Communication with Noncausal Channel-State Information at the Transmitter

Covert Communication with Noncausal Channel-State Information at the Transmitter Covert Commuicatio with Nocausal Chael-State Iformatio at the Trasmitter Si-Hyeo Lee, Ligog Wag, Ashish Khisti, ad Gregory W. Worell POSTECH, Pohag, South Korea (e-mail:sihyeo@postech.ac.kr ETIS Uiversité

More information

Nonuniform Codes for Correcting Asymmetric Errors

Nonuniform Codes for Correcting Asymmetric Errors 2011 IEEE Iteratioal Symposium o Iformatio Theory Proceedigs Nouiform Codes for Correctig Asymmetric Errors Hogchao Zhou Electrical Egieerig Departmet Califoria Istitute of Techology Pasadea, CA 91125

More information

Hashing and Amortization

Hashing and Amortization Lecture Hashig ad Amortizatio Supplemetal readig i CLRS: Chapter ; Chapter 7 itro; Sectio 7.. Arrays ad Hashig Arrays are very useful. The items i a array are statically addressed, so that isertig, deletig,

More information

Analytic Continuation

Analytic Continuation Aalytic Cotiuatio The stadard example of this is give by Example Let h (z) = 1 + z + z 2 + z 3 +... kow to coverge oly for z < 1. I fact h (z) = 1/ (1 z) for such z. Yet H (z) = 1/ (1 z) is defied for

More information

2000 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes

2000 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes 2000 IEEE. Persoal use of this material is permitted. However, permissio to reprit/republish this material for advertisig or promotioal purposes or for creatig ew collective works for resale or redistributio

More information

Rademacher Complexity

Rademacher Complexity EECS 598: Statistical Learig Theory, Witer 204 Topic 0 Rademacher Complexity Lecturer: Clayto Scott Scribe: Ya Deg, Kevi Moo Disclaimer: These otes have ot bee subjected to the usual scrutiy reserved for

More information

The Choquet Integral with Respect to Fuzzy-Valued Set Functions

The Choquet Integral with Respect to Fuzzy-Valued Set Functions The Choquet Itegral with Respect to Fuzzy-Valued Set Fuctios Weiwei Zhag Abstract The Choquet itegral with respect to real-valued oadditive set fuctios, such as siged efficiecy measures, has bee used i

More information

Lecture 3: August 31

Lecture 3: August 31 36-705: Itermediate Statistics Fall 018 Lecturer: Siva Balakrisha Lecture 3: August 31 This lecture will be mostly a summary of other useful expoetial tail bouds We will ot prove ay of these i lecture,

More information

THE ASYMPTOTIC COMPLEXITY OF MATRIX REDUCTION OVER FINITE FIELDS

THE ASYMPTOTIC COMPLEXITY OF MATRIX REDUCTION OVER FINITE FIELDS THE ASYMPTOTIC COMPLEXITY OF MATRIX REDUCTION OVER FINITE FIELDS DEMETRES CHRISTOFIDES Abstract. Cosider a ivertible matrix over some field. The Gauss-Jorda elimiatio reduces this matrix to the idetity

More information

Chapter 6 Infinite Series

Chapter 6 Infinite Series Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat

More information

Lecture 19. sup y 1,..., yn B d n

Lecture 19. sup y 1,..., yn B d n STAT 06A: Polyomials of adom Variables Lecture date: Nov Lecture 19 Grothedieck s Iequality Scribe: Be Hough The scribes are based o a guest lecture by ya O Doell. I this lecture we prove Grothedieck s

More information

The multiplicative structure of finite field and a construction of LRC

The multiplicative structure of finite field and a construction of LRC IERG6120 Codig for Distributed Storage Systems Lecture 8-06/10/2016 The multiplicative structure of fiite field ad a costructio of LRC Lecturer: Keeth Shum Scribe: Zhouyi Hu Notatios: We use the otatio

More information

1 Hash tables. 1.1 Implementation

1 Hash tables. 1.1 Implementation Lecture 8 Hash Tables, Uiversal Hash Fuctios, Balls ad Bis Scribes: Luke Johsto, Moses Charikar, G. Valiat Date: Oct 18, 2017 Adapted From Virgiia Williams lecture otes 1 Hash tables A hash table is a

More information

6.895 Essential Coding Theory October 20, Lecture 11. This lecture is focused in comparisons of the following properties/parameters of a code:

6.895 Essential Coding Theory October 20, Lecture 11. This lecture is focused in comparisons of the following properties/parameters of a code: 6.895 Essetial Codig Theory October 0, 004 Lecture 11 Lecturer: Madhu Suda Scribe: Aastasios Sidiropoulos 1 Overview This lecture is focused i comparisos of the followig properties/parameters of a code:

More information

Seunghee Ye Ma 8: Week 5 Oct 28

Seunghee Ye Ma 8: Week 5 Oct 28 Week 5 Summary I Sectio, we go over the Mea Value Theorem ad its applicatios. I Sectio 2, we will recap what we have covered so far this term. Topics Page Mea Value Theorem. Applicatios of the Mea Value

More information

Lecture 5: April 17, 2013

Lecture 5: April 17, 2013 TTIC/CMSC 350 Mathematical Toolkit Sprig 203 Madhur Tulsiai Lecture 5: April 7, 203 Scribe: Somaye Hashemifar Cheroff bouds recap We recall the Cheroff/Hoeffdig bouds we derived i the last lecture idepedet

More information

The log-behavior of n p(n) and n p(n)/n

The log-behavior of n p(n) and n p(n)/n Ramauja J. 44 017, 81-99 The log-behavior of p ad p/ William Y.C. Che 1 ad Ke Y. Zheg 1 Ceter for Applied Mathematics Tiaji Uiversity Tiaji 0007, P. R. Chia Ceter for Combiatorics, LPMC Nakai Uivercity

More information

Supplementary Materials for Statistical-Computational Phase Transitions in Planted Models: The High-Dimensional Setting

Supplementary Materials for Statistical-Computational Phase Transitions in Planted Models: The High-Dimensional Setting Supplemetary Materials for Statistical-Computatioal Phase Trasitios i Plated Models: The High-Dimesioal Settig Yudog Che The Uiversity of Califoria, Berkeley yudog.che@eecs.berkeley.edu Jiamig Xu Uiversity

More information

Series III. Chapter Alternating Series

Series III. Chapter Alternating Series Chapter 9 Series III With the exceptio of the Null Sequece Test, all the tests for series covergece ad divergece that we have cosidered so far have dealt oly with series of oegative terms. Series with

More information

The Growth of Functions. Theoretical Supplement

The Growth of Functions. Theoretical Supplement The Growth of Fuctios Theoretical Supplemet The Triagle Iequality The triagle iequality is a algebraic tool that is ofte useful i maipulatig absolute values of fuctios. The triagle iequality says that

More information

SRC Technical Note June 17, Tight Thresholds for The Pure Literal Rule. Michael Mitzenmacher. d i g i t a l

SRC Technical Note June 17, Tight Thresholds for The Pure Literal Rule. Michael Mitzenmacher. d i g i t a l SRC Techical Note 1997-011 Jue 17, 1997 Tight Thresholds for The Pure Literal Rule Michael Mitzemacher d i g i t a l Systems Research Ceter 130 Lytto Aveue Palo Alto, Califoria 94301 http://www.research.digital.com/src/

More information

f(x) dx as we do. 2x dx x also diverges. Solution: We compute 2x dx lim

f(x) dx as we do. 2x dx x also diverges. Solution: We compute 2x dx lim Math 3, Sectio 2. (25 poits) Why we defie f(x) dx as we do. (a) Show that the improper itegral diverges. Hece the improper itegral x 2 + x 2 + b also diverges. Solutio: We compute x 2 + = lim b x 2 + =

More information

sin(n) + 2 cos(2n) n 3/2 3 sin(n) 2cos(2n) n 3/2 a n =

sin(n) + 2 cos(2n) n 3/2 3 sin(n) 2cos(2n) n 3/2 a n = 60. Ratio ad root tests 60.1. Absolutely coverget series. Defiitio 13. (Absolute covergece) A series a is called absolutely coverget if the series of absolute values a is coverget. The absolute covergece

More information

Discrete Mathematics for CS Spring 2007 Luca Trevisan Lecture 22

Discrete Mathematics for CS Spring 2007 Luca Trevisan Lecture 22 CS 70 Discrete Mathematics for CS Sprig 2007 Luca Trevisa Lecture 22 Aother Importat Distributio The Geometric Distributio Questio: A biased coi with Heads probability p is tossed repeatedly util the first

More information

Chapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities

Chapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other

More information

Fundamental Theorem of Algebra. Yvonne Lai March 2010

Fundamental Theorem of Algebra. Yvonne Lai March 2010 Fudametal Theorem of Algebra Yvoe Lai March 010 We prove the Fudametal Theorem of Algebra: Fudametal Theorem of Algebra. Let f be a o-costat polyomial with real coefficiets. The f has at least oe complex

More information

Math 312 Lecture Notes One Dimensional Maps

Math 312 Lecture Notes One Dimensional Maps Math 312 Lecture Notes Oe Dimesioal Maps Warre Weckesser Departmet of Mathematics Colgate Uiversity 21-23 February 25 A Example We begi with the simplest model of populatio growth. Suppose, for example,

More information

5.1 A mutual information bound based on metric entropy

5.1 A mutual information bound based on metric entropy Chapter 5 Global Fao Method I this chapter, we exted the techiques of Chapter 2.4 o Fao s method the local Fao method) to a more global costructio. I particular, we show that, rather tha costructig a local

More information

The Capacity Region of the Degraded Finite-State Broadcast Channel

The Capacity Region of the Degraded Finite-State Broadcast Channel The Capacity Regio of the Degraded Fiite-State Broadcast Chael Ro Dabora ad Adrea Goldsmith Dept. of Electrical Egieerig, Staford Uiversity, Staford, CA Abstract We cosider the discrete, time-varyig broadcast

More information