On Interactive Encoding and Decoding for Distributed Lossless Coding of Individual Sequences

Transcription

1 O Iteractve Ecodg ad Decodg or Dstruted Lossless Codg o Idvdual Sequeces E-Hu Yag ad J Meg Departmet o Electrcal ad Computer Egeerg Uversty o Waterloo Waterloo Otaro N2L 6P6 Emal: {ehyagj4meg}@uwaterlooca Astract Dstruted ear lossless codg o dvdual sequeces X ad Y s cosdered where X ad Y are rst ecoded separately ad the set to a jot decoder Ulke dstruted ear lossless codg o correlated radom sources the jot decoder dstruted codg o dvdual sequeces does ot help at all I other words the mmum umers o ts to e set rom X ad Y respectvely to the jot decoder are the same as two depedet parallel systems where X ad Y are ecoded separately ad decoded separately I ths paper however we show that y usg teractve ecodg ad decodg where the jot decoder s allowed to teract wth oth separate ecoders the mmum umer o total ts to e exchaged etwee the jot decoder ad two separate ecoders or each ad every par o dvdual sequeces X ad Y s the same as the system where X ad Y are jotly ecoded ad the jotly decoded whle X ad Y ca e recovered y the jot decoder a ear lossless maer I INTRODUCTION Cosder dstruted ear lossless codg o two sources X ad Y where X ad Y are rst ecoded separately ad the set to a jot decoder Whe X ad Y are radom correlated ad memoryless sources Slepa ad Wol showed ther semar work [1] that the mmum umer o total ts per symol par eeded to e udrectoally trasmtted rom two separated ecoders to the jot decoder order or X Y to e recovered wth hgh proalty y the jot decoder s stll equal to the jot etropy the mmum umer o trasmtted ts the system where X ad Y are jotly ecoded ad the jotly decoded The same result was later show [2] [3] to e vald as well or statoary ergodc sources X Y I ths paper we shall reer to these results collectvely as the Slepa- Wol result ad to the correspodg type o dstruted codg paradgm as Slepa-Wol codg SWC The Slepa-Wol result however does ot hold ay more whe X ad Y are dvdual sequeces I other words Slepa-Wol codg o dvdual sequeces the jot decoder does ot help at all ad the mmum umers o ts to e set udrectoally rom X ad Y respectvely to the jot decoder are the same as two depedet parallel systems where X ad Y are ecoded separately ad decoded separately Ths work was supported part y the Natural Sceces ad Egeerg Research Coucl o Caada uder Grats RGPIN ad RGPIN ad y the Caada Research Chars Program To overcome the aove prolem ths paper we cosder teractve ecodg ad decodg IED or dstruted ear lossless codg o dvdual sequeces as show Fgure 1 where two dvdual sequeces X = x ad Y = y are ecoded separately ad decoded jotly ad where the jot decoder s allowed to teract wth two separate ecoders The cocept o IED was rst ormalzed [4] [5] wth emphass o IED or ear lossless source codg wth decoder sde ormato where oe source s trasmtted rom the ecoder to the decoder ad aother source s avalale oly to the decoder as sde ormato By measurg the perormace o a IED scheme terms o the average umer o ts per symol exchaged etwee the ecoder ad the decoder the decodg error proalty ad the umer o teractos t was show [4] [5] that IED s superor to Slepa-Wol codg several aspects or ear lossless source codg wth decoder sde ormato I partcular uversal IED schemes or ear lossless source codg wth decoder sde ormato ca e costructed whch ca acheve the codtoal etropy rate or ay statoary source par wthout kowg the statstcs o the sources O the other had o such SWC schemes exst I ths paper we exted the advatage o IED over SWC to the source codg etwork show Fgure 1 ad show that y usg IED the mmum umer o total ts to e exchaged etwee the jot decoder ad two separate ecoders or each ad every par o dvdual sequeces X ad Y s the same as the system where X ad Y are jotly ecoded ad the jotly decoded whle X ad Y ca e recovered y the jot decoder a ear lossless maer Speccally gve ay our classcal source codg schemes oe scheme ecodg x aloe oe scheme ecodg y codtoally gve x oe scheme ecodg y aloe ad oe scheme ecodg x codtoally gve y we demostrate how to costruct a uversal radom IED scheme or Fgure 1 such that or each ad every par o dvdual sequeces x ad y the umer o total ts to e exchaged etwee the jot decoder ad two separate ecoders o the IED scheme s roughly the same as the jot ecodg ad decodg o x ad y va the ecodg o x plus the codtoal ecodg o y gve x or the ecodg o y plus the codtoal ecodg o x gve y ad at the same tme x ad y

2 ca e recovered y the jot decoder o the IED scheme wth hgh proalty Ths s sharp cotrast to the case o SWC where or dvdual sequeces there s o etter SWC scheme tha two depedet parallel systems where x ad y are separately ecoded ad decoded Thereore our result mples that allowg teractos strctly mproves the codg perormace or dstruted lossless codg o dvdual sequeces I addto t s show that y chagg the parameters o the uversal radom IED scheme we ca adjust the umer o ts o each lk whle keepg the umer o total ts exchaged etwee the jot decoder ad two ecoders tact x y Fg 1 put put Ecoder E X Ecoder E Y Decoder output ˆx ŷ Iteractve ecodg ad decodg or dstruted lossless codg The rest o the paper s orgazed as ollows Secto II troduces detos ad coveto to e used later our dscusso Secto III presets the uversal radom IED scheme alog wth ts perormace aalyss Fally coclusos wll e draw Secto IV II DEFINITIONS AND CONVENTION To acltate our susequet dscusso ths secto we troduce several detos ad coveto For ay sequece x over alphaet X x B s the ary sequece o legth log X geerated y covertg each symol o x to ts ary represetato o legth log X The otato S stads or the legth o S S s a sequece ad the cardalty o S S s a te set Let C = { g } e a classcal lossless codg scheme o order over alphaet Z where s a ecoder ecodg sequeces o legth over alphaet Z to ary sequeces rom a ary prex set ad g s ts correspodg decoder The the ormalzed codeword legth ucto l o C s deed as l z = z Let A l/ e the set o sequeces z deed as A l/ ={z : l z l/} The ollowg lemma s proved [5] Lemma 1: For ay lossless codg scheme C o order ad ay l 0 log Z A l/ 2 l Pck ay our classcal lossless codg schemes C X C Y C X Y ad C where C X ad C Y are classcal lossless codg schemes o order over alphaets X ad Y respectvely ad C X Y C respectvely s classcal codtoal lossless codg scheme wth ts source over alphaet X Y respectvely ad sde ormato over alphaet Y X respectvely avalale to oth the ecoder ad decoder Note that gve each y C X Y s a lossless code o order over alphaet X wth y servg as sde ormato avalale to oth ts ecoder ad decoder Lkewse gve each x C s a lossless code o order over alphaet Y wth x servg as sde ormato avalale to oth ts ecoder ad decoder Accordgly wth respect to C X C Y C X Y ad C the respectve ormalzed ucodtoal ad codtoal codeword legth uctos l X l Y l X Y y ad l x as well as sets A l/ X Al/ Y A l/ X Y y ad A l/ Y X x are well deed It s easy to check that Lemma 1 apples to ay o A l/ X Al/ Y Al/ X Y y ad A l/ Y X x Further dee A l/ XY A l/ Y X = {x y : l X x + l y x l/} = {x y : l Y y + l X Y x y l/} Sce l X x + l y x s the ormalzed codeword legth ucto o the jot ecodg o x ad y va the ecodg o x y C X plus the codtoal ecodg o y gve x y C Lemma 1 apples to A l/ XY Smlarly Lemma 1 apples to A l/ XY as well For coveece we also dee L M x y L m x y = max = m { { } l X x + l y x l Y y + l X Y x y } l X x + l y x l Y y + l X Y x y Wth reerece to Fgure 1 let I e a determstc IED scheme o order Gve ay x ad y we use x y I ad R Y x y I to represet the umer o ts per symol the orward drecto o each lk rom the ecoders E X ad E Y to the decoder respectvely ad x y I ad R Y x y I to represet the umer o ts per symol the ackward drecto o each lk rom the decoder to the ecoders E X ad E Y respectvely The total umer o ts per symol par set the orward drecto s the gve y R x y I = x y I + R Y x y I Moreover gve x ad y let PeI x y deote the dcator varale o error evet o I deed as { PeI x y = 0 ˆx = x ŷ = y 1 otherwse where ˆx ad ŷ are the outputs o the decoder For radom IED where I s chose radomly rom a IED esemle I

3 we urther dee x y I = E[ x y I ] R Y x y I = E[R Y x y I ] R x y I = E[R x y I ] x y I = E[ x y I ] R Y x y I = E[R Y x y I ] PeI x y = E[PeI x y ] where x ad y are gve ad E[ ] s the stadard expectato operator take over I III UNIVERSAL IED: ALGORITHMS AND PERFORMANCE Fx our classcal lossless codg schemes C X C Y C X Y ad C wth ther respectve ormalzed codeword legth uctos l X l Y l X Y ad l I ths secto we rst show how to covert these our classcal lossless codg schemes to a uversal radom IED scheme or Fgure 1 ad the aalyze the perormace o the resultg IED scheme A Uversal Radom IED Schemes Let ad e two teger actors o log X ad respectvely Radomly geerate two sequeces o log X ary matrces {H X } =0 ad {H Y } =0 where each H X s o sze log X each H Y s o sze ad elemets o H X ad H Y are geerated depedetly ad uormly over the ary alphaet {0 1} Wth reerece to Fgure 1 the sequece {H X } log X =0 s revealed to the ecoder E X ad jot decoder; lkewse the sequece {H Y } =0 s revealed to the ecoder E Y ad jot decoder log X Based o {H X } =0 ad {H Y } =0 we covert the our gve classcal lossless codg schemes C X C Y C X Y ad C to a uversal radom IED scheme I o order whch ecodes each par o dvdual sequeces x ad y as ollows: Set j = 0 The decoder seds t 0 to oth E X ad E Y to talze the trasmsso Stage I [ Trasmsso o Source Party Check Bts rom E X ad E Y ]: Upo recevg t 0 E X trasmts S X j = H X j x B to the decoder Upo recevg t 0 E Y trasmts S Y j = H Y j yb to the decoder Ater recevg S X j ad S Y j the decoder computes ẋ = argm v :H X v B =SX 0 j ẏ = argm w :H Y w B =SY 0 j l X v 1 l Y w 2 ad check the ollowg two codtos: C1: l X ẋ j 1X C2: l Y ẏ j 1Y or j or j log X ; I oth C1 ad C2 are true the decoder outputs ẋ ẏ as ˆx ŷ ad seds ts 10 to E X ad E Y ; ths IED scheme the termates I oly C1 s true the the decoder outputs ẋ as ˆx seds ts 10 to E X to termate E X ad the updates ẏ to ẏ = argm w :H Y w B =SY 0 j l w ˆx 3 I l ẏ ˆx j 1Y the the decoder outputs ẏ as ŷ ad seds ts 10 to E Y to termate E Y ad hece the whole IED process; otherwse the decoder leaves ŷ udecded ad seds t 0 to E Y ad ater creasg j y 1 the IED process moves to Stage III I oly C2 s true the the decoder outputs ẏ as ŷ seds ts 10 to E Y to termate E Y ad the updates ẋ to ẋ = v :H X argm v B =SX 0 j l X Y v ŷ 4 I l X Y ẋ ŷ j 1X the the decoder outputs ẋ as ˆx ad seds ts 10 to E X to termates E X ad hece the whole IED process; otherwse the decoder leaves ˆx udecded ad seds t 0 to E X ad ater creasg j y 1 the IED process moves to Stage IV I ether C1 or C2 s true the decoder updates ẋ ẏ to ẋ ẏ = argm v w Λ j L m v w 5 where Λ j s a set o v w satsyg H X vb = SX H Y wb = SY or 0 j ad check the ollowg codto C3: L m ẋ ẏ j 1X+Y I C3 s true the the decoder records ẋ ẏ as a tetatve estmato x ỹ o x y ad seds ts 11 ollowed y S X = H X x B to E X ad ts 11 ollowed y S Y = H Y ỹb to E Y where H X = H X j+1 HY = H Y j+1 ad hereater the IED process moves to Stage II I C3 s ot true the decoder seds t 0 to E X ad E Y ; crease j y 1 ad the IED process remas at Stage I Stage II [Trasmsso o Party Comparso Bt rom E X ad E Y ]:

4 Upo recevg ts 11 ad S X E X computes H X x B ; the ecoder E X trasmts t 1 to the decoder S X = H X x B ad t 0 otherwse Upo recevg ts 11 ad S Y E Y computes H Y yb ; the ecoder E Y trasmts t 1 to the decoder S Y = H Y yb ad t 0 otherwse I the decoder receves t 1 rom E X ad t 0 rom E Y t outputs x as ˆx seds ts 10 to E X to termate E X ad the updates ẏ accordg to 3 I l ẏ ˆx j 1Y the the decoder urther outputs ẏ as ŷ ad seds ts 10 to E Y to termate E Y ad hece the whole IED process; otherwse the decoder leaves ŷ udecded ad seds t 0 to E Y ad ater creasg j y 1 the IED process moves to Stage III I the decoder receves t 0 rom E X ad t 1 rom E Y t outputs ỹ as ŷ seds ts 10 to E Y to termate E Y ad the updates ẋ accordg to 4 l X Y ẋ ŷ j 1X the the decoder urther outputs ẋ as ˆx ad seds ts 10 to E X to termate E X ad hece the whole IED process; otherwse the decoder leaves ˆx udecded ad seds t 0 to E X ad ater creasg j y 1 the IED process moves to Stage IV I the decoder receves t 0 or t 1 rom oth E X ad E Y t outputs x ỹ as ˆx ŷ ad seds ts 10 to E X ad E Y to termate oth E X ad E Y ad hece the whole IED process Stage III [Trasmsso o Source Party Check Bts rom E Y Oly]: Upo recevg t 0 E Y trasmts S Y j = H Y j yb to the decoder Ater recevg S Y j the decoder computes 2 ad check the codto C2 I C2 s true the the decoder outputs ẏ as ŷ ad seds ts 10 to E Y to termate E Y ad hece the whole IED process I C2 s ot true the decoder updates ẏ accordg to 3 I l ẏ ˆx j 1Y the the decoder outputs ẏ as ŷ ad seds ts 10 to E Y to termate E Y ad hece the whole IED process; otherwse the decoder leaves ŷ udecded ad seds t 0 to E Y ad ater creasg j y 1 the IED process remas at Stage III Stage IV [Trasmsso o Source Party Check Bts rom E X Oly]: Upo recevg t 0 E X trasmts S X j = H X j x B to the decoder; Ater recevg S Y j the decoder computes 1 ad check the codto C1 I C1 s true the the decoder outputs ẋ as ˆx ad seds ts 10 to E X to termate E X ad hece the whole IED process I C1 s ot true the decoder updates ẋ accordg to 4 I l X Y ẋ ŷ j 1X the the decoder urther outputs ẋ as ˆx ad seds ts 10 to E X to termate E X ad hece the whole IED process; otherwse the decoder leaves ˆx udecded ad seds t 0 to E X ad ater creasg j y 1 the IED process remas at Stage IV Remark 1: Note that the uversal radom IED scheme I o order costructed aove depeds o the our classcal lossless codg schemes C X C Y C X Y ad C oly through ther respectve ormalzed codeword legth uctos B Perormace The ollowg theorem shows that or each ad every par o dvdual sequeces x ad y the average umer o total ts per symol par to e exchaged etwee the jot decoder ad two separate ecoders o the uversal radom IED scheme I s roughly upper ouded y the total umer o ts per symol par the jot ecodg ad decodg o x ad y va the ecodg o x plus the codtoal ecodg o y gve x or the ecodg o y plus the codtoal ecodg o x gve y Theorem 1: For ay sequece x y x y I l X x R Y x y I l Y y R x y I L M x y Q X + log X x y I R Y x y I where log X PeI x y Q X 11 Q X = 2 X+1+log log X +1 X +2 Y +1+log +1 Y +2 X Y +1+log m{ log{ X } } Y A sketch o the proo o Theorem 1 s provded Appedx A I Theorem 1 we maly ocus o the oud R x y I I act the rato o ad s gve uder some mld codtos o the code legth uctos t s possle to derve tghter ouds or x y I ad R Y x y I separately whle matag the same oud o R x y I as Theorem 1 I the ollowg dscusso we assume l X Y x y l X x 12 l y x l Y 13

5 The assumptos ca e terpreted as that sde ormato ca ot make the codg perormace worse Theorem 2: Uder the assumptos o 12 ad 13 each o the ollowg hold I the l Y y l X Y x y x y I l X Y x y Q X log X ad I the R Y ad I the ad R Y x y I l Y y l y x l X x x y I l y x Q X x y I l X x l y x < l Y y < l X x l X Y x y R Y x y I l X x + l y x Q X Y x y I l Y y + l X Y x y Q X Y log X The proo o Theorem 2 provded the ull paper [6] s the reemet o that o Theorem 1 where the rato o ad s take to accout Theorem 2 ca e terpreted graphcally as show Fgure 2 where we assume that l X x + l y x = l Y y + l X Y x y or the smplcato o the dscusso As ca ee ths gure deret values o the rato / wll result deret values o x y I ad R Y x y I whle R x y I remas the same R Y / l Y / IX Y l X Y l Y l Y X Graphc Iterpretato o Theorem 2 / I X / ly / IX Y l X ly X 05 / l Y X / I X Y X Fg 2 Graphc Iterpretato o Theorem 2 IV CONCLUSION I ths paper Iteractve Ecodg ad Decodg IED or dstruted lossless codg o dvdual sequeces s cosdered Coupled wth classcal source codes we propose IED schemes whch ca e appled to ay dvdual sequece par Whle wthout teractos jot decoder or dstruted lossless codg o dvdual sequeces does ot mprove the codg perormace compared to the system whch each sequece s decoded separately we show that our schemes ca approach the same sum rate perormace as the system o jotly ecodg ad decodg sequece pars Moreover our aalyss shows that we ca adjust the rate o each lks whle matag the sum rate y choosg the proper parameters or the schemes APPENDIX A SKETCH OF THE PROOF OF THEOREM 1 Gve dvdual sequeces x ad y let j X x y I ad j Y x y I e the umers o tmes whe E X ad E Y trasmt source party check ts to the decoder respectvely From the descrpto o I t s ot hard to see that j X x y I s always upper-ouded y log X +1 ad x y I j Xx y I + 1 A1 x y I j Xx y I A2 From ths 9 ollows To complete the proo o 6 oserve that upo recevg the j X x y I 1th packet o source party check ts rom E X l X x m v :H X v B =SX j Xx y I 3 0 j Xx y I 2 l X v A3

6 o matter what stage the IED process o I s o at that pot Pluggg A3 to A1 yelds 6 whle 7 ad 10 ca e proved the same maer Thereore the proo o Theorem 1 s shed oce 8 ad 11 are proved From the descrpto o I aga oe ca show that Pe{I x y } = 1 the at least oe o the ollowg evets happes: F 1 : For some j 0 j < log{ X } ˆx x H X ˆx B = HX l X F 2 : For some j 0 j < x B ˆx j 1 or 0 j ad ŷ y H Y ŷb = H Y yb or 0 j ad F 3 : For j = log{ X } F 4 : For j = l Y ŷ j 1 ˆx x H X ˆx B = HX x B or 0 j ŷ y H Y ŷ B = H Y y B or 0 j F 5 : For some j 0 j < m{ log{ X } ˆx x ŷ y or 0 j ad H X ˆx B = H X x B H Y ŷb = HY yb L m ˆx ŷ j 1 + F 6 : x x ad H X j+1 x B = HX j+1 x B } where j s the dex o the teracto mmedately eore the IED scheme eters stage II F 7 : ỹ y H Y j+1ỹ B = H Y j+1 y B where j s the dex o the teracto mmedately eore the IED scheme eters stage II F 8 : For some j 0 j < log{ X } ˆx x H X ˆx B = HX l X Y x B ˆx y j 1 F 9 : For some j 0 j < or 0 j ad ŷ y H Y ŷb = H Y yb or 0 j ad l ŷ x j 1 By the uo oud PeI x y = E[PeI x y ] 9 Pr{F } =1 Thereore PeI x y wll e ouded we gve the oud o the proalty o each o F For F 1 y the uo oud aga Pr{F 1 } j A j 1 X 2 jx 2 X+log log X I the smlar maer we ca derve the ouds or F 2 F 3 F 4 F 5 F 8 ad F 9 Pr{F 2 } 2 Y +log Pr{F 3 } 2 X Pr{F 4 } 2 Y Pr{F 5 } 2 X Y +1+log Pr{F 8 } 2 X+log log X Pr{F 9 } 2 Y +log m{ log{ X } } Y Now or F 6 oth j ad x are radom varales Sce x t exsts s determed y {H X } j =0 ad {HY } j =0 t s depedet wth H X j+1 Thereore we have Pr{F 6 } Pr{H X x = H X x x x } = 2 X ad smlarly All all we have Pr{F 7 } 2 Y PeI x y 2 X+1+log log X +1 X +2 Y +1+log +1 Y +2 X Y +1+log m{ log{ X } } Y ad 11 has ee proved Towards provg 8 gve x ad y dvde I to three su-esemles I III I III I IV or whch the whole IED process o I termates at Stages I or II Stage III ad Stage IV respectvely The R x y I = Pr{I I III }R x y I III + Pr{I I III }R x y I III + Pr{I I IV }R x y I IV

7 For I I III let jx y I represet the j whe the whole IED process o I termates Note that ths case ad j X x y I = j Y x y I = jx y I + 1 R x y I jx y I O the other had at the jx y I 1 roud o teracto L m x y m L m v w v w Λ jx y I 1 jx y I 2 + where Λ jx y I 1 ollows the deto the descrpto o IED schemes Thereore R x y I L m x y A4 For I I III upo recevg j Y x y I 1th packet o source party check ts rom E Y thereore l Y X y ˆx m w :H Y w B =SY 0 j Y x y I 2 j Y x y I 3 l w ˆx Smlar or the case I I IV R x y I l Y y m{l X Y x ŷ log X } I total we have R x y I L M x y Pr{ˆx x } + Pr{ŷ y } log X The 8 s proved y otg that Pr{ˆx x } ad Pr{ŷ y } are ouded y PeI x y REFERENCES [1] D Slepa ad J K Wol Noseless codg o correlated ormato sources IEEE Tras I Theory vol IT-19 pp July 1973 [2] T M Cover A proo o the data compresso theorem o slepa ad wol or ergodc sources IEEE Tras I Theory vol IT-19 pp Mar 1975 [3] R F Ahlswede ad J Körer Source codg wth sde ormato ad a coverse or degraded roadcast chaels IEEE Tras I Theory vol IT [4] E-H Yag ad D-K He O teractve ecodg ad decodg or lossless source codg wth decoder oly sde ormato Proc o ISIT 08 July 2008 pp [5] Iteractve ecodg ad decodg or oe way learg: Near lossless recovery wth sde ormato at the decoder accepted or pulcato IEEE Tras o Iormato Theory 2009 [6] E-H Yag ad J Meg Iteractve ecodg ad decodg or dstruted lossless codg o dvdual sequeces preparato or IEEE Tras Iorm Theory R Y x y I l Now or x y I we have y ˆx thereore x y I l X x R x y I l X I the meatme we have whch mples that j Y x y I x + l y ˆx R x y I l X x m{l y ˆx }