Finite Block-Length Gains in Distributed Source Coding

Transcription

1 Decoder Fiite Block-Legth Gais i Distributed Source Codig Farhad Shirai EECS Departmet Uiversity of Michiga A Arbor,USA fshirai@umichedu S Sadeep Pradha EECS Departmet Uiversity of Michiga A Arbor,USA pradhav@umichedu Abstract A ew codig scheme for the distributed source codig problem for geeral discrete memoryless sources is preseted The scheme ivolves a two-layered codig strategy, the first layer code is of costat fiite block-legth whereas the secod layer cotais codes of block-legth approachig ifiity It is argued that small block-legth codes preserve correlatios betwee sources more efficietly, while sufferig rate-loss i a poit-to-poit compressio perspective Cosequetly, there is a sweet-spot for the legth of the code A achievable ratedistortio regio is characterized usig sigle-letter distributios It is show that this regio strictly cotais previous kow achievable rate-distortio regios for the distributed source codig problem I INTRODUCTION Due to the iheret difficulty i characterizatio of fudametal limits of reliable commuicatio, most of the works i iformatio theory have cosidered the aalysis of large block-legth codes, without ay costraits o memory ad computatioal complexity This allows operatig i the realm of the laws of large umbers ad thus sigificatly simplifies the aalysis However, codig theory deals with codes with short block-legths with computatioally efficiet ecodig ad decodig algorithms While there is iterest i applicatio of such small-legth commuicatios strategies, it is largely assumed that performace improves whe the block-legth icreases This turs out to be ot true i certai codig schemes used i distributed source codig as discussed below The two user distributed source-codig problem is depicted i figure 1 Two correlated discrete-memoryless-sources (DMS) are fed to the ecoders Each ecoder compresses its respective source ad trasmits the compressed versio to the decoder The decoder recostructs each of the sources with respect to a distortio criteriothere has bee a substatial effort to derive the optimal RD regio for this set-up, however the regio has bee characterized oly i special cases [2], [3] The best kow ier boud to the optimal rate-distortio (RD) regio for distributed codig of two correlated sources is the Berger-Tug (BT) boud [4] The BT boud is based o a codig strategy called quatize ad bi I this strategy the two sources are quatized usig two idepedet, ifiite legth radom vector quatizers ad the quatizers are bied to This work was supported by NSF grat CCF Fig 1 X 1 Ec 1 X 2 Ec 2 ˆX 1, ˆX2 The two-user distributed source codig problem reduce the trasmissio rate The idepedet quatizatio approach leads to the so-called log Markov chai The Markov chai implies that coditioed o the sources, the sigle-letter distributio of the quatized versios of the sources decompose ito the product of coditioal margial distributios I [5] it was poited out that i the presece of commo compoets, further correlatio ca be iduced betwee the quatized versios, i other words the Markov chai ca be relaxed usig the commo compoet (CC) Based o this observatio the authors i [5] propose a codig scheme which outperforms the BT strategy i the presece of commo compoets, but reduces to the latter i their absece The CC achievable RD regio shriks discotiuously i source probability distributio as commo compoets are replaced with highly correlated compoets From the cotiuity of the optimal RD regio, it was proved i [5] that the CC scheme is also sub-optimal sice it is discotiuous Hece the optimal RD regio strictly cotais the CC regio, however it was ot clear how to achieve poits outside of the CC ratedistortio regio Toward improvig the CC boud, we used two idetical radom quatizers istead of two idepedet oes We foud [1] that whe applied to the biary-oe-helpoe (BOHO) example of [5], as the block-legth is icreased, the performace improves iitially, reaches a plateau, ad the decreases after that So the best performace is achieved whe the legth of the code is at some fiite value Based o this observatio, a codig strategy was proposed for the BOHO example which improves the CC achievable RD regio The mai difficulty i aalyzig fiite block-legth codig strategies is that i the absece of simplifyig theorems such as laws of large umbers, the resultig characterizatios of achievable ier bouds are i terms of multi-letter prob-

2 Fig 2 ɛ X + E Q(X(1 : )) X Q((X + E)(1 :)) Quatizatio oises become idepedet at large block-legth ability distributios This makes the computatio of such ier bouds very complex I this paper we first preset a codig scheme for the distributed source codig problem i the geeral discrete, memoryless settig The scheme utilizes both small-legth codes ad codes with block-legth approachig ifiity A multi-letter characterizatio of the rate-distortio regio achievable usig this scheme is give, ad it is show that there is a approximatig sigle-letter characterizatio for the ier boud We prove that the resultig ier boud outperforms the previous kow codig schemes for this commuicatio settig The rest of the paper is orgaized as follows: I sectio II, we explai the ituitive reaso behid our codig scheme Sectio III icludes the ew codig strategy ad the resultig RD regio Sectio IV cocludes the paper II BINARY-ONE-HELP ONE EXAMPLE I [1] quatizatio of two highly correlated BSS s usig the same radomly geerated code was cosidered More specifically, let X be a BSS, ad E be a Be(ɛ) radom variable Defie the sources as X 1 = X ad X 2 = X +E We took a radomly chose -legth quatizer Q ad quatized X1 ad X2 As the legth of the quatizer approaches ifiity, the quatizatio oises become idepedet; this happes irrespective of the value of ɛ The ituitive reaso is that as the quatizer legth icreases, most of the vectors are cocetrated o the edges of the Vorooi regios Hece as the legth goes to ifiity, with high probability the two source vectors fall ito two differet quatizatio regios Sice codewords are pairwise idepedet, the quatizatio oises become idepedet This is illustrated i figure 2 Idepedet quatizatio oises prevet ecoders from refiig each other s quatizatios Whe the sources are exactly equal (ie commo compoets), the quatizatios would also be equal We argue that the discotiuity i the CC scheme is caused by the discotiuity i the correlatio betwee quatizatio oises This implies applicatio of small-legth quatizers is beeficial sice they preserve correlatio betwee the sources, however there is also a rate-loss associated with usig small-legth quatizers The evidece suggests that the existece of a trade-off betwee avoidig rate-loss ad preservig correlatio It was show that there is a sweet-spot for the legth of the quatizer where the trade-off is optimized Based o these observatios a ew codig scheme for the special case of BOHO problem was preseted The scheme uses two layers of codes, the first layer cosists of a codebook with fiite legth (ie the codeword legth is ot take to ifiity), ad the secod codebook has large block-legth I this codig strategy the ecoders apply the fiite-legth quatizer i first layer to quatize the highly correlated compoets of the sources, ad these quatizatios are trasmitted to the decoder Sice the compressed sequeces are highly correlated, the ecoders are able to cooperate i trasmittig them I the ext step, the CC scheme is applied while cosiderig the previous quatizatios as side iformatio completely kow at the decoder ad partially kow at each ecoder III THE NEW CODING STRATEGY Let X 1 ad X 2 be two correlated DMS s Assume there exist fuctios f 1 : X 1 Zad f 2 : X 2 Z, such that P (f 1 (X 1 ) f 2 (X 2 )) ɛ,ɛ (0, 05) Here X i ad Z are the uderlyig alphabets for X i ad Z Let ɛ =1 (1 ɛ ) Also defie S i = f i (X i ),i {1, 2} The ext theorem presets the mai result of this paper: Theorem 1: The followig RD vectors are achievable: R 1 I(X 1 ; U 1 U 2 W )+E,ɛ +2 X 1 U 1 log( ɛ ) p 2 R 2 I(X 2 ; U 2 U 1 W )+2E,ɛ +2 X 2 U 2 log( p 2 ɛ ) R 1 + R 2 I(X 1 X 2 ; U 1 U 2 W )+3E,ɛ +2 X 1 U 1 log( ɛ )+θ D i E{d i hi (U 1,U 2,W),X i } For every distributio P X1,X 2,W,U 1,U 2 satisfyig the followig costraits: 1)U 1 (X 1,W) (X 2,W) U 2 2)W S 1 (X 1,X 2 ) 3)p i >ɛ,i=1, 2 Also p i ad E,ɛ are defied as follows: = mi ({P U1 X x 1,W,U 2 }, {P U1 W,U 2 }) 1,w,u 1,u 2 p 2 = mi ({P U2 X x 2,W,U 1 }, {P U2 W,U 1 }) 2,w,u 1,u 2 E,ɛ = h(ɛ) + ɛ log W I the above formulas, θ is a sequece approachig 0 which ca be bouded for a give P S,W Also U i ad W are the alphabets for U i ad W Remark 1: The above RD regio reduces to the CC regio whe ɛ =0(ie whe S 1 = S 2 ) Also if S 1 ad S 2 are take to be trivial, the boud reduces to the BT regio Remark 2: As becomes larger, ɛ icreases, which i tur causes E,ɛ to icrease O the other had, θ is a decreasig

3 fuctio of This illustrates the trade-off betwee rate-loss due to applicatio of small block-legth codes θ, ad the gais from preservatio of correlatio betwee the sources E,ɛ Remark 3: Note that fidig the achievable rate-distortio regio ivolves sweepig over all possible choices of f 1 ad f 2 However log( p1 p ) icreases as f 1 ɛ 1 ad f 2 become less correlated (ie as ɛ icreases) This suggests choosig highly correlated fuctios gives larger achievable regios Remark 4: The above ier boud is ot symmetric with respect to the two ecoders, oe ca symmetrize the regio by swappig the idices for ecoders 1 ad 2 i the theorem ad takig the uio of the two resultig regios To prove theorem 1, a achievable RD regio usig fiitelegth codig schemes is derived, the it is show that the regio cotais the ier boud i the theorem Let W 1 be a radom variable with alphabet W 1 Take a arbitrary probability distributio P S1,W 1 Also let I be a radom variable uiformly distributed o {1, 2, 3,,} Cosider Q, a -legth quatizer which quatizes S1 to W1 such that: P S1(I),W 1(I) = 1 P S1(i),W 1(i) = P S1,W 1 i [1:] By radom codig argumets Q exists with rate R = I PS1,W 1 (S 1 ; W 1 )+θ, where θ ca be bouded give the distributio P S1,W 1 ad approaches 0 as goes to ifiity [6] Let W2 = Q (S2 ) W2 ca be perceived as the secod ecoder s estimate of W1 Wehave: P X 1,X2,W 1,W 2 = P X 1,X2 P S 1,S S 2 1,S 2,W 1,W 2 s 1,s 2 Defie P X1,X 2,W 1,W 2 = P X1(I),X 2(I),W 1(I),W 2(I) Also defie P as the set of all probability distributios o X 1,X 2,W 1,W 2,U 1,U 2 such that P X1,X 2,W 1,W 2 is produced by the above process ad U 1 ad U 2 satisfy U 1 (X 1,W 1 ) (X 2,W 2 ) U 2 The esuig theorem states the -letter achievable boud for the ew codig strategy Theorem 2: RD vectors satisfyig the followig bouds are achievable: R 1 I(X 1 ; U 1 U 2 W 1 W 2 )+E,ɛ R 2 I(X 2 ; U 2 U 1 W 1 W 2 )+E,ɛ R 1 + R 2 I(W 1 ; S 1 )+I(X 1 ; U 1 W 1,W 2 )+ I(X 2 ; U 2 W 1,W 2 ) I(U 1 ; U 2 W 1 W 2 )+θ + E,ɛ D i E{d i hi (U 1,U 2,W 1,W 2 ),X i } For every probability distributio P X1,X 2,W 1,W 2,U 1,U 2 chose form P Here h i : W 1 W 2 U 1 U 2 X i are the recostructio fuctios at the decoder Remark 5: Q completely determies P X1,X 2,W 1,W 2, also from the Markov chai P U1 X 1,W 1, P U2 X 2,W 2 ad Q fix the iduced joit probability distributio P X1,X 2,W 1,W 2,U 1,U 2 Hece determiig the RD regio give i theorem 2 ivolves takig the uio of RD vectors satisfyig the above bouds for some give f i, h i, Q, P U1 X 1,W 1 ad P U2 X 2,W 2 Proof: First we preset a summary of the proof Fix f i, h i, Q, P U1 X 1,W ad P U2 X 2,W 2 Usig Q the ecoders quatize Si to Wi From radom codig argumets, if multiple realizatios of W1 s are available at the decoder, ecoder 2 ca trasmit the correspodig sequece of W2 1 s usig rate at most H(W 2 W1 ) So the ecoders trasmit the sequeces of W i s with sum-rate less tha R + 1 max{h(w 2 W1 ),H(W1 W2 )} Sice S i s are highly correlated, the vectors Si are almost always equal Cosequetly the quatizatios Wi are almost always equal, ad usig this the term 1 max{h(w 2 W1 ),H(W1 W2 )} ca be bouded I the ext step (W i,x i ) is trasformed ito a DMS by applyig the iterleavig method explaied i [1] The rest of the problem ca be viewed as distributed source codig with sources (W i,x i ) ad side-iformatio (W 1,W 2 ) available at the decoder A more rigorous proof is preseted ext Codebook Geeratio: Three codebooks are used for the quatizatio C is the uderlyig codebook for Q The other two codebooks Ci m, are costructed by choosig each of their elemets from U i based o distributio P Ui Ci m have rates I(X i,w i ; U i )+λ m where λ m 0 Let Q i,m be the quatizers associated with these codebooks Each of the codebooks Ci m are radomly bied at rate I(X i ; U i W 1,W 2 ) r i, where r 1 + r 2 = I(U 1 ; U 2 W 1,W 2 )Givet [0, 1], radomly ad uiformly bi the space of all vectors W tm W tm with rate E,ɛ, ad defie B 1 as the biig fuctio Also bi the space of all vectors W (1 t)m usig the same rate, let B 2 be the biig fuctio Fially choose m permutatios π j,j [1 : m] radomly ad uiformly from the set of all -legth permutatios S Ecodig: Commuicatio is carried out over blocks of legth m Deote the source sequece i oe block as the matrix X i (1:m, 1:) The ith ecoder calculates W i (j, 1:) = Q (S i (j, 1:)) for all j The ecoder wishes to utilize the codebooks Ci m for quatizig (X i,w i ), however the source (X i,w i ) is ot a DMS sice W i is produced by a fiite-legth quatizer To overcome this difficulty we use the method explaied i [1] Let Xi (j, 1:) =π j (X(j, 1:)), defie W i i the same maer As show i [1], ( X i, W i )(1:m, l) would behave like a DMS with probability distributio P Xi,W i Each ecoder calculates Ũi(1:m, l) =Q i,m (( X i, W i )(1:m, l)) For rows (1:tm), the first ecoder trasmits W 1 (j, 1:) while the secod ecoder seds the bi idex B 1 (W 2 (1 : tm, 1:)) For the rest of the rows ecoder 1 seds the bi idex B 2 (W 1 (tm +1:m, 1:)) while ecoder 2 seds W 2 (j, 1:) I other words the ecoders time-share betwee two strategies I the first strategy ecoder 1 trasmits W 1 while ecoder 2 oly seds the bi umber for the sequece of W 2,ithe secod strategy the ecoders reverse roles For every colum l, the i th ecoder also seds the bi idex of Ũi(1 : m, l) i

4 C m i The resultig rates are: Also: R 1 = tr +(1 t)e,ɛ + I(X 1 ; U 1 W 1 W 2 ) r 1 R 2 =(1 t)r + te,ɛ + I(X 2 ; U 2 W 1 W 2 ) r 2 If we prove that the above rates are achievable, the proof is complete, sice they iclude both corer poits of the regio i theorem 2 Decodig: The decoder first decodes W i (1 : m, 1:) For elemets (1 : tm, 1:), W1 tm are available, while oly the bi idex of W2 tm is available Sice m is goig to ifiity, by radom codig argumets W2 tm is losslessly recovered as log the biig rate is more tha 1 H(W 2 W1 ) By lemma 1 i the appedix we have 1 H(W 2 W1 ) E,ɛ By the same argumet (W 1,W 2 ) are recovered losslessly for the rest of the rows The bi size for each vector Ũ1(1:m, l) is: I(X 1,W 1 ; U 1 ) I(X 1 ; U 1 W 1,W 2 )+r 1 = I(X 1,W 1,W 2 ; U 1 ) I(X 1 ; U 1 W 1,W 2 )+r 1 = I(W 1,W 2 ; U 1 )+r 1 The log Markov chai is used i the secod equatio By the same calculatios the bi size for Ũ2 is I(W 1,W 2 ; U 2 )+r 2 Usig typicality argumets oe ca show that for these bi sizes, there is oly oe pair (U 1,U 2 )(1 : m, l), joitly typical with (W 1,W 2 )(1 : m, l) Due to space limitatios we oly preset a summary of the proof The decoder first creates two ambiguity sets L i from the sequeces of U i (1 : m, l) s i the correspodig bis Each of these sets cotais all sequeces U i (1 : m, l) i the bi, which are typical with (W 1,W 2 )(1:m, l) There is roughly oe such sequece i each 2 mi(w1,w2;ui) vectors So the size of L i is close to 2 mri The decoder fids a pair of vectors i the two ambiguity sets which are typical with each other Sice all these vectors are typical with W 1 ad W 2, as log as r 1 + r 2 I(U 1 ; U 2 W 1,W 2 ) there is o more tha oe pair (U 1,U 2 )(1 : m, l) typical with respect to P U1,U 2 W 1,W 2 This completes the proof From remark 3, calculatio of the RD regio i the theorem requires takig uio over all possible -legth quatizers This reders the characterizatio practically icomputable The ext proof shows that the RD regio i theorem 1 is cotaied i the oe i theorem 2, so it is achievable usig the fiitelegth codig scheme Proof: First we elimiate W 2 from the iequalities Restrictig the recostructio fuctios to oly use W 1,U 1,U 2 results i a achievable ier boud I the ext step, W 2 is removed from the mutual iformatio terms: I(X 1 ; U 1 U 2,W 1,W 2 ) = H(U 1 U 2,W 1,W 2 ) H(U 1 X 1,W 1,U 2 ) I(U 1 ; X 1 W 1,U 2 ) I(X 2 ; U 2 U 1,W 1,W 2 ) I(X 2 ; W 2,U 2 W 1,U 1 ) I(X 2 ; U 2 W 1,U 1 )+H(W 2 W 1 ) I(X 2 ; U 2 W 1,U 1 )+E,ɛ W 2 i the terms I(X 1 ; U 1 W 1,W 2 ) ad I(X 2 ; U 2 W 1,W 2 ) i the sum-rate boud ca be removed usig the same method For I(U 1 ; U 2 W 1,W 2 ) a upper-boud is ecessary: I(U 1 ; U 2 W 1,W 2 ) I(U 1 ; U 2 W 1 ) H(W 2 W 1 ) I(U 1 ; U 2 W 1 ) E,ɛ It is straightforward to show I(W 1 ; S 1 )=I(W 1 ; X 1 )Sofar we have the followig ier boud: Q R 1 I(X 1 ; U 1 W 1,U 2 )+E,ɛ R 2 I(X 2 ; U 2 W 1,U 1 )+2E,ɛ R 1 + R 2 I(W 1 ; X 1 )+3E,ɛ + I(X 1 ; U 1 W 1 ) + I(X 2 ; U 2 W 1 ) I(U 1 ; U 2 W 1 )+θ D i E{d i hi (U 1,U 2,W 1 ),X i } is still playig a role i the calculatio of the RD regio by determiig P X1,X 2,W 1,W 2, which i tur affects P U2 X 1,X 2,W 1 Lemma 2 i the appedix provides the meas to elimiate this depedecy o Q The lemma shows that for every probability distributio i theorem 1, there is a probability distributio i theorem 2, for which the above bouds are well-approximated whe calculated usig oe of the distributios istead of the other Hece we ca provide a ier boud for the above rates by cosiderig the distributios from the first theorem ad boudig the estimatio error usig lemma 2 Applyig the estimatio bouds gives the regio preseted i theorem 1 Fially, we show that the RD regio i theorem 1 strictly cotais the CC rate-distortio regio Theorem 3: For the BOHO problem i [1], the RD regio i theorem 2 achieves poits outside of the CC rate-distortio regio Proof: Take U 1 = φ, U 2 = Z + X + W + N δ0 ad W = X + N ɛ + N δ, where N δ is Be(δ),N δ0 is Be(δ 0 ), ad the quatizatio oises are idepedet of the sources ad each other The recostructio fuctio is U 2 + W = X + Z + N δ0 Cosider the corer poit where ecoder 1 is trasmittig W by itself ad ecoder 2 is biig its correlated quatizatio at rate E,ɛ The resultig RD vector approaches (R 1,R 2,D)= (1 h b (δ),h b (p δ) h b (δ 0 ),δ 0 ) as ɛ 0 ad I [5] it was show that these RD vectors are ot achievable by the CC scheme whe ɛ 0 By the same argumet as i [1], it ca be proved that there exist ad ɛ for which the resultig rates are ot achievable by the CC scheme There ca be highly correlated compoets betwee the sources give W 1 ad W 2 I this case, there must be several

5 fiite-legth codebooks super-imposed o each other, oe for each of the highly correlated compoets The ier boud preseted here ca be exteded to iclude these ew layers IV CONCLUSION A ew codig scheme for the distributed source codig problem was preseted The scheme used a combiatio of fiite legth ad large block-legth codes The applicatio of fiite legth quatizers was justified by showig that they preserve correlatio betwee sources more efficietly The achievable regio for the codig scheme was derived A sigleletter characterizatio of a ier boud to the achievable regio was preseted It was show the ier boud strictly cotais previous kow achievable RD regios APPENDIX Lemma 1: Let S 1 ad S 2 be two DMS s such that P (S 1 S 2 ) ɛ for some ɛ > 0 Also Let Wi = f i (Si ) be -letter fuctios of Si to alphabet W Let ɛ =1 (1 ɛ ) The the followig are true: 1) P (W1 W2 ) ɛ 2) 1 H(W 2 W1 ) h(ɛ) + ɛ log W 2 Proof: 1) P (W1 = W2 ) P (S1 = S2 )=1 ɛ The first iequality is true sice if S1 = S2, the W1 = W2 2) Let 1 A be the idicator fuctio of evet A Wehave: H(W2 W1 )=H(1 {W 1 =W2 },W2 W1 ) H(1 {W 1 =W2 } )+H(W2 W1, 1 {W 1 =W2 } ) h b (ɛ)+p(w1 W2 )H(W2 W1, 1 {W 1 =W2 } =0) h b (ɛ)+p(w1 W2 )H(W2 ) h b (ɛ)+ɛ log W 2 Lemma 2: Cosider a probability distributio P X1,X 2,W 1,U 1,U 2 satisfyig the Markov chais U 1 (X 1,W 1 ) (X 2,W 1 ) U 2 ad W 1 S 1 (X 1,X 2 ), where S 1 ad S 2 are as defied previously let P S1,W 1 be the margial distributio of (S 1,W 1 ), take a quatizer Q from Q PS,W Assume P X 1,X 2,W 1,W 2 is the probability distributio iduced by Q Let P U 1 X 1,W 1 = P U1 X 1,W 1 ad P U 2 X 2,W 2 = Defie: = P X 1,X 2,W 1,W 2 P U 1 X 1,W 1 P U 2 X 2,W 2 The followig hold: 1)P X 1,X 2,W 1,U 1,U 2 =P X1,X 2,W 1,U 1,U 2 ± ɛ 2)I P (X 1 ; U 1 W 1,U 2 ) =I P (X 1 ; U 1 W 1,U 2 ) ± 2 X 1 U 1 log ( ɛ ) 3)I P (X 2 ; U 2 W 2,U 1 ) =I P (X 2 ; U 2 W 2,U 1 ) p 2 ± 2 X 2 U 2 log ( p 2 ɛ ) I the above equatios, a =b ± ɛ meas a [b ɛ, b + ɛ] Also I P (A; B C) deotes the mutual iformatio with respect to P Proof: 1) P X 1,X 2,W 1,U 1,U 2 w 2 1 {W1=W 2} P U1 X 1,W 1 = P U1 X 1,W 1 w 2 a P U1 X 1,W 1 (P X1,X 2,W 1 ɛ) P X1,X 2,W 1,U 1,U 2 ɛ Where part (a) results from the followig: P X1,X 2,W 1 = P X 1,X 2,W 1 P X 1,X 2,W 1,W 2 w 2 + P (1 W1 W 2 ) Now we prove the other side of the iequality: P X 1,X 2,W 1,U 1,U 2 1 W1=W 2 + ɛ w 2 P X1,X 2,W 1 P U1 X 1,W 1 P U2 X 1,W 1 + ɛ = P X1,X 2,W 1,U 1,U 2 + ɛ 2) ad 3) follow from 1 i a straightforward maer by expadig the mutual iformatios, the maximum differece betwee the terms i the mutual iformatios is 2log for the sake of brevity we omit the proofs for these parts REFERENCES pi p, i ɛ [1] F Shirai, AG Sahebi, SS Pradha, Distributed source codig i absece of commo compoets, i IEEE Iteratioal Symp o Iformatio Theory IEEE, pp1362,1366, 7-12 July 2013 [2] D Slepia, J Wolf, Noiseless codig of correlated iformatio sources, IEEE Trasactios o Iformatio Theory, vol19, o4, pp , Jul 1973 [3] T Berger, R W Yeug, Multitermial source ecodig with oe distortio criterio,, IEEE Trasactios o Iformatio Theory, vol 35, o 2, pp , 1989 [4] S-Y Tug, Multitermial source codig, PhD Thesis, Corell Uiversity, Ithaca, NY, 1974 [5] AB Wager, BG Kelly, Y Altuğ, Distributed Rate-Distortio With Commo Compoets, IEEE Trasactios o Iformatio Theory, vol57, o7, pp , July 2011 [6] V Kostia, S Verdu, Fixed-Legth Lossy Compressio i the Fiite Blocklegth Regime, IEEE Trasactios o Iformatio Theory, vol58, o6, pp3309,3338, Jue 2012