Universal source coding for complementary delivery

Transcription

1 SITA2006 i Hakodate p. Uiversal source codig for complemetary delivery Akisato Kimura, 2, Tomohiko Uyematsu 2, Shigeaki Kuzuoka 2 Media Iformatio Laboratory, NTT Commuicatio Sciece Laboratories, NTT Corporatio 2 Departmet of Commuicatios ad Itegrated Systems, Graduate School of Sciece ad Egieerig, Tokyo Istitute of Techology

2 SITA2006 i Hakodate p. 2 Abstract Preset eplicit costructios of uiversal codes for the followig multitermial source codig system Side iformatio Y Iput sequeces X Y X Side iformatio 2 Ecoder Decoder Decoder 2 Decoded sequece Xˆ Yˆ Decoded sequece 2 Fied-to-fied legth (FF) ad fied-to-variable legth (FV) lossless codig schemes are cosidered. Key techiques: type theory, graph theory

3 SITA2006 i Hakodate p. 3 Cotets Itroductio FF uiversal codig Codig scheme Lemmas that support the codig scheme FF codig theorems FV uversal codig Codig scheme Codig theorems Coclusio (See the proceedigs.)

4 SITA2006 i Hakodate p. 4 Itroductio Codig problems for correlated iformatio sources were origially ivestigated by Slepia ad Wolf [SW73] Iput sequeces X Y Ecoder Ecoder 2 Decoder Decoded sequece Xˆ Yˆ Decoded sequece 2 The problem of uiversal codig for this systems was first ivestigated by Csiszár ad Körer [CK80]. Subsequet work has maily focused o the Slepia-Wolf codig system Sice it appears to be difficult to costruct uiversal codes for most of the other codig systems.

5 SITA2006 i Hakodate p. 5 Mai cotributios Costruct uiversal codes for the complemetary delivery codig system Side iformatio Y Iput sequeces X Y X Side iformatio 2 Ecoder Decoder Decoder 2 Decoded sequece Xˆ Yˆ Decoded sequece 2 Our previous report [KU06]: Cosidered the lossy cofiguratio ad clarified the rate-distortio fuctio. This report: Cosider a lossless cofiguratio ad costruct uiversal codes.

6 SITA2006 i Hakodate p. 6 Prelimiaries (/2) X, Y: fiite alphabets, B: a biary set, B : a set of all fiite sequeces i B, X : cardiality of X. = (, 2,, ): member of X. Whe the dimesio is clear from the cotet, vectors will be deoted by boldface letters, i.e., X M(X ): the set of all probability distributios o X, M(X P Y ): the set of all probability distributios o X give a distributio P Y M(Y) X: a discrete memoryless source takig values i X with a geeric distributio P X M(X )

7 SITA2006 i Hakodate p. 7 Prelimiaries (2/2) H(X), H(P X ): etropy of X, H(X Y ), H(P X Y P Y ): coditioal etropy of X give Y, D(P X P Y ): divergece betwee P X ad P Y. TQ : the set with sequeces of type Q, TV (): the set of sequeces with coditioal type V : X Y for a give X. P (X ): the set of types of sequeces i X, V (Y Q): the set of coditioal types of sequeces i Y for a give Q P (X ) s.t. TV () for ay T Q.

8 SITA2006 i Hakodate p. 8 Miimum achievable rate Theorem. [WWW02] R F (X, Y ) = ma{h(x Y ), H(Y X)} = ma{h(p X Y P Y ), H(P Y X P X )}. Side iformatio Y Iput sequeces X ( X Y) H Decoder Decoded sequece Xˆ Ecoder Y X Side iformatio 2 ( Y X ) H Decoder 2 Yˆ Decoded sequece 2

9 SITA2006 i Hakodate p. 9 Cotets Itroductio FF uiversal codig Codig scheme Lemmas that support the codig scheme FF codig theorems Coclusio

10 SITA2006 i Hakodate p. 0 FF Codig scheme (/4). (Type selectio) For a give codig rate R > 0, select joit types Q XY P (X Y) that satisfy ma{h(w Q Y ), H(V Q X )} R, where Q X P (X ), Q Y P (Y), V V (Y Q X ) ad W V (X Q Y ) that satisfy Q X V = Q Y W = Q XY. If ma{h(w Q Y ), H(V Q X )} > R codewords will ot be assiged.

11 SITA2006 i Hakodate p. FF Codig scheme (2/4) 2. (Table creatio) Create a codig table for each joit type Q XY selected i Step. Each row of the codig table correspods to a sequece T Q X, ad each colum correspods to a sequece y T Q Y. QXY T QX y y2 y3 y4 y5 T QY

12 SITA2006 i Hakodate p. 2 FF Codig scheme (3/4) 3. (Cell markig) Mark cells that correspod to sequece pairs (, y) T Q XY Codewords will be give oly to sequece pairs that correspod to marked cells. Q XY y y2 y3 y4 y ( ) T QXY,y 4 ( ) T QXY,y 3

13 SITA2006 i Hakodate p. 3 Codig scheme (4/4) 4. (Codeword assigmet) Fill the marked cells with ep(r) differet symbols such that each symbol occurs at most oce i each row ad at most oce i each colum. 5. For a give (, y) X Y, the codeword is y y2 y3 y4 y5 ide assiged to the type of the sequece pair

14 SITA2006 i Hakodate p. 4 Lemmas that support the codig scheme (/3) Lemma. For a give codig table of a joit type Q XY P (X Y), the umber of marked cells i every row of the codig table is a costat N s.t. N ep(r), the umber of marked cells i every colum of the codig table is also a costat N y s.t. N y ep(r) both of which deped solely o the joit type Q XY. Sketch of the proof. For a give type Q XY ad (, y) T Q XY, both N = T V N y = T W (y) are costat. () ad

15 SITA2006 i Hakodate p. 5 Lemmas that support the codig scheme (2/3) Lemma 2. For give positive itegers m, m y, ad y that satisfy m ad m y y, there eists a m m y table filled with ma(, y ) differet symbols such that at most y cells are filled with a certai symbol for each row (blak cells are possible), at most cells are filled with a certai symbol for each colum (blak cells are possible), each symbol occurs at most oce i each row ad at most oce i each colum.

16 SITA2006 i Hakodate p. 6 Lemmas that support the codig scheme (3/3) Sketch of the proof. y y2 y3 y4 y row colum y y2 y3 y4 y5 The followig lemma esures the eistece of the above bipartite graph. Lemma 3. (Köig [K. 6]) If a graph G is bipartite, the miimum umber of colors ecessary for edge colorig of the graph G equals the maimum degree of G.

17 SITA2006 i Hakodate p. 7 Cotets Itroductio FF codig scheme Codig scheme Lemmas that support the codig scheme FF codig theorems Coclusio

18 SITA2006 i Hakodate p. 8 FF codig theorems (/2) Theorem 2. (Direct part) For a give real umber R > 0, there eists a sequece of uiversal FF codes {(ϕ, ϕ (), ϕ (2) )} = such that for ay iteger ad ay source (X, Y ) log M R + X Y log( + ), { ρ (X) + ρ (Y ) 2( + ) X Y ep where mi D(Q XY P XY ) Q XY S (R) S (R) = {Q XY P (X Y) : ma{h(v Q X ), H(W Q Y )} > R, Q X P (X ), Q Y P (Y), V V (Y Q X ), W V (X Q Y ), Q XY = Q X V = Q Y W }. },

19 SITA2006 i Hakodate p. 9 FF codig theorems (2/2) Theorem 3. (Coverse part) Ay sequece of FF codes {(ϕ, ϕ (), ϕ (2) )} = must satisfy ρ (X) + ρ (Y ) for the source (X, Y ) { } ep mi 2( + ) X Y D(Q XY P XY ) Q XY S (R+ɛ ) for ay iteger ad a give codig rate R = / log M > 0, where ɛ 0 ( ). This theorem implies that the error epoet obtaied i Theorem 2 is asymptotically optimal.

20 SITA2006 i Hakodate p. 20 Coclusios Ivestigated a uiversal codig problem for the complemetary delivery codig system. Preseted a eplicit costructio of uiversal FF codes The proposed codig scheme ca achieve the optimal error epoet. Ca apply the FF codig scheme to costructio of FV uiversal codes Codeword legths coverge to the miimum achievable rate almost surely. Future work Eted the results to other classes of iformatio sources Costruct uiversal codes for the lossy cofiguratio

21 SITA2006 i Hakodate p. 2 Thak you Refereces [CK80] I. Csiszár ad J. Körer. Towards a geeral theory of source etworks. IEEE Tras. Iform. Theory, 26(2):55 65, March 980. [K. 6] D. Köig. Graphok és alkalmazásuk a determiások és a halmazok elméletére. Mathematikai és Természettudomáyi Értesitö, 34:04 9, 96. (i Hugaria). [KU06] [SW73] A. Kimura ad T. Uyematsu. Multitermial source codig with complemetary delivery. I Proc. Iteratioal Symposium o Iformatio Theory ad its Applicatios (ISITA), pages 89 94, October D. Slepia ad J. K. Wolf. Noiseless codig of correlated iformatio sources. IEEE Tras. Iform. Theory, 9(4):47 480, July 973. [WWW02] A. D. Wyer, J. K. Wolf, ad F. M. J. Willems. Commuicatig via a processig broadcast satellite. IEEE Tras. Iform. Theory, 48(6): , Jue Some materials will be available at

22 Appedi SITA2006 i Hakodate p. 22

23 SITA2006 i Hakodate p. 23 Variable-legth codig Basically the same as the FF codig scheme. The codeword is y y2 y3 y4 y5 ide assiged to the type Codig tables are created for every type Q P(X Y), ad therefore codewords are assiged to every sequece pair. ma{ep(h(v Q X )), ep(h(w Q Y ))} differet symbols are ecessary to fill the codig table with.

24 SITA2006 i Hakodate p. 24 FV codig theorems (/2) Theorem 4. (Direct part) There eists a sequece of uiversal FV codes {(ϕ, ϕ (), ϕ (2) )} = such that for ay iteger ad ay source (X, Y ) the overflow probability is bouded as e (R) def. = Pr {l(ϕ (X, Y )) > (R + ɛ )} { } ( + ) X Y ep mi D(Q XY P XY ) Q XY S (R), where ɛ 0 ( ). This implies that there eists a sequece of uiversal FV codes {(ϕ, ϕ (), ϕ (2) )} = that satisfies lim sup l(ϕ (X, Y )) R F (X, Y ) a.s.

25 SITA2006 i Hakodate p. 25 FV codig theorems (2/2) Theorem 5. (Coverse part) Ay sequece of FV codes {(ϕ, ϕ (), ϕ (2) )} = must satisfy e (R) for the source (X, Y ) { } ep mi ( + ) X Y D(Q XY P XY ). Q XY S (R+ɛ ) for a give real umber R > 0 ad ay iteger, where ɛ 0 ( ).