On Network Coding of Independent and Dependent Sources in Line Networks

Transcription

1 On Network Codng of Independent and Dependent Sources n Lne Networks Mayank Baksh, Mchelle Effros, WeHsn Gu, Ralf Koetter Department of Electrcal Engneerng Department of Electrcal Engneerng Calforna Insttute of Technology Unversty of Illnos at Urbana-Champagn Pasadena, Calforna, CA 95, USA Urbana, IL 680, USA Emal: {mayank, effros, wgu}@caltech.edu Emal: koetter@uuc.edu Abstract We nvestgate the network codng capacty for lne networks. For ndependent sources and a specal class of dependent sources, we fully characterze the capacty regon of lne networks for all possble demand structures (e.g., multple uncast, mxtures of uncasts and multcasts, etc.) Our achevablty bound s derved by frst decomposng a lne network nto sngle-demand components and then addng the component rate regons to get rates for the parent network. For general dependent sources, we gve an achevablty result and provde examples where the result s and s not tght. I. INTRODUCTION To date, the feld of network codng has focused prmarly on fndng solutons for famles of problems defned by a broad class of networks (e.g., networks representable by drected, acyclc graphs) and a narrow class of demands (e.g., multcast or multple uncast demands). We here nvestgate a famly of network codng problems defned by a completely general demand structure and a narrow famly of networks. Precsely, we gve the complete soluton to the problem of network codng wth ndependent sources and arbtrary demands on a drected lne network. We then generalze that soluton to accommodate specal cases of dependent sources. Theorem summarzes those results. Theorem : Gven an n-node lne network N (shown n Fg. ) wth memoryless sources,..., n and demands,..., n satsfyng H(,..., ) 0 for all {,..., n}, the rate vector (R,..., R n ) s achevable f and only f, for < n, R H( j +,..., j, +,..., j ), () j+ provded one of the followng condtons holds: A. Sources,..., n are ndependent and each s a subset of the sources,...,. B. Sources,..., n have arbtrary dependences and each s ether a constant or the vector (,..., n ). C. Each source s a (potentally) dstnct subset of ndependent sources W,..., W k and each demand s any subset of those W,..., W k that appear n,...,. Lemmas 3, 4, and 5 of Sectons III-B, III-C, and III-D gve formal statements and proofs of ths result under condtons A, B, and C, respectvely. Case B s the multcast result of [] 3 n n R R R n n 3 Fg.. An n-node lne network wth sources,..., n and demands,..., n generalzed to many sources and specalzed to lne networks. We nclude a new proof of ths result for ths specal case as t provdes an mportant example n developng our approach. Central to our dscusson s a formal network decomposton descrbed n Secton II. The decomposton breaks an arbtrary lne network nto a famly of component lne networks. Each component network preserves the orgnal node demands at exactly one node and assumes that all demands at pror nodes n the lne network have already been met. (See Fg. for an llustraton. Formal defntons follow n Secton II.) Sequentally applyng the component network solutons n the parent network to meet frst the frst node s demands, then the second node s demands assumng that the frst node s demands are met, and so on, acheves a rate equal to the sum of the rates on the component networks. The gven soluton always yelds an achevablty result. The proofs of Lemmas 3, 4, and 5 addtonally demonstrate that the gven achevablty result s tght under each of condtons A, B, and C. Theorem shows that the achevablty result gven by our addtve soluton s tght for an extremely broad class of sources and demands n 3-node lne networks. In partcular, ths result allows arbtrary dependences between the sources and also allows demands that can be (restrcted) functons of those sources (rather than smply the sources themselves). The form of our soluton lends nsght nto the types of codng suffcent to acheve optmal performance n the gven famles of problems. Prmary among these are entropy codes, ncludng Slepan-Wolf codes for examples wth dependent courses. These codes can be mplemented, for example, usng lnear encoders and typcal set or mnmum entropy decoders []. The other feature llustrated by our decomposton s the need to retreve nformaton from the nearest precedng node where t s avalable (whch may be a snk), thereby n n n

2 avodng sendng multple copes of the same nformaton over any lnk (as can happen n pure routng solutons). Unfortunately, the gven decomposton fals to capture all of the nformaton known to pror nodes n some cases, and thus the achevablty result gven by the addtve constructon s not tght n general. Theorem 3 gves a 3-node network where the bound s provably loose. The falure of addtvty n ths functonal source codng example arses snce the component network decomposton fals to capture nformaton that ntermedate nodes can learn beyond ther explct demands. The same problem can also be replcated n a 4-node network, where all demands also appear as sources n the network. Theorem 4 shows that the gap between the addtve soluton and the optmal soluton can be large. II. PRELIMINARIES The followng notaton s useful n the dscusson that follows. For random varables A,..., A n, set S {,..., n}, and ndces, j {,..., n}, A S (A : S) and A j (A,..., A j ) denote vectors of A varables and A j ( m) (A j (),..., Aj (m)) denotes consecutve samples of Aj. An n-node lne network (N, n, n ) s a drected graph N (V, E) wth V {,..., n} and E {(, ),..., (n, n)}, as shown n Fg.. Node observes source and requres demand. The random process ( n ( ), n ( )) s drawn..d. accordng to probablty mass functon p(x n, y n ). A rate allocaton for the n- node lne network N s a vector (R : < n), where R s the rate on lnk (, + ). We assume that there are no errors on any of the lnks. Lne networks have been studed earler n the context of relable communcaton (e.g., [3]). A smple lne network s a lne network wth exactly one demand ( c at all but one node n the network). We next defne the component networks N,..., N n for an n-node lne network N wth sources n and demands n. (See Fg..) For each n, component N s an -node smple lne network. For each j <, the source and demand at node j of network N j are () j ( j, ) and () j c, respectvely; the source and demand at node are ()... and (). 3 n n R R R n N N N n R 3 n n Fg.. Component networks n n III. RESULTS A. Cutset bounds and Lne Networks Lemmas and relate the cutset bound to the achevable rate regon. Lemma : In an n-node lne network (N, n, n ), the cutset bounds are satsfed f and only f R max H( j + j+ j + ) < n () Proof: The reverse part s mmedate snce () s a subset of the cutset bounds. For the forward part, let (R : < n) satsfy (), and let T be a cut. Each cut s a unon of ntervals (T l kt (k) wth T (k) {m(k),..., m(k)+l(k) } {,..., n} and m(k ) + l(k ) < m(k)). Then, k R m(k) k max j0 m(k)+j H(m(k) m(k)+j m(k) ) H( T (k) T (k) ) k H( T T ). (3) Snce T s arbtrary, (3) gves the cutset bounds. Lemma : Let (N, 3, 3 ) be a 3-node lne network for whch the cutset bounds are tght on each of the component networks. Then, the achevable rate regon s the set R {(R, R ) : R > R m }, where R m H( ) + H( 3 3, ), R m H( 3 3 ). Proof: Converse: Let C and C be rate R and R m- dmensonal codes for lnks (, ) and (, 3) of N, and suppose (/m)h( ( m) ( m), C ) ɛ. Then, mr H(C ) mh( 3 3 ), mr H(C ) H( ( m), C ( m)) Now, mh( ) + H(C ( m), ( m)). H(C ( m), ( m)) I(C ; 3 ( m) ( 3, )( m)) mh( 3 3, ) H( 3 ( m) C, 3 ( m)) mh( 3, 3 ) mɛ. So R H( )+H( 3 3, ) ɛ and R H( 3 3 ), mplyng that by pckng arbtrarly small ɛ, (R, R ) R. and H(C C ) < ɛ. Achevablty: Let R R m + ɛ. Snce the cutset bound s tght on the components N and N 3 for suffcently large m, there exst m-dmensonal codes C, C, 3 and C 3 for the lnks (, ) n N, (, ) n N 3, and (, 3) n N 3 for whch m H(C ) H( ) + ɛ 3, (4) m H(C3 ) H( 3 3, ) + ɛ 3, (5) m H(C3 ) H( 3 3 ) + ɛ 3, (6)

3 and m H( ( m) ( m), C ) 3 for, 3. Let C CC 3 and C C. 3 Then, (C, C ) s a code for N satsfyng m H(C ) < R for, 3, m H( ( m) ( m), C ) < ɛ for,, and H(C C ) < ɛ. Thus, (C, C ) acheves rate (R, R ). B. Independent sources, arbtrary demands Lemma 3: Consder an n-node lne network (N, n, n ) wth n ndependent and D(), D() {,..., } for each n. Then, the rate regon s fully characterzed by the cutset bounds. In partcular, R H( jk+ D()\{k+,...,n}) < n s achevable. Proof: Let Q(, k) jk D(j). By Lemma, the cut-set bound gves R max k H( Q(,k), k +) max max k k H( Q(,k)\{,...,k}) j Q(,k)\{,...,k} H( j ). (7) Snce D() {,..., }, Q(, k) {,..., k} and Q(, k) \ {,..., k} Q(, k + ) \ {,..., k + }. Thus (7) becomes H( j ). R j Q(,n)\{,...,n} We wsh to acheve ths bound by codng for the component networks separately. Let R H( Q(+,n)\{+,...,n} ) + ɛ for < n. Snce the demand n network N j s D(j), by calculatng the demand across the varous lnks startng from the very last lnk, the followng rate allocaton s feasble: R j j H( D(j)\{j} ) + ɛ n R j j H( (D(j)\{j})\(D(j ) {j }) ) + ɛ n H( D(j)\({j,j} Q(j,j )) ) + ɛ n. R j H( D(j)\({,...,j} Q(,j )) ) + ɛ n. Addng the rates from component networks gves R j H( D(j)\({+,...,j} Q(+,j )) ) + ɛ n j+ j+ j+ j+ H( D(j)\({+,...,n} Q(+,j )) ) + ɛ n H( (D(j)\Q(+,j ))\{+,...,n} ) + ɛ n H( ( n j+ D(j))\{+,...,n}) + ɛ H( Q(+,n)\{+,...,n} ) + ɛ R C. Dependent sources, multcast Lemma 4: Let (N, n, n ) be an n-node lne network wth n arbtrarly dependent and n feasble multcast demands ( c or n for each ). Then Theorem holds. Proof: Let M{ n : n } be the multcast recevers. We denote the vertces of M by {m,..., m k }, where m < m j when < j. For each n, let d() mn{m M : m > }. Consder an achevable rate allocaton (R : < n) for N. For any < n, the cutset bound on R s tghtest possble f we choose the set of vertces for the cutset to be the set { +,..., d()}. Ths s true because addng extra vertces to ths set adds addtonal sources to t wthout ncreasng the set of demands. Therefore, for all < n, R H( n d() + ) H(d() d() + ) H( d() + ) (8) where the frst equalty follows snce H( n d() ) 0 as the demand s feasble for the network. Next, we show that for the component networks {N j } jn, there exst rate allocatons {(R j : j )} n j whch come arbtrarly close to satsfyng the above bounds wth equalty. Observe that t suffces to restrct our attenton to the networks {N j } j M. Fx ɛ > 0 and consder any m M. When, t suffces to encode j at rate H( j m j+ )+ ɛ/n [4]. Summng the over all sources that use use lnk (r, r+ ) rate achevable rate R m r R m r r j for network N m, where H( j m j+ ) + ɛ n H( r m r+ ) + ɛ H( r d(r) r+ ) + ɛ. (9) When >, n are avalable at the node m. Hence, the rate requred over the lnk (r, r + ) s zero for all r < m. for m r < m, [4] agan gves r Rr m H( n m m ) + + H( j m j+ ) + ɛ n H( r m r+ ) + ɛ jm + H( r d(r) r+ ) + ɛ. (0) Fnally, addng the rates over component networks gves R j Rd() H( d() + ) + ɛ R + ɛ. j+ Ths shows that the cutset bound n (8) s tght and s achevable by the approach based on component networks. Further, each R j s of the form H( j j +, j + ). D. A class of dependent sources wth dependent demands In ths secton, we consder sources and demands that are dependent n the followng way. We assume the exstence of underlyng ndependent sources W k such that the sources are W S() and the demands are W D() for n for {S()} n and {D()}n subsets of {,..., k}. In order

4 for the demands to be feasble, we requre D() j S(j) for each n. Lemma 5 characterzes the rate regon for lne networks wth the above knd of sources and demands. We need the followng notaton n order to state the lemma. For j < n, defne d (j) and s (j) as the frst occurrence after the vertex j of W n a demand and source respectvely. Lemma 5: Let (N, n, n ) be an n-node lne network wth W S() and W D() as defned above. Then the achevable rate regon s R j :s (j)>d (j) H(W ) j < n. Proof: We proceed by frst decomposng the network N nto k dfferent networks {N } k, each correspondng to a dfferent W out W k. For each k and A {,..., k}, let A W f A and A c otherwse. Let N be an n-node lne network wth sources S(),..., S(n) and demands D(),..., D(n). Note that each N s a lne network n whch both the sources and demands are ether W or constant. By result of Sec. III-C, t follows that the cutset bound s tght for such network and the rate allocaton (R j, : j < n}), defned by R j, { H(W ) f s (j) > d (j) 0 otherwse s achevable. Thus, for the parent network N, the rate allocaton (R j : j {,..., n}) s achevable, where R j k R j,. Further, as all the sources are ndependent, ths approach s optmal. Therefore, the rate regon for the network N s gven by all (R j : j {,..., n}) such that R j :s (j)>d (j) H(W ). Next, we show that the same can be obtaned by decomposng N nto smple networks N,..., N n. To ths end, we frst decompose the network N nto smple networks N,..., Nn, notng that the mnmum rate Rj, l requred for the lnk (j, j + ) n Nl s 0 f there s a demand or a source present n one of the nodes j +,..., l and H(W ) otherwse. Hence, Rj, l H( D(l) S(j+),..., S(l), D(j+),..., D(l ) ). Ths s an optmal decomposton, snce R j, Rj,. l lj+ Addng the rates over components N, R l j R l j, H( D(l) l tj+ S(t), l tj+ D(t)) H( D(l) l tj+, l tj+ D(t) ) () s achevable for N l. Fnally, note that lj+ R l j lj+ Rj, l R j, R j. Thus the rates for N can also be obtaned by summng lnkwse the rates for the component networks. By (), the sum s of the form clamed n Theorem -C. E. 3-node lne networks wth dependent sources We now restrct our attenton to 3-node lne networks of the knd shown n Fg. 3. Sources 3 are arbtrarly dependent and ther alphabets are fnte; demands 3 take the form f( ) and 3 g( 3 ) for some f : and g : 3. The followng result shows the tghtness of our decomposton appoach n ths case. Theorem : Gven ɛ > 0, for every rate vector (R, R ) achevable for N, there exst achevable rate allocatons R and (R, 3 R) 3 for component networks N and N 3 such that R + R 3 < R + ɛ and R 3 R + ɛ. 3 Fg. 3. f( ) g(,, ) 3 3 The three node lne network Proof: Let (R, R ) be achevable for N. Then, for m large enough, there exst codes (a m, b m ) for the frst and second lnk, respectvely such that (/m)h(a m ( ( m))) < R + ɛ, (/m)h(b m (a m ( ( m)))) < R + ɛ, and P r(ŷ 3 ( m) 3 ( m)) < ɛ. Let B m (k) b m (a m ( (m(k )+ mk))), F m (k) (f( (m(k )+),..., f( (mk))),,m (k) (m(k ) + mk). Allowng a probablty of error ɛ, the problem of codng for the network N can be reformulated as a functonal source codng problem for the network N () wth sources, and, and the demand (F, B ) as shown n Fg 4. Pror results on functonal codng ([5], [6], [7]) can Fg. 4.,, F An equvalent functonal codng problem now be appled. Specfcally, by evaluatng the functonal rate dstorton functon n [6], [7] at zero dstorton, the mnmal rate at whch, can be coded s gven by R nf b, I(, ;,, ) B

5 where, the nfmum s over the set P consstng of all, for whch,,, forms a Markov chan and H(F, B,,, ) 0. We show that the above rate can be splt nto two parts - the rate requred to to encode, so as to reconstruct F wth, as the sde nformaton, and the rate requred to be able to reconstruct B wth, and F as the sde nformaton. To ths end, let F,B P. Then, the followng hold: I( F,B ;,, ) I( F,B, F ;,, ) I(F ;,,, F,B ) I( F,B, F ;,, ) H( F,B, F, ) H( F,B, F,,, ) H( F,B, F, ) H( F,B,,,,, F ) H(F, ) + H( F,B F,, ) H( F,B,,,,, F ) H(F, ) + I( F,B ;, F,, ). () Snce F s a functon of,, H(F, ) s an achevable rate for the network N. Further, snce F,B P, t follows that F,B, (F,, ) s a Markov chan. Combnng t wth the fact that, H(B ) F,B,,, F ) 0, we note that I( F,B ;, F,, ) s a suffcent rate for functonal source codng wth regards to the functon B gven, and F as the sde nformaton. Therefore, ( I( F,B;, F,, ), H(B )) s an achevable rate for the network N 3, hence provng the theorem. F. Networks where addtvty does not hold Theorem 3: There exsts a 3-node network (N, 3, 3 ), for whch addng the best rate allocatons from the component networks does not yeld an optmal code. Proof: Consder the example shown n Fg. 5(a). Rate 3 f(,) (a) Rate (b) f(,) Fg. 5. (a) A set of achevable rates for N. (b) An optmal rate allocaton for component N. Let be dstrbuted unformly on {0, } and be ndependently dstrbuted unformly on {0,,, 3}. Let { 0 (x, y) {(0, 0), (0, ), (, ), (, )} f(x, y) (x, y) {(0, ), (0, 3), (, 0), (, 3)} Fg. 5(a) shows an achevable rate allocaton that can be acheved by the transmttng over both lnks. We next show that best possble rateallocaton achevable by optmzng over the achevable rate regons of the component networks s strctly greater than the above rate allocaton. To ths end, consder N (see Fg. 5(b)). By [5], the best possble rate on lnk (,) s. To prove our clam, t suffces to show that the rate requred over lnk (, ) N s non-zero. Ths follows because H(, f(, )) > 0, and the cutset bound requres at least rate H(, f(, )) > 0 across lnk (, ). Theorem 4: For any n 3, there exsts a n-node network (N, n, n ), such that for any achevable (R j : < j) for N j, there exsts an achevable (R : < n) for N wth j+ R j > R + Ω(n). Proof: Let and be ndependent sources unformly dstrbuted over {0, } and {0,..., n} respectvely. Defne f : {0, } {0,..., n} {0,..., n} as { y f y {,..., n } f(x, y) x f y n Consder the n-node lne network (N, n, n ), where, 3..., n, n c, and f(, ( ) for n. By the functonal source codng bound, the rate requred on each lnk of the -th component s at least. Ths rate s acheved by sendng on all the lnks. Thus, n j+ Rj n. In contrast, rate s suffcent for network N (sendng over all the lnks). Therefore, n j+ Rj R n. The left sde s at most O(n) for a network wth n components. Hence, n j+ Rj R Ω(n). ACKNOWLEDGMENT Ths materal s based upon work partally supported by NSF Grant Nos. CCF-00039, CCR and Caltech s Lee Center for Advanced Networkng. REFERENCES [] T. Ho, R. Koetter, M. Médard, M. Effros, J. Sh, and D. Karger. A random lnear network codng approach to multcast. IEEE Transactons on Informaton Theory, 5(0): , October 006. [] Imre Csszar. Lnear codes for sources and source networks: Error exponents, unversal codng. IEEE Transactons on Informaton Theory, IT-8:585 59, July 98. [3] P. Pakzad, C. Fragoul, and A. Shokrollah. Codng schemes for lne networks. In Proceedngs of the IEEE Internatonal Symposum on Informaton Theory, Adelade, Australa, September 005. [4] D. Slepan and J. K. Wolf. Noseless codng of correlated nformaton sources. IEEE Transactons on Informaton Theory, IT-9:47 480, 973. [5] A. Orltsky and J. R. Roche. Codng for computng. In IEEE Symposum on Foundatons of Computer Scence, pages 50 5, 995. [6] H. amamoto. Wyner-zv theory for a general functon of correlated sources. IEEE Transactons on Informaton Theory, IT-8(5): , September 98. [7] H. Feng, M. Effros, and S. A. Savar. Functonal source codng for networks wth recever sde nformaton. In Proceedngs of the Allerton Conference on Communcaton, Control, and Computng, Montcello, IL, September 004.