On Achievable Rates for Multicast in the Presence of Side Information

Transcription

1 On Achievable Raes for Mulicas in he Presence of ide Informaion Mayank Bakshi, Michelle ffros Deparmen of lecrical ngineering California Insiue of echnology Pasadena, California 91125, UA mail: mayank, II 2008, orono, Canada, July 6-11, 2008 Absrac We invesigae he nework source coding rae region for neworks wih muliple sources and mulicas demands in he presence of side informaion, generalizing earlier resuls on mulicas rae regions wihou side informaion. When side informaion is presen only a he erminal nodes, we show ha he rae region is precisely characerized by he cu-se bounds and ha random linear coding suffices o achieve he opimal performance. When side informaion is presen a a non-erminal node, we presen an achievable region. Finally, we apply hese resuls o obain an inner bound on he rae region for neworks wih general source-demand srucures. Fig. 1. Y X s 1 1 Y 2 Y m A mulicas nework wih side informaion a he sink I. INRODUCION he rae region for a mulicas nework was characerized for independen sources by Ahlswede e al. in [1], wherein he achievabiliy of he cu-se bounds was shown. For dependen sources, Ho e al. [2] proved ha cu-se bounds are again igh for mulicas demands. In his paper, we generalize hese resuls o incorporae joinly disribued source and side informaion random variables in mulicas neworks. In heorem 1, we show ha he cu-se region is achievable wih linear coding when each sink is a erminal node wih access o a disinc side informaion random variable. (ee Fig. 1. heorem 2 gives an alernaive proof of his achievabiliy resul ha does no rely on linear codes; his approach allows us o use unsrucured random binning in he derivaions ha follow. heorem 3 generalizes heorem 2 o allow side informaion a one non-sink node. heorem 4 applies he above resuls o find an achievable region for neworks wih general source-demand srucures. II. PRLIMINARI A nework N =(V, is a direced, acyclic graph wih verex se V and noiseless edge se (V V. For each v V, we use Γ i (v and Γ o (v o denoe he incoming and ougoing edges, respecively, for node v. We likewise use Γ i (A = v A Γ i (v \ (A A and Γ o (A = v A Γ o (v\(a A, respecively, o represen he se of edges coming ino and emerging from a se of verices A V.A cu is a subse of he verex se V. For any cu C V and 0 his maerial is based upon work parially suppored by NF Gran No. CCF and Calech s Lee Cener for Advanced Neworking. verex v V, 1 if v C I C (v 0 oherwise. We use V and V o denoe he ses of source and sink nodes, respecively. ach source node s has no inpu edges and one oupu edge; each sink node has no oupu edges. While here is no loss of generaliy in he firs assumpion, requiring sink nodes o be erminal nodes is a resricive assumpion when side informaion is available only a sink nodes. ach source node s observes a source random process X s X s, and each sink node observes a side-informaion random process Y Y. For any index se I, we denoe he ordered I uple (r i : i I by he shor-hand noaion r I. hus, for A and B,we use X A =(X s : s A and Y B =(Y : B o denoe he vecors of source and side-informaion random variables respecively. ach inermediae node v V \ ( has access only o he codewords received on he edges in Γ i (v. he random process ( (i,y (i} i=1 is drawn i.i.d. from known probabiliy mass funcion P (. We consider neworks wih mulicas demands and receiver side informaion, as shown in Fig. 1. hus each demands he complee collecion of sources ; he side informaion may differ from one sink o he nex. Rae regions for mulicas neworks wihou side informaion appear in [1] for independen sources and [2] for dependen sources. For any collecion of raes R 0, an(n, (2 nre e /08/$ I 1661

2 II 2008, orono, Canada, July 6-11, 2008 nework code (f f defines encoders (v,v : X n v 1,...,2 nr (v,v } v, (v, v f (v,v : 1,...,2 nre } 1,...,2 nr (v,v } e Γ i(v v, (v, v and decoders g : e Γ i(1,...,2 nre } Y n X s. s During ransmission, he above maps are appropriaely sequenced o ensure ha a each node, he maps corresponding o he incoming edges are applied (and heir oupus received prior o applying he maps corresponding o he ougoing edges. We say ha (f } n=1 is a valid sequence of codes if he probabiliy of an error a he receivers vanishes as n increases wihou bound; more precisely, if he random variable F e denoes he codewords observed on he edge e corresponding o he inpu X n, hen (F lim n Pr(g Γ i(,yn X n 0 for each. he se of achievable rae vecors R for he nework is all raes R for which valid sequences of (n, (2 nre e codes exis. III. MULICA WIH ID INFORMAION A H INK he proofs of previous mulicas resuls wihou side informaion use random binning for code design on independen sources [1] and random linear coding for code design on (boh independen and dependen sources [2]. Random linear code design is a form of random binning ha adds addiional srucure o he random bin choices. his srucure is useful in pracice. heorem 1 generalizes he proof of [2] o allow decoder side informaion. o make he discussion precise, a (n, (2 nre e linear code (f is a se of mappings such ha f (v,v = b v (Xv n v, (v, v e Γ a i(v ef e v/, (v, v Here, b s : X n s F nre 2 is an arbirary (possibly non-linear funcion and a e F nrin,e nre 2 defines a linear map wih R in,(v,v = e Γ R i(v e. he decoder mappings (g : s are suiably chosen (ofen non-linear funcions. An (n, (2 nre e linear code (f is a random linear code if he coefficiens a e } and mappings b v } are chosen independenly and uniformly a random. Le R and R L denoe he se of rae vecors ha are achievable hrough arbirary codes and linear codes, respecively, on nework N. Le R C be he se of rae vecors ha saisfy he cu-se bounds on N [3], [2], i.e., R R C if and only if for any C V, R e H(X C \C,Y \C. (1 Xs 1 Fig. 2. Y he nework N heorem 1 characerizes he rae region for mulicas neworks in he presence of side informaion a he sinks and also shows he sufficiency of linear codes for achieving his region. heorem 1: R = R L = R C. Proof: All raes achievable hrough random linear coding lie in R C. hus, R L R R C (cf. [3]. In he following, we show ha R C R L. Define N o be he nework obained from N by deleing all he sink nodes excep he node. he resuling nework has he edge se ( \ Γ i ( \}, sources, and side informaion Y a he only sink. (ee Fig. 2. ince he side informaion is available a he sink, nework N is equivalen o a (muli-source mulicas problem wih sources (,Y. hus, random linear codes achieve any rae R ( ha saisfies R e H(X C \C,Y (2 for all C V [2, heorem 6]. Denoe by R C, he se of rae vecors R such ha R ( saisfies (2. Now, consider any R R C,, and le (f } be a code obained by assigning random coefficiens o a linear code a rae R. For his code, Pr(g (F Γ,Y i( for some Pr(g (F Γ,Y i( < ɛ for sufficienly large n by he union bound since rae R ( is achievable for each N and he random linear encoding operaion depends only on he inpu raes and oupu raes a each node. herefore, R is achievable for he nework N. Nex, we show ha R C = R C,. ince all R R C, are achievable, R C, R C. o see he reverse inclusion, le R R C. For any C V, for which C le C = C ( \}, and le C = C \ ( \}. hen, Γ o (C =Γ o ( C and he cu-se inequaliy corresponding o he cu C in he nework N is R e H( C X C c,y \C, (3 e Γ o(c which is he same as he cu-se inequaliy corresponding o he cu C in N. An (n, (2 nre e code (f is generaed by random binning if for each v, (v, v, and x n X n v, f (v,v (xn is chosen uniformly a random from alphabe 1,...,2 nr (v,v }; and for each v, (v, v and each i e Γ 1, i(v 2,...,2nRe }, f (v,v (i is chosen uniformly 1662

3 II 2008, orono, Canada, July 6-11, 2008 X s 1 Fig. 3. Z v z Y 1 Y m Y 2 ide informaion a a non-sink node v Z a random from alphabe 1,...,2 nr (v,v }.WeuseRB o denoe he se of rae vecors ha are achievable for a given nework using random binning. While random linear coding is a low-complexiy mehod for achieving random binning, he codewords corresponding o differen inpus are no necessarily independen of each oher. hus, heorem 1 does no imply R = R B = R C. heorem 2 generalizes [1] firs o dependen sources and hen o allow side informaion, hereby giving an alernaive proof ha R = R C. he proof of heorem 2, which appears in he Appendix, is used in he proof of heorem 3 in he nex secion. heorem 2: R = R B = R C. IV. MULICA WIH ID INFORMAION A A NON-INK NOD Now suppose ha side informaion random variable Z is presen a node v Z /. (ee Fig. 3. We again denoe he sources by and assume ha each sink node has access o side informaion Y and wishes o reconsruc.o obain an achievabiliy bound we firs encode Z ino a separae codeword for each subse of he sinks and hen sequenially mulicas each codeword o is corresponding sinks using he earlier codewords as side informaion. We hen mulicas o all of he receivers using he codewords received by each sink as he side informaion for ha sink. We calculae he rae required for each mulicas using heorem 1. Le τ : τ } be he subses of. For any permuaion σ of, any, and any τ for which τ le (σ,, τ τ : τ,σ 1 (τ <σ 1 (τ} be he indices of hose codewords received by node before he codeword for τ. Finally, for any A V, le (σ, A, τ A (σ,, τ. heorem 3: Le U = U τ : τ } be a se of random variables saisfying he Markov chains U τ Z (,Y. hen rae vecor R is achievable if here exiss a permuaion σ of for which R e I C (v Z τ:τ C [ H ( Uτ Y \C,U (σ, \C,τ H(Uτ Z ] +H ( C \C, (U τ : τ C,Y \C for all C V. (4 Proof: Again, he ouline of our sraegy is as follows. Firs, we encode he side informaion source Z ino codewords U, where for each τ, U τ is he codeword for subse τ of he sinks. Nex, we ransmi each U τ o he sinks in τ in he order described by permuaion σ. We rea each ransmission as a single-source mulicas wih side informaion U (σ,,τ a each receiver τ. Finally, we describe sources o he receivers; by his ime, he codewords available a receiver are (U τ : τ. We apply heorem 1 o calculae he rae required for each such mulicas. he auxiliary random variable U τ capures informaion presen in Z ha is useful o all τ. We follow he approach of previously solved coded side-informaion problems (cf. [4], [5], [6] in designing he codeword using each U τ. For each τ, we choose a random variable U τ ha saisfies he given Markov chain condiion. Nex, we generae 2 n R e τ lengh-n codewords U (1,τ,...,U (2n Rτ e,τ such ha each U (i,τ 1,...,U n (i,τ is drawn i.i.d. according o he marginal of U τ. Define he encoder mapping α τ : Z n 1,...,2 n R e τ }, where α τ (z n is an index i for which (z n,u (i,τ A ɛ (Z, U τ. Following he proof in [4], he exisence of such an index occurs wih probabiliy approaching 1 provided R τ I(Z; U τ. Le I τ = α τ (Z n. o ransmi he indices I o he respecive sinks, we fix a permuaion σ of and hen for each i 1,...,2 1} we mulicas I σ (i wih rae allocaion R σ(i o he verices in σ(i. Noe ha he indices I (σ,,τ are available a he sink earlier han he index I τ. hus, in order o achieve asympoically vanishing error probabiliy for decoding I τ a he sink τ, i suffices o perform random binning a each inermediae node while ensuring ha he rae R τ saisfies R τ e I C (v Z [I(U τ ; Z I(U τ ; Y,U (σ,,τ ] for each C V. he above can be seen by combining he binning argumen of [5] (see also [6] and he proof from heorem 2. Replacing he righ side of he above bound by is maximal value across all sinks C c, we find ha any R τ saisfying R τ e I C (v Z [I(U τ ; Z min \C I(U τ ; Y,U (σ,,τ ] for each C V is sufficien o make he error probabiliy vanish asympoically. ubracing H(U τ from each of he above erms gives [ Re τ I C (v Z H(U τ Z ] + max H(U τ Y,U (σ,,τ \C Furher simplificaion using an argumen similar o he one ha les us obain he region in (3 from he one given in (2 1663

4 II 2008, orono, Canada, July 6-11, 2008 in heorem 1 equals he se of rae allocaions R τ ha saisfy Re τ I C (v Z [ H(U τ Y \C,U σ, \C,τ H(U τ Z]. Adding he raes over all τ, and finally, adding he rae required o mulicas wih Y and (U τ : τ presen as side informaion a he sink node, we obain he achievabiliy resul given in (4. V. AN INNR BOUND ON H RA RGION WIH GNRAL DMAND RUCUR In his secion, we use he resul of he previous secion o find an inner bound on he rae region for general demands. We again denoe he sources by and he demands by Y, where for each, Y = ( for some (. We obain an achievable region by saisfying he demands Y hrough sequence of mulicas sessions. For each mulicas ransmission, he demands me in previous mulicas sessions are reaed as side informaion for he curren session.for s, le s = : s (}. Le = σ : σ is a permuaion of }. For σ, le R σ denoe he se of rae vecors R saisfying he following inequaliy for all C V : R e k=1 max H(X σ(k} ( X σ(1,...,σ(k 1} (,(5 \C and le R denoe he convex hull of σ K R σ. he following heorem assers he achievabiliy of R. heorem 4: Le R denoe he se of achievable raes for a nework N. hen, R R. Proof: For any ordering σ of, consider he sequence of mulicas sessions such ha he k-h session has mulicas demands X σ(k a he sink nodes for which σ(k (. By heorem 1, he rae vecor R (σ,k is sufficien o mee he demands for he k-h mulicas session, if he following condiion is saisfied: R (σ,k e max H(X σ(k} ( X σ(1,...,σ(k 1} (. (6 \C Adding he raes required for each of he mulicas sessions gives he achievabiliy of he region R σ given in (5 for each σ. By he convexiy of he rae region, R is achievable. While he closed-form expression for he above rae-region may be difficul o analyze, i is easily compuable algorihmically. I should be noed, however, ha he above rae region is no igh in general. In he following example, R has no igh rae poins. X1 X 2 X 3 a b Fig. 4. xample 1 X 1 X 2 X 2 X 3 X 3 X 1 xample 1: Consider he nework shown in Fig 4. Le X 1 ake values uniformly in 0, 1, 2}, p X2 X 1 (y x =1/2 for y x, (x + 1(mod 3} and X 3 =(1 X 2 X 1 (mod 3. hen, for all possible σ 1,2,3},anyR R σ saisfies: R (a,b log = log On he oher hand, for any achievable rae, he vecor (X 1,X 2,X 3 is decodable a he node a. hus, is suffices o ensure a rae of H(X 1,X 2,X 3 = log on he link (a, b. his proves ha he rae poin any rae poin R R is sricly subopimal. VI. CONCLUION We generalize earlier mulicas rae region bounds o allow side informaion a he decoders. We also generalize o neworks wih side informaion a one inermediae node in addiion o he side informaion a he sinks. he generalizaion akes an approach similar o ha used in he coded side informaion problem. he given bounds are ineresing boh on heir own and for heir applicabiliy in proving oher ineresing bounds. For example, we can bound he rae region for a nework wih muliple mulicass by considering each mulicas in urn and reaing informaion received from earlier mulicass as side informaion available o he corresponding sinks. APPNDIX Proof of heorem 2: ince all raes in R B are achievable, R B R C. o prove R B R C, we firs show ha R B is a convex se. Le R 1,,R 2, R B. Le f 1,,g 1, } and f 2,,g 2, } be wo valid sequences of codes, henceforh called he componen codes, ha achieve he raes R 1, and R 2, respecively. Define a (n, (2 nλ R1,e+ n(1 λ R2,e e code ( f, g by ime-sharing f 1, and f 2, and appropriaely defining he decoder funcions g as follows: f e g = f ( λn 1,e f ( (1 λn 2,e e = g ( λn 1, g ( (1 λn 2,. 1664

5 II 2008, orono, Canada, July 6-11, 2008 For he composie code hus formed, Pr( g ( F Γ i(,yn X n Pr(g ( λn 1, (F ( λn 1,Γ i(,y λn X λn +Pr(g ( (1 λn 2, (F ( (1 λn 2,Γ i(, (Y n λn +1 f ( n λn +1 hus, (, g } is a valid sequence of codes. ince he componen encoders are chosen independenly and each of hem is uniformly random mapping, i follows ha he composie encoders are also uniform and independen, and hence, have he same disribuion as a code ha would have been formed by random binning. Furher, he rae vecor corresponding o he code hus consruced approaches λr 1, + (1 λr 2, asympoically. hus, λr 1, +(1 λr 2, R B. herefore, R B is a convex se. Now, le R be a boundary poin of R C, ha is R is a rae vecor such ha if R R C and R e R e for some e, hen here exiss an e for which R e >R e. We claim ha e Γ R i( e = H(. o see his, le us assume oherwise. By he achievabiliy of R C proved in [2], here exiss a sequence of valid (n, (2 nre e codes, say (f }. Now, perform random binning on each Xr s o obain he nework N which has he same se of verices and edges as N, while he r-dimensional sources X r are replaced by sources B, where, B s = (b (Xs r : s, where b s : X r s X r s is a random binning operaion a rae R s r saisfying x r b 1 s (x r } x r b s (X r s. he las condiion can be ensured by appropriaely relabeling he value of b s ( in each bin. ince he code sequence (f } is valid for a mulicas wih as sources, i is valid for mulicas wih sources B oo. hus, sequence of codes ( f, g } is a valid code for he nework, where he encoding funcions f are consruced as f e b = s f e e = e s for some s oherwise f e and he decoding funcions g are suiably defined. By lepian-wolf heorem ([7], B is sufficien o reconsruc X r wih error probabiliy vanishing asympoically wih r, as long as R s s r rh(. his shows ha here is an achievable rae R s.. Re R R e e and s es = H(, which conradics he assumpion ha R is a igh rae. hus, s R e s = H(X s : s. Define he nework N r wih he same se of verices and edges as N and he sources replaced by B, where each b s is a random binning operaion a rae R s + ɛ. Le Ñr be he nework obained by replacing B in N r by B, where he sources B are independen of each oher, bu have he same firs-order marginal disribuion as B. By he proof used in he achievabiliy resul in [1] for he case of mulicas wih independen sources, he error probabiliy for random binning codes on Ñr approaches zero asympoically. Furher, since R is a igh poin, s H( B s = s H(B s < H(B +r ɛ. ince ɛ is arbirary, i follows ha by using he same code on each link as Ñr, we see ha random binning achieves he rae vecor rr in N r, and hence, R in N. o esablish ha he error probabiliy for a code formed by random binning approaches 0, consider any sequence of codes f (, g } ha is valid for he nework Ñr. By using he same codes on he nework N r, he error probabiliy saisfies he following: Pr( g ( F Γi( B = y n ( Q s bs(xr s n y n :eg y n ( Q s bs(xr s n y n :eg P B (y n P eb (y n + d V (P B,P eb, where d V (p, q denoe he variaional disance beween he disribuions p and q. he sequence of inequaliies is furhered by he use of Pinsker s inequaliy as follows: Pr( g ( F Γi( (B n P eb (y n y n ( Q s bs(xr s n y n :eg + y n Q s Bs(Xr s n y n :eg 2nD(P B P eb P eb (y n + 2nɛ ince he choice of ɛ is independen of n, we can make he second erm vanish by choosing ɛ =1/n 2. he firs erm vanishes because he code ( f, g is a valid code for he nework Ñr. hus, R R B. RFRNC [1] R. Ahlswede, N. Cai,. Y. R. Li, and R. Yeung. Nework informaion flow. I ransacions on Informaion heory, I-46(4: , July [2]. Ho, R. Koeer, M. Médard, M. ffros, J. hi, and D. Karger. A random linear nework coding approach o mulicas. I ransacions on Informaion heory, 52(10: , Ocober [3]. M. Cover and J. A. homas. lemens of Informaion heory. Wiley, [4] R. Ahlswede and J. Körner. ource coding wih side informaion and a converse for degraded broadcas channels. I ransacions on Informaion heory, I-21(6: , November [5] A. D. Wyner and J. Ziv. he rae-disorion funcion for source coding wih side informaion a he decoder. I ransacions on Informaion heory, I-22(1:1 10, January [6]. Berger, K.B. Housewrigh, J.K. Omura,. ing, and J. Wolfowiz. An upper bound on he rae disorion funcion for source coding wih parial side informaion a he decoder. I ransacions on Informaion heory, I-25(6: , November [7] D. lepian and J. K. Wolf. Noiseless coding of correlaed informaion sources. I ransacions on Informaion heory, I-19: ,