The Capacity Region for Multi-source Multi-sink Network Coding

Transcription

1 The Capacity Region for Multi-source Multi-sink Network Coding Xijin Yan Dept. of Electrical Eng. - Systems University of Southern California Los Angeles, CA, U.S.A. xyan@usc.edu Raymond W. Yeung Dept. of Information Eng. The Chinese University of Hong Kong N.T., Hong Kong whyeung@ie.cuhk.edu.hk Zhen Zhang Dept. of Electrical Eng. - Systems University of Southern California Los Angeles, CA, U.S.A. zzhang@commsci1.usc.edu Abstract The capacity problem for general acyclic multisource multi-sink networks with arbitrary transmission requirements has been studied in [1], [2] and [3]. Specifically, inner and outer bounds of the capacity region were derived respectively in terms of Γ n and Γ n, the fundamental regions of the entropy function. In this paper, we show that by carefully bounding the constrained regions in the entropy space, we obtain the exact characterization of the capacity region, thus closing the existing gap between the above inner and outer bounds. I. INTRODUCTION Consider a multi-source multi-sink network in which more than one mutually independent information sources are generated at possibly different nodes and each of the information sources is multicast to a specific set of sink nodes. We assume the network is acyclic and the channels are free of error. Unlike the single-source multicast network coding problem, the capacity region has an explicit Max-flow Mincut representation [4], the counterpart problem for multisource multi-sink network coding with arbitrary transmission requirements is considerably more complex. Except for a few explicit outer bounds that have been discovered recently in [2], [5], [6] and [7], the tightest theoretical characterization which has been obtained so far makes use of the tools developed in the theory of information inequalities [1]. Specifically, an inner bound and an outer bound were derived in terms of Γ n and Γ n respectively, which are fundamental regions of the entropy function. In this paper, we determine the exact capacity region for general acyclic multi-source multi-sink networks using an entropy function characterization. In particular, we show that by carefully bounding the constrained regions in the entropy space, we obtain the exact characterization of the capacity region, thus closing the existing gap between the above inner and outer bounds. The rest of the paper is organized as follows. In section II, we present a formal problem formulation and introduce some preliminaries on strongly typical sequences and entropy functions. In section III, we present the main result, i.e., an exact characterization of the capacity region. Proofs of the main result are given in section IV and final conclusions are drawn in section V. A. Network Model II. PRELIMINARIES Let G = V, E) denote an acyclic multi-source multi-sink communication network, V and E are the set of all nodes and the set of all channels. We assume each channel e E is error-free with a positive capacity constraint R e and all nodes i V are ordered in a way such that if there exists a channel from a node i to a node j, then the node i precedes the node j. We further define Ini) = j, i) E : j V to be the set of channels directed into node i and Outi) = i, j) E : j V to be the set of channels directed from i. Let S V be the set of all source nodes and T V be the set of all sink nodes. Without loss of generality, we assume G has the structure such that each source node has no input channels and each sink node has no outgoing channels. We assume all sources are uniformly distributed and mutually independent with finite alphabet X s = 1, 2,, 2 nτs for all s S, τ s is the source information rate at s S. A sink node t T requires the data from a set of sources βt) S to be decoded. In the case when βt) = S for all t T, the given network can be simply treated as a single source multicast network [4]. To allow for a general treatment of networks with arbitrary transmission requirements, we assume that βt) can be any subset of S for all t T. For clarity of notation, we sometimes use a superscript on a vector e.g. x n ) to specify the dimension of the vector, which will be distinguished from a superscript in parentheses e.g. x k) ) used to specify the index of a vector in a sequence e.g. x 1), x 2), ). The complement of a set A is represented by A c. For consistency, we further assume all the logarithms are in the base 2. B. Capacity Region Consider a block code of length n. Definition 1: An n, η e, e E), τ s, s S), t, t T )) block code of length n on a given communication network is defined by 1) for all source node s S and all channel e Outs), a local encoding mapping k e : X s 0, 1,, η e ; 1)

2 2) for all node i V \ S T ) and all channel e Outi), a local encoding mapping k e : 0, 1,, η d 0, 1,, η e ; 2) d Ini) 3) for all sink node t T, a decoding mapping g t : 0, 1,, η d X s ; 3) d Int) s βt) 4) for all sink node t T, a decoding error probability t = Pr g t X S ) X βt), 4) g t X S ) is the value of g t as a function of X S. Definition 2: An information rate tuple ω = ω s : s S), ω 0 componentwise) is achievable if for any ɛ > 0, there exists for sufficient large n an n, η e, e E), τ s, s S), t, t T )) code such that for all e E, for all s S, and n 1 log η e R e + ɛ 5) τ s ω s ɛ 6) t ɛ 7) for all t T. Definition 3: The capacity region denoted by R is the set of all achievable information rate tuple ω. C. Strongly Typical Sequences Consider an information source X k, k 1 X k are i.i.d. with probability distribution px). Let X denote the generic random variable, HX) < and S X be the support of X. Definition 4: The strong δ-typical set T[X]δ n with respect to px) is the set of sequences x n = x 1, x 2,, x n ) X n such that Nx; x n ) = 0 for x / S X, and 1 n Nx; xn ) px) δ, x Nx; x n ) is the number of occurrences of x in x n, and δ is an arbitrarily small positive real number. Lemma 1: Strong AEP) Let η be a small positive quantity such that η 0 as δ 0. 1) If x n T n [X]δ, then 2 nhx)+η) px n ) 2 nhx) η). 8) 2) For n sufficiently large, P r X n T[X]δ n > 1 δ. 9) 3) For n sufficiently large, 1 δ)2 nhx) η) T n [X]δ 2 nhx)+η). 10) For an i.i.d. bivariate information source X k, Y k ) : k 1 with probability distribution px, y). Let X, Y ) denote the pair of generic random variables, HX, Y ) <. Definition 5: The strongly jointly δ-typical set T[XY n ]δ with respect to px, y) is the set of x n, y n ) X n Y n such that Nx, y; x n, y n ) = 0 for x, y) S XY, and 1 n Nx, y; xn, y n ) px, y) δ, x y Nx, y; x n, y n ) is the number of occurrences of x, y) in x n, y n ) and δ is an arbitrarily small positive real number. Strong typicality satisfies the following properties. Lemma 2: Consistency) If x n, y n ) T[XY n ]δ, then xn T[X]δ n and yn T[Y n ]δ. Lemma 3: Preservation) Let Y = fx). If x n = x 1, x 2,, x n ) T[X]δ n, then fxn ) = y 1, y 2,, y n ) T[Y n ]δ, y i = fx i ) for 1 i n. Lemma 4: Strong JAEP) Let X n, Y n ) = X 1, Y 1 ), X 2, Y 2 ),, X n, Y n )), X i, Y i ) are i.i.d. with generic pair of random variables X, Y ). Let λ be a small positive quantity such that λ 0 as δ 0. 1) If x n, y n ) T n [XY ]δ, then 2 nhx,y )+λ) px n, y n ) 2 nhx,y ) λ). 11) 2) For n sufficiently large, Pr X n, Y n ) T[XY n ]δ > 1 δ. 12) 3) For n sufficiently large, 1 δ)2 nhx,y ) λ) T n [XY ]δ Lemma 5: For any x n T[X]δ n, define T n [Y X]δ xn ) = If T[Y n X]δ xn ) 1, then 2 nhy X) γ) T n 2 nhx,y )+λ). 13) y n T n [Y ]δ : xn, y n ) T n [XY ]δ [Y X]δ. 14) 2 nhy X)+γ), 15) γ 0 as n and δ 0. The generalization to a multivariate distribution is straightforward. A more thorough introduction to strongly δ-typical sequences can be found in [2] Chapter 5. D. The Region Γ N Let N be a nonempty set of random variables and Q N = 2 N \ φ with cardinality Q N = 2 N 1. Let H N be the Q N -dimensional Euclidean space with the coordinates labeled by h A, A Q N. A vector h = h A : A Q N ) in H N is said to be an entropy function if there exists a joint distribution for all X N such that h A = HX : X A) for all A Q N. We then define the region Γ N = h H N : h is an entropy function. 16) Therefore, by the above definition, there exists an oneto-one mapping between each vector h in Γ N and some set of random variables whose joint entropies correspond to

3 the elements in h. Since an arbitrary information inequality equality) can be regarded as a half-space hyperplane) in H N, it cuts Γ N into a subregion that maps to all sets of random variables that possess this property. Lemma 6: Basic properties of Γ N : 1) Γ N contains the origin. 2) Γ N, the closure of Γ N, is convex. 3) Γ N is in the nonnegative orthant of the space H N, i.e., Γ N h H N : h A 0 for all A Q N. III. MAIN RESULT Consider the set of all information rate tuples ω such that there exist auxiliary random variables, s S and U e, e E which satisfy the following conditions: H ) ω s, s S 17) HY S ) = s S H ) 18) HU Outs) ) = 0, s S 19) HU Outi) U Ini) ) = 0, i V \ S T ) 20) HU e ) R e, e E 21) HY βt) U Int) ) = 0, t T, 22) U e is an auxiliary random variable associated with the codeword sent on channel e, and Y S, U Ini) denote respectively the sets : s S, U e : e Ini), etc. For a given acyclic multi-source multi-sink network G, let N = : s S; U e : e E and define the following constrained regions in H N : C 1 = h H N : h YS = h Ys s S 23) C 2 = h H N : h UOuts) = 0, s S 24) C 3 = h H N : h UOuti) U Ini) = 0, i V \ S T ) 25) C 4 = h H N : h Ue R e, e E 26) C 5 = h H N : h Yβt) U Int) = 0, t T, 27) we used the notations h A A, h AA and h A for HA A ), HAA ) and HA ) for brevity. Clearly, 23) to 27) are the corresponding forms of 18) to 22) in H N. Let conγ N ) be the convex hull of Γ N. By time sharing, for any h conλ), Λ Γ N, there exists a random variable W and a set of jointly distributed random variables X : X N such that for any A Q N, h A W = HX : X A W ). Let C α = i α C i, α 1, 2, 3, 4, 5, we have the following theorem. Theorem 1: The capacity region for an arbitrary acyclic multi-source multi-sink network is characterized by )) R = Λ proj YS conγ N C 123) C 4 C 5, 28) for any A H N, proj YS A) = h YS : h A is the projection of A on the coordinates h Ys, s S, ΛA) = h H N : 0 h h, h A and Ā is the closure of region A. A. Proof of Achievability IV. PROOF OF THEOREM Let ω proj YS conγ N C 123) C 4 C 5 ). Then there exists an h conγ N C 123) C 4 C 5 such that ω = proj YS h). This implies that there exists a sequence h k) conγ N C 123) such that h = lim k h k). Note that h k) might not be in Γ N, thus is not necessarily an entropy function. However, this can be resolved by using time-sharing. Let W k) be a time-sharing variable, thus by the definition of Γ N, there exists a set of random variables N k) = W k) ; k) : s S; U e k) : e E whose entropy function corresponds to h k), i.e., H k) W k)) = ω s k), s S 29) H W k)) = H W k)) 30) s S H H U k) U k) Outs) Outi) Y k) S Y k) s k), W k)) = 0, s S 31) U k) Ini), W k)) = 0, i V \ S T ), 32) lim k ω s k) = ω s for all s S. Since h C 4 C 5 and h = lim k h k), it is implied that N k) must also satisfy H Y k) βt) H U e k) W k)) R e + ɛ k, e E 33) U k) Int), W k)) = δ k, t T, 34) ɛ k 0, δ k 0 as k. From the Caratheodory s Theorem, we may assume that the support S W of W k) has size at most 2 N. k) and U e k) are random variables representing the information source X s and the codeword sent on the channel e. View W k) as a time-sharing random variable. To prove the result, we need to construct a code for each value w of W k). The time sharing of the codes gives the overall performance of the code. By 30), are conditionally independent given W k) = w, and we consider n i.i.d. copies of them having generic distributions py s w) : s S. For sequences Y n s with block length n, the corresponding joint distribution is py n s w) = pn y s w) y n s = y1) s,, y s n) ). Since we next consider only the random variables in N k), we temporarily drop the index k for convenience. Similarly, being aware of the conditioning on W k) = w, we drop the conditioning index w unless otherwise specified. Now we construct a random n, η e, e E), ω s ɛ, s S), t, t T )) code by the following procedure: 1. Codebook a) For each source s S, generate 2 nτ s sequences of length n randomly and independently according to p n y s w). We call this codebook C s with indices 0, 1,, 2 nτs 1 and cardinality M s = 2 nτs, we assume 2 nτs is an integer for convenience. b) The construction of the random codes C s for the sources s S are done independently.

4 2. Encoding a) If the message is j at source node s, map it to the jth codeword in C s and call this sequence y n s j. b) By 31) and 32), for each channel e Outi), i V \ S T ), there exists a deterministic function u e such that U e = u e U d : d Ini)). Since the network is acyclic, we can show inductively that there exists another deterministic function ũ e such that U e = ũ e, s S). Let ζ e = T[U n e W =w]δ, by Lemma 5, we have ζ e 2 nhu e W =w)+η), η 0 as δ 0. Then from 33), by time sharing the code with respect to the probability distribution of W and letting δ be sufficiently small, we have the average rate over channel e being HU e W ) + η R e + ɛ k + η. 35) Thus for the fixed value W = w, by choosing an integer η e such that 2 nhu e W =w)+η) η e 2 nr e+ɛ k +η), 36) we can define a local encoding function k e such that if U n Ini) T [U n Ini) W =w]δ, Un e T[U n e W =w]δ. Thus we transmit the index of U n e in T[U n e W =w]δ as the codeword on channel e. Otherwise, we transmit a zero codeword on e. 3. Decoding For t T, define the decoding function g t : 0, 1,, η d C s. 37) d Int) s βt) Let C βt) = s βt) C s be the joint codebook for all the sources in βt) with cardinality M βt) = s βt) M s. We use the following strong typicality decoding. If the received codeword is nonzero for all d Int) and there exists a unique codeword y n βt) in C βt) such that U n Int), yn βt) ) T[U n Int) Y βt) W =w]δ, then let g tu n Int) ) = yn βt). Otherwise, declare a decoding error. For the above coding procedure, assuming that y n S is the source message from all sources and the codewords of C βt) are ordered as c n βt) i : i = 1,, M βt), the joint codeword c n βt) i is concatenated by βt) codewords from C s, s βt). For notational convenience, we omit the dimension index on c n βt) i and use c i instead. Thus Pr error W = w = Pr error y n S T n + Pr error y n S T n = Pr y n S T [Y n λ n + Pr error y n S T n + Pr Pr y n S T n Pr y n S T n error y n S T n 38) λ n 0 as n. Since y n S T [Y n implies Un Int) T [U n Int) W =w]δ, a decoding error occurs if and only if there are more than one codeword in C βt) that satisfy the strong joint typicality condition. Let c i C βt) be such a codeword, that is c i, U n Int) ) T [Y n βt) U Int) W =w]δ and c i y n βt). Call this event E i, then we have Pr error y n S T [Y n = Pr ci y n E i y n βt) S T [Y n = 1 Pr ci y n Ec βt) i y n S T [Y n a) = 1 Pr E1 c y n S T [Y n Mβt) 1) ) y n 1 1 Pr E 1 S T[Y n Mβt) b) c) 1 1 δ 2 nhy βt) W =w) δ k 3η) ) M βt) 1 1 M βt) δ 2 nhy βt) W =w) δ k 3η) ) = M βt) δ 2 nhy βt) W =w) δ k 3η) d) = M βt) δ 2 np s βt) HYs W =w) δ k 3η), 39) δ = 1 1 δ 1 as δ 0. The noted inequalities are explained as follows: a) follows since c i C βt) are i.i.d. uniformly distributed we abuse the notation here by assuming c 1 y n βt) ), and the total number of such codewords is M βt) 1. b) follows from Lemma 4, 5 and the conditioning on W = w. We omit the details for brevity. c) follows from the fact that 1 + a) n 1 + na. d) follows by applying 30). Now combine 38) and 39), we have Prerror W = w λ n + M βt) δ 2 np s βt) H W =w) δ k 3η) = λ n + δ 2 nµ δ k 3η) by choosing the codebook cardinalities M s = 2 nτ s = 2 nhys W =w) µ) and µ > δ k + 3η for all s S. Since the support size S W of W is uniformly bounded above, the time shared code has a vanishing error probability, i.e., Prerror = E[Prerror W = w] S W λ ) n + δ 2 nµ δ k 3η) 0, as n. Thus we proved the existence of N k), which asserts that proj YS h k) ) R. Letting k, we obtain ω = proj YS h) R. Invoking the definition of achievability, we obtain Λω) R, completing the proof of achievability. B. Proof of Converse Let ω R, then for 0 < ɛ k 0, we have a sequence of ) n k, η e k), e E), τ s k), s S), k) t, t T ) codes satisfying n 1 k log ηk) e R e + ɛ k, e E 40) τ s k) ω s ɛ k, s S 41) k) t ɛ k, t T. 42)

5 In the following discussion, we fix the value of k, and drop k from all notations. Let X s be uniformly distributed in the codebook X s and X s : s S be independent, then HX S ) = s S HX s ) 43) HU Outs) X s ) = 0, s S 44) HU Outi) U Ini) ) = 0, i V \ S T ) 45) HU e ) nr e + 2ɛ k ), e E 46) HX βt) U Int) ) nφ t n, ɛ k ), t T, 47) HX s ) nω s ɛ k ), s S 48) 43) follows from the independence of X s, s S. 44), 45) follows from the definition of the code. 46) follows from 40). 47) can be proved similarly as in [3] section 6.3 φ t n, ɛ k ) = 1 n + 2ɛ k R e + ɛ k ). 49) e Int) 48) follows from 41) and the fact that X s is uniformly distributed, i.e., for all s S HX s ) = log X s = log 2 nτs nτ s nω s ɛ k ). By letting = X s for all s S, we see there exists a sequence h k) such that for all s S, and h k) nω s ɛ k ), 50) h k) Γ N C 123 C n 4ɛ k C n 5ɛ k, 51) C n 4ɛ k = h H N : h Ue nr e + 2ɛ k ), e E C n 5ɛ k = h H N : h Yβt) U Int) nφ t n, ɛ k ), t T. Dividing 50) by n, we obtain n 1 h k) ω s ɛ k 52) for all s S. Since conγ N C 123) is convex and contains the zero vector, we have n 1 h k) conγ N C 123) C 4ɛk C 5ɛk, 53) C 4ɛk = h H N : h Ue R e + 2ɛ k, e E C 5ɛk = h H N : h Yβt) U Int) φ t n, ɛ k ), t T. Now define the set B n,k) = h conγ N C 123) C 4ɛk C 5ɛk : h Ys ω s ɛ k, for all s S. 54) Without loss of generality, we let ɛ k 0 monotonically. Note from 49) that φ t n, ɛ k ) is monotonic with respect to n and k, thus B n+1,k) B n,k) and B n,k+1) B n,k). For any fixed k, for sufficiently large n, from 52) and 53), we see that B n,k) is nonempty. Since each B n,k) is compact closeness by the definition, boundedness by the constraint in C 4ɛk ), we see that lim n Bn,k) is both compact and nonempty. By the same argument, lim lim k n Bn,k) is also nonempty. Thus there exists some h such that h conγ N C 123) C 4 C 5, 55) h ω s, for all s S. 56) Let r = proj YS h ), then we have r proj YS conγ N C 123) C 4 C 5 ), 57) r ω componentwise). 58) By 57) and 58), we finally obtain that )) ω Λ proj YS conγ N C 123) C 4 C 5. 59) This completes the proof. V. CONCLUSION In this work, we extended the previous work in [1], [2] and [3] on the capacity region for general acyclic multisource multi-sink networks. Specifically, we closed the gap between the existing inner and outer bounds by refining the constrained regions in the entropy space. This leads to an exact characterization of the capacity region for general acyclic multi-source multi-sink networks with arbitrary transmission requirements and thus completes the work along this line of research. However, how to explicitly evaluate the obtained capacity region remains an open problem in general. ACKNOWLEDGMENT The authors would like to give special thanks to Prof. Ning Cai and Dr. Lihua Song for their valuable comments. REFERENCES [1] L. Song and R. W. Yeung, Zero-error network coding for acyclic network, IEEE Trans. Inform. Theory, vol. 49, no. 12, pp , Dec [2] R. W. Yeung, A First Course in Information Theory. New York: Kluwer/Plenum, [3] R. W. Yeung, N. Cai, S.-Y. R. Li, and Z. Zhang, Network coding theory, Foundations and Trends in Communications and Information Theory, vol. 2, no. 4 and 5, pp , [4] R. Ahlswede, N. Cai, S.-Y. R. Li, and R. W. Yeung, Network information flow, IEEE Trans. Inform. Theory, vol. 46, no. 7, pp , July [5] X. Yan, J. Yang, and Z. Zhang, An outer bound for multisource multisink network coding with minimum cost consideration, IEEE Trans. Inform. Theory & IEEE/ACM Trans. Networking joint issue), vol. 52, no. 6, pp , June [6] N. Harvey, R. Kleinberg, and A. Lehman, On the capacity of information networks, IEEE Trans. Inform. Theory, vol. 52, no. 6, pp , June [7] G. Kramer and S. A. Savari, Edge-cut bounds on network coding rates, Journal of Network and Systems Management, vol. 14, no. 1, pp , Mar