Strong Converse Theorems for Classes of Multimessage Multicast Networks: A Rényi Divergence Approach

Transcription

1 Strong Converse Theorems for Classes of Multimessage Multicast Networks: A Rényi Divergence Approach Silas Fong (Joint work with Vincent Tan) Department of Electrical & Computer Engineering National University of Singapore {silas fong, vtan}@nus.edu.sg January 21, 2015 Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

2 Overview 1 Background Multimessage Multicast Networks Weak Converse vs Strong Converse 2 Problem Formulation Network Model Cut-Set Outer Bound and Noisy Network Coding Inner Bound 3 Main Result Main Theorem Strong Converse for Classes of MMNs 4 Proof Sketch Rényi Divergence Proof Sketch 5 Conclusion Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

3 Multimessage Multicast Networks (MMNs) Multiple sources transmit messages to multiple destinations. Each source transmits 1 message. Each destination decodes all the messages. Examples include the butterfly network and the relay channel. s 1 d 1 r 1 r 2 s 2 d 2 r s d Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

4 Some MMNs with Known Capacity Regions Y 2 2 X 2 1 X 1 Y 3 3 Finite-field linear deterministic network [Avestimehr et al., 2011] Y 3 = X 1 + X 2 GF(p n ) for some finite field GF(p n ) MMN consisting of independent DMCs [Köetter et al., 2011] Y 3 = (X 1 + Z 1, X 2 + Z 2) where Z 1 and Z 2 are independent noises. The linear network coding model is a special case when Z 1 = Z 2 = 0 [Li et al., 2003]. Wireless erasure network [Dana et al., 2006] { Y 3 = ( ˆX 1, ˆX 2) where ˆX erasure with prob. ε i i = X i with prob. 1 ε i. The direct part uses random coding. The converse part uses Fano s inequality, which yields a weak converse. Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

5 Weak Converse Assumption: The probability of decoding error vanishes as the blocklength increases. If the rate tuple of a code falls outside the capacity region, the probability of decoding error must be bounded away from 0 as the blocklength increases. For the DMC example, consider any sequence of the optimal length-n schemes with rate R which minimize the error probability: Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

6 Strong Converse Weaker assumption: The asymptotic probability of decoding error is upper bounded by some ε [0, 1) as the blocklength increases. If the rate tuple of a code falls outside the capacity region, the probability of decoding error must tend to 1 as the blocklength increases. Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

7 Motivation of This Work Although the capacity regions of the aforementioned DM-MMNs are well-known, only the weak converse has been proved using Fano s inequality. Therefore we are motivated to prove the strong converse using Rényi divergence: Powerful technique for establishing strong converse theorems. Has been employed previously to establish strong converse for DMC with output feedback [Polyanskiy and Verdú, 2010], classical-quantum channels [Ogawa and Nagaoka, 1999], and entanglement-breaking quantum channels [Wilde et al., 2013]. Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

8 Network Model Let I {1, 2,..., N} be the index set of the nodes. Let S I and D I denote the set of source and destination nodes respectively. The sources in S transmit information to the destinations in D in n time slots: Each source i S chooses W i to transmit. Message W i is uniform on {1, 2,..., 2 nr i } where R i denotes the rate of W i. Each destination j D wants to decode W S. Each node i transmits X i,k and receives Y i,k in time slot k. X i,k is a function of (W i, Y k 1 i ). The channel is characterized by q YI X I : For each k {1, 2,..., n}, Pr{Y I,k = y I,k W I = w I, X k I = x k I, Y k 1 I = y k 1 I } = q YI X I (y I,k x I,k ) Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

9 ε-capacity Region Define R I (R 1, R 2,..., R N ) to be a rate tuple. Assume wlog R i = 0 i / S. A length-n code operating at rate R I is called an (n, R I, ε n )-code if the average probability of decoding error is ε n. R I is ε-achievable if a sequence of (n, R I, ε n )-codes such that lim sup ε n ε. n ε-capacity region C ε {R I : R I is ε-achievable} Fano s inequality yields an outer bound for only C 0 Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

10 Cut-Set Outer Bound A well-known outer bound on the capacity region of DM-MMN developed by El Gamal in C 0 {R R i I pxi q I YI X I (X T ; Y T c X T c ) } i T p XI T I: T c D Applying it to the relay channel, we have R max min{i (X 1 ; Y 2, Y 3 X 2 ), I (X p X1,X 2 }{{} 1, X 2 ; Y 3 ) }{{} cut T 1 cut T 2 } Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

11 Simplified Noisy Network Coding (NNC) Inner Bound Simplified noisy network coding (NNC) inner bound [Lim et al., 2011]: C 0 {R R i I (X T ; Y T c X T c ) H(Y T X I, Y T c ) } I p XI :p XI = T I: N i=1 p X T c D i i T Similar to the cut-set bound: C 0 {R R i I pxi q I YI X I (X T ; Y T c X T c ) i T p XI T I: T c D } For the finite-field linear deterministic network, MMN consisting of independent channels and the wireless erasure network, NNC inner bound = cut-set bound Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

12 Main Theorem Theorem (Strong Converse Outer Bound) For each ε [0, 1), C ε {R R i I pxi q I YI X I (X T ; Y T c X T c ) } T I: p XI T c D i T Compare with cut-set outer bound: C 0 {R R i I pxi q I YI X I (X T ; Y T c X T c ) i T p XI T I: T c D = Reason for the gap: Both proofs first fix an arbitrary T and then find a distribution p (T ) X I that R i I (T ) p (X q T ; Y T c X T c ) for the ε-achievable R I. X YI X I I i T } such In the proof of cut-set bound, p (T ) X I are the same for all T. In the proof of strong converse bound, p (T ) X I can be different for different T. Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

13 Strong Converse for Classes of MMNs Proposition For the finite-field linear deterministic network, MMN consisting of independent channels and the wireless erasure network, strong converse bound = cut-set bound known = NNC inner bound. Corollary (Strong converse) Consider a network belonging to one of the above three classes. For each ε [0, 1), our main theorem implies that C ε strong converse bound. Combining with the proposition above, we have C ε NNC inner bound. Since NNC inner bound C ε, we have C ε = NNC inner bound. Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

14 Rényi Divergence Definition The conditional Rényi divergence with parameter λ [1, ) between p X Z and q X Z given r Z is 1 D λ (p X Z q X Z r Z ) log r λ 1 Z (z) (p X Z (x z)) λ if λ > 1, (q z Z x X X Z (x z)) λ 1 D(p X Z q X Z r Z ) if λ = 1 where D(p X Z q X Z r Z ) z Z r Z (z) x X p X Z (x z) log p X Z (x z) q X Z (x z) is the relative entropy. Data processing inequality (DPI) D λ (p X q X ) D λ (p g(x ) q g(x ) ) for any function g. In particular, D λ (p X,Y q X,Y ) D λ (p X q X ). Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

15 Properties of Rényi Divergence Lemma 1 ( Mutual information approximation) D λ (p XY Z p X Z p Y Z r Z ) D(p XY Z p X Z p Y Z r Z ) + 8(λ 1) X 5 Y 5 = I rz p XY Z (X ; Y Z) + 8(λ 1) X 5 Y 5 Lemma 2 (Chain rule) Given n where r (λ) X k p Yk X k, n q Yk X k D λ ( n and r X n I, we have ) n p Yk X k q Yk X k r X n I = which is determined by λ, k 1 m=1 n p Ym Xm, k 1 D λ (p Yk X k q Yk X k r (λ) X k ) m=1 q Ym X m and r X k I Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

16 Recalling Proof Steps for Cut-Set Bound 1 Lower bounding the error probability in terms of mutual information using Fano s inequality n i T R i I (W T ; ŴT,d W T c ) ε n n i T R 2 Using the DPI of the mutual information to introduce the channel output I (W T ; ŴT,d W T c ) I (W T ; Y n T c W T c ) 3 Single-letterizing the mutual information I (W T ; Y n T c W T c ) n I (X T,k; Y T c,k X T c,k) 4 Introduction of a time-sharing random variable Q n n I (X T,k; Y T c,k X T c,k) ni (X T,Qn ; Y T c,q n X T c,q n ) 5 Combining the above inequalities, using lim n ε n = 0 i T R i I (X T ; Y T c X T c ) Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

17 Proof Steps Using Rényi Divergence 1) Lower bounding the error probability in terms of the Rényi divergence ( ) nr i D λ (p WT,Ŵ T,d p WT sŵt,d ) + λ(λ 1) 1 1 log 1 ε n i T for any choice of sŵt,d. 2) Using the DPI of the Rényi divergence to introduce the channel input and output with a proper choice of s X n I,Y I n,ŵt,d ( n ) n D λ (p WT,Ŵ T,d p WT sŵt,d ) D λ q YT c,k X I,k s YT c,k X I,k p X n I 3) Single-letterizing the Rényi divergence using the chain rule D λ ( n n q YT c,k X I,k s YT c,k X I,k p X n I )= n D λ (q YT c X I s YT c,k X T c,k p (λ) X I,k ). Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

18 Proof Steps Using Rényi Divergence 4) Representing distributions in the Rényi divergence by a single distribution Due to the careful choice of s X n I,Y I n, we can define,ŵt p(λ) {d} X I,k,Y T c,k s.t. n D λ (q YT c X I s YT c,k X T c,k p (λ) X I,k ) n D λ ( p (λ) Y T c,k X I,k p (λ) Y T c,k X T c,k p (λ) X I,k ). 5) Introduction of a time-sharing variable followed by approximating I with D λ. Using a time-sharing variable Q n and letting λ = n, n D 1+ 1 ( p (1+ 1 ) n n Y T c,k X I,k p (1+ 1 ) n Y T c,k X p (1+ 1 T c,k ) n X I,k ) nd 1+ 1 ( p (1+ 1 ) n n Y T c,qn X I,Qn p (1+ 1 ) n Y T c,qn X p (1+ 1 T c,qn ni (X T ; Y T c X T c ) + 8 X 5 Y 5 n Combining the steps and letting n, R i I (X T ; Y T c X T c ). i T ) n X I,Qn ) Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

19 Conclusion In a multimessage multicast network (MMN), every source node transmits a message and every destination node decodes all the messages. A strong converse outer bound for the discrete memoryless MMN have been established using Rényi divergence, i.e., outer bound on C ε. For any sequence of codes that operate at a rate tuple outside the strong converse bound, the average probability of decoding error must tend to 1. Our strong converse bound implies the strong converse some classes of MMNs including The finite-field linear deterministic network. The MMN consisting of independent DMCs. The wireless erasure network. For the aforementioned MMNs, we have fully characterized their ε-capacity regions, which coincide with the NNC inner bound and the cut-set bound. Open problem: Prove or disprove that the cut-set bound contains C ε. Silas Fong (NUS) Strong Converse for MMNs January 21, / 20

20 References Avestimehr, A. S., Diggavi, S. N., and Tse, D. N. (2011). Wireless network information flow: A deterministic approach. IEEE Trans. Inf. Theory, 57(4): Dana, A. F., Gowaikar, R., Palanki, R., Hassibi, B., and Effros, M. (2006). Capacity of wireless erasure networks. IEEE Trans. Inf. Theory, 52(3): Köetter, R., Effros, M., and Médard, M. (2011). A theory of network equivalence Part I: Point-to-point channels. IEEE Trans. Inf. Theory, 57(2): Li, S.-Y. R., Yeung, R. W., and Cai, N. (2003). Linear network coding. IEEE Trans. Inf. Theory, 49(2): Lim, S. H., Kim, Y.-H., El Gamal, A., and Chung, S.-Y. (2011). Noisy network coding. IEEE Trans. on Inform. Th., 57(5): Ogawa, T. and Nagaoka, H. (1999). Strong converse to the quantum channel coding theorem. IEEE Trans. on Inform. Th., 45(7): Polyanskiy, Y. and Verdú, S. (2010). Arimoto channel coding converse and Rényi divergence. In Proc. Allerton Conference on Communication, Control and Computing, pages Wilde, M. M., Winter, A., and Yang, D. (2013). Strong converse for the classical capacity of entanglement-breaking and hadamard channels via a sandwiched Rényi relative entropy. ArXiv Preprint. Silas Fong (NUS) Strong Converse for MMNs January 21, / 20