Improving on the Cutset Bound via a Geometric Analysis of Typical Sets

Transcription

1 Imroving on the Cutset Bound via a Geometric Analysis of Tyical Sets Ayfer Özgür Stanford University CUHK, May 28, 2016 Joint work with Xiugang Wu (Stanford). Ayfer Özgür (Stanford) March 16 1 / 33

2 Gaussian Relay Channel Z = X + W 1 Y = X + X r + W 2 Caacity is the largest achievable end-to-end reliable communication rate between the source and the destination. Ayfer Özgür (Stanford) March 16 2 / 33

3 Early work One of the central roblems in information theory: Introduced by van der Meulen in Seminal work by Cover and El Gamal in Two achievable Schemes: Decode-Forward, Comress-Forward. A general uer bound: Cutset Bound. Ayfer Özgür (Stanford) March 16 3 / 33

4 Over the Following 35 Years Major research effort over the last two decades however the roblem remains oen. Ayfer Özgür (Stanford) March 16 4 / 33

5 Over the Following 35 Years Major research effort over the last two decades however the roblem remains oen. Decode-Forward and Comress-Forward have been extended to multi-relay networks (Xie-Kumar 05,Kramer-Gastar-Guta 05) and unified (Wu-Xie 14). Ayfer Özgür (Stanford) March 16 4 / 33

6 Over the Following 35 Years Major research effort over the last two decades however the roblem remains oen. Decode-Forward and Comress-Forward have been extended to multi-relay networks (Xie-Kumar 05,Kramer-Gastar-Guta 05) and unified (Wu-Xie 14). Many new relaying strategies have been discovered: e.g., Amlify-Forward (Schein-Gallager 00), Hash-Forward (Kim 08), Comute-Forward (Nazer-Gastar 11), Quantize-Ma-Forward (Avestimehr-Diggavi-Tse 11), Noisy Network Coding (Lim-Kim-El Gamal-Chung 11). Ayfer Özgür (Stanford) March 16 4 / 33

7 Over the Following 35 Years Major research effort over the last two decades however the roblem remains oen. Decode-Forward and Comress-Forward have been extended to multi-relay networks (Xie-Kumar 05,Kramer-Gastar-Guta 05) and unified (Wu-Xie 14). Many new relaying strategies have been discovered: e.g., Amlify-Forward (Schein-Gallager 00), Hash-Forward (Kim 08), Comute-Forward (Nazer-Gastar 11), Quantize-Ma-Forward (Avestimehr-Diggavi-Tse 11), Noisy Network Coding (Lim-Kim-El Gamal-Chung 11). Cutset bound remains as the only uer bound on caacity of the Gaussian Relay Channel. Consistently used as a benchmark for erformance. It is not known if this bound is indeed achievable or not. Ayfer Özgür (Stanford) March 16 4 / 33

8 Gaussian Primitive Relay Channel W 1 N(0,N) W 2 N(0,N) Z Relay R 0 Source X Y Destination Cutset Bound: (Cover-El Gamal 79) If R achievable, some X with E[X 2 ] P such that: R I (X ; Y, Z) R I (X ; Y ) + R 0 Broadcast Bound Multile Access Bound Ayfer Özgür (Stanford) March 16 5 / 33

9 Gaussian Primitive Relay Channel W 1 N(0,N) W 2 N(0,N) Z Relay R 0 Source X Y Destination Cutset Bound: (Cover-El Gamal 79) If R achievable, some X with E[X 2 ] P such that: R I (X ; Y, Z) R I (X ; Y ) + R 0 Broadcast Bound Multile Access Bound Ayfer Özgür (Stanford) March 16 5 / 33

10 Gaussian Primitive Relay Channel W 1 N(0,N) W 2 N(0,N) Z Relay R 0 Source X Y Destination Cutset Bound: (Cover-El Gamal 79) If R achievable, then R 1 ( 2 log 1 + 2P ) N R 1 ( 2 log 1 + P ) + R 0 N Ayfer Özgür (Stanford) March 16 6 / 33

11 Cutset Bound is not Tight W 1 N(0,N) W 2 N(0,N) Z Relay R 0 Source X Y Destination Theorem: (Wu-Ozgur 15) If R achievable, then some X with E[X 2 ] P and a 0 such that R I (X ; Y, Z) R I (X ; Y ) + R 0 a R I (X ; Y ) + a + 2a ln 2 log e Broadcast Bound Modified Multile Access Bound New constraint involving a Ayfer Özgür (Stanford) March 16 7 / 33

12 Cutset Bound is not Tight W 1 N(0,N) W 2 N(0,N) Z Relay R 0 Source X Y Destination Theorem: (Wu-Ozgur 15) If R achievable, then some a 0 such that R 1 ( 2 log 1 + 2P ) N R 1 ( 2 log 1 + P ) + R 0 a N R 1 ( 2 log 1 + P ) + a + 2a ln 2 log e N Ayfer Özgür (Stanford) March 16 8 / 33

13 Cutset Bound is not Tight W 1 N(0,N) W 2 N(0,N) Z Relay R 0 Source X Theorem: (Wu-Ozgur 15) If R achievable, then R 1 ( 2 log 1 + 2P ) N R 1 ( 2 log 1 + P ) + R 0 a N Y Destination where a is such that R 0 = 2a + 2a ln 2 log e New bound is strictly tighter than the cutset bound. Ayfer Özgür (Stanford) March 16 9 / 33

14 Ga to the Cutset Bound Cut-set bound New bound ale 0.5 X-Y single-user caacity Largest ga among all ( P N, R 0) : Ayfer Özgür (Stanford) March / 33

15 Caacity Aroximation for Gaussian Relay Networks N relays Source Destination Theorem: (Avestimehr-Diggavi-Tse 11) The caacity of any Gaussian relay network can be aroximated with the cutset bound within a ga that is indeendent of the channel configurations and deends on the network toology only through the number of nodes. Aroximation ga: Ga 7.5N 1.5N 0.6N 0.5N (Avestimehr-Diggavi-Tse 11) (Ozgur-Diggavi 13) (Lim-Kim-El Gamal-Chung 13) (Lim-Kim-Kim 15) Ayfer Özgür (Stanford) Can we do better? March / 33

16 The Gaussian N-Relay Diamond Network 1 h 1s h 1d 2 S h 2s. h 2d D h Ns N h Nd Chern and Ozgur 12 We can aroximate the caacity of any Gaussian N-Relay Diamond network with its cutset bound within a Ga log N indeendent of the channel coefficients and the SNR s. Ayfer Özgür (Stanford) March / 33

17 Sublinear ga to the cut-set bound? N relays Source Destination Theorem:(Courtade-Ozgur 15) Sublinear ga is achievable iff cutset bound is tight for all Gaussian relay networks. Linear ga to the cutset bound is order-otimal: 0.5N Ga 0.01N. Ayfer Özgür (Stanford) March / 33

18 A Tensorization Argument Ga Source Destination M Ayfer Özgür (Stanford) March / 33

19 A Tensorization Argument 1 Ga M 0.01N Source Destination M Ayfer Özgür (Stanford) March / 33

20 Derivation of the Cutset Bound W 1 N(0,N) W 2 N(0,N) Z n Relay R 0 I n = f(z n ) Source X n Destination Y n Ste 1) Aly Fano s inequality: nr I (X n ; Y n, I n ) + nɛ n Ste 2) Bound with single-letter exressions. Ayfer Özgür (Stanford) March / 33

21 Derivation of the Cutset Bound BC Constraint: nr I (X n ; Y n, I n ) + nɛ n I (X n ; Y n, Z n ) + nɛ n n(i (X ; Y, Z) + ɛ n ) Source X n W 1 N(0,N) W 2 N(0,N) Z n Relay R I n = f(z n 0 ) Destination Y n MAC Constraint: nr I (X n ; Y n, I n ) + nɛ n I (X n ; Y n ) + I (X n ; I n Y n ) + nɛ n I (X n ; Y n ) + H(I n Y n ) H(I }{{} n Y n, X n ) +nɛ }{{} n nr 0 0 n(i (X ; Y ) + R 0 + ɛ n ) Ayfer Özgür (Stanford) March / 33

22 Derivation of the New Bound BC Constraint: nr I (X n ; Y n, I n ) + nɛ n Z n Relay I (X n ; Y n, Z n ) + nɛ n n(i (X ; Y, Z) + ɛ n ) Source X n W 1 N(0,N) W 2 N(0,N) R 0 I n = f(z n ) Destination Y n MAC Constraint: nr I (X n ; Y n, I n ) + nɛ n I (X n ; Y n ) + I (X n ; I n Y n ) + nɛ n I (X n ; Y n ) + H(I n Y n ) H(I }{{} n Y n, X n ) +nɛ }{{} n nr 0 n(i (X ; Y ) + R 0 a + ɛ n ) Need a lower bound on a. =H(I n X n )=na Ayfer Özgür (Stanford) March / 33

23 a cannot be arbitrarily small Y n X n nn Z n W 1 N(0,N) W 2 N(0,N) Z n Relay R I n = f(z n 0 ) Source X n Destination Y n Tyical set of Z n /Y n I n -th bin If H(I n X n ) = na = 0, Ayfer Özgür (Stanford) March / 33

24 a cannot be arbitrarily small Y n X n nn Z n W 1 N(0,N) W 2 N(0,N) Z n Relay R I n = f(z n 0 ) Source X n Destination Y n Tyical set of Z n /Y n I n -th bin If H(I n X n ) = na = 0, then H(I n Y n ) = 0 R I (X n ; Y n ) + H(I n Y n ) H(I }{{} n X n ) +nɛ }{{} n =0 =0 n(i (X ; Y ) + ɛ n ) Ayfer Özgür (Stanford) March / 33

25 In general W 1 N(0,N) W 2 N(0,N) Z n Relay R I n = f(z n 0 ) Source X n Destination Y n I n = f (Z n ) Z n X n Y n, and Y n and Z n are i.i.d. given X n. R I (X n ; Y n ) + H(I n Y n ) H(I }{{} n X n ) +nɛ }{{} n? =na I (X n ; Y n ) + I (I n ; X n ) I (I }{{} n ; Y n ) +nɛ }{{} n =n(r 0 a)? Ayfer Özgür (Stanford) March / 33

26 In general W 1 N(0,N) W 2 N(0,N) Z n Relay R I n = f(z n 0 ) Source X n Destination Y n I n = f (Z n ) Z n X n Y n, and Y n and Z n are i.i.d. given X n. R I (X n ; Y n ) + H(I n Y n ) H(I }{{} n X n ) +nɛ }{{} n? =na I (X n ; Y n ) + I (I n ; X n ) I (I }{{} n ; Y n ) +nɛ }{{} n =n(r 0 a)? Strong Data Processing: Let U X Y. Fix I (U; X ) and bound I (U; Y ). Ayfer Özgür (Stanford) March / 33

27 H(I n X n ) = na and a > 0 Tyical set of Z n nn X n Multile bins # of bins =? P(each bin) =? Ayfer Özgür (Stanford) March / 33

28 From n- to nb- Dimensional Sace B-length i.i.d. extension {(X n (b), Y n (b), Z n (b), I n (b))} B b=1. Signals in nb-dimensional sace: X, Y, Z, I. Law of Large Numbers: n dimensional sace: H(In X n ) = na. nb dimensional sace: For any tyical (x, i), P(Z i th bin x) =. 2 nba. Tyical set of Z/Y nbnx ith Pr =2. bin nba Ayfer Özgür (Stanford) March / 33

29 Gaussian Measure Concentration Theorem (Functional) If X N (0, I k ), and Z = f (X ) such that f : R k R is L-Lischitz, i.e. f (x) f (y) L x y 2, x, y then P(Z E[Z] > t) e t2 /(2L 2), t > 0. Ayfer Özgür (Stanford) March / 33

30 Gaussian Measure Concentration Theorem (Functional) If X N (0, I k ), and Z = f (X ) such that f : R k R is L-Lischitz, i.e. f (x) f (y) L x y 2, x, y then Theorem (Geometric) P(Z E[Z] > t) e t2 /(2L 2), t > 0. If X N (0, I k ), and let B t = {x R k : y B s.t. y x 2 t} for B R k, then ( ) 2 P(X / B t ) e 1 t 2 log P(B) 2. Ayfer Özgür (Stanford) March / 33

31 Gaussian Measure Concentration Theorem (Functional) If X N (0, I k ), and Z = f (X ) such that f : R k R is L-Lischitz, i.e. f (x) f (y) L x y 2, x, y then Theorem (Geometric) P(Z E[Z] > t) e t2 /(2L 2), t > 0. If X N (0, I k ), and let B t = {x R k : y B s.t. y x 2 t} for B R k, then ( ) 2 P(X / B t ) e 1 t 2 log P(B) 2. If P(B) = e kb, blow u by a radius t = k( 2b + ɛ), then P(X / B t ) e kɛ2. Ayfer Özgür (Stanford) March / 33

32 Gaussian Measure Concentration Tyical set of Z/Y nbnx ith Pr =2. bin nba Ayfer Özgür (Stanford) March / 33

33 Gaussian Measure Concentration nb 2Naln 2 nbnx nbnx Tyical set of Z/Y ith Pr =2. bin nba Tyical set of Z/Y blown-u set of i-th bin Pr 1 Ayfer Özgür (Stanford) March / 33

34 Gaussian Measure Concentration nb 2Naln 2 nbnx nbnx Tyical set of Z/Y ith Pr =2. bin nba Tyical set of Z/Y blown-u set of i-th bin Pr 1 P(Z blown-u set of i th bin x) 1. P(Y blown-u set of i th bin x) 1. Ayfer Özgür (Stanford) March / 33

35 Geometry of Tyical Sets n-dimensional sace: Tyical (X n, Y n, Z n, I n ) nb-dimensional sace: Tyical (x, y, i) Y n y nbn nb 2Naln 2 z! i X n nn Z n! I n x Ayfer Özgür (Stanford) March / 33

36 Geometry of Tyical Sets n-dimensional sace: Tyical (X n, Y n, Z n, I n ) nb-dimensional sace: Tyical (x, y, i) Y n y nbn nb 2Naln 2 z! i X n nn Z n! I n x Ayfer Özgür (Stanford) March / 33

37 Translating Geometry to Information Inequalities Assume y, z were discrete: B(r): Ball of radius r around y. B(r) : number of z sequences. H(I Y) log B( 2nBNa log 2). y nbn x nb 2Naln 2 z! i If I (I n ; X n ) = n(r 0 a), I (I n ; Y n ) nr 0 1 B log B( 2nBNa log 2). Ayfer Özgür (Stanford) March / 33

38 Bounding f (y i) for a tyical (y, i) x 2 A(X z) y nb 2Naln 2 f(y x) z 2 ith bin nbn d(x, y) ale d(x, z)+d(z, y) ale nb( N + 2Naln 2) f (y i) (x i)f (y x) x A(X n z) A(X n z). = 2 BH(X n Z n ) (x i). = 2 BH(X n I n) f (y x) 1 nb( N+ 2Na ln 2) 2 e 2N (2πN) nb/2 Ayfer Özgür (Stanford) March / 33

39 Bounding f (y i) For a tyical (y, i): 1. f (y i) 2 B(H(X n I n) H(X n Z n )+h(y n X n )+n(a+ 2a ln 2 log e)) 2. f (y i). = 2 Bh(Y n I n) A new entroy inequality: h(y n I n ) H(X n I n ) H(X n Z n )+h(y n X n )+n(a+ 2a ln 2 log e). Ayfer Özgür (Stanford) March / 33

40 Bounding f (y i) For a tyical (y, i): 1. f (y i) 2 B(H(X n I n) H(X n Z n )+h(y n X n )+n(a+ 2a ln 2 log e)) 2. f (y i). = 2 Bh(Y n I n) A new entroy inequality: h(y n I n ) H(X n I n ) H(X n Z n )+h(y n X n )+n(a+ 2a ln 2 log e). Equivalently: I (I n ; X n ) I (I n ; Y n ) n(a + 2a ln 2 log e). Ayfer Özgür (Stanford) March / 33

41 Bounding f (y i) For a tyical (y, i): 1. f (y i) 2 B(H(X n I n) H(X n Z n )+h(y n X n )+n(a+ 2a ln 2 log e)) 2. f (y i). = 2 Bh(Y n I n) A new entroy inequality: h(y n I n ) H(X n I n ) H(X n Z n )+h(y n X n )+n(a+ 2a ln 2 log e). Equivalently: I (I n ; X n ) I (I n ; Y n ) n(a + 2a ln 2 log e). A new constraint: R I (X ; Y ) + I (I n ; X n ) I (I n ; Y n ) I (X ; Y ) + a + 2a ln 2 log e. Ayfer Özgür (Stanford) March / 33

42 Discrete Memoryless Primitive Relay Channel Y and Z are conditionally I.I.D. given X. Joint work with Liang-Liang Xie. Ayfer Özgür (Stanford) March / 33

43 An Oen Problem Cover, Oen roblems in Communication and Comutation, 1987 What is the minimum R 0 (denoted by R0 ) needed to achieve C XYZ = max (x) I (X ; Y, Z)? For examle, when the channel from X to Y and Z is BSC(), R 0 1. Ayfer Özgür (Stanford) March / 33

44 Bounds on R 0 (BSC) Lower bound on R Hash-Forward Cut-Set Bound A striking dichotomy when 0.5 C XYZ 0: Achievable schemes require R 0 1. Uer bounds allow for R 0 0. Ayfer Özgür (Stanford) March / 33

45 Bounds on R 0 (BSC) Lower bound on R Hash-Forward Cut-Set Bound New Bound Strictly ositive R 0 needed to achieve C XYZ 0! Ayfer Özgür (Stanford) March / 33

46 Conclusion We develoed new uer bounds on the caacity of the relay channel that are tighter than the cutset bound. Our roof used ideas from tyicality and measure concentration. It would be interesting to see if the aroach we develo in this aer, i.e. deriving information inequalities by studying the geometry of tyical sets, in articular using measure concentration, can be used to make rogress on other long-standing oen roblems in network information theory. Ayfer Özgür (Stanford) March / 33