Cover s Open Problem: The Capacity of the Relay Channel Ayfer Özgür Stanford University Advanced Networks Colloquia Series University of Maryland, March 2017 Joint work with Xiugang Wu and Leighton Pate Barnes. Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 1 / 26
Father of the Information Age Claude Shannon (1916-2001) Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 2 / 26
The Bell System Technical Journal Vol. XXVII J Illy, 1948 No.3 A Mathematical Theory of Communication By c. E. SHANNON IXTRODUCTION HE recent development of various methods of modulation such as rem T and PPM which exchange bandwidth for signal-to-noise ratio has intensified the interest in a general theory of communication. A basis for such a theory is contained in the important papers of Nyquist! and Hartley" on this subject. In the present paper we will extend the theory to include a number of new factors, in particular the effect of noise in the channel, and Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 3 / 26
A method is developed for representing any communication system geometrically... Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 4 / 26
AWGN Channel P Transmitter Receiver Capacity ( C = log 1 + P ) N Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 5 / 26
Converse: Sphere Packing p nn X n (1) X n (2) p nn 0 p n(p + N) Y sphere X n (3) p nn noise sphere ( n(p ) Sphere + N) # of X n. 2 n log 2πe(P+N) 2 ( nn ) = Sphere 2 n 2 log 2πeN = 2 n 2 log(1+ P N ) Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 6 / 26
Achievability: Geometric Random Coding Pr( false X n Lens ) ( np ) 2 nr Sphere. = 2 n 2 2 n 2 PN log 2πe P+N 2nR log 2πeP = 2 n 2 log(1+ P N ) 2 nr Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 7 / 26
The story goes... Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 8 / 26
Cover s Open Problem Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 9 / 26
Gaussian case W 1 N(0,N) W 2 N(0,N) Z n Relay C 0 I n = f(z n ) Source X n Achievability: C 0 =. C( ) = 1 2 log ( 1 + 2P N Destination Y n Cutset-bound (Cover and El Gamal 79): C0 1 ( 2 log 1 + 2P ) 12 ( N log 1 + P ). N ) Potentially, C 0 0 as P/N 0. Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 10 / 26
Main Result W 1 N(0,N) W 2 N(0,N) Z n Relay C 0 I n = f(z n ) Source X n Destination Y n Theorem C 0 = Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 11 / 26
Upper Bound on the Capacity 3.1 3 Cut-set bound C-F Old bound New bound SNR = 15 db 2.9 2.8 2.7 2.6 2.5 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 C 0 (bit/channel use) Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 12 / 26
Cutset Bound W 1 N(0,N) W 2 N(0,N) Z n C 0 Relay I n = f(z n ) Source X n Destination Y n nr I (X n ; Y n, I n ) + nɛ n = I (X n ; Y n ) + I (X n ; I n Y n ) + nɛ n = I (X n ; Y n ) + H(I n Y n ) H(I }{{} n Y n, X n ) +nɛ }{{} n nc 0 0 n(i (X ; Y ) + C 0 + ɛ n ) Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 13 / 26
Cutset Bound W 1 N(0,N) W 2 N(0,N) Z n C 0 Relay I n = f(z n ) Source X n Destination Y n nr I (X n ; Y n, I n ) + nɛ n = I (X n ; Y n ) + I (X n ; I n Y n ) + nɛ n = I (X n ; Y n ) + H(I n Y n ) H(I }{{} n X n ) +nɛ }{{} n nc 0 0 n(i (X ; Y ) + C 0 + ɛ n ) Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 13 / 26
Z n X n p nn Y n Typical set of Z n /Y n I n -th bin Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 14 / 26
Z n X n p nn Y n Typical set of Z n /Y n I n -th bin If H(I n X n ) = 0, Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 14 / 26
Z n X n p nn Y n Typical set of Z n /Y n I n -th bin If H(I n X n ) = 0, then H(I n Y n ) = 0. Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 14 / 26
W 1 N(0,N) W 2 N(0,N) Z n Relay C 0 I n = f(z n ) Source X n Destination Y n R I (X n ; Y n ) + H(I n Y n ) H(I }{{} n X n ) +nɛ }{{} n? = n log sin θ n Goal: I n = f (Z n ) Z n X n n Y }{{} H(I n X n )= n log sin θ n } {{ } H(I n Y n )? Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 15 / 26
H(I n X n ) 0 Typical set of Z n p nn X n Multiple bins # of bins =? P(each bin) =? Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 16 / 26
From n- to nb- Dimensional Space X, Y, Z, I : B-length i.i.d. from {(X n (b), Y n (b), Z n (b), I n (b))} B b=1. If H(I n X n ) = n log sin θ n, then for any typical (x, i) p(i x). = 2 nb log sin θn, P(Z A(i) x) =. nb log sin θn 2 p nbnx ith bin A x (i) Pr. =2 nb log sin n A x (i). =2 nb( 1 2 log 2 en sin2 n) Typical set of Z/Y Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 17 / 26
Isoperimetric Inequalities Isoperimetric Inequality in the Plane (Steiner 1838) Among all closed curves in the plane with a given enclosed area, the circle has the smallest perimeter. Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 18 / 26
Isoperimetric Inequalities Isoperimetric Inequality on the Sphere (Levy 1919) Among all sets on the sphere with a given volume, the spherical cap has the smallest boundary, or the smallest volume of ω-neighborhood for any ω > 0. Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 19 / 26
Blowing-up Lemma A x (i). =2 nb( 1 2 log 2 en sin2 n) Isoperimetric Inequality on the Sphere + Measure Concentration: P(Z blow-up of A x (i) x) 1. P(Y blow-up of A x (i) x) 1. Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 20 / 26
Geometry of Typical Sets n-dimensional space: Almost all (X n, Y n, Z n, I n ) Z n! I n X n p nn Y n nb-dimensional space: Almost all (x, y, i) p nbn z! i x 2 n y Information Inequality: (Wu and Ozgur, 2015) H(I n Y n ) n(2 log sin θ n + 2 log sin θ n ln 2 log e). Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 21 / 26
A new approach A x (i) A x (i) Y Y Control the intersection of a sphere drawn around a randomly chosen Y and A x (i). Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 22 / 26
Easy if A x (i) is a spherical cap Cap(z 0, n) z 0 Cap(z 0, n) \ Cap(y 0,! n) p nbn x n! n y 0 Cap(y 0,! n) Cap(z 0, θ n ) Cap(Y, ω n ). = 2 nb( 1 2 log 2πeN(sin2 θ n cos 2 ω n)) Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 23 / 26
Strengthening of the Isoperimetric Inequality Strengthening of the Isoperimetric Inequality: Among all sets on the sphere with a given volume, the spherical cap has minimal intersection volume at distance ω for almost all points on the sphere for any ω > π/2 θ. Proof: builds on the Riesz rearrangement inequality. Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 24 / 26
A Packing Argument Given Y, # of I ( ) Sphere Y, nbn4sin 2 ωn 2 2 nb( 1 2 log 2πeN(sin2 θ n cos 2 ω n)) Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 25 / 26
Summary Solved an problem posed by Cover and named The Capacity of Relay Channel in Open Problems in Communication and Computation, Springer-Verlag, 1987. Developed a converse technique that significantly deviates from standard converse techniques based on single-letterization and has some new ingredients: Typicality Measure Concentration Isoperimetric Inequality Ayfer Özgür (Stanford) The Capacity of the Relay Channel March 17 26 / 26