Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities

Transcription

1 Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities Vincent Y. F. Tan Dept. of ECE and Dept. of Mathematics National University of Singapore (NUS) September 2014 Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

2 Acknowledgements This is joint work with 1 Sy-Quoc Le and Mehul Motani (NUS) 2 Jon Scarlett (Cambridge, now at EPFL) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

3 Outline 1 Motivation, Background and History Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

4 Outline 1 Motivation, Background and History 2 Gaussian Interference Channel with Very Strong Interference Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

5 Outline 1 Motivation, Background and History 2 Gaussian Interference Channel with Very Strong Interference 3 Gaussian MAC with Degraded Message Sets Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

6 Outline 1 Motivation, Background and History 2 Gaussian Interference Channel with Very Strong Interference 3 Gaussian MAC with Degraded Message Sets 4 Conclusion Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

7 Outline 1 Motivation, Background and History 2 Gaussian Interference Channel with Very Strong Interference 3 Gaussian MAC with Degraded Message Sets 4 Conclusion Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

8 Transmission of Information Shannon s Figure 1 INFORMATION SOURCE TRANSMITTER RECEIVER DESTINATION SIGNAL RECEIVED SIGNAL MESSAGE MESSAGE NOISE SOURCE Shannon abstracted away information meaning, semantics treat all data equally Shannon s bits as a universal Figure currency 1 crucial abstraction for modern communication and computing systems Also relaxed computation and delay constraints to discover a fundamental limit: capacity, providing a goal-post to work toward Information theory Finding fundamental limits for reliable information transmission Saturday, June 11, 2011 Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

9 Transmission of Information Shannon s Figure 1 INFORMATION SOURCE TRANSMITTER RECEIVER DESTINATION SIGNAL RECEIVED SIGNAL MESSAGE MESSAGE NOISE SOURCE Shannon abstracted away information meaning, semantics treat all data equally Shannon s bits as a universal Figure currency 1 crucial abstraction for modern communication and computing systems Also relaxed computation and delay constraints to discover a fundamental limit: capacity, providing a goal-post to work toward Information theory Finding fundamental limits for reliable information transmission Saturday, June 11, 2011 Channel coding: Concerned with the maximum rate of communication in bits/channel use Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

10 Channel Coding (One-Shot) M X Y M e W d A code is an triple C = {M, e, d} where M is the message set and b(e(m)) S for some cost function b( ) and cost S Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

11 Channel Coding (One-Shot) M X Y M e W d A code is an triple C = {M, e, d} where M is the message set and b(e(m)) S for some cost function b( ) and cost S The average error probability p err (C) is [ ] p err (C) := Pr M M where M is uniform on M Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

12 Channel Coding (One-Shot) M X Y M e W d A code is an triple C = {M, e, d} where M is the message set and b(e(m)) S for some cost function b( ) and cost S The average error probability p err (C) is [ ] p err (C) := Pr M M where M is uniform on M A non-asymptotic fundamental limit can be defined as M (W, ε, S) := sup { m N C s.t. m = M, p err (C) ε } Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

13 Channel Coding (n-shot) for AWGN M X n Y e W n n M d Consider n independent uses of an additive white Gaussian channel W n Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

14 Channel Coding (n-shot) for AWGN M X n Y e W n n M d Consider n independent uses of an additive white Gaussian channel W n For vectors x = (x 1,..., x n ) R n and y := (y 1,..., y n ) R n, with the channel law is W n (y x) = i=1 x 2 2 ns, n 1 exp ( (y i x i ) 2 ) 2π 2 Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

15 Channel Coding (n-shot) for AWGN M X n Y e W n n M d Consider n independent uses of an additive white Gaussian channel W n For vectors x = (x 1,..., x n ) R n and y := (y 1,..., y n ) R n, with the channel law is W n (y x) = i=1 x 2 2 ns, n 1 exp ( (y i x i ) 2 ) 2π 2 Non-asymptotic fundamental limit for n uses of W M (W n, ε, S) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

16 Single-User Asymptotic Evaluation Theorem (Hayashi (2009), Polyanskiy-Poor-Verdú (2010), Tan-Tomamichel (2014)) For every ε (0, 1), we have log M (W n, ε, S) = nc(s) + nv(s)φ 1 (ε) + 1 log n + O(1) 2 where C(S) and V(S) are the capacity and dispersion defined as C(S) = 1 2 log(1 + S), V(S) = S(S + 2) (S + 1) 2 log2 e Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

17 Single-User Asymptotic Evaluation Theorem (Hayashi (2009), Polyanskiy-Poor-Verdú (2010), Tan-Tomamichel (2014)) For every ε (0, 1), we have log M (W n, ε, S) = nc(s) + nv(s)φ 1 (ε) + 1 log n + O(1) 2 where C(S) and V(S) are the capacity and dispersion defined as C(S) = 1 2 log(1 + S), V(S) = S(S + 2) (S + 1) 2 log2 e M. Hayashi Polyanskiy-Poor-Verdú M. Tomamichel Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

18 Single-User Asymptotic Evaluation: Interpretation R (W n, ε, S) := log M (W n, ε, S) n V(S) ( log n ) = C(S) + n Φ 1 (ε) +O }{{} n Gaussian approximation Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

19 Single-User Asymptotic Evaluation: Interpretation R (W n, ε, S) := log M (W n, ε, S) n V(S) ( log n ) = C(S) + n Φ 1 (ε) +O }{{} n Gaussian approximation Interpretation: The backoff from C(S) at finite blocklength n and tolerable error probability is approximately V(S) Φ 1 (1 ε) n Small ε implies large backoff Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

20 Single-User Asymptotic Evaluation: Interpretation R (W n, ε, S) := log M (W n, ε, S) n V(S) ( log n ) = C(S) + n Φ 1 (ε) +O }{{} n Gaussian approximation Interpretation: The backoff from C(S) at finite blocklength n and tolerable error probability is approximately V(S) Φ 1 (1 ε) n Small ε implies large backoff Can compare to actual finite blocklength bounds Gaussian approximation is good for some n and ε Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

21 Background: Shannon s Channel Coding Theorem Shannon s noisy channel coding theorem and Shannon s (1959), Yoshihara s (1964) and Wolfowitz s (1978) strong converse state that Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

22 Background: Shannon s Channel Coding Theorem Shannon s noisy channel coding theorem and Shannon s (1959), Yoshihara s (1964) and Wolfowitz s (1978) strong converse state that Theorem (Shannon (1949), Shannon (1959)) 1 lim n n log M (W n, ε, S) = C(S), ε (0, 1) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

23 Background: Shannon s Channel Coding Theorem 1 lim n n log M (W n, ε, S) = C(S) bits/channel use Channel coding theorem for AWGN channels is independent of ε (0, 1) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

24 Background: Shannon s Channel Coding Theorem 1 lim n n log M (W n, ε, S) = C(S) bits/channel use Channel coding theorem for AWGN channels is independent of ε (0, 1) 1 lim p err(c n ) n 0 C(S) R Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

25 Background: Shannon s Channel Coding Theorem 1 lim n n log M (W n, ε, S) = C(S) bits/channel use Channel coding theorem for AWGN channels is independent of ε (0, 1) 1 lim p err(c n ) n 0 C(S) R Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

26 Background: Shannon s Channel Coding Theorem 1 lim n n log M (W n, ε, S) = C(S) bits/channel use Channel coding theorem for AWGN channels is independent of ε (0, 1) 1 lim p err(c n ) n 0 C(S) R Phase transition at capacity Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

27 Background: Second-Order Coding Rates What happens at capacity? Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

28 Background: Second-Order Coding Rates What happens at capacity? More precisely, what happens when log M n nc(s) + L n for some L R? Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

29 Background: Second-Order Coding Rates What happens at capacity? More precisely, what happens when log M n nc(s) + L n for some L R? Here L is known as the second-order coding rate of the code Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

30 Background: Second-Order Coding Rates What happens at capacity? More precisely, what happens when log M n nc(s) + L n for some L R? Here L is known as the second-order coding rate of the code Note that L can be negative (cf. Hayashi (2008), Hayashi (2009)) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

31 Background: Second-Order Coding Rates Assume rate of the code satisfies 1 n log M n = C(S) + L ( 1 ) + o n n Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

32 Background: Second-Order Coding Rates Assume rate of the code satisfies 1 n log M n = C(S) + L ( 1 ) + o n n 1 lim n perr(cn) L Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

33 Background: Second-Order Coding Rates Assume rate of the code satisfies 1 n log M n = C(S) + L ( 1 ) + o n n 1 lim n perr(cn) 0.5 p err (C n ) = Φ( L V(S) ) + o(1) 0 L Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

34 Background: Second-Order Coding Rates Assume rate of the code satisfies 1 n log M n = C(S) + L ( 1 ) + o n n 1 lim n perr(cn) 0.5 p err (C n ) = Φ( L V(S) ) + o(1) 0 L For an error probability ε, the optimum second-order coding rate is L (ε) := V(S)Φ 1 (ε) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

35 Network Information Theory Network Information Theory by A. El Gamal and Y.-H. Kim Many problems unsolved Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

36 Network Information Theory Network Information Theory by A. El Gamal and Y.-H. Kim Many problems unsolved My agenda is to understand second-order behavior of solved NIT problems Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

37 Network Information Theory Network Information Theory by A. El Gamal and Y.-H. Kim Many problems unsolved My agenda is to understand second-order behavior of solved NIT problems Gain new insights on second-order optimal coding schemes Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

38 Network Information Theory Network Information Theory by A. El Gamal and Y.-H. Kim Many problems unsolved My agenda is to understand second-order behavior of solved NIT problems Gain new insights on second-order optimal coding schemes We study two simple NIT problems in the rest of the talk Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

39 Outline 1 Motivation, Background and History 2 Gaussian Interference Channel with Very Strong Interference 3 Gaussian MAC with Degraded Message Sets 4 Conclusion Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

40 Gaussian Interference Channel M 1 f 1 X1 n Y n 1 ϕ 1 ˆM 1 W n M 2 f 2 X2 n Y n 2 ϕ 2 ˆM 2 Two-sender two-receiver GIC is given as Y 1i = g 11 X 1i + g 12 X 2i + Z 1i, Y 2i = g 21 X 1i + g 22 X 2i + Z 2i Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

41 Gaussian Interference Channel M 1 f 1 X1 n Y n 1 ϕ 1 ˆM 1 W n M 2 f 2 X2 n Y n 2 ϕ 2 ˆM 2 Two-sender two-receiver GIC is given as Y 1i = g 11 X 1i + g 12 X 2i + Z 1i, Y 2i = g 21 X 1i + g 22 X 2i + Z 2i Channel inputs are power limited n Xji 2 ns j, j = 1, 2 i=1 Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

42 Gaussian Interference Channel M 1 f 1 X1 n Y n 1 ϕ 1 ˆM 1 W n M 2 f 2 X2 n Y n 2 ϕ 2 ˆM 2 Two-sender two-receiver GIC is given as Y 1i = g 11 X 1i + g 12 X 2i + Z 1i, Y 2i = g 21 X 1i + g 22 X 2i + Z 2i Channel inputs are power limited n Xji 2 ns j, j = 1, 2 i=1 Capacity region is an open problem in NIT Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

43 Very Strong Inference Define the signal-to-noise ratios as snr 1 = g 2 11S 1, snr 2 = g 2 22S 2 Define the interference-to-noise ratios as inr 1 = g 2 12S 2, inr 2 = g 2 21S 1 Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

44 Very Strong Inference Define the signal-to-noise ratios as snr 1 = g 2 11S 1, snr 2 = g 2 22S 2 Define the interference-to-noise ratios as inr 1 = g 2 12S 2, inr 2 = g 2 21S 1 A GIC is said to have strictly very strong interference (SVSI) if Equivalently snr 1 < inr snr 2, snr 2 < inr snr 1 C(snr 1 ) + C(snr 2 ) < min{c(snr 1 + inr 1 ), C(snr 2 + inr 2 )} Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

45 Very Strong Inference Define the signal-to-noise ratios as snr 1 = g 2 11S 1, snr 2 = g 2 22S 2 Define the interference-to-noise ratios as inr 1 = g 2 12S 2, inr 2 = g 2 21S 1 A GIC is said to have strictly very strong interference (SVSI) if Equivalently snr 1 < inr snr 2, snr 2 < inr snr 1 C(snr 1 ) + C(snr 2 ) < min{c(snr 1 + inr 1 ), C(snr 2 + inr 2 )} Intuition: Receiver 1 decodes interference M 2, then uses that to decode intended message M 1 (and vice versa) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

46 Capacity Region for GICs with SVSI R 2 C 2 (ii) (iii) (i) C 1 R 1 A. Carleial Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

47 Capacity Region for GICs with SVSI R 2 C 2 (ii) (iii) (i) C 1 R 1 A. Carleial Carleial (1975) showed that the capacity region is R 1 C 1 := C(snr 1 ) R 2 C 2 := C(snr 2 ) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

48 Capacity Region for GICs with SVSI R 2 C 2 (ii) (iii) (i) C 1 R 1 A. Carleial Carleial (1975) showed that the capacity region is R 1 C 1 := C(snr 1 ) R 2 C 2 := C(snr 2 ) Examine deviations of order 1 n away from the boundary Three distinct regions. Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

49 Second-Order Coding Rate Region (L 1, L 2 ) is an (ε, R 1, R 2 )-achievable second-order coding rate pair if there exists a sequence of (n, M 1n, M 2n, ε n )-codes such that the code sizes M jn satisfy lim inf n 1 n (log M jn nr j ) L j, j = 1, 2 Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

50 Second-Order Coding Rate Region (L 1, L 2 ) is an (ε, R 1, R 2 )-achievable second-order coding rate pair if there exists a sequence of (n, M 1n, M 2n, ε n )-codes such that the code sizes M jn satisfy lim inf n 1 n (log M jn nr j ) L j, j = 1, 2 and the average error probabilities ε n satisfy lim sup ε n ε n Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

51 Second-Order Coding Rate Region (L 1, L 2 ) is an (ε, R 1, R 2 )-achievable second-order coding rate pair if there exists a sequence of (n, M 1n, M 2n, ε n )-codes such that the code sizes M jn satisfy lim inf n 1 n (log M jn nr j ) L j, j = 1, 2 and the average error probabilities ε n satisfy lim sup ε n ε n If (L 1, L 2 ) is (ε, R 1, R 2 )-achievable, then there exists a sequence of codes such that log M jn nr j + nl j + o( n) and ε n ε + o(1). Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

52 Optimum Second-Order Coding Rate Region The set of all (ε, R 1, R 2 )-achievable second-order coding rate pairs is called the (ε, R 1, R 2 )-optimum second-order coding rate region L(ε; R 1, R 2) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

53 Optimum Second-Order Coding Rate Region The set of all (ε, R 1, R 2 )-achievable second-order coding rate pairs is called the (ε, R 1, R 2 )-optimum second-order coding rate region L(ε; R 1, R 2) Note: In the single-user case, L (ε) := sup L(ε; R 1 = C 1 ) = V(S 1 )Φ 1 (ε) This follows from Hayashi s work (2009) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

54 Characterize L(ε; R 1, R 2 ) This is joint work with S.-Q. Le (NUS) M. Motani (NUS) We want to characterize L(ε; R 1, R 2 ) for all points (R 1, R 2 ) on the boundary of the capacity region Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

55 Characterize L(ε; R 1, R 2 ) This is joint work with S.-Q. Le (NUS) M. Motani (NUS) We want to characterize L(ε; R 1, R 2 ) for all points (R 1, R 2 ) on the boundary of the capacity region For all (R 1, R 2 ) in the interior of the capacity region L(ε; R 1, R 2) = R 2 For all (R 1, R 2 ) in the exterior of the capacity region implying the strong converse L(ε; R 1, R 2) = Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

56 Main Result: Vertical Boundary R 2 C 2 (i) C 1 R 1 In case (i), we are far from the horizontal boundary R 2 < C 2 Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

57 Main Result: Vertical Boundary R 2 C 2 (i) C 1 R 1 In case (i), we are far from the horizontal boundary R 2 < C 2 Error event for user 2 vanishes (large deviations) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

58 Main Result: Vertical Boundary R 2 C 2 (i) C 1 R 1 In case (i), we are far from the horizontal boundary R 2 < C 2 Error event for user 2 vanishes (large deviations) Hence second-order asymptotics only pertains to user 1 L(ε; R 1, R 2) = { (L 1, L 2 ) : L 1 V 1 Φ 1 (ε) }. Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

59 Main Result: Corner Point R 2 C 2 (iii) C 1 R 1 In case (iii), we are operating near both boundaries Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

60 Main Result: Corner Point R 2 C 2 (iii) C 1 R 1 In case (iii), we are operating near both boundaries So both constraints are active and we see L 1 and L 2 in the optimum second-order coding rate region is { ( L(ε; R 1, R 2) = (L 1, L 2 ) : Φ L ( 1 )Φ L ) } 2 1 ε. V1 V2 Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

61 Heuristic Derivation of Case (iii) Let G j := { ˆM j = ˆM} be the event that message j = 1, 2 is decoded correctly Pr(G 1 G 2 ) 1 ε Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

62 Heuristic Derivation of Case (iii) Let G j := { ˆM j = ˆM} be the event that message j = 1, 2 is decoded correctly Pr(G 1 G 2 ) 1 ε Assuming independence (which does not hold generally), Pr(G 1 ) Pr(G 2 ) 1 ε Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

63 Heuristic Derivation of Case (iii) Let G j := { ˆM j = ˆM} be the event that message j = 1, 2 is decoded correctly Pr(G 1 G 2 ) 1 ε Assuming independence (which does not hold generally), Pr(G 1 ) Pr(G 2 ) 1 ε But we know from the single-user result that the error probability ( Pr(Gj c Lj ) ( ) Φ = Pr(G j ) Φ L j ). Vj Vj So the set of all (ε, C 1, C 2 )-achievable second-order coding rates is ( Φ L ( 1 )Φ L ) 2 1 ε V1 V2 Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

64 Illustration of Case (iii) L1 (nats/ use) V 1 = V 2 = 2 V 1 = V 2 = L 1 (nats/ use) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

65 Intuition Gleaned from GIC with SVSI Carleial (1975) mentioned that Very strong interference is as innocuous as no interference at all Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

66 Intuition Gleaned from GIC with SVSI Carleial (1975) mentioned that Very strong interference is as innocuous as no interference at all We show that SVSI is innocuous in the sense that the capacities C j and dispersions V j are not affected Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

67 Intuition Gleaned from GIC with SVSI Carleial (1975) mentioned that Very strong interference is as innocuous as no interference at all We show that SVSI is innocuous in the sense that the capacities C j and dispersions V j are not affected Error events are approximately independent Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

68 Intuition Gleaned from GIC with SVSI Carleial (1975) mentioned that Very strong interference is as innocuous as no interference at all We show that SVSI is innocuous in the sense that the capacities C j and dispersions V j are not affected Error events are approximately independent Intuition that the GIC with SVSI is analogous to 2 independent direct channels carries over, even in the finer second-order sense Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

69 Outline 1 Motivation, Background and History 2 Gaussian Interference Channel with Very Strong Interference 3 Gaussian MAC with Degraded Message Sets 4 Conclusion Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

70 Gaussian MAC with Degraded Message Sets M 1 M 2 f 1 f 2 X1 n X2 n Y n ( ˆM W n 1, ˆM 2 ) ϕ Channel law is Y i = X 1i + X 2i + Z i Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

71 Gaussian MAC with Degraded Message Sets M 1 M 2 f 1 f 2 X1 n X2 n Y n ( ˆM W n 1, ˆM 2 ) ϕ Channel law is Y i = X 1i + X 2i + Z i Channel inputs are power limited n Xji 2 ns j, j = 1, 2 i=1 Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

72 Capacity Region Capacity region is an exercise in NIT Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

73 Capacity Region Capacity region is an exercise in NIT The capacity region is the set of all (R 1, R 2 ) such that R 1 C(S 1 (1 ρ 2 )) R 1 + R 2 C(S 1 + S 2 + 2ρ S 1 S 2 ) for some 0 ρ 1. Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

74 Capacity Region Capacity region is an exercise in NIT The capacity region is the set of all (R 1, R 2 ) such that R 1 C(S 1 (1 ρ 2 )) R 1 + R 2 C(S 1 + S 2 + 2ρ S 1 S 2 ) for some 0 ρ 1. Uses superposition coding (Cover (1972)) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

75 Capacity Region The Capacity Region CR Boundary ρ = 0 ρ = 1/3 ρ = 2/ R2 (nats/use) R 1 (nats/use) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

76 Characterize L(ε; R 1, R 2 ) This is joint work with Jon Scarlett We want to characterize L(ε; R 1, R 2 ) for all points (R 1, R 2 ) on the boundary of the capacity region Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

77 Characterize L(ε; R 1, R 2 ) This is joint work with Jon Scarlett We want to characterize L(ε; R 1, R 2 ) for all points (R 1, R 2 ) on the boundary of the capacity region For all (R 1, R 2 ) in the interior of the capacity region L(ε; R 1, R 2) = R 2 For all (R 1, R 2 ) in the exterior of the capacity region implying the strong converse L(ε; R 1, R 2) = Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

78 Some Basic Definitions Mutual informations [ ] [ I1 (ρ) C ( (1 ρ I(ρ) := = 2 ) ] )S 1 I 12 (ρ) C ( S 1 + S 2 + 2ρ ) S 1 S 2 Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

79 Some Basic Definitions Mutual informations [ ] [ I1 (ρ) C ( (1 ρ I(ρ) := = 2 ) ] )S 1 I 12 (ρ) C ( S 1 + S 2 + 2ρ ) S 1 S 2 Derivative of mutual informations D(ρ) := ρ I(ρ) = [ S 1 ρ 1+S 1 (1 ρ 2 ) S1 S 2 ] 1+S 1 +S 2 +2ρ S 1 S 2 Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

80 Some Basic Definitions Mutual informations [ ] [ I1 (ρ) C ( (1 ρ I(ρ) := = 2 ) ] )S 1 I 12 (ρ) C ( S 1 + S 2 + 2ρ ) S 1 S 2 Derivative of mutual informations D(ρ) := ρ I(ρ) = [ S 1 ρ 1+S 1 (1 ρ 2 ) S1 S 2 ] x(y+2) 2(x+1)(y+1) 1+S 1 +S 2 +2ρ S 1 S 2 Dispersions V(x, y) := and V(x) := V(x, x) [ ] V1 (ρ) V V(ρ) := 1,12 (ρ) V 1,12 (ρ) V 12,12 (ρ) where V 1 (ρ) := V ( (1 ρ 2 ) )S 1, V12,12 (ρ) := V ( S 1 + S 2 + 2ρ ) S 1 S 2 V 1,12 (ρ) := V ( (1 ρ 2 )S 1, S 1 + S 2 + 2ρ ) S 1 S 2 Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

81 Generalization of Inverse CDF of a Gaussian For a positive semi-definite matrix V, Ψ(z 1, z 2, V) = z1 z2 N (0, V) du Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

82 Generalization of Inverse CDF of a Gaussian For a positive semi-definite matrix V, Ψ(z 1, z 2, V) = Given ε (0, 1), z1 z2 N (0, V) du Ψ 1 (V, ε) = {(z 1, z 2 ) : Ψ( z 1, z 2, V) 1 ε} ρ = ρ = z2 z ε = 0.01 ε = z ε = 0.01 ε = z1 Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

83 The Main Result: Vertical Boundary Points on vertical boundary reduce to scalar dispersion as sum rate constraint is in error exponents regime L(ε; R 1, R 2) = {(L 1, L 2 ) : L 1 V 1 (0)Φ 1 (ε)} The Capacity Region CR Boundary ρ =0 ρ =1/3 ρ =2/ R2 (nats/use) R1 (nats/use) (R 1, R 2 ) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

84 The Main Result: Vertical Boundary Points on vertical boundary reduce to scalar dispersion as sum rate constraint is in error exponents regime L(ε; R 1, R 2) = {(L 1, L 2 ) : L 1 V 1 (0)Φ 1 (ε)} R2 (nats/use) The Capacity Region CR Boundary ρ =0 ρ =1/3 ρ =2/ R1 (nats/use) (R 1, R 2 ) Following expansion holds for cardinality of first codebook log M 1n V1 (0) I 1 (0) + Φ 1 (ε) n n Far from sum rate constraint 1 lim n n log(m 1nM 2n ) < I 12 (0) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

85 The Main Result: Curved Boundary Different behavior in the curved region { [ ] L(ε; R 1, R L 2) = (L 1, L 2 ) : 1 } βd(ρ) + Ψ 1 (V(ρ), ε) L 1 + L 2 β R The Capacity Region CR Boundary ρ =0 ρ =1/3 ρ =2/3 R2 (nats/use) (R 1, R 2 ) R1 (nats/use) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

86 The Main Result: Curved Boundary Different behavior in the curved region { [ ] L(ε; R 1, R L 2) = (L 1, L 2 ) : 1 } βd(ρ) + Ψ 1 (V(ρ), ε) L 1 + L 2 β R R2 (nats/use) The Capacity Region CR Boundary ρ =0 ρ =1/3 ρ =2/3 (R 1, R 2 ) D(ρ) doesn t appear usually R1 (nats/use) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

87 The Main Result: Curved Boundary Different behavior in the curved region { [ ] L(ε; R 1, R L 2) = (L 1, L 2 ) : 1 } βd(ρ) + Ψ 1 (V(ρ), ε) L 1 + L 2 β R R2 (nats/use) The Capacity Region CR Boundary ρ =0 ρ =1/3 ρ =2/3 (R 1, R 2 ) D(ρ) doesn t appear usually Ψ 1 (V(ρ), ε): corresponds using Gaussian with covariance matrix [ S Σ(ρ) = 1 ρ ] S 1 S 2 ρ S 1 S 2 S R1 (nats/use) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

88 Achieving all Second-Order Rate Pairs L(ε; R 1, R 2) = { [ ] L (L 1, L 2 ) : 1 L 1 + L 2 } βd(ρ) + Ψ 1 (V(ρ), ε) β R Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

89 Achieving all Second-Order Rate Pairs L(ε; R 1, R 2) = { [ ] L (L 1, L 2 ) : 1 L 1 + L 2 } βd(ρ) + Ψ 1 (V(ρ), ε) β R Non-empty regions in CR not in trapezium achievable by ( [ S N 0, 1 ρ ]) S 1 S 2 ρ S 1 S 2 S 2 Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

90 Achieving all Second-Order Rate Pairs L(ε; R 1, R 2) = { [ ] L (L 1, L 2 ) : 1 L 1 + L 2 } βd(ρ) + Ψ 1 (V(ρ), ε) β R Non-empty regions in CR not in trapezium achievable by ( [ S N 0, 1 ρ ]) S 1 S 2 ρ S 1 S 2 S 2 Use the above distribution dependent on blocklength: By a Taylor expansion, ρ n = ρ + β n I(ρ n ) I(ρ) + (ρ n ρ)d(ρ) = I(ρ) + βd(ρ) n explaining the slope term. Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

91 Illustration of Second-Order Coding Rates Ψ 1 (V,ε) L(ε;R 1,R 2) L2(nats/ use) S 1 = S 2 = 1 and ρ = 1 2 L(ε; R 1, R 2 ) is a half-space L 1 (nats/ use) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

92 Illustration of Second-Order Coding Rates Ψ 1 (V,ε) L(ε;R 1,R 2) L2(nats/ use) S 1 = S 2 = 1 and ρ = 1 2 L(ε; R 1, R 2 ) is a half-space L 1 (nats/ use) Second-order rates achieved using a single input distribution N (0, Σ(ρ)) is not optimal Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

93 Illustration of Second-Order Coding Rates Ψ 1 (V,ε) L(ε;R 1,R 2) L2(nats/ use) S 1 = S 2 = 1 and ρ = 1 2 L(ε; R 1, R 2 ) is a half-space L 1 (nats/ use) Second-order rates achieved using a single input distribution N (0, Σ(ρ)) is not optimal Need to vary input distribution with blocklength to achieve all points in L(ε; R 1, R 2 ) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

94 Outline 1 Motivation, Background and History 2 Gaussian Interference Channel with Very Strong Interference 3 Gaussian MAC with Degraded Message Sets 4 Conclusion Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

95 Conclusion Asymptotic expansions for NIT problems with non-vanishing error probabilities is a very fertile area of research Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

96 Conclusion Asymptotic expansions for NIT problems with non-vanishing error probabilities is a very fertile area of research Other single-user that are solved include: 1 Quasi-static MIMO fading channels (Yang-Durisi-Koch-Polyanskiy) 2 Channels with discrete state (Tomamichel-Tan) 3 Dirty-paper coding (Scarlett) Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39

97 Conclusion Asymptotic expansions for NIT problems with non-vanishing error probabilities is a very fertile area of research Other single-user that are solved include: 1 Quasi-static MIMO fading channels (Yang-Durisi-Koch-Polyanskiy) 2 Channels with discrete state (Tomamichel-Tan) 3 Dirty-paper coding (Scarlett) Shameless self-promotion: V. Y. F. Tan Asymptotic expansions in IT with non-vanishing error probabilities Now Publishers Foundations and Trends in Comms and Inf. Th. Vincent Tan (NUS) IT with Non-Vanishing Errors Chalmers University / 39