A Tight Upper Bound on the Second-Order Coding Rate of Parallel Gaussian Channels with Feedback

Transcription

1 A Tight Upper Bound on the Second-Order Coding Rate of Parallel Gaussian Channels with Feedback Vincent Y. F. Tan (NUS) Joint work with Silas L. Fong (Toronto) 2017 Information Theory Workshop, Kaohsiung, Taiwan Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

2 Channel Model: The Parallel Gaussian Channel Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

3 Channel Model: The Parallel Gaussian Channel Channel Law (g 1 = g 2 =... = g d = 1) Y k = X k + Z k, Y 1,k Y 2,k. Y d,k = X 1,k X 2,k and Z l,k i.i.d. N (0, N l ) where l [1 : d].. X d,k + Z 1,k Z 2,k. Z d,k, k [1 : n] Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

4 Capacity of the Parallel Gaussian Channel Consider a peak power constraint { } 1 d n Pr Xl,k 2 P = 1 n l=1 k=1 Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

5 Capacity of the Parallel Gaussian Channel Consider a peak power constraint { } 1 d n Pr Xl,k 2 P = 1 n l=1 k=1 Define the capacity functions d ( ) sl C(s) = C, where C(P) := 1 log(1 + P). 2 l=1 N l Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

6 Capacity of the Parallel Gaussian Channel Consider a peak power constraint { } 1 d n Pr Xl,k 2 P = 1 n l=1 k=1 Define the capacity functions d ( ) sl C(s) = C, where C(P) := 1 log(1 + P). 2 l=1 N l Vanishing-error capacity is given by the water-filling solution and C(P ), where P = (P 1, P 2,..., P d) P l = (λ N l ) + and λ > 0 satisfies d P l = P. l=1 Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

7 Capacity vs. Non-Asymptotic Fundamental Limits Define the non-asymptotic fundamental limit M (n, ε, P) := max{m N : (n, M, ε, P)-code}. Maximum number of messages that can be supported by a length-n code with peak power P and average error prob. ε. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

8 Capacity vs. Non-Asymptotic Fundamental Limits Define the non-asymptotic fundamental limit M (n, ε, P) := max{m N : (n, M, ε, P)-code}. Maximum number of messages that can be supported by a length-n code with peak power P and average error prob. ε. Capacity result can be stated as 1 lim lim inf ε 0 n n log M (n, ε, P) = C(P ). Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

9 Capacity vs. Non-Asymptotic Fundamental Limits Define the non-asymptotic fundamental limit M (n, ε, P) := max{m N : (n, M, ε, P)-code}. Maximum number of messages that can be supported by a length-n code with peak power P and average error prob. ε. Capacity result can be stated as 1 lim lim inf ε 0 n n log M (n, ε, P) = C(P ). In fact, the strong converse holds, and we have lim inf n 1 n log M (n, ε, P) = C(P ), ε (0, 1). Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

10 Strong Converse ε = lim sup P (n) 1 e, R = lim inf n n n log M (n, ε, P) ε 0.5 Peak C(P ) R Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

11 Second-Order Asymptotics Let V(P) := P(P + 2) 2(P + 1) 2 be the dispersion of the point-to-point AWGN channel. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

12 Second-Order Asymptotics Let V(P) := P(P + 2) 2(P + 1) 2 be the dispersion of the point-to-point AWGN channel. Define the sum dispersion function to be d ( ) sl V(s) := V. l=1 N l Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

13 Second-Order Asymptotics Let V(P) := P(P + 2) 2(P + 1) 2 be the dispersion of the point-to-point AWGN channel. Define the sum dispersion function to be d ( ) sl V(s) := V. l=1 Then Polyanskiy (2010) and Tomamichel-T. (2015) showed that log M (n, ε, P) = nc(p ) + nv(p )Φ 1 (ε) + O(log n). where P = (P 1, P 2,..., P d ) is the optimal power allocation and b 1 ( Φ(b) = exp u2 ) du. 2π 2 N l Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

14 Feedback If feedback is present, then the encoder at time k has access to message W and previous channel outputs (Y 1, Y 2,..., Y k 1 ), i.e., X k = X k (W, Y k 1 ), k [1 : n]. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

15 Feedback If feedback is present, then the encoder at time k has access to message W and previous channel outputs (Y 1, Y 2,..., Y k 1 ), i.e., X k = X k (W, Y k 1 ), k [1 : n]. Define the non-asymptotic fundamental limit MFB (n, ε, P) := max{m N : (n, M, ε, P)-feedback code}. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

16 Feedback If feedback is present, then the encoder at time k has access to message W and previous channel outputs (Y 1, Y 2,..., Y k 1 ), i.e., X k = X k (W, Y k 1 ), k [1 : n]. Define the non-asymptotic fundamental limit MFB (n, ε, P) := max{m N : (n, M, ε, P)-feedback code}. Since feedback does not increase capacity of point-to-point memoryless channels [Shannon (1956)], 1 lim lim inf ε 0 n n log M FB (n, ε, P) = C(P ). Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

17 Feedback If feedback is present, then the encoder at time k has access to message W and previous channel outputs (Y 1, Y 2,..., Y k 1 ), i.e., X k = X k (W, Y k 1 ), k [1 : n]. Define the non-asymptotic fundamental limit MFB (n, ε, P) := max{m N : (n, M, ε, P)-feedback code}. Since feedback does not increase capacity of point-to-point memoryless channels [Shannon (1956)], 1 lim lim inf ε 0 n n log M FB (n, ε, P) = C(P ). Natural question: What happens to the second-order terms? Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

18 Main Result Theorem (Fong-T. (2017)) Feedback does not affect the second-order term for parallel Gaussian channels with feedback, i.e., log M FB (n, ε, P) = nc(p ) + nv(p )Φ 1 (ε) + o( n). Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

19 Main Result Theorem (Fong-T. (2017)) Feedback does not affect the second-order term for parallel Gaussian channels with feedback, i.e., log M FB (n, ε, P) = nc(p ) + nv(p )Φ 1 (ε) + o( n). Recall that without feedback log M (n, ε, P) = nc(p ) + nv(p )Φ 1 (ε) + O(log n). Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

20 Main Result Theorem (Fong-T. (2017)) Feedback does not affect the second-order term for parallel Gaussian channels with feedback, i.e., log M FB (n, ε, P) = nc(p ) + nv(p )Φ 1 (ε) + o( n). Recall that without feedback log M (n, ε, P) = nc(p ) + nv(p )Φ 1 (ε) + O(log n). No guarantees on third-order term. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

21 Main Result Theorem (Fong-T. (2017)) Feedback does not affect the second-order term for parallel Gaussian channels with feedback, i.e., log M FB (n, ε, P) = nc(p ) + nv(p )Φ 1 (ε) + o( n). Recall that without feedback log M (n, ε, P) = nc(p ) + nv(p )Φ 1 (ε) + O(log n). No guarantees on third-order term. For achievability, encoder can ignore feedback. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

22 Main Result Theorem (Fong-T. (2017)) Feedback does not affect the second-order term for parallel Gaussian channels with feedback, i.e., log M FB (n, ε, P) = nc(p ) + nv(p )Φ 1 (ε) + o( n). Recall that without feedback log M (n, ε, P) = nc(p ) + nv(p )Φ 1 (ε) + O(log n). No guarantees on third-order term. For achievability, encoder can ignore feedback. Only need to prove converse part. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

23 Comparison to AWGN channel with feedback For the AWGN channel with feedback (d = 1), information density log pn Y X (Yn x n ) p n Y (Yn ) has the same distribution for all x n such that x n 2 = np. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

24 Comparison to AWGN channel with feedback For the AWGN channel with feedback (d = 1), information density log pn Y X (Yn x n ) p n Y (Yn ) = Yn x n 2 2N + c has the same distribution for all x n such that x n 2 = np. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

25 Comparison to AWGN channel with feedback For the AWGN channel with feedback (d = 1), information density log pn Y X (Yn x n ) p n Y (Yn ) = Yn x n 2 2N = xn + Z n, x n N + c + c has the same distribution for all x n such that x n 2 = np. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

26 Comparison to AWGN channel with feedback For the AWGN channel with feedback (d = 1), information density log pn Y X (Yn x n ) p n Y (Yn ) = Yn x n 2 + c 2N = xn + Z n, x n + c N = 1 n Z k x k + c N k=1 has the same distribution for all x n such that x n 2 = np. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

27 Comparison to AWGN channel with feedback For the AWGN channel with feedback (d = 1), information density log pn Y X (Yn x n ) p n Y (Yn ) = Yn x n 2 + c 2N = xn + Z n, x n + c N = 1 n Z k x k + c N k=1 has the same distribution for all x n such that x n 2 = np. This spherical symmetry argument [Polyanskiy (2011, 2013) and Fong-T. (2015)] yields log M FB (n, ε, P) = nc(p) + nv(p)φ 1 (ε) + O(log n). Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

28 Comparison to AWGN channel with feedback For the AWGN channel with feedback (d = 1), information density log pn Y X (Yn x n ) p n Y (Yn ) = Yn x n 2 + c 2N = xn + Z n, x n + c N = 1 n Z k x k + c N k=1 has the same distribution for all x n such that x n 2 = np. This spherical symmetry argument [Polyanskiy (2011, 2013) and Fong-T. (2015)] yields log M FB (n, ε, P) = nc(p) + nv(p)φ 1 (ε) + O(log n). Symmetry argument doesn t work for parallel Gaussians. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

29 Elements of Converse Proof Want to show lim sup n log M FB (n, ε, P) nc(p ) n V(P )Φ 1 (ε). Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

30 Elements of Converse Proof Want to show lim sup n log M FB (n, ε, P) nc(p ) n V(P )Φ 1 (ε). Discretizing the power allocation: Turn power allocations into power types Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

31 Elements of Converse Proof Want to show lim sup n log M FB (n, ε, P) nc(p ) n V(P )Φ 1 (ε). Discretizing the power allocation: Turn power allocations into power types Apply the meta-converse [Polyanskiy-Poor-Verdú (2010)] Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

32 Elements of Converse Proof Want to show lim sup n log M FB (n, ε, P) nc(p ) n V(P )Φ 1 (ε). Discretizing the power allocation: Turn power allocations into power types Apply the meta-converse [Polyanskiy-Poor-Verdú (2010)] Apply Curtiss theorem to the information spectrum term for close-to-optimal power types Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

33 Elements of Converse Proof Want to show lim sup n log M FB (n, ε, P) nc(p ) n V(P )Φ 1 (ε). Discretizing the power allocation: Turn power allocations into power types Apply the meta-converse [Polyanskiy-Poor-Verdú (2010)] Apply Curtiss theorem to the information spectrum term for close-to-optimal power types Apply large deviations bounds to far-from-optimal power types Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

34 Turn Power Allocation into Types: Part I Given a channel input x n R d n, define its power type to be φ(x n ) = 1 n [ n = 1 n [ x n 1 2, x n 2 2,..., x n d 2] k=1 x 2 1,k, n x2,k, 2..., n k=1 k=1 x 2 d,k ] R d +. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

35 Turn Power Allocation into Types: Part I Given a channel input x n R d n, define its power type to be φ(x n ) = 1 n [ n = 1 n [ x n 1 2, x n 2 2,..., x n d 2] k=1 x 2 1,k, n x2,k, 2..., n k=1 k=1 x 2 d,k ] R d +. Create a new code such that almost surely, n Xl,k 2 = m l P, for some m l [1 : n] and l [1 : d] k=1 and d n Xl,k 2 = np, l=1 k=1 i.e., d m l = n. l=1 Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

36 Turn Power Allocation into Types: Part II Power allocation vector set { S (n) P := n a a = (a 1, a 2,..., a d ) Z d +, } d a l = n l=1 Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

37 Turn Power Allocation into Types: Part II Power allocation vector set { } S (n) P := n a d a = (a 1, a 2,..., a d ) Z d +, a l = n l=1 a 2 (0, n) P/n P/n a 1 (n, 0) Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

38 Turn Power Allocation into Types: Part III Set of quantized power allocation vectors s with quantization level P/n that satisfy the equality power constraint d s l = P. l=1 Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

39 Turn Power Allocation into Types: Part III Set of quantized power allocation vectors s with quantization level P/n that satisfy the equality power constraint d s l = P. l=1 With the above construction, almost surely, φ(x n ) S (n). Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

40 Turn Power Allocation into Types: Part III Set of quantized power allocation vectors s with quantization level P/n that satisfy the equality power constraint d s l = P. l=1 With the above construction, almost surely, φ(x n ) S (n). So we can pretend that the codewords are discrete and leverage some existing theory from second-order analysis for DMCs. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

41 Application of the Meta-Converse (PPV 10) By a relaxation of the meta-converse, for any ξ > 0, { log MFB (1 (n, ε, P) log ξ log ε Pr log pn Y X (Yn X n ) q Y n(y n ) log ξ }) Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

42 Application of the Meta-Converse (PPV 10) By a relaxation of the meta-converse, for any ξ > 0, { log MFB (1 (n, ε, P) log ξ log ε Pr log pn Y X (Yn X n ) q Y n(y n ) log ξ }) Choose auxiliary output distribution q Y n carefully. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

43 Application of the Meta-Converse (PPV 10) By a relaxation of the meta-converse, for any ξ > 0, { log MFB (1 (n, ε, P) log ξ log ε Pr log pn Y X (Yn X n ) q Y n(y n ) log ξ }) Choose auxiliary output distribution q Y n carefully. Inspired by Hayashi (2009), we choose q Y n(y n )= S (n) N (y l,k ; 0, s l +N l ) N (y l,k ; 0, P l +N l ) s S (n) l,k l,k }{{}}{{} q (1) Y n (y n ) q (2) Y n (y n ) Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

44 Application of the Meta-Converse (PPV 10) By a relaxation of the meta-converse, for any ξ > 0, { log MFB (1 (n, ε, P) log ξ log ε Pr log pn Y X (Yn X n ) q Y n(y n ) log ξ }) Choose auxiliary output distribution q Y n carefully. Inspired by Hayashi (2009), we choose q Y n(y n )= S (n) N (y l,k ; 0, s l +N l ) N (y l,k ; 0, P l +N l ) s S (n) l,k l,k }{{}}{{} q (1) Y n (y n ) q (2) Y n (y n ) q (1) Y n (y n ): Unif. mixture of output dist. induced by input power types Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

45 Application of the Meta-Converse (PPV 10) By a relaxation of the meta-converse, for any ξ > 0, { log MFB (1 (n, ε, P) log ξ log ε Pr log pn Y X (Yn X n ) q Y n(y n ) log ξ }) Choose auxiliary output distribution q Y n carefully. Inspired by Hayashi (2009), we choose q Y n(y n )= S (n) N (y l,k ; 0, s l +N l ) N (y l,k ; 0, P l +N l ) s S (n) l,k l,k }{{}}{{} q (1) Y n (y n ) q (2) Y n (y n ) q (1) Y n (y n ): Unif. mixture of output dist. induced by input power types q (2) Y n (y n ): Capacity-achieving output dist. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

46 Almost-Optimal Types: Part I Define the set of almost-optimal power types { Π (n) := s S (n) s P 2 1 } n 1/6 Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

47 Almost-Optimal Types: Part I Define the set of almost-optimal power types { Π (n) := s S (n) s P 2 1 } n 1/6 S (n) Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

48 Almost-Optimal Types: Part I Define the set of almost-optimal power types { Π (n) := s S (n) s P 2 1 } n 1/6 S (n) P 3 Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

49 Almost-Optimal Types: Part I Define the set of almost-optimal power types { Π (n) := s S (n) s P 2 1 } n 1/6 Π (n) S (n) P 3 Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

50 Almost-Optimal Types: Part II Define threshold log ξ := nc(p ) + ) n( V(P )Φ 1 (ε + τ) ( ) + log 2 S (n) Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

51 Almost-Optimal Types: Part II Define threshold log ξ := nc(p ) + ) n( V(P )Φ 1 (ε + τ) Contribution of information spectrum term on Π (n) is { } 1 n Pr U (P ) nv(p k Φ 1 (ε + τ) ) where k=1 ( ) + log 2 S (n) U (P ) k := d l=1 ( P l N l )Z 2 l,k + 2X l,kz l,k + P l 2(P l + N l ) k [1 : n] and φ(x n ) Π (n). Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

52 Almost-Optimal Types: Part II Define threshold log ξ := nc(p ) + ) n( V(P )Φ 1 (ε + τ) Contribution of information spectrum term on Π (n) is { } 1 n Pr U (P ) nv(p k Φ 1 (ε + τ) ) where k=1 ( ) + log 2 S (n) U (P ) k := d l=1 ( P l N l )Z 2 l,k + 2X l,kz l,k + P l 2(P l + N l ) k [1 : n] and φ(x n ) Π (n). Because of feedback, U (P ) k is not independent across time! Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

53 Almost-Optimal Types: Part III Would like to approximate the nasty { U (P ) k k [1 : n]} with the independent and identically distributed random variables V (P ) k := d l=1 ( P l N l )Z 2 l,k + 2 P l Z l,k + P l 2(P l + N l ) k [1 : n] Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

54 Almost-Optimal Types: Part III Would like to approximate the nasty { U (P ) k k [1 : n]} with the independent and identically distributed random variables V (P ) k := d l=1 ( P l N l )Z 2 l,k + 2 P l Z l,k + P l 2(P l + N l ) k [1 : n] Due to our discretization procedure and the fact that φ(x n ) Π (n) lim E n [ for all t R. exp ( t n n k=1 ) ] U (P ) k = lim E n [ exp ( t n n k=1 ) ] V (P ) k Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

55 Almost-Optimal Types: Part IV Curtiss (aka Lévy s continuity) theorem: implies that lim E[exp(tX n)] = lim E[exp(tY n)], t R, n n lim Pr{X n a} = lim Pr{Y n a}, a R. n n Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

56 Almost-Optimal Types: Part IV Curtiss (aka Lévy s continuity) theorem: implies that Thus, lim E[exp(tX n)] = lim E[exp(tY n)], t R, n n lim Pr{X n a} = lim Pr{Y n a}, a R. n n lim Pr n { 1 nv(p ) n k=1 U (P ) k Φ 1 (ε + τ) } Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

57 Almost-Optimal Types: Part IV Curtiss (aka Lévy s continuity) theorem: implies that Thus, lim E[exp(tX n)] = lim E[exp(tY n)], t R, n n lim Pr{X n a} = lim Pr{Y n a}, a R. n n lim Pr n { = lim n Pr 1 nv(p ) { n k=1 1 nv(p ) U (P ) k Φ 1 (ε + τ) n k=1 } V (P ) k Φ 1 (ε + τ) } Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

58 Almost-Optimal Types: Part IV Curtiss (aka Lévy s continuity) theorem: implies that Thus, lim E[exp(tX n)] = lim E[exp(tY n)], t R, n n lim Pr{X n a} = lim Pr{Y n a}, a R. n n lim Pr n { = lim n Pr 1 nv(p ) { CLT = 1 (ε + τ). n k=1 1 nv(p ) U (P ) k Φ 1 (ε + τ) n k=1 } V (P ) k Φ 1 (ε + τ) } Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

59 Far-From-Optimal Types Contribution of information spectrum term on S (n) \ Π (n) is { } 1 n Pr U (s) k C(P ) C(s), s S (n) \ Π (n) n k=1 Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

60 Far-From-Optimal Types Contribution of information spectrum term on S (n) \ Π (n) is { } 1 n Pr U (s) k C(P ) C(s), s S (n) \ Π (n) n Recall k=1 S (n) \ Π (n) = { s S (n) s P 2 > 1 } n 1/6 Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

61 Far-From-Optimal Types Contribution of information spectrum term on S (n) \ Π (n) is { } 1 n Pr U (s) k C(P ) C(s), s S (n) \ Π (n) n Recall On this set, k=1 S (n) \ Π (n) = { s S (n) s P 2 > 1 } n 1/6 ( ) 1 C(P ) C(s) = Ω n 1/3. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

62 Far-From-Optimal Types Contribution of information spectrum term on S (n) \ Π (n) is { } 1 n Pr U (s) k C(P ) C(s), s S (n) \ Π (n) n Recall k=1 S (n) \ Π (n) = { s S (n) s P 2 > 1 } n 1/6 On this set, ( ) 1 C(P ) C(s) = Ω n 1/3. Thus, by a standard exponential Markov inequality argument { } 1 n max Pr U (s) s S (n) \Π (n) k C(P ) C(s) exp ( O(n 1/3 ) ). n k=1 Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

63 Far-From-Optimal Types Contribution of information spectrum term on S (n) \ Π (n) is { } 1 n Pr U (s) k C(P ) C(s), s S (n) \ Π (n) n Recall k=1 S (n) \ Π (n) = { s S (n) s P 2 > 1 } n 1/6 On this set, ( ) 1 C(P ) C(s) = Ω n 1/3. Thus, by a standard exponential Markov inequality argument { } 1 n max Pr U (s) s S (n) \Π (n) k C(P ) C(s) exp ( O(n 1/3 ) ). n k=1 Polynomially many power types. Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

64 Wrap-Up and Future Work For parallel Gaussian channels with feedback, log M FB (n, ε, P) = nc(p ) + nv(p )Φ 1 (ε) + o( n). Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

65 Wrap-Up and Future Work For parallel Gaussian channels with feedback, log M FB (n, ε, P) = nc(p ) + nv(p )Φ 1 (ε) + o( n). Careful quantization of power allocation vector into power types Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

66 Wrap-Up and Future Work For parallel Gaussian channels with feedback, log MFB (n, ε, P) = nc(p ) + nv(p )Φ 1 (ε) + o( n). Careful quantization of power allocation vector into power types Bounding main information spectrum term using Curtiss theorem Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

67 Wrap-Up and Future Work For parallel Gaussian channels with feedback, log MFB (n, ε, P) = nc(p ) + nv(p )Φ 1 (ε) + o( n). Careful quantization of power allocation vector into power types Bounding main information spectrum term using Curtiss theorem For third-order, need speed of convergence, i.e., sup E[exp(tX n )] E[exp(tY n )] f (n), t R for some f (n) = o(1), do we have sup Pr{X n a} Pr{Y n a} g(n), a R for some desirable g(n)? Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

68 Wrap-Up and Future Work For parallel Gaussian channels with feedback, log MFB (n, ε, P) = nc(p ) + nv(p )Φ 1 (ε) + o( n). Careful quantization of power allocation vector into power types Bounding main information spectrum term using Curtiss theorem For third-order, need speed of convergence, i.e., sup E[exp(tX n )] E[exp(tY n )] f (n), t R for some f (n) = o(1), do we have sup Pr{X n a} Pr{Y n a} g(n), a R for some desirable g(n)? Esséen smoothing lemma? Stein s method? Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

69 Full paper Oct 2017 issue of IEEE Transactions on Information Theory Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22

70 Advertisement: Postdoc Positions in IT and ML at NUS Vincent Tan (NUS) Second-Order Parallel AWGN with Feedback ITW / 22