Fundamental Limits in Asynchronous Communication

Transcription

1 Fundamental Limits in Asynchronous Communication Aslan Tchamkerten Telecom ParisTech V. Chandar D. Tse G. Wornell L. Li G. Caire D.E. Shaw Stanford MIT TPT TU Berlin

2 What is asynchronism? When timing of information transmission is unknown to the receiver.

3 Why important? Because virtually all communication systems are a priori asynchronous.

4 Channel burstiness Steady data generation time x 1 x 2 x 4 x 5 x 6 x 7 x 8 x 3 x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 Asynchronous channel Receiver Example: internet.

5 Channel burstiness: models Insertion-deletion-substitution channel (Dobrushin 1967) W (y x), x 2 X, y 2 Y? Channel coding theorem exists: C = lim n!1 1 n max X n I(Xn ; Y n ) but no known single letter expression, even for the purely deletion channel (Kanoria-Montanari 2013).

6 assumes that positions are known Timing channels (Anantharam-Verdù 1996) (a 1,a 2,...,a n ) queue (d 1,d 2,...,d n ) (x a1,x a2,...,x an ) (y d1,y d2,...,y dn ) W Timing information embedded into a i+1 a i and d i+1 d i. Well understood for exponential service, less so for other service times.

7 this tutorial will focus mostly on one type of asynchronism, here we briefly review other setups Source burstiness Data time Synchronous channel Receiver

8 time Small amounts of data because sources produces small amounts of data limited resources for transmission (energy harvesting systems, Ulukus et al. 2015)

9 This tutorial: source burstiness time long Packets sent once in a long while Fundamental limits (bits/channel use or /joule)? Efficient communication strategies?

10 Roadmap Preliminaries Detection Capacity & capacity per unit cost Receiver sampling constraint Finite length analysis

11 Preliminaries

12 Channel W = {W (y x),x2 X,y 2 Y} DMC X input random variable to channel W Y output random variable to channel W Hence if X P then (X, Y ) PW = P ( )W ( )

13 Notation W a = W ( a) Y a W a D(Y a Y b )=D(W ( a) W ( b)) = X y W (y a) log W (y a) W (y b)

14 Typicality empirical distribution: ˆP x n(a) = 1 n {i : x i = a} typical sequence: x n 2 T (V ) if ˆP x V n 1 = X n if i.i.d. P then Pr(X n 2 T (P )) = 1 o(1) Pr(X n 2 T (V )). =2 nd(v P ) o(1)!(1/ p n) where D(V P )= X x V (x) log V (x) P (x) a n. = bn log a n b n n!1! 0

15 Typicality (cont.) Hence, if (X n,y n ) i.i.d. PW then Pr((X n,y n ) 2 T (J)). =2 nd(j PW) and if (X n,y n ) i.i.d. PW a then Pr((X n,y n ) 2 T (J)). =2 nd(j P,W a) where D(J P, W a )= X x,y J(x, y) log J(x, y) P (x)w (y a)

16 Energy-limited synchronous communication msg m,bbits x n (m) Input cost: x 7! k(x) 2 R + Encoding: Cost for sending m : Channel: x m 7! x n (m)! Y W ( x) k(x n (m)) def = nx i=1 k(x i (m)) Decoding: ˆm(Y n )

17 Energy-limited synchronous communication msg m,bbits x n (m) R = B max m k(x n (m)) bits/joule R achievable if Pr(ˆm 6= m) n!1! 0 C sync. =sup{achievable R}

18 msg m,bbits x n (m) Theorem (Gallager, Verdú, late 80 s) C sync. = max X I(X; Y ) Ek(X) If there exists a zero cost symbol C sync. = max x D(Y x Y 0 ) k(x)

19 Detection

20 Detection x B?,?,...??,?,...? A Single message x B Sent at random time {1, 2,...,A} across W Receiver: Y B W x B over {, +1,, + B 1} and Y W? otherwise.

21 ?,?,...??,?,...? x B A Given Y A+B 1 produce estimate ˆ(Y A+B 1 ) of Largest A for which Pr(ˆ 6= ) B!1! 0? x B Optimal?

22 A natural scheme a, a,..., a Y B A Suppose slotted model, i.e., receiver knows mod B Suppose x B = a, a,..., a Declare ˆ such that is -typical Y ˆ+B 1 ˆ W a

23 Analysis a, a,..., a Y B Ȳ B A P (ˆ 6= ) apple P (miss detection) + P (false-alarm) = P (Y +B 1 /2 T (W a )) + (A/B)P (Ȳ B 2 T (W a )) where is i.i.d. Ȳ B W?

24 Analysis a, a,..., a Y B Ȳ B A P (Y +B 1 /2 T (W a )) = o(1) Pr(Ȳ B 2 T (W a )). =2 BD(Y a Y? ) ) (A/B)P (Ȳ B 2 T (W a )) = o(1) if A =2 B with <D(Y a Y? )

25 Analysis a, a,..., a Y B Ȳ B A =2 B Hence Pr(ˆ 6= ) =o(1) as B!1 if A =2 B with <D(Y a Y? ) largest : o = max a D(Y a Y? ) which corresponds to ā = arg max a D(Y a Y? )

26 All observations??,?,...? a, a,..., a?,?,...? Y B Ȳ B A =2 B Sliding window sequential detection: =min{t 1:Y t+b 1 t 2 T (Wā)} achieves same performance as non-sequential.

27 Non-slotted model? Slotted model guarantees pure hypothesis ā B?,?,...??,?,...? ˆ A =2 B If non-slotted Pr( ˆ 1) = (1)!

28 Non-slotted model??,?,...??,?,...? ā B A =2 B Modify ā B slightly to get low autocorrelation; Hamming distance with any of its shift is (B). ā B =ā x B?,?,...,? ā B i?,?,...,? x B i

29 > o : converse (intuition) a, a,..., a Y B Ȳ B A =2 B Assume x B = a B Reveal mod B Typicality decoding MAP (can be shown) : Pr(Y B 2 T (W a )) = 1 o(1) : Pr(Ȳ B 2 T (W a )). =2 BD(Y a Y? ) Pr(false-alarm). =2 B(D(Y a Y? ) ) Non constant x B, no gain (can be shown).

30 Summary x B?,?,...??,?,...? A =2 B Theorem (Chandar-T-Wornell 2008): Pr(ˆ 6= ) =o(1) as B!1 i A =2 B with Optimal rule: < max a =min{t 1:Y t+b 1 t D(Y a Y? ) 2 T (W x B)}

31 Information transmission

32 Bursty communication model msg m, B bits x n (m) A =2 B when transmission starts when information is available

33 Information transmission msg m, B bits?,?,...? x n (m)?,?,...? A =2 B Y n i.i.d. W? i.i.d. W? Delay constraint (for a meaningful problem): given d we want Pr( apple d) =1 o(1).

34 msg m, B bits?,?,...? x n (m)?,?,...? A =2 B Y n i.i.d. W? i.i.d. W? C(A, d)? How does it compare with? C = max X I(X; Y )

35 Small delay capacity msg m, B bits x n (m) A =2 B Theorem (Chandar-T-Tse 2013): C(,d=2 o(b) ) = max X d min ( )= B min max X2P I(X; Y ) I(X; Y ), D(XY X, Y?) 1+ where P is the set of inputs that achieve C(, = 0).

36 Remark C C( ) C(,d=2 o(b) ) = max X min I(X; Y ), D(XY X, Y?) 1+ ) C(,d=2 o(b) ) apple C

37 Remark (cont.) C C( ) C(,d=2 o(b) ) = max X min c =sup{ : C( )=C} > 0 c I(X; Y ), D(XY X, Y?) 1+ iff (unique) capacity achieving output channel differs from Y? since Ỹ of sync D(XY X, Y? )=I(X; Y )+D(Ỹ Y?) = C + D(Ỹ Y?) {z } >0

38 Small delay capacity: alternative scaling msg m, B bits x n (m) A =2 n A =2 B =2 n, = R Theorem (Chandar-T-Tse 2013): C(,d=2 o(n) ) = max I(X; Y ) X:D(Y Y? ) d min ( ) =n

39 Achievability msg m, B bits x n (m) A =2 B {x n (m)} =min{t i.i.d. P, = 1:9m s.t. (x n (m),y t t n+1) 2 T (PW)}

40 Simplified analysis (slotted) Ỹ n A =2 B Assume m =1 is sent P ((X n (1),Y n ) 2 T (PW)) = 1 P ((X n (2),Y n ) 2 T (PW)). =2 o(1) nd(xy X,Y ) P ((X n (2), Ỹ n ) 2 T (PW)). =2 nd(xy X,Y?)

41 Simplified analysis (slotted) Ỹ n A =2 B Hence Pr(error)! 0 if 2 nr 2 nd(xy X,Y ) = o(1) (A/n)2 nr 2 nd(xy X,Y?). =2 n(d(xy X,Y? ) R(1+ )) i.e. = o(1) R<D(XY X, Y )=I(X; Y ) R(1 + ) <D(XY X, Y? ) # ) C(, = 0) max min I(X; Y ), D(XY X, Y?) X 1+

42 Converse: intuition (small delay, =0) x n (m) d A =2 B Effective output process: pure noise from + d onwards Differs from real output process when + n> + d Reveal effective output process to receiver

43 Converse: intuition ( =0) x n (m) d A =2 B Consider extended codewords: symbols sent from to + d x d (m) x n (m)??? d or x d (m) x n ( ) (m) d + d + n + d< + n

44 Converse: intuition ( =0) x d (m) A =2 B Without essential loss of generality assume constant composition P. {x d (m)} is of From synchronous communication: R = log M d apple I(X; Y ) with X P

45 Converse (cont.) Decoding delay implies the receiver can locate codeword Reveal effective output process to receiver Reveal mod d Ask receiver to detect and isolate sent codeword d A =2 B

46 d A =2 B Typic-decoding is essentially optimal (can be shown).

47 Typic-decoding error probability d Ȳ d A =2 B Assuming m =1 is sent: P ((X d (1),Y d ) 2 T (PW)) = 1 P ((X d (2),Y d ) 2 T (PW)). =2 o(1) dd(xy X,Y ) P ((X d (2), Ȳ d ) 2 T (PW)). =2 dd(xy X,Y?) Union bound over msg: R apple D(XY X, Y )=I(X; Y ) Union bound over msg/slots: R(1 + ) apple D(XY X, Y? )

48 Typic-decoding error probability d Ȳ d A =2 B R apple D(XY X, Y )=I(X; Y ) R(1 + ) apple D(XY X, Y? ), R apple min ) C(, = 0) apple max X I(X; Y ), D(XY X, Y?) 1+ min I(X; Y ), D(XY X, Y?) 1+ #

49 Large delay capacity d =2 B 2 B A =2 B Theorem (Chandar-T-Tse 2013): C(, )=C(, 0) for any > 0 Proof: reduce asynchronism by delaying transmission

50 Energy to transmit asynchronously

51 Capacity per unit cost Theorem (Chandar-T-Tse 2013): C(, = 0) = max X d min ( )= min B max X2P I(X; Y ) I(X; Y ) Ek(X), D(XY X, Y?) (1 + )Ek(X) where P is the set of inputs that achieve C(, = 0). C(, )=C(, 0) 0 Proof: follow capacity arguments.

52 A natural case: k(?) =0 C(, = 0) = 1 1+ max x = 1 1+ C D(Y x Y? ) k(x) Example: Gaussian channel x! x + Z(0,N o /2) k(x) =x 2,? =0 C( )= log 2 e N o (1 + )

53 Alternative proof for C(, = 0) when k(?) =0 m, B bits d A Equivalent scenarios Async. within d m,w ˆm, ˆ Sync. m, W ˆm, ˆ PPM

54 Alternative proof for when k(?) =0 C( ) m, B bits A =2 B Since k(?) =0: cost to transmit B bits asynchronously = cost to transmit log(a/d)+b. = B + B bits synchronously via PPM.

55 Alternative proof for when k(?) =0 C( ) m, B bits Hence, minimum cost to transmit 1 bit asynchronously minimum cost to transmit 1+ bits synchronously, i.e. 1 C( ) (1 + ) 1 C, (1 + )C( ) apple C

56 Alternative proof for when k(?) =0 C( ) m, B bits Converse: optimality of PPM ) (1 + )C( ) C( ) #

57 Energy to transmit and receive asynchronously

58 msg m, B bits x n (m) A =2 B Transmission cost: k(x) Reception cost: sampling rate Motivation: power at receivers scales linearly with.

59 msg m, B bits x n (m) A =2 B Transmitter: x n (m) at time Receiver: sampling, stop, decode = number of until Receiver operates at rate 30% if Pr( apple 30%) = 1 o(1)

60 msg m, B bits x n (m) A =2 B C async. (, )?

61 Theorem (non-adaptive sampling, T-Chandar-Caire 2014): For any and > 0 0 < < 1 C async. (, ) =C async. (, 1) d min (, ) = 1 d min(, 1)

62 Achievability (asynchronism reduction) consider coding scheme under full sampling stretch codewords by 1/, sample each 1/ output Converse Non-adaptive sampling budget A implies delay 1/. (change of measure argument).

63 Theorem (adaptive sampling, T-Chandar-Caire 2014): For any 0 and 0 < apple 1 C async. (, ) =C async. (, 1) d min (, ) =d min (, 1)(1 + o(1)) Minimum input energy, minimum delay, arbitrarily small sampling rate.

64 Achievability Transmitter: random coding, = Receiver: decoding function stopping rule sampling strategy

65 Two-phase scheme Sparse sampling: at multiples of 1/ compute empirical distribution ˆP of the last log(n) samples. If P? (T ( ˆP )) apple " switch to full sampling for period n. At the end of the full sampling period, typic-decode or return to sparse mode if no codeword is found.

66 Theorem (adaptive sampling, T-Chandar-Caire 2014): For any 0 and 0 < apple 1 C async. (, ) =C async. (, 1) d min (, ) =d min (, 1)(1 + o(1)) No loss for constant sampling rate but what if! 0?

67 Theorem (min,min,min Chandar-T 2016): For any 0 and 0 < apple 1 if =!(1/B) C async. (, ) =C async. (, 1) d min (, ) =d min (, 1)(1 + o(1)) if = o(1/b) unreliable communication

68 H? 0 H msg Multi-phase H? 1 H msg 0 < 1 <...< ` =1... H? H msg skip samples repeat ` H? H msg stop and decode

69 skip skip H? H? N 0 N 1 N 1 H msg N 0 H msg At each s-instant {t = j,j 2 N} test N 0,N 1,...,N` samples if a test skip samples until next s-instant, repeat! H? if ` consecutive test! Hmsg, decode

70 skip skip H? H? N 0 N 1 N 1 H msg N 0 H msg Parameters can be chosen such that =!(1/B)

71 Theorem (min,min,min Chandar-T 2016): For any 0 and 0 < apple 1 if =!(1/B) C async. (, ) =C async. (, 1) d min (, ) =d min (, 1)(1 + o(1)) if = o(1/b) unreliable communication Artefacts of asymptotic analysis?

72 Finite length analysis ( k(x) =1)

73 Synchronous communication x n (m) W y n Pr(ˆm 6= m) apple " Theorem (Strassen 1962, Polyanskiy et al. 2010) B (n, ") =nc p nv " (W )Q 1 (")+O(log n) largest message size dispersion, gap to capacity

74 msg m, B bits x n (m) d A =2 n asynchronism? sampling rate? dispersion delay d?

75 x n (m) A =2 n Recall C( ) = max I(X; Y ) X:D(Y Y? ) d min ( ) =n Definition: V " (W, ) dispersion of synchronous channel under input constraint W V " (W ) V " (W, ) D(Y Y?). c

76 Minimum delay: d n = d min = n x n (m) A =2 n Theorem (full sampling, Polyanskiy 2013, Li-T. 2017) B (n, ",, n = 1) = nc( ) p nv" (W, )Q 1 (")+O(log n) Theorem (sparse sampling, Li-T. 2017) B (n, ",,!(1/ p n)) = nc( ) p nv" (W, )Q 1 (")+o( p n) B (n, ",,!(1/n)) =nc( ) (1/ n ) + O( p n) & O(1/ p n)

77 Close to minimum delay: d n = n(1 + o(1)) x n (m) A =2 n Theorem (sparse sampling, Li-T. 2017) B (n, ",,!(1/n)) = nc( ) p nv" (W, )Q 1 (")+O(log n) minimum sampling rate minimum delay & & d = n(1 + o(1)) n =!(1/ p dispersion n)

78 Intuition: d n = d min = n x n (m) A =2 n B (n, ",,!(1/n)) =nc( ) (1/ n ) + O( p n) At sampling rate we miss (1/ ) symbols of the sent codeword. So if = o(1/ p n) decoder will miss!( p n) of the sent codeword. Codes of length n!( p n) have!( p n) order synchronous dispersion.

79 Summary msg m, B bits x n (m) A Capacity, capacity per unit cost Tradeoffs between rate, delay, sampling rate Finite length analysis

80 Open issues Bit asynchronism: capacity per unit cost? Random/multiple access: bursty interference (Chandar-T.-Caire 2014, Shahi et. al. 2016, Farkas- Kói 2016) Asynchronous networks (Shomorony et al. 2014, Gallager 1976)

81 Bibliography 1. Dobrushin, Roland L'vovich. "Shannon's theorems for channels with synchronization errors." Problemy Peredachi Informatsii 3.4 (1967): Kanoria, Yashodhan, and Andrea Montanari. "Optimal coding for the binary deletion channel with small deletion probability." IEEE Transactions on Information Theory (2013): Anantharam, Venkat, and Sergio Verdu. "Bits through queues." IEEE Transactions on Information Theory 42.1 (1996): Ulukus, Sennur, et al. "Energy harvesting wireless communications: A review of recent advances." IEEE Journal on Selected Areas in Communications 33.3 (2015): Verdu, Sergio. "On channel capacity per unit cost." IEEE Transactions on Information Theory 36.5 (1990): LIDS-P-1714, November Gallager, R. G., "Energy Limited Channels: Coding, Multiaccess, and Spread Spectrum", M.I.T., Laboratory for Information and Decision Systems, Report 7. Chandar, Venkat, Aslan Tchamkerten, and Gregory Wornell. "Optimal sequential frame synchronization." IEEE Transactions on Information Theory 54.8 (2008): Chandar, Venkat, Aslan Tchamkerten, and David Tse. "Asynchronous capacity per unit cost." IEEE Transactions on Information Theory 59.3 (2013): Tchamkerten, Aslan, Venkat Chandar, and Giuseppe Caire. "Energy and sampling constrained asynchronous communication." IEEE Transactions on Information Theory (2014): Chandar, Venkat, and Aslan Tchamkerten. "How to Quickly Detect a Change while Sleeping Almost all the Time." arxiv preprint arxiv: (2015). 11. Strassen, Volker. "Asymptotische abschätzungen in Shannons informationstheorie." Trans. Third Prague Conf. Inf. Theory Polyanskiy, Yury, H. Vincent Poor, and Sergio Verdú. "Channel coding rate in the finite blocklength regime." IEEE Transactions on Information Theory 56.5 (2010): Polyanskiy, Yury. "Asynchronous communication: Exact synchronization, universality, and dispersion." IEEE Transactions on Information Theory 59.3 (2013):

82 Bibliography 14. Li Longguang and Aslan Tchamkerten. Infinite dispersion in bursty communication." ISIT Shahi, Sara, Daniela Tuninetti, and Natasha Devroye. "On the capacity of strong asynchronous multiple access channels with a large number of users." Information Theory (ISIT), 2016 IEEE International Symposium on. IEEE, Farkas Lóránt, Tamás Kói. "Universal Random Access Error Exponents for Codebooks of Different Word-Lengths." arxiv preprint arxiv: (2016). 17. Shomorony, Ilan, et al. "Diamond networks with bursty traffic: Bounds on the minimum energy-perbit." IEEE Transactions on Information Theory 60.1 (2014): Gallager, Robert. "Basic limits on protocol information in data communication networks." IEEE Transactions on Information Theory 22.4 (1976):

83

84

85 Typical scenarios

86 msg m, B bits c n (m) A =2 B d =2 B C async. (,, )?

87 msg m, B bits c n (m) A =2 B C async. (, )?

88

89 Synchronous communication time Voice, video Packets sent contiguously Negligible cost to acquire synchronization

90

91

92

93 How does asynchronism impact capacity per unit cost?

94 Bursty communication model msg m, B bits?,?,...? x n (m)?,?,...? idle idle A Y n i.i.d. W (?) i.i.d. W (?) noise noise : stopping time with respect to Y 1,Y 2,...

95 Bursty communication model msg m, B bits x n (m) time A A:level of asynchronism :transmission start time One message sent over the [1, A]

96 msg m, B bits x n (m) A Given A Find C async. (A) =sup{achievable R}

97 msg m, B bits A natural scheme x n (m) A Random code {x n (m)} i.i.d. P Sequential typicality: =inf{t 1:(x n (m),y t t n+1) PW-typical for some m}

98 Analysis msg m, B bits A Random code {x n (m)} i.i.d. P Sequential typicality: =inf{t 1:(x n (m),y t t n+1) PW-typical for some m}

99 msg m, B bits c n (m) A C async. (A, )?

100 Bursty communication model msg m, B bits c n (m) time A

101 Bursty communication model msg m, B bits?,?,...??,?,...? c n (m) idle Y n idle A i.i.d. W (?) i.i.d. W (?)

102 msg m, B bits c n (m) A Input cost Output cost: sampling rate = number of until

103 msg m, B bits c n (m) A C async. (A, )?

104 =1 msg m, B bits c n (m) A =2 B Theorem (Chandar, T., Tse 2010): C async. (A, = 1) = max X min I(X; Y ) Ek(X), I(X; Y )+D(Y Y?) (1 + )Ek(X) Minimum delay d min (, = 1)(= (B))

105 0 < apple 1 msg m, B bits c n (m) A =2 B Theorem (Chandar, T., Caire 2013): for any > 0 C async. (, ) =C async. (, 1) d min (, ) d min (, = 1)

106 msg m, B bits c n (m) A =2 B No loss for constant sampling rate.! 0?

107 B bits Main result A =2 B Theorem: for any > 0 if =!(1/B) C async. (A, ) =C async. (A, 1) d min (, ) d min (, = 1) if = o(1/b) unreliable communication

108 Achievability Transmitter: random coding Receiver: decoding function stopping rule sampling strategy Message detection with =!(1/B) samples?

109 H? 0 H msg Multi-zoom H? 1 H msg 0 < 1 <...< ` =1... H? H msg skip samples repeat ` H? H msg stop and decode

110 skip skip H? H? N 0 N 1 N 1 H msg N 0 H msg At each s-instant {t = j,j 2 N} test N 0,N 1,...,N` samples if a test skip samples until s-instant, repeat! H? if ` consecutive test! H msg, decode

111 skip skip H? H? N 0 N 1 N 1 H msg N 0 H msg Parameters can be chosen such that =!(1/B)

112 Summary Fundamental tradeoffs for bursty communication To combat asynchronism we only need strong signals; better output sampling does not help Decoder can sleep almost all the time and yet be maximally efficient (asympt.).

113 skip skip H? H? N 0 N 1 N 1 H msg N 0 H msg Design tests such that first phase dominates: ) #{noise samples at s-instant} N 0 N0 =!(1) )!(1) But = o(b) otherwise change is missed. )!(1/B)