A New Metaconverse and Outer Region for Finite-Blocklength MACs

Transcription

1 A New Metaconverse Outer Region for Finite-Blocklength MACs Pierre Moulin Dept of Electrical Computer Engineering University of Illinois at Urbana-Champaign Urbana, IL 680 Abstract An outer rate region for the Discrete Memoryless Multiple Access Channel is given, is shown to coincide with a recently derived inner rate region within a o(/ n term except in the vicinity of the corner points of the Ahlswede-Liao pentagonal region, where the gap is O(/ n. The derivation is based on a new metaconverse under the maximum error probability criterion on strong large deviations for Neyman- Pearson tests. I. INTRODUCTION The study of the fundamental communication limits in the finite-blocklength regime has received tremendous attention in the information theory community, following the works by Strassen ], Polyanskiy et al. 2], Hayashi 3]. Extension of the techniques therein to multiuser settings are quite challenging 4], 5], 6], 7], 8], 9], 0]. In this paper we derive a new outer region for multiple-access channels which essentially matches the inner region of 7]. More precisely, we obtain the same second-order coding rate as in 7], except in the vicinity of the corner points of the Ahlswede-Liao pentagonal region. To obtain the outer region, we derive a strong converse under the maximum error probability criterion. A strong converse for the Ahlswede-Liao region was given by Ahlswede ] under the average error probability criterion. However Ahlswede s method requires the use of sophisticated wringing techniques which seem difficult to extend for a more refined asymptotic analysis. It should be noted that while the outer regions for the DM-MAC under the maximum average error probability criteria do not coincide in general for deterministic codes 2], they do coincide if romization of the code is allowed. As in ], the primary technical difficulty is to manage the pairs of codewords that result in atypically good decoding performance. To this end, we derive a new metaconverse, which is inspired by Strassen s approach for deterministic codes on singleuser channels ]. The problem is stated as follows. Consider a discrete memoryless multiple access channel (DM- MAC with two input alphabets X X 2, output alphabet Y, channel law W (y x, x 2, x X, x 2 X 2, y Y. We will often denote by W xx 2 the conditional distribution W ( x, x 2 over Y. Let M {, 2,, M } M 2 {, 2,, M 2 }. A (n, M, M 2 MAC code C consists of two codebooks {x (m, m M } {x 2 (m 2, m 2 M 2 } a (possibly romized decoder φ : Y n M M 2. The decoding error probability for message pair (m, m 2 is e(m, m 2 ; C y Y n W n (y x (m, x 2 (m 2 {φ(y (m, m 2 }. The average error probability for the code is e avg (C M M 2 m,m 2 e(m, m 2 ; C. The maximum error probability for the code is e max (C max m,m 2 e(m, m 2 ; C. A rate pair (R, R 2 is (n, ɛ achievable if there exists a (n, M, M 2 MAC code with maximum error probability ɛ. Given a joint distribution P UXX 2Y In particular, romized permutation of codeword assignments equalizes the decoding error probability for all possible message pairs.

2 P U P X U P X2 U W, define the rom vector W (Y XX2 L ln P (Y X 2U L L 2 W (Y XX2 ln P (Y X U. ( L 3 ln I I 2 I 3 L L 2 L 3 W (Y XX2 P (Y U The expectation of L is I(X ; Y X 2 U I E I(X 2 ; Y X U (2 I(X X 2 ; Y U its conditional covariance matrix given U is V CovL U]. (3 have unique solutions. The following holds for every (n, M, M 2 MAC code with maximum error probability ɛ. There exists a joint probability distribution P U P X U P X2 U W such that U 3 log M ni(x ; Y X 2, U nv (X ; Y X 2, U Q (ɛ + o( n log M 2 ni(x 2 ; Y X, U nv (X 2 ; Y X, U Q (ɛ + o( n log(m M 2 ni(x X 2 ; Y U nv (X X 2 ; Y U Q (ɛ + o( n. The diagonal elements of V are the conditional variances V V (X ; Y X 2 U Var ln W (Y X ] X 2 P (Y X 2 U U V 2 V (X 2 ; Y X U Var ln W (Y X ] X 2 P (Y X U U V 3 V (X X 2 ; Y U Var ln W (Y X ] X 2 P (Y U U. (4 Let Z be a three-dimensional Gaussian vector with mean zero covariance matrix V. Define the region Q inv (V, ɛ {z R 3 : P (Z z ɛ} where the inequality Z z holds componentwise. The following result was given in 7]. Theorem.: For any joint probability distribution P U P X U P X2 U W, the vectors log M log M 2 ni nq inv (V, ɛ + O(log n log(m M 2 multiplied by n, are (n, ɛ achievable rate vectors under the average error probability criterion. Our main result in this paper is as follows. Theorem.2: Assume the maximization problems max P X P X2 I(X ; Y X 2 I max P X P X2 I(X 2 ; Y X I 2 max P X P X2 I(X ; X 2 ; Y I 3 By the properties of the function Q inv (V, ɛ, the inner region of Theorem. the outer region of Theorem.2 agree up to o(/ n except in the vicinity of the corners of the Ahlswede-Liao pentagonal region, where the gap is O(/ n. II. META CONVERSE Given two probability distributions P Q for a rom variable Z Z, denote by δ : Z 0, ] a romized decision rule returning δ(z PrSay P Z z], z Z, by β ɛ (P, Q the type-ii error probability of the Neyman-Pearson test at significance level ɛ, i.e. β ɛ (P, Q min E Qδ(Z]. δ : E P δ(z] ɛ Proposition 2.: For any (n, M, M 2 MAC code with maximum error probability ɛ over channel W n, the following holds for any subset Ω of M M 2 any distribution Q Y over Y n : β ɛ (W n ( x (m, x 2 (m 2, Q Y. (5 In particular, for any three distributions Q Y, Q Y X2,

3 Q Y X we have M 2 M β ɛ (W n ( x (m, x 2 (m 2, Q Y (6 β ɛ (W n ( x (m, x 2 (m 2, Q Y X2x 2(m 2 (7 β ɛ (W n ( x (m, x 2 (m 2, Q Y Xx (m. (8 This strengthens the metaconverse of 7, Theorem 5], which is based on the single-user metaconverse of 2] states min β ɛ(w n ( x (m, x 2 (m 2, Q Y in particular, (9 M M 2 min m,m 2 β ɛ (W n ( x (m, x 2 (m 2, Q Y (0 min β ɛ (W n ( x (m, x 2 (m 2, M m,m 2 Q Y X2x 2(m 2 ( min β ɛ (W n ( x (m, x 2 (m 2, M 2 m,m 2 Q Y Xx (m (2 but is insufficient to derive the results claimed in this paper. The disadvantage of (0 (2 is that there may exist a few codeword pairs for which β ɛ is unusually low; to eliminate those, one typically seeks a large subset Ω of the joint message set that excludes them. As we shall see in Sec. III-A, this is sufficient to get a strong converse a second-order coding rate, but not the best second-order coding rate. The advantage of the new bound (6 (8 is that the pairs of concern contribute weakly to the sum that no joint message set pruning technique is needed. Proof of Prop. 2.. The proof is based on Strassen s method for the converse, Sec. 4] is easiest to follow in the case of deterministic encoding decoding rules. In that case, the space Y n is partitioned into decoding regions D(m, m 2 such that W n (D(m, m 2 x (m, x 2 (m 2 ɛ for each (m, m 2. For any Ω M M 2 any distribution Q Y over Y n, we therefore have (a (b Q Y (D(m, m 2 min D Y n : W n (D x (m,x 2(m 2 ɛ Q Y (D β ɛ (W n ( x (m, x 2 (m 2, Q Y where (a holds because the decoding regions are disjoint, (b holds by definition of the function β ɛ. This proves (5, which reduces to (6 when Ω M M 2. Similarly, let M (m 2 {m : (m, m 2 Ω}. For any distribution Q Y X2, we have, for each m 2, m M (m 2 Q Y X2x 2(m 2(D(m, m 2 min D Y n : W n (D x (m,x 2(m 2 ɛ m M (m 2 m M (m 2 Q Y X2x 2(m 2(D(m, m 2 β ɛ (W n ( x (m, x 2 (m 2, Q Y X2x 2(m 2. Summing these inequalities over m 2 proves (7. The final inequality (8 is proved the same way. The claim holds when the encoders decoder are stochastic mappings P X m, P X2 m 2, P ˆM ˆM2 Y, respectively. To derive (6 in that case, let W(y m, m 2 x,x 2 W n (y x x 2 P X m (x P X2 m 2 (x 2. For each (m, m 2 we have P ˆM ˆM2 Y (m, m 2 yw(y m, m 2 ɛ. y

4 For any (m, m 2 any distribution Q Y we have P ˆM ˆM2 Y (m, m 2 yq Y (y y min P ˆM ˆM2 Y : y P ˆM ˆM2 Y (m,m2 yw(y m,m2 ɛ P ˆM ˆM2 Y (m, m 2 yq Y (y y β ɛ (W( m, m 2, Q Y. Summing over all (m, m 2 Ω we obtain P ˆM ˆM2 Y (m, m 2 yq Y (y y β ɛ (W( m, m 2, Q Y similarly for any conditional distributions Q Y m2 Q Y m, β ɛ (W( m, m 2, Q Y m2, β ɛ (W( m, m 2, Q Y m. III. EARLY ATTEMPTS Before sketching the proof of Theorem.2, we present two instructive attempts at deriving a strong converse a second-order coding rate. The first is based on the metaconverse of 7, Theorem 5], the second on the new metaconverse of Prop 2.. A. First Metaconverse The following technique was presented in 8] yields an O(/ n backoff from the Ahlswede-Liao capacity region, albeit not with the desired secondorder coding rate. Define n pairs of rom variables (X i, X 2i, i n that are equal to (x i (m, x 2i (m 2 where (m, m 2 is drawn uniformly from M,n M 2,n. The joint distribution of (X i, X 2i, Y i is given by P XiX 2iY i (x, x 2, y P Xi (x P X2i (x 2 W (y x, x 2, i n. (3 n Let I i I(X i; Y i X 2i, I 2 n i I(X 2i; Y i X i, I 3 n i I(X i, X 2i ; Y i, Q Y n i P Y i D(m, m 2 n D(W xi(m n,x 2i(m 2 P Yi, V (m, m 2 n i n V (W xi(m,x 2i(m 2 P Yi. i The following asymptotics hold for the NP test: log β ɛ (W n ( x, x 2, Q Y nd(m, m 2 nv (m, m 2 Q (ɛ + log n + O(. (4 2 We have D(m, m 2 I 3 M M 2 m,m 2 hence by Markov s inequality, there exists a set Ω M M 2 such that Then log max D(m, m 2 < I 3 + n > M M 2 ni 3 +. (5 max log β ɛ(w n ( x (m, x 2 (m 2, max {nd(m, m 2 } nv (m, m 2 Q (ɛ + log n + O( 2 ni 3 nv min,3 Q (ɛ + O(log n hence Q Y log(m M 2 ni 3 nv min,3 Q (ɛ + O(log n. The same technique can be used to derive the bounds log M ni nv min, Q (ɛ + O(log n log M 2 ni 2 nv min,2 Q (ɛ + O(log n. By taking the supremum with respect to all possible distributions P Xi, P X2i, i n, we obtain a

5 strong converse for the DM-MAC under the maximum error probability criterion, with a O(/ n backoff from the Ahlswede-Liao bound on the sum-rate. Hence the codeword pairs (m, m 2 that have atypically large D(m, m 2 can be managed by excluding them from the set Ω. It would be nice to use the same technique again to find a subset Ω Ω from which the codeword pairs (m, m 2 that have atypically small V (m, m 2 are excluded as well. Unfortunately such a set might not be sufficiently large, in fact might be empty. B. New Metaconverse Lemma 3.: Fix A > 0 n > 0. The function f(t, u e nt+a nu, t, u 0, is convex in t for all t > 0 convex in u for all u > A 2 n but is not jointly convex in (t, u. Proof: The partial first derivatives of f are f(t,u t nf(t, u f(t,u u A n 2 f(t, u its Hessian u is ( A 2 2 nu f(t, u ( A 2 A 2 nu 4nu n 2 f(t, u. A nu The partial second derivatives of f with respect to t u are positive over their range, hence f is convex in t u separately. The determinant of the Hessian equals An5/2 f 2 (t, u < 0, hence f is not jointly 4u 3/2 convex in (t, u. Remark: As n, the (unnormalized eigenvectors of the Hessian are asymptotic to (, A 2 nu A (, are associated with the positive 2 nu negative eigenvalues + A2 4un + o(n ]n 2 f(t, u A2 + o(n 3/2 ]n 2 f(t, u, respectively. 4n 3/2 u 2 Lemma 3.2: Let f be defined as in Lemma 3. E P f(t, U be a function of a probability distribution P, to be minimized subject to the mean-value constraints E P (T D E P (U V the domain constraint (t, u R + 0, u max ]. The minimum is equal to f e nd+av n/u max achieved by P that has support at the points ( (t, u D (t 2, u 2 AV numax, 0 ( D + A ( umax V, u max, n umax (6 the probability of the first point being V u max. Proof. By the fundamental theorem of linear programming, the minimum is achieved by P that has support at three points (t j, u j, j, 2, 3 at most, with respective probabilities α j, j, 2, 3. It may be verified that (6 satisfies the first- secondorder Kuhn-Tucker conditions for a minimum, with f(t, u f(t 2, u 2 f. Lemma 3.2 suggests a second attempt at deriving a second-order coding rate. Using the metaconverse of Prop. 2. the asymptotics of the NP test (4, we obtain M M 2 M M 2 m,m 2 exp {nd(m, m 2 nv (m, m 2 Q (ɛ + 2 log n + O( }. Applying Lemma 3.2 yields a strong converse again, but still not the desired second-order coding rate. IV. SKETCH OF PROOF OF THEOREM.2 The desired coding rate can be obtained by working with rom variables that are defined over blocks of samples instead of single samples as done above. Specifically, we partition the codewords into blocks of length k log 2 n consider the output distribution over such blocks. The auxiliary distribution over Y n for the NP test is blockwise memoryless with block size k. To prove Theorem.2, we will apply Prop. 2. three times use the asymptotics of the corresponding NP tests. Step. Let k log 2 n partition the set {,, n} into blocks B(j, j n k of size k. Given any sequence x X n, denote by x(j X k its restriction to block B(j, i.e., x(j {x i, i B(j}. We denote by x (j, m X k the subsequence associated with codeword x (m, likewise x 2 (j, m 2 X2 k the subsequence associated with codeword x 2 (m 2. Define the pairs of rom variables (X (j, X 2 (j, j n k that take values (x (j, m, x 2 (j, m 2 where (m, m 2 is drawn uniformly from M,n M 2,n. Hence X (j X 2 (j are independent for each j. Let Y(j be the restriction

6 to B(j of the rom sequence Y at the channel output, i.e., Y(j {Y i, i B(j}. Hence P Y(j (y(j M M 2 m,m 2 W k (y(j x (j, m, x 2 (j, m 2. The joint distribution of (X (j, X 2 (j, Y(j is given by P X(j,X 2(j,Y(j(x (j, x 2 (j, y(j P X(j(x (j P X2(j(x 2 (j W k (y(j x (j, x 2 (j. Fix the following three product distributions: Q Y P Y(j, Q Y X2 ij P Y(j X2(j, Q Y X P Y(j X(j.(7 Step 2. Asymptotics of NP tests. For Q Y defined in (7, we have log β ɛ (W n ( x, x 2, Q Y Similarly D(W x(j,x 2(j P Y(j k/n V (W x(j,x 2(j P Y(j Q (ɛ + 2 log n + O( x X n, x 2 X n 2.(8 log β ɛ (W n ( x, x 2, Q Y X2x 2 D(W x(j,x 2(j P Y(j X2(jx 2(j k/n V (W x(j,x 2(j P Y(j X2(jx 2(j Q (ɛ + 2 log n + O( x X n, x 2 X n 2 (9 log β ɛ (W n ( x, x 2, Q Y Xx D(W x(j,x 2(j P Y(j X(jx (j k/n V (W x(j,x 2(j P Y(j X(jx (j Q (ɛ + 2 log n + O( x X n, x 2 X n 2. (20 Step 3. Let d j (m, m 2 D(W x(j,m,x 2(j,m 2 P Y(j (2 v j (m, m 2 V (W x(j,m,x 2(j,m 2 P Y(j (22 I j I(X (j, X (j; Y(j d j (m, m 2 (23 M M 2 m,m 2 V j V (X (j, X (j; Y(j v j (m, m 2 (24 M M 2 m,m 2 I3 sup I(X, X 2 ; Y P X P X2 achieved at unique (P X, P X 2 (25 V3 V (X, X 2 ; Y where X PX X 2 PX 2 (26 define the set of block indices { J j {, 2,, n k } : k I j I3 } log 3. n (27 By definition of (PX, PX 2, we have j J ( k V j > V3 O. (28 log 3/2 n For j J, we have ki3 I j ki3 log n thus D(P X(jP X2(j (PX k (PX 2 k 0 as k. For any such block j J, the rom variable V j V (W x(j,m,x 2(j,m 2 P Y(j v j (m, m 2, multiplied by k, converges to an average of k iid rom variables therefore converges in probability

7 to its expectation k V j as k. Moreover P V j < V j k ] log k The probability of the event { ( } k exp O log 2 k j J. E V j < V j k J log k j J j J is upper-bounded by (29 P E] P j J : V j < V j k ] log k J max P V j < V j k ] j J log k { ( } k J exp O log 2 k { ( log 2 } n exp O log 2 (30 log n which vanishes as n. Recalling (29, let Ω (m, m 2 : v j (m, m 2 V j k J log k j J j J (3 where (30 implies that M M 2 as n. Step 4. We have (a β ɛ (W n ( x (m, x 2 (m 2, Q Y (b exp d j (m, m 2 + k/n v j (m, m 2 Q (ɛ log n + O( 2 (c exp d j (m, m 2 + V j k J log k Q (ɛ log n + O( 2 j J (d exp d j (m, m 2 + V j k J log k Q (ɛ log n + O( 2 j J (e exp I j + V j k J log k Q (ɛ j J } 2 log n + O( (f (g (h exp j J + j J ( I j J c k I3 log 3 n V j k J log k Q (ɛ 2 { exp ni3 + k J c log 3 n ( ( + k J V3 O log 3/2 n Q (ɛ } 2 log n + O( min J exp + { ni 3 + k J c log 3 n (k J V 3 Q (ɛ + o(] log n + O( log k { exp ni3 + nv3 Q (ɛ + o( } n }

8 where (a follows from (5, (b from (8 (2 (22, (c from definition of the set Ω in (3, (d from the convexity of the exp( function, (e from (23, (f from the definition of J in (27, (g from (28 the fact that I j ki3, (h holds because the minimum over J is achieved by the largest possible J, namely, J {,, n k }. Since M M 2 as n, we obtain log(m M 2 ni3 nv3 (ɛ + o( n sup ni(x X 2 ; Y P X P X2 nv (X X 2 ; Y Q (ɛ] +o( n. (32 Step 4. Similarly to (32, we obtain log M sup P X P X2 ni(x ; Y X 2 nv (X ; Y X 2 Q (ɛ] +o( n (33 log M 2 sup P X P X2 ni(x 2 ; Y X nv (X 2 ; Y X Q (ɛ] +o( n. (34 Step 5. Choosing U {, 2, 3} defining the three pairs (P X Uu, P X2 Uu, u, 2, 3 as those that achieve the supremum in (33, (34, (32 respectively, proves the claim. V. EXAMPLE Consider the binary erasure MAC channel with input alphabets X X 2 { {0, }, output alphabet X + X Y {0,, 2, e} Y 2 wp λ e wp λ (erasure probability λ. No time-sharing is needed; the optimal input distributions PX PX 2 are uniform, { λ PY (PX PX 2 W, λ, λ }, λ The likelihood ratio vectors over Y are given by W 00 PY W 0 P Y {4, 0, 0, }, W 0 P Y {0, 0, 2, }, W P Y {0, 2, 0, }, {0, 0, 4, }. The mean variance of the loglikelihood ratios are given by D(W 00 P Y D(W P Y 2( λ D(W 0 P Y D(W 0 P Y λ (35 V (W 00 P Y V (W P Y 4λ( λ V (W 0 P Y V (W 0 P Y λ( λ (36 hence I3 I(PX PX 2 ; W 3 2 ( λ V3 V (PX PX 2 ; W 5 2λ( λ. Note from (35 that the most informative input pairs are (x, x 2 (0, 0 (,, however (36 shows that the information variance is also largest in this case. Acknowledgements. This work was supported by NSF under grant CCF The author thanks Yen-Wei Huang for valuable comments about an earlier version of this paper. REFERENCES ] V. Strassen, Asymptotische Abschätzungen in Shannon s Informationstheorie, Trans. Third Prague Conf. Information Theory, 962. English translation by M. Luthy available from pmlut/strassen.pdf. 2] Y. Polyanskij, H. V. Poor S. Verdú, Channel Coding Rate in the Finite Blocklength Regime, IEEE Trans. Information Theory, Vol. 56, No. 5, pp , May ] M. Hayashi, Information Spectrum Approach to Second-Order Coding Rate in Channel Coding, IEEE Trans. Information Theory, Vol. 55, No., pp , Nov ] P. Moulin, Finite-Blocklength Universal Coding for Multiple- Access Channels, presented at DARPA ITMANET workshop, Stanford, CA, Jan ] V. Y. F. Tan O. Kosut, On the Dispersions of Three Network Information Theory Problems, Feb ] E. MolavianJazi J. N. Laneman, Multiaccess Communication in the Finite Blocklength regime, presented in ITA Workshop, San Diego, CA, Feb ] Y. -W. Huang P. Moulin, Finite Blocklength Coding for Multiple Access Channels, ISIT, Cambridge, MA, July ] P. Moulin, Finite Blocklength Capacity Region for Multiple Access Channels, presented at Network Science Workshop, Chinese U. of Hong Kong, July ] E. MolavianJazi J. N. Laneman, A Rom Coding Approach to Gaussian Multiple Access Channels with Finite Blocklength, presented at 50th Allerton Conf., Monticello, IL, Oct ] S. Verdú, Non-Asymptotic Achievability Bounds in Multiuser Information Theory, presented at 50th Allerton Conf., Monticello, IL, Oct ] R. Ahlswede, An Elementary Proof of the Strong Converse Theorem for the Multiple-Access Channel, J. Combinatorics, Information System Sci., Vol. 7, No. 3, pp , ] G. Dueck, Maximal Error Capacity Regions are Smaller than Average Error Capacity Regions for Multi-user Channels, Probl. Control Information Theory, Vol. 7, No., pp. 9, 978.