A General Formula for Compound Channel Capacity

Transcription

1 A General Formula for Compound Channel Capacity Sergey Loyka, Charalambos D. Charalambous University of Ottawa, University of Cyprus ETH Zurich (May 2015), ISIT-15 1/32

2 Outline 1 Introduction 2 Channel Model 3 Definitions 4 Capacity for a given channel state 5 General Formula for Compound Channel Capacity 6 Uniform Compound Channel 7 Examples & mistakes 8 Conclusion 2/32

3 Introduction Impact of channel state information (CSI) on capacity/system design Real world: measurement/modeling uncertainty Wireless channels dynamic nature estimation error feedback limitations Uncertain CSI: deterministic vs. stochastic models 3/32

4 Introduction Impact of channel state information (CSI) on capacity/system design Real world: measurement/modeling uncertainty Wireless channels dynamic nature estimation error feedback limitations Uncertain CSI: deterministic vs. stochastic models 4/32

5 Introduction Impact of channel state information (CSI) on capacity/system design Real world: measurement/modeling uncertainty Wireless channels dynamic nature estimation error feedback limitations Uncertain CSI: deterministic vs. stochastic models 5/32

6 Introduction Impact of channel state information (CSI) on capacity/system design Real world: measurement/modeling uncertainty Wireless channels dynamic nature estimation error feedback limitations Uncertain CSI: deterministic vs. stochastic models 6/32

7 Introduction Impact of channel uncertainty: extensive studies since 1950s 12 Several approaches compound channel model mixed/composite channel arbitrary varying channel CDI instead of CSI 1 A. Lapidoth and P. Narayan, Reliable Communication Under Channel Uncertainty, IEEE Trans. Inform. Theory, vol. 44, No. 6, Oct E. Biglieri, J. Proakis, and S. Shamai, Fading Channels: Information-Theoretic and Communications Aspects, IEEE Trans. Inform. Theory, vol. 44, No. 6, Oct /32

8 Introduction Impact of channel uncertainty: extensive studies since 1950s 12 Several approaches compound channel model mixed/composite channel arbitrary varying channel CDI instead of CSI 1 A. Lapidoth and P. Narayan, Reliable Communication Under Channel Uncertainty, IEEE Trans. Inform. Theory, vol. 44, No. 6, Oct E. Biglieri, J. Proakis, and S. Shamai, Fading Channels: Information-Theoretic and Communications Aspects, IEEE Trans. Inform. Theory, vol. 44, No. 6, Oct /32

9 Introduction Compound channel/capacity Prior work: information-stable channels (e.g. stationary ergodic) Many/all channels are non-stationary, non-ergodic modulation-induced channels quasi-static fading channels how to measure ergodicity??? This work: information-unstable channels no ergodicity/stationarity is required via information density/spectrum 34 3 S. Verdu, T.S. Han, A General Formula for Channel Capacity, IEEE Trans. Info. Theory, vol. 40, no. 4, July T. S. Han, Information-Spectrum Method in Information Theory, New York: Springer, /32

10 Introduction Compound channel/capacity Prior work: information-stable channels (e.g. stationary ergodic) Many/all channels are non-stationary, non-ergodic modulation-induced channels quasi-static fading channels how to measure ergodicity??? This work: information-unstable channels no ergodicity/stationarity is required via information density/spectrum 34 3 S. Verdu, T.S. Han, A General Formula for Channel Capacity, IEEE Trans. Info. Theory, vol. 40, no. 4, July T. S. Han, Information-Spectrum Method in Information Theory, New York: Springer, /32

11 Introduction Compound channel/capacity Prior work: information-stable channels (e.g. stationary ergodic) Many/all channels are non-stationary, non-ergodic modulation-induced channels quasi-static fading channels how to measure ergodicity??? This work: information-unstable channels no ergodicity/stationarity is required via information density/spectrum 34 3 S. Verdu, T.S. Han, A General Formula for Channel Capacity, IEEE Trans. Info. Theory, vol. 40, no. 4, July T. S. Han, Information-Spectrum Method in Information Theory, New York: Springer, /32

12 Introduction Compound channel/capacity Prior work: information-stable channels (e.g. stationary ergodic) Many/all channels are non-stationary, non-ergodic modulation-induced channels quasi-static fading channels how to measure ergodicity??? This work: information-unstable channels no ergodicity/stationarity is required via information density/spectrum 34 3 S. Verdu, T.S. Han, A General Formula for Channel Capacity, IEEE Trans. Info. Theory, vol. 40, no. 4, July T. S. Han, Information-Spectrum Method in Information Theory, New York: Springer, /32

13 Compound Channel Model I a channel is selected from a given uncertainty set stays fixed during entire transmission design a single code good for any channel in the set largest achievable rate = compound capacity No CSI at Tx, complete CSI at Rx 13/32

14 Compound Channel Model II X n = {X 1...X n } - (random) sequence of n input symbols Y n - corresponding output sequence; s S - channel state (which may also be a sequence) p(x n ), p s (y n ) - input/output distributions under state s. p s (y n x n ) - channel transition probability; information density ( unexpected mutual information): i(x n ;y n,s) = ln p s(x n,y n ) p(x n )p s (y n ) = i(xn ;y n s) (1) 14/32

15 Compound Channel Model III If information-stable: If information-unstable: 1 n i(xn ;y n s) I(X;Y s) as n (2) 1 n i(xn ;y n s) RV (3) 15/32

16 Definitions I (n, r n, ε ns )-code: n - the block length ε ns - the error probability (for channel state s) r n = lnm n /n - the code rate M n - the number of codewords compound error probability: achievable rate R: (n, r n, ε ns )-code such that compound channel capacity C: ε n = supε ns (4) s S lim supε n = 0, liminf r n R (5) n n C = sup{r : R is achievable} (6) R 16/32

17 Capacity for a given channel state Very general - information-unstable channels Given channel state s, the capacity is [Verdu-Han 94] 5 C(s) = supi(x;y s) (7) p(x) where I(X; Y s) is the inf-information rate I(X;Y s) = sup R { R : lim n Pr{ n 1 i(x n ;Y n s) R } = 0 } (8) Proof: via Feinstein and Verdu-Han Lemmas 5 S. Verdu, T.S. Han, A General Formula for Channel Capacity, IEEE Trans. Info. Theory, vol. 40, no. 4, July /32

18 Capacity for a given channel state Lemma (Feinstein) For arbitrary input X n, any r n and a given channel state s, there exists a code satisfying the following inequality, for any γ > 0. Lemma (Verdu-Han 94) ε ns Pr { n 1 i(x n ;Y n s) r n +γ } +e γn (9) Every (n,r n,ε ns )-code satisfies the following inequality, ε ns Pr { n 1 i(x n ;Y n s) r n γ } e γn (10) for any γ > 0, where X n is uniformly distributed over all codewords and Y n is the corresponding channel output under channel state s. 18/32

19 Capacity for a given channel state Lemma (Feinstein) For arbitrary input X n, any r n and a given channel state s, there exists a code satisfying the following inequality, for any γ > 0. Lemma (Verdu-Han 94) ε ns Pr { n 1 i(x n ;Y n s) r n +γ } +e γn (9) Every (n,r n,ε ns )-code satisfies the following inequality, ε ns Pr { n 1 i(x n ;Y n s) r n γ } e γn (10) for any γ > 0, where X n is uniformly distributed over all codewords and Y n is the corresponding channel output under channel state s. 19/32

20 General Formula for Compound Channel Capacity Theorem Consider a general compound channel; the Rx knows s S, but not the Tx; the Tx knows the (arbitrary) S. The compound channel capacity is C c = supi(x;y) (11) p(x) where I(X;Y) = sup R {R Ω} is the compound inf-information rate, Ω = { R : lim sup n s S { } } 1 Pr n i(xn ;Y n s) R = 0 (12) Proof: via Feinstein and Verdu-Han Lemmas extended to compound setting 20/32

21 General Formula for Compound Channel Capacity Theorem Consider a general compound channel; the Rx knows s S, but not the Tx; the Tx knows the (arbitrary) S. The compound channel capacity is C c = supi(x;y) (11) p(x) where I(X;Y) = sup R {R Ω} is the compound inf-information rate, Ω = { R : lim sup n s S { } } 1 Pr n i(xn ;Y n s) R = 0 (12) Proof: via Feinstein and Verdu-Han Lemmas extended to compound setting 21/32

22 Compound Lemmas Lemma (compound Feinstein) For any X n, S and r n, there exists a (n,r n,ε n )-code (where the codewords are independent of channel state s), such that, for any γ > 0, ε n suppr { n 1 i(x n ;Y n s) r n +γ } +e γn (13) s S Lemma (compound Verdu-Han) Every (n,r n,ε n )-code satisfies ε n suppr { n 1 i(x n ;Y n s) r n γ } e γn (14) s S for any γ > 0, where X n is uniformly distributed over all codewords and Y n is the corresponding channel output under channel state s. 22/32

23 Compound Lemmas Lemma (compound Feinstein) For any X n, S and r n, there exists a (n,r n,ε n )-code (where the codewords are independent of channel state s), such that, for any γ > 0, ε n suppr { n 1 i(x n ;Y n s) r n +γ } +e γn (13) s S Lemma (compound Verdu-Han) Every (n,r n,ε n )-code satisfies ε n suppr { n 1 i(x n ;Y n s) r n γ } e γn (14) s S for any γ > 0, where X n is uniformly distributed over all codewords and Y n is the corresponding channel output under channel state s. 23/32

24 Properties Remark I(X,Y) is an extension of I(X,Y s) to the compound channel setting, not inf s I(X,Y s), in the general case. The relationship between I(X,Y) and inf s I(X,Y s) is established below. Proposition The following inequality holds for a general compound channel I(X,Y) I(X,Y) = infi(x,y s) (15) s Strict inequality can be shown by examples. 24/32

25 Properties Remark I(X,Y) is an extension of I(X,Y s) to the compound channel setting, not inf s I(X,Y s), in the general case. The relationship between I(X,Y) and inf s I(X,Y s) is established below. Proposition The following inequality holds for a general compound channel I(X,Y) I(X,Y) = infi(x,y s) (15) s Strict inequality can be shown by examples. 25/32

26 Uniform Compound Channel When does the equality hold? Definition A compound channel is uniform if Pr { n 1 i(x n ;Y n s) I(X,Y) γ } 0 γ > 0 (16) uniformly in s S as n. Proposition The following equality holds for a uniform compound channel I(X,Y) = I(X,Y) = infi(x,y s) (17) s 26/32

27 Uniform Compound Channel When does the equality hold? Definition A compound channel is uniform if Pr { n 1 i(x n ;Y n s) I(X,Y) γ } 0 γ > 0 (16) uniformly in s S as n. Proposition The following equality holds for a uniform compound channel I(X,Y) = I(X,Y) = infi(x,y s) (17) s 27/32

28 Uniform Compound Channel Theorem The capacity of a uniform compound channel: C c = sup inf I(X;Y s) (18) p(x) s S Does not hold for non-uniform channels, contrary to what Theorem in [Han 03] 6 claims. 6 T. S. Han, Information-Spectrum Method in Information Theory, New York: Springer, /32

29 Examples & mistakes Binary non-stationary channel with memory: p s (y n x n ) = p s (y n ) if n s (19) = BSC(0) if n > s (20) Models the noise coherence time τ = s S = {1,2,...}. It follows that I(X;Y) = 0 < I(X;Y) = infi(x;y s) = ln2 s (21) C c = supi(x;y) = 0 < ln2 = sup inf I(X;Y s) p(x) p(x) s S (22) Mistake: Theorem in [Han 03] claims C c = ln2. 29/32

30 Examples & mistakes Mistake: Theorem in [Han 03] claims C c = ln2. Reason: improper error probability definition for compound channel: In general, In our example, lim ε ns = 0 s lim n sup s 0 = sup s sup n s lim ε ns lim n sup n s lim ε ns < lim n sup n s ε ns = 0 (23) ε ns (24) ε ns = 1 (25) 30/32

31 Examples & mistakes Binary compound channel with additive noise: Y k = X k +Z ks, Z n s = {w 1,w 2,...w s,0,0...0} (26) and w 1...w s are i.i.d. equiprobable. Its capacity Why C c = 0? C c = supi(x;y) = 0 < ln2 = sup p(x) p(x) inf I(X;Y s) (27) For any n, does not matter how large, there are always channel states s n for which the channel is BSC(1/2), i.e. useless. The standard sup inf expression falls short of the channel capacity because this compound channel is not uniform. s S 31/32

32 Conclusion CSI uncertainty - compound channel Real world - information-unstable channels General formula for compound channel capacity Uniform compound channels Examples & mistakes 32/32