Variable Rate Channel Capacity. Jie Ren 2013/4/26

Transcription

1 Variable Rate Channel Capacity Jie Ren 2013/4/26

2 Reference This is a introduc?on of Sergio Verdu and Shlomo Shamai s paper. Sergio Verdu and Shlomo Shamai, Variable- Rate Channel Capacity, IEEE Transac?ons on Informa?on Theory, vol. 56, No. 6, June 2010.

3 Outline (1 st talk) Conven?onal Channel Coding Setup of Variable Rate Coding

4 Conven?onal Channel Capacity X P(y x)

5 Conven?onal Channel Capacity What if Channel state varies? X P(y x,m)! = inf! sup!!!(!;!! )!

6 Example: BSC Crossover Probability p

7 Example: BSC Crossover Probability p=0.5! = 1 h 0.5 = 0!

8 Example: BSC Crossover Probability p~uni(0,0.5)! = inf! 1 h! = 0!

9 Conven?onal Channel Capacity What if Channel state varies? X P(y x,m)! = inf! sup!!!(!;!! )!

10 Outline Conven?onal Channel Coding Setup of Variable Rate Coding Variable- to- Fixed Capacity Fixed- to- Variable Capacity Variable- Blocklength Capacity Examples

11 Setup of Variable Rate Coding Fixed- to- fixed coding Variable- to- fixed coding Fixed- to- variable coding Variable- blocklength coding

12 Fixed- to- fixed B 1 B K X 1 X n Channel B 1 B k

13 Fixed- to- fixed B 1 B K X 1 X n Channel B 1 B k

14 Variable- to- Fixed! :! : {0,1}!!!!!!!: :!! {0,1}!!! B 1 B m X 1 X n Channel B 1 Bm k

15 Variable- to- Fixed B 1 B m X 1 X n Channel B 1 Bm k

16 Fixed- to- variable!!: : {0,1}!!!! B 1 B k X 1 Channel B 1 B k

17 Fixed- to- variable B 1 B k X 1 Channel B 1 B k

18 Variable- blocklength!! : {0,1}!!!!!! :!! {0,1}!!

19 Nota?ons

20 Basic Rela?onship

21 Example: BSC

22 Example: BSC

23 Variable Rate Channel Capacity Jie Ren 2013/4/30

24 Outline (2 nd talk) Reviews Theorems Examples

25 Review: Defini?ons Fixed- to- fixed capacity

26 Review: Defini?ons Variable- to- fixed capacity

27 Review: Defini?ons Fixed- to- variable & upper fixed- to- variable

28 Review: Defini?ons ε capacity lim!!!!!!

29 Basic Rela?onship

30 State- Dependent Channels

31 State- Dependent Channels Finite alphabets

32 Outline (2 nd talk) Reviews Theorems Examples

33 Theorem 7 Two possible states!!!!!!!!! =!!!! (!!!! ) Variable- to- fixed capacity in SDC!!!!! +!!! (!!!! )!!!!! = max!!" {min!!,!!,!!,!! + max{!!!!,!!!,!!!!,!!! }}!

34 Theorem 12 Fixed- to- fixed capacity in SDC

35 Theorem 13 Fixed- to- variable capacity in SDC

36 Theorem 14 Suppose Then the fixed- to- variable capacity in SDC

37 Theorem 15 Upper- fixed- to- variable capacity in SDC

38 Outline (2 nd talk) Reviews Theorems Examples

39 Example 1 With probability q With probability 1- q C0

40 By defini?on Example 1

41 Example 2 With probability π 0 : BSC(δ 0 ) With probability π 1 : BSC(δ 1 ), 0.5>δ 1 >δ 0

42 Example 2! =!!!!!!!!!!!!!!!

43 By theorem 12 Example 2

44 By theorem 7 Example 2

45 By theorem 14 & 15 Example 2

46 Example 3 Channel alphabet {0 1023} With probability 0.9: no error With probability 0.1: y i =(x i >0)

47 By theorem 13 Example 3

48 Subop?mal scheme Example 3

49 Example 4 Binary erasure channel BEC(E) E random variable Stay constant during the dura?on of the codeword

50 Example 4

51 E~Uni(0,1) Example 4

52 Example 5 Gaussian channel with nonergodic fading

53 Example 5 Gaussian channel with nonergodic fading

54 Example 5 Gaussian channel with nonergodic fading

55 Example 5 Gaussian channel with nonergodic fading

56 Variable Rate Channel Capacity Jie Ren 2013/5/7

57 Theorems and proofs Outline (3 rd talk)

58 Theorems and Proofs

59 For all channels Theorem 1

60 Proof Consider a fixed- to- fixed code Let

61 Theorem 2 If the channel input alphabet is finite

62 Proof Prove by contradic?on

63 Theorem 3 State- dependent channel, suppose each component sa?sfies strong converse

64 Proof The bound would be?ght if the encoder knows the channel state Hence it s an upper bound in general

65 Theorem 4 Capacity region or 2- user DMBC with degraded message sets

66 Proof Thomas M. Cover, Elements of Informa?on Theory, page 568

67 Theorem 5 In state- dependent channels

68 Proof Assume iden?ty permuta?on Show

69 Theorem 6 Theorem 5 holds with equality if the K- user broadcast channels sa?sfy the strong converse

70 Proof Number the channel states in order of increasing expected number of recovered bits

71 Proof For an arbitrarily small δ

72 Hence, Proof

73 Theorem 7 Suppose the channel has finite input/output alphabets and is memoryless.

74 Proof When the cons?tuent channels are discrete memoryless, the broadcast channel with degraded message sets sa?sfies the strong converse. Theorem 7 follows from 4 and 6

75 For all channels Theorem 8

76 Proof Consider a fixed- to- fixed code Define fixed- to- variable code

77 Theorem 9 Suppose the channel has finite memory

78 Proof Similar proof as theorem 8

79 Theorem 10 Assume either the input or output alphabets are finite

80 Proof Jensen s inequality Counterpart of Theorem 2

81 For all channels Theorem 11

82 Assume Proof

83 Proof Non- an?cipatory sesng Compa?ble variable- to- fixed encoder

84 Proof Construct a sequence of rateless encoders Let the ith component be equal to the ith component of variable- to- fixed code For n sufficiently large

85 Proof

86 Proof!!!!!!!!!!!!!!!!!!

87 Theorem 12 In state- dependent channel

88 Proof Shannon capacity Has to sa?sfy the worst channel

89 Theorem 13 In state- dependent channel

90 Proof Achievability proof Scheme: Fixed- to- variable Repi?on auer the first transmission

91 Theorem 14 In state- dependent channel, suppose Then theorem 13 holds with equality! = min!!(!!,!! )!!!!!!!!!!

92 Proof Converse proof Data- processing inequality

93 Theorem 15 In state- dependent channel

94 Proof Achievability Assume the decoder has knowledge of the ergodic mode Apply theorem 8!! = max!!!(!!,! )!!!!!!!

95 Converse Proof