Interactions of Information Theory and Estimation in Single- and Multi-user Communications

Size: px

Start display at page:

Download "Interactions of Information Theory and Estimation in Single- and Multi-user Communications"

Giles Kenneth Wright
5 years ago
Views:

1 Interactions of Information Theory and Estimation in Single- and Multi-user Communications Dongning Guo Department of Electrical Engineering Princeton University March 8, 2004 p 1 Dongning Guo

2 Communications S Source X Encoder Noisy Y channel S Decoder Dest p 2 Dongning Guo

3 Communications S Source X Encoder Noisy Y channel S Decoder Dest Tension: Noise causes errors (X Y ) Reliable transmission at a good rate (S S ) p 2 Dongning Guo

4 Communications S Source X Encoder Noisy Y channel S Decoder Dest Tension: Noise causes errors (X Y ) Reliable transmission at a good rate (S S ) Shannon (1948): Given a noisy channel, arbitrarily reliable transmission is possible up to a certain rate p 2 Dongning Guo

5 Communications S Source X Encoder Noisy Y channel S Decoder Dest Tension: Noise causes errors (X Y ) Reliable transmission at a good rate (S S ) Shannon (1948): Given a noisy channel, arbitrarily reliable transmission is possible up to a certain rate Estimation vs Information Analog vs digital p 2 Dongning Guo

6 Communications S Source X Encoder Noisy Y channel S Decoder Dest Tension: Noise causes errors (X Y ) Reliable transmission at a good rate (S S ) Shannon (1948): Given a noisy channel, arbitrarily reliable transmission is possible up to a certain rate Estimation vs Information Analog vs digital This talk: Interactions, (new) surprises, and applications p 2 Dongning Guo

7 Example: Mars Rover Received power as low as watt Rate as high as 10 Kbits/sec p 3 Dongning Guo

8 Wireless Network (X 1, Y 1 ) (X k, Y k ) p 4 Dongning Guo

9 Wireless Network (X 1, Y 1 ) (X k, Y k ) Tension: Throughput Interferences (complicated) Resources are scarce and shared p 4 Dongning Guo

10 Wireless Network (X 1, Y 1 ) (X k, Y k ) Tension: Throughput Interferences (complicated) Resources are scarce and shared Interferences are information Think smart! p 4 Dongning Guo

11 Wireless Network (X 1, Y 1 ) (X k, Y k ) Tension: Throughput Interferences (complicated) Resources are scarce and shared Interferences are information Think smart! Wanted: Simple laws of the seemingly disordered system p 4 Dongning Guo

12 Example: Sensor Network p 5 Dongning Guo

13 Mutual Information & MMSE Probabilistic point of view: X P X X P Y X Y X and Y are random variables/vectors or processes p 6 Dongning Guo

14 Mutual Information & MMSE Probabilistic point of view: X P X X P Y X Y X and Y are random variables/vectors or processes Minimum mean-squared error (MMSE): mmse(x; Y ) = min f E X f(y ) 2, Achieved by X(Y ) = E {X Y } p 6 Dongning Guo

15 Mutual Information & MMSE Probabilistic point of view: X P X X P Y X Y X and Y are random variables/vectors or processes Minimum mean-squared error (MMSE): mmse(x; Y ) = min f E X f(y ) 2, Achieved by X(Y ) = E {X Y } Mutual information: I(X; Y ) = E { log p } XY (X, Y ) p X (X)p Y (Y ) p 6 Dongning Guo

16 Multi-user Perspective Examples: cellular telephony, sensor networks, DSL X 1 X K Channel P Y X Y Multiuser detector X 1 X K p 7 Dongning Guo

17 Multi-user Perspective Examples: cellular telephony, sensor networks, DSL X 1 X K Channel P Y X Y Multiuser detector X 1 X K Mean-square error: E X k X k 2 p 7 Dongning Guo

18 Multi-user Perspective Examples: cellular telephony, sensor networks, DSL X 1 X K Channel P Y X Y Multiuser detector X 1 X K Mean-square error: E X k X k 2 Mutual information: I (X k ; X k ) p 7 Dongning Guo

19 Outline Part I: Canonical Gaussian channel: Y = α X + W A fundamental I-mmse relationship By-products and applications p 8 Dongning Guo

20 Outline Part I: Canonical Gaussian channel: Y = α X + W A fundamental I-mmse relationship By-products and applications Part II: Multiuser (vector) channel: Y = S N K X + W Large-system analysis via statistical physics Result: Multiuser channel can be decoupled p 8 Dongning Guo

21 Part I: Y = α X + W p 9 Dongning Guo

22 An Observation Scalar Gaussian channel: Y = snr X + W, W N (0, 1) p 10 Dongning Guo

23 An Observation Scalar Gaussian channel: Y = snr X + W, W N (0, 1) Gaussian input: X N (0, 1) I(X; Y ) = 1 2 mmse(snr) = E log(1 + snr), [ snr X 1 + snr Y ] 2 = snr p 10 Dongning Guo

24 An Observation Scalar Gaussian channel: Y = snr X + W, W N (0, 1) Gaussian input: X N (0, 1) I(X; Y ) = 1 2 mmse(snr) = E log(1 + snr), [ snr X 1 + snr Y ] 2 = snr Immediately, d dsnr I(X; Y ) = 1 mmse(snr) log e 2 ( ) p 10 Dongning Guo

25 Another Observation Scalar Gaussian channel: Y = snr X + W, W N (0, 1) p 11 Dongning Guo

26 Another Observation Scalar Gaussian channel: Y = snr X + W, W N (0, 1) Binary input: X = ±1 equally likely I(snr) = snr e y2 2 2π log cosh(snr snr y) dy, mmse(snr) = 1 e y2 2 2π tanh(snr snr y) dy p 11 Dongning Guo

27 Another Observation Scalar Gaussian channel: Y = snr X + W, W N (0, 1) Binary input: X = ±1 equally likely I(snr) = snr e y2 2 2π log cosh(snr snr y) dy, mmse(snr) = 1 e y2 2 2π tanh(snr snr y) dy Again, d dsnr I(X; Y ) = 1 2 mmse(snr) ( ) p 11 Dongning Guo

28 d dsnr I(snr) = 1 2 mmse(snr) ( ) mmse snr I snr snr Gaussian input p 12 Dongning Guo

29 d dsnr I(snr) = 1 2 mmse(snr) ( ) mmse snr I snr snr Gaussian input, binary input p 12 Dongning Guo

30 I-mmse Theorem Theorem 1 (Guo, Shamai & Verdú) Y = snr X + W P X with EX 2 <, d dsnr I(snr) = 1 2 mmse(snr) ( ) p 13 Dongning Guo

31 I-mmse Theorem Theorem 1 (Guo, Shamai & Verdú) Y = snr X + W P X with EX 2 <, ( ) also proved for: Vector channel: Continuous-time: Discrete-time: d dsnr I(snr) = 1 2 mmse(snr) Y = snr HX + W Y t = snr X t + W t, t [0, T ] Y n = snr X n + W n, n = 1, 2, ( ) p 13 Dongning Guo

32 Information vs Estimation Information theory I(X; Y ): (coded) reliable rate detection theory likelihood ratio estimation theory mmse(x; Y ): (uncoded) accuracy p 14 Dongning Guo

33 Information vs Estimation Information theory I(X; Y ): (coded) reliable rate detection theory likelihood ratio estimation theory mmse(x; Y ): (uncoded) accuracy History: Wiener, Shannon, Kolmogorov Price, Kailath, s Duncan, 1970: Mutual information causal MMSE in continuous-time filtering p 14 Dongning Guo

34 Proof of d dsnr I(snr) = 1 2 mmse(snr) ( ) Equivalent to: I(snr + δ) I(snr) = δ 2 mmse(snr) + o(δ) p 15 Dongning Guo

35 Proof of d dsnr I(snr) = 1 2 mmse(snr) ( ) Equivalent to: I(snr + δ) I(snr) = δ 2 mmse(snr) + o(δ) Incremental channel: X σ 1 W 1 σ 2 W 2 Y 1 snr + δ Y 2 snr p 15 Dongning Guo

36 Proof of d dsnr I(snr) = 1 2 mmse(snr) ( ) Equivalent to: I(snr + δ) I(snr) = δ 2 mmse(snr) + o(δ) Incremental channel: Markov property: X σ 1 W 1 σ 2 W 2 Y 1 snr + δ Y 2 snr I(X; Y 1 ) I(X; Y 2 ) = I(X; Y 1, Y 2 ) I(X; Y 2 ) = I(X; Y 1 Y 2 ) p 15 Dongning Guo

37 Proof of d dsnr I(snr) = 1 2 mmse(snr) ( ) Equivalent to: I(snr + δ) I(snr) = δ 2 mmse(snr) + o(δ) Incremental channel: Markov property: X σ 1 W 1 σ 2 W 2 Y 1 snr + δ Y 2 snr I(X; Y 1 ) I(X; Y 2 ) = I(X; Y 1, Y 2 ) I(X; Y 2 ) = I(X; Y 1 Y 2 ) Given Y 2, the channel X Y 1 is Gaussian: (snr + δ) Y 1 = snr Y 2 + δ X + N (0, δ) p 15 Dongning Guo

38 Proof of d dsnr I(snr) = 1 mmse(snr) (Cont) 2 Lemma 1 (Verdú 90, Lapidoth-Shamai 02, Guo et al 04) Y = δ Z + U, U N (0, 1) As δ 0, I(Y ; Z) = δ 2 E (Z EZ)2 + o(δ) p 16 Dongning Guo

39 Proof of d dsnr I(snr) = 1 mmse(snr) (Cont) 2 Lemma 1 (Verdú 90, Lapidoth-Shamai 02, Guo et al 04) Y = δ Z + U, U N (0, 1) As δ 0, I(Y ; Z) = δ 2 E (Z EZ)2 + o(δ) Apply Lemma 1 to X Y 1 conditioned on Y 2 : I(X; Y 1 Y 2 ) = δ 2 E (X E {X Y 2}) 2 + o(δ) p 16 Dongning Guo

40 Proof of d dsnr I(snr) = 1 mmse(snr) (Cont) 2 Lemma 1 (Verdú 90, Lapidoth-Shamai 02, Guo et al 04) Y = δ Z + U, U N (0, 1) As δ 0, I(Y ; Z) = δ 2 E (Z EZ)2 + o(δ) Apply Lemma 1 to X Y 1 conditioned on Y 2 : I(X; Y 1 Y 2 ) = δ 2 E (X E {X Y 2}) 2 + o(δ) Increase due to SNR increment: I(snr + δ) I(snr) = I(X; Y 1 Y 2 ) = δ mmse(snr) + o(δ) 2 p 16 Dongning Guo

41 Proof of d dsnr I(snr) = 1 mmse(snr) (Cont) 2 Lemma 1 (Verdú 90, Lapidoth-Shamai 02, Guo et al 04) Y = δ Z + U, U N (0, 1) As δ 0, I(Y ; Z) = δ 2 E (Z EZ)2 + o(δ) Apply Lemma 1 to X Y 1 conditioned on Y 2 : I(X; Y 1 Y 2 ) = δ 2 E (X E {X Y 2}) 2 + o(δ) Increase due to SNR increment: I(snr + δ) I(snr) = I(X; Y 1 Y 2 ) = δ mmse(snr) + o(δ) 2 Key property: infinite divisibility of Gaussian p 16 Dongning Guo

42 Why d dsnr I(snr) = 1 2 mmse(snr)? snr 1 snr 2 snr 3 0 X Y 1 Y 2 Y 3 p 17 Dongning Guo

43 Why d dsnr I(snr) = 1 2 mmse(snr)? snr 1 snr 2 snr 3 0 X Y 1 Y 2 Y 3 Using mutual information chain rule: I(snr 1 ) = I(X; Y 1 ) = I(X; Y 1 Y 2 ) + I(X; Y 2 ) = n=1 I(X; Y n Y n+1 ) 1 (snr n snr n+1 ) mmse(snr n ) n=1 snr1 0 mmse(γ) dγ p 17 Dongning Guo

44 Continous-time Channel Channel model: R t = dy t = snr X t + N t, snr X t dt + db t or, equivalently, {B t } Brownian motion (Wiener process) p 18 Dongning Guo

45 Continous-time Channel Channel model: R t = snr X t + N t, or, equivalently, dy t = snr X t dt + db t {B t } Brownian motion (Wiener process) Consider t [0, T ] Information rate: I(snr) = 1 T I(XT 0 ; Y T 0 ) Causal and non-causal MMSEs: cmmse(t, snr) = E ( X t E { }) 2, X t Y0 t mmse(t, T, snr) = E ( X t E { }) 2 X t Y T 0 p 18 Dongning Guo

46 Triangle Relationship Theorem 2 If T 0 EX2 t dt < (finite-power input), d dsnr I(snr) = 1 2 T 0 mmse(t, T, snr) dt T Proof: Radon-Nikodym derivatives Stochastic calculus p 19 Dongning Guo

47 Triangle Relationship Theorem 2 If T 0 EX2 t dt < (finite-power input), d dsnr I(snr) = 1 2 T 0 mmse(t, T, snr) dt T Proof: Radon-Nikodym derivatives Stochastic calculus Theorem 3 (Duncan 1970) For finite-power input, I(snr) = snr 2 T 0 cmmse(t, snr) dt T p 19 Dongning Guo

48 Triangle Relationship Theorem 2 If T 0 EX2 t dt < (finite-power input), d dsnr I(snr) = 1 2 T 0 mmse(t, T, snr) dt T Proof: Radon-Nikodym derivatives Stochastic calculus Theorem 3 (Duncan 1970) For finite-power input, I(snr) = snr 2 T 0 cmmse(t, snr) dt T Theorem 4 a T 0 cmmse(t, snr) dt T = T 0 snr 0 mmse(t, T, γ) dγ snr dt T p 19 Dongning Guo

49 Stationary Inputs Theorem 5 For stationary finite-power input, d dsnr I(snr) = 1 2 mmse(snr) ( ) p 20 Dongning Guo

50 Stationary Inputs Theorem 5 For stationary finite-power input, d dsnr I(snr) = 1 2 mmse(snr) ( ) Theorem 6 For stationary finite-power input, cmmse(snr) = 1 snr snr 0 mmse(γ) dγ ( ) p 20 Dongning Guo

51 Stationary Inputs Theorem 5 For stationary finite-power input, d dsnr I(snr) = 1 2 mmse(snr) ( ) Theorem 6 For stationary finite-power input, cmmse(snr) = 1 snr snr 0 mmse(γ) dγ ( ) Can be checked: Gaussian input with spectrum S X (ω) I(snr) [Shannon 49], cmmse(snr) [Yovits-Jackson 55] Random telegraph waveform (2-state Markov) input MMSEs due to Wonham 65 and Yao 85 p 20 Dongning Guo

52 cmmse(snr) = 1 snr snr 0 mmse(γ) dγ ( ) 1 An example: Random telegraph waveform input cmmse snr mmse snr snr p 21 Dongning Guo

53 Extensions Vector channel Y = snr H X + W : d dsnr I(X; Y ) = 1 2 E H X H E {X Y } 2 p 22 Dongning Guo

54 Extensions Vector channel Y = snr H X + W : d dsnr I(X; Y ) = 1 2 E H X H E {X Y } 2 Discrete-time model: Y n = snr X n + W n, n = 1, 2, Via piecewise constant continuous-time input p 22 Dongning Guo

55 Extensions Vector channel Y = snr H X + W : d dsnr I(X; Y ) = 1 2 E H X H E {X Y } 2 Discrete-time model: Y n = snr X n + W n, n = 1, 2, Via piecewise constant continuous-time input More general models: dy t = snr h t (X t ) dt + db t, t R MMSEs errors in estimating channel input h t (X t ) p 22 Dongning Guo

56 Other Channels & Applications Incremental channel device works if noise has independent increments (Lévy processes) Eg, Gaussian channel MMSE: X 2 t X 2 t ; Poisson channel φ(λ t ) φ( λ t ), where φ(x) = x log x p 23 Dongning Guo

57 Other Channels & Applications Incremental channel device works if noise has independent increments (Lévy processes) Eg, Gaussian channel MMSE: X 2 t X 2 t ; Poisson channel φ(λ t ) φ( λ t ), where φ(x) = x log x Bounds on MMSE bounds on mutual information Linear estimation upper bound Intersymbol interference channel p 23 Dongning Guo

58 Other Channels & Applications Incremental channel device works if noise has independent increments (Lévy processes) Eg, Gaussian channel MMSE: X 2 t X 2 t ; Poisson channel φ(λ t ) φ( λ t ), where φ(x) = x log x Bounds on MMSE bounds on mutual information Linear estimation upper bound Intersymbol interference channel MMSE estimator is often a key component in capacity achieving receivers p 23 Dongning Guo

59 Other Channels & Applications Incremental channel device works if noise has independent increments (Lévy processes) Eg, Gaussian channel MMSE: X 2 t X 2 t ; Poisson channel φ(λ t ) φ( λ t ), where φ(x) = x log x Bounds on MMSE bounds on mutual information Linear estimation upper bound Intersymbol interference channel MMSE estimator is often a key component in capacity achieving receivers More to be discovered p 23 Dongning Guo

60 Part II: Y = SX + W p 24 Dongning Guo

61 Multiuser Channel Channel model: K Y = s k snrk X k + W where W N (0, I) k=1 = S N K X + W, p 25 Dongning Guo

62 Multiuser Channel Channel model: Y = where W N (0, I) K s k snrk X k + W k=1 Assumptions: Iid input: X k P X Random signatures: s k Fading: snr k P snr = S N K X + W, p 25 Dongning Guo

63 Examples Code-division multiple access (CDMA): X 1 X 2 snr1 s 1 snr2 s 2 W N (0, I) K Y = s k snrk X k + W k=1 X K snrk s K Y = S N K X + W Raondom signatures in Qualcomm CDMA, 3G wireless Also: multiple-antenna, multi-carrier systems p 26 Dongning Guo

64 Joint Decoding Joint decoding Encoder Encoder Encoder X 1 snr1 s 1 X 2 snr2 s 2 X K snrk s K N (0, I) Y Joint decoding p 27 Dongning Guo

65 Joint Decoding Joint decoding Encoder Encoder Encoder X 1 snr1 s 1 X 2 snr2 s 2 X K snrk s K N (0, I) Y Joint decoding Spectral efficiency: C joint = 1 N I(X; Y ) p 27 Dongning Guo

66 Separate Decoding Multiuser detection + single-user decoding: Encoder Encoder Encoder X 1 snr1 s 1 X 2 snr2 s 2 X K snrk s K N (0, I) Y Multiuser detector Decoder X 1 Decoder X 2 Decoder X K p 28 Dongning Guo

67 Separate Decoding Multiuser detection + single-user decoding: Encoder Encoder Encoder X 1 snr1 s 1 Xk snrk s k X K snrk s K N (0, I) Multiuser Y detector Decoder X 1 X k Decoder Decoder X K Channel for user k: Capacity of user k: P Xk X k I(X k ; X k ) p 28 Dongning Guo

68 Multiuser Detection Detection function: X k = f k (Y, S) p 29 Dongning Guo

69 Multiuser Detection Detection function: X k = f k (Y, S) Optimal: X k = E {X k Y, S} Wrt p Xk Y,S (induced from p X and p Y X,S ) p 29 Dongning Guo

70 Multiuser Detection Detection function: X k = f k (Y, S) Optimal: X k = E {X k Y, S} Wrt p Xk Y,S (induced from p X and p Y X,S ) Suboptimal: X k q = E q {X k Y, S} Wrt q Xk Y,S (induced from q X and q Y X,S ) p 29 Dongning Guo

71 Multiuser Detection Detection function: X k = f k (Y, S) Optimal: X k = E {X k Y, S} Wrt p Xk Y,S (induced from p X and p Y X,S ) Suboptimal: X k q = E q {X k Y, S} Wrt q Xk Y,S (induced from q X and q Y X,S ) Postulated input q X Postulated channel: Y = S X + σ W p 29 Dongning Guo

72 Special Cases If q X N (0, 1), then X q = [ S S + σ 2 I ] 1 S Y σ : matched filter; σ = 1: linear MMSE; σ 0: decorrelator p 30 Dongning Guo

73 Special Cases If q X N (0, 1), then X q = [ S S + σ 2 I ] 1 S Y σ : matched filter; σ = 1: linear MMSE; σ 0: decorrelator If q X = p X, then σ 0: jointly optimal; σ = 1: individually optimal p 30 Dongning Guo

74 Special Cases If q X N (0, 1), then X q = [ S S + σ 2 I ] 1 S Y σ : matched filter; σ = 1: linear MMSE; σ 0: decorrelator If q X = p X, then σ 0: jointly optimal; σ = 1: individually optimal In principle, multiuser detector X q = E q {X Y, S}, parameterized by q X and σ p 30 Dongning Guo

75 Problem Multiuser detection + single-user codes Encoder Encoder Encoder X 1 snr1 s 1 Xk snrk s k X K snrk s K N (0, I) Multiuser Y detector Decoder X 1 q Decoder X k q Decoder X K q What is P Xk q X k? I(X k ; X k q )? I(X; Y )? p 31 Dongning Guo

76 Problem Multiuser detection + single-user codes Encoder Encoder Encoder X 1 snr1 s 1 Xk snrk s k X K snrk s K N (0, I) Multiuser Y detector Decoder X 1 q Decoder X k q Decoder X K q What is P Xk q X k? I(X k ; X k q )? I(X; Y )? Large system: user number (K), dimensionality (N), lim K K N β p 31 Dongning Guo

77 New Result: Decoupling Multiuser channel: Encoder Encoder Encoder X 1 snr1 s 1 Xk snrk s k X K snrk s K N (0, I) Multiuser Y detector Decoder X 1 q Decoder X k q Decoder X K q p 32 Dongning Guo

78 New Result: Decoupling Multiuser channel: Encoder Encoder Encoder X 1 snr1 s 1 Xk snrk s k X K snrk s K N (0, I) Multiuser Y detector Decoder X 1 q Decoder X k q Decoder X K q Equivalent single-user scalar channel: N ( 0, η 1) Encoder X k snrk Y Decision function Decoder X k q p 32 Dongning Guo

79 Decision Function Implicit decision function: N ( 0, η 1) X p X snr + Z Decision function X q p 33 Dongning Guo

80 Decision Function Implicit decision function: N ( 0, η 1) X p X snr + Z Decision function X q Best estimate of X given Z assuming a postulated channel: N ( 0, ξ 1) X q X snr + Z Ie, X q = E q {X Z, snr; ξ} p 33 Dongning Guo

81 Retrochannel The equivalent channel & retrochannel: N ( 0, η 1) X p X snr + Z Decision function E q {X Z, snr; ξ} X q Retrochannel q X Z,snr;ξ X p 34 Dongning Guo

82 Retrochannel The equivalent channel & retrochannel: N ( 0, η 1) X p X snr + Z Decision function E q {X Z, snr; ξ} X q Retrochannel q X Z,snr;ξ [ ] 2; The MSE: E(snr; η, ξ) = E X X q [ ] 2 the variance: V(snr; η, ξ) = E X X q X p 34 Dongning Guo

83 Main Result: Decoupling Theorem 7 Equivalent channel for a user with snr k = snr: N ( 0, η 1) X p X snr + Z Decision function X q E q {X Z, snr; ξ} Retrochannel X q X Z,snr;ξ ie, Gaussian followed by a decision function, where η 1 =1 + β E {snr E(snr; η, ξ)}, ξ 1 =σ 2 + β E {snr V(snr; η, ξ)} ( ) E(snr; η, ξ) = E[X X q ] 2 ; V(snr; η, ξ) = E[X X q ] 2 p 35 Dongning Guo

84 Spectral Efficiencies Corollary 1 The capacity of user k is I(η snr k ) N ( 0, η 1) X p X snrk + Z p 36 Dongning Guo

85 Spectral Efficiencies Corollary 1 The capacity of user k is I(η snr k ) N ( 0, η 1) X p X snrk + Z Corollary 2 The overall spectral efficiency: C sep = β E {I(η snr)} p 36 Dongning Guo

86 Spectral Efficiencies Corollary 1 The capacity of user k is I(η snr k ) N ( 0, η 1) X p X snrk + Z Corollary 2 The overall spectral efficiency: C sep = β E {I(η snr)} Theorem 8 The spectral efficiency under joint decoding is C joint = C sep + (η 1 log η)/2 Generalization of Shamai-Verdú, Tanaka, Müller-Gerstaker p 36 Dongning Guo

87 Special Case Postulate q X N (0, 1), σ 2 = 1 Then linear MMSE p 37 Dongning Guo

88 Special Case Postulate q X N (0, 1), σ 2 = 1 Then linear MMSE MMSE: mmse(snr; η) = η snr p 37 Dongning Guo

89 Special Case Postulate q X N (0, 1), σ 2 = 1 Then linear MMSE MMSE: mmse(snr; η) = η snr Tse-Hanly 99 is a special case of ( ): { } η 1 1 = 1 + β E snr 1 + η snr p 37 Dongning Guo

90 Special Case Postulate q X N (0, 1), σ 2 = 1 Then linear MMSE MMSE: mmse(snr; η) = η snr Tse-Hanly 99 is a special case of ( ): { } η 1 1 = 1 + β E snr 1 + η snr Single-user capacity C(snr; η) = 1 2 log (1 + ηsnr) p 37 Dongning Guo

91 Joint vs Separate Theorem 9 P X, C joint (β) = β 0 1 β C sep(β ) dβ p 38 Dongning Guo

92 Joint vs Separate Theorem 9 P X, Proof: Use C joint (β) = β d dβ C joint = 1 β C sep + dη dβ 0 1 β C sep(β ) dβ [ { } ] d E dη I(ηsnr) + 1 η 1 d dsnr I(snr) = 1 mmse(snr) ( ), and fixed-point eqn ( ) 2 p 38 Dongning Guo

93 Joint vs Separate Theorem 9 P X, β 1 C joint (β) = 0 β C sep(β ) dβ d Proof: dβ C joint = 1 β C sep + dη [ { } ] d E dβ dη I(ηsnr) + 1 η 1 d Use dsnr I(snr) = 1 mmse(snr) ( ), and fixed-point eqn ( ) 2 Mutual information chain rule Successive cancellation 1 N I(X; Y S) = 1 K I(X k ; Y S, X k+1,, X K ) N = 1 N k=1 K I k=1 ( η ( k N ) ) snr k p 38 Dongning Guo

94 Statistical Physics From microscopic interactions to macroscopic properties p 39 Dongning Guo

95 Statistical Physics From microscopic interactions to macroscopic properties Spin glass: p 39 Dongning Guo

96 Statistical Physics From microscopic interactions to macroscopic properties Spin glass: Posterior distribution of transmitted symbols = configuration distribution under external field p X Y,S (x y, S) exp [ H y,s (x)] p 39 Dongning Guo

97 Statistical Physics From microscopic interactions to macroscopic properties Spin glass: Posterior distribution of transmitted symbols = configuration distribution under external field p X Y,S (x y, S) exp [ H y,s (x)] Multiuser system spin glass Spins bits External field received signal & channel state p 39 Dongning Guo

98 Replica Trick Mutual information 1 K I(X; Y S) = 1 K E { log p Y S (Y S) S } const = F(S) const p 40 Dongning Guo

99 Replica Trick Mutual information 1 K I(X; Y S) = 1 K E { log p Y S (Y S) S } const = F(S) const Asymptotic equipartition property Free energy F(S) F = lim K (1/K) E { log p Y S (Y S) } p 40 Dongning Guo

100 Replica Trick Mutual information 1 K I(X; Y S) = 1 K E { log p Y S (Y S) S } const = F(S) const Asymptotic equipartition property Free energy F(S) F = lim K (1/K) E { log p Y S (Y S) } Replica trick: lim u 0 u log E E {Θ u log Θ} {Θu } = lim u 0 E {Θ u } = E {log Θ} p 40 Dongning Guo

101 Replica Trick Mutual information 1 K I(X; Y S) = 1 K E { log p Y S (Y S) S } const = F(S) const Asymptotic equipartition property Free energy F(S) F = lim K (1/K) E { log p Y S (Y S) } Replica trick: lim u 0 u log E E {Θ u log Θ} {Θu } = lim u 0 E {Θ u } = E {log Θ} Introducing replicas X ak, a = 1,, u, p u Y S (y S) E { u a=1 p Y X,S(y X a, S) S } p 40 Dongning Guo

102 Conclusion I-mmse relationship: Under arbitrary input distribution, d dsnr I(snr) = 1 2 mmse(snr) ( ) Incremental channel proof Relationship between causal & non-causal MMSEs p 41 Dongning Guo

103 Conclusion I-mmse relationship: Under arbitrary input distribution, d dsnr I(snr) = 1 2 mmse(snr) ( ) Incremental channel proof Relationship between causal & non-causal MMSEs Multiuser channel: Multiuser detector optimal for a postulated system Decoupling: Equivalent Gaussian single-user channel Multiuser efficiency, spectral efficiencies p 41 Dongning Guo

104 Future Work Application and practical implications of d dsnr I(snr) = 1 2 mmse(snr) ( ) Eg, ISI channels, coding Further extensions MMSE structures in capacity-achieving receivers Eg, ISI channel, CDMA channel, lattice codes Applications of the decoupling result Eg, power control, signaling optimization Applications of statistical physics methodologies Eg, large sensor networks p 42 Dongning Guo

Decoupling of CDMA Multiuser Detection via the Replica Method

Decoupling of CDMA Multiuser Detection via the Replica Method Dongning Guo and Sergio Verdú Dept. of Electrical Engineering Princeton University Princeton, NJ 08544, USA email: {dguo,verdu}@princeton.edu