On the Shamai-Laroia Approximation for the Information Rate of the ISI Channel

Transcription

1 On the Shamai-Laroia Approximation for the Information Rate of the ISI Channel Yair Carmon and Shlomo Shamai (Shitz) Department of Electrical Engineering, Technion - Israel Institute of Technology 2014 Information Theory and Applications Workshop San Diego, USA February 2014 Acknowledgment: Prof. Tsachy Weissman, FP7 Network of Excellence in Wireless COMmunications NEWCOM#, Israel Science Foundation (ISF). Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

2 Outline 1 Introduction ISI Channel and I.I.D. Information Rate Analytical Lower Bounds on the Information Rate The Shamai-Laroia Conjecture (SLC) 2 Low SNR Analysis 3 Counterexamples 4 High SNR Analysis 5 Conclusion Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

3 Introduction ISI Channel and I.I.D. Information Rate Inter-Symbol Interference Channel Model Input-output relationship: L 1 y k = h l x k l + n k l=0 Input Sequence x is assumed i.i.d, with P x = Ex 2 0 Noise Sequence n is Gaussian and i.i.d., with N 0 = En 2 0 Inter-symbol interference (ISI) coefficients h L 1 0 Channel frequency response H (θ) = L 1 k=0 h ke jkθ The simplest (non-discrete) model for a channel with memory Ubiquitous in wireless and wireline communications Results presented here extend straightforwardly to a complex setting Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

4 Introduction ISI Channel and I.I.D. Information Rate Inter-Symbol Interference Channel Model Input-output relationship: L 1 y k = h l x k l + n k l=0 Input Sequence x is assumed i.i.d, with P x = Ex 2 0 Noise Sequence n is Gaussian and i.i.d., with N 0 = En 2 0 Inter-symbol interference (ISI) coefficients h L 1 0 Channel frequency response H (θ) = L 1 k=0 h ke jkθ The simplest (non-discrete) model for a channel with memory Ubiquitous in wireless and wireline communications Results presented here extend straightforwardly to a complex setting Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

5 Mutual Information Rate Introduction ISI Channel and I.I.D. Information Rate Given by 1 I = lim K 2K + 1 I(yK K ; xk K ) = I(y ; x 0 x 1 ) Is the rate of reliable communications achievable by a random code with codewords distributed as x For Gaussian input I has a simple expression I Gaussian = 1 2π π π ( log 1 + P ) x H(θ) 2 dθ N 0 When the input is distributed on a finite set (constellation), no closed form expression is known Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

6 Introduction ISI Channel and I.I.D. Information Rate Approximating the I.I.D. Information Rate Two main ways to investigate I: 1 Approximations and bounds based on Monte-Carlo simulations Provide the best accuracy But little theoretic insight High computational complexity, that grows quickly with the number of dominant ISI taps c.f. [Arnold-Loeliger-Vontobel-Kavcic-Zeng 06] 2 Analytical lower bounds Not as tight as their simulation-based counterparts But much easier to compute Useful in benchmarking communication schemes and other bounds May provide theoretical insight Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

7 Introduction ISI Channel and I.I.D. Information Rate Approximating the I.I.D. Information Rate Two main ways to investigate I: 1 Approximations and bounds based on Monte-Carlo simulations Provide the best accuracy But little theoretic insight High computational complexity, that grows quickly with the number of dominant ISI taps c.f. [Arnold-Loeliger-Vontobel-Kavcic-Zeng 06] 2 Analytical lower bounds Not as tight as their simulation-based counterparts But much easier to compute Useful in benchmarking communication schemes and other bounds May provide theoretical insight Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

8 Introduction Data Processing Lower Bounds Analytical Lower Bounds on the Information Rate For any sequence of coefficients a, I = I(y ; x 0 x 1 ) I( k a ky k ; x 0 x 1 ) Can be simplified to, I I(x 0 ; x 0 + k 1 α kx k + m) }{{} additive noise term with α k = l a lh l k / l a lh l and m N (0, N 0 l a2 l /( l a lh l ) 2 ) independent of x 0 Different choices of a yield different bounds Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

9 Introduction Analytical Lower Bounds on the Information Rate Data Processing with a Sample Whitened Matched Filter a are chosen so that α k = 0 for every k > 0 (non-causal ISI eliminated) In this case the noise term is purely Gaussian The resulting bound was first proposed in [Shamai-Ozarow-Wyner 91]: I I(x 0 ; x 0 + m) = I x (SNR ZF-DFE ) with I x (γ) the MI of a scalar Gaussian channel at SNR γ with input x 0 SNR ZF-DFE the output SNR of the zero-forcing decision feedback equalizer (DFE): SNR ZF-DFE = P { x 1 π ( exp log H(θ) 2) } dθ N 0 2π π Very simple, but quite loose in medium and low SNR s Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

10 Introduction Analytical Lower Bounds on the Information Rate Data Processing with a Mean Square WMF Choose a so that the noise term has minimum variance I I(x 0 ; x 0 + k 1 ˆα kx k + ˆm ) I MMSE }{{} min variance A tight bound in many cases Still difficult to compute and analyze Some techniques for further bounding were proposed Using probability-of-error bounds and Fano s inequality [Shamai-Laroia 96] Using a mismatched mutual information approach [Jeong-Moon 12] However, none of the resulting bounds is both simple and tight Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

11 Introduction Analytical Lower Bounds on the Information Rate Data Processing with a Mean Square WMF Choose a so that the noise term has minimum variance I I(x 0 ; x 0 + k 1 ˆα kx k + ˆm ) I MMSE }{{} min variance A tight bound in many cases Still difficult to compute and analyze Some techniques for further bounding were proposed Using probability-of-error bounds and Fano s inequality [Shamai-Laroia 96] Using a mismatched mutual information approach [Jeong-Moon 12] However, none of the resulting bounds is both simple and tight Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

12 Introduction The Shamai-Laroia Conjecture The Shamai-Laroia Conjecture (SLC) [Shamai-Laroia 96] conjectured that I MMSE is lower bounded by replacing x 1 with g 1, i.i.d. Gaussian of equal variance: I MMSE = I(x 0 ; x 0 + k 1 ˆα kx k + ˆm) I(x 0 ; x 0 + k 1 ˆα kg k + ˆm) = I x (SNR MMSE-DFE-U ) I SL SNR MMSE-DFE-U is the output SNR of the unbiased MMSE DFE: { 1 π ( SNR MMSE-DFE-U = exp log 1 + P ) } x H(θ) 2 dθ 1 2π N 0 I SL a simple, tight and useful approximation for I MMSE, I π Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

13 Introduction The Shamai-Laroia Conjecture (SLC) The Shamai-Laroia Conjecture Example BPSK input, h = [0.408, 0.817, 0.408] (moderate ISI severity) IMMSE ISLC Gaussian upper bound Shamai-Ozarow-Wyner lower bound Information [bits] Px/N0 [db] Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

14 Introduction The Shamai-Laroia Conjecture (SLC) The Shamai-Laroia Conjecture Example, cont BPSK input, h = [0.408, 0.817, 0.408] (moderate ISI severity) Information [bits] Information [bits] Px/N0 [db] Px/N0 [db] 1.2 IMMSE ISLC Gaussian upper bound Shamai-Ozarow-Wyner lower bound 1 Y. Carmon and S. Shamai 0.8 On the Shamai-Laroia Approximation ITA / 31

15 Introduction The Strong SLC and its Refutation The Shamai-Laroia Conjecture (SLC) A stronger version of the SLC reads I(x 0 ; x 0 + k 1 α kx k + m) I(x 0 ; x 0 + k 1 α kg k + m) for every α and m (not just ˆα and ˆm) [Abbe-Zheng 12] gave a counterexample Based on a geometrical tool using the Hermite polynomials Cannot straightforwardly refute the original SLC Uses continuous-valued input distributions (finite alphabet is more interesting) Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

16 Introduction The Strong SLC and its Refutation The Shamai-Laroia Conjecture (SLC) A stronger version of the SLC reads I(x 0 ; x 0 + k 1 α kx k + m) I(x 0 ; x 0 + k 1 α kg k + m) for every α and m (not just ˆα and ˆm) [Abbe-Zheng 12] gave a counterexample Based on a geometrical tool using the Hermite polynomials Cannot straightforwardly refute the original SLC Uses continuous-valued input distributions (finite alphabet is more interesting) Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

17 Outline of Results Introduction The Shamai-Laroia Conjecture (SLC) We present the following, Low SNR Analysis For sufficiently low SNR, I SL > I MMSE (essentially always), disproving the original SLC Counterexample Carefully constructed clearly disprove the SLC Moreover, I SL > I is shown (in a pathological case) High SNR Analysis For sufficiently high SNR, I > I SL (for finite entropy source) Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

18 Low SNR Universal Low SNR Behavior Define the normalized comulants of our (zero-mean) input: Skewness s x = Ex 3 0/[Ex 2 0] 3/2 Excess Kurtosis κ x = Ex 4 0/(Ex 2 0) 2 3 For s x 0, I MMSE I SL = 1 6 C 3s 2 xɛ 3 + O(ɛ 4 ) For s x = 0, I MMSE I SL = 1 24 C 4κ 2 xɛ 4 + O(ɛ 5 ) where k 1 ˆα kx k + ˆm the minimum variance noise term C m = k 1 ˆαm k / ( k 0 ˆα2 k ) m ɛ = k 0 ˆα2 k P x/e ˆm 2 the smallness parameter ɛ 0 as P x /N 0 0 Hence, I MMSE < I SL for sufficiently low P x /N 0 Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

19 Low SNR Universal Low SNR Behavior Define the normalized comulants of our (zero-mean) input: Skewness s x = Ex 3 0/[Ex 2 0] 3/2 Excess Kurtosis κ x = Ex 4 0/(Ex 2 0) 2 3 For s x 0, I MMSE I SL = 1 6 C 3s 2 xɛ 3 + O(ɛ 4 ) For s x = 0, I MMSE I SL = 1 24 C 4κ 2 xɛ 4 + O(ɛ 5 ) where k 1 ˆα kx k + ˆm the minimum variance noise term C m = k 1 ˆαm k / ( k 0 ˆα2 k ) m ɛ = k 0 ˆα2 k P x/e ˆm 2 the smallness parameter ɛ 0 as P x /N 0 0 Hence, I MMSE < I SL for sufficiently low P x /N 0 Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

20 Low SNR Universal Low SNR Behavior Define the normalized comulants of our (zero-mean) input: Skewness s x = Ex 3 0/[Ex 2 0] 3/2 Excess Kurtosis κ x = Ex 4 0/(Ex 2 0) 2 3 For s x 0, I MMSE I SL = 1 6 C 3s 2 xɛ 3 + O(ɛ 4 ) For s x = 0, I MMSE I SL = 1 24 C 4κ 2 xɛ 4 + O(ɛ 5 ) where k 1 ˆα kx k + ˆm the minimum variance noise term C m = k 1 ˆαm k / ( k 0 ˆα2 k ) m ɛ = k 0 ˆα2 k P x/e ˆm 2 the smallness parameter ɛ 0 as P x /N 0 0 Hence, I MMSE < I SL for sufficiently low P x /N 0 Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

21 Low SNR Universal Low SNR Behavior Define the normalized comulants of our (zero-mean) input: Skewness s x = Ex 3 0/[Ex 2 0] 3/2 Excess Kurtosis κ x = Ex 4 0/(Ex 2 0) 2 3 For s x 0, I MMSE I SL = 1 6 C 3s 2 xɛ 3 + O(ɛ 4 ) For s x = 0, I MMSE I SL = 1 24 C 4κ 2 xɛ 4 + O(ɛ 5 ) where k 1 ˆα kx k + ˆm the minimum variance noise term C m = k 1 ˆαm k / ( k 0 ˆα2 k ) m ɛ = k 0 ˆα2 k P x/e ˆm 2 the smallness parameter ɛ 0 as P x /N 0 0 Hence, I MMSE < I SL for sufficiently low P x /N 0 Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

22 Outline of Proof Low SNR Using the chain rule, rewrite I MMSE as I MMSE = I 0 MMSE I1 MMSE, where I i MMSE I( k i ˆα kx k ; k i ˆα kx k + ˆm) and α 0 1 By [Guo-Wu-Shamai-Verdú 11], for any RV ξ and independent Gaussian ν, [ ] I(ξ ; ξ + ν) = ρ 2 ρ2 4 + ρ3 2 s2 ξ ρ4 [ κ ξ 12s 2 ξ + 6] + O ( ρ 5) with ρ = Eξ 2 /Eν 2 Apply the above series expansion to I 0 MMSE, I1 MMSE and I SL to obtain the desired result Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

23 Outline of Proof Low SNR Using the chain rule, rewrite I MMSE as I MMSE = I 0 MMSE I1 MMSE, where I i MMSE I( k i ˆα kx k ; k i ˆα kx k + ˆm) and α 0 1 By [Guo-Wu-Shamai-Verdú 11], for any RV ξ and independent Gaussian ν, [ ] I(ξ ; ξ + ν) = ρ 2 ρ2 4 + ρ3 2 s2 ξ ρ4 [ κ ξ 12s 2 ξ + 6] + O ( ρ 5) with ρ = Eξ 2 /Eν 2 Apply the above series expansion to I 0 MMSE, I1 MMSE and I SL to obtain the desired result Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

24 Counterexamples Counterexample Settings Inter-symbol interference Same as in the previous example (Moderate ISI severity) A three taps impulse response, h 0 = h 2 = 0.408, h 1 = Channel B from Chapter 10 of [Proakis 87] Input distribution 1 Symmetric trinary alphabet { 1, 0, 1} Pr(x = 1) = Pr(x = 1) = 0.01, Pr(x = 0) = 0.98 Zero skewness, high excess kurtosis: κ x = 47 Input distribution 2 Highly skewed binary distribution Pr(x > 0) = 1 Pr(x < 0) = s x 22.3, κ x 495 Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

25 Counterexamples Counterexample Settings Inter-symbol interference Same as in the previous example (Moderate ISI severity) A three taps impulse response, h 0 = h 2 = 0.408, h 1 = Channel B from Chapter 10 of [Proakis 87] Input distribution 1 Symmetric trinary alphabet { 1, 0, 1} Pr(x = 1) = Pr(x = 1) = 0.01, Pr(x = 0) = 0.98 Zero skewness, high excess kurtosis: κ x = 47 Input distribution 2 Highly skewed binary distribution Pr(x > 0) = 1 Pr(x < 0) = s x 22.3, κ x 495 Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

26 Counterexamples Demonstration of Low SNR Behaviour Numerical computation agrees with low-snr expansion for up to ɛ 0.02 At low SNR s and moderate to high ISI, I SL I MMSE 10 8 s 2 x κ 2 x Hence, s x 1 or κ x 1 is a must Difference in information [bits] 2 Explains why similar low-snr 4 counterexamples were not previously reported 6 0 x 10 6 Symmetric Trinary input IMMSE ISL Low SNR approximation ɛ (right scale) Px/N0 [db] Difference in information [bits] Difference in information [bits] x 10 6 Symmetric Trinary input IMMSE ISL 0.01 Low SNR approximation ɛ (right scale) Px/N0 [db] x 10 4 Skewed Binary Input Px/N0 [db] Difference in information [bits] Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

27 Counterexamples Demonstration of Low SNR Behaviour Numerical computation agrees with low-snr expansion for up to ɛ 0.02 At low SNR s and moderate to high ISI, I SL I MMSE 10 8 s 2 x κ 2 x Hence, s x 1 or κ x 1 is a must Difference in information [bits] 2 Explains why similar low-snr 4 counterexamples were not previously reported 6 0 x 10 6 Symmetric Trinary input IMMSE ISL Low SNR approximation ɛ (right scale) Px/N0 [db] Difference in information [bits] Difference in information [bits] x 10 6 Symmetric Trinary input IMMSE ISL 0.01 Low SNR approximation ɛ (right scale) Px/N0 [db] x 10 4 Skewed Binary Input Px/N0 [db] Difference in information [bits] Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

28 Counterexamples Demonstration of Low SNR Behaviour Numerical computation agrees with low-snr expansion for up to ɛ 0.02 At low SNR s and moderate to high ISI, I SL I MMSE 10 8 s 2 x κ 2 x Hence, s x 1 or κ x 1 is a must Difference in information [bits] 2 Explains why similar low-snr 4 counterexamples were not previously reported 6 0 x 10 6 Symmetric Trinary input IMMSE ISL Low SNR approximation ɛ (right scale) Px/N0 [db] Difference in information [bits] Difference in information [bits] x 10 6 Symmetric Trinary input IMMSE ISL 0.01 Low SNR approximation ɛ (right scale) Px/N0 [db] x 10 4 Skewed Binary Input Px/N0 [db] Difference in information [bits] Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

29 Counterexamples Higher SNR s Trinary Source Information [bits] P x /N 0 [db] I evaluated via Monte-Carlo method I MMSE I SL I I SL significantly exceeds I MMSE! (but not I) Information [bits] Qualitative explanation The noise k 1 ˆα kx k + ˆm has excess kurtosis proportional to κ x The SL approximating noise term has excess kurtosis 0 (it s Gaussian) Quality of the approximation is expected to deteriorate as κ x grows 14 P x /N Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

30 Counterexamples Higher SNR s Trinary Source Information [bits] P x /N 0 [db] I evaluated via Monte-Carlo method I MMSE I SL I I SL significantly exceeds I MMSE! (but not I) Information [bits] Qualitative explanation The noise k 1 ˆα kx k + ˆm has excess kurtosis proportional to κ x The SL approximating noise term has excess kurtosis 0 (it s Gaussian) Quality of the approximation is expected to deteriorate as κ x grows 14 P x /N Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

31 Counterexamples Higher SNR s Trinary Source Information [bits] P x /N 0 [db] I evaluated via Monte-Carlo method I MMSE I SL I I SL significantly exceeds I MMSE! (but not I) Information [bits] Qualitative explanation The noise k 1 ˆα kx k + ˆm has excess kurtosis proportional to κ x The SL approximating noise term has excess kurtosis 0 (it s Gaussian) Quality of the approximation is expected to deteriorate as κ x grows 14 P x /N Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

32 Counterexamples Higher SNR s Trinary Source Information [bits] P x /N 0 [db] I evaluated via Monte-Carlo method I MMSE I SL I I SL significantly exceeds I MMSE! (but not I) Information [bits] Qualitative explanation The noise k 1 ˆα kx k + ˆm has excess kurtosis proportional to κ x The SL approximating noise term has excess kurtosis 0 (it s Gaussian) Quality of the approximation is expected to deteriorate as κ x grows 14 P x /N Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

33 Counterexamples Higher SNR s Skewed Binary Source [db] I MMSE I SL I 2 4 Information [bits] I MMSE I SL I P x /N 0 [db] I evaluated via Monte-Carlo method This time I SL > I as well as I SL > I MMSE Thus, even the relaxed SLC, I I SL, does not hold for all SNR s Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

34 Counterexamples Higher SNR s Skewed Binary Source [db] I MMSE I SL I 2 4 Information [bits] I MMSE I SL I P x /N 0 [db] I evaluated via Monte-Carlo method This time I SL > I as well as I SL > I MMSE Thus, even the relaxed SLC, I I SL, does not hold for all SNR s Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

35 High SNR Universal High SNR Behavior For a finite entropy input distribution, let R[I] = lim log[h(x 0) I] P x/n 0 P x /N 0 be the rate of convergence to the input entropy For non-trivial ISI, we show that R[I SL ] 1 ( ) 2 dmin g ZF-DFE < ( min 2 ) 2 R[I] where d min is the minimum distance between (unit-power) input values g ZF-DFE = ( Px N 0 ) 1 SNR ZF-DFE is the zero-forcing DFE gain factor min is a free distance associated with optimal equalization Hence, I > I SL for sufficiently high P x /N 0 Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

36 High SNR Universal High SNR Behavior For a finite entropy input distribution, let R[I] = lim log[h(x 0) I] P x/n 0 P x /N 0 be the rate of convergence to the input entropy For non-trivial ISI, we show that R[I SL ] 1 ( ) 2 dmin g ZF-DFE < ( min 2 ) 2 R[I] where d min is the minimum distance between (unit-power) input values g ZF-DFE = ( Px N 0 ) 1 SNR ZF-DFE is the zero-forcing DFE gain factor min is a free distance associated with optimal equalization Hence, I > I SL for sufficiently high P x /N 0 Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

37 High SNR Universal High SNR Behavior For a finite entropy input distribution, let R[I] = lim log[h(x 0) I] P x/n 0 P x /N 0 be the rate of convergence to the input entropy For non-trivial ISI, we show that R[I SL ] 1 ( ) 2 dmin g ZF-DFE < ( min 2 ) 2 R[I] where d min is the minimum distance between (unit-power) input values g ZF-DFE = ( Px N 0 ) 1 SNR ZF-DFE is the zero-forcing DFE gain factor min is a free distance associated with optimal equalization Hence, I > I SL for sufficiently high P x /N 0 Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

38 High SNR Outline of proof Convergence Rate of I SL Information-Estimation Identity for H(x 0 ) I SL Recall that I SL = I x (SNR MMSE-DFE-U ) By [Guo-Shamai-Verdú 05] (assuming Ex 2 = 1), where H (x) I x (snr) = 1 2 snr mmse x (γ) dγ mmse x (γ) E (x E [x γx + n]) 2 with n N (0, 1) and independent of x We therefore need to find an exponentially tight lower bound for mmse x Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

39 High SNR Outline of proof Convergence Rate of I SL Information-Estimation Identity for H(x 0 ) I SL Recall that I SL = I x (SNR MMSE-DFE-U ) By [Guo-Shamai-Verdú 05] (assuming Ex 2 = 1), where H (x) I x (snr) = 1 2 snr mmse x (γ) dγ mmse x (γ) E (x E [x γx + n]) 2 with n N (0, 1) and independent of x We therefore need to find an exponentially tight lower bound for mmse x Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

40 High SNR Outline of proof Convergence Rate of I SL Information-Estimation Identity for H(x 0 ) I SL Recall that I SL = I x (SNR MMSE-DFE-U ) By [Guo-Shamai-Verdú 05] (assuming Ex 2 = 1), where H (x) I x (snr) = 1 2 snr mmse x (γ) dγ mmse x (γ) E (x E [x γx + n]) 2 with n N (0, 1) and independent of x We therefore need to find an exponentially tight lower bound for mmse x Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

41 High SNR Lower bound on mmse x (γ) Outline of proof Convergence Rate of I SL We construct a Genie 1 G distributed on {0, 1} such that given G = 1, x is uniformly distributed two values with distance d min We have, mmse x (γ) Pr (G = 1) mmse x G=1 (γ) and mmse x G=1 (γ) = ( dmin ) ( 2 (dmin ) 2 mmse b γ) 2 2 where b is uniformly distributed on { 1, 1} Finally, mmse b (ρ) 2Q ( ρ) 1 This is based on the analysis in [Lozano-Tulino-Verdú 06], but slightly altered to accommodate non-uniformly distributed inputs Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

42 High SNR Lower bound on mmse x (γ) Outline of proof Convergence Rate of I SL We construct a Genie 1 G distributed on {0, 1} such that given G = 1, x is uniformly distributed two values with distance d min We have, mmse x (γ) Pr (G = 1) mmse x G=1 (γ) and mmse x G=1 (γ) = ( dmin ) ( 2 (dmin ) 2 mmse b γ) 2 2 where b is uniformly distributed on { 1, 1} Finally, mmse b (ρ) 2Q ( ρ) 1 This is based on the analysis in [Lozano-Tulino-Verdú 06], but slightly altered to accommodate non-uniformly distributed inputs Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

43 High SNR Lower bound on mmse x (γ) Outline of proof Convergence Rate of I SL We construct a Genie 1 G distributed on {0, 1} such that given G = 1, x is uniformly distributed two values with distance d min We have, mmse x (γ) Pr (G = 1) mmse x G=1 (γ) and mmse x G=1 (γ) = ( dmin ) ( 2 (dmin ) 2 mmse b γ) 2 2 where b is uniformly distributed on { 1, 1} Finally, mmse b (ρ) 2Q ( ρ) 1 This is based on the analysis in [Lozano-Tulino-Verdú 06], but slightly altered to accommodate non-uniformly distributed inputs Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

44 High SNR Putting everything together Outline of proof Convergence Rate of I SL Using the above results, ( C H (x 0 ) I SL exp 1 Px /N 0 2 ( dmin for some C > 0 and sufficiently high P x /N 0 It is straightforward (but technical) to show that SNR MMSE-DFE-U P x N 0 g ZF-DFE + K for some K, ε > 0 and sufficiently high P x /N 0 This establishes R[I SL ] = lim log[h(x 0) I SL ] P x/n 0 P x /N 0 2 ) 2 SNR MMSE-DFE-U) ( ) 1 ε Px N 0 ( dmin 2 ) 2 g ZF-DFE Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

45 High SNR Putting everything together Outline of proof Convergence Rate of I SL Using the above results, ( C H (x 0 ) I SL exp 1 Px /N 0 2 ( dmin for some C > 0 and sufficiently high P x /N 0 It is straightforward (but technical) to show that SNR MMSE-DFE-U P x N 0 g ZF-DFE + K for some K, ε > 0 and sufficiently high P x /N 0 This establishes R[I SL ] = lim log[h(x 0) I SL ] P x/n 0 P x /N 0 2 ) 2 SNR MMSE-DFE-U) ( ) 1 ε Px N 0 ( dmin 2 ) 2 g ZF-DFE Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

46 High SNR Putting everything together Outline of proof Convergence Rate of I SL Using the above results, ( C H (x 0 ) I SL exp 1 Px /N 0 2 ( dmin for some C > 0 and sufficiently high P x /N 0 It is straightforward (but technical) to show that SNR MMSE-DFE-U P x N 0 g ZF-DFE + K for some K, ε > 0 and sufficiently high P x /N 0 This establishes R[I SL ] = lim log[h(x 0) I SL ] P x/n 0 P x /N 0 2 ) 2 SNR MMSE-DFE-U) ( ) 1 ε Px N 0 ( dmin 2 ) 2 g ZF-DFE Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

47 High SNR Outline of proof Convergence Rate of I Data Processing with the ML Sequence Detector Recall that I = I(x 0 ; y x 1 ) Since conditioning decreases entropy H (x 0 ) I = H ( x 0 y, x 1 ) ( H x0 ˆx ML ) 0 } where {ˆx ML i is the maximum likelihood sequence estimate of i= x given y By Fano s inequality H ( x 0 ˆx ML ) ( ( 0 h2 Pr x0 ˆx ML )) ( 0 + Pr x0 ˆx ML ) 0 log X with h 2 (x) the binary entropy function and X the input alphabet cardinality Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

48 High SNR Outline of proof Convergence Rate of I Data Processing with the ML Sequence Detector Recall that I = I(x 0 ; y x 1 ) Since conditioning decreases entropy H (x 0 ) I = H ( x 0 y, x 1 ) ( H x0 ˆx ML ) 0 } where {ˆx ML i is the maximum likelihood sequence estimate of i= x given y By Fano s inequality H ( x 0 ˆx ML ) ( ( 0 h2 Pr x0 ˆx ML )) ( 0 + Pr x0 ˆx ML ) 0 log X with h 2 (x) the binary entropy function and X the input alphabet cardinality Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

49 High SNR Outline of proof Convergence Rate of I Data Processing with the ML Sequence Detector Recall that I = I(x 0 ; y x 1 ) Since conditioning decreases entropy H (x 0 ) I = H ( x 0 y, x 1 ) ( H x0 ˆx ML ) 0 } where {ˆx ML i is the maximum likelihood sequence estimate of i= x given y By Fano s inequality H ( x 0 ˆx ML ) ( ( 0 h2 Pr x0 ˆx ML )) ( 0 + Pr x0 ˆx ML ) 0 log X with h 2 (x) the binary entropy function and X the input alphabet cardinality Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

50 High SNR Forney s Error Probability Bound Outline of proof Convergence Rate of I In [Forney 72] it was shown that Pr ( x 0 ˆx ML 0 ) K Q P x N 0 ( min 2 ) 2 for some K > 0 and 2 min = inf N 1 min x N 1 0, x N 1 0 s.t. x 0 x 0, x N 1 x N 1 ( ) 2 x N 1 0, x N 1 0 where ( ) 2 x N 1 0, x N 1 0 = L+N 2 k=0 N 1 2 (x l x l ) h k l is the Euclidean distance between finite sequences h x, h x l=0 Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

51 High SNR Outline of proof Convergence Rate of I Bounding min with g ZF-DFE Putting Fano and Forney together, we establish R[I] = lim log[h(x 0) I] 1 P x/n 0 P x /N 0 2 ( ) 2 min Finally, notice that for any non-trivial ISI channel (L > 1) ( ) 2 x N 1 0, x N 1 0 (x 0 x 0 ) h (x N 1 x N 1 ) h L 1 2 ( d 2 min h h L 1 2) > d 2 min h 0 2 Assuming w.l.o.g. h0 L 1 is minimal phase, { 1 π ( h 0 = exp log H(θ) 2) } dθ 2π π Hence, 2 min > d2 min g ZF-DFE 2 = g ZF-DFE Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

52 High SNR Outline of proof Convergence Rate of I Bounding min with g ZF-DFE Putting Fano and Forney together, we establish R[I] = lim log[h(x 0) I] 1 P x/n 0 P x /N 0 2 ( ) 2 min Finally, notice that for any non-trivial ISI channel (L > 1) ( ) 2 x N 1 0, x N 1 0 (x 0 x 0 ) h (x N 1 x N 1 ) h L 1 2 ( d 2 min h h L 1 2) > d 2 min h 0 2 Assuming w.l.o.g. h0 L 1 is minimal phase, { 1 π ( h 0 = exp log H(θ) 2) } dθ 2π π Hence, 2 min > d2 min g ZF-DFE 2 = g ZF-DFE Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

53 High SNR Outline of proof Convergence Rate of I Bounding min with g ZF-DFE Putting Fano and Forney together, we establish R[I] = lim log[h(x 0) I] 1 P x/n 0 P x /N 0 2 ( ) 2 min Finally, notice that for any non-trivial ISI channel (L > 1) ( ) 2 x N 1 0, x N 1 0 (x 0 x 0 ) h (x N 1 x N 1 ) h L 1 2 ( d 2 min h h L 1 2) > d 2 min h 0 2 Assuming w.l.o.g. h0 L 1 is minimal phase, { 1 π ( h 0 = exp log H(θ) 2) } dθ 2π π Hence, 2 min > d2 min g ZF-DFE 2 = g ZF-DFE Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

54 Conclusion Conclusion Conjectured in 1996, the SLC I MMSE I SL is now finally disproved I I SL also doesn t hold in general (skewed binary input) However, it will hold above some SNR threshold In most cases, it seems this threshold is 0... Open issues Can a threshold independent on the ISI channel be found? For conventional inputs, such as BPSK, can the threshold be proven to be very low or even 0? A new use for I SL : upper bound on the OFDM information rate w/ i.i.d. inputs more details at IZS 2014 (or online)! Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

55 Conclusion Conclusion Conjectured in 1996, the SLC I MMSE I SL is now finally disproved I I SL also doesn t hold in general (skewed binary input) However, it will hold above some SNR threshold In most cases, it seems this threshold is 0... Open issues Can a threshold independent on the ISI channel be found? For conventional inputs, such as BPSK, can the threshold be proven to be very low or even 0? A new use for I SL : upper bound on the OFDM information rate w/ i.i.d. inputs more details at IZS 2014 (or online)! Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

56 Conclusion Conclusion Conjectured in 1996, the SLC I MMSE I SL is now finally disproved I I SL also doesn t hold in general (skewed binary input) However, it will hold above some SNR threshold In most cases, it seems this threshold is 0... Open issues Can a threshold independent on the ISI channel be found? For conventional inputs, such as BPSK, can the threshold be proven to be very low or even 0? A new use for I SL : upper bound on the OFDM information rate w/ i.i.d. inputs more details at IZS 2014 (or online)! Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

57 References I References D.M. Arnold, H.A. Loeliger, P.O. Vontobel, A. Kavcic and W. Zeng Simulation-based computation of information rates for channels with memory Information Theory, IEEE Transactions on, 52(8): , S. Shamai, L.H. Ozarow, and A.D. Wyner Information rates for a discrete-time Gaussian channel with intersymbol interference and stationary inputs Information Theory, IEEE Transactions on, 37(6): , S. Shamai and R. Laroia The intersymbol interference channel: Lower bounds on capacity and channel precoding loss Information Theory, IEEE Transactions on, 42(5): , S. Jeong and J. Moon Easily computed lower bounds on the information rate of intersymbol interference channels Information Theory, IEEE Transactions on, 58(2): , E. Abbe and L. Zheng A coordinate system for Gaussian Networks Information Theory, IEEE Transactions on, 58(2): , Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

58 References II References Dongning Guo, Yihong Wu, Shlomo Shamai, and Sergio Verdú Estimation in gaussian noise: Properties of the minimum mean-square error Information Theory, IEEE Transactions on, 57(4): , J.G. Proakis Digital communications McGraw-hill, D. Guo, S. Shamai, and S. Verdú Mutual information and minimum mean-square error in gaussian channels Information Theory, IEEE Transactions on, 51(4): , A. Lozano, A.M. Tulino, and S. Verdú Optimum power allocation for parallel gaussian channels with arbitrary input distributions Information Theory, IEEE Transactions on, 52(7): , G. Forney Jr. Maximum-likelihood sequence estimation of digital sequences in the presence of intersymbol interference Information Theory, IEEE Transactions on, 18(3): , Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

59 Thank You Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31

60 Yair Carmon and Shlomo Shamai (Shitz) On the Shamai-Laroia Approximation for the Information Rate of the ISI Channel Abstract: The approximation proposed by Shamai and Laroia for the achievable rate in the intersymbol interference (ISI) channel with fixed i.i.d. inputs is considered. We investigate the conjecture that this approximation is a lower bound on the achievable rate. This conjecture is a slightly relaxed version of the original Shamai-Laroia conjecture, which has recently been disproved. It is shown that for any discrete input distribution and ISI channel, the lower bound holds in the high-snr regime. Numerical evidence indicates, however, that even this relaxed version of the conjecture does not hold in general. Y. Carmon and S. Shamai On the Shamai-Laroia Approximation ITA / 31