A Relation between Conditional and Unconditional Soft Bit Densities of Binary Input Memoryless Symmetric Channels

Transcription

1 A Relation between Conditional and Unconditional Soft Bit Densities of Binary Input Memoryless Symmetric Channels Wolfgang Rave Vodafone Chair Mobile Communications Systems, Technische Universität Dresden D-6 Dresden, Germany Abstract We show that a general very simple relation holds between the variance of the unconditional and the mean of the conditional expected bit: both are identical for channels that are symmetric and in addition possess the property of exponential symmetry also called consistency) Special cases are the binary erasure channel, the binary symmetric channel or the binary AWGN channel This equality allows to obtain the conditional soft bit mean from observing the unconditional channel or decoder output which is of interest for estimators using expected bit or symbol values We prove our result using results from Sharon et al [], where properties of conditional soft bit densities had been used to study EXIT functions We exploit the relation by applying it to the problem of soft turbo channel estimation I SYMMETRY OF LOG-LIKELIHOOD RATIOS To introduce the problem let us consider log-likelihood ratios also denoted as LLR or L-values) [ ] PrX Y ) L ln, PrX Y ) derived from observations Y at the output of a binary symmetric memoryless channel with equiprobable input In the context of density evolution for low-density parity check codes Richardson et al [] showed that the conditional L-value density f L l) f L X l ) of a r v L X ) of a binary symmetric memoryless channel BMC) has the following exponential symmetry or consistency property which is maintained under convolution): f L l) f L l)e l ) As an alternative to the LLR an estimate of a transmitted antipodally modulated) bit X {±} can also be expressed as the expected or soft bit: ΛX) E{X Y } PrX Y ) PrX Y ) tanhlx)/) Specific cases of interest for BMCs are the binary erasure channel BEC), the binary symmetric channel BSC) or the binary symmetric memoryless AWGN channel biawgnc) Recently we showed in [3] that for the biawgnc the unimodal conditional soft bit density f Λ and the bimodal unconditional density f Λ are connected by the following We denote random variables r v) such as X, LX) and ΛX) with capital letters and their realizations x, l and in lower case relation: the mean µ Λ X of the conditional soft bit distribution is equal to the variance σλ of the unconditional soft bit for every given value of the channel parameter σl 4/σ n of the Gaussian channel y x + n, ie µ Λ X ) σλ ) We also showed that this property is rooted in the consistency property of the associated L-value density Here we show that the converse can be shown much more simply We build on a theorem proved in [] developed in the context of EXIT [4] functions for the BEC and more general BMCs There mutual information between channel input bits X and outputs Y was expressed in terms of moments of the soft bit Λ, but the authors had only considered conditional densities of LLRs and soft bits) and not drawn the connection to unconditional densitities which are the observables usually available at the output of a channel or an APP decoder The rest of the paper is organized as follows: First we review the concept of Λ-consistency called T -consistency in []) and recall some important properties Then we state our main theorem and give a simple proof Examples for the moment identity are given for the BEC, the BSC and the biawgnc for which we also list some dual relations between L-value densities and soft bit densities An application to soft channel estimation in a turbo equalizer follows before we summarize our results II Λ-CONSISTENT RANDOM VARIABLES The consistency property of L-values implies a similar property for another rv that is a function of L Two of these properties denoted as Λ-consistency were presented in [] as proposition 3 and lemma 3: A first consequence of consistency is the relation f Λ f Λ ) + ) This follows directly from the transformation of the density of the r v l note that l ln + )/ )): f Λ dl d f L ln + ) ) l f L ln ) ) + + } {{ l } e l f Λ )

2 A second less obvious consequence is the fact that any even conditional moment equals the next lower odd moment of the conditional soft bit density: E{Λ ) i } i f Λ ) d i f Λ ) d E{Λ ) i } 3) For a proof of this lemma using mirror) symmetry and consistency, see [] Now we are ready to state the relation which will be exploited in section IV to an estimation problem: Theorem : Given a consistent and symmetric density f L l) of log-likelihood ratios based on observations at the output of a binary symmetric and consistent channel, the associated conditional and unconditional densities f Λ X ) and f Λ of the soft bit Λ tanhl/) have the following property: Proof: µ Λ σ Λ 4) σ Λ E{Λ } E{Λ ) } E{Λ )} µ Λ The equality of variance and second moment follows from the symmetry of the unconditional density The second equality is a consequence of the fact that conditioning only affects moments of odd order The cited lemma implies immediately the third equality A Example : BEC III EXAMPLES The channel is defined by its erasure probability ε Describing the discrete L-value density by a pair of sign and magnitude variable [] the sign of a rv with magnitude is ± with probability /), we have the discrete unconditional density f L l) at the sign/magnitude pairs { / ), ±/), / )} with values { ε)/, ε, ε)/} Similarly the conditional density f L l) has values {ε, ε} at {±/), / )} This corresponds to the densities of the expected bit and fλ BEC {,, }) { ε)/, ε, ε)/} fλ BEC {, }) {ε, ε} Calculation of the moments gives µ Λ ε σλ as well as E{Λ ) } ε B Example : BSC The channel is defined by its crossover probability δ Describing the discrete L-value density again by a sign and magnitude variable, we have the unconditional density f L l) with values {/, /} at { / ln δ)/δ)), / ln δ)/δ))} and the conditional density f L l) with values {δ, δ} at { / ln δ)/δ)), / ln δ)/δ))} The corresponding densities of the expected bit are and f BSC Λ { + δ, δ}) {/, /} fλ BSC { + δ, δ}) {δ, δ} We calculate σ Λ δ) and µ Λ δ + δ) + δ) δ) 4δ + 4δ E{Λ ) } C Example 3: biawgnc The most interesting case for applications is probably the binary symmetric additive white Gaussian noise channel for which we present the relations between L- and Λ-values and their associated densities in more detail in the following together with the particular way Theorem appears in this case L-value Density of a Gaussian Channel: Its bimodal unconditional density f L l) of L-values is the superposition of two Gaussian densities Mirror) symmetry relates the two Gaussian modes of the unconditional density, enforcing that the means ±µ L are equal in magnitude but of opposite sign and the variances of both modes are equal Thus we have: f biawgnc L l) /)N µ L, σ L) + /)N µl, σ L) If the consistency property more appropriately called exponential symmetry [5]) holds, the remaining two degrees of freedom mean and variance) for the density of a Gaussian channel become coupled as µ L σl /, implying that the whole density is characterized by the single parameter as f L l) / N σl /, L) σ + N σ L /, σl) ) Assuming antipodal BPSK transmission with symbols x {±} over a Gaussian channel an extension to a flat fading Rayleigh channel is possible) y x + n with noise variance σ n, the L-values are related to the channel output y i by l i /σ n)y i The conditional density f L X l), conditioned on one of the channel inputs x ±, is unimodal Gaussian with variance σ L 4/σ n and given as f L X l) N ± σ L/, σ L) Note, that the variance of the bimodal) unconditional density is σl + σ4 L /4 It has to be distinguished from the variance σl of one of the individual modes and is available at the output of a Gaussian channel when the input is unknown) It is also observed approximately at the output of a soft-in soft-out decoder due to the central limit theorem We use σl as parameter on which the L-value and soft bit densities depend

3 3 Equivalent soft bit density: The density of the expected bit f Λ can be obtained in closed form as f Λ [ / ) df L d f L 5) exp ) atanh /) σ L + exp atanh+ ) /) ] σl Again the density of the conditioned soft bit follows immediately as one branch of the upper equation: atanh ± σ f Λ X ) exp L / ) ) f Λ f Λ X f Λ F Λ σ L f Λ X F Λ X 5 5 x Fig Unconditional two symmetric branches, top) and conditional single branch, bottom) soft bit densities and distributions To illustrate the difference between uni- and bimodal soft bit densities and distributions an exemplary comparison for the parameter value σl 4 is presented in Fig Mean of the Conditional Soft Bit: The mean or first moment of the unimodal soft bit density is by definition µ Λ ) f Λ d 6) σl ) l tanh e l σ L / ) σ L dl tanhx)e x σ L /4 ) σ L /4 dx Variance of the Unconditional Soft Bit: The variance of the bimodal soft bit density equals its second moment due to symmetry, so that σ Λ ) + f Λ d tanh l ) fl l)dl f Λ X d 7) tanh x)e x σ L /4 ) σ L /4 dx 4x σ L )tanhx)e x σ L /4 ) σ L /4 dx The two expression for µ Λ ) and σλ ) are equivalent due to the general proof based on Λ-consistency [] that describes consistency of L-values in the domain of expected bits The alternative proof in the converse direction can be found in [3], showing that consistency is required in order that the equality between 6) and 7) holds As a special case of our general theorem relating conditional soft bit mean and variance of the unconditional soft bit density for the biawgnc we thus have the following statement: Given a consistent and symmetric density f L l) of loglikelihood ratios specified by the single parameter, the associated conditional and unconditional densities f Λ X and f Λ of the soft bit Λ tanhl/) have the property: µ Λ X, σ Λ X, σ Λ µ Λ X ) σ Λ ) 8) µ Λ X σ Λ σ Λ X σ L Fig Mean and variance of conditional and unconditional soft bit densities as a function of the variance σl of the associated L-values A graphical illustration of mean and variances of conditional and unconditional soft bit densitites is presented in Fig The conditional mean µ Λ X increases with increasing reliability σl of the underlying L-values towards exactly) in the same way as the unconditional variance σλ ) that is derived from two branches of the soft bit density In contrast the conditional variance σλ X of a single branch decreases again to zero with increasing σl after passing through a maximum at 4, because the soft bit density becomes concentrated at ± see Fig )

4 4 D Second Order Statistics of Conditional Soft Bits The variance of the conditional soft bit σ Λ X ) can be obtained easily from our previous results as σ Λ X ) E { x) } E { x} σ Λ ) µ Λ X ) µ Λ X ) µ Λ X ) 9) The relations of the second order statistics for conditional L- values and soft bits that connect the observation of the bimodal unconditional densities at the output of the channel or a soft decoder to the conditional densities are collected in Table I The relations due to consistency might be interpreted as LLR-domain Λ-domain µ L X σ L X / consistency µ Λ X σλ µ L mirror µ Λ µ L µ L symmetry µ Λ µ Λ σl σλ σ Λ σl X + σ4 L X /4 σ Λ X µ Λ X µ Λ X σλ σ4 Λ σ L X + σl σ Λ + 4σΛ X ) TABLE I DUALITY BETWEEN LLR AND SOFT BIT DENSITIES OF A BINARY INPUT SYMMETRIC AND CONSISTENT AWGN CHANNEL constituent equations which, together with the trivial mirror symmetry, lead to the consequences stated in the third row note that σλ X does not exceed 5 for a bimodal Gaussian density of L-values) IV APPLICATION: SOFT TURBO CHANNEL ESTIMATION As an application of identity 8) we consider the problem of iterative soft channel estimation in a turbo equalizer [6] as it was treated eg by Song et al in [7] Our method is based on interference cancelation and correlation which we denote as IC-Correlation Essentially it is the same idea as presented in [8] but refined to a recursive version Assume that apart from the received symbol vector r a vector of corresponding L-values based on the decoder output for the current codeword c is available as well as the channel impulse response estimate ĥ from the last iteration of the turbo equalizer The received symbols are given as the convolution sum between channel impulse response h length L) and a block of transmitted data symbols s length N, assumed iid with symbol power σs) disturbed by zero mean additive Gaussian noise samples w[n]: r[n] h[l]s[n l] + L i,i l h[i]s[n i] } {{ } ISI for s[n l] + w[n] AWGN ) Computing expected symbols ŝ[n] from the bit probabilities Prc[k] ) / + e L[k] )) allows to compute intersymbol interference ISI) reduced receive symbols r l) [n] for each observation sample index n) and associated channel tap h[l]: r l) [n] r[n] L i,i l ĥ[i]ŝ[n i] h[l]s[n l] + ISI + w[n] w[n] ) Correlating the available tap specific sequence of ISI reduced symbols with corresponding delayed symbol estimates ŝ[n l] and exploiting that w[n] is zero mean we get { } E ŝ [n l] r l) [n] E {ŝ [n l] h[l]s[n l] + w[n])} h[l]e {ŝ [n l]s[n l]} ) This estimate is biased due to the reduced average magnitude of the expected symbol E{ ŝ[n] } making E{ ŝ[n] s[n] } < σs However, for symbol alphabets with constant magnitude such as M-PSK this bias can be compensated for applying the soft bit moment identity, ie by scaling the estimate with the online available estimate of the conditional soft bit mean For fixed soft decoder output an unbiased estimate of the channel is obtained based on the expectation in ) that can be approximated with the arithmetic mean ĥ[l] N ˆµ Λ X σs N ŝ [n l] r l) [n] 3) n More efficiently we compute the tap estimates recursively by forming a growing arithmetic mean of the individual correlation results To this end see Algorithm below) the weight a is used to update the arithmetic mean evaluation from observation n to n by calculating /n from /n ) in the third line 3 This also allows the immediate reuse of the improving estimates in the ISI cancelation Line 6 cancels the ISI in the received symbol for tap L while in line 7 a single correlation estimate is obtained Algorithm Recursive Soft Interference Canceler Correlator Require: L, N,r[ : N + L ],ŝ[ : N + L ],h init [ : L], ˆµ Λ X, σs : initialize ĥ[l L ] h init, a : for n L to N + L ) do 3: a a/ + a) 4: // ISI reduction and correlation 5: for l to L ) do 6: r l) [n] r[n] L ĥ[i] ŝ[n i] i,i l 7: ĥ[l] new ŝ [n l] r l) [n] / ˆµ Λ X σs ) 8: end for 9: // estimator update : ĥ[n n] ĥ[n n ] + aĥnew ĥ[n n ]) : end for : return ĥ[n + L N + L ] This is the simplest possibility, alternatively a weighting with the inverse soft symbol variance combined with the noise variance σ w can be used 3 If we replace a/ + a) by a/ + a) with a forgetting factor the algorithm can already be used for tracking here the parameter should not be confused with the expected bit)

5 5 Line combines the new estimate of the channel impulse response based on r[n] with the accumulated previous estimates updating the arithmetic mean of the correlation results An extension to larger alphabets is possible and is described in [8]) for linear mappings 4 In summary the algorithm that we propose acts as an adaptive filter and is described by the pseudo code in Algorithm a tilde indicates residual quantities, a hat above a symbol denotes an estimate) Performance example: We compare our method with the soft Kalman channel estimator of Song et al described in [7]) see Eqs 8)-3) In that work the performance was studied for an L 4 tap Rayleigh fading channel with equal power delay profile which we use as a benchmark A reproduction of the channel estimation results green curve) shown as Fig in [7] for BPSK transmission at an SNR of db and a block length of N symbols is presented in Fig 3 and compared to our proposed algorithm red curve) For orientation we also show the upper and lower bounds derived for the mean-square identification error MSIE) All curves are plotted as a function of the soft bit reliability standard deviation of the conditional Gaussian L-value density): MSIE 3 lower bound soft Kalman channel estimator SNR db Rayleigh fading equal PDP, L 4, N upper bound soft IC correlator Fig 3 Comparison of the mean-square identification error of the soft Kalman estimator [7] and the IC-Correlation method as a function of the standard deviation of the L-values for a four tap equal PDP fading channel Note that for 4 parameters to be estimated L 4), SNR db and N observations the Cramér-Rao lower bound becomes MSIE 4 4 which is achieved for high soft bit reliability, ie We observe a small performance advantage for the Kalman approach with respect to the IC-Correlation method in the transition region of intermediate reliability under these boundary conditions note that the asymptotic behaviour for large is the same) However, the performance difference appears to be small the MSIE has to be seen in relation to the noise power σw, ie the operating point of the code) and both methods are close to the upper bound 4 Also for nonlinear mappings such as Gray mapping for QAMconstellations up to 56-QAM we checked that the error when scaling the correlation estimate using µ Λ X stays smaller than 5% for > We attribute the small performance difference to the fact that the soft Kalman estimator uses a time-variant noise variance by taking the instantaneous soft symbol variance in the interference cancelation into account This leads to a better weighting of the individual estimates contributing to the combined estimate On the other hand the complexity of the Kalman estimator is of the order OL 3 N) due to the update of the error covariance matrix of size L L) while the operation count for the IC-Correlator scales as OLN), because the interference reduction in line 6 of algorithm can be computed more efficiently by subtracting the whole signal related part completely before the inner loop and adding just one term again inside it Therefore the latter approach appears to be at least an interesting alternative V SUMMARY We demonstrated a simple relation between the mean of the conditional and the variance of unconditional soft bit densities that is valid for general binary mirror) symmetric and consistent memoryless channels This observation provides an interesting connection between coding and estimation theory similar to the observation in [9] that the computation of EXIT charts is possible by only observing the magnitude of the L- values As a specific example for the application of our main result we presented an unbiased soft channel estimator based on interference cancelation and correlation that can be used in a turbo equalizer We showed that it appears to be competitive compared to the established method of the soft Kalman channel estimator Acknowledgment: This work was supported by the German Science Foundation DFG) within the priority programme UKoLoS under project grant Fe43/- REFERENCES [] E Sharon, A Ashikhmin and S Litsyn, EXIT Function for Binary Input Memoryless Symmetric Channels,, IEEE Trans Comm, vol 54, pp 7-4, 6 [] T J Richardson, A Shokrollahi, and R Urbanke, Design of provably good low-density parity-check codes, IEEE Trans Inf Theory, vol IT- 47, pp , Feb [3] W Rave, On a Property of the Soft Bit Distribution associated to Log- Likelihood Ratios, ITG Conf on Source and Channel Coding, SCC, Siegen, Germany, [4] S ten Brink, Convergence behavior of iteratively decoded parallel concatenated codes,, IEEE Trans Comm, vol 49, pp 77-37, [5] P Hoeher, I Land, and U Sorger, Log-likelihood values and Monte Carlo simulation - some fundamental results, Proc Int Symp on Turbo Codes & Related Topics, Brest, France, pp 43-46, Sep [6] C Douillard, A Picart, P Didier, M Jezequel, C Berrou and A Glavieux, Iterative Correction of Intersymbol Interference: Turbo Equalization, European Trans on Telecomm, vol 6, no 5, pp 57-5, 995 [7] S Song, A C Singer and K-M Sung, Soft Input Channel Estimation for Turbo Equalization, IEEE Trans Sig Proc vol 5, pp , 4 [8] A Santos, W Rave and G Fettweis, Turbo Channel Estimation using the Iterative Soft Correlator Estimator I-SCE),, subm to IEEE Trans Sig Proc [9] I Land, P A Hoeher, S Gligorevic, Computation of Symbol-Wise Mutual Information in Transmission Systems with LogAPP Decoders and Application to EXIT Charts, ITG Conf on Source and Channel Coding, SCC 6, Erlangen, Germany, 4