Research Article Rapidly-Converging Series Representations of a Mutual-Information Integral

Transcription

1 International Scholarly Research Network ISRN Counications and Networking Volue 11, Article ID 5465, 6 pages doi:1.54/11/5465 Research Article Rapidly-Converging Series Representations of a Mutual-Inforation Integral Don Torrieri 1 and Matthew Valenti 1 Coputational and Inforation Sciences Directorate, U.S. Ary Research Laboratory, Adelphi, MD , USA Lane Departent of Coputer Science and Electrical Engineering, West Virginia University, Morgantown, WV 656, USA Correspondence should be addressed to Matthew Valenti, atthew.valenti@ail.wvu.edu Received 3 Noveber 1; Accepted 13 Deceber 1 Acadeic Editors: C. Carbonelli and K. Teh Copyright 11 D. Torrieri and M. Valenti. This is an open access article distributed under the Creative Coons Attribution License, which perits unrestricted use, distribution, and reproduction in any ediu, provided the original work is properly cited. This paper evaluates a utual-inforation integral that appears in EXIT-chart analysis and the capacity equation for the binaryinput AWGN channel. Rapidly-converging series representations are derived and shown to be coputationally efficient. Over a wide range of channel signal-to-noise ratios, the series are ore accurate than the Gauss-Herite quadrature and coparable to Monte Carlo integration with a very large nuber of trials. 1. Introduction The utual-inforation between a binary-valued rando variable and a consistent, conditionally Gaussian rando variable with variance is J() = 1 exp [ ( 1/ )( y / ) ] log π (1e y )dy. (1) This function and its inverse play a central role in EXITchart analysis, which ay be used to design and predict the perforance of turbo codes [1], low-density parity-check codes [], and bit-interleaved coded odulation with iterative decoding (BICM-ID) [3]. Achangeofvariableyields exp ( y J() = 1 / ) ( log ) π 1e y / dy () which has the sae for as the equation for the capacity of a binary-input AWGN channel [4]. Despite its origins in the early years of inforation theory, the integral in () has not been expressed in closed for or represented as a rapidly-converging infinite series. Consequently, () has been evaluated by nuerical integration, Monte Carlo siulation, or approxiation []. As will be shown subsequently, a rapidly-converging series representation for (1)and()is J() = 1 1 { ( exp /8 ) ( ) ( ) ln π 1 Q (1) [ ( 1) exp ( )] (3) ( Q ) }, Q(x) = 1 ( ) ( ) x erfc 1 = exp y dy. (4) x π Nuerical evaluation of (3) requires truncation to a finite nuber of ters. Since an alternating series satisfying the Leibniz convergence criterion [5] appears in (3), siple upper and lower bounds are easily obtained by series truncation, and the axiu error is easily coputed fro the first oitted ter. Only six ters in the suation over in (3) are needed to copute J() with an error less than.1.

2 ISRN Counications and Networking Define J M () tobe(3) with the upper liit on the suation over replaced with M. LetE(M, ) bethe agnitude of the error when the series that is truncated after M ters, i.e. E(M, ) = J() J M (). SinceQ(x) exp(x /)/, x, the error agnitude E(M, )satisfies E(M, ) exp( /8 ) (5) (M 1)(M ) which indicates a rapid convergence of (3). Because of the dependence on in the bound given by (5), the nuber of ters required to achieve a very sall error ay becoe large for low. This otivates the use of an alternative series expression for (1)and()thatissuitable for sall. A rapidly-converging series representation for <.5 is J() = 1 { exp( b / ) ( ) ln π ln Q(b) (1) [ exp ( )] Q( b) 1 n n=1 n n (1) k n k= k [ ( exp k k )] } Q(k b), (6) b = ln 3. (7) Note that (6) isvalidfor = ifweuseq( ) = 1and Q( ) =. Both series entail the coputation of the Q function, which is itself an integral. However, the central iportance of the Q function has led to the developent of very efficient approxiations to copute it, for instance, by using the erfc function found in MATLAB or the approxiation proposed in [6]. In EXIT-chart analysis [1], theinverseofj() is required. The truncated series can be easily inverted by using nuerical algoriths such as the MATLAB function fzero, which uses a cobination of bisection and inverse quadratic interpolation. The reainder of this paper is organized as follows. The series are derived in Section. Coputational aspects of (3) and(6) are discussed in Section 3. Theseriesare copared against Gauss-Herite quadrature in Section 4, and the paper concludes in Section 5.. Derivation of Series To derive the desired representations, we first derive a faily of series representations of a ore general integral. Consider the following integral: I(a, ) = G(x) ln(1ae x )dx, a,, (8) ( ) G(x) = 1 exp x π. (9) Since I(, ) = andi(a,)= ln(1 a), a>and>are assued henceforth. For any β, define b = 1 ( ) 1β ln. (1) a Dividing the integral in (8) into two integrals, changing variables in the second one, and using the fact that G(x) isan even function, we obtain I(a, ) = I 1 (a, )I (a, ), I 1 (a, ) = G(x) ln(1ae x )dx, (11) b I (a, ) = G(x) ln(1ae x )dx. (1) b A uniforly convergent Taylor series expansion of the logarith over the interval [, 1 β]is ln ( 1y ) = ln ( 1β ) (1) 1 ( ) ( ) 1β y β, (13) y 1 β. The unifor convergence can be proved by application of the Leibniz criterion for alternating series [5]. Since a exp(x) 1β for x b, (13) indicates that ln(1 ae x )in(11) ay be expressed as a uniforly convergent series of continuous functions in the interval of integration. Since b G(x)dx is bounded, the infinite suation and the integration ay be interchanged and I 1 (a, ) = ln ( 1β ) Q(b) (1) 1 ( ) 1β G(x) ( ae x β ) dx, b (14) Q(x) = 1 ( ) x erfc = G ( y ) dy. (15) x Substituting the binoial expansion, interchanging the finite suation and integration, copleting the square of the arguent of the exponential in the integrand, changing variables, and then evaluating the integral, we obtain I 1 (a, ) = ln ( 1β ) 1 Q(b) ( 1β ) (1) k k= (16) β k a kk Q(k b). k

3 ISRN Counications and Networking 3 Since ln(1 ae x ) = ln[ae x (1 a 1 e x )] = x lna ln(1 a 1 e x ), a, (1) ay be expressed as I (a, ) = b G(x)(x lna)dx b G(x) ln ( 1a 1 e x) dx. (17) Since a 1 e x (1 β) 1 1β for x b, the substitution of (13) into(17), calculations siilar to previous ones, and the evaluation of the first integral yield I (a, ) = exp( b / ) (ln a)q(b) π (1) Q( b). a (18) The addition of (16) and(18) gives the general faily of series representations for I(a, ) whena >, >, b is defined by (1), and β : I(a, ) = exp[ b / ] (ln a)q(b) ln ( 1β ) Q(b) π 1 ( 1β ) (1) k β k a kk k Q(k b) k= (1) a Q( b). (19) If a = exp( /) and β = in(19), then, for, an algebraic siplification yields I ( e /, ) = exp( /8 ) ( ) ( ) π 1 Q (1) [ ( 1) exp ( )] () ( Q the validity of this equation when = can be separately verified. The substitution of (8), (), and ln x = log x/ ln into () proves the series representation of (3). Siilarly, if a = exp( /) and β = 1in(19), then, for, an algebraic siplification yields I ( e /, ) ), = exp( b / ) Q(b) ln()q(b) π (1) [ exp ( )] Q( b) 1 n n=1 n n (1) k n [ ( exp k k )] Q(k b), k= k (1) JM() M Value of truncated series Error bounds Figure 1: Truncated large- series J M ()for =.5 and a bound on the error. b = ln 3, () and we use Q( ) = 1andQ( ) = when =. The substitution of (8), (1), Q(b) = 1 Q(b), and ln x = log x/ ln into () proves the series representation of (6). 3. Evaluation of Series To nuerically evaluate (3) and(6), the suations ust be evaluated with a finite nuber of ters. Let the large series be J M (), that is, (3) with the upper liit on the suation over replaced with M. Siilarly, define J M,N () to be the sall- series given by (6) with the upper liits of the suations over and n replaced with M and N, respectively. The rapid convergence of the large- series is illustrated in Figure 1, which shows the value of the truncated series J M () as a function of M for =.5. The error bounds deterined by (5), which are also shown, are observed to be pessiistic. Figure shows the nuber of ters required for the large- series to converge to attain various error agnitudes as a function of. Figure shows that (3) isnotefficiently coputed for sall values of and error agnitude, which otivates the use of (6)forsall. Evaluation of (6) is coplicated by the presence of two infinite suations. For the range of of interest, the suation over n is the doinant of the two infinite suations. This behavior is illustrated in Figure 3, which shows the values of the suations over and n for =.5 as a function of the nuber of ters. When coputed to 1 ters, the value of the suation over n is , while the value of the suation over is only

4 4 ISRN Counications and Networking Nuber ters M Figure : Miniu nuber of ters required to attain various error agnitudes E(M, ). For lower values of, the agnitude of the suation over is even saller and becoes negligible as approaches zero. For this reason, we evaluate the suation over first and select the upper liit on the suation M such that M = in : { 1 exp [ ( )] } Q( b) <ζ, (3) ζ is a sall threshold value. If the nuerical accuracy requireents are odest, the suation over can be oitted. After coputing the suation over to M ters, (6) is evaluated with the nuber of ters N in the suation over n chosen to satisfy { N = in : n 1 J } M,n() J M,n1 () < ɛ. (4) After each ter is added to the suation over n in (6), the absolute value in (4) is evaluated, and the process halts once the threshold ɛ is reached. The nuber of ters M and N to achieve convergence criterion (3)withζ = 1 4 and (4)withɛ = 1 is shown in Figure 4 for.5.forhighervaluesof, evaluation of (6) becoes unstable because the large upper liit on the suation over n results in large binoial coefficients that cause nuerical overflow in typical ipleentations. Also shown in Figure 4 is the nuber of ters M required to copute the truncated large- series (3)withconvergence threshold ɛ = 1,M is related to ɛ by { M = in : 1 J } () J 1 () < ɛ. (5) Fro Figure 4, it ight appear that the sall- series (6) is less coplex to evaluate than the large- series (3) for all.5. However, due to the presence of the double Value of su over Value of su over n (a) M (b) Figure 3: Evaluation of the two suations in (6) for =.5. (a) Value of the suation over as a function of the nuber of ters M; (b) Value of the suation over n as a function of the nuber of ters N. Nuber of ters (M or N) M in (3) 1 5 N in (6) M in (6) Figure 4: The nuber of ters M and N required to evaluate the sall- series (6) withζ = 1 4 and ɛ = 1. Also shown is the nuber of ters M required to evaluate the large- series (3)with ɛ = 1. suation in (6), this is not necessarily true. To deterine a threshold below which the sall- series is preferable coputationally, one should consider the total nuber of ters containing an exponential and/or Q-function. To evaluate the truncated large- series (3), a total of M ters involving exp and/or Q ust be coputed. On the other hand, to evaluate the truncated sall- series (6), a total of MN(N 1)/ ters involving exp and/or Q ust be coputed. Figure 5 copares the total nuber of ters involving exp and/or Q that ust be coputed for each N

5 ISRN Counications and Networking 5 Nuber of ters 1 Tersin(3) Ters in (6) Figure 5: The total nuber of ters involving exp and/or Q in the sall- series (6)andthelarge- series (3). J() = = 1 = M J() Monte Carlo Series Figure 6: The truncated series representation (6) for.5 and (3) for>.5 and the integral evaluated using Monte Carlo integration. of the two series representations as a function of.frothis figure, it is seen that for.6, fewer ters are required for the sall- series. For all values of,thenuberofters is fewer than 8, and for ost it is significantly saller. Figure 6 copares the series representations against the value of (1) found using Monte Carlo integration with one illion trials per value of. The sall- series is used for.5, and the large- series is used for >.5. As before, ɛ = 1 for both series, and ζ = 1 4 for the sall- series. There is no discernible difference between the series representations and the Monte Carlo integration, and any sall differences can be attributed ainly to the finite nuber of Monte Carlo trials. Figure 7: The truncated large- series J () with =, = 1, and = M required to satisfy the convergence criterion with ɛ = 1 4. Given the rapid rate of convergence of (3), it is interesting to see the value of the large- series when only one ter is aintained in the suation or if the suation is dropped copletely. Figure 7 copares the value of the truncated large- series for M ={, 1} and the M required to satisfy the convergence criterion with ɛ = 1 4.UsingM = provides an upper bound that is tight only for large values of,suchas>, as using M = 1providesatightlower bound even for relatively sall values of.usingm = 4gives two decial places of accuracy for all Coparison with Gauss-Herite Quadrature The integral given by (1) and() ay also be solved using a for of nuerical integration known as Gauss-Herite quadrature [7]. After a change of variables (z = y/ ), the integral ay be written as J() = 1 1 e z f (z)dz, (6) π { f (z) = log (1exp z }). (7) With the Gauss-Herite quadrature, the integral in (6) is evaluated using n e z f (z)dz w i f (z i ), (8) i=1

6 6 ISRN Counications and Networking Magnitudeoferror Gauss-Herite large- series Sall- series Monte Carlo Figure 8: Error (relative to infinite series) of truncated series with 6 ters and Gauss-Herite quadrature with 6 ters. Monte Carlo with 1 illion trials shown for coparison purposes. f (z)isgivenby(7), z i are the roots of the nth-degree Herite polynoial H n (z) = (1) n e z dz n, (9) ez and w i are the associated weights w i = n1 n! π n [H n1 (z i )]. (3) The roots {z 1,..., z n } of H n (z)aybefoundusingnewton s ethod [7]. We copare five realizations of the J() functionas follows: (1) the infinite large- series representation,coputed with very large M (i.e., M = 75); () the truncated large- series representation (3), truncated to M ters; (3) the truncated sall- series representation (6), with the first suation truncated to M = 1 ters and the second (double) suation truncated to an upper liit on the outer suation of N; (4) the Gauss-Herite quadrature with n = M ters, that is, the sae nuber of ters as the truncated series representation; (5) Monte Carlo integration with 1 illion trials. For each realization, we deterine the error to be the agnitude of the difference between the calculated value and the value coputed by the infinite series. Figure 8 shows the errors of the two truncated series and the Gauss-Herite quadrature. The Gauss-Herite quadrature is evaluated with M = 6ters,andthelarge- series dn is truncated to M = 6 ters. In order to have a coparable nuber of ters in the suations in (6), M = 1andN = are used for the sall- series. The error of the Monte Carlo approach is also shown (dotted line). When >1.6, the truncated large- series usually provides saller error than Gauss-Herite quadrature (except at =.5and = 3.9, the Gauss-Herite quadrature briefly has a saller error). Since the aount of coputation required per ter of the series is roughly the sae, the large series is preferable to the Gauss-Herite quadrature. For <1.6, the error of the Gauss-Herite quadrature is saller than that of the large- series. In this region, the sall- series could be used, since it provides a saller error than the large- series below = Conclusions Series representations have been derived for a utualinforation function that is used in EXIT-chart analysis and the evaluation of the capacity of a binary-input AWGN channel. Truncated versions of the series are coputationally copetitive with the Gauss-Herite quadrature and do not require finding roots of Herite polynoials. The series are useful for coputation and to provide siple lower and upper bounds. References [1] S. ten Brink, Convergence behavior of iteratively decoded parallel concatenated codes, IEEE Transactions on Counications, vol. 49, no. 1, pp , 1. [] S. ten Brink, G. Kraer, and A. Ashikhin, Design of lowdensity parity-check codes for odulation and detection, IEEE Transactions on Counications, vol. 5, no. 4, pp , 4. [3] S. ten Brink, Convergence of iterative decoding, Electronics Letters, vol. 35, no. 1, pp , [4] J. G. Proakis and M. Salehi, Digital Counications, McGraw- Hill, New York, NY, USA, 5th edition, 8. [5] E. Kreyszig, Advanced Engineering Matheatics, Wiley,New York, NY, USA, 9th edition, 6. [6] G. K. Karagiannidis and A. S. Lioupas, An iproved approxiation for the Gaussian Q-function, IEEE Counications Letters, vol. 11, no. 8, pp , 7. [7] P. Moin, Fundaentals of Engineering Nuerical Analysis, Cabridge University Press, Cabridge, UK, 1.