Extremal Problems of Information Combining

Transcription

1 Extreml Problems of Informtion Combining Yibo Jing, Alexei Ashikhmin, Rlf Koetter, nd Andrew C Singer Coordinted Science Lbortory University of Illinois t Urbn-Chmpign, Urbn, IL 680, USA Emil: {yjing, koetter, csinger}@uiucedu Bell Lbortories, Lucent Technologies 600 Mountin Avenue, Murry Hill, NJ 07974, USA Emil: e@reserchbell-lbscom Abstrct In this pper we study moments of soft-bits of binry-input symmetric-output chnnels nd solve some extreml problems of the moments We use these results to solve the extreml informtion combining problem Further, we extend the informtion combining problem by dding constrint on the second moment of soft-bits, nd find the extreme distributions for this new problem I INTRODUCTION The extrinsic informtion trnsfer (EXIT) chrt method [] hs proven useful in prctice, for exmple, in the design of low-density prity-check (LDPC) codes [2], nd simple since it trcks only one prmeter, nmely mutul informtion It is usully ssumed tht priori informtion is conditionlly Gussin distributed In the belief propgtion decoding of LDPC codes [3], however, the distributions of messges re often non-gussin Therefore, the EXIT chrt method only provides n pproximtion to the rel decoding trjectory Thus it is worthwhile to conduct more rigorous nlysis Bounds on the extrinsic mutul informtion t either vrible node decoder (VND) or check node decoder (CND) will be useful nd led to upper nd lower bounds on the decoding trjectory of mutul informtion This motivtes the study of informtion combining problems The notion of informtion combining ws introduced by Huettinger et l in [4], [5] for the study of conctented coding systems In prticulr, in the belief propgtion decoding of LDPC codes, the processings t vrible nd check nodes cn be interpreted s opertions of informtion combining, ie combining the mutul informtion contined messges incoming to node Let (X,,X d ) be codeword of either [d, ] repetition code or [d, d ] single prity-check code, which models degree d vrible node or degree d check node respectively Assume d 3 to void trivil cses Assume X i is binry phse shift keying (BPSK) modulted under the mpping 0 + nd, nd trnsmitted through binry-input symmetric-output chnnel with output Y i, i d, ie p(y i = y X i =)=p(y i = y X i = ) These d chnnels re ssumed to be independent The nottion X i Y i is used to indicte the i-th chnnel The soft-bit T i for the i-th chnnel is defined s T i = p(x i = Y i ) p(x i = Y i ), nd is sufficient sttistic Although the trnsition probbility distribution of ech chnnel is unknown, the mutul informtion of ech chnnel is ssumed to be known Without loss of generlity, one cn focus on X d nd sk the following question: cn we find tight lower nd upper bounds on the combined informtion I(X d ; Y,Y 2,,Y d ), ie the extrinsic mutul informtion? Such n extreml problem ws first considered in [6] for [3, ] repetition code It ws shown tht I(X 3 ; Y,Y 2 ) is mximized (or minimized) when both X Y nd X 2 Y 2 re binry ersure chnnels (BECs) (or binry symmetric chnnels (BSCs)) with prescribed mutul informtion vlues In [7], the cse of [3, 2] single pritycheck codes ws studied, nd it ws shown tht BSCs chieve the upper bound, nd BECs chieve the lower bound In [8], [9], [0], [] the bove results were extended to rbitrry codeword length d for both repetition codes nd single pritycheck codes In [6], [7], [8], [9], the extrinsic mutul informtion I(X d ; Y,Y 2,,Y d ) is optimized over ll chnnels subject to their individul mutul informtion constrints In [0], [], the problems were generlized by optimizing with respect to single chnnel Fix the trnsition probbility distributions of X i Y i, i d 2 nd fix the mutul informtion of X d Y d It ws shown in [] tht for repetition code, when X d Y d is BEC (BSC), the extrinsic mutul informtion I(X d ; Y,Y 2,,Y d ) is mximized (minimized) For single prity-check code, the roles of BEC nd BSC re reversed All the results bove were obtined by proving new mutul informtion inequlities nd using vrious existing inequlities nd identities of mutul informtion The work by Shron et l, [2], [3], provides frmework of the moments of soft-bits nd expresses both the mutul informtion of binry-input symmetric-output chnnel nd the extrinsic mutul informtion of [d, d ] single prity-check code s serieses of moments of chnnel soft-bits In this pper, we study vrious properties of the moments nd moment sequences We prove tht mong ll binry-input symmetric-output chnnels with fixed mutul informtion, the BSC (BEC) mximizes (minimizes) the second moment of chnnel soft-bits We lso determine the ordering between the moment sequences of BSC (or BEC) nd ny other chnnel By using the properties of the moment sequences, we solve

2 the extreml informtion combining problem t the check nodes proposed in [] Lter, we extend the problem by dding constrint on the second moment of chnnel softbits It is lso solved by moments pproch, nd the best nd worst chnnel distributions in the sense of mximizing nd minimizing the extrinsic mutul informtion re determined The following nottions re used in this pper Underscores denote vectors The symbol y [j] is used to denote the vector obtined by deleting the j-th element of y Subscript b nd w stnd for best nd worst, respectively All proofs re omitted due to the limittion of spce II T-CONSISTENCY AND MUTUAL INFORMATION In [2], [3], the concept of T-consistency nd some relted mutul informtion results were proposed We give short summry for completeness Let X nd Y be rndom vribles t the input nd output of binry-input symmetric-output chnnel respectively, with trnsition probbility density function f stisfying f Y X (Y X =)=f Y X ( Y X = ) Define the chnnel soft-bit T s T = Pr(X = Y ) Pr(X = Y ) It is esy to see tht T =(e L )/(e L +)=tnh(l/2) where L is the log-likelihood-rtio (LLR) It ws shown in [3] tht the conditionl probbility density p(t X) stisfies p(t = t X =)=p(t = t X =) +t t, () nd p(t = t X =)=p(t = t X = ) Such rndom vrible is clled T-consistent For BSC chnnel with cpcity I, its T vrible tkes two vlues { 2h [ I], 2h [ I] }, where h[x] = x log 2 x ( x)log 2 ( x) is the binry entropy function nd h [x] [0, 05] For BEC chnnel, its T vrible lwys tkes three vlues {, 0, } Assume X is n equiprobble binry rndom vrible The mutul informtion between X nd T is [2], [3] + I(X; T )= log 2 ( + t)p(t = t X =)dt = (2) ln 2 2i(2i ) m 2i, where m 2i = + Note tht 0 m 2i nd ln 2 t 2i p(t = t X =)dt (3) 2i(2i ) =log 2( + t) t= =, therefore the right-hnd-side of Eq (2) is convergent We point out tht the singulrity of log 2 ( + t) t t = does not ffect the integrl (2) since one knows p(t = X =)=0from () Consider n [n, k] binry liner code Its codeword is BPSK modulted nd trnsmitted through n binry-input symmetricoutput independent chnnels Let x nd y indicte the input nd output vector The extrinsic soft-bit for X j posteriori probbility (APP) decoder is defined s of n T E,j = Pr(X j = Y [j] ) Pr(X j = Y [j] ) (4) Let c 0 be the ll-zero codeword It is shown [3] tht T E,j is T-consistent nd p(t E,j = t X j =)=p(t E,j = t c 0 trnsmitted) (5) Furthermore, for n [n, n ] single prity-check code, T E,j = n k=,k j T k where T k = Pr(X k = Y k ) Pr(X k = Y k ) Thus the extrinsic mutul informtion is I E,j = I(X j ; T E,j ) = ln 2 2i(2i ) n k=,k j E[T 2i k X k =] III TCHEBYCHEFF SYSTEM Tchebycheff system (T-system) theory is used to solve extreml problems of moments For completeness, we give n introduction to some of the key concepts nd results of T-system theory For <bfinite nd rel, set of rel continuous functions {u i (t)} n i=0 defined on [, b] is clled T-system [4], [5] if every nontrivil rel liner combintion n i=0 iu i (t) hs t most n distinct zeros in [, b] It is esy to show tht {u i (t)} n i=0 is T-system if nd only if the determinnt u 0 (t 0 ) u 0 (t ) u 0 (t n ) u (t 0 ) u (t ) u (t n ) det (7) u n (t 0 ) u n (t ) u n (t n ) does not vnish whenever t 0 <t < <t n b Since the determinnt (7) is continuous function of t i,itis equivlent to require tht the determinnt (7) mintins fixed strict sign Without loss of generlity, in [5], [4], {u i (t)} n i=0 is clled T-system if the determinnt (7) is strictly positive whenever t 0 <t < <t n b Let us introduce the concept of distribution [4, p5] on [, b] A distribution is nondecresing, right-continuous (except t the left endpoint) function For distribution σ(t), the mss t point ξ (, b] is σ(ξ) σ(ξ 0), nd the mss t the left endpoint is σ( +0) σ(), where σ(ξ 0) nd σ( +0) indicte the left nd right limits respectively Note tht distribution is not necessrily probbility distribution (totl mss on [, b] is, ie dσ =) The moment spce M n+ induced by the T-system {u i (t)} n i=0 is { } M n+ = (c 0,,c n ) c i = u i (t)dσ(t), 0 i n (8) where σ goes through the set of ll vlid distributions Geometriclly, M n+ is closed convex cone Assume c = {c k } n 0 M n+ Define set V (c) to be V (c) ={σ u i(t)dσ(t) =c i, 0 i n} In generl, V (c) contins either one distribution (if nd only if c is boundry point of M n+ ) or infinitely mny distributions (if nd only if c is n interior point of M n+ (denoted by c IntM n+ )) Assume {u i (t)} n i=0 is T-system Let σ be distribution in V (c) Ifσ(t) hs finitely mny points of increse t < (6)

3 t 2 < <t m b, the representtion for c becomes m c i = u i (t)dσ(t) = ρ k u i (t k ), 0 i n, (9) k= where ρ k is the mss t the point t k The points {t k } m re clled roots of the representtion (9), nd σ(t) is clled the distribution ssocited with the representtion (9) As in [4], n index function ɛ(t) is defined s ɛ(t) =2for <t<b, nd ɛ() = ɛ(b) = The index of the representtion (9) is defined s the sum m k= ɛ(t k) Now we introduce two importnt concepts, cnonicl representtion nd principle representtion If the index of representtion is n +2,the representtion is sid to be cnonicl If the index is n +, the representtion is sid to be principle Furthermore, if cnonicl (principle) representtion hs root t b, it is further clled n upper cnonicl (principle) representtion On the other hnd, if b is not root, it is clled lower cnonicl (principle) representtion From [5, Coro 3, Sec II3] or [4, Theorem 5, Sec III5], it is true tht Theorem : For ech c IntM n+, there exists exctly one lower principle representtion nd exctly one upper principle representtion The roots of these two representtions strictly interlce Theorem 2: (p77 of [4], p45 of [5]): The roots of lower nd upper principle representtions hve the following properties () for n odd (n =2q ): lower principle representtion: ll mss is concentrted t q interior points of [, b], ie <t <t 2 < <t q < b; upper principle representtion: ll mss is concentrted t q interior points of [, b], nd t both endpoints, b, ie = s <s 2 < <s q <s q+ = b (2) for n even (n =2q): lower principle representtion: ll mss is concentrted t q interior points of [, b], nd t the endpoint, ie = t <t 2 < <t q+ <b; upper principle representtion: ll mss is concentrted t q interior points of [, b], nd t the endpoint b, ie <s <s 2 < <s q <s q+ = b Let Ω(t) be continuous function Define u n+ (t) =Ω(t) Theorem 3: ([4, Theorem, Sec IV], [5, Theorem, Sec III]): Let c IntM n+ If both {u i (t)} n i=0 nd the ugmented system {u i(t)} n+ i=0 re T- systems, mx Ω(t)dσ(t) is ttined uniquely for the distribution σ ssocited with the upper principle σ V (c) representtion of c, nd min σ V (c) Ω(t)dσ(t) is ttined uniquely for the distribution σ ssocited with the lower principle representtion of c It is remrkble tht s long s the ugmented system {u i (t)} n+ i=0 is T-system, the mximizing nd the minimizing distributions (σ nd σ ) re independent of the function Ω(t) IV THE EXTREMAL INFORMATION COMBINING PROBLEM In this section, we re focusing on the type of extreml problems of informtion combining proposed in [] Let us first consider check node with degree d One cn tret such node s [d, d ] single prity-check code Its codeword is BPSK modulted Let x nd y stnd for BPSK-modulted codeword nd chnnel output vector respectively All chnnels X i Y i re binry-input symmetric-output, ie T-consistent, but not necessrily hve the sme distribution The chnnels re independent, ie p(y x) = d p(y i x i ) Without loss of generlity, the extreml problem of informtion combining t check node cn be stted s follows Problem : Fix the chnnels X i Y i, i d 2 Find T-consistent probbility densities P b nd P w for X d Y d tht mximize nd minimize the extrinsic mutul informtion I(X d ; T E,d ) respectively, subject to I(X d ; T d )= I d Let p(t) p(t d = t X d =)nd p(t) should be T- consistent, ie p(t) = p( t)( + t)/( t) In the constrint I d = I(X d ; T d ), where the right-hnd-side is relted to p(t) by (2), one cn see the integrnd log( + t) is not continuous on [, ] One knows T-system theory requires ll the integrnd functions to be continuous on closed intervl Thus we use the T-consistency of p(t) to trnsform the integrnd into continuous function First, p(t) is mpped into new density ˆp(t) on [0, ] ˆp(t) = Then one obtins { p(t)+p( t) =p(t)2/( + t), t (0, ] p(0), t =0 + (0) I d = I(X d ; T d )= ( h[( t)/2])ˆp(t)dt 0 = ln 2 2i(2i ) m 2i () where m 2i = + 0 t 2i ˆp(t)dt Note tht h[( t)/2] is continuous function on [0, ] Another merit of (0) is tht the T-consistency requirement on p(t) is utomticlly tken into ccount in () Define β 2i = d 2 2i k= E[Tk X k =], from (6), one obtins I E,d = I(X d ; T E,d )= ln 2 2i(2i ) β 2im 2i (2) Fix β 2i for i The extreml informtion combining problem t the check node cn be reformulted s: subject to (), determine the best or worst densities of ˆp such tht I E,d (2) is mximized or minimized respectively In wht follows we will show tht the extreme densities for this optimiztion problem re exctly the densities corresponding to the extreme distributions of the following one Problem 2: Among ll probbility distributions on [0, ] which stisfy the constrint (), determine the probbility distribution σ b (σ w ) which mximizes (minimizes) the 2nd moment m 2

4 T-system theory is perfect tool to solve it Let u 0 (t), u (t) h[( t)/2], Ω(t) t 2 nd c (,I d ) We prove tht Lemm : Both {u 0,u } nd the ugmented system {u 0,u, Ω} re T-systems on [0, ] V (c) ={σ 0 u i(t)dσ(t) =c i,i=0, } is the set of distributions which re probbility distributions nd stisfy () By Theorem 3 one concludes tht the distribution σ ssocited with the upper principle representtion of c mximizes m 2, thus σ w = σ The distribution σ ssocited with the lower principle representtion of c minimizes m 2, thus σ b = σ According to Theorem 2, since n =is odd, q =,ll probbility mss of σ b concentrtes t n interior point t, nd ll probbility mss of σ w concentrtes t two endpoints {0, } By solving () for σ b, ie u (t )=I d, one cn obtin t = 2h [ I d ] According to (0), the T-consistent distribution corresponding to σ b hs probbility mss of ( + t )/2 t t nd probbility mss of ( t )/2 t t This exctly corresponds to BSC chnnel with cpcity I d Similrly, for σ w, one cn esily obtin tht the probbility mss t is I d The T-consistent distribution corresponding to σ w hs probbility mss of I d t, nd probbility mss of I d t 0 This exctly corresponds to BEC chnnel with cpcity I d Thus we conclude tht Theorem 4: Among binry-input symmetric-output chnnels with fixed mutul informtion, BSC (BEC) mximizes (minimizes) m 2 Lemm 2: A moment sequence {m 2i } hs the following properties: ) {m 2i } is nonincresing; 2) The rtio sequence {r 2i = m2i m 2i+2 } is nonincresing For BSC, its moment sequence {m b,2i = t 2i } is geometric sequence For BEC, its moment sequence {m w,2i = I d } is constnt sequence Using these fcts, Theorem 4, Lemm 2 nd the constrint (), we show tht Lemm 3: There re two possible types of ordering for the moment sequence {m b,2i } of BSC nd the moment sequence {m σ,2i } for ny non-bsc σ V (c) ) m b,2i >m σ,2i for i<i 0, nd m b,2i <m σ,2i for i i 0 ; 2) m b,2i >m σ,2i for i<i 0, m b,2i0 = m σ,2i0 nd m b,2i <m σ,2i for i>i 0 ; where i 0 is n integer depending on σ Similrly, we cn show Lemm 4: There re two possible types of ordering for the moment sequence {m w,2i } of BEC nd the moment sequence {m σ,2i } for ny non-bec σ V (c) ) m σ,2i >m w,2i for i<i, nd m σ,2i <m w,2i for i i ; 2) m σ,2i >m w,2i for i<i, m σ,2i = m w,2i nd m σ,2i <m w,2i for i>i, where i is n integer depending on σ The following lemm is useful We prove tht Lemm 5: Assume i > 0 for i, b b 2 b i 0 For sequences {x i }, {y i} which stisfy ) x i 0, y i 0, fori ; 2) ix i = iy i ; 3) x i >y i for i k, nd x i <y i for i>k, it is true tht ib i x i ib i y i Using Lemm 3, Lemm 4, Lemm 5, () nd (2), we conclude tht subject to the mutul informtion constrint (), BSC mximizes nd BEC minimizes I E,d (2), ie P b =BSC nd P w =BEC As pointed out before, they re lso the extreme distributions with respect to optimizing m 2 For the cse [7] where the distribution of ech chnnel is llowed to vry subject to mutul informtion constrint, one cn esily conclude tht when ll of them re BSCs (BECs), the extrinsic mutul informtion is mximized (minimized) For the extreml informtion combining problem t the vrible nodes, by Lemm 3 [, SecIV], one cn conclude tht BEC nd BSC re the mximizer nd minimizer respectively V EXTENSION OF THE ORIGINAL EXTREMAL INFORMATION COMBINING PROBLEM We consider n extension of the originl extreml informtion combining problem by dding constrint on the 2nd moment m 2 Problem 3: Fix the chnnels X i Y i, i d 2 Find T-consistent probbility densities P b nd P w for X d Y d tht mximize nd minimize the extrinsic mutul informtion I(X d ; T E,d ) respectively, subject to constrints: I(X d ; T d )=I d ; m 2 = θ The motivtion for considering this problem is to obtin smller gp between mximized nd minimized I E,d compred to Problem First we gin consider different optimiztion problem Lter we will show tht the solution to the following problem leds to the solution of Problem 3 by tking n inverse mpping of (0) Problem 4: Among ll probbility distributions on [0, ] which stisfy the constrint () nd m 2 = θ, determine the probbility distribution σ b ( σ w ) which mximizes (minimizes) the 4th moment m 4 Agin, we use T-system theory Define u 0 (t), u (t) t 2, u 2 (t) h[( t)/2], Ω(t) t 4 nd c (,θ,i d ) We prove tht Lemm 6: Both {u 0,u,u 2 } nd the ugmented system {u 0,u,u 2, Ω} re T-systems on [0, ] V (c) = {σ 0 u i(t)dσ(t) = c i,i= 0,, 2} is set of distributions which re probbility distributions nd stisfy both () nd m 2 = θ By Theorem 3 one concludes tht the distribution σ ssocited with the upper principle representtion of c mximizes m 4, thus σ w = σ The distribution σ ssocited with the lower principle representtion of c minimizes m 4, thus σ b = σ

5 According to Theorem 2, since n = 2 is even, q =, ll probbility mss of σ b concentrtes t two points {0,t } where t is n interior point, nd ll probbility mss of σ w concentrtes t two points {s, } where 0 s t The T-consistent density P 4,b corresponding to σ b hs probbility mss of p 0 t 0, probbility mss of p t t nd probbility mss of p 0 p t t From the constrints nd the distribution structure, we cn determine tht t = f ( Id θ ) h[( x)/2], where f(x) = x 2, p = θ ( + t ) 2t 2, p 0 = 2p +t Similrly, the T-consistent density P 4,w corresponding to σ w s hs probbility mss of p t s, probbility mss of p +s t s nd probbility mss of p 2 t From the constrints nd the distribution structure, we cn determine tht ( ) s = g Id h[( x)/2], where g(x) = θ x 2, θ p = 2( s ), p 2 = θ s2 s 2 In summry, we hve Lemm 7: Among binry-input symmetric-output chnnels with fixed mutul informtion nd fixed m 2, the chnnel corresponding to P 4,b (P 4,w ) mximizes (minimizes) m 4 Observe tht the moment sequence { m b,2i of the best density P 4,b is geometric sequence Along with Lemm 2, Lemm 7 nd (), we show tht Lemm 8: There re two possible types of ordering for the moment sequence { m b,2i } of P 4,b nd the moment sequence {m σ,2i } of ny non-p 4,b σ V (c) ) m b,2i >m σ,2i for 2 i i 0, nd m b,2i <m σ,2i for i>i 0 ; 2) m b,2i >m σ,2i for 2 i<i 0, m b,2i0 = m σ,2i0 nd m b,2i <m σ,2i for i>i 0 ; where i 0 is n integer depending on σ Obviously m b,2 = m σ,2 = θ Although its proof is quite complex, we still cn show Lemm 9: There re two possible types of ordering for the moment sequence { m w,2i } of P 4,w nd the moment sequence {m σ,2i } of ny non-p 4,w σ V (c) ) m σ,2i > m w,2i for 2 i i, nd m σ,2i < m w,2i for i>i ; 2) m σ,2i > m w,2i for 2 i<i, m σ,2i = m w,2i nd m σ,2i < m w,2i for i>i ; where i is n integer depending on σ Obviously m w,2 = m σ,2 = θ By Lemm 8, Lemm 9, Lemm 5, we cn conclude tht subject to the mutul informtion constrint () nd m 2 = θ, the distributions P 4,b nd P 4,w mximize nd minimize the extrinsic mutul informtion I E,d respectively Thus s pointed out before, Pb = P 4,b nd P w = P 4,w Extended to the cse where the distribution of ech chnnel is llowed to vry subject to () nd m 2 = θ constrints, = θt 2i 2 } it is not difficult to see tht when ll of the chnnels re of the type of P b (with pproprite prmeters), the extrinsic mutul informtion is mximized Similrly, when ll of the chnnels re of the type of P w (with pproprite prmeters), the extrinsic mutul informtion is minimized In [6], we use the results of this section to bound the CND EXIT functions with Gussin priors, nd s prt of procedure which computes (potentilly) improved overll best nd worst performnce bounds on the mutul informtion trjectory of the belief propgtion decoding of LDPC codes VI CONCLUSIONS In this pper, we showed tht BSC (BEC) mximizes (minimizes) m 2 mong ll binry-input symmetric-output chnnels with fixed mutul informtion We lso proved some ordering properties of moment sequences We solved both the originl nd extended extreml problems of informtion combining Ongoing nd future work includes finding more pplictions for the extension problem, considering more complicted codes nd chnnels REFERENCES [] S ten Brink, Convergence behvior of itertively decoded prllel conctented codes, IEEE Trns Commun, vol 49, no 0, pp , Oct 200 [2] S ten Brink, G Krmer, nd A Ashikhmin, Design of low-density prity-check codes for modultion nd detection, IEEE Trns Commun, vol 52, no 4, pp , Apr 2004 [3] T Richrdson nd R Urbnke, The cpcity of low-density pritycheck codes under messge-pssing decoding, IEEE Trns Inform Theory, vol 47, no 2, pp , Feb 200 [4] S Huettinger, J Huber, R Johnnesson, nd R Fischer, Informtion processing in soft-output decoding, in Proc Allerton Conf on Communictions, Control, nd Computing, Monticello, IL, Oct 200 [5] S Huettinger, J Huber, R Fischer, nd R Johnnesson, Soft-outputdecoding: Some spects from informtion theory, in Proc Int ITG Conf on Source nd Chnnel Coding, Berlin, Germny, Jn 2002, pp 8 90 [6] I Lnd, S Huettinger, P A Hoeher, nd J Huber, Bounds on informtion combining, in Proc Int Symp on Turbo Codes & Rel Topics, Brest, Frnce, Sept 2003, pp [7], Bounds on informtion combining, IEEE Trns Inform Theory, vol 5, no 2, pp 62 69, Feb 2005 [8] I Lnd, P A Hoeher, nd J Huber, Bounds on informtion combining for prity-check equtions, in Proc Int Zurich Seminr on Communictions (IZS), Zurich, Switzerlnd, Feb 2004, pp 68 7 [9] I Lnd, S Huettinger, P A Hoeher, nd J Huber, Bounds on mutul informtion for simple codes using informtion combining, Annls of Telecomm, Jn 2005, ccepted for publiction [0] I Sutskover, S Shmi, nd J Ziv, Extremes of informtion combining, in Proceedings of the 4st Annul Allerton Conference on Communictions, Conrol nd Computing, Monticello, IL, Oct 2003, pp [], Extremes of informtion combining, IEEE Trns Inform Theory, vol 5, no 4, pp , Apr 2005 [2] E Shron, A Ashikhmin, nd S Litsyn, EXIT functions for the Gussin chnnel, in Proceedings of 4st Annul Allerton Conference on Communiction, Control nd Computing, Monticello, Illinois, Oct 2003, pp [3], EXIT functions for continuous chnnels: Prt I Constituent codes, IEEE Trns Commun, submitted for publiction, 2004 [4] M G Krein nd A A Nudel mn, The Mrkov Moment Problem nd Extreml Problems Providence, Rhode Islnd: Americn Mthemticl Society, 977 [5] S Krlin nd W J Studden, Tchebycheff Systems: with Applictions in Anlysis nd Sttistics New York: Interscience Publishers, 966 [6] Y Jing, A Ashikhmin, R Koetter, nd A C Singer, Extreml problems of informtion combining, 2005, in preprtion