Ensemble-Tight o Exponents fo Mismtched Decodes Jonthn Sclett Univesity of Cmbidge jms265@cm.c.uk Alfonso Mtinez Univesitt Pompeu Fb lfonso.mtinez@ieee.og Albet Guillén i Fàbegs ICREA & Univesitt Pompeu Fb Univesity of Cmbidge guillen@ieee.og Abstct This ppe studies chnnel coding fo discete memoyless chnnels with given (possibly suboptiml) decoding ule. Using uppe nd lowe bounds on the ndom-coding eo pobbility, the exponentil behvio of thee ndomcoding ensembles is chcteized. The ensemble tightness of existing chievble eo exponents is poven fo the i.i.d. nd constnt-composition ensembles, nd new ensemble-tight eo exponent is given fo the cost-constined i.i.d. ensemble. Connections e dwn between the ensembles unde both mismtched decoding nd mximum-likelihood decoding. I. INTRODUCTION It is well known tht ndom coding techniques cn be used to pove the chievbility pt of Shnnon s chnnel coding theoem, s well s chcteizing the exponentil behviou of the best code fo nge of tes unde mximumlikelihood (ML) decoding []. In pctice, howeve, ML decoding is often uled out due to chnnel uncetinty nd implementtion constints. In this ppe, we conside the poblem of mismtched decoding [2] [8], in which the decoding ule is fixed nd not necessily optiml. In this setting, the following ndom-coding ensembles hve been consideed (see Section III fo detils): ) the i.i.d. ensemble, in which ech symbol of ech codewod is geneted independently; 2) the constnt-composition ensemble, in which ech codewod hs the sme empiicl distibution; 3) the cost-constined i.i.d. ensemble, in which ech codewod stisfies given cost constint. The i.i.d. ensemble cn be used to pove the chievbility of Genelized Mutul Infomtion (GMI) [2], while the ltte two ensembles cn be used to pove the chievbility of the highe LM te [3], []. It is known tht the LM te is equl to the mismtched cpcity in the cse tht the input lphbet is biny [5], but this is not tue in genel when the input is non-biny. It is theefoe of inteest to detee whethe the wekness is due to the ndom-coding ensemble itself, o the bounding techniques used in the nlysis. This question ws ddessed in [6] [8], whee it ws shown tht the GMI nd LM te e tight with espect to This wok hs been poted in pt by the Euopen Resech Council unde ERC gnt geement 259663. A. Mtinez eceived funding fom the Ministy of Science nd Innovtion (Spin) unde gnt RYC-20-0850 nd fom the Euopen Union s 7th Fmewok Pogmme (PEOPLE-20- CIG) unde gnt geement 303633. the ensemble vege fo the i.i.d. nd constnt-composition ensembles espectively; see Section I-B fo detils. In this ppe, we stengthen these esults by obtining ensembletight eo exponents fo ech of the bove ensembles. The nlysis is pefomed in unified fshion, nd connections e dwn between the thee ensembles. A. System Setup The input nd output lphbets e denoted by nd Y espectively, nd the chnnel W (y x) is ssumed to be discete memoyless chnnel (DMC). We conside block coding, in which the codebook C = {x (),...,x (M) } is known t both the encode nd decode. The encode chooses messge m equipobbly fom the set {,...,M} nd tnsmits the coesponding codewod x (m). The decode eceives the vecto y t the output of the chnnel, nd foms the estimte ˆm = g mx ny j2{,...,m} q(x (j) i,y i ) () whee n is the length of ech codewod nd x (j) i is the i-th enty of x (j) (similly fo y i ). The function q(x, y) is clled the decoding metic, nd is ssumed to be non-negtive. In the cse of tie, ndom codewod chieving the mximum in () is selected. Thoughout the ppe, we wite q(x, y) s shothnd fo Q n q(x i,y i ), nd similly fo W (y x). The mismtched cpcity is defined to be the emum of ll tes R = n log M such tht the eo pobbility (C) cn be mde bitily smll fo sufficiently lge n. An eo exponent E(R) is sid to be chievble if thee exists sequence of codebooks of length n nd te R such tht lim n! n log (C) E(R). (2) We let denote the vege eo pobbility with espect to given ndom-coding ensemble. The ndom-coding eo exponent E (R) is sid to exhibit ensemble tightness if lim n! B. Contibutions nd Pevious Wok I GMI (Q) = E s 0 n log = E (R). (3) The GMI nd LM te e espectively defined s pple log q(, Y ) s E[q(,Y ) s Y ] ()
pple I LM (Q) = E log s 0, q(, Y ) s e () E[q(,Y ) s e () Y ] whee (, Y, ) Q(x)W (y x)q(x), nd Q(x) is n bity input distibution. Fo the i.i.d. ensemble with input distibution Q, it is known tht! 0 s n! when R < I GMI (Q), whees! s n! when R > I GMI (Q) [2], [7]. Similly, fo the constntcomposition ensemble with input distibution Q, it hs been shown tht! 0 s n!when R<I LM (Q), whees! s n!when R>I LM (Q) [6], [8]. While chievble eo exponents exist in the litetue fo ech of the foementioned ensembles [2], [9], [0], we e not we of ny complete esults on ensemble tightness. The ensemble tightness of eo exponents in the mtched egime ws ddessed in [], but n extension of these techniques to the mismtched setting only poves ensemble tightness t low tes. In this ppe, we give uppe nd lowe bounds to the ndom-coding eo pobbility, nd deive ensemble-tight eo exponents fo ech ensemble. Fo the i.i.d. ensemble nd constnt-composition ensemble, ou esults pove the ensemble tightness of the chievble exponents pesented in [2] nd [9] espectively. The exponent fo the costconstined i.i.d. ensemble ppes to be new, nd cn be wekened to tht of [0]. We dw connections between the thee ensembles unde both mismtched decoding nd ML decoding. C. Nottion The set of ll pobbility distibutions on n lphbet A is denoted by P(A), nd the set of ll empiicl distibutions on vecto in A n (i.e. types) is denoted by P n (A). The type of vecto x is denoted by p x ( ). Fo given Q 2P n (A), the type clss T (Q) is defined to be the set of ll sequences in A n with type Q. We efe the ede to [2], [3] fo n intoduction to the method of types. The pobbility of n event is denoted by P[ ], nd the symbol mens distibuted s. The mginls of joint distibution P Y (x, y) e denoted by P (x) nd P Y (y).we wite P = P e to denote element-wise equlity between two pobbility distibutions on the sme lphbet. Expecttion with espect to joint distibution P Y (x, y) is denoted by E P [ ]. When the ssocited pobbility distibution is undestood fom the context, the expecttion is witten s E[ ]. Similly, mutul infomtion with espect to P Y is witten s I P (; Y ), o simply I(; Y ) when the distibution is undestood fom the context. Given distibution Q(x) nd conditionl distibution W (y x), we wite Q W to denote the joint distibution defined by Q(x)W (y x). Fo two sequences f(n) nd g(n), we wite f(n) =. g(n) f(n) if lim n! n log g(n) =0, nd similly fo pple nd. All logithms hve bse e, nd ll tes e in units of nts except in the exmples, whee bits e used. We define [c] + = mx{0,c}, nd denote the indicto function by { }. (5) II. RANDOM-CODING ERROR PROBABILITY In this section, we pesent non-symptotic nlysis of the ndom-coding eo pobbility fo n bity codewod distibution Q (x). While it is possible to wite n exct expession fo [], its computtion is usully infesible even fo modete vlues of n. Similly to [], we cn uppe bound the eo pobbility by ssug tht ties e boken t ndom, nd pply the union bound to obtin [5] pple RCU(n, M) = E, (M )P[q(, Y ) q(, Y ), Y ] (6) whee (, Y, ) Q (x)w (y x)q (x). This is the ndom-coding union (RCU) bound fo mismtched decodes. In ode to lowe bound the ensemble vege eo pobbility, we mke use of lowe bound on the pobbility of union of events due to de Cen, which sttes tht [6] " k [ P A i k P[A i ] 2 P k j= P[A i \ A j ] fo n bity sequence of pobbilistic events A,...,A k. In the cse tht the events e piwise independent nd identiclly distibuted, we theefoe obtin " k [ P A i (7) kp[a ] +(k )P[A ]. (8) Theoem. The ndom-coding eo pobbility fo the mismtched decode which esolves ties ndomly stisfies whee RCU L (n, M) RCU(n, M) (9) RCU L (n, M) = pple (M )P[q(, Y ) q(, Y ), Y ] 2 E +(M 2)P[q(, Y ) q(, Y ), Y ] + 2 E pple (M )P[q(, Y ) >q(, Y ), Y ] +(M 2)P[q(, Y ) >q(, Y ), Y ] (0) nd (, Y, ) Q (x)w (y x)q (x). Poof: Conside fixed codebook C with eo pobbility (C). Let B 0 be the event tht one o moe codewods yield stictly highe metic thn the tnsmitted one, nd let B` (` ) be the event tht the tnsmitted codewod yields metic which is equl highest with ` othe codewods. We
hve M (C) =P[B 0 ]+ P[B`] P[B 0 ]+ 2 `= M ` ` + () P[B`] (2) `= = 2 p0 e(c)+ 2 P[B 0] (3) whee p 0 e(c) = P[B 0 ]+ P M `= P[B`] is the eo pobbility of decode which decodes ties s eos. Aveging (3) ove the ndom-coding distibution, we obtin 2 P " M [ i=2 + 2 P " M [ i=2 q( (i), Y ) q( (), Y ) q( (i), Y ) >q( (), Y ) () whee ( (), Y ) Q (x)w (y x) nd the (i) (i 2) e geneted ccoding to Q (x), independently of () nd Y. The fist inequlity in (9) follows by witing ech pobbility in () s n expecttion given () nd Y, nd pplying the lowe bound in (8). The second inequlity in (9) follows by lowe bounding (0) by the fist of the two tems, eplcing M 2 in the denoto with M, nd pplying the inequlity + 2 {, }. The complexity of the computtion of RCU L (n, M) is essentilly identicl to tht of RCU(n, M). The second inequlity in (9) shows tht RCU(n, M) is ensemble-tight to within fcto of fou, which will be useful fo obtining ensemble-tight eo exponents in Section III. We compe the uppe nd lowe bounds numeiclly by consideing the chnnel defined by the enties of the mtix 2 3 2 0 0 0 2 5 (5) 2 2 2 2 with = Y = {0,, 2}. The mismtched decode chooses the codewod which is closest to y in tems of Hmg distnce. Fo exmple, the decoding metic cn be tken to be the enties of the mtix 2 3 2 2 5 (6) 2 fo ny 2 (0, 3 ). Tht is, the decode uses metic which is mtched to symmetic chnnel, but the tue chnnel is symmetic. We set 0 =0.0, =0.05 nd 2 =0.25 nd conside the ensemble in which ech symbol of ech codewod is geneted independently ccoding to Q = ( 3, 3, 3 ). Unde these pmetes, we hve tht I GMI (Q) = 0.63, I LM (Q) = 0.728 nd I(; Y ) = 0.763 bits/use. Figue plots RCU(n, M) nd RCU L (n, M) fo n = 50, unde both mismtched decoding nd ML decoding. We obseve 0 0 0 2 0 0 6 0 8 0 0 RCU(n, M )(mismtched) RCU L (n, M )(mismtched) RCU(n, M )(ML) RCU L (n, M )(ML) 0 0.2 0. 0.6 0.8 R (bits/chnnel use) Figue. Rndom coding uppe nd lowe bounds fo the chnnel defined in (5) with n =50, 0 =0.0, =0.05, 2 =0.25 nd Q =( 3, 3, 3 ). The mismtched decode uses the imum Hmg distnce metic given in (6). vey close mtch between the uppe nd lowe bounds coss ll tes, pticully in the cse of ML decoding. The slightly lge gp in the mismtched cse is due to n incesed pobbility of decoding ties, ising fom the fct tht simple decoding metic is being used. III. RANDOM-CODING ERROR EPONENTS In this section, we conside thee fmilies of the ndomcoding distibution Q (x), ech of which depends on n input distibution Q(x). ) The i.i.d. ensemble is given by ny Q (x) = Q(x i ). (7) In wods, ech symbol of ech codewod is geneted independently ccoding to Q. 2) The constnt-composition ensemble is given by Q (x) = T (Q n ) x 2 T (Q n) (8) whee Q n is the most pobble type unde Q. Tht is, ech codewod is geneted unifomly ove the type clss T (Q n ), nd hence ech codewod hs the sme composition. 3) The cost-constined i.i.d. ensemble is given by ( ) Q (x) = ny n Q(x i ) (x i ) pple µ n n n (9) whee (x) is cost function, = E Q [()], is positive constnt which does not vy with n, nd µ n is nomlizing constnt. Roughly speking, ech codewod is geneted ccoding to n i.i.d. distibution
conditioned on the empiicl men of (x) being vey close to the tue men. The cost function should not be viewed s being chosen to meet system constint (e.g. powe limittions). Rthe, it is intoduced in ode to impove the pefomnce of the ndom-coding ensemble; see [7], [0] fo detils. We cn ewite ech of the ndom-coding distibutions in (7) (9) s n i.i.d. distibution conditioned on the empiicl distibution of x being in pticul set of types. To this end, we intoduce the genel ensemble defined by Q (x) = µ 0 n ny Q(x i ){p x 2G n } (20) whee G n 2P n ( ) is the set of possible codewod types nd µ 0 n is nomlizing constnt. The i.i.d. ensemble is ecoveed by setting G n = P n ( ), the constnt-composition ensemble is ecoveed by setting G n = {Q n }, nd the costconstined i.i.d. ensemble is ecoveed by setting G n = {P : E P [()] pple n }. We detee the exponentil behviou of the genel ensemble in (20) using known popeties of types, nlogous to the nlysis of the ML decode given in [3]. We define the sets S n (G n ) = P Y 2P n ( Y):P 2G n (2) T n (P Y, G n ) = e P Y 2P n ( Y): e P 2G n, ep Y = P Y, E ep [log q(, Y )] E P [log q(, Y )]. (22) Roughly speking, S n is the set of possible types of ( (m), Y ), nd T n is the set of types of ( (m0), Y ) which led to eos given tht ( (m), Y ) 2 T (P Y ), whee m is the tnsmitted messge nd m 0 is diffeent messge. Theoem 2. Suppose G n is such tht P[ 0. 2 G n ] =, whee 0 Q n Q(x i). Then the ndom-coding eo pobbility fo the ndom-coding ensemble in (20) stisfies whee E,n (Q, R, G n ) = whee. =exp n,n (Q, R, G n ) (23) D(P Y kq W )+ P Y 2S n(g n) Poof: Fom (6) nd (9), we hve ep Y 2T n(p Y,G n) h D( e P Y kq e P Y ) Ri +. (2). = RCU(n, M) =E [ (, Y )] (25) (x, y) =, (M )P q(, y) q(x, y). (26) Let P Y denote the joint type of (x, y). Since (x, y) depends only on P Y, we wite (P Y ) = (x, y). Fom the definition of T n (P Y, G n ), we hve tht q(, y) q(x, y) if nd only if (, y) 2T n (P Y, G n ), nd hence n (P Y )=, (M ) P (, y) 2 T ( P e Y ) o. (27) ep Y 2T n(p Y,G n) Fom (20), the distibution of is the sme s tht of 0 Q n Q(x i) conditioned on the event tht 0 2G n. Hence, using the ssumption tht P[ 0 2G n ]. =, we obtin (P Y ) =., (M ) ep Y 2T n(p Y,G n). = mx, ep Y 2T n(p Y,G n) P ( 0, y) 2 T ( e P Y ) (28) (M ) exp nd( e P Y kq e P Y ) (29) whee (29) follows fom the popety of types in [3, Eq. (8)], nd the fct tht the numbe of joint types is polynomil in n. Expnding the expecttion in (25), we obtin. = P [(, Y ) 2 T (P Y )] (P Y ) (30) P Y 2S n(g n) nd nely identicl gument to (27) (29) yields. = mx P Y 2S n(g n) exp nd(p Y kq W ) (P Y ). (3) The poof is concluded by substituting (29) into (3). Using Theoem 2, we obtin ensemble-tight eo exponents fo the ensembles defined in (7) (9). Specificlly, defining the sets nd S iid = P( Y) (32) S cc (Q) = P Y 2P( Y):P = Q (33) S cost () = P Y 2P( Y):E P [()] = (3) T iid (P Y ) = P e Y 2P( Y): ep Y = P Y, E ep [log q(, Y )] E P [log q(, Y )] (35) T cc (P Y ) = e P Y 2P( Y): e P = P, ep Y = P Y, E ep [log q(, Y )] E P [log q(, Y )] (36) T cost (P Y,) = e P Y 2P( Y):E ep [()] =, ep Y = P Y, E ep [log q(, Y )] E P [log q(, Y )] (37) we obtin the following coolly.
Coolly 3. The ndom-coding eo exponents fo the ensembles defined in (7) (9) e espectively given by (Q, R) = P Y 2S iid D(P Y kq W )+ E cc (Q, R) = P Y 2S cc (Q) ep Y 2T iid (P Y ) D(P Y kq W )+ (Q, R, ) = P Y 2S cost () D(P Y kq W )+ h D( e P Y kq e P Y ) Ri + (38) ep Y 2T cc (P Y ) h I ep (; Y ) Ri + (39) ep Y 2T cost (P Y,) h D( e P Y kq e P Y ) Ri +. (0) Poof: The ssumption P[ 0. 2G n ] =in Theoem 2 holds tivilly fo the i.i.d. ensemble, nd ws shown to hold in [2, Eq. (2.6)] nd [0, Eq. (88)] fo the constntcomposition ensemble nd cost-constined i.i.d. ensemble espectively. Since ny pobbility distibution cn be ppoximted bitily well by type fo sufficiently lge n, the imiztions ove types in (2) cn be eplced by imiztions ove ll pobbility distibutions [2]. Similly, the constint P = Q n fo the constnt-composition ensemble cn be eplced by P = Q, nd the constint E P [()] pple n fo the cost-constined i.i.d. ensemble cn be eplced by E P [()] =, egdless of the vlue of. Simil guments pply to the constints on P e fo the constnt-composition nd cost-constined i.i.d. ensembles. The optimiztion poblems in (38) (0) e ll convex fo fixed input distibution nd cost function. Using the method of Lgnge dulity [7], ech exponent cn be witten in n ltentive fom. Theoem. The eo exponents in (38) (0) cn be expessed s whee iid (Q, R) = mx 2[0,] Eiid 0 (Q, ) R () E cc (Q, R) = mx 2[0,] Ecc 0 (Q, ) R (2) cost (Q, R, ) = mx 2[0,] Ecost 0 (Q,, ) R (3) 0 (Q, ) = s 0 E cc " E q(,y ) s Y log E q(, Y ) s 0 (Q, ) = s 0, " pple E q(,y ) s e () Y E log E q(, Y ) s e () () (5) 0 (Q,, ) = s 0,, " E q(,y ) s e (() ) Y log E q(, Y ) s e(() ) nd (, Y, ) Q(x)W (y x)q(x). (6) Poof: The poofs e simil fo ech of the thee ensembles, so we povide sketch only fo cc. Applying [] + = mx 2[0,] to (39) nd using Fn s imx theoem [8], we obtin whee cc (Q, R) = mx Ê0 cc (Q, ) R (7) 2[0,] Ê0 cc (Q, ) = P Y 2S cc (Q) ep Y 2T cc (P Y ) D(P Y kq W )+ I ep (; Y ). (8) It emins to show vi Lgnge dulity tht Ê0 cc (Q, ) = E0 cc (Q, ). We fist fix P Y nd conside the poblem I ep (; Y ) (9) ep Y 2T cc (P Y ) the Lgnge dul of which is given by [6] P Y (x, y) log s 0, q(x, y) s e (x) Px P (x)q(x, y) s e (x) (50) whee s nd (x) e Lgnge multiplies. Substituting (50) into (8) yields - poblem, whee the imiztion is ove P Y 2 S(P ) nd the emum is ove s 0 nd. Using Fn s imx theoem [8], the ode of these optimiztions cn be swpped. The poof is concluded by fog the Lgnge dul poblem of the esulting optimiztion ove P Y. The expessions in (39) nd () ppe in [9] nd [2] espectively, though both deivtions e diffeent to ous. To ou knowledge, the ltentive expessions in (38) nd (2) hve not ppeed peviously, nd the exponent cost is new. We note tht the function (x) epesents diffeent quntities fo the constnt-composition ensemble nd cost-constined i.i.d. ensemble. In the fome cse, ises s mthemticl optimiztion pmete, whees in the ltte cse, is design pmete fo the ndom-coding ensemble. The exponents in () (3) cn be deived diectly, the thn vi Lgnge dulity. The deivtion of () is pesented in [2], nd the expession in (2) follows by combining the chievble eo exponent of [9] with the fct tht unde constnt-composition codes, the metic q(x, y) is equivlent to the metic q(x, y) s e (x) fo ny s 0 nd [3]. The diect deivtion of (3) is simil to tht of [0], whee it ws shown tht n chievble eo exponent fo the cost-constined i.i.d. ensemble is given by cost0 (Q, R, ) = mx 0 2[0,] Ecost 0 (Q,, ) R (5)
"! E0 cost0 (Q,, ) = E[q(,Y ) s e () Y ] log E. s 0 q(, Y ) s e () (52) e Roughly speking, the dditionl fcto of (x) in (52) e (x) comped to the i.i.d. ensemble in () ises fom the fct tht e P i (xi) is close to e P i (xi) fo ny codewods x nd x. Fo the function E0 cost in (6), the dditionl fcto e ((x) ) ises fom the fct tht e P i (xi) is close to e n, nd similly fo the fcto e ((x) ). We will see tht the efined exponent cost impoves on cost0 in genel. While none of the bove diect deivtions pove ensemble tightness, they ech hve the dvntge of extending immeditely to continuous lphbets fte eplcing the ppopite sums by integls, except tht the input lphbet must be finite fo the constnt-composition ensemble. A. Connections Between the o Exponents The constints on P Y nd P e Y in () (3) e given by the sets defined in (32) (37). Since the constint E P [()] = holds by definition when P = Q, we hve tht S iid S cost () S cc (Q) fo ny given cost function nd input distibution Q. A simil obsevtion pplies to the constints on P e, nd it follows tht (Q, R) pple cost (Q, R, ) pple cc (Q, R). (53) This indictes tht the constnt-composition ensemble yields the best eo exponent of the thee ensembles unde considetion. Futhemoe, by setting = =in (6), we obtin the inequlity cost0 (Q, R, ) pple cost (Q, R, ) (5) whee stict inequlity is possible. In the cse tht (x) does not depend on x, we obtin (Q, R, ) =0 nd hence we hve in genel tht (Q, R, ) = (Q, R) (55) (Q, R) pple cost0 (Q, R, ). (56) The following theoem gives the connection between cost0 nd cc, nd shows tht the two e equl fte optimiztion ove Q nd. This esult is nlogous to connection between the i.i.d. ensemble nd constnt-composition ensemble unde ML decoding [9]. Theoem 5. E cc 0 (Q, ) cn be expessed s E0 cc (Q, ) = mx eq2p( ) Consequently, mx Q2P( ) Ecc (Q, R) = 0 0 ( e Q,, ) ( + )D(Qk e Q) (57) mx Q2P( ) 0 (Q, R, ). (58) Poof: Since the emum ove in (5) is ove ll el-vlued functions on, n equivlent expession is obtined by defining e(x) such tht e e(x) = e (x) Q(x) eq(x) some e Q(x), nd insted tking the emum ove e. Simple lgebic mnipultions yield eq(x) 0 (Q, ) = Q(x) log W (y x) s 0,e Q(x) x y Px Q(x)q(x, e y)e e(x) ( + )D(QkQ) q(x, y) s e e (59) e(x) E cc s 0,e log eq(x)w (y x) Px Q(x)q(x, e y)e e(x) q(x, y) s e e(x) fo ( + )D(Qk e Q) (60) whee (60) follows fom Jensen s inequlity. It emins to show tht equlity holds in (60) fte mximizing ove Q. e Fist, by nlyzing the Kush-Kuhn-Tucke (KKT) conditions fo the optimiztion poblem ove Q, e s nd e ssocited with (60), it cn be shown tht eq(x) Q(x) W (y x) y s q(x, y) e eq(x) e(x) q(x, y) x e e(x)! (6) is constnt fo ll x such tht Q(x) > 0 unde the optiml pmetes, implying equlity in (60). The poof is concluded by showing tht the esulting vlues of s nd (the ltte being computed using Q e nd e) lso stisfy the KKT conditions fo the optimiztion poblem ove s nd ssocited with (5). Detils e omitted fo the ske of edbility. The equlity in (58) follows immeditely fom (57) nd the fct tht D(QkQ) e 0 with equlity if Q = Q. e It follows fom (53) nd Theoem 5 tht cost0 is tight with espect to the ensemble vege when the optiml vlues of both Q nd e used. Hence, lthough cost is tighte eo exponent thn cost0 in genel, it does not impove on the best chievble eo exponent using cost-constined i.i.d. ndom coding. Futhemoe, both exponents cn be used to pove the chievbility of the LM te nd no bette. Howeve, the efined exponent cost is useful in the cse tht one does not hve complete feedom in choosing Q nd, o when exct optimiztion ove ech is not fesible. Fo exmple, if the codebook designe does not know the chnnel, then the objective in (52) cnnot be computed in ode to pefom the optimiztion. The following theoem gives two futhe connections between the eo exponents in the cse of ML decoding. Theoem 6. If q(x, y) =W (y x) then cost0 (Q, R, ) = iid (Q, R) (62) We ssume tht Q(x) e > 0 wheeve Q(x) > 0, since ll othe choices of Q e mke the objective in (57) equl to nd hence cnnot chieve the mximum.
nd fo ny given Q nd R. cost (Q, R, ) = cc (Q, R) (63) Poof: We obtin (62) by optimizing the objective in () ove s, nd optimizing the objective in (52) ove s nd. It ws shown in [, Ex. 5.6] tht the optiml vlue of s in () is equl to +. Following the sme steps, we obtin tht tht the optiml vlue of s is in (52) is lso equl to, nd the optiml cost function (x) does not depend on + x. Combining these esults, we obtin (62). Fom (53), it suffices to pove tht cost cc in ode to pove (63). To this end, we will show tht E0 cost (Q,, ) E0 cc (Q, ) fo ll 2 [0, ]. The cse =0is tivil, so we ssume tht >0. We set = nd = nd wite the expecttion inside the logithm of (6) s Px Q(x)W (y x) Q(x)q(x, y)s e (x) q(x, y) s e e (x) e. (6) We ssume without loss of genelity tht Q(x) > 0 fo ll x. Intoducing the distibution Q(x) e = Q(x)e(x) E Q, we cn wite [e () ] (6) s EQ [e () + ] Px eq(x)w (y x) e! Q(x)q(x, y) s e q(x, y) s. (65) The summtion in (65) coincides with the expecttion inside the logithm of (), nd with some simple lgebic mnipultion it cn be shown tht E Q [e () ] e =exp D(Qk e Q). (66) Substituting (65) nd (66) into (6) nd noting tht suitble choice of cn yield ny distibution e Q such tht e Q(x) > 0 fo ll x, we obtin 0 (Q,, ) mx eq 0 ( e Q, ) ( + )D(Qk e Q). (67) Tking ccount of (62) nd Theoem 5, the ight-hnd side of (67) is equl to E0 cc (Q, ), nd the poof is complete. Unde ML decoding, iid is simply Gllge s ndomcoding eo exponent [], while cc is Csiszá s ndomcoding eo exponent fo constnt-composition codes [2]. We hve thus shown tht using n optimized cost function, we cn chieve Csiszá s exponent in the mtched setting without using constnt-composition codes. Howeve, the nlysis thus f does not give pecise connection between nd E cc B. Multiple Cost Constints when the decoding metic diffes fom ML. In this subsection, we outline how the exponent cc cn be chieved in the mismtched setting using cost-constined i.i.d. ndom coding. To this end, we intoduce the costconstined i.i.d. ensemble with L cost constints, given by Q (x) = ( ny Q(x i ) µ n n n l (x i ) l pple l n,l=,...,l ) (68) whee fo ech l 2 {,...,L}, l is cost function, l = E Q [ l ()] nd l is positive constnt. By using the multidimensionl centl limit theoem nd extending the nlysis of [0, Eq. (88)], it cn be shown tht the nomlizing constnt µ n decys to zeo sub-exponentilly,. i.e. µ n =. Using this esult nd following the nlysis of the cse L =t the beginning of this section, we obtin the exponent cost (Q, R, { l }) = mx 2[0,] Ecost 0 (Q,, { l }) R (69) whee 0 (Q,, { l }) = s 0,{ l },{ l } " E q(,y ) s e P L l= l( l () l) Y log E q(, Y ) s e P L. (70) l= l( l () l) Hee we wite { l } s shothnd fo {,..., L }, nd similly fo { l } nd { l }. The constnt-composition ensemble is in fct specil cse of the ensemble in (68), since it is obtined by setting L = nd l < fo ll l, nd choosing the cost functions =(, 0,...,0), 2 =(0,, 0,...,0), etc. We will show, howeve, tht the exponent cc cn be ecoveed using only two cost functions, egdless of the cdinlity of. Setting L =2nd choosing = = 2 =nd =, the expecttion in (70) becomes Px Q(x)W (y x) Q(x)q(x, y)s e (x)+2(x) e 2(x). q(x, y) s e (x)+ 2 e 2 (7) Defining Q(x) e = Q(x)e 2 (x) nd following identicl steps to E Q [e 2 () ] (65) (67), we obtin E0 cost (Q,, {, 2 }), 2 mx eq 0 0 ( Q, e, ) ( + )D(QkQ). e (72) By Theoem 5, the ight-hnd side of (72) is equl to E0 cc (Q, ), nd we hve thus ecoveed the exponent cc. C. Numeicl Results In this subsection, we plot the exponents fo the chnnel defined in (5) unde both the imum Hmg distnce nd ML decoding metics, gin using the pmetes 0 = 0.0, =0.05 nd 2 =0.25. We set Q =(0., 0.3, 0.6), which we hve intentionlly chosen suboptimlly to highlight
o Exponent 0.2 0.8 0.6 0. 0.2 0. 0.08 0.06 0.0 0.02 cc iid cc (ML) (ML) (L =2) (L =) 0 0 0 0. 0.2 0.3 0. 0.5 R (bits/chnnel use) Figue 2. o exponents fo the chnnel defined in (5) with 0 =0.0, =0.05, 2 =0.25 nd Q =(0., 0.3, 0.6). The mismtched decode uses the imum Hmg distnce metic given in (6). The coesponding chievble tes I GMI (Q), I LM (Q) nd I(; Y ) e espectively mked on the hoizontl xis. the diffeences between the eo exponents when the input distibution is fixed. Unde these pmetes we hve tht I GMI (Q) =0.387, I LM (Q) =0.9 nd I(; Y )=0.7 bits/use. We evlute the exponents using the optimiztion softwe YALMIP [20]. The exponent cost0 is optimized ove, pefog the optimiztion jointly with s in (52) nd llowing the cost function to vy coss tes. The cost function fo cost (L =) is chosen using n ltenting optimiztion between nd (s,, ) in (6), using the which mximizes E0 cost0 s stting point nd teting when the chnge in the exponent between itetions becomes negligible. Similly, the cost functions fo cost (L =2) e chosen using n ltenting optimiztion between (, 2 ) nd (s,, 2,, 2 ), initilly setting both nd 2 to be equl to the which mximizes E0 cost0. Fom Figue 2, we see tht cc nd cost (L =2) e indistinguishble, indicting tht the ltenting optimiztion technique ws effective in finding the tue exponent. The exponent cost (L =) is only mginlly lowe, while the gp to cost0 is moe significnt. The exponent iid is not only lowe thn ech of the othe exponents, but lso yields wose chievble te. This exmple demonsttes tht fo fixed Q, the efined exponent cost (L =) cn outpefom even when is optimized. 0 IV. CONCLUSION We hve developed tight chcteiztion of the ndomcoding eo pobbility fo mismtched decodes. A new chievble eo exponent hs been deived fo the costconstined i.i.d. ensemble, nd ltentive foms of existing exponents fo the i.i.d. ensemble nd constnt-composition ensemble hve been given. The exponent fo ech ensemble is tight with espect to the ensemble vege fo ny discete memoyless chnnel nd decoding metic. Fo ny given input distibution nd cost function, the eo exponent fo the constnt-composition ensemble is t lest s high s tht of the cost-constined i.i.d. ensemble, which in tun is t lest s high s tht of the i.i.d. ensemble. We hve shown the eo exponent fo the constnt-composition ensemble cn be ecoveed using cost-constined i.i.d. ndom coding with t most two cost constints. REFERENCES [] R. Gllge, Infomtion Theoy nd Relible Communiction. John Wiley & Sons, 968. [2] G. Kpln nd S. Shmi, Infomtion tes nd eo exponents of compound chnnels with ppliction to ntipodl signling in fding envionment, AEU, vol. 7, no., pp. 228 239, 993. [3] I. Csiszá nd P. Nyn, Chnnel cpcity fo given decoding metic, IEEE Tns. Inf. Theoy, vol. 5, no., pp. 35 3, Jn. 995. [] J. Hui, Fundmentl issues of multiple ccessing, Ph.D. dissettion, MIT, 983. [5] V. Blkisky, A convese coding theoem fo mismtched decoding t the output of biny-input memoyless chnnels, IEEE Tns. Inf. Theoy, vol., no. 6, pp. 889 902, Nov. 995. [6] N. Mehv, G. Kpln, A. Lpidoth, nd S. Shmi, On infomtion tes fo mismtched decodes, IEEE Tns. Inf. Theoy, vol. 0, no. 6, pp. 953 967, Nov. 99. [7] A. Gnti, A. Lpidoth, nd E. Telt, Mismtched decoding evisited: genel lphbets, chnnels with memoy, nd the wide-bnd limit, IEEE Tns. Inf. Theoy, vol. 6, no. 7, pp. 235 2328, Nov. 2000. [8] A. Lpidoth, Mismtched decoding nd the multiple-ccess chnnel, IEEE Tns. Inf. Theoy, vol. 2, no. 5, pp. 39 52, Sep. 996. [9] I. Csiszá nd J. Köne, Gph decomposition: A new key to coding theoems, IEEE Tns. Inf. Theoy, vol. 27, no., pp. 5 2, Jn. 98. [0] S. Shmi nd I. Sson, Vitions on the Gllge bounds, connections, nd pplictions, IEEE Tns. Inf. Theoy, vol. 8, no. 2, pp. 3029 305, Dec. 2002. [] R. Gllge, The ndom coding bound is tight fo the vege code, IEEE Tns. Inf. Theoy, vol. 9, no. 2, pp. 2 26, Mch 973. [2] I. Csiszá nd J. Köne, Infomtion Theoy: Coding Theoems fo Discete Memoyless Systems, 2nd ed. Cmbidge Univesity Pess, 20. [3] R. Gllge, Fixed composition guments nd lowe bounds to eo pobbility, http://web.mit.edu/gllge/www/notes/notes5.pdf. [] Y. Polynskiy, V. Poo, nd S. Vedú, Chnnel coding te in the finite blocklength egime, IEEE Tns. Inf. Theoy, vol. 56, no. 5, pp. 2307 2359, My 200. [5] A. Mtinez nd A. Guillén i Fàbegs, Sddlepoint ppoximtion of ndom-coding bounds, in Inf. Theoy App. Wokshop, L Joll, CA, 20. [6] D. de Cen, A lowe bound on the pobbility of union, Discete Mthemtics, vol. 69, pp. 27 220, 997. [7] S. Boyd nd L. Vndenbeghe, Convex Optimiztion. Cmbidge Univesity Pess, 200. [8] K. Fn, Minimx theoems, Poc. Nt. Acd. Sci., vol. 39, pp. 2 7, 953. [9] G. Poltyev, Rndom coding bounds fo discete memoyless chnnels, Pob. Inf. Tnsm., vol. 8, no., pp. 9 2, 982. [20] J. Löfbeg, YALMIP : A toolbox fo modeling nd optimiztion in MATLAB, in Poc. CACSD Conf., Tipei, Tiwn, 200.