Shannon meets von Neumann: A Minimax Theorem for Channel Coding in the Presence of a Jammer

Transcription

1 Shao meet vo Neuma: A Mii Theorem for Chael Codig i the Preece of a Jammer Sharu Therea Joe Akur A. Kulkari arxiv: v c.it] 8 Nov 08 Abtract We tudy the ettig of chael codig over a family of chael whoe tate i cotrolled by a adverarial jammer by viewig it a a zero-um game betwee a fiite blocklegth ecoder-decoder team, ad the jammer. The ecoder-decoder team chooe tochatic ecodig ad decodig trategie to miimize the average probability of error i tramiio, while the jammer chooe a ditributio o the tate-pace to imize thi probability. The mi- value of thi game i equivalet to chael codig for a compoud chael we call thi the Shao olutio of the problem. The -mi value correpod to fidig a mixed chael with the larget value of the miimum achievable probability of error. Whe the mi- ad -mi value are equal, the problem i aid to admit a addle-poit or vo Neuma olutio. While a Shao olutio alway exit, a vo Neuma olutio eed ot, owig to iheret ocovexity i the commuicatig team problem. Depite thi, we how that the mi- ad -mi value become equal aymptotically i the large blocklegth limit, for all but fiitely may rate. We explicitly characterize thi limitig value a a fuctio of the rate ad obtai tight fiite blocklegth boud o the mi- ad -mi value. A a corollary we get a explicit expreio for the ɛ-capacity of a compoud chael uder tochatic code the firt uch reult, to the bet of our kowledge. Our reult demotrate a deeper relatio betwee the compoud chael ad mixed chael tha wa previouly kow. They alo how that the covetioal iformatio-theoretic viewpoit, articulated via the Shao olutio, coicide aymptotically with the game-theoretic oe articulated via the vo Neuma olutio. Key to our reult i the derivatio of ew fiite blocklegth upper boud o the mi- value of the game via a ovel achievability cheme, ad lower boud o the -mi value obtaied via the liear programmig relaxatio baed approach we itroduced i ]. I. INTRODUCTION Commuicatio theory ha traditioally tudied the problem of commuicatio i the preece of a jammer, oly from the commuicater perpective. The olutio ought have cocetrated o what the commuicatio ytem ca achieve i the wort cae over all poible ceario; we call thi the Shao olutio. However, alogide oe could alo ak a dual quetio, amely, what i the imum damage that the jammer could achieve i the preece of a itelliget commuicatio ytem? A atural vatage poit for aalyzig the ytem from both poit of view together i that of a game betwee the commuicatio ytem ad the jammer. Oe could the employ olutio cocept from game theory to olve the game ad aalyze the ytem; we refer to the reultig olutio Akur with the Sytem ad Cotrol Egieerig group, Idia Ititute of Techology Bombay, Mumbai, , Idia. Thi work wa doe partly while Sharu wa a Ph.D. tudet i thi group. They ca be cotacted at harutherea@iitb.ac.i, kulkari.akur@iitb.ac.i. Thi work will be preeted i part at the IEEE Coferece o Deciio ad Cotrol, to be held i December 08 ]. a the vo Neuma olutio. The focu of thi paper i a game of the above kid i the fiite blocklegth regime ad the relatio betwee the Shao ad vo Neuma olutio of commuicatio problem i the preece of adverarial actio. We defie the followig zero-um game betwee a fiite blocklegth commuicatio ytem ad a fiite tate jammer. The commuicatio ytem attempt to commuicate a dicrete uiform ource acro a dicrete memoryle chael by makig ue of the chael. The chael law i determied by it tate which i cotrolled by the jammer. The commuicatio ytem comprie of a team of two deciio maker a ecoder ad a decoder that together attempt to miimize the average probability of error icurred i the tramiio of the ource. The jammer chooe a tate radomly o a to imize the error icurred i tramiio. Followig Fig, let S deote a ource meage of rate R draw uiformly at Fig. : A commuicatio ytem with a fiite tate jammer radom from S :=,..., R. Suppoe the jammer chooe a chael tate from a et Θ radomly accordig to a ditributio q. The ecoder ad decoder chooe tochatic trategie. A tochatic ecoder map the ource S to a - legth chael iput X radomly accordig to a coditioal ditributio Q X S. The chael, i tate, output a -legth trig deoted Y, which i decoded by a tochatic decoder that output Ŝ S accordig to a coditioal ditributio QŜ Y. We aume that the et of tate, Θ, i fiite, ad that the tate oce choe remai fixed through the ue of the chael. The choice of the chael tate by the jammer i idepedet of the ource meage ad the ecoded meage, ad the actual tate realized i ot kow to either ecoder or decoder. The mi- value or upper value of thi game i deoted ϑ(; R) ad i the optimal value of the followig problem, P(; R).t mi Q X S,QŜ Y q EIS Ŝ] Q X S P(X S), QŜ Y P(Ŝ Y), q P(Θ). where X, Y are the et of -legth chael iput ad output, repectively ad P( ) i the et of probability ditributio o. The expectatio E above i uder the ditributio iduced by the code (Q X S, QŜ Y ) ad the ditributio q. Problem P(; R) thu correpod to the miimum probability

2 of error achievable by the ecoder-decoder team i the wort cae over all ditributio q choe by the jammer. The mi value or lower value of the game, deoted ϑ(; R), i the optimal value of the followig problem, P(; R).t q mi Q X S,QŜ Y EIS Ŝ] Q X S P(X S), QŜ Y P(Ŝ Y), q P(Θ), ad correpod to the imum probability of error achievable by the jammer i the wort cae over all tochatic code employed by the ecoder-decoder team. Note that the followig relatio alway hold: ϑ(; R) ϑ(; R), N, R 0, ). () The above zero-um game i aid to admit a addle poit value, or a vo Neuma olutio, if equality hold, i.e., if ϑ(; R) = ϑ(; R). Havig defied the game, we ak our firt quetio. Doe the game admit a addle poit? For each trategy of the jammer, the team problem of the ecoder ad decoder ha oclaical iformatio tructure 3]. A argued i 4], i the pace of tochatic code, the commuicatig team problem i ocovex for each jammer trategy. Thi lack of covexity implie that the exitece of a addle poit i ot guarateed. Moreover, uig the code that form a addle poit (aumig oe exit) may imply capacitie, error expoet ad uchlike that are ditict from thoe obtaied from the Shao olutio. Thi lead u to our ecod quetio: do the awer obtaied from the addle poit, i.e., the vo Neuma olutio, coicide with thoe obtaied from the Shao olutio? We fid that, depite the kepticim voiced above, awer to both thee quetio are i the affirmative i the large blocklegth limit, for all but fiite may value of the rate R of the ource. The mai reult i thi paper how that the aymptotic value of the miimum probability of error achievable by a code of rate R, i the wort cae over all jammer trategie, equal the aymptotic value of the imum probability of error that a jammer ca iduce, i the wort cae over all poible code of rate of R, for all rate R barrig ome fiitely may pecific value. I other word, lim ϑ(; R) = lim ϑ(; R), R 0, )\D, where D i a fiite et. Moreover, ϑ(r), which i defied a the value of the above limit whe they are equal i give by ϑ(r) = mi q() R < C(Θ ), q P(Θ) Θ Θ Θ where C(Θ ) i the capacity of the compoud chael formed by Θ Θ, ad D i preciely the et of poit of dicotiuity of the right-had ide above whe viewed a a fuctio of R 0, ). At a fiite blocklegth a approximate mii theorem hold, i.e., the differece ϑ(; R) ϑ(; R) become vaihigly mall with for all R 0 except i D. Iteretigly, the upper value, ϑ(; R) alo happe to equal the probability of error guarateed by the Shao olutio. Coequetly, we fid that a the blocklegth become large, the Shao olutio come i harmoy with the vo Neuma olutio. A a corollary of thee reult we alo obtai the ɛ-capacity of a compoud chael uder tochatic code, the firt uch reult, to the bet of our kowledge. Depite the ocovexity of the commuicatig team problem argued i 4], our recet reult ], 5], 6] ad 7] have how that all poit-to-poit problem ad everal etwork problem (without a jammer) admit a ear-covexity, i the ee that they ca be approximated aymptotically by a liear program. Thi lead u to cojecture that a approximate mii theorem may be withi reach, whoe approximatio become icreaigly accurate a. Our reult how that thi ituitio i correct for almot all rate. Sice the tate oce choe i held fixed throughout the tramiio, problem P(; R) correpod to codig for a fiite blocklegth compoud chael 8] uder tochatic code. Problem P(; R) o the other had amout to foitig a ditributio o the tate pace o that the reultig fiite blocklegth mixed chael 9] ha the larget probability of error. To the bet of our kowledge, our i the firt reult howig that thee are aymptotically the ame. The compoud ad mixed chael have bee kow to be itimately related (ee 8], 9]); i particular they have the ame capacity. Our reult how that there i a eve deeper relatio betwee them. Iteretigly, thi mii theorem break dow if oe coider the imum probability of error criterio; there i alway a iterval of rate where there i a gap betwee the upper ad lower value. Furthermore, to the bet of our udertadig, local radomizatio produced by tochatic code eem eetial ad we have ot bee able to how imilar reult with determiitic code. Local radomizatio by the ecoder allow the code to radomly commuicate with the correct codebook with a poitive probability, regardle of the actio of the jammer. Correctly chooig thi radomizatio achieve the optimal performace. It i uclear if the ame ca be achieved with determiitic code. Ideed, formulatig the problem i the pace of tochatic code wa alo key to our ear-covexity reult for codig problem foud i ], 5], 6] ad 7]. There are two regime where our reult i rather eay to claim. Defie, C = mi I P X (X; Y ), P X Θ C = mi Θ P X I PX (X; Y ), () where I PX (X; Y ) i the mutual iformatio betwee the (igle letter) chael iput X ad chael output Y whe the tate i ad the chael iput ditributio i P X. C i the capacity of the compoud chael (uder determiitic ad tochatic code). Clearly, by defiitio of capacity, for R < C, the upper value ad hece the lower value of the game mut go to zero a. C i the mallet of the capacitie of idividual DMC defied by the tate of the chael. For R > C, the jammer ca chooe the tate with mallet capacity (with probability oe), whereby by the trog covere for chael codig for thi DMC, the lower, ad hece upper value mut both approach uity. Thu the aymptotic mii theorem hold for R < C ad R > C. The otrivial cae i

3 3 To how our reult, we fid two categorie of boud: a lower boud o ϑ(; R), which i equivaletly a fiite blocklegth covere for the mixed chael, ad a upper boud o ϑ(; R), which i equivaletly a fiite blocklegth achievability reult for the compoud chael. The former i obtaied via the liear programmig (LP) approach we itroduced i ]. For the latter, i the regime R < C ad R > C, we employ a determiitic achievability cheme. I the itermediate regio C < R < C, we employ a tochatic cotructio that radomly chooe betwee codebook for compoud chael formed by ubet of Θ. Thi achievability cheme i ovel i the ee that it wa derived uig the covere a a referece, reiterpretig the latter via vo Neuma mii theorem, the ad attemptig to match it, rather tha the commo approach which attempt to fid a matchig covere for a achievability cheme. A. Related Work Fig. : ϑ(r) for four-tate jammer where C(Θ ) for Θ Θ =,, 3, 4 are aumed to atify: C < C(a) < C(b) < C(c) < C(d) < C(e) < C(f) < C(g) < C(h) < C(i) < C(j) < C = C() where a =,, 3, b =,, 4, c =, 3, 4, d =,, e =, 3, f =, 4, g =, 3, 4, h =, 3, i =, 4, j = 3, 4. The hollow circle idicate a excluded ed-poit rate. C < R < C where we alo how the mii theorem with a ew argumet that exploit the tochatic ecodig allowed. The limitig addle poit value ϑ(r) happe to take a peculiar form. It i a tep fuctio wherei the poit where jump occur deped o the capacitie of compoud chael that correpod to ubet of Θ. For example, i the cae where Θ =, we have 0 if R < C, ϑ(r) = if C < R < C, if R > C. For Θ = 3 (ay Θ =,, 3), aumig that the capacitie of compoud chael correpodig to ubet Θ atify, C < C(, ) < C(, 3) < C(, 3) < C = C(), we get 0 if R < C, 3 if C < R < C(, ), ϑ(r) = if C(, ) < R C(, 3), if C(, 3) < R < C(, 3), 3 if C(, 3) < R < C, if R > C. The addle poit value for Θ = 4 i how i Fig. The exitece of a addle poit olutio for a game compriig of a commuicatio ytem ad a jammer ha bee tudied multiple time i the LQG ettig (e.g., 0], ]). For tramittig Gauia ource over Gauia chael cotrolled by a power-cotraied jammer with acce to the ource, the exitece of a mixed addle poit olutio i how i ]. A exteio of the above work to geeral ource over additive oie chael ha bee coidered i 3]. Cloely related to our problem etup i the Arbitrarily Varyig Chael (AVC) model (ee 4]), where a jammer chage the tate of the chael at each itat durig the tramiio of the ecoded meage. AVC i ued to model packet-dropout adverarial chael i 5], where the cotrol problem over a adverarial chael i the poed a a zero-um game betwee the jammer ad the cotroller. For gauia AVC with power cotrait, the problem of evaluatig the aymptotic radom codig capacity i how to have a equivalet zero-um game formulatio betwee the tramitter ad the jammer i 6]. To the bet of our kowledge our i the oly work that tudie the ettig we coider, where the chael i the compoud chael. B. Orgaizatio Sectio II formulate the zero-um game betwee the fiite blocklegth commuicatio ytem ad the fiite tate jammer. We alo give a backgroud o zero-um game ad compoud ad mixed chael. I Sectio III, we preet the LPrelaxatio baed framework to obtai a lower boud o the -mi value of the game. Sectio IV preet a ew upper boud o the mi- value of the game ad aalye the aymptotic tighte of thi boud to the lower boud obtaied via LP for rate R < C ad R < C. I Sectio V, we coider the itermediate rate regio C < R < C. We derive a ovel fiite blocklegth upper boud o the mi- value of the game ad how that thi boud i aymptotically equal to the LP-baed lower boud for all but fiitely may rate i thi rage. Fially, we coclude with Sectio VI.

4 4 C. Notatio Throughout thi paper, upper cae letter A, B repreet radom variable takig value i pace repreeted by calligraphic letter A, B repectively ad lower cae letter a, b repreet the pecific value thee radom variable take. Let I repreet the idicator fuctio which i oe whe i true ad i zero otherwie. P( ) deote the et of all probability ditributio o. Let Z := S X Y Ŝ ad z := (, x, y, ŝ) Z. Let Q X S QŜ Y P Y X, (z, ) Q X S (x )QŜ Y (ŝ y)p Y X, (y x, ). If P repreet a optimizatio problem, OPT(P ) repreet it optimal value. If A repreet a et, A c repreet it complimet. Let L(R)HS repreet Left (Right) Had Side. Let x A be a -legth trig. The, i= P (a x) = Ix i = a, a A, repreet the type of x (ee 7]). Note that P (a x) 0 for all a A ad a A P (a x) =, whereby P P(A). Let T repreet the et of all type i A. From type coutig lemma 7], we the have, T ( + ) A. (3) II. BACKGROUND AND PROBLEM FORMULATION A. Zero-um game A zero-um game comprie of two player P, P, with trategie deoted z Z ad z Z, repectively, ad a fuctio c : Z Z R of thee trategie. P attempt to chooe z Z o a to miimize c ad P attempt to chooe z Z o a to imize thi fuctio, but the value each get deped ot oly what the player chooe but alo what the other player chooe. How would thee player the play? vo Neuma urmied that P, P would each chooe z, z, repectively, o a to miimize the wort cae damage that the other player could do. i.e., ad z arg mi z Z c(z ), z arg z Z c(z ), c(z ) := z Z c(z, z ), (4) c(z ) := mi z Z c(z, z ). (5) Clearly, thee calculatio may ot match up. Specifically, if P play z it may ot be optimal for P to play z, or if P play z, P may ot wat to tick to playig z. Remarkably, vo Neuma howed that i certai clae of game 8], thee calculatio do match up exactly ad we get c(z ) = c(z ). I thi cae, we get the cetral olutio cocept i zero-um game, propoed by vo Neuma, amely, a addle poit. (z, z ) are aid to form a addle poit of the game if c(z, z ) c(z, z ) c(z, z ) z Z, z Z. (6) The exitece of a addle poit i equivalet to the followig iterchageability of the mi ad operatio o c, i.e., mi z Z c(z, z ) = mi c(z, z ). (7) z Z z Z z Z I geeral, oe alway ha that the left had ide of (7) (called the upper value of the game, ad equal to c(z )) i greater tha or equal to the right had ide (called the lower value, ad equal to c(z )). vo Neuma mii theorem 8] howed that equality hold whe Z, Z are fiite dimeioal probability implice ad c(z, z ) z Az for a matrix A. B. Commuicatio i the preece of a jammer a a game We ow preet the problem of chael codig i preece of a jammer a a zero-um game. Let S repreet the radom ource meage ditributed uiformly o S =,..., M with M = R ad R i the rate of tramiio. Coider a family of chael P Y X, P(B A) with commo fiite iput ad output alphabet A ad B, repectively, parametrized by a tate takig value i a fiite et Θ. Defie X := A ad Y := B a the pace of -legth trig of chael iput ad output of ay chael from thi family. A ecoder map S to a -legth ( < ) trig X X radomly accordig to the ditributio Q X S P(X S). X i ubequetly et through a chael i tate Θ, repreeted by the tochatic kerel P Y X, to get the chael output Y Y. For each Θ, we aume that the chael i dicrete ad memoryle, i.e. P Y X, (y x, ) = P Y X, (y i x i, ), x X, y Y. (8) i= Subequetly, a decoder map the chael output Y radomly accordigly to QŜ Y P(S Y) to get Ŝ S. Here, Q X S repreet a tochatic ecoder, QŜ Y repreet a tochatic decoder ad together, they cotitute a tochatic code i the ee of Cizar ad Korer 7] or behavioral trategie i the laguage of game theory 9]. Note that thi i ditict from a radom code. A determiitic code i a pair of fuctio f : S X, g : Y S ad a radom code i a radomly choe pair of fuctio f, g. A jammer cotrol the tate of the chael ad chooe a tate Θ radomly accordig to ome ditributio q P(Θ) idepedet of S, X. State oce choe i held fixed through -ue of the chael ad the chael tate choe i ot kow to both the ecoder ad decoder. Thi ettig ca be formulated a a zero-um game by takig P a a team compriig of the tochatic ecoder ad decoder (Q X S, QŜ Y ) ad it trategy pace a Z = P(X S) P(S Y), the et of all tochatic code. The jammer i player P with it trategy pace a P(Θ) ad the cot fuctio c i the average probability of error over all meage, EIS Ŝ] evaluated uder the probability ditributio iduced by the trategie of the ecoder-decoder team ad the jammer. Sice for each value of, the radom variable S, X, Y, Ŝ form a Markov chai i that order, we have EIS Ŝ S =, ] = x,y,ŝ Q X S (x )QŜ Y (ŝ y)p Y X, (y x, )I ŝ, (9)

5 5 ad hece EIS Ŝ S = ] = q()eis Ŝ S =, ], Θ ad EIS Ŝ] = EIS Ŝ S = ]. M S The cetral quet i thi paper i to ivetigate whether a mii theorem ca be claimed for thi game. However, the awer to thi quetio i i the egative i geeral whe i fiite ice while EIS Ŝ] i cocave (i fact, liear) i q, it i ocovex i (Q X S, QŜ Y ), the ocovexity ariig due to the preece of biliear product Q X S (x )QŜ Y (ŝ y) (ee e.g., 4]) i (9). Coequetly, a addle poit value may ot exit for thi game. I thi cotext, we explore a approximate mi theorem for the zero-um game, the approximatio becomig icreaigly accurate a blocklegth icreae. Toward thi, we derive ew upper boud o ϑ(; R) ad lower boud o ϑ(; R) ad how that a earaddle poit hold for the zero-um game, i.e., for each N, 0 ϑ(; R) ϑ(; R) ɛ, ɛ 0, ], where for all but fiitely may value of the rate R, we have lim ɛ = 0. Coequetly, for uch rate we get lim ϑ(; R) = lim ϑ(; R) =: ϑ(r), where ϑ(r) i ow the limitig addle-poit value of the zeroum game. C. Backgroud o mixed ad compoud chael Our problem i itimately related to compoud ad mixed chael. We recall ome backgroud about them here ad relate them to our problem 4]. A compoud chael i defied by a family of chael P Y X, Θ whoe tate i held fixed throughout the tramiio. The average probability of error icurred by a code Q X S, QŜ Y over thi compoud chael i defied a ɛ com (Q X S, QŜ Y ; ) := Θ EIS Ŝ S =, ]. M For ay q P(Θ), a mixed chael of blocklegth i defied a P (q) Y X (y x) = Θ q()p Y X, (y x, ), where P Y X, (y x, ) i a defied i (8). The average probability of error icurred by a code i a mixed chael i defied i the uual ee a ɛ mix (q, Q X S, QŜ Y ; ) := EIS Ŝ]. From the liearity of EIS Ŝ] i q, it i eay to ee that problem P(; R) i equivalet to the fiite blocklegth chael codig of the compoud chael uder the average probability of error criterio (where the average i take over all meage), employig tochatic code. Moreover, for each trategy q P(Θ) of the jammer, the ier miimizatio i problem P(; R) correpod to the fiite blocklegth chael codig of the mixed chael uder the average probability of error criterio. It i the evidet that the boud ɛ metioed at the ed of the previou ectio ca be etablihed uig a combiatio of a fiite blocklegth achievability for a compoud chael ad a fiite blocklegth covere for a mixed chael. Our mai cotributio i i howig that thee aymptotically the ame. A rate R i aid to be achievable for a compoud chael (or mixed chael) if there exit a equece of tochatic code (Q X S, QŜ Y ) with M = R uch that ɛ com (Q X S, QŜ Y ; ) (or ɛ mix (q, Q X S, QŜ Y ; )) goe to 0 a. The upremum over all uch achievable rate the give the capacity of a compoud chael (or mixed chael) uder tochatic code. Iteretigly, the capacity (uder both determiitic ad tochatic code) of a compoud chael i (ee 0]) C = mi I P X (X; Y ), P X Θ ad the capacity of the mixed chael i (ee 4]) P X mi I P X (X; Y ), :q()>0 where the imizatio i over P X P(A) ad I PX (X; Y ) := a A,b B P X (a)p Y X, (b a, ) log P Y X, (b a, ) a A P X(a)P Y X, (b a, ), repreet the mutual iformatio betwee radom variable X A ad Y B. Notice that the mixed chael capacity deped oly o the upport of q, ad whe the upport i Θ, it equal C, the capacity of the compoud chael. We aume throughout thi paper that C > 0. Uder the average probability of error criterio, a trog covere doe ot hold for mixed ad compoud chael (ee ], 9]). However, if the average probability of error criterio i replaced with imum probability of error (over all meage), i.e., S, Θ EIS Ŝ S =, ], a trog covere hold for the compoud chael (4]) but ot the mixed chael (for the latter the criterio i S EIS Ŝ S = ]). We will ee that thi ubtle differece play a importat role i our reult. III. FINITE BLOCKLENGTH LOWER BOUNDS I thi ectio, we obtai a lower boud o ϑ(; R) via a liear programmig relaxatio of the miimizatio part of problem P(; R). For each q P(Θ), EIS Ŝ] i ocovex i (Q X S, QŜ Y ), the ocovexity reultig from the preece of biliear product. Coequetly, we relax thi ocovexity by a liear programmig (LP) relaxatio which reult i a ew mi- problem over relaxed code. Toward thi, we reort to a lift-ad-project-like idea a illutrated i ]. For ake of completee, we quickly outlie the techique here. I thi approach, we itroduce ew variable, W (, x, y, ŝ) to replace the biliear product term Q X S (x )QŜ Y (ŝ y) for each S, x X, y Y, ŝ Ŝ, thereby liftig the ocovex problem to a higher dimeioal pace of variable, that ow iclude (Q X S, QŜ Y, W ). New valid iequalitie i term of W are obtaied i thi pace. Toward thi, for each S, we multiply both ide of the equatio x Q X S(x ) = with QŜ Y (ŝ y) for all ŝ, y ad

6 6 for each y Y, multiply both ide of ŝ Q Ŝ Y (ŝ y) = with Q X S (x ) for all x,. Replace the biliear product i the reultig et of equatio with the ew variable W (, x, y, ŝ) ad add thee to the origial cotrait. Thu, a lower boud o ϑ(; R) i give by the optimal value of the followig problem, LP(; R) mi Q X S,QŜ Y,W q.t where err(q, W ) = z, err(q, W ) Q X S P(X S), QŜ Y P(Ŝ Y), q P(Θ) x W (z) Q Ŝ Y (ŝ y) = 0, ŝ, y, ŝ W (z) Q X S(x ) = 0, x, y, M W (z)i ŝq()p Y X,(y x, ). A collectio (Q X S, QŜ Y, W ) atifyig the cotrait above i called a relaxed code. Clearly ϑ(; R) OPT(LP(; R)). Note that err(q, W ) i covex i (Q X S, QŜ Y, W ) ad liear i q. Moreover, the et of relaxed code i covex ad compact, ad o i the et of P(Θ). Coequetly, a mi theorem hold, ad the order of the miimizatio ad imizatio ca be iterchaged i LP(; R). Thu, OPT(LP(; R)) = q mi (Q X S,QŜ Y,W ) err(q, W ), (0) where the miimizatio i over relaxed code ad the imizatio i over q P(Θ). Now, for each q P(Θ), the ier miimizatio over relaxed code i (0) i a lower boud o the ier miimizatio mi QX S,QŜ Y EIS Ŝ] i P(; R); it i preciely what oe would obtai if oe applied the above relaxatio directly to thi miimizatio. Coequetly, OPT(LP(; R)) i a lower boud alo o ϑ(; R), whereby, ϑ(; R) ϑ(; R) OPT(LP(; R)). () Thi ubtatiate our motivatio if LP(; R) i a tight relaxatio of P(; R), the the upper ad lower value of the game ought to approach each other. To obtai a lower boud from LP(; R), we replace the ier miimizatio over relaxed code i (0) by it dual. The dual of the ier miimizatio i (0) ca be writte a, DP(q, ; R).t where γ (q) a γ (q) a,γ (q) b,, γ a (q) () + y γ (q) b (y) () y λ(q) c (, x, y) 0 x, (D) γ (q) b (y) λ(q) (, ŝ, y) 0 ŝ, y (D) (, ŝ, y) + c (, x, y) Λ (q) (z) z (D3) Λ (q) (z) = M I ŝ P Y X, (y x, )q(), ad : S Ŝ Y R, λ(q) c : S X Y R, γ a (q) : S R ad γ (q) b : Y R are Lagrage multiplier (ee ] for detail). From duality of liear programmig, it the follow that OPT(LP(; R)) = q P(Θ) OPT(DP(q, ; R)). Toward evaluatig OPT(DP(q, ; R)), ote that it i optimal to take γ a (q) () = mi x c (, x, y) ad γ (q) b (y) = miŝ OPT(DP(q, ; R)) = mi x c.t, λ(q) y y λ(q) (, ŝ, y). Coequetly, we get c (, x, y) + y mi ŝ (, ŝ, y) + c (, x, y) Λ (q) (z) z. (, ŝ, y) The followig lemma the outlie our framework for derivig lower boud o ϑ(; R). Lemma 3.: Ay choice of fuctio : S Ŝ Y R, c : X S Y R atifyig cotrait (D3) yield the followig lower boud o ϑ(; R), ϑ(; R) OPT(DP(q, ; R)) q mi c (, x, y) + q x y y mi ŝ (, ŝ, y). I fact, a particular choice of fuctio, c atifyig cotrait (D3) of DP(q, ; R) for each q P(Θ) give the followig lower boud o ϑ(; R). Theorem 3.: The followig lower boud o ϑ(; R) hold, ϑ(; R) q q up P Ȳ,γ>0 OPT(DP(q, ; R)) mi q() mi P Y X, (y x, ), x y P Ȳ (y )M exp( γ) exp( γ), () where P Ȳ i a arbitrary probability ditributio i the pace of Y for Θ. Proof : For each q P(Θ), coider the followig choice of fuctio, c (, x, y) = c, (, ŝ, y) = M q() M mi P Y X, (y x, ), M PȲ (y ) exp(γ) q()p Ȳ (y )M exp( γ)i = ŝ. The feaibility of thee fuctio with repect to (D3) of DP(q, ; R) ca be verified a i the proof of Theorem. 7] ad we kip the proof here. Takig upremum over P Ȳ (y ) ad γ > 0 of the reultig dual cot ad applyig Lemma 3. give the required lower boud. Lower boudig mi P Y X, (y x, ), P Ȳ Θ (y )M exp( γ) i () with P Y X, (y x, )I i X; Ȳ (x; Y ) log M γ, where i X; Ȳ (x; y ) = log P Y X,(y x, ), P Ȳ (y ) the yield the followig boud, up mi q() P i X; q P,γ>0 Ȳ x (x; Y ) log M γ] Ȳ exp( γ). (3),

7 7 (3) i tur reult i the followig lower boud o ϑ(; R). Corollary 3.3: The followig lower boud o ϑ(; R) hold, ϑ(; R) q q()i q P(Θ) OPT(DP(q, ; R)) I P (q)(x; Y ) R ξ log T A(ξ) exp( ξ), (4) where ξ > 0 i a arbitrary cotat, P Ȳ (y ) T (P P Y X, ) (y), (5) P T ad P (q) i the type of x A that miimize q() P Y X, (y x, )I log P Y X,(y x, ) P y Ȳ (y ) Proof : The proof i icluded i Appedix A. R ξ. (6) Havig obtaied lower boud o the lower value of the game, we ow move o to aalyig the aymptotic equality of ϑ(; R) ad ϑ(; R) i the limit a. Toward thi, we divide the rate regio ito two differet cae (a) R > C or R < C, (b) C < R < C ad we obtai ew upper boud o the ϑ(; R) i thee regio. Together with the LP-baed lower boud, we the how i Sectio IV ad V that the mi- ad -mi value of the game ideed approach each other i the limit a i thee regio. IV. ASYMPTOTIC EQUALITY OF ϑ(; R) AND ϑ(; R) WHEN R < C AND R > C I thi ectio, we aalyze the aymptotic equality of ϑ(; R) ad ϑ(; R) i the limit a whe R < C ad R > C. Sice a lower boud o ϑ(; R) follow from the LP relaxatio-baed boud i (4), we firt derive a ew fiite blocklegth upper boud (or achievability boud) o the mi- value ϑ(; R). Toward thi, ote that the objective value of P(; R) correpodig to ay choice of ditributio Q X S, QŜ Y yield a upper boud o ϑ(; R). I particular, we cotruct a determiitic code (f, g) o a to obtai the followig upper boud o ϑ(; R). Theorem 4.: Let P X P(X ) be ay iput ditributio. The, for ay tochatic code, the followig upper boud hold, ϑ(; R) EIS Ŝ S =, ] Θ, S P log P ] Y X,(Y X, ) α + δ Θ P Y (Y ) + exp( δ) + M Θ exp(α), (7) where EIS Ŝ S =, ] i the probability of error i tramittig meage uder thi code whe the chael tate i, P i the probability with repect to P X P Y X,, α, δ > 0 are arbitrary cotat ad P Y (y ) = x X P X(x)P Y X, (y x, ). Proof : Proof i icluded i Appedix B. I particular, fixig α = log M +, δ = with > 0 a arbitrary poitive cotat, (7) reult i the followig upper boud, ϑ(; R) P log P ] Y X,(Y X, ) log M + Θ P Y (Y ) + exp( /) Θ + ]. (8) Recall from Sectio II that evaluatig ϑ(; R) i equivalet to the problem of miimizig the average probability of error over all tochatic code i a compoud chael. Earliet upper boud kow for uch compoud chael ha bee obtaied by Blackwell et al. 8]. Our ew boud i (7) improve o the boud of Blackwell ] et al. by replacig P log P Y X,(Y X,) P Y (Y ) ] α + δ. P log P Y X,(Y X,) P Y (Y ) α + δ therei with Employig thi ewly derived upper boud o ϑ(; R) together with the lower boud o ϑ(; R) obtaied i (4), the followig theorem the how that the differece betwee the upper ad lower value of the game vaihe aymptotically a for R < C ad R > C. Theorem 4.: Coider the zero-um game with M = R ad chael coditioal ditributio a give i (8). The, lim ϑ(; R) = lim ϑ(; R) = 0 R < C, ϑ(; R) = R > C. lim ϑ(; R) = lim Proof : Proof i icluded i Appedix C. Recall from the itroductio that thi reult i alog expected lie. After all, C beig the capacity of the compoud chael, we mut have that for R < C, ϑ(; R) ad hece ϑ(; R) approach 0 for large. Similarly C beig the mallet amog the capacitie of the idividual DMC correpodig to tate Θ, by the trog covere of chael codig for R > C, ϑ(; R) ad hece ϑ(; R) mut approach uity. A additioal ubtlety here i that we have fiite blocklegth etimate o the approximate mii value, which follow from Theorem 4. ad Corollary 3.3. Coequetly, eve though for each fiite the game may ot admit a addle poit value, for large ad R < C ad R > C, it admit earaddle poit value (a i evidet from (46) i the Appedix). Moreover, for the cla of chael where C = C = C, Theorem 4. how that a ear-addle poit value exit for all R except whe R = C. Example where C = C hold are familie of dicrete memoryle chael where the capacity achievig iput ditributio i idepedet of 4], that iclude the biary ymmetric chael ad biary eraure chael amogt other. V. ASYMPTOTIC EQUALITY OF ϑ(; R) AND ϑ(; R) FOR C < R < C We ow come to the aymptotic equality of the mi- ad -mi value of the game whe the rate of commuicatio R lie i the rage C < R < C ad exact characterizatio of thi limitig value. Toward thi, we firt etablih a

8 8 fudametal lower boud o the -mi value of the game by employig the LP relaxatio from Sectio III. From the lower boud we the ipire a fiite blocklegth achievability cheme whoe performace aymptotically matche thi lower boud a for all but fiitely may value of the rate R. A. A LP-Relaxatio Baed Fudametal Lower Boud o Max-mi problem We employ the LP relaxatio to derive a lower boud o lim if ϑ(; R). Recall that for each q P(Θ), the LP relaxatio yield that ϑ(; R) OPT(DP(q)) whereby the followig limitig boud hold: lim if ϑ(; R) lim if OPT(DP(q)). q P(Θ) Together with the covere obtaied i (4), the LP relaxatio thu give the followig limitig boud o the -mi value of the game. Theorem 5.: The LP-relaxatio yield the followig fudametal lower boud o the -mi value of the game, lim if ϑ(; R) lim if OPT(DP(q)) L(R), q P(Θ) (9) where L(R) = mi q() R C(Θ ), q P(Θ) Θ Θ Θ for R 0, C] ad, C(Θ ) = mi I P P X Θ X (X; Y ), (0) i the capacity of the compoud chael formed by Θ Θ. Proof : For ay q P(Θ), we have from (4), ϑ(; R) OPT(DP(q)) (a) Θ q() A(ξ) exp( ξ), where i (a), ξ > 0 i a arbitrary cotat, A(ξ) i idepedet of ad Θ := Θ R I P (q)(x; Y )+ ξ + log T. Sice R < I P (q)(x; Y )+ ξ + log T we mut have (Θ ) c, R < mi I (Θ P (q)(x; Y )+ ξ + log T. )c It i clear that Θ atifie that R < C((Θ ) c )+ξ + log T, whereby for ay q P(Θ), ϑ(; R) mi Θ Θ Θ A(ξ) q() R < C( Θ c ) + ξ + log T exp( ξ), () Subequetly, takig limit o both ide of (), oticig that there are oly fiitely may Θ Θ, ad the imizig over q P(Θ) yield that lim if ϑ(; R) mi q() R < C( Θ c ) + ξ. q P(Θ) Θ Θ Θ Our reult the follow by takig ξ 0. ) The quatitie L(R) ad U(R): L(R) defied i (0) play a key role i our aalyi. Thi ectio i devoted to udertadig it propertie. By makig a light chage i the defiitio of L, we defie the followig quatity, U(R) := mi q() R < C(Θ ), () q P(Θ) Θ Θ Θ for R 0, C]. For coveiece, we exted the defiitio of L ad U to R > C by defiig, L(R) = U(R) := R > C. Oberve that we alway have L(R) U(R). To ee thi, ote that for ay rate R, Υ(R) := Θ Θ R C(Θ ) Υ(R) := Θ Θ R < C(Θ ) (3) hold i geeral, whereby for ay q P(Θ), mi Θ Υ(R) Θ q()] mi Θ Υ(R) Θ q()]. I particular, for thoe rate R uch that R = C( Θ) for ome Θ Θ, it i eay to ee that Θ Υ(R) ad Θ Υ(R), whereby Υ(R) Υ(R), a trict icluio. I ome cae thi implie L(R) < U(R) a we ee i the example below. Employig Υ(R), U(R) ca be equivaletly writte a, U(R) = mi q(). (4) q P(Θ) Θ Υ(R) Θ Iteretigly, the followig lemma how that U(R) defied above i i fact equal to the followig expreio, Ũ(R) = if ɛ 0, ) R < C ɛ (q), (5) q P(Θ) where C ɛ (q) i the ɛ-capacity defiitio of a mixed chael P (q) Y X. Uder the average probability of error criterio, the ɛ- capacity of thi chael ha bee obtaied i, Thm ad Lem (a)] for ɛ 0, ) a C ɛ (q) = up R P X q()ii PX (X; Y ) R ɛ, (6) where I PX (X; Y ) i the mutual iformatio betwee X P X P(A) ad Y x A P Y X,(y x)p X (x). Lemma 5.: Ũ(R) i (5) evaluate to U(R) for R C, C), i.e. Ũ(R) = q P(Θ) if ɛ 0, ) R < C ɛ (q) = U(R).

9 9 Proof : From the defiitio of ɛ-capacity i (6), it follow that whe R < C ɛ (q) for a give q P(Θ), there exit P X P(A) uch that q()ii P X (X; Y ) R ɛ. Coequetly, Ũ(R) = if ɛ 0, ) P X P(A).t q P(Θ) q()ii PX (X; Y ) R ɛ. It i eay to ee that for a give q P(Θ), the ier ifimum of ɛ i the above expreio for Ũ(R) i equivalet to fidig the miimum of um Θ q() over trict ubet Θ Θ correpodig to which there exit a P X P(A) uch that R I PX (X; Y ) for all Θ ad R < To illutrate thi relatio betwee L(R) ad U(R), we coider the followig example of a three tate jammer. mi Θ c I PX (X; Y ). Sice a miimum i take over ubet Θ, the firt coditio i redudat. Coequetly, Ũ(R) ca Example: Coider a three tate jammer with Θ =,, 3. be equivaletly writte a the followig optimizatio problem, Let C(Θ ) for Θ Θ be uch that, mi q() q P(Θ) Θ Θ P X P(A).t R < mi I P Θ c X (X; Y ). C < C(, ) < C(, 3) < C(, 3) < C = C(). Θ Evaluatig U(R) for thi three-tate jammer the reult i, Now, R < mi Θ c I PX (X; Y ) for ome P X hold if ad oly if R < C(Θ c 0 if R < C ), which give the followig equivalet expreio, 3 if C R < C(, ) mi q P(Θ) Θ Θ q() R < C(Θ ). Θ It i the eay to ee that the above expreio i equivalet to U(R) for R C, which exclude the feaibility of Θ = Θ i (). The followig propoitio decribe the form of L ad U preciely ad give the coditio whe L(R) = U(R). Propoitio 5.3: ) L ad U are o-decreaig tep fuctio. U i rightcotiuou ad L i left-cotiuou. ) L ad U are dicotiuou at the ame poit. 3) L(R) = U(R) hold for all rate R where thee fuctio are cotiuou. I particular, L(R) = U(R) for all rate R uch that R C(Θ ) for ay Θ Θ. Proof : ) We firt argue that U i a o-decreaig tep fuctio. Note that for rate R, R uch that R R, Υ(R ) Υ(R ) (where Υ(R) i a defied i (3)), which i tur give that U(R ) U(R ). Thu, U(R) i odecreaig. Moreover, if for a give R, Θ, Θ Θ are uch that C(Θ ) = Θ Θ:R C(Θ ) C(Θ ) ad C(Θ ) = mi Θ Θ:R<C(Θ ) C(Θ ), the U(R) i cotat i the iterval C(Θ ), C(Θ )). Together, we thu have that U(R) i a o-decreaig tep fuctio which i cotiuou i the rage C(Θ ) < R < C(Θ ) ad i right-cotiuou everywhere. Similarly, it i eay to verify that L(R) i a o-decreaig fuctio. Further, if Θ 3, Θ 4 Θ are uch that C(Θ 3 ) = Θ Θ:R>C(Θ ) C(Θ ) ad C(Θ 4 ) = mi Θ Θ:R C(Θ ) C(Θ ), L(R) i cotat i the iterval (C(Θ 3 ), C(Θ 4 )]. Hece, L(R) i a o-decreaig tep fuctio which i cotiuou i the rage C(Θ 3 ) < R < C(Θ 4 ) ad i left-cotiuou everywhere. ad 3) We firt how that L(R) = U(R) for R where U i cotiuou. Coider the rate R where U i cotiuou. Two cae arie (a) R C(Θ ) for ay Θ Θ i which cae it i eay to verify that Υ(R) = Υ(R), whereby L(R) = U(R), ad (b) R = C(Θ ) for ome Θ Θ ad U(R) i cotiuou. I the ecod cae, the cotiuity of U(R) eure that U(R) = U(R δ) for ome mall δ > 0. However, Υ(R δ) = Υ(R) whereby U(R δ) = L(R). Thu U(R) = L(R) for rate R where U(R) i cotiuou. It follow that L ad U have the ame poit of dicotiuity, thereby provig the ecod claim. Moreover, U(R) ad L(R) i tur coicide for all rate R uch that R C(Θ ) for ay Θ Θ. U(R) = if C(, ) R < C(, 3) if C(, 3) R < C(, 3) 3 if C(, 3) R < C if R C To ee thi, let u evaluate U(R) whe C(, 3) R < C(, 3). From the defiitio i (), we get that, U(R) = mi q(), q() + q(3), q() + q(3), q() + q() = q P(Θ), which follow by takig q() = ad q() = q(3) = 4. We ow evaluate L(R) for the coidered three-tate jammer. Thi reult i 0 if R C 3 if C < R C(, ) L(R) = if C(, ) < R C(, 3) if C(, 3) < R C(, 3) 3 if C(, 3) < R C if R > C Note that U(R) L(R) at poit of dicotiuity of U(R) ad a trict iequality hold. To ee thi, coider the cae whe R = C(, 3). While U(R) = 3, L(R) yield. However, at rate R where U(R) i cotiuou, ay R = C(, 3), where > 0 i mall eough, U(R) = L(R) =. Thu, L(R) ad U(R) coicide for all rate R where U(R) i cotiuou. Figure 3 how the U(R) ad Figure 4 how L(R) for the three-tate jammer. B. Fiite Blocklegth Achievability Upper Boud o ϑ(; R) I thi ectio, we preet a fiite blocklegth achievability cheme for a compoud chael to obtai a upper boud o

10 0 output x X a x = f(v, ) where f i a fuctio that map V S to X. Note however that V i ot available to the decoder, whereby the radomizatio i local to the ecoder. Thu, the tochatic ecoder i our achievability cheme i take to be, Q X S (x ) = v V Ix = f(v, )P V (v). (7) Fig. 3: U(R) for three-tate jammer. A hollow circle idicate a excluded ed-poit rate ad a filled circle idicate that the ed-poit rate i icluded. Fig. 4: L(R) for three-tate jammer. A hollow circle idicate a excluded ed-poit rate ad a filled circle idicate that the ed-poit rate i icluded. the mi- value ϑ(; R) of the game whe rate R lie i the rage C < R < C. While Theorem 5. howed a lower boud of L(R) i thi rage, our cheme will achieve a upper boud of U(R), implyig, thak to Propoitio 5.3, that the upper ad lower boud are equal, ad equal to L(R), everywhere except at fiitely may poit. The propoed cheme ha two key feature which ditiguihe itelf from the achievability cheme i Theorem 4. employed for the cae whe R < C ad R > C. Firtly, i cotrat to the determiitic ecoder employed i Theorem 4., the ecoder i our ew cheme avail a local radomizatio trategy, with it output depedig o the outcome of a radom experimet. Preciely tatig, for edig each meage S, the ecoder perform a radom experimet idepedet of, the outcome of which i repreeted by the radom variable V takig value i a kow fiite pace V ad ditributed accordig to a kow ditributio P V P(V). Correpodig to the iput meage S ad depedig o the outcome V of the radom experimet, the ecoder the To develop a achievability cheme we employ a plitachievability techique the ecod key feature of the cheme. I the cheme we coider, the pace V i take a Υ(R), i.e., the collectio of ubet Θ Θ that each defie a compoud chael P Y X, Θ with C(Θ ) > R, ad the radom experimet chooe a compoud chael from the above collectio. The value of V pecifie the codebook that i ued for ecodig the meage; whe V = v, the codebook for compoud chael P Y X, v i ued to ecode meage. f i deiged uch that it ecode both, the value of V ad the meage. V i ecoded ito a trig of legth ; thi trig i prefixed before each codeword from the codebook for the compoud chael formed by V to get the actual chael iput trig. From the received output, the decoder attempt to decode V, ad the decode the meage baed o the codebook aociated with V. We get a error if (a) the chael tate Θ doe ot lie i the choice of the compoud chael choe by the radom experimet, i.e., V, or (b) the chael tate V, but the decoder fail to correctly decode either the meage et or the choice of the compoud chael V. To implemet the plit-achievability techique outlied above, we plit the chael iput pace a X = A A ad the chael output pace a Y = B B where alo agreed upo by the decoder. We ue the followig otatio. Let X = A ad X = A. A geeric chael iput X X i writte a X = ( X, X) where X X, X X ad imilarly x = ( x, x) where x X, x X. Similarly, let Ŷ = B repreet the chael output pace correpodig to X ad Ỹ = repreet the chael output pace B correpodig to X. Let Y = (Ŷ, Ỹ ) with Ŷ Ŷ, Ỹ Ỹ ad y = (ŷ, ỹ) with ŷ Ŷ, ỹ Y. With thi otatio we have that for each Θ, P Y X, (y x, ) = P Y X, (y i x i, ) where i= = PŶ X, (ŷ x, )PỸ X, (ỹ x, ), (8) PŶ X, (ŷ x, ) = P Y X, (ŷ i x i, ) (9) ad PỸ X, (ỹ x, ) = = i= i= + i= P Y X, (y i x i, ) P Y X, (ỹ i x i, ) (30) The followig theorem give our fiite blocklegth achievability reult.

11 Theorem 5.4: For ay rate R lyig i C < R < C, let V = Υ(R) ad V V be a radom variable ditributed accordig to P V arg mi P V P(V) Θ v V P V (v)i v. (3) The, for ay α, δ, α, δ > 0, ad ay <, the followig upper boud hold, where λ = ϑ(; R) U(R) + λ, (3) v V err(v) S, err V, with (33) err V = P log P Ŷ X, (Ŷ X, ) Θ PŶ (Ŷ ) α + δ err (v) S + V Θ exp( α) + exp( δ), (34) = P V =v log P Ỹ X, (Ỹv X ] v, ) v PỸv (Ỹv ) α + δ + v M exp( α) + exp( δ) for each v V. (35) Here for each v V, Xv P Xv where P Xv i ay ditributio i P( X ), Ỹ v deote the output of Xv uder the chael PỸ X, defied i (30), ad PỸv (ỹ ) x X P ( x)pỹ Xv X, (ỹ x, ). Likewie, X P X P( X ) ad Ŷ i the output of X uder the chael PŶ X, defied i (9) ad PŶ (ŷ ) x X P X( x)pŷ X, (ŷ x, ). Fially, P V =v deote the probability with repect to PỸ X, P Xv ad P i with repect to PŶ X, P X. Proof : To obtai the required boud, we coider the radomized ecoder Q X S a i (7) with f : V S X X ad P V a defied i (3), ad a determiitic decoder, g : Ŷ Ỹ,..., M. We adopt the followig plitachievability trategy. For codig the choice of V, we coider a determiitic code of ize V i the pace of X ad Ŷ, i.e. a ecoder f : V X ad a decoder ĝ : Ŷ V, uch that the imum probability of erroeou tramiio of ay v V over the compoud chael P Y X, Θ i λ. Sice λ err V, Theorem 4. applied with M replaced by V guaratee the exitece of uch a code. Subequetly, for each choice of v V, we coider aother determiitic code of ize M i the pace of X ad Ỹ, i.e., a ecoder f v : S X, ad a decoder g v : Ỹ S, uch that the imum average probability of erroeou tramiio over the compoud chael P Y X, v i λ. Sice λ err (v) S, Theorem 4. applied with Θ replaced by v, the guaratee the exitece of uch a code. The ecoder f ad decoder g are the aembled a follow, f(v, ) = ( f(v), f v ()) v V, S, g(ŷ, ỹ) = gĝ(ŷ) (ỹ) ŷ Ŷ, ỹ Ỹ. I other word, the ecoder f ecode the value of V uig f ad the value of S uig f v whe V = v. The decoder g map Ỹ to a meage i S uig a fuctio g v where v i ] obtaied a ĝ(ŷ ). Thu the decoder firt decode v from Ŷ ad the ue the reultig value of v to chooe g v, which i the ued to decode the meage from Ỹ. We ow how that thi cheme achieve the boud i (3). Recall that V = Υ(R), whereby V comprie of compoud chael formed from thoe ubet v of Θ uch that R < C(v). Thu whe V = v, error i tramiio of S occur whe either the chael tate / v or whe v but either v i ot correctly decoded or, v i correctly decoded, but S i ot correctly decoded. Cocretely, let P (S Ŝ V = v) repreet the average probability of error i tramittig S give that V = v uder the code (f, g) cotructed above whe the chael tate i. Clearly, P (S Ŝ V = v) equal the um P (S Ŝ, / v V = v) + P (S Ŝ, v V = v). (36) We upper boud the firt term i (36) by I / v. For the ecod term, ote that P (S Ŝ, v V = v) = P (S Ŝ, v, ĝ(ŷ ) = v V = v) + P (S Ŝ, v, ĝ(ŷ ) v V = v) which are probabilitie correpodig to the evet that the decoder correctly decode V but make a error i decodig S, ad the evet that decoder icorrectly decode V ad S. Coequetly, P V (v)p (S Ŝ, v V = v) v v λ, ] P V (v) P ( g v (Ỹ ) S V = v) + P (ĝ(ŷ ) V V = v) where the lat iequality follow from the cotructio of our code. Coequetly, ϑ(; R) P V (v) P (S Ŝ, v V = v)+ Θ v V ] P (S Ŝ, v V = v) λ + Θ v V P V (v)i v. By the choice of P V i (3), the ecod term i the above boud ca be equivaletly writte a, P V (v)i v (37) Θ v V = mi P V P(V) Θ (a) = mi P V P(V) q P(Θ) (b) = mi q P(Θ) P V P(V) (c) = q P(Θ) v Υ(R) = mi mi q P(Θ) v Υ(R) v V P V (v)i v v V Θ v V Θ q()i v Θ v q()p V (v)i v q()p V (v)i v q() = U(R),

12 where (a) hold due to the liearity of the expreio i the curly brace i q P(Θ), (b) follow from vo Neuma mii theorem ad (c) follow agai from the liearity of the expreio i curly brace i P V. Comparig the reultig expreio with (4), the yield the required boud. Oberve that the boud i Theorem 5.4 i valid for ay <. We ow how that U(R) i achievable aymptotically for rate lyig i the rage C < R < C by lettig grow to ifiity uch that = o(). Theorem 5.5: Let C > 0. For C < R < C, the upper boud o the mi- value of the game i (3) yield the followig limitig value a, i.e., lim up ϑ(; R) U(R). (38) Proof : It uffice to argue that for V = Υ(R), we ca chooe P X, P Xv, α, α, δ, δ ad uch that for each v V, err (v) S 0 ad err V 0 a. Toward thi, a icreae, we let grow to ifiity at a rate of o(). err V i a upper boud guarateed by Theorem 4. for edig V meage over the compoud chael P Y X, Θ uig a code of blocklegth. Sice a, ad V Θ, a cotat, the rate of thi code i aymptotically zero. Sice C > 0, arguig a i the proof of Theorem 4., we ca chooe P X, α, δ uch that err V 0. For each v V, err (v) S i a upper boud guarateed by Theorem 4. for edig M meage over the compoud chael P Y X, v uig a code of blocklegth. Sice = o(), the rate of uch a code i aymptotically R. Now ice V = Υ(R), we have from (3) that R < C(v) for each v V. Coequetly, arguig agai a i the proof of Theorem 4., we ca chooe X v, α, δ uch that err (v) S 0 for each v V. That give the upper boud we were lookig for. Remark V.. We fid it rather pleaig to ote how, via vo Neuma mii theorem, the achievable error term for the compoud chael i (37), i.e., Θ v V P V (v)i v become exactly the required quatity U(R), which we arrived at from the covere for the mixed chael a differet chael altogether. A outlied i the itroductio, our ituitio for thi are grouded i the ear-covexity of codig problem we dicovered i ]. There may be other operatioal iterpretatio of thi pheomeo that are probably worthy of further ivetigatio. The above cheme, though atural i hidight, occurred to u oly after firt derivig the covere expreio ad reiterpretig it i thee dual term. It would be illumiatig to fid a ab iitio operatioal jutificatio for the optimality of thi cheme. C. Aymptotic Tighte of the Mi- ad Max-mi Value We ow tie our tory together by coolidatig the coequece of Theorem 5.5, Theorem 5. ad Theorem 4.. The followig reult the hold. Theorem 5.6: Let C > 0. For rate R 0 uch that U(R) i cotiuou at R, the mi- ad -mi value of the game approach ϑ(r) := U(R) = L(R) a, i.e., ϑ(r) := lim ϑ(; R) = lim ϑ(; R) = L(R) = U(R). I particular, for rate R uch that R C(Θ ) for ay Θ Θ, the above equatio hold. Proof : Notice that for R < C, L(R) = U(R) = 0. Moreover for R > C, we defied L(R) = U(R) =. Thu, Theorem 4. cofirm the above claim for R < C ad R > C. By combiig (9) with (38), we get that for rate R uch that C < R < C, the followig boud hold i the limit a, i.e., U(R) lim up ϑ(; R) lim if ϑ(; R) L(R). (39) I particular, for rate R uch that U(R) i cotiuou at R, L(R) = U(R) from Propoitio 5.3, whereby the claim hold. Thu, except for thoe fiitely may rate R where L or U are dicotiuou, the mi- ad -mi value of the game coicide i the limit a, ad they coicide to ϑ(r) = U(R) = L(R), a value oe ca explicitly compute. Theorem 5.6 alo give a cloed-form expreio for the ɛ-capacity of a compoud chael P Y X, Θ uder tochatic code a how i the followig corollary. Corollary 5.7: For ay fixed ɛ 0, ), the ɛ-capacity of the compoud chael P Y X, Θ uder tochatic code ad average error probability criterio, deoted C ɛ, i give a, C ɛ := up R lim ϑ(; R) ɛ = up R L(R) ɛ. (40) Proof : Deote κ(ɛ) := up R L(R) ɛ. Sice L ad U are tep fuctio that are equal everywhere except at poit of jump-dicotiuity, it follow that κ(ɛ) = up R U(R) ɛ. We firt how that if R i ɛ-achievable for the compoud chael, i.e. lim ϑ(; R) ɛ, the R κ(ɛ). Toward thi, ote from (39) that R i ɛ-achievable implie that L(R) ɛ, which i tur give that R κ(ɛ). Coverely, if R < κ(ɛ), the U(R) ɛ which how that lim ϑ(; R) ɛ. To the bet of our kowledge, our i the firt characterizatio of the ɛ-capacity of a compoud chael uder tochatic code. We coclude with ome fial remark. For thoe rate R at which U(R) i dicotiuou, Theorem 5.6 give that the limitig value of the differece betwee mi- ad mi value of the game amout to at mot U(R) L(R). The quetio the arie if we could modify our achievability cheme o a to yield a limitig value of L(R) at the poit of dicotiuity. Let R = C(Θ ), Θ Θ be a poit of dicotiuity of U(R). Toward modifyig the achievability cheme, oe ca itead take V = Υ(R). Our modified cheme the give a limitig value of L(R) at the poit of dicotiuity R = C(Θ ) if zero probability of error i achievable at

13 3 rate R = C(Θ ) for the compoud chael P Y X, Θ. Whether thi i poible deped o the pecific chael law, ot o the capacity aloe. Thi i topic of eparate reearch, which i beyod our preet cope. VI. CONCLUSION We coidered a game betwee a team compriig of a fiite blocklegth ecoder ad decoder ad a fiite tate jammer where the former team attempt to miimize the probability of error ad the jammer attempt to imize it. The oclaicality of the iformatio tructure reder the team deciio problem ocovex whereby there may ot exit a addle poit value to thi game. Depite thi, we howed that for all but fiitely may rate, a aymptotic addle-poit value exit for thi game ad derived a exact characterizatio of thi value. Our reult demotrate a deeper relatio betwee compoud ad mixed chael ad provide a ew characterizatio of the ɛ-capacity of a compoud chael uder tochatic code. ACKNOWLEDGMENT The author would like to thak Himahu Tyagi from the Idia Ititute of Sciece for helpful dicuio o thi topic. APPENDIX A PROOF OF COROLLARY 3.3 Crucial to provig Corollary 3.3 i the followig lemma from ], which i ueful i aalyzig the aymptotic tighte of the boud i (3). Lemma A.: For ay fixed x A, let P deote it type. Let δ > 0 be a arbitrary cotat ad defie C () (δ) := y B : x log P Y X, (y x, ) (P P Y X, ) (y ) I P (X; Y ), δ for all Θ, where (P P Y X, ) (y) = i= a A P (a)p Y X=a,(y i a, ) ad I P (X; Y Θ) i the mutual iformatio evaluated whe X ha the ditributio P. The, PY C () A(δ) x (δ)], where P i with repect to P Y X=x,, A(δ) i a cotat idepedet of, P ad. We ow prove Corollary 3.3. Proof of Corollary 3.3: To obtai the boud i (4), lower boud (3) by fixig ome q P(Θ), γ = ξ, ξ > 0 ad P Ȳ a i (5). Note that the choice of γ ad P Ȳ are idepedet of q. Let the miimum i the reultig expreio be attaied by x 0 X with type P (q). Subequetly, apply Lemma A. a illutrated i 5, Proof of Theorem.] to get the followig lower boud o (3), q()i I P (q)(x; Y ) R ξ log T A(ξ) exp( ξ). (4) Subequetly, take upremum over q P(Θ) to get the required lower boud. APPENDIX B PROOF OF THEOREM 4. Proof of Theorem 4.: To obtai a upper boud o ϑ(; R), we deig a determiitic code (f, g) for the compoud chael P Y X, Θ. The code will be cotructed o that f(i) = u i, u i X, i S =,..., M ad g : Y S partitio Y ito M dijoit decodig et D,..., D M to yield a (, M, λ) code uch that P Y X, (Y D i X = u i, ) λ, Θ, i M], ad λ = P log P ] Y X,(Y X, ) α + M Θ P Y (Y ) exp(α), with P Y (y) = Θ P Y (y ) = x P X(x)P Y X, (y x, ). We will the upper boud λ to get the required reult. Notice that thi trategy eure a boud o the imum probability over all meage, a required by the theorem. Toward thi, correpodig to each x X, we aociate B (x) = y Y : log P Y X,(y x, ) α. P Y (y) The, for ay x X ad Θ, the followig relatio hold: P Y B (x) X = x] P Y (y) exp(α)i P Y X, (y x, ) P Y (y) exp(α) y = exp(α)p Y (B (x)). (4) We ow cotruct our codebook ad decodig et alog the lie of a imal code cotructio of Feitei (ee 3]). ) Chooe if poible u X uch that mi P B (u ) X = u ] λ. Aig D, = B (u ) a the decodig et of u with repect to the chael parameter. Subequetly, take D = D, to be the decodig et correpodig to u. ) Chooe u k X \u,..., u k uch that k mi P B (u k )/ D j X = u k ] λ. j= Take D k = D,k = B (u k )/ k j= D j] to be the decodig regio correpodig to u k, where D,k repreet the decodig regio for u k with repect to the chael parameter. It i eay to verify that the et D i are mutually dijoit. Suppoe thi proce top after K tep, which reult i a codebook u,..., u K with (D,..., D K ) a the correpodig decodig regio. We ow how that K M. The toppig coditio implie that for all x, there exit = 0 uch that, K P 0 B 0 (x)/ D j X = x] < λ, j=