Strong Converse, Feedback Channel Capacity. and Hypothesis Testing. Journal of the Chinese Institute of Engineers, to appear November 1995.

Transcription

1 Strong Convrs, Fdback Channl Capacity and Hypothsis Tsting Po-Ning Chn Computr & Communication Rsarch Laboratoris Industrial Tchnology Rsarch Institut Taiwan 30, Rpublic of China Fady Alajaji Dpartmnt of Mathmatics & Statistics Qun's Univrsity Kingston, ON K7L 3N6, Canada Journal of th Chins Institut of Enginrs, to appar Novmbr 995 Abstract In light of rcnt rsults by Vrdu and Han on channl capacity, w xamin thr problms: th strong convrs condition to th channl coding thorm, th capacity of arbitrary channls with fdback and th Nyman-Parson hypothsis tsting typ-ii rror xponnt. It is rst rmarkd that th strong convrs condition holds if and only if th squnc of normalizd channl information dnsitis convrgs in probability to a constant. Exampls illustrating this condition ar also providd. A gnral formula for th capacity of arbitrary channls with output fdback is thn obtaind. Finally, a gnral xprssion for th Nyman-Parson typ-ii rror xponnt basd on arbitrary obsrvations subjct to a constant bound on th typ-i rror probability is drivd. Ky Words: Strong convrs, channl capacity, channls with fdback, hypothsis tsting. Parts of this papr wr prsntd at th 995 Confrnc on Information Scincs and Systms, Th John Hopkins Univrsity, Baltimor, MD, USA, March 995.

2 Introduction In this papr, w invstigat thr problms inspird by th rcnt work of Vrdu and Han on th gnral capacity formula of arbitrary singl-usr channls [6]. W rst addrss th strong convrs condition obtaind in [6] and provid xampls of channls for which th strong convrs holds. W nxt driv a gnral capacity formula for arbitrary singl-usr channls with output fdback. Finally, w analyz th Nyman-Parson hypothsis tsting problm basd on arbitrary obsrvations. In [6], Vrdu and Han giv a ncssary and sucint condition for th validity of th strong convrs to th channl coding thorm. Thy stat that th strong convrs holds if and only if th channl capacity is qual to th channl rsolvability. W rmark that if thr xists an input distribution P X n achiving th channl capacity, thn th strong convrs condition is actually quivalnt to th convrgnc in probability to a constant (or in distribution to a dgnrat random variabl) of th squnc of normalizd information dnsitis according to a joint inputoutput distribution with P X n as its inducd marginal. W furthrmor not that th xprssion of th strong capacity, which will b dnd latr, is givn by th channl rsolvability. W also obtain xampls of discrt channls satisfying th strong convrs condition. Th main tool usd in [6] to driv a gnral xprssion for th (nonfdback) channl capacity is a nw approach to th (wak) convrs of th coding thorm basd on a simpl lowr bound on rror probability. W utiliz this rsult to gnraliz th capacity xprssion for channls with fdback. Fdback capacity is shown to qual th suprmum, ovr all fdback ncoding stratgis, of th input-output inf-information rat which is dnd as th liminf in probability of th normalizd information dnsity. W nally considr th Nyman-Parson hypothsis tsting problm basd on arbitrary obsrvations. W driv a gnral xprssion for th typ-ii rror xponnt subjct to a xd bound on th typ-i rror probability. W obsrv that this xprssion is indd th dual of th gnral "-capacity formula givn in [6]. 2

3 On th strong convrs of th singl-usr channl. Strong convrs condition Considr an arbitrary singl-usr channl with input alphabt A and output alphabt B and n-dimnsional transition distribution givn by W (n) = P Y n jx n : An! B n ; n = ; 2; : : :. Dnition ([6]) An (n; M; ) cod has blocklngth n, M codwords, and (avrag) rror probability not largr than. R 0 is an -achivabl rat if for vry > 0 thr xists, for all sucintly larg n, (M; n; ) cods with rat log 2 M n > R : Th maximum -achivabl rat is calld th -capacity, C. Th channl capacity, C, is dnd as th maximal rat that is -achivabl for all 0 < <. It follows immdiatly from th dnition that C = lim!0 C. Dnition 2 ([6]) A channl with capacity C is said to satisfy th strong convrs if for vry > 0 and vry squnc of (n; M; n ) cods with rat log 2 M n > C + ; it holds that n! as n!. In [6], Vrdu and Han driv a gnral formula for th oprational capacity of arbitrary singlusr channls (not ncssarily stationary, rgodic, information stabl, tc.). Th (nonfdback) capacity was shown to qual th suprmum, ovr all input procsss, of th input-output infinformation rat dnd as th liminf in probability of th normalizd information dnsity: C = sup I (X n ; Y n ); () X n whr X n = (X ; : : : ; X n ), for n = ; 2; : : :, is th block input vctor and Y n = (Y ; : : : ; Y n ) is th corrsponding block output vctor inducd by X n via th channl. 3

4 Th symbol I (X n ; Y n ) apparing in () is th inf-information rat btwn X n and Y n and is dnd as th liminf in probability of th squnc of normalizd information dnsitis i n X n Y n(x n ; Y n ), whr P i X n Y n(an ; b n Y n jx ) = log n(bn ja n ) 2 : (2) P Y n(b n ) Likwis, th sup-information rat dnotd as I(X n ; Y n ) is dnd as th limsup in probability of th squnc of normalizd information dnsitis. Th liminf in probability of a squnc [6] of random variabls is dnd as follows: If A n is a squnc of random variabls, its liminf in probability is th largst xtndd ral numbr such that for all > 0, lim sup n! P r[a n ] = 0. Similarly, its limsup in probability is th smallst xtndd ral numbrs such that for all > 0, lim sup n! P r[a n + ] = 0. Not that ths two quantitis ar always dnd; if thy ar qual, thn th squnc of random variabls convrgs in probability to a constant (which is ). In Thorm 6 in [6], Vrdu and Han stablish gnral xprssions for -capacity. Thy also giv a ncssary and sucint condition for th validity of th strong convrs (Thorm 7 in [6]), which stats that th strong convrs condition is quivalnt to th condition sup I(X n ; Y n ) = sup I(X n ; Y n ); (3) X n X n i.. C = S, whr S 4 = sup X n I(X n ; Y n ) dnots th channl rsolvability, which is dnd as th minimum numbr of random bits rquird pr channl us in ordr to gnrat an input that achivs arbitrarily accurat approximation of th output statistics for any givn input procss [4]. Furthrmor, if channl input alphabt is nit, thn C = S = n! lim sup X n n I(X n ; Y n ): Lmma If (3) holds and thr xists ~ X n such that thn sup X n I(X n ; Y n ) = I( ~ X n ; Y n ); I( ~ X n ; Y n ) = I( ~ X n ; Y n ): 4

5 P roof: W know that I( X ~ n ; Y n ) = sup I(X n ; Y n ) = sup I(X n ; Y n ) I( X ~ n ; Y n ): X n X n But I(X n ; Y n ) I( ~ X n ; Y n ), for all ~ X n. Hnc I( ~ X n ; Y n ) = I( ~ X n ; Y n ): Rmark: Th abov lmma stats that if (3) holds and thr xists an input distribution that achivs th channl capacity, thn it also achivs th channl rsolvability. Howvr, th convrs is not tru in gnral; i.., if (3) holds and thr xists an input distribution that achivs th channl rsolvability, thn it dos not ncssarily achiv th channl capacity. 2 Obsrvation If w assum that thr xists an input distribution P X n capacity, thn th following two conditions ar quivalnt: that achivs th channl. sup X n I(X n ; Y n ) = sup X n I(X n ; Y n ). 2. n i X n W n(x n ; Y n ) convrgs to a constant (which is th capacity C) in probability according to th joint input-output distribution P X n Y n, such that its inducd marginal is P X n inducd conditional distribution P Y n jx n is givn by th channl transition distribution. and th W will hraftr us th condition statd in th abov obsrvation to vrify th validity of th strong convrs. But rst, w not th following rsult. Dn th strong convrs capacity (or strong capacity) C SC as th inmum of th rats R such that for all block cods with rat R and blocklngth n, lim inf P (n) n! = ; whr P (n) is probability of dcoding rror. It follows from th dnition that C SC = lim "! C " : 5

6 Lmma 2 C SC = sup X n I(X n ; Y n ): P roof:. C SC sup X n I(X n ; Y n ): From th dnition of th strong convrs capacity, w only nd to show that if th probability of dcoding rror of a (squnc of) block cod satiss lim inf n! P (n) =, its rat must b gratr than sup X n I(X n ; Y n ). Lt f X n b th input distribution satisfying I( f X n ; Y n ) > sup X n I(X n ; Y n ) ", and lt M = nr. Also lt P (n) satisfy lim inf n! P (n) =. From Thorm in [6] (also from Finstins's lmma), thr xists an (n; M; P (n) ) cod that satiss P (n) P for any > 0, which implis Th abov rsult is idntical to n i X nw n( f X n ; Y n ) n log M + + xp f ng ; (8 > 0) lim inf P n! n i X ( f X n ; Y n ) R + n W n (8 > 0) lim sup P n! n i X ( f X n ; Y n ) > R + n W n Finally, by th dnition of sup-information rat, R must b gratr than I( f X n ; Y n ) > sup X n I(X n ; Y n ) ". Sinc " can b mad arbitrarily small, w hav th dsird rsult. 2. C SC = sup X n I(X n ; Y n ): If C SC > sup X n I(X n ; Y n ), thn thr xists a cod with rat = : = 0: C SC > R = n log M > sup X n I(X n ; Y n ) + " such that lim inf P (n) n! < ; (4) To mak it clar, w r-phras Thorm in [6] as follows. Fix n and 0 (n) < P <, and also x th input distribution P X n on A n. Thn for vry > 0, thr xists an (n) (n; M; P ) cod for th givn transition probability W n that satiss P (n) P n i X n W n ( X n ; Y n ) n log M + + xp f ng : 6

7 for som " > 0. From [6,Thorm 4], vry (n; M) cod satiss, P (n) P n i X nw n(x n ; Y n ) n log M " 2 whr X n placs probability mass =M on ach codword. Hnc, lim inf P n! n i n(x X n W n ; Y n ) n log M " 2 = lim inf n! P n i X n W n(x n ; Y n ) R " 2 = ; xp f "n=2g ; xp f "n=2g xp f "n=2g lim inf n! P n i X n W n(x n ; Y n ) I(X n ; Y n ) + "=2 xp f "n=2g which implis lim inf n! P (n) =, and contradicts (4). 2 It can b asily shown that for any input distribution X n, I(X n ; Y n ) supfr : F X (R) "g I(X n ; Y n ); whr F X (R) = 4 lim sup P n! n i n(x X n W n ; Y n ) R : Hnc, from Thorm 6 in [6], if w assum that sup X n supfr : F X (R) "g is continuous in ", w obtain that C C " C SC : Th abov quation lads to th following rsult. Corollary C = S = C SC i C " = C for all " 2 (0; ). 2. Exampls of channls satisfying th strong convrs A. Additiv nois channl Considr th channl with common input, nois, and output alphabt, A = f0; ; : : : ; q g, dscribd by Y n = X n Z n ; 7

8 whr dnots addition modulo q and X n, Z n and Y n ar rspctivly th input, nois, and output symbols of th channl at tim n, n = ; 2; : : :. W assum that th input and nois squncs ar indpndnt of ach othr. W also assum that th nois procss is stationary and rgodic. Sinc th channl is symmtric, th input procss that achivs (3) is uniform i.i.d., which yilds a uniform i.i.d. output procss. It follows from th Shannon-McMillian thorm that th information spctrum convrgs to C whr C = log q H(Z ). Hr, H(Z ) dnots th nois ntropy rat. Thrfor, th strong convrs holds, and C " = C SC = C for all " 2 (0; ). Obsrvation 2 If th nois procss is only stationary, thn th strong convrs dos not hold in gnral. Indd, by th rgodic dcomposition thorm [2], w can show that th additiv nois channl is an avragd channl whos componnts ar q-ary channls with stationary rgodic additiv nois. In this cas, w obtain using Thorm 6 in [6], a gnral -capacity formula for this channl: C " = log q F U ( "); whr U is a random variabl with cumulativ distribution function F U () 2 such that th squnc n log P (Z n ) convrgs to U in probability. Furthrmor, it is known that U = H (Z ) whr H (Z ) is th ntropy rat of th rgodic componnts dnd on th spac (; (); G) 3. Th distribution of U can hnc b drivd using th mixing distribution G of th avrag channl. Finally, w rmark that as xpctd. lim!0 C " = log q F U () = log q ss sup H (Z ) = C; 2 W assum th CDF F U() admits an invrs. Othrwis, w can rplac F () by F U (x)4 = supfy : FU(y) < xg: U 3 W assum that th probability spac (; (); G) satiss crtain rgularity conditions [2]. 8

9 B. Additiv nois channl with input cost constraints In gnral, th us of th channl is not fr; w associat with ach input lttr x a nonngativ numbr b(x), that w call th \cost" of x. Th function b() is calld th cost function. If w us th channl n conscutiv tims, i.., w snd an input vctor x n = (x ; x 2 ; : : : ; x n ), th cost associatd with this input vctor is \additiv"; i.., b(x n ) = nx i= b(x i ): For an input procss fx i g i= with block input distribution P (n) (X n = x n ) th avrag cost for snding X n is dnd by E [b(x n )] = X x n P (n) (x n ) b(x n ) = nx i= E [b(x i )] : W assum that th cost function is \boundd"; i.., thr xists a nit b max such that b(x) b max for all x in th st f0; : : : ; q g. Dnition 3 An n-dimnsional input random vctor X n = (X ; X 2 ; : : : ; X n ) that satiss n E [b(x n )] ; is calld a -admissibl input vctor. W dnot th st of n-dimnsional -admissibl input distributions by n (): n () = P (n) (X n ) : n E [b(x n )] : Rcall that a channl is said to b stationary if for vry stationary input, th joint inputoutput procss is stationary. Furthrmor, a channl is said to b rgodic if for vry rgodic input procss, th joint input-output procss is rgodic. It is known that a channl with stationary mixing additiv nois is rgodic [2,5]. Lmma 3 If th nois procss is stationary and mixing, thn th strong convrs holds: C " () = C() = lim n! C n (); 9

10 whr C n () is th n'th capacity-cost function givn by C n () 4 = max P (n) (X n )2 n() n I(X n ; Y n ): P roof: Sinc th channl is a causal, historylss 4 and stationary rgodic channl, and th cost function is additiv and boundd, thn thr xists a stationary rgodic input procss that achivs C(). This follows from th dual rsult on th distortion rat function D(R) of stationary rgodic sourcs, which stats that for a stationary rgodic sourc with additiv and boundd distortion masur, thr xists a stationary rgodic input-output procss P X n Y n that th inducd marginal P X n is th sourc distribution [2,3]. that achivs D(R) such Thrfor, if w form th joint input-output procss f(x n ; Y n )g n= using th stationary rgodic input procss that achivs C(), w obtain that f(x n ; Y n )g n= is stationary rgodic. Hnc, i n X n Y n(x n ; Y n ) convrgs to C() in probability. 2 Gnral capacity formula with fdback Considr a discrt channl with output fdback. By this w man that thr xists a \rturn channl" from th rcivr to th transmittr; w assum this rturn channl is noislss, dlaylss, and has larg capacity. Th rcivr uss th rturn channl to inform th transmittr what lttrs wr actually rcivd; ths lttrs ar rcivd at th transmittr bfor th nxt lttr is transmittd, and thrfor can b usd in choosing th nxt transmittd lttr. A fdback cod with blocklngth n and rat R consists of squnc of ncodrs f i : f; 2; : : : ; 2 nr g B i! A for i = ; 2; : : : ; n, along with a dcoding function g : B n! f; 2; : : : ; 2 nr g; 4 Rcall that a channl is said to b causal (with no anticipation) if for a givn input and a givn input-output history, its currnt output is indpndnt of futur inputs. Furthrmor, a channl is said to b historylss (with no input mmory) if its currnt output is indpndnt of prvious inputs. Rfr to [2] for mor rigorous dnitions of causal and historylss channls. 0

11 whr A and B ar th input and output alphabts, rspctivly. Th intrprtation is simpl: If th usr wishs to convy mssag V 2 f; 2; : : : ; 2 nr g thn th rst cod symbol transmittd is X = f (V ); th scond cod symbol transmittd is X 2 = f 2 (V; Y ), whr Y is th channl's output du to X. Th third cod symbol transmittd is X 3 = f 3 (V; Y ; Y 2 ), whr Y 2 is th channl's output du to X 2. This procss is continud until th ncodr transmits X n = f n (V; Y ; Y 2 ; : : : ; Y n ). At this point th dcodr stimats th mssag to b g(y n ), whr Y n = [Y ; Y 2 ; : : : ; Y n ]. W assum that V is uniformly distributd ovr f; 2; : : : ; 2 nr g. Th probability of dcoding rror is thus givn by: P (n) = 2X nr 2 nr k= P rfg(y n ) 6= V jv = kg = P rfg(y n ) 6= V g: W say that a rat R is achivabl (admissibl) if thr xists a squnc of cods with blocklngth n and rat R such that lim n! P (n) = 0: W will dnot th capacity of th channl with fdback by C F B. As bfor, C F B suprmum of all admissibl fdback cod rats. is th Lmma 4 Th gnral capacity formula of an arbitrary channl with fdback is C F B = sup X n I(V ; Y n ); whr th suprmum is takn ovr all possibl fdback ncoding schms. 5 5 P roof:. C F B sup (f ;:::;f n) I(V ; Y n ). W rst stat th following rsult. sup X n I(V ; Y n ) = sup X n =(f (V );f 2(V;Y );:::;fn(v;y n )) I(V ; Y n ) = sup (f ;f 2;:::;fn) I(V ; Y n ):

12 Proposition For a fdback cod of blocklngth n and siz M, th probability of rror satiss P n { W Y n(w ; Y n ) n log M P (n) xp f ng ; for vry > 0, whr P W (W = w) = =M for all w. Th proof of th proposition is as follows. Lt = xp f ng. Dn L 4 = n (w; b n ) 2 f; 2; : : : ; Mg Y n : P W jy n(wjb n ) o = (w; b n ) 2 f; 2; : : : ; Mg Y n : = [ M w= fwg B w; n { n(w; W Y bn ) n log M whr B w 4 = fbn 2 Y n : P W jy n(wjb n ) g. By dning D w 2 Y n b th dcoding st corrsponding to w, w obtain P W Y n(l) = = = MX w= MX w= MX w= MX w= P W Y n(fwg B w ) P W Y n(fwg (B w \ D c w)) + M P Y n jw (B w \ D c wjw) + MX w= MX w= M P Y n jw (D c w jw) + P Y n([m w= D w); P W Y n(fwg (B w \ D w )) P W Y n(fwg (B w \ D w )) bcaus D w ar pair wis disjoint: P (n) + : Basd on this proposition, w can show that using proof-by-contradiction [6]. 2. C F B sup (f ;:::;f n) I(V ; Y n ). C F B sup I(V ; Y n ) (f ;:::;f n) This follows dirctly using Finstin's lmma as in [6]. 2 2

13 Gnral formula for th Nyman-Parson hypothsis tsting rror xponnt In this sction, w considr a Nyman-Parson hypothsis tsting problm for tsting a null hypothsis H 0 : P X n against an altrnativ hypothsis H : Q X n basd on a squnc of random obsrvations X n = (X ; : : : ; X n ), which is supposd to xhibit a probability distribution of ithr P X n or Q X n. Upon rcipt of th n obsrvations, a nal dcision about th natur of th random obsrvations is mad so that th typ-ii rror probability n, subjct to a xd uppr bound " on th typ-i rror probability n, is minimizd. Th typ-i rror probability is dnd as th probability of accpting hypothsis H whn actually H 0 is tru; whil th typ-ii rror probability is dnd as th probability of accpting hypothsis H 0 whn actually H is tru []. For arbitrary obsrvations (not ncssarily stationary, rgodic), w driv a gnral formula for th typ-ii rror xponnt subjct to a constant uppr bound " on th typ-i rror probability. This is givn in th following lmma. Lmma 5 Givn a squnc of random obsrvations X n = (X ; : : : ; X n ) which is assumd to hav a probability distribution ithr P X n or Q X n, th typ-ii rror xponnt satiss whr supfd : F(D) < "g lim sup n! n log n(") supfd : F (D) "g; (5) supfd : F (D) < "g lim inf n! n log n (") supfd : F (D) "g; (6) " # F(D) = 4 lim inf P n! Q(X n ) D " # ; and F(D) = 4 lim sup P n! Q(X n ) D ; and n (") rprsnts th minimum typ-ii rror probability subjct to a xd typ-i rror bound " 2 (0; ). P roof: W rst prov th lowr bound of lim sup n! (=n) log n("). For any D satisfying F(D) < ", thr xists > 0 such that F(D) < " 2; and hnc, by th dnition of F(D), (9 3

14 a subsqunc fn j g and N) such that for j > N, " nj P (X ) P log D n j Q(X n j ) # " < ": " : : : nj P (X ) n j (") Q log > D n j Q(X n j ) " nj P (X ) P log > D n j Q(X n j ) # # xp f n j Dg xp f n j Dg : (7) Thrfor, for any D with F(D) < ". lim sup n! n log n(") lim sup log n n j (") D; j j! For th proof of th uppr bound of lim sup n! (=n) log n("), lt U n b th optimal accptanc rgion for altrnativ hypothsis undr liklihood ratio partition, which is dnd as follows. ( ) ( ) 4 U n = Q(X n ) < n + n Q(X n ) = n ; (8) for som n and possibl randomization factor n 2 [0; ). Thn P (U n ) = ". Lt D = supfd : F(D) "g. Thn F(D + ) > " for any > 0. Hnc, (9 = () > 0), F(D + ) > " + : By th dnition of F(D + ), (9 N)(8 n > N) P " Q(X n ) D + # > " + 2 : Thrfor, " n(") = Q Q Q(X n ) > n " D + P # " + ( n ) Q Q(X n ) > n " D + Q(X n ) > n # xp f n(d + )g 4 Q(X n ) = n " # + ( n ) Q Q(X n ) = n # " + ( n ) P Q(X n ) = n # #!

15 " # = P Q(X n ) D + P " #! n P Q(X n ) = n " + 2 " xp f n(d + )g ; " # Q(X n ) < n xp f n(d + )g for n > N = xp f n(d + )g ; for n > N: (9) 2 Sinc can b mad arbitrarily small, : : : lim sup n! n log n(") D + : lim sup n! n log n(") D: Similarly, to prov th lowr bound of lim inf n! (=n) log n("), w rst not that for any D satisfying F(D) < ", (9 > 0) such that F(D) < " 2; and hnc, by th dnition of F(D), (9 N)(8 n > N), P " Q(X n ) D # By following th sam procdur of (7), w hav for n > N, n(") xp f ndg ; " < ": Thrfor, for any D with F(D) < ". lim inf n! n log n (") D; Thn for th proof of th uppr bound of lim inf n! (=n) log n ("), lt D = supfd : F(D) "g. Thn F( D + ) > " for any > 0. Hnc, (9 = () > 0), F( D + ) > " + : By th dnition of F( D + ), (9 a subsqunc fn j g and N) such that for j > N, " # nj P (X ) P log D n j Q(X + > " + n j ) 2 : Thrfor, by following th sam procdur as (9), w hav for j > N, n j (") 2 xp n n j ( D + ) o 5

16 : : : lim inf n! n log n(") lim inf j! n j log n j (") D + : Sinc can b mad arbitrarily small, lim inf n! n log n(") D: 2 Rmarks: Both F(D) and F (D) ar non-dcrasing; hnc, th numbr of discontinuous points for both functions is countabl. Whn th normalizd log-liklihood ratio convrgs in probability to a constant D c undr null distribution which is th cas for most dtction problms of intrst, th typ-ii rror xponnt is that constant D c, and is indpndnt of th typ-i rror bound ". For xampl, in a spcial cas of i.i.d. data sourc with je P [log P (X)=Q(X)]j <, both functions dgnrat to th form F (D) = F(D) = if D > D c F (D) = F(D) = 0 if D < D c ; whr D c 4 = EP [log P (X)=Q(X)]. As a rsult, for " 2 (0; ), lim sup n! n log n(") = lim inf n! n log n(") = D c : Th signicanc of th gnral typ-ii rror xponnt formula of xd lvl bcoms transparnt whn th spctrum (th cumulativ distribution function) of th normalizd logliklihood ratio convrgs in probability undr P (which is wakr than convrgnc in man) to a random variabl Z with invrtibl cumulativ distribution function F (). In this cas, th typ-ii rror xponnt can b xplicitly writtn as lim n! n log n (") = F ("); 6

17 for " 2 (0; ). A mor xtrm cas is that Z is almost surly a constant which is lim n! n D (P X nkq n) X ; if th limit xists, whr D(k) is th Kullback-Liblr divrgnc of two probability masurs. This rsult coincids with that obtaind from Stin's Lmma. This is also th countrpart rsult of th strong convrs condition (i.., th "-capacity is indpndnt of ") for discrt mmorylss channls (DMC) [6]. Summary In this papr, w considrd thr dirnt problms rlatd to th work of Vrdu and Han on channl capacity [6]. Prtinnt obsrvations concrning th validity of th strong convrs to th channl coding thorm, as wll as xampls of channls for which th strong convrs holds, wr providd. Gnral xprssions for th fdback capacity of arbitrary channls and th Nyman-Parson typ-ii rror xponnt of constant tst lvl wr also drivd. Rfrncs. R. E. Blahut, Principls and Practic of Information Thory, Addison Wsly, Nw York (987). 2. R. M. Gray, Entropy and Information Thory, Springr-Vrlag Nw York Inc. (990). 3. R. M. Gray, Sourc Coding Thory, Kluwr Acadmic Publishrs, Norwll, MA (990). 4. T. S. Han and S. Vrdu, \Approximation Thory of Output Statistics", IEEE Transactions on Information Thory, Vol. 39, No. 3, pp (993). 5. M. S. Pinskr, Information and Information Stability of Random Variabls and Procsss, Holdn-Day, San Francisco (964). 6. S. Vrdu and T. S. Han, \A Gnral Formula for Channl Capacity", IEEE Transactions on Information Thory, vol. 40, pp , July