Abstract: The asympttically ptimal hypthesis testig prblem with the geeral surces as the ull ad alterative hyptheses is studied uder expetial-type err

Transcription

1 Hypthesis Testig with the Geeral Surce y Te Su HAN z April 26, 2000 y This paper is a exteded ad revised versi f Sectis i Chapter 4 f the Japaese bk f Ha [8]. z Te Su Ha is with the Graduate Schl f Ifrmati Systems, Uiversity f Electr- Cmmuicatis, Chfugaka -5-, Chfu, Tky , Japa. ha@is.uec.ac.jp

2 Abstract: The asympttically ptimal hypthesis testig prblem with the geeral surces as the ull ad alterative hyptheses is studied uder expetial-type errr cstraits the rst kid f errr prbability. Our fudametal philsphy i dig s is rst t cvert all f the hypthesis testig prblems cmpletely t the pertiet cmputati prblems i the large deviati-prbability thery. It turs ut that this kid f methdlgically ew apprach eables us t establish quite cmpact geeral frmulas f the ptimal expets f the secd kid f errr ad crrect testig prbabbilities fr the geeral surces icludig all statiary ad/r ergdic surces with arbitrary abstract alphabet (cutable r ucutable). Such geeral frmulas are preseted frm the ifrmati-spectrum pit f view. Idex terms: geeral surce, hypthesis testig, rst kid f errr prbability, secd kid f errr prbability, ifrmati spectrum, ifrmati spectrum slicig, large deviati, abstract alphabet 0

3 Itrducti The hypthesis testig prblem is very imprtat t ly frm the theretical viewpit but als frm the egieerig pit f view. This fudametal research subject i the hypthesis testig prblem seems t have started earlier i the 930's with the asympttic study that fr e sht surces with real alphabet (e.g., see Neyma ad Pears []) ad subsequetly has bee geeralized it varius kids f directis icludig that f the asympttic apprachtadiversity f surce prcesses. I the preset paperwe csider a wide class f geeral surces (geeralized prcesses) as ull ad alterative hyptheses. Let us rst dee the geeral surce as a iite sequece X = fx =(X () X () )g = f -dimesial radm variables X where each cmpet radm variable X () i ( i ) takes values i a arbitrary abstract set X that we call the surce alphabet (cf. Ha [20]). It shuld be ted here that each cmpet fx may chage depedig blck legth. This implies that the sequece X is quite geeral i the sese that it may t satisfy eve the csistecy cditi as usual prcesses, where the csistecy cditi meas that fr ay itegers m such that m<it hlds that X (m) i X () i fr all i = 2 m: The class f surces thus deed cvers a very wide rage f surces icludig all statiary ad/r ergdic surces. The itrducti f such a class f geeral surces is crucial i the whle argumet i the sequel. Thus, give tw arbitrary geeral surces X = fx g = ad X = fx g = takig values i the same surce alphabet fx g =, wemay dee the geeral hypthesis testig prblem with X = fx g = as the ull hypthesis ad X = fx g = as the alterative hypthesis. emark. A mre reasable deiti f the geeral surce is the fllwig. Let fz g = be ay sequece f arbitrary surce alphabets Z ad let Z be ay radm variable takig values i Z ( = 2 ). The, the sequece Z = fz g = f radm variables Z is called a geeral surce. The abve deiti is a special case f this geeral surce with Z = X ( = 2 ). The key results i this paper (Therem 2. ad Therem 4. ) ctiue t be valid as well als i this mre geeral settig with fx g = (surce alphabet), X = fx g = (ull hypthesis), X = fx g = (alterative hypthesis) replaced by fz g = (surce alphabet), Z = fz g = (ull hypthesis), Z = fz g = (alterative hypthesis), respectively, where bth f Z ad Z take values i Z ( = 2 ). 2 I the preset paper, with this kid f geeral hypthesis testigs we ivestigate the ptimal expet prblem fr the prbability f testig errr as well as the ptimal expet prblem fr the prbability f crrect testig. Frmally,letA be ay subset f X ( = 2 ) that we call the acceptace regi f the hypthesis testig, ad dee fx 62 A g fx 2A g (:) where are called the rst kid f errr prbability ad the secd kid f errr prbability, respectively.

4 Oe f the basic prblems i the hypthesis testig is t determie the supremum B e (rjxjjx) fachievable expets fr the secd kid f errr prbability uder asympttic cstraits f the frm e ;r the rst kid f errr prbability (r>0is a prescribed arbitrary cstat) which meas that the rst kid f errr prbability isrequired at mst t decay expetially fast with the expet r. Ather basic prblem i the hypthesis testig is t determie the imum Be (rjxjjx) fachievable expets fr the secd kid f crrect prbability ; uder asympttic cstraits f the same frm as abve e ;r the rst kid f errr prbability (r>0 is agai a prescribed arbitrary cstat). I the fllwig sectis we fcus these tw basic prblems fr the geeral hypthesis testigs. We establish a geeral frmula (Therem 2.) fr B e (rjxjjx) i Secti 2 alg with several typical examples i Secti 3, whereas we establish a geeral frmula (Therem 4.) fr Be (rjxjjx) i Secti 4 alg with several typical examples i Secti 5. I rder t drive the geeral frmula fr B e (rjxjjx) aswell as that fr Be (rjxjjx) i a surprisigly uifyig way, we shall take aifrmatispectrum apprach that had bee eectively ivked already i Ha ad Verdu [], Verdu ad Ha [5], Ha [7, 9, 20], where the substatially vel techique f ifrmati spectrum slicig, as explited i Ha [7, 8], plays the key rle. Our fudametal philsphy here is rst t cvert all f the hypthesis testig prblems cmpletely t the pertiet cmputati prblems i the large deviati-prbability thery. We ca the expel all the acceptace-regi argumets frm the rigial hypthesis testig prblems thereby, all f what we shuld d bils dw slely t hw t cmpute the relevat large deviati prbabilities (r, i may stadard cases, the relevat rate fuctis). It turs ut that this kid f methdlgically ew apprach eables us t establish quite cmpact geeral frmulas f the expet fuctis B e (rjxjjx), Be (rjxjjx) fr geeral surces icludig all statiary ad/r ergdic surces with abstract alphabet. Such geeral frmulas are preseted i this paper. Fially, i Secti 6 we pleasigly bserve that all the argumets develped i Sectis 25 ctiue t be valid eve if we replace the geeral alterative hypthesis X = fx g = by ay sequece X = fg g = f egative measures (fr example, cutig measures t ecessarily prbability measures), ad as a csequece i Secti 7 it is revealed that there exists a itrisic e-t-e peratial crrespdece betwee the prblem f s geeralized hypthesis testigs ad the prblem f geeral xed-legth surce cdigs. As a illustrative case, it is shw i the case f cutably iite surce alphabet X that the geeral frmula f Ha [20] frthe imum e (rjx) fachievable cdig rates uder asympttic cstraits f the frm " e ;r (r > 0) the errr prbability " with xed-legth surce cdig immediately fllws frm the geeral frmula (Therem 2.) fr B e (rjxjjx) (withthe sequece X = fc g = f cutig measures) as drive i Secti 2. It thus turs ut that the geeral xed-legth surce cdig prblem is just a special case f the s geeralized hypthesis testig prblem. 2

5 2 Hypthesis Testig ad Large Deviati: bability f Testig Errr I this secti we ivestigate the prblem f determiig the supremum B e (rjxjjx) f achievable expets fr the secd kid f errr prbability uder asympttic cstraits f the frm e ;r the rst kid f errr prbability (r>0is a prescribed arbitrary cstat). Let us rst give the frmal deitits, where X = fx g =, X = fx g = idicate the ull hypthesis ad the alterative hypthesis, respectively. Deiti 2. A rate E is called r- achievable if there exists a acceptace regi A such that lim if! lg r ad lim if! lg E: Deiti 2.2 (The supremum f r-achievable errr expets) B e (rjxjjx) = sup fe j E is r-achievableg : The purpse f this secti is t determie B e (rjxjjx) as a fucti f r. T this ed, we csider the radm variable lg (X ) P X (X that we call the divergece-desity ) rate, ad dee the key fucti () by () = lim if! lg lg P (2:) X (X ) (X ) where i the sequel we use the cveti that P Z () detes the prbability distributi f a radm variable Z. It is bvius that this fucti () is mte decreasig i but t ecessarily ctiuus. Next, dee the spectral if-divergece rate D(XjjX) f the radm variable lg (X ) (X ) as Deiti 2.3 D(XjjX) = p- lim if! Lemma 2. If >D(XjjX), the () =0. lg (X ) (X ) :y I the case where the surce alphabet X is abstract i geeral, it is uderstd that g(x) (x) (x) (x 2 X ) detes the ad-nikdym derivative betwee tw prbability measures X P with values a sigular set assumed cvetially t be +. The, X (X (X ) is deed as (X (X ) g(x ), which isbviusly a radm variable. The prbability distributi f the divergece-desity rate is called the divergece-spectrum r mre geerally the ifrmati-spectrum (cf. Ha ad Verdu []). y Fr ay sequece fzg = f real-valued radm variables, we dee the limit iferir i prbability (cf. Ha ad Verdu []) f fz g = by p- lim if Z = supfj lim fz <g =0g:!! 3

6 f: If >D(XjjX), the by the deiti f D(XjjX) there exists a 0 <" 0 < such that hlds fr iitely may. Hece, lg (X ) (X ) >" 0 () lim if! lg " 0 =0: We w have the fllwig quite geeral frmula: Therem 2. Fr ay r 0, B e (rjxjjx) = if where B e (0jXjjX) =+ (r = 0). f + () j () <rg (2:2) emark 2. We tice here that () < r the right-had side f (2.2) is t () r. This is a essetial dierece, as will be see i the prf belw. Als, it is t dicult t check that + () 0 fr all ; << emark 2.2 Sice it fllws frm Lemma 2. that if f + () j () <rg = if >D(XjjX) >D(XjjX) ad if the right-had side is attaied by = D(XjjX), we may replace if the right-had side f (2.2) by if () is ctiuus at = D(XjjX). 2 f f Therem 2. z ) Direct part: We use the tati that S (a) = if D(XjjX) x 2X lg (x) (x) >a : (2:3) Let = if f j () <rg (2:4) z Oe f the referees suggested that the prf belw based the ifrmati-spectrum slicig is substatially similar t that f Varadha's itegral lemma (cf. Demb ad Zeitui [4]), but this fact des ever mea that Therem 2. is a csequece f Varadha's itegral lemma, because the latter assumes the existece f a gd rate fucti. 4

7 ad csider the hypthesis testig with the acceptace regi A = S ( ; ) with a arbitrarily small >0. The, the rst kid f errr prbabbility is give by Hece, = fx =2A g = lim if! lg (X ) (X ) ; : lg = ( ; ): O the ther had, (2.4) implies ( ; ) r. Therefre, lim if! lg r: (2:5) Next, let us evaluate the secd kid f errr prbability. First, put 0 = if f + () j () <rg : (2:6) We take K large eugh s as t satisfy K > 0 ad put L = (K ; + )=(2). Divide the iterval ( ; K] it L subitervals with equal width 2 t dee I i =(b i ; 2 b i ] (i = 2 L) (2:7) where b i ; +2i. Accrdig t this iterval partiti, divide the set T 0 = x 2X ; < lg (x) (x) K it the fllwig L subsets (Ifrmati-spectrum slicig): S (i) = Mrever, we dee t have x 2X lg (x) (x) 2 I i Sice fr i = 2 L it hlds that S (0) = x 2X lg (x) (x) >K X 2 S (i) S ( ; ) = L[ i=0 lg (X ) (i = 2 L): S (i) : (2:8) (X ) b i 5

8 we have Hece, lim if! lg (b i ): X 2 S (i) X 2 S (i) e ;((b i);) (8 0 ): (2:9) Mrever, if x 2 S (i) the ad s As a result, by meas f (2.9) we have x X 2 S (i) Sice b i + fr all i = 2 L, lg (x) (x) >b i ; 2 (x) (x)e ;(bi;2) : X x2s (i) (x)e ;(bi;2) e ;(bi+(bi);3) : (2.0) b i + (b i ) 0 (i = 2 L): Substituti f this it (2.0) yields X 2 S (i) e ;( 0;3) (i = 2 L): (2:) O the ther had, takig accut thatx 2 S (0) have X 2 S (0) Csequetly, frm (2.8), (2.), (2.2), = = X x2s (0) X e ;K implies (x) (x)e ;K,we (x) x2s (0) (x) e ;K : (2.2) X 2 S ( ; ) Le ;( 0;3) + e ;K : x I the case where the surce alphabet X is abstract i geeral, the summati P is uderstd t dete the itegral. 6

9 We tice here that K> 0 ; 3 ( >0) because K> 0. Thus, lim if! lg 0 ; 3 which tgether with (2.5) ccludes that 0 ; 3 is r-achievable (Ntice here that >0 is arbitrarily small). ) Cverse part: { Let ad 0 be deed as i (2.4), (2.6), respectively. The, sice () is mte decreasig i, there exists a 0 such that 0 ad Let us csider the set S 0 = lim( 0 + " + ( 0 + ")) = 0 : (2:3) "#0 x 2X lg (x) (x) 0 + where >0 is a arbitrarily small cstat. The, by the deiti f (), there exists sme diverget sequece < 2 <! f itegers such that fx j 2 S 0 ge ;j ((0+)+ ) (8j j 0 ) (2:4) where > 0 is a arbitrarily small cstat. Nw let us use the ctradicti argumet. T d s, assume that E = 0 +2 ( >0 is a xed cstat) is r-achivable, i.e., assume that there exists a acceptace regi A such that ad Sice x 2 S 0 implies we have lim if! lim if! lg r (2:5) lg E 0 +2: (2:6) (x) (x)e (0+) fx 2 S 0 \A g = X X x2s 0\A (x) (x)e ( 0+) x2s 0\A X e (0+) (x) x2a = e (0+) : (2.7) { Althugh it is usual i hypthesis testig prblems t ivke the Neyma-Pears lemma i rder t prve the cverse part, here we will give ather simple elemetary prf withut recurse t the Neyma-Pears lemma. This is t shw that several alterative prfs are pssible. 7

10 Furthermre, it fllws frm (2.6) that Substituti f this it (2.7) yields e ;(E;) (8 0 ): fx 2 S 0 \A g e ;(E;0;2) By virtue f (2.3), fr ay >0 small eugh, Therefre, by (2.8) we have = e ;(0;0+2;2) : (2.8) ( 0 + ) ; : fx 2 S 0 \A ge ;((0+)+;) : Next, let us take >0, >0 s small as t satisfy >2 +, the fx 2 S 0 \A ge ;((0+)+2 ) (2:9) where > 0 is the same e as i (2.4). O the ther had, by usig (2.5), we btai fx 2 S 0 \A c g fx 2A c g = e ;(r; ) (8 0 ): (2.20) We bserve here that ( 0 + ) <rfr all >0, ad hece, fr ay sucietly small > 0, ( 0 + )+2<r; : The, it fllws frm (2.9), (2.20) that fx 2 S 0 g = fx 2 S 0 \A g +fx 2 S 0 \A c g e ;((0+)+2 ) + e ;(r; ) 2e ;((0+)+2 ) (2.2) fr all 0. Hwever, sice > 0, (2.2) ctradicts (2.4). Thus, the rate E = 0 +2 cat be r-achievable. Sice >0 is arbitrary, it is ccluded that ay E such that E> 0 cat be r-achievable. 2 3 Examples I this secti we demstrate several typical applicatis f Therem 2.. This is t verify the ptetialities f Therem 2.. 8

11 Example 3. Let the surce alphabet X be ite, ad csider the hypthesis testig where the ull hypthsis X = (X X 2 ) ad the alterative hypthesis X = (X X 2 ) are statiary irreducible Markv surces subject t trasiti prbabilities P (x 2 jx )=fx 2 = x 2 jx = x g, P (x 2 jx )= X 2 = x 2 jx = x (x x 2 2X), respectively. Let P(XX) dete the set f all prbability distributis XX, ad, fr ay Q 2P(X X X) dee the cditial divergeces as D(QjjPjq) = q(x )D(Q(jx )jjp (jx )) x 2X X D(QjjPjq) = q(x )D(Q(jx )jjp (jx )) x 2X where D(jj) is the divergece (cf. Csiszar ad Krer [6]), ad q() adq(j) dete the margial distributi ad the cditial distributi f Q, respectively, which are deed as X q(x ) = Q(x x 2 ) x 22X Q(x 2 jx ) = Q(x x 2 ) : q(x ) The, by usig Sav therem the statiary irreducible Markv surce (cf. Demb ad Zeitui [4]), wehave () = 0 fr D(PjjPjp)(p is the statiary distributi fr P ) ad, fr D(PjjPjp), () =D(P jjpjp ) (3:) + () =D(P jjpjp ) (3:2) where, lettig P 0 be the set all prbability distributis Q 2P(XX) satisfyig the statiarity, i.e., P 0 = ( Q 2P(X X) X x 2X Q(x x)= X x 22X P 2P 0 detes the prjecti f P the plae: as specied by = 8 < : Q 2P 0 X x x 22X Q(x x 2 ) fr all x 2X Q(x x 2 ) lg P (x 2jx ) P (x 2 jx ) = 9 = ) (3:3) (3:4) if D(QjjPjq) =D(P jjpjp ) (3:5) Q2 with q beig the margial distributi f Q, ad p is the margial distributi f P. Ntice here that, sice Q mves, (3.5) implies als that if D(QjjPjq) =D(P jjpjp ): (3:6) Q2 9

12 It is easy t see that D(XjjX) =D(PjjPjp) (cf. Barr [7]) ad the fucti () give by (3.) is ctiuus at = D(PjjPjp). Therefre, i view f emark 2.2, it suces t csider ly 's such that D(PjjPjp) the right-had side f (2.2). (Such a bservati applies als t all the subsequet examples except fr Example 3.6.) Thus, Therem 2. leads us t D(P jjpjp ) j D(P jjpjp ) <r B e (rjxjjx) = if = if D(QjjPjq) (8r >0): (3.7) Q2P 0:D(QjjP jq)<r This result has bee btaied by Nataraja [4]. This frmula tells als that B e (rjxjjx) = 0 wheever r D(PjjPjp) (p is the statiary distributi crrespdig t P ). If we csider the special case where surces X, X are bth statiary memryless subject t distributis P, P X, respectively, the frmula (3.7) reduces t B e (rjxjjx) = if D(QjjP ): (3:8) Q:D(QjjP )<r This is thig but Hedig's therem [3] as is well kw i the eld f statistics. This tells als that B e (rjxjjx) = 0 wheever r D(P jjp ). 2 Example 3.2 Let us geeralize Example 3. t the case with uilar ite-state surces istead f statiary irreducible Markv surces. With the surce alphabet X (ite) ad the state set S (ite), let the ull hypthesis X = fx =(X X )g = be the uilar ite-state surce specied by (x) = Y i= P (x i js i ) (x =(x x 2 x ) 2X ) (3.9) s i+ = f(x i s i ) (s i 2S i = 2 +) (3.0) ad the let alterative hypthesis X = fx =(X X )g = be the uilar itestate surce specied by (x) = Y i= P (x i js i ) (x =(x x 2 x ) 2X ) (3.) s i+ = f(x i s i ) (s i 2S i = 2 +): (3.2) Give ay xed iitial state s 2 S, let S 0 dete the set f all states s 2 S that ca be reached frm s with psitive prbability with respect t. Next, lettig XS (X S) beay radm variable takig values i XS 0, put S 0 = f(x S): (3:3) Mrever, let V 0 dete the set f all the jit prbability distributis S f radm variables XS satisfyig bth f the statiarity cditi P S 0() =P S () 0

13 ad the cditi that the trasiti prbability matrix P S 0jS(j) is irreducible. Let the prjecti S 2V 0 f P (j) the plae be deed by where = if D(S jjpjp S )=D(S jjpjp S ) (3:4) S2 8 < : S 2V 0 X x2x s2s 0 S (x s) lg P (xjs) P (xjs) = 9 = : (3:5) The, Sav therem the uilar ite-state surce (cf. Ha [2]) yields () = D(S jjpjp S ) (3.6) + () = D(S jjpjp S ): (3.7) Ntice here that, sice S mves, (3.4) implies als that if D(S jjpjp S )=D(S jjpjp S ): (3:8) S2 Thus, by Therem 2. we have the fllwig frmula fr the hypthesis testig X agaist X with uilar ite-state surces: B e (rjxjjx) = if D(PXS jjpjp S ) jd(s jjpjp S ) <r = if D(SjjPjP S ) (8r >0): (3.9) S2V 0:D(SjjP jp S)<r I the abve argumet we have take accut that i geeral the uilar ite-state surce is asympttically a mixture f statiary r peridic irreducible surces. 2 Example 3.3 Let us csider the hypthesis testig with a mixed surce as the ull hypthesis, whe the surce alphabet X is ite. Let the alterative hypthesis X = fx g = be a statiary memryless surce subject t prbability distributi P. Mrever, with ay statiary memryless surces X = fx g = X 2 = fx 2 g = subject t prbability distributis P P 2, respectively, let the ull hypthesis X = fx g = (called the mixed surce f X ad X 2 )bedeedby (x) = (x)+ 2 2 (x) (8x 2X ) (3:20) where > 0 2 > 0 are cstats such that + 2 =. I rder t drive the required frmula fr this case, let the half-spaces 2 be deed by ( ) = Q 2P(X X ) Q(x) lg P (x) P 2 (x) 0 (3:2) x2x

14 2 = ( Q 2P(X ) X x2x x2x Q(x) lg P (x) P 2 (x) 0 ) (3:22) where P(X ) is the set f all prbability distributis X. Mrever, dee ther half-spaces i P(X )as ( ) () = Q 2P(X X ) Q(x) lg P (x) P (x) (3.23) x2x ( ) (2) = Q 2P(X X ) Q(x) lg P 2(x) P (x) : (3.24) The, lettig the prjectis f P P 2 \ () 2 \ (2) () be deted by P, P (2), respectively, Sav therem cmbied with the argumet ftypes (cf. Ha [20]) gives () = mi(d(p () jjp ) D(P (2) jjp 2 )): (3:25) Substitutig this () it the right-had side f (2.2) i Therem 2., we ca cmpute the value f B e (rjxjjx) as a fucti f r fr the hypthesis testig with mixed surces. Here, it easily fllws frm (3.25) that if mi(d(p jjp ) D(P 2 jjp )) the () = 0, ad that () is a mte decreasig ctiuus fucti f. Hece, B e (rjxjjx) mi(d(p jjp ) D(P 2 jjp )) (8r >0): (3:26) O the ther had, it fllws agai frm (3.25) als that (h) > 0 fr ay h such that h<mi(d(p jjp ) D(P 2 jjp )), ad s if f + ()j() <(h)gh which implies that h is (h)-achievable. Hece, it hlds that lim B e (rjxjjx) = mi(d(p jjp ) D(P 2 jjp )): (3:27) r#0 emark 3. I fact, hwever, it is pssible t drive a mre geeral ad much simpler frmula fr B e (rjxjjx) with mixed surces, withut ay calculati f ifrmati spectra.. With abstract surce alphabet X i geeral, let X = fx g = X 2 = fx 2 g = X = fx g = X 2 = fx 2g = be ay geeral surces. Csider the mixed surce X = fx g = f X ad X 2 ad the mixed surce X = fx g = f X ad X 2, i the sese f Example 3.3, respectively. The, fr the hypthesis testig X agaist X, we have the geeral frmula: B e (rjxjjx) = mi B e (rjx i jjx j ) (8r >0): (3:28) i j2 As fr the detailed prf f (3.28), see Ha [8]

15 Example 3.4 Let us here csider the case with cutably iite surce alphabet X,say, X = f 2 g. I this case, Sav therem as i Examples des t ecessarily hld, while, sice Cramer therem (cf. Demb ad Zeitui [4]) always hlds, we ca ivke here Cramer therem istead f Sav therem. First, let P =(p p 2 ), P =(p p 2 )beay prbablity distributis X, ad let X, X dete the radm variables such that fx = kg = p k,fx = kg = p k. Let X = fx =(X X 2 X )g, X = = fx = (X X 2 X )g = be the statiary memryless surces specied by X, X, respectively. The, sice the divergecedesity rate is decmpsed as () i (2.) ca be expressed as lg (X ) (X ) = X i= lg i (X i ) i (X i ) (3:29) () = if I(x) (3:30) x where I(x) is the large deviati rate fucti fr (3.29). geeratig fucti M() f lg is deed by PX (X) (X) M() = Ee lg (X) (X) = = X i= X i= p i e lg p i p i As usual, the mmet p + i p ; i : (3.3) If we set () = lg M(), Cramer therem tells us that the rate fucti I(x) is give by I(x) = sup(x ; ()) (3:32) where ; lg M() is called the Cher's -distace (cf. Thmas [9]). The expectati f lg is cmputed as E lg (X) (X) = X i= PX (X) (X) p i lg p i p i D(PjjP ) Blahut [8], Cver ad (the divergece): Therefre, frm (3.30) we see that if D(P jjp ) the () = 0, ad if D(P jjp ) the () = I(). (It shuld be ted that I(x) is mte icreasig i the rage f x D(PjjP ), ad mte decreasig i the rage f x D(PjjP ) ad I(x) =0 fr x = D(PjjP ).) The, substitutig (3.30) it (2.2) i Therem 2., we ca btai the frmula fr cmputig the values f B e (rjxjjx). Substituti f (3.3) it (3.32) with x = yields I() = sup( ; lg 3 X i= p + i p ; i ) (3:33)

16 which eables us t cmpute the values f I(). T cmpute this, dieretiate the term i the bracket the right-had side f (3.33) with respect t ad put it t zer t have the equati with respect t : = X p + i p ; i i= X i= lg p i p i p + i p ; i '(): (3:34) As far as P 6= P, it is easy t check by usig Schwarz iequality (cf. Gallager [0]) that '() the right-had side is ctiuus ad strictly mte icreasig i, because M() is term-by-term ctiuusly dietiable (cf. Demb ad Zeitui [4]). As a result, D f; <'() < +jg frms a iterval the real lie. Therefre, i the case with 2D, I() ca be cmputed as I() = ; lg X i= p + i p ; i (3:35) where is the e as specied by (3.34). I this case, lettig P(X ) dete the set f all prbability distributis X ad Q dete the prjecti f the distributi P the plae i P(X ): = ( Q 2P(X ) X i= we ca ascertai by a direct calculati that ad Q (i) = ) Q(i) lg p i = p i I() =D(Q jjp ) (3:36) p + i p ; i P i= p+ i p ; i (i 2X) (3:37) with specied by the equati (3.34). Csequetly, i the cutably iite alphabet case with 2D, Cramer therem equivaletly reduces t Sav therem as i (3.8) f Example 3. with ite alphabet. O the ther had, hwever, i the case with =2D, the relati such as (3.36) des t hld. It the matters what iterval D frms i geeral. I particular, if the D(PjjP ) < + D(PjjP ) < + (3:38) [;D(PjjP ) D(PjjP )] D: I this case, therefre, by usig Sav therem i the same maer as i (3.8) f Example 3., we have fr 0 <r D(PjjP ) the frmula B e (rjxjjx) = if D(QjjP ) (3:39) Q:D(QjjP )<r 4

17 where it is easy t check that (3.39) hlds als fr r>d(pjjp )withb e (rjxjjx) =0. The frmula (3.39) gives a exteded versi with cutably iite surce alphabet X f Hedig's therem with ite surece alphabet X. It shuld be emphasized here that the frmula (3.39) actually hlds eve with ay abstract surce alphabet X uder the mdest cditi (3.38). I fact, the whle argumet develped abve ctiues t be valid, if ly we equivaletly rewrite p + i p ; i i the frm p i pi p i where bth f pi p ad p i i p i i the latter frm are well-deed as the ad-nikdym derivatives (cf. Billigsley [2]) eve with ay abstract surce alphabet X, i that the cditi (3.38) is equivalet t the prperty that the prbability measure P is abslutely ctiuus with respect t the prbability measure P ad cversely the prbability measure P is abslutely ctiuus with respect t the prbability measure P. The Cramer type equivalet f the frmula (3.39) uder cditi (3.38) is fud i Demb ad Zeitui [4] where the Neyma-Pears lemma is directly ivked, while here Therem 2. is ivked. 2 Example 3.5 Let us csider the hypthesis testig where the ull hypthesis X ad the alterative hypthesis X are bth statiary memryless surces subject t Gaussia distributis N( 2 ), N( 2 ), respectively. Let the prbability desities f these Gaussia distributis be writte as P (x) = p e ; (x;) P (x) = p 2 e ; (x;)2 2 2 : Detig by X the radm variable subject t the prbability desity P, the mmet geeratig fucti M() =E(e Y )f Y = lg P (X) P (X) (3:40) is cmputed as s that M() =e (;)2 (+ 2 ) 2 2 x ; lg M() =x ; ( ; )2 ( + 2 ) 2 2 : The, the large deviati rate fucti I(x) f (3.40) is give by I(x) =sup(x ; lg M()) = 2 (x ; a) 2 2( ; ) (3:4) 2 5

18 where, fr simplicity,wehave put a = (;) Icidetally,we bserve that D(P jjp )= a. The, by meas f Cramer therem, () i Therem 2. ca be cmputed as 2 (x ; a) 2 () = if I(x) = mi x x 2( ; ) 2 = mi [a ; ] + 2 ( ; a) 2 (3.42) 2( ; ) 2 frm which it fllws that + () = mi +[a ; ] ( ; a) 2 2( ; ) 2 = mi +[a ; ] + 2 ( + a) 2 : (3.43) 2( ; ) 2 Thus, substitutig (3.42) ad (3.43) it the right-had side f (2.2) i Therem 2., we have B e (rjxjjx) = mi [a ; r] + ( p r ; p a) 2 = ( p r ; p a) 2 [r a] where [ ] stads fr the characteristic fucti. This frmula tells us that B e (rjxjjx) is mte decreasig i r fr 0 <r<a, ad als that B e (0jXjjX) =a = D(P jjp ) ad B e (rjxjjx) =0frr a. 2 Example 3.6 I all the examples that we have shw s far, the fuctis () were ctiuus i. Here, we shw aexample i which () is disctius i, where emark 2.2 des t wrk. Let the surce alphabet be X = f0 g, ads be a subset f X with size js j =2, where is a cstat such that0<<. Mrever, let tw elemets x 0 x 2 X ; S be xed s that x 0 6= x. The ull hypthesis X = fx g = be deed by (x) = 8 >< >: 2 ;2 fr x 2 S 2 ;3 fr x = x ; 2 ; ; 2 ;3 fr x = x 0 0 fr x 62 S [fx x 0 g (3:44) where it is bvius that (S )=2 ;. The alterative hypthesis X = fx g = be deed by (x) =2 ; (8x 2X ). The, by a simple calculati, we see that the divergece-spectrum f this hypthesis testig csists f three pits lcated at + lg( ; 2; ; 2 ;3 ) ; 2 ; 3 with prbabilities ; 2 ; ; 2 ;3 2 ; 2 ;3, respectively. Therefre, by deiti, the fucti () isgive by () = 8 >< >: + fr < ; 3 3 fr ; 3 < ; 2 fr ; 2 < 0 fr : (3:45) 6

19 Hece, + () isgive by + () = 8 >< >: + fr < ; 3 +3 fr ; 3 < ; 2 + fr fr ; 2 < : (3:46) The, by Therem 2., we have the frmula B e (rjxjjx) = ; fr r> fr 0 <r : (3:47) We bserve here that, i the case f r >, if attaied by = ; 2, i.e., the right-had side f (2.2) is if f + () j () <rg = + ( ) ( ; 2) = ; : I particular, we see that, if r>3, if is t attaied by the budary pit iffj() <rg =;3 f fj() <rg, but bytheiteral pit = ;2. This kid f pheme has ever take place i the previus examples. Als, we shuld tice that frmula (3.47) cat be drive via the stadard rate fucti methd, dierig frm the previus examples, because i this case there des t exist ay (lwer semictiuus) rate fucti. 2 4 Hypthesis Testig ad Large Deviati: bability f Crrect Testig I this secti we ivestigate the prblem f determiig the imum B e (rjxjjx) f achievable expets fr the secd kid f crrect prbability; uder asympttic cstraits f the frm e ;r the rst kid f errr prbability (r>0isa prescribed arbitrary cstat), where is the secd kid f errr prbability. Let us rst give the frmal deitits, where X = fx g =, X = fx g = idicate the ull hypthesis ad the alterative hypthesis, respectively. Deiti 4. A rate E is called r- achievable if there exists a acceptace regi A such that lim if! lg r ad lim sup! lg E: ; Deiti 4.2 (The imum f r-achievable crrect expets) B e (rjxjjx) = if fe j E is r-achievableg : 7

20 The purpse f this secti is t determie B e (rjxjjx) as a fucti f r. T this ed, let us dee the fucti () by () = lim! lg lg (X ) (X ) : (4:) This fucti is the same e as () deed by (2.) i Secti 2, but here we assume that the right-had side f (4.) has the limit. We tice here that () is mte decreasig i, adif>d(xjjx) the () = 0 (cf. Lemma 2.). The reas why we assume the existece f the limit i (4.), the ctrary t i Secti 2, will be made apparet belw frm the prf f Therem 4.. Furthermre, fr sme techical reas, we assume i the sequel the fllwig prperty abutthe ifrmati-spectrum that fr ay cstat M > 0 there exists sme cstat K>0such that lim if! lg ( lg (X ) M: (4:2) ) (X ) K emark 4. This assumpti k meas that the ifrmati-spectrum f X with respect t X des t shift t the right faster tha with ay specied expetial speed f decay, whe teds t +. Fr example, if X, X are statiary memryless surces with ite surce alphabet subject t prbability distributis,, respectively, ad there des t exist a x 2X fr which (x) = 0 ad (x) > 0, the it is evidet that the cditi (4.2) is satised. 2 We w have the fllwig quite geeral frmula, which is a dual cuterpart f Therem 2.: Therem 4. Assume that the limit i (4.) exists ad the cditi (4.2) is satis- ed. The, fr ay r 0, Be (rjxjjx) = if + ()+[r ; ()] + (4:3) where [x] + = max(x 0) ad we have putb e (0jXjjX) =0(r = 0). emark 4.2 Sice it fllws frm Lemma 2. that + ()+[r ; ()] + = if >D(XjjX) if ( + r) >D(XjjX) the if the right-had side is attaied by = D(XjjX). Therefre, if the righthad side f (4.3) may be replaced by if if ()isctiuus at = D(XjjX). D(XjjX) 2 k Oe f the referees suggests the strikig similarity betwee the cditi (4.2) ad the stadard ccept f expetial tightess i large deviati thery (e.g., cf. Demb ad Zeitui [4]). 8

21 f f Therem 4.. ) Direct part: I the prf f the direct part we d t eed the assumti (4.2). First, keep i mid that () i + ()+[r ; ()] + the right-had side f (4.3) is mte decreasig, ad set 0 = if + ()+[r ; ()] + : (4:4) The, there exists a 0 such that 0 is expressed as which we rewrite as 0 = lim "#0 ( 0 + " + ( 0 + ")+[r ; ( 0 + ")] + ) (4:5) 0 = ( 0 + )+[r ; ( 0 + )] + ; () (4:6) where >0is a arbitrarily small cstat ad ()! 0as! 0. We use here the tati that S (a) = x 2X lg (x) (x) a : (4:7) The, sice the existece f the limit i (4.) was assumed, we have e ;((0+)+ ) fx 2 S ( 0 + )ge ;((0+); ) (8 0 ) (4:8) where > 0 is a arbitrarily small cstat. Next, dee a subset C f S ( 0 + ) as fllws if ( 0 + ) r the set C = S ( 0 + ), therwise if ( 0 + ) <rthe set C = T where T is ay subset f S ( 0 + ) suchthat lim! lg fx = r: (4:9) 2 T g It shuld be ted here that it is always pssible t chse such asubsett, because i the case with ( 0 + ) <rwe ca make ( 0 + )+ <rhld with > 0 small eugh, where we may csider a radmized hypthesis testig if ecessary. Nw, csider the hypthesis testig with C as the critical regi. First, we evaluate the value f the rst kid f errr prbablity. I the case with ( 0 + ) r, sice C = S ( 0 + ), by meas f (4.8) we have fx 2C g e ;((0+); ) e ;(r; ) (8 0 ) while i the case with ( 0 + ) <r,by meas f (4.9) we have fx 2C ge ;(r; ) (8 0 ): 9

22 The, i either case, it hlds that fx 2C ge ;(r; ) : (4:0) Therefre, the rst kid f errr prbablity is evaluated as Hece, fx 2C ge ;(r; ) : lim if! Sice > 0 is arbitrary, it is ccluded that lim if! lg r ; : lg r: (4:) Next, we evaluate the value f the secd kid f crrect prbability ;, where is the secd kid f errr prbability. First, we bserve that if x 2 S ( 0 + ) the (x) (x)e ;(0+) (4:2) hlds. The, i the case with ( 0 + ) r, sice C = S( 0 + ), it fllws frm (4.8) that X 2C X = (x) x2c X x2c (x)e ;(0+) = e ;(0+) fx 2 S ( 0 + )g e ;(0++(0+)+ ) (8 0 ): (4.3) Similarly, i the case with ( 0 + ) <r, sice C = T, it fllws frm (4.9) that X 2C e ;( 0++r+ ) Summarizig (4.3) ad (4.4), i either case we have (8 0 ): (4:4) X 2C e ;( 0++( 0+)+[r;( 0+)] + + ) : (4:5) Substituti f (4.6) it (4.5) yields X 2C e ;( 0 + +()) : Hece, ; = X 2C e ;( 0 + +()) 20

23 frm which it fllws that lim sup! lg ; (): (4:6) We tice here that we ca make + ()! 0, because > 0 ad > 0 are bth made arbitrarily small. Thus, by virtue f (4.) ad (4.6) we cclude that ay rate E such thate> 0 is r-ahievable. ) Cverse part: I the prf f the cverse part we eed the assumpti (4.2). First, let K>0 be a cstat large eugh (t be specied belw) ad >0be a arbitrarily small cstat. Puttig L = 2K,we divide the iterval (;K K] it L subitervals with equal width t have I i =(c i ; c i ] (i = 2 L) where c i K ; (i ; ). Accrdig t this iterval partiti, divide the set it the L subsets S (i) = T = x 2X ;K< lg (x) (x) K x 2X lg (x) (x) 2 I i (i = 2 L): This perati is called the ifrmati-spectrum slicig. Mrever, we dee S (0) = x 2X lg (x) (x) ;K S (;) = x 2X lg (x) (x) >K where it is bvius that X S L = j=; S(j) : Suppse that E is r-achievable, i.e., that there exists a critical regi C such that The, frm (4.7) we have lim if! lim sup! where > 0 is a arbitrarily small cstat. X 2C, let us rst evaluate the value f lg r (4:7) lg E: (4:8) ; e ;(r; ) (8 0 ) (4:9) X 2C (i) I rder t evaluate the value f (i = 2 L) 2

24 where C (i) S (i) \C (i = ; 0 2 L). Wewevaluate the value f (i = 2 L)itw ways as fllws. First, we bserve that X 2C (i) which tgether with (4.9) yields fx 2C g = X 2C (i) Next, by the deitis f (c i ) ad S (i),we see that Hece, fx 2 S (i) g X 2C (i) X 2C (i) e ;(r; ) : (4:20) lg (X ) (X ) c i e ;((ci); ) (8 0 ): X 2 S (i) e ;((ci); ) : (4.2) A csequece f (4.20) ad (4.2) is X 2C (i) e ;((c i)+[r;(c i)] + ; ) (i = 2 L): (4:22) We ca w evaluate the value f x 2 S (i) X 2C (i) as fllws. Sice x 2C (i) (i = 2 L) ad hece als P (x) (x)e ;(ci;),wehave X 2C X (i) = (x) X x2c (i) x2c (i) = e ;(ci;) (x)e ;(ci;) X 2C (i) implies e ;(ci+(ci)+[r;(ci)]+ ;; ) (4.23) fr i = 2 L, where we have used (4.22) i the last iequality. Furthermre, let us evaluate the values f X 2 S (;) implies (x) (x)e ;K,we btai X 2 S (;) = ad X X x2s (;) x2s (;) (x) X 2 S (0) (x)e ;K. Sice x 2 S (;) e ;K : (4.24) 22

25 ecallig here that X 2 S (0) = = ( ( ) ) (X ) ;K lg (X ) ) (X ) K lg (X : ad tig the assumpti (4.2), we see that fr ay M > 0 there exists a K > 0 large eugh such that X 2 S (0) Summarizig up (4.23)(4.25), we have ; = X 2C = LX i= e ;(M; ) LX i=; X 2C (i) (8 0 ): (4:25) e ;(ci+(ci)+[r;(ci)]+ ;; ) + e ;K + e ;(M; ) : (4.26) O the ther had, sice, by the deiti (4.4) f 0, c i + (c i )+[r ; (c i )] + 0 (i = 2 L) it fllws frm (4.26) that ; Le ;( 0;; ) + e ;K + e ;(M; ) : Thus, if we take M>0 ad K>0 large eugh, the lim sup! lg ; 0 ; ; : (4:27) Therefre, E 0 ; ; hlds, wig t (4.8), (4.27). Sice bth f > 0 ad > 0 are arbitrary, we ca let! 0,! 0tgetE 0. Thus, it is ccluded that ay r-achievable rate E cat be smaller tha Examples I this secti we demstrate several typical applicatis f Therem 4.. This is t verify the ptetialities f Therem

26 Example 5. As i Example 3., let us csider the hypthesis testig with statiary irreducible Markv surces X, X with ite surce alphabet. With the same tati as i Example 3., it fllws als here with Sav therem that (3.) ad (3.2) hld, i.e., () =0fr D(PjjPjp) ad, fr D(PjjPjp), () = D(P jjpjp ) + () = D(P jjpjp ) ad s, by Therem 4. we have Be (rjxjjx) = if D(P jjpjp )+[r; D(P jjpjp )] + : (5:) It is easy t check that, if r<d(pjjpjp) (p is the statiary distriibuti fr P )the Be (rjxjjx) = 0, whereas if r D(P jjpjp) theif the right-had side f (5.) is attaied by a such that D(P jjpjp ) r ad hece i this latter case we have Be (rjxjjx) = if D(P jjpjp )+r; D(P jjpjp ) :D(P jjp jp )r = if Q2P 0:D(QjjP jq)r D(QjjPjq)+r ; D(QjjPjq) : (5.2) This frmula has bee drive by Nakagawa ad Kaaya [6]. Here, Be (rjxjjx) =0 wheever r D(P jjpjp). Let us csider the special case where surces X, X are bth statiary memryless subject t prbability distributis P, P X, respectively. The, i the case f r D(PjjP ), frmula (5.2) reduces t B e (rjxjjx) = if Q:D(QjjP )r D(QjjP )+r ; D(QjjP ) : (5:3) This frmula has rst bee established by Ha ad Kbayashi [5] based the methd f types. O the ther had, wehave B e (rjxjjx) = 0 wheever r<d(pjjp ). 2 Example 5.2 I rder t geeralize Example 5., as i Example 3.2 f x2 let us csider the hypthesis testig with uilar ite-state surces X, X as the ull ad alterative hyptheses, respectively. With the same tati as i Example 3.2, Therem 4. tgether with (3.6) ad (3.7) gives the frmula fr the hypthesis testig X agaist X: Be (rjxjjx) = if D(S jjpjp S )+[r; D (S jjpjp S )] + = if D(S jjpjp S )+[r; D (S jjpjp S )] + : (5.4) S2V 0 24

27 2 Example 5.3 Let the surce alphabet X be ite, ad, as i Example 3.3, let us csider the hypthesis testig with a mixed surce X = fx g = as the ull hypthesis ad a statiary memryless surce X = fx g = subject t prbability distributi P as the alterative hypthesis. I rder t satisfy the assumpti (4.2), let P (x) > 0, P 2 (x) > 0(8x 2X). Here, recall that the mixed surce X = fx g = was deed as (x) = (x)+ 2 2 (x) (8x 2X ) (5:5) where X = fx g X = 2 = fx 2 g = are statiary memryless surces subject t prbability distributis P P 2, respectively. Dee 2, () (2) as i (3.2) (3.24) f Example 3.3, ad similarly, let the prjectis f P P 2 \ () 2 \ (2) be deted by P (), P (2), respectively. The, applicati f Sav therem gives () = mi(d(p () jjp ) D(P (2) jjp 2 )) (5:6) frm which we see that if mi(d(p jjp ) D(P 2 jjp )) the () =0. Fially, by substitutig () f (5.6) it the right-had side f (4.3) i Therem 4. we have the cmputable frmula fr B e (rjxjjx) as a fucti f r. 2 emark 5. Ufrtuately, fr B e (rjxjjx) that we are csiderig here, such a simple frmula fr mixed surces as (3.28) i emark 3. des t hld. 2 emark 5.2 S far, we have csidered ly the case with ite surce alphabet X where Sav therem played the key rle. O the ther had, i the case f geeral statiary memryless surces with cutably iite r abstract surce alphabet X, Sav therem des t ecessarily hld. Hwever, sice Cramer therem always wrks, we ca ivke Cramer therem, istead f Sav therem, i rder t cmpute the value f B e (rjxjjx), whe X, X are bth statiary memryless surces. The, it suces t use the same rate fucti I(x) as specied i (3.30) f Example 3.4, i.e., () = if I(x): (5:7) x With the same tati as i Example 3.4, we see that we ca write the right-had side f (5.7) i terms f divergeces (with Sav therem) ly whe 2D. 2 Example 5.4 Let us csider the hypthesis testig with statiary memryless Gaussia surces X = fp g, X = fp g as i Example 3.5. Sice () ad + () are give by (3.42), (3.43), substituti f these (3.42), (3.43) it (4.3) i Therem 4. ad sme simple calculati yield the frmula B e (rjxjjx) =( p r ; p a) 2 [r a] (5:8) 25

28 where a = D(P jjp ). We itce here that the fucti (5.8) is symmetric t the fucti B e (rjxjjx) i Example 3.5 with respect t the y-axis. The frmula (5.8) tells us that B e (rjxjjx) is a mte icreasig fucti f r, ad that B e (rjxjjx) =0 wheever r a. 2 6 Geeralized Hypthesis Testig S far, we have studied the hypthesis testig prblem with geeral surces X = fx g =, X = fx g = as ull ad alterative hyptheses, respectively. Hwever, it is easy t bserve that Therem 2. ad Therem 4. i the previus sectis ctiue t be valid as they are, eve if we replace the prbability distributi f the alterative hypthesis by ay egative measure G with G ( ) = 0 (t ecessarily a prbability measure), where the secd kid f errr prbability fx 2A g shuld be iterpreted i tur as detig the value f the egative measure G (A ). This is called the geeralized hypthesis testig. The, if we dee G (X ) ( = 2 ) lim sup! lg G (X ) Therem 4. is meaigful ly whe < +, where B e (0jXjjX) = 0 i Therem 4. eeds t be replaced by B e (0jXjjX) =, ad ; i Deiti 4. eeds t be replaced by ;. As examples f such egative measures G ( = 2 ), we may csider G (x) =(8x 2X 8 = 2 )withcutably iite surce alphabet X (called the cutig measure X )rthe-dimesial Lebesgue measure with real surce alphabet X. I particular, the case f the cutig measure has the deep structural relatiship with the xed-legth surce cdig prblem, which will be elucidated i the ext secti. emark 6. As will be easily see frm the prfs, eve if we i tur replace the prbability measure f the ull hypthesis by egative measures F with F ( ) = 0, bth f Therem 2. ad Therem 4. ctiue t hld with the due reiterpretati fr prbabilities as abve. 2 7 Hypthesis Testig ad Fixed-Legth Surce Cdig Thus far, we have shw tw key therems (Therem 2. ad Therem 4.) ccerig the geeral hypthesis testig. I this geeral settig, we ca shw als may ther elegat systematic results the hypthesis testig (as fr the details, refer t Ha 26

29 [8]). I parallel with these systematic results, the crrespdig may results i the geeral xed-legth surce cdig prblem have bee established (cf. Ha [8, 20]). This crrespdece is f very itrisic ature t ly at the techical level but als at the cceptual level, which ca be made very trasparet byitrducig the geeralized hypthesis testig prblem as abve. Frm this pit f view, it turs ut that all the therems that hld i the xed-legth surce cdig prblem ca be regarded as frmig a special class f thse hldig i the geeralized hypthesis testig prblem. As a illustrative case, we will shw that Therem 2. f Ha [20] immediately fllws as a special case f Therem 2. (i Secti 2) with the cutig measure C (x) (8x 2X ) as the alterative hypthesis. T shw this, let us rst state the frmal deiti f the geeral xed-legth surce cdig prblem. Let X = fx g =g be ay geeral surce with cutably iite surce alphabet X, ad let M f 2 M g be a iteger set. The, mappigs ' : X!M, : M! X are called the ecder ad the decder, where we call " fx 6= (' (X ))g the errr prbability f the xed-legth surce cdig. We dete the pair (' ) with the errr prbability " by( M " ) (called a cde). I the xed-legth surce cdig prblem, we areiterested i the prlem f determiig the imum e (rjx) f achievable rates uder asympttic cstraits f the frm " e ;r (r > 0isa prescribed cstat) the errr prbability ". Frmally, we dee as fllws. Deiti 7. is called r-achievable if there exists a cde ( M " )such that lim if! lim sup! lg " r lg M : Deiti 7.2 (The imum f r-achievable xed-legth cdig rates) Deiti 7.3 e (rjx) = if f j is r-achievableg : () = lim if! lg : (7:) lg (X ) With these deitis, the fllwig geeral therem has bee established based the etrpy-spectrum argumet which is a dieret versi f the ifrmatispectrum demstrated i this paper. Therem 7. (Ha [8, 20]) Let X = fx g = be a geeral surce with cutably iite alphabet X,thefray r 0wehave e (rjx) = sup f ; () j () <rg (7:2) 0 where e (0jX) =0(r =0). 2 27

30 Let us w shw that Therem 7. directly fllws just by rewritig Therem 2. with the cutig measure C (x) (8x 2X ) as the alterative hypthesis. Let this alterative hypthesis be deted by C = fc g =. First, whe we are give a acceptace regi A X fr a hypthesis testig, set M = ja j ad we csider the ecder ' : X! M such that ' maps i the e-t-e maer all the elemets f A it M i the rder f 2, ad maps all the elemets f A c it 2M, where the decder : M!X is the iverse mappig f ' j A. The, it is bvius that A = fx 2X j (' (x)) = xg which meas that the rst kid f errr prbability =fx =2A g fr the hypthesis testig cicides with the errr prbability " fr the xed-legth surce cdig. We tice that this kid f crrespdece betwee hypthesis testigs ad xed-legth surce cdigs becmes the e-t-e mappig if we idieretly idetify all the cdes ( M " ) which have the same set A = fx 2X j (' (x)) = xg f the elemets f x 2X that ca be crrectly decded uder xed-legth surce cdig. O the ther had, the secd kid f errr prbability uder the cutig measure C ca be writte as where = C (A )=ja j = M = e r (7.3) r = lg M : The, uder this crrespdece it fllws frm (7.3) that lim if! lg = ; lim sup r! which meas that is r-achievable fr (geeralized) hypthesis testig if ad ly if ; is r-achievable fr xed-legth surce cdig. Thus, frm Deiti 7. Deiti 7.2 ad Deiti 2. Deiti 2.2, we have the fllwig equati cectig B e (rjxjjc) t e (rjx): B e (rjxjjc) =; e (rjx) (8r >0): (7:4) Next, sice we are csiderig the cutig measure C as the alterative hypthesis, the prbability appearig the righ-had side f (2.) deig () is writte as lg (X ) (X ) = lg (X ) C (X ) = lg (X ) = lg (X ) ; : 28

31 Therefre, we have () = (;) by the deiti (7.) f (), i.e., () =(;): (7:5) The, Therem 2. with the cutig measure as the alterative hypthesis tgether with (7.4) yields e (rjx) = ;B e (rjxjjc) = ; if f + () j () <rg = sup f; ; () j () <rg : As a csequece, if we replace by ; ad use (7.5), it is ccluded that e (rjx) = sup f ; () j () <rg : 0 This is thig but Therem 7. the xed-legth surce cdig. efereces [] T.S. Ha ad S. Verdu, \Apprximati thery f utput statistics," IEEE Trasactis Ifrmati Thery, vl.it-39,.3, pp , 993 [2] T. S. Ha, Basic Csideratis Large Deviati Therems, IS Techical eprts UEC-IS-998-4, Graduate Schl f Ifrmati Systems, Uiversity f Electr-Cmmuicatis, Chfu, Tky, , Japa, Octber 998 (i Japaese) [3] J. A. Bucklew, Large Deviati Techiques i Decisi, Simulati ad Estimati, Jh Wiley & Ss, New Yrk, 990 [4] A. Demb ad O. Zeitui, Large Deviatis Techiques ad Applicatis, Jes ad Bartlett Publishers, Bst, 993 [5] S. Verdu ad T.S. Ha, \A geeral frmula fr chael capacity," IEEE Trasactis Ifrmati Thery, vl.it-40,.4, pp.47-57, 994 [6] I. Csiszar ad J. Krer, Ifrmati Thery: Cdig Therems fr Discrete Memryless Systems, Academic ess, New Yrk, 98 [7] A.. Barr, \The strg ergdic therem fr desities: geeralized Sha- McMilla-Breima therem," Aals f bability, vl.3,.4, pp , 985 [8]. E. Blahut, iciples ad actice f Ifrmati Thery, Addis-Wesley, Massachusetts,

32 [9] T. M. Cver ad J. A. Thmas, Elemets f Ifrmati Thery, Wiley, New Yrk, 99 [0]. G. Gallager, Ifrmati Thery ad eliable Cmmuicati, Jh Wiley & Ss, New Yrk, 968 [] J. Neyma ad E. S. Pears, \O the prblem f the mst eciet tests f statistical hyptheses," Phil. Tras. yal Sc. Ld, Series A, vl.23, pp , 933 [2] P. Billigsley, bability ad Measure, 3rd ed., Jh Wiley & Ss, New Yrk, 995 [3] W. Hedig, \Asympttically ptimal test fr multimial distributis," Aals f Mathematical Statistics, vl.36, pp , 965 [4] S. Nataraja, \Large deviatis, hyptheses testig, ad surce cdig fr ite Markv chais," IEEE Trasactis Ifrmati Thery, vl.it-3,.3, pp , 985 [5] T. S. Ha ad K. Kbayashi, \The strg cverse therem fr hypthesis testig," IEEE Trasactis Ifrmati Thery, vl.it-35,., pp.78-80, 989 [6] K. Nakagawa ad F. Kaaya, \O the cverse therem i statistical hypthesis testig fr Markv chais," IEEE Trasactis Ifrmati Thery, vl.it- 39,.2, pp , 993 [7] T. S. Ha, \A ifrmati-spectrum apprach t surce cdig therems with a delity criteri," IEEE Trasactis Ifrmati Thery, vl.it-43,.4, pp.45-64, 997 [8] T. S. Ha, Ifrmati-Spectrum Methds i Ifrmati Thery, Baifuka- ess, Tky, 998 (i Japaese). [9] T. S. Ha, \A ifrmati-spectrum apprach t capacity therems fr the geeral multiple-access chael," IEEE Trasactis Ifrmati Thery, vl.it-44,.7, pp , 998 [20] T. S. Ha, \The reliability fuctis f the geeral surce with xed-legth cdig," IEEE Trasactis Ifrmati Thery, vl.it-46,.5,