Journal Afrka Statstka Vol. 6, 2011, pages 307 315. DOI: http://dx.do.org/10.4314/afst.v61.1 Journal Afrka Statstka ISS 0825-0305 On error probablty exponents of many hypotheses optmal testng llustratons Leader avae Payame oor Unversty, 19395-4697, Tehran, I.R. of Iran Receved 28 January 2011; Accepted 23 September 2011 Copyrght c 2011, Journal Afrka Statstka. All rghts reserved Abstract. In ths paper we study a model of hypotheses testng consstng of wth to smple homogeneous statonary Markov chans th fnte number of states such that havng dfferent dstrbutons from four possble transmsson probabltes.for solvng ths problem we apply the method of type and large devaton technques (LTD). The case of two objects havng dfferent dstrbutons from to gven probablty dstrbuton as examned by Ahlswedeh and Haroutunan. Résumé. Dans cet artcle nous étudons un modèle de tests d hypothèses composé de deux chanes de Markov statonnares homogènes et smples avec un nombre fn d états ayant dfférentes dstrbutons parm quatre probabltés de transton possbles. Pour résoudre ce problème, nous applquons la méthode des types et des technques de grandes devatons. Le cas de deux objets ayant dfférentes dstrbutons ssues d une dstrbuton de probablté donnée, a été examné par Ahlswedeh et Haroutunan. Key words: Markov chans; Error probabltes; Dfferent dstrbutons; Transton probabltes; Relabltes. AMS 2010 Mathematcs Subject Classfcaton : 62P30; 62M02. 1. Introducton Applcatons of nformaton-theoretcal methods n mathematcal statstcs are reflected n the monographs by Kullback [10], Csszár and Körner [4], Blahut [2], Csszár and Shelds [5], Zetoun and Gutman [14]. In the book of Csszár and Shelds [5] dfferent asymptotc aspects of two hypotheses testng for ndependent dentcally dstrbuted observatons are consdered va theory of large devatons. Smlar problems for Markov dependence of experments were nvestgated by atarajan [13], Haroutunan [7], [8], Haroutunan and avae [9] and others. Ahlswede and Haroutunan n [1] formulated an ensemble of problems on multple hypotheses testng for many objects and on dentfcaton of hypotheses under relablty requrement. Leader avae: leadernavae@yahoo.com
of many hypotheses optmal testng llustratons 308 The problem of many (L > 2) hypotheses testng on dstrbutons of ndependent observatons s studed n [13], [11] va large devatons technques. In ths paper we nvestgate a model wth two smple homogeneous statonary Markov chans wth fnte number of states such that havng dfferent dstrbutons from four possble transton probabltes. In Secton 2 we ntroduce the concept of Markov chan and the method of type [3] and n Secton 3, we apply the result Secton 2 for hypotheses testng. 2. Prelmnares Let y = (y 0, y 1, y 2,..., y ), y n Y = {1, 2,..., I}, y Y +1, = 0, 1, 2,..., be a vectors of observatons of a smple homogeneous statonary Markov chan wth fnte number I of states. The l = 1, L hypotheses concern the rreducble matrces of the transton probabltes P l = {P l (j ), = 1, I, j = 1, I}, l = 1, L. The statonarty of the chan provdes exstence for each l = 1, L of the unque statonary dstrbuton Q l = {Q l (), = 1, I}, such that Q l ()P l (j ) = Q l (j), We defne the jont dstrbutons Q l () = 1, = 1, I, j = 1, I. Q l P l = {Q l ()P l (j ), = 1, I, j = 1, I}, l = 1, L. Let us denote D(Q P Q l P l ) Kullback-Lebler dvergence D(Q P Q l P l ) =,j Q()P (j )[log Q()P (j ) log Q l ()P l (j )] of the dstrbuton = D(Q Q l ) + D(Q P Q P l ), wth respect to dstrbuton Q l P l where Q P = {Q()P (j ), = 1, I, j = 1, I}, D(Q Q l ) = Q()[log Q() log Q l ()], l = 1, L. Let us name the second order type of vector y the square matrx of I 2 relatve frequences {(, j) 1, = 1, I, j = 1, I} of the smultaneous appearance n y of the states and j on the pars of neghbor places. It s clear that j (, j) =. Denote by T Q P the set of vectors from Y +1 whch have the second order type such that for some jont PD Q P (, j) = Q()P (j ), = 1, I, j = 1, I. The set of all jont PD Q P on Y s denoted by Q P(Y) and the set of all possble the second order types for jont PD Q P s denoted by Q P (Y). ote that f vector y T Q P, then
of many hypotheses optmal testng llustratons 309 (, j) = Q(), = 1, I, j (, j) = Q (j), j = 1, I, for somewhat dfferent from PD Q, but n accordance wth the defnton of (, j) we have Q() Q () 1, = 1, I, and then n the lmt, when, the dstrbuton Q concdes wth Q and may be taken as statonary for condtonal PD P : Q()P (j ) = Q(j), j Y. The probablty of vector y Y +1 of the Markov chan wth transton probabltes P l and statonary dstrbuton Q l, s the followng Q l P l (y) Q l (y 0 ) P l (y n y n 1 ), l = 1, I, n=1 Q l Pl (A) Q l Pl (y), A Y +1. y A ote also that f Q P s absolutely contnuous relatve to Q l P l, then from [3],[7] we have where Q l Pl (TQ P ) = exp{ (D(Q P Q P l )) + o(1)}, o(1) = max(max 1 log Q l () : Q l () > 0), (max 1 log Q l () : Q l () > 0) 0, when. and also accordng [5],[8] ths s not dffcult to verfy takng nto account that the number TQ P of vectors n T Q P s equal to exp{ (,j Q()P (j ) log P (j )) + o(1)}. In the next secton we use the results of ths secton for the case of L = 12 Hypotheses LAO testng. 3. Problem Statement and Formulaton of Results Let Y 1 and Y 2 be random varables (RV) takng values n the same fnte set Y wth one of L = 4 PDs. Let (y 1, y 2 ) = ((y 1 0, y 2 0),..., (y 1 n, y 2 n),..., (y 1, y2 )), y Y, = 1, 2, n = 0,, be a sequence of results of + 1 ndependent observatons of a smple homogeneses statonary Markov chan wth fnte number I of states. The goal of the statstcan s to defne whch
of many hypotheses optmal testng llustratons 310 jont of dstrbutons corresponds to observed sample (y 1, y 2 ), whch we denote by ϕ. For ths model the vector (Y 1, Y 2 ) can have one of sx jont probablty dstrbutons Q l 1,l 2 P l 1,l 2 (y 1, y 2 ), l 1 l 2, l 1, l 2 = 1, 4 where Q l 1,l 2 l 1,l 2 (y 1, y 2 ) = Q l 1 l 1 (y 1 )Q l 2 l 2 (y 2 ). We can take (Y 1, Y 2 ) = X, Y Y = X and x = (x 0, x 1, x 2,..., x ), x n X, x X +1, where x n = (y 1 n, y 2 n), n = 0,, then we wll have sx new hypotheses for one object. Q 1,2 P 1,2(y 1, y 2 ) = Q 1 P 1 (x), Q 1,3 P 1,3(y 1, y 2 ) = Q 2 P 2 (x), Q 1,4 P 1,4(y 1, y 2 ) = Q 3 P 3 (x), Q 2,1 P 2,1(y 1, y 2 ) = Q 4 P 4 (x), Q 2,3 P 2,3(y 1, y 2 ) = Q 5 P 5 (x), Q 2,4 P 2,4(y 1, y 2 ) = Q 6 P 6 (x), Q 3,1 P 3,1(y 1, y 2 ) = Q 7 P 7 (x), Q 3,2 P 3,2(y 1, y 2 ) = Q 8 P 8 (x), Q 3,4 P 3,4(y 1, y 2 ) = Q 9 P 9 (x), Q 4,1 P 4,1(y 1, y 2 ) = Q 10 P 10 (x), Q 4,2 P 4,2(y 1, y 2 ) = Q 11 P 11 (x), Q 4,3 P 4,3(y 1, y 2 ) = Q 12 P 12 (x), and thus we have brought the orgnal problem to the dentfcaton problem for one object of observaton of Markov chan wth fnte number of states wth L = 12 hypotheses. ow, accordng non-randomzed test ϕ (x) accepts one of the hypotheses H l, l = 1, 12 on the bass of the trajectory x = (x 0, x 1,..., x ) of the + 1 observatons. Let us denote α () l m (ϕ ) the probablty to accept the hypothess H l under the condton that H m, m l, s true. For l = m we denote α () m m (ϕ ) the probablty to reject the hypothess H m. It s clear that α () m m (ϕ ) = α () l m (ϕ ), m = 1, 12. (1) l m Ths probablty s called the error probablty of the m-th knd of the test ϕ. The quadratc matrx of 144 error probabltes {α () l m (ϕ), m = 1, 12, l = 1, 12} sometmes s called the power of the tests. To every trajectory x the test ϕ puts n correspondence one from 6 hypotheses. So the space X +1 wll be dvded nto 12 parts, Gl = {x, ϕ (x) = l}, l = 1, 12, and αl m (ϕ ) = Q m P m (Gl ), m, l = 1, 12. We study the matrx of relabltes, E l m (ϕ) = lm 1 log α l m(ϕ ), m, l = 1, 12. (2) ote that from defntons (1) and (2) t follows that E m m = mn l m E l m. (3)
of many hypotheses optmal testng llustratons 311 E 1 1... E 1 m... E 1 12... E(ϕ) = E l 1... E l m... E l 12.... E 12 1... E 12 m... E 12 12 Defnton 1. The test sequence Φ = (ϕ 1, ϕ 2,...) s called LAO f for gven famly of postve numbers E 1 1, E 2 2,..., E 11 11, the relablty matrx contans n the dagonal these numbers and the remaned 133 ts components take the maxmal possble values. Let P = {P (j )} be a rreducble matrx of transton probabltes of some statonary Markov chan wth the same set X of states, and Q = {Q(), = 1, I} be the correspondng statonary PD. For gven famly of postve numbers E 1 1, E 2 2,..., E 11 11, let us defne the decson rule ϕ by the sets R l {Q P : D(Q P Q P l ) E l l, D(Q Q l ) < }, l = 1, 11, (4) R 6 {Q P : D(Q P Q P l ) > E l l, l = 1, 11}, and ntroduce the functons: E l m (E l l) = R l R l Q P (X ), l = 1, 12. E l l (E l l) E l l, l = 1, 11, nf D(Q P Q P m ), m = 1, 12, l m, l = 1, 11, (5) Q P R l and E 12 m (E 1 1,..., E 11 11 ) nf D(Q P Q P m), m = 1, 11, Q P R 12 E12 12 (E 1 1,..., E 11 11 ) mn El 12. l=1,11 We cte the statement of the general case of large devaton result for types by atarajan [13]. Theorem 1. : Let X = {1, 2,..., I} be a dscrete topologcal space of fnte set of the states of the statonary Markov chan possessng an rreducble transton matrx P and (X, A) be a measurable space such that A be a nonempty and open subset or convex subset of jont dstrbutons Q P and Q m s statonary dstrbuton for P m, then for the type Q P (x) of a vector x from Q m P m on X : lm 1 log Q m Pm {x : Q P (x) A} = nf D(Q P Q P m). Q P A
of many hypotheses optmal testng llustratons 312 In ths secton we use the followng lemma. Lemma 1. :If elements E m l (φ ), m, l = 1, 12, = 1, 2, are strctly postve, then the followng equaltes hold for Φ = (φ 1, φ 2 ) : E m1,m 2 l 1,l 2 (Φ) = E m1 l 1 (φ 1 ) + E m2 l 2 (φ 2 ), f m 1 l 1, m 2 l 2, (a) E m1,m 2 l 1,l 2 (Φ) = E m l (φ ), f m 3 = l 3 m l, = 1, 2. (b) Proof : From the ndependence of the objects we can wrte: α m 1,m 2 l 1,l 2 (Φ ) = α m1 l 1 (φ 1 )α m2 l 2 (φ 2 ), f m 1 l 1, m 2 l 2, (c) α m 1,m 2 l 1,l 2 (Φ ) = α m l (φ )[1 α m3 l 3 ](φ 3 ), f m 3 l 3, m l (d) Accordng to the defntons (1) and (2) we obtan (a) and (b) from equaltes (c) and (d). otce that usng Lemma 1, for jont probablty dstrbutons D(Q l 1,l 2 l 1,l 2 Q m 1,m 2 ) and defnton of α l m (ϕ ) = Q m P m (G l ), m, l = 1, 12, t s clear that: When m, l = 1, 4, = 1, 2, m 1 m 2, l 1 l 2, we have D(Q l 1,l 2 l 1,l 2 Q m 1,m 2 m 1,m 2 ) = D(Q (1) l 1 (1) l 1 Q (1) m 1 (1) m 1 ) + D(Q (2) l 2 (2) l 2 Q (2) m 2 (2) m 2 ), and for m l, m 3 = l 3, = 1, 2, For example D(Q l 1,l 2 l 1,l 2 Q m 1,m 2 m 1,m 2 ) = D(Q () l () l Q () m () m ). D(Q 1,2 1,2 Q 4,2 4,2) = D(Q (1) 1 (1) 1 Q (1) 4 (1) 4 ). ow we formulate the theorem from [9], whch we prove by applcaton of Theorem 1. Theorem 2. Let X be a fxed fnte set, and P 1,, P 12 be a famly of dstnct dstrbutons of a Markov chan. Consder the followng condtons for postve fnte numbers E 1 1,, E 11 11 : 0 < E 1 1 < mn[d(q m P m Q m P 1 ), m = 2, 12], (6) 0 < E l l < mn[mn E l m (E m m) m=1,l 1, mn D(Q m P m Q m P l ) m=l+1,12 ], Two followng statements hold: l = 2, 11.
of many hypotheses optmal testng llustratons 313 a). f condtons (6) are verfed, { then} here exsts a LAO sequence of tests ϕ, the relablty matrx of whch E = El m (ϕ ) s defned n (5), and all elements of t are postve, b). even f one of condtons (6) s volated, then the relablty matrx of an arbtrary test havng n dagonal numbers E 1 1,, E 11 11 necessarly has an element equal to zero (the correspondng error probablty does not tend exponentally to zero). Proof: Frst we remark that D(Q P l Q P m ) > 0, for l m, because all measures P l, l = 1, 12, are dstnct. Let us prove the statement a) of the Theorem 2 about the exstence of the sequence correspondng to a gven E 1 1,, E 11 11 satsfyng condton (6). Consder the followng sequence of tests ϕ gven by the sets B l = Q P R l T Q P (x), l = 1, 12. (7) otce that on account of condton (6) and the contnuty of dvergence D for large enough the sets R l, l = 1, 12 from (4) are not empty. The sets B l, l = 1, 12, satsfy condtons : B l B m =, l m, 12 Bl = X. ow let us show that, exponent E l m (ϕ ) for sequence of tests ϕ defned n (7) s equal to E l m. We know from (4) that R l, l = 1, 11, are convex subset and R 12 s open subset of the decson rule of ϕ, therefore R l, l = 1, 12, satsfy n condton of Theorem 1. Wth relatons (4), (5), by Theorem 1 we have l=1 lm 1 log α l m (ϕ ) = lm 1 log Q m Pm (R l ) = nf D(Q P Q P m ). (8) Q P R l ow usng (2) and (8) we can wrte E l m (ϕ ) = nf D(Q P Q P m ) m, l = 1, 12. (9) Q P R l Usng (8), (4) and (5) we can see that all El m are strctly postve. The proof of part (a) wll be fnshed f one demonstrates that the sequence of the tests ϕ s LAO, that s for gven fnte E 1 1,, E 11 11 for any other sequence of tests ϕ E l m (ϕ ) E l m (ϕ ), m, l = 1, 12. Let us consder another sequence of tests ϕ, whch s defned by the sets G 1,, G 12 such that Ths condton s equvalent to the nequalty E l m (ϕ ) E l m (ϕ ), m, l = 1, 12. α l m (ϕ ) α l m (ϕ ). (10)
of many hypotheses optmal testng llustratons 314 We examne the sets Gl case B l, l = 1, 11. Ths ntersecton can not be empty, because n that α () l l (ϕ ) = Q l Pl (Gl ) Q l Pl (Bl ) max Q l P () l (TQ P (x)) exp{ (E l l + o(1))} Q P :D(Q P Q l P l ) E l l Let us show that Gl B m =, l = 1, 11. If there exsts Q P such that D(Q P Q l P l ) E l l and TQ P (x) G l, then α () l m (ϕ ) = Q m P m (G l ) > Q m P m (T Q P (x)) exp{ (E m m + o(1))} When 0 G l T Q P (x) TQ P (x), we also obtan that α () l m (ϕ ) = Q m P m (G l ) > Q m P m (G l T Q P (x)) exp{ (E m m + o(1))} Thus t follows f a). l < m from (6)we obtan that E l m (ϕ ) E m m < El m (ϕ ). b). l > m then E l m (ϕ ) E m m < El m (ϕ ), whch contradcts our assumpton. Hence we obtan that Gl B l = Bl, l = 1, 11. The followng ntersecton G12 B 12 = B12 s empty too, because otherwse α 12 l (ϕ ) α 12 l (ϕ ), whch contradcts to (10), n ths case G l = B l, l = 1, 12. Accordng the prevous explanng the statement of part b) of theorem s evdent, snce the volaton of one of the condtons (8) reduces to the equalty to zero of a least one of the elements El m defned n (5). References [1] Ahlswede, R. F. and Haroutunan, E. A., 2006. Testng of hypotheses and dentfcaton. Lecture otes n Computer Scence, vol. 4123. General Theory of Informaton Transfer and Combnatons, Sprnger, pp. 462-478. [2] Blahut, R. E., 1987. Prncple and Practce of Informaton Theory, readng, MA, Addson-wesley. [3] Csszár, I., 1998. Method of types, IEEE Transacton on Informaton Theory, vol. (44).. 6. pp. 2505-2523. [4] Csszár, I. and Körner, J., 1981. Informaton Theory: Codng Theorem for Dscrete Memoryless Systems, Academc press, ewyork. [5] Csszár, I. and Shelds, P., 2004. Informaton Theory and Statstcs. Fundementals and Trends n Communcatons and Informaton Theory, vol. (1),. 4. [6] Dembo, A. and Zetoun, O., 1993. Large Devatons Technques and Applcatons, Jons and Bartlet. Publshers, London.
of many hypotheses optmal testng llustratons 315 [7] Haroutunan, E. A., 1988. On asymptotcally optmal testng of hypotheses concernng Markov chan, (n Russan). Izvesta Acad. auk Armenan SSR. Sera Mathem. vol. 22,. 1. pp. 76-80. [8] Haroutunan, E. A, Haroutunan, M. E. and Harutyunyan, A.., 2008. Relablty Crtera n Informaton Theory and n Statstcal Hypothess Testng, ow Publshers Inc. [9] Haroutunan E. A. and avae, L., 2009. On statstcal dentfcaton of Markov chan dstrbuton subject to the relablty crteran, Mathematcal Problems of Computer Scence, vol. 32, pp. 65-69. [10] Kullback S., 1959. Informaton Theory and Statstcs, Wley, ew York. [11] avae, L., 2007. Large devatons technques for error exponents to many hypotheses LAO testng, Journal of Modern Appled Statstcal Methods, USA, vol. 6,. 3. pp. 487-491. [12] avae, L., 2010. Identfcaton of Optmalty of Multple Hypotheses Testng, Lambert Academc Publshng, Germany. [13] atarajan, S., 1985. Large devatons, hypotheses testng, and source codng for fnte Markov chan, IEEE Transacton on Informaton Theory, vol. 31,. 3, pp. 360-365. [14] Zetoun, O. and Gutman, M., 1991. On unversal hypotheses testng va large devatons. IEEE Transacton on Informaton Theory, vol. 37,. 2, pp.285-290.