013 IEEE International Symosium on Information Theory Interactive Hyothesis Testing Against Indeendence Yu Xiang and Young-Han Kim Deartment of Electrical and Comuter Engineering University of California, San Diego La Jolla, CA 9093, USA Email: {yxiang,yhk}@ucsd.edu Abstract A hyothesis testing roblem with communication constraints is studied, in which two nodes interactively communicate with each other in q rounds to erform a simle binary hyothesis testing on whether the observed random sequences at the nodes are generated indeendently or not. The otimal tradeoff between the communication rates in q rounds interaction and the testing erformance measured by the tye II error exonent such that the tye I error robability asymtotically vanishes. An examle is rovided that shows that a two-way test outerforms the otimal one-way test and thus that interaction hels for hyothesis testing. I. INTRODUCTION Berger [1], in an insiring attemt at combining information theory and statistical inference, formulated the roblem of hyothesis testing with communication constraints as deicted in Fig. 1. Let X1 n,x n n,x x 1i,x i beaair of indeendent and identically distributed i.i.d. n-sequences generated by a two-comonent discrete memoryless source - DMS,X. Suose that there are two hyotheses on the joint distribution of,x, namely, H 0 :,X 0 x 1,x, H 1 :,X 1 x 1,x. In order to decide which hyothesis is true, nodes 1 and that observe X1 n and X n, resectively, comress their observed sequences into indices of rates and R, and communicate them over noiseless links to node 3, which then makes a decision {H 0,H 1 } based on the received comression indices. What is the otimal tradeoff between the communication rates and the tye I and II error robabilities? Desite many natural alications, however, theoretical understanding of this roblem is far from comlete and a simle characterization of this rate exonent tradeoff remains oen in general. X n Fig. 1. Node 1 Node R Node 3 Multiterminal hyothesis testing with communication constraints. In their celebrated aer [], Ahlswede and Csiszár studied the secial case in which the sequence X n is fully available at the destination node, i.e., R. They established singleletter inner and outer bounds on the otimal tradeoff and showed that these bounds are tight for testing against indeendence, i.e., the alternative hyothesis H 1 is 1 x 1,x 0 x 1 0 x. Later, Han [3] and Shimokawa, Han, and Amari [4] rovided a new coding scheme that imroves uon the Ahlswede and Csiszar inner bound for the general hyothesis testing roblem. A more comrehensive survey on the earlier literature can be found in [5]. Several variations of this setu have been studied, including successive refinement hyothesis testing [6] and testing against conditional indeendence [7]. This aer studies an interactive version of hyothesis testing with communication constraints. Two nodes communicate with each other in q rounds through noiseless links and one of the nodes is to erform hyothesis testing at the end of interactive communication. For the secial case of hyothesis testing against indeendence, we establish a single-letter characterization of the otimal tradeoff between the communication rates and the tye II error robability when the tye I error robability is arbitrarily small. The q case has been reorted in [8]. The rest of the aer is organized as follows. In Section II, we review the roblem of one-way hyothesis testing with communication constraints. In Section III, we formulate the roblem of interactive hyothesis testing with communication constraints and resent our main theorem. In Section IV, we comare the interactive hyothesis testing roblem with the interactive lossy source coding roblem by Kasi [9]. Throughout the aer, we closely follow the notation in [10]. In articular, for X x and ɛ 0, 1, we define the set of ɛ-tyical n-sequences or the tyical set in short [11] as T ɛ n X { : #{i : x i x}/n x ɛx for all x X}. We say that X Y Z form a Markov chain if x, y, z xy xz y, thatis,x and Z are conditionally indeendent of each other given X. II. ONE-WAY HYPOTHESIS TESTING WITH COMMUNICATION CONSTRAINTS As before, let X1 n,xn n,x x 1i,x i be a air of i.i.d. sequences generated by a -DMS,X and U.S. Government work not rotected by U.S. coyright 840
013 IEEE International Symosium on Information Theory consider hyothesis testing against indeendence 1/ 0 0 3/4 3/4 0 0 1/ H 0 :,X 0 x 1,x, H 1 :,X 1 x 1,x 0 x 1 0 x. Here 0 x 1 and 0 x are marginal distributions of 0 x 1,x. We consider the secial case of the roblem deicted in Fig. 1, in which R ; see Fig.. 1/ 1 0.35 1/ a X 1 1/4 X 1/4 1 b /3 1 1/ 0.3 Fig.. Node 1 Node X n One-way hyothesis testing with communication constraint. θ 0.5 0. 0.15 θ 1 A nr1,n hyothesis test consists of an encoder that assigns an index m 1 1 [1 : nr1 ] to each sequence 1 Xn 1,and a tester that assigns ĥm 1, {H 0,H 1 } to each m 1, [1 : nr1 ] X n. The accetance region is defined as A n : {m 1, [1 : nr1 ] X n : ĥm 1, H 0}. Then the tye I error robability is P 0 A c n 1,xn :m1xn 1,xn Ac n 0 1,xn and the tye II error robability is P 1 A n 1 1,. 1,xn :m1xn 1,xn An For ɛ 0, 1, define the otimal tye II error robability as β n,ɛ:minp 1 A n, where the minimum is over all nr1,n tests such that P 0 A c n ɛ. Further define the otimal tye II error exonent as θ 1,ɛ : lim n n log β n,ɛ. Theorem 1 Ahlswede and Csiszár []: For every ɛ 0, 1, θ 1,ɛ max IU 1; X, 1 u 1 x 1: IU 1; where the cardinality bound for U 1 is U 1 +1. We illustrate the theorem with the following. Examle 1 [8]: Consider the following forward Zbinary sources,x deicted in Fig. 3a, where X is the outut of through a Z channel and X1,X 0, 0 1/, X1,X 0, 1 0, X1,X 1, 0 1/4, X1,X 1, 1 1/4. The entire curve of the otimal tye II error exonent, denoted by θ1,ɛ, is lotted in Fig. 3c. Examle : Now consider the following backward Zbinary sources,x deicted in Fig. 3b, where is the outut 0.1 0.05 0 0 0. 0.4 0.6 0.8 1 c Fig. 3. Comarison of error exonents for a forward Z binary sources and b backward Z binary sources. The solid black curve corresonds to θ1,ɛ and the dotted red curve corresonds to θ1,ɛ. of X through an inverted Z channel. The entire curve of the otimal tye II error exonent, denoted by θ 1,ɛ this time, is lotted in Fig. 3c. Observe that for every 0, 1, θ 1,ɛ >θ 1,ɛ. III. INTERACTIVE HYPOTHESIS TESTING WITH COMMUNICATION CONSTRAINTS Suose now that instead of making an immediate decision based on one round of communication, the two nodes can interactively communicate over a noiseless bidirectional link before one of the nodes erforms hyothesis testing. We wish to characterize the otimal tradeoff between the communication rates and the erformance of hyothesis testing. Fig. 4. M l,ml 1 θ 1 Node 1 M l+1 X n,ml Node X n Interactive hyothesis testing with communication constraints. As before, we consider testing against indeendence. Assume without loss of generality that node 1 sends the first index and that the number of rounds of communication q is even. A nr1,..., nrq,n hyothesis test consists of two encoders, one for each node, where in round l j {j, j +,...,q +j}, encoder j {1, } sends an index m lj j,mlj 1 [1 : nr l j ], that is, a function of its sequence and all reviously transmitted indices, and a tester that assigns ĥmq, 1 {H 0,H 1 } to each m q, 1 [1 : ] [1 : nr1 nrq ] X1 n. 841
013 IEEE International Symosium on Information Theory The tye I and II error robabilities are defined similarly as in the one-way case. In articular, the otimal tye II error exonent is θ q,...,r q,ɛ : lim 1 n n log β n,...,r q,ɛ. We establish the otimal tradeoff between the rate constraints and the testing erformance by characterizing θ q,...,r q,ɛ in the limit. Theorem : lim θ q,...,r q,ɛmax IU l ; X jl U l 1, 3 ɛ 0 where the maximum is over all q u l u l 1,x jl with U l X jl l 1 j1 U j +1 such that R l IU l ; X jl U l 1 for l [1 : q] and j l 1if l is odd and j l if l is even. Remark 1: By setting U l and R l 0for l,...,q, Theorem recovers the otimal one-way tye II error exonent in Theorem 1. Remark : We recover the following result [8] for q : lim θ,r,ɛmax IU 1 ; X +IU ; U 1, ɛ 0 where the maximum is over all u 1 x 1 u u 1,x with U 1 +1 and U X U 1 +1such that IU 1 ;, R IU ; X U 1. Remark 3: We can exress the otimal tradeoff between communication constraints and the tye II error exonent by the rate exonent region that consists of all rate exonent tules,...,r q,θ such that R l IU l ; X jl U l 1, l [1 :: q], θ IU l ; X jl U l 1 for some mfs q u l u l 1,x jl. Examle 3 Interaction hels: This examle is motivated by [1]. We revisit the Z binary sources in Examles 1 and. Recall that θ 1,ɛ and θ 1,ɛ denote the otimal tye II error exonents for the forward and backward Z binary sources, resectively. Now consider the following double Z binary sources as deicted in Fig. 5, where,x is indeendent of Y 1,Y.Let θ R, ɛ : max θ,r,ɛ.,r :+R R It can be easily verified that if R R, R,whereR min{r : θ R, ɛ I ; X 0.3113}, then θ R, ɛ θ R/,ɛ, Fig. 5. 1/ 0 1/ 1 3/4 0 Y 1 1/4 1 1/ /3 0 3/4 X 1 1/4 0 1/ Y 1 1/ Double Z binary sources. while θ 1 R, ɛ θ R,ɛ+θ R R,ɛ a <θ R,ɛ+θ R R,ɛ b θ R/,ɛ, where a follows by and b follows by the concavity of θ1 R over [0,R ] see, for examle, [, Lemma 1]. For examle, when R 3/, wehaveθ 1 3/,ɛ 0.5548 and θ 3/,ɛ 0.5934. Thus there is strict imrovement by using interaction. In the following we rove the converse for Theorem. The roof of achievability is similar for the two-round case [8] and omitted for brevity. Proof of the Converse for Theorem : Consider q is odd and let j l 1 if l is odd and j l if l is even. Given a nr1,..., nrq,n test characterized by the encoding functions m l, l 1,...,q, and the accetance region A n,we can aly the data rocessing inequality for relative entroy as for the two-round case [8] to have the multiletter uer bound lim θ 1,...,R q,ɛ lim ɛ 0 n n D 0 1,m q 1 1,m q, where 0 1,m q : 0 1, m l m l 1, j l and 0 1 m l m l 1, 1 l odd 0 xn 1 m l m l 1, 1 1,mq : 0 1 0 m l m l 1, j l 0 1 m l m l 1, 1 l odd 0 m l m l 1,. 84
013 IEEE International Symosium on Information Theory It is easy to verify that 0 m l m l 1, 1 1 m l m l 1, 1 We now rove that 0 xn 1 0 D 0 1,mq 1 1,mq m l m l 1,, m l m l 1,. IM l ; Xj n l+1 M l 1. 4 To show this, we exand the relative entroy term in 4 as 5 at the bottom of this age. The second term in 5 can be uer bounded as 6 at the bottom of the age, which establishes 4. To comlete the roof, we single-letterize the uer bound in 4 as IM l ; X1 n M l 1 IM l ; i M l 1 a IM l ; i M l + IM l ; X i 1 M l 1,X1 i 1 IM l ; X i 1 M l 1,X1 i IM l ; i M l 1,X1 i, where a follows from IM l ; X i 1 M l 1,X1 i 1 IM l ; X i 1 M l 1,X1 i HX i 1 M l 1,X1 i 1 HX i 1 M l,x1 i 1 HX i 1 M l 1,X1+HX i i 1 M l,x1 i IX i 1 ; i M l 1,X1 i 1 IX i 1 ; i M l,x1 i 1 0. To bound the rate constraints, consider for, nr l HM l IM l ; X1 n,x n M l 1 IM l ; i,x i M l 1,X1 i IM l ; X i M l. When l>1 is odd, the rate constraints and the terms in 4 can be bounded similarly. The case of l 1 needs to be D 0 1,mq 1 1,mq 0 1,m q log 0m q mq 1,xn1 0 m q m q 3, 1 0 m 1 1 1 m q m q 1 1 m q m q 3 1 m 1 1,mq 0 0 m q m q 1, 1,mq 1 log 0 m q m q 1 1,mq IM l ; X1 n M l 1 + m l 0 m q m q 1 1 m q m q 1 0 m q m q 3, 1 0 m q m q 3 0m 1 1 0 m q m q 3 1 m q m q 3 0 m 1 0 m 1 1 m 1 0 m l log 0m l m l 1 1 m l m l 1, 5 0 m l log 0m l ml 1 1 m l m l 1 m l 0 m l 0 m l,m l 1 m l log 0m l ml 1 m l m l,m 1 m l m l 1 l 1 0 m l D 0 m l m l 1 0 m l 1 m l 1 m l m l 1 0 m l 1 m l m l m l m l 0 m l D 0 m l 1 m l 0 m l m l 1, 0 m l 1 0 m l 1 m l 0 m l D 0 m l 1 m l 0 m l m l 1, 0 m l 1 0 m l 1 m l 0 m l m l 1, 0 m l 0 m l m l 1, 0 m l IM l 1 ; X n M l. 6 843
013 IEEE International Symosium on Information Theory considered searately and it can be easily verified that n IM 1,X1 i ; X i IM 1 ; n IM,X i 1 1 ; i. Identify U 1i M and U li M l for l. Define the time-sharing random variable Q to be uniformly distributed over [1 : n] and indeendent of M q,,xn, and let U l Q, U lq, Q,andX X Q. Clearly, U l U l 1,X jl X jl+1 form Markov chains. Finally, the cardinality bounds on U l follow the standard technique, in articular, the one used in the -round interactive lossy source coding roblem [9]. This comletes the converse roof. IV. RELATIONSHIP TO INTERACTIVE LOSSY COMPRESSION In this section, we comare the two-round interactive hyothesis testing roblem with the two-round interactive lossy source coding roblem. Consider the interactive lossy source coding roblem deicted in Fig. 6. Here two nodes interactively communicate with each other so that each node can reconstruct the source observed by the other node with rescribed distortions. Kasi [9] established the otimal tradeoff between communication constraints and the distortion air D 1,D. See also Ma and Ishwar [13] for an ingenious examle demonstrating that interactive lossy comression can strictly outerforms one-way lossy comression. The otimal tradeoff between communication and distortion is characterized by the rate distortion region, the formal definition of which can be found in [9] or [10, Section 0.3]. Theorem 3 Kasi [9]: The two-round rate distortion region is the set of all rate airs,r such that I ; U q X, R IX ; U q for some conditional mf q u l u l 1,x jl with U l X jl l j1 U j +1 and functions ˆx 1 u q,x and ˆx 1 u q,x 1 that satisfy Ed j X j, ˆX j D j, j 1,, wherej l 1if l is odd and j l if l is even. Achievability is established by erforming Wyner Ziv coding [14] in each round, i.e., joint tyicality encoding followed by binning. By contrast, the scheme we used for the interactive hyothesis testing roblem is joint tyicality encoding in each round without binning. It turns out, however, that this distinction between binning and no binning is not fundamental. By using Wyner Ziv coding in the interactive hyothesis testing X1 n M l X1 n,ml 1 X n Node 1 M l+1 X n Node,Ml ˆX n,d ˆ,D 1 Fig. 6. Interactive lossy comression. roblem, we can establish the following tradeoff between communication constraints and the testing erformance. Proosition 1: The rate exonent region for q-round interactive hyothesis testing is the set of rate exonent triles,...,r q,θ such that θ IU l ; X jl U l 1, R l IU l ; X jl U l 1 IU l ; X jl+1 U l 1, l [1 : q], R l θ IUl ; X jl U l 1 IU l ; X jl U l 1 for some q u l u l 1,x jl,wherej l 1if l is odd and j l if l is even. It can be shown that the rate exonent region in Proosition 1 is equivalent to the region in Remark 3. As ointed out by Rahman and Wagner [7] in the one-way setu, binning never hurts. Therefore, the coding scheme for q-round interactive lossy source coding leads to an essentially identical scheme for q-round interactive hyothesis testing. It is refreshing to note that the same scheme is otimal for both roblems. REFERENCES [1] T. Berger, Decentralized estimation and decision theory, in Proc. IEEE Inf. Theory Worksho, Mt. Kisco, NY, Se. 1979. [] R. Ahlswede and I. Csiszár, Hyothesis testing with communication constraints, IEEE Trans. Inf. Theory, vol. 3, no. 4,. 533 54, 1986. [3] T. S. Han, Hyothesis testing with multiterminal data comression, IEEE Trans. Inf. Theory, vol. 33, no. 6,. 759 77, 1987. [4] H. Shimokawa, T. S. Han, and S. Amari, Error bound of hyothesis testing with data comression, in Proc. IEEE Internat. Sym. Inf. Theory, Jun. 1994,. 9. [5] T. S. Han and S. Amari, Statistical inference under multiterminal data comression, IEEE Trans. Inf. Theory, vol. 44, no. 6,. 300 34, Oct. 1998. [6] C. Tian and J. Chen, Successive refinement for hyothesis testing and lossless one-heler roblem, IEEE Trans. Inf. Theory, vol. 54, no. 10,. 4666 4681, Oct. 008. [7] M. S. Rahman and A. Wagner, On the otimality of binning for distributed hyothesis testing, IEEE Trans. Inf. Theory, vol. 58, no. 10,. 68 6303, Oct. 01. [8] Y. Xiang and Y.-H. Kim, Interactive hyothesis testing with communication constraints, in Proc. 50th Ann. Allerton Conf. Commun. Control Comut., Se. 01. [9] A. H. Kasi, Two-way source coding with a fidelity criterion, IEEE Trans. Inf. Theory, vol. 31, no. 6,. 735 740, 1985. [10] A. El Gamal and Y.-H. Kim, Network Information Theory. Cambridge: Cambridge University Press, 011. [11] A. Orlitsky and J. R. Roche, Coding for comuting, IEEE Trans. Inf. Theory, vol. 47, no. 3,. 903 917, 001. [1] A. A. Gohari and V. Anantharam, Information-theoretic key agreement of multile terminals I: Source model, IEEE Trans. Inf. Theory, vol. 56, no. 8,. 3973 3996, Aug. 010. [13] N. Ma and P. Ishwar, Some results on distributed source coding for interactive function comutation, IEEE Trans. Inf. Theory, vol. 57, no. 9,. 6180 6195, Se. 011. [14] A. D. Wyner and J. Ziv, The rate distortion function for source coding with side information at the decoder, IEEE Trans. Inf. Theory, vol., no. 1,. 1 10, 1976. 844