Interactive Hypothesis Testing Against Independence
|
|
- Alice Payne
- 5 years ago
- Views:
Transcription
1 013 IEEE International Symosium on Information Theory Interactive Hyothesis Testing Against Indeendence Yu Xiang and Young-Han Kim Deartment of Electrical and Comuter Engineering University of California, San Diego La Jolla, CA 9093, USA Abstract A hyothesis testing roblem with communication constraints is studied, in which two nodes interactively communicate with each other in q rounds to erform a simle binary hyothesis testing on whether the observed random sequences at the nodes are generated indeendently or not. The otimal tradeoff between the communication rates in q rounds interaction and the testing erformance measured by the tye II error exonent such that the tye I error robability asymtotically vanishes. An examle is rovided that shows that a two-way test outerforms the otimal one-way test and thus that interaction hels for hyothesis testing. I. INTRODUCTION Berger [1], in an insiring attemt at combining information theory and statistical inference, formulated the roblem of hyothesis testing with communication constraints as deicted in Fig. 1. Let X1 n,x n n,x x 1i,x i beaair of indeendent and identically distributed i.i.d. n-sequences generated by a two-comonent discrete memoryless source - DMS,X. Suose that there are two hyotheses on the joint distribution of,x, namely, H 0 :,X 0 x 1,x, H 1 :,X 1 x 1,x. In order to decide which hyothesis is true, nodes 1 and that observe X1 n and X n, resectively, comress their observed sequences into indices of rates and R, and communicate them over noiseless links to node 3, which then makes a decision {H 0,H 1 } based on the received comression indices. What is the otimal tradeoff between the communication rates and the tye I and II error robabilities? Desite many natural alications, however, theoretical understanding of this roblem is far from comlete and a simle characterization of this rate exonent tradeoff remains oen in general. X n Fig. 1. Node 1 Node R Node 3 Multiterminal hyothesis testing with communication constraints. In their celebrated aer [], Ahlswede and Csiszár studied the secial case in which the sequence X n is fully available at the destination node, i.e., R. They established singleletter inner and outer bounds on the otimal tradeoff and showed that these bounds are tight for testing against indeendence, i.e., the alternative hyothesis H 1 is 1 x 1,x 0 x 1 0 x. Later, Han [3] and Shimokawa, Han, and Amari [4] rovided a new coding scheme that imroves uon the Ahlswede and Csiszar inner bound for the general hyothesis testing roblem. A more comrehensive survey on the earlier literature can be found in [5]. Several variations of this setu have been studied, including successive refinement hyothesis testing [6] and testing against conditional indeendence [7]. This aer studies an interactive version of hyothesis testing with communication constraints. Two nodes communicate with each other in q rounds through noiseless links and one of the nodes is to erform hyothesis testing at the end of interactive communication. For the secial case of hyothesis testing against indeendence, we establish a single-letter characterization of the otimal tradeoff between the communication rates and the tye II error robability when the tye I error robability is arbitrarily small. The q case has been reorted in [8]. The rest of the aer is organized as follows. In Section II, we review the roblem of one-way hyothesis testing with communication constraints. In Section III, we formulate the roblem of interactive hyothesis testing with communication constraints and resent our main theorem. In Section IV, we comare the interactive hyothesis testing roblem with the interactive lossy source coding roblem by Kasi [9]. Throughout the aer, we closely follow the notation in [10]. In articular, for X x and ɛ 0, 1, we define the set of ɛ-tyical n-sequences or the tyical set in short [11] as T ɛ n X { : #{i : x i x}/n x ɛx for all x X}. We say that X Y Z form a Markov chain if x, y, z xy xz y, thatis,x and Z are conditionally indeendent of each other given X. II. ONE-WAY HYPOTHESIS TESTING WITH COMMUNICATION CONSTRAINTS As before, let X1 n,xn n,x x 1i,x i be a air of i.i.d. sequences generated by a -DMS,X and U.S. Government work not rotected by U.S. coyright 840
2 013 IEEE International Symosium on Information Theory consider hyothesis testing against indeendence 1/ 0 0 3/4 3/ / H 0 :,X 0 x 1,x, H 1 :,X 1 x 1,x 0 x 1 0 x. Here 0 x 1 and 0 x are marginal distributions of 0 x 1,x. We consider the secial case of the roblem deicted in Fig. 1, in which R ; see Fig.. 1/ / a X 1 1/4 X 1/4 1 b /3 1 1/ 0.3 Fig.. Node 1 Node X n One-way hyothesis testing with communication constraint. θ θ 1 A nr1,n hyothesis test consists of an encoder that assigns an index m 1 1 [1 : nr1 ] to each sequence 1 Xn 1,and a tester that assigns ĥm 1, {H 0,H 1 } to each m 1, [1 : nr1 ] X n. The accetance region is defined as A n : {m 1, [1 : nr1 ] X n : ĥm 1, H 0}. Then the tye I error robability is P 0 A c n 1,xn :m1xn 1,xn Ac n 0 1,xn and the tye II error robability is P 1 A n 1 1,. 1,xn :m1xn 1,xn An For ɛ 0, 1, define the otimal tye II error robability as β n,ɛ:minp 1 A n, where the minimum is over all nr1,n tests such that P 0 A c n ɛ. Further define the otimal tye II error exonent as θ 1,ɛ : lim n n log β n,ɛ. Theorem 1 Ahlswede and Csiszár []: For every ɛ 0, 1, θ 1,ɛ max IU 1; X, 1 u 1 x 1: IU 1; where the cardinality bound for U 1 is U We illustrate the theorem with the following. Examle 1 [8]: Consider the following forward Zbinary sources,x deicted in Fig. 3a, where X is the outut of through a Z channel and X1,X 0, 0 1/, X1,X 0, 1 0, X1,X 1, 0 1/4, X1,X 1, 1 1/4. The entire curve of the otimal tye II error exonent, denoted by θ1,ɛ, is lotted in Fig. 3c. Examle : Now consider the following backward Zbinary sources,x deicted in Fig. 3b, where is the outut c Fig. 3. Comarison of error exonents for a forward Z binary sources and b backward Z binary sources. The solid black curve corresonds to θ1,ɛ and the dotted red curve corresonds to θ1,ɛ. of X through an inverted Z channel. The entire curve of the otimal tye II error exonent, denoted by θ 1,ɛ this time, is lotted in Fig. 3c. Observe that for every 0, 1, θ 1,ɛ >θ 1,ɛ. III. INTERACTIVE HYPOTHESIS TESTING WITH COMMUNICATION CONSTRAINTS Suose now that instead of making an immediate decision based on one round of communication, the two nodes can interactively communicate over a noiseless bidirectional link before one of the nodes erforms hyothesis testing. We wish to characterize the otimal tradeoff between the communication rates and the erformance of hyothesis testing. Fig. 4. M l,ml 1 θ 1 Node 1 M l+1 X n,ml Node X n Interactive hyothesis testing with communication constraints. As before, we consider testing against indeendence. Assume without loss of generality that node 1 sends the first index and that the number of rounds of communication q is even. A nr1,..., nrq,n hyothesis test consists of two encoders, one for each node, where in round l j {j, j +,...,q +j}, encoder j {1, } sends an index m lj j,mlj 1 [1 : nr l j ], that is, a function of its sequence and all reviously transmitted indices, and a tester that assigns ĥmq, 1 {H 0,H 1 } to each m q, 1 [1 : ] [1 : nr1 nrq ] X1 n. 841
3 013 IEEE International Symosium on Information Theory The tye I and II error robabilities are defined similarly as in the one-way case. In articular, the otimal tye II error exonent is θ q,...,r q,ɛ : lim 1 n n log β n,...,r q,ɛ. We establish the otimal tradeoff between the rate constraints and the testing erformance by characterizing θ q,...,r q,ɛ in the limit. Theorem : lim θ q,...,r q,ɛmax IU l ; X jl U l 1, 3 ɛ 0 where the maximum is over all q u l u l 1,x jl with U l X jl l 1 j1 U j +1 such that R l IU l ; X jl U l 1 for l [1 : q] and j l 1if l is odd and j l if l is even. Remark 1: By setting U l and R l 0for l,...,q, Theorem recovers the otimal one-way tye II error exonent in Theorem 1. Remark : We recover the following result [8] for q : lim θ,r,ɛmax IU 1 ; X +IU ; U 1, ɛ 0 where the maximum is over all u 1 x 1 u u 1,x with U 1 +1 and U X U 1 +1such that IU 1 ;, R IU ; X U 1. Remark 3: We can exress the otimal tradeoff between communication constraints and the tye II error exonent by the rate exonent region that consists of all rate exonent tules,...,r q,θ such that R l IU l ; X jl U l 1, l [1 :: q], θ IU l ; X jl U l 1 for some mfs q u l u l 1,x jl. Examle 3 Interaction hels: This examle is motivated by [1]. We revisit the Z binary sources in Examles 1 and. Recall that θ 1,ɛ and θ 1,ɛ denote the otimal tye II error exonents for the forward and backward Z binary sources, resectively. Now consider the following double Z binary sources as deicted in Fig. 5, where,x is indeendent of Y 1,Y.Let θ R, ɛ : max θ,r,ɛ.,r :+R R It can be easily verified that if R R, R,whereR min{r : θ R, ɛ I ; X }, then θ R, ɛ θ R/,ɛ, Fig. 5. 1/ 0 1/ 1 3/4 0 Y 1 1/4 1 1/ /3 0 3/4 X 1 1/4 0 1/ Y 1 1/ Double Z binary sources. while θ 1 R, ɛ θ R,ɛ+θ R R,ɛ a <θ R,ɛ+θ R R,ɛ b θ R/,ɛ, where a follows by and b follows by the concavity of θ1 R over [0,R ] see, for examle, [, Lemma 1]. For examle, when R 3/, wehaveθ 1 3/,ɛ and θ 3/,ɛ Thus there is strict imrovement by using interaction. In the following we rove the converse for Theorem. The roof of achievability is similar for the two-round case [8] and omitted for brevity. Proof of the Converse for Theorem : Consider q is odd and let j l 1 if l is odd and j l if l is even. Given a nr1,..., nrq,n test characterized by the encoding functions m l, l 1,...,q, and the accetance region A n,we can aly the data rocessing inequality for relative entroy as for the two-round case [8] to have the multiletter uer bound lim θ 1,...,R q,ɛ lim ɛ 0 n n D 0 1,m q 1 1,m q, where 0 1,m q : 0 1, m l m l 1, j l and 0 1 m l m l 1, 1 l odd 0 xn 1 m l m l 1, 1 1,mq : m l m l 1, j l 0 1 m l m l 1, 1 l odd 0 m l m l 1,. 84
4 013 IEEE International Symosium on Information Theory It is easy to verify that 0 m l m l 1, 1 1 m l m l 1, 1 We now rove that 0 xn 1 0 D 0 1,mq 1 1,mq m l m l 1,, m l m l 1,. IM l ; Xj n l+1 M l 1. 4 To show this, we exand the relative entroy term in 4 as 5 at the bottom of this age. The second term in 5 can be uer bounded as 6 at the bottom of the age, which establishes 4. To comlete the roof, we single-letterize the uer bound in 4 as IM l ; X1 n M l 1 IM l ; i M l 1 a IM l ; i M l + IM l ; X i 1 M l 1,X1 i 1 IM l ; X i 1 M l 1,X1 i IM l ; i M l 1,X1 i, where a follows from IM l ; X i 1 M l 1,X1 i 1 IM l ; X i 1 M l 1,X1 i HX i 1 M l 1,X1 i 1 HX i 1 M l,x1 i 1 HX i 1 M l 1,X1+HX i i 1 M l,x1 i IX i 1 ; i M l 1,X1 i 1 IX i 1 ; i M l,x1 i 1 0. To bound the rate constraints, consider for, nr l HM l IM l ; X1 n,x n M l 1 IM l ; i,x i M l 1,X1 i IM l ; X i M l. When l>1 is odd, the rate constraints and the terms in 4 can be bounded similarly. The case of l 1 needs to be D 0 1,mq 1 1,mq 0 1,m q log 0m q mq 1,xn1 0 m q m q 3, 1 0 m m q m q 1 1 m q m q 3 1 m 1 1,mq 0 0 m q m q 1, 1,mq 1 log 0 m q m q 1 1,mq IM l ; X1 n M l 1 + m l 0 m q m q 1 1 m q m q 1 0 m q m q 3, 1 0 m q m q 3 0m m q m q 3 1 m q m q 3 0 m 1 0 m 1 1 m 1 0 m l log 0m l m l 1 1 m l m l 1, 5 0 m l log 0m l ml 1 1 m l m l 1 m l 0 m l 0 m l,m l 1 m l log 0m l ml 1 m l m l,m 1 m l m l 1 l 1 0 m l D 0 m l m l 1 0 m l 1 m l 1 m l m l 1 0 m l 1 m l m l m l m l 0 m l D 0 m l 1 m l 0 m l m l 1, 0 m l 1 0 m l 1 m l 0 m l D 0 m l 1 m l 0 m l m l 1, 0 m l 1 0 m l 1 m l 0 m l m l 1, 0 m l 0 m l m l 1, 0 m l IM l 1 ; X n M l
5 013 IEEE International Symosium on Information Theory considered searately and it can be easily verified that n IM 1,X1 i ; X i IM 1 ; n IM,X i 1 1 ; i. Identify U 1i M and U li M l for l. Define the time-sharing random variable Q to be uniformly distributed over [1 : n] and indeendent of M q,,xn, and let U l Q, U lq, Q,andX X Q. Clearly, U l U l 1,X jl X jl+1 form Markov chains. Finally, the cardinality bounds on U l follow the standard technique, in articular, the one used in the -round interactive lossy source coding roblem [9]. This comletes the converse roof. IV. RELATIONSHIP TO INTERACTIVE LOSSY COMPRESSION In this section, we comare the two-round interactive hyothesis testing roblem with the two-round interactive lossy source coding roblem. Consider the interactive lossy source coding roblem deicted in Fig. 6. Here two nodes interactively communicate with each other so that each node can reconstruct the source observed by the other node with rescribed distortions. Kasi [9] established the otimal tradeoff between communication constraints and the distortion air D 1,D. See also Ma and Ishwar [13] for an ingenious examle demonstrating that interactive lossy comression can strictly outerforms one-way lossy comression. The otimal tradeoff between communication and distortion is characterized by the rate distortion region, the formal definition of which can be found in [9] or [10, Section 0.3]. Theorem 3 Kasi [9]: The two-round rate distortion region is the set of all rate airs,r such that I ; U q X, R IX ; U q for some conditional mf q u l u l 1,x jl with U l X jl l j1 U j +1 and functions ˆx 1 u q,x and ˆx 1 u q,x 1 that satisfy Ed j X j, ˆX j D j, j 1,, wherej l 1if l is odd and j l if l is even. Achievability is established by erforming Wyner Ziv coding [14] in each round, i.e., joint tyicality encoding followed by binning. By contrast, the scheme we used for the interactive hyothesis testing roblem is joint tyicality encoding in each round without binning. It turns out, however, that this distinction between binning and no binning is not fundamental. By using Wyner Ziv coding in the interactive hyothesis testing X1 n M l X1 n,ml 1 X n Node 1 M l+1 X n Node,Ml ˆX n,d ˆ,D 1 Fig. 6. Interactive lossy comression. roblem, we can establish the following tradeoff between communication constraints and the testing erformance. Proosition 1: The rate exonent region for q-round interactive hyothesis testing is the set of rate exonent triles,...,r q,θ such that θ IU l ; X jl U l 1, R l IU l ; X jl U l 1 IU l ; X jl+1 U l 1, l [1 : q], R l θ IUl ; X jl U l 1 IU l ; X jl U l 1 for some q u l u l 1,x jl,wherej l 1if l is odd and j l if l is even. It can be shown that the rate exonent region in Proosition 1 is equivalent to the region in Remark 3. As ointed out by Rahman and Wagner [7] in the one-way setu, binning never hurts. Therefore, the coding scheme for q-round interactive lossy source coding leads to an essentially identical scheme for q-round interactive hyothesis testing. It is refreshing to note that the same scheme is otimal for both roblems. REFERENCES [1] T. Berger, Decentralized estimation and decision theory, in Proc. IEEE Inf. Theory Worksho, Mt. Kisco, NY, Se [] R. Ahlswede and I. Csiszár, Hyothesis testing with communication constraints, IEEE Trans. Inf. Theory, vol. 3, no. 4, , [3] T. S. Han, Hyothesis testing with multiterminal data comression, IEEE Trans. Inf. Theory, vol. 33, no. 6, , [4] H. Shimokawa, T. S. Han, and S. Amari, Error bound of hyothesis testing with data comression, in Proc. IEEE Internat. Sym. Inf. Theory, Jun. 1994,. 9. [5] T. S. Han and S. Amari, Statistical inference under multiterminal data comression, IEEE Trans. Inf. Theory, vol. 44, no. 6, , Oct [6] C. Tian and J. Chen, Successive refinement for hyothesis testing and lossless one-heler roblem, IEEE Trans. Inf. Theory, vol. 54, no. 10, , Oct [7] M. S. Rahman and A. Wagner, On the otimality of binning for distributed hyothesis testing, IEEE Trans. Inf. Theory, vol. 58, no. 10, , Oct. 01. [8] Y. Xiang and Y.-H. Kim, Interactive hyothesis testing with communication constraints, in Proc. 50th Ann. Allerton Conf. Commun. Control Comut., Se. 01. [9] A. H. Kasi, Two-way source coding with a fidelity criterion, IEEE Trans. Inf. Theory, vol. 31, no. 6, , [10] A. El Gamal and Y.-H. Kim, Network Information Theory. Cambridge: Cambridge University Press, 011. [11] A. Orlitsky and J. R. Roche, Coding for comuting, IEEE Trans. Inf. Theory, vol. 47, no. 3, , 001. [1] A. A. Gohari and V. Anantharam, Information-theoretic key agreement of multile terminals I: Source model, IEEE Trans. Inf. Theory, vol. 56, no. 8, , Aug [13] N. Ma and P. Ishwar, Some results on distributed source coding for interactive function comutation, IEEE Trans. Inf. Theory, vol. 57, no. 9, , Se [14] A. D. Wyner and J. Ziv, The rate distortion function for source coding with side information at the decoder, IEEE Trans. Inf. Theory, vol., no. 1,. 1 10,
Interactive Hypothesis Testing with Communication Constraints
Fiftieth Annual Allerton Conference Allerton House, UIUC, Illinois, USA October - 5, 22 Interactive Hypothesis Testing with Communication Constraints Yu Xiang and Young-Han Kim Department of Electrical
More informationImproved Capacity Bounds for the Binary Energy Harvesting Channel
Imroved Caacity Bounds for the Binary Energy Harvesting Channel Kaya Tutuncuoglu 1, Omur Ozel 2, Aylin Yener 1, and Sennur Ulukus 2 1 Deartment of Electrical Engineering, The Pennsylvania State University,
More informationImproving on the Cutset Bound via a Geometric Analysis of Typical Sets
Imroving on the Cutset Bound via a Geometric Analysis of Tyical Sets Ayfer Özgür Stanford University CUHK, May 28, 2016 Joint work with Xiugang Wu (Stanford). Ayfer Özgür (Stanford) March 16 1 / 33 Gaussian
More informationOn Code Design for Simultaneous Energy and Information Transfer
On Code Design for Simultaneous Energy and Information Transfer Anshoo Tandon Electrical and Comuter Engineering National University of Singaore Email: anshoo@nus.edu.sg Mehul Motani Electrical and Comuter
More informationSampling and Distortion Tradeoffs for Bandlimited Periodic Signals
Samling and Distortion radeoffs for Bandlimited Periodic Signals Elaheh ohammadi and Farokh arvasti Advanced Communications Research Institute ACRI Deartment of Electrical Engineering Sharif University
More informationECE 534 Information Theory - Midterm 2
ECE 534 Information Theory - Midterm Nov.4, 009. 3:30-4:45 in LH03. You will be given the full class time: 75 minutes. Use it wisely! Many of the roblems have short answers; try to find shortcuts. You
More informationEECS 750. Hypothesis Testing with Communication Constraints
EECS 750 Hypothesis Testing with Communication Constraints Name: Dinesh Krithivasan Abstract In this report, we study a modification of the classical statistical problem of bivariate hypothesis testing.
More informationAnalysis of some entrance probabilities for killed birth-death processes
Analysis of some entrance robabilities for killed birth-death rocesses Master s Thesis O.J.G. van der Velde Suervisor: Dr. F.M. Sieksma July 5, 207 Mathematical Institute, Leiden University Contents Introduction
More informationConvex Optimization methods for Computing Channel Capacity
Convex Otimization methods for Comuting Channel Caacity Abhishek Sinha Laboratory for Information and Decision Systems (LIDS), MIT sinhaa@mit.edu May 15, 2014 We consider a classical comutational roblem
More informationEE/Stats 376A: Information theory Winter Lecture 5 Jan 24. Lecturer: David Tse Scribe: Michael X, Nima H, Geng Z, Anton J, Vivek B.
EE/Stats 376A: Information theory Winter 207 Lecture 5 Jan 24 Lecturer: David Tse Scribe: Michael X, Nima H, Geng Z, Anton J, Vivek B. 5. Outline Markov chains and stationary distributions Prefix codes
More informationDistributed Lossy Interactive Function Computation
Distributed Lossy Interactive Function Computation Solmaz Torabi & John MacLaren Walsh Dept. of Electrical and Computer Engineering Drexel University Philadelphia, PA 19104 Email: solmaz.t@drexel.edu &
More informationVarious Proofs for the Decrease Monotonicity of the Schatten s Power Norm, Various Families of R n Norms and Some Open Problems
Int. J. Oen Problems Comt. Math., Vol. 3, No. 2, June 2010 ISSN 1998-6262; Coyright c ICSRS Publication, 2010 www.i-csrs.org Various Proofs for the Decrease Monotonicity of the Schatten s Power Norm, Various
More informationLDPC codes for the Cascaded BSC-BAWGN channel
LDPC codes for the Cascaded BSC-BAWGN channel Aravind R. Iyengar, Paul H. Siegel, and Jack K. Wolf University of California, San Diego 9500 Gilman Dr. La Jolla CA 9093 email:aravind,siegel,jwolf@ucsd.edu
More informationLattices for Distributed Source Coding: Jointly Gaussian Sources and Reconstruction of a Linear Function
Lattices for Distributed Source Coding: Jointly Gaussian Sources and Reconstruction of a Linear Function Dinesh Krithivasan and S. Sandeep Pradhan Department of Electrical Engineering and Computer Science,
More informationOn Scalable Coding in the Presence of Decoder Side Information
On Scalable Coding in the Presence of Decoder Side Information Emrah Akyol, Urbashi Mitra Dep. of Electrical Eng. USC, CA, US Email: {eakyol, ubli}@usc.edu Ertem Tuncel Dep. of Electrical Eng. UC Riverside,
More informationOn Scalable Source Coding for Multiple Decoders with Side Information
On Scalable Source Coding for Multiple Decoders with Side Information Chao Tian School of Computer and Communication Sciences Laboratory for Information and Communication Systems (LICOS), EPFL, Lausanne,
More informationDistributed Lossless Compression. Distributed lossless compression system
Lecture #3 Distributed Lossless Compression (Reading: NIT 10.1 10.5, 4.4) Distributed lossless source coding Lossless source coding via random binning Time Sharing Achievability proof of the Slepian Wolf
More informationResearch Article An iterative Algorithm for Hemicontractive Mappings in Banach Spaces
Abstract and Alied Analysis Volume 2012, Article ID 264103, 11 ages doi:10.1155/2012/264103 Research Article An iterative Algorithm for Hemicontractive Maings in Banach Saces Youli Yu, 1 Zhitao Wu, 2 and
More informationElementary Analysis in Q p
Elementary Analysis in Q Hannah Hutter, May Szedlák, Phili Wirth November 17, 2011 This reort follows very closely the book of Svetlana Katok 1. 1 Sequences and Series In this section we will see some
More informationOn the capacity of the general trapdoor channel with feedback
On the caacity of the general tradoor channel with feedback Jui Wu and Achilleas Anastasooulos Electrical Engineering and Comuter Science Deartment University of Michigan Ann Arbor, MI, 48109-1 email:
More informationHypothesis Testing with Communication Constraints
Hypothesis Testing with Communication Constraints Dinesh Krithivasan EECS 750 April 17, 2006 Dinesh Krithivasan (EECS 750) Hyp. testing with comm. constraints April 17, 2006 1 / 21 Presentation Outline
More informationHomework Set #3 Rates definitions, Channel Coding, Source-Channel coding
Homework Set # Rates definitions, Channel Coding, Source-Channel coding. Rates (a) Channels coding Rate: Assuming you are sending 4 different messages using usages of a channel. What is the rate (in bits
More informationLossy Distributed Source Coding
Lossy Distributed Source Coding John MacLaren Walsh, Ph.D. Multiterminal Information Theory, Spring Quarter, 202 Lossy Distributed Source Coding Problem X X 2 S {,...,2 R } S 2 {,...,2 R2 } Ẑ Ẑ 2 E d(z,n,
More informationSHARED INFORMATION. Prakash Narayan with. Imre Csiszár, Sirin Nitinawarat, Himanshu Tyagi, Shun Watanabe
SHARED INFORMATION Prakash Narayan with Imre Csiszár, Sirin Nitinawarat, Himanshu Tyagi, Shun Watanabe 2/40 Acknowledgement Praneeth Boda Himanshu Tyagi Shun Watanabe 3/40 Outline Two-terminal model: Mutual
More informationOn The Binary Lossless Many-Help-One Problem with Independently Degraded Helpers
On The Binary Lossless Many-Help-One Problem with Independently Degraded Helpers Albrecht Wolf, Diana Cristina González, Meik Dörpinghaus, José Cândido Silveira Santos Filho, and Gerhard Fettweis Vodafone
More informationA Comparison of Superposition Coding Schemes
A Comparison of Superposition Coding Schemes Lele Wang, Eren Şaşoğlu, Bernd Bandemer, and Young-Han Kim Department of Electrical and Computer Engineering University of California, San Diego La Jolla, CA
More informationON THE LEAST SIGNIFICANT p ADIC DIGITS OF CERTAIN LUCAS NUMBERS
#A13 INTEGERS 14 (014) ON THE LEAST SIGNIFICANT ADIC DIGITS OF CERTAIN LUCAS NUMBERS Tamás Lengyel Deartment of Mathematics, Occidental College, Los Angeles, California lengyel@oxy.edu Received: 6/13/13,
More informationRobustness of classifiers to uniform l p and Gaussian noise Supplementary material
Robustness of classifiers to uniform l and Gaussian noise Sulementary material Jean-Yves Franceschi Ecole Normale Suérieure de Lyon LIP UMR 5668 Omar Fawzi Ecole Normale Suérieure de Lyon LIP UMR 5668
More informationSHARED INFORMATION. Prakash Narayan with. Imre Csiszár, Sirin Nitinawarat, Himanshu Tyagi, Shun Watanabe
SHARED INFORMATION Prakash Narayan with Imre Csiszár, Sirin Nitinawarat, Himanshu Tyagi, Shun Watanabe 2/41 Outline Two-terminal model: Mutual information Operational meaning in: Channel coding: channel
More informationIntroduction to Probability and Statistics
Introduction to Probability and Statistics Chater 8 Ammar M. Sarhan, asarhan@mathstat.dal.ca Deartment of Mathematics and Statistics, Dalhousie University Fall Semester 28 Chater 8 Tests of Hyotheses Based
More informationDistributed K-means over Compressed Binary Data
1 Distributed K-means over Comressed Binary Data Elsa DUPRAZ Telecom Bretagne; UMR CNRS 6285 Lab-STICC, Brest, France arxiv:1701.03403v1 [cs.it] 12 Jan 2017 Abstract We consider a networ of binary-valued
More informationInequalities for the L 1 Deviation of the Empirical Distribution
Inequalities for the L 1 Deviation of the Emirical Distribution Tsachy Weissman, Erik Ordentlich, Gadiel Seroussi, Sergio Verdu, Marcelo J. Weinberger June 13, 2003 Abstract We derive bounds on the robability
More informationB8.1 Martingales Through Measure Theory. Concept of independence
B8.1 Martingales Through Measure Theory Concet of indeendence Motivated by the notion of indeendent events in relims robability, we have generalized the concet of indeendence to families of σ-algebras.
More informationUniversal Finite Memory Coding of Binary Sequences
Deartment of Electrical Engineering Systems Universal Finite Memory Coding of Binary Sequences Thesis submitted towards the degree of Master of Science in Electrical and Electronic Engineering in Tel-Aviv
More information#A47 INTEGERS 15 (2015) QUADRATIC DIOPHANTINE EQUATIONS WITH INFINITELY MANY SOLUTIONS IN POSITIVE INTEGERS
#A47 INTEGERS 15 (015) QUADRATIC DIOPHANTINE EQUATIONS WITH INFINITELY MANY SOLUTIONS IN POSITIVE INTEGERS Mihai Ciu Simion Stoilow Institute of Mathematics of the Romanian Academy, Research Unit No. 5,
More informationMultiuser Successive Refinement and Multiple Description Coding
Multiuser Successive Refinement and Multiple Description Coding Chao Tian Laboratory for Information and Communication Systems (LICOS) School of Computer and Communication Sciences EPFL Lausanne Switzerland
More informationSimultaneous Nonunique Decoding Is Rate-Optimal
Fiftieth Annual Allerton Conference Allerton House, UIUC, Illinois, USA October 1-5, 2012 Simultaneous Nonunique Decoding Is Rate-Optimal Bernd Bandemer University of California, San Diego La Jolla, CA
More informationTests for Two Proportions in a Stratified Design (Cochran/Mantel-Haenszel Test)
Chater 225 Tests for Two Proortions in a Stratified Design (Cochran/Mantel-Haenszel Test) Introduction In a stratified design, the subects are selected from two or more strata which are formed from imortant
More informationNew Information Measures for the Generalized Normal Distribution
Information,, 3-7; doi:.339/info3 OPEN ACCESS information ISSN 75-7 www.mdi.com/journal/information Article New Information Measures for the Generalized Normal Distribution Christos P. Kitsos * and Thomas
More informationFAST AND EFFICIENT SIDE INFORMATION GENERATION IN DISTRIBUTED VIDEO CODING BY USING DENSE MOTION REPRESENTATIONS
18th Euroean Signal Processing Conference (EUSIPCO-2010) Aalborg, Denmark, August 23-27, 2010 FAST AND EFFICIENT SIDE INFORMATION GENERATION IN DISTRIBUTED VIDEO CODING BY USING DENSE MOTION REPRESENTATIONS
More informationOn Doob s Maximal Inequality for Brownian Motion
Stochastic Process. Al. Vol. 69, No., 997, (-5) Research Reort No. 337, 995, Det. Theoret. Statist. Aarhus On Doob s Maximal Inequality for Brownian Motion S. E. GRAVERSEN and G. PESKIR If B = (B t ) t
More informationOn Wald-Type Optimal Stopping for Brownian Motion
J Al Probab Vol 34, No 1, 1997, (66-73) Prerint Ser No 1, 1994, Math Inst Aarhus On Wald-Tye Otimal Stoing for Brownian Motion S RAVRSN and PSKIR The solution is resented to all otimal stoing roblems of
More informationImproved Bounds on Bell Numbers and on Moments of Sums of Random Variables
Imroved Bounds on Bell Numbers and on Moments of Sums of Random Variables Daniel Berend Tamir Tassa Abstract We rovide bounds for moments of sums of sequences of indeendent random variables. Concentrating
More informationOptimal Design of Truss Structures Using a Neutrosophic Number Optimization Model under an Indeterminate Environment
Neutrosohic Sets and Systems Vol 14 016 93 University of New Mexico Otimal Design of Truss Structures Using a Neutrosohic Number Otimization Model under an Indeterminate Environment Wenzhong Jiang & Jun
More informationSecret Key Agreement Using Conferencing in State- Dependent Multiple Access Channels with An Eavesdropper
Secret Key Agreement Using Conferencing in State- Dependent Multiple Access Channels with An Eavesdropper Mohsen Bahrami, Ali Bereyhi, Mahtab Mirmohseni and Mohammad Reza Aref Information Systems and Security
More informationMultiterminal Source Coding with an Entropy-Based Distortion Measure
Multiterminal Source Coding with an Entropy-Based Distortion Measure Thomas Courtade and Rick Wesel Department of Electrical Engineering University of California, Los Angeles 4 August, 2011 IEEE International
More informationRemote Source Coding with Two-Sided Information
Remote Source Coding with Two-Sided Information Basak Guler Ebrahim MolavianJazi Aylin Yener Wireless Communications and Networking Laboratory Department of Electrical Engineering The Pennsylvania State
More informationTopic: Lower Bounds on Randomized Algorithms Date: September 22, 2004 Scribe: Srinath Sridhar
15-859(M): Randomized Algorithms Lecturer: Anuam Guta Toic: Lower Bounds on Randomized Algorithms Date: Setember 22, 2004 Scribe: Srinath Sridhar 4.1 Introduction In this lecture, we will first consider
More informationJohn Weatherwax. Analysis of Parallel Depth First Search Algorithms
Sulementary Discussions and Solutions to Selected Problems in: Introduction to Parallel Comuting by Viin Kumar, Ananth Grama, Anshul Guta, & George Karyis John Weatherwax Chater 8 Analysis of Parallel
More informationFormal Modeling in Cognitive Science Lecture 29: Noisy Channel Model and Applications;
Formal Modeling in Cognitive Science Lecture 9: and ; ; Frank Keller School of Informatics University of Edinburgh keller@inf.ed.ac.uk Proerties of 3 March, 6 Frank Keller Formal Modeling in Cognitive
More informationMATH 250: THE DISTRIBUTION OF PRIMES. ζ(s) = n s,
MATH 50: THE DISTRIBUTION OF PRIMES ROBERT J. LEMKE OLIVER For s R, define the function ζs) by. Euler s work on rimes ζs) = which converges if s > and diverges if s. In fact, though we will not exloit
More informationA Formula for the Capacity of the General Gel fand-pinsker Channel
A Formula for the Capacity of the General Gel fand-pinsker Channel Vincent Y. F. Tan Institute for Infocomm Research (I 2 R, A*STAR, Email: tanyfv@i2r.a-star.edu.sg ECE Dept., National University of Singapore
More informationA Public-Key Cryptosystem Based on Lucas Sequences
Palestine Journal of Mathematics Vol. 1(2) (2012), 148 152 Palestine Polytechnic University-PPU 2012 A Public-Key Crytosystem Based on Lucas Sequences Lhoussain El Fadil Communicated by Ayman Badawi MSC2010
More informationCombining Logistic Regression with Kriging for Mapping the Risk of Occurrence of Unexploded Ordnance (UXO)
Combining Logistic Regression with Kriging for Maing the Risk of Occurrence of Unexloded Ordnance (UXO) H. Saito (), P. Goovaerts (), S. A. McKenna (2) Environmental and Water Resources Engineering, Deartment
More informationThe Gallager Converse
The Gallager Converse Abbas El Gamal Director, Information Systems Laboratory Department of Electrical Engineering Stanford University Gallager s 75th Birthday 1 Information Theoretic Limits Establishing
More informationCommon Information. Abbas El Gamal. Stanford University. Viterbi Lecture, USC, April 2014
Common Information Abbas El Gamal Stanford University Viterbi Lecture, USC, April 2014 Andrew Viterbi s Fabulous Formula, IEEE Spectrum, 2010 El Gamal (Stanford University) Disclaimer Viterbi Lecture 2
More informationA New Perspective on Learning Linear Separators with Large L q L p Margins
A New Persective on Learning Linear Searators with Large L q L Margins Maria-Florina Balcan Georgia Institute of Technology Christoher Berlind Georgia Institute of Technology Abstract We give theoretical
More informationCoding Along Hermite Polynomials for Gaussian Noise Channels
Coding Along Hermite olynomials for Gaussian Noise Channels Emmanuel A. Abbe IG, EFL Lausanne, 1015 CH Email: emmanuel.abbe@efl.ch Lizhong Zheng LIDS, MIT Cambridge, MA 0139 Email: lizhong@mit.edu Abstract
More informationECE Information theory Final (Fall 2008)
ECE 776 - Information theory Final (Fall 2008) Q.1. (1 point) Consider the following bursty transmission scheme for a Gaussian channel with noise power N and average power constraint P (i.e., 1/n X n i=1
More informationA Comparison between Biased and Unbiased Estimators in Ordinary Least Squares Regression
Journal of Modern Alied Statistical Methods Volume Issue Article 7 --03 A Comarison between Biased and Unbiased Estimators in Ordinary Least Squares Regression Ghadban Khalaf King Khalid University, Saudi
More informationApproximating min-max k-clustering
Aroximating min-max k-clustering Asaf Levin July 24, 2007 Abstract We consider the roblems of set artitioning into k clusters with minimum total cost and minimum of the maximum cost of a cluster. The cost
More informationOn Common Information and the Encoding of Sources that are Not Successively Refinable
On Common Information and the Encoding of Sources that are Not Successively Refinable Kumar Viswanatha, Emrah Akyol, Tejaswi Nanjundaswamy and Kenneth Rose ECE Department, University of California - Santa
More informationVARIANTS OF ENTROPY POWER INEQUALITY
VARIANTS OF ENTROPY POWER INEQUALITY Sergey G. Bobkov and Arnaud Marsiglietti Abstract An extension of the entroy ower inequality to the form N α r (X + Y ) N α r (X) + N α r (Y ) with arbitrary indeendent
More informationResearch Article Positive Solutions of Sturm-Liouville Boundary Value Problems in Presence of Upper and Lower Solutions
International Differential Equations Volume 11, Article ID 38394, 11 ages doi:1.1155/11/38394 Research Article Positive Solutions of Sturm-Liouville Boundary Value Problems in Presence of Uer and Lower
More informationThe non-stochastic multi-armed bandit problem
Submitted for journal ublication. The non-stochastic multi-armed bandit roblem Peter Auer Institute for Theoretical Comuter Science Graz University of Technology A-8010 Graz (Austria) auer@igi.tu-graz.ac.at
More informationLecture 21: Quantum Communication
CS 880: Quantum Information Processing 0/6/00 Lecture : Quantum Communication Instructor: Dieter van Melkebeek Scribe: Mark Wellons Last lecture, we introduced the EPR airs which we will use in this lecture
More informationA construction of bent functions from plateaued functions
A construction of bent functions from lateaued functions Ayça Çeşmelioğlu, Wilfried Meidl Sabancı University, MDBF, Orhanlı, 34956 Tuzla, İstanbul, Turkey. Abstract In this resentation, a technique for
More informationProblemsWeCanSolveWithaHelper
ITW 2009, Volos, Greece, June 10-12, 2009 ProblemsWeCanSolveWitha Haim Permuter Ben-Gurion University of the Negev haimp@bgu.ac.il Yossef Steinberg Technion - IIT ysteinbe@ee.technion.ac.il Tsachy Weissman
More informationOn Function Computation with Privacy and Secrecy Constraints
1 On Function Computation with Privacy and Secrecy Constraints Wenwen Tu and Lifeng Lai Abstract In this paper, the problem of function computation with privacy and secrecy constraints is considered. The
More informationSmall Zeros of Quadratic Forms Mod P m
International Mathematical Forum, Vol. 8, 2013, no. 8, 357-367 Small Zeros of Quadratic Forms Mod P m Ali H. Hakami Deartment of Mathematics, Faculty of Science, Jazan University P.O. Box 277, Jazan, Postal
More informationOn a Markov Game with Incomplete Information
On a Markov Game with Incomlete Information Johannes Hörner, Dinah Rosenberg y, Eilon Solan z and Nicolas Vieille x{ January 24, 26 Abstract We consider an examle of a Markov game with lack of information
More informationLECTURE 7 NOTES. x n. d x if. E [g(x n )] E [g(x)]
LECTURE 7 NOTES 1. Convergence of random variables. Before delving into the large samle roerties of the MLE, we review some concets from large samle theory. 1. Convergence in robability: x n x if, for
More information#A64 INTEGERS 18 (2018) APPLYING MODULAR ARITHMETIC TO DIOPHANTINE EQUATIONS
#A64 INTEGERS 18 (2018) APPLYING MODULAR ARITHMETIC TO DIOPHANTINE EQUATIONS Ramy F. Taki ElDin Physics and Engineering Mathematics Deartment, Faculty of Engineering, Ain Shams University, Cairo, Egyt
More informationA note on the random greedy triangle-packing algorithm
A note on the random greedy triangle-acking algorithm Tom Bohman Alan Frieze Eyal Lubetzky Abstract The random greedy algorithm for constructing a large artial Steiner-Trile-System is defined as follows.
More informationErasure/List Exponents for Slepian Wolf Decoding
Erasure/List Exonents for Sleian Wolf Decoding Neri Merhav May 24, 2013 Deartment of Electrical Engineering Technion - Israel Institute of Technology Technion City, Haifa 32000, ISRAEL E mail: merhav@ee.technion.ac.il
More informationHomework 2: Solution
0-704: Information Processing and Learning Sring 0 Lecturer: Aarti Singh Homework : Solution Acknowledgement: The TA graciously thanks Rafael Stern for roviding most of these solutions.. Problem Hence,
More informationInteractive Decoding of a Broadcast Message
In Proc. Allerton Conf. Commun., Contr., Computing, (Illinois), Oct. 2003 Interactive Decoding of a Broadcast Message Stark C. Draper Brendan J. Frey Frank R. Kschischang University of Toronto Toronto,
More informationHENSEL S LEMMA KEITH CONRAD
HENSEL S LEMMA KEITH CONRAD 1. Introduction In the -adic integers, congruences are aroximations: for a and b in Z, a b mod n is the same as a b 1/ n. Turning information modulo one ower of into similar
More informationDETC2003/DAC AN EFFICIENT ALGORITHM FOR CONSTRUCTING OPTIMAL DESIGN OF COMPUTER EXPERIMENTS
Proceedings of DETC 03 ASME 003 Design Engineering Technical Conferences and Comuters and Information in Engineering Conference Chicago, Illinois USA, Setember -6, 003 DETC003/DAC-48760 AN EFFICIENT ALGORITHM
More informationLecture 6. 2 Recurrence/transience, harmonic functions and martingales
Lecture 6 Classification of states We have shown that all states of an irreducible countable state Markov chain must of the same tye. This gives rise to the following classification. Definition. [Classification
More information1 Extremum Estimators
FINC 9311-21 Financial Econometrics Handout Jialin Yu 1 Extremum Estimators Let θ 0 be a vector of k 1 unknown arameters. Extremum estimators: estimators obtained by maximizing or minimizing some objective
More informationarxiv: v2 [math.na] 6 Apr 2016
Existence and otimality of strong stability reserving linear multiste methods: a duality-based aroach arxiv:504.03930v [math.na] 6 Ar 06 Adrián Németh January 9, 08 Abstract David I. Ketcheson We rove
More informationDistributed Functional Compression through Graph Coloring
Distributed Functional Compression through Graph Coloring Vishal Doshi, Devavrat Shah, Muriel Médard, and Sidharth Jaggi Laboratory for Information and Decision Systems Massachusetts Institute of Technology
More informationA Social Welfare Optimal Sequential Allocation Procedure
A Social Welfare Otimal Sequential Allocation Procedure Thomas Kalinowsi Universität Rostoc, Germany Nina Narodytsa and Toby Walsh NICTA and UNSW, Australia May 2, 201 Abstract We consider a simle sequential
More informationLocation of solutions for quasi-linear elliptic equations with general gradient dependence
Electronic Journal of Qualitative Theory of Differential Equations 217, No. 87, 1 1; htts://doi.org/1.14232/ejqtde.217.1.87 www.math.u-szeged.hu/ejqtde/ Location of solutions for quasi-linear ellitic equations
More informationShadow Computing: An Energy-Aware Fault Tolerant Computing Model
Shadow Comuting: An Energy-Aware Fault Tolerant Comuting Model Bryan Mills, Taieb Znati, Rami Melhem Deartment of Comuter Science University of Pittsburgh (bmills, znati, melhem)@cs.itt.edu Index Terms
More informationEstimation of the large covariance matrix with two-step monotone missing data
Estimation of the large covariance matrix with two-ste monotone missing data Masashi Hyodo, Nobumichi Shutoh 2, Takashi Seo, and Tatjana Pavlenko 3 Deartment of Mathematical Information Science, Tokyo
More informationsubstantial literature on emirical likelihood indicating that it is widely viewed as a desirable and natural aroach to statistical inference in a vari
Condence tubes for multile quantile lots via emirical likelihood John H.J. Einmahl Eindhoven University of Technology Ian W. McKeague Florida State University May 7, 998 Abstract The nonarametric emirical
More informationArithmetic and Metric Properties of p-adic Alternating Engel Series Expansions
International Journal of Algebra, Vol 2, 2008, no 8, 383-393 Arithmetic and Metric Proerties of -Adic Alternating Engel Series Exansions Yue-Hua Liu and Lu-Ming Shen Science College of Hunan Agriculture
More informationAn Introduction to Information Theory: Notes
An Introduction to Information Theory: Notes Jon Shlens jonshlens@ucsd.edu 03 February 003 Preliminaries. Goals. Define basic set-u of information theory. Derive why entroy is the measure of information
More informationInformation Masking and Amplification: The Source Coding Setting
202 IEEE International Symposium on Information Theory Proceedings Information Masking and Amplification: The Source Coding Setting Thomas A. Courtade Department of Electrical Engineering University of
More informationHotelling s Two- Sample T 2
Chater 600 Hotelling s Two- Samle T Introduction This module calculates ower for the Hotelling s two-grou, T-squared (T) test statistic. Hotelling s T is an extension of the univariate two-samle t-test
More informationResearch of PMU Optimal Placement in Power Systems
Proceedings of the 5th WSEAS/IASME Int. Conf. on SYSTEMS THEORY and SCIENTIFIC COMPUTATION, Malta, Setember 15-17, 2005 (38-43) Research of PMU Otimal Placement in Power Systems TIAN-TIAN CAI, QIAN AI
More informationSome Results on the Generalized Gaussian Distribution
Some Results on the Generalized Gaussian Distribution Alex Dytso, Ronit Bustin, H. Vincent Poor, Daniela Tuninetti 3, Natasha Devroye 3, and Shlomo Shamai (Shitz) Abstract The aer considers information-theoretic
More informationOn the Necessity of Binning for the Distributed Hypothesis Testing Problem
On the Necessity of Binning for the Distributed Hypothesis Testing Problem Gil Katz, Pablo Piantanida, Romain Couillet, Merouane Debbah To cite this version: Gil Katz, Pablo Piantanida, Romain Couillet,
More informationGraph Coloring and Conditional Graph Entropy
Graph Coloring and Conditional Graph Entropy Vishal Doshi, Devavrat Shah, Muriel Médard, Sidharth Jaggi Laboratory for Information and Decision Systems Massachusetts Institute of Technology Cambridge,
More informationProbability Estimates for Multi-class Classification by Pairwise Coupling
Probability Estimates for Multi-class Classification by Pairwise Couling Ting-Fan Wu Chih-Jen Lin Deartment of Comuter Science National Taiwan University Taiei 06, Taiwan Ruby C. Weng Deartment of Statistics
More informationOn the Toppling of a Sand Pile
Discrete Mathematics and Theoretical Comuter Science Proceedings AA (DM-CCG), 2001, 275 286 On the Toling of a Sand Pile Jean-Christohe Novelli 1 and Dominique Rossin 2 1 CNRS, LIFL, Bâtiment M3, Université
More informationSome Unitary Space Time Codes From Sphere Packing Theory With Optimal Diversity Product of Code Size
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 5, NO., DECEMBER 4 336 Some Unitary Sace Time Codes From Shere Packing Theory With Otimal Diversity Product of Code Size Haiquan Wang, Genyuan Wang, and Xiang-Gen
More information6196 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 9, SEPTEMBER 2011
6196 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 9, SEPTEMBER 2011 On the Structure of Real-Time Encoding and Decoding Functions in a Multiterminal Communication System Ashutosh Nayyar, Student
More information