Bounds on the Effective-length of Optimal Codes for Interference Channel with Feedback

Size: px
Start display at page:

Download "Bounds on the Effective-length of Optimal Codes for Interference Channel with Feedback"

Transcription

1 Bounds on the Effectve-length of Optmal Codes for Interference Channel wth Feedback Mohsen Hedar EECS Department Unversty of Mchgan Ann Arbor,USA Emal: Farhad Shran ECE Department New York Unversty New York, New York, Emal: S. Sandeep Pradhan EECS Department Unversty of Mchgan Ann Arbor,USA Emal: arxv: v1 [cs.it] 16 Jan 2018 Abstract In ths paper, we nvestgate the necessty of fnte blocklength codes n dstrbuted transmsson of ndependent message sets over channels wth feedback. Prevously, t was shown that fnte effectve length codes are necessary n dstrbuted transmsson and compresson of sources. We provde two examples of three user nterference channels wth feedback where codes wth asymptotcally large effectve lengths are suboptmal. As a result, we conclude that coded transmsson usng fnte effectve length codes s necessary to acheve optmalty. We argue that the sub-optmal performance of large effectve length codes s due to ther neffcency n preservng the correlaton between the nputs to the dstrbuted termnals n the communcaton system. Ths correlaton s made avalable by the presence of feedback at the termnals and s used as a means for coordnaton between the termnals when usng fnte effectve length codng strateges. I. INTRODUCTION Most of the codng strateges developed n nformaton theory are based on random code ensembles whch are constructed usng ndependent and dentcally dstrbuted (IID) sequences of random varables [1] [4]. The codes assocated wth dfferent termnals n the network are mutually ndependent. Moreover, the blocklengths assocated wth these codes are asymptotcally large. Ths allows the applcaton of the laws of large numbers and concentraton of measure theorems when analyzng the performance of codng strateges; and leads to characterzatons of ther achevable regons n terms of nformaton quanttes that are the functonals of the underlyng dstrbuton used to construct the codes. These characterzatons are often called sngle-letter characterzatons. Although the orgnal problem s to optmze the performance of codes wth asymptotcally large blocklengths, the soluton s characterzed by a functonal (such as mutual nformaton) of just one realzaton of the source or the channel under consderaton. It s well-known that unstructured random codes wth asymptotcally large blocklength can be used to acheve optmalty n terms of achevable rates n pont-to-pont communcatons. In fact, t can be shown that large blocklength codes are necessary to approach optmal performance. At a hgh level, ths s due to the fact that the effcency of fundamental tasks of communcaton such as coverng and packng ncreases as the nput dmenson s ncreased [5]. In network communcaton, one needs to (a) remove redundancy among correlated nformaton sources [2], [4] n a dstrbuted manner n the source codng problems, and (b) nduce redundancy among dstrbuted termnals to facltate [1], [3] cooperaton among them. For example, n the network source codng problems such as dstrbuted source codng and multple descrpton codng, the objectve s to explot the statstcal correlaton of the dstrbuted nformaton sources. Smlarly, n the network channel codng problems, such as the nterference channels and broadcast channels, correlaton of nformaton among dfferent termnals s nduced for better cooperaton among them. At a hgh level, n addton to the basc objectves of effcent packng and coverng at every termnal, the network codng strateges need to explot statstcal correlaton among dstrbuted nformaton sources or nduce statstcal correlaton among nformaton accessed by termnals n the network. Wtsenhausen [6] and Gacs-Korner [7] made the observaton that dstrbuted processng of pars of sequences of random varables leads to outputs whch are less correlated than the orgnal nput sequences. In the network communcatons context, ths mples that the outputs of encodng functons at dfferent termnals n a network are less correlated wth each other than the orgnal nput sequences. In [8], [9], we bult upon these observatons and showed that the correlaton between the outputs of pars of encodng functons operatng over correlated sequences s nversely proportonal to the effectve length of the encodng functons. Based on these results, t can be concluded that whle random unstructured codng strateges wth asymptotcally large blocklengths are effcent n performng the tasks of coverng and packng, they are neffcent n facltatng coordnaton between dfferent termnals. Usng these results, we showed that fnte effectve codes are necessary to acheve optmalty n varous setups nvolvng the transmsson of correlated sources. Partcularly, we showed that the effectve length of optmalty achevng codes s bounded from above n the dstrbuted source codng problem as well as the problem of transmsson of correlated sources over the multple access channel (MAC) and the nterference channel (IC) [8], [10]. So far, all of the results showng the necessty of fnte effectve length codes pertan to stuatons nvolvng the ds-

2 trbuted transmsson of sources over channels and dstrbuted compresson of sources. However, the queston of whether such codes are necessary n mult-termnal channel codng has remaned open. The reason s that the applcaton of the results n [8], [9] requres the presence of correlated nputs n dfferent termnals of the network. In the case of dstrbuted processng of sources, such correlaton s readly avalable n the form of the dstrbuted source. Whereas, n dstrbuted transmsson of ndependent message t s unclear how such a correlaton can be created and exploted. In ths work, we argue that n channel codng wth feedback, correlaton s nduced because of the feedback lnk. More precsely, the feedback sequence at one termnal s correlated wth the message set n the other termnal. In order to explot ths correlaton effcently, fnte effectve length codes are necessary. The contrbutons of ths paper can be summarzed as follows. We provde two examples of nterference channels wth feedback where fnte effectve length codes are necessary to approach optmalty. For each of these examples, we provde an outer bound on the achevable regon as a functon of the effectve-length of the encodng functons used at the transmtters. Furthermore, we use fnte effectve length codes to prove the achevablty of certan rate vectors whch le outsde of the outer bound when the effectve length s large. The combnaton of these two results shows that n these examples any codng strategy whch uses encodng functons wth asymptotcally large effectve lengths s sub-optmal. The rest of the paper s organzed as follows: In Secton II we ntroduce the problem formulaton. Secton III provdes the pror results whch are used n ths paper. Secton IV explans our man results. Fnally, secton V concludes the paper. A. Notatons II. DEFINITIONS AND MODEL Random varables are denoted usng captal letters such as X,Y. The random vector (X 1,X 2,...,X n ) s represented by X n. Smlarly, we use underlne letters to denote vectors of numbers and functons. For shorthand, vectors are sometmes represented usng underlne letters wthout any superscrpt such as X,f, and a. Callgraphc letters such as C and M are used to represent sets. B. Model The problem of Interference Channel wth Feedback (IC- FB) s studed n [11] and [12]. A three-user nterference channel wth generalzed feedback (IC-FB) s characterzed by three nput alphabets (X 1,X 2,X 3 ), three output alphabets (Y 1,Y 2,Y 3 ), three feedback alphabets (Z 1,Z 2,Z 3 ), and transton probablty dstrbutons (Q Y X,P Z Y ). We assume that all the alphabets are fnte and that the channel s memoryless. Let x n,yn,zn, [1,3], be the channel nputs, outputs and Enc. 1 Enc. 2 Enc. 3 Z 1 X 1 X 2 X 3 Z 3 P Z1jY1 Q Y1Y2Y3jX1X2X3 P Z3jY3 Y 1 Y 2 Y 3 Dec. 1 Dec. 2 Dec. 3 Fg. 1. An nstance of the three-user IC wth generalzed feedback. Here transmtters 1 and 3 receve nosy feedback, whereas transmtter 2 does not receve feedback. the channel feedback after n uses of the channel, respectvely. The memoryless property mples that: p(y j,n,z j,n,j [1,3] y n 1,z n 1,x n, [1,3]) = Q Y X (y 1,n,y 2,n,y 3,n x 1,n,x 2,n,x 3,n ) P Z Y (z 1,n,z 2,n,z 3,n y 1,n,y 2,n,y 3,n ) In the three user IC-FB, there are three transmtters and three recevers. The th transmtter [1,3], ntends to transmt the message ndex W to the th recever. It s also assumed that the feedback Z, [1,3] s causally avalable at transmtter wth one unt of delay. An example of such setup s depcted n Fgure 1. In ths fgure, Z 2 s trval, and P Z1,Z 3 Y 1,Y 2,Y 3 = P Z1 Y 1 P Z3 Y 3 (.e. the second transmtter does not receve any feedback). For postve ntegers M 1,M 2,M 3 and N are arbtrary postve ntegers. Defnton 1. A (M 1,M 2,M 3,N) feedback-block-code for the three user IC-FB conssts of Three sets of messages M = {1,2,...,M }, [1,3]. Three sequences of encodng functons f,n : M Z n 1 X, 1 n N, Three decodng functons: g : Y N M, [1,3]. The message for transmtter s denoted by a random varable W. It s assumed that the messages W are mutually ndependent and unformly dstrbuted on M, [1,3]. The output of the th transmtter at the nth use of the channel s denoted by X,n = f,n (W,Z n 1 ). The rate-trple of an (M 1,M 2,M 3,N) code s defned as R = logm N, [1,3]. Let Ŵ, [1,3], be the decoded message at the recever. Then, the probablty of the error s defned as P e P((W 1,W 2,W 3 ) (Ŵ1,Ŵ2,Ŵ3)). For ths problem, at tme n, each transmtter can chose an encodng functon randomly usng a probablty measure defned over the set of all encodng functons f N descrbed

3 n Defnton 1. The followng defnes a randomzed codng strategy. Defnton 2. A(M 1,M 2,M 3,N)-randomzed codng strategy s characterzed by a probablty measure P N on the set of all functons (f N ), [1,3] descrbed n Defnton 1. Next, we defne the achevable regon for the three user IC-FB. Defnton 3. For ǫ > 0, a rate-trple (R 1,R 2,R 3 ) s sad to be ǫ-achevable by a feedback-block-code wth parameters (M 1,M 2,M 3,N), f the followng condtons are satsfed. P e ǫ, 1 N logm R ǫ, [1,3]. Defnton 4. For ǫ > 0, a rate-trple (R 1,R 2,R 3 ) s sad to be ǫ-achevable by a (M 1,M 2,M 3,N)-randomzed codng strategy wth probablty measure P, f, wth probablty one wth respect to P, there exsts a feedback-block-code for whch (R 1,R 2,R 3 ) s ǫ-achevable. Defnton 5. For ǫ > 0, a rate-trple (R 1,R 2,R 3 ) s sad to be ǫ-achevable, f there exst a (M 1,M 2,M 3,N) feedback-block-code (randomzed codng strategy) for whch (R 1,R 2,R 3 ) s ǫ-achevable. Defnton 6. A rate-trple (R 1,R 2,R 3 ) s sad to be achevable, f t sǫ-achevable for anyǫ > 0. Gven an IC-FB, the set of all achevable rate-trples s called the feedback-capacty. III. BACKGROUND AND PRIOR RESULTS In ths secton, we summarze the results n [8] on the correlaton between the outputs of Boolean functons of pars of sequences of random varables. These results are used n the next secton to prove the necessty of fnte effectve length codes. Defnton 7. (X,Y) s called a par of DMS s f we have P Xn,Y n(xn,y n ) = [1,n] P X,Y (x,y ), n N,x n X n,y n Y n, where P X,Y = P X,Y, [1,n], for some jont dstrbuton P X,Y. Defnton 8. A Bnary-Block-Encoder (BBE) s characterzed by the trple (e,x,n), where e s a mappng e : X n {0,1} n, X s a fnte set, and n s an nteger. Defnton 9. For a BBE (e, X, n) and DMS X, let P (e (X n ) = 1) = q. For each Boolean functone, [1,n], the real-valued functon correspondng to e s defned as follows: { ẽ (X n 1 q, f e (X n ) = 1, ) = (1) q. otherwse. Defnton 10. For a BBE (e,x,n), defne the decomposton ẽ = ẽ, whereẽ = E X n X (ẽ X ) j<ẽj. Then,ẽ s the component of ẽ whch s only a functon of {X j j = 1}. The collecton {ẽ j [1,n] j = k}, s called the set of k-letter components of ẽ. Defnton 11. For a functon e : X n {0,1}, wth real decomposton vector (ẽ ) {0,1} n, the dependency spectrum s defned as the vector (P ) {0,1} n of the varances, where P = Var(ẽ ), {0,1} n. The effectve length s defned as the expected value L = 1 n {0,1} w H() P n, where w H ( ) s the Hammng weght. Lemma 1. Let ψ sup(e(e(x)f(y)), where the supremum s taken over all sngle-letter functons e : X R, and f : Y R such that h(x) and g(y) have unt varance and zero mean. the followng bound holds: 2 P Q 2 C P 1 2 Q 1 2 P(e(X n ) f(y n )) 1 2 P Q +2 C P 1 2 Q 1 2, where 1) C ψ N, 2) P s the varance of ẽ, 3) ẽ s the real functon correspondng to e, 4) Q s the varance of f, and 5) N w H (). Remark 1. The value C s decreasng n N. So, P(e(X n ) f(y n )), s maxmzed when most of the varance P s dstrbuted on ẽ whch have lower N (.e. operate on smaller blocks). Ths mples that encodng functons wth smaller effectve-lengths can have hgher correlaton between ther outputs. IV. MAIN RESULTS In ths secton, we ntroduce two examples of three user IC-FBs where fnte effectve length codes are necessary to approach optmalty. Example 1. Consder the setup shown n Fgure 2. Here, (X 11,X 12 ),X 2,(X 32,X 33 ) are the outputs of the th Encoder [1,3], respectvely. The channel outputs Y 1, (Y 2,Y 2 ), and Y 3 are receved at decoders 1,2 and 3, respectvely. The channel correspondng to the transton probabltyp Y 2 X 12X 32 s descrbed by the followng relaton: Y 2 = X 12 +N δ +(X 12 X 32 ) E, where N δ and E are ndependent Bernoull random varables wth P(N δ = 1) = δ and P(E = 1) = 1 2. Also, the random varablesn ǫ, andn p n Fgure 2 are Bernoull random varables wth P(N ǫ = 1) = ǫ and P(N p = 1) = p, respectvely. The varables N δ,e,n ǫ and N P are mutually ndependent. In ths setup feedback s only avalable at encoder 1 and 3. The feedback at the frst transmtter s Z 1 = Y 1 wth probablty one. The feedback at the thrd transmtter s Z 3 = Y 3 wth probablty one. In other words the two transmtters receve noseless feedback. The followng provdes an outer bound on the achevable rates after n channel uses. The bound s provded as a functon of the average probablty of agreement between the encoder outputs X 12 and X 32.

4 Z 1 Enc. 1 Enc. 2 Enc. 3 Z 3 N p X 11 Y 1 X 12 X 2 X 32 X 33 P Y 0 2 jx12x32 N p N p N ǫ Y 0 2 Y 2 Y 3 Dec. 1 Dec. 2 Dec. 3 Fg. 2. The dagram of the IC-FB gven n Example 1. In ths setup, Z 1 s the feedback at Transmtter 1, and Z 3 s the feedback at transmtter 3. Theorem 1. Any (M 1,M 2,M 3,n)-randomzed codng strategy for the channel n Example 1 acheves a rate vector (R 1,R 2,R 3 ) satsfyng the followng nequaltes: R 1 1 h(p), R 2 1 h b (p) (1 h b (δ))( 1 n R 3 1 h b (p ǫ). n P(X 12, = X 32, ) + ), =1 Proof. The proof s gven n Appendx A. Corollary 1. Defne the set R as the unon of all rate-trples (R 1,R 2,R 3 ) such that R 1 1 h b (p), R 2 1 h b (p) (1 h b (δ)) +, R 3 1 h b (p ǫ). Then, the feedback-capacty of the channel n Example 1 s contaned n R. Corollary 2. Suppose p and δ are such that h(p) 1 h(δ), and ǫ = 0. Then the feedback-capacty of Example 1 s characterzed by the followng R 1 1 h(p),r 2 1,R 3 1 h(p). Proof. The converse follows by Theorem 1. For the achevablty, we use standard Shannon random codes at encoder 1 and 3. Then, the rates R 1 1 h(p), and R 3 1 h(p) are achevable. Because of the feedbacks Z 1,Z 3, the nose N p s avalable at encoder 1 and 3. Transmtters one and three send N p to recever 2. We requre N p to be decoded at recever 2 losslessly. Consder a good source-channel code for transmsson of N p over a Bnary Symmetrc Channel wth nose bas δ. We use ths codebook both at encoder 1 and 3. Snce the source N p and the codebook are avalable at encoder 1 and 3, then X 12 = X 32 wth probablty one. As a result, the channel P Y 2 X 12X 32 becomes a bnary symmetrc channel wth bas δ. Therefore, as h b (p) 1 h(δ) then N p s reconstructed at recever 2 wthout any nose. By subtractng N p from Y 2 the channel from X 2 to Y 2 becomes a nosless channel. Thus, R 2 = 1 s achevable. Lemma 2. Let C ǫ denote the feedback-capacty regon of the IC-FB n Example 1. For any(r 1,R 2,R 3 ) C 0, there exsts a contnuous functon ζ(ǫ) such that for suffcently small ǫ > 0 the rate-trple (R 1 ζ(ǫ),r 2 ζ(ǫ),r 3 ζ(ǫ)) C ǫ, where ζ(ǫ) 0, as ǫ 0. Proof. The proof s gven n Appendx B. Theorem 2. There exst γ > 0 and ǫ > 0, such that for any codng strategy achevng the rate-trple (1 h b (p),1 γ,1 h b (p)) the effectve length of the encodng functons producng X 12 and X 32 are bounded from above by a constant. Furthermore, the effectve length s greater than 1 (.e. uncoded transmsson s not optmal). proof outlne. From Theorem 1 the followng upper-bound holds for R 2. R 2 1 h b (p)(1 1 n P(X 12, = X 32, )) n Therefore, t s requred that 1 N =1 N P(X 12, = X 32, ) 1. =1 Ths mples that n N, P(X 12,n = X 32,n ) 1. However, by Lemma 1, ths requres that the effectve length be bounded from above. If the effectve length s equal to 1, then P n 1 for all n N, ths mples that P(F 1,n (Z1 n 1 ) = Z 1,n 1 ) 1. Thus, P(Y 2,n = N p +N δ ) 1. However, 1 N H(NN p,n 0 Y N 2,N0 ) (1 N 0 N )(2h b(p) h b (p p)) 2h b (p) h b (p p) > 0 As a result t s not possble to reconstruct N p at the decoder losslessly. More precsely, R 2 1+h b (p p) 2h b (p) Ths contradcts wth R 2 1. A. The Second Example In ths subsecton, we provde another example to llustrate more the necessty of codng strategy wth effectve fnte length for communcatons over IC-FB. Example 2. Consder the IC shown n 3. The outputs of encoder 1 are denoted by (X 11,X 12 ), the output of encoder 2 s X 2, and the outputs of encoder 3 are (X 32,X 33,X 33 ). In ths setup, Z 1 and Z 2 represent the feedback avalable at encoder 1 and encoder 3, respectvely. All the nputs alphabets n ths channel are bnary. All the output alphabets are bnary; except Y 1 whch a ternary. In ths setup N 1, N 3,N δ,n ǫ and E are mutually ndependent Bernoull random varables wth parameter p 1,p 3,δ,ǫ, and 1/2, respectvely. Fnally, t s assumed that p 1,p 3,δ,ǫ < 1/2.

5 Z 1 Enc. 1 N ǫ E X 11 Y 1 X 12 N 1 3 Dec. 1 d s dscontnuous as a functon of ǫ. Ths mples that the achevable rates usng sngle letter codng scheme strctly decreases comparng to the case when ǫ = 0. Hence, there exsts a γ > 0 such that any rate-trple (R 1,R 2,R 3 ) wth R 2 > 1 h b (δ) γ s not achevable usng sngle-letter codng strateges, where d s as n Lemma 3. More precsely, the followng Lemma holds. Enc. 2 X 2 X 32 N δ E N 3 N δ Y 12 Y 22 Y 32 Dec. 2 Lemma 4. There exst γ > 0 and ǫ > 0, such that for any codng strategy achevng the rate-trple (log 3 1,1 h b (d) γ,1 h b (δ)) the effectve length of the encodng functons producng X 12 and X 32 are bounded from above by a constant. Furthermore, the effectve length s greater than 1 (.e. uncoded transmsson s not optmal). Enc. 3 Z 3 X 33 X33 0 E Y 33 Y 0 33 Dec. 3 Proof. The proof for ths lemma follows from a smlar argument as n Theorem 2. V. CONCLUSION Fg. 3. The dagram of the IC-FB n Example 2. In ths setup, Z 1, the feedback at Transmtter 1, s a nosy verson of Y 1. We frst study the case n whch ǫ = 0. The followng lemma provdes an achevable rates for ths example. Lemma 3. For ǫ = 0 n the setup gven n Example 2, the rate-trple (log3 1,1 h b (d),1 h b (δ)) s achevable, where d = h 1 b ( h b (p 1 δ)+h(p 3 ) 1 + ). proof outlne. The bounds on R 1,R 3 follow from the standard arguments as n pont-to-pont channel codng (Fano s nequalty). Next, we show the bound on R 2. Upon recevng Z 1, the frst encoder recovers E. The thrd encoder receves Z 3 and recovers (E,N δ ). Encoder 1 and 3 employ a sourcechannel codng scheme to encode the sources E,N δ such that decoder 2 be able to reconstruct E +N δ wthn a Hammng dstorton d. Ths s a smlar problem to the Common-Bt One-Help-One Problem ntroduced n [13]. Usng the results from [13] (Theorem 3), we can show that decoder 2 s able to reconstruct E +N δ wthn a Hammng dstorton d, f the bounds R 32 h b ( δ) h b (d), and R 12 1 h b ( ), hold for some 0 1/2. From standard channel codng arguments the transmtted codewords from encoder 1 and 3 are decoded at recever 2 wth small probablty of error, f R 12 1 h b (p 1 ) ζ, R 32 1 h b (p 3 ) ζ, where ζ > 0 s a suffcently small number. Fnally the proof follows by settng p 1, and d as n the statement of the lemma. For the case when ǫ > 0, there s no common nformaton between encoder 1 and 3. From the dscontnuty argument as n [13], we can show that the mnmum dstorton level We provded two examples of channel codng wth feedback over nterference networks where fnte effectve length codng s necessary to acheve optmal performance. We showed that n these examples, optmalty achevng codng strateges utlze the feedback avalable n dfferent termnals to coordnate ther outputs. We showed that codng strateges wth asymptotcally large effectve lengths are neffcent n preservng the correlaton among ther outputs and are hence unable to coordnate ther nputs to the channel effectvely. APPENDIX A PROOF OF THEOREM 1 Proof. The boundsr 1 1 h b (p) andr 3 1 h b (p) follows from standard arguments as n pont-to-pont channel codng problem. Note that the feedback does not ncrease the rate of R 1 and R 3 snce these upper-bounds correspond to the pontto-pont capacty and feedback does not ncrease pont-to-pont capacty. To bound R 2 we use Fano s nequalty. Therefore, ; nr 2 H(W 2 ) (a) = H(W 2 Y n 2 ) (b) I(W 2 ;Y n 2 Y n = H(Y n 2 Y n 2 ) H(Y n 2 W 2,Y n (c) = H(Y n 2 Y n 2 ) H(Y n 2 W 2,X n 2,Y n (d) n H(Y n 2 W 2,X n 2,Y n (e) = n H(N n p Y n, (2) where (a), (c) and (e) follow from the fact that X n 12,Xn 32,Y n 2 are ndependent of W 2 and that X n 2 s a functon of W 2 snce the second transmtter does not receve feedback. (b) follows from Fano s nequalty and (d) follows from the fact that Y 2 s bnary.

6 Defne the random vector Z, [1,n] as the ndcator functon of the event that X 12, = X 32,. Then, H(Np n Y n 2 ) H(Np n Y n 2,Z n ) = p(z n = z)h(np n Y n 2,z) z {0,1} n For the nnermost term n the above nequalty we have: H(N n p Y n 2,z) (a) = H(N n p Xz 12 Nz δ ) = H(N n p,x z 12 Nz δ ) H(Xz 12 Nz δ ) (b) H(N n p,xz 12 Nz δ ) w H(z) (c) H(N n p,nz δ ) w H(z) = H(N n p)+h(n z δ ) w H(z) = nh b (p) w H (z)(1 h b (δ)), (3) where (a) follows from the defnton of Y 2, (b) follows from the fact that the bnary entropy s upper bounded by one and fnally, (c) follows form the fact that X 12 s ndependent of N δ. Combnng equatons (2) and (3), we get: H(N n p Y n 2 ) n h b (p) (1 h b (δ)) 1 n E[w H(z)] + APPENDIX B PROOF OF LEMMA 2 Proof. We can show that C ǫ s the set of all rate-trples (R 1,R 2,R 3 ) for whch N N such that R 1 1 N I(XN 1 ;Y N 1 ) R 3 1 N I(XN 33X N 33 ;Y N 33Y N 33 ) R 2 1 N I(XN 2 ;Y N 2 Y N 2 ). where the jont dstrbuton of the varables s n some set P ǫ. The proof for ths statement follows by a converse and an achevablty argument. For the converse, we use Fanoe s nequalty as n Theorem 1. The achevablty s straghtforward and follows by employng a mult-letter random codng scheme. For the case when ǫ = 0, and any achevable rate-trple (R 1,R 2,R 3 ) there exst N and γ > 0 such that R 1 1 h b (p) γ (4) R 3 1 h b (p) γ (5) R 2 1 N I(XN 2 ;Y N 2 Y N 2 ) γ, (6) where the jont dstrbuton of the random varables nvolved s denoted by P 0 P 0. Snce the bnary entropy functon s contnuous, then there exsts ζ(ǫ) such that 1 h b (p) 1 h b (p ǫ)+ζ(ǫ). Next, we show contnuty for the rghthand sde of (6). Fx N, P ǫ P ǫ, and consder the thrd nequalty n the characterzaton of C ǫ. Note that the only probablty dstrbuton dependng on ǫ s P(y 3,n x 3,n ). Snce ths condtonal probablty s contnuous wth ǫ then so s P ǫ. Thus, for any fxed N, the bound on R 2 n C ǫ s contnuous as a functon functon of ǫ. As a result, there exsts a functon ζ (ǫ) such that the rght-hand sde of the nequalty (6) s upper bounded by 1 N I(XN 2 ;Y 2 N Y N 2 ) + ζ (ǫ) γ for some jont dstrbuton P ǫ P ǫ. As a result, the followng bounds hold for (R 1,R 2,R 3 ). R 1 1 h b (p)+ζ(ǫ) γ R 3 1 h b (p ǫ)+ζ(ǫ) γ R 2 1 N I(XN 2 ;Y N 2 Y N 2 )+ζ(ǫ) γ. Ths mples that there exsts ǫ > 0 suffcently small such that (R 1 ζ(ǫ),r 2 ζ(ǫ),r 3 ζ(ǫ)) C ǫ. Thus, we establsh the contnuty of C ǫ at ǫ = 0. REFERENCES [1] Te Han and K. Kobayash. A new achevable rate regon for the nterference channel. IEEE Transactons on Informaton Theory, 27(1):49 60, Jan [2] S. Y. Tung. Multtermnal Source Codng. PhD thess, Cornell Unversty, Ithaca, NY, [3] K. Marton. A codng theorem for the dscrete memoryless broadcast channel. IEEE Transactons on Informaton Theory, 25(3): , May [4] Zhen Zhang and T. Berger. New results n bnary multple descrptons. IEEE Transactons on Informaton Theory, 33(4): , Jul [5] I. Csszár and J. Korner. Informaton Theory: Codng Theorems for Dscrete Memoryless Systems. Academc Press Inc. Ltd., [6] H. S. Wtsenhausen. On sequences of par of dependent random varables. SIAM Journal of Appled Mathematcs, 28(1): , [7] P. Gacs and J. Körner. Common nformaton s far less than mutual nformaton. Problems of Control and Informaton Theory, 2(2): , [8] Farhad Shran and S Sandeep Pradhan. On the sub-optmalty of sngle-letter codng n mult-termnal communcatons. arxv preprnt arxv: , [9] Farhad Shran Chaharsoogh and S Sandeep Pradhan. On the correlaton between boolean functons of sequences of random varables. In Informaton Theory (ISIT), 2017 IEEE Internatonal Symposum on, pages IEEE, [10] F. Shran and S. S. Pradhan. Fnte block-length gans n dstrbuted source codng. In 2014 IEEE Internatonal Symposum on Informaton Theory, pages , June [11] S. Yang and D. Tunnett, Interference Channel Wth Generalzed Feedback (a.k.a. Wth Source Cooperaton): Part I: Achevable Regon," n IEEE Transactons on Informaton Theory, vol. 57, no. 5, pp , May [12] G. Kramer, Feedback strateges for whte Gaussan nterference networks," IIEEE Trans. Inf. Theory, vol. 48, no. 6, pp , Jun [13] A. B. Wagner, B. G. Kelly and Y. Altug, Dstrbuted Rate-Dstorton Wth Common Components," n IEEE Transactons on Informaton Theory, vol. 57, no. 7, pp , July 2011.

On the Correlation between Boolean Functions of Sequences of Random Variables

On the Correlation between Boolean Functions of Sequences of Random Variables On the Correlaton between Boolean Functons of Sequences of Random Varables Farhad Shran Chaharsoogh Electrcal Engneerng and Computer Scence Unversty of Mchgan Ann Arbor, Mchgan, 48105 Emal: fshran@umch.edu

More information

Lecture 3: Shannon s Theorem

Lecture 3: Shannon s Theorem CSE 533: Error-Correctng Codes (Autumn 006 Lecture 3: Shannon s Theorem October 9, 006 Lecturer: Venkatesan Guruswam Scrbe: Wdad Machmouch 1 Communcaton Model The communcaton model we are usng conssts

More information

Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:

More information

Rate-Memory Trade-off for the Two-User Broadcast Caching Network with Correlated Sources

Rate-Memory Trade-off for the Two-User Broadcast Caching Network with Correlated Sources Rate-Memory Trade-off for the Two-User Broadcast Cachng Network wth Correlated Sources Parsa Hassanzadeh, Antona M. Tulno, Jame Llorca, Elza Erkp arxv:705.0466v [cs.it] May 07 Abstract Ths paper studes

More information

ECE 534: Elements of Information Theory. Solutions to Midterm Exam (Spring 2006)

ECE 534: Elements of Information Theory. Solutions to Midterm Exam (Spring 2006) ECE 534: Elements of Informaton Theory Solutons to Mdterm Eam (Sprng 6) Problem [ pts.] A dscrete memoryless source has an alphabet of three letters,, =,, 3, wth probabltes.4,.4, and., respectvely. (a)

More information

EGR 544 Communication Theory

EGR 544 Communication Theory EGR 544 Communcaton Theory. Informaton Sources Z. Alyazcoglu Electrcal and Computer Engneerng Department Cal Poly Pomona Introducton Informaton Source x n Informaton sources Analog sources Dscrete sources

More information

More metrics on cartesian products

More metrics on cartesian products More metrcs on cartesan products If (X, d ) are metrc spaces for 1 n, then n Secton II4 of the lecture notes we defned three metrcs on X whose underlyng topologes are the product topology The purpose of

More information

State Amplification and State Masking for the Binary Energy Harvesting Channel

State Amplification and State Masking for the Binary Energy Harvesting Channel State Amplfcaton and State Maskng for the Bnary Energy Harvestng Channel Kaya Tutuncuoglu, Omur Ozel 2, Ayln Yener, and Sennur Ulukus 2 Department of Electrcal Engneerng, The Pennsylvana State Unversty,

More information

Chapter 7 Channel Capacity and Coding

Chapter 7 Channel Capacity and Coding Wreless Informaton Transmsson System Lab. Chapter 7 Channel Capacty and Codng Insttute of Communcatons Engneerng atonal Sun Yat-sen Unversty Contents 7. Channel models and channel capacty 7.. Channel models

More information

Chapter 7 Channel Capacity and Coding

Chapter 7 Channel Capacity and Coding Chapter 7 Channel Capacty and Codng Contents 7. Channel models and channel capacty 7.. Channel models Bnary symmetrc channel Dscrete memoryless channels Dscrete-nput, contnuous-output channel Waveform

More information

Lecture 14 (03/27/18). Channels. Decoding. Preview of the Capacity Theorem.

Lecture 14 (03/27/18). Channels. Decoding. Preview of the Capacity Theorem. Lecture 14 (03/27/18). Channels. Decodng. Prevew of the Capacty Theorem. A. Barg The concept of a communcaton channel n nformaton theory s an abstracton for transmttng dgtal (and analog) nformaton from

More information

ECE559VV Project Report

ECE559VV Project Report ECE559VV Project Report (Supplementary Notes Loc Xuan Bu I. MAX SUM-RATE SCHEDULING: THE UPLINK CASE We have seen (n the presentaton that, for downlnk (broadcast channels, the strategy maxmzng the sum-rate

More information

Error Probability for M Signals

Error Probability for M Signals Chapter 3 rror Probablty for M Sgnals In ths chapter we dscuss the error probablty n decdng whch of M sgnals was transmtted over an arbtrary channel. We assume the sgnals are represented by a set of orthonormal

More information

j) = 1 (note sigma notation) ii. Continuous random variable (e.g. Normal distribution) 1. density function: f ( x) 0 and f ( x) dx = 1

j) = 1 (note sigma notation) ii. Continuous random variable (e.g. Normal distribution) 1. density function: f ( x) 0 and f ( x) dx = 1 Random varables Measure of central tendences and varablty (means and varances) Jont densty functons and ndependence Measures of assocaton (covarance and correlaton) Interestng result Condtonal dstrbutons

More information

VQ widely used in coding speech, image, and video

VQ widely used in coding speech, image, and video at Scalar quantzers are specal cases of vector quantzers (VQ): they are constraned to look at one sample at a tme (memoryless) VQ does not have such constrant better RD perfomance expected Source codng

More information

Refined Coding Bounds for Network Error Correction

Refined Coding Bounds for Network Error Correction Refned Codng Bounds for Network Error Correcton Shenghao Yang Department of Informaton Engneerng The Chnese Unversty of Hong Kong Shatn, N.T., Hong Kong shyang5@e.cuhk.edu.hk Raymond W. Yeung Department

More information

Entropy Coding. A complete entropy codec, which is an encoder/decoder. pair, consists of the process of encoding or

Entropy Coding. A complete entropy codec, which is an encoder/decoder. pair, consists of the process of encoding or Sgnal Compresson Sgnal Compresson Entropy Codng Entropy codng s also known as zero-error codng, data compresson or lossless compresson. Entropy codng s wdely used n vrtually all popular nternatonal multmeda

More information

Edge Isoperimetric Inequalities

Edge Isoperimetric Inequalities November 7, 2005 Ross M. Rchardson Edge Isopermetrc Inequaltes 1 Four Questons Recall that n the last lecture we looked at the problem of sopermetrc nequaltes n the hypercube, Q n. Our noton of boundary

More information

Lossy Compression. Compromise accuracy of reconstruction for increased compression.

Lossy Compression. Compromise accuracy of reconstruction for increased compression. Lossy Compresson Compromse accuracy of reconstructon for ncreased compresson. The reconstructon s usually vsbly ndstngushable from the orgnal mage. Typcally, one can get up to 0:1 compresson wth almost

More information

Approximately achieving Gaussian relay network capacity with lattice codes

Approximately achieving Gaussian relay network capacity with lattice codes Approxmately achevng Gaussan relay network capacty wth lattce codes Ayfer Özgür EFL, Lausanne, Swtzerland ayfer.ozgur@epfl.ch Suhas Dggav UCLA, USA and EFL, Swtzerland suhas@ee.ucla.edu Abstract Recently,

More information

Channel Encoder. Channel. Figure 7.1: Communication system

Channel Encoder. Channel. Figure 7.1: Communication system Chapter 7 Processes The model of a communcaton system that we have been developng s shown n Fgure 7.. Ths model s also useful for some computaton systems. The source s assumed to emt a stream of symbols.

More information

NP-Completeness : Proofs

NP-Completeness : Proofs NP-Completeness : Proofs Proof Methods A method to show a decson problem Π NP-complete s as follows. (1) Show Π NP. (2) Choose an NP-complete problem Π. (3) Show Π Π. A method to show an optmzaton problem

More information

The Order Relation and Trace Inequalities for. Hermitian Operators

The Order Relation and Trace Inequalities for. Hermitian Operators Internatonal Mathematcal Forum, Vol 3, 08, no, 507-57 HIKARI Ltd, wwwm-hkarcom https://doorg/0988/mf088055 The Order Relaton and Trace Inequaltes for Hermtan Operators Y Huang School of Informaton Scence

More information

Games of Threats. Elon Kohlberg Abraham Neyman. Working Paper

Games of Threats. Elon Kohlberg Abraham Neyman. Working Paper Games of Threats Elon Kohlberg Abraham Neyman Workng Paper 18-023 Games of Threats Elon Kohlberg Harvard Busness School Abraham Neyman The Hebrew Unversty of Jerusalem Workng Paper 18-023 Copyrght 2017

More information

APPENDIX A Some Linear Algebra

APPENDIX A Some Linear Algebra APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,

More information

Pulse Coded Modulation

Pulse Coded Modulation Pulse Coded Modulaton PCM (Pulse Coded Modulaton) s a voce codng technque defned by the ITU-T G.711 standard and t s used n dgtal telephony to encode the voce sgnal. The frst step n the analog to dgtal

More information

Lectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix

Lectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could

More information

A note on almost sure behavior of randomly weighted sums of φ-mixing random variables with φ-mixing weights

A note on almost sure behavior of randomly weighted sums of φ-mixing random variables with φ-mixing weights ACTA ET COMMENTATIONES UNIVERSITATIS TARTUENSIS DE MATHEMATICA Volume 7, Number 2, December 203 Avalable onlne at http://acutm.math.ut.ee A note on almost sure behavor of randomly weghted sums of φ-mxng

More information

Quantum and Classical Information Theory with Disentropy

Quantum and Classical Information Theory with Disentropy Quantum and Classcal Informaton Theory wth Dsentropy R V Ramos rubensramos@ufcbr Lab of Quantum Informaton Technology, Department of Telenformatc Engneerng Federal Unversty of Ceara - DETI/UFC, CP 6007

More information

A new construction of 3-separable matrices via an improved decoding of Macula s construction

A new construction of 3-separable matrices via an improved decoding of Macula s construction Dscrete Optmzaton 5 008 700 704 Contents lsts avalable at ScenceDrect Dscrete Optmzaton journal homepage: wwwelsevercom/locate/dsopt A new constructon of 3-separable matrces va an mproved decodng of Macula

More information

Complete subgraphs in multipartite graphs

Complete subgraphs in multipartite graphs Complete subgraphs n multpartte graphs FLORIAN PFENDER Unverstät Rostock, Insttut für Mathematk D-18057 Rostock, Germany Floran.Pfender@un-rostock.de Abstract Turán s Theorem states that every graph G

More information

College of Computer & Information Science Fall 2009 Northeastern University 20 October 2009

College of Computer & Information Science Fall 2009 Northeastern University 20 October 2009 College of Computer & Informaton Scence Fall 2009 Northeastern Unversty 20 October 2009 CS7880: Algorthmc Power Tools Scrbe: Jan Wen and Laura Poplawsk Lecture Outlne: Prmal-dual schema Network Desgn:

More information

Assortment Optimization under MNL

Assortment Optimization under MNL Assortment Optmzaton under MNL Haotan Song Aprl 30, 2017 1 Introducton The assortment optmzaton problem ams to fnd the revenue-maxmzng assortment of products to offer when the prces of products are fxed.

More information

Consider the following passband digital communication system model. c t. modulator. t r a n s m i t t e r. signal decoder.

Consider the following passband digital communication system model. c t. modulator. t r a n s m i t t e r. signal decoder. PASSBAND DIGITAL MODULATION TECHNIQUES Consder the followng passband dgtal communcaton system model. cos( ω + φ ) c t message source m sgnal encoder s modulator s () t communcaton xt () channel t r a n

More information

ON A DETERMINATION OF THE INITIAL FUNCTIONS FROM THE OBSERVED VALUES OF THE BOUNDARY FUNCTIONS FOR THE SECOND-ORDER HYPERBOLIC EQUATION

ON A DETERMINATION OF THE INITIAL FUNCTIONS FROM THE OBSERVED VALUES OF THE BOUNDARY FUNCTIONS FOR THE SECOND-ORDER HYPERBOLIC EQUATION Advanced Mathematcal Models & Applcatons Vol.3, No.3, 2018, pp.215-222 ON A DETERMINATION OF THE INITIAL FUNCTIONS FROM THE OBSERVED VALUES OF THE BOUNDARY FUNCTIONS FOR THE SECOND-ORDER HYPERBOLIC EUATION

More information

Simultaneous Optimization of Berth Allocation, Quay Crane Assignment and Quay Crane Scheduling Problems in Container Terminals

Simultaneous Optimization of Berth Allocation, Quay Crane Assignment and Quay Crane Scheduling Problems in Container Terminals Smultaneous Optmzaton of Berth Allocaton, Quay Crane Assgnment and Quay Crane Schedulng Problems n Contaner Termnals Necat Aras, Yavuz Türkoğulları, Z. Caner Taşkın, Kuban Altınel Abstract In ths work,

More information

Tornado and Luby Transform Codes. Ashish Khisti Presentation October 22, 2003

Tornado and Luby Transform Codes. Ashish Khisti Presentation October 22, 2003 Tornado and Luby Transform Codes Ashsh Khst 6.454 Presentaton October 22, 2003 Background: Erasure Channel Elas[956] studed the Erasure Channel β x x β β x 2 m x 2 k? Capacty of Noseless Erasure Channel

More information

Two-Way and Multiple-Access Energy Harvesting Systems with Energy Cooperation

Two-Way and Multiple-Access Energy Harvesting Systems with Energy Cooperation Two-Way and Multple-Access Energy Harvestng Systems wth Energy Cooperaton Berk Gurakan, Omur Ozel, Jng Yang 2, and Sennur Ulukus Department of Electrcal and Computer Engneerng, Unversty of Maryland, College

More information

Digital Modems. Lecture 2

Digital Modems. Lecture 2 Dgtal Modems Lecture Revew We have shown that both Bayes and eyman/pearson crtera are based on the Lkelhood Rato Test (LRT) Λ ( r ) < > η Λ r s called observaton transformaton or suffcent statstc The crtera

More information

On Network Coding of Independent and Dependent Sources in Line Networks

On Network Coding of Independent and Dependent Sources in Line Networks On Network Codng of Independent and Dependent Sources n Lne Networks Mayank Baksh, Mchelle Effros, WeHsn Gu, Ralf Koetter Department of Electrcal Engneerng Department of Electrcal Engneerng Calforna Insttute

More information

Comparison of the Population Variance Estimators. of 2-Parameter Exponential Distribution Based on. Multiple Criteria Decision Making Method

Comparison of the Population Variance Estimators. of 2-Parameter Exponential Distribution Based on. Multiple Criteria Decision Making Method Appled Mathematcal Scences, Vol. 7, 0, no. 47, 07-0 HIARI Ltd, www.m-hkar.com Comparson of the Populaton Varance Estmators of -Parameter Exponental Dstrbuton Based on Multple Crtera Decson Makng Method

More information

Connecting Multiple-unicast and Network Error Correction: Reduction and Unachievability

Connecting Multiple-unicast and Network Error Correction: Reduction and Unachievability Connectng Multple-uncast and Network Error Correcton: Reducton and Unachevablty Wentao Huang Calforna Insttute of Technology Mchael Langberg Unversty at Buffalo, SUNY Joerg Klewer New Jersey Insttute of

More information

On mutual information estimation for mixed-pair random variables

On mutual information estimation for mixed-pair random variables On mutual nformaton estmaton for mxed-par random varables November 3, 218 Aleksandr Beknazaryan, Xn Dang and Haln Sang 1 Department of Mathematcs, The Unversty of Msssspp, Unversty, MS 38677, USA. E-mal:

More information

arxiv: v1 [math.co] 1 Mar 2014

arxiv: v1 [math.co] 1 Mar 2014 Unon-ntersectng set systems Gyula O.H. Katona and Dánel T. Nagy March 4, 014 arxv:1403.0088v1 [math.co] 1 Mar 014 Abstract Three ntersecton theorems are proved. Frst, we determne the sze of the largest

More information

2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification

2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton

More information

The Gaussian classifier. Nuno Vasconcelos ECE Department, UCSD

The Gaussian classifier. Nuno Vasconcelos ECE Department, UCSD he Gaussan classfer Nuno Vasconcelos ECE Department, UCSD Bayesan decson theory recall that we have state of the world X observatons g decson functon L[g,y] loss of predctng y wth g Bayes decson rule s

More information

The Two-User Gaussian Fading Broadcast Channel

The Two-User Gaussian Fading Broadcast Channel The Two-User Gaussan Fadng Broadcast Channel Amn Jafaran Lab of Informatcs, Networks & Communcatons LINC Department of Electrcal & Computer Engneerng Unversty of Texas at Austn Austn, TX 787, USA Emal:

More information

MAXIMUM A POSTERIORI TRANSDUCTION

MAXIMUM A POSTERIORI TRANSDUCTION MAXIMUM A POSTERIORI TRANSDUCTION LI-WEI WANG, JU-FU FENG School of Mathematcal Scences, Peng Unversty, Bejng, 0087, Chna Center for Informaton Scences, Peng Unversty, Bejng, 0087, Chna E-MIAL: {wanglw,

More information

On Information Theoretic Games for Interference Networks

On Information Theoretic Games for Interference Networks On Informaton Theoretc Games for Interference Networks Suvarup Saha and Randall A. Berry Dept. of EECS Northwestern Unversty e-mal: suvarups@u.northwestern.edu rberry@eecs.northwestern.edu Abstract The

More information

Low Complexity Soft-Input Soft-Output Hamming Decoder

Low Complexity Soft-Input Soft-Output Hamming Decoder Low Complexty Soft-Input Soft-Output Hammng Der Benjamn Müller, Martn Holters, Udo Zölzer Helmut Schmdt Unversty Unversty of the Federal Armed Forces Department of Sgnal Processng and Communcatons Holstenhofweg

More information

MMA and GCMMA two methods for nonlinear optimization

MMA and GCMMA two methods for nonlinear optimization MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons

More information

The Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction

The Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction ECONOMICS 5* -- NOTE (Summary) ECON 5* -- NOTE The Multple Classcal Lnear Regresson Model (CLRM): Specfcaton and Assumptons. Introducton CLRM stands for the Classcal Lnear Regresson Model. The CLRM s also

More information

Notes prepared by Prof Mrs) M.J. Gholba Class M.Sc Part(I) Information Technology

Notes prepared by Prof Mrs) M.J. Gholba Class M.Sc Part(I) Information Technology Inverse transformatons Generaton of random observatons from gven dstrbutons Assume that random numbers,,, are readly avalable, where each tself s a random varable whch s unformly dstrbuted over the range(,).

More information

Notes on Frequency Estimation in Data Streams

Notes on Frequency Estimation in Data Streams Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to

More information

Exact-Regenerating Codes between MBR and MSR Points

Exact-Regenerating Codes between MBR and MSR Points Exact-Regeneratng Codes between BR SR Ponts arxv:1304.5357v1 [cs.dc] 19 Apr 2013 Abstract In ths paper we study dstrbuted storage systems wth exact repar. We gve a constructon for regeneratng codes between

More information

Lecture 3: Probability Distributions

Lecture 3: Probability Distributions Lecture 3: Probablty Dstrbutons Random Varables Let us begn by defnng a sample space as a set of outcomes from an experment. We denote ths by S. A random varable s a functon whch maps outcomes nto the

More information

Application of Nonbinary LDPC Codes for Communication over Fading Channels Using Higher Order Modulations

Application of Nonbinary LDPC Codes for Communication over Fading Channels Using Higher Order Modulations Applcaton of Nonbnary LDPC Codes for Communcaton over Fadng Channels Usng Hgher Order Modulatons Rong-Hu Peng and Rong-Rong Chen Department of Electrcal and Computer Engneerng Unversty of Utah Ths work

More information

x = x 1 + :::+ x K and the nput covarance matrces are of the form ± = E[x x y ]. 3.2 Dualty Next, we ntroduce the concept of dualty wth the followng t

x = x 1 + :::+ x K and the nput covarance matrces are of the form ± = E[x x y ]. 3.2 Dualty Next, we ntroduce the concept of dualty wth the followng t Sum Power Iteratve Water-fllng for Mult-Antenna Gaussan Broadcast Channels N. Jndal, S. Jafar, S. Vshwanath and A. Goldsmth Dept. of Electrcal Engg. Stanford Unversty, CA, 94305 emal: njndal,syed,srram,andrea@wsl.stanford.edu

More information

3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X

3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number

More information

Estimation: Part 2. Chapter GREG estimation

Estimation: Part 2. Chapter GREG estimation Chapter 9 Estmaton: Part 2 9. GREG estmaton In Chapter 8, we have seen that the regresson estmator s an effcent estmator when there s a lnear relatonshp between y and x. In ths chapter, we generalzed the

More information

Maximizing the number of nonnegative subsets

Maximizing the number of nonnegative subsets Maxmzng the number of nonnegatve subsets Noga Alon Hao Huang December 1, 213 Abstract Gven a set of n real numbers, f the sum of elements of every subset of sze larger than k s negatve, what s the maxmum

More information

IN THE paradigm of network coding, a set of source nodes

IN THE paradigm of network coding, a set of source nodes 4496 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 64, NO. 6, JUNE 2018 Sngle-Uncast Secure Network Codng and Network Error Correcton are as Hard as Multple-Uncast Network Codng Wentao Huang, Tracey Ho,

More information

(1 ) (1 ) 0 (1 ) (1 ) 0

(1 ) (1 ) 0 (1 ) (1 ) 0 Appendx A Appendx A contans proofs for resubmsson "Contractng Informaton Securty n the Presence of Double oral Hazard" Proof of Lemma 1: Assume that, to the contrary, BS efforts are achevable under a blateral

More information

COMPARISON OF SOME RELIABILITY CHARACTERISTICS BETWEEN REDUNDANT SYSTEMS REQUIRING SUPPORTING UNITS FOR THEIR OPERATIONS

COMPARISON OF SOME RELIABILITY CHARACTERISTICS BETWEEN REDUNDANT SYSTEMS REQUIRING SUPPORTING UNITS FOR THEIR OPERATIONS Avalable onlne at http://sck.org J. Math. Comput. Sc. 3 (3), No., 6-3 ISSN: 97-537 COMPARISON OF SOME RELIABILITY CHARACTERISTICS BETWEEN REDUNDANT SYSTEMS REQUIRING SUPPORTING UNITS FOR THEIR OPERATIONS

More information

Research Article Green s Theorem for Sign Data

Research Article Green s Theorem for Sign Data Internatonal Scholarly Research Network ISRN Appled Mathematcs Volume 2012, Artcle ID 539359, 10 pages do:10.5402/2012/539359 Research Artcle Green s Theorem for Sgn Data Lous M. Houston The Unversty of

More information

Appendix B. Criterion of Riemann-Stieltjes Integrability

Appendix B. Criterion of Riemann-Stieltjes Integrability Appendx B. Crteron of Remann-Steltes Integrablty Ths note s complementary to [R, Ch. 6] and [T, Sec. 3.5]. The man result of ths note s Theorem B.3, whch provdes the necessary and suffcent condtons for

More information

arxiv:cs.cv/ Jun 2000

arxiv:cs.cv/ Jun 2000 Correlaton over Decomposed Sgnals: A Non-Lnear Approach to Fast and Effectve Sequences Comparson Lucano da Fontoura Costa arxv:cs.cv/0006040 28 Jun 2000 Cybernetc Vson Research Group IFSC Unversty of São

More information

Introduction to Information Theory, Data Compression,

Introduction to Information Theory, Data Compression, Introducton to Informaton Theory, Data Compresson, Codng Mehd Ibm Brahm, Laura Mnkova Aprl 5, 208 Ths s the augmented transcrpt of a lecture gven by Luc Devroye on the 3th of March 208 for a Data Structures

More information

A be a probability space. A random vector

A be a probability space. A random vector Statstcs 1: Probablty Theory II 8 1 JOINT AND MARGINAL DISTRIBUTIONS In Probablty Theory I we formulate the concept of a (real) random varable and descrbe the probablstc behavor of ths random varable by

More information

Graph Reconstruction by Permutations

Graph Reconstruction by Permutations Graph Reconstructon by Permutatons Perre Ille and Wllam Kocay* Insttut de Mathémathques de Lumny CNRS UMR 6206 163 avenue de Lumny, Case 907 13288 Marselle Cedex 9, France e-mal: lle@ml.unv-mrs.fr Computer

More information

Power Allocation/Beamforming for DF MIMO Two-Way Relaying: Relay and Network Optimization

Power Allocation/Beamforming for DF MIMO Two-Way Relaying: Relay and Network Optimization Power Allocaton/Beamformng for DF MIMO Two-Way Relayng: Relay and Network Optmzaton Je Gao, Janshu Zhang, Sergy A. Vorobyov, Ha Jang, and Martn Haardt Dept. of Electrcal & Computer Engneerng, Unversty

More information

Introduction to Random Variables

Introduction to Random Variables Introducton to Random Varables Defnton of random varable Defnton of random varable Dscrete and contnuous random varable Probablty functon Dstrbuton functon Densty functon Sometmes, t s not enough to descrbe

More information

Online Appendix: Reciprocity with Many Goods

Online Appendix: Reciprocity with Many Goods T D T A : O A Kyle Bagwell Stanford Unversty and NBER Robert W. Stager Dartmouth College and NBER March 2016 Abstract Ths onlne Appendx extends to a many-good settng the man features of recprocty emphaszed

More information

Lecture Notes on Linear Regression

Lecture Notes on Linear Regression Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume

More information

Secret Communication using Artificial Noise

Secret Communication using Artificial Noise Secret Communcaton usng Artfcal Nose Roht Neg, Satashu Goel C Department, Carnege Mellon Unversty, PA 151, USA {neg,satashug}@ece.cmu.edu Abstract The problem of secret communcaton between two nodes over

More information

Affine transformations and convexity

Affine transformations and convexity Affne transformatons and convexty The purpose of ths document s to prove some basc propertes of affne transformatons nvolvng convex sets. Here are a few onlne references for background nformaton: http://math.ucr.edu/

More information

Real-Time Systems. Multiprocessor scheduling. Multiprocessor scheduling. Multiprocessor scheduling

Real-Time Systems. Multiprocessor scheduling. Multiprocessor scheduling. Multiprocessor scheduling Real-Tme Systems Multprocessor schedulng Specfcaton Implementaton Verfcaton Multprocessor schedulng -- -- Global schedulng How are tasks assgned to processors? Statc assgnment The processor(s) used for

More information

ISSN On error probability exponents of many hypotheses optimal testing illustrations

ISSN On error probability exponents of many hypotheses optimal testing illustrations Journal Afrka Statstka Vol. 6, 2011, pages 307 315. DOI: http://dx.do.org/10.4314/afst.v61.1 Journal Afrka Statstka ISS 0825-0305 On error probablty exponents of many hypotheses optmal testng llustratons

More information

Winter 2008 CS567 Stochastic Linear/Integer Programming Guest Lecturer: Xu, Huan

Winter 2008 CS567 Stochastic Linear/Integer Programming Guest Lecturer: Xu, Huan Wnter 2008 CS567 Stochastc Lnear/Integer Programmng Guest Lecturer: Xu, Huan Class 2: More Modelng Examples 1 Capacty Expanson Capacty expanson models optmal choces of the tmng and levels of nvestments

More information

CLOUD Radio Access Network (C-RAN) is an emerging

CLOUD Radio Access Network (C-RAN) is an emerging 7402 EEE TRANSACTONS ON NFORMATON THEORY VOL. 62 NO. 2 DECEMBER 206 On the Optmal Fronthaul Compresson and Decodng Strateges for Uplnk Cloud Rado Access Networks Yuhan Zhou Member EEE YnfeXuStudent Member

More information

CSCE 790S Background Results

CSCE 790S Background Results CSCE 790S Background Results Stephen A. Fenner September 8, 011 Abstract These results are background to the course CSCE 790S/CSCE 790B, Quantum Computaton and Informaton (Sprng 007 and Fall 011). Each

More information

Supplementary material: Margin based PU Learning. Matrix Concentration Inequalities

Supplementary material: Margin based PU Learning. Matrix Concentration Inequalities Supplementary materal: Margn based PU Learnng We gve the complete proofs of Theorem and n Secton We frst ntroduce the well-known concentraton nequalty, so the covarance estmator can be bounded Then we

More information

n α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0

n α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0 MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector

More information

BOUNDEDNESS OF THE RIESZ TRANSFORM WITH MATRIX A 2 WEIGHTS

BOUNDEDNESS OF THE RIESZ TRANSFORM WITH MATRIX A 2 WEIGHTS BOUNDEDNESS OF THE IESZ TANSFOM WITH MATIX A WEIGHTS Introducton Let L = L ( n, be the functon space wth norm (ˆ f L = f(x C dx d < For a d d matrx valued functon W : wth W (x postve sem-defnte for all

More information

CHAPTER III Neural Networks as Associative Memory

CHAPTER III Neural Networks as Associative Memory CHAPTER III Neural Networs as Assocatve Memory Introducton One of the prmary functons of the bran s assocatve memory. We assocate the faces wth names, letters wth sounds, or we can recognze the people

More information

Power law and dimension of the maximum value for belief distribution with the max Deng entropy

Power law and dimension of the maximum value for belief distribution with the max Deng entropy Power law and dmenson of the maxmum value for belef dstrbuton wth the max Deng entropy Bngy Kang a, a College of Informaton Engneerng, Northwest A&F Unversty, Yanglng, Shaanx, 712100, Chna. Abstract Deng

More information

Chapter 11: Simple Linear Regression and Correlation

Chapter 11: Simple Linear Regression and Correlation Chapter 11: Smple Lnear Regresson and Correlaton 11-1 Emprcal Models 11-2 Smple Lnear Regresson 11-3 Propertes of the Least Squares Estmators 11-4 Hypothess Test n Smple Lnear Regresson 11-4.1 Use of t-tests

More information

Continuous Time Markov Chain

Continuous Time Markov Chain Contnuous Tme Markov Chan Hu Jn Department of Electroncs and Communcaton Engneerng Hanyang Unversty ERICA Campus Contents Contnuous tme Markov Chan (CTMC) Propertes of sojourn tme Relatons Transton probablty

More information

Lecture 5 Decoding Binary BCH Codes

Lecture 5 Decoding Binary BCH Codes Lecture 5 Decodng Bnary BCH Codes In ths class, we wll ntroduce dfferent methods for decodng BCH codes 51 Decodng the [15, 7, 5] 2 -BCH Code Consder the [15, 7, 5] 2 -code C we ntroduced n the last lecture

More information

Amiri s Supply Chain Model. System Engineering b Department of Mathematics and Statistics c Odette School of Business

Amiri s Supply Chain Model. System Engineering b Department of Mathematics and Statistics c Odette School of Business Amr s Supply Chan Model by S. Ashtab a,, R.J. Caron b E. Selvarajah c a Department of Industral Manufacturng System Engneerng b Department of Mathematcs Statstcs c Odette School of Busness Unversty of

More information

Perfect Competition and the Nash Bargaining Solution

Perfect Competition and the Nash Bargaining Solution Perfect Competton and the Nash Barganng Soluton Renhard John Department of Economcs Unversty of Bonn Adenauerallee 24-42 53113 Bonn, Germany emal: rohn@un-bonn.de May 2005 Abstract For a lnear exchange

More information

Coding over an Erasure Channel with a Large Alphabet Size

Coding over an Erasure Channel with a Large Alphabet Size 1 Codng over an Erasure Channel wth a Large Alphabet Sze Shervan Fashand, Shahab Oves Gharan and Amr K. Khandan ECE Dept., Unversty of Waterloo, Waterloo, O, Canada, 2L3G1 emal: sfashand,shahab,khandan}@cst.uwaterloo.ca

More information

Dimensionality Reduction Notes 1

Dimensionality Reduction Notes 1 Dmensonalty Reducton Notes 1 Jelan Nelson mnlek@seas.harvard.edu August 10, 2015 1 Prelmnares Here we collect some notaton and basc lemmas used throughout ths note. Throughout, for a random varable X,

More information

FACTORIZATION IN KRULL MONOIDS WITH INFINITE CLASS GROUP

FACTORIZATION IN KRULL MONOIDS WITH INFINITE CLASS GROUP C O L L O Q U I U M M A T H E M A T I C U M VOL. 80 1999 NO. 1 FACTORIZATION IN KRULL MONOIDS WITH INFINITE CLASS GROUP BY FLORIAN K A I N R A T H (GRAZ) Abstract. Let H be a Krull monod wth nfnte class

More information

Predictive Analytics : QM901.1x Prof U Dinesh Kumar, IIMB. All Rights Reserved, Indian Institute of Management Bangalore

Predictive Analytics : QM901.1x Prof U Dinesh Kumar, IIMB. All Rights Reserved, Indian Institute of Management Bangalore Sesson Outlne Introducton to classfcaton problems and dscrete choce models. Introducton to Logstcs Regresson. Logstc functon and Logt functon. Maxmum Lkelhood Estmator (MLE) for estmaton of LR parameters.

More information

Mathematical Models for Information Sources A Logarithmic i Measure of Information

Mathematical Models for Information Sources A Logarithmic i Measure of Information Introducton to Informaton Theory Wreless Informaton Transmsson System Lab. Insttute of Communcatons Engneerng g Natonal Sun Yat-sen Unversty Table of Contents Mathematcal Models for Informaton Sources

More information

Lecture 10 Support Vector Machines II

Lecture 10 Support Vector Machines II Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed

More information

MATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS

MATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS MATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS These are nformal notes whch cover some of the materal whch s not n the course book. The man purpose s to gve a number of nontrval examples

More information