The Capacity Region of the Degraded Finite-State Broadcast Channel

Size: px
Start display at page:

Download "The Capacity Region of the Degraded Finite-State Broadcast Channel"

Transcription

1 The Capacity Regio of the Degraded Fiite-State Broadcast Chael Ro Dabora ad Adrea Goldsmith Dept. of Electrical Egieerig, Staford Uiversity, Staford, CA Abstract We cosider the discrete, time-varyig broadcast chael with memory uder the assumptio that the chael states belog to a set of ite cardiality. We rst dee the physically degraded ite-state broadcast chael for which we derive the capacity regio. We the dee the stochastically degraded ite-state broadcast chael ad derive the capacity regio for this sceario as well. I both scearios we cosider the oidecomposable ite-state chael as well as the idecomposable oe. I. INTRODUCTION The broadcast chael (BC) was itroduced by Cover i 972. I this sceario a sigle seder trasmits three messages, oe commo ad two private, to two receivers over a chael deed by X, p(y, z x), Y Z. Here, X is the chael iput from the trasmitter, Y is the chael output at Rx ad Z is the chael output at Rx 2. I the years followig its itroductio the study of the BC focused o memoryless scearios, i.e., whe the probability of a block of trasmissios is give by p(y, z x ) p(y i, z i x i ). I recet years, models of time-varyig broadcast chaels with memory have attracted a lot of attetio, especially i Gaussia setups. This was motivated by the proliferatio of mobile commuicatios, for which the chael is subject to time-varyig correlated fadig. The correlatio of the fadig process itroduces memory to the BC. The fadig BC is oe istace of the geeral BC with chael states. While fadig BCs received cosiderable attetio, discrete BCs with chael states received oly little attetio. A otable exceptio is the degraded arbitrarily varyig BC (DAVBC) cosidered i [2] ad [3]. I [2] DAVBCs with causal ad o-causal side iformatio at the trasmitter were cosidered. The states are assumed i.i.d. ad the chael is memoryless: p(y, z x, s ) p(y i, z i x i, s i ). I [3], the capacity regio for DAVBCs with causal side iformatio at the trasmitter ad o-causal side iformatio at the good receiver was derived. I [3] the state distributio is geeral ad is ot subject to the i.i.d. restrictio, but the chael outputs, give the states ad the chael iputs are agai memoryless. The arbitrarily varyig chael (AVC) is oe model for a time-varyig chael with states. It models a memoryless chael whose law varies i time i a arbitrary maer. The state trasitios are idepedet of the chael iputs ad outputs. I this work we study the discrete time-varyig BC with memory i the framework of ite-state chaels (FSCs). I cotrast to the AVC, i the FSC both the chael output ad the curret state deped o both the chael iput ad the previous state. The authors are with the Wireless Systems Lab, Departmet of Electrical Egieerig, Staford Uiversity, Staford, CA ro,adrea@wsl.staford.edu. This work was supported i part by the DARPA ITMANET program uder grat TFIND. The ite-state chael model was cosidered for the poit-to-poit sceario as early as 953 []. This chael is characterized by the distributio p(y, s x, s ) where S is the curret state ad S is the previous state. For a block of trasmissios, the p.m.f. at the i'th symbol time satises p(y i, s i x i, s i, y i, s 0 ) p(y i, s i x i, s i ), () where s 0 was the state of the chael whe trasmissio bega. Equatio () implies that S i cotais all the history iformatio for time i. Recetly, the ite-state multipleaccess chael was studied i [5]. This sceario is characterized by the chael distributio p(y, s x, x 2, s ), ad the work i [5] also cosidered the effect of feedback o the rates. I the preset work we study the ite-state broadcast chael (FSBC). Here, the chael from the trasmitter to the receivers is govered by a state sequece that depeds o the chael iputs, outputs ad previous states. The way these symbols iteract with each other is captured by the trasitio fuctio p(y, z, s x, s ). Mai Cotributios ad Orgaizatio I this paper we cosider for the rst time the capacity of the FSBC. Here, there is a uique aspect ot ecoutered i the poit-to-poit ad the MAC couterparts, amely the applicatio of superpositio codig to the FSC. We iitially dee the physically degraded FSBC ad d the capacity regio of this sceario. We the dee the stochastically degraded FSBC ad give examples of commuicatio scearios represeted by this model. We derive the capacity regio for this chael as well. The rest of this paper is orgaized as follows: Sectio II itroduces the chael model. Sectio III presets a summary of the results together with a discussio. Lastly, Sectio IV outlies the proof of the capacity regio for the physically degraded FSBC. II. CHANNEL MODEL AND DEFINITIONS First, a word about otatio. I the followig we deote radom variables with upper case letters, e.g. X, Y, ad their realizatios with lower case letters x, y. A radom variable (RV) X takes values i a set X. We use X to deote the cardiality of a ite, discrete set X, X to deote the -fold cartesia product of X, ad p X (x) to deote the probability mass fuctio (p.m.f.) of a discrete RV X o X. For brevity we may omit the subscript X whe it is obvious from the cotext. We use p X Y (x y) to deote the coditioal p.m.f. of X give Y. We deote vectors with boldface letters, e.g. x, y; the i'th elemet of a vector x is deoted with x i ad we use x j i where i < j to deote the vector (x i, x i+,..., x j, x j );

2 is short form otatio for x j, ad x x. A vector of radom variables is deoted by X, ad similarly we dee X j i (X i, X i+,..., X j, X j ) for i < j. We use H( ) to deote the etropy of a discrete radom variable ad I( ; ) to deote the mutual iformatio betwee two radom variables, as deed i [6, Chapter 2]. I( ; ) q deotes the mutual iformatio evaluated with a p.m.f. q o the radom variables. Fially, co R deotes the covex hull of the set R. Deitio : The discrete, ite-state broadcast chael is deed by the triplet X S, p(y, z, s x, s ), Y Z S where X is the iput symbol, Y ad Z are the output symbols, S is the state of the chael at the ed of the previous symbol trasmissio ad S is the state of the chael at the ed of the curret symbol trasmissio. S, X, Y ad Z are discrete alphabets of ite cardialities. The p.m.f of a block of trasmissios is x j p(y, z, s, x s 0 ) p(y i, z i, s i, x i y i, z i, s i, x i, s 0 ) p(x i x i )p(y i, z i, s i y i, z i, s i, x i, s 0 ) (a) p(x ) p(y i, z i, s i x i, s i ), (2) where s 0 is the iitial chael state. Here (a) captures the fact that give S i, the symbols at time i are idepedet of the past. Deitio 2: The FSBC is called physically degraded if its p.m.f. satises p(y i x i, y i, z i, s 0 ) p(y i x i, y i, s 0 ), p(z i x i, y i, z i, s 0 ) p(z i y i, z i, s 0 ). (3a) (3b) Coditio (3a) captures the ituitive otio of degradedess, amely that Z i is a degraded versio of Y i, thus it does ot add iformatio whe Y i is give. Note that i the memoryless case this coditio is ot ecessary as, give X i, Y i is idepedet of the history. Coditio (3b) follows from the stadard otio of degradedess. Usig coditios (3a) ad (3b) we obtai (whe p(y, x s 0 ) > 0) p(z y, x, s 0 ) p(z, y, x s 0 ) p(y, x s 0 ) p(z i, y i, x i z i, y i, x i, s 0 ) p(y i, x i y i, x i, s 0 ) p(x i z i, y i, x i ) p(z i, y i z i,y i,x i,s 0 ) p(x i y i, x i ) p(y i y i, x i, s 0 ) (a) p(x i x i ) p(z i, y i z i, y i, x i, s 0 ) p(x i x i ) p(y i y i, x i, s 0 ) (b) p(y i y i, x i, s 0 ) p(z i z i, y i, x i, s 0 ) p(y i y i, x i, s 0 ) (c) p(z i z i, y i, s 0 ), (4) where (a) is because there is o feedback, (b) follows from (3a) ad (c) follows from (3b). We coclude that whe (3) holds, p(z y, x, s 0 ) p(z y, s 0 ). Hece, p(y, z x, s 0 ) p(y x, s 0 )p(z y, s 0 ). (5) Note that (4) shows how to obtai p(z y, x, s 0 ) i a causal maer. Also ote that Z is a degraded versio of Y but still depeds o the state sequece (i.e. degradedess does ot elimiate the memory). A special case of the physically degraded FSBC occurs whe i (3b) it holds that p(z i x i, y i, z i, s 0 ) p(z i y i ). Hece, p(z y, x, s 0 ) p(z y ) p(z i y i ). (6) Equatio (6) is similar to the deitio of degradedess for the DAVBC used i [2]. Deitio 3: The FSBC is called stochastically degraded if there exists a p.m.f. p(z y) such that p(z, s x, s ) Y p(y, s x, s )p(z y, s, x, s ) p(y, s x, s ) p(z y). (7) Y Note that whe (7) holds the p(z x, s 0 ) p(z, s x, s 0 ) (a) p(z i, s i x i, s i ) p(y i, s i x i, s i ) p(z i y i ) y i Y Y (b) p(y i, s i x i, s i ) p(z i y i ) Y p(y, s x, s 0 ) Y p(y x, s 0 ) p(z i y i ) p(z i y i ), (8) where (a) ad (b) follow from (2). Deitio 3 does ot costitute oly a mathematical coveiece, but represets a physical sceario. For example, cosider a sceario i which a base statio trasmits to two mobile uits, located approximately o the same lie-of-sight from the base statio (BS), as idicated by the dashed lie i Figure. Let the BS trasmit a BPSK sigal ad let the received sigals be subject to additive Gaussia thermal oise due to the receivers' frot-eds. Whe decodig at the receivers takes place after a hard threshold at zero, the resultig sceario is the biary symmetric broadcast chael (BSBC). Deote the situatio where there is o trafc o the road betwee the BS ad the mobiles as state A. Let the chael BS Rx have a crossover probability ɛ (A) 0. ad the chael BS Rx 2 have a crossover probability ɛ 2 (A) 0.5. This ca be represeted as a stochastically degraded BC with a degradig chael whose crossover probability is ɛ 2 (A) ɛ 2 ɛ 2ɛ

3 Assume that o occasios, a car passes o the road betwee the BS ad the mobiles. This causes atteuatio i both chaels simultaeously. Call this state B ad let ɛ (B) 0.8 ad ɛ 2 (B) Agai we have ɛ 2 (B) Hece, the degradig chael is the same for both states, irrespective of the state sequece (i this example the state sequece represets the trafc patter, ad is ot a idepedet sequece). This satises coditio (8). Note that we assume o kowledge of the states at the trasmitter ad receivers. Deitio 6: The average probability of error of a code for the FSBC is give by P () e max s0 S P () e (s 0 ), where, P e () (s 0 )Pr ( g y (Y ) (M 0, M ) or g z (Z ) ) (M 0, M 2 ) s 0, where each of the messages M 0 M 0, M M ad M 2 M 2 is selected idepedetly ad uiformly. Deitio 7: A rate triplet (R 0, R, R 2 ) is called achievable for the FSBC if for every ɛ > 0 ad δ > 0 there exists a (ɛ, δ) such that for all > (ɛ, δ) a (R 0 δ, R δ, R 2 δ, ) code with P e () ɛ ca be costructed. Deitio 8: The capacity regio of the FSBC is the covex hull of all achievable rate triplets. Base Statio Mobile Mobile 2 Fig.. A degraded FSBC sceario: the mobile uits are located o the same lie-of-sight from the base-statio (idicated by the dashed lie). Passig cars affect the chaels to both mobile uits simultaeously. More geerally, we ca dee a set of states for this sceario, e.g. S, 2,..., K, with Y Z 0, ad p(z i, s i y i, s i ) p(s i s i )p(z i y i, s i ) ɛ2 (k), z y p(z y, s k), ɛ 2 (k), z y ɛ 2 (k) (0, 0.5), k S. This results i a collectio of physically degraded BSBCs that ca give more exibility i modelig the sceario of Figure, as the degradig chael may deped o the state. However, for this reaso, this model does ot satisfy our deitio of stochastic degradedess i Deitio 3. Deitio 4: (see [4, Sectio 4.6]) The FSBC is called idecomposable if for every ɛ > 0 there exists N 0 (ɛ) such that for all > N 0 (ɛ), p(s x, s 0 ) p(s x, s 0) < ɛ, for all s, x, ad iitial states s 0 ad s 0. Deitio 5: A (R 0, R, R 2, ) determiistic code for the FSBC cosists of three message sets, M 0, 2,..., 2 R0, M, 2,..., 2 R ad M2, 2,..., 2 R2, ad three mappigs (f, g y, g z ) such that f : M 0 M M 2 X (9) is the ecoder ad g y : Y M 0 M, g z : Z M 0 M 2, are the decoders. Here, M 0 is the set of commo messages ad M ad M 2 are the sets of private messages to Rx ad Rx 2 respectively. The sceario parameters assumed i this example are: Two-ray propagatio model, Rx decodig scheme is maximum-likelihood, Base statio Tx power 30 dbm, Base statio atea gai 0 dbi, Rx atea gai 0 dbi, Rx oise oor 90 dbm, Base statio atea height 0 m, Rx atea height.5 m, BS Rx distace 7.2 Km ad BS Rx 2 distace 8 Km. We also assume a passig car icreases the path atteuatio by 3 db. Dee rst III. MAIN RESULTS AND DISCUSSION R, (p, s 0 ) I(X ; Y U, s 0 ) p log 2 S R 2, (p, s 0 ) I(U ; Z s 0 ) p log 2 S. The mai result is stated i the followig theorem, whose proof is outlied i Sectio IV: Theorem : Let Q be the set of all joit distributios o ( U i, X ) such that the cardiality of the radom vector U is bouded by U i mi X, Y, Z. For the physically degraded FSBC of Deitio 2, dee the regio R (s 0 ) as R (s 0 ) co q Q (R 0, R, R 2 ) : R 0 0, R 0, R 2 0, R R, (q, s 0 ), R 0 + R 2 R 2, (q, s 0 ).(0) The capacity regio of the physically degraded FSBC is give by C pd lim R (s 0 ), () s 0 S ad the limit exists. Sice the capacity of the broadcast chael depeds oly o the coditioal margials p(y x, s 0 ) ad p(z x, s 0 ) (see [6, Chapter 4.6]) the the capacity regio of the stochastically degraded FSBC is the same as the correspodig physically degraded FSBC: Corollary : For the stochastically degraded FSBC of Defiitio 3, the capacity regio is give by Theorem where p(z s, y, x, s ) is replaced by p(z y) that satises equatio (7). Whe the FSBC is idecomposable, the the effect of the iitial state fades away as icreases. Therefore we have the followig corollary: Corollary 2: For the idecomposable physically degraded FSBC, the capacity regio is give by Theorem. For the idecomposable stochastically degraded FSBC, the capacity regio is obtaied from Corollary. I both cases the parameter s 0 i R, (q, s 0 ) ad R 2, (q, s 0 ) ad the itersectio over S i the expressio for C pd are omitted.

4 Proof outlie: Loosely speakig, the corollary is true sice for large eough the effect of the iitial state fades away. Therefore, for asymptotically large the maximum over all iitial states s 0 S equals the miimum. Discussio First, ote that if lim R (s 0 ) exists for all s 0 S the the capacity regio () ca be writte as C pd lim R (s 0 ) (a) lim R (s 0 ). Here, (a) is permitted because S is ite. Thus, the capacity regio ca be viewed as the itersectio of all the capacity regios obtaied whe the iitial state is kow at the receivers (but ot at the trasmitter). We also ote the followig coclusios: ) Sice the limit of the regio exists, the as icreases, optimizig the code will result i better performace (which is ot guarateed whe the limits caot be show to exist, cosider for example a o-statioary chael with oise that oscillates with time). 2) The codebook structure that achieves capacity is a superpositio codebook. This itroduces a structural costrait whe optimizig the codebook for achievig the maximum rate triplets. 3) The auxiliary RV U itroduces difculties maily i places where we eed to rely o the its cardiality. This is because we caot traslate the boud o the cardiality of U ito a boud o the cardiality of a subset of U. I particular, we caot use the cardiality of U whe derivig the capacity regio for the idecomposable FSBC. Moreover, lettig m + m 2, the from Equatio () we have that p(z m, y m, s m x, s 0 ) p(z m, y m, s m x m, s 0 ). But because p(x m u ) p(x m u m ) the p(z m, y m, s m u, s 0 ) p(z m, y m, s m u m, s 0 ). This is a major differece from the poit-to-poit ad the MAC chaels. Cosider, for example, the expressio max max p(u,x ) s 0 I(U ; Z s 0 )+λ max s 0 I(X ; Y U, s 0). (2) While i the MAC ad the poit-to-poit chaels the correspodig expressios coverge for all chaels, for the FSBC (2) ca be show to coverge oly for the idecomposable case. Therefore, usig superpositio codig, the chael betwee U ad (Y, Z ) is fudametally differet from the chael betwee X ad (Y, Z ). This is i cotrast also to the discrete, memoryless BC. IV. PROOF OUTLINE I the derivatio we focus o the physically degraded FSBC. The derivatio requires oly that coditio (5) holds. I the derivatio we shall cosider oly the two private messages case as the commo message ca be icorporated by splittig the rate to Rx 2 ito private ad commo rates, as i [6, Theorem 4.6.4]. R 2 0 s A Achievable Regio R Fig. 2. Lies boudig the achievable regios for the FSBC for iitial states s A ad s B, ad the resultig regio of positive error expoets. A. Achievability Theorem Due to space limitatios we omit the details of the achievability proof ad give oly the coclusio. For complete details see [7]. Dee rst F (λ) max p(u,x ) 0 s B mi R 2,(p, s 0 ) + λ mi R,(p, s 0) Followig [8, Sectio 2], the boudary of the regio of positive error expoets for a give ca be writte as R2 (R ) if F (λ) λr. (3) 0 λ This characterizatio is illustrated i Figure 2. I the achievability proof we show that for a give p(u (, x ), whe trasmittig at the positive rate pair mis 0 S R, (p, s 0), mi s0 S R 2, (p, s 0 ) ), the the error expoet is positive ad bouded away from zero. Hece, the probability of error ca be made less tha ay arbitrary ɛ > 0 by takig a block legth K with a large eough iteger K. Furthermore, i sectio IV-D we show that the largest regio is obtaied by takig the limit R 2 (R ) if 0 λ lim F (λ) λr., (4) ad that this limit exists ad is ite. The fact that the limit exists ad is ite implies that we ca approach the rates of Theorem arbitrarily close by takig large eough, thus by Deitio 7 these rates are achievable. Before cosiderig the coverse we discuss the cardiality of the auxiliary RV U, as the evaluatio of R 2 (R ) of (3) depeds o the existece of such a boud. B. Cardiality Bouds From the derivatio i [8], it follows that maximizig the regio R (s 0 ) of Equatio (0) over all joit distributios p(u, x ), ca be carried out while the cardiality of the auxiliary radom variable U is bouded by U i mi X, Y, Z. (5) Now ote that from (), the achievable regio for a xed is give by the itersectio s 0 S R (s 0 ). As for each R (s 0 ), s 0 S we have the same cardiality boud, the this boud also holds for maximizig the itersectio of the regios R (s 0 ), s 0 S.

5 C. Coverse Lemma : If for some ɛ > 0, λ 0, R 2 + λr > lim F (λ) + ɛ, the there exists a pair of iitial states s 0 ad s 0 such that P () e2 (s 0)R 2 +λ ( P () e (s 0)R ) > ɛ (+λ)( + log 2 S ). The implicatio of this iequality, as explaied i [8, Sectio 3], is that for large eough the probability of error P e () caot be made arbitrarily small outside the regio (4). Proof: From Fao's iequality we have that H(M 2 Z, s 0 ) P () e2 (s 0)R 2 + (6a) H(M Y, s 0 ) P () e (s 0)R +. (6b) Next write mi I(M 2; Z s 0 ) R 2 max H(M 2 Z, s 0 ) (7) s 0 S s 0 S mi I(M ; Y M 2, s 0) R max H(M Y, M 2, s 0) R max H(M Y, s 0).(8) Now ote that I(M 2 ; Z s 0 ) H(Z s 0 ) H(Z M 2, s 0 ) I(U ; Z s 0 ), (9) where U i M 2, i, 2,...,. We also have I(M ; Y M 2, s 0) H(Y M 2, s 0) H(Y M, M 2, s 0) H(Y U, s 0) H(Y X, U, s 0) I(X ; Y U, s 0), (20) where the deitio of U satises the Markov relatioship U s 0 X s 0 Y s 0. Combiig (9) ad (20) we have that for this choice of U : mi I(M 2; Z s 0 ) + λ mi I(M ; Y M 2, s 0) mi I(U ; Z s 0 ) + λ mi I(X ; Y U, s 0) F (λ) + ( + λ) log 2 S, (2) sice F (λ) is obtaied by maximizig over all joit distributios p(u, x ) subject to the cardiality costrait (5), which is also satised by our choice of U. Let s 0, ad s 0, be the maximizig states for H(M 2 Z, s 0 ) ad H(M Y, s 0) respectively. Pluggig (7) ad (8) ito (2) yields R 2 H(M 2 Z, s 0, ) + λ(r H(M Y, s 0,)) ( + λ) log 2 S F (λ). Thus, H(M 2 Z, s 0, ) + λh(m Y, s 0,) + ( + λ) log 2 S (R 2 + λr F (λ)) (R 2 + λr lim F (λ)) > ɛ. Combied with (6), this completes the proof of the lemma. D. Covergece I this subsectio we show that lim F (λ) exists ad is ite for the chael uder cosideratio, whe λ [0, ]. The proof of covergece exteds the argumets i [4, Appedix 4A] to the FSBC. The mai difculty here is the itroductio of the auxiliary RV U ad its iteractio with the other RVs,, X, Y ad Z. We actually show that lim F (λ) sup F (λ) which implies that the limit exists. Due to its legth, the full proof is omitted ad oly the mai poits are highlighted. Let s 0 s z 0(l) miimize l I(U l ; Z l s 0 ) ad ( s 0 s y 0 (l) miimize l I(Xl ; Y l U l, s 0), for the triplet q (u l, x l ), s z 0(l), s y 0 (l)) that achieves the max-mi solutio for F l (λ), ad let (q 2 (u m, x m ), s z 0(m), s y 0 (m)) achieve the max-mi solutio F m (λ). Fially, let s z 0() ad s y 0 () be the states that achieve the max-mi solutio for F (λ). We show that F (λ) is sup-additive, i.e., for every iteger m, l [0, ] with m + l we have F (λ) lf l (λ) + mf m (λ). Sup-additivity is veried by breakig the legth expressios ito expressios of legth l ad expressios of legth m. The critical part here is to cosider the legth m sequece from l+ to. Here we use the fact that give the iitial state the chael is statioary, so p(zl+, Y l+ x l+, s l s 0 ) p(z m, Y m x m x l+, s 0). This, combied with the fact the cardiality boud depeds oly o the legth of the sequece, leads to the coclusio that the joit distributio q 2 (u m, x m ) that maximizes F m (λ) will maximize the segmet from l + to (i.e. is the maximizig distributio for (Ul+, X l+ ), with the same iitial state). Additioally, both I(U ; Z s 0 ) ad I(X ; Y U, s 0) are bouded from above, idepedet of : I(U ; Z s 0 ) log 2 Z, sice all the Z i 's are deed over the same alphabet Z i Z, ad similarly I(X ; Y U, s 0) log 2 X. Thus, F (λ) log 2 Z + λ log 2 X < for ay λ [0, ]. The fact that F (λ) is bouded from above idepedet of ad is also sup-additive implies that lim F (λ) exists ad is ite. Combiig the fact that the limit exists with sectios IV-A, IV-B ad IV-C gives the capacity of the FSBC of Theorem. REFERENCES [] B. McMilla. The Basic Theorems of Iformatio Theory. The Aals of Mathematical Statistics, Vol-24(2):96 29, 953. [2] Y. Steiberg. Codig for the Degraded Broadcast Chael With Radom Parameters, With Causal ad Nocausal Side Iformatio. IEEE Tras. Iform. Theory, IT-5(8): , [3] A. Wishtok ad Y. Steiberg. The Arbitrarily Varyig Degraded Broadcast Chael with States Kow at the Ecoder. Iteratioal Symposium o Iformatio Theory (ISIT) 2006, Seattle, WA, pp [4] R. G. Gallager. Iformatio Theory ad Reliable Commuicatio. Joh Wiley ad Sos Ic., 968. [5] H. Permuter ad T. Weissma. Capacity Regio of the Fiite-State Multiple Access Chael with ad without Feedback. Submitted to the IEEE Tras. Iform. Theory, [6] T. M. Cover ad J. Thomas. Elemets of Iformatio Theory. Joh Wiley ad Sos Ic., 99. [7] R. Dabora ad A. Goldsmith. The Capacity Regio of the Degraded Fiite-State Broadcast Chael. Submitted to the IEEE Tras. Iform. Theory, [8] R. G. Gallager. Capacity ad Codig for Degraded Broadcast Chaels. Problemy Peredachi Iformatsii, vol. 0(3):3 4, 974.

Capacity Theorems for the Finite-State Broadcast Channel with Feedback

Capacity Theorems for the Finite-State Broadcast Channel with Feedback Capacity Theorems for the Fiite-State Broadcast Chael with Feedback Ro abora ad Adrea Goldsmith ept of Electrical Egieerig, Staford Uiversity Abstract We cosider the discrete, time-varyig broadcast chael

More information

The Capacity Region of the. Degraded Finite-State Broadcast Channel

The Capacity Region of the. Degraded Finite-State Broadcast Channel Submitted to the IEEE Trasactios o Iformatio Theory, 2008. The Capacity Regio of the Degraded Fiite-State Broadcast Chael Ro Dabora ad Adrea Goldsmith Wireless Systems Lab Departmet of Electrical Egieerig

More information

Information Theory Tutorial Communication over Channels with memory. Chi Zhang Department of Electrical Engineering University of Notre Dame

Information Theory Tutorial Communication over Channels with memory. Chi Zhang Department of Electrical Engineering University of Notre Dame Iformatio Theory Tutorial Commuicatio over Chaels with memory Chi Zhag Departmet of Electrical Egieerig Uiversity of Notre Dame Abstract A geeral capacity formula C = sup I(; Y ), which is correct for

More information

Asymptotic Coupling and Its Applications in Information Theory

Asymptotic Coupling and Its Applications in Information Theory Asymptotic Couplig ad Its Applicatios i Iformatio Theory Vicet Y. F. Ta Joit Work with Lei Yu Departmet of Electrical ad Computer Egieerig, Departmet of Mathematics, Natioal Uiversity of Sigapore IMS-APRM

More information

Entropies & Information Theory

Entropies & Information Theory Etropies & Iformatio Theory LECTURE I Nilajaa Datta Uiversity of Cambridge,U.K. For more details: see lecture otes (Lecture 1- Lecture 5) o http://www.qi.damtp.cam.ac.uk/ode/223 Quatum Iformatio Theory

More information

Lecture 11: Channel Coding Theorem: Converse Part

Lecture 11: Channel Coding Theorem: Converse Part EE376A/STATS376A Iformatio Theory Lecture - 02/3/208 Lecture : Chael Codig Theorem: Coverse Part Lecturer: Tsachy Weissma Scribe: Erdem Bıyık I this lecture, we will cotiue our discussio o chael codig

More information

Lecture 15: Strong, Conditional, & Joint Typicality

Lecture 15: Strong, Conditional, & Joint Typicality EE376A/STATS376A Iformatio Theory Lecture 15-02/27/2018 Lecture 15: Strog, Coditioal, & Joit Typicality Lecturer: Tsachy Weissma Scribe: Nimit Sohoi, William McCloskey, Halwest Mohammad I this lecture,

More information

Lecture 7: Channel coding theorem for discrete-time continuous memoryless channel

Lecture 7: Channel coding theorem for discrete-time continuous memoryless channel Lecture 7: Chael codig theorem for discrete-time cotiuous memoryless chael Lectured by Dr. Saif K. Mohammed Scribed by Mirsad Čirkić Iformatio Theory for Wireless Commuicatio ITWC Sprig 202 Let us first

More information

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i

More information

Lecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound

Lecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound Lecture 7 Ageda for the lecture Gaussia chael with average power costraits Capacity of additive Gaussia oise chael ad the sphere packig boud 7. Additive Gaussia oise chael Up to this poit, we have bee

More information

Symmetric Two-User Gaussian Interference Channel with Common Messages

Symmetric Two-User Gaussian Interference Channel with Common Messages Symmetric Two-User Gaussia Iterferece Chael with Commo Messages Qua Geg CSL ad Dept. of ECE UIUC, IL 680 Email: geg5@illiois.edu Tie Liu Dept. of Electrical ad Computer Egieerig Texas A&M Uiversity, TX

More information

Cooperative Communication Fundamentals & Coding Techniques

Cooperative Communication Fundamentals & Coding Techniques 3 th ICACT Tutorial Cooperative commuicatio fudametals & codig techiques Cooperative Commuicatio Fudametals & Codig Techiques 0..4 Electroics ad Telecommuicatio Research Istitute Kiug Jug 3 th ICACT Tutorial

More information

Entropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP

Entropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP Etropy ad Ergodic Theory Lecture 5: Joit typicality ad coditioal AEP 1 Notatio: from RVs back to distributios Let (Ω, F, P) be a probability space, ad let X ad Y be A- ad B-valued discrete RVs, respectively.

More information

The Maximum-Likelihood Decoding Performance of Error-Correcting Codes

The Maximum-Likelihood Decoding Performance of Error-Correcting Codes The Maximum-Lielihood Decodig Performace of Error-Correctig Codes Hery D. Pfister ECE Departmet Texas A&M Uiversity August 27th, 2007 (rev. 0) November 2st, 203 (rev. ) Performace of Codes. Notatio X,

More information

On Random Line Segments in the Unit Square

On Random Line Segments in the Unit Square O Radom Lie Segmets i the Uit Square Thomas A. Courtade Departmet of Electrical Egieerig Uiversity of Califoria Los Ageles, Califoria 90095 Email: tacourta@ee.ucla.edu I. INTRODUCTION Let Q = [0, 1] [0,

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak

More information

On the Capacity of Symmetric Gaussian Interference Channels with Feedback

On the Capacity of Symmetric Gaussian Interference Channels with Feedback O the Capacity of Symmetric Gaussia Iterferece Chaels with Feedback La V Truog Iformatio Techology Specializatio Departmet ITS FPT Uiversity, Haoi, Vietam E-mail: latv@fpteduv Hirosuke Yamamoto Dept of

More information

Lecture 7: October 18, 2017

Lecture 7: October 18, 2017 Iformatio ad Codig Theory Autum 207 Lecturer: Madhur Tulsiai Lecture 7: October 8, 207 Biary hypothesis testig I this lecture, we apply the tools developed i the past few lectures to uderstad the problem

More information

Convergence of random variables. (telegram style notes) P.J.C. Spreij

Convergence of random variables. (telegram style notes) P.J.C. Spreij Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space

More information

Lecture 7: MIMO Architectures Theoretical Foundations of Wireless Communications 1. Overview. CommTh/EES/KTH

Lecture 7: MIMO Architectures Theoretical Foundations of Wireless Communications 1. Overview. CommTh/EES/KTH : Theoretical Foudatios of Wireless Commuicatios 1 Thursday, May 19, 2016 12:30-15:30, Coferece Room SIP 1 Textbook: D. Tse ad P. Viswaath, Fudametals of Wireless Commuicatio 1 / 1 Overview Lecture 6:

More information

SUCCESSIVE INTERFERENCE CANCELLATION DECODING FOR THE K -USER CYCLIC INTERFERENCE CHANNEL

SUCCESSIVE INTERFERENCE CANCELLATION DECODING FOR THE K -USER CYCLIC INTERFERENCE CHANNEL Joural of Theoretical ad Applied Iformatio Techology 31 st December 212 Vol 46 No2 25-212 JATIT & LLS All rights reserved ISSN: 1992-8645 wwwatitorg E-ISSN: 1817-3195 SCCESSIVE INTERFERENCE CANCELLATION

More information

Multiterminal source coding with complementary delivery

Multiterminal source coding with complementary delivery Iteratioal Symposium o Iformatio Theory ad its Applicatios, ISITA2006 Seoul, Korea, October 29 November 1, 2006 Multitermial source codig with complemetary delivery Akisato Kimura ad Tomohiko Uyematsu

More information

Universal source coding for complementary delivery

Universal source coding for complementary delivery SITA2006 i Hakodate 2005.2. p. Uiversal source codig for complemetary delivery Akisato Kimura, 2, Tomohiko Uyematsu 2, Shigeaki Kuzuoka 2 Media Iformatio Laboratory, NTT Commuicatio Sciece Laboratories,

More information

Information Theory and Statistics Lecture 4: Lempel-Ziv code

Information Theory and Statistics Lecture 4: Lempel-Ziv code Iformatio Theory ad Statistics Lecture 4: Lempel-Ziv code Łukasz Dębowski ldebowsk@ipipa.waw.pl Ph. D. Programme 203/204 Etropy rate is the limitig compressio rate Theorem For a statioary process (X i)

More information

Finite Block-Length Gains in Distributed Source Coding

Finite Block-Length Gains in Distributed Source Coding Decoder Fiite Block-Legth Gais i Distributed Source Codig Farhad Shirai EECS Departmet Uiversity of Michiga A Arbor,USA Email: fshirai@umichedu S Sadeep Pradha EECS Departmet Uiversity of Michiga A Arbor,USA

More information

Overview of Gaussian MIMO (Vector) BC

Overview of Gaussian MIMO (Vector) BC Overview of Gaussia MIMO (Vector) BC Gwamo Ku Adaptive Sigal Processig ad Iformatio Theory Research Group Nov. 30, 2012 Outlie / Capacity Regio of Gaussia MIMO BC System Structure Kow Capacity Regios -

More information

Lecture 19: Convergence

Lecture 19: Convergence Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may

More information

A Partial Decode-Forward Scheme For A Network with N relays

A Partial Decode-Forward Scheme For A Network with N relays A Partial Decode-Forward Scheme For A etwork with relays Yao Tag ECE Departmet, McGill Uiversity Motreal, QC, Caada Email: yaotag2@mailmcgillca Mai Vu ECE Departmet, Tufts Uiversity Medford, MA, USA Email:

More information

Generalized Semi- Markov Processes (GSMP)

Generalized Semi- Markov Processes (GSMP) Geeralized Semi- Markov Processes (GSMP) Summary Some Defiitios Markov ad Semi-Markov Processes The Poisso Process Properties of the Poisso Process Iterarrival times Memoryless property ad the residual

More information

Increasing timing capacity using packet coloring

Increasing timing capacity using packet coloring 003 Coferece o Iformatio Scieces ad Systems, The Johs Hopkis Uiversity, March 4, 003 Icreasig timig capacity usig packet colorig Xi Liu ad R Srikat[] Coordiated Sciece Laboratory Uiversity of Illiois e-mail:

More information

Optimally Sparse SVMs

Optimally Sparse SVMs A. Proof of Lemma 3. We here prove a lower boud o the umber of support vectors to achieve geeralizatio bouds of the form which we cosider. Importatly, this result holds ot oly for liear classifiers, but

More information

1 Review and Overview

1 Review and Overview DRAFT a fial versio will be posted shortly CS229T/STATS231: Statistical Learig Theory Lecturer: Tegyu Ma Lecture #3 Scribe: Migda Qiao October 1, 2013 1 Review ad Overview I the first half of this course,

More information

A New Achievability Scheme for the Relay Channel

A New Achievability Scheme for the Relay Channel A New Achievability Scheme for the Relay Chael Wei Kag Seur Ulukus Departmet of Electrical ad Computer Egieerig Uiversity of Marylad, College Park, MD 20742 wkag@umd.edu ulukus@umd.edu October 4, 2007

More information

Infinite Sequences and Series

Infinite Sequences and Series Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet

More information

Hybrid Coding for Gaussian Broadcast Channels with Gaussian Sources

Hybrid Coding for Gaussian Broadcast Channels with Gaussian Sources Hybrid Codig for Gaussia Broadcast Chaels with Gaussia Sources Rajiv Soudararaja Departmet of Electrical & Computer Egieerig Uiversity of Texas at Austi Austi, TX 7871, USA Email: rajivs@mailutexasedu

More information

The Fading Number of Multiple-Input Multiple-Output Fading Channels with Memory

The Fading Number of Multiple-Input Multiple-Output Fading Channels with Memory The Fadig Number of Multiple-Iput Multiple-Output Fadig Chaels with Memory Stefa M. Moser Departmet of Commuicatio Egieerig Natioal Chiao Tug Uiversity NCTU Hsichu, Taiwa Email: stefa.moser@ieee.org Abstract

More information

Journal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula

Journal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula Joural of Multivariate Aalysis 102 (2011) 1315 1319 Cotets lists available at ScieceDirect Joural of Multivariate Aalysis joural homepage: www.elsevier.com/locate/jmva Superefficiet estimatio of the margials

More information

Lecture 6: Source coding, Typicality, and Noisy channels and capacity

Lecture 6: Source coding, Typicality, and Noisy channels and capacity 15-859: Iformatio Theory ad Applicatios i TCS CMU: Sprig 2013 Lecture 6: Source codig, Typicality, ad Noisy chaels ad capacity Jauary 31, 2013 Lecturer: Mahdi Cheraghchi Scribe: Togbo Huag 1 Recap Uiversal

More information

Properties of Fuzzy Length on Fuzzy Set

Properties of Fuzzy Length on Fuzzy Set Ope Access Library Joural 206, Volume 3, e3068 ISSN Olie: 2333-972 ISSN Prit: 2333-9705 Properties of Fuzzy Legth o Fuzzy Set Jehad R Kider, Jaafar Imra Mousa Departmet of Mathematics ad Computer Applicatios,

More information

Disjoint Systems. Abstract

Disjoint Systems. Abstract Disjoit Systems Noga Alo ad Bey Sudaov Departmet of Mathematics Raymod ad Beverly Sacler Faculty of Exact Scieces Tel Aviv Uiversity, Tel Aviv, Israel Abstract A disjoit system of type (,,, ) is a collectio

More information

Channel coding, linear block codes, Hamming and cyclic codes Lecture - 8

Channel coding, linear block codes, Hamming and cyclic codes Lecture - 8 Digital Commuicatio Chael codig, liear block codes, Hammig ad cyclic codes Lecture - 8 Ir. Muhamad Asial, MSc., PhD Ceter for Iformatio ad Commuicatio Egieerig Research (CICER) Electrical Egieerig Departmet

More information

On Routing-Optimal Network for Multiple Unicasts

On Routing-Optimal Network for Multiple Unicasts O Routig-Optimal Network for Multiple Uicasts Chu Meg, Athia Markopoulou Abstract I this paper, we cosider etworks with multiple uicast sessios. Geerally, o-liear etwork codig is eeded to achieve the whole

More information

Information-based Feature Selection

Information-based Feature Selection Iformatio-based Feature Selectio Farza Faria, Abbas Kazeroui, Afshi Babveyh Email: {faria,abbask,afshib}@staford.edu 1 Itroductio Feature selectio is a topic of great iterest i applicatios dealig with

More information

Random Walks on Discrete and Continuous Circles. by Jeffrey S. Rosenthal School of Mathematics, University of Minnesota, Minneapolis, MN, U.S.A.

Random Walks on Discrete and Continuous Circles. by Jeffrey S. Rosenthal School of Mathematics, University of Minnesota, Minneapolis, MN, U.S.A. Radom Walks o Discrete ad Cotiuous Circles by Jeffrey S. Rosethal School of Mathematics, Uiversity of Miesota, Mieapolis, MN, U.S.A. 55455 (Appeared i Joural of Applied Probability 30 (1993), 780 789.)

More information

Lecture 14: Graph Entropy

Lecture 14: Graph Entropy 15-859: Iformatio Theory ad Applicatios i TCS Sprig 2013 Lecture 14: Graph Etropy March 19, 2013 Lecturer: Mahdi Cheraghchi Scribe: Euiwoog Lee 1 Recap Bergma s boud o the permaet Shearer s Lemma Number

More information

On Multipath Fading Channels at High SNR

On Multipath Fading Channels at High SNR O Multipath Fadig Chaels at High SNR Tobias Koch Amos Lapidoth ETH Zurich, Switzerlad Email: toch, lapidoth}@isi.ee.ethz.ch ISIT 008, Toroto, Caada, July 6 -, 008 Abstract This paper studies the capacity

More information

Problem Set 4 Due Oct, 12

Problem Set 4 Due Oct, 12 EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios

More information

Fixed-Threshold Polar Codes

Fixed-Threshold Polar Codes Fixed-Threshold Polar Codes Jig Guo Uiversity of Cambridge jg582@cam.ac.uk Albert Guillé i Fàbregas ICREA & Uiversitat Pompeu Fabra Uiversity of Cambridge guille@ieee.org Jossy Sayir Uiversity of Cambridge

More information

TEACHER CERTIFICATION STUDY GUIDE

TEACHER CERTIFICATION STUDY GUIDE COMPETENCY 1. ALGEBRA SKILL 1.1 1.1a. ALGEBRAIC STRUCTURES Kow why the real ad complex umbers are each a field, ad that particular rigs are ot fields (e.g., itegers, polyomial rigs, matrix rigs) Algebra

More information

Axioms of Measure Theory

Axioms of Measure Theory MATH 532 Axioms of Measure Theory Dr. Neal, WKU I. The Space Throughout the course, we shall let X deote a geeric o-empty set. I geeral, we shall ot assume that ay algebraic structure exists o X so that

More information

Capacity of Compound State-Dependent Channels with States Known at the Transmitter

Capacity of Compound State-Dependent Channels with States Known at the Transmitter Capacity of Compoud State-Depedet Chaels with States Kow at the Trasmitter Pablo Piataida Departmet of Telecommuicatios SUPELEC Plateau de Moulo, 992 Gif-sur-Yvette, Frace Email: pablo.piataida@supelec.fr

More information

arxiv: v1 [cs.it] 13 Jul 2012

arxiv: v1 [cs.it] 13 Jul 2012 O the Sum Capacity of the Discrete Memoryless Iterferece Chael with Oe-Sided Weak Iterferece ad Mixed Iterferece Fagfag Zhu ad Biao Che Syracuse Uiversity Departmet of EECS Syracuse, NY 3244 Email: fazhu{biche}@syr.edu

More information

If a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero?

If a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero? 2 Lebesgue Measure I Chapter 1 we defied the cocept of a set of measure zero, ad we have observed that every coutable set is of measure zero. Here are some atural questios: If a subset E of R cotais a

More information

Inequalities for Entropies of Sets of Subsets of Random Variables

Inequalities for Entropies of Sets of Subsets of Random Variables Iequalities for Etropies of Sets of Subsets of Radom Variables Chao Tia AT&T Labs-Research Florham Par, NJ 0792, USA. tia@research.att.com Abstract Ha s iequality o the etropy rates of subsets of radom

More information

Product measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.

Product measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014. Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the

More information

lim za n n = z lim a n n.

lim za n n = z lim a n n. Lecture 6 Sequeces ad Series Defiitio 1 By a sequece i a set A, we mea a mappig f : N A. It is customary to deote a sequece f by {s } where, s := f(). A sequece {z } of (complex) umbers is said to be coverget

More information

Are Slepian-Wolf Rates Necessary for Distributed Parameter Estimation?

Are Slepian-Wolf Rates Necessary for Distributed Parameter Estimation? Are Slepia-Wolf Rates Necessary for Distributed Parameter Estimatio? Mostafa El Gamal ad Lifeg Lai Departmet of Electrical ad Computer Egieerig Worcester Polytechic Istitute {melgamal, llai}@wpi.edu arxiv:1508.02765v2

More information

Entropy Rates and Asymptotic Equipartition

Entropy Rates and Asymptotic Equipartition Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,

More information

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1 EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum

More information

1 Duality revisited. AM 221: Advanced Optimization Spring 2016

1 Duality revisited. AM 221: Advanced Optimization Spring 2016 AM 22: Advaced Optimizatio Sprig 206 Prof. Yaro Siger Sectio 7 Wedesday, Mar. 9th Duality revisited I this sectio, we will give a slightly differet perspective o duality. optimizatio program: f(x) x R

More information

The multiplicative structure of finite field and a construction of LRC

The multiplicative structure of finite field and a construction of LRC IERG6120 Codig for Distributed Storage Systems Lecture 8-06/10/2016 The multiplicative structure of fiite field ad a costructio of LRC Lecturer: Keeth Shum Scribe: Zhouyi Hu Notatios: We use the otatio

More information

Lecture 2. The Lovász Local Lemma

Lecture 2. The Lovász Local Lemma Staford Uiversity Sprig 208 Math 233A: No-costructive methods i combiatorics Istructor: Ja Vodrák Lecture date: Jauary 0, 208 Origial scribe: Apoorva Khare Lecture 2. The Lovász Local Lemma 2. Itroductio

More information

Advanced Stochastic Processes.

Advanced Stochastic Processes. Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.

More information

Chapter 7: The z-transform. Chih-Wei Liu

Chapter 7: The z-transform. Chih-Wei Liu Chapter 7: The -Trasform Chih-Wei Liu Outlie Itroductio The -Trasform Properties of the Regio of Covergece Properties of the -Trasform Iversio of the -Trasform The Trasfer Fuctio Causality ad Stability

More information

Lecture 3 : Random variables and their distributions

Lecture 3 : Random variables and their distributions Lecture 3 : Radom variables ad their distributios 3.1 Radom variables Let (Ω, F) ad (S, S) be two measurable spaces. A map X : Ω S is measurable or a radom variable (deoted r.v.) if X 1 (A) {ω : X(ω) A}

More information

Random Models. Tusheng Zhang. February 14, 2013

Random Models. Tusheng Zhang. February 14, 2013 Radom Models Tusheg Zhag February 14, 013 1 Radom Walks Let me describe the model. Radom walks are used to describe the motio of a movig particle (object). Suppose that a particle (object) moves alog the

More information

Non-Asymptotic Achievable Rates for Gaussian Energy-Harvesting Channels: Best-Effort and Save-and-Transmit

Non-Asymptotic Achievable Rates for Gaussian Energy-Harvesting Channels: Best-Effort and Save-and-Transmit No-Asymptotic Achievable Rates for Gaussia Eergy-Harvestig Chaels: Best-Effort ad Save-ad-Trasmit Silas L. Fog, Jig Yag, ad Ayli Yeer arxiv:805.089v [cs.it] 30 May 08 Abstract A additive white Gaussia

More information

REGRESSION WITH QUADRATIC LOSS

REGRESSION WITH QUADRATIC LOSS REGRESSION WITH QUADRATIC LOSS MAXIM RAGINSKY Regressio with quadratic loss is aother basic problem studied i statistical learig theory. We have a radom couple Z = X, Y ), where, as before, X is a R d

More information

The source coding game with a cheating switcher

The source coding game with a cheating switcher The source codig game with a cheatig switcher Hari Palaiyaur hpalaiya@eecsberkeleyedu Cheg Chag cchag@eecsberkeleyedu Departmet of Electrical Egieerig ad Computer Scieces Uiversity of Califoria at Berkeley

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece

More information

5.1 A mutual information bound based on metric entropy

5.1 A mutual information bound based on metric entropy Chapter 5 Global Fao Method I this chapter, we exted the techiques of Chapter 2.4 o Fao s method the local Fao method) to a more global costructio. I particular, we show that, rather tha costructig a local

More information

IET Commun., 2009, Vol. 3, Iss. 1, pp doi: /iet-com: & The Institution of Engineering and Technology 2009

IET Commun., 2009, Vol. 3, Iss. 1, pp doi: /iet-com: & The Institution of Engineering and Technology 2009 Published i IET Commuicatios Received o 4th Jue 28 Revised o 3th July 28 doi: 1.149/iet-com:28373 ISSN 1751-8628 Symmetric relayig based o partial decodig ad the capacity of a class of relay etworks L.

More information

ECE 564/645 - Digital Communication Systems (Spring 2014) Final Exam Friday, May 2nd, 8:00-10:00am, Marston 220

ECE 564/645 - Digital Communication Systems (Spring 2014) Final Exam Friday, May 2nd, 8:00-10:00am, Marston 220 ECE 564/645 - Digital Commuicatio Systems (Sprig 014) Fial Exam Friday, May d, 8:00-10:00am, Marsto 0 Overview The exam cosists of four (or five) problems for 100 (or 10) poits. The poits for each part

More information

(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3

(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3 MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special

More information

Resilient Source Coding

Resilient Source Coding Resiliet Source Codig Maël Le Treust Laboratoire des Sigaux et Systèmes CNRS - Supélec - Uiv. Paris Sud 11 91191, Gif-sur-Yvette, Frace Email: mael.letreust@lss.supelec.fr Samso Lasaulce Laboratoire des

More information

R. van Zyl 1, A.J. van der Merwe 2. Quintiles International, University of the Free State

R. van Zyl 1, A.J. van der Merwe 2. Quintiles International, University of the Free State Bayesia Cotrol Charts for the Two-parameter Expoetial Distributio if the Locatio Parameter Ca Take o Ay Value Betwee Mius Iity ad Plus Iity R. va Zyl, A.J. va der Merwe 2 Quitiles Iteratioal, ruaavz@gmail.com

More information

Rademacher Complexity

Rademacher Complexity EECS 598: Statistical Learig Theory, Witer 204 Topic 0 Rademacher Complexity Lecturer: Clayto Scott Scribe: Ya Deg, Kevi Moo Disclaimer: These otes have ot bee subjected to the usual scrutiy reserved for

More information

Discrete-Time Systems, LTI Systems, and Discrete-Time Convolution

Discrete-Time Systems, LTI Systems, and Discrete-Time Convolution EEL5: Discrete-Time Sigals ad Systems. Itroductio I this set of otes, we begi our mathematical treatmet of discrete-time s. As show i Figure, a discrete-time operates or trasforms some iput sequece x [

More information

Multiterminal Source Coding with an Entropy-Based Distortion Measure

Multiterminal Source Coding with an Entropy-Based Distortion Measure 20 IEEE Iteratioal Symposium o Iformatio Theory Proceedigs Multitermial Source Codig with a Etropy-Based Distortio Measure Thomas A. Courtade ad Richard D. Wesel Departmet of Electrical Egieerig Uiversity

More information

The Boolean Ring of Intervals

The Boolean Ring of Intervals MATH 532 Lebesgue Measure Dr. Neal, WKU We ow shall apply the results obtaied about outer measure to the legth measure o the real lie. Throughout, our space X will be the set of real umbers R. Whe ecessary,

More information

Empirical Processes: Glivenko Cantelli Theorems

Empirical Processes: Glivenko Cantelli Theorems Empirical Processes: Gliveko Catelli Theorems Mouliath Baerjee Jue 6, 200 Gliveko Catelli classes of fuctios The reader is referred to Chapter.6 of Weller s Torgo otes, Chapter??? of VDVW ad Chapter 8.3

More information

Randomized Algorithms I, Spring 2018, Department of Computer Science, University of Helsinki Homework 1: Solutions (Discussed January 25, 2018)

Randomized Algorithms I, Spring 2018, Department of Computer Science, University of Helsinki Homework 1: Solutions (Discussed January 25, 2018) Radomized Algorithms I, Sprig 08, Departmet of Computer Sciece, Uiversity of Helsiki Homework : Solutios Discussed Jauary 5, 08). Exercise.: Cosider the followig balls-ad-bi game. We start with oe black

More information

6.3 Testing Series With Positive Terms

6.3 Testing Series With Positive Terms 6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial

More information

1 Convergence in Probability and the Weak Law of Large Numbers

1 Convergence in Probability and the Weak Law of Large Numbers 36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec

More information

The Three-Terminal Interactive. Lossy Source Coding Problem

The Three-Terminal Interactive. Lossy Source Coding Problem The Three-Termial Iteractive 1 Lossy Source Codig Problem Leoardo Rey Vega, Pablo Piataida ad Alfred O. Hero III arxiv:1502.01359v3 cs.it] 18 Ja 2016 Abstract The three-ode multitermial lossy source codig

More information

The Poisson Process *

The Poisson Process * OpeStax-CNX module: m11255 1 The Poisso Process * Do Johso This work is produced by OpeStax-CNX ad licesed uder the Creative Commos Attributio Licese 1.0 Some sigals have o waveform. Cosider the measuremet

More information

Lecture 15: Learning Theory: Concentration Inequalities

Lecture 15: Learning Theory: Concentration Inequalities STAT 425: Itroductio to Noparametric Statistics Witer 208 Lecture 5: Learig Theory: Cocetratio Iequalities Istructor: Ye-Chi Che 5. Itroductio Recall that i the lecture o classificatio, we have see that

More information

Binary classification, Part 1

Binary classification, Part 1 Biary classificatio, Part 1 Maxim Ragisky September 25, 2014 The problem of biary classificatio ca be stated as follows. We have a radom couple Z = (X,Y ), where X R d is called the feature vector ad Y

More information

Basics of Probability Theory (for Theory of Computation courses)

Basics of Probability Theory (for Theory of Computation courses) Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.

More information

The z-transform. 7.1 Introduction. 7.2 The z-transform Derivation of the z-transform: x[n] = z n LTI system, h[n] z = re j

The z-transform. 7.1 Introduction. 7.2 The z-transform Derivation of the z-transform: x[n] = z n LTI system, h[n] z = re j The -Trasform 7. Itroductio Geeralie the complex siusoidal represetatio offered by DTFT to a represetatio of complex expoetial sigals. Obtai more geeral characteristics for discrete-time LTI systems. 7.

More information

Lecture 10 October Minimaxity and least favorable prior sequences

Lecture 10 October Minimaxity and least favorable prior sequences STATS 300A: Theory of Statistics Fall 205 Lecture 0 October 22 Lecturer: Lester Mackey Scribe: Brya He, Rahul Makhijai Warig: These otes may cotai factual ad/or typographic errors. 0. Miimaxity ad least

More information

An Introduction to Randomized Algorithms

An Introduction to Randomized Algorithms A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis

More information

Lecture 3 The Lebesgue Integral

Lecture 3 The Lebesgue Integral Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified

More information

Vector Permutation Code Design Algorithm. Danilo SILVA and Weiler A. FINAMORE

Vector Permutation Code Design Algorithm. Danilo SILVA and Weiler A. FINAMORE Iteratioal Symposium o Iformatio Theory ad its Applicatios, ISITA2004 Parma, Italy, October 10 13, 2004 Vector Permutatio Code Desig Algorithm Dailo SILVA ad Weiler A. FINAMORE Cetro de Estudos em Telecomuicações

More information

Binary Fading Interference Channel with No CSIT

Binary Fading Interference Channel with No CSIT Biary Fadig Iterferece Chael with No CSIT Alireza Vahid, Mohammad Ali Maddah-Ali, A. Salma Avestimehr, ad Ya Zhu arxiv:405.003v3 [cs.it] 4 Mar 07 Abstract We study the capacity regio of the two-user Biary

More information

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters

More information

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1. Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio

More information

Statistical Pattern Recognition

Statistical Pattern Recognition Statistical Patter Recogitio Classificatio: No-Parametric Modelig Hamid R. Rabiee Jafar Muhammadi Sprig 2014 http://ce.sharif.edu/courses/92-93/2/ce725-2/ Ageda Parametric Modelig No-Parametric Modelig

More information

Inseparability of the Multiple Access Wiretap Channel

Inseparability of the Multiple Access Wiretap Channel Iseparability of the Multiple Access Wiretap Chael Jiawei Xie Seur Ulukus Departmet of Electrical ad Computer Egieerig Uiversity of Marylad, College Park, MD 074 xiejw@umd.edu ulukus@umd.edu Abstract We

More information

THE interference channel problem describes a setup where multiple pairs of transmitters and receivers share a communication

THE interference channel problem describes a setup where multiple pairs of transmitters and receivers share a communication Trade-off betwee Commuicatio ad Cooperatio i the Iterferece Chael Farhad Shirai EECS Departmet Uiversity of Michiga A Arbor,USA Email: fshirai@umich.edu S. Sadeep Pradha EECS Departmet Uiversity of Michiga

More information