Inseparability of the Multiple Access Wiretap Channel

Size: px
Start display at page:

Download "Inseparability of the Multiple Access Wiretap Channel"

Transcription

1 Iseparability of the Multiple Access Wiretap Chael Jiawei Xie Seur Ulukus Departmet of Electrical ad Computer Egieerig Uiversity of Marylad, College Park, MD 074 Abstract We examie the separability of the parallel multiple access wiretap chael. Separability, whe exists, is useful as it eables us to code separately over parallel chaels, ad still achieve the optimum overall performace. It is well-kow that the parallel sigle-user chael, parallel multiple access chael (MAC) ad parallel broadcast chael (BC) are all separable, however, the parallel iterferece chael (IC) is ot separable i geeral. I this paper, we show that, while MAC is separable MAC wiretap chael is ot separable i geeral. We prove this via a specific liear determiistic MAC wiretap chael. We the show that eve the Gaussia MAC wiretap chael is iseparable i geeral. Fially, we show that, whe the chael gais are draw from cotiuous distributios, ad whe the secure degrees of freedom (s.d.o.f.) regio is cosidered, the the Gaussia MAC wiretap chael is almost surely separable. I. INTRODUCTION Separability, whe exists, is useful as it eables us to code separately over parallel chaels, ad still achieve the optimum overall performace. It is well-kow that the parallel sigleuser chael, parallel multiple access chael (MAC) ad parallel broadcast chael (BC) 3 are all separable, however, the parallel iterferece chael (IC) is ot separable i geeral 4 7. I particular, referece 4 studied the twouser oe-sided ergodic fadig IC ad showed that separatio ca be strictly sub-optimal i certai cases. Referece 5 studied the separability i a parallel Gaussia IC, ad showed that the parallel Gaussia IC is ot always separable by presetig a specific example where joit ecodig over the parallel chaels outperforms idividually optimal ecodig i each parallel chael. Referece 6 further cofirmed the iseparability of the parallel IC by examiig the topological iterferece chael where the parallel chaels correspod to differet etwork topologies some of which had asymmetric coectivity. Recetly, referece 7 showed that eve symmetric parallel ICs are iseparable by characterizig the capacity regio of parallel symmetric liear determiistic ICs. I this paper, we cosider the MAC wiretap chael, which is a combiatio of a MAC to the legitimate receiver ad a MAC to the eavesdropper. The MAC wiretap chael was itroduced i 8, 9 ad studied further i 0 6. Eve though, i the absece of ay secrecy costraits, MAC is the most well-uderstood multi-user chael model, its wiretap versio is sigificatly more complex. The secrecy capacity This work was supported by NSF Grats CNS , CCF , CCF ad CNS regio of the MAC wiretap chael is still ukow today, ad its secure degrees of freedom (s.d.o.f.) regio has bee fully characterized oly recetly 7, 8. I this paper, we focus o the separability of the parallel MAC wiretap chael ad show that it is ot separable i geeral. Ituitively, this ca be attributed to the observatio that, eve though MAC wiretap chael is composed of MAC legitimate ad eavesdroppig liks, as a whole, it resembles the IC more, as it has two idepedet trasmitters ad two idepedet receivers. To show the iseparability of the parallel MAC wiretap chael, we costruct a specific liear determiistic MAC wiretap chael i each compoet chael. We fid the exact secrecy capacity of each of these compoet MAC wiretap chaels, ad the determie the optimum secrecy rates achievable by separate ecodig. This step is challegig as the secrecy capacity of MAC wiretap chaels is ukow i geeral; we provide a specific achievability ad coverse for the capacity of each of the compoet chaels. We the provide a ecodig scheme that codes over the parallel chaels which outperforms the optimum separable scheme. Next, we cosider the parallel Gaussia MAC wiretap chael. Sice the secrecy capacity regio of the geeral MAC wiretap chael, icludig the Gaussia MAC wiretap chael, is ukow but exact s.d.o.f. regio is kow 7, 8, we ivestigate the sum s.d.o.f. of parallel Gaussia MAC wiretap chaels ad prove that it is iseparable. This implies the iseparability of the secrecy regio as well. Next, we observe that, if the differet chael gais which give rise to differet parallel chaels are draw idepedetly from cotiuous distributios, the the chael gai cofiguratios which give rise to iseparability fall ito a set with zero Lebesgue measure. To cofirm this observatio, ad prove the almost sure s.d.o.f. separability of parallel Gaussia MAC wiretap chaels, we cosider the flat chael, where we put the idividual chael uses of each compoet chael ito a sigle chael uses. We utilize the coverse techiques i 7, 8 to show the separability i this case. Fially, we ote that, while iseparability i s.d.o.f. implies iseparability i the secrecy capacity, separability i s.d.o.f. does ot imply separability i secrecy capacities. The almost sure separability proved for the parallel Gaussia MAC wiretap chael i this paper holds oly for the s.d.o.f., which is the pre-log factor of the secrecy capacity, ad is a weaker measure of separability.

2 II. SYSTEM MODEL AND DEFINITIONS I a two-user MAC wiretap chael p(y, y x, x ), each trasmitter i, i =,, has a message W i iteded for the legitimate receiver whose chael output is Y. For each i, message W i is uiformly ad idepedetly chose from set W i. The rate of message i is R i = log W i. Trasmitter i uses a stochastic fuctio f i : W i Xi, where the - legth vector Xi deotes the ith user s chael iput i chael uses. All messages are eeded to be kept secret from the eavesdropper whose chael output is Y. A secrecy rate pair (R, R ) is said to be achievable if for ay ɛ > 0 there exist -legth codes such that the legitimate receiver ca decode the messages reliably, i.e., the probability of decodig error is less tha ɛ Pr (W, W ) (Ŵ, Ŵ) ɛ () A X X Y U X Y (a) V B C Y X Y Y X Y X (b) A U A U B C V V B ad the messages are kept iformatio-theoretically secure agaist the eavesdropper H(W, W Y ) H(W, W ) ɛ () where Ŵ, Ŵ are the estimates of the messages based o the legitimate receiver s observatio Y. The secrecy capacity regio C is the closure of the set cotaiig all achievable secrecy rate pairs. The sum secrecy capacity is C Σ = sup(r + R ), where the supremum is over all achievable secrecy rate pairs (R, R ) C. For Gaussia MAC wiretap chael with average power costrait P for both trasmitters, the s.d.o.f. regio is defied as: { } R i D s = (d, d ) : (R, R ) C, d i = lim P log P (3) ad the sum s.d.o.f. is defied as: D s,σ = lim P C Σ log P (4) Let p(y a, y a x a, x a ) ad p(y b, y b x b, x b ) be two two-user MAC wiretap chaels. The parallel two-user MAC wiretap chael is a two-user MAC wiretap chael i which the chael iputs of trasmitter ad are (x a, x b ) ad (x a, x b ), respectively, ad the chael iputs are set simultaeously i parallel. The chael outputs of the legitimate receiver ad the eavesdropper are (y a, y b ) ad (y a, y b ), respectively, ad are distributed accordig to p (y a, y a, y b, y b x a, x a, x b, x b ) = p(y a, y a x a, x a )p(y b, y b x b, x b ) (5) We refer to each MAC wiretap chael, p(y a, y a x a, x a ) ad p(y b, y b x b, x b ), as a compoet chael of the overall parallel MAC wiretap chael. III. INSEPARABILITY OF THE MAC WIRETAP CHANNEL I this sectio, we show that the parallel MAC wiretap chael is ot separable i geeral. To this ed, we provide a specific couter example. A V B U X X (c) Y Y A V B U A U B V Fig.. A iseparable liear determiistic parallel MAC wiretap chael. There are three compoet chaels: (a), (b) ad (c). A achievable scheme that codes across the parallel chaels is show i color mageta. Cosider the liear determiistic parallel discrete memoryless MAC wiretap chael show i Fig., which has three compoet chaels: (a), (b) ad (c). I the first compoet chael, (a), trasmitter has two sub-chael iputs, i.e., (X, X ), ad trasmitter has oly oe sub-chael iput X. The legitimate receiver observes (Y, Y ) ad the eavesdropper observes Y. I the secod compoet chael, (b), the roles of the two trasmitters are swapped. I the third compoet chael, (c), the legitimate receiver ad the eavesdropper have idetical observatios. Specifically, the iput/output relatioships for sub-chael (a) are: Y = X, Y = X X, Y = X X (6) where all symbols are biary, ad additio is modulo-. While trasmitters sed idepedet data, they ca each code their data joitly across their parallel chaels. I the followig two sub-sectios, we show that the optimum separable (i.e., idepedet) codig yields bits/chael-use for the sum secrecy rate, while through codig joitly across the compoet chaels a sum secrecy rate of 3 bits/chael-use is achievable, ad hece separatio is strictly sub-optimal. A. Optimum Sum Secrecy Rate with Separable Ecodig Due to idepedet codig across the compoet chaels: C Σ,idep = C Σ,(a) + C Σ,(b) + C Σ,(c) = C Σ,(a) (7) where C Σ,(a) = C Σ,(b) is due to symmetry, ad C Σ,(c) = 0 is due to the fact that the legitimate receiver ad the eavesdropper have idetical observatios. Therefore, we oly eed to show C Σ,(a) = i order to show C Σ,idep =. The achievability of

3 this follows by the followig sigallig: The first user seds a bit (uiform) iformatio sigal i X, ad seds o sigal i the other sub-chael which leaks to the eavesdropper, i.e., X = 0, ad the secod user does ot sed ay iformatio, i.e., X = 0. This gives bit secure rate for the first user, ad hece bit sum secrecy rate for the system, i.e., C Σ,(a). Next, we eed to prove that the sum secrecy rate i the compoet chael (a) is upper bouded by, i.e., C Σ,(a). For coveiece, let us deote R Σ = (R + R ) ɛ i order ot to carry +ɛ throughout the derivatio. The, by defiitio, ad Fao s iequality, we have R Σ = H(W, W ) ɛ (8) I(W, W ; Y, Y) I(W, W ; Y ) (9) Usig the chai rule o both terms o the right had side, R Σ I(W, W ; Y) + I(W, W ; Y Y ) I(W ; Y ) I(W ; Y W ) (0) = I(W ; Y) + I(W ; Y W ) I(W ; Y ) + I(W, W ; Y Y ) I(W ; Y W ) () = I(W ; Y) + I(W ; Y, W ) I(W ; Y ) + I(W, W ; Y Y ) I(W ; Y W ) () = I(W ; Y) I(W ; Y ) + I(W, W ; Y Y ) I(W ; Y W ) (3) = I(W ; Y) I(W ; Y ) + I(W, W ; Y Y ) I(W ; Y, W ) (4) where () ad (4) come from the idepedece of W ad W, ad (3) comes from the idepedece of W ad (W, Y ). For the first part i (4), we have I(W ; Y) I(W ; Y ) I(W ; Y, Y ) I(W ; Y ) (5) = I(W ; Y Y ) (6) = I(W ; X Y ) (7) = H(X Y ) H(X Y, W ) (8) where we refer to (6). For the secod part i (4), we have I(W, W ; Y Y ) I(W ; Y, W ) = I(W ; Y Y ) + I(W ; Y Y, W ) I(W ; Y, W ) (9) = I(W ; Y Y ) + I(W ; Y, Y, W ) I(W ; Y, W ) (0) I(W ; Y Y ) + I(W ; Y, Y, Y, W ) I(W ; Y, W ) () = I(W ; Y Y ) + I(W ; Y, Y Y, W ) () I(X, X; Y Y ) + I(X ; Y, Y Y, W ) (3) = I(X; Y X ) + H(X Y, W ) (4) = I(X; Y X ) + H(X Y, W ) (5) where (0) follows from the idepedece of W ad (W, Y ), (3) follows from the Markov chais W (Y, X, X) Y W (X, Y, W ) (Y, Y), we obtai (4) by usig the chael model i (6) ad the fact that by kowig (Y, Y ) = (X, Y ), X ca be determied, ad fially, we reach (5) by usig the chael model i (6) ad through the followig derivatio H(X Y, W ) = H(X, Y, W ) H(Y, W ) (6) = H(X, X, W ) H(Y, W ) (7) = H(X, Y, W ) H(Y, W ) (8) = H(X Y, W ) (9) Substitutig (8) ad (5) ito (4), we obtai R Σ H(X Y ) + I(X ; Y X ) (30) = H(X X X ) + I(X ; X X X ) (3) where meas bitwise modulo plus. Now, ituitively, as show i (3), if trasmitter iteds to trasmit -bit message via X, the to protect it, trasmitter must sed Beroulli ( ) i.i.d radom oise; however, by performig that, the sub-chael capacity betwee X ad Y is costraied ad reduced to zero. To cofirm this, we cotiue from (3) R Σ H(X X X ) + I(X ; X X X ) (3) = H(X, X ) H(X X ) + H(X X X ) H(X X, X ) (33) = H(X ) + H(X ) H(X X ) + H(X X X ) H(X ) (34) = H(X ) H(X X ) + H(X X X ) (35) = H(X X ) H(X X ) + H(X X X ) (36) = H(X X X ) H(X X ) + H(X X X ) (37) = H(X X X) I(X ; X X ) (38) H(X X X) = H(Y X ) H(Y) (39) (40) where we repeatedly use the idepedece of X ad X, ad also the idepedece of X ad (X, X ). Fially, (40) implies C Σ,(a), cocludig, together with the achievability, that C Σ,(a) =, ad hece C Σ,idep =. B. Joit Ecodig Based Achievable Scheme Here, we provide a achievable scheme to trasmit 3 bits securely by codig across the compoet chaels, i.e., by itroducig correlatio betwee the chael iputs of compoet chaels. Let {A, B, C, U, V } be mutually idepedet Beroulli ( ) radom variables. Here, {A, B, C} represet the message carryig sigals, ad {U, V } represet the jammig sigals. The joit ecodig based achievable scheme is show

4 i color mageta i Fig., where trasmitter seds A, V ad A V i three compoet chaels, respectively (ote that we choose X = 0), ad trasmitter seds U, (B, C) ad B U i three compoet chaels, respectively. With this scheme, the legitimate receiver observes A, U, B, C V, A V B U from three compoet chaels, which meas that the legitimate receiver ca decode message A from trasmitter ad messages B, C from trasmitter with zero probability of error, i.e., the legitimate receiver ca decode 3 bits reliably. O the other had, the eavesdropper observes A U, B V ad A U B V, which implies I(A, B, C; A U, B V, A U B V ) = I(A, B, C; A U, B V ) (4) = H(A U, B V ) H(A U, B V A, B, C) (4) = H(A U, B V ) H(U, V ) (43) = = 0 (44) where we use the idepedece of {A, B, C, U, V } ad also that they are all Beroulli ( ). This derivatio implies that the eavesdropper lears othig about the messages, ad therefore, 3 bits are set to the legitimate receiver reliably ad securely. IV. GAUSSIAN MAC WIRETAP CHANNEL A. Geeral Iseparability I this sectio, we show that eve the parallel Gaussia MAC wiretap chael is ot separable i geeral. We prove this by providig a specific example. Also ote that, it suffices to show the iseparability from the s.d.o.f. poit of view, sice it implies the iseparability of the secrecy capacity. Cosider the special two-user parallel Gaussia MAC wiretap chael show i Fig., i which each compoet chael is a two-user Gaussia MAC wiretap chael defied by, Y k = h k X k + h k X k + N k (45) Y k = g k X k + g k X k + N k (46) where k = a, b, ad (h ia, h ib ) ad (g ia, g ib ) are the timeivariat chael gais of user i to the legitimate receiver ad the eavesdropper, respectively. We let h b = h b = α, ad g b = g b = β (47) The, the six radom variables {h a, h a, g a, g a, α, β} are mutually idepedetly distributed accordig to the same cotiuous distributio, ad N a, N a, N b, N b are mutually idepedet Gaussia radom variables with zero-mea ad uit-variace. The chael iputs of each user satisfy average power costraits, E Xia + ib X P, for i =,. From 7, for almost all chael gais {h a, h a, g a, g a }, the sum s.d.o.f. for compoet chael (a) is 3. From 8, compoet chael (b) is degraded, ad its sum s.d.o.f. is zero. This implies that, by idepedet ecodig across the compoet chaels, the optimum sum s.d.o.f. is 3. O the other had, by selectig X a = g a V, X a = g a U, X b = β V, X b = β U (48) Fig.. X a X a X b X b g a g b = β h a g a h b = α g b = β (a) (b) N a h a N a N b h b = α N b A example two-user parallel Gaussia MAC wiretap chael. where V ad U are idepedet radom variable draw from the followig discrete PAM costellatio: C(a, Q) = a{ Q, Q +,..., Q, Q} (49) Here, V represets the message-carryig sigal ad U represets the jammig sigal. Let us defie Ŷ as Ŷ = g a Y a β h a α Y b (50) ga h a = V + g a N a β g a h a h a α N b (5) The factor i frot of V is o-zero for almost all chael gais. Let us defie ˆV as the estimate of V obtaied by selectig the closest poit i C(a, Q) based o the observatio Ŷ. For ay small eough δ > 0, let us choose Q = P δ ad a = γp δ, where γ is a costat idepedet of P to meet the average power costrait. The, due to the Markov chai V (Y a, Y b ) Ŷ ˆV, we have I(V ; Y a, Y b ) I(V ; Ŷ ) I(V ; ˆV ) (5) = H(V ) H(V ˆV ) (53) = log(q + ) H(V ˆV ) (54) log(q + ) Pr V ˆV log(q + ) (55) { Pr V ˆV } δ log P (56) Now, due to the PAM structure, probability of error is Pr V ˆV exp ( γ a ) exp ( γ P δ) (57) where γ, γ are costats idepedet of P. The, from (56) ad (57), at high SNR (large eough P ), we have I(V ; Y a, Y b ) δ log P + o(log P ) (58) where o( ) is the little-o fuctio. Y a Y a Y b Y b

5 O the other had, for the iformatio leakage rate, I(V ; Y a, Y b ) I(V ; V + U) (59) H(V + U) H(V ) (60) log 4Q + Q + (6) By 3, Theorem, we ca achieve the sum secrecy rate of sup (R + R ) I(V ; Y a, Y b ) I(V ; Y a, Y b ) (6) δ log P + o(log P ) (63) for ay δ 0, which implies that we ca achieve sum s.d.o.f. This meas that by joit ecodig across compoet chaels, we achieve sum s.d.o.f. outperformig optimum idepedet ecodig, which ca at most achieve 3 sum s.d.o.f. B. Separability i s.d.o.f. for Almost All Chael Gais Although the Gaussia MAC wiretap chael is ot always separable, the special costructio provided i the last subsectio is ot geeral, i.e., for almost all chael gais, the costraits i (47) are ever met. Based o this observatio, we show that the s.d.o.f. regio of the parallel Gaussia MAC wiretap chael is separable for almost all chael gais. From 8, the s.d.o.f. regios of the compoet Gaussia MAC wiretap chaels are idetical, i.e., D s,(a) = D s,(b), ad D s,(a) = {(d, d ) : d + d, d + d } (64) Therefore, it suffices to show that for the overall parallel Gaussia MAC chael the s.d.o.f. regio is D s = {(d, d ) : d + d, d + d } (65) The achievability follows from 8 for almost all chael gais. I the achievability, we scale the power i each compoet chael, to meet the overall power costrait; however, this does ot affect the s.d.o.f. calculatios. For the coverse, we first flatte the parallel chael by cocateatig the chael iputs ad outputs of compoet chaels ito -legth vectors. Istead of studyig the parallel chael i chael uses, we study the flat chael i chael uses. The power costrait remais the same over chael uses. I additio, sice itroducig correlatio i time ad i compoet chaels has the same effect, the flat chael must have the same coverse as the origial oe. The, similar to the steps i 7, Eqs. (7)-(6), we have (R + R ) h( X, X, Y, Y ) h( X, X Y, Y ) h(y ) + c 0 (66) where vectors i bold-face are -legth vectors. The compoets of -vectors X j, for j =,, are X ji = X ji + Ñji, for i =,...,. Here, the sequece Ñ j is i.i.d. over time, is idepedet of all other radom variables, ad Ñji is a Gaussia radom variable with zero-mea ad variace σji, such that { } σji < mi,,, (67) h ja g ja h jb g jb The, all the remaiig steps i 7 follow, ad we have ( ) R i +R +R h(y )+c log P +c (68) for i =,. This implies d + d, ad d + d (69) which completes the proof of the coverse for this case. V. CONCLUSIONS We showed that the parallel MAC wiretap chael is ot always separable by providig a specific example i which the sum secrecy rate by joit ecodig over parallel chaels outperforms the best rate achievable by idividually optimal ecodig for each compoet chael. The, we showed that the parallel Gaussia MAC wiretap chael is iseparable i geeral as well. Fially, we showed, from a s.d.o.f. poit of view, that the parallel Gaussia MAC wiretap chael is separable almost surely, however, separability i s.d.o.f. is weaker tha separability i secrecy capacity. REFERENCES T. M. Cover ad J. A. Thomas. Elemets of Iformatio Theory. Wiley- Itersciece, secod editio, 006. D. Tse ad S. V. Haly. Multiaccess fadig chaels-part I: Polymatroid structure, optimal resource allocatio ad throughput capacities. IEEE Tras. If. Theory, 44(7):796 85, November D. Tse. Optimal power allocatio over parallel Gaussia broadcast chaels. I IEEE ISIT, Jue L. Sakar, X. Shag, E. Erkip, ad H. V. Poor. Ergodic two-user iterferece chaels: Is separability optimal? I Allerto Coferece, September V. R. Cadambe ad S. A. Jafar. Parallel Gaussia iterferece chaels are ot always separable. IEEE Tras. If. Theory, 55(9): , September H. Su, C. Geg, ad S. A. Jafar. Topological iterferece maagemet with alteratig coectivity. Available at arxiv: P. Mukherjee, R. Tado, ad S. Ulukus. Eve symmetric parallel liear determiistic iterferece chaels are iseparable. I Allerto Coferece, October E. Teki ad A. Yeer. The Gaussia multiple access wire-tap chael. IEEE Tras. If. Theory, 54(): , December E. Teki ad A. Yeer. The geeral Gaussia multiple-access ad twoway wiretap chaels: Achievable rates ad cooperative jammig. IEEE Tras. If. Theory, 54(6):735 75, Jue E. Ekrem ad S. Ulukus. O the secrecy of multiple access wiretap chael. I Allerto Coferece, September 008. E. Ekrem ad S. Ulukus. Cooperative secrecy i wireless commuicatios. Securig Wireless Commuicatios at the Physical Layer, W. Trappe ad R. Liu, Eds., Spriger-Verlag, 009. X. He ad A. Yeer. Providig secrecy with structured codes: Two-user Gaussia chaels. IEEE Tras. If. Theory, 60(4): 38, April G. Bagherikaram, A. S. Motahari, ad A. K. Khadai. O the secure degrees-of-freedom of the multiple-access-chael. IEEE Tras. If. Theory, submitted March 00. Also available at arxiv: R. Bassily ad S. Ulukus. Ergodic secret aligmet. IEEE Tras. If. Theory, 58(3):594 6, March 0. 5 N. Liu ad W. Kag. The secrecy capacity regio of a special class of multiple access chaels. I IEEE ISIT, July 0. 6 M. Wiese ad H. Boche. A achievable regio for the wiretap multipleaccess chael with commo message. I IEEE ISIT, July 0. 7 J. Xie ad S. Ulukus. Secure degrees of freedom of the Gaussia multiple access wiretap chael. I IEEE ISIT, July J. Xie ad S. Ulukus. Secure degrees of freedom regio of the Gaussia multiple access wiretap chael. I Asilomar Coferece, November 03.

Symmetric Two-User Gaussian Interference Channel with Common Messages

Symmetric Two-User Gaussian Interference Channel with Common Messages Symmetric Two-User Gaussia Iterferece Chael with Commo Messages Qua Geg CSL ad Dept. of ECE UIUC, IL 680 Email: geg5@illiois.edu Tie Liu Dept. of Electrical ad Computer Egieerig Texas A&M Uiversity, TX

More information

arxiv: v1 [cs.it] 13 Jul 2012

arxiv: v1 [cs.it] 13 Jul 2012 O the Sum Capacity of the Discrete Memoryless Iterferece Chael with Oe-Sided Weak Iterferece ad Mixed Iterferece Fagfag Zhu ad Biao Che Syracuse Uiversity Departmet of EECS Syracuse, NY 3244 Email: fazhu{biche}@syr.edu

More information

Lecture 11: Channel Coding Theorem: Converse Part

Lecture 11: Channel Coding Theorem: Converse Part EE376A/STATS376A Iformatio Theory Lecture - 02/3/208 Lecture : Chael Codig Theorem: Coverse Part Lecturer: Tsachy Weissma Scribe: Erdem Bıyık I this lecture, we will cotiue our discussio o chael codig

More information

Information Theory Tutorial Communication over Channels with memory. Chi Zhang Department of Electrical Engineering University of Notre Dame

Information Theory Tutorial Communication over Channels with memory. Chi Zhang Department of Electrical Engineering University of Notre Dame Iformatio Theory Tutorial Commuicatio over Chaels with memory Chi Zhag Departmet of Electrical Egieerig Uiversity of Notre Dame Abstract A geeral capacity formula C = sup I(; Y ), which is correct for

More information

Asymptotic Coupling and Its Applications in Information Theory

Asymptotic Coupling and Its Applications in Information Theory Asymptotic Couplig ad Its Applicatios i Iformatio Theory Vicet Y. F. Ta Joit Work with Lei Yu Departmet of Electrical ad Computer Egieerig, Departmet of Mathematics, Natioal Uiversity of Sigapore IMS-APRM

More information

Secure Degrees of Freedom of the MIMO Multiple Access Wiretap Channel

Secure Degrees of Freedom of the MIMO Multiple Access Wiretap Channel Secure Degrees of Freedom of the MIMO Multiple Access Wiretap Channel Pritam Mukherjee Sennur Ulukus Department of Electrical and Computer Engineering University of Maryland, College Park, MD 074 pritamm@umd.edu

More information

Lecture 7: Channel coding theorem for discrete-time continuous memoryless channel

Lecture 7: Channel coding theorem for discrete-time continuous memoryless channel Lecture 7: Chael codig theorem for discrete-time cotiuous memoryless chael Lectured by Dr. Saif K. Mohammed Scribed by Mirsad Čirkić Iformatio Theory for Wireless Commuicatio ITWC Sprig 202 Let us first

More information

A Partial Decode-Forward Scheme For A Network with N relays

A Partial Decode-Forward Scheme For A Network with N relays A Partial Decode-Forward Scheme For A etwork with relays Yao Tag ECE Departmet, McGill Uiversity Motreal, QC, Caada Email: yaotag2@mailmcgillca Mai Vu ECE Departmet, Tufts Uiversity Medford, MA, USA Email:

More information

Lecture 6: Source coding, Typicality, and Noisy channels and capacity

Lecture 6: Source coding, Typicality, and Noisy channels and capacity 15-859: Iformatio Theory ad Applicatios i TCS CMU: Sprig 2013 Lecture 6: Source codig, Typicality, ad Noisy chaels ad capacity Jauary 31, 2013 Lecturer: Mahdi Cheraghchi Scribe: Togbo Huag 1 Recap Uiversal

More information

On the Capacity of Symmetric Gaussian Interference Channels with Feedback

On the Capacity of Symmetric Gaussian Interference Channels with Feedback O the Capacity of Symmetric Gaussia Iterferece Chaels with Feedback La V Truog Iformatio Techology Specializatio Departmet ITS FPT Uiversity, Haoi, Vietam E-mail: latv@fpteduv Hirosuke Yamamoto Dept of

More information

Hybrid Coding for Gaussian Broadcast Channels with Gaussian Sources

Hybrid Coding for Gaussian Broadcast Channels with Gaussian Sources Hybrid Codig for Gaussia Broadcast Chaels with Gaussia Sources Rajiv Soudararaja Departmet of Electrical & Computer Egieerig Uiversity of Texas at Austi Austi, TX 7871, USA Email: rajivs@mailutexasedu

More information

Cooperative Communication Fundamentals & Coding Techniques

Cooperative Communication Fundamentals & Coding Techniques 3 th ICACT Tutorial Cooperative commuicatio fudametals & codig techiques Cooperative Commuicatio Fudametals & Codig Techiques 0..4 Electroics ad Telecommuicatio Research Istitute Kiug Jug 3 th ICACT Tutorial

More information

Entropies & Information Theory

Entropies & Information Theory Etropies & Iformatio Theory LECTURE I Nilajaa Datta Uiversity of Cambridge,U.K. For more details: see lecture otes (Lecture 1- Lecture 5) o http://www.qi.damtp.cam.ac.uk/ode/223 Quatum Iformatio Theory

More information

Lecture 14: Graph Entropy

Lecture 14: Graph Entropy 15-859: Iformatio Theory ad Applicatios i TCS Sprig 2013 Lecture 14: Graph Etropy March 19, 2013 Lecturer: Mahdi Cheraghchi Scribe: Euiwoog Lee 1 Recap Bergma s boud o the permaet Shearer s Lemma Number

More information

Lecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound

Lecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound Lecture 7 Ageda for the lecture Gaussia chael with average power costraits Capacity of additive Gaussia oise chael ad the sphere packig boud 7. Additive Gaussia oise chael Up to this poit, we have bee

More information

Lecture 19: Convergence

Lecture 19: Convergence Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may

More information

STAT Homework 1 - Solutions

STAT Homework 1 - Solutions STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better

More information

Lecture 15: Strong, Conditional, & Joint Typicality

Lecture 15: Strong, Conditional, & Joint Typicality EE376A/STATS376A Iformatio Theory Lecture 15-02/27/2018 Lecture 15: Strog, Coditioal, & Joit Typicality Lecturer: Tsachy Weissma Scribe: Nimit Sohoi, William McCloskey, Halwest Mohammad I this lecture,

More information

A New Achievability Scheme for the Relay Channel

A New Achievability Scheme for the Relay Channel A New Achievability Scheme for the Relay Chael Wei Kag Seur Ulukus Departmet of Electrical ad Computer Egieerig Uiversity of Marylad, College Park, MD 20742 wkag@umd.edu ulukus@umd.edu October 4, 2007

More information

SUCCESSIVE INTERFERENCE CANCELLATION DECODING FOR THE K -USER CYCLIC INTERFERENCE CHANNEL

SUCCESSIVE INTERFERENCE CANCELLATION DECODING FOR THE K -USER CYCLIC INTERFERENCE CHANNEL Joural of Theoretical ad Applied Iformatio Techology 31 st December 212 Vol 46 No2 25-212 JATIT & LLS All rights reserved ISSN: 1992-8645 wwwatitorg E-ISSN: 1817-3195 SCCESSIVE INTERFERENCE CANCELLATION

More information

Information Theory and Coding

Information Theory and Coding Sol. Iformatio Theory ad Codig. The capacity of a bad-limited additive white Gaussia (AWGN) chael is give by C = Wlog 2 ( + σ 2 W ) bits per secod(bps), where W is the chael badwidth, is the average power

More information

Properties and Hypothesis Testing

Properties and Hypothesis Testing Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.

More information

Problem Set 4 Due Oct, 12

Problem Set 4 Due Oct, 12 EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios

More information

THE interference channel problem describes a setup where multiple pairs of transmitters and receivers share a communication

THE interference channel problem describes a setup where multiple pairs of transmitters and receivers share a communication Trade-off betwee Commuicatio ad Cooperatio i the Iterferece Chael Farhad Shirai EECS Departmet Uiversity of Michiga A Arbor,USA Email: fshirai@umich.edu S. Sadeep Pradha EECS Departmet Uiversity of Michiga

More information

Channel coding, linear block codes, Hamming and cyclic codes Lecture - 8

Channel coding, linear block codes, Hamming and cyclic codes Lecture - 8 Digital Commuicatio Chael codig, liear block codes, Hammig ad cyclic codes Lecture - 8 Ir. Muhamad Asial, MSc., PhD Ceter for Iformatio ad Commuicatio Egieerig Research (CICER) Electrical Egieerig Departmet

More information

Lecture 7: MIMO Architectures Theoretical Foundations of Wireless Communications 1. Overview. CommTh/EES/KTH

Lecture 7: MIMO Architectures Theoretical Foundations of Wireless Communications 1. Overview. CommTh/EES/KTH : Theoretical Foudatios of Wireless Commuicatios 1 Thursday, May 19, 2016 12:30-15:30, Coferece Room SIP 1 Textbook: D. Tse ad P. Viswaath, Fudametals of Wireless Commuicatio 1 / 1 Overview Lecture 6:

More information

Overview of Gaussian MIMO (Vector) BC

Overview of Gaussian MIMO (Vector) BC Overview of Gaussia MIMO (Vector) BC Gwamo Ku Adaptive Sigal Processig ad Iformatio Theory Research Group Nov. 30, 2012 Outlie / Capacity Regio of Gaussia MIMO BC System Structure Kow Capacity Regios -

More information

Entropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP

Entropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP Etropy ad Ergodic Theory Lecture 5: Joit typicality ad coditioal AEP 1 Notatio: from RVs back to distributios Let (Ω, F, P) be a probability space, ad let X ad Y be A- ad B-valued discrete RVs, respectively.

More information

Universal source coding for complementary delivery

Universal source coding for complementary delivery SITA2006 i Hakodate 2005.2. p. Uiversal source codig for complemetary delivery Akisato Kimura, 2, Tomohiko Uyematsu 2, Shigeaki Kuzuoka 2 Media Iformatio Laboratory, NTT Commuicatio Sciece Laboratories,

More information

Lecture 7: October 18, 2017

Lecture 7: October 18, 2017 Iformatio ad Codig Theory Autum 207 Lecturer: Madhur Tulsiai Lecture 7: October 8, 207 Biary hypothesis testig I this lecture, we apply the tools developed i the past few lectures to uderstad the problem

More information

On Random Line Segments in the Unit Square

On Random Line Segments in the Unit Square O Radom Lie Segmets i the Uit Square Thomas A. Courtade Departmet of Electrical Egieerig Uiversity of Califoria Los Ageles, Califoria 90095 Email: tacourta@ee.ucla.edu I. INTRODUCTION Let Q = [0, 1] [0,

More information

The Gaussian Multiple Access Wire-Tap Channel with Collective Secrecy Constraints

The Gaussian Multiple Access Wire-Tap Channel with Collective Secrecy Constraints IIT 6, eattle, UA, July 9-4, 6 The Gaussia Multiple Access Wire-Tap Chael with Collective ecrecy Costraits Eder Tei tei@psu.edu Ayli Yeer yeer@ee.psu.edu Wireless Commuicatios ad Networig Laboratory Electrical

More information

The Capacity Region of the Degraded Finite-State Broadcast Channel

The Capacity Region of the Degraded Finite-State Broadcast Channel The Capacity Regio of the Degraded Fiite-State Broadcast Chael Ro Dabora ad Adrea Goldsmith Dept. of Electrical Egieerig, Staford Uiversity, Staford, CA Abstract We cosider the discrete, time-varyig broadcast

More information

Increasing timing capacity using packet coloring

Increasing timing capacity using packet coloring 003 Coferece o Iformatio Scieces ad Systems, The Johs Hopkis Uiversity, March 4, 003 Icreasig timig capacity usig packet colorig Xi Liu ad R Srikat[] Coordiated Sciece Laboratory Uiversity of Illiois e-mail:

More information

Lecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables

Lecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze

More information

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1. Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio

More information

IET Commun., 2009, Vol. 3, Iss. 1, pp doi: /iet-com: & The Institution of Engineering and Technology 2009

IET Commun., 2009, Vol. 3, Iss. 1, pp doi: /iet-com: & The Institution of Engineering and Technology 2009 Published i IET Commuicatios Received o 4th Jue 28 Revised o 3th July 28 doi: 1.149/iet-com:28373 ISSN 1751-8628 Symmetric relayig based o partial decodig ad the capacity of a class of relay etworks L.

More information

Multiterminal source coding with complementary delivery

Multiterminal source coding with complementary delivery Iteratioal Symposium o Iformatio Theory ad its Applicatios, ISITA2006 Seoul, Korea, October 29 November 1, 2006 Multitermial source codig with complemetary delivery Akisato Kimura ad Tomohiko Uyematsu

More information

The Maximum-Likelihood Decoding Performance of Error-Correcting Codes

The Maximum-Likelihood Decoding Performance of Error-Correcting Codes The Maximum-Lielihood Decodig Performace of Error-Correctig Codes Hery D. Pfister ECE Departmet Texas A&M Uiversity August 27th, 2007 (rev. 0) November 2st, 203 (rev. ) Performace of Codes. Notatio X,

More information

Lecture 7: Properties of Random Samples

Lecture 7: Properties of Random Samples Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ

More information

Sequences and Series of Functions

Sequences and Series of Functions Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges

More information

Inequalities for Entropies of Sets of Subsets of Random Variables

Inequalities for Entropies of Sets of Subsets of Random Variables Iequalities for Etropies of Sets of Subsets of Radom Variables Chao Tia AT&T Labs-Research Florham Par, NJ 0792, USA. tia@research.att.com Abstract Ha s iequality o the etropy rates of subsets of radom

More information

The Fading Number of Multiple-Input Multiple-Output Fading Channels with Memory

The Fading Number of Multiple-Input Multiple-Output Fading Channels with Memory The Fadig Number of Multiple-Iput Multiple-Output Fadig Chaels with Memory Stefa M. Moser Departmet of Commuicatio Egieerig Natioal Chiao Tug Uiversity NCTU Hsichu, Taiwa Email: stefa.moser@ieee.org Abstract

More information

Non-Asymptotic Achievable Rates for Gaussian Energy-Harvesting Channels: Best-Effort and Save-and-Transmit

Non-Asymptotic Achievable Rates for Gaussian Energy-Harvesting Channels: Best-Effort and Save-and-Transmit 08 IEEE Iteratioal Symposium o Iformatio Theory ISIT No-Asymptotic Achievable Rates for Gaussia Eergy-Harvestig Chaels: Best-Effort ad Save-ad-Trasmit Silas L Fog Departmet of Electrical ad Computer Egieerig

More information

A Rank Ratio Inequality and Its Applications

A Rank Ratio Inequality and Its Applications A Rak Ratio Iequality ad Its Applicatios Salma Avestimehr I collaboratio with Sia Lashgari (Corell) ad Chagho Suh (KAIST) Allerto Coferece October 2013 Motivatio If the chaels are -me- varyig ad the trasmi5ers

More information

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1 EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum

More information

The Binary Energy Harvesting Channel with On-Off Fading

The Binary Energy Harvesting Channel with On-Off Fading The Biary Eergy Harvestig Chael with O-Off Fadig Omur Ozel, Kaya Tutucuoglu 2, Seur Ulukus, ad Ayli Yeer 2 Deartmet of Electrical ad Comuter Egieerig, Uiversity of Marylad, College Park, MD 20742 2 Deartmet

More information

Communicating under Temperature and Energy Harvesting Constraints

Communicating under Temperature and Energy Harvesting Constraints Commuicatig uder Temperature ad Eergy Harvestig Costraits Omur Ozel, Seur Ulukus 2, ad Pulkit Grover Departmet of Electrical ad Computer Egieerig, Caregie Mello Uiversity, Pittsburgh, PA 2 Departmet of

More information

ECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002

ECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002 ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom

More information

6.3 Testing Series With Positive Terms

6.3 Testing Series With Positive Terms 6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial

More information

On Routing-Optimal Network for Multiple Unicasts

On Routing-Optimal Network for Multiple Unicasts O Routig-Optimal Network for Multiple Uicasts Chu Meg, Athia Markopoulou Abstract I this paper, we cosider etworks with multiple uicast sessios. Geerally, o-liear etwork codig is eeded to achieve the whole

More information

Non-Asymptotic Achievable Rates for Gaussian Energy-Harvesting Channels: Best-Effort and Save-and-Transmit

Non-Asymptotic Achievable Rates for Gaussian Energy-Harvesting Channels: Best-Effort and Save-and-Transmit No-Asymptotic Achievable Rates for Gaussia Eergy-Harvestig Chaels: Best-Effort ad Save-ad-Trasmit Silas L. Fog, Jig Yag, ad Ayli Yeer arxiv:805.089v [cs.it] 30 May 08 Abstract A additive white Gaussia

More information

Quick Review of Probability

Quick Review of Probability Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter & Teachig Material.

More information

1 Convergence in Probability and the Weak Law of Large Numbers

1 Convergence in Probability and the Weak Law of Large Numbers 36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec

More information

Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara

Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio

More information

A Non-Asymptotic Achievable Rate for the AWGN Energy-Harvesting Channel using Save-and-Transmit

A Non-Asymptotic Achievable Rate for the AWGN Energy-Harvesting Channel using Save-and-Transmit 016 IEEE Iteratioal Symposium o Iformatio Theory A No-Asymptotic Achievable Rate for the AWGN Eergy-Harvestig Chael usig Save-ad-Trasmit Silas L. Fog, Vicet Y. F. Ta, ad Jig Yag Departmet of Electrical

More information

Sequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence

Sequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece 1, 1, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet

More information

Chapter 6 Sampling Distributions

Chapter 6 Sampling Distributions Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to

More information

Multiterminal Source Coding with an Entropy-Based Distortion Measure

Multiterminal Source Coding with an Entropy-Based Distortion Measure 20 IEEE Iteratioal Symposium o Iformatio Theory Proceedigs Multitermial Source Codig with a Etropy-Based Distortio Measure Thomas A. Courtade ad Richard D. Wesel Departmet of Electrical Egieerig Uiversity

More information

Quick Review of Probability

Quick Review of Probability Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter 2 & Teachig

More information

Reliability and Queueing

Reliability and Queueing Copyright 999 Uiversity of Califoria Reliability ad Queueig by David G. Messerschmitt Supplemetary sectio for Uderstadig Networked Applicatios: A First Course, Morga Kaufma, 999. Copyright otice: Permissio

More information

Lecture 20: Multivariate convergence and the Central Limit Theorem

Lecture 20: Multivariate convergence and the Central Limit Theorem Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece

More information

Information-based Feature Selection

Information-based Feature Selection Iformatio-based Feature Selectio Farza Faria, Abbas Kazeroui, Afshi Babveyh Email: {faria,abbask,afshib}@staford.edu 1 Itroductio Feature selectio is a topic of great iterest i applicatios dealig with

More information

IN many applications, the average delay packets experience. Delay-Minimal Transmission for Average Power Constrained Multi-Access Communications

IN many applications, the average delay packets experience. Delay-Minimal Transmission for Average Power Constrained Multi-Access Communications 754 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 9, NO. 9, SEPTEMBER 010 Delay-Miimal Trasmissio for Average Power Costraied Multi-Access Commuicatios Jig Yag, Studet Member, IEEE, ad Seur Ulukus,

More information

Lecture Chapter 6: Convergence of Random Sequences

Lecture Chapter 6: Convergence of Random Sequences ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite

More information

Probability and statistics: basic terms

Probability and statistics: basic terms Probability ad statistics: basic terms M. Veeraraghava August 203 A radom variable is a rule that assigs a umerical value to each possible outcome of a experimet. Outcomes of a experimet form the sample

More information

First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise

First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >

More information

Capacity Theorems for the Finite-State Broadcast Channel with Feedback

Capacity Theorems for the Finite-State Broadcast Channel with Feedback Capacity Theorems for the Fiite-State Broadcast Chael with Feedback Ro abora ad Adrea Goldsmith ept of Electrical Egieerig, Staford Uiversity Abstract We cosider the discrete, time-varyig broadcast chael

More information

REGRESSION WITH QUADRATIC LOSS

REGRESSION WITH QUADRATIC LOSS REGRESSION WITH QUADRATIC LOSS MAXIM RAGINSKY Regressio with quadratic loss is aother basic problem studied i statistical learig theory. We have a radom couple Z = X, Y ), where, as before, X is a R d

More information

Estimation for Complete Data

Estimation for Complete Data Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of

More information

Chapter 6 Principles of Data Reduction

Chapter 6 Principles of Data Reduction Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a

More information

Section 14. Simple linear regression.

Section 14. Simple linear regression. Sectio 14 Simple liear regressio. Let us look at the cigarette dataset from [1] (available to dowload from joural s website) ad []. The cigarette dataset cotais measuremets of tar, icotie, weight ad carbo

More information

Keyless Authentication and Authenticated Capacity

Keyless Authentication and Authenticated Capacity Keyless Autheticatio ad Autheticated Capacity Wewe Tu ad Lifeg Lai Abstract We cosider the problem of keyless message autheticatio over oisy chaels i the presece of a active adversary. Differet from the

More information

Confidence interval for the two-parameter exponentiated Gumbel distribution based on record values

Confidence interval for the two-parameter exponentiated Gumbel distribution based on record values Iteratioal Joural of Applied Operatioal Research Vol. 4 No. 1 pp. 61-68 Witer 2014 Joural homepage: www.ijorlu.ir Cofidece iterval for the two-parameter expoetiated Gumbel distributio based o record values

More information

Lecture 2. The Lovász Local Lemma

Lecture 2. The Lovász Local Lemma Staford Uiversity Sprig 208 Math 233A: No-costructive methods i combiatorics Istructor: Ja Vodrák Lecture date: Jauary 0, 208 Origial scribe: Apoorva Khare Lecture 2. The Lovász Local Lemma 2. Itroductio

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties

More information

Journal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula

Journal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula Joural of Multivariate Aalysis 102 (2011) 1315 1319 Cotets lists available at ScieceDirect Joural of Multivariate Aalysis joural homepage: www.elsevier.com/locate/jmva Superefficiet estimatio of the margials

More information

Unbiased Estimation. February 7-12, 2008

Unbiased Estimation. February 7-12, 2008 Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom

More information

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i

More information

NICK DUFRESNE. 1 1 p(x). To determine some formulas for the generating function of the Schröder numbers, r(x) = a(x) =

NICK DUFRESNE. 1 1 p(x). To determine some formulas for the generating function of the Schröder numbers, r(x) = a(x) = AN INTRODUCTION TO SCHRÖDER AND UNKNOWN NUMBERS NICK DUFRESNE Abstract. I this article we will itroduce two types of lattice paths, Schröder paths ad Ukow paths. We will examie differet properties of each,

More information

Problem Set 2 Solutions

Problem Set 2 Solutions CS271 Radomess & Computatio, Sprig 2018 Problem Set 2 Solutios Poit totals are i the margi; the maximum total umber of poits was 52. 1. Probabilistic method for domiatig sets 6pts Pick a radom subset S

More information

Lecture #20. n ( x p i )1/p = max

Lecture #20. n ( x p i )1/p = max COMPSCI 632: Approximatio Algorithms November 8, 2017 Lecturer: Debmalya Paigrahi Lecture #20 Scribe: Yua Deg 1 Overview Today, we cotiue to discuss about metric embeddigs techique. Specifically, we apply

More information

Fig. 2. Block Diagram of a DCS

Fig. 2. Block Diagram of a DCS Iformatio source Optioal Essetial From other sources Spread code ge. Format A/D Source ecode Ecrypt Auth. Chael ecode Pulse modu. Multiplex Badpass modu. Spread spectrum modu. X M m i Digital iput Digital

More information

Lecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting

Lecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would

More information

The Likelihood Encoder with Applications to Lossy Compression and Secrecy

The Likelihood Encoder with Applications to Lossy Compression and Secrecy The Likelihood Ecoder with Applicatios to Lossy Compressio ad Secrecy Eva C. Sog Paul Cuff H. Vicet Poor Dept. of Electrical Eg., Priceto Uiversity, NJ 8544 {csog, cuff, poor}@priceto.edu Abstract A likelihood

More information

Binary Fading Interference Channel with No CSIT

Binary Fading Interference Channel with No CSIT Biary Fadig Iterferece Chael with No CSIT Alireza Vahid, Mohammad Ali Maddah-Ali, A. Salma Avestimehr, ad Ya Zhu arxiv:405.003v3 [cs.it] 4 Mar 07 Abstract We study the capacity regio of the two-user Biary

More information

The Capacity Per Unit Energy of Large Wireless Networks

The Capacity Per Unit Energy of Large Wireless Networks The Capacity Per Uit Eergy of Large Wireless Networks Sudeep Kamath EECS Departmet, UC Berkeley Berkeley, CA 9470 sudeep@eecs.berkeley.edu Urs Niese Bell Laboratories, Alcatel-Lucet Murray Hill, NJ 07974

More information

It is always the case that unions, intersections, complements, and set differences are preserved by the inverse image of a function.

It is always the case that unions, intersections, complements, and set differences are preserved by the inverse image of a function. MATH 532 Measurable Fuctios Dr. Neal, WKU Throughout, let ( X, F, µ) be a measure space ad let (!, F, P ) deote the special case of a probability space. We shall ow begi to study real-valued fuctios defied

More information

Finite Block-Length Gains in Distributed Source Coding

Finite Block-Length Gains in Distributed Source Coding Decoder Fiite Block-Legth Gais i Distributed Source Codig Farhad Shirai EECS Departmet Uiversity of Michiga A Arbor,USA Email: fshirai@umichedu S Sadeep Pradha EECS Departmet Uiversity of Michiga A Arbor,USA

More information

Convergence of random variables. (telegram style notes) P.J.C. Spreij

Convergence of random variables. (telegram style notes) P.J.C. Spreij Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space

More information

ACO Comprehensive Exam 9 October 2007 Student code A. 1. Graph Theory

ACO Comprehensive Exam 9 October 2007 Student code A. 1. Graph Theory 1. Graph Theory Prove that there exist o simple plaar triagulatio T ad two distict adjacet vertices x, y V (T ) such that x ad y are the oly vertices of T of odd degree. Do ot use the Four-Color Theorem.

More information

Rademacher Complexity

Rademacher Complexity EECS 598: Statistical Learig Theory, Witer 204 Topic 0 Rademacher Complexity Lecturer: Clayto Scott Scribe: Ya Deg, Kevi Moo Disclaimer: These otes have ot bee subjected to the usual scrutiy reserved for

More information

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

More information

CEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering

CEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio

More information

Element sampling: Part 2

Element sampling: Part 2 Chapter 4 Elemet samplig: Part 2 4.1 Itroductio We ow cosider uequal probability samplig desigs which is very popular i practice. I the uequal probability samplig, we ca improve the efficiecy of the resultig

More information

Entropy Rates and Asymptotic Equipartition

Entropy Rates and Asymptotic Equipartition Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece

More information

Lecture 2: Concentration Bounds

Lecture 2: Concentration Bounds CSE 52: Desig ad Aalysis of Algorithms I Sprig 206 Lecture 2: Cocetratio Bouds Lecturer: Shaya Oveis Ghara March 30th Scribe: Syuzaa Sargsya Disclaimer: These otes have ot bee subjected to the usual scrutiy

More information

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

More information

11 Correlation and Regression

11 Correlation and Regression 11 Correlatio Regressio 11.1 Multivariate Data Ofte we look at data where several variables are recorded for the same idividuals or samplig uits. For example, at a coastal weather statio, we might record

More information