Information Theory. Lecture 10. Network Information Theory (CT15); a focus on channel capacity results
|
|
- Megan Hamilton
- 6 years ago
- Views:
Transcription
1 Information Theory Lecture 10 Network Information Theory (CT15); a focus on channel capacity results The (two-user) multiple access channel (15.3) The (two-user) broadcast channel (15.6) The relay channel (15.7) Some remarks on general multiterminal channels (15.10) Mikael Skoglund, Information Theory 1/25 Joint Typicality Extension of previous results to an arbitrary number of variables (most basic defs here, many additional results in CT) Notation For any k-tuple x k 1 = (x 1, x 2,..., x k ) X 1 X 2 X k and subset of indices S {1, 2,..., k} let x S = (x i ) i S denote the restriction of x k 1 to components with indices in S. Assume x i Xi n, any i, and let x S be a matrix with x i as rows for i S. Let the S -tuple x S,j be the jth column of x S.. As in CT, a n = 2 n(c±ε) means 1 n log a n c < ε, for all sufficiently large n Mikael Skoglund, Information Theory 2/25
2 For random variables X1 k with joint distribution p(xk 1 ): Generate X S via n independent copies of x S,j, j = 1,..., n. Then, n Pr(X S = x S ) = p(x S,j ) p(x S ) j=1 For S {1, 2,..., k}, define the set of ε-typical n-sequences { A (n) ε (S) = x S : Pr(X S = x S ) =. } 2 n[h(x S )±ε], S S Then, for any ε > 0, sufficiently large n, and S {1,..., k}, P (A (n) ε (S)) 1 ε p(x S ). = 2 n[h(x S)±ε] A (n) ε (S) =. 2 n[h(x S)±2ε] if x S A (n) ε (S) Mikael Skoglund, Information Theory 3/25 The Multiple Access Channel encoder 1 W 1 α 1 ( ) encoder 2 W 2 α 2 ( ) X 1 X 2 channel p(y x 1, x 2 ) Y decoder β( ) Ŵ 1 Ŵ 2 Two users communicating over a common channel. (The generalization to more than two is straightforward.) Mikael Skoglund, Information Theory 4/25
3 Coding: Memoryless pmf (or pdf): p(y x 1, x 2 ), x 1 X 1, x 2 X 2, y Y Data: W 1 I 1 = {1,..., M 1 } and W 2 I 2 = {1,..., M 2 } Assume W 1 and W 2 uniformly distributed and independent Encoders: α 1 : I 1 X n 1 and α 2 : I 2 X n 2 Rates: R 1 = 1 n log M 1 and R 2 = 1 n log M 2 Decoder: β : Y n I 1 I 2, β(y n ) = (Ŵ1, Ŵ2) Error probability: P e (n) = Pr ( (Ŵ1, Ŵ2) (W 1, W 2 ) ) Mikael Skoglund, Information Theory 5/25 Capacity: We have two (or more) rates, R 1 and R 2 = cannot consider one maximum achievable rate = study sets of achievable rate-pairs (R 1, R 2 ) = trade-off between R 1 and R 2 Achievable rate-pair: (R 1, R 2 ) is achievable if (α 1, α 2, β) exists such that P e (n) 0 as n Capacity region: The closure of the set of all achievable rate-pairs (R 1, R 2 ) Mikael Skoglund, Information Theory 6/25
4 Capacity Region for the Multiple Access Channel Fix π(x 1, x 2 ) = p 1 (x 1 )p 2 (x 2 ) on X 1 and X 2. Draw { X n 1 (i) : i I 1} and { X n 2 (j) : j I 2 } in an i.i.d. manner according to p 1 and p 2. Symmetry of codebook generation = P (n) e = Pr ( (Ŵ1, Ŵ2) (W 1, W 2 ) ) = Pr { (Ŵ1, Ŵ2) (1, 1) (W 1, W 2 ) = (1, 1) } where the second Pr is with respect to the channel and the random codebook design. Mikael Skoglund, Information Theory 7/25 Also Pr ( (Ŵ1, Ŵ2) (1, 1) ) = Pr(Ŵ1 1, Ŵ2 1) + Pr(Ŵ1 1, Ŵ2 = 1) + Pr(Ŵ1 = 1, Ŵ2 1) = P (n) 12 + P (n) 1 + P (n) 2 conditioned that (W 1, W 2 ) = (1, 1) everywhere. Joint typicality decoding, declare (Ŵ1, Ŵ2) = (1, 1) if only for i = j = 1 ( X n 1 (i), X n 2 (j), Y n) A (n) ε P (n) 12 2n[R 1+R 2 I(X 1,X 2 ;Y )+4ε] P (n) 1 2 n[r 1 I(X 1 ;Y X 2 )+3ε] P (n) 2 2 n[r 2 I(X 2 ;Y X 1 )+3ε] Mikael Skoglund, Information Theory 8/25
5 I(X 2 ; Y X 1 ) R 2 A I(X 2 ; Y ) B R 1 I(X 1 ; Y ) I(X 1 ; Y X 2 ) Hence, for a fixed π(x 1, x 2 ) = p 1 (x 1 )p 2 (x 2 ) the capacity region contains at least all pairs (R 1, R 2 ) in the set Π defined by R 1 < I(X 1 ; Y X 2 ) R 2 < I(X 2 ; Y X 1 ) R 1 + R 2 < I(X 1, X 2 ; Y ) Mikael Skoglund, Information Theory 9/25 The corner points Consider the point A R 1 = I(X 1 ; Y ) R 2 = I(X 2 ; Y X 1 ) } R 1 + R 2 = I(X 1, X 2 ; Y ) User 1 ignores the presence of user 2 R 1 = I(X 1 ; Y ) Decode user 1 s codeword User 2 sees an equivalent channel with input X2 n and output (Y n, X1 n ) R 2 = I(X 2 ; Y, X 1 ) = I(X 2 ; Y X 1 ) + I(X 1 ; X 2 ) = I(X 2 ; Y X 1 ) The above can be repeated with 1 2 and A B Points on the line A B can be achieved by time sharing Mikael Skoglund, Information Theory 10/25
6 Each particular choice of distribution π gives an achievable region Π; for two different π s, R 2 π 1 π 2 R 1 Fixed π = Π is convex. Varying π = Π can be non-convex. However all rates on a line connecting two achievable rate-pairs are achievable by time-sharing. Mikael Skoglund, Information Theory 11/25 The capacity region for the multiple access channel is the closure of the convex hull of the set of points defined by the three inequalities R 1 < I(X 1 ; Y X 2 ) R 2 < I(X 2 ; Y X 1 ) R 1 + R 2 < I(X 1, X 2 ; Y ) over all possible product distributions p 1 (x 1 )p 2 (x 2 ) for (X 1, X 2 ). Mikael Skoglund, Information Theory 12/25
7 Example: A Gaussian Channel Bandlimited AWGN channel with two additive users Y (t) = X 1 (t) + X 2 (t) + Z(t). The noise Z(t) is zero-mean Gaussian with power spectral density N 0 /2, and X 1 (t) and X 2 (t) are subject to the power constraints P 1 and P 2, respectively. The available bandwidth is W. The capacity of the corresponding single-user channel (with power constraint P ) is ( ) P W C [bits/second] where C(x) = log(1 + x). Mikael Skoglund, Information Theory 13/25 Time-Division Multiple-Access (TDMA): Let user 1 use all of the bandwidth with power P 1 /α a fraction α [0, 1] of time, and let user 2 use all the bandwidth with power P 2 /(1 α) the remaining fraction 1 α of time. The achievable rates then are ( ) ( ) P1 /α P2 /(1 α) R 1 < W α C R 2 < W (1 α) C Frequency-Division Multiple-Access (FDMA): Let user 1 transmit with power P 1 using a fraction α of the available bandwidth W, and let user two transmit with power P 2 the remaining fraction (1 α)w. The achievable rates are ( ) P1 R 1 < αw C α ( P 1 R 2 < (1 α)w C (1 α) TDMA and FDMA are equivalent from a capacity perspective! ) Mikael Skoglund, Information Theory 14/25
8 Code-Division Multiple-Access (CDMA): Defined, in our context, as all schemes that can be implemented to achieve the rates in the true capacity region R 1 R 2 R 1 + R 2 ( ) ( P1 W C = W log 1 + P ) 1 ( ) ( P2 W C = W log 1 + P ) 2 ( ) ( P1 + P 2 W C = W log 1 + P ) 1 + P 2 Mikael Skoglund, Information Theory 15/25 I R 2 T/FDMA CDMA R 2 T/FDMA CDMA I I R 1 2I R 1 Capacity region for P 1 = P 2 Capacity region for P 1 = 2P 2 Note that T/FDMA is only optimal when α 1 α = P 1 P 2. Mikael Skoglund, Information Theory 16/25
9 The Broadcast Channel (W 0, W 1, W 2 ) encoder α( ) X channel p(y 1, y 2 x) Y 1 Y 2 decoder 1 β 1 ( ) decoder 2 β 2 ( ) (Ŵ0, Ŵ1) (Ŵ0, Ŵ2) One transmitter, several receivers Message W 0 is a public message for both receivers, whereas W 1 and W 2 are private messages Mikael Skoglund, Information Theory 17/25 The Degraded Broadcast Channel Y 1 X p(y1, y 2 x) Y 2 X p(y 1 x) Y 1 p(y 2 y 1 ) Y 2 A broadcast channel is degraded if it can be split as in the figure. That is, Y 2 is a noisier version of X than Y 1, p(y 1, y 2 x) = p(y 2 y 1 )p(y 1 x). The Gaussian and the binary symmetric broadcast channels are degraded (see the examples in CT). Mikael Skoglund, Information Theory 18/25
10 Superposition Coding for the Degraded Broadcast Channel W 1 W 2 Codebook Y 1 -receiver Y 2 -receiver Assume there is no common information (for simplicity). Let W 2 chose a cloud of possible W 1 -codewords. The Y 1 -receiver sees all codewords, whereas the Y 2 -receiver is only able to distinguish between clouds. Mikael Skoglund, Information Theory 19/25 The capacity region of the degraded broadcast channel (with no common information), is the closure of the convex hull of all rates satisfying R 2 < I(U; Y 2 ) R 1 < I(X; Y 1 U) for some distribution p 1 (u)p 2 (x u). Proof: Choose W 2 -codewords i.i.d. according to p 1 (u), and for each one, choose W 1 -codewords i.i.d. according to p 2 (x u). The overall channel from W 2 to Y 2 (the clouds) can be made error-free as long as R 2 < I(U; Y 2 ), and conditioned on W 2 the channel from W 1 to Y 1 can be made error-free as long as R 1 < I(X; Y 1 U). Converse proved in Problem (based on Fano, as usual). Mikael Skoglund, Information Theory 20/25
11 The capacity region of the degraded broadcast channel (with common information): If the pair (R 1 = a, R 2 = b) is achievable for independent messages, as before, the triple (R 0, R 1 = a, R 2 = b R 0 ) is achievable with common information at rate R 0 (as long as R 0 < b). Since the better receiver can decode both W 1 and W 2, part of the W 2 -message can be made to include common information! Mikael Skoglund, Information Theory 21/25 The Relay Channel relay encoder { fi (Y i 1 1,1 )} n i=1 W encoder α( ) X Y 1 p(y, y 1 x, x 1 ) X 1 Y decoder β( ) Ŵ One sender, one receiver, and one intermediate node The problem does not define the set of relay functions {f i ( )} n i=1. The relay s strategy might be to decode the message, or compress its channel observation, or amplify it and retransmit it, or... Mikael Skoglund, Information Theory 22/25
12 Capacity is not known in general. Some known bounds: Cut-set upper bound: The relay is assumed to be co-located with the transmitter or with the receiver. R max p(x,x 1 ) min { I(X, X 1; Y ), I(X; Y, Y 1 X 1 ) } Decode-and-forward lower bound: R max p(x,x 1 ) min { I(X, X 1; Y ), I(X; Y 1 X 1 ) } Proof: Split transmission in b blocks. Choose 2 n R codewords i.i.d. p(x 2 ), and for each one, choose 2 nr codewords i.i.d. p(x 1 x 2 ) and distribute them in 2 n R bins. The relay can decode the message if R < I(X; Y 1 X 1 ) and then it sends the bin index in the next block. The receiver can decode the bin index if R < I(X1 ; Y ) and, knowing this index, it can decode the message from the previous block if R R < I(X; Y X 1 ). These bounds coincide if the relay channel is degraded: p(y, y 1 x, x 1 ) = p(y 1 x, x 1 )p(y y 1, x 1 ) Mikael Skoglund, Information Theory 23/25 General Multiterminal Systems M different nodes, each transmitting X m and receiving Y m. The message from node i to node j is W i,j with rate R i,j. The channel between nodes is p(y 1,..., y M x 1,..., x M ). Although significant progress in recent years, still only a few general results are known. One of them is El Gamal s cut-set bound: If the rates {R i,j } are achievable there exists a p(x 1,..., x m ) such that for all S {1,..., M}. i S,j S c R i,j < I(X (S) ; Y (Sc) X (Sc) ) Mikael Skoglund, Information Theory 24/25
13 The source channel separation principle: For general multiterminal networks the source and channel codes cannot be designed separately without loss. Essentially the source code needs to know the channel to provide optimal dependencies between channel input variables. Feedback: Feedback can increase the capacity of a multi-terminal channel, since it can help transmitters to cooperate to reduce interference. Mikael Skoglund, Information Theory 25/25
Lecture 10: Broadcast Channel and Superposition Coding
Lecture 10: Broadcast Channel and Superposition Coding Scribed by: Zhe Yao 1 Broadcast channel M 0M 1M P{y 1 y x} M M 01 1 M M 0 The capacity of the broadcast channel depends only on the marginal conditional
More informationA Comparison of Two Achievable Rate Regions for the Interference Channel
A Comparison of Two Achievable Rate Regions for the Interference Channel Hon-Fah Chong, Mehul Motani, and Hari Krishna Garg Electrical & Computer Engineering National University of Singapore Email: {g030596,motani,eleghk}@nus.edu.sg
More informationA Half-Duplex Cooperative Scheme with Partial Decode-Forward Relaying
A Half-Duplex Cooperative Scheme with Partial Decode-Forward Relaying Ahmad Abu Al Haija, and Mai Vu, Department of Electrical and Computer Engineering McGill University Montreal, QC H3A A7 Emails: ahmadabualhaija@mailmcgillca,
More informationLecture 1: The Multiple Access Channel. Copyright G. Caire 12
Lecture 1: The Multiple Access Channel Copyright G. Caire 12 Outline Two-user MAC. The Gaussian case. The K-user case. Polymatroid structure and resource allocation problems. Copyright G. Caire 13 Two-user
More informationEfficient Use of Joint Source-Destination Cooperation in the Gaussian Multiple Access Channel
Efficient Use of Joint Source-Destination Cooperation in the Gaussian Multiple Access Channel Ahmad Abu Al Haija ECE Department, McGill University, Montreal, QC, Canada Email: ahmad.abualhaija@mail.mcgill.ca
More informationLecture 3: Channel Capacity
Lecture 3: Channel Capacity 1 Definitions Channel capacity is a measure of maximum information per channel usage one can get through a channel. This one of the fundamental concepts in information theory.
More informationSuperposition Encoding and Partial Decoding Is Optimal for a Class of Z-interference Channels
Superposition Encoding and Partial Decoding Is Optimal for a Class of Z-interference Channels Nan Liu and Andrea Goldsmith Department of Electrical Engineering Stanford University, Stanford CA 94305 Email:
More informationInformation Theory for Wireless Communications. Lecture 10 Discrete Memoryless Multiple Access Channel (DM-MAC): The Converse Theorem
Information Theory for Wireless Communications. Lecture 0 Discrete Memoryless Multiple Access Channel (DM-MAC: The Converse Theorem Instructor: Dr. Saif Khan Mohammed Scribe: Antonios Pitarokoilis I. THE
More informationOn Multiple User Channels with State Information at the Transmitters
On Multiple User Channels with State Information at the Transmitters Styrmir Sigurjónsson and Young-Han Kim* Information Systems Laboratory Stanford University Stanford, CA 94305, USA Email: {styrmir,yhk}@stanford.edu
More informationRelay Networks With Delays
Relay Networks With Delays Abbas El Gamal, Navid Hassanpour, and James Mammen Department of Electrical Engineering Stanford University, Stanford, CA 94305-9510 Email: {abbas, navid, jmammen}@stanford.edu
More informationThe Gallager Converse
The Gallager Converse Abbas El Gamal Director, Information Systems Laboratory Department of Electrical Engineering Stanford University Gallager s 75th Birthday 1 Information Theoretic Limits Establishing
More informationAchievable Rates and Outer Bound for the Half-Duplex MAC with Generalized Feedback
1 Achievable Rates and Outer Bound for the Half-Duplex MAC with Generalized Feedback arxiv:1108.004v1 [cs.it] 9 Jul 011 Ahmad Abu Al Haija and Mai Vu, Department of Electrical and Computer Engineering
More informationDistributed Lossless Compression. Distributed lossless compression system
Lecture #3 Distributed Lossless Compression (Reading: NIT 10.1 10.5, 4.4) Distributed lossless source coding Lossless source coding via random binning Time Sharing Achievability proof of the Slepian Wolf
More informationA Comparison of Superposition Coding Schemes
A Comparison of Superposition Coding Schemes Lele Wang, Eren Şaşoğlu, Bernd Bandemer, and Young-Han Kim Department of Electrical and Computer Engineering University of California, San Diego La Jolla, CA
More informationMulticoding Schemes for Interference Channels
Multicoding Schemes for Interference Channels 1 Ritesh Kolte, Ayfer Özgür, Haim Permuter Abstract arxiv:1502.04273v1 [cs.it] 15 Feb 2015 The best known inner bound for the 2-user discrete memoryless interference
More informationAn Achievable Rate Region for the 3-User-Pair Deterministic Interference Channel
Forty-Ninth Annual Allerton Conference Allerton House, UIUC, Illinois, USA September 8-3, An Achievable Rate Region for the 3-User-Pair Deterministic Interference Channel Invited Paper Bernd Bandemer and
More informationOn the Capacity Region of the Gaussian Z-channel
On the Capacity Region of the Gaussian Z-channel Nan Liu Sennur Ulukus Department of Electrical and Computer Engineering University of Maryland, College Park, MD 74 nkancy@eng.umd.edu ulukus@eng.umd.edu
More informationELEC546 Review of Information Theory
ELEC546 Review of Information Theory Vincent Lau 1/1/004 1 Review of Information Theory Entropy: Measure of uncertainty of a random variable X. The entropy of X, H(X), is given by: If X is a discrete random
More informationA Summary of Multiple Access Channels
A Summary of Multiple Access Channels Wenyi Zhang February 24, 2003 Abstract In this summary we attempt to present a brief overview of the classical results on the information-theoretic aspects of multiple
More informationNational University of Singapore Department of Electrical & Computer Engineering. Examination for
National University of Singapore Department of Electrical & Computer Engineering Examination for EE5139R Information Theory for Communication Systems (Semester I, 2014/15) November/December 2014 Time Allowed:
More informationII. THE TWO-WAY TWO-RELAY CHANNEL
An Achievable Rate Region for the Two-Way Two-Relay Channel Jonathan Ponniah Liang-Liang Xie Department of Electrical Computer Engineering, University of Waterloo, Canada Abstract We propose an achievable
More informationMultiaccess Channels with State Known to One Encoder: A Case of Degraded Message Sets
Multiaccess Channels with State Known to One Encoder: A Case of Degraded Message Sets Shivaprasad Kotagiri and J. Nicholas Laneman Department of Electrical Engineering University of Notre Dame Notre Dame,
More informationShannon s noisy-channel theorem
Shannon s noisy-channel theorem Information theory Amon Elders Korteweg de Vries Institute for Mathematics University of Amsterdam. Tuesday, 26th of Januari Amon Elders (Korteweg de Vries Institute for
More informationHalf-Duplex Gaussian Relay Networks with Interference Processing Relays
Half-Duplex Gaussian Relay Networks with Interference Processing Relays Bama Muthuramalingam Srikrishna Bhashyam Andrew Thangaraj Department of Electrical Engineering Indian Institute of Technology Madras
More informationOn the Capacity of the Interference Channel with a Relay
On the Capacity of the Interference Channel with a Relay Ivana Marić, Ron Dabora and Andrea Goldsmith Stanford University, Stanford, CA {ivanam,ron,andrea}@wsl.stanford.edu Abstract Capacity gains due
More informationCut-Set Bound and Dependence Balance Bound
Cut-Set Bound and Dependence Balance Bound Lei Xiao lxiao@nd.edu 1 Date: 4 October, 2006 Reading: Elements of information theory by Cover and Thomas [1, Section 14.10], and the paper by Hekstra and Willems
More informationSimultaneous Nonunique Decoding Is Rate-Optimal
Fiftieth Annual Allerton Conference Allerton House, UIUC, Illinois, USA October 1-5, 2012 Simultaneous Nonunique Decoding Is Rate-Optimal Bernd Bandemer University of California, San Diego La Jolla, CA
More informationList of Figures. Acknowledgements. Abstract 1. 1 Introduction 2. 2 Preliminaries Superposition Coding Block Markov Encoding...
Contents Contents i List of Figures iv Acknowledgements vi Abstract 1 1 Introduction 2 2 Preliminaries 7 2.1 Superposition Coding........................... 7 2.2 Block Markov Encoding.........................
More informationIET Commun., 2009, Vol. 3, Iss. 4, pp doi: /iet-com & The Institution of Engineering and Technology 2009
Published in IET Communications Received on 10th March 2008 Revised on 10th July 2008 ISSN 1751-8628 Comprehensive partial decoding approach for two-level relay networks L. Ghabeli M.R. Aref Information
More informationCapacity Region of Reversely Degraded Gaussian MIMO Broadcast Channel
Capacity Region of Reversely Degraded Gaussian MIMO Broadcast Channel Jun Chen Dept. of Electrical and Computer Engr. McMaster University Hamilton, Ontario, Canada Chao Tian AT&T Labs-Research 80 Park
More informationarxiv: v2 [cs.it] 28 May 2017
Feedback and Partial Message Side-Information on the Semideterministic Broadcast Channel Annina Bracher and Michèle Wigger arxiv:1508.01880v2 [cs.it] 28 May 2017 May 30, 2017 Abstract The capacity of the
More informationVariable Length Codes for Degraded Broadcast Channels
Variable Length Codes for Degraded Broadcast Channels Stéphane Musy School of Computer and Communication Sciences, EPFL CH-1015 Lausanne, Switzerland Email: stephane.musy@ep.ch Abstract This paper investigates
More informationLECTURE 13. Last time: Lecture outline
LECTURE 13 Last time: Strong coding theorem Revisiting channel and codes Bound on probability of error Error exponent Lecture outline Fano s Lemma revisited Fano s inequality for codewords Converse to
More informationLecture 8: Channel and source-channel coding theorems; BEC & linear codes. 1 Intuitive justification for upper bound on channel capacity
5-859: Information Theory and Applications in TCS CMU: Spring 23 Lecture 8: Channel and source-channel coding theorems; BEC & linear codes February 7, 23 Lecturer: Venkatesan Guruswami Scribe: Dan Stahlke
More informationLecture 6 I. CHANNEL CODING. X n (m) P Y X
6- Introduction to Information Theory Lecture 6 Lecturer: Haim Permuter Scribe: Yoav Eisenberg and Yakov Miron I. CHANNEL CODING We consider the following channel coding problem: m = {,2,..,2 nr} Encoder
More informationPhysical-Layer MIMO Relaying
Model Gaussian SISO MIMO Gauss.-BC General. Physical-Layer MIMO Relaying Anatoly Khina, Tel Aviv University Joint work with: Yuval Kochman, MIT Uri Erez, Tel Aviv University August 5, 2011 Model Gaussian
More informationX 1 : X Table 1: Y = X X 2
ECE 534: Elements of Information Theory, Fall 200 Homework 3 Solutions (ALL DUE to Kenneth S. Palacio Baus) December, 200. Problem 5.20. Multiple access (a) Find the capacity region for the multiple-access
More informationNetwork coding for multicast relation to compression and generalization of Slepian-Wolf
Network coding for multicast relation to compression and generalization of Slepian-Wolf 1 Overview Review of Slepian-Wolf Distributed network compression Error exponents Source-channel separation issues
More informationPrimary Rate-Splitting Achieves Capacity for the Gaussian Cognitive Interference Channel
Primary Rate-Splitting Achieves Capacity for the Gaussian Cognitive Interference Channel Stefano Rini, Ernest Kurniawan and Andrea Goldsmith Technische Universität München, Munich, Germany, Stanford University,
More informationTwo Applications of the Gaussian Poincaré Inequality in the Shannon Theory
Two Applications of the Gaussian Poincaré Inequality in the Shannon Theory Vincent Y. F. Tan (Joint work with Silas L. Fong) National University of Singapore (NUS) 2016 International Zurich Seminar on
More informationLecture 22: Final Review
Lecture 22: Final Review Nuts and bolts Fundamental questions and limits Tools Practical algorithms Future topics Dr Yao Xie, ECE587, Information Theory, Duke University Basics Dr Yao Xie, ECE587, Information
More informationLecture 5 Channel Coding over Continuous Channels
Lecture 5 Channel Coding over Continuous Channels I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw November 14, 2014 1 / 34 I-Hsiang Wang NIT Lecture 5 From
More informationOn Capacity Under Received-Signal Constraints
On Capacity Under Received-Signal Constraints Michael Gastpar Dept. of EECS, University of California, Berkeley, CA 9470-770 gastpar@berkeley.edu Abstract In a world where different systems have to share
More informationEE/Stat 376B Handout #5 Network Information Theory October, 14, Homework Set #2 Solutions
EE/Stat 376B Handout #5 Network Information Theory October, 14, 014 1. Problem.4 parts (b) and (c). Homework Set # Solutions (b) Consider h(x + Y ) h(x + Y Y ) = h(x Y ) = h(x). (c) Let ay = Y 1 + Y, where
More informationLecture 4 Noisy Channel Coding
Lecture 4 Noisy Channel Coding I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw October 9, 2015 1 / 56 I-Hsiang Wang IT Lecture 4 The Channel Coding Problem
More informationECE Information theory Final (Fall 2008)
ECE 776 - Information theory Final (Fall 2008) Q.1. (1 point) Consider the following bursty transmission scheme for a Gaussian channel with noise power N and average power constraint P (i.e., 1/n X n i=1
More informationThe Capacity Region of the Gaussian Cognitive Radio Channels at High SNR
The Capacity Region of the Gaussian Cognitive Radio Channels at High SNR 1 Stefano Rini, Daniela Tuninetti and Natasha Devroye srini2, danielat, devroye @ece.uic.edu University of Illinois at Chicago Abstract
More informationLecture 4 Channel Coding
Capacity and the Weak Converse Lecture 4 Coding I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw October 15, 2014 1 / 16 I-Hsiang Wang NIT Lecture 4 Capacity
More informationInterference Channel aided by an Infrastructure Relay
Interference Channel aided by an Infrastructure Relay Onur Sahin, Osvaldo Simeone, and Elza Erkip *Department of Electrical and Computer Engineering, Polytechnic Institute of New York University, Department
More informationOn the Capacity of the Two-Hop Half-Duplex Relay Channel
On the Capacity of the Two-Hop Half-Duplex Relay Channel Nikola Zlatanov, Vahid Jamali, and Robert Schober University of British Columbia, Vancouver, Canada, and Friedrich-Alexander-University Erlangen-Nürnberg,
More informationInteractive Decoding of a Broadcast Message
In Proc. Allerton Conf. Commun., Contr., Computing, (Illinois), Oct. 2003 Interactive Decoding of a Broadcast Message Stark C. Draper Brendan J. Frey Frank R. Kschischang University of Toronto Toronto,
More informationRandom Access: An Information-Theoretic Perspective
Random Access: An Information-Theoretic Perspective Paolo Minero, Massimo Franceschetti, and David N. C. Tse Abstract This paper considers a random access system where each sender can be in two modes of
More informationLECTURE 10. Last time: Lecture outline
LECTURE 10 Joint AEP Coding Theorem Last time: Error Exponents Lecture outline Strong Coding Theorem Reading: Gallager, Chapter 5. Review Joint AEP A ( ɛ n) (X) A ( ɛ n) (Y ) vs. A ( ɛ n) (X, Y ) 2 nh(x)
More informationCapacity bounds for multiple access-cognitive interference channel
Mirmohseni et al. EURASIP Journal on Wireless Communications and Networking, :5 http://jwcn.eurasipjournals.com/content///5 RESEARCH Open Access Capacity bounds for multiple access-cognitive interference
More informationCapacity Theorems for Relay Channels
Capacity Theorems for Relay Channels Abbas El Gamal Department of Electrical Engineering Stanford University April, 2006 MSRI-06 Relay Channel Discrete-memoryless relay channel [vm 7] Relay Encoder Y n
More informationSource and Channel Coding for Correlated Sources Over Multiuser Channels
Source and Channel Coding for Correlated Sources Over Multiuser Channels Deniz Gündüz, Elza Erkip, Andrea Goldsmith, H. Vincent Poor Abstract Source and channel coding over multiuser channels in which
More informationInterference Channels with Source Cooperation
Interference Channels with Source Cooperation arxiv:95.319v1 [cs.it] 19 May 29 Vinod Prabhakaran and Pramod Viswanath Coordinated Science Laboratory University of Illinois, Urbana-Champaign Urbana, IL
More informationOn the Duality of Gaussian Multiple-Access and Broadcast Channels
On the Duality of Gaussian ultiple-access and Broadcast Channels Xiaowei Jin I. INTODUCTION Although T. Cover has been pointed out in [] that one would have expected a duality between the broadcast channel(bc)
More informationFeedback Capacity of the Gaussian Interference Channel to Within Bits: the Symmetric Case
1 arxiv:0901.3580v1 [cs.it] 23 Jan 2009 Feedback Capacity of the Gaussian Interference Channel to Within 1.7075 Bits: the Symmetric Case Changho Suh and David Tse Wireless Foundations in the Department
More informationJoint Source-Channel Coding for the Multiple-Access Relay Channel
Joint Source-Channel Coding for the Multiple-Access Relay Channel Yonathan Murin, Ron Dabora Department of Electrical and Computer Engineering Ben-Gurion University, Israel Email: moriny@bgu.ac.il, ron@ee.bgu.ac.il
More information5958 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 56, NO. 12, DECEMBER 2010
5958 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 56, NO. 12, DECEMBER 2010 Capacity Theorems for Discrete, Finite-State Broadcast Channels With Feedback and Unidirectional Receiver Cooperation Ron Dabora
More informationAn Achievable Rate for the Multiple Level Relay Channel
An Achievable Rate for the Multiple Level Relay Channel Liang-Liang Xie and P. R. Kumar Department of Electrical and Computer Engineering, and Coordinated Science Laboratory University of Illinois, Urbana-Champaign
More informationApproximately achieving the feedback interference channel capacity with point-to-point codes
Approximately achieving the feedback interference channel capacity with point-to-point codes Joyson Sebastian*, Can Karakus*, Suhas Diggavi* Abstract Superposition codes with rate-splitting have been used
More informationLECTURE 15. Last time: Feedback channel: setting up the problem. Lecture outline. Joint source and channel coding theorem
LECTURE 15 Last time: Feedback channel: setting up the problem Perfect feedback Feedback capacity Data compression Lecture outline Joint source and channel coding theorem Converse Robustness Brain teaser
More informationSum Capacity of Gaussian Vector Broadcast Channels
Sum Capacity of Gaussian Vector Broadcast Channels Wei Yu, Member IEEE and John M. Cioffi, Fellow IEEE Abstract This paper characterizes the sum capacity of a class of potentially non-degraded Gaussian
More informationCognitive Multiple Access Networks
Cognitive Multiple Access Networks Natasha Devroye Email: ndevroye@deas.harvard.edu Patrick Mitran Email: mitran@deas.harvard.edu Vahid Tarokh Email: vahid@deas.harvard.edu Abstract A cognitive radio can
More informationIN this paper, we show that the scalar Gaussian multiple-access
768 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004 On the Duality of Gaussian Multiple-Access and Broadcast Channels Nihar Jindal, Student Member, IEEE, Sriram Vishwanath, and Andrea
More informationChapter 9. Gaussian Channel
Chapter 9 Gaussian Channel Peng-Hua Wang Graduate Inst. of Comm. Engineering National Taipei University Chapter Outline Chap. 9 Gaussian Channel 9.1 Gaussian Channel: Definitions 9.2 Converse to the Coding
More informationClassical codes for quantum broadcast channels. arxiv: Ivan Savov and Mark M. Wilde School of Computer Science, McGill University
Classical codes for quantum broadcast channels arxiv:1111.3645 Ivan Savov and Mark M. Wilde School of Computer Science, McGill University International Symposium on Information Theory, Boston, USA July
More informationEE5585 Data Compression May 2, Lecture 27
EE5585 Data Compression May 2, 2013 Lecture 27 Instructor: Arya Mazumdar Scribe: Fangying Zhang Distributed Data Compression/Source Coding In the previous class we used a H-W table as a simple example,
More informationOn the Rate-Limited Gelfand-Pinsker Problem
On the Rate-Limited Gelfand-Pinsker Problem Ravi Tandon Sennur Ulukus Department of Electrical and Computer Engineering University of Maryland, College Park, MD 74 ravit@umd.edu ulukus@umd.edu Abstract
More informationOn Source-Channel Communication in Networks
On Source-Channel Communication in Networks Michael Gastpar Department of EECS University of California, Berkeley gastpar@eecs.berkeley.edu DIMACS: March 17, 2003. Outline 1. Source-Channel Communication
More informationCapacity of a Class of Deterministic Relay Channels
Capacity of a Class of Deterministic Relay Channels Thomas M. Cover Information Systems Laboratory Stanford University Stanford, CA 94305, USA cover@ stanford.edu oung-han Kim Department of ECE University
More informationSecret Key Agreement Using Conferencing in State- Dependent Multiple Access Channels with An Eavesdropper
Secret Key Agreement Using Conferencing in State- Dependent Multiple Access Channels with An Eavesdropper Mohsen Bahrami, Ali Bereyhi, Mahtab Mirmohseni and Mohammad Reza Aref Information Systems and Security
More informationLecture 11: Quantum Information III - Source Coding
CSCI5370 Quantum Computing November 25, 203 Lecture : Quantum Information III - Source Coding Lecturer: Shengyu Zhang Scribe: Hing Yin Tsang. Holevo s bound Suppose Alice has an information source X that
More informationEE 4TM4: Digital Communications II. Channel Capacity
EE 4TM4: Digital Communications II 1 Channel Capacity I. CHANNEL CODING THEOREM Definition 1: A rater is said to be achievable if there exists a sequence of(2 nr,n) codes such thatlim n P (n) e (C) = 0.
More informationECE Information theory Final
ECE 776 - Information theory Final Q1 (1 point) We would like to compress a Gaussian source with zero mean and variance 1 We consider two strategies In the first, we quantize with a step size so that the
More informationLecture 15: Conditional and Joint Typicaility
EE376A Information Theory Lecture 1-02/26/2015 Lecture 15: Conditional and Joint Typicaility Lecturer: Kartik Venkat Scribe: Max Zimet, Brian Wai, Sepehr Nezami 1 Notation We always write a sequence of
More informationApproximate Capacity of Fast Fading Interference Channels with no CSIT
Approximate Capacity of Fast Fading Interference Channels with no CSIT Joyson Sebastian, Can Karakus, Suhas Diggavi Abstract We develop a characterization of fading models, which assigns a number called
More informationEE5139R: Problem Set 7 Assigned: 30/09/15, Due: 07/10/15
EE5139R: Problem Set 7 Assigned: 30/09/15, Due: 07/10/15 1. Cascade of Binary Symmetric Channels The conditional probability distribution py x for each of the BSCs may be expressed by the transition probability
More informationReliable Computation over Multiple-Access Channels
Reliable Computation over Multiple-Access Channels Bobak Nazer and Michael Gastpar Dept. of Electrical Engineering and Computer Sciences University of California, Berkeley Berkeley, CA, 94720-1770 {bobak,
More informationLecture 5: Channel Capacity. Copyright G. Caire (Sample Lectures) 122
Lecture 5: Channel Capacity Copyright G. Caire (Sample Lectures) 122 M Definitions and Problem Setup 2 X n Y n Encoder p(y x) Decoder ˆM Message Channel Estimate Definition 11. Discrete Memoryless Channel
More informationOn Gaussian MIMO Broadcast Channels with Common and Private Messages
On Gaussian MIMO Broadcast Channels with Common and Private Messages Ersen Ekrem Sennur Ulukus Department of Electrical and Computer Engineering University of Maryland, College Park, MD 20742 ersen@umd.edu
More informationThe Capacity of the Semi-Deterministic Cognitive Interference Channel and its Application to Constant Gap Results for the Gaussian Channel
The Capacity of the Semi-Deterministic Cognitive Interference Channel and its Application to Constant Gap Results for the Gaussian Channel Stefano Rini, Daniela Tuninetti, and Natasha Devroye Department
More informationSource-Channel Coding Theorems for the Multiple-Access Relay Channel
Source-Channel Coding Theorems for the Multiple-Access Relay Channel Yonathan Murin, Ron Dabora, and Deniz Gündüz Abstract We study reliable transmission of arbitrarily correlated sources over multiple-access
More informationAn Outer Bound for the Gaussian. Interference channel with a relay.
An Outer Bound for the Gaussian Interference Channel with a Relay Ivana Marić Stanford University Stanford, CA ivanam@wsl.stanford.edu Ron Dabora Ben-Gurion University Be er-sheva, Israel ron@ee.bgu.ac.il
More informationEquivalence for Networks with Adversarial State
Equivalence for Networks with Adversarial State Oliver Kosut Department of Electrical, Computer and Energy Engineering Arizona State University Tempe, AZ 85287 Email: okosut@asu.edu Jörg Kliewer Department
More informationBounds on Achievable Rates for General Multi-terminal Networks with Practical Constraints
Bounds on Achievable Rates for General Multi-terminal Networs with Practical Constraints Mohammad Ali Khojastepour, Ashutosh Sabharwal, and Behnaam Aazhang Department of Electrical and Computer Engineering
More informationOn Network Interference Management
On Network Interference Management Aleksandar Jovičić, Hua Wang and Pramod Viswanath March 3, 2008 Abstract We study two building-block models of interference-limited wireless networks, motivated by the
More informationDegrees of Freedom Region of the Gaussian MIMO Broadcast Channel with Common and Private Messages
Degrees of Freedom Region of the Gaussian MIMO Broadcast hannel with ommon and Private Messages Ersen Ekrem Sennur Ulukus Department of Electrical and omputer Engineering University of Maryland, ollege
More informationCapacity of All Nine Models of Channel Output Feedback for the Two-user Interference Channel
Capacity of All Nine Models of Channel Output Feedback for the Two-user Interference Channel Achaleshwar Sahai, Vaneet Aggarwal, Melda Yuksel and Ashutosh Sabharwal 1 Abstract arxiv:1104.4805v3 [cs.it]
More informationThe Unbounded Benefit of Encoder Cooperation for the k-user MAC
The Unbounded Benefit of Encoder Cooperation for the k-user MAC Parham Noorzad, Student Member, IEEE, Michelle Effros, Fellow, IEEE, and Michael Langberg, Senior Member, IEEE arxiv:1601.06113v2 [cs.it]
More informationThe Poisson Channel with Side Information
The Poisson Channel with Side Information Shraga Bross School of Enginerring Bar-Ilan University, Israel brosss@macs.biu.ac.il Amos Lapidoth Ligong Wang Signal and Information Processing Laboratory ETH
More informationAppendix B Information theory from first principles
Appendix B Information theory from first principles This appendix discusses the information theory behind the capacity expressions used in the book. Section 8.3.4 is the only part of the book that supposes
More informationEnergy State Amplification in an Energy Harvesting Communication System
Energy State Amplification in an Energy Harvesting Communication System Omur Ozel Sennur Ulukus Department of Electrical and Computer Engineering University of Maryland College Park, MD 20742 omur@umd.edu
More informationOn the Capacity of the Binary-Symmetric Parallel-Relay Network
On the Capacity of the Binary-Symmetric Parallel-Relay Network Lawrence Ong, Sarah J. Johnson, and Christopher M. Kellett arxiv:207.3574v [cs.it] 6 Jul 202 Abstract We investigate the binary-symmetric
More informationSolutions to Homework Set #2 Broadcast channel, degraded message set, Csiszar Sum Equality
1st Semester 2010/11 Solutions to Homework Set #2 Broadcast channel, degraded message set, Csiszar Sum Equality 1. Convexity of capacity region of broadcast channel. Let C R 2 be the capacity region of
More informationSolutions to Homework Set #3 Channel and Source coding
Solutions to Homework Set #3 Channel and Source coding. Rates (a) Channels coding Rate: Assuming you are sending 4 different messages using usages of a channel. What is the rate (in bits per channel use)
More informationOn the Duality between Multiple-Access Codes and Computation Codes
On the Duality between Multiple-Access Codes and Computation Codes Jingge Zhu University of California, Berkeley jingge.zhu@berkeley.edu Sung Hoon Lim KIOST shlim@kiost.ac.kr Michael Gastpar EPFL michael.gastpar@epfl.ch
More informationError Exponent Region for Gaussian Broadcast Channels
Error Exponent Region for Gaussian Broadcast Channels Lihua Weng, S. Sandeep Pradhan, and Achilleas Anastasopoulos Electrical Engineering and Computer Science Dept. University of Michigan, Ann Arbor, MI
More information