On Multiple User Channels with State Information at the Transmitters

Transcription

1 On Multiple User Channels with State Information at the Transmitters Styrmir Sigurjónsson and Young-Han Kim* Information Systems Laboratory Stanford University Stanford, CA 94305, USA Abstract We extend Shannon s result on the capacity of channels with state information to multiple user channels. More specifically, we characterize the capacity (region) of degraded broadcast channels and physically degraded relay channels where the channel state information is causally available at the transmitters. We also obtain inner and outer bounds on the capacity region for multiple access channels with causal state information at the transmitters. I. INTRODUCTION In his 1958 paper [1], Shannon considered a communication system where additional side information about the channel is causally available at the transmitter. (See Figure 1.) He showed W Encoder X i (W, ) S n Q n p(s i ) p(y x, s) Y i Decoder Fig. 1. Communication channel with state information causally known at the transmitter. that the capacity of this channel can be achieved by adding a physical device in front of the channel, which depends on the current state (and the message to be sent) only. Using the modern language, we can write the capacity of this channel as C max I(U; Y ) p(u)p(x u,s) max I(U; Y ), (1) p(u), xf(u,s) where U is an auxilliary random variable with finite cardinality. The achievability of the rate in (1) is clear from Shannon s original argument of attaching a physical device in front of the channel. The optimality of this rate is deceptively easy to show. Indeed, by recognizing U i : (W, Y i 1, 1 ) to be independent of and by observing that X i can be written as * This work was partly supported by NSF Grant CCR Ŵ a function of (U i, ), we have I(W; ) H( ) H( W) i i i H(Y i ) H(Y i W, Y i 1 ) H(Y i ) H(Y i W, Y i 1, 1 ) I(U i ; Y i ) nc. () It is also worth pointing out the similarity of (1) to the capacity C when the state information is noncausally available at the transmitter, as shown by Gel fand and Pinsker [] and Heegard and El Gamal [3]: C max I(U; Y ) I(U; S). p(u s)p(x u,s) Since the capacity C for the causal case can be written as max p(u)p(x u,s) I(U; Y ) I(U; S), the loss of causality merely lies in the independence between U and S. In this paper, we try to extend Shannon s result to multiple user channels. Research in this direction is not new. Most notably, Steinberg [4] obtained bounds on the capacity region of degraded broadcast channels when the state information is noncausally available at the transmitter, although the tightness of these bounds is still open. Causality of the state information makes the problem much easier. Recently, Steinberg [5] reported the capacity region of the degraded broadcast channel when the state information is causally available at the transmitter. In Section II, we give an independent treatment of his result by scaling the converse technique we just developed for the single-user case. In Section III, we run the same program for the physically degraded relay channel and obtain the capacity. Unfortunately, the optimality of the similar coding scheme for multiple access channels is yet to be established, and we present inner and outer bounds on the capacity region in Section IV. II. DEGRADED BROADCAST CHANNELS Definition 1: The discrete memoryless broadcast channel with state information consists of input alphabet X, state alphabet S, output alphabet Y 1 Y, and a probability transition function p(y 1, y x, s), as in Figure /05/$ IEEE 7 ISIT 005

2 We assume the state information S n is available causally to the transmitter. We also assume that the channel is physically degraded, i.e., p(y 1, y x, s) p(y 1 x, s)p(y y 1 ). (W 1, W ) X i(w 1, W, ) p(y 1, y x, s) 1 Ŵ 1 Ŵ thus have ( ) P E,1 c i 1 E,i P(E c,1) + i 1 P(E,i ) ǫ + nr n(i(u;y) ǫ). Hence, 0 if R < I(U ; Y ) ǫ. Similarily, ( ) 1 P E1,1,1 c i 1 E 1,i j 1 E 1,1,j P(E c 1,1,1) + i 1 P(E 1,i ) + j 1 P(E 1,1,j ) Fig.. Broadcast channel with state causally available at the transmitter. Theorem 1: The capacity region of the degraded broadcast channel with state information available causally at the transmitter is the set of all rate pairs (R 1, R ) satisfying R 1 I(U 1 ; Y 1 U ) R I(U ; Y ) for some joint distribution p(u 1, u )p(s), x f(u 1, u, s) where the auxiliary random variables U 1 and U have finite alphabets. Proof: Although the proof of the achievability can be found in [5], we repeat it here for the sake of completeness. Fix R 1, R and p(u )p(u 1 u )p(x u 1, u, s). Codebook Generation. Generate nr independent codewords u n (w ) according to n p(u i). For each of these codewords, generate nr1 independent codewords u n 1(w 1, w ) according to n p(u 1i u i ). Codebook assignments are fixed and revealed to the transmitter and the receivers. Encoder. To send w 1 {1,,..., nr1 } to receiver 1 and w {1,,..., nr } to receiver, we select the corresponding codewords u n (w ) and u n 1(w 1, w ). At time i, upon observing the channel state s i, the transmitter sends x i f(u 1,i, u,i, s i ). Decoding. Receiver declares that a message ŵ was sent if there is a unique ŵ such that u n (ŵ ) and y n are jointly typical; otherwise an error is declared. Receiver 1 declares that a message pair (ŵ 1, ŵ ) was sent if there is a unique (ŵ 1, ŵ ) such that u n 1(ŵ 1, ŵ ), u n (ŵ ) and y n are jointly typical; otherwise an error is declared. Define the following events: E,i {(U n (i), ) A (n) E 1,i {(U n (i), Y1 n ) A (n) E 1,i,j {(U n (i), U1 n (j, i), Y1 n ) A (n) Without loss of generality, assume the sent messages were w 1 1 and w 1 and denote the probabilities that receivers 1 and declare an error with 1 and, respectively. We By the asymptotic equipartition property (AEP), P(E c 1,1,1) 0 and i 1 P(E 1,i) nr n(i(u;y1) ǫ) 0 if R < I(U ; Y 1 ). But we already have R < I(U ; Y ) and the degradedness implies I(U ; Y ) < I(U ; Y 1 ). Similarly, we can show that P(E 1,1,j ) n(i(u1;y1 U) 3ǫ). Thus j 1 P(E 1,1,j) nr1 n(i(u1;y1 U) 3ǫ) 0 if R 1 < I(U 1 ; Y 1 U ) 3ǫ, which completes the proof of achievability. Let us now prove the converse. We wish to show that given any sequence of (( nr1, nr ), n) codes (X i (W 1, W, ), Ŵ 1 (Y1 n ), Ŵ ( )) such that e : P(W 1 Ŵ1 or W Ŵ) 0 for uniform and independent message indices W 1 and W, the rate pair (R 1, R ) must satisfy the conditions in the theorem. By Fano s inequality, H(W 1 Y1 n ) nr 1 P e (n) + 1 nǫ 1n H(W ) nr P e (n) + 1 nǫ n since P e (n) max{ 1, }. Define the following auxiliary random variables, U 1i : (W 1, Y1 i 1, 1 ) and U i : (W, Y1 i 1 ). We now have Similarily, nr H(W ) I(W ; ) + H(W ) I(W ; ) + nǫ n nr 1 H(W 1 ) I(W ; Y i Y i 1 ) + nǫ n I(W, Y i 1 ; Y i ) + nǫ n I(W, Y1 i 1 ; Y i) + nǫ n }{{} U i I(U i ; Y i ) + nǫ n. 73

3 I(W 1 ; Y1 n ) + H(W 1 Y1 n ) I(W 1 ; Y1 n ) + nǫ 1n I(W 1 ; Y1 n W ) + nǫ 1n I(W 1 ; Y 1i W, Y i 1 1 ) + nǫ 1n I(W 1, Y1 i 1, 1 i 1 ; Y 1i W, Y1 ) + nǫ 1n }{{}}{{} U 1,i U,i I(U 1i ; Y 1i U i ) + nǫ 1n. We define a random variable Q independent of everything else, uniformly distributed over {1,,...,n} and define U 1 (Q, U 1Q ), U (Q, U Q ), X X Q, S S Q, Y 1 Y 1Q, and Y Y Q. Then, we have and R 1 1 n 1 n I(U 1i ; Y 1i U i ) + ǫ 1n I(U 1i ; Y 1i U i, Q i) + ǫ 1n I(U 1Q ; Y 1Q U Q, Q) + ǫ 1n I(U 1Q, Q; Y 1Q U Q, Q) + ǫ 1n I(U 1 ; Y 1 U ) + ǫ 1n, R 1 n I(U i ; Y i ) + ǫ n I(U Q ; Y Q Q) + ǫ n I(U Q, Q; Y Q ) + ǫ n I(U ; Y ) + ǫ n. But it is see to see that (U 1, U ) is independent of S, that X is a deterministic function of (U 1, U, S), and that the joint distribution of (X, S, Y 1, Y ) is consistent with the channel p(y 1, y x, s). Thus, we have established the converse of the theorem. III. PHYSICALLY DEGRADED RELAY CHANNELS Definition : The discrete memoryless relay channel with state information consists of input alphabet X, relay input alphabet X 1, state alphabet S, relay output alphabet Y 1, output alphabet Y, and a probability transition function p(y, y 1 x, x 1, s). In the following, we will assume that the state variable S is causally available to the transmitter and the relay. We also assume that the channel is physically degraded, i.e., p(y, y 1 x, x 1, s) p(y 1 x, x 1, s)p(y y 1, x 1, s). The main result of this section is the following: Theorem : The capacity of the degraded relay channel with state information causally available at the transmitter and Relay Y 1 X 1 W X i (W, ) p(y, y 1 x, x 1, s) Fig. 3. Relay channel with state information available causally at the transmitter and relay relay is given by C max min{i(u, U 1 ; Y ), I(U; Y 1 U 1, S)}. p(u,u 1) xf(u,s) x 1f 1(u 1,s) Proof: We first prove the achievability of the rate region. The approach follows Shannon s method of attaching physical devices [1] and the relay coding theorem by Cover and El Gamal [6], which transforms the original relay channel into one with auxilliary inputs U and U 1. More specifically, we use the block Markov encoding. A sequence of B 1 messages, w i W, each selected independly and uniformly over W, are to be sent over the channel in nb transmissions. Within each block of length n, the sender and the relay use a doubly-indexed set of codewords C {(u n (w t), u n 1(t)) : w {1,..., nr }, Ŵ t {, 1,..., nr0 }} Fix a probability distribution p(u, u 1 ), x f(u, s), x 1 f 1 (u 1, s). Codebook Generation: Generate at random nr0 independent n-sequences u n 1 (t), t {1,...,nR0 }, each drawn according to n p(u 1i). For each u n 1(t) sequence, generate nr conditionally independent u n (w t) sequences drawn according to n p(u i u 1i ). This defines the random codebook C. For each message w {1,..., nr } assign an index t(w) at random from {1,..., nr }. The set of messages with the same index form a bin T t W. The codebook and bin assignments are revealed to all parties. Encoding: Let w(b) {1,..., nr } be the new index to be sent in block b, and assume that w(b 1) T t(b). The encoder then selects u n (w(b) t(b)). At time i in block b, upon receiving s i (b), the sender sends x i (b) f(u i (w(b) t(b)), s i (b)). The relay will have an estimate ŵ(b 1) of the previous index w(b 1). Assume that ŵ(b 1). Then upon receiving Tˆt(b) s(b) the relay encoder sends x 1i f 1 (u 1i (ˆt(b)), s i (b)). Decoding: We assume that at the end of block b 1, the receiver knows (w(1),...,w(b )) and (t(1),...,t(b 1)) and the relay knows (w(1),...,w(b 1)) and consequently (t(1),...,t(b)). The decoding procedures at the end of block b are as follows: 1) With (t(b), y1 n (b), s n (b)), the relay estimates the message of the transmitter as ŵ(b) if there exists a unique ŵ(b) such that (u n (ŵ(b) t(b)), u n 1(t(b)), y1 n (b), s n (b)) are jointly typical. 74

4 It can be shown that ŵ(b) w(b) with arbitrarily small probability of error, if R < I(U; Y 1 U 1, S) and n sufficiently large. ) The receiver declares that ˆt(b) was sent if there exists exactly one ˆt(b) such that (u n 1 (ˆt(b)), y n (b)) is jointly typical. It can be shown that ˆt(b) t(b) with arbitrarily small probability of error, if R 0 < I(U 1 ; Y ) (3) and n sufficiently large. 3) Assuming that t(b) is decoded succesfully at the receiver, then ŵ(b 1) is declared to be the index sent in block b 1 if there is a unique ŵ(b 1) T t(b) that is jointly typical with y n (b 1). It can be shown that if n is sufficiently large and if R < I(U; Y U 1 ) + R 0 (4) then ŵ(b 1) w(b 1)x with arbitrarily small probability of error. Combining (3) and (4) yields the condition R < I(U, U 1 ; Y ). Let us now prove the converse for any rate-r code with encoding functions X i (W, ) and X 1i (, Y i 1 ). We define the auxiliary random variables U i (W, Y i 1, 1 ) and U 1i (Y1 i 1, 1 ). It is easy to see that (U i, U 1i ) is independent of and that X i and X 1i are deterministic functions of (U i, ) and (U 1i, ), respectively. We have I(W; ) I(W; Y i Y i 1 ) H(Y i Y i 1 ) H(Y i Y i 1, W) H(Y i ) H(Y i Y i 1, Y i 1 1, W, 1 ) I(U i, U 1i ; Y i ) I(W; ) I(W;, Y1 n S n ) H(Y i, Y 1i Y i 1, Y1 i 1, S n ) H(Y i, Y 1i Y i 1, Y1 i 1, S n, W) H(Y i, Y 1i Y i 1, Y1 i 1, S n ) H(Y i, Y 1i Y i 1, Y1 i 1, S n, W, X i, X 1i ) H(Y i, Y 1i Y1 i 1, 1, ) H(Y i, Y 1i X i, X 1i, ) H(Y i, Y 1i Y1 i 1, 1, ) H(Y i, Y 1i X i, X 1i,, Y i 1, Y i 1 1, 1, W) H(Y i, Y 1i U 1i, ) H(Y i, Y 1i U i, U 1i, ) I(U i ; Y i, Y 1i U 1i, ). Fano s inequality and the use of the usual time-sharing random variable will show R min {I(U, U 1 ; Y ), I(U; Y, Y 1 U 1, S)} + ǫ n. Combining the following Markov relationship U (S, U 1, Y 1 ) Y with the degradedness of the channel and the fact that X and X 1 are functions of (U, S) and (U 1, S), we can easily show that I(U; Y, Y 1 U 1, S) I(U; Y 1 U 1, S). This completes the proof of the converse. IV. MULTIPLE ACCESS CHANNELS Definition 3: The discrete memoryless multiple access channel with state information consists of input alphabet X 1 X, state alphabet S, output alphabet Y, and a probability transition function p(y x 1, x, s); see Figure 4. Again we consider the case when the state variable s causally available to the transmitters. W 1 W X 1i(W 1, ) X i(w, ) p(y x 1, x, s) (Ŵ1, Ŵ) Fig. 4. Multiple access channel with state causally available to the transmitters. Define the region Rp(u mac 1,u ) to be the convex hull of all rate pairs (R 1, R ) satisfying R 1 I(U 1 ; Y U ) R I(U ; Y U 1 ) R 1 + R I(U 1, U ; Y ) over all p(u 1, u ), x 1 f 1 (u 1, s), x f(u, s). Similarly, define Rp(u mac 1)p(u ) to be the convex hull of all rate pairs (R 1, R ) satisfying the same set of inequalities over all p(u 1, u ), x 1 f 1 (u 1, s), and x f(u, s). For the multiple access channel case, the capacity theorem is yet to be established, mostly because of the coupling of two auxiliary random variables in the proof of the converse. Here we give bounds on the capacity region instead. Theorem 3: Let C mac denote the capacity region of the multiple access channel with state causally available at both transmitters. Then Rp(u mac 1)p(u ) Cmac Rp(u mac. 1,u ) Proof: Following the same arguments as before, it is not hard to show that C mac Rp(u mac 1,u ) ; hence, we skip the details. 75

5 For the lower bound, we first fix p(u 1 )p(u ), f 1 (u 1, s), and f (u, s). Codebook Generation. Generate nr1 independent codewords u n 1(w! ), w 1 {1,,..., nr1 }, generating each element i.i.d. n p(u 1i) and nr independent codewords u n (w ), w {1,,..., nr }, generating each element i.i.d. n p(u i). These codewords form the codebook, which is revealed to the senders and the receiver. Encoding. To send message indices w 1 {1,,..., nr1 } and w {1,,..., nr }, select the corresponding codewords u n 1 (w 1) and u n (w ). At time i, upon receiving s i, transmitter 1 sends x 1,i f 1 (u 1i (w 1 ), s i ) and transmitter acts similarly. Decoding. Let A ǫ (n) denote the set of typical (U1 n, Un, ) sequences. If there exists a unique pair Ŵ1 and Ŵ such that (U1 n(ŵ1), U n(ŵ), ) A ǫ (n) declare that (Ŵ1, Ŵ) was sent, otherwise declare an error. We now analyze the probability of error. Define the events, E i,j {(U n 1 (i), U n (j), ) A (n). Without loss of generality assume the messages sent were W 1 1 and W, the probability of error becomes, { } P e (n) P E1,1 c (i,j) (1,1) E i,j P(E1,1) c + P(E i,1 ) + P(E 1,j ) i 1 j 1 + P(E i,j ) (i,j) (1,1) From the AEP, P(E c 1,1) 0. For the second term we get P(E i,1 ) P {(U1 n (i), U n (1), ) A (n) p(u n 1)p(u n, y n ) (u n 1,un,yn ) A (n) ǫ (u n 1,un,yn ) A (n) ǫ n(h(u1) ǫ) n(h(u,y ) ǫ) n(h(u1,u,y )+ǫ) n(h(u1) ǫ) n(h(u,y ) ǫ) n(h(u1)+h(u,y ) H(U1,U,Y ) 3ǫ) n(i(u1;u,y ) 3ǫ) n(i(u1;y U) 3ǫ) where the last equality follows since U 1 and U are independent. Similarily for j 1, and for (i, j) (1, 1) P(E 1,j ) n(i(u;y U1) 3ǫ), P(E i,j ) n(i(u1,u;y ) 4ǫ). We can now write the probability of error as, e P(E1,1) c + nr1 n(i(u1;y U) 3ǫ) + nr n(i(u;y U1) 3ǫ) + n(r1+r) n(i(u1,u;y ) 4ǫ) which tends to zero if the conditions of the theorem are met. V. CONCLUDING REMARKS We characterized the capacity region of a few simple multiple user channels with state information, in single-letter formulas. The value of this result may lie in expanding the set of toy examples in network information theory and thus giving a glimpse on the structure of the theory. Questions still remain on other multiple user channels and on the cost of causality of state information. REFERENCES [1] C. E. Shannon, Channels with side information at the transmitter, IBM Journal of Research and Development, vol., pp , [] S. I. Gel fand and M. S. Pinsker, Coding for channel with random parameters, Problems of Control and Information Theory, vol. 9, no. 1 pp , [3] C. Heegard and A. A. El Gamal, On the capacity of computer memories with defects, IEEE Trans. Info. Theory, vol. 9, pp , September [4] Y. Steinberg, On the broadcast channel with random parameters, in Proc. IEEE ISIT 00 (Lausanne, Switzerland), p. 5, June 00. [5] Y. Steinberg, Coding for the degraded broadcast channel with random parameters, with causal and non-causal side information, accepted for publication in IEEE Trans. Info. Theory. [6] T. M. Cover and A. A. El Gamal, Capacity theorems for the relay channel, IEEE Trans. Info. Theory, vol. IT-5, pp , Sep