Homework Set # Rates definitions, Channel Coding, Source-Channel coding. Rates (a) Channels coding Rate: Assuming you are sending 4 different messages using usages of a channel. What is the rate (in bits er channel use) that you send. (b) Source coding Rate: Assuming you have a file with 6 Ascii characters, where the alhabet of Ascii characters is 56. Assume that each Ascii character is reresented by bits (binary alhabet before comression). After comressing it we get 4 6 bits. What is the comression rate?. Prerocessing the outut. One is given a communication channel with transition robabilities (y x) and channel caacity C = max (x) I(X;Y). A helful statistician rerocesses the outut by forming Ỹ = g(y), yielding a channel (ỹ x). He claims that this will strictly imrove the caacity. (a) Show that he is wrong. (b) Under what conditions does he not strictly decrease the caacity?. The Z channel. The Z-channel has binary inut and outut alhabets and transition robabilities (y x) given by the following matrix: [ ] Q = x,y {,} / / Find the caacity of the Z-channel and the maximizing inut robability distribution. 4. Using two channels at once. Consider two discrete memoryless channels (X,(y x ),Y ) and (X,(y x ),Y ) with caacities C and C resectively. A new channel (X X,(y x ) (y x ),Y Y ) is formed in which x X and x X, are simultaneously sent, resulting in y,y. Find the caacity of this channel.
5. A channel with two indeendent looks at Y. Let Y and Y be conditionally indeendent and conditionally identically distributed given X. Thus (y,y x) = (y x)(y x). (a) Show I(X;Y,Y ) = I(X;Y ) I(Y ;Y ). (b) Conclude that the caacity of the channel X (Y,Y ) is less than twice the caacity of the channel X Y 6. Choice of channels. Find the caacity C of the union of channels (X, (y x ),Y ) and (X, (y x ),Y ) where, at each time, one can send a symbol over channel or over channel but not both. Assume the outut alhabets are distinct and do not intersect. (a) Show C = C + C. (b) What is the caacity of this Channel? 7. Cascaded BSCs. Considerthetwodiscretememorylesschannels(X, (y x),y)and(y, (z y),z). Let (y x) and (z y) be binary symmetric channels with crossover robabilities λ and λ resectively.
X λ λ λ λ λ λ λ λ Y Z (a) What is the caacity C of (y x)? (b) What is the caacity C of (z y)? (c) Wenowcascadethesechannels. Thus (z x) = y (y x) (z y). What is the caacity C of (z x)? Show C min{c,c }. (d) Now let us actively intervene between channels and, rather than assively transmitting y n. What is the caacity of channel followed by channel if you are allowed to decode the outut y n of channel and then reencode it as ỹ n for transmission over channel? (Think W x n (W) y n ỹ n (y n ) z n Ŵ.) (e) What is the caacity of the cascade in art c) if the receiver can view both Y and Z? 8. Channel caacity (a) What is the caacity of the following channel in Fig. (aears on the next age). (b) Provide a simle scheme that can transmit at rate R = log bits through this channel.
(Inut) X Figure : A channel 4 Y (Outut) 9. A channel with a switch. Consider the channel that is deicted in Fig., there are two channels with the conditional robabilities (y x) and (y x). These two channels have common inut alhabet X and two disjoint outut alhabets Y,Y (a symbol that aears in Y can t aear in Y ). The osition of the switch is determined by R.V Z which is indeendent of X, where Pr(Z = ) = λ. (y x) Y Z = X Y (y x) Y Z = Figure : The channel. (a) Show that I(X;Y) = λi(x;y )+ λi(x;y ). () (b) The caacity of this system is given by C = max (x) I(X;Y). Show that C λc + λc, () 4
where C i = max (x) I(X;Y i ). When is equality achieved? (c) The sub-channels defined by (y x) and (y x) are now given in Fig.9, where =. Find the inut robability (x) that maximizes I(X; Y). For this case, does the equality stand in Eq. ()? exlain! (a) X Y (b) X Y Figure : (a) describes channel - BSC with transition robability. (b) describes channel - Z channel with transition robability.. Channel with state A discrete memoryless (DM) state deendent channel with state sace S is defined by an inut alhabet X, an outut alhabet Y and a set of channel transition matrices {(y x,s)} s S. Namely, for each s S the transmitter sees a different channel. The caacity of such a channel where the state is know causally to both encoder and decoder is given by: C = maxi(x;y S). () (x s) Let S = and the three different channels (one for each state s S) are as illustrated in Fig.4 The state rocess is i.i.d. according to the distribution (s). (a) Find an exression for the caacity of the S-channel (the channel the transmitter sees given S = ) as a function of ǫ. (b) Find an exression for the caacity of the BSC (the channel the transmitter sees given S = ) as a function of δ. 5
ǫ ǫ δ δ ǫ δ δ ǫ S = S = S = Figure 4: The three state deendent channel. (c) Find an exression for the caacity of the Z-channel (the channel the transmitter sees given S = ) as a function of ǫ. (d) Find an exression for the caacity of the DM state deendent channel (using formula ()) for (s) = [ ] as a function of ǫ 6 and δ. (e) LetusdefineaconditionalrobabilitymatrixP X S fortworandom variables X and S with X = {,} and S = {,,}, by: [ ], PX S = (x = j s = i). (4) i=,j= WhatistheinutconditionalrobabilitymatrixP X S thatachieves the caacity you have found in (d)?. Modulo Channel (a) Consider the DMC defined as follows: Outut Y = X Z where X, taking values in {,}, is the channel inut, is the modulo- summation oeration, and Z is binary channel noise uniform over {,} and indeendent of X. What is the caacity of this channel? (b) Consider the channel of the revious art, but suose that instead of modulo- addition Y = X Z, we erform modulo- addition Y = X Z. Now what is the caacity? 6
. Cascaded BSCs: Given is a cascade of k identical and indeendent binary symmetric channels, each with crossover robability α. (a) In the case where no encoding or decoding is allowed at the intermediate terminals, what is the caacity of this cascaded channel as a function of k,α. (b) Now, assume that encoding and decoding is allowed at the intermediate oints, what is the caacity as a function of k,α. (c) What is the caacity of each of the above settings in the case where the number of cascaded channels, k, goes to infinity? 7