Capacity Theorems for the Finite-State Broadcast Channel with Feedback

Transcription

1 Capacity Theorems for the Fiite-State Broadcast Chael with Feedback Ro abora ad Adrea Goldsmith ept of Electrical Egieerig, Staford Uiversity Abstract We cosider the discrete, time-varyig broadcast chael with memory uder the assumptio that the chael states belog to a set of ite cardiality We study the achievable rates i two scearios where feedback (ad cooperatio is available Oe sceario is the geeral ite-state broadcast chael (FSBC where both receivers sed feedback to the trasmitter, ad i additio oe receiver seds his chael outputs to the other receiver through a cooperatio lik The secod sceario is the degraded FSBC where oly the strog receiver seds feedback to the trasmitter We d the capacity regios for both cases I both scearios we cosider o-idecomposable as well as a class of idecomposable FSBCs I INTROUCTION The ite-state chael (FSC was itroduced as early as 953 to model time-varyig chaels with memory [] I this model memory is captured by the state of the chael at the ed of the previous symbol trasmissio To make this precise, let X deote the chael iput, Y the chael output ad S the chael state The trasitio fuctio of the FSC at symbol time i satises p(y i, s i x i, y i, s i p(y i, s i x i, s i Capacity aalysis of time-varyig chaels with memory has bee the focus of cosiderable iterest, especially i the Gaussia setup This has bee motivated by the proliferatio of mobile commuicatios i which the chael is subject to multipath ad correlated fadig The correlatio of the fadig process itroduces memory ito the time-varyig chael The capacity of o-idecomposable FSCs without feedback was origially studied by Gallager [2] ad has bee applied to specic chaels, see [3] ad refereces therei Recetly, the capacity of FSCs with feedback has bee studied Specically, it was show i [4] that feedback ca icrease the capacity of some FSCs with memory The capacity of the FS-MAC with ad without feedback was also cosidered recetly [5] I this work we focus o the capacity of ite-state broadcast chaels with feedback I this sceario a sigle trasmitter commuicates with two receivers over a itestate chael characterized by p(y, z, s x, s (see Figure for otatios, where s deotes the state of the chael at the ed of the previous symbol trasmissio I a previous work we cosidered the degraded FSBC without feedback [6] Specically, we rst deed the otios of physical ad stochastic degradedess for chaels with memory ad the showed that capacity is achieved with a superpositio codebook with memory The capacity regio was obtaied as a limitig expressio by takig the blocklegth to iity I this work we study the effect of feedback ad cooperatio The authors are with the Wireless Systems Lab, epartmet of Electrical Egieerig, Staford Uiversity, Staford, CA ro,adrea@wslstafordedu This work was supported i part by the ARPA ITMANET program uder grat TFIN o the capacity regio of the FSBC The degraded FSBC with feedback is of special iterest, as it was show i [7] that feedback does ot icrease the capacity regio of the physically degraded memoryless broadcast chael (BC, ad it would be iterestig to cotrast this with the case with memory (see discussio i Sectio III I the cotext of discrete multi-user chaels with states, we ote that the capacity of the degraded arbitrarily varyig BC (AVBC has bee recetly ivestigated i [8] ad [9] This chael is characterized by the trasitio fuctio p(y, z x, s p(y i, z i x i, s i This models a memoryless chael whose parameters vary with time i a arbitrary maer I [8] AVBCs with causal ad o-causal side iformatio at the trasmitter were cosidered, ad i [9] the capacity for AVBCs with causal side iformatio at the trasmitter ad o-causal side iformatio at the good receiver was derived The most geeral sceario for the FSBC with feedback ad cooperatio is depicted i Figure M M 2 Ecoder X i B Broadcast Chael p(z,y,s x,s A 2 Y i- Z i- Y i Z i ecoder (Rx A Z i- ecoder 2 (Rx 2 ^ ^ M ^ ^ M 2 Fig The geeral FSBC with feedback ad cooperatio idicates a sigle symbol-time delay Each of the three switches A, A 2 or B ca be either ope or closed Switch A eables cooperatio from Rx 2 to Rx, switch A 2 eables feedback from Rx 2 to the trasmitter, ad switch B eables feedback from Rx to the trasmitter From Figure we see that there are eight possible coguratios I this work we cosider two of the eight scearios (the sceario with all switches ope was cosidered i [6] We deote them by SC ad SC2: SC: The geeral FSBC with all switches closed Therefore, Rx 2 seds its chael output to both Rx ad the trasmitter This results i a physically degraded chael X s 0 (Y, Z s 0 Z s 0, but it is ot required that X s 0 Y s 0 Z s 0 form a Markov chai The sceario i which the chael output at oe receiver is available to the other receiver is also called the augmeted BC [0] Note that the delay from Rx 2 to Rx is of o

2 sigicace as the receivers ca wait util the ed of the etire block before begiig to decode SC2: The physically degraded BC with oly switch B closed Therefore, Rx 2 does ot sed feedback, ad the codebook is geerated without kowledge of the chael output at Rx 2 For SC2 we shall assume that the degradedess coditio X s 0 Y s 0 Z s 0 holds The sceario SC ca be viewed as represetig a uplik sceario that combies receiver cooperatio ad feedback: the weak (eg far receiver seds its chael output to the strog receiver, ad the feedback from the strog receiver to the trasmitter cotais both its ow feedback ad the feedback of the weak receiver, obtaied through the cooperatio lik The sceario SC2 ca represet a cellular situatio without cooperatio i which the mobile Rx 2 is too far from the base statio to reliably sed feedback to it, but the mobile Rx is close to the base statio ad thus has a reliable feedback path to it I Sectio III we explai why, despite the fact that oly oe receiver seds feedback, the rates to both receivers icrease This is a importat beet of feedback i multi-user scearios II CHANNEL MOEL First, a word about otatio I the followig we deote radom variables with upper case letters, eg X, Y, ad their realizatios with lower case letters x, y A radom variable (RV X takes values i a set X We use X to deote the cardiality of a ite, discrete set X, X to deote the -fold Cartesia product of X, ad p X (x to deote the probability mass fuctio (pmf of a discrete RV X o X For brevity we may omit the subscript X whe it is obvious from the cotext We use p X Y (x y to deote the coditioal pmf of X give Y We deote vectors with boldface letters, eg x, y; the i'th elemet of a vector x is deoted with x i ad we use x j i where i < j to deote the vector (x i, x i+,, x j, x j ; x j is short form otatio for x j, ad x x A vector of radom variables is deoted by X X, ad similarly we dee X j i (X i, X i+,, X j, X j for i < j We use H( to deote the etropy of a discrete radom variable ad I( ; to deote the mutual iformatio betwee two radom variables, as deed i [2, Chapter 2] I( ; q deotes the mutual iformatio evaluated with a pmf q o the radom variable ad co R deotes the covex hull of the set R Fially, we recall the deitios of directed mutual iformatio ad causal coditioig (see also [5]: I(X Y Z I(X i ; Y i Y i, Z I(X Y Z Q(x y, z I(X i ; Y i Y i, Z i p(x i x i, y i, z i eitio : The ite-state broadcast chael is deed by the triplet X S, p(y, z, s x, s, Y Z S where X is the iput symbol, Y ad Z are the output symbols, S is the chael state at the ed of the previous symbol trasmissio ad S is the chael state at the ed of the curret symbol trasmissio S, X, Y ad Z are discrete alphabets of ite cardialities The pmf of a block of trasmissios is p(y, z, s, x s 0 p(y i, z i, s i, x i y i, z i, s i, x i, s 0 (a (b p(x i y i, z i, x i p(y i, z i, s i y i, z i, s i, x i, s 0 p(x i y i, z i, x i p(y i, z i, s i x i, s i, where s 0 is the iitial state Here, (a is because the trasmitter is oblivious of the chael states ad (b captures the fact that give S i, the symbols at time i are idepedet of the past eitio 2: The FSBC is called physically degraded if its pmf satises p(y i x i, y i, z i, s 0 p(y i x i, y i, s 0, (a p(z i x i, y i, z i, s 0 p(z i y i, z i, s 0 (b Coditio (a captures the ituitive otio of degradedess, amely that Z i is a degraded versio of Y i, thus it does ot add iformatio whe Y i is give Note that i the memoryless case this coditio is ot ecessary as, give X i, Y i is idepedet of the history Coditio (b follows from the stadard otio of degradedess, amely that Y i makes Z i idepedet of X i Note that coditio (b does ot elimiate memory Usig coditios (a ad (b i sceario SC2 we obtai (whe p(y, x s 0 > 0 p(z y, x, s 0 p(z, y, x s 0 p(y, x s 0 p(z i, y i, x i z i, y i, x i, s 0 p(y, x s 0 p(x i z i, y i, x i p(z i, y i z i, y i, x i, s 0 p(y, x s 0 (a p(x i y i, x i p(z i, y i z i, y i, x i, s 0 p(x i y i, x i p(y i y i, x i, s 0 (b p(y i y i, x i, s 0 p(z i z i, y i, x i, s 0 p(y i y i, x i, s 0 (c p(z i z i, y i, s 0 where (a is because A 2 is ope i SC2, (b follows from (a ad (c follows from (b We therefore coclude that whe ( holds, p(z y, x, s 0 p(z y, s 0 Hece, p(y, z x, s 0 p(y x, s 0 p(z y, s 0 (2 Note that Z is a degraded versio of Y but still depeds o the state sequece (ie degradedess does ot elimiate the memory A special case of the physically degraded FSBC occurs whe i (b we have p(z i x i, y i, z i, s 0 p(z i y i Hece, p(z y, s 0 p(z y p(z i y i (3 Equatio (3 is similar to the deitio of degradedess for the AVBC used i [8] Coditio (3 does ot costitute oly a

3 mathematical coveiece, but represets a physical sceario, as show i [6] eitio 3: The FSBC is called stochastically degraded if there exists a pmf p(z y such that p(z x, s 0 Y p(z, y x, s 0 Y p(y x, s 0 p(z i y i (4 eitio 4: (see [2, Sectio 46] The FSBC is called idecomposable if for every ɛ > 0 there exists N 0 (ɛ such that for all > N 0 (ɛ, p(s x, s 0 p(s x, s 0 < ɛ, for all s, x, ad iitial states s 0 ad s 0 eitio 5: A (R 0, R, R 2, determiistic code for the FSBC with feedback cosists of three message sets,, 2,, 2 R0, M, 2,, 2 R ad M 2, 2,, 2 R2, ad a collectio of mappigs (f i, g y, g z such that f i : M M 2 Z i Y i X (5 is the ecoder, ad g y : Y M, g z : Z M 2, are the decoders Here, is the set of commo messages ad M ad M 2 are the sets of private messages to Rx ad Rx 2, respectively Note that we assume o kowledge of the states at the trasmitter ad receivers eitio 6: The average probability of error of a code of blocklegth is give by max s0 S P e ( (s 0 where P ( e (s 0 Pr (g y (Y (, M or g z (Z (, M 2 s 0, ad the messages, M M ad M 2 M 2 are selected idepedetly ad uiformly Remark: I eitios 5 ad 6 we assume o receiver cooperatio However, sice i SC Z is also available at Rx, the for this sceario we modify eitios 5 ad 6 to iclude cooperatio by replacig g y (Y with g y (Y, Z eitio 7: A rate triplet (R 0, R, R 2 is called achievable for the FSBC if for every ɛ > 0 ad δ > 0 there exists a (ɛ, δ N such that > (ɛ, δ a (R 0 δ, R δ, R 2 (s 0 ɛ, s 0 S ca be costructed eitio 8: The capacity regio of the FSBC is the covex hull of all achievable rate triplets δ, code with P ( e III MAIN RESULTS AN ISCUSSION The mai results are stated i the followig theorems The proof of Theorem is outlied i Sectio IV The proof of Theorem 2 follows from similar argumets ad is thus omitted Theorem (capacity regio for SC: For the FSBC p(y, z, s x, s, let Q SC be the set of all distributios o ( U i X Y Z S that satisfy q(u, x, y, z, s s 0 : q(u, x, y, z, s s 0 p(u i u i, z i Q SC p(x i x i, z i, y i, u i p(y i, z i, s i x i, s i for all s 0 S, N ee the regio R SC as R SC co (R 0, R, R 2 : R 0 0, R 0, q Q SC R 2 0, R mi s 0 S I(X Z, Y U, s 0 q, R 0 + R 2 mi s 0 S I(U Z s 0 q For the geeral FSBC with feedback such that switches A, A 2 ad B are closed, the capacity regio is give by C SC fb lim RSC, (6 ad the limit exists The deitio of the limit of regios ca be foud, eg, i [5] Here, the auxiliary RV U represets the iformatio trasmitted to Rx 2 Whe feedback ad cooperatio are ot available from Rx 2 ad the chael is physically degraded we obtai the followig capacity regio: Theorem 2 (capacity regio for SC2: Let Q SC2 be the set of all distributios o (U X Y Z S that satisfy Q SC2 q(u, x, y, z, s s 0 : q(u, x, y, z, s s 0 p(u p(x i x i, y i, u p(y i, z i, s i x i, s i for all s 0 S ad U mi X Y, Z +, N, where p(y, z, s x, s is the physically degraded FSBC satisfyig Equatio (2 ee the regio R SC2 as R SC2 co (R 0, R, R 2 : R 0 0, R 0, q Q SC2 R 2 0, R mi s 0 S R 0 + R 2 mi s 0 S I(X Y U, s 0 q, I(U ; Z s 0 q For the geeral FSBC with feedback such that switch B is closed, the capacity regio is give by C SC2 fb lim RSC2, (7 ad the limit exists Sice the capacity of the broadcast chael depeds oly o the coditioal margials p(y x, s 0 ad p(z x, s 0 (see [2, Chapter 46], the capacity regio of the stochastically degraded FSBC for sceario SC2 is the same as the correspodig physically degraded FSBC: Corollary : For the stochastically degraded FSBC of efiitio 3, the capacity regio is give by Theorem 2 with the appropriate p(z y satisfyig (4 The ite-state Markov BC (FSMBC is deed by p(y, z, s x, s p(s s p(y, z x, s Whe the Markov chai is homogeeous, irreducible ad aperiodic the the FSMBC is a idecomposable chael,

4 deoted H-FSMBC For the H-FSMBC we have the followig corollary: Corollary 2: For the H-FSMBC of sceario SC, the capacity regio is give by Theorem where, i the deitio of R SC, the coditioig o s 0 ad s 0 are omitted from the mutual iformatio expressios 2 For the physically degraded H-FSMBC of sceario SC2, the capacity regio is obtaied from Theorem 2 where, i the deitio of R SC2, s 0 ad s 0 are omitted from the mutual iformatio expressios 3 For the stochastically degraded H-FSMBC of sceario SC2, the capacity regio is obtaied from Corollary where, i the deitio of R SC2, s 0 ad s 0 are omitted from the mutual iformatio expressios Proof outlie: Loosely speakig, the corollary is true sice for large eough the effect of the iitial state fades away Therefore, the maximum over all s 0 S equals the miimum Hece, for all iitial states the limits for are the same iscussio We make the followig observatios: We ote that i both scearios we actually have physically degraded situatios I SC the physical degradedess is due to the cooperatio lik (switch A ad the Markov chai X (Y, Z Z I SC2 the chael is physically degraded by deitio of the sceario Therefore, i the derivatio we eed to cosider oly the two private messages case as the commo message ca be icorporated by splittig the rate to Rx 2 ito private ad commo rates, as i [2, Theorem 464] 2 Sice i SC the chael is effectively a physically degraded broadcast chael, a superpositio codetree achieves capacity Iterpretig superpositio codig i terms of cloud ceters (U ad cloud elemets (X U we ote that i SC the cloud ceters are i fact codetrees o which cloud elemets, which are also codetrees, are superimposed 3 It was show i [7] that feedback does ot icrease the capacity of the physically degraded memoryless BC The ituitio is as follows: i the physically degraded memoryless BC, Y is a memoryless trasformatio of X ad Z is a memoryless trasformatio of Y Now, as feedback does ot help the memoryless poit-to-poit chael it caot help the cascade of two such chaels As explaied earlier, whe the chael has memory, feedback ca icrease its capacity Feedback also helps the stochastically degraded BC [0] Therefore, feedback ca icrease the capacity of the FSBC 4 I SC, both receivers decode the cloud ceter based o Z, as Z is available also at Rx through the cooperatio lik Rx ow proceeds to decode the cloud elemet However, sice the chael is ot physically degraded, after decodig the cloud ceter usig Z, Rx uses both (Z, Y rather tha oly Y to decode the cloud elemet This is because p(z x, y, s 0 p(z y, s 0 5 A superpositio codebook structure achieves capacity for both SC ad SC2 This itroduces a structural costrait whe optimizig the codebook for achievig the maximal rate pairs Note, however, that for SC we are ot able to establish a cardiality boud for the auxiliary RV 6 I SC2 feedback is available oly from Rx However, due to the physically degraded structure, this helps both receivers: degradedess costrais the sum-rate to be bouded by the sum-rate at Rx Whe this sum-rate is icreased, the for the same rate R it is possible to obtai a higher rate R 2 The maximum rate to Rx 2, though, remais the same IV PROOF OUTLINE FOR THEOREM (SC A Achievability The achievability proof cosists of the followig steps: Fix a collectio of probability distributios p(ui u i, z i, p(x i u i, y i, z i, x i Q, Q 2 Usig maximum-likelihood decodig at Rx 2 accordig to arg max p(z i z i, u i, s 0 m 2 S s 0 S we coclude that a positive error expoet for decodig M 2 ca be obtaied as log as R 2 mi s0 S I(U Z s 0 log 2 S I2 (Q, Q 2 As Rx receives Z through the cooperatio lik, the also for decodig M 2 at Rx the error expoet is positive Usig maximum-likelihood decodig at Rx accordig to arg max p(y i, z i x i, y i, z i, s 0 m S s 0 S we coclude that a positive error expoet for decodig M at Rx, give that M 2 was correctly decoded, ca be achieved as log as R mi s0 S I(X Y, Z U, s 0 log 2 S I (Q, Q 2 Next, for a xed ad some iteger b let 0 b Costruct distributios by takig the product of the basic distributio for a block of symbols b times: Q 2 (x 0 u 0, y 0, z 0 b ( p b Q (u 0 z 0 b ( p b x (b +i x (b +i (b +, y (b +i (b +, z (b +i (b +, u (b +i (b + u (b +i z (b +i (b +, u (b +i (b + Now, takig b large eough results i a average probability of error that is arbitrarily small, hece (I (Q, Q 2, I 2 (Q, Q 2 is achievable Fially, for λ > 0 dee C fb,sc (λ: Cfb,SC(λ F (λ, Q, Q 2 max Q (u z Q 2 (x u,z,y mi s 0 S F (λ, Q, Q 2, (8 I(U Z s 0 ( + λ +λ mi s 0 S I(X Z, Y U, s 0 log S We show that Cfb,SC (λ is sup-additive: C fb,sc (λ lim Cfb,SC (λ sup Cfb,SC (λ Therefore, the boudary of the achievable regio ca be writte as ( R 2 (R C fb,sc (λ λr (9 if 0 λ

5 B Coverse Theorem 3: If for some λ > 0, R 2 + λr > C fb,sc(λ + ɛ, the there exist iitial states s 0, s 0 S for which P ( e2 (s 0R 2 + λp ( + log S e (s 0R > ɛ ( + λ (0 The implicatio of (0, as explaied i [], is that for large eough the probability of error caot be made arbitrarily small, outside the regio whose boudary is give by (9 Proof: Recall that P ( e2 (s 0 ad P ( e (s 0 deote probabilities of error for iitial state s 0, whe the decoders are igorat of the iitial state From Fao's iequality we have that for iitial state s 0 H(M 2 Z, s 0 P ( e2 (s 0R 2 + (a H(M Z, Y, s 0 P ( e (s 0R + (b eote with s 0, the iitial state that maximizes H(M 2 Z, s 0 ad with s 0, the iitial state that maximizes H(M Z, Y, s 0 Now, ote that H(M2 s 0 H(M 2 Z, s 0 mi I(M 2; Z s 0 mi s 0 S s 0 S R 2 max H(M 2 Z, s 0, (2 s 0 S mi I(M ; Z, Y M 2, s 0 s 0 S We ext have I(M 2 ; Z s 0 R max s 0 S H(M Y, Z, M 2, s 0 R max s 0 S H(M Y, Z, s 0 (3 ( H(Zi Z i, s 0 H(Z i Z i, M 2, s 0 ( H(Zi Z i, s 0 H(Z i Z i, U i, s 0 I(U Z s 0 where U i (M 2, Z i, i, 2,, Note that U i (U i, Z i Y i ad also U i Z i, Y i, s 0 X i Z i, Y i, s 0 Y i, Z i Z i, Y i, s 0 We also have that I(M ; Y, Z M 2, s 0 ( H(Zi, Y i Z i, Y i, M 2, s 0 H(Z i, Y i Z i, Y i, M, M 2, s 0 ( H(Zi, Y i Z i, Y i, U i, s 0 H(Z i, Y i Z i, Y i, X i, U i, s 0 I(X Z, Y U, s 0 Combiig the above we have that for our choice of U : mi I(M 2; Z s 0 + λ mi I(M ; Z, Y M 2, s s 0 0 S s 0 S mi s 0 S I(U Z s 0 + λ mi s 0 S I(X Z, Y U, s 0 C fb,sc(λ + ( + λ log S C fb,sc(λ + ( + λ log S, (4 sice Cfb,SC (λ is obtaied by maximizig over all joit distributios Q (u z Q 2 (x u, z, y ad also because Cfb,SC (λ is sup-additive Pluggig (2 ad (3 ito (4 yields R 2 H(M 2 Z, s 0, + λ(r H(M Z, Y, s 0, C fb,sc(λ + ( + λ log S H(M 2 Z, s 0, + λh(m Z, Y, s 0, + ( + λ log S ( R 2 + λr C fb,sc(λ > ɛ Combied with Fao's iequalities (, we obtai (0, which meas that at least oe of the states s 0,, s 0, results i a probability of error (at the respective receiver that is bouded away from zero, completig the proof of the coverse V CONCLUSIONS We have derived the capacity regio of a two-user FSBC with feedback uder two differet assumptios about the ature of the feedback ad user cooperatio A importat property of the rst sceario is that the chael output at oe receiver is also available to the other receiver through cooperatio Whe this cooperatio is ot possible, the the rst user's receiver caot use iformatio about received sigal at the secod receiver to decode its ow message This implies that a superpositio codebook is ot ecessarily optimal, eve if the chael is physically degraded I our secod model, as decoder 2's chael output is ot available at the trasmitter ad the chael is physically degraded, a superpositio codebook i which the cloud ceters are geerated without feedback is optimal A importat property of feedback i multi-user scearios is that feedback from oe user ca help other users as well REFERENCES [] B McMilla The Basic Theorems of Iformatio Theory The Aals of Mathematical Statistics, Vol-24(2:96 29, 953 [2] R G Gallager Iformatio Theory ad Reliable Commuicatio Joh Wiley ad Sos Ic, 968 [3] T Holliday, A Goldsmith, ad P Gly O Etropy ad Lyapuov Expoets for Fiite State Chaels IEEE Tr o IT, IT-52(8, 2006 [4] H Permuter, P Cuff, B Va Roy, ad T Weissma Capacity of the Trapdoor Chael with Feedback Submitted to IEEE Tras o IT [5] H Permuter ad T Weissma Capacity Regio of the Fiite-State Multiple Access Chael with ad without Feedback Submitted to the IEEE Tras Iform Theory, 2007 [6] R abora ad A Goldsmith The Capacity of the egraded Fiite- State Broadcast Chael Accepted to the Iformatio Theory Workshop (ITW, 2008 [7] A El-Gamal The Feedback Capacity of egraded Broadcast Chaels IEEE Tras Iform Theory, IT-24(3:379 38, 978 [8] Y Steiberg Codig for the egraded Broadcast Chael With Radom Parameters, With Causal ad Nocausal Side Iformatio IEEE Tras Iform Theory, IT-5(8: , 2005 [9] A Wishtok ad Y Steiberg The Arbitrarily Varyig egraded Broadcast Chael with States Kow at the Ecoder Iteratioal Symp o Iform Theory (ISIT 2006, Seattle, WA, pp [0] L H Ozarow Codig ad Capacity for Additive White Gaussia Noise Multi-User Chaels with Feedback Ph dissertatio, Mass Ist Tech, Cambridge, MA, May 979 [] R G Gallager Capacity ad Codig for egraded Broadcast Chaels Problemy Peredachi Iformatsii, vol 0(3:3 4, 974 [2] T M Cover ad J Thomas Elemets of Iformatio Theory Joh Wiley ad Sos Ic, 99