Lossy Coding of Correlated Sources Over a Multiple Access Channel: Necessary Conditions and Separation Results

Transcription

1 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 64, NO. 9, SETEMBER Lossy Codig of Correlated Sources Over a Multiple Access Chael: Necessary Coditios ad Separatio Results Başak Güler, Member, IEEE, Deiz Güdüz, Seior Member, IEEE, ad Ayli Yeer, Fellow, IEEE Abstract Lossy codig of correlated sources over a multiple access chael MAC is studied. First, a joit source-chael codig scheme is preseted whe the decoder has correlated side iformatio. Next, the optimality of separate source ad chael codig that emerges from the availability of a commo observatio at the ecoders or side iformatio at the ecoders ad the decoder is ivestigated. It is show that separatio is optimal whe the ecoders have access to a commo observatio whose lossless recovery is required at the decoder, ad the two sources are idepedet coditioed o this commo observatio. Optimality of separatio is also proved whe the ecoder ad the decoder have access to shared side iformatio coditioed o which the two sources are idepedet. These separatio results obtaied i the presece of side iformatio are the utilized to provide a set of ecessary coditios for the trasmissio of correlated sources over a MAC without side iformatio. Fially, by specializig the obtaied ecessary coditios to the trasmissio of biary ad Gaussia sources over a MAC, it is show that they ca potetially be tighter tha the existig results i the literature, providig a ovel coverse for this fudametal problem. Idex Terms Commo iformatio, coditioal idepedece, hybrid codig, joit source ad chael codig, multiple access chael, rate-distortio theory, separatio theorem. I. INTRODUCTION THIS paper cosiders the lossy codig of correlated discrete memoryless DM sources over a DM multiple access chael MAC. Separate source ad chael codig is kow to be suboptimal for this setup i geeral, eve Mauscript received November 30, 06; revised August 6, 07 ad March 6, 08; accepted May 9, 08. Date of publicatio Jue 7, 08; date of curret versio August 6, 08. This work was supported i part by the U.S. Army Research Laboratory through the Network Sciece Collaborative Techology Alliace uder Agreemet Number W9NF ad i part by the Europea Research Coucil through the Startig Grat roject BEACON uder roject The material i this paper was preseted i part at the 06 IEEE Iteratioal Symposium o Iformatio Theory ISIT 6 ad the 07 IEEE Iteratioal Symposium o Iformatio Theory ISIT 7. B. Güler was with the Departmet of Electrical Egieerig, The esylvaia State Uiversity, Uiversity ark, A 680 USA. She is ow with the Departmet of Electrical Egieerig, Uiversity of Souther Califoria, Los Ageles, CA USA bguler@usc.edu. D. Güdüz is with the Departmet of Electrical ad Electroic Egieerig, Imperial College Lodo, Lodo SW7 AZ, U.K. d.guduz@imperial.ac.uk. A. Yeer is with the Departmet of Electrical Egieerig, The esylvaia State Uiversity, Uiversity ark, A 680 USA yeer@ee.psu.edu. Commuicated by J. Che, Associate Editor for Shao Theory. Color versios of oe or more of the figures i this paper are available olie at Digital Object Idetifier 0.09/TIT whe the lossless recostructio of the sources is required []. This is i cotrast to the poit-to-poit sceario for which the separatio of source ad chael codig is optimal, also kow as the separatio theorem []. The characterizatio of the achievable distortio regio whe trasmittig correlated sources over a MAC is oe of the fudametal ope problems i etwork iformatio theory, solved oly for some special cases. This problem is also related to aother log-stadig ope problem, amely the multi-termial lossy source-codig problem, which refers to the sceario whe the uderlyig MAC cosists of two orthogoal fiite-capacity error-free liks. Despite the lack of a geeral sigle-letter characterizatio for the multi-termial source codig problem, separate source ad chael codig is optimal whe the uderlyig MAC is orthogoal [3]. Separatio is also optimal whe oe of the sources is shared betwee the two ecoders [4], or for the lossless case, whe the decoder has access to side iformatio coditioed o which the two sources are idepedet [5]. However, due to the lack of a geeral separatio result, the achievable distortio regio is ukow eve i scearios for which the correspodig source codig problem ca be solved. I the absece of sigle-letter ecessary ad sufficiet coditios, the goal is to obtai computable ier ad outer bouds. A fairly geeral joit source-chael codig scheme was itroduced i [6] by leveragig hybrid codig. This scheme subsumes most other kow codig schemes. A ovel outer boud was preseted i [7] for the Gaussia settig, which uses the fact that the correlatio amog chael iputs is limited by the correlatio available amog source sequeces. Other bouds were proposed i [8] ad [9] ad more recetly i [0] ad []. Optimality of source-chael separatio was studied i [5] ad [], ad the optimality of ucoded trasmissio was ivestigated for Gaussia sources over multitermial Gaussia chaels i [3]. This paper studies the achievable distortio regio for sedig correlated sources over a MAC. I the first part of the paper, it is assumed that the ecoders ad/or the decoder may have access to side iformatio correlated with the sources see Fig.. Iitially, a joit source-chael codig scheme is proposed whe side iformatio is available oly at the decoder. The, we ivestigate separatio theorems that emerge from the availability of a commo observatio at the ecoders, or from the availability of side iformatio at the IEEE. ersoal use is permitted, but republicatio/redistributio requires IEEE permissio. See for more iformatio.

2 608 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 64, NO. 9, SETEMBER 08 Fig.. Commuicatio of correlated sources over a MAC. ecoders ad the decoder. I doig so, we first focus o the sceario i which the ecoders share a commo observatio coditioed o which the two sources are idepedet. For this setup, we show that separatio is optimal whe the decoder is required to recover the commo observatio losslessly, but ca tolerate some distortio for the parts kow oly at a sigle ecoder. Correspodig ecessary ad sufficiet coditios are idetified for the optimality of separatio. Next, we cosider the sceario i which the ecoders ad the decoder have access to shared side iformatio, ad show that separatio is agai optimal if the two sources are coditioally idepedet give the side iformatio. I the secod part of the paper, we leverage the separatio theorems derived i the first part to obtai a ew set of ecessary coditios for the achievability of a distortio pair whe trasmittig correlated sources over a MAC without ay side iformatio. I particular, we obtai our computable ecessary coditios by providig particular side iformatio sequeces to the ecoders ad the decoder to iduce the optimality of separatio. Based o the results of the first part, this ca be achieved whe the two sources are coditioally idepedet give the side iformatio. Optimality of separatio coditioed o the provided side iformatio allows us to characterize the correspodig ecessary coditios explicitly. Coditioal idepedece iducig side iformatio sequeces have previously bee used to obtai coverse results i some multi-termial source codig problems [4], [5]. I this paper, they are used to obtai coverse results i a multi-termial joit source-chael codig problem. The ecessary coditios are the specialized to the case of bivariate Gaussia sources over a Gaussia MAC as well as doubly symmetric biary sources DSBS over a Gaussia MAC. By providig comparisos betwee the ew ecessary coditios ad the kow bouds i the literature, we show that the proposed techique ca potetially provide tighter coverse bouds tha the previous results i the literature. I the remaider of the paper, X represets a radom variable, ad x is its realizatio. X X,...,X is a radom vector of legth, adx x,...,x deotes its realizatio. X is a set with cardiality X. E[X] is the expected value ad varx is the variace of X. II. SYSTEM MODEL We cosider the trasmissio of DM sources S ad S over a DM MAC as illustrated i Fig.. Ecoder observes S S,...,S, whereas ecoder observes S S,...,S. If switch SW i Fig. is closed, the two ecoders also have access to a commo observatio Z correlated with S ad S. Ecoders ad map their observatios to the chael iputs X ad X, respectively. The chael is characterized by the coditioal distributio py x, x. If switch SW i Fig. is closed, the decoder has access to side iformatio Z. Upo observig the chael output Y ad side iformatio Z wheever it is available, the decoder costructs the estimates Ŝ, Ŝ,adẐ. Correspodig average distortio values for the source sequece Ŝ j, j,, is give by j E[d j S ji, Ŝ ji ], i where d j, < is the distortio measure for source S j. A distortio pair D, D is achievable for the source pair S, S ad chael py x, x if there exists a sequece of ecodig ad decodig fuctios such that lim sup j D j, j,, ad Z Ẑ 0as whe at least oe of the switches is closed. Radom variables S, S, Z, X, X, Y, Ŝ, Ŝ, Ẑ are defied over the correspodig alphabets S, S, Z, X, X, Y, Sˆ, Sˆ, Z. ˆ Note that, whe switch SW is closed, error probability i decodig Z becomes irrelevat sice it is readily available at the decoder, ad serves as side iformatio. Throughout the paper, we use the followig defiitios extesively. Defiitio. Coditioal rate distortio fuctio [6] Give correlated radom variables S ad U, defie the miimum average distortio for S give U as [4], [7]: ES U if E[dS, f U], 3 f :U S ˆ where the miimum is over all fuctios f from U to the recostructio alphabet S. ˆ The, the coditioal rate distortio fuctio for source S whe correlated side iformatio Z is shared betwee the ecoder ad the decoder is give by, R S Z D mi I S; U Z, 4 pu s,z: ES U,Z D where the miimum is over all coditioal distributios pu s, z such that the miimum average distortio for S give U ad Z is less tha or equal to D. Defiitio. Gács-Körer commo iformatio [8] Defie the fuctio f j : S j {,...,k} for j,, with the largest iteger k such that f j S j u 0 > 0 for u 0 {,...,k}, j,, ad f S f S. The, U 0 f S f S is defied as the commo part betwee S ad S, ad the Gács-Körer commo iformatio is give by C GK S, S H U 0. 5 Defiitio 3. Wyer s commo iformatio [9] Wyer s commo iformatio betwee S ad S is defied as, C W S, S mi I S, S ; V. 6 pv s,s S V S

3 GÜLER et al.: LOSSY CODING OF CORRELATED SOURCES OVER A MAC 6083 III. JOINT SOURCE-CHANNEL CODING WITH DECODER SIDE INFORMATION We first assume that oly SW is closed i Fig., ad preset a geeral achievable scheme for the lossy codig of correlated sources i the presece of decoder side iformatio. Theorem. Whe sedig correlated DM sources S ad S over a DM MAC with py x, x ad decoder side iformatio Z, distortio pair D, D is achievable if there exists a joit distributio pu, u, s, s, z pu s pu s ps, s, z, ad fuctios x j u j, s j,g j u, u, y, z for j,, such that I U ; S U, Z <I U ; Y U, Z 7 I U ; S U, Z <I U ; Y U, Z 8 I U, U ; S, S Z <I U, U ; Y Z 9 ad E[d j S j, g j U, U, Y, Z] D j for j,. roof. Our achievable scheme builds upo the hybrid codig framework of [6], by geeralizig it to the case with decoder side iformatio. The detailed proof is available i Appedix A. IV. SEARATION THEOREMS We ow focus o the coditios uder which separatio is optimal for lossy codig of correlated sources over a MAC. For the remaider of this sectio, we assume that S ad S are idepedet coditioed o Z, i.e., the Markov coditio S Z S holds. Separatio i the resece of Commo Observatio: Here, we assume that oly switch SW i Fig. is closed, ad show the optimality of separatio if the lossless recostructio of the commo observatio Z is required. Theorem. Cosider the commuicatio of correlated sources S,S, ad Z, where Z is observed by both ecoders. If S Z S holds, the separatio is optimal, ad D, D is achievable if R S Z D <IX ; Y X, W 0 R S ZD <IX ; Y X, W R S Z D + R S ZD <IX, X ; Y W H Z + R S Z D + R S ZD <IX, X ; Y 3 for some px, x, y,w py x, x px wpx wpw. Coversely, if a distortio pair D, D is achievable, the 0-3 must hold with < replaced with. roof. We provide a detailed proof i Appedix B. Corollary. A special case of Theorem is the trasmissio of two correlated sources over a MAC with oe distortio criterio, whe oe source is available at both ecoders as cosidered i [4], which correspods to S beig a costat i Theorem. A related sceario is whe the two sources share a commo part i the sese of of Gács-Körer. The followig result states that, i accordace with Theorem, if the two sources are idepedet whe coditioed o the Gács-Körer commo part, the separate source ad chael codig is optimal if lossless recostructio of the commo part is required. Corollary. Cosider the trasmissio of correlated sources S ad S with a commo part U 0 f S f S from Defiitio. If S U 0 S ad the commo part U 0 of S ad S is to be recovered losslessly, the, separate source ad chael codig is optimal. roof. From Defiitio, the two ecoders ca separately recostruct U 0. The result the follows by lettig Z U 0 i Theorem. Separatio i the resece of Shared Ecoder-Decoder Side Iformatio: We ext assume that both switches i Fig. are closed, ad show the optimality of separatio if the two sources are idepedet give the side iformatio that is shared betwee the ecoders ad the decoder. Theorem 3. Cosider commuicatio of two correlated sources S ad S with side iformatio Z shared betwee the ecoders ad the decoder. If S Z S holds, the separatio is optimal, ad D, D is achievable if R S ZD <IX ; Y X, Q 4 R S Z D <IX ; Y X, Q 5 R S ZD + R S Z D <IX, X ; Y Q 6 for some px, x, y, q py x, x px qpx qpq. Coversely, for ay achievable D, D pair, 4-6 must hold with < replaced with. roof. See Appedix C. Whe side iformatio Z is available oly at the decoder, i.e., whe oly switch SW is closed, separatio is kow to be optimal for the lossless trasmissio of sources S ad S wheever S Z S [5]. I light of Theorem 3, we show that a similar result holds for the lossy case wheever the Wyer-Ziv rate distortio fuctio of each source is equal to its coditioal rate distortio fuctio. Corollary 3. Cosider the commuicatio of correlated sources S ad S with decoder oly side iformatio Z. If where R S j Z D j R WZ S j Z D j, 7 R WZ S j Z D j mi I S j ; U j Z for j,, pu j s j,gu j,z: E[d j S j,gu j,z] D j U j S j Z is the Wyer-Ziv rate distortio fuctio of S j with decoderoly side iformatio Z [0], ad S Z S form a Markov chai, the separatio is optimal, with the ecessary ad sufficiet coditios i 4-6. roof. Corollary 3 follows from the fact that wheever 7 holds, coditioal rate distortio fuctios i Theorem 3 are achievable by relyig o decoder side iformatio oly. We ote that Gaussia sources are a example for 7. Remark. We would like to ote that the optimality/suboptimality of separatio for the case of decoder-oly side

4 6084 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 64, NO. 9, SETEMBER 08 for some Q for which X Q X form a Markov chai, where Fig.. Correlated sources over a MAC. iformatio coditioed o which the two sources are idepedet is ope i geeral. I additio to the settig i Corollary 3, the optimality of separatio holds also for lossless recostructio [5]. Lastly, we cosider the trasmissibility of correlated sources with a commo part whe the commo part is available at the decoder. The followig result states that if the two sources are idepedet whe coditioed o the Gács-Körer commo part, separatio is agai optimal if the decoder has access to the commo part. Corollary 4. Cosider the trasmissio of sources S ad S with a commo part U 0 f S f S from Defiitio. The, separatio is optimal if S U 0 S ad the commo part U 0 is available at the decoder. roof. Sice both ecoders ca extract U 0 idividually, each source ca achieve the correspodig coditioal rate distortio fuctio. Corollary 4 the follows from Theorem 3 by lettig Z U 0. I the followig, we leverage these separatio results to obtai ecessary coditios for the lossy codig of correlated sources over a MAC without side iformatio. V. NECESSARY CONDITIONS FOR TRANSMITTING CORRELATED SOURCES OVER A MAC We cosider i this sectio the lossy codig of correlated sources over a MAC whe both switches i Fig. are ope; see Fig.. We provide ecessary coditios for the achievability of a distortio pair D, D usig our results from Sectio IV. This will be achieved by providig correlated side iformatio to the ecoders ad the decoder, coditioed o which the two sources are idepedet. From Theorem 3, separatio is optimal i this settig, ad the correspodig ecessary ad sufficiet coditios for the achievability of a distortio pair serve as ecessary coditios for the origial problem. Correspodig ecessary coditios are preseted i Theorem 4 below. Theorem 4. Cosider the commuicatio of correlated sources S ad S over a MAC. If a distortio pair D, D is achievable, the for every Z satisfyig the Markov coditio S Z S, we have R S Z D I X ; Y X, Q, 8 R S ZD I X ; Y X, Q, 9 R S Z D + R S ZD I X, X ; Y Q, 0 R S S D, D I X, X ; Y, R S S D, D mi I S, S ; Ŝ, Ŝ pŝ,ŝ s,s E[d S,Ŝ ] D E[d S,Ŝ ] D is the rate distortio fuctio of the joit source S, S with target distortios D ad D for sources S ad S, respectively. roof. For ay Z that satisfies the Markov coditio S Z S, we cosider the geie-aided settig i which Z is provided to the ecoders ad the decoder. The, we obtai the settig i Theorem 3. Coditios 8-0 follow from Theorem 3, whereas coditio follows from the cut-set boud. A. Correlated Sources Over a Gaussia MAC I this sectio, we focus o a memoryless MAC with additive Gaussia oise: Y X + X + N, where N is a stadard Gaussia radom variable. We impose the iput power costraits i E[X ji ], j,. I the followig, we specialize the ecessary coditios of Theorem 4 to a Gaussia MAC. Corollary 5. If a distortio pair D, D is achievable for sources S, S over the Gaussia MAC i, the for every Z that forms a Markov chai S Z S, we have R S ZD + R S Z D log + β + β 3 R S S D, D log + + β β 4 for some 0 β,β. roof. The corollary follows by cosiderig oly 0-, ad from the fact that the right had sides RHSs of these iequalities are maximized by Gaussia Q, X, ad X []. Gaussia Sources Over a Gaussia MAC: This sectio studies the ecessary coditios for trasmittig correlated Gaussia sources over a Gaussia MAC. Cosider a bivariate Gaussia source S, S such that S 0 ρ N,, 5 S 0 ρ trasmitted over the DM Gaussia MAC i, uder the squared error distortio measures d j S j, Ŝ j S j Ŝ j for j,. For this setup, various otable results exist, each presetig differet sets of ecessary coditios. The followig ecessary coditio is obtaied i [7, Theorem IV.]: R S S D, D log + + ρ. 6

5 GÜLER et al.: LOSSY CODING OF CORRELATED SOURCES OVER A MAC 6085 Aother set of ecessary coditios is proposed i [8, Theorem ]. By substitutig σz σ σ ade E i [8, Theorem ], these coditios ca be stated as follows: ρ ˆρ l, k,, 7 D k l R S S D, D +ˆρ, 8 for some 0 ˆρ ρ. Other sets of ecessary coditios have recetly bee preseted i [0, Theorem ], [3, ropositio ], ad [, Theorems ad 4], all icorporatig various auxiliary radom variables. It is ot possible i geeral to compare Theorem 4 over the full set of coditios preseted i these results, sice this ivolves optimizatio of auxiliary radom variables ad a large umber of parameters. For this reaso, here we compare Corollary 5 with 6, 7-8, alog with the coditios from [0, Corollary.], which is a relaxed versio of [0, Theorem ]. Note that Corollary 5 is also a weaker versio of Theorem 4, where, for fairess, the first two sigle rate coditios are removed as i [0, Corollary.]. The set of ecessary coditios from [0, Corollary.] ca be stated as: R S S D, D log + ρ ρ log + β + β 9 R S S D, D log + + β β 30 for some 0 β,β. For the ecessary coditios i Corollary 5, we let Z be the commo part of S, S with respect to Wyer s commo iformatio from 6. The commo part ca be characterized as follows [, ropositio ]. Let Z, N,adN be stadard radom variables. The, S,adS ca be expressed as S i ρ Z + ρ N i, i,, 3 where I S, S ; Z +ρ log ρ ad I S, S ; Z > +ρ log ρ for all S Z S with Z Z. The rate distortio fuctio for S i with ecoder ad decoder side iformatio Z is [3]: { R Si ZD i ρ log D i if 0 < D i < ρ 0 if D i ρ for i,. We also have, from [7] ad [4], that, R S S D, D log mid,d log+ ρ D D log+ ρ D D ρ D D where log + x max{0, logx}, ad { D D, D : 0 D ρ, if D, D D 3 if D, D D if D, D D 3 33 Fig. 3. a Regios D, D,adD 3. b artitioed distortio regios for D, D. D ρ + ρ D or ρ < D, D ρ + ρ D, D D ρ } ρ 34 { D D, D : 0 D ρ, 0 D } <ρ D 35 D { D 3 D, D : 0 D ρ, ρ D D < ρ + ρ D or D ρ < D, D ρ ρ < D < ρ +ρ D }. 36 Fig. 3a illustrates the regios D, D,adD 3 as i [7]. By aalyzig the correspodig expressios from Corollary 5, 6, 7-8, ad 9-30, the ext propositio shows that there exist D, D values for which Corollary 5 is tighter; that is, while other results caot make ay judgemet o the achievability of such D, D pairs, they are show ot to be achievable thaks to Corollary 5. ropositio. There exist distortio pairs that are icluded i the outer bouds of [7, Theorem IV.], [8, Theorem ], ad [0, Corollary.], but ot i the outer boud of Corollary 5.

6 6086 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 64, NO. 9, SETEMBER 08 roof. The details are give i Appedix D. A graphical illustratio of the bouds from Corollary 5, [7, Theorem IV.], ad [0, Corollary.] ca be provided as follows. Defie r β,β log + + β β, 37 r β,β log + β + β, 38 ad cosider the regio R {R, R : R r β,β, R r β,β }. 0 β,β 39 The ecessary coditios i Corollary 5 state that, if a D, D pair is achievable, the RS S D, D, R S Z D + R S ZD R. 40 The ecessary coditios i 9-30 state that, if a D, D pair is achievable, the R S S D, D, R S S D, D log + ρ R. ρ 4 Let D 0.45 < ρ. Cosider first Regio B, forwhich D ρ ad ρ D ρ D D.ForaD, D pair i Regio B, i.e.,d 0.45 ad ρ D ρ D D, we have from 3 ad 33 that RS S D, D, R S ZD + R S Z D ρ log, D D log ρ. 4 D The R S S D, D, R S Z D + R S ZD pairs obtaied from 4 for icreasig D values withi Regio B are illustrated with a gree + sig i Fig. 4a. The regio R from 39 is the regio shaded i blue i the same figure. Wheever a poit from 4 falls outside of R, we coclude that the correspodig D, D pair is ot achievable accordig to Corollary 5. We also evaluate R S S D, D, R S S D, D log + ρ ρ log, D D ρ log ρ D D 43 for poits 0.45, D i Regio B, usig 33. The poits correspodig to 43 for differet D values are marked with a dark blue * i Fig. 4a. Wheever a poit from 43 is ot cotaied withi R, the the correspodig D, D pair is ot achievable accordig to Next, we cosider D, D pairs from Regio D, forwhich D ρ ad ρ D D D ρ +ρ D.Weevaluate RS S D, D, R S Z D + R S ZD ρ log+ D D ρ D D, ρ log D 44 Fig. 4. Compariso of the ecessary coditios from Corollary 5 with the ecessary coditios from 6 ad 9-30, respectively, for, ρ 0.5, ad a D 0.45, b D 0.6. from The values obtaied for D 0.45 ad D ρ D D, ρ + ρ D are marked with a purple + i Fig. 4a. Similarly, from 33, for D, D Regio D, R S S D, D, R S S D, D log + ρ ρ ρ log+ D D ρ D D, ρ log+ D D ρ D D. 45 Correspodig poits for D 0.45 ad icreasig D values i Regio D are illustrated with a red x markig i Fig. 4a. Fially, we cosider D, D Regio G, whered ρ, ρ + ρ D D, ad RS S D, D, R S Z D + R S ZD log, D log ρ. 46 D

7 GÜLER et al.: LOSSY CODING OF CORRELATED SOURCES OVER A MAC 6087 Correspodig poits are marked with a pik + i Fig. 4a. Note that sice 46 depeds oly o D, these poits appear as a sigle poit. We also evaluate R S S D, D, R S S D, D log + ρ ρ log, D log ρ 47 D + ρ for ρ + ρ D D from 33. This is marked with a black * i Fig. 4a. Sice 47 also depeds oly o D, they appear as a sigle poit. Oe ca observe from 4-43, as well as from ad 46-47, that the poits that share the same value o the horizotal axis i Fig. 4a correspod to the same D, D pairs, as the first terms of both 4-43 ad as well as are equal. Lastly, we illustrate the RHS of 6 with a straight lie i Fig. 4a. The poits o the RHS of this lie correspod to D, D pairs that are ot achievable accordig to 6, sice for these poits oe has R S S D, D > log + + ρ. 48 I order to compare the three bouds, we ivestigate the D, D pairs that caot be achieved by Corollary 5, 6, ad 9-30, respectively. From Fig. 4a, we fid that whe D 0.45, some D, D pairs i Regios G ad D from Fig. 3b satisfy both 6 ad 9-30, but ot Corollary 5, as ca be observed from the pik ad purple poits marked with the + sig that are o the left had side LHS of the straight lie, but outside of R. Therefore, we ca coclude that there exist distortio pairs for which Corollary 5 provides tighter coditios tha both 6 ad 9-30 i Regios G ad D. We also compare the correspodig bouds whe D 0.6 i Fig. 4b. From the gree poits marked with the + sig that are o the LHS of the straight lie but are outside of R, we observe that there exist distortio pairs i Regio B for which Corollary 5 provides tighter coditios tha both 6 ad We ote, however, that Corollary 5 is ot ecessarily strictly tighter for all D, D pairs. The ext propositio states that there exist D, D pairs for which 6 is tighter tha Corollary 5. ropositio. There exist distortio pairs that are i the outer boud of Corollary 5, but ot i the outer boud of [7, Theorem IV.]. roof. The details are available i Appedix E. Biary Sources Over a Gaussia MAC: We ext study the trasmissio of a doubly symmetric biary source DSBS over a Gaussia MAC. Cosider a DSBS with joit distributio ps s, S s α s s + α s s, 49 a memoryless Gaussia MAC from, ad Hammig distortio d j S j, Ŝ j S j Ŝ j where Sˆ j S j {0, } for j,. For the coditios i Corollary 5, we choose the variable Z as illustrated i Fig. 5a. The the joit distributio for Fig. 5. a Z-chael structure. b ps i, Z for i,. S i, Z is as give i Fig. 5b for i,. Note that Z forms a Z-chael both with S ad S while satisfyig S Z S. Usig the coditioal rate-distortio fuctio for the Z-chael settig from [5], oe ca evaluate Corollary 5. We compare Corollary 5 first with the set of ecessary coditios from [7, Remark IV.], R S S D, D log + + ρ max, 50 where R S S D, D is as i [6, Theorem ], ad ρ max is the Hirschfield-Gebeli-Réyi maximal correlatio for DSBS give by [7]: ρ max α + α. 5 We ext cosider the ecessary coditios from [0, Corollary.], R S S D, D hα + hθ log+β +β 5 R S S D, D log + + β β 53 for some 0 β,β, where θ / α ad hλ λlog λ λ log λ is the biary etropy fuctio, ad C W S, S from 6 is as i [9]. The last set of ecessary coditios we cosider is obtaied from [0, Theorem ] by removig 9a ad 9b ad lettig W Z, wherez is as defied i Fig. 5, R S S D, D + α α hα log + β + β, 54 R S S D, D log + + β β, 55 for some 0 β,β. I the followig, we compare Corollary 5 with the ecessary coditios from 50 ad 5-53 as well as from ropositio 3. There exist distortio pairs that satisfy the outer bouds of [7, Remark IV.], [0, Corollary.], ad but ot the outer boud of Corollary 5 for the biary setup. roof. The details are provided i Appedix F.

8 6088 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 64, NO. 9, SETEMBER 08 VI. CONCLUSIONS We have cosidered the lossy codig of correlated sources over a MAC. We have provided a achievable scheme for the trasmissio of correlated sources i the presece of decoder side iformatio, ad ivestigated the coditios uder which separate source ad chael codig is optimal whe the ecoder ad/or decoder has access to side iformatio. By leveragig the obtaied separatio theorem i the presece of a commo side iformatio coditioed o which the two sources are idepedet, we derived a simple ad computable set of ecessary coditios for the lossy codig of correlated sources over a MAC. The compariso of the ew ecessary coditios with the kow results from the literature are provided for the Gaussia settig, i.e., Gaussia sources trasmitted over a Gaussia MAC, as well as for a DSBS over a Gaussia MAC. Idetifyig ecessary coditios for the trasmissibility of correlated sources is a active ope research directio. A direct compariso of the proposed ecessary coditios appear to be difficult aalytically, ad, due to the dimesioality of the search space, umerically. Accordigly, we poit to this problem as a iterestig future directio. Aother iterestig ope problem is the optimality/suboptimality of separatio i the presece of decoder-oly side iformatio, coditioed o which the two sources are idepedet. Other future directios iclude the suboptimality of separatio i other multi-termial scearios with side iformatio. AENDIX A ROOF OF THEOREM Our achievable scheme is alog the lies of [6]. For completeess, we provide the details i the sequel. Geeratio of the codebook: Choose ɛ > ɛ > 0. Fix pu s, pu s, x u, s, x u, s, ŝ u, u, y, z ad ŝ u, u, y, z with E[d j S j, Ŝ j ] D j +ɛ for j,. For each j,, geerate R j sequeces u j m j for m j {,..., R j } idepedetly at radom coditioed o the distributio i p U j u ji. The codebook is kow by the two ecoders ad the decoder. Ecodig: Ecoder j, observes a sequece s j ad tries to fid a idex m j {,..., R j } such that the correspodig u j m j is joitly typical with s j, i.e., s j, u j m j T ɛ. If more tha oe idex exist, the ecoder selects oe of them uiformly at radom. If o such idex exists, it selects a radom idex uiformly. Upo selectig the idex, ecoder j seds x ji x j u ji m j, s ji for i,..., to the decoder. Decodig: The decoder observes the chael output y ad side iformatio z, ad tries to fid a uique pair of idices ˆm, ˆm such that u ˆm, u ˆm, y, z T ɛ ad sets ŝ ji ŝ j u i m, u i m, y i, z i for i,..., for j,. Expected Distortio Aalysis: Let M ad M deote the idices selected by ecoder ad ecoder. Defie E{S, S, U ˆM, U ˆM, Y, Z / T ɛ } 56 Fig. 6. Distributed source codig for correlated sources Y 0, Y, Y where Y j is observed by ecoder j 0,,. The decoder recostructs Y 0 losslessly, while Y ad Y are recostructed i a lossy maer, with respect to the distortio criterio i 63. such that the distortio pair D, D is satisfied if E 0 as.let E j {S j, U j m j / T ɛ m j }, j, 57 E 3 {S, S, U M, U M, Y, Z / T ɛ } 58 E 4 {U m, U m, Y, Z T ɛ for some m M, m M } 59 E 5 {U m, U M, Y, Z T ɛ for some m M } 60 E 6 {U M, U m, Y, Z T ɛ for some m M } 6 The, E E + E + E 3 E c Ec + E 4 +E 5 + E 6. 6 Through stadard techiques based o joit typicality codig, it ca be show that E 0as ad oe ca boud the expected distortios for E c for the two sources S ad S, whe the sufficiet coditios i 7-9 are satisfied. AENDIX B ROOF OF THEOREM A. Achievability Our source codig part is based o the distributed source codig scheme with a commo part from [8]. For completeess, we briefly outlie the problem setup i [8], also depicted i Fig. 6. This problem cosiders the trasmissio of correlated DM sources Y 0, Y, Y such that Y j is observed by ecoder j 0,,. Lossless recostructio of source Y 0 is required at the decoder, while the remaiig two sources, Y ad Y, are recovered i a lossy maer, with respect to correspodig per-letter distortio costraits. I other words, we have lim sup E[d j Y ji, Ŷ ji ] D j, j,. 63 i ad Y 0 Ŷ 0 0 as. Sources Y ad Y also have a commo compoet X such that, for a pair of determiistic fuctios f ad g, X f Y gy ad H X > 0. A achievable rate-distortio regio for

9 GÜLER et al.: LOSSY CODING OF CORRELATED SOURCES OVER A MAC 6089 the distributed source codig system i Fig. 6 is give i [8, Theorem ]. By lettig Y 0 Z, Y j S j, Z for j,, ad X Z i Fig. 6, we observe for this setup that ay achievable rate pair for the system i Fig. 6 is also achievable for our system. This follows from the fact that i our setup Z is available to both ecoders, as a result, the ecoders ca cooperate to sed it to the decoder ad realize ay achievable scheme i [8]. Lettig U X i [8, Theorem ] ad substitutig X Z, Y 0 Z, Ŷ 0 Ẑ, Y j S j, Z, V j U j, Ŷ j Ŝ j,ad d j Y j, Ŷ j d j S j, Ŝ j for j,, we fid that a distortio pair D, D is achievable for the rate triplet R 0, R, R if R 0 H Z Z, U, U 64 R I S, Z; U Z, U 65 R I S, Z; U Z, U 66 R 0 + R H Z Z, U + I S, Z; U Z, U 67 R 0 + R H Z Z, U + I S, Z; U Z, U 68 R + R I S, S, Z; U, U, Z Z 69 R 0 + R + R H Z + I S, S, Z; U, U, Z Z 70 ad E[d j S j, Ŝ j ] D j for j,, for some distributio pz, s, s, u, u, ŝ, ŝ pz, s, s pu s, zpu s, z pŝ, ŝ z, u, u. 7 Coditio 64 ca be removed without loss of geerality. We ca write 65 as, R I S, Z; U Z, U 7 H U Z, U H U S, Z, U 73 H U Z H U S, Z 74 I S ; U Z 75 where 74 is from U S Z U ad U Z U sice pu, u z s,s pu s, zpu s, zps zps z 76 s,s pu, s zpu, s z 77 pu zpu z 78 where 76 is from U S Z S U ad U S Z S as well as S Z S. Followig the steps i 7-75, we ca write 67 as R 0 + R I S ; U Z, 79 which, comparig with 75, idicates that 67 ca be removed without loss of geerality. Followig similar steps, we ca write 66 ad 68 as R I S ; U Z 80 R 0 + R I S ; U Z 8 respectively, which show that coditio 68 ca also be removed. For 69-70, we fid that I S, S, Z; U, U, Z Z I S, S ; U, U Z 8 H U Z + H U Z, U H U Z, S H U Z, S 83 H U Z+ H U Z H U Z, S H U Z, S 84 I S ; U Z + I S ; U Z 85 where 83 holds as U ZS S ad U ZS S U ; ad 84 follows from U Z U show i 78. Combiig 75, 79, 80, ad 8 with 85, we restate 64-7 as follows. A distortio pair D, D is achievable for the rate triplet R 0, R, R if R I S ; U Z 86 R I S ; U Z 87 R + R I S ; U Z + I S ; U Z 88 R 0 + R + R H Z+ I S ; U Z + I S ; U Z 89 ad E[d j S j, Ŝ j ] D j for j,, for some distributio pz, s, s pu s, zpu s, zpŝ, ŝ z, u, u. 90 We ext show that oe ca set Ŝ j f j Z, U, U for j, without loss of optimality. To do so, we write E[d S, Ŝ ] ps, ŝ d s, ŝ 9 s,ŝ pŝ, ŝ z, u, u, s s,ŝ,ŝ,z,u,u pz, u, u, s d s, ŝ 9 pŝ, ŝ z, u, u s,ŝ,ŝ,z,u,u pz, u, u, s d s, ŝ 93 pŝ z, u, u z,u,u ŝ s pz, u, u, s d s, ŝ 94 pz, u, u, s d s, f z, u, u z,u,u,s 95 E[d S, f Z, U, U ] 96 where we defie a fuctio f : Z U U Sˆ i 95 such that, f z, u, u arg mi pz, u, u, s d s, ŝ 97 ŝ s ad set pŝ z, u, u for ŝ f z, u, u ad pŝ z, u, u 0otherwise. A similar argumet follows for S by defiig a fuctio f : Z U U Sˆ leadig to E[d S, Ŝ ] E[d S, f Z, U, U ]. 98 Therefore, we ca set Ŝ j f j Z, U, U for j,.

10 6090 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 64, NO. 9, SETEMBER 08 We ext show for j, that wheever there exists a fuctio f j Z, U, U such that E[d j S j, f j Z, U, U ] D j, 99 the there exists a fuctio g j Z, U j such that E[d j S j, g j Z, U j ] E[d j S j, f j Z, U, U ] D j. 00 We show this result alog the lies of [9]. Cosider a fuctio f Z, U, U such that E[d S, f Z, U, U ] D. From the law of iterated expectatios, E[d S, f Z, U, U ] E S,U,Z[E S,U S,U,Z[d S, f Z, U, U ]] 0 E S,U,Z[E S,U Z[d S, f Z, U, U ]] 0 0 holds due to U S Z U S, see Defie φ : Z U such that φz arg mi E S,U u Zz[d S, f z, U, u ]. 03 The for each Z z, E S,U Zz[E S,U Zz[d S, f z, U, U ]] E S,U Zz[d S, f z, U,φz], 04 ad hece, E[d S, f Z, U, U ] E Z [E S,U Zz[E S,U Zz[d S, f z, U, U ]]] 05 E Z [E S,U Zz[d S, f z, U,φz]] 06 E S,U,Z[d S, f Z, U,φZ] 07 E[d S, g Z, U ] 08 where g Z, U f Z, U,φZ. Followig similar steps, for ay f Z, U, U that achieves E[d S, f Z, U, U ] D we ca fid a fuctio g Z, U such that E[d S, f Z, U, U ] E[d S, g Z, U ]. 09 Combiig 96, 98, 08, 09 with 3 ad 4, we ca state the rate regio i as follows. A distortio pair D, D is achievable for the rate triplet R 0, R, R if R R S Z D 0 R R S ZD R + R R S Z D + R S ZD R 0 + R + R H Z + R S Z D + R S ZD 3 sice for ay ps j, u j, z pu j s j, zps j zpz ad g j z, u j with E[d j S j, g j Z, U j ] D j, I S j ; U j Z R S j Z D j, j,, 4 where R S j Z D j is defied i 4. This completes the source codig part. Our chael codig is based o codig for a MAC with a commo message [30], for which ay triplet of rates R 0, R, R is achievable if R I X ; Y X, W 5 R I X ; Y X, W 6 R + R I X, X ; Y W 7 R 0 + R + R I X, X ; Y 8 for some px, x, y,w py x, x px wpx wpw. B. Coverse Our proof is alog the lies of [4] ad [7]. Suppose there exist ecodig fuctios e j : S j Z X j for j,, decodig fuctios g j : Y Sˆ j for j, adg 0 : Y Ẑ such that i E[d j S ji, Sˆ ji ] D j + ɛ for j, adz Ẑ e where ɛ 0, e 0 as. Defie U ji Y, S i j, Zi c for j, where Zi c Z,...,Z i, Z i+,...,z. The, I X ; Y X, Z H Y X, Z H Y X, X, Z, S 9 H Y X, Z H Y X, Z, S 0 I S ; Y, X Z I S ; Y Z i I S i ; Y S i, Z I S i ; U i Z i i 3 R S ZES i U i, Z i i 4 R S ZES i Y i 5 R S ZE[d S i, Ŝ i ] i 6 R S ZD + ɛ 7 9 is from Y X X Z S, 0 holds sice coditioig caot icrease etropy, ad is from I S ; X Z 0siceS Z X as follows. px, s z s px, s, s z 8 px s, z ps z ps z s 9 px z ps z 30 where 9 holds sice X S Z S ad S Z S. Equatio 3 is from the defiitio of U i ad the memoryless property of the sources; 4 is from 3 ad 4;

11 GÜLER et al.: LOSSY CODING OF CORRELATED SOURCES OVER A MAC is from the fact that coditioig caot icrease 3; 6 follows as Ŝ i is a fuctio of Y ad 7 as R S ZD is covex ad mootoe i D. By defiig a discrete radom variable Q uiformly distributed over {,...,} idepedet of everythig else, we fid that I X ; Y X, Z H Y i X i, Z H Y i X i, X i, Z 3 i I X i ; Y i X i, Q i, Z 3 i I X Q ; Y Q X Q, Q, Z 33 I X ; Y X, W 34 where we let X X Q, X X Q, Y Y Q ad W Q, Z. Combiig 34 with 9 ad 7 leads to 0. We obtai by followig similar steps. Next, we show that I X, X ; Y Z H Y Z H Y Z, X, X 35 H Y Z H Y Z, X, X, S, S 36 H Y Z H Y Z, S, S 37 I S ; Y Z + H S Z H S Y, S, Z 38 I S ; Y Z + H S Z H S Y, Z 39 R S ZD + ɛ + R S ZD + ɛ 40 where 36 is from Y X X S S Z, 37 is from the fact that coditioig caot icrease etropy, 38 is from S Z S, 39 is from coditioig caot icrease etropy, 40 is from followig the steps -7 twice, where the role of S ad S are chaged for the secod term. Moreover, we have I X, X ; Y Z H Y i Z H Y i X i, X i, Z 4 i I X i, X i ; Y i Q i, Z 4 i I X Q, X Q ; Y Q Q, Z 43 I X, X ; Y W 44 where X X Q, X X Q, Y Y Q ad W Q, Z. Combiig 44 with 35 ad 40 leads to. We lastly show that I X, X ; Y I S, S, Z ; Y 45 I Z ; Y + I S ; Y Z +H S Z H S Y, S, Z 46 I Z ; Y + I S ; Y Z +H S Z H S Y, Z 47 H Z + I S ; Y Z + I S ; Y Z δ e 48 H Z+ R S Z D +ɛ+ R S ZD +ɛ δ e 49 where 45 is from Y X X S S Z, 46 is from S Z S, 47 is from the fact that coditioig caot icrease etropy, 48 is from Fao s iequality combied with the data processig iequality, i.e., H Z Y H Z Ẑ δ e 50 where δ e 0 as e 0 [3]. Equatio 49 is from the memoryless property of Z ad from followig -7 twice, the secod oe is whe the role of S is replaced with S. Lastly, usig radom variable Q that has bee defied uiformly over {,...,} ad idepedet of everythig else, we derive the followig. I X, X ; Y H Y i H Y i X i, X i 5 i I X i, X i ; Y i Q i i 5 I X Q, X Q ; Y Q Q 53 I X, X ; Y Q 54 H Y H Y X, X 55 I X, X ; Y 56 where X X Q, X X Q, Y Y Q. Combiig 45, 49, 5, ad 56 leads to 3. I order to complete our proof, we demostrate that px, x w px wpx w for w i, z. To this ed, we show that X x, X x W w X i x, X i x Q i, Z z 57 X i x Q i, Z z X i x Q i, Z z 58 X x W wx x W w 59 where 58 holds sice X i Z X i for i,..., as follows. px, x z px, x, s, s z 60 s,s px s, z px s, z ps z ps z 6 s,s px z px z 6

12 609 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 64, NO. 9, SETEMBER 08 where 6 is from X S Z S X ad X S Z S as well as S Z S. From 6, we observe that X Z X, which implies X i Z X i. AENDIX C ROOF OF THEOREM 3 A. Achievability The source codig part is based o lossy source codig at the two ecoders coditioed o the side iformatio Z shared betwee the ecoder ad decoder [6], after which the coditioal rate distortio fuctios give i 4 ca be achieved for S ad S, respectively. Chael codig part is based o codig for a classical MAC with idepedet chael iputs [3]. B. Coverse Suppose there exist ecodig fuctios e j : S j Z X j, j,, ad decodig fuctios g j : Y Z Sˆ j such that i E[d j S ji, Sˆ ji ] D j + ɛ, whereɛ 0as. The, I X ; Y X, Z I S ; Y X, Z 63 I S ; Y, X Z 64 I S ; Y Z 65 H S Z H S Y, Z, Ŝ 66 H S Z H S Z, Ŝ 67 H S i Z i H S i Z i, Ŝ i i 68 I S i ; Ŝ i Z i 69 i R S ZE[d S i, Ŝ i ] i 70 R S Z D + ɛ 7 63 is from Y X X S Z ad coditioig caot icrease etropy, ad 64 is from X Z S which holds sice px, s z px, s, s z s px s, z ps z ps z s px z ps z 7 from X S Z S ad S Z S. Equatio 65 is due to the oegativity of mutual iformatio; 66 follows from Ŝ g Y, Z ; 67 holds sice coditioig caot icrease etropy; 68 is from the memoryless property of the sources ad the side iformatio as well as the chai rule ad the fact that coditioig caot icrease etropy; 7 holds as R S Z D is covex ad mootoe i D. By defiig a discrete uiform radom variable Q over {,...,} idepedet of everythig else, ad followig steps 3-34 by W Q, Z replaced with Q Q, Z,wefidthat I X ; Y X, Z I X ; Y X, Q 73 where X X Q, X X Q, Y Y Q. Combiig 63, 7, ad 73 yields 4. Followig similar steps we obtai 5, R S Z D + ɛ I X ; Y X, Q. 74 Lastly, we have I X, X ; Y Z I X ; Y X, Z + I X ; Y Z 75 R S Z D + ɛ + I S ; Y Z 76 R S Z D + ɛ + R S Z D + ɛ 77 where the first term i 76 is from 63-7, ad 77 follows similarly to To obtai the secod term i 76, we first show that Y Z X S : py, s z, x ps z, x py s, z, x 78 ps z, x py x, x px s, z ps z s,x ps z, x x 79 py x, x px z is from Y X X S S Z ad X S Z S X as well as S Z S X, which holds sice ps, s, x z px s, z ps z ps z px, s z ps z, 8 due to X S Z S ad S Z S. Note that py z, x py x, x px s, z ps z s,x x py x, x px z, 8 as X S Z X ad S Z X holds sice S Z S X. From 8 ad 80, py, s z, x ps z, x py z, x, 83 ad hece, Y Z X S. The, we use the followig i 75, I X ; Y Z H Y Z H Y X, Z, S 84 H Y Z H Y Z, S I S ; Y Z, 85

13 GÜLER et al.: LOSSY CODING OF CORRELATED SOURCES OVER A MAC 6093 where 84 is from Y Z X S, ad 85 holds sice coditioig caot icrease etropy, which leads to the secod term i 76. The, by replacig W Q, Z with Q Q, Z i 4-44, we ca show by followig the same steps that, I X, X ; Y Z I X, X ; Y Q 86 Combiig 75, 77 ad 86 recovers 6. Lastly, we show px, x q px qpx q alog the lies of [5]. For q i, z, X x, X x Q q X i x, X i x Q i, Z z 87 X i x Q i, Z z X i x Q i, Z z 88 X x Q qx x Q q 89 where 88 holds sice X i Z X i for i,...,. AENDIX D ROOF OF ROOSITION Let ρ 0.5 ad. artitio the set of all distortio pairs D, D for 0 D, D as i Fig. 3b. First, cosider D For this case, oe ca observe that 6 is satisfied with equality whe D , by otig that D, D D for D, D 0.45, ad solvig the resultig equatio. Accordigly, for all distortio pairs 0.45, D with D, the ecessary coditio from 6 is satisfied. Cosider ow the ecessary coditios from Corollary 5 give i 3-4 alog with the distortio pair D, D 0.45,, ρ log D log + β + β 90 log log + + β β, D 9 which follows from R S ZD 0wheD ρ. By rearragig the terms i 90, ρ D β β 9 from which, by combiig with 9, we have the coditio ρ D β β D β β, 93 leadig to ρ ρ β + D D β + D By substitutig D 0.45, ρ 0.5, ad, we fid that the left had side LHS of 94 is a cocave quadratic polyomial whose maximum value is , ρ D attaied whe β 0.6. Hece, 94 is ot satisfied for ay 0 β, ad o distortio pair 0.45, D for which 0 D is achievable accordig to coditios 3-4. Lastly, cosider the ecessary coditios Cosider the distortio pair D, D 0.45, Observe that 0.45, D, as a result, 9-30 ca be writte as log ρ D D ρ D D log + β + β 95 log ρ D D ρ D D log + + β β. 96 ρ Defie α D D ρ,adsetβ D D α β, which satisfies 95. The, 96 ca be expressed as β + α β + α θ 0, 97 ρ where θ / D D ρ D D. The LHS of 97 is a cocave polyomial whose maximum value is 0.945, attaied whe β α , which satisfies 97. The correspodig β ca be computed from β α β α Hece, for all distortio pairs 0.45, D with D, ecessary coditios from 9-30 are satisfied. Accordigly, we coclude that there exist distortio pairs D, D i regios G ad D that satisfy the coditios 6 ad 9-30 but ot 3-4. Next, cosider D 0.6. For this case, 6 holds with equality whe D , by otig that 0.6, D B for D, D 0.6, 0.70 ad solvig the resultig equatio. The ecessary coditio from 6 is the satisfied for all distortio pairs 0.6, D such that D. Cosider ext the coditios from 3-4 for D, D B, ρ log D log + β + β 98 ρ log D D log + + β β, 99 from which, as i 93, we ca obtai the coditio ρ D β β ρ β β D D, 00

14 6094 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 64, NO. 9, SETEMBER 08 ad ρ β + D β + ρ D ρ D D 0. 0 By substitutig D 0.6, ρ 0.5, ad, we observe that the LHS of 0 is a cocave quadratic polyomial whose maximum value occurs at β We ote that wheever D < 0.688, the LHS of 0 is egative for all 0 β, hece the ecessary coditios from Corollary 5 caot be satisfied. Cosider ext coditios 9-30 for D, D 0.6, Sice 0.6, B, oe ca write 9-30 as ρ log D D log + β + β 0 ρ log D D log + + β β. 03 Defie ᾱ ρ D D. By lettig β ᾱ β, which satisfies 0, we restate 03 as β + ᾱ β + ᾱ θ 0, 04 ρ. where θ D D The LHS of 04 is a cocave polyomial with a maximum value of 0.943, attaied whe β ᾱ , which satisfies 04. The correspodig β is computed from β ᾱ β ᾱ Therefore, for all distortio pairs 0.6, D such that D, ecessary coditios i 9-30 are satisfied. Sice 0.6, D B for all D 0.688, we coclude that there exist distortio pairs i Regio B that satisfy the ecessary coditios from 6 ad from 9-30, but ot from Corollary 5. Lastly, cosider the coditios from 7-8. Note that D D i regios B, D, adg, therefore 7-8 ca be stated as, ρ ˆρ l 05 D l R S S D, D +ˆρ 06 for some 0 ˆρ ρ. Note that, if R S S D, D l, 07 the, 06 is satisfied for ay ˆρ. For Regio B, we fid from 07 that, ρ log 08 D ρ l by lettig D ρ, which the leads to D + ρ l. 09 If 07 is satisfied for some D, D, it will be satisfied for all D, D such that D D. Accordigly, if ρ D + ρ l, the coditio 06 is satisfied for all D ρ, irrespective of ˆρ. Next, cosider coditio 05 ad select ˆρ 0, from which we have ρ l, 0 or equally D D ρ e. For adρ 0.5, 09 becomes D ad becomes D Hece, both 7 ad 8 are satisfied whe D 0.45 ad D 0.6. These examples demostrate that there exist distortio pairs i regios B, D, adg, ad from symmetry, i regios C, F, ad I, for which the ecessary coditios from Corollary 5 is tighter tha both 6, 7-8, ad Lastly, we compare Corollary 5 with the coditios from 9-30 by ivestigatig the LHS of both coditios for various regios i Fig. 3b, as the regio defied by the RHS of both 3-4 ad 9-30 is the same. For D, D A, we observe from 3 ad 33 that, R S S D, D C W S, S ρ log + ρ D D log ρ R S ZD + R S Z D, hece, i this regio, Corollary 5 ad the 9-30 boud are equivalet. For D, D B, we fid from 3 ad 33 that, R S S D, D C W S, S ρ log + ρ D D log 3 ρ log ρ R S Z D + R S ZD, 4 D sice D ρ ad D ρ for D, D B. Hece, i this regio, Corollary 5 is at least as tight as By swappig the roles of D ad D, we ca exted the same argumet to Regio C as well. For D, D D, we have from 3 ad 33 that, R S Z D + R S ZD log ρ, 5 D whereas R S S D, D C W S, S { max log ρ +ρ, log ρ } D D ρ D D 6 log ρ D + D + ρ + ρ D D, 7 where the last equatio follows from D D 4ρ D D ρ D D + ρ D D 0 8

15 GÜLER et al.: LOSSY CODING OF CORRELATED SOURCES OVER A MAC 6095 ad therefore, D + D + ρ + ρ D D ρ. 9 The, by comparig 7 with 5, we fid that, Corollary 5 provides ecessary coditios at least as tight as 9-30 if where ρ {ρ : τ D + τ ρ τ + D + τ, D + τ }, τ D + D D. 0 By symmetry, for regio D, D F, Corollary 5 is at least as tight as 9-30 if where ρ {ρ : λ D + λ ρ λ + D + λ, D + τ }, λ D + D D. For D, D G, we observe from 3 ad 33 that, R S S D, D C W S, S log + ρ D log 3 ρ log ρ R S ZD + R S Z D. 4 D Therefore, Corollary 5 is agai at least as tight as It follows by symmetry that Corollary 5 is at least as tight as 9-30 i Regio I as well. For D, D H, we have from 3 ad 33 that, R S S D, D C W S, S log + ρ mid, D log 5 ρ log ρ mid, D + ρ 6 0 R S ZD + R S Z D 7 sice mid, D ρ whe D, D H. From 7, coditios 3 ad 9 are both trivially satisfied i this regio, ad therefore Corollary 5 ad the coditios from 9-30 are equivalet. Same coclusio follows for Regio J. For regio D, D E, we have from 3 ad 33 that, R S ZD + R S Z D 0, 8 hece, coditio 3 is trivially satisfied, whereas R S S D, D C W S, S is as give i 6 ad 7. If D D, we have from 6 ad D ρ that, R S S D, D C W S, S { max log ρ } + ρ, log ρ D ρ D 9 0 R S ZD + R S Z D, 30 ad 9 is also trivially satisfied. Hece, for all D D i Regio E, Corollary 5 ad the coditios from 9-30 are equivalet. We ext cosider the case whe ρ 0.5 ford, D E. Without loss of geerality, we assume that D D. Notig that D ρ, wehave D + D + ρ + ρ D D D + D + ρ + ρ D 3 D ρ + D ρ 3 ρ 33 from which, alog with 8 ad 7, we fid that R S S D, D C W S, S 0 R S ZD + R S Z D. 34 Therefore, for all ρ 0.5, Corollary 5 ad the coditios 9-30 are equivalet. By comparig 8 with 7, we ca show that, Corollary 5 is equivalet to 9-30 if { D + D ρ ρ : + ρ D + D + +, D + D } + 35 where + D D. We therefore fid that the ecessary coditios from Corollary 5 are at least as tight as coditios 9-30 i all regios but E, D, adf. Remark. We ote that Corollary 5 is ot ecessarily strictly tighter i ay of these regios, sice the ecessary coditios ivolve also the RHS of 3-4 ad 9-30, whichca be used to claim the impossibility of achievig certai distortio pairs based o the relative value of the rate distortio fuctios with respect to the rate regio characterized by the RHS. It is possible that, eve though the LHS of Corollary 5 is lower tha the LHS of 9-30, either both or oe of the ecessary coditios may be satisfied, leadig exactly to the same coclusio regardig the achievability of the correspodig distortio pair. AENDIX E ROOF OF ROOSITION Cosider D 0.3, ρ 0.5, ad. For this case, 6 holds with equality whe D 0.65, ad 0.3, 0.65 B. Accordigly, o distortio pair 0.3, D, with 0.5 D < 0.65, satisfies 6. The ecessary coditios of Corollary 5 for D, D B are give by ρ log D log + β + β 36 ρ log D D log + + β β. 37 By defiig ˆα ρ D, ad settig β ˆα β,which satisfies 36, coditio 37 becomes, β + ˆα β + ˆα ˆθ 0, 38

16 6096 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 64, NO. 9, SETEMBER 08 where ˆθ ρ. D D The LHS of 04 is cocave, ad attais its maximum value at β ˆα The correspodig β is computed from β ˆα β From 38, it ca be show that Corollary 5 is satisfied wheever D Accordigly, for the distortio pairs 0.3, D with D < 0.65, the ecessary coditios of Corollary 5 are satisfied whereas the boud i 6 is ot. AENDIX F ROOF OF ROOSITION 3 α α. Let D 0.5, α 0., 0.9, ad 0 D Cosider iitially the coditio from 50. Let D ad observe that for this case R S S D, D hd. The, R S S D, D log+ + ρ max0.978, 39 hece 50 is satisfied for all D Next, cosider the coditios from Let D ad β hθhd hα β ad observe that 5 is satisfied. By rearragig 5-53, we obtai β hθhd hα + β + hθhd hα hd 0 40 whose LHS reaches its maximum value at β hθhd hα Therefore, ecessary coditios 5-53 are satisfied for all D Next, cosider the ecessary coditios i Similar to the previous case, let D ad β α α hαhd β which satisfies 54. Rearrage to obtai α α hαhd β + α α hαhd β + hd 0 4 whose LHS reaches a maximum of 0.44 at β α α hαhd Hece, ecessary coditios from are satisfied for all D Lastly, cosider the ecessary coditios from Corollary 5 ad let D From 3, we have β R S Z D +R S Z D β, from which, by combiig with 4, we obtai R S Z D +R S Z D β + β + R S Z D +R S Z D R S S D,D 0 4 ad observe that the polyomial o the LHS attais its R S Z D +R S Z D maximum value at β Hece, for this example, Corollary 5 caot be satisfiedforay0 β,β. We therefore coclude that there exist distortio pairs for which the two ecessary coditios are satisfied while Corollary 5 is ot. REFERENCES [] T. M. Cover, A. El Gamal, ad M. Salehi, Multiple access chaels with arbitrarily correlated sources, IEEE Tras. If. Theory, vol. IT-6, o. 6, pp , Nov [] C. E. Shao, A mathematical theory of commuicatio, Bell Syst. Tech. J., vol. 7, o. 3, pp , 948. [3] J.-J. Xiao ad Z.-Q. Luo, Multitermial source chael commuicatio over a orthogoal multiple-access chael, IEEE Tras. If. Theory, vol. 53, o. 9, pp , Sep [4] D. Güdüz ad E. Erkip, Correlated sources over a asymmetric multiple access chael with oe distortio criterio, i roc. 4st Au. Cof. If. Sci. Syst., CISS, Baltimore, MD, USA, Mar. 007, pp [5] D. Güdüz, E. Erkip, A. Goldsmith, ad H. V. oor, Source ad chael codig for correlated sources over multiuser chaels, IEEE Tras. If. Theory, vol. 55, o. 9, pp , Sep [6]. Miero, S. H. Lim, ad Y.-H. Kim, A uified approach to hybrid codig, IEEE Tras. If. Theory, vol. 6, o. 4, pp , Apr. 05. [7] A. Lapidoth ad S. Tiguely, Sedig a bivariate Gaussia over a Gaussia MAC, IEEE Tras. If. Theory, vol. 56, o. 6, pp , Ju. 00. [8] A. Jai, D. Güdüz, S. R. Kulkari, H. V. oor, ad S. Verdú, Eergydistortio tradeoffs i Gaussia joit source-chael codig problems, IEEE Tras. If. Theory, vol. 58, o. 5, pp , May 0. [9] W. Kag ad S. Ulukus, A ew data processig iequality ad its applicatios i distributed source ad chael codig, IEEE Tras. If. Theory, vol. 57, o., pp , Ja. 0. [0] A. Lapidoth ad M. Wigger, A ecessary coditio for the trasmissibility of correlated sources over a MAC, i roc. IEEE It. Symp. If. Theory, ISIT, Barceloa, Spai, Jul. 06, pp [] L. Yu, H. Li, ad C. W. Che, Distortio bouds for trasmittig correlated sources with commo part over MAC, i roc. 54th Au. Allerto Cof. Commu., Cotrol, Comput., Sep. 06, pp [] C. Tia, J. Che, S. N. Diggavi, ad S. Shamai Shitz, Optimality ad approximate optimality of source-chael separatio i etworks, IEEE Tras. If. Theory, vol. 60, o., pp , Feb. 04. [3] C. Tia, J. Che, S. N. Diggavi, ad S. Shamai Shitz, Matched multiuser Gaussia source chael commuicatios via ucoded schemes, IEEE Tras. If. Theory, vol. 63, o. 7, pp , Jul. 07. [4] L. Ozarow, O a source-codig problem with two chaels ad three receivers, Bell Syst. Tech. J., vol. 59, o. 0, pp , Dec [5] A. B. Wager ad V. Aatharam, A improved outer boud for multitermial source codig, IEEE Tras. If. Theory, vol. 54, o. 5, pp , May 008. [6] R. M. Gray, Coditioal rate-distortio theory, If. Sys. Lab., Staford Electro. Lab., Staford, CA, USA, Tech. Rep. SU-SEL-7-047, 97. [7] S. Shamai, S. Verdú, ad R. Zamir, Systematic lossy source/chael codig, IEEE Tras. If. Theory, vol. 44, o., pp , Mar. 998.