A Hybrid Random-Structured Coding Scheme for the Gaussian Two-Terminal Source Coding Problem Under a Covariance Matrix Distortion Constraint

Size: px

Start display at page:

Download "A Hybrid Random-Structured Coding Scheme for the Gaussian Two-Terminal Source Coding Problem Under a Covariance Matrix Distortion Constraint"

Maude Willa Booker
5 years ago
Views:

1 A Hybrid Radom-Structured Codig Scheme for the Gaussia Two-Termial Source Codig Problem Uder a Covariace Matrix Distortio Costrait Yag Yag ad Zixiag Xiog Dept of Electrical ad Computer Egieerig Texas A&M Uiversity College Statio TX yagyag@tamu.edu ad zx@ece.tamu.edu Abstract This paper focuses o the Gaussia two-termial source codig problem uder a covariace matrix distortio costrait which subsumes the quadratic Gaussia two-termial source codig problem with idividual distortio costraits whose complete solutio is kow. Differet from existig schemes which are either radom or structured we propose a ew hybrid radom-structured scheme with a sum-rate strictly smaller tha the quatize-ad-bi (QB) upper boud i certai cases. The first layer of our scheme is a QB radom codig scheme attemptig to achieve a itermediate distortio matrix that is as symmetric as possible. The secod layer is a structured scheme that targets at recostructig a weighted differece of the observed sources coditioed o their quatized versios i the first layer. We prove that the gap betwee the sum-rate of our scheme ad its lower boud is o larger tha two bits per sample i particular this gap decreases to exactly oe bit per sample whe the source covariace matrix is symmetrifiable i the sese that the itermediate covariace matrix ca be made purely symmetric. I. INTRODUCTION Distributed source codig (DSC) of correlated sources has attracted a icreasig amout of attetio sice Slepia ad Wolf s celebrated work [ i 973. However lossy DSC is still a ope problem i geeral despite of some partial solutios provided for certai special cases. This paper cosiders a subclass of lossy DSC where two Gaussia sources are separately compressed ad joitly recostructed subject to a covariace matrix distortio costrait. This problem is of particular iterest because it is the most geeral Gaussia two-termial source codig problem i the sese that its achievable rate regio completely determies those of the followig problems (as poited out by Oohama i [). The quadratic Gaussia two-termial source codig problem with idividual MSE distortio costraits whose rate regio has bee characterized [3. The DSC problem of liear fuctios [4 [5 [6 with a sigle MSE distortio costrait o a liear fuctio of the two sources. The geeralized Gaussia CEO problem [7 with a sum MSE distortio costrait o some remote sources that are joitly Gaussia with the two observed sources. For a give source covariace matrix the target distortio matrix has three degrees of freedom. However the cove- Work supported by the NSF grat 0789 ad the Qatar Natioal Research Fud. tioal radom codig scheme with a quatize-ad-bi (QB) framework ca oly achieve distortio matrices o a twodimesioal surface parameterized by the two quatizatio oise variaces. Cosequetly amog those distortio matrices with uit diagoal elemets oly oe is achievable by a QB scheme while others require time-sharig betwee at least two QB schemes. Deote the off-diagoal elemet of the QB-achievable distortio matrix as θ. Two lower sum-rate bouds [3 [8 have bee give for the covariace matrix costraied two-termial problem. Specifically Wager [3 provided a composite sum-rate lower boud usig those for the µ-sum problem ad the cooperative twotermial problem respectively ad showed that the QB sumrate upper boud is tight for ay distortio matrix with uit diagoal elemets ad off-diagoal elemet θ that is o larger tha θ. O the other had Wager s composite lower boud degeerates to the cooperative boud which is relatively loose ad diverges from the QB upper boud as the off-diagoal elemet icreases beyod the threshold θ. Aother sumrate lower boud is provided i [8 which improves the cooperative boud by a ubouded amout whe θ > θ. However the maximum sum-rate gap betwee the improved lower boud ad the QB upper boud is also ubouded. The mai cotributio of this work is a ew hybrid radomstructured scheme that superimposes a distributed differeceformig structured codig compoet proposed by Krithivasa ad Pradha [4 o a QB radom codig scheme with a miimum sum-rate withi two b/s from the lower boud i [8. Our proposed scheme has a similar structure as Ahlswede ad Ha s scheme [9 Sectio VI which cocateates a QB radom codig scheme with Körer ad Marto s structured liear codig scheme [0 for ecodig the modulo- sum of two biary sources (that are asymmetric i geeral). The first layer of our hybrid scheme is a QB scheme where both ecoders quatize theirs sources usig radom quatizers with the resultig quatizatio idices further compressed by a pair of Slepia-Wolf ecoders. The at each ecoder with the availability of the quatized versio of its observed source a correlatio tuer forms a tued source by liearly combiig the observed source with its quatized versio i such a way that a certai weighted differece betwee the resultig tued sources is idepedet of the coarsely quatized versios. Next the secod layer comes i by applyig the Krithivasa

2 ad Pradha s structured scheme [4 o the tued sources to recostruct the weighted differece lossily at the decoder usig two fie lattice quatizers ad the same coarse lattice chael code. The joit decoder liearly combies the three pieces of recostructios i a MMSE maer to geerate the fial estimatios of the sources. Our scheme is parameterized by the four quatizatio oise variaces (two i each layer) ad the weight factor i the secod layer. By assumig the target distortio matrix to have uit diagoal elemet without loss of geerality we show that the optimal parameter set that achieves the miimum sum-rate is such that The first layer ad the correlatio tuers adjust the correlatio betwee the tued sources such that the itermediate covariace matrix of the sources give their quatized versios is as symmetric as possible. If the source covariace matrix is symmetrifiable i the sese that the itermediate covariace matrix has equal diagoal elemets the gap betwee the sum-rate of our hybrid scheme ad the lower boud i [8 is exactly oe b/s while this gap is capped at two b/s i geeral. A ituitive reaso for the first layer to symmetrify the itermediate covariace matrix is that the secod structured layer is most efficiet i the symmetric case (withi oe b/s from optimality [4 [5). Compared to the QB sum-rate boud the sum-rate savig i our hybrid scheme is ubouded. However sice the secod layer of our scheme has a miimum of oe b/s trasmissio rate there are cases whe the QB sum-rate is strictly smaller. Moreover to get a covex composite sum-rate upper boud time-sharig betwee QB schemes ad our scheme is required with a maximum of three differet schemes ivolved due to Carathéodory s Theorem [. II. PROBLEM SETUP Let Y ad Y be joitly Gaussia with zero mea ad covariace matrix [ σ Σ Y ρσ σ ρσ σ σ ad Y (Y Y... Y ) Y (Y Y... Y ) be two legth- blocks of idepedet drawigs of Y ad Y respectively. Each of the two ecoders seds a fuctio of their ow source amely φ () i : R... M () i } for i. The decoder attempts to recostruct Y ad Y usig fuctio ψ () :... M () }... M () } R R. Deote the recostructed versios as Ŷ i i. For a target distortio matrix D [ θ θ () Note that without loss of geerality we assume i this paper that the target distortio matrix D have uit diagoal elemets ad a off-diagoal elemet θ sice otherwise oe ca set Σ Y µσ Y µ T ad D µdµ T with µ diag(([d ) ([D ) ) to obtai a equivalet problem with D takig the form of (). such that 0 D Σ Y we say a triple (R R D) is achievable if there exists a sequece of schemes (φ () φ() ψ() ) such that lim sup log M () i R i i lim sup i Cov(Y Y W W ) D ad deote the rate-distortio regio RD as the covex closure of the set of all achievable triple i.e. } RD cov (R R D) is achievable. The achievable rate regio is defied as R (D) (R R ) : (R R D) RD }. Ad the miimum sum-rate is defied as R (D) mi (R R ) R (D) R R. A. Existig lower ad upper bouds o R (D) For ay pair of Σ Y ad D oe ca always apply the covetioal QB scheme that employs idepedet Gaussia quatizatio followed by biig (Slepia-Wolf codig) to obtai the followig rate-distortio regio [ (R RD QB cov R Cov(Y Y G G ) ) : (G G ) U(Y Y ) R I(Y ; G G ) R I(Y ; G G ) } R R I(Y Y ; G G ). where cov( ) meas covex closure ad U(Y Y ) cotais all (G G ) pairs such that G Y N (0 σq ) G Y N (0 σq ) are idepedetly corrupted versios of Y ad Y. Cosequetly the QB ier rate regio ad the QB sum-rate upper boud are defied as R QB (D) (R R ) : (R R D) RD QB} R QB (D) mi R R (R R ) R QB (D) with R QB (D) R (D) R QB (D) R (D). O the other had Wager et al. [3 provided the followig composite sum-rate lower boud (which was used to prove the QB sum-rate tightess i the origial quadratic Gaussia two-termial problem) that combies a µ-sum boud with the cooperative boud } R (D) R W (D) max R µ (D) R coop (D) R µ (D) log (( ρ )σ σ ρσ σ ( θ)) ( θ) R coop (D) log ( ρ )σσ θ. This lower boud was partially improved i [8 as R (D) R(D) Rµ (D) θ θ R lb (D) θ > θ

3 where ( θ ρ ) σ σ 4ρ ( ρ )σ σ ρ R lb (D) log ( ρ ) σ 3 σ 3 ( θ) (( ρ )σ σ ρ( θ)). I additio R(D) is partially tight whe θ θ sice R(D) R W (D) R (D) R QB (D) whe θ θ. III. THE PROPOSED HYBRID RANDOM-STRUCTURED SCHEME We describe a hybrid radom-structured codig scheme ad give a ew ier rate-distortio regio RD for the covariace matrix costraied Gaussia two-termial problem. Let Λ Λ ad Λ C be three -dimesioal lattices with ormalized secod momets q q ad q C respectively such that Λ i Λ C i. Assume that (Λ Λ ) are good for source codig while Λ C good for chael codig. Also defie the dithered quatizer associated with a -dimesioal lattice Λ ad a legth- dither sequece T as Q Λ(T )(x ) arg mi q Λ x T q T. The block diagram of our proposed two-layer scheme is depicted i Fig.. I the first layer of our scheme the two ecoders quatize Y ad Y usig two radom quatizers with auxiliary radom variables U i Y i P i i () where P i N (0 p i ) are idepedet additive Gaussia oises. The idices of the codewords U ad U are compressed by a pair of Slepia-Wolf ecoders to trasmissio rates of R () ad R () respectively. Y Y Correlatio Tuer Correlatio Tuer Z Z Ecoder Radom Quatizer I Q (T ) U Ecoder c Fig.. Q (T ) Radom Quatizer II U V V Slepia-Wolf Ecoder I Slepia-Wolf Ecoder II R R R R Slepia-Wolf Decoder Decoder Block diagram of our proposed scheme. ^Z Liear Estimator ^U ; ^U The at the each ecoder a correlatio tuer computes where Z i Y i λ i U i i ( ρ )σ λ σ p (σ cρσ ) ( ρ )σ σ p σ p σ p p λ ( ρ )cσσ p σ (cσ ρσ ) ( ρ )σ σ p σ p σ p p ^Y ^Y with c > 0 beig a positive umber. The resultig tued sources (Z Z ) are scaled ad quatized usig aother two dithered lattices quatizers as Ṽ Q Λ(T ) (Z ) Ṽ Q Λ(T ) (cz ) with T ad T beig legth- radom dithers uiformly distributed over the Vorooi regios of Λ ad Λ respectively. The idices of Si Ṽ i mod Λ C i are set to the decoder usig trasmissio rates R () i log q C qi i. The decoder also cosists of two layers. The first layer recostructs Û ad Û usig the Slepia-Wolf Decoder while the secod layer receives S ad S ad computes Ẑ (S S ) mod Λ C. The last step at the decoder recostructs Y ad Y as Ŷ i α i Û α i Û α i3 Ẑ i (3) where α ij s are liear MMSE estimatio coefficiets of (Y Y ) give (U U ) i () ad Z V V with V Z Q V cz Q Z i Yi λ i U i (4) ad Q Q beig idepedet additive Gaussia oises with variaces q ad q respectively. Remarks: The correlatio tuers betwee the two layers of the ecoders select λ ad λ such that Z cz (Y cy ) E [ Y cy U U i.e. they completely remove the portio of Y cy that is correlated to (U U ) which ca be added back at the decoder oce (U U ) is recovered. There is o loss of geerality by usig a sigle weight factor c o Y sice recostructig Y cy as Ẑ is equivalet to restorig Y c Y as Ẑ c. A. Performace of our proposed scheme Our proposed scheme provides the followig ier ratedistortio regio with detailed proof give i Sectio IV. Theorem : It holds that RD RD where RD cotais all triples (R R D) satisfyig R I(Y ; U U ) I(Z; Y V U U ) R I(Y ; U U ) I(Z; V Y U U ) R R I(Y Y ; U U ) I(Z; Y V U U ) I(Z; V Y U U ) for some (U U Z V V ) defied i () ad (4) such that Cov(Y Y U U Z) D. A atural corollary of Theorem is that the covex closure of the QB rate-distortio regio RD QB ad the rate-distortio regio RD of our ew scheme is a subset of RD.

4 Corollary : It holds that ( cov RD RD QB) RD. (5) O the other had the rate-distortio regio RD of our ew scheme gives a sum-rate upper boud stated i the followig theorem. The proof is also outlied i Sectio IV. Theorem : For a D i () with d d ad θ θ it holds that R(D) R(D) (6) log 4σ 3 σ3 ( ρ ) ( θ) (( ρ )σ σ ρ(θ)) δ δ 0 log 4σ 4 σ4 ( ρ ) (( ρ )σ ( θ )) ( θ ) (( ρ )σ σ σ ρσ θρσσ) δ <0 δ 0 log 4σ 4 σ4 ( ρ ) (( ρ )σ ( θ )) ( θ ) (( ρ )σ σ σ ρσ θρσσ) δ 0 δ <0 where for i j } i j δ i σ i σ j( ρ )( θ) ( ρ )σ i σ j (σ j ρσ i )( θ). Comparig our ew sum-rate upper boud with the lower boud R(D) we obtaied the followig theorem which states that the sum-rate gap betwee R(D) ad R(D) is o larger tha two b/s sadwichig the sum-rate-distortio fuctio R(D) i a costat ad relatively small rage. The detailed proof is omitted. Theorem 3: For a D i () with d d ad θ θ it holds that R(D) R(D). (7) Furthermore the followig theorem states that the gap betwee R(D) ad R(D) is exactly oe b/s i a special symmetrifiable case. Theorem 4: If D takes the form i () with d d ad θ θ ad Σ Y is symmetrifiable i the sese that there exists p p 0 such that [Σ Z (p p ) [Σ Z (p p ) (8) [ with Σ Z (p p ) Σ Y diag( p p ) the R(D) R(D). (9) IV. PROOF OF MAIN RESULTS This sectio cotais proof outlies of Theorems ad. A. Proof of Theorem Proof: Before provig Theorem we states two lemmas with detailed proofs omitted. Lemma : There exist sequeces of lattices Λ i ()} N i j that are good for source codig i the sese that Ṽ i Ẑ Ṽ Ṽ Z V i i with (Z V V ) defied i (4) ad X Y meas X coverges to Y i the Kullback-Leibler divergece sese as. Lemma : There exist a sequece of lattices Λ C ()} N that is good for chael codig i the sese that q C Var(Z) ad Pr(Û i Ui ) 0 i Pr(Ẑ Ṽ Ṽ ) 0. Cosequetly for Ŷ i s defied i (3) Cov(Y Ŷ Y Ŷ ) Cov(Y Y U U Z). (0) To prove Theorem we ca simply apply Lemmas ad to show achievability of (R R D) with R I(Y ; U ) I(Z; Y V U U ) R I(Y ; U U ) I(Z; V Y U U ). The usig the same techique we ca prove that the dual scheme ca achieve (R R D) with R I(Y ; U U ) I(Z; Y V U U ) R I(Y ; U ) I(Z; V Y U U ) ad Theorem follows by takig the covex closure of (R R D) ad (R R D) triples. B. Proof of Theorem Proof: With Theorem we oly eed to provide proper (c p p q q ) parameters for the three differet cases. Whe δ δ 0 let c () σ σ (p () p() ) (δ δ ) q () σ σ ( ρ )( θ ) i (θσ σ ( ρ ) ρ( θ i. )) Whe δ < 0 ad δ 0 let p () c () θσ σ ( ρ ) ρ( θ ) σ ( ρ ) ( θ ) p () σ (( ρ )σ ( θ )) (σ )(σ ) ρ σ σ θ(ρσ σ θ) q () σ i ( ρ )( θ ) (( ρ )σ ( i. θ )) Whe δ < 0 ad δ 0 let p (3) c (3) ( ρ )σ ρ( θ ) θσ σ ( ρ ) ρ( θ ) p (3) σ (( ρ )σ ( θ )) (σ )(σ ) ρ σ σ θ(ρσ σ θ) q (3) i σ ( ρ )( θ )(( ρ )σ ( θ )) (θσ σ ( ρ ) ρ( θ )) i. The verificatios of (6) is straightforward.

5 ρ V. NUMERICAL EXAMPLES Fig. compares the sum-rate lower ad upper bouds R(D) R W (D) R QB (D) R(D) ad R T S (D) respectively with σ 0 σ 0 ρ 0.9 d d ad ( R T S (D) mi R R :(R R D) cov RD RD QB)}. We observe that for this case R(D) does ot directly improves R QB (D). However by time-sharig betwee our proposed two-layer scheme ad the QB scheme R T S (D) is strictly smaller tha R QB (D) for all θ > θ. I additio due to Theorem 3 the gap betwee R(D) ad R(D) is o larger tha two b/s. sum-rate (b/s) BT sum-rate upper boud R BT (D) New sum-rate upper boud R(D) Sum-rate upper boud R TS (D) w/ time-sharig Sum-rate lower boud R(D) Wager et al. s composite lower boud R W (D) θ θ b/s θ Fig.. A compariso amog differet lower ad upper sum-rate bouds whe σ 0 σ 0 ρ 0.9 ad d d. Aother compariso whe σ 00 9 σ 0 ρ 0.99 ad d d. We observe that for this case R(D) itself already improves R QB (D). Ad oe ca show that (ρ D) is always symmetrifiable for ay θ > θ hece due to Theorem 4 the gap betwee R(D) ad R(D) is exact oe b/s. sum-rate (b/s) BT upper boud R BT (D) New sum-rate upper boud R(D) Sum-rate upper boud R TS (D) w/ time-sharig Sum-rate lower boud R(D) Wager et al. s composite sum-rate lower boud θ θ b/s symmetrifiable θ Fig. 3. A compariso amog differet lower ad upper sum-rate bouds whe σ 00 9 σ 0 ρ 0.99 ad d d. Fig. 4 shows the ( σ σ ) regio where R(D) < R QB (D) for some θ whe ρ is fixed. Aalytical forms of the boudaries are very hard to compute hece the plot is obtaied through umerical comparisos betwee R(D) ad R QB (D). We observe that i order for R(D) < R QB (D) to hold ρ must be at least σ Fig. 4. ( fixed. 0.4 σ ρ 0.6 ρ 0.65 ρ 0.65 ρ ρ 0.7 ρ 0.75 ρ 0.75 ρ ρ 0.8 ρ 0.85 ρ 0.85 ρ ρ 0.9 ρ 0.95 ρ 0.95 ρ ρ σ σ ) regio where R(D) < R QB (D) for some θ whe ρ is Oe ca show that as ρ goes to oe the maximum sumrate savig betwee R(D) ad R QB (D) is ubouded with detailed proof omitted. It is worth otig that R T S (D) ca be achieved by at most three differet schemes due to Carathéodory s Theorem [. VI. CONCLUSIONS We have proposed a two-layer structured scheme for the covariace matrix costraied Gaussia two-termial source codig problem. The first layer is a QB compoet that adjusts the source correlatio while the secod layer directly aims at recostructig a weighted differece of the sources give the iformatio set from the first layer to the decoder. By choosig appropriate parameters the proposed scheme ca strictly improve the QB rate-distortio regio with a sumrate that is withi two b/s from optimality. Coceptually our proposed hybrid scheme brigs together the two differet worlds of radom QB codig ad structured lattice codig. REFERENCES [ D. Slepia ad J. Wolf Noiseless codig of correlated iformatio sources IEEE Tras. If. Theory vol. 9 pp July 973. [ Y. Oohama Distributed source codig of correlated Gaussia sources preprit available at cache/arxiv/pdf/007/ v.pdf. [3 A. Wager S. Tavildar ad P. Viswaath The rate regio of the quadratic Gaussia two-termial source-codig problem IEEE Tras. If. Theory vol. 54 pp May 008. [4 D. Krithivasa S. S. Pradha Lattices for distributed source codig: joitly Gaussia sources ad recostructio of a liear fuctio IEEE Tras. If. Theory vol. 55 pp Dec [5 A. B. Wager O distributed compressio of liear fuctios IEEE Tras. If. Theory vol. 57 pp Jauary 0. [6 M. A. Maddah-Ali ad D. Tse Iterferece eutralizatio i distributed lossy source codig Proc. ISIT 0 Ja. 00. [7 Y. Yag ad Z. Xiog O the geeralized Gaussia CEO problem IEEE Tras. If. Theory to appear. [8 Y. Yag Y. Zhag ad Z. Xiog A ew sufficiet coditio for sum-rate tightess i quadratic Gaussia multitermial source codig submitted to IEEE Tras. If. Theory Jue 00. [9 R. Ahlswede ad T. S. Ha O source codig with side iformatio via a multiple-access chael ad related problems i multi-user iformatio theory IEEE Tras. If. Theory vol. 9 pp May 983. [0 J. Körer ad K. Marto How to ecode the modulo-two sum of biary sources IEEE Tras. Iform. Theory vol. 5 pp 9- March 979. [ T. Cover ad J. Thomas Elemets of Iformatio Theory Wiley 99.

Cooperative Communication Fundamentals & Coding Techniques

Cooperative Communication Fundamentals & Coding Techniques 3 th ICACT Tutorial Cooperative commuicatio fudametals & codig techiques Cooperative Commuicatio Fudametals & Codig Techiques 0..4 Electroics ad Telecommuicatio Research Istitute Kiug Jug 3 th ICACT Tutorial