On Source-Channel Communication in Networks

Transcription

1 On Source-Channel Communication in Networks Michael Gastpar Department of EECS University of California, Berkeley DIMACS: March 17, 2003.

2 Outline 1. Source-Channel Communication seen from the perspective of the separation theorem 2. Source-Channel Communication seen from the perspective of measure-matching Acknowledgments Gerhard Kramer Bixio Rimoldi Emre Telatar Martin Vetterli

3 Source-Channel Communication Consider the transmission of a discrete-time memoryless source across a discrete-time memoryless channel. Source S F X Channel Y G Ŝ Destination F (S n ) = X n G(Y n ) = Ŝn The fundamental trade-off is cost versus distortion, = Ed(S n, Ŝn ) Γ = Eρ(X n ) What is the set of achievable trade-offs (Γ, )? optimal trade-offs (Γ, )?

4 The Separation Theorem Source S F X Channel Y G Ŝ Destination F (S n ) = X n G(Y n ) = Ŝn For a fixed source (p S, d) and a fixed channel (p Y X, ρ): A cost-distortion pair (Γ, ) is achievable if and only if R( ) C(Γ). A cost-distortion pair (Γ, ) is optimal if and only if R( ) = C(Γ), subject to certain technicalities. Rate-matching: In an optimal communication system, the minimum source rate is matched (i.e., equal) to the maximum channel rate.

5 Source-Channel Communication in Networks Simple source-channel network: Src 1 S 1 X 1 F 1 Y 1 Ŝ G 11 1 Dest 1 Channel Src 2 S 2 X 2 F 2 Y 2 Ŝ 12, G Ŝ22 12 Dest 2 Trade-off between cost (Γ 1, Γ 2 ) and distortion ( 11, 12, 22 ). Achievable cost-distortion tuples? Optimal cost-distortion tuples? For the sketched topology, the (full) answer is unknown.

6 These Trade-offs Are Achievable: For a fixed network topology and fixed probability distributions and cost/distortion functions: If a cost-distortion tuple satisfies R( 1, 2,...) C(Γ 1, Γ 2,...), then it is achievable. R 2 R C R 1 When is it optimal?

7 Example: Multi-access source-channel communication Src 1 Src 2 S 1 X F 1 {0, 1} 1 Ŝ 1, Ŝ2 Y = X 1 + X 2 G 12 S 2 F 2 X 2 {0, 1} Dest Capacity region of this channel is contained inside R 1 + R Goal: Reconstruct S 1 and S 2 perfectly. S 1 and S 2 are correlated: S 1 = 0 S 1 = 1 S 2 = 0 1/3 1/3 S 2 = 1 0 1/3 R 1 + R 2 log R and C do not intersect. Yet uncoded transmission works. This example appears in T. M. Cover, A. El Gamal, M. Salehi, Multiple access channels with arbitrarily correlated sources. IEEE Trans IT-26, 1980.

8 So what is capacity? The capacity region is computed assuming independent messages. In a source-channel context, the underlying sources may be dependent. MAC example: Allowing arbitrary dependence of the channel inputs, the capacity is log 2 3 = 1.585, fixing the example: R C =. Can we simply redefine capacity appropriately? Remark: Multi-access with dependent messages is still an open problem. T. M. Cover, A. El Gamal, M. Salehi, Multiple access channels with arbitrarily correlated sources. IEEE Trans IT-26, 1980.

9 Separation Strategies for Networks In order to retain a notion of capacity: Discrete messages are transmitted reliably Src 1 S 1 F 1 F 1 X 1 Y 1 G 1 G 1 Ŝ 11 Dest 1 Channel Src 2 S 2 F 2 F 2 X 2 Y 2 G 12 G 12 Ŝ 12, Ŝ22 Dest 2 What is the best achievable performance for such a system? The general answer is unknown.

10 Example: Broadcast Z 1 Y 1 G Ŝ 1 1 Dest 1 Src S F X Z 2 Y 2 G Ŝ 2 2 Dest 2 S, Z 1, Z 2 are i.i.d. Gaussian. Goal: Minimize the meansquared errors 1 and 2. Denote by 1 and 2 the singleuser minima. 1 and 2 cannot be achieved simultaneously by sending messages reliably: The messages disturb one another. But uncoded transmission achieves 1 and 2 simultaneously. This cannot be fixed by altering the definitions of capacity and/or rate-distortion regions.

11 Alternative approach Eρ(X n ) Γ Source S F X Channel Y G Ŝ Destination F (S n ) = X n G(Y n ) = Ŝn A code (F, G) performs optimally if and only if it satisfies R( ) = C(Γ) (subject to certain technical conditions). Equivalently, a code (F, G) performs optimally if and only if ρ(x n ) = c 1 D(p Y n x n p Y n ) + ρ 0 d(s n, ŝ n ) = c 2 log 2 p(s n ŝ n ) + d 0 (s) I(S n ; Ŝn ) = I(X n ; Y n ) We call this the measure-matching conditions.

12 Single-source Broadcast Src S F Y 1 G Ŝ 1 1 Dest 1 X Chan Y 2 G Ŝ 2 2 Dest 2 Measure-matching conditions for single-source broadcast: If the single-source broadcast communication system satisfies ρ(x) = c (1) 1 D(p Y 1 x p Y1 ) + ρ (1) 0 = c (2) 1 D(p Y 2 x p Y2 ) + ρ (2) 0, d 1 (s, ŝ 1 ) = c (1) 2 log 2 p(s ŝ 1 ) + d (1) 0 (s), d 2 (s, ŝ 2 ) = c (2) 2 log 2 p(s ŝ 2 ) + d (2) 0 (s), I(X; Y 1 ) = I(S; Ŝ1), and I(X; Y 2 ) = I(S; Ŝ2), then it performs optimally.

13 Sensor Network M wireless sensors measure physical phenomena characterized by S. U 1 F 1 X 1 U 2 F 2 X 2 Source S Y G Ŝ Dest U M F M X M

14 Gaussian Sensor Network The observations U 1, U 2,..., U k are noisy versions of S. W 1 W 2 U 1 U 2 F 1 F 2 X 1 X 2 M k=1 E X k 2 MP Z Source S Y G Ŝ Dest W M U M F M X M

15 Gaussian Sensor Network: Bits Consider the following communication strategy: W 1 U 1 F 1 Bits F 1 M k=1 E X k 2 MP W 2 U 2 F 2 Bits F 2 X 1 X 2 Z Source S Y G Bits G Ŝ Dest W M U M F M Bits F M X M

16 Gaussian Sensor Network: Bits (1/2) Source coding part. CEO problem. See Berger, Zhang, Viswanathan (1996); Viswanathan and Berger (1997); Oohama (1998). W 1 W 2 U 1 U 2 F 1 F 2 T 1 T 2 S N c (0, σ 2 S) and for k = 1,..., M, W k N c (0, σ 2 W) Src S G Ŝ Dest For large R tot, the behavior is W M U M F M T M D CEO (R tot ) = σ2 W R tot.

17 Gaussian Sensor Network: Bits (2/2) Channel coding part. Additive white Gaussian multi-access channel: ( R sum log MP ) σz 2. However, the codewords may be dependent. Therefore, the sum rate may be up to ( R sum log M 2 ) P σz 2. Hence, the distortion for a system that satisfies the rate-matching condition is at least Is this optimal? D rm (M) σ 2 W log 2 (1 + M 2 P σ 2 Z )

18 Gaussian Sensor Network: Uncoded transmission Consider instead the following coding strategy: W 1 U 1 α 1 X 1 M k=1 E X k 2 MP W 2 U 2 α 2 X 2 Z Source S Y G Ŝ Dest W M U M α M X M

19 Gaussian Sensor Network: Uncoded transmission Strategy: The sensors transmit whatever they measure, scaled to their power constraint, without any coding at all. Y [n] = P σ 2 S + σ2 W ( MS[n] + ) M W k [n] + Z[n]. If the decoder is the minimum mean-squared error estimate of S based on Y, the following distortion is incurred: k=1 Proposition 1. Uncoded transmission achieves D 1 (MP ) = σs 2σ2 W M 2 M+(σZ 2 /σ2 W )(σ2 S +σ2 W )/P σ2 S + σ2 W. This is better than separation (D rm 1/ log M). In this sense, uncoded transmission beats capacity. Is it optimal?

20 Gaussian Sensor Network: An outer bound Suppose the decoder has direct access to U 1, U 2,..., U M. W 1 W 2 U 1 U 2 The smallest distortion for our sensor network cannot be smaller than the smallest distortion for the idealization. Src S G Ŝ Dest D min,ideal = σ 2 S σ2 W Mσ 2 S + σ2 W W M U M

21 Gaussian Sensor Network: Asymptotic optimum Rate-matching: D rm (MP ) σ 2 W log 2 (1 + M 2 P σ 2 Z ) Uncoded transmission: D 1 (MP ) = σs 2σ2 W M 2 M+(σZ 2 /σ2 W )(σ2 S +σ2 W )/P σ2 S + σ2 W. Proposition 2. As the number of sensors becomes large, the optimum trade-off is D(MP ) σ 2 S σ2 W MσS 2 +. σ2 W

22 Gaussian Sensor Network: Conclusions Two conclusions from the Gaussian sensor network example: 1. Uncoded transmission is asymptotically optimal. This leads to a general measure-matching condition. 2. Even for finite M, uncoded transmission considerably outperforms the best separation-based coding strategies. This suggests an alternative coding paradigm for source-channel networks.

23 Sensor Network: Measure-matching Theorem. If the coding system F 1, F 2,..., F M, G satisfies the cost constraint Eρ(X 1, X 2,..., X M ) Γ, and then it performs optimally. d(s, ŝ) = log 2 p(s ŝ) I(S; U 1 U 2... U M ) = I(S; Ŝ), U 1 F 1 X 1 U 2 F 2 X 2 Src S Chan Y G Ŝ Dest U M F M X M

24 Proof: Cut-sets Outer bound on the capacity region of a network: X 1. X M S S c Y If the rates (R 1, R 2,..., R M ) are achievable, they must satisfy, for every cut S: R k max I(X S; Y S c X S c) S S c p(x 1,x 2,...,x M ) Hence, if a scheme satisfies, for some cut S, the above with equality, then it is optimal (with respect to S). Remark. This can be sharpened. If the rates (R 1, R 2,..., R M ) are achievable, then there exists some joint probability distribution p(x 1, x 2,..., x M ) such that for every cut S: S S c R k I(X S ; Y S c X S c)

25 Source-channel Cut-sets (1/2) Fix the coding scheme (F 1, F 2,..., F M, G). Is it optimal? Place any source-channel cut through the source-channel network. S U 1. U M X 1. X M Y Ŝ Sufficient condition for optimality: R S ( ) = C (X1,X 2,...,X M ) Y (Γ). Gaussian: D σ2 S σ2 Z M 2 P + σz 2. Equivalently, using measure-matching conditions, ρ(x 1, x 2,..., x M ) = D(p Y x1,x 2...,x M p Y ) d(s, ŝ) = log 2 p(s ŝ) I(S; Ŝ) = I(X 1X 2... X M ; Y )

26 Source-channel Cut-sets (2/2) Fix the coding scheme (F 1, F 2,..., F M, G). Is it optimal? Place any source-channel cut through the source-channel network. S U 1. U M X 1. X M Y Ŝ Sufficient condition for optimality: R S ( ) = C S (U1,U 2,...,U M )(Γ). Gaussian: D σ2 S σ2 W MσS 2 +. σ2 W Equivalently, using measure-matching conditions, ρ(s) = D(p U1,U 2...,U M s p U1,U 2...,U M ) d(s, ŝ) = log 2 p(s ŝ) I(S; Ŝ) = I(S; U 1U 2... U M )

27 Sensor Network: Measure-matching Theorem. If the coding system F 1, F 2,..., F M, G satisfies the cost constraint Eρ(X 1, X 2,..., X M ) Γ, and then it performs optimally. d(s, ŝ) = log 2 p(s ŝ) I(S; U 1 U 2... U M ) = I(S; Ŝ), U 1 F 1 X 1 U 2 F 2 X 2 Src S Chan Y G Ŝ Dest U M F M X M

28 Gaussian Example 1. The uncoded scheme satisfies the condition d(s, ŝ) = log 2 p(s ŝ) for any M since p(s ŝ) is Gaussian. More generally, this is true as soon as the sum of the measurement noises W k, k = 1,..., M, is Gaussian. 2. For the mutual information, for large M, I(S; U 1 U 2... U M ) I(S; Ŝ) c 1 log 2 ( 1 + c 2 M 2 ) M, hence the second measure-matching condition is approached as M.

29 Measure-matching as a coding paradigm Second observation from the Gaussian sensor network example: 2. Even for finite M, uncoded transmission considerably outperforms the best separation-based coding strategies. Coding Paradigm. The goal of the coding scheme in the sensor network topology is to approach as closely as possible. d(s, ŝ) = log 2 p(s ŝ) I(S; U 1 U 2... U M ) = I(S; Ŝ), The precise meaning of as closely as possible remains to be determined.

30 Slightly Extended Topologies Communication between the sensors S U 1. U M X 1. X M Y Ŝ Sensors assisted by relays S U 1. U M X 1 R 1. X M Y Ŝ

31 Slightly Extended Topologies Key insight: The same outer bound applies. Hence, But: the same measure-matching condition applies, and in the Gaussian scenario, uncoded transmission, ignoring the communication between the sensors, and/or the relay, is asymptotically optimal. Communication between the sensors simplifies the task of matching the measures. Relays simplify the task of matching the measures. Can this be quantified?

32 Conclusions Rate-matching: Yields some achievable cost-distortion pairs for arbitrary network topologies. Measure-matching: Yields some optimal cost-distortion pairs for certain network topologies, including single-source broadcast sensor network sensor network with communication between the sensors sensor network with relays

33 What is Information? Point-to-point: Network: Information = Bits Information =??? References: 1. M. Gastpar and M. Vetterli, Source-channel communication in sensor networks, IPSN 2003 and Springer Lecture Notes in Computer Science, April M. Gastpar, Cut-set bounds for source-channel networks, in preparation.