Amobile satellite communication system, like Motorola s

Transcription

1 I TRANSACTIONS ON INFORMATION THORY, VOL. 45, NO. 4, MAY Distributed Source Coding for Satellite Communications Raymond W. Yeung, Senior Member, I, Zhen Zhang, Senior Member, I Abstract Inspired by mobile satellite communications systems, we consider a source coding system which consists of multiple sources, multiple encoders, multiple decoders. ach encoder has access to a certain subset of the sources, each decoder has access to certain subset of the encoders, each decoder reconstructs a certain subset of the sources almost perfectly. The connectivity between the sources the encoders, the connectivity between the encoders the decoders, the reconstruction requirements for the decoders are all arbitrary. Our goal is to characterize the admissible coding rate region. Despite the generality of the problem, we have developed an approach which enables us to study all cases on the same footing. We obtain inner outer bounds of the admissible coding rate region in terms of 0 3 N 0N, 3 respectively, which are fundamental regions in the entropy space recently defined by Yeung. So far, there has not been a full characterization of 0N, 3 so these bounds cannot be evaluated explicitly except for some special cases. Nevertheless, we obtain an alternative outer bound which can be evaluated explicitly. We show that this bound is tight the special cases for which the admissible coding rate region is known. The model we study in this paper is more general than all previously reported models on multilevel diversity coding, the tools we use are new in multiuser information theory. Index Terms Diversity coding, multiterminal source coding, multiuser information theory, satellite communication. I. INTRODUCTION Amobile satellite communication system, like Motorola s Iridium TM System Qualcomm s GlobalStar TM System, provides telephone data services to mobile users within a certain geographical range through satellite links. For example, the Iridium TM System covers users anywhere in the world, while the GlobalStar TM System covers users between 70 latitude. At any time, a mobile user is covered by one or more satellites. Through satellite links, the message from a mobile user is transmitted to a certain set of mobile users within the system. In a generic mobile satellite communication system, transmitters receivers are not necessarily colocated. In principle, a transmitter can transmit information to all the satellites Manuscript received December 12, 1996; revised November 4, The work of R. W. Yeung was supported in part by the Research Grant Council of Hong Kong under armarked Grant CUHK 332/96. The work of Z. Zhang was supported in part by the National Science Foundation under Grant NCR R. W. Yeung is with Department of Information ngineering, The Chinese University of Hong Kong, N.T., Hong Kong, China ( whyeung@ie.cuhk.edu.hk). Z. Zhang is with the Communication Sciences Institute, Department of lectrical ngineering Systems, University of Southern California, Los Angeles, CA USA ( zzhang@milly.usc.edu). Communicated by K. Zeger, ditor At Large. Publisher Item Identifier S (99) within the line of sight simultaneously; a satellite can combine, encode, broadcast the information it receives from all the transmitters it covers; a receiver can combine decode the information it receives from all the satellites within the line of sight. We are motivated to study a new problem called the distributed source coding problem. A distributed source coding system consists of multiple sources, multiple encoders, multiple decoders. ach encoder has access to a certain subset of the sources, each decoder has access to a certain subset of the encoders, each decoder reconstructs a certain subset of the sources almost perfectly. The connectivity between the sources the encoders, the connectivity between the encoders the decoders, the reconstruction requirements for the decoders are all arbitrary. The sources, the encoders, the decoders in a distributed coding system correspond to the transmitters, the satellites, the receivers in a mobile satellite communication system, respectively. The set of encoders connected to a source refers to the set of satellites within the line of sight of a transmitter, while the set of decoders connected to an encoder refers to the set of receivers covered by a satellite. Throughout the paper, we will use a boldfaced letter to denote a vector. The th component of a vector is denoted by unless otherwise specified. For a rom variable, we will use to denote the alphabet set of, to denote a generic outcome of. For a set, we will use to denote the closure of. We will use to denote the probability of an event, to denote the entropy of a set of rom variables in base. Let us now present the formal description of the problem. A distributed source coding system consists of the following elements: 1), the index set of the information sources; 2), the index set of the encoders; 3), the index set of the decoders; 4), the set of connections between the sources the encoders; 5), the set of connections between the encoders the decoders; 6), which specify the reconstruction requirements of the decoders. The th source is denoted by,. We assume that are independent, are independent identically distributed (i.i.d.) copies of a generic rom variable, where. The th encoder /99$ I

2 1112 I TRANSACTIONS ON INFORMATION THORY, VOL. 45, NO. 4, MAY 1999 An -tuple is admissible if for every, there exists for sufficiently large an code such that Fig. 1. A distributed source coding system. Let, let is admissible is denoted by,, the th decoder is denoted by,. The sets specify the distributed source coding system as follows: has access to if only if, has access to if only if, reconstructs. Let us illustrate the above notations by a small example. For the distributed source coding system in Fig. 1, To facilitate the description of our model, we define contains the indices of the sources which are accessed by, contains the indices of the encoders which are accessed by. Let be the Hamming distortion measure in Let code is defined by.an if if. ; i.e., for any be the admissible coding rate region (henceforth the coding rate region if there is no ambiguity). The goal of this paper is to characterize. In Section II, we prove an inner bound for. In Section III, we prove an outer bound for. In Section IV, we give geometrical interpretations of. The regions are defined in terms of, which are fundamental regions recently defined by Yeung [10]. So far, there has not been a full characterization of either or, so explicit evaluation of is not possible. Based on the geometrical interpretation of these regions, we obtain an outer bound for called the LP bound (for linear programming bound) which can be evaluated explicitly. This is described in Section V. In this section, we also show that the LP bound is tight the special cases for which the admissible coding rate region is known. Concluding remarks are in Section VI. II. INNR BOUND Before we define, we first introduce some notation from the framework for information inequalities developed in [10]. Let be a finite set of discrete rom variables whose joint distribution is unspecified, let. Note that. Let denote the coordinates of. A vector is said to be constructible if there exists a joint distribution for such that. We then define is constructible To simplify notation in the sequel, for any nonempty, we further define where we have used juxtaposition to denote the union of two sets. In using the above definitions, we will not distinguish elements singletons of ; i.e., for a rom variable, is the same as. Let be discrete rom variables whose joint distribution is unspecified, let where i.e., the set containing all the rom variables. is an auxiliary rom variable associated

3 YUNG AND ZHANG: DISTRIBUTD SOURC CODING FOR SATLLIT COMMUNICATIONS 1113 with the source, is an auxiliary rom variable associated with, the output of the encoder. The actual meaning of will become clear later. Let be the set of all such that there exists which satisfies the following conditions: (1) for (2) for (3) for (4) for (5) Note that (1) (3) are hyperplanes, (4) is an open halfspace in. We then define. Theorem 1:. Note that can be defined more conventionally as the set of all such that there exist auxiliary rom variables satisfying (6) for (7) for (8) for (9) for (10) Although this alternative definition is more intuitive, the region so defined appears to be totally different from case to case. On the other h, defining in terms of enables us to study all cases on the same footing. In particular, if is an explicit inner bound on, upon replacing by in the definition of, we immediately obtain an explicit inner bound on cases. (Unfortunately, no explicit inner bound on for is available at this point; a nontrivial inner bound for has been obtained in [6] [12].) The introduction of in Section V as an explicit outer bound on is in the same spirit. Let us give the motivations of the conditions in (1) (5) before we prove the theorem, although the meaning of these conditions cannot be fully explained until we come to the proof. The condition (1) corresponds to the assumption that the sources are independent. The condition (2) corresponds to the fact that the encoder has access to the sources. The condition (3) corresponds to the requirement that the sources can be reconstructed by the decoder. The condition (4) means that the entropy of the auxiliary rom variable is strictly greater than the entropy rate of the source, the condition (5) means that the coding rate of the encoder is greater than or equal to the entropy of the auxiliary rom variable. We will state the following lemma before we present the proof of the theorem. Since this lemma is a stard result, its proof will be omitted. We first recall the definitions of strong typicality of sequences [2]. A sequence is -typical with respect to a distribution if where is the number of occurrences of the symbol in. Similarly, a pair of sequences is -typical with respect to a distribution if In the sequel, the -typical notations in [4] will be adopted. Lemma 1: Let be -vectors drawn according to If, then (11) where denotes any function such that for some constant in a neighborhood of. Proof of Theorem 1: We will prove the theorem by describing a rom coding scheme showing that it has the desired performance. Let be small positive quantities be a large integer to be specified later. Let. Then there exist rom variables such that the left-h sides of (1) (5) are the corresponding Shannon information measures; i.e., (1) (5) can be written as (6) (10). The encoding scheme is described in the following steps: 1. Let be the distribution of, let. For each, independently generate vectors in according to the distribution where. Let be the set of the vectors generated, denote these vectors by (12) 2. Let index the vectors in by the set. Since by (4), by taking small enough sufficiently large, we can make Define a mapping (13) as follows. For any, if, then ;if, then is equal to the index of in defined above. We note that for, is a constant vector in,

4 1114 I TRANSACTIONS ON INFORMATION THORY, VOL. 45, NO. 4, MAY 1999 for, are distinct vectors in (for all, by (13), so is properly defined). 3. Let be the conditional distribution of given. Let index the vectors in by the set. For each, define a mapping 2) For each, define a mapping by if otherwise (14) romly as follows. For each, send it through a discrete memoryless channel with transition probability by using the channel times. If the received vector is not in, then, otherwise, is equal to the index of the received vector in defined above. 4. The encoding function where is an arbitrary constant vector in (cf. step 2 of the encoding scheme). 3) The decoding function is defined as where is defined as follows. For each We now analyze the performance of this rom code. Let be any positive quantity. First,, since if for some, then, otherwise, The decoding scheme is described in the following steps: 1) For each, define a mapping we have by taking (15) (16) sufficiently small. Therefore, (17) as follows. Let be the vector in with index (cf. step 3 in the encoding scheme). For any by (10). We now introduce the following notations: if there exists a unique set of indices Define the events such that then otherwise is the event that is -typical with respect to for all. is the event that Decoder decodes correctly the index assigned to. By the weak law of large numbers, we see that for sufficiently large for some (18)

5 YUNG AND ZHANG: DISTRIBUTD SOURC CODING FOR SATLLIT COMMUNICATIONS 1115 Let be the joint distribution of. By (6) (7), we have following. To see that the claim is true, consider (27) at the top of the following page. Note that in in (27) (19) Since is drawn i.i.d. according to the above distribution, we see from the weak law of large numbers that for a sufficiently large (20) By noting that implies, we see that for any Assuming without loss of generality that all (28) for (29) Further, since are independent events, we have (21) (22) Therefore its reciprocal is well-defined. We now bound the probability in (23). By the symmetry of the problem, we assume without loss of generality that for. To simplify notation, define the sets Now for any sufficiently large, we have where. Note that is a partition of, iff Given (23) where the last inequality follows because subset of. Then (30) is always a proper (24) for some Note that the last equality follows from (14) because (25) given. Therefore, (26) The discussion that follows is a variation of the Channel Coding Theorem. We claim that can be obtained by sending through a discrete memoryless channel with transition probability, where is the conditional distribution of given. To simply notation, we assume denote by in the (31) We now prove an identity by considering the information diagram [8] for the rom variables, in Fig. 2. By (8), we see from the diagram that (32) (33)

6 1116 I TRANSACTIONS ON INFORMATION THORY, VOL. 45, NO. 4, MAY 1999 (27) (34) By (6), are independent, so (35) From the diagram, we see that this implies (36) (cf. the definition of the mutual information among more than two rom variables in [8]). We mark each of these atoms with measure zero by an asterisk. Then we see immediately

7 YUNG AND ZHANG: DISTRIBUTD SOURC CODING FOR SATLLIT COMMUNICATIONS 1117 We note that the definition of (recall that )is very similar to the definition of except that 1) is replaced by. 2) The inequality in (4) is strict while the inequality in (44) is nonstrict. It is clear that is a subset of. As a consequence of the above discrepancies, there is a gap between. It is not apparent that the gap between the two regions has zero measure in general. Proof of Theorem 2: Let be admissible. Then for any, there exists for sufficiently large an code such that Fig. 2. The information diagram for (Y j ;j 2 Fmn9); (Y j ;j2 Fm \ 9); (Z l ;l 2 Vm). that (46) (47) We first consider such a code for a fixed. Now for, by Lemma 1 (37) (37) From (30), (31), (38), we have (38) (48) In the above, follows because is a function of, is the vector version of Fano s inequality where denotes the binary entropy function, follows from (47). From (46), we also have (49). Thus for this code, we have (39) when is small enough that is sufficiently large. Finally, from (23) (39), we obtain (40) Hence, from (17) (40), is admissible, which implies. III. OUTR BOUND Let be the set of all such that there exists satisfying the following conditions: (50) for (51) for (52) for (53) for (54) The inequality in (53) is in fact an equality, which follows from the i.i.d. assumption of the source. We note the oneto-one correspondence between (50) (54) (41) (45). By letting, we see that there exists such that Theorem 2: (41) for (42) for (43) for (44) for (45) (55) for (56) for (57) for (58) for (59) It was shown in the recent work of Zhang Yeung [11] that is a convex cone. Therefore, if, then.

8 1118 I TRANSACTIONS ON INFORMATION THORY, VOL. 45, NO. 4, MAY 1999 Dividing (55) (59) by replacing by, we see that there exists such that (60) for (61) for (62) for (63) for (64) We then let to conclude that there exists which satisfies (41) (45), completing the proof. IV. GOMTRICAL INTRPRTATIONS OF AND In Sections II III, are specified in a way which facilitates analysis. In this section, we will present geometrical interpretations of these regions which give further insight. For a set, define for some where we write to mean greater than or equal to componentwise. For a set, define to be the projection of the set on the coordinates. Define the sets (65) Following [10], we define such that for any nonempty ) V. TH LP BOUND to be the set of (recall that (71) (72) (73) (74) These inequalities are referred to as the basic inequalities, which are linear inequalities in. It is easy to see that. Since is a closed set,. Therefore, upon replacing in the definition of by, we immediately obtain an outer bound on. We call this outer bound the LP bound (for linear programming bound) denote it by. Thus (75) The geometrical interpretation of in the last section can also be applied to, so it is not difficult to see that can be evaluated explicitly. The multilevel diversity coding problem (with independent data streams) studied in [9] is a special case of the problem we study in the current paper. For such a problem with levels, there are sources,, whose importance decreases in the order. ach encoder has access to all the sources. Further, Decoder belongs to Level, where, it reconstructs, the most important sources. The problem is specified by for (66) for (67) for (68) Note that the independence relation in (1) is interpreted as the hyperplane in. Similarly, the Markov conditions the functional dependencies in (2) (3) are interpreted as the hyperplanes, respectively. Then we see that So far, the coding rate region of the problem we study in this paper has been determined for certain special cases [5], [7], [9], [13], these are all special cases of multilevel diversity coding. The coding rate region for each of these cases has the form (69) Similarly, we see that (70) The bounds are specified in terms of, respectively. So far, there exists no full characterization of either or [11], [12]. Therefore, although these bounds are in single-letter form, they cannot be evaluated explicitly. Nevertheless, in view of the geometrical interpretation of, one can easily obtain an outer bound on which can be evaluated explicitly. This will be described in the next section. (76) where, is some subset of,, are some nonnegative functions depending only on. Further, the necessity of the coding rate region (i.e., the converse coding theorem) these cases can be proved by invoking the basic inequalities. We claim that the LP bound is tight the special cases for which the coding rate region has the form in (76) the converse coding theorem is a consequence of the basic inequalities. Among these is the -encoder symmetrical

9 YUNG AND ZHANG: DISTRIBUTD SOURC CODING FOR SATLLIT COMMUNICATIONS 1119 multilevel diversity coding (SMDC) problem recently studied by the authors [13]. In the rest of the section, we will prove the tightness of the LP bound for the SMDC problem; the techniques involved in proving the other cases are exactly the same. We further conjecture that the LP bound is tight for any special case as long as the converse coding theorem is a consequence of the basic inequalities, but we cannot prove it. In the following we will use to denote the th component of a vector. In the SMDC problem 1) ; 2) such that ; 3) for. then (78) With the above specifications, we implicitly have. From [13], the coding rate region for the SMDC problem is given by Comparing with (76), we see that for the SMDC problem. Note that the definition of implies for. This can be seen by letting be the -vector whose components are equal to except that the th component is equal to, so that is lower-bounded by. Lemma 2:. Proof: Trivial. In the proof for the necessity of in [13], the authors consider a code with blocklength. Careful examination of the proof for the zero-error case reveals that the same region can be obtained by considering a code with. (This is due to the assumption that the sources are i.i.d.) In the following, we use to denote the collection of rom variables such that. Basically, it is proved in [13] that Proposition 1: For any discrete rom variables,if 1) ; 2) such that, then Note that in the above proposition, the quantities are interpreted as constants. Lemma 3: Propositions 1 2 are equivalent. Proof: We first show that Proposition 1 implies Proposition 2. Proposition 1 says that 1) 2) implies (77). Together with 3), we have since, which proves Proposition 2. We now show that Proposition 2 implies Proposition 1. Suppose 1) 2) are satisfied. We let for so that 3) is satisfied with equality. Then by Proposition 2, we have Thus we see that 1) 2) imply (77), which proves Proposition 1. The constraints 1), 2), 3) in Proposition 2 correspond to the sets,,, respectively, defined in Section IV, which are subsets of. Since the Proof of Proposition 1 in [13] involves only the basic inequalities (i.e., Proposition 1, hence Proposition 2, are implied by the basic inequalities), using the results in [10], we see that So (79) (77). Let us now state another proposition. Proposition 2: For any discrete rom variables,if (80)

10 1120 I TRANSACTIONS ON INFORMATION THORY, VOL. 45, NO. 4, MAY 1999 By projecting onto the coordinates proj, we have for a class of special cases, including all those for which the coding rate region is known. would be tight if, but this has recently been disproved by the authors [11], [12]. Nevertheless, we believe that it is tight for most cases. A problem for future research is to determine the conditions for which is tight. We point out that the rom code we have constructed in Section III has an arbitrarily small probability of error which does not go away even if we are allowed to compress the sources first by variable rate codes. This characteristic is undesirable for many applications. A challenging problem for future research is to construct simple zero-error algebraic codes for distributed source coding. Then ACKNOWLDGMNT The authors wish to thank Reviewer A for the detailed comments on the manuscript. where the last step follows from Lemma 2. Since outer bound on, we have is an which implies. Thus is tight. Compared with, has the advantage that it is more explicit can easily be evaluated. VI. CONCLUSION In this paper, we introduce a new multiterminal source coding problem called the distributed source coding problem. Our model is more general than all previously reported models on multilevel diversity coding [5], [7], [9], [13]. We mention that an even more general model has been formulated in [1]. We have obtained an inner bound an outer bound on the coding rate region. Both are implicit in the sense that they are specified in terms of, respectively, which are fundamental regions in the entropy space yet to be determined. Our work is an application of the results in [10] to information theory problems. We have also obtained an explicit outer bound for the coding rate region. We have shown that this bound is tight RFRNCS [1] R. Ahlswede, N. Cai, S.-Y. R. Li, R. W. Yeung, Network information flow theory, I Trans. Inform. Theory, to be published. [2] T. Berger, Multiterminal source coding, in The Information Theory Approach to Communications, G. Longo, d. New York: Springer- Verlag, [3] T. M. Cover J. A. Thomas, lements of Information Theory. New York: Wiley, [4] I. Csiszár J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, [5] K. P. Hau, Multilevel diversity coding with independent data streams, M.Phil. thesis, The Chinese University of Hong Kong, May [6] F. Matú s M. Studený, Conditional independences among four rom variables I, Combin., Prob. Comput., vol. 4, pp , [7] J. R. Roche, R. W. Yeung, K. P. Hau, Symmetrical multilevel diversity coding, I Trans. Inform. Theory, vol. 43, pp , May [8] R. W. Yeung, A new outlook on Shannon s information measures, I Trans. Inform. Theory, vol. 37, pp , May [9], Multilevel diversity coding with distortion, I Trans. Inform. Theory, vol. 41, pp , May, [10], A framework for linear information inequalities, I Trans. Inform. Theory, vol. 43, pp , Nov [11] Z. Zhang R. W. Yeung, A non-shannon-type conditional inequality of information quantities, I Trans. Inform. Theory, vol. 43, pp , Nov [12], On characterization of entropy function via information inequalities, I Trans. Inform. Theory, vol. 44, pp , July [13] R. W. Yeung Z. Zhang, On symmetrical multilevel diversity coding, I Trans. Inform. Theory, vol. 45, pp , Mar