SYMMETRICAL MULTILEVEL DIVERSITY CODING AND SUBSET ENTROPY INEQUALITIES. A Dissertation JINJING JIANG

Transcription

1 SYMMETRICA MUTIEVE DIVERSITY CODING AND SUBSET ENTROPY INEQUAITIES A Dissertation by JINJING JIANG Submitted to the Office of Graduate Studies of Texas A&M University in partia fufiment of the requirements for the degree of DOCTOR OF PHIOSOPHY Chair of Committee, Tie iu Committee Members, Serap Savari Anxiao Jiang Srinivas Shakkottai Head of Department, Chanan Singh August 2013 Major Subject: Department of Eectrica and Computer Engineering Copyright 2013 Jinjing Jiang

2 ABSTRACT Symmetrica mutieve diversity coding (SMDC) is a cassica mode for coding over distributed storage. In this setting, a simpe separate encoding strategy known as superposition coding was shown to be optima in terms of achieving the minimum sum rate and the entire admissibe rate region of the probem in the iterature. The proofs utiized carefuy constructed induction arguments, for which the cassica subset entropy inequaity of Han payed a key roe. This thesis incudes two parts. In the first part the existing optimaity proofs for cassica SMDC are revisited, with a focus on their connections to subset entropy inequaities. First, a new siding-window subset entropy inequaity is introduced and then used to estabish the optimaity of superposition coding for achieving the minimum sum rate under a weaker source-reconstruction requirement. Second, a subset entropy inequaity recenty proved by Madiman and Tetai is used to deveop a new structura understanding to the proof of Yeung and Zhang onthe optimaity of superposition coding for achieving the entire admissibe rate region. Buiding on the connections between cassica SMDC and the subset entropy inequaities deveoped in the first part, in the second part the optimaity of superposition coding is further extended to the cases where there is an additiona a-access encoder, an additiona secrecy constraint or an encoder hierarchy. ii

3 To my parents

4 ACKNOWEDGEMENTS I woud ike to thank my advisor, Prof. Tie iu, who beieved in me, ed me into the exciting area of information theoretic research and worked with me, side by side, on various probems. I can never forget those hours-ong discussion in his office and even intensive phone cas at eary mornings and midnights when we were at dawn of truy cracking our probem. I can never forget he aways did some pioneer work even before I knew the probem, tod me where to start and made everything so easy to begin with. I can never forget he woud never be satisfied with something we had achieved and aways tried to refine our understanding in a bigger picture, which aways gave me deeper insight and new inspiration. I can never forget he worked harder on my tak and my paper than mysef. I do not beieve I can expect a better advisor and a better roe mode. I truy earn a ot from him not ony as a researcher but aso as a human being. I woud ike to thank Dr. Savari, Dr. Shakkottai and Dr. Jiang for serving on my thesis committee and for providing their thoughtfu comments on my research. I woud ike to thank Hung y, Jae Won Yoo and Neeharika Marukaa for making our office an exceptionay wonderfu pace! Many thanks aso go to my ong-time friends Cong Shen and Yan Zhu for their hep these many years! Finay, I woud ike to extend my greatest gratitude towards my parents and my wife ing. Their endess ove is the ony origin of a my courage. iv

5 TABE OF CONTENTS Page ABSTRACT DEDICATION ACKNOWEDGEMENTS TABE OF CONTENTS IST OF FIGURES ii iii iv v vii 1. INTRODUCTION SYMMETRICA MUTIEVE DIVERSITY CODING REVISITED Probem Statement And Optimaity Of Superposition Coding Probem Statement Superposition Coding Rate Region Optimaity Of Superposition Coding: Known Proofs Minimum Sum Rate Via A Siding-window Subset Entropy Inequaity A Siding-window Subset Entropy Inequaity The Minimum Sum Rate The Subset Entropy Inequaity Of Yeung And Zhang Revisited A Subset Entropy Inequaity Of Madiman And Tetai Connections To The Subset Entropy Inequaities Of Han And Yeung Zhang A Conditiona Subset Entropy Inequaity Of Yeung and Zhang SYMMETRICA MUTIEVE DIVERSITY CODING WITH AN A- ACCESS ENCODER Probem Statement Superposition Coding Rate Region Optimaity Of Superposition Coding Rate Aocation At The A-access Encoder SECURE SYMMETRICA MUTIEVE DIVERSITY CODING Probem Statement Superposition Coding Rate Region Optimaity Of Superposition Coding v

6 5. HIERARCHICA MUTIEVE DIVERSITY CODING Probem Statement Superposition Coding Rate Region Optimaity Of Superposition Coding For Minimum Sum Rate Optimaity Of Superposition Coding Beyond Minimum Sum Rate CONCUSION REFERENCES APPENDIX A. PROOFS OF SUBSET ENTROPY INEQUAITIES A.1 Proof Of Han s Subset Inequaity A.2 Proof Of Madiman Tetai s Inequaity APPENDIX B. PROOF OF THEOREM APPENDIX C. HMDC: 2 STRONG ENCODERS AND 4 WEAK ENCODERS 92 vi

7 IST OF FIGURES FIGURE Page 1.1 An exampe of distributed storage system with faiures Decoding threshod determine the tradeoff between system robustness and storage efficiency The cassica SMDC probem where a tota of independent discrete memoryess sources S 1,...,S are to be encoded by a tota of encoders. The decoder, which has access to a subset U of the encoder outputs, needs to neary perfecty reconstruct the sources S 1,...,S U no matter what the reaization of U is An iustration of the siding windows of ength α when the integers 1,..., are circuary paced (cockwise) based on their natura order An exampe of hypergraph representation of a set U and its coections of subsets V. Here, the eements in U are vertices and the subsets are edges, each of which is represented by distinct coor. Two vertices are connected if they beong to the same subset SMDC with an a-access encoder 0 and randomy accessibe encoders 1 to. A tota of independent discrete memoryess sources (S 1,...,S ) are to be encoded at the encoders. The decoder, which has access to encoder 0 and a subset U of the randomy accessibe encoders, needs to neary perfecty reconstruct the sources (S 1,...,S U ) no matter what the reaization of U is The eft-hand and right-hand sides of (3.21) as a function of λ 0 for a fixed (R 0,R 1,...,R ) (R + ) +1 and λ (R + ) S-SMDCwithrandomyaccessibe encoders 1 to. Atotaof N independent discrete memoryess sources (S 1,...,S N ) are to be encoded at the encoders. The egitimate receiver, which has access to a subset U of the encoder outputs, needs to neary perfecty reconstruct the sources (S 1,...,S U N ) whenever U N +1. The eavesdropper has access to a subset A of the encoder ouputs. A sources (S 1,...,S N ) need to be kept perfecty secret from the eavesdropper whenever A N vii

8 5.1 The HMDC probem where a tota of independent discrete memoryess sources S 1,...,S are to be encoded by a tota of J = A + B encoders of two types. Encoders from 1 to A are strong encoders and the others are weak encoders. The decoder, which has access to a subset U of the encoder outputs, needs to neary perfecty reconstruct the sources S 1,...,S u T w no matter what the reaization of U is A singe source network muticast exampe for coding a singe source S α (α = 2) with 2 strong encoders and 2 weak encoders viii

9 1. INTRODUCTION In recent years, the boom of coud computing appications has mounted great chaenges on the design of the distributed storage systems, where users woud ike to store the information on severa distributed servers. The motivation of this thesis is in terms of the robustness issue in distributed storage systems as shown in Fig Figure 1.1: An exampe of distributed storage system with faiures. 1

10 Particuary, when a network erasure happens temporariy or permanenty due to ink faiure or disk mafunction, it is sti desirabe to recover the information through the erasure-resiient coding schemes. The schemes such as repetition codes or maximum distance separabe (MDS) codes can decode information if at east α out of the tota servers remains avaiabe. Genera speaking, these erasure coding schemes are we understood in the iterature. However, from the system design point of view, the probem eft is how to choose the decoding threshod α, which contros the tradeoff between robustness and efficiency as shown in Fig One extreme is to set α = 1, in which the system is the most robust as ong as there is one node avaiabe. However, apparenty the information shoud be repeated at every node, which may need huge amount of storage in tota. On the contrary, when α =, the system is the most efficient but the east robust since the information is not decodabe if any singe node fais. Meanwhie, not a information are created equa, i.e., some information are more important than others. Take binary representations as an exampe: MSBs are more important than SBs. Due to the information hierarchy, the design that adapts the decoding threshod to the importance of the source is preferred. From the architectura eve, the fundamenta information theoretic question about this adaptive design is which coding scheme is optima, separate encoding or joint encoding. Separate encoding is easy to impement and manage whie joint encoding is potentiay more efficient in terms of minimizing the storage space due to the principe of network coding. Symmetrica mutieve diversity coding (SMDC) is a cassica mode arising from the coding over distributed storage, which was first introduced by Roche [1] and Yeung [2]. In this setting, there are a tota of independent discrete memoryess sources S 1,...,S, where theimportance ofthesource S is assumed to decrease with the subscript. The sources are to be encoded by a tota of randomy accessibe 2

11 α = Efficiency α = 1 Robustness Figure 1.2: Decoding threshod determine the tradeoff between system robustness and storage efficiency. encoders. The goa of encoding is to ensure that the number of sources that can be neary perfecty reconstructed grows with the number of avaiabe encoder outputs at the decoder. More specificay, denote by U Ω := {1,...,} the set of accessibe encoders. The reaization of U is unknown a priori at the encoders. However, the sources S 1,...,S α need to be neary perfecty reconstructed whenever U α at the decoder. The word symmetrica here refers to the fact that the sources that need to be neary perfecty reconstructed depend on the set of accessibe encoders ony via its cardinaity. The rate aocations at different encoders, however, can be 3

12 different and are not necessariy symmetrica. A natura strategy for SMDC is to encode the sources separatey at each of the encoders (no coding across different sources) known as superposition coding [2]. To show that the natura superposition coding strategy is aso optima, however, turned out to be rather nontrivia. The optimaity of superposition coding in terms of achieving the minimum sum rate was estabished by Roche, Yeung, and Hau[3]. The proof used a carefuy constructed induction argument, for which the cassica subset entropy inequaity of Han[4] payed a key roe. ater, the optimaity of superposition coding in terms of achieving the entire admission rate region was estabished by Yeung and Zhang [5]. Their proof was based on a new subset entropy inequaity, which was estabished by carefuy combining Han s subset inequaity with severa highy technica resuts on the anaysis of a sequence of inear programs (which are used to characterize the performance of superposition coding). This thesis incudes two parts. In the first part (Section 2), the optimaity proofs of [3] and [5] are revisited in ight of two new subset entropy inequaities: First, a new siding-window subset entropy inequaity is introduced, which not ony impies the cassica subset entropy inequaity of Han [4] in a trivia way, but aso eads to a new proof of the optimaity of superposition encoding for achieving the minimum sum rate under a weaker source-reconstruction requirement. Second, a subset entropy inequaity recenty proved by Madiman and Tetai [6] is everaged to provide a new structura understanding to the subset entropy inequaity of Yeung and Zhang [5]. Based on this new understanding, a conditiona version of the subset entropy inequaity of Yeung and Zhang [5] is further estabished, which pays a key roe in extending the optimaity of superposition 4

13 coding to the case where there is an additiona secrecy constraint. In the second part of the thesis (Section 3 to Section 5), three extensions of cassica SMDC are considered: The first extension, which we sha refer to as Symmetrica Mutieve Diversity Coding with an A-Access Encoder (SMDC-A), features an a-access encoder, in addition to the randomy accessibe encoders in the cassica setting, whose output is avaiabe at the decoder at a time. This mode is mainy motivated by the proiferation of mobie computing devices (aptop computers, tabets, smart phones etc.), which can access both remote storage nodes via unreiabe wireess inks and oca hard disks which are aways avaiabe but are of imited capacity. It is shown that in this setting, superposition coding remains optima in terms of achieving the entire admissibe rate region. Key to our proof is to identify the supporting hyperpanes that define the superposition coding rate region and then appy the subset entropy inequaity of Yeung and Zhang [5]. The second extension, which we sha refer to as Secure Mutieve Diversity Coding (S-SMDC), extends the probem of SMDC to the secure communication setting. The probem was first introduced in [7], where the optimaity of superposition coding for achieving the minimum sum rate was estabished via the cassica subset entropy inequaity of Han [4]. Through the conditiona version of the subset entropy inequaity of Yeung and Zhang [5] estabished in the first part, here we show that superposition coding can, in fact, achieve the entire admissibe rate region of the probem, resoving the conjecture of [7] by positive. The third extension, which we sha refer as Hierarchica Mutieve Diversity Coding (HMDC), further extends the probem of SMDC to a specia asymmet- 5

14 rica setting. This mode is a natura generaization of symmetrica mutieve diversity coding to heterogeneous distributed storage systems. Reca that in symmetrica mutieve diversity coding, the number of messages that needs to be recovered by the decoder depends on the avaiabe subset of the encoder outputs ony via its cardinaity. Therefore, the underying assumption for symmetrica mutieve diversity coding is that a distributed storage nodes have the same reiabiity. For heterogeneous distributed storage systems, one may associate each storage node with a reiabiity score and design a coding scheme such that the number of messages that needs to be recovered by the decoder depends on the avaiabe subset of the encoder outputs via its accumuative reiabiity score. Then the encoders are cassified into different ranks due to their different reiabiity scores. In our setting, the encoders show two reiabiity scores. The encoders with higher reiabiity score are referred as the strong encoders, whie the others are weak encoders. With this encoder hierarchy, it is shown that superposition coding remains optima in achieving the minimum sum rate but the optimaity of achieving the entire admissibe rate region is sti unknown. We conjecture that superposition coding is optima in achieving the entire admissibe rate region. 6

15 2. SYMMETRICA MUTIEVE DIVERSITY CODING REVISITED 2.1 Probem Statement And Optimaity Of Superposition Coding Probem Statement As iustrated in Figure 2.1, the probem of SMDC consists of: a tota of independent discrete memoryess sources {S α [t]} t=1, where α = 1,..., and t is the time index; a set of encoders (encoder 1 to ); a decoder which can access a nonempty subset U Ω of the encoder outputs. The reaization of U is unknown a priori at the encoders. However, no matter which U actuay materiaizes, the decoder needs to neary perfecty reconstruct the sources S 1,...,S α whenever U α. R 1 Encoder 1 X 1 Sources (S1,...,S n ) n R 2 Encoder 2... X 2 X U Decoder (Ŝn 1,...,Ŝn U ) R Encoder X Figure 2.1: The cassica SMDC probem where a tota of independent discrete memoryess sources S 1,...,S are to be encoded by a tota of encoders. The decoder, which has access to a subset U of the encoder outputs, needs to neary perfecty reconstruct the sources S 1,...,S U no matter what the reaization of U is. 7

16 Formay, an (n,(m 1,...,M )) code is defined by a coection of encoding functions: e : Sα n {1,...,M }, = 1,..., (2.1) and 2 1 decoding functions: d U : U } Sα U{1,...,M n, U Ω s.t. U. (2.2) Anonnegative ratetupe(r 1,...,R )issaidto beadmissibe ifforevery ǫ > 0, there exits, for sufficienty arge bock-ength n, an (n,(m 1,...,M )) code such that: (Rate constraints at the encoders) 1 n ogm R +ǫ, = 1,...,; (2.3) (Asymptoticay perfect reconstructions at the decoder) Pr { d U (X U ) (S n 1,...,Sn U )} ǫ, U Ω s.t. U (2.4) where Sα n := {S α[t]} n t=1, X := e (S1 n,...,sn ) is the output of encoder, and X U := {X : U}. Theadmissiberate region Risthecoectionofa admissiberatetupes(r 1,...,R ). The minimum sum rate R ms is defined as R ms := min (R 1,...,R ) R R. (2.5) =1 8

17 2.1.2 Superposition Coding Rate Region As mentioned previousy, a natura strategy for SMDC is superposition coding, i.e., toencodethesourcesseparateyattheencodersandthereisno codingacrossdifferent sources. Formay, the probem of encoding a singe source S α can be viewed as a specia case of the genera SMDC probem, where the sources S m are deterministic for a m α. In this case, the source S α needs to be neary perfecty reconstructed whenever the decoder can access at east α encoder outputs. Thus, the probem is essentiay to transmit S α over an erasure channe, and the foowing simpe source-channe separation scheme is known to be optima [1, 2]: Firstcompressthesourcesequence Sα n intoasourcemessage W α using aossess source code. It is we known [8, Ch. 5] that the rate of the source message W α can be made arbitrariy cose to the entropy rate H(S α ) for sufficienty arge bock-ength n. Next, the source message W α is encoded at encoders 1 to using a maximum distance separabe code [9]. It is we known [1,2] that the source message W α can be perfecty recovered at the decoder whenever U R 1 n H(W α), U Ω (α) (2.6) for sufficienty arge bock ength n, where Ω (α) subsets of Ω of size α. denotes the coection of a Combining the above two steps, we concude that the admissibe rate region for encoding a singe source S α is given by the coection of a nonnegative rate tupes 9

18 (R 1,...,R ) satisfying U R H(S α ), U Ω (α). (2.7) By definition, the superposition coding rate region R sup for encoding the sources S 1,...,S is given by the coection of a nonnegative rate tupes (R 1,...,R ) such that R := r (α) (2.8) for some nonnegative r (α), α = 1,..., and = 1,...,, satisfying U r (α) H(S α ), U Ω (α). (2.9) In principe, an expicit characterization of the superposition coding rate region R sup can be obtained by eiminating r (α), α = 1,..., and = 1,...,, via a Fourier-Motzkin eimination from(2.8) and(2.9). However, the eimination process is unmanageabe even for moderate, as there are simpy too many equations invoved. On the other hand, note that the superposition coding rate region R sup is a convex poyhedron with poyhedra cone being (R + ), so an equivaent characterization is to characterize the supporting hyperpanes: where λ R f(λ), λ := (λ 1,...,λ ) (R + ) (2.10) =1 f(λ) = min (R 1,...,R ) R sup λ R. (2.11) =1 10

19 To soving for f(λ), (2.11) can be expicity written as the foowing inear program, min ( =1 λ ) r(α) subject to U r(α) r (α) H(S α ), U Ω (α) and α = 1,..., 0, α = 1,..., and = 1,...,. (2.12) The inear program can be further written as min ( ) =1 λ r (α) subject to U r(α) r (α) H(S α ), U Ω (α) and α = 1,..., 0, α = 1,..., and = 1,...,. (2.13) Ceary, the above optimization probem can be separated into the foowing sub-optimization probems: where f(λ) = f α(λ) (2.14) f α(λ) = min =1 λ r (α), (2.15) subject to U r (α) H(S α ), U Ω (α), (2.16) r (α) 0, = 1,...,. (2.17) 11

20 The minimization of f α(λ) can be expicity written as the foowing inear program max ( U Ω (α) ) c λ (U) H(S α ) subject to {U Ω (α) :U }c λ(u) λ, = 1,..., c λ (U) 0, U Ω (α). (2.18) and (2.18) foows from the strong duaity for inear programs. For any λ (R + ) and any α = 1,...,, et f α (λ) := max U Ω (α) c λ (U) subject to {U Ω (α) :U }c λ(u) λ, = 1,..., c λ (U) 0, U Ω (α). (2.19) Then, we have f α(λ) = f α (λ)h(s α ) and hence f(λ) = f α (λ)h(s α ) (2.20) for any λ (R + ). Substituting (2.20) into (2.10), we concude that the superposition coding rate region R sup is given by the coection of nonnegative rate tupes (R 1,...,R ) satisfying λ R =1 f α (λ)h(s α ), λ (R + ). (2.21) For a genera λ, the inear program (2.19) does not admit a cosed-form soution. However, for λ = 1 := (1,...,1) it can be easiy verified that c (α) 1 = {c 1 (U) : U Ω (α) } where c 1 (U) := ( 1 1 ) (2.22) α 1 12

21 is an optima soution to the inear program (2.19), and we thus have f α (1) = U Ω (α) c 1 (U) = ( α) ( 1 ) = α α 1 (2.23) for any α = 1,...,. Hence, the minimum sum rate that can be achieved by superposition coding is given by min (R 1,...,R ) R sup R = f(1) = =1 f α (1)H(S α ) = (/α)h(s α ). (2.24) Optimaity Of Superposition Coding: Known Proofs To show that superposition coding is optima in terms of achieving the entire admissibe rate region, we need to show that for any λ (R + ) we have λ R f α (λ)h(s α ), (R 1,...,R ) R. (2.25) =1 In particuar, to show that superposition coding is optima in terms of achieving the minimum sum rate, we need to show that R f α (1)H(S α ) = (/α)h(s α ), (R 1,...,R ) R. (2.26) =1 Note that for any admissibe rate tupe (R 1,...,R ) R and ǫ > 0, by the rate constraints (2.3) we have n(r +ǫ) H(X ), = 1,..., (2.27) for sufficienty arge bock-ength n. Furthermore, by the asymptoticay perfect 13

22 reconstruction requirement (2.4) and the we-known Fano s inequaity we have H(S n 1,...,S n α X U ) nδ (n) α (2.28) for any U Ω (α) Thus, for any V Ω (α 1) and α = 1,...,, where δ(n) α 0 in the imit as n and ǫ 0. we have H(X V S n 1,...,Sn α 2 ) = H(X V S n 1,...,Sn α 1 )+I(X V;S n α 1 Sn 1,...,Sn α 2 ) (2.29) = H(X V S n 1,...,Sn α 1 )+H(Sn α 1 Sn 1,...,Sn α 2 ) H(S n α 1 Sn 1,...,Sn α 2,X V) (2.30) H(X V S n 1,...,S n α 1)+H(S n α 1) H(S n 1,...,S n α 1 X V ) (2.31) H(X V S n 1,...,Sn α 1 )+nh(s α 1) nδ (n) α 1 (2.32) where (2.31) foows from the facts that a sources are independent so H(S n α 1 S n 1,...,S n α 2) = H(S n α 1) and that H(S n α 1 S n 1,...,S n α 2,X V ) = H(S n 1,...,S n α 1 X V ) H(S n 1,...,S n α 2 X V ) H(S n 1,...,Sn α 1 X V). (2.33) Therefore, starting with (2.27) and appying (2.32) iterativey may ead us towards a proof of (2.25) and (2.26). Note, however, that to appy (2.32) iterativey we sha need to bound from beow H(X V S n 1,...,Sn α 1 ) in terms of H(X U S n 1,...,Sn α 1 ) for 14

23 some U Ω (α). The key observation of [3] and [5] is that such bounds exist, not for an arbitrary individua pair of U and V, but rather at the eve of an appropriate averaging among V Ω (α 1) and U Ω (α). More specificay, [3] considered the cassica subset entropy inequaity of Han [4], which can be written as foows. Theorem 1 (A subset entropy inequaity of Han [4]). For any coection of jointy distributed random variabes (X 1,...,X ), we have 1 ( ) α 1 V Ω (α 1) H(X V ) α 1 1 ) ( α U Ω (α) H(X U ) α (2.34) for any α = 2,...,. Essentiay, considering the average joint entropy of a subsets of fixed size, (2.34) says that the average joint entropy per eement decreases with the size of the subsets. The proof of Theorem 1 can be found in Appendix A. Iterativey appying (2.32) and (2.34), we may obtain 1 =1 H(X ) = ( 1 ) H(X V ) (2.35) 1 V Ω (1) m m ( 1 ) H(X U S1,...,S n m) n m m U Ω (m) +n H(S α ) α for any m = 1,...,. In particuar, et m =, and we have 1 =1 H(X ) ( 1 ) H(X U S1,...,S n n) U Ω () n H(S α ) α n 15 +n H(S α ) α n n δ (n) α α δ (n) α α (2.36) δ (n) α α. (2.37)

24 Substituting (2.27) into (2.37) and dividing both sides of the inequaity by n, we have 1 (R +ǫ) =1 H(S α ) α δ (n) α α. (2.38) Finay, etting n and ǫ 0 competes the proof of (2.26), i.e., superposition coding can achieve the minimum sum rate for the genera SMDC probem. To prove that superposition coding can in fact achieve the entire admissibe rate region, Yeung and Zhang [5] proved the foowing key subset entropy inequaity. Theorem 2 (A subset entropy inequaity of Yeung and Zhang [5]). For any λ (R + ), there exists a function c λ : 2 Ω \ R + such that: 1) for each α = 1,...,, c (α) λ := {c λ(u) : U Ω (α) } is an optima soution to the inear program (2.19); and 2) for each α = 2,...,, V Ω (α 1) c λ (V)H(X V ) U Ω (α) c λ (U)H(X U ) (2.39) for any coection of jointy distributed random variabes (X 1,...,X ). Iterativey appying (2.32) and (2.39), we may obtain V Ω (1) c λ (V)H(V) n U Ω (m) c λ (U)H(X U S n 1,...,Sn m )+ m f α (λ)h(s α ) n m f α (λ)δ (n) α (2.40) for any m = 1,...,. In particuar, et m =, and note that for α = 1 the optima 16

25 soution to the inear program (2.19) is unique and is given by c λ ({}) = λ, Ω. (2.41) We have λ H(X ) =1 n U Ω () c λ (U)H(X U S n 1,...,Sn )+ f α (λ)h(s α ) n n f α (λ)h(s α ) n f α (λ)δ (n) α (2.42) f α (λ)δ (n) α. (2.43) Substituting (2.27) into (2.43) and dividing both sides of the inequaity by n, we have λ (R +ǫ) =1 f α (λ)h(s α ) f α (λ)δ (n) α. (2.44) Finay, etting n and ǫ 0 competes the proof of (2.25), i.e., superposition coding can achieve the entire admissibe rate region for the genera SMDC probem. 2.2 Minimum Sum Rate Via A Siding-window Subset Entropy Inequaity In this section, we prove a new siding-window subset entropy inequaity and then use it to provide an aternative proof of the optimaity of superposition coding for achieving the minimum sum rate. 17

26 For any integer et A Siding-window Subset Entropy Inequaity mod, if mod 0 :=, if mod = 0 (2.45) and for any = 1,..., and α = 1,..., et W (α) := {, +1,..., +α 1 }. (2.46) As iustrated in Figure 2.2, W (α) represents a siding window of ength α starting withwhentheintegers1,...,arecircuarypaced(cockwiseorcountercockwise) based on their natura order. We have the foowing siding-window subset entropy inequaity. Theorem 3 (A siding-window subset entropy inequaity). For any coection of jointy distributed random variabes (X 1,...,X ), we have =1 H(X (α 1) W ) α 1 =1 H(X (α) W ) α (2.47) for any α = 2,...,. The equaities hod when X 1,...,X are mutuay independent of each other. 18

27 W (α) 1 2 +α 1 W (α) 1 α Figure 2.2: An iustration of the siding windows of ength α when the integers 1,..., are circuary paced (cockwise) based on their natura order. Proof. Consider a proof via an induction on α. First, for α = 2 we have =1 H(X (1) W ) = = = H(X ) (2.48) =1 =1 =1 =1 H(X )+H(X +1 ) 2 H(X,X +1 ) 2 H(X (2) W ) 2 (2.49) (2.50) (2.51) where (2.50) foows from the independence bound on entropy. Next, assume that the inequaity (2.47) hods for α = r for some r {2,..., 19

28 1}, i.e., =1 H(X (r 1) W ) r 1 H(X (r) W ). (2.52) r =1 We have =1 H(X (r) W ) = = 1 2 = [ ] H(X (r) W )+H(X (r) W ) +1 =1 [ =1 =1 =1 =1 H(X W (r+1) H(X (r+1) W )+ 1 2 H(X (r+1) W )+ 1 2 ] )+H(X (r 1) W ) +1 =1 =1 H(X (r+1) W )+ 1 2 r 1 r (2.53) (2.54) H(X (r 1) W ) (2.55) +1 H(X (r 1) W ) (2.56) where (2.54) foows from the submoduarity of entropy [14, Ch. 14.A] =1 H(X (r) W ) (2.57) H(X U )+H(X V ) H(X U V )+H(X U V ) (2.58) for U = W (r) and V = W (r) +1 so U V = W(r+1) and U V = W (r 1) +1, and (2.57) foows from the induction assumption (2.52). Moving the second term on the right-hand side of (2.57) to the eft and mutipying both sides by 2, we have r+1 1 r =1 H(X (r) W ) 1 r +1 =1 H(X (r+1) W ). (2.59) We have thus proved that the inequaity (2.47) aso hods for α = r

29 Finay, note that when X 1,...,X are mutuay independent, we have =1 H(X (α) W ) α = H(X ), α = 1,...,. (2.60) =1 This competes the proof of Theorem 3. Note that for α =, the cassica subset entropy inequaity of Han (2.34) and the siding-window subset entropy inequaity (2.47) are equivaent, and both can be equivaenty written as 1 1 H(X Ω \{}) H(X Ω ). (2.61) =1 For a genera α, the cassica subset entropy inequaity of Han (2.34) can be derived from the siding-window subset entropy inequaity (2.47) via a simpe permutation argument as foows. et π be a permutation on Ω. For any = 1,..., and α = 1,...,, et W (α) π, := {π 1 (),π 1 ( +1 ),...,π 1 ( +α 1 )}. (2.62) By Theorem 3, we have 1 α 1 =1 H(X (α 1) W ) 1 π, α =1 H(X (α) W ) (2.63) π, for any α = 2,...,. Averaging (2.63) over a possibe permutations π, we have 1! [ π 1 α 1 =1 H(X (α 1) W ) π, ] 1! π [ 1 α =1 H(X (α) W ) π, ]. (2.64) 21

30 Note that for any α = 1,...,, π =1 H(X W (α) π, ) = α!( α)! U Ω (α) H(X U ). (2.65) Substituting (2.65) into(2.64) and dividing both sides of the inequaity by estabish the cassica subset entropy inequaity of Han (2.34) The Minimum Sum Rate The siding-window subset entropy inequaity (2.47) can be used to provide an aternative proof of the optimaity of superposition coding for achieving the minimum sum rate as foows. et us first show that 1 H(X ) = 1 =1 1 =1 =1 H(X (1) W ) (2.66) H(X (m) W S1 n,...,sn m ) m +n m H(S α ) α n m δ (n) α α (2.67) for any m = 1,...,. Consider a proof via an induction on m. When m = 1, (2.67) can be written as 1 H(X ) 1 =1 =1 H(X S n 1 )+nh(s 1) nδ (n) 1 (2.68) which can be obtained via a uniform averaging of (2.32) for α = 2 and V = {} for = 1,...,. Now assume that the inequaity (2.67) hods for m = r 1 for some 22

31 r {2,...,}. We have 1 H(X ) 1 =1 1 =1 =1 H(X (r 1) W S1,...,S n r 1) n r 1 H(X (r) W S1 n,...,sn r 1 ) r r 1 +n r 1 +n H(S α ) α H(S α ) α r 1 n r 1 n δ (n) α α δ (n) α α (2.69) (2.70) where (2.70) foows from the siding-window subset entropy inequaity (2.47) with α = r. etting α = r+1 and V = W (r) in (2.32), we have H(X W (r) S1 n,...,sn r 1 ) H(X S W (r) 1 n,...,sn r )+nh(s r) nδ r (n). (2.71) Substituting (2.71) into (2.70) gives 1 H(X ) 1 =1 =1 H(X (r) W S1 n,...,sn r ) r +n r H(S α ) α n r δ (n) α α. (2.72) This competes the proof of the induction step and hence (2.67). Now et m =, and we have 1 H(X ) 1 =1 n =1 H(X () W S1 n,...,sn ) H(S α ) α n +n H(S α ) α n δ (n) α α (2.73) δ (n) α α. (2.74) Substituting (2.27) into (2.74) and dividing both sides of the inequaity by n, we have 1 (R +ǫ) =1 H(S α ) α δ (n) α α. (2.75) Finay, etting n and ǫ 0 competes the proof of (2.26), i.e., superposition coding can achieve the minimum sum rate for the genera SMDC probem. 23

32 Note that unike the origina proof of [3], which uses the cassica subset entropy inequaity of Han [4] and hence invoves a nonempty subsets U of Ω, our proof reies on the siding-window subset entropy inequaity (2.47) and hence ony invoves the subsets U of a siding-window type, i.e., U = W (α) for some = 1,..., and α = 1,...,. Therefore, based on our proof, the converse resut (2.26) remains to be true even if we weaken the asymptoticay perfect reconstruction requirement (2.4) to Pr { d U (X U ) (S n 1,...,Sn U )} ǫ, U { } W (α) : = 1,..., and α = 1,...,. (2.76) This is the definitive advantage of our proof over that based on the cassica subset entropy inequaity of Han [4]. 2.3 The Subset Entropy Inequaity Of Yeung And Zhang Revisited In this section, we revisit the subset entropy inequaity of Yeung and Zhang(2.39), which payed a key in their proof [5] of the optimaity of superposition coding for achieving the entire admissibe rate region of the probem. As mentioned previousy, in [5] the subset entropy inequaity (2.39) was proved by combining the cassica subset entropy inequaity of Han [4] and a number of anaysis resuts on the sequence of inear programs (2.19). However, the inequaity, as stated in Theorem 2, does not even directy impy the cassica subset entropy inequaity of Han [4]. The reason is that Theorem 2 merey asserts the existence of a set of optima soutions c (α) λ, α = 1,...,, thatsatisfiesthesubsetentropyinequaity (2.39),ratherthanproviding a sufficient condition for the inequaity to hod. Beow, we sha use a subset entropy inequaity recenty proved by Madiman and Tetai [6] to summarize the anaysis resuts of [5] on the sequence of inear programs (2.19) into a succinct sufficient 24

33 condition for the subset entropy inequaity (2.39) to hod A Subset Entropy Inequaity Of Madiman And Tetai Consider a hypergraph (U,V) where U is a finite ground set and V is a coection of subsets of U. An exampe is shown in Fig e 1 2 e e U = {1,2,3,4,5,6,7} V = {e 1,e 2,e 3,e 4 } = {{1,2,3},{2,3},{3,5,6},{4}} Figure 2.3: An exampe of hypergraph representation of a set U and its coections of subsets V. Here, the eements in U are vertices and the subsets are edges, each of which is represented by distinct coor. Two vertices are connected if they beong to the same subset. 25

34 A function g : V R + is caed a fractiona cover of (U,V) if it satisfies g(v) 1, i U. (2.77) {V V:V i} Theorem 4 (A subset entropy inequaity of Madiman and Tetai [6]). et (U,V) be a hypergraph, and et g be a fractiona cover of (U,V). Then g(v)h(x V ) H(X U ) (2.78) V V for any coection of jointy distributed random variabes X U. The proof of Theorem 4 can be found in Appendix A. The foowing coroary provides a chain form of the subset entropy inequaity (2.78). et M be a positive integer, and et Σ be a finite ground set. et Σ (α) be a coection of subsets of Σ for each α = 1,...,M,. Assuming that Σ (α), α = 1,...,M, are mutuay excusive, then {Σ (α) : α = 1,...,M} induces a coection of hypergraphs {(U,V U ) : U M α=2 Σ(α) } where V U := {V Σ (α 1) : V U}, U Σ (α). (2.79) We sha term each subset V V U a chid of U. For convenience, we sha aso define U V := {U Σ (α) : U V}, V Σ (α 1) (2.80) and term each subset U U V a parent of V. Coroary 1. et c : M Σ(α) R +. For any α = 2,...,M, if there exists a 26

35 coection of functions {g U : U Σ (α) } for which each g U is a fractiona cover of (U,V U ) and such that c(v) = U U V g U (V)c(U), V Σ (α 1) (2.81) we have c(v)h(x V ) c(u)h(x U ) (2.82) V Σ (α 1) U Σ (α) for any coection of jointy distributed random variabes X Σ. Proof. Fix α {2,...,M}. For any U Σ (α), g U is a fractiona cover of (U,V U ). By the subset entropy inequaity of Madiman and Tetai (2.78), we have g U (V)H(X V ) H(X U ), U Σ (α). (2.83) V V U Mutipying both sides of (2.83) by c(u) and summing over U Σ (α), we have Note that U Σ (α) U Σ (α) c(u)g U (V)H(X V ) c(u)h(x U ). (2.84) V V U U Σ (α) V V U c(u)g U (V)H(X V ) = ( V Σ (α 1) U U V g U (V)c(U) ) H(X V ) (2.85) = c(v)h(x V ) (2.86) V Σ (α 1) where (2.86) foows (2.81). Substituting (2.86) into (2.84) competes the proof of the coroary. 27

36 2.3.2 Connections To The Subset Entropy Inequaities Of Han And Yeung Zhang Specifying Σ = Ω, M =, and Σ (α) = Ω (α) for α = 1,...,, the subset entropy inequaity of Madiman and Tetai can be used to provide a unifying proof for both the subset entropy inequaity of Han and the subset entropy inequaity of Yeung and Zhang. Note that the choice {Σ (α) = Ω (α) : α = 1,...,} is reguar in that each subset U Ω (α) has exacty α chidren in Ω(α 1), and each subset V Ω (α 1) exacty (α 1) parents in Ω (α). To see how the subset entropy inequaity of Madiman and Tetai (2.78) impies the subset entropy inequaity of Han (2.34), et has c(u) := 1 α ( ), U Ω (α) α and α = 1,..., (2.87) and g U (V) := 1, U Ω(α) α 1, V V U, and α = 2,...,. (2.88) For any α = 2,..., and U Ω (α), {V V U :V i} g U (V) = {V V U : V i} α 1 = α 1 = 1, i U (2.89) α 1 so g U is a uniform fractiona cover of (U,V U ). Furthermore, for any α = 2,..., and V Ω (α 1) we have U U V g U (V)c(U) = U V (α 1)α ( ) = (α 1) (α 1)α ( ) = α α 1 (α 1) ( ) = c(v). (2.90) α 1 Substituting (2.87) into (2.82) immediatey gives the subset entropy inequaity of 28

37 Han (2.34). To see how the subset entropy inequaity of Madiman and Tetai (2.78) impies the subset entropy inequaity of Yeung and Zhang (2.39), we sha need the foowing resut, which is a synthesis of the anaytica resuts on the sequence of inear programs (2.19) estabished in [5]. (For competeness, a sketched proof based on the resuts of [5] is incuded in Appendix B.) Theorem 5 (A inear programing resut of Yeung and Zhang [5]). For any λ (R + ), any α = 2,...,, and any c (α) λ which is an optima soution to the inear program (2.19) with the optima vaue f α (λ) > 0, there exists a coection of functions {g U : U Ω (α) } for which each g U is a fractiona cover of (U,V U ) and such that c (α 1) λ = {c λ (V) : V Ω (α 1) } where c λ (V) := U U V g U (V)c λ (U) (2.91) is an optima soution to the inear program (2.19) with α repaced by α 1. Now fix λ (R + ), and consider the foowing construction of c λ = c (α) λ. For α =, choose c () λ (2.19). For α = 1,..., 1, construct c (α) λ to be an arbitrary optima soution to the inear program is aready in pace for some α = 2,..., such that c (α) λ iterativey as foows. Suppose that c(α) λ is an optima soution to the inear program (2.19). If the optima vaue f α (λ) > 0, construct c (α 1) λ (2.81) so c (α 1) λ according to is an optima soution to the inear program (2.19) with α repaced by α 1. Moreover, by Coroary1c (α 1) λ andc (α) λ satisfythesubset entropy inequaity of Yeung and Zhang (2.39). If, on the other hand, f α (λ) = 0, we have c λ (U) = 0 for a U Ω (α). In this case, choose c(α 1) λ to be an arbitrary optima soution to the inear program (2.19) with α repaced by α 1, and c (α 1) λ and c (α) λ wi triviay satisfy the subset entropy inequaity of Yeung and Zhang (2.39). We have thus constructed for 29

38 any λ (R + ), a sequence of c (α) λ, α = 1,...,, such that each c(α) λ is an optima soution to the inear program (2.19), and the subset entropy inequaity of Yeung and Zhang (2.39) hods for each α = 2,...,. We mention here that even though both the subset entropy inequaity of Han and the subset entropy inequaity of Yeung and Zhang can be directy estabished from the subset entropy inequaity of Madiman and Tetai, this is not the case for the siding-window subset entropy inequaity (2.47) except for α = 2 and. This can be seen as foows. et Σ = Ω, M =, and Σ (α) = {W (α) that for any α = 1,..., 1, each siding window W (α) : = 1,...,} for α = 1,...,. Note represents a different subset for different. (For α =, a siding windows W (), = 1,...,, represent the same subset Ω.) Furthermore, for any α = 2,..., 1 each siding window W (α) has ony two chidren: W (α 1) two parents: W (α) and W (α 1) +1, and each siding window W(α 1) has ony and W (α) 1. Now consider the eements and + α 1 from W (α). Note that among the two chidren W (α 1) and W (α 1) +1 of W (α), beongs ony to W (α 1), and +α 1 beong ony to W (α 1) +1. Thus, any fractiona cover g W (α) of the hypergraph (W (α),w (α 1) }) must satisfy,{w (α 1) +1 g W (α) (W (α 1) ) 1 and g W (α) (W (α 1) +1 ) 1. (2.92) Now et c(w (α) ) := 1/α for a = 1,..., and α = 1,..., 1. We have g W (α) (W (α 1) )c(w (α) )+g (α) W 1 (W (α 1) )c(w (α) 1 ) 2 α > 1 α 1 = c(w(α 1) ) (2.93) for any α > 2. We thus concude that for any 2 < α <, the siding-window subset entropy inequaity (2.47) cannot be directy inferred from the subset entropy 30

39 inequaity of Madiman and Tetai A Conditiona Subset Entropy Inequaity Of Yeung and Zhang We concude this section by providing a conditiona extension of the subset entropy inequaity of Yeung and Zhang, which wi pay a key roe in proving the optimaity of superposition coding for achieving the entire admissibe rate region of the genera S-SMDC probem. We sha start with the foowing generaization of Coroary 1. et Σ be a finite ground set, and et Σ (α), α = 1,...,M, be a coection of subsets of Σ. As before, we sha assume that the coections Σ (α), α = 1,...,M, are mutuay excusive, so {Σ (α) : α = 1,...,M} induces a hypergraph (U,V U ) for every U M α=2σ (α). For each U Σ (M) et A U be a coection of subsets of Σ, and et A (M) := {A U : U Σ (M) }. For α = 1,...,M 1, define A (α) := {A U : U Σ (α) } iterativey as foows. Suppose that A (α) is aready in pace for some α = 2,...,M. et A (α 1) = {A V : V Σ (α 1) } where A V := U UV A U. (2.94) Proposition 1. For each U M Σ (α), et s(u, ) : A U R +. For any α = 2,...,M, if there exists a coection of functions {g U : U Σ (α) } for which each g U is a fractiona cover of (U,V U ) and such that s(v,a) = g U (V)s(U,A), V Σ (α 1) and A A V (2.95) {U U V :A U A} 31

40 we have A A V s(v,a)h(x V X A ) V Σ (α 1) U Σ (α) A A U s(u,a)h(x U X A ) (2.96) for any coection of jointy distributed random variabes X Σ. Proof. Fix α {2,...,M}. For any U Σ (α), g U is a fractiona cover of (U,V U ). By the subset entropy inequaity of Madiman and Tetai (2.78), we have g U (V)H(X V X A ) H(X U X A ), U Σ (α) and A A U. (2.97) V V U Mutipying both sides of (2.97) by s(u,a) and summing over A A U and U Σ (α), we have U Σ (α) A A U U Σ (α) Note that U Σ (α) A A U V V U s(u,a)g U (V)H(X V X A ) s(u,a)g U (V)H(X V X A ) V V U = U Σ (α) V V U = V Σ (α 1) U U V = V Σ (α 1) A A V = V Σ (α 1) A A U s(u,a)h(x U X A ). (2.98) s(u,a)g U (V)H(X V X A ) (2.99) A A U s(u,a)g U (V)H(X V X A ) (2.100) A A U s(u,a)g U (V) H(X V X A ) (2.101) {U U V :A U A} A A V s(v,a)h(x V X A ) (2.102) where (2.102) foows from (2.95). Substituting (2.102) into (2.98) competes the 32

41 proof of the proposition. Theorem 6 (A conditiona subset entropy inequaity of Yeung and Zhang). For any λ (R + ) and N = 0,..., 1, there exists for each U N Ω(α) subsets A U of Ω such that: a coection of A = N and A U =, A A U (2.103) and a function s λ (U, ) : A U R + such that: 1) for each α = 1,..., N, c (α) λ = {c λ(u) : U Ω (α) } where c λ (U) := is an optima soution to the inear program (2.19); and A A U s λ (U,A) (2.104) 2) for each α = 2,..., N, V Ω (α 1) A A V s(v,a)h(x V X A ) U Ω (α) A A U s(u,a)h(x U X A ) (2.105) for any coection of jointy distributed random variabes (X 1,...,X ). Proof. Fix λ (R + ) and N {0,..., 1}, and et Σ = Ω, M = N, and Σ (α) = Ω (α) for α = 1,..., N. Consider the foowing construction of A(α) and s (α) λ := {s(u, ) : U Ω(α) }, α = 1,..., N. For α = N, et A ( N) = {A U : U Ω ( N) } where A U := {Ω \ U}, i.e., each A U contains a singe subset A = Ω \ U of size A = ( N) = N and such that A U =. Furthermore, et c ( N) λ = {c λ (U) : U Ω ( N) } be an optima 33

42 soution to the inear program (2.19) for α = N, and et s λ (U,Ω \U) := c λ (U), U Ω ( N). (2.106) Since by construction each A U, U Ω ( N), contains a singe subset A = Ω \ U, we triviay have A A U s λ (U,A) = c λ (U), U Ω ( N). (2.107) For α = 1,..., N 1, et us construct A (α) and s (α) λ iterativey as foows. Suppose that A (α) and s (α) λ are aready in pace for some α = 2,..., N such that A = N and A U = for any U Ω (α) and A A U, and c (α) λ = {c λ(u) : U Ω (α) } where c λ (U) is given by (2.104) is an optima soution to the inear program (2.19). First, construct A (α 1) according to (2.94). Based on this construction, for any V Ω (α 1) and A A V we have A A U for some U U V Ω (α). Therefore, by the induction assumption we must have A = N and A V A U = (2.108) for any V Ω (α 1) and A A V. Next, construct s (α 1) λ as foows. If the optima vaue f α (λ) > 0, by Theorem 5 there exists a coection of functions {g U : U Ω (α) } for which each g U is a fractiona cover of (U,V U ) and such that c (α 1) λ = {c λ (V) : V Ω (α 1) } where c λ (V) is given by (2.91) is an optima soution to the inear program (2.19) with α repaced by α 1. In this case, et s (α 1) λ = {s λ (V, ) : V Ω (α 1) } 34

43 where s λ (V,A) := g U (V)s λ (U,A). (2.109) {U U V :A U A} Thus, for each V Ω (α 1) we have s λ (V,A) = A A V g U (V)s λ (U,A) (2.110) A A V = {U U V :A U A} [ g U (V) s λ (U,A) U U V A A U = ] (2.111) g U (V)c λ (U) (2.112) U U V = c λ (V) (2.113) Furthermore, by Proposition 1 s (α 1) λ and s (α) λ satisfy the subset entropy inequaity (2.105). If, on the other hand, f α (λ) = 0, we have s λ (U,A) = 0 for a U Ω (α) and A A U. In this case, choose an arbitrary s (α 1) λ Ω (α 1) } where c λ (V) := such that c (α 1) λ = {c λ (V) : V A A V s λ (V,A) (2.114) is an optima soution to the inear program (2.19) with α being repaced by α 1, and s (α 1) λ and s (α) λ wi triviay satisfy the subset entropy inequaity (2.105). We have thus constructed for any λ (R + ) and N = 0,..., 1, a sequence of A (α) and c (α) λ, α = 1,..., N, such that a conditions of Theorem 6 are met simutaneousy. This competes the proof of the theorem. 35

44 3. SYMMETRICA MUTIEVE DIVERSITY CODING WITH AN A-ACCESS ENCODER 3.1 Probem Statement As iustrated in Figure 3.1, the probem of SMDC-A consists of: a tota of independent discrete memoryess sources {S α [t]} t=1, where α = 1,..., and t is the time index; a set of +1 encoders (encoder 0 to ); a decoder who has access to a subset {0} U of the encoder outputs for some nonempty U Ω. The reaization of U is unknown a priori at the encoders. However, no matter which U actuay materiaizes, the decoder needs to neary perfecty reconstruct the sources (S 1,...,S α ) whenever U α. Formay, an (n,(m 0,M 1,...,M )) code is defined by a coection of + 1 encoding functions e : Sα n {1,...,M }, = 0,1,..., (3.1) and 2 1 decoding functions d U : {1,...,M 0 } U } Sα, U{1,...,M n U Ω s.t. U. (3.2) A nonnegative rate tupe (R 0,R 1,...,R ) is said to be admissibe if for every ǫ > 0, there exits, for sufficienty arge bock ength n, an (n,(m 0,M 1,...,M )) code such 36

45 R 0 Encoder 0 R 1 Encoder 1 X 0 X 1 Sources R 2 X 2 (S1,...,S n ) n Encoder 2 Decoder (Ŝn 1,...,Ŝn U )... X U R Encoder X Figure 3.1: SMDC with an a-access encoder 0 and randomy accessibe encoders 1 to. A tota of independent discrete memoryess sources (S 1,...,S ) are to be encoded at the encoders. The decoder, which has access to encoder 0 and a subset U of the randomy accessibe encoders, needs to neary perfecty reconstruct the sources (S 1,...,S U ) no matter what the reaization of U is. that: (Rate constraints at the encoders) 1 n ogm R +ǫ, = 0,1,...,; (3.3) (Asymptoticay perfect reconstructions at the decoder) Pr { d U (X {0} U ) (S n 1,...,Sn U )} ǫ, U Ω s.t. U (3.4) where Sα n := {S α[t]} n t=1, X := e (S1 n,...,sn ) is the output of encoder, and X {0} U := {X : {0} U}. Theadmissiberate region Risthecoectionofa admissiberatetupes(r 0,R 1,...,R ). 37

46 3.2 Superposition Coding Rate Region Simiar to cassica SMDC, a natura strategy for SMDC-A is superposition coding, i.e., to encode the sources separatey at the encoders and there is no coding across different sources. Formay, the probem of encoding a singe source S α can be viewedasaspeciacaseofthegeneraprobemwherethesourcess m aredeterministic for a m α. In this case, the source S α needs to be neary perfecty reconstructed whenever the decoder can access at east α randomy accessibe encoders in addition to the a-access encoder 0. The foowing scheme is a natura extension of the simpe source-channe separation scheme considered previousy for cassica SMDC: Firstcompressthesourcesequence S n α intoasourcemessage W α using aossess source code. It is we known [8, Ch. 5] that the rate of the source message W α can be made arbitrariy cose to the entropy rate H(S α ) for sufficienty arge bock ength n. Next, divide the source message W α into two independent sub-messages W (0) α and W (1) α so we have H(W α ) = H(W (0) α )+H(W (1) α ). (3.5) The sub-message W (0) α is stored at the a-access encoder 0 without any coding, which requires R 0 1 n H(W(0) α ). (3.6) The sub-message W (1) α is encoded by the randomy accessibe encoders 1 to using a maximum distance separabe code [9]. Ceary, the sub-message W (1) α 38

47 can be perfecty recovered at the decoder whenever U R 1 n H(W(1) α ), U Ω (α) (3.7) for sufficienty arge bock ength n. Eiminating H(W (0) α ) and H(W (1) α ) from (3.5) (3.7), we concude that the source message W α can be perfecty recovered at the decoder whenever R 0 + U R 1 n H(W α), U Ω (α) (3.8) Combining the above two steps, we concude that the rate region that can be achieved by the above source-channe separation scheme is given by the coection of a nonnegative rate tupes (R 0,R 1,...,R ) satisfying R 0 + U R H(S α ), U Ω (α). (3.9) Foowing the same footsteps as those for cassica SMDC [1, 2], it is straightforward to show that the above rate region is in fact the admissibe rate region for encoding the singe source S α. By definition, the superposition coding rate region R sup for SMDC-A is given by the coection of a nonnegative rate tupes (R 0,R 1,...,R ) such that R := r (α) (3.10) for some nonnegative r (α), α = 1,..., and = 0,1,...,, satisfying r (α) 0 + r (α) H(S α ), U Ω (α). (3.11) U 39

48 Simiar to cassica SMDC, the superposition coding rate region R sup for SMDC- A is a poyhedron with poyhedra cone being the nonnegative orthant in R +1 and hence can be competey characterized by the supporting hyperpanes where λ R f(λ 0,λ), λ 0 0 and λ := (λ 1,...,λ ) (R + ) (3.12) =0 f(λ 0,λ) = = min (R 0,R 1,...,R ) R sup λ R (3.13) =0 min =0 ( λ r (α) subject to r (α) 0 + U r(α) H(S α ), U Ω (α) r (α) ) 0, α = 1,..., and = 0,...,. and α = 1,..., (3.14) Ceary, the above optimization probem can be separated into the foowing suboptimization probems: where f(λ 0,λ) = f α (λ 0,λ) (3.15) f α(λ 0,λ) = = min =0 λ r (α) subject to r (α) 0 + U r(α) H(S α ), U Ω (α) max r (α) 0, = 0,..., ) c λ0,λ(u) H(S α ) ( U Ω (α) subject to c U Ω (α) λ0,λ(u) λ 0 {U Ω (α) :U }c λ 0,λ(U) λ, = 1,..., c λ0,λ(u) 0, U Ω (α). (3.16) (3.17) 40