Approximating the Gaussian multiple description rate region under symmetric distortion constraints

Approximating the Gaussian multiple description rate region under symmetric distortion constraints Chao Tian AT&T Labs-Research Florham Park, NJ 0793, USA. tian@research.att.com Soheil Mohajer Suhas Diggavi Ecole Polytechnique Fédérale de Lausanne (EPFL) Lausanne, Switzerl, CH 05 Email: {soheil.mohajer, suhas.diggavi}@epfl.ch Abstract We consider multiple description coding for the Gaussian source with descriptions under the symmetric mean squared error distortion constraints. Inner outer bounds for the achievable rate region are derived carefully tailored, such that they can be compared conveniently. The inner bound is based on a generalization of the multilayer scheme previously proposed by Puri et al., through a more flexible binning method. The resulting achievable region has the same geometric structure as the rate region of the lossless multilevel diversity coding problem, which reveals a strong connection between them. The outer bound is derived by combining the bounding technique for the sum rate in our earlier work, together with the α-resolution method introduced by Yeung Zhang. Comparison between the inner outer bounds shows that the gap in between is upper bounded by some constants. Particularly for the three description problem, the bounds can be written explicitly, both the inner outer bounds can be represented by ten planes with matching normal directions, between which the pairwise difference is small. I. INTRODUCTION In the multiple description (MD) problem, a source is encoded into several descriptions such that any one of them can be used to reconstruct the source with certain quality, more descriptions can improve the reconstruction. The problem is well motivated by source transmission over network with path failure or packet network with loss. In the early works [], [] on this problem, only two descriptions are considered. Even in this setting, the quadratic Gaussian problem is the only completely solved case [], for which the achievable region in [] is tight. Through a counterexample, Zhang Berger showed that this achievable region is however not tight in general [3], a complete characterization of the rate-distortion (R-D) region is yet to be found. In [4][5], an achievable rate was provided for symmetric description problem, where each description has the same rate, the distortion constraint depends only on the number of descriptions available. The coding scheme is based on joint binning of the codebooks, similar to the method often used in distributed source coding. Wang Viswanath [6], [7] generalized the MD problem to vector Gaussian source with descriptions, tight sum rate lower bound was established for certain cases with only two levels of distortion constraints. The current work is built upon two related works. In a previous paper [8], we developed a novel lower bound on the sum rate of description Gaussian problem under symmetric distortion constraints. This bound was derived by generalizing Ozarow s well-known bounding technique for the two-description MD problem [], introducing more than one additional auxiliary rom variables. Special structure among these rom variables facilities the derivation of a lower bound involving levels of distortion constraints. Another thread of results related to the current work is the symmetric multilevel diversity (MLD) source coding problem, considered by Roche et al. [9] by Yeung Zhang [0], which is essentially a lossless version of the MD problem. As we shall show, the lossless MLD coding problem provides not only important guidelines as how to represent the achievability rate region of the MD problem, but also important tools in establishing the MD rate region outer bound. Both inner outer bounds of the MD rate region are considered in this work. An inner bound is given by generalizing through more flexible binning the multilayer coding scheme in [5], which originally is only for symmetric rate. This inner bound is a polytope bearing the same geometry as the MLD rate region. We illustrate the result with = 3, since the polytope inner outer bounds can be written explicitly as the intersection of ten half spaces with the same normal directions. We show the pairwise differences between these planes are less than some small constants; these constants can be made universal, i.e., independent of the distortion constraints. For the general -description problem, we combine the α-resolution method in [0] with the lower bounding technique to derive lower bound on the bounding planes of the rate region, which together with the afore-given inner bound, provides an approximate rate region characterization. II. NOTATION AND REVIEWS In this section we first provide a formal problem definition, then review some necessary definitions results on α- resolutions in [0], as well as some results from [8]. A. Notation problem formulation Let {X(i)} i=,,... be a memoryless stationary Gaussian source with zero-mean unit-variance. The vector X(),X(),...,X(n) will be denoted as X n. The mean squared error (MSE) distortion d(x n,y n ) = n n i= (x(i) y(i)) will be used. We adopt the notation in [0]. Boldface letters are used to denote -vectors. For the general - description problem, a length-n block of the source samples

is encoded into descriptions. Let v be a vector in {0,}, denote the i-th component of v by v i. Define Ω α = {v {0,} : v = α}, α =,,..., () where v is the Hamming weight of v, define Ω = Ωα. Decoder v, v Ω has access to the v descriptions in the set Gv = {i : v i = }. The symmetric distortion constraints are given such that any decoder v can reconstruct the source to satisfy a certain distortion D v, i.e., the distortion constraint depends only on the number of descriptions the decoder has access to, but not the particular combination of descriptions. The notation I k is used to denote the set {,,...,k}. Formally, an (n,(m i,i I ),( v,v Ω )) code is defined by encoding functions S i decoding functions Tv S i : X n I Mi, i I, Tv : I Mi X n, v Ω, i Gv v = Ed(X n,tv(s i (X n ),i Gv)), v Ω, () where E is the expectation operator. A rate tuple (R,R,...,R ) is (D,D,...,D ) achievable if for every ǫ > 0, there exists for sufficiently large n an (n,(m i,i I ),( v,v Ω )) code such that n log M i R i + ǫ, i I, v D v + ǫ, v Ω. We are interested in the collection of all the achievable rate tuples, denoted as R(D). B. Review of α-resolution results We quote directly a few definitions results from [0] for α-resolution in the sequel, which essentially is a duality result. Let u v be two vectors in R. Define u v if only if u i v i, i I. Similar notation holds for u,v {0,}. For any A = (A,A,...,A ) 0, a mapping c α : Ω α R + satisfying the following properties c α (v) 0, for all v Ω α,, c α (v)v A (3) is called an α-resolution for A; it will be denoted as c α. Define a function f α : (R + ) R + for α I by f α (A) = max c α (v), (4) where the maximum is taken over all the α-resolution of A. If c α achieves f α (A), then it is called an optimal α-resolution for A. Without loss of generality, we may assume A A... A. Definition : Let c α be an α-resolution of A, then v Ω c α α (v)v is called the profile of {c α (v)}. Lemma ([0], Lemma ): Let c α be α-optimal for A, let (Ă,Ă,...,Ă) be its profile. If there exist i such that A i Ăi > 0, then c α (v) > 0 implies v i =. Lemma ([0], Lemma ): Let c α be α-optimal for A, let (Ă,Ă,...,Ă) be its profile, then there exists 0 l α α such that A i Ăi > 0 if only if i l α. Definition : For α, let c α c α be α- optimal (α )-optimal for A, respectively. Then c α covers c α, denoted by c α c α, if c α (u)h(s i,i Gu) c α (v)h(s i,i Gv), u Ω α for any jointly distributed rom variable S,S,...,S. Theorem ([0], Theorem 3): For any A 0, there exist c α, α, where c α is α-optimal for A, such that C. Review of MD sum rate results c c... c. (5) The following sum-rate lower bound was proved in [8][]. Theorem ([8], Theorem 3): For the Gaussian source, the sum rate under the -description symmetric distortion satisfies R i i= α [log + d α + d α + log D α + d α D α + d α ], (6) where d d... d > 0 are arbitrary non-negative values, d 0 d 0. The following lemma can be extracted from the proof of the above theorem, is included below for convenience. Lemma 3: Let S i, i I be a set of encoding functions such that there exist decoding functions to satisfy the distortion constraints D. Let Y = X + Z Y = X + Z + Z, where Z i, i =,, are mutually independent Gaussian rom variables independent of the Gaussian source X, with variance σ i. Then I(S i,i Gv;Y n ) n log + σ D v + σ, (7) I(S i,i Gv;Y n ) I(S i,i Gv;Y n ) n log ( + σ )(D v + σ + σ) ( + σ + σ )(D v + σ (8) ). In the multilayer scheme in [5], at each layer α I, there are rom variables Y α,j, for j I. A decoder with α descriptions can decode the first α layers of information, each rom variable Y α,j is essentially the information intended for the receiver having the j-th description. The - th layer uses a more conventional conditional codebook. This scheme was carefully analyzed in [8][], it was shown that the original symmetric requirement on the auxiliary rom variables can be relaxed. More specifically, the following rom variables are valid choice for the multilayer scheme: Y α,k = X + N i,k, α I, k I (9) i=α where N i,k are mutually independent zero-mean Gaussian rom variables, also independent of X, with σ i,k = σ i to be specified later. For the last layer we use Y = X E(X Y α,k,α I,k I ) + N, (0) where N is a zero-mean Gaussian rom variable independent of everything else, with variance σ. It was shown in [8][] that associated with distortion vector D, there exists an enhanced distortion vector D, such that D D D D... D. Moreover the choice of rom variables (9) (0) can always achieve D

with equality using linear least mean squared error (LLMSE) estimation, by proper choice of the variances σ i. III. GENERALIZATION OF THE MULTI-LAYER CODING SCHEME AND THE ACHIEVABLE REGION A. The multilayer coding scheme [5] At layer α I for any k I, codebooks of size nr α,k are generated using the marginal distribution of Yα,k, respectively. The rate R α,k should be sufficiently large such that for any source codeword, with high probability there exist codewords in the codebooks (α,k), α I k I that are jointly typical with it. This can be done if we choose where R α,k > h(y α, ) h α, h α h(y α,k,k I X, {Y j,k,j I α,k I }) () We choose R α,k = R α,k for any k k to simplify the resulting achievable region; i.e., we choose R α,k = h(y α, ) h α + δ, () for an arbitrarily small but positive δ. Each codebook is romly assigned into bin of size nr α,k, α I k I. At the decoder, with any k descriptions such that k I, the first k layers are decoded. More precisely, with descriptions in Gv, such that v = k, if there exists a unique set of codewords {y n α,j,α I k,j G v} in the specified bins that are jointly typical, then the decoder reconstructs using the single-letter decoding function gv( ); otherwise a decoding failure occurs. To succeed with high probability for any k I, the rates need to satisfy where 0 R α,j R α,j, α I, j I. (3) (R α,j R α,j ) < αh(y α, ) h α, (4) j Gv h α = (Y α,i,i I α Y k,j,k I α,j I α ). (5) for all v Ω such that v = α, for all α I. Rewriting (4) using (), we have R α,j > h α α h α + αδ. (6) j Gv The last layer codebook is generated using the more conventional method, i.e., the conditional codebook, the following condition is sufficient R,k > I(X;Y Y α,k,α I,k I ). (7) k= An achievable symmetric rate is then given by collecting the constraints on non-negative rates R α,j in (3), (6) (7), defining R = R k = R α,k, which is the main result in [5]. However, instead of a single rate, we are interested in the rate region. Moreover, the upper bound in (3) introduces additional difficulty when comparing to the outer bound derived in the next section. Next we show a region without the upper bound in (3) is indeed achievable. B. The rate region of the generalized multilayer scheme For a fixed set of symmetric rom variables {{Y α,k,α I,k I },Y }, define the following quantities H α = h α α h α, α I H = I(X;Y Y α,k,α I,k I ). (8) Let R(Y ) be the set of non-negative rate vectors (R,R,...,R ), such that R i ri α, i, (9) for some ri α 0, α, satisfying r v i H v, v Ω. (0) i Gv For a given distortion vector D its corresponding enhanced distortion vector D, let RG (D ) be R(Y ) under the specific choice of Gaussian auxiliary rom variables given in (9) (0), such that the distortion constraints D are satisfied with equality under LMMSE. Theorem 3: Let D be the enhanced distortion vector of D, then R G (D ) R(D ). () Before presenting the proof of this theorem, we quote a result from [0]. Let denote the usual inner product in the Euclidean space. Let R (Y ) be the set of all R 0 such that for all A (R + ) but A 0 A R f α (A) H α. () Theorem 4 ([0] Theorem ): R(Y ) = R (Y ). (3) Proof of Theorem 3: Since both R(D ) R(Y ) are convex, they can be characterized by the bounding planes. As such if we can prove that for any A (R + ) A 0, the following inequality holds min A R min A R, (4) R R(D ) R R(Y ) then it follows that the region R(Y ) is an achievable region. By Theorem 4, we have min R R(Y ) A R = f α (A) H α. (5) Thus it suffices to prove there exists R R(D ) such that A R = f α (A) H α, (6) for any A (R + ) A 0. This would imply (4), which subsequently implies the claimed result. Thus we only need to prove the existence of R R(D ) for (6) to hold. For a given A, let l α be the non-negative integer defined in Lemma for the α-level. For any α I, let { 0 if k lα ; R α,k = H α α l α if l α + k. (7)

R α,k will be rate assigned to the α-th layer for the k-th description; denote (R α,,r α,,...,r α, ) as R α. It is clear if each of the bin has size approximately n H α/α, then any of the α descriptions can guarantees decoding with high probability, as proved in the original scheme in [5]. However, because the first l α descriptions are not given any rate for the α-th layer, this can not be achieved directly without proper generalization. Now we construct codes with these rates generalizing the multi-layer coding scheme. For any level α I, the last l α codebooks binning are constructed as in the original scheme; denote the bin index as b α,k (yα,k n ), or simply as b α,k. For each of the first l α description, the codebooks are generated the same as before, but we introduce l α orthogonal binning structures: for each binning structure, the codewords are assigned uniformly at rom into one of n H α/[α(α l α)] bins independently. For a codeword yα,k n, its bin index in the j-th binning structure is denoted as b α,k,j. During encoding, the encoder finds the jointly typical codewords in the same way as in the original scheme. For each of the last l α descriptions, for example the k -th description, we concatenate the bin index b α,k with the l α other bin indices b α,i,k, i =,,...,l α, to form the α-th layer of the k -th description. The rate of this description for the α-th layer is as we claimed. Moreover, given any Gv descriptions where v = α, we always have sufficient information to decode with high probability l α codewords in the first l α codewords, as well as those other codewords in the codebooks belonging to the set Gv, by the same argument as in the original multilayer scheme. Thus this is indeed a code of rates as in (7). It remains to show (6) is true with the given rate vector. Let {c(v)} be an optimal α-resolution for A. We have A R α = c α (v)(v R α ) + (A c α (v)v) R α. By Lemma, for any v such that c α (v) > 0, v i = for i =,,...,l α ; moreover, exactly α l α of the remaining components are s. Since the first l α components of R α are 0 s, the remaining components are equal, we have H α v R α = (α l α ) = α l H α for v : c α (v) > 0. α It follows that c α (v)(v R α ) = H α c α (v) = f α (A) H α. (8) Since A c α (v)v = A Ă (9) has zeros in the last l α components, R α has zeros in the complement positions, we have (A c α (v)v) R α = 0. (30) It follows A R α = f α (A) H α. (3) Summing over α I completes the proof. R R 3 R + + R R3 R + R + R 3 R + R 3 Fig.. R(D ) is drawn in bold lines, R G (D ) in dashed line. It is clear that RG (D ) R(D ) it is the gaps between the corresponding planes that are shown in terms of the Euclidean distance. IV. APPROXIMATING THE RATE REGION FOR = 3 Using the Fourier-Motzkin s algorithm [3], it is straightforward to show that for = 3 the region R G (D ) is the set of rates satisfying the following constraints, with D being the enhanced distortion vector of D. R i /log Di, i =,,3, R i + R j D α log( D ), i,j I 3, R i + R j + R k 3 3 + log( D) log 3 D, i,j,k I 3, R + R + R 3 = 3 3 3 4 log( D ) log 3 D, Define R(D ), which is an outer bound of the rate region R(D), to be the set of rates satisfying the following R i /log Di, i =,,3, R i + R j D log( D), i,j I 3, α R + R + R 3 3 3 + log( D ) log(3 D ) + log, i,j,k I 3, R + R + R 3 3 3 α log D α 3 log( D ) 3 4 log(3 D ) + log. We omitted the proof for brevity; see []. The bounds are R

illustrated in Fig.. Note that the bound on the gap can be made universal by replacing α Dk with α. V. THE GENERAL -DESCRIPTION GAUSSIAN PROBLEM The result given in the last section is difficult to generalize directly. In this section, we follow the approach taken by Yeung Zhang [0] for the MLD problem, consider the bounding planes of the rate region R(D). Define the following function R A (D ) f α (A)log D α D α f α (A)log(α ) + α= f α (A)log(α ). α= Theorem 5: For the Gaussian source any A 0, f α(a) H G α (D ) min R R(D) A R R A (D ). Moreover, define A i= A i A min = min{a i }, f α (A) H α G (D ) R A (D ) A [f α (A) f α (A)]log(α ) α= [ α α ]log(α D α ) α= + ( A A min)log( D ). (3) Proof: The upper bound follows from Theorem 3 4. For the lower bound, let c,c,...,c be a set of optimal resolutions as in Theorem. Clearly the -resolution satisfies that v = with v i =, c (v) = A i ; also for v such that v =, c (v) = f (A) = A min. Consider the inequality n A i (R i + ǫ) A i H(S i ) (a) = i= i= c α (v)h(s i,i Gv) Gv : v =α c α+ (v)h(s i,i Gv) Gv : v =α+ + A min H(S i,i I ) A min H(S i,i I X n ) I (S), where (a) is by adding subtracting the same terms, by the fact that S i,i I are deterministic functions of X n. A i H(S i ) (b) I (S) i= c α (v)h(s i,i Gv Yα n ) Gv : v =α c α+ (v)h(s i,i Gv Yα n ), (33) Gv : v =α+ where Y α = X + i=α N i, for α I, with N i being independent Gaussian rom variables with variances ˆσ i ; denote d α = α σα; (b) is by the covering property of the given sequence of the optimal α-resolutions given in Definition. After some algebra, we arrive at A i H(S i ) A min (I(S i,i I ;X n ) I(S i,i I ;Y )) n i= i= + α= v: v =α c α (v) ( I(S i,i Gv;Y n α ) I(S i,i Gv;Y n α ) ) + c (v)i(s i,i Gv;Y n ), (34) Gv : v = at this point, application of Lemma 3 leads to [ f α (A) A i H(S i ) log + d α + log D ] α + d α. + d α D α + d α Using a specific set of values of d α for α =,3,..., algebraic simplification lead to the formula in the theorem. The details are omitted here; see []. VI. CONCLUSION Given the difficulty of the general Gaussian MD problem, it is satisfying to see there exists a simple scheme provably within constants of optimality. In fact we show in [] that similar bounds can be developed for an even simpler scheme based on successive refinement coding MLD coding. The MD result given in this work relies on the MLD coding result, which demonstrates their intimate connection. REFERENCES [] L. Ozarow, On a source-coding problem with two channels three receivers, Bell Syst. Tech. Journal, vol. 59, pp. 909 9, Dec. 980. [] A. El Gamal T. M. Cover, Achievable rates for multiple descriptions, IEEE Trans. Inform. Th., vol. 8, no. 6, pp. 85 857, Nov. 98. [3] Z. Zhang T. Berger, New results in binary multiple descriptions, IEEE Trans. Information Theory, vol. 33, no. 4, pp. 50 5, Nov. 98. [4] S. S. Pradhan, R. Puri,. Ramchran, n-channel symmetric multiple descriptions - Part I: (n, k) source-channel erasure codes, IEEE Trans. Information Theory, vol. 50, pp. 47 6, Jan. 004. [5] R. Puri, S.S. Pradhan,. Ramchran, n-channel symmetric multiple descriptions - Part II: an achievable rate-distortion region, IEEE Trans. Information Theory, vol. 5, pp. 377 39, Apr. 005. [6] H. Wang P. Viswanath, Vector Gaussian multiple description with individual central receivers, IEEE Trans. Information Theory, vol. 53, no. 6, pp. 33 53, Jun. 007. [7] H. Wang P. Viswanath, Vector Gaussian multiple description with two levels of receivers, IEEE Trans. Information Theory, submitted for publication. [8] C. Tian, S. Mohajer S. Diggavi, On the symmetric Gaussian multiple description rate-distortion function, In Proceedings, 007 IEEE Data Compression Conference, Snowbird, Utah, Mar. 007. [9] J. R. Roche, R. W. Yeung,. P. Hau, Symmetrical multilevel diversity coding, IEEE Trans. Information Theory, vol. 43, no. 5, pp. 059 064, May 997. [0] R. W. Yeung Z. Zhang, On symmetrical multilevel diversity coding, IEEE Trans. Inform. Th., vol. 45, pp. 609 6, Mar. 999. [] C. Tian, S. Mohajer S. Diggavi, Approximating the multiple description rate region under symmetric distortion constraints, EPFL Technical report, Jan. 008. Available at http://infoscience.epfl.ch [] T. M. Cover J. A. Thomas, Elements of information theory, New York: Wiley, 99. [3] G. M. Ziegler, Lectures on polytopes, volume 5 of Graduate Texts in Mathematics, Springer-Verlag, 995.