A Coordinate System for Gaussian Networks

Transcription

1 A Coordinate System for Gaussian Networs The MIT Faculty has made this article oenly available. Please share how this access benefits you. Your story matters. Citation As Published Publisher Abbe, Emmanuel, and Lizhong Zheng. A Coordinate System for Gaussian Networs. IEEE Transactions on Information Theory ): htt://dx.doi.org/ /tit Institute of Electrical and Electronics Engineers IEEE) Version Author's final manuscrit Accessed Sat Oct 0 17:05:18 EDT 018 Citable Lin htt://hdl.handle.net/171.1/73671 Terms of Use Creative Commons Attribution-Noncommercial-Share Alie 3.0 Detailed Terms htt://creativecommons.org/licenses/by-nc-sa/3.0/

2 A Coordinate System for Gaussian Networs Emmanuel A. Abbe IPG, EPFL Lausanne, 1015 CH Lizhong Zheng RLE, MIT Cambridge, MA arxiv:100.71v1 [cs.it] 11 Feb 010 Abstract This aer studies networ information theory roblems where the external noise is Gaussian distributed. In articular, the Gaussian broadcast channel with coherent fading and the Gaussian interference channel are investigated. It is shown that in these roblems, non-gaussian code ensembles can achieve higher rates than the Gaussian ones. It is also shown that the strong Shamai-Laroia conjecture on the Gaussian ISI channel does not hold. In order to analyze non-gaussian code ensembles over Gaussian networs, a geometrical tool using the Hermite olynomials is roosed. This tool rovides a coordinate system to analyze a class of non-gaussian inut distributions that are invariant over Gaussian networs. I. INTRODUCTION Let a memoryless additive white Gaussian noise AWGN) channel be described by Y = X + Z, where Z N 0, v) is indeendent of X. If the inut is imosed an average ower constraint given by EX, the inut distribution maximizing the mutual information is Gaussian. This is due to the fact that under second moment constraint, the Gaussian distribution maximizes the entroy, hence arg max X:EX = hx + Z) N 0, ). 1) On the other hand, if we use a Gaussian inut distribution, i.e., X N 0, ), the worst noise that can occur, i.e., the noise minimizing the mutual information, among noises with bounded second moment, is again Gaussian distributed. This can be shown by using the entroy ower inequality EPI), cf. [11], which reduces in this setting to and imlies arg min hx + Z) N 0, v) ) Z: hz)= 1 log πev arg min Z:EZ =v hx + Z) hz) N 0, v). 3) Hence, in the single-user setting, when otimizing the mutual information as above, a Gaussian inut is the best inut for a Gaussian noise and a Gaussian noise is the worst noise for a Gaussian inut. This rovides a game equilibrium between user and nature, as defined in [7],. 63. With these results, many roblems in information theory dealing with Gaussian noise can be solved. However, in Gaussian networs, that is, in multi-user information theory roblems where the external noise is Gaussian distributed, several new henomena mae the search for the otimal inut ensemble more comlex. Besides for some secific cases of Gaussian networs, we still do not now how interference should be treated in general. Let us consider two users interfering on each other in addition to suffering from Gaussian external noise and say that the receivers treat interference as noise. Then, if the first user has drawn its code from a Gaussian ensemble, the second user faces a frustration henomenon: using a Gaussian ensemble maximizes its mutual information but minimizes the mutual information of the first user. It is an oen roblem to find the otimal inut distributions for this roblem. This is one illustration of the comlications aearing in the networ setting. Another examle is regarding the treatment of the fading. Over a single-user AWGN channel, whether the fading is deterministic or random, but nown at the receiver, does not affect the otimal inut distribution. From 1), it is clear that maximizing IX; X + Z) or IX; HX + Z H) under an average ower constraint is achieved by a Gaussian inut. However, the situation is different if we consider a Gaussian broadcast channel BC). When there is a deterministic fading, using 1) and 3), the otimal inut distribution can be shown to be Gaussian. However, it has been an oen roblem to show whether Gaussian inuts are otimal or not for a Gaussian BC with a random fading nown at the receiver, even if the fading is such that it is a degraded BC. A reason for these oen questions in the networ information theoretic framewor, is that Gaussian ensembles are roughly the only ensembles that can be analyzed over Gaussian networs, as non-gaussian ensembles have left most roblems in an intractable form. In this aer, a novel technique is develoed to analyze a class of non-gaussian inut distributions over Gaussian noise channels. This technique is efficient to analyze the cometitive situations occurring in the networ roblems described below. It allows in articular to find certain non-gaussian ensembles that outerform Gaussian ones on a Gaussian BC with coherent fading channel, a two user interference channel, and it allows to disrove the strong Shamai-Laroia conjecture on the Gaussian intersymbol interference channel. This tool rovides a new insight on Gaussian networs and confirms that non-gaussian ensembles do have a role to lay in these networs. We now introduce with more details the notion of cometitive situations. A. Cometitive Situations 1) Fading Broadcast Channel: Consider a degraded Gaussian BC with coherent memoryless fading, where the fading

3 is indeed the same for both receivers, i.e. Y 1 = HX + Z 1, Y = HX + Z but Z 1 N 0, v 1 ) and Z N 0, v ), with v 1 < v. The inut X is imosed a ower constraint denoted by. Because the fading is coherent, each receiver also nows the realization of H, at each channel use. The fading and the noises are memoryless iid) rocesses. Since this is a degraded broadcast channel, the caacity region is given by all rate airs IX; Y 1 U, H), IU; Y H)) with U X Y 1, Y ). The otimal inut distributions, i.e., the distributions of U, X) achieving the caacity region boundary, are given by the following otimization, where µ R, arg max IX; Y 1 U, H) + µiu; Y H). 4) U,X): U X Y 1,Y ) EX Note that the objective function in the above maximization is given by hy 1 U, H) hz 1 ) + µhy H) µhy U, H). Now, each term in this exression is individually maximized by a Gaussian distribution for U and X, but these terms are combined with different signs, so there is a cometitive situation and the maximizer is not obvious. When µ 1, one can show that Gaussian distributions are otimal. Also, if H is comactly suorted, and if v is small enough as to mae the suort of H and 1/vH non overlaing, the otimal distribution of U, X) is jointly Gaussian cf. [13]). However, in general the otimal distribution is unnown. We do not now if it because we need more theorems, or if it is really that with fading, non-gaussian codes can actually erform better than the Gaussian ones. ) Interference Channel: We consider the symmetric memoryless interference channel IC) with two users and white Gaussian noise. The average ower is denoted by, the interference coefficients by a, and the resective noise by Z 1 and Z indeendent standard Gaussian). We define the following exression ms a, X m 1, X m ) 5) = IX m 1 ; X m 1 + ax m + Z m 1 ) + IX m ; X m + ax m 1 + Z m ) = hx m 1 + ax m + Z m 1 ) hax m + Z m 1 ) + hx m + ax m 1 + Z m ) hax m 1 + Z m ), where X1 m and X m are indeendent random vectors of dimension m with a covariance having a trace bounded by m and Zi m, i = 1,, are iid standard Gaussian. For any dimension m and any distributions of X1 m and X m, S a, X1 m, X m ) is a lower bound to the sum-caacity. Moreover, it is tight by taing m arbitrarily large and X1 m and X m maximizing 5). Now, a similar cometitive situation as for the fading broadcast roblem taes lace: Gaussian distributions maximize each entroy term, but these terms are combined with different signs. Would we then refer to tae X 1 and X Gaussian or not? This should deend on the value of a. If a = 0, we have two arallel AWGN channels with no interference, and Gaussian inuts are otimal. We can then exect that this might still hold for small values of a. It has been roved recently in [], [8], [10], that the sum-caacity is achieved by treating interference as noise and with iid Gaussian inuts, as long as a 3 + a 1/ 0. Hence, in this regime, the iid Gaussian distribution maximizes 5) for any m. But if a is above that threshold and below 1, the roblem is oen. Let us now review the notion of treating interference as noise. For each user, we say that the decoder is treating interference as noise, if it does not require the nowledge of the other user s code boo. However, we allow such decoders to have the nowledge of the distribution, under which the other user s code boo may be drawn. This is for examle necessary to construct a sum-caacity achieving code 1 in [], [8], [10], where the decoder of each user treats interference as noise but uses the fact that the other user s code boo is drawn from an iid Gaussian distribution. But, if we allow this distribution to be of arbitrarily large dimension m in our definition of treating interference as noise, we can get a misleading definition. Indeed, no matter what a is, if we tae m large enough and a distribution of X m 1, X m maximizing 5), we can achieve rates arbitrarily close to the sum-caacity, yet, formally treating interference as noise. The roblem is that the maximizing distributions in 5) may not be iid for an arbitrary a, and nowing it at the receiver can be as much information as nowing the other user s code boo for examle, if the distribution is the uniform distribution over a code boo of small error robability). Hence, one has to be careful when taing m large. In this aer, we will only wor with situations that are not ambiguous with resect to our definition of treating interference as noise. It is indeed an interesting roblem to discuss what ind of m-dimensional distributions would cature the meaning of treating interference as noise that we want. This also oints out that studying the maximizers of 5) relates to studying the concet of treating interference as noise or information. Since for any chosen distributions of the inuts we can achieve 5), the maximizers of 5) must have a different structure when a grows. For a small enough, iid Gaussian are maximizing distributions, but for a 1, since we do not want to treat interference as noise, the maximizing distributions must have a heavy structure, whose characterization requires as much information as giving the entire code boo. This underlines that an encoder can be drawn from a distribution which does not maximize 5) for any value of m, but yet, a decoder may exist in order to have a caacity achieving code. This haens if a 1, iid Gaussian inuts will achieve the sum-caacity if the receiver decodes the message of both users one can show that the roblem is equivalent to having two MAC s). However, if a 1, the iid Gaussian distribution does not maximize S a, X 1, X ) for the dimension 1, hence for arbitrary dimensions). 1 in a low interference regime

4 In any cases, if the Gaussian distribution does not maximize 5) for the dimension 1, it means that iid Gaussian inuts and treating interference as noise is not caacity achieving, since a code which treats interference as noise and whose encoder is drawn from a distribution can be caacity achieving only if the encoder is drawn from a distribution maximizing 5). Hence, understanding better how to resolve the cometitive situation of otimizing 5) is a consequent roblem for the interference channel. B. ISI channel and strong Shamai-Laroia Conjecture Conjecture 1: Let h,, v R +, X1 G N 0, ) and Z N 0, v) indeendent of X1 G ). For all X, X 1 i.i.d. with mean 0 and variance, we have IX; X + hx G 1 + Z) IX; X + hx 1 + Z). 6) This conjecture has been brought to our attention by Shlomo Shamai Shitz), who referred to the strong conjecture for a slightly more general statement, where an arbitrary memory for the interference term is allowed, i.e., where n i=1 h ix i stands for hx 1. The strong conjecture then claims that icing all X i s Gaussian gives a minimizer. However, we will show that even for the memory one case, the conjecture does not hold. The wea conjecture, also referred to as the Shamai-Laroia conjecture, corresonds to a secific choice of the h i s, which arises when using an MMSE decision feedbac equalizer on a Gaussian noise ISI channel, cf. [9]. This conjecture is investigated in a wor in rogress. There are many other examles in networ information theory where such cometitive situations occur. Our goal in this aer is to exlore the degree of freedom rovided by non- Gaussian inut distributions. We show that the neighborhood of Gaussian distributions can be arametrized in a secific way, as to simlify greatly the comutations arising in cometitive situations. We will be able to recisely quantify how much a certain amount of non-gaussianness, which we will characterize by means of the Hermite olynomials, affects or hels us in maximizing the cometitive entroic functional of reviously mentioned roblems. A. Fading BC II. PROBLEM STATEMENT For the fading BC roblem roblem described in I-A1, we want to determine if/when the distribution of U, X) maximizing 4) is Gaussian or not. B. IC For the interference channel roblem described in I-A, we now from [], [8], [10] that treating interference as noise and using iid Gaussian inuts is otimal when a 3 + a 1/ > 0. We question when this coding scheme is no longer otimal. More generally, we want to analyze the maximizers of 5). We distinguish the imlication of such a threshold in both the synchronized and asynchronized users setting, as there will be an interesting distinction between these two cases. We recall how the synch and asynch settings are defined here. In the synch setting, each user of the IC sends their code words of a common bloc length n simultaneously, i.e., at time 1, they both send the first comonent of their code word, at time the second comonent, etc. In the asynch setting, each user is still using code words of the same bloc length n, however, there might be a shift between the time at which the first and second users start sending their code words. We denote this shift by τ, and assume w.l.o.g. that 0 τ n. In the totally asynch setting, we assume that τ is drawn uniformly at random within {0,..., n}. We may also distinguish the cases where τ is not nown at the transmitter but at the receiver, and when τ is not nown at both. Note that if iid inut distributions are used to draw the code boos, and interference is treated as noise, whether the users are synch or asynch is not affecting the rate achievability. However, if the users want to time-share over the channel uses, such as to fully avoid their interference, they will need synchronization. Definition 1: Time sharing over a bloc length n assumed to be even) with Gaussian inuts refers to using X 1 Gaussian with covariance P În/ and X Gaussian with covariance P Îc n/, where În/ is a diagonal matrix with n/ 1 s and 0 s, and Îc n/ flis the 1 s and 0 s on the diagonal. C. ISI Channel and Strong Shamai-Laroia Conjecture We want to determine whether conjecture 1 holds or not. D. General Problem Our more general goal is to understand better the roblem osed by any cometitive situations. For this urose, we formulate the following mathematical roblem. We start by changing the notation and rewrite 1) and 3) as arg max hf g v) = g 7) f: m f)= arg min hf g v) hf) = g 8) f: m f)= where g denotes the Gaussian density with zero mean and variance, and the functions f are density functions on R, i.e., ositive functions integrating to 1, and having a welldefined entroy and second moment m f) = R x fx)dx. We consider the local geometry by looing at densities of the form f ε x) = g x)1 + εlx)), x R, 9) where L : R R satisfies inf Lx) > 10) x R Lx)g x)dx = 0. 11) R hence, 5) with an iid distribution for X 1 and X can still be defined for the totally asynch IC

5 With these two constraints on L, f ε is a valid density for ε sufficiently small. It is a erturbed Gaussian density, in a direction L. Observe that, m 1 f ε ) = 0 iff M 1 L) = xlx)g x)dx = 0 1) R m f ε ) = iff M L) = x Lx)g x)dx = 0. 13) We are now interested in analyzing how these erturbations affect the outut distributions through an AWGN channel. Note that, if the inut distribution is a Gaussian g erturbed in the direction L, the outut is a Gaussian g +v erturbed in the direction gl) gv g +v, since f ε g v = g +v 1 + ε g L) g v g +v ). Convention: g L g v refers to g L) g v, i.e., the multilicative oerator recedes the convolution one. For simlicity, let us assume in the following that the function L is a olynomial satisfying 10), 11). Lemma 1: We have where Df ε g ) = 1 ε L g + oε ) Df ε g v g g v ) = 1 ε g L g v g +v g +v + oε ). L g = R R L x)g x)dx. Moreover, note that for any density f, if m 1 f) = a and m f) = + a, we have hf) = hg a, ) Df g a, ). 14) Hence, the extremal entroic results of 7) and 8) are locally exressed as arg min g L g v g L: M L)=0 g +v = 0 15) +v arg max g L g v g L: M L)=0 g +v L g = 0, 16) +v where 0 denotes here the zero function. If 15) is obvious, 16) requires a roof which will be done in section V. Let us define the following maing, Γ +) : L L g ) g L g v g +v L g +v ; R), 17) where L g ) denotes the sace of real functions having a finite g norm. This linear maing gives, for a given erturbed direction L of a Gaussian inut g, the resulting erturbed direction of the outut through additive Gaussian noise g v. The norm of each direction in their resective saces, i.e., in L g ) and L g +v ), gives how far from the Gaussian distribution these erturbations are u to a scaling factor). Note that if L satisfies 10)-11), so does Γ +) L for the measure g +v. The result in 16) worst noise case) tells us that this maing is a contraction, but for our goal, what would be helful is a sectral analysis of this oerator, to allow more quantitative results than the extreme-case results of 15) and 16). In order to do so, one can exress Γ +) as an oerator defined and valued in the same sace, namely L with the Lebesgue measure λ, which is done by inserting the Gaussian measure in the oerator argument. We then roceed to a singular function/value analysis. Formally, let K = L g, which gives K λ = L g, and let g K g v Λ : K L λ) L λ) 18) g+v which gives Γ +) L g+v = ΛK λ. Denoting by Λ the adjoint oerator of Λ, we want to find the singular functions of Λ, i.e., the eigenfunctions K of Λ Λ: Λ ΛK = γk. III. RESULTS A. General Result: Local Geometry and Hermite Coordinates The following theorem gives the singular functions and values of the oerator Λ defined in revious section. Theorem 1: Λ ΛK = γk, K 0 holds for each air K, γ) { ) g H [], )} 0, + v where H [] x) = 1! H x/ ) H x) = 1) e x / d dx e x /, 0, x R. The olynomials H [] are the normalized Hermite olynomials for a Gaussian distribution having variance ) and g H [] are called the Hermite functions. For any > 0, {H [] } 0 is an orthonormal basis of L g ), this can be found in [1]. One can chec that H 1, resectively H erturb a Gaussian distribution into another Gaussian distribution, with a different first moment, resectively second moment. For 3, the H erturbations are not modifying the first two moments and are moving away from Gaussian distributions. Since H [] 0 = 1, the orthogonality roerty imlies that H [] satisfies 11) for any > 0. However, it is formally only for even values of that 13) is verified although we will see in section V that essentially any can be considered in our roblems). The following result contains the roerty of Hermite olynomials mostly used in our roblems, and exresses Theorem 1 with the Gaussian measures. The following result contains the roerty of Hermite olynomials mostly used in our roblems, and exresses Proosition 1 with the Gaussian measures.

6 Theorem : Γ +) H [] Γ ) H [+v] = g H [] g v = g +v = H [+v] g v = Last Theorem imlies Theorem 1, since for ) / H [+v], 19) + v ) / H [] + v. 0) Γ ) Γ +) L = γl Λ ΛK = γk K = L g. Comment: the results that we have just derived are related to roerties of the Ornstein-Uhlenhec rocess. Summary: In words, we just saw that H is an eigenfunction of the inut/outut erturbation oerator Γ +), in the sense ) / that Γ +) H [] = [+v] +v H. Hence, over an additive Gaussian noise channel g v, if we erturb the inut g in the direction H [] by an amount ε, we will erturb the outut +v ) / ε. Such g +v in the direction H [+v] by an amount a erturbation in H imlies that the outut entroy is reduced comared to not erturbing) by B. Fading BC Result +v ) ε if 3). The following result states that the caacity region of a degraded fading BC with Gaussian noise is not achieved by a Gaussian suerosition code in general. Theorem 3: Let Y 1 = HX + Z 1, Y = HX + Z with X such that EX, Z 1 N 0, v), 0 < v < 1, Z N 0, 1) and H, X, Z 1, Z mutually indeendent. There exists a fading distribution and a value of v for which the caacity achieving inut distribution is non-gaussian. More recisely, let U be any auxiliary random variable, with U X Y 1, Y ). Then, there exists, v, a distribution of H and µ such that U, X) IX; Y 1 U, H) + µiu; Y H) 1) is maximized by a non jointly Gaussian distribution. In the roof, we resent a counter-examle to Gaussian being otimal for H binary. In order to defeat Gaussian distributions, we construct inut distributions using the Hermite coordinates. The roof also gives a condition on the fading distribution and the noise variance v for which a non-gaussian distribution strictly imroves on the Gaussian one. C. IC Result Definition : Let [ F a, ) = lim lim Sa, δ 0 ε 0 ε X 1, X ) S a, X1 G, X G ) ], where X1 G, X G iid g, X 1 g 1 + ε H ) and X g 1 ε H ), with H defined in ) below as exlain in section IV, H is a formal modification of H to ensure the ositivity of the erturbed densities). In other words, F a, ) reresents the gain ositive or negative) of using X 1 erturbed along H and X erturbed along H with resect to using Gaussian distributions. Note that the distributions we chose for X 1 and X are not the most general ones, as we could have chosen arbitrary directions sanned by the Hermite basis to erturb the Gaussian densities. However, as exlained in the roof of the theorem 4, this choice is sufficient for our urose. Theorem 4: We have for [ a F a, ) = a + 1 ] a 1) a + + 1). For any fixed, the function F, ) has a unique ositive root, below which it is negative and above which it is ositive. Theorem 5: Treating interference as noise with iid Gaussian inuts does not achieve the sum-caacity of the symmetric IC synch or asynch) and is outerformed by X 1 g 1 + ε H 3 ) and X g 1 ε H 3 ), if F 3 a, ) > 0. This Theorem is a direct consequence of Theorem 4. Proosition 1: For the symmetric synch IC, time sharing imroves on treating interference as noise with iid Gaussian distribution if F a, ) > 0. We now introduce the following definition. Definition 3: Blind time sharing over a bloc length n assumed to be even) between two users, refers to sending non-zero ower symbols only at the instances mared with a 1 in 1, 0, 1, 0, 1, 0,... 1, 0) for the first user, and zero ower symbols only at the instances mared with a 1 in 1, 1,..., 1, 0, 0,..., 0) for the second user. Proosition : For the symmetric totally asynch IC, if the receivers but not transmitters) now the asynchronization delay, blind time sharing imroves on treating interference as noise with iid Gaussian distributions if B a, ) > 0, where B a, ) = 1 4 log1+)+log1+ 1+a )) log1+ 1+a ). If the receivers do not now the asynchronization delay, blind time sharing cannot imrove on treating interference as noise with iid Gaussian distributions if B a, ) 0. How to read these results: We have four thresholds to ee trac of: T 1 ) is when a 3 + a 1 = 0. If a T 1), we now from [], [8], [10] that iid Gaussian inuts and treating interference as noise is sum-caacity achieving. T ) is when F a, ) = 0. If a > T ), we now from Pro. 1 that, if synchronization is ermitted, time sharing imroves on treating interference as noise with iid

7 Gaussian inuts. This regime matches with the so-called moderate regime defined in [5]. T 3 ) is when F 3 a, ) = 0. If a > T 3 ), we now from Pro. 5 that treating interference as noise with iid non- Gaussian distributions oosites in H 3 ) imroves on the iid Gaussian ones. T 4 ) is when B a, ) = 0. If a > T 4 ), we now from Pro. that, even if the users are totally asynchronized, but if the receivers now the asynchronization delay, blind time sharing imroves on treating interference as noise with iid Gaussian inuts. If the receivers do not now the delay, the threshold can only aear for larger values of a. The question is now, how are these thresholds raned. It turns out that 0 < T 1 ) < T ) < T 3 ) < T 4 ). And if = 1, the above inequality reads as 0.44 < < < This imlies the following for a decoder that treats interference as noise. Since T ) < T 3 ), it is first better to time share than using non-gaussian distributions along H 3. But this is useful only if time-sharing is ermitted, i.e., for the synch IC. However, for the asynchronized IC, since T 3 ) < T 4 ), we are better off using the non-gaussian distributions along H 3 before a Gaussian inut scheme, even with blind time-sharing, and even if the receiver could now the delay. We notice that there is still a ga between T 1 ) and T ), and we cannot say if, in this range, iid Gaussian inuts are still otimal, or if another class of non-gaussian inuts far away from Gaussians) can outerform them. In [4], another technique which is related to ours but not equivalent) is used to find regimes where non-gaussian inuts can imrove on Gaussian ones on the same roblem that we consider here. The threshold found in [4] is equal to 0.95 for = 1, which is looser than the value of found here. Finally, the following interesting and curious fact has also been noticed. In theorem 4, we require. Nevertheless, if we lug = 1 in the right hand side of theorem 4 and as for this exressions to be ositive, we recisely get a 3 + a 1 > 0, i.e., the comlement range delimited by T 1 ). However, the right hand side of theorem 4 for = 1 is not equal to F 1 a, ) this is exlained in more details in the roof of theorem 4). Indeed, it would not mae sense that moving along H 1, which changes the mean with a fixed second moment within Gaussians, would allow us to imrove on the iid Gaussian scheme. Yet, getting to the exact same condition, when woring on the roblem of imroving on the iid Gaussian scheme, seems to be a strange coincidence. D. Strong Shamai-Laroia Conjecture We show in Section V that conjecture 1 does not hold. We rovide counter-examles to the conjecture, ointing out that the range of h for which the conjecture does not hold increases with SNR. IV. HERMITE CODING: FORMALITIES The Hermite olynomial corresonding to = 0 is H [] 0 = 1 and is clearly not a valid direction as it violates 11). Using the orthogonality roerty of the Hermite basis and since H [] 0 = 1, we conclude that H [] satisfies 11) for any > 0. However, it is only for even that H [] satisfies 10). On the other hand, for any δ > 0, we have that H [] + δh[] 4 satisfies 10), whether is even or not we chose 4 instead of for reasons that will become clear later). Now, if we consider the direction H [], 10) is not satisfied for both even and odd. But again, for any δ > 0, we have that H [] +δh[] 4 satisfies 10). Hence, in order to ensure 10), we will often wor in the roofs with ±H [] +δh[] 4, although it will essentially allow us to reach the erformance achieved by any ±H [] odd or even), since we will then tae δ arbitrarily small and use continuity arguments. Convention: We dro the variance uer scrit in the Hermite terms whenever a Gaussian density with secified variance is erturbed, i.e., the density g 1 + εh ) always denotes g 1 + εh [] ), and g H always denotes g H [], no matter what is. Same treatment is done for g and. Now, in order to evaluate the entroy of a erturbation, i.e., hg 1 + εl)), we can exress it as the entroy of hg ) minus the divergence ga, as in 14), and then use Lemma 1 for the aroximation. But this is correct if g 1 + εl) has the same first two moments as g. Hence, if L contains only H s with 3, the revious argument can be used. But if L contains H 1 and/or H terms, the situation can be different. Next Lemma describes this. Lemma : Let δ, > 0 and { bh [] + δh[] 4 ), if b 0, [] b H = bh [] We have for any α R, 1, ε > 0 hg 1 + ε 1 α H )) = hg ) Dg 1 + ε 1 δh[] 4 ), if b < 0. ) α H ) g ) + εα. Finally, when we convolve two erturbed Gaussian distributions, we get g a 1 + εh j ) g b 1 + εh ) = g a+b + ε[g a H j g b + g a g b H ] + ε g a H j g b H. We already now from Theorem how to describe the terms in ε, what we still need is to describe the term in ε. We have the following. Lemma 3: We have g a H [a] g bh [b] l = Cg a+b H [a+b] +l, where C is a constant deending only on a, b, and l. In articular if = l = 1, we have C = ab a+b. V. PROOFS We start by reviewing the roof of 16), as it brings interesting facts. We then rove the main result. Proof of 16): We first assume that f ε has zero mean and variance. Using the Hermite basis, we exress L as L = 3 α H [] L must have such an exansion, since it must have a finite L g )

8 norm, to mae sense of the original exressions). Using 19), we can then exress 16) as ) α α 3) + v 3 3 which is clearly negative. Hence, we have roved that g L g v g +v g +v L g 4) and 16) is maximized by taing L = 0. Note that we can get tighter bounds than the one in revious inequality, indeed the tightest, holding for H 3, is given by g L g v g +v g +v + v ) 3 L g 5) this clearly holds if written as a series lie in 3)). Hence, locally the contraction roerty can be tightened, and locally, we have stronger EPI s, or worst noise case. Namely, if ν +v ) 3, we have arg min hf g v) νhf) = g 6) f: m 1f)=0,m f)= ) and if ν < 3, +v g is outerformed by non-gaussian distributions. Now, if we consider the constraint m f), which, in articular, allows to consider m 1 f) > 0 and m f) =, we get that if ν +v, arg min hf g v) νhf) = g 7) f: m f) and if ν < +v, g is outerformed by g δ for some δ > 0. It would then be interesting to study if these tighter results hold in a greater generality than for the local setting. Proof of Theorem : We want to show H [+v] g v = + v )/ H [], g H [] g v = + v )/ g +v H [+v], which is roved by an induction on, using the following roerties Aell sequence and recurrence relation) of Hermite olynomials: x H[] +1 x) = ) g x)h [] x x) = + 1 H[] x) + 1 g x)h [] +1 x). Proof of Theorem 3: We refer to 1) as the mu-rate. Let us first consider Gaussian codes, i.e., when U, X) is jointly Gaussian, and see what mu-rate they can achieve. Without loss of generality, we can assume that X = U + V, with U and V indeendent and Gaussian, with resective variance Q and R satisfying P = Q + R. Then, 1) becomes 1 RH E log1 + v ) + µ1 1 + P H E log 1 + RH. 8) Now, we ic a µ and loo for the otimal ower R that must be allocated to V in order to maximize the above exression. We are interested in cases for which the otimal R is not at the boundary but at an extremum of 8), and if the maxima is unique, the otimal R is found by the first derivative chec, which gives E H v+rh = µe H 1+RH. Since we will loo for µ, v, with R > 0, revious condition can be written as E RH v + RH = µe RH 1 + RH. 9) We now chec if we can imrove on 8) by moving away from the otimal jointly Gaussian U, X). There are several ways to erturb U, X), we consider first the following case. We ee U and V indeendent, but erturb them away from Gaussian s in the following way: Uε u) = g Q u)1 + εh [Q] 3 u) + δh 4 )) 30) Vε v) = g R v)1 εh [R] 3 v) δh 4)) 31) with ε, δ > 0 small enough. Note that these are valid density functions and that they reserve the first two moments of U and V. The reason why we add δh 4, is to ensure that 13) is satisfied, but we will see that for our urose, this can essentially be neglected. Then, using Lemma, the new distribution of X is given by Q X x) = g P x)1 + ε P ) 3 H [P ] 3 ε Q P ) 3 R [P ] H 3 ) + fδ) P ) 4 H [P ] 4 + ) 4 R P H [P ] 4 ), which where fδ) = δg P x)ε tends to zero when δ tends to zero. Now, by icing P = R, we have X x) = g P x) + fδ). 3) Hence, by taing δ arbitrarily small, the distribution of X is arbitrarily close to the Gaussian distribution with variance P. We now want to evaluate how these Hermite erturbations erform, given that we want to maximize 1), i.e., hy 1 U, H) hz 1 ) + µhy H) µhy U, H). 33) We wonder if, by moving away from Gaussian distributions, the gain achieved for the term hy U, H) is higher than the loss suffered from the other terms. Using Theorem, Lemma 1 and Lemma, we are able to recisely measure this and we get hy 1 U = u, H = h) ) 3 Rh = hg hu,v+rh 1 ε v + Rh H [hu,v+rh ] 3 )) = 1 ) log πev + Rh ) ε Rh 3 v + Rh + oε ) + oδ)

9 hy U = u, H = h) = 1 log πe1 + Rh ) ε and because of 3) Rh 1 + Rh ) 3 + oε ) + oδ) hy H = h) = 1 log πe1 + P h ) + oε ) + oδ). Therefore, collecting all terms, we find that for U ε and V ε defined in 30) and 31), exression 41) reduces to ) I G ε RH 3 ) E v + RH + µ ε RH 3 E 1 + RH + oε ) + oδ) 34) where I G is equal to 8) and is the mu-rate obtained with Gaussian inuts). Hence, if for some distribution of H and some v, we have that ) RH ) RH µe 1 + RH E v + RH > 0, 35) when = 3 and R is otimal for µ, we can tae ε and δ small enough in order to mae 34) strictly larger than I G. We have shown how, if verified, inequality 35) leads to counter-examles of the Gaussian otimality, but with similar exansions, we would also get counter-examles if the following inequality holds for any ower instead of 3, as long as 3. Let us summarize what we obtained: Let R be otimal for µ, which means that 9) holds if there is only one maxima not at the boarder). Then, non-gaussian codes along Hermite s strictly outerforms Gaussian codes, if, for some 3, 35) holds. If the maxima is unique, this becomes where ET v) ET v) < ET 1) ET 1) T v) = RH v + RH. So we want the Jensen ga of T v) for the ower to be small enough comared to the Jensen ga of T 1). We now give an examle of a fading distribution for which the above conditions can be verified. Let H be binary, taing values 1 and 10 with robability half and let v = 1/4. Let µ = 5/4, then for any values of P, the maximizer of 8) is at R = , cf. Figure 1, which corresonds in this case to the unique value of R for which 9) is satisfied. Hence if P is larger than this value of R, there is a corresonding fading BC for which the best Gaussian code slits the ower on U and V with R = to achieve the best murate with µ = 5/4. To fit the counter-examles with the choice of Hermite erturbations made reviously, we ic P = R. Finally, for these values of µ and R, 35) can be verified for = 8, cf. Figure, and the corresonding Hermite code along H 8 ) strictly outerforms any Gaussian codes. Note that we can consider other non-gaussian encoders, such as when U and V are indeendent with U Gaussian and Fig. 1. Gaussian mu-rate, i.e., exression 8), lotted as a function of R for µ = 5/4, v = 1/4, P = and H binary {1; 10}. Maxima at R = Fig.. LHS of 35) as a function of R, for µ = 5/4, v = 1/4, = 8 and H binary {1; 10}, ositive at R = V non-gaussian along Hermite s. Then, we get the following condition. If for 3 and R otimal for µ, we have ) RH E v + RH 36) [ ) RH ) ) R P H ] < µ E 1 + RH E P 1 + P H, 37) then Gaussian encoders are not otimal. Notice that revious inequality is stronger than the one in 35) for fixed values of the arameters. Yet, it can still be verified for valid values of the arameters and there are also codes with U Gaussian and V non-gaussian that outerform Gaussian codes for some degraded fading BCs.

10 Proof of Theorem 4: Let ε, δ > 0 and let X 1 and X be resectively distributed as g 1 + ε[h + δh 4 ]) and g 1 ε[h δh 4 ]), where 1,. We have IX 1, X 1 + ax + Z 1 ) = hx 1 + ax + Z 1 ) hax + Z 1 ) where X G are indeendent Gaussian 0-mean and -variance random variables. Hence, we need to evaluate the contribution of each divergence aearing in revious exression, in order to now if the erturbations are imroving on the Gaussian distributions. Let us first analyze hx 1 + ax + Z 1 ). The density of X 1 + ax + Z 1 is given by g 1 + ε[h + δh 4 ]) g a 1 ε[h δh 4 ]) g 1, 38) which, from Theorem, is equal to g +a +11+ [ ε{ + a + 1 [ a + a + 1 where εl}) ) H + δ ) H δ ) + a H 4] + 1 a ) + a H 4] + 1 L = g [H + δh 4 ] g a [H δh 4 ] g 1. g +a +1 Note that each direction in each line of the bracet { } above, including L, satisfy 10) and 11). Using Lemma 3, we have ) g [H + δh 4 ] g a a +1[ a L = +1 H a = C 1 H [+a +1] + C H [+a +1] 5 g +a +1 a +1) δh4 ] + C 3 H [+a +1] 8, 39) where C 1, C, C 3 are constants. Therefore, the density of X 1 + ax +Z 1 is a Gaussian g +a +1 erturbed along the direction H in the order ε and several H l with l in the order ε and other directions but that have a δ order). So we can use Lemma and write hx 1 + ax + Z 1 ) = hx G 1 + ax G + Z 1 ) DX 1 + ax + Z 1 X G 1 + ax G + Z 1 ) Using Lemma 1, we have DX 1 + ax + Z 1 X1 G + ax G + Z 1 ) [ ) ) = ε H + a + δ a H 4] + 1 [ a ) a ) + a H δ a H 4] + 1 ) = ε 1 a ) + a + ε + 1 oδ). Hence hx 1 + ax + Z 1 ) = hx1 G + ax G + Z 1 ) ) ε 1 a ) + a + ε + 1 oδ). Similarly, we get and DaX + Z 1 ax G + Z 1 ) = ε a ) a + ε + 1 oδ) IX 1, X 1 + ax + Z 1 ) = IX1 G, X1 G + ax G + Z 1 ) [ + ε a ) ) ] a 1 a ) a + ε + 1 oδ). Finally, we have and IX, X + ax 1 + Z ) = IX 1, X 1 + ax + Z 1 ) IX 1, X 1 + ax + Z 1 ) + IX, X + ax 1 + Z ) = IX1 G, X1 G + ax G + Z 1 ) + IX G, X G + ax1 G + Z ) + ε [ a a + 1 ) 1 a ) + a + 1 Hence, if for some 3 we have ) a a 1) a + 1 ) ] + ε oδ). + a + 1) > 0 40) we can imrove on the iid Gaussian distributions g by using the resective Hermite erturbations. Now, we could have started with X 1 and X distributed as g 1 + εb H ) and g 1 + εc H ), where H is defined in ). With similar exansions, we would then get that we can imrove on the Gaussian distributions if for some some b, c and 1, we have [ a ) ] a a ) a + 1) b + c ) ) a 4 + a b c > But the quadratic function b, c) R γb + c ) δbc, with δ > 0, can be made ositive if and only if γ +δ > 0, and is made so by taing b = c. Hence, the initial choice we made about X 1 and X is otimal. Moreover, note that for this distribution of X 1 and X, we could have actually chosen = as well. Because, even if Lemma tells us that we must use correction terms, these correction terms will cancel out when we consider the sum-rate, since b = c and since the correction is in ε. There is however another roblem when using =, which is that g 1 + εh ) has a larger second moment than. However, if we use a scheme of bloc length

11 , we can comensate this excess on the first channel use with the second channel use, and because of the symmetry, we can achieve the desired rate. But this is allowed only with synchronization. We could also have used erturbations that are mixtures of Hermite s, such as g 1 + ε b H ). We would then get mixtures of revious equations as our condition. But in the current roblem this will not be helful. Finally, erturbing iid Gaussian inuts in a indeendent but non i.d. way, i.e., to erturb different comonents in different Hermite directions, cannot imrove on our scheme, from revious arguments. The only otion which is not investigated here but in a wor in rogress), is to erturb iid Gaussian inuts in a non indeendent manner. Finally, if we wor with = 1, the roof sees the following modification. In 39), we now have a term in H. However, even if this term is in the order ε, we can no longer neglect it, since from Lemma, a ε H term in the direction comes out as a ε term in the entroy. Hence, we do not get the above condition for = 1, but the one obtained by relacing a 1) with a +1), and the condition for ositivity can never be fulfilled. Proof of Proosition 1: From Theorem 4, we now that when treating interference as noise and when F a, ) > 0, it is better to use encoders drawn from the dimensional distributions X 1 and X, where X 1 ) 1 g 1 + ε H ), X ) 1 g 1 ε H ), X 1 ) g 1 ε H ) and X ) g 1 + ε H ), as oosed to using Gaussian distributions. But erturbations in H are changing the second moment of the inut distribution. Hence, this scheme is mimicing a time-sharing in our local setting. Moreover, a direct comutation also allows to show that, constraining each user to use Gaussian inuts of arbitrarily bloc length n, with arbitrary covariances having a trace bounded by np, the otimal covariances are I n if F a, ) 0, and otherwise, are given by a time-sharing scheme cf. Definition 1 for the definition of a Gaussian time-sharing scheme and covariance matrices). Proof of Proosition : Note that when using blind time-sharing, no matter what the delay in the asynchronization of each user is, the users are interfering in n/4 channel uses and have each a non-intefering channel in n/4 channel uses the rest of the n/4 channel uses are not used by any users). Hence, if the receiver have the nowledge of the asynchronization delay, the following sum-rate can be achieved: 1 4 log1 + ) + log1 + 1+a )). And if the delay is unnown to the receivers, the revious sum-rate can surely not be imroved on. Disroof of Conjecture 1: This roof uses similar stes as revious roofs. Using Lemma 14), we exress IX; X + hx G 1 + Z) IX; X + hx 1 + Z). as DX + hx G 1 + Z X G + hx G 1 + Z) DX + hx 1 + Z X G + hx G 1 + Z) + DhX 1 + Z hx G 1 + Z). 41) We then ic X, X 1 g 1+ε H ) and assume that is even for now. We then have and Similarly, and Finally, and X + hx1 G + Z g 1 + εh ) g h +v = g +h +v1 + ε + h + v )/ H ) DX + hx G 1 + Z X G + hx G 1 + Z) = ε + h + v ). X + hx 1 + Z g 1 + εh ) g h 1 + H ) g v h = g 1 + εh ) g h +v1 + h + v )/ H ) g +h +v1 + ε + h + v )/ H h + ε + h + v )/ H ) DX + hx 1 + Z X G + hx G 1 + Z) = ε + h + v ) 1 + h ). hx 1 + Z g h 1 + εh ) g v h = g h +v1 + ε h + v )/ H ) DhX 1 + Z hx G 1 + Z) = ε h h + v ), Therefore, 41) is given by ) + h + v ) ) h + h 1 + h ) + + v h + o1) + v and if ) ) + h 1 + h ) + v + h + v ) h h > 0 4) + v

12 Fig. 3. for some even and greater than 4, we have a counter examle to the strong conjecture. Note that, using the same tric as in revious roofs, that is, erturbing along H instead of H, we get that if 4) holds for any 3, we have a counter examle to the strong conjecture. Defining u := v/ = 1/SNR, 4) is equivalent to Gh, u, ) := 1 + h ) 1 + h + u) ) 1 h ) > 0. 43) 1 + h + u h + u As shown in Figure 3, this can indeed haen. An interesting observation is that the range where 43) holds is broader when u is larger, i.e., when SNR is smaller. Indeed, when u = v = 0, which corresonds to droing the additive noise Z, we do not get a counter-examle to the conjecture. But in the resence of Gaussian noise, the conjecture does not hold for some distributions of X, X 1. The conjecture had been numerically checed with binary inuts at low SNR, and in this regime, it could not be disroved. With the hint described above, we checed numerically the conjecture with binary inuts at high SNR, and there we found counter-examles. VI. DISCUSSION We have develoed a technique to analyze codes drawn from non-gaussian ensembles using the Hermite olynomials. If the erformance of non-gaussian inuts is usually hard to analyze, we showed how with this tool, it reduces to the analysis of analytic ower series. This allowed us to show that Gaussian inuts are in general not otimal for degraded fading Gaussian BC, although they might still be otimal for many fading distributions. For the IC roblem, we found that in the asynchronous setting and when treating interference as noise, using non-gaussian code ensembles H 3 erturbations) can strictly imrove on using Gaussian ones, when the interference coefficient is above a given threshold, which significantly imroves on the existing threshold cf. [4]). We have also recovered the threshold of the moderate regime by using H erturbations in the synch setting, showing that this global threshold is reflected in our local setting. We also met mysteriously in our local setting the other global threshold found in [], [8], [10], below which treating interference as noise with iid Gaussian inuts is otimal. It is worth noting that this two global thresholds moderate regime and noisy interference) are recovered with our tool from a common analytic function. We hoe to understand this better with a wor in rogress. The Hermite technique rovides not only counter-examles to the otimality of Gaussian inuts but it also gives insight on the cometitive situations in Gaussian networ roblems. For examle, in the fading BC roblem, the Hermite technique gives a condition on what ind of fading distributions and degradedness values of v) non-gaussian inuts must be used. It also oints out that the erturbation in H 3 are most effective when carried in an oosite manner for the two users, so as to mae the distribution of X close to Gaussian. Finally, in a different context, local results could be lifted to corresonding global results in [1]. There, the localization is made with resect to the channels and not the inut distribution, yet, it would be interesting to comare the local with the global behavior for the current roblem too. The fact that we have observed some global results locally, as mentioned reviously, gives hoe for ossible local to global extensions. A wor in rogress aims to use our tool beyond the local setting, in articular, by analyzing all sub-gaussian distributions. Moreover, there are interesting connections between the results develoed in this aer and the roerties of the Ornstein- Uhlenbec rocess. Indeed, some of these roerties have already been used in [3] to solve the long standing entroy monotonicity conjecture, and we are currently investigating these relations from closer. ACKNOWLEDGMENT The authors would lie to than Tie Liu and Shlomo Shamai for ointing out roblems relevant to the alication of the roosed tool, Daniel Strooc for discussions on Hermite olynomials, and Emre Telatar for stimulating discussions and helful comments. REFERENCES [1] E. Abbe and L. Zheng, Linear universal decoding: a local to global geometric aroach, Submitted IEEE Trans. Inform. Theory, 008. [] V.S. Annaureddy and V.V. Veeravalli, Gaussian Interference Networ: sum Caacity in the Low Interference Regime and New Outer Bounds on the Caacity Region, Submitted IEEE Trans. Inform. Theory, 008. [3] S. Artstein, K. Ball, F. Barthe and A. Naor, Solution of Shannon s Problem on the Monotonicity of Entroy, Journal of the American Mathematical Society 17, ). [4] E. Calvo, J. Fonollosa, and J. Vidal, The Totally Asynchronous Interference Channel with Single-User Receivers, ISIT 009, Seoul. [5] M. Costa, On the Gaussian interference channel, IEEE Trans. Inform. Theory, vol. IT-31, , Set [6] T. M. Cover, Comments on broadcast channel, IEEE Trans. on Inform. Theory vol. 44, n. 6, , Oct

13 [7] T. M. Cover and J. A. Thomas, Elements of Information Theory, John Wiley & Sons, New Yor, NY, [8] A. S. Motahari, A. K. Khandani, Caacity Bounds for the Gaussian Interference Channel, CoRR abs/ : 008). [9] S. Shamai Shitz) and R. Laroia, The intersymbol interference channel: Lower bounds on caacity and channel recoding loss, IEEE Trans. Inform. Theory, vol. 4, , Set [10] X. Shang, G. Kramer, B. Chen, Outer bound and noisy-interference sum-rate caacity for symmetric Gaussian interference channels, CISS 008: [11] A.J. Stam, Some inequalities satisfied by the quantities of information of Fisher and Shannon, Information and Control, : , [1] D. W. Strooc, A Concise Introduction to the Theory of Integration, Third Edition, Birhäuser, [13] D. Tuninetti and S. Shamai, On two-user fading Gaussian broadcast channels with erfect channel state information at the receivers, Int. Sym. on Inform. Theory ISIT), 003.