FOR a continuous random variable X with density g(x), the differential entropy is defined as

Transcription

1 Higher Order Derivatives in Costa s Entrop Power Inequalit Fan Cheng, Member, IEEE and Yanlin Geng, Member, IEEE arxiv:09.v [cs.it] Aug 0 Abstract Let X be an arbitrar continuous random variable and Z be an independent Gaussian random variable with zero mean and unit variance. For t > 0, Costa proved that e hx+ tz is concave in t, where the proo hinged on the irst and second order derivatives o hx + tz. Speciicall, these two derivatives are signed, i.e., hx + tz 0 and hx + tz 0. t t In this paper, we show that the third order derivative o hx + tz is nonnegative, which implies that the Fisher inormation JX + tz is convex in t. We urther show that the ourth order derivative o hx + tz is nonpositive. Following the irst our derivatives, we make two conjectures on hx + tz: the irst is that n hx + tz is nonnegative in t i n is odd, and t n nonpositive otherwise; the second is that log JX + tz is convex in t. The irst conjecture can be rephrased in the context o completel monotone unctions: JX + tz is completel monotone in t. The histor o the irst conjecture ma date back to a problem in mathematical phsics studied b McKean in 9. Apart rom these results, we provide a geometrical interpretation to the covariance-preserving transormation and stud the concavit o h tx + tz, revealing its connection with Costa s EPI. Index Terms Costa s EPI, Completel monotone unction, Dierential entrop, Entrop power inequalit, Fisher inormation, Heat equation, McKean s problem. I. INTRODUCTION FOR a continuous random variable X with densit gx, the dierential entrop is deined as + hx : gx log gxdx, where log is the natural logarithm. The Fisher inormation e.g., Cover [, p. ] is deined as [ + x JX : gx gx ] dx. gx The entrop power inequalit EPI introduced b Shannon [] states that or an two independent continuous random variables X and Y, e hx+y e hx + e hy, where the equalit holds i and onl i both X and Y are Gaussian. Shannon did not give a proo and there was a gap in his argument. The irst rigorous proo was made b Stam in [], where he applied an equalit that connected Fisher inormation and dierential entrop and the so-called Fisher inormation inequalit FII was proved; i.e., JX + Y JX + JY. Later, Stam s proo was simpliied b Blachman []. Zamir [] proved the FII via a data processing argument in Fisher inormation. Lieb [] showed an equivalent orm o EPI and proved the equivalent orm via Young s inequalit. Lieb s argument has been widel used as a common step in the subsequent proos o EPI. Recentl, Verdú and Guo [] gave a proo b invoking an equalit that related minimum mean square error estimation and dierential entrop. Rioul [] devised a Markov chain on X, Y, and the additive Gaussian noise, rom which EPI can be proved via the data processing inequalit and properties o mutual inormation. F. Cheng was with the Institute o Network Coding, The Chinese Universit o Hong Kong, N.T., Hong Kong. He is now with the department o ECE, NUS, Singapore. chengan@gmail.com Y. Geng was with the Department o Inormation Engineering, The Chinese Universit o Hong Kong, N.T., Hong Kong. Now he is with the School o Inormation Science and Technolog, ShanghaiTech Universit, China. genganlin@gmail.com The work o F. Cheng was partiall unded b a grant rom the Universit Grants Committee o the Hong Kong Special Administrative Region Project No. AoE/E-0/0 and Shenzhen Ke Laborator o Network Coding Ke Technolog and Application, Shenzhen, China ZSDY0099. The work o Y. Geng was partiall supported b a GRF grant rom the Universit Grants Committee o the Hong Kong Special Administrative Region Project No. 0 and the Science and Technolog Commission o Shanghai Municipalit YF0900. This paper was presented in part at IEEE Iran Workshop on Communication and Inormation Theor, 0 [0].

2 There are several generalizations o EPI. Costa [] proved that the entrop power e hx+ tz is concave in t, where the irst and second order derivatives o hx + tz were obtained. Moreover, these two derivatives are signed, i.e., positive or negative. Dembo [9] gave a simple proo to Costa s EPI via FII. Villani [] simpliied the proo in [] b using some advanced techniques as well as the heat equation noticed b [], which is instrumental in our work. The generalization o EPI in matrix orm was obtained in Zamir and Feder [0]. Liu and Viswanath [] generalized EPI b considering a covariance-constrained optimization problem which was motivated b multi-terminal coding problems. Wang and Madiman [] discussed EPI rom the perspective o rearrangement. As one o the most important inormation inequalities, EPI FII has numerous proos, generalizations, and applications. In Barron [9], FII was emploed to strengthen the central limit theorem. The relationships o EPI to inequalities in other branches o mathematics can be ound in Dembo et al. []. The literature is so vast that instead o tring to be complete, we onl mention the results that are most relevant to our discussion. A recent comprehensive surve can be ound in [], and the book b El Gamal and Kim [] also serves as a ver good repositor. In this paper, inspired b [], we make some progress and introduce related conjectures which reveal even more undamental acts about Gaussian random variables in the view o inormation theor. B harnessing the power o the techniques in [], i.e, heat equation and integration b parts, we obtain the third and ourth order derivatives o hx + tz, which are also signed. Summarizing all the derivatives o hx + tz, we conjecture that n t hx + tz is signed or an n. Corresponding to n Costa s EPI, we urther conjecture that log JX + tz is convex in t. We investigate the concavit o h tx + tz, showing that it is concave in t and is equivalent to Costa s EPI. We provide a geometrical interpretation to the covariancepreserving transormation. The connection between the convexities o J tx + tz and log JX + tz is also revealed. Finall, we state some results rom the literature, including McKean s problem and completel monotone unctions. The paper is organized as ollows. In Section II, we introduce the background and the main result on derivatives. In Section III, some preliminaries are stated. In Section IV and V, the derivatives are veriied. We discuss the uniqueness o the signed orm in Section VI. The conjecture is introduced in Section VII. In Section VIII, we give a geometrical interpretation to the covariance-preserving transormation and prove an inequalit which is equivalent to Costa s EPI. In Section IX, we discuss some urther issues. We conclude the paper in Section X. II. THE HIGH-ORDER DERIVATIVES Consider a random variable X with densit gx, and an independent standard Gaussian random variable Z, denoted as Z N 0,. For t 0, let Y t : X + tz. The densit o Y t is, t + gx e x t dx. πt Notation: For the derivatives, in addition to the usages o, t and, b n we alwas mean Sometimes, or ease o notation we also denote n : n n. n : n n n. The integration interval, usuall, +, will be omitted, unless it is not clear rom the context. In this paper, the main result is the ollowing two theorems. Theorem. For t > 0, This implies that JY t is convex in t. t hy t + + d.

3 Theorem. For t > 0, This implies that t hy t 0. t hy t Our work is highl related to the ollowing theorem Theorem Costa s EPI []. e hyt is concave in t, where t > 0. d. There are several methods to prove Costa s EPI, and a straightorward wa is to calculate the irst and second order derivatives o hy t and show some inequalit holds. The expressions on t hy t and t hy t are alread obtained in Lemma. Lemma. t hy t JY t; t hy t d. The proo can be ound in [], []. The irst equation is called de Bruijn s identit in the literature and is due to de Bruijn. Using Lemma, one can readil show that t hy t 0 and t hy t 0. In Theorem, we have presented the expressions o t hy t and showed that t hy t 0. A much more complicated result on t hy t is stated in Theorem. We notice that in Guo et al. [, Proposition 9], the third and ourth order derivatives o h tx + Z have been computed, but these derivatives cannot determine the corresponding signs o hx + tz. However, the signs o hx + tz are determined b Theorem and. In the next section, we introduce the necessar tools to prove Theorem and. III. PRELIMINARIES The dierential entrop and Fisher inormation ma not be well deined due to the integration issue. In the literature, there are no simple and general conditions which can guarantee their existence c.. []. In general, the behavior o the dierential entrop and Fisher inormation ma be unpredictable as shown b Wu and Verdú []. However, this work studies the higher order derivatives o hy t, where t > 0 is imposed. Under this assumption, Y t has some good properties; e.g., in [], the densit o Y t is proved to be ininitel dierentiable everwhere. A. Properties o, t The ollowing propert is well known e.g., [, Lemma ]. Proposition. For an ixed t > 0 and an n Z +, all the derivatives n, t exist, are bounded, and satis lim n, t 0. The ollowing propert is used repeatedl in the rest o the paper, or dealing with integration b parts. The proo is presented in Appendix A. Proposition. For an r, m i, k i Z +, the ollowing integral exists r [ mi ] ki d. In particular, this implies that i ki r lim [ mi ] ki 0. ki i

4 B. The heat equation For a Gaussian random variable ˆX N µ, σ with densit unction ˆx, one can show that the ollowing heat equation holds σ ˆ x ˆ. The heat equation also holds or Y t [], and was used b [] to simpli Costa s proo. Lemma. t, t, t. 9 Proo: The proo is known in the literature and we present it here or completeness. B some calculus, t gx e x x t πt t dx, t gx e x t t x dx, πt [ ] gx e x t x dx. πt t t B comparing with t, the lemma can be proved. C. Proo to Lemma The proo to Lemma is known in the literature. Here we slightl modi the proo, so that the idea carries over to the proo o Theorem and even the cases with higher-order derivatives. Proo: For the irst order derivative we have t hy t [ ], t log, td t t + log d 9 + log d + log d a + + log + b 0 + d JY t. d In a we appl integration b parts. In b the limits are zero, because + log + log, where 0 rom Proposition, 0 as, and log 0 because x log x 0 as x 0. For the second order derivative, similarl For the second term t hy t 9 t t d d. d d + 0 d + d d.

5 Hence For the irst term Combining these two terms we have d d d d. 0 0 t hy t 0 d + d + d. d Now it suices to show that the right-hand side term in has the same orm: d d. d + d Thus the proo is inished. One ma notice that we irst use the heat equation to deal with t, then appl integration b parts to eliminate those terms whose highest-order derivatives have power one. Equation explains this elimination, as one can see that in the inal expression the highest-order derivatives are and, whose powers are bigger than one. IV. PROOF TO THEOREM The ollowing lemma is instrumental in proving Theorem. Lemma. d d d + d d + d d + d Proo: See Appendix C. This lemma is similar to what we did in equations 0 and : For the terms on the let-hand side, the highest-order derivatives have power one; while or the right-hand side, the are bigger than one. Next, we prove Theorem. Proo: From Thus hy t t + d hy t t + d. t + d.

6 B repeatedl appling the heat equation, d t Substitute these terms and use Lemma : d t hy t t t t d 9 d d t t d 9 d Lemma d d d + d Then we do the same manipulations to t hy t in Theorem. That is, appling Lemma to the corresponding terms and we have Thus the expression is proved. Finall, which means JY t is convex in t. + d d Lemma d t JY t t hy t 0, V. PROOF TO THEOREM + d The proo is the same as that to Theorem, except there are more manipulations. The ollowing lemma is instrumental in proving Theorem.

7 Lemma. Proo: See Appendix D. Next, we prove Theorem. Proo: According to We irst appl the heat equation: d d d + 0 d 9 d + d 0 d + d d + 0 d d + d d + d d + d d + d hy t t + + d t d t d t t t d 9 d d t t d 9 d d d t t + t t d d + 9 d

8 Substitute these terms and use Lemma : d t t t 0 d 9 0 d d hy t t d Lemma d d d Then we do the same manipulations to t hy t in Theorem. That is, appling Lemma to the corresponding terms.

9 9 To simpli the calculation, we irst consider the ollowing general expression x 0 + x + x + x + x d x 0 + x + x + x + x + x 0 x + x 0 x + x 0 x + x 0 x + x x + x x + x x + x x + x x + x x d x 0 + x + x + x + x + x 0x + Lemma + x 0 x + + x 0 x + + x 0 x x x + + x x + + x x x x + x x + x x x 0 + x + x 0 x x 0 x + x x 0x + x 0x x x + x + x 0 x x 0 x + x x 0x x + x x + x + 0 x 0x + 0 x x + x x + x 0 x x 0 x + x 0 x x 0 x + x 0 x + x x d x x + x x d,

10 0 With this general simpliication, we have d d d d d d d d d 0000

11 B 9, 0 and which completes the proo o Theorem d d d d, VI. ALTERNATIVE SIGNED REPRESENTATIONS In this section, we discuss alternative signed representations o n t n hy t in Lemma, Theorem, and Theorem. For the irst order derivative, the representation is unique due to its simplicit. For the second and third order derivatives, we have the ollowing alternative representations stated in Corollar and. The proo o Corollar, though simple, contains the idea o how we obtain the ormulae in Theorem and. Corollar. t hy t α + β + γ where + α + β γ αβ d α 0 β γ αβ 0

12 One set o solution is where the case β corresponds to the result in Lemma. α, γ 0, β, Proo: Ater appling the heat equation, the orders o derivatives in each term o t hy t have sum equals our. Thus we consider expressing the second derivative as t hy t i α i + β i d, where α i and β i are coeicients. Since or the reals A, B, C, a, b, the ollowing equalit holds aa + B + ba + C a + b A + it suices to consider the ollowing expression t hy t a a + b B + b a + b C b + a + b B a a + b C, α + β + γ d. Now similar to the proo to Theorem, α + β + γ d α + αβ + β + γ d 0 α + β + γ + αβ d. Comparing with, one obtains t hy t α + β To show that the second derivative is negative, one requires And it is eas to veri the set o solution + γ + α + β γ αβ d. α 0 β γ αβ 0. α, γ 0, β. For the third derivative, similar to Corollar, one could determine the coeicients c i in the ollowing t hy t c 0 + c + c + c + c + c d. Since there is no essential dierence, we would not present the general expression or the third derivative, but just prove the ollowing corollar.

13 Corollar. t hy t + β where β + 9. Proo: We have + β + β β d, + β + β + β β d + β + β + β β d Lemma + β + β + + β + β d + + d. The interval o β ensures that the coeicients are positive. For the second and third order derivatives o hy t, the representations can be obtained b hand. For the ourth order derivative, we consider the ollowing representation t hy t c 0 + c c c 9 + c + c + c + c + c + c 0 + c c + c + c + c d. B, we can obtain some constraints similar to, and inall ind the easible set o coeicients in Theorem b numerical methods. The process is much more complicated, and we would not present it here. VII. CONJECTURES Motivated b Theorem, we would like to introduce the ollowing conjectures. Conjecture. The n-th order derivative o hy t satisies. n t hy n t 0 when n is even;. n t hy n t 0 when n is odd; i.e., n t hy n t is either convex or concave in t or a ixed n. It is eas to see that when X is Gaussian, the above conjectures hold. Conjecture speculates that or a ixed n, the convexit or concavit o n t hy n t remains as i X is Gaussian. Conjecture has been veriied or n in the literature Lemma, and or n, b Theorem and.

14 a JY t b t JY t Fig.. is neither concave nor convex. JY t Remark. The general pattern or the signed orm o the n-th order derivative is that, irst we need to ind all the partitions o n, and then each partition is an item in the squares. But the exact coeicients are hard to obtain. One can appl the same technique to deal with the ith derivative, or even higher. However, the manipulation b hand is huge and hence it is prohibitive in computational cost, unless one can ind some patterns or the coeicients in the signed representations. Some sotwares like Mathematica ma be useul to veri the higher order derivatives based on the simple rules observed rom the ourth derivative, but we still need a mathematical proo. The second conjecture is on the log-convexit o Fisher inormation. From the grand picture o dierential entrop and Fisher inormation, nearl ever result on dierent entrop has a counterpart in Fisher inormation, e.g., Shannon EPI and FII, the concavit o hy t and the convexit o JY t as well as de Bruijn identit. Corresponding to Costa s EPI, there ma be a strengthened convexit o JY t. Conjecture log-convex. log JY t is convex in t. When X is standard white Gaussian, JY t t+. We ma speculate JY or log JY t t is convex in t. Simulations show that JY is neither convex nor concave. Fig. illustrates an example o t JY t, where X is mixed Gaussian with p.d.. gx 0. G 0, G 0, 0. and G µ, σ is the p.d.. o Gaussian N µ, σ. Limited simulations show that log JY t is convex. Remark. Ater inishing this paper, we realized that Conjecture implies Conjecture. See Section IX or the details. For 0 < t <, let VIII. CONCAVITY OF h tx + tz W t : tx + tz, where Z N 0, is independent o X. In this section, we stud the concavit and convexit o hw t and JW t, respectivel. Lieb showed that Shannon EPI is equivalent to h λx + λx λhx + λhx or an 0 λ. Here we use X and X in lieu o X and Y as the independent random variables. In the literature, X, X λx + λx is reerred to as the covariance-preserving transormation, which can be ound in man generalizations o Shannon EPI []. The original proo o Lieb is a little trick. Next, we give a geometrical interpretation o this transormation which can help us to have a better appreciation on λx + λx. A. Covariance-preserving Transormation Recall that a convex unction has the ollowing three equivalent statements. Let x be a unction which is twice dierentiable, where x R n. Then the ollowing are equivalent:. x is convex in x.

15 . The Hessian matrix o x is positive semi-deinite; i.e.,. For an ixed point x 0, 0. x x 0 + x x 0 T x 0. Furthermore, x 0 + x x 0 T x 0 can be viewed as the tangent plane at point x 0, x 0 or unction x. In the ollowing, we shall appl the above argument on convex unctions to stud the so-called covariance-preserving transormation. Shannon EPI can be equivalentl transormed to hx + X log e hx + e hx. Let s stud unction x, x log e x + e x. B some manipulations,, e x x x e x + e, e x x e x + e [ x ] [ ] ex e x x i x j e x + e x. ij It is eas to see that x, x is convex since 0. B, the tangent plane o x, x at point x, x logσ, logσ is log σ + σ + x σ logσ σ + σ + x σ logσ σ +. σ Hence, is equivalent to Let Notice that hax hx + log a, we have Substitute X, X with λx, λx, hx + X log σ + σ + hx σ logσ σ + σ + hx σ logσ σ +. 9 σ λ σ σ +. σ hx + X λhx / λ + λhx / λ. 0 h λx + λx λhx + λhx, which is exactl the inequalit. In the above proo, the points share the same tangent plane 0 as long as the admit the same λ. In act, all the results see [] that applied covariance-preserving transormation can be proved in this manner. B. The concavit o hw t Theorem. hw t is concave in t, 0 < t <. Proo: Since hw t hx + /t Z + log t,, b some algebra, we obtain t hw t JX + /t Z t + t

16 and To show we need to prove That is B, Costa s EPI is equivalent to or an s > 0. Thereore, t hw t J X + /t Z t + JX + /t Z t t. t hw t 0, J X + /t Z t + t JX + /t Z t. J X + /t Z + t tjx + /t Z. J X + sz JX + sz, J X + /t Z + t JX + /t Z + t tjx + /t Z, which is. In all the results above, as t > 0, X + tz can be replaced b X + sz s0, where X X + tẑ and Ẑ is the standard Gaussian and is independent o X and Z. In this manner, we onl need to prove that the result holds or an such X at point s 0. In light o the smoothness introduced b tz where t > 0, without weakening our result, we can just assume that when t 0, the n-th order derivative o hx + tz exists in the sequel. Next, we show that Theorem can impl Costa s EPI. In the above proo, i JX and J X are well deined, then let t in, J X + JX. Let ˆX X/ JX, then Substitute X with ˆX in, which is just Costa s EPI b. J ˆX J X JX, and J ˆX. J X, JX C. The convexit o JW t In this section, we stud the convexit o JW t via the relations among the convexities o JW t, JY t, and log JY t. Claim. JW t is not convex. B some algebra, log JY t is convex in t i and onl i J Y t JY t J Y t. JY is convex in t i and onl i t J Y t JY t J Y t, 9 and concave i and onl i J Y t JY t J Y t. 0

17 The irst and second order derivatives o JW t are t JW t t t JX + /t Z t JX + /t Z t J X + /t Z and t JW t t JX + /t Z + t J X + /t Z + t J X + /t Z + t J X + /t Z t JX + /t Z + t J X + /t Z + t J X + /t Z. I we can show that holds or an s > 0, then holds and J X + szjx + sz J X + sz t JW t t JX + /t Z + t J X + /t Z + t J X + /t Z t J t J + t J t J + t J 0. Conversel, i t JW t 0 holds, we can show that also holds. In, let t, we obtain that Substitute X b X ax, where a > 0, JX + J X + J X 0. JX a + J X a Note that J 0 and J 0. Choose proper a such that Hence Thereore, becomes JX a JX a J X a JX a JX a J X a + J X a + J X a 0. J X a. a JXJ X. + J X a a JXJ X + J X a 0,

18 which is JXJ X J X. Hence, t JW t 0 i and onl i holds or arbitrar X. Because JY t is neither convex nor concave or arbitrar X, which means neither 9 nor 0 holds alwas, thus JW t is neither convex nor concave. IX. FURTHER DISCUSSION Ater the third and ourth derivatives are obtained, we consult the literature to ind more connections and implications. The irst inding is that in the literature o mathematical phsics, Conjecture was studied in a 9 paper [, Sec. ] b McKean, who studied the signs o the third and ourth derivatives but ailed to prove them. B this means, our results provide an airmative answer to McKean s problem up to the ourth order. Furthermore, ollowing the routine rules obtained in our paper, one ma tr to veri the conjecture up to an inite order. McKean s work has man other conjectures regarding thermodnamics and has remained unknown to inormation theor communit until ver recentl. For more details on McKean s work, one ma reer to Villani [, pp. -]. Another inding is that Conjecture can be discussed in the context o completel monotone unctions Widder []. Deinition. A unction t, t 0, is completel monotone, i or all n 0,,,..., Hence, Conjecture can be restated as: n dn t 0. dtn Conjecture Completel Monotone Conjecture. JY t is completel monotone in t 0,. A ver interesting result on completel monotone unctions is due to Fink []: I t is completel monotone in t, then t is log-convex. B this means, Conjecture can impl Conjecture. Another result on completel monotone unctions is the ollowing theorem Widder [, p. 0]. Theorem Bernstein s theorem. A necessar and suicient condition that t should be completel monotone in [0, is that t e tx dαx, where αx is bounded and non-decreasing and the above integral converges or 0 t <. 0 That is, i Conjecture is true, an equivalent expression or Fisher inormation will be obtained. Noting that αx can be regarded as a measure deined on [0,. In this paper, to simpli the problem, we consider onl the univariate case o random variables. For the multivariate case, the computation will be much more involved. Some sophisticated techniques that have been developed in probabilit theor ma be useul; e.g., the Γ calculus, which can be ound in Villani [] and Bakr et al. []. X. CONCLUSION The Gaussian random variables have man ascinating properties. In this paper, we have obtained the third and the ourth order derivatives o hx + tz. The signed representations have a ver interesting orm. We wish to show that, though we cannot obtain a closed-orm expression on hx + tz when X is arbitrar, we can still obtain its convexit or concavit or an order derivative. Our progress veriies a small part o the conjectures and has nearl exhausted the power o undamental calculus. A new approach ma be needed towards solving these conjectures. In the literature, the approach that emploed heat equation and integration b parts is merel one o man dierent approaches to prove Costa s EPI. For the approaches like data processing argument in [] [], and the advanced tools in [], it is unknown whether the can go urther than what we have done. However, i these conjectures are correct, a rather undamental act about the Gaussian random variable will be revealed in the language o dierential entrop. APPENDIX A. Proo to Proposition The technique used in this proo is essentiall the same as that b Costa. One ma reer to [] or more details. Proo: One can obtain the ormulae or the derivatives as n, t gx e x t H n xdx, πt

19 9 where H 0 and H n satisies the recursion ormula In general H n can be expressed as H n x x H n + t H n. H n x n α n,j x nj where α n,j s are some constants that also depend on t and actuall these constants are zeroes or odd j. Notice that n gx e x t H n x πt, t dx E [H n Y t X Y t ] n α n,j E [ Y t X nj Y t ]. j0 Let α n : l α n,l. We prove Proposition b induction on r. When r, [ n k n k ] d E n E α n,j E [ Y t X nj Y t ] k j0 αne k n α n,j E [ Y t X nj Y t ] k α n α k n α k n α k n α k n < +, n j0 n j0 n j0 n j0 j0 j0 α n,j [ E [ E Yt X nj Y t ] k] α n α n,j [ [ ]] E E Y t X knj Y t α n α n,j E [ Y t X knj] α n [ α n,j ] knj E tz α n where and are due to Jensen s inequalit. When r, b induction, [ r [ mi ] r ki ki d E i i [ r E mi k i ] [ E < +, i mr mi k i ] k r ] where is b the Cauch-Schwartz inequalit. The act that r i vanishes as can be obtained rom the existence o integral. [ m i ] k i k i

20 0 B. Proo to Costa s EPI Proo: Costa s EPI is equivalent to B some algebra, one needs to show or i.e., t ehyt 0. 9 t hy t t hy t, 0 JY t J Y t, d d, which can be proved b the inequalit o arithmetic and geometric means: d d d d d. In, d t d t C. Proo to Lemma d t 0. Proo: We use integration b parts to eliminate the high-order terms: d d d d d. d d + 0 d d + d + d.

21 d d d + d 0 d + d. d d + d 0 d + d + d. In the above, the limits are zero due to Proposition. Since all the integrals and limits exist, all the steps which use integration b parts are valid. D. Proo to Lemma Proo: We use integration b parts to eliminate the high-order terms: d d d d d d d + d 0 d + d + 0 d

22 d d d + 0 d d + d d d d + d 0 d + d 9 d d + d 0 d + d d + 0 d 0

23 d d + d 0 + d + d d + d d d + d 0 d + d d + d d d d + d 0 d + d

24 d d + d 0 d + d + d ACKNOWLEDGMENT The authors would like to thank the Associate Editor and two anonmous reviewers or their comments and suggestions; especiall the second reviewer or providing the reerences on McKean s Problem and Γ calculus. F. Cheng would like to express his gratitude to Pro. Ramond Yeung or introducing him the topic on EPI when he was a student at CUHK and the support when he was working at the Institute o Network Coding. He would like to thank Pro. Chandra Nair or teaching him the proo in Costa s paper and the help in preparing the manuscript. He also would like to thank Pro. Venkat Anantharam or his valuable discussion and suggestion, which have greatl improved the paper. He is grateul to Pro. Cédric Villani or sharing his expertise as well as the critical comments and providing the reerences in mathematical phsics. The help rom Pro. Amir Dembo and Pro. Andrew Barron is sincerel appreciated. REFERENCES [] C. E. Shannon, A mathematical theor o communication, Bell Sstem Tech. J., vol., : pp. 9-, : pp. -, 9. [] A. Stam, Some inequalities satisied b the quantities o inormation o Fisher and Shannon, Inormation and Control, vol., pp. 0-, 99. [] N. Blachman, The convolution inequalit or entrop powers, IEEE Trans. Inorm. Theor, vol., no., pp. -, Apr. 9. [] R. Zamir, A Proo o the Fisher Inormation Inequalit via a Data Processing Argument, IEEE Trans. Inorm. Theor, vol., no., pp. -0, Ma 99. [] E. H. Lieb, Proo o an entrop conjecture o Wehrl, Comm. Math. Phs., vol., no., pp. -, 9. [] S. Verdú and D. Guo, A simple proo o the entrop-power inequalit, IEEE Trans. Inorm. Theor, vol., no., pp. -, Ma 00. [] O. Rioul, Inormation theoretic proos o entrop power inequalities, IEEE Trans. Inorm. Theor, vol., no., pp. -, 0. [] M. H. M. Costa, A new entrop power inequalit, IEEE Trans. Inorm. Theor, vol., no., pp. -0, Nov. 9. [9] A. Dembo, Simple proo o the concavit o the entrop power with respect to added Gaussian noise, IEEE Trans. Inorm. Theor, vol., no., pp. -, Jul. 99. [0] R. Zamir and M. Feder, A Generalization o the Entrop Power Inequalit with Applications, IEEE Trans. Inorm. Theor, vol. 9, no., pp. -, Sep. 99. [] T. Liu and P. Viswanath, An extremal inequalit motivated b multiterminal inormation theoretic problems, IEEE Trans. Inorm. Theor, vol., no., pp. 9-, Ma 00. [] L. Wang and M. Madiman, A New Approach to the Entrop Power Inequalit, via Rearrangements, IEEE International Smposium on Inormation Theor, 0. [] A. El Gamal and Y.-H. Kim, Network Inormation Theor, Cambridge Univ. Press, 0. [] A. Dembo, T. M. Cover, and J. A. Thomas, Inormation Theoretic Inequalities, IEEE Trans. Inorm. Theor, vol., no., pp. 0-, Ma 99. [] T. M. Cover and J. A. Thomas, Elements o Inormation Theor, Second Edition, Wile. [] C. Villani, A short proo o the concavit o entrop power, IEEE Trans. Inorm. Theor, vol., no., pp. 9-9, 000. [] Y. Wu and S. Verdú, MMSE dimension, IEEE Trans. Inorm. Theor, vol., no., pp. -9, Aug. 0. [] D. Guo, S. Shamai Shitz, and S. Verdú, Estimation in Gaussian noise: properties o the minimum mean-square error, IEEE Trans. Inorm. Theor, vol., no., pp. -, Apr. 0. [9] A. R. Barron, Entrop and the central limit theorem, Annals o Probabilit, vol., pp. -, 9. [0] F. Cheng and Y. Geng, Convexit o Fisher Inormation with Respect to Gaussian Perturbation, IEEE Iran Workshop on Communication and Inormation Theor, 0. [] H. P. McKean, Jr., Speed o approach to equilibrium or Kac s caricature o a Maxwellian gas, Arch. Rational Mech. Anal., :-, 9. [] C. Villani, A review o mathematical topics in collisional kinetic theor, Handbook o Mathematical Fluid Dnamics Vol., Elsevier Science, 00. [] D. V. Widder, The Laplace Transorm, Princeton Universit Press, 9. [] A. M. Fink, Kolmogorov-Landau inequalities or monotone unctions, J. Math. Anal. Appl. 90 9, -. [] D. Bakr, I. Gentil, and M. Ledoux, Analsis and geometr o Markov diusion operators, Volume o Grundlehren der Mathematischen Wissenschaten [Fundamental Principles o Mathematical Sciences]. Springer, Cham, 0. Fan Cheng S -M received the bachelor degree in computer science and engineering rom Shanghai Jiao Tong Universit in 00, and the PhD degree in inormation engineering rom The Chinese Universit o Hong Kong in 0. From 0-0, he had been a postdoctoral ellow in the Institute o Network Coding, The Chinese Universit o Hong Kong. As o 0, he has been a research ellow in the Department o ECE, NUS, Singapore.

25 Yanlin Geng M received his B.Sc. mathematics and M.Eng. signal and inormation processing rom Peking Universit, and Ph.D. inormation engineering rom the Chinese Universit o Hong Kong in 00, 009, and 0, respectivel. He is currentl an Assistant Proessor in the School o Inormation Science and Technolog, ShanghaiTech Universit. His research interests are mainl on problems in network inormation theor.