EQUIVALENT FORMULATIONS OF HYPERCONTRACTIVITY USING INFORMATION MEASURES

Transcription

1 1 EQUIVALENT FORMULATIONS OF HYPERCONTRACTIVITY USING INFORMATION MEASURES CHANDRA NAIR 1 1 Dept. of Information Engineering, The Chinese Universit of Hong Kong Abstract We derive alternate characterizations for the hpercontractive region of a pair of random variables using information measures. 1. Introduction A pair of random variables (X, Y) defined on some probabilit space (Ω, F, µ), is said to be (p, q)-hpercontractive for 1 q p < if the inequalit E[g(Y) X] p g(y) q holds for ever bounded measurable function g(y). The following is another well-known equivalent definition (equivalence being a direct application of Hölder s inequalit). A pair of random variables (X, Y) defined on some probabilit space (Ω, F, µ), is said to be (p, q)- hpercontractive for 1 q p < if the inequalit E[ f (X)g(Y)] f (X) p g(y) q holds for ever pair of bounded measurable functions f (X), g(y). Here p = of p. For an p 1 one can define q p (X; Y) = inf{q : (X, Y) is (p, q)-hpercontractive}. p p 1 denotes the Hölder conjugate Define the ratio r p (X; Y) = q p(x;y) p. Hpercontractive inequalities have found a variet of applications in quantum phsics [3], theoretical computer science [4], analsis [6], and in information theor [1, 2]. In this talk we present the following alternate characterizations of r p (X; Y) using information measures. A ver useful propert of the hpercontractive parameter r p (X; Y) is the so-called tensorization propert which states: if {(X i, Y i )} n i=1 are independent random variables, then r p (X n ; Y n ) = n ma i=1 r p(x i ; Y i ). For the purpose of this manuscript, let us assume that the random variables X, Y take values in a finite alphabet space X Y. Thus we can talk about a probabilit mass function µ XY (, ) and the induced marginal distributions µ X (), µ Y (). We assume, without loss of generalit, that µ X () > 0, X and µ Y () > 0, Y. We will omit the subscripts on the measures when there is no ambuguit. Corresponding author: Chandra Nair, chandra.nair@gmail.com c The Author(s) 2014.

2 C. Nair 2 Consider two measures ν and µ on a space, sa X. If the measure ν is absolutel continuous with respect to the measure µ, then we denote it as ν µ. Let D(ν() µ()) = ν() X ν() log 2 µ() be the relative entrop between the two measures ν() and µ(), when ν µ. For p 1, define k p (X; Y) := ν(x,y) µ(x,y) ν(x,y) µ(x,y) D(ν() µ()) pd(ν(, ) D(µ(, )) (p 1)D(ν() D(µ()). Let P µ = {ν UXY (u,, ) : u U, U <, u U ν(u,, ) = µ(, ) X, Y} be the collection of measures induced b random variables (U, X, Y) such that the marginal law of (X, Y) is consistent with µ(x, Y). Here U is a random variable taking finitel man values and is often referred to as an auiliar random variable in multiuser information theor. Later we will see that we can restrict the size of U to X Y. For p 1 define u p (X; Y) = Theorem 1. The following equivalence holds: Further the common value is also equal to pi(u; XY) (p 1)I(U; X). r p (X; Y) = k p (X; Y) = u p (X; Y). inf{λ : H(Y) λph(x, Y) + λ(p 1)H(X) = K[H(Y) λph(x, Y) + λ(p 1)H(X)] µxy }, where K[ f ] represents the lower conve envelope of the function f evaluated at. Remark: The above result generalizes the equivalence results in both [1] and [2] which deal with the limiting case p. 2. Proof of the main result The equalit in Theorem 1 will be established b showing a sequence of inequalities. In particular, we will show that (i) r p (X; Y) k p (X; Y) (ii) k p (X; Y) u p (X; Y) (iii) u p (X; Y) r p (X; Y). Proof of r p (X; Y) k p (X; Y): satisf Given p > 1, and ɛ > 0 arbitrar, let non-negative functions f (X) and g(y) W.l.o.g. assume that f (X) p = g(y) (rp ɛ)p = 1. Define f () p = h() and g() (rp ɛ)p = j(). We have µ()h() = µ() j() = 1. E( f (X)g(Y)) > f (X) p g(y) (rp ɛ)p. (1) Since E( f (X)g(Y)) > 1, let C < 1 be such that Cµ(, )h() 1 p 1 (rp ɛ)p j() = 1. Define One easil verifies that ν XY µ XY and ν XY µ XY., ν(, ) := Cµ(, )h() 1 p j() 1 (rp ɛ)p.

3 manuscript - talk at IZS, Now observe that pd(ν(, ) µ(, )) (p 1)D(ν() µ()) = p log C + p 1 p ν() log h() + ν() log j() (p 1) ν() log ν() (r p ɛ) µ() = p log C + (p 1) ν() log µ()h() 1 µ() j() + ν() log ν() (r p ɛ) ν() 1 + (r p ɛ) 1 (r p ɛ) ν() log ν() µ() ν() log ν() µ() = 1 (r p ɛ) D(ν() µ()). Here the last inequalit follows since C < 1, µ()h() = 1, µ() j() = 1. Since ɛ > 0 is arbitrar we are done. Proof of k p (X; Y) u p (X; Y): The argument below is identical to the argument in [2] which in turn is motivated b similar arguments in [5]. Let δ (0, k p (X; Y)) be arbitrar. Let ν XY µ XY, ν XY µ XY be an distribution satisfing D(ν() µ()) pd(ν(, ) D(µ(, )) (p 1)D(ν() D(µ()) > k p(x; Y) δ. Let U ɛ := {1, 2}. Fi a sufficientl small 1 ɛ > 0 and define a triple (U ɛ, X, Y) according to: P(U ɛ = 1) = ɛ; Conditional distribution of (X, Y) (U ɛ = 1) = ν XY, P(U ɛ = 2) = 1 ɛ; Conditional distribution of (X, Y) (U ɛ = 2) = µ XY + ɛ 1 ɛ (µ XY ν XY ) = 1 1 ɛ µ XY ɛ 1 ɛ ν XY. Note that for an ɛ > 0 the distribution of (U ɛ, X, Y) belongs to P µ. For an 0 < λ < k p (X; Y) δ define the function g(ɛ) := I(U ɛ ; Y) λ(pi(u ɛ ; X, Y) (p 1)I(U ɛ ; X)). Elementar calculations ield that dg(ɛ) = D ( ν() µ() ) λ(pd ( ν(, ) µ(, ) ) (p 1)D ( ν() µ() ) > 0, dɛ ɛ 0 where the last inequalit is because of the choice of ν(x, Y) and as 0 < λ < k p (X; Y) δ. Since g(0) = 0 this implies that for some ɛ > 0 we have I(U ɛ; Y) λ(pi(u ɛ; X, Y) (p 1)I(U ɛ; X)) > 0. Note also that I(U ɛ; X, Y) > 0 since ν XY µ XY. This implies that pi(u; XY) (p 1)I(U; X) I(U ɛ ; Y) pi(u ɛ; X, Y) (p 1)I(U ɛ; X) > λ. Since the above holds for all λ < k p (X; Y) δ we have u p (X; Y) = Finall, since δ > 0 is arbitrar, we are done. pi(u; XY) (p 1)I(U; X) k p(x; Y) δ. 1 For instance ɛ < min (,):ν(,)>0 µ(,) ν(,) and ɛ > 1 ma (,) µ(, ) suffices. Observe that such an ɛ eists if X and Y are not constants.

4 C. Nair 4 Proof of u p (X; Y) r p (X; Y): This proof uses the tensorization propert of r p (X; Y) as well as the notion of tpical sequences often emploed in multi-terminal information theor. Consider an (U, X, Y) ν UXY P µ. Consider (U n, X n, Y n ) distributed according to i ν UXY (u i, i, i ), i.e. the components are independent and identicall distributed. Pick a single u n such that { {i : u i = u} nν U (u) U }. For instance, let U = {1, 2,..., m} and set the first nν U (1) entries of u n to be 1, then net nν U (2) entries of u n to be 2, and so on. At the end one would have at least m 1 m 1 n nν U (i) n (m 1) n ν U (i) = nν U (m) (m 1) i=1 entries taking the final value m. Define two sets according to and A = { n : {i : (ui, i ) = (u, )} nν UX (u, ) n log(n)νux (u, ) for all (u, )} B = { n : {i : (ui, i ) = (u, )} nν UY (u, ) n log(n)νuy (u, ) for all (u, )}. In the language used in network information theor, these are the sets of sequences n and n respectivel that are jointl tpical with the u n sequence chosen earlier. Note that for an set A and B we have (Lemma 1 in [1]) i=1 P(X n A, Y n B) = E(1 A E(1 B X n )) P(A) 1 1 p E(1B X n ) p P(A) 1 1 p P(B) 1 rp p. (2) The first inequalit follows from Hölder and the second one b the definition and tensorization propert of r p (X; Y) (which implies r p (X n ; Y n ) = r p (X; Y) when {X i, Y i } are i.i.d. according to (X, Y)). It is known (b a simple counting argument) that 1 n log 2 P(A) I(U; X) and 1 n log 2 P(B) as n. Define C = {( n, n ) : {i : (ui, i, i ) = (u,, )} nν UXY (u,, ) n log(n)νuxy (u,, ) for all (u,, )}. Clearl if ( n, n ) C then n A and n B. Thus P(C) P(X n A, Y n B). A counting argument again shows that 1 n log 2 P(C) I(U; XY). Since we have P(C) P(X n A, Y n B) P(A) p P(B) rp p b taking logarithms and dividing b n and letting n go to infinit, we obtain that ( I(U; XY) 1 1 ) I(U; X) 1. p r p p This implies that r p (pi(u; XY) (p 1)I(U; X)) for ever ν UXY P µ. When I(U; XY) > 0 we see that pi(u; XY) (p 1)I(U; X) > 0 impling r p pi(u; XY) (p 1)I(U; X) = u p(x; Y), as desired. The remaining part of the proof is to show that the common value is also given b inf{λ : H(Y) λph(x, Y) + λ(p 1)H(X) = K[H(Y) λph(x, Y) + λ(p 1)H(X)] µxy }. (3)

5 manuscript - talk at IZS, It is an eas eercise to observe that K[H(Y) λph(x, Y) + λ(p 1)H(X)] µxy = inf H(Y U) λph(x, Y U) + λ(p 1)H(X U). Thus if the equalit H(Y) λph(x, Y) + λ(p 1)H(X) = K[H(Y) λph(x, Y) + λ(p 1)H(X)] µxy holds for some λ then for ever ν UXY P µ Rearrangement ields impling H(Y U) λph(x, Y U) + λ(p 1)H(X U) H(Y) λph(x, Y) + λ(p 1)H(X). λ λ(pi(u; XY) (p 1)I(U; X)), pi(u; XY) (p 1)I(U; X) = u p(x; Y). Let λ p (X; Y) denote the infimum of λ satisfing (3). Then we have λ p (X; Y) u p (X; Y). On the other hand, for an ɛ > 0 there must eist a ν UXY P µ such that Re-arrangement ields H(Y U) (λ p (X; Y) ɛ)ph(x, Y U) + (λ p (X; Y) ɛ)(p 1)H(X U) For such a ν UXY clearl I(U; XY) > 0, hence < H(Y) (λ p (X; Y) ɛ)ph(x, Y) + (λ p (X; Y) ɛ)(p 1)H(X). (λ p (X; Y) ɛ)(pi(u; XY) (p 1)I(U; X)) <. λ p (X; Y) ɛ < pi(u; XY) (p 1)I(U; X) u p(x; Y). Taking ɛ 0 ields the desired equalit that λ p (X; Y) u p (X; Y) completing the proof of the equivalence. The last part of this section is to show that in the above calculations one can restrict to random variables U that take at most X Y distinct values. (These are also standard arguments in network information theor.) For an distribution ν XY on X Y define the function f (ν) = H(Y) λph(x, Y) + λ(p 1)H(X) where the entropies are evaluated at the distribution ν XY. Observe that (b Caratheodor-Fenchel-Bunt theorem) the lower conve envelope of f (ν) and the distribution µ XY denoted earlier as K[H(Y) λph(x, Y) + λ(p 1)H(X)] µxy can be computed as a conve combination of at most X Y distributions ν (i) XY, i = 1,.., X Y. Consider ν(i) XY to be the conditional distribution of (X, Y) when U = i and thus observe that we have K[H(Y) λph(x, Y) + λ(p 1)H(X)] µxy = inf H(Y U) λph(x, Y U) + λ(p 1)H(X U). U X Y Remark 1. This remark is for those unfamiliar with the cardinalit bounding arguments in network information theor. To appl Caratheodor-Fenchel-Bunt theorem one considers the continuous mapping from ν XY to S R X Y where the first X Y 1 co-ordinates represent the values of ν XY (i, j) (ecept the entr i = X, j = Y, whose value is forced b the remaining entries since ν XY is a probabilit vector) and the last co-ordinate is the value f (ν). One is interested in obtaining a particular point in the conve hull of S using as few points from S as possible. Since S is connected it suffices to use X Y distinct points. This improvement over Caratheodor is due to Fenchel and later generalized b Bunt.

6 C. Nair 6 Historical background and Acknowledgements This work is directl motivated b the work of the author with Venkat Anantharam, Amin Gohari, and Sudeep Kamath in [2] which deals with the equivalent version of Theorem 1 when p. Tring to generalize this result to finite p, the author had obtained that both u p (X; Y) and k p (X; Y) were lower bounds to r p (X; Y). Numerical eperiments conducted b Sudeep Kamath suggested that these bounds were tight, prompting the author to complete the proof of the equivalence. The author wishes to thank Venkat Anantharam, Amin Gohari, and Sudeep Kamath for various stimulating discussions relating to the connections between hpercontractivit parameters and information measures. This work was ported b an AoE grant (Project No. AoE/E-02/08) and two GRF grants (Project Nos and ) from the UGC of the Hong Kong Special Administrative Region, China. References [1] Rudolf Ahlswede and Peter Gács, Spreading of sets in product spaces and hpercontraction of the markov operator, The Annals of Probabilit (1976), [2] Venkat Anantharam, Amin Aminzadeh Gohari, Sudeep Kamath, and Chandra Nair, On maimal correlation, hpercontractivit, and the data processing inequalit studied b erkip and cover, CoRR abs/ (2013). [3] E. Brian Davies, Leonard Gross, and Barr Simon, Hpercontractivit: a bibliographic review, Ideas and methods in quantum and statistical phsics (Oslo, 1988), Cambridge Univ. Press, Cambridge, 1992, pp MR (93g:47052) [4] Jeff Kahn, G. Kalai, and Nathan Linial, The influence of variables on boolean functions, Foundations of Computer Science, 1988., 29th Annual Smposium on, 1988, pp [5] J Körner and K Marton, Comparison of two nois channels, Topics in Inform. Theor(ed. b I. Csiszar and P.Elias), Keszthel, Hungar (August, 1975), [6] Michel Talagrand, On russo s approimate zero-one law, The Annals of Probabilit 22 (1994), no. 3, pp (English).