Logarithmic Sobolev inequalities in discrete product spaces: proof by a transportation cost distance

Transcription

1 Logarithmic Sobolev inequalities in discrete product spaces: proof by a transportation cost distance Katalin Marton Alfréd Rényi Institute of Mathematics of the Hungarian Academy of Sciences

2 Relative entropy Definition Z: measurable space, µ, ν: probability measures on Z. Z, U: random variables, L(Z) = µ, L(U) = ν. Relative entropy: D(µ ν) = D(Z U) = Z log dµ dµ (1) dν If Z is finite: D(µ ν) = D(Z U) = z Z µ(z) log µ(z) ν(z). (2)

3 Entropy contraction of Markov kernels Definition (Z, ν) : probability space, P(Z): measures on Z, Γ: Markov kernel on Z with invariant measure ν. Entropy contraction for (Z, ν, Γ) with rate 1 c, 0 < c 1: D(µΓ ν) (1 c) D(µ ν). (3) Equivalently: c D(µ ν) ( D(µ ν) D(µΓ ν) ), for all µ P(Z) (4)

4 Gibbs sampler governed by local specifications of q n Definition Γ i : Markov kernel X n X n Γ: Markov kernel X n X n : Definition Γ i (z n y n ) = δ(ȳ i, z i ) q i (z i ȳ i ), Γ = 1 n n Γ i. q n has the entropy contraction property if: its Gibbs sampler Γ has.

5 Entropy contraction in product space (X n, q n ): probability space Question Which measures q n have the entropy contraction property with a reasonable constant c? c cannot be larger than O(1/n). Changing notation: WHEN c n D(pn q n ) ( D(p n q n ) D(p n Γ q n ) ) for all p n P(X n )? (5) Equivalently: WHEN D(p n q n ) (1 c n ) D(pn Γ q n )? (6)

6 Conditional relative entropy Definition Z: measurable space, V: another measurable space, π: probability measure on V, V : random variable, L(V ) = π For v V probability measures on Z: µ( v) = L(Z V = v) ν( v) = L(U V = v) Conditional relative entropy: D ( µ( V ) ν( V ) ) = D ( Z V ) U V ) ) D ( µ( v) ν( v) ) dπ V (7)

7 Cont d If V is finite: D ( µ( V ) ν( V ) = D ( Z V ) U V ) ) = v V π(v)d ( µ( v) ν( v) ). (8)

8 Sufficient condition for entropy contraction in product space (X n, q n, Γ): probability space with Gibbs sampler, q n = L(X n ) p n = L(Y n ): another distribution on X n Recall: Γ = 1 n n Γ i Ȳ i = (Y 1,..., Y i 1, Y i+1,..., Y n ), Proposition c D(p n q n ) = n D ( ) p i ( Ȳi) q i ( Ȳi D(p n Γ q n ) ( 1 c n) D(p n q n ). If q n is a product measure then: c = 1. ( ) (9)

9 Notation For x n = (x 1, x 2,..., x n ) X n : x i (x 1,..., x i 1, x i+1,..., x n ) p i L(Ȳi), q i L( X i ) Expansion formula D(p n q n ) = D( p i q i ) + D ( p i ( Ȳi) q i ( Ȳi) ) = D(p n q n ) = 1 n n D( p i q i ) + 1 n n D ( p i ( Ȳi) q i ( Ȳi) )

10 Proof of Proposition D(p n q n ) D(p n Γ q n ) (by convexity of entropy) D(p n q n ) 1 n n D( p i q i ) = 1 n n D ( p i ( Ȳi q i ( Ȳi) ) (by assumption) c n D(pn q n )

11 Entropy contraction in DISCRETE product spaces X finite (X n, q n, Γ) Wanted: inequality n D(p n q n ) C D ( p i ( Ȳi q i ( Ȳi) ) for all p n P(X n ) ( ) To get (*): use a Wassersteine-like distance.

12 A reverse Pinsker s inequality µ, ν: probability measures on X X finite Notation: Variational distance µ ν = 1 2 Lemma Set Then x X µ(x) ν(x) X + {x X : ν(x) > 0}, α min { ν(x) : x X + } D(µ ν) 4 µ ν 2 α Follows from the inequality D(µ ν) x X µ(x) ν(x) 2 ν(x)

13 A distance between measures on product spaces X n : product space µ n = L(Z n ), ν n = L(U n ): probability measures on X n Definition (P. Massart) The square of the W 2 -distance of µ n and ν n : W2 2 (µ n, ν n ) min inf: on couplings of µ n and ν n. n P r 2{ } Z i U i

14 Notation For I [1, n], p n = L(Y n ) and y n X n : y I (y i : i I), ȳ I (y i : i / I) p I L(Y I ), p I ( ȳ I ) L(Y I ȲI = ȳ I ) For Theorem 1 we need the inequality W2 2 ( n p n q n ) C E p i ( Ȳi) q i ( Ȳi) 2. in a MORE GENERAL FORM: We require a bound for W 2 2 ( pi ( ȳ I ), q I ( ȳ I ) ) for ALL subsets I [1, n] (not just I = [1, n]), and all ȳ I.

15 Main Theorem (X n, q n ), X finite! Theorem 1 Set α = min { q i (x i x i ) : q n (x n ) > 0, 1 i n } (10) Fix a p n = L(Y n ) P(X n ); assume q n (x n ) = 0 = p n (x n ) = 0. (11) Main assumption: ( ) )) pi ( ȳi, qi ( ȳi W 2 2 { C E p i ( Ȳi) q i ( Ȳi) 2 Ȳ I = ȳ I }, i I (12) for all I [1, n], ȳ I X [1,n]\I.

16 Main Theorem continued Assume all the inequalities ( ) )) pi ( ȳi, qi ( ȳi W 2 2 { C E p i ( Ȳi) q i ( Ȳi) 2 Ȳ I = ȳ I }, i I (13) where I [1, n] and ȳ I X [1,n]\I is fixed. Then D(p n q n ) 4C n α E pi ( Ȳi) q i ( Ȳi) 2 2C α n D ( p i ( Ȳi) q i ( Ȳi) ). (14)

17 An analogous result for densities in R n Theorem f(x n ) = exp( V (x n ) : density on R n, q n : probability measure with density f. Assume: conditional densities a f(x i x i ) satisfy a logarithmic Sobolev inequality with constant ρ, for all i, x i. Under some conditions on 1 ρ Hess V (xn ) (expressing that V is not too far from being uniformly convex): D(p n q n ) C (C = C(q n )) n D ( p i ( Ȳi) q i ( Ȳi) ) for all p n

18 Proof of Theorem 1 By induction on n. Assume Theorem 1 for n 1 Notation p i ( y i ) L(Ȳi Y i = y i ) For every i [1, n] D(p n q n ) = D(Y n X n ) = D(Y i X i ) + D ( p i ( Y i ) q i ( Y i ) ) = D(p n q n ) = 1 n n D(Y i X i ) + 1 n n D ( p i ( Y i ) q i ( Ȳi) ) By the induction hypothesis the second term is (15) ( 1 1 n ) 4C α n E p i ( Ȳi) q i ( Ȳi) 2

19 Proof of Theorem 1 Cont d Second term: First term: 1 n ( 1 1 n n D(Y i X i ) ) 4C α n E p i ( Ȳi) q i ( Ȳi) 2 (by the Lemma) n P r 2{ } Y i X i in any coupling of p n, q n = 1 n 4 α W 2 2 (p n, q n ) for the best coupling (by the assumption of Theorem 1) 1 n 4C n α pi ( Ȳi) q i ( Ȳi) 2 1 n 4 α L(Y i ) L(X i ) 2 (16)

20 Entropy contraction (X n, q n, Γ), X finite Γ: Gibbs sampler Corollary 1 If q n satisfies the conditions of Theorem 1 then ( D(p n Γ q n ) 1 α ) D(p n q n ). (17) 2nC

21 Logarithmic Sobolev inequality Notation E: Dirichlet form associated with Γ is the quadratic form E(f, g) = (Id Γ)f, g q n Definition q n satisfies a logarithmic Sobolev inequality with constant c > 0 if: ( ) p c D(p n q n n p ) E q n, n q n for all p n P(X n )

22 Logarithmic Sobolev inequality for Gibbs sampler (X n, q n, Γ), X finite Corollary 2 Under conditions of Theorem 1 the logarithmic Sobolev inequality holds true: 1 n D(pn q n ) 4C ( ) p α E n p Γ q n, n q n = 4C n ( ( αn ( ( ) E 1 p i yi Ȳi) ) 2) qi yi Ȳi. y i X (18) = hypercontractivity

23 ***********************************?????????????????

24 Application: Gibbs measures with Dobrushin s uniqueness condition (X n, q n ), X finite Definition q n satisfies (an L 2 -version of) Dobrushin s uniqueness condition with coupling matrix if: A = ( a k,i ) n k,, a i,i = 0, (i) max q i ( z i ) q i ( s i ) a k,i, k i, max : for all z i, s i differing only in the k-th coordinate, (19) and (ii) A 2 < 1.

25 (X n, q n ), X finite Theorem 2 Assume Dobrushin s uniqueness condition with coupling matrix A, A 2 < 1. Then conditions of Theorem 1 hold with C = 1/ ( 1 A ) 2. Thus D(p n q n ) 4 α 2 α 1 ( 1 A ) 2 1 ( 1 A ) 2 n E p i ( Ȳi) q i ( Ȳi) 2 n D ( p i ( Ȳi) q i ( Ȳi) ), (20)

26 Cont d Proof Dobrushin s uniqueness condition implies that Γ contracts W 2 -distance with rate 1 1 ) (1 n A 2.

27 Application: Gibbs measures on Z d Notation Z d : d-dimensional lattice, i Z d : site ρ(k, i) = max 1 ν d k ν i ν : distance on Z d, Λ Z d : finite set of sites X finite: spin space x Zd = (x i : i Z d ) X Zd : spin configuration X Zd : configuration space, For x Zd and Λ Z d x Λ = (x i : i Λ), x Λ = (x i : i / Λ), x Λ is called an outside configuration for Λ.

28 Local specificatons Definition q Λ ( x Λ ), Λ Z d : conditional distributions on X Λ. Assume compatibility conditions. There exists at least one probability measure q on the space of configurations: q = L(X) P(X Zd ) satisfying L(X Λ X Λ = x Λ ) = q Λ ( x Λ ), all Λ Z d. q Λ ( x Λ ): local specifications of q.

29 Finite range interactions Definition The local specifications have finite range of interactions if: there is an R > 0: q Λ ( x Λ ) only depends on coordinates k / Λ with ρ(k, Λ) R. q may not be uniquely defined by the local specifications, even for finite range interactions.

30 Dobrushin-Shlosman s strong mixing condition Given local specifications q Λ ( x Λ ). Assumption There exists a function ϕ(ρ) of the distance such that: (i) ( ) ρ(k, i) <, and: (ii) for every i Z d ϕ Λ Z d, M Λ, k / Λ and every ȳ Λ, z Λ, differing only at k : qm ( ȳ Λ ) q M ( z Λ ) ( ) ϕ ρ(k, M).

31 In case of finite range interactions: If Dobrushin-Shlosman s strong mixing condition holds then ϕ(ρ) = C exp( γ ρ) can be taken

32 ρ(k, M) = max length of red segments q M ( y Λ ) q M ( z Λ ) ϕ(ρ(k, M)) y Λ, z Λ differ only at k Λ

33 Dobrushin-Shlosman s strong mixing condition Cont d Meaning: The influence of the spin at k / Λ on the spins in M Λ that are far away from k is small. Essential: The spins over Λ are fixed in two different ways. The spins over Λ \ M are not fixed.

34 Logarithmic Sobolev inequality for strongly mixing measures Earlier results for the case of finite range interactions: D. Stroock, B. Zegarlinski 1992, F. Cesi 2001 F. Martinelli, E. Olivieri Theorem 3 (X Λ, q Λ ( ȳ Λ )) for fixed Λ and ȳ Λ α = min { q i (x i x i ) : q i (x i x i ) > 0 }. (Finite range is not assumed.) { Dobrushin-Shlosman s strong mixing condition + {α > 0} } = conditions of Theorem 1 for q Λ ( ȳ Λ ), with uniform constant = logarithmic Sobolev inequality for q Λ ( ȳ Λ ) with uniform constant

35 Logarithmic Sobolev inequality for strongly mixing measures Cont d The proof uses a Gibbs sampler updating cubes of size m depending on the dimension and on the function ϕ(ρ). We get W 2 2 ( pλ, q Λ ( ȳ Λ ) ) C m I:m-sided cube E p I Λ ( ȲI Λ) q I Λ ( ȲI Λ) 2 C m,α E ) ) p i ( YΛ\i qi ( YΛ\i, ȳ Λ 2 i Λ for an appropriate m that is good enough for any Λ and ȳ Λ.