Stein s method, logarithmic Sobolev and transport inequalities

Transcription

1 Stein s method, logarithmic Sobolev and transport inequalities M. Ledoux University of Toulouse, France and Institut Universitaire de France

2 Stein s method, logarithmic Sobolev and transport inequalities joint work with I. Nourdin, G. Peccati (Luxemburg) links between Stein s approximation method functional and transport inequalities

3 Stein s method normal approximation central limit theorems rates of convergence, Edgeworth expansions, strong approximations Poisson approximation (L. Chen), other distributions statistics, machine learning, engineering...

4 logarithmic Sobolev inequalities quantum field theory, hypercontractivity infinite dimensional Wiener analysis convergence to equilibrium of kinetic equations, Markov chains, interacting particle systems measure concentration (entropy method) information theory, optimal transport (C. Villani) Perelman s Ricci flow

5 Plan 1. logarithmic Sobolev inequalities 2. Stein s method 3. links and applications

6 classical logarithmic Sobolev inequality L. Gross (1975) A. Stam (1959), P. Federbush (1969)

7 classical logarithmic Sobolev inequality L. Gross (1975) γ standard Gaussian (probability) measure on R d dγ(x) = e x 2 /2 dx (2π) d/2 h > 0 smooth, R h dγ = 1 d entropy h log h dγ 1 Rd h 2 dγ R d 2 h Fisher information

8 classical logarithmic Sobolev inequality h log h dγ 1 Rd h 2 dγ, R d 2 h R d h dγ = 1 ν << γ dν = h dγ H ( ν γ ) 1 2 I( ν γ ) (relative) H-entropy H ( ν γ ) = R d h log h dγ (relative) Fisher Information I ( ν γ ) = Rd h 2 dγ h

9 logarithmic Sobolev inequalities hypercontractivity (integrability of Wiener chaos) concentration inequalities (entropy method) links with transportation cost inequalities

10 logarithmic Sobolev inequality and concentration Herbst argument (1975) h log h dγ 1 Rd h 2 dγ, R d 2 h R d h dγ = 1 ϕ : R d R 1-Lipschitz, R ϕ dγ = 0 d h = e λϕ R d e λϕ dγ, λ R Z(λ) = R d e λϕ dγ

11 logarithmic Sobolev inequality and concentration Herbst argument (1975) λz (λ) Z(λ) log Z(λ) λ2 2 Z(λ) integrate Z(λ) = e λϕ dγ e λ2 /2 R d ϕ : R d R 1-Lipschitz γ ( ϕ R n ϕ dγ + r ) e r2 /2, r 0 (dimension free) Gaussian concentration

12 Gaussian processes F collection of functions f : S R G(f ), f F centered Gaussian process M = sup G(f ), M Lipschitz f F Gaussian concentration P ( M m r ) 2 e r2 /2σ 2, r 0 m mean or median, σ 2 = sup E ( G(f ) 2) f F Gaussian isoperimetric inequality C. Borell, V. Sudakov, B. Tsirel son, I. Ibragimov (1975)

13 extension to empirical processes M. Talagrand (1996) X 1,..., X n independent in (S, S) F collection of functions f : S R M = sup f F n f (X i ) i=1 M Lipschitz and convex Talagrand s convex distance inequality concentration inequalities on P ( M m r ), r 0

14 convex distance inequality ϕ : R d R 1-Lipschitz convex for any product probability P on [0, 1] d P ( ϕ m r ) 4 e r2 /4, r 0 m median of ϕ for P M. Talagrand (1996) isoperimetric ideas (induction) entropy method Herbst argument S. Boucheron, G. Lugosi, P. Massart (2013)

15 M = sup f F extension to empirical processes n f (X i ) i=1 f 1, E ( f (X i ) ) = 0, f F P ( M m r ) ( C exp rc ( )) log r 1 + σ 2, r 0 + m m mean or median, σ 2 = sup f F n E ( f 2 (X i ) ) i=1 early motivation from Probability in Banach spaces (law of the iterated logarithm)

16 logarithmic Sobolev and transport inequalities Otto-Villani theorem (2000) logarithmic Sobolev inequality for µ (on R d ) H ( ν µ) C 2 I( ν µ ), ν << µ implies quadratic transportation cost inequality W 2 2 (ν, µ) 2C H( ν µ), ν << µ W2 2 (ν, µ) = inf x y 2 dπ(x, y) ν π µ R d R d Kantorovich-Wasserstein distance µ = γ M. Talagrand (1996)

17 Plan 1. logarithmic Sobolev inequalities 2. Stein s method 3. links and applications

18 Stein s method C. Stein (1972)

19 R Stein s method C. Stein (1972) γ standard normal on R x φ dγ = φ dγ, φ : R R smooth R characterizes γ Stein s inequality ν probability measure on R ν γ TV sup φ π/2, φ 2 [ x φ dν R R ] φ dν

20 R Stein s kernel ν (centered) probability measure on R Stein s kernel for ν : x τ ν (x) x φ dν = τ ν φ dν, φ : R R smooth R γ standard normal τ γ = 1 dν = f dx τ ν (x) = [ f (x) ] 1 x y f (y)dy, x supp(f ) (τ ν polynomial: Pearson class) zero/size-biased coupling L. Goldstein, Y. Rinott, G. Reinert ( )

21 R Stein s kernel ν (centered) probability measure on R Stein s kernel for ν : x τ ν (x) x φ dν = τ ν φ dν, φ : R R smooth R γ standard normal τ γ = 1 Stein s discrepancy S(ν γ) S 2( ν γ ) = τ ν 1 2 dν Stein s inequality ν γ TV 2 S ( ν γ ) R

22 Stein s method normal approximation rates of convergence in central limit theorems convergence of Wiener chaos Lindeberg s replacement method, random matrices exchangeable pairs, concentration inequalities

23 Stein s method and central limit theorem central limit theorem X, X 1,..., X n iid real random variables mean zero, variance one S n = 1 n (X X n ) S 2( L(S n ) γ ) 1 n S2( L(X) γ ) = 1 n Var( τ L(X) (X) ) L(S n ) γ 2 TV 4 S2( L(S n ) γ ) ( 1 = O n)

24 Stein s method and Wiener chaos Wiener multiple integrals (chaos) multilinear Gaussian polynomials (degree k) F = X 1,..., X N N a i1,...,i k X i1 X ik i 1,...,i k =1 independent standard normals a i1,...,i k R symmetric, vanishing on diagonals E(F 2 ) = 1

25 Stein s method and Wiener chaos D. Nualart, G. Peccati (2005) fourth moment theorem F = F n, n N k-chaos (fixed degree k) N = N n E(F 2 n) = 1 (or 1) F n converges to a standard normal if and only if E(F 4 n) 3 ( = R ) x 4 dγ

26 Stein s method and Wiener chaos F Wiener functional τ F (x) = E ( DF, D L 1 F F = x ) L Ornstein-Uhlenbeck operator, D Malliavin derivative S 2( L(F) γ ) k 1 3k [ E(F 4 ) 3 ] I. Nourdin, G. Peccati (2009), I. Nourdin, J. Rosinski (2012) multidimensional versions

27 Stein s method and Lindeberg s replacement strategy J. Lindeberg (1922) X = (X 1,..., X n ) iid, mean zero, variance one, third moment φ : R n R smooth, i = 1,..., n E ( φ(x ) ) ( n ) φ dγ = O i 3 φ R n i=1 Taylor expansion E ( X i i φ(x ) ) = E ( 2 i φ(x ) ) + O ( 3 i φ ) E ( X i i φ(x ) ) = E ( τ(x i ) i 2 φ(x )) E ( Lφ(X ) ) Lφ dγ = R n n i=1 E ([ τ(x i ) 1 ] i 2 φ(x ) ) ( n ) = O i 3 φ i=1

28 Stein s method and Lindeberg s replacement strategy S. Chatterjee (2006) comparison between arbitrary samples, strong embeddings universality in random matrix theory T. Tao, V. Vu ( ) four moment theorem L. Erdös, H. T. Yau, A. Knowles, J. Yin ( ) Green s function comparison

29 Stein s method and concentration exchangeable pairs (X, X ) ϕ(x) = E ( F(X, X ) X ) S. Chatterjee (2005) F antisymmetric ( ϕ(x) ϕ(x ) ) F(X, X ) C P ( ϕ(x) r ) 2 e r2 /2C, r 0 S. Chatterjee, P. Dey ( ) S. Ghosh, L. Goldstein (2011) size-biased couplings graphs, matching, Gibbs measures, Ising model, matrix concentration, machine learning...

30 R Stein s kernel ν (centered) probability measure on R Stein s kernel for ν : x τ ν (x) x φ dν = τ ν φ dν, φ : R R smooth R γ standard normal τ γ = 1 Stein s discrepancy S(ν γ) S 2( ν γ ) = τ ν 1 2 dν Stein s inequality ν γ TV 2 S ( ν γ ) R

31 multidimensional Stein s kernel ν (centered) probability measure on R d (existence) Stein s kernel (matrix) for ν : R d x φ dν = x τ ν (x) = ( τ ij ν (x) ) 1 i,j d R d τ ν φ dν, φ : R d R smooth Stein s discrepancy S(ν γ) S 2( ν γ ) = τ ν Id 2 HS dν R d no Stein s inequality in general

32 entropy and total variation Stein s inequality (on R) ν γ TV 2 S ( ν γ ) stronger distance in entropy ν probability measure on R d, ν << γ dν = h dγ relative H-entropy H ( ν γ ) = R d h log h dγ Pinsker s inequality ν γ 2 TV 1 2 H( ν γ )

33 convergence in entropy of Wiener chaos I. Nourdin, G. Peccati, Y. Swan (2013) (F n ) n N sequence of Wiener chaos, fixed degree H ( L(F n ) γ ) 0 as S ( L(F n ) γ ) 0 (fourth moment theorem S(L(F n ) γ) 0)

34 Plan 1. logarithmic Sobolev inequalities 2. Stein s method 3. links and applications

35 Stein and logarithmic Sobolev γ standard Gaussian measure on R d, dν = h dγ logarithmic Sobolev inequality H ( ν γ ) 1 2 I( ν γ ) (relative) H-entropy H ( ν γ ) = h log h dγ R d (relative) Fisher Information I ( ν γ ) Rd h 2 = dγ h (relative) Stein discrepancy S 2( ν γ ) = τ ν Id 2 HS dν R d

36 HSI inequality new HSI inequality H-entropy-Stein-Information H ( ν γ ) 1 2 S2( ν γ ) ( ) I(ν γ) log 1 + S 2, ν << γ (ν γ) log(1 + x) x improves upon the logarithmic Sobolev inequality potential towards concentration inequalities

37 HSI inequality new HSI inequality H-entropy-Stein-Information H ( ν γ ) 1 2 S2( ν γ ) ( ) I(ν γ) log 1 + S 2, ν << γ (ν γ) entropic convergence if S(ν n γ) 0 and I(ν n γ) bounded, then H ( ν n γ ) 0

38 HSI and entropic convergence entropic central limit theorem X, X 1,..., X n iid real random variables, mean zero, variance one S n = 1 n (X X n ) J. Linnik (1959), A. Barron (1986) H ( L(S n ) γ ) 0

39 HSI and entropic convergence entropic central limit theorem X, X 1,..., X n iid real random variables, mean zero, variance one S n = 1 n (X X n ) S 2( L(S n ) γ ) 1 n Var( τ L(X) (X) ) Stam s inequality I ( L(S n ) γ ) I ( L(X) γ ) <

40 HSI inequality HSI inequality H ( ν γ ) 1 2 S2( ν γ ) ( ) I(ν γ) log 1 + S 2, ν << γ (ν γ) H-entropy H(ν γ) Fisher Information I(ν γ) Stein discrepancy S(ν γ)

41 HSI and entropic convergence entropic central limit theorem X, X 1,..., X n iid real random variables, mean zero, variance one S n = 1 n (X X n ) S 2( L(S n ) γ ) 1 n Var( τ L(X) (X) ) Stam s inequality HSI inequality I ( L(S n ) γ ) I ( L(X) γ ) < H ( L(S n ) γ ) ( ) log n = O n optimal O( 1 n ) under fourth moment on X S. Bobkov, G. Chistyakov, F. Götze ( )

42 logarithmic Sobolev inequality and concentration ϕ : R d R 1-Lipschitz, R ϕ dγ = 0 d γ(ϕ r) e r2 /2, r 0 Gaussian concentration equivalent (up to numerical constants) ( ) 1/p ϕ p dγ C p, p 1 R d moment growth: concentration rate

43 HSI and concentration ν (centered) probability measure on R d ϕ : R d R 1-Lipschitz, R ϕ dν = 0 d moment growth in p 2, C > 0 numerical ( ) 1/p ( ) ϕ p ( ) 1/p] dν C[S p ν γ + p τ ν p/2 Op dν R d R d ( ) ( ) 1/p S p ν γ = τ ν Id R p HS dν d

44 HSI and concentration S n = 1 n (X X n ) X, X 1,..., X n iid mean zero, covariance identity in R d ϕ : R d R 1-Lipschitz ( ϕ(sn P ) E ( ϕ(s n ) ) ) r C e r2 /C 0 r r n according to the growth in p of S p ( ν γ )

45 HWI inequality F. Otto, C. Villani (2000) H ( ν γ) W 2 (ν, γ) I ( ν γ ) 1 2 W2 2 (ν, γ) Kantorovich-Wasserstein distance W2 2 (ν, γ) = inf ν π γ R d R d x y 2 dπ(x, y) covers both the logarithmic Sobolev inequality and Talagrand s transportation cost inequality W 2 2 (ν, γ) 2 H( ν γ )

46 WSH inequality Talagrand s inequality W 2 2 (ν, γ) 2 H( ν γ ) ν << γ (centered) probability measure on R d WSH inequality W 2 (ν, γ) S ( ν γ ) ) H(ν γ) arccos (e S 2 (ν γ) arccos(e r ) 2r Stein s inequality (in R d ) W 2 (ν, γ) S ( ν γ )

47 HSI inequality: elements of proof HSI inequality H ( ν γ ) 1 2 S2( ν γ ) ( ) I(ν γ) log 1 + S 2, ν << γ (ν γ) H-entropy H(ν γ) Fisher Information I(ν γ) Stein discrepancy S(ν γ)

48 HSI inequality: elements of proof Ornstein-Uhlenbeck semigroup (P t ) t 0 P t f (x) = f ( e t x + 1 e 2t y ) dγ(y) R d dν = h dγ, dν t = P t h dγ (ν 0 = ν, ν = γ) de Bruijin s formula H ( ν γ ) = I ( ν t γ ) e 2t I ( ν γ ) Bakry-Émery criterion classical logarithmic Sobolev inequality 0 I ( ν t γ ) dt

49 H ( ν γ ) = HSI inequality: elements of proof 0 I ( ν t γ ) dt classical I ( ν t γ ) e 2t I ( ν γ ) new main ingredient I ( ν t γ ) e 4t 1 e 2t S2( ν γ ) (information theoretic) representation of I(ν t γ) e 2t [ (τν (x) Id ) ( y v t e t x + 1 e 2t y )] dν(x)dγ(y) 1 e 2t R d R d v t = log P t h optimize small t > 0 and large t > 0

50 HSI inequalities for other distributions H ( ν µ ) 1 2 S2( ν µ ) ( ) I(ν µ) log 1 + S 2 (ν µ) µ gamma, beta distributions multidimensional families of log-concave distributions µ Markov Triple (E, µ, Γ) (abstract Wiener space)

51 Markov Triple Markov Triple (E, µ, Γ) Markov (diffusion) operator L with state space E µ invariant and symmetric probability measure Γ bilinear gradient operator (carré du champ) Γ(f, g) = 1 2 [ L(f g) f Lg g Lf ], f, g A E integration by parts f ( Lg) dµ = Γ(f, g)dµ E

52 examples of Markov Triple second order differential operator on E = R d d d L = a ij ijf 2 + b i i f i,j=1 i=1 d Γ(f, g) = a ij i f j g i,j=1 Lf dµ = 0 R d example: Ornstein-Uhlenbeck operator d d Lf = f x f = ijf 2 x i i f i,j=1 i=1 γ invariant measure

53 Stein kernel for diffusion operator L = d d a ij ijf 2 + b i i f = a, Hess(f ) HS + b f i,j=1 i=1 µ invariant measure ( R d Lf dµ = 0 ) Stein kernel τ ν b f dν = τν, Hess(f ) R d R d HS dν Stein discrepancy S 2( ν µ ) = a 1 2 τν a 1 2 Id 2 R d HS dν approximation of diffusions A. Barbour (1990), F. Götze (1991)

54 Markov Triple (E, µ, Γ) abstract HSI inequality Markov (diffusion) operator L with state space E µ invariant and symmetric probability measure Γ bilinear gradient operator (carré du champ) Γ(f, g) = 1 [ ] 2 L(f g) f Lg g Lf, f, g A logarithmic Sobolev inequality Γ 2 iterated gradient operator D.Bakry, M. Émery (1985) Γ 2 (f, g) = 1 [ ] 2 L Γ(f, g) Γ(f, Lg) Γ(g, Lf ), f, g A

55 abstract HSI inequality P t = e t L, t 0, dν t = P t hdµ H ( ν µ ) = 0 I ( ν t µ ) dt classical I ( ν t µ ) e 2ρt I ( ν µ ) Bakry-Émery Γ 2 criterion Γ 2 (f ) = Γ 2 (f, f ) ρ Γ(f, f ) = Γ(f ), d dt I( ν t µ ) = 2 E E P t h Γ 2 (log P t h)dµ f A 2ρ P t h Γ(log P t h)dµ = 2ρ I ( ν t µ )

56 abstract HSI inequality I ( ν t µ ) c e 2ct 1 S2( ν µ ) new feature: Γ 3 operator Γ 3 (f ) c Γ 2 (f ), f A Γ 3 (f, g) = 1 2 [ L Γ2 (f, g) Γ 2 (f, Lg) Γ 2 (g, Lf ) ], f, g A control of Hessians (Stein s kernel) P t ( Γ(h) ) e 2ct 1 c a 1 2 Hess(Pt h)a HS

57 Gammas Ornstein-Uhlenbeck operator Lf = f x f on R d Γ(f ) = f 2 Γ 2 (f ) = 2 f 2 + f 2 Γ 3 (f ) = 3 f f f 2 Bochner s formula for Laplacian on (M, g) Riemannian manifold Γ 2 (f ) = 2 f 2 + Ric g ( f, f ) Γ 2 (f ) ρ Γ(f ) Ric g ρ

58 HSI inequalities for other distributions H ( ν µ ) C S 2( ν µ ) ( ) C I(ν µ) Ψ S 2 (ν µ) Ψ(r) = 1 + log r, r 1 µ gamma, beta distributions multidimensional families of log-concave distributions µ discrete models (Poisson) under investigation

59 beyond the Fisher information Markov Triple (E, µ, Γ) (abstract Wiener space) I(ν γ) difficult to control in general F = (F 1,..., F d ) : E R d with law L(F) H ( L(F) γ ) C F S 2( L(F) γ ) ( ) C F Ψ S 2 (L(F) γ) Ψ(r) = 1 + log r, r 1 C F > 0 depend on integrability of F, Γ(F i, F j ) and inverse of the determinant of (Γ(F i, F j )) 1 i,j d (Malliavin calculus)

60 beyond the Fisher information towards entropic convergence via HSI I(ν γ) difficult to control in general Wiener chaos or multilinear polynomial F = X 1,..., X N N a i1,...,i k X i1 X ik i 1,...,i k =1 independent standard normals a i1,...,i k R symmetric, vanishing on diagonals law L(F) of F? Fisher information I ( L(F) γ )?

61 beyond the Fisher information I. Nourdin, G. Peccati, Y. Swan (2013) (F n ) n N sequence of Wiener chaos, fixed degree H ( L(F n ) γ ) 0 as S ( L(F n ) γ ) 0 (fourth moment theorem S(L(F n ) γ) 0)

62 HSI without I Markov Triple (E, µ, Γ) (abstract Wiener space) F = (F 1,..., F d ) : E R d with law L(F) H ( L(F) γ ) C F S 2( L(F) γ ) ( ) C F Ψ S 2 (L(F) γ) Ψ(r) = 1 + log r, r 1 C F > 0 depend on integrability of F, Γ(F i, F j ) and inverse of the determinant of (Γ(F i, F j )) 1 i,j d (Malliavin calculus)

63 HSI without I H ( L(F) γ ) C α S 2( L(F) γ ) ( ) 2(AF + d(b F + 1)) Ψ S 2 (L(F) γ) A F < moment assumptions B F = E 1 det( Γ) α dµ, α > 0 Γ = ( Γ(F i, F j ) ) 1 i,j d (Malliavin matrix)

64 B F = E 1 det( Γ) α dµ, α > 0 HSI without I Gaussian vector chaos F = (F 1,..., F d ) Γ(F i, F j ) = DF i, DF j H L(F) density: E(det( Γ)) > 0 P ( det( Γ) λ ) cnλ 1/N E ( det( Γ) ) 1/N, λ > 0 N degrees of the F i s A. Carbery, J. Wright (2001) log-concave models

65 Thank you for your attention