The Lovász Local Lemma: constructive aspects, stronger variants and the hard core model

Transcription

1 The Lovász Local Lemma: constructive aspects, stronger variants and the hard core model Jan Vondrák 1 1 Dept. of Mathematics Stanford University joint work with Nick Harvey (UBC)

2 The Lovász Local Lemma Theorem (Symmetric LLL, Lovász 1975) If E 1,..., E n are events on a probability space Ω such that Each event is independent of all but d other events 1 The probability of each event is at most e(d+1) (e = ) then n Pr[ E i ] > 0. i=1 Needle in a haystack" problem: 1. LLL implies that it is possible to avoid all events E 1,..., E n 2. but the probability of n i=1 E i could be exponentially small

3 Example: the r-partite Turán problem Consider an r-partite graph, at least ρ V i V j edges between every pair (V i, V j ). V 3 V 4 V 1 V 2 Question: at what density ρ must G contain K r?

4 Application of the LLL X 3 X 4 V 3 V 4 V 1 V 2 X i = random vertex in V i E ij = the event that (X i, X j ) / E We want: (X 1,..., X r ) such that no event E ij occurs. X 1 X 2

5 Application of the LLL X 3 X 4 V 3 V 4 V 1 V 2 X i = random vertex in V i E ij = the event that (X i, X j ) / E We want: (X 1,..., X r ) such that no event E ij occurs. X 1 X 2 Parameters: Pr[E ij ] = 1 ρ, d = 2(r 1) (dependencies only between E ij, E i j sharing an index).

6 Application of the LLL X 3 X 4 V 3 V 4 V 1 V 2 X i = random vertex in V i E ij = the event that (X i, X j ) / E We want: (X 1,..., X r ) such that no event E ij occurs. X 1 X 2 Parameters: Pr[E ij ] = 1 ρ, d = 2(r 1) (dependencies only between E ij, E i j sharing an index). LLL implies: If ρ 1 1 e(2r 1) then G contains a K r.

7 Application of the LLL X 3 X 4 V 3 V 4 V 1 V 2 X i = random vertex in V i E ij = the event that (X i, X j ) / E We want: (X 1,..., X r ) such that no event E ij occurs. X 1 X 2 Parameters: Pr[E ij ] = 1 ρ, d = 2(r 1) (dependencies only between E ij, E i j sharing an index). LLL implies: If ρ 1 1 e(2r 1) then G contains a K r. (Roughly correct: There is a graph with ρ = 1 1 r 1 without a K r.)

8 The General (asymmetric) Lovász Local Lemma Theorem (General LLL) If E 1,..., E n are events with a dependency graph", Γ(i) = neighborhood of i, so that Each event E i is independent of all the events E j, j / Γ(i) {i} There are x i (0, 1) such that Pr[E i ] x i E 3 E 2 (1 x j ). j Γ(i) E 4 E 1 E 5 E 6 Then n n Pr[ E i ] (1 x i ). i=1 i=1 (Symmetric variant can be obtained by setting x i = e Pr[E i ].)

9 Shearer s Lemma ( optimal form of the local lemma") For events E 1,..., E n with probabilities p 1,..., p n and a dependency graph G, define q S (p 1,..., p n ) = indep. I S( 1) I i I (alternating-sign independence polynomial of the dependency graph). p i Lemma (Shearer 1985) If S [n], q S (p 1,..., p n ) > 0, then n Pr[ E i ] q [n] (p 1,..., p n ). i=1 (If not, then Pr[ n i=1 E i] could be 0.)

10 Connection with statistical physics [Scott-Sokal 2005] Shearer s Lemma is closely related to the hard core model of repulsive gas in statistical physics. Model: particles on a graph G, two particles never adjacent; activity parameters w i. Pr[I] i I w i if I independent. Partition function: Z (w) = indep. I V i I w i. Fact: log Z (w) has an alternating-sign Taylor series around 0 ("Mayer expansion").

11 Hard core model vs. Shearer s Lemma [Scott-Sokal 2005] The following are equivalent: 1. Mayer expansion of log Z (w) is convergent for w i R i. 2. Z ( λr) > 0 for all 0 λ Z S ( R) > 0 for all subsets of vertices S, where Z S (w) = w i. indep. I S i I

12 Hard core model vs. Shearer s Lemma [Scott-Sokal 2005] The following are equivalent: 1. Mayer expansion of log Z (w) is convergent for w i R i. 2. Z ( λr) > 0 for all 0 λ Z S ( R) > 0 for all subsets of vertices S, where Z S (w) = indep. I S i I w i. Note: Z S ( p) = q S (p) are the quantities in Shearer s Lemma (whose positivity implies that all events can be avoided).

13 Hard core model vs. Lovász Local Lemma Let Γ(i) = neighborhood of i, and Γ + (i) = {i} Γ(i). Various sufficient conditions for the convergence of log Z (w) have been investigated. [Dobrushin 1996] If w i y i / j Γ + (i) (1 + y j) for some y i > 0, then the Mayer expansion for log Z (w) converges. Corresponds exactly to the LLL (substitute y i = x i 1 x i ).

14 Hard core model vs. Lovász Local Lemma Let Γ(i) = neighborhood of i, and Γ + (i) = {i} Γ(i). Various sufficient conditions for the convergence of log Z (w) have been investigated. [Dobrushin 1996] If w i y i / j Γ + (i) (1 + y j) for some y i > 0, then the Mayer expansion for log Z (w) converges. Corresponds exactly to the LLL (substitute y i = x i 1 x i ). [Fernandez-Procacci 2007] If w i y i / indep. I Γ + (i) i I y i for some y i > 0, then the Mayer expansion for log Z (w) converges. New criterion previously unknown to combinatorialists.

15 The Cluster Expansion Lemma Theorem (Bissacot-Fernandez-Procacci-Scoppola 2011) If E 1,..., E n are events with a dependency graph G, Each event E i is independent of its non-neighbor events. There are y i > 0 such that Pr[E i ] y i indep. I Γ + (i) i I y. i (To compare: in LLL, we sum up over all subsets I Γ + (i).) Then n Pr[ E i ] > 0. (Analytic Proof.) i=1

16 Combinatorial proof of Cluster Expansion [Harvey-V. 15] Define: P S = Pr[ i S E i], Y S = indep. I S We assume: Pr[E i ] y i /Y Γ + (i). i I y i.

17 Combinatorial proof of Cluster Expansion [Harvey-V. 15] Define: P S = Pr[ i S E i], Y S = indep. I S We assume: Pr[E i ] y i /Y Γ + (i). i I y i. Recursive bounds: P S = Pr[ i S a E i] Pr[E a i S a E i] P S a p a P S\Γ + (a), Y T +a = Y T +y a Y T \Γ + (a) Y T + p a Y T Γ + (a).

18 Combinatorial proof of Cluster Expansion [Harvey-V. 15] Define: P S = Pr[ i S E i], Y S = indep. I S We assume: Pr[E i ] y i /Y Γ + (i). i I y i. Recursive bounds: P S = Pr[ i S a E i] Pr[E a i S a E i] P S a p a P S\Γ + (a), Y T +a = Y T +y a Y T \Γ + (a) Y T + p a Y T Γ + (a). We claim, by induction, P S P S a Y S Y S a > 0.

19 Combinatorial proof of Cluster Expansion [Harvey-V. 15] Define: P S = Pr[ i S E i], Y S = indep. I S We assume: Pr[E i ] y i /Y Γ + (i). i I y i. Recursive bounds: P S = Pr[ i S a E i] Pr[E a i S a E i] P S a p a P S\Γ + (a), Y T +a = Y T +y a Y T \Γ + (a) Y T + p a Y T Γ + (a). We claim, by induction, Proof: P S P S a P S P S a 1 p a P S\Γ+ (a) P S a Y S Y S a > 0. 1 p a Y S Γ+ (a) Y S+a Y S Y S a.

20 Hierarchy of the Local Lemmas Symmetric LLL General LLL Shearer s Cluster Lemma Expansion

21 Application of Cluster Expansion to r-partite Turán X 3 X 4 V 3 V 4 V 1 V 2 X i = random vertex in V i E ij = the event that (X i, X j ) / E p ij = Pr[E ij ] = 1 ρ. X 1 X 2 Dependency graph G = line graph of K r. Neighborhood of E ij : two cliques, events incident to i and events incident to j. indep. I Γ + (ij) (i j ) I Set y = 1 r : y i j (1 + y (1+ry) 2 = 1 4r. r y ij )(1 + j =1 G always contains a K r for ρ 1 1 4r r y i j) = (1 + ry) 2 i =1 (when all y ij equal). (improvement from 1 1 2er )

22 Application of Shearer s Lemma [Csikváry-Nagy 2012] X 3 X 4 V 3 V 4 V 1 V 2 X i = random vertex in V i E ij = the event that (X i, X j ) / E p ij = Pr[E ij ] = 1 ρ. X 1 X 2 Dependency graph G = line graph of K r (events E ij, E i j dependent if they share an index). Independence polynomial q(p) of G = matching polynomial of K r = the Hermite polynomial.

23 Application of Shearer s Lemma [Csikváry-Nagy 2012] X 3 X 4 V 3 V 4 V 1 V 2 X i = random vertex in V i E ij = the event that (X i, X j ) / E p ij = Pr[E ij ] = 1 ρ. X 1 X 2 Dependency graph G = line graph of K r (events E ij, E i j dependent if they share an index). Independence polynomial q(p) of G = matching polynomial of K r = the Hermite polynomial. Roots of q(p) well understood: minimum positive root 1 4(r 2). G always contains a K r for ρ 1 1 4(r 2).

24 Tight bound for the r-partite Turán problem? V 2 V 3 Shearer s Lemma: For K 3, the matching polynomial is q 3 (p) = 1 3p. V 1 Minimum root p 0 = 1/3. This implies a K 3 subgraph for density ρ 2/3.

25 Tight bound for the r-partite Turán problem? V 2 V 3 Shearer s Lemma: For K 3, the matching polynomial is q 3 (p) = 1 3p. V 1 Minimum root p 0 = 1/3. This implies a K 3 subgraph for density ρ 2/3. But this is not tight: [Bondy-Shen-Thomassé-Thomassen 2006] The optimal density for K 3 is ρ = Open question: What is the optimal density that guarantees the appearance of K r in an r-partite graph, for r 4? (roughly between 1 1 4r and 1 1 2r )

26 The non-constructive aspect of the LLL The proof of LLL is essentially non-constructive: Pr[ n i=1 E i] is proved to be positive, but it could be exponentially small. How do we find a state ω n i=1 E i efficiently, given an instance where the LLL applies?

27 The non-constructive aspect of the LLL The proof of LLL is essentially non-constructive: Pr[ n i=1 E i] is proved to be positive, but it could be exponentially small. How do we find a state ω n i=1 E i efficiently, given an instance where the LLL applies? Example: Given an r-partite graph on n vertices, density of each pair ρ 1 1 4r. Can you find a K r subgraph in poly(n, r) time?

28 The Moser-Tardos framework Independent random variables X 1,..., X m. "Bad events" E 1,..., E n. Event E i depends on variables var(e i ). A dependency graph G: i j iff var(e i ) var(e j ). There are x 1,..., x n (0, 1) s.t. i; Pr[E i ] x i j Γ(i) (1 x j). (asymmetric LLL condition) E 1 E 2 E 3 E 4 X 0 X 1 X 2 X 3 X 4 X 5 X 6 X 7 X 8 X 9 Xa X b Xc X d Xe X f E 5 E 6 E 7 E 8 Moser-Tardos Algorithm: Start with random variables X 1,..., X m. As long as some event E i occurs, resample the variables in var(e i ).

29 The Moser-Tardos framework Independent random variables X 1,..., X m. "Bad events" E 1,..., E n. Event E i depends on variables var(e i ). A dependency graph G: i j iff var(e i ) var(e j ). There are x 1,..., x n (0, 1) s.t. i; Pr[E i ] x i j Γ(i) (1 x j). (asymmetric LLL condition) E 1 E 2 E 3 E 4 X 0 X 1 X 2 X 3 X 4 X 5 X 6 X 7 X 8 X 9 Xa X b Xc X d Xe X f E 5 E 6 E 7 E 8 Moser-Tardos Algorithm: Start with random variables X 1,..., X m. As long as some event E i occurs, resample the variables in var(e i ). Theorem (Moser-Tardos 08) This algorithm finds ω n i=1 E i after n operations (in expectation). i=1 x i 1 x i resampling

30 Beyond independent random variables The LLL gives interesting applications also in spaces with more structure: permutations Hamiltonian cycles matchings trees

31 Beyond independent random variables The LLL gives interesting applications also in spaces with more structure: permutations Hamiltonian cycles matchings trees Follow-up work: [Kolipaka-Szegedy 11] extension of Moser-Tardos to Shearer s setting. [Harris-Srinivasan 14] handle applications with random permutations. [Achlioptas-Iliopoulos 14] general approach based on random walks; handle Hamiltonian cycles, matchings.

32 We would like to have: "Algorithmic proof" of the LLL? given a probability space with events satisfying the LLL conditions, a (randomized) procedure that quickly finds ω n i=1 E i. Ω

33 Our Main Result "Algorithmic proof" of Shearer s Lemma: Arbitrary probability space Ω. Events E 1,..., E n with a dependency graph G. Each E i independent of non-neighbors (or more generally, "positively associated" with non-neighbors) p i = (1 + ɛ) Pr[E i ] satisfy Shearer s conditions. E 3 E 2 E 4 E 1 E 5 E 6

34 Our Main Result "Algorithmic proof" of Shearer s Lemma: Arbitrary probability space Ω. Events E 1,..., E n with a dependency graph G. Each E i independent of non-neighbors (or more generally, "positively associated" with non-neighbors) p i = (1 + ɛ) Pr[E i ] satisfy Shearer s conditions. E 3 E 2 E 4 E 1 E 5 E 6 Theorem (Harvey-V. 15) There is a randomized procedure which finds ω n i=1 E i under these assumptions after O( n ɛ log 1 ɛ ) "resampling operations" w.h.p.

35 Resampling operations Assume a space Ω with a probability measure µ, and events E 1,..., E n with a neighborhood structure denoted Γ(i). E i Ω ω r i (ω) Definition: A resampling operation r i for event E i is a random r i (ω) Ω for each ω Ω, such that 1. If ω has distribution µ conditioned on E i r i (ω) has distribution µ. (removes conditioning on E i ) 2. If k / Γ + (i) and ω / E k r i (ω) / E k. (does not cause non-neighbor events)

36 Why should resampling operations exist? Lemma (Harvey-V. 15) Resampling operations for events E 1,..., E n w.r.t. G exist whenever each E i is independent of its non-neighbor events.

37 Why should resampling operations exist? Lemma (Harvey-V. 15) Resampling operations for events E 1,..., E n w.r.t. G exist whenever each E i is independent of its non-neighbor events. More generally: Resampling operations exist if and only if each E i is "positively associated" with its non-neighbor events: E[Z E i ] E[Z ] for every monotonic function Z of (E j : j / Γ + (i)). Remark: necessary to handle permutations and matchings; not for trees.

38 The algorithm Our algorithm: Sample ω from µ. While any violated events exist, repeat: J As long as j / Γ + (J), E j occurs { ω r j (ω), (resample E i ) J J {j} } Note: In each iteration we resample an independent set of events J. In the next iteration, all violated events are in Γ + (J) = J Γ(J). (Γ(J) = neighbors of J in the dependency graph G)

39 Analysis of our algorithm Def.: Stab = {(I 1, I 2,..., I t ) : I i Ind(G) \ { }, I i+1 Γ + (I i )}. Coupling lemma: The probability that the algorithm resamples a sequence of independent sets (I 1, I 2,..., I t ) Stab is at most t p(i 1,..., I t ) = (p i = Pr [E i]), µ E[#iterations] p i s=1 i I s t=0 (I 1,...,I t ) Stab p(i 1,..., I t ).

40 Analysis of our algorithm Def.: Stab = {(I 1, I 2,..., I t ) : I i Ind(G) \ { }, I i+1 Γ + (I i )}. Coupling lemma: The probability that the algorithm resamples a sequence of independent sets (I 1, I 2,..., I t ) Stab is at most t p(i 1,..., I t ) = (p i = Pr [E i]), µ E[#iterations] p i s=1 i I s t=0 (I 1,...,I t ) Stab p(i 1,..., I t ). Summation identity: [Kolipaka-Szegedy 11] If Shearer s conditions are satisfied, then 1 p(i 1,..., I t ) = q(p 1,..., p n ) t=0 (I 1,...,I t ) Stab where q(p 1,..., p n ) = I Ind ( 1) I i I p i.

41 Analysis with slack Conditions with slack: Suppose p i = (1 + ɛ)p i and q S (p 1, p 2,..., p n) > 0 S [n]. Then Pr[#iterations t] (I 1,I 2,...,I t ) Stab p(i 1,..., I t ) We also prove: under an ɛ slack, q(p 1,..., p n) ɛ n. e ɛt q(p 1,..., p n).

42 Analysis with slack Conditions with slack: Suppose p i = (1 + ɛ)p i and q S (p 1, p 2,..., p n) > 0 S [n]. Then Pr[#iterations t] (I 1,I 2,...,I t ) Stab p(i 1,..., I t ) We also prove: under an ɛ slack, q(p 1,..., p n) ɛ n. e ɛt q(p 1,..., p n). Corollary: With high prob., the algorithm stops within O( n ɛ log 1 ɛ ) iterations.

43 Automatic slack for LLL conditions Lemma (Harvey-V. 15) If p i x i j Γ + (i) (1 x j) then for p i = ( n i=1 q S (p 1,..., p n) 1 (1 x i ). 2 i S x i 1 x i ) p i, I.e., when the LLL conditions are tight, there is still a slack of 1 ɛ = 2 n x i i=1 1 x i w.r.t. Shearer s conditions. LLL Shearer cluster expansion Corollary: E[#iterations] = O(( n x i i=1 1 x i ) 2 ) under LLL conditions.

44 [Achlioptas-Iliopoulos 14]: The latest news do not draw a formal connection with LLL; on the other hand claim to go beyond the LLL in some ways (orthogonal to Shearer s extension); neither framework subsumes the other. Update: [Achlioptas-Iliopoulos 16], [Kolmogorov 16] extended their framework to incorporate resampling operations Meanwhile, we extended our framework as well... both frameworks are becoming one unifying concept approximate resampling operations this captures exactly the following form of Shearer s Lemma...

45 Shearer s Lemma with lopsided conditioning Lemma Let E 1,..., E n be events with a graph G such that for every E i and every event F monotonically depending on (E j : j / Γ + (i)), Pr[E i F] p i. Let q S (p 1,..., p n ) = indep. I S ( 1) I i I p i. If S [n], q S (p 1,..., p n ) > 0, then n Pr[ E i ] > 0. i=1

46 Open questions Is there a deterministic algorithm to find ω n i=1 E i? Can we generate a random sample from µ n i=1 E i?