The coupling method - Simons Counting Complexity Bootcamp, 2016

Transcription

1 The coupling method - Simons Counting Complexity Bootcamp, 2016 Nayantara Bhatnagar (University of Delaware) Ivona Bezáková (Rochester Institute of Technology) January 26, 2016

2 Techniques for bounding the mixing time Probabilistic techniques - coupling, martingales, strong stationary times, coupling from the past. Eigenvalues and eigenfunctions. Functional, isoperimteric and geometric inequalities - Cheeger s inequality, conductance, Poincaré and Nash inequalities, discrete curvature. (Levin-Peres-Wilmer 2009, Aldous-Fill 1999/2014) have comprehensive accounts.

3 In this part of the tutorial Coupling distributions Coupling Markov chains Path Coupling Exact sampling - coupling from the past

4 Definition - Total variation distance 1 γ = α = β = µ ν tv Let µ and ν be probability measures on the same measurable space (Ω, F). The total variation distance between µ and ν is given by Here Ω is finite and F = 2 Ω µ ν tv = sup µ(a) ν(a) A F µ ν tv = max A Ω µ(a) ν(a) = 1 2 µ(x) ν(x) x Ω

5 Definition - Coupling of distributions (Doeblin 1938) Let µ and ν be probability measures on the same measurable space (Ω, F). A coupling of µ and ν is a pair of random variables (X, Y ) on the probability space (Ω Ω, F F, P) such that the marginals coincide P(X A) = µ(a), P(Y A) = ν(a), A F.

6 Example - Biased coins µ: Bernoulli(p), probability p of heads (= 1). ν: Bernoulli(q), probability q > p of heads. H µ (n) = no. of heads in n tosses. H ν (n) = no. of heads in n tosses. Proposition 1. P(H µ (n) > k) P(H ν (n) > k)

7 Example - Biased coins µ: Bernoulli(p), probability p of heads (= 1). ν: Bernoulli(q), probability q > p of heads. H µ (n) = no. of heads in n tosses. H ν (n) = no. of heads in n tosses. Proposition 1. P(H µ (n) > k) P(H ν (n) > k) Proof. For 1 i n, (X i, Y i ) a coupling of µ and ν. Indep. (X i, Y i ). Let X i µ. If X i = 1, set Y i = 1. If X i = 0, set Y i = 1 w.p. q p 1 p and 0 otherwise. P(X X n > k) P(Y Y n > k)

8 Coupling and Total Variation Distance Lemma 2. Let µ and ν be distributions on Ω. Then µ ν tv inf {P(X Y )} couplings (X,Y ) Proof. For any coupling (X, Y ) and A Ω µ(a) ν(a) = P(X A) P(Y A) P(X A, Y / A) P(X Y ). Similarly, ν(a) µ(a) P(X Y ). Therefore, µ ν tv P(X Y ).

9 Maximal coupling 1 γ = α = β = µ ν tv Lemma 3. Let µ and ν be distributions on Ω. Then µ ν tv = inf {P(X Y )} couplings (X,Y )

10 Coupling Markov chains (Pitman 1974, Griffeath 1974/5, Aldous 1983) A coupling of Markov chains with transition matrix P is a process (X t, Y t ) t=0 such that both (X t) and (Y t ) are Markov chains with transition matrix P. Applied to bounding the rate of convergence to stationarity of MC s for sampling tilings of lattice regions, particle processes, card shuffling, random walks on lattices and other natural graphs, Ising and Potts models, colorings, independent sets... (Aldous-Diaconis 1986, Bayer-Diaconis 1992, Bubley-Dyer 1997, Luby-Vigoda 1999, Luby-Randall-Sinclair 2001, Vigoda 2001, Wilson 2004,... )

11 Simple random walk on {0, 1..., n} SRW: Move up or down with probability 1 2 if possible. Do nothing if attempt to move outside interval. Claim 4. If 0 x y n, P t (y, 0) P t (x, 0).

12 Simple random walk on {0, 1..., n} SRW: Move up or down with probability 1 2 if possible. Do nothing if attempt to move outside interval. Claim 4. If 0 x y n, P t (y, 0) P t (x, 0). A coupling (X t, Y t ) of P t (x, ) and P t (y, ): X 0 = x, Y 0 = y. Let b 1, b 2... be i.i.d. {±1}-valued Bernoulli(1/2). At the ith step, attempt to add b i to both X i 1 and Y i 1.

13 Simple random walk on {0, 1..., n} For all t, X t Y t. Therefore, P t (y, 0) = P(Y t = 0) P(X t = 0) = P t (x, 0).

14 Simple random walk on {0, 1..., n} For all t, X t Y t. Therefore, P t (y, 0) = P(Y t = 0) P(X t = 0) = P t (x, 0). Note: Can modify any coupling so that the chains stay together after the first time they meet. We ll assume this to be the case.

15 Distance to stationarity and the mixing time Define d(t) := max x Ω Pt (x, ) π tv, d(t) := max x,y Ω Pt (x, ) P t (y, ) tv By the triangle inequality, d(t) d(t) 2d(t). The mixing time is t mix (ε) := min{t : d(t) ε} It s standard to work with t mix := t mix (1/4) (any constant ε < 1/2) because t mix (ε) log 2 ε 1 t mix.

16 Bound on the mixing time Theorem 5. Let (X t, Y t ) be a coupling with X 0 = x and Y 0 = y. Let τ be the first time they meet (and thereafter, coincide). Then P t (x, ) P t (y, ) tv P x,y (τ > t) Proof. By Lemma 2, P t (x, ) P t (y, ) tv P(X t Y t ) = P x,y (τ > t)

17 Bound on the mixing time Theorem 5. Let (X t, Y t ) be a coupling with X 0 = x and Y 0 = y. Let τ be the first time they meet (and thereafter, coincide). Then P t (x, ) P t (y, ) tv P x,y (τ > t) Proof. By Lemma 2, P t (x, ) P t (y, ) tv P(X t Y t ) = P x,y (τ > t) Corollary 6. t mix 4 max x,y E x,y (τ ). Proof. d(t) d(t) max x,y P x,y (τ > t) max x,y E x,y (τ ) t

18 Example - Card shuffling by random transpositions Shuffling MC on Ω = S n : Choose card X t and an independent position Y t uniformly. Exchange X t with σ t (Y t ) (the card at Y t ). Stationary distribution is uniform over permutations S n.

19 Example - Card shuffling by random transpositions Coupling of σ t, σ t: Choose card X t and independent position Y t uniformly. Use X t and Y t to update both σ t and σ t Let M t = number of cards at the same position in σ and σ.

20 Example - Card shuffling by random transpositions Coupling of σ t, σ t: Choose card X t and independent position Y t uniformly. Use X t and Y t to update both σ t and σ t Let M t = number of cards at the same position in σ and σ. Case 1: X t in same position M t+1 = M t.

21 Example - Card shuffling by random transpositions Coupling of σ t, σ t: Choose card X t and independent position Y t uniformly. Use X t and Y t to update both σ t and σ t Let M t = number of cards at the same position in σ and σ. Case 2: X t in different pos. σ(y t ) = σ (Y t ). M t+1 = M t.

22 Example - Card shuffling by random transpositions Mt+1 = Mt + 1 Mt+1 = Mt + 2 Case 3: Mt+1 = Mt + 3 I Xt in different pos. I σ(yt ) 6= σ 0 (Yt ). I Mt+1 > Mt.

23 Example - Card shuffling by random transpositions Proposition 7 (Broder). any x, y, Let τ be the first time M t = n. For E x,y (τ ) < π2 6 n2, t mix = O(n 2 ). Proof. Let τ i = steps to increase M t from i 1 to i so τ = τ 1 + τ τ n. P(M t+1 > M t M t = i) = Therefore, for any x, y (n i)2 n 2 E(τ i+1 M t = i) = n2 (n i) 2. n 1 E x,y (τ ) n 2 1 (n i) 2 < π2 6 n2. i=0

24 In this part of the tutorial Coupling distributions Coupling Markov chains Coloring MC Path Coupling Exact sampling - coupling from the past

25 A few remarks on yesterday s talk Shuffling by random transpositions has t mix O(n log n). Strong stationary times - random times at which the MC is guaranteed to be at stationarity. Maximal/optimal coupling 1 γ = α = β = µ ν tv Lemma 3. Let µ and ν be distributions on Ω. Then µ ν tv = inf {P(X Y )} couplings (X,Y )

26 Sampling Colorings Graph G = (V, E). Ω set of proper colorings of G. Metropolis MC: Select v V and k [q] uniformly. If k is allowed at v, update. Stationary distribution: uniform over colorings Ω (q > + 1).

27 Sampling Colorings Graph G = (V, E). Ω set of proper colorings of G. Metropolis MC: Select v V and k [q] uniformly. If k is allowed at v, update. Stationary distribution: uniform over colorings Ω (q > + 1).

28 Sampling Colorings Graph G = (V, E). Ω set of proper colorings of G. Metropolis MC: Select v V and k [q] uniformly. If k is allowed at v, update. Stationary distribution: uniform over colorings Ω (q > + 1).

29 A coupling for colorings Recall, Theorem 8. Let (X t, Y t ) be a coupling with X 0 = x and Y 0 = y. Let τ be the first time they meet (and thereafter, coincide). Then d(t) max P x,y (τ > t). x,y Theorem 9. Let G have max. degree. Let q > 4. Then, 1 t mix (ε) 1 4 /q n(log n + log(ε 1 )). Coupling: Generate a vertex-color pair (v, k) uniformly. Update the colorings X t and Y t by recoloring v with k if it is allowed.

30 Case analysis For colorings X t, Y t, let D t := {v X t (v) Y t (v)}. D t+1 = D t 1 w.p. Dt(q 2 ) qn

31 Case analysis For colorings X t, Y t, let D t := {v X t (v) Y t (v)}. D t+1 = D t 1 w.p. Dt(q 2 ) qn D t+1 = D t + 1 w.p. Dt(2 ) qn

32 Case analysis For colorings X t, Y t, let D t := {v X t (v) Y t (v)}. D t+1 = D t 1 w.p. Dt(q 2 ) qn E(D t+1 X t, Y t ) D t Dt(q 2 ) qn D t+1 = D t + 1 w.p. Dt(2 ) qn ( ) + 2Dt qn = D t 1 q 4 qn

33 Case analysis For colorings X t, Y t, let D t := {v X t (v) Y t (v)}. D t+1 = D t 1 w.p. Dt(q 2 ) qn E(D t+1 X t, Y t ) D t Dt(q 2 ) qn ( Iterating, E(D t X 0, Y 0 ) D 0 D t+1 = D t + 1 w.p. Dt(2 ) qn ( ) + 2Dt qn = D t 1 q 4 qn ) t ( n 1 q 4 qn 1 q 4 qn ) t

34 Mixing time We ve shown E(D t X 0 = x, Y 0 = y) n By Theorem 8, ( 1 q 4 ) t ( n exp q 4 qn q d(t) max P x,y (τ > t) = max P(D t > 1 X 0 = x, Y 0 = y) x,y x,y max x,y E(D t X 0 = x, Y 0 = y) t n ). Therefore, 1 t mix (ε) 1 4 /q n(log n + log(ε 1 )).

35 Fast mixing for q > 2 (Jerrum 1995, Salas-Sokal 1997) Theorem 10. Let G have max. degree. If q > 2, the mixing time of the Metropolis chain on colorings is ( ) q t mix (ε) n(log n + log(ε 1 )) q 2 Use path metrics on Ω to couple only colorings with a single difference and simplify the proof. Coupling colorings with just one difference in a smarter way.

36 Contraction in D t Recall, we showed contraction in one step: for some α > 0 E(D t+1 X t, Y t ) D t e α d(t) ne αt log n + log ε 1 t mix (ε) α

37 Contraction in D t Recall, we showed contraction in one step: for some α > 0 E(D t+1 X t, Y t ) D t e α d(t) ne αt log n + log ε 1 t mix (ε) α

38 Contraction in D t Recall, we showed contraction in one step: for some α > 0 E(D t+1 X t, Y t ) D t e α d(t) ne αt log n + log ε 1 t mix (ε) α

39 Path Coupling (Bubley-Dyer 1997) Connected graph (Ω, E 0 ) on Ω. Length function l : E 0 [1, ). A path from x 0 to x r is ξ = (x 0, x 1,..., x r ), and (x i 1, x i ) E 0. Length of path ξ is l(ξ) := r i=1 l((x i 1, x i ). Path metric on Ω is ρ(x, y) = min{l(ξ) ξ is a path between x, y}

40 Path Coupling (Bubley-Dyer 1997) Theorem 11 (Bubley-Dyer 97). If for each (x, y) E 0 there is a coupling (X 1, Y 1 ) of P(x, ) and P(y, ) so for some α > 0, E x,y (ρ(x 1, Y 1 )) e α ρ(x, y), then log(diam(ω) + log(ε)) t mix (ε) α where diam(ω) = max x,y ρ(x, y).

41 Metropolis chain on extended space Graph G = (V, E). Ω set of all colorings of G (including improper). Metropolis MC: Select v V and k [q] uniformly. If k is allowed at v, update. Stationary distribution: uniform on Ω, proper colorings.

42 Metropolis chain on extended space Graph G = (V, E). Ω set of all colorings of G (including improper). Metropolis MC: Select v V and k [q] uniformly. If k is allowed at v, update. Stationary distribution: uniform on Ω, proper colorings.

43 Metropolis chain on extended space Graph G = (V, E). Ω set of all colorings of G (including improper). Metropolis MC: Select v V and k [q] uniformly. If k is allowed at v, update. Stationary distribution: uniform on Ω, proper colorings.

44 Path coupling for colorings Let x y in ( Ω, E 0 ) if their color differs at 1 vertex. For (x, y) E 0, let l(x, y) = 1.

45 Path coupling for colorings Let x y in ( Ω, E 0 ) if their color differs at 1 vertex. For (x, y) E 0, let l(x, y) = 1.

46 Path coupling for colorings Let x y in ( Ω, E 0 ) if their color differs at 1 vertex. For (x, y) E 0, let l(x, y) = 1.

47 Coupling Let v be the disagreeing vertex in colorings x and y. Pick a vertex u and color k uniformly at random. If u / N(v) attempt to update u with k. If u N(v) If k / {x(v), y(v)} attempt to update u with k. Otherwise, attempt to update u in x with k and in y with the color in {x(v), y(v)} \ {k}. E x,y (ρ(x 1, Y 1 )) ρ(x, y) q qn + qn = q 2 qn

48 Sampling colorings Conjecture: O(n log n) mixing for q + 2. (Vigoda 1999) O(n 2 log n) mixing for q (Hayes-Sinclair 2005) Ω(n log n) lower bound. Restricted cases - triangle free graphs, large max. degree. Non-Markovian couplings. (Dyer, Flaxman, Frieze, Hayes, Molloy, Vigoda)

49 Exact Sampling - Coupling from the past (Propp-Wilson 1996) Two copies of the chain are run until they meet. When the chains meet, are they at stationarity? No: π(a) = 1 3, π(b) = 2 3, but the chains never meet at a. Coupling from the past (CFTP): If we run the chain backwards/ from the past from a time so that all trajectories meet, guaranteed to be at stationarity.

50 Random function representation of a MC Ergodic MC with transition matrix P. A distribution F over functions f : Ω Ω is a random function representation (RFR) iff P F (f (x) = y) = P(x, y) e.g. for the SRW on {0, 1..., n} let f (i) = min{i + 1, n}, f (i) = max{i 1, 0}. and F uniform on f and f. Proposition 12. Every transition matrix has an RFR, not necessarily unique. F defines a coupling on Ω via (x, y) f F (f (x), f (y))

51 Simulating forwards and backwards in time Associate a random function f t F to each t {,..., }. Forward simulation of the chain from x for t steps: F t 0(x) = f t 1 f 0 (x), P t (x, y) = P(F t 0(x) = y). Backward simulation of the chain from x for t steps: F 0 t(x) = f 1 f t (x). Let S be time so that F0 S (Ω) = 1. May not be stationary at S. CFTP: Let S be such that F S 0 (Ω) = 1. The chain is stationary!

52 Why does CFTP work? lim P(F t t 0(x) = y) = π(y) = lim P(F 0 t t(x) = y) Let S be such that F S 0 (Ω) = 1 and t > S. F 0 t(x) = f 1 f t (x) = F 0 S f S 1 f t (x) = F 0 S (y) = F 0 S (x). Since t > S was arbitrary, F 0 t has the same distribution as F 0.

53 Implementing CFTP Issues/Benefits: Choosing F so E(S) is small. When Ω is large, detecting when F S 0 (Ω) = 1. Exact samples even in absence of bounds on mixing time. Monotone CFTP: Partial order on states respected by the coupling with a maximal state x max and minimal state x min. Run CFTP from x max and x min. All trajectories will have converged when these coalesce. Ising Model, dimer configs. of hexagonal grid, bipartite independent set.

54 Strong stationary times A random variable τ is a stopping time for (X t ) if 1 τ=t is a function only of (X 0,..., X t ). A strong stationary time (SST) for (X t ) with stationary dist π is a randomized stopping time such that P x (τ = t, X τ = y) = P x (τ = t)π(y) Theorem 13. If τ is an SST then d(t) max P x (τ > t). x

55 Broder stopping time for random shuffling MC on S n : Choose L t and R t u.a.r. and transpose if different. Stopping time: Mark R t if both R t is unmarked Either L t is marked or L t = R t. Time τ for all cards to be marked is an SST. τ = τ τ n 1 where τ k = number of transpositions after kth card is marked and upto and including when k + 1st card is marked. ( ) (k + 1)(n k) τ k Geom n 2

56 Coupon collector estimate Can also calculate ( n 2 ) ( = n2 1 (k + 1)(n k) n + 1 k ) n k n 1 E(τ) = E(τ k ) = 2n(log n + O(1)) k=0 Var(τ) = O(n 2 ). Let t 0 = E(τ) + 2 Var(τ). By Chebyshev, P(τ > t 0 ) 1 4. t mix (2 + o(1))n log n.