Lecture 7: Advanced Coupling & Mixing Time via Eigenvalues

Transcription

1 Counting and Sampling Fall 07 Lecture 7: Advanced Coupling & Miing Time via Eigenvalues Lecturer: Shaan Oveis Gharan October 8 Disclaimer: These notes have not been subjected to the usual scrutin reserved for formal publications. In this lecture first we discuss [HV05] to prove that the Glauber dnamics generates a random coloring of a graph G with maimum degree using q.764 colors in On log n). Our main motivation is to introduce additional more technical tools in coupling, beond the path coupling technique. We prove the following theorem Theorem 7.. Let α satisfies α = e /α. If G is triangle free with ma degree, and q α + ɛ) and ɛ logn), then the heat-bath chain mies in O n ɛ logn)) time. The main idea behind the proof is a local uniformit propert which holds with high probabilit at the stationarit distribution. The show that for an triangle-free graph an eas lower bound on the number of available colors for all vertices of graph which holds with high probabilit at stationarit. We use the following notation throughout this section and the net one. For a coloring configuration X and a verte v let AX, v) be the set of colors available to v. In other words, AX, v) is all colors that are not appeared on an neighbors of v in X. Lemma 7.. Let G be a triangle-free graph with maimum degree, ɛ > 0 and q > +. For a uniforml random q-coloring of G, we have [ ] P v : AX, v) qe /q ɛ) ne ɛq. Proof. Fi a verte v. We condition on the colors of all vertices which are not neighbors of v. We call the conditional information F. Observe that, conditioned on F, all neighbors of v are colored independentl. This is because the form an independent set b the traingle-free assumption). Let X c,u be the random variable indicating that a neighbor u of v is colored with c. AX, v) = X c,u ). c u v Therefore, E [ AX, v) F] = c = c q c = q u v E [X c,u F]) u v u v:c AX,u) u v:c AX,u) c AX,u) q u v e /q = qe /q. ) AX, u) ) /q AX, u) ) /q AX, u) 7-

2 7- Lecture 7: Advanced Coupling & Miing Time via Eigenvalues where the first inequalit follows b the AM-GM inequalit. Note that since X c,u s are independent under F, and AX, v) is a Lipschitz function of X c,us it follows that we have strong concentration on the above epected bound. Therefore, the claim simpl follows b McDiarmid s inequalit and a union bound. Recall that McDiarmids inequalit sas that for an function fx,..., X n ) which is a Lipshitz function with Lipshitz constant of independent variables X,..., X n we have P [ fx,..., X n ) EfX,..., X n ) ɛ] e ɛ /n. Equipped with the above lemma, let α be such that α = e /α. Then, for q = α, and for Ωlog n) we have qe /q qe /α = q α = Therefore, if we take a slightl larger q, sa q = α + ɛ), for ɛ log n, we have that with probabilit /n a random coloring X satisifies the following: For all v, AX, v) > + Ωɛ)). 7. Improved bound on Coupling In this section step we show that if in a coloring X all vertices have strictl more than available colors, then we can couple X with an proper coloring Y such that the distance decreases in epectation. Note that as a special case, this will also give us a proof of the q + bound from last lecture using ordinar coupling. We work with a slightl different Markov chain: We choose a random verte v and we assign a color that is not appeared in an of its neighbors uniforml at random, i.e., we work with the Heat-Bath chain as opposed to the Metropolis rule. Lemma 7.3. Let X be a proper coloring of G such that for some 0 < β < and all vertices v, AX, v) β. Then, for ever Y Ω, there eists a coupling X, Y ) X, Y ) such that E [dx, Y ) X, Y ] β/n) dx, Y ). Note that in particular, if q +, then, we can let β Ω/ ), and the above lemma would simpl impl the miing. Proof. We use a coupling strateg similar to the one we considered last time. First of all, both X, Y choose the same verte v. For ever color c AX, v) AY, v) both chains choose c with probabilit min{ AX,v), AY,v) }. Note that in all other cases v will end up with different colors in X, Y. So, we don t

3 Lecture 7: Advanced Coupling & Miing Time via Eigenvalues 7-3 need to specif how to match the rest of the events. Now, observe that P [c X v) c Y v) v] = AX, v) AY, v) ma{ AX, v), AY, v) } = ma{ax, v), AY, v)} AX, v) AY, v) ma{ax, v), AY, v)} {u v : c Xu) c Y u)} ma{ax, v), AY, v)} β {u v : c Xu) c Y u)} Note that the second to last inequalit can be justified as follows: Sa AX, v) > AY, v). Then, for ever color in c AX, v) \ AX, v) AY, v)) there must be a verte u that is a neighbor of v such that c Y u) = c c X u). The last inequalit follows b the assumption of the lemma. Therefore, for ever verte v, P [c X v) c Y v)] β n {u v : c Xu) c Y u)} + Summing this over all v, we get as desired. E [dx, Y ) X, Y ] β dx, Y )) + n ) I [c X v) c Y v)]. n ) dx, Y ) = β ) dx, Y ) n n 7. Coupling with stationarit Using lemmas 7. and 7.3, to prove Theorem 7., it remains to show that for a coupling proof it is enough to show that for most states X the distance decreases: Theorem 7.4. Suppose that the distance function d.,.) is integral. For ɛ > 0 and S Ω suppose that for all X Ω and Y S there is a coupling X, Y ) X, Y ) such that if πs) ɛ diamω), then τ mi O E [dx, Y ) X, Y ] ɛ)dx, Y ). logdiamω)) Proof. The main insight of the proof is that in a coupling proof we can alwas assume X 0 = for some state Ω and Y 0 π. Therefore, for all t, P [Y t / S] = πs). As usual if X = Y, then X = Y. Otherwise, if Y t is not in S we just follow an independent coupling of X t, Y t. Otherwise, we use the coupling promised in the statement of the theorem. Now, observe that on the event that Y t / S, we have no control on the dx t+, Y t+ ) but we know in the worst case it is at most diamω). Otherwise, we know the distance decreases. Therefore, E [dx, Y )] = E [dx, Y ) Y S] P [Y S] + E [dx, Y ) Y / S] P [Y / S] ɛ ɛ)dx, Y ) + diamω) ɛ/)dx, Y ). diamω) ɛ ).

4 7-4 Lecture 7: Advanced Coupling & Miing Time via Eigenvalues where we used that dx, Y ) because X Y and d.,.) is integer. O ɛ logdiamω))). So, the chain mies in time This completes the proof of Theorem Miing Time Using Eigenvalues In this section we discuss a spectral theor of reversible Markov chains. Based on this theor in the net lecture we introduce the path technolog which will be one of the strongest tools to analze Markov chains. Note that we particular focus on reversible chains because ever chain that we will use in our counting tasks is reversible. In this section we will treat the Markov kernel K as an operator. It turns out that K is not a smmetric matri operator). But, it has real eigenvalues. There are several was to see this. Perhaps, one of the cleanest was is to stud K in a different linear product space, in which K is self-adjoint, i.e., smmetric. The inner product space L π) is defined as follows: f, g = f)g)π). In particular, f = f) π). Recall that for an vector/function f, Kf) = K, )f). Although K is not a smmetric matri, it is self-adjoint with respect the above inner product, i.e., for an two functions f, g, Kf, g = f, Kg. This is because, Kf, g = Kf)g)π) = ) K, )f) g)π) = K, )f)g)π), =, K, )f)g)π) = f, Kg. Now, we can appl the spectral theorem: K has orthonormal set of eigenvalues λ λ Ω with corresponding eigenvectors ψ,..., ψ Ω such that for all i, and for all i j, Kψ i = λ i ψ i, ψ i, ψ j = 0. It is not hard to see that we alwas have λ = and ψ = because K is stochastic. Furthermore, also b stochasticit, λ i for all i. We can translate the ergodicit properties of a Markov chain to this language. A reversible) Markov chain is irreducible iff λ <, and a reversible) markov chain is aperiodic, i.e., it is not bipartite, iff λ Ω >. Let λ = ma i> λ i = ma{λ, λ n }.

5 Lecture 7: Advanced Coupling & Miing Time via Eigenvalues 7-5 Net, we show that if λ <, then we can bound the miing time b logn)/ λ ). Firstl, it turns out that it is more convenient to measure the miing time in L distance as opposed to the total variation distance; this is because eigenvalues are L properties of the chain. Definition 7.5 L P distance). For a kernel K, the L p distance of K t,.) to stationar is defined as K t,.) π.) = π) K t, ) ) p /p p π). Here, we are thinking of K t, )/π) as a function of. Observe that K t,.) π T V = K t, ) π) = π) π) K t, ) π) K t, ) π) = K t,.) π. 7.) So, if we prove that the L distance is small, it automaticall implies that the total variation distance is small. Note that if p =, then the distance of K t /π from is equal to ma K t, )/π). Theorem 7.6. Let λ = ma i> λ i = ma{λ, λ n }. Then, K t π λ t π). Therefore, b 7.), ) log ɛ π) τ ɛ) λ. Note that the above bound this bound is ideal in applications where π is ver far from a uniform distribution. In these scenarios we want to start the chain from a state with large π). This idea is usuall called a warm start. Proof. First, observe that for an eigenvector ψ i we have K t π, ψ i = K t, ) ψ i )π) = λ t π) iψ i ). Note that b above, Kt π, =. Therefore, K t π = i> K t π, ψ i ψ i = λ t iψ i )ψ i. i> And, K t π = λ t i ψ i ) λ t ψ i ) λ t π) i> i> The onl nontrivial step is the last inequalit which holds because ψ i ) = ψ i, π = ψ i, π ψ i = π = π). i i i where is the indicator function of.

6 7-6 Lecture 7: Advanced Coupling & Miing Time via Eigenvalues References [HV05] Tom Haes and Eric Vigoda. Coupling with the stationar distribution and improved sampling for colorings and independent sets. In SODA, pages Societ for Industrial and Applied Mathematics,