215 Problem 1. (a) Define the total variation distance µ ν tv for probability distributions µ, ν on a finite set S. Show that

Transcription

1 15 Problem 1. (a) Define the total variation distance µ ν tv for probability distributions µ, ν on a finite set S. Show that µ ν tv = (1/) x S µ(x) ν(x) = x S(µ(x) ν(x)) + where a + = max(a, 0). Show that if P is the transition matrix of an irreducible, aperiodic Markov chain on a state space S with invariant distribution π, and if d(t) = sup x P t (x, ) π( ) tv then d(t) d(t) d(t) where d(t) = sup x P t (x, ) P t (y, ) tv. (b) Define what is meant by a coupling of µ and ν, and show that if (X, Y ) is such a coupling then µ ν tv P(X Y ). (c) Using a coupling or otherwise, show that d(t + s) d(t) d(s). Hence deduce that ρ = lim t d(t) 1/t exists. [Hint: you can use without proof the following lemma: if f is subadditive, i.e., if f(t + s) f(t) + f(s) for all s, t 0 then lim t f(t)/t exists in R { }.] (d) In the above setting, if the chain is also assumed to be reversible, what is the value of ρ? [You can use without proof any result from the course, provided it is clearly stated]. Page 1017 Part III Paper Mixing times of Markov Chains//15Draft 6th May 017

2 Marks Comments (a) Definition. The total variation distance between µ and ν is [] For the proof of d(t) d(t), note that µ ν tv = sup µ(a) ν(a) A S π(a) = y S π(y)p t (y, A) Lectures [3] Therefore, by the triangular inequality: π P t (x, ) tv = max A S = max A S max A S P t (x, A) π(a) π(y)[p t (x, A) P t (y, A)] y S π(y) P t (x, A) P t (y, A)] y S d(t) y S π(y) = d(t). (b) A coupling of µ and ν is the realisation of a pair of random variables (X, Y ) on the same probability space such that X µ and Y ν. For all couplings (X, Y ) of µ and ν, we have: Example class µ ν tv P(X Y ). (1) Furthermore, there always is a coupling (X, Y ) which achieves equality in (1). We start with the proof of the inequality (1), which is practice the only thing we use. (But it is reassuring to know that this is a sharp inequality!) Let (X, Y ) denote a coupling of µ and ν. If A is any subset of S, then we have: µ(a) ν(a) = P(X A) P(Y A) P(X A, X = Y ) P(Y A, X = Y ) + P(X A, X Y ) P(Y A, X Y ) [3] Note that the first term is in fact equal to zero, while the second is less or equal to P(X Y ), as desired. (c) We use the converse to the previous inequality (no need to prove): there is a coupling which achieves equality. Fix x, y S. Let (X s, Y s ) be an optimal coupling of Page 017 Part III Paper Mixing times of Markov Chains//15Draft 6th May 017

3 [6] P s (x, ) and P s (y, ), so P s (x, ) P s (y, ) tv = P(X s Y s ). Then P s+t (x, ) P s+t (y, ) tv = 1 P(X s+t = z) P(Y s+t = z) z = 1 E(P t (X s, z)) E(P t (Y s, z)) Taking the sup over x, y finishes the proof. y = E( P t (X s, ) P t (Y s, ) tv ) d(t)e(1 Xs Y s ) d(t)p(x s Y s ) d(t) d(s). Unseen, not easy to get it right Let f(t) = log d(t). Then f(t + s) f(t) + f(s), so by the hint, lim t f(t)/t exists in R { }, call it α. Then lim t d(t) 1/t = ρ := e α [0, ). Since [] (1/) d(t) d(t) d(t), and since (1/) 1/t 1 as t we deduce from the sandwich theorem that also d(t) 1/t ρ. Unseen (d) Recall that by a result of the course, since the chain is reversible, we have: P x (X t = y)/π(y) = λ t jf j (x)f j (y) where f j are an orthonormal basis of eigenfucntions (wrt l (π)) and λ j are the eigenvalues. so P t (x, y) π(y) = λ t jf j (x)f j (y)π(y). j= Taking the absolute value and summing over of y: P t (x, ) π( ) tv = 1 y j λ t jf j (x)f j (y)π(y) Here n is fixed and we let t : since each of the summand is asymptotic to C xy λ t j, we deduce that as t, P t (x, ) π( ) tv (1 γ ) t [4] where γ is the absolute spectral gap and a n b n means c 1 b n a n c b n for some constants c 1, c. Taking the sup over all x, and raising to the power 1/t we deduce that ρ = 1 γ. Unseen, not easy [0] Page 3017 Part III Paper Mixing times of Markov Chains//15Draft 6th May 017

4 15 Problem. (a) Let P be the transition matrix of an irreducible, aperiodic and reversible Markov chain on a finite state space S of size n with invariant distribution (π(x)) x S, with eigenvalues λ 1... λ n. Define the Dirichlet form E(f, f) associated to P, and give without proof an equivalent expression. State and prove the variational characterisation of the spectral gap in terms of E(f, f). State without proof a similar characterisation for higher order eigenvalues. (b) Let P, P be two transitive Markov chains on S, with corresponding Dirichlet forms E, Ẽ respectively. Suppose that if A > 0 is such that Ẽ(f, f) AE(f, f). State and prove a theorem concerning their respective mixing behaviours in L, defining carefully the expressions you introduce. (You can use without proof a relation between eigenvalues and L distance to stationarity, provided that this is stated clearly). (c) Define the interchange process on a connected graph G = (V, E). State a theorem giving a bound of mixing time of the interchange on G in terms of geometric quantities associated with G. (d) Suppose G = (V, E) = [0, n) Z is the n n square, and E is the set of nearest neighbour edges, so (u, v) E if and only if u v 1 = 1 for u, v V (here u 1 = u 1 + u for u = (u 1, u )). Show that the interchange process (in continuous time) satisfies t mix = O(n 4 log n). On the other hand, explain briefly, e.g. by considering the position of a single card, why t mix cn 4 for some c > 0. Page 4017 Part III Paper Mixing times of Markov Chains//15Draft 6th May 017

5 Marks Comments (a) Let f, g : S R. The Dirichlet form associated with P is defined by E(f, g) = (I P )f, g π. [1] It can be checked that E(f, f) = (1/) x,y π(x)p (x, y)(f(y) f(x)). Lectures [1] Theorem. Assume (P, π) is reversible, let γ be the spectral gap. Then γ = Equality is attained for f = f. min E(f, f) = f:s R E π(f)=0, f =1 min f:s R E π(f)=0 E(f, f) f. Proof: By scaling it suffices to prove the first equality. Now note that E π (f) = 0 = f, 1 π so the condition E π (f) = 0 means that f 1 f 1. Thus consider any function f with f = 1 with f 1, we have f = f, f j π f j = f, f j π f j since f, f 1 π = 0 by assumption. Using orthonormality of the eigenfunctions, and the fact that f = 1: j= E(f, f) = (I P )f, f π = f, f j π (1 λ j ) (1 λ ) f, f j = (1 λ ) f = γ. j= [3] On the other hand there is clearly equality for f = f. Lectures Theorem. Suppose P is irreducible, aperiodic, and reversible, let λ j be the eigenvalues ordered in nonincreasing order, so 1 = λ 1 λ... λ n. Then for 1 j n 1 λ j = max φ 1,...,φ j 1 min{e(f, f) : f = 1, f φ 1,..., φ j 1 }. [1] Lectures We let H t (x, y) denote the transition probabilities of the Marjov chain in continuous time, and call introduce d (t) defined by d (t) = H t (x, ) 1 π( ) Then we have the following theorem. Theorem. Assume that P, P are two transitive Markov chains on a set S, and suppose that Ẽ(f, f) AE(f, f) for all functions f : S R. Then if d (t), d (t) denote the respective l distance to stationarity of their continuous-time versions, d (t) d (t/a). Page 5017 Part III Paper Mixing times of Markov Chains//15Draft 6th May 017

6 [] Lectures Proof: For a fixed j 1, and for fixed functions φ 1,..., φ j 1, if W = Span(φ 1,..., φ j 1 ), min{ẽ(f, f) : f = 1; f W } A min{e(f, f) : f = 1, f W }. Taking the maximum over φ 1,..., φ j 1, we get so µ j A µ j. 1 λ j A(1 λ j ), Since d (t) = n [3] j= e tµ j, the result follows. Lectures [1] (c) Interchange process: Consider a fix connected graph G = (V, E) on n vertices labelled {v 1,..., v n }. One can define a random walk on the permutations of V by imagining that there is a card on each vertex of V and at each step, we exchange two neighbouring cards at random, or do nothing with probability 1/n (where n = V ). In other words the group is G = S(V ) S n and the set S of generators consists of the identity along with the transpositions (v i, v j ) for every pair of neighbouring vertices v i, v j. The kernel p is defined to be p(id) = 1/n, and p((v i, v j )) = (1 1/n)(1/ E ). Note that this set of generators is symmetric and the kernel is symmetric. Moreover since G is connected every transposition (v, w) can be built from neighbouring transpositions, and hence the set S generates all of S(V ). For each vertex x, y V fix γ xy a path on G from x to y, and set = max γ xy, x,y (where γ xy is the length of the path γ xy, measured in the number of edges), K = max e E {(x, y) V : e γ x,y }. Theorem The comparison Ẽ AE holds with A = 8 E K n(n 1). [] As a result, if t = (A/)n(log n + c), d (t) αe c for some universal constant α > 0. Lectures (d) Applications. We take the canonical paths γ xy on the graph G to be the one where we go in two segments from x to y, first chaning only the first coordinate and then changing only the second coordinate. Note that E = O(n ), = O(n), and K = O(n 3 ) since for a fixed edge e (assume wlog only the second coordinate is changing on the edge e), there are are most O(n) choices for y and and most O(n ) choices for x. [4] So we get A = O(n n n 3 /n (n 1)) = O(n ) (recall that now the base graph has not n elements but n ). Applying the Diaconis-Shahshahahani result this gives us a mixing time of at most An log(n ) in L and thus in total variation by Cauchy Schwarz. This implies t mix = O(n 4 log n), as desired. Similar to lectures Page 6017 Part III Paper Mixing times of Markov Chains//15Draft 6th May 017

7 Conversely, a single card performs a SRW on G, roughly every n steps. Since it takes t mix (G) cn steps for SRW to mix on G by a result of the course, we deduce that [] the mixing time is at least t mix cn 4, as desired. Unseen [0] Page 7017 Part III Paper Mixing times of Markov Chains//15Draft 6th May 017

8 15 Problem 3. (a) Let (X t, t = 0, 1,...) be an irreducible, aperiodic and reversible Markov chain on a finite state space S with invariant distribution π(y), y S. Define the notion of mixing time t mix (α) at level α (0, 1). Give the definition of the absolute spectral gap γ of the chain, as well as that of the relaxation time t rel, and give without proof the statement of a relation between relaxation time and mixing time at level ε > 0. (b) Define the bottleneck (or isoperimetric) ratio Φ of an irreducible, reversible Markov chain on a finite state space S. State Cheeger s inequality, and prove that if γ is the spectral gap, then γ Φ. (c) Let S = {1,..., n} be the n-cycle and consider the Markov chain on S which is the lazy simple random walk on S. Show that Φ = (1/n)(1 + o(1)) as n. Deduce that γ (1 + o(1))1/(n ), and hence show that t mix (1/4) O(n log n). (d) Compute all the eigenvalues of this Markov chain, and compare the estimate above with the actual value of the spectral gap. Page 8017 Part III Paper Mixing times of Markov Chains//15Draft 6th May 017

9 Marks Comments Definition. Let P be an irreducible, aperiodic transition matrix on a finite state space S, and let π(x) denote its stationary distribution, defined by the identity πp = π. Define the distance function for all t = 0, 1,... by: d(t) = max x S P t (x, ) π( ) tv. We also extend the definition of d(t) to all [0, ) by setting d(t) = d( t ). Note that by the ergodic theorem, d(t) 0 as t. Hence we can define, for 0 < ε < 1: t mix (ε) = inf{t 0 : d(t) ε}. [] Lectures Definition. Supose P irreducible and aperiodic. Let λ = max{ λ : λ eigenvalue 1}. γ = 1 λ is called the absolute spectral gap, and γ = 1 λ is called the spectral gap of P. The relaxation time t rel is defined by t rel = 1 γ. Theorem. ( ) ( ) 1 1 (t rel 1) log t mix (ε) log ε ε t rel π min [] Lectures (b) Recall our notation Q(e) = π(x)p (x, y) for the equilibrium flow through the edge e = (x, y). Let Q(A, B) = x A,y B Q(x, y) and define the bottleneck ratio of a set A to be Φ(A) = Q(A, Ac ) π(a) Essentially this is a measure of the size of the boundary relative to the total size of the set A. Definition The bottleneck ratio of the Markov chain is defined by [3] Φ = We now state Cheeger s inequality: min Φ(A). A:π(A) 1/ Theorem Suppose P is reversible and let γ = 1 λ be the spectral gap. Then [] Proof of upper bound. Φ γ Φ. Page 9017 Part III Paper Mixing times of Markov Chains//15Draft 6th May 017

10 We start with the proof that γ Φ which is easier. By the variational characterisation of the spectral gap, γ = min f 0;E π(f)=0 E(f, f) var π (f) Define f to be the function which is constant on A and A c by f(x) = π(a c ) if x A, f(x) = π(a) if x A c. Then note that E π (f) = 0. Moreover, E(f, f) = 1 π(x)p (x, y)(f(x) f(y)) = 1 [Q(A, Ac ) + Q(A c, A)] = Q(A, A c ). x,y On the other hand, var π (f) = E π (f ) = π(x)π(a c ) + π(x)π(a ) x A x A c = π(a)π(a c ) + π(a c )π(a) = π(a)π(a c ) π(a)/. [3] Consequently, γ Q(A, Ac ) π(a)/. Taking the minimum over all sets A such that π(a) 1/ gives γ Φ, as desired. (c) It is clear that the set A which realises the infimum must be an arc, since given A, the hull of A (the interval between left most point and rightmost point) or its complement has bigger mass and smaller boundary. Furthermore, we see that 1 A or n A since translating A to contain one or the other will likewise reduce φ(a). We deduce that Lectures Φ = min (1/)/a = 1/( n/ ) = (1/n)(1 + o(1) 1 a n/ Hence γ Φ / (/n )(1+o(1)). Hence since the walk is lazy, γ = γ and t rel = O(n ). [4] Applying the above theorem, and since π min = 1/n, we deduce that t mix = O(n log n). Unseen (d) We view S as a subset of the complex plane W n = {1, ω, ω,..., ω n 1 } with ω = e iπ/n. Let P be the matrix of this walk. To be an eigenfunction f with eigenvalue λ for P means that λf(ω k ) = P f(ω k ) = 1 (f(ωk+1 ) + f(ω k 1 )) for all 1 k n. We claim that the functions φ j (z) = z j, 1 j n, give us n eigenvalues. Indeed, φ j (ω k+1 ) + φ j (ω k 1 ) = ω jk ωj + ω j = φ j (ω k )Re(ω j ) = φ j (ω k ) cos ( πj n ). Page Part III Paper Mixing times of Markov Chains//15Draft 6th May 017

11 Thus φ j is an eigenfunction with eigenvalue cos(πj/n). Since the φ j are linearly independent, we deduce that these are all the eigenfunctions. If n is even, the chain is periodic and the absolute spectral gap is 0. If n is odd, the chain is aperiodic and the absolute spectral gap is equal to ( ) π[(n + 1)/] ( π 1 + cos = 1 cos n n) π n [4] as n. Thus So the estimate was off by a factor 1/π. t rel n π. [0] Page Part III Paper Mixing times of Markov Chains//15Draft 6th May 017

12 15 Problem 4. (a) Let (X t, t = 0, 1,...) be an irreducible, aperiodic and reversible Markov chain on a finite state space S with invariant distribution π(y), y S. Show that if P t (x, y) denote the t-step transition probabilities of the chain, P t (x, y) π(y) = λ t jf j (x)f j (y) where λ j are the eigenvalues and f j are functions which you should specify. [You can assume without proof that there exists an orthonormal basis of eigenfunctions for the inner product associated with the l norm f = ( x f(x) π(x)) 1/ ]. (b) Define the relaxation time t rel, and the l distance d (t) to equilibrium. Show that d(t) (1/)d (t) where d(t) is the total variation distance to equilibrium, and show that ( ) 1 t mix (ε) log ε t rel π min where π min = min{π(x) : x S}, and t mix (ε) is the mixing time at level ε. (c) Show that P t (x, x) is a decreasing sequence (as a function of t = 0, 1,...). Show however with an example that P t (x, x) is not in general monotone, as a function of t. (d) Show that d (t) is a contraction: for all t, s 0: d (t + s) d (s)e t/ t rel. [You can use freely without proof the inequality 1 x e x, valid for all x R]. Page 1017 Part III Paper Mixing times of Markov Chains//15Draft 6th May 017

13 Marks Comments (a) Let δ y be the function s S δ y (s) equal to 1 if s = y and 0 otherwise. By the spectral theorem (for symmetric matrices), we can expand this function on the orthonormal basis of eigenfunctions f j with respect to the inner product f, g π = x f(x)g(x)π(x): δ y = δ y, f j π f j = f j (y)π(y)f j. Hence, since P t (x, y) is nothing else but (P t δ y )(x) and λ t j is an eignevalue of P t we get: P t (x, y) = f j (y)π(y)λ t jf j (x) [3] as required. Lectures (b) Definition. Supose P irreducible and aperiodic. Let λ = max{ λ : λ eigenvalue 1}. γ = 1 λ is called the absolute spectral gap, and γ = 1 λ is called the spectral gap of P. The relaxation time t rel is defined by [1] [1] [3] t rel = 1 γ. P Definition Let d (t) = sup t (x, ) x S π( ) 1. We call d (t), the l distance to stationarity. Lemma Assume that P is irreducible, aperiodic (but not necessarily reversible). Then we have d(t) (1/)d (t). Proof: Recall that one of the basic identities for the definition of the total variation distance is P t (x, ) π = 1 P t (x, y) π(y) = π(y) P t (x, y) 1 π(y) = P t (x, ) 1 π( ) y y 1 where 1 refers to the l 1 (π) norm. Taking the square and using Jensen s inequality, we get 4 P t (x, ) π P t (x, ) 1 π( ) d (t). Taking the maximum over x gives the result. Theorem. ( t mix (ε). log 1 ε π min ) t rel Proof: Expanding the function on the eigenfunction basis f j, we get P t (x, ) 1 π( ) = f j (x)f j ( )λ t j j= = λ t j f j (x) λ t f j (x). j= j Page Part III Paper Mixing times of Markov Chains//15Draft 6th May 017

14 Now, we claim that n f j(x) = π(x) 1. Indeed, by decomposition: Hence π(x) = δ x, δ x π = f j (x) π(x). 4 P t (x, ) π λ t π(x) 1 λ t π 1 min (1 γ ) t π min e γ t π 1 min. Maximising over x and taking the square root, we get d(t) 1 e γ t π 1 min. Solving for the right-hand side equal to ε gives us d(t) ε as soon as t 1 1 [4] γ log( ε π min ). Lectures (c) When we specialise the eigenfunction expansion to y = x then we get P t (x, x) = π(x) j λ t j f j (x). [] All terms are positive and λ j 1 so the sum is decreasing (nonincreasing). Unseen Counterexample: take the n circle. Then P 0 (x, x) = 1, P 1 (x, x) = 0 but [] P (x, x) > 0. (Basically this is locally periodic.) Unseen (d) Observe that an exact expression (from the above proof) is so d (t + s) = d (t) = j= j= λ t j f j (x) λ t+s j f j (x) λ s d (t) (1 γ ) s d (s) e sγ d (s). Taking the square root, and by definition of t rel, d (t + s) e s/ t rel d (t), [4] as desired. Unseen [0] Page Part III Paper Mixing times of Markov Chains//15Draft 6th May 017