Characterization of cutoff for reversible Markov chains

Transcription

1 Characterization of cutoff for reversible Markov chains Yuval Peres Joint work with Riddhi Basu and Jonathan Hermon 3 December 2014 Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 1 / 34

2 Notation Transition matrix - P (reversible). Stationary dist. - π. Reversibility: π(x)p (x, y) = π(y)p (y, x), x, y Ω. Laziness P (x, x) 1/2, x Ω. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 2 / 34

3 TV distance For any 2 dist. µ, ν on Ω, their total-variation distance is: µ ν TV = d max µ(a) ν(a). A Ω d(t, x) d = P t x π TV, d(t) = d max d(t, x). x Ω Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 3 / 34

4 TV distance For any 2 dist. µ, ν on Ω, their total-variation distance is: µ ν TV = d max µ(a) ν(a). A Ω d(t, x) d = P t x π TV, d(t) = d max d(t, x). x Ω The ɛ-mixing-time (0 < ɛ < 1) is: t mix(ɛ) d = min {t : d(t) ɛ} t mix d = t mix(1/4). Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 3 / 34

5 Cutoff - definition Def: a sequence of MCs (X (n) t ) exhibits cutoff if t (n) mix (ɛ) t(n) (1 ɛ) = o(t(n) ), 0 < ɛ < 1/4. (1) mix mix Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 4 / 34

6 Cutoff - definition Def: a sequence of MCs (X (n) t ) exhibits cutoff if t (n) mix (ɛ) t(n) (1 ɛ) = o(t(n) ), 0 < ɛ < 1/4. (1) mix mix ( (w n) is called a cutoff window for (X (n) t ) if: w n = o t (n) mix t (n) mix ), and (ɛ) t(n) (1 ɛ) cɛwn, n 1, ɛ (0, 1/4). mix Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 4 / 34

7 Cutoff Figure : cutoff Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 5 / 34

8 Background Cutoff was first identified for random transpositions Diaconis & Shashahani 81 and RW on the hypercube by Aldous 83. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 6 / 34

9 Background Cutoff was first identified for random transpositions Diaconis & Shashahani 81 and RW on the hypercube by Aldous 83. Many chains are believed to exhibit cutoff. Verifying the occurrence of cutoff rigorously is usually hard. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 6 / 34

10 Background Cutoff was first identified for random transpositions Diaconis & Shashahani 81 and RW on the hypercube by Aldous 83. Many chains are believed to exhibit cutoff. Verifying the occurrence of cutoff rigorously is usually hard. The name cutoff was coined by Aldous and Diaconis in their seminal 86 paper. Aldous & Diaconis 86 - the most interesting open problem : Find verifiable conditions for cutoff. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 6 / 34

11 Background Cutoff was first identified for random transpositions Diaconis & Shashahani 81 and RW on the hypercube by Aldous 83. Many chains are believed to exhibit cutoff. Verifying the occurrence of cutoff rigorously is usually hard. The name cutoff was coined by Aldous and Diaconis in their seminal 86 paper. Aldous & Diaconis 86 - the most interesting open problem : Find verifiable conditions for cutoff. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 6 / 34

12 Spectral gap & relaxation-time Let λ 2 be the largest non-trivial e.v. of P. Definition: gap = 1 λ 2 - the spectral gap. Def: t rel := gap 1 - the relaxation-time. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 7 / 34

13 The product condition (Prod. cond.) In a 2004 Aim workshop I proposed that The product condition (Prod. Cond.) - gap (n) t (n) mix (equivalently, t(n) rel = o(t (n) mix )) should imply cutoff for nice reversible chains. (It is a necessary condition for cutoff) Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 8 / 34

14 The product condition (Prod. cond.) In a 2004 Aim workshop I proposed that The product condition (Prod. Cond.) - gap (n) t (n) mix (equivalently, t(n) rel = o(t (n) mix )) should imply cutoff for nice reversible chains. (It is a necessary condition for cutoff) It is not always sufficient - examples due to Aldous and Pak. Problem: Find families of MCs s.t. Prod. Cond. = cutoff. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 8 / 34

15 Aldous example 1/12 x 3/4 1/6 3/4 10n 1/12 - The mass is concentrated in a small neighborhood of y. - Last step away from z before Ty determines Ty. z 3/4 1/12 1/6 1/12 - each 1/6 n n 1/2 1/3 y Figure : Fixed bias to the right conditioned on a non-lazy step. Different laziness probabilities along the 2 paths. t (n) rel = O(1). d n(t) P x[t y > t] = ɛ d n(130n) d n(128n) 1 ɛ, for some ɛ.. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 9 / 34

16 Aldous example d n (t) 126n 132n Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 10 / 34

17 Hitting and Mixing Def: The hitting time of a set A Ω = T A := min{t : X t A}. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 11 / 34

18 Hitting and Mixing Def: The hitting time of a set A Ω = T A := min{t : X t A}. Hitting times of worst sets are related to mixing - mid 80 s (Aldous). Refined independently by Oliviera (2011) and Peres-Sousi (2011) (case α = 1/2 due to Griffiths-Kang-Oliviera-Patel 2012): for any irreducible reversible lazy MC and 0 < α 1/2: t H(α) = Θ α(t mix), where t H(α) := max E x[t A]. (2) x,a: π(a) α Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 11 / 34

19 Hitting and Mixing Def: The hitting time of a set A Ω = T A := min{t : X t A}. Hitting times of worst sets are related to mixing - mid 80 s (Aldous). Refined independently by Oliviera (2011) and Peres-Sousi (2011) (case α = 1/2 due to Griffiths-Kang-Oliviera-Patel 2012): for any irreducible reversible lazy MC and 0 < α 1/2: t H(α) = Θ α(t mix), where t H(α) := max E x[t A]. (2) x,a: π(a) α We relate d(t) and max x,a: π(a) α P x[t A > t] and refine (2) by also allowing 1/2 < α 1 exp[ Ct mix/t rel ] and improving Θ α to Θ. Remark: (2) may fail for α > 1/2. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 11 / 34

20 counter-example Kn Kn Figure : n is the index of the chain Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 12 / 34

21 Hitting and Cutoff Concentration of hitting times of worst sets is related to cutoff in birth and death (BD) chains. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 13 / 34

22 Hitting and Cutoff Concentration of hitting times of worst sets is related to cutoff in birth and death (BD) chains. Diaconis & Saloff-Coste (06) (separation cutoff) and Ding-Lubetzky-Peres (10) (TV cutoff): A seq. of BD chains exhibits cutoff iff the Prod. Cond. holds. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 13 / 34

23 Hitting and Cutoff Concentration of hitting times of worst sets is related to cutoff in birth and death (BD) chains. Diaconis & Saloff-Coste (06) (separation cutoff) and Ding-Lubetzky-Peres (10) (TV cutoff): A seq. of BD chains exhibits cutoff iff the Prod. Cond. holds. We extend their results to weighted nearest-neighbor RWs on trees. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 13 / 34

24 Cutoff for trees Theorem Let (V, P, π) be a lazy Markov chain on a tree T = (V, E) with V 3. Then t mix(ɛ) t mix(1 ɛ) C log ɛ t rel t mix, for any 0 < ɛ 1/4. In particular, the Prod. Cond. implies cutoff with a cutoff window w n = t (n) rel t(n) mix and c ɛ = C log ɛ. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 14 / 34

25 Cutoff for trees Theorem Let (V, P, π) be a lazy Markov chain on a tree T = (V, E) with V 3. Then t mix(ɛ) t mix(1 ɛ) C log ɛ t rel t mix, for any 0 < ɛ 1/4. In particular, the Prod. Cond. implies cutoff with a cutoff window w n = t (n) rel t(n) mix and c ɛ = C log ɛ. Ding Lubetzky Peres (10) - For BD chains t mix(ɛ) t mix(1 ɛ) O(ɛ 1 t rel t ( ) mix) and in some cases w n = Ω t (n) rel t(n) mix. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 14 / 34

26 To mix - escape and then relax Definition: hit α := hit α(1/4), where hit α,x(ɛ) := min{t : P x[t A > t] ɛ : for all A Ω s.t. π(a) α}, hit α(ɛ) := max x Ω hitα,x(ɛ) Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 15 / 34

27 To mix - escape and then relax Definition: hit α := hit α(1/4), where hit α,x(ɛ) := min{t : P x[t A > t] ɛ : for all A Ω s.t. π(a) α}, hit α(ɛ) := max x Ω hitα,x(ɛ) Easy direction: to mix, the chain must first escape from small sets = first stage of mixing. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 15 / 34

28 To mix - escape and then relax Definition: hit α := hit α(1/4), where hit α,x(ɛ) := min{t : P x[t A > t] ɛ : for all A Ω s.t. π(a) α}, hit α(ɛ) := max x Ω hitα,x(ɛ) Easy direction: to mix, the chain must first escape from small sets = first stage of mixing. Loosely speaking - we show that in the 2nd stage of mixing, the chain mixes at the fastest possible rate (governed by its relaxation-time). Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 15 / 34

29 Hitting times when X 0 π Fact: Let A Ω be such that π(a) 1/2. Then (under reversibility) P π[t A > 2st rel ] e s, for all s 0. 2 Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 16 / 34

30 Hitting times when X 0 π Fact: Let A Ω be such that π(a) 1/2. Then (under reversibility) P π[t A > 2st rel ] e s, for all s 0. 2 By a coupling argument, P x[t A > t + 2st rel ] d(t) + P π[t A > 2st rel ]. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 16 / 34

31 Hitting of worst sets For any reversible irreducible finite lazy chain and any 0 < ɛ 1/4, hit 1/2 (3ɛ) t rel log(2ɛ) t mix(2ɛ) hit 1/2 (ɛ) + t rel log(4ɛ) Terms involving t rel are negligible under the Prod. Cond.. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 17 / 34

32 Hitting of worst sets For any reversible irreducible finite lazy chain and any 0 < ɛ 1/4, hit 1/2 (3ɛ) t rel log(2ɛ) t mix(2ɛ) hit 1/2 (ɛ) + t rel log(4ɛ) Terms involving t rel are negligible under the Prod. Cond.. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 17 / 34

33 Hitting of worst sets For any reversible irreducible finite lazy chain and any 0 < ɛ 1/4, hit 1/2 (3ɛ) t rel log(2ɛ) t mix(2ɛ) hit 1/2 (ɛ) + t rel log(4ɛ) Terms involving t rel are negligible under the Prod. Cond.. A similar two sided inequality holds for t mix(1 2ɛ). Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 17 / 34

34 Main abstract result Definition: A sequence has hit α-cutoff if hit (n) α (ɛ) hit α (n) (1 ɛ) = o(hit α (n) ) for all 0 < ɛ < 1/4. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 18 / 34

35 Main abstract result Definition: A sequence has hit α-cutoff if hit (n) α Main abstract result: Theorem (ɛ) hit α (n) (1 ɛ) = o(hit α (n) ) for all 0 < ɛ < 1/4. Let (Ω n, P n, π n) be a seq. of finite reversible lazy MCs. Then TFAE: The seq. exhibits cutoff. The seq. exhibits a hit α-cutoff for some α (0, 1/2). The seq. exhibits a hit α-cutoff for some α [1/2, 1) and the Prod. Cond. holds. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 18 / 34

36 Main abstract result Definition: A sequence has hit α-cutoff if hit (n) α Main abstract result: Theorem (ɛ) hit α (n) (1 ɛ) = o(hit α (n) ) for all 0 < ɛ < 1/4. Let (Ω n, P n, π n) be a seq. of finite reversible lazy MCs. Then TFAE: The seq. exhibits cutoff. The seq. exhibits a hit α-cutoff for some α (0, 1/2). The seq. exhibits a hit α-cutoff for some α [1/2, 1) and the Prod. Cond. holds. The equivalence of cutoff to hit 1/2 -cutoff under the Prod. Cond. follows from the ineq. from the prev. slide together with the fact that hit (n) 1/2 = Θ(t(n) mix ). Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 18 / 34

37 Main abstract result Definition: A sequence has hit α-cutoff if hit (n) α Main abstract result: Theorem (ɛ) hit α (n) (1 ɛ) = o(hit α (n) ) for all 0 < ɛ < 1/4. Let (Ω n, P n, π n) be a seq. of finite reversible lazy MCs. Then TFAE: The seq. exhibits cutoff. The seq. exhibits a hit α-cutoff for some α (0, 1/2). The seq. exhibits a hit α-cutoff for some α [1/2, 1) and the Prod. Cond. holds. The equivalence of cutoff to hit 1/2 -cutoff under the Prod. Cond. follows from the ineq. from the prev. slide together with the fact that hit (n) 1/2 = Θ(t(n) mix ). For general α we show under the Prod. Cond. (using the tail decay of T A/t rel when X 0 π): hit α-cutoff for some α (0, 1) = hit β -cutoff for all β (0, 1). Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 18 / 34

38 Tools Def: For f R Ω, t 0, define P t f R Ω by P t f(x) := E x[f(x t)] = P t (x, y)f(y). y Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 19 / 34

39 Tools Def: For f R Ω, t 0, define P t f R Ω by P t f(x) := E x[f(x t)] = P t (x, y)f(y). y For f R Ω define E π[f] := x Ω π(x)f(x) and f 2 2 := E π[f 2 ]. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 19 / 34

40 Tools Def: For f R Ω, t 0, define P t f R Ω by P t f(x) := E x[f(x t)] = P t (x, y)f(y). y For f R Ω define E π[f] := x Ω π(x)f(x) and f 2 2 := E π[f 2 ]. For g R Ω denote Var πg := g E π[g] 2 2. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 19 / 34

41 Tools Def: For f R Ω, t 0, define P t f R Ω by P t f(x) := E x[f(x t)] = P t (x, y)f(y). y For f R Ω define E π[f] := x Ω π(x)f(x) and f 2 2 := E π[f 2 ]. For g R Ω denote Var πg := g E π[g] 2 2. The following is well-known and follows from elementary linear-algebra. Lemma (Contraction Lemma) Let (Ω, P, π) be a finite rev. irr. lazy MC. Let A Ω. Let t 0. Then Var πp t 1 A e 2t/t rel. (3) Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 19 / 34

42 Maximal Inequality The main ingredient in our approach is Starr s maximal-inequality (66) (refines Stein s max-inequality (61)) Theorem (Maximal inequality) Let (Ω, P, π) be a lazy irreducible reversible Markov chain. Let f R Ω. Define the corresponding maximal function f R Ω as f (x) := sup P k (f)(x) = sup E x[f(x k )]. 0 k< 0 k< Then for 1 < p <, f p q f p 1/p + 1/q = 1 (4) Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 20 / 34

43 Combining the Max-in. with the Contraction Lemma Goal: want for every A Ω to have G = G s(a) Ω s.t. T G t serves as a certificate of being ɛ-mixed w.r.t. A and to control its π measure from below. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 21 / 34

44 Combining the Max-in. with the Contraction Lemma Goal: want for every A Ω to have G = G s(a) Ω s.t. T G t serves as a certificate of being ɛ-mixed w.r.t. A and to control its π measure from below. Let σ s := e s/t rel Var πp s 1 A (contraction lemma). Consider } G = G s(a) := {g : s s, P s g(a) π(a) 4σ s. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 21 / 34

45 Combining the Max-in. with the Contraction Lemma Goal: want for every A Ω to have G = G s(a) Ω s.t. T G t serves as a certificate of being ɛ-mixed w.r.t. A and to control its π measure from below. Let σ s := e s/t rel Var πp s 1 A (contraction lemma). Consider } G = G s(a) := {g : s s, P s g(a) π(a) 4σ s. Want precision 4σ s = ɛ = s := t rel log(4/ɛ). Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 21 / 34

46 Combining the Max-in. with the Contraction Lemma Goal: want for every A Ω to have G = G s(a) Ω s.t. T G t serves as a certificate of being ɛ-mixed w.r.t. A and to control its π measure from below. Let σ s := e s/t rel Var πp s 1 A (contraction lemma). Consider } G = G s(a) := {g : s s, P s g(a) π(a) 4σ s. Want precision 4σ s = ɛ = s := t rel log(4/ɛ). Claim π(g) 1/2. (5) Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 21 / 34

47 Combining the Max-in. with the Contraction Lemma Goal: want for every A Ω to have G = G s(a) Ω s.t. T G t serves as a certificate of being ɛ-mixed w.r.t. A and to control its π measure from below. Let σ s := e s/t rel Var πp s 1 A (contraction lemma). Consider } G = G s(a) := {g : s s, P s g(a) π(a) 4σ s. Want precision 4σ s = ɛ = s := t rel log(4/ɛ). Claim π(g) 1/2. (5) Proof: Set f s := P s (1 A π(a)). Then G c {fs > 4 f s 2}. Apply Starr s inequality. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 21 / 34

48 Main idea Claim: Proof: Recall G := G s(a, m) := t mix(2ɛ) hit 1/2 (ɛ) + t rel log(4/ɛ). { } g : s s, P s g(a) π(a) ɛ, s := t rel log(4/ɛ) Set t := hit 1/2 (ɛ). By prev. claim π(g) 1/2 = P x[t G > t] ɛ (by def. of t). Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 22 / 34

49 Main idea Claim: Proof: Recall G := G s(a, m) := t mix(2ɛ) hit 1/2 (ɛ) + t rel log(4/ɛ). { } g : s s, P s g(a) π(a) ɛ, s := t rel log(4/ɛ) Set t := hit 1/2 (ɛ). By prev. claim π(g) 1/2 = P x[t G > t] ɛ (by def. of t). For any x, A: P t+s x (A) π(a) P x[t G > t] + max g G, s s P s g(a) π(a) 2ɛ. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 22 / 34

50 Trees Let: T := (V, E) be a finite tree. (V, P, π) a lazy MC corresponding to some (lazy) weighted nearest-neighbor walk on T (i.e. P (x, y) > 0 iff {x, y} E or y = x). Fact: (Kolmogorov s cycle condition) every MC on a tree is reversible. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 23 / 34

51 Trees Can the tree structure be used to determine the identity of the worst sets? Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 24 / 34

52 Trees Can the tree structure be used to determine the identity of the worst sets? Easier question: what set of π measure 1/2 is the hardest to hit in a birth & death chain with state space [n] := {1, 2,..., n}? Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 24 / 34

53 Trees Can the tree structure be used to determine the identity of the worst sets? Easier question: what set of π measure 1/2 is the hardest to hit in a birth & death chain with state space [n] := {1, 2,..., n}? Answer: take a state m with π([m]) 1/2 and π([m 1]) < 1/2. Then the set worst set would be either [m] or [n] \ [m 1]. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 24 / 34

54 Trees How to generalize this to trees? Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 25 / 34

55 Central vertex C2 o (Ci) 1/2 for all i C1 C3 C4 Figure : A vertex o V is called a central-vertex if each connected component of T \ {o} has stationary probability at most 1/2. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 26 / 34

56 Trees There is always a central-vertex (and at most 2). We fix one, denote it by o and call it the root. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 27 / 34

57 Trees There is always a central-vertex (and at most 2). We fix one, denote it by o and call it the root. It follows from our analysis that for trees the Prod. Cond. holds iff T o is concentrated (from worst leaf). A counterintuitive result = such unweighed trees (Peres-Sousi (13)). Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 27 / 34

58 Trees o Let A be s.t. (A) 1/2. Partition V to B and D=V\B s.t. B is connected, o is in B and (A ) 1/4, where A :=(D U {o}) A. Po[TA>s] Po[TA >s] P B[TA >s] 2P [TA >s], where B is conditioned on B. Take s:=ctrel log(ε). =>Po[TA>s] ε. => hit1/2(a+ε) min{t:px[to>t] a, for all x} + s. trivially: min{t:px[to>t] a, for all x} hit1/2(a) Figure : Hitting the worst set is roughly like hitting o. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 28 / 34

59 Trees Cutoff would follow if we show that T o is concentrated (under the Prod. Cond.). More precisely, we need to show that E x[t o] = Ω(t mix) = T yβ (x) is concentrated if X 0 = x. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 29 / 34

60 Trees Cutoff would follow if we show that T o is concentrated (under the Prod. Cond.). More precisely, we need to show that E x[t o] = Ω(t mix) = T yβ (x) is concentrated if X 0 = x. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 29 / 34

61 Trees o=vk vk-1 v3 v2 v1 x=v0 Figure : Let v 0 = x,v 1,..., v k = o be the vertices along the path from x to o. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 30 / 34

62 Trees Proof of Concentration: Var x[t o] Ct rel t mix: It suffices to show that Var x[t o] 4t rel E x[t o]. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 31 / 34

63 Trees Proof of Concentration: Var x[t o] Ct rel t mix: It suffices to show that Var x[t o] 4t rel E x[t o]. If X 0 = x then T o is the sum of crossing times of the edges along the path between x: τ i := T vi T vi 1 d = T vi underx 0 = v i 1 Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 31 / 34

64 Trees Proof of Concentration: Var x[t o] Ct rel t mix: It suffices to show that Var x[t o] 4t rel E x[t o]. If X 0 = x then T o is the sum of crossing times of the edges along the path between x: τ i := T vi T vi 1 d = T vi underx 0 = v i 1 τ 1,..., τ k are independent = it suffices to bound the sum of their 2nd moments Var x[t o] = Var x[τ i] = Var vi 1 [T vi ] E vi 1 [T 2 v i ]. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 31 / 34

65 Trees Proof of Concentration: Var x[t o] Ct rel t mix: It suffices to show that Var x[t o] 4t rel E x[t o]. If X 0 = x then T o is the sum of crossing times of the edges along the path between x: τ i := T vi T vi 1 d = T vi underx 0 = v i 1 τ 1,..., τ k are independent = it suffices to bound the sum of their 2nd moments Var x[t o] = Var x[τ i] = Var vi 1 [T vi ] E vi 1 [T 2 v i ]. Denote the subtree rooted at v (the set of vertices whose path to o goes through v) by W v. For A Ω let π A be π conditioned on A. Kac formula implies that for any A, there exists a dist. µ on the external vertex boundary of A s.t. E µ[t 2 A] 2E µ[t A]E πa c [T A] = By the tree structure E vi 1 [T 2 v i ] 2E vi 1 [T vi ]E πwvi 1 [T vi ]. Not hard to show E πwvi 1 [T vi ] 2t rel (generally π(a c )E πa c [T A] t rel ) so Evi 1 [T 2 v i ] 4t rel E vi 1 [Tv i ] = 4t rele x[t o]. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 31 / 34

66 Beyond trees The tree assumption can be relaxed. In particular, we can treat jumps to vertices of bounded distance on a tree (i.e. the length of the path from u to v in the tree (which is now just an auxiliary structure) is > r = P (u, v) = 0) under some mild necessary assumption. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 32 / 34

67 Beyond trees The tree assumption can be relaxed. In particular, we can treat jumps to vertices of bounded distance on a tree (i.e. the length of the path from u to v in the tree (which is now just an auxiliary structure) is > r = P (u, v) = 0) under some mild necessary assumption. Previously the BD assumption could not be relaxed mainly due to it being exploited via a representation of hitting times result for BD chains. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 32 / 34

68 Beyond trees The tree assumption can be relaxed. In particular, we can treat jumps to vertices of bounded distance on a tree (i.e. the length of the path from u to v in the tree (which is now just an auxiliary structure) is > r = P (u, v) = 0) under some mild necessary assumption. Previously the BD assumption could not be relaxed mainly due to it being exploited via a representation of hitting times result for BD chains. In particular, if P (u, v) δ > 0 for all u, v s.t. d T (u, v) r (and otherwise P (u, v) = 0), then cutoff the Prod. Cond. holds. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 32 / 34

69 Last remark: Previously good expansion of small sets can improve mixing. Now know - considering expansion only of small sets and t rel suffices to bound t mix! t mix(ɛ) hit 1 ɛ/4 (3ɛ/4) + 3t rel log (4/ɛ). 2 From which it follows that t mix 5 max Ex[TA] + 3t rel log (4/ɛ). x,a:π(a) 1 ɛ/4 2 For any x and A with π(a) 1 ɛ/4 we can bound E x[t A] using the expansion profile of sets only of π measure at most ɛ/4 (by an integral of the form used to bound the mixing time via the expansion profile). Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 33 / 34

70 Last remark: Previously good expansion of small sets can improve mixing. Now know - considering expansion only of small sets and t rel suffices to bound t mix! t mix(ɛ) hit 1 ɛ/4 (3ɛ/4) + 3t rel log (4/ɛ). 2 From which it follows that t mix 5 max Ex[TA] + 3t rel log (4/ɛ). x,a:π(a) 1 ɛ/4 2 For any x and A with π(a) 1 ɛ/4 we can bound E x[t A] using the expansion profile of sets only of π measure at most ɛ/4 (by an integral of the form used to bound the mixing time via the expansion profile). In practice, we can take ɛ = exp[ ct mix/t rel ] to determine t mix up to a constant. Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 33 / 34

71 Open problems What can be said about the geometry of the worst sets in some interesting particular cases (say, transitivity or monotonicity)? Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 34 / 34

72 Open problems What can be said about the geometry of the worst sets in some interesting particular cases (say, transitivity or monotonicity)? When can the worst sets be described as { f 2 C} (P f 2 = λ 2f 2)? (would imply several new cutoff results if true in certain cases) When can one relate escaping time from balls of π-measure ɛ to escaping time from sets of π-measure ɛ 100 /100? When can monotonicity w.r.t. a partial order (preserved by the chain) be used to describe the worst sets and their hitting time distributions? Joint work with Riddhi Basu and Jonathan Hermon Characterization of cutoff for reversible Markov chains 34 / 34