Phase transitions in discrete structures

Size: px
Start display at page:

Download "Phase transitions in discrete structures"

Transcription

1 Phase transitions in discrete structures Amin Coja-Oghlan Goethe University Frankfurt

2 Overview 1 The physics approach. [following Mézard, Montanari 09] Basics. Replica symmetry ( Belief Propagation ). The 1RSB cavity method. 2 Classical rigorous work. The second moment method. Quiet planting. 3 A physics-inspired rigorous approach. The Kauzmann transition. The free entropy in the 1RSB phase. 4 Random k-sat. A rigorous Belief Propagation-based approach. Amin Coja-Oghlan (Frankfurt) Random k-sat 2 / 40

3 The Boltzmann distribution Let X be a finite set of spins and let N be a large integer. Consider a probability distribution on X N defined by µ(x) = 1 Z M ψ a (x), a=1 with Z = M ψ a (x). x X N a=1 Assume that ψ a depends only on the components a [N] of x. Thus, ψ a (x) = ψ a (x a ). We call µ the Boltzmann distribution and Z the partition function. Amin Coja-Oghlan (Frankfurt) Random k-sat 3 / 40

4 The factor graph Suppose we are given the Boltzmann distribution µ(x) = 1 Z M ψ a (x a ). a=1 We set up the bipartite factor graph. Square vertices: the factors ψa, a = 1,..., M. Round vertices: the variables x 1,..., x M. Conncect each factor ψ a with all variables x i, i a. Amin Coja-Oghlan (Frankfurt) Random k-sat 4 / 40

5 The factor graph Example: the 1-dimensional Ising model There are N sites with spins in X = { 1, 1}. For any a {1,..., N 1} we define ψ a (x) = exp(βx a x a+1 ) (β > 0). Hence, a = {a, a + 1}. This yields µ(x) [ M a=1 exp(βx ax a+1 ) = exp β ] N 1 a=1 x ax a+1. Amin Coja-Oghlan (Frankfurt) Random k-sat 5 / 40

6 The factor graph Example: the Potts antiferromagnet Let X = [K] = {1, 2,..., K}. Let G = (V, E) be a graph on V = [N]. For an edge e joining vertices i, j in G let ψ e (x) = exp [ ] β 1 xi =x j. This gives rise to µ(x) = ψ e (x) = exp [ β #monochromatic edges]. e E Amin Coja-Oghlan (Frankfurt) Random k-sat 6 / 40

7 The factor graph Example: the Potts antiferromagnet Let X = [K] = {1, 2,..., K}. Let G = (V, E) be a graph on V = [N]. For an edge e joining vertices i, j in G let ψ e (x) = exp [ ] β 1 xi =x j. This gives rise to µ(x) = ψ e (x) = exp [ β #monochromatic edges]. e E Amin Coja-Oghlan (Frankfurt) Random k-sat 6 / 40

8 The free entropy Generally, the partition function Z scales exponentially with N. Therefore, we are interested in the free entropy 1 N ln Z. In fact, we care mostly about the theormdynamic limit N, i.e., the free entropy density. lim N 1 ln Z, N Amin Coja-Oghlan (Frankfurt) Random k-sat 7 / 40

9 The free entropy Generally, the partition function Z scales exponentially with N. Therefore, we are interested in the free entropy 1 N ln Z. In fact, we care mostly about the theormdynamic limit N, i.e., lim N 1 ln Z, N the free entropy density. Does this limit exist? If so, can we compute/approximate it? Is the limit an analytic function, or are there phase transitions? Amin Coja-Oghlan (Frankfurt) Random k-sat 7 / 40

10 The free entropy Generally, the partition function Z scales exponentially with N. Therefore, we are interested in the free entropy 1 N ln Z. In fact, we care mostly about the theormdynamic limit N, i.e., lim N 1 ln Z, N the free entropy density. Does this limit exist? If so, can we compute/approximate it? Is the limit an analytic function, or are there phase transitions? Example: the Ising model 1-dim: the free entropy density is analytic, i.e., no phase transition. 2-dim: there is a phase transition. [Onsager 1949] Amin Coja-Oghlan (Frankfurt) Random k-sat 7 / 40

11 Disordered systems Two levels of randomness The Boltzmann distribution itself is random. Hence, the free entropy density becomes [ ] 1 lim E N N ln Z. Amin Coja-Oghlan (Frankfurt) Random k-sat 8 / 40

12 Disordered systems Two levels of randomness The Boltzmann distribution itself is random. Hence, the free entropy density becomes [ ] 1 lim E N N ln Z. Example: the Edwards-Anderson model With x as in the 1-dim Ising model, we let [ ] N 1 µ(x) exp β J i x i x i+1. i=1 The J i are independent Gaussian random variables. Amin Coja-Oghlan (Frankfurt) Random k-sat 8 / 40

13 Disordered systems Example: the diluted mean-filed Potts antiferromagnet Let X = {1,..., K} for some small K 2. Consider a random graph G(N, M) on N vertices with M edges. Amin Coja-Oghlan (Frankfurt) Random k-sat 9 / 40

14 Disordered systems Example: the diluted mean-filed Potts antiferromagnet Let X = {1,..., K} for some small K 2. Consider a random graph G(N, M) on N vertices with M edges. For any edge e of G(N, M) joining vertices i, j, define ψ e (x) = exp [ ] β 1 xi =x j. Amin Coja-Oghlan (Frankfurt) Random k-sat 9 / 40

15 Disordered systems Example: the diluted mean-filed Potts antiferromagnet Let X = {1,..., K} for some small K 2. Consider a random graph G(N, M) on N vertices with M edges. For any edge e of G(N, M) joining vertices i, j, define ψ e (x) = exp [ ] β 1 xi =x j. The Boltzmann distribution µ(x) e ψ e (x) = exp [ β #monochromatic edges] is random as it depends on the graph G(N, M). Amin Coja-Oghlan (Frankfurt) Random k-sat 9 / 40

16 Disordered systems Example: the diluted mean-filed Potts antiferromagnet Let X = {1,..., K} for some small K 2. Consider a random graph G(N, M) on N vertices with M edges. For any edge e of G(N, M) joining vertices i, j, define ψ e (x) = exp [ ] β 1 xi =x j. The Boltzmann distribution µ(x) e ψ e (x) = exp [ β #monochromatic edges] is random as it depends on the graph G(N, M). New parameter: the denisty α = M/N. Amin Coja-Oghlan (Frankfurt) Random k-sat 9 / 40

17 Disordered systems Example: coloring random graphs Let X = {1,..., K} for some small K 3. For any edge e of G(N, M) joining vertices i, j, define ψ e (x) = exp [ 1 xi =x j ] = 1xi x j [ zero temperature ]. Then µ(x) e ψ e (x) = 1 no edge is monochromatic. is the uniform distribution over proper K-colorings of G(N, M). Amin Coja-Oghlan (Frankfurt) Random k-sat 10 / 40

18 Disordered systems Question [Erdős, Rényi 1960] Is there a phase transition in this model (in terms of α = M/N)? Amin Coja-Oghlan (Frankfurt) Random k-sat 10 / 40

19 Disordered systems Example: random k-sat Let X = {0, 1} and fix some small K 3. Think of x 1,..., x N as Boolean variables. Let Φ be an expression of the form (x 1 x 17 x 29 ) ( x }{{} 11 x 2 x 1 ) }{{} k-clause Φ 1 k-clause Φ 2 with M clauses, chosen uniformly at random. For i = 1,..., M let ψ i (x) = 1 clause Φi evaluates to true. Then µ(x) = M i=1 ψ i(x) is uniform over satisfying assignments. Amin Coja-Oghlan (Frankfurt) Random k-sat 11 / 40

20 Belief Propagation Goals To compute the free entropy on trees. the Belief Propagation equations. the Bethe free entropy. The replica symmetric ansatz. Belief Propagation as a distributional fixed point equation. application to diluted mean-field models. Amin Coja-Oghlan (Frankfurt) Random k-sat 12 / 40

21 Example: the 1-dimensional Ising model Let X = {±1}, x X N and µ(x) exp [ β N 1 i=1 x i x i+1 + βb Goal: to compute the marginal µ(x i ) of x i. ] N x i. i=1 Amin Coja-Oghlan (Frankfurt) Random k-sat 13 / 40

22 Example: the 1-dimensional Ising model For j [N] define two distributions ˆν j, ˆν j on X by ˆν j (x j ) ˆν j (x j ) x 1,...,x j 1 X x j+1,...,x N X [ ] j 1 j 1 exp β x i x i+1 + βb x i, exp β i=1 N 1 i=j x i x i+1 + βb i=1 N i=j+1 x i. Amin Coja-Oghlan (Frankfurt) Random k-sat 14 / 40

23 Example: the 1-dimensional Ising model For j [N] define two distributions ˆν j, ˆν j on X by ˆν j (x j ) ˆν j (x j ) x 1,...,x j 1 X x j+1,...,x N X [ ] j 1 j 1 exp β x i x i+1 + βb x i, exp β i=1 N 1 i=j x i x i+1 + βb i=1 N i=j+1 x i. Then µ(x j ) ˆν j (x j ) exp(βbx j ) ν j (x j ). Amin Coja-Oghlan (Frankfurt) Random k-sat 14 / 40

24 Example: the 1-dimensional Ising model For j [N] define two distributions ˆν j, ˆν j on X by ˆν j (x j ) ˆν j (x j ) x 1,...,x j 1 X x j+1,...,x N X [ ] j 1 j 1 exp β x i x i+1 + βb x i, exp β i=1 N 1 i=j x i x i+1 + βb i=1 N i=j+1 x i. Then µ(x j ) ˆν j (x j ) exp(βbx j ) ν j (x j ). Moreover, we have the recurrence ˆν j+1 (x j+1 ) = x j X ˆν j (x j ) exp [βx j x j+1 + βbx j ]. Amin Coja-Oghlan (Frankfurt) Random k-sat 14 / 40

25 Example: the 1-dimensional Ising model We have ˆν j+1 (x j+1 ) = x j X ν j(x j ) exp [βx j x j+1 + βbx j ]. Setting we can rephrase the above as u j = 1 2β ln ˆν j(1) ˆν j ( 1), u j+1 = f (u j + B) with f (x) = β 1 atanh(tanh(β) tanh(βx)). The function f has a unique fixed point u. Hence, for j sufficiently far from the boundary, we find µ(x j ) tanh(β(2u + B)). Amin Coja-Oghlan (Frankfurt) Random k-sat 15 / 40

26 Belief Propagation on trees Suppose that the factor graph of µ(x) is a tree. Generalising the above example, we define messages recursively by ν (t+1) j a (x j) ˆν (t) b j (x j), b j\{a} ˆν (t) b j (x j) x b\j ψ b (x b ) i b\j ν (t) i b (x i). t counts the number of recursive steps, j is the neighborhood of j, the sum ranges over X b\j, with x j is given upfront. Amin Coja-Oghlan (Frankfurt) Random k-sat 16 / 40

27 Belief Propagation on trees Suppose that the factor graph of µ(x) is a tree. Generalising the above example, we define messages recursively by ν (t+1) j a (x j) ˆν (t) b j (x j), b j\{a} ˆν (t) b j (x j) x b\j ψ b (x b ) i b\j ν (t) i b (x i). t counts the number of recursive steps, j is the neighborhood of j, the sum ranges over X b\j, with x j is given upfront. Define the Belief Propagation marginal of variable x j as ν (t) (x j ) b j ˆν (t 1) b j (x j ). Amin Coja-Oghlan (Frankfurt) Random k-sat 16 / 40

28 Belief Propagation on trees Theorem Assume that the factor graph is a tree of diameter T. 1 The BP equations have a unique fixed point ν. 2 For t > T we have ν (t) = ν, regardless of the initialisation. 3 For all variables x j we have ν (x j ) = µ(x j ), i.e., the BP marginals coincide with the Boltzmann marginals. Amin Coja-Oghlan (Frankfurt) Random k-sat 17 / 40

29 Belief Propagation on trees Theorem (the Bethe free entropy) Given the messages ν, define M N F (ν) = F a (ν) + F ia (ν), where a=1 F i (ν) i=1 (i,a) F a (ν) = ln ψ a (x a ) ν i a (x i ), x a i a F i (ν) = ln ˆν b i (x i ), x i b i F ia (ν) = ln x i ν i a (x i )ˆν a i (x i ). If the factor graph is a tree, then ln Z = F (ν ). Amin Coja-Oghlan (Frankfurt) Random k-sat 18 / 40

30 Belief Propagation on trees On trees, BP converges to its unique fixed point. This fixed point yields the correct marginals, the free entropy. Example: the 1-dim Ising model. Amin Coja-Oghlan (Frankfurt) Random k-sat 19 / 40

31 Belief Propagation on infinite trees Question Does this formalism survive the thermodynamic limit? That is, does it extend to infinite trees? Amin Coja-Oghlan (Frankfurt) Random k-sat 20 / 40

32 Belief Propagation on infinite trees Question Does this formalism survive the thermodynamic limit? That is, does it extend to infinite trees? Examples of infinite trees The infinite regular tree. Each variable/factor node has the same number of outgoing edges. The factors ψ a are identical. Infinite random trees. The numbers of outgoing edges is are independent Poisson variables. The factors ψa might have some randomness, too. Generally, we assume that sub-trees are identically distributed. Amin Coja-Oghlan (Frankfurt) Random k-sat 20 / 40

33 Belief Propagation on infinite trees The distributional BP equations We interpret the Belief Propagation equations ν j a (x j ) ˆν b j (x j ), b j\{a} ˆν b j (x j ) ψ b (x b ) ν i b (x i ). x b\j i b\j as distributional fixed point equations. That is, if ψ b, b, j are i.i.d. from the distribution defining the tree, the ν i b are i.i.d. from a distribution ν over distributions, then ν j a has distribution ν. The Bethe free entropy is determined by ν. Amin Coja-Oghlan (Frankfurt) Random k-sat 21 / 40

34 Belief Propagation on infinite trees Example: coloring regular trees Consider the graph K-coloring problem on the d-ary tree. By symmetry, all the marginals are identical, i.e., ν j a (x j ) = 1/K for all j, x j. Thus, ν is the measure concentrated on the uniform distribution. Amin Coja-Oghlan (Frankfurt) Random k-sat 22 / 40

35 Belief Propagation on infinite trees Example: random K-SAT Each variable node has a Po(α) number of children. The factors ψ a are i.i.d. random clauses of length K. The fixed point distribution ν is not known to exist, but is conjectured to be highly non-trivial and not discrete, based on numerical evidence ( population dynamics ). This mirrors the asymmetric combinatorics of the problem. Amin Coja-Oghlan (Frankfurt) Random k-sat 23 / 40

36 The replica symmetric ansatz Diluted mean-field models Think of the factor graphs in models such as random graph K-coloring, random K-SAT. The factor graph is an Erdős-Rényi-like sparse random graph. Thus, roughly speaking, the vertices are bounded degree, the shortest cycle has length Ω(ln N). In other words, locally the graph looks like the infinite random tree! Amin Coja-Oghlan (Frankfurt) Random k-sat 24 / 40

37 The replica symmetric ansatz The replica symmetric solution Let ν be the BP fixed point on the tree. Let φ be the resulting Bethe free entropy. Then the replica symmetric prediction is that E[ln Z] φ = lim N N, with Z the partition function in, e.g., random K-SAT. Amin Coja-Oghlan (Frankfurt) Random k-sat 25 / 40

38 The replica symmetric ansatz The replica symmetric solution Let ν be the BP fixed point on the tree. Let φ be the resulting Bethe free entropy. Then the replica symmetric prediction is that E[ln Z] φ = lim N N, with Z the partition function in, e.g., random K-SAT. Hypotheses The fixed point ν exists and is only one relevant fixed point. There is correlation decay. Amin Coja-Oghlan (Frankfurt) Random k-sat 25 / 40

39 The replica symmetric ansatz Example: ferromagnetic Ising on sparse random graphs [DM 08] The replica symmetric solution is correct. Paramagnetic/ferromagnetic phase transition. Amin Coja-Oghlan (Frankfurt) Random k-sat 26 / 40

40 The replica symmetric ansatz Example: coloring random graphs Let K = #colors, α = M/N density of the graph. Then Z = #K-colorings of G(N, M). According to the replica symmetric solution, 1 N E[ln Z] 1 ln E[Z] = ln K + α ln(1 1/K). N The col/uncol transition occurs at α = (K 1 2 ) ln K + o K (1). Rigorous work: [ACO, Vilenchik 13] The r.s. solution is correct only for α < (K 1 2 ) ln K ln 2 + o K (1). The col/uncol transition occurs at α (K 1 2 ) ln K o K (1). Amin Coja-Oghlan (Frankfurt) Random k-sat 27 / 40

41 Multiple Belief Propagation fixed points Random factor graphs In random graph K-coloring, the factor graph is similar to G(N, M). Thus, it s far from being a tree. In effect, there can be multiple BP fixed points. These mirror long range correlations. Amin Coja-Oghlan (Frankfurt) Random k-sat 28 / 40

42 Multiple Belief Propagation fixed points x i w The reconstruction problem Suppose µ is defined by a factor graph G. Pick a variable x i and let G ω be the depth-ω neighborhood. What is the impact of the boundary on x i? Amin Coja-Oghlan (Frankfurt) Random k-sat 29 / 40

43 Multiple Belief Propagation fixed points Correlation decay: Gibbs uniqueness Let y ω be a boundary configuration. The worst-case influence of the boundary can be cast as E[sup y ω µ (x i y ω ) µ(x i ) ]. Here the expectation is over the choice of µ. If lim ω we say there is Gibbs uniqueness. lim E[sup µ (x i y ω ) µ(x i ) ] = 0, N y ω Observe that this is a purely local property. Amin Coja-Oghlan (Frankfurt) Random k-sat 30 / 40

44 Multiple Belief Propagation fixed points Correlation decay: non-reconstruction The average influence of the boundary configuration can be cast as E y ω µ(y ω ) µ (x i y ω ) µ(x i ). If lim ω lim E µ(y ω ) µ (x i y ω ) µ(x i ) = 0, N y ω we say there is non-reconstruction. Otherwise reconstruction is possible. Amin Coja-Oghlan (Frankfurt) Random k-sat 31 / 40

45 Multiple Belief Propagation fixed points Theorem [Achlioptas, ACO 08] If the density satisfies α = M/N > ( ) ε K ln K, then in random graph K-coloring reconstruction is possible. Amin Coja-Oghlan (Frankfurt) Random k-sat 32 / 40

46 Multiple Belief Propagation fixed points Theorem [Achlioptas, ACO 08] If the density satisfies α = M/N > ( ) ε K ln K, then in random graph K-coloring reconstruction is possible. In fact, suppose we initialise BP with one random K-coloring. Then for Ω(n) variables x i the BP marginals satisfy ν (x i ) = 1. Thus, these variables are locally frozen. Interpretation: µ decomposes into a large number of clusters. Amin Coja-Oghlan (Frankfurt) Random k-sat 32 / 40

47 The 1RSB cavity method 1-step replica symmetry breaking In the above scenario, µ is a convex combination µ = N w i µ i i=1 of probability measures µ i (corresponding to clusters ). Here w i exp(f i N), with F i the free entropy of µ i. Key idea: perform Belief Propagation on the level of the µ i! The parameter β is replaced by the Parisi parameter y. Amin Coja-Oghlan (Frankfurt) Random k-sat 33 / 40

48 The 1RSB cavity method Idea A Boltzmann distribution on the µ i. This distribution is defined by P y (µ i ) w y i. In terms of the complexity function Σ(φ) = 1 N ln # {i [N ] : F i φ}, the partition function Ξ(y) of ξ y satisfies Ξ(y) = yφ + Σ(φ), with φ s.t. Σ φ = y. Amin Coja-Oghlan (Frankfurt) Random k-sat 34 / 40

49 The 1RSB cavity method Belief Propagation one level up What is the factor graph of P y? The old messages ν i a, ˆν a i are the variables. There are factor nodes enforcing the BP equations. Additional factor nodes implement the Parisi parameter y. (This is possible due to Bethe s formula.) Amin Coja-Oghlan (Frankfurt) Random k-sat 35 / 40

50 The 1RSB cavity method Belief Propagation one level up What is the factor graph of P y? The old messages ν i a, ˆν a i are the variables. There are factor nodes enforcing the BP equations. Additional factor nodes implement the Parisi parameter y. (This is possible due to Bethe s formula.) On this factor graph, write the Belief Propagation equations. 1RSB cavity equations. Complication: the spins are continuous. Amin Coja-Oghlan (Frankfurt) Random k-sat 35 / 40

51 The 1RSB cavity method Belief Propagation one level up What is the factor graph of P y? The old messages ν i a, ˆν a i are the variables. There are factor nodes enforcing the BP equations. Additional factor nodes implement the Parisi parameter y. (This is possible due to Bethe s formula.) On this factor graph, write the Belief Propagation equations. 1RSB cavity equations. Complication: the spins are continuous. The BP fixed point for P y yields Σ(φ). Analogous to the Bethe free entropy. Amin Coja-Oghlan (Frankfurt) Random k-sat 35 / 40

52 The 1RSB cavity method Belief Propagation one level up What is the factor graph of P y? The old messages ν i a, ˆν a i are the variables. There are factor nodes enforcing the BP equations. Additional factor nodes implement the Parisi parameter y. (This is possible due to Bethe s formula.) On this factor graph, write the Belief Propagation equations. 1RSB cavity equations. Complication: the spins are continuous. The BP fixed point for P y yields Σ(φ). Analogous to the Bethe free entropy. The 1RSB equations can be viewed as distributional equations. the thermodynamic limit. Setting y = 0 yields the Survey Propagation equations. Amin Coja-Oghlan (Frankfurt) Random k-sat 35 / 40

53 The 1RSB cavity method ical, connd q-col entedstatic in Talarge k [11]. 1RSB (aka condensation, Kauzmann transition). confirming r q-coloring 14], the val- l c l s [14] α d,+ α d α c α s The geometry of Fig. the 2. Boltzmann distribution Pictorial representation of the different phase transiti known The that replica solutions symmetric of aphase. rcsp. At α d,+ some clusters appear, but for α d,+ putforward Dynamic 1RSB. solutions are split among about e NΣ clusters of size e Ns.I comprise only an exponentially small fraction of solutions. Forα d the set of solutions is dominated by a few large clusters (with str weights), and above α s the problem does not admit solutions an corresponding to two distinct intuitions. According to Amin Coja-Oghlan (Frankfurt) Random k-sat 36 / 40

54 The 1RSB cavity method Example: for what d is the random d-regular graph K-colorable? Setting y = 0, we consider the distribution P 0 (µ i ) = 1 N, N = #clusters. Amin Coja-Oghlan (Frankfurt) Random k-sat 37 / 40

55 The 1RSB cavity method Example: for what d is the random d-regular graph K-colorable? Setting y = 0, we consider the distribution P 0 (µ i ) = 1 N, N = #clusters. Suppose that each µ i is characterized by frozen variables. Encode µi as a map ζ i : [N] {1, 2,..., K, }. = unfrozen ( joker color ). Amin Coja-Oghlan (Frankfurt) Random k-sat 37 / 40

56 The 1RSB cavity method Example: for what d is the random d-regular graph K-colorable? Setting y = 0, we consider the distribution P 0 (µ i ) = 1 N, N = #clusters. Suppose that each µ i is characterized by frozen variables. Encode µi as a map ζ i : [N] {1, 2,..., K, }. = unfrozen ( joker color ). This gives rise to SP messages: Qi j (x i ) =P 0 -probability that w/out j, i is frozen to color x i. Qi j (x i ) =P 0 -probability that w/out j, i is unfrozen. Amin Coja-Oghlan (Frankfurt) Random k-sat 37 / 40

57 The 1RSB cavity method Example: for what d is the random d-regular graph K-colorable? We have the fixed point equations x i\j N (x i ) k i\j Q i j (x i ) = Q k i(x k ) x i\j D k i\j Q k i(x k ) (x i [K]). D contains all (c 1,..., c d 1 ) such that [K] {c 1,..., c d 1 }. N (x) contains all (c 1,..., c d 1 ) D with [K] \ {x} {c 1,..., c d 1 }. Let Q i j ( ) = 1 x i Q i j (x i ). Amin Coja-Oghlan (Frankfurt) Random k-sat 38 / 40

58 The 1RSB cavity method Example: for what d is the random d-regular graph K-colorable? We have the fixed point equations x i\j N (x i ) k i\j Q i j (x i ) = Q k i(x k ) x i\j D k i\j Q k i(x k ) (x i [K]). D contains all (c 1,..., c d 1 ) such that [K] {c 1,..., c d 1 }. N (x) contains all (c 1,..., c d 1 ) D with [K] \ {x} {c 1,..., c d 1 }. Let Q i j ( ) = 1 x i Q i j (x i ). Simplification: it might seem reasonable to assume that Q i j (x i ) = a for some fixed 0 a 1/q and all x i [K]. Qi j ( ) = 1 Ka. The above equation becomes ( q 1 ) r=0 a = ( 1)r K 1 r (1 (r + 1)a) d 1 ( q 1 ). r=0 ( 1)r q r+1 (1 (r + 1)a) d 1 Amin Coja-Oghlan (Frankfurt) Random k-sat 38 / 40

59 The 1RSB cavity method Example: for what d is the random d-regular graph K-colorable? We obtain the complexity ( 1 N ln #clusters) [ q 1 ( ] q Σ = ln ( 1) )(1 r (r + 1)a) d d r ln(1 Ka2 ) r=0 with a the solution to a = ( q 1 ) r=0 ( 1)r K 1 r (1 (r + 1)a) d 1 ( q 1 ) r=0 ( 1)r q. r+1 (1 (r + 1)a) d 1 Now, K-colorability should occur iff Σ > 0. Asymptotically, this yields a threshold of d K col = (2K 1) ln K 1 + o K (1). Amin Coja-Oghlan (Frankfurt) Random k-sat 39 / 40

60 Conclusion The cavity method as a non-rigorous formalism. free entropy, phase transitions. There are two variants: the replica symmetric version ( Belief Propagation ). the 1RSB version ( Survey Propagation ). Can we develop a rigorous foundation? Amin Coja-Oghlan (Frankfurt) Random k-sat 40 / 40

Chasing the k-sat Threshold

Chasing the k-sat Threshold Chasing the k-sat Threshold Amin Coja-Oghlan Goethe University Frankfurt Random Discrete Structures In physics, phase transitions are studied by non-rigorous methods. New mathematical ideas are necessary

More information

The condensation phase transition in random graph coloring

The condensation phase transition in random graph coloring The condensation phase transition in random graph coloring Victor Bapst Goethe University, Frankfurt Joint work with Amin Coja-Oghlan, Samuel Hetterich, Felicia Rassmann and Dan Vilenchik arxiv:1404.5513

More information

Random Graph Coloring

Random Graph Coloring Random Graph Coloring Amin Coja-Oghlan Goethe University Frankfurt A simple question... What is the chromatic number of G(n,m)? [ER 60] n vertices m = dn/2 random edges ... that lacks a simple answer Early

More information

Predicting Phase Transitions in Hypergraph q-coloring with the Cavity Method

Predicting Phase Transitions in Hypergraph q-coloring with the Cavity Method Predicting Phase Transitions in Hypergraph q-coloring with the Cavity Method Marylou Gabrié (LPS, ENS) Varsha Dani (University of New Mexico), Guilhem Semerjian (LPT, ENS), Lenka Zdeborová (CEA, Saclay)

More information

Phase Transitions in the Coloring of Random Graphs

Phase Transitions in the Coloring of Random Graphs Phase Transitions in the Coloring of Random Graphs Lenka Zdeborová (LPTMS, Orsay) In collaboration with: F. Krząkała (ESPCI Paris) G. Semerjian (ENS Paris) A. Montanari (Standford) F. Ricci-Tersenghi (La

More information

Statistical Physics on Sparse Random Graphs: Mathematical Perspective

Statistical Physics on Sparse Random Graphs: Mathematical Perspective Statistical Physics on Sparse Random Graphs: Mathematical Perspective Amir Dembo Stanford University Northwestern, July 19, 2016 x 5 x 6 Factor model [DM10, Eqn. (1.4)] x 1 x 2 x 3 x 4 x 9 x8 x 7 x 10

More information

Phase Transitions (and their meaning) in Random Constraint Satisfaction Problems

Phase Transitions (and their meaning) in Random Constraint Satisfaction Problems International Workshop on Statistical-Mechanical Informatics 2007 Kyoto, September 17 Phase Transitions (and their meaning) in Random Constraint Satisfaction Problems Florent Krzakala In collaboration

More information

Information, Physics, and Computation

Information, Physics, and Computation Information, Physics, and Computation Marc Mezard Laboratoire de Physique Thdorique et Moales Statistiques, CNRS, and Universit y Paris Sud Andrea Montanari Department of Electrical Engineering and Department

More information

The cavity method. Vingt ans après

The cavity method. Vingt ans après The cavity method Vingt ans après Les Houches lectures 1982 Early days with Giorgio SK model E = J ij s i s j i

More information

Perturbed Message Passing for Constraint Satisfaction Problems

Perturbed Message Passing for Constraint Satisfaction Problems Journal of Machine Learning Research 16 (2015) 1249-1274 Submitted 4/13; Revised 9/14; Published 7/15 Perturbed Message Passing for Constraint Satisfaction Problems Siamak Ravanbakhsh Russell Greiner Department

More information

The Random Matching Problem

The Random Matching Problem The Random Matching Problem Enrico Maria Malatesta Universitá di Milano October 21st, 2016 Enrico Maria Malatesta (UniMi) The Random Matching Problem October 21st, 2016 1 / 15 Outline 1 Disordered Systems

More information

Reconstruction for Models on Random Graphs

Reconstruction for Models on Random Graphs 1 Stanford University, 2 ENS Paris October 21, 2007 Outline 1 The Reconstruction Problem 2 Related work and applications 3 Results The Reconstruction Problem: A Story Alice and Bob Alice, Bob and G root

More information

The Tightness of the Kesten-Stigum Reconstruction Bound for a Symmetric Model With Multiple Mutations

The Tightness of the Kesten-Stigum Reconstruction Bound for a Symmetric Model With Multiple Mutations The Tightness of the Kesten-Stigum Reconstruction Bound for a Symmetric Model With Multiple Mutations City University of New York Frontier Probability Days 2018 Joint work with Dr. Sreenivasa Rao Jammalamadaka

More information

On the number of circuits in random graphs. Guilhem Semerjian. [ joint work with Enzo Marinari and Rémi Monasson ]

On the number of circuits in random graphs. Guilhem Semerjian. [ joint work with Enzo Marinari and Rémi Monasson ] On the number of circuits in random graphs Guilhem Semerjian [ joint work with Enzo Marinari and Rémi Monasson ] [ Europhys. Lett. 73, 8 (2006) ] and [ cond-mat/0603657 ] Orsay 13-04-2006 Outline of the

More information

arxiv: v2 [cond-mat.dis-nn] 29 Feb 2008

arxiv: v2 [cond-mat.dis-nn] 29 Feb 2008 Clusters of solutions and replica symmetry breaking in random k-satisfiability Andrea Montanari Depts of Electrical Engineering and Statistics, Stanford University, USA. arxiv:080.367v [cond-mat.dis-nn]

More information

THE PHYSICS OF COUNTING AND SAMPLING ON RANDOM INSTANCES. Lenka Zdeborová

THE PHYSICS OF COUNTING AND SAMPLING ON RANDOM INSTANCES. Lenka Zdeborová THE PHYSICS OF COUNTING AND SAMPLING ON RANDOM INSTANCES Lenka Zdeborová (CEA Saclay and CNRS, France) MAIN CONTRIBUTORS TO THE PHYSICS UNDERSTANDING OF RANDOM INSTANCES Braunstein, Franz, Kabashima, Kirkpatrick,

More information

Physics on Random Graphs

Physics on Random Graphs Physics on Random Graphs Lenka Zdeborová (CNLS + T-4, LANL) in collaboration with Florent Krzakala (see MRSEC seminar), Marc Mezard, Marco Tarzia... Take Home Message Solving problems on random graphs

More information

Message passing and approximate message passing

Message passing and approximate message passing Message passing and approximate message passing Arian Maleki Columbia University 1 / 47 What is the problem? Given pdf µ(x 1, x 2,..., x n ) we are interested in arg maxx1,x 2,...,x n µ(x 1, x 2,..., x

More information

Phase Transitions in Physics and Computer Science. Cristopher Moore University of New Mexico and the Santa Fe Institute

Phase Transitions in Physics and Computer Science. Cristopher Moore University of New Mexico and the Santa Fe Institute Phase Transitions in Physics and Computer Science Cristopher Moore University of New Mexico and the Santa Fe Institute Magnetism When cold enough, Iron will stay magnetized, and even magnetize spontaneously

More information

Upper-bounding the k-colorability threshold by counting covers

Upper-bounding the k-colorability threshold by counting covers Upper-bounding the k-colorability threshold by counting covers Amin Coja-Oghlan Goethe University Mathematics Institute Frankfurt am Main, Germany acoghlan@math.uni-frankfurt.de Submitted: May 2, 2013;

More information

arxiv: v2 [cond-mat.dis-nn] 4 Dec 2008

arxiv: v2 [cond-mat.dis-nn] 4 Dec 2008 Constraint satisfaction problems with isolated solutions are hard Lenka Zdeborová Université Paris-Sud, LPTMS, UMR8626, Bât. 00, Université Paris-Sud 9405 Orsay cedex CNRS, LPTMS, UMR8626, Bât. 00, Université

More information

5. Density evolution. Density evolution 5-1

5. Density evolution. Density evolution 5-1 5. Density evolution Density evolution 5-1 Probabilistic analysis of message passing algorithms variable nodes factor nodes x1 a x i x2 a(x i ; x j ; x k ) x3 b x4 consider factor graph model G = (V ;

More information

The Condensation Phase Transition in the Regular k-sat Model

The Condensation Phase Transition in the Regular k-sat Model The Condensation Phase Transition in the Regular k-sat Model Victor Bapst 1 and Amin Coja-Oghlan 2 1 Mathematics Institute, Goethe University, Frankfurt, Germany bapst@math.uni-frankfurt.de 2 Mathematics

More information

Hard-Core Model on Random Graphs

Hard-Core Model on Random Graphs Hard-Core Model on Random Graphs Antar Bandyopadhyay Theoretical Statistics and Mathematics Unit Seminar Theoretical Statistics and Mathematics Unit Indian Statistical Institute, New Delhi Centre New Delhi,

More information

SPIN SYSTEMS: HARDNESS OF APPROXIMATE COUNTING VIA PHASE TRANSITIONS

SPIN SYSTEMS: HARDNESS OF APPROXIMATE COUNTING VIA PHASE TRANSITIONS SPIN SYSTEMS: HARDNESS OF APPROXIMATE COUNTING VIA PHASE TRANSITIONS Andreas Galanis Joint work with: Jin-Yi Cai Leslie Ann Goldberg Heng Guo Mark Jerrum Daniel Štefankovič Eric Vigoda The Power of Randomness

More information

Phase Transitions in Networks: Giant Components, Dynamic Networks, Combinatoric Solvability

Phase Transitions in Networks: Giant Components, Dynamic Networks, Combinatoric Solvability in Networks: Giant Components, Dynamic Networks, Combinatoric Solvability Department of Physics UC Davis April 27, 2009 Outline Historical Prospective Old School New School Non-Physics 1 Historical Prospective

More information

Approximate Message Passing

Approximate Message Passing Approximate Message Passing Mohammad Emtiyaz Khan CS, UBC February 8, 2012 Abstract In this note, I summarize Sections 5.1 and 5.2 of Arian Maleki s PhD thesis. 1 Notation We denote scalars by small letters

More information

A variational approach to Ising spin glasses in finite dimensions

A variational approach to Ising spin glasses in finite dimensions . Phys. A: Math. Gen. 31 1998) 4127 4140. Printed in the UK PII: S0305-447098)89176-2 A variational approach to Ising spin glasses in finite dimensions R Baviera, M Pasquini and M Serva Dipartimento di

More information

Independence and chromatic number (and random k-sat): Sparse Case. Dimitris Achlioptas Microsoft

Independence and chromatic number (and random k-sat): Sparse Case. Dimitris Achlioptas Microsoft Independence and chromatic number (and random k-sat): Sparse Case Dimitris Achlioptas Microsoft Random graphs W.h.p.: with probability that tends to 1 as n. Hamiltonian cycle Let τ 2 be the moment all

More information

Approximate counting of large subgraphs in random graphs with statistical mechanics methods

Approximate counting of large subgraphs in random graphs with statistical mechanics methods Approximate counting of large subgraphs in random graphs with statistical mechanics methods Guilhem Semerjian LPT-ENS Paris 13.03.08 / Eindhoven in collaboration with Rémi Monasson, Enzo Marinari and Valery

More information

arxiv: v1 [cs.cc] 4 Nov 2010

arxiv: v1 [cs.cc] 4 Nov 2010 Adversarial Satisfiability Problem arxiv:1011.1273v1 [cs.cc] 4 Nov 2010 Michele Castellana 1,2,3, Lenka Zdeborová 3,4 1 Dipartimento di Fisica, Università di Roma La Sapienza, 00185 Rome, Italy 2 LPTMS,

More information

Lecture 5: Random Energy Model

Lecture 5: Random Energy Model STAT 206A: Gibbs Measures Invited Speaker: Andrea Montanari Lecture 5: Random Energy Model Lecture date: September 2 Scribe: Sebastien Roch This is a guest lecture by Andrea Montanari (ENS Paris and Stanford)

More information

Gibbs Measures and Phase Transitions on Sparse Random Graphs

Gibbs Measures and Phase Transitions on Sparse Random Graphs Gibbs Measures and Phase Transitions on Sparse Random Graphs Amir Dembo, Andrea Montanari Stanford University. e-mail: amir@math.stanford.edu; montanari@stanford.edu Abstract: Many problems of interest

More information

The Sherrington-Kirkpatrick model

The Sherrington-Kirkpatrick model Stat 36 Stochastic Processes on Graphs The Sherrington-Kirkpatrick model Andrea Montanari Lecture - 4/-4/00 The Sherrington-Kirkpatrick (SK) model was introduced by David Sherrington and Scott Kirkpatrick

More information

Growth Rate of Spatially Coupled LDPC codes

Growth Rate of Spatially Coupled LDPC codes Growth Rate of Spatially Coupled LDPC codes Workshop on Spatially Coupled Codes and Related Topics at Tokyo Institute of Technology 2011/2/19 Contents 1. Factor graph, Bethe approximation and belief propagation

More information

Min-Max Message Passing and Local Consistency in Constraint Networks

Min-Max Message Passing and Local Consistency in Constraint Networks Min-Max Message Passing and Local Consistency in Constraint Networks Hong Xu, T. K. Satish Kumar, and Sven Koenig University of Southern California, Los Angeles, CA 90089, USA hongx@usc.edu tkskwork@gmail.com

More information

NETADIS Kick off meeting. Torino,3-6 february Payal Tyagi

NETADIS Kick off meeting. Torino,3-6 february Payal Tyagi NETADIS Kick off meeting Torino,3-6 february 2013 Payal Tyagi Outline My background Project: 1) Theoretical part a) Haus master equation and Hamiltonian b) First steps: - Modelling systems with disordered

More information

Reconstruction in the Generalized Stochastic Block Model

Reconstruction in the Generalized Stochastic Block Model Reconstruction in the Generalized Stochastic Block Model Marc Lelarge 1 Laurent Massoulié 2 Jiaming Xu 3 1 INRIA-ENS 2 INRIA-Microsoft Research Joint Centre 3 University of Illinois, Urbana-Champaign GDR

More information

Ising models on locally tree-like graphs

Ising models on locally tree-like graphs Ising models on locally tree-like graphs Amir Dembo and Andrea Montanari April 30, 2008 Abstract We consider Ising models on graphs that converge locally to trees. Examples include random regular graphs

More information

Message Passing Algorithms: A Success Looking for Theoreticians

Message Passing Algorithms: A Success Looking for Theoreticians Message Passing Algorithms: A Success Looking for Theoreticians Andrea Montanari Stanford University June 5, 2010 Andrea Montanari (Stanford) Message Passing June 5, 2010 1 / 93 What is this talk about?

More information

A New Look at Survey Propagation and Its Generalizations

A New Look at Survey Propagation and Its Generalizations A New Look at Survey Propagation and Its Generalizations ELITZA MANEVA, ELCHANAN MOSSEL, AND MARTIN J. WAINWRIGHT University of California Berkeley, Berkeley, California Abstract. This article provides

More information

Sparse Superposition Codes for the Gaussian Channel

Sparse Superposition Codes for the Gaussian Channel Sparse Superposition Codes for the Gaussian Channel Florent Krzakala (LPS, Ecole Normale Supérieure, France) J. Barbier (ENS) arxiv:1403.8024 presented at ISIT 14 Long version in preparation Communication

More information

XVI International Congress on Mathematical Physics

XVI International Congress on Mathematical Physics Aug 2009 XVI International Congress on Mathematical Physics Underlying geometry: finite graph G=(V,E ). Set of possible configurations: V (assignments of +/- spins to the vertices). Probability of a configuration

More information

arxiv: v2 [cond-mat.dis-nn] 23 Jul 2016

arxiv: v2 [cond-mat.dis-nn] 23 Jul 2016 On one-step replica symmetry breaking in the Edwards-Anderson spin glass model Gino Del Ferraro, 1 Chuang Wang, 2, 3 Hai-Jun Zhou, 2 and Erik Aurell1, 4, 5 1 AlbaNova University Centre, KTH-Royal Institute

More information

The non-backtracking operator

The non-backtracking operator The non-backtracking operator Florent Krzakala LPS, Ecole Normale Supérieure in collaboration with Paris: L. Zdeborova, A. Saade Rome: A. Decelle Würzburg: J. Reichardt Santa Fe: C. Moore, P. Zhang Berkeley:

More information

A Better Algorithm for Random k-sat

A Better Algorithm for Random k-sat A Better Algorithm for Random k-sat Amin Coja-Oghlan University of Edinburgh, School of Informatics, 10 Crichton Street, Edinburgh EH8 9AB acoghlan@inf.ed.ac.uk Abstract. Let Φ be a uniformly distributed

More information

Reconstruction in the Sparse Labeled Stochastic Block Model

Reconstruction in the Sparse Labeled Stochastic Block Model Reconstruction in the Sparse Labeled Stochastic Block Model Marc Lelarge 1 Laurent Massoulié 2 Jiaming Xu 3 1 INRIA-ENS 2 INRIA-Microsoft Research Joint Centre 3 University of Illinois, Urbana-Champaign

More information

Long Range Frustration in Finite Connectivity Spin Glasses: Application to the random K-satisfiability problem

Long Range Frustration in Finite Connectivity Spin Glasses: Application to the random K-satisfiability problem arxiv:cond-mat/0411079v1 [cond-mat.dis-nn] 3 Nov 2004 Long Range Frustration in Finite Connectivity Spin Glasses: Application to the random K-satisfiability problem Haijun Zhou Max-Planck-Institute of

More information

arxiv: v3 [cond-mat.dis-nn] 21 May 2008

arxiv: v3 [cond-mat.dis-nn] 21 May 2008 Zero temperature solutions of the Edwards-Anderson model in random Husimi lattices Alejandro Lage-Castellanos and Roberto Mulet Henri-Poincaré Group of Complex Systems, Physics Faculty, University of Havana,

More information

Spectra of Large Random Stochastic Matrices & Relaxation in Complex Systems

Spectra of Large Random Stochastic Matrices & Relaxation in Complex Systems Spectra of Large Random Stochastic Matrices & Relaxation in Complex Systems Reimer Kühn Disordered Systems Group Department of Mathematics, King s College London Random Graphs and Random Processes, KCL

More information

Phase Transitions in Spin Glasses

Phase Transitions in Spin Glasses p.1 Phase Transitions in Spin Glasses Peter Young http://physics.ucsc.edu/ peter/talks/bifi2008.pdf e-mail:peter@physics.ucsc.edu Work supported by the and the Hierarchical Systems Research Foundation.

More information

Performance of Sequential Local Algorithms for the Random NAE-$K$-SAT Problem

Performance of Sequential Local Algorithms for the Random NAE-$K$-SAT Problem Performance of Sequential Local Algorithms for the Random NAE-$K$-SAT Problem The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation

More information

Probabilistic Graphical Models

Probabilistic Graphical Models School of Computer Science Probabilistic Graphical Models Variational Inference II: Mean Field Method and Variational Principle Junming Yin Lecture 15, March 7, 2012 X 1 X 1 X 1 X 1 X 2 X 3 X 2 X 2 X 3

More information

1 Mechanistic and generative models of network structure

1 Mechanistic and generative models of network structure 1 Mechanistic and generative models of network structure There are many models of network structure, and these largely can be divided into two classes: mechanistic models and generative or probabilistic

More information

On the method of typical bounded differences. Lutz Warnke. Georgia Tech

On the method of typical bounded differences. Lutz Warnke. Georgia Tech On the method of typical bounded differences Lutz Warnke Georgia Tech What is this talk about? Motivation Behaviour of a function of independent random variables ξ 1,..., ξ n : X = F (ξ 1,..., ξ n ) the

More information

PHASE TRANSITIONS IN THE COMPLEXITY OF COUNTING

PHASE TRANSITIONS IN THE COMPLEXITY OF COUNTING PHASE TRANSITIONS IN THE COMPLEXITY OF COUNTING A Thesis Presented to The Academic Faculty by Andreas Galanis In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in Algorithms,

More information

Decay of Correlation in Spin Systems

Decay of Correlation in Spin Systems (An algorithmic perspective to the) Decay of Correlation in Spin Systems Yitong Yin Nanjing University Decay of Correlation hardcore model: random independent set I v Pr[v 2 I ] (d+)-regular tree `! σ:

More information

A New Look at Survey Propagation and its Generalizations

A New Look at Survey Propagation and its Generalizations A New Look at Survey Propagation and its Generalizations Elitza Maneva Elchanan Mossel Martin J. Wainwright September 7, 2004 Technical Report 669 Department of Statistics University of California, Berkeley

More information

ON STRUCTURAL AND ALGORITHMIC BOUNDS

ON STRUCTURAL AND ALGORITHMIC BOUNDS ON STRUCTURAL AND ALGORITHMIC BOUNDS IN RANDOM CONSTRAINT SATISFACTION PROBLEMS Dissertation zur Erlangung des Doktorgrades der Naturwissenschaften vorgelegt beim Fachbereich 12 der Johann Wolfgang Goethe-Universität

More information

Structured Variational Inference

Structured Variational Inference Structured Variational Inference Sargur srihari@cedar.buffalo.edu 1 Topics 1. Structured Variational Approximations 1. The Mean Field Approximation 1. The Mean Field Energy 2. Maximizing the energy functional:

More information

Lecture 18 Generalized Belief Propagation and Free Energy Approximations

Lecture 18 Generalized Belief Propagation and Free Energy Approximations Lecture 18, Generalized Belief Propagation and Free Energy Approximations 1 Lecture 18 Generalized Belief Propagation and Free Energy Approximations In this lecture we talked about graphical models and

More information

The Last Survivor: a Spin Glass Phase in an External Magnetic Field.

The Last Survivor: a Spin Glass Phase in an External Magnetic Field. The Last Survivor: a Spin Glass Phase in an External Magnetic Field. J. J. Ruiz-Lorenzo Dep. Física, Universidad de Extremadura Instituto de Biocomputación y Física de los Sistemas Complejos (UZ) http://www.eweb.unex.es/eweb/fisteor/juan

More information

Propagating beliefs in spin glass models. Abstract

Propagating beliefs in spin glass models. Abstract Propagating beliefs in spin glass models Yoshiyuki Kabashima arxiv:cond-mat/0211500v1 [cond-mat.dis-nn] 22 Nov 2002 Department of Computational Intelligence and Systems Science, Tokyo Institute of Technology,

More information

Directed Dominating Set Problem Studied by Cavity Method: Warning Propagation and Population Dynamics

Directed Dominating Set Problem Studied by Cavity Method: Warning Propagation and Population Dynamics Commun. Theor. Phys. 70 (28) 785 794 Vol. 70, No. 6, December 1, 28 Directed Dominating Set Problem Studied by Cavity Method: Warning Propagation and Population Dynamics Yusupjan Habibulla ( 玉素甫 艾比布拉 )

More information

A Tale of Two Cultures: Phase Transitions in Physics and Computer Science. Cristopher Moore University of New Mexico and the Santa Fe Institute

A Tale of Two Cultures: Phase Transitions in Physics and Computer Science. Cristopher Moore University of New Mexico and the Santa Fe Institute A Tale of Two Cultures: Phase Transitions in Physics and Computer Science Cristopher Moore University of New Mexico and the Santa Fe Institute Computational Complexity Why are some problems qualitatively

More information

MAP Examples. Sargur Srihari

MAP Examples. Sargur Srihari MAP Examples Sargur srihari@cedar.buffalo.edu 1 Potts Model CRF for OCR Topics Image segmentation based on energy minimization 2 Examples of MAP Many interesting examples of MAP inference are instances

More information

Statistical Mechanics and Algorithms on Sparse and Random Graphs

Statistical Mechanics and Algorithms on Sparse and Random Graphs Statistical Mechanics and Algorithms on Sparse and Random Graphs Andrea Montanari 1 October 8, 2014 1 A. Montanari is with the Departments of Electrical Engineering and Statistics, Stanford University

More information

13 : Variational Inference: Loopy Belief Propagation and Mean Field

13 : Variational Inference: Loopy Belief Propagation and Mean Field 10-708: Probabilistic Graphical Models 10-708, Spring 2012 13 : Variational Inference: Loopy Belief Propagation and Mean Field Lecturer: Eric P. Xing Scribes: Peter Schulam and William Wang 1 Introduction

More information

Guilhem Semerjian. Mean-field disordered systems : glasses and optimization problems, classical and quantum

Guilhem Semerjian. Mean-field disordered systems : glasses and optimization problems, classical and quantum LABORATOIRE DE PHYSIQUE THEORIQUE DE L ECOLE NORMALE SUPERIEURE THESE D HABILITATION A DIRIGER DES RECHERCHES présentée par Guilhem Semerjian Mean-field disordered systems : glasses and optimization problems,

More information

A Spectral Approach to Analysing Belief Propagation for 3-Colouring

A Spectral Approach to Analysing Belief Propagation for 3-Colouring University of Pennsylvania ScholarlyCommons Statistics Papers Wharton Faculty Research 11-2009 A Spectral Approach to Analysing Belief Propagation for 3-Colouring Amin Coja-Oghlan Elchanan Mossel University

More information

Phase transition phenomena of statistical mechanical models of the integer factorization problem (submitted to JPSJ, now in review process)

Phase transition phenomena of statistical mechanical models of the integer factorization problem (submitted to JPSJ, now in review process) Phase transition phenomena of statistical mechanical models of the integer factorization problem (submitted to JPSJ, now in review process) Chihiro Nakajima WPI-AIMR, Tohoku University Masayuki Ohzeki

More information

Spin models on random graphs

Spin models on random graphs Spin models on random graphs Sander Dommers Spin models on random graphs Sander Dommers, 2013. A catalogue record is available from the Eindhoven University of Technology Library. ISBN: 978-90-386-3348-0

More information

The typical structure of sparse K r+1 -free graphs

The typical structure of sparse K r+1 -free graphs The typical structure of sparse K r+1 -free graphs Lutz Warnke University of Cambridge (joint work with József Balogh, Robert Morris, and Wojciech Samotij) H-free graphs / Turán s theorem Definition Let

More information

Algorithmic Barriers from Phase Transitions

Algorithmic Barriers from Phase Transitions Algorithmic Barriers from Phase Transitions Dimitris Achlioptas Amin Coja-Oghlan Abstract For many random Constraint Satisfaction Problems, by now there exist asymptotically tight estimates of the largest

More information

Notes 6 : First and second moment methods

Notes 6 : First and second moment methods Notes 6 : First and second moment methods Math 733-734: Theory of Probability Lecturer: Sebastien Roch References: [Roc, Sections 2.1-2.3]. Recall: THM 6.1 (Markov s inequality) Let X be a non-negative

More information

Constraint satisfaction problems (CSPs) arise in a large spectrum

Constraint satisfaction problems (CSPs) arise in a large spectrum Gibbs states and the set of solutions of random constraint satisfaction problems Florent Krza kała a, Andrea Montanari b,c,d, Federico Ricci-Tersenghi e, Guilhem Semerjian c, and Lenka Zdeborová f a Laboratoire

More information

The glass transition as a spin glass problem

The glass transition as a spin glass problem The glass transition as a spin glass problem Mike Moore School of Physics and Astronomy, University of Manchester UBC 2007 Co-Authors: Joonhyun Yeo, Konkuk University Marco Tarzia, Saclay Mike Moore (Manchester)

More information

CS Lecture 19. Exponential Families & Expectation Propagation

CS Lecture 19. Exponential Families & Expectation Propagation CS 6347 Lecture 19 Exponential Families & Expectation Propagation Discrete State Spaces We have been focusing on the case of MRFs over discrete state spaces Probability distributions over discrete spaces

More information

Analysis of Algorithms I: Asymptotic Notation, Induction, and MergeSort

Analysis of Algorithms I: Asymptotic Notation, Induction, and MergeSort Analysis of Algorithms I: Asymptotic Notation, Induction, and MergeSort Xi Chen Columbia University We continue with two more asymptotic notation: o( ) and ω( ). Let f (n) and g(n) are functions that map

More information

Learning from and about complex energy landspaces

Learning from and about complex energy landspaces Learning from and about complex energy landspaces Lenka Zdeborová (CNLS + T-4, LANL) in collaboration with: Florent Krzakala (ParisTech) Thierry Mora (Princeton Univ.) Landscape Landscape Santa Fe Institute

More information

Szemerédi s regularity lemma revisited. Lewis Memorial Lecture March 14, Terence Tao (UCLA)

Szemerédi s regularity lemma revisited. Lewis Memorial Lecture March 14, Terence Tao (UCLA) Szemerédi s regularity lemma revisited Lewis Memorial Lecture March 14, 2008 Terence Tao (UCLA) 1 Finding models of large dense graphs Suppose we are given a large dense graph G = (V, E), where V is a

More information

Phase Transitions in Random Discrete Structures

Phase Transitions in Random Discrete Structures Institut für Optimierung und Diskrete Mathematik Phase Transition in Thermodynamics The phase transition deals with a sudden change in the properties of an asymptotically large structure by altering critical

More information

Spin glasses and Adiabatic Quantum Computing

Spin glasses and Adiabatic Quantum Computing Spin glasses and Adiabatic Quantum Computing A.P. Young alk at the Workshop on heory and Practice of Adiabatic Quantum Computers and Quantum Simulation, ICP, rieste, August 22-26, 2016 Spin Glasses he

More information

Bayesian Machine Learning - Lecture 7

Bayesian Machine Learning - Lecture 7 Bayesian Machine Learning - Lecture 7 Guido Sanguinetti Institute for Adaptive and Neural Computation School of Informatics University of Edinburgh gsanguin@inf.ed.ac.uk March 4, 2015 Today s lecture 1

More information

On convergence of Approximate Message Passing

On convergence of Approximate Message Passing On convergence of Approximate Message Passing Francesco Caltagirone (1), Florent Krzakala (2) and Lenka Zdeborova (1) (1) Institut de Physique Théorique, CEA Saclay (2) LPS, Ecole Normale Supérieure, Paris

More information

13: Variational inference II

13: Variational inference II 10-708: Probabilistic Graphical Models, Spring 2015 13: Variational inference II Lecturer: Eric P. Xing Scribes: Ronghuo Zheng, Zhiting Hu, Yuntian Deng 1 Introduction We started to talk about variational

More information

Recitation 9: Loopy BP

Recitation 9: Loopy BP Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.438 Algorithms For Inference Fall 204 Recitation 9: Loopy BP General Comments. In terms of implementation,

More information

arxiv: v1 [cond-mat.dis-nn] 18 Dec 2009

arxiv: v1 [cond-mat.dis-nn] 18 Dec 2009 arxiv:0912.3563v1 [cond-mat.dis-nn] 18 Dec 2009 Belief propagation for graph partitioning Petr Šulc 1,2,3, Lenka Zdeborová 1 1 Theoretical Division and Center for Nonlinear Studies, Los Alamos National

More information

Lecture 6: September 22

Lecture 6: September 22 CS294 Markov Chain Monte Carlo: Foundations & Applications Fall 2009 Lecture 6: September 22 Lecturer: Prof. Alistair Sinclair Scribes: Alistair Sinclair Disclaimer: These notes have not been subjected

More information

Low-Density Parity-Check Codes

Low-Density Parity-Check Codes Department of Computer Sciences Applied Algorithms Lab. July 24, 2011 Outline 1 Introduction 2 Algorithms for LDPC 3 Properties 4 Iterative Learning in Crowds 5 Algorithm 6 Results 7 Conclusion PART I

More information

Spin glasses and Stein s method

Spin glasses and Stein s method UC Berkeley Spin glasses Magnetic materials with strange behavior. ot explained by ferromagnetic models like the Ising model. Theoretically studied since the 70 s. An important example: Sherrington-Kirkpatrick

More information

CSC 412 (Lecture 4): Undirected Graphical Models

CSC 412 (Lecture 4): Undirected Graphical Models CSC 412 (Lecture 4): Undirected Graphical Models Raquel Urtasun University of Toronto Feb 2, 2016 R Urtasun (UofT) CSC 412 Feb 2, 2016 1 / 37 Today Undirected Graphical Models: Semantics of the graph:

More information

UNDERSTANDING BELIEF PROPOGATION AND ITS GENERALIZATIONS

UNDERSTANDING BELIEF PROPOGATION AND ITS GENERALIZATIONS UNDERSTANDING BELIEF PROPOGATION AND ITS GENERALIZATIONS JONATHAN YEDIDIA, WILLIAM FREEMAN, YAIR WEISS 2001 MERL TECH REPORT Kristin Branson and Ian Fasel June 11, 2003 1. Inference Inference problems

More information

Expectation Propagation in Factor Graphs: A Tutorial

Expectation Propagation in Factor Graphs: A Tutorial DRAFT: Version 0.1, 28 October 2005. Do not distribute. Expectation Propagation in Factor Graphs: A Tutorial Charles Sutton October 28, 2005 Abstract Expectation propagation is an important variational

More information

Lecture 8: February 8

Lecture 8: February 8 CS71 Randomness & Computation Spring 018 Instructor: Alistair Sinclair Lecture 8: February 8 Disclaimer: These notes have not been subjected to the usual scrutiny accorded to formal publications. They

More information

FACTOR GRAPHS AND GRAPH ENSEMBLES

FACTOR GRAPHS AND GRAPH ENSEMBLES 9 FACTOR GRAPHS AND GRAPH ENSEMBLES Systems involving a large number of simple variables with mutual dependencies (or constraints, or interactions) appear recurrently in several fields of science. It is

More information

Lecture 9: PGM Learning

Lecture 9: PGM Learning 13 Oct 2014 Intro. to Stats. Machine Learning COMP SCI 4401/7401 Table of Contents I Learning parameters in MRFs 1 Learning parameters in MRFs Inference and Learning Given parameters (of potentials) and

More information

Phase Transitions and Local Search for k-satisfiability

Phase Transitions and Local Search for k-satisfiability Phase Transitions and Local Search for k-satisfiability Pekka Orponen (joint work with many others) Department of Information and Computer Science Helsinki University of Technology TKK Outline Part I:

More information

Lecture 8: PGM Inference

Lecture 8: PGM Inference 15 September 2014 Intro. to Stats. Machine Learning COMP SCI 4401/7401 Table of Contents I 1 Variable elimination Max-product Sum-product 2 LP Relaxations QP Relaxations 3 Marginal and MAP X1 X2 X3 X4

More information

Lecture 16 Deep Neural Generative Models

Lecture 16 Deep Neural Generative Models Lecture 16 Deep Neural Generative Models CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago May 22, 2017 Approach so far: We have considered simple models and then constructed

More information