Improved FPTAS for Multi-Spin Systems

Transcription

1 Improved FPTAS for Multi-Spin Systems Pinyan Lu and Yitong Yin 2 Microsoft Research Asia, China. pinyanl@microsoft.com 2 State Key Laboratory for Novel Software Technology, Nanjing University, China. yinyt@nju.edu.cn Abstract. We design deterministic fully polynomial-time approximation scheme (FPTAS) for computing the partition function for a class of multi-spin systems, extending the known approximable regime by an exponential scale. As a consequence, we have an FPTAS for the Potts models with inverse temperature β up to a critical threshold β = O( ) where is the maximum degree, confirming a conjecture in [0]. We also give an improved FPTAS for a generalization of counting q-colorings, namely the counting list-colorings. As a consequence we have an FP- TAS for counting q-colorings in graphs with maximum degree when q α + for α greater than α This is so far the best bound achieved by deterministic approximation algorithms for counting q-colorings. All these improvements are obtained by applying a potential analysis to the correlation decay on computation trees for multi-spin systems. Introduction Spin systems in Statistical Physics are the stochastic models defined by local interactions. In Computer Science, spin systems are used as a theoretical framework for counting or inference problems arising from constraint satisfaction problems, e.g. counting independent sets or q-colorings in graphs, and probability inference in graphical models. A central problem in this framework is the computation of the partition function, which may solve both counting and inference. The problem is #P-hard for almost all nontrivial spin systems [3,4]. A classic approach for approximation of partition function is the Markov Chain Monte Carlo (MCMC) method which relies on the rapid mixing of random walks in the configuration space [5 7, 3 8, 2, 27]. A more contemporary approach is the correlation decay technique introduced by Bandyopadhyay and Gamarnik [] and Weitz [28], which leads to deterministic fully polynomial-time approximation scheme (FPTAS) for #Phard counting problems [2, 0, 9, 20, 22, 23, 29]. In these algorithms, the computation of a marginal probability (which is equivalent to the computation of partition function by self-reduction) is reduced Supported by the National Science Foundation of China under Grant No , and

2 to evaluating an exponential-size tree-structured dynamical system. The correlation decay property guarantees that the far-away variables can be disregarded without substantially affecting the marginal probability of interest, thus the true values can be efficiently approximated by evaluating truncated dynamical systems. Two such dynamical systems were proposed: () the self-avoiding-walk (SAW) tree [28] for two-state spin systems and (2) the computation tree [0] for all spin systems. For two-state spin systems, FPTAS based on SAW-trees may approach the approximability boundaries, such as [9, 20, 23, 28]. This is because a SAW-tree is a faithful construction of the original spin system on a tree, hence long-range correlations can be used as gadgets in the reduction for the inapproximability [8, 25, 26]. Very recently, the similar long-range correlations were used to prove inapproximability for multi-spin systems [9]. On the algorithmic side, due to a barrier result of Sly in [24], the original spin systems on trees are no longer capable of simulating all marginal probabilities. The computation tree introduced by Gamarnik and Katz in [0] overcomes this fundamental issue by creating a dynamical system consisting of different instances of spin systems.. Our results We design efficient computation trees for multi-spin systems: For a vertex of degree d, our computation tree expends to d branches while the previous one in [0] has exp(ω(d)) many branches. We apply a potential analysis to the decay of correlation between variables in computation trees. The potential analysis has been used in [9,20,22,23] for analyzing the correlation decay on the self-avoiding walk trees for two-state spin systems. We show for the first time that this powerful technique can be applied to computation trees for multi-spin systems. Our new construction of efficient computation trees and potential analysis greatly extend the regimes of correlation decay and deterministic FPTAS for these systems. One of the most well-studied multi-spin systems is the Potts model. Theorem. For any constant q 2, there exists an FPTAS for computing the partition function for q-state Potts models with inverse temperature β and maximum degree satisfying 3 (e β ). For large, the condition 3 (e β ) is translated ( to that β = O( ), which greatly improves the best previous bound β = O due to Gamarnik q ) and Katz [0] and also confirms a conjecture in [0]. For the anti-ferromagnetic case (β < 0), our condition is asymptotically tight due to a very recent inapproximability result of Galanis, Štefankovič, and Vigoda [9]. Theorem is a special case of a much more general theorem for the q-state spin systems, also called the Markov random fields. As suggested by [0], the regime of correlation decay for these models is described in terms of c A, the maximum ratio between edge parameters. We show that there exists an FPTAS for a family of Markov random fields if 3 (c A ). This exponentially improves the previous best known condition (c A c A ) q < proved in [0]. This general result is formally stated as Theorem 3 in Section 2.

3 We next study the problem of counting proper q-colorings in an undirected graph. For this problem, the mixing or the tractability condition is usually given in form of q α + β for some constant α and β where is the maximum degree of the graph. The previous best bound for deterministic FPTAS was achieved in [0] for an α and some sufficiently large β on triangle-free graphs. Better bounds (with α < 2) were known for randomized approximation algorithms [6, 5, 27] or correlation decay only [, 2]. We prove the following theorem for a constant α which is formally defined by (2) in Section 2. Theorem 2. There exists an FPTAS for counting q-colorings on graphs with maximum degree if q and are constants and q α + for α > α. This is a new record for the deterministic FPTAS for counting q-colorings on general graphs, and we remove the triangle-free requirement in previous correlationdecay based results such as [0, 2]. Theorem 2 is proved as a special case of a theorem for a generalization of q-colorings, called the list-colorings, which is formally stated as Theorem 4 in Section 2. All the above FPTAS require the degree of the graph and the number of states (colors) to be constant. If we remove this restriction, the algorithms compute a ( ± ɛ)-approximation of the true value for any fixed 0 < ɛ < in time n O(log n). This complexity bound was only known previously for simple models like list-colorings but was not known for general multi-spin systems since for such systems the previous computation tree proposed in [0] tries to enumerate all configurations of the local neighborhood at each step. We give a more efficient computation tree which uses exponentially less branches. 2 Definitions and Statements of Results An instance of a q-state spin system or a pair-wise Markov random field (MRF) is a tuple Ω = (G, X, A, F ), where G = (V, E) is an undirected graph called the underlying graph; X = [q] = {, 2,..., q} is a domain of spin states; A = (A e, e E) is a tuple where each A e : X X R 0 is a symmetric function specifying the activity of edge e; F = (F v, v V ) is a tuple where each F v : X R 0 specifies the external field at vertex v. The size of an MRF instance is defined as Ω = max{ V, X }. We consider only those MRF instances such that the number of bits used to encode A and F is in polynomial of n = V and q = X. This does not affect the generality of the problem since we are interested in the approximation algorithms. The partition function of an MRF instance Ω = (G, X, A, F ) is defined as Z(Ω) x X V e=uv E A e (x u, x v ) v V F v (x v ).

4 This gives rise to a probability distribution P Ω, called the Gibbs measure, over all configurations x X V, such that e=uv E P Ω (X = x) = A e(x u, x v ) v V F v(x v ). Z(Ω) Given an MRF instance Ω = (G, X, A, F ) with underlying graph G = (V, E), we denote by G the maximum degree of G and let c A max e E w,x,y,z X A e (x, y) A e (w, z). Theorem 3. Let M be a family of MRF instances with bounded degree and bounded size of domain. There exists an FPTAS for computing the partition function of MRFs in M if it holds that Ω = (G, X, A, F ) M, 3 G (c A ). () For any family M of MRFs satisfying () without any restriction on the degree or the size of domain, the algorithm computes a ( ± ɛ)-approximation of Z(Ω) for any fixed 0 < ɛ < in time n O(log n) where n = Ω. The q-state Potts model is a special class of MRFs Ω = (G, X, A, F ) with for every e E, A e = A such that A(x, y) = e β if x = y and A(x, y) = otherwise. The parameter β is called the inverse temperature. It is easy to see that Theorem is a special case of Theorem 3 on Potts models. Next we consider the proper q-colorings in an undirected graph, which can be easily seen as a special case of MRF. The problem of counting q-colorings is solved by solving its generalization called the list-colorings. A list-coloring instance is a tuple Ω = (G, X, L) with that G = (V, E) is an undirected graph; X = [q] is a domain of q colors; L = (L v, v V ) such that each L v X is a list of colors for vertex v. A proper coloring in a list-coloring instance is a proper q-coloring x X V of vertices such that x v L v for every v V. The list-coloring is a special case of MRFs (G, X, A, F ) with that for every e E, A e = A such that A(x, y) = 0 if x = y and A(x, y) = if otherwise, and for every v V, the external field F v is a Boolean function indicating the color list L v. For the list-colorings we have c A =, thus Theorem 3 does not apply, so we use different algorithm and analysis to prove the following theorem. Let α be the solution to the equation α ( ) 4α 3 3 2α + 4α 3 exp =. (2) 2α(α ) 4(α ) 3 The LHS of (2) is in fact monotonously decreasing from + to 0 for α >, so there is a unique solution α.

5 Theorem 4. There exists a deterministic FPTAS for counting proper colorings in list-coloring instances Ω = (G, X, L) with bounded degree G and bounded number of colors q = X satisfying that there is an α > α such that v V, L v α G +. (3) Obviously Theorem 2 is a special case of Theorem 4 as colorings are just listclorings with L v = X for every v V. 3 Markov Random Fields Given an MRF instance defined on the underlying graph G = (V, E), we suppose that for each vertex v V, the neighbors of v are enumerated as v, v 2,..., v deg(v) where deg(v) is the degree of v. We define operations called pinning and partial pinning on MRF instances as follows. Definition. Given an MRF instance Ω = (G, X, A, F ), a vertex v V and its neighbors v, v 2,..., v d in G, where d = deg(v), for each spin state x X and each i d +, the partial pinning of Ω, denoted as Ω i v,x, is a new MRF instance augmented from Ω as Ω i v,x = (G v, X, Ã, F ), where G v = G \ {v} is the subgraph of G induced by V \ {v}, Ã = (A e, e E \ {vv, vv 2,..., vv d }) is the restriction of A on the set of edges in G v, and F = ( F u, u V \ {v}) where y X, Fu (y) = { A uv (x, y)f u (y) if u {v,..., v i }, F u (y) otherwise. The pinning of Ω is a partial pinning by choosing i = d +, which is denoted as Ω v,x = Pin v,x (Ω) = Ω d+ v,x. The following identity can be seen as a generalization of the recursion for list-colorings derived in [0]. Compared to the recursion for MRFs in [0], it uses substantially less variables. Proposition. Let Ω = (G, X, A, F ) be an MRF instance. For every vertex v V G and its neighbors v, v 2,..., v d where d = deg(v), and every spin state x X, it holds that P Ω (X v = x) = Proof. We define that ( Fv(x) d i= A vvi (x,x) ) z x(a vvi (x,x) A vvi (x,z))p Ω i (X vi =z) v,x y X Fv(y) ( d i= A vvi (y,y) z y(a vvi (y,y) A vvi (y,z))p Ω i (X vi =z) v,y Z Ω (X v = x) x X V xv =x uw E A uw (x u, x w ) u V F u (x u ). ).

6 It can be verified that Z Ω (X v = x) = F v (x)z(ω v,x ) where Ω v,x = Pin v,x (Ω) is the pinning of Ω. Then P Ω (X v = x) = Z Ω (X v = x) y X Z Ω(X v = y) = F v (x)z(ω v,x) y X F v(y)z(ω v,y ). (4) By the Definition, it holds that Ω v,x = Ω d+ v,x, and Ω v,x is simply the MRF instance deleting vertex v, which is independent of the choice of x. Therefore, (4) = F v (x)z(ωv,x d+ )/Z(Ωv,x) y X F v(y)z(ωv,y d+ )/Z(Ωv,y) = d F v (x) i= y X F v(y) d i= The partition function of a partial pinning of Ω expands as: Z(Ω i v,x) = x X V \{v} uw E u v w v A uw (x u, x w ) u V \{v} j= Z(Ω i+ v,x ) Z(Ω i v,x ). (5) Z(Ωv,y i+ ) Z(Ωv,y i ) i F u (x u ) A vvj (x, x vj ). It can be verified that Z(Ωv,x i+ ) = z X A vv i (x, z) Z Ω i v,x (X vi = z). Therefore, (5) = F v (x) d i= z X A vv i (x, z) ZΩ i (X vi =z) v,x Z(Ωv,x i ) y X F v(y) d i= z X A vv i (y, z) ZΩ i (X vi =z) v,y Z(Ωv,y i ) F v (x) d i= = y X F v(y) d ( = Fv(x) d i= y X Fv(y) ( d i= z X A vv i (x, z) P Ω i v,x (X vi = z) z X A vv i (y, z) P Ω i v,y (X vi = z) i= A vvi (x,x) ) z x(a vvi (x,x) A vvi (x,z))p Ω i (X vi =z) v,x A vvi (y,y) z y(a vvi (y,y) A vvi (y,z))p Ω i v,y (X vi =z) where the last equation uses the fact that z X P Ω i v,y (X v i = z) =. ), 3. Algorithms based on the computation tree recursion Given an MRF instance Ω = (G, X, A, F ) on underlying graph G = (V, E), a vertex v V with d neighbors v, v 2,..., v d in G and a spin state x X, we define the following function: f Ω,v,x (p) Fv(x) d i=(a vvi (x,x) z x(a vvi (x,x) A vvi (x,z))p i,x,z) y X Fv(y) d i=(a vvi (y,y) z y(a vvi (y,y) A vvi (y,z))p i,y,z) (6) over the domain of vectors p = (p i,y,z, i d; y, z X ; y z) [0, ] dq(q ) satisfying that z y p i,y,z for every i d and y X. Due to Proposition we have P Ω (X v = x) = f Ω,v,x (p) where p i,y,z = P Ω i v,y (X vi = z) for each i d and y, z X that y z. This already gives us a procedure, called

7 the computation tree recursion, for computing the exact value of a marginal probability P Ω (X v = x). Note that it terminates since each partial pinning Ωv,y i deletes a vertex v from the current underlying graph. It is easy to verify the following closure property of the computation tree recursion: If each p i,y,z is replaced by an estimation P Ω i v,y (X vi = z) of marginal P Ω i v,y (X vi = z) such that P Ω i v,y (X vi = z) [0, ] and z y P Ω i v,y (X vi = z) then the outcome of the recursion P Ω (X v = x) = f Ω,v,x (p) as an estimation of P Ω (X v = x) still satisfies that P Ω (X v = x) [0, ] and P x X Ω (X v = x) =. The size of the computation tree can be easily of exponential in the size of the underlying graph. We can run the computation tree recursion up to t levels and use a naive estimation of marginals for the base cases. Formally, for t 0, the quantity P Ω (X v = x) is recursively defined as follows: If t = 0, let P (0) Ω (X v = x) = P F v(x) y X Fv(y). If t > 0, let Ω (X v = x) = f Ω,v,x ( p) where ˆp i,y,z = i d and y, z X that y z. (t ) P Ω (X v,y i vi = z) for each (0) The value of the base case P Ω (X v = x) is not important due to a correlation decay property we prove later. As shown in [0], on graphs of constant maximum degrees, the quantity P Ω (X v = x) can be efficiently computed by dynamic programming when t = O(log n). The partition function can be approximated from estimations of marginals by the following standard procedure. Enumerate the vertices in V as v, v 2,..., v n.. Let Ω = Ω. For k =, 2,..., n, assuming that the is well-defined, use the computation tree recursion to compute P (X vk = x) for all x X, choose x k to be the x which maximizes the P (X vk = x) and construct + = Pin vk,x k ( ) as a pinning of. 2. Compute that Ẑ(Ω) = e=uv E Ae(xu,xv) v V Fv(xv) and return Ẑ(Ω). n k= P (X vk =x vk ) This algorithm is the same as the one proposed in [0], except for using a simplified computation tree recursion, thus by the same analysis as in [0], we have the following proposition. Proposition 2. Let Ω = (G, X, A, F ) be an MRF instance such that G has maximum degree and q = X. The value of Ẑ(Ω) can be computed in time poly( Ω ) (q ) O. 3.2 Correlation decay on the computation tree The above algorithm approximates the marginal probabilities by simulating a tree-structured dynamical system for a limited number of iterations. The accuracy of this approximation relies on the following property of correlation decay.

8 Definition 2 (Correlation Decay). Let M be a family of MRFs. We say that the computation tree recursion exhibits exponential correlation decay over M if there exists a constant C > 0 such that given any MRF instance Ω M, for all t, it holds that max v V Ω x X P Ω (X v = x) P Ω (X v = x) poly( Ω ) exp( C t). A sufficient condition for the exponential correlation decay is that the error of estimation decays by a constant factor in every iteration. However, in general, the systems exhibiting correlation decay may not necessarily decay in every step. This issue has been addressed by a potential-based analysis in [9, 20, 22, 23] for self-avoiding walk trees for 2-spin systems, which is now formalized as the following condition for computation trees for multi-spin systems. Definition 3 (The Amortized Decay Condition). Let M be a family of q- state MRFs. We say that M satisfies the Amortized Decay Condition if there exists a strictly increasing differentiable function ϕ : [0, ] R satisfying the following conditions:. Let Φ(x) = d ϕ(x) d x denote the derivative of function ϕ. We call Φ( ) the potential function. The values of Φ( ) and Φ( ) are bounded by poly(q) over domain [0, ]. 2. Given an MRF instance Ω M, a vertex v V Ω with d = deg(v) and a spin state x X, let f = f Ω,v,x be the computation tree recursion defined by (6), and define the amortized decay rate as κ(p) i d y z f(p) p i,y,z Φ(f(p)) Φ(p i,y,z ). (7) There exists a constant 0 < κ <, such that for every MRF instance Ω M, vertex v V Ω and spin state x X, it holds that κ(p) κ for all p = (p i,y,z, i d y, z X y z) [0, ] dq(q ) satisfying that z y p i,y,z for all i and y. We may replace the first condition by a more sophisticated bound on the values of Φ( ) and Φ( ), which will give us more freedom to choose potential functions, although the current simple bound is sufficient for our analysis. We say a family M of MRF instances is closed under partial pinning if for every Ω = (G, X, A, F ) M, every vertex v V G with d = deg(v), spin state x X and i d, it holds for the partial pinning Ωv,x i of Ω that Ωv,x i M. Lemma. Let M be a family of MRFs which is closed under partial pinning. If M satisfies the amortized decay condition then the computation tree recursion exhibits exponential correlation decay over M.

9 Proof. Pick an MRF instance Ω M, a vertex v V Ω with d neighbors v, v 2,..., v d and a spin state x X. Let ϕ : [0, ] R be the monotone differentiable function and Φ( ) be its derivative, as required by the amortized decay condition. Consider the corresponding recursion f = f Ω,v,x. We define the following notations: Let p = P Ω (X v = x), ˆp = P Ω (X v = x), and for every i d and y, z X that y z, let p i,y,z = P Ω i v,y (X vi = z) and (t ) ˆp i,y,z = P Ω (X v,y i vi = z). Obviously, we have p = f(p) and ˆp = f( p). We also denote that ξ = ϕ(p), ˆξ = ϕ(ˆp), ξ i,y,z = ϕ(p i,y,z ) and ˆξ i,y,z = ϕ(ˆp i,y,z ), respectively. Let ɛ = p ˆp = f(p) f(ˆp), δ = ϕ(p) ϕ(ˆp) = ϕ(f(p)) ϕ(f(ˆp)), ɛ i,y,z = p i,y,z ˆp i,y,z, and δ i,y,z = ϕ(p i,y,z ) ϕ(ˆp i,y,z ) be the respective errors. We have δ = ξ ˆξ = ϕ(f(p)) ϕ(f( p)) = ( ( ϕ f ϕ (q) )) ϕ ( f ( ϕ ( q) )). Due to the Mean Value Theorem, there exist ξ i,y,z [0, ] and accordingly p i,y,z = ϕ ( ξ i,y,z ), i d, y, z X, y z, such that δ = f( p) Φ(f( p)) p i,y,z Φ( p i,y,z ) δ i,y,z κ( p) max δ i,y,z, i d y z i d y z where κ(p) is defined by (7). Since M satisfies the amortized decay condition, there exists a universal constant κ < such that κ( p) κ. And since M is closed under partial pinning, every Ωv,y i still belongs to M. Therefore, by induction we have that δ κ t δ 0, where δ 0 = ϕ(p 0 ) ϕ(ˆp 0 ) such that p 0 = P Ω (X u = w) and (0) ˆp 0 = P Ω (X u = w) for some Ω M, u V Ω and x X, where Ω is an MRF instance resulting from applying t partial pinnings on the original Ω. By the Mean Value Theorem, there exists a p 0 [0, ] such that δ 0 = ϕ(p 0 ) ϕ(ˆp 0 ) Φ( p 0 ), which is upper bounded by q c for some constant c due to the requirement of amortized decay condition, thus δ κ t δ 0 q c κ t. Recall that δ = ϕ(p) ϕ(ˆp). Also by the Mean Value Theorem there exists p [0, ] such that δ = ϕ(p) ϕ(ˆp) = Φ( p) p ˆp = Φ( p) ɛ, thus ɛ = δ Φ( p) qc δ. Altogether we have that P Ω (X v = x) P Ω (X v = x) = ɛ q c δ q c κ t δ 0 q 2c κ t. And this holds for every Ω M, v V Ω, x X and t, with the universal constants c and κ <, which implies the exponential correlation decay of computation tree recursion over M. The following lemma is proved by verifying the amortized decay condition. Lemma 2. Let M be a family of MRFs satisfying (). The computation tree recursion exhibits exponential correlation decay over M. Proof. Let M be the closure of M under partial pinning, thus every instance from M is either an instance Ω M or an outcome of successive partial pinnings

10 of it, and the family M is closed under partial pinning. We show that M satisfies the amortized decay condition. We choose a monotone function ϕ : [0, ] R ( ). so that its derivative Φ satisfies that Φ(p) = p + 00q Thus both Φ( ) and Φ( ) are bounded by polynomial of q over [0, ]. Let Ω = (G, X, A, F ) M be an MRF instance on an underlying graph G with maximum degree, v V G a vertex with d = deg(v), and x X a spin state. Let f = f Ω,v,x be the recursion defined by (6). We define some shorthand notations. For each i d and y, z X that y z, let a i,y,z = Avv i (y,z) A vvi (y,y) and b y = F v (y) d i= A vv i (y, y), and denote that s i,y = z y a d i,y,z p i,y,z, s y = b y i= s i,y, and s = y X s y. Then we have ( ) z x a i,x,z p i,x,z f(p) = d b x i= y X b d y i= ( z y a i,y,z p i,y,z ) = s x s. For p = (p i,y,z, i d; y, z X ; y z) [0, ] dq(q ) such that z y p i,y,z for all i and y, it holds that s i,y 0 for any i and y, and f(p) [0, ]. The partial derivatives satisfy: f(p) p i,x,z = a i,x,z s x (s s x ) s 2 s i,x = f(p)( f(p)) a i,x,z, s i,x f(p) p i,y,z = y x y x a i,y,z s x s y s 2 s i,y = f(p) s y s y x The amortized decay rate defined by (7) is then bounded as κ(p) = f(p) Φ(f(p)) Φ(p i,y,z ) i d y z p i,y,z = f(p)( f(p))φ(f(p)) d i= + f(p)φ(f(p)) s y d s y x i= ( a i,x,z p i,y,z + s i,x 00q i d z x 0 50 max a i,y,z. i d s z y i,y a i,x,z s i,x Φ(p i,x,z ) z x s i,y z y ) + max y x a i,y,z Φ(p i,y,z ) i d z y d i= a i,y,z s i,y. ( a i,y,z p i,y,z + ) s i,y 00q For every i d and y, z X that y z, it can be verified that s i,y = p i,y,y + A vvi (y,z) z y A vvi (y,y) p i,y,z c A, and a i,y,z = A { } vv i (y, z) A vvi (y, y) max ca, c A c A. c A

11 Note that the partial pinning does not affect the edge activity A, thus for M satisfying (), for every Ω M the c A still satisfies that 3 (c A ). Therefore, κ(p) 0 50 c A(c A ) 0 50 ( + 3 ) <. Therefore, the MRF family M satisfies the amortized decay condition. By Lemma, the computation tree recursion exhibits exponential correlation decay over M thus also over its subfamily M. Proof of Theorem 3: Let Ω = (G, X, A, F ) M be an MRF instance and G = (V, E). Enumerate the vertices in V as v, v 2,..., v n. For each k n, let P (X vk = x k ) be computed by the algorithm in Section 3., where Ω = Ω and + = Pin vk,x k ( ). It is easy to verify that still satisfies the condition () for every k since pinning increases neither nor c A. Let w(x) Ẑ(Ω) = n k= P where w(x) = Ω (X vk =x vk ) e=uv E A e(x u, x v ) v V F v(x v ). It k holds that n P Ω (X = x) = P Ω (X vk = x k i < k, X vi = x i ). k= As observed in [0], the marginal probability P Ω (X vk = x k i < k, X vi = x i ) = P Ωk (X vk = x k ). Since satisfies the condition (), by Lemma 2, there exists constant C > 0 such that PΩk (X = x vk vk ) P (X vk = x vk ) poly( Ω ) exp( C t). Thus by choosing appropriate t = O ( log ɛ + log q + log n), it holds for every k that PΩk (X = x vk vk ) P (X vk = x vk ) ɛ 4qn, and since in the algorithm we always choose the x vk maximizing the value of P (X vk = x vk ), we have P (X vk = x vk ) q thus P (X vk = x vk ) q ɛ 4qn 2q. By definition we have P Ω (X = x) = w(x) w(x) Z(Ω), thus Z(Ω) = n k= P (X vk =x vk ). Therefore, we have ɛ ( ɛ ) n Z(Ω) 2n Ẑ(Ω) = n k= P (X vk = x vk ) ( P Ωk (X vk = x vk ) + ɛ ) n + ɛ, 2n which is simplified as that ɛ Ẑ(Ω) Z(Ω) + ɛ. By Proposition 2, the total running time is bounded by poly( Ω )(q ) O. Since t = O ( log ɛ + log q + log n), the algorithm is an FPTAS if q and are constants, and in general the running time is bounded by Ω O(log Ω ) for any fixed 0 < ɛ <.

12 4 List-coloring We consider list-coloring instances Ω = (G, X, L) satisfying the condition (3). Let = G be the maximum degree of G and define that = (α ) +. The condition (3) implies the following weaker condition: v V, L v deg(v). (8) A merit of considering this weaker condition is that it is closed under partial pinning and pinning. The pinning and partial pinning can be defined on listcoloring instances as they are special cases of MRFs. Given a list-coloring instance Ω = (G, X, L) with underlying graph G = (V, E) and a vertex v V with d neighbors v, v 2,..., v d, for each color x L v, the pinning of Ω is a new list-coloring instance Ω v,x = Pin v,x (Ω) = (G v, X, L), where G v is the subgraph of G induced by V \ {v} and L = ( L u, u V \ {v}) such that L u = L u \ {x} if u is adjacent to v and L u = L u if otherwise; and for each i d +, the partial pinning of Ω is a new list-coloring instance Ω i v,x = (G v, X, L), where L = ( L u, u V \ {v}) such that L u = L u \ {x} for u = v j with j < i and L u = L u for all other u in V \ {v}. The pinning and the partial pinning does not violate the condition (8) since it never increases the maximum degree, and if L v decreases by then also deg(v) decreases by. The following identity for marginals of list-coloring is proved in [0]. Proposition 3. Let Ω = (G, X, L) be a list-coloring instance on graph G = (V, E), v V a vertex with d neighbors v, v 2,..., v d where d = deg(v), and x L v a color. It holds that P Ω (X v = x) = ( ) d i= P Ω i v,x (X vi = x) ( ). d y L v i= P Ω i v,x (X vi = y) Some simple lower and upper bounds hold for the marginals, similar to the ones proved in [0]. Lemma 3. Let Ω = (G, X, L) be a list-coloring instance with the maximum degree of G, satisfying the condition (8). For any vertex v V G and any color x L v, it holds for the marginal probability that P q e α Ω (X v = x). Proof. The upper bound is easy: conditioning on any coloring of the neighbos of v, the number of remaining colors for v is at least L v deg(v), thus marginal probability is at most. Applying the upper bound to the marginals in the numerator of the recursion in Proposition 3 and the trivial upper bound q to the denominator, we have the lower bound. q e α

13 4. The computation tree recursion with adjustment Given a list-coloring instance Ω = (G, X, L) on graph G = (V, E), a vertex v V with d neighbors v, v 2,..., v d and a color x L v, the computation tree recursion f Ω,v,x can be defined on the domain of all p = (p i,y, i d y L v ) [0, ] d Lv : d i= f Ω,v,x (p) ( p i,x) d y L v i= ( p i,y). (9) For t 0, the quantity If t = 0, let If t > 0, let that ˆp i,y = P Ω (X v = x) is recursively defined as follows: P (0) Ω (X v = x) = { P Ω (X v = x) = min L, f v d Ω,v,x( p) (t ) P Ω (X v,x i vi = y) for each i d and y L v. L v. }, where the p is taken as Note that the only difference from the MRF case is the truncation of the value of f( p) so that P Ω (X v = x) never goes beyond the naive upper bound L. We v d call this procedure the computation tree recursion with adjustment. It is the same as the procedure proposed in [0] except with a more simplified value truncation. The estimation Ẑ(Ω) of the partition function is computed from these estimations P Ω (X v = x) of marginal probabilities by the same algorithm as in Section 3.. The same complexity bound as in Proposition 2 still holds. 4.2 Correlation Decay The correlation decay of the computation tree recursion with adjustment can be defined in the same way as in Definition 2. Lemma 4. The computation tree recursion with adjustment exhibits exponential correlation decay on list-coloring instances satisfying the condition (8) with α > α where α is defined by (2) in Section 2. Proof. Let Ω = (G, X, L) be a list-coloring instance on the underlying graph G = (V, E) with the maximum degree = (G) satisfying the condition (8). It can be verified that all the list-coloring instances generated by recursively applying partial pinnings on Ω still satisfy the condition (8). Let v V be a vertex with d neighbors v, v 2,..., v d, x L v a color, and f = f Ω,v,x the recursion defined by (9). It holds that P Ω (X v = x) = f(p) where p = (p i,y, i d y L v ) and each p i,y = P Ω i v,x (X vi =y). We choose the monotone differentiable function ϕ : [0, ] R so that its derivative is Φ(p) = d ϕ(p) d p = ( p) p. We define the amortized decay rate in the same way as (7) by: κ(p) f(p) Φ(f(p)) p i,y Φ(p i,y ). i d y Lv

14 By the same analysis as in Lemma, due to the mean value theorem, we have ( P ϕ (P Ω (X v = x)) ϕ Ω (X v = x)) ( ) ( P(t ) κ(p) max ϕ P i d Ω i v,x (X vi = y) ϕ Ω (X v,x i vi = y)), y Lv for some p = (p i,y, i d y L v ) such that the value of each p i,y is between (t ) P Ω i v,x (X vi = y) and P Ω (X v,x i vi = y). By Lemma 3, we have P Ω (X vi = y) and due to the definition of the algorithm, P Ω (X v i = y) L v d, thus p i,y for any i d and y L v. By our choice of Φ( ), it can be verified that d κ(p) = f(p) Φ(f(p)) p i,x Φ(p i,x ) + f(p) Φ(f(p)) p i,y Φ(p i,y ) i= ( d f(p) pi,x + i= i d y Lv \{x} d i= ) max pi,y y x ( d d i= ( p i,x) ( ) pi,x d + (d + ) i= where the last inequality is due to that p i,x ( d ) d ) d, (0) and L v d +. Let p = i= ( p i,x). Then p since all p i,x satisfy so, and d i= ( p i,x) = ( p) d. Let l i = ln( p i,x ), thus d i= l i = d ln( p). The function g(x) = e x is concave over x 0, thus by Jensen s inequality, d pi,x = i= d g(l i ) d g i= ( d ) d l i = d p. Therefore, (0) can be bounded by its symmetrized form as follows: ( ) d ( ) 2 d p κ(p) κ( p) p + d + ( + p ) where the last inequality is due to that p Let p = i= ( ) 2 p +. and d. α(α ) exp ρ for ρ [0, ]. It holds that κ( p) ( ρ+) ( ) ρ 2(α ), whose maximum is achieved when ρ = 2 (2α 4α 3), such that ( ) 2 + 2α 4α 3 3 2α + 4α 3 κ( p) κ α exp. 2α(α ) 4(α )

15 It can be verified that κ α is is monotonously decreasing from + to 0 for α >, so κ α < if α > α where α is the unique solution to κ α =, as defined by (2). Since the condition (8) is closed under partial pinning, by induction we have ( P ϕ (P Ω (X v = x)) ϕ Ω (X v = x)) κ t ϕ ( P(0) (PΩ (X u = z)) ϕ Ω (X u = z)), where Ω = (G, X, L ) is a list-coloring instance resulting from recursively applying t partial pinnings on the original Ω. By the same mean value theorem argument as in Lemma, we have P Ω (X v = x) P Ω (X v = x) Φ( p0) Φ( p) κt α, for (0) some p [0, ] and some p 0 between P Ω (X u = w) and P Ω (X u = w) = L. u Recall that the condition (8) is closed under partial pinning. It holds that q L v χ( (G )), and by Lemma 3 it also holds that P q e /(α ) Ω (X u = [ ] w) χ( (G )). Therefore, p 0. By our choice of Φ(p), we have Φ( p0) Φ( p) q e 2(α ) χ( (G )) = O( q)., q e /(α ) χ( (G )) In conclusion, if the condition (8) is satisfied with α > α , there exists a constant κ < such that P Ω (X v = x) P Ω (X v = x) O( q)κ t. Proof of Theorem 4: We first prove the theorem under the weaker condition (8), which is closed under pinning and partial pinning. The proof is the same as the proof of Theorem 3. The theorem with the stronger condition (3) follows as a consequence. References. A. Bandyopadhyay and D. Gamarnik. Counting without sampling: Asymptotics of the log-partition function for certain statistical physics models. Random Structures & Algorithms, 33(4): , M. Bayati, D. Gamarnik, D. Katz, C. Nair, and P. Tetali. Simple deterministic approximation algorithms for counting matchings. In Proceedings of STOC, pages 22 27, J.-Y. Cai and X. Chen. A decidable dichotomy theorem on directed graph homomorphisms with non-negative weights. In Proceedings FOCS, pages , J.-Y. Cai, X. Chen, and P. Lu. Graph homomorphisms with complex values: A dichotomy theorem. In Proceedings of ICALP, pages , M. Dyer, M. Jerrum, and E. Vigoda. Rapidly mixing markov chains for dismantleable constraint graphs. In RANDOM, pages M. E. Dyer, A. M. Frieze, T. P. Hayes, and E. Vigoda. Randomly coloring constant degree graphs. In Proceedings of FOCS, pages , M. E. Dyer and C. S. Greenhill. On markov chains for independent sets. Journal of Algorithms, 35():7 49, 2000.

16 8. A. Galanis, D. Štefankovič, and E. Vigoda. Inapproximability of the partition function for the antiferromagnetic ising and hard-core models. Arxiv preprint arxiv: , A. Galanis, D. Štefankovič, and E. Vigoda. Inapproximability for antiferromagnetic spin systems in the tree non-uniqueness region. arxiv preprint arxiv: , D. Gamarnik and D. Katz. Correlation decay and deterministic FPTAS for counting colorings of a graph. Journal of Discrete Algorithms, 2:29 47, D. Gamarnik, D. Katz, and S. Misra. Strong spatial mixing for list coloring of graphs. arxiv preprint arxiv: , L. Goldberg, R. Martin, and M. Paterson. Strong spatial mixing with fewer colors for lattice graphs. SIAM Journal on Computing, 35(2):486, L. A. Goldberg and M. Jerrum. A polynomial-time algorithm for estimating the partition function of the ferromagnetic ising model on a regular matroid. In Proceedings of ICALP, pages , L. A. Goldberg, M. Jerrum, and M. Paterson. The computational complexity of two-state spin systems. Random Structures & Algorithms, 23(2):33 54, T. P. Hayes. Randomly coloring graphs of girth at least five. In Proceedings of STOC, pages , M. Jerrum. A very simple algorithm for estimating the number of k-colorings of a low-degree graph. Random Structures & Algorithms, 7(2):57 66, M. Jerrum and A. Sinclair. Polynomial-time approximation algorithms for the ising model. SIAM Journal on Computing, 22(5):087 6, M. Jerrum, A. Sinclair, and E. Vigoda. A polynomial-time approximation algorithm for the permanent of a matrix with nonnegative entries. Journal of the ACM, 5:67 697, July L. Li, P. Lu, and Y. Yin. Approximate counting via correlation decay in spin systems. In Proceedings of SODA, pages , L. Li, P. Lu, and Y. Yin. Correlation decay up to uniqueness in spin systems. In Proceedings of SODA, pages 67 84, M. Luby and E. Vigoda. Approximately counting up to four (extended abstract). In Proceedings of STOC, pages , R. Restrepo, J. Shin, P. Tetali, E. Vigoda, and L. Yang. Improved mixing condition on the grid for counting and sampling independent sets. In Proceedings of FOCS, pages 40 49, A. Sinclair, P. Srivastava, and M. Thurley. Approximation algorithms for twostate anti-ferromagnetic spin systems on bounded degree graphs. In Proceedings of SODA, pages , A. Sly. Uniqueness thresholds on trees versus graphs. The Annals of Applied Probability, 8(5): , A. Sly. Computational transition at the uniqueness threshold. In Proceedings of FOCS, pages , A. Sly and N. Sun. The computational hardness of counting in two-spin models on d-regular graphs. In Proceedings of FOCS, pages , E. Vigoda. Improved bounds for sampling coloring. In Proceedings of FOCS, pages 5 59, D. Weitz. Counting independent sets up to the tree threshold. In Proceedings of STOC, pages 40 49, Y. Yin and C. Zhang. Approximate counting via correlation decay on planar graphs. In Proceedings of SODA, pages 47 66, 203.