13 : Variational Inference: Loopy Belief Propagation and Mean Field

Transcription

1 10-708: Probabilistic Graphical Models , Spring : Variational Inference: Loopy Belief Propagation and Mean Field Lecturer: Eric P. Xing Scribes: Peter Schulam and William Wang 1 Introduction Inference problems involve answering a query that concerns the likelihood of observed data. For example, to answer the query on a marginal p(x A ), we can perform the marginalization operation to derive C/A p(x). Or, for queries concern the conditionals, such as p(x A x B ), we can first compute the joint, and divide by the marginals p(x B ). Sometimes to answer a query, we might also need to compute the mode of density ˆx = arg max x X m p(x). So far, in the class, we have covered the exact inference problem. To perform exact inference, we know that brute force search might be too inefficient for large graphs with complex structures, so a family of message passing algorithm such as forward-backward, sum-product, max-product, and junction tree, was introduced. However, although we know that these message-passing based exact inference algorithms work well for treestructured graphical models, it was also shown in the class that they might not yield consistent results for loopy graphs, or, the convergence might not be guaranteed. Also, for complex graphical models such as the Ising model, we cannot run exact inference algorithm such as the junction tree algorithm, because it is computationally intractable. In this lecture, we look at two variational inference algorithms: loopy belief propagation (yww) and mean field approximation (pschulam). 2 Loopy Belief Propagation The general idea for loopy belief propagation is that even though we know graph contains loops and the messages might circulate indefinitely, we still let it run anyway and hope for the best. In this section, we first review the basic belief propagation algorithms. Then, we discuss an experimental study by Murphy et al. (1999), and show some empirical results on the effects of loopy belief propagation. Most importantly, we start from the notion of KL divergence, and show how to explain the LBP algorithm from the perspective of minimizing the Bethe free energy. 2.1 Belief Propagation: a Quick Review The basic idea of belief propagation is very simple: to update the belief on a node, we just need to calculate the doubleton potentials from its neighboring nodes, and multiply with the target node s singleton potential. To give a more concrete example, let s consider the example on Figure 1. On the part (a) of the figure, we see that in order to compute the message M i j (x j ), we will need to calculate the message from all the neighboring nodes x k to. Then, multiply with the singletons and doubletons concerning and, x j : M i j (x j ) Φ ij (, x j )Φ i ( ) k M k i ( ) (1) 1

2 2 13 : Variational Inference: Loopy Belief Propagation and Mean Field Figure 1: Belief propagation: an example. Here the doubleton potential Φ ij (, x j ) is also called compatibilities, and is used to model the interactions of the two nodes, whereas the singleton potential Φ i ( ) is also called the external evidence. On the right-hand side (part b), we can simple update the belief of using the similar formulation: b i ( ) Φ i ( ) k M k (x k ) (2) Similarly, for factor graphs, we can also have the notion of messages and update the belief of node by multiplying its factor and the messages coming from neighboring nodes: b i ( ) f i ( ) m a i ( ) (3) a N(i) If we want to calculate message from node X a to node, we should sum up all the products: m a i ( ) f a (X a ) m j a (x j ) (4) X a\ j N(a)\i In the class, we know that running BP on trees always converges to the exact solution. However, it is not always the case for loopy graphs. The problem is that when the message is sent into a loop structure, it might circulate indefinitely, so it does not guarantee the convergence or it might converge to the wrong solution. 2.2 Loopy Belief Propagation Algorithm The idea of loopy belief propagation algorithm is to use a fixed point iteration procedure to minimize the Bethe free energy. Basically, if the convergence criteria is not met, we can update the messages and the believes: b i ( ) m a i ( ) (5) a N(i) b a (X a ) f a (X a ) m new i a( ) = c N(i)\a i N(a) m new a i( ) = f a (X a ) X a\ m i a ( ) (6) m c i ( ) (7) j N(a)\i m j a (x j ) (8)

3 13 : Variational Inference: Loopy Belief Propagation and Mean Field 3 Therefore, we know that stationary properties are guaranteed when it converges. However, the big problem here is that the convergence is not guaranteed, and the reason is intuitive: when BP algorithm is running on graphs that include loops, the messages might be circulating in the loops forever. Interestingly, Murphy et al. (1999 UAI) has studied the empirical behaviors of the loopy belief propagation algorithm, and found that LBP can still achieve good approximations: The program is stopped after a fixed number of iterations. Stop when there is no significant difference in belief updates. When the solution converges, it is usually a good approximation. And this is probably the reason why LBP is still a very popular inference algorithm, even though convergence might be guaranteed. Also, it was mentioned in class that, in order to test the empirical performance of an approximate inference algorithm on large intractable problems, one can always start simple by testing on a small example of the problem (e.g. a 20 x 20 nodes Ising model). 2.3 Understanding LBP: a F Bethe Minimization Perspective To understand the LBP algorithm, let s first define the true distribution P as: P (X) = 1 f a (X a ) (9) Z f a F where Z is the partition function, and we are interested in the product of factors. Since this is often intractable, we can approximate the distribution P with Q. To do this, we can utilize the KL-divergence method: KL(Q P ) = X Q(X) log Q(X) P (X) Note that the KL divergence is asymmetric. The value from KL should be non-negative, and has the minimum value when P = Q. KL is very useful in our problem, because it means that we can know use KL to approximate Q. To do this, we can compute KL(Q P ) without performing inference: KL(Q P ) = X Q(X) log Q(X) P (X) (10) (11) = X Q(X) log Q(X) X Q(X) log P (X) (12) = H Q (X) E Q log P (X) (13) If we replace P (X) with our earlier definition on the true probability, we can get: ( 1 ) KL(Q P ) = H Q (X) E Q log f a (X a ) (14) Z f a F = H Q (X) log 1 Z E Q log f a (X a ) (15) Note that if we re-arrange the terms on the right side of the equation, we can get KL(Q P ) = H Q (X) E Q log f a (X a ) + log Z (16) f a F f a F

4 4 13 : Variational Inference: Loopy Belief Propagation and Mean Field And the Physicists define the first two terms on the right side of equation as (Gibbs) free energy F (P, Q). So, now, our goal can be boiled down to compute the F (P, Q). In order to do this, we know that f E a F Q log f a (X a ) can be computed by summing up all the marginals, where as computing H Q (X) is a much harder task that needs to sum over all possible values, which is very expensive. However, we can always approximate F (P, Q) by computing ˆF (P, Q). Before we show how to approximate the Gibbs free energy, let s first consider the case with tree graphical models in Fig. 2 : Here, we know the probability can Figure 2: Calculating the tree energy: an example. be written as: b(x) = a b a(x a ) i b i( ) 1 di, and the H tree and F tree can be written as: H tree = b a (x a ) log b a (x a ) + (d i 1) b i ( ) log b i ( ) (17) a x a i F tree = b a (x a ) log b a(x a ) f a x a (x a ) + (1 d i ) b i ( ) log b i ( ) (18) a i = F 12 + F F 67 + F 78 F 1 F 5 F 2 F 6 F 3 F 7 (19) It can be seen that from the above derivation, we only need to sum over the singletons and doubletons, which is easy to compute. Similarly, we can also use the above idea to approximate the Gibbs free energy. For example, in a general graph, such as Fig. 3, we also have: Figure 3: Calculting the loopy graph Bethe energy: an example. H Bethe = b a (x a ) log b a (x a ) + (d i 1) b i ( ) log b i ( ) a x a i (20) F Bethe = b a (x a ) log b a(x a ) f a x a (x a ) + (1 d i ) b i ( ) log b i ( ) = f a (x a ) H Bethe a i (21) = F 12 + F F 67 + F 78 F 1 F 5 2F 2 2F 6 F 8 (22)

5 13 : Variational Inference: Loopy Belief Propagation and Mean Field 5 So, this is called the Bethe approximation of the Gibbs free energy. The idea is simple: we just need to sum over the singletons and the doubletons to derive the entropy. However, we need to notice that this approximation might not be well connected to the Gibbs free energy. Now, to minimize the Bethe free energy, we can write out the objective with Lagrangian dual form: l = F Bethe + i γ i {1 b i ( )} + a i N(a) λ ai ( ) {b i ( ) } b a (X a ) X a\ l Now to solve this, we need to take the partial derivatives and set them into zeros ( b = 0 and l i() b = 0). a(x a) Then, we have: ( 1 b i ( ) exp d i 1 a N(i) ) λ ai ( ) ( b a (X a ) exp E a (X a ) + i N(a) ) λ ai ( ) Interestingly, if we let λ ai ( ) = log(m i a ( )) = log b N(i) a m b i( ), and use b a i ( ) = X a\ b a (X a ), then we can obtain exactly the same BP formulations in equations 3 and 4. This is very attractive, because we have shown how to derive the message passing algorithm from the perspective of minimizing the Bethe free energy. So, in general, the variational methods can be summarize as: { } q = arg min F Bethe (p, q) q S where q is a now a tractable problem. Note that here we do not want to optimize q(x) directly. Instead, we want to focus on a relaxed feasible set and approximate the objective: { } b = arg min E F (b) b M o b where b covers the edge potentials (doubletons) and the node potentials (singletons). To solve for b, we typically use a fixed point iteration algorithm. (23) (24) (25) (26) (27) 3 Mean Field Approximation Recall that the purpose of approximate inference methods is to allow us to compute the posterior distribution over a model s latent variables even when the posterior involves an intractable integral or summation. As a motivating example, we will look at a Bayesian mixture of Gaussians with known variance b 2. To review, a mixture of K Gaussians has the following generative story: θ Dir(α) µ k N (0, a 2 ) for k {1,..., K} For i {1,..., n} z i Mult(θ) N (µ zi, b 2 )

6 6 13 : Variational Inference: Loopy Belief Propagation and Mean Field Suppose that we wanted to compute a posterior distribution over the cluster assignments z i and cluster mean vectors µ k. This would require us to compute the following quantity where µ = {µ 1,..., µ K }, z = {z 1,..., z n }, and x = {x 1,..., x n } p(µ, z x) = µ K k=1 p(µ k) n i=1 p(z i)p( z i, µ) K z k=1 p(µ k) n i=1 p(z i)p( z i, µ) (28) We can easily compute the numerator, but the denominator will be intractable because it involves a summation over all configuration of the latent cluster variables z. If there are K clusters, then the number of configurations that we would need to sum over is K n. This is difficult by itself, and, in order to compute the denominator, we also need compute the integral over all mean vectors µ. In the above posterior distribution, the denominator is difficult to compute because the latent variables are not easily factored. Note, in particular, that they are coupled in the conditional density of a particular data point. In general, when the latent variables are coupled, we must sum an exponentially large number of terms in order to compute the normalizing quantity in a posterior distribution. If, however, the latent variables could be easily factored, then we might be able to more easily compute the normalizing term in the posterior distribution. Broadly, mean field variational inference is a technique used to design a new family of distributions over the latent variables that do factorize well, and can then be used to approximate the posterior distribution over the Gaussian mixture model parameters and latent variables shown above. In symbols, the mean field approximation assumes that the variational distribution over the latent variables factorizes q(z 1,..., z m ) = m q(z i ; ν i ) (29) More generally, we do not need to assume that the joint distribution over the latent variables factorizes into a separate term for each variable. We can include more broad families of variational distributions by instead assuming that the joint factorizes into independent distributions over clusters of the latent variables: i=1 q(z 1,..., z m ) = q(c i ; ν i ) (30) C i C Where C is some set of disjoint sets of the latent variables. 3.1 Variational Inference Objective Functions Since we are approximating a distribution with our variational distribution q, a natural way to measure the quality of our approximation is using the Kullback-Leibler (KL) divergence between the true density p and our approximation q: KL(p q) = x p(x) log p(x) q(x) (31)

7 13 : Variational Inference: Loopy Belief Propagation and Mean Field 7 This metric, however, is a problem since it requires pointwise evaluation of p(x), which is the problem we are trying to solve in the first place. An alternative is to reverse the directionality of the KL divergence: KL(q p) = x q(x) log q(x) p(x) (32) Assuming that our approximation q(x) is tractable to compute, this metric is a slight improvement, but still involves evaluating p(x) in the denominator of the log. Note, however, that the unnormalized measure p(x) can be written as p(x)z where Z is the normalizing factor of the distribution. When p(x) is a posterior p(x D), then the normalizing constant Z is p(d). Using this fact, we can define a new objective function J(q): J(q) = KL(q p) (33) = x = x = x = x q(x) log q(x) p(x) q(x) log q(x) p(x)z q(x) log q(x) p(x) x (34) (35) q(x) log Z (36) q(x) log q(x) log Z (37) p(x) (38) Since log Z = log p(d) is a constant, minimizing J(q) = KL(q p) is equivalent to minimizing an upper bound on the negative log likelihood of the evidence by minimizing the KL divergence between our approximation and the true distribution p(x). An alternative objective is to maximize J(q), which is known as the energy functional. 3.2 Interpretations of the Objective Function We can rewrite our objective function J(q) as J(q) = E q [log q(x)] + E q [ log p(x)] (39) = H(q) + E q [E(x)] (40) Where E(x) = log p(x) is the energy. Intuitively, since we are minimizing J(q), we can see by breaking the objective function down in this way that minimizing our objective is attempting to do two things. First, we want to minimize the negative entropy (or increase the entropy), which, as we know from the maximum entropy principle, is a good way to measure how well a distribution will generalize. That is, we do not want to make unwarranted assumptions about the distribution, and should, in general, always seek to choose the distribution that maximizes entropy. Second, we want to minimize the expected energy E q [ log p(x)].

8 8 13 : Variational Inference: Loopy Belief Propagation and Mean Field Recall that the energy is lower when probability is high, so we would like to minimize energy if we want to maximize likelihood. The second term, at an intuitive level, is making sure that our approximate distribution puts more mass on x with low energy according to p(x), and less mass on x with high energy. Another interpretation of the objective function J(q) is J(q) = E q [log q(x) log p(x)p(d x)] (41) = E q [log q(x) log p(x) log p(d x)] (42) = E q [ log p(d x)] + KL(q p) (43) We can again see that breaking down the objective function can give us an intuitive feel for what we are minimizing. We see that we are minimizing the expected negative log likelihood of the data conditioned on x, which prefers distributions that put more probability mass on x that increase the likelihood of our observed data. In addition, there is a term that penalizes distributions q that are too far from p. 3.3 Optimizing the Variational Distribution Now that we understand the objective functions that we want to minimize (maximize in the case of the energy functional), we can address the issue of actually finding the variational distribution that minimizes (or maximizes) the objective function. In what follows, we use the energy functional objective function: J(q) = x q(x) log p(x) q(x) (44) Recall that we wish to minimize this function. We will also use the most simple approximating distribution that assumes that the joint density over all hidden variables x can factor completely. That is m q(x 1,..., x m ) = q i ( ; ν i ) (45) i=1 Our strategy will be to use coordinate ascent to maximize the J(q) with respect to each q i. By deriving results that optimize the energy functional with respect to q i, it is relatively straight forward to extend the coordinate ascent updates to optimize the parameters ν i for each local distribution q i. Let us first view J(q) as a function of one of the local distributions q i, which we will write as J(q i ). We can then rewrite the objective function

9 13 : Variational Inference: Loopy Belief Propagation and Mean Field 9 J(q i ) = q(x) log p(x) (46) q(x) x = ( m q i ( )) m log p(x) log q j (x j ) (47) =1 j=1 = q i ( ) q k (x k ) log p(x) log q j (x j ) log q i ( ) (48) x i k i j i = q i ( ) q k (x k ) log p(x) q i ( ) q k (x k ) log q j (x j ) + log q i ( ) (49) x i k i x i k i j i q i ( ) q k (x k ) log p(x) q i ( ) q k (x k ) log q i ( ) (50) x i k i x i k i = q i ( ) q k (x k ) log p(x) q i ( ) log q i ( ) (51) x i k i = q i ( )E qi [log p(x)] q i ( ) log q i ( ) (52) With this modified form, we can now define E qi [log p(x)] to be the log of some function of : log f i ( ) = E qi [log p(x)] (53) which allows us to rewrite the final expression in our derivation above as: q i ( ) log f i ( ) q i ( ) log q i ( ) = KL(q i f i ) (54) We then maximize our objective J(q i ) by minimizing KL(q i f i ), which is clearly done by setting q i ( ) = f i ( ). Thus log f i ( ) = E qi [log p(x)] (55) f i ( ) = exp (E qi [log p(x)]) (56) Therefore, the distribution for each q i that maximizes our objective function is q i ( ) = 1 Z i exp (E qi [log p(x)]) (57) where Z i is some normalizing constant to ensure that q i is a proper distribution. From this we can see that the approximate distribution over a particular hidden variable depends on the mean values of the rest of

10 10 13 : Variational Inference: Loopy Belief Propagation and Mean Field the hidden variables. In the expectation, we can drop all terms that do not involve, which will remove the means of all variables that are not neighbors of. We see then that the distribution for a variable depends on the mean values of its neighbors. This is known as the mean field, which is where the name mean field approximation comes from.