Chapter 10: Random Fields

LEARNING AND INFERENCE IN GRAPHICAL MODELS Chapter 10: Random Fields Dr. Martin Lauer University of Freiburg Machine Learning Lab Karlsruhe Institute of Technology Institute of Measurement and Control Systems Learning and Inference in Graphical Models. Chapter 10 p. 1/38

References for this chapter Christopher M. Bishop, Pattern Recognition and Machine Learning, ch. 8, Springer, 2006 Michael Ying Yang and Wolfgang Förstner, A hierarchical conditional random field model for labeling and classifying images of man-made scenes. In: IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 196-203, 2011 Learning and Inference in Graphical Models. Chapter 10 p. 2/38

Motivation Bayesian networks model clear dependencies, often causal dependencies. Bayesian networks are acyclic. How can we model mutual and cyclic dependencies? Example (economy): demand and supply determine the price high price fosters supply low price fosters demand Learning and Inference in Graphical Models. Chapter 10 p. 3/38

Motivation Example (physics): modeling ferromagnetism in statistical mechanics a grid of magnetic dipoles in a volume every dipole causes a force on its neighbors every dipole is forced by its neighbors The dipoles might change their orientation. Every configuration of the magnetic dipole field can be characterized by its energy. The probability of a certain configuration depends on its energy: high energy configurations are less probable, low energy configurations are more probable. Ising-model (Ernst Ising, 1924) Learning and Inference in Graphical Models. Chapter 10 p. 4/38

Markov random fields a Markov random field (MRF) is a undirected, connected graph each node represents a random variable open circles indicate non-observed random variables filled circles indicate observed random variables dots indicate given constants links indicate an explicitly modeled stochastic dependence B A D C Learning and Inference in Graphical Models. Chapter 10 p. 5/38

Markov random fields Joint probability distribution of a MRF is defined over cliques in the graph Definition: A clique of sizek is a subsetc ofk nodes of the MRF so that for each pairx,y C withx Y holds thatx andy are connected by an edge. Example: The MRF on the right has X 2 X 1 X 4 one clique of size 3: {X 2,X 3,X 4 } X 3 four cliques of size 2: {X 1,X 2 },{X 2,X 3 },{X 2,X 4 },{X 3,X 4 } four cliques of size 1: {X 1 },{X 2 },{X 3 },{X 4 } Learning and Inference in Graphical Models. Chapter 10 p. 6/38

Markov random fields For every cliquec in the MRF we specify a potential function ψ C : C R >0 large values ofψ C indicate that a certain configuration of the random variables in the clique is more probable small values ofψ C indicate that a certain configuration of the random variables in the clique is less probable The joint distribution of the MRF is defined as the product of the potential functions for all cliques p(x 1,...,X n ) = 1 Z C Cliques ψ C (C) withz = C Cliques ψ C(C)d(X 1,...,X n ) the partition function Remark: calculating Z might be very hard in practice Learning and Inference in Graphical Models. Chapter 10 p. 7/38

Markov random fields Potential functions are usually given in terms of Gibbs/Boltzmann distributions ψ C (C) = e E C(C) withe C : C R an energy function large energy means low probability small energy means large probability Hence, the overall probability distribution of an MRF is p(x 1,...,X n ) = 1 Z e C Cliques E C(C) Learning and Inference in Graphical Models. Chapter 10 p. 8/38

Markov random fields Example: let us model the food preferences of a group of four persons: Antonia, Ben, Charles, and Damaris. They might choose between pasta, fish, and meat Ben likes meat and pasta but hates fish Antonia, Ben, and Charles prefer to choose the same Charles is vegetarian Damaris prefers to choose something else than all the other create an MRF on the blackboard that models the food preferences of the four persons and assign potential functions to the cliques. Learning and Inference in Graphical Models. Chapter 10 p. 9/38

Markov random fields One way to model the food preference task Random variables A, B, C, D model Antonias, Bens, Charles, and Damaris choice. Discrete variables with values1=pasta,2=fish,3=meat Energy functions which are relevant (all others are constant): { 0 ifb {1,3} E {B} (b)= 100 ifb = 2 { 0 ifa = b = c E {A,B,C} (a,b,c)= E {C} (c)= 30 otherwise 0 ifc = 1 50 ifc = 2 200 ifc = 3 A B E {A,D} (a,d)= E {B,D} (b,d)= E {C,D} (c,d)= C D { 0 ifa d 10 ifa = d { 0 ifb d 10 ifb = d { 0 ifc d 10 ifc = d Learning and Inference in Graphical Models. Chapter 10 p. 10/38

Factor graphs Like for Bayesian networks we can define factor graphs over MRFs. A factor graph is a bipartite graph with two kind of nodes: variable nodes that model random variables factor nodes that model a probabilistic relationship between variable nodes. Each factor node is assigned with a potential function Variable nodes and factor nodes are connected by undirected links. For each MRF we can create a factor graph as follows: the set of variable nodes is taken from the nodes of the MRF for each non-constant potential functionψ C we create a new factor nodef we connectf with all variable nodes in cliquec we assign the potential functionψ C tof Hence, the joint probability of the MRF is equal to the Gibbs distribution over the sum of all factor potentials Learning and Inference in Graphical Models. Chapter 10 p. 11/38

Factor graphs The factor graph of the food preference task looks likes E {C} A C E {A,B,V} E {A,D} E {C,D} B D E {B} E {B,D} Learning and Inference in Graphical Models. Chapter 10 p. 12/38

Stochastic inference in Markov random fields How can we calculatep(u = u O = o) andargmax u p(u = u O = u)? if the factor graph related to a MRF is a tree, we can use the sum-product and max-sum algorithm introduced in chapter 4. in the general case there are no efficient exact algorithms we can build variational approximations (chapter 6) for approximate inference we can use MCMC samplers (chapter 7) for numerical inference we can use local optimization (chapter 8) Example: in the food preference task, what is the overall best choice of food? what is the best choice of food if Antonia eats fish? Learning and Inference in Graphical Models. Chapter 10 p. 13/38

Special types of MRFs MRFs are very general and can be used for many purposes. Some models have been shown to be very useful. In this lecture, we introduce the Potts model. Useful for image segmentation and noise removal Conditional random fields. Usefule for image segmentation the Boltzmann machine. Useful for unsupervised and supervised learning Markov logic networks. Useful for logic inference on noisy data (chapter 11) Learning and Inference in Graphical Models. Chapter 10 p. 14/38

Potts Model Learning and Inference in Graphical Models. Chapter 10 p. 15/38

Potts model The Potts model can be used for segmentation and noise removal in images and other sensor data. We discuss it in the image segmentation case Assume, an image is composed out of several areas (e.g. foreground/background, object A/object B/background) each area has a characteristic color or gray value pixels in the image are corrupted by noise neighboring pixels are very likely to belong to the same area How can we model these assumptions with a MRF? Learning and Inference in Graphical Models. Chapter 10 p. 16/38

Potts model every pixel belongs to a certain area. We model it with a discrete random variable X i,j. The true class label is unobserved. the color/gray value of each pixel is described by a random variabley i,j. The color value is observed. X i,j andy i,j are stochastically dependent. This dependency can be described by an energy function Y i 1,j 1 Y i 1,j Y i 1,j+1 X i 1,j 1 X i 1,j X i 1,j+1 Y i,j 1 Y i,j Y i,j+1 X i,j 1 X i,j X i,j+1 Y i+1,j 1 Y i+1,j Y i+1,j+1 X i+1,j 1 X i+1,j X i+1,j+1 the class labels of neighboring pixels are stochastically dependent. This can be described by an energy functions. we can provide priors for the class label as energy function on individualx i,j Learning and Inference in Graphical Models. Chapter 10 p. 17/38

Potts model energy functions on cliques: similarity of neighboring nodes E {Xi,j,X i+1,j }(x i,j,x i+1,j ) = E {Xi,j,X i,j+1 }(x i,j,x i,j+1 ) = { 0 ifx i,j = x i+1,j 1 ifx i,j x i+1,j { 0 ifx i,j = x i,j+1 1 ifx i,j x i,j+1 dependecy between observed color/gray value and class label. Assume each classk can be characterized by a typical color/gray valuec k E {Xi,j,Y i,j }(x i,j,y i,j ) = Y i,j c Xi,j overall preference for certain classes. Assume a prior distribution p over the classese {Xi,j }(x i,j ) = logp(x i,j ) Learning and Inference in Graphical Models. Chapter 10 p. 18/38

Potts model energy function for the whole Potts model: E =κ i,j E {Xi,j,Y i,j }(x i,j,y i,j ) +λ i,j E {Xi,j,X i+1,j } +λ i,j +µ i,j E {Xi,j,X i,j+1 } E {Xi,j }(x i,j ) with weighting factorsκ,λ,µ 0 Learning and Inference in Graphical Models. Chapter 10 p. 19/38

Potts model for image segmentation Let us apply the Potts model to image segmentation as described before Determining a segmentation is done by maximizing the conditional probability p(...,x i,j,......,y i,j,...) wherey i,j are the color/gray values of a given picture. This is equal to minimizing the overall energy keeping they i,j values fixed. Solution techniques: finding an exact solution is NP-hard in general, in the two-class-caseo(n 3 ) ifnis the number of pixels (solution using graph cuts) local optimization MCMC sampling Matlab-demo Think about extensions of the Potts model that can cope with cases in which the reference colors of the segments are a priori vague or unknown homework Learning and Inference in Graphical Models. Chapter 10 p. 20/38

Conditional Random Fields Learning and Inference in Graphical Models. Chapter 10 p. 21/38

Segmentation with Potts model revisited Using a Potts model for segmentation requires adequate energy functions E {Xi,j,Y i,j } easy for a color segmentation task with pre-specified segment colors possible for a color segmentation task with roughly pre-specified segment colors almost impossible for texture-based segmentation Task: segment picture into areas of road, buildings, vegetation, sky, cars. Idea: combine random field based segmentation with traditional classifiers (e.g. neural networks, support vector machines, decision trees, etc.) apply classifier on small patches of the image use a random field to integrate neighborhood relationships Learning and Inference in Graphical Models. Chapter 10 p. 22/38

Combination of random fields and classifiers A classifier is a mapping from a vector of observations (features) to class labels a mapping from a vector of observations (features) to class probabilities With the second definition, the classifier provides a distribution p(x Y) with X the class label and Y the observation vector. A classifier does not provide a distribution ony nor onx. Learning and Inference in Graphical Models. Chapter 10 p. 23/38

Combination of random fields and classifiers Let us try to build a Potts model integrating the classifiers to model p(x Y) we can model the prior on the class labels as before using a potential function we can model the relationship between neighboring X nodes by a potential function as before Y i 1,j 1 Y i 1,j Y i 1,j+1 X i 1,j 1 X i 1,j X i 1,j+1 Y i,j 1 Y i,j Y i,j+1 X i,j 1 X i,j X i,j+1 Y i+1,j 1 Y i+1,j Y i+1,j+1 X i+1,j 1 X i+1,j X i+1,j+1 we can modelp(x i,j Y i,j ) with the classifier How does the joint distributionp({x i,j,y i,j }) over all(i,j) look like? The joint distribution is not fully specified since we do not knowp({y i,j }) Learning and Inference in Graphical Models. Chapter 10 p. 24/38

Conditional random fields Conditional random fields (CRF) overcome the problem of missingp({y i,j }) by modeling only p({x i,j } {Y i,j }). This is sufficient if we do not want to make inference on{y i,j } but only on {X i,j } A E A conditional random field consists of a set of observed nodeso a set of unobserved random variables U B C edges between pairs of unobserved nodes edges between observed and unobserved nodes D Note that cliques in a conditional random field contain at most one observed node. Learning and Inference in Graphical Models. Chapter 10 p. 25/38

Conditional random fields For every clique that contains at least one unobserved node we specifiy a potential function ψ C : C R >0 A CRF specifies the conditional distribution p(u O) as p(u O) = 1 Z C Cliques ψ C (C) B A E C D Learning and Inference in Graphical Models. Chapter 10 p. 26/38

Example: facade segmentation Segmentation of pictures into categories building/car/door/pavement/road/ sky/vegetation/window. Work of Michael Ying Yang Approach: Hierarchical CRF combined with random decision forest. Result: c.f. Yang and Förstner, 2011 Learning and Inference in Graphical Models. Chapter 10 p. 27/38

Boltzmann Machines Learning and Inference in Graphical Models. Chapter 10 p. 28/38

Boltzmann machines Definition: A Boltzmann machine is a fully connected MRF with binary random variables. Its energy function is defined over 1-cliques and 2-cliques by: E X (x)= θ X x E X,Y (x,y)= w X,Y x y withθ X,w X,Y non-negative real weight factors. Hence, if we enumerate all random variables withx 1,...,X n p(x 1,...,x n ) = 1 n i 1 Z e i=1 j=1 (w X i,x j x i x j )+ n i=1 (θ X i x i ) Note, thatw X,X = 0 andw X,Y = w Y,X. Learning and Inference in Graphical Models. Chapter 10 p. 29/38

Boltzmann machines What is a Boltzmann machine good for? Two tasks: pattern classification denoising of patterns Learning and Inference in Graphical Models. Chapter 10 p. 30/38

Boltzmann machines for pattern classification Goal: we assume some patterns (data) which belong to different categories. Applying a pattern to the Boltzmann machine we want the Boltzman machine to return the appropriate class label. Structure of a Boltzmann machine for classification There are three different types of nodes: observed nodes O. We apply a pattern to the observed nodes by setting their value to the respective value of the pattern and never change it afterwards label nodes L. These serve as output of the Boltzmann machine. We have one label node for each class. Finally, the label nodes indicate the class probabilities for each class hidden nodes H. These nodes are unobserved and used for stochastic inference on the pattern Learning and Inference in Graphical Models. Chapter 10 p. 31/38

Boltzmann machines for pattern classification Process of class predicition: 1. we apply a pattern to the observed nodes, i.e. the value ofi-th observed node is set to thei-th value of the pattern. Afterwards, we do not change the observed nodes any more 2. we use Gibbs sampling to update the values of all hidden nodesh and label nodes L, i.e. we try to determine most probable configurations of p(l,h O). If we are only interested in the most probable configuraton we might also use simulated annealing to find it. 3. after a while we interpret the label nodes. We might assume that the value of the i-th label node is proportional to the posterior probability of the i-th class Learning and Inference in Graphical Models. Chapter 10 p. 32/38

Gibbs sampling for Boltzmann machines To implement Gibbs sampling we need to know p(x i X 1,...,X i 1,X i+1,...,x n ) W.l.o.g. we get p(x n X 1,...,X n 1 ) p(x n,x 1,...,X n 1 ) e n i 1 i=1 j=1 (w X i,x j x i x j )+ n i=1 (θ X i x i ) Hence, = e x n n 1 j=1 (w Xn,X j x j )+θ X n x n+ n 1 i=1 = e x n n 1 j=1 (w Xn,X j x j )+θ X n xn e n 1 i=1 e x n n 1 j=1 (w Xn,X j x j )+θ X n x n p(x n = 0 X 1,...,X n 1 )= 1 Z e0 p(x n = 1 X 1,...,X n 1 )= 1 Z e n 1 j=1 (w Xn,X j x j )+θ X n i 1 j=1 (w X i,x j x i x j )+ n 1 i=1 (θ X i x i ) i 1 j=1 (w X i,x j x i x j )+ n 1 i=1 (θ X i x i ) Fromp(X n = 0 X 1,...,X n 1 )+p(x n = 1 X 1,...,X n 1 ) = 1 follows Z = 1+e n 1 j=1 (w Xn,X j x j )+θ X n Learning and Inference in Graphical Models. Chapter 10 p. 33/38

Boltzmann machines for denoising Goal: we assume that all patterns have a typical structure. Applying a pattern we want the Boltzmann machine to return a typical pattern that is most similar to the pattern applied. Structure of a Boltzmann machine for denoising There are two different types of nodes: observed nodes O. We apply a pattern to the observed nodes by setting their value to the respective value of the pattern and never change it afterwards hidden nodes H. These nodes are unobserved and used for stochastic inference on the pattern Learning and Inference in Graphical Models. Chapter 10 p. 34/38

Boltzmann machines for denoising Process of denoising: 1. we apply a pattern to the observed nodes, i.e. the value ofi-th observed node is set to thei-th value of the pattern. 2. we use Gibbs sampling (or simulated annealing) to update the values of all hidden nodesh and observed nodeso, i.e. we try to determine most probable configurations of p(h, O). 3. after a while we consider the values of the observed nodes as pattern after denoising Learning and Inference in Graphical Models. Chapter 10 p. 35/38

Training of Boltzmann machines For both tasks, we need to train a Boltzmann machine before we can use it, i.e. determine appropriate parametersw X,Y andθ X Assume we are given T training examples (patterns and labels for the classification task, only patterns for the denoising task). Now, we want to maximize the likelihood w.r.t. w X,Y andθ X T p(o (t),l (t) {w X,Y X,Y O H L},{θ X X O H L}) t=1 gradient ascent (calculating the gradient is not trivial) Learning and Inference in Graphical Models. Chapter 10 p. 36/38

Boltzmann machines Some remarks on Boltzmann machines: training Boltzmann machines is very time-consuming however, there are more efficient variants (restricted Boltzmann machines, deep belief networks) which are subject to recent research and which are better suitable for pattern recognition and machine learning we do not want to discuss Boltzmann machines in depth in this lecture since they have been discussed in Prof. Sperschneider s machine learning lecture already Learning and Inference in Graphical Models. Chapter 10 p. 37/38

Summary definition of Markov random fields joint probability distribution factor graph Potts model image segmentation example Conditional random fields image segmentation example of Michael Ying Yang Boltzmann machines Learning and Inference in Graphical Models. Chapter 10 p. 38/38