MAP Examples. Sargur Srihari

Transcription

1 MAP Examples Sargur 1

2 Potts Model CRF for OCR Topics Image segmentation based on energy minimization 2

3 Examples of MAP Many interesting examples of MAP inference are instances of structured prediction, which involves doing inference in a conditional random field (CRF) model p(y x): Example of OCR given next Where x is a character image and y is its recognized label 3

4 Potts models A generalization of the Ising model and Boltzmann model to multiple values

5 MAP inference in 1-D structure We are given images x i [0,1] d d of characters in the form of pixel matrices; MAP inference is to jointly recognizing the most likely word (y i ) n i=1 encoded by the images Recognized characters Observed images Chain-structured conditional random field for OCR 5

6 MAP inference in a 2-D structure We are interested in locating an entity in an image and label all its pixels. Our input x [0,1]d d is a matrix of image pixels, our task is to predict the label y {0,1}d d, indicating whether each pixel encodes object we want to recover yi xi Intuitively, neighboring pixels should have similar values in y, i.e. pixels associated with the horse should form one continuous blob (rather than white noise) 6

7 Potts model Prior knowledge modeled as PGMs We can introduce potentials ϕ(y i,x i ) that encode the likelihood that any given pixel is from our subject. We then augment them with pairwise potentials ϕ(y i,y j ) for neighboring y i,y j, which encourages adjacent y s to have the same value with higher probability ϕ(y i,y j ) y i ϕ(y i,x i ) 7 x i

8 Ising Model in Statistical Physics Energy of interacting atoms Determined from their spin Atom s spin is sum of its electron spins Each atom associated with binary random variable X i {+1,-1} whose value is direction of atom s spin Energy function parametric form ε ij (x i,x j ) = -w ij x i x j Symmetric in X i, X j : note scope is pairwise When w ij >0 model prefers aligned spins: ferromagnetism w ij <0 : antiferromagnetic w ij =0: non-interacting Makes contribution w ij to energy when X i =X j (same spin) -w ij otherwise Probability distribution over atoms (energy function) P(ξ) = 1 Z exp w ijx i x j u i x i i< j i ξ ε Val(Χ)

9 Boltzmann Distribution Variant of Ising Model Variables X i have value {0,1} instead of {+1,-1} Energy function has same parametric form ε ij (x i,x j )=-w ij x i x j Nonzero contribution w ij from edge X i -X j only when X i =X j =1 Ising model has contribution w ij when variables are same and w ij when they are different Has the same energy function as Ising model P(ξ) = 1 Z exp w x x u x ij i j i i Mapping 0 to -1 i< j i 9

10 CRF for pixel labeling x i = unknown pixel label y i = known noisy pixel A conditional probability distribution p(x y) Neighboring pixel labels x i and x j are strongly correlated 10

11 Energy Functions Graph has two types of pairwise cliques {x i,y i } expresses strength of label to pixel values Energy function η x i y i {x i,x j } are neighboring pixel labels Choose β x i x j 11

12 Potential Function Complete energy function of model å å å E h x x x x y (x,y) = i -b i j -h i i i {, i j} i The hx i term biases towards pixel values that have one particular sign Defines a joint distribution over x and y given by 1 p(x, y) = exp{ -E(x, y)} Z We now fix elements of y to the observed values given by the pixels It implicitly defines a conditional distribution p(x y) over the labeled image 12

13 Metric MRF for Labeling Task: We are given a graph with nodes X 1,..X n, edges E Task is to assign to each X i a label in V={v 1,..v k } E.g., labeling super-pixels as v 1 =cow, v 2 = grass Each node, in isolation, has a preferred label E.g., when pixel x i has color=brown and label is cow,! i (x i ) is high When pixel x i has color=green and label is cow,! i (x i ) is low However, we want smoothness constraint over neighbors Neighboring nodes should have similar values 13

14 Solution for Labeling Solution: In a pairwise MRF Encode node preferences as node potentials! i takes higher value when x i is brown and label is cow Smoothness preferences as edge potentials Model in negative log-space, using energy functions Specify the energy function: E(x 1,..x n ) = ε i (x i ) + ε ij (x i, x j ) i i, j E For MAP objective, ignore partition function Goal: Minimize the energy (MAP objective) How to define smoothness? Next. arg min x 1,..x n E(x 1,..x n ) 14

15 Smoothness for Metric MRF 15

16 Generalizations of Smoothness for Metric MRF 1. Potts model (when there are more than two labels) 2. Distance Function on labels Prefer neighboring nodes to have labels smaller distance apart Metric MRF Need a metric µ(v k,v l ) on labels 16