Basics on the Graphcut Algorithm. Application for the Computation of the Mode of the Ising Model

Transcription

1 Basics on the Graphcut Algorithm. Application for the Computation of the Mode of the Ising Model Bruno Galerne MAP5, Université Paris Descartes Master MVA Cours Méthodes stochastiques pour l analyse d images Lundi 13 mars 2017

2 Outline The Max-Flow Min-Cut Theorem Computation of the Mode of the Ising Model

3 Reference The main reference for this course is Exercise 2 of Chap. 4 of the book Pattern Theory: The Stochastic Analysis of Real-World Signals by D. Mumford and A. Desolneux [Mumford and Desolneux 2010] Graph illustrations and proof of Max-flow min-cut theorem from [Cormen et al., 2009]. Original research paper for applying graphcut to the Ising model [Greig et al., 1989]

4 Outline The Max-Flow Min-Cut Theorem Computation of the Mode of the Ising Model

5 Directed Graphs We consider an arbitrary finite directed graph (V, E) with vertices V and directed edges E V V, with no self-loop. The vertices will be denoted u, v, V. The directed edges will be denoted uv E or (u, v) E. The graph (V, E) is given with a capacity function c : E [0, + ). For each edge (u, v) E, c(u, v) is the maximal amount of current allowed to go from u to v. Two special vertices: the vertex s (the source) and the vertex t (the sink). We call a flow network the 5-uplet G = (V, E, c, s, t).

6 Flows in a flow network For a vertex u V define the two sets of vertices Γ + (u) = {v, uv E} and Γ (u) = {v, vu E}. Definition (Flow) A flow f is a function f : E R that satisfies the two following properties: Capacity constraint: For all (u, v) E, 0 f (u, v) c(u, v). Flow conservation: For all u V \ {s, t}, f (u, v) = f (v, u). v Γ + (u) v Γ (u)

7 Flows in a flow network Definition (Value of a flow) The value f of a flow is the net amount of flow getting out of the source f = f (s, v) f (v, s). v Γ + (s) v Γ (s) f is also the amount of flow getting out through the sink t... f is also the amount of flow going through the partition of the vertices f = 19 = etc. (proof in a couple of slides)

8 Max-flow problem Since f f is continuous (it is a linear map) and the set of flow is compact (it is a compact polytope), there exists flows f having maximal flow value f. Max-flow problem: Given a a network flow G, determine a flow f that reaches the maximal flow value f associated with the network.

9 Cut of Flow Networks Definition (Cut) A cut (S, T ) of flow network G = (V, E) is a partition of V into S and T = V \ S such that s S and t T. The capacity of the cut (S, T ) is the total amount of current that can pass from S to T : c(s, T ) = c(u, v). (u,v) S T E If f is a flow, the net flow across the cut (S, T ) is defined as f (u, v) f (v, u). (u,v) S T E (v,u) T S E S = {s, v 1, v 2 }, T = {t, v 3, v 4 } c(s, T ) = = 26 Net flow across (S, T ) is = 19 f = 19...

10 Cut and Flows Proposition (The capacity of a cut bounds the value of a flow) Let (S, T ) be any cut and f be any flow. Then, (a) The net flow across the cut (S, T ) is equal to the flow value f : f (u, v) f (v, u) = f. (u,v) S T E (v,u) T S E (b) The capacity of the cut (S, T ) is larger than the flow value f : f c(s, T ) = c(u, v). (u,v) S T E

11 Cut and Flows: Proof I Proof of (a) : The net flow across the cut (S, T ) is equal to the flow value f : f (u, v) f (v, u) = f. (u,v) S T E (v,u) T S E Using the flow conservation at each u S \ {s}, f = f (s, v) f (v, s) + v Γ + (s) v Γ (s) = f (u, v) u S v Γ + (u) = f (u, v) (u,v) S T E v Γ (u) (v,u) T S E u S\{s} f (v, u) f (v, u) v Γ + (u) f (u, v) v Γ (u) f (v, u) } {{ } =0

12 Cut and Flows: Proof II Proof of (b) : The capacity of the cut (S, T ) is larger than the flow value f : f c(s, T ) = c(u, v). (u,v) S T E Recall that by definition the flow is bounded by the capacity: From (a), For all (u, v) E, f = (u,v) S T E (u,v) S T E 0 f (u, v) c(u, v). f (u, v) }{{} c(u,v) (v,u) T S E c(u, v) = c(s, T ). f (v, u) }{{} 0

13 Min-Cut Problem Max-flow problem: Given a a network flow G, determine a cut (S, T ) such that the cut capacity c(s, T ) reaches the minimal value among all cuts. Remarks: This is a partition problem that is a priori combinatorial: Find the minimum value among the 2 V 2 possible cuts. But this can be solved in polynomial time!

14 The Max-Flow Min-Cut Theorem Theorem (The Max-Flow Min-Cut Theorem) The maximal flow value from s to t is equal to the minimum of the capacities of cuts separating s from t: max f = min c(s, T ). flow f cut (S,T ) Remarks: The max-flow and min-cut problems can be interpreted as two primal-dual linear programs. The theorem then boils down to a consequence of the strong duality theorem. We will give a more constructive proof that leads to an algorithm: 1. Construct iteratively a maximal flow f. 2. Given a maximal flow f, construct a cut (S, T ) such that c(s, T ) = f. Each step of this algorithm relies on the concept of residual network associated with a flow f...

15 Residual Network Associated with a Flow Definition (Residual network associated with a flow f ) Let f be a flow. The residual capacity c f : V V [0, + ) associated with f is defined by c f (u, v) = max(c(u, v) f (u, v), f (v, u)). (with the convention c(u, v) = 0 if (u, v) / E). The set of residual edges is E f = {(u, v) V V, c f (u, v) > 0}. The residual network associated with f is G f = (V, E f, c f, s, t). Intuitively, the residual network associated with the flow f is made of all edges (u, v) such that the flow can be increased from u to v or decreased from v to u. Recall that we always have f c(s, T ). To have equality one needs that for each boundary edge (u, v) with u S and v T, f (u, v) = c(u, v) and f (v, u) = 0, that is, c f (u, v) = 0.

16 Residual Network Associated with a Flow Network G with flow f with value f = 19. Residual network G f. Augmenting path with residual capacity 4. New flow with value f + 4 = 23

17 Residual Network and Augmenting Path Definition Let f be a flow. We call augmenting path any simple path p = (s = v 1, v 2,..., v k = t) from the source s to the sink t in the residual graph G f. The residual capacity of an augmenting path p is the minimal residual capacity along p, that is, c f (p) = min{c f (v i, v i+1 ), 1 i k 1}. Proposition Let p = (s = v 1, v 2,..., v k = t) be an augmenting path. Define g as follows: For all edges (u, v) / p, g(u, v) = f (u, v) and for all edges (u, v) p, { f (u, v) = f (u, v) + c f (p) if c(u, v) f (u, v) > f (v, u), otherwise, { f (u, v) = f (u, v) f (v, u) = f (v, u) c f (p) f (v, u) = f (v, u) Then g is a flow of G with value g = f + c f (p) > f. Proof: g is a flow (respect capacity and conservation constraint), and the net value at s is +c f (p).

18 The Max-Flow Min-Cut Theorem Theorem (The Max-Flow Min-Cut Theorem) The maximal flow value from s to t is equal to the minimum of the capacities of cuts separating s from t: max f = min c(s, T ). flow f cut (S,T ) More precisely, if f is a flow, then the following conditions are equivalent: (a) f is a maximum flow in G. (b) The residual network G f contains no augmenting paths. (c) The flow of f is equal to the capacity of a cut: f = c(s, T ) for some cut (S, T ).

19 Proof of The Max-Flow Min-Cut Theorem Theorem (The Max-Flow Min-Cut Theorem) The maximal flow value from s to t is equal to the minimum of the capacities of cuts separating s from t: max f = min c(s, T ). flow f cut (S,T ) More precisely, if f is a flow, then the following conditions are equivalent: (a) f is a maximum flow in G. (b) The residual network G f contains no augmenting paths. (c) The flow of f is equal to the capacity of a cut: f = c(s, T ) for some cut (S, T ). Proof of (c) (a): We know that for any cut (S, T ) and any flow f, f c(s, T ). Hence, if there exists (S, T ) such that f = c(s, T ), then the flow value is necessary maximal. Proof of (a) (b): This is the contrapositive of the previous proposition: if the residual network contains an augmenting path with capacity c f (p), then we can augment f into a flow with value f + c f (p), and thus f is not maximal.

20 Proof of The Max-Flow Min-Cut Theorem Theorem (The Max-Flow Min-Cut Theorem) The maximal flow value from s to t is equal to the minimum of the capacities of cuts separating s from t: max f = min c(s, T ). flow f cut (S,T ) More precisely, if f is a flow, then the following conditions are equivalent: (a) f is a maximum flow in G. (b) The residual network G f contains no augmenting paths. (c) The flow of f is equal to the capacity of a cut: f = c(s, T ) for some cut (S, T ). Proof of (c) (a): We know that for any cut (S, T ) and any flow f, f c(s, T ). Hence, if there exists (S, T ) such that f = c(s, T ), then the flow value is necessary maximal. Proof of (a) (b): This is the contrapositive of the previous proposition: if the residual network contains an augmenting path with capacity c f (p), then we can augment f into a flow with value f + c f (p), and thus f is not maximal.

21 Proof of The Max-Flow Min-Cut Theorem Theorem (The Max-Flow Min-Cut Theorem) The maximal flow value from s to t is equal to the minimum of the capacities of cuts separating s from t: max f = min c(s, T ). flow f cut (S,T ) More precisely, if f is a flow, then the following conditions are equivalent: (a) f is a maximum flow in G. (b) The residual network G f contains no augmenting paths. (c) The flow of f is equal to the capacity of a cut: f = c(s, T ) for some cut (S, T ). Proof of (c) (a): We know that for any cut (S, T ) and any flow f, f c(s, T ). Hence, if there exists (S, T ) such that f = c(s, T ), then the flow value is necessary maximal. Proof of (a) (b): This is the contrapositive of the previous proposition: if the residual network contains an augmenting path with capacity c f (p), then we can augment f into a flow with value f + c f (p), and thus f is not maximal.

22 Proof of The Max-Flow Min-Cut Theorem Proof of (b) (c): Suppose that G f contains no augmenting paths, that is, no path from s to t. Then define S = {v V, there exists a path from s to v in G f } and T = V \ S. Remark that s S and t T (since by hypothesis t / S), and thus (S, T ) is a cut. Now consider a pair of vertices u S and v T. By construction, u and v are not connected in G f and thus c f (u, v) = 0, that is, f (u, v) = c(u, v) and f (v, u) = 0. Hence, f = f (u, v) f (v, u) = (u,v) S T E (u,v) S T E = c(s, T ). c(u, v) 0 (v,u) T S E The flow of f is equal to the capacity of the cut (S, T ). Note that the construction of the cut (S, T ) is explicit.

23 Algorithm for Min-Cut Computation 1. Compute a max-flow f with the following iterative algorithm: Initialize the flow f with zero While there exists an augmenting path p in the residual network Gf : Augment the flow f along p. 2. Given the max-flow f, compute the cut (S, T ) defined by S = {v V, there exists a path from s to v in G f } and T = V \ S. Remarks: The iterative procedure of 1. is called the Ford-Fulkerson method. The convergence is not guaranteed. One should chose the augmenting path p among the shortest paths linking s and t. This is then called the Edmonds-Karp algorithm [Cormen et al., 2009]. It then converges in O( V E 2 ) times [Cormen et al., 2009].

24 Outline The Max-Flow Min-Cut Theorem Computation of the Mode of the Ising Model

25 The Ising Model Notation: Ω M,N = {0,..., M 1} {0,..., N 1} is the set of pixel indexes. For keeping the notation short we will denote by α or β pixel coordinates (k, l), (m, n), so that I(α) = I(k, l) for some (k, l). We note α β to mean that pixels α and β are neighbors for the 4-connectivity (each pixel has 4 neighbors, except at the border). Energy: To each couple of images (I, J), I R M N, J { 1, 1} M N, one associates the energy E(I, J) defined by E(I, J) = c (I(α) J(α)) 2 + (J(α) J(β)) 2 α Ω M,N α β where c > 0 is a positive constant and the sum α β means that each pair of connected pixels is summed once (and not twice). The first term of the energy measures the similarity between I and J. The second term measures the similarity between pixels of J that are neighbors.

26 The Ising Model Energy: E(I, J) = c (I(α) J(α)) 2 + (J(α) J(β)) 2 α Ω M,N α β The Ising model associated with I: For any fixed image I, this energy enables to define a discrete probability distribution on { 1, 1} M N by p T (J) = 1 Z T e 1 T E(I,J), where T > 0 is a constant called the temperature and Z T is the normalizing constant Z T = e 1 T E(I,J). J { 1,1} M N The probability distribution p T is the Ising model associated with I (and constant c and temperature T ).

27 The Ising Model Energy: E(I, J) = c (I(α) J(α)) 2 + (J(α) J(β)) 2 α Ω M,N α β Probability distribution: p T (J) = 1 Z T e 1 T E(I,J) Minimizing with respect to J the energy E(I, J) is equivalent to find the most probable state of the discrete distribution p T. Note that this most probable state (also called the mode of the distribution) is the same for all temperatures T. As T tends to 0, the distribution p T tends to be concentrated at this mode. As T tends to +, p T tends to be uniform over all the possible image configurations. Hence the temperature parameter T controls the amount of allowed randomness around the most probable state.

28 The Ising Model The main questions regarding the Ising model are the following: 1. How can we sample efficiently from the discrete distribution p T? Gibbs sampler (approximate simulation) 2. How can we compute the common mode(s) of the distributions p T, that is (one of) the most probable state of the distributions p T? Simulated annealing (approximate computation) Graphcut algorithm (exact computation)

29 The Ising Model: Simulated Annealing

30 Graphcut for the Ising Model Main idea: [Greig et al., 1989] Propose a directed graph such that the capacity c(s, T ) of a cut (S, T ) corresponds to the energy E(J) of some configuration ({J = 1}, {J = 1}). Compute the configuration with minimal energy by solving a max-flow problem. Proposed graph: V = (s, t, Ω M,N ) Each edge (s, α) has capacity (I(α) 1) 2. Each edge (α, t) has capacity (I(α) + 1) 2. Each edge (α, β) Ω M,N Ω M,N has capacity 4 = (1 ( 1)) 2. There is more efficient algorithms for this specific graphs [Boykov, Kolmogorov, 2004]

31 Graphcut for the Ising Model V = (s, t, Ω M,N ) Each edge (s, α) has capacity (I(α) 1) 2. Each edge (α, t) has capacity (I(α) + 1) 2. Each edge (α, β) Ω M,N Ω M,N has capacity 4 = (1 ( 1)) 2. Capacity of a cut: Let (S, T ) be a cut of V. Let J such that J(α) = 1 if α S and 1 if α T. c(s, T ) = = (u,v) S T E α T s.t.(s,α) E c(u, v) c(s, α) + α S s.t.(α,t) E c(α, t) + = (I(α) 1) ) α T α S(I(α) 2 + = (I(α) J(α)) 2 + (J(α) J(β)) 2 = E(J). α Ω M,N α β (α,β) S\{s} T \{t} E (α,β) S\{s} T \{t} E (1 ( 1)) 2 c(α, β)

32 Bibliographic references I Boykov, Y. and Kolmogorov, V., An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision IEEE TPAMI, Cormen, T. H., Leiserson, C. E., Rivest, R. L. and Stein, C., Introduction to Algorithms, Third Edition, The MIT Press, 2009 D. Mumford and A. Desolneux, Pattern Theory: The Stochastic Analysis of Real-World Signals, Ak Peters Series, 2010 D. M. Greig and B. T. Porteous and A. H. Seheult, Exact Maximum A Posteriori Estimation for Binary Images, Journal of the Royal Statistical Society. Series B (Methodological), 1989