A note on the primal-dual method for the semi-metric labeling problem

Transcription

1 A note on the primal-dual method for the semi-metric labeling problem Vladimir Kolmogorov Technical report June 4, 2007 Abstract Recently, Komodakis et al. [6] developed the FastPD algorithm for the semi-metric labeling problem, which extends the expansion move algorithm of Boykov et al. [2]. We present a slightly different derivation of the FastPD method.. Preliaries Consider the following energy function: Ex = u V u x v + u,v E uv x u, x v Here G = V, E is an undirected graph. Variables x u for nodes u V belong to a discrete set x u X u. Thus, labeling x belongs to the set X = u V X u. We denote E = {u v, v u u, v E}, i.e. V, E is the directed graph corresponding to undirected graph V, E. The energy function is specified by unary terms θ u i and pairwise terms θ uv i, j i X u, j X v. It will be convenient to denote them as u;i and uv;ij, respectively. We can concatenate all these values into a single vector = { α α I} where the index set is I = {u; i} {uv; ij}. Note that uv; ij vu; ji, so uv;ij and vu;ji are the same element. We will use the notation u to denote a vector of size X u and uv to denote a vector of size X u X v... LP relaxation In this section we describe a linear programg LP relaxation of energy which plays a crucial role for algorithms in [5, 6]. This natural relaxation was studied extensively in the literature, in particular by Schlesinger [0] for a special case when θ uv i, j {0, + }, Koster et al. [7], Chekuri et al. [3], and Wainwright et al. []. Primal problem Let us introduce binary indicator variables: τ u;i = [x u = i], τ uv;ij = [x u = i, x v = j] where [ ] is the Iverson bracket: it is if its argument is true, and 0 otherwise. Variables {τ u;i }, {τ uv;ij } must belong to the following constraint set: Λ = { τ R I + τ u;i = u V } u v E, τ uv;ij = τ v;j j X v Clearly, the problem of imizing function can be formulated as follows: imize, τ subject to τ Λ τ uv;ij {0, } The LP relaxation in [0, 7, 3,, 5, 6] is obtained by dropping the integrality constraint: imize, τ subject to τ Λ 2a Reparameterization and dual problem The dual problem to 2a can be defined using the notion of reparameterization [0, ]. Definition. Suppose vectors θ and define the same energy function, i.e. Ex θ = Ex for all configurations x. Then θ is called a reparameterization of. Let us introduce message M uv;j R for directed edge u v E and label j X v. We denote M uv = {M uv;j j X v } to be a vector of size X v, and M = {M uv u v E} to be the vector of all messages. This vector defines reparameterization θ = [M] as follows: θ u;i = u + u,v E M vu;i θ uv;ij = uv;ij M uv;j M vu;i

2 In this paper denotes the original parameter vector, and θ = [M] denotes its reparameterization defined by some message vector M. To simplify notation, we denote the corresponding energy function as Ex = Ex θ = Ex. Let us define Φθ = θ u;i + u V θ uv;ij u,v E It is easy to see that for any messages M value Φ[M] is a lower bound on the energy: Φ[M] x Ex. This motivates the following maximization problem: maximize subject to Φθ θ = [M] 2b In other words, the goal is to find the tightest possible bound. This maximization problem is dual to 2a see e.g. [2]..2. Optimality conditions Consider fractional labeling τ Λ and messages M corresponding to reparameterization θ = [M]. It is wellknown that τ, M is an optimal primal-dual pair i.e. τ is an optimal solution of 2a and M is an optimal solution of 2a if and only if the following complementary slackness conditions hold: τ u;i > 0 θ u;i = θ u;i i X u τ uv;ij > 0 θ uv;ij = θ uv;i i X j u j X v 3a 3b Algorithms in [5, 6] maintain an integer primal vector τ Λ or equivalently a labeling x X. With such a restriction reparameterization θ = [M] satisfying 3 may not exist. Thus, conditions 3 must be relaxed. Given number, let us say that x, M satisfies -relaxed complementary slackness conditions if the following holds for all nodes and edges: θ u x u θ u;i + θ uv x u, x v θ uv;ij + u x u 4a uv x u, x v 4b Theorem 2 cf. [5]. Suppose that pair x, M satisfies eq. 4. Then x is an -approximation, i.e. Ex y Ey. Proof. Let us sum 4a over nodes u V and 4b over edges u, v E. We obtain Ex Φθ + Ex Ex Φθ Ey y.3. LP relaxation for submodular functions of binary variables The expansion move algorithm [2] and algorithms in [5, 6] rely on solving the imization problem with at most binary variables. In this section we consider the case when X u = {0, } or X u = {0} for all nodes u. Furthermore, we assume that function E is submodular, i.e. each term θ uv with X u = X v = {0, } satisfies θ uv;00 +θ uv; θ uv;0 + θ uv;0. Note that expressionθ uv;00 +θ uv; θ uv;0 θ uv;0 is invariant to reparameterization. This case has several important properties see e.g. []. First, it can solved efficiently by computing a maximum flow in a graph with V + 2 nodes and V + E edges. Second, the algorithm produces an integer optimal solution τ Λ or equivalently labeling x X which is a global imum of E. Thus, optimality conditions 3 are reduced to θ u x u = θ u;i θ uv x u, x v = θ uv;ij 5a 5b Finally, reparameterization θ is in a normal form, i.e. terms θ uv satisfy θ uv 0, 0 = θ uv i max, j max = θ uv;ij 6 where i max is the maximal label in X u and j max is the maximal label in X v. Handling singleton nodes In a practical implementation nodes u with X u = {0} can be handled as follows. First, for each boundary edge u, v with X u = {0}, X v = {0, } we choose message M uv so that θ uv;00 = θ uv;0. Namely, M uv;0 = 0, M uv; = uv;0 uv;00. Then we solve the LP relaxation for nodes u with X u = {0, } ignoring singleton nodes and their incident edges. In this step only messages M uv for edges u v with X u = X v = {0, } are allowed to be modified. Clearly, upon teration conditions 5 and 6 will hold for all nodes and edges. Restricting messages It is easy to see that adding a constant to vector M uv preserves optimality of M. Thus, when solving problem 2b we can require that M uv;0 = 0 for all u v E. This constraint will be used later. Given term uv and reparameterization θ uv for edge u, v with X u = X v = {0, }, messages M uv, M vu can be computed as follows: add constant C = uv;00 θ uv;00 to vector θ uv so that we get θ uv;00 = uv;00 ; 2 set M uv;0 = M vu;0 = 0, M uv; = uv;0 θ uv;0, M vu; = uv;0 θ uv;0.

3 2. FastPD algorithm [6] From now on we assume that the set X u is the same for all nodes: X u = L, and that vector satisfies the following for all nodes and edges: u;i 0 i L 7a uv;ii = 0 i L 7b uv;ij > 0 i, j L, i j 7c We also consider a special case when the triangular inequalities hold: uv;ij uv;ik + uv;kj i, j, k L 7d Note that if we add the symmetry condition uv i, j = uv j, i then eq. 7a-7c give the definition of a semimetric, and eq. 7a-7d give the definition of a metric, However, the symmetry will not be needed. We denote uv = i,j L,i =j uv;ij, max uv = max i,j L uv;ij and = 2 max max uv u,v E. 2.. General overview uv FastPD algorithm [6] maintains integer primal configuration x X and dual variables messages M which define reparameterization θ = [M]. The following invariants hold during the execution: θ uv x u, x v = 0 8a θ uv i, i = 0 i L 8b These properties can easily be ensured in the beginning: for any x it is straightforward to find messages M so that 8 holds. At each step, the algorithm selects some label k L and performs the k-expansion operation described in section 2.2. After the first pass over labels in L the following condition holds for all nodes u: θ u x u = θ u;i 9 i L The algorithm terates when there was a pass over all labels k L but configuration x has not changed. At this point an additional property holds for all edges. In the case of triangular inequalities 7d we have θ uv;ij 0 i, j L, i=x u or j=x v 0 Without triangular inequalities eq. 0 cannot be guaranteed. Instead, the following weaker version holds: θ uv;ij uv;ij max uv i, j L, i=x u or j=x v 0 The final step of the algorithm is to scale messages down: M uv;j := M uv;j / After that the -relaxed complementary slackness conditions 4 will hold and therefore labeling x will be within factor from the optimum k-expansion step The input to this step is label k and primal-dual pair x, M. Let us denote X u = {x u, k}, and let Ẽ be the restriction of function E to configurations x with x u X u. Ẽ can be viewed as an energy function of at most binary variables. We assume that label x u corresponds to 0 and label k corresponds to if k x u. Note that messages M define reparameterization not only for the original energy E, but also for the restriction Ẽ. Submodular case First, let us consider the case when function Ẽ is submodular. We get this case if, for example, triangular inequalities 7d hold. The k-expansion step solves the LP relaxation for the restriction Ẽ, as described in section.3. The LP relaxation has an integer solution. Thus, the goal is to find a global imum of Ẽ and messages M that give optimal reparameterization for Ẽ: x := arg x u e X u Ẽx M := arg max Φθ M:θ=[M] 2a 2b where the objective function in the maximization problem is Φθ = θ u;i + u V i X e u θ uv;ij i X u,v E e u j X e v To achieve efficiency, one could start with reparameterization θ = [M ] when solving 2b. This can be formulated as follows: i find message increment M δ and corresponding reparameterization θ = θ [M δ ] which maximizes Φθ; ii set M := M + M δ. When solving problem 2b, only components M uv;k for edges u v with x v k will be allowed to change. In other words, M uv;j = Muv;j for labels j L v where L v = L {k} {x v }. This is not a severe restriction: as discussed in section.3, it still allows to find an optimal solution to 2b. Function Ẽ may have several global ima. If their cost equals Ex then x is not updated, i.e. x is chosen as x. This guarantees convergence since the number of configurations is finite. Note that in the primal domain the method is equivalent the expansion move algorithm of Boykov et al. [2]. Non-submodular case If function Ẽ is non-submodular then the following operations are performed. For each nonsubmodular edge u, v with uv x u, x v > uv x u, k + uv k, x v terms uv x u, k and uv k, x v are increased by nonnegative constants until we get an equality: uv x u, x v = uv x u, x v = uv x u, k + uv k, x v

4 where is the modified parameter vector. The algorithm then proceeds in the same way as before. Namely, define Ẽ to be the restriction of function E to labelings x with x u X u. It is easy to see that Ẽ is submodular. The LP relaxation of Ẽ is now solved yielding pair x, M and corresponding parameter vector θ = [M]. Finally, vector is restored to its original value, and vector θ is changed to θ = [M]. Note that in the primal domain this algorithm is a special case of the more general majorize-imize technique see e.g. [8]. It is also equivalent to the truncation trick described in [9]. In the context of the labeling problem truncation was proposed independently in [4] and in [9]. Both papers state that the method produces an - approximation, but in [9] this fact is mentioned without proof Correctness of FastPD Theorem 3 cf. [6]. i After the k-expansion reparameterization θ = [M] satisfies conditions 8. ii After the first pass over all labels k condition 9 holds. iii Upon convergence pair x, M satisfies condtion 0 in the case of triangular inequalities 7d or condition 0 without triangular inequalities. Proof. Submodular case By assumption, conditions 8 hold for the initial reparameterization θ = [M ]: θ uv x u, x v = 0 3a θ uvi, i = 0 i L 3b We have M uv;j = M uv;j for labels j L v. Therefore, θ u i = θ ui i L u 3c θ uv i, j = θ uv i, j i L u, j L v 3d Eq. 3d and 3a yield θ uv x u, x v = 0 3e Using eq. 3e and condition 6 of the normal form we obtain θ uv k, k = 0 3f θ uv x u, k 0, θ uv k, x v 0 3g Finally, from the optimality conditions 5 we obtain θ u x u θ u i i X u 3h θ uv x u, x u = 0 3i Non-submodular case Let θ = [M ] and θ = [M] be the reparameterizations before changing and after restoring, respectively. We claim that conditions 3 hold except that 3g is replaced with the following: θ uv x u, k uv x max u, k θ uv k, x v uv k, x v max 3g Indeed, eq. 3a-3f, 3h-3i can be shown in the same way as before, using the fact that θ uv k, k = θ uvk, k, θ uvx u, x v = θ uv x u, x v where θ = [M] is the reparameterization of obtained after solving the LP relaxation. Instead of 3g we now have θ uvx u, k 0, θ uvk, x v 0 If the term for edge u, v is submodular then θ uv = θ uv, so 3g clearly holds. Otherwise, M vu;x u + M uv;k = uvx u, k θ uvx u, k uv x u, k = uv x u, x v uvk, x v uv x u, x v θmax uv which implies the first inequality in 3g. The second inequality follows from the first inequality applied to the reverse edge. We now prove the theorem using equations 3. Proof of i We already showed 8a see 3i. Eq. 8b follows from 3b, 3d and 3f. Proof of ii Let L be the set of labels processed so far at least once. Let us prove by induction on the number of steps that θ u x u θ u i for each node u V and label i L. The base of the induction is trivial: in the beginning L is empty. Suppose that the claim holds for vector θ and labeling x in the beginning of the k-expansion step. The fact θ u x v θ u i for label i = k follows from 3h. For label i L {k} we have θ u x u θ u x u = θ u x u θ u i = θ ui The first inequality follows from 3h, the second inequality is by the induction hypothesis, and the two equalities follow from 3c. Proof of iii We consider only the case with triangular inequalities 7d. The proof for the general case is entirely analogous; we just need to use eq. 3g instead of 3g. Consider the last iteration, i.e. a complete pass over labels k L in which configurationx has not changed. Let L be the set of labels processed in this iteration. Let us prove by induction on the number of steps that θ uv x, j 0 for labels j L. The base of the induction is trivial: in the beginning L is empty. Suppose that the claim holds for vector θ in the beginning of the k-expansion step. Eq. 3d and 3g imply

5 that the claim also holds for vector θ and set L {k} after the step. The analogous fact for term θ uv k, x v can be shown in the same way. Theorem 4 cf. [5, 6]. If reparameterization θ = [M] satisfies conditions 8, 9 and 0 then after message scaling the -relaxed slackness conditions 4 hold. Proof. Let M = M/ be the messages after scaling and θ = [M ] be the corresponding reparameterization. Proof of 4a Consider node u. There holds θ u;i = u;i + = u,v E θ u;i + M vu;i = u;i + θ u;i u;i Using this equality and eq. 9 we obtain θ ux u = i L θ u;i + u;i θ u;i + i L u x u u x u Proof of 4b Now consider edge u, v. Let us show that θ uv;ij 0 for all i, j L. If i = j then this follows from 8b. Suppose that i j. From 0 we get M vu;xu + M uv;j M vu;i + M uv;xv M vu;xu + M uv;xv = 0 max uv max uv Adding the first two equations and subtracting the third one yields M vu;i + M uv;j 2 max uv M vu;i + M uv;j = M vu;i + M uv;j θ uv;ij = uv;ij M vu;i + M uv;j uv;ij as claimed. Thus, i,j L θ uv;ij 0. Finally, using 8a we get θ uv x u, x v = uv x u, x v M uv;x v M vu;x u uv uv 0 = uv x u, x v uv x u, x v θ uv x u, x v = uv x u, x v. References [] E. Boros and P. L. Hammer. Pseudo-boolean optimization. Discrete Applied Mathematics, 23-3:55 225, November [2] Y. Boykov, O. Veksler, and R. Zabih. Fast approximate energy imization via graph cuts. PAMI, 23, November 200. [3] C. Chekuri, S. Khanna, J. Naor, and L. Zosin. Approximation algorithms for the metric labeling problem via a new linear programg formulation. In SODA, pages 09 8, [4] N. Komodakis and G. Tziritas. Approximate labeling via the primal-dual schema. Technical Report CSD- TR , University of Crete, February [5] N. Komodakis and G. Tziritas. Approximate labeling via graph-cuts based on linear programg. PAMI, To appear. [6] N. Komodakis, G. Tziritas, and N. Paragios. Fast, approximately optimal solutions for single and dynamic MRFs. In CVPR, [7] A. Koster, C. van Hoesel, and A. Kolen. The partial constraint satisfaction problem: Facets and lifting theorems. Operations Research Letters, 233-5:89 97, 998. [8] K. Lange. Optimization. Springer Verlag, New York, [9] C. Rother, S. Kumar, V. Kolmogorov, and A. Blake. Digital tapestry. In CVPR, [0] M. I. Schlesinger. Syntactic analysis of twodimensional visual signals in noisy conditions. Kibernetika, 4:3 30, 976. [] M.J. Wainwright, T.S. Jaakkola, and A.S. Willsky. MAP estimation via agreement on trees: Messagepassing and linear-programg approaches. IEEE Transactions on Information Theory, 5: , November [2] T. Werner. A linear programg approach to maxsum problem: A review. PAMI, 297:65 79, July Acknowledgements I thank Pushmeet Kohli and Philip Torr for discussions on connections between primal-dual schema and tree-reweighted message passing.