Gaussian Conditional Random Field Network for Semantic Segmentation - Supplementary Material

Transcription

1 Gaussan Condtonal Random Feld Networ for Semantc Segmentaton - Supplementary Materal Ravtea Vemulapall, Oncel Tuzel *, Mng-Yu Lu *, and Rama Chellappa Center for Automaton Research, UMIACS, Unversty of Maryland, College Par. * Mtsubsh Electrc Research Laboratores, Cambrdge, MA. Abstract In Secton 1 of ths supplementary materal, we derve the mean feld update equaton for the Gaussan dstrbuton used n ths paper. Secton 2 provdes the relevant dervatve formulas for bacpropagaton and Secton 3 presents a detaled algorthmc descrpton of the proposed Gaussan CRF networ. Notatons: We use bold face small letters to denote vectors and bold face captal letters to denote matrces. We use A, A 1, A and tracea) to denote the transpose, nverse, determnant and trace of a matrx A, respectvely. We use b 2 2 to denote the squared l 2 norm of a vector b. A 0 means A s symmetrc and postve semdefnte. We use R to denote the set of real numbers and E to denote expectaton. 1. Mean feld nference In ths wor, we model the condtonal probablty densty P y X) as a Gaussan dstrbuton gven by P y X) exp { 12 } Ey X), where E y X) y r y y ) W y y ) 1) y I + W y 2 r y + r r 2 y W y The standard mean feld approach approxmates the ont Gaussan dstrbuton P y X) usng a smpler Gaussan dstrbuton Qy X) whch can be wrtten as a product of ndependent margnals,.e, Qy X) Q y X) 1, where Qy X) s a Gaussan dstrbuton wth mean µ R K and covarance Σ R K K. The parameters {µ, Σ } of Q are obtaned by mnmzng the KL-dvergence between the dstrbutons Q and P. 1 Note that nstead of usng margnals of scalar varables y, we are usng margnals of vector varables y.

2 KLQ P ) [ ] Qy X) P y X) Qy X) log Qy X) log [Qy X)] Qy X) log [P y X)] Q y X) log [Q y X)] Qy X) log [P y X)] usng Qy X) Q y X) ) 2) 1 2 log [ 2πe) K Σ ] Qy X) log [P y X)] {µ, Σ } argmn {µ,σ } argmn {µ,σ } argmn {µ,σ } argmn {µ,σ } argmn {µ,σ } argmn {µ,σ } argmn {µ,σ } KLQ P ) log [ 2πe) K Σ ] log [ Σ ] + Qy X) y W y log [ Σ ] + 2 E [ y ] W y E Qy X) y y Qy X) log [P y X)] I + I + W log [ Σ ] + E trace y y I + 2 E [ trace y y )] W log [ Σ ] + trace E [ y y ] I + 2 trace E [ y y ] ) W log [ Σ ] + trace Σ + µ µ 2 trace µ µ ) W W y 2 y 2 E [ r ] y W W ) I W 2 Qy X) r y E [ r y ] E [ r y ] r µ 3) Note that n the last step, we have used the fact that y and y are ndependent under the dstrbuton Q. From 3) we have, Σ argmn trace Σ I + W log [ Σ ] 4) Σ Note that 4) s a convex problem. Dfferentatng the cost functon and settng the gradent to zero, we get Σ I + W ) 1.

3 From 3) we have, µ argmn trace µ µ µ argmn µ µ I + I + W W 2r µ 2 µ 2r µ 2µ W µ trace µ µ ) W 5) Note that 5) s a convex problem. Dfferentatng the cost functon and settng the gradent to zero, we get µ I + W 1 r + w µ. 6) Hence, for the Gaussan dstrbuton n 1), the mean feld update for computng the means {µ } s gven by µ I + W ) 1 r + W µ ). 7) 2. Bacpropagaton Let L be the fnal loss functon. Bacpropagatng through the matrx generaton layer: Gven the dervatves /dw of the loss functon wth respect to the output of the matrx generaton layer, we can compute the dervatves of L wth respect to ts nput s and parameters C usng ) trace C), ds dw dc 8) s. dw Bacpropagatng through the smlarty layer: Gven the dervatves /ds of the loss functon wth respect to the output of the smlarty layer, we can compute the dervatves of L wth respect to ts nput z and parameters f m usng M ) 2 f m fm s z z ), dz ds m1 9) 2 s z z ) z z ) f m. df m ds

4 Bacpropagatng through the odd update layer: Gven the dervatves / of the loss functon wth respect to the output of an odd update layer, we can compute the dervatves of L wth respect to ts nputs r, W and µ n usng { I + W ) 1 dr 0 elsewse, f node s on an odd column I + dw dµ n W ) 1 µ n µ out ), for n odd columns, { + W I + ) W ) 1 0 elsewse. f node s on an even column 10) Bacpropagatng through the even update layer: Gven the dervatves / of the loss functon wth respect to the output of an even update layer, we can compute the dervatves of L wth respect to ts nputs r, W and µ n usng { I + W ) 1 dr 0 elsewse, f node s on an even column I + dw dµ n W ) 1 µ n µ out ), for n even columns, { + W I + ) W ) 1 0 elsewse. f node s on an odd column 11)

5 3. Algorthmc descrpton of the proposed Gaussan CRF networ Algorthm 1 Gaussan CRF Networ Input: Image X Unary Networ 1: Apply the DeepLab CNN wth parameters θ CNN u to mage X to compute the unary predctons r {r }. Parwse Networ r DeepLabCNN X, θu CNN ). 2: Apply the parwse networ wth paramters {θ CNN p, {f m }, C 0} to mage X to compute the parwse matrces {W } used n the energy functon. a) DeepLab CNN parameters θ CNN p ): Compute per-pxel features z {z }. z DeepLabCNN X, θp CNN ). b) Smlarty layer parameters {f m }): Compute the smlarty measure s [0, 1] for every par of connected pxels and usng the features z. s e M m1f m z z))2. c) Matrx generaton layer parameters C 0): Compute the matrx W 0 for every par of connected pxels and usng the smlarty measure s. W s C. GMF Networ 3: Intalze the GMF networ nput µ 1 r, and partton the nodes nto even and odd columns µ {µ e, µ o }. 4: for t 1 to 5 a) Even update layer: Update the even column nodes µ t+1 e usng r, {W } and µ t o. µ t+1 I + W 1 r + W µ t, µ µ e, µ µ o. b) Odd update layer: Update the odd column nodes µ t+1 o usng r, {W } and µ t+1 e. µ t+1 I + W 1 r + W µ t+1, µ µ o, µ µ e. 5: Upsample µ 6 to the nput mage resoluton usng blnear nterpolaton to obtan the class predcton scores at each pxel. 6: For each pxel, select the class label correspondng to the hghest score. Output: Class label at each pxel.