KillingFusion: Non-rigid 3D Reconstruction without Correspondences Supplementary Material Miroslava Slavcheva 1, Maximilian Baust 1 Daniel Cremers 1 Slobodan Ilic 1, {mira.slavcheva,maximilian.baust,cremers}@tum.de, slobodan.ilic@siemens.com 1 Technische Universität München, Munich, Germany Siemens Corporate Technology, Munich, Germany This supplementary document gives a more detailed derivation of our non-rigid reconstruction model in Sections 1 and. Then it provides more insights into the properties of approximately Killing vector fields in Section 3. Finally, further results are included in Section 4. 1. Non-rigid Registration Energy Recall that we are estimating a 3D vector field Ψ: N 3 R 3 and our energy is formulated as an SDF-based data alignment term, regularized by motion smoothness and rigidity, and a level property preservation term: 1.1. Data Term E non Ψ E dataψ ω k E Killing Ψ ω s E level Ψ. 1 rigid The data term aligns the projective SDF φ n of frame number n with the cumulative SDF φ global, driving their voxel-wise difference to be minimal: E data Ψ 1 φn x u, y v, z w φ global x, y, z. Note that in the above formula, and elsewhere, the dependence of u, v and w on the voxel location x, y, z and on the frame number n has been omitted for brevity. 1.. Level Set Property Maintaining the property of unity gradient ensures geometrically correct SDF evolution: E level Ψ 1 φn x u, y v, z w 1. 3 Here φ denotes the spatial gradient of the SDF φ. Note that when the implementation is over a truncated signed distance field, the gradient magnitude is unity in the narrow band and 0 in the truncated ±1 regions. We do not write this explicitly in the equations, since the deformation field is calculated over the narrow band only. 1.3. Motion Regularization / / / Let us denote the Jacobian of the vector field Ψ as J Ψ v/ v/ v/. Its transpose is JΨ. w/ w/ w/ 1
As explained in the main paper, a Killing vector field generates isometric motion and satisfies the Killing condition J Ψ JΨ F 0. An approximately Killing vector field AKVF generates locally nearly isometric motion, thus balancing volume and angular distortion, and minimizes the Frobenius norm of the Killing condition: E AKV F Ψ 1 J Ψ JΨ F. 4 Next, let us rewrite Eq. 4 using the column-wise stacking operator veca, which denotes the vectorized matrix A. Thus, vecj Ψ R 9 1 is the 9-element vector of stacked elements from J Ψ, and similarly vecj Ψ R9 1 contains the elements from J Ψ. Finally, vecj Ψ R 1 9 denotes the transpose of vecj Ψ. vecj Ψ u x v x w x u y v y w y u z v z w z 5 We obtain the following: E AKV F Ψ 1 vecj Ψ JΨ vecj Ψ JΨ 1 vecj Ψ vecj Ψ vecjψ vecj Ψ vecjψ vecjψ vecj Ψ vecj Ψ vecj Ψ vecj Ψ. 6 However, this condition is too restrictive for cases of large deformation. We notice that the first term of Eq. 6 can be rewritten as follows: u x u y u x vx vy vz wx wy wz vecj Ψ vecj Ψ u v w, 7 which is the motion regularizer typically used in scene flow. This regularizer requires smoothness of the motion, but not necessarily rigidity. Therefore, we propose to reduce the weight of the other term in Eq. 6 in order to be able to capture non-rigid motions. Thus we obtain our damped Killing regularizer: E Killing Ψ vecjψ vecj Ψ γ vecj Ψ vecj Ψ. 8 The factor γ controls the balance between the strictly rigid and non-rigid components of the regularization. A choice of γ 1 would lead to the AKVF condition of Eq. 4. As we aim to alleviate the effect of the rigidity constraint, we use values of γ < 1 in our optimization.. Solution Here we give the detailed derivations of the Euler-Lagrange equations..1. Data Term E data 1 [ φn x u, y v, z w φ global x, y, z 1 φ n x u, y v, z w φ global x, y, z div φ n x u, y v, z w φ global x, y, z ] u
1 φ n x u, y v, z w φ global x, y, z φ n x u, y v, z w φ global x, y, z φ n x u, y v, z w φ global x, y, z φ n x u, y v, z w φ n x u, y v, z w φ global x, y, z x φ n x u, y v, z w In the above x φ is the x-component of the spatial gradient of the SDF φ. E data / E data / v φ n x u, y v, z w φ global x, y, z x φ n x u, y v, z w y φ n x u, y v, z w E data / w z φ n x u, y v, z w φ n x u, y v, z w φ global x, y, z φ n x u, y v, z w φ n Ψ φ global φn Ψ 9 10 Above we used φ n Ψ to refer to the evolved SDF after the application of the deformation field vector u, v, w, i.e. equivalently to φ n x u, y v, z w. We will use this shorthand notation from here onwards... Level Set Property 1 [ φn x u, y v, z w 1 div φ n x u, y v, z w 1 ] u 1 φ n x u, y v, z w 1 1 φ n x u, y v, z w 1 φ n x u, y v, z w 1 φ n Ψ 1 φnψ φ n Ψ 1 1 φ n Ψ ε φ nψ 1 φ nψ φ n Ψ ε φ nψ 1 φ n Ψ ε φnψ φnψ φ n Ψ φnψ φnψ φ n Ψ 1/ φnψ φ n Ψ φ n Ψ φ n Ψ x φ n Ψ xx φ n Ψ y φ n Ψ xy φ n Ψ z φ n x u, y v, z w xz φ n Ψ φ nψ 1 φ n Ψ ε xx φ n Ψ xy φ n Ψ xz φ n Ψ φ n Ψ 11 Here ɛ denotes the norm plus a small constant ɛ which avoids division by zero. / / v φ nψ 1 xx φ n Ψ xy φ n Ψ xz φ n Ψ yx φ n Ψ yy φ n Ψ yz φ n Ψ φ n Ψ φ n Ψ ε / w zx φ n Ψ zy φ n Ψ zz φ n Ψ φ nψ 1 φ n Ψ ε H φnψ φ n Ψ 1 Above we have denoted the Hessian matrix of φ n Ψ as H φnψ. 3
.3. Motion Regularization Expanding the terms in Eq. 8, we obtain: E Killing Ψ 1γu xu yu zv x1γv yv z w xw y1γw z γu y v x γu z w x γw y v z. 13 Then, and similarly: 1 γu x u y u z vx 1 γvy vz wx wy 1 γwz γu y v x γu z w x γw y v z 1 γu x u y u z vx 1 γvy vz wx wy 1 γwz γu y v x γu z w x γw y v z x 1 γu x u y u z vx 1 γvy vz wx wy 1 γwz γu y v x γu z w x γw y v z y 1 γu x u y u z vx 1 γvy vz wx wy 1 γwz γu y v x γu z w x γw y v z z 0 1 γu x u y γv x u z γw x u xx u yy u zz γu xx v xy w xz, 14 Finally, v xx v yy v zz γu xy v yy w yz v w xx w yy w zz γu xz v yz w zz. w u xx u yy u zz u xx v xy w xz E KillingΨ v xx v yy v zz γ u xy v yy w yz w xx w yy w zz u xz v yz w zz u divψ/ v γ divψ/, w divψ/ 15 16 where divψ u x v y w z is the divergence of the warp field. Please note that the derivative given in the paper is with respect to x, y, and z, while here we have the one with respect to u, v and w - this has to be used for the correct gradient descent update of Ψ. 4
3. AKVF Visualization The Killing constraint has advantages over classical motion smoothness because it enforces a divergence-free flow field, i.e. a vector field with no sources or sinks. According to the Helmholz theorem, any vector field can be decomposed into a combination of curl-free and a divergence-free term. Therefore, our regularizer enforces the curl-free part to vanish, as shown below in a D example visualized via []. The stronger the influence of the Killing regularizer, the closer to zero the curl-free part will become. Figure 1. Curl-free component of a D approximately Killing vector field. 4. Additional Results Figure. More canonical pose reconstructions of sequences from the VolumeDeform paper [1]. References [1] M. Innmann, M. Zollhöfer, M. Nießner, C. Theobalt, and M. Stamminger. VolumeDeform: Real-time Volumetric Non-rigid Reconstruction. In European Conference on Computer Vision ECCV, 016. 5 [] G. Peyré. The Numerical Tours of Signal Processing - Advanced Computational Signal and Image Processing. IEEE Computing in Science and Engineering, 134:94 97, 011. 5 5