CSE 554 Lecture 6: Deformation I

Transcription

1 CSE 554 Lecture 6: Deformation I Fall 20 CSE554 Deformation I Slide Review Alignment Registering source to target by rotation and translation Methods Rigid-body transformations Aligning principle directions (PCA) Aligning corresponding points (SVD) Iterative improvement (ICP) Combines PCA and SVD Input After PCA After ICP Source Target CSE554 Deformation I Slide 2

2 Non-rigid Registration Rigid alignment cannot account for shape variance Non-rigid deformation can give a better fit Source Target Rigid alignment After non-rigid deformation CSE554 Deformation I Slide 3 Non-rigid Registration A minimization problem Minimizing the distance between the deformed source and the target Fitting term Minimizing the distortion to the source shape Distortion term CSE554 Deformation I Slide 4 2

3 Intrinsic vs. Extrinsic Intrinsic methods Deforms points on the source curve/surface App: boundary curve or surface matching Extrinsic methods Deforms all points on and interior to the source curve/surface App: image or volume matching CSE554 Deformation I Slide 5 Laplacian-based Deformation An intrinsic method Simple to implement, produces reasonable results Preserving local shape features Widely used in graphics applications for interactive deformation Reference: Laplacian surface editing, by Sorkine et al., 2004 (citation > 300) CSE554 Deformation I Slide 6 3

4 Setup Input Source with n points: p,,p n Target with m points (m<=n): q,,q m Corresponding to first m source points Output Deformed locations of source points: p,,p n Deformed q 2 Source p =q p 3 =q 3 p 2 An example with 3 target points, two of which are stationary (red) CSE554 Deformation I Slide 7 Overview Finding deformed locations p i that minimize: E E f E d E f : fitting term Measures how close are the deformed source to the target E d : distortion term Measures how much the source shape is changed CSE554 Deformation I Slide 8 4

5 Fitting Term Sum of squared distances between corresponding points Over all pairs of source and target points m E f p i q i 2 q 2 p 2 CSE554 Deformation I Slide 9 Distortion Term Q: How to measure shape? A: By bumpiness at each vertex Laplacian: vector from the centroid of neighbors to the vertex Recall that in fairing, we reduced this vector to smooth out bumps A linear operator over point locations Lp i p i N i p j jni p i where N i ={i,i2, } are indices of neighboring vertices of p i p i p i p i2 2 p i2 CSE554 Deformation I Slide 0 5

6 Distortion Term Minimizing changes in Laplacians during deformation Over all source points n E d Lp i i 2 d i : Laplacian at p i before deformation p i p i i Lp i CSE554 Deformation I Slide Putting Together Finding deformed locations p i that minimize: A quadratic equation in terms of variables (p ix, p iy, p iz ) q i, d i are constants E E f E d m L[] is a linear operator n p i q i 2 Lp i i 2 CSE554 Deformation I Slide 2 6

7 Quadratic Minimization A general form of quadratic minimization: k min a i T x b i 2 There are s variables: x=(x,,x s ) T Each a,, a k is a length-s column vector (linear coefficients) Each b,, b k is a scalar (constant coefficients) k should be greater than s CSE554 Deformation I Slide 3 Quadratic Minimization Re-writing our minimization in the general form E E f E d m n p i q i 2 Lp i i 2 In 2D, there are 2n variables: x = (p x,, p nx, p y,, p ny ) T In 3D, there are 3n variables We will next re-write each quadratic term in 2D as (a i x-b i ) 2 Can be extended easily to 3D CSE554 Deformation I Slide 4 7

8 Quadratic Minimization The a i and b i in the fitting term m E f p i q i 2 p i x q ix 2 p i y q iy 2 There are 2m quadratic terms In the first set of m terms: For i=,,m, b i =q ix, a i contains all zero, except its (i)th entry is. In the second set of m terms: m 2 m a i T x b i 2 For i=,,m, b i+m =q iy, a i+m contains all zero, except its (i+n)th entry is m CSE554 Deformation I Slide 5 Quadratic Minimization The a i and b i in the fitting term m E f p i q i 2 p i x q ix 2 p i y q iy 2 There are 2m quadratic terms m Example with 3 vertices and 2 fitting constraints (n=3; m=2): a T a 2 T a 3 T a 4 T m a i T x b i 2 x p x p 2 x p 3 x p y p 2 y p 3 y m b q x b 2 q 2x b 3 q y b 4 q 2y CSE554 Deformation I Slide 6 8

9 Quadratic Minimization The a i and b i in the distortion term: Lp i p i N i p j jni n E d n Lp i i 2 n Lp i x ix 2 Lp i y iy 2 There are 2n quadratic terms The first set of n terms: 2 n a i T x b i 2 For i=,,n, a i is all zero except the (i)th entry is, the (j)th entries are -/ N i for all jœn i, and b i =d ix The second set of n terms: For i=,,n, a i+n is all zero except the (i+n)th entry is, the (j+n)th entries are -/ N i for all jœn i, and b i+n =d iy CSE554 Deformation I Slide 7 Quadratic Minimization The a i and b i in the distortion term: Lp i p i N i p j jni n E d n Lp i i 2 There are 2n quadratic terms Example with 3 vertices (n=3): n Lp i x ix 2 2 n a i T x b i 2 Lp i y iy 2 a T a T a T a T a T x p x p 2 x p 3 x p y p 2 y p 3 y b x b 2 2x b 3 3x b 4 y b 5 2y b 6 3y p 3 p p 2 a 6 T CSE554 Deformation I Slide 8 9

10 Quadratic Minimization To solve: Re-write in matrix form: min A x B 2 k min a T i x b i 2 where A B a T ª a k T b ª b k is a k by s matrix is a length-k vector CSE554 Deformation I Slide 9 Quadratic Minimization The minimizer is where the partial derivatives are all zero Ax B2 0 x 2A T Ax2A T B A T A x A T B We have: x A T A A T B CSE554 Deformation I Slide 20 0

11 Summary x A T A A T B CSE554 Deformation I Slide 2 Results Deformed A small deformation CSE554 Deformation I Slide 22

12 Results Deformed A larger deformation CSE554 Deformation I Slide 23 Results Deformed Rotation CSE554 Deformation I Slide 24 2

13 Results Deformed Stretching CSE554 Deformation I Slide 25 Results Deformed Shrinking CSE554 Deformation I Slide 26 3

14 Discussion Limitations Local features are skewed, and they don t scale with the model Reason: Laplacian changes with rotation or scale Two bumps that differ by rotation or scale have different Laplacians Which will be penalized by our distortion term p i p i Lp i Lp i Lp i Lp i CSE554 Deformation I Slide 27 A Better Distortion Term Not penalizing rotation and scaling of local features Transforming the original Laplacian vectors before comparing to the deformed Laplacians n E d Lp i T i i 2 T i is a matrix that describes how the local shape around p i is deformed Including translation, rotation and isotropic scaling Collectively called similarity transformations Entries of T i are linear forms of variables p i So that the minimization problem is still quadratic CSE554 Deformation I Slide 28 4

15 Some Questions How to represent similarity transformations as matrices? How to compute T i so that it is linear in the deformed points? We will focus in the derivations of the 2D case 3D results will be briefly presented at the end CSE554 Deformation I Slide 29 Similarity Transforms (2D) Homogeneous coordinates A 2D point: (x,y,) A 2D vector: (x,y,0) A 3D point: (x,y,z,) A 3D vector: (x,y,z,0) CSE554 Deformation I Slide 30 5

16 Similarity Transforms (2D) Translation Cartesian coordinates: vector addition p x v x v y p x Homogeneous coordinates: matrix product p x 0 v x 0 v y 0 0 p x CSE554 Deformation I Slide 3 Similarity Transforms (2D) Isotropic scaling Cartesian coordinates: vector scaling p x s p x Homogeneous coordinates: matrix product p x s s p x CSE554 Deformation I Slide 32 6

17 Similarity Transforms (2D) Rotation Cartesian coordinates: matrix product p x Cos Sin Sin Cos p x Homogeneous coordinates: matrix product p x Cos Sin 0 Sin Cos p x CSE554 Deformation I Slide 33 Similarity Transforms (2D) Summary of elementary similarity transformations To combine transformations: take the product of these matrices p x M p x Trsv Scls 0 v x 0 v y 0 0 s s Translation by vector v Scaling by scalar s Rot Cos Sin 0 Sin Cos Rotation by angle a CSE554 Deformation I Slide 34 7

18 Similarity Transforms (2D) General similarity transformations T a w t x w a t y 0 0 The product of any set of elementary matrices can be written this way Any choice of (a, w, t x, t y ) can be written as a sequence of rotation, isotropic scaling and translation a w t x w a t y 0 0 Trst x,t y.scl Note that a and w can t be both zero a 2 w 2.RotArcTan a w CSE554 Deformation I Slide 35 Computing T i (2D) Suppose we know the deformed locations p i Compute T i as the similarity transform that best fits the neighborhood of p i to that of p i min T i p i p i 2 T i p j p j 2 jni This is a quadratic minimization problem for entries of T i E.g., a, w, t x, t y CSE554 Deformation I Slide 36 8

19 Computing T i (2D) The matrix form of the minimization is: min C a w t x t y p i x p i y p i x p i y ª 2 p i x p i y 0 p i y p i x 0 where C p i x p i y 0 p i y p ix 0 ª ª ª ª is a 2 N i +2 by 4 matrix, and N i ={i, i2, } are indices of neighboring vertices of p i CSE554 Deformation I Slide 37 Computing T i (2D) By quadratic minimization: a w t x t y C T C C T p i x p i y p i x p i y ª Linear expressions of variables (p ix, p iy ) CSE554 Deformation I Slide 38 9

20 Distortion Term (2D) Two parts of each distortion term: Lp i T i i 2 Transformed Laplacian: T i i D a w t x t y D C T C C T p i x p i y p i x p i y ª where D ix i y 0 0 i y ix 0 0 Laplacian of the deformed locations: Lp i L p i x p i y p i x p i y ª where L Ni Ni is a 2 by 2 N i +2 matrix CSE554 Deformation I Slide 39 Distortion Term (2D) Putting together: E d n n Lp i T i i 2 H p i x p i y p i x p i y 2 n H2 p i x p i y p i x p i y 2 where H L D C T C C T and H, H2 are its rows ª ª They form 2n quadratic terms (a i x-b i ) 2 for x = (p x,, p nx, p y,, p ny ) T All b i are zero Each a i can be extracted from H CSE554 Deformation I Slide 40 20

21 Results (2D) Old distortion term New distortion term CSE554 Deformation I Slide 4 Results (2D) Old distortion term New distortion term CSE554 Deformation I Slide 42 2

22 Results (2D) Old distortion term New distortion term CSE554 Deformation I Slide 43 Results (2D) Old distortion term New distortion term CSE554 Deformation I Slide 44 22

23 Registration Use nearest neighbors as corresponding target locations Assuming the source is already close to the target Iterative closest point (ICP). For each point on the source, assign its closest point on the target as its corresponding point. Compute Laplacian-based deformation. A threshold on the closest distance can be used to throw away unlikely correspondences 2. Repeat step () until a termination criteria is met. Maximum iteration or minimum RMSD improvement CSE554 Deformation I Slide 45 Result After rigid alignment iteration of Laplacian 7 iterations of Laplacian Overlaying all curves CSE554 Deformation I Slide 46 23

24 Result Weighting the distortion term E E f w E d large w medium w small w CSE554 Deformation I Slide 47 Similarity Transforms (3D) Elementary transformation matrices To perform a sequence of transformations: take the product of these matrices 0 0 v x 0 0 v Trsv y 0 0 v z p x p y p z M p x p z Scls s s s Translation by vector v Scaling by scalar s RotX, Cos Sin 0 0 Sin Cos Rotation by angle a around X axis CSE554 Deformation I Slide 48 24

25 Similarity Transforms (3D) General similarity transformations in 3D T s h 3 h 2 t x h 3 s h t y h 2 h s t z Approximates the product of a set of elementary matrices Up to a small rotation angle May introduce skewing for large rotations CSE554 Deformation I Slide 49 Computing T i (3D) Assuming known deformation, by quadratic minimization: s h h 2 h 3 t x t y t z C T C C T p i x p i y p i z p i x p i y p i z ª where C p i x 0 p i z p i y 0 0 p i y p i z 0 p i x 0 0 p i z p i y p i x p i x 0 p i z p i y 0 0 p i y p iz 0 p i x 0 0 p i z p i y p i x ª ª ª ª ª ª ª Linear expressions of the deformed points p i C is a 3 N i +3 by 7 matrix CSE554 Deformation I Slide 50 25

26 Distortion Term (3D) Constructing transformed Laplacian: T i i D s h h 2 h 3 t x t y t z p i x p i y p i where D C T C C T z p i x p i y p i z ª where D i x 0 i z i y i y iz 0 i x i z i y i x CSE554 Deformation I Slide 5 Results (3D) Moving targets Stationary targets CSE554 Deformation I Slide 52 26

27 Results (3D) CSE554 Deformation I Slide 53 27