Point Distribution Models

Size: px

Start display at page:

Download "Point Distribution Models"

Clara Blair
5 years ago
Views:

1 Point Distribution Models Jan Kybic winter semester 2007

2 Point distribution models (Cootes et al., 1992) Shape description techniques A family of shapes = mean + eigenvectors (eigenshapes) Shapes described by points

3 Point distribution model procedure Input: M training samples N points each x i = (x i 1, y i 1, x i 2, y i 2,..., x i N, y i N )T Procedure: Rigidly align all shapes Calculate the mean and the covariance matrix PCA (eigen analysis) find principal modes

4 Rigid alignment before alignment after alignment

5 Aligning two shapes x (1) = (x (1) 1, y (1) 1, x (1) 2, y (1) 2,..., x (1) N, y (1) N )T x (2) = (x (2) 1, y (2) 1, x (2) 2, y (2) 2,..., x (2) N, y (2) N )T Find a transformation (rotation, translation, scaling) of x (2) [ ] T (x (2) x (2) [ ] [ ] ) = s R i tx x (2) y (2) + = i s cos θ y (2) i s sin θ t y i x (2) i s sin θ + y (2) + i s cos θ such that a sum of squared distances is minimized [ tx t y ] E = M [ ] [ ] cos θ sin θ x (2) w i s i sin θ cos θ y (2) i i=1 + [ tx t y ] [ ] x (1) i y (1) i 2

6 Aligning two shapes E = M [ ] [ ] cos θ sin θ x (2) w i s i sin θ cos θ y (2) i i=1 + [ tx t y ] [ ] x (1) i y (1) i Minimize E(θ, s, t x, t y ) as min θ min s,tx,t y E θ (s, t x, t y ) Inner minimization wrt s, t x, t y 2 E t x = 0, E t y = 0, E s = 0,

7 Aligning two shapes Inner minimization wrt s, t x, t y E t x = 0, leads to linear equations: E t y = 0, E s = 0, s M i=1 w i q( y i, x i, θ) N t x = M i=1 w i x i s M i=1 w i q( x i, y i, θ) N t y = M i=1 w i y i s ( ) M i=1 w i 2 q 2 (y i, x i, θ) + q 2 M (x i, y i, θ) t x i=1 w i q(y i, x i, θ) M t y i=1 w i q( x i, y i, θ) M M = w i x i q(y i, x i, θ) + w i y i q(x i, y i, θ) i=1 where q(a, b, θ) = a sin θ + b cos θ. i=1

8 Aligning two shapes Inner minimization wrt s, t x, t y Outer minimization wrt θ One dimensional functional minimization, e.g. Brent s routine or golden section search.

9 Aligning all training shapes Before alignment After alignment

10 Aligning all training shapes Align each x i with x 1, for i = 2, 3,..., M, obtaining {x 1, ˆx 2, ˆx 3,..., ˆx M }. Calculate the mean x = [ x 1, ȳ 1, x 2, ȳ 2,..., x N, ȳ N ] of the aligned shapes {x 1, ˆx 2, ˆx 3,..., ˆx M }. x j = 1 M M ˆx j i and ȳ j = 1 M i=1 M i=1 ŷ i j.

11 Aligning all training shapes Align each x i with x 1, for i = 2, 3,..., M, obtaining {x 1, ˆx 2, ˆx 3,..., ˆx M }. Calculate the mean x = [ x 1, ȳ 1, x 2, ȳ 2,..., x N, ȳ N ] of the aligned shapes {x 1, ˆx 2, ˆx 3,..., ˆx M }. Align the mean shape x with x 1. (Necessary for convergence.)

12 Aligning all training shapes Align each x i with x 1, for i = 2, 3,..., M, obtaining {x 1, ˆx 2, ˆx 3,..., ˆx M }. Calculate the mean x = [ x 1, ȳ 1, x 2, ȳ 2,..., x N, ȳ N ] of the aligned shapes {x 1, ˆx 2, ˆx 3,..., ˆx M }. Align the mean shape x with x 1. (Necessary for convergence.) Align ˆx 2, ˆx 3,..., ˆx M to the adjusted mean.

13 Aligning all training shapes Align each x i with x 1, for i = 2, 3,..., M, obtaining {x 1, ˆx 2, ˆx 3,..., ˆx M }. Calculate the mean x = [ x 1, ȳ 1, x 2, ȳ 2,..., x N, ȳ N ] of the aligned shapes {x 1, ˆx 2, ˆx 3,..., ˆx M }. Align the mean shape x with x 1. (Necessary for convergence.) Align ˆx 2, ˆx 3,..., ˆx M to the adjusted mean. Repeat until convergence. We have obtained M (mutually aligned) boundaries ˆx 1, ˆx 2,..., ˆx M and the mean x.

14 Deriving the model We have M boundaries ˆx 1, ˆx 2,..., ˆx M and the mean x. Variation from the mean for each training shape δx i = ˆx i x. Covariance matrix S (2N 2N) S = 1 M M δx i (δx i ) T i=1

15 Deriving the model We have M boundaries ˆx 1, ˆx 2,..., ˆx M and the mean x. Variation from the mean for each training shape δx i = ˆx i x. Covariance matrix S (2N 2N) S = 1 M M δx i (δx i ) T i=1 Principal component analysis

16 Principal component analysis Eigen decomposition Sp i = λ i p i P = [ p 1 p 2 p 3... p 2N] We know eigenvalues λ i are real because S is symmetric, positive definite. Eigenvectors (principal components) p i are orthogonal, so P is a basis and any vector x can be represented as x = x + P b

17 Principal component analysis Eigen decomposition Sp i = λ i p i P = [ p 1 p 2 p 3... p 2N] We know eigenvalues λ i are real because S is symmetric, positive definite. Eigenvectors (principal components) p i are orthogonal, so P is a basis and any vector x can be represented as x = x + P b Order eigenvectors p i and eigenvalues λ i such that λ 1 λ 2 λ 3... λ 2N. Most changes are then described by the first few eigenvectors.

18 Principal component analysis Eigen decomposition Sp i = λ i p i P = [ p 1 p 2 p 3... p 2N] x = x + P b Order eigenvectors p i and eigenvalues λ i such that λ 1 λ 2 λ 3... λ 2N. Most changes are then described by the first few eigenvectors. Consider only K largest eigenvalues. Approximation x x + P K b K with P K = [ p 1 p 2 p 3... p K ] b t = [ b 1, b 2,..., b K ] T Choose the smallest K, such that K i=1 λ i α N i=1 λ i.

19 Point distribution model Input: M non-aligned boundaries x 1, x 2,..., x M. Output: mean x and reduced eigvector matrix P K

20 Point distribution model Input: M non-aligned boundaries x 1, x 2,..., x M. Output: mean x and reduced eigvector matrix P K New shape generation: For well-behaved shapes x = x + P K b K 3 λ i b i 3 λ i

21 PDM example Before alignment After alignment, mean shape

22 PDM example First mode Second mode The mean shape is in red, the shape corresponding to 3 λ in blue and the shape corresponding to +3 λ in green.

23 Active shape models PDM Image to fit Fit a learned point distribution model (PDM) to a given image.

24 Pose and shape parameters Point distribution model (PDM) consists of mean p eigenvectors P

25 Pose and shape parameters Point distribution model (PDM) consists of mean p eigenvectors P Fitted model given by: pose parameters: θ, s, t x, t y shape parameters: b [ xi y i p = Pb + p p = [ p 1 p 2... p N ] p = [ x ] 1 ỹ 1 x 2 ỹ 2... x N ỹ N ] [ ] ] [ ] cos θ sin θ [ xi tx = s + sin θ cos θ ỹ i t y p = [ ] x 1 y 1 x 2 y 2... x N y N

26 Fitting p = T s,θ,tx,t y ( p) = s Q θ p + r tx,t y where p = Pb + p Calculate an edge map of the image For each landmark p i we find a line normal to the shape contour. New position p i is the maximum of the edge map on the line. (If maximum too weak, no change.)

27 Fitting New position p i is the maximum of the edge map on the line. (If maximum too weak, no change.)

28 Fitting p = T s,θ,tx,t y ( p) = s Q θ p + r tx,t y where p = Pb + p Calculate an edge map of the image For each landmark p i we find a line normal to the shape contour. New position p i is the maximum of the edge map on the line. (If maximum too weak, no change.) Adjust pose parameters θ, s, t x, t y by the alignment algorithm.

29 Fitting p = T s,θ,tx,t y ( p) = s Q θ p + r tx,t y where p = Pb + p Calculate an edge map of the image For each landmark p i we find a line normal to the shape contour. New position p i is the maximum of the edge map on the line. (If maximum too weak, no change.) Adjust pose parameters θ, s, t x, t y by the alignment algorithm. Adjust shape parameters b as follows: p i = T 1 (p i) b = P 1 ( p i p) = P T ( p i p)

30 Fitting p = T s,θ,tx,t y ( p) = s Q θ p + r tx,t y where p = Pb + p Calculate an edge map of the image For each landmark p i we find a line normal to the shape contour. New position p i is the maximum of the edge map on the line. (If maximum too weak, no change.) Adjust pose parameters θ, s, t x, t y by the alignment algorithm. Adjust shape parameters b as follows: p i = T 1 (p i) Repeat until convergence b = P 1 ( p i p) = P T ( p i p)

31 Example Hand image Edge map + initial shape. The image is smoothed and a gradient magnitude image calculated in each color channel. The edge map is a maximum over the three color channels, thresholded to obtain a clean background.

32 Example First iteration Final postition

Principal Component Analysis

Principal Component Analysis CSci 5525: Machine Learning Dec 3, 2008 The Main Idea Given a dataset X = {x 1,..., x N } The Main Idea Given a dataset X = {x 1,..., x N } Find a low-dimensional linear projection The Main Idea Given