Polynomial Regression on Riemannian Manifolds

Transcription

1 Polynomial Regression on Riemannian Manifolds Jacob Hinkle, Tom Fletcher, Sarang Joshi May 11, 212 arxiv:

2 Nonparametric Regression Number of parameters tied to amount of data present Example: kernel regression on images using diffeomorphisms (Davis27) Polynomial Regression on Riemannian Manifolds 2

3 Parametric Regression Small number of parameters can be estimated more efficiently Fletcher 211 Geodesic regression (Niethammer211, Fletcher211) has recently received attention. Polynomial Regression on Riemannian Manifolds 3

4 Polynomial Regression Dependent Variable Independent Variable Polynomials provide a more flexible framework for parametric regression on Riemannian manifolds Polynomial Regression on Riemannian Manifolds 4

5 Riemannian Polynomials At least three ways to define polynomial in R d Algebraic: γ(t) = c + 1 1! c 1t + 1 2! c 2t k! c kt k Polynomial Regression on Riemannian Manifolds 5

6 Riemannian Polynomials At least three ways to define polynomial in R d Algebraic: γ(t) = c + 1 1! c 1t + 1 2! c 2t k! c kt k Variational: γ = argmin ϕ ( ) k+1 d 2 dt ϕ(t) 2 dt s.t. BC/ICs Polynomial Regression on Riemannian Manifolds 5

7 Riemannian Polynomials At least three ways to define polynomial in R d Algebraic: γ(t) = c + 1 1! c 1t + 1 2! c 2t k! c kt k Variational: γ = argmin ϕ ( d dt ) k+1 2 ϕ(t) 2 dt s.t. BC/ICs Differential: ( d ) k+1 ( dt γ(t) = s.t. initial conditions d ) i dt γ() = ci Polynomial Regression on Riemannian Manifolds 5

8 Riemannian Polynomials At least three ways to define polynomial in R d Algebraic: γ(t) = c + 1 1! c 1t + 1 2! c 2t k! c kt k Variational: γ = argmin ϕ ( d dt ) k+1 2 ϕ(t) 2 dt s.t. BC/ICs Differential: ( d ) k+1 ( dt γ(t) = s.t. initial conditions d ) i dt γ() = ci Covariant derivative: replace d dt of vectors with γ Polynomial Regression on Riemannian Manifolds 5

9 Riemannian Polynomials At least three ways to define polynomial in R d Algebraic: γ(t) = c + 1 1! c 1t + 1 2! c 2t k! c kt k Variational: γ = argmin ϕ ( d dt ) k+1 2 ϕ(t) 2 dt s.t. BC/ICs Differential: ( d ) k+1 ( dt γ(t) = s.t. initial conditions d ) i dt γ() = ci Covariant derivative: replace d dt of vectors with γ Geodesic (k = 1) has both forms γ = argmin ϕ ϕ(t) 2 dt γ γ = s.t. initial conditions γ(), γ() Well-studied (Fletcher, Younes, Trouve, ) Polynomial Regression on Riemannian Manifolds 5

10 Riemannian Polynomials At least three ways to define polynomial in R d Algebraic: γ(t) = c + 1 1! c 1t + 1 2! c 2t k! c kt k Variational: γ = argmin ϕ ( d dt ) k+1 2 ϕ(t) 2 dt s.t. BC/ICs Differential: ( d ) k+1 ( dt γ(t) = s.t. initial conditions d ) i dt γ() = ci Covariant derivative: replace d dt of vectors with γ Cubic spline satisfies (Noakes1989, Leite, Machado, ) γ = argmin ϕ ϕ ϕ(t) 2 dt Euler-Lagrange equation: ( γ ) 3 γ = R( γ, γ γ) γ Shape splines (Trouve-Vialard) Polynomial Regression on Riemannian Manifolds 5

11 Riemannian Polynomials At least three ways to define polynomial in R d Algebraic: γ(t) = c + 1 1! c 1t + 1 2! c 2t k! c kt k Variational: γ = argmin ϕ ( d dt ) k+1 2 ϕ(t) 2 dt s.t. BC/ICs Differential: ( d ) k+1 ( dt γ(t) = s.t. initial conditions d ) i dt γ() = ci Covariant derivative: replace d dt of vectors with γ k-order polynomial satisfies ( γ ) k γ = subject to initial conditions γ(), ( γ ) i γ(), i =,..., k 1 Introduced via rolling maps by Jupp&Kent1987 Studied by Leite (28), in rolling map setting Polynomial Regression on Riemannian Manifolds 5

12 Riemannian Polynomials At least three ways to define polynomial in R d Algebraic: γ(t) = c + 1 1! c 1t + 1 2! c 2t k! c kt k Variational: γ = argmin ϕ ( d dt ) k+1 2 ϕ(t) 2 dt s.t. BC/ICs Differential: ( d ) k+1 ( dt γ(t) = s.t. initial conditions d ) i dt γ() = ci Covariant derivative: replace d dt of vectors with γ k-order polynomial satisfies ( γ ) k γ = subject to initial conditions γ(), ( γ ) i γ(), i =,..., k 1 Introduced via rolling maps by Jupp&Kent1987 Studied by Leite (28), in rolling map setting Polynomial Regression on Riemannian Manifolds 5

13 Riemannian Polynomials At least three ways to define polynomial in R d Algebraic: γ(t) = c + 1 1! c 1t + 1 2! c 2t k! c kt k Variational: γ = argmin ϕ ( d dt ) k+1 2 ϕ(t) 2 dt s.t. BC/ICs Differential: ( d ) k+1 ( dt γ(t) = s.t. initial conditions d ) i dt γ() = ci Covariant derivative: replace d dt of vectors with γ k-order polynomial satisfies ( γ ) k γ = subject to initial conditions γ(), ( γ ) i γ(), i =,..., k 1 Introduced via rolling maps by Jupp&Kent1987 Studied by Leite (28), in rolling map setting Polynomial Regression on Riemannian Manifolds 5

14 Rolling maps Leite 28 Unroll curve α on manifold to curve α dev on R d without twisting or slipping. Then ( ) d k ( α ) k α = α dev = dt Polynomial Regression on Riemannian Manifolds 6

15 Rolling maps Leite 28 Unroll curve α on manifold to curve α dev on R d without twisting or slipping. Then ( ) d k ( α ) k α = α dev = dt Unknown whether this satisfies a variational principle Polynomial Regression on Riemannian Manifolds 6

16 Riemannian Polynomials Generate via forward evolution of linearized system of first-order covariant ODEs Forward Polynomial Evolution repeat w v 1 for i = 1,..., k 1 do v i ParallelTransport γ ( tw, v i + tv i+1 ) end for v k ParallelTransport γ ( tw, v k ) γ Exp γ ( tw) t t + t until t = T Parametrized by ICs: γ() position v 1 () velocity v 2 () acceleration v 3 () jerk Polynomial Regression on Riemannian Manifolds 7

17 Polynomial Regression ( γ ) k γ = becomes linearized system γ = v 1 γ v i = v i+1 i = 1,..., k 1 γ v k =. Want to find initial conditions for this ODE that minimize E(γ) = g i (γ(t i )) i=1 Polynomial Regression on Riemannian Manifolds 8

18 Lagrange multiplier (adjoint) vector fields λ i along γ: E (γ, {v i }, {λ i }) = g i (γ(t i )) + i=1 k 1 + i=1 λ, γ v 1 dt λ i, γ v i v i+1 dt + Euler-Lagrange for {λ i } gives forward system. Vector field integration by parts: λ i, γ v i dt = [ λ i, v i ] T γ λ i, v i dt λ k, γ v k dt Polynomial Regression on Riemannian Manifolds 9

19 Lagrange multiplier (adjoint) vector fields λ i along γ: E (γ, {v i }, {λ i }) = g i (γ(t i )) + i=1 k 1 + i=1 λ, γ v 1 dt λ i, γ v i v i+1 dt + Euler-Lagrange for {λ i } gives forward system. Vector field integration by parts: λ i, γ v i dt = [ λ i, v i ] T γ λ i, v i dt λ k, γ v k dt Polynomial Regression on Riemannian Manifolds 9

20 Rewrite using integration by parts E (γ, {v i }, {λ i }) = g i (γ(t i )) + i=1 k 1 k 1 + [ λ i, v i ] T i=1 k 1 i=1 + [ λ k, v k ] T λ, γ v 1 dt i=1 λ i, v i+1 dt γ λ i, v i dt γ λ k, v k dt Polynomial Regression on Riemannian Manifolds 1

21 Rewrite using integration by parts E (γ, {v i }, {λ i }) = g i (γ(t i )) + i=1 k 1 k 1 + [ λ i, v i ] T i=1 k 1 i=1 + [ λ k, v k ] T So variation w.r.t. {v i } gives λ, γ v 1 dt i=1 λ i, v i+1 dt δ vi E = = γ λ i λ i 1 δ vi (T )E = = λ i (T ) δ vi ()E = λ i () γ λ i, v i dt γ λ k, v k dt Polynomial Regression on Riemannian Manifolds 1

22 Variation with respect to the curve γ: Let {γ s : s ( ɛ, ɛ)} a smooth family of curves, with: γ = γ W (t) := d ds γ s(t) s= Extend v i, λ i away from curve via parallel transport: Then W v i = W λ i = δ γ E (γ, {v i }, {λ i }), W dt = d ds E (γ s, {v i }, {λ i }) s= Polynomial Regression on Riemannian Manifolds 11

23 For any smooth family of curves γ s (t), we have [ d ds γ s(t), d ] dt γ s(t) = [W, γ s ] = so We also need the Leibniz rule W γ = γ W. d ds X, Y s= = W X, Y + X, W Y, And the Riemann curvature tensor R(X, Y )Z = X Y Z Y X Z [X,Y ] Z W γ Z = γ W Z + R(W, γ)z Polynomial Regression on Riemannian Manifolds 12

24 For first term, T 1 = λ, γ s dt d ds T 1(γ s ) s= = d ds = = λ, γ s dt s= W λ, γ s + λ, W γ s dt s=, γ s + λ, γ W dt s= = [ λ, W ] T Variation of this term with respect to γ is δ γ(t) T 1 = γ λ δ γ(t ) T 1 = = λ (T ) δ γ() T 1 = λ () γ λ, W dt Polynomial Regression on Riemannian Manifolds 13

25 Now do the same with another term T i d ds T i(γ s ) s= = d ds = = + = + λ i, γ v i dt W λ i, γ v i + λ i, W γ v i dt λ i, γ W v i + R(W, γ)v i dt R(λ i, v i ) γ, W dt where we used Bianchi identities to rearrange the curvature term. So δ γ(t) T i = R(λ i, v i ) γ Polynomial Regression on Riemannian Manifolds 14

26 Combine all terms to get adjoint equations k γ λ = δ(t t i )(grad g i (γ(t))) + R(λ i, v i )v 1 i=1 i=1 γ λ i = λ i 1 Initialization for λ i at t = T is λ i (T ) =, Parameter gradients are δ γ() E = λ () δ vi ()E = λ i () Polynomial Regression on Riemannian Manifolds 15

27 Combine all terms to get adjoint equations k γ λ = δ(t t i )(grad g i (γ(t))) + R(λ i, v i )v 1 i=1 i=1 γ λ i = λ i 1 Initialization for λ i at t = T is λ i (T ) =, Parameter gradients are δ γ() E = λ () δ vi ()E = λ i () Polynomial Regression on Riemannian Manifolds 15

28 Combine all terms to get adjoint equations k γ λ = δ(t t i )(grad g i (γ(t))) + R(λ i, v i )v 1 i=1 i=1 γ λ i = λ i 1 Initialization for λ i at t = T is λ i (T ) =, Parameter gradients are δ γ() E = λ () δ vi ()E = λ i () Typically, g i (γ) = d(γ, y i ) 2, so that (grad g i (γ)) = Log γ y i Polynomial Regression on Riemannian Manifolds 15

29 Polynomial Regression Algorithm repeat Integrate γ, {v i } forward to t = T Initialize λ i (T ) =, i =,..., k Integrate {λ i } via adjoint equations back to t = Gradient descent step: γ() n+1 = Exp γ() n(ɛλ ()) v i () n+1 = ParTrans γ() n(ɛλ (), v i () n + ɛλ i ()) until convergence Polynomial Regression on Riemannian Manifolds 16

30 Special Case: Geodesic (k = 1) Adjoint system is γ λ = δ(t t i )(grad g i (γ(t))) + R(λ 1, v 1 )v 1 i=1 γ λ 1 = λ Between data points this is ( γ ) 2 λ 1 = R(λ 1, γ) γ This is the Jacobi equation, λ 1 is a Jacobi field. Polynomial Regression on Riemannian Manifolds 17

31 Kendall Shape Space Space of N landmarks in d dimensions, R Nd, modulo translation, scale, rotation Prevents skewed statistics due to similarity transformed data d = 2, complex projective space C P N 2 Polynomial Regression on Riemannian Manifolds 18

32 Kendall Shape Space Geometry (d = 2) Center point-set and scale so that N i=1 x i 2 = 1 (resulting object is called a preshape) Preshapes lie on sphere S 2N 1, represented as vectors in (R 2 ) N = C N Riemannian submersion from preshape to shape space: vertical direction holds rotations of R 2 Exponential and log map available in closed form (for d = 2) Polynomial Regression on Riemannian Manifolds 19

33 Covariant derivative in shape space in terms of preshape (O Neill1966): X Y = H X Y Vertical direction is JN, where N is outward unit normal at the preshape, J is almost complex structure on C N. So parallel transport in small steps in upstairs space then do horizontal projection. Polynomial Regression on Riemannian Manifolds 2

34 Curvature on preshape sphere S 2N 1, R(X, Y )Z is: R(X, Y )Z = X, Z Y Y, Z X For curvature, need first fundamental form A. For horizontal vf s X, Y, Curvature downstairs is A X Y = 1 2 V[X, Y ] R (X, Y )Z, H = R(X, Y )Z, H + 2 A X Y, A Z H A Y Z, A X H A Z X, A Y H Polynomial Regression on Riemannian Manifolds 21

35 First fundamental form is (O Neill) A X Y = X, JY JN Adjoint of A Z : A X Y, A Z H = J X, JY Z, H Curvature then is R (X, Y )Z = R(X, Y )Z 2J X, JY Z + J Y, JZ X + J Z, JX Y Polynomial Regression on Riemannian Manifolds 22

36 Bookstein Rat Calivarium Growth.3 B landmark points 18 subjects 8 ages D.1 C A A B C D k R Polynomial Regression on Riemannian Manifolds 23

37 Corpus Collosum Aging ( Geodesic Fletcher 211 N = 32 patients Age range landmarks using ShapeWorks sci.utah.edu k R Quadratic Cubic Polynomial Regression on Riemannian Manifolds 24

38 Corpus Collosum Aging γ() γ γ() ( γ ) 2 γ() Initial conditions are collinear, implying time reparametrization Polynomial Regression on Riemannian Manifolds 25

39 Landmark Space Space L of N points in R d. Geodesic equations: d dt x i = γ( x i x j 2 )α j j=1 d dt α i = 2 (x i x j )γ ( x i x j ) 2 αi T α j j=1 Usually use Gaussian kernel γ(r) = e r/(2σ2 ) x L and α T x L is a covector (momentum) Polynomial Regression on Riemannian Manifolds 26

40 Landmark Space Space L of N points in R d. Geodesic equations: d dt x i = γ( x i x j 2 )α j j=1 d dt α i = 2 (x i x j )γ ( x i x j ) 2 αi T α j j=1 Usually use Gaussian kernel γ(r) = e r/(2σ2 ) x L and α T x L is a covector (momentum) Polynomial Regression on Riemannian Manifolds 26

41 Have simple formula for cometric g ij (the kernel) Parallel transport in terms of covectors, cometric: d dt β l = 1 2 g ilg in,j g jm (α m β n α n β m ) 1 2 gmn,l α m β n Curvature more complicated (Mario s Formula): 2R ursv = g ur,sv g rv,us + g rs,uv + g uv,rs + 2Γ rv ρ Γ us σ g ρσ 2Γ rs ρ Γ uv σ g ρσ + g rλ,u g λµ g µv,s g rλ,u g λµ g µs,v + g uλ,r g λµ g µs,v g uλ,r g λµ g µv,s + g rλ,s g λµ g µv,u + g uλ,v g λµ g µs,r g rλ,v g λµ g µs,u g uλ,s g λµ g µv,r. Polynomial Regression on Riemannian Manifolds 27

42 Landmark parallel transport in momenta (Younes28): d dt β i = K 1( N (x i x j ) T ((Kβ) i (Kβ) j )γ ( x i x j 2 )α j j=1 ) (x i x j ) T ((Kα) i (Kα) j )γ ( x i x j 2 )β j j=1 (x i x j )γ ( x i x j 2 )(αj T β i + αi T β j ) j=1 This is enough to integrate polynomials Polynomial Regression on Riemannian Manifolds 28

43 For curvature, need Christoffel symbols and their derivatives: (Γ(u, v)) i = (x i x j) T (v i v j)γ ( x i x j 2 )(K 1 u) j j=1 (x i x j) T (u i u j)γ ( x i x j 2 )(K 1 v) j j=1 + γ( x i x j 2 ) (x j x k )γ ( x j x k 2 )((K 1 u) T k (K 1 v) j + (K 1 j=1 k=1 Take derivative with respect to x, and combine using R l ijk = Γl ki,j Γl ji,k + Γl jmγ m ki Γl km Γm ji Polynomial Regression on Riemannian Manifolds 29

44 ((DΓ(u, v))w)i = (wi wj) T (ui uj)γ ( xi xj 2 )(K 1 v)j j=1 + 2 (xi xj) T (ui uj)(xi xj) T (wi wj)γ ( xi xj 2 )(K 1 v)j j=1 + (xi xj) T (ui uj)γ ( xi xj 2 )(( d dɛ K 1 )v)j j=1 + (wi wj) T (vi vj)γ ( xi xj 2 )(K 1 u)j j=1 + 2 (xi xj) T (vi vj)(xi xj) T (wi wj)γ ( xi xj 2 )(K 1 u)j j=1 + (xi xj) T (vi vj)γ ( xi xj 2 )(( d dɛ K 1 )u)j j=1 2 (xi xj) T (wi wj)γ ( xi xj 2 ) (xj xk)γ ( xj xk 2 )((K 1 u) T k (K 1 v)j + (K 1 u) T j (K 1 v)k) j=1 k=1 γ( xi xj 2 ) (wj wk)γ ( xj xk 2 )((K 1 u) T k (K 1 v)j + (K 1 u) T j (K 1 v)k) j=1 k=1 2 γ( xi xj 2 ) (xj xk)(xj xk) T (wj wk)γ ( xj xk 2 )((K 1 u) T k (K 1 v)j + (K 1 u) T j (K 1 v)k) j=1 k=1 γ( xi xj 2 ) (xj xk)γ ( xj xk 2 ) j=1 k=1 (( d dɛ K 1 u) T k (K 1 v)j + (K 1 u) T k ( d dɛ K 1 v)j + ( d dɛ K 1 u) T j (K 1 v)k + (K 1 u) T j ( d dɛ K 1 v)k) ( ( d ) dɛ K 1 )v = (K 1 d i dɛ KK 1 v)i N = 2(K 1 (xk xj) T (wk wj)γ ( xk xj 2 )(K 1 v)j j=1 Polynomial Regression on Riemannian Manifolds 3

45 Landmark Regression Results Same Bookstein rat data. Procrustes alignment, no scaling R 2 =.92 geodesic,.94 quadratic Polynomial Regression on Riemannian Manifolds 31

46 Polynomial Regression on Riemannian Manifolds 32

47 Thank You!