Body Models I-2. Gerard Pons-Moll and Bernt Schiele Max Planck Institute for Informatics

Transcription

1 Body Models I-2 Gerard Pons-Moll and Bernt Schele Max Planck Insttute for Informatcs December 18, 2017

2 What s mssng Gven correspondences, we can fnd the optmal rgd algnment wth Procrustes. PROBLEMS: How do we fnd the correspondences between shapes? How do we algn shapes non-rgdly?

3 Today Optmsng algnment and correspondences usng Iteratve Closest Pont (ICP). Algnment through gradent descent based optmsaton.

4 Ideas?

5 Ideas?

6 ? Ideas?

7 Ideas The dea was to mnmse the sum of dstances between the one set of ponts and the other set, transformed E X ksrx + t y k 2 X? kf(x ) y k 2 compact notaton: f contans translaton, rotaton and sotropc scale What f we make up some reasonable correspondences? x j+1 teraton = arg mn kf j (x) y k 2 x2x X f j+1 = arg mn f kf(x j+1 ) y k 2 Gven current best transformaton, whch are the closest correspondences? Gven current best correspondences, whch s the best transformaton?

8 Ideas The dea was to mnmse the sum of dstances between the one set of ponts and the other set, transformed E X ksrx + t y k 2 X? kf(x ) y k 2 compact notaton: f contans translaton, rotaton and sotropc scale What f we make up some reasonable correspondences? x j+1 teraton = arg mn kf j (x) y k 2 x2x X f j+1 = arg mn f kf(x j+1 ) y k 2 Gven current best transformaton, whch are the closest correspondences? Gven current best correspondences, whch s the best transformaton?

9 Ideas The dea was to mnmse the sum of dstances between the one set of ponts and the other set, transformed E X ksrx + t y k 2 X? kf(x ) y k 2 compact notaton: f contans translaton, rotaton and sotropc scale What f we fnd some reasonable correspondences? x j+1 teraton orgnal unsorted ponts = arg mn kf j (x) y k 2 x2x X f j+1 = arg mn f kf(x j+1 ) y k 2 Gven current best transformaton, whch are the closest correspondences? Gven current best correspondences, whch s the best transformaton?

10 Ideas The dea was to mnmse the sum of dstances between the one set of ponts and the other set, transformed E X ksrx + t y k 2 X? kf(x ) y k 2 compact notaton: f contans translaton, rotaton and sotropc scale What f we make up some reasonable correspondences? x j+1 teraton = arg mn kf j (x) y k 2 x2x X f j+1 = arg mn f kf(x j+1 ) y k 2 Gven current best transformaton, whch are the closest correspondences? Gven current best correspondences, whch s the best transformaton?

11 Make up reasonable correspondences X Y

12 Make up reasonable correspondences X f 0 (X) x 1 0 y 0 Neutral ntalsaton. Intalsng t to algn centrods should work better! f 0 = {R = I, t = 0,s=1} x 1 0 = arg mn x2x kf 0 (x) y 0 k 2

13 Make up reasonable correspondences x 1 0 x 1 1 y 0 y 1 f 0 = {R = I, t = 0,s=1} x 1 = arg mn x2x kf 0 (x) y k 2

14 Solve for the best transformaton solve wth procrustes x 1 = arg mn kf 0 (x) y k 2 x2x f 1 X = arg mn kf(x 1 ) y k 2 f

15 f 1 (X) Apply t

16 and terate! f 1 (X) f 1 = arg mn f X kf(x 1 ) y k 2 x 2 = arg mn x2x kf 1 (x) y k 2

17 and terate! f j (X) x j+1 f j = arg mn f X kf(x j ) y k 2 = arg mn x2x kf j (x) y k 2

18 and terate! f j (X) x j+1 f j = arg mn f X kf(x j ) y k 2 = arg mn x2x kf j (x) y k 2

19 and terate! f j (X) x j+1 f j = arg mn f X kf(x j ) y k 2 = arg mn x2x kf j (x) y k 2

20 and terate! f j (X) x j+1 f j = arg mn f X kf(x j ) y k 2 = arg mn x2x kf j (x) y k 2

21 and terate! f j (X) x j+1 f j = arg mn f X kf(x j ) y k 2 = arg mn x2x kf j (x) y k 2

22 Iteratve Closest Pont (ICP) typcally better than 0 1. ntalse 2. compute correspondences accordng to current best transform 3. compute optmal transformaton ( s, R, t )wth Procrustes 4. termnate f converged (error below a threshold), otherwse terate 5. converges to local mnma f 0 = {R = I, t = x j+1 f j+1 = arg mn f P y N P x N,s=1} = arg mn x2x kf j (x) y k 2 X kf(x j+1 ) y k 2

23 Iteratve Closest Pont (ICP) 1. ntalse 2. compute correspondences accordng to current best transform 3. compute optmal transformaton ( s, R, t )wth Procrustes 4. termnate f converged (error below a threshold), otherwse terate 5. converges to local mnma f 0 = {R = I, t = x j+1 f j+1 = arg mn f P y N P x N,s=1} = arg mn x2x kf j (x) y k 2 X kf(x j+1 ) y k 2

24 Iteratve Closest Pont (ICP) 1. ntalse 2. compute correspondences accordng to current best transform 3. compute optmal transformaton ( s, R, t ) wth Procrustes 4. termnate f converged (error below a threshold), otherwse terate 5. converges to local mnma f 0 = {R = I, t = x j+1 f j+1 = arg mn f P y N P x N,s=1} = arg mn x2x kf j (x) y k 2 X kf(x j+1 ) y k 2

25 Iteratve Closest Pont (ICP) 1. ntalse 2. compute correspondences accordng to current best transform 3. compute optmal transformaton ( s, R, t )wth Procrustes 4. termnate f converged (error below a threshold), otherwse terate 5. converges to local mnma f 0 = {R = I, t = x j+1 f j+1 = arg mn f P y N P x N,s=1} = arg mn x2x kf j (x) y k 2 X kf(x j+1 ) y k 2

26 Iteratve Closest Pont (ICP) 1. ntalse 2. compute correspondences accordng to current best transform 3. compute optmal transformaton ( s, R, t )wth Procrustes 4. termnate f converged (error below a threshold), otherwse terate (go to step 2) 5. converges to local mnma f 0 = {R = I, t = x j+1 f j+1 = arg mn f P y N P x N,s=1} = arg mn x2x kf j (x) y k 2 X kf(x j+1 ) y k 2

27 Is ICP the best we can do? teraton j compute closest ponts compute optmal transformaton wth Procrustes apply transformaton termnate f converged, otherwse terate

28 Brute force s n^2 Closest ponts

29 Closest ponts Tree based methods (e.g. kdtree) have avg. complexty log(n) Random pont samplng also reduces the runnng tme

30 Is ICP the best we can do? teraton j compute closest ponts compute optmal transformaton wth Procrustes apply transformaton termnate f converged, otherwse terate

31 Best transformaton? Procrustes gves us the optmal rgd transformaton and scale gven correspondences What f the deformaton model s not rgd? Can we generalse ICP to non-rgd deformaton?

32 Iteratve Closest Pont (ICP) teraton j compute closest ponts In whch drecton should I move? compute optmal transformaton wth Procrustes apply transformaton termnate f converged, otherwse terate

33 Iteratve Closest Pont (ICP) teraton j compute closest ponts In whch drecton should I move? compute optmal transformaton wth Procrustes apply transformaton compute a transform that reduces the error termnate f converged, otherwse terate

34 Gradent-based ICP teraton j compute closest ponts Jacoban of dstance-based energy compute optmal transformaton wth Procrustes apply transformaton compute descent step by lnearsng the energy termnate f converged, otherwse terate

35 Gradent-based ICP arg mn f E(f) = arg mn f X kf(x j+1 ) y k 2 If f s a rgd transformaton we can solve ths mnmsaton usng Procrustes If f s a general non-lnear functon? Gradent descent: f k+1 = f k r f E(f) For least squares, s there a better optmsaton method? yes: Gauss-Newton based methods.

36 Gradent-based ICP 1. Energy: E X k mn x f(x) y k 2 2. Consder the correspondences fxed n each teraton j+1 x j+1 = arg mn x2x kf j (x) y k 2 3. Compute gradent of the energy around current estmaton g j+1 = re(f j ) 4. Apply step (gradent descent, dogleg, LM, BFGS ) f j+1 = k step (g 0...j+1,f 0...j ) (for example f j+1 = f j g j+1 ) 5. termnate f converged, otherwse terate (go to step 2)

37 Try t!

38 Gradent-based ICP Energy: Consder the correspondences fxed n each teraton j+1 Compute gradent of the energy around current estmaton Apply step (gradent descent, dogleg, LM, BFGS ) termnate f converged, otherwse terate

39 Gradent-based ICP E X k mn x f(x) y k 2 gradent: dervatve of the sum of squared dstances between target ponts and scale, rotated and translated source ponts, wth respect to the the scale, rotaton and translaton Each dervatve s easy Who takes the chalk and wrtes t down? g j+1 = re(f j ) Chan rule and automatc dfferentaton!

40 Gradent-based ICP E X k mn x f(x) y k 2 gradent: dervatve of the sum of squared dstances between target ponts and scale, rotated and translated source ponts, wth respect to the the scale, rotaton and translaton Each dervatve s easy Who takes the chalk and wrtes t down? g j+1 = re(f j ) Chan rule and automatc dfferentaton!

41 Gradent-based ICP E X k mn x f(x) y k 2 gradent: dervatve of the sum of squared dstances between target ponts and scale, rotated and translated source ponts, wth respect to the the scale, rotaton and translaton Each dervatve s easy Who takes the chalk and wrtes t down? g j+1 = re(f j ) Chan rule and automatc dfferentaton!

42 Chumpy Automatc dfferentaton compatble wth numpy Jacoban: matrx encodng partal dervatve of outputs (rows) 2 b 1 b wth respect to nputs (columns) c c n The Jacobans of each operaton are encoded for you The composed Jacoban s computed wth the chan rule J a b (c) =J a (b(c))j b (c) J = db dc = 6 4. b m c b m cn 3 7 5

43 Chumpy E = X ksrx + t y k 2 wrte as f t was numpy code results n expresson tree wth jacobans avalable at each step

44 Gradent-based ICP Energy: Consder the correspondences fxed n each teraton j+1 Compute gradent of the energy around current estmaton Apply step (gradent descent, dogleg, LM, BFGS ) termnate f converged, otherwse terate f j+1 = k step (g 0...j+1,f 0...j )

45 Gradent-based ICP However, lots of standard ways are avalable n scentfc lbrares lke scpy And chumpy ntegrates well wth t Mnmsaton n a sngle lne: ch.mnmze(fun=energy, x0=[scale, rot, trans], method= dogleg')

46 Why Gradent-based ICP? Formulaton s much more generc: the energy can ncorporate other terms, more parameters, etc A lot of avalable software for solvng ths least squares problem (cvx, ceres, ) However, the resultng energy s non-convex for general deformaton models. Optmsaton can get trapped n local mnma.

47 Take-home message Procrustes s optmal gven optmal correspondences and for rgd algnment problems. For other problems: We can compute correspondences and solve for the best transformaton teratvely wth Iteratve Closest Pont (ICP)