Invariant Histograms and Signatures for Object Recognition and Symmetry Detection

Transcription

1 Invariant Histograms and Signatures for Object Recognition and Symmetry Detection Peter J. Olver University of Minnesota olver North Carolina, October, 2011

2 References Boutin, M., & Kemper, G., On reconstructing n-point configurations from the distribution of distances or areas, Adv. Appl. Math. 32 (2004) Brinkman, D., & Olver, P.J., Invariant histograms, Amer. Math. Monthly 118 (2011) Calabi, E., Olver, P.J., Shakiban, C., Tannenbaum, A., & Haker, S., Differential and numerically invariant signature curves applied to object recognition, Int. J. Computer Vision 26 (1998)

3 The Distance Histogram Definition. The distance histogram of a finite set of points P = {z 1,..., z n } V is the function η P (r) = # { (i, j) 1 i < j n, d(zi, z j ) = r }.

4 The Distance Set The support of the histogram function, supp η P = P R + is the distance set of P. Erdös distinct distances conjecture (1946): If P R m, then # P c m,ε (# P ) 2/m ε

5 Characterization of Point Sets Note: If P = g P is obtained from P R m by a rigid motion g E(n), then they have the same distance histogram: η P = η P. Question: Can one uniquely characterize, up to rigid motion, a set of points P {z 1,..., z n } R m by its distance histogram? = Tinkertoy problem.

6 Yes: η = 1, 1, 1, 1, 2, 2.

7 No: Kite Trapezoid η = 2, 2, 2, 10, 10, 4.

8 No: P = {0, 1, 4, 10, 12, 17} Q = {0, 1, 8, 11, 13, 17} R η = 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 17 = G. Bloom, J. Comb. Theory, Ser. A 22 (1977)

9 Theorem. (Boutin Kemper) Suppose n 3 or n m + 2. Then there is a Zariski dense open subset in the space of n point configurations in R m that are uniquely characterized, up to rigid motion, by their distance histograms. = M. Boutin, G. Kemper, Adv. Appl. Math. 32 (2004)

10 Limiting Curve Histogram

11 Limiting Curve Histogram

12 Limiting Curve Histogram

13 Sample Point Histograms Cumulative distance histogram: n = #P : Λ P (r) = 1 n + 2 n 2 s r η P (s) = 1 n 2 # { (i, j) d(zi, z j ) r }, Note η(r) = 1 2 n2 [ Λ P (r) Λ P (r δ) ] δ 1. Local distance histogram: λ P (r, z) = 1 n # { j d(z, zj ) r } = 1 n #(P B r (z)) Ball of radius r centered at z: B r (z) = { v V d(v, z) r } Note: Λ P (r) = 1 n z P λ P (r, z) = 1 n 2 z P #(P B r (z)).

14 Limiting Curve Histogram Functions Length of a curve l(c) = Local curve distance histogram function C ds < h C (r, z) = l(c B r (z)) l(c) z V = The fraction of the curve contained in the ball of radius r centered at z. Global curve distance histogram function: H C (r) = 1 l(c) h C (r, z(s)) ds. C

15 Convergence Theorem. Let C be a regular plane curve. Then, for both uniformly spaced and randomly chosen sample points P C, the cumulative local and global histograms converge to their continuous counterparts: λ P (r, z) h C (r, z), Λ P (r) H C (r), as the number of sample points goes to infinity.

16 Square Curve Histogram with Bounds

17 Kite and Trapezoid Curve Histograms

18 Histogram Based Shape Recognition 500 sample points Shape (a) (b) (c) (d) (e) (f) (a) triangle (b) square (c) circle (d) 2 3 rectangle (e) 1 3 rectangle (f) star

19 Curve Histogram Conjecture Two sufficiently regular plane curves C and C have identical global distance histogram functions, so H C (r) = H C(r) for all r 0, if and only if they are rigidly equivalent: C C.

20 Proof Strategies Show that any polygon obtained from (densely) discretizing a curve does not lie in the Boutin Kemper exceptional set. Polygons with obtuse angles: taking r small, one can recover (i) the set of angles and (ii) the shortest side length from H C (r). Further increasing r leads to further geometric information about the polygon... Expand H C (r) in a Taylor series at r = 0 and show that the corresponding integral invariants characterize the curve.

21 Taylor Expansions Local distance histogram function: L h C (r, z) = 2r κ2 r 3 + ( 1 40 κκ ss κ2 s κ4 ) r 5 +. Global distance histogram function: H C (r) = 2r L + r3 12L 2 C κ2 ds + r5 40L 2 C ( 3 8 κ4 1 9 κ2 s ) ds +.

22 Space Curves Saddle curve: z(t) = (cos t, sin t, cos 2t), 0 t 2π. Convergence of global curve distance histogram function:

23 Surfaces Local and global surface distance histogram functions: h S (r, z) = area (S B r (z)) area (S) Convergence for sphere:, H S (r) = 1 area (S) S h S (r, z) ds.

24 Area Histograms Rewrite global curve distance histogram function: H C (r) = 1 L h C (r, z(s)) ds = 1 C L 2 χ r (d(z(s), C C z(s )) ds ds { 1, t r, where χ r (t) = 0, t > r, Global curve area histogram function A C (r) = 1 L 3 χ r (area (z(ŝ), C C C z(ŝ ), z(ŝ )) dŝ dŝ dŝ, dŝ equi-affine arc length element L = Discrete cumulative area histogram 1 A P (r) = n(n 1)(n 2) z z z P χ r (area (z, z, z )), Boutin & Kemper: the area histogram uniquely determines generic point sets P R 2 up to equi-affine motion C dŝ

25 Area Histogram for Circle Joint invariant histograms convergence???

26 Triangle Distance Histograms Z = (... z i...) M sample points on a subset M R n (curve, surface, etc.) T i,j,k triangle with vertices z i, z j, z k. Side lengths: σ(t i,j,k ) = ( d(z i, z j ), d(z i, z k ), d(z j, z k ) ) Discrete triangle histogram: Triangle inequality cone S = σ(t ) K K = { (x, y, z) x, y, z 0, x + y z, x + z y, y + z x } R 3.

27 Triangle Histogram Distributions Circle Triangle Square = Madeleine Kotzagiannidis

28 Practical Object Recognition Scale-invariant feature transform (SIFT) (Lowe) Shape contexts (Belongie Malik Puzicha) Integral invariants (Krim, Kogan, Yezzi, Pottman,... ) Shape distributions (Osada Funkhouser Chazelle Dobkin) Surfaces: distances, angles, areas, volumes, etc. Gromov Hausdorff and Gromov-Wasserstein distances (Mémoli) = lower bounds

29 Signature Curves Definition. The signature curve S R 2 of a curve C R 2 is parametrized by the two lowest order differential invariants S = { ( κ, dκ ds ) } R 2 = One can recover the signature curve from the Taylor expansion of the local distance histogram function.

30 Other Signatures Euclidean space curves: C R 3 S = { ( κ, κ s, τ ) } R 3 Euclidean surfaces: S R 3 (generic) S = { ( H, K, H,1, H,2, K,1, K,2 ) } κ curvature, τ torsion R 3 H mean curvature, K Gauss curvature Equi affine surfaces: S R 3 (generic) S = { ( P, P,1, P,2, P,11 ) } R 3 P Pick invariant

31 Equivalence and Signature Curves Theorem. Two regular curves C and C are equivalent: C = g C if and only if their signature curves are identical: S = S = object recognition

32 Symmetry and Signature Theorem. The dimension of the symmetry group G N = { g g N N } of a nonsingular submanifold N M equals the codimension of its signature: dim G N = dim N dim S

33 Discrete Symmetries Definition. The index of a submanifold N equals the number of points in N which map to a generic point of its signature: ι N = min { # Σ 1 {w} } w S = Self intersections Theorem. The cardinality of the symmetry group of a submanifold N equals its index ι N. = Approximate symmetries

34 The Index Σ N S

35 = Steve Haker

36

37

38 Signature Metrics Hausdorff Monge Kantorovich transport Electrostatic repulsion Latent semantic analysis (Shakiban) Histograms (Kemper Boutin) Diffusion metric Gromov Hausdorff

39 Signatures κ s Classical Signature κ s Original curve κ Differential invariant signature

40 Signatures κ s Classical Signature κ s Original curve κ Differential invariant signature

41 Occlusions κ s Classical Signature κ s Original curve κ Differential invariant signature

42 The Baffler Jigsaw Puzzle

43 The Baffler Solved = Dan Hoff

44 Advantages of the Signature Curve Purely local no ambiguities Symmetries and approximate symmetries Extends to surfaces and higher dimensional submanifolds Occlusions and reconstruction Main disadvantage: Noise sensitivity due to dependence on high order derivatives.

45 Noise Reduction Use lower order invariants to construct a signature: joint invariants joint differential invariants integral invariants topological invariants...

46 Joint Invariants A joint invariant is an invariant of the k-fold Cartesian product action of G on M M: I(g z 1,..., g z k ) = I(z 1,..., z k ) A joint differential invariant or semi-differential invariant is an invariant depending on the derivatives at several points z 1,..., z k N on the submanifold: I(g z (n) 1,..., g z (n) k ) = I(z(n) 1,..., z (n) k )

47 Joint Euclidean Invariants Theorem. Every joint Euclidean invariant is a function of the interpoint distances d(z i, z j ) = z i z j z j z i

48 Joint Equi Affine Invariants Theorem. Every planar joint equi affine invariant is a function of the triangular areas [ i j k ] = 1 2 (z i z j ) (z i z k ) z j z i z k

49 Joint Projective Invariants Theorem. Every joint projective invariant is a function of the planar cross-ratios [ z i, z j, z k, z l, z m ] = A B C D A D C B

50 Three point projective joint differential invariant tangent triangle ratio: [ ] [ ] [ ] [ ] [ ] [ ] z 0 z 1 z 0 z 1 z 2 z 2

51 Joint Invariant Signatures If the invariants depend on k points on a p-dimensional submanifold, then you need at least l > k p distinct invariants I 1,..., I l in order to construct a syzygy. Typically, the number of joint invariants is l = k m r = (#points) (dim M) dim G Therefore, a purely joint invariant signature requires at least r k m p + 1 points on our p-dimensional submanifold N M.

52 Joint Euclidean Signature z 0 a z 1 b e c d z 3 f z 2

53 Joint signature map: Σ : C 4 S R 6 a = z 0 z 1 b = z 0 z 2 c = z 0 z 3 d = z 1 z 2 e = z 1 z 3 f = z 2 z 3 = six functions of four variables Syzygies: Φ 1 (a, b, c, d, e, f) = 0 Φ 2 (a, b, c, d, e, f) = 0 Universal Cayley Menger syzygy C R 2 det 2a 2 a 2 + b 2 d 2 a 2 + c 2 e 2 a 2 + b 2 d 2 2b 2 b 2 + c 2 f 2 a 2 + c 2 e 2 b 2 + c 2 f 2 2c 2 = 0

54 Joint Equi Affine Signature Requires 7 triangular areas: [ ], [ ], [ ], [ ], [ ], [ ], [ ] z 1 z 2 z 0 z 5 z 3 z 4

55 Joint Invariant Signatures The joint invariant signature subsumes other signatures, but resides in a higher dimensional space and contains a lot of redundant information. Identification of landmarks can significantly reduce the redundancies (Boutin) It includes the differential invariant signature and semidifferential invariant signatures as its coalescent boundaries. Invariant numerical approximations to differential invariants and semi-differential invariants are constructed (using moving frames) near these coalescent boundaries.

56 Statistical Sampling Idea: Replace high dimensional joint invariant signatures by increasingly dense point clouds obtained by multiply sampling the original submanifold. The equivalence problem requires direct comparison of signature point clouds. Continuous symmetry detection relies on determining the underlying dimension of the signature point clouds. Discrete symmetry detection relies on determining densities of the signature point clouds.