Founda'ons of Large- Scale Mul'media Informa'on Management and Retrieval. Lecture #4 Similarity. Edward Chang

Transcription

1 Founda'ons of Large- Scale Mul'media Informa'on Management and Retrieval Lecture #4 Similarity Edward Y. Chang Edward Chang Foundations of LSMM 1

2 Edward Chang Foundations of LSMM 2

3 Similar? Edward Chang Foundations of LSMM 3

4 Two Key Technical Problems Curse of Dimensionality Modeling Subjec'vity Query/User/App Dependent Edward Chang Foundations of LSMM 4

5 Dimensionality Curse D: Data Dimension When D increases Nearest neighbors are not local All points are equally distanced Edward Chang Foundations of LSMM 5

6 Sparse High- D Space [C. Aggarwal, etc. ICDT 2001] Hyper- cube Range Queries d P [ s] = s d Edward Chang Foundations of LSMM 6

7 Range Coverage à 0% Edward Chang Foundations of LSMM 7

8 Sparse High- D Space Spherical Range Queries Edward Chang Foundations of LSMM 8

9 P[ R sp d ( Q,0.5)] = π d (0.5) d d Γ( + 1) 2 Edward Chang Foundations of LSMM 9

10 No Point in the Nearest Neighborhood Edward Chang Foundations of LSMM 10

11 Dimensionality Curse Edward Chang Foundations of LSMM 11

12 Equidistant Points 4D 512D Edward Chang Foundations of LSMM 12

13 Are We Doomed? How does the curse affect classifica'on? Similar objects tend to cluster together Dimensionality reduc'on Edward Chang Foundations of LSMM 13

14 Summary of Approaches Dynamic Par'al Func'on Restricted Es'mators Specifying the nature of local neighborhood E.g., Manifold learning Adap've Feature Reduc'on PCA, LDA Edward Chang Foundations of LSMM 14

15 Distribu'on of Distances Edward Chang Foundations of LSMM 15

16 Some Solu'ons to High- D Restricted Es'mators Specifying the nature of local neighborhood Manifold learning Adap've Feature Reduc'on PCA, LDA Dynamic Par'al Func'on Edward Chang Foundations of LSMM 16

17 Three Major Paradigms Preserve data descrip'on in a lower dimensional space PCA Maximize discriminability in a lower dimensional space LDA Ac'vate only similar channels DPF Edward Chang Foundations of LSMM 17

18 Minkowski Distance Objects P and Q D = (Σ M (pi - qi) n ) 1/n Similar images are similar in all M features Edward Chang Foundations of LSMM 18

19 1.0E E-02 Frequency 1.0E E E E Feature Distance 1.0E E-02 Frequency 1.0E E E E Edward Chang Feature Distance Foundations of LSMM 19

20 Weighted Minkowski Distance D = (Σ M wi(pi - qi) n ) 1/n Similar images are similar in the same subset of the M features Edward Chang Foundations of LSMM 20

21 Average Distance GIF Feature Number Scale 0 up/down Feature Number Average Distance Average Distance Cropping Rotation Feature Number Feature Number Edward Chang Foundations of LSMM 21 Average Distance

22 Similarity Theories Objects are similar in all respects (Richardson 1928) Objects are similar in some respects (Tversky 1977) Similarity is a process of determining respects, rather than using predefined respects (Goldstone 94) Edward Chang Foundations of LSMM 22

23 DPF Which Place is Similar to Kyoto? Par'al Dynamic Dynamic Par'al Func'on Edward Chang Foundations of LSMM 23

24 Precision/Recall Edward Chang Foundations of LSMM 24

25 Summary of Approaches Dynamic Par'al Func'on Restricted Es'mators Specifying the nature of local neighborhood E.g., Manifold learning Adap've Feature Reduc'on PCA, LDA Edward Chang Foundations of LSMM 25

26 Manifold Learning Algorithms Auto. NN KPCA Principal curves SOM GTM MDS ISOMAP LLE Explicit Manifold No No Yes No Yes No Yes Yes Parametric Yes Yes No No Yes No No No Dissimilarity matrix Local neighborhood No No(?) No No No Yes Yes No(?) No No No(?) No No No Yes Yes Edward Chang Foundations of LSMM 26

27 Geodesic Distance Geodesic: the shortest curve on a manifold that connects two points on the manifold Example: on a sphere, geodesics are great circles Geodesic distance: length of the geodesic A B Figure from mathworld.wolfram.com /GreatCircle.html Edward Chang Foundations of LSMM 27

28 Geodesic Distance Euclidean distance needs not be a good measure between two points on a manifold Length of geodesic is more appropriate Example: Swiss roll Figure from LLE paper Edward Chang Foundations of LSMM 28

29 Isometric Feature Mapping (ISOMAP) Take a distance matrix {g ij } as input Es'mate geodesic distance between any two points by a chain of short paths Formulate this as a graph theory problem Perform classical scaling on the matrix of geodesic distances to obtain final projec'on Edward Chang Foundations of LSMM 29

30 Steps to Es'mate Geodesic Distances 1. Find the neighbors of all data items z i Two possible defini'ons of neighbors Set of items whose distances are less than e The K closest items 2. Construct a weighted undirected graph Vertex i corresponds to z i An edge between the vertex i and j iff z i and z j are neighbors, and its weight is g ij Edward Chang Foundations of LSMM 30

31 Steps to Es'mate Geodesic Distances 3. Find the shortest distance between all pairs of ver'ces in the graph Floyd (O(m 3 )) or Dijkstra (O(m 2 log m+mp)) The shortest distance between ver'ces i and j in the graph is the es'mated geodesic distance between z i and z j Edward Chang Foundations of LSMM 31

32 Ra'onale for the Geodesic Distance Es'ma'on Figures from ISOMAP paper Edward Chang Foundations of LSMM 32

33 A Run of ISOMAP Figure from isomap.stanford.edu/ handfig.html Edward Chang Foundations of LSMM 33

34 A Run of ISOMAP Figures from ISOMAP paper Edward Chang Foundations of LSMM 34

35 Interpola'on on Straight Lines in the Projected Co- ordinates Figures from ISOMAP paper Edward Chang Foundations of LSMM 35

36 Summary of Approaches Dynamic Par'al Func'on Restricted Es'mators Specifying the nature of local neighborhood E.g., Manifold learning Adap've Feature Reduc'on PCA, LDA Edward Chang Foundations of LSMM 36

37 Two Key Technical Problems Curse of Dimensionality Modeling Subjec'vity Query/User/App Dependent Edward Chang Foundations of LSMM 37

38 Distance Func'on? Foundations 38 of LSMM Edward Chang

39 Group by Proximity Foundations 39 of LSMM Edward Chang

40 Group by Proximity x1 x2 x3 x4 x5 x6 x7 x8 x X X x x x x7 1.7 X8 1 Foundations 40 of LSMM Edward Chang

41 Group by Shape Foundations 41 of LSMM Edward Chang

42 Group by Shape x1 x2 x3 x4 x5 x6 x7 x8 x X X x x x x7 1.7 X8 1 Foundations 42 of LSMM Edward Chang

43 Group by Color Foundations 43 of LSMM Edward Chang

44 Group by Color x1 x2 x3 x4 x5 x6 x7 x8 x x x x x x x7 1.7 x8 1 Foundations 44 of LSMM Edward Chang

45 Naïve Alignment Rules Increasing the scores of similar pairs Decreasing the scores of dissimilar pairs S ij > D ij Foundations 45 of LSMM Edward Chang

46 Our Work [ACM KDD 2005, ACM MM 05] kij = β 1 kij if (xi, xj) D kij = β 2 kij + (1 - β 2 ) if (xi, xj) S 0 β 1 β 2 1 Theorem #1 The resul'ng matrix is psd Theorem #2 The resul'ng matrix is beser aligned with the ideal kernel Foundations 46 of LSMM Edward Chang

47 Personaliza'on & Scalability Unsupervised Method Clustering Mul'- version Clustering Ac've Learning Reinforcement Learning Foundations of LSMM 47 Edward Chang

48 Pairs 1,2 3,4 5,6 7,8 Are stable pairs Foundations of LSMM 48 Edward Chang

49 ULP: Unified Learning Paradigm Stable Pairs x1 x2 x3 x4 x5 x6 x7 x8 x X X x x x7 1.7 X8 1 Foundations of LSMM 49 Edward Chang

50 ULP Stable Pairs (green circles) Found via shot- gun clustering Selected Uncertain Pairs (red circles) Iden'fied via the maximum informa'on or fastest convergence rule Propaga'on (green arrow) Foundations of LSMM 50 Edward Chang

51 ULP [EITC 05] D Input: D = L + U K = CalcInitKernel(D) L K M = DoClustering(K) [K,M] [T,Xu] = DoSimilarityReinforce(K,M, M, L) Xu M M =DoActiveLearning(Xu) T K = TransformKernel(K, T) K K=K false IsConverge() true Output: K* Foundations of LSMM 51 Edward Chang

52 Convex Optimization SOCP SDP QCQP LP QP Foundations of LSMM 52 Edward Chang

53 Learning Similarity from Data Please refer to Chap 5 of FLSMIMR Edward Chang Foundations of LSMM 53

54 Summary Curse of Dimension Dynamic Par'al Func'on Manifold learning PCA, LDA Learning Distance Func'on from Data Kernel Alignment Unified Learning Paradigm Edward Chang Foundations of LSMM 54

55 Reading Founda'ons of Large- Scale Mul'media Informa'on Management and Retrieval, E. Y. Chang, Springer, 2011 Chapter #4 Similarity Chapter #5 Learning Distance Func'on Edward Chang Foundations of LSMM 55