Characterising Structural Variations in Graphs

Transcription

1 Characterising Structural Variations in Graphs Edwin Hancock Department of Computer Science University of York Supported by a Royal Society Wolfson Research Merit Award

2 In collaboration with Fan Zhang Tatjiana Aleksic Bin Luo Huaijun Qiu Peng Ren Bai Xiao David Emms Richard Wilson

3 Learning with graph data Problems based on graphs arise in areas such as language processing, proteomics/chemoinformatics, data mining, computer vision and complex systems. Relatively little methodology available, and vectorial methods from statistical machine learning not easily applied since there is no canonical ordering of the nodes in a graph. Can make considerable progress if we develop permutation invariant characterisations of variations in graph structure.

4 Learning with graph data Problems based on graphs arise in areas such as language processing, proteomics/chemoinformatics, data mining, computer vision and complex systems. Relatively little methodology available, and vectorial methods from statistical machine learning not easily applied since there is no canonical ordering of the nodes in a graph. Can make considerable progress if we develop permutation invariant characterisations of variations in graph structure.

5 Learning with graph data Problems based on graphs arise in areas such as language processing, proteomics/chemoinformatics, data mining, computer vision and complex systems. Relatively little methodology available, and vectorial methods from statistical machine learning not easily applied since there is no canonical ordering of the nodes in a graph. Can make considerable progress if we develop permutation invariant characterisations of variations in graph structure.

6 Structural Variations

7 Problem In computer vision graph-structures are used to abstract image structure. However, the algorithms used to segment the image primitives are not reliable. As a result there are both additional and missing nodes (due to segmentation error) and variations in edge-structure. Hence image matching and recognition can not be reduced to a graph isomorphism or even a subgraph isomorphism problem. Instead inexact graph matching methods are needed.

8 Problem In computer vision graph-structures are used to abstract image structure. However, the algorithms used to segment the image primitives are not reliable. As a result there are both additional and missing nodes (due to segmentation error) and variations in edge-structure. Hence image matching and recognition can not be reduced to a graph isomorphism or even a subgraph isomorphism problem. Instead inexact graph matching methods are needed.

9 Problem In computer vision graph-structures are used to abstract image structure. However, the algorithms used to segment the image primitives are not reliable. As a result there are both additional and missing nodes (due to segmentation error) and variations in edge-structure. Hence image matching and recognition can not be reduced to a graph isomorphism or even a subgraph isomorphism problem. Instead inexact graph matching methods are needed.

10 Learning is difficult because Graphs are not vectors: There is no natural ordering of nodes and edges. Correspondences must be used to establish order. Structural variations: Numbers of nodes and edges are not fixed. They can vary due to segmentation error. Not easily summarised: Since they do not reside in a vector space, mean and covariance hard to characterise.

11 Measuring similarity of graphs Early work on graph-matching is vision ( Barrow and Popplestone) introduced association graph and showed how it could be used to locate maximum common subgraph. Work on syntactic and structural pattern recognition in 1980 s unearthed problems with inexact matching (Sanfeliu+Eshera and Fu. Haralick and Shapiro, Wong etc) and extended concept of edit distance from strings to graphs. Recent work has aimed to develop probability distributions for graph matching (Christmas, Kittler and Petrou, Wilson and Hancock, Seratosa and Sanfeliu) and match using advanced optimisation methods(simic, Gold and Rangarjan). Renewed interest in placing classical methods such as edit distance (Bunke) and max-clique (Pelillo) on a more rigorous footing.

12 Learning with graphs Work with (dis) similarities: Can perform pariwise clustering or embed sets of graphs in a vector space using multidimensional scaling on similarities. Non metricity of similarities may pose problems. Embed individual graphs in a low dimensional space: Characterise structural variations in terms of statistical variation in a point-pattern. Learn modes of structural variation: Understand how edge (connectivity) structure varies for graphs belonging to the same class. Requires correspondences of raw structure or alignment of an embedded one. Can also be effected using permutation invariant characteristics (path length, commute-times, cycle frequencies). Construct generative model: Borrow ideas from graphical model to construct model for raw structures or point distribution model to for embedded graphs.

13 Approaches Work with permutation invariant graph characteristics, which can relate to the topological structure. Embed graphs in a vector space and manipulate as point patterns (align, characterise statistically). Establish correspondences and learn variations in edge structure. Determine likelihood that particular nodes and edges co-occur.

14 Methods Graph-spectra lead to straightforward methods for characterising structure and embedding, that do not require correspondences and are permutation invariant. Random walks provide an intuitive way of understanding spectral methods in terms of distributions of path and cycle length. When correspondences are to hand detailed generative models can be constructed using information theory.

15 Our contributions IJCV 2007 (Torsello, Robles-Kelly, Hancock) shape classes from edit distance using pairwise clustering. PAMI 06 and Pattern Recognition 05 (Wilson, Luo and Hancock) graph clustering using spectral features and polynomials. PAMI 07 (Torsello and Hancock) generative model for variations in tree structure using description length. CVIU09 (Xiao, Wilson and Hancock) generative model from heat-kernel embedding of graphs.

16 Delaunay Graph

17 Synthetic Sequence

18 PCA on Adjacency matrices Convert adjacency matrices into long-vectors and perform PCA on sequence. Left column: Eigenspaces Right column: Graph distances in eigenspace Top row: Unweighted graph Middle row: Weighted graph proximity weights Bottom row: Fully connected weighted graph (point proximity matrix)

19 Image Sequence

20 PCA+Eigenvalues

21 ICA+Eigenvalues

22 Shock graphs Type 1 shock (monotonically increasing radius) Type 2 shock (minimum radius) Type 3 shock (constant radius) Type 4 shock (maximum radius)

23 Pairwise clustering Edit distancees Shape comparison Multidimensional scaling on distances

24 Outline Some spectral graph theory and links to random walk. Characterisations based on commute-time, heat-kernel and zeta function. More detailed structure from quantum walks. Conclusions and future.

25 Problem studied How can we find efficient means of characterising graph structure which does not involve exhaustive search? Enumerate properties of graph structure without explicit search, e.g. count cycles, path length frequencies, etc.. Can we analyse the structure of sets of graphs without solving the graph-matching problem? Inexact graph matching is computational bottleneck for most problems involving graphs. Answer: let a random walker do the work.

26 Graph Spectral Methods Use eigenvalues and eigenvectors of adjacency graph (or Laplacian matrix) - Biggs, Cvetokovic, Fan Chung Singular value methods for exact graph-matching and point-set alignment). (Umeyama, Scott and Longuet-Higgins, Shapiro and Brady). Use of eigenvectors for image segmentation (Shi and Malik) and for perceptual grouping (Freeman and Perona, Sarkar and Boyer). Graph-spectral methods for indexing shock-trees (Dickinson and Shakoufandeh)

27 Random walks on graphs Determined by the Laplacian spectrum (and in continuous time case by heatkernel). Can be used to interpret, and analyse, spectral methods since they can be understood intuitively as path-based.

28 Graph spectra and random walks Use spectrum of Laplacian matrix to compute hitting and commute times for random walk on a graph

29 Laplacian Matrix Weighted adjacency matrix Degree matrix Laplacian matrix = otherwise E v u v w u v u W 0 ), ( ), ( ), ( = V v v u W u u D ), ( ), ( W D L =

30 Laplacian spectrum Spectral Decomposition of Laplacian L = ΦΛΦ Element-wise T = λ φ k Λ = diag( λ1,..., λ V ) Φ = φ... φ ) L( u, v) = λ φ ( u) φ ( v) k k k k k φ T k ( 1 V k 0 = λ1 λ2... λ V

31 Properties of the Laplacian Eigenvalues are positive and smallest eigenvalue is zero 0 = λ 1 < λ2 <... < λ V Multiplicity of zero eigenvalue is number connected components of graph. Zero eigenvalue is associated with all-ones vector. Eigenvector associated with the second smallest eigenvector is Fiedler vector.

32 Discrete-time random walk State-vector p t t = Tpt 1 = T p 0 Transition matrix T = D 1 A Steady state p s = Tp s Determined by leading eigenvector of adjacency matrix A, or equivalently the second (Fiedler) eigenvector of the Laplacian.

33 Continuous time random walk

34 Heat Kernels Solution of heat equation and measures information flow across edges of graph with time: T ht L = D W = ΦΛΦ = Lht t Solution found by exponentiating Laplacian eigensystem h t = exp[ λ t] φ φ = Φ exp[ Λt] Φ k k k T k T

35 Heat kernel and random walk State vector of continuous time random walk satisfies the differential equation Solution p t t = Lp t p exp[ Lt] p = = t 0 h t p 0

36 Heat kernel and path lengths In terms of number of paths of length k from node u to node v! ), ( ] exp[ ), ( 1 k t v u P t v u h k k k t = = ), ( v u P k ) ( ) ( ) (1 ), ( v u v u P i i k V i i k φ φ λ =

37 Example. Graph shows spanning tree of heat-kernel. Here weights of graph are elements of heat kernel. As t increases, then spanning tree evolves from a tree rooted near centre of graph to a string (with ligatures). Low t behaviour dominated by Laplacian, high t behaviour dominated by Fiedler-vector.

38 Embedding using commute time Commute time is robust to edge deletion or edge weight errors. Qiu and Hancock, IEEE TPAMI 2007.

39 Hitting time Q(u,v): Expected number of steps of a random walk before node v is visited commencing from node u.

40 Commute time CT(u,v): Expected time taken for a random walk to travel from node u to node v and then return again CT(u,v)=Q(u,v)+Q(v,u)

41 Idea: compute edge attribute that is robust to modifications in edgestructure. Commute time: averages over all paths connecting a pair of nodes. Effects of edge deletion reduced.

42 Illustration Spanning tree vs commute time spanning tree Spanning tree Commte time spanning tree.

43 Green s function Spectral representation V 1 G( u, v) = λi φi ( u) φi ( v) i= 2 Meaning: psuedo inverse of Laplacian

44 Commute Time Commute time Hitting time and the Green s function Commute time and Laplacian eigen-spectrum ), ( ), ( ), ( u u G d vol v v G d vol v Q u u v = ), ( ), ( ), ( ), ( ), ( ), ( ), ( u v G d vol v u G d vol v v G d vol u u G d vol u v Q v Q u v u CT u u v u + = + = ( ) = = 2 2 ) ( ) ( 1 ), ( V i i i i v u vol v u CT φ φ λ

45 Commute Time Embedding Embedding that preserves commute time has co-ordinate matrix (vectors of co-ordinates are columns): Y = vol Λ Φ T

46 Diffusion map Transition probability matrix P = D 1 W Eigensystem Diffusion time D PΨ = ΛΨΨ m 2 2 t ( u, v) = λi [ ϕi ( u) ϕi ( v i= 1 )] 2 Embedding Y = Λ t Ψ T Commute time is sum of diffusion time over all timesteps t.

47 Example embedding

48 Examples

49 Multi-body Motion Analysis Use Costeira and Kanade factorisation method to decompose movement of a set of points over a series of frames into a shape-matrix and a shape-interaction matrix. [X/Y]=M S. Shape interaction matrix is used a adjacency weight matrix and Laplacian computed. W=S. Points are embedded in vector-space using commutetime embedding. Independently moving objects correspond to clusters in embedding space.

50 Multibody motion analysis

51 Videos

52 Videos

53 Videos

54 Videos

55 Videos

56 Motions

57 Embeddings

58 Separation accuracy

59 Invariants from the heat-kernel Extract permutation invariant characteristics from heat-kernel trace. Links with Ihara zeta-function and path-length distribution on graph. Xiao, Wilson and Hancock, Pattern Recognition 2009.

60 Moments of the heat-kernel trace.can we characterise graph by the shape of its heat-kernel trace function?

61 Heat Kernel Trace Tr[ ht ] = exp[ λit] Shape of heat-kernel distinguishes graphs can we characterise its shape using moments Trace i Time (t)->

62 Rosenberg Zeta function Definition of zeta function ς ( s) λ 0 = ( λ ) k k s

63 Heat-kernel moments Mellin transform s 1 s 1 λi = t exp[ λit] dt Γ( s) 0 s Γ( s) = t 0 1 exp[ t] dt Trace and number of connected components Tr[ h t ] = C + λ 0 i exp[ λ t] i C is multiplicity of zero eigenvalue or number of connected components in graph. Zeta function ς ( s) 1 Γ( s) s = λi = 0 λ 0 i t s 1 [ Tr[ h ] C]dt t Zeta-function is related to moments of heatkernel trace.

64 Zeta-function behavior

65 Objects 72 views of each object taken in 5 degree intervals as camera moves in circle around object. Feature points extracted using corner detector. Construct Voronoi tesselation image plane using corner points as seeds. Delaunay graph is region adjacency graph for Voronoi regions.

66 Heat kernel moments (zeta(s), s=1,2,3,4)

67 PCA using zeta(s), s=1,2,3,4)

68 PCA on Laplace spectrum

69

70

71 Ox-Caltech database

72 Line-patterns Use Huet+Hancock representation (TPAMI-99). Extract straight line segments from Canny edgemap. Weight computed using continuity and proximity. Captures arrangement Gestalts.

73

74 Zeta function derivative Zeta function in terms of natural exponential ς ( s) = Derivative λ 0 ς '( s) = ( λ ) s Derivative at origin ς '(0) = ln λk = k λ 0 k λ 0 k k = λ 0 k exp[ s ln λ ] ln λ exp[ s ln λ ] k ln λ 0 k 1 λ k k k

75 Meaning Number of spanning trees in graph du u V ' τ ( G) = exp[ ζ ( o)] d u V u

76 COIL

77 Ox-Cal

78 Eigenvalue polynomials (COIL) Determinant Trace

79 Eigenvalue Polynomials (Ox- Cal)

80 Spectral polynomials (COIL)

81 Spectral Polynomials (Ox-Cal)

82 COIL: node and edge frequency

83 Ox-Cal: node+edge frequency

84 Performance Rand index=correct/(correct+wrong). Zeta func Sym. Poly. (evals) Sym. Poly. (matrix) Laplacian spectrum 0,

85 Deeper probes of structure Ihara zeta function

86 Zeta functions Used in number theory to characterise distribution of prime numbers. Can be extended to graphs by replacing notion of prime number with that of a prime cycle.

87 Characterising graphs Topological: e.g. average degree, degree distribution, edge-density, diameter, cycle frequencies etc. Spectral: use eigenvalues of adjacency matrix or Laplacian. Algebraic: co-efficients of characteristic polynomial.

88 Ihara Zeta function Determined by distribution of prime cycles. Transform graph to oriented line graph (OLG) with edges as nodes and edges indicating incidence at a common vertex. Zeta function is reciprocal of characteristic polynomial for OLG adjacency matrix. Coefficients of polynomial determined by eigenvalues of OLG adjacency matrix. Coefficients linked to topological quantities such as cycle frequencies, number of spanning trees.

89 Oriented Line Graph

90 Ihara Zeta Function Ihara Zeta Function for a graph G(V,E) Defined over prime cycles of graph Rational expression in terms of characteristic polynomial of oriented line-graph A is adjacency matrix of line digraph Q =D-I (degree matrix minus identity

91 Characteristic Polynomials from IZF Perron-Frobenius operator is the adjacency matrix T H of the oriented line graph Determinant Expression of IZF Each coefficient,i.e. Ihara coefficient, can be derived from the elementary symmetric polynomials of the eigenvalue set Pattern Vector in terms of

92 Analysis of determinant From matrix logs ζ ( s) = 1 det[ I Ts] = exp[ Tr[ T k s ] k > 1 k k ] Tr[T^k] is symmetric polynomial of 1 eigenvalues of T Tr[ T ] = λ λn Tr[ T... Tr[ T 2 N 2 ] = λ + λ λ +... λ 1 ] = λ λ... λ N N

93 Distribution of prime cycles Frequency distribution for cycles of length l s d ds lnζ ( s) = l N l s l Cycle frequencies l 1 d N l = lnζ ( s) l s= 0 = ( l 1)! ds Tr[ T l ]

94 Experiments: Edge-weighted Graphs Feature Distance & Edit Distance Three Classes of Randomly Generated Graphs

95 Experiments: Hypergraphs

96 Complexity

97 Non-Neuman Entropy Given in terms of normalised Laplacian eigenvalues H λk ln λk = k Quadratic approximation H = λ (1 λ ) Variance of elements of weighted adjacency matrix. k k k

98 Birkhoff polytopes Representation of permutation group using stochastic basis matrices. Doubly stochastic matrix can be represented as a convex combination of permutation matrices Complexity K = p P k C = k k p k n! ln k p k

99 Shannon-Jensen Divergence In terms of Shannon entropy J = H ( ) k Gk k H ( G k ) Related to edit distance on trees (Torsello+Hancock). Can lead to family of kernels

100 Behaviour

101

102

103

104

105

106

107 Quantum Walks Have both discrete time (DQW) and continuous time (CQW) variants. Use qubits to represent state of walk. State-vector is composed of complex numbers rather than an probabilities. Governed by unitary matrices rather than stochastic matrices. Admit interference of different feasible walks. Reversible, non ergodic, no limiting distribution.

108 Literature Fahri and Gutmann: CQW can penetrate family of trees in polynomial time, whereas classical walk can not. Childs: CQW hits exponentially faster than classical walk. Kempe: exploits polynomial hitting time to solve 2-SAT problem and suggest solution to routing problem. Shenvi, Kempe and Whaley: search of unordered database.

109 Quantum walks.have richer structure due to interference and complex representation. What can a quantum walker discover about a graph that a classical walker can not?

110 New possibilities Interference in quantum walks used to develop alternatives to classical random walks algorithm.

111 Consequences of interference Quantum walks hit exponentially faster than classical ones. Local symmetry reflected in cancellation of complex amplitude. Can be exploited to lift co-spectrality of strongly regular graphs. Quantum version of commute time embedding can be used to detect symmetries.

112 Strongly Regular Graphs Two SRGs with parameters (16,6,2,2) A B

113 Is the sp + (U 3 ) able to distinguish all There is no method proven to be able to decide whether two SRGs are isomorphic in polynomial time. There are large families of strongly regular graphs that we can test the method on SRGs? MDS embeddings of the SRGs with parameters (25,12,5,6)-red, (26,10,3,4)-blue, (29,14,6,7)-black, (40,12,2,4)-green using the adjacency spectrum (top) and the spectrum of S+(U^3) (bottom).

114 Testing The algorithm takes time O( E ^3). We have tested it for all pairs of graphs from the table opposite and found that it successfully determines whether the graphs are isomorphic in all cases.

115 Continuous Time Quantum Walk Evolution governed by Schrodinger s equation d dt ψ t >= il ψ t > Solution ψ t >= exp[ ilt] ψ 0 >

116 Walks compared Classical walk Quantum walk

117 Quantum hitting time Probability that walk is at node v at time t t P( X = v) = < v exp[ ilt] u > u 2 t T P( X = v) = < v Φ exp[ iλt] Φ u > u 2 Hazard function R(t) satisfies Hitting time is expectation of time weighted by hazard function: d dt t R( t) = P( X = v)[1 R( t)] 0 Q( u, v) = R( t) tdt u

118 Average commute vs path length Quantum walk reduces effect of path length. Quantum Classical

119 Example Embedding

120 Wheel with spokes

121 Symmetries

122 Symmetries of 3x3 Grid

123 8x8 Grid

124 Conclusions Interference of quantum walks can be exploited in algorithm design. Leads to a method for resolving cospectrality of graphs and trees Can also be used to design algorithm for inexact graph matching. Quantum commute time differs from its classical counterpart, and leads to embeddings that can be used to characterise symmetries in graphs..

125 Conclusions Random walks provide powerful framework for pathbased or flow-based analysis in vision and pattern recognition. Convenient since they rely on the Laplacian eigensystem, and although eigendecompisition is time consuming implementation is easy. Allow both low level and high level problems to be addressed. Quantum walks offer fascinating alternative which allows for interference of alternative walks.