A Random Walk Through Spectral Graph Theory letting a random walker do the hard work of analysing graphs

Transcription

1 A Random Walk Through Spectral Graph Theory letting a random walker do the hard work of analysing graphs Edwin Hancock Department of Computer Science University of York Supported by a Royal Society Wolfson Research Merit Award

2 In collaboration with Fan Zhang Huaijun Qiu Hongfang Wang Bai Xiao David Emms Richard Wilson

3 Pattern analysis and learning with graphs is difficult Graphs are not vectors: There is no natural ordering of nodes and edges. Correspondences must be used to establish order. Structural variations: Numbers of nodes and edges are not fixed. They can vary due to segmentation error. Not easily summarised: Since they do not reside in a vector space, mean and covariance hard to characterise.

4 Structural Variations

5 Problem studied How can find efficient means of characterising graph structure which does not involve exhaustive search? Enumerate properties of graph structure without explicit search, e.g. count cycles, path length frequencies, etc.. Can we analyse the structure of sets of graphs without solving the graph-matching problem? Inexact graph matching is computational bottleneck for most problems involving graphs. Answer: let a random walker do the work.

6 Graph Spectral Methods Use eigenvalues and eigenvectors of adjacency graph (or Laplacian matrix) - Biggs, Cvetokovic, Fan Chung Singular value methods for exact graph-matching and point-set alignment). (Umeyama, Scott and Longuet-Higgins, Shapiro and Brady). Use of eigenvectors for image segmentation (Shi and Malik) and for perceptual grouping (Freeman and Perona, Sarkar and Boyer). Graph-spectral methods for indexing shock-trees (Dickinson and Shakoufandeh)

7 Random walks on graphs Determined by the Laplacian spectrum (and in continuous time case by heatkernel). Can be used to interpret, and analyse, spectral methods since they can be understood intuitively as path-based.

8 Outline Some spectral graph theory and links to random walk. Characterisations based on commute-time, heat-kernel and zeta function. Quantum walks, and how interference leads to some powerful alternative algorithms. Conclusions and future.

9 Graph spectra and random walks Use spectrum of Laplacian matrix to compute hitting and commute times for random walk on a graph

10 Laplacian Matrix Weighted adjacency matrix Degree matrix Laplacian matrix = otherwise E v u v w u v u W 0 ), ( ), ( ), ( = V v v u W u u D ), ( ), ( W D L =

11 Laplacian spectrum Spectral Decomposition of Laplacian L = ΦΛΦ Element-wise T = λ φ k Λ = diag( λ1,..., λ V ) Φ = φ... φ ) L( u, v) = λ φ ( u) φ ( v) k k k k k φ T k ( 1 V k 0 = λ1 λ2... λ V

12 Properties of the Laplacian Eigenvalues are positive and smallest eigenvalue is zero 0 = λ 1 < λ2 <... < λ V Multiplicity of zero eigenvalue is number connected components of graph. Zero eigenvalue is associated with all-ones vector. Eigenvector associated with the second smallest eigenvector is Fiedler vector.

13 Discrete-time random walk State-vector p t t = Tpt 1 = T p 0 Transition matrix T = D 1 A Steady state p s = Tp s Determined by leading eigenvector of adjacency matrix A, or equivalently the second (Fiedler) eigenvector of the Laplacian.

14 Continuous time random walk

15 Heat Kernels Solution of heat equation and measures information flow across edges of graph with time: T ht L = D W = ΦΛΦ = Lht t Solution found by exponentiating Laplacian eigensystem h t = exp[ λ t] φ φ = Φ exp[ Λt] Φ k k k T k T

16 Heat kernel and random walk State vector of continuous time random walk satisfies the differential equation Solution p t t = Lp t p exp[ Lt] p = = t 0 h t p 0

17 Heat kernel and path lengths In terms of number of paths of length k from node u to node v! ), ( ] exp[ ), ( 1 k t v u P t v u h k k k t = = ), ( v u P k ) ( ) ( ) (1 ), ( v u v u P i i k V i i k φ φ λ =

18 Example. Graph shows spanning tree of heat-kernel. Here weights of graph are elements of heat kernel. As t increases, then spanning tree evolves from a tree rooted near centre of graph to a string (with ligatures). Low t behaviour dominated by Laplacian, high t behaviour dominated by Fiedler-vector.

19 Diffusion smoothing Use heat-kernel to smooth noisy data. Zhang and Hancock, Pattern Recognition 2008 and 2010 (in press)

20 Literature General diffusion-based partial differential equation (PDE): u = div( D u) t If D=1, isotropic linear diffusion (Gaussian filter). If D is a scalar function, inhomogeneous diffusion, e.g. Perona- Malik diffusion: D= g( u ) If D is a 2 2diffusion tensor, anisotropic diffusion, e.g. g( u ) 0 D = 0 1 (Weickert, etc).

21 Motivation: Why Graph Diffusion? Assumption for continuous PDEs An image is a continuous function on R 2 Consider discretisation for numerical implementation. Weakness: Noisy images may not be sufficiently smooth to give reliable derivatives. Fast, accurate, and stable implementation is difficult to achieve. Solution: diffusion on graphs An image is a smooth function on a graph. Purely combinatorial operators that require no discretisation are used.

22 Steps Set-up diffusion process as problem involving weighted graph, where anisotropic smoothing is modelled by an edge-weight matrix. Diffusion is heat-flow on the associated graph-structure. Nodes are pixels, and diffusion of grey-scale information is along edges with time. Solution is given by exponentiating the spectrum of the associated Laplacian matrix.

23 Graph Representation of Images I An image is represented using a weighted graph Γ= ( V, E). The nodes V are the pixels. An edge eij E is formed between two nodes v i and v j. The weight of an edge, e, is denoted by wi (, j). ij

24 Graph Edge Weight Characterise each pixel i by a n n window Ni of neighbors instead of using a single pixel alone. The similarity between nodes v i and v j is measured by the Gaussian weighted Euclidean distance between windows, i.e. d (, i j) = N N = G * N N σ i j 2, σ σ i j 2 Thus, edge weight is computed by wi (, j) exp( = 2 dσ (, i j) ) if X(i)-X(j) κ 0 otherwise 2 2 r

25 Anisotropic diffusion as heat Vertices: pixel values flow on a graph Edge weight: c.f. thermal conductivity Diffusion-equation for pixel values: I Pixel values stored as vector of t = LIt stacked image columns. t Connectivity structure encoded by Laplacian. I t = exp[ Lt] I0 = h t I 0 Heat kernel is smoothing filter.

26 Results It takes 3~6 seconds to process a 256 square image.

27 Root-Mean-Square Error comparison

28 Root-Mean-Square Error comparison

29

30 Smoothing in Diffusion Tensor MRI Tensor characterisation of water diffusion direction at each voxel location, Assign tensors to prolate or oblate classes using eigenvalues of tensors (FA). For pairs of prolate tensors weight is determined by angle between principle eigenvectors. Otherwise weight is determined geodesic distance between tensors on half-cone.

31 Raw image

32 Add noise

33 Graph-based diffusion smoothing

34 Pennec Riemannian smoothing of tensors

35 Root mean square error

36 Root mean square error

37 Effect of smoothing

38 Details

39 Details

40

41 Raw and smoothed

42 Tracked fibers (raw and smoothed)

43 Details

44 Principal direction field FA map Smoothed FA Principal direction field Smoothed principal direction field.

45 Consistent labelling Run probability diffusion backwards using Fokker-Planck equation to restore label consistency

46 Relaxation Labelling It aims at assigning consistent and unambiguous labels to a given set of objects. Relies on contextual information provided by topology of object arrangement and sets of compatibility relations on object-configutations. Involves evidence combination to update label probabilities and propagate label consistency constraints. Requires initial label probability assignment.

47 Relaxation Labelling Support functions for evidence combination: Hummer & Zucker: Kittler & Hancock: Probability update formula: Ω = N j i i i k j i ij j k i P R j S ω ω ω ω ω ) ( ) ( ) ( ) (, ) ( Ω + = i i k i k j k j k j k j S j P j S j P j P ω ω ω ω ω ω ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( 1) ( Ω = N j i i i k j i ij j k i P R j S ω ω ω ω ω ) ( ) ( ) ( ) (, ) (

48 Graph spectral relaxation labeling Set up a support graph where nodes represent possible object-label assignments. Edges represent label compatibility. Set up a random walk on the graph. Probability of visiting node is probability of object-label assignment (state-vector). Evolve state-vector of random with time to update probabilities and combine evidence.

49 Method Exploit the similarities between diffusion processes and relaxation labelling to develop a iterative process for consistent labelling. A compositional graph structure is used to represent the labelling. Each node on the graph is an object-label assignment. Edge weights encode label compatibility information. Run diffusion process to update label probabilities.

50 Label-probability diffusion process Given label-set M, object-set O nodes of graph are Cartesian pairs V=OxM. Edge weights are label compatibilities, weighted Laplacian L=D-W. Probability update formula: run diffusion process backwards in time (Fokker-Planck equation): p t = exp[ t( D W )] p = Φ Λ 0 exp[ t ] Φ T p 0

51 Experiments Five synthetic data-sets: I. Two rings (R2) II. Ring-Gaussian (RG) III. Three Gaussian (G3) V. Four Gaussian G4_2) IV. Four Gaussian (G4_1)

52 Experimental Results Resutls of synthetic data-sets: II. Results of Ring-Gaussian data (RG) I. Results of Three Gaussian data (G3) III. Results of Four Gaussian data (G4-1)

53 Experimental Results Real world data (from UCI machine learning data repository) III. Results of Iris data-set IV. Results of Wine data

54 Embedding using commute time Commute time is robust to edge deletion or edge weight errors. Qiu and Hancock, IEEE TPAMI 2007.

55 Hitting time Q(u,v): Expected number of steps of a random walk before node v is visited commencing from node u.

56 Commute time CT(u,v): Expected time taken for a random walk to travel from node u to node v and then return again CT(u,v)=Q(u,v)+Q(v,u)

57 Idea: compute edge attribute that is robust to modifications in edgestructure. Commute time: averages over all paths connecting a pair of nodes. Effects of edge deletion reduced.

58 Illustration Spanning tree vs commute time spanning tree Spanning tree Commte time spanning tree.

59 Green s function Spectral representation V 1 G( u, v) = λi φi ( u) φi ( v) i= 2 Meaning: psuedo inverse of Laplacian

60 Commute Time Commute time Hitting time and the Green s function Commute time and Laplacian eigen-spectrum ), ( ), ( ), ( u u G d vol v v G d vol v Q u u v = ), ( ), ( ), ( ), ( ), ( ), ( ), ( u v G d vol v u G d vol v v G d vol u u G d vol u v Q v Q u v u CT u u v u + = + = ( ) = = 2 2 ) ( ) ( 1 ), ( V i i i i v u vol v u CT φ φ λ

61 Commute Time Embedding Embedding that preserves commute time has co-ordinate matrix (vectors of co-ordinates are columns): Y = vol Λ Φ T

62 Diffusion map Transition probability matrix P = D 1 W Eigensystem Diffusion time D PΨ = ΛΨΨ m 2 2 t ( u, v) = λi [ ϕi ( u) ϕi ( v i= 1 )] 2 Embedding Y = Λ t Ψ T Commute time is sum of diffusion time over all timesteps t.

63 Example embedding

64 Examples

65 Multi-body Motion Analysis Use Costeira and Kanade factorisation method to decompose movement of a set of points over a series of frames into a shape-matrix and a shape-interaction matrix. [X/Y]=M S. Shape interaction matrix is used a adjacency weight matrix and Laplacian computed. W=S. Points are embedded in vector-space using commutetime embedding. Independently moving objects correspond to clusters in embedding space.

66 Multibody motion analysis

67 Videos

68 Videos

69 Videos

70 Videos

71 Videos

72 Motions

73 Embeddings

74 Separation accuracy

75 Invariants from the heat-kernel Extract permutation invariant characteristics from heat-kernel trace. Links with Ihara zeta-function and path-length distribution on graph. Xiao, Wilson and Hancock, Pattern Recognition 2009.

76 Setting the scene.probe structure of graph without explicit search.

77 Moments of the heat-kernel trace.can we characterise graph by the shape of its heat-kernel trace function?

78 Heat Kernel Trace Tr[ ht ] = exp[ λit] Shape of heat-kernel distinguishes graphs can we characterise its shape using moments Trace i Time (t)->

79 Rosenberg Zeta function Definition of zeta function ς ( s) λ 0 = ( λ ) k k s

80 Heat-kernel moments Mellin transform s 1 s 1 λi = t exp[ λit] dt Γ( s) 0 s Γ( s) = t 0 1 exp[ t] dt Trace and number of connected components Tr[ h t ] = C + λ 0 i exp[ λ t] i C is multiplicity of zero eigenvalue or number of connected components in graph. Zeta function ς ( s) 1 Γ( s) s = λi = 0 λ 0 i t s 1 [ Tr[ h ] C]dt t Zeta-function is related to moments of heatkernel trace.

81 Zeta-function behavior

82 Objects 72 views of each object taken in 5 degree intervals as camera moves in circle around object. Feature points extracted using corner detector. Construct Voronoi tesselation image plane using corner points as seeds. Delaunay graph is region adjacency graph for Voronoi regions.

83 Heat kernel moments (zeta(s), s=1,2,3,4)

84 PCA using zeta(s), s=1,2,3,4)

85 PCA on Laplace spectrum

86

87

88 Ox-Caltech database

89 Line-patterns Use Huet+Hancock representation (TPAMI-99). Extract straight line segments from Canny edgemap. Weight computed using continuity and proximity. Captures arrangement Gestalts.

90

91 Zeta function derivative Zeta function in terms of natural exponential ς ( s) = Derivative λ 0 ς '( s) = ( λ ) s Derivative at origin ς '(0) = ln λk = k λ 0 k λ 0 k k = λ 0 k exp[ s ln λ ] ln λ exp[ s ln λ ] k ln λ 0 k 1 λ k k k

92 Meaning Number of spanning trees in graph du u V ' τ ( G) = exp[ ζ ( o)] d u V u

93 COIL

94 Ox-Cal

95 Eigenvalue polynomials (COIL) Determinant Trace

96 Eigenvalue Polynomials (Ox- Cal)

97 Spectral polynomials (COIL)

98 Spectral Polynomials (Ox-Cal)

99 COIL: node and edge frequency

100 Ox-Cal: node+edge frequency

101 Performance Rand index=correct/(correct+wrong). Zeta func Sym. Poly. (evals) Sym. Poly. (matrix) Laplacian spectrum 0,

102 Quantum Walks Have both discrete time (DQW) and continuous time (CQW) variants. Use qubits to represent state of walk. State-vector is composed of complex numbers rather than an probabilities. Governed by unitary matrices rather than stochastic matrices. Admit interference of different feasible walks. Reversible, non ergodic, no limiting distribution.

103 Literature Fahri and Gutmann: CQW can penetrate family of trees in polynomial time, whereas classical walk can not. Childs: CQW hits exponentially faster than classical walk. Kempe: exploits polynomial hitting time to solve 2-SAT problem and suggest solution to routing problem. Shenvi, Kempe and Whaley: search of unordered database.

104 Quantum walks.have richer structure due to interference and complex representation. What can a quantum walker discover about a graph that a classical walker can not?

105 New possibilities Interference in quantum walks used to develop alternatives to classical random walks algorithm.

106 Consequences of interference Quantum walks hit exponentially faster than classical ones. Local symmetry reflected in cancellation of complex amplitude. Can be exploited to lift co-spectrality of strongly regular graphs. Quantum version of commute time embedding can be used to detect symmetries.

107 Strongly Regular Graphs Two SRGs with parameters (16,6,2,2) A B

108 Is the sp + (U 3 ) able to distinguish all There is no method proven to be able to decide whether two SRGs are isomorphic in polynomial time. There are large families of strongly regular graphs that we can test the method on SRGs? MDS embeddings of the SRGs with parameters (25,12,5,6)-red, (26,10,3,4)-blue, (29,14,6,7)-black, (40,12,2,4)-green using the adjacency spectrum (top) and the spectrum of S+(U^3) (bottom).

109 Testing The algorithm takes time O( E ^3). We have tested it for all pairs of graphs from the table opposite and found that it successfully determines whether the graphs are isomorphic in all cases.

110 Continuous Time Quantum Walk Evolution governed by Schrodinger s equation d dt ψ t >= il ψ t > Solution ψ t >= exp[ ilt] ψ 0 >

111 Walks compared Classical walk Quantum walk

112 Quantum hitting time Probability that walk is at node v at time t t P( X = v) = < v exp[ ilt] u > u 2 t T P( X = v) = < v Φ exp[ iλt] Φ u > u 2 Hazard function R(t) satisfies Hitting time is expectation of time weighted by hazard function: d dt t R( t) = P( X = v)[1 R( t)] 0 Q( u, v) = R( t) tdt u

113 Average commute vs path length Quantum walk reduces effect of path length. Quantum Classical

114 Example Embedding

115 Wheel with spokes

116 Symmetries

117 Symmetries of 3x3 Grid

118 8x8 Grid

119 Conclusions Interference of quantum walks can be exploited in algorithm design. Leads to a method for resolving cospectrality of graphs and trees Can also be used to design algorithm for inexact graph matching. Quantum commute time differs from its classical counterpart, and leads to embeddings that can be used to characterise symmetries in graphs..

120 Conclusions Random walks provide powerful framework for pathbased or flow-based analysis in vision and pattern recognition. Convenient since they rely on the Laplacian eigensystem, and although eigendecompisition is time consuming implementation is easy. Allow both low level and high level problems to be addressed. Quantum walks offer fascinating alternative which allows for interference of alternative walks.