Diffusion and random walks on graphs

Transcription

1 Diffusion and random walks on graphs Leonid E. Zhukov School of Data Analysis and Artificial Intelligence Department of Computer Science National Research University Higher School of Economics Structural Analysis and Visualization of Networks Leonid E. Zhukov (HSE) Lecture / 22

2 Module 4 1 Diffusions and random walks on graphs 2 Epidemics on networks 3 Information flow 4 Diffusion of innovations 5 Social influence 6 Trust propagation 7 Segregation model on networks 8 Link prediction 9 Node labeling 10 Time evolution of networks Leonid E. Zhukov (HSE) Lecture / 22

3 Lecture outline 1 Random walks on graph 2 Diffusion on graph Diffusion equation Laplace operator 3 Spectral graph theory Normalized laplacian Leonid E. Zhukov (HSE) Lecture / 22

4 Random walks on graph A random walk on graph on graph G is a sequence of vertices v 0, v 1,...v t.., where each v t+1 is chosen to be a random neighbor of v t, {v t, v t+1 } E(G) and probability of the transition is given by P ij = P(x t+1 = v j x t = v i ), where i P ij = 1, matrix P - row stochastic Leonid E. Zhukov (HSE) Lecture / 22

5 Random walks on graph 2D grid (k=2 regular graph) image from wikipedia.org Leonid E. Zhukov (HSE) Lecture / 22

6 Random walks on graph We will be considering undirected connected unweighted graphs Transition matrix { 1/d(i), if e(i, j), i and j adjacent, P ij = 0, otherwise Using adjacency matrix P ij = A ij d i = D 1 ii A ij, where D ij = d i δ ij Let p i (t) - probability, that a walk is at node i at moment t (probability distribution vector, value per node) Random walk p j (t + 1) = P ij p i (t) = p i (t) A ij d i i i Matrix form p(t + 1) = p(t)p = p(t)(d 1 A) Leonid E. Zhukov (HSE) Lecture / 22

7 Random walks on graph Starting from initial distribution p(0) after t steps p(t) = p(0)p t Random walk on connected non-bipartite graphs converges to limiting distribution lim p(t) = lim t t p(0)pt = π Limiting distribution = stationary distribution lim p(t + 1) = lim p(t)p t t π = πp Left eigenvalue corresponding to λ = 1 λ π = πp Leonid E. Zhukov (HSE) Lecture / 22

8 Random walks on graph Random walk is reversible if π i P ij = π j P ji On undirected graph: and i π i = 1 π i A ij d i = π j A ji d j π i d i = π j d j = const Stationary (stable) distribution π i = d i j d j = d i 2 E Leonid E. Zhukov (HSE) Lecture / 22

9 Random walks on graph Lazy random walk Matrix form p j (t + 1) = 1 2 p j(t) i p i (t) A ij d i p(t + 1) = 1 2 p(t)(i + D 1 A) Converges (always!) to the same stationary distribution (2λ 1) π = π(d 1 A) Leonid E. Zhukov (HSE) Lecture / 22

10 Random walks on graph Theorem Let λ 2 denote second largest eigenvalue of transition matrix P = D 1 A, p(t) probability distribution vector and π stationary distribution. If walk starts from the vertex i, p i (0) = 1, then after t steps for every vertex: d j p j (t) π j λ t 2 d i For P = D 1 A, λ 1 = 1, λ 2 < 1 For P = 1 2 (I + D 1 A), λ 2 = 1 2 (1 + λ 2) Leonid E. Zhukov (HSE) Lecture / 22

11 Physics of Diffusion Let Φ(r, t) -concentration Fik s Law J = C Φ r = C Φ Continuity equation (conserved quantity) Φ t + J = 0 Diffusion equation ( heat equation) Φ(r, t) t = C Φ(r, t) Leonid E. Zhukov (HSE) Lecture / 22

12 Diffusion Laplacian 2D f = 2 f = 2 f x f y 2 Discretized Laplacian in 2D f (x, y) = f (x + h, y) + f (x h, y) + f (x, y + h) + f (x, y h) 4f (x, y) h 2 Leonid E. Zhukov (HSE) Lecture / 22

13 Diffusion on network Some substance that occupy vertices, on each time step diffuses out φ i (t) - quantity per node φ i (t + 1) = φ i (t) + j A ij (φ j (t) φ i (t))cδt dφ i (t) dt = C j A ij (φ j (t) φ i (t)) dφ i dt = C( j A ij φ j j A ij φ i ) = C( j A ij φ j d i φ i ) = C j (A ij δ ij d j )φ j dφ i dt = C j L ij φ j Leonid E. Zhukov (HSE) Lecture / 22

14 Graph Laplacian Graph Laplacian L ij = d j δ ij A ij = D ij A ij, D ij = d j δ ij d(i), if i = j, L ij = 1, if e(i, j) i and j adjacent, 0, otherwise Matrix form L = D A Leonid E. Zhukov (HSE) Lecture / 22

15 Diffusion on Graph Diffusion equation Eigenvector basis φ(t) = k dφ dt + CLφ = 0 Lv k = λv k a k (t)v k, a k (t) = φ(t) T v k ODE Solution ( dak (t) dt k da k (t) dt ) + Cλ k a k (t) v k = 0 + Cλ k a k (t) = 0 a k (t) = a k (0)e Cλ kt φ(t) = k a k (0)v k e Cλ kt Leonid E. Zhukov (HSE) Lecture / 22

16 Laplace matrix L - symmetric positive semidefinite φ T Lφ = L ij φ i φ j = (d i δ ij A ij )φ i φ j = 1 A ij (φ i φ j ) 2 2 ij ij ij Spectral properties Lv i = λv i real non-negative eigenvalues λ i 0 and orthogonal eigenvectors v i smallest eigenvalue always λ 1 = 0 for v 1 = e = [1, 1, 1...1] T Le = (D A)e = 0 Number of zero eigenvalues = number of connected components In connected graph λ algebraic connectivity of a graph (spectral gap), v 2 - Fiedler vector Leonid E. Zhukov (HSE) Lecture / 22

17 Diffusion on Graph Solution φ(t) = k a k (0)v k e Cλ kt all λ i > 0 for i > 1, λ 1 = 0: Normalized solution v 1 = 1 N e Steady state lim t φ(t) = a 1(0)v 1 a 1 (0) = φ(0) T v 1 = 1 φ j (0) N lim t φ(t) = ( 1 N φ j (0))e = const j j Leonid E. Zhukov (HSE) Lecture / 22

18 Diffusion on Graph Leonid E. Zhukov (HSE) Lecture / 22

19 Smoothing operator Smoothing operator (Lφ) i = j (D ij A ij )φ j = j (d i δ ij φ j A ij φ j ) = d i (φ i 1 A ij φ j ) d i j Laplace equation φ = 0, (Lφ) i = 0, solution - harmonic function φ i = 1 A ij φ j d i Regression on graphs j Leonid E. Zhukov (HSE) Lecture / 22

20 Normalized Laplacian Normalized Laplacian L = D 1/2 LD 1/2 1, if i = j, L ij = 1, if e(i, j) i and j adjacent, di d j 0, otherwise Connection to random walks: P = D 1 A = D 1/2 (I L)D 1/2 Similar matrices, share properties of represented linear operators, i.e. eigenvalues: λ max (P) = 1, λ 1 (L) = 0. Leonid E. Zhukov (HSE) Lecture / 22

21 Normalized Laplacian Conductance of a vertex set S φ(s) = cut(s, V \S) min(vol(s), vol(v \S)) where vol(s) = i S k i - sum of all node degrees in the set Cheeger s inequality λ 2 (L)/2 min φ(s) 2λ 2 (L) S Leonid E. Zhukov (HSE) Lecture / 22

22 References Chung, Fan R.K. (1997). Spectral graph theory (2ed.). Providence, RI: American Math. Soc. Daniel A. Spielman. Spectral Graph theory. Combinatorial Scientific Computing. Chapman and Hall/CRC Press Lovasz, L. (1993). Random walks on graphs: a survey. In Combinatorics, Paul Erdos is eighty (pp ). Budapest: Janos Bolyai Math. Soc. Leonid E. Zhukov (HSE) Lecture / 22