Portland State Univ., Feb Diffusion and Consensus on Digraphs

Size: px

Start display at page:

Download "Portland State Univ., Feb Diffusion and Consensus on Digraphs"

Aron Nichols
5 years ago
Views:

1 Portland State Univ., Feb 209 Diffusion and Consensus on Digraphs Based on: []: J. S. Caughman, J. J. P. Veerman, Kernels of Directed Graph Laplacians, Electronic Journal of Combinatorics,, No, [2]: J. J. P. Veerman, E. Kummel, Diffusion and Consensus on Weakly Connected Directed Graphs, submitted, 208. Math/Stat, Portland State Univ., Portland, OR 9720, USA.

2 SUMMARY: * This is a review of two basic dynamical processes on a weakly connected, directed graph G: consensus and diffusion, as well their discrete time analogues. * We consider them as dual processes defined on G by: ẋ = Lx for consensus and ṗ = pl for diffusion. * We give a complete characterization of the asymptotic behavior of both diffusion and consensus discrete and continuous in terms of the null space of the Laplacian (defined below). * As an application of these ideas, we present a treatment of the pagerank algorithm that is dual to the usual one, namely cast in terms of consensus (and not random walk). * Many of the ideas presented here can be found scattered in the literature, though mostly outside mainstream mathematics and not always with complete proofs. 2

3 OUTLINE: The headings of this talk are color-coded as follows: Definitions Peculiarities of Directed Graphs Consensus and Diffusion Left and Right Kernels of L Asymptotics The Pagerank Algorithm *More Remarks

4 . D E F I N I T I O N S 4

5 Definitions: Digraphs Definition: A directed graph (or digraph) is a set V = {, n} of vertices together with set of ordered pairs E V V (the edges) A directed edge j i, also written as ji. A directed path from j to i is written as j i. Digraphs are everywhere: models of the internet [5], social networks [6], food webs [9], epidemics [8], chemical reaction networks [2], databases [4], communication networks [], and networks of autonomous agents in control theory [7], to name but a few. A BIG topic: Much of mathematics can be translated into graph theory (discretization, triangulation, etc). In addition, many topics in graph theory that do not translate back to continuous mathematics. 5

6 Definitions: Connectedness of digraphs Undirected graphs are connected or not. But Definition: * A digraph G is strongly connected if for every ordered pair of vertices (i, j), there is a path i j. * A digraph G is unilaterally connected if for every ordered pair of vertices (i, j), there is a path i j or a path j i. * A digraph G is weakly connected if the underlying UNdirected graph is connected. * A digraph G is not connected: if it is not weakly connected. 6

7 Definitions: Graph Structure Definition: Definitions underlined are used downstream. * Reachable Set R(i) V : j R(i) if i j. * Reach R V : A maximal reachable set. Or: a maximal unilaterally connected set. * Exclusive part H R: vertices in R that do not see vertices from other reaches. If not in cabal, called minions. * Common part C R: vertices in R that also see vertices from other reaches. * Cabal B H: set of vertices from which the entire reach R is reachable. If single, called leader. * Gaggle Z R: an SCC with no outgoing edges. If single, called goose. So gaggles and cabals are SCC s. If we reverse edge orientation, then gaggles turn into cabals, and so on. SCC s remain SCC s. Reaches are not preserved. 7

8 Definitions: Reaches common part = common part 2 = {6,7} exclusive part 2 cabal 2 4 exclusive part 2 cabal 6 reach 7 reach 2 5 cabal = scc w. no incoming edges gaggle = scc w. no outgoing edges {2} = goose = minion {} = leader {2} and {6,7} Fun exercise: Invert orientation and do the taxonomy again. Surprising exercise: The number of reaches may change if orientation is reversed! (Therefore the spectrum is not invariant.) Example: o o o 8

9 Definitions: Laplacian Definition: The combinatorial adjacency matrix Q of the graph G is the matrix whose entry Q ij = if there is an edge ji and equals 0 otherwise. If vertex i has no incoming edges, set Q ii = (create a loop). Remark: Instead of creating a loop, sometimes all elements of the ith row are given the value /n. This is called Teleporting! The matrix is denoted by Q t. Definition: The in-degree matrix D is a diagonal matrix whose i diagonal entry equals the number of (directed) edges xi, x V. Definition: The matrices S D Q and S t D Q t are called the normalized adjacency matrices. By construction, they are row-stochastic (non-negative, every row adds to ). Definition: Laplacians describe decentralized or relative observation. Common cases: The combinatorial Laplacian: L D Q. The random walk (rw) Laplacian: L I D Q. The rw Laplacian with teleporting: L I D Q t. 9

10 Definitions: Laplacian So Q = L I D Q = D = diag / /2 0 0 /2 0 0 /2 0

11 . P E C U L I A R I T I E S O F D I R E C T E D G R A P H S

12 Directed and Undirected In the math community, directed graphs are still much less studied than undirected graphs (especially true for the algebraic aspects). As a consequence, very few good text books. What are the reasons for this? Directed graphs are a lot messier than undirected graphs: - Combinatorial Laplacians of undirected graphs are symmetric. So: real eigenvalues, orthogonal basis of eigenvectors, no non-trivial Jordan blocks, etc. - Connectedness of undirected graphs is much simpler. - No standard convention on how to orient a digraph. rw Laplacians of undirected graphs are almost symmetric, because they are conjugate to symmetric matrices:. D Q = D 2 D 2QD 2 D 2. Proposition: G undirected. Then the eigenvectors of the rw Laplacian form a complete basis, and the eigenvalues are real. (Well-known result: mathematicians like clean, not messy.) 2

13 Digraphs are Messy Two strongly connected digraphs. The first has rw Laplacian 0 0 L = /2 /2 0 /2 0 /2 0 /2 /2 with spectrum {0,.62 ± 0.40i, 0.76} (approximately). The second has rw Laplacian 0 0 L = /2 0 /2 /2 0 /2 0 0 with spectrum {0, (2), 2}. The eigenvalue has an associated 2-dimensional Jordan block.

14 Which Direction?? In this review, we are interested in information flow, as opposed to a physical flow (oil, traffic, for example). We propose a new convention: The direction of the edges should be the same as the direction of the flow of the information. In many cases, this makes sense. In a food web, the predator needs to locate the prey. Thus arrows go from prey to predator. See this food web. Taken from the US Geological Survey []. 4

15 . D U A L P R O C E S S E S : C O N S E N S U S A N D D I F F U S I O N 5

16 Consensus and Diffusion The Laplacian L has the form I S or I S t where S and S t are row-stochastic. From now on x is a column vector and p is a row vector. Consensus: ẋ = Lx. (Usual matrix multiplication.) Properties: The all ones vector is a solution. Given an edge ki, this edge will give a contribution to ẋ i proportional to x k x i. Diffusion: ṗ = pl. (Usual matrix multiplication.) Properties: i ṗi = 0 (row-sum L is zero). Given an edge ki, then this edge will give a contribution to ṗ k proportional to p k p i. Remark: The physicist s definition of L would be the negative of the one we use here. Graph theorists like eigenvalues of symmetric Laplacians to be non-negative. Theorem : The eigenvalues of S lie within the closed unit disk (Gersgorin). So the non-zero eigenvalues of L = I S have positive real part. 6

17 . L E F T A N D R I G H T K E R N E L S O F L 7

18 First: Eigenvalue Zero SCC: i j if i and j are in same SCC. This is an equivalence. Partial order on SCC s: S < S 2 if S S 2. Topol. sorting: extend partial order to total order. Theorem 2: S and L are block triangular with SCC s as blocks L = / /2 0 0 /2 0 0 /2 st and rd block both give a zero eigenvalue. To understand how SCC s are connected, we will look at their eigenvectors, i.e.: the kernel of L. 8

19 The Right Kernel of L Recall for a digraph G: reach R i, exclusive part H i, cabal B i, and common part C i. FROM NOW ON assume there are exactly k reaches {R i } k i=. Theorem []: The algebraic and geometric multiplicity of the eigenvalue 0 of L = I S equals k. Thus: no non-trivial Jordan blocks in kernel! Theorem 4 []: The right kernel of L consists of the column vectors {γ,, γ k }, where: γ m (j) = if j H m (excl.) γ m (j) (0, ) if j C m (common) γ m (j) = 0 if j R m (reach) k m= γ m(j) = γ T = ( ) and γ T 2 = ( ) 9

20 The Left Kernel of L Theorem 5 [2]: The left kernel of L consists of the row vectors { γ,, γ k }, where: γ m (j) > 0 if j B m (cabal) γ m (j) = 0 if j B m k j= γ m(j) = { γ m } k m= are orthogonal Mnemonic: the horizontal bar on γ indicates a (horizontal) row vector. Thus in this case the row vectors { γ,, γ k } are a set of orthogonal invariant probability measures γ = ( ) and γ 2 = ( ) 20

21 Observations about the Kernels Theorem 6 (folklore, [2]): A random walker starting at vertex j has a chance γ m (j) of ending up in the mth cabal B m. Definition: For a digraph G with n vertices with k reaches, we define the n n matrix Γ whose entries are given by: k k Γ ij γ m (i) γ m (j) or Γ = γ m γ m m= m= 2

22 . A S Y M P T O T I C B E H A V I O R 22

23 Asymptotics of Consensus If L is a symmetric (or self-adjoint) square matrix with eigenpairs λ m and η m, then is solved by or x (t) (i) = x (t) = ẋ = Lx n (η m, x (0) )e λmt η m m= ( n n ) η m (i)η m (j)e λ mt x (0) (j) j= m= The terms with Re(λ m ) positive, converge to 0. But non-orthogonality and Jordan blocks destroy this! However, for our bases for kernels of L, we still get the following. Theorem 7 [2]: The consensus problem: satisfies or lim t x(t) (i) = ẋ = Lx ( n k j= m= γ m (i) γ m (j) lim t x(t) = Γx (0) ) x (0) (j) 2

24 Asymptotics of the Random Walk Theorem 8 [2]: The random walk: satisfies lim n p (n+) = p (n) S n n i=0 p (i) = p (0) Γ The p are probability row vectors. Note: in the discrete case we must first average, then take limit! Similar theorems can be formulated for discrete consensus and continuous diffusion. 24

25 Asymptotics: Example γ T = ( ) and γ T 2 = ( 0 0 γ = ( ) and γ 2 = ( 0 0 So k Γ = γ m γ m = m= Let x (0) and p (0) be concentrated on vertex 7 only. Then lim t x(t) = 0 and lim n n n i=0 2 ) 0 0 ) p (i) = (, 0, 2, 2, 2, 0, 0) 9 25

26 . P A G E R A N K 26

27 The Pagerank Algorithm Recall: consensus flows with the arrows, random walk goes against them. The original pagerank algorithm by Page and Brin (as discussed in []). Definition (Pagerank): Let J be the n n all ones matrix. Define, for β = 0.85, say, S p βs + β n J Determine unique invariant probability measure for the random walk S p. Pagerank(i) equals (i). 27

28 Crash Course Pagerank S p βs + β n J S p strictly positive (every vertex sees every other vertex). Therefore: one reach! Thus is unique (thms, 4, 5). S and J are simultaneously diagonalizable. Leading eigenpair: with eigenvector (the all ones vector). All other eigenvectors have eigenvalue at most β Very fast convergence: Can formulate the whole thing without using matrices. Observation: Original algorithm uses S t instead of S. [2] shows that the two rankings are trivially related. 28

29 Dual Approach to Pagerank Thm 7: Displacements in consensus caused by initial displacement x 0 : ẋ = Lx = lim t x (t) = Γx (0) Definition: The influence I(i) of the vertex i is average of the displacements caused by unit displacement e i : I(i) n T Γ e i = n T ( k m= is the all ones vector. γ m γ m ) e i Problem: Non-zero only if γ m e i 0 for some m. Thus I(i) > 0 only if i is in a cabal (by defn γ m ). Not interesting! Definition: The extended graph G α. for every vertex v in V, attach a new vertex b v and an edge b v v with strength α. Think of b v as the boss/owner/administrator of the page v. 29

30 Dual Approach to Pagerank 2 b 4 b 2 2 b b 6 6 b 7 7 b 4 5 b 5 The graph G α has n leaders b i and each of these has a non-zero influence Ĩ(b i). Theorem 9 [2]: If we choose α = β β, then the pagerank (i) of i equals 2Ĩ(b i) n. The factor 2 is because the pagerank in G α is averaged over 2n vertices. We have to subtract n because we do not want to count the displacement of the virtual page b i. From this, [2] derives a trivial relation between the pageranks derived from S p = βs + β n J and the one derived from S p = βs t + β n J (with extra teleporting). 0

31 . *M O R E R E M A R K S

32 *Processes: from Continuous to Discrete Start with the continuous processes: ẋ = Lx (consensus). ṗ = pl (diffusion) Soln: x (t) = e Lt x (0). Time one map: x (n+) = e L x (n). () S (d) e L = I L + L2 2 + ( ) (2) S (d) e L = e S I = e I + S + S2 2 + Properties of e L : () row-sum one, (2) non-negative. Thus S (d) is row-stochastic matrix. So... Obtain Discrete Consensus: x (n+) = S (d) x (n). and Discrete Diffusion: p (n+) = p (n) S (d). (The usual term is random walk.) Define the discrete Laplacian: L (d) = I S (d). From (): Theorem 0 [2]: L (d) and L have the same kernels. As before: the leading eigenspace of S (d) is kernel of L (d). Corollary: The discrete processes have the same asymptotic behavior as the original continuous ones. 2

33 *Processes: Every Discrete Process?? One more Property of e L : Recall ( ) (2) S (d) = e L = e S I = e I + S + S2 2 + Thus e L is transitively closed: if there is a path i j, then there is an edge ij. So, the ( answer ) is NO! Systems with periodic behavior like 0 S = cannot occur as time one maps. 0 Another obstruction is that e L cannot have 0 as eigenvalue. The question exactly which maps can be considered as a time one map of a Laplacian system is open, though several obstructions are known (such as the ones above).

34 *Processes: Periodic Behavior Possibility of periodic behavior changes asymptotics: Consider: Consensus (continuous): ẋ = Lx. Consensus (discrete): x (n+) = Sx (n). The eigenvalues of S lie within the closed unit disk. Asymptotic behavior as t is determined by Continuous: null space of L. Discrete: (i) eigenspace of S assoc. to eigenvalue or. (ii) eigenspaces of S assoc. to roots of unity. All else converges to zero. eigenvalues S eigenvalues Lapl=S I +i i To get asymptotics For discrete: must average: lim n n For continuous, no need: lim t x (t). n k=0 x(k). 4

The pagerank algorithm employs these links to make random walks following links.

35 *Remarks: The Web A web page can be linked to another one (see picture). This means that there is a reference to data in another page that you can land on by tapping or clicking. The pagerank algorithm employs these links to make random walks following links. The stationary measure determines the expected frequency of visits to pages. The higher the frequency, the more important the pages. Important Remark: The flow of information is opposite to the direction of the links. In other words, with our convention the orientation of the edges is reversed. 5

36 *Remarks: Bow-tie Structure of Web (These arrows run against the information flow!) - LSCC or core: Large strongly connected component. - IN component: there is directed path to core. - OUT component: directed path from core; - TENDRILS: pages reachable from IN, or that can reach OUT. - TUBES: paths from IN to OUT. - DISCONNECTED: All other pages. (Sources: [5] in 2000, and [0] in 205.) 6

37 *Definitions: the Usual Laplacian Crude discretization of 2nd deriv. of function f : IR IR: f (j) (f(j + ) f(j)) (f(j) f(j ) or f (j) f(j ) 2f(j) + f(j + ) Suppose has period n (large). Get (combinatorial) Laplacian L = Graph theorists add a - to get eigenvalues 0. Random walk Laplacian: Divide by 2 (and multiply by ). The corresponding graph G: n n 2 7

38 Definitions: Weak Component Example strong component 8

39 *Eigenvalues: Topological Sorting Collapse strong components. Get an acyclic graph. C 0 perturbation of vertices and project to horizontal axis. 9

40 References [] J. S. Caughman, J. J. P. Veerman, Kernels of Directed Graph Laplacians, Electronic Journal of Combinatorics,, No, [2] J. J. P. Veerman, E. Kummel, Diffusion and Consensus on Weakly Connected Directed Graphs, submitted, 208. [] R. Ahlswede et al., Network Information Flow, IEEE Transactions on Information Theory, Vol. 46, No. 4, pp , [4] R. Angles, C. Guiterrez, Survey of Graph Database Models, ACM Computing Surveys, Vol. 40, No., pp. -9, [5] A. Broder et al., Graph Structure of the Web, Computer Networks,, pp , [6] P. Carrington, J. Scott, S. Wasserman, Models and Methods in Social Network Analysis, Cambridge University Press, [7] J. Fax, R Murray, Information Flow and Cooperative Control of Vehicle Formations, IEEE Transactions on Automatic Control, Vol. 49, No. 9, [8] T. Jombert et al., Reconstructing disease outbreaks from genetic data: a graph approach, Heredity 06, 8-90,

41 [9] Robert M. May, Qualitative Stability in Model Ecosystems, Ecology, Vol. 54, No.. (May, 97), pp [0] R. Meusel et al., Graph Structure in the Web Analyzed on Different Aggreegation Levels, The Journal of Web Science,, -47, 205. [] Phillips, S.W. Synthesis of U.S. Geological Survey science for the Chesapeake Bay ecosystem and implications for environmental management, U.S. Geological Survey Circular 6, [2] S. Rao, A. van der Schaft, B. Jayawardhana, A graphtheoretical approach for the analysis and model reduction of complex-balanced chemical reaction networks, J. Math. Chem., Vol. 5, No. 9, pp , 20. [] S. Sternberg, Dynamical Systems, Dover Publications, Mineola, NY, 200, revised edition 20. 4

Kernels of Directed Graph Laplacians. J. S. Caughman and J.J.P. Veerman

Kernels of Directed Graph Laplacians J. S. Caughman and J.J.P. Veerman Department of Mathematics and Statistics Portland State University PO Box 751, Portland, OR 97207. caughman@pdx.edu, veerman@pdx.edu