Local properties of PageRank and graph limits. Nelly Litvak University of Twente Eindhoven University of Technology, The Netherlands MIPT PDF Free Download

Local properties of PageRank and graph limits Nelly Litvak University of Twente Eindhoven University of Technology, The Netherlands MIPT 2018

Centrality in networks Network as a graph G = (V, E)

Centrality in networks Network as a graph G = (V, E) Undirected:

Centrality in networks Network as a graph G = (V, E) Undirected: Facebook, roads networks, airline networks, power grids, co-authorship networks

Centrality in networks Network as a graph G = (V, E) Undirected: Facebook, roads networks, airline networks, power grids, co-authorship networks Directed:

Centrality in networks Network as a graph G = (V, E) Undirected: Facebook, roads networks, airline networks, power grids, co-authorship networks Directed: Twitter, bank transactions, food webs, WWW, scientific citations V set of vertices, E set of edges

Google search Brin and Page 1998 The anatomy of a large-scale hypertextual web search engine. Page, Brin, Motwani and Winograd 1999 The PageRank Citation Ranking: Bringing Order to the Web.

Google search Brin and Page 1998 The anatomy of a large-scale hypertextual web search engine. Page, Brin, Motwani and Winograd 1999 The PageRank Citation Ranking: Bringing Order to the Web. Yahoo, AltaVista,...

Google PageRank PageRank r i of page i = 1,..., n is defined as: r i = j : j i α d j r j + (1 α)q i, i = 1,..., n d j = # out-links of page j α (0, 1), damping factor originally 0.85 q i 0, i q i = 1, originally, q i = 1/n.

Easily bored surfer model r i = j : j i α d j r j + (1 α)q i, i = 1,..., n r i is a stationary distribution of a Markov chain

Easily bored surfer model r i = j : j i α d j r j + (1 α)q i, i = 1,..., n r i is a stationary distribution of a Markov chain with probability α follow a randomly chosen outgoing link with probability 1 α random jump (to page i w.p. q i ) Dangling nodes, d j = 0: Random jump from dangling nodes

PageRank beyond web search Applications: Topic-sensitive search (Haveliwala 2002); Spam detection (Gyöngyi et al. 2004) Finding related entities (Chakrabarti 2007); Link prediction (Liben-Nowell and Kleinberg 2003; Voevodski, Teng, Xia 2009); Finding local cuts (Andersen, Chung, Lang 2006); Graph clustering (Tsiatas, Chung 2010); Person name disambiguation (Smirnova, Avrachenkov, Trousse 2010); Finding most influential people in Wikipedia (Shepelyansky et al 2010, 2013)

Example: food web

Example: food web Allesina and Pascual 2009

Matrix expansion r = αrp + (1 α)q r = (1 α)q[i αp] 1 = (1 α)q Computation by matrix iterations: r (0) = (1/n,..., 1/n) r (k) = αr (k 1) P + (1 α)q α t P t. t=0 k 1 = r (0) α k P k + (1 α)q α t P t t=0

Matrix expansion r = αrp + (1 α)q r = (1 α)q[i αp] 1 = (1 α)q Computation by matrix iterations: r (0) = (1/n,..., 1/n) r (k) = αr (k 1) P + (1 α)q α t P t. t=0 k 1 = r (0) α k P k + (1 α)q α t P t t=0 Exponentially fast convergence due to α (0, 1)

Power laws in complex networks Power laws: Internet, WWW, social networks, biological networks, etc...

Power laws in complex networks Power laws: Internet, WWW, social networks, biological networks, etc... degree of the node = # (in-/out-) links [fraction nodes degree at least k] = p k, Power law: p k const k γ 1, α > 0. Power law is the model for high variability: some nodes (hubs) have extremely many connections

Power law hypothesis for PageRank Pandurangan, Raghavan, Upfal 2002.

Power law hypothesis for PageRank Pandurangan, Raghavan, Upfal 2002. Power law hypothesis for PageRank: When (in-)degree distribution follows a power law then the PageRank follows power law with the same exponent.

Stochastic model for PageRank r i = j : j i α d j r j + (1 α)q i, i = 1,..., n

Stochastic model for PageRank r i = j : j i α d j r j + (1 α)q i, i = 1,..., n Rescale: R i = nr i, Q i n(1 α)q i so that E(R) = 1 Idea: model R as a solution of stochastic equation (Volkovich&L 2010): R d = α N j=1 1 D j R j + Q N: in-degree of the randomly chosen page D: out-degree of page that links to the randomly chosen page R j is distributed as R; N, D, R j are independent Denote C j = α/d j.

Stochastic model for PageRank r i = j : j i α d j r j + (1 α)q i, i = 1,..., n Rescale: R i = nr i, Q i n(1 α)q i so that E(R) = 1 Idea: model R as a solution of stochastic equation (Volkovich&L 2010): R d = N C j R j + Q j=1 N: in-degree of the randomly chosen page D: out-degree of page that links to the randomly chosen page R j is distributed as R; N, D, R j are independent Denote C j = α/d j.

Results for stochastic recursion R d = N C j R j + Q j=1 Theorem (Volkovich&L 2010) If P(Q > x) = o(p(n > x)), then the following are equivalent: P(N > x) L(x)x γ as x, P(R > x) al(x)x γ as x, where a = (E[C]) γ (1 E[N]E[C γ ]) 1 Here a b means a/b 1 Note that E[C] = 1/E(N), the role of out-degrees is minimal!

Results for stochastic recursion R d = N C j R j + Q j=1 Olvera-Cravioto, Jelenkovic 2010, 2012 analyzed the recursion under most general assumptions on C j s. For example, R can be heavy-tailed even when N is light-tailed.

Results for stochastic recursion N R = d C j R j + Q j=1 Olvera-Cravioto, Jelenkovic 2010, 2012 analyzed the recursion under most general assumptions on C j s. For example, R can be heavy-tailed even when N is light-tailed. However, this does not completely explain the behavior of PageRank in networks because the recursion implicitly assumes an underlying tree structure.

PageRank in a random graph Chen, L, Olvera-Cravioto 2017 M = M(n) R n n is related to the adjacency matrix of the graph: { C i P ij, if there are s ij edges from i to j, M i,j = 0, otherwise. Q R n is a personalization vector. Assume E(Q i ) < H. We are interested in the distribution of one coordinate, R (n) 1, of the vector R (n) R n defined by R (n) = R (n) M + Q

Finite approximation of PageRank Assume C i α, E Q i = 1 α. Then R (n,n) R (n, ) 1 We want to bound R (n, ) i Nodes are not identical Q 1 α N+i α N = Q 1 1 α. i=0 R (n,n) i for individual nodes The standard results on mixing times do not help to get rid of the factor n

Local approximation of PageRank Idea: Average over a random node [ [ ] E R Vn (n) R (N) 1 n V n (n) = E n i=1 [ 1 n E α k [ ] QP k] n i i=1 k=n+1 = E 1 n n α k Q j Pji k n = 1 n n i=1 k=n+1 j=1 k=n+1 j=1 α k (1 α) k=n+1 n Pji k α N+1, i=1 [ QM k ] i ]

Local approximation of PageRank w.h.p. Markov inequality: ([ ] ) P R (n, ) 1 R (n,n) 1 > ε = O ( εα k) Approximation with arbitrary precision in finite number of iterations It is a weaker result than bounding R (n, ) i R (n,k) i, but it is good enough

Directed Configuration Model (DCM)

Directed Configuration Model (DCM) F + v 1 v 1 F v 2 v 2.. v n v n

Directed Configuration Model (DCM) F + v 1 v 1 F v 2 v 2.. v n v n We keep self-loops and double edges. The result is a multi-graph

Limiting distribution of PageRank in the DCM Chen, L, Olvera-Cravioto 2017 R D = N C j R j + Q, j=1 Let R denote the endogenous solution to the SFPE above. The endogenous solution is the limit of iterations of the recursion starting, say, from R 0 = 1. Main result: R (n) 1 R, n, where denotes weak convergence and R is given by N 0 R := C j R j + Q 0, j=1

Methodology 1. Finite approximation. PageRank is accurately approximated by a finite number of matrix iterations.

Methodology 1. Finite approximation. PageRank is accurately approximated by a finite number of matrix iterations. 2. Coupling with a tree. Construct a coupling of the DCM graph and a thorny branching tree (TBT). The coupling between the graph and the TBT will hold for a number of generations in the tree that is logarithmic in n. 3. Convergence to a weighted branching process. Show that the rank of the root node of the TBT converges weakly to the stated limit. Chen and Olvera-Cravioto (2014)

Coupling with a branching tree We start with random node (node 1) and explore its neighbors, labeling the stubs that we have already seen τ the number of generations of WBP completed before coupling breaks

Coupling with branching tree Lemma (The Coupling Lemma) Suppose (N n, D n, C n, Q n ) satisfies WLLN, µ = E(ND)/E(D). Then, for any 1 k h log n with 0 < h < 1/(2 log µ), if µ > 1, for any 1 k n b with b < 1/2, if µ 1, we have as n. O ( (n/µ 2k ) 1/2), µ > 1, P (τ k Ω n ) = O ( (n/k 2 ) 1/2), µ = 1, O ( n 1/2), µ < 1, Remark: µ corresponds to the average number of offspring of a node in TBT.

The Coupling Lemma: idea of the proof Ẑ s # individuals in generation s of the tree ˆV s # outbound stubs of all nodes in generation s

The Coupling Lemma: idea of the proof Ẑ s # individuals in generation s of the tree ˆV s # outbound stubs of all nodes in generation s Ẑ s, ˆV s are not much larger than their means: ] [ ] E n [Ẑs µ s+1, E n ˆV s λµ s, λ = E[D 2 ]/µ. An inbound stub of a node in the rth generation will be the first one to be paired with a labeled outbound stub with a probability not larger than P r := 1 L n r ˆV s s=0 λµr n(µ 1).

PageRank and local weak convergence Garavaglia, vd Hofstad, L 2018 Alternative approach:

PageRank and local weak convergence Garavaglia, vd Hofstad, L 2018 Alternative approach: Consider a limit of a graph in a local weak convergence sense

PageRank and local weak convergence Garavaglia, vd Hofstad, L 2018 Alternative approach: Consider a limit of a graph in a local weak convergence sense Prove that PageRank in a random graph converges in distribution to the PageRank of a limit. Note: This does not prove the power law hypothesis! The hope is that in the limiting graph the analysis is easier

Local weak convergence (LWC) LWC for undirected graphs (Benjamini&Schramm 2001)

Local weak convergence (LWC) LWC for undirected graphs (Benjamini&Schramm 2001) (G n ) n N converges to a rooted graph (G, )

Local weak convergence (LWC) LWC for undirected graphs (Benjamini&Schramm 2001) (G n ) n N converges to a rooted graph (G, ) (G, ) is a (possibly) random object

Local weak convergence (LWC) LWC for undirected graphs (Benjamini&Schramm 2001) (G n ) n N converges to a rooted graph (G, ) (G, ) is a (possibly) random object One vertex in (G, ) is labeled as a root

LWC through finite neighborhoods U k (i), i G n, is a neighborhood of i up to distance k in G n

LWC through finite neighborhoods U k (i), i G n, is a neighborhood of i up to distance k in G n (G, ) random rooted graph, is a root

LWC through finite neighborhoods U k (i), i G n, is a neighborhood of i up to distance k in G n (G, ) random rooted graph, is a root G n G in distribution in the LWC sense if for any finite rooted graph (H, y), and any k N, 1 n P ( U k (i) = (H, y) ) P ( U k ( ) = (H, y) ) i [n] Local properties of the graph can pass to the limit

Rooted directed graphs G = (V (G), E(G)) is a directed graph

Rooted directed graphs G = (V (G), E(G)) is a directed graph For every vertex i V (G), di is the in-degree and d i + out-degree is the

Rooted directed graphs G = (V (G), E(G)) is a directed graph For every vertex i V (G), di is the in-degree and d i + is the out-degree Exploring the graph using in-bound edges D i + is a mark assigned to i V (G) such that d i + D i + <. 2 2 1 4 1 4 3 3 M(G) = {D + i } i V (G). (G,, M(G)) is a rooted marked directed graph.

k-neighborhoods in rooted marked directed graphs

k-neighborhoods in rooted marked directed graphs (U k ( ),, M(U k ( ))) k-neighborhood of root

k-neighborhoods in rooted marked directed graphs (U k ( ),, M(U k ( ))) k-neighborhood of root Obtained by exploring in-degree only!

k-neighborhoods in rooted marked directed graphs (U k ( ),, M(U k ( ))) k-neighborhood of root Obtained by exploring in-degree only! Example of a two-neighborhood 2 7 6 5 1 3 5 4 4 6 1 3

Distance for rooted marked directed graphs (G,, M(G)) = (G,, M(G )) isomorphic

Distance for rooted marked directed graphs (G,, M(G)) = (G,, M(G )) isomorphic γ : V (G) V (G ) such that (i, j) E(G) if and only if (γ(i), γ(j)) E(G ); γ( ) = ; for every i V (G), D + i = D + γ(i). For directed graphs (G,, M(G)) and (G,, M(G )), where d loc ( (G,, M(G)), (G,, M(G )) ) = 1 1 + κ, κ = inf { k 0 : (U k ( ),, M(U n ( ))) = (U n ( ),, M(U k ( ))) }, and the inf of the empty set is defined as +

LWC for marked directed graphs For any graph G define 1 P(G) = V (G) i V (G) δ (G,i,M(G)).

LWC for marked directed graphs For any graph G define 1 P(G) = V (G) i V (G) δ (G,i,M(G)). (G n, M(G n )) converges in the LWC sense to a probability P on G if, for any bounded continuous function f : G R, E P(Gn ) [f ] E P [f ] Theorem. G n P in the LW sense iff k N and finite directed rooted marked graph (H, y, M(H)), 1 1 { U k (i) = n (H, y, M(H)) } i [n] P ( U k ( ) = (H, y, M(H)) ).

Main result Theorem (Existence of asymptotic PageRank distribution) Consider a sequence of directed random graphs (G n ) n N. Then: if G n converges in distribution in the local weak sense, then there exists a limiting distribution R, with E[R ] 1, such that d R Vn (n) R ; if G n converges in probability in the local weak sense, then there exists a limiting distribution R, with E[R ] 1, such that, for every r > 0, 1 n 1{R i (n) > r} i [n] P P (R > r).

Scheme of the proof R Vn (n) (A) n R (a) finite N (c) N R (N) (n) Vn (b) n R (N)

Scheme of the proof R Vn (n) (A) n R (a) finite N (c) N R (N) (n) Vn (b) n R (N) ( ) (a) Matrix iterations: P R Vn (n) R (N) V n (n) ε αn+1 ε

Scheme of the proof R Vn (n) (A) n R (a) finite N (c) N R (N) (n) Vn (b) n R (N) ( ) (a) Matrix iterations: P R Vn (n) R (N) V n (n) ε (b) LWC: [ ] ( ) lim E 1{R (N) n V n > r} = lim P R (N) n V n (n) > r αn+1 ε ( ) = P R (N) > r

Scheme of the proof R Vn (n) (A) n R (a) finite N (c) N R (N) (n) Vn (b) n R (N) ( ) (a) Matrix iterations: P R Vn (n) R (N) V n (n) ε (b) LWC: [ ] ( ) lim E 1{R (N) n V n > r} = lim P R (N) n V n (n) > r (c) Monotone convergence of (R (N) ) N N αn+1 ε ( ) = P R (N) > r

Examples where convergence holds Directed Configuration Model with independent in- and out-degrees Chen, L, Olvera-Cravioto 2017

Examples where convergence holds Directed Configuration Model with independent in- and out-degrees Chen, L, Olvera-Cravioto 2017 Directed Configuration Model with dependent in- and out-degrees Garavaglia, vd Hofstad, L 2018 Directed Generalized Random Graphs Lee and Olvera-Cravioto 2017

Ranking algorithms/centrality measures Boldi and Vigna 2014 Based on distances: (in-)degree: number of nodes on distance 1 Closeness centrality (Bavelas 1950) Harmonic centrality (Boldi and Vigna 2014) Based on paths: Betweenness centrality (Anthonisse 1971) Katz s index (Katz 1953) Based on spectrum: Seeley index (Seeley 1949) HITS (Kleinberg 1997) PageRank (Brin, Page, Motwani and Vinograd 1999)

Open Wikipedia ranking http://wikirank.di.unimi.it/

Thank you!

Local properties of PageRank and graph limits. Nelly Litvak University of Twente Eindhoven University of Technology, The Netherlands MIPT 2018