Local properties of PageRank and graph limits. Nelly Litvak University of Twente Eindhoven University of Technology, The Netherlands MIPT 2018

Size: px
Start display at page:

Download "Local properties of PageRank and graph limits. Nelly Litvak University of Twente Eindhoven University of Technology, The Netherlands MIPT 2018"

Transcription

1 Local properties of PageRank and graph limits Nelly Litvak University of Twente Eindhoven University of Technology, The Netherlands MIPT 2018

2 Centrality in networks Network as a graph G = (V, E)

3 Centrality in networks Network as a graph G = (V, E) Undirected:

4 Centrality in networks Network as a graph G = (V, E) Undirected: Facebook, roads networks, airline networks, power grids, co-authorship networks

5 Centrality in networks Network as a graph G = (V, E) Undirected: Facebook, roads networks, airline networks, power grids, co-authorship networks Directed:

6 Centrality in networks Network as a graph G = (V, E) Undirected: Facebook, roads networks, airline networks, power grids, co-authorship networks Directed: Twitter, bank transactions, food webs, WWW, scientific citations

7 Centrality in networks Network as a graph G = (V, E) Undirected: Facebook, roads networks, airline networks, power grids, co-authorship networks Directed: Twitter, bank transactions, food webs, WWW, scientific citations V set of vertices, E set of edges

8 Centrality in networks Network as a graph G = (V, E) Undirected: Facebook, roads networks, airline networks, power grids, co-authorship networks Directed: Twitter, bank transactions, food webs, WWW, scientific citations V set of vertices, E set of edges V = n, we can let n

9 Centrality in networks Network as a graph G = (V, E) Undirected: Facebook, roads networks, airline networks, power grids, co-authorship networks Directed: Twitter, bank transactions, food webs, WWW, scientific citations V set of vertices, E set of edges V = n, we can let n Which nodes are most central in a network?

10 Google search Brin and Page 1998 The anatomy of a large-scale hypertextual web search engine. Page, Brin, Motwani and Winograd 1999 The PageRank Citation Ranking: Bringing Order to the Web.

11 Google search Brin and Page 1998 The anatomy of a large-scale hypertextual web search engine. Page, Brin, Motwani and Winograd 1999 The PageRank Citation Ranking: Bringing Order to the Web. Yahoo, AltaVista,...

12 Google search Brin and Page 1998 The anatomy of a large-scale hypertextual web search engine. Page, Brin, Motwani and Winograd 1999 The PageRank Citation Ranking: Bringing Order to the Web. Yahoo, AltaVista,... Directory-based, comparable to telephone books

13 Google search Brin and Page 1998 The anatomy of a large-scale hypertextual web search engine. Page, Brin, Motwani and Winograd 1999 The PageRank Citation Ranking: Bringing Order to the Web. Yahoo, AltaVista,... Directory-based, comparable to telephone books

14 Google PageRank PageRank r i of page i = 1,..., n is defined as: r i = j : j i α d j r j + (1 α)q i, i = 1,..., n d j = # out-links of page j α (0, 1), damping factor originally 0.85 q i 0, i q i = 1, originally, q i = 1/n.

15 Easily bored surfer model r i = j : j i α d j r j + (1 α)q i, i = 1,..., n r i is a stationary distribution of a Markov chain

16 Easily bored surfer model r i = j : j i α d j r j + (1 α)q i, i = 1,..., n r i is a stationary distribution of a Markov chain with probability α follow a randomly chosen outgoing link with probability 1 α random jump (to page i w.p. q i )

17 Easily bored surfer model r i = j : j i α d j r j + (1 α)q i, i = 1,..., n r i is a stationary distribution of a Markov chain with probability α follow a randomly chosen outgoing link with probability 1 α random jump (to page i w.p. q i ) Dangling nodes, d j = 0: Random jump from dangling nodes

18 Easily bored surfer model r i = j : j i α d j r j + (1 α)q i, i = 1,..., n r i is a stationary distribution of a Markov chain with probability α follow a randomly chosen outgoing link with probability 1 α random jump (to page i w.p. q i ) Dangling nodes, d j = 0: Random jump from dangling nodes Stationary distribution π = r/ r 1

19 Easily bored surfer model r i = j : j i α d j r j + (1 α)q i, i = 1,..., n r i is a stationary distribution of a Markov chain with probability α follow a randomly chosen outgoing link with probability 1 α random jump (to page i w.p. q i ) Dangling nodes, d j = 0: Random jump from dangling nodes Stationary distribution π = r/ r 1 The page is important if many important pages link to it!

20 PageRank beyond web search Applications: Topic-sensitive search (Haveliwala 2002); Spam detection (Gyöngyi et al. 2004) Finding related entities (Chakrabarti 2007); Link prediction (Liben-Nowell and Kleinberg 2003; Voevodski, Teng, Xia 2009); Finding local cuts (Andersen, Chung, Lang 2006); Graph clustering (Tsiatas, Chung 2010); Person name disambiguation (Smirnova, Avrachenkov, Trousse 2010); Finding most influential people in Wikipedia (Shepelyansky et al 2010, 2013)

21 PageRank beyond web search Applications: Topic-sensitive search (Haveliwala 2002); Spam detection (Gyöngyi et al. 2004) Finding related entities (Chakrabarti 2007); Link prediction (Liben-Nowell and Kleinberg 2003; Voevodski, Teng, Xia 2009); Finding local cuts (Andersen, Chung, Lang 2006); Graph clustering (Tsiatas, Chung 2010); Person name disambiguation (Smirnova, Avrachenkov, Trousse 2010); Finding most influential people in Wikipedia (Shepelyansky et al 2010, 2013) Ranking in sports (Beggs et al. 2017, Rojas-Mora et al. 2017, Lazova and Basnarkov 2015) Global characteristic of the graph

22 Example: food web

23 Example: food web Allesina and Pascual 2009

24 Matrix expansion r = αrp + (1 α)q r = (1 α)q[i αp] 1 = (1 α)q Computation by matrix iterations: r (0) = (1/n,..., 1/n) r (k) = αr (k 1) P + (1 α)q α t P t. t=0 k 1 = r (0) α k P k + (1 α)q α t P t t=0

25 Matrix expansion r = αrp + (1 α)q r = (1 α)q[i αp] 1 = (1 α)q Computation by matrix iterations: r (0) = (1/n,..., 1/n) r (k) = αr (k 1) P + (1 α)q α t P t. t=0 k 1 = r (0) α k P k + (1 α)q α t P t t=0 Exponentially fast convergence due to α (0, 1)

26 Matrix expansion r = αrp + (1 α)q r = (1 α)q[i αp] 1 = (1 α)q Computation by matrix iterations: r (0) = (1/n,..., 1/n) r (k) = αr (k 1) P + (1 α)q α t P t. t=0 k 1 = r (0) α k P k + (1 α)q α t P t t=0 Exponentially fast convergence due to α (0, 1) Matrix iterations are used to compute PageRank in practice Langville&Meyer 2004, Berkhin 2005

27 Power laws in complex networks Power laws: Internet, WWW, social networks, biological networks, etc...

28 Power laws in complex networks Power laws: Internet, WWW, social networks, biological networks, etc... degree of the node = # (in-/out-) links [fraction nodes degree at least k] = p k, Power law: p k const k γ 1, α > 0. Power law is the model for high variability: some nodes (hubs) have extremely many connections

29 Power laws in complex networks Power laws: Internet, WWW, social networks, biological networks, etc... degree of the node = # (in-/out-) links [fraction nodes degree at least k] = p k, Power law: p k const k γ 1, α > 0. Power law is the model for high variability: some nodes (hubs) have extremely many connections log p k = log(const) (γ + 1) log k

30 Power laws in complex networks Power laws: Internet, WWW, social networks, biological networks, etc... degree of the node = # (in-/out-) links [fraction nodes degree at least k] = p k, Power law: p k const k γ 1, α > 0. Power law is the model for high variability: some nodes (hubs) have extremely many connections log p k = log(const) (γ + 1) log k Straight line on the log-log scale

31 Power law hypothesis for PageRank Pandurangan, Raghavan, Upfal 2002.

32 Power law hypothesis for PageRank Pandurangan, Raghavan, Upfal Power law hypothesis for PageRank: When (in-)degree distribution follows a power law then the PageRank follows power law with the same exponent.

33 Stochastic model for PageRank r i = j : j i α d j r j + (1 α)q i, i = 1,..., n

34 Stochastic model for PageRank r i = j : j i α d j r j + (1 α)q i, i = 1,..., n Rescale: R i = nr i, Q i n(1 α)q i so that E(R) = 1 Idea: model R as a solution of stochastic equation (Volkovich&L 2010): R d = α N j=1 1 D j R j + Q N: in-degree of the randomly chosen page D: out-degree of page that links to the randomly chosen page R j is distributed as R; N, D, R j are independent Denote C j = α/d j.

35 Stochastic model for PageRank r i = j : j i α d j r j + (1 α)q i, i = 1,..., n Rescale: R i = nr i, Q i n(1 α)q i so that E(R) = 1 Idea: model R as a solution of stochastic equation (Volkovich&L 2010): R d = N C j R j + Q j=1 N: in-degree of the randomly chosen page D: out-degree of page that links to the randomly chosen page R j is distributed as R; N, D, R j are independent Denote C j = α/d j.

36 Results for stochastic recursion R d = N C j R j + Q j=1 Theorem (Volkovich&L 2010) If P(Q > x) = o(p(n > x)), then the following are equivalent: P(N > x) L(x)x γ as x, P(R > x) al(x)x γ as x, where a = (E[C]) γ (1 E[N]E[C γ ]) 1 Here a b means a/b 1 Note that E[C] = 1/E(N), the role of out-degrees is minimal!

37 Results for stochastic recursion R d = N C j R j + Q j=1 Olvera-Cravioto, Jelenkovic 2010, 2012 analyzed the recursion under most general assumptions on C j s. For example, R can be heavy-tailed even when N is light-tailed.

38 Results for stochastic recursion N R = d C j R j + Q j=1 Olvera-Cravioto, Jelenkovic 2010, 2012 analyzed the recursion under most general assumptions on C j s. For example, R can be heavy-tailed even when N is light-tailed. However, this does not completely explain the behavior of PageRank in networks because the recursion implicitly assumes an underlying tree structure.

39 Results for stochastic recursion N R = d C j R j + Q j=1 Olvera-Cravioto, Jelenkovic 2010, 2012 analyzed the recursion under most general assumptions on C j s. For example, R can be heavy-tailed even when N is light-tailed. However, this does not completely explain the behavior of PageRank in networks because the recursion implicitly assumes an underlying tree structure.

40 PageRank in a random graph Chen, L, Olvera-Cravioto 2017 M = M(n) R n n is related to the adjacency matrix of the graph: { C i P ij, if there are s ij edges from i to j, M i,j = 0, otherwise. Q R n is a personalization vector. Assume E(Q i ) < H. We are interested in the distribution of one coordinate, R (n) 1, of the vector R (n) R n defined by R (n) = R (n) M + Q

41 Finite approximation of PageRank Assume C i α, E Q i = 1 α. Then R (n,n) R (n, ) 1 We want to bound R (n, ) i Nodes are not identical Q 1 α N+i α N = Q 1 1 α. i=0 R (n,n) i for individual nodes The standard results on mixing times do not help to get rid of the factor n

42 Local approximation of PageRank Idea: Average over a random node [ [ ] E R Vn (n) R (N) 1 n V n (n) = E n i=1 [ 1 n E α k [ ] QP k] n i i=1 k=n+1 = E 1 n n α k Q j Pji k n = 1 n n i=1 k=n+1 j=1 k=n+1 j=1 α k (1 α) k=n+1 n Pji k α N+1, i=1 [ QM k ] i ]

43 Local approximation of PageRank w.h.p. Markov inequality: ([ ] ) P R (n, ) 1 R (n,n) 1 > ε = O ( εα k) Approximation with arbitrary precision in finite number of iterations It is a weaker result than bounding R (n, ) i R (n,k) i, but it is good enough

44 Directed Configuration Model (DCM)

45 Directed Configuration Model (DCM)

46 Directed Configuration Model (DCM) F + v 1 v 1 F v 2 v 2.. v n v n

47 Directed Configuration Model (DCM) F + v 1 v 1 F v 2 v 2.. v n v n

48 Directed Configuration Model (DCM) F + v 1 v 1 F v 2 v 2.. v n v n

49 Directed Configuration Model (DCM) F + v 1 v 1 F v 2 v 2.. v n v n

50 Directed Configuration Model (DCM) F + v 1 v 1 F v 2 v 2.. v n v n We keep self-loops and double edges. The result is a multi-graph

51 Limiting distribution of PageRank in the DCM Chen, L, Olvera-Cravioto 2017 R D = N C j R j + Q, j=1 Let R denote the endogenous solution to the SFPE above. The endogenous solution is the limit of iterations of the recursion starting, say, from R 0 = 1. Main result: R (n) 1 R, n, where denotes weak convergence and R is given by N 0 R := C j R j + Q 0, j=1

52 Methodology 1. Finite approximation. PageRank is accurately approximated by a finite number of matrix iterations.

53 Methodology 1. Finite approximation. PageRank is accurately approximated by a finite number of matrix iterations. 2. Coupling with a tree. Construct a coupling of the DCM graph and a thorny branching tree (TBT). The coupling between the graph and the TBT will hold for a number of generations in the tree that is logarithmic in n.

54 Methodology 1. Finite approximation. PageRank is accurately approximated by a finite number of matrix iterations. 2. Coupling with a tree. Construct a coupling of the DCM graph and a thorny branching tree (TBT). The coupling between the graph and the TBT will hold for a number of generations in the tree that is logarithmic in n. 3. Convergence to a weighted branching process. Show that the rank of the root node of the TBT converges weakly to the stated limit. Chen and Olvera-Cravioto (2014)

55 Coupling with a branching tree We start with random node (node 1) and explore its neighbors, labeling the stubs that we have already seen τ the number of generations of WBP completed before coupling breaks

56 Coupling with branching tree Lemma (The Coupling Lemma) Suppose (N n, D n, C n, Q n ) satisfies WLLN, µ = E(ND)/E(D). Then, for any 1 k h log n with 0 < h < 1/(2 log µ), if µ > 1, for any 1 k n b with b < 1/2, if µ 1, we have as n. O ( (n/µ 2k ) 1/2), µ > 1, P (τ k Ω n ) = O ( (n/k 2 ) 1/2), µ = 1, O ( n 1/2), µ < 1, Remark: µ corresponds to the average number of offspring of a node in TBT.

57 The Coupling Lemma: idea of the proof Ẑ s # individuals in generation s of the tree ˆV s # outbound stubs of all nodes in generation s

58 The Coupling Lemma: idea of the proof Ẑ s # individuals in generation s of the tree ˆV s # outbound stubs of all nodes in generation s Ẑ s, ˆV s are not much larger than their means: ] [ ] E n [Ẑs µ s+1, E n ˆV s λµ s, λ = E[D 2 ]/µ.

59 The Coupling Lemma: idea of the proof Ẑ s # individuals in generation s of the tree ˆV s # outbound stubs of all nodes in generation s Ẑ s, ˆV s are not much larger than their means: ] [ ] E n [Ẑs µ s+1, E n ˆV s λµ s, λ = E[D 2 ]/µ. An inbound stub of a node in the rth generation will be the first one to be paired with a labeled outbound stub with a probability not larger than P r := 1 L n r ˆV s s=0 λµr n(µ 1).

60 The Coupling Lemma: idea of the proof Ẑ s # individuals in generation s of the tree ˆV s # outbound stubs of all nodes in generation s Ẑ s, ˆV s are not much larger than their means: ] [ ] E n [Ẑs µ s+1, E n ˆV s λµ s, λ = E[D 2 ]/µ. An inbound stub of a node in the rth generation will be the first one to be paired with a labeled outbound stub with a probability not larger than P r := 1 L n r ˆV s s=0 λµr n(µ 1). {τ = r} is equivalent to the event that Binomial(Ẑ r, P r ) r.v. is greater or equal than 1.

61 The Coupling Lemma: idea of the proof Ẑ s # individuals in generation s of the tree ˆV s # outbound stubs of all nodes in generation s Ẑ s, ˆV s are not much larger than their means: ] [ ] E n [Ẑs µ s+1, E n ˆV s λµ s, λ = E[D 2 ]/µ. An inbound stub of a node in the rth generation will be the first one to be paired with a labeled outbound stub with a probability not larger than P r := 1 L n r ˆV s s=0 λµr n(µ 1). {τ = r} is equivalent to the event that Binomial(Ẑ r, P r ) r.v. is greater or equal than 1. Markov s inequality: P(τ = r) Ẑr P r = O ( µ 2r n 1), r k.

62 PageRank and local weak convergence Garavaglia, vd Hofstad, L 2018 Alternative approach:

63 PageRank and local weak convergence Garavaglia, vd Hofstad, L 2018 Alternative approach: Consider a limit of a graph in a local weak convergence sense

64 PageRank and local weak convergence Garavaglia, vd Hofstad, L 2018 Alternative approach: Consider a limit of a graph in a local weak convergence sense Prove that PageRank in a random graph converges in distribution to the PageRank of a limit.

65 PageRank and local weak convergence Garavaglia, vd Hofstad, L 2018 Alternative approach: Consider a limit of a graph in a local weak convergence sense Prove that PageRank in a random graph converges in distribution to the PageRank of a limit. Note: This does not prove the power law hypothesis!

66 PageRank and local weak convergence Garavaglia, vd Hofstad, L 2018 Alternative approach: Consider a limit of a graph in a local weak convergence sense Prove that PageRank in a random graph converges in distribution to the PageRank of a limit. Note: This does not prove the power law hypothesis! The hope is that in the limiting graph the analysis is easier

67 PageRank and local weak convergence Garavaglia, vd Hofstad, L 2018 Alternative approach: Consider a limit of a graph in a local weak convergence sense Prove that PageRank in a random graph converges in distribution to the PageRank of a limit. Note: This does not prove the power law hypothesis! The hope is that in the limiting graph the analysis is easier Example: if the limit is a tree then the PageRank satisfies the stochastic recursion

68 Local weak convergence (LWC) LWC for undirected graphs (Benjamini&Schramm 2001)

69 Local weak convergence (LWC) LWC for undirected graphs (Benjamini&Schramm 2001) (G n ) n N converges to a rooted graph (G, )

70 Local weak convergence (LWC) LWC for undirected graphs (Benjamini&Schramm 2001) (G n ) n N converges to a rooted graph (G, ) (G, ) is a (possibly) random object

71 Local weak convergence (LWC) LWC for undirected graphs (Benjamini&Schramm 2001) (G n ) n N converges to a rooted graph (G, ) (G, ) is a (possibly) random object One vertex in (G, ) is labeled as a root

72 LWC through finite neighborhoods U k (i), i G n, is a neighborhood of i up to distance k in G n

73 LWC through finite neighborhoods U k (i), i G n, is a neighborhood of i up to distance k in G n (G, ) random rooted graph, is a root

74 LWC through finite neighborhoods U k (i), i G n, is a neighborhood of i up to distance k in G n (G, ) random rooted graph, is a root G n G in distribution in the LWC sense if for any finite rooted graph (H, y), and any k N, 1 n P ( U k (i) = (H, y) ) P ( U k ( ) = (H, y) ) i [n]

75 LWC through finite neighborhoods U k (i), i G n, is a neighborhood of i up to distance k in G n (G, ) random rooted graph, is a root G n G in distribution in the LWC sense if for any finite rooted graph (H, y), and any k N, 1 n P ( U k (i) = (H, y) ) P ( U k ( ) = (H, y) ) i [n] Local properties of the graph can pass to the limit

76 LWC through finite neighborhoods U k (i), i G n, is a neighborhood of i up to distance k in G n (G, ) random rooted graph, is a root G n G in distribution in the LWC sense if for any finite rooted graph (H, y), and any k N, 1 n P ( U k (i) = (H, y) ) P ( U k ( ) = (H, y) ) i [n] Local properties of the graph can pass to the limit V n a uniformly chosen vertex in G n, then lim E [f (G n, V n )] = E [f (G, )] n

77 Rooted directed graphs G = (V (G), E(G)) is a directed graph

78 Rooted directed graphs G = (V (G), E(G)) is a directed graph For every vertex i V (G), di is the in-degree and d i + out-degree is the

79 Rooted directed graphs G = (V (G), E(G)) is a directed graph For every vertex i V (G), di is the in-degree and d i + is the out-degree Exploring the graph using in-bound edges D i + is a mark assigned to i V (G) such that d i + D i + < M(G) = {D + i } i V (G). (G,, M(G)) is a rooted marked directed graph.

80 k-neighborhoods in rooted marked directed graphs

81 k-neighborhoods in rooted marked directed graphs (U k ( ),, M(U k ( ))) k-neighborhood of root

82 k-neighborhoods in rooted marked directed graphs (U k ( ),, M(U k ( ))) k-neighborhood of root Obtained by exploring in-degree only!

83 k-neighborhoods in rooted marked directed graphs (U k ( ),, M(U k ( ))) k-neighborhood of root Obtained by exploring in-degree only! Example of a two-neighborhood

84 Distance for rooted marked directed graphs (G,, M(G)) = (G,, M(G )) isomorphic

85 Distance for rooted marked directed graphs (G,, M(G)) = (G,, M(G )) isomorphic γ : V (G) V (G ) such that (i, j) E(G) if and only if (γ(i), γ(j)) E(G ); γ( ) = ; for every i V (G), D + i = D + γ(i).

86 Distance for rooted marked directed graphs (G,, M(G)) = (G,, M(G )) isomorphic γ : V (G) V (G ) such that (i, j) E(G) if and only if (γ(i), γ(j)) E(G ); γ( ) = ; for every i V (G), D + i = D + γ(i). For directed graphs (G,, M(G)) and (G,, M(G )), where d loc ( (G,, M(G)), (G,, M(G )) ) = κ, κ = inf { k 0 : (U k ( ),, M(U n ( ))) = (U n ( ),, M(U k ( ))) }, and the inf of the empty set is defined as +

87 Distance for rooted marked directed graphs (G,, M(G)) = (G,, M(G )) isomorphic γ : V (G) V (G ) such that (i, j) E(G) if and only if (γ(i), γ(j)) E(G ); γ( ) = ; for every i V (G), D + i = D + γ(i). For directed graphs (G,, M(G)) and (G,, M(G )), where d loc ( (G,, M(G)), (G,, M(G )) ) = κ, κ = inf { k 0 : (U k ( ),, M(U n ( ))) = (U n ( ),, M(U k ( ))) }, and the inf of the empty set is defined as + d loc is not a metric but a pseudometric

88 Distance for rooted marked directed graphs (G,, M(G)) = (G,, M(G )) isomorphic γ : V (G) V (G ) such that (i, j) E(G) if and only if (γ(i), γ(j)) E(G ); γ( ) = ; for every i V (G), D + i = D + γ(i). For directed graphs (G,, M(G)) and (G,, M(G )), where d loc ( (G,, M(G)), (G,, M(G )) ) = κ, κ = inf { k 0 : (U k ( ),, M(U n ( ))) = (U n ( ),, M(U k ( ))) }, and the inf of the empty set is defined as + d loc is not a metric but a pseudometric (G,, M(G)) G is an equivalence class of the relation d loc = 0

89 LWC for marked directed graphs For any graph G define 1 P(G) = V (G) i V (G) δ (G,i,M(G)).

90 LWC for marked directed graphs For any graph G define 1 P(G) = V (G) i V (G) δ (G,i,M(G)). (G n, M(G n )) converges in the LWC sense to a probability P on G if, for any bounded continuous function f : G R, E P(Gn ) [f ] E P [f ]

91 LWC for marked directed graphs For any graph G define 1 P(G) = V (G) i V (G) δ (G,i,M(G)). (G n, M(G n )) converges in the LWC sense to a probability P on G if, for any bounded continuous function f : G R, E P(Gn ) [f ] E P [f ] Theorem. G n P in the LW sense iff k N and finite directed rooted marked graph (H, y, M(H)), 1 1 { U k (i) = n (H, y, M(H)) } i [n] P ( U k ( ) = (H, y, M(H)) ).

92 Main result Theorem (Existence of asymptotic PageRank distribution) Consider a sequence of directed random graphs (G n ) n N. Then: if G n converges in distribution in the local weak sense, then there exists a limiting distribution R, with E[R ] 1, such that d R Vn (n) R ; if G n converges in probability in the local weak sense, then there exists a limiting distribution R, with E[R ] 1, such that, for every r > 0, 1 n 1{R i (n) > r} i [n] P P (R > r).

93 Scheme of the proof R Vn (n) (A) n R (a) finite N (c) N R (N) (n) Vn (b) n R (N)

94 Scheme of the proof R Vn (n) (A) n R (a) finite N (c) N R (N) (n) Vn (b) n R (N) ( ) (a) Matrix iterations: P R Vn (n) R (N) V n (n) ε αn+1 ε

95 Scheme of the proof R Vn (n) (A) n R (a) finite N (c) N R (N) (n) Vn (b) n R (N) ( ) (a) Matrix iterations: P R Vn (n) R (N) V n (n) ε (b) LWC: [ ] ( ) lim E 1{R (N) n V n > r} = lim P R (N) n V n (n) > r αn+1 ε ( ) = P R (N) > r

96 Scheme of the proof R Vn (n) (A) n R (a) finite N (c) N R (N) (n) Vn (b) n R (N) ( ) (a) Matrix iterations: P R Vn (n) R (N) V n (n) ε (b) LWC: [ ] ( ) lim E 1{R (N) n V n > r} = lim P R (N) n V n (n) > r (c) Monotone convergence of (R (N) ) N N αn+1 ε ( ) = P R (N) > r

97 Examples where convergence holds Directed Configuration Model with independent in- and out-degrees Chen, L, Olvera-Cravioto 2017

98 Examples where convergence holds Directed Configuration Model with independent in- and out-degrees Chen, L, Olvera-Cravioto 2017 Directed Configuration Model with dependent in- and out-degrees Garavaglia, vd Hofstad, L 2018

99 Examples where convergence holds Directed Configuration Model with independent in- and out-degrees Chen, L, Olvera-Cravioto 2017 Directed Configuration Model with dependent in- and out-degrees Garavaglia, vd Hofstad, L 2018 Directed Generalized Random Graphs Lee and Olvera-Cravioto 2017

100 Examples where convergence holds Directed Configuration Model with independent in- and out-degrees Chen, L, Olvera-Cravioto 2017 Directed Configuration Model with dependent in- and out-degrees Garavaglia, vd Hofstad, L 2018 Directed Generalized Random Graphs Lee and Olvera-Cravioto 2017 Directed Preferential Attachment Model Garavaglia, vd Hofstad, L 2018

101 Ranking algorithms/centrality measures Boldi and Vigna 2014 Based on distances: (in-)degree: number of nodes on distance 1 Closeness centrality (Bavelas 1950) Harmonic centrality (Boldi and Vigna 2014) Based on paths: Betweenness centrality (Anthonisse 1971) Katz s index (Katz 1953) Based on spectrum: Seeley index (Seeley 1949) HITS (Kleinberg 1997) PageRank (Brin, Page, Motwani and Vinograd 1999)

102 Open Wikipedia ranking

103 Thank you!

Finding central nodes in large networks

Finding central nodes in large networks Finding central nodes in large networks Nelly Litvak University of Twente Eindhoven University of Technology, The Netherlands Woudschoten Conference 2017 Complex networks Networks: Internet, WWW, social

More information

Online Social Networks and Media. Link Analysis and Web Search

Online Social Networks and Media. Link Analysis and Web Search Online Social Networks and Media Link Analysis and Web Search How to Organize the Web First try: Human curated Web directories Yahoo, DMOZ, LookSmart How to organize the web Second try: Web Search Information

More information

DATA MINING LECTURE 13. Link Analysis Ranking PageRank -- Random walks HITS

DATA MINING LECTURE 13. Link Analysis Ranking PageRank -- Random walks HITS DATA MINING LECTURE 3 Link Analysis Ranking PageRank -- Random walks HITS How to organize the web First try: Manually curated Web Directories How to organize the web Second try: Web Search Information

More information

Link Analysis Ranking

Link Analysis Ranking Link Analysis Ranking How do search engines decide how to rank your query results? Guess why Google ranks the query results the way it does How would you do it? Naïve ranking of query results Given query

More information

Analysis of Google s PageRank

Analysis of Google s PageRank Analysis of Google s PageRank Ilse Ipsen North Carolina State University Joint work with Rebecca M. Wills AN05 p.1 PageRank An objective measure of the citation importance of a web page [Brin & Page 1998]

More information

Link Analysis. Leonid E. Zhukov

Link Analysis. Leonid E. Zhukov Link Analysis Leonid E. Zhukov School of Data Analysis and Artificial Intelligence Department of Computer Science National Research University Higher School of Economics Structural Analysis and Visualization

More information

Monte Carlo methods in PageRank computation: When one iteration is sufficient

Monte Carlo methods in PageRank computation: When one iteration is sufficient Monte Carlo methods in PageRank computation: When one iteration is sufficient Nelly Litvak (University of Twente, The Netherlands) e-mail: n.litvak@ewi.utwente.nl Konstantin Avrachenkov (INRIA Sophia Antipolis,

More information

Mathematical Properties & Analysis of Google s PageRank

Mathematical Properties & Analysis of Google s PageRank Mathematical Properties & Analysis of Google s PageRank Ilse Ipsen North Carolina State University, USA Joint work with Rebecca M. Wills Cedya p.1 PageRank An objective measure of the citation importance

More information

Personalized PageRank with node-dependent restart

Personalized PageRank with node-dependent restart Personalized PageRank with node-dependent restart Konstantin Avrachenkov Remco van der Hofstad Marina Sokol May 204 Abstract Personalized PageRank is an algorithm to classify the improtance of web pages

More information

Uncertainty and Randomization

Uncertainty and Randomization Uncertainty and Randomization The PageRank Computation in Google Roberto Tempo IEIIT-CNR Politecnico di Torino tempo@polito.it 1993: Robustness of Linear Systems 1993: Robustness of Linear Systems 16 Years

More information

CS 277: Data Mining. Mining Web Link Structure. CS 277: Data Mining Lectures Analyzing Web Link Structure Padhraic Smyth, UC Irvine

CS 277: Data Mining. Mining Web Link Structure. CS 277: Data Mining Lectures Analyzing Web Link Structure Padhraic Smyth, UC Irvine CS 277: Data Mining Mining Web Link Structure Class Presentations In-class, Tuesday and Thursday next week 2-person teams: 6 minutes, up to 6 slides, 3 minutes/slides each person 1-person teams 4 minutes,

More information

LINK ANALYSIS. Dr. Gjergji Kasneci Introduction to Information Retrieval WS

LINK ANALYSIS. Dr. Gjergji Kasneci Introduction to Information Retrieval WS LINK ANALYSIS Dr. Gjergji Kasneci Introduction to Information Retrieval WS 2012-13 1 Outline Intro Basics of probability and information theory Retrieval models Retrieval evaluation Link analysis Models

More information

Slides based on those in:

Slides based on those in: Spyros Kontogiannis & Christos Zaroliagis Slides based on those in: http://www.mmds.org High dim. data Graph data Infinite data Machine learning Apps Locality sensitive hashing PageRank, SimRank Filtering

More information

Analysis and Computation of Google s PageRank

Analysis and Computation of Google s PageRank Analysis and Computation of Google s PageRank Ilse Ipsen North Carolina State University, USA Joint work with Rebecca S. Wills ANAW p.1 PageRank An objective measure of the citation importance of a web

More information

Online Social Networks and Media. Link Analysis and Web Search

Online Social Networks and Media. Link Analysis and Web Search Online Social Networks and Media Link Analysis and Web Search How to Organize the Web First try: Human curated Web directories Yahoo, DMOZ, LookSmart How to organize the web Second try: Web Search Information

More information

6.207/14.15: Networks Lecture 7: Search on Networks: Navigation and Web Search

6.207/14.15: Networks Lecture 7: Search on Networks: Navigation and Web Search 6.207/14.15: Networks Lecture 7: Search on Networks: Navigation and Web Search Daron Acemoglu and Asu Ozdaglar MIT September 30, 2009 1 Networks: Lecture 7 Outline Navigation (or decentralized search)

More information

ECEN 689 Special Topics in Data Science for Communications Networks

ECEN 689 Special Topics in Data Science for Communications Networks ECEN 689 Special Topics in Data Science for Communications Networks Nick Duffield Department of Electrical & Computer Engineering Texas A&M University Lecture 8 Random Walks, Matrices and PageRank Graphs

More information

Link Analysis Information Retrieval and Data Mining. Prof. Matteo Matteucci

Link Analysis Information Retrieval and Data Mining. Prof. Matteo Matteucci Link Analysis Information Retrieval and Data Mining Prof. Matteo Matteucci Hyperlinks for Indexing and Ranking 2 Page A Hyperlink Page B Intuitions The anchor text might describe the target page B Anchor

More information

Complex Social System, Elections. Introduction to Network Analysis 1

Complex Social System, Elections. Introduction to Network Analysis 1 Complex Social System, Elections Introduction to Network Analysis 1 Complex Social System, Network I person A voted for B A is more central than B if more people voted for A In-degree centrality index

More information

PAGERANK PARAMETERS. Amy N. Langville. American Institute of Mathematics Workshop on Ranking Palo Alto, CA August 17th, 2010

PAGERANK PARAMETERS. Amy N. Langville. American Institute of Mathematics Workshop on Ranking Palo Alto, CA August 17th, 2010 PAGERANK PARAMETERS 100David F. Gleich 120 Amy N. Langville American Institute of Mathematics Workshop on Ranking Palo Alto, CA August 17th, 2010 Gleich & Langville AIM 1 / 21 The most important page on

More information

Quick Detection of Top-k Personalized PageRank Lists

Quick Detection of Top-k Personalized PageRank Lists Quick Detection of Top-k Personalized PageRank Lists K. Avrachenkov, N. Litvak 2, D. Nemirovsky, E. Smirnova, and M. Sokol INRIA Sophia Antipolis {k.avrachenkov,dnemirov,esmirnov,msokol}@sophia.inria.fr

More information

Wiki Definition. Reputation Systems I. Outline. Introduction to Reputations. Yury Lifshits. HITS, PageRank, SALSA, ebay, EigenTrust, VKontakte

Wiki Definition. Reputation Systems I. Outline. Introduction to Reputations. Yury Lifshits. HITS, PageRank, SALSA, ebay, EigenTrust, VKontakte Reputation Systems I HITS, PageRank, SALSA, ebay, EigenTrust, VKontakte Yury Lifshits Wiki Definition Reputation is the opinion (more technically, a social evaluation) of the public toward a person, a

More information

Inf 2B: Ranking Queries on the WWW

Inf 2B: Ranking Queries on the WWW Inf B: Ranking Queries on the WWW Kyriakos Kalorkoti School of Informatics University of Edinburgh Queries Suppose we have an Inverted Index for a set of webpages. Disclaimer Not really the scenario of

More information

Node and Link Analysis

Node and Link Analysis Node and Link Analysis Leonid E. Zhukov School of Applied Mathematics and Information Science National Research University Higher School of Economics 10.02.2014 Leonid E. Zhukov (HSE) Lecture 5 10.02.2014

More information

Introduction to Search Engine Technology Introduction to Link Structure Analysis. Ronny Lempel Yahoo Labs, Haifa

Introduction to Search Engine Technology Introduction to Link Structure Analysis. Ronny Lempel Yahoo Labs, Haifa Introduction to Search Engine Technology Introduction to Link Structure Analysis Ronny Lempel Yahoo Labs, Haifa Outline Anchor-text indexing Mathematical Background Motivation for link structure analysis

More information

Introduction to Data Mining

Introduction to Data Mining Introduction to Data Mining Lecture #9: Link Analysis Seoul National University 1 In This Lecture Motivation for link analysis Pagerank: an important graph ranking algorithm Flow and random walk formulation

More information

Web Ranking. Classification (manual, automatic) Link Analysis (today s lesson)

Web Ranking. Classification (manual, automatic) Link Analysis (today s lesson) Link Analysis Web Ranking Documents on the web are first ranked according to their relevance vrs the query Additional ranking methods are needed to cope with huge amount of information Additional ranking

More information

Web Structure Mining Nodes, Links and Influence

Web Structure Mining Nodes, Links and Influence Web Structure Mining Nodes, Links and Influence 1 Outline 1. Importance of nodes 1. Centrality 2. Prestige 3. Page Rank 4. Hubs and Authority 5. Metrics comparison 2. Link analysis 3. Influence model 1.

More information

1 Searching the World Wide Web

1 Searching the World Wide Web Hubs and Authorities in a Hyperlinked Environment 1 Searching the World Wide Web Because diverse users each modify the link structure of the WWW within a relatively small scope by creating web-pages on

More information

CS246: Mining Massive Datasets Jure Leskovec, Stanford University

CS246: Mining Massive Datasets Jure Leskovec, Stanford University CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu 2/7/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 2 Web pages are not equally important www.joe-schmoe.com

More information

PageRank. Ryan Tibshirani /36-662: Data Mining. January Optional reading: ESL 14.10

PageRank. Ryan Tibshirani /36-662: Data Mining. January Optional reading: ESL 14.10 PageRank Ryan Tibshirani 36-462/36-662: Data Mining January 24 2012 Optional reading: ESL 14.10 1 Information retrieval with the web Last time we learned about information retrieval. We learned how to

More information

Math 304 Handout: Linear algebra, graphs, and networks.

Math 304 Handout: Linear algebra, graphs, and networks. Math 30 Handout: Linear algebra, graphs, and networks. December, 006. GRAPHS AND ADJACENCY MATRICES. Definition. A graph is a collection of vertices connected by edges. A directed graph is a graph all

More information

Slide source: Mining of Massive Datasets Jure Leskovec, Anand Rajaraman, Jeff Ullman Stanford University.

Slide source: Mining of Massive Datasets Jure Leskovec, Anand Rajaraman, Jeff Ullman Stanford University. Slide source: Mining of Massive Datasets Jure Leskovec, Anand Rajaraman, Jeff Ullman Stanford University http://www.mmds.org #1: C4.5 Decision Tree - Classification (61 votes) #2: K-Means - Clustering

More information

Computing PageRank using Power Extrapolation

Computing PageRank using Power Extrapolation Computing PageRank using Power Extrapolation Taher Haveliwala, Sepandar Kamvar, Dan Klein, Chris Manning, and Gene Golub Stanford University Abstract. We present a novel technique for speeding up the computation

More information

Node Centrality and Ranking on Networks

Node Centrality and Ranking on Networks Node Centrality and Ranking on Networks Leonid E. Zhukov School of Data Analysis and Artificial Intelligence Department of Computer Science National Research University Higher School of Economics Social

More information

Calculating Web Page Authority Using the PageRank Algorithm

Calculating Web Page Authority Using the PageRank Algorithm Jacob Miles Prystowsky and Levi Gill Math 45, Fall 2005 1 Introduction 1.1 Abstract In this document, we examine how the Google Internet search engine uses the PageRank algorithm to assign quantitatively

More information

MAE 298, Lecture 8 Feb 4, Web search and decentralized search on small-worlds

MAE 298, Lecture 8 Feb 4, Web search and decentralized search on small-worlds MAE 298, Lecture 8 Feb 4, 2008 Web search and decentralized search on small-worlds Search for information Assume some resource of interest is stored at the vertices of a network: Web pages Files in a file-sharing

More information

PageRank algorithm Hubs and Authorities. Data mining. Web Data Mining PageRank, Hubs and Authorities. University of Szeged.

PageRank algorithm Hubs and Authorities. Data mining. Web Data Mining PageRank, Hubs and Authorities. University of Szeged. Web Data Mining PageRank, University of Szeged Why ranking web pages is useful? We are starving for knowledge It earns Google a bunch of money. How? How does the Web looks like? Big strongly connected

More information

Personalized PageRank with Node-dependent Restart

Personalized PageRank with Node-dependent Restart Reprint from Moscow Journal of Combinatorics and Number Theory 204, vol. 4, iss. 4, pp. 5 20, [pp. 387 402] Scale =.7 PS:./fig-eps/Logo-MJCNT.eps Personalized PageRank with Node-dependent Restart Konstantin

More information

Algebraic Representation of Networks

Algebraic Representation of Networks Algebraic Representation of Networks 0 1 2 1 1 0 0 1 2 0 0 1 1 1 1 1 Hiroki Sayama sayama@binghamton.edu Describing networks with matrices (1) Adjacency matrix A matrix with rows and columns labeled by

More information

CS224W: Social and Information Network Analysis Jure Leskovec, Stanford University

CS224W: Social and Information Network Analysis Jure Leskovec, Stanford University CS224W: Social and Information Network Analysis Jure Leskovec, Stanford University http://cs224w.stanford.edu How to organize/navigate it? First try: Human curated Web directories Yahoo, DMOZ, LookSmart

More information

Link Analysis. Stony Brook University CSE545, Fall 2016

Link Analysis. Stony Brook University CSE545, Fall 2016 Link Analysis Stony Brook University CSE545, Fall 2016 The Web, circa 1998 The Web, circa 1998 The Web, circa 1998 Match keywords, language (information retrieval) Explore directory The Web, circa 1998

More information

Page rank computation HPC course project a.y

Page rank computation HPC course project a.y Page rank computation HPC course project a.y. 2015-16 Compute efficient and scalable Pagerank MPI, Multithreading, SSE 1 PageRank PageRank is a link analysis algorithm, named after Brin & Page [1], and

More information

The Push Algorithm for Spectral Ranking

The Push Algorithm for Spectral Ranking The Push Algorithm for Spectral Ranking Paolo Boldi Sebastiano Vigna March 8, 204 Abstract The push algorithm was proposed first by Jeh and Widom [6] in the context of personalized PageRank computations

More information

CSI 445/660 Part 6 (Centrality Measures for Networks) 6 1 / 68

CSI 445/660 Part 6 (Centrality Measures for Networks) 6 1 / 68 CSI 445/660 Part 6 (Centrality Measures for Networks) 6 1 / 68 References 1 L. Freeman, Centrality in Social Networks: Conceptual Clarification, Social Networks, Vol. 1, 1978/1979, pp. 215 239. 2 S. Wasserman

More information

Lecture 12: Link Analysis for Web Retrieval

Lecture 12: Link Analysis for Web Retrieval Lecture 12: Link Analysis for Web Retrieval Trevor Cohn COMP90042, 2015, Semester 1 What we ll learn in this lecture The web as a graph Page-rank method for deriving the importance of pages Hubs and authorities

More information

Applications to network analysis: Eigenvector centrality indices Lecture notes

Applications to network analysis: Eigenvector centrality indices Lecture notes Applications to network analysis: Eigenvector centrality indices Lecture notes Dario Fasino, University of Udine (Italy) Lecture notes for the second part of the course Nonnegative and spectral matrix

More information

Thanks to Jure Leskovec, Stanford and Panayiotis Tsaparas, Univ. of Ioannina for slides

Thanks to Jure Leskovec, Stanford and Panayiotis Tsaparas, Univ. of Ioannina for slides Thanks to Jure Leskovec, Stanford and Panayiotis Tsaparas, Univ. of Ioannina for slides Web Search: How to Organize the Web? Ranking Nodes on Graphs Hubs and Authorities PageRank How to Solve PageRank

More information

Four graph partitioning algorithms. Fan Chung University of California, San Diego

Four graph partitioning algorithms. Fan Chung University of California, San Diego Four graph partitioning algorithms Fan Chung University of California, San Diego History of graph partitioning NP-hard approximation algorithms Spectral method, Fiedler 73, Folklore Multicommunity flow,

More information

Data Mining and Matrices

Data Mining and Matrices Data Mining and Matrices 10 Graphs II Rainer Gemulla, Pauli Miettinen Jul 4, 2013 Link analysis The web as a directed graph Set of web pages with associated textual content Hyperlinks between webpages

More information

Spectral Graph Theory and You: Matrix Tree Theorem and Centrality Metrics

Spectral Graph Theory and You: Matrix Tree Theorem and Centrality Metrics Spectral Graph Theory and You: and Centrality Metrics Jonathan Gootenberg March 11, 2013 1 / 19 Outline of Topics 1 Motivation Basics of Spectral Graph Theory Understanding the characteristic polynomial

More information

Lecture: Local Spectral Methods (1 of 4)

Lecture: Local Spectral Methods (1 of 4) Stat260/CS294: Spectral Graph Methods Lecture 18-03/31/2015 Lecture: Local Spectral Methods (1 of 4) Lecturer: Michael Mahoney Scribe: Michael Mahoney Warning: these notes are still very rough. They provide

More information

Data and Algorithms of the Web

Data and Algorithms of the Web Data and Algorithms of the Web Link Analysis Algorithms Page Rank some slides from: Anand Rajaraman, Jeffrey D. Ullman InfoLab (Stanford University) Link Analysis Algorithms Page Rank Hubs and Authorities

More information

Degree Distribution: The case of Citation Networks

Degree Distribution: The case of Citation Networks Network Analysis Degree Distribution: The case of Citation Networks Papers (in almost all fields) refer to works done earlier on same/related topics Citations A network can be defined as Each node is

More information

IR: Information Retrieval

IR: Information Retrieval / 44 IR: Information Retrieval FIB, Master in Innovation and Research in Informatics Slides by Marta Arias, José Luis Balcázar, Ramon Ferrer-i-Cancho, Ricard Gavaldá Department of Computer Science, UPC

More information

CS6220: DATA MINING TECHNIQUES

CS6220: DATA MINING TECHNIQUES CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun yzsun@ccs.neu.edu November 16, 2015 Methods to Learn Classification Clustering Frequent Pattern Mining Matrix Data Decision

More information

CS6220: DATA MINING TECHNIQUES

CS6220: DATA MINING TECHNIQUES CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun yzsun@ccs.neu.edu March 16, 2016 Methods to Learn Classification Clustering Frequent Pattern Mining Matrix Data Decision

More information

Thanks to Jure Leskovec, Stanford and Panayiotis Tsaparas, Univ. of Ioannina for slides

Thanks to Jure Leskovec, Stanford and Panayiotis Tsaparas, Univ. of Ioannina for slides Thanks to Jure Leskovec, Stanford and Panayiotis Tsaparas, Univ. of Ioannina for slides Web Search: How to Organize the Web? Ranking Nodes on Graphs Hubs and Authorities PageRank How to Solve PageRank

More information

INTRODUCTION TO MCMC AND PAGERANK. Eric Vigoda Georgia Tech. Lecture for CS 6505

INTRODUCTION TO MCMC AND PAGERANK. Eric Vigoda Georgia Tech. Lecture for CS 6505 INTRODUCTION TO MCMC AND PAGERANK Eric Vigoda Georgia Tech Lecture for CS 6505 1 MARKOV CHAIN BASICS 2 ERGODICITY 3 WHAT IS THE STATIONARY DISTRIBUTION? 4 PAGERANK 5 MIXING TIME 6 PREVIEW OF FURTHER TOPICS

More information

IEOR 6711: Stochastic Models I Fall 2013, Professor Whitt Lecture Notes, Thursday, September 5 Modes of Convergence

IEOR 6711: Stochastic Models I Fall 2013, Professor Whitt Lecture Notes, Thursday, September 5 Modes of Convergence IEOR 6711: Stochastic Models I Fall 2013, Professor Whitt Lecture Notes, Thursday, September 5 Modes of Convergence 1 Overview We started by stating the two principal laws of large numbers: the strong

More information

Data Mining and Analysis: Fundamental Concepts and Algorithms

Data Mining and Analysis: Fundamental Concepts and Algorithms Data Mining and Analysis: Fundamental Concepts and Algorithms dataminingbook.info Mohammed J. Zaki 1 Wagner Meira Jr. 2 1 Department of Computer Science Rensselaer Polytechnic Institute, Troy, NY, USA

More information

MultiRank and HAR for Ranking Multi-relational Data, Transition Probability Tensors, and Multi-Stochastic Tensors

MultiRank and HAR for Ranking Multi-relational Data, Transition Probability Tensors, and Multi-Stochastic Tensors MultiRank and HAR for Ranking Multi-relational Data, Transition Probability Tensors, and Multi-Stochastic Tensors Michael K. Ng Centre for Mathematical Imaging and Vision and Department of Mathematics

More information

DS504/CS586: Big Data Analytics Graph Mining II

DS504/CS586: Big Data Analytics Graph Mining II Welcome to DS504/CS586: Big Data Analytics Graph Mining II Prof. Yanhua Li Time: 6-8:50PM Thursday Location: AK233 Spring 2018 v Course Project I has been graded. Grading was based on v 1. Project report

More information

Web Ranking. Classification (manual, automatic) Link Analysis (today s lesson)

Web Ranking. Classification (manual, automatic) Link Analysis (today s lesson) Link Analysis Web Ranking Documents on the web are first ranked according to their relevance vrs the query Additional ranking methods are needed to cope with huge amount of information Additional ranking

More information

Updating PageRank. Amy Langville Carl Meyer

Updating PageRank. Amy Langville Carl Meyer Updating PageRank Amy Langville Carl Meyer Department of Mathematics North Carolina State University Raleigh, NC SCCM 11/17/2003 Indexing Google Must index key terms on each page Robots crawl the web software

More information

Link Mining PageRank. From Stanford C246

Link Mining PageRank. From Stanford C246 Link Mining PageRank From Stanford C246 Broad Question: How to organize the Web? First try: Human curated Web dictionaries Yahoo, DMOZ LookSmart Second try: Web Search Information Retrieval investigates

More information

1998: enter Link Analysis

1998: enter Link Analysis 1998: enter Link Analysis uses hyperlink structure to focus the relevant set combine traditional IR score with popularity score Page and Brin 1998 Kleinberg Web Information Retrieval IR before the Web

More information

Deterministic Decentralized Search in Random Graphs

Deterministic Decentralized Search in Random Graphs Deterministic Decentralized Search in Random Graphs Esteban Arcaute 1,, Ning Chen 2,, Ravi Kumar 3, David Liben-Nowell 4,, Mohammad Mahdian 3, Hamid Nazerzadeh 1,, and Ying Xu 1, 1 Stanford University.

More information

CS246: Mining Massive Datasets Jure Leskovec, Stanford University.

CS246: Mining Massive Datasets Jure Leskovec, Stanford University. CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu What is the structure of the Web? How is it organized? 2/7/2011 Jure Leskovec, Stanford C246: Mining Massive

More information

6.207/14.15: Networks Lecture 12: Generalized Random Graphs

6.207/14.15: Networks Lecture 12: Generalized Random Graphs 6.207/14.15: Networks Lecture 12: Generalized Random Graphs 1 Outline Small-world model Growing random networks Power-law degree distributions: Rich-Get-Richer effects Models: Uniform attachment model

More information

Adventures in random graphs: Models, structures and algorithms

Adventures in random graphs: Models, structures and algorithms BCAM January 2011 1 Adventures in random graphs: Models, structures and algorithms Armand M. Makowski ECE & ISR/HyNet University of Maryland at College Park armand@isr.umd.edu BCAM January 2011 2 Complex

More information

1 Complex Networks - A Brief Overview

1 Complex Networks - A Brief Overview Power-law Degree Distributions 1 Complex Networks - A Brief Overview Complex networks occur in many social, technological and scientific settings. Examples of complex networks include World Wide Web, Internet,

More information

Google Matrix, dynamical attractors and Ulam networks Dima Shepelyansky (CNRS, Toulouse)

Google Matrix, dynamical attractors and Ulam networks Dima Shepelyansky (CNRS, Toulouse) Google Matrix, dynamical attractors and Ulam networks Dima Shepelyansky (CNRS, Toulouse) wwwquantwareups-tlsefr/dima based on: OGiraud, BGeorgeot, DLS (CNRS, Toulouse) => PRE 8, 267 (29) DLS, OVZhirov

More information

Facebook Friends! and Matrix Functions

Facebook Friends! and Matrix Functions Facebook Friends! and Matrix Functions! Graduate Research Day Joint with David F. Gleich, (Purdue), supported by" NSF CAREER 1149756-CCF Kyle Kloster! Purdue University! Network Analysis Use linear algebra

More information

arxiv: v1 [math.pr] 8 May 2018

arxiv: v1 [math.pr] 8 May 2018 Analysis of Relaxation Time in Random Walk with Jumps arxiv:805.03260v [math.pr] 8 May 208 Konstantin Avrachenkov and Ilya Bogdanov 2 Inria Sophia Antipolis, France k.avrachenkov@inria.fr 2 Higher School

More information

ELE 538B: Mathematics of High-Dimensional Data. Spectral methods. Yuxin Chen Princeton University, Fall 2018

ELE 538B: Mathematics of High-Dimensional Data. Spectral methods. Yuxin Chen Princeton University, Fall 2018 ELE 538B: Mathematics of High-Dimensional Data Spectral methods Yuxin Chen Princeton University, Fall 2018 Outline A motivating application: graph clustering Distance and angles between two subspaces Eigen-space

More information

Kristina Lerman USC Information Sciences Institute

Kristina Lerman USC Information Sciences Institute Rethinking Network Structure Kristina Lerman USC Information Sciences Institute Università della Svizzera Italiana, December 16, 2011 Measuring network structure Central nodes Community structure Strength

More information

INTRODUCTION TO MCMC AND PAGERANK. Eric Vigoda Georgia Tech. Lecture for CS 6505

INTRODUCTION TO MCMC AND PAGERANK. Eric Vigoda Georgia Tech. Lecture for CS 6505 INTRODUCTION TO MCMC AND PAGERANK Eric Vigoda Georgia Tech Lecture for CS 6505 1 MARKOV CHAIN BASICS 2 ERGODICITY 3 WHAT IS THE STATIONARY DISTRIBUTION? 4 PAGERANK 5 MIXING TIME 6 PREVIEW OF FURTHER TOPICS

More information

How does Google rank webpages?

How does Google rank webpages? Linear Algebra Spring 016 How does Google rank webpages? Dept. of Internet and Multimedia Eng. Konkuk University leehw@konkuk.ac.kr 1 Background on search engines Outline HITS algorithm (Jon Kleinberg)

More information

Information Retrieval and Search. Web Linkage Mining. Miłosz Kadziński

Information Retrieval and Search. Web Linkage Mining. Miłosz Kadziński Web Linkage Analysis D24 D4 : Web Linkage Mining Miłosz Kadziński Institute of Computing Science Poznan University of Technology, Poland www.cs.put.poznan.pl/mkadzinski/wpi Web mining: Web Mining Discovery

More information

Distances in Random Graphs with Finite Variance Degrees

Distances in Random Graphs with Finite Variance Degrees Distances in Random Graphs with Finite Variance Degrees Remco van der Hofstad, 1 Gerard Hooghiemstra, 2 and Piet Van Mieghem 2 1 Department of Mathematics and Computer Science, Eindhoven University of

More information

Graph Models The PageRank Algorithm

Graph Models The PageRank Algorithm Graph Models The PageRank Algorithm Anna-Karin Tornberg Mathematical Models, Analysis and Simulation Fall semester, 2013 The PageRank Algorithm I Invented by Larry Page and Sergey Brin around 1998 and

More information

0.1 Naive formulation of PageRank

0.1 Naive formulation of PageRank PageRank is a ranking system designed to find the best pages on the web. A webpage is considered good if it is endorsed (i.e. linked to) by other good webpages. The more webpages link to it, and the more

More information

Basics and Random Graphs con0nued

Basics and Random Graphs con0nued Basics and Random Graphs con0nued Social and Technological Networks Rik Sarkar University of Edinburgh, 2017. Random graphs on jupyter notebook Solu0on to exercises 1 is out If your BSc/MSc/PhD work is

More information

Probability & Computing

Probability & Computing Probability & Computing Stochastic Process time t {X t t 2 T } state space Ω X t 2 state x 2 discrete time: T is countable T = {0,, 2,...} discrete space: Ω is finite or countably infinite X 0,X,X 2,...

More information

Detecting Sharp Drops in PageRank and a Simplified Local Partitioning Algorithm

Detecting Sharp Drops in PageRank and a Simplified Local Partitioning Algorithm Detecting Sharp Drops in PageRank and a Simplified Local Partitioning Algorithm Reid Andersen and Fan Chung University of California at San Diego, La Jolla CA 92093, USA, fan@ucsd.edu, http://www.math.ucsd.edu/~fan/,

More information

Available online at ScienceDirect. IFAC PapersOnLine 51-7 (2018) 64 69

Available online at   ScienceDirect. IFAC PapersOnLine 51-7 (2018) 64 69 Available online at www.sciencedirect.com ScienceDirect IFAC PapersOnLine 51-7 (2018) 64 69 Efficient Updating of Node Importance in Dynamic Real-Life Networks J. Berkhout B.F. Heidergott Centrum Wiskunde

More information

Models and Algorithms for Complex Networks. Link Analysis Ranking

Models and Algorithms for Complex Networks. Link Analysis Ranking Models and Algorithms for Complex Networks Link Analysis Ranking Why Link Analysis? First generation search engines view documents as flat text files could not cope with size, spamming, user needs Second

More information

1 Mechanistic and generative models of network structure

1 Mechanistic and generative models of network structure 1 Mechanistic and generative models of network structure There are many models of network structure, and these largely can be divided into two classes: mechanistic models and generative or probabilistic

More information

Lab 8: Measuring Graph Centrality - PageRank. Monday, November 5 CompSci 531, Fall 2018

Lab 8: Measuring Graph Centrality - PageRank. Monday, November 5 CompSci 531, Fall 2018 Lab 8: Measuring Graph Centrality - PageRank Monday, November 5 CompSci 531, Fall 2018 Outline Measuring Graph Centrality: Motivation Random Walks, Markov Chains, and Stationarity Distributions Google

More information

arxiv:cond-mat/ v1 3 Sep 2004

arxiv:cond-mat/ v1 3 Sep 2004 Effects of Community Structure on Search and Ranking in Information Networks Huafeng Xie 1,3, Koon-Kiu Yan 2,3, Sergei Maslov 3 1 New Media Lab, The Graduate Center, CUNY New York, NY 10016, USA 2 Department

More information

MPageRank: The Stability of Web Graph

MPageRank: The Stability of Web Graph Vietnam Journal of Mathematics 37:4(2009) 475-489 VAST 2009 MPageRank: The Stability of Web Graph Le Trung Kien 1, Le Trung Hieu 2, Tran Loc Hung 1, and Le Anh Vu 3 1 Department of Mathematics, College

More information

Algorithmic Primitives for Network Analysis: Through the Lens of the Laplacian Paradigm

Algorithmic Primitives for Network Analysis: Through the Lens of the Laplacian Paradigm Algorithmic Primitives for Network Analysis: Through the Lens of the Laplacian Paradigm Shang-Hua Teng Computer Science, Viterbi School of Engineering USC Massive Data and Massive Graphs 500 billions web

More information

Mean-field dual of cooperative reproduction

Mean-field dual of cooperative reproduction The mean-field dual of systems with cooperative reproduction joint with Tibor Mach (Prague) A. Sturm (Göttingen) Friday, July 6th, 2018 Poisson construction of Markov processes Let (X t ) t 0 be a continuous-time

More information

Random Surfing on Multipartite Graphs

Random Surfing on Multipartite Graphs Random Surfing on Multipartite Graphs Athanasios N. Nikolakopoulos, Antonia Korba and John D. Garofalakis Department of Computer Engineering and Informatics, University of Patras December 07, 2016 IEEE

More information

AXIOMS FOR CENTRALITY

AXIOMS FOR CENTRALITY AXIOMS FOR CENTRALITY From a paper by Paolo Boldi, Sebastiano Vigna Università degli Studi di Milano! Internet Math., 10(3-4):222 262, 2014. Proc. 13th ICDMW. IEEE, 2013. Proc. 4th WebScience. ACM, 2012.!

More information

Pr[positive test virus] Pr[virus] Pr[positive test] = Pr[positive test] = Pr[positive test]

Pr[positive test virus] Pr[virus] Pr[positive test] = Pr[positive test] = Pr[positive test] 146 Probability Pr[virus] = 0.00001 Pr[no virus] = 0.99999 Pr[positive test virus] = 0.99 Pr[positive test no virus] = 0.01 Pr[virus positive test] = Pr[positive test virus] Pr[virus] = 0.99 0.00001 =

More information

Jeffrey D. Ullman Stanford University

Jeffrey D. Ullman Stanford University Jeffrey D. Ullman Stanford University 2 Web pages are important if people visit them a lot. But we can t watch everybody using the Web. A good surrogate for visiting pages is to assume people follow links

More information

Randomization and Gossiping in Techno-Social Networks

Randomization and Gossiping in Techno-Social Networks Randomization and Gossiping in Techno-Social Networks Roberto Tempo CNR-IEIIT Consiglio Nazionale delle Ricerche Politecnico ditorino roberto.tempo@polito.it CPSN Social Network Layer humans Physical Layer

More information

Conditioning of the Entries in the Stationary Vector of a Google-Type Matrix. Steve Kirkland University of Regina

Conditioning of the Entries in the Stationary Vector of a Google-Type Matrix. Steve Kirkland University of Regina Conditioning of the Entries in the Stationary Vector of a Google-Type Matrix Steve Kirkland University of Regina June 5, 2006 Motivation: Google s PageRank algorithm finds the stationary vector of a stochastic

More information