Motivation Data mining: ~ find patterns (rules, outliers) Problem#1: How do real graphs look like? Problem#2: How do they evolve? Problem#3: How to ge

Size: px

Start display at page:

Download "Motivation Data mining: ~ find patterns (rules, outliers) Problem#1: How do real graphs look like? Problem#2: How do they evolve? Problem#3: How to ge"

Britton Robbins
6 years ago
Views:

1 Graph Mining: Laws, Generators and Tools Christos Faloutsos CMU Thank you! Sharad Mehrotra UCI 2007 C. Faloutsos 2 Outline Problem definition / Motivation Static & dynamic laws; generators Tools: CenterPiece graphs; Tensors Other projects (Virus propagation, e-bay fraud detection) Conclusions UCI 2007 C. Faloutsos 3 1

Motivation Data mining: ~ find patterns (rules, outliers) Problem#1:

Problem#3: How to generate realistic graphs TOOLS Problem#4: Who is

Faloutsos 4 Problem#1: Joint work with Dr.

Faloutsos 5 Graphs - why should we care? Internet Map [lumeta.

2 Motivation Data mining: ~ find patterns (rules, outliers) Problem#1: How do real graphs look like? Problem#2: How do they evolve? Problem#3: How to generate realistic graphs TOOLS Problem#4: Who is the master-mind? Problem#5: Track communities over time UCI 2007 C. Faloutsos 4 Problem#1: Joint work with Dr. Deepayan Chakrabarti (CMU/Yahoo R.L.) UCI 2007 C. Faloutsos 5 Graphs - why should we care? Internet Map [lumeta.com] Food Web [Martinez 91] Friendship Network [Moody 01] Protein Interactions [genomebiology.com] UCI 2007 C. Faloutsos 6 2

3 Graphs - why should we care? IR: bi-partite graphs (doc-terms) D T 1 web: hyper-text graph D N T M... and more: UCI 2007 C. Faloutsos 7 Graphs - why should we care? network of companies & board-of-directors members viral marketing web-log ( blog ) news propagation computer network security: /ip traffic and anomaly detection... UCI 2007 C. Faloutsos 8 Problem #1 - network and graph mining How does the Internet look like? How does the web look like? What is normal / abnormal? which patterns/laws hold? UCI 2007 C. Faloutsos 9 3

4 Graph mining Are real graphs random? UCI 2007 C. Faloutsos 10 Laws and patterns Are real graphs random? A: NO!! Diameter in- and out- degree distributions other (surprising) patterns UCI 2007 C. Faloutsos 11 Solution#1 Power law in the degree distribution [SIGCOMM99] internet domains att.com log(degree) ibm.com log(rank) UCI 2007 C. Faloutsos 12 4

5 Solution#1 : Eigen Exponent E Eigenvalue Exponent = slope E = May 2001 Rank of decreasing eigenvalue A2: power law in the eigenvalues of the adjacency matrix UCI 2007 C. Faloutsos 13 But: How about graphs from other domains? UCI 2007 C. Faloutsos 14 The Peer-to-Peer Topology [Jovanovic+] Frequency versus degree Number of adjacent peers follows a power-law UCI 2007 C. Faloutsos 15 5

6 More power laws: citation counts: (citeseer.nj.nec.com 6/2001) 100 cited.pdf log(count) log count 10 Ullman log # citations log(#citations) UCI 2007 C. Faloutsos 16 More power laws: web hit counts [w/ A. Montgomery] log(count) Web Site Traffic Zipf ``ebay users sites log(in-degree) UCI 2007 C. Faloutsos 17 count epinions.com who-trusts-whom [Richardson + Domingos, KDD 2001] trusts-2000-people user (out) degree UCI 2007 C. Faloutsos 18 6

Outline Problem definition / Motivation Static & dynamic laws; generators Tools: CenterPiece graphs; Tensors Other projects (Virus propagation, e-bay

Faloutsos 19 Motivation Data mining: ~ find patterns (rules, outliers) Problem#1: How do real graphs look like? Problem#2: How do they evolve?

7 Outline Problem definition / Motivation Static & dynamic laws; generators Tools: CenterPiece graphs; Tensors Other projects (Virus propagation, e-bay fraud detection) Conclusions UCI 2007 C. Faloutsos 19 Motivation Data mining: ~ find patterns (rules, outliers) Problem#1: How do real graphs look like? Problem#2: How do they evolve? Problem#3: How to generate realistic graphs TOOLS Problem#4: Who is the master-mind? Problem#5: Track communities over time UCI 2007 C. Faloutsos 20 Problem#2: Time evolution with Jure Leskovec (CMU/MLD) and Jon Kleinberg (Cornell CMU) UCI 2007 C. Faloutsos 21 7

8 Evolution of the Diameter Prior work on Power Law graphs hints at slowly growing diameter: diameter ~ O(log N) diameter ~ O(log log N) What is happening in real data? UCI 2007 C. Faloutsos 22 Evolution of the Diameter Prior work on Power Law graphs hints at slowly growing diameter: diameter ~ O(log N) diameter ~ O(log log N) What is happening in real data? Diameter shrinks over time UCI 2007 C. Faloutsos 23 Diameter ArXiv citation graph Citations among physics papers One graph per year diameter time [years] UCI 2007 C. Faloutsos 24 8

9 Diameter Autonomous Systems Graph of Internet One graph per day diameter number of nodes UCI 2007 C. Faloutsos 25 Diameter Affiliation Network Graph of collaborations in physics authors linked to papers 10 years of data diameter time [years] UCI 2007 C. Faloutsos 26 Diameter Patents Patent citation network 25 years of data diameter time [years] UCI 2007 C. Faloutsos 27 9

10 Temporal Evolution of the Graphs N(t) nodes at time t E(t) edges at time t Suppose that N(t+1) = 2 * N(t) Q: what is your guess for E(t+1) =? 2 * E(t) UCI 2007 C. Faloutsos 28 Temporal Evolution of the Graphs N(t) nodes at time t E(t) edges at time t Suppose that N(t+1) = 2 * N(t) Q: what is your guess for E(t+1) =? 2 * E(t) A: over-doubled! But obeying the ``Densification Power Law UCI 2007 C. Faloutsos 29 Densification Physics Citations Citations among physics papers 2003: 29,555 papers, 352,807 citations E(t)?? UCI 2007 C. Faloutsos 30 N(t) 10

11 Densification Physics Citations Citations among physics papers 2003: 29,555 papers, 352,807 citations E(t) 1.69 UCI 2007 C. Faloutsos 31 N(t) Densification Physics Citations Citations among physics papers 2003: 29,555 papers, 352,807 citations E(t) : tree UCI 2007 C. Faloutsos 32 N(t) Densification Physics Citations Citations among physics papers 2003: 29,555 papers, 352,807 citations E(t) clique: UCI 2007 C. Faloutsos 33 N(t) 11

12 Densification Patent Citations Citations among patents granted million nodes 16.5 million edges Each year is a datapoint E(t) 1.66 N(t) UCI 2007 C. Faloutsos 34 Densification Autonomous Systems Graph of Internet ,000 nodes 26,000 edges One graph per day E(t) 1.18 N(t) UCI 2007 C. Faloutsos 35 Densification Affiliation Network Authors linked to their publications ,000 nodes 20,000 authors 38,000 papers 133,000 edges E(t) 1.15 N(t) UCI 2007 C. Faloutsos 36 12

13 Outline Problem definition / Motivation Static & dynamic laws; generators Tools: CenterPiece graphs; Tensors Other projects (Virus propagation, e-bay fraud detection) Conclusions UCI 2007 C. Faloutsos 37 Motivation Data mining: ~ find patterns (rules, outliers) Problem#1: How do real graphs look like? Problem#2: How do they evolve? Problem#3: How to generate realistic graphs TOOLS Problem#4: Who is the master-mind? Problem#5: Track communities over time UCI 2007 C. Faloutsos 38 Problem#3: Generation Given a growing graph with count of nodes N 1, N 2, Generate a realistic sequence of graphs that will obey all the patterns UCI 2007 C. Faloutsos 39 13

14 Problem Definition Given a growing graph with count of nodes N 1, N 2, Generate a realistic sequence of graphs that will obey all the patterns Static Patterns Power Law Degree Distribution Power Law eigenvalue and eigenvector distribution Small Diameter Dynamic Patterns Growth Power Law Shrinking/Stabilizing Diameters UCI 2007 C. Faloutsos 40 Problem Definition Given a growing graph with count of nodes N 1, N 2, Generate a realistic sequence of graphs that will obey all the patterns Idea: Self-similarity Leads to power laws Communities within communities UCI 2007 C. Faloutsos 41 Kronecker Product a Graph Intermediate stage Adjacency matrix Adjacency matrix UCI 2007 C. Faloutsos 42 14

15 Kronecker Product a Graph Continuing multiplying with G 1 we obtain G 4 and so on G 4 adjacency matrix UCI 2007 C. Faloutsos 43 Kronecker Product a Graph Continuing multiplying with G 1 we obtain G 4 and so on G 4 adjacency matrix UCI 2007 C. Faloutsos 44 Kronecker Product a Graph Continuing multiplying with G 1 we obtain G 4 and so on G 4 adjacency matrix UCI 2007 C. Faloutsos 45 15

16 Properties: We can PROVE that Degree distribution is multinomial ~ power law Diameter: constant Eigenvalue distribution: multinomial First eigenvector: multinomial See [Leskovec+, PKDD 05] for proofs UCI 2007 C. Faloutsos 46 Problem Definition Given a growing graph with nodes N 1, N 2, Generate a realistic sequence of graphs that will obey all the patterns Static Patterns Power Law Degree Distribution Power Law eigenvalue and eigenvector distribution Small Diameter Dynamic Patterns Growth Power Law Shrinking/Stabilizing Diameters First and only generator for which we can prove all these properties UCI 2007 C. Faloutsos 47 Stochastic Kronecker Graphs skip Create N 1 N 1 probability matrix P 1 Compute the k th Kronecker power P k For each entry p uv of P k include an edge (u,v) with probability p uv P Kronecker multiplication P k Instance Matrix G 2 flip biased coins UCI 2007 C. Faloutsos 48 16

17 Experiments How well can we match real graphs? Arxiv: physics citations: 30,000 papers, 350,000 citations 10 years of data U.S. Patent citation network 4 million patents, 16 million citations 37 years of data Autonomous systems graph of internet Single snapshot from January ,400 nodes, 26,000 edges We show both static and temporal patterns UCI 2007 C. Faloutsos 49 Arxiv Degree Distribution Real graph Deterministic Kronecker Stochastic Kronecker count degree degree degree UCI 2007 C. Faloutsos 50 Arxiv Scree Plot Real graph Deterministic Kronecker Stochastic Kronecker Eigenvalue Rank Rank Rank UCI 2007 C. Faloutsos 51 17

18 Arxiv Densification Real graph Deterministic Kronecker Stochastic Kronecker Edges Nodes(t) Nodes(t) Nodes(t) UCI 2007 C. Faloutsos 52 Arxiv Effective Diameter Real graph Deterministic Kronecker Stochastic Kronecker Diameter Nodes(t) Nodes(t) Nodes(t) UCI 2007 C. Faloutsos 53 (Q: how to fit the parm s?) A: Stochastic version of Kronecker graphs + Max likelihood + Metropolis sampling [Leskovec+, ICML 07] UCI 2007 C. Faloutsos 54 18

19 Experiments on real AS graph Degree distribution Hop plot Adjacency matrix eigen values Network value UCI 2007 C. Faloutsos 55 Conclusions Kronecker graphs have: All the static properties Heavy tailed degree distributions Small diameter All the temporal properties Densification Power Law Shrinking/Stabilizing Diameters We can formally prove these results Multinomial eigenvalues and eigenvectors UCI 2007 C. Faloutsos 56 Outline Problem definition / Motivation Static & dynamic laws; generators Tools: CenterPiece graphs; Tensors Other projects (Virus propagation, e-bay fraud detection) Conclusions UCI 2007 C. Faloutsos 57 19

20 Motivation Data mining: ~ find patterns (rules, outliers) Problem#1: How do real graphs look like? Problem#2: How do they evolve? Problem#3: How to generate realistic graphs TOOLS Problem#4: Who is the master-mind? Problem#5: Track communities over time UCI 2007 C. Faloutsos 58 Problem#4: MasterMind CePS w/ Hanghang Tong, KDD 2006 htong <at> cs.cmu.edu UCI 2007 C. Faloutsos 59 Center-Piece Subgraph(Ceps) Given Q query nodes Find Center-piece ( b ) B App. Social Networks Law Inforcement, A C Idea: Proximity -> random walk with restarts UCI 2007 C. Faloutsos 60 20

Jordan 1 4 Corinna 6 Daryl Cortes Pregibon UCI 2007 C. Faloutsos 62 13 Case Study: AND query 15 H.V. Jagadish 10 Laks V.S. Lakshmanan R.

21 Case Study: AND query R. Agrawal Jiawei Han V. Vapnik M. Jordan UCI 2007 C. Faloutsos 61 Case Study: AND query 15 H.V. Jagadish 10 Laks V.S. Lakshmanan R. Agrawal Jiawei Han 10 Heikki 1 1 Mannila Christos 1 Padhraic 1 Faloutsos Smyth 1 V. Vapnik 3 M. Jordan 1 4 Corinna 6 Daryl Cortes Pregibon UCI 2007 C. Faloutsos Case Study: AND query 15 H.V. Jagadish 10 Laks V.S. Lakshmanan R. Agrawal Jiawei Han 10 Heikki 1 1 Mannila Christos Padhraic 1 1 Faloutsos Smyth 1 V. Vapnik 3 M. Jordan 1 4 Corinna 6 Daryl Cortes Pregibon UCI 2007 C. Faloutsos

22 A B C 15 H.V. Jagadish 10 Laks V.S. Lakshmanan databases 13 R. Agrawal Jiawei Han Umeshwar 3 3 Dayal 5 Bernhard Scholkopf 2 Peter L. Bartlett ML/Statistics 2 V. Vapnik M. Jordan _SoftAnd query Alex J. Smola UCI 2007 C. Faloutsos 64 Conclusions Q1:How to measure the importance? A1: RWR+K_SoftAnd Q2: How to find connection subgraph? A2: Extract Alg. Q3:How to do it efficiently? A3:Graph Partition (Fast CePS) ~90% quality 6:1 speedup; 150x speedup (ICDM 06, b.p. award) UCI 2007 C. Faloutsos 65 Outline Problem definition / Motivation Static & dynamic laws; generators Tools: CenterPiece graphs; Tensors Other projects (Virus propagation, e-bay fraud detection) Conclusions UCI 2007 C. Faloutsos 66 22

23 Motivation Data mining: ~ find patterns (rules, outliers) Problem#1: How do real graphs look like? Problem#2: How do they evolve? Problem#3: How to generate realistic graphs TOOLS Problem#4: Who is the master-mind? Problem#5: Track communities over time UCI 2007 C. Faloutsos 67 Tensors for time evolving graphs [Jimeng Sun+ KDD 06] [, SDM 07] [ CF, Kolda, Sun, SDM 07 tutorial] UCI 2007 C. Faloutsos 68 Social network analysis Static: find community structures 1990 Keywords Authors DB UCI 2007 C. Faloutsos 69 23

24 Social network analysis Static: find community structures Dynamic: monitor community structure evolution; spot abnormal individuals; abnormal time-stamps 2004 DM Keywords 1990 DB Authors DB UCI 2007 C. Faloutsos authors Application 1: Multiway latent semantic indexing (LSI) 2004 DB DM keyword DB Pattern U keyword Query U authors Philip Yu Projection matrices specify the clusters Core tensors give cluster activation level Michael Stonebraker UCI 2007 C. Faloutsos 71 Bibliographic data (DBLP) Papers from VLDB and KDD conferences Construct 2nd order tensors with yearly windows with <author, keywords> Each tensor: timestamps (years) UCI 2007 C. Faloutsos 72 24

25 Authors michael carey, michael stonebraker, h. jagadish, hector garcia-molina surajit chaudhuri,mitch cherniack,michael stonebraker,ugur etintemel Multiway LSI DB jiawei han,jian pei,philip s. yu, jianyong wang,charu c. aggarwal DM Keywords queri,parallel,optimization,concurr, objectorient distribut,systems,view,storage,servic,pr ocess,cache streams,pattern,support, cluster, index,gener,queri Year Two groups are correctly identified: Databases and Data mining People and concepts are drifting over time UCI 2007 C. Faloutsos 73 Conclusions Tensor-based methods (WTA/DTA/STA): spot patterns and anomalies on time evolving graphs, and on streams (monitoring) UCI 2007 C. Faloutsos 74 Outline Problem definition / Motivation Static & dynamic laws; generators Tools: CenterPiece graphs; Tensors Other projects (Virus propagation, e-bay fraud detection) Conclusions UCI 2007 C. Faloutsos 75 25

26 Virus propagation How do viruses/rumors propagate? Will a flu-like virus linger, or will it become extinct soon? UCI 2007 C. Faloutsos 76 The model: SIS Flu like: Susceptible-Infected-Susceptible Virus strength s= β/δ N1 Infected Prob. β Prob. δ N Prob. β N2 N3 Healthy UCI 2007 C. Faloutsos 77 Epidemic threshold τ of a graph: the value of τ, such that if strength s = β / δ < τ an epidemic can not happen Thus, given a graph compute its epidemic threshold UCI 2007 C. Faloutsos 78 26

27 Epidemic threshold τ What should τ depend on? avg. degree? and/or highest degree? and/or variance of degree? and/or third moment of degree? and/or diameter? UCI 2007 C. Faloutsos 79 Epidemic threshold [Theorem] We have no epidemic, if β/δ <τ = 1/ λ 1,A UCI 2007 C. Faloutsos 80 Epidemic threshold [Theorem] We have no epidemic, if epidemic threshold recovery prob. β/δ <τ = 1/ λ 1,A attack prob. Proof: [Wang+03] largest eigenvalue of adj. matrix A UCI 2007 C. Faloutsos 81 27

Experiments (Oregon) Number of Infected Nodes 500 400 300 200 100 Oregon β = 0.001 β/δ > τ (above threshold) β/δ = τ (at the threshold) 0 0 250 500 750 1000 Time δ: 0.05 0.06 0.

28 Experiments (Oregon) Number of Infected Nodes Oregon β = β/δ > τ (above threshold) β/δ = τ (at the threshold) Time δ: β/δ < τ (below threshold) UCI 2007 C. Faloutsos 82 Outline Problem definition / Motivation Static & dynamic laws; generators Tools: CenterPiece graphs; Tensors Other projects (Virus propagation, e-bay fraud detection) Conclusions UCI 2007 C. Faloutsos 83 E-bay Fraud detection w/ Polo Chau & Shashank Pandit, CMU UCI 2007 C. Faloutsos 84 28

E-bay Fraud detection - NetProbe UCI 2007 C. Faloutsos 85 OVERALL CONCLUSIONS Graphs pose a wealth of fascinating problems self-similarity and power laws work, when textbook methods fail!

29 E-bay Fraud detection - NetProbe UCI 2007 C. Faloutsos 85 OVERALL CONCLUSIONS Graphs pose a wealth of fascinating problems self-similarity and power laws work, when textbook methods fail! New patterns (shrinking diameter!) New generator: Kronecker UCI 2007 C. Faloutsos 86 Philosophical observation Graph mining brings together: ML/AI / IR Stat, Num. analysis, Systems (DB (Gb/Tb), Networks ) sociology, epidemiology physics, ++ UCI 2007 C. Faloutsos 87 29

30 References Hanghang Tong, Christos Faloutsos, and Jia-Yu Pan Fast Random Walk with Restart and Its Applications ICDM 2006, Hong Kong. Hanghang Tong, Christos Faloutsos Center-Piece Subgraphs: Problem Definition and Fast Solutions, KDD 2006, Philadelphia, PA UCI 2007 C. Faloutsos 88 References Jure Leskovec, Jon Kleinberg and Christos Faloutsos Graphs over Time: Densification Laws, Shrinking Diameters and Possible Explanations KDD 2005, Chicago, IL. ("Best Research Paper" award). Jure Leskovec, Deepayan Chakrabarti, Jon Kleinberg, Christos Faloutsos Realistic, Mathematically Tractable Graph Generation and Evolution, Using Kronecker Multiplication (ECML/PKDD 2005), Porto, Portugal, UCI 2007 C. Faloutsos 89 References Jure Leskovec and Christos Faloutsos, Scalable Modeling of Real Graphs using Kronecker Multiplication, ICML 2007, Corvallis, OR, USA Jimeng Sun, Dacheng Tao, Christos Faloutsos Beyond Streams and Graphs: Dynamic Tensor Analysis, KDD 2006, Philadelphia, PA UCI 2007 C. Faloutsos 90 30

Minneapolis, Minnesota, Apr 2007. [pdf] UCI 2007 C.

31 References Jimeng Sun, Yinglian Xie, Hui Zhang, Christos Faloutsos. Less is More: Compact Matrix Decomposition for Large Sparse Graphs, SDM, Minneapolis, Minnesota, Apr [pdf] UCI 2007 C. Faloutsos 91 Contact info: www. cs.cmu.edu /~christos (w/ papers, datasets, code, etc) UCI 2007 C. Faloutsos 92 31

CMU SCS. Large Graph Mining. Christos Faloutsos CMU

CMU SCS. Large Graph Mining. Christos Faloutsos CMU Large Graph Mining Christos Faloutsos CMU Thank you! Hillol Kargupta NGDM 2007 C. Faloutsos 2 Outline Problem definition / Motivation Static & dynamic laws; generators Tools: CenterPiece graphs; fraud