Large Graph Mining: Power Tools and a Practitioner s guide

Transcription

1 Large Graph Mining: Power Tools and a Practitioner s guide Task 5: Graphs over time & tensors Faloutsos, Miller, Tsourakakis CMU KDD '9 Faloutsos, Miller, Tsourakakis P5-1

2 Outline Introduction Motivation Task 1: Node importance Task 2: Community detection Task 3: Recommendations Task 4: Connection sub-graphs Task 5: Mining graphs over time Task 6: Virus/influence propagation Task 7: Spectral graph theory Task 8: Tera/peta graph mining: hadoop Observations patterns of real graphs Conclusions KDD '9 Faloutsos, Miller, Tsourakakis P5-2

3 Tamara Kolda (Sandia) Thanks to for the foils on tensor definitions, and on TOPHITS KDD '9 Faloutsos, Miller, Tsourakakis P5-3

4 Detailed outline Motivation Definitions: PARAFAC and Tucker Case study: web mining KDD '9 Faloutsos, Miller, Tsourakakis P5-4

5 Examples of Matrices: Authors and terms John Peter Mary Nick... data mining classif. tree KDD '9 Faloutsos, Miller, Tsourakakis P5-5

6 Motivation: Why tensors? Q: what is a tensor? KDD '9 Faloutsos, Miller, Tsourakakis P5-6

7 Motivation: Why tensors? A: N-D generalization of matrix: KDD 9 John Peter Mary Nick... data mining classif. tree KDD '9 Faloutsos, Miller, Tsourakakis P5-7

8 Motivation: Why tensors? A: N-D generalization of matrix: KDD 7 John Peter Mary Nick... KDD 8 KDD 9 data mining classif. tree KDD '9 Faloutsos, Miller, Tsourakakis P5-8

9 Tensors are useful for 3 or more modes Terminology: mode (or aspect ): Mode#3 data mining classif. tree... Mode# Mode (== aspect) #1 KDD '9 Faloutsos, Miller, Tsourakakis P5-9

10 Notice 3 rd mode does not need to be time we can have more than 3 modes Dest. port IP source IP destination KDD '9 Faloutsos, Miller, Tsourakakis P5-1

11 Notice 3 rd mode does not need to be time we can have more than 3 modes Eg, ffmri: x,y,z, time, person-id, task-id From DENLAB, Temple U. (Prof. V. Megalooikonomou +) KDD '9 Faloutsos, Miller, Tsourakakis P5-11

12 Motivating Applications Why tensors are useful? web mining (TOPHITS) environmental sensors Intrusion detection (src, dst, time, dest-port) Social networks (src, dst, time, type-of-contact) face recognition etc KDD '9 Faloutsos, Miller, Tsourakakis P5-12

13 Detailed outline Motivation Definitions: PARAFAC and Tucker Case study: web mining KDD '9 Faloutsos, Miller, Tsourakakis P5-13

14 Tensor basics Multi-mode extensions of SVD recall that: KDD '9 Faloutsos, Miller, Tsourakakis P5-14

15 Reminder: SVD n n m A m Σ V T U Best rank-k approximation in L2 KDD '9 Faloutsos, Miller, Tsourakakis P5-15

16 Reminder: SVD n σ 1 u 1 v 1 σ 2 u 2 v 2 m A + Best rank-k approximation in L2 KDD '9 Faloutsos, Miller, Tsourakakis P5-16

17 Goal: extension to >=3 modes I x J x K I x R K x R C J x R ¼ B = + + A R x R x R KDD '9 Faloutsos, Miller, Tsourakakis P5-17

18 Main points: 2 major types of tensor decompositions: PARAFAC and Tucker both can be solved with ``alternating least squares (ALS) KDD '9 Faloutsos, Miller, Tsourakakis P5-18

19 Specially Structured Tensors Tucker Tensor Kruskal Tensor Our Notation Our Notation core W K x T W K x R I x J x K I x R J x S I x J x K w 1 I x R w R J x R = U R x S x T V = = u 1 v 1 U + + V R x R x R u R v R KDD '9 Faloutsos, Miller, Tsourakakis P5-19

20 Tucker Decomposition - intuition K x T C I x J x K I x R J x S ¼ A R x S x T B author x keyword x conference A: author x author-group B: keyword x keyword-group C: conf. x conf-group G: how groups relate to each other KDD '9 Faloutsos, Miller, Tsourakakis P5-2

21 Intuition behind core tensor 2-d case: co-clustering [Dhillon et al. Information-Theoretic Coclustering, KDD 3] KDD '9 Faloutsos, Miller, Tsourakakis P5-21

22 KDD '9 Faloutsos, Miller, Tsourakakis P5-22 CMU SCS [ ] = m m n n l k k l eg, terms x documents

23 med. doc cs doc term group x doc. group med. terms cs terms common terms term x term-group [ ] doc x doc group KDD '9 Faloutsos, Miller, Tsourakakis P5-23 =

24 Two main tools PARAFAC Tucker Tensor tools - summary Both find row-, column-, tube-groups but in PARAFAC the three groups are identical ( To solve: Alternating Least Squares ) KDD '9 Faloutsos, Miller, Tsourakakis P5-24

25 Detailed outline Motivation Definitions: PARAFAC and Tucker Case study: web mining KDD '9 Faloutsos, Miller, Tsourakakis P5-25

26 Web graph mining How to order the importance of web pages? Kleinberg s algorithm HITS PageRank Tensor extension on HITS (TOPHITS) KDD '9 Faloutsos, Miller, Tsourakakis P5-26

27 Kleinberg s Hubs and Authorities (the HITS method) Sparse adjacency matrix and its SVD: authority scores for 1 st topic authority scores for 2 nd topic to from hub scores for 1 st topic hub scores for 2 nd topic KDD '9 Faloutsos, Miller, Tsourakakis P5-27 Kleinberg, JACM, 1999

28 from CMU SCS authority scores for 1 st topic to HITS Authorities on Sample Data 1st Principal Factor nd Principal Factor.8 www-128.ibm.com.99 We started our crawl from.5 www2.lehigh.edu 3rd Principal Factor.6 and crawled 47 pages,.1 java.sun.com.6 resulting in 56.1 news.com.com cross-linked hosts..36 developers.sun.com.2 4th Principal Factor.24 see.sun.com.2 lewisweb.cc.lehigh.edu docs.sun.com blogs.sun.com.2 fp1.cc.lehigh.edu.31 travel.state.gov 6th Principal Factor.8 sunsolve.sun.com.22 mathpost.asu.edu math.la.asu.edu.8 news.com.com authority scores for 2 nd topic archives.math.utk.edu hub scores KDD '9 Faloutsos, Miller, Tsourakakis for 1 st hub scores topic P5-28 for 2 nd topic

29 Three-Dimensional View of the Web Observe that this tensor is very sparse! KDD '9 Faloutsos, Miller, Tsourakakis P5-29 Kolda, Bader, Kenny, ICDM5

30 Three-Dimensional View of the Web Observe that this tensor is very sparse! KDD '9 Faloutsos, Miller, Tsourakakis P5-3 Kolda, Bader, Kenny, ICDM5

31 Three-Dimensional View of the Web Observe that this tensor is very sparse! KDD '9 Faloutsos, Miller, Tsourakakis P5-31 Kolda, Bader, Kenny, ICDM5

32 Topical HITS (TOPHITS) Main Idea: Extend the idea behind the HITS model to incorporate term (i.e., topical) information. term term scores for 1 st topic term scores for 2 nd topic to from hub scores for 1 st topic authority scores for 1 st topic authority scores for 2 nd topic hub scores for 2 nd topic KDD '9 Faloutsos, Miller, Tsourakakis P5-32

33 Topical HITS (TOPHITS) Main Idea: Extend the idea behind the HITS model to incorporate term (i.e., topical) information. term term scores for 1 st topic term scores for 2 nd topic to from hub scores for 1 st topic authority scores for 1 st topic authority scores for 2 nd topic hub scores for 2 nd topic KDD '9 Faloutsos, Miller, Tsourakakis P5-33

34 TOPHITS Terms & Authorities on Sample Data 1st Principal Factor.23 JAVA.86 java.sun.com.18 SUN.38 developers.sun.com 2nd Principal Factor.17 PLATFORM.16 docs.sun.com.2 NO-READABLE-TEXT.99 TOPHITS uses 3D analysis to find.16 SOLARIS.14 see.sun.com.16 FACULTY.6.16 DEVELOPER rd www2.lehigh.edu Principal Factor the dominant groupings of web.16 SEARCH.15 EDITION.15 NO-READABLE-TEXT pages and terms..16 NEWS.15 DOWNLOAD.15 IBM.7 developer.sun.com.18 LIBRARIES.14 INFO.12 SERVICES.6 sunsolve.sun.com.7 4th www-128.ibm.com Principal Factor.16 COMPUTING.12 SOFTWARE.12 WEBSPHERE.5 access1.sun.com.26 INFORMATION LEHIGH.12 NO-READABLE-TEXT.12 WEB.5 iforce.sun.com.24 FEDERAL DEVELOPERWORKS.23 CITIZEN.1 6th Principal Factor w.11 LINUX.22 OTHER.19 travel.state.gov k = # unique links using term k.26 PRESIDENT.87 RESOURCES.19 CENTER.25 NO-READABLE-TEXT TECHNOLOGIES.19 LANGUAGES.25 BUSH.9 travel.state.gov.1 DOWNLOADS.15 U.S.25 WELCOME th Principal Factor.1 PUBLICATIONS.75 OPTIMIZATION.17 WHITE CONSUMER.58 SOFTWARE.16 U.S FREE.8 DECISION th plato.la.asu.edu Principal Factor.15 HOUSE.7 NEOS.46 ADOBE BUDGET.6 TREE.45 READER PRESIDENTS.5 GUIDE.45 ACROBAT th Principal Factor.11 OFFICE.2 SEARCH.3 FREE.5 WEATHER Tensor PARAFAC.5 ENGINE.3 NO-READABLE-TEXT.24 OFFICE.25 www-fp.mcs.anl.gov.41 CONTROL.29 HERE.23 CENTER.22 lwf.ncdc.noaa.gov.5 ILOG.29 COPY.19 NO-READABLE-TEXT th Principal Factor term scores term scores for 1 st topic for 2.5 DOWNLOAD.17 ORGANIZATION.22 TAX nd topic.15 NWS.17 TAXES.9 travel.state.gov to.15 SEVERE.15 CHILD.7 aviationweather.gov.22 FIRE.15 RETIREMENT authority scores authority scores.15 POLICY.14 BENEFITS for 1 st topic for 2 nd topic.14 CLIMATE.14 STATE.3 hub scores.14 INCOME.3 for 2 KDD '9 hub scores nd topic Faloutsos, Miller, Tsourakakis.13 SERVICE.2 for 1 P5-34 st topic.13 REVENUE.2 CREDIT.1 from term

35 Conclusions Real data are often in high dimensions with multiple aspects (modes) Tensors provide elegant theory and algorithms PARAFAC and Tucker: discover groups KDD '9 Faloutsos, Miller, Tsourakakis P5-35

36 References T. G. Kolda, B. W. Bader and J. P. Kenny. Higher-Order Web Link Analysis Using Multilinear Algebra. In: ICDM 25, Pages , November 25. Jimeng Sun, Spiros Papadimitriou, Philip Yu. Window-based Tensor Analysis on Highdimensional and Multi-aspect Streams, Proc. of the Int. Conf. on Data Mining (ICDM), Hong Kong, China, Dec 26 KDD '9 Faloutsos, Miller, Tsourakakis P5-36

37 Resources See tutorial on tensors, KDD 7 (w/ Tamara Kolda and Jimeng Sun): KDD '9 Faloutsos, Miller, Tsourakakis P5-37

38 Tensor tools - resources Toolbox: from Tamara Kolda: csmr.ca.sandia.gov/~tgkolda/tensortoolbox T. G. Kolda and B. W. Bader. Tensor Decompositions and Applications. SIAM Review, Volume 51, Number 3, September 29 csmr.ca.sandia.gov/~tgkolda/pubs/bibtgkfiles/tensorreview-preprint.pdf T. Kolda KDD ICDE 9 '9and J. Sun: Scalable Copyright: Faloutsos, Tensor Faloutsos, Miller, Tsourakakis Tong Decomposition (29) for Multi-Aspect P Data Mining (ICDM 28)