Bi-Relational Network Analysis Using a Fast Random Walk with Restart

Transcription

1 B-Relatonal Network Analyss Usng a Fast Random Walk wth Restart Jng Xa, Dona Caragea and Wllam Hsu Departmet of Computng and Informaton Scences Kansas State Unversty, Manhattan, USA {xajng, dcaragea, bhsu}@ksuedu Abstract Identfcaton of nodes relevant to a gven node n a relatonal network s a basc problem n network analyss wth great practcal mportance Most exstng network analyss algorthms utlze one sngle relaton to defne relevancy among nodes However, n real world applcatons multple relatonshps exst between nodes n a network Therefore, network analyss algorthms that can make use of more than one relaton to dentfy the relevance set for a node are needed In ths paper, we show how the Random Walk wth Restart (RWR) approach can be used to study relevancy n a b-relatonal network from the bblographc doman, and show that makng use of two relatons results n better results as compared to approaches that use a sngle relaton As relatonal networks can be very large, we also propose a fast mplementaton for RWR by adaptng an exstng Iteratve Aggregaton and Dsaggregaton (IAD) approach The IAD-based RWR explots the block-wse structure of real world networks Expermental results show sgnfcant ncrease n runnng tme for the IAD-based RWR compared to the tradtonal power method based RWR Keywords-Relatonal data mnng; node relevancy; random walk; teratve aggregaton and dsaggregaton approach I INTRODUCTION Identfcaton of nodes relevant to a gven node n a relatonal network s of sgnfcant practcal mportance Node relevancy nformaton enables the study of complex propertes of a network As an example, n a bblographc network, nformaton about researchers relevant to a gven researcher can be used to predct potental co-author relatonshps or communtes that a researcher should jon Among several approaches to the problem of dentfyng a relevancy set for a gven node n a network, random walk based algorthms have proven very effectve [1] Tradtonally, random walk algorthms explot one type of relaton (eg, co-author relatonshps) when fndng relevance scores for a node (n our example, an author) However, n real world applcatons, for a partcular doman there always exst several types of objects (eg, papers, authors, venues) and relatons among objects of nterest (eg, co-author relatonshps, ctaton relatonshps, authorpaper relatonshps) Whle each relaton can be exploted by tself for solvng a partcular network analyss task, more nsghts nto the propertes of the network can be ganed f multple relatons are used together In ths work, we focus on b-relatonal network analyss usng a Random Walk wth Restart (RWR) approach and show ts advantages as compared to sngle relatonal network analyss Gven the large scale of network data avalable nowadays, fast mplementatons of the RWR algorthms are needed even n the case of sngle relatonal network analyss [2] For the analyss of networks wth two or more relatons, tme and memory effcent algorthms are mperatve To address ths challenge, we propose a fast mplementaton of the RWR algorthm for b-relatonal networks Ths mplementaton takes advantage of a nce property that real world networks present, specfcally ther block-wse structure Based on ths property, an teratve aggregaton and decomposton (IAD) algorthm s adapted to RWR The rest of the paper s organzed as follows We defne relevance scores and ntroduce sngle and b-relatonal networks n Secton II We ntroduce the RWR network analyss approach n Secton III Our adaptaton of the RWR approach to b-relatonal networks s presented n Secton IV, whle the IAD-based RWR and a dscusson on the effcency of the method are presented n Secton V Secton VI descrbes the expermental evaluaton of the proposed approach We conclude wth a summary and dscusson of the related work n Secton VII II BACKGROUND AND MOTIVATION A Relevance of Nodes Generally, a relatonal network can be represented as a graph G =< V, E >, where V s the set of nodes and E s the set of edges representng relatonshps between nodes n the network Smlar to prevous work [1], the man queston we address n ths paper can be stated as follows: gven a node a V, whch nodes n V are most relevant to a? To answer ths queston, for each node b V, we use RWR to compute a relevance score to a All scores together form a relevance score vector wth respect to the node a We expect nodes that are hghly relevant to a to have hgher scores than nodes that are not relevant to a Thus, the score vector can help dentfy nodes relevant to a and also quantfy relevancy B B-Relatonal Networks We wll use a smple academc communty example to motvate and descrbe b-relatonal networks In a typcal academc communty, gven a researcher a, one may be nterested n fndng the most relevant researchers b for a Here, we assume that b s relevant to a f a and b share smlar research nterests The set of nodes for ths example

2 author has paper relatonshp smlarty relatonshp authors papers Fgure 1 B-relatonal network: authors are assocated wth papers through author-has-paper relatonshps; papers are lnked through smlarty relatonshps conssts of researchers and papers In prncple, we can compute the relevancy score vector for researcher a based on co-author relatonshps (sngle-relatonal network) n a bblographc data set However, we can also buld a brelatonal network n whch papers can be lnked to each other based on content smlarty relatonshps (e, two papers are smlar f they share smlar words); authors can be lnked to papers through author-has-paper relatonshps Fgure 1 shows a vew of the b-relatonal network nduced by these two relatonshps Intutvely, authors who have smlar papers share smlar nterests Thus, usng these relatonshps together to fnd authors most relevant to a gven author should result n a stronger relevancy compared wth the relevancy obtaned usng each relaton ndependently We wll use the followng defntons for sngle and brelatonal networks: A sngle-relatonal network s a network nduced by a sngle relaton among nodes If G =< V, E > s the graph correspondng to the network, then E s the set of edges defned by the sngle relaton among nodes V A b-relatonal network s a network nduced by two types of relatons among nodes Formally, a brelatonal network s gven by G =< V 1 V 2, E 1 E 2 >, where E 1 V 1 V 2 and E 2 V 2 V 2 III RANDOM WALK WITH RESTART The notatons used n ths paper are shown n Table I The RWR method defnes a transton matrx P n n (where n s the number of nodes) Ths matrx models the probablty of transton between every two nodes n the network If P s row normalzed (e, the sum of elements n a row s 1), then P s rreducble and aperodc Therefore, accordng to the Perron-Frobenus theorem, there s a unque statonary dstrbuton of the matrx P Gven the transton matrx P, a RWR can be seen as a non-homogeneous Markov chan A RWR s defned by the followng formula: π (t+1) = (1 c)π (t) P + c e k (1) Table I SYMBOLS AND DEFINITIONS Symbol Defnton π 1 n statonary dstrbuton vector by runnng RWR from startng node k π (t) dstrbuton vector after t teratons runs c the restart probablty, 0 < c < 1 e k 1 n startng vector, the k th element s 1 and all the other elements are 0 c k 1 n startng vector c k = ce k n the number of nodes n the graph N the number of parttons P the orgnal transton matrx M the transformed transton matrx, M = (1 c)p A N N couplng matrx of M m c k π c 1 N the sze of each sub matrx M 1 m k sub-startng vector for the sub-matrx M kk where one element correspondng to the startng node k s 1 and 0 for others sub-egenvector of sub-matrx M 1 N vector where one element correspondng to M kk s 1 and the others are 0 where π s the probablty dstrbuton of a partcle startng at node k, c (0, 1) s the restartng probablty, and e k s the ntal vector As can be seen n the equaton, at each teraton, a constant c e k nterpreted as the restart s added Eq (1) converges as the number of teratons approaches nfnty [3] Therefore, we have: π = (1 c)π P + c e k = π M + c k (2) The statonary dstrbuton π, whch represents the probablty dstrbuton of reachng any node a from k, can be seen as the relevance score vector correspondng to k IV RWR FOR SINGLE AND BI-RELATIONAL NETWORKS In what follows, we explan how we apply RWR to sngle and b-relatonal networks, respectvely A Sngle-Relatonal Networks Obvously, we can drectly apply RWR for sngle relatonal networks To do that, P s constructed from the network G =< V, E > (whch s a weghted graph); e a represents the vector startng wth node a The statonary dstrbuton π (a) can be used as the relevance score vector correspondng to the node a B B-Relatonal Networks Remember that a b-relatonal network s defned by G =< V 1 V 2, E 1 E 2 >, where E 1 V 1 V 2 and E 2 V 2 V 2 Our goal s to use RWR to dentfy nodes b n V 1 relevant to a node a n V 1 by makng use of both relatonshps n E 1 and E 2 To acheve that, we construct the transton matrx P from < V 2, E 2 > Then, for the gven node a V 1 and every edge (a, p) E 1 (consequently for every node p V 2 that s lnked to a), we run RWR wth

3 transton probablty P and startng vector e p The resultng statonary dstrbuton represents the relevance score vector correspondng to the startng node p Based on the relevance score vector, we choose a set of nodes V 2 V 2 that are most relevant to node p Fnally, the nodes relevant to a are defned as those nodes b n V 1 for whch there exsts an edge (b, q) (where q V 2) For example, let us assume that an author a has four papers These papers are part of a paper smlarty network To dentfy authors related to a, we run RWR wth a transton matrx gven by the paper smlarty network and startng vectors correspondng to each of the author s papers, n turn Thus, we wll obtan a set of four relevance score vectors From each vector, we choose the most related papers and nfer that ther authors are most relevant for the gven author V SCALING UP RWR The straghtforward mplementaton of RWR requres many teratons over the transton matrx or, even worse, calculatng the nverse of the matrx [2] As mult-relatonal networks are usually large, usng ths mplementaton s mpractcal for most real world applcatons To address ths lmtaton, we propose an approach for scalng up the RWR method The theory behnd our fast RWR approach and the algorthm used n our experments are descrbed n what follows A RWR Property In ths subsecton, we wll show a nce property of π (the statonary dstrbuton of the RWR startng at k), assumng that the matrx M n (2) has the followng dagonal block-structure: M M 22 0 M = (3) 0 0 M N,N where each block sub-matrx M s of sze m, for = 1, 2,, N and M kk contans the startng node k Then, by replacng M wth (3) n (2), we get: (π 1,, π k,, π N ) = (π 1,, π k,, π N ) M M kk 0 + c ḳ 0 0 M NN 0 where π k s the sub-egenvector for the block sub-matrx M kk and c k s a (1 m k) vector correspondng to the submatrx M kk (the element correspondng to the startng node k s 1 and all the other elements are 0) As a consequence, each π can be obtaned from T π M = π, k and π k M kk + c k = π k Note that ρ(m ) < 1 (ρ s the spectral radus of a matrx, e the max egenvalue of a matrx), therefore π = 0, for = 1, 2,, N, k Thus, we only need to solve the equaton π k M kk + c k = π k Ths property explans the observaton made n [1], where the authors notced that most elements n the dstrbuton are close to zero and therefore proposed to perform RWR on the parttoned local block only In most real network applcatons, the network naturally forms a block-wse structure, although not necessarly a perfect dagonal block-structure lke the one above For nstance, n the academc communty network example, the author network has a block-wse communty structure wth respect to authors nterests and publcatons Smlarly, the papers network has a block-wse structure wth respect to papers topcs and smlarty We wll explot ths type of structure to scale up the RWR approach To do that, we frst construct a block-wse partton for M that looks lke: M 11 M 12 M 1N M 21 M 22 M 2N M = (4) M N1 M N2 M NN where M represents the lnks wthn a communty and M j, j represents the lnks between communtes and j To construct such a partton for M, we use CLUTO [4] algorthm, whch maxmzes the edge weght wthn the communty, whle mnmzng the weght between communtes Once a partton of M s constructed, we can compute the left egenvector for each dagonal sub-matrx M : u M = λ u (for k) and u k M kk + c k = u k (5) where λ (1 c) We wll use the egenvectors u of M as an approxmaton to π (the sub-vector n π correspondng to M ) and further combne the u local egenvalue vectors nto one global egenvector for the whole matrx by adaptng the Iteratve Aggregaton/Dsaggregaton (IAD) [3], [5], [6] method Ths wll allow to quckly fnd the steady dstrbuton π B Fast RWR Usng the IAD Method The combnaton of the local egenvectors correspondng to matrces M nto a global egenvector for M needs to take nto account the weght of each sub-block matrx The frst part of the IAD algorthm s used to derve ths weght vector by constructng an aggregated matrx A from M, n two steps Assumng that π s known for = 1, 2,, N, the two steps are as follows: 1) replace each row of each sub-block matrx M j wth the sum of the elements n that row; ths results n a matrx n N (one column for each M j ); 2) multply each of the resultng columns by a weght vector φ, where φ = π / π 1, for

4 = 1, 2,, N; ths results n an aggregated matrx A N N (7) back nto Eq (6) We wll construct a matrx W, = (one element for each sub-block matrx M j ) 1, 2,, N such that The elements of the matrx, A N N can be wrtten as: ( ) M s a j = φ M j e j, where φ s a row vector wth m elements W = r and e j s the 1 column vector wth m j elements Havng T (8) q constructed the aggregated matrx A, the goal s to fnd a where r T s a 1 m vector defned as: weght vector for each sub-block matrx M j by solvng the 1 equaton ξ = ξa + c 1 N Indeed, we can show that A has 1 ξ ξ j φ j M j f k such a statonary dstrbuton Ths follows from: r T j = 1 ( π 1 1, π 2 1,, π N 1 )A = ( π 1 1, π 2 1,, π N 1 ) 1 ξ k (c k + (9) ξ j φ j M jk ) f = k j k π 1 π π 0 2 π s s an m 1 vector defned as: s = e M e for = Me 1, 2,, N and q s a scalar defned as: q = 1 n N + c 1 N r T e for = 1, 2,, N Therefore, we obtan: π 0 0 N ( ) π N 1 M s (π, 1 ξ ) e 0 0 r T = (π q, 1 ξ ) (10) 0 e 0 =π Me n N + c 1 N = (π c k ) + c Wth the constructed block W, we can obtan a new π and 1 Nξ through solvng the Eq (10) Fnally we update φ = 0 0 e π /ξ wth the new values for π and ξ obtaned from W =( π 1 1, π 2 1,, π k c The steps for scalng up RWR are shown n Algorthm 1 k 1,, π N 1 ) + c 1 N =( π 1 1, π 2 1,, π k 1,, π N 1 ) Algorthm 1 IAD-based RWR We used the fact that π M = (π c k ) (Eq 1) and π k c k 1 + c = π k 1 (whch can be easly proved usng the defnton of the norm 1) If ξ = ( π 1 1, π 2 1,, π k 1,, π N 1 ) s the statonary dstrbuton of A, we consder ths to be the weght vector for the sub-block matrx M Note that φ ( = 1, 2,, N) depends on s π, whch s not known n advance; therefore, we wll use u as an approxmaton for π For practcal problems, ths approxmaton should not result n a sgnfcant error as the structure of M s presumably close to the structure n Eq (3) and M 1 s maxmzed when creatng the block-wse partton of M Therefore, an approxmaton s made such that φ = u / u 1 φ = π / π 1 (6) We use Eq (6) to compute an approxmaton A to the aggregated matrx A Each element of A s gven by a j = φ M je j Next, we determne an approxmaton egenvector ξ from ξ A + c 1 N = ξ and use t to derve the statonary dstrbuton of M: π = ( ) ξ1φ 1, ξ2φ 2,, ξn φ N (7) The second part of the IAD s used to mprove the approxmaton n Eq (7) The smplest way to do ths s to ncorporate Eq (7) back nto Eq (6) and reterate wth the goal of obtanng a better soluton However, drectly usng Eq (7) wll have no effect on the approxmaton [3] Therefore, smlar to [3], we adapt Takahash s approach [7] to mprove the approxmaton before ncorporatng Eq Input: a normalzed matrx P, the startng vector e k and the error threshold ɛ Output: the statonary dstrbuton π 1 Construct the transformed matrx M from P 2 Partton M nto N parttons usng CLUTO [4] 3 Let π (0) = (π (0) 1, π(0) 2,, π(0) N ) be a gven ntal approxmaton to the soluton and set m = 1 4 For = 1, 2,, N, compute φ (m 1) as: φ (m 1) = π (m 1) / π (m 1) 1 5 Construct aggregated matrx A (m 1) whose elements are (A (m 1) ) j = φ (m 1) M j e j 6 Solve the egenvector problem ξ (m 1) A (m 1) + c 1 N = ξ (m 1) 7 For = 1, 2,, N, construct W and derve π (m) ξ (m) by solvng Eq (10); update φ (m) = π (m) /ξ (m) and 8 Convergence test: f the the dfference between two consecutve estmates π (m) π (m 1) 2 < ɛ, then stop π (m) = (ξ (m 1) 1 φ (m) 1, ξ (m 1) 2 φ (m) 2,, ξ (m 1) Otherwse, set m = m + 1 and go to step (4) C Effcency of the IAD-based RWR N φ (m) N ) IAD-based RWR s a dvde-and-conquer method whch takes advantage of the block-wse structure of real world

5 networks The runnng tme of the algorthm depends manly on two factors: number of teratons and, for each teraton, the tme t takes to solve the Eq (10) for N block submatrces Solvng Eq (10) takes tme proportonal to the sze of the matrx M ) The CLUTO algorthm that we use to partton M takes as nput the number N of blocks needed and optmzes block sze to avod parttons wth a lot of small blocks and several large block Thus, the resultng parttons are well suted for the IAD approach The global convergence of the IAD method s stll an open problem However, we wll show that for real world networks that have a natural block-wse structure the algorthm converges very fast As for space, the algorthm stores the dagonal matrces and the sparse matrx of the cross-lnk network The aggregated matrx requres O(N 2 ) space VI EXPERIMENTAL EVALUATION A Data Sets and Questons The data set used for the experments n ths paper (called paper & co-author data) s constructed from the Cora data set mccallum, whch contans research papers For each paper, the followng felds are avalable: ttle, authors, topc, abstract, avenue (eg, conference name), among others The data set s obtaned from Cora as follows: We frst extract publcatons for whch ttle and authors names are avalable From the resultng set of papers, we select those for whch abstracts are avalable Ths results n a data set that contans 4,100 papers and 10,830 authors Two networks are constructed from ths data Frst, we construct a sngle relatonal network G =< V, E > based on the co-author relaton The weght of an edge (a, b) from author a to author b s defned as the number of publcatons co-authored by a and b, dvded by the total number of the publcatons authored by a Second, we construct a b-relatonal network G =< V 1 V 2, E 1 E 2 > based on author-has-paper and paper smlarty relatons The weght on an edge (a, p) V 1 from an author a to a paper p s 1 f a s among paper s p authors and 0 otherwse The weght of an edge (p, q) E 2 between two papers p and q s gven by the cosne smlarty between the abstracts of the two papers To calculate cosne smlarty we buld an nverted ndex over the merged vocabulary of all abstracts Usng the nverted ndex, each abstract s represented usng the TF-IDF (term frequency, ndex document frequency) weghtng scheme The questons that we want to answer about the paper & co-author data set are the followng: (Q1) What are the most relevant authors to an author a, as dentfed through the analyss of the sngle and b-relatonal networks, respectvely? Intutvely, the more smlar the papers that two authors share, the more related the authors are (Q2) How accurate s the process of mnng nformaton from the sngle and b-relatonal networks, respectvely? B Expermental Desgn and Results We answer (Q1) through a case study We run RWR on the sngle and b-relatonal networks descrbed n secton VI-A, respectvely, to get relevance score vectors We use the author Jawe Han as a startng node Table II left column shows the top 10 relevant authors for Jawe Han, as dentfed from the sngle-relatonal co-author network As expected, these are mostly hs collaborators (researchers that have co-authored papers wth hm) Table II rgh column shows the top 10 relevant authors for Jawe Han, as dentfed from the b-relatonal network These are researchers whose nterests are smlar to Jawe Han s nterests (specfcally, database and data mnng) Ths case study shows the advantage of usng the b-drectonal network n the analyss: t produces results that can be used for predctng potental future collaboratons or even potental revewers for a researcher Table II AUTHORS RELEVANT TO JIAWEI HAN Sngle-relatonal network B-relatonal network n stefanovc l lakshmanan j chang t topaloglou w gong j mylopoulos b xa r mssaou o r zaane r ramakrshnan m kamber h hrsh l lakshmanan s sudarshan k kopersk m j zak w wang a j bonner a pang d srvastava We conduct two experments to answer (Q2) The frst experment (Q21) s desgned to evaluate the accuracy of the process of labelng papers wth categores usng the smlarty network only The second experment (Q22) s desgned to test the accuracy of the process of predctng co-authors based on the b-relatonal network (Q21) In the Cora data set, each paper has a label ndcatng the research category assocated wth the paper We consder the k-th most related papers to a gven paper n the data set (accordng to ther relevance scores) The accuracy s defned as the number of papers categorzed n the same category as the gven paper, dvded by k Fgure 2 shows the results As expected, the accuracy decreases wth the number of papers k consdered (Q22) To test the accuracy of the process of predctng co-author relatonshps from the b-relatonal network, we randomly select three dstnct sets of author pars The authors n a par have co-authored some papers, whch are removed from the network We assume that n addton to the papers that a par of authors have co-authored (removed), the two authors mght have publshed other smlar papers Our ntuton s that f a par of authors share smlar papers, then they wll be predcted to be co-authors based on the brelatonal author-paper network

6 Fgure 2 Paper labelng predcton accuracy, as a functon of the total number k of papers for whch labels are predcted, based on mnng the paper smlarty network For each par, we run RWR starng at an arbtrary paper of one of the authors n the par (ths wll not be a co-authored paper, as those have been removed) and dentfy the k-th most related authors A par s predcted correctly f the coauthor n the par s among the related authors We defne the accuracy as the number of co-authors dentfed dvded by the number of pars n a data set Fgure 3 shows the results As expected, the more related authors k are retreved, the better the predcton accuracy Fgure 3 Co-author predcton accuracy, as a functon of the number k of authors retreved for each par of potental co-authors, based on mnng the b-relatonal network C Effcency of Random Walk Table III shows a comparson between the tradtonal power method (whch multples the transton matrx wth π untl the L 2 norm of successve estmates of π goes below the threshold ɛ) and the IAD-based RWR n terms of the π (1) π (0) 2 values after the frst teraton and number of steps to convergence Results for three parttons are shown Table III VALUES OF π (1) π (0) 2 AFTER THE FIRST ITERATION AND NUMBER OF ITERATIONS FOR CONVERGENCE WHEN ɛ = 10 5, FOR THE TRADITIONAL POWER METHOD VS IAD APPROACH π (1) π (0) 2 # Iteratons # Parttons Power IAD Power IAD e 2 179e e 2 159e e 2 133e VII SUMMARY AND RELATED WORK In ths paper, we have shown how to use the RWR approach to analyze a b-relatonal network A smlar analyss has been performed for networks wth three relatons, but was omtted here due to space lmtatons (to be publshed as a techncal report) Generalzaton of our approach to multrelatonal networks s possble, accordng to the semantcs of the relatons n a partcular network We have also proposed an IAD-based fast RWR mplementaton Ths mplementaton makes use of the blockwse structure that many networks present Expermental results on a data set from the bblographc doman show the benefts of usng b-relatonal networks as opposed to sngle networks The relevance scores defned by RWR have many useful propertes Compared wth other parwse metrcs, the relevance scores can capture the global structure of the graph as well as the mult-facet relatonshps between nodes RWR s a popular method for network analyss Many applcatons use random walk and related methods as a buldng block Tong et al [2] provdes an excellent revew of RWR An exact soluton for RWR usually requres the nverson of a large matrx Therefore, fast approxmate solutons to the problem have been proposed before [1] Smlar to our proposed approach, other exstng solutons make use of the block-wse structure of real world networks Tong et al [2] approxmate the statonary dstrbuton of RWR by heurstc-based low rank approxmaton Sun et al [1] perform RWR only on the partton that contans the startng pont Ther method outputs a local estmaton of the statonary dstrbuton REFERENCES [1] J Sun, H Qu, D Chakrabart, and C Faloutsos, Neghborhood formaton and anomaly detecton n bpartte graph, Proc of the Sxth Int Conf on Data Mnng (ICDM 05), pp , 2005 [2] H Tong, C Faloutsos, and J-Y Pan, Fast random walk wth restart and ts applcatons, Proc of the Sxth Int Conf on Data Mnng (ICDM 06), vol 418, pp , 2006 [3] W J Stewart, Introducton to the Numercal Soluton of Markov Chans Prnceton Unversty Press, 1996 [4] G Karyps and V Kumar, Multlevel k-way parttonng scheme for rregular graphs, J Parallel Dstrb Comput, vol 48, no 1, pp , 1998 [5] W J Stewart, Numercal methods for computng statonary dstrbuton of fnte rreducble markov chans, J of Adv n Computatonal Probablty, vol 418, pp , 1998 [6] D P O Leary, Iteratve methods for fndng the statonary vector for markov chans, Lnear Algebra, Markov Chans, and Queung Models, vol 418, pp , 1992 [7] Y Takahash, A lumpng method for numercal calculatons of statonary dstrbutons of markov chans Techncal Report B-18, Tokyo Inst of Tech, vol 418, pp , 1975