Cheetah: Fast Graph Kernel Tracking on Dynamic Graphs

Transcription

1 Cheetah: Fast Gaph Kenel Tacking on Dynamic Gaphs Pesente: Liangyue Li Joint wok with Hanghang Tong (ASU), Yanghua Xiao (Fudan), Wei Fan (Baidu) 1 Aizona State Univesity

2 Gaphs ae Eveywhee Collaboation Netwoks US Powe Gid Bus Netwok 2 Bain Netwoks Patient Netwoks Hospital Netwoks Aizona State Univesity

3 Application 1: Web Mining Q: How simila ae the two gaphs? A: Gaph Kenel 1. Fo each entity, constuct a neighbohood gaph by beadth-fist seach up to depth k 2. Apply gaph kenel in kenel based leaning methods 3 Lösch, Uta, Stephan Bloehdon, and Achim Rettinge. "Gaph kenels fo RDF data." The Semantic Web: Reseach and Applications. Spinge Belin Heidelbeg, Aizona State Univesity

4 Application 2: Compute Vision ea, sun, ( sea, sun,, waves ) sky, waves ) (a) (a) ( cat, foest, ( cat, foest, gass, tige ) gass, tige ) (b) (b) - no caption - no caption (c) (c) (c) (f) (f) 5 (e) (d) 5 (d)4 (e) gaphs? 2 Q: How 27 simila ae the two A: Gaph Kenel sea, sun, ( sea, sun,, waves ) 1 waves ) 1 sky, (a)4 (a) ( cat, foest, ( cat, foest, gass, tige ) gass, tige ) 5 5 (b) (b) - no 8caption - no 8caption 9 (c) (d) (e) (d) (f) (e) (f) 1. Fo each image, epesent it as a segmentation gaph i3 methods 2. Apply in kenel based ii1 2 gaph kenel i2 i3 i3 leaning i i with segmentation 1Hachaoui, 2 3 Zaïd, 4 and5fancis 6 Bach. 7 "Image 8 classification gaph kenels." Aizona State Univesity CVPR 2007.

5 Application 3: Neuoscience Q: How simila ae the two gaphs? A: Gaph Kenel 1. Fo each bain image, epesent it as a gaph 2. Apply gaph kenel in kenel based leaning methods 5 L. Shi, H. Tong, X. Mu: BainQuest: Peception-Guided Visual Bain Aizona Compaison, State Univesity

6 Random Walk based Gaph Kenel h b a e i 3 4 c d j f g Gaph 1 Gaph 2 Intuitions: 1. Compae similaity of evey pai of nodes fom each gaph Eg: (1,2) vs (a, j)! less simila (1,5) vs (a,e)! moe simila 2. Node pai similaity is measued by andom walks 3. Two gaphs ae simila if they shae many simila node pais 6 Aizona State Univesity

7 Konecke Poduct Gaph Gaph Illustation 2' 2 3' A 1 3 1' 1 X A 2 11' 21' 34' 24' 14' 33' 23' 31' 12' 4' 22' 32' 13' A 1 A 2 A 1 Matix Desciption = A 2 = A 1 A 2 = Konecke poduct One Random Walk on + = One Random Walk on One Random Walk on A 1 A 2 A 1 A 2 = A 7 S. V. N. Vishwanathan, Nicol N. Schaudolph, Ime Risi Kondo, and Kasten M. Bogwadt. Gaph Kenels. Jounal of Machine Leaning Reseach, 11: , Apil Aizona State Univesity

8 RWR Gaph Kenel Fomulation Taking expectations instead of summing Ke(G 1,G 2 ) = P k ck q 0 A k p = q 0 (I ca ) 1 p Computational challenge: A is of size n 2 n 2 O(n 6 ) (Diect computation) o O(n 3 ) (Sylveste equation) Time > 1h, n= S. V. N. Vishwanathan, Nicol N. Schaudolph, Ime Risi Kondo, and Kasten M. Bogwadt. Gaph Kenels. Jounal of Machine Leaning Reseach, 11: , Apil Aizona State Univesity

9 Speed up ARK Idea: pefom low-ank appox on both gaphs Ke(G 1, G 2 )=(q 1 0 q 2 0 )(I ca 1 0 A 2 0 ) 1 (p 1 p 2 ) Step 1: Top- low-ank appox: Step 2: Matix-invese Lemma: 2 2 U 1 1 V 0 1 U 2 2 V 0 2 Ke(G 1, G 2 ) (q 0 1p 1 )(q 0 2p 2 )+c(q 0 1U 1 q 0 2U 2 ) (V 0 1p 1 V 0 2p 2 ) =(( 1 2 ) 1 c(v 0 1 V 0 2)(U 1 U 2 )) 1 Matix of size, easy to invese Oveall complexity: O(n m + 6 ) Time = 7.5s, n=3328 Can be educed to O(n 2 + m + 6 ) 9 U. Kang, Hanghang Tong, Jimeng Sun. Fast Random Walk Gaph Aizona Kenel. State SDM Univesity 2012

10 Challenges ARK: Good fo static gaphs What if gaphs ae evolving ove time Static,ea, sun, sun, waves ), waves ) ) (a) ( cat, foest, ( cat, foest, gass, tige ) gass, tige ) (b)(b) ) (d) 1 ea (e)(e) i1 i1 sun sun sky sky ( sea, sun, ( sea, sun, ( cat, foest, ( cat, foest,- no caption - no caption - no caption - no caption ( sea, sun, ( sea, ( cat, foest, - no caption sun, ( cat, foest, - no caption sky, waves ) gass, tige ) sky, waves ) gass, tige ) ( sea, sun, sky, ( cat, foest, - no caption ( sea, waves ) sun, ( cat, foest, - no caption sky, waves ) gass, tige ) gass, tige ) (a) (b) (c) (a) (b) (c) (c) sky, e waves ) gass, tige ) (c) sky, waves ) gass, tige ) m (a) (c) i (a) (b) (c) T(b) (a) (b) (c) (a) (b) (c) (d) (e) (f) (f) (d) (d) (d) (d) (d) (e) (e) e im T(e) (e) i3 ii33 i 3 i2 i2 t1 t t 2 t t3 t t 4 t t 5 t t6 t t 7 t t sea Dynamic (e) (f) (f) (f) (f)(f) cat waves cat foest foest gass tige gass tige i3 i 3 i2 i3 i 3 i i1 i 2 i1 i1 3 i3 i 2 i i1 i 2 i1 i3 i 3 i2 e i3 i3 3 i 3 i1 i2 m i t t t t Tt t t t t1 t2 t3 t1t4 1 t2t5 2 t3t6 3 t4t7 4 t5t8 5 t6 6 t7 7 t8 8 t t t t t t t t t1 1 t2 2 t3 3 t4t1 4 t5t2 5 t6t3 6 t7t4 7 t8t5 8 t6 t7 (g)t8 (g) (g) sea sea sea sun sun sky sky sea waves sun sun cat cat waves sea sun cat waves sky cat waves gass sky foest waves cat tige foest gass tige cat waves sky foest gass tige foest foest gass gass e Tim (f) Q: How to tack gaph kenel efficiently? waves e Tim e Tim tige tige (g) (g) (g) (g) (g) 10 Aizona State Univesity Figue Thee sample gaphimages, two of them annotated; thei egions (d,e,f); and thei e images, two of them annotated; thei egions (d,e,f); and 1. thei sea sun sky foest gass tige

11 Roadmap Motivation Cheetah-D fo Diected Gaphs Expeimental Results Conclusion 11 Aizona State Univesity

12 Cheetah-D: gaph kenel tacking Ke(G 1, G 2 )=(q 1 0 q 2 0 )(I ca 1 0 A 2 0 ) 1 (p 1 p 2 ) top- low-ank appox: U 1 1 V 0 1 U 2 2 V 0 2 A 1 A 2 Ideas: Avoid: e-computing low-ank appox [ARK] Goal: tack low-ank stuctue efficiently [Cheetah-D] (SVD in this pape) 12 Aizona State Univesity

13 Step 0: Low ank appox on A Intuition: A 0 v1 v2 (m =5, = 2) u 1 u 2 0 Details: A A 0 U 0 0 V 0 0 = x 1 A = XY Z 0 =[U 0 X] A = A 0 + A Y z1 apple Y [V 0 Z] 0 (m 0 =2, 0 = 1) Popety: SVD on SVD on A 0 A takes: takes: O(m + n) O(m n 0 ) O(m + n) [m 0 m, 0 ] 13 Aizona State Univesity

14 Step 1: Patial QR Decomposition Intuition: Case 1: x 1 2 span(u 1,u 2 ) u 2 x 1 u 1 u 1 u 2 x 1 = u 1 u 2 S Details: [U 0 X]=U 0 S apple 0 0 A =[U 0 X] 0 Y [V 0 Z] 0 apple 0 0 = U 0 S 0 Y T 0 V0 0 O(n 02 ) Popety: Efficiency: takes [ 0 ] Effectiveness: No exta eo 14 Aizona State Univesity

15 Step 1: Patial QR Decomposition Intuition: Case 2: x 1 /2 span(u 1,u 2 ) x 1 u 1 u 2 x 1 = q u 2 u 1 u 1 u 2 q S Details: [U 0 X]=[U 0 q]s A =[U 0 X] =[U 0 Popety: Efficiency: takes O(n 02 ) [ 0 ] Effectiveness: No exta eo apple Y [V 0 Z] 0 apple 0 0 Q]S 0 Y T 0 [V 0 Z] 0 Simila Patial QR decomp on Z 15 Aizona State Univesity

16 Step 2: Full SVD on a Small Matix Intuition: M ( + 0 ) ( + 0 ) = S 0 Y T 0 Details: M = S = L R 0 Updated apple Y T 0 = L R 0 A =[U 0 apple 0 0 Q]S 0 Y T 0 [V 0 Z] 0 =[U 0 Q]L R 0 T 0 [V 0 Z] 0 Popety: [ 0 ] Efficiency: takes O(( + 0 ) 3 ) Effectiveness: No exta eo 16 Aizona State Univesity

17 Step 3: Rotate Othonomal Basis Intuition: Rotate u 1 u 2 q by L Rotate v 1 v 2 z by Details: U =[U 0 V =[V 0 R Q]L Z]R Popety: 17 Complexity: O(n 2 ) Oveall SVD Update Complexity: O(n 2 + m n 02 ) Re-compute SVD: O(n 2 + m) Aizona State Univesity

18 Analysis and Vaiants Time complexity of Cheetah-D: Compaison Example ARK:7.5s Ous:0.4s Vaiants Undiected gaphs Attibuted gaphs O(n 2 + n ) ARK: O(n m + 6 ) (n = 3328, = 500, 0 = 5) Cheetah-D Algoithm Sketch t = 1, Initialize SVD of A1 and A2 fo t=2,3, Update SVD fo A1 Update SVD fo A2 Update Ke(A1,A2) end O(n( )) O(n( )) O(n ) 18 Aizona State Univesity

19 Roadmap Motivation Cheetah-D fo Diected Gaphs Expeimental Results Conclusion 19 Aizona State Univesity

20 Case Study MTA Bus Taffic Gaph constuction Monito taffic volume of 30 bus stops on 3 outes, fom Monday, 03/24/2014 Sunday, 03/30/2014 Repesent each stop as a time seies whee each timestamp is taffic volume within each hou On each day, build a gaph fo the 30 stops using Gange causality test Gaph kenel computation Gaph kenel is computed between two gaphs of two consecutive days 20 Aizona State Univesity

21 Case Study MTA Bus Taffic Nomalized Ke(G1,G2) Tue Wed Fi Sat (Mon,Tue) (Tue, Wed) (Wed,Thu) (Thu,Fi) (Fi,Sat) (Sat,Sun) Weekdays Schedule Weekends Schedule 21 Aizona State Univesity

22 Relative Eo Relative Eo =100 =200 =300 =400 = % 0.018% 0.016% 0.014% 0.012% 0.010% 0.008% 0.006% 0.004% 0.002% 0.000% All cases, E <0.02% n = 4183, m = Aizona State Univesity

23 Relative Eo Running Time (Seconds) Avg Eo vs. Rank 0.25% % '=5 '= % '=40 '= % '=80 '= % % Reduced ank 0 0 When >50, E<0.05% n = 4183, m = 5692 Figue 6: di eent ga 23 Aizona State Univesity 0.8

24 Running Time (Seconds) Running Time (Seconds) Running Time vs. Rank di een Cheetah-U ARK-U Reduced Rank ARK-U+ 10x OURS Figue Aizona State Univesity

25 Running Time (Seconds) Scalability Figue 6: Gaph Size (n) Scale nea linealy =50 =100 =150 =200 =250 Running time of Cheetah-U on AS with Aizona State Univesity

26 Relative Eo Quality vs. Speed > Ous ARK-U+ Cheetah-U Fist-ode Second-ode Running Time (Seconds) 26 Aizona State Univesity

27 Roadmap Motivation Cheetah-D fo Diected Gaphs Expeimental Results Conclusion 27 Aizona State Univesity

28 Conclusion Goal: tack gaph kenel of dynamic gaphs Ou Solution: Cheetah-D Key idea: tack low-ank appox Results: Complexity: O(n 2 + n ) In pactice: ~15x faste, E<0.05% Moe in pape: Cheetah-U fo undiected gaphs Eo bound analysis 28 Aizona State Univesity