THE POWER OF CERTAINTY

Transcription

1 A DIRICHLET-MULTINOMIAL APPROACH TO BELIEF PROPAGATION Dhivya Eswaran* CMU Stephan Guennemann TUM Christos Faloutsos CMU

2 PROBLEM MOTIVATION NETCONF GUARANTEES PROBLEM

3 RECOMMENDATION BOB JOHN BOB ALICE CAROL SMITH ALICE CAROL SMITH ADS PLACEMENT

4 GRAPH LABELING / NODE CLASSIFICATION H GIVEN a graph of nodes & edges SMITH labels for a few nodes label compatibility ALICE BOB FIND labels of all nodes H JOHN CAROL

5 NODE CLASSIFICATION IS HARD 4 x 30 x 1 x 15 x SMITH JOHN Higher fraction of android friends Who is more likely to buy an Android phone? Higher number of android friends 5

6 PROBLEM MOTIVATION NETCONF GUARANTEES MOTIVATION

7 CLASSIFICATION BY PROPAGATION INITIALIZE: Set nodes to random/given values. BOB PROPAGATE: Update each node s value based on the values of its neighbors. ALICE SMITH CAROL CONVERGENCE: If no value changes, terminate. Else continue propagation. Q1. What are values here? Q2. How are they updated? 7

8 BELIEF PROPAGATION [0.5, 0.5] DETAILS! A1. Values : beliefs (probability vectors) [0.4, 0.6] BOB [0.8, 0.2] (i) A2. Update : 2 stages Each neighbor sends a message m vu (i) kx H(i, j)e v (j) Y m wu (i) ALICE CAROL j=1 v2n (v)\u SMITH [0.73, 0.27] b u (i) e u (i) (ii) Node updates its belief based on messages Y v2n (u) m vu (i) 8

9 BP LEADS TO COUNTER-INTUITIVE RESULTS Who is more likely to buy an Android phone? 13 x 110 x SMITH 3 x 100 x JOHN BP INTUITION 9

10 MAIN IDEA belief distributions as values probability certainty / confidence (absolute count of neighbors) belief / leaning (ratio of android : apple neighbors) 10

11 PROBLEM MOTIVATION NETCONF GUARANTEES NETCONF

12 NODE CLASSIFICATION WITH CERTAINTY GIVEN a graph of nodes & edges belief distributions for a few nodes H label compatibility FIND SMITH belief distributions of all nodes SUBJECT TO ALICE BOB theoretically-grounded algorithm JOHN fast & scalable implementation CAROL 12

13 DIRICHLET BELIEF DISTRIBUTIONS (1/2) BACKGROUND 2D Dirichlet distribution (Beta distribution) p(x; +1, + 1) / x (1 x) Belief/Leaning : # # 13

14 DIRICHLET BELIEF DISTRIBUTIONS (2/2) BACKGROUND Certainty: + EXTENDS TO ANY NUMBER OF DIMENSIONS!! Example: 3 dimensions (talkative / silent / confused) Image source: UBC Wiki 14

15 MULTINOMIAL MESSAGE DISTRIBUTIONS DETAILS! BP belief update rule b u (i) e u (i) Y v2n (u) m vu (i) DIRICHLET DISTRIBUTION MULTINOMIAL DISTRIBUTION! NETCONF belief update rule b u ĕ u + X v2n (u) m vu parameters of belief distribution parameters of message distribution 15

16 MESSAGES FROM BELIEFS (a, b) SMITH (a, b) JOHN perfect homophily no network effects perfect heterophily (0, 0) (b, a) M = k k 1 H 1 k + 16

17 NETCONF BELIEF & MESSAGE UPDATE RULES message update m vu M u + X v2n (v)\u m wu 1 A belief update b u ĕ u + X v2n (u) m vu M b u m uv 17

18 NETCONF MATRIX BELIEF UPDATE B Ĕ +(A BM D BM 2 )(I M 2 ) 1 final belief distribution prior belief distribution adjacency diagonal degree modulation ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ 18

19 PROBLEM MOTIVATION NETCONF GUARANTEES GUARANTEES

20 KEY THEORETICAL QUESTIONS UNIQUENESS Is the steady state solution unique? CONVERGENCE Can we predict convergence? 20

21 NETCONF HAS CLOSED-FORM SOLUTION DETAILS! ITERATIVE UPDATE B Ĕ +(A BM D BM 2 )(I M 2 ) 1 Roth s Column Lemma CLOSED FORM vec( B) = D 1 I (M ˆM) T A +(M 2 ˆM) T vec( Ĕ) 21

22 GUARANTEES PRECISE CONVERGENCE GUARANTEES CLOSED FORM vec( B) = I (M ˆM) T A +(M 2 ˆM) T D 1 vec( Ĕ) DETAILS! NECESSARY & SUFFICIENT CONDITION (M ˆM) T A +(M 2 ˆM) T D < 1 GRAPH STRUCTURE LABEL COMPATIBILITY 22

23 KEY THEORETICAL GUARANTEES CLOSED-FORM Closed-form solution and unique fixed point! CONVERGENCE Necessary and sufficient conditions for convergence! 23

24 PROBLEM MOTIVATION NETCONF GUARANTEES

25 KEY QUESTIONS FOR EFFECTIVENESS Improves accuracy & precision? SCALABILITY Fast and scalable? INTERPRETABILITY Are final results interpretable? 25

26 DATA POLBLOGS DBLP POKEC 1.5K nodes, 19K edges 28K nodes, 67K edges 1.6M nodes, 30M edges 2 classes 4 classes 10 classes 26

27 NETCONF IS ACCURATE AND PRECISE HIGHER ACCURACY % BETTER PRECISION Ideal DATASET BP NETCONF POLBLOGS DBLP POKEC

28 NETCONF IS FAST AND SCALABLE 30M edges in ~7 seconds! Linear scaling with graph size 28

29 NETCONF GIVES INTERPRETABLE RESULTS TOP DATABASES AUTHORS IN DBLP AUTHOR H-index AUTHOR H-index Michael J Carey 48 Jiawei Han 139 Rakesh Agrawal 96 Annie W Shum - Jiawei Han 139 Werner Keibling - Hamid Pirahesh 40 Xiaofang Zhou 36 David J Dewitt 81 Bertram Ludascher 45 Serge Abiteboul 77 Amarnath Gupta - NETCONF Many papers & high H1 indices! BP 29

30 KEY EXPERIMENTAL FINDINGS EFFECTIVENESS Improves accuracy & precision! SCALABILITY Scales linearly with graph size! INTERPRETABILITY Certainty scores reflect intuition! 30

31 SUMMARY SUMMARY: NETCONF Theoretically grounded Closed-form solution Convergence guarantees Improved performance Fast and scalable Interpretable results Questions? 31