Bayesian Poisson Tensor Factorization for Inferring Multilateral Relations from Sparse Dyadic Event Counts

Transcription

1 Bayesian Poisson Tensor Factorization for Inferring Multilateral Relations from Sparse Dyadic Event Counts KDD 2015 Aaron Schein UMass Amherst Joint work with: John Paisley, Dave Blei & Hanna Wallach Columbia University Microsoft Research

2 International relations are multilateral

3 From Schrodt (1993) Event Data in Foreign Policy Analysis:

4 From Schrodt (1993) Event Data in Foreign Policy Analysis:

5 From Schrodt (1993) Event Data in Foreign Policy Analysis:

6 From Schrodt (1993) Event Data in Foreign Policy Analysis:

7 From Schrodt (1993) Event Data in Foreign Policy Analysis:

8 From Schrodt (1993) Event Data in Foreign Policy Analysis:

9 From Schrodt (1993) Event Data in Foreign Policy Analysis:

10 From Schrodt (1993) Event Data in Foreign Policy Analysis:

11 From Schrodt (1993) Event Data in Foreign Policy Analysis:

12

13

14 Multilateral relation A coherent thread of international events.

15 Multilateral relation A coherent thread of international events. Characterized by: 1. sender countries 2. receiver countries 3. action types 4. time steps

16 Multilateral relation A coherent thread of international events. Characterized by: 1. sender countries 2. receiver countries 3. action types 4. time steps

17 Multilateral relation A coherent thread of international events. Characterized by: 1. sender countries 2. receiver countries 3. action types 4. time steps Who is doing what to whom, when?

18 Goal: Infer multilateral relations Tease apart the coherent threads of:

19 Goal: Infer multilateral relations Tease apart the coherent threads of: (millions of unstructured dyadic events)

20 Sample result: time steps senders receivers action types

21 Sample result: Afghanistan War time steps senders receivers action types

22 Sparse event counts Express intent to cooperate Threaten United States Russia Israel Iraq China United Kingdom Iran Palestine France Japan Europe Pakistan Germany Turkey Afghanistan Egypt Africa Syria Lebanon North Korea South Korea Australia Jordan Italy Saudi Arabia Canada Indonesia Nigeria Sudan Spain Philippines Ukraine Kenya NATO Poland July 2002 United States Russia Israel Iraq China United Kingdom Iran Palestine France Japan Europe Pakistan Germany Turkey Afghanistan Egypt Africa Syria Lebanon North Korea South Korea Australia Jordan Italy Saudi Arabia Canada Indonesia Nigeria Sudan Spain Philippines Ukraine Kenya NATO Poland United States Russia Israel Iraq China United Kingdom Iran Palestine France Japan Europe Pakistan Germany Turkey Afghanistan Egypt Africa Syria Lebanon North Korea South Korea Australia Jordan Italy Saudi Arabia Canada Indonesia Nigeria Sudan Spain Philippines Ukraine Kenya NATO Poland

23 Sparse event counts Express intent to cooperate Threaten United States Russia Israel Iraq China United Kingdom Iran Palestine France Japan Europe Pakistan Germany Turkey Afghanistan Egypt Africa Syria Lebanon North Korea South Korea Australia Jordan Italy Saudi Arabia Canada Indonesia Nigeria Sudan Spain Philippines Ukraine Kenya NATO Poland July 2002 United States Russia Israel Iraq China United Kingdom Iran Palestine France Japan Europe Pakistan Germany Turkey Afghanistan Egypt Africa Syria Lebanon North Korea South Korea Australia Jordan Italy Saudi Arabia Canada Indonesia Nigeria Sudan Spain Philippines Ukraine Kenya NATO Poland United States Russia Israel Iraq China United Kingdom Iran Palestine France Japan Europe Pakistan Germany Turkey Afghanistan Egypt Africa Syria Lebanon North Korea South Korea Australia Jordan Italy Saudi Arabia Canada Indonesia Nigeria Sudan Spain Philippines Ukraine Kenya NATO Poland

24 Poisson matrix factorization Australia New Zealand China Vietnam Cambodia Laos Australia New Zealand China Vietnam Cambodia Laos

25 Poisson matrix factorization Australia New Zealand China Vietnam Cambodia Laos Australia New Zealand China Vietnam Cambodia Laos

26 Poisson matrix factorization Australia New Zealand China Vietnam Cambodia Laos Australia New Zealand China Vietnam Cambodia Laos

27 Poisson matrix factorization Australia New Zealand China Vietnam Cambodia Laos Australia New Zealand China Vietnam Cambodia Laos

28 Poisson matrix factorization Australia New Zealand China Vietnam Cambodia Laos Australia New Zealand China Vietnam Cambodia Laos

29 Poisson matrix factorization Y = N N

30 Poisson matrix factorization Y = N y ij Poisson X k N (1) (2) ik jk

31 Poisson matrix factorization Y = N y ij Poisson X k N (1) (2) ik jk how active country i is as a sender in community k

32 Poisson matrix factorization Y = N y ij Poisson X k N (1) (2) ik jk how active country i is as a sender in community k how active country j is as a receiver in community k

33 Poisson matrix factorization Y Pois (1) (2)

34 Poisson matrix factorization Y Pois (1) (2) Fitting this model is a form of nonnegative matrix factorization: Y (1) N K 2 R + (2) K N 2 R +

35 Poisson matrix factorization Australia New Zealand Australia New Zealand China Vietnam Cambodia Laos China Vietnam (1) 1 (2) 1 Cambodia Laos component 1

36 Poisson tensor factorization y ijat Poisson X k! (1) (2) (3) (4) ik jk ak tk

37 Poisson tensor factorization y ijat Poisson X k! (1) (2) (3) (4) ik jk ak tk how active country i is as a sender in multilateral relation k j how active country is as a receiver in multilateral relation k

38 Poisson tensor factorization how relevant is action type to multilateral relation k y ijat Poisson a X k! (1) (2) (3) (4) ik jk ak tk how active country i is as a sender in multilateral relation k j how active country is as a receiver in multilateral relation k

39 Poisson tensor factorization how relevant is action type to multilateral relation k y ijat Poisson a X k how active at time step t is multilateral relation k! (1) (2) (3) (4) ik jk ak tk how active country i is j how active country is as a sender in multilateral relation k as a receiver in multilateral relation k

40 Poisson tensor factorization y ijat Poisson X k! (1) (2) (3) (4) ik jk ak tk

41 Poisson tensor factorization y ijat Poisson X k! (1) (2) (3) (4) ik jk ak tk Fitting this model is a form of nonnegative tensor factorization:

42 Poisson tensor factorization y ijat Poisson X k! (1) (2) (3) (4) ik jk ak tk Fitting this model is a form of nonnegative tensor factorization: (1) N K 2 R + sender factors Y (2) (3) 2 R + N K receiver factors 2 R + A K action type factors (4) 2 R + T K time step factors

43 Sample component time-steps senders receivers action types

44 Sample component: Yugoslav Wars Components correspond to multilateral relations

45 Parameter estimation: Bayesian inference y ijat Poisson X k (1) ik (2) jk (3) ak (4) tk!

46 Parameter estimation: Bayesian inference y ijat Poisson X k (1) ik (2) jk (3) ak (4) tk! (1) ik (2) jk Gamma, (1) Gamma, (2) (3) ak Gamma, (3) (4) tk Gamma, (4)

47 Parameter estimation: Bayesian inference y ijat Poisson X k (1) ik (2) jk (3) ak (4) tk! (1) ik (2) jk Gamma, (1) Gamma, (2) (3) ak Gamma, (3) (4) tk Gamma, (4) } Sparsity-inducing Gamma priors < 1

48 Parameter estimation: Bayesian inference P (1), (2), (3), (4) Y

49 Parameter estimation: Bayesian inference P (1), (2), (3), (4) Y cannot compute this analytically

50 Parameter estimation: Variational inference P (1), (2), (3), (4) Y

51 Parameter estimation: Variational inference P (1), (2), (3), (4) Y cannot compute this analytically

52 Parameter estimation: Variational inference P (1), (2), (3), (4) Y cannot compute this analytically Define a convenient family of distributions: Q (1), (2), (3), (4)

53 Parameter estimation: Variational inference P (1), (2), (3), (4) Y cannot compute this analytically Define a convenient family of distributions: Q (1), (2), (3), (4) Optimize the parameters of Q to minimize: KL(Q P )

54 Parameter estimation: Variational inference (m)

55 Parameter estimation: Variational inference ˆ (m) ik

56 Parameter estimation: Variational inference Q( (m) ik )

57 Parameter estimation: Variational inference Q( (m) ik ) = Gamma (m) ik ; (m) ik, (m) ik

58 Parameter estimation: Variational inference Q( (m) ik ) = Gamma (m) ik ; (m) ik, (m) ik { variational parameters

59 Parameter estimation: Variational inference Q( (m) ik ) = Gamma (m) ik ; (m) ik, (m) ik After inference, if we need a point estimate: ˆ (m) ik := E Q [ (m) ik ]= ik (m) (m) ik

60 Parameter estimation: Variational inference Q( (m) ik ) = Gamma (m) ik ; (m) ik, (m) ik After inference, if we need a point estimate: ˆ (m) ik := G Q [ (m) ik ik )) (m) ik ]=exp( ( (m)

61 Parameter estimation: Variational inference

62 Parameter estimation: Variational inference Bayesian PTF generalizes much better than maximum likelihood PTF when the data is very sparse!

63 Parameter estimation: Variational inference Bayesian PTF generalizes much better than maximum likelihood PTF when the data is very sparse! No sacrifice in efficiency!

64 Code and more sample results available:

65 Our favorite component time steps senders receivers action types Jan 2011 Mar 2011 May 2011 Jun 2011 Aug 2011 Sep 2011 Nov 2011 Jan 2012 Feb 2012 Apr 2012 May 2012 Jul 2012 Sep 2012 Oct 2012 Dec Ecuador United Kingdom United States Sweden Australia Spain Russia Japan Belarus Cuba Ecuador United Kingdom United States Sweden Australia Russia Belarus Spain Paraguay Europe Consult Aid Appeal Make Statement Coerce Disapprove Intend to Cooperate Cooperate (Diplomatic) Reject Cooperate (Material)

66 Our favorite component time steps senders receivers action types Jan 2011 Mar 2011 May 2011 Jun 2011 Aug 2011 Sep 2011 Nov 2011 Jan 2012 Feb 2012 Apr 2012 May 2012 Jul 2012 Sep 2012 Oct 2012 Dec Ecuador United Kingdom United States Sweden Australia Spain Russia Japan Belarus Cuba Ecuador United Kingdom United States Sweden Australia Russia Belarus Spain Paraguay Europe Consult Aid Appeal Make Statement Coerce Disapprove Intend to Cooperate Cooperate (Diplomatic) Reject Cooperate (Material)

67 Our favorite component time steps senders receivers action types Jan 2011 Mar 2011 May 2011 Jun 2011 Aug 2011 Sep 2011 Nov 2011 Jan 2012 Feb 2012 Apr 2012 May 2012 Jul 2012 Sep 2012 Oct 2012 Dec Ecuador United Kingdom United States Sweden Australia Spain Russia Japan Belarus Cuba Ecuador United Kingdom United States Sweden Australia Russia Belarus Spain Paraguay Europe Consult Aid Appeal Make Statement Coerce Disapprove Intend to Cooperate Cooperate (Diplomatic) Reject Cooperate (Material)

68 Thanks! Come to poster 897