Bayesian Poisson Tensor Factorization for Inferring Multilateral Relations from Sparse Dyadic Event Counts

Bayesian Poisson Tensor Factorization for Inferring Multilateral Relations from Sparse Dyadic Event Counts KDD 2015 Aaron Schein UMass Amherst Joint work with: John Paisley, Dave Blei & Hanna Wallach Columbia University Microsoft Research

International relations are multilateral

From Schrodt (1993) Event Data in Foreign Policy Analysis:

Multilateral relation A coherent thread of international events.

Multilateral relation A coherent thread of international events. Characterized by: 1. sender countries 2. receiver countries 3. action types 4. time steps

Multilateral relation A coherent thread of international events. Characterized by: 1. sender countries 2. receiver countries 3. action types 4. time steps Who is doing what to whom, when?

Goal: Infer multilateral relations Tease apart the coherent threads of:

Goal: Infer multilateral relations Tease apart the coherent threads of: (millions of unstructured dyadic events)

Sample result: time steps senders receivers action types

Sample result: Afghanistan War time steps senders receivers action types

Sparse event counts Express intent to cooperate Threaten United States Russia Israel Iraq China United Kingdom Iran Palestine France Japan Europe Pakistan Germany Turkey Afghanistan Egypt Africa Syria Lebanon North Korea South Korea Australia Jordan Italy Saudi Arabia Canada Indonesia Nigeria Sudan Spain Philippines Ukraine Kenya NATO Poland July 2002 United States Russia Israel Iraq China United Kingdom Iran Palestine France Japan Europe Pakistan Germany Turkey Afghanistan Egypt Africa Syria Lebanon North Korea South Korea Australia Jordan Italy Saudi Arabia Canada Indonesia Nigeria Sudan Spain Philippines Ukraine Kenya NATO Poland United States Russia Israel Iraq China United Kingdom Iran Palestine France Japan Europe Pakistan Germany Turkey Afghanistan Egypt Africa Syria Lebanon North Korea South Korea Australia Jordan Italy Saudi Arabia Canada Indonesia Nigeria Sudan Spain Philippines Ukraine Kenya NATO Poland

Poisson matrix factorization Australia New Zealand China Vietnam Cambodia Laos Australia New Zealand China Vietnam Cambodia Laos

Poisson matrix factorization Y = N N

Poisson matrix factorization Y = N y ij Poisson X k N (1) (2) ik jk

Poisson matrix factorization Y = N y ij Poisson X k N (1) (2) ik jk how active country i is as a sender in community k

Poisson matrix factorization Y = N y ij Poisson X k N (1) (2) ik jk how active country i is as a sender in community k how active country j is as a receiver in community k

Poisson matrix factorization Y Pois (1) (2)

Poisson matrix factorization Y Pois (1) (2) Fitting this model is a form of nonnegative matrix factorization: Y (1) N K 2 R + (2) K N 2 R +

Poisson matrix factorization Australia New Zealand Australia New Zealand China Vietnam Cambodia Laos China Vietnam (1) 1 (2) 1 Cambodia Laos component 1

Poisson tensor factorization y ijat Poisson X k! (1) (2) (3) (4) ik jk ak tk

Poisson tensor factorization y ijat Poisson X k! (1) (2) (3) (4) ik jk ak tk how active country i is as a sender in multilateral relation k j how active country is as a receiver in multilateral relation k

Poisson tensor factorization how relevant is action type to multilateral relation k y ijat Poisson a X k! (1) (2) (3) (4) ik jk ak tk how active country i is as a sender in multilateral relation k j how active country is as a receiver in multilateral relation k

Poisson tensor factorization how relevant is action type to multilateral relation k y ijat Poisson a X k how active at time step t is multilateral relation k! (1) (2) (3) (4) ik jk ak tk how active country i is j how active country is as a sender in multilateral relation k as a receiver in multilateral relation k

Poisson tensor factorization y ijat Poisson X k! (1) (2) (3) (4) ik jk ak tk

Poisson tensor factorization y ijat Poisson X k! (1) (2) (3) (4) ik jk ak tk Fitting this model is a form of nonnegative tensor factorization:

Poisson tensor factorization y ijat Poisson X k! (1) (2) (3) (4) ik jk ak tk Fitting this model is a form of nonnegative tensor factorization: (1) N K 2 R + sender factors Y (2) (3) 2 R + N K receiver factors 2 R + A K action type factors (4) 2 R + T K time step factors

Sample component time-steps senders receivers action types

Sample component: Yugoslav Wars Components correspond to multilateral relations

Parameter estimation: Bayesian inference y ijat Poisson X k (1) ik (2) jk (3) ak (4) tk!

Parameter estimation: Bayesian inference y ijat Poisson X k (1) ik (2) jk (3) ak (4) tk! (1) ik (2) jk Gamma, (1) Gamma, (2) (3) ak Gamma, (3) (4) tk Gamma, (4)

Parameter estimation: Bayesian inference y ijat Poisson X k (1) ik (2) jk (3) ak (4) tk! (1) ik (2) jk Gamma, (1) Gamma, (2) (3) ak Gamma, (3) (4) tk Gamma, (4) } Sparsity-inducing Gamma priors < 1

Parameter estimation: Bayesian inference P (1), (2), (3), (4) Y

Parameter estimation: Bayesian inference P (1), (2), (3), (4) Y cannot compute this analytically

Parameter estimation: Variational inference P (1), (2), (3), (4) Y

Parameter estimation: Variational inference P (1), (2), (3), (4) Y cannot compute this analytically

Parameter estimation: Variational inference P (1), (2), (3), (4) Y cannot compute this analytically Define a convenient family of distributions: Q (1), (2), (3), (4)

Parameter estimation: Variational inference P (1), (2), (3), (4) Y cannot compute this analytically Define a convenient family of distributions: Q (1), (2), (3), (4) Optimize the parameters of Q to minimize: KL(Q P )

Parameter estimation: Variational inference (m)

Parameter estimation: Variational inference ˆ (m) ik

Parameter estimation: Variational inference Q( (m) ik )

Parameter estimation: Variational inference Q( (m) ik ) = Gamma (m) ik ; (m) ik, (m) ik

Parameter estimation: Variational inference Q( (m) ik ) = Gamma (m) ik ; (m) ik, (m) ik { variational parameters

Parameter estimation: Variational inference Q( (m) ik ) = Gamma (m) ik ; (m) ik, (m) ik After inference, if we need a point estimate: ˆ (m) ik := E Q [ (m) ik ]= ik (m) (m) ik

Parameter estimation: Variational inference Q( (m) ik ) = Gamma (m) ik ; (m) ik, (m) ik After inference, if we need a point estimate: ˆ (m) ik := G Q [ (m) ik ik )) (m) ik ]=exp( ( (m)

Parameter estimation: Variational inference

Parameter estimation: Variational inference Bayesian PTF generalizes much better than maximum likelihood PTF when the data is very sparse!

Parameter estimation: Variational inference Bayesian PTF generalizes much better than maximum likelihood PTF when the data is very sparse! No sacrifice in efficiency!

Code and more sample results available: https://github.com/aschein/bptf

Our favorite component 0.40 0.35 0.30 0.25 time steps senders receivers action types 0.20 0.15 0.10 0.05 0.00 Jan 2011 Mar 2011 May 2011 Jun 2011 Aug 2011 Sep 2011 Nov 2011 Jan 2012 Feb 2012 Apr 2012 May 2012 Jul 2012 Sep 2012 Oct 2012 Dec 2012 0.45 0.40 0.35 0.30 0.25 0.20 0.15 0.10 0.05 0.00 Ecuador United Kingdom United States Sweden Australia Spain Russia Japan Belarus Cuba Ecuador United Kingdom United States Sweden Australia Russia Belarus Spain Paraguay Europe Consult Aid Appeal Make Statement Coerce Disapprove Intend to Cooperate Cooperate (Diplomatic) Reject Cooperate (Material)

Thanks! Come to poster 897