Learning with Temporal Point Processes t Manuel Gomez Rodriguez MPI for Software Systems Isabel Valera MPI for Intelligent Systems Slides/references: http://learning.mpi-sws.org/tpp-icml18 ICML TUTORIAL, JULY 2018
Many discrete events in continuous Disease dynamics Q m ee, 2013 Online actions Financial trading Mobility dynamics 2
Variety of processes behind these events Events are (noisy) observations of a variety of complex dynamic processes FAST Stock trading News spread in Twitter Flu spreading Ride-sharing requests Article creation in Wikipedia Reviews and sales in Amazon A user s reputation in Quora SLOW in a wide range of temporal scales. 3
Example I: Information propagation S D means D follows S 3.25pm Bob Christine 3.00pm Beth 3.27pm Joe David 4.15pm t Friggeri et al., 2014 They can have an impact in the off-line world 4
Example II: Knowledge creation Addition Refutation t Question Answer Upvote t t
Example III: Human learning 1st year computer science student Introduction to programming Discrete math Project presentation For/do-while loops Define Set theory functions Graph Theory Powerpoint vs. Keynote Class inheritance Export Geometry pptx to pdf t If else How to write Logic switch Private functions PP templates Class destructor Plot library 6
Aren t these event traces just series? t t t t Discrete and continuous s series What about aggregating events in epochs? Discrete events in continuous The framework of temporal point processes provides a native How representation long is each epoch? How to aggregate events per epoch? What if no event in one epoch? What about -related queries? Epoch 1 Epoch 2 Epoch 3 t 7
Outline of the Seminar TEMPORAL POINT PROCESSES (TPPS): INTRO 1. Intensity function 2. Basic building blocks 3. Superposition 4. Marks and SDEs with jumps Next MODELS & INFERENCE 1. Modeling event sequences 2. Clustering event sequences 3. Capturing complex dynamics 4. Causal reasoning on event sequences RL & CONTROL 1. Marked TPPs: a new setting 2. Stochastic optimal control 3. Reinforcement learning Slides/references: learning.mpi-sws.org/tpp-icml18 8
Temporal Point Processes (TPPs): Introduction 1. Intensity function 2. Basic building blocks 3. Superposition 4. Marks and SDEs with jumps 9
Temporal point processes Temporal point process: A random process whose realization consists of discrete events localized in Discrete events History, Dirac delta function Formally: 10
Model as a random variable Prob. between [t, t+dt) density History, Prob. not before t Likelihood of a line: 11
Problems of density parametrization (I) It is difficult for model design and interpretability: 1. Densities need to integrate to 1 (i.e., partition function) 2. Difficult to combine lines 12
Intensity function density Prob. between [t, t+dt) History, Prob. not before t Intensity: Probability between [t, t+dt) but not before t Observation: It is a rate = # of events / unit of 13
Advantages of intensity parametrization (I) Suitable for model design and interpretable: 1. Intensities only need to be nonnegative 2. Easy to combine lines 14
Relation between f*, F*, S*, λ* Central quantity we will use! 15
Representation: Temporal Point Processes 1. Intensity function 2. Basic building blocks 3. Superposition 4. Marks and SDEs with jumps 16
Poisson process Intensity of a Poisson process Observations: 1. Intensity independent of history 2. Uniformly random occurrence 3. Time interval follows exponential distribution 17
Fitting & sampling from a Poisson Fitting by maximum likelihood: Sampling using inversion sampling: 18
Inhomogeneous Poisson process Intensity of an inhomogeneous Poisson process Example: (Independent of history) 19
Fitting & sampling from inhomogeneous Poisson Fitting by maximum likelihood: Sampling using thinning (reject. sampling) + inverse sampling: 1. Sample from Poisson process with intensity using inverse sampling 2. Generate Keep sample with 3. Keep the sample if prob. 20
Terminating (or survival) process Intensity of a terminating (or survival) process Observations: 1. Limited number of occurrences Try sampling and fitting! 21
Self-exciting (or Hawkes) process History, Intensity of self-exciting (or Hawkes) process: Triggering kernel Observations: 1. Clustered (or bursty) occurrence of events 2. Intensity is stochastic and history dependent 22
Fitting a Hawkes process from a recorded line Fitting by maximum likelihood: The max. likelihood is jointly convex in and Sampling using thinning (reject. sampling) + inverse sampling: Key idea: the maximum of the intensity over changes
Summary Building blocks to represent different dynamic processes: Poisson processes: Inhomogeneous Poisson processes: We know how to fit them and how to sample from them Terminating point processes: Self-exciting point processes: 24
Representation: Temporal Point Processes 1. Intensity function 2. Basic building blocks 3. Superposition 4. Marks and SDEs with jumps 25
Mutually exciting process Bob History Christine History Clustered occurrence affected by neighbors 26
Mutually exciting terminating process Bob Christine History Clustered occurrence affected by neighbors 27
Representation: Temporal Point Processes 1. Intensity function 2. Basic building blocks 3. Superposition 4. Marks and SDEs with jumps 28
Marked temporal point processes Marked temporal point process: A random process whose realization consists of discrete marked events localized in History, 29
Independent identically distributed marks Distribution for the marks: Observations: 1. Marks independent of the temporal dynamics 2. Independent identically distributed (I.I.D.) 30
Dependent marks: SDEs with jumps History, Marks given by stochastic differential equation with jumps: Observations: Drift Event influence 1. Marks dependent of the temporal dynamics 2. Defined for all values of t 31
Dependent marks: distribution + SDE with jumps History, Distribution for the marks: Observations: Drift Event influence 1. Marks dependent on the temporal dynamics 2. Distribution represents additional source of uncertainty 32
Mutually exciting + marks Bob Christine Marks affected by neighbors Drift Neighbor influence 33
Marked TPPs as stochastic dynamical systems Example: Susceptible-Infected-Susceptible (SIS) SDE with jumps Susceptible Infected Susceptible Infection rate Node is susceptible It gets infected It recovers If friends are infected, higher infection rate SDE with jumps Recovery rate Self-recovery rate when node gets infected If node recovers, rate to zero Rate increases 34 if node gets treated
Outline of the Seminar TEMPORAL POINT PROCESSES (TPPS): INTRO 1. Intensity function 2. Basic building blocks 3. Superposition 4. Marks and SDEs with jumps MODELS & INFERENCE 1. Modeling event sequences 2. Clustering event sequences 3. Capturing complex dynamics 4. Causal reasoning on event sequences Next RL & CONTROL 1. Marked TPPs: a new setting 2. Stochastic optimal control 3. Reinforcement learning Slides/references: learning.mpi-sws.org/tpp-icml18 35