State-Space Methods for Inferring Spike Trains from Calcium Imaging

Size: px
Start display at page:

Download "State-Space Methods for Inferring Spike Trains from Calcium Imaging"

Transcription

1 State-Space Methods for Inferring Spike Trains from Calcium Imaging Joshua Vogelstein Johns Hopkins April 23, 2009 Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

2 Outline Introduction 1 Introduction 2 General Methods Generative Model Inverting the model 3 Simplifying our model How to simplify Approximations to our model Gaussian approximation 4 Fast non-negative spike inference (FANSI) Main idea Algorithmic details Discussion 5 Particle-filter-smoother (PFS) spike inference The forward recursion The backward recursion PFS algorithm for our model 6 PFS results Joshua Vogelstein Main result (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

3 What is our goal? Introduction Inferring spike trains using only calcium imaging Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

4 Introduction Why is this a hard problem? Many reasons... 1 Too many spike trains to search through them all 2 Noise is non-gaussian 3 Observation are non-linear 4 Parameters are unknown Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

5 Introduction What are we going to do? Our strategy Write down a generative model, explaining the causal relationship between spikes and movies Develop an algorithm to invert that model, to obtain spike trains and microcircuits from the movies Test our approach on real data Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

6 Outline General Methods 1 Introduction 2 General Methods 3 Simplifying our model 4 Fast non-negative spike inference (FANSI) 5 Particle-filter-smoother (PFS) spike inference 6 PFS results 7 Concluding thoughts Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

7 General Methods Generative Model Generative Model is a state-space model Our generative model Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

8 General Methods Generative Model Generative Model is a state-space model Our generative model n t Poisson(λ ) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

9 General Methods Generative Model Generative Model is a state-space model Our generative model n t Poisson(λ ) τ C t C t 1 = C t 1 + n t Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

10 General Methods Generative Model Generative Model is a state-space model Our generative model τ C t C t 1 n t Poisson(λ ) = C t 1 + n t Y x,t = α x [C t + β] + σ Y ε x,t, ε x,t iid N (0, 1) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

11 General Methods Generative Model Generative Model is a state-space model Our generative model τ C t C t 1 n t Poisson(λ ) = C t 1 + n t General state-space formalism Y x,t = α x [C t + β] + σ Y ε x,t, ε x,t iid N (0, 1) C t = γc t 1 + n t, Y t = αc t + ɛ t, n t Possion(λ ) ( ɛ t N αβ, σy 2 ) I Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

12 General Methods Generative Model Generative Model is a state-space model General state-space formalism C t = γc t 1 + n t, Y t = αc t + ɛ t, n t Possion(λ ) ( ɛ t N αβ, σy 2 ) I State-space distributions Transition distribution: P θ (C t C t 1 ) is Linear-Poisson Observation distribution: P θ ( Y t C t ) is Linear-Gaussian Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

13 General Methods Generative Model Generative Model for a single neuron Some Notation n = {n t } T t=0 is the spike train ε N (µ, Σ) means ε is distributed according to a Gaussian with mean µ and covariance Σ Y t = {Y x,t } P x=0 is the t-th image frame Y = { Y t } T t=1 is the entire movie θ = {λ, τ, σ c, α, β, σ Y } is the set of model parameters Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

14 General Methods Generative Model Generative Model for a single neuron Simulation Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

15 General Methods Generative Model Generative Model for a single neuron Schematic Spatially Filtered Fluorescence Calcium Spike Train Time (sec) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

16 Inverting the model General Methods Inverting the model What does this even mean? Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

17 Inverting the model General Methods Inverting the model What does this even mean? The model defines: Likelihood: P θ ( Y n) = N ( Y; µ, Σ) Prior: P θ (n) = Poisson(λ ) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

18 Inverting the model General Methods Inverting the model What does this even mean? The model defines: Likelihood: P θ ( Y n) = N ( Y; µ, Σ) Prior: P θ (n) = Poisson(λ ) We want the posterior: P θ (n Y) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

19 Inverting the model General Methods Inverting the model What does this even mean? The model defines: Likelihood: P θ ( Y n) = N ( Y; µ, Σ) Prior: P θ (n) = Poisson(λ ) We want the posterior: P θ (n Y) We know Bayes Rule: P θ (n Y) P θ ( Y n)p θ (n) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

20 Inverting the model General Methods Inverting the model What does this even mean? The model defines: Likelihood: P θ ( Y n) = N ( Y; µ, Σ) Prior: P θ (n) = Poisson(λ ) We want the posterior: P θ (n Y) We know Bayes Rule: P θ (n Y) P θ ( Y n)p θ (n) So, that should be no problem, right? Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

21 Inverting the model General Methods Inverting the model Problems Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

22 Inverting the model General Methods Inverting the model Problems Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

23 Inverting the model General Methods Inverting the model Problems We can compute P θ (n Y) for any individual n, but we may want a point estimate of this distribution Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

24 Inverting the model General Methods Inverting the model Problems We can compute P θ (n Y) for any individual n, but we may want a point estimate of this distribution For instance, a desirable quantity might be n MAP = argmax n P θ (n Y) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

25 Inverting the model General Methods Inverting the model Problems We can compute P θ (n Y) for any individual n, but we may want a point estimate of this distribution For instance, a desirable quantity might be n MAP = argmax n P θ (n Y) Another point estimate of interest may be n mean = E[n Y] Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

26 Inverting the model General Methods Inverting the model Problems We can compute P θ (n Y) for any individual n, but we may want a point estimate of this distribution For instance, a desirable quantity might be n MAP = argmax n P θ (n Y) Another point estimate of interest may be n mean = E[n Y] Both of these point estimates require having all possible spike trains Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

27 Inverting the model General Methods Inverting the model Problems We can compute P θ (n Y) for any individual n, but we may want a point estimate of this distribution For instance, a desirable quantity might be n MAP = argmax n P θ (n Y) Another point estimate of interest may be n mean = E[n Y] Both of these point estimates require having all possible spike trains Because n Poisson(λ ): there are an infinite number of possible spike trains we lack the calculus to integrate over Poisson Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

28 General Methods What to do? Approximate! Inverting the model Two general options Simplify the assumptions to get something tractable Monte Carlo sample from the model, to approximate the distributions Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

29 Outline Simplifying our model 1 Introduction 2 General Methods 3 Simplifying our model 4 Fast non-negative spike inference (FANSI) 5 Particle-filter-smoother (PFS) spike inference 6 PFS results 7 Concluding thoughts Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

30 How can we simplify? Simplifying our model How to simplify Simplification steps 1 Explicitly state the all assumptions of our model 2 Determine which ones are giving us trouble 3 Try approximating them Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

31 Simplifying our model How can we simplify our model? How to simplify Stating our assumptions Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

32 Simplifying our model How can we simplify our model? How to simplify Stating our assumptions Spikes are Poisson Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

33 Simplifying our model How can we simplify our model? How to simplify Stating our assumptions Spikes are Poisson Calcium jumps instantaneously after each spike Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

34 Simplifying our model How can we simplify our model? How to simplify Stating our assumptions Spikes are Poisson Calcium jumps instantaneously after each spike Calcium decays mono-exponentially Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

35 Simplifying our model How can we simplify our model? How to simplify Stating our assumptions Spikes are Poisson Calcium jumps instantaneously after each spike Calcium decays mono-exponentially Calcium jumps the same size with each spike Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

36 Simplifying our model How can we simplify our model? How to simplify Stating our assumptions Spikes are Poisson Calcium jumps instantaneously after each spike Calcium decays mono-exponentially Calcium jumps the same size with each spike There are no other sources of fluctuations in calcium Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

37 Simplifying our model How can we simplify our model? How to simplify Stating our assumptions Spikes are Poisson Calcium jumps instantaneously after each spike Calcium decays mono-exponentially Calcium jumps the same size with each spike There are no other sources of fluctuations in calcium Observations are a linear-gaussian function of calcium Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

38 Simplifying our model How can we simplify our model? How to simplify Stating our assumptions Spikes are Poisson Calcium jumps instantaneously after each spike Calcium decays mono-exponentially Calcium jumps the same size with each spike There are no other sources of fluctuations in calcium Observations are a linear-gaussian function of calcium Which of these assumptions screws us? Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

39 Simplifying our model How can we simplify our model? How to simplify Stating our assumptions Spikes are Poisson Calcium jumps instantaneously after each spike Calcium decays mono-exponentially Calcium jumps the same size with each spike There are no other sources of fluctuations in calcium Observations are a linear-gaussian function of calcium Which of these assumptions screws us? Poisson spikes! Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

40 Simplifying our model How to simplify How can we approximate Poisson spikes? Some reasonable options Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

41 Simplifying our model How to simplify How can we approximate Poisson spikes? Some reasonable options 1 As 0, Poisson Bernoulli Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

42 Simplifying our model How to simplify How can we approximate Poisson spikes? Some reasonable options 1 As 0, Poisson Bernoulli 2 When Poisson rate is low, Poisson Exponential Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

43 Simplifying our model How to simplify How can we approximate Poisson spikes? Some reasonable options 1 As 0, Poisson Bernoulli 2 When Poisson rate is low, Poisson Exponential 3 When Poisson rate is high, Poisson Gaussian Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

44 Simplifying our model How to simplify How can we approximate Poisson spikes? Some reasonable options 1 As 0, Poisson Bernoulli 2 When Poisson rate is low, Poisson Exponential 3 When Poisson rate is high, Poisson Gaussian Slow rate Poisson Fast rate Poisson poisson exponential gaussian Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

45 Simplifying our model Approximations to our model A closer look at the problem at hand Rigorously stating the problem to solve n MAP = argmax n P θ (n Y) = argmax P θ ( Y n)p θ (n) n Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

46 Simplifying our model Approximations to our model A closer look at the problem at hand Rigorously stating the problem to solve n MAP = argmax n = argmax n t t P θ (n Y) = argmax P θ ( Y n)p θ (n) n T t=1 P θ (Y t n t )P θ (n t ) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

47 Simplifying our model Approximations to our model A closer look at the problem at hand Rigorously stating the problem to solve n MAP = argmax n = argmax n t t = argmax n t t P θ (n Y) = argmax P θ ( Y n)p θ (n) n T t=1 P θ (Y t n t )P θ (n t ) T log P θ (Y t C t ) + log P θ (n t ) t=1 Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

48 Simplifying our model Approximations to our model A closer look at the problem at hand Rigorously stating the problem to solve n MAP = argmax n = argmax n t t = argmax n t t = argmax n t t P θ (n Y) = argmax P θ ( Y n)p θ (n) n T t=1 P θ (Y t n t )P θ (n t ) T log P θ (Y t C t ) + log P θ (n t ) t=1 T log N ( Y t αc t 1β, σy 2 I) + log P θ (n t ) t=1 Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

49 Simplifying our model Approximations to our model A closer look at the problem at hand Rigorously stating the problem to solve n MAP = argmax n = argmax n t t = argmax n t t = argmax n t t = argmax n t t P θ (n Y) = argmax P θ ( Y n)p θ (n) n T t=1 P θ (Y t n t )P θ (n t ) T log P θ (Y t C t ) + log P θ (n t ) t=1 T log N ( Y t αc t 1β, σy 2 I) + log P θ (n t ) t=1 T 1 2σ 2 t=1 Y Y t αc t 1β 2 + log P θ (n t ) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

50 Simplifying our model Approximations to our model Writing the prior in terms of C t Rewriting the calcium dynamics τ C t C t 1 = C t 1 + n t 1 C t = γc t 1 + n t 1 n t 1 = (C t γc t 1 ) n = MC Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

51 Simplifying our model Approximations to our model Writing the prior in terms of C t Rewriting the calcium dynamics τ C t C t 1 = C t 1 + n t 1 C t = γc t 1 + n t 1 n t 1 = (C t γc t 1 ) n = MC Rewriting the optimization in terms of C t C MAP = argmax C t t T 1 2σ 2 t=1 Y Y t αc t 1β 2 + log P θ (C t γc t 1 ) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

52 Simplifying our model Gaussian approximation A closer look at the Gaussian approximation Gaussian approximation of P θ (n) C MAP = argmax C t t argmax C t t = argmax C t t T 1 2σ 2 t=1 Y T 1 2σ 2 t=1 Y T 1 2σ 2 t=1 Y Y t αc t 1β Y t αc t 1β Y t αc t 1β 2 + log P θ (n t ) 2 + log N (n t ; λ, λ ) (λ ) 2 (C t γc t 1 λ ) 2 Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

53 Simplifying our model Gaussian approximation A closer look at the Gaussian approximation Gaussian approximation of P θ (n) C MAP = argmax C t t T 1 2σ 2 t=1 Y Y t αc t 1β Thoughts on the Gaussian approximation We have a quadratic problem It is, in fact, a Wiener filter We can solve this in O(T ) 2 + Spikes are not constrained to be non-negative Spikes are not constrained to be integers 1 2(λ ) 2 (C t γc t 1 λ ) 2 Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

54 Simplifying our model Gaussian approximation A closer look at the Gaussian approximation Wiener filter for slow and fast firing rate simulated neurons Slow Firing Rate Fast Firing Rate 1 Fluorescence Wiener Filter Time (sec) Time (sec) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

55 Outline Fast non-negative spike inference (FANSI) 1 Introduction 2 General Methods 3 Simplifying our model 4 Fast non-negative spike inference (FANSI) 5 Particle-filter-smoother (PFS) spike inference 6 PFS results 7 Concluding thoughts Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

56 Fast non-negative spike inference (FANSI) Main idea A closer look at the exponential approximation Exponential approximation of P θ (n) C MAP = argmax C t γc t 1 0 t argmax C t γc t 1 0 t = argmax C t γc t 1 0 t T 1 2σ 2 t=1 Y T 1 2σ 2 t=1 Y T 1 2σ 2 t=1 Y Y t αc t 1β Y t αc t 1β Y t αc t 1β 2 + log P θ (n t ) 2 + log Exp(n t ; λ ) 2 λ (C t γc t 1 ) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

57 Fast non-negative spike inference (FANSI) Main idea A closer look at the exponential approximation Exponential approximation of P θ (n) C MAP = argmax C t γc t 1 0 t T 1 2σ 2 t=1 Y Y t αc t 1β Thoughts on the exponential approximation We have a concave (but not quadratic) problem We can solve this in O(T ) Spikes are constrained to be non-negative Spikes are not constrained to be integers 2 λ (C t γc t 1 ) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

58 Fast non-negative spike inference (FANSI) Main idea A closer look at the exponential approximation FANSI for slow and fast firing rate simulated neurons Slow Firing Rate Fast Firing Rate Fluorescence 1 Fast Filter Time (sec) Time (sec) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

59 Fast non-negative spike inference (FANSI) Algorithmic details FANSI algorithm for solving non-negative state-space problems Direct method: optimize C MAP directly C MAP = argmax C t γc t 1 0 t = argmax C t γc t 1 0 t T 1 2σ 2 t=1 Y 1 2σ 2 Y Y t αc t 1β Y t αc t 1β Barrier method: optimize C z iteratively C z = argmax C t t 2σ 2 Y 2 λ (C t γc t 1 ) 2 λ (MC) 1 1 Y t αc t 1β 2 λ (MC) 1 + z log(mc) 1 Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

60 Fast non-negative spike inference (FANSI) Algorithmic details Finding C z It s concave, so we just use Newton-Raphson L z = 1 2σ 2 Y Y t αc t 1β 2 λ (MC) 1 + z log(mc) 1 g = α σy 2 ( Y αc 1β) + λ M 1 zm (MC z ) 1 H = α α σy 2 + zm (MC z ) 2 M C z C z + sd Hd = g d = H\g Some thoughts Because M is bidiagonal, H is tridiagonal This means that we can use Gaussian elimination Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

61 Fast non-negative spike inference (FANSI) Estimating the parameters Algorithmic details Pseudo-EM EM requires sufficient statistics that we don t compute However, we can estimate θ = argmax P( Y n, θ)p(n θ)dn argmax P( Y n, θ)p( n θ) θ Θ θ Θ We iterate finding n and θ The likelihood for the parameters is concave In practice, we can always find good parameters, without modifying the initial values, with relatively little data Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

62 Fast non-negative spike inference (FANSI) Using our FANSI filter Algorithmic details in vivo data Fluorescence Projection Fast Filter Time (sec) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

63 Fast non-negative spike inference (FANSI) Summary of results so far Discussion Fast non-negative spike inference (FANSI) Due to state-space nature, requires only O(T ) Outperforms Wiener filter for both fast and slow spiking neurons Works for in vivo data Parameter estimation is simple and straightforward (and unsupervised) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

64 Fast non-negative spike inference (FANSI) Straightforward generalizations Discussion Relaxing various assumptions, but staying within the state-space framework Time varying rate more flexible prior Slow rise time add another C with different dynamics Poisson observations likelihood maintains concavity (unlike Poisson dynamics) Optimal thresholding initialize an integer programming algorithm with n Multiple neurons even overlapping spatial filters is ok Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

65 Fast non-negative spike inference (FANSI) Discussion Not straightforward generalizations Desirata not easily incorporated into our FANSI framework Non-Poisson spiking eg, spike history dependence Non-linear observations fluorescence saturates Errorbars FANSI only provides a MAP estimate (which is not conducive to a Laplace approximation) Coupling between neurons an extension of spike history dependence Better parameter estimation requires sufficient stats not available from FANSI Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

66 Fast non-negative spike inference (FANSI) Discussion So, what can we do, to achieve these desirata? Remember this slide? Simplify the assumptions to get something tractable Monte Carlo sample from the model, to approximate the distributions We already did the simplify option, let s try the Monte Carlo option. Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

67 Particle-filter-smoother (PFS) spike inference Outline 1 Introduction 2 General Methods 3 Simplifying our model 4 Fast non-negative spike inference (FANSI) 5 Particle-filter-smoother (PFS) spike inference 6 PFS results 7 Concluding thoughts Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

68 Particle-filter-smoother (PFS) spike inference Remember the state-space formalism? General state-space formalism C t = γc t 1 + n t, Y t = αc t + ɛ t, n t Possion(λ ) ( ɛ t N αβ, σy 2 ) I Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

69 Particle-filter-smoother (PFS) spike inference Remember the state-space formalism? General state-space formalism C t = γc t 1 + n t, Y t = αc t + ɛ t, n t Possion(λ ) ( ɛ t N αβ, σy 2 ) I So, what does sampling mean here? Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

70 Particle-filter-smoother (PFS) spike inference Remember the state-space formalism? General state-space formalism C t = γc t 1 + n t, Y t = αc t + ɛ t, n t Possion(λ ) ( ɛ t N αβ, σy 2 ) I So, what does sampling mean here? Sample spike trains! Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

71 Particle-filter-smoother (PFS) spike inference Remember the state-space formalism? General state-space formalism C t = γc t 1 + n t, Y t = αc t + ɛ t, n t Possion(λ ) ( ɛ t N αβ, σy 2 ) I So, what does sampling mean here? Sample spike trains! How do we do it? Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

72 Particle-filter-smoother (PFS) spike inference What are some ways we can sample spike trains? Three general approaches Naïve too slow: > 2 T possible spike trains Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

73 Particle-filter-smoother (PFS) spike inference What are some ways we can sample spike trains? Three general approaches Naïve too slow: > 2 T possible spike trains Markov Chain Monte Carlo (MCMC) too difficult: because space of spike trains is so non-convex Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

74 Particle-filter-smoother (PFS) spike inference What are some ways we can sample spike trains? Three general approaches Naïve too slow: > 2 T possible spike trains Markov Chain Monte Carlo (MCMC) too difficult: because space of spike trains is so non-convex Sequential Monte Carlo (or particle filter; SMC) just right! Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

75 Particle-filter-smoother (PFS) spike inference What are some ways we can sample spike trains? Three general approaches Naïve too slow: > 2 T possible spike trains Markov Chain Monte Carlo (MCMC) too difficult: because space of spike trains is so non-convex Sequential Monte Carlo (or particle filter; SMC) just right! What s the big idea? Let s say we have a state-space model and we want to infer some statistics about the hidden state what do we do? Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

76 Particle-filter-smoother (PFS) spike inference What are some ways we can sample spike trains? Three general approaches Naïve too slow: > 2 T possible spike trains Markov Chain Monte Carlo (MCMC) too difficult: because space of spike trains is so non-convex Sequential Monte Carlo (or particle filter; SMC) just right! What s the big idea? Let s say we have a state-space model and we want to infer some statistics about the hidden state what do we do? forward-backward algorithm Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

77 Particle-filter-smoother (PFS) spike inference Forward backward algorithms Two familiar special cases Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

78 Particle-filter-smoother (PFS) spike inference Forward backward algorithms Two familiar special cases Discrete state-space models (HMM) remember the Baum-Welch algorithm? Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

79 Particle-filter-smoother (PFS) spike inference Forward backward algorithms Two familiar special cases Discrete state-space models (HMM) remember the Baum-Welch algorithm? Linear-Gaussian dynamics remember the Kalman filter-smooth? Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

80 Particle-filter-smoother (PFS) spike inference Forward backward algorithms how do they work again? Forward recursion First compute the probability of the hidden state, given all previous observations, P θ (H t Y 0:t ) This is called the filter (or forward) distribution Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

81 Particle-filter-smoother (PFS) spike inference Forward backward algorithms how do they work again? Forward recursion First compute the probability of the hidden state, given all previous observations, P θ (H t Y 0:t ) This is called the filter (or forward) distribution Backward recursion Then compute the probability of the hidden state, given all observations (both past and future), P θ (H t Y 0:T ) This is called the smooth (or backward) distribution Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

82 Particle-filter-smoother (PFS) spike inference What are these recursions? Forward recursion P θ (H t Y 0:t ) = 1 Z P θ( Y t H t ) P θ (H t H t 1 )P θ (H t 1 Y 0:t 1 )dh t 1 Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

83 Particle-filter-smoother (PFS) spike inference What are these recursions? Forward recursion P θ (H t Y 0:t ) = 1 Z P θ( Y t H t ) P θ (H t H t 1 )P θ (H t 1 Y 0:t 1 )dh t 1 Backward recursion P θ (H t, H t 1 Y) = P θ (H t Y) P θ (H t H t 1 )P θ (H t 1 Y 0:t 1 ) Pθ (H t H t 1 )P θ (H t 1 Y 0:t 1 )dh t 1 P θ (H t 1 Y) = P θ (H t, H t 1 Y)dH t Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

84 Particle-filter-smoother (PFS) spike inference The forward recursion A closer look at the forward recursion Consider the integral P θ (H t H t 1 )P θ (H t 1 Y 0:t 1 )dh t 1 Under which circumstances can we analytically evaluate that integral? When H t takes finite possible values (ie, for HMM) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

85 Particle-filter-smoother (PFS) spike inference The forward recursion A closer look at the forward recursion Consider the integral P θ (H t H t 1 )P θ (H t 1 Y 0:t 1 )dh t 1 Under which circumstances can we analytically evaluate that integral? When H t takes finite possible values (ie, for HMM) When both distributions are Gaussian, as the product of Gaussians are... Gaussian Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

86 Particle-filter-smoother (PFS) spike inference The forward recursion A closer look at the forward recursion Consider the integral P θ (H t H t 1 )P θ (H t 1 Y 0:t 1 )dh t 1 What can we do to approximate this integral? Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

87 Particle-filter-smoother (PFS) spike inference The forward recursion A closer look at the forward recursion Consider the integral P θ (H t H t 1 )P θ (H t 1 Y 0:t 1 )dh t 1 What can we do to approximate this integral? Discretize on a grid Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

88 Particle-filter-smoother (PFS) spike inference The forward recursion A closer look at the forward recursion Consider the integral P θ (H t H t 1 )P θ (H t 1 Y 0:t 1 )dh t 1 What can we do to approximate this integral? Discretize on a grid Approximation distributions as Gaussians (Laplace approximation) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

89 Particle-filter-smoother (PFS) spike inference The forward recursion A closer look at the forward recursion Consider the integral P θ (H t H t 1 )P θ (H t 1 Y 0:t 1 )dh t 1 What can we do to approximate this integral? Discretize on a grid Approximation distributions as Gaussians (Laplace approximation) Sample! Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

90 Particle-filter-smoother (PFS) spike inference Approximating P θ (H t Y 0:t ) The forward recursion Approximate with a histogram P θ (H t Y 0:t ) N ( ) w (i) t δ H t H (i) t i=1 N particles at each time step w (i) t H (i) t indicates the weight (likelihood) particle i at time t indicates the position of particle i at time t Collectively, they comprise the approximation to our distribution Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

91 Particle-filter-smoother (PFS) spike inference Approximating P θ (H t Y 0:t ) The forward recursion Approximate with a histogram P θ (H t Y 0:t ) N ( ) w (i) t δ H t H (i) t i=1 Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

92 Particle-filter-smoother (PFS) spike inference The forward recursion Substituting this approximation into the integral Given samples, what do we do? P θ (H t Y 0:t ) = 1 Z P θ( Y t H t ) P θ (H t H t 1 )P θ (H t 1 Y 0:t 1 )dh t 1 w (i) t w (i) t = = P θ ( Yt H (i) t P θ ( Yt H (i) t N j=1 w (i) t (j) w t ) N ( P θ j=1 ) P θ ( H (i) t H (i) t H (j) t 1 H (i) t 1 ) w (i) t 1 ) w (j) t 1 Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

93 Particle-filter-smoother (PFS) spike inference The forward recursion But how do we actually get the samples? Importance sampling w (i) t ( ) ( P Yt θ H (i) t P θ ( q H (i) t H (i) t 1 H (i) t ) ) w (i) t 1 q( ) is called the proposal (or importance) distribution (or density) { } q( ) can depend on anything in the past, ie, H (j) N t 1 and Y 0:t j=1 we have various standard options Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

94 Particle-filter-smoother (PFS) spike inference The forward recursion Typical proposal distribution options Transition distribution sampler ( ) If q( ) = P θ H (i) t H (i) t 1, then the computation of the forward distribution is straightforward: ( ) w (i) t = P Yt θ H (i) t w (i) t 1 Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

95 Particle-filter-smoother (PFS) spike inference The forward recursion Typical proposal distribution options Transition distribution sampler ( ) If q( ) = P θ H (i) t H (i) t 1, then the computation of the forward distribution is straightforward: ( ) w (i) t = P Yt θ H (i) t w (i) t 1 One step ahead sampler ( q( ) = P θ H (i) t H (i) t 1, Y ) ( t P θ H (i) t H (i) t 1 ) ( ) P Yt θ H (i) t more efficient than the transition distribution, as it also considers observations Computing the forward distribution no longer simplifies It is sometimes difficult to sample from Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

96 Particle-filter-smoother (PFS) spike inference The forward recursion Typical proposal distribution options One step ahead sampler ( q( ) = P θ H (i) t H (i) t 1, Y ) ( t P θ H (i) t H (i) t 1 ) ( ) P Yt θ H (i) t more efficient than the transition distribution, as it also considers observations Computing the forward distribution no longer simplifies It is sometimes difficult to sample from Optimal sampler q( ) = P θ ( H (i) t H (i) t 1, Y ) ( t:t P θ Uses all available information H (i) t H (i) t 1 ) ( ) P Yt:T θ H (i) t Sometimes (approximately) possible using a backwards recursion Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

97 Particle-filter-smoother (PFS) spike inference The procedure so far The forward recursion For each time step, for each particle 1 Sample from the proposal distribution 2 Update weights 3 Normalize weights Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

98 Particle-filter-smoother (PFS) spike inference The procedure so far The forward recursion For each time step, for each particle 1 Sample from the proposal distribution 2 Update weights 3 Normalize weights Anybody see a problem? Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

99 Particle-filter-smoother (PFS) spike inference The procedure so far The forward recursion For each time step, for each particle 1 Sample from the proposal distribution 2 Update weights 3 Normalize weights Anybody see a problem? Weights degenerate Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

100 Particle-filter-smoother (PFS) spike inference The procedure so far The forward recursion For each time step, for each particle 1 Sample from the proposal distribution 2 Update weights 3 Normalize weights Anybody see a problem? Weights degenerate Draw an example on the board Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

101 Particle-filter-smoother (PFS) spike inference Anybody see a solution The forward recursion Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

102 Particle-filter-smoother (PFS) spike inference Anybody see a solution The forward recursion Resample Sample particles (with replacement) according to their weights If we do it too often, we reduce to our particle diversity If we do it too infrequently, the weights degenerate Thus, it is standard to resample when the effective number of particles is too small: N eff = N ( i=1 w (i) t ) 2 The threshold is typically taken to be N/2 Goal: expectation of resampled distribution should be equal to original distribution Given this constraint, we want to minimize variance Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

103 Particle-filter-smoother (PFS) spike inference Resampling schemes The forward recursion Multinomial resampling Draw particle i with probability w (i) t This is the simplest strategy, but not the best Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

104 Particle-filter-smoother (PFS) spike inference Resampling schemes The forward recursion Multinomial resampling Draw particle i with probability w (i) t This is the simplest strategy, but not the best Stratified resampling Discretize (0, 1] into N equal partitions Sample once from each partition Call cumsum on the weights to generate a cumulative sum of weights Each time a sample falls into the interval for a particular particle, keep that particle This approach has a lower conditional variance than multinomial sampling Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

105 Particle-filter-smoother (PFS) spike inference The forward recursion Putting the whole forward recursion (ie, particle filter) together For each time step, for each particle 1 Sample from the proposal distribution 2 Update weights 3 Normalize weights 4 Stratified resample, if necessary Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

106 Particle-filter-smoother (PFS) spike inference The forward recursion Putting the whole forward recursion (ie, particle filter) together For each time step, for each particle 1 Sample from the proposal distribution 2 Update weights 3 Normalize weights 4 Stratified resample, if necessary Demo the procedure on the board Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

107 Particle-filter-smoother (PFS) spike inference Remember the backward recursion The backward recursion Backward recursion P θ (H t, H t 1 Y) = P θ (H t Y) P θ (H t H t 1 )P θ (H t 1 Y 0:t 1 ) Pθ (H t H t 1 )P θ (H t 1 Y 0:t 1 )dh t 1 P θ (H t 1 Y) = P θ (H t, H t 1 Y)dH t Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

108 Particle-filter-smoother (PFS) spike inference Remember the backward recursion The backward recursion Backward recursion P θ (H t, H t 1 Y) = P θ (H t Y) P θ (H t H t 1 )P θ (H t 1 Y 0:t 1 ) Pθ (H t H t 1 )P θ (H t 1 Y 0:t 1 )dh t 1 P θ (H t 1 Y) = P θ (H t, H t 1 Y)dH t Substituting the results from our forward recursion, obtaining a particle-filter-smoother (PFS) P θ ( H (i) t ) (, H (j) t 1 Y = P θ ( ) P θ H (j) t 1 Y = H (i) t N ( P θ ( Y ) P θ H (i) t H (j) t 1 N j=1 P θ (H (i) t H (j) t 1 ) ( ) P θ H (j) t 1 Y 0:t 1 ) ( ) P θ H (j) t 1 Y 0:t 1 H (i) t, H (j) t 1 Y i=1 Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78 )

109 Particle-filter-smoother (PFS) spike inference Remember the backward recursion The backward recursion Backward recursion P θ (H t, H t 1 Y) = P θ (H t Y) P θ (H t H t 1 )P θ (H t 1 Y 0:t 1 ) Pθ (H t H t 1 )P θ (H t 1 Y 0:t 1 )dh t 1 P θ (H t 1 Y) = P θ (H t, H t 1 Y)dH t Some thoughts on the backwards recursion Plug and chug (ie, does not require computing any new distributions) Scales with O ( N 2 T ) because we need the probability of going from any N particles at time t 1, to any N particles at time t For applications like ours, can really refine our estimates (as most of the information comes from after the spike) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

110 Particle-filter-smoother (PFS) spike inference The backward recursion The whole PFS algorithm for non-analytic state-space models For t = 1,..., T, for each particle Sample from the proposal distribution Update weights Normalize weights Stratified resample, if necessary For t = T,..., 1, for each particle Update backward distribution Things to choose Proposal Number of particles Resampling details Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

111 Particle-filter-smoother (PFS) spike inference PFS algorithm for our model PFS algorithm for our model Our model (slightly modified) n t B(λ ) C t = γc t 1 + An t + σ c ε t, ε t N (0, 1) ( Y t = αc t + ɛ t, ɛ t N αβ, σy 2 ) I The necessary distributions P θ (C t C t 1, n t ) { N (γc t 1 + An t, σ 2 c) if n t = 1 N (γc t 1, σ 2 c) if n t = 0 P θ ( Y t C t ) N (αc t + αβ, σ 2 Y I) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

112 Particle-filter-smoother (PFS) spike inference PFS algorithm for our model One step ahead sampler for our model Defining q( ) ( ) q {C, n} (i) t ( ) ( = P Yt θ C (i) t P θ C (i) t {C t 1, n t } (i)) ( P θ n (i) t ) Procedure First sample spikes, by integrating out C t Then, given spikes, sample C t Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

113 ( Sampling from q Particle-filter-smoother (PFS) spike inference n (i) t ) PFS algorithm for our model by integrating out C t Solving the integral ( q n (i) t ) = = P θ ( = P θ ( = P θ ( ( q {C, n} (i) t n (i) t n (i) t n (i) t ) dc (i) t ) ( ) ( P Yt θ C (i) P θ t C (i) t {C t 1, n t } (i)) dc (i) ) ( ) ( N C (i) t ; µ 1, σ1 2 N C (i) t ; µ 2 (n (i) t ), σ2 2 ) ( N Yt ; µ 3 ({C t 1, n t } (i) ), σ3) 2 t ) dc (i) t So, we compute the probability of n (i) t = 0 and 1 by plugging those values into the above equation, and then we sample from that distribution Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

114 ( Sampling from q Particle-filter-smoother (PFS) spike inference C (i) t ) PFS algorithm for our model Solving the integral ( q C (i) t ) = 1 Z P θ = 1 ( Z N = N ( ) ( Yt C (i) t P θ C (i) C (i) t {C t 1, n t } (i)) ( C (i) ) N t ; µ 4 ( Y t ), σ4 2 ( C (i) t, µ 6 ( Y t, {C t 1, n t } (i) ), σ6 2 t ; µ 5 ({C t 1, n t } (i) ), σ5 2 ) ) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

115 Particle-filter-smoother (PFS) spike inference Weighting the samples PFS algorithm for our model Plug in the proposal distributions ( ( ) P w (i) t = w (i) t 1 P Yt θ C (i) θ C (i) t q ) ( t {C t 1, n t }(i) P θ ( ) ( ) q n (i) t C (i) t n (i) t ) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

116 Particle-filter-smoother (PFS) spike inference PFS algorithm for our model Estimating the parameters in our model The PFS algorithm provides the sufficient statistics for all the parameters Each parameter depends on either P θ (H t Y) or P θ (H t, H t 1 Y), both of which we have from the forward-backward algorithm Parameters are jointly concave, given the inferred hidden distributions In practice, a small number of iterations is required to converge to reasonable answers Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

117 Particle-filter-smoother (PFS) spike inference PFS algorithm for our model Some thoughts on our PFS algorithm Comparison with FANSI Should be more accurate than FANSI, as the approximation is better Provides errorbars (FANSI doesn t) Parameters can be estimated better than FANSI Requires O(N 2 T ) (FANSI only requires O(T ) Can be further generalized Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

118 Outline PFS results 1 Introduction 2 General Methods 3 Simplifying our model 4 Fast non-negative spike inference (FANSI) 5 Particle-filter-smoother (PFS) spike inference 6 PFS results 7 Concluding thoughts Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

119 PFS results Main result Main result using our Particle-Filter-Smoother (PFS) Inferring a spike train from noisy observations Simulated Spike Train Simulated Calcium Simulated Fluorescence Wiener Filter Linear Observation PFS Spike Inference Time (sec) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

120 PFS results Nonlinear observations Incorporating nonlinear observations Modifying the model C t = γc t 1 + C b + An t + σ c εt, ε t N (0, 1) Y t = α[s(c t ) + β] + ξs(c t ) + ηɛ t, ɛ t N (0, I) S(C t ) = C n d t C n d t + k d Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

121 PFS results Nonlinear observations Incorporating nonlinear observations Modifying the model C t = γc t 1 + C b + An t + σ c εt, ε t N (0, 1) Y t = α[s(c t ) + β] + ξs(c t ) + ηɛ t, ɛ t N (0, I) S(C t ) = C n d t C n d t + k d What must change? Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

122 PFS results Nonlinear observations Incorporating nonlinear observations Modifying the model C t = γc t 1 + C b + An t + σ c εt, ε t N (0, 1) Y t = α[s(c t ) + β] + ξs(c t ) + ηɛ t, ɛ t N (0, I) S(C t ) = C n d t C n d t + k d What must change? Observation distribution, P θ ( Y t C t ) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

123 PFS results Nonlinear observations Incorporating nonlinear observations Modifying the model C t = γc t 1 + C b + An t + σ c εt, ε t N (0, 1) Y t = α[s(c t ) + β] + ξs(c t ) + ηɛ t, ɛ t N (0, I) S(C t ) = C n d t C n d t + k d What must change? Observation distribution, P θ ( Y t C t ) Proposal distribution, q( ) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

124 PFS results Nonlinear observations How do we modify the relevant distributions Observation distribution P θ ( Y ( t C t ) = N Yt ; µ 7 (S(C t )), σ 7 (S(C t )) 2) We want a Gaussian function of C t We don t have it, but we compute a Laplacian approximation Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

125 PFS results Nonlinear observations How do we modify the relevant distributions Observation distribution P θ ( Y ( t C t ) = N Yt ; µ 7 (S(C t )), σ 7 (S(C t )) 2) Laplace We want a Gaussian function of C t We don t have it, but we compute a Laplacian approximation Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

126 Nonlinear observation PFS results Nonlinear observations in silico data Simulated Spike Train Simulated Calcium Simulated Fluorescence Wiener Filter Nonlinear Observation PFS Spike Inference Nonlinear Observation PFS [Ca 2+ ] Inference Time (sec) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

127 Nonlinear observation PFS results Nonlinear observations in vitro bursts in vitro Fluorescence Wiener Filter Nonlinear Observation PFS Spike Inference Nonlinear Observation PFS [Ca 2+ ] Inference Time (sec) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

128 Nonlinear observation PFS results Nonlinear observations in vitro spike train in vitro Fluorescence Wiener Filter Nonlinear Observation PFS Spike Inference Nonlinear Observation PFS [Ca 2+ ] Inference Time (sec) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

129 Intermittent observations PFS results Intermittent observations Generalizing the model Y t = α[s(c t ) + β] + ξs(c t ) + ηɛ t, t/d Z Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

130 Intermittent observations PFS results Intermittent observations Generalizing the model Y t = α[s(c t ) + β] + ξs(c t ) + ηɛ t, t/d Z Generalizing the observation distribution P θ ( Y t C t ) = { N (α[s(c t ) + β], ξs(c t ) + η) if t/d Z 1 otherwise Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

131 PFS results One observation ahead sampler Intermittent observations Let v be the time step of the next observation ( q H (i) t ) ( ) ( = P Yv θ H (i) t P θ H (i) t H (i) t 1 ) We can start at time v and use the nonlinear observation sampler Then, we can recursively step backward, using the standard backward recursion Note, however, that the number of possible spike trains is 2 v t Thus, if d >> 1, we have a mixture with too many components Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

132 PFS results Approximating P θ ( Yv H (i) t ) Intermittent observations Reduce from 2 v t to v t + 1 Exact Approximate 0 0 [Ca 2+ ] (A µm) v 4 v 2 v Time Step v 4 v 2 v Time Step Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

133 PFS results Approximating P θ ( Yv H (i) t ) Intermittent observations Reduce from 2 v t to v t + 1 Exact Approximate 0 0 [Ca 2+ ] (A µm) v 4 v 2 v Time Step v 4 v 2 v Time Step Plug in our approximate observation distribution and sample ( q H (i) t ) = P ( ) ( Yv θ H (i) t P θ H (i) t H (i) t 1 ) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

134 PFS results Intermittent observations One observation ahead sampler improves performance One observation ahead sampler Particles Prior sampler Inferred Distributions Weighted Spike History Spike Train Calcium Concentration u Time (sec) v u Time (sec) v One observation ahead sampler Particles Inferred Distributions Weighted Spike History Spike Train Calcium Concentration u Time (sec) v u Time (sec) v Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

135 PFS results Intermittent observations Array of results varying noise and intermittency One observation ahead sampler 20 ms Increasing Frame Rate 40 ms 80 ms 160 ms Time (sec) 2! F 4! F 8! F Increasing Observation Noise Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

136 PFS results Incorporating stimulus and spike history dependence Incorporating stimulus and spike history dependence Generalizing the model Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

137 PFS results Incorporating stimulus and spike history dependence Incorporating stimulus and spike history dependence Generalizing the model P θ (n t ) = B ( n t ; f (b + kx t + ωh t ) ) h t = γ h h t 1 + n t 1 + σ h εt, ε t N (0, 1) Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

138 PFS results Incorporating stimulus and spike history dependence Incorporating stimulus and spike history dependence Generalizing the model P θ (n t ) = B ( n t ; f (b + kx t + ωh t ) ) h t = γ h h t 1 + n t 1 + σ h εt, ε t N (0, 1) Constraints on dynamics Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, / 78

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 11 Project

More information

Sean Escola. Center for Theoretical Neuroscience

Sean Escola. Center for Theoretical Neuroscience Employing hidden Markov models of neural spike-trains toward the improved estimation of linear receptive fields and the decoding of multiple firing regimes Sean Escola Center for Theoretical Neuroscience

More information

STA 414/2104: Machine Learning

STA 414/2104: Machine Learning STA 414/2104: Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistics! rsalakhu@cs.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 9 Sequential Data So far

More information

Machine Learning for OR & FE

Machine Learning for OR & FE Machine Learning for OR & FE Hidden Markov Models Martin Haugh Department of Industrial Engineering and Operations Research Columbia University Email: martin.b.haugh@gmail.com Additional References: David

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning Brown University CSCI 1950-F, Spring 2012 Prof. Erik Sudderth Lecture 25: Markov Chain Monte Carlo (MCMC) Course Review and Advanced Topics Many figures courtesy Kevin

More information

Hidden Markov Models. Aarti Singh Slides courtesy: Eric Xing. Machine Learning / Nov 8, 2010

Hidden Markov Models. Aarti Singh Slides courtesy: Eric Xing. Machine Learning / Nov 8, 2010 Hidden Markov Models Aarti Singh Slides courtesy: Eric Xing Machine Learning 10-701/15-781 Nov 8, 2010 i.i.d to sequential data So far we assumed independent, identically distributed data Sequential data

More information

Computer Intensive Methods in Mathematical Statistics

Computer Intensive Methods in Mathematical Statistics Computer Intensive Methods in Mathematical Statistics Department of mathematics johawes@kth.se Lecture 16 Advanced topics in computational statistics 18 May 2017 Computer Intensive Methods (1) Plan of

More information

Hidden Markov Models. By Parisa Abedi. Slides courtesy: Eric Xing

Hidden Markov Models. By Parisa Abedi. Slides courtesy: Eric Xing Hidden Markov Models By Parisa Abedi Slides courtesy: Eric Xing i.i.d to sequential data So far we assumed independent, identically distributed data Sequential (non i.i.d.) data Time-series data E.g. Speech

More information

A new look at state-space models for neural data

A new look at state-space models for neural data A new look at state-space models for neural data Liam Paninski Department of Statistics and Center for Theoretical Neuroscience Columbia University http://www.stat.columbia.edu/ liam liam@stat.columbia.edu

More information

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA Contents in latter part Linear Dynamical Systems What is different from HMM? Kalman filter Its strength and limitation Particle Filter

More information

CPSC 540: Machine Learning

CPSC 540: Machine Learning CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is

More information

Introduction to Machine Learning CMU-10701

Introduction to Machine Learning CMU-10701 Introduction to Machine Learning CMU-10701 Hidden Markov Models Barnabás Póczos & Aarti Singh Slides courtesy: Eric Xing i.i.d to sequential data So far we assumed independent, identically distributed

More information

The Kalman Filter ImPr Talk

The Kalman Filter ImPr Talk The Kalman Filter ImPr Talk Ged Ridgway Centre for Medical Image Computing November, 2006 Outline What is the Kalman Filter? State Space Models Kalman Filter Overview Bayesian Updating of Estimates Kalman

More information

Bayesian Methods for Machine Learning

Bayesian Methods for Machine Learning Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),

More information

Sequential Monte Carlo Methods for Bayesian Computation

Sequential Monte Carlo Methods for Bayesian Computation Sequential Monte Carlo Methods for Bayesian Computation A. Doucet Kyoto Sept. 2012 A. Doucet (MLSS Sept. 2012) Sept. 2012 1 / 136 Motivating Example 1: Generic Bayesian Model Let X be a vector parameter

More information

Hidden Markov Models Part 2: Algorithms

Hidden Markov Models Part 2: Algorithms Hidden Markov Models Part 2: Algorithms CSE 6363 Machine Learning Vassilis Athitsos Computer Science and Engineering Department University of Texas at Arlington 1 Hidden Markov Model An HMM consists of:

More information

Computer Intensive Methods in Mathematical Statistics

Computer Intensive Methods in Mathematical Statistics Computer Intensive Methods in Mathematical Statistics Department of mathematics johawes@kth.se Lecture 7 Sequential Monte Carlo methods III 7 April 2017 Computer Intensive Methods (1) Plan of today s lecture

More information

ECO 513 Fall 2008 C.Sims KALMAN FILTER. s t = As t 1 + ε t Measurement equation : y t = Hs t + ν t. u t = r t. u 0 0 t 1 + y t = [ H I ] u t.

ECO 513 Fall 2008 C.Sims KALMAN FILTER. s t = As t 1 + ε t Measurement equation : y t = Hs t + ν t. u t = r t. u 0 0 t 1 + y t = [ H I ] u t. ECO 513 Fall 2008 C.Sims KALMAN FILTER Model in the form 1. THE KALMAN FILTER Plant equation : s t = As t 1 + ε t Measurement equation : y t = Hs t + ν t. Var(ε t ) = Ω, Var(ν t ) = Ξ. ε t ν t and (ε t,

More information

Why do we care? Measurements. Handling uncertainty over time: predicting, estimating, recognizing, learning. Dealing with time

Why do we care? Measurements. Handling uncertainty over time: predicting, estimating, recognizing, learning. Dealing with time Handling uncertainty over time: predicting, estimating, recognizing, learning Chris Atkeson 2004 Why do we care? Speech recognition makes use of dependence of words and phonemes across time. Knowing where

More information

Linear Dynamical Systems

Linear Dynamical Systems Linear Dynamical Systems Sargur N. srihari@cedar.buffalo.edu Machine Learning Course: http://www.cedar.buffalo.edu/~srihari/cse574/index.html Two Models Described by Same Graph Latent variables Observations

More information

An introduction to Sequential Monte Carlo

An introduction to Sequential Monte Carlo An introduction to Sequential Monte Carlo Thang Bui Jes Frellsen Department of Engineering University of Cambridge Research and Communication Club 6 February 2014 1 Sequential Monte Carlo (SMC) methods

More information

Lecture 2: From Linear Regression to Kalman Filter and Beyond

Lecture 2: From Linear Regression to Kalman Filter and Beyond Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing

More information

Sequential Monte Carlo and Particle Filtering. Frank Wood Gatsby, November 2007

Sequential Monte Carlo and Particle Filtering. Frank Wood Gatsby, November 2007 Sequential Monte Carlo and Particle Filtering Frank Wood Gatsby, November 2007 Importance Sampling Recall: Let s say that we want to compute some expectation (integral) E p [f] = p(x)f(x)dx and we remember

More information

Pattern Recognition and Machine Learning

Pattern Recognition and Machine Learning Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability

More information

Sequence labeling. Taking collective a set of interrelated instances x 1,, x T and jointly labeling them

Sequence labeling. Taking collective a set of interrelated instances x 1,, x T and jointly labeling them HMM, MEMM and CRF 40-957 Special opics in Artificial Intelligence: Probabilistic Graphical Models Sharif University of echnology Soleymani Spring 2014 Sequence labeling aking collective a set of interrelated

More information

STA414/2104 Statistical Methods for Machine Learning II

STA414/2104 Statistical Methods for Machine Learning II STA414/2104 Statistical Methods for Machine Learning II Murat A. Erdogdu & David Duvenaud Department of Computer Science Department of Statistical Sciences Lecture 3 Slide credits: Russ Salakhutdinov Announcements

More information

Why do we care? Examples. Bayes Rule. What room am I in? Handling uncertainty over time: predicting, estimating, recognizing, learning

Why do we care? Examples. Bayes Rule. What room am I in? Handling uncertainty over time: predicting, estimating, recognizing, learning Handling uncertainty over time: predicting, estimating, recognizing, learning Chris Atkeson 004 Why do we care? Speech recognition makes use of dependence of words and phonemes across time. Knowing where

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 7 Approximate

More information

Auxiliary Particle Methods

Auxiliary Particle Methods Auxiliary Particle Methods Perspectives & Applications Adam M. Johansen 1 adam.johansen@bristol.ac.uk Oxford University Man Institute 29th May 2008 1 Collaborators include: Arnaud Doucet, Nick Whiteley

More information

Probabilistic Graphical Models

Probabilistic Graphical Models Probabilistic Graphical Models Brown University CSCI 2950-P, Spring 2013 Prof. Erik Sudderth Lecture 12: Gaussian Belief Propagation, State Space Models and Kalman Filters Guest Kalman Filter Lecture by

More information

CPSC 540: Machine Learning

CPSC 540: Machine Learning CPSC 540: Machine Learning Expectation Maximization Mark Schmidt University of British Columbia Winter 2018 Last Time: Learning with MAR Values We discussed learning with missing at random values in data:

More information

An Brief Overview of Particle Filtering

An Brief Overview of Particle Filtering 1 An Brief Overview of Particle Filtering Adam M. Johansen a.m.johansen@warwick.ac.uk www2.warwick.ac.uk/fac/sci/statistics/staff/academic/johansen/talks/ May 11th, 2010 Warwick University Centre for Systems

More information

STA 4273H: Sta-s-cal Machine Learning

STA 4273H: Sta-s-cal Machine Learning STA 4273H: Sta-s-cal Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 2 In our

More information

Part 1: Expectation Propagation

Part 1: Expectation Propagation Chalmers Machine Learning Summer School Approximate message passing and biomedicine Part 1: Expectation Propagation Tom Heskes Machine Learning Group, Institute for Computing and Information Sciences Radboud

More information

Maximum Likelihood (ML), Expectation Maximization (EM) Pieter Abbeel UC Berkeley EECS

Maximum Likelihood (ML), Expectation Maximization (EM) Pieter Abbeel UC Berkeley EECS Maximum Likelihood (ML), Expectation Maximization (EM) Pieter Abbeel UC Berkeley EECS Many slides adapted from Thrun, Burgard and Fox, Probabilistic Robotics Outline Maximum likelihood (ML) Priors, and

More information

State Space and Hidden Markov Models

State Space and Hidden Markov Models State Space and Hidden Markov Models Kunsch H.R. State Space and Hidden Markov Models. ETH- Zurich Zurich; Aliaksandr Hubin Oslo 2014 Contents 1. Introduction 2. Markov Chains 3. Hidden Markov and State

More information

STA414/2104. Lecture 11: Gaussian Processes. Department of Statistics

STA414/2104. Lecture 11: Gaussian Processes. Department of Statistics STA414/2104 Lecture 11: Gaussian Processes Department of Statistics www.utstat.utoronto.ca Delivered by Mark Ebden with thanks to Russ Salakhutdinov Outline Gaussian Processes Exam review Course evaluations

More information

Kalman filtering and friends: Inference in time series models. Herke van Hoof slides mostly by Michael Rubinstein

Kalman filtering and friends: Inference in time series models. Herke van Hoof slides mostly by Michael Rubinstein Kalman filtering and friends: Inference in time series models Herke van Hoof slides mostly by Michael Rubinstein Problem overview Goal Estimate most probable state at time k using measurement up to time

More information

Machine Learning & Data Mining Caltech CS/CNS/EE 155 Hidden Markov Models Last Updated: Feb 7th, 2017

Machine Learning & Data Mining Caltech CS/CNS/EE 155 Hidden Markov Models Last Updated: Feb 7th, 2017 1 Introduction Let x = (x 1,..., x M ) denote a sequence (e.g. a sequence of words), and let y = (y 1,..., y M ) denote a corresponding hidden sequence that we believe explains or influences x somehow

More information

Parametric Inference Maximum Likelihood Inference Exponential Families Expectation Maximization (EM) Bayesian Inference Statistical Decison Theory

Parametric Inference Maximum Likelihood Inference Exponential Families Expectation Maximization (EM) Bayesian Inference Statistical Decison Theory Statistical Inference Parametric Inference Maximum Likelihood Inference Exponential Families Expectation Maximization (EM) Bayesian Inference Statistical Decison Theory IP, José Bioucas Dias, IST, 2007

More information

Lecture: Gaussian Process Regression. STAT 6474 Instructor: Hongxiao Zhu

Lecture: Gaussian Process Regression. STAT 6474 Instructor: Hongxiao Zhu Lecture: Gaussian Process Regression STAT 6474 Instructor: Hongxiao Zhu Motivation Reference: Marc Deisenroth s tutorial on Robot Learning. 2 Fast Learning for Autonomous Robots with Gaussian Processes

More information

Introduction to Machine Learning Midterm, Tues April 8

Introduction to Machine Learning Midterm, Tues April 8 Introduction to Machine Learning 10-701 Midterm, Tues April 8 [1 point] Name: Andrew ID: Instructions: You are allowed a (two-sided) sheet of notes. Exam ends at 2:45pm Take a deep breath and don t spend

More information

Answers and expectations

Answers and expectations Answers and expectations For a function f(x) and distribution P(x), the expectation of f with respect to P is The expectation is the average of f, when x is drawn from the probability distribution P E

More information

Learning Energy-Based Models of High-Dimensional Data

Learning Energy-Based Models of High-Dimensional Data Learning Energy-Based Models of High-Dimensional Data Geoffrey Hinton Max Welling Yee-Whye Teh Simon Osindero www.cs.toronto.edu/~hinton/energybasedmodelsweb.htm Discovering causal structure as a goal

More information

Advanced Computational Methods in Statistics: Lecture 5 Sequential Monte Carlo/Particle Filtering

Advanced Computational Methods in Statistics: Lecture 5 Sequential Monte Carlo/Particle Filtering Advanced Computational Methods in Statistics: Lecture 5 Sequential Monte Carlo/Particle Filtering Axel Gandy Department of Mathematics Imperial College London http://www2.imperial.ac.uk/~agandy London

More information

+ + ( + ) = Linear recurrent networks. Simpler, much more amenable to analytic treatment E.g. by choosing

+ + ( + ) = Linear recurrent networks. Simpler, much more amenable to analytic treatment E.g. by choosing Linear recurrent networks Simpler, much more amenable to analytic treatment E.g. by choosing + ( + ) = Firing rates can be negative Approximates dynamics around fixed point Approximation often reasonable

More information

Note Set 5: Hidden Markov Models

Note Set 5: Hidden Markov Models Note Set 5: Hidden Markov Models Probabilistic Learning: Theory and Algorithms, CS 274A, Winter 2016 1 Hidden Markov Models (HMMs) 1.1 Introduction Consider observed data vectors x t that are d-dimensional

More information

Math 350: An exploration of HMMs through doodles.

Math 350: An exploration of HMMs through doodles. Math 350: An exploration of HMMs through doodles. Joshua Little (407673) 19 December 2012 1 Background 1.1 Hidden Markov models. Markov chains (MCs) work well for modelling discrete-time processes, or

More information

Unsupervised Learning

Unsupervised Learning Unsupervised Learning Bayesian Model Comparison Zoubin Ghahramani zoubin@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc in Intelligent Systems, Dept Computer Science University College

More information

Approximate Inference

Approximate Inference Approximate Inference Simulation has a name: sampling Sampling is a hot topic in machine learning, and it s really simple Basic idea: Draw N samples from a sampling distribution S Compute an approximate

More information

GWAS V: Gaussian processes

GWAS V: Gaussian processes GWAS V: Gaussian processes Dr. Oliver Stegle Christoh Lippert Prof. Dr. Karsten Borgwardt Max-Planck-Institutes Tübingen, Germany Tübingen Summer 2011 Oliver Stegle GWAS V: Gaussian processes Summer 2011

More information

Particle Filters. Pieter Abbeel UC Berkeley EECS. Many slides adapted from Thrun, Burgard and Fox, Probabilistic Robotics

Particle Filters. Pieter Abbeel UC Berkeley EECS. Many slides adapted from Thrun, Burgard and Fox, Probabilistic Robotics Particle Filters Pieter Abbeel UC Berkeley EECS Many slides adapted from Thrun, Burgard and Fox, Probabilistic Robotics Motivation For continuous spaces: often no analytical formulas for Bayes filter updates

More information

Particle Filters. Outline

Particle Filters. Outline Particle Filters M. Sami Fadali Professor of EE University of Nevada Outline Monte Carlo integration. Particle filter. Importance sampling. Degeneracy Resampling Example. 1 2 Monte Carlo Integration Numerical

More information

The 1d Kalman Filter. 1 Understanding the forward model. Richard Turner

The 1d Kalman Filter. 1 Understanding the forward model. Richard Turner The d Kalman Filter Richard Turner This is a Jekyll and Hyde of a document and should really be split up. We start with Jekyll which contains a very short derivation for the d Kalman filter, the purpose

More information

Approximate inference in Energy-Based Models

Approximate inference in Energy-Based Models CSC 2535: 2013 Lecture 3b Approximate inference in Energy-Based Models Geoffrey Hinton Two types of density model Stochastic generative model using directed acyclic graph (e.g. Bayes Net) Energy-based

More information

Sequential Monte Carlo Methods (for DSGE Models)

Sequential Monte Carlo Methods (for DSGE Models) Sequential Monte Carlo Methods (for DSGE Models) Frank Schorfheide University of Pennsylvania, PIER, CEPR, and NBER October 23, 2017 Some References These lectures use material from our joint work: Tempered

More information

Introduction to Machine Learning CMU-10701

Introduction to Machine Learning CMU-10701 Introduction to Machine Learning CMU-10701 Markov Chain Monte Carlo Methods Barnabás Póczos & Aarti Singh Contents Markov Chain Monte Carlo Methods Goal & Motivation Sampling Rejection Importance Markov

More information

AUTOMOTIVE ENVIRONMENT SENSORS

AUTOMOTIVE ENVIRONMENT SENSORS AUTOMOTIVE ENVIRONMENT SENSORS Lecture 5. Localization BME KÖZLEKEDÉSMÉRNÖKI ÉS JÁRMŰMÉRNÖKI KAR 32708-2/2017/INTFIN SZÁMÚ EMMI ÁLTAL TÁMOGATOTT TANANYAG Related concepts Concepts related to vehicles moving

More information

Pattern Recognition and Machine Learning. Bishop Chapter 11: Sampling Methods

Pattern Recognition and Machine Learning. Bishop Chapter 11: Sampling Methods Pattern Recognition and Machine Learning Chapter 11: Sampling Methods Elise Arnaud Jakob Verbeek May 22, 2008 Outline of the chapter 11.1 Basic Sampling Algorithms 11.2 Markov Chain Monte Carlo 11.3 Gibbs

More information

(5) Multi-parameter models - Gibbs sampling. ST440/540: Applied Bayesian Analysis

(5) Multi-parameter models - Gibbs sampling. ST440/540: Applied Bayesian Analysis Summarizing a posterior Given the data and prior the posterior is determined Summarizing the posterior gives parameter estimates, intervals, and hypothesis tests Most of these computations are integrals

More information

Online appendix to On the stability of the excess sensitivity of aggregate consumption growth in the US

Online appendix to On the stability of the excess sensitivity of aggregate consumption growth in the US Online appendix to On the stability of the excess sensitivity of aggregate consumption growth in the US Gerdie Everaert 1, Lorenzo Pozzi 2, and Ruben Schoonackers 3 1 Ghent University & SHERPPA 2 Erasmus

More information

Lecture 2: From Linear Regression to Kalman Filter and Beyond

Lecture 2: From Linear Regression to Kalman Filter and Beyond Lecture 2: From Linear Regression to Kalman Filter and Beyond Department of Biomedical Engineering and Computational Science Aalto University January 26, 2012 Contents 1 Batch and Recursive Estimation

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear

More information

Bayesian Calibration of Simulators with Structured Discretization Uncertainty

Bayesian Calibration of Simulators with Structured Discretization Uncertainty Bayesian Calibration of Simulators with Structured Discretization Uncertainty Oksana A. Chkrebtii Department of Statistics, The Ohio State University Joint work with Matthew T. Pratola (Statistics, The

More information

Approximate Bayesian Computation and Particle Filters

Approximate Bayesian Computation and Particle Filters Approximate Bayesian Computation and Particle Filters Dennis Prangle Reading University 5th February 2014 Introduction Talk is mostly a literature review A few comments on my own ongoing research See Jasra

More information

Announcements. CS 188: Artificial Intelligence Fall VPI Example. VPI Properties. Reasoning over Time. Markov Models. Lecture 19: HMMs 11/4/2008

Announcements. CS 188: Artificial Intelligence Fall VPI Example. VPI Properties. Reasoning over Time. Markov Models. Lecture 19: HMMs 11/4/2008 CS 88: Artificial Intelligence Fall 28 Lecture 9: HMMs /4/28 Announcements Midterm solutions up, submit regrade requests within a week Midterm course evaluation up on web, please fill out! Dan Klein UC

More information

Pattern Recognition and Machine Learning. Bishop Chapter 2: Probability Distributions

Pattern Recognition and Machine Learning. Bishop Chapter 2: Probability Distributions Pattern Recognition and Machine Learning Chapter 2: Probability Distributions Cécile Amblard Alex Kläser Jakob Verbeek October 11, 27 Probability Distributions: General Density Estimation: given a finite

More information

Exercises Tutorial at ICASSP 2016 Learning Nonlinear Dynamical Models Using Particle Filters

Exercises Tutorial at ICASSP 2016 Learning Nonlinear Dynamical Models Using Particle Filters Exercises Tutorial at ICASSP 216 Learning Nonlinear Dynamical Models Using Particle Filters Andreas Svensson, Johan Dahlin and Thomas B. Schön March 18, 216 Good luck! 1 [Bootstrap particle filter for

More information

Variational Autoencoder

Variational Autoencoder Variational Autoencoder Göker Erdo gan August 8, 2017 The variational autoencoder (VA) [1] is a nonlinear latent variable model with an efficient gradient-based training procedure based on variational

More information

Nonparametric Bayesian Methods (Gaussian Processes)

Nonparametric Bayesian Methods (Gaussian Processes) [70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent

More information

The Particle Filter. PD Dr. Rudolph Triebel Computer Vision Group. Machine Learning for Computer Vision

The Particle Filter. PD Dr. Rudolph Triebel Computer Vision Group. Machine Learning for Computer Vision The Particle Filter Non-parametric implementation of Bayes filter Represents the belief (posterior) random state samples. by a set of This representation is approximate. Can represent distributions that

More information

Lecture 6: Bayesian Inference in SDE Models

Lecture 6: Bayesian Inference in SDE Models Lecture 6: Bayesian Inference in SDE Models Bayesian Filtering and Smoothing Point of View Simo Särkkä Aalto University Simo Särkkä (Aalto) Lecture 6: Bayesian Inference in SDEs 1 / 45 Contents 1 SDEs

More information

Linear Dynamical Systems (Kalman filter)

Linear Dynamical Systems (Kalman filter) Linear Dynamical Systems (Kalman filter) (a) Overview of HMMs (b) From HMMs to Linear Dynamical Systems (LDS) 1 Markov Chains with Discrete Random Variables x 1 x 2 x 3 x T Let s assume we have discrete

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate

More information

L11: Pattern recognition principles

L11: Pattern recognition principles L11: Pattern recognition principles Bayesian decision theory Statistical classifiers Dimensionality reduction Clustering This lecture is partly based on [Huang, Acero and Hon, 2001, ch. 4] Introduction

More information

ECE276A: Sensing & Estimation in Robotics Lecture 10: Gaussian Mixture and Particle Filtering

ECE276A: Sensing & Estimation in Robotics Lecture 10: Gaussian Mixture and Particle Filtering ECE276A: Sensing & Estimation in Robotics Lecture 10: Gaussian Mixture and Particle Filtering Lecturer: Nikolay Atanasov: natanasov@ucsd.edu Teaching Assistants: Siwei Guo: s9guo@eng.ucsd.edu Anwesan Pal:

More information

Generative Clustering, Topic Modeling, & Bayesian Inference

Generative Clustering, Topic Modeling, & Bayesian Inference Generative Clustering, Topic Modeling, & Bayesian Inference INFO-4604, Applied Machine Learning University of Colorado Boulder December 12-14, 2017 Prof. Michael Paul Unsupervised Naïve Bayes Last week

More information

Computational statistics

Computational statistics Computational statistics Markov Chain Monte Carlo methods Thierry Denœux March 2017 Thierry Denœux Computational statistics March 2017 1 / 71 Contents of this chapter When a target density f can be evaluated

More information

Gaussian Process Optimization with Mutual Information

Gaussian Process Optimization with Mutual Information Gaussian Process Optimization with Mutual Information Emile Contal 1 Vianney Perchet 2 Nicolas Vayatis 1 1 CMLA Ecole Normale Suprieure de Cachan & CNRS, France 2 LPMA Université Paris Diderot & CNRS,

More information

Reading: Karlin and Taylor Ch. 5 Resnick Ch. 3. A renewal process is a generalization of the Poisson point process.

Reading: Karlin and Taylor Ch. 5 Resnick Ch. 3. A renewal process is a generalization of the Poisson point process. Renewal Processes Wednesday, December 16, 2015 1:02 PM Reading: Karlin and Taylor Ch. 5 Resnick Ch. 3 A renewal process is a generalization of the Poisson point process. The Poisson point process is completely

More information

Towards inference for skewed alpha stable Levy processes

Towards inference for skewed alpha stable Levy processes Towards inference for skewed alpha stable Levy processes Simon Godsill and Tatjana Lemke Signal Processing and Communications Lab. University of Cambridge www-sigproc.eng.cam.ac.uk/~sjg Overview Motivation

More information

Gaussian Process Approximations of Stochastic Differential Equations

Gaussian Process Approximations of Stochastic Differential Equations Gaussian Process Approximations of Stochastic Differential Equations Cédric Archambeau Centre for Computational Statistics and Machine Learning University College London c.archambeau@cs.ucl.ac.uk CSML

More information

Human Pose Tracking I: Basics. David Fleet University of Toronto

Human Pose Tracking I: Basics. David Fleet University of Toronto Human Pose Tracking I: Basics David Fleet University of Toronto CIFAR Summer School, 2009 Looking at People Challenges: Complex pose / motion People have many degrees of freedom, comprising an articulated

More information

Bayesian Computations for DSGE Models

Bayesian Computations for DSGE Models Bayesian Computations for DSGE Models Frank Schorfheide University of Pennsylvania, PIER, CEPR, and NBER October 23, 2017 This Lecture is Based on Bayesian Estimation of DSGE Models Edward P. Herbst &

More information

Based on slides by Richard Zemel

Based on slides by Richard Zemel CSC 412/2506 Winter 2018 Probabilistic Learning and Reasoning Lecture 3: Directed Graphical Models and Latent Variables Based on slides by Richard Zemel Learning outcomes What aspects of a model can we

More information

Graphical Models for Collaborative Filtering

Graphical Models for Collaborative Filtering Graphical Models for Collaborative Filtering Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012 Sequence modeling HMM, Kalman Filter, etc.: Similarity: the same graphical model topology,

More information

Markov Models. CS 188: Artificial Intelligence Fall Example. Mini-Forward Algorithm. Stationary Distributions.

Markov Models. CS 188: Artificial Intelligence Fall Example. Mini-Forward Algorithm. Stationary Distributions. CS 88: Artificial Intelligence Fall 27 Lecture 2: HMMs /6/27 Markov Models A Markov model is a chain-structured BN Each node is identically distributed (stationarity) Value of X at a given time is called

More information

13: Variational inference II

13: Variational inference II 10-708: Probabilistic Graphical Models, Spring 2015 13: Variational inference II Lecturer: Eric P. Xing Scribes: Ronghuo Zheng, Zhiting Hu, Yuntian Deng 1 Introduction We started to talk about variational

More information

A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models

A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models Jeff A. Bilmes (bilmes@cs.berkeley.edu) International Computer Science Institute

More information

Bayesian Hidden Markov Models and Extensions

Bayesian Hidden Markov Models and Extensions Bayesian Hidden Markov Models and Extensions Zoubin Ghahramani Department of Engineering University of Cambridge joint work with Matt Beal, Jurgen van Gael, Yunus Saatci, Tom Stepleton, Yee Whye Teh Modeling

More information

University of Cambridge. MPhil in Computer Speech Text & Internet Technology. Module: Speech Processing II. Lecture 2: Hidden Markov Models I

University of Cambridge. MPhil in Computer Speech Text & Internet Technology. Module: Speech Processing II. Lecture 2: Hidden Markov Models I University of Cambridge MPhil in Computer Speech Text & Internet Technology Module: Speech Processing II Lecture 2: Hidden Markov Models I o o o o o 1 2 3 4 T 1 b 2 () a 12 2 a 3 a 4 5 34 a 23 b () b ()

More information

Doing Bayesian Integrals

Doing Bayesian Integrals ASTR509-13 Doing Bayesian Integrals The Reverend Thomas Bayes (c.1702 1761) Philosopher, theologian, mathematician Presbyterian (non-conformist) minister Tunbridge Wells, UK Elected FRS, perhaps due to

More information

Monte Carlo Methods. Geoff Gordon February 9, 2006

Monte Carlo Methods. Geoff Gordon February 9, 2006 Monte Carlo Methods Geoff Gordon ggordon@cs.cmu.edu February 9, 2006 Numerical integration problem 5 4 3 f(x,y) 2 1 1 0 0.5 0 X 0.5 1 1 0.8 0.6 0.4 Y 0.2 0 0.2 0.4 0.6 0.8 1 x X f(x)dx Used for: function

More information

Particle Filtering for Data-Driven Simulation and Optimization

Particle Filtering for Data-Driven Simulation and Optimization Particle Filtering for Data-Driven Simulation and Optimization John R. Birge The University of Chicago Booth School of Business Includes joint work with Nicholas Polson. JRBirge INFORMS Phoenix, October

More information

Statistical Data Analysis Stat 3: p-values, parameter estimation

Statistical Data Analysis Stat 3: p-values, parameter estimation Statistical Data Analysis Stat 3: p-values, parameter estimation London Postgraduate Lectures on Particle Physics; University of London MSci course PH4515 Glen Cowan Physics Department Royal Holloway,

More information

Lecture 3. G. Cowan. Lecture 3 page 1. Lectures on Statistical Data Analysis

Lecture 3. G. Cowan. Lecture 3 page 1. Lectures on Statistical Data Analysis Lecture 3 1 Probability (90 min.) Definition, Bayes theorem, probability densities and their properties, catalogue of pdfs, Monte Carlo 2 Statistical tests (90 min.) general concepts, test statistics,

More information

Computer Intensive Methods in Mathematical Statistics

Computer Intensive Methods in Mathematical Statistics Computer Intensive Methods in Mathematical Statistics Department of mathematics johawes@kth.se Lecture 5 Sequential Monte Carlo methods I 31 March 2017 Computer Intensive Methods (1) Plan of today s lecture

More information

Infering the Number of State Clusters in Hidden Markov Model and its Extension

Infering the Number of State Clusters in Hidden Markov Model and its Extension Infering the Number of State Clusters in Hidden Markov Model and its Extension Xugang Ye Department of Applied Mathematics and Statistics, Johns Hopkins University Elements of a Hidden Markov Model (HMM)

More information

PROBABILISTIC REASONING OVER TIME

PROBABILISTIC REASONING OVER TIME PROBABILISTIC REASONING OVER TIME In which we try to interpret the present, understand the past, and perhaps predict the future, even when very little is crystal clear. Outline Time and uncertainty Inference:

More information

A new look at state-space models for neural data

A new look at state-space models for neural data A new look at state-space models for neural data Liam Paninski, Yashar Ahmadian, Daniel Gil Ferreira, Shinsuke Koyama, Kamiar Rahnama Rad, Michael Vidne, Joshua Vogelstein, Wei Wu Department of Statistics

More information