Bayesian Inference with Oscillator Models: A Possible Role of Neural Rhythms

Transcription

1 Bayesian Inference with Oscillator Models: A Possible Role of Neural Rhythms Qualcomm/Brain Corporation/INC Lecture Series on Computational Neuroscience University of California, San Diego, March 5, 2012 Prashant G. Mehta Department of Mechanical Science and Engineering and the Coordinated Science Laboratory University of Illinois at Urbana-Champaign Research supported by NSF and AFOSR

2 Application 2 Gait Cycle Biological Rhythm

3 Application 2 Gait Cycle Biological Rhythm

4 Application 2 Gait Cycle Biological Rhythm

5 Application 2 Gait Cycle Biological Rhythm

6 Application 2 Gait Cycle Biological Rhythm

7 Application 2 Gait Cycle Biological Rhythm

8 Application 2 Gait Cycle Biological Rhythm

9 Application Application: Ankle-foot Orthoses Estimation of gait cycle using sensor measurements Ankle-foot orthoses (AFOs) : For lower-limb neuromuscular impairments. Provides dorsiflexor (toe lift) and plantarflexor (toe push) torque assistance Solenoid valves: control the flow of CO2 to the actuator Actuator Compressed CO2 Sensors: heel, toe, and ankle joint AFO system components: Power supply, Valves, Actuator, Sensors. Professor Liz Hsiao-Wecksler Acknowledgement: Professor Liz Hsiao-Wecksler for sharing the AFO device picture and sensor data. 3

10 Application Application: Ankle-foot Orthoses Estimation of gait cycle using sensor measurements Ankle-foot orthoses (AFOs) : For lower-limb neuromuscular impairments. Provides dorsiflexor (toe lift) and plantarflexor (toe push) torque assistance Solenoid valves: control the flow of CO2 to the actuator Actuator Compressed CO2 Sensors: heel, toe, and ankle joint AFO system components: Power supply, Valves, Actuator, Sensors. Professor Liz Hsiao-Wecksler Acknowledgement: Professor Liz Hsiao-Wecksler for sharing the AFO device picture and sensor data. 3

11 Application 4 Gait Cycle Signal model Stance phase Swing phase Model (Noisy oscillator) θ(t) = ω }{{} 0 + noise natural frequency

12 Application 4 Gait Cycle Signal model Stance phase Swing phase Model (Noisy oscillator) θ(t) = ω }{{} 0 + noise natural frequency

13 Application 4 Gait Cycle Signal model Stance phase Swing phase Model (Noisy oscillator) θ(t) = ω }{{} 0 + noise natural frequency

14 Application 4 Gait Cycle Signal model Stance phase Swing phase Model (Noisy oscillator) θ(t) = ω }{{} 0 + noise natural frequency

15 Application 4 Gait Cycle Signal model Stance phase Swing phase Model (Noisy oscillator) θ(t) = ω }{{} 0 + noise natural frequency

16 Application 4 Gait Cycle Signal model Stance phase Swing phase Model (Noisy oscillator) θ(t) = ω }{{} 0 + noise natural frequency

17 Application 4 Gait Cycle Signal model Stance phase Swing phase Model (Noisy oscillator) θ(t) = ω }{{} 0 + noise natural frequency

18 Application 4 Gait Cycle Signal model Stance phase Swing phase Model (Noisy oscillator) θ(t) = ω }{{} 0 + noise natural frequency

19 Application 5 Problem: Estimate Gait Cycle θ(t) Sensor model Observation model: y(t) = h(θ(t))+ noise Problem: What is θ(t) given noisy observations?

20 Application 5 Problem: Estimate Gait Cycle θ(t) Sensor model Observation model: y(t) = h(θ(t))+ noise Problem: What is θ(t) given noisy observations?

21 Application Problem: Estimate Gait Cycle θ(t) Sensor model Observation model: y(t) = h(θ(t))+ noise Problem: What is θ(t) given noisy observations? 5

22 Application Problem: Estimate Gait Cycle θ(t) Sensor model Observation model: y(t) = h(θ(t))+ noise Problem: What is θ(t) given noisy observations? 5

23 Application Problem: Estimate Gait Cycle θ(t) Sensor model Observation model: y(t) = h(θ(t))+ noise Problem: What is θ(t) given noisy observations? 5

24 Application 6 Solution: Particle Filter Algorithm to approximate posterior distribution Large number of oscillators Posterior distribution: P(φ 1 < θ(t) < φ 2 Sensor readings) = Fraction of θ i (t) in interval (φ 1,φ 2 ) Circuit: θ i (t) = ω i }{{} natural freq. of ith oscillator + noise i + u }{{} i, i = 1,...,N mean-field control Feedback Particle Filter: Design control law u i (t)

25 Application 6 Solution: Particle Filter Algorithm to approximate posterior distribution Large number of oscillators Posterior distribution: P(φ 1 < θ(t) < φ 2 Sensor readings) = Fraction of θ i (t) in interval (φ 1,φ 2 ) Circuit: θ i (t) = ω i }{{} natural freq. of ith oscillator + noise i + u }{{} i, i = 1,...,N mean-field control Feedback Particle Filter: Design control law u i (t)

26 Application 6 Solution: Particle Filter Algorithm to approximate posterior distribution Large number of oscillators Posterior distribution: P(φ 1 < θ(t) < φ 2 Sensor readings) = Fraction of θ i (t) in interval (φ 1,φ 2 ) Circuit: θ i (t) = ω i }{{} natural freq. of ith oscillator + noise i + u }{{} i, i = 1,...,N mean-field control Feedback Particle Filter: Design control law u i (t)

27 Application 6 Solution: Particle Filter Algorithm to approximate posterior distribution Large number of oscillators Posterior distribution: P(φ 1 < θ(t) < φ 2 Sensor readings) = Fraction of θ i (t) in interval (φ 1,φ 2 ) Circuit: θ i (t) = ω i }{{} natural freq. of ith oscillator + noise i + u }{{} i, i = 1,...,N mean-field control Feedback Particle Filter: Design control law u i (t)

28 Application 6 Solution: Particle Filter Algorithm to approximate posterior distribution Large number of oscillators Posterior distribution: P(φ 1 < θ(t) < φ 2 Sensor readings) = Fraction of θ i (t) in interval (φ 1,φ 2 ) Circuit: θ i (t) = ω i }{{} natural freq. of ith oscillator + noise i + u }{{} i, i = 1,...,N mean-field control Feedback Particle Filter: Design control law u i (t)

29 Application 6 Solution: Particle Filter Algorithm to approximate posterior distribution Large number of oscillators Posterior distribution: P(φ 1 < θ(t) < φ 2 Sensor readings) = Fraction of θ i (t) in interval (φ 1,φ 2 ) Circuit: θ i (t) = ω i }{{} natural freq. of ith oscillator + noise i + u }{{} i, i = 1,...,N mean-field control Feedback Particle Filter: Design control law u i (t)

30 Application 7 Simulation Results Solution for the gait cycle estimation problem [Click to play the movie]

31 Application 8

32 Application 8

33 Part I Oscillators

34 Oscillators in Biology 10 Oscillator Models in Neuroscience Literature: Dynamical Systems & Neuroscience

35 Oscillators in Biology Normal Form Reduction Derivation of oscillator model C dv dt = g T m 2 (V ) h (V E T ) g h r (V E h )... dh dt = h (V ) h τ h (V ) dr dt = r (V ) r τ r (V ) [5] J. Guckenheimer, J. Math. Biol., 1975; [1] J. Moehlis et al., Neural Computation,

36 Oscillators in Biology Normal Form Reduction Derivation of oscillator model C dv dt = g T m 2 (V ) h (V E T ) g h r (V E h )... dh dt = h (V ) h τ h (V ) dr dt = r (V ) r τ r (V ) [5] J. Guckenheimer, J. Math. Biol., 1975; [1] J. Moehlis et al., Neural Computation,

37 Oscillators in Biology Normal Form Reduction Derivation of oscillator model C dv dt = g T m 2 (V ) h (V E T ) g h r (V E h )... dh dt = h (V ) h τ h (V ) dr dt = r (V ) r τ r (V ) Normal form reduction θ i = ω i + u i Φ(θ i ) [5] J. Guckenheimer, J. Math. Biol., 1975; [1] J. Moehlis et al., Neural Computation,

38 Oscillators in Biology 12 Collective Dynamics of a Large Number of Oscillators Synchrony, Neural rhythms

39 Oscillators in Biology Synchronization Kuramoto coupled oscillator model ( d dt θ i(t) = ω i + κ N N sin(θ j (t) θ i (t)) j=1 ω i taken from distribution g(ω) over [1 γ,1 + γ] γ measures the heterogeneity of the population κ measures the strength of coupling ) + σ ξ i (t), i = 1,...,N [9] Y. Kuramoto, 1975; [14] Strogatz et al., J. Stat. Phy.,

40 Oscillators in Biology Synchronization Kuramoto coupled oscillator model ( d dt θ i(t) = ω i + κ N N sin(θ j (t) θ i (t)) j=1 ω i taken from distribution g(ω) over [1 γ,1 + γ] γ measures the heterogeneity of the population κ measures the strength of coupling ) + σ ξ i (t), i = 1,...,N [9] Y. Kuramoto, 1975; [14] Strogatz et al., J. Stat. Phy.,

41 Oscillators in Biology Synchronization Kuramoto coupled oscillator model ( d dt θ i(t) = ω i + κ N N j=1 sin(θ j (t) θ i (t)) ω i taken from distribution g(ω) over [1 γ,1 + γ] γ measures the heterogeneity of the population κ measures the strength of coupling ) + σ ξ i (t), i = 1,...,N [9] Y. Kuramoto, 1975; [14] Strogatz et al., J. Stat. Phy.,

42 Oscillators in Biology Synchronization Kuramoto coupled oscillator model ( d dt θ i(t) = ω i + κ N N sin(θ j (t) θ i (t)) j=1 ω i taken from distribution g(ω) over [1 γ,1 + γ] γ measures the heterogeneity of the population κ measures the strength of coupling ) + σ ξ i (t), i = 1,...,N κ 0.3 Synchrony Locking R κ < κ c (γ) Incoherence γ [9] Y. Kuramoto, 1975; [14] Strogatz et al., J. Stat. Phy.,

43 Oscillators in Biology 14 Movies of incoherence and synchrony solution Incoherence [Click to play] Synchrony [Click to play]

44 Oscillators in Biology 15 Functional Role of Neural Rhythms Is synchronization useful? Does it have a functional role? Books/review papers: Buzsaki, Destexhe, Ermentrout, Izhikevich, Kopell, Trout and Whittington (2009), Llinas and Ribary (2001), Pareti and Palma (2004), Sejnowski and Paulsen (2006), Singer (1993)... Computations: Computing with intrinsic network states Destexhe and Contreras (2006); Izhikevich (2006); Zhang and Ballard (2001). Synaptic plasticity: Neurons that fire together wire together And several other hypotheses: Communication and information flow (Laughlin and Sejnowski); Binding by synchrony (Singer); Memory formation (Jutras and Fries); Probabilistic decision making (Wang); Stimulus competition and attention selection (Kopell); Sleep/wakefulness/disease (Steriade)

45 Part II Bayesian Inference

46 Bayesian inference in Neuroscience 17 Prediction Brain as a reality emulator [Prediction] is the primary function of the neocortex, and the foundation of intelligence. If we want to understand how your brain works, and how to build intelligent machines, we must understand the nature of these predictions and how the cortex makes them. The capacity to predict the outcome of future events critical to successful movement is, most likely, the ultimate and most common of all brain functions.

47 Bayesian inference in Neuroscience 17 Prediction Brain as a reality emulator [Prediction] is the primary function of the neocortex, and the foundation of intelligence. If we want to understand how your brain works, and how to build intelligent machines, we must understand the nature of these predictions and how the cortex makes them. The capacity to predict the outcome of future events critical to successful movement is, most likely, the ultimate and most common of all brain functions.

48 Bayesian inference in Neuroscience 17 Prediction Brain as a reality emulator [Prediction] is the primary function of the neocortex, and the foundation of intelligence. If we want to understand how your brain works, and how to build intelligent machines, we must understand the nature of these predictions and how the cortex makes them. The capacity to predict the outcome of future events critical to successful movement is, most likely, the ultimate and most common of all brain functions.

49 Bayesian inference in Neuroscience 17 Prediction Brain as a reality emulator [Prediction] is the primary function of the neocortex, and the foundation of intelligence. If we want to understand how your brain works, and how to build intelligent machines, we must understand the nature of these predictions and how the cortex makes them. The capacity to predict the outcome of future events critical to successful movement is, most likely, the ultimate and most common of all brain functions.

50 Bayesian inference in Neuroscience 18 Bayesian Inference in Neuroscience Edited volumes

51 Bayesian inference in Neuroscience 19 Bayesian Inference in Neuroscience Mathematics of prediction: Bayes rule Signal (hidden): X X P(X ), (prior, known) Solution Bayes rule: P(X Y ) }{{} Posterior P(Y X )P(X ) }{{} Prior Challenge: Implementing Bayes rule in dynamic, nonlinear, non-gaussian settings!

52 Bayesian inference in Neuroscience 19 Bayesian Inference in Neuroscience Mathematics of prediction: Bayes rule Signal (hidden): X X P(X ), (prior, known) Observation: Y (known) Solution Bayes rule: P(X Y ) }{{} Posterior P(Y X )P(X ) }{{} Prior Challenge: Implementing Bayes rule in dynamic, nonlinear, non-gaussian settings!

53 Bayesian inference in Neuroscience Bayesian Inference in Neuroscience Mathematics of prediction: Bayes rule Signal (hidden): X X P(X ), (prior, known) Observation: Y (known) Observation model: P(Y X ) (known) Solution Bayes rule: P(X Y ) }{{} Posterior P(Y X )P(X ) }{{} Prior Challenge: Implementing Bayes rule in dynamic, nonlinear, non-gaussian settings! 19

54 Bayesian inference in Neuroscience Bayesian Inference in Neuroscience Mathematics of prediction: Bayes rule Solution Signal (hidden): X X P(X ), (prior, known) Observation: Y (known) Observation model: P(Y X ) (known) Problem: What is X? Bayes rule: P(X Y ) }{{} Posterior P(Y X )P(X ) }{{} Prior Challenge: Implementing Bayes rule in dynamic, nonlinear, non-gaussian settings! 19

55 Bayesian inference in Neuroscience Bayesian Inference in Neuroscience Mathematics of prediction: Bayes rule Solution Signal (hidden): X X P(X ), (prior, known) Observation: Y (known) Observation model: P(Y X ) (known) Problem: What is X? Bayes rule: P(X Y ) }{{} Posterior P(Y X )P(X ) }{{} Prior Challenge: Implementing Bayes rule in dynamic, nonlinear, non-gaussian settings! 19

56 Bayesian inference in Neuroscience Bayesian Inference in Neuroscience Mathematics of prediction: Bayes rule Solution Signal (hidden): X X P(X ), (prior, known) Observation: Y (known) Observation model: P(Y X ) (known) Problem: What is X? Bayes rule: P(X Y ) }{{} Posterior P(Y X )P(X ) }{{} Prior Challenge: Implementing Bayes rule in dynamic, nonlinear, non-gaussian settings! 19

57 Part III Nonlinear Filtering

58 Nonlinear Filtering 21 Nonlinear Filtering Mathematical Problem Signal model: X t = a(x t ) + Ḃt, X 0 p 0( ) Posterior is an information state P(X t A Y0 t ) = E(X t Y0 t ) = A R p (x,t)dx xp (x,t)dx

59 Nonlinear Filtering 21 Nonlinear Filtering Mathematical Problem Signal model: Observation model: X t = a(x t ) + Ḃ t, X 0 p 0( ) Y t = h(x t ) + Ẇ t Posterior is an information state P(X t A Y0 t ) = E(X t Y0 t ) = A R p (x,t)dx xp (x,t)dx

60 Nonlinear Filtering 21 Nonlinear Filtering Mathematical Problem Signal model: Observation model: X t = a(x t ) + Ḃ t, X 0 p 0( ) Y t = h(x t ) + Ẇ t Problem: What is X t? given obs. till time t =: Y t 0 Posterior is an information state P(X t A Y0 t ) = E(X t Y0 t ) = A R p (x,t)dx xp (x,t)dx

61 Nonlinear Filtering 21 Nonlinear Filtering Mathematical Problem Signal model: Observation model: X t = a(x t ) + Ḃ t, X 0 p 0( ) Y t = h(x t ) + Ẇ t Problem: What is X t? given obs. till time t =: Y t 0 Answer in terms of posterior: P(X t Y t 0 ) =: p (x,t). Posterior is an information state P(X t A Y0 t ) = E(X t Y0 t ) = A R p (x,t)dx xp (x,t)dx

62 Nonlinear Filtering 21 Nonlinear Filtering Mathematical Problem Signal model: Observation model: X t = a(x t ) + Ḃ t, X 0 p 0( ) Y t = h(x t ) + Ẇ t Problem: What is X t? given obs. till time t =: Y t 0 Answer in terms of posterior: P(X t Y t 0 ) =: p (x,t). Posterior is an information state P(X t A Y0 t ) = E(X t Y0 t ) = A R p (x,t)dx xp (x,t)dx

63 Nonlinear Filtering 21 Nonlinear Filtering Mathematical Problem Signal model: Observation model: X t = a(x t ) + Ḃ t, X 0 p 0( ) Y t = h(x t ) + Ẇ t Problem: What is X t? given obs. till time t =: Y t 0 Answer in terms of posterior: P(X t Y t 0 ) =: p (x,t). Posterior is an information state P(X t A Y0 t ) = E(X t Y0 t ) = A R p (x,t)dx xp (x,t)dx

64 Nonlinear Filtering Kalman filter Solution in linear Gaussian settings X t = αx t + Ḃt (1) Y t = γx t + Ẇ t (2) [8] R. E. Kalman, Trans. ASME, Ser. D: J. Basic Eng.,

65 Nonlinear Filtering Kalman filter Solution in linear Gaussian settings X t = αx t + Ḃt (1) Y t = γx t + Ẇ t (2) Kalman filter: p = N( ˆX t,σ t ) ˆX t = α ˆX t + K(Y t γ ˆX t ) }{{} Update [8] R. E. Kalman, Trans. ASME, Ser. D: J. Basic Eng.,

66 Nonlinear Filtering Kalman filter Solution in linear Gaussian settings Kalman filter: p = N( ˆX t,σ t ) ˆX t = α ˆX t + K(Y t γ ˆX t ) }{{} Update X t = αx t + Ḃt (1) Y t = γx t + Ẇ t (2) - + Kalman Filter [8] R. E. Kalman, Trans. ASME, Ser. D: J. Basic Eng.,

67 Nonlinear Filtering Kalman filter Solution in linear Gaussian settings X t = αx t + Ḃt (1) Y t = γx t + Ẇ t (2) Kalman filter: p = N( ˆX t,σ t ) ˆX t = α ˆX t + K(Y t γ ˆX t ) }{{} Update Observation: Kalman Filter Y t = γx t + Ẇ t - + Kalman Filter [8] R. E. Kalman, Trans. ASME, Ser. D: J. Basic Eng.,

68 Nonlinear Filtering Kalman filter Solution in linear Gaussian settings X t = αx t + Ḃt (1) Y t = γx t + Ẇ t (2) Kalman filter: p = N( ˆX t,σ t ) ˆX t = α ˆX t + K(Y t γ ˆX t ) }{{} Update Observation: Prediction: Kalman Filter Y t = γx t + Ẇ t Ŷ t = γ ˆX t - + Kalman Filter [8] R. E. Kalman, Trans. ASME, Ser. D: J. Basic Eng.,

69 Nonlinear Filtering Kalman filter Solution in linear Gaussian settings X t = αx t + Ḃt (1) Y t = γx t + Ẇ t (2) Kalman filter: p = N( ˆX t,σ t ) ˆX t = α ˆX t + K(Y t γ ˆX t ) }{{} Update Observation: Prediction: Kalman Filter Y t = γx t + Ẇ t Ŷ t = γ ˆX t - + Innov. error: I t = Y t Ŷ t = Y t γ ˆX t Kalman Filter [8] R. E. Kalman, Trans. ASME, Ser. D: J. Basic Eng.,

70 Nonlinear Filtering Kalman filter Solution in linear Gaussian settings X t = αx t + Ḃt (1) Y t = γx t + Ẇ t (2) Kalman filter: p = N( ˆX t,σ t ) ˆX t = α ˆX t + K(Y t γ ˆX t ) }{{} Update Observation: Prediction: Kalman Filter Y t = γx t + Ẇ t Ŷ t = γ ˆX t - + Innov. error: I t = Y t Ŷ t = Y t γ ˆX t Kalman Filter Control: U t = KI t [8] R. E. Kalman, Trans. ASME, Ser. D: J. Basic Eng.,

71 Nonlinear Filtering Kalman filter Solution in linear Gaussian settings X t = αx t + Ḃt (1) Y t = γx t + Ẇ t (2) Kalman filter: p = N( ˆX t,σ t ) ˆX t = α ˆX t + K(Y t γ ˆX t ) }{{} Update Observation: Prediction: Kalman Filter Y t = γx t + Ẇ t Ŷ t = γ ˆX t - + Innov. error: I t = Y t Ŷ t = Y t γ ˆX t Kalman Filter Control: U t = KI t Gain: Kalman gain [8] R. E. Kalman, Trans. ASME, Ser. D: J. Basic Eng.,

72 Nonlinear Filtering Applications in Engineering Filtering is a mature field with many applications Filtering is important to: Air moving target indicator (AMTI) systems, Space situational awareness Remote sensing and surveillance: Air traffic management, weather surveillance, geophysical surveys Autonomous navigation & robotics: Simultaneous localization and map building (SLAM) 23

73 Nonlinear Filtering Applications in Engineering Filtering is a mature field with many applications Filtering is important to: Air moving target indicator (AMTI) systems, Space situational awareness Remote sensing and surveillance: Air traffic management, weather surveillance, geophysical surveys Autonomous navigation & robotics: Simultaneous localization and map building (SLAM) 23

74 Nonlinear Filtering Applications in Engineering Filtering is a mature field with many applications Filtering is important to: Air moving target indicator (AMTI) systems, Space situational awareness Remote sensing and surveillance: Air traffic management, weather surveillance, geophysical surveys Autonomous navigation & robotics: Simultaneous localization and map building (SLAM) 23

75 Nonlinear Filtering Applications in Engineering Filtering is a mature field with many applications Filtering is important to: Air moving target indicator (AMTI) systems, Space situational awareness Remote sensing and surveillance: Air traffic management, weather surveillance, geophysical surveys Autonomous navigation & robotics: Simultaneous localization and map building (SLAM) 23

76 Nonlinear Filtering Applications in Engineering Filtering is a mature field with many applications Filtering is important to: Air moving target indicator (AMTI) systems, Space situational awareness Remote sensing and surveillance: Air traffic management, weather surveillance, geophysical surveys Autonomous navigation & robotics: Simultaneous localization and map building (SLAM) 23

77 Nonlinear Filtering Filtering in Brain? Bayesian model of sensory signal processing Theory: Lee and Mumford, Hierarchical Bayesian inference Framework (2003) Rao; Rao and Ballard; Rao and Sejnowski. Predictive coding framework (2002) Dayan, Hinton, Neal and Zemel. The Helmholtz machine (1995) Lewicki and Sejnowski. Bayesian unsupervised learning (1995) Ma, Beck, Latham and Pouget. Probabilistic population codes (2006) And others: See Doya, Ishii, Pouget and Rao, Bayesian Brain, MIT Press (2007) Rao, Olshausen & Lewicki, Probabilistic Models of Brain, MIT Press (2002) 24

78 Nonlinear Filtering Filtering in Brain? Bayesian model of sensory signal processing Theory: Lee and Mumford, Hierarchical Bayesian inference Framework (2003) Rao; Rao and Ballard; Rao and Sejnowski. Predictive coding framework (2002) Dayan, Hinton, Neal and Zemel. The Helmholtz machine (1995) Lewicki and Sejnowski. Bayesian unsupervised learning (1995) Ma, Beck, Latham and Pouget. Probabilistic population codes (2006) And others: See Doya, Ishii, Pouget and Rao, Bayesian Brain, MIT Press (2007) Rao, Olshausen & Lewicki, Probabilistic Models of Brain, MIT Press (2002) 24

79 Nonlinear Filtering 25 Filtering in Brain? Bayesian model of sensory signal processing Experiments (see reviews): Gold & Shadlen, The neural basis of decision making, Ann. Rev. of Neurosci. (2007) R. T. Knight, Neural networks debunk phrenology, Science (2007) Such theories naturally feed into computer vision & more generally on how to make computer intelligent

80 Nonlinear Filtering 25 Filtering in Brain? Bayesian model of sensory signal processing Experiments (see reviews): Gold & Shadlen, The neural basis of decision making, Ann. Rev. of Neurosci. (2007) R. T. Knight, Neural networks debunk phrenology, Science (2007) Such theories naturally feed into computer vision & more generally on how to make computer intelligent

81 Nonlinear Filtering 26 Bayesian Inference in Neuroscience Lee and Mumford s hierarchical Bayesian inference framework Bayes rule Bayes rule Bayes rule... Similar ideas also appear in: 1 Dayan, Hinton, Neal and Zemel. The Helmholtz machine (1995) 2 Lewicki and Sejnowski. Bayesian unsupervised learning (1995) 3 Rao and Ballard; Rao and Sejnowski. Predictive coding framework (1999;2002)

82 Nonlinear Filtering 26 Bayesian Inference in Neuroscience Lee and Mumford s hierarchical Bayesian inference framework Bayes rule Bayes rule Bayes rule... Part. Filter Part. Filter Part. Filter... Similar ideas also appear in: 1 Dayan, Hinton, Neal and Zemel. The Helmholtz machine (1995) 2 Lewicki and Sejnowski. Bayesian unsupervised learning (1995) 3 Rao and Ballard; Rao and Sejnowski. Predictive coding framework (1999;2002)

83 Nonlinear Filtering 27 What is a Particle Filter An algorithm to solve nonlinear filtering problem Approximate posterior in terms of particles p (x,t) = 1 N Algorithm outline 1 Initialization at time 0: X i 0 p 0( ) 2 At each discrete time step: Importance sampling (Bayes update step) Resampling (for variance reduction) N i=1 δ X i t (x)

84 Nonlinear Filtering 27 What is a Particle Filter An algorithm to solve nonlinear filtering problem Approximate posterior in terms of particles p (x,t) = 1 N Algorithm outline 1 Initialization at time 0: X i 0 p 0( ) 2 At each discrete time step: Importance sampling (Bayes update step) Resampling (for variance reduction) N i=1 δ X i t (x)

85 Nonlinear Filtering 27 What is a Particle Filter An algorithm to solve nonlinear filtering problem Approximate posterior in terms of particles p (x,t) = 1 N Algorithm outline 1 Initialization at time 0: X i 0 p 0( ) 2 At each discrete time step: Importance sampling (Bayes update step) Resampling (for variance reduction) N i=1 δ X i t (x)

86 Nonlinear Filtering 27 What is a Particle Filter An algorithm to solve nonlinear filtering problem Approximate posterior in terms of particles p (x,t) = 1 N Algorithm outline 1 Initialization at time 0: X i 0 p 0( ) 2 At each discrete time step: Importance sampling (Bayes update step) Resampling (for variance reduction) N i=1 δ X i t (x)

87 Nonlinear Filtering What is a Particle Filter An algorithm to solve nonlinear filtering problem Approximate posterior in terms of particles p (x,t) = 1 N Algorithm outline 1 Initialization at time 0: X i 0 p 0( ) 2 At each discrete time step: Importance sampling (Bayes update step) Resampling (for variance reduction) N i=1 δ X i t (x) It is unclear how to implement this with neurons? 27

88 Nonlinear Filtering What is a Particle Filter An algorithm to solve nonlinear filtering problem Approximate posterior in terms of particles p (x,t) = 1 N Algorithm outline 1 Initialization at time 0: X i 0 p 0( ) 2 At each discrete time step: Importance sampling (Bayes update step) Resampling (for variance reduction) N i=1 δ X i t (x) It is unclear how to implement this with neurons? 27

89 Part IV Feedback Particle Filter

90 Feedback particle filter Control-oriented Formulation of Particle Filter Use feedback control to implement Bayes rule Signal & Observations Controlled system (N particles): X t = a(x t ) + σ B Ḃ t (1) Y t = h(x t ) + σ W Ẇ t (2) X t i = a(xt i ) + σ B Ḃt i + Ut i, i = 1,...,N (3) }{{} mean field control {Ḃ i t} N i=1 are ind. standard white noises. Objective: Choose control Ut, i as a function of history {Y s,xs i : 0 s t}, such that the two posteriors coincide: P{X t A Z t } p (x,t) dx = p(x,t) dx = P{Xt i A Z t } x A x A Huang, Caines and Malhame IEEE TAC (2007); Lasry and Lions (2007) 29

91 Feedback particle filter Control-oriented Formulation of Particle Filter Use feedback control to implement Bayes rule Signal & Observations Controlled system (N particles): X t = a(x t ) + σ B Ḃ t (1) Y t = h(x t ) + σ W Ẇ t (2) X t i = a(xt i ) + σ B Ḃt i + Ut i, i = 1,...,N (3) }{{} mean field control {Ḃ i t} N i=1 are ind. standard white noises. Objective: Choose control Ut, i as a function of history {Y s,xs i : 0 s t}, such that the two posteriors coincide: P{X t A Z t } p (x,t) dx = p(x,t) dx = P{Xt i A Z t } x A x A Huang, Caines and Malhame IEEE TAC (2007); Lasry and Lions (2007) 29

92 Feedback particle filter Control-oriented Formulation of Particle Filter Use feedback control to implement Bayes rule Signal & Observations Controlled system (N particles): X t = a(x t ) + σ B Ḃ t (1) Y t = h(x t ) + σ W Ẇ t (2) X t i = a(xt i ) + σ B Ḃt i + Ut i, i = 1,...,N (3) }{{} mean field control {Ḃ i t} N i=1 are ind. standard white noises. Objective: Choose control Ut, i as a function of history {Y s,xs i : 0 s t}, such that the two posteriors coincide: P{X t A Z t } p (x,t) dx = p(x,t) dx = P{Xt i A Z t } x A x A Huang, Caines and Malhame IEEE TAC (2007); Lasry and Lions (2007) 29

93 Feedback particle filter 30 Feedback Particle Filter Filtering in nonlinear non-gaussian settings Signal model: Observation model: X t = a(x t ) + Ḃt, X 0 p 0( ) Y t = h(x t ) + Ẇ t FPF: Ẋ i t = a(x i t ) + Ḃ i t + K(X i t )I i t }{{} Update Innovations: I i t =:Y t 1 2 (h(x i t ) + ĥ), with cond. mean ĥ = p,h.

94 Feedback particle filter 30 Feedback Particle Filter Filtering in nonlinear non-gaussian settings Signal model: Observation model: X t = a(x t ) + Ḃt, X 0 p 0( ) Y t = h(x t ) + Ẇ t FPF: Ẋ i t = a(x i t ) + Ḃ i t + K(X i t )I i t }{{} Update Innovations: I i t =:Y t 1 2 (h(x i t ) + ĥ), with cond. mean ĥ = p,h.

95 Feedback particle filter 31 Update Step How does feedback particle filter implement Bayes rule? Feedback particle filter Linear Kalman filter Observation: Y t = h(x t ) + Ẇ t Y t = γx t + Ẇ t

96 Feedback particle filter 31 Update Step How does feedback particle filter implement Bayes rule? Feedback particle filter Linear Kalman filter Observation: Y t = h(x t ) + Ẇ t Y t = γx t + Ẇ t Prediction: Ŷt i = h(x t i )+ĥ 2 ĥ = 1 N N i=1 h(x t i ) Ŷ t = γ ˆX t

97 Feedback particle filter 31 Update Step How does feedback particle filter implement Bayes rule? Feedback particle filter Linear Kalman filter Observation: Y t = h(x t ) + Ẇ t Y t = γx t + Ẇ t Prediction: Ŷt i = h(x t i )+ĥ 2 ĥ = 1 N N i=1 h(x t i ) Ŷ t = γ ˆX t Innov. error: It i = Y t Ŷ t i I t = Y t Ŷt = Y t h(x t i )+ĥ 2 = Y t γ ˆX t

98 Feedback particle filter 31 Update Step How does feedback particle filter implement Bayes rule? Feedback particle filter Linear Kalman filter Observation: Y t = h(x t ) + Ẇ t Y t = γx t + Ẇ t Prediction: Ŷt i = h(x t i )+ĥ 2 ĥ = 1 N N i=1 h(x t i ) Ŷ t = γ ˆX t Innov. error: It i = Y t Ŷ t i I t = Y t Ŷt = Y t h(x t i )+ĥ 2 = Y t γ ˆX t Control: U i t = K(X i t )I i t U t = KI t

99 Feedback particle filter 31 Update Step How does feedback particle filter implement Bayes rule? Feedback particle filter Linear Kalman filter Observation: Y t = h(x t ) + Ẇ t Y t = γx t + Ẇ t Prediction: Ŷt i = h(x t i )+ĥ 2 ĥ = 1 N N i=1 h(x t i ) Ŷ t = γ ˆX t Innov. error: It i = Y t Ŷ t i I t = Y t Ŷt = Y t h(x t i )+ĥ 2 = Y t γ ˆX t Control: U i t = K(X i t )I i t U t = KI t Gain: K is a solution of a linear BVP K is the Kalman gain

100 Feedback particle filter Feedback Particle Filter Filtering in nonlinear non-gaussian settings Signal model: Observation model: FPF: X t = a(x t ) + Ḃt, X 0 p 0( ) Y t = h(x t ) + Ẇ t Ẋ i t = a(x i t ) + Ḃ i t + K(X i t )I i t }{{} Update Innovations: It i =:Y t 1 2 (h(x t i ) + ĥ), with cond. mean ĥ = p,h. - + Feedback Particle Filter T. Yang, P. G. Mehta and S. P. Meyn, A Control-oriented Approach for Particle Filtering, ACC 2011, CDC

101 Feedback particle filter Feedback Particle Filter Filtering in nonlinear non-gaussian settings Signal model: Observation model: FPF: X t = a(x t ) + Ḃt, X 0 p 0( ) Y t = h(x t ) + Ẇ t Ẋ i t = a(x i t ) + Ḃ i t + K(X i t )I i t }{{} Update Innovations: It i =:Y t 1 2 (h(x t i ) + ĥ), with cond. mean ĥ = p,h. - + Feedback Particle Filter T. Yang, P. G. Mehta and S. P. Meyn, A Control-oriented Approach for Particle Filtering, ACC 2011, CDC

102 Feedback particle filter 33 Robustness of feedback particle filter Variance reduction Mean-square error: ( 1 T T 0 Σ (N) t ) 2 Σ t dt Σ t MSE 10 1 Bootstrap (BPF) 10 2 Feedback (FPF) N (number of particles)

103 Part V Application: Filtering with Rhythms

104 35 Filtering of Biological Rhythms with Brain Rhythms Connection to Lee and Mumford s hierarchical Bayesian inference framework Noisy input Part. Filter Part. Filter Part. Filter... Prior

105 35 Filtering of Biological Rhythms with Brain Rhythms Connection to Lee and Mumford s hierarchical Bayesian inference framework Noisy input Part. Filter Part. Filter Part. Filter... Prior Normal form reduction Mumford s box with neurons Rhythmic movement Noisy measurements Mumford s box with oscillators Normal form reduction Prior Estimate

106 35 Filtering of Biological Rhythms with Brain Rhythms Connection to Lee and Mumford s hierarchical Bayesian inference framework Noisy input Part. Filter Part. Filter Part. Filter... Prior Normal form reduction Mumford s box with neurons Rhythmic movement Noisy measurements Mumford s box with oscillators Normal form reduction Prior Estimate

107 35 Filtering of Biological Rhythms with Brain Rhythms Connection to Lee and Mumford s hierarchical Bayesian inference framework Noisy input Part. Filter Part. Filter Part. Filter... Prior Normal form reduction Mumford s box with neurons Rhythmic movement Noisy measurements Mumford s box with oscillators Normal form reduction Prior Estimate

108 Application Signal & Observation models Ankle-foot orthoses (AFOs) : For lower-limb neuromuscular impairments. Provides dorsiflexor (toe lift) and plantarflexor (toe push) torque assistance Heel Force Regression Model Experimental Data Solenoid valves: control the flow of CO2 to the actuator Actuator Compressed CO2 Toe Force Sensors: heel, toe, and ankle joint AFO system components: Power supply, Valves, Actuator, Sensors. Ankle Angle Percent Gait Cycle Cycles of sensor data. Acknowledgement: Professor Liz Hsiao-Wecksler for sharing the AFO device picture and sensor data. 36

109 Application Signal & Observation models Ankle-foot orthoses (AFOs) : For lower-limb neuromuscular impairments. Provides dorsiflexor (toe lift) and plantarflexor (toe push) torque assistance Heel Force Regression Model Experimental Data Solenoid valves: control the flow of CO2 to the actuator Actuator Compressed CO2 Toe Force Sensors: heel, toe, and ankle joint AFO system components: Power supply, Valves, Actuator, Sensors. Ankle Angle Percent Gait Cycle Cycles of sensor data. Acknowledgement: Professor Liz Hsiao-Wecksler for sharing the AFO device picture and sensor data. 36

110 Filtering with rhythms Feedback particle filter The gait cycle is a single sequence of functions of one limb. Compressed CO2 Solenoid valves: control the flow of CO2 to the actuator Actuator Sensors: heel, toe, and ankle joint Oscillator model to estimate gait Gait-State Estimation Oscillator Model of gait: θ t = ω + B t mod 2π Observations: h(θ ) pulse function: π 0 π 37

111 Filtering with rhythms Feedback particle filter The gait cycle is a single sequence of functions of one limb. Compressed CO2 Solenoid valves: control the flow of CO2 to the actuator Actuator Sensors: heel, toe, and ankle joint Oscillator model to estimate gait Gait-State Estimation Oscillator Model of gait: θ t = ω + B t mod 2π Observations: h(θ ) pulse function: π 0 π 37

112 38 Filtering for Oscillators Signal & Observations θ t = ω + Ḃt mod 2π Y t = h(θ t ) + Ẇ t π 0 π Particle evolution, θ t i = ω i + Ḃt i + K(θt i )[Y t 1 2 (h(θ t i ) + ĥ)] mod 2π, i = 1,...,N. where ω i is sampled from a distribution.

113 38 Filtering for Oscillators Signal & Observations θ t = ω + Ḃt mod 2π Y t = h(θ t ) + Ẇ t π 0 π Particle evolution, θ t i = ω i + Ḃt i + K(θt i )[Y t 1 2 (h(θ t i ) + ĥ)] mod 2π, i = 1,...,N. where ω i is sampled from a distribution.

114 38 Filtering for Oscillators Signal & Observations θ t = ω + Ḃ t mod 2π Y t = h(θ t ) + Ẇt π 0 π Particle evolution, θ t i = ω i + Ḃt i + K(θt i )[Y t 1 2 (h(θ t i ) + ĥ)] mod 2π, i = 1,...,N. where ω i is sampled from a distribution. - + Feedback Particle Filter

115 39 Simulation Results Solution of the Estimation of Gait Cycle Problem [Click to play the movie]

116 Thank you! Website: Collaborators Adam Tilton Tao Yang Huibing Yin Liz Hsiao-Wecksler Sean Meyn Synchronization of Coupled Oscillators is a Game, ACC 2010, IEEE TAC 2012 Learning in Mean-field Oscillator Game, CDC 2010 A Control-oriented Approach for Particle Filtering, ACC 2011, CDC 2011 Filtering with Rhythms: Application to Estimation of Gait Cycle, ACC 2012

117 41

118 Bibliography 41 Eric Brown, Jeff Moehlis, and Philip Holmes. On the phase reduction and response dynamics of neural oscillator populations. Neural Computation, 16(4): , A. Doucet, N. de Freitas, and N. Gordon. Sequential Monte-Carlo Methods in Practice. Springer-Verlag, April R. Ericson and A. Pakes. Markov-perfect industry dynamics: A framework for empirical work. The Review of Economic Studies, 62(1):53 82, N. J. Gordon, D. J. Salmond, and A. F. M. Smith. Novel approach to nonlinear/non-gaussian Bayesian state estimation. IEE Proceedings F Radar and Signal Processing, 140(2): , J. Guckenheimer. Isochrons and phaseless sets. J. Math. Biol., 1: , M. Huang, P. E. Caines, and R. P. Malhame. Large-population cost-coupled LQG problems with nonuniform agents: Individual-mass behavior and decentralized ε-nash equilibria. 52(9): , Minyi Huang, Peter E. Caines, and Roland P. Malhame. Large-population cost-coupled LQG problems with nonuniform agents: Individual-mass behavior and decentralized ε-nash equilibria. IEEE transactions on automatic control, 52(9): , R. E. Kalman. A new approach to linear filtering and prediction problems. Journal of Basic Engineering, 82(1):35 45, Y. Kuramoto. International Symposium on Mathematical Problems in Theoretical Physics, volume 39 of Lecture Notes in Physics. Springer-Verlag, 1975.

119 Bibliography 41 H. J. Kushner. On the differential equations satisfied by conditional probability densities of markov process. SIAM J. Control, 2: , J. Lasry and P. Lions. Mean field games. Japanese Journal of Mathematics, 2(2): , Jean-Michel Lasry and Pierre-Louis Lions. Mean field games. Japan. J. Math., 2: , R. L. Stratonovich. Conditional Markov processes. SIAM Theory Probab. Appl., 5: , S. H. Strogatz and R. E. Mirollo. Stability of incoherence in a population of coupled oscillators. Journal of Statistical Physics, 63: , May Huibing Yin, Prashant G. Mehta, Sean P. Meyn, and Uday V. Shanbhag. Synchronization of coupled oscillators is a game. In Proc. of 2010 American Control Conference, pages , Baltimore, MD, Mehta P. G. Meyn S. P. Yin, H. and U. V. Shanbhag. Synchronization of coupled oscillators is a game. IEEE Trans. Automat. Control. G. Y. Weintraub, L. Benkard, and B. Van Roy. Oblivious equilibrium: A mean field approximation for large-scale dynamic games. In Advances in Neural Information Processing Systems, volume 18. MIT Press, G. Y. Weintraub, L. Benkard, and B. V. Roy. Markov perfect industry dynamics with many firms. Econometrica, 76(6): , 2008.