Feedback Particle Filter and its Application to Coupled Oscillators Presentation at University of Maryland, College Park, MD

Feedback Particle Filter and its Application to Coupled Oscillators Presentation at University of Maryland, College Park, MD Prashant Mehta Dept. of Mechanical Science and Engineering and the Coordinated Science Laboratory University of Illinois at Urbana-Champaign May 1, 2015

Bayesian Inference/Filtering Mathematics of prediction: Bayes rule Signal (hidden): X X P(X), (prior, known) Feedback Particle Filter Prashant Mehta 2 / 34

Bayesian Inference/Filtering Mathematics of prediction: Bayes rule Signal (hidden): X X P(X), (prior, known) Observation: Y (known) Feedback Particle Filter Prashant Mehta 2 / 34

Bayesian Inference/Filtering Mathematics of prediction: Bayes rule Signal (hidden): X X P(X), (prior, known) Observation: Y (known) Observation model: P(Y X) (known) Feedback Particle Filter Prashant Mehta 2 / 34

Bayesian Inference/Filtering Mathematics of prediction: Bayes rule Signal (hidden): X X P(X), (prior, known) Observation: Y (known) Observation model: P(Y X) (known) Problem: What is X? Solution Bayes rule: P(X Y) P(Y X) P(X) }{{}}{{} Posterior Prior Feedback Particle Filter Prashant Mehta 2 / 34

Applications Target state estimation Feedback Particle Filter Prashant Mehta 3 / 34

Applications Bayesian model of sensory signal processing Feedback Particle Filter Prashant Mehta 3 / 34

Nonlinear Filtering Mathematical Problem Signal model: dx t = a(x t )dt + db t, X 0 p 0 ( ) A. Bain and D. Crisan, Fundamentals of Stochastic Filtering. Springer, 2010. Feedback Particle Filter Prashant Mehta 4 / 34

Nonlinear Filtering Mathematical Problem Signal model: dx t = a(x t )dt + db t, X 0 p 0 ( ) Observation model: dz t = h(x t )dt + dw t A. Bain and D. Crisan, Fundamentals of Stochastic Filtering. Springer, 2010. Feedback Particle Filter Prashant Mehta 4 / 34

Nonlinear Filtering Mathematical Problem Signal model: dx t = a(x t )dt + db t, X 0 p 0 ( ) Observation model: Problem: dz t = h(x t )dt + dw t What is X t? given obs. till time t =: Z t A. Bain and D. Crisan, Fundamentals of Stochastic Filtering. Springer, 2010. Feedback Particle Filter Prashant Mehta 4 / 34

Nonlinear Filtering Mathematical Problem Signal model: dx t = a(x t )dt + db t, X 0 p 0 ( ) Observation model: Problem: Answer in terms of posterior: dz t = h(x t )dt + dw t What is X t? given obs. till time t =: Z t P(X t Z t ) =: p (x,t). A. Bain and D. Crisan, Fundamentals of Stochastic Filtering. Springer, 2010. Feedback Particle Filter Prashant Mehta 4 / 34

Nonlinear Filtering Mathematical Problem Signal model: dx t = a(x t )dt + db t, X 0 p 0 ( ) Observation model: Problem: Answer in terms of posterior: dz t = h(x t )dt + dw t What is X t? given obs. till time t =: Z t P(X t Z t ) =: p (x,t). Posterior is an information state P(X t A Z t ) = p (x,t)dx A E(X t Z t ) = xp (x,t)dx R A. Bain and D. Crisan, Fundamentals of Stochastic Filtering. Springer, 2010. Feedback Particle Filter Prashant Mehta 4 / 34

Kalman filter Solution in linear Gaussian settings dx t = αx t dt + db t (1) dz t = γx t dt + dw t (2) R. E. Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng. (1961); R. E. Kalman and R. S. Bucy. New Results in Liner Filtering and Prediction Theory. J. Basic Eng. (1961). Feedback Particle Filter Prashant Mehta 5 / 34

Kalman filter Solution in linear Gaussian settings dx t = αx t dt + db t (1) dz t = γx t dt + dw t (2) Kalman filter: p = N( ˆX t,σ t ) d ˆX t = α ˆX t dt + K(dZ t γ ˆX t dt) }{{} Update R. E. Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng. (1961); R. E. Kalman and R. S. Bucy. New Results in Liner Filtering and Prediction Theory. J. Basic Eng. (1961). Feedback Particle Filter Prashant Mehta 5 / 34

Kalman filter Solution in linear Gaussian settings Kalman filter: p = N( ˆX t,σ t ) dx t = αx t dt + db t (1) dz t = γx t dt + dw t (2) d ˆX t = α ˆX t dt + K(dZ t γ ˆX t dt) }{{} Update - + Kalman Filter R. E. Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng. (1961); R. E. Kalman and R. S. Bucy. New Results in Liner Filtering and Prediction Theory. J. Basic Eng. (1961). Feedback Particle Filter Prashant Mehta 5 / 34

Kalman filter Solution in linear Gaussian settings Kalman filter: p = N( ˆX t,σ t ) d ˆX t = α ˆX t dt + K(dZ t γ ˆX t dt) }{{} Update dx t = αx t dt + db t (1) dz t = γx t dt + dw t (2) Kalman Filter Observation: dz t = γx t dt + dw t - + Kalman Filter R. E. Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng. (1961); R. E. Kalman and R. S. Bucy. New Results in Liner Filtering and Prediction Theory. J. Basic Eng. (1961). Feedback Particle Filter Prashant Mehta 5 / 34

Kalman filter Solution in linear Gaussian settings Kalman filter: p = N( ˆX t,σ t ) d ˆX t = α ˆX t dt + K(dZ t γ ˆX t dt) }{{} Update dx t = αx t dt + db t (1) dz t = γx t dt + dw t (2) Kalman Filter Observation: Prediction: dz t = γx t dt + dw t dẑ t = γ ˆX t dt - + Kalman Filter R. E. Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng. (1961); R. E. Kalman and R. S. Bucy. New Results in Liner Filtering and Prediction Theory. J. Basic Eng. (1961). Feedback Particle Filter Prashant Mehta 5 / 34

Kalman filter Solution in linear Gaussian settings Kalman filter: p = N( ˆX t,σ t ) d ˆX t = α ˆX t dt + K(dZ t γ ˆX t dt) }{{} Update dx t = αx t dt + db t (1) dz t = γx t dt + dw t (2) Kalman Filter Observation: Prediction: dz t = γx t dt + dw t dẑ t = γ ˆX t dt - + Innov. error: di t = dz t dẑ t = dz t γ ˆX t dt Kalman Filter R. E. Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng. (1961); R. E. Kalman and R. S. Bucy. New Results in Liner Filtering and Prediction Theory. J. Basic Eng. (1961). Feedback Particle Filter Prashant Mehta 5 / 34

Kalman filter Solution in linear Gaussian settings Kalman filter: p = N( ˆX t,σ t ) d ˆX t = α ˆX t dt + K(dZ t γ ˆX t dt) }{{} Update dx t = αx t dt + db t (1) dz t = γx t dt + dw t (2) Kalman Filter Observation: Prediction: dz t = γx t dt + dw t dẑ t = γ ˆX t dt - + Innov. error: di t = dz t dẑ t = dz t γ ˆX t dt Control: du t = K di t Kalman Filter R. E. Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng. (1961); R. E. Kalman and R. S. Bucy. New Results in Liner Filtering and Prediction Theory. J. Basic Eng. (1961). Feedback Particle Filter Prashant Mehta 5 / 34

Kalman filter Solution in linear Gaussian settings Kalman filter: p = N( ˆX t,σ t ) d ˆX t = α ˆX t dt + K(dZ t γ ˆX t dt) }{{} Update dx t = αx t dt + db t (1) dz t = γx t dt + dw t (2) Kalman Filter Observation: Prediction: dz t = γx t dt + dw t dẑ t = γ ˆX t dt - + Innov. error: di t = dz t dẑ t = dz t γ ˆX t dt Control: du t = K di t Kalman Filter Gain: Kalman gain R. E. Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng. (1961); R. E. Kalman and R. S. Bucy. New Results in Liner Filtering and Prediction Theory. J. Basic Eng. (1961). Feedback Particle Filter Prashant Mehta 5 / 34

Kalman filter d ˆX t = α ˆX t dt }{{} Prediction + K(dZ t γ ˆX t dt) }{{} Update Simple enough to be included in the first undergraduate course on control! Feedback Particle Filter Prashant Mehta 6 / 34

Kalman filter d ˆX t = α ˆX t dt }{{} Prediction + K(dZ t γ ˆX t dt) }{{} Update - + Kalman Filter Simple enough to be included in the first undergraduate course on control! Feedback Particle Filter Prashant Mehta 6 / 34

Kalman filter d ˆX t = α ˆX t dt }{{} Prediction + K(dZ t γ ˆX t dt) }{{} Update - + Kalman Filter This illustrates the key features of feedback control: 1 Use error to obtain control (du t = K di t ) 2 Negative gain feedback serves to reduce error (K = γ σw 2 Σ t ) }{{} SNR Simple enough to be included in the first undergraduate course on control! Feedback Particle Filter Prashant Mehta 6 / 34

Pretty Formulae in Mathematics More often than not, these are simply stated Euler s identity e iπ = 1 Euler s formula v e + f = 2 Pythagoras theorem x 2 + y 2 = z 2 Kenneth Chang. What Makes an Equation Beautiful? in The New York Times on October 24, 2004 Feedback Particle Filter Prashant Mehta 7 / 34

Filtering Problem Nonlinear Model: Kushner-Stratonovich PDE Signal & Observations dx t = a(x t )dt + db t, (1) dz t = h(x t )dt + dw t (2) Posterior distribution p is a solution of a stochastic PDE: dp = L (p )dt + 1 (h ĥ)(dz t ĥdt)p σw 2 where ĥ = E[h(X t ) Z t ] = h(x)p (x,t)dx L (p ) = (p a(x)) + 1 2 p x 2 x 2 R. L. Stratonovich. Conditional Markov Processes. Theory Probab. Appl. (1960); H. J. Kushner. On the differential equations satisfied by conditional probability densities of Markov processes. SIAM J. Control (1964). Feedback Particle Filter Prashant Mehta 8 / 34

Filtering Problem Nonlinear Model: Kushner-Stratonovich PDE Signal & Observations dx t = a(x t )dt + db t, (1) dz t = h(x t )dt + dw t (2) Posterior distribution p is a solution of a stochastic PDE: dp = L (p )dt + 1 (h ĥ)(dz t ĥdt)p σw 2 where ĥ = E[h(X t ) Z t ] = h(x)p (x,t)dx L (p ) = (p a(x)) + 1 2 p x 2 x 2 No closed-form solution in general. Closure problem. R. L. Stratonovich. Conditional Markov Processes. Theory Probab. Appl. (1960); H. J. Kushner. On the differential equations satisfied by conditional probability densities of Markov processes. SIAM J. Control (1964). Feedback Particle Filter Prashant Mehta 8 / 34

Particle Filter An algorithm to solve nonlinear filtering problem Approximate posterior in terms of particles p (x,t) = 1 N N δ X i(x) t i=1 Algorithm outline 1 Initialization at time 0: X i 0 p 0 ( ) 2 At each discrete time step: Importance sampling (Bayes update step) Resampling (for variance reduction) J. E. Handschin and D. Q. Mayne. Monte-Carlo techniques to estimate conditional expectation in nonlinear filtering. Int. J. Control (1969); N. Gordon, D. Salmond, A. Smith. Novel approach to nonlinear/non-gaussian Bayesian state estimation. IEE Proc. F Radar Signal Process (1993); J. Xiong. Particle approximation to filtering problems in continuous time. The Oxford handbook of nonlinear filtering (2011); A. Budhiraja, L. Chen, C. Lee. A survey of numerical methods for nonlinear filtering problems. Physica D (2007). Feedback Particle Filter Prashant Mehta 9 / 34

Particle Filter An algorithm to solve nonlinear filtering problem Approximate posterior in terms of particles p (x,t) = 1 N N δ X i(x) t i=1 Algorithm outline 1 Initialization at time 0: X i 0 p 0 ( ) 2 At each discrete time step: Importance sampling (Bayes update step) Resampling (for variance reduction) e.g. dz t = X t dt + small noise J. E. Handschin and D. Q. Mayne. Monte-Carlo techniques to estimate conditional expectation in nonlinear filtering. Int. J. Control (1969); N. Gordon, D. Salmond, A. Smith. Novel approach to nonlinear/non-gaussian Bayesian state estimation. IEE Proc. F Radar Signal Process (1993); J. Xiong. Particle approximation to filtering problems in continuous time. The Oxford handbook of nonlinear filtering (2011); A. Budhiraja, L. Chen, C. Lee. A survey of numerical methods for nonlinear filtering problems. Physica D (2007). Feedback Particle Filter Prashant Mehta 9 / 34

Particle Filter An algorithm to solve nonlinear filtering problem Approximate posterior in terms of particles p (x,t) = 1 N N δ X i(x) t i=1 Algorithm outline 1 Initialization at time 0: X i 0 p 0 ( ) 2 At each discrete time step: Importance sampling (Bayes update step) Resampling (for variance reduction) Innovation error, feedback? And most importantly, is this pretty? J. E. Handschin and D. Q. Mayne. Monte-Carlo techniques to estimate conditional expectation in nonlinear filtering. Int. J. Control (1969); N. Gordon, D. Salmond, A. Smith. Novel approach to nonlinear/non-gaussian Bayesian state estimation. IEE Proc. F Radar Signal Process (1993); J. Xiong. Particle approximation to filtering problems in continuous time. The Oxford handbook of nonlinear filtering (2011); A. Budhiraja, L. Chen, C. Lee. A survey of numerical methods for nonlinear filtering problems. Physica D (2007). Feedback Particle Filter Prashant Mehta 9 / 34

Feedback Particle Filter A control-oriented approach Signal & Observations dx t = a(x t )dt + db t (1) dz t = h(x t )dt + dw t (2) Motivation: Work of Huang, Caines and Malhame on Mean-field games (IEEE TAC 2007). Related approaches: D. Crisan and J. Xiong, Approximate McKean-Vlasov representations for a class of SPDEs. Stochastics (2009); S. K. Mitter and N. J. Newton. A variational approach to nonlinear estimation. SIAM J. Control Optimiz. (2003); F. Daum and J. Huang. Generalized particle flow for nonlinear filters. Proc. SPIE (2010); S. Reich, A dynamical systems framework for intermittent data assimilation. BIT Numer. Math. (2011). Feedback Particle Filter Prashant Mehta 10 / 34 P

Feedback Particle Filter A control-oriented approach Controlled system (N particles): Signal & Observations dx t = a(x t )dt + db t (1) dz t = h(x t )dt + dw t (2) dxt i = a(xt)dt i + db i t + dut i, i = 1,...,N (3) }{{} mean-field control {B i t} N i=1 are ind. standard white noises. Motivation: Work of Huang, Caines and Malhame on Mean-field games (IEEE TAC 2007). Related approaches: D. Crisan and J. Xiong, Approximate McKean-Vlasov representations for a class of SPDEs. Stochastics (2009); S. K. Mitter and N. J. Newton. A variational approach to nonlinear estimation. SIAM J. Control Optimiz. (2003); F. Daum and J. Huang. Generalized particle flow for nonlinear filters. Proc. SPIE (2010); S. Reich, A dynamical systems framework for intermittent data assimilation. BIT Numer. Math. (2011). Feedback Particle Filter Prashant Mehta 10 / 34 P

Feedback Particle Filter A control-oriented approach Controlled system (N particles): Variational approach: Signal & Observations dx t = a(x t )dt + db t (1) dz t = h(x t )dt + dw t (2) dxt i = a(xt)dt i + db i t + dut i, i = 1,...,N (3) }{{} mean-field control 1. Gradient flow construction: Nonlinear filter is shown to be a gradient flow (steepest descent) 2. Optimal transport: Derivation of the feedback particle filter {B i t} N i=1 are ind. standard white noises. Motivation: Work of Huang, Caines and Malhame on Mean-field games (IEEE TAC 2007). Related approaches: D. Crisan and J. Xiong, Approximate McKean-Vlasov representations for a class of SPDEs. Stochastics (2009); S. K. Mitter and N. J. Newton. A variational approach to nonlinear estimation. SIAM J. Control Optimiz. (2003); F. Daum and J. Huang. Generalized particle flow for nonlinear filters. Proc. SPIE (2010); S. Reich, A dynamical systems framework for intermittent data assimilation. BIT Numer. Math. (2011). Feedback Particle Filter Prashant Mehta 10 / 34 P

Update Step How does feedback particle filter implement Bayes rule? Feedback particle filter Kalman filter Observation: dz t = h(x t )dt + dw t dz t = γx t dt + dw t Yang, Mehta and Meyn. Feedback Particle Filter. IEEE TAC (2013). Feedback Particle Filter Prashant Mehta 11 / 34 P

Update Step How does feedback particle filter implement Bayes rule? Feedback particle filter Kalman filter Observation: dz t = h(x t )dt + dw t dz t = γx t dt + dw t Prediction: dẑ i t = h(xi t )+ĥ 2 dt dẑ t = γ ˆX t dt ĥ = 1 N N i=1 h(xi t) Yang, Mehta and Meyn. Feedback Particle Filter. IEEE TAC (2013). Feedback Particle Filter Prashant Mehta 11 / 34 P

Update Step How does feedback particle filter implement Bayes rule? Feedback particle filter Kalman filter Observation: dz t = h(x t )dt + dw t dz t = γx t dt + dw t Prediction: dẑ i t = h(xi t )+ĥ 2 dt dẑ t = γ ˆX t dt ĥ = 1 N N i=1 h(xi t) Innov. error: dit i = dz t dẑt i di t = dz t dẑ t = dz t h(xi t )+ĥ 2 dt = dz t γ ˆX t dt Yang, Mehta and Meyn. Feedback Particle Filter. IEEE TAC (2013). Feedback Particle Filter Prashant Mehta 11 / 34 P

Update Step How does feedback particle filter implement Bayes rule? Feedback particle filter Kalman filter Observation: dz t = h(x t )dt + dw t dz t = γx t dt + dw t Prediction: dẑ i t = h(xi t )+ĥ 2 dt dẑ t = γ ˆX t dt ĥ = 1 N N i=1 h(xi t) Innov. error: dit i = dz t dẑt i di t = dz t dẑ t = dz t h(xi t )+ĥ 2 dt = dz t γ ˆX t dt Control: du i t = K(X i t) di i t du t = K di t Yang, Mehta and Meyn. Feedback Particle Filter. IEEE TAC (2013). Feedback Particle Filter Prashant Mehta 11 / 34 P

Update Step How does feedback particle filter implement Bayes rule? Feedback particle filter Kalman filter Observation: dz t = h(x t )dt + dw t dz t = γx t dt + dw t Prediction: dẑ i t = h(xi t )+ĥ 2 dt dẑ t = γ ˆX t dt ĥ = 1 N N i=1 h(xi t) Innov. error: dit i = dz t dẑt i di t = dz t dẑ t = dz t h(xi t )+ĥ 2 dt = dz t γ ˆX t dt Control: du i t = K(X i t) di i t du t = K di t Gain: K is a solution of a linear BVP K is the Kalman gain Main Result: FPF is an exact algorithm (in the mean-field, N, limit). Yang, Mehta and Meyn. Feedback Particle Filter. IEEE TAC (2013). Feedback Particle Filter Prashant Mehta 11 / 34 P

Variance Reduction Filtering for a linear model. Mean-square error: 1 T ( T 0 Σ (N) t ) 2 Σ t dt Σ t MSE 10 1 Bootstrap (BPF) 10 2 Feedback (FPF) 10 3 10 2 10 3 N (number of particles) Feedback Particle Filter Prashant Mehta 12 / 34 P

The Next Few Slides Results 1 and 2 1. Gradient flow construction: 1 Nonlinear filter is shown to be a gradient flow 2. Optimal transport: 1 Derivation of feedback particle filter Poisson s equation is central to both 1 and 2 Details appear in: Laugesen, Mehta, Meyn and Raginsky. Poisson s equation in nonlinear filtering. SIAM J. Control Optimiz. (2015); Also see: Yang, Mehta and Meyn. Feedback particle filter. IEEE Trans. Automat. Control (2013); Yang, Laugesen, Mehta and Meyn. Multivariable feedback particle filter. Automatica (To Appear). Feedback Particle Filter Prashant Mehta 13 / 34 P

Poisson s Equation Review of various mathematical forms Strong form: }{{} Laplacian. = 2 φ(x) = h(x) c on domain Ω Feedback Particle Filter Prashant Mehta 14 / 34 P

Poisson s Equation Review of various mathematical forms Strong form: }{{} Laplacian. = 2 φ(x) = h(x) c on domain Ω Weak form: φ ψ dx = (h c)ψ dx test fns. ψ H 1 Feedback Particle Filter Prashant Mehta 14 / 34 P

Poisson s Equation Review of various mathematical forms Strong form: }{{} Laplacian. = 2 φ(x) = h(x) c on domain Ω Weak form: Generalization: φ ψ dx = (h c)ψ dx test fns. ψ H 1 φ ψ p(x)dx = (h c)ψ p(x)dx ψ H 1 Feedback Particle Filter Prashant Mehta 14 / 34 P

Poisson s Equation in Physics This equation is fundamental to many fields! Electric potential: 1 4π 2 φ = ρ charge density Gravitational potential: Temperature: 1 4πG 2 φ = ρ mass density κ 2 φ = q heat-flux density Walter Strauss. Partial Differential Equations. Wiley (1992). Feedback Particle Filter Prashant Mehta 15 / 34 P

Gradient flow An elementary example Time stepping procedure x (t + t) = arg min y 1 2 y x (t) 2 + t h(y) Feedback Particle Filter Prashant Mehta 16 / 34 P

Gradient flow An elementary example Time stepping procedure x (t + t) = arg min y 1 2 y x (t) 2 + t h(y) Calculus 101: x (t + t) = x (t) t h(x (t + t)) Feedback Particle Filter Prashant Mehta 16 / 34 P

Gradient flow An elementary example Time stepping procedure x (t + t) = arg min y 1 2 y x (t) 2 + t h(y) Calculus 101: Cont. limit: x (t + t) = x (t) t h(x (t + t)) dx dt = h(x ). Feedback Particle Filter Prashant Mehta 16 / 34 P

Gradient flow An elementary example Time stepping procedure x (t + t) = arg min y 1 2 y x (t) 2 }{{} metric + t h(y) }{{} min Feedback Particle Filter Prashant Mehta 17 / 34 P

Gradient flow An elementary example Time stepping procedure x (t + t) = arg min y 1 2 y x (t) 2 }{{} metric + t h(y) }{{} min Calc. of Variation: x (t + t),ψ = x (t),ψ t h(x (t + t)),ψ Feedback Particle Filter Prashant Mehta 17 / 34 P

Gradient flow An elementary example Time stepping procedure x (t + t) = arg min y 1 2 y x (t) 2 }{{} metric + t h(y) }{{} min Calc. of Variation: Cont. limit: x (t + t),ψ = x (t),ψ t h(x (t + t)),ψ t x (t),ψ = x (0),ψ h(x (s)),ψ ds. 0 Feedback Particle Filter Prashant Mehta 17 / 34 P

Gradient Flow Interpretation of Heat Equation 1998 paper of Jordon, Kinderlehrer and Otto Time stepping procedure p 1 t+ t = arg min ρ P 2 W2 2 (ρ,p t ) + t ρ(x) lnρ(x) dx Jordon, Kinderlehrer and Otto. The variational formulation of the Fokker-Planck equation. SIAM J. Math. Anal., 29 (1998) Feedback Particle Filter Prashant Mehta 18 / 34 P

Gradient Flow Interpretation of Heat Equation 1998 paper of Jordon, Kinderlehrer and Otto Time stepping procedure p 1 t+ t = arg min ρ P 2 W2 2 (ρ,p t ) + t ρ(x) lnρ(x) dx E-L equation:... Jordon, Kinderlehrer and Otto. The variational formulation of the Fokker-Planck equation. SIAM J. Math. Anal., 29 (1998) Feedback Particle Filter Prashant Mehta 18 / 34 P

Gradient Flow Interpretation of Heat Equation 1998 paper of Jordon, Kinderlehrer and Otto Time stepping procedure p 1 t+ t = arg min ρ P 2 W2 2 (ρ,p t ) + t ρ(x) lnρ(x) dx E-L equation:... Cont. limit: p t = 2 p Jordon, Kinderlehrer and Otto. The variational formulation of the Fokker-Planck equation. SIAM J. Math. Anal., 29 (1998) Feedback Particle Filter Prashant Mehta 18 / 34 P

Gradient Flow for Nonlinear Filter Construction via a time-stepping procedure Signal & Observations dx t = 0, dz t = h(x t )dt + dw t Time stepping procedure p t+ t = arg min ρ P where Y t := Z t+ t Z t. t D(ρ p t ) + t ρ(x)(y t h(x)) 2 dx, 2 Laugesen, Mehta, Meyn and Raginsky. Poisson s Equation in Nonlinear Filtering. SIAM J. Control Optim (2015) Feedback Particle Filter Prashant Mehta 19 / 34 P

Gradient Flow Interpretation of Nonlinear Filter Construction via a time-stepping procedure Time stepping procedure p t+ t = arg min ρ P D(ρ p t ) + t ρ(x)(y t h(x)) 2 dx, 2 S. K. Mitter and N. J. Newton. A variational approach to nonlinear estimation. SIAM J. Control Optimiz. (2003); Feedback Particle Filter Prashant Mehta 20 / 34 P

Gradient Flow Interpretation of Nonlinear Filter Construction via a time-stepping procedure Time stepping procedure p t+ t = arg min ρ P D(ρ p t ) + t ρ(x)(y t h(x)) 2 dx, 2 E-L equation:... S. K. Mitter and N. J. Newton. A variational approach to nonlinear estimation. SIAM J. Control Optimiz. (2003); Feedback Particle Filter Prashant Mehta 20 / 34 P

Gradient Flow Interpretation of Nonlinear Filter Construction via a time-stepping procedure Time stepping procedure p t+ t = arg min ρ P D(ρ p t ) + t ρ(x)(y t h(x)) 2 dx, 2 E-L equation:... Cont. limit: dp = (h ĥ)(dz t ĥdt)p S. K. Mitter and N. J. Newton. A variational approach to nonlinear estimation. SIAM J. Control Optimiz. (2003); Feedback Particle Filter Prashant Mehta 20 / 34 P

Gradient Flow Interpretation of Nonlinear Filter Poisson s equation? E-L equation: E p t+ t [ψ] = E p t [ψ] + E p t+ t [( Z t h t) h ς] Feedback Particle Filter Prashant Mehta 21 / 34 P

Gradient Flow Interpretation of Nonlinear Filter Poisson s equation? E-L equation: E p t+ t [ψ] = E p t [ψ] + E p t+ t [( Z t h t) h ς] Poisson s equation: (p t (x) ς(x)) = (ψ(x) ˆψ t )p t (x) Feedback Particle Filter Prashant Mehta 21 / 34 P

Gradient Flow Interpretation of Nonlinear Filter Poisson s equation? E-L equation: E p t+ t [ψ] = E p t [ψ] + E p t+ t [( Z t h t) h ς] Poisson s equation: (p t (x) ς(x)) = (ψ(x) ˆψ t )p t (x) Assumption: Spectral gap For some λ 0 > 0, and for all functions ψ H 1 with E p 0 [ψ] = 0, ψ(x) 2 p 0 (x)dx 1 ψ(x) 2 p 0 λ (x)dx. [PI(λ 0)] 0 Feedback Particle Filter Prashant Mehta 21 / 34 P

Gradient Flow Interpretation of Nonlinear Filter Result 1: Derivation of nonlinear filter Result 1 Certain Technical conditions. The density p is a weak solution of the nonlinear filter with prior p 0. That is, for any test function ψ C c(r d ), t ψ,p t = ψ,p 0 + (h ĥ s )(dz s ĥ s ds)ψ,p s, 0 where ψ,p t =. ψ(x)p (x,t)dx. Feedback Particle Filter Prashant Mehta 22 / 34 P

Optimal Transport Result 2: Derivation of feedback particle filter Optimization problem J (N) (s ). = min s ( I tn (s # tn (p t n )) t ) t n 2 Y2 t n, Feedback Particle Filter Prashant Mehta 23 / 34 P

Optimal Transport Result 2: Derivation of feedback particle filter Optimization problem J (N) (s ). = min s ( I tn (s # tn (p t n )) t ) t n 2 Y2 t n, I t (ρ) =. D(ρ p t ) + t ρ(x)(y t h(x)) 2 dx 2 Feedback Particle Filter Prashant Mehta 23 / 34 P

Optimal Transport Result 2: Derivation of feedback particle filter Optimization problem J (N) (s ). = min s ( I tn (s # tn (p t n )) t ) t n 2 Y2 t n, I t (ρ) =. D(ρ p t ) + t ρ(x)(y t h(x)) 2 dx 2 s t # : Optimal transport Feedback Particle Filter Prashant Mehta 23 / 34 P

Feedback Particle Filter Algorithm summary Signal: Observations: dx t = a(x t )dt + db t dz t = h(x t )dt + dw t Feedback Particle Filter Prashant Mehta 24 / 34 P

Feedback Particle Filter Algorithm summary Signal: Observations: dx t = a(x t )dt + db t dz t = h(x t )dt + dw t Problem: Approximate the posterior distribution p (x,t). Feedback Particle Filter Prashant Mehta 24 / 34 P

Feedback Particle Filter Algorithm summary Signal: Observations: dx t = a(x t )dt + db t dz t = h(x t )dt + dw t Problem: Approximate the posterior distribution p (x,t). FPF Algo.: dxt i = a(xt)dt i + db i t ( + K(X i,t) dz t 1 ) 2 (h(xi t) + ĥ t )dt }{{} FPF control Feedback Particle Filter Prashant Mehta 24 / 34 P

Boundary Value Problem Euler-Lagrange equation for the variational problem Multi-dimensional boundary value problem Gain Fn.: K = φ (p φ) = (h ĥ)p solved at each time-step. Feedback Particle Filter Prashant Mehta 25 / 34 P

Boundary Value Problem Euler-Lagrange equation for the variational problem Multi-dimensional boundary value problem Gain Fn.: K = φ (p φ) = (h ĥ)p solved at each time-step. Linear case: Feedback Particle Filter Prashant Mehta 25 / 34 P

Summary Kalman Filter Feedback Particle Filter - + - + Kalman Filter Innovation Error: di t = dz t h( ˆX)dt Feedback Particle Filter Innovation Error: dit i = dz t 1 ( ) h(x i 2 t ) + ĥ t dt Gain Function: K = Kalman Gain Gain Function: K is solution of a linear BVP. Yang, Mehta and Meyn. Feedback Particle Filter. IEEE TAC (2013). Feedback Particle Filter Prashant Mehta 26 / 34 P

Coupled Oscillators Kuramoto model dθ i t = ( ω i + κ N ) N sin(θt j θt i ) j=1 dt + σ dξ i t, i = 1,...,N ω i taken from distribution g(ω) over [1 γ,1 + γ] γ measures the heterogeneity of the population κ measures the strength of coupling Y. Kuramoto. Self-entrainment of a population of coupled nonlinear oscillators (1975); Strogatz and Mirollo. Stability of incoherence in a population of coupled oscillators. J. Stat. Phy. (1991); N. Kopell and G. B. Ermentrout. Symmetry and phaselocking in chains of weakly coupled oscillators. Commun. Pure Appl. Math. (1986) Feedback Particle Filter Prashant Mehta 27 / 34 P

Coupled Oscillators Kuramoto model dθ i t = ( ω i + κ N ) N sin(θt j θt i ) j=1 dt + σ dξ i t, i = 1,...,N ω i taken from distribution g(ω) over [1 γ,1 + γ] γ measures the heterogeneity of the population κ measures the strength of coupling 1-1 1+ Y. Kuramoto. Self-entrainment of a population of coupled nonlinear oscillators (1975); Strogatz and Mirollo. Stability of incoherence in a population of coupled oscillators. J. Stat. Phy. (1991); N. Kopell and G. B. Ermentrout. Symmetry and phaselocking in chains of weakly coupled oscillators. Commun. Pure Appl. Math. (1986) Feedback Particle Filter Prashant Mehta 27 / 34 P

Coupled Oscillators Kuramoto model dθ i t = ( ω i + κ N ) N sin(θt j θt i ) j=1 dt + σ dξ i t, i = 1,...,N ω i taken from distribution g(ω) over [1 γ,1 + γ] γ measures the heterogeneity of the population κ measures the strength of coupling 0.3 Synchrony 0.2 Incoherence 0.1 0 0.1 0.2 Y. Kuramoto. Self-entrainment of a population of coupled nonlinear oscillators (1975); Strogatz and Mirollo. Stability of incoherence in a population of coupled oscillators. J. Stat. Phy. (1991); N. Kopell and G. B. Ermentrout. Symmetry and phaselocking in chains of weakly coupled oscillators. Commun. Pure Appl. Math. (1986) Feedback Particle Filter Prashant Mehta 27 / 34 P

Hodgkin-Huxley type Neuron model Normal form reduction C dv = g T m 2 dt (V) h (V E T ) g h r (V E h )... dh dt = h (V) h τ h (V) dr dt = r (V) r τ r (V) Voltage 100 50 0 50 100 150 Neural spike train 2000 2200 2400 2600 2800 3000 3200 3400 3600 3800 4000 time J. Guckenheimer. Isochrons and phaseless sets. J. Math. Biol. (1975); Brown, Moehlis and Holmes. On the phase reduction and response dynamics of neural oscillator populations. Neural Computation (2004); E. M. Izhikevich. Dynamical Systems in Neuroscience. in Chapter 10. The MIT Press (2006). Feedback Particle Filter Prashant Mehta 28 / 34 P

r Hodgkin-Huxley type Neuron model Normal form reduction C dv = g T m 2 dt (V) h (V E T ) g h r (V E h )... dh dt = h (V) h τ h (V) dr dt = r (V) r τ r (V) Voltage 100 50 0 50 100 150 Neural spike train 2000 2200 2400 2600 2800 3000 3200 3400 3600 3800 4000 time r 0.4 0.3 Limit cyle 0.2 0.1 0 1 0.8 0.6 0.4 h h 0.2 0 100 50 V 0 v 50 100 J. Guckenheimer. Isochrons and phaseless sets. J. Math. Biol. (1975); Brown, Moehlis and Holmes. On the phase reduction and response dynamics of neural oscillator populations. Neural Computation (2004); E. M. Izhikevich. Dynamical Systems in Neuroscience. in Chapter 10. The MIT Press (2006). Feedback Particle Filter Prashant Mehta 28 / 34 P

r Hodgkin-Huxley type Neuron model Normal form reduction C dv = g T m 2 dt (V) h (V E T ) g h r (V E h )... dh dt = h (V) h τ h (V) dr dt = r (V) r τ r (V) Voltage 100 50 0 50 100 150 Neural spike train 2000 2200 2400 2600 2800 3000 3200 3400 3600 3800 4000 time r 0.4 0.3 Limit cyle 0.2 0.1 Normal form reduction 0 1 0.8 0.6 0.4 h h 0.2 0 100 50 V 0 v 50 100 θ i = ω i + u i Φ(θ i ) J. Guckenheimer. Isochrons and phaseless sets. J. Math. Biol. (1975); Brown, Moehlis and Holmes. On the phase reduction and response dynamics of neural oscillator populations. Neural Computation (2004); E. M. Izhikevich. Dynamical Systems in Neuroscience. in Chapter 10. The MIT Press (2006). Feedback Particle Filter Prashant Mehta 28 / 34 P

Gait Cycle Signal model Stance phase Swing phase Model (Noisy oscillator) dθ t = ω 0 dt }{{} + noise natural frequency Feedback Particle Filter Prashant Mehta 29 / 34 P

Simulation Results Solution of the Estimation of Gait Cycle Problem noisy measurements feedback particle filter dynamics [Click to play the movie] estimate Tilton, Hsiao-Wecksler and Mehta. Filtering with rhythms: Application to estimation of gait cycle. American Control Conference (2012). Feedback Particle Filter Prashant Mehta 30 / 34 P

Geometric Control Locomotion Systems shape variables: x 1,x 2 group variable: ψ 3-body system P. S. Krishnaprasad. Geometric phases and optimal reconfiguration for multibody systems. UM Tech. Report (1990); P. S. Krishnaprasad. Motion control and coupled oscillators. Procs. of Symp. on Motion, Control & Geometry. Natl. Acad. Sciences (1995); R. Brockett. Pattern generation and the control of nonlinear systems. IEEE TAC (2003); S. Kelly and R. Murray. Geometric phases and robotic locomotion. J. Robotic Systems (1995); R. Murray and S. Sastry. Nonholonomic motion planning: Steering using sinusoids. IEEE TAC (1993); J. Blair and T. Iwasaki. Optimal gaits for mechanical rectifier systems. IEEE TAC (2011); Feedback Particle Filter Prashant Mehta 31 / 34 P

Geometric Control Locomotion Systems shape variables: x 1,x 2 group variable: ψ 3-body system ẍ = f (x,ẋ,τ) τ: torque input P. S. Krishnaprasad. Geometric phases and optimal reconfiguration for multibody systems. UM Tech. Report (1990); P. S. Krishnaprasad. Motion control and coupled oscillators. Procs. of Symp. on Motion, Control & Geometry. Natl. Acad. Sciences (1995); R. Brockett. Pattern generation and the control of nonlinear systems. IEEE TAC (2003); S. Kelly and R. Murray. Geometric phases and robotic locomotion. J. Robotic Systems (1995); R. Murray and S. Sastry. Nonholonomic motion planning: Steering using sinusoids. IEEE TAC (1993); J. Blair and T. Iwasaki. Optimal gaits for mechanical rectifier systems. IEEE TAC (2011); Feedback Particle Filter Prashant Mehta 31 / 34 P

Geometric Control Locomotion Systems shape variables: x 1,x 2 group variable: ψ 3-body system ẍ = f (x,ẋ,τ) τ: torque input ψ = a 1 (x)ẋ 1 + a 2 (x)ẋ 2 Reconstruction equation P. S. Krishnaprasad. Geometric phases and optimal reconfiguration for multibody systems. UM Tech. Report (1990); P. S. Krishnaprasad. Motion control and coupled oscillators. Procs. of Symp. on Motion, Control & Geometry. Natl. Acad. Sciences (1995); R. Brockett. Pattern generation and the control of nonlinear systems. IEEE TAC (2003); S. Kelly and R. Murray. Geometric phases and robotic locomotion. J. Robotic Systems (1995); R. Murray and S. Sastry. Nonholonomic motion planning: Steering using sinusoids. IEEE TAC (1993); J. Blair and T. Iwasaki. Optimal gaits for mechanical rectifier systems. IEEE TAC (2011); Feedback Particle Filter Prashant Mehta 31 / 34 P

Control of Locomotion Gaits 2-body System ψ = f (x)ẋ A. Taghvaei, S. Hutchinson and P. G. Mehta. A coupled-oscillators-based control architecture for locomotory gaits. IEEE CDC (2014). Feedback Particle Filter Prashant Mehta 32 / 34 P

Control of Locomotion Gaits 2-body System ψ = f (x)ẋ = f (θ) A. Taghvaei, S. Hutchinson and P. G. Mehta. A coupled-oscillators-based control architecture for locomotory gaits. IEEE CDC (2014). Feedback Particle Filter Prashant Mehta 32 / 34 P

Control of Locomotion Gaits 2-body System ψ = f (x)ẋ = f (θ,u) A. Taghvaei, S. Hutchinson and P. G. Mehta. A coupled-oscillators-based control architecture for locomotory gaits. IEEE CDC (2014). Feedback Particle Filter Prashant Mehta 32 / 34 P

Control of Locomotion Gaits 2-body System ψ = f (x)ẋ = f (θ,u) [ min E u [0,T] ψ(t) ψ(0) }{{} Geom. phase + 1 T ] u(t) 2 dt 2ε 0 A. Taghvaei, S. Hutchinson and P. G. Mehta. A coupled-oscillators-based control architecture for locomotory gaits. IEEE CDC (2014). Feedback Particle Filter Prashant Mehta 32 / 34 P

2-body system, Simulation Result x(t) Observation: y(t) Particles True Phase x π 3 0 π 3 t 0.2 0.0 0.2 0.4 q1 0.6 0.8 1.0 open loop: q1(t) close loop: q1(t) 0 10 20 30 40 50 60 70 80 t [Click to play the movie] A. Taghvaei, S. Hutchinson and P. G. Mehta. A coupled-oscillators-based control architecture for locomotory gaits. IEEE CDC (2014). Feedback Particle Filter Prashant Mehta 33 / 34 P

Acknowledgement Students in red 1. Feedback particle filter: Tao Yang, Rick Laugesen, Sean Meyn, Max Raginsky 2. Coupled oscillators for estimation: Adam Tilton, Shane Ghiotto, Liz Hsiao-Wecksler 3. Coupled oscillators for control: Amirhossein Taghvaei, Seth Hutchinson Research supported by NSF Feedback Particle Filter Prashant Mehta 34 / 34 P