Nonlinear and/or Non-normal Filtering. Jesús Fernández-Villaverde University of Pennsylvania
|
|
- Rebecca Hodges
- 5 years ago
- Views:
Transcription
1 Nonlinear and/or Non-normal Filtering Jesús Fernández-Villaverde University of Pennsylvania 1
2 Motivation Nonlinear and/or non-gaussian filtering, smoothing, and forecasting (NLGF) problems are pervasive in economics. Macroeconomics: evaluating likelihood of DSGE models Finance: time-varying variance of time series. However, NLGF is a complicated endeavor with no simple and exact algorithm. 2
3 Environment Discrete time t {1, 2,...}. States S t. Initial state S 0 is either known or it comes from p (S 0 ; γ). Properties of p (S 0 ; γ)? Stationarity? 3
4 Nonlinear and/or Non-gaussian State Space Representations Transition equation: S t = f (S t 1,W t ; γ) Measurement equation: Y t = g (S t,v t ; γ) Interpretation. 4
5 Shocks {W t } and {V t } are independent of each other. W t and V t have zero mean. ThevarianceofW t is given by R t and the variance of V t by Q t. Generalizations? 5
6 Conditional Densities From S t = f (S t 1,W t ; γ), we can compute p (S t S t 1 ; γ). From Y t = g (S t,v t ; γ), we can compute p (Y t S t ; γ). From S t = f (S t 1,W t ; γ) andy t = g (S t,v t ; γ), we have: Y t = g (f (S t 1,W t ; γ),v t ; γ) and hence we can compute p (Y t S t 1 ; γ). 6
7 Filtering, Smoothing, and Filtering Filtering: we are concerned with what we have learned up to current observation. Smoothing: we are concerned with what we learn with the full sample. Forecasting: we are concerned with future realizations. 7
8 Goal of Filtering I Compute conditional densities: p ³ s t y t 1 ; γ and p ³ s t y t ; γ. Why? 1. It allows probability statements regarding the situation of the system. 2. Compute conditional moments: mean, s t t and s t t 1 and variances P t t and P t t Other functions of the states. 8
9 Goals of Filtering II Compute condition the likelihood function of a sequence of realizations of the observable y T at a particular parameter value γ: p ³ y T ; γ Given the markov structure of our state space representation, we can factorize the likelihood function as p ³ y T ; γ = TY t=1 p ³ y t y t 1 ; γ 9
10 Goals of Filtering III Then, p ³ y T ; γ = p (y 1 γ) = Z TY t=1 p ³ y t y t 1 ; γ p (y 1 s 0 ; γ) ds 0 T Y t=1 Z p (y t s t ; γ) p ³ s t y t 1 ; γ ds t Hence, knowledge of n p ³ s t y t 1 ; γ o T t=1 and p (s 0; γ) allow the evaluation of the likelihood of the model. 10
11 Two Fundamental Tools 1. Chapman-Kolmogorov equation: p ³ s t y t 1 ; γ = Z p (s t s t 1 ; γ) p ³ s t 1 y t 1 ; γ ds t 1 2. Bayes theorem: p ³ s t y t ; γ = p (y t s t ; γ) p ³ s t y t 1 ; γ p ³ y t y t 1 ; γ where: p ³ y t y t 1 ; γ = Z p (y t s t ; γ) p ³ s t y t 1 ; γ ds t 11
12 Interpretation 1. Chapman-Kolmogorov equation is one-step ahead predictor. 2. Bayes theorem updates the conditional density of states given the new observation. 12
13 Recursion for p ³ s t y t ; γ Combining the Chapman-Kolmogorov and the Bayes theorem: p ³ R p (st s t y t s t 1 ; γ) p ³ s t 1 y ; γ t 1 ; γ ds t 1 = R n R p (st s t 1 ; γ) p ³ s t 1 y t 1 o p (y t s t ; γ) ; γ ds t 1 p (yt s t ; γ) ds t To initiate that recursion, we only need a value for s 0 or p (s 0 ; γ). Applying the Chapman-Kolmogorov equation once more to the outcome of the recursion, the get the sequence of n p ³ s t y t 1 ; γ o T t=1 to evaluate the likelihood function. 13
14 Smoothing We are interested on the distribution of the state conditional on all the observations, on p ³ s t y T ; γ and p ³ y t y T ; γ. We compute: p ³ s t y T ; γ = p ³ s t y t ; γ Z p ³ s t+1 y T ; γ p (s t+1 s t ; γ) p (s t+1 y t ds t+1 ; γ) a backward recursion that we initialize with p ³ s T y T ; γ and the sequences n p ³ s t y t ; γ o T t=1 and n p ³ s t y t 1 ; γ o T we obtained from t=1 filtering. 14
15 Forecasting We apply the Chapman-Kolmogorov equation recursively, we can get p ³ s t+j y t ; γ, j 1. Integrating recursively: Z p ³ y l+1 y l ; γ = p y l+1 s l+1 ; γ p ³ s l+1 y l ; γ ds l+1 from t +1tot + j, wegetp ³ y t+j y T ; γ. Clearly smoothing and forecasting require to solve the filtering problem first! 15
16 Problem of Filtering We have the recursion p ³ R p (st s t y t s t 1 ; γ) p ³ s t 1 y ; γ t 1 ; γ ds t 1 = R n R p (st s t 1 ; γ) p ³ s t 1 y t 1 o p (y t s t ; γ) ; γ ds t 1 p (yt s t ; γ) ds t A lot of complicated and high dimensional integrals. In general, we do not have close for solution for them. Nonlinear and non-gaussianity translate, spread, and deform (TSD) the conditional densities in ways that impossibilities to accommodate them with any known parametric family. 16
17 Exception There is one exception: linear and gaussian case. Why? Because if the system is linear and gaussian, all the conditional probabilities are also gaussian. Linear and gaussian state spaces models translate and spread the conditional distributions, but they do not deform them. For gaussian distributions, we only need to track mean and variance (sufficient statistics). Kalman filter accomplishes this goal efficiently. 17
18 Different Approaches Deterministic filtering: 1. Kalman family. 2. Grid-based filtering. Simulation filtering: 1. McMc. 2. Sequential Monte Carlo. 18
19 Kalman Family of Filters Use ideas of Kalman filtering to NLGF problems. Non-optimal filters. Different implementations: 1. Extended Kalman filter. 2. Iterated Extended Kalman filter. 3. Second-order Extended Kalman filter. 4. Unscented Kalman filter. 19
20 The Extended Kalman Filter EKF is historically the first descendant of the Kalman filter. EKF deals with nonlinearities with a first order approximation to the system and applying the Kalman filter to this approximation. Non-gaussianities are ignored. 20
21 Algorithm I Given s t 1 t 1 : s t t 1 = f ³ s t 1 t 1, 0; γ Then: P t t 1 = Q t 1 + F t P t 1 t 1 F 0 t where F t = df (S t 1,W t ; γ) ds t 1 St 1 =s t 1 t 1,W t =0 21
22 Algorithm II Kalman gain, K t,is: K t = P t t 1 G 0 ³ t Gt P t t 1 G 0 t + R t 1 where G t = dg (S t 1,v t ; γ) ds t 1 St 1 =s t t 1,v t =0 Then s t t = s t t 1 + K t ³ yt g ³ s t t 1, 0; γ P t t = P t t 1 K t G t P t t 1 22
23 Problems of EKF 1. It ignores the non-gaussianities of W t and V t. 2. It ignores the non-gaussianities of states distribution. 3. Approximation error incurred by the linearization. 4. Biased estimate of the mean and variance. 5. We need to compute Jacobian and Hessians. As the sample size grows, those errors accumulate and the filter diverges. 23
24 Iterated Extended Kalman Filter I Compute s t t 1 and P t t 1 as in EKF. Iterate N times on: where K i t = P t t 1 G i0 t G i t = dg (S t 1,v t ; γ) ds t 1 ³ G i t P t t 1 G i0 t + R t 1 St 1 =s i t t 1,v t=0 and s i t t = s t t 1 + Ki t ³ yt g ³ s t t 1, 0; γ 24
25 Iterated Extended Kalman Filter II Why are we iterating? How many times? Then: ³ s t t = s t t 1 + K t yt g ³ s N t t 1, 0; γ P t t = P t t 1 K N t G N t P t t 1 25
26 Second-order Extended Kalman Filter We keep second-order terms of the Taylor expansion of transition and measurement. Theoretically, less biased than EKF. Messy algebra. In practice, not much improvement. 26
27 Unscented Kalman Filter I Recent proposal by Julier and Uhlmann (1996). Based around the unscented transform. A set of sigma points is selected to preserve some properties of the conditional distribution (for example, the first two moments). Then, those points are transformed and the properties of the new conditional distribution are computed. 27
28 Unscented Kalman Filter II The UKF computes the conditional mean and variance accurately up to a third order approximation if the shocks W t and V t are gaussian and up to a second order if they are not. The sigma points are chosen deterministically and not by simulation asinamontecarlomethod. The UKF has the advantage with respect to the EKF that no jacobian or hessians is required, objects that may be difficult to compute. 28
29 New State Variable We modify the state space by creating a new augmented state variable: S t = [S t,w t,v t ] that includes the pure state space and the two random variables W t and V t. We initialize the filter with s 0 0 = E (S t )=E (S 0, 0, 0) P 0 0 = P R Q 0 29
30 Sigma Points Let L be the dimension of the state variable S t. For t = 1, we calculate the 2L + 1 sigma points: S 0,t 1 t 1 = s t 1 t 1 S i,t 1 t 1 = s t 1 t 1 ³ (L + λ) P t 1 t for i =1,..., L S i,t 1 t 1 = s t 1 t 1 + ³ (L + λ) P t 1 t for i = L +1,...,2L 30
31 Parameters λ = α 2 (L + κ) L is a scaling parameter. α determines the spread of the sigma point and it must belong to the unit interval. κ is a secondary parameter usually set equal to zero. Notation for each of the elements of S: S i =[S s i, Sw i, Sv i ] for i =0,...,2L 31
32 Weights Weights for each point: W0 m λ = L + λ W0 c = λ L + λ + ³ 1 α 2 + β W m 0 = X c 0 = 1 2(L + λ) for i =1,...,2L β incorporates knowledge regarding the conditional distributions. For gaussian distributions, β = 2 is optimal. 32
33 Algorithm I: Prediction of States We compute the transition of the pure states: S s i,t t 1 = f ³ S s i,t t 1, Sw i,t 1 t 1 ; γ Weighted state s t t 1 = 2LX Wi m Ss i,t t 1 i=0 Weighted variance: P t t 1 = 2LX Wi c i=0 ³ S s i,t t 1 s t t 1 ³ S s i,t t 1 s t t
34 Algorithm II: Prediction of Observables Predicted sigma observables: Y i,t t 1 = g ³ S s i,t t 1, Sv i,t t 1 ; γ Predicted observable: y t t 1 = 2LX i=0 W m i Y i,t t 1 34
35 Algorithm III: Update Variance-covariance matrices: P yy,t = P xy,t = 2LX Wi c i=0 2LX Wi c i=0 ³ Yi,t t 1 y t t 1 ³ Yi,t t 1 y t t 1 0 ³ S s i,t t 1 s t t 1 ³ Yi,t t 1 y t t 1 0 Kalman gain: K t = P xy,t P 1 yy,t 35
36 Algorithm IV: Update Update of the state: s t t = s t t + K t ³ yt y t t 1 the update of the variance: P t t = P t t 1 + K t P yy,t K 0 t Finally: P t t = P t t R t Q t 36
37 Grid-Based Filtering Remember that we have the recursion p ³ R p (st s t y t s t 1 ; γ) p ³ s t 1 y ; γ t 1 ; γ ds t 1 = R n R p (st s t 1 ; γ) p ³ s t 1 y t 1 o p (y t s t ; γ) ; γ ds t 1 p (yt s t ; γ) ds t This recursion requires the evaluation of three integrals. This suggests the possibility of addressing the problem by computing those integrals by a deterministic procedure as a grid method. Kitagawa (1987) and Kramer and Sorenson (1988). 37
38 Grid-Based Filtering We divide the state space into N cells, with center point s i t, n s i t : i =1,...,No. We substitute the exact conditional densities by discrete densities that put all the mass at the points n s i N to i=1. We denote δ (x) is a dirac function with mass at 0. 38
39 Approximated Densities and Weights p ³ s t y t 1 ; γ p ³ s t y t ; γ ' ' NX i=1 NX i=1 ω i t t 1 δ ³ s t s i t ω i t t 1 δ ³ s t s i t X ω i N t t 1 = ω j t 1 t 1 p ³ s i t s j t 1 ; γ j=1 ω i ω i t t 1 t t = p ³ y t s i t ; γ P Nj=1 ω j t t 1 p ³ y t s j t ; γ 39
40 Approximated Recursion PNj=1 ω j t 1 t 1 p ³ s i t sj t 1 ; γ p ³ y t s i t ; γ p ³ s t y t ; γ = NX i=1 P Nj=1 PNj=1 ω j t 1 t 1 p ³ γ s i t sj t 1 ; p ³ y t s j γ δ t ; ³ st s i t Compare with p ³ s t y t ; γ = R p (st s t 1 ; γ) p ³ s t 1 y t 1 ; γ ds t 1 R n R p (st s t 1 ; γ) p ³ s t 1 y t 1 o p (y t s t ; γ) ; γ ds t 1 p (yt s t ; γ) ds t given that p ³ s t 1 y t 1 ; γ ' NX i=1 ω i t 1 t 1 δ ³ s i t 40
41 Problems Grid filters require a constant readjustment to small changes in the model or its parameter values. Too computationally expensive to be of any practical benefit beyond very low dimensions. Grid points are fixed ex ante and the results are very dependent on that choice. Can we overcome those difficulties and preserve the idea of integration? Yes, through Monte Carlo Integration. 41
Filtering and Likelihood Inference
Filtering and Likelihood Inference Jesús Fernández-Villaverde University of Pennsylvania July 10, 2011 Jesús Fernández-Villaverde (PENN) Filtering and Likelihood July 10, 2011 1 / 79 Motivation Introduction
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond Department of Biomedical Engineering and Computational Science Aalto University January 26, 2012 Contents 1 Batch and Recursive Estimation
More informationEKF, UKF. Pieter Abbeel UC Berkeley EECS. Many slides adapted from Thrun, Burgard and Fox, Probabilistic Robotics
EKF, UKF Pieter Abbeel UC Berkeley EECS Many slides adapted from Thrun, Burgard and Fox, Probabilistic Robotics Kalman Filter Kalman Filter = special case of a Bayes filter with dynamics model and sensory
More informationEKF, UKF. Pieter Abbeel UC Berkeley EECS. Many slides adapted from Thrun, Burgard and Fox, Probabilistic Robotics
EKF, UKF Pieter Abbeel UC Berkeley EECS Many slides adapted from Thrun, Burgard and Fox, Probabilistic Robotics Kalman Filter Kalman Filter = special case of a Bayes filter with dynamics model and sensory
More informationIntroduction to Unscented Kalman Filter
Introduction to Unscented Kalman Filter 1 Introdution In many scientific fields, we use certain models to describe the dynamics of system, such as mobile robot, vision tracking and so on. The word dynamics
More informationLecture 6: Bayesian Inference in SDE Models
Lecture 6: Bayesian Inference in SDE Models Bayesian Filtering and Smoothing Point of View Simo Särkkä Aalto University Simo Särkkä (Aalto) Lecture 6: Bayesian Inference in SDEs 1 / 45 Contents 1 SDEs
More informationROBOTICS 01PEEQW. Basilio Bona DAUIN Politecnico di Torino
ROBOTICS 01PEEQW Basilio Bona DAUIN Politecnico di Torino Probabilistic Fundamentals in Robotics Gaussian Filters Course Outline Basic mathematical framework Probabilistic models of mobile robots Mobile
More informationA new unscented Kalman filter with higher order moment-matching
A new unscented Kalman filter with higher order moment-matching KSENIA PONOMAREVA, PARESH DATE AND ZIDONG WANG Department of Mathematical Sciences, Brunel University, Uxbridge, UB8 3PH, UK. Abstract This
More informationAutonomous Navigation for Flying Robots
Computer Vision Group Prof. Daniel Cremers Autonomous Navigation for Flying Robots Lecture 6.2: Kalman Filter Jürgen Sturm Technische Universität München Motivation Bayes filter is a useful tool for state
More informationECE276A: Sensing & Estimation in Robotics Lecture 10: Gaussian Mixture and Particle Filtering
ECE276A: Sensing & Estimation in Robotics Lecture 10: Gaussian Mixture and Particle Filtering Lecturer: Nikolay Atanasov: natanasov@ucsd.edu Teaching Assistants: Siwei Guo: s9guo@eng.ucsd.edu Anwesan Pal:
More informationProbabilistic Fundamentals in Robotics. DAUIN Politecnico di Torino July 2010
Probabilistic Fundamentals in Robotics Gaussian Filters Basilio Bona DAUIN Politecnico di Torino July 2010 Course Outline Basic mathematical framework Probabilistic models of mobile robots Mobile robot
More informationSequential Monte Carlo Methods (for DSGE Models)
Sequential Monte Carlo Methods (for DSGE Models) Frank Schorfheide University of Pennsylvania, PIER, CEPR, and NBER October 23, 2017 Some References These lectures use material from our joint work: Tempered
More informationEstimating Macroeconomic Models: A Likelihood Approach
Estimating Macroeconomic Models: A Likelihood Approach Jesús Fernández-Villaverde University of Pennsylvania, NBER, and CEPR Juan Rubio-Ramírez Federal Reserve Bank of Atlanta Estimating Dynamic Macroeconomic
More informationThe Kalman Filter ImPr Talk
The Kalman Filter ImPr Talk Ged Ridgway Centre for Medical Image Computing November, 2006 Outline What is the Kalman Filter? State Space Models Kalman Filter Overview Bayesian Updating of Estimates Kalman
More informationNonlinear State Estimation! Extended Kalman Filters!
Nonlinear State Estimation! Extended Kalman Filters! Robert Stengel! Optimal Control and Estimation, MAE 546! Princeton University, 2017!! Deformation of the probability distribution!! Neighboring-optimal
More informationLecture 7: Optimal Smoothing
Department of Biomedical Engineering and Computational Science Aalto University March 17, 2011 Contents 1 What is Optimal Smoothing? 2 Bayesian Optimal Smoothing Equations 3 Rauch-Tung-Striebel Smoother
More informationNonlinear State Estimation! Particle, Sigma-Points Filters!
Nonlinear State Estimation! Particle, Sigma-Points Filters! Robert Stengel! Optimal Control and Estimation, MAE 546! Princeton University, 2017!! Particle filter!! Sigma-Points Unscented Kalman ) filter!!
More informationProbabilistic Graphical Models
Probabilistic Graphical Models Brown University CSCI 2950-P, Spring 2013 Prof. Erik Sudderth Lecture 13: Learning in Gaussian Graphical Models, Non-Gaussian Inference, Monte Carlo Methods Some figures
More informationA Time-Varying Threshold STAR Model of Unemployment
A Time-Varying Threshold STAR Model of Unemloyment michael dueker a michael owyang b martin sola c,d a Russell Investments b Federal Reserve Bank of St. Louis c Deartamento de Economia, Universidad Torcuato
More informationAutonomous Mobile Robot Design
Autonomous Mobile Robot Design Topic: Extended Kalman Filter Dr. Kostas Alexis (CSE) These slides relied on the lectures from C. Stachniss, J. Sturm and the book Probabilistic Robotics from Thurn et al.
More informationEfficient Monitoring for Planetary Rovers
International Symposium on Artificial Intelligence and Robotics in Space (isairas), May, 2003 Efficient Monitoring for Planetary Rovers Vandi Verma vandi@ri.cmu.edu Geoff Gordon ggordon@cs.cmu.edu Carnegie
More informationThe Unscented Particle Filter
The Unscented Particle Filter Rudolph van der Merwe (OGI) Nando de Freitas (UC Bereley) Arnaud Doucet (Cambridge University) Eric Wan (OGI) Outline Optimal Estimation & Filtering Optimal Recursive Bayesian
More informationKalman filtering and friends: Inference in time series models. Herke van Hoof slides mostly by Michael Rubinstein
Kalman filtering and friends: Inference in time series models Herke van Hoof slides mostly by Michael Rubinstein Problem overview Goal Estimate most probable state at time k using measurement up to time
More informationOn a Data Assimilation Method coupling Kalman Filtering, MCRE Concept and PGD Model Reduction for Real-Time Updating of Structural Mechanics Model
On a Data Assimilation Method coupling, MCRE Concept and PGD Model Reduction for Real-Time Updating of Structural Mechanics Model 2016 SIAM Conference on Uncertainty Quantification Basile Marchand 1, Ludovic
More informationDual Estimation and the Unscented Transformation
Dual Estimation and the Unscented Transformation Eric A. Wan ericwan@ece.ogi.edu Rudolph van der Merwe rudmerwe@ece.ogi.edu Alex T. Nelson atnelson@ece.ogi.edu Oregon Graduate Institute of Science & Technology
More informationThe Scaled Unscented Transformation
The Scaled Unscented Transformation Simon J. Julier, IDAK Industries, 91 Missouri Blvd., #179 Jefferson City, MO 6519 E-mail:sjulier@idak.com Abstract This paper describes a generalisation of the unscented
More informationA NEW NONLINEAR FILTER
COMMUNICATIONS IN INFORMATION AND SYSTEMS c 006 International Press Vol 6, No 3, pp 03-0, 006 004 A NEW NONLINEAR FILTER ROBERT J ELLIOTT AND SIMON HAYKIN Abstract A discrete time filter is constructed
More informationStatistics 910, #15 1. Kalman Filter
Statistics 910, #15 1 Overview 1. Summary of Kalman filter 2. Derivations 3. ARMA likelihoods 4. Recursions for the variance Kalman Filter Summary of Kalman filter Simplifications To make the derivations
More informationMarkov Chain Monte Carlo Methods for Stochastic
Markov Chain Monte Carlo Methods for Stochastic Optimization i John R. Birge The University of Chicago Booth School of Business Joint work with Nicholas Polson, Chicago Booth. JRBirge U Florida, Nov 2013
More informationGaussian Process Approximations of Stochastic Differential Equations
Gaussian Process Approximations of Stochastic Differential Equations Cédric Archambeau Dan Cawford Manfred Opper John Shawe-Taylor May, 2006 1 Introduction Some of the most complex models routinely run
More informationCSC487/2503: Foundations of Computer Vision. Visual Tracking. David Fleet
CSC487/2503: Foundations of Computer Vision Visual Tracking David Fleet Introduction What is tracking? Major players: Dynamics (model of temporal variation of target parameters) Measurements (relation
More informationMarkov Chain Monte Carlo Methods for Stochastic Optimization
Markov Chain Monte Carlo Methods for Stochastic Optimization John R. Birge The University of Chicago Booth School of Business Joint work with Nicholas Polson, Chicago Booth. JRBirge U of Toronto, MIE,
More informationParticle Filtering Approaches for Dynamic Stochastic Optimization
Particle Filtering Approaches for Dynamic Stochastic Optimization John R. Birge The University of Chicago Booth School of Business Joint work with Nicholas Polson, Chicago Booth. JRBirge I-Sim Workshop,
More informationSensor Fusion: Particle Filter
Sensor Fusion: Particle Filter By: Gordana Stojceska stojcesk@in.tum.de Outline Motivation Applications Fundamentals Tracking People Advantages and disadvantages Summary June 05 JASS '05, St.Petersburg,
More information(Extended) Kalman Filter
(Extended) Kalman Filter Brian Hunt 7 June 2013 Goals of Data Assimilation (DA) Estimate the state of a system based on both current and all past observations of the system, using a model for the system
More informationKalman Filter. Predict: Update: x k k 1 = F k x k 1 k 1 + B k u k P k k 1 = F k P k 1 k 1 F T k + Q
Kalman Filter Kalman Filter Predict: x k k 1 = F k x k 1 k 1 + B k u k P k k 1 = F k P k 1 k 1 F T k + Q Update: K = P k k 1 Hk T (H k P k k 1 Hk T + R) 1 x k k = x k k 1 + K(z k H k x k k 1 ) P k k =(I
More informationState Estimation of Linear and Nonlinear Dynamic Systems
State Estimation of Linear and Nonlinear Dynamic Systems Part III: Nonlinear Systems: Extended Kalman Filter (EKF) and Unscented Kalman Filter (UKF) James B. Rawlings and Fernando V. Lima Department of
More informationNonlinear Estimation Techniques for Impact Point Prediction of Ballistic Targets
Nonlinear Estimation Techniques for Impact Point Prediction of Ballistic Targets J. Clayton Kerce a, George C. Brown a, and David F. Hardiman b a Georgia Tech Research Institute, Georgia Institute of Technology,
More informationSequential Monte Carlo Methods for Bayesian Computation
Sequential Monte Carlo Methods for Bayesian Computation A. Doucet Kyoto Sept. 2012 A. Doucet (MLSS Sept. 2012) Sept. 2012 1 / 136 Motivating Example 1: Generic Bayesian Model Let X be a vector parameter
More informationROBUST CONSTRAINED ESTIMATION VIA UNSCENTED TRANSFORMATION. Pramod Vachhani a, Shankar Narasimhan b and Raghunathan Rengaswamy a 1
ROUST CONSTRINED ESTIMTION VI UNSCENTED TRNSFORMTION Pramod Vachhani a, Shankar Narasimhan b and Raghunathan Rengaswamy a a Department of Chemical Engineering, Clarkson University, Potsdam, NY -3699, US.
More informationLinear and nonlinear filtering for state space models. Master course notes. UdelaR 2010
Linear and nonlinear filtering for state space models Master course notes UdelaR 2010 Thierry CHONAVEL thierry.chonavel@telecom-bretagne.eu December 2010 Contents 1 Introduction 3 2 Kalman filter and non
More informationMonetary and Exchange Rate Policy Under Remittance Fluctuations. Technical Appendix and Additional Results
Monetary and Exchange Rate Policy Under Remittance Fluctuations Technical Appendix and Additional Results Federico Mandelman February In this appendix, I provide technical details on the Bayesian estimation.
More informationOpen Economy Macroeconomics: Theory, methods and applications
Open Economy Macroeconomics: Theory, methods and applications Lecture 4: The state space representation and the Kalman Filter Hernán D. Seoane UC3M January, 2016 Today s lecture State space representation
More information2D Image Processing. Bayes filter implementation: Kalman filter
2D Image Processing Bayes filter implementation: Kalman filter Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de
More informationTSRT14: Sensor Fusion Lecture 8
TSRT14: Sensor Fusion Lecture 8 Particle filter theory Marginalized particle filter Gustaf Hendeby gustaf.hendeby@liu.se TSRT14 Lecture 8 Gustaf Hendeby Spring 2018 1 / 25 Le 8: particle filter theory,
More information1 Kalman Filter Introduction
1 Kalman Filter Introduction You should first read Chapter 1 of Stochastic models, estimation, and control: Volume 1 by Peter S. Maybec (available here). 1.1 Explanation of Equations (1-3) and (1-4) Equation
More informationAn introduction to particle filters
An introduction to particle filters Andreas Svensson Department of Information Technology Uppsala University June 10, 2014 June 10, 2014, 1 / 16 Andreas Svensson - An introduction to particle filters Outline
More informationWORKING PAPER. Sigma point filters for dynamic nonlinear regime switching models NORGES BANK RESEARCH AUTHORS: ANDREW BINNING JUNIOR MAIH
WORKING PAPER Sigma point filters for dynamic nonlinear regime switching models NORGES BANK RESEARCH 1 215 AUTHORS: ANDREW BINNING JUNIOR MAIH Working papers fra Norges Bank, fra 1992/1 til 29/2 kan bestilles
More informationParticle Filtering for Data-Driven Simulation and Optimization
Particle Filtering for Data-Driven Simulation and Optimization John R. Birge The University of Chicago Booth School of Business Includes joint work with Nicholas Polson. JRBirge INFORMS Phoenix, October
More informationWhy do we care? Measurements. Handling uncertainty over time: predicting, estimating, recognizing, learning. Dealing with time
Handling uncertainty over time: predicting, estimating, recognizing, learning Chris Atkeson 2004 Why do we care? Speech recognition makes use of dependence of words and phonemes across time. Knowing where
More informationParticle Filters. Outline
Particle Filters M. Sami Fadali Professor of EE University of Nevada Outline Monte Carlo integration. Particle filter. Importance sampling. Degeneracy Resampling Example. 1 2 Monte Carlo Integration Numerical
More information2D Image Processing (Extended) Kalman and particle filter
2D Image Processing (Extended) Kalman and particle filter Prof. Didier Stricker Dr. Gabriele Bleser Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz
More informationSEQUENTIAL MONTE CARLO METHODS WITH APPLICATIONS TO COMMUNICATION CHANNELS. A Thesis SIRISH BODDIKURAPATI
SEQUENTIAL MONTE CARLO METHODS WITH APPLICATIONS TO COMMUNICATION CHANNELS A Thesis by SIRISH BODDIKURAPATI Submitted to the Office of Graduate Studies of Texas A&M University in partial fulfillment of
More informationThe Ensemble Kalman Filter:
p.1 The Ensemble Kalman Filter: Theoretical formulation and practical implementation Geir Evensen Norsk Hydro Research Centre, Bergen, Norway Based on Evensen 23, Ocean Dynamics, Vol 53, No 4 p.2 The Ensemble
More informationRESEARCH ARTICLE. Online quantization in nonlinear filtering
Journal of Statistical Computation & Simulation Vol. 00, No. 00, Month 200x, 3 RESEARCH ARTICLE Online quantization in nonlinear filtering A. Feuer and G. C. Goodwin Received 00 Month 200x; in final form
More informationA Comparitive Study Of Kalman Filter, Extended Kalman Filter And Unscented Kalman Filter For Harmonic Analysis Of The Non-Stationary Signals
International Journal of Scientific & Engineering Research, Volume 3, Issue 7, July-2012 1 A Comparitive Study Of Kalman Filter, Extended Kalman Filter And Unscented Kalman Filter For Harmonic Analysis
More informationConstrained State Estimation Using the Unscented Kalman Filter
16th Mediterranean Conference on Control and Automation Congress Centre, Ajaccio, France June 25-27, 28 Constrained State Estimation Using the Unscented Kalman Filter Rambabu Kandepu, Lars Imsland and
More informationLagrangian Data Assimilation and Manifold Detection for a Point-Vortex Model. David Darmon, AMSC Kayo Ide, AOSC, IPST, CSCAMM, ESSIC
Lagrangian Data Assimilation and Manifold Detection for a Point-Vortex Model David Darmon, AMSC Kayo Ide, AOSC, IPST, CSCAMM, ESSIC Background Data Assimilation Iterative process Forecast Analysis Background
More informationNON-LINEAR NOISE ADAPTIVE KALMAN FILTERING VIA VARIATIONAL BAYES
2013 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING NON-LINEAR NOISE ADAPTIVE KALMAN FILTERING VIA VARIATIONAL BAYES Simo Särä Aalto University, 02150 Espoo, Finland Jouni Hartiainen
More informationRECURSIVE OUTLIER-ROBUST FILTERING AND SMOOTHING FOR NONLINEAR SYSTEMS USING THE MULTIVARIATE STUDENT-T DISTRIBUTION
1 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEPT. 3 6, 1, SANTANDER, SPAIN RECURSIVE OUTLIER-ROBUST FILTERING AND SMOOTHING FOR NONLINEAR SYSTEMS USING THE MULTIVARIATE STUDENT-T
More informationLecture 9. Time series prediction
Lecture 9 Time series prediction Prediction is about function fitting To predict we need to model There are a bewildering number of models for data we look at some of the major approaches in this lecture
More informationAN EFFICIENT TWO-STAGE SAMPLING METHOD IN PARTICLE FILTER. Qi Cheng and Pascal Bondon. CNRS UMR 8506, Université Paris XI, France.
AN EFFICIENT TWO-STAGE SAMPLING METHOD IN PARTICLE FILTER Qi Cheng and Pascal Bondon CNRS UMR 8506, Université Paris XI, France. August 27, 2011 Abstract We present a modified bootstrap filter to draw
More informationTAKEHOME FINAL EXAM e iω e 2iω e iω e 2iω
ECO 513 Spring 2015 TAKEHOME FINAL EXAM (1) Suppose the univariate stochastic process y is ARMA(2,2) of the following form: y t = 1.6974y t 1.9604y t 2 + ε t 1.6628ε t 1 +.9216ε t 2, (1) where ε is i.i.d.
More informationLessons in Estimation Theory for Signal Processing, Communications, and Control
Lessons in Estimation Theory for Signal Processing, Communications, and Control Jerry M. Mendel Department of Electrical Engineering University of Southern California Los Angeles, California PRENTICE HALL
More information1 Bayesian Linear Regression (BLR)
Statistical Techniques in Robotics (STR, S15) Lecture#10 (Wednesday, February 11) Lecturer: Byron Boots Gaussian Properties, Bayesian Linear Regression 1 Bayesian Linear Regression (BLR) In linear regression,
More informationX t = a t + r t, (7.1)
Chapter 7 State Space Models 71 Introduction State Space models, developed over the past 10 20 years, are alternative models for time series They include both the ARIMA models of Chapters 3 6 and the Classical
More informationIntroduction to Particle Filters for Data Assimilation
Introduction to Particle Filters for Data Assimilation Mike Dowd Dept of Mathematics & Statistics (and Dept of Oceanography Dalhousie University, Halifax, Canada STATMOS Summer School in Data Assimila5on,
More informationWhy do we care? Examples. Bayes Rule. What room am I in? Handling uncertainty over time: predicting, estimating, recognizing, learning
Handling uncertainty over time: predicting, estimating, recognizing, learning Chris Atkeson 004 Why do we care? Speech recognition makes use of dependence of words and phonemes across time. Knowing where
More informationStatistical Inference and Methods
Department of Mathematics Imperial College London d.stephens@imperial.ac.uk http://stats.ma.ic.ac.uk/ das01/ 31st January 2006 Part VI Session 6: Filtering and Time to Event Data Session 6: Filtering and
More information2D Image Processing. Bayes filter implementation: Kalman filter
2D Image Processing Bayes filter implementation: Kalman filter Prof. Didier Stricker Dr. Gabriele Bleser Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche
More informationThe Ensemble Kalman Filter:
p.1 The Ensemble Kalman Filter: Theoretical formulation and practical implementation Geir Evensen Norsk Hydro Research Centre, Bergen, Norway Based on Evensen, Ocean Dynamics, Vol 5, No p. The Ensemble
More informationSequential Monte Carlo methods for filtering of unobservable components of multidimensional diffusion Markov processes
Sequential Monte Carlo methods for filtering of unobservable components of multidimensional diffusion Markov processes Ellida M. Khazen * 13395 Coppermine Rd. Apartment 410 Herndon VA 20171 USA Abstract
More informationLecture 6: Multiple Model Filtering, Particle Filtering and Other Approximations
Lecture 6: Multiple Model Filtering, Particle Filtering and Other Approximations Department of Biomedical Engineering and Computational Science Aalto University April 28, 2010 Contents 1 Multiple Model
More informationState-space Model. Eduardo Rossi University of Pavia. November Rossi State-space Model Fin. Econometrics / 53
State-space Model Eduardo Rossi University of Pavia November 2014 Rossi State-space Model Fin. Econometrics - 2014 1 / 53 Outline 1 Motivation 2 Introduction 3 The Kalman filter 4 Forecast errors 5 State
More informationLecture 4: Numerical Solution of SDEs, Itô Taylor Series, Gaussian Approximations
Lecture 4: Numerical Solution of SDEs, Itô Taylor Series, Gaussian Approximations Simo Särkkä Aalto University, Finland November 18, 2014 Simo Särkkä (Aalto) Lecture 4: Numerical Solution of SDEs November
More informationBayesian Monte Carlo Filtering for Stochastic Volatility Models
Bayesian Monte Carlo Filtering for Stochastic Volatility Models Roberto Casarin CEREMADE University Paris IX (Dauphine) and Dept. of Economics University Ca Foscari, Venice Abstract Modelling of the financial
More informationOn Continuous-Discrete Cubature Kalman Filtering
On Continuous-Discrete Cubature Kalman Filtering Simo Särkkä Arno Solin Aalto University, P.O. Box 12200. FI-00076 AALTO, Finland. (Tel: +358 50 512 4393; e-mail: simo.sarkka@aalto.fi) Aalto University,
More informationProbabilistic Graphical Models
Probabilistic Graphical Models Lecture 12 Dynamical Models CS/CNS/EE 155 Andreas Krause Homework 3 out tonight Start early!! Announcements Project milestones due today Please email to TAs 2 Parameter learning
More informationBayesian Dynamic Linear Modelling for. Complex Computer Models
Bayesian Dynamic Linear Modelling for Complex Computer Models Fei Liu, Liang Zhang, Mike West Abstract Computer models may have functional outputs. With no loss of generality, we assume that a single computer
More informationCONDENSATION Conditional Density Propagation for Visual Tracking
CONDENSATION Conditional Density Propagation for Visual Tracking Michael Isard and Andrew Blake Presented by Neil Alldrin Department of Computer Science & Engineering University of California, San Diego
More informationMarkov Chain Monte Carlo methods
Markov Chain Monte Carlo methods Tomas McKelvey and Lennart Svensson Signal Processing Group Department of Signals and Systems Chalmers University of Technology, Sweden November 26, 2012 Today s learning
More informationDynamic System Identification using HDMR-Bayesian Technique
Dynamic System Identification using HDMR-Bayesian Technique *Shereena O A 1) and Dr. B N Rao 2) 1), 2) Department of Civil Engineering, IIT Madras, Chennai 600036, Tamil Nadu, India 1) ce14d020@smail.iitm.ac.in
More informationRecursive Noise Adaptive Kalman Filtering by Variational Bayesian Approximations
PREPRINT 1 Recursive Noise Adaptive Kalman Filtering by Variational Bayesian Approximations Simo Särä, Member, IEEE and Aapo Nummenmaa Abstract This article considers the application of variational Bayesian
More informationMobile Robot Localization
Mobile Robot Localization 1 The Problem of Robot Localization Given a map of the environment, how can a robot determine its pose (planar coordinates + orientation)? Two sources of uncertainty: - observations
More informationECO 513 Fall 2008 C.Sims KALMAN FILTER. s t = As t 1 + ε t Measurement equation : y t = Hs t + ν t. u t = r t. u 0 0 t 1 + y t = [ H I ] u t.
ECO 513 Fall 2008 C.Sims KALMAN FILTER Model in the form 1. THE KALMAN FILTER Plant equation : s t = As t 1 + ε t Measurement equation : y t = Hs t + ν t. Var(ε t ) = Ω, Var(ν t ) = Ξ. ε t ν t and (ε t,
More informationStochastic Analogues to Deterministic Optimizers
Stochastic Analogues to Deterministic Optimizers ISMP 2018 Bordeaux, France Vivak Patel Presented by: Mihai Anitescu July 6, 2018 1 Apology I apologize for not being here to give this talk myself. I injured
More informationPATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA Contents in latter part Linear Dynamical Systems What is different from HMM? Kalman filter Its strength and limitation Particle Filter
More informationThe Kalman filter, Nonlinear filtering, and Markov Chain Monte Carlo
NBER Summer Institute Minicourse What s New in Econometrics: Time Series Lecture 5 July 5, 2008 The Kalman filter, Nonlinear filtering, and Markov Chain Monte Carlo Lecture 5, July 2, 2008 Outline. Models
More informationResearch Division Federal Reserve Bank of St. Louis Working Paper Series
Research Division Federal Reserve Bank of St Louis Working Paper Series Kalman Filtering with Truncated Normal State Variables for Bayesian Estimation of Macroeconomic Models Michael Dueker Working Paper
More informationBayesian Computations for DSGE Models
Bayesian Computations for DSGE Models Frank Schorfheide University of Pennsylvania, PIER, CEPR, and NBER October 23, 2017 This Lecture is Based on Bayesian Estimation of DSGE Models Edward P. Herbst &
More informationBlind Equalization via Particle Filtering
Blind Equalization via Particle Filtering Yuki Yoshida, Kazunori Hayashi, Hideaki Sakai Department of System Science, Graduate School of Informatics, Kyoto University Historical Remarks A sequential Monte
More informationPaul Karapanagiotidis ECO4060
Paul Karapanagiotidis ECO4060 The way forward 1) Motivate why Markov-Chain Monte Carlo (MCMC) is useful for econometric modeling 2) Introduce Markov-Chain Monte Carlo (MCMC) - Metropolis-Hastings (MH)
More informationEvolution Strategies Based Particle Filters for Fault Detection
Evolution Strategies Based Particle Filters for Fault Detection Katsuji Uosaki, Member, IEEE, and Toshiharu Hatanaka, Member, IEEE Abstract Recent massive increase of the computational power has allowed
More informationEKF and SLAM. McGill COMP 765 Sept 18 th, 2017
EKF and SLAM McGill COMP 765 Sept 18 th, 2017 Outline News and information Instructions for paper presentations Continue on Kalman filter: EKF and extension to mapping Example of a real mapping system:
More informationState Estimation Based on Nested Particle Filters
Cleveland State University EngagedScholarship@CSU ETD Archive 2013 State Estimation Based on Nested Particle Filters Swathi Srinivasan Cleveland State University How does access to this work benefit you?
More informationThe Metropolis-Hastings Algorithm. June 8, 2012
The Metropolis-Hastings Algorithm June 8, 22 The Plan. Understand what a simulated distribution is 2. Understand why the Metropolis-Hastings algorithm works 3. Learn how to apply the Metropolis-Hastings
More informationA Comparison of the EKF, SPKF, and the Bayes Filter for Landmark-Based Localization
A Comparison of the EKF, SPKF, and the Bayes Filter for Landmark-Based Localization and Timothy D. Barfoot CRV 2 Outline Background Objective Experimental Setup Results Discussion Conclusion 2 Outline
More informationPosterior Cramer-Rao Lower Bound for Mobile Tracking in Mixed Line-of-Sight/Non Line-of-Sight Conditions
Posterior Cramer-Rao Lower Bound for Mobile Tracking in Mixed Line-of-Sight/Non Line-of-Sight Conditions Chen Liang 1,2, Wu Lenan 2, Robert Piché 1 1 Tampere University of Technology, Finland 2 Southeast
More informationBayes Filter Reminder. Kalman Filter Localization. Properties of Gaussians. Gaussians. Prediction. Correction. σ 2. Univariate. 1 2πσ e.
Kalman Filter Localization Bayes Filter Reminder Prediction Correction Gaussians p(x) ~ N(µ,σ 2 ) : Properties of Gaussians Univariate p(x) = 1 1 2πσ e 2 (x µ) 2 σ 2 µ Univariate -σ σ Multivariate µ Multivariate
More information