# Kalman filtering and friends: Inference in time series models. Herke van Hoof slides mostly by Michael Rubinstein

Save this PDF as:

Size: px
Start display at page:

Download "Kalman filtering and friends: Inference in time series models. Herke van Hoof slides mostly by Michael Rubinstein"

## Transcription

1 Kalman filtering and friends: Inference in time series models Herke van Hoof slides mostly by Michael Rubinstein

2 Problem overview Goal Estimate most probable state at time k using measurement up to time k k <k: prediction k =k: filtering k >k: smoothing Input (Noisy) Sensor measurements Known or learned system model (see last lecture) Many problems require estimation of the state of systems that change over time using noisy measurements on the system

3 Applications Ballistics Robotics Robot localization Tracking hands/cars/ Econometrics Stock prediction Navigation Many more

4 Example: noisy localization Position at t=0 Measurement at t=1 t=2 t=3 t=4 t=5 t=6

5 Example: noisy localization Position at t=0 Measurement at t=1 Smoothing: where was I in the past t=2 t=3 t=4 t=5 t=6

6 Example: noisy localization Position at t=0 Measurement at t=1 Smoothing: where was I in the past Filtering: where am I now t=2 t=3 t=4 t=5 t=6

7 Example: noisy localization Position at t=0 Measurement at t=1 Smoothing: where was I in the past Filtering: where am I now t=2 t=3 t=4 t=5 t=6 Prediction: where will I be in the future

8 Today s lecture Fundamentals Formalizing time series models Recursive filtering Two cases with optimal solutions Linear Gaussian models Discrete systems Suboptimal solutions

9 Stochastic Processes Stochastic process Collection of random variables indexed by some set Ie. R.V. x " for every element i in index set Time series modeling Sequence of random states/variables Measurements available at discrete times Modeled as stochastic process indexed by N

10 Stochastic Processes Stochastic process Collection of random variables indexed by some set Ie. R.V. x " for every element i in index set Time series modeling Sequence of random states/variables Measurements available at discrete times Modeled as stochastic process indexed by N p(x \$ ) x \$ (location at t=1)

11 Stochastic Processes Stochastic process Collection of random variables indexed by some set Ie. R.V. x " for every element i in index set Time series modeling Sequence of random states/variables Measurements available at discrete times Modeled as stochastic process indexed by N p(x \$ ) p(x ( ) p(x ) )

12 (First-order) Markov process The Markov property the likelihood of a future state depends on present state only Markov chain A stochastic process with Markov property k-1 k k+1 time x k-1 x k x k+1 States

13 Hidden Markov Model (HMM) the state is not directly visible, but output dependent on the state is visible k-1 k k+1 time x k-1 x k x k+1 States (hidden) z k-1 z k z k+1 Measurements (observed)

14 State space The state vector contains all available information to describe the investigated system usually multidimensional: The measurement vector represents observations related to the state vector Generally (but not necessarily) of lower dimension than the state vector

15 State space Tracking: Econometrics: Monetary flow Interest rates Inflation

16 Hidden Markov Model (HMM) the state is not directly visible, but output dependent on the state is visible k-1 k k+1 time x k-1 x k x k+1 States (hidden) z k-1 z k z k+1 Measurements (observed)

17 Dynamic System k-1 k k+1 x k-1 x k x k+1 z k-1 z k z k+1 Stochastic diffusion State equation: state vector at time instant k state transition function, i.i.d process noise Observation equation: observations at time instant k observation function, i.i.d measurement noise

18 A simple dynamic system (4-dimensional state space) Constant velocity motion: Only position is observed:

19 Gaussian distribution μ Yacov Hel-Or

20 Today s lecture Fundamentals Formalizing time series models Recursive filtering Two cases with optimal solutions Linear Gaussian models Discrete systems Suboptimal solutions

21 Recursive filters For many problems, estimate is required each time a new measurement arrives Batch processing Requires all available data Sequential processing New data is processed upon arrival Need not store the complete dataset Need not reprocess all data for each new measurement Assume no out-of-sequence measurements (solutions for this exist as well )

22 Bayesian filter Construct the posterior probability density function of the state based on all available information Posterior Thomas Bayes Sample space By knowing the posterior many kinds of estimates for can be derived mean (expectation), mode, median, Can also give estimation of the accuracy (e.g. covariance)

23 Recursive Bayes filters Given: System models in probabilistic forms Markovian process Measurements conditionally independent given state (known statistics of v k, w k ) Initial state also known as the prior Measurements z \$,, z -

24 Recursive Bayes filters Prediction step (a-priori) Uses the system model to predict forward Deforms/translates/spreads state pdf due to random noise Update step (a-posteriori) Update the prediction in light of new data Tightens the state pdf

25 Prior vs posterior? It can seem odd to regard p x - z \$:-/\$ Compare to likelihood posterior P(x - z -, z \$:-/\$ ) = p z - x -, z \$:-/\$ P(x - z \$:-/\$ ) p(z - z \$:-/\$ ) evidence prior P(x - z -, z \$:-/\$ ) = p z - x -, z \$:-/\$ P(x - z \$:-/\$ ) p(z - z \$:-/\$ ) as prior In update with z -, it is a working prior

26 General prediction-update framework Assume is given at time k-1 Prediction: System model Previous posterior (1) Using Chapman-Kolmogorov identity + Markov property

27 General prediction-update framework Update step Measurement model Current prior (2) Where Normalization constant

28 Generating estimates Knowledge of enables to compute optimal estimate with respect to any criterion. e.g. Minimum mean-square error (MMSE) Maximum a-posteriori

29 General prediction-update framework So (1) and (2) give optimal solution for the recursive estimation problem! Unfortunately only conceptual solution integrals are intractable Cannot represent arbitrary pdfs! However, optimal solution does exist for several restrictive cases

30 Today s lecture Fundamentals Formalizing time series models Recursive filtering Two cases with optimal solutions Linear Gaussian models Discrete systems Suboptimal solutions

31 Restrictive case #1 Posterior at each time step is Gaussian Completely described by mean and covariance If is Gaussian it can be shown that is also Gaussian provided that: are Gaussian are linear

32 Restrictive case #1 Why Linear? Yacov Hel-Or

33 Restrictive case #1 Why Linear? Yacov Hel-Or

34 Restrictive case #1 Linear system with additive noise Simple example again /\$ /\$

35 The Kalman filter Rudolf E. Kalman Substituting into (1) and (2) yields the predict and update equations

36 The Kalman filter Predict: Update:

37 Intuition via 1D example Lost at sea Night No idea of location For simplicity let s assume 1D Not moving * Example and plots by Maybeck, Stochastic models, estimation and control, volume 1

38 Example cont d Time t1: Star Sighting Denote z(t1)=z1 Uncertainty (inaccuracies, human error, etc) Denote σ1 (normal) Can establish the conditional probability of x(t1) given measurement z1

39 Example cont d Probability for any location, based on measurement For Gaussian density 68.3% within ±σ1 Best estimate of position: Mean/Mode/Median

40 Example cont d Time t2: friend (more trained) x(t2)=z2, σ(t2)=σ2 Since she has higher skill: σ2<σ1

41 Example cont d f(x(t2) z1,z2) also Gaussian

42 Example cont d σ less than both σ1 and σ2 σ1= σ2: average σ1> σ2: more weight to z2 Rewrite:

43 Example cont d The Kalman update rule: Best estimate Given z2 (a posteriori) Best Prediction prior to z2 (a priori) Optimal Weighting (Kalman Gain) Residual

44 The Kalman filter Predict: Update: Generally increases variance Generally decreases variance

45 Kalman gain Small measurement error, H invertible: Small prediction error:

46 The Kalman filter Pros (compared to e.g. particle filter) Optimal closed-form solution to the tracking problem (under the assumptions) No algorithm can do better in a linear-gaussian environment! All logical estimations collapse to a unique solution Simple to implement Fast to execute Cons If either the system or measurement model is nonlinear à the posterior will be non-gaussian Smoothing possible with a backward message (cf HMMs, lecture 10)

47 Restrictive case #2 The state space (domain) is discrete and finite Assume the state space at time k-1 consists of states Let be the conditional probability of the state at time k-1, given measurements up to k-1

48 The Grid-based filter The posterior pdf at k-1 can be expressed as sum of delta functions Again, substitution into (1) and (2) yields the predict and update equations Equivalent to belief monitoring in HMMs (Lecture 10)

49 The Grid-based filter Prediction (1) New prior is also weighted sum of delta functions New prior weights are reweighting of old posterior weights using state transition probabilities

50 The Grid-based filter Update (2) Posterior weights are reweighting of prior weights using likelihoods (+ normalization)

51 The Grid-based filter Pros: assumed known, but no constraint on their (discrete) shapes Easy extension to varying number of states Optimal solution for the discrete-finite environment! Cons: Curse of dimensionality Inefficient if the state space is large Statically considers all possible hypotheses Smoothing possible with a backward message (cf HMMs, lecture 10)

52 Today s lecture Fundamentals Formalizing time series models Recursive filtering Two cases with optimal solutions Linear Gaussian models Discrete systems Suboptimal solutions

53 Suboptimal solutions In many cases these assumptions do not hold Practical environments are nonlinear, non-gaussian, continuous Approximations are necessary Extended Kalman filter (EKF) Approximate grid-based methods Multiple-model estimators Unscented Kalman filter (UKF) Particle filters (PF) Analytic approximations Numerical methods Gaussian-sum filters Sampling approaches

54 The extended Kalman filter The idea: local linearization of the dynamic system might be sufficient description of the nonlinearity The model: nonlinear system with additive noise

55 The extended Kalman filter f, h are approximated using a first-order Taylor series expansion (eval at state estimations) Predict: Update:

56 The extended Kalman filter

57 The extended Kalman filter Pros Good approximation when models are near-linear Efficient to calculate (de facto method for navigation systems and GPS) Cons Only approximation (optimality not proven) Still a single Gaussian approximations Nonlinearity à non-gaussianity (e.g. bimodal) If we have multimodal hypothesis, and choose incorrectly can be difficult to recover Inapplicable when f,h discontinuous

58 The unscented Kalman filter Yacov Hel-Or Can give more accurately approximates posterior

59 Challenges Detection specific Full/partial occlusions False positives/false negatives Entering/leaving the scene Efficiency Multiple models and switching dynamics Multiple targets

60 Conclusion Inference in time series models: Past: smoothing Present: filtering Future: prediction Recursive Bayes filter optimal Computable in two cases Linear Gaussian systems: Kalman filter Discrete systems: Grid filter Approximate solutions for other systems

### Lecture 2: From Linear Regression to Kalman Filter and Beyond

Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing

### Linear Dynamical Systems

Linear Dynamical Systems Sargur N. srihari@cedar.buffalo.edu Machine Learning Course: http://www.cedar.buffalo.edu/~srihari/cse574/index.html Two Models Described by Same Graph Latent variables Observations

### Lecture 2: From Linear Regression to Kalman Filter and Beyond

Lecture 2: From Linear Regression to Kalman Filter and Beyond Department of Biomedical Engineering and Computational Science Aalto University January 26, 2012 Contents 1 Batch and Recursive Estimation

### 2D Image Processing. Bayes filter implementation: Kalman filter

2D Image Processing Bayes filter implementation: Kalman filter Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de

### Lecture 6: Bayesian Inference in SDE Models

Lecture 6: Bayesian Inference in SDE Models Bayesian Filtering and Smoothing Point of View Simo Särkkä Aalto University Simo Särkkä (Aalto) Lecture 6: Bayesian Inference in SDEs 1 / 45 Contents 1 SDEs

### Autonomous Navigation for Flying Robots

Computer Vision Group Prof. Daniel Cremers Autonomous Navigation for Flying Robots Lecture 6.2: Kalman Filter Jürgen Sturm Technische Universität München Motivation Bayes filter is a useful tool for state

### 2D Image Processing. Bayes filter implementation: Kalman filter

2D Image Processing Bayes filter implementation: Kalman filter Prof. Didier Stricker Dr. Gabriele Bleser Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche

### 2D Image Processing (Extended) Kalman and particle filter

2D Image Processing (Extended) Kalman and particle filter Prof. Didier Stricker Dr. Gabriele Bleser Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz

### The Kalman Filter ImPr Talk

The Kalman Filter ImPr Talk Ged Ridgway Centre for Medical Image Computing November, 2006 Outline What is the Kalman Filter? State Space Models Kalman Filter Overview Bayesian Updating of Estimates Kalman

### Autonomous Mobile Robot Design

Autonomous Mobile Robot Design Topic: Extended Kalman Filter Dr. Kostas Alexis (CSE) These slides relied on the lectures from C. Stachniss, J. Sturm and the book Probabilistic Robotics from Thurn et al.

### Probabilistic Graphical Models

Probabilistic Graphical Models Brown University CSCI 2950-P, Spring 2013 Prof. Erik Sudderth Lecture 12: Gaussian Belief Propagation, State Space Models and Kalman Filters Guest Kalman Filter Lecture by

### The Kalman Filter (part 1) Definition. Rudolf Emil Kalman. Why do we need a filter? Definition. HCI/ComS 575X: Computational Perception.

The Kalman Filter (part 1) HCI/ComS 575X: Computational Perception Instructor: Alexander Stoytchev http://www.cs.iastate.edu/~alex/classes/2007_spring_575x/ March 5, 2007 HCI/ComS 575X: Computational Perception

### ROBOTICS 01PEEQW. Basilio Bona DAUIN Politecnico di Torino

ROBOTICS 01PEEQW Basilio Bona DAUIN Politecnico di Torino Probabilistic Fundamentals in Robotics Gaussian Filters Course Outline Basic mathematical framework Probabilistic models of mobile robots Mobile

### STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 11 Project

### Bayes Filter Reminder. Kalman Filter Localization. Properties of Gaussians. Gaussians. Prediction. Correction. σ 2. Univariate. 1 2πσ e.

Kalman Filter Localization Bayes Filter Reminder Prediction Correction Gaussians p(x) ~ N(µ,σ 2 ) : Properties of Gaussians Univariate p(x) = 1 1 2πσ e 2 (x µ) 2 σ 2 µ Univariate -σ σ Multivariate µ Multivariate

### Sensor Fusion: Particle Filter

Sensor Fusion: Particle Filter By: Gordana Stojceska stojcesk@in.tum.de Outline Motivation Applications Fundamentals Tracking People Advantages and disadvantages Summary June 05 JASS '05, St.Petersburg,

### CSE 473: Artificial Intelligence

CSE 473: Artificial Intelligence Hidden Markov Models Dieter Fox --- University of Washington [Most slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188 materials

### Note Set 5: Hidden Markov Models

Note Set 5: Hidden Markov Models Probabilistic Learning: Theory and Algorithms, CS 274A, Winter 2016 1 Hidden Markov Models (HMMs) 1.1 Introduction Consider observed data vectors x t that are d-dimensional

### Markov localization uses an explicit, discrete representation for the probability of all position in the state space.

Markov Kalman Filter Localization Markov localization localization starting from any unknown position recovers from ambiguous situation. However, to update the probability of all positions within the whole

### CS325 Artificial Intelligence Ch. 15,20 Hidden Markov Models and Particle Filtering

CS325 Artificial Intelligence Ch. 15,20 Hidden Markov Models and Particle Filtering Cengiz Günay, Emory Univ. Günay Ch. 15,20 Hidden Markov Models and Particle FilteringSpring 2013 1 / 21 Get Rich Fast!

### PROBABILISTIC REASONING OVER TIME

PROBABILISTIC REASONING OVER TIME In which we try to interpret the present, understand the past, and perhaps predict the future, even when very little is crystal clear. Outline Time and uncertainty Inference:

### Probabilistic Fundamentals in Robotics. DAUIN Politecnico di Torino July 2010

Probabilistic Fundamentals in Robotics Gaussian Filters Basilio Bona DAUIN Politecnico di Torino July 2010 Course Outline Basic mathematical framework Probabilistic models of mobile robots Mobile robot

### CS491/691: Introduction to Aerial Robotics

CS491/691: Introduction to Aerial Robotics Topic: State Estimation Dr. Kostas Alexis (CSE) World state (or system state) Belief state: Our belief/estimate of the world state World state: Real state of

### Hidden Markov Models. AIMA Chapter 15, Sections 1 5. AIMA Chapter 15, Sections 1 5 1

Hidden Markov Models AIMA Chapter 15, Sections 1 5 AIMA Chapter 15, Sections 1 5 1 Consider a target tracking problem Time and uncertainty X t = set of unobservable state variables at time t e.g., Position

### Lecture 7: Optimal Smoothing

Department of Biomedical Engineering and Computational Science Aalto University March 17, 2011 Contents 1 What is Optimal Smoothing? 2 Bayesian Optimal Smoothing Equations 3 Rauch-Tung-Striebel Smoother

### PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA Contents in latter part Linear Dynamical Systems What is different from HMM? Kalman filter Its strength and limitation Particle Filter

### Mobile Robot Localization

Mobile Robot Localization 1 The Problem of Robot Localization Given a map of the environment, how can a robot determine its pose (planar coordinates + orientation)? Two sources of uncertainty: - observations

### Dynamic System Identification using HDMR-Bayesian Technique

Dynamic System Identification using HDMR-Bayesian Technique *Shereena O A 1) and Dr. B N Rao 2) 1), 2) Department of Civil Engineering, IIT Madras, Chennai 600036, Tamil Nadu, India 1) ce14d020@smail.iitm.ac.in

### Modeling and state estimation Examples State estimation Probabilities Bayes filter Particle filter. Modeling. CSC752 Autonomous Robotic Systems

Modeling CSC752 Autonomous Robotic Systems Ubbo Visser Department of Computer Science University of Miami February 21, 2017 Outline 1 Modeling and state estimation 2 Examples 3 State estimation 4 Probabilities

### Mobile Robot Localization

Mobile Robot Localization 1 The Problem of Robot Localization Given a map of the environment, how can a robot determine its pose (planar coordinates + orientation)? Two sources of uncertainty: - observations

### L09. PARTICLE FILTERING. NA568 Mobile Robotics: Methods & Algorithms

L09. PARTICLE FILTERING NA568 Mobile Robotics: Methods & Algorithms Particle Filters Different approach to state estimation Instead of parametric description of state (and uncertainty), use a set of state

### Particle Filters. Pieter Abbeel UC Berkeley EECS. Many slides adapted from Thrun, Burgard and Fox, Probabilistic Robotics

Particle Filters Pieter Abbeel UC Berkeley EECS Many slides adapted from Thrun, Burgard and Fox, Probabilistic Robotics Motivation For continuous spaces: often no analytical formulas for Bayes filter updates

### Parametric Models. Dr. Shuang LIANG. School of Software Engineering TongJi University Fall, 2012

Parametric Models Dr. Shuang LIANG School of Software Engineering TongJi University Fall, 2012 Today s Topics Maximum Likelihood Estimation Bayesian Density Estimation Today s Topics Maximum Likelihood

### Dynamic models 1 Kalman filters, linearization,

Koller & Friedman: Chapter 16 Jordan: Chapters 13, 15 Uri Lerner s Thesis: Chapters 3,9 Dynamic models 1 Kalman filters, linearization, Switching KFs, Assumed density filters Probabilistic Graphical Models

### Bayesian Methods in Positioning Applications

Bayesian Methods in Positioning Applications Vedran Dizdarević v.dizdarevic@tugraz.at Graz University of Technology, Austria 24. May 2006 Bayesian Methods in Positioning Applications p.1/21 Outline Problem

### AUTOMOTIVE ENVIRONMENT SENSORS

AUTOMOTIVE ENVIRONMENT SENSORS Lecture 5. Localization BME KÖZLEKEDÉSMÉRNÖKI ÉS JÁRMŰMÉRNÖKI KAR 32708-2/2017/INTFIN SZÁMÚ EMMI ÁLTAL TÁMOGATOTT TANANYAG Related concepts Concepts related to vehicles moving

### L06. LINEAR KALMAN FILTERS. NA568 Mobile Robotics: Methods & Algorithms

L06. LINEAR KALMAN FILTERS NA568 Mobile Robotics: Methods & Algorithms 2 PS2 is out! Landmark-based Localization: EKF, UKF, PF Today s Lecture Minimum Mean Square Error (MMSE) Linear Kalman Filter Gaussian

### Markov Models. CS 188: Artificial Intelligence Fall Example. Mini-Forward Algorithm. Stationary Distributions.

CS 88: Artificial Intelligence Fall 27 Lecture 2: HMMs /6/27 Markov Models A Markov model is a chain-structured BN Each node is identically distributed (stationarity) Value of X at a given time is called

### Bayesian Machine Learning - Lecture 7

Bayesian Machine Learning - Lecture 7 Guido Sanguinetti Institute for Adaptive and Neural Computation School of Informatics University of Edinburgh gsanguin@inf.ed.ac.uk March 4, 2015 Today s lecture 1

### Introduction to Mobile Robotics Probabilistic Robotics

Introduction to Mobile Robotics Probabilistic Robotics Wolfram Burgard 1 Probabilistic Robotics Key idea: Explicit representation of uncertainty (using the calculus of probability theory) Perception Action

### Markov Models and Hidden Markov Models

Markov Models and Hidden Markov Models Robert Platt Northeastern University Some images and slides are used from: 1. CS188 UC Berkeley 2. RN, AIMA Markov Models We have already seen that an MDP provides

### Rao-Blackwellized Particle Filter for Multiple Target Tracking

Rao-Blackwellized Particle Filter for Multiple Target Tracking Simo Särkkä, Aki Vehtari, Jouko Lampinen Helsinki University of Technology, Finland Abstract In this article we propose a new Rao-Blackwellized

Miscellaneous Regarding reading materials Reading materials will be provided as needed If no assigned reading, it means I think the material from class is sufficient Should be enough for you to do your

### L11. EKF SLAM: PART I. NA568 Mobile Robotics: Methods & Algorithms

L11. EKF SLAM: PART I NA568 Mobile Robotics: Methods & Algorithms Today s Topic EKF Feature-Based SLAM State Representation Process / Observation Models Landmark Initialization Robot-Landmark Correlation

### Introduction to Artificial Intelligence (AI)

Introduction to Artificial Intelligence (AI) Computer Science cpsc502, Lecture 10 Oct, 13, 2011 CPSC 502, Lecture 10 Slide 1 Today Oct 13 Inference in HMMs More on Robot Localization CPSC 502, Lecture

### Stochastic Models, Estimation and Control Peter S. Maybeck Volumes 1, 2 & 3 Tables of Contents

Navtech Part #s Volume 1 #1277 Volume 2 #1278 Volume 3 #1279 3 Volume Set #1280 Stochastic Models, Estimation and Control Peter S. Maybeck Volumes 1, 2 & 3 Tables of Contents Volume 1 Preface Contents

### Sequential Monte Carlo Methods for Bayesian Computation

Sequential Monte Carlo Methods for Bayesian Computation A. Doucet Kyoto Sept. 2012 A. Doucet (MLSS Sept. 2012) Sept. 2012 1 / 136 Motivating Example 1: Generic Bayesian Model Let X be a vector parameter

### CS 5522: Artificial Intelligence II

CS 5522: Artificial Intelligence II Hidden Markov Models Instructor: Wei Xu Ohio State University [These slides were adapted from CS188 Intro to AI at UC Berkeley.] Pacman Sonar (P4) [Demo: Pacman Sonar

### Robotics 2 Target Tracking. Kai Arras, Cyrill Stachniss, Maren Bennewitz, Wolfram Burgard

Robotics 2 Target Tracking Kai Arras, Cyrill Stachniss, Maren Bennewitz, Wolfram Burgard Slides by Kai Arras, Gian Diego Tipaldi, v.1.1, Jan 2012 Chapter Contents Target Tracking Overview Applications

### CS 5522: Artificial Intelligence II

CS 5522: Artificial Intelligence II Hidden Markov Models Instructor: Alan Ritter Ohio State University [These slides were adapted from CS188 Intro to AI at UC Berkeley. All materials available at http://ai.berkeley.edu.]

### CSEP 573: Artificial Intelligence

CSEP 573: Artificial Intelligence Hidden Markov Models Luke Zettlemoyer Many slides over the course adapted from either Dan Klein, Stuart Russell, Andrew Moore, Ali Farhadi, or Dan Weld 1 Outline Probabilistic

### Probabilistic Robotics. Slides from Autonomous Robots (Siegwart and Nourbaksh), Chapter 5 Probabilistic Robotics (S. Thurn et al.

robabilistic Robotics Slides from Autonomous Robots Siegwart and Nourbaksh Chapter 5 robabilistic Robotics S. Thurn et al. Today Overview of probability Representing uncertainty ropagation of uncertainty

### Probabilistic Graphical Models

Probabilistic Graphical Models Lecture 12 Dynamical Models CS/CNS/EE 155 Andreas Krause Homework 3 out tonight Start early!! Announcements Project milestones due today Please email to TAs 2 Parameter learning

### 9 Multi-Model State Estimation

Technion Israel Institute of Technology, Department of Electrical Engineering Estimation and Identification in Dynamical Systems (048825) Lecture Notes, Fall 2009, Prof. N. Shimkin 9 Multi-Model State

### STA 414/2104: Machine Learning

STA 414/2104: Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistics! rsalakhu@cs.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 9 Sequential Data So far

### Particle Filters; Simultaneous Localization and Mapping (Intelligent Autonomous Robotics) Subramanian Ramamoorthy School of Informatics

Particle Filters; Simultaneous Localization and Mapping (Intelligent Autonomous Robotics) Subramanian Ramamoorthy School of Informatics Recap: State Estimation using Kalman Filter Project state and error

### Mathematical Formulation of Our Example

Mathematical Formulation of Our Example We define two binary random variables: open and, where is light on or light off. Our question is: What is? Computer Vision 1 Combining Evidence Suppose our robot

### Why do we care? Examples. Bayes Rule. What room am I in? Handling uncertainty over time: predicting, estimating, recognizing, learning

Handling uncertainty over time: predicting, estimating, recognizing, learning Chris Atkeson 004 Why do we care? Speech recognition makes use of dependence of words and phonemes across time. Knowing where

### Introduction to Machine Learning

Introduction to Machine Learning Brown University CSCI 1950-F, Spring 2012 Prof. Erik Sudderth Lecture 25: Markov Chain Monte Carlo (MCMC) Course Review and Advanced Topics Many figures courtesy Kevin

### Machine Learning for OR & FE

Machine Learning for OR & FE Hidden Markov Models Martin Haugh Department of Industrial Engineering and Operations Research Columbia University Email: martin.b.haugh@gmail.com Additional References: David

### Hidden Markov Models (recap BNs)

Probabilistic reasoning over time - Hidden Markov Models (recap BNs) Applied artificial intelligence (EDA132) Lecture 10 2016-02-17 Elin A. Topp Material based on course book, chapter 15 1 A robot s view

### Lecture: Gaussian Process Regression. STAT 6474 Instructor: Hongxiao Zhu

Lecture: Gaussian Process Regression STAT 6474 Instructor: Hongxiao Zhu Motivation Reference: Marc Deisenroth s tutorial on Robot Learning. 2 Fast Learning for Autonomous Robots with Gaussian Processes

### Bayesian Networks BY: MOHAMAD ALSABBAGH

Bayesian Networks BY: MOHAMAD ALSABBAGH Outlines Introduction Bayes Rule Bayesian Networks (BN) Representation Size of a Bayesian Network Inference via BN BN Learning Dynamic BN Introduction Conditional

### Why do we care? Measurements. Handling uncertainty over time: predicting, estimating, recognizing, learning. Dealing with time

Handling uncertainty over time: predicting, estimating, recognizing, learning Chris Atkeson 2004 Why do we care? Speech recognition makes use of dependence of words and phonemes across time. Knowing where

### Probabilistic Graphical Models Homework 2: Due February 24, 2014 at 4 pm

Probabilistic Graphical Models 10-708 Homework 2: Due February 24, 2014 at 4 pm Directions. This homework assignment covers the material presented in Lectures 4-8. You must complete all four problems to

### EKF and SLAM. McGill COMP 765 Sept 18 th, 2017

EKF and SLAM McGill COMP 765 Sept 18 th, 2017 Outline News and information Instructions for paper presentations Continue on Kalman filter: EKF and extension to mapping Example of a real mapping system:

### Content.

Content Fundamentals of Bayesian Techniques (E. Sucar) Bayesian Filters (O. Aycard) Definition & interests Implementations Hidden Markov models Discrete Bayesian Filters or Markov localization Kalman filters

### Human-Oriented Robotics. Temporal Reasoning. Kai Arras Social Robotics Lab, University of Freiburg

Temporal Reasoning Kai Arras, University of Freiburg 1 Temporal Reasoning Contents Introduction Temporal Reasoning Hidden Markov Models Linear Dynamical Systems (LDS) Kalman Filter 2 Temporal Reasoning

### Hidden Markov Models,99,100! Markov, here I come!

Hidden Markov Models,99,100! Markov, here I come! 16.410/413 Principles of Autonomy and Decision-Making Pedro Santana (psantana@mit.edu) October 7 th, 2015. Based on material by Brian Williams and Emilio

### Probabilistic Robotics

University of Rome La Sapienza Master in Artificial Intelligence and Robotics Probabilistic Robotics Prof. Giorgio Grisetti Course web site: http://www.dis.uniroma1.it/~grisetti/teaching/probabilistic_ro

### Introduction to Mobile Robotics Bayes Filter Particle Filter and Monte Carlo Localization

Introduction to Mobile Robotics Bayes Filter Particle Filter and Monte Carlo Localization Wolfram Burgard, Cyrill Stachniss, Maren Bennewitz, Kai Arras 1 Motivation Recall: Discrete filter Discretize the

### Reasoning Under Uncertainty: Conditioning, Bayes Rule & the Chain Rule

Reasoning Under Uncertainty: Conditioning, Bayes Rule & the Chain Rule Alan Mackworth UBC CS 322 Uncertainty 2 March 13, 2013 Textbook 6.1.3 Lecture Overview Recap: Probability & Possible World Semantics

### Lecture 3. Linear Regression II Bastian Leibe RWTH Aachen

Advanced Machine Learning Lecture 3 Linear Regression II 02.11.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de/ leibe@vision.rwth-aachen.de This Lecture: Advanced Machine Learning Regression

### Temporal probability models. Chapter 15, Sections 1 5 1

Temporal probability models Chapter 15, Sections 1 5 Chapter 15, Sections 1 5 1 Outline Time and uncertainty Inference: filtering, prediction, smoothing Hidden Markov models Kalman filters (a brief mention)

### Probabilistic Robotics

Probabilistic Robotics Overview of probability, Representing uncertainty Propagation of uncertainty, Bayes Rule Application to Localization and Mapping Slides from Autonomous Robots (Siegwart and Nourbaksh),

### Introduction to Bayesian Learning. Machine Learning Fall 2018

Introduction to Bayesian Learning Machine Learning Fall 2018 1 What we have seen so far What does it mean to learn? Mistake-driven learning Learning by counting (and bounding) number of mistakes PAC learnability

### Bayesian Networks: Construction, Inference, Learning and Causal Interpretation. Volker Tresp Summer 2014

Bayesian Networks: Construction, Inference, Learning and Causal Interpretation Volker Tresp Summer 2014 1 Introduction So far we were mostly concerned with supervised learning: we predicted one or several

### Vlad Estivill-Castro. Robots for People --- A project for intelligent integrated systems

1 Vlad Estivill-Castro Robots for People --- A project for intelligent integrated systems V. Estivill-Castro 2 Probabilistic Map-based Localization (Kalman Filter) Chapter 5 (textbook) Based on textbook

### State Estimation Based on Nested Particle Filters

Cleveland State University EngagedScholarship@CSU ETD Archive 2013 State Estimation Based on Nested Particle Filters Swathi Srinivasan Cleveland State University How does access to this work benefit you?

### Linear Dynamical Systems (Kalman filter)

Linear Dynamical Systems (Kalman filter) (a) Overview of HMMs (b) From HMMs to Linear Dynamical Systems (LDS) 1 Markov Chains with Discrete Random Variables x 1 x 2 x 3 x T Let s assume we have discrete

### Inference and estimation in probabilistic time series models

1 Inference and estimation in probabilistic time series models David Barber, A Taylan Cemgil and Silvia Chiappa 11 Time series The term time series refers to data that can be represented as a sequence

### SEQUENTIAL MONTE CARLO METHODS WITH APPLICATIONS TO COMMUNICATION CHANNELS. A Thesis SIRISH BODDIKURAPATI

SEQUENTIAL MONTE CARLO METHODS WITH APPLICATIONS TO COMMUNICATION CHANNELS A Thesis by SIRISH BODDIKURAPATI Submitted to the Office of Graduate Studies of Texas A&M University in partial fulfillment of

### ECE521 Lecture 19 HMM cont. Inference in HMM

ECE521 Lecture 19 HMM cont. Inference in HMM Outline Hidden Markov models Model definitions and notations Inference in HMMs Learning in HMMs 2 Formally, a hidden Markov model defines a generative process

### Hidden Markov Models. Vibhav Gogate The University of Texas at Dallas

Hidden Markov Models Vibhav Gogate The University of Texas at Dallas Intro to AI (CS 4365) Many slides over the course adapted from either Dan Klein, Luke Zettlemoyer, Stuart Russell or Andrew Moore 1

### Lecture Note 2: Estimation and Information Theory

Univ. of Michigan - NAME 568/EECS 568/ROB 530 Winter 2018 Lecture Note 2: Estimation and Information Theory Lecturer: Maani Ghaffari Jadidi Date: April 6, 2018 2.1 Estimation A static estimation problem

### Nonlinear Estimation Techniques for Impact Point Prediction of Ballistic Targets

Nonlinear Estimation Techniques for Impact Point Prediction of Ballistic Targets J. Clayton Kerce a, George C. Brown a, and David F. Hardiman b a Georgia Tech Research Institute, Georgia Institute of Technology,

### Expectation Propagation in Dynamical Systems

Expectation Propagation in Dynamical Systems Marc Peter Deisenroth Joint Work with Shakir Mohamed (UBC) August 10, 2012 Marc Deisenroth (TU Darmstadt) EP in Dynamical Systems 1 Motivation Figure : Complex

### 9 Forward-backward algorithm, sum-product on factor graphs

Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.438 Algorithms For Inference Fall 2014 9 Forward-backward algorithm, sum-product on factor graphs The previous

### Nonlinear and/or Non-normal Filtering. Jesús Fernández-Villaverde University of Pennsylvania

Nonlinear and/or Non-normal Filtering Jesús Fernández-Villaverde University of Pennsylvania 1 Motivation Nonlinear and/or non-gaussian filtering, smoothing, and forecasting (NLGF) problems are pervasive

### Robotics. Mobile Robotics. Marc Toussaint U Stuttgart

Robotics Mobile Robotics State estimation, Bayes filter, odometry, particle filter, Kalman filter, SLAM, joint Bayes filter, EKF SLAM, particle SLAM, graph-based SLAM Marc Toussaint U Stuttgart DARPA Grand

### Hidden Markov models 1

Hidden Markov models 1 Outline Time and uncertainty Markov process Hidden Markov models Inference: filtering, prediction, smoothing Most likely explanation: Viterbi 2 Time and uncertainty The world changes;

### Announcements. CS 188: Artificial Intelligence Fall VPI Example. VPI Properties. Reasoning over Time. Markov Models. Lecture 19: HMMs 11/4/2008

CS 88: Artificial Intelligence Fall 28 Lecture 9: HMMs /4/28 Announcements Midterm solutions up, submit regrade requests within a week Midterm course evaluation up on web, please fill out! Dan Klein UC

### From Bayes to Extended Kalman Filter

From Bayes to Extended Kalman Filter Michal Reinštein Czech Technical University in Prague Faculty of Electrical Engineering, Department of Cybernetics Center for Machine Perception http://cmp.felk.cvut.cz/

### Probability and Information Theory. Sargur N. Srihari

Probability and Information Theory Sargur N. srihari@cedar.buffalo.edu 1 Topics in Probability and Information Theory Overview 1. Why Probability? 2. Random Variables 3. Probability Distributions 4. Marginal

### Part 1: Expectation Propagation

Chalmers Machine Learning Summer School Approximate message passing and biomedicine Part 1: Expectation Propagation Tom Heskes Machine Learning Group, Institute for Computing and Information Sciences Radboud

### Hidden Markov Models. By Parisa Abedi. Slides courtesy: Eric Xing

Hidden Markov Models By Parisa Abedi Slides courtesy: Eric Xing i.i.d to sequential data So far we assumed independent, identically distributed data Sequential (non i.i.d.) data Time-series data E.g. Speech

### Simultaneous Localization and Mapping (SLAM) Corso di Robotica Prof. Davide Brugali Università degli Studi di Bergamo

Simultaneous Localization and Mapping (SLAM) Corso di Robotica Prof. Davide Brugali Università degli Studi di Bergamo Introduction SLAM asks the following question: Is it possible for an autonomous vehicle

### Particle Filters. Outline

Particle Filters M. Sami Fadali Professor of EE University of Nevada Outline Monte Carlo integration. Particle filter. Importance sampling. Degeneracy Resampling Example. 1 2 Monte Carlo Integration Numerical