CS 5522: Artificial Intelligence II

Similar documents
CS 5522: Artificial Intelligence II

CS 343: Artificial Intelligence

Markov Chains and Hidden Markov Models

Announcements. CS 188: Artificial Intelligence Fall Markov Models. Example: Markov Chain. Mini-Forward Algorithm. Example

CS 188: Artificial Intelligence Spring 2009

CSE 473: Artificial Intelligence

CSE 473: Ar+ficial Intelligence

CS 188: Artificial Intelligence Spring Announcements

CSE 473: Ar+ficial Intelligence. Probability Recap. Markov Models - II. Condi+onal probability. Product rule. Chain rule.

Markov Models. CS 188: Artificial Intelligence Fall Example. Mini-Forward Algorithm. Stationary Distributions.

CSEP 573: Artificial Intelligence

Announcements. CS 188: Artificial Intelligence Fall VPI Example. VPI Properties. Reasoning over Time. Markov Models. Lecture 19: HMMs 11/4/2008

CSE 473: Ar+ficial Intelligence. Example. Par+cle Filters for HMMs. An HMM is defined by: Ini+al distribu+on: Transi+ons: Emissions:

CSE 473: Artificial Intelligence Spring 2014

Hidden Markov Models. Hal Daumé III. Computer Science University of Maryland CS 421: Introduction to Artificial Intelligence 19 Apr 2012

CS 5522: Artificial Intelligence II

CS 188: Artificial Intelligence

CS 5522: Artificial Intelligence II

CS 343: Artificial Intelligence

Hidden Markov Models. Vibhav Gogate The University of Texas at Dallas

Approximate Inference

CS 343: Artificial Intelligence

Probability, Markov models and HMMs. Vibhav Gogate The University of Texas at Dallas

CS 5522: Artificial Intelligence II

CSE 473: Artificial Intelligence

Our Status in CSE 5522

Markov Models and Hidden Markov Models

CSE 473: Artificial Intelligence Probability Review à Markov Models. Outline

CS 188: Artificial Intelligence Fall Recap: Inference Example

CS 188: Artificial Intelligence

CS188 Outline. We re done with Part I: Search and Planning! Part II: Probabilistic Reasoning. Part III: Machine Learning

CS188 Outline. CS 188: Artificial Intelligence. Today. Inference in Ghostbusters. Probability. We re done with Part I: Search and Planning!

Mini-project 2 (really) due today! Turn in a printout of your work at the end of the class

Hidden Markov Models. AIMA Chapter 15, Sections 1 5. AIMA Chapter 15, Sections 1 5 1

CS 188: Artificial Intelligence Fall 2011

Outline. CSE 573: Artificial Intelligence Autumn Agent. Partial Observability. Markov Decision Process (MDP) 10/31/2012

CS 188: Artificial Intelligence. Our Status in CS188

CS 188: Artificial Intelligence Spring Announcements

CSCI 360 Introduc/on to Ar/ficial Intelligence Week 2: Problem Solving and Op/miza/on. Professor Wei-Min Shen Week 8.1 and 8.2

Our Status. We re done with Part I Search and Planning!

CSE 473: Ar+ficial Intelligence. Hidden Markov Models. Bayes Nets. Two random variable at each +me step Hidden state, X i Observa+on, E i

Hidden Markov models 1

Human-Oriented Robotics. Temporal Reasoning. Kai Arras Social Robotics Lab, University of Freiburg

Introduction to Artificial Intelligence (AI)

Introduction to Artificial Intelligence (AI)

CS 188: Artificial Intelligence Fall 2011

CS 343: Artificial Intelligence

Reinforcement Learning Wrap-up

Probability and Time: Hidden Markov Models (HMMs)

CS 188: Artificial Intelligence Fall 2009

CS 5522: Artificial Intelligence II

15-381: Artificial Intelligence. Hidden Markov Models (HMMs)

CS325 Artificial Intelligence Ch. 15,20 Hidden Markov Models and Particle Filtering

Linear Dynamical Systems

Hidden Markov Models. Aarti Singh Slides courtesy: Eric Xing. Machine Learning / Nov 8, 2010

Approximate Q-Learning. Dan Weld / University of Washington

Hidden Markov Models (recap BNs)

Introduction to Machine Learning CMU-10701

CS532, Winter 2010 Hidden Markov Models

Hidden Markov models

Why do we care? Measurements. Handling uncertainty over time: predicting, estimating, recognizing, learning. Dealing with time

Hidden Markov Models. By Parisa Abedi. Slides courtesy: Eric Xing

CS 5522: Artificial Intelligence II

Markov localization uses an explicit, discrete representation for the probability of all position in the state space.

Reasoning Under Uncertainty Over Time. CS 486/686: Introduction to Artificial Intelligence

Bayesian Networks BY: MOHAMAD ALSABBAGH

PROBABILISTIC REASONING OVER TIME

Reasoning Under Uncertainty: Bayesian networks intro

CS 188: Artificial Intelligence Fall 2009

CS 343: Artificial Intelligence

Artificial Intelligence Bayes Nets: Independence

Extensions of Bayesian Networks. Outline. Bayesian Network. Reasoning under Uncertainty. Features of Bayesian Networks.

CS 343: Artificial Intelligence

Kalman filtering and friends: Inference in time series models. Herke van Hoof slides mostly by Michael Rubinstein

CS 188: Artificial Intelligence Spring Announcements

Hidden Markov Models,99,100! Markov, here I come!

CS 7180: Behavioral Modeling and Decision- making in AI

Hidden Markov Models

Bayesian Machine Learning - Lecture 7

Nonlinear Optimization for Optimal Control Part 2. Pieter Abbeel UC Berkeley EECS. From linear to nonlinear Model-predictive control (MPC) POMDPs

Intelligent Systems (AI-2)

Bayes Nets: Sampling

Introduction to Mobile Robotics Probabilistic Robotics

STA 4273H: Statistical Machine Learning

Final Exam December 12, 2017

Note Set 5: Hidden Markov Models

CS Lecture 3. More Bayesian Networks

8: Hidden Markov Models

A.I. in health informatics lecture 8 structured learning. kevin small & byron wallace

ECE521 Lecture 19 HMM cont. Inference in HMM

Bayesian Networks. instructor: Matteo Pozzi. x 1. x 2. x 3 x 4. x 5. x 6. x 7. x 8. x 9. Lec : Urban Systems Modeling

CS 188 Fall Introduction to Artificial Intelligence Midterm 2

Final Exam December 12, 2017

Why do we care? Examples. Bayes Rule. What room am I in? Handling uncertainty over time: predicting, estimating, recognizing, learning

COMP90051 Statistical Machine Learning

Partially Observable Markov Decision Processes (POMDPs)

Forward algorithm vs. particle filtering

Probability. CS 3793/5233 Artificial Intelligence Probability 1

Miscellaneous. Regarding reading materials. Again, ask questions (if you have) and ask them earlier

Probability Hal Daumé III. Computer Science University of Maryland CS 421: Introduction to Artificial Intelligence 27 Mar 2012

Transcription:

CS 5522: Artificial Intelligence II Hidden Markov Models Instructor: Wei Xu Ohio State University [These slides were adapted from CS188 Intro to AI at UC Berkeley.]

Pacman Sonar (P4) [Demo: Pacman Sonar No Beliefs(L14D1)

Video of Demo Pacman Sonar (no beliefs)

Probability Recap Conditional probability Product rule Chain rule X, Y independent if and only if: X and Y are conditionally independent given Z if and only if:

Hidden Markov Models

Hidden Markov Models Markov chains not so useful for most agents Need observations to update your beliefs Hidden Markov models (HMMs) Underlying Markov chain over states X You observe outputs (effects) at each time step X 1 X 2 X 3 X 4 X 5 E 1 E 2 E 3 E 4 E 5

Example: Weather HMM Rain t-1 Rain t Rain t+1 Umbrella t-1 Umbrella t Umbrella t+1 An HMM is defined by: Initial distribution: Transitions: Emissions: R t R t+1 P(R t+1 R t ) +r +r 0.7 +r -r 0.3 -r +r 0.3 -r -r 0.7 R t U t P(U t R t ) +r +u 0.9 +r -u 0.1 -r +u 0.2 -r -u 0.8

Example: Ghostbusters HMM P(X 1 ) = uniform 1/9 1/9 1/9 P(X X ) = usually move clockwise, but sometimes move in a random direction or stay in place 1/9 1/9 1/9 1/9 1/9 1/9 P(X 1 ) P(R ij X) = same sensor model as before: red means close, green means far away. 1/6 1/6 0 1/6 1/2 0 X 1 X 2 X 3 X 4 0 0 0 X 5 P(X X =<1,2>) R i,j R i,j R i,j R i,j [Demo: Ghostbusters Circular Dynamics HMM (L14D2)]

Video of Demo Ghostbusters Circular Dynamics -- HMM

Joint Distribution of an HMM X 1 X 2 X 3 X 5 E 1 E 2 E 3 E 5 Joint distribution: More generally: Questions to be resolved: Does this indeed define a joint distribution? Can every joint distribution be factored this way, or are we making some assumptions about the joint distribution by using this factorization?

Chain Rule and HMMs X 1 X 2 X 3 E 1 E 2 E 3 From the chain rule, every joint distribution over written as: can be Assuming that gives us the expression posited on the previous slide:

Chain Rule and HMMs X 1 X 2 X 3 E 1 E 2 E 3 From the chain rule, every joint distribution over written as: can be Assuming that for all t: State independent of all past states and all past evidence given the previous state, i.e.: Evidence is independent of all past states and all past evidence given the current state, i.e.: gives us the expression posited on the earlier slide:

Implied Conditional Independencies X 1 X 2 X 3 E 1 E 2 E 3 Many implied conditional independencies, e.g., To prove them Approach 1: follow similar (algebraic) approach to what we did in the Markov models lecture Approach 2: directly from the graph structure (3 lectures from now) Intuition: If path between U and V goes through W, then [Some fineprint later]

Real HMM Examples Speech recognition HMMs: Observations are acoustic signals (continuous valued) States are specific positions in specific words (so, tens of thousands) Machine translation HMMs: Observations are words (tens of thousands) States are translation options Robot tracking: Observations are range readings (continuous) States are positions on a map (continuous)

Filtering / Monitoring Filtering, or monitoring, is the task of tracking the distribution B t (X) = P t (X t e 1,, e t ) (the belief state) over time We start with B 1 (X) in an initial setting, usually uniform As time passes, or we get observations, we update B(X) The Kalman filter was invented in the 60 s and first implemented as a method of trajectory estimation for the Apollo program

Example: Robot Localization Example from Michael Pfeiffer Prob 0 t=0 Sensor model: can read in which directions there is a wall, never more than 1 mistake Motion model: may not execute action with small prob. 1

Example: Robot Localization Prob 0 t=1 Lighter grey: was possible to get the reading, but less likely b/c required 1 mistake 1

Example: Robot Localization Prob 0 1 t=2

Example: Robot Localization Prob 0 1 t=3

Example: Robot Localization Prob 0 1 t=4

Example: Robot Localization Prob 0 1 t=5

Inference: Base Cases X 1 X 1 X 2 E 1

Passage of Time Assume we have current belief P(X evidence to date) X 1 X 2 Then, after one time step passes: Or compactly: Basic idea: beliefs get pushed through the transitions With the B notation, we have to be careful about what time step t the belief is about, and what evidence it includes

Example: Passage of Time As time passes, uncertainty accumulates (Transition model: ghosts usually go clockwise) T = 1 T = 2 T = 5

Observation Assume we have current belief P(X previous evidence): X 1 Then, after evidence comes in: E 1 Or, compactly: Basic idea: beliefs reweighted by likelihood of evidence Unlike passage of time, we have to renormalize

Example: Observation As we get observations, beliefs get reweighted, uncertainty decreases Before observation After observation

Example: Weather HMM B (+r) = 0.5 B (-r) = 0.5 B (+r) = 0.627 B (-r) = 0.373 B(+r) = 0.5 B(-r) = 0.5 B(+r) = 0.818 B(-r) = 0.182 B(+r) = 0.883 B(-r) = 0.117 Rain 0 Rain 1 Rain 2 Umbrella 1 Umbrella 2 R t R t+1 P(R t+1 R t ) +r +r 0.7 +r -r 0.3 -r +r 0.3 -r -r 0.7 R t U t P(U t R t ) +r +u 0.9 +r -u 0.1 -r +u 0.2 -r -u 0.8

The Forward Algorithm We are given evidence at each time and want to know We can derive the following updates We can normalize as we go if we want to have P(x e) at each time step, or just once at the end

Online Belief Updates Every time step, we start with current P(X evidence) We update for time: X 1 X 2 We update for evidence: X 2 The forward algorithm does both at once (and doesn t normalize) E 2

Pacman Sonar (P4) [Demo: Pacman Sonar No Beliefs(L14D1)]

Video of Demo Pacman Sonar (with beliefs)

Next Time: Particle Filtering and Applications of HMMs