# Learning Static Parameters in Stochastic Processes

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Learning Static Parameters in Stochastic Processes Bharath Ramsundar December 14, Introduction Consider a Markovian stochastic process X T evolving (perhaps nonlinearly) over time variable T. We may observe this process only through observations Y T. This process is parameterized by vector of parameters Θ, which is typically unknown and must be learned from the sequence {Y T }. Learning the parameter Θ is critical to gaining understanding of this class of stochastic processes. Nonlinear stochastic processes arise in a variety of situations. For example, understanding the time evolution of stock value through a standard stochastic volatility model requires the ability to track the nonlinear evolution of price X T. Such problems are typically solved by Sequential Monte Carlo (SMC) Methods, such as particle filtering, but only when parameter vector Θ is provided to the Monte Carlo Algorithm. Standard algorithms such as particle filtering encounter difficulties when Θ is unknown. Consequently, a large literature has developed focusing on algorithms to learn Θ in this setting. For example, researchers from statistics and econometrics have introduced the Particle MCMC Algorithm [1] which mixes Markov Chain Monte Carlo (MCMC) methods with Sequential Monte Carlo algorithms in order to learn parameters Θ. Unfortunately, such algorithms tend to be slow, and often require at least quadratic time complexity [3]. In this project, we introduce a new algorithm that learns parameter Θ while performing inference. Namely, we modify the Decayed MCMC Filtering algorithm [4] to learn static parameters. We then perform empirical analysis showing the robustness of this algorithm in handling static parameters and also prove preliminary correctness results. Direction and guidance for this research were provided by Professor Stuart Russell of UC Berkeley. 2 Nonlinear State Space Model The following toy nonlinear stochastic problem [2] is routinely used to evaluate algorithms for their ability to handle nonlinear time evolution. X n = X n X n cos(1.2n) + V 1 + Xn 1 2 n Y n = X2 n 20 + W n X 1 N (0, 5), the V n are IID drawn from N (0, σ 2 v), and the W n are IID drawn from N (0, σ 2 w) (N (m, σ 2 ) denotes the Gaussian distribution of mean m and variance σ 2 and IID means independent 1

2 and identically distributed). We set parameter vecter Θ = (σ v, σ w ). Henceforth, we maintain the convention in figures that blue lines are observations, green lines are true states, and red lines are the products of inference. Figure 1: Nonlinear State Space Model, σ w = σ v = 3 3 Prior Literature on Static Parameter Learning In this section we review various algorithms created for Static Parameter Learning Problems. 3.1 Particle Filtering The Particle Filter does not learn Static Parameters, but effectively draws samples from nonlinear processes given such parameters. We implemented a simple particle filter to draw samples from the nonlinear state space model. Figure 2 shows that the particle filter can almost infer the true state. 3.2 Particle Markov Chain Monte Carlo The PMCMC algorithm [1] is an extension of the Markov Chain Monte Carlo (MCMC) framework to handle nonlinearities. The PMCMC extends the reach of this framework by using a particle filter to sample from nonlinear distributions. We implemented PMCMC and used it to calculate distributions for parameters σ v, σ w. See Figure 3 for details. Figure 2: Nonlinear State Space Model Particle Filtering, 50 particles, 100 time steps Figure 3: Histogram PMCMC Parameter Estimation 2

3 Algorithm 1: DMCMC: Decayed MCMC Filtering Input: g(t): Decay Function; S: Total Number of Time Steps; K: Gibbs Moves Per Time Step; Output: Approximate Filtering Distributions D s 1 for s = 1 to S do 2 for i = 1 to K do 3 Choose t from g(s); 4 sample X t from P (X t X t 1, X t+1, Y t ); 5 increment count for X s in D s ; 6 return D 1,..., D T ; 4 SDMCMC: Decayed MCMC with Static Parameters In this section we define a variant of the Decayed MCMC Filtering algorithm (introduced in [4]) for the static parameter estimation problem. 4.1 Decayed MCMC Algorithm We start by giving a brief overview of Decayed MCMC Filtering. Assume as before hidden state variables X 1,..., X T and evidence variables Y 1,..., Y T. Decayed MCMC creates a markov chain to target the distribution X T Y 1,... Y T through Gibbs updates. To save time, the algorithm spends progressively less time updating past elements and focuses on recent history. The rate of this decay is given by function g(t) on window [0, T ] which controls sampling. For example g(t ) = 1 gives T the standard Gibbs Sampling Methodology. The result achieved in [4] is that for g α,δ (t) = α(t t + 1) (1+δ) Algorithm 1 converges to the true filtering distribution for X T in time not dependent on T. 4.2 Decayed MCMC for Static Parameters MCMC techniques have long been used in the statistical literature to estimate static parameters. We consequently propose Algorithm (2) as a method to dynamically learn static parameters while filtering Empirical Results Empirical results on the nonlinear state space model shows that Algorithm 2 works well in practice. However, the effectiveness of Algorithm 2 becomes most clear when we emphasize the fact that this inference is performed without prior knowledge of Θ = (σ v, σ w ). For comparison, we perform inference in the Particle Filter with Θ initialized according to P Θ. From Figures 4 and 5, we see that the particle filter noticeably diverges from the true state, while SDMCMC achieves near perfect accuracy. In fact, with growing sequence size S, Algorithm 2 does not lose accuracy, while the particle filter does. Figure 6 compares the L 2 distances of the inferred and true solutions for SDMCMC and 3

4 Algorithm 2: SDMCMC: Decayed MCMC Filtering for Static Parameters Input: g(t): Decay Function; S: Total Number of Time Steps; K: Gibbs Moves Per Time Step; Output: Approximate Filtering Distributions D s, parameter Θ 1 Sample Θ from prior P Θ ; 2 for s = 1 to S do 3 for i = 1 to K do 4 Choose t from g(s); 5 Sample u from U([0, 1]); 6 if u < 1 then S 7 Resample Θ from P (Θ X 1,..., X s ); else sample X t from P (X t X t 1, X t+1, Y t, Θ); increment count for X s in D s ; Sample X s+1 from P (X s+1 X s, Y s+1, Θ); return D 1,..., D T, Θ; Figure 4: Decayed MCMC, K = 3 Figure 5: Particle Filter, 50 particles SMC (Particle Filter) methods for S = 200 timesteps. This figure indicates that the L 2 distance is divergent for growing T with SMC methods but is convergent for SDMCMC. Finally, we consider the parameter learning capabilities of SDMCMC. Figures 7 and 8 show the histograms of σ v, σ w considered in a run of Algorithm 2. Although the distribution does not center around the true parameters σ v = σ w = 3, it is close. The diagrams suggest that there might be some bias. Further analysis is required to clarify this point Preliminary Mathematical Correctness Analysis We present some preliminary mathematical analysis of Algorithm 2. Theorem 4.1. For all s < S in the outer for loop of Algorithm 2, the inner for loop defines a Markov process with stationary distribution X 1,..., X s, Θ Y 1,..., Y s. Proof. It suffices to show that the distribution X 1,..., X s, Θ Y 1,..., Y s is invariant under an action of the inner for loop. To do so, we will consider the two cases of the if condition separately. Suppose u < 1/S. Note that the marginal distribution (summing out Θ) on X 1,... X s is exactly 4

5 Figure 6: Log-Log Comparison of L 2 distance for SDM- Figure 7: σ v Histogram, True CMC and SMC σ v = 3 Figure 8: σ w Histogram, True σ w = 3 the conditional X 1,..., X s Y 1,..., Y s. Algorithm 2 samples Θ P (Θ X 1,..., X s ). It follows that the joint distribution after the sample is P (Θ X 1,..., X s )P (X 1,..., X s Y 1,..., Y s ) = P (Θ X 1,..., X s, Y 1,..., Y s )P (X 1,..., X s Y 1,..., Y s ) = P (Θ, X 1,..., X s Y 1,..., Y s ) Now suppose that u 1/S. Suppose that i is sampled according to decay g(s). Then the marginal distribution (summing out X i ) is Θ, X 1,..., ˆX i,..., X s P (Θ, X 1,..., ˆX i,..., X s Y 1,..., Y s ) The hat signifies exclusion. Conditional independence shows that the joint distribution after the Gibbs sample is then P (X i Θ, X i 1, X i+1, Y i )P (Θ, X 1,..., ˆX i,... X s Y 1,..., Y s ) = P (X i Θ, X 1,..., ˆX i,..., X s, Y 1,..., Y s )P (Θ, X 1,..., ˆX i,..., X s Y 1,..., Y s ) = P (Θ, X 1,..., X s Y 1,..., Y s ) References [1] C. Andrieu, A. Doucet, and R. Holenstein. Particle markov chain monte carlo methods. Journal of the Royal Statistical Society, [2] N.J. Gordon, D.J. Salmond, and A.F.M. Smith. Novel approach to nonlinear/non-gaussian bayesian state estimation. In Radar and Signal Processing, IEE Proceedings F, volume 140, pages IET, [3] N. Kantas. An overview of sequential monte carlo methods for parameter estimation in general state-space models. In Proc. IFAC Symposium on System Identification (SYSID), [4] B. Marthi, H. Pasula, S. Russell, and Y. Peres. Decayed mcmc filtering. In Proceedings of the Eighteenth conference on Uncertainty in artificial intelligence, pages Morgan Kaufmann Publishers Inc.,

### The Hierarchical Particle Filter

and Arnaud Doucet http://go.warwick.ac.uk/amjohansen/talks MCMSki V Lenzerheide 7th January 2016 Context & Outline Filtering in State-Space Models: SIR Particle Filters [GSS93] Block-Sampling Particle

### PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA Contents in latter part Linear Dynamical Systems What is different from HMM? Kalman filter Its strength and limitation Particle Filter

### Advanced Computational Methods in Statistics: Lecture 5 Sequential Monte Carlo/Particle Filtering

Advanced Computational Methods in Statistics: Lecture 5 Sequential Monte Carlo/Particle Filtering Axel Gandy Department of Mathematics Imperial College London http://www2.imperial.ac.uk/~agandy London

### An Brief Overview of Particle Filtering

1 An Brief Overview of Particle Filtering Adam M. Johansen a.m.johansen@warwick.ac.uk www2.warwick.ac.uk/fac/sci/statistics/staff/academic/johansen/talks/ May 11th, 2010 Warwick University Centre for Systems

### An introduction to Sequential Monte Carlo

An introduction to Sequential Monte Carlo Thang Bui Jes Frellsen Department of Engineering University of Cambridge Research and Communication Club 6 February 2014 1 Sequential Monte Carlo (SMC) methods

### Efficient Monitoring for Planetary Rovers

International Symposium on Artificial Intelligence and Robotics in Space (isairas), May, 2003 Efficient Monitoring for Planetary Rovers Vandi Verma vandi@ri.cmu.edu Geoff Gordon ggordon@cs.cmu.edu Carnegie

### Introduction to Particle Filters for Data Assimilation

Introduction to Particle Filters for Data Assimilation Mike Dowd Dept of Mathematics & Statistics (and Dept of Oceanography Dalhousie University, Halifax, Canada STATMOS Summer School in Data Assimila5on,

### Markov Chain Monte Carlo Methods for Stochastic

Markov Chain Monte Carlo Methods for Stochastic Optimization i John R. Birge The University of Chicago Booth School of Business Joint work with Nicholas Polson, Chicago Booth. JRBirge U Florida, Nov 2013

### Sensor Fusion: Particle Filter

Sensor Fusion: Particle Filter By: Gordana Stojceska stojcesk@in.tum.de Outline Motivation Applications Fundamentals Tracking People Advantages and disadvantages Summary June 05 JASS '05, St.Petersburg,

### Particle Filters. Outline

Particle Filters M. Sami Fadali Professor of EE University of Nevada Outline Monte Carlo integration. Particle filter. Importance sampling. Degeneracy Resampling Example. 1 2 Monte Carlo Integration Numerical

### Hidden Markov Models. Hal Daumé III. Computer Science University of Maryland CS 421: Introduction to Artificial Intelligence 19 Apr 2012

Hidden Markov Models Hal Daumé III Computer Science University of Maryland me@hal3.name CS 421: Introduction to Artificial Intelligence 19 Apr 2012 Many slides courtesy of Dan Klein, Stuart Russell, or

Self Adaptive Particle Filter Alvaro Soto Pontificia Universidad Catolica de Chile Department of Computer Science Vicuna Mackenna 4860 (143), Santiago 22, Chile asoto@ing.puc.cl Abstract The particle filter

### CS 188: Artificial Intelligence Spring Announcements

CS 188: Artificial Intelligence Spring 2011 Lecture 18: HMMs and Particle Filtering 4/4/2011 Pieter Abbeel --- UC Berkeley Many slides over this course adapted from Dan Klein, Stuart Russell, Andrew Moore

### Contrastive Divergence

Contrastive Divergence Training Products of Experts by Minimizing CD Hinton, 2002 Helmut Puhr Institute for Theoretical Computer Science TU Graz June 9, 2010 Contents 1 Theory 2 Argument 3 Contrastive

### Lecture 2: From Linear Regression to Kalman Filter and Beyond

Lecture 2: From Linear Regression to Kalman Filter and Beyond Department of Biomedical Engineering and Computational Science Aalto University January 26, 2012 Contents 1 Batch and Recursive Estimation

### Learning of state-space models with highly informative observations: a tempered Sequential Monte Carlo solution

Learning of state-space models with highly informative observations: a tempered Sequential Monte Carlo solution Andreas Svensson, Thomas B. Schön, and Fredrik Lindsten Department of Information Technology,

### Density Estimation. Seungjin Choi

Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/

### An Introduction to Particle Filtering

An Introduction to Particle Filtering Author: Lisa Turner Supervisor: Dr. Christopher Sherlock 10th May 2013 Abstract This report introduces the ideas behind particle filters, looking at the Kalman filter

### CS 188: Artificial Intelligence. Bayes Nets

CS 188: Artificial Intelligence Probabilistic Inference: Enumeration, Variable Elimination, Sampling Pieter Abbeel UC Berkeley Many slides over this course adapted from Dan Klein, Stuart Russell, Andrew

### A Nearly-Black-Box Online Algorithm for Joint Parameter and State Estimation in Temporal Models

A Nearly-Black-Box Online Algorithm for Joint Parameter and State Estimation in Temporal Models Yusuf B. Erol Yi Wu Lei Li Stuart Russell EECS, UC Berkeley {yberol,jxwuyi,russell}@eecs.berkeley.edu Toutiao

### Kalman filtering and friends: Inference in time series models. Herke van Hoof slides mostly by Michael Rubinstein

Kalman filtering and friends: Inference in time series models Herke van Hoof slides mostly by Michael Rubinstein Problem overview Goal Estimate most probable state at time k using measurement up to time

### An introduction to particle filters

An introduction to particle filters Andreas Svensson Department of Information Technology Uppsala University June 10, 2014 June 10, 2014, 1 / 16 Andreas Svensson - An introduction to particle filters Outline

### CS 188: Artificial Intelligence

CS 188: Artificial Intelligence Hidden Markov Models Instructor: Anca Dragan --- University of California, Berkeley [These slides were created by Dan Klein, Pieter Abbeel, and Anca. http://ai.berkeley.edu.]

### CPSC 540: Machine Learning

CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is

Sensor Tasking and Control Sensing Networking Leonidas Guibas Stanford University Computation CS428 Sensor systems are about sensing, after all... System State Continuous and Discrete Variables The quantities

### Sequential Monte Carlo and Particle Filtering. Frank Wood Gatsby, November 2007

Sequential Monte Carlo and Particle Filtering Frank Wood Gatsby, November 2007 Importance Sampling Recall: Let s say that we want to compute some expectation (integral) E p [f] = p(x)f(x)dx and we remember

Adaptive Monte Carlo methods Jean-Michel Marin Projet Select, INRIA Futurs, Université Paris-Sud joint with Randal Douc (École Polytechnique), Arnaud Guillin (Université de Marseille) and Christian Robert

### Divide-and-Conquer Sequential Monte Carlo

Divide-and-Conquer Joint work with: John Aston, Alexandre Bouchard-Côté, Brent Kirkpatrick, Fredrik Lindsten, Christian Næsseth, Thomas Schön University of Warwick a.m.johansen@warwick.ac.uk http://go.warwick.ac.uk/amjohansen/talks/

### Sequential Monte Carlo methods for system identification

Technical report arxiv:1503.06058v3 [stat.co] 10 Mar 2016 Sequential Monte Carlo methods for system identification Thomas B. Schön, Fredrik Lindsten, Johan Dahlin, Johan Wågberg, Christian A. Naesseth,

### STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 7 Approximate

### Approximate Bayesian Computation: a simulation based approach to inference

Approximate Bayesian Computation: a simulation based approach to inference Richard Wilkinson Simon Tavaré 2 Department of Probability and Statistics University of Sheffield 2 Department of Applied Mathematics

### Markov Chain Monte Carlo Lecture 6

Sequential parallel tempering With the development of science and technology, we more and more need to deal with high dimensional systems. For example, we need to align a group of protein or DNA sequences

### An ABC interpretation of the multiple auxiliary variable method

School of Mathematical and Physical Sciences Department of Mathematics and Statistics Preprint MPS-2016-07 27 April 2016 An ABC interpretation of the multiple auxiliary variable method by Dennis Prangle

### A FEASIBILITY STUDY OF PARTICLE FILTERS FOR MOBILE STATION RECEIVERS. Michael Lunglmayr, Martin Krueger, Mario Huemer

A FEASIBILITY STUDY OF PARTICLE FILTERS FOR MOBILE STATION RECEIVERS Michael Lunglmayr, Martin Krueger, Mario Huemer Michael Lunglmayr and Martin Krueger are with Infineon Technologies AG, Munich email:

### CS 188: Artificial Intelligence Spring 2009

CS 188: Artificial Intelligence Spring 2009 Lecture 21: Hidden Markov Models 4/7/2009 John DeNero UC Berkeley Slides adapted from Dan Klein Announcements Written 3 deadline extended! Posted last Friday

### Computer Intensive Methods in Mathematical Statistics

Computer Intensive Methods in Mathematical Statistics Department of mathematics johawes@kth.se Lecture 16 Advanced topics in computational statistics 18 May 2017 Computer Intensive Methods (1) Plan of

### Variational Bayesian Inference Techniques

Advanced Signal Processing 2, SE Variational Bayesian Inference Techniques Johann Steiner 1 Outline Introduction Sparse Signal Reconstruction Sparsity Priors Benefits of Sparse Bayesian Inference Variational

### Computer Practical: Metropolis-Hastings-based MCMC

Computer Practical: Metropolis-Hastings-based MCMC Andrea Arnold and Franz Hamilton North Carolina State University July 30, 2016 A. Arnold / F. Hamilton (NCSU) MH-based MCMC July 30, 2016 1 / 19 Markov

### Bayesian Inference: Probit and Linear Probability Models

Utah State University DigitalCommons@USU All Graduate Plan B and other Reports Graduate Studies 5-1-2014 Bayesian Inference: Probit and Linear Probability Models Nate Rex Reasch Utah State University Follow

### Sampling Algorithms for Probabilistic Graphical models

Sampling Algorithms for Probabilistic Graphical models Vibhav Gogate University of Washington References: Chapter 12 of Probabilistic Graphical models: Principles and Techniques by Daphne Koller and Nir

### CONDENSATION Conditional Density Propagation for Visual Tracking

CONDENSATION Conditional Density Propagation for Visual Tracking Michael Isard and Andrew Blake Presented by Neil Alldrin Department of Computer Science & Engineering University of California, San Diego

### Sequential Bayesian Updating

BS2 Statistical Inference, Lectures 14 and 15, Hilary Term 2009 May 28, 2009 We consider data arriving sequentially X 1,..., X n,... and wish to update inference on an unknown parameter θ online. In a

### Paul Karapanagiotidis ECO4060

Paul Karapanagiotidis ECO4060 The way forward 1) Motivate why Markov-Chain Monte Carlo (MCMC) is useful for econometric modeling 2) Introduce Markov-Chain Monte Carlo (MCMC) - Metropolis-Hastings (MH)

### Nonlinear State Estimation! Particle, Sigma-Points Filters!

Nonlinear State Estimation! Particle, Sigma-Points Filters! Robert Stengel! Optimal Control and Estimation, MAE 546! Princeton University, 2017!! Particle filter!! Sigma-Points Unscented Kalman ) filter!!

### Sampling Rejection Sampling Importance Sampling Markov Chain Monte Carlo. Sampling Methods. Oliver Schulte - CMPT 419/726. Bishop PRML Ch.

Sampling Methods Oliver Schulte - CMP 419/726 Bishop PRML Ch. 11 Recall Inference or General Graphs Junction tree algorithm is an exact inference method for arbitrary graphs A particular tree structure

### Modeling Ultra-High-Frequency Multivariate Financial Data by Monte Carlo Simulation Methods

Outline Modeling Ultra-High-Frequency Multivariate Financial Data by Monte Carlo Simulation Methods Ph.D. Student: Supervisor: Marco Minozzo Dipartimento di Scienze Economiche Università degli Studi di

### Bayesian Computations for DSGE Models

Bayesian Computations for DSGE Models Frank Schorfheide University of Pennsylvania, PIER, CEPR, and NBER October 23, 2017 This Lecture is Based on Bayesian Estimation of DSGE Models Edward P. Herbst &

### Particle filters, the optimal proposal and high-dimensional systems

Particle filters, the optimal proposal and high-dimensional systems Chris Snyder National Center for Atmospheric Research Boulder, Colorado 837, United States chriss@ucar.edu 1 Introduction Particle filters

### Monte Carlo Solution of Integral Equations

1 Monte Carlo Solution of Integral Equations of the second kind Adam M. Johansen a.m.johansen@warwick.ac.uk www2.warwick.ac.uk/fac/sci/statistics/staff/academic/johansen/talks/ March 7th, 2011 MIRaW Monte

### State Estimation using Moving Horizon Estimation and Particle Filtering

State Estimation using Moving Horizon Estimation and Particle Filtering James B. Rawlings Department of Chemical and Biological Engineering UW Math Probability Seminar Spring 2009 Rawlings MHE & PF 1 /

### Research Division Federal Reserve Bank of St. Louis Working Paper Series

Research Division Federal Reserve Bank of St Louis Working Paper Series Kalman Filtering with Truncated Normal State Variables for Bayesian Estimation of Macroeconomic Models Michael Dueker Working Paper

Particle Metropolis-adjusted Langevin algorithms Christopher Nemeth, Chris Sherlock and Paul Fearnhead arxiv:1412.7299v3 [stat.me] 27 May 2016 Department of Mathematics and Statistics, Lancaster University,

### Probabilistic Graphical Models

Probabilistic Graphical Models Brown University CSCI 295-P, Spring 213 Prof. Erik Sudderth Lecture 11: Inference & Learning Overview, Gaussian Graphical Models Some figures courtesy Michael Jordan s draft

### Statistical Machine Learning Lecture 8: Markov Chain Monte Carlo Sampling

1 / 27 Statistical Machine Learning Lecture 8: Markov Chain Monte Carlo Sampling Melih Kandemir Özyeğin University, İstanbul, Turkey 2 / 27 Monte Carlo Integration The big question : Evaluate E p(z) [f(z)]

### Latent Dirichlet Allocation (LDA)

Latent Dirichlet Allocation (LDA) A review of topic modeling and customer interactions application 3/11/2015 1 Agenda Agenda Items 1 What is topic modeling? Intro Text Mining & Pre-Processing Natural Language

### A nested sampling particle filter for nonlinear data assimilation

Quarterly Journal of the Royal Meteorological Society Q. J. R. Meteorol. Soc. : 14, July 2 A DOI:.2/qj.224 A nested sampling particle filter for nonlinear data assimilation Ahmed H. Elsheikh a,b *, Ibrahim

### 2D Image Processing. Bayes filter implementation: Kalman filter

2D Image Processing Bayes filter implementation: Kalman filter Prof. Didier Stricker Dr. Gabriele Bleser Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche

### Bayesian Quadrature: Model-based Approximate Integration. David Duvenaud University of Cambridge

Bayesian Quadrature: Model-based Approimate Integration David Duvenaud University of Cambridge The Quadrature Problem ˆ We want to estimate an integral Z = f ()p()d ˆ Most computational problems in inference

### Notes on pseudo-marginal methods, variational Bayes and ABC

Notes on pseudo-marginal methods, variational Bayes and ABC Christian Andersson Naesseth October 3, 2016 The Pseudo-Marginal Framework Assume we are interested in sampling from the posterior distribution

### The Kalman Filter ImPr Talk

The Kalman Filter ImPr Talk Ged Ridgway Centre for Medical Image Computing November, 2006 Outline What is the Kalman Filter? State Space Models Kalman Filter Overview Bayesian Updating of Estimates Kalman

### Artificial Intelligence Bayesian Networks

Artificial Intelligence Bayesian Networks Stephan Dreiseitl FH Hagenberg Software Engineering & Interactive Media Stephan Dreiseitl (Hagenberg/SE/IM) Lecture 11: Bayesian Networks Artificial Intelligence

### Lecture 2: From Linear Regression to Kalman Filter and Beyond

Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing

### Gaussian Process Volatility Model

Gaussian Process Volatility Model Yue Wu Cambridge University wu5@post.harvard.edu José Miguel Hernández Lobato Cambridge University jmh33@cam.ac.uk Zoubin Ghahramani Cambridge University zoubin@eng.cam.ac.uk

### Monetary and Exchange Rate Policy Under Remittance Fluctuations. Technical Appendix and Additional Results

Monetary and Exchange Rate Policy Under Remittance Fluctuations Technical Appendix and Additional Results Federico Mandelman February In this appendix, I provide technical details on the Bayesian estimation.

### SAMPLING ALGORITHMS. In general. Inference in Bayesian models

SAMPLING ALGORITHMS SAMPLING ALGORITHMS In general A sampling algorithm is an algorithm that outputs samples x 1, x 2,... from a given distribution P or density p. Sampling algorithms can for example be

### Stat 535 C - Statistical Computing & Monte Carlo Methods. Arnaud Doucet.

Stat 535 C - Statistical Computing & Monte Carlo Methods Arnaud Doucet Email: arnaud@cs.ubc.ca 1 1.1 Outline Introduction to Markov chain Monte Carlo The Gibbs Sampler Examples Overview of the Lecture

### Introduction to Bayesian methods in inverse problems

Introduction to Bayesian methods in inverse problems Ville Kolehmainen 1 1 Department of Applied Physics, University of Eastern Finland, Kuopio, Finland March 4 2013 Manchester, UK. Contents Introduction

### Approximating a single component of the solution to a linear system

Approximating a single component of the solution to a linear system Christina E. Lee, Asuman Ozdaglar, Devavrat Shah celee@mit.edu asuman@mit.edu devavrat@mit.edu MIT LIDS 1 How do I compare to my competitors?

Adaptive Dual Control Björn Wittenmark Department of Automatic Control, Lund Institute of Technology Box 118, S-221 00 Lund, Sweden email: bjorn@control.lth.se Keywords: Dual control, stochastic control,

### Particle Methods as Message Passing

Particle Methods as Message Passing Justin Dauwels RIKEN Brain Science Institute Hirosawa,2-1,Wako-shi,Saitama,Japan Email: justin@dauwels.com Sascha Korl Phonak AG CH-8712 Staefa, Switzerland Email: sascha.korl@phonak.ch

### Distributed Bayesian Learning with Stochastic Natural-gradient EP and the Posterior Server

Distributed Bayesian Learning with Stochastic Natural-gradient EP and the Posterior Server in collaboration with: Minjie Xu, Balaji Lakshminarayanan, Leonard Hasenclever, Thibaut Lienart, Stefan Webb,

### Markov Chain Monte Carlo, Numerical Integration

Markov Chain Monte Carlo, Numerical Integration (See Statistics) Trevor Gallen Fall 2015 1 / 1 Agenda Numerical Integration: MCMC methods Estimating Markov Chains Estimating latent variables 2 / 1 Numerical

### Data Analysis and Uncertainty Part 2: Estimation

Data Analysis and Uncertainty Part 2: Estimation Instructor: Sargur N. University at Buffalo The State University of New York srihari@cedar.buffalo.edu 1 Topics in Estimation 1. Estimation 2. Desirable

### Tracking. Readings: Chapter 17 of Forsyth and Ponce. Matlab Tutorials: motiontutorial.m. 2503: Tracking c D.J. Fleet & A.D. Jepson, 2009 Page: 1

Goal: Tracking Fundamentals of model-based tracking with emphasis on probabilistic formulations. Examples include the Kalman filter for linear-gaussian problems, and maximum likelihood and particle filters

### PART I INTRODUCTION The meaning of probability Basic definitions for frequentist statistics and Bayesian inference Bayesian inference Combinatorics

Table of Preface page xi PART I INTRODUCTION 1 1 The meaning of probability 3 1.1 Classical definition of probability 3 1.2 Statistical definition of probability 9 1.3 Bayesian understanding of probability

### Kurume University Faculty of Economics Monograph Collection 18. Theoretical Advances and Applications in. Operations Research. Kyushu University Press

Kurume University Faculty of Economics Monograph Collection 18 Theoretical Advances and Applications in Operations Research Managing Editor Kangrong Tan General Advisor Joe Gani Kyushu University Press

### Particle-Based Approximate Inference on Graphical Model

article-based Approimate Inference on Graphical Model Reference: robabilistic Graphical Model Ch. 2 Koller & Friedman CMU, 0-708, Fall 2009 robabilistic Graphical Models Lectures 8,9 Eric ing attern Recognition

### Recurrent Latent Variable Networks for Session-Based Recommendation

Recurrent Latent Variable Networks for Session-Based Recommendation Panayiotis Christodoulou Cyprus University of Technology paa.christodoulou@edu.cut.ac.cy 27/8/2017 Panayiotis Christodoulou (C.U.T.)

### Regression with Input-Dependent Noise: A Bayesian Treatment

Regression with Input-Dependent oise: A Bayesian Treatment Christopher M. Bishop C.M.BishopGaston.ac.uk Cazhaow S. Qazaz qazazcsgaston.ac.uk eural Computing Research Group Aston University, Birmingham,

### Sequential Monte Carlo for Graphical Models

Sequential Monte Carlo for Graphical Models Christian A. Naesseth Div. of Automatic Control Linköping University Linköping, Sweden chran60@isy.liu.se Fredrik Lindsten Dept. of Engineering The University

### Nonparametric Drift Estimation for Stochastic Differential Equations

Nonparametric Drift Estimation for Stochastic Differential Equations Gareth Roberts 1 Department of Statistics University of Warwick Brazilian Bayesian meeting, March 2010 Joint work with O. Papaspiliopoulos,

### Petr Volf. Model for Difference of Two Series of Poisson-like Count Data

Petr Volf Institute of Information Theory and Automation Academy of Sciences of the Czech Republic Pod vodárenskou věží 4, 182 8 Praha 8 e-mail: volf@utia.cas.cz Model for Difference of Two Series of Poisson-like

### Particle Metropolis-Hastings using gradient and Hessian information

Particle Metropolis-Hastings using gradient and Hessian information Johan Dahlin, Fredrik Lindsten and Thomas B. Schön September 19, 2014 Abstract Particle Metropolis-Hastings (PMH) allows for Bayesian

### Learning in Markov Random Fields An Empirical Study

Learning in Markov Random Fields An Empirical Study Sridevi Parise, Max Welling University of California, Irvine sparise,welling@ics.uci.edu Abstract Learning the parameters of an undirected graphical

### Monte Carlo Methods for Computation and Optimization (048715)

Technion Department of Electrical Engineering Monte Carlo Methods for Computation and Optimization (048715) Lecture Notes Prof. Nahum Shimkin Spring 2015 i PREFACE These lecture notes are intended for

### Extended Object and Group Tracking with Elliptic Random Hypersurface Models

Extended Object and Group Tracing with Elliptic Random Hypersurface Models Marcus Baum Benjamin Noac and Uwe D. Hanebec Intelligent Sensor-Actuator-Systems Laboratory ISAS Institute for Anthropomatics

### Cover Page. The handle holds various files of this Leiden University dissertation

Cover Page The handle http://hdl.handle.net/1887/39637 holds various files of this Leiden University dissertation Author: Smit, Laurens Title: Steady-state analysis of large scale systems : the successive

### Generalized Method of Moments Estimation

Generalized Method of Moments Estimation Lars Peter Hansen March 0, 2007 Introduction Generalized methods of moments (GMM) refers to a class of estimators which are constructed from exploiting the sample

### Gate Volume Estimation for Target Tracking

Gate Volume Estimation for Target Tracking Darko Mušicki Mark R. Morelande Dept of Electrical Engineering Dept of Electrical Engineering University of Melbourne University of Melbourne Victoria 30 Victoria

### Kazuhiko Kakamu Department of Economics Finance, Institute for Advanced Studies. Abstract

Bayesian Estimation of A Distance Functional Weight Matrix Model Kazuhiko Kakamu Department of Economics Finance, Institute for Advanced Studies Abstract This paper considers the distance functional weight

### Bayesian linear regression

Bayesian linear regression Linear regression is the basis of most statistical modeling. The model is Y i = X T i β + ε i, where Y i is the continuous response X i = (X i1,..., X ip ) T is the corresponding

### A Monte Carlo Sampling Approach to Testing Separate Families of Hypotheses: Monte Carlo Results

A Monte Carlo Sampling Approach to Testing Separate Families of Hypotheses: Monte Carlo Results by Nouhoun Coulibaly and B. Wade Brorsen * * Nouhoun Coulibaly is a Ph.D. candidate and B. Wade Brorsen is

### Monte Carlo Methods. Geoff Gordon February 9, 2006

Monte Carlo Methods Geoff Gordon ggordon@cs.cmu.edu February 9, 2006 Numerical integration problem 5 4 3 f(x,y) 2 1 1 0 0.5 0 X 0.5 1 1 0.8 0.6 0.4 Y 0.2 0 0.2 0.4 0.6 0.8 1 x X f(x)dx Used for: function

### Risk Estimation and Uncertainty Quantification by Markov Chain Monte Carlo Methods

Risk Estimation and Uncertainty Quantification by Markov Chain Monte Carlo Methods Konstantin Zuev Institute for Risk and Uncertainty University of Liverpool http://www.liv.ac.uk/risk-and-uncertainty/staff/k-zuev/

### Mixed Membership Matrix Factorization

Mixed Membership Matrix Factorization Lester Mackey University of California, Berkeley Collaborators: David Weiss, University of Pennsylvania Michael I. Jordan, University of California, Berkeley 2011

### An introduction to adaptive MCMC

An introduction to adaptive MCMC Gareth Roberts MIRAW Day on Monte Carlo methods March 2011 Mainly joint work with Jeff Rosenthal. http://www2.warwick.ac.uk/fac/sci/statistics/crism/ Conferences and workshops

### The Origin of Deep Learning. Lili Mou Jan, 2015

The Origin of Deep Learning Lili Mou Jan, 2015 Acknowledgment Most of the materials come from G. E. Hinton s online course. Outline Introduction Preliminary Boltzmann Machines and RBMs Deep Belief Nets

### Bayesian Heteroskedasticity-Robust Regression. Richard Startz * revised February Abstract

Bayesian Heteroskedasticity-Robust Regression Richard Startz * revised February 2015 Abstract I offer here a method for Bayesian heteroskedasticity-robust regression. The Bayesian version is derived by