Risk Estimation and Uncertainty Quantification by Markov Chain Monte Carlo Methods
|
|
- Garry Booth
- 6 years ago
- Views:
Transcription
1 Risk Estimation and Uncertainty Quantification by Markov Chain Monte Carlo Methods Konstantin Zuev Institute for Risk and Uncertainty University of Liverpool 8 Nov, 2013 Workshop on Risk and Uncertainty Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 1 / 11
2 MCMC Revolution P. Diaconis (2009), The Markov chain Monte Carlo revolution :...asking about applications of Markov chain Monte Carlo (MCMC) is a little like asking about applications of the quadratic formula... you can take any area of science, from hard to social, and find a burgeoning MCMC literature specifically tailored to that area. decoupl i ngbl og. com Konstantin Zuev (University of Liverpool) sl owmuse. wor dpr ess. com sl ow RE and UQ by MCMC wwwt hphys. physi cs. ox. ac. uk comput er weekl y. com Workshop on Risk and Uncertainty 2 / 11
3 MCMC Revolution P. Diaconis (2009), The Markov chain Monte Carlo revolution :...asking about applications of Markov chain Monte Carlo (MCMC) is a little like asking about applications of the quadratic formula... you can take any area of science, from hard to social, and find a burgeoning MCMC literature specifically tailored to that area. decoupl i ngbl og. com Main message: sl owmuse. wor dpr ess. com sl ow wwwt hphys. physi cs. ox. ac. uk comput er weekl y. com MCMC algorithms can be efficiently used for solving engineering problems involving risk and uncertainty Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 2 / 11
4 Outline 1 What problems is MCMC meant to solve? 2 Why is MCMC useful in Engineering? 3 How does MCMC work? Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 3 / 11
5 Outline 1 What problems is MCMC meant to solve? 2 Why is MCMC useful in Engineering? 3 How does MCMC work? Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 3 / 11
6 Purpose of MCMC Problem: To estimate I = h(θ)π(θ)dθ R d parameter space h : R function of interest π(θ) target PDF on Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 4 / 11
7 Purpose of MCMC Problem: To estimate I = h(θ)π(θ)dθ = E π [h] R d parameter space h : R function of interest π(θ) target PDF on Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 4 / 11
8 Purpose of MCMC Problem: To estimate I = h(θ)π(θ)dθ = E π [h] R d parameter space h : R function of interest π(θ) target PDF on Easy Cases: d is small (d = 1, 2, 3) numerical integration π(θ) is easy to sample from Monte Carlo method Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 4 / 11
9 Purpose of MCMC Problem: To estimate I = h(θ)π(θ)dθ = E π [h] R d parameter space h : R function of interest π(θ) target PDF on Easy Cases: d is small (d = 1, 2, 3) numerical integration π(θ) is easy to sample from Monte Carlo method Typical Case in Applications: d is large (d ) π(θ) is known only up to a constant, π(θ) = cf(θ) Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 4 / 11
10 Purpose of MCMC Problem: To estimate I = h(θ)π(θ)dθ = E π [h] R d parameter space h : R function of interest π(θ) target PDF on Easy Cases: d is small (d = 1, 2, 3) numerical integration π(θ) is easy to sample from Monte Carlo method Typical Case in Applications: d is large (d ) π(θ) is known only up to a constant, π(θ) = cf(θ) Solution: Use an appropriate MCMC method Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 4 / 11
11 Outline 1 What problems is MCMC meant to solve? 2 Why is MCMC useful in Engineering? Bayesian Inference Performance-Based Design Optimization Reliability Problem 3 How does MCMC work? Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 5 / 11
12 Bayesian Inference M the assumed model class for the target dynamic system: set of I/O probability models p(y θ, u) u System y θ the uncertain model parameters prior PDF π0(θ) over Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 6 / 11
13 Bayesian Inference M the assumed model class for the target dynamic system: set of I/O probability models p(y θ, u) u System y θ the uncertain model parameters prior PDF π0(θ) over Bayesian approach: Update π 0 (θ) to posterior PDF π(θ D) via Bayes theorem: π(θ D) = L(D θ)π 0(θ) Z D the measured data from the system L(D θ) the likelihood function Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 6 / 11
14 Bayesian Inference M the assumed model class for the target dynamic system: set of I/O probability models p(y θ, u) u System y θ the uncertain model parameters prior PDF π0(θ) over Bayesian approach: Update π 0 (θ) to posterior PDF π(θ D) via Bayes theorem: π(θ D) = L(D θ)π 0(θ) Z D the measured data from the system L(D θ) the likelihood function Problems: Posterior expectations E π [h] = h(θ)π(θ D)dθ Evidence Z = L(D θ)π 0 (θ)dθ Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 6 / 11
15 Performance-Based Design Optimization M the assumed model class for the target dynamic system θ the uncertain model parameters ϕ Φ design variables: controllable parameters that define the system design Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 7 / 11
16 Performance-Based Design Optimization M the assumed model class for the target dynamic system θ the uncertain model parameters ϕ Φ design variables: controllable parameters that define the system design The objective function for a robust-to-uncertainties design: E π [h(ϕ, θ)] = h(ϕ, θ)π(θ ϕ)dθ h(ϕ, θ) a performance function of the system, e.g. stress in a component π(θ ϕ) a PDF, which incorporates available knowledge about the system Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 7 / 11
17 Performance-Based Design Optimization M the assumed model class for the target dynamic system θ the uncertain model parameters ϕ Φ design variables: controllable parameters that define the system design The objective function for a robust-to-uncertainties design: E π [h(ϕ, θ)] = h(ϕ, θ)π(θ ϕ)dθ h(ϕ, θ) a performance function of the system, e.g. stress in a component π(θ ϕ) a PDF, which incorporates available knowledge about the system Problem: ϕ = arg min ϕ Φ E π[h(ϕ, θ)] Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 7 / 11
18 Reliability Problem Reliability Problem: To estimate the probability of failure p F p F = P(θ F ) = π(θ)i F (θ)dθ R d Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 8 / 11
19 Reliability Problem Reliability Problem: To estimate the probability of failure p F p F = P(θ F ) = π(θ)i F (θ)dθ R d Notation: θ R d represents the uncertain input load θ is a random vector and has joint PDF π F R d a failure domain (unacceptable performance) F = {θ : g(θ) b } g(θ) a performance function b a critical threshold for performance I F (θ) = 1 if θ F and I F (θ) = 0 if θ / F Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 8 / 11
20 Outline 1 What problems is MCMC meant to solve? 2 Why is MCMC useful in Engineering? 3 How does MCMC work? Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 9 / 11
21 MCMC: The Main Idea Monte Carlo method h(θ)π(θ)dθ 1 N N h(θ i ), i=1 θ i i.i.d π Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 10 / 11
22 MCMC: The Main Idea Monte Carlo method h(θ)π(θ)dθ 1 N N h(θ i ), i=1 θ i i.i.d π Main difficulty: How to obtain i.i.d. samples from π? Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 10 / 11
23 MCMC: The Main Idea Monte Carlo method h(θ)π(θ)dθ 1 N N h(θ i ), i=1 θ i i.i.d π Main difficulty: How to obtain i.i.d. samples from π? MCMC samples from π and computes integrals using Markov chains: 1 h(θ)π(θ)dθ N N 0 N h(x i ) i=n 0+1 Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 10 / 11
24 Papers Applications of MCMC to Engineering Problems: Bayesian Inference J.L. Beck and K.M. Zuev (2013), Asymptotically independent Markov sampling: a new Markov chain Monte Carlo scheme for Bayesian inference, Int. J. for Uncertainty Quant., 3(5), Performance-Based Design Optimization K.M. Zuev and J.L. Beck (2013), Global optimization using the asymptotically independent Markov sampling method, Computers & Structures, 126, Reliability Problem K.M. Zuev, J.L. Beck, S.K. Au, and L.S. Katafygiotis (2012), Bayesian post-processor and other enhancements of Subset Simulation for estimating failure probabilities in high dimensions, Computers & Structures, 92-93, K.M. Zuev and L.S. Katafygiotis (2011), Modified Metropolis-Hastings algorithm with delayed rejection, Probabilistic Engineering Mechanics, 26(3), K.M. Zuev and L.S. Katafygiotis (2011), The Horseracing Simulation algorithm for evaluation of small failure probabilities, Probabilistic Engineering Mechanics, 26(2), Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 11 / 11
25 Papers Applications of MCMC to Engineering Problems: Bayesian Inference J.L. Beck and K.M. Zuev (2013), Asymptotically independent Markov sampling: a new Markov chain Monte Carlo scheme for Bayesian inference, Int. J. for Uncertainty Quant., 3(5), Performance-Based Design Optimization K.M. Zuev and J.L. Beck (2013), Global optimization using the asymptotically independent Markov sampling method, Computers & Structures, 126, Reliability Problem K.M. Zuev, J.L. Beck, S.K. Au, and L.S. Katafygiotis (2012), Bayesian post-processor and other enhancements of Subset Simulation for estimating failure probabilities in high dimensions, Computers & Structures, 92-93, K.M. Zuev and L.S. Katafygiotis (2011), Modified Metropolis-Hastings algorithm with delayed rejection, Probabilistic Engineering Mechanics, 26(3), K.M. Zuev and L.S. Katafygiotis (2011), The Horseracing Simulation algorithm for evaluation of small failure probabilities, Probabilistic Engineering Mechanics, 26(2), Konstantin Zuev (University of Liverpool) RE and UQ by MCMC Workshop on Risk and Uncertainty 11 / 11
On the Optimal Scaling of the Modified Metropolis-Hastings algorithm
On the Optimal Scaling of the Modified Metropolis-Hastings algorithm K. M. Zuev & J. L. Beck Division of Engineering and Applied Science California Institute of Technology, MC 4-44, Pasadena, CA 925, USA
More informationMODIFIED METROPOLIS-HASTINGS ALGORITHM WITH DELAYED REJECTION FOR HIGH-DIMENSIONAL RELIABILITY ANALYSIS
SEECCM 2009 2nd South-East European Conference on Computational Mechanics An IACM-ECCOMAS Special Interest Conference M. Papadrakakis, M. Kojic, V. Papadopoulos (eds.) Rhodes, Greece, 22-24 June 2009 MODIFIED
More informationMonte Carlo in Bayesian Statistics
Monte Carlo in Bayesian Statistics Matthew Thomas SAMBa - University of Bath m.l.thomas@bath.ac.uk December 4, 2014 Matthew Thomas (SAMBa) Monte Carlo in Bayesian Statistics December 4, 2014 1 / 16 Overview
More informationBayesian Inference and MCMC
Bayesian Inference and MCMC Aryan Arbabi Partly based on MCMC slides from CSC412 Fall 2018 1 / 18 Bayesian Inference - Motivation Consider we have a data set D = {x 1,..., x n }. E.g each x i can be the
More informationBayesian Estimation of DSGE Models 1 Chapter 3: A Crash Course in Bayesian Inference
1 The views expressed in this paper are those of the authors and do not necessarily reflect the views of the Federal Reserve Board of Governors or the Federal Reserve System. Bayesian Estimation of DSGE
More informationBAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA
BAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA Intro: Course Outline and Brief Intro to Marina Vannucci Rice University, USA PASI-CIMAT 04/28-30/2010 Marina Vannucci
More informationA Probabilistic Framework for solving Inverse Problems. Lambros S. Katafygiotis, Ph.D.
A Probabilistic Framework for solving Inverse Problems Lambros S. Katafygiotis, Ph.D. OUTLINE Introduction to basic concepts of Bayesian Statistics Inverse Problems in Civil Engineering Probabilistic Model
More informationMarkov Chain Monte Carlo methods
Markov Chain Monte Carlo methods Tomas McKelvey and Lennart Svensson Signal Processing Group Department of Signals and Systems Chalmers University of Technology, Sweden November 26, 2012 Today s learning
More informationTEORIA BAYESIANA Ralph S. Silva
TEORIA BAYESIANA Ralph S. Silva Departamento de Métodos Estatísticos Instituto de Matemática Universidade Federal do Rio de Janeiro Sumário Numerical Integration Polynomial quadrature is intended to approximate
More informationExample: Ground Motion Attenuation
Example: Ground Motion Attenuation Problem: Predict the probability distribution for Peak Ground Acceleration (PGA), the level of ground shaking caused by an earthquake Earthquake records are used to update
More informationLearning the hyper-parameters. Luca Martino
Learning the hyper-parameters Luca Martino 2017 2017 1 / 28 Parameters and hyper-parameters 1. All the described methods depend on some choice of hyper-parameters... 2. For instance, do you recall λ (bandwidth
More informationDynamic System Identification using HDMR-Bayesian Technique
Dynamic System Identification using HDMR-Bayesian Technique *Shereena O A 1) and Dr. B N Rao 2) 1), 2) Department of Civil Engineering, IIT Madras, Chennai 600036, Tamil Nadu, India 1) ce14d020@smail.iitm.ac.in
More informationarxiv: v1 [stat.co] 23 Apr 2018
Bayesian Updating and Uncertainty Quantification using Sequential Tempered MCMC with the Rank-One Modified Metropolis Algorithm Thomas A. Catanach and James L. Beck arxiv:1804.08738v1 [stat.co] 23 Apr
More informationComputationalToolsforComparing AsymmetricGARCHModelsviaBayes Factors. RicardoS.Ehlers
ComputationalToolsforComparing AsymmetricGARCHModelsviaBayes Factors RicardoS.Ehlers Laboratório de Estatística e Geoinformação- UFPR http://leg.ufpr.br/ ehlers ehlers@leg.ufpr.br II Workshop on Statistical
More informationASYMPTOTICALLY INDEPENDENT MARKOV SAMPLING: A NEW MARKOV CHAIN MONTE CARLO SCHEME FOR BAYESIAN INFERENCE
International Journal for Uncertainty Quantification, 3 (5): 445 474 (213) ASYMPTOTICALLY IDEPEDET MARKOV SAMPLIG: A EW MARKOV CHAI MOTE CARLO SCHEME FOR BAYESIA IFERECE James L. Beck & Konstantin M. Zuev
More informationProbabilistic Machine Learning
Probabilistic Machine Learning Bayesian Nets, MCMC, and more Marek Petrik 4/18/2017 Based on: P. Murphy, K. (2012). Machine Learning: A Probabilistic Perspective. Chapter 10. Conditional Independence Independent
More informationEfficient Aircraft Design Using Bayesian History Matching. Institute for Risk and Uncertainty
Efficient Aircraft Design Using Bayesian History Matching F.A. DiazDelaO, A. Garbuno, Z. Gong Institute for Risk and Uncertainty DiazDelaO (U. of Liverpool) DiPaRT 2017 November 22, 2017 1 / 32 Outline
More informationSTAT 499/962 Topics in Statistics Bayesian Inference and Decision Theory Jan 2018, Handout 01
STAT 499/962 Topics in Statistics Bayesian Inference and Decision Theory Jan 2018, Handout 01 Nasser Sadeghkhani a.sadeghkhani@queensu.ca There are two main schools to statistical inference: 1-frequentist
More informationTransitional Markov Chain Monte Carlo: Observations and Improvements
Transitional Markov Chain Monte Carlo: Observations and Improvements Wolfgang Betz, Iason Papaioannou, Daniel Straub Engineering Risk Analysis Group, Technische Universität München, 8333 München, Germany
More informationMCMC algorithms for fitting Bayesian models
MCMC algorithms for fitting Bayesian models p. 1/1 MCMC algorithms for fitting Bayesian models Sudipto Banerjee sudiptob@biostat.umn.edu University of Minnesota MCMC algorithms for fitting Bayesian models
More informationEvaluating the value of structural heath monitoring with longitudinal performance indicators and hazard functions using Bayesian dynamic predictions
Evaluating the value of structural heath monitoring with longitudinal performance indicators and hazard functions using Bayesian dynamic predictions C. Xing, R. Caspeele, L. Taerwe Ghent University, Department
More informationBayesian Inverse Problems
Bayesian Inverse Problems Jonas Latz Input/Output: www.latz.io Technical University of Munich Department of Mathematics, Chair for Numerical Analysis Email: jonas.latz@tum.de Garching, July 10 2018 Guest
More informationWho was Bayes? Bayesian Phylogenetics. What is Bayes Theorem?
Who was Bayes? Bayesian Phylogenetics Bret Larget Departments of Botany and of Statistics University of Wisconsin Madison October 6, 2011 The Reverand Thomas Bayes was born in London in 1702. He was the
More informationBayesian Phylogenetics:
Bayesian Phylogenetics: an introduction Marc A. Suchard msuchard@ucla.edu UCLA Who is this man? How sure are you? The one true tree? Methods we ve learned so far try to find a single tree that best describes
More informationBayesian Phylogenetics
Bayesian Phylogenetics Bret Larget Departments of Botany and of Statistics University of Wisconsin Madison October 6, 2011 Bayesian Phylogenetics 1 / 27 Who was Bayes? The Reverand Thomas Bayes was born
More informationApproximate Bayesian Computation: a simulation based approach to inference
Approximate Bayesian Computation: a simulation based approach to inference Richard Wilkinson Simon Tavaré 2 Department of Probability and Statistics University of Sheffield 2 Department of Applied Mathematics
More informationPrinciples of Bayesian Inference
Principles of Bayesian Inference Sudipto Banerjee University of Minnesota July 20th, 2008 1 Bayesian Principles Classical statistics: model parameters are fixed and unknown. A Bayesian thinks of parameters
More informationParameter Estimation Method Using Bayesian Statistics Considering Uncertainty of Information for RBDO
th World Congress on Structural and Multidisciplinary Optimization 7 th - 2 th, June 205, Sydney Australia Parameter Estimation Method Using Bayesian Statistics Considering Uncertainty of Information for
More informationThe Jackknife-Like Method for Assessing Uncertainty of Point Estimates for Bayesian Estimation in a Finite Gaussian Mixture Model
Thai Journal of Mathematics : 45 58 Special Issue: Annual Meeting in Mathematics 207 http://thaijmath.in.cmu.ac.th ISSN 686-0209 The Jackknife-Like Method for Assessing Uncertainty of Point Estimates for
More informationMonte Carlo Techniques for Regressing Random Variables. Dan Pemstein. Using V-Dem the Right Way
: Monte Carlo Techniques for Regressing Random Variables Workshop Goals (Re)familiarize you with Monte Carlo methods for estimating functions of random variables, integrating/marginalizing (Re)familiarize
More informationAnswers and expectations
Answers and expectations For a function f(x) and distribution P(x), the expectation of f with respect to P is The expectation is the average of f, when x is drawn from the probability distribution P E
More informationParameter Estimation. William H. Jefferys University of Texas at Austin Parameter Estimation 7/26/05 1
Parameter Estimation William H. Jefferys University of Texas at Austin bill@bayesrules.net Parameter Estimation 7/26/05 1 Elements of Inference Inference problems contain two indispensable elements: Data
More informationAdaptive Monte Carlo methods
Adaptive Monte Carlo methods Jean-Michel Marin Projet Select, INRIA Futurs, Université Paris-Sud joint with Randal Douc (École Polytechnique), Arnaud Guillin (Université de Marseille) and Christian Robert
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond Department of Biomedical Engineering and Computational Science Aalto University January 26, 2012 Contents 1 Batch and Recursive Estimation
More informationStatistical Methods in Particle Physics Lecture 1: Bayesian methods
Statistical Methods in Particle Physics Lecture 1: Bayesian methods SUSSP65 St Andrews 16 29 August 2009 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan
More informationTutorial on ABC Algorithms
Tutorial on ABC Algorithms Dr Chris Drovandi Queensland University of Technology, Australia c.drovandi@qut.edu.au July 3, 2014 Notation Model parameter θ with prior π(θ) Likelihood is f(ý θ) with observed
More information(5) Multi-parameter models - Gibbs sampling. ST440/540: Applied Bayesian Analysis
Summarizing a posterior Given the data and prior the posterior is determined Summarizing the posterior gives parameter estimates, intervals, and hypothesis tests Most of these computations are integrals
More informationDavid Giles Bayesian Econometrics
David Giles Bayesian Econometrics 1. General Background 2. Constructing Prior Distributions 3. Properties of Bayes Estimators and Tests 4. Bayesian Analysis of the Multiple Regression Model 5. Bayesian
More informationIntroduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak
Introduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak 1 Introduction. Random variables During the course we are interested in reasoning about considered phenomenon. In other words,
More informationTutorial on Probabilistic Programming with PyMC3
185.A83 Machine Learning for Health Informatics 2017S, VU, 2.0 h, 3.0 ECTS Tutorial 02-04.04.2017 Tutorial on Probabilistic Programming with PyMC3 florian.endel@tuwien.ac.at http://hci-kdd.org/machine-learning-for-health-informatics-course
More informationGeometric insight into the challenges of solving high-dimensional reliability problems
Probabilistic Engineering Mechanics 23 (2008) 208 28 www.elsevier.com/locate/probengmech Geometric insight into the challenges of solving high-dimensional reliability problems L.S. Katafygiotis, K.M. Zuev
More informationPart III. A Decision-Theoretic Approach and Bayesian testing
Part III A Decision-Theoretic Approach and Bayesian testing 1 Chapter 10 Bayesian Inference as a Decision Problem The decision-theoretic framework starts with the following situation. We would like to
More informationReminder of some Markov Chain properties:
Reminder of some Markov Chain properties: 1. a transition from one state to another occurs probabilistically 2. only state that matters is where you currently are (i.e. given present, future is independent
More informationMonte Carlo Dynamically Weighted Importance Sampling for Spatial Models with Intractable Normalizing Constants
Monte Carlo Dynamically Weighted Importance Sampling for Spatial Models with Intractable Normalizing Constants Faming Liang Texas A& University Sooyoung Cheon Korea University Spatial Model Introduction
More informationMarkov chain Monte Carlo
1 / 26 Markov chain Monte Carlo Timothy Hanson 1 and Alejandro Jara 2 1 Division of Biostatistics, University of Minnesota, USA 2 Department of Statistics, Universidad de Concepción, Chile IAP-Workshop
More informationBayesian search for other Earths
Bayesian search for other Earths Low-mass planets orbiting nearby M dwarfs Mikko Tuomi University of Hertfordshire, Centre for Astrophysics Research Email: mikko.tuomi@utu.fi Presentation, 19.4.2013 1
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationMONTE CARLO METHODS. Hedibert Freitas Lopes
MONTE CARLO METHODS Hedibert Freitas Lopes The University of Chicago Booth School of Business 5807 South Woodlawn Avenue, Chicago, IL 60637 http://faculty.chicagobooth.edu/hedibert.lopes hlopes@chicagobooth.edu
More informationMCMC notes by Mark Holder
MCMC notes by Mark Holder Bayesian inference Ultimately, we want to make probability statements about true values of parameters, given our data. For example P(α 0 < α 1 X). According to Bayes theorem:
More informationLECTURE 15 Markov chain Monte Carlo
LECTURE 15 Markov chain Monte Carlo There are many settings when posterior computation is a challenge in that one does not have a closed form expression for the posterior distribution. Markov chain Monte
More informationA Novel Vibration-Based Two-Stage Bayesian System Identification Method
A Novel Vibration-Based Two-Stage Bayesian System Identification Method *Feng-Liang Zhang 1) and Siu-Kui Au 2) 1) Research Institute of Structural Engineering and isaster Reduction, Tongji University,
More informationPrinciples of Bayesian Inference
Principles of Bayesian Inference Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department
More informationStatistical Inference: Maximum Likelihood and Bayesian Approaches
Statistical Inference: Maximum Likelihood and Bayesian Approaches Surya Tokdar From model to inference So a statistical analysis begins by setting up a model {f (x θ) : θ Θ} for data X. Next we observe
More informationBayesian Estimation of Input Output Tables for Russia
Bayesian Estimation of Input Output Tables for Russia Oleg Lugovoy (EDF, RANE) Andrey Polbin (RANE) Vladimir Potashnikov (RANE) WIOD Conference April 24, 2012 Groningen Outline Motivation Objectives Bayesian
More informationMarkov Chain Monte Carlo methods
Markov Chain Monte Carlo methods By Oleg Makhnin 1 Introduction a b c M = d e f g h i 0 f(x)dx 1.1 Motivation 1.1.1 Just here Supresses numbering 1.1.2 After this 1.2 Literature 2 Method 2.1 New math As
More informationIntroduction to Probabilistic Machine Learning
Introduction to Probabilistic Machine Learning Piyush Rai Dept. of CSE, IIT Kanpur (Mini-course 1) Nov 03, 2015 Piyush Rai (IIT Kanpur) Introduction to Probabilistic Machine Learning 1 Machine Learning
More informationTheory of Stochastic Processes 8. Markov chain Monte Carlo
Theory of Stochastic Processes 8. Markov chain Monte Carlo Tomonari Sei sei@mist.i.u-tokyo.ac.jp Department of Mathematical Informatics, University of Tokyo June 8, 2017 http://www.stat.t.u-tokyo.ac.jp/~sei/lec.html
More informationIntroduction to Bayes
Introduction to Bayes Alan Heavens September 3, 2018 ICIC Data Analysis Workshop Alan Heavens Introduction to Bayes September 3, 2018 1 / 35 Overview 1 Inverse Problems 2 The meaning of probability Probability
More informationVCMC: Variational Consensus Monte Carlo
VCMC: Variational Consensus Monte Carlo Maxim Rabinovich, Elaine Angelino, Michael I. Jordan Berkeley Vision and Learning Center September 22, 2015 probabilistic models! sky fog bridge water grass object
More informationForward Problems and their Inverse Solutions
Forward Problems and their Inverse Solutions Sarah Zedler 1,2 1 King Abdullah University of Science and Technology 2 University of Texas at Austin February, 2013 Outline 1 Forward Problem Example Weather
More informationBayesian Regression Linear and Logistic Regression
When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we
More informationSubset Simulation Method for Rare Event Estimation: An Introduction
Subset Simulation Method for Rare Event Estimation: An Introduction Synonyms Engineering reliability; Failure probability; Markov chain Monte Carlo; Monte Carlo simulation; Rare events; Subset Simulation
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing
More informationPrinciples of Bayesian Inference
Principles of Bayesian Inference Sudipto Banerjee and Andrew O. Finley 2 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department
More informationSC7/SM6 Bayes Methods HT18 Lecturer: Geoff Nicholls Lecture 2: Monte Carlo Methods Notes and Problem sheets are available at http://www.stats.ox.ac.uk/~nicholls/bayesmethods/ and via the MSc weblearn pages.
More informationMarkov Chain Monte Carlo
Markov Chain Monte Carlo Recall: To compute the expectation E ( h(y ) ) we use the approximation E(h(Y )) 1 n n h(y ) t=1 with Y (1),..., Y (n) h(y). Thus our aim is to sample Y (1),..., Y (n) from f(y).
More informationBayesian parameter estimation in predictive engineering
Bayesian parameter estimation in predictive engineering Damon McDougall Institute for Computational Engineering and Sciences, UT Austin 14th August 2014 1/27 Motivation Understand physical phenomena Observations
More informationNotes on pseudo-marginal methods, variational Bayes and ABC
Notes on pseudo-marginal methods, variational Bayes and ABC Christian Andersson Naesseth October 3, 2016 The Pseudo-Marginal Framework Assume we are interested in sampling from the posterior distribution
More informationSTAT J535: Introduction
David B. Hitchcock E-Mail: hitchcock@stat.sc.edu Spring 2012 Chapter 1: Introduction to Bayesian Data Analysis Bayesian statistical inference uses Bayes Law (Bayes Theorem) to combine prior information
More informationMarkov Chain Monte Carlo, Numerical Integration
Markov Chain Monte Carlo, Numerical Integration (See Statistics) Trevor Gallen Fall 2015 1 / 1 Agenda Numerical Integration: MCMC methods Estimating Markov Chains Estimating latent variables 2 / 1 Numerical
More informationAssessing Reliability Using Developmental and Operational Test Data
Assessing Reliability Using Developmental and Operational Test Data Martin Wayne, PhD, U.S. AMSAA Mohammad Modarres, PhD, University of Maryland, College Park Presented at the RAMS-04 Symposium Colorado
More informationAssessing Regime Uncertainty Through Reversible Jump McMC
Assessing Regime Uncertainty Through Reversible Jump McMC August 14, 2008 1 Introduction Background Research Question 2 The RJMcMC Method McMC RJMcMC Algorithm Dependent Proposals Independent Proposals
More informationIntroduction to Bayesian Methods. Introduction to Bayesian Methods p.1/??
to Bayesian Methods Introduction to Bayesian Methods p.1/?? We develop the Bayesian paradigm for parametric inference. To this end, suppose we conduct (or wish to design) a study, in which the parameter
More informationBayesian Graphical Models
Graphical Models and Inference, Lecture 16, Michaelmas Term 2009 December 4, 2009 Parameter θ, data X = x, likelihood L(θ x) p(x θ). Express knowledge about θ through prior distribution π on θ. Inference
More informationMachine Learning. Probabilistic KNN.
Machine Learning. Mark Girolami girolami@dcs.gla.ac.uk Department of Computing Science University of Glasgow June 21, 2007 p. 1/3 KNN is a remarkably simple algorithm with proven error-rates June 21, 2007
More informationeqr094: Hierarchical MCMC for Bayesian System Reliability
eqr094: Hierarchical MCMC for Bayesian System Reliability Alyson G. Wilson Statistical Sciences Group, Los Alamos National Laboratory P.O. Box 1663, MS F600 Los Alamos, NM 87545 USA Phone: 505-667-9167
More informationMCMC for big data. Geir Storvik. BigInsight lunch - May Geir Storvik MCMC for big data BigInsight lunch - May / 17
MCMC for big data Geir Storvik BigInsight lunch - May 2 2018 Geir Storvik MCMC for big data BigInsight lunch - May 2 2018 1 / 17 Outline Why ordinary MCMC is not scalable Different approaches for making
More informationThe University of Auckland Applied Mathematics Bayesian Methods for Inverse Problems : why and how Colin Fox Tiangang Cui, Mike O Sullivan (Auckland),
The University of Auckland Applied Mathematics Bayesian Methods for Inverse Problems : why and how Colin Fox Tiangang Cui, Mike O Sullivan (Auckland), Geoff Nicholls (Statistics, Oxford) fox@math.auckland.ac.nz
More informationPreliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com
1 School of Oriental and African Studies September 2015 Department of Economics Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com Gujarati D. Basic Econometrics, Appendix
More informationLecture 7 and 8: Markov Chain Monte Carlo
Lecture 7 and 8: Markov Chain Monte Carlo 4F13: Machine Learning Zoubin Ghahramani and Carl Edward Rasmussen Department of Engineering University of Cambridge http://mlg.eng.cam.ac.uk/teaching/4f13/ Ghahramani
More informationModel comparison. Christopher A. Sims Princeton University October 18, 2016
ECO 513 Fall 2008 Model comparison Christopher A. Sims Princeton University sims@princeton.edu October 18, 2016 c 2016 by Christopher A. Sims. This document may be reproduced for educational and research
More informationLarge Scale Bayesian Inference
Large Scale Bayesian I in Cosmology Jens Jasche Garching, 11 September 2012 Introduction Cosmography 3D density and velocity fields Power-spectra, bi-spectra Dark Energy, Dark Matter, Gravity Cosmological
More informationComputer Practical: Metropolis-Hastings-based MCMC
Computer Practical: Metropolis-Hastings-based MCMC Andrea Arnold and Franz Hamilton North Carolina State University July 30, 2016 A. Arnold / F. Hamilton (NCSU) MH-based MCMC July 30, 2016 1 / 19 Markov
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate
More informationMultifidelity Approaches to Approximate Bayesian Computation
Multifidelity Approaches to Approximate Bayesian Computation Thomas P. Prescott Wolfson Centre for Mathematical Biology University of Oxford Banff International Research Station 11th 16th November 2018
More informationProbabilistic Graphical Networks: Definitions and Basic Results
This document gives a cursory overview of Probabilistic Graphical Networks. The material has been gleaned from different sources. I make no claim to original authorship of this material. Bayesian Graphical
More informationEco517 Fall 2013 C. Sims MCMC. October 8, 2013
Eco517 Fall 2013 C. Sims MCMC October 8, 2013 c 2013 by Christopher A. Sims. This document may be reproduced for educational and research purposes, so long as the copies contain this notice and are retained
More informationBayesian Inference. Chapter 1. Introduction and basic concepts
Bayesian Inference Chapter 1. Introduction and basic concepts M. Concepción Ausín Department of Statistics Universidad Carlos III de Madrid Master in Business Administration and Quantitative Methods Master
More informationPhylogenetics: Bayesian Phylogenetic Analysis. COMP Spring 2015 Luay Nakhleh, Rice University
Phylogenetics: Bayesian Phylogenetic Analysis COMP 571 - Spring 2015 Luay Nakhleh, Rice University Bayes Rule P(X = x Y = y) = P(X = x, Y = y) P(Y = y) = P(X = x)p(y = y X = x) P x P(X = x 0 )P(Y = y X
More informationBayesian Methods and Uncertainty Quantification for Nonlinear Inverse Problems
Bayesian Methods and Uncertainty Quantification for Nonlinear Inverse Problems John Bardsley, University of Montana Collaborators: H. Haario, J. Kaipio, M. Laine, Y. Marzouk, A. Seppänen, A. Solonen, Z.
More informationIntroduction to Machine Learning
Introduction to Machine Learning Brown University CSCI 1950-F, Spring 2012 Prof. Erik Sudderth Lecture 25: Markov Chain Monte Carlo (MCMC) Course Review and Advanced Topics Many figures courtesy Kevin
More informationIntroduction to Machine Learning
Outline Introduction to Machine Learning Bayesian Classification Varun Chandola March 8, 017 1. {circular,large,light,smooth,thick}, malignant. {circular,large,light,irregular,thick}, malignant 3. {oval,large,dark,smooth,thin},
More informationIterative Markov Chain Monte Carlo Computation of Reference Priors and Minimax Risk
Iterative Markov Chain Monte Carlo Computation of Reference Priors and Minimax Risk John Lafferty School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 lafferty@cs.cmu.edu Abstract
More informationMultivariate Normal & Wishart
Multivariate Normal & Wishart Hoff Chapter 7 October 21, 2010 Reading Comprehesion Example Twenty-two children are given a reading comprehsion test before and after receiving a particular instruction method.
More informationan introduction to bayesian inference
with an application to network analysis http://jakehofman.com january 13, 2010 motivation would like models that: provide predictive and explanatory power are complex enough to describe observed phenomena
More informationMARKOV CHAIN MONTE CARLO
MARKOV CHAIN MONTE CARLO RYAN WANG Abstract. This paper gives a brief introduction to Markov Chain Monte Carlo methods, which offer a general framework for calculating difficult integrals. We start with
More informationST 740: Model Selection
ST 740: Model Selection Alyson Wilson Department of Statistics North Carolina State University November 25, 2013 A. Wilson (NCSU Statistics) Model Selection November 25, 2013 1 / 29 Formal Bayesian Model
More informationMCMC algorithms for Subset Simulation
To appear in Probabilistic Engineering Mechanics June 2015 MCMC algorithms for Subset Simulation Iason Papaioannou *, Wolfgang Betz, Kilian Zwirglmaier, Daniel Straub Engineering Risk Analysis Group, Technische
More informationBayesian Inference in Astronomy & Astrophysics A Short Course
Bayesian Inference in Astronomy & Astrophysics A Short Course Tom Loredo Dept. of Astronomy, Cornell University p.1/37 Five Lectures Overview of Bayesian Inference From Gaussians to Periodograms Learning
More informationMIT Spring 2016
MIT 18.655 Dr. Kempthorne Spring 2016 1 MIT 18.655 Outline 1 2 MIT 18.655 Decision Problem: Basic Components P = {P θ : θ Θ} : parametric model. Θ = {θ}: Parameter space. A{a} : Action space. L(θ, a) :
More information