BUGS Bayesian inference Using Gibbs Sampling
|
|
- Charlotte McBride
- 6 years ago
- Views:
Transcription
1 BUGS Bayesian inference Using Gibbs Sampling Glen DePalma Department of Statistics May 30, / 20
2 Bayesian Philosophy I [Pearl] turned Bayesian in 1971, as soon as I began reading Savage s monograph The Foundations of Statistical Inference [Savage, 1962]. The arguments were unassailable: i. It is plain silly to ignore what we know, ii. iii. It is natural and useful to cast what we know in the language of probabilities, and If our subjective probabilities are erroneous, their impact will get washed out in due time, as the number of observations increases. Note: He later would doubt the validity of (iii) 2 / 20
3 Bayesian Statistics Becoming more and more popular due to ease of simulation tools (R, SAS, BUGS,...) The main problem with frequentist statistics is there is a natural tendency to treat the p-value as if it was a Bayesian a-posterior probability that the null hypothesis is true (and hence 1-p is the probability that the alternative is true) or treating a frequentist confidence interval as a Bayesian credible interval (and hence assuming there is a 95% probability that the true value lies within a 95% confidence interval for the particular sample of data we have) I challenge you to find me in any published statistical analysis, outside of an introductory textbook, of a confidence interval given the correct interpretation. If you can find even one instance where the confidence interval is not interpreted as a credible interval, then I will eat your hat. - William Briggs 2012 The main objection to Bayesian statistics is the subjectivity in the prior (interestingly the subjectivity of the likelihood for frequentist approach is rarely mentioned) 3 / 20
4 Andrew Gelman Quotes The difference between significant and non-significant is not itself statistically significant I ve never in my professional life made a Type I error or a Type II error. But I ve made lots of errors. How can this be? A Type 1 error occurs only if the null hypothesis is true (typically if a certain parameter, or difference in parameters, equals zero). In the applications I ve worked on, in social science and public health, I ve never come across a null hypothesis that could actually be true, or a parameter that could actually be zero. A Type 2 error occurs only if I claim that the null hypothesis is true, and I would certainly not do that, given my statement above! 4 / 20
5 BUGS? Much of Bayesian analysis is done using Markov chain Monte Carlo (MCMC) to sample from the posterior. What does BUGS do? You define the model by specifying the relationships between the variables, BUGS handles the MCMC. Four (?) flavors: 1. WinBUGS (the original, development has ceased) 2. OpenBUGS (open source, still in development (?)) 3. JAGS (Just Another Gibbs Sampler) 4. STAN (New, Hamiltonian Monte Carlo) The syntax is very similar among the programs, examples in this presentation use JAGS: 5 / 20
6 Example 1 Out of 2,430 parts produced 219 were not shipped due to being defective. Our interest is in the probability of a defective part. Let us assume (unlikely) we know nothing apriori about the proportion of scrapped parts. y θ Binomial(2430, θ) θ Beta(1, 1) Exact Posterior is Beta( , ) 6 / 20
7 MCMC burnin =1000 nsim =5000 theta =.5 n=2430 y=219 a=1 b=1 t h e t a S a v e=rep (NA, nsim b u r n i n ) l l k =f u n c t i o n ( t h e t a ) r e t u r n ( dbinom ( y, n, t h e t a, l o g=true) ) l p r i o r=f u n c t i o n ( t h e t a ) r e t u r n ( dbeta ( t h e t a, a, b, l o g=true) ) f o r ( i i n 1 : nsim ){ # p r o p o s e new Theta thetanew=t h e t a+r u n i f ( 1,. 0 5,. 0 5 ) #Compute M H Acceptance i f ( thetanew > 0){ proposalllk= l l k ( thetanew )+l p r i o r ( thetanew ) l l k ( t h e t a ) l p r i o r ( t h e t a ) i f ( proposalllk>l o g ( r u n i f ( 1 ) ) ) t h e t a=thetanew i f ( i>b u r n i n ) thetasave [ i burnin ]= theta p l o t ( ( b u r n i n +1) : nsim, t h e tasave, t y p e= l, x l a b= I t e r a t i o n, y l a b= Theta ) summary ( t h e t a S a v e ) 7 / 20
8 BUGS Code model{ # Model y d b i n ( t h e t a, n ) # P r i o r t h e t a dbeta ( a1, b1 ) 8 / 20
9 Example 2 - One Sample Normal Likelihood: y i N (µ, 1/τ) Priors: µ N (0,.001) τ Gamma(.001,.001) Posterior: n i=1 τ 2π exp ( τ(y i µ) 2 /2 ).001 2π exp (.001µ 2 /2 ) τ e.001τ Γ(.001) We could calculate the exact posteriors of µ and τ but let s use BUGS! 9 / 20
10 BUGS Code model{ # Model f o r ( i i n 1 : n ){ y [ i ] dnorm (mu, tau ) # P r i o r mu dnorm ( 0, ) tau dgamma ( , ) # E x t r a sigma < 1/ s q r t ( tau ) prob < s t e p (mu 4.8) 10 / 20
11 Example 3 - Regression Model: Likelihood: L(β, σ 2 ) = (2πσ 2 ) n/2 exp y i N (X T i β, σ 2 ) ( ) 1 2σ 2 (Y X β)t (Y X β) Priors: We will use non-informative priors for β and σ 2 and let BUGS handle the details. 11 / 20
12 BUGS Code model { f o r ( i i n 1 : n ){ l o s s [ i ] dnorm (mu [ i ], tau ) mu[ i ] < beta [1]+ beta [ 2 ] a i r [ i ]+ beta [ 3 ] water [ i ]+ beta [ 4 ] a c i d [ i ] #p r i o r s f o r ( i i n 1 : 4 ) { beta [ i ] dnorm ( 0, ) tau dgamma ( , ) sigma < 1/ s q r t ( tau ) #t e s t b e t a 4 beta4prob < step ( beta [4] 0) #P r e d i c t i o n s mu1 < beta [1]+ beta [ 2 ] beta [ 3 ] beta [ 4 ] pred1 dnorm (mu1, tau ) 12 / 20
13 Example 4 - One-way ANOVA Model: Lamb data: y ij = µ i + ɛ ij µ i N (0,.001) ɛ ij N (0, 1/τ) y ij = µ i + ɛ ij µ i N (β i, λ i ) β i N (5,.01) λ i Gamma(10, 1) ɛ ij N (0, 1/τ) i = / 20
14 BUGS Code model { f o r ( i i n 1 : n ){ y [ i ] dnorm (mu [ group [ i ] ], tau ) f o r ( i i n 1 : 5 ) { mu[ i ] dnorm ( beta [ i ], lambda [ i ] ) beta [ i ] dnorm ( 5,. 0 1 ) lambda [ i ] dgamma (1 0,1) tau dgamma ( , ) sigma < 1/ s q r t ( tau ) #########d i f f e r e n c e i n mus############ prob1< step (mu[2] mu [ 1 ] ) prob2< step (mu[2] mu [ 3 ] ) prob3< step (mu[2] mu [ 4 ] ) prob4< step (mu[2] mu [ 5 ] ) 14 / 20
15 Example 5 - Agricultural Experiment - 2 Way ANOVA Fertilizer was applied to four corn types and three different fertilizers. Let us be real Bayesians and use prior knowledge we have gathered from talking to the farmers and industry experts. Cell Means ANOVA Model: Y ij = θ i + ɛ ij where θ i N ( i=1 ) X i, σ2 4 i = j = ɛ ij N (0, σ 2 ) The variance of each observation is known to be around 20 (σ 2 = 20), therefore the variance of the thetas equals 5. Theta is expected to be around 125 with a possible range of Given this information, we use the following prior for theta: θ i N (µ i, 5) µ i N (125, 225) 15 / 20
16 BUGS Code ##c e l l means anova model model { f o r ( i i n 1 : 1 2 ) { f o r ( j i n 1 : 4 ) { #x [ i, j ] i s t h e t a x [ i, j ] dnorm ( mu [ i ], 1/ 20) # p r i o r on t h e t a mu[ i ] dnorm (125,1 / 225) # Extra! ### Compute Main E f f e c t s o f F e r t i l i z e r and Corn c o r n [ 1 ] < mean (mu [ 1 : 3 ] ) c o r n [ 2 ] < mean (mu [ 4 : 6 ] ) c o r n [ 3 ] < mean (mu [ 7 : 9 ] ) c o r n [ 4 ] < mean (mu [ 1 0 : 1 2 ] ) f e r t [ 1 ] < (mu[1]+mu[4]+mu[7]+mu [ 1 0 ] ) /4 f e r t [ 2 ] < (mu[2]+mu[5]+mu[8]+mu [ 1 1 ] ) /4 f e r t [ 3 ] < (mu[3]+mu[6]+mu[9]+mu [ 1 2 ] ) /4 ### Prob main e f f e c t s b e s t f o r ( i i n 1 : 4 ) { probcorn [ i ] < e q u a l s ( c o r n [ i ], max( c o r n [ 1 : 4 ] ) ) f o r ( i i n 1 : 3 ) { p r o b F e r t [ i ] < e q u a l s ( f e r t [ i ], max( f e r t [ 1 : 3 ] ) ) ### Prob c e l l means b e s t f o r ( i i n 1 : 1 2 ) { probtrt [ i ] < e q u a l s (mu[ i ], max(mu [ 1 : 1 2 ] ) ) ### prob [ 4, 3 ] > [ 2, 1 ] prob1 < s t e p (mu[12] mu [ 4 ] ) 16 / 20
17 Reporting a Bayesian Analysis 1. Motivate the use of Bayesian analysis Richer and more informative, no reliance on p-values 2. Clearly describe the model and its parameters The posterior distribution is a distribution over the parameters 3. Clearly describe and justify the prior 4. Mention MCMC details 5. Interpret the posterior Report summary statistics of the parameters that are theoretically meaningful 6. Robustness of the posterior for different priors Conduct the analysis with different priors as a sensitivity test 7. Posterior Predictive Check Generate data from the posterior, do they match the actual data? 17 / 20
18 Convergence How do we know if we have converged to the correct posterior? Quick answer: we can t. Many fancy diagnostic theorems have been proposed, but none of them prove the property you really want a diagnostic to have. These theorems say that if the chain converges, then the diagnostic will probably say the chain has converged, but they do not say that if the chain pseudo-converges, then the diagnostic will probably say the chain did not converge. How do we know if we have converged to the correct posterior? Theorems that do claim pseudo-convergence have unverifiable conditions that make them useless. 18 / 20
19 Convergence II Your humble author has a dictum that the least one can do is make an overnight run. What better way for your computer to spend its time? In many problems that are not too complicated, this is millions or billions of iterations. If you do not make runs like that, you are not simply serious about MCMC. - Charles J. Geyer 19 / 20
20 Summary Frequentist statistics are a useful tool, but the future of effective data analysis is moving closer to Bayesian analyses BUGS makes it easy to do Bayesian Analysis Can handle a wide variety of models Presented only basic (but common!) examples Roughly 100 examples: However BUGS can t do everything (but it is getting close..): Models that require a specialized likelihood (not Normal, Binomial,...) Reversible Jump Markov Chain Monte Carlo (RJMCMC) Still very important to learn how to fully program a Bayesian analysis Frequentist methods still have a place / 20
Bayesian Networks in Educational Assessment
Bayesian Networks in Educational Assessment Estimating Parameters with MCMC Bayesian Inference: Expanding Our Context Roy Levy Arizona State University Roy.Levy@asu.edu 2017 Roy Levy MCMC 1 MCMC 2 Posterior
More informationAdvanced Statistical Modelling
Markov chain Monte Carlo (MCMC) Methods and Their Applications in Bayesian Statistics School of Technology and Business Studies/Statistics Dalarna University Borlänge, Sweden. Feb. 05, 2014. Outlines 1
More informationPrinciples of Bayesian Inference
Principles of Bayesian Inference Sudipto Banerjee University of Minnesota July 20th, 2008 1 Bayesian Principles Classical statistics: model parameters are fixed and unknown. A Bayesian thinks of parameters
More informationBayesian Graphical Models
Graphical Models and Inference, Lecture 16, Michaelmas Term 2009 December 4, 2009 Parameter θ, data X = x, likelihood L(θ x) p(x θ). Express knowledge about θ through prior distribution π on θ. Inference
More informationBayesian philosophy Bayesian computation Bayesian software. Bayesian Statistics. Petter Mostad. Chalmers. April 6, 2017
Chalmers April 6, 2017 Bayesian philosophy Bayesian philosophy Bayesian statistics versus classical statistics: War or co-existence? Classical statistics: Models have variables and parameters; these are
More information36-463/663Multilevel and Hierarchical Models
36-463/663Multilevel and Hierarchical Models From Bayes to MCMC to MLMs Brian Junker 132E Baker Hall brian@stat.cmu.edu 1 Outline Bayesian Statistics and MCMC Distribution of Skill Mastery in a Population
More information(5) Multi-parameter models - Gibbs sampling. ST440/540: Applied Bayesian Analysis
Summarizing a posterior Given the data and prior the posterior is determined Summarizing the posterior gives parameter estimates, intervals, and hypothesis tests Most of these computations are integrals
More informationPrinciples of Bayesian Inference
Principles of Bayesian Inference Sudipto Banerjee and Andrew O. Finley 2 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department
More informationBayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
More informationBayesian Inference for Regression Parameters
Bayesian Inference for Regression Parameters 1 Bayesian inference for simple linear regression parameters follows the usual pattern for all Bayesian analyses: 1. Form a prior distribution over all unknown
More informationMCMC for Cut Models or Chasing a Moving Target with MCMC
MCMC for Cut Models or Chasing a Moving Target with MCMC Martyn Plummer International Agency for Research on Cancer MCMSki Chamonix, 6 Jan 2014 Cut models What do we want to do? 1. Generate some random
More informationProbabilistic Machine Learning
Probabilistic Machine Learning Bayesian Nets, MCMC, and more Marek Petrik 4/18/2017 Based on: P. Murphy, K. (2012). Machine Learning: A Probabilistic Perspective. Chapter 10. Conditional Independence Independent
More informationPIER HLM Course July 30, 2011 Howard Seltman. Discussion Guide for Bayes and BUGS
PIER HLM Course July 30, 2011 Howard Seltman Discussion Guide for Bayes and BUGS 1. Classical Statistics is based on parameters as fixed unknown values. a. The standard approach is to try to discover,
More informationRobust Bayesian Regression
Readings: Hoff Chapter 9, West JRSSB 1984, Fúquene, Pérez & Pericchi 2015 Duke University November 17, 2016 Body Fat Data: Intervals w/ All Data Response % Body Fat and Predictor Waist Circumference 95%
More informationMH I. Metropolis-Hastings (MH) algorithm is the most popular method of getting dependent samples from a probability distribution
MH I Metropolis-Hastings (MH) algorithm is the most popular method of getting dependent samples from a probability distribution a lot of Bayesian mehods rely on the use of MH algorithm and it s famous
More informationWeakness of Beta priors (or conjugate priors in general) They can only represent a limited range of prior beliefs. For example... There are no bimodal beta distributions (except when the modes are at 0
More informationIntroduction to Bayesian Statistics and Markov Chain Monte Carlo Estimation. EPSY 905: Multivariate Analysis Spring 2016 Lecture #10: April 6, 2016
Introduction to Bayesian Statistics and Markov Chain Monte Carlo Estimation EPSY 905: Multivariate Analysis Spring 2016 Lecture #10: April 6, 2016 EPSY 905: Intro to Bayesian and MCMC Today s Class An
More informationHypothesis Testing and Estimation under a Bayesian Approach
Hypothesis Testing and Estimation under a Bayesian Approach L.R. Pericchi* and M.E. Pérez 1 1 Department of Mathematics Universidad de Puerto Rico, Río Piedras Campus *Co-Leader of Biostatistics, Epidemiology
More informationPrinciples of Bayesian Inference
Principles of Bayesian Inference Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department
More informationHierarchical Linear Models
Hierarchical Linear Models Statistics 220 Spring 2005 Copyright c 2005 by Mark E. Irwin The linear regression model Hierarchical Linear Models y N(Xβ, Σ y ) β σ 2 p(β σ 2 ) σ 2 p(σ 2 ) can be extended
More informationLecture 13 Fundamentals of Bayesian Inference
Lecture 13 Fundamentals of Bayesian Inference Dennis Sun Stats 253 August 11, 2014 Outline of Lecture 1 Bayesian Models 2 Modeling Correlations Using Bayes 3 The Universal Algorithm 4 BUGS 5 Wrapping Up
More informationMetric Predicted Variable on One Group
Metric Predicted Variable on One Group Tim Frasier Copyright Tim Frasier This work is licensed under the Creative Commons Attribution 4.0 International license. Click here for more information. Prior Homework
More informationBayes: All uncertainty is described using probability.
Bayes: All uncertainty is described using probability. Let w be the data and θ be any unknown quantities. Likelihood. The probability model π(w θ) has θ fixed and w varying. The likelihood L(θ; w) is π(w
More informationPrinciples of Bayesian Inference
Principles of Bayesian Inference Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department
More informationMarkov Chain Monte Carlo (MCMC) and Model Evaluation. August 15, 2017
Markov Chain Monte Carlo (MCMC) and Model Evaluation August 15, 2017 Frequentist Linking Frequentist and Bayesian Statistics How can we estimate model parameters and what does it imply? Want to find the
More informationLinear Regression. Data Model. β, σ 2. Process Model. ,V β. ,s 2. s 1. Parameter Model
Regression: Part II Linear Regression y~n X, 2 X Y Data Model β, σ 2 Process Model Β 0,V β s 1,s 2 Parameter Model Assumptions of Linear Model Homoskedasticity No error in X variables Error in Y variables
More informationBayesian modelling. Hans-Peter Helfrich. University of Bonn. Theodor-Brinkmann-Graduate School
Bayesian modelling Hans-Peter Helfrich University of Bonn Theodor-Brinkmann-Graduate School H.-P. Helfrich (University of Bonn) Bayesian modelling Brinkmann School 1 / 22 Overview 1 Bayesian modelling
More informationBayesian Computation
Bayesian Computation CAS Centennial Celebration and Annual Meeting New York, NY November 10, 2014 Brian M. Hartman, PhD ASA Assistant Professor of Actuarial Science University of Connecticut CAS Antitrust
More informationThe Bayesian Choice. Christian P. Robert. From Decision-Theoretic Foundations to Computational Implementation. Second Edition.
Christian P. Robert The Bayesian Choice From Decision-Theoretic Foundations to Computational Implementation Second Edition With 23 Illustrations ^Springer" Contents Preface to the Second Edition Preface
More informationTutorial on Probabilistic Programming with PyMC3
185.A83 Machine Learning for Health Informatics 2017S, VU, 2.0 h, 3.0 ECTS Tutorial 02-04.04.2017 Tutorial on Probabilistic Programming with PyMC3 florian.endel@tuwien.ac.at http://hci-kdd.org/machine-learning-for-health-informatics-course
More informationBayesian Inference in GLMs. Frequentists typically base inferences on MLEs, asymptotic confidence
Bayesian Inference in GLMs Frequentists typically base inferences on MLEs, asymptotic confidence limits, and log-likelihood ratio tests Bayesians base inferences on the posterior distribution of the unknowns
More informationBayesian Modeling of Accelerated Life Tests with Random Effects
Bayesian Modeling of Accelerated Life Tests with Random Effects Ramón V. León Avery J. Ashby Jayanth Thyagarajan Joint Statistical Meeting August, 00 Toronto, Canada Abstract We show how to use Bayesian
More informationMetric Predicted Variable on Two Groups
Metric Predicted Variable on Two Groups Tim Frasier Copyright Tim Frasier This work is licensed under the Creative Commons Attribution 4.0 International license. Click here for more information. Goals
More informationConfidence Intervals. CAS Antitrust Notice. Bayesian Computation. General differences between Bayesian and Frequntist statistics 10/16/2014
CAS Antitrust Notice Bayesian Computation CAS Centennial Celebration and Annual Meeting New York, NY November 10, 2014 Brian M. Hartman, PhD ASA Assistant Professor of Actuarial Science University of Connecticut
More informationMarkov Chain Monte Carlo
Markov Chain Monte Carlo Recall: To compute the expectation E ( h(y ) ) we use the approximation E(h(Y )) 1 n n h(y ) t=1 with Y (1),..., Y (n) h(y). Thus our aim is to sample Y (1),..., Y (n) from f(y).
More informationBayesian Inference and Decision Theory
Bayesian Inference and Decision Theory Instructor: Kathryn Blackmond Laskey Room 2214 ENGR (703) 993-1644 Office Hours: Tuesday and Thursday 4:30-5:30 PM, or by appointment Spring 2018 Unit 6: Gibbs Sampling
More informationWeakness of Beta priors (or conjugate priors in general) They can only represent a limited range of prior beliefs. For example... There are no bimodal beta distributions (except when the modes are at 0
More informationWhy Bayesian approaches? The average height of a rare plant
Why Bayesian approaches? The average height of a rare plant Estimation and comparison of averages is an important step in many ecological analyses and demographic models. In this demonstration you will
More informationPreface 5. Introduction 6. A Hands-on Example 8. Regression Models 12. Model Checking & Diagnostics 20. Contents. Bayesian Probability 7
M I C H A E L C L A R K C E N T E R F O R S TAT I S T I C A L C O N S U LTAT I O N A N D R E S E A R C H U N I V E R S I T Y O F M I C H I G A N B AY E S I A N B A S I C S A C O N C E P T U A L I N T R
More informationStat 451 Lecture Notes Markov Chain Monte Carlo. Ryan Martin UIC
Stat 451 Lecture Notes 07 12 Markov Chain Monte Carlo Ryan Martin UIC www.math.uic.edu/~rgmartin 1 Based on Chapters 8 9 in Givens & Hoeting, Chapters 25 27 in Lange 2 Updated: April 4, 2016 1 / 42 Outline
More informationMIT /30 Gelman, Carpenter, Hoffman, Guo, Goodrich, Lee,... Stan for Bayesian data analysis
MIT 1985 1/30 Stan: a program for Bayesian data analysis with complex models Andrew Gelman, Bob Carpenter, and Matt Hoffman, Jiqiang Guo, Ben Goodrich, and Daniel Lee Department of Statistics, Columbia
More informationStat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
More informationWinBUGS : part 2. Bruno Boulanger Jonathan Jaeger Astrid Jullion Philippe Lambert. Gabriele, living with rheumatoid arthritis
WinBUGS : part 2 Bruno Boulanger Jonathan Jaeger Astrid Jullion Philippe Lambert Gabriele, living with rheumatoid arthritis Agenda 2! Hierarchical model: linear regression example! R2WinBUGS Linear Regression
More informationPARAMETER ESTIMATION: BAYESIAN APPROACH. These notes summarize the lectures on Bayesian parameter estimation.
PARAMETER ESTIMATION: BAYESIAN APPROACH. These notes summarize the lectures on Bayesian parameter estimation.. Beta Distribution We ll start by learning about the Beta distribution, since we end up using
More informationBayesian Phylogenetics:
Bayesian Phylogenetics: an introduction Marc A. Suchard msuchard@ucla.edu UCLA Who is this man? How sure are you? The one true tree? Methods we ve learned so far try to find a single tree that best describes
More informationBiol 206/306 Advanced Biostatistics Lab 12 Bayesian Inference
Biol 206/306 Advanced Biostatistics Lab 12 Bayesian Inference By Philip J. Bergmann 0. Laboratory Objectives 1. Learn what Bayes Theorem and Bayesian Inference are 2. Reinforce the properties of Bayesian
More informationDAG models and Markov Chain Monte Carlo methods a short overview
DAG models and Markov Chain Monte Carlo methods a short overview Søren Højsgaard Institute of Genetics and Biotechnology University of Aarhus August 18, 2008 Printed: August 18, 2008 File: DAGMC-Lecture.tex
More informationMonte Carlo Inference Methods
Monte Carlo Inference Methods Iain Murray University of Edinburgh http://iainmurray.net Monte Carlo and Insomnia Enrico Fermi (1901 1954) took great delight in astonishing his colleagues with his remarkably
More information36-463/663: Hierarchical Linear Models
36-463/663: Hierarchical Linear Models Taste of MCMC / Bayes for 3 or more levels Brian Junker 132E Baker Hall brian@stat.cmu.edu 1 Outline Practical Bayes Mastery Learning Example A brief taste of JAGS
More informationBayesian hypothesis testing for the distribution of insurance claim counts using the Gibbs sampler
Journal of Computational Methods in Sciences and Engineering 5 (2005) 201 214 201 IOS Press Bayesian hypothesis testing for the distribution of insurance claim counts using the Gibbs sampler Athanassios
More information0.1 normal.bayes: Bayesian Normal Linear Regression
0.1 normal.bayes: Bayesian Normal Linear Regression Use Bayesian regression to specify a continuous dependent variable as a linear function of specified explanatory variables. The model is implemented
More informationBayesian Linear Models
Bayesian Linear Models Sudipto Banerjee 1 and Andrew O. Finley 2 1 Department of Forestry & Department of Geography, Michigan State University, Lansing Michigan, U.S.A. 2 Biostatistics, School of Public
More informationComputer intensive statistical methods
Lecture 13 MCMC, Hybrid chains October 13, 2015 Jonas Wallin jonwal@chalmers.se Chalmers, Gothenburg university MH algorithm, Chap:6.3 The metropolis hastings requires three objects, the distribution of
More informationProbabilities for climate projections
Probabilities for climate projections Claudia Tebaldi, Reinhard Furrer Linda Mearns, Doug Nychka National Center for Atmospheric Research Richard Smith - UNC-Chapel Hill Steve Sain - CU-Denver Statistical
More informationDIC: Deviance Information Criterion
(((( Welcome Page Latest News DIC: Deviance Information Criterion Contact us/bugs list WinBUGS New WinBUGS examples FAQs DIC GeoBUGS DIC (Deviance Information Criterion) is a Bayesian method for model
More informationBayesian Linear Regression
Bayesian Linear Regression Sudipto Banerjee 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. September 15, 2010 1 Linear regression models: a Bayesian perspective
More informationStatistics in Environmental Research (BUC Workshop Series) II Problem sheet - WinBUGS - SOLUTIONS
Statistics in Environmental Research (BUC Workshop Series) II Problem sheet - WinBUGS - SOLUTIONS 1. (a) The posterior mean estimate of α is 14.27, and the posterior mean for the standard deviation of
More informationINTRODUCTION TO BAYESIAN STATISTICS
INTRODUCTION TO BAYESIAN STATISTICS Sarat C. Dass Department of Statistics & Probability Department of Computer Science & Engineering Michigan State University TOPICS The Bayesian Framework Different Types
More informationDown by the Bayes, where the Watermelons Grow
Down by the Bayes, where the Watermelons Grow A Bayesian example using SAS SUAVe: Victoria SAS User Group Meeting November 21, 2017 Peter K. Ott, M.Sc., P.Stat. Strategic Analysis 1 Outline 1. Motivating
More informationMetropolis-Hastings Algorithm
Strength of the Gibbs sampler Metropolis-Hastings Algorithm Easy algorithm to think about. Exploits the factorization properties of the joint probability distribution. No difficult choices to be made to
More informationST 740: Markov Chain Monte Carlo
ST 740: Markov Chain Monte Carlo Alyson Wilson Department of Statistics North Carolina State University October 14, 2012 A. Wilson (NCSU Stsatistics) MCMC October 14, 2012 1 / 20 Convergence Diagnostics:
More informationBayesian Linear Models
Bayesian Linear Models Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department
More informationDeciding, Estimating, Computing, Checking
Deciding, Estimating, Computing, Checking How are Bayesian posteriors used, computed and validated? Fundamentalist Bayes: The posterior is ALL knowledge you have about the state Use in decision making:
More informationDeciding, Estimating, Computing, Checking. How are Bayesian posteriors used, computed and validated?
Deciding, Estimating, Computing, Checking How are Bayesian posteriors used, computed and validated? Fundamentalist Bayes: The posterior is ALL knowledge you have about the state Use in decision making:
More informationA Parameter Expansion Approach to Bayesian SEM Estimation
A Parameter Expansion Approach to Bayesian SEM Estimation Ed Merkle and Yves Rosseel Utrecht University 24 June 2016 Yves Rosseel A Parameter Expansion Approach to Bayesian SEM Estimation 1 / 51 overview
More informationBayesian Inference and MCMC
Bayesian Inference and MCMC Aryan Arbabi Partly based on MCMC slides from CSC412 Fall 2018 1 / 18 Bayesian Inference - Motivation Consider we have a data set D = {x 1,..., x n }. E.g each x i can be the
More informationMarkov Chain Monte Carlo methods
Markov Chain Monte Carlo methods By Oleg Makhnin 1 Introduction a b c M = d e f g h i 0 f(x)dx 1.1 Motivation 1.1.1 Just here Supresses numbering 1.1.2 After this 1.2 Literature 2 Method 2.1 New math As
More informationBayesian Statistics: An Introduction
: An Introduction Tim Frasier Copyright Tim Frasier This work is licensed under the Creative Commons Attribution 4.0 International license. Click here for more information. Outline 1. Bayesian statistics,
More informationParameter Estimation. William H. Jefferys University of Texas at Austin Parameter Estimation 7/26/05 1
Parameter Estimation William H. Jefferys University of Texas at Austin bill@bayesrules.net Parameter Estimation 7/26/05 1 Elements of Inference Inference problems contain two indispensable elements: Data
More informationFast Likelihood-Free Inference via Bayesian Optimization
Fast Likelihood-Free Inference via Bayesian Optimization Michael Gutmann https://sites.google.com/site/michaelgutmann University of Helsinki Aalto University Helsinki Institute for Information Technology
More information0.1 factor.bayes: Bayesian Factor Analysis
0.1 factor.bayes: Bayesian Factor Analysis Given some unobserved explanatory variables and observed dependent variables, the Normal theory factor analysis model estimates the latent factors. The model
More informationIntroduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Markov Chain Monte Carlo Methods Barnabás Póczos & Aarti Singh Contents Markov Chain Monte Carlo Methods Goal & Motivation Sampling Rejection Importance Markov
More informationFAV i R This paper is produced mechanically as part of FAViR. See for more information.
Bayesian Claim Severity Part 2 Mixed Exponentials with Trend, Censoring, and Truncation By Benedict Escoto FAV i R This paper is produced mechanically as part of FAViR. See http://www.favir.net for more
More informationModel comparison: Deviance-based approaches
Model comparison: Deviance-based approaches Patrick Breheny February 19 Patrick Breheny BST 701: Bayesian Modeling in Biostatistics 1/23 Model comparison Thus far, we have looked at residuals in a fairly
More informationA Comparison of Two MCMC Algorithms for Hierarchical Mixture Models
A Comparison of Two MCMC Algorithms for Hierarchical Mixture Models Russell Almond Florida State University College of Education Educational Psychology and Learning Systems ralmond@fsu.edu BMAW 2014 1
More informationIntroduction to Markov Chain Monte Carlo
Introduction to Markov Chain Monte Carlo Jim Albert March 18, 2018 A Selected Data Problem Here is an interesting problem with selected data. Suppose you are measuring the speeds of cars driving on an
More informationAdvanced Statistical Methods. Lecture 6
Advanced Statistical Methods Lecture 6 Convergence distribution of M.-H. MCMC We denote the PDF estimated by the MCMC as. It has the property Convergence distribution After some time, the distribution
More informationLecture 6. Prior distributions
Summary Lecture 6. Prior distributions 1. Introduction 2. Bivariate conjugate: normal 3. Non-informative / reference priors Jeffreys priors Location parameters Proportions Counts and rates Scale parameters
More informationBayesian Statistical Methods. Jeff Gill. Department of Political Science, University of Florida
Bayesian Statistical Methods Jeff Gill Department of Political Science, University of Florida 234 Anderson Hall, PO Box 117325, Gainesville, FL 32611-7325 Voice: 352-392-0262x272, Fax: 352-392-8127, Email:
More informationMarkov Chain Monte Carlo
Markov Chain Monte Carlo Jamie Monogan University of Georgia Spring 2013 For more information, including R programs, properties of Markov chains, and Metropolis-Hastings, please see: http://monogan.myweb.uga.edu/teaching/statcomp/mcmc.pdf
More informationCSC321 Lecture 18: Learning Probabilistic Models
CSC321 Lecture 18: Learning Probabilistic Models Roger Grosse Roger Grosse CSC321 Lecture 18: Learning Probabilistic Models 1 / 25 Overview So far in this course: mainly supervised learning Language modeling
More informationLatent Variable Models for Binary Data. Suppose that for a given vector of explanatory variables x, the latent
Latent Variable Models for Binary Data Suppose that for a given vector of explanatory variables x, the latent variable, U, has a continuous cumulative distribution function F (u; x) and that the binary
More informationComputational statistics
Computational statistics Markov Chain Monte Carlo methods Thierry Denœux March 2017 Thierry Denœux Computational statistics March 2017 1 / 71 Contents of this chapter When a target density f can be evaluated
More informationReducing The Computational Cost of Bayesian Indoor Positioning Systems
Reducing The Computational Cost of Bayesian Indoor Positioning Systems Konstantinos Kleisouris, Richard P. Martin Computer Science Department Rutgers University WINLAB Research Review May 15 th, 2006 Motivation
More informationAn Introduction to Reversible Jump MCMC for Bayesian Networks, with Application
An Introduction to Reversible Jump MCMC for Bayesian Networks, with Application, CleverSet, Inc. STARMAP/DAMARS Conference Page 1 The research described in this presentation has been funded by the U.S.
More informationBayesian Methods in Multilevel Regression
Bayesian Methods in Multilevel Regression Joop Hox MuLOG, 15 september 2000 mcmc What is Statistics?! Statistics is about uncertainty To err is human, to forgive divine, but to include errors in your design
More informationMCMC algorithms for fitting Bayesian models
MCMC algorithms for fitting Bayesian models p. 1/1 MCMC algorithms for fitting Bayesian models Sudipto Banerjee sudiptob@biostat.umn.edu University of Minnesota MCMC algorithms for fitting Bayesian models
More informationBayesian Regression Linear and Logistic Regression
When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we
More informationProbabilistic programming and Stan. mc-stan.org
Probabilistic programming and Stan mc-stan.org Outline What is probabilistic programming Stan now Stan in the future A bit about other software Probabilistic programming Probabilistic programming framework
More information(4) One-parameter models - Beta/binomial. ST440/550: Applied Bayesian Statistics
Estimating a proportion using the beta/binomial model A fundamental task in statistics is to estimate a proportion using a series of trials: What is the success probability of a new cancer treatment? What
More informationProbabilistic Graphical Models Lecture 17: Markov chain Monte Carlo
Probabilistic Graphical Models Lecture 17: Markov chain Monte Carlo Andrew Gordon Wilson www.cs.cmu.edu/~andrewgw Carnegie Mellon University March 18, 2015 1 / 45 Resources and Attribution Image credits,
More informationMarkov chain Monte Carlo
1 / 26 Markov chain Monte Carlo Timothy Hanson 1 and Alejandro Jara 2 1 Division of Biostatistics, University of Minnesota, USA 2 Department of Statistics, Universidad de Concepción, Chile IAP-Workshop
More informationMCMC Methods: Gibbs and Metropolis
MCMC Methods: Gibbs and Metropolis Patrick Breheny February 28 Patrick Breheny BST 701: Bayesian Modeling in Biostatistics 1/30 Introduction As we have seen, the ability to sample from the posterior distribution
More informationSTAT 425: Introduction to Bayesian Analysis
STAT 425: Introduction to Bayesian Analysis Marina Vannucci Rice University, USA Fall 2017 Marina Vannucci (Rice University, USA) Bayesian Analysis (Part 2) Fall 2017 1 / 19 Part 2: Markov chain Monte
More informationBiol 206/306 Advanced Biostatistics Lab 12 Bayesian Inference Fall 2016
Biol 206/306 Advanced Biostatistics Lab 12 Bayesian Inference Fall 2016 By Philip J. Bergmann 0. Laboratory Objectives 1. Learn what Bayes Theorem and Bayesian Inference are 2. Reinforce the properties
More informationMotivation Scale Mixutres of Normals Finite Gaussian Mixtures Skew-Normal Models. Mixture Models. Econ 690. Purdue University
Econ 690 Purdue University In virtually all of the previous lectures, our models have made use of normality assumptions. From a computational point of view, the reason for this assumption is clear: combined
More informationPart 7: Hierarchical Modeling
Part 7: Hierarchical Modeling!1 Nested data It is common for data to be nested: i.e., observations on subjects are organized by a hierarchy Such data are often called hierarchical or multilevel For example,
More informationStatistical Computing with R
Statistical Computing with R Eric Slud, Math. Dept., UMCP October 21, 2009 Overview of Course This course was originally developed jointly with Benjamin Kedem and Paul Smith. It consists of modules as
More informationRegression: Main Ideas Setting: Quantitative outcome with a quantitative explanatory variable. Example, cont.
TCELL 9/4/205 36-309/749 Experimental Design for Behavioral and Social Sciences Simple Regression Example Male black wheatear birds carry stones to the nest as a form of sexual display. Soler et al. wanted
More informationLecture 5. G. Cowan Lectures on Statistical Data Analysis Lecture 5 page 1
Lecture 5 1 Probability (90 min.) Definition, Bayes theorem, probability densities and their properties, catalogue of pdfs, Monte Carlo 2 Statistical tests (90 min.) general concepts, test statistics,
More information