Statistics 360/601 Modern Bayesian Theory
|
|
|
- Herbert Goodwin
- 5 years ago
- Views:
Transcription
1 Statistics 360/601 Modern Bayesian Theory Alexander Volfovsky Lecture 8 - Sept 25, 2018
2 Monte Carlo 1
3 2 Monte Carlo approximation Want to compute Z E[ y] = p( y)d without worrying about actual integration... I Let be a parameter of interest I Let y 1,...,y n be numerical values of a sample from a distribution p(y 1,...,y n ) I Let 1,..., S p( y 1,...,y n ) be iid samples from the posterior. I We estimate our posterior quantity of interest as 1/S P g( i )
4 3 Consistent parameters?? I Let the data come from Y Geometric(p). I Recall that a geometric variable has pdf Pr(Y = k) =(1 p) k p and it captures a success on the k +1trialafterk failures. I The mean of a geometric is (1 p)/p. I Let the model we think the data comes from be Poisson( ). I Let the prior be a Gamma(, ). I We know the posterior is Gamma( + P y i, + n).
5 4 Consistent parameters?? mean= mean= x, n= x, n= 100 mean= mean= x, n= x, n= 20000
6 5 Consistent parameters?? mean= mean= x, n= x, n= 100 mean= mean= x, n= x, n= 20000
7 Posterior Predictive Checks 6
8 Lets look at the data (from the book). Pr(Y i = y i ) empirical distribution predictive distribution number of children Data: twice as many women with 2 children as with 1. Posterior predictive: fewer women with 2 children than with 1. 7
9 > t.mc <- t2.mc <-NULL > for(s in 1:10000) { > theta1<-rgamma(1,a+sum(y1), b+length(y1)) > y1.mc<-rpois(length(y1),theta1) > t.mc <- c(t.mc,mean(y1.mc)) > t2.mc<-c(t2.mc,sum(y1.mc==2)/sum(y1.mc==1)) > } t(y ~ 2.5 ) Where t(y) isthemean! 8
10 t(y ~ ) Where t(y) is the ratio of 2 s to 1 s in a dataset.
11 10 I Lets look at the data (Gelman, Meng and Stern).
12 10 I Lets look at the data (Gelman, Meng and Stern). I Gelman (1990,92) describe positron emission tomography experiment.
13 10 I Lets look at the data (Gelman, Meng and Stern). I Gelman (1990,92) describe positron emission tomography experiment. I Goal: Estimate the density of a radioactive isotope in a cross-section of the brain.
14 10 I Lets look at the data (Gelman, Meng and Stern). I Gelman (1990,92) describe positron emission tomography experiment. I Goal: Estimate the density of a radioactive isotope in a cross-section of the brain. I 2-d image is estimate from gamma-ray counts in a ring of detectors around the head.
15 10 I Lets look at the data (Gelman, Meng and Stern). I Gelman (1990,92) describe positron emission tomography experiment. I Goal: Estimate the density of a radioactive isotope in a cross-section of the brain. I 2-d image is estimate from gamma-ray counts in a ring of detectors around the head. I n bins of counts based on positions of detectors 6 million counts.
16 10 I Lets look at the data (Gelman, Meng and Stern). I Gelman (1990,92) describe positron emission tomography experiment. I Goal: Estimate the density of a radioactive isotope in a cross-section of the brain. I 2-d image is estimate from gamma-ray counts in a ring of detectors around the head. I n bins of counts based on positions of detectors 6 million counts. I Bin count y i are modeled as independent Poisson( i )
17 10 I Lets look at the data (Gelman, Meng and Stern). I Gelman (1990,92) describe positron emission tomography experiment. I Goal: Estimate the density of a radioactive isotope in a cross-section of the brain. I 2-d image is estimate from gamma-ray counts in a ring of detectors around the head. I n bins of counts based on positions of detectors 6 million counts. I Bin count y i are modeled as independent Poisson( i ) I =Ag + r where g is the unknown image, A is a known linear operator and r are known corrections.
18 10 I Lets look at the data (Gelman, Meng and Stern). I Gelman (1990,92) describe positron emission tomography experiment. I Goal: Estimate the density of a radioactive isotope in a cross-section of the brain. I 2-d image is estimate from gamma-ray counts in a ring of detectors around the head. I n bins of counts based on positions of detectors 6 million counts. I Bin count y i are modeled as independent Poisson( i ) I =Ag + r where g is the unknown image, A is a known linear operator and r are known corrections. I A, g and r are non-negative.
19 10 I Lets look at the data (Gelman, Meng and Stern). I Gelman (1990,92) describe positron emission tomography experiment. I Goal: Estimate the density of a radioactive isotope in a cross-section of the brain. I 2-d image is estimate from gamma-ray counts in a ring of detectors around the head. I n bins of counts based on positions of detectors 6 million counts. I Bin count y i are modeled as independent Poisson( i ) I =Ag + r where g is the unknown image, A is a known linear operator and r are known corrections. I A, g and r are non-negative. I This is an easy problem without the constraint.
20 11 I Poisson noise + model problems makes exact non-negative solutions impossible.
21 11 I Poisson noise + model problems makes exact non-negative solutions impossible. I Use an estimate ĝ and capture the discrepancy between y and ˆ = Aĝ + r.
22 11 I Poisson noise + model problems makes exact non-negative solutions impossible. I Use an estimate ĝ and capture the discrepancy between y and ˆ = Aĝ + r. I Consider 2 discrepancy. X 2 (y; ˆ ) = X (y i ˆ i ) 2 ˆ i
23 11 I Poisson noise + model problems makes exact non-negative solutions impossible. I Use an estimate ĝ and capture the discrepancy between y and ˆ = Aĝ + r. I Consider 2 discrepancy. X 2 (y; ˆ ) = X (y i ˆ i ) 2 ˆ i I Fitting to real data: y with n =
24 11 I Poisson noise + model problems makes exact non-negative solutions impossible. I Use an estimate ĝ and capture the discrepancy between y and ˆ = Aĝ + r. I Consider 2 discrepancy. X 2 (y; ˆ ) = X (y i ˆ i ) 2 ˆ i I Fitting to real data: y with n = I The best-fit non-negative image ĝ was not an exact fit leading to the discrepancy between y and ˆ to be X 2 (y; ˆ ) 30, 000.
25 11 I Poisson noise + model problems makes exact non-negative solutions impossible. I Use an estimate ĝ and capture the discrepancy between y and ˆ = Aĝ + r. I Consider 2 discrepancy. X 2 (y; ˆ ) = X (y i ˆ i ) 2 ˆ i I Fitting to real data: y with n = I The best-fit non-negative image ĝ was not an exact fit leading to the discrepancy between y and ˆ to be X 2 (y; ˆ ) 30, 000. I Reject the model!
26 11 I Poisson noise + model problems makes exact non-negative solutions impossible. I Use an estimate ĝ and capture the discrepancy between y and ˆ = Aĝ + r. I Consider 2 discrepancy. X 2 (y; ˆ ) = X (y i ˆ i ) 2 ˆ i I Fitting to real data: y with n = I The best-fit non-negative image ĝ was not an exact fit leading to the discrepancy between y and ˆ to be X 2 (y; ˆ ) 30, 000. I Reject the model! I Possible failures:
27 11 I Poisson noise + model problems makes exact non-negative solutions impossible. I Use an estimate ĝ and capture the discrepancy between y and ˆ = Aĝ + r. I Consider 2 discrepancy. X 2 (y; ˆ ) = X (y i ˆ i ) 2 ˆ i I Fitting to real data: y with n = I The best-fit non-negative image ĝ was not an exact fit leading to the discrepancy between y and ˆ to be X 2 (y; ˆ ) 30, 000. I Reject the model! I Possible failures: I Error in the specification of A, r
28 11 I Poisson noise + model problems makes exact non-negative solutions impossible. I Use an estimate ĝ and capture the discrepancy between y and ˆ = Aĝ + r. I Consider 2 discrepancy. X 2 (y; ˆ ) = X (y i ˆ i ) 2 ˆ i I Fitting to real data: y with n = I The best-fit non-negative image ĝ was not an exact fit leading to the discrepancy between y and ˆ to be X 2 (y; ˆ ) 30, 000. I Reject the model! I Possible failures: I Error in the specification of A, r I Lack of independence
29 11 I Poisson noise + model problems makes exact non-negative solutions impossible. I Use an estimate ĝ and capture the discrepancy between y and ˆ = Aĝ + r. I Consider 2 discrepancy. X 2 (y; ˆ ) = X (y i ˆ i ) 2 ˆ i I Fitting to real data: y with n = I The best-fit non-negative image ĝ was not an exact fit leading to the discrepancy between y and ˆ to be X 2 (y; ˆ ) 30, 000. I Reject the model! I Possible failures: I Error in the specification of A, r I Lack of independence I super-poisson variance in the counts
30 I Poisson noise + model problems makes exact non-negative solutions impossible. I Use an estimate ĝ and capture the discrepancy between y and ˆ = Aĝ + r. I Consider 2 discrepancy. X 2 (y; ˆ ) = X (y i ˆ i ) 2 ˆ i I Fitting to real data: y with n = I The best-fit non-negative image ĝ was not an exact fit leading to the discrepancy between y and ˆ to be X 2 (y; ˆ ) 30, 000. I Reject the model! I Possible failures: I Error in the specification of A, r I Lack of independence I super-poisson variance in the counts I Error from discretizing the continuous g. 11
31 12 I The critically bad level of X 2 (y; ˆ ) isn +2 p 2n 23, 000.
32 12 I The critically bad level of X 2 (y; ˆ ) isn +2 p 2n 23, 000. I We reassess and change the model.
33 12 I The critically bad level of X 2 (y; ˆ ) isn +2 p 2n 23, 000. I We reassess and change the model. I We get a 2 discrepancy that is 22, 000 or even 20, 000.
34 12 I The critically bad level of X 2 (y; ˆ ) isn +2 p 2n 23, 000. I We reassess and change the model. I We get a 2 discrepancy that is 22, 000 or even 20, 000. I Should we just accept this new model?
35 12 I The critically bad level of X 2 (y; ˆ ) isn +2 p 2n 23, 000. I We reassess and change the model. I We get a 2 discrepancy that is 22, 000 or even 20, 000. I Should we just accept this new model? I No.
36 12 I The critically bad level of X 2 (y; ˆ ) isn +2 p 2n 23, 000. I We reassess and change the model. I We get a 2 discrepancy that is 22, 000 or even 20, 000. I Should we just accept this new model? I No. I Be skeptical!
37 Another graph example 13
38 Relational Data 0 Y = Y 21 Y 31. Y m1 1 Y 12 Y 13 Y 1m C A Y ij is the relationship between nodes i and j. When Y ij 2 {0,1} this is a sociomatrix.
39 High school in Adolescent Health data set I 181 male respondents I Each one nominated at most 5 friends I There are people who nominated no one and were nominated by no one
40 Model - standard approach (SRM) I The social relations model (SRM) was introduced by Warner, Kenny and Stoto (1979). I Probit model with row and column effects: (a i b i ) Y ij = 1 Zij >0 Z ij = b t X ij + a i + b j + e ij cor(e ij,e ji ) = r iid normal(0, ab ) I Note that cov(z ij,z kl )=0unlessZ ij and Z kl are in the same column, same row, or are reciprocal.
41 Summary statistics I Posterior predictive checks (PPC): at each iteration of the MCMC procedure we 1. Sample from the full conditionals of Z (s). 2. Simulate new data, Y (s). 3. Calculate test statistics. I Statistics to consider: I Binary row correlation: t row Y (s) = average of cor Y (s) (s) 2. i, (i,j),y j, (i,j) I Binary column correlation: tcol Y (s) = average of cor Y (s) (s) 2,Y (i,j),i (i,j),j I Binary joint correlation: tjoint Y (s) = average of cor Y (s) (s) i, (i,j),y, Y (s) (s) 2 (i,j),i j, (i,j),y (i,j),j
42 Summary statistics Density Density Density Row Column Joint Observed statistic. Excess correlation is present.
43 Aparticularmodelforrowandcolumncovariance I SRM is able to capture covariance within a row or within a column: 8 >< sa 2 i = k,j 6= l cov(z ij,z kl ) = sb >: 2 i 6= k,j = l 0 (i 6= k,j 6= l) I We propose a matrix normal model that can capture correlation between Z ij and Z kl which are in different rows and columns.
Statistics 360/601 Modern Bayesian Theory
Statistics 360/601 Modern Bayesian Theory Alexander Volfovsky Lecture 5 - Sept 12, 2016 How often do we see Poisson data? 1 2 Poisson data example Problem of interest: understand different causes of death
Appendix: Modeling Approach
AFFECTIVE PRIMACY IN INTRAORGANIZATIONAL TASK NETWORKS Appendix: Modeling Approach There is now a significant and developing literature on Bayesian methods in social network analysis. See, for instance,
Practical Statistics
Practical Statistics Lecture 1 (Nov. 9): - Correlation - Hypothesis Testing Lecture 2 (Nov. 16): - Error Estimation - Bayesian Analysis - Rejecting Outliers Lecture 3 (Nov. 18) - Monte Carlo Modeling -
Review. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Review DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing with
Part 2: One-parameter models
Part 2: One-parameter models 1 Bernoulli/binomial models Return to iid Y 1,...,Y n Bin(1, ). The sampling model/likelihood is p(y 1,...,y n ) = P y i (1 ) n P y i When combined with a prior p( ), Bayes
Stat 516, Homework 1
Stat 516, Homework 1 Due date: October 7 1. Consider an urn with n distinct balls numbered 1,..., n. We sample balls from the urn with replacement. Let N be the number of draws until we encounter a ball
Approximate Bayesian Computation
Approximate Bayesian Computation Michael Gutmann https://sites.google.com/site/michaelgutmann University of Helsinki and Aalto University 1st December 2015 Content Two parts: 1. The basics of approximate
ST 740: Markov Chain Monte Carlo
ST 740: Markov Chain Monte Carlo Alyson Wilson Department of Statistics North Carolina State University October 14, 2012 A. Wilson (NCSU Stsatistics) MCMC October 14, 2012 1 / 20 Convergence Diagnostics:
Bayesian Learning. HT2015: SC4 Statistical Data Mining and Machine Learning. Maximum Likelihood Principle. The Bayesian Learning Framework
HT5: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Maximum Likelihood Principle A generative model for
Bernoulli and Poisson models
Bernoulli and Poisson models Bernoulli/binomial models Return to iid Y 1,...,Y n Bin(1, ). The sampling model/likelihood is p(y 1,...,y n ) = P y i (1 ) n P y i When combined with a prior p( ), Bayes rule
Sampling and incomplete network data
1/58 Sampling and incomplete network data 567 Statistical analysis of social networks Peter Hoff Statistics, University of Washington 2/58 Network sampling methods It is sometimes difficult to obtain a
Random Effects Models for Network Data
Random Effects Models for Network Data Peter D. Hoff 1 Working Paper no. 28 Center for Statistics and the Social Sciences University of Washington Seattle, WA 98195-4320 January 14, 2003 1 Department of
Bayesian Networks: Construction, Inference, Learning and Causal Interpretation. Volker Tresp Summer 2016
Bayesian Networks: Construction, Inference, Learning and Causal Interpretation Volker Tresp Summer 2016 1 Introduction So far we were mostly concerned with supervised learning: we predicted one or several
Computer Vision Group Prof. Daniel Cremers. 10a. Markov Chain Monte Carlo
Group Prof. Daniel Cremers 10a. Markov Chain Monte Carlo Markov Chain Monte Carlo In high-dimensional spaces, rejection sampling and importance sampling are very inefficient An alternative is Markov Chain
Bayesian GLMs and Metropolis-Hastings Algorithm
Bayesian GLMs and Metropolis-Hastings Algorithm We have seen that with conjugate or semi-conjugate prior distributions the Gibbs sampler can be used to sample from the posterior distribution. In situations,
Statistical Methods for Particle Physics Lecture 1: parameter estimation, statistical tests
Statistical Methods for Particle Physics Lecture 1: parameter estimation, statistical tests http://benasque.org/2018tae/cgi-bin/talks/allprint.pl TAE 2018 Benasque, Spain 3-15 Sept 2018 Glen Cowan Physics
Penalized Loss functions for Bayesian Model Choice
Penalized Loss functions for Bayesian Model Choice Martyn International Agency for Research on Cancer Lyon, France 13 November 2009 The pure approach For a Bayesian purist, all uncertainty is represented
Chapter 5. Chapter 5 sections
1 / 43 sections Discrete univariate distributions: 5.2 Bernoulli and Binomial distributions Just skim 5.3 Hypergeometric distributions 5.4 Poisson distributions Just skim 5.5 Negative Binomial distributions
Bagging During Markov Chain Monte Carlo for Smoother Predictions
Bagging During Markov Chain Monte Carlo for Smoother Predictions Herbert K. H. Lee University of California, Santa Cruz Abstract: Making good predictions from noisy data is a challenging problem. Methods
Computer Vision Group Prof. Daniel Cremers. 11. Sampling Methods: Markov Chain Monte Carlo
Group Prof. Daniel Cremers 11. Sampling Methods: Markov Chain Monte Carlo Markov Chain Monte Carlo In high-dimensional spaces, rejection sampling and importance sampling are very inefficient An alternative
Generalized Linear Models for Non-Normal Data
Generalized Linear Models for Non-Normal Data Today s Class: 3 parts of a generalized model Models for binary outcomes Complications for generalized multivariate or multilevel models SPLH 861: Lecture
Part 6: Multivariate Normal and Linear Models
Part 6: Multivariate Normal and Linear Models 1 Multiple measurements Up until now all of our statistical models have been univariate models models for a single measurement on each member of a sample of
STA 4273H: Sta-s-cal Machine Learning
STA 4273H: Sta-s-cal Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 2 In our
Lecture 5 : The Poisson Distribution
Random events in time and space Lecture 5 : The Poisson Distribution Jonathan Marchini Many experimental situations occur in which we observe the counts of events within a set unit of time, area, volume,
Modeling networks: regression with additive and multiplicative effects
Modeling networks: regression with additive and multiplicative effects Alexander Volfovsky Department of Statistical Science, Duke May 25 2017 May 25, 2017 Health Networks 1 Why model networks? Interested
Quantile POD for Hit-Miss Data
Quantile POD for Hit-Miss Data Yew-Meng Koh a and William Q. Meeker a a Center for Nondestructive Evaluation, Department of Statistics, Iowa State niversity, Ames, Iowa 50010 Abstract. Probability of detection
Statistics 352: Spatial statistics. Jonathan Taylor. Department of Statistics. Models for discrete data. Stanford University.
352: 352: Models for discrete data April 28, 2009 1 / 33 Models for discrete data 352: Outline Dependent discrete data. Image data (binary). Ising models. Simulation: Gibbs sampling. Denoising. 2 / 33
Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.
1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n
Part 8: GLMs and Hierarchical LMs and GLMs
Part 8: GLMs and Hierarchical LMs and GLMs 1 Example: Song sparrow reproductive success Arcese et al., (1992) provide data on a sample from a population of 52 female song sparrows studied over the course
Lecture 5: Spatial probit models. James P. LeSage University of Toledo Department of Economics Toledo, OH
Lecture 5: Spatial probit models James P. LeSage University of Toledo Department of Economics Toledo, OH 43606 jlesage@spatial-econometrics.com March 2004 1 A Bayesian spatial probit model with individual
Default Priors and Effcient Posterior Computation in Bayesian
Default Priors and Effcient Posterior Computation in Bayesian Factor Analysis January 16, 2010 Presented by Eric Wang, Duke University Background and Motivation A Brief Review of Parameter Expansion Literature
CS839: Probabilistic Graphical Models. Lecture 7: Learning Fully Observed BNs. Theo Rekatsinas
CS839: Probabilistic Graphical Models Lecture 7: Learning Fully Observed BNs Theo Rekatsinas 1 Exponential family: a basic building block For a numeric random variable X p(x ) =h(x)exp T T (x) A( ) = 1
Econometrics Problem Set 10
Econometrics Problem Set 0 WISE, Xiamen University Spring 207 Conceptual Questions Dependent variable: P ass Probit Logit LPM Probit Logit LPM Probit () (2) (3) (4) (5) (6) (7) Experience 0.03 0.040 0.006
STAT Lecture 11: Bayesian Regression
STAT 491 - Lecture 11: Bayesian Regression Generalized Linear Models Generalized linear models (GLMs) are a class of techniques that include linear regression, logistic regression, and Poisson regression.
Math 50: Final. 1. [13 points] It was found that 35 out of 300 famous people have the star sign Sagittarius.
![Math 50: Final. 1. [13 points] It was found that 35 out of 300 famous people have the star sign Sagittarius.](/thumbs/95/124875477.jpg "Math 50: Final. 1. [13 points] It was found that 35 out of 300 famous people have the star sign Sagittarius.")
Math 50: Final 180 minutes, 140 points. No algebra-capable calculators. Try to use your calculator only at the end of your calculation, and show working/reasoning. Please do look up z, t, χ 2 values for
Part 4: Multi-parameter and normal models
Part 4: Multi-parameter and normal models 1 The normal model Perhaps the most useful (or utilized) probability model for data analysis is the normal distribution There are several reasons for this, e.g.,
Bayesian Methods with Monte Carlo Markov Chains II
Bayesian Methods with Monte Carlo Markov Chains II Henry Horng-Shing Lu Institute of Statistics National Chiao Tung University hslu@stat.nctu.edu.tw http://tigpbp.iis.sinica.edu.tw/courses.htm 1 Part 3
BTRY 4090: Spring 2009 Theory of Statistics
BTRY 4090: Spring 2009 Theory of Statistics Guozhang Wang September 25, 2010 1 Review of Probability We begin with a real example of using probability to solve computationally intensive (or infeasible)
Bayesian Regression Linear and Logistic Regression
When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we
Model Checking and Improvement
Model Checking and Improvement Statistics 220 Spring 2005 Copyright c 2005 by Mark E. Irwin Model Checking All models are wrong but some models are useful George E. P. Box So far we have looked at a number
Comparison of Three Calculation Methods for a Bayesian Inference of Two Poisson Parameters
Journal of Modern Applied Statistical Methods Volume 13 Issue 1 Article 26 5-1-2014 Comparison of Three Calculation Methods for a Bayesian Inference of Two Poisson Parameters Yohei Kawasaki Tokyo University
PROBABILISTIC REASONING SYSTEMS
PROBABILISTIC REASONING SYSTEMS In which we explain how to build reasoning systems that use network models to reason with uncertainty according to the laws of probability theory. Outline Knowledge in uncertain
STA 216, GLM, Lecture 16. October 29, 2007
STA 216, GLM, Lecture 16 October 29, 2007 Efficient Posterior Computation in Factor Models Underlying Normal Models Generalized Latent Trait Models Formulation Genetic Epidemiology Illustration Structural
(5) Multi-parameter models - Gibbs sampling. ST440/540: Applied Bayesian Analysis
Summarizing a posterior Given the data and prior the posterior is determined Summarizing the posterior gives parameter estimates, intervals, and hypothesis tests Most of these computations are integrals
Other Noninformative Priors
Other Noninformative Priors Other methods for noninformative priors include Bernardo s reference prior, which seeks a prior that will maximize the discrepancy between the prior and the posterior and minimize
Quiz 1. Name: Instructions: Closed book, notes, and no electronic devices.
Quiz 1. Name: Instructions: Closed book, notes, and no electronic devices. 1. What is the difference between a deterministic model and a probabilistic model? (Two or three sentences only). 2. What is the
Bayesian model selection: methodology, computation and applications
Bayesian model selection: methodology, computation and applications David Nott Department of Statistics and Applied Probability National University of Singapore Statistical Genomics Summer School Program
Bayesian Hierarchical Models
Bayesian Hierarchical Models Gavin Shaddick, Millie Green, Matthew Thomas University of Bath 6 th - 9 th December 2016 1/ 34 APPLICATIONS OF BAYESIAN HIERARCHICAL MODELS 2/ 34 OUTLINE Spatial epidemiology
L 36 Modern Physics [3] The atom and the nucleus. Structure of the nucleus. The structure of the nucleus SYMBOL FOR A NUCLEUS FOR A CHEMICAL X
![L 36 Modern Physics [3] The atom and the nucleus. Structure of the nucleus. The structure of the nucleus SYMBOL FOR A NUCLEUS FOR A CHEMICAL X](/thumbs/87/95852290.jpg "L 36 Modern Physics [3] The atom and the nucleus. Structure of the nucleus. The structure of the nucleus SYMBOL FOR A NUCLEUS FOR A CHEMICAL X")
L 36 Modern Physics [3] [L36] Nuclear physics what s inside the nucleus and what holds it together what is radioactivity carbon dating [L37] Nuclear energy nuclear fission nuclear fusion nuclear reactors
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate
UC Berkeley Department of Electrical Engineering and Computer Sciences. EECS 126: Probability and Random Processes
UC Berkeley Department of Electrical Engineering and Computer Sciences EECS 6: Probability and Random Processes Problem Set 3 Spring 9 Self-Graded Scores Due: February 8, 9 Submit your self-graded scores
Foundations of Statistical Inference
Foundations of Statistical Inference Julien Berestycki Department of Statistics University of Oxford MT 2015 Julien Berestycki (University of Oxford) SB2a MT 2015 1 / 16 Lecture 16 : Bayesian analysis
27 : Distributed Monte Carlo Markov Chain. 1 Recap of MCMC and Naive Parallel Gibbs Sampling
10-708: Probabilistic Graphical Models 10-708, Spring 2014 27 : Distributed Monte Carlo Markov Chain Lecturer: Eric P. Xing Scribes: Pengtao Xie, Khoa Luu In this scribe, we are going to review the Parallel
Bayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
Statistical Analysis of List Experiments
Statistical Analysis of List Experiments Graeme Blair Kosuke Imai Princeton University December 17, 2010 Blair and Imai (Princeton) List Experiments Political Methodology Seminar 1 / 32 Motivation Surveys
Lecture: Gaussian Process Regression. STAT 6474 Instructor: Hongxiao Zhu
Lecture: Gaussian Process Regression STAT 6474 Instructor: Hongxiao Zhu Motivation Reference: Marc Deisenroth s tutorial on Robot Learning. 2 Fast Learning for Autonomous Robots with Gaussian Processes
Lecture 3. G. Cowan. Lecture 3 page 1. Lectures on Statistical Data Analysis
Lecture 3 1 Probability (90 min.) Definition, Bayes theorem, probability densities and their properties, catalogue of pdfs, Monte Carlo 2 Statistical tests (90 min.) general concepts, test statistics,
Computer Vision Group Prof. Daniel Cremers. 14. Sampling Methods
Prof. Daniel Cremers 14. Sampling Methods Sampling Methods Sampling Methods are widely used in Computer Science as an approximation of a deterministic algorithm to represent uncertainty without a parametric
Stat 451 Lecture Notes Markov Chain Monte Carlo. Ryan Martin UIC
Stat 451 Lecture Notes 07 12 Markov Chain Monte Carlo Ryan Martin UIC www.math.uic.edu/~rgmartin 1 Based on Chapters 8 9 in Givens & Hoeting, Chapters 25 27 in Lange 2 Updated: April 4, 2016 1 / 42 Outline
Source Term Estimation for Hazardous Releases
Source Term Estimation for Hazardous Releases Fukushima Workshop NCAR, 22 nd 23 rd February 2012 Dstl/DOC61790 Dr. Gareth Brown Presentation Outline Dstl s approach to Source Term Estimation Recent developments
Randomized Quasi-Monte Carlo for MCMC
Randomized Quasi-Monte Carlo for MCMC Radu Craiu 1 Christiane Lemieux 2 1 Department of Statistics, Toronto 2 Department of Statistics, Waterloo Third Workshop on Monte Carlo Methods Harvard, May 2007
Exam 2 Practice Questions, 18.05, Spring 2014
Exam 2 Practice Questions, 18.05, Spring 2014 Note: This is a set of practice problems for exam 2. The actual exam will be much shorter. Within each section we ve arranged the problems roughly in order
CSC 2541: Bayesian Methods for Machine Learning
CSC 2541: Bayesian Methods for Machine Learning Radford M. Neal, University of Toronto, 2011 Lecture 3 More Markov Chain Monte Carlo Methods The Metropolis algorithm isn t the only way to do MCMC. We ll
Lecture 25: Models for Matched Pairs
Lecture 25: Models for Matched Pairs Dipankar Bandyopadhyay, Ph.D. BMTRY 711: Analysis of Categorical Data Spring 2011 Division of Biostatistics and Epidemiology Medical University of South Carolina Lecture
CS 2750: Machine Learning. Bayesian Networks. Prof. Adriana Kovashka University of Pittsburgh March 14, 2016
CS 2750: Machine Learning Bayesian Networks Prof. Adriana Kovashka University of Pittsburgh March 14, 2016 Plan for today and next week Today and next time: Bayesian networks (Bishop Sec. 8.1) Conditional
2.11. The Maximum of n Random Variables 3.4. Hypothesis Testing 5.4. Long Repeats of the Same Nucleotide
2.11. The Maximum of n Random Variables 3.4. Hypothesis Testing 5.4. Long Repeats of the Same Nucleotide Prof. Tesler Math 283 Fall 2018 Prof. Tesler Max of n Variables & Long Repeats Math 283 / Fall 2018
Markov Chain Monte Carlo
Markov Chain Monte Carlo Recall: To compute the expectation E ( h(y ) ) we use the approximation E(h(Y )) 1 n n h(y ) t=1 with Y (1),..., Y (n) h(y). Thus our aim is to sample Y (1),..., Y (n) from f(y).
Introduction to Probability and Statistics (Continued)
Introduction to Probability and Statistics (Continued) Prof. icholas Zabaras Center for Informatics and Computational Science https://cics.nd.edu/ University of otre Dame otre Dame, Indiana, USA Email:
Machine Learning for Data Science (CS4786) Lecture 24
Machine Learning for Data Science (CS4786) Lecture 24 Graphical Models: Approximate Inference Course Webpage : http://www.cs.cornell.edu/courses/cs4786/2016sp/ BELIEF PROPAGATION OR MESSAGE PASSING Each
SAMPLING ALGORITHMS. In general. Inference in Bayesian models
SAMPLING ALGORITHMS SAMPLING ALGORITHMS In general A sampling algorithm is an algorithm that outputs samples x 1, x 2,... from a given distribution P or density p. Sampling algorithms can for example be
Lecture 13 Fundamentals of Bayesian Inference
Lecture 13 Fundamentals of Bayesian Inference Dennis Sun Stats 253 August 11, 2014 Outline of Lecture 1 Bayesian Models 2 Modeling Correlations Using Bayes 3 The Universal Algorithm 4 BUGS 5 Wrapping Up
Directed Graphical Models
CS 2750: Machine Learning Directed Graphical Models Prof. Adriana Kovashka University of Pittsburgh March 28, 2017 Graphical Models If no assumption of independence is made, must estimate an exponential
Computational Genomics
Computational Genomics http://www.cs.cmu.edu/~02710 Introduction to probability, statistics and algorithms (brief) intro to probability Basic notations Random variable - referring to an element / event
One-Way Tables and Goodness of Fit
Stat 504, Lecture 5 1 One-Way Tables and Goodness of Fit Key concepts: One-way Frequency Table Pearson goodness-of-fit statistic Deviance statistic Pearson residuals Objectives: Learn how to compute the
Lecture 14: Introduction to Poisson Regression
Lecture 14: Introduction to Poisson Regression Ani Manichaikul amanicha@jhsph.edu 8 May 2007 1 / 52 Overview Modelling counts Contingency tables Poisson regression models 2 / 52 Modelling counts I Why
Markov Chains and MCMC
Markov Chains and MCMC CompSci 590.02 Instructor: AshwinMachanavajjhala Lecture 4 : 590.02 Spring 13 1 Recap: Monte Carlo Method If U is a universe of items, and G is a subset satisfying some property,
Modelling counts. Lecture 14: Introduction to Poisson Regression. Overview
Modelling counts I Lecture 14: Introduction to Poisson Regression Ani Manichaikul amanicha@jhsph.edu Why count data? Number of traffic accidents per day Mortality counts in a given neighborhood, per week
Bayesian data analysis in practice: Three simple examples
Bayesian data analysis in practice: Three simple examples Martin P. Tingley Introduction These notes cover three examples I presented at Climatea on 5 October 0. Matlab code is available by request to
Summary of Extending the Rank Likelihood for Semiparametric Copula Estimation, by Peter Hoff
Summary of Extending the Rank Likelihood for Semiparametric Copula Estimation, by Peter Hoff David Gerard Department of Statistics University of Washington gerard2@uw.edu May 2, 2013 David Gerard (UW)
Statistical Machine Learning Lecture 8: Markov Chain Monte Carlo Sampling
1 / 27 Statistical Machine Learning Lecture 8: Markov Chain Monte Carlo Sampling Melih Kandemir Özyeğin University, İstanbul, Turkey 2 / 27 Monte Carlo Integration The big question : Evaluate E p(z) [f(z)]
Generalized Linear Models
Generalized Linear Models STAT 489-01: Bayesian Methods of Data Analysis Spring Semester 2017 Contents 1 Multi-Variable Linear Models 1 2 Generalized Linear Models 4 2.1 Illustration: Logistic Regression..........
Statistics 360/601 Modern Bayesian Theory
Statistics 360/601 Modern Bayesian Theory Alexander Volfovsky Lecture 20something - Nov 27, 2018 The Challenger disaster This is an example from Monte Carlo Statistical Methods by Robert and Casella (2004).
Lecture 21: October 19
36-705: Intermediate Statistics Fall 2017 Lecturer: Siva Balakrishnan Lecture 21: October 19 21.1 Likelihood Ratio Test (LRT) To test composite versus composite hypotheses the general method is to use
PubH 5450 Biostatistics I Prof. Carlin. Lecture 13
PubH 5450 Biostatistics I Prof. Carlin Lecture 13 Outline Outline Sample Size Counts, Rates and Proportions Part I Sample Size Type I Error and Power Type I error rate: probability of rejecting the null
Gibbs Sampling in Latent Variable Models #1
Gibbs Sampling in Latent Variable Models #1 Econ 690 Purdue University Outline 1 Data augmentation 2 Probit Model Probit Application A Panel Probit Panel Probit 3 The Tobit Model Example: Female Labor
Spatial Discrete Choice Models
Spatial Discrete Choice Models Professor William Greene Stern School of Business, New York University SPATIAL ECONOMETRICS ADVANCED INSTITUTE University of Rome May 23, 2011 Spatial Correlation Spatially
Learning Bayesian Networks
Learning Bayesian Networks Probabilistic Models, Spring 2011 Petri Myllymäki, University of Helsinki V-1 Aspects in learning Learning the parameters of a Bayesian network Marginalizing over all all parameters
Chapter 5 continued. Chapter 5 sections
Chapter 5 sections Discrete univariate distributions: 5.2 Bernoulli and Binomial distributions Just skim 5.3 Hypergeometric distributions 5.4 Poisson distributions Just skim 5.5 Negative Binomial distributions
STAT 135 Lab 11 Tests for Categorical Data (Fisher s Exact test, χ 2 tests for Homogeneity and Independence) and Linear Regression
STAT 135 Lab 11 Tests for Categorical Data (Fisher s Exact test, χ 2 tests for Homogeneity and Independence) and Linear Regression Rebecca Barter April 20, 2015 Fisher s Exact Test Fisher s Exact Test
Fundamental Probability and Statistics
Fundamental Probability and Statistics "There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are
Open book, but no loose leaf notes and no electronic devices. Points (out of 200) are in parentheses. Put all answers on the paper provided to you.
ISQS 5347 Final Exam Spring 2017 Open book, but no loose leaf notes and no electronic devices. Points (out of 200) are in parentheses. Put all answers on the paper provided to you. 1. Recall the commute
ADJUSTED POWER ESTIMATES IN. Ji Zhang. Biostatistics and Research Data Systems. Merck Research Laboratories. Rahway, NJ
ADJUSTED POWER ESTIMATES IN MONTE CARLO EXPERIMENTS Ji Zhang Biostatistics and Research Data Systems Merck Research Laboratories Rahway, NJ 07065-0914 and Dennis D. Boos Department of Statistics, North
Brandon C. Kelly (Harvard Smithsonian Center for Astrophysics)
Brandon C. Kelly (Harvard Smithsonian Center for Astrophysics) Probability quantifies randomness and uncertainty How do I estimate the normalization and logarithmic slope of a X ray continuum, assuming
Recall that the AR(p) model is defined by the equation
Estimation of AR models Recall that the AR(p) model is defined by the equation X t = p φ j X t j + ɛ t j=1 where ɛ t are assumed independent and following a N(0, σ 2 ) distribution. Assume p is known and
Kevin James. MTHSC 3110 Section 2.1 Matrix Operations
MTHSC 3110 Section 2.1 Matrix Operations Notation Let A be an m n matrix, that is, m rows and n columns. We ll refer to the entries of A by their row and column indices. The entry in the i th row and j
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear
The subjective worlds of Frequentist and Bayesian statistics
Chapter 2 The subjective worlds of Frequentist and Bayesian statistics 2.1 The deterministic nature of random coin throwing Suppose that, in an idealised world, the ultimate fate of a thrown coin heads
Module 4: Bayesian Methods Lecture 9 A: Default prior selection. Outline
Module 4: Bayesian Methods Lecture 9 A: Default prior selection Peter Ho Departments of Statistics and Biostatistics University of Washington Outline Je reys prior Unit information priors Empirical Bayes
17 : Markov Chain Monte Carlo
10-708: Probabilistic Graphical Models, Spring 2015 17 : Markov Chain Monte Carlo Lecturer: Eric P. Xing Scribes: Heran Lin, Bin Deng, Yun Huang 1 Review of Monte Carlo Methods 1.1 Overview Monte Carlo
Bayesian Graphical Models
Graphical Models and Inference, Lecture 16, Michaelmas Term 2009 December 4, 2009 Parameter θ, data X = x, likelihood L(θ x) p(x θ). Express knowledge about θ through prior distribution π on θ. Inference