HW3 Solutions : Applied Bayesian and Computational Statistics

Size: px
Start display at page:

Download "HW3 Solutions : Applied Bayesian and Computational Statistics"

Transcription

1 HW3 Solutions : Applied Bayesian and Computational Statistics March 2, 2006 Problem 1 a Fatal Accidents Poisson(θ I will set a prior for θ to be Gamma, as it is the conjugate prior. I will allow the paramters to be α = 0.5, β = 0.01 as it is near the non-informative Jeffrey s prior shown in the last problem. Thus, the posterior distribution can be calculated. p(θ y p(y θp(θ = e nθ θ yi θ 1 2 e 0.01θ = e θ θ(n+0.01 yi 1 2 Note that this is a Gamma(α = y i + 1 2, β = n I get the 95% predictive interval using simulation. I found the interval to be [14, 35]. > theta=rgamma(1000, sum(y+0.5, length(y+0.01 > ypred=rpois(1000, theta > sort(ypred[c(25, 975] [1] b Let θ = α + βt (a noninformative priors What do we know about α + βt? We know it must be greater than zero since it is the mean of a Poisson. The prior for (α, β 1 is one option. Given a fixed range of time values we may simply truncate the prior to ensure that α + βt > 0 on the support of the prior (technically the interior of the support. Another option is Jeffrey s prior, however with multiple parameters, the use of Jeffreys prior is more controversial. (b Informative Prior: One may postulate that the number of accidents decreases over time and choose a prior which favors negative values of β. Since, we do not hear of plane crashes all the time in the news, we 1

2 may believe that α is a fairly small positive number. To put these beliefs into practice, we can take, for example, independent normals with means 50 and 5 and standard deviations of 20 and 5. Another informative prior would be to do a regression analysis on the data to get estimates and distributions for α and β, and assume then they are independent (not necessarily realistic. However, this would be double counting the data, and not really a prior. I will draw the contours assuming the regression analysis. (Again, this is not an ideal method. We know that the regression coefficients are normally distributed. Performing a regression analysis, we get the following. Call: lm(formula = y t Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t (Intercept e-06 *** t Signif. codes: 0 *** ** 0.01 * Residual standard error: on 8 degrees of freedom Multiple R-Squared: , Adjusted R-squared: F-statistic: on 1 and 8 DF, p-value: So I will let α N(28.87, std=2.75 and β N( 0.92, std=0.44. The contours are in Figure 1. > z=matrix(na, 100, 100 > alpha.grid=seq(0, 50, length=100 > beta.grid=seq(-30, 20, length=100 > funval=function(alpha, beta{ + return(dnorm(alpha, 28.87, 2.75*dnorm(beta, -0.92, 0.44 > for (i in 1:100{ + for (j in 1:100{ + z[i,j]=funval(alpha.grid[i], beta.grid[j] contour(alpha.grid, beta.grid, z, xlab="alpha", ylab="beta", xlim=c(20, 40 (c Posterior Density & Sufficient Statistics Using the non-informative prior (α, β 1 we can compute the posterior. p(α, β y 1 10 i=1 = e (nα+β t i i (α + βt i y i e (α+βt i (α + βt i y i 2

3 beta alpha Figure 1: Prior Contours for 4b In this case, there is no way to write the posterior as a reduced function of the data, so the data are the sufficient statistics, i.e. (y 1, t 1,..., (y 10, t 10 (d Proper Posterior To check if the posterior is proper, we need to show p(α, β ydαdβ <. The handwavy answer is that the exponential term will dominate any polynomial, and thus it must be integrable. A more precise answer is the following. If we let Ω be the parameter space (possible values for (α, β, we get sup (α,β Ω {(α + βt i e (α+βt i sup u [0, {ue u = M for some constant M. Hence, we can create an upper bound for the integral as M 9 (α + βt 1 e (α+βt 1 dαdβ which is integrable. (e Linear Regression See the above section on Prior Distributions. We found α N(28.87, std=2.75 and β N( 0.92, std=0.44. (f Posterior Recall that the prior on (α, β 1. Recall that the posterior is proportional to e (nα+β t i i (α+ βt i y i. A contour plot of the posterior is in Figure 2. > alpha.grid=seq(15, 50, length=100 > beta.grid=seq(-3, 1, length=100 > for (i in 1:100{ + for (j in 1:100{ + z[i,j]=postfun(alpha.grid[i], beta.grid[j] > contour(alpha.grid, beta.grid, z, xlim=c(20, 40, ylim=c(-2.5, 0.5, xlab="alpha", ylab="beta" (g Posterior density for expected number of accidents in Below is the code on how I sampled from the joint posterior. I then computed α + β 11 for each of the sampled values and then I plotted the histogram in Figure 3 3

4 beta alpha Figure 2: Posterior Contours for 4f > alpha.grid=seq(20, 40, length=100 > beta.grid=seq(-2.5, 0.5, length=100 > for (i in 1:100{ + for (j in 1:100{ + z[i,j]=postfun(alpha.grid[i], beta.grid[j] > zvec=c(z > post.sample=sample(length(alpha.grid*length(beta.grid, length(alpha.grid, replace=t, prob=zvec > alpha=rep(na, length(alpha.grid > beta=rep(na, length(beta.grid > for (m in 1:100{ + j=post.sample[m]%/%100 + i=post.sample[m]%%100 + j=j+1 + if ( i ==0 {i=100; j=j-1 + alpha[m]=alpha.grid[i] + beta[m]=beta.grid[j] + > hist(alpha+11*beta (h Number of fatal accidents For this, I generate 100 new values of y based on the values of α and β. Doing this, I find the confidence interval to be [9,30]. > newy=rpois(100, alpha+beta*11 > quantile(newy, c(0.025, % 97.5% (i Informative prior vs. posterior 4

5 Histogram of alpha + 11 * beta Frequency alpha + 11 * beta Figure 3: Posterior Expected Number of Accidents for 4g Problem 2 The informative prior does vary from the posterior, in that my informative prior assumes independence, and α and β are clearly not independent in the posterior. Though I did know when I made the informative prior that the independence assumption was not accurate. a L(θ y = i p i y i = (2 + θ p 1 (1 θ p 2+p 3 θ p 4 /4 n b We use simple acceptance/rejection sampling to do our Monte Carlo simulation. We get the following summary statistics mean: sd: accept. rate: Plots of the posterior density and histograms of the samples for 2b and 2d are given below. 5

6 2d 2d Density Density samples samples # This is the code for 2b y=c(125,18,20,34; loglik = function(theta { logp = y[1]*log(2+theta+(y[2]+y[3]*log(1-theta+ y[4]*log(theta; fudge = 1e-3; M = -nlm( function(t { -loglik(t,.5$minimum + fudge; n = ; samples = runif(n,0,1; U = runif(n,0,1; accept = (log(u < loglik(samples - M; samples = samples[accept]; sum(accept/n; mean(samples; sd(samples; hist(samples,br=100,freq=f; s = seq(0,1, length=1000; gridpost = exp(loglik(s; gridpost = gridpost/sum(gridpost*length(s; lines(s,gridpost; c The estimated mean and standard deviation from the Laplace approximation are and respectively. The code is given below. h = function(t { -loglik(t/n 6

7 hstar = function(theta { logp = -(y[1]*log(2+theta+(y[2]+y[3]* log(1-theta+(y[4]+1*log(theta/n; hstar2 = function(theta { logp = -(y[1]*log(2+theta+(y[2]+y[3]* log(1-theta+(y[4]+2*log(theta/n; min = nlm( h,.5, hessian=t; min.star = nlm( hstar,.5, hessian=t; min.star2 = nlm( hstar2,.5, hessian=t; mu = sqrt(1/min.star$hessian*exp(-n*min.star$minimum / (sqrt(1/min$hessian*exp(-n*min$minimum mom2 = sqrt(1/min.star2$hessian*exp(-n*min.star2$minimum / (sqrt(1/min$hessian*exp(-n*min$minimum sqrt(mom2 - muˆ2 d The likelihood function is the same. We get the following summary statistics from running the same code as in part a with a different data vector. mean: sd: accept. rate: e We have that our old posterior is cl(θ y where c is the appropriate normalizing constant. We have as the posterior mean under the new Beta(5, 15 prior E(θ y = θl(θ yp(θdθ θp(θcl(θ ydθ θi dbeta(θ i, 5, 15 = L(θ yp(θdθ p(θcl(θ ydθ dbeta(θi, 5, 15 (1 where θ i is the i th draw from our old posterior. Similarly, we may estimate the second moment. The estimated posterior mean and standard deviation from this Beta(5, 15 prior are and respectively. Problem 3 We are given that y ij = θ j + ɛ ij, ɛ ij N(0, σ 2, θ j = µ + γ j, γ j N(0, τ 2. Combining these, we see that y ij = µ + γ j + ɛ ij, ɛ ij N(0, σ 2, γ j N(0, τ 2 where the γ s are independent of each other, and independent of the ɛ s, and the ɛ s are independent of each other. a Corr(y i1,j, y i2,j, i 1 i 2 Corr(y i1,j, y i2,j = = Cov(y i1,j, y i2,j Var(y i1,j Var(y i2,j Cov(µ + γ j + ɛ i1 j, µ + γ j + ɛ i2 j Var(µ + γ j + ɛ i1 j Var(µ + γ j + ɛ i2 j 7

8 µ is constant = Cov(γ j + ɛ i1 j, γ j + ɛ i2 j Var(γ j + ɛ i1 j Var(γ j + ɛ i2 j = Cov(γ j, γ j + Cov(γ j, ɛ i1,j + Cov(γ j, ɛ i2,j + Cov(ɛ i1,j, ɛ i2,j Var(γ j + ɛ i1,j Var(γ j + ɛ i2 j by independence = τ σ 2 + τ 2 b Corr(y i1 j 1, y i2 j 2 Problem 4 Corr(y i1 j 1, y i2 j 2 = µ is constant = a Graphical Summaries Corr(µ + γ j1 + ɛ i1 j 1, µ + γ j2 + ɛ i2 j 2 Var(µ + γ j1 + ɛ i1 j 1 Var(µ + γ j2 + ɛ i2 j 2 Corr(γ j1 + ɛ i1 j 1, γ j2 + ɛ i2 j 2 Var(γ j1 + ɛ i1 j 1 Var(γ j2 + ɛ i2 j 2 = Corr(γ j 1, γ j2 + Corr(γ j2, ɛ i1 j 1 + Corr(γ j1, ɛ i2 j 2 + Corr(ɛ i1 j 1, ɛ i2 j 2 Var(γ j1 + ɛ i1 j 1 Var(γ j2 + ɛ i2 j 2 = = Var(γ j1 + ɛ i1 j 1 Var(γ j2 + ɛ i2 j 2 See included pdf file for history, autocorrelation density plots and the numerical summary for sigma.theta and theta[1]. My bugs code is below. list(j=8, y=c(28, 8, -3, 7, -1, 1, 18, 12, sigma.y=c(15, 10, 16, 11, 9, 11, 10, 18 model{ for (j in 1:J{ y[j] dnorm(theta[j], tau.y[j] theta[j] dnorm(mu, tau.theta tau.y[j]<- pow(sigma.y[j], -2 mu dnorm(0.0, 1.0E-6 tau.theta<-pow(sigma.theta,-2 sigma.theta dunif(0, 1000 b Change for sigma.theta See included pdf file for history, autocorrelation density plots and the numerical summary for sigma.theta and theta[1]. My bugs code is below. 8

9 list(j=8, y=c(28, 8, -3, 7, -1, 1, 18, 12, sigma.y=c(15, 10, 16, 11, 9, 11, 10, 18 model{ for (j in 1:J{ y[j] dnorm(theta[j], tau.y[j] theta[j] dnorm(mu, tau.theta tau.y[j]<- pow(sigma.y[j], -2 mu dnorm(0.0, 1.0E-6 tau.theta<-pow(sigma.theta,-2 v dgamma(1,1 sigma.theta<-sqrt(1/v Note: I choose just to compare sigma.theta and theta[1] for a write up comparison to save space. Other variables could have been reviewed. Comparing parts a and b, we see that the values for sigma.theta are generally larger for part a (mean 6.4, sd 5.51 than for part b (mean 1.52, sd Both histories look as if the chain is stable and converged. However it is clear that the autocorrelation for the part a is larger than the autocorrelation for part b. The densities appear similarly shaped, with the changes in the summaries (i.e. mean and sd reflected. For theta[1] (chosen arbitrarily for comparison, we see similar items. For part a, the mean was 10.99, and the sd was For part b, the mean is smaller, at 7.5 and the sd is also smaller at Both histories look stable, indicating the chain is converged. Though here, the autocorrelation for part a is smaller than the autocorrelation for part b. This could explain the smaller sd for part b. 9

Introduction to Bayesian Computation

Introduction to Bayesian Computation Introduction to Bayesian Computation Dr. Jarad Niemi STAT 544 - Iowa State University March 20, 2018 Jarad Niemi (STAT544@ISU) Introduction to Bayesian Computation March 20, 2018 1 / 30 Bayesian computation

More information

One-parameter models

One-parameter models One-parameter models Patrick Breheny January 22 Patrick Breheny BST 701: Bayesian Modeling in Biostatistics 1/17 Introduction Binomial data is not the only example in which Bayesian solutions can be worked

More information

Hierarchical Linear Models

Hierarchical Linear Models Hierarchical Linear Models Statistics 220 Spring 2005 Copyright c 2005 by Mark E. Irwin The linear regression model Hierarchical Linear Models y N(Xβ, Σ y ) β σ 2 p(β σ 2 ) σ 2 p(σ 2 ) can be extended

More information

Lecture 6. Prior distributions

Lecture 6. Prior distributions Summary Lecture 6. Prior distributions 1. Introduction 2. Bivariate conjugate: normal 3. Non-informative / reference priors Jeffreys priors Location parameters Proportions Counts and rates Scale parameters

More information

Statistics in Environmental Research (BUC Workshop Series) II Problem sheet - WinBUGS - SOLUTIONS

Statistics in Environmental Research (BUC Workshop Series) II Problem sheet - WinBUGS - SOLUTIONS Statistics in Environmental Research (BUC Workshop Series) II Problem sheet - WinBUGS - SOLUTIONS 1. (a) The posterior mean estimate of α is 14.27, and the posterior mean for the standard deviation of

More information

Linear Regression. Data Model. β, σ 2. Process Model. ,V β. ,s 2. s 1. Parameter Model

Linear Regression. Data Model. β, σ 2. Process Model. ,V β. ,s 2. s 1. Parameter Model Regression: Part II Linear Regression y~n X, 2 X Y Data Model β, σ 2 Process Model Β 0,V β s 1,s 2 Parameter Model Assumptions of Linear Model Homoskedasticity No error in X variables Error in Y variables

More information

The binomial model. Assume a uniform prior distribution on p(θ). Write the pdf for this distribution.

The binomial model. Assume a uniform prior distribution on p(θ). Write the pdf for this distribution. The binomial model Example. After suspicious performance in the weekly soccer match, 37 mathematical sciences students, staff, and faculty were tested for the use of performance enhancing analytics. Let

More information

Lecture 13 Fundamentals of Bayesian Inference

Lecture 13 Fundamentals of Bayesian Inference Lecture 13 Fundamentals of Bayesian Inference Dennis Sun Stats 253 August 11, 2014 Outline of Lecture 1 Bayesian Models 2 Modeling Correlations Using Bayes 3 The Universal Algorithm 4 BUGS 5 Wrapping Up

More information

Bayesian Graphical Models

Bayesian Graphical Models Graphical Models and Inference, Lecture 16, Michaelmas Term 2009 December 4, 2009 Parameter θ, data X = x, likelihood L(θ x) p(x θ). Express knowledge about θ through prior distribution π on θ. Inference

More information

Bayesian Regression Linear and Logistic Regression

Bayesian Regression Linear and Logistic Regression When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we

More information

INTRODUCTION TO BAYESIAN ANALYSIS

INTRODUCTION TO BAYESIAN ANALYSIS INTRODUCTION TO BAYESIAN ANALYSIS Arto Luoma University of Tampere, Finland Autumn 2014 Introduction to Bayesian analysis, autumn 2013 University of Tampere 1 / 130 Who was Thomas Bayes? Thomas Bayes (1701-1761)

More information

Stat 5102 Final Exam May 14, 2015

Stat 5102 Final Exam May 14, 2015 Stat 5102 Final Exam May 14, 2015 Name Student ID The exam is closed book and closed notes. You may use three 8 1 11 2 sheets of paper with formulas, etc. You may also use the handouts on brand name distributions

More information

Other Noninformative Priors

Other Noninformative Priors Other Noninformative Priors Other methods for noninformative priors include Bernardo s reference prior, which seeks a prior that will maximize the discrepancy between the prior and the posterior and minimize

More information

Classical and Bayesian inference

Classical and Bayesian inference Classical and Bayesian inference AMS 132 January 18, 2018 Claudia Wehrhahn (UCSC) Classical and Bayesian inference January 18, 2018 1 / 9 Sampling from a Bernoulli Distribution Theorem (Beta-Bernoulli

More information

Introduction to Markov Chain Monte Carlo

Introduction to Markov Chain Monte Carlo Introduction to Markov Chain Monte Carlo Jim Albert March 18, 2018 A Selected Data Problem Here is an interesting problem with selected data. Suppose you are measuring the speeds of cars driving on an

More information

One Parameter Models

One Parameter Models One Parameter Models p. 1/2 One Parameter Models September 22, 2010 Reading: Hoff Chapter 3 One Parameter Models p. 2/2 Highest Posterior Density Regions Find Θ 1 α = {θ : p(θ Y ) h α } such that P (θ

More information

Weakness of Beta priors (or conjugate priors in general) They can only represent a limited range of prior beliefs. For example... There are no bimodal beta distributions (except when the modes are at 0

More information

CS 361: Probability & Statistics

CS 361: Probability & Statistics March 14, 2018 CS 361: Probability & Statistics Inference The prior From Bayes rule, we know that we can express our function of interest as Likelihood Prior Posterior The right hand side contains the

More information

STAT Lecture 11: Bayesian Regression

STAT Lecture 11: Bayesian Regression STAT 491 - Lecture 11: Bayesian Regression Generalized Linear Models Generalized linear models (GLMs) are a class of techniques that include linear regression, logistic regression, and Poisson regression.

More information

Introduction to Bayesian Methods

Introduction to Bayesian Methods Introduction to Bayesian Methods Jessi Cisewski Department of Statistics Yale University Sagan Summer Workshop 2016 Our goal: introduction to Bayesian methods Likelihoods Priors: conjugate priors, non-informative

More information

Bayesian Inference for Regression Parameters

Bayesian Inference for Regression Parameters Bayesian Inference for Regression Parameters 1 Bayesian inference for simple linear regression parameters follows the usual pattern for all Bayesian analyses: 1. Form a prior distribution over all unknown

More information

Data Analysis and Uncertainty Part 2: Estimation

Data Analysis and Uncertainty Part 2: Estimation Data Analysis and Uncertainty Part 2: Estimation Instructor: Sargur N. University at Buffalo The State University of New York srihari@cedar.buffalo.edu 1 Topics in Estimation 1. Estimation 2. Desirable

More information

Using Bayes Factors for Model Selection in High-Energy Astrophysics

Using Bayes Factors for Model Selection in High-Energy Astrophysics Using Bayes Factors for Model Selection in High-Energy Astrophysics Shandong Zhao Department of Statistic, UCI April, 203 Model Comparison in Astrophysics Nested models (line detection in spectral analysis):

More information

David Giles Bayesian Econometrics

David Giles Bayesian Econometrics David Giles Bayesian Econometrics 1. General Background 2. Constructing Prior Distributions 3. Properties of Bayes Estimators and Tests 4. Bayesian Analysis of the Multiple Regression Model 5. Bayesian

More information

Foundations of Statistical Inference

Foundations of Statistical Inference Foundations of Statistical Inference Julien Berestycki Department of Statistics University of Oxford MT 2016 Julien Berestycki (University of Oxford) SB2a MT 2016 1 / 20 Lecture 6 : Bayesian Inference

More information

CS 361: Probability & Statistics

CS 361: Probability & Statistics October 17, 2017 CS 361: Probability & Statistics Inference Maximum likelihood: drawbacks A couple of things might trip up max likelihood estimation: 1) Finding the maximum of some functions can be quite

More information

Linear Models A linear model is defined by the expression

Linear Models A linear model is defined by the expression Linear Models A linear model is defined by the expression x = F β + ɛ. where x = (x 1, x 2,..., x n ) is vector of size n usually known as the response vector. β = (β 1, β 2,..., β p ) is the transpose

More information

GOV 2001/ 1002/ E-2001 Section 3 Theories of Inference

GOV 2001/ 1002/ E-2001 Section 3 Theories of Inference GOV 2001/ 1002/ E-2001 Section 3 Theories of Inference Solé Prillaman Harvard University February 11, 2015 1 / 48 LOGISTICS Reading Assignment- Unifying Political Methodology chs 2 and 4. Problem Set 3-

More information

WinBUGS : part 2. Bruno Boulanger Jonathan Jaeger Astrid Jullion Philippe Lambert. Gabriele, living with rheumatoid arthritis

WinBUGS : part 2. Bruno Boulanger Jonathan Jaeger Astrid Jullion Philippe Lambert. Gabriele, living with rheumatoid arthritis WinBUGS : part 2 Bruno Boulanger Jonathan Jaeger Astrid Jullion Philippe Lambert Gabriele, living with rheumatoid arthritis Agenda 2! Hierarchical model: linear regression example! R2WinBUGS Linear Regression

More information

David Giles Bayesian Econometrics

David Giles Bayesian Econometrics David Giles Bayesian Econometrics 5. Bayesian Computation Historically, the computational "cost" of Bayesian methods greatly limited their application. For instance, by Bayes' Theorem: p(θ y) = p(θ)p(y

More information

Lecture 3. Univariate Bayesian inference: conjugate analysis

Lecture 3. Univariate Bayesian inference: conjugate analysis Summary Lecture 3. Univariate Bayesian inference: conjugate analysis 1. Posterior predictive distributions 2. Conjugate analysis for proportions 3. Posterior predictions for proportions 4. Conjugate analysis

More information

Predictive Distributions

Predictive Distributions Predictive Distributions October 6, 2010 Hoff Chapter 4 5 October 5, 2010 Prior Predictive Distribution Before we observe the data, what do we expect the distribution of observations to be? p(y i ) = p(y

More information

Bayesian inference. Fredrik Ronquist and Peter Beerli. October 3, 2007

Bayesian inference. Fredrik Ronquist and Peter Beerli. October 3, 2007 Bayesian inference Fredrik Ronquist and Peter Beerli October 3, 2007 1 Introduction The last few decades has seen a growing interest in Bayesian inference, an alternative approach to statistical inference.

More information

Principles of Bayesian Inference

Principles of Bayesian Inference Principles of Bayesian Inference Sudipto Banerjee and Andrew O. Finley 2 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department

More information

DAG models and Markov Chain Monte Carlo methods a short overview

DAG models and Markov Chain Monte Carlo methods a short overview DAG models and Markov Chain Monte Carlo methods a short overview Søren Højsgaard Institute of Genetics and Biotechnology University of Aarhus August 18, 2008 Printed: August 18, 2008 File: DAGMC-Lecture.tex

More information

Bayesian Methods. David S. Rosenberg. New York University. March 20, 2018

Bayesian Methods. David S. Rosenberg. New York University. March 20, 2018 Bayesian Methods David S. Rosenberg New York University March 20, 2018 David S. Rosenberg (New York University) DS-GA 1003 / CSCI-GA 2567 March 20, 2018 1 / 38 Contents 1 Classical Statistics 2 Bayesian

More information

Weakness of Beta priors (or conjugate priors in general) They can only represent a limited range of prior beliefs. For example... There are no bimodal beta distributions (except when the modes are at 0

More information

Stats 579 Intermediate Bayesian Modeling. Assignment # 2 Solutions

Stats 579 Intermediate Bayesian Modeling. Assignment # 2 Solutions Stats 579 Intermediate Bayesian Modeling Assignment # 2 Solutions 1. Let w Gy) with y a vector having density fy θ) and G having a differentiable inverse function. Find the density of w in general and

More information

INTRODUCTION TO BAYESIAN METHODS II

INTRODUCTION TO BAYESIAN METHODS II INTRODUCTION TO BAYESIAN METHODS II Abstract. We will revisit point estimation and hypothesis testing from the Bayesian perspective.. Bayes estimators Let X = (X,..., X n ) be a random sample from the

More information

Bayesian Analysis (Optional)

Bayesian Analysis (Optional) Bayesian Analysis (Optional) 1 2 Big Picture There are two ways to conduct statistical inference 1. Classical method (frequentist), which postulates (a) Probability refers to limiting relative frequencies

More information

Introduction to Applied Bayesian Modeling. ICPSR Day 4

Introduction to Applied Bayesian Modeling. ICPSR Day 4 Introduction to Applied Bayesian Modeling ICPSR Day 4 Simple Priors Remember Bayes Law: Where P(A) is the prior probability of A Simple prior Recall the test for disease example where we specified the

More information

Principles of Bayesian Inference

Principles of Bayesian Inference Principles of Bayesian Inference Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department

More information

Introduction. Start with a probability distribution f(y θ) for the data. where η is a vector of hyperparameters

Introduction. Start with a probability distribution f(y θ) for the data. where η is a vector of hyperparameters Introduction Start with a probability distribution f(y θ) for the data y = (y 1,...,y n ) given a vector of unknown parameters θ = (θ 1,...,θ K ), and add a prior distribution p(θ η), where η is a vector

More information

Chapter 5. Bayesian Statistics

Chapter 5. Bayesian Statistics Chapter 5. Bayesian Statistics Principles of Bayesian Statistics Anything unknown is given a probability distribution, representing degrees of belief [subjective probability]. Degrees of belief [subjective

More information

36-463/663Multilevel and Hierarchical Models

36-463/663Multilevel and Hierarchical Models 36-463/663Multilevel and Hierarchical Models From Bayes to MCMC to MLMs Brian Junker 132E Baker Hall brian@stat.cmu.edu 1 Outline Bayesian Statistics and MCMC Distribution of Skill Mastery in a Population

More information

STAT 425: Introduction to Bayesian Analysis

STAT 425: Introduction to Bayesian Analysis STAT 425: Introduction to Bayesian Analysis Marina Vannucci Rice University, USA Fall 2017 Marina Vannucci (Rice University, USA) Bayesian Analysis (Part 1) Fall 2017 1 / 10 Lecture 7: Prior Types Subjective

More information

Prior Choice, Summarizing the Posterior

Prior Choice, Summarizing the Posterior Prior Choice, Summarizing the Posterior Statistics 220 Spring 2005 Copyright c 2005 by Mark E. Irwin Informative Priors Binomial Model: y π Bin(n, π) π is the success probability. Need prior p(π) Bayes

More information

BAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA

BAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA BAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA Intro: Course Outline and Brief Intro to Marina Vannucci Rice University, USA PASI-CIMAT 04/28-30/2010 Marina Vannucci

More information

Bayesian inference. Rasmus Waagepetersen Department of Mathematics Aalborg University Denmark. April 10, 2017

Bayesian inference. Rasmus Waagepetersen Department of Mathematics Aalborg University Denmark. April 10, 2017 Bayesian inference Rasmus Waagepetersen Department of Mathematics Aalborg University Denmark April 10, 2017 1 / 22 Outline for today A genetic example Bayes theorem Examples Priors Posterior summaries

More information

Bayesian Inference: Posterior Intervals

Bayesian Inference: Posterior Intervals Bayesian Inference: Posterior Intervals Simple values like the posterior mean E[θ X] and posterior variance var[θ X] can be useful in learning about θ. Quantiles of π(θ X) (especially the posterior median)

More information

Principles of Bayesian Inference

Principles of Bayesian Inference Principles of Bayesian Inference Sudipto Banerjee University of Minnesota July 20th, 2008 1 Bayesian Principles Classical statistics: model parameters are fixed and unknown. A Bayesian thinks of parameters

More information

Remarks on Improper Ignorance Priors

Remarks on Improper Ignorance Priors As a limit of proper priors Remarks on Improper Ignorance Priors Two caveats relating to computations with improper priors, based on their relationship with finitely-additive, but not countably-additive

More information

Statistical Methods III Statistics 212. Problem Set 2 - Answer Key

Statistical Methods III Statistics 212. Problem Set 2 - Answer Key Statistical Methods III Statistics 212 Problem Set 2 - Answer Key 1. (Analysis to be turned in and discussed on Tuesday, April 24th) The data for this problem are taken from long-term followup of 1423

More information

First Year Examination Department of Statistics, University of Florida

First Year Examination Department of Statistics, University of Florida First Year Examination Department of Statistics, University of Florida August 19, 010, 8:00 am - 1:00 noon Instructions: 1. You have four hours to answer questions in this examination.. You must show your

More information

Hypothesis Testing. Econ 690. Purdue University. Justin L. Tobias (Purdue) Testing 1 / 33

Hypothesis Testing. Econ 690. Purdue University. Justin L. Tobias (Purdue) Testing 1 / 33 Hypothesis Testing Econ 690 Purdue University Justin L. Tobias (Purdue) Testing 1 / 33 Outline 1 Basic Testing Framework 2 Testing with HPD intervals 3 Example 4 Savage Dickey Density Ratio 5 Bartlett

More information

Probability and Estimation. Alan Moses

Probability and Estimation. Alan Moses Probability and Estimation Alan Moses Random variables and probability A random variable is like a variable in algebra (e.g., y=e x ), but where at least part of the variability is taken to be stochastic.

More information

MAS3301 Bayesian Statistics Problems 5 and Solutions

MAS3301 Bayesian Statistics Problems 5 and Solutions MAS3301 Bayesian Statistics Problems 5 and Solutions Semester 008-9 Problems 5 1. (Some of this question is also in Problems 4). I recorded the attendance of students at tutorials for a module. Suppose

More information

p y (1 p) 1 y, y = 0, 1 p Y (y p) = 0, otherwise.

p y (1 p) 1 y, y = 0, 1 p Y (y p) = 0, otherwise. 1. Suppose Y 1, Y 2,..., Y n is an iid sample from a Bernoulli(p) population distribution, where 0 < p < 1 is unknown. The population pmf is p y (1 p) 1 y, y = 0, 1 p Y (y p) = (a) Prove that Y is the

More information

BUGS Bayesian inference Using Gibbs Sampling

BUGS Bayesian inference Using Gibbs Sampling BUGS Bayesian inference Using Gibbs Sampling Glen DePalma Department of Statistics May 30, 2013 www.stat.purdue.edu/~gdepalma 1 / 20 Bayesian Philosophy I [Pearl] turned Bayesian in 1971, as soon as I

More information

Exercise 5.4 Solution

Exercise 5.4 Solution Exercise 5.4 Solution Niels Richard Hansen University of Copenhagen May 7, 2010 1 5.4(a) > leukemia

More information

Example using R: Heart Valves Study

Example using R: Heart Valves Study Example using R: Heart Valves Study Goal: Show that the thrombogenicity rate (TR) is less than two times the objective performance criterion R and WinBUGS Examples p. 1/27 Example using R: Heart Valves

More information

Statistics 360/601 Modern Bayesian Theory

Statistics 360/601 Modern Bayesian Theory Statistics 360/601 Modern Bayesian Theory Alexander Volfovsky Lecture 5 - Sept 12, 2016 How often do we see Poisson data? 1 2 Poisson data example Problem of interest: understand different causes of death

More information

10. Exchangeability and hierarchical models Objective. Recommended reading

10. Exchangeability and hierarchical models Objective. Recommended reading 10. Exchangeability and hierarchical models Objective Introduce exchangeability and its relation to Bayesian hierarchical models. Show how to fit such models using fully and empirical Bayesian methods.

More information

INTRODUCTION TO BAYESIAN STATISTICS

INTRODUCTION TO BAYESIAN STATISTICS INTRODUCTION TO BAYESIAN STATISTICS Sarat C. Dass Department of Statistics & Probability Department of Computer Science & Engineering Michigan State University TOPICS The Bayesian Framework Different Types

More information

Metropolis-Hastings Algorithm

Metropolis-Hastings Algorithm Strength of the Gibbs sampler Metropolis-Hastings Algorithm Easy algorithm to think about. Exploits the factorization properties of the joint probability distribution. No difficult choices to be made to

More information

Computational statistics

Computational statistics Computational statistics Markov Chain Monte Carlo methods Thierry Denœux March 2017 Thierry Denœux Computational statistics March 2017 1 / 71 Contents of this chapter When a target density f can be evaluated

More information

Markov Chain Monte Carlo (MCMC) and Model Evaluation. August 15, 2017

Markov Chain Monte Carlo (MCMC) and Model Evaluation. August 15, 2017 Markov Chain Monte Carlo (MCMC) and Model Evaluation August 15, 2017 Frequentist Linking Frequentist and Bayesian Statistics How can we estimate model parameters and what does it imply? Want to find the

More information

Homework 2. For the homework, be sure to give full explanations where required and to turn in any relevant plots.

Homework 2. For the homework, be sure to give full explanations where required and to turn in any relevant plots. Homework 2 1 Data analysis problems For the homework, be sure to give full explanations where required and to turn in any relevant plots. 1. The file berkeley.dat contains average yearly temperatures for

More information

Part 2: One-parameter models

Part 2: One-parameter models Part 2: One-parameter models 1 Bernoulli/binomial models Return to iid Y 1,...,Y n Bin(1, ). The sampling model/likelihood is p(y 1,...,y n ) = P y i (1 ) n P y i When combined with a prior p( ), Bayes

More information

First Year Examination Department of Statistics, University of Florida

First Year Examination Department of Statistics, University of Florida First Year Examination Department of Statistics, University of Florida May 6, 2011, 8:00 am - 12:00 noon Instructions: 1. You have four hours to answer questions in this examination. 2. You must show your

More information

Comparing Nested Models

Comparing Nested Models Comparing Nested Models ST 370 Two regression models are called nested if one contains all the predictors of the other, and some additional predictors. For example, the first-order model in two independent

More information

Learning the hyper-parameters. Luca Martino

Learning the hyper-parameters. Luca Martino Learning the hyper-parameters Luca Martino 2017 2017 1 / 28 Parameters and hyper-parameters 1. All the described methods depend on some choice of hyper-parameters... 2. For instance, do you recall λ (bandwidth

More information

Stat 451 Lecture Notes Markov Chain Monte Carlo. Ryan Martin UIC

Stat 451 Lecture Notes Markov Chain Monte Carlo. Ryan Martin UIC Stat 451 Lecture Notes 07 12 Markov Chain Monte Carlo Ryan Martin UIC www.math.uic.edu/~rgmartin 1 Based on Chapters 8 9 in Givens & Hoeting, Chapters 25 27 in Lange 2 Updated: April 4, 2016 1 / 42 Outline

More information

Bayesian Estimation of DSGE Models 1 Chapter 3: A Crash Course in Bayesian Inference

Bayesian Estimation of DSGE Models 1 Chapter 3: A Crash Course in Bayesian Inference 1 The views expressed in this paper are those of the authors and do not necessarily reflect the views of the Federal Reserve Board of Governors or the Federal Reserve System. Bayesian Estimation of DSGE

More information

Bayesian course - problem set 5 (lecture 6)

Bayesian course - problem set 5 (lecture 6) Bayesian course - problem set 5 (lecture 6) Ben Lambert November 30, 2016 1 Stan entry level: discoveries data The file prob5 discoveries.csv contains data on the numbers of great inventions and scientific

More information

AMS-207: Bayesian Statistics

AMS-207: Bayesian Statistics Linear Regression How does a quantity y, vary as a function of another quantity, or vector of quantities x? We are interested in p(y θ, x) under a model in which n observations (x i, y i ) are exchangeable.

More information

Chapter 8: Sampling distributions of estimators Sections

Chapter 8: Sampling distributions of estimators Sections Chapter 8: Sampling distributions of estimators Sections 8.1 Sampling distribution of a statistic 8.2 The Chi-square distributions 8.3 Joint Distribution of the sample mean and sample variance Skip: p.

More information

Foundations of Statistical Inference

Foundations of Statistical Inference Foundations of Statistical Inference Julien Berestycki Department of Statistics University of Oxford MT 2015 Julien Berestycki (University of Oxford) SB2a MT 2015 1 / 16 Lecture 16 : Bayesian analysis

More information

Advanced Statistical Modelling

Advanced Statistical Modelling Markov chain Monte Carlo (MCMC) Methods and Their Applications in Bayesian Statistics School of Technology and Business Studies/Statistics Dalarna University Borlänge, Sweden. Feb. 05, 2014. Outlines 1

More information

Part 6: Multivariate Normal and Linear Models

Part 6: Multivariate Normal and Linear Models Part 6: Multivariate Normal and Linear Models 1 Multiple measurements Up until now all of our statistical models have been univariate models models for a single measurement on each member of a sample of

More information

Linear Regression Models P8111

Linear Regression Models P8111 Linear Regression Models P8111 Lecture 25 Jeff Goldsmith April 26, 2016 1 of 37 Today s Lecture Logistic regression / GLMs Model framework Interpretation Estimation 2 of 37 Linear regression Course started

More information

Module 22: Bayesian Methods Lecture 9 A: Default prior selection

Module 22: Bayesian Methods Lecture 9 A: Default prior selection Module 22: Bayesian Methods Lecture 9 A: Default prior selection Peter Hoff Departments of Statistics and Biostatistics University of Washington Outline Jeffreys prior Unit information priors Empirical

More information

Computational Cognitive Science

Computational Cognitive Science Computational Cognitive Science Lecture 8: Frank Keller School of Informatics University of Edinburgh keller@inf.ed.ac.uk Based on slides by Sharon Goldwater October 14, 2016 Frank Keller Computational

More information

UNIVERSITY OF MASSACHUSETTS. Department of Mathematics and Statistics. Basic Exam - Applied Statistics. Tuesday, January 17, 2017

UNIVERSITY OF MASSACHUSETTS. Department of Mathematics and Statistics. Basic Exam - Applied Statistics. Tuesday, January 17, 2017 UNIVERSITY OF MASSACHUSETTS Department of Mathematics and Statistics Basic Exam - Applied Statistics Tuesday, January 17, 2017 Work all problems 60 points are needed to pass at the Masters Level and 75

More information

Introduction to Bayesian Methods. Introduction to Bayesian Methods p.1/??

Introduction to Bayesian Methods. Introduction to Bayesian Methods p.1/?? to Bayesian Methods Introduction to Bayesian Methods p.1/?? We develop the Bayesian paradigm for parametric inference. To this end, suppose we conduct (or wish to design) a study, in which the parameter

More information

PARAMETER ESTIMATION: BAYESIAN APPROACH. These notes summarize the lectures on Bayesian parameter estimation.

PARAMETER ESTIMATION: BAYESIAN APPROACH. These notes summarize the lectures on Bayesian parameter estimation. PARAMETER ESTIMATION: BAYESIAN APPROACH. These notes summarize the lectures on Bayesian parameter estimation.. Beta Distribution We ll start by learning about the Beta distribution, since we end up using

More information

The joint posterior distribution of the unknown parameters and hidden variables, given the

The joint posterior distribution of the unknown parameters and hidden variables, given the DERIVATIONS OF THE FULLY CONDITIONAL POSTERIOR DENSITIES The joint posterior distribution of the unknown parameters and hidden variables, given the data, is proportional to the product of the joint prior

More information

Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.

Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A. 1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n

More information

Chapter 4 - Fundamentals of spatial processes Lecture notes

Chapter 4 - Fundamentals of spatial processes Lecture notes Chapter 4 - Fundamentals of spatial processes Lecture notes Geir Storvik January 21, 2013 STK4150 - Intro 2 Spatial processes Typically correlation between nearby sites Mostly positive correlation Negative

More information

STAT J535: Chapter 5: Classes of Bayesian Priors

STAT J535: Chapter 5: Classes of Bayesian Priors STAT J535: Chapter 5: Classes of Bayesian Priors David B. Hitchcock E-Mail: hitchcock@stat.sc.edu Spring 2012 The Bayesian Prior A prior distribution must be specified in a Bayesian analysis. The choice

More information

Estimation of Quantiles

Estimation of Quantiles 9 Estimation of Quantiles The notion of quantiles was introduced in Section 3.2: recall that a quantile x α for an r.v. X is a constant such that P(X x α )=1 α. (9.1) In this chapter we examine quantiles

More information

CPSC 540: Machine Learning

CPSC 540: Machine Learning CPSC 540: Machine Learning Empirical Bayes, Hierarchical Bayes Mark Schmidt University of British Columbia Winter 2017 Admin Assignment 5: Due April 10. Project description on Piazza. Final details coming

More information

Principles of Bayesian Inference

Principles of Bayesian Inference Principles of Bayesian Inference Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department

More information

Time Series and Dynamic Models

Time Series and Dynamic Models Time Series and Dynamic Models Section 1 Intro to Bayesian Inference Carlos M. Carvalho The University of Texas at Austin 1 Outline 1 1. Foundations of Bayesian Statistics 2. Bayesian Estimation 3. The

More information

Bayesian modelling. Hans-Peter Helfrich. University of Bonn. Theodor-Brinkmann-Graduate School

Bayesian modelling. Hans-Peter Helfrich. University of Bonn. Theodor-Brinkmann-Graduate School Bayesian modelling Hans-Peter Helfrich University of Bonn Theodor-Brinkmann-Graduate School H.-P. Helfrich (University of Bonn) Bayesian modelling Brinkmann School 1 / 22 Overview 1 Bayesian modelling

More information

Metropolis Hastings. Rebecca C. Steorts Bayesian Methods and Modern Statistics: STA 360/601. Module 9

Metropolis Hastings. Rebecca C. Steorts Bayesian Methods and Modern Statistics: STA 360/601. Module 9 Metropolis Hastings Rebecca C. Steorts Bayesian Methods and Modern Statistics: STA 360/601 Module 9 1 The Metropolis-Hastings algorithm is a general term for a family of Markov chain simulation methods

More information

Problem Selected Scores

Problem Selected Scores Statistics Ph.D. Qualifying Exam: Part II November 20, 2010 Student Name: 1. Answer 8 out of 12 problems. Mark the problems you selected in the following table. Problem 1 2 3 4 5 6 7 8 9 10 11 12 Selected

More information

Model Checking and Improvement

Model Checking and Improvement Model Checking and Improvement Statistics 220 Spring 2005 Copyright c 2005 by Mark E. Irwin Model Checking All models are wrong but some models are useful George E. P. Box So far we have looked at a number

More information

Robust Bayesian Regression

Robust Bayesian Regression Readings: Hoff Chapter 9, West JRSSB 1984, Fúquene, Pérez & Pericchi 2015 Duke University November 17, 2016 Body Fat Data: Intervals w/ All Data Response % Body Fat and Predictor Waist Circumference 95%

More information

ST430 Exam 1 with Answers

ST430 Exam 1 with Answers ST430 Exam 1 with Answers Date: October 5, 2015 Name: Guideline: You may use one-page (front and back of a standard A4 paper) of notes. No laptop or textook are permitted but you may use a calculator.

More information