Statistics 360/601 Modern Bayesian Theory

Size: px

Start display at page:

Download "Statistics 360/601 Modern Bayesian Theory"

Erica Lewis
5 years ago
Views:

1 Statistics 360/601 Modern Bayesian Theory Alexander Volfovsky Lecture 20something - Nov 27, 2018

By Kapitel, vectorisation by Adam Rdzikowski - Own work, GFDL, curid=24312838 Simplified cross section of the joints between rocket segments SRB;

2 The Challenger disaster This is an example from Monte Carlo Statistical Methods by Robert and Casella (2004). In 1986, the Challenger space shuttle exploded on take off. Suspected cause of explosion: O-ring failure. By Kapitel, vectorisation by Adam Rdzikowski - Own work, GFDL, curid= Simplified cross section of the joints between rocket segments SRB; outside to left. Legend: A - steel wall 0.5 inches (12.7 mm) thick B - base O-ring gasket, C - backup O-ring gasket, D - Strengthening-Cover band, E - insulation, F - insulation, G - carpeting, H - sealing paste, I - fixed propellant

3 The Challenger disaster O-rings are rigorously tested. Anomaly during the launch: unusually cold temperature (31 degrees F). There is reason to believe that O-ring failure probabilities increase as temperature decreases. We have data on 23 other launches: Flight Failure Temperature Hard to read... Looks like more 1s at lower temperatures.

4 The Challenger disaster 80 temp as.factor(fail) Linear relationship between odds or log odds of failure and temperature seems reasonable. Use logistic regression!

5 4 The Challenger disaster Our model: Pr(failure = 1) = exp(α + β temp) 1 + exp(α + β temp) We have 23 data points that we will assume are conditionally independent. failure 1,..., failure 23 α, β, temp 1,..., temp 23 exp(α + β temp ( i ) 1 + exp(α + β temp i ) )failure i 1 ( 1 + exp(α + β temp i ) )1 failure i

6 5 Dissecting the code Initialize θ (0) Propose θ from proposal J θ (θ θ (0) ) Evaluate the likelihood under the proposal v 1 = p(y i θ ) and under the current θ (0) : v 2 = p(y i θ (0) ) Evaluate the prior under the proposal v 3 = π θ (θ ) and under the current value v 4 = π θ (θ (0) ) Evaluate the proposal probability for the proposed value v 5 = J θ (θ θ (0) ) and for the current value v 6 = J θ (θ (0) θ ) Compute the acceptance ratio: r = min(1, v 1 v 2 v 3 v 4 v 6 v 5 ) Simulate u unif (0, 1) and accept θ if u < r.

7 Dissecting the code Initialize θ (0) = (α (0), β (0 ) Propose θ from proposal π α (α ˆb)φ(β). Evaluate the likelihood under the proposal: v 1 = p(y i α, β, x i ) and under θ (0) : v 2 = p(y i α (0), β (0), x i ) Evaluate the prior under the proposal v 3 = π α (α ˆb)π β (β ) and under the current value v 4 = π α (α (0) ˆb)π β (β (0) ) Evaluate the proposal probability for the proposed value v 5 = π α (α ˆb)φ(β ) and for the current value v 6 = π α (α (0) ˆb)φ(β (0) ) Compute the acceptance ratio: r = min(1, v 1 v 2 v 3 v 4 v 6 v 5 ) Simulate u unif (0, 1) and accept θ if u < r.

8 7 Dissecting the code Initialize θ (0) = (α (0), β (0 ) Propose θ from proposal π α (α ˆb)φ(β). Evaluate the likelihood under the proposal: v 1 = p(y i α, β, x i ) and under θ (0) : v 2 = p(y i α (0), β (0), x i ) Evaluate the prior under the proposal v 3 = π α (α ˆb)π β (β ) and under the current value v 4 = π α (α (0) ˆb)π β (β (0) ) Evaluate the proposal probability for the proposed value v 5 = π α (α ˆb)φ(β ) and for the current value v 6 = π α (α (0) ˆb)φ(β (0) ) Compute the acceptance ratio: r = min(1, v 1 v 2 v 4 Simulate u unif (0, 1) and accept θ if u < r. v 3 v 6 φ(β (0) ) v 5 φ(β ) )

9 Proposal and evaluation > rprop <- function(theta0) { + a <- log(rexp(1, 1 / exp(a.mle ))) + b <- rnorm(1, b.mle, sqrt(0.1^2)) + return(c(a, b)) + } > > dprop <- function(theta, theta0) { + a <- theta[1] + b <- theta[2] + pr1 <- exp(a) * exp(-exp(a) / b.mme) / b.mme + pr2 <- dnorm(b, b.mle, sqrt(var.b.mle)) + return(pr1 * pr2) + }

10 Posterior evaluation > dpost <- function(theta) { + a <- theta[1] + b <- theta[2] + p <- 1-1 / (1 + exp(a + b * x)) + lik <- exp(sum(dbinom(y, size=1, prob=p, log=true))) + dprior <- exp(a) * exp(-exp(a) / b.mme) / b.mme + return(lik * dprior) + }

11 One iteration > theta_star <- rprop(theta[i-1,]) > r <- log(dpost(theta_star)) - + log(dpost(theta[i-1,])) + + log(dprop(theta[i-1,],theta_star)) - + log(dprop(theta_star,theta[i-1,])) > r <- min(exp(r),1) > if(runif(1)<= R){ + theta[i,] <- theta_star; ct <- ct + 1} > else{ theta[i,] <- theta[i-1,] }

Here θ = (α, β) is the unknown parameter. The likelihood function is

Here θ = (α, β) is the unknown parameter. The likelihood function is Stat 591 Notes Logistic regression and Metropolis Hastings example Ryan Martin (rgmartin@uic.edu) November 2, 2013 Introduction (This part taken from Example 1.13 in Robert and Casella, 2004.) In 1986,