Other Noninformative Priors

Size: px

Start display at page:

Download "Other Noninformative Priors"

Bernice Anderson
6 years ago
Views:

1 Other Noninformative Priors Other methods for noninformative priors include Bernardo s reference prior, which seeks a prior that will maximize the discrepancy between the prior and the posterior and minimize the discrepancy between the likelihood and the posterior (a dominant likelihood prior ). An improper prior, in which p(θ) =. Θ A highly diffuse proper prior, e.g., for normal data with µ unknown, a N(0, ) prior for µ. (This is very close to the improper prior p(µ) 1.)

2 Informative Prior Forms Informative prior information is usually based on expert opinion or previous research about the parameter(s) of interest. Power Priors Suppose we have access to previous data x 0 that is analogous to the data we will gather. Then our power prior could be p(θ x 0, a 0 ) p(θ)[l(θ x 0 )] a 0 where p(θ) is an ordinary prior and a 0 [0, 1] is an exponent measuring the influence of the previous data.

3 Power Priors As a 0 0, the influence of the previous data is lessened. As a 0 1, the influence of the previous data is strengthened. The posterior, given our actual data x, is then π(θ x, x 0, a 0 ) p(θ x 0, a 0 )L(θ x) To avoid specifying a single a 0 value: We could put a, say, beta distribution p(a 0 ) on a 0 and average over values of a 0 in [0, 1]: p(θ x 0 ) = 1 0 p(θ)[l(θ x 0 )] a 0 p(a 0 ) da 0

4 Prior Elicitation A challenge is putting expert opinion into a form where it can be used as a prior distribution. Strategies: Requesting guesses for several quantiles (maybe {0.1, 0.25, 0.5, 0.75, 0.9}?) from a few experts. For a normal prior, note that a quantile q(α) is related to the z-value Φ 1 (α) by: q(α) = mean + Φ 1 (α) (std. dev.) Via regression on the provided [q(α), Φ 1 (α)] values, we can get estimates for the mean and standard deviation of the normal prior.

5 Prior Elicitation Another strategy asks the expert to provide a predictive modal value (most likely value) for the parameter. Then a rough 67% interval is requested from the expert. With a normal prior, the length of this interval is twice the prior standard deviation. For a prior on a Bernoulli probability, the most likely probability of success is often clear.

6 Spike-and-Slab Priors for Linear Models In regression, the priors on the regression coefficients are crucial. Whether or not β j = 0 defines whether X j is important in the regression. For any j, a useful prior for β j is: h_0j p(β) h_1j f j 0 f j β j

7 Spike-and-Slab Priors for Linear Models Here: P(β j = 0) = h 0j ( = prior probability that X j is not needed in the model) P(β j 0) = 1 h 0j = h 1j ( fj ( f j ) ) = 2f j h 1j (where [ f j, f j ] contains all reasonable values for β j ) To include X j in the model with certainty, set h 0j = 0. To reflect more doubt that X j should be in the model, increase the ratio h 0j h 1j = h 0j (1 h 0j )/2f j = 2f j h 0j 1 h 0j Recently, nonparametric priors have become popular, typically involving a mixture of Dirichet processes.

8 CHAPTER 6 SLIDES START HERE

9 The Monte Carlo Method The Monte Carlo method involves studying a distribution (e.g., a posterior) and its characteristics by generating many random observations having that distribution. If θ (1),..., θ (S) iid π(θ x), then the empirical distribution of {θ (1),..., θ (S) } approximates the posterior, when S is large. By the law of large numbers, as S. 1 S S g(θ (s) ) E[g(θ) x] s=1

10 The Monte Carlo Method So as S : 1 S 1 θ = 1 S S θ (s) posterior mean s=1 S (θ (s) θ) 2 posterior variance s=1 #{θ (s) c} P[θ c x] S median{θ (1),..., θ (S) } posterior median (and similarly for any posterior quantile).

11 The Monte Carlo Method If the posterior is a common distribution, as in many conjugate analyses, we could draw samples from the posterior using R functions. Example 1: (General Social Survey) Sample 1: # of children for women age 40+, no bachelor s degree. Sample 2: # of children for women age 40+, bachelor s degree or higher. Assume Poisson(θ 1 ) and Poisson(θ 2 ) models for the data. We use gamma(2,1) priors for θ 1 and for θ 2.

12 The Monte Carlo Method Data: n 1 = 111, i x i1 = 217 Data: n 2 = 44, i x i2 = 66 Posterior for θ 1 is gamma(219,112). Posterior for θ 2 is gamma(68, 45). Find P[θ 1 > θ 2 x 1, x 2 ]. Find posterior distribution of the ratio θ 1 θ 2. See R example using Monte Carlo method on course web page.

Bayesian Inference: Posterior Intervals

Bayesian Inference: Posterior Intervals Simple values like the posterior mean E[θ X] and posterior variance var[θ X] can be useful in learning about θ. Quantiles of π(θ X) (especially the posterior median)