Predictive Distributions

Transcription

1 Predictive Distributions October 6, 2010 Hoff Chapter 4 5 October 5, 2010

2 Prior Predictive Distribution Before we observe the data, what do we expect the distribution of observations to be? p(y i ) = p(y i θ)p(θ) dθ Θ p(y 1,..., y n ) = p(y 1,..., y n θ)p(θ) dθ Θ What we would predict for y given no data. Useful for assessing whether choice of prior distribution does capture prior beliefs. Recall de Finetti s exchangability representation

3 SPF example Normal model from last class for SPF data: Y i µ, φ iid N(µ, 1/φ) µ φ N(log(16), 1/(25φ)) φ G(24/2, 185.7/2) Need to integrate out µ and φ Result is a Student-t distribution ( Y i St 24, log(16), (1 + 1 ) 25 )

4 Some Normal Results If X µ, σ 2 N(µ, σ 2 ) then X is equal in distribution to µ + σz where Z N(0, 1) Linear combinations of Normal Random variables are Normal µ N(m 0, σ 2 /p 0 ) σz N(0, σ 2 ) independent of µ σ 2 Y i d = µ + σz Y i σ 2 N(m 0, σ 2 (1 + 1/p 0 )) Add means and variances (in case of independent normals)

5 Integrate out σ 2 Re-expression of result from last class: So X σ 2 N(m, σ 2 p 1 ) 1/σ 2 G(ν/2, SS/2) ( X St ν, m, SS ) ν p 1 Y i σ 2 N(log(16), σ 2 (1 + 1/25)) 1/σ 2 G(24/2, 187.5/2) ( Y i St 24, 0, ) (1 + 1/25) 24 To get to SPF s exponentiate Y i

6 Sample Quantiles > mu=rt(10000,24)*sqrt((1/25)*187.5/24)+log(16) > quantile(exp(mu)) 1% 5% 25% 50% 75% 95% 99% > Y=rt(10000,24)*sqrt((1+1/25)*187.5/24)+log(16) > quantile(exp(y)) 1% 5% 25% 50% 75% Note both have the same median, but the variance in the prior predictive is much greater! Believable? Was I really thinking about exp(y ) rather than exp(µ) when I constructed the prior? (yes)

7 Posterior Predictive What is the predictive distribution of a new observaton Y given the current data Y? p(y Y ) = p(y µ, φ, Y )p(µ, φ Y ) dµ dφ = p(y µ, φ)p(µ, φ Y ) dµ dφ Use assumption that given µ, φ, Y is independent of Y, normal trick to integrate out µ, and results about t ( Y Y t v n, m n, SS n (1 + 1 ) ) v n p n

8 Predictive Distribution for New Subject Y = log(trt) log(baseline) D = µ + Z/φ TRT/BASELINE = exp(y ) Y D = exp{t (37, 2.5, 5.32 (1 + 1/38))} Distribution is the exponential of a Student t Simulate from predictive distribution 50% HPD interval is (0.0003, 12.4) from CODA Predict that with sunscreen there is a 50% chance that the next subject could tolerate 0 to 12 times longer sun exposure with the sunscreen than without sunscreen. Prior influence?

9 Recall Posterior ( ) 1 µ φ, Y N m n, p n φ φ Y G( v n 2, v ns 2 n 2 ) = µ Y St(v n, m n, s 2 n /p n) where p n = p 0 + n m n = nȳ + p 0m 0 p n v n = v 0 + n SS n = SS 0 + SS + np 0 p n (ȳ m 0 ) 2 s 2 n = SS n/v n

10 Limiting Case Let p 0 0, v 0 1 (and SS 0) Then µ Y St(n 1, Ȳ, s2 /n) or µ Ȳ s/ n t n 1 Before you see the data, the sampling distribution of the t statistic conditional on θ has a Student t distribution After you see the data, the distribution of µ given the data also has the same Student t distribution. Formal Bayes posterior based on the improper prior p(µ, φ) 1/φ

11 Jeffreys prior(s) Jeffreys for the multi-dimensional θ is p(θ) I (θ) 1/2 I (θ) is the determinant of the expected Fisher Information [ ] [I (θ)] ij = E 2 L(θ) θ i θ j For the normal model ( ) µ I = φ [ φ 0 normal φ 2 ] Jeffreys prior p J (µ, φ) φ 1/2

12 Independent Jeffreys Prior In higher dimensions (µ 1,..., µ d, φ) p J (µ 1,... µ d, φ) φ 1+d/2 (adds degrees of freedom as the dimension increases which is not desirable) Alternatively, find the Jefferys prior for µ for fixed φ, p(µ) 1 find the Jeffreys prior for φ for fixed µ, p(φ) 1/φ Independent Jeffreys prior is P IJ (µ, φ) p(µ)p(φ) 1/φ

13 Estimators of µ Under the independent Jeffreys prior, E[µ Y ] = Ȳ = ˆµ IJ Under the conjugate Normal-Gamma prior where w n = n/(n + p 0 ) E[µ Y ] = w n Ȳ + (1 w n )m 0 = ˆµ C Frequentist properties of the posterior mean as an estimator of µ when the truth is µ 0.

14 Bias Variance Trade-off True mean is µ 0 MSE : E Y [ˆµ µ 0 ] 2 = Bias 2 + Variance Bias: E Y [ˆµ IJ ] µ 0 = µ 0 µ 0 Bias: E Y [ˆµ C ] µ 0 = w n µ 0 + (1 w n )m 0 Variance: V Y [ˆµ IJ ] = σ2 n Variance: V Y [ˆµ C ] = w 2 n σ 2 n < σ2 n

15 Which is Best in terms of MSE? Can show (some algebra) that the MSE of using the posterior mean under a proper conjugate prior distribution is less than the MSE of using Ȳ (posterior mean of the formal posterior under the Independent Jeffreys prior) if (m 0 µ 0 ) 2 < σ 2 ( 1 n + 2 p 0 ) Can find a prior such that the Bayesian estimator will a lower average squared distance from the truth than the sample mean.

16 Better Prior for SPF? 1. Think about Prior Predictive for prior eliciation 2. Simulate from Prior Distributions (particularly prior predictive distribution, preferably before looking at the data) to check that prior beliefs are consistent with selected dsitribution 3. Caution: Repeating analysis with different prior sensitivity analysis (OK) changing prior until posterior results are better (not OK)

17 New Prior > p0 = > v0 = > m0 = > SS0 = v0*((q.99 - m0)/(qt(.99, v0)))^2 /(1 + 1/p0) > SS0