Conjugate Priors for Normal Data

Transcription

1 Conjugate Priors for Normal Data September 22, 2008 Gill Chapter 3. Sections 4, 7-8 Conjugate Priors for Normal Data p.1/17

2 Normal Model IID observations Y = (Y 1,Y 2,...Y n ) Y i N(µ,σ 2 ) unknown parameters µ and σ 2 or precision, φ = 1/σ 2. Likelihood: L(µ,φ Y ) φ n/2 exp{ 1 2 φss)} exp{ 1 2 φn(µ ȳ)2 } Sufficient statistics sample mean ȳ = n i=1 y i sample sum of squares SS = i (y i ȳ) 2 Conjugate Priors for Normal Data p.2/17

3 Conjugate Prior Distribution for (µ, φ) is Normal-Gamma. µ φ N(m 0, 1/(p 0 φ)) φ G(v 0 /2, SS 0 /2) p(µ,φ) φ v 0/2 1 exp{ φ SS 0 2 }φ1/2 exp{ φ p 0 2 (µ m 0) 2 } µ,φ NG(m 0,p 0,v 0 /2, SS 0 /2) Normal-Gamma family µ,φ Y NG(m n,p n,v n /2, SS n /2) Posterior is Normal- Gamma Conjugate Priors for Normal Data p.3/17

4 Updating the Posterior Parameters Under the Normal-Gamma prior distribution: ( ) 1 µ φ,y N m n, p n φ φ Y G( v n 2, SS n 2 ) where p n = p 0 + n m n = nȳ + p 0m 0 p n v n = v 0 + n SS n = SS 0 + SS + np 0 p n (ȳ m 0 ) 2 Conjugate Priors for Normal Data p.4/17

5 Interpretation p n precision for estimating µ after n observations m n expected value for µ after n observations m n = n p n ȳ + p 0 p n m 0 weighted average v n degrees of freedom φ G(a/2,b/2) φb χ 2 a with degrees of freedom a SS n = SS 0 + SS + np 0 p n (ȳ m 0 ) 2 posterior variation: prior variation, observed variation (sum of squares), variation between prior mean and sample mean Conjugate Priors for Normal Data p.5/17

6 Derivation p(µ,φ Y ) φ v 0/2 1 exp{ φ SS 0 2 } φn/2 exp{ φ SS 2 } φ 1/2 exp{ φ p 0 2 (µ m 0) 2 } exp{ φ n 2 (µ ȳ)2 } Hint: Expand quadratics in µ to read off the posterior precision p n and mean m n then complete the square and factor 1 2 (p nµ 2 2p n m n µ + p n m 2 n) = 1 2 p n(µ m n ) 2 (Note: page 80 in Gill has errors; need to keep track of terms after completing the square.) Conjugate Priors for Normal Data p.6/17

7 Derivation Take second line and complete the square: p(µ φ,y ) φ 1/2 e { φ p 0 2 (µ m 0 ) 2} e { φ n 2 (µ ȳ)2 } = φ 1/2 e { φ 2(p 0 µ 2 2p 0 m 0 µ+p 0 m 2 0+nµ 2 2nȳµ+nȳ 2 )} = φ 1/2 e { φ 2((p 0 +n)µ 2 2(p 0 m 0 +nȳ)µ+p 0 m 2 0+nȳ 2 )} = φ 1/2 e ( φ 2 h i (p 0 +n)µ 2 (p 2p 0 m 0 +nȳ) n µ+p n m 2 pn n ) e ( φ 2(p 0 m 2 0+nȳ 2 p n m 2 n)) Read off posterior mean and precision from terms in [ ]. Conjugate Priors for Normal Data p.7/17

8 Derivation Factor and combine terms from earlier p(µ,φ Y ) φ 1/2 e { φ p n 2 (µ m n) 2} φ v 0 +n 2 1 e { φ ( SS 0 +SS 2 )} e { φ 2(p 0 m 2 0+nȳ 2 p n m 2 n)} The first line is the (unnormalized) posterior density for µ given φ and the second line is proportional to the posterior density for φ. The last term comes from the left-over terms after completing the square. Simplifying p(φ Y ) φ v 0 +n 2 1 exp{ φ ( SS0 + SS + p 0n p n (ȳ m 0 ) 2 2 ) } Conjugate Priors for Normal Data p.8/17

9 Marginal for µ If then µ φ N ( m, ) 1 pφ φ G(v/2, SS/2) µ t ( v,m, SS v µ = D SS m + t v vp µ m t(v, 0, 1) SS vp ) 1 p Conjugate Priors for Normal Data p.9/17

10 Example: SPF Construct an informative prior distribution for µ: Take prior median SPF to be 16 P(µ > 64) = 0.01 information in prior is worth 25 observations Solve for hyperparameters that are consistent with these quantiles: m 0 = log(16), p 0 = 25, v 0 = p 0 1 P(µ < log(64)) = 0.99 where µ m 0 SS 0 /(v 0 p 0 ) t v 0 SS 0 = Conjugate Priors for Normal Data p.10/17

11 Posterior Distribution Summary statistics ybar = SS = n = 13 Posterior hyperparameters: p n = = 38 m n = ( )/38 = v n = = 37 SS n = ( ) 2 (13 25)/38 = µ Y t(37, 2.508, /(38 37)) Conjugate Priors for Normal Data p.11/17

12 Samples from the Posterior To draw samples of SPF from the posterior distribution: Draw φ Y phi = rgamma(10000, vn/2, rate=ssn/2) Draw µ φ,y mu = rnorm(10000, mn, 1/sqrt(phi*pn)) or Draw µ Y directly mu = rt(10000,vn)*sqrt(ssn/(pn*vn))+ mn transform exp(mu) HPDinterval(exp(mu)) Conjugate Priors for Normal Data p.12/17

13 Distributions Posterior Distribution of µ Density µ Posterior Distribution of σ Density σ Conjugate Priors for Normal Data p.13/17

14 Distribution for SPF Posterior Distribution of SPF Density exp(µ) 95% HPD Interval 4.54 to Reference 95% HPD Interval 4.47 to Conjugate Priors for Normal Data p.14/17

15 Predictive Distribution What is the predictive distribution of a new observaton Y given the current data Y? p(y Y )? Y µ,φ independent of Y p(y Y ) = p(y µ,φ)p(µ,φ Y )dµdφ Conjugate Priors for Normal Data p.15/17

16 Integrals Use normal trick to integrate out µ: If X N(m,s 2 ) then X is equal in distribution to sz + m where Z N(0, 1) Y D = µ + Z/ φ Sum of two independent normals: ( Y φ,y N m n, 1 φ (1 + 1 ) ) p n Use previous result about t distributions ( Y Y t v n,m n, SS n (1 + 1 ) ) v n p n Conjugate Priors for Normal Data p.16/17

17 Predictive Distribution for New Subject Y = log(trt) log(baseline) D = µ + Z/φ TRT/BASELINE = exp(y ) Y D = exp{t (37, 2.5, 5.32 (1 + 1/38))} Distribution is the exponential of a Student t Simulate from predictive distribution 50% HPD interval is (0.0003, 12.4) from CODA Predict that with sunscreen there is a 50% chance that the next subject could be exposed from 0 to 12 times longer than without sunscreen. Prior influence? Conjugate Priors for Normal Data p.17/17