Conjugate Priors for Normal Data

Transcription

1 Conjugate Priors for Normal Data September 23, 2009 Hoff Chapter 5 Conjugate Priors for Normal Data p.1/22

2 Normal Model IID observations Y = (Y 1,Y 2,...Y n ) Y i µ,σ 2 N(µ,σ 2 ) unknown parameters µ and σ 2. From a Bayesian perspective, it is easier to work with the precision, φ, where φ = 1/σ 2. Likelihood L(µ,φ Y ) n i=1 1 2π φ 1/2 exp{ 1 2 φ(y i µ) 2 } φ n/2 exp{ 1 2 φ i (Y i µ) 2 } Conjugate Priors for Normal Data p.2/22

3 Likelihood Factorization L(µ,φ Y ) φ n/2 exp{ 1 2 φ i (Y i µ) 2 } φ n/2 exp{ 1 2 φ i [ (Yi Ȳ ) (µ Ȳ )] 2 } φ n/2 exp{ 1 2 φ [ i (Y i Ȳ )2 + n(µ Ȳ )2 ]} φ n/2 exp{ 1 2 φs2 (n 1)} exp{ 1 2 φn(µ Ȳ )2 } s 2 = i (Y i Ȳ )2 /(n 1) is the sample variance. Conjugate Priors for Normal Data p.3/22

4 Likelihood for Normal Model L(µ,φ Y ) φ n/2 exp{ 1 2 φss)} exp{ 1 2 φn(µ ȳ)2 } Sufficient statistics sample mean ȳ = n i=1 y i sample sum of squares SS = i (y i ȳ) 2 Conjugate Priors for Normal Data p.4/22

5 Conjugate Prior Distribution for (µ, φ) is Normal-Gamma. µ φ N(m 0, 1/(p 0 φ)) φ G(v 0 /2, SS 0 /2) p(µ,φ) φ v 0/2 1 exp{ φ SS 0 2 }φ1/2 exp{ φ p 0 2 (µ m 0) 2 } µ,φ NG(m 0,p 0,v 0 /2, SS 0 /2) Normal-Gamma family µ,φ Y NG(m n,p n,v n /2, SS n /2) Posterior is Normal- Gamma Conjugate Priors for Normal Data p.5/22

6 Updating the Posterior Parameters Under the Normal-Gamma prior distribution: ( ) 1 µ φ,y N m n, p n φ φ Y G( v n 2, SS n 2 ) where p n = p 0 + n m n = nȳ + p 0m 0 p n v n = v 0 + n SS n = SS 0 + SS + np 0 p n (ȳ m 0 ) 2 Conjugate Priors for Normal Data p.6/22

7 Interpretation p n precision for estimating µ after n observations m n expected value for µ after n observations m n = n p n ȳ + p 0 p n m 0 weighted average v n degrees of freedom φ G(a/2,b/2) φb χ 2 a with degrees of freedom a SS n = SS 0 + SS + np 0 p n (ȳ m 0 ) 2 posterior variation: prior variation, observed variation (sum of squares), variation between prior mean and sample mean Conjugate Priors for Normal Data p.7/22

8 Derivation p(µ,φ Y ) φ v 0/2 1 exp{ φ SS 0 2 } φn/2 exp{ φ SS 2 } φ 1/2 exp{ φ p 0 2 (µ m 0) 2 } exp{ φ n 2 (µ ȳ)2 } Hint: Expand quadratics in µ to read off the posterior precision p n and mean m n then complete the square and factor 1 2 (p nµ 2 2p n m n µ + p n m 2 n) = 1 2 p n(µ m n ) 2 Conjugate Priors for Normal Data p.8/22

9 Derivation Take second line and complete the square: p(µ φ,y ) φ 1/2 e { φ p 0 2 (µ m 0 ) 2} e { φ n 2 (µ ȳ)2 } = φ 1/2 e { φ 2(p 0 µ 2 2p 0 m 0 µ+p 0 m 2 0+nµ 2 2nȳµ+nȳ 2 )} = φ 1/2 e { φ 2((p 0 +n)µ 2 2(p 0 m 0 +nȳ)µ+p 0 m 2 0+nȳ 2 )} = φ 1/2 e ( φ 2 h i (p 0 +n)µ 2 (p 2p 0 m 0 +nȳ) n µ+p n m 2 pn n ) e ( φ 2(p 0 m 2 0+nȳ 2 p n m 2 n)) Read off posterior mean and precision from terms in [ ]. Conjugate Priors for Normal Data p.9/22

10 Derivation Factor and combine terms from earlier p(µ,φ Y ) φ 1/2 e { φ p n 2 (µ m n) 2} φ v 0 +n 2 1 e { φ ( SS 0 +SS 2 )} e { φ 2(p 0 m 2 0+nȳ 2 p n m 2 n)} The first line is the (unnormalized) posterior density for µ given φ and the second line is proportional to the posterior density for φ. The last term comes from the left-over terms after completing the square. Simplifying p(φ Y ) φ v 0 +n 2 1 exp{ φ ( SS0 + SS + p 0n p n (ȳ m 0 ) 2 2 ) } Conjugate Priors for Normal Data p.10/22

11 Marginal Distribution for µ Y p(µ Y ) = p(µ,φ Y )dφ { φ v n+1 1 SSn + p 2 n (µ m n ) 2 } exp[ φ ]dφ 2 This has the form of a Gamma integral with a = (v + 1)/2 and b equal to the mess multiplying φ in the exponential term, so that the result is b a (at least that is all that matters) p(µ Y ) { SS n + p n (µ m n ) 2} (v n+1) 2 Conjugate Priors for Normal Data p.11/22

12 Student t Distribution X has a Student t distribution with location l, scale S and degrees of freedom δ: X t δ (l, S) p(t;δ,l, S) Rearrange posterior distribution: (δ + (x l)2 S ) (δ+1)/2 p(µ Y ) { SS n + p n (µ m n ) 2} (v n+1) 2 ( v n + (µ m n) 2 1 p n SS n v n ) (vn +1)/2 Student t vn (m n,s 2 n) location m n, df = v n, scale s 2 n = 1 p n SS n v n Conjugate Priors for Normal Data p.12/22

13 Standard Student t Standardize X t δ (l, S) by subtracting location and dividing by square root of the scale: X l S t δ (0, 1) (new location 0 and scale 1) µ m n s n t vn (0, 1) Use rt, qt, pt, dt in R µ D = m n + t vn s n Conjugate Priors for Normal Data p.13/22

14 Example: SPF A Sunlight Protection Factor (SPF) of 5 means an individual that can tolerate X minutes of sunlight without any sunscreen can tolerate 5X minutes with sunscreen. Tolerance (min) PreTreatment During Treatment Conjugate Priors for Normal Data p.14/22

15 Pairing A paired design may be more powerful than two sample design because of patient to patient variability, particularly if there is positive correlation V (d) = V (Y T c) + V (Y B ) 2Cov(Y B,Y T ) Analysis should take into account pairing which induces dependence between observations use differences use ratios or log(ratios) difference in logs Ratios make more sense given the goals: how much longer can a person be exposed to the sun relative to their baseline. Conjugate Priors for Normal Data p.15/22

16 Data Normal Q Q Plot Theoretical Quantiles Normal Q Q Plot Theoretical Quantiles Conjugate Priors for Normal Data p.16/ Sample Quantiles Sample Quantiles

17 Model for SPF Model Y = log(trt) - log(control) as N(µ, 1/φ) E(log(TRT/CONTROL)) = µ = log(spf) Want distribution of exp µ SPF Summary statistics ybar = s2 = n = 13 Conjugate Priors for Normal Data p.17/22

18 Informative Prior for SPF Construct an informative prior distribution for µ: Take prior median SPF to be 16 P(µ > 64) = 0.01 information in prior is worth 25 observations Solve for hyperparameters that are consistent with these quantiles: m 0 = log(16), p 0 = 25, v 0 = p 0 1 P(µ < log(64)) = 0.99 where µ m 0 SS 0 /(v 0 p 0 ) t v 0 SS 0 = Conjugate Priors for Normal Data p.18/22

19 Posterior Distribution Summary statistics ybar = SS = n = 13 Posterior hyperparameters: p n = = 38 m n = ( )/38 = v n = = 37 SS n = ( ) 2 (13 25)/38 = µ Y t(37, 2.508, /(38 37)) Conjugate Priors for Normal Data p.19/22

20 Samples from the Posterior To draw samples of SPF from the posterior distribution: Draw φ Y phi = rgamma(10000, vn/2, rate=ssn/2) Draw µ φ,y mu = rnorm(10000, mn, 1/sqrt(phi*pn)) or Draw µ Y directly mu = rt(10000,vn)*sqrt(ssn/(pn*vn))+ mn transform exp(mu) HPDinterval(exp(mu)) Conjugate Priors for Normal Data p.20/22

21 Distributions Posterior Distribution of µ Density µ Posterior Distribution of σ Density σ Conjugate Priors for Normal Data p.21/22

22 Distribution for SPF Posterior Distribution of SPF Density exp(µ) 95% HPD Interval 4.54 to Reference 95% HPD Interval 4.47 to Conjugate Priors for Normal Data p.22/22