Bayesian Inference in a Normal Population September 17, 2008 Gill Chapter 3. Sections 1-4, 7-8 Bayesian Inference in a Normal Population p.1/18
Normal Model IID observations Y = (Y 1,Y 2,...Y n ) Y i N(µ,σ 2 ) unknown parameters µ and σ 2. From a Bayesian perspective, it is easier to work with the precision, φ, where φ = 1/σ 2. Likelihood L(µ,φ Y ) n i=1 1 2π φ 1/2 exp{ 1 2 φ(y i µ) 2 } φ n/2 exp{ 1 2 φ i (Y i µ) 2 } Bayesian Inference in a Normal Population p.2/18
Likelihood Factorization L(µ,φ Y ) φ n/2 exp{ 1 2 φ i (Y i µ) 2 } φ n/2 exp{ 1 2 φ i [ (Yi Ȳ ) (µ Ȳ )] 2 } φ n/2 exp{ 1 2 φ [ i (Y i Ȳ )2 + n(µ Ȳ )2 ]} φ n/2 exp{ 1 2 φs2 (n 1)} exp{ 1 2 φn(µ Ȳ )2 } s 2 = i (Y i Ȳ )2 /(n 1) is the sample variance. Bayesian Inference in a Normal Population p.3/18
Prior Distributions Conjugate prior distribution for (µ, φ) is Normal-Gamma. µ φ N(m 0, 1/(n 0 φ)) φ Gamma(v 0 /2, (v 0 s 2 0)/2) p(φ) φ v 0/2 1 exp{ φv 0 s 2 0 /2} Non-informative prior distribution reference prior (more in Ch 5) p(µ,φ) = 1/φ µ is uniform on the real line (improper) log(φ) is uniform on real line (improper) invariant to scale and location changes of the data. Bayesian Inference in a Normal Population p.4/18
Reference Posterior Distribution p(µ, φ Y ) L(µ, φ)p(µ, φ) = φ n 2 exp{ 1 = 2 φs2 (n 1)} exp{ 1 2 φn(µ Ȳ )2 }φ 1 {φ n 1 2 1 e { 1 2 φs2 (n 1)} }{ φ 1 2 e 1 2 Gamma ( n 1 2 = p(φ Y )p(µ φ,y ), (n 1) s2 2 ) N φn(µ Ȳ )2} ( Ȳ, ) 1 φn Bayesian Inference in a Normal Population p.5/18
Marginal Distribution for µ Y Obtain the marginal distribution for µ by integrating out φ from the joint posterior distribution, and recognizing the kernel of the distribution! p(µ Y ) p(µ,φ Y )dφ = φ n/2 1 exp[ 1 2 {φs2 (n 1) + φn(µ Ȳ )2 }]dφ Bayesian Inference in a Normal Population p.6/18
Continued p(µ Y ) φ n/2 1 exp{ φ 1 2 [s2 (n 1) + n(µ Ȳ )2 ]}dφ This has the form of a Gamma integral with a = n/2 and b equal to the mess multiplying φ in the exponential term, so that the result is b a (at least that is all that matters) p(µ Y ) ( s 2 (n 1) + n(µ Ȳ )2) n/2 Bayesian Inference in a Normal Population p.7/18
Student t Distribution X has a Student t distribution with location l, scale S and degrees of freedom δ: X t δ (l, S) p(t;δ,l, S) Rearrange posterior distribution: ( 1 + 1 δ (x l) 2 S ) (δ+1)/2 p(µ Y ) (s 2 (n 1) + n(µ Ȳ )2 ) n/2 ( 1 + 1 (µ Ȳ )2 n 1 s 2 /n ) (n 1+1)/2 Student t n 1 (Ȳ,s2 /n) location Ȳ, df = n-1, scale s2 /n) Bayesian Inference in a Normal Population p.8/18
Standard Student t Standardize X t δ (l, S) by subtracting location and dividing by square root of the scale: X l S t δ (0, 1) (new location 0 and scale 1) Use rt, qt, pt, dt in R µ Ȳ s/ n t n 1(0, 1) Bayesian Inference in a Normal Population p.9/18
Proper Conjugate Priors Under the Normal-Gamma Prior (proper prior distributions): Find the (conditional) posterior distribution µ φ Find the (marginal) posterior distribution of φ Find the (marginal) posterior distribution of µ Hint: Expand quadratics in µ to read off the posterior precision p n and mean m n then complete the square and factor 1 2 (p nµ 2 2p n m n µ + p n m 2 n) = 1 2 p n(µ m n ) 2 See page 80 in Gill Bayesian Inference in a Normal Population p.10/18
Example: SPF A Sunlight Protection Factor (SPF) of 5 means an individual that can tolerate X minutes of sunlight without any sunscreen can tolerate 5X minutes with sunscreen. Tolerance (min) 0 100 200 300 400 PreTreatment During Treatment Bayesian Inference in a Normal Population p.11/18
Pairing A paired design may be more powerful than two sample design because of patient to patient variability. Analysis should take into account pairing which induces dependence between observations use differences use ratios or log(ratios) difference in logs Ratios make more sense given the goals: how much longer can a person be exposed to the sun relative to their baseline. Bayesian Inference in a Normal Population p.12/18
Data Normal Q Q Plot 1.5 1.0 0.5 0.0 0.5 1.0 1.5 Theoretical Quantiles Normal Q Q Plot 1.5 1.0 0.5 0.0 0.5 1.0 1.5 Theoretical Quantiles Bayesian Inference in a Normal Population p.13/18 1.0 1.5 2.0 2.5 3.0 100 200 300 400 Sample Quantiles 1.0 1.5 2.0 2.5 3.0 Sample Quantiles 100 200 300 400
Model for SPF Model Y = log(trt) - log(control) as N(µ, 1/φ) E(log(TRT/CONTROL)) = µ = log(spf) Want distribution of exp µ SPF Summary statistics ybar = 1.998 s2 = 0.525 n = 13 Bayesian Inference in a Normal Population p.14/18
Samples from the Posterior To draw samples of SPF from the posterior distribution: Draw φ Y phi = rgamma(10000, (n-1)/2, rate=(n-1)*s2/2) Draw µ φ,y mu = rnorm(10000, ybar, 1/sqrt(phi*n)) or Draw µ Y directly mu = ybar + rt(10000, n-1)*sqrt(s2/n) transform exp(mu) summarize quantile(exp(mu), c(.025,.5,.975)) Bayesian Inference in a Normal Population p.15/18
Distributions Posterior Distribution of µ Density 0.0 0.5 1.0 1.5 0.5 1.0 1.5 2.0 2.5 3.0 3.5 µ Posterior Distribution of σ Density 0.0 0.5 1.0 1.5 2.0 2.5 0.5 1.0 1.5 2.0 2.5 σ Bayesian Inference in a Normal Population p.16/18
Distribution for SPF Posterior Distribution of SPF Density 0.00 0.05 0.10 0.15 0.20 0.25 5 10 15 exp(µ) 95% Probability Interval 4.76 to 11.31 (equal tail area) 95% HPD Interval 4.47 to 10.89 Bayesian Inference in a Normal Population p.17/18
Exact Posterior Distribution p(4.42 < exp(µ) < 11.06 y) = 0.95) posterior Density 0.00 0.05 0.10 0.15 0.20 0.25 4 6 8 10 12 14 16 exp(µ) Bayesian Inference in a Normal Population p.18/18