EXERCISES FOR SECTION 1 AND 2

Transcription

1 EXERCISES FOR SECTION AND Exercise. (Conditional probability). Suppose that if θ, then y has a normal distribution with mean and standard deviation σ, and if θ, then y has a normal distribution with mean and standard deviation σ. Also, suppose P r(θ ).5 and P r(θ ).5. Two cases for θ: θ P r(θ ).5, y N(, σ ) θ P r(θ ).5, y N(, σ ) (a). For σ, write the formula for the marginal probability density (marginal p.d.f.) for y and sketch it. The marginal p.d.f. of y, p(y), is given by p(y) θ p(y, θ) θ p(y θ)p(θ) p(y θ )p(θ ) + p(y θ )p(θ ) N(y, ) + N(y, ) [ N(y, ) + N(y, ) ].. Figure.: marginal p.d.f. for y p(y) y Bertil Wegmann, Department of Computer and Information Science, Linkping University SE Linkping. bertil.wegmann@liu.se.

2 EXERCISES, SESSION (b). What is P r(θ y ), again supposing σ? p(θ y ) p(θ, y ) p(y ) p(y θ )p(θ ) p(y ). π exp [ ( ) ] π [exp [ ( ) ] + exp [ ( ) ]] + exp [ ], 53 8 (c). Describe how the posterior density of θ changes in shape as σ is increased and as it is decreased. The posterior density is given by p(θ y) p(y θ)p(θ) p(y) The posterior probability of θ is given by Similarly, Thus, p(θ y) exp [ (y θ) ] σ exp [ (y ) ] + exp [ (y ) ]. σ σ + exp [ [(y ) σ (y ) ] ] + exp [ y 3 p(θ y) + exp [ y 3 σ ]. σ p(θ y) p(θ) σ scenarios y < 3, σ p(θ y). σ ]. y > 3, σ p(θ y). Exercise.6 (Conditional probability). approximately /5 of all births are fraternal twins and /3 of births are identical twins. Elvis Presley had a twin brother (who died at birth). What is the probability that Elvis was an identical twin? (You may approximate the probability of a boy or girl birth as.)

3 EXERCISES, SESSION 3 Events: A Elvis had a twin brother, B Elvis was an identical twin, C Elvis was a fraternal twin. P (boy) P (girl) gives This gives, P (B A) P (B) 3 6, P (C) 5 5. P (A B) P (A) P (A B)P (B) P (A B)P (B) + P (A C)P (C) Exercise. (Posterior inference). suppose there is Beta(4, 4) prior distribution on the probability θ that a coin will yield a head when spun in a specified manner. The coin is independently spun ten times, and heads appear fewer than 3 times. You are not told how many heads were seen, only that the number is less than 3. Calculate your exact posterior density (up to a proportionality constant) for θ and sketch it. The prior distribution for θ is p(θ) θ 3 ( θ) 3 Let y total number of heads in n spuns. Then, We have that so that This gives y θ Bin(n, θ) p(y θ) p(y < 3 θ) p(θ y < 3) ( ) θ y ( θ) y. y p(y θ) ( θ) + θ( θ) θ ( θ) 8, y p(θ, y < 3) p(y < 3) p(y < 3 θ)p(θ) p(y < 3) p(y < 3 θ)p(θ). p(θ y < 3) θ 3 ( θ) 3 + θ 4 ( θ) + 45θ 5 ( θ). 4 x 3 Figure.: posterior density of θ given y < p(θ y<3) θ

4 4 EXERCISES, SESSION Exercise.5 (posterior distribution as a compromise between prior information and data). Let y be the number of heads in n spins of a coin, whose probability of heads is θ. (a). If your prior distribution for θ is uniform on the range [, ], derive your prior predictive distribution for y, P r(y k) for each k,,..., n. P r(y k θ)dθ, y Bin(n, θ) p(y k θ) ( n k) θ y ( θ) n y, so ( ) n p(y k) θ k ( θ) n k dθ. k If θ Beta(k +, n k + ), then This gives that p(y k) p(θ) Γ(n + ) Γ(k + )Γ(n k + ) ( ) n Γ(k + )Γ(n k + ) k Γ(n + ) θ k ( θ) n k dθ. n! k!(n k)! k!(n k)! (n + )! n +. (b). Suppose you assign a Beta(α, β) prior distribution for θ, and then you observe y heads out of n spins. Show algebraically that your posterior mean of θ always lies α between your prior mean,, and the observed relative frequency of heads, y. α+β n This gives Hence, θ Beta(α, β) p(θ) θ α ( θ) β y θ Bin(n, θ) p(y θ) θ y ( θ) n y. p(θ y) θ y+α ( θ) n y+β Beta(θ α + y, β + n y). E[θ y] α + y α + β + n α ( ) α + β α + β + y ( α + β + n n n α + β + n ). (c). Show that, if the prior distribution on θ is uniform, the posterior variance of θ is always less than the prior variance.

5 We have that This gives V ar(θ y) EXERCISES, SESSION 5 p(θ y) θ y ( θ) n y Beta(θ y +, n y + ). (y + )(n y + ) (n + ) (n + 3) ( n + )(n n + ) (n + ) (n + 3) 6 < V ar(θ). (d). Give an example of a Beta(α, β) prior distribution and data y, n, in which the posterior variance of θ is higher than the prior variance. p(θ) θ α ( θ) β and V ar(θ) αβ (α + β) (α + β + ) p(θ y) θ y+α ( θ) n y+β Beta(θ α + y, β + n y) and V ar(θ y) (α + y)(β + n y) (α + β + n) (α + β + n + ). For example, n, y, α, β 9 gives V ar(θ) 9 < 936 V ar(θ y). Exercise.8 (normal distribution with unknown mean). a random sample of n students is drawn from a large population, and their weights are measured. The average weight of the n sampled students is ȳ 5 pounds. Assume the weights in the population are normally distributed with unknown mean θ and known standard deviation pounds. Suppose your prior distribution for θ is normal with mean 8 and standard deviation 4. (a). Give your posterior distribution for θ. (Your answer will be a function of n.) Equation (.) (.) gives ȳ θ N ) (θ, n θ N(8, 4 ). p(θ ȳ) N(θ µ n, τ n),

6 6 EXERCISES, SESSION where and µ n τ n µ τ + n ȳ σ τ τ + n σ, + n σ. (b). A new student is sampled at random from the same population and has a weight of ỹ pounds. Give a posterior predictive distribution for ỹ. The posterior predictive distribution is given by (page 47-48) [ p(ỹ ȳ) p(ỹ θ)p(θ ȳ)dθ exp ] [ (ỹ θ) exp (θ µ σ τn n ) ]dθ. This gives that ỹ y is normal distributed with and E(ỹ ȳ) E[E(ỹ θ, ȳ) ȳ] E[θ ȳ] µ n V ar[ỹ ȳ] E[V ar(ỹ θ, ȳ) ȳ] + V ar[e(ỹ θ, ȳ) ȳ] E[σ ȳ] + V ar(θ ȳ) σ + τ n. (c). For n, give a 95% posterior interval for θ and a 95% predictive interval for ỹ. A 95% posterior interval for θ is given by E[θ ȳ 5] ±, 96 V ar(θ ȳ 5). This gives the interval to 5, 73 ±, 96 39, 4. A 95% posterior predictive interval for ỹ is given by E(ỹ ȳ) ±, 96 V ar[ỹ ȳ]. This gives the interval to 5, 73 ±, , 4. (d). For n, give a 95% posterior interval for θ and a 95% predictive interval for ỹ.

7 EXERCISES, SESSION 7 The 95% posterior interval for θ becomes [46, 54]. The 95% posterior predictive interval for ỹ becomes [, 89]. Exercise.. Figure for part (a):.4 Figur., normaliserad posterior funktion p(theta y). normaliserad posterior funktion theta