MAS3301 Bayesian Statistics Problems 2 and Solutions

Size: px

Start display at page:

Download "MAS3301 Bayesian Statistics Problems 2 and Solutions"

Morgan Robinson
5 years ago
Views:

1 MAS33 Bayesian Statistics Problems Solutions Semester 8-9 Problems Useful integrals: In solving these problems you might find the following useful Gamma functions: Let a b be positive Then where Γ(a) If a is a positive integer then Γ(a) (a )! x a e bx dx Γ(a) b a Beta functions: Let a b be positive Then x a e x dx (a )Γ(a ) x a ( x) b dx Γ(a)Γ(b) Γ(a + b) We are interested in the mean, λ, of a Poisson distribution We have a prior distribution for λ with density f () (λ) (λ ) k ( + λ)e λ (λ > ) (a) i Find the value of k ii Find the prior mean of λ iii Find the prior stard deviation of λ (b) We observe data x,, x n where, given λ, these are independent observations from the Poisson(λ) distribution i Find the likelihood ii Find the posterior density of λ iii Find the posterior mean of λ We are interested in the parameter, θ, of a binomial(n, θ) distribution We have a prior distribution for θ with density f () (θ) k θ ( θ) + θ( θ) ( < θ < ) (otherwise) (a) i Find the value of k ii Find the prior mean of θ iii Find the prior stard deviation of θ (b) We observe x, an observation from the binomial(n, θ) distribution

2 i Find the likelihood ii Find the posterior density of θ iii Find the posterior mean of θ 3 We are interested in the parameter θ, of a binomial(n, θ) distribution We have a prior distribution for θ with density f () (θ) k θ ( θ) 3 ( < θ < ) (otherwise) (a) i Find the value of k ii Find the prior mean of θ iii Find the prior stard deviation of θ (b) We observe x, an observation from the binomial(n, θ) distribution i Find the likelihood ii Find the posterior density of θ iii Find the posterior mean of θ 4 In a manufacturing process packages are made to a nominal weight of kg All underweight packages are rejected but the remaining packages may be slightly overweight It is believed that the excess weight X, in g, has a continuous uniform distribution on (, θ) but the value of θ is unknown Our prior density for θ is (a) i Find the value of k ii Find the prior median of θ (θ < ) f () (θ) k / ( θ < ) k θ ( θ < ) (b) We observe packages their excess weights, in g, are as follows Assume that these are independent observations, given θ i Find the likelihood ii Find a function h(θ) such that the posterior density of θ is f () (θ) k h(θ), where k is a constant iii Evaluate the constant k (Note that it is a very large number but you should be able to do the evaluation using a calculator) 5 Repeat the analysis of the Chester Road example in section 63, using the same likelihood but with the following prior density (a) Find the value of k f () (λ) k [ + (8λ) ] ( < λ < ) (otherwise) (b) Use numerical methods in R to do the following i Find the posterior density plot a graph showing both the prior posterior densities ii Find the posterior mean stard deviation Note: For the numerical calculations the plot in part (b) I suggest that you use a range λ When plotting the graph, it is easiest to plot the posterior first as this will determine the length of the vertical axis The value of k can be found analytically If you do use numerical integration to find it, you will need a much wider range of values of λ

3 6 We are interested in the parameter λ of a Poisson(λ) distribution We have a prior distribution for λ with density f () (λ) (λ < ) k λ 3 e λ (λ ) (a) i Find the value of k ii Find the prior mean of λ iii Find the prior stard deviation of λ (b) We observe x,, x n which are independent observations from the Poisson(λ) distribution i Find the likelihood function ii Find the posterior density of λ iii Find the posterior mean of λ 7 In a fruit packaging factory apples are examined to see whether they are blemished A sample of n apples is examined, given the value of a parameter θ, representing the proportion of aples which are blemished, we regard x, the number of blemished apples in the sample, as an obervation from the binomial(n, θ) distribution The value of θ is unknown Our prior density for θ is f () (θ) k (θ( θ) 3 + ) ( θ ) (otherwise) (a) i Show that, for θ, the prior density can be written as f () (θ) Γ(6) Γ()Γ(4) θ ( θ) 4 + Γ() Γ()Γ() θ ( θ) ii Find the prior mean of θ iii Find the prior stard deviation of θ (b) We observe n apples x 4 Homework i Find the likelihood function ii Find the posterior density of θ iii Find the posterior mean of θ iv Use R to plot a graph showing both the prior posterior densities of θ (Hint: It is easier to get the vertical axis right if you plot the posterior density then superimpose the prior density, rather than the other way round) Solutions to Questions 6, 7 of Problems are to be submitted in the Homework Letterbox no later than 4pm on Monday February 3rd 3

4 Solutions (a) i ii iii k / E (λ) E (λ ) f () (λ) dλ k e λ dλ + k + k λf () (λ) dλ λe λ dλ λ f () (λ) dλ λ e λ dλ + λe λ dλ λ e λ dλ λ 3 e λ dλ So ) 6 9 (b) i Likelihood where L var (λ) 8 ( 3 stddev (λ) n e λ λ xi i ii Posterior density proportional to x i! e nλ λ xi xi! S n x i i e nλ λ S xi! The posterior density is f () (λ)l ( + λ)e λ e nλ λ S e (n+)λ λ S + e (n+)λ λ S+ f () (λ) k e (n+)λ λ S + e (n+)λ λ S+ where Γ(S + ) f () Γ(S + ) (λ) dλ k + (n + ) S+ (n + ) S+ Γ(S + ) Γ(S + ) k + (n + ) S+ (n + ) S+ Γ(S + ) f () Γ(S + ) (λ) + (n + ) S+ (n + ) S+ e (n+)λ λ S + e (n+)λ λ S+ 4

5 (a) i iii Posterior mean E (λ) k 6 ii Prior mean iii λf () (λ) dλ k λ S+ e (n+)λ dλ + Γ(S + ) Γ(S + 3) k + (n + ) S+ (n + ) S+3 (S+) (n+) + (S+)(S+) (n+) + (S+) (n+) f () (θ) dθ k θ ( θ) dθ + k Γ(3)Γ() Γ(5) k Γ(3)Γ() Γ(5) λ S+ e (n+)λ dλ + Γ()Γ(3) Γ(5) k!! 4! k 6 E (θ) θf () (θ) dθ k θ 3 ( θ) dθ + Γ(4)Γ() k + Γ(3)Γ(3) Γ(6) Γ(6) 3!! +!! k k 5! E (θ ) θ f () (θ) dθ k θ 4 ( θ) dθ + k Γ(5)Γ() k 4!! + 3!! 6! var (θ) 3 + Γ(4)Γ(3) θ( θ) dθ θ 3 ( θ) dθ 3 ( ) 6 5 stddev (θ) 36 θ ( θ) dθ (b) i Likelihood L ( n x ) θ x ( θ) n x ii Posterior density f () (θ) proportional to f () (θ)l f () (θ) k θ ( θ) + θ( θ) θ x ( θ) n x k θ x+ ( θ) n x+ + θ x+ ( θ) n x+ 5

6 Now f () (θ) dθ k θ x+ ( θ) n x+ dθ + k Γ(x + 3)Γ(n x + ) + θ x+ ( θ) n x+ dθ Γ(x + )Γ(n x + 3) Γ(x + 3)Γ(n x + ) k + f () (θ) Γ(x + 3)Γ(n x + ) + Γ(x + )Γ(n x + 3) Γ(x + )Γ(n x + 3) θ x+ ( θ) n x+ + θ x+ ( θ) n x+ iii Posterior mean E (θ) θf () (θ) dθ k θ x+3 ( θ) n x+ dθ + θ x+ ( θ) n x+ dθ Γ(x + 4)Γ(n x + ) Γ(x + 3)Γ(n x + 3) k + Γ(n + 6) Γ(n + 6) Γ(x+4)Γ(n x+) Γ(n+6) + Γ(x+3)Γ(n x+3) Γ(n+6) Γ(x+3)Γ(n x+) Γ(n+5) + Γ(x+)Γ(n x+3) Γ(n+5) n + 5 n + 5 Γ(x + 4)Γ(n x + ) + Γ(x + 3)Γ(n x + 3) Γ(x + 3)Γ(n x + ) + Γ(x + )Γ(n x + 3) Γ(x + 4) + Γ(x + 3)(n x + ) Γ(x + 3) + Γ(x + )(n x + ) (x + 3)(x + ) + (x + )(n x + ) n + 5 (x + ) + (n x + ) ( ) x + (x + 3) + (n x + ) n + 5 (x + ) + (n x + ) x + n + 4 6

7 3 (a) i ii Prior mean f () (θ) dθ k k E (θ) θ ( θ) 3 Γ(3)Γ(4) dθ k Γ(3)Γ(4) 6!!3! Γ(4)Γ(4) Γ(8) θf () (θ) dθ k θ 3 ( θ) 3 dθ Γ(3)Γ(4) iii E (θ ) Γ(5)Γ(4) Γ(9) θ f () (θ) dθ k θ 4 ( θ) 3 dθ Γ(3)Γ(4) (b) i Likelihood ii Posterior density Now var (θ) 3 4 stddev (θ) L(θ) ( n x ( ) ) θ x ( θ) n x f () (θ) f () (θ)l(θ) k θ x+ ( θ) n x+3 f () (θ) dθ k f () (θ) iii Posterior mean θ x+ ( θ) n x+3 Γ(x + 3)Γ(n x + 4) dθn k Γ(n + 7) k Γ(n + 7) Γ(x + 3)Γ(n x + 4) Γ(n + 7) Γ(x + 3)Γ(n x + 4) θx+ ( θ) n x+3 ( < θ < ) E (θ) θf () (θ) dθ k θ x+3 ( θ) n x+3 dθ Γ(n + 7) Γ(x + 3)Γ(n x + 4) Γ(x + 4)Γ(n x + 4) Γ(n + 8) x + 3 n

8 6 (a) i Value of k : λ 3 e λ dλ λ 4 e λ dλ Γ(4) 3! 6 k 6 ii Prior mean: ( mark) E (λ) λk λ 3 e λ dλ k λ 5 e λ dλ k Γ(5) 4! 3! 4 ( mark) iii Prior stddev: E (λ ) λ k λ 3 e λ dλ k λ 6 e λ dλ k Γ(6) 5! 3! var (λ) E (λ ) [E (λ)] 6 4 prior sd is 4 (b) i Likelihood: ( marks) n e λ λ xi L x i! i e nλ λ xi xi! ii Posterior density proportional to ( mark) λ 3 e λ e nλ λ xi That is Now the posterior density is λ xi+4 e (n+)λ λ a e bλ dλ Γ(a) b a xi+4 (n + ) Γ( x i + 4) λ xi+4 e (n+)λ ( marks) 8

9 iii To find the posterior mean, increase the power of λ by integrate Posterior mean is (n + ) xi+4 Γ( Γ( x i + 5) x i + 4) (n + ) xi + 4 xi+5 n + 7 (a) i The expression given is proportional to the prior density since Now we only need to show that this follows since Γ(6) Γ()Γ(4) Γ() Γ()Γ() Γ(6) Γ()Γ(4) θ ( θ) 4 + 5! 3! 3 Γ() Γ()Γ() θ ( θ) dθ ( mark) ii Prior mean: E (θ) θf () (θ) dθ θ ( θ) 4 dθ Γ()Γ(4) Γ(6) θ ( θ) dθ Γ()Γ() Γ() ( mark) Γ(6) Γ()Γ(4) θ3 ( θ) 4 + Γ() Γ()Γ() θ ( θ) dθ Γ(6) Γ(3)Γ(4) + Γ() Γ()Γ() Γ()Γ(4) Γ()Γ() Γ(3) ( marks) iii Prior std dev: E (θ ) θ f () (θ) dθ Γ(6) Γ()Γ(4) θ4 ( θ) 4 + Γ() Γ()Γ() θ3 ( θ) Γ(6) Γ(4)Γ(4) + Γ() Γ(3)Γ() Γ()Γ(4) Γ(8) Γ()Γ() Γ(4) so the prior std dev is var (θ) 5 ( ) dθ

10 ( marks) (b) i Likelihood: L ( 4 ) θ 4 ( θ) 6 ii Posterior density: ( mark) Posterior propto P rior Likelihood Γ(6) f () (θ) k Γ()Γ(4) θ6 ( θ) + Γ() Γ()Γ() θ5 ( θ) 7 Γ(6) f () Γ(6)Γ() (θ) dθ k + Γ() Γ(5) Γ()Γ(4) Γ(6) Γ()Γ() Γ() k k k k 33 k k Posterior density: f () (θ) 9 θ 6 ( θ) + θ 5 ( θ) 7 iii Posterior mean: ( marks) E (θ) θf () (θ) dθ 9 θ 7 ( θ) + θ 6 ( θ) 7 dθ 9 Γ() + Γ(6) Γ(3) ( marks) iv Plot: suitable R comms: > theta<-seq(,,) > prior<-5*(*theta*((-theta)^3)+) > post<-9*(*(theta^5)*((-theta)^9)+(theta^4)*((-theta)^6)) > plot(theta,post,type"l",xlabexpression(theta),ylab"density") > lines(theta,prior,lty) The plot is shown in Figure ( marks)

11 Density θ Figure : Prior (dashes) posterior (solid line) density functions for θ

MAS3301 Bayesian Statistics

MAS3301 Bayesian Statistics MAS331 Bayesian Statistics M. Farrow School of Mathematics and Statistics Newcastle University Semester 2, 28-9 1 9 Conjugate Priors II: More uses of the beta distribution 9.1 Geometric observations 9.1.1