Homework 1: Solution

Size: px

Start display at page:

Download "Homework 1: Solution"

Julius Wade
5 years ago
Views:

1 University of California, Santa Cruz Department of Applied Mathematics and Statistics Baskin School of Engineering Classical and Bayesian Inference - AMS 3 Homework : Solution. Suppose that the number of defects on a roll of magnetic recording tape has a Poisson distribution for which the mean θ is unknown. [5 pts.] (a) Write the statistical model. [5 pts.] Let X be the r.v. describing the number of defects on a roll of magnetic recording tape. The p.f. of the statistical model is f(x θ) = e θ θ x /x!, x = 0,,,..., and θ Ω = R +. (b) Assume that the mean number of defects on a roll of magnetic recording tape is either.0 or.5, and the prior p.f. of θ is ξ(.0) = 0.4 and ξ(.5) = 0.6. If a roll of tape selected at random is found to have three defects, what is the posterior p.f. of θ? [5 pts.] Here Ω =,.5, and ξ(.0) = 0.4 and ξ(.5) = 0.6. The marginal p.f. of X is given by g(x) = θ Ω f(x θ)ξ(θ) = e () x x! e.5 (.5) x x! 0.6. Then the posterior p.f. of θ given X = 3 is ( ) ( e e () 3 ξ( X = 3) = 0.4 / e.5 (.5) 3 ) 0.6 = 0.457, ξ(.5 X = 3) = ξ( X = 3) = (c) Under the same assumptions of (b), find the conditional p.f. of X given X = 3. [5 pts.] The conditional p.f. of X given X = 3 is given by f(x X = 3) = θ Ω Note that, x =0 f(x X = 3) =. f(x θ)ξ(θ X = 3) = e () x x! e.5 (.5) x x! Suppose that a single observation X is to be taken from the uniform distribution on the interval [θ /, θ + /], where the value of θ is unknown, and the prior distribution of θ is the uniform distribution on the interval [0, 0]. If the observed value of X is, what is the posterior distribution of θ? [5 pts.] Since θ / < x < θ + /, it follows that x / < θ < x + /. Additionally, 0 < θ < 0. Therefore, maxx /, 0 < θ < minx + /, 0.

2 Since ξ(θ x) f(x θ)ξ(θ) = /0, it follows that the posterior distribution of θ is constant on the interval (maxx /, 0, minx + /, 0). Finally, since X = is observed, we have that the posterior distribution of θ given X = is a uniform distribution on the interval (.5,.5). 3. Automobile engines emit a number of undesirable pollutants when they burn gasoline. One class of polutants consists of the oxides of nitrogen. Suppose that X,..., X n describe the emissions of oxides of nitrogen, that can be described by a normal distribution with unknown mean θ and known variance σ. Also, assume that a normal prior p.d.f. for θ is considered with mean µ 0 and variance v 0. [5 pts.] (a) Show that the posterior distribution of θ is a normal distribution with mean and variance given by µ = σ µ 0 +nv0 xn, and v σ +nv0 = σ v0. [5 pts.] σ +nv0 n ξ(θ x) σ (x i θ) i= ( n σ + v0 σ v0 σ +nv0 (θ µ v0 0 ), ) θ ( n i= x i/σ + µ 0 /v ( ) 0) θ, ( θ σ µ 0 + nv 0x n σ + nv 0 n σ + v 0 ). Therefore, the posterior distribution of θ given X = x,..., X n = x n is a normal distribution with mean µ and v.

3 (b) Find the distribution of X n+ given X = x,..., X n = x n. [5 pts.] f(x n+ x) = = = πσ = πσ = f(x n+ θ)ξ(θ x)dθ, πσ πv πv σ (x n+ θ) x n+ σ µ, v (θ x n+ θ) (θ µ θ) σ v x n+ σ µ, v ( (θ µ) dθ, πv v dθ, ) θ (x n+/σ + µ /v) θ ( ) σ + v dθ, σ +v. π(σ + v) (σ + v) (x n+ µ ) Therefore, X n+ given X = x,..., X n = x n has a normal distribution with mean µ and variance σ + v. (c) Assume that σ = 0.5, µ 0 =.0, and v 0 =.0. In a random sample of n = 46 engines, the average of the emissions of oxides of nitrogen was.39. Compute P (X 47 <.5 x,..., x n ). [5 pts.] With the given information, µ =.3367 and v = , hence P (X 47 <.5 x,..., x n ) = Φ(Z < ( )/ ) = , where Z N(0, ) and Φ denotes the cumulative distribution function. 4. Suppose that X,..., X n form a random sample from a distribution for which the p.d.f. f(x θ) is f(x θ) = θe θx, for x > 0, and f(x θ) = 0, otherwise. Suppose that the value of the parameter θ is unknown (θ > 0), and the prior distribution of θ is the gamma distribution with parameters α and β (α > 0 and β > 0). [5 pts.] If Y a, b Gamma(a, b), then f(y a, b) = ba Γ(a) ya e by, E(Y a, b) = a/b and V ar(y a, b) = a/b. (a) Write the statistical model. [5 pts.] Let X be the r.v. of interest. The p.d.f. of the statistical model is f(x θ) = θe θx, x > 0, and θ Ω = R +. (b) Determine the mean and the variance of θ after observing X = x,..., X n = x n. [5 pts.] First we find the posterior distribution of θ given the observations and then its mean and variance. ξ(θ x) θ n e θ n i= x i θ α e βθ θ α+n e (β+ n i= x i)θ. 3

4 Therefore θ x Gamma(α + n, β + n i= x i). So, E(θ x) = (α + n)/(β + n i= x i) and V ar(θ x) = (α + n)/(β + n i= x i). (c) If α = β = 0, is the prior p.d.f. of θ improper? Is the posterior p.d.f. of θ improper? [5 pts.] If α = β = 0, then ξ(θ) θ and θ dθ = ln(θ) 0 θ=0 =, so the prior p.d.f. of θ given the observations is improper. The posterior distribution of θ given the observations is not improper as long as n i= x i > 0 and n. 5. Show that the family of beta distributions with parameters a, b > 0, is a conjugate family of prior distributions for samples from a negative binomial distribution with a known value of the parameter r and an unknown value of the parameter p (0 < p < ). [5 pts.] If X p NegativeBinomial(r, p), then P (X = x p) p r ( p) x. The posterior distribution of p given the observations is ξ(p x) n i= p r ( p) x i p a ( p) b = p a+nr ( p) b+ n i= x i. So, we recognize the kernel of a beta distribution with parameters a + nr and b + n i= x i, and so, the family of beta distributions is a conjugate family of prior distributions for samples from a negative binomial distribution 6. Suppose that the time in minutes required to serve a customer at a certain facility has an onential distribution for which the value of the parameter θ is unknown and that the prior distribution of θ is a gamma distribution. [5 pts.] If Y a, b Gamma(a, b), then f(y a, b) = ba Γ(a) ya e by, E(Y a, b) = a/b and V ar(y a, b) = a/b. (a) The average time required to serve a random sample of 0 customers is observed to be 3.8 minutes. If the prior distribution of θ has mean equal to 0. and the standard deviation equal to, what is the posterior distribution of θ?[5 pts.] Let X be a r.v. describing the time in minutes required to serve a customer at a certain facility. Then X θ (θ), and we are assuming that θ a, b Gamma(a, b). We know that the gamma prior is conjugate for the onential distribution, therefore θ x Gamma(a + n, b + n i= x i) Since a/b = 0. and a/b =, it follows that a = 0.04 and b = 0.. Additionally, n = 0 and n i= x i = = 76, so we conclude that, θ x Gamma(0.04, 76.). (b) For a distribution with mean µ 0 and standard deviation σ > 0, the coefficient of variation of the distribution is defined as σ/ µ. Suppose that the coefficient of variation of the prior gamma distribution of θ is. What is the smallest number of customers that must be observed in order to reduce the coefficient of variation of the posterior distribution to 0. or less? [5 pts.] 4

5 Since the gamma prior is conjugate for the onential distribution, we have that θ x Gamma(a + n, b + n i= x i). a/b Since the coefficient of variation of the prior distribution is, it follows that = a/b a =, therefore a = 0.5. Now, for the posterior distribution of θ given the observations, we have that the coefficient of variation is given by a+n. Therefore, / n 0. implies that at least 00 customers must be observed. (c) Suppose now that the coefficient of variation of the prior gamma distribution of θ is 0.. What is the smallest number of customers that must be observed in order to reduce the coefficient of variation of the posterior distribution to 0.? [5 pts.] Since the coefficient of variation of the prior distribution is 0., it follows that a = Now, / n 0. implies that at least 75 customers must be observed. 5

Classical and Bayesian inference

Classical and Bayesian inference AMS 132 January 18, 2018 Claudia Wehrhahn (UCSC) Classical and Bayesian inference January 18, 2018 1 / 9 Sampling from a Bernoulli Distribution Theorem (Beta-Bernoulli