Chapter 4 HOMEWORK ASSIGNMENTS. 4.1 Homework #1

Chapter 4 HOMEWORK ASSIGNMENTS These homeworks may be modified as the semester progresses. It is your responsibility to keep up to date with the correctly assigned homeworks. There may be some errors in the statements of these problems, due to typographical and conceptual errors on my part. I will give 1% addition to the first exam scores for all those finding errors, under the following conditions: (i) typos are fixed, stated properly and then worked out; (ii) conceptual errors on my part are explained by you. 4.1 Homework #1 Problem #1 Problem 6.1 in Casella & Berger (p. 280). Problem #2 Problem 6.6 in Casella & Berger (p. 280). (a) What are the sufficient statistics for α and β? (b) If α is known, show that the gamma density is a member of the class of exponential families. (c) If β is known, is the gamma density a member of the class of exponential families? Why or why not? (d) With neither α nor β known, is the gamma density a member of the multiple parameter exponential family? Why or why not? Problem #3 Suppose that x 1,..., x n are fixed constants. distributed with mean β 0 + β 1 x i and variance σ 2. (a) What are the sufficient statistics for (β 0,β 1,σ 2 )? Suppose further that Y i is normally Problem #4 The Rayleigh family has the density f(x, θ) =2(x/θ 2 )exp ( x 2 /θ 2),x>0,θ >0. Use the fact that this is an exponential family to compute the mean, variance and 3rd and 4th moments of X 2,whereXis Rayleigh. 13

14 CHAPTER 4. HOMEWORK ASSIGNMENTS Problem #5 Suppose I have a sample X 1,..., X n from the normal distribution with mean θ and variance θ. LetXbe the sample mean, and s 2 be the sample variance. Remember that X and s 2 are independent. (a) For any 0 α 1, compute the mean and variance of the statistic T (α) =αx+(1 α)s 2. (b) Compute the limiting distribution of T (α), i.e., what does n 1/2 {T (α) θ) converge to in distribution. (c) Is there a unique best value of α as n? Problem #6 Suppose that we have a sample X 1,..., X n from the density f(x, θ) = Find a minimal sufficient statistic for θ. x!γ(θ)γ(x + θ) 2 x+θ. Problem #7 Work problem 6.20 in Casella and Berger (page 280). A function T (X) isacomplete sufficient statistic if E [g{t (X) θ}] = 0 for all θ and for all g = T (X) 0.

4.2. HOMEWORK #2 15 4.2 Homework #2 Problem #1 Find the mle of θ in the Rayleigh family of Homework #1. Problem #2 Find the mle s of (β 0,β 1,σ 2 ) in the linear regression problem of Homework #1. Problem #3 Suppose that X 1,..., X n are a sample with mass function Find the mle of θ. pr(x = k) = (k 2)(k 1) (1 θ) k 3 θ 3. 2 Problem #4 Suppose that X 1,..., X n are i.i.d. uniform on the interval [θ, θ 2 ], where θ>1. (a) Show that a method of moments estimator of θ is ( θ(mm)= 8n 1 n i=1 1/2 X i +1) 1 /2. (b) Find the mle for θ. (c) By combining the central limit theorem and the delta method (Taylor-Slutsky), compute the limiting distribution of θ(mm). Problem #5 Work Problem 7.7 in Casella & Berger (page 332). Problem #6 Work Problem 7.12 of Casella & Berger (page 333). Problem #7 Suppose that z 1,..., z n are fixed constants, and that the responses Y 1,..., Y n are independent and normally distributed with mean z i β and variance σ 2 v(z i ), where v(z i )areknown constants. (a) Compute the mle of the parameters. (b) Compute the mean and variance of β. Problem #8 Suppose that z 1,..., z n are fixed constants, and that the responses Y 1,..., Y n are independently distributed according to a gamma distribution with mean exp z i β and variance σ 2 exp 2z i β. (a) It turns out that there is a function ψ(y,z,β) such that the mle for β solves n i=1 ψ(y i,z i,β)= 0. What is ψ( )? (b) What is the mle for σ 2?

16 CHAPTER 4. HOMEWORK ASSIGNMENTS 4.3 Homework #3 Problem #1 If X Poisson(θ), show that X is UMVUE for θ. Problem #2 If X Binomial(n, θ), show that there exists no unbiased estimator of the odds ratio g(θ) = θ 1 θ. HINT: Suppose there does exist an S(X) which is unbiased. Write out E θ {S(X)} and then find a contradiction. Problem #3 Suppose that X has the mass function Pr(X = k θ) =θ(1 θ) k, k =0,1,2,.../ Find the mle for θ from a sample of size n, and discuss its properties, namely: (a) mean (b) variance (c) is it UMVUE? Problem #4 Suppose that (z 1,..., z n ) are fixed constants, and that for i =1,..., n, X i is normally distributed with mean z i and variance θz 2 i. Find the mle for θ from a sample of size n, and discuss its properties, namely: (a) mean (b) variance (c) is it UMVUE? HINT: If Z Normal(0, 1), E(Z 3 )=0andE(Z 4 )=3. Problem #5 Work problem 7.56 in Casella & Berger (page 341).

4.4. HOMEWORK #4 17 4.4 Homework #4 Problem #1 Find the Fisher information for the Rayleigh family. Problem #2 If X 1,..., X n are i.i.d. and normally distributed with mean equal to its variance, find the mle and the Fisher information for θ. Problem #3 Let X be Poisson(λ x )andlety be independent of X and distributed as a Poisson(λ y ). Define θ = λ x /(λ x + λ y )andξ=λ x +λ y. (a) Suppose that θ is known. Show that T = X + Y is sufficient for ξ. (b) Compute the conditional distribution of X given T. (c) Conditioning on T, find the UMVUE for θ. I want you to show that this is really a conditional UMVUE, so I want you to cite theorems from class to justify your steps. Problem #4 Suppose I have a sample X 1,..., X n from the normal distribution with mean θ and variance θ 2.LetXbe the sample mean, and s 2 bethesamplevariance. (a) For any 0 α 1, compute the mean and variance of the statistic (b) Compute the limiting distribution of T (α). T (α) =αx 2 +(1 α)s 2. (c) Compute the limiting distribution of T (α). Problem #5 Work Problem 7.55(a) in Casella & Berger (p. 340). Hint #1: An unbiased estimator is I(X 1 =0),whereI( ) is the indicator function. Hint #2: what is the distribution of X 1 given the sufficient statistic? Problem #6 Suppose that (z 1,..., z n ) are fixed constants, and that for i =1,..., n, X i is normally distributed with mean z i and variance θz 2 i. Find the mle for θ from a sample of size n. Does the mle achieve the Fisher information bound? Does this in two ways: (a) by direct calculation (b) by using properties of OPEF s. Problem #7 Suppose that X 1,..., X n follow the Weibull model with density f(x λ, κ) =κλ(λx) κ 1 exp { (λy) κ }. (a) What equations must be solved to compute the mle? (b) Show that the mle of (λ, κ) is unique.

18 CHAPTER 4. HOMEWORK ASSIGNMENTS 4.5 Homework #5 Problem #1 In the Rayleigh family, show directly using the weak law of large numbers that the mle is consistent. Also show it is consistent using the general theory in class about consistency of mle s in exponential families. Problem #2 What is the asymptotic limit distribution of the mle in the Rayleigh family? Problem #3 Let X 1,..., X n be i.i.d. negative exponential with mean θ. (a) Find the mle for θ. (b) Find the mle for pr θ (X>t 0 ). (c) Prove that the mle for pr θ (X>t 0 ) is consistent. (d) Compute the limit distribution for the mle of pr θ (X>t 0 ). Problem #4 Let X 1,..., X n be i.i.d. Poisson with mean θ. It moment generating function is known to be E {exp(tx)} =exp[θ{exp(t) 1}]. (a) Show that E(X θ) 2 = θ, E(X θ) 3 = θ and E(X θ) 4 = θ +3θ 2. Imayhavemadean error here, so correct it if I have. (b) Compute the limiting distribution for the mle of θ. (c) The sample variance s 2 is unbiased for θ. Compute its limiting distribution. (d) Compare the limiting variances you found in parts (b) and (c). Problem #5 Let X 1,..., X n be i.i.d. from a one parameter exponential family in canonical form, with the density function p(x θ) = S(x)exp {θx + d(θ)}. (a) Show that if the mle exists, it must satisfy X = {E θ (X)} θ= θ. (b) Cite a theorem from class showing that the mle must be consistent. Problem #6 Suppose that X 1,..., X n follow the Weibull model with density f(x λ, κ) =κλ(λx) κ 1 exp { (λy) κ }. (a) Suppose that κ is known. What is the limit distribution for the mle of λ?

4.5. HOMEWORK #5 19 Problem #7 In many problems, time to event data would naturally be modeled via a negative exponential density. However, in some of these problems, there is the worry that there is a certain probability that the event will never occur. Such a model has the distribution (not density) function F (x, θ, κ) =κ+(1 κ){1 exp(x/θ)}, for0 x. Note that the value of x = has a positive probability. This model is not in the form of an exponential family, and in fact the data do not even have a density function. (a) interpret κ as a cure rate. (b) Show that the likelihood function for this model is κ I(x= ) {(1 κ)exp( x/θ)/θ} I(x< ). (c) Show that E(X) = and hence standard method of moments will not work. (d) Compute the mle for κ and θ. (e) Compute the limit distribution for the mle of κ.

20 CHAPTER 4. HOMEWORK ASSIGNMENTS 4.6 Homework #6 Problem #1 Suppose that, given θ, X is Poisson with mean θ. Letθhave a negative exponential prior distribution with mean θ 0. Let the loss function be L(θ, t) =(t θ) 2 /θ. (a) Show that the posterior distribution of θ is a gamma random variable. (b) What is the Bayes estimator of θ? Hint: you have been told a characterization of Bayes estimators in terms of minimizing a certain function. You should try to do this minimization explicitly here. Problem #2 Let X be Binomial(n, θ 1 )andlety be Binomial(n, θ 2 ). Suppose the loss function is L(θ 1,θ 2,t)=(θ 1 θ 2 t) 2.Letθ 1 and θ 2 have independent prior beta-distributions with parameters (α, β). Find the Bayes estimator for this loss function. Problem #3 Work problem 7.24 in Casella & Berger (p. 335). Problem #4 Let X 1,..., X n be i.i.d. Normal(0, variance = θ). Suppose I am interested only in the special class of estimators of θ defined by { F = T : T n (m) =(n+m) 1 n }. Xi 2 i=1 Suppose that the loss function is L(t, θ) =θ 2 (t θ) 2. (a) In this class of estimators, which values of m, if any, yield an admissible estimator? (b) Is m = 0 minimax? (c) Answer (a) if the loss function is changed to L(t, θ) =θ 1 (t θ) 2. (c) What is the asymptotic limiting distribution of the mle, in terms of derivatives of the function d(θ)? Problem #5 One of the more difficult aspects of Bayesian inference done by frequentists is to find a noninformative prior. Jeffreys Prior is the one in which the prior density is proportional to the square root of the Fisher information. Suppose that X 1,..., X n are independent and identically distributed Bernoulli(θ). (a) Find the Jeffreys prior for this model. (b) Interpret the Jeffreys prior as a uniform prior for arcsin( θ). Problem #6 One criticism of the use of beta priors for Bernoulli sampling is that they are unimodel. Thus, various people have proposed the use of a mixture of betas prior, namely π(θ) =ɛg B (θ a, b)+(1 ɛ)g B (θ c, d), where g B (θ a, b) isthebeta(a, b) density. Show that this prior is conjugate for Bernoulli sampling.

4.6. HOMEWORK #6 21 Problem #7 Suppose that X 1,..., X n are iid with a negative exponential distribution with mean 1/θ. (a) Find the Jeffreys prior for θ. (b) Compute the posterior distribution for θ. (c) Compute the posterior distribution for λ = 1/θ. (d) Discuss computing the posterior mean and model for λ.

22 CHAPTER 4. HOMEWORK ASSIGNMENTS 4.7 Homework #7 Problem #1 If X 1,..., X n are i.i.d. normal with mean θ and variance 1.0, consider testing the hypothesis H 0 : θ 0 against the alternative H 1 : θ>0. What is the power function of the UMP level α test? Problem #2 In Problem #1, suppose that θ has a prior Normal distribution with mean 0.0 and variance σ 2. Consider the 0 1 loss function discussed in class, i.e., the loss is zero if a correct decision is made, and the loss is one otherwise. What is the Bayes procedure for this problem? Problem #3 Let X 1,..., X n be i.i.d. with a common density p(x θ) =exp{ (x θ)}, x θ. Let U =min(x 1,..., X n ). (a) Show that U and U (1/n) are (respectively) an mle and a UMVUE for θ. (b) In testing H 0 : θ θ 0 against H 1 : θ>θ 0 at level α, show that the UMP level α test is of the form to reject H 0 when U>c. (c) In part (b), express c as a function of θ 0 and α. (d) In parts (b) (c), what is the power function for the test? Problem #4 Suppose that X 1,..., X n are a sample with mass function pr(x = k) = (k 2)(k 1) (1 θ) k 3 θ 3. 2 (a) Show that if we wish to test H 0 : θ θ 0 against H 1 : θ>θ 0, find the form of the the UMP test. (b) What is the Fisher information and the asymptotic distribution of the mle here? Problem #5 Let X be a Binomial random variable based on a sample of size n = 10 with success probability θ. LetS= X 5, and suppose this is all that is observed, i.e., I only observe S, and I cannot observe X. Consider testing H 0 : θ 1/3 orθ 2/3 against H 1 : θ =1/2. Suppose I use the test which rejects H 0 when S =0orS=1. (a) What is the distribution of S? (b) Find the level of this test. Remember to consider carefully what level means with this composite hypothesis. (c) Is the test UMP of its level? Why or why not? Problem #6 Suppose I take n observations from a multinomial distribution with cell probabilities as arranged in Table 4.1, and data as in Table 4.2 I am interested in testing the hypothesis H 0 : θ yy θ yn against the alternative H 0 : θ yy >θ yn. By thinking carefully, find an appropriate

4.7. HOMEWORK #7 23 Yes No Yes θ yy θ yn No θ ny θ nn Table 4.1: Table of probabilities for Problem #7. The θ s sum to 1.0. Yes No Yes N yy N yn No N ny N nn Table 4.2: Table of probabilities for Problem #7. The N s sum to n. conditional test for this hypothesis. By a conditional test, I mean that you should condition on part or all of the data.

24 CHAPTER 4. HOMEWORK ASSIGNMENTS 4.8 Homework #8 Problem #1 Let X 1,..., X n be i.i.d. Poisson(θ). Suppose I want to test the hypothesis H 0 : θ = θ 0 against the alternative H 1 : θ θ 0. (a) What is the form of the GLR test here? (b) What is the form of the Wald test? (c) What is the form of the score test? (d) Prove directly that as n, the score test achieves its nominal level α. Problem #2 Repeat problem #1 but for the case of sampling from the normal distribution with mean and variance both equal to θ. Problem #3 Let X be Binomial(n, θ 1 )andlety be Binomial(n, θ 2 ). Let S = X + Y. (a) What is the distribution of X given S? You may find it useful to reparameterize θ 1 = {1+exp( )} 1 and θ 2 = {1+exp( η)} 1. (c) Is this distribution a member of the one-parameter exponential family with a monotone likelihood ratio? (c) Use the result in (a) to find a UMP conditional test of the hypothesis H 0 : θ 1 θ 2 against the alternative H 1 : θ 1 >θ 2. (d) What is the conditional Wald test for this problem? This is a one-sided test, and we did not cover one-sided testing in class. I m asking that you come up with a reasonable guess. Problem #4 Suppose we are concerned with the lower endpoint of computer generated random numbers which purport to be uniform on (0, 1). We have a sample X 1,..., X n are consider the density f(x, θ) = I(θ x) 1 θ. Consider the following observations: (.87,.84,.79,.33,.02,.97,.20,.47,.51,.29,.58,.69). Suppose we adopt a prior distribution with density π(θ) =(1+a)(1 θ) a. (a) What kind of prior beliefs does this prior represent? (b) Compute the posterior density for θ. (c) Plot this posterior density for a few values of a. (d) Compute a 95% credible confidence interval for θ, i.e., one which covers 95% of the posterior density. Problem #5 Consider the same model as in problem #4, but this time compute a 95% likelihood ratio confidence interval for θ.