Statistics Ph.D. Qualifying Exam

Transcription

1 Department of Statistics Carnegie Mellon University May Statistics Ph.D. Qualifying Exam You are not expected to solve all five problems. Complete solutions to few problems will be preferred to partial solutions to many problems. The last page of this exam contains a list of facts you may find useful.

2 1. Let X = (X 1,..., X n ) iid Poisson(λ) and let X n be the sample mean. (a) Show that the quantity R 1 (X, λ) = (Xn λ) is asymptotically a pivot. Construct a 1 α asymptotic confidence interval for λ using [R 1 (X, λ)] 2 λ/n. (b) Show that the quantity R 2 (X, λ) = X (Xn λ) is asymptotically a pivot. Construct a 1 α asymp- n/n totic confidence interval for λ using [R 2 (X, λ)] 2. (c) Show that Rao score statistic for Ω H : λ = λ 0 versus Ω H : λ λ 0 is [R 1 (X, λ 0 )] 2 and therefore the asymptotic confidence interval from (a) coincides with the confidence interval obtained from inverting Rao score test. (d) Show that Wald statistic for Ω H : λ = λ 0 versus Ω H : λ λ 0 is [R 2 (X, λ 0 )] 2, and therefore the asymptotic confidence interval from (b) coincides with the confidence interval obtained from inverting Wald test. 2

3 2. Consider the linear regression model Y = Xβ + ɛ, where X is a known n k deterministic matrix and the entries of ɛ R n are independent random variables identically distributed with mean 0 and variance σ 2. Let β be the ordinary least squares estimator of β, i.e. β = (X X) 1 X Y. Assume that (a) 1 n (X X) D, where D is positive definite; (b) for any vector t R k, where b = t (X X) 1 X R n. max b i 0, as n, i Show that ( β β) d Nk (0, Σ), and identify Σ as a function of D and σ 2. 3

4 3. Suppose X Multinomial k (n, θ), where θ Ω = (0, 1) k. Let Λ α be a Dirichlet(α) prior for Θ. (a) Show that the posterior distribution of Θ given X is a Dirichlet(α + X). (b) Show that the mode of the posterior distribution is θ mode = α + X 1 k j=1 α j + n k, where 1 is the k-dimensional vector of ones. For which value of α does θ mode equal the MLE θ = X n? (c) Assume squared error loss function. i. Show that, when α = a1 for some a > 0, the Bayes rule is δ Λα (X) = ka 1 ka + n k + n ka + n θ; ii. show that, when α = k 1, the corresponding Bayes rule δ Λα (X) = + n 1 k + n + n θ is minimax. (d) Find the limiting distribution of ( δλα (X) θ ). 4

5 4. Let (X 1,..., X n ) iid U(0, θ), where 0 < θ <. Let θ and T be the MLE and the UMVUE of θ, respectively. (a) Compute the mean squared errors of θ and T and show that, under squared error loss function, the MLE is inadmissible when n 2. (b) Prove that i. n(θ θ) d Z; ii. n(θ T ) d Z θ, where Z Exponential(θ). (c) Show that the asymptotic relative efficiency of θ with respect to T is 1/2. 5

6 5. Let (X 1,..., X n ) be a sample of independent and identically distributed random variables from a distribution with mean µ, variance σ 2 and finite fourth moment. Consider the estimation of µ 2 using the following three estimators: (i) T 1,n = X 2 n, where X n is the sample mean; (ii) T 2,n = X 2 1 n S2 n, where S 2 n is the sample variance; (iii) T 3,n = max{0, T 2,n }. (a) When µ 0, show that the three estimators have the same asymptotic efficiency by proving that they have the same asymptotic mean squared error 4µ2 σ 2 n. (b) When µ = 0, show that i. the asymptotic mean squared error of T 1,n is 3σ4 n 2 ; ii. the asymptotic mean squared error of T 2,n is 2σ4 n 2 ; iii. T 2,n is asymptotically more efficient than T 1,n ; iv. T 3,n is asymptotically more efficient than T 2,n. 6

7 Useful Facts 1. The probability density function of a Dirichlet(α), with α R k a vector with positive entries, is p α (x 1,..., x k ) = Γ(α 0 ) Γ(α 1 )... Γ(α k ) xα x α k 1 k, where x j 0 for all 1 j k, k j=1 x j = 1 and α 0 = k j=1 α j. Furthermore, if X = (X 1,..., X k ) Dirichlet(α), then, for j = 1,..., k, EX j = α j α 0 and VarX j = α j(α 0 α j ) α0 2(α 0 + 1). 2. If X Multinomial k (n, θ), where θ (0, 1) k, and θ = X n, then ( θ θ ) d Nk ( 0, diag(θ) θθ ). 3. If (X 1,..., X n ) iid U(0, θ), then max i X i is a complete sufficient statistic for θ. 4. The probability density function of an Exponential(θ), where θ > 0, is p θ (x) = 1 x exp θ 1{x 0}, θ and the mean and variance are θ and θ 2, respectively. 5. Let P = {P θ, θ Ω R k } be a regular parametric model. Suppose (X 1,..., X n ) is an i.i.d. sample from P and consider the hypothesis Ω H : θ = θ 0 versus Ω A : θ θ 0. Let I(θ) be the Fisher information matrix and Z n (θ) be the normalized score function, i.e. the k-dimensional random vector whose j-th 1 entry is n n i=1 The Wald statistic is θ j log p θ (X i ). ( θ θ0 ) Î( θ) ( θ θ0 ), where Î( ) is the observed Fisher information, and the Rao score statistic is Z n (θ 0 ) I 1 (θ 0 )Z n (θ 0 ). 6. The probability density function of a Beta(α, β), where α, β > 0, is and the mean and variance are p α,β (x) = α α+β and 7. If X χ 2 k, then E(X) = k and Var(X) = 2k. Γ(α + β) Γ(α)Γ(β) xα 1 (1 x) β 1 1 {x [0,1]}, αβ (α+β) 2 (α+β+1), respectively. 8. (A version of the Lindeberg-Feller CLT). Let {X ni } be a row independent triangular array with E(X ni ) = 0 and Var(X ni ) = σni 2. Let Y n = n i=1 X ni and σn 2 = n i=1 σ2 ni. Then, if 1 σ 2 n Y n /σ n d N(0, 1) n E(Xni1 2 { Xni ɛσ n}) 0 ɛ > 0. i=1 7