Probability and Statistics qualifying exam, May 2015 Name: Instructions: 1. The exam is divided into 3 sections: Linear Models, Mathematical Statistics and Probability. You must pass each section to pass the exam. 2. M.S. students: in the Mathematical Statistics section, please choose a total of 4 questions (say, 4 from Group 1 plus 0 from Group 2; or 2 from Group 1 plus 2 from Group 2). Ph.D. students: do the 4 questions from Group 2. 3. M.S. students: in the Probability section, please choose a total of 4 questions (say, 4 from Group 1 plus 0 from Group 2; or 2 from Group 1 plus 2 from Group 2). Ph.D. students: do the 4 questions from Group 2. 4. remark: we consider a M.S. student to be someone whose goal is to get a M.S. degree in Statistics, irrespective of whether (s)he is also enrolled in the Ph.D. program. 5. Please justify all your answers. 6. Good luck!
Page 2 Linear Models 1. Consider data (X i, Y i ) R 2 related by Y i = β 0 + β 1 X i + ϵ i i = 1,..., n where β 0, β 1 R are constants, ϵ i N(0, σ 2 ) are iid, and σ 2 is some known constant. (a) Write the least squares estimate ˆβ of β = [β 0, β 1 ] in matrix form and identify its distribution. (b) Under what conditions are the estimates ˆβ 0, ˆβ 1 uncorrelated? (c) Suppose you have a new X 0 and believe that Y 0 = β 0 + β 1 X 0 + ϵ 0 where ϵ 0 N(0, σ 2 ) is independent of all other ϵ i. Find a 95% prediction interval for Y 0. 2. Consider a model Y = Xβ + ϵ where X = 1 1 1 1 1 0, β i = [ β0 β 1 ] and ϵ N(0, σ 2 I). (a) Demonstrate that this model s parameterization is identifiable. (b) Describe how you would test the hypothesis β 1 = 0 if σ 2 is a known constant. What is the rejection region for a size α test? (c) Describe an estimate of σ 2 and prove that it is (un)biased. 3. Let X R m n be some nonzero matrix and M the PPO onto C(X). (a) Prove that tr(m) = r(x). (b) Prove that 0 M ii 1. (c) Prove that if M has full rank, then M = I. 4. Let Y N(µ, Σ) where Y = [Y 1, Y 2, Y 3 ], µ = [0, 0, 0], Σ = σ 2 + ρ ρ 0 ρ σ 2 + ρ 0 0 0 σ 2 = σ 2 I + ρee where ρ is not necessarily positive and e = [1, 1, 0]. (a) For a given σ 2 > 0, which restrictions on ρ ensure that Σ is positive definite?
Page 3 (b) Suppose that µ, σ 2 are known and you have observed Y = y. Find an equation for the MLE ˆρ of ρ. It may be helpful to note that det(a + uv ) = (1 + v A 1 u) det(a) (A + uv ) 1 = A 1 A 1 uv A 1 1+v A 1 u
Page 4 Mathematical Statistics Group 1 1. Consider a sample X 1,..., X n U[θ, θ + 1]. (a) Prove that T (X) = (X (1), X (n) ) is a minimal and sufficient statistic for θ. (b) Is the statistic T (X) complete? 2. Let X Bin(n, p), 0 < p < 1, and suppose that the prior distribution of p is Beta(α, β), i.e. where π(p) = Γ(α + β) Γ(α)Γ(β) pα 1 (1 p) β 1 1 (0,1) (p), α, β > 0, Γ(α) = (a) Find the posterior distribution of p, i.e., f(p x). 0 t α 1 e t dt. (b) Find the Bayes estimator of p under the squared loss function L(p, d) = (p d) 2. 3. Consider a sample X 1,..., X n f(x 1 θ), where the density f( θ) is given by Define the parameter g(θ) = θ 1. (a) Find the UMVU estimator of g(θ). f(x 1 θ) = θx θ 1 1 1 (0,1) (x 1 ), θ > 1. (b) For a given n N, find a maximum likelihood estimator of g(θ). Describe its asymptotic distribution as n. 4. Consider the sample X 1,..., X n U[0, θ], θ > 0, and the composite hypotheses For a fixed α (0, 1), find the UMP test. H 0 : θ θ 0 vs H 1 : θ > θ 0. Group 2 1. Let X 1,..., X n N(µ, σ 2 ), where µ = σ > 0. (a) Show that ( n T (X) = X i, i=1 is minimal sufficient for this parametric family. (b) Is T (X) also complete? Please justify your answer. 2. Let X 1,..., X n N(θ, aθ 2 ), θ > 0, and a > 0 is known. (a) Find an explicit expression for an EL (efficient likelihood) estimator θ EL (X) of θ (this includes proving that θ EL (X) is, indeed, an EL estimator). n i=1 X 2 i )
Page 5 (b) Let g(θ) = log θ be a reparametrization. Show that g( θ EL (X)) is asymptotically unbiased for g(θ). 3. Suppose we have two independent samples X 1,..., X n λ 1, λ 2 > 0. (a) Find the GLR test for the hypotheses exp(λ 1 ), Y 1,..., Y m exp(λ 2 ), H 0 : λ 1 = λ 2 vs H 1 : λ 1 λ 2. (b) Show that the test in part (a) can be based on the statistic n i=1 T (X, Y) = X i n i=1 X i + m j=1 Y. j (c) Find the distribution of T (X, Y) when H 0 is true. 4. Let X 1,..., X n, n 2, be an sample from a Poi(λ), λ > 0, parametric family. (a) Prove that E(S 2 X) = X a.s. (b) Use part (a) to establish that S 2 is not UMVU for λ.
Page 6 Probability Group 1 1. For Borel sets A, B, define d(a, B) = P (A B), where A B = (A B)\(A B). Let {A n } n N, A be a collection of Borel sets. Prove that d(a n, A) 0 if and only if 1 An 1 A in the L 2 (P ) sense. 2. Suppose that {A n } n N are independent events such that P (A n ) < 1, n N. Prove that ( P n=1 A n ) = 1 P (A n i.o.) = 1. 3. Let X Cauchy(0, 1), i.e., f X (x) = 1 1 π 1 + x 2, x R. Find the density of the random variable Y = 1. 1+X 2 4. Prove the classical central limit theorem. In other words, let {X n } n N be an sequence such that EX 1 = µ and Var(X i ) = σ 2. Then, ( Xn µ ) d n N(0, 1), n, σ where X n = 1 n n i=1 X i. Group 2 1. Let {X n } n N be an sequence of non-degenerate random variables defined on a given probability space. Show that P (X n converges) = 0. 2. Let {X n } n N, X be random variables, and let {F n } n N, F be their respective distribution d functions. Assume that X n X, and that F is continuous. Prove that 3. Please answer the following questions. sup F n (x) F (x) 0, n. x R (a) Let X and Y be two a.c. random variables with joint density f X,Y (x, y). Show that a conditional density f X Y (x y) exists and establish its form. (b) Assume that P (W = c) = 1 for some c R, and let Z be some other random variable. Show that P (W = c Z = z) = 1, z R. 4. Let {X n } n N be an sequence on a given probability space such that P (X n = 0) = 1 2 = P (X n = 1), n N. Show that the random variable is well-defined and find its distribution. X := n=1 X n 2 n End of exam.