Ph.D. Qualifying Exam Friday Saturday, January 6 7, 2017

Transcription

1 Ph.D. Qualifying Exam Friday Saturday, January 6 7, 2017 Put your solution to each problem on a separate sheet of paper. Problem 1. (5106) Let X 1, X 2,, X n be a sequence of i.i.d. observations from a logistic distribution with the probability density function f(x θ) = exp( (x θ)) (1 + exp( (x θ))) 2. Our goal is to find the maximum likelihood estimate (MLE) of θ with the observations {X i } n. Derive an expression for the log-likelihood function l(θ) = such that the MLE is given by n log(f(x i θ)), ˆθ = argmax θ l(θ). (c) Find the expressions for l(θ) and l(θ), the first and second derivatives of l with respect to θ. Verify that l(θ) < 0. Write out the Newton-Raphson algorithm to find the root of l(θ). Problem 2. (5106) Let Y be a continuous random variable with probability density function: Y α 1 f 1 (y; µ 1, σ 2 1) + α 2 f 2 (y; µ 2, σ 2 2), where f 1 and f 2 are two Gaussian density functions with means µ 1, µ 2 and variances σ 2 1, σ 2 2, respectively. Also, 0 α 1, α 2 1, such that α 1 + α 2 = 1. Given n i.i.d. observations {Y i } n, our goal is to find the maximum likelihood estimate of θ = (α 1, µ 1, σ 1, α 2, µ 2, σ 2 ). Use the EM algorithm for iteratively estimating θ. Let θ (m) be the current values of the unknown. Derive the mathematical formula to update for θ (m+1). Let L(θ) denote the likelihood with parameter θ. Prove that L(θ (m+1) ) L(θ (m) ). 1

2 Problem 3. (5166) Consider the following linear model for a randomized block design: y ti = µ + β i + τ t + ɛ ti, t = 1,..., k; i = 1,..., n, where µ is an overall mean, τ t is the effect of tth treatment, β i is the effect of ith block, {ɛ ti : t = 1,..., k; i = 1,..., n} are assumed to be i.i.d. N(0, σ 2 ). The least squares estimate of β i is ˆβ i = ȳ i ȳ. Find the expectation and variance of ˆβ i. Show the decomposition of variation for the experiment: S D = S B + S T + S R where S D : Total Variation of the observations, S B : Sum of Squares for Blocks, S T : Sum of Squares for Treatments, S R : Sum of Squares for Experimental Errors. (c) Find the expectation of S B. Problem 4. (5166) model: Consider the following unbalanced one-way random-effects y ij = µ + α i + ɛ ij, i = 1,..., k; j = 1,..., n i, where {α i, i = 1,..., k} are i.i.d. N(0, σα), 2 {ɛ ij, i = 1,..., k; j = 1,..., n i } are i.i.d. N(0, σɛ 2 ), and the α i s and ɛ ij s are independent. Define k k n i SSA = n i (ȳ i ȳ ) 2, SSE = (y ij ȳ i ) 2, where j=1 n i y i = y ij, ȳ i = y i ; y = n i j=1 k n i y ij, ȳ = j=1 y k n. i Find Cov(y ij, y i j ). What is the distribution of ȳ? Show that SSA = k yi 2 y2 n k i n. Show that SSA and SSE are independent. i (c) Find unbiased estimates for the variances σ 2 α and σ 2 ɛ. 2

3 Problem 5. (5167) Suppose that {(x 1, y 1 ),..., (x n, y n )} is a sample from a bivariate normal distribution, i.e., ( ) (( ) ( )) xi µ1 σ 2 N, 1 ρσ 1 σ 2 y i µ 2 ρσ 1 σ 2 σ2 2, i = 1,, n. Show that the conditional distribution of y i given x i is normal and ( y i x i N µ 2 + ρ σ ) 2 (x i µ 1 ), σ 2 σ 2(1 ρ 2 ), i = 1,, n. 1 Define β 1 = ρ σ 2 σ 1, β 0 = µ 2 β 1 µ 1, σ 2 = σ 2 2(1 ρ 2 ). (1) Then y i given x i follows the simple regression model: y i = β 0 + β 1 x i + ɛ i, i = 1,, n, where {ɛ i, i = 1,, n} are i.i.d. N(0, σ 2 ). The moment estimates of β 0, β 1, and σ 2, denoted by β 0, β1, and σ 2, respectively, are obtained by simply substituting the sample means ( x and ȳ), sample variances (ˆσ 2 1 and ˆσ 2 2), and sample correlation ˆρ into (1). Are the moment estimates the same as the least squares estimates ˆβ 0, ˆβ 1, and ˆσ 2? Problem 6. (5167) Consider the linear regression model: Y = Xβ + ξ, where Y = (y 1,..., y n ), ξ = (ξ 1,..., ξ n ), β = (β 1,..., β p ) and X is an n p full-rank matrix. The process {ξ i } is generated by the moving-average model: ξ i = ɛ i θ 1 ɛ i 1 θ 2 ɛ i 2, where {ɛ i, i = 1, 0, 1,..., n} are i.i.d. N(0, σ 2 ) variables. Calculate the variance-covariance matrix of ξ. Describe how to use the least squares method, the weighted least squares method, and the maximum likelihood method to estimate the coefficients β in the model. Give details of the three procedures. Suppose that θ 1 = 0.5 and θ 2 = 2. Let ˆβ be the weighted least squares estimate of β. Give the expression of ˆβ in this case. Define Ŷ = X ˆβ, and ˆξ = Y Ŷ. Are Ŷ and ˆξ independent? Show your reasons. 3

4 Put your solution to each problem on a separate sheet of paper. Problem 7. (5326) Suppose that a Geiger counter is turned on at time zero and that clicks on this Geiger counter occur according to a Poisson process with a rate of λ clicks per second. (c) Let S t denote the random number of clicks during the time interval (0, t). State a formula for P (S t = k) valid for nonnegative integers k. (No proof is needed. Just state the answer.) Let T 1 < T 2 < T 3 < be the times of the first click, second click, third click,..., and X 1 = T 1, X 2 = T 2 T 1, X 3 = T 3 T 2,... be the times between clicks (the interarrival times). What is the joint distribution of X 1, X 2, X 3,...? In particular, give an explicit formula for the joint density of (X 1, X 2, X 3 ). (No proof is needed. Just state the answers.) Use the facts in parts and or any other approach to prove that t λ r Γ(r) zr 1 e λz dz = r 1 y=0 where λ > 0 and t > 0. Give a detailed argument. (λt) y e λt, r = 1, 2, 3,... y! (d) Let S t be as defined in for any value of t. Find P (S 2 1, S 5 2, S 10 = 3). Problem 8. (5326) Suppose the random variables (X, Y ) have the joint density f(x, y) = x 2 ye x(y+1) for x > 0, y > 0 (and f(x, y) = 0 otherwise). Answer the following. Carefully specify the support of any density or joint density. Find the marginal density of Y. Find the joint density of (U, V ) where U = X(Y + 1) and V = X. (c) Find the density of U = X(Y + 1). 4

5 Problem 9. (5327) Let X 1, X 2,..., X n be i.i.d. from a distribution with pdf given by { θ 1 x (1 θ)/θ if 0 x 1 f(x θ) = 0 otherwise, where θ > 0 is the unknown parameter. Show that T = 2 n log(x i) is a minimal sufficient statistic for θ. Find the distribution of Y = 2 log X 1. (c) Using Basu s theorem or otherwise, find E[Y T ], the conditional expectation of the random variable Y given T. Problem 10. (5327) Suppose that X 1, X 2,..., X n are independent and identically distributed Poisson(λ) random variables for an unknown parameter λ > 0. Instead of observing the random variables X i, we only observe the events X i = 0 or X i > 0 for i = 1,..., n. Find the maximum likelihood estimate (MLE) of λ and discuss when the MLE is not finite. Compute the probability that the MLE is not finite based on a sample of size n, assuming that the true value of λ is λ 0 > 0. 5

6 Problem 11. (6346) Let (Ω, F, µ) be a measure space. A statement about elements ω Ω is true locally almost everywhere (µ) if i. the statement is true for all ω A, where A F, and ii. µ(a F ) = 0 for any F F with µ(f ) <. (Note: A is the complement of A.) Show that if something is true almost everywhere (µ) then it is true locally almost everywhere (µ). If µ is σ-finite, show that if something is true locally almost everywhere (µ) then it is true almost everywhere (µ). Problem 12. (6346) Let (Ω = [0, 1], F = B[0, 1], µ) be a probability space where µ is Lebesgue measure on [0, 1] and B[0, 1] is the restriction of the Borel σ-field to [0, 1]. Let X n be a sequence of random variables given by { n X n (ω) = 2, ω [0, 1/n], 0, ω (1/n, 1]. Show that X n P X by using the definition of convergence in probability, and give X. Show that X n X a.s. for the same X as in part. (c) Prove or disprove: For p 1, X n L p X for the same X as in parts and. 6