Statement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.

MATHEMATICAL STATISTICS Homework assignment Instructions Please turn in the homework with this cover page. You do not need to edit the solutions. Just make sure the handwriting is legible. You may discuss the problems with your peers but the final solutions should be your work. There is no specific deadline but you need to complete everything to get the grade. Statement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.

Homework assignment Random variables and random vectors 1. Suppose Z 1, Z 2,..., Z n are independent, identically ditributed random variables having the Beta(1, q) distribution. a. Define X 1 = Z 1 X 2 = Z 2 (1 Z 1 ) X 3 = Z 3 (1 Z 2 )(1 Z 1 ). Find the joint distribution of X 1, X 2 and X 3. Consider taking logarithms. b. More generally, define i 1 X i = Z i (1 Z j ). for i = 1, 2,..., n. Find the joint distribution of (X 1, X 2,..., X n ). 2. Let X 1, X 2,..., X n, X n+1 be random variables such that E(X k ) = 0 for k = 1,..., n + 1 and covariance matrix Σ (a (n + 1) (n + 1) matrix). We would like to find the best linear predictor for X n+1 based on the variables X 1, X 2,..., X n. This means that we are looking for the linear combination ˆX n+1 = b 0 + b 1 X 1 + + b n X n for which the expected square error j=1 E(X n+1 ˆX n+1 ) 2 will be as small as possible. Find the coefficients b 0, b 1,..., b n. Hint: Write the square error as a function of b 0, b 1,..., b n and use partial derivatives. 3. Let X and Y be random variables with density for x, y 0. f X,Y (x, y) = xe x(y+1) a. Find the conditional densities f X Y =y (x) and f Y X=x (y). b. Find E(X Y ) and E(Y X) and check that E(Xg(Y )) = E(E(X Y )g(y )) and E(Y g(x)) = E(E(Y X)g(X)) for an arbitrary bounded function g. 2

Homework assignment 4. The conditional variance of Y given (X 1, X 2,..., X n ) is defined as v(x 1,..., x n ) = var(y X 1 = x 1, X 2 = x 2,..., X n = x n ) = (y ψ(x 1,..., x n )) 2 f Y X=x (y) dy. where ψ(x 1,..., x n ) = E(Y X 1 = x 1,..., X n = x n ). We can interpret the conditional variance as a random variable given as a. Show that var(y X 1,..., X n ) = v(x 1, X 2,..., X n ). v(x 1, X 2,..., X n ) = E [ (Y E(Y X 1,..., X n )) 2 X1, X 2,..., X n ]. b. Convince yourself that var(y ) = E(var(Y X 1,..., X n )) + var(e(y X 1,..., X n )). Can you explain this formula in words? Multivariate normal distribution 5. Suppose Z is a random vector whose components are independent standard normal random variables and let A be a rectangular matrix such that AA T is invertible. Prove that the density of X = AZ + µ is still given by the formula ( 1 f X (x) = exp (2π) det(aa 1 ) n/2 T 2 (x µ)t (AA T ) 1 (x µ). ) 6. Suppose X is a mutivariate normal vector with expectation 0 and variance Σ. Write ( ) ( ) X1 Σ11 Σ X = and Σ = 12. X 2 Σ 21 Σ 22 Assume Σ is invertible. Compute the conditional density of X 2 given X 1 = x 1 by using the usual formula Hint: Use the inversion lemma ( Σ 1 = f X2 X 1 =x 1 (x 2 ) = f X(x) f X1 (x 1 ). (Σ 11 Σ 12 Σ 1 22 Σ 21 ) 1 (Σ 11 Σ 12 Σ 1 22 Σ 21 ) 1 Σ 12 Σ 1 22 Σ 1 22 Σ 21 (Σ 11 Σ 12 Σ 1 22 Σ 21 ) 1 (Σ 22 Σ 21 Σ 1 11 Σ 12 ) 1 Compare this proof to the slicker one using independence of linear transformations of multivariate normal vectors. Comment. ) 3

Homework assignment 7. Suppose X N p (µ, Σ) and that the matrix QΣQ T is an invertile q q matrix. Show that the conditional distribution of X given Qx = q is normal with conditional mean µ + ΣQ T (QΣQ T ) 1 (q Qµ) and conditional (singular) covariance matrix Σ ΣQ T (QΣQ T ) 1 QΣ. 8. Suppose X and Y are p-dimensional random vectors such that ( ) X N Y 2p (0, Σ) where the covariance matrix is of the form ( ) I ρ11 T Σ = ρ11 T I The matrix I represents the p p identity matrix, 1 = (1, 1,..., 1) T and ρ is a scalar constant such that ρ 1/ p(p 1). a. Compute E(X T X). b. Compute E(X T X Y). 9. If A and B are p p symmetric idempotent matrices of rank r and s and if AB = 0, show that, for X N p (0, σ 2 I) a. Show that b. Show that c. Show that X T AX/r X T BX/s F r,s. X T AX X T (A + B)X B(r/2, s/2). (p r)x T AX rx T (I A)X F r,p r. Central limit theorem 4

Homework assignment 10. The central limit theorem is also valid for vectors. For roulette we can code the outcomes in vectors ξ 1, ξ 2,... such that for i = 1, 2,..., 37 { 1 if in spin n the oucome is i 1 ξi n = 0 else. By central limit theorem one has ξ 1 + + ξ n ne(ξ 1 ) n d N(0, Σ). a. How can one interpret the expressions ξ 1 + ξ n ne(ξ 1 ) n? b. Find E(ξ 1 ) and var(ξ 1 ) = Σ. c. Usually one would use the χ 2 -statistic to test whether the wheel is biased. One defines 36 χ 2 (n j np) 2 = np j=0 where p = 1/37, n j is the number of occurences of outcome j and n is the number of spins. Use b. to prove that the distribution of χ 2 is approximately χ 2 (36). Parameter estimation 11. The log-normal distribution has the density for x > 0. f X (x) = 1 2πσx e (log x µ)2 /(2σ 2 ) a. Assume that σ is known and you have i.i.d. observations X 1, X 2,..., X n. Find the maximum likelihood estimate for µ. b. Find the approximate standard error of your estimator. 12. The Pareto distribution with parameters α and λ has density for x > 0 where α, λ > 0. f(x, α, λ) = αλ α (λ + x) α+1 5

Homework assignment a. Write down the equations for the MLE of the parameters given i.i.d. observations x 1, x 2,..., x n. b. Compute the approximate standard error for the MLE od α. 13. Let X 1, X 2,..., X n be an i.i.d. sample from the inverse Gaussian distribution I(µ, τ) with density { τ 2πx exp τ } (x 3 µ)2, x > 0, τ > 0, µ > 0. 2xµ 2 The expectation of the inverse Gaussian distribution is E(X 1 ) = µ. Assume that all densities are smooth enough to apply the asymptotic theorems. a. (10) Find the MLE for (µ, τ) based on observations x 1,..., x n. b. (10) Compute the Fisher information matrix I(µ, τ). c. (5) Give a formula for the approximate 95% confidence interval for µ based on x 1, x 2,..., x n. 14. Suppose X = (X 1, X 2,..., X n ) N(µ, σ 2 Σ) where σ 2 is an unknown parameter and Σ is a known invertible matrix. a. Suppose the expectation µ is known and you have one observation X 1. How would you estimate σ 2? Is your estimate unbiased? What is the variance of the estimate you found? Hint: What is the distribution of Σ 1/2 X? b. How would you go about the questions in a. if µ was not known but you knew that all components of µ were the same? Hypothesis testing 15. We have observations X 1, X 2,..., X n from the normal distribution N(µ, σ 2 ). We would like to test H 0 : µ = 0 versus H 1 : µ 0. a. One can test H 0 at confidence level α in two ways: - H 0 is rejected if X > c for a suitable c. - One estimates µ in σ 2 and sets up a confidence interval. If the confidence interval does not cover 0 the null-hypothesis is rejected. Are the above tests the same? Comment. What is the answer if we assume that the parameter σ is known? 6

Homework assignment b. Find the likelohood ratio statistic for the above testing problem in both cases, when σ is known and when σ is unknown. 16. Bartlett s test is a commonly used test for equal variances. The testing problem assumes that all observations {x ij } for i = 1, 2,..., k and j = 1, 2,..., n i for each i are like independent random variables where X ij N(µ i, σ 2 i ). One tests versus H 0 : σ 2 1 = σ 2 2 =... = σ 2 k H 1 : the σ 2 i are not all equal Assume we have samples of size n i from the i-th population, i = 1, 2,..., k, and the usual variance estimates from each sample where s 2 i = 1 n i 1 s 2 1, s 2 2,..., s 2 k n i j=1 (x ij x i ) 2. Introduce the following notation ν j = n j 1 and ν = k ν i and s 2 = 1 ν k ν i s 2 i The Bartlett s test statistic M is defined by M = ν log s 2 k ν i log s 2 i. a. The approximate distribution of Bartlett s M is χ 2 (r). What is in your opinion r? Explain why. b. Assume that the maximum likelihood estimates for parameters µ i and σi 2 are ˆµ i = x i = 1 n i x ij and ˆσ i 2 = 1 n i (x ij x i ) 2 n i n i 7

Homework assignment for i = 1, 2,..., k. Write down the likelihood ratio statistic for the testing problem in question. What is its approximate distribution? Any similarity to Bartlett s test? Comment. Hint: If you assume σ 2 1 = σ 2 = = σ 2 k, the MLE estimates for µ i are still the means x i for i = 1, 2,..., k. 17. The one sample Wilcoxon test is used to test whether a continuous distribution is symmetric. On the basis of n i.i.d. observations X 1, X 2,..., X n from an unknown continuous distribution F one tests the hypothesis H 0 : F (x) = 1 F ( x) for all x versus H 1 : F (x) < 1 F ( x) for some x. Let R i be the rank of X i among the X 1, X 2,..., X n. The sign test is based on the statistic n W = 1(X i > 0)R i i.e. the sum of ranks of positive X i s. a. Show that if H 0 is true W has a distribution that does not depend on F. b. Show that n W = 1( X i X j ). i,j=1 c. Show that if H 0 is true then E(W ) = n(n + 1)/4. d. Compute the variance of W. e. How would you find critical values for testing H 0? 8

Homework assignment On the following pages you will find the take-home finals from previous years. Do two of the three as part of your homework. 9

MATHEMATICAL STATISTICS Final take-home examination April 8 th -April 15 th, 2013 Instructions You do not need to edit the solutions. Just make sure the handwriting is legible. The final solutions should be your work. The deadline for completion is March 15th, 2013 by 4pm. Turn in your solutions to Petra Vranješ. For any questions contact me by e-mail. Statement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.

1. (25) Suppose a population of size N is divided into K = N/M groups of size M. We select a sample of size km the following way: First we select k groups out of K groups by simple random sampling with replacement. We then select m units in each group selected on the first step by simple random sample with replacement. The estimate of the population mean is the average Ȳ of the sample. Let µ i be the population average in the i-the group for i = 1, 2,..., K. Let σu 2 = 1 K (µ i µ) 2, K where µ = K µ i/k. Let σ 2 w = 1 N K M (y ij µ i ) 2, where y ij denotes the value of the variable for the j-the unit in the i-th group. j=1 a. Let k = 1. Show that we can write the estimator as K Ȳ = I i Y i, where { 1 if the i-th group is selected. I i = 0 otherwise and var(y i ) = σ 2 i /m. Argue that it is reasonable to assume that Y i and I i are all independent. Let σ 2 i be the population variance for the i-th subgroup. Compute var(ȳ ). b. If we repeat the procedure we get independent estimators Ȳ1, Ȳ2,..., Ȳk, and estimate the population average by Show that Ȳ = 1 k k Ȳ k. var(ȳ ) = σ2 u k + σ2 w km. Argue that this expression is the variance of the estimator described in the introduction. 2

c. The assumption that we sample with replacement is unrealistic. Let k = 1 and assume that the sample of size m is selected by simple random sample without replacement. Argue that K Ȳ = I i Y i, where { 1 if we select the ith subgroup. I i = 0 otherwise Compute the variance of the estimator in this case. d. Assume that the k groups are selected by simple random sample without replacement. In this case the estimator is Ȳ = 1 k K I i Y i, where { 1 if we select the ith subgroup. I i = 0 otherwise Argue that it is reasonable to assume that I 1,..., I K and Y 1,..., Y K are independent. Compute the standard error of the estimator. e. Explain why the sampling distribution in d. is approximately normal. 3

2. (25) Suppose {p(x, θ), θ Θ R k } is a (regular) family of distributions. Define the vector valued score function s as the column vector with components s(x, θ) = θ log(p(x, θ)) = grad(log(p(x, θ)). and the Fisher information matrix as I(θ) = var(s). Remark: If p(x, θ) = 0 define log (p(x, θ)) = 0. a. Let t(x) be an unbiased estimator of θ based on the likelihood function, i.e. Prove that Deduce that cov(s, t) = I. E θ (t(x)) = θ. E(s) = 0 and E(st T ) = I. Remark: Make liberal assumptions about interchanging integration and differentiation. b. Let a, c be two arbitrary k dimensional vectors. Prove that corr 2 ( a T t, c T s ) = (a T c) 2 a T var(t)a c T I(θ)c. The correlation coefficient squared is always less or equal 1. Maximize the expression for the correlation coefficient over c and deduce the Rao-Cramér inequality. 4

3. (25) Suppose X 1, X 2,..., X n are i.i.d. observations from a multivariate normal distribution N(µ, Σ) where Σ is known. Further assume that R is a given matrix and r a given vector. Use the likelihood ratio procedure to produce a test statistic for H 0 : Rµ = r vs. H 1 : Rµ r. Give explicit formulae for the test statistic and the critical values. 5

4. (25) Let Y = Xβ + ɛ be a linear model where we assume E(ɛ) = 0 in var(ɛ) = σ 2 Σ for a known invertible matrix Σ. a. Show that the BLUE for β is given by ˆβ = (X T Σ 1 X) 1 X T Σ 1 Y. Assume that X T Σ 1 X is invertible and use the Gauss-Markov theorem. b. Assume that the linear model is of the form Y kl = α + βx kl + u k + ɛ kl, k = 1, 2,..., K in l = 1, 2,..., L k where ɛ kl are N(0, σ 2 ) and u k are N(0, τ 2 ) and all random quantities are independent. Assume that the ratio τ 2 /σ 2 is known. Show that the BLUE is given by where ( ˆαˆβ ) = ( wk wk x k wk x k S xx + w k x 2 k ) 1 ( wk ȳ k w k = L k σ 2 /(σ 2 + L k τ 2 ) S xx = (x kl x k ) 2 k l S xy = (x kl x k )(y kl ȳ k ). k l S xy + w k x k ȳ k ), Hint: For c 1/n one has (I + c11 T ) 1 = I c(1 + nc) 1 11 T where 1 T = (1, 1,..., 1). c. What would you do if the ratio τ 2 /σ 2 were unknown? d. How would you test the hypothesis H 0 : β = 0 versus H 1 : β 0? What is the distribution of the test statistic under the null-hypothesis? 6

MATHEMATICAL STATISTICS Final take-home examination May 7 th -May 16 th, 2014 Instructions You do not need to edit the solutions. Just make sure the handwriting is legible. The final solutions should be your work. The deadline for completion is May 16th, 2014 by 4pm. Turn in your solutions to Petra Vranješ. For any questions contact me by e-mail. Statement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.

1. (25) Suppose a population of size N is divided into K = N/M groups of size M. We select a sample of size n = km the following way: First we select k groups out of K groups by simple random sampling. We then select m units in each group selected on the first step by simple random sampling. The estimate of the population mean is the average Ȳ of the sample. Let µ i be the population average in the i-th group for i = 1, 2,..., K, and let σ 2 i be the population variance in the i-th group for i = 1, 2,..., K. a. (10) Show that we can write the estimator as Ȳ = 1 k K Ȳ i I i, where { 1 if the i-th group is selected. I i = 0 otherwise and Ȳi is the sample average in the i-th group for i = 1, 2,..., K. Argue that it is reasonable to assume that the random variables Ȳ1,..., ȲK are independent and independent from I 1,..., I K. Show that Ȳ is an unbiased estimator of the population mean µ and show that the variance of Ȳ is var(ȳ ) = M m k(m 1)m 1 K K σi 2 + K k k(k 1) 1 K b. (15) Suggest an estimate for the quantity K (µ i µ) 2. σ 2 b = 1 K K (µ k µ) 2 = 1 K K µ 2 k µ 2. Is your estimate unbiased? Can you modify it to be an unbiased estimate? 2

2. (25) Suppose Θ 1, Θ 2,..., Θ n are i.i.d. random variables with values in [0, 2π) each having the von Mises density f(θ; µ, k) = 1 exp (k cos(θ µ)) 2πI 0 (k) for 0 θ < 2π where k 0 and µ [0, 2π] are the unknown parameters. I 0 is the modified Bessel function of the first kind and order 0. Suppose you have an i.i.d. sample θ 1, θ 2,..., θ n. a. (10) Let ν = (cos(µ), sin(µ)). Derive the MLE for ν. b. (5) Describe how you would find the MLE for k. c. (10) Let a = k cos(µ) and b = k sin(µ) and let â n and ˆb n be their respective MLE based on n i.i.d. observations. Show that the asymptotic distribution of n(ân a, ˆb n b) is bivariate normal N(0, Σ 1 ), where Σ is the covariance matrix of the random vector (cos(θ 1 ), sin(θ 1 )). 3

3. (25) Suppose X 1, X 2,..., X n are i.i.d. observations from a multivariate normal distribution N(µ, Σ) where Σ is known. Further assume that a is a given vector. Use the likelihood ratio procedure to produce a test statistic for H 0 : a T µ = 0 vs. H 1 : a T µ 0 a. (15) Give explicit formulae for the test statistic and the critical values. b. (10) What changes if the covariance matrix of the X i is of the form σ 2 Σ with unknown σ 2 and known Σ? 4

4. (25) Let Y = Xβ +ɛ be a linear model where we assume E(ɛ) = 0 in var(ɛ) = σ 2 Σ for a known invertible matrix Σ. a. (10) Show that the BLUE for β is given by ˆβ = (X T Σ 1 X) 1 X T Σ 1 Y. Assume that X T Σ 1 X is invertible and use the Gauss-Markov theorem. b. (15) Assume that the linear model is of the form Y kl = α + βx kl + u k + ɛ kl, k = 1, 2,..., K in l = 1, 2,..., L k where ɛ kl are N(0, σ 2 ) and u k are N(0, τ 2 ) and all random quantities are independent. Assume that the ratio τ 2 /σ 2 is known. Show that the BLUE is given by where ( ˆαˆβ ) = ( wk wk x k wk x k S xx + w k x 2 k ) 1 ( wk ȳ k w k = L k σ 2 /(σ 2 + L k τ 2 ) S xx = (x kl x k ) 2 k l S xy = (x kl x k )(y kl ȳ k ). k l S xy + w k x k ȳ k ), Hint: For c 1/n one has (I + c11 T ) 1 = I c(1 + nc) 1 11 T where 1 T = (1, 1,..., 1). 5

MATHEMATICAL STATISTICS Final take-home examination May 4 th -May 12 th, 2015 Instructions You do not need to edit the solutions. Just make sure the handwriting is legible. The final solutions should be your work. The deadline for completion is May 12th, 2015 by 4pm. Turn in your solutions to Petra Vranješ. For any questions contact me by e-mail or call me at 041 725 497. Statement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.

2. (25) Assume the data pairs (y 1, z 1 ),..., (y n, z n ) are an i.i.d. sample from the distribution with density for y > 0 and σ > 0. f(y, z, θ, σ) = e y 1 e (z θy) 2yσ 2 2πyσ a. Find the maximum likelihood estimators of θ and σ 2. Are the estimators unbiased? b. Find the exact standard errors of ˆθ and ˆσ 2. c. Compute the Fisher information matrix. d. Find the standard errors of the maximum likelihood estimators using the Fisher information matrix. Comment on your findings. 2 3

3. (25) Assume that the data x 1, x 2,..., x n are an i.i.d. sample from the multivariate normal distribution of the form (( ) ( )) µ (1) Σ11 Σ X 1 N µ (2), 12. Σ 21 Σ 22 Assume that the parameters µ and Σ are unknown. Assume the following theorem: If A(p p) is a given symmetric positive definite matrix then the positive definite matrix Σ that maximizes the expression 1 exp ( 12 det(σ) Tr ( Σ 1 A )) n/2 is the matrix Σ = 1 n A. The testing problem is H 0 : Σ 12 = 0 versus H 1 : Σ 12 0. a. Find the maximum likelihood estimates of µ and Σ in the unconstrained case. b. Find the maximum likelihood estimates of µ and Σ in the constrained case. c. Write the likelihood ratio statistic for the testing problem as explicitly as possible. d. What can you say about the distribution of the likelihood ratio statistic? 4

4. (25) Assume the regression model Y i1 = α + βx i1 + ɛ i Y i2 = α + βx i2 + η i for i = 1, 2,..., n. In other words the observation come in pairs. Assume that E(ɛ i ) = E(η i ) = 0, var(ɛ i ) = var(η i ) = σ 2 and corr(ɛ i, η i ) = ρ ( 1, 1). Assume that the pairs (ɛ 1, η 1 ),..., (ɛ n, η n ) are uncorrelated. Furthermore assume that n x i1 x i2 = 0. a. Assume that ρ is known. Find the best linear unbiased estimate of the regression parameters α and β. Find an unbiased estimator of σ 2. b. Assume that ρ is unknown and let ˆα and ˆβ be the ordinary least squares estimators of the regression parameters. Compute the standard errors of the two estimators. c. Let ˆɛ i and ˆη i be the residuals from ordinary least squares. Express [ n ) 2 E (ˆɛ ] i + ˆη i 2 and with the elements of the hat matrix H. [ n ] E ˆɛ iˆη i d. Give an estimate of var(ˆα) and var( ˆβ). Are the estimators unbiased? itemize 5