Non-parametric Inference and Resampling

Size: px

Start display at page:

Download "Non-parametric Inference and Resampling"

June Sullivan
5 years ago
Views:

1 Non-parametric Inference and Resampling Exercises by David Wozabal (Last update. Juni 010) 1 Basic Facts about Rank and Order Statistics students were asked about the amount of time they spend surfing Internet per month : Student i Amount of time x i (in Hours) (a) What are the corresponding values for the ordered statistics? (b) Determine the rank statistics (r 1,..., r 10 ). 1. Suppose that we have a sample of size n for the setting of Exercise 1.1. In terms of the order statistics, what statistic will you consider to study the following : (a) the least amount of time spent (b) the highest amount of time spent (c) the median of the amount of time spent. Hint: distinguish between cases, when n is odd and when n is even. (d) the range of the amount of time spent (e) Determine the values of the statistics discussed in (a)-(d) based on the data in Exercise The empirical distribution function ˆF n corresponding to the sample x 1,..., x n is given by: ˆF n (t) = 1. Number of samples not exceeding t. n (a) Calculate (and draw the graph of) the empirical distribution function for Exercise 1.1. (b) Can you express the empirical distribution function in terms of order statistics. 1.4 Let X = (X 1,..., X n ) be a random vector with a continuous density, such that R is uniformly distributed on S n (the permutations of n elements). Show that (a) P(R i = k) = 1 n, 1 i, k n (b) P(R i = k, R j = h) = 1 n(n 1), i j, k h (c) P(R i = R j ) = 0, 1 i j n. 1.5 Let X 1,..., X n be i.i.d. random variables. Show that Hint: Use the fact that f X(1),X (n) (x, y) = n(n 1) (F (y) F (x)) n f(x)f(y), < x y <. P(X (1) x, X (n) y) = P(X (n) y) P(X (1) > x, X (n) y) to obtain the joint distribution of (X (1), X (n) ).

2 1.6 Show that the density of the midrange V n = (X (1) + X (n) )/ for an i.i.d. sample X 1,... X n from an absolutely continuous distribution F with density f is given by f Vn (v) = n(n 1) v {F (v x 1 ) F (x 1 )} n f(x 1 )f(v x 1 )dx 1, < v <. Hint: Use the joint density of (X (1), X (n) ) from 1.5 to calculate the joint density of (X (1), V n ) by finding a transformation T : R R, such that T (X (1), X (n) ) = (X (1), V n ) and noting that using the change of variable formula P(T (X) A) = P(X T 1 (A)) = f X (x)dx = f X (T 1 (x)) J T 1(x) dx x T 1 (A) where J T 1 is the determinant of the Jacobian of T 1. Now integrate out the dependence on x (1). 1.7 Calculate the explicit form for the density of the midrange V n, when X i F = U(0, 1) for all 1 i n. 1.8 For an i.i.d. sample X 1,..., X n, calculate the density of the order statistic X (i) for 1 i n. 1.9 Using example 1.8 calculate the density of U (i) for U i i.i.d. with U i U(0, 1) for all 1 i n For the U 1,... U n defined in Exercise 1.9 prove that E(U (i) ) = Hint: Use the following result (Beta function) where Γ is the Gamma function. B(x, y) = For the U 1,... U n defined in Exercise 1.9 prove that 0 Var(U (i) ) = i n + 1. x A t x 1 (1 t) y 1 dt = Γ(x)Γ(y) Γ(x + y) i(n i + 1) (n + 1) (n + ). Hint: Again use the Beta function and it s relation to the Gamma function. 1.1 Show that in the i.i.d. case the joint density of two arbitrary order statistics (X (j), X (i) ) with 1 i < j n is given by (for < x i < x j < ) f X(i),X (j) (x i, x j ) = n! (i 1)!(j i 1)!(n j)! F (x i) i 1 (F (x j ) F (x i )) j i 1 (1 F (x j )) n j f(x i )f(x j ). Hint: Use the formula for the joint density f of all the order statistics (why does the last equality hold?) f(x (1),..., x (n) ) = r S n f(x (r1 ),..., x (rn)) = n!f(x (1),..., x (n) ) and integrate out the orders k : k i, k j Let U 1,..., U n be a i.i.d. sample from the standard uniform distribution U(0, 1). Use 1.1 and 1.8 to show that the random variables U (i) /U (j) and U (j) for 1 i < j n, are statistically independent and calculate the respective distribution functions.

3 Simple Non-Parametric Tests.1 Use the sign test and the following i.i.d. sample 1.13, 1.990, , 0.491, 0.710, , , 0.40, 1.583, to test the null hypothesis that the median φ of the underlying distribution equals 0, against H 1 : φ 0 and H 1 : φ > 0 (at the level α = 0.05).. Repeat the test in.1 with the Wilcoxon test..3 Let X 1,..., X 48 be a random sample from a distribution with distribution function F. To test H 0 : F (41) = 1 4 against H 1 : F (41) < 1 4, use the statistic Y, which is the number of samples less than or equal to 41. If the observed value of Y is smaller than 7 reject H 0 and accept H 1. If p = F (41), find the power function K(p), 0 < p < 1 4 of the test. Approximate α = K( 1 4 )..4 A statistician decides that working with numbers is a tedious job and designs a universal test, which rejects the null hypothesis in favor of the alternative hypothesis if a coin toss (whose outcome is independent of the testing problem in question) yields heads. What is the power of this test?.5 Derive the probability generating function of V = n i=1 sign(x i)r i, where (X 1,..., X n ) is an i.i.d. sample from a distribution which fulfills the assumptions of the signed rank (Wilcoxon) test and (R 1,..., R n ) are the ranks of the sample ( X 1,..., X n )..6 Write a computer programm that calculates the pdf of V from.5 for different values of n..7 Write a computer program that samples m samples of size n from a (a) normal distribution with mean µ and σ = 1; (b) a shifted t-distribution with ν degrees of freedom and mean µ; (c) a uniform distribution on [µ c, µ + c]; (d) an exponential distribution with mean 1; (e) a distribution of your choice. Use the samples generated above to assess the power of the (a) t-test (b) sign test (c) Wilcoxon test (d) Mann-Whitney-U test when testing H 0 : µ = 0 against H1 a : µ 0 and Hb 1 : µ > 0 for different values of µ and the above distributions. Plot the empirically found power of the tests as a function of the true mean µ. How does the power depend on n and α? Generate meaningful plots to illustrate your results..8 A generalized Wilcoxon statistic can be defined by W = n i=1 sign(x i)d i (X i ) where d i (X i ) = c Ri for some c 1 < c <... < c n and the ranks R i of X i. Calculate the mean and the variance of the so called binary statistic W g = n i=1 sign(x i)d i (X i ) where c j = j..9 Check whether the distribution of the test statistic W g = n i=1 sign(x i)d i (X i ) from.8 is asymptotically normally distributed by checking the Liapounov condition n i=1 E( V i µ i 3 ) lim n ( n i=1 σ i ) 3 = 0 is satisfied for V i = sign(x i )d i (X i ) with mean µ i and variance σ i. 3

4 .10 Show that if d i (X i ) 1, then the generalized Wilcoxon test based on W = n i=1 sign(x i)d i (X i ) is equivalent to the sign test..11 Using the code from.7 empirically approximate the asymptotic relative efficiency of the tests in.7 for the case that the real data is distributed according to the distributions listed in.7. Which test is asymptotically the best for which family of distributions? Generate meaningful plots to illustrate your results..1 Use the asymptotic distribution of the Wilcoxon statistic to evaluate the efficacy of the Wilcoxon test statistic where the distribution F X of the data generating process X is given by (a) X N(µ X, σ ); (b) X U( 0.5, 05); (c) X is double exponential, i.e. F X (x) = { (1/)e λx, x 0 1 (1/)e λx, x > 0. Hint: Use the fact that for W N = N i=1 sign(x i)rank( X i ), and Var(W N ) θ=0 = ( E(W N ) = Np 1 + where p 1 = P(X i > 0) = 1 F X (θ) and p = P(X i + X j > 0) = Use the above to show that the efficacy is given by where I = f X (x)dx. (N + 1)(N + 1)(N) 6 ) N(N 1) N(N + 1) p F X (x + θ)df X (x). 4N(N 1) ( ) fx (0) (N + 1)(N + 1) N 1 + I.13 Calculate the efficacy of the Student s t test statistic in cases (b) and (c) of the above problem..14 Use the answers to Problem.1 and.13 to verify the following results for the Asymptotic Relative Efficacy of the Wilcoxon signed-rank test to the Students t test Normal 3/π Uniform 1 Double exponential 3/ 3 Estimation of CDFs and plug-in estimates 3.1 Generate 100 observations from a N(0, 1) distribution. Compute a 95 percent confidence band for the CDF based on the sample using the results covered in the course. Repeat this 100 times and see how often the confidence band contains the true distribution function. Repeat the experiment using data from a Cauchy distribution. 4

5 3. Let X 1,... X n Bernoulli(p). Find the plug-in estimator and estimated standard error of p. Find an approximate 95% confidence interval for p. 3.3 Let X 1,... X n F and let ˆF n be the empirical distribution function. Let a < b be fixed numbers and define θ = T (F ) = F (b) F (a). Let ˆθ = T ( ˆF n ) = ˆF n (b) ˆF n (a). Find the influence function and the estimated standard error of ˆθ. Find an expression for an approximate 1 α confidence interval for θ. 3.4 Let X 1,..., X n F and let ˆF n (x) be the empirical distribution function. For a fixed x, find the limiting distribution of ˆF n (x). 3.5 Let x and y be two distinct points. Find Cov( ˆF n (x), ˆF n (y)). 3.6 Download the data on magnitudes of earthquakes near Fiji from Estimate the CDF F(x). Compute and plot a 95% confidence envelope for F. Find an approximate 95% confidence intervall for F (4.9) F (4.3). 3.7 Suppose that T (F ) T (G) C sup F (x) G(x) (1) x for some C > 0. Prove that T ( ˆF n ) a.s. T (F ). Suppose that X M < and show that T (F ) = xdf (x) fulfills (1). 5

Non-parametric Inference and Resampling

Non-parametric Inference and Resampling Exercises by David Wozabal (Last update 3. Juni 2013) 1 Basic Facts about Rank and Order Statistics 1.1 10 students were asked about the amount of time they spend