Lecture 28: Asymptotic confidence sets

Size: px

Start display at page:

Download "Lecture 28: Asymptotic confidence sets"

Lee Cummings
5 years ago
Views:

1 Lecture 28: Asymptotic confidence sets 1 α asymptotic confidence sets Similar to testing hypotheses, in many situations it is difficult to find a confidence set with a given confidence coefficient or level 1 α. A common approach is to find a confidence set whose confidence coefficient or level is approximately 1 α when the sample size n is large. Based on a sample X, a confidence set C(X) for ϑ has asymptotic level 1 α if lim n P θ (ϑ C(X)) 1 α Often, we can replace by = in the previous expression. In any case, we call C(X) a 1 α asymptotic confidence set. Asymptotic pivotal quantities A known function q n (X,ϑ) is said to be asymptotically pivotal iff the limiting distribution of q n (X,ϑ) does not depend on any unknown quantity. UW-Madison (Statistics) Stat 610 Lecture / 10

2 Confidence sets based on asymptotic pivots Like a pivotal quantity in constructing confidence sets with a given confidence coefficient or level, an asymptotically pivotal quantity can be used in constructing 1 α asymptotic confidence sets. Because of the wide applications of the CLT, Slutsky s theorem, and the delta-method, many asymptotic pivots can be constructed with the 1/2 form V n ( ϑ n ϑ), where ϑ n is an estimator of ϑ that is asymptotically normal, i.e., V 1/2 n ( ϑ n ϑ) converges in distribution to N(0,I k ) and V n is a consistent estimator of V n (called the asymptotic covariance matrix of ϑ n ) in the sense that if v ij and v ij are the (i,j)th elements of V n and V n, respectively, then n( v ij v ij ) converges in probability to 0. The resulting 1 α asymptotically correct confidence sets are of the form C(X) = {ϑ : V 1/2 n ( ϑ n ϑ) 2 χk,α 2 }, UW-Madison (Statistics) Stat 610 Lecture / 10

3 where χk,α 2 is the 100(1 α)th percentile of the chi-square distribution with k degrees of freedom and b 2 = b b for any vector b. If ϑ is real-valued, then this C(X) is a 1 α asymptotic confidence interval; ϑ is multivariate, C(X) is an ellipsoid. Eample. Let X 1,...,X n be iid random variables with E(X 1 ) = µ and Var(X 1 ) = σ 2, and let ϑ = µ. Then n( X µ)/s is an asymptotic pivot, since by the CLT and Slutsky s theorem, it converges in distribution to N(0, 1). Based on this asymptotic pivot, we obtain a 1 α asymptotic confidence interval [ X z α/2 S/ n, X + z α/2 S/ n] If we also know that X i has the Poisson(λ) distribution with unknown λ > 0, then µ = λ and σ 2 = λ. Hence, we have two other asymptotic pivots: UW-Madison (Statistics) Stat 610 Lecture / 10

4 where χk,α 2 is the 100(1 α)th percentile of the chi-square distribution with k degrees of freedom and b 2 = b b for any vector b. If ϑ is real-valued, then this C(X) is a 1 α asymptotic confidence interval; ϑ is multivariate, C(X) is an ellipsoid. Eample. Let X 1,...,X n be iid random variables with E(X 1 ) = µ and Var(X 1 ) = σ 2, and let ϑ = µ. Then n( X µ)/s is an asymptotic pivot, since by the CLT and Slutsky s theorem, it converges in distribution to N(0, 1). Based on this asymptotic pivot, we obtain a 1 α asymptotic confidence interval [ X z α/2 S/ n, X + z α/2 S/ n] If we also know that X i has the Poisson(λ) distribution with unknown λ > 0, then µ = λ and σ 2 = λ. Hence, we have two other asymptotic pivots: UW-Madison (Statistics) Stat 610 Lecture / 10

5 n( X λ) X and n( X λ) λ. Based on the first one we obtain a 1 α asymptotic confidence interval [ X zα/2 X/ n, X + zα/2 X/ n ] For the second asymptotic pivot, we can construct a 1 α asymptotic confidence set C(X) = {λ : ( X λ) 2 } { zα/2 2 = λ : λ 2 (2 λ/n X + n 1 zα/2 2 )λ + X } 2 0 Note that Q(λ) = λ 2 (2 X + n 1 z 2 α/2 )λ + X 2 is a convex quadratic function. Thus, C(X) is also a confidence interval [L(X),U(X)] whose two limits are the two solutions to Q(λ) = 0: L(X) = X n 1 z 2 α/2 Xz 2 α/2 /n + z 4 α/2 /n2 U(X) = X n 1 z 2 α/2 + Xz 2 α/2 /n + z 4 α/2 /n2 UW-Madison (Statistics) Stat 610 Lecture / 10

6 Taking the difference of the lower limits of the two confidence intervals we get zα/2 2 2n z 2 X α/2 + z4 α/2 n n 2 zα/2 2 X n / = z2 α/2 2n z4 α/2 z 2 X α/2 n 2 + z4 α/2 n n 2 + zα/2 2 X n which converges to 0 at the rate n 1 and, hence, the two intervals are asymptotically equivalent. From the previous example, we find that, if ϑ = θ is the only unknown parameter in the population and thus V n = V n (θ), then we may not need to estimate V n in order to obtain a confidence interval. We can directly solve ( ϑ n ϑ) 2 /V n (ϑ) χ1,α 2 and obtain a confidence set { C(X) = ϑ : ( ϑ } n ϑ) 2 χ1,α 2 V n(ϑ) Often, C(X) is a confidence interval. UW-Madison (Statistics) Stat 610 Lecture / 10

7 Taking the difference of the lower limits of the two confidence intervals we get zα/2 2 2n z 2 X α/2 + z4 α/2 n n 2 zα/2 2 X n / = z2 α/2 2n z4 α/2 z 2 X α/2 n 2 + z4 α/2 n n 2 + zα/2 2 X n which converges to 0 at the rate n 1 and, hence, the two intervals are asymptotically equivalent. From the previous example, we find that, if ϑ = θ is the only unknown parameter in the population and thus V n = V n (θ), then we may not need to estimate V n in order to obtain a confidence interval. We can directly solve ( ϑ n ϑ) 2 /V n (ϑ) χ1,α 2 and obtain a confidence set { C(X) = ϑ : ( ϑ } n ϑ) 2 χ1,α 2 V n(ϑ) Often, C(X) is a confidence interval. UW-Madison (Statistics) Stat 610 Lecture / 10

8 A sufficient condition for C(X) to be an interval is that, as a function of ϑ, χ 2 1,α V n(ϑ) ( ϑ n ϑ) 2 is unimodal. This is actually the case for the second confidence interval in the previous example for the Poisson case. We may replace V n (ϑ) by V n ( ϑ n ), which is consistent if V n (ϑ) is a continuous function of ϑ. This ensures that we obtain a confidence interval, and the first interval in the previous example for the Poisson case is this kind. Suppose now that ϑ is not the only parameter, i.e., θ = (ϑ,ϕ) and we want a confidence set or interval for ϑ. If V n (θ) does not depend on ϕ, then the procedures in the previous discussion can still be applied. If V n (θ) = V n (ϑ,ϕ) depends on ϕ, then we first find an estimator ϕ n of ϕ, and then use V n (ϑ, ϕ n ) or V n ( ϑ n, ϕ n ) and apply the previous discussed procedure. We may also find an estimator of V n (θ) in a different way. UW-Madison (Statistics) Stat 610 Lecture / 10

9 Example. Let X 1,...,X n be iid from Double-exponential(µ,σ) with unknown µ R and σ 2 > 0. Consider setting a confidence interval for ϑ = µ based on the asymptotic distribution of X (not the MLE of µ). By the CLT and Var(X 1 ) = 2σ 2, n( X µ) converges in distribution to N(0,1) 2σ If we estimator 2σ 2 by the sample variance S 2, then we obtain the following 1 α asymptotic confidence interval (noting that zα/2 2 = χ2 1,α ) C(X) = {µ : n( X µ) 2 χ1,α 2 S2} = [ X zα/2 S/ n, X + z α/2 S/ n ] Suppose now that we know σ 2 = µ 2. If we estimate 2σ 2 by 2 X 2 and inverting n( X µ) 2 χ1,α 2 (2 X 2 ), the resulting confidence interval is the same as C(X) with S replaced by 2 X. UW-Madison (Statistics) Stat 610 Lecture / 10

10 If we invert n( X µ) 2 χ 2 1,α (2µ2 ), the resulting confidence interval is [L(X),U(X)] with L(X) = X 2z α/2 X / n 1 2z 2 α/2 /n, U(X) = X + 2z α/2 X / n 1 2z 2 α/2 /n If we estimate 2σ 2 = 2µ 2 by S 2, then we can still use interval C(X). Typically, in a given problem there exist many different asymptotic pivots that lead to different 1 α asymptotic confidence sets for θ. The following result indicates that we can compare volumes of confidence sets constructed using two asymptotic pivots. Theorem. Let C j (X), j = 1,2, be the confidence sets for ϑ using asymptotic V 1/2 pivots jn ( ϑ jn ϑ), j = 1,2, respectively. If V 1n < V 2n for sufficiently large n, where A is the determinant of the matrix A, then lim n P( vol(c 1 (X)) < vol(c 2 (X)) ) = 1. UW-Madison (Statistics) Stat 610 Lecture / 10

11 If we invert n( X µ) 2 χ 2 1,α (2µ2 ), the resulting confidence interval is [L(X),U(X)] with L(X) = X 2z α/2 X / n 1 2z 2 α/2 /n, U(X) = X + 2z α/2 X / n 1 2z 2 α/2 /n If we estimate 2σ 2 = 2µ 2 by S 2, then we can still use interval C(X). Typically, in a given problem there exist many different asymptotic pivots that lead to different 1 α asymptotic confidence sets for θ. The following result indicates that we can compare volumes of confidence sets constructed using two asymptotic pivots. Theorem. Let C j (X), j = 1,2, be the confidence sets for ϑ using asymptotic V 1/2 pivots jn ( ϑ jn ϑ), j = 1,2, respectively. If V 1n < V 2n for sufficiently large n, where A is the determinant of the matrix A, then lim n P( vol(c 1 (X)) < vol(c 2 (X)) ) = 1. UW-Madison (Statistics) Stat 610 Lecture / 10

12 Proof. The result follows from the consistency of V jn and the fact that the volume of the ellipsoid C(X) is equal to vol(c(x)) = π k/2 (χ 2 k,α )k/2 V n 1/2/ Γ(1 + k/2). If ϑ 1n is asymptotically more efficient than ϑ 2n, then V 1n V 2n and ϑ 1n results in a better confidence set in terms of volume. If ϑ n is asymptotically efficient (such as the MLE), then C(X) obtained by using the pivot based on ϑ n is asymptotically optimal in terms of volume. In the example for the Poisson distribution, X is the MLE and hence the first confidence interval is asymptotically optimal in terms of length. Although the theorem is not directly applicable to the second confidence interval in the Poisson example, actually we have shown in the end of the example that the second confidence interval is also asymptotically optimal in terms of length, because the difference between the lengths of the two intervals is of the order n 1, whereas the length is of the order n 1/2. UW-Madison (Statistics) Stat 610 Lecture / 10

13 Proof. The result follows from the consistency of V jn and the fact that the volume of the ellipsoid C(X) is equal to vol(c(x)) = π k/2 (χ 2 k,α )k/2 V n 1/2/ Γ(1 + k/2). If ϑ 1n is asymptotically more efficient than ϑ 2n, then V 1n V 2n and ϑ 1n results in a better confidence set in terms of volume. If ϑ n is asymptotically efficient (such as the MLE), then C(X) obtained by using the pivot based on ϑ n is asymptotically optimal in terms of volume. In the example for the Poisson distribution, X is the MLE and hence the first confidence interval is asymptotically optimal in terms of length. Although the theorem is not directly applicable to the second confidence interval in the Poisson example, actually we have shown in the end of the example that the second confidence interval is also asymptotically optimal in terms of length, because the difference between the lengths of the two intervals is of the order n 1, whereas the length is of the order n 1/2. UW-Madison (Statistics) Stat 610 Lecture / 10

14 In the double exponential example, the MLE of µ is the sample median Q 0.5, which has an asymptotic distribution derived previously: ( ) 1 n( Q0.5 µ) converges in distribution to N 0, 4(2σ) 2 = σ 2 By inverting n( Q 0.5 µ) 2 χ1,α 2 S2 /2 (σ 2 should be estimated by S 2 /2, not S 2 ), we obtain the 1 α asymptotic confidence interval [ C 1 (X) = X zα/2 S/ 2n, X + z α/2 S/ ] 2n which is shorter than the C(X) previously derived, by a factor of 2. We may replace the estimator S 2 /2 of σ 2 by the MLE of σ 2, which is equal to (verify) σ 2 = 1 n X i n Q 0.5 i=1 However, replacing S 2 /2 by σ 2 does not reduce the length. Finally, there are situations where an MLE does not exist or is very hard to calculate and, thus, we have to use other asymptotic pivots. UW-Madison (Statistics) Stat 610 Lecture / 10

Lecture 32: Asymptotic confidence sets and likelihoods

Lecture 32: Asymptotic confidence sets and likelihoods Asymptotic criterion In some problems, especially in nonparametric problems, it is difficult to find a reasonable confidence set with a given confidence