Stat 710: Mathematical Statistics Lecture 40

Transcription

1 Stat 710: Mathematical Statistics Lecture 40 Jun Shao Department of Statistics University of Wisconsin Madison, WI 53706, USA Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, / 11

2 Lecture 40: Simultaneous confidence intervals and Scheffe s method So far we have studied confidence sets for a real-valued θ or a vector-valued θ with a finite dimension k. In some applications, we need a confidence set for real-valued θ t with t T, where T is an index set that may contain infinitely many elements, for example, T = [0,1] or T = R. Definition 7.6 Let X be a sample from P P, let θ t, t T, be real-valued parameters related to P, and let C t (X), t T, be a class of (one-sided or two-sided) confidence intervals. (i) Intervals C t (X), t T, are level 1 α simultaneous confidence intervals for θ t, t T, iff inf P( θ t C t (X) for all t T ) 1 α. P P The left-hand side of the above expression is the confidence coefficient of C t (X), t T. Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, / 11

3 Lecture 40: Simultaneous confidence intervals and Scheffe s method So far we have studied confidence sets for a real-valued θ or a vector-valued θ with a finite dimension k. In some applications, we need a confidence set for real-valued θ t with t T, where T is an index set that may contain infinitely many elements, for example, T = [0,1] or T = R. Definition 7.6 Let X be a sample from P P, let θ t, t T, be real-valued parameters related to P, and let C t (X), t T, be a class of (one-sided or two-sided) confidence intervals. (i) Intervals C t (X), t T, are level 1 α simultaneous confidence intervals for θ t, t T, iff inf P( θ t C t (X) for all t T ) 1 α. P P The left-hand side of the above expression is the confidence coefficient of C t (X), t T. Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, / 11

4 Definition 7.6 (continued) (ii) Intervals C t (X), t T, are simultaneous confidence intervals for θ t, t T, with asymptotic confidence level 1 α iff lim P( θ t C t (X) for all t T ) 1 α. n Intervals C t (X), t T, are 1 α asymptotically correct iff the equality in the above expression holds. If the index set T contains k < elements, then θ = (θ t,t T ) is a k-vector and the methods studied in the previous sections can be applied to construct a level 1 α confidence set C(X) for θ. If C(X) can be expressed as t T C t (X) for some intervals C t (X), then C t (X), t T, are level 1 α simultaneous confidence intervals. This simple method, however, does not always work. A simple method for the case of T containing finite many elements is introduced next. Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, / 11

5 Definition 7.6 (continued) (ii) Intervals C t (X), t T, are simultaneous confidence intervals for θ t, t T, with asymptotic confidence level 1 α iff lim P( θ t C t (X) for all t T ) 1 α. n Intervals C t (X), t T, are 1 α asymptotically correct iff the equality in the above expression holds. If the index set T contains k < elements, then θ = (θ t,t T ) is a k-vector and the methods studied in the previous sections can be applied to construct a level 1 α confidence set C(X) for θ. If C(X) can be expressed as t T C t (X) for some intervals C t (X), then C t (X), t T, are level 1 α simultaneous confidence intervals. This simple method, however, does not always work. A simple method for the case of T containing finite many elements is introduced next. Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, / 11

6 Bonferroni s method Bonferroni s method, which works when T contains k < elements, is based on the following simple inequality for k events A 1,...,A k : ( ) ( ) k k k k P A i P(A i ) i.e., P 1 P(A i ). i=1 i=1 A c i i=1 For each t T, let C t (X) be a level 1 α t confidence interval for θ t. If α t s are chosen so that t T α t = α (e.g., α t = α/k for all t), then Bonferroni s simultaneous confidence intervals are C t (X), t T, since P(θ t C t (X),t T ) 1 t T i=1 P(θ t C t (X)) = 1 α t = 1 α t T Bonferroni s intervals are of level 1 α, but they are not of confidence coefficient 1 α even if C t (X) has confidence coefficient 1 α t for any fixed t. Bonferroni s method does not require that C t (X), t T, be independent. Bonferroni s method is easy to use, but may be too conservative. Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, / 11

7 Bonferroni s method Bonferroni s method, which works when T contains k < elements, is based on the following simple inequality for k events A 1,...,A k : ( ) ( ) k k k k P A i P(A i ) i.e., P 1 P(A i ). i=1 i=1 A c i i=1 For each t T, let C t (X) be a level 1 α t confidence interval for θ t. If α t s are chosen so that t T α t = α (e.g., α t = α/k for all t), then Bonferroni s simultaneous confidence intervals are C t (X), t T, since P(θ t C t (X),t T ) 1 t T i=1 P(θ t C t (X)) = 1 α t = 1 α t T Bonferroni s intervals are of level 1 α, but they are not of confidence coefficient 1 α even if C t (X) has confidence coefficient 1 α t for any fixed t. Bonferroni s method does not require that C t (X), t T, be independent. Bonferroni s method is easy to use, but may be too conservative. Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, / 11

8 Scheffé s method Consider the normal linear model X = N n (Z β,σ 2 I n ), where β is a p-vector of unknown parameters, σ 2 > 0 is unknown, and Z is an n p known matrix of rank r p. Let L be an s p matrix of rank s r. Suppose that R(L) R(Z) and we would like to construct simultaneous confidence intervals for t τ Lβ, where t T = R s {0}. Using the argument in Example 7.15, for each t T, we can obtain the following confidence interval for t τ Lβ with confidence coefficient 1 α: [ t τ L β t n r,α/2 σ t τ Dt, t τ L β + t n r,α/2 σ t τ Dt ], where β is the LSE of β, σ 2 = X Z β 2 /(n r), D = L(Z τ Z) L τ, and t n r,α is the (1 α)th quantile of the t-distribution t n r. However, these intervals are not level 1 α simultaneous confidence intervals for t τ Lβ, t T. Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, / 11

9 Scheffé s method Scheffé s (1959) method of constructing simultaneous confidence intervals for t τ Lβ is based on the following equality (exercise): x τ A 1 x = max y R k,y 0 (y τ x) 2 y τ Ay, where x R k and A is a k k positive definite matrix. Theorem 7.10 Assume the normal linear model. Let L be an s p matrix of rank s r. Assume that R(L) R(Z) and D = L(Z τ Z) L τ is of full rank. Then C t (X) = [ t τ L β σ sc α t τ Dt, t τ L β + σ sc α t τ Dt ], t T, are simultaneous confidence intervals for t τ Lβ, t T, with confidence coefficient 1 α, where σ 2 = X Z β 2 /(n r), T = R s {0}, and c α is the (1 α)th quantile of the F-distribution F s,n r. Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, / 11

10 Scheffé s method Scheffé s (1959) method of constructing simultaneous confidence intervals for t τ Lβ is based on the following equality (exercise): x τ A 1 x = max y R k,y 0 (y τ x) 2 y τ Ay, where x R k and A is a k k positive definite matrix. Theorem 7.10 Assume the normal linear model. Let L be an s p matrix of rank s r. Assume that R(L) R(Z) and D = L(Z τ Z) L τ is of full rank. Then C t (X) = [ t τ L β σ sc α t τ Dt, t τ L β + σ sc α t τ Dt ], t T, are simultaneous confidence intervals for t τ Lβ, t T, with confidence coefficient 1 α, where σ 2 = X Z β 2 /(n r), T = R s {0}, and c α is the (1 α)th quantile of the F-distribution F s,n r. Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, / 11

11 Proof Note that t τ Lβ C t (X) for all t T is equivalent to i.e., From the inequality we have (t τ L β t τ Lβ) 2 s σ 2 t τ c α for all t T, Dt (t τ L β t τ Lβ) 2 max t T s σ 2 t τ c α. Dt x τ A 1 x = max y R k,y 0 (y τ x) 2 y τ Ay, (t τ L β t τ Lβ) 2 max t T s σ 2 t τ = (L β Lβ) τ D 1 (L β Lβ) Dt s σ 2 = F. Note that F has the F-distribution F s,n r. Then P (t τ Lβ C t (X) for all t T ) = P (F c α ) = 1 α. Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, / 11

12 Remarks If the normality assumption is removed but conditions in Theorem 3.12 are assumed, then Scheffé s intervals in Theorem 7.10 are 1 α asymptotically correct (exercise). The choice of the matrix L depends on the purpose of analysis. One particular choice is L = Z, in which case t τ Lβ is the mean of t τ X. When Z is of full rank, we can choose L = I p, in which case {t τ Lβ : t T } is the class of all linear functions of β. Another L commonly used when Z is of full rank is the following (p 1) p matrix: L = It can be shown (exercise) that for this L, { } t τ Lβ : t R p 1 {0} = { c τ β : c R p {0},c τ J = 0 }, where J is the p-vector of ones. Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, / 11

13 Remarks Functions c τ β satisfying c τ J = 0 are called contrasts. Setting simultaneous confidence intervals for t τ Lβ, t T, with the above L is the same as setting simultaneous confidence intervals for all nonzero contrasts. Example 7.28 (Simple linear regression) Consider the special case where X i = N(β 0 + β 1 z i,σ 2 ), i = 1,...,n, and z i R satisfying S z = n i=1 (z i z) 2 > 0, z = n 1 n i=1 z i. In this case, we are usually interested in simultaneous confidence intervals for the regression function β 0 + β 1 z, z R. Note that the result in Theorem 7.10 (with L = I 2 ) can be applied to obtain simultaneous confidence intervals for β 0 y + β 1 z, t T = R 2 {0}, where t = (y,z). If we let y 1, Scheffé s intervals in Theorem 7.10 are Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, / 11

14 Remarks Functions c τ β satisfying c τ J = 0 are called contrasts. Setting simultaneous confidence intervals for t τ Lβ, t T, with the above L is the same as setting simultaneous confidence intervals for all nonzero contrasts. Example 7.28 (Simple linear regression) Consider the special case where X i = N(β 0 + β 1 z i,σ 2 ), i = 1,...,n, and z i R satisfying S z = n i=1 (z i z) 2 > 0, z = n 1 n i=1 z i. In this case, we are usually interested in simultaneous confidence intervals for the regression function β 0 + β 1 z, z R. Note that the result in Theorem 7.10 (with L = I 2 ) can be applied to obtain simultaneous confidence intervals for β 0 y + β 1 z, t T = R 2 {0}, where t = (y,z). If we let y 1, Scheffé s intervals in Theorem 7.10 are Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, / 11

15 Example 7.28 (continued) [ β0 + β 1 z σ 2c α D(z), β 0 + β 1 z + σ 2c α D(z) ], z R, (1) where D(z) = n 1 +(z z) 2 /S z. Unless ( β 0 + max β 1 z β 0 β 1 z) 2 ( β 0 y + = max β 1 z β 0 y β 1 z) 2 z R D(z) t=(y,z) T t τ (Z τ Z) 1 t (2) holds with probability 1, where Z is the n 2 matrix whose ith row is the vector (1,z i ), the confidence coefficient of the intervals in (1) is larger than 1 α. We now show that (2) actually holds with probability 1 so that the intervals in (1) have confidence coefficient 1 α. First, P ( n( β 0 β 0 )+n( β 1 β 1 ) z 0 ) = 1. Second, it can be shown (exercise) that the maximum on the right-hand side of (2) is achieved at Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, / 11

16 Example 7.28 (continued) ( ) y Z τ Z t = = z n( β 0 β 0 )+n( β 1 β 1 ) z ( β0 β 0 β 1 β 1 Finally, (2) holds since y in (3) is equal to 1 (exercise). ). (3) Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, / 11