General Linear Test of a General Linear Hypothesis. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

Transcription

1 General Linear Test of a General Linear Hypothesis Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

2 Suppose the NTGMM holds so that y = Xβ + ε, where ε N(0, σ 2 I). opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

3 Suppose C is a known q p matrix and d is a known q 1 vector. The general linear hypothesis H 0 : Cβ = d is testable if rank(c) = q and each component of Cβ is estimable. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

4 Suppose p m A of rank s. Can H 0 : β C(A) be written as a testable general linear hypothesis? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

5 β C(A) P A β = β β P A β = 0 (I P A )β = 0 w 1. w p s β = 0, where w 1,..., w p s form a basis for C((I P A ) ) = C(I P A ). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

6 Suppose y ijk = µ + α i + β j + γ ij + ε ijk, i = 1, 2; j = 1, 2; k = 1,..., n ij. µ α 1 α 2 β 1 Let β = β 2. γ 11 γ 12 γ 21 γ 22 Write a testable general linear hypothesis for no interaction. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

7 In this case, no interaction means E(y 11k ) E(y 12k ) = E(y 21k ) E(y 22k ) E(y 11k ) E(y 12k ) E(y 21k ) + E(y 22k ) = 0 µ + α 1 + β 1 + γ 11 (µ + α 1 + β 2 + γ 12 ) (µ + α 2 + β 1 + γ 21 ) + (µ + α 2 + β 2 + γ 22 ) = 0 γ 11 γ 12 γ 21 + γ 22 = 0. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

8 Thus, H 0 : Cβ = 0 is testable GLH of no interaction if C = [0, 0, 0, 0, 0, 1, 1, 1, 1] because C is 1 p is of rank 1 and Cβ = γ 11 γ 12 γ 21 + γ 22 = E(y 111 ) E(y 121 ) E(y 211 ) + E(y 221 ) is estimable as a LC of elements of E(y). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

9 Suppose H 0 : Cβ = d is testable. Find the distribution of the BLUE of Cβ. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

10 Cˆβ = C(X X) X y N(Cβ, σ 2 C(X X) C ) where C(X X) C is a PD q q matrix of rank q based on previous results. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

11 Find the distribution of (Cˆβ d) (σ 2 C(X X) C ) 1 (Cˆβ d). opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

12 By Result 5.10, the distribution is χ 2 q(φ), where φ = 1 2 (Cβ d) (σ 2 C(X X) C ) 1 (Cβ d) = 1 2σ 2 (Cβ d) (C(X X) C ) 1 (Cβ d). opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

13 Show that Cˆβ and SSE are independent. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

14 [ŷ ] ˆε = [ P X y (I P X )y ([ PX ] N I P X ([ ] Xβ N 0 ] = Thus, ŷ and ˆε are independent. [ PX Xβ, I P X [ PX ] y I P X ], σ 2 [ PX 0 0 I P X [ ] ) σ 2 I P X I P X ]). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

15 Cβ estimable = A C = AX. Cˆβ = C(X X) X y = AX(X X) X y = AP X y = Aŷ. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

16 SSE = ˆε ˆε. Cˆβ is a function of only ŷ and SSE a function of only ˆε, Cˆβ and SSE are independent. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

17 We could alternatively have used Result 5.16: C(X X) X (σ 2 I)(I P X ) = σ 2 C(X X) X (I P X ) = σ 2 C(X X) (X X P X ) = σ 2 C(X X) (X X ) = 0. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

18 Now note that independence of Cˆβ and SSE = (Cˆβ d) (σ 2 C(X X) C ) 1 (Cˆβ d) are independent. and SSE σ 2 (n r) = y (I P X )y σ 2 = ˆσ2 (n r) σ 2 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

19 We have previously shown that SSE σ 2 χ 2 n r. Thus, and (n r)ˆσ 2 σ 2 χ 2 n r ˆσ 2 σ 2 χ2 n r/(n r). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

20 It follows that (Cˆβ d) (σ 2 C(X X) C ) 1 (Cˆβ d)/q ˆσ 2 /σ 2 = (Cˆβ d) (C(X X) C ) 1 (Cˆβ d) qˆσ 2 F q,n r (φ), F where opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

21 as defined previously. φ = 1 2σ 2 (Cβ d) (C(X X) C ) 1 (Cβ d) We can use F to test H 0 : φ = 0 H 0 : Cβ d = 0 ( (C(X X) C ) 1 is PD.) H 0 : Cβ = d. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

22 To test H 0 : Cβ = d at level α, we reject H 0 iff F F q,n r,α where F q,n r,α is the upper α quantile of the F q,n r distribution. By Result 5.13, the power of the test is a strictly increasing function of φ. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

23 Now suppose that H 0 : c β = d is testable. By arguments analogous to the previous F case, it is straightforward to show that ( ) c ˆβ d t ˆσ 2 c (X X) c t c β d n r. σ 2 c (X X) c Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

24 We can conduct tests of H 0 : c β = d against H A1 : c β < d H A2 : c β > d, H A : c β d or by comparing the observed value of t to the t n r distribution. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

25 Returning to the F-test of testable GLH H 0 : Cβ = d, note that there are multiple ways to express the same null hypothesis. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

26 For example, suppose y ij = µ i + ε ij (i = 1, 2, 3; j = 1,..., n i ). Find different matrices C 1, C 2 and C 3 C k µ 1 µ 2 µ 3 = 0 µ 1 = µ 2 = µ 3 k = 1, 2, 3. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

27 [ ] [ ] µ1 µ 2 C 1 =, C 1 µ = µ 1 µ 3 [ ] [ ] µ1 µ 2 C 2 =, C 2 µ = µ 2 µ 3 [ ] [ ] µ1 µ 2 C 3 =, C 3 µ =. µ 1/2 1/ µ 2 2 µ 3 opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

28 Suppose H 01 : C 1 β = d 1 and H 02 : C 2 β = d 2 are both testable and S 1 {β : C 1 β = d 1 } = {β : C 2 β = d 2 } S 2. Show that the F-test of H 01 is the same as the F-test of H 02. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

29 Recall that X = X P X = X X(X X) X. Thus, X(X X) is a GI of X X. It follows that C k (C kc k ) is a GI of C k. Thus, S k = {C k(c k C k) d k + (I C k(c k C k) C k )z : z R p } for k = 1, 2. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

30 Because S 1 = S 2, C 1 times any element of S 2 equal d 1 ; i.e., β S 2 = C 1 β = d 1. Thus C 1 [C 2(C 2 C 2) d 2 + (I C 2(C 2 C 2) C 2 )z] = d 1 z R p = C 1 C 2(C 2 C 2) d 2 d 1 + (C 1 C 1 C 2(C 2 C 2) C 2 )z = 0 z R p = C 1 = C 1 C 2(C 2 C 2) C 2 and C 1 C 2(C 2 C 2) d 2 = d 1 by Result A.8. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

31 Now C 1 = C 1 C 2(C 2 C 2) C 2 = C 1 = C 1 P C 2 = C 1 = P C 2 C 1 = C(C 1) C(P C 2 ) = C(C 2). Repeating the entire argument with the roles of C 1 and C 2 reversed gives C(C 2) C(C 1) so that C(C 2) = C(C 1). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

32 Because C 1 β = d 1 and C 2 β = d 2 are both testable, C 1 and C 2 are both full-column rank = q. a unique nonsingular matrix q q B C 1 = C 2B. We have previously shown that C 1 = P C 2 C 1 = C 2[(C 2 C 2) ] C 2 C 1. Thus, B = [(C 2 C 2) ] C 2 C 1 and B = C 1 C 2(C 2 C 2). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

33 We have also shown previously that C 1 C 2(C 2 C 2) d 2 = d 1 = B d 2 = d 1. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

34 Now consider the quadratic form (C 1 b d 1 ) (C 1 (X X) C 1) 1 (C 1 b d 1 ) = (B C 2 b B d 2 ) (B C 2 (X X) C 2B) 1 (B C 2 b B d 2 ) = (C 2 b d 2 ) BB 1 (C 2 (X X) C 2) 1 (B ) 1 B (C 2 b d 2 ) = (C 2 b d 2 ) (C 2 (X X) C 2) 1 (C 2 b d 2 ). This is true for b R p including ˆβ and the true parameter vector β. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35

35 the F statistics for testing H 01 : C 1 β = d 1 and H 02 : C 2 β = d 2 are identical, as are the noncentrality parameters associated with those F statistics. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 35