General Linear Test of a General Linear Hypothesis Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 1 / 35
Suppose the NTGMM holds so that y = Xβ + ε, where ε N(0, σ 2 I). opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 2 / 35
Suppose C is a known q p matrix and d is a known q 1 vector. The general linear hypothesis H 0 : Cβ = d is testable if rank(c) = q and each component of Cβ is estimable. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 3 / 35
Suppose p m A of rank s. Can H 0 : β C(A) be written as a testable general linear hypothesis? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 4 / 35
β C(A) P A β = β β P A β = 0 (I P A )β = 0 w 1. w p s β = 0, where w 1,..., w p s form a basis for C((I P A ) ) = C(I P A ). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 5 / 35
Suppose y ijk = µ + α i + β j + γ ij + ε ijk, i = 1, 2; j = 1, 2; k = 1,..., n ij. µ α 1 α 2 β 1 Let β = β 2. γ 11 γ 12 γ 21 γ 22 Write a testable general linear hypothesis for no interaction. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 6 / 35
In this case, no interaction means E(y 11k ) E(y 12k ) = E(y 21k ) E(y 22k ) E(y 11k ) E(y 12k ) E(y 21k ) + E(y 22k ) = 0 µ + α 1 + β 1 + γ 11 (µ + α 1 + β 2 + γ 12 ) (µ + α 2 + β 1 + γ 21 ) + (µ + α 2 + β 2 + γ 22 ) = 0 γ 11 γ 12 γ 21 + γ 22 = 0. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 7 / 35
Thus, H 0 : Cβ = 0 is testable GLH of no interaction if C = [0, 0, 0, 0, 0, 1, 1, 1, 1] because C is 1 p is of rank 1 and Cβ = γ 11 γ 12 γ 21 + γ 22 = E(y 111 ) E(y 121 ) E(y 211 ) + E(y 221 ) is estimable as a LC of elements of E(y). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 8 / 35
Suppose H 0 : Cβ = d is testable. Find the distribution of the BLUE of Cβ. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 9 / 35
Cˆβ = C(X X) X y N(Cβ, σ 2 C(X X) C ) where C(X X) C is a PD q q matrix of rank q based on previous results. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 10 / 35
Find the distribution of (Cˆβ d) (σ 2 C(X X) C ) 1 (Cˆβ d). opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 11 / 35
By Result 5.10, the distribution is χ 2 q(φ), where φ = 1 2 (Cβ d) (σ 2 C(X X) C ) 1 (Cβ d) = 1 2σ 2 (Cβ d) (C(X X) C ) 1 (Cβ d). opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 12 / 35
Show that Cˆβ and SSE are independent. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 13 / 35
[ŷ ] ˆε = [ P X y (I P X )y ([ PX ] N I P X ([ ] Xβ N 0 ] = Thus, ŷ and ˆε are independent. [ PX Xβ, I P X [ PX ] y I P X ], σ 2 [ PX 0 0 I P X [ ] ) σ 2 I P X I P X ]). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 14 / 35
Cβ estimable = A C = AX. Cˆβ = C(X X) X y = AX(X X) X y = AP X y = Aŷ. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 15 / 35
SSE = ˆε ˆε. Cˆβ is a function of only ŷ and SSE a function of only ˆε, Cˆβ and SSE are independent. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 16 / 35
We could alternatively have used Result 5.16: C(X X) X (σ 2 I)(I P X ) = σ 2 C(X X) X (I P X ) = σ 2 C(X X) (X X P X ) = σ 2 C(X X) (X X ) = 0. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 17 / 35
Now note that independence of Cˆβ and SSE = (Cˆβ d) (σ 2 C(X X) C ) 1 (Cˆβ d) are independent. and SSE σ 2 (n r) = y (I P X )y σ 2 = ˆσ2 (n r) σ 2 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 18 / 35
We have previously shown that SSE σ 2 χ 2 n r. Thus, and (n r)ˆσ 2 σ 2 χ 2 n r ˆσ 2 σ 2 χ2 n r/(n r). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 19 / 35
It follows that (Cˆβ d) (σ 2 C(X X) C ) 1 (Cˆβ d)/q ˆσ 2 /σ 2 = (Cˆβ d) (C(X X) C ) 1 (Cˆβ d) qˆσ 2 F q,n r (φ), F where opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 20 / 35
as defined previously. φ = 1 2σ 2 (Cβ d) (C(X X) C ) 1 (Cβ d) We can use F to test H 0 : φ = 0 H 0 : Cβ d = 0 ( (C(X X) C ) 1 is PD.) H 0 : Cβ = d. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 21 / 35
To test H 0 : Cβ = d at level α, we reject H 0 iff F F q,n r,α where F q,n r,α is the upper α quantile of the F q,n r distribution. By Result 5.13, the power of the test is a strictly increasing function of φ. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 22 / 35
Now suppose that H 0 : c β = d is testable. By arguments analogous to the previous F case, it is straightforward to show that ( ) c ˆβ d t ˆσ 2 c (X X) c t c β d n r. σ 2 c (X X) c Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 23 / 35
We can conduct tests of H 0 : c β = d against H A1 : c β < d H A2 : c β > d, H A : c β d or by comparing the observed value of t to the t n r distribution. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 24 / 35
Returning to the F-test of testable GLH H 0 : Cβ = d, note that there are multiple ways to express the same null hypothesis. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 25 / 35
For example, suppose y ij = µ i + ε ij (i = 1, 2, 3; j = 1,..., n i ). Find different matrices C 1, C 2 and C 3 C k µ 1 µ 2 µ 3 = 0 µ 1 = µ 2 = µ 3 k = 1, 2, 3. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 26 / 35
[ ] [ ] 1 1 0 µ1 µ 2 C 1 =, C 1 µ = 1 0 1 µ 1 µ 3 [ ] [ ] 1 1 0 µ1 µ 2 C 2 =, C 2 µ = 0 1 1 µ 2 µ 3 [ ] [ ] 1 1 0 µ1 µ 2 C 3 =, C 3 µ =. µ 1/2 1/2 1 1 +µ 2 2 µ 3 opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 27 / 35
Suppose H 01 : C 1 β = d 1 and H 02 : C 2 β = d 2 are both testable and S 1 {β : C 1 β = d 1 } = {β : C 2 β = d 2 } S 2. Show that the F-test of H 01 is the same as the F-test of H 02. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 28 / 35
Recall that X = X P X = X X(X X) X. Thus, X(X X) is a GI of X X. It follows that C k (C kc k ) is a GI of C k. Thus, S k = {C k(c k C k) d k + (I C k(c k C k) C k )z : z R p } for k = 1, 2. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 29 / 35
Because S 1 = S 2, C 1 times any element of S 2 equal d 1 ; i.e., β S 2 = C 1 β = d 1. Thus C 1 [C 2(C 2 C 2) d 2 + (I C 2(C 2 C 2) C 2 )z] = d 1 z R p = C 1 C 2(C 2 C 2) d 2 d 1 + (C 1 C 1 C 2(C 2 C 2) C 2 )z = 0 z R p = C 1 = C 1 C 2(C 2 C 2) C 2 and C 1 C 2(C 2 C 2) d 2 = d 1 by Result A.8. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 30 / 35
Now C 1 = C 1 C 2(C 2 C 2) C 2 = C 1 = C 1 P C 2 = C 1 = P C 2 C 1 = C(C 1) C(P C 2 ) = C(C 2). Repeating the entire argument with the roles of C 1 and C 2 reversed gives C(C 2) C(C 1) so that C(C 2) = C(C 1). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 31 / 35
Because C 1 β = d 1 and C 2 β = d 2 are both testable, C 1 and C 2 are both full-column rank = q. a unique nonsingular matrix q q B C 1 = C 2B. We have previously shown that C 1 = P C 2 C 1 = C 2[(C 2 C 2) ] C 2 C 1. Thus, B = [(C 2 C 2) ] C 2 C 1 and B = C 1 C 2(C 2 C 2). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 32 / 35
We have also shown previously that C 1 C 2(C 2 C 2) d 2 = d 1 = B d 2 = d 1. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 33 / 35
Now consider the quadratic form (C 1 b d 1 ) (C 1 (X X) C 1) 1 (C 1 b d 1 ) = (B C 2 b B d 2 ) (B C 2 (X X) C 2B) 1 (B C 2 b B d 2 ) = (C 2 b d 2 ) BB 1 (C 2 (X X) C 2) 1 (B ) 1 B (C 2 b d 2 ) = (C 2 b d 2 ) (C 2 (X X) C 2) 1 (C 2 b d 2 ). This is true for b R p including ˆβ and the true parameter vector β. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 34 / 35
the F statistics for testing H 01 : C 1 β = d 1 and H 02 : C 2 β = d 2 are identical, as are the noncentrality parameters associated with those F statistics. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 35 / 35