Associated Hypotheses in Linear Models for Unbalanced Data

Transcription

1 University of Wisconsin Milwaukee UWM Digital Commons Theses and Dissertations May 5 Associated Hypotheses in Linear Models for Unbalanced Data Carlos J. Soto University of Wisconsin-Milwaukee Follow this and additional works at: Part of the Mathematics Commons, and the Statistics and Probability Commons Recommended Citation Soto, Carlos J., "Associated Hypotheses in Linear Models for Unbalanced Data" (5). Theses and Dissertations. Paper 84. This Thesis is brought to you for free and open access by UWM Digital Commons. It has been accepted for inclusion in Theses and Dissertations by an authorized administrator of UWM Digital Commons. For more information, please contact kristinw@uwm.edu.

2 ASSOCIATED HYPOTHESES I LIEAR MODELS FOR UBALACED DATA by Carlos J. Soto A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Science in Mathematics at The University of Wisconsin-Milwaukee May 5

3 ABSTRACT ASSOCIATED HYPOTHESES I LIEAR MODELS FOR UBALACED DATA by Carlos J. Soto The University of Wisconsin-Milwaukee, 5 Under the Supervision of Professor Dr. Jay Beder When looking at factorial experiments there are several natural hypotheses that can be tested. In a two-factor or a b design, the three null hypotheses of greatest interest are the absence of each main effect and the absence of interaction. There are two ways to construct the numerator sum of squares for testing these, namely either adjusted or sequential sums of squares (also known as type I and type III in SAS). Searle has pointed out that, for unbalanced data, a sequential sum of squares for one of these hypotheses is equal (with probability ) to an adjusted sum of squares for a non-standard associated hypothesis. In his view, then, sequential sums of squares may test the wrong hypotheses. We give an exposition of this topic to show how to derive the hypothesis associated to a given sequential sum of squares. ii

4 TABLE OF COTETS Introduction Linear Models. otation Least Squares Hypotheses Restrictions Hypothesis in Linear Models 6 3. Linear Hypothesis ested Hypothesis and Sequential Sum of Squares Associated Hypothesis Some Associated Hypotheses 9 4. Rows Columns Additivity Conclusion 9 REFERECES 3 iii

5 LIST OF TABLES 4. Soil Data iv

6 ACKOWLEDGMETS The author wishes to thank Professor Jay Beder and my committee members Professor Jugal K Ghorai and Professor Vytaras Brazauskas. Also I d like to thank my peers, brothers and friends, especially Andrew Kaldunski, who have all supported me. v

7 Introduction When looking at factorial experiments there are several natural hypotheses that can be tested. In a two-factor or a b design, the three null hypotheses of greatest interest are the absence of each main effect and the absence of interaction. We are under the assumption that no cell in our experiment is empty and further the cells could be unbalanced, that is that, they do not necessarily have the same sample size in each cell. To test our hypothesis we first fit a linear model. Then there are two ways to construct the numerator sum of squares for testing these hypotheses, namely either adjusted or sequential sums of squares (also known as type I and type III in SAS). In Linear Models [, Section 7.(f)] Searle derives the hypotheses associated with various sequential sum of squares in a a b design. He has pointed out that, for unbalanced data, a sequential sum of squares for one of these hypotheses is equal (with probability ) to an adjusted sum of squares for a non-standard associated hypothesis. In his view, then, sequential sums of squares may test the wrong hypotheses. Some of the hypotheses are not directly given in terms of the cell means µ ij, and the computations are somewhat complicated and ad hoc. We then give an exposition to build up to Theorem 3.3 and Corollary 3., and then derive from them hypotheses associated with sequential sum of squares in a a b design. Linear models are first explored and the usual methods of testing hypotheses are given. Several theorems that are presented without proofs are from Linear Models and Design [].

8 Linear Models For testing hypotheses we will be fitting a linear full model of E(Y ) = Xβ and comparing it to a restricted model as that under our null hypothesis, H. To fit our model the method of least squares will be used. For the full and restricted models the sum of squared errors are denoted as SSE and SSE R respectively. If SSE and SSE R are significantly close to each other, then we will not reject H. It s important to note that SSE R SSE since a restricted model s error can t be less than a full model. Thus, if SSE R is significantly greater than SSE we will reject H which is equivalent to SSE R /SSE being significantly large.. otation Working with an a b design there is several useful notation which we ll be using. First, let p = ab which is the number of cells. Let µ ij denote the mean of the cell ij. ext, n i is used to denote the sum of observations in the ith row. That is n i = b j= n ij. Similarly for columns n j = a observations is = n = b j= i= n ij. Thus the total number of a i= n ij = b j= n j = a i= n i In general when the lower and upper bounds of summations are obvious the following notation will be used: i = a i= and j = b j=.. Least Squares Since the method of least squares will be used, a few notes about the method will be made. However, since the method is quite standard, derivations of most equations are omitted. Let X denote the p design matrix. Let Y represent the vector of observed values and let Ŷ represents the vector of fitted values, both of which are. The

9 3 method of least squares is that which minimizes SSE := Y Ŷ = j= (Y j Ŷj) To minimize the SSE, Ŷ = X(X X) X Y and since Ŷ = X ˆβ then ˆβ = (X X) X Y. Throughout this paper, the hat matrix is defined to be P = X(X X) X. The matrix P is a projection matrix from R to R where is the total number observations as defined in the previous section. It is useful to think of R as the observation space. Ŷ = P Y is the orthogonal projection of Y onto V := R(X), the column space of X..3 Hypotheses In a a b design there are six main hypotheses of interest: only A present, only B present, absence of each main effect and absence of interaction. The hypotheses of only A present consists of the equations µ = = µ b µ = = µ b. µ a = = µ ab Let ρ i denote the ith row mean. That is ρ i = b b j= µ ij. A natural hypotheses would be H : ρ = ρ = = ρ a, that is all rows have the same mean. This is referred to as A effect absent. The hypotheses of only B present consists of the equations µ = = µ a

10 4 µ = = µ a. µ b = = µ ab Let γ j denote the jth column mean. That is γ j = a a i= µ ij. A natural hypotheses would be H : γ = γ = = γ b, that is all columns have the same mean. This is referred to as B effect absent. Another main hypothesis tested is that of interaction of A and B, defined to be lack of additivity. The factors A and B are considered to be additive if a change from row i to i can be achieved by adding the same constant to each cell mean of row i. ote that the constant need not be the same amongst different pairs of rows. Whilst it is true that additivity holds between any pair of rows it s more useful to equivalently consider additivity between row and row i, i =,..., a. The hypothesis of additivity consists of the (a )(b ) equations µ ij µ j = µ i µ, i =,..., a, j =,..., b The last main hypothesis tested is that of no effect present, that is, H : µ ij equal for all ij..4 Restrictions In our hypothesis of interest some sort of restriction or linear constraint is made on β where β R p. It is useful to think of R p as the parameter space of the model. A hypothesis which makes a linear constraint is considered a linear hypothesis. There are several ways in which a linear constraint can be written and few are given below.

11 5 Lemma.. The following are equivalent (i) There exists a subspace U R p such that β U (ii) There exists a subspace W R p such that β W (iii) There exists a matrix W and a vector β such that β = W β U is a set of contrast vectors. From this Lemma it s important to note that W and U are orthogonal complements of each other. Since the hypotheses we will be testing are defined by a linear constraint then it s important to know how the model acts under such a linear constraint. It seems natural that a linear model under a linear constraint should still be a linear model. We thus have the following theorem Theorem.. If E(Y ) = Xβ and if β is subject to a linear constraint β W, then the constrained model is also a linear model E(Y ) = X β where X = XW and appropriate β. We will be assuming that X has full rank and thus W (X) = (), where (X) is the nullspace of X, that is (X) = {c Xc = }. Therefore X has full rank as well.

12 6 3 Hypothesis in Linear Models 3. Linear Hypothesis We have already shown how to fit the unrestricted model and now that we know how to restrict our β then fitting a restricted model is similar in method. Let H : β W be our null hypothesis, where W is a subspace of R p and let V := X(W ), a subspace of V. To fit the restricted model, we find the value of Ŷ V that minimizes the distance to Y. Thus, Ŷ is the orthogonal projection of Y onto V, and we define SSE R := Y Ŷ. At this point, we have Y, Ŷ, and Ŷ and since Ŷ, Ŷ V = R(X) then we can see that these three vectors form a right triangle. Thus we have, Y Ŷ = Y Ŷ + Ŷ Ŷ. If we denote SS(H ) = Ŷ Ŷ (notation will become apparent shortly) then SSE R = SSE + SS(H ) SSE R SSE = + SS(H ) SSE Recall that we will reject H when SSE R /SSE is significantly large. Thus this is equivalent to rejecting when SS(H ) SSE for testing H. is large. Thus, we call SS(H ) the sum of squares 3. ested Hypothesis and Sequential Sum of Squares Thus far, we have been interested in testing a single linear hypothesis. Generally an analysis of variance tests several hypotheses. Several interesting things occur when

13 7 there are more than one hypothesis and we ll begin to explore this. Definition 3.. A hypothesis H () is nested in hypothesis H () if H () implies H (). Since in our linear model β R p, then from Lemma. a general hypothesis H (i) may be expressed as β W (i) for an appropriate subspace W (i) R p. Thus H () is nested in H () if and only if W W. Applying Lemma., H (i) may be expressed as β U (i). Thus H () is nested in H () if and only if U U. The choice of H is the simplest possible hypothesis model. Generally this is the mean of all cells being equal. In general, suppose W W which thus means the hypothesis β W is nested in β W. Let V i = X(W i ) be the image of W i so thus V = R(X) the column space of X. Thus V V V Theorem 3.. We have Ŷ Ŷ V V and Ŷ Ŷ V V. In particular, Y Ŷ, Ŷ Ŷ and Ŷ Ŷ are pairwise orthogonal. Suppose we wanted to test β W, given that β W. ote that Y Ŷ = Y Ŷ + Ŷ Ŷ + Ŷ Ŷ = Y Ŷ + Ŷ Ŷ As when we defined SS(H ), if we divided both sides by SSE = Y Ŷ then, Y Ŷ SSE = + Ŷ Ŷ SSE Thus Ŷ Ŷ is appropriate for testing β W, given that β W. We thus denote this as SS(β W β W ) = Ŷ Ŷ

14 8 Keeping with this notation Ŷ Ŷ = Ŷ Ŷ + Ŷ Ŷ Ŷ Ŷ = Ŷ Ŷ Ŷ Ŷ SS(β W β W ) = SS(β W ) SS(β W ) Since SS(β W ) = Ŷ Ŷ is the simplest model, it is generally referred to as the sum of squares for the model. Thus since SS(β W ) = S(β W β W ) + SS(β W ), then S(β W β W ) and SS(β W ) are referred to as sequential sums of squares. This is so because if we started with Y Ŷ and add SS(β W ) = Ŷ Ŷ and SS(β W β W ) = Ŷ Ŷ sequentially, then we build the sum of squares for the model. To generalize Theorem 3. we can consider H : β W i where W W W k R p and the W i are distinct. Then V V V. Theorem 3.. We have Ŷ Ŷk V V k and Ŷj Ŷj V j V j. In particular, Y Ŷ, Ŷ Ŷk, Ŷk Ŷk,..., Ŷ Ŷ are pairwise orthogonal. 3.3 Associated Hypothesis Definition 3.. Given a hypothesis H () nested in H (), the hypothesis H is associated to SS(H () H () ) and SS(H () H () ) = SS(H ). Theorem 3.3. Consider the model E(Y ) = Xβ, where X p has full rank, and let W W R p. Then there is a unique subspace W satisfying SS(β W ) = SS(β W β W ), and we have df(β W ) = df(β W β W ).

15 9 The subspace is given by W = R(T P ), where T = (X X) X and P is defined as follows. Let V i = X(W i ), V = R(X) be the column space of X, and let P be P i be the orthogonal projections of R on V and on V i, respectively. Then P = P P + P. Corollary 3.. The subspace U is given by U = (P T ) 4 Some Associated Hypotheses Throughout the rest of this paper instead of working with an a b design, we ll be working with a 3 design. 4. Rows Theorem 4.. Consider the hypotheses H () :all µ ij equal and H () :Only A present. Clearly H () is nested in H (). The hypothesis associated with the sequential sum of squares SS(H () H () ) is H : ρ = ρ = = ρ a where ρ i = n i j n ijµ ij. Thus let H () : µ = µ = µ 3 = µ = µ = µ 3 and H () : µ = µ = µ 3 and µ = µ = µ 3. The hypothesis associated with SS(H () H () ) of is H : ρ = ρ. Proof. We need P = P P + P by Theorem 3.3. Let k be the k vector of s, let J n m be the n m matrix of s, and J n the n n matrix of s. For our unrestricted full model we have

16 n n X = n3 n n n3 A B C Thus P = X(X X) X = where D E F A = n J n B = n J n C = n 3 J n3 D = n J n E = n J n F = n 3 J n3 Under our null hypothesis of equality of means in rows we want µ = µ = µ 3 and µ = µ = µ 3. We have two equations so we have two free parameters, say µ and µ. So by Theorem. we want a matrix W such that µ µ µ 3 µ µ µ 3 = W µ µ

17 We can see that W = since Therefore, we have X = XW. µ µ µ µ µ µ = W µ. µ Thus X = n n and P = X (X X ) X = A B where A = n J n and B = n J n. For the simplest model we have µ = µ = µ = µ = µ 3 = µ 3. We have one free parameter, say µ. By Theorem. so we need a W such that µ µ µ 3 µ µ µ 3 [ = W µ ].

18 It is clear that, W = since µ µ µ µ µ µ = W [ µ ] Therefore, we have X = XW =. Further P = X (X X ) X = J Since P is a projection matrix from R to R, all P matrices are matrices. We have P = P P + P. The only matrix we need now is T where T = (X X) X =. n n n n T = n 3 n3 n n n n n 3 n3 Remark 4.. The very next step is to calculate P T and then according to Corollary 3.. find its nullspace. Let us introduce the notation C(P T ) which we ll think of as the core of our matrix. The matrix P T has several repeated rows, in fact only 6 unique rows. Thus 6 rows can be eliminated when finding the nullspace. The 6 unique rows compose C(P T ) and thus C(P T ) is 6 6. To picture P T, the first row of C(P T ) is repeated n times, the second row is repeated n times, the third is repeated n 3 times, etc. ote that (P T ) = (C(P T )) since eliminated repeated rows do not effect the nullspace. C(P T ) = A J 3 B J 3 6

19 3 where A = n n + n n n + n n n n n n 3 n + and B = n n + n n n + n n n n n n 3 n + The next step is to put the matrix in reduced row echelon form. It s important to note that the Gaussian elimination method is not efficient and should be avoided if replicating this result. The reduced matrix is as follows. n n n n 3 n n n n 3 n 3n n n 3. n n 3 n n 3

20 4 ext to find the nullspace we need to solve n n n n 3 n n n n 3 n 3n n n 3 n n 3 n n 3 c c c 3 c c c 3 =. Thus we have c = n n n n 3 c 3 c = n n n n 3 c 3 c 3 = n 3n n n 3 c 3 c = n n 3 c 3 c = n n 3 c 3 If we let c 3 = n 3 n, then c = n n c = n n c 3 = n 3 n. c = n n c = n n Thus n n µ + n n µ + n 3 n µ 3 n n µ n n µ n 3 n µ 3 = n n µ + n n µ + n 3 n µ 3 = n n µ + n n µ + n 3 n µ 3

21 5 n [n µ + n µ + n 3 µ 3 ] = n [n µ + n µ + n 3 µ 3 ] Recall that ρ i = n i j n ijµ ij. Therefore, the associated hypothesis is ρ = ρ, as claimed. Example 4.. Lets see the hypothesis H : ρ = ρ as applied to the following experiment. Consider the following data: Searle claims that the sequential sum of Variety Soil 3 6,, 3, 5 4,, 5, 9, 8 3 8, 9, Table 4.: This is a table of soil and variety squares for Soil only tests the hypothesis (3µ 7 +µ +µ 3 ) = (4µ 8 +µ +3µ 3 ). Using the previous theorem to derive this hypothesis we have 3 X = 4 3

22 6 Using statistical software we get P T = As we can see the first row is repeated n = 3 times, the fourth row is repeated n = times and etc. Since our total sample size is = 5 we ended up with a 5 6 matrix. The core of P T is C(P T ) =

23 7 Thus we have c = 8c 7 3 c = 6c 3 c 3 = 6c 3 c = 4 3 c 3 c = c 3 3 If we let c 3 = n 3 n = 3 8 then c = 8 7 ( 3 8 ) = 3 7 c = 6 ( 3) = 8 7 c 3 = 6( 3) = 8 7 c = 4 3 ( 3 8 ) = 4 8 c = ( 3) = Therefore H : 3µ 7 + µ 7 + µ 7 3 4µ 8 µ 8 3µ 8 3 =. So, H : 3µ 7 + µ 7 + µ 7 3 = 4µ 8 + µ 8 + 3µ 8 3, or H : (3µ 7 + µ + µ 3 ) = (4µ 8 + µ + 3µ 3 ), as Searle claims. 4. Columns Theorem 4.. Consider the hypotheses H () :all µ ij equal and H () :Only B present. Clearly H () is nested in H (). The hypothesis associated with the sequential sum of squares SS(H () H () ) is H : γ = γ = = γ b where γ i = n j i n ijµ ij. Thus let H () : µ = µ = µ 3 = µ = µ = µ 3 and H () : µ = µ, µ = µ, and µ 3 = µ 3. The hypothesis associated with SS(H () H () ) of is H : γ = γ = γ 3.

24 8 Proof. First we will note that the full model and simplest model are the same as in the hypothesis of row equality in the previous section. Thus P, P, and T are the same as in the previous section. Under our null hypothesis of equality of means in columns we want µ = µ, µ = µ, and µ 3 = µ 3. We have three equations so we have three free parameters, say µ, µ, and µ 3. So by Theorem. we want a matrix W such that µ µ µ 3 µ µ µ 3 = W µ µ µ 3 Thus we can see that W =

25 9 since Thus we have µ µ µ 3 µ µ µ 3 = W µ µ µ 3. n n X = XW = n3. n n n3 Further J n J n n n J n J n n n J P = X (X X ) X = n3 J n3 n 3 n 3 J n J n n n J n J n n n J n3 J n3 n 3 n 3 [ Again, P T has 6 unique rows and C(P T ) = A B ]

26 where and A = B = n + n n + n n n 3 + n 3 n n n + n n 3 n n n + n 3 n 3 + n 3 The matrix C(P T ) in reduced row echelon form is as follows: n n n 3 n n n n n 3 n n n 3 n 3 n n n 3 n n n n n 3

27 Thus to find the nullspace of C(P T ) we need to solve n n n 3 n n n n n 3 n n n 3 n 3 n n n 3 n n n n n 3 c c c 3 c c c 3 = Thus we have c = n n n n c n 3n n n 3 c 3 c = n n c c 3 = n 3 n 3 c 3 If we let c = n n and c 3 = n 3 n, then c = n n n n c n 3n c = n n n n 3 c 3 c = n n c 3 = n 3 n 3. c = n n So n n µ + n n µ + n 3 n 3 µ 3 n n µ + n n µ + n 3 n µ 3 = [( n n µ + n n µ ) ( n n µ + n n µ )] [( n n µ + n n µ ) ( n 3 n 3 µ 3 + n 3 n µ 3 )] = (n µ +n µ ) n n (n µ +n µ ) = n (n µ +n µ ) n 3 (n 3 µ 3 +n 3 µ 3 )

28 Recall that γ i = n j i n ijµ ij. Thus γ γ = γ γ 3. Similarly, if we go back to our system of equations and let c = n n and c 3 = then γ γ = Thus γ = γ and by our previous equations, γ = γ = γ 3, as claimed. Example 4.. Continuing using the data from Example 4., we will be examining the hypothesis of of variety only. C(P T ) = After reducing the matrix we get

29 3 Thus we have c = 9 7 c 5 7 c 3 c = c c 3 = 3 c 3 c = c 7 If we let c = n n = 3 and c 3 = n 3 n 3 = 3 5 then c = 6 7 c = 3 c 3 c 3 = 5 c = 8 7 Thus 6 7 µ + 3 µ + 5 µ µ + 3 µ µ 3 = 3 µ + 3 µ 3 7 µ 4 7 µ = 3 7 µ µ 5 µ µ 3 3 (µ + µ ) 7 (3µ + 4µ ) = 7 (3µ + 4µ ) 5 (µ 3 + 3µ 3 ) Recall that γ i = n j i n ijµ ij, so, γ γ = γ γ 3.

30 4 Similarly if we let c = n n = 3 and c 3 = then we get c = 3 7 c = 3 c 3 = c = 4 7 Thus 3 7 µ + 3 µ 4 7 µ + 3 µ =, or µ 3 + µ 3 = 3µ 7 + 4µ 7, so γ = γ and thus γ = γ = γ Additivity Theorem 4.3. Consider the hypotheses H () :Only A present and H () :Additivity. H () is nested in H (). Then the hypothesis associated with the sequential sum of squares SS(H () H () ) is H : γ j = n j i n ijρ i j where γ i = n j i n ijµ ij. Thus for our 3 case we have H () : µ = µ = µ 3 and µ = µ = µ 3 and H () : µ µ = µ µ and µ 3 µ 3 = µ µ. The hypothesis associated with SS(H () H () ) is H : γ = n (n ρ + n ρ ) and γ = n (n ρ + n ρ ). Proof. This hypothesis actually ending up being the most challenging to calculate, so instead of doing this with unbalanced data we will be using balanced data.

31 5 For the unrestriced model we have n n X = n n n n and J n n J n n P = X(X X) X = J n n J. n n n J n J n n For the model of A only in the balanced case we have P = X (X X ) X 3n = n. 3n n

32 6 Under our hypothesis of additivity we have µ µ = µ µ and µ 3 µ 3 = µ µ. Thus we want W such that µ µ µ 3 µ µ µ 3 = W µ µ µ 3 µ Thus we want Therefore, we have W =. n n X = XW = n n n n n n n n

33 7 and P = X (X X ) X = J 3n n J n J n J 3n n J n 3n J n 3n J n 3n J n 3n J n 3n J n 3n J n 3n J n 3n J n J n J n J 3n n J n J n 3n J n. Thus we have P = P P + P = J 3n n J n J n J 3n n J n 3n J n 3n J n 3n J n 3n J n 3n J n 3n J n 3n J n 3n J n J n J n J 3n n J n J n 3n J n and C(P T ) = 3n 3n 3n 3n 3n 3n 3n 3n 3n 3n 3n 3n.

34 8 After some algebra we have the reduced form of. To find the nullspace of this matrix, so we want to solve c c c 3 c c c 3 = We have c = c c 3 c = c c 3 = c 3 c = c c 3. In this case, deriving the associated hypothesis isn t straightforward. So instead, we can verify that the associated hypothesis is correct. Consider γ = (ρ + ρ ) then n (nµ + nµ ) = ( 3n (nµ + nµ + nµ 3 ) + 3n (nµ + nµ + nµ 3 ))

35 9 (µ + µ ) = ( 3 (µ + µ + µ 3 ) + 3 (µ + µ + µ 3 )) µ + µ = 3 (µ + µ + µ 3 + µ + µ + µ 3 ) 3 µ 3 µ 3 µ µ 3 µ 3 µ 3 = Thus if we let c = 3 and c 3 = 3, then c = 3 c = 3 c 3 = 3, c = 3 thus verifying the associated hypothesis. 5 Conclusion We have shown to how use Theorem 3. and Corollary 3. [] to derive certain associated hypotheses, namely those for sequential sum of squares of the form SS(no ef f ect Only A present). The corresponding computations for sequential sum of squares such as SS(Only A present o interaction) turn out to be much more difficult for unbalanced data. This needs to be carried out to make the theorem truly usable.

36 3 REFERECES [] Beder, J. Linear Models and Design. Unpublished Manuscript, 4. [] Searle, S.R. Linear Models. Wiley, ew York, 97. [3] Searle, S.R. Linear Models For Unbalanced Data. Wiley, ew York, 987.