One-Way ANOVA Model. group 1 y 11, y 12 y 13 y 1n1. group 2 y 21, y 22 y 2n2. group g y g1, y g2, y gng. g is # of groups,

Transcription

1 One-Way ANOVA Model group 1 y 11, y 12 y 13 y 1n1 group 2 y 21, y 22 y 2n2 group g y g1, y g2, y gng g is # of groups, n i denotes # of obs in the i-th group, and the total sample size n = g i=1 n i. 1

2 Model Equation The unknown parameters are y ij = µ + α i + e ij, e ij iid N(0, σ 2 ). (µ, α 1,..., α g ). 2

3 Matrix Form For simplicity, consider a simple case g = 3, n 1 = 3, n 2 = 2 and n 3 = 2. Write the one-way ANOVA model in matrix form y 11 y 12 y 13 y 21 y 22 y 31 y 32 = µ α 1 α 2 α 3 + err. The model is over-parameterized, i.e, columns of X are linearly dependent. 3

4 Projection Matrix Let Z denote the design matrix X without the 1st column. Is it easy to show that C(Z) = C(X) and Z is of full rank. So the projection matrix can be computed via Z: H = Z(Z t Z) 1 Z t. For the g = 3 example, we have H n n = 1 3 J J J2 2, where J m m denotes an m m matrix with all entries being 1. 4

5 The LS fit for y ij is the corresponding group mean ŷ ij = ȳ i Residuals r ij = y ij ŷ ij = y ij ȳ i RSS g n i ( yij ȳ i ) 2, i=1 j=1 i.e., the within-group variation. 5

6 Comparing Nested Modesl and F -tests Are all groups really different? State the hypothesis in terms of models H a : y ij = µ + α i + e ij H 0 : y ij = µ + e ij They are two nested models, then we can use F -test. (RSS 0 RSS a )/(g 1) RSS a /(n g) F g 1,n g, under H 0. The test statistic can also written as g i=1 n i(y i y ) 2 /(g 1) i,j (y ij y i ) 2 /(n g) = Between-group Variation/(g 1) Within-group Variation/(n g). How to use F-test to decide whether we should merge some groups? 6

7 LS Coefficient Estimation As mentioned before, the LS estimates for (µ, α 1,..., α g ) are not unique since the design matrix is not of full rank. If you fit a one-way ANOVA model using lm, R will return you a g-dim regression coefficient vector. The LS estimates returned by R vary depending on the contrast option. 7

8 Since the one-way ANOVA model is over-parameterized, to obtain an estimate of the coefficients, we need to put some constraint on µ or α i s. µ = 0: What s the design matrix X? How to interpret the parameters? (the default case; lm X -1) α 1 = 0: What s the design matrix X? How to interpret the parameters? (contr.treatment) α i = 0: What s the design matrix X? How to interpret the parameters? (contr.sum) Suffices to remember the default case; the interpretations are not important. 8

9 µ = 0 (i.e., regression without the intercept): y 11 y 12 y 13 y 21 y 22 y 31 y 32 = α 1 α 2 α 3 + err = α α α err. So α i denotes the (population) mean of group i. 9

10 contr.treatment (α 1 = 0) y 11 y 12 y 13 y 21 y 22 y 31 y 32 = µ α 2 α 3 + err = µ1 3 (µ + α 2 )1 2 (µ + α 3 )1 2 + err. So µ denotes the (population) mean of group 1 and α 2 denotes the difference between the mean of group 1 and mean of group 2, etc. 10

11 contr.sum ( i α i = 0) y 11 y 12 y 13 y 21 y 22 y 31 y 32 = µ α 1 α 2 + err = (µ + α 1 )1 3 (µ + α 2 )1 2 (µ α 1 α 2 )1 2 + err. So µ denotes the average of the group (population) means, and α i denotes the difference between the mean of group i and µ. 11

12 Contrasts A linear combination of the group means g i=1 c iα i is called a contrast if i c i = 0. α 1 α 2 : c 1 = 1, c 2 = 1, and other c i s = 0. (α 1 + α 2 )/2 α 3 : c 1 = c 2 = 1/2, c 3 = 1, and other c i s = 0. The LS estimate (i.e., BLUE) of g i=1 c iα i is g i=1 c iȳ i with s.e. ˆσ i c2 i /n i. The (individual) (1 α) CI for g i=1 c iα i is given by i c i ȳ i ± t α/2 n g ˆσ i c 2 i n i. 12

13 Decompose BSS Using Orthogonal Contrasts There is a one-to-one correspondence between the following two subspaces: each has dimension (g 1), but one is from R g and one is from R n. The contrast space {c R g : c c g = 0.}. M A.0 = {x R n : x M A and x M 0 } where M A denotes the estimation space for one-way ANOVA model and M 0 denotes the estimation space for an intercept-only model. 13

14 Vectors in M A.0 take the following form b 1 1 n1 b 2 1 n2, where g n i b i = 0. i=1 b g 1 ng Apparently, this vector above corresponds to a contrast vector (n 1 b 1,..., n g b g ) t. For any contrast vector c, the corresponding LS estimate of c t α is given by and vector a M A.0. a t y = ( c1 n 1 1 t n 1,..., c g n g 1 t n g ) y, 14

15 Let a 1,..., a g 1 be an orthogonal basis for M A.0, then the norm-square of any vector projected to M A.0 can be decomposed as sum of norm-square of its projection to each one-dim space spanned by a j : SS(M A.0 ) = SS SS g 1, SS j = (y t a j ) 2 / a j 2. Define orthogonal contrasts to be two contrast vectors c 1 and c 2 such that i c i1c i2 /n i = 0. For balanced design, it means c 1 and c 2 are also orthogonal as two g-dim vectors. Let c 1,..., c g 1 be a set of (pairwise) orthogonal contrasts. For each c i, define a j = ( c j1 n 1 1 t n 1,..., c jg n g 1 t n g ) t. Then (a 1,..., a g 1 ) form an orthogonal basis for M A.0, and ( ) 2 i ȳi c ji SS j = (y t a j ) 2 / a j 2 = Which contrast has the largest SS? i c2 ji /n i. 15