Analysis of Variance. ภาว น ศ ร ประภาน ก ล คณะเศรษฐศาสตร มหาว ทยาล ยธรรมศาสตร

Transcription

1 Analysis of Variance ภาว น ศ ร ประภาน ก ล คณะเศรษฐศาสตร มหาว ทยาล ยธรรมศาสตร

2 Outline Introduction One Factor Analysis of Variance Two Factor Analysis of Variance ANCOVA MANOVA

3 Introduction A statistical technique used for making inferences about a multitude of parameters relating to population means. We do this not by analyzing variance, but by analyzing variation One Factor ANOVA: Samples from m populations Hypothesis: All the population means are equal Two Factor ANOVA: Two factors may affect the means Hypothesis: A specified factor does not affect the mean

4 1. ONE FACTOR ANOVA

5 One Factor ANOVA Consider m samples, each of size n Suppose that for i = 1,,m these samples are independent Sample i comes from population i that is normally distributed with mean µ i and variance σ 2 Null hypothesis is: H 0 : μ 1 = μ 2 =... = μ m = μ Against H 1 : Not all means are equal

6 One Factor ANOVA If H 0 is reasonable, then practically there is only one population Suppose we express mean of sample i as µ i = µ + γ i Let Y ij be the j th sample observation from population i, then we have Y ij = µ + γ i + ε ij Thus H 0 can be restated as γ 1 = γ 2 = = γ m = 0

7 One Factor ANOVA The sample data structure is: Population 1: Y 11, Y 12,, Y 1n Population 2: Y 21, Y 22,, Y 2n Population m: Y m1, Y m2,, Y mn The total sample size is N = m*n

8 One Factor ANOVA Assumptions The random errors, ε ij, are: Normally distributed with mean zero Independent of one another Each population has the same variance, σ 2 Note: We can test the validity of these assumptions before performing ANOVA as well

9 One Factor ANOVA Notations: be mean of the sample i Y i We use Y i to estimate µ i Y be mean of the overall sample That is 1 (We use to estimate µ ) Y m i = 1 γ i is estimated by i is estimated by m = Y Y $ γ i = Yi Y ε $ ε i = Yij Yi i

10 Sums of Squares Decomposition If we had only 1 population, we would view the variance as deviations of observations from the overall mean However, we potentially have m distinct populations The deviations can be composed of Deviations of observations from each population mean, Yij μi, and Deviations of population means from the overall mean, μ μ i

11 Sums of Squares Decomposition This can be expressed as: Summed across i and j, we have: 2 2 ij μ = Yij μi + μi μ ( Y ) ( ) = [( Y μ ) + ( μ μ)] ij i i ij μi Yij μi μi μ μi μ = ( Y ) + 2( )( ) + ( ) ( Yij μ) = ( Yij μi ) + n( μi μ) i j i j i 2 2 ( Yij μi ) nγi i j i = +

12 Sums of Squares Decomposition The expression is estimated by: ( ) 2 Yij 2 N Y = ( n 1) Si 2 + nγˆ i 2 ij, i i Where S 2 i = j ( Y ) 2 ij Yi n 1 Or SST = SSE + SSP

13 Variation The TOTAL variation about the common mean, SST, can be decomposed into two portions: A portion due to error, SSE, that represents variation within a population which we are unable to explain further, and A portion due to process, SSP, that represents variation from population to population due to differences in the population means

14 F Statistic To fairly compare the two portions of the total variation, we divide each portion by its degrees of freedom to obtain the corresponding mean squares : MSE = SSE/(N m) MSP = SSP/(m 1) The ratio F calc = MSP/MSE

15 Hypothesis Testing Reject H 0 if: F calc F m 1, N m, α Where α is the significant level Do not reject H 0 : Otherwise

16 Table for One Factor ANOVA In conclusion: SSP = SSE = i i 2 i i 2 i nˆ γ = n( Y Y) ( n 1) S i 2

17 Table for One Factor ANOVA Source SS df MS (variation) (variance) Process SSP m -1 MSP F MSP/ MSE Error SSE N - m MSE Total SST N -1

18 One Factor ANOVA Example EXAMPLE 11.1 in Ross (1996) An investigator organized a study of mileages obtainable from three different brands of gasoline. Using 15 identical motors set to run at the same speed, the investigator randomly assigned each brand of gasoline to 5 of the motors. Each motors was then run on 10 gallons of gasoline, with the total mileages obtained as given below

19 One Factor ANOVA Example Gas 1 Gas 2 Gas

20 One Factor ANOVA Example Source SS df MS F (variation) (variance) Process Error 1, Total 2, Y 1

21 2. TWO FACTOR ANOVA

22 Two Factor ANOVA Example EXAMPLE 11.3 in Ross (1996) Four different standardized reading achievement test were administered to each of five students. Their score were as follows:

23 Test No. Two Factor ANOVA Example Student

24 Two Factor ANOVA Example Each value in the set of 20 data points is affected by two factors: The test paper The student who did the test

25 Two Factor ANOVA Suppose there are 2 factors There are m possible levels of factor 1 (Row factor) There are n possible levels of factor 2 (Column factor) Let X ij denote the value of the data obtained when: The first factor is at level i The second factor is at level j

26 Two Factor ANOVA Assume that: All data values are independent Common variance is σ 2 Suppose that the mean value of the data point depends on both factors E[X ij ] = µ + α i + β j µ is grand mean α i is deviation from factor 1 β j is deviation from factor 2

27 Suppose also that Let: Two Factor ANOVA n m i i= 1 j= 1 X ij be the average when factor 1 is i j= 1 = (average the same row) X i n m X ij i= 1 X be the average when factor 2 is j j = m (average the same column) m n Xij i= 1 j= 1 X = be the average of all data values mn α n = β = j 0

28 Two Factor ANOVA We can see that: As well as: EX [ ] i EX [ ] j EX [ ] = μ + α = μ + β = μ i i EX [ X] = α i EX [ X] = β j i i

29 Two Factor ANOVA Then, it is best to estimate these parameters as: ˆ μ = X ˆ α = X ˆ β i j i j X = X X

30 Hypothesis Testing Now there are 2 hypotheses First, there is no row effect H 0 : All α i are 0 H 1 : Not all α i are 0 Second, there is no column effect H 0 : All β j are 0 H 1 : Not all β j are 0

31 Variation Let: SS1 = SS2 = SSE = m n ( X X ) i= 1 n j= 1 m n i m ( X X ) i= 1 j= 1 j 2 2 ( X X X + X ) ij i j 2

32 F Statistic Let: MS1 = SS1/m 1 MS2 = SS2/n 1 MSE = SSE/(m 1)(n 1) F calc1 = MS1/MSE F calc2 = MS2/MSE

33 Hypothesis Testing Reject H 0 for factor 1 if: F calc1 F m 1, (m 1)(n 1), α Where α is the significant level Reject H 0 for factor 2 if: F calc2 F n 1, (m 1)(n 1), α Do not reject H 0 : Otherwise

34 Table of Two Factor ANOVA Source Factor 1 (Row) Factor 2 (Column) SS df MS (variation) (variance) SS1 m 1 MS1 SS2 n 1 MS2 F MS1/ MSE MS2/ MSE Error SSE (m 1)(n 1) MSE

35 Back to the Example Test No. Student

36 Table of Two Factor ANOVA Source Factor 1 (Row) Factor 2 (Column) SS df MS F (variation) (variance) , Error

37 3. ANCOVA

38 Analysis of Covariance The Analysis of Covariance (ANCOVA) is a technique that sits between ANOVA and regression analysis Purposes To increase precision of comparisons between groups by accounting to variation on important variables To adjust comparisons between groups for imbalances in important variables

39 Analysis of Covariance ANCOVA extends ANOVA by including one or more continuous variables that predict the outcome Continuous variables such as these are not part of the main interests, but have an influence on the dependent variables These continuous variables are called covariates

40 Same as ANOVA + Analysis of Covariance Assumptions Independence of the covariate and treatment effect Homogeneity of regression slopes

41 Analysis of Covariance Rough Idea Because the covariate can affect the dependent variable, we adjust for the variation in the covariate between groups using the technique of regression After taking out effects of the covariate, we perform ANOVA on differences of dependent variable between groups

42 Analysis of Covariance Example

43 Table of ANCOVA Source SS df MS F (variation) (variance) Process SSP (adj.) m 1 Error SSE (adj.) N m 1 MSP (adj.) MSE (adj.) MSP/ MSE Total SST (adj.) N 2

44 4. MANOVA

45 Multivariate Analysis of Variance MANOVA is a generalized from of ANOVA, used in cases when there are two or more dependent variables For example, we may conduct a study where we try two different textbooks, and we are interested in the students' improvements in math and physics our hypothesis is that both together are affected by the difference in textbooks

46 Multivariate Analysis of Variance Instead of a univariate F value, we would obtain a multivariate F value (Wilks' lambda) based on a comparison of the error variance/covariance matrix and the effect variance/covariance matrix If the overall multivariate test is significant, we conclude that the respective effect (e.g., textbook) is significant However, our next question would of course be whether only math skills improved, only physics skills improved, or both

47 References Ross, Sheldon M. (1996) Introductory Statistics. McGraw Hill cs/ancova.pdf