Analysis of variance

Transcription

1 Analysis of variance 1

2 Method If the null hypothesis is true, then the populations are the same: they are normal, and they have the same mean and the same variance. We will estimate the numerical value of this common variance in two distinct ways: we will compute the between-groups variance and the withingroups variance. If the null hypothesis is true, then these two distinct estimates of the variance should be equal, their equality can be tested by an F ratio test. The fact that we can compute variances in two ways stems from the break-down of the total sum of squares into a between-groups sum of squares and a within-groups sum of squares. The total number of degrees of freedom N-1, where N is the sum of all sample sizes, is also broken down into the appropriate between-groups and within-groups degrees of freedom: t-1 and N-t, respectively. The results of computations of ANOVA methods are usually tabulated. The rows of such tables give the source of the variance; the columns contain the sum of squares, the number of degrees of freedom, the variances, the F-value (variance ratio), and the p-value. HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach

3 ANOVA table Source of variation Between groups Within groups Sum of squares Q = n ( x x) k h h i = 1 n i i i Qb = ( xij xi ) s i= 1 j = 1 Total h ni N-1 Q = ( xij x) i = 1 j = 1 Degrees of freedom h-1 N-h s Variance b = w = Qb h 1 Q N w h F F = s s b w HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach3 3

4 Multiple comparisons If the result of the ANOVA is not significant at the specified level, the analysis is complete. We expect all samples be drawn from the same population; the differences between sample means are due to random effects If the result of the ANOVA is significant, then we have to accept the alternative hypothesis: there is at least one group different from one of the others. To find these groups, we have to compare each group with each of the others. As the two-sample t-test is inappropriate to do this, there are special tests for multiple comparisons that keep the probability of Type I error as α. HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach4 4

5 Pairwise methods I. The most often used multiple comparisons are the modified t- tests. Another often used method is the so-called Bonferroni method: to achieve a level of not more than α for a set of a number of c tests, we need to choose a level α /c for the individual tests. xi x j For example for three comparisons a p-value less than 0.05/3=0.017 has to be considered significant instead of p=0.05. This method is conservative. We know only that the probability does not exceed α for the set. This method can be used in cases involving small numbers of comparisons. t = s b n n i j HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach5 5

6 Scheffé test The Scheffé test performs simultaneous joint pairwise comparisons for all possible pairwise combinations of means. This test can be used to examine special linear combinations of group means, not simply pairwise comparisons. s b xi x j ( )( h 1) ni n j HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach6 6

7 Tukey-test (only in the case of equal sample sizes The followin quantity has to be compared with a value from the special table of studentized ranges T = x i s b n x j HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach7 7

8 Dunnett test Another multiple comparison method is the Dunnett test: a test comparing a given group (control) with the others. HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach8 8

9 Example Assume that we have recorded the biomass of 3 bacteria in flasks of glucose broth, and we used 3 replicate flasks for each bacterium. Replicate Bacterium A Bacterium B Bacterium C HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach9 9

10 Anova: Single Factor Descriptive statistics Groups Count Sum Average Variance Bacterium A Bacterium B 3 6 0, , Bacterium C HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 10 10

11 ANOVA table Source of Variation SS df MS F P-value F crit Between Groups 1140, 570, , ,61E-05 5,14353 Within Groups 5, , Total 119,889 8 HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 11 11

12 Two-way analysis of variance Main Effect The main effect involves the independent variables one at a time. The interaction is ignored for this part. Just the rows or just the columns are used, not mixed. This is the part which is similar to the one-way analysis of variance. Each of the variances calculated to analyze the main effects are like the between variances Interaction Effect The interaction effect is the effect that one factor has on the other factor. The degrees of freedom here is the product of the two degrees of freedom for each factor. Within Variation The Within variation is the sum of squares within each treatment group. You have one less than the sample size (remember all treatment groups must have the same sample size for a two-way ANOVA) for each treatment group. The total number of treatment groups is the product of the number of levels for each factor. The within variance is the within variation divided by its degrees of freedom. The within group is also called the error. F-Tests There is an F-test for each of the hypotheses, and the F-test is the mean square for each main effect and the interaction effect divided by the within variance. The numerator degrees of freedom come from each effect, and the denominator degrees of freedom is the degrees of freedom for the within variance in each case. HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 1 1

13 Hypothesis There are three sets of hypothesis with the two-way ANOVA. The null hypotheses for each of the sets are given below. The population means of the first factor are equal. This is like the oneway ANOVA for the row factor. The population means of the second factor are equal. This is like the one-way ANOVA for the column factor. There is no interaction between the two factors. This is similar to performing a test for independence with contingency tables. HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 13 13

14 ANOVA table without replication Factor B p T j = x ij i = 1 j=1 j=... j=q Z i i=1 x 11 x 1... x 1q Z 1 xi x1 T Z i = x ij x = i = q q j = 1 j = 1 Z q T j i F a c t o r A i= x 1 x... x q Z x x j = T j p i=p x k1 x k... x pq Z p T j T 1 T... T q T x p Intercept / correction factor = T pq x j x 1 x x q x HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 14 14

15 ANOVA table without replication Source of variation Correction factor Sum of squares Degrees of freedom Variance (Var=Q/f) F - value (F=Var/Varerror) Q CF f CF = 1 Var CF =Q CF /f CF F CF =Var CF /Var error Faktor A Q A f A =p 1 Var A =Q A /f A F A =Var A /Var error Faktor B Q B f B =q 1 Var B =Q B /f B F B =Var B /Var error Error Q error f error =(p 1)(q 1) Var error =Q error /f error Total Q total f total =pq-1 HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 15 15

16 ANOVA table without replication Source of variation Sum of squares (SS) Degrees of freedom (df) Variance (MS=SS/df) F T pq Intercept 1 s CF SS CF = F = CF sh s Factor A 1 T SS Z p-1 s A A = i q pq i F = s s A h Factor B 1 T SS T B j q-1 s B Error (Residual) SS Error = = p pq j SS Total SS A SS B (p-1)(q-1) s h F = s s B h = pq i, j Total T N-1 SS Total x ij HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 16 16

17 ANOVA table with replication Source of variation Correction factor Sum of squares Degrees of freedom Variance (Var=Q/f) F - value (F=Var/Varerror) Q CF f CF = 1 Var CF =Q CF /f CF F CF =Var CF /Var error Faktor A Q A f A =p 1 Var A =Q A /f A F A =Var A /Var error Faktor B Q B f B =q 1 Var B =Q B /f B F B =Var B /Var error A x B Q AB f AB =(p 1)(q 1) Var AB =Q AB /f AB F AB =Var AB /Var error Error Q error f error =N-pq(=0) Var error =Q error /f error Total Q total f total =pq-1 HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 17 17

18 ANOVA table with replication Source of variation Sum of squares (SS) Degrees of freedom (df) Variance (MS=SS/df) F Intercept T 1 s CF Factor A p-1 s A q Factor B q-1 s B Interaction (AxB) Error (Within) SS SS CF = SS SS SS AxB A = B within n = = = p nq np n pq q n p q Total ( ) N-1 SStotal = xhij x p i = 1 i = 1 i = 1 j = 1 p h = 1 i = 1 j = 1 ( xi. x) ( x. j x) ( xij xi. x. j + x) q h = 1 i = 1 j = 1 ( x xij ) hij (p-1)(q-1) s AB N-pq s h F = F = F = F = CF sh s s s A h s s B h AB sh s HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 18 18

19 y The two-way ANOVA model Let us denote the numbers of levels of factors 1 and by t and l, respectively, and by N the total number of observations. The two-way ANOVA model is ijk = µ + α + β + Θ + ε i j ij ijk, i = 1,..., t j = 1,..., l, k = 1,..., n ij HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 19 19

20 where we use the following notations: y ijk = the k-th observed value of the dependent variable when we are using level i of factor 1 and level j of factor, µ = an overall mean, (unknown constant α i = the effect due to level i of factor 1 (an unknown constant), β j = the effect due to level j of factor, (an unknown constant), θ ij = the effect due to the interaction of level i of factor 1 and level j of factor (an unknown constant), ε ijk = the k-th error term when we are using level i of factor 1 and level j of factor (assumed to be distributed as N(0,σ)) HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 0 0

21 According to the above questions, the following null hypotheses can be tested H 1 : α 1 = α =...= α t H : β 1 = β =...= β t H 3 : θ ij = 0 HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 1 1

22 Example: Suppose that factor A has 3 levels (i=3) and factor B has levels (j=). Them theoretically, we might have the following situations HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach

23 Only the effect of A is significant, the only significant factor is A y ijk = µ + α i + ε ijk, HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 3 3

24 Only the effect of B is significant, the only significant factor is B y = µ + β ijk j + ε ijk OSSZFVS 1 1 LPS LPS nem volt -,5 0,0,5 1,0 1,5,0,5 HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 4 4

25 Both factors are significant y ijk = µ + α + β + i j ε ijk LPS OSSZFVS 0 LPS nem volt -,5 0,0,5 1,0 1,5,0,5 HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 5 5

26 None of them is significant HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 6 6

27 Interaction The differences in A depend on the level of factor B OSSZFVS 0 LPS LPS nem volt -,5 0,0,5 1,0 1,5,0,5 HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 7 7

28 Two-way ANOVA In two-way ANOVA, the total sum of squares is decomposed into four terms, according to the effects in the model. The results are generally written into an ANOVA table which contains rows for the effects of factors 1 and, the interaction and the error with (p-1), (q-1), (p-1)(q-1) and (N-pq) degrees of freedom, respectively HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 8 8

29 The rows of this tables give the components for the effects of factor 1, factor, the interaction and the error term, while the columns contain the sum of squares, the number of degrees of freedom, the variances, the F-values (variance ratio), and the p-value of F. HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 9 9

30 Decision There are three F-values in this table according to the three hypotheses. The interaction, is tested first. If it is not significant, the significance of each of factors 1 and can be tested separately. If H1 is rejected, we can say that at least two means of the factor 1 differ. If t (which is more than two), the number of levels of factor 1, we again have to use multiple comparisons to find pairwise differences. HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 30 30

31 Decision If the interaction is significant, then the relationship between the means of factor 1 depends on the level of factor. Multiple comparisons can be performed for each combination of one factor with a given level of the other factor. There are special methods against the increase of Type I error, and the use of t-tests independently is an incorrect solution HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 31 31

32 Example Suppose that we have grown one bacterium in broth culture at 3 different ph levels at 4 different temperatures. We have 1 flasks in all, but no replicates. Growth was measured by optical density (O.D.). HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 3 3

33 Anova: Two-Factor Without Replication DATA Temp o C ph 5.5 ph 6.5 ph HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 33 33

34 Descriptive statistics SUMMARY Count Sum Average Variance , , , ,3333 ph ,66667 ph ph HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 34 34

35 Two-way Anova Source of Variation SS df MS F P-value F crit Rows 38, , ,4634 0,008 4,75706 Columns ,19,8E-06 5,143 Error 7, ,55556 Total HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 35 35

36 Of interest, another piece of information is revealed by this analysis - the effects of temperature do not interact with effects of ph. In other words, a change of temperature does not change the response to ph, and vice-versa.we can deduce this because the residual (error) mean square (MS) is small compared with the mean squares for temperature (columns) or ph (rows). [A low residual mean square tells us that most variation in the data is accounted for by the separate effects of temperature and ph]. HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 36 36

37 Example II Temp o C ph 5.5 ph 6.5 ph HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 37 37

38 ANOVA without replication Source of Variation SS df MS F P-value F crit Rows , , , , Columns 660, ,3333 0, , ,14353 Error Total 399, HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 38 38

39 ANOVA with replication The data of serum IGG level were measured in triplicate of three patients. Is there any difference in individual mean IGG level values using diferent method of measurement? HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 39 39

40 ANOVA with replication Dataset help species method value a1 A a A 1 11 a3 A 1 5. b1 B b B b3 B 1 5. c1 C c C c3 C a1 A 10.7 a A 8.6 a3 A 10.6 b1 B 1.9 b B 3. b3 B 1. c1 C 9.7 c C 7.8 c3 C 6.5 HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 40 40

41 The role of Column help Using Excel Pivot Table command to get the ANOVA table we need to use an extra column let s name it as help. Without this we get the following table (there is no replicant). Sum of value method species 1 Grand Total A B C Grand Total HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 41 41

42 Pivot table using help column Sum of value method help 1 Grand Total a a a b b b c c c Grand Total HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 4 4

43 The settings of ANOVA command G H I 4 Species Method_1 Method_ 5 A rep rep B rep rep C rep rep HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 43 43

44 Anova: Two-Factor With Replication I. SUMMARY Method_1 Method_ Total A Count Sum Average Variance B Count Sum Average Variance C Count Sum Average Variance HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 44 44

45 Anova: Two-Factor With Replication II. ANOVA Source of Variation SS df MS F P-value F crit Sample/Species Columns/Meth Interaction Within Total HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 45 45

46 SPSS results I Dependent Variable: value species A B C Total method Method1 Method Total Method1 Method Total Method1 Method Total Method1 Method Total Descriptive Statistics Mean Std. Deviation N 0,0000 7, ,9667 1, ,9833 7, ,5667,7091 3,1000 1, ,3333 3, ,8000, ,0000 1, ,4000 3, ,1 7, ,6889 3, ,9056 6, Levene's Test of Equality of Error Variances a Dependent Variable: value F df1 df Sig. 6,40 5 1,004 Tests the null hypothesis that the error variance of the dependent variable is equal across groups. a. Design: Intercept+species+method+species * method HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 46 46

47 SPSS results II. Dependent Variable: value Source Corrected Model Intercept species method species * method Error Total Corrected Total Tests of Between-Subjects Effects Type III Sum of Squares df Mean Square F Sig. 557,956 a 5 111,591 8,430, , , ,418,000 34, ,34 1,935, , ,45 14,069,003 9,43 14,6 1,105, , ,38 48, , a. R Squared =,778 (Adjusted R Squared =,686) HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 47 47

48 Post-Hoc test Dependent Variable: value LSD Multiple Comparisons (I) species A B C (J) species B C A C A B Based on observed means. *. The mean difference is significant at the,05 level. Mean Difference 95% Confidence Interval (I-J) Std. Error Sig. Lower Bound Upper Bound 10,6500*,1006,000 6, ,69 4,5833*,1006,050,0065 9,160-10,6500*,1006,000-15,69-6,0731-6,0667*,1006,014-10,6435-1,4898-4,5833*,1006,050-9,160 -,0065 6,0667*,1006,014 1, ,6435 HUSRB/0901/1/088 Teaching Mathematics and Statistics in Sciences: Modeling and Computer-aided Approach 48 48