Statistics For Economics & Business

Transcription

1 Statistics For Economics & Business Analysis of Variance In this chapter, you learn: Learning Objectives The basic concepts of experimental design How to use one-way analysis of variance to test for differences among the means of several populations (also referred to as groups in this chapter) To learn the basic structure and use of a randomized block design How to use two-way analysis of variance and interpret the interaction effect How to perform multiple comparisons in a one-way analysis of variance and a two-way analysis of variance

2 Chapter Overview Analysis of Variance (ANOVA) One-Way ANOVA F-test Tukey- Kramer Multiple Comparisons Levene Test For Homogeneity of Variance Randomized Block Design Tukey Multiple Comparisons Two-Way ANOVA Interaction Effects Tukey Multiple Comparisons General ANOVA Setting Investigator controls one or more factors of interest Each factor contains two or more levels Levels can be numerical or categorical Different levels produce different groups Think of each group as a sample from a different population Observe effects on the dependent variable Are the groups the same? Experimental design: the plan used to collect the data

3 Completely Randomized Design Experimental units (subjects) are assigned randomly to groups Subjects are assumed homogeneous Only one factor or independent variable With two or more levels Analyzed by one-factor analysis of variance (ANOVA) One-Way Analysis of Variance Evaluate the difference among the means of three or more groups Examples: Accident rates for st, nd, and 3 rd shift Expected mileage for five brands of tires Assumptions Populations are normally distributed Populations have equal variances Samples are randomly and independently drawn

4 Hypotheses of One-Way ANOVA H = 0 : µ = µ = µ 3 =! µ c All population means are equal i.e., no factor effect (no variation in means among groups) H :Not all of At least one population mean is different i.e., there is a factor effect the populationmeans are the same Does not mean that all population means are different (some pairs may be the same) One-Way ANOVA H = 0 : µ = µ = µ 3 =! µ c H :Not all µ j are thesame The Null Hypothesis is True All Means are the same: (No Factor Effect) µ = = µ µ 3

5 One-Way ANOVA H = 0 : µ = µ = µ 3 =! µ c H :Not all µ j are thesame The Null Hypothesis is NOT true At least one of the means is different (Factor Effect is present) (continued) or µ = µ µ 3 µ µ µ 3 Partitioning the Variation Total variation can be split into two parts: SST = SSA + SSW SST = Total Sum of Squares (Total variation) SSA = Sum of Squares Among Groups (Among-group variation) SSW = Sum of Squares Within Groups (Within-group variation)

6 Partitioning the Variation SST = SSA + SSW (continued) Total Variation = the aggregate variation of the individual data values across the various factor levels (SST) Among-Group Variation = variation among the factor sample means (SSA) Within-Group Variation = variation that exists among the data values within a particular factor level (SSW) Partition of Total Variation Total Variation (SST) Variation Due to Factor (SSA) = + Variation Due to Random Error (SSW)

7 Where: Total Sum of Squares SST = SSA + SSW SST = c n j j= i= ( X ij SST = Total sum of squares X c = number of groups or levels ) n j = number of observations in group j X ij = i th observation from group j X = grand mean (mean of all data values) Total Variation (continued) SST = ( X X X ) + ( X X ) + + ( X cn ) c Response, X X Group Group Group 3

8 Where: Among-Group Variation SST = SSA + SSW c SSA = n j= j ( X j X ) SSA = Sum of squares among groups c = number of groups n j = sample size from group j X j = sample mean from group j X = grand mean (mean of all data values) Among-Group Variation c SSA = nj( X j X ) j= Variation Due to Differences Among Groups µ i µ j SSA MSA = c (continued) Mean Square Among = SSA/degrees of freedom

9 Among-Group Variation (continued) SSA = n ( X X) + n( X X) + + nc ( X c X) Response, X X X X 3 X Group Group Group 3 Within-Group Variation SSW SST = SSA + SSW c j= i= ( X ij X j Where: SSW = Sum of squares within groups c = number of groups n j = sample size from group j X j = sample mean from group j X ij = i th observation in group j n j = )

10 Within-Group Variation (continued) SSW c j= n j = i= ( X ij X j Summing the variation within each group and then adding over all groups ) SSW MSW = n c Mean Square Within = SSW/degrees of freedom µ j Within-Group Variation (continued) SSW = c ( X X ) + ( X X ) + + ( X cn X c ) Response, X X X X 3 Group Group Group 3

11 Obtaining the Mean Squares The Mean Squares are obtained by dividing the various sum of squares by their associated degrees of freedom SSA MSA = c Mean Square Among (d.f. = c-) MSW = SSW n c Mean Square Within (d.f. = n-c) MST SST = n Mean Square Total (d.f. = n-) One-Way ANOVA Table Source of Variation Degrees of Freedom Sum Of Squares Mean Square (Variance) F Among Groups Within Groups c - SSA SSA MSA = c - n - c SSW SSW MSW = n - c F STAT = MSA MSW Total n SST c = number of groups n = sum of the sample sizes from all groups df = degrees of freedom

12 One-Way ANOVA F Test Statistic H 0 : µ = µ = = µ c H : At least two population means are different Test statistic F STAT = MSA MSW MSA is mean squares among groups MSW is mean squares within groups Degrees of freedom df = c (c = number of groups) df = n c (n = sum of sample sizes from all populations) Interpreting One-Way ANOVA F Statistic The F statistic is the ratio of the among estimate of variance and the within estimate of variance The ratio must always be positive df = c - will typically be small df = n - c will typically be large Decision Rule: Reject H 0 if F STAT > F α, otherwise do not reject H 0 0 Do not reject H 0 α Reject H 0 F α

13 One-Way ANOVA F Test Example You want to see if when three different golf clubs are used, they hit the ball different distances. You randomly select five measurements from trials on an automated driving machine for each club. At the 0.05 significance level, is there a difference in mean distance? Club Club Club One-Way ANOVA Example: Scatter Plot Club Club Club X = 49. X = 6.0 X3 X = 7.0 = 05.8 Distance X X X 3 X 3 Club

14 One-Way ANOVA Example Computations Club Club Club X = 49. X = 6.0 X 3 = 05.8 X = 7.0 n = 5 n = 5 n 3 = 5 n = 5 c = 3 SSA = 5 (49. 7) + 5 (6 7) + 5 (05.8 7) = 4,76.4 SSW = (54 49.) + (63 49.) + + ( ) =,9.6 MSA = 4,76.4 / (3-) =,358. MSW =,9.6 / (5-3) = 93.3,358. F STAT = = One-Way ANOVA Example Solution H 0 : µ = µ = µ 3 H : µ j not all equal α = 0.05 df = df = Test Statistic: MSA 358. F STAT = = = 5.75 MSW Do not reject H 0 Critical Value: F α = 3.89 α =.05 Reject H 0 F α = 3.89 F STAT = 5.75 Decision: Reject H 0 at α = 0.05 Conclusion: There is evidence that at least one µ j differs from the rest

15 The Tukey-Kramer Procedure Tells which population means are significantly different e.g.: µ = µ µ 3 Done after rejection of equal means in ANOVA Allows paired comparisons Compare absolute mean differences with critical range µ = µ µ 3 x Tukey-Kramer Critical Range Critical Range = Q α MSW n j + n j' where: Q α = Upper Tail Critical Value from Studentized Range Distribution with c and n - c degrees of freedom (see appendix E.7 table) MSW = Mean Square Within n j and n j = Sample sizes from groups j and j

16 The Tukey-Kramer Procedure: Example. Compute absolute mean Club Club Club 3 differences: x x = = x x3 = = x x = = 0.. Find the Q α value from the table in appendix E.7 with c = 3 and (n c) = (5 3) = degrees of freedom: 3 Q = 3.77 α The Tukey-Kramer Procedure: Example 3. Compute Critical Range: MSW Critical Range = Q + = α n j n j' 5 5 = 5. All of the absolute mean differences are greater than critical range. Therefore there is a significant difference between each pair of means at 5% level of significance. Thus, with 95% confidence we can conclude that the mean distance for club is greater than club and 3, and club is greater than club Compare: x x x x x 3 x 3 (continued) = 3. = 43.4 = 0.

17 ANOVA Assumptions Randomness and Independence Select random samples from the c groups (or randomly assign the levels) Normality The sample values for each group are from a normal population Homogeneity of Variance All populations sampled from have the same variance Can be tested with Levene s Test ANOVA Assumptions Levene s Test Tests the assumption that the variances of each population are equal. First, define the null and alternative hypotheses: H 0 : σ = σ = =σ c H : Not all σ j are equal Second, compute the absolute value of the difference between each value and the median of each group. Third, perform a one-way ANOVA on these absolute differences.

18 Levene Homogeneity Of Variance Test Example H0: σ = σ = σ 3 H: Not all σ j are equal Calculate Medians Calculate Absolute Differences Club Club Club Median Club Club Club Levene Homogeneity Of Variance Test Example (continued) Anova: Single Factor SUMMARY Groups Count Sum Average Variance Club Club Club Source of Variation SS df MS F P- value F crit Between Groups Within Groups Total 4 4 Since the p-value is greater than 0.05 there is insufficient evidence of a difference in the variances

19 The Randomized Block Design (optional) Like One-Way ANOVA, we test for equal population means (for different factor levels, for example)......but we want to control for possible variation from a second factor (with two or more levels) Levels of the secondary factor are called blocks Partitioning the Variation (optional) Total variation can now be split into three parts: SST = SSA + SSBL + SSE SST = Total variation SSA = Among-Group variation SSBL = Among-Block variation SSE = Random variation

20 Sum of Squares for Blocks SST = SSA + SSBL + SSE (optional) SSBL = c r i= ( X i. X ) Where: c = number of groups r = number of blocks X i. = mean of all values in block i X = grand mean (mean of all data values) Partitioning the Variation Total variation can now be split into three parts: SST = SSA + SSBL + SSE SST and SSA are computed as they were in One-Way ANOVA SSE = SST (SSA + SSBL)

21 Mean Squares (optional) MSBL = Meansquare blocking = SSBL r MSA = Meansquare among groups = SSA c MSE = Mean square error = SSE ( r )( c ) Randomized Block ANOVA Table (optional) Source of Variation SS df MS F Among Groups SSA c - MSA MSA MSE Among Blocks SSBL r - MSBL MSBL MSE Error SSE (r )(c-) MSE Total SST rc - c = number of populations rc = total number of observations r = number of blocks df = degrees of freedom

22 Testing For Factor Effect (optional) H = 0 :µ. = µ. = µ.3 = µ.c H : Not all populationmeansare equal F STAT = MSA MSE Main Factor test: df = c df = (r )(c ) Reject H 0 if F STAT > F α Test For Block Effect H : µ = µ = µ =... = µ H :Not all block means are equal r. (optional) F STAT = MSBL MSE Blocking test: df = r df = (r )(c ) Reject H 0 if F STAT > F α

23 Randomized Block Design Example Ratings at Four Restaurants of a Fast-Food Chain RESTAURANTS (optional) RATERS A B C D Totals Means Totals ,887 Raters are the blocks so r = 6. Restaurants are the groups of interest so c = 4. n = rc = 4 c r Xij j= i=,887 X = = = rc 4 Means Hypothesis Tests For This Example To decide whether there is a difference in average rating among the restaurants: (optional) H 0 : µ A = µ B = µ C = µ D vs H : At least one of the µ s is different To decide whether there is a difference in average rating among the raters and the blocking has reduced error: H 0 : µ = µ = µ 3 = µ 4 = µ 5 = µ 6 vs H : At least one of the µ s is different

24 ANOVA Output From Excel (optional) Do the restaurants differ in average rating? Since the p-value (0.0000) < 0.05 conclude there is a difference in avg. rating. Do the raters differ in average rating? Since the p-value (0.005) < 0.05 conclude there is a difference in the avg. rating of raters. This indicates the blocking has reduced error. Factorial Design: Two-Way ANOVA Examines the effect of Two factors of interest on the dependent variable e.g., Percent carbonation and line speed on soft drink bottling process Interaction between the different levels of these two factors e.g., Does the effect of one particular carbonation level depend on at which level the line speed is set?

25 Two-Way ANOVA (continued) Assumptions Populations are normally distributed Populations have equal variances Independent random samples are drawn Two-Way ANOVA Sources of Variation Two Factors of interest: A and B r = number of levels of factor A c = number of levels of factor B n = number of replications for each cell n = total number of observations in all cells n = (r)(c)(n ) X ijk = value of the k th observation of level i of factor A and level j of factor B

26 Two-Way ANOVA Sources of Variation SST = SSA + SSB + SSAB + SSE SSA Factor A Variation (continued) Degrees of Freedom: r SST Total Variation n - SSB Factor B Variation SSAB Variation due to interaction between A and B SSE Random variation (Error) c (r )(c ) rc(n ) Two-Way ANOVA Equations Total Variation: Factor A Variation: Factor B Variation: SST = r c nʹ i= j= k= i= ( X ijk X ) SSA = cnʹ ( X X r c j= i.. ) SSB = rnʹ ( X X. j. )

27 Two-Way ANOVA Equations (continued) Interaction Variation: SSAB r c n ʹ (X + = i= j= ij. Xi.. X.j. X) Sum of Squares Error: SSE = r c nʹ i= j= k= ( X ijk X ij.) where: c j= k= Two-Way ANOVA Equations nʹ X ijk X = r c nʹ i= j= k= rcnʹ X ijk = Grand Mean th Xi.. = = Mean of i level of factor A (i =,,...,r) cnʹ X r nʹ Xijk i= k= th X. j. = = Mean of j level of factor B (j =, rnʹ X n ijk ij. = ʹ = k= nʹ Mean of cell ij r = number of levels of factor A c = number of levels of factor B (continued),..., c) n = number of replications in each cell

28 Mean Square Calculations MSA = Mean square factor A = MSB = Mean square factor B = SSA r SSB c MSAB = Mean square interaction = SSAB ( r )( c ) MSE = Mean square error = SSE rc( n' ) Two-Way ANOVA: The F Test Statistics F Test for Factor A Effect H 0 : µ.. = µ.. = µ 3.. = = µ r.. H : Not all µ i.. are equal F STAT = MSA MSE Reject H 0 if F STAT > F α F Test for Factor B Effect H 0 : µ.. = µ.. = µ.3. = = µ.c. H : Not all µ.j. are equal F STAT = MSB MSE Reject H 0 if F STAT > F α H 0 : the interaction of A and B is equal to zero H : interaction of A and B is not zero F Test for Interaction Effect F STAT = MSAB MSE Reject H 0 if F STAT > F α

29 Two-Way ANOVA Summary Table Source of Variation Degrees Of Freedom Sum of Squares Factor A r SSA Factor B c - SSB Mean Squares MSA MSA = SSA /(r ) MSE MSB = SSB /(c ) MSB MSE F AB (Interaction) (r )(c-) SSAB MSAB MSAB = SSAB / (r )(c ) MSE Error rc(n ) SSE Total n - SST MSE = SSE/rc(n ) Features of Two-Way ANOVA F Test Degrees of freedom always add up n- = rc(n -) + (r-) + (c-) + (r-)(c-) Total = error + factor A + factor B + interaction The denominators of the F Test are always the same but the numerators are different The sums of squares always add up SST = SSA + SSB + SSAB + SSE Total = factor A + factor B + interaction + error

30 Examples: Interaction vs. No Interaction No interaction: line segments are parallel Interaction is present: some line segments not parallel Factor B Level Mean Response Factor B Level 3 Factor B Level Mean Response Factor B Level Factor B Level Factor B Level 3 Factor A Levels Factor A Levels Multiple Comparisons: The Tukey Procedure Unless there is a significant interaction, you can determine the levels that are significantly different using the Tukey procedure Consider all absolute mean differences and compare to the calculated critical range Example: Absolute differences for factor A, assuming three levels: X X.... X X X.. X 3..

31 Multiple Comparisons: The Tukey Procedure Critical Range for Factor A: Critical Range = Q α MSE cn' (where Q α is from Table E.7 with r and rc(n ) d.f.) Critical Range for Factor B: Critical Range = Q α MSE r n' (where Q α is from Table E.7 with c and rc(n ) d.f.) Do ACT Prep Course Type & Length Impact Average ACT Scores ACT Scores for Different Types and Lengths of Courses LENGTH OF COURSE TYPE OF COURSE Condensed Regular Traditional Traditional Traditional Traditional Traditional Online 7 4 Online Online Online Online

32 Plotting Cell Means Shows A Strong Interaction Nonparallel lines indicate the effect of condensing the course depends on whether the course is taught in the traditional classroom or by online distance learning The online course yields higher scores when condensed while the traditional course yields higher scores when not condensed (regular). Excel Analysis Of ACT Prep Course Data The interaction between course length & type is significant because its p-value is While the p-values associated with both course length & course type are not significant, because the interaction is significant you cannot directly conclude they have no effect.

33 With The Significant Interaction Collapse The Data Into Four Groups After collapsing into four groups do a one way ANOVA The four groups are. Traditional course condensed. Traditional course regular length 3. Online course condensed 4. Online course regular length Excel Analysis Of Collapsed Data Group is a significant effect. p-value of < Traditional regular > Traditional condensed. Online condensed > Traditional condensed 3. Traditional regular > Online regular 4. Online condensed > Online regular If the course is take online should use the condensed version and if the course is taken by traditional method should use the regular.

34 Summary Described one-way analysis of variance The logic of ANOVA ANOVA assumptions F test for difference in c means The Tukey-Kramer procedure for multiple comparisons The Levene test for homogeneity of variance Examined the basic structure and use of a randomized block design Described two-way analysis of variance Examined effects of multiple factors Examined interaction between factors