χ 2 Test for Frequencies January 15, 2019 Contents

Transcription

1 χ 2 Test for Frequencies January 15, 2019 Contents Chi squared (χ 2 ) test for frequencies Example 1: left vs right handers in our class The χ 2 distribution One or two tailed? Example 2: Birthdays by month Using R to run a hypothesis test for frequencies Questions Answers Happy birthday to Sara Mikaela Heller! Chi squared (χ 2 ) test for frequencies This is a hypothesis test on the frequency of samples that fall into different discrete categories For example, are the number of left and right-handed people in our class distributed like you d expect from the population? Or, is the freqency distribution of birthdays by month for the students in our class distributed evenly across months? For these tests the dependent measure is a frequency, not a mean Here s how to get to the χ 2 test for frequencies with the flow chart: 1

2 START HERE Test for = 0 Ch number of correlations correlation (r) measurement scale frequency number of variables 1 2 test frequency Ch Test for 1 = 2 Ch 174 Means 2 test independence Ch 199 z-test Ch 131 Yes Do you know? 1 number of means More than 2 number of 2 factors 2-factor ANOVA Ch 21 No 1 one sample t-test Ch factor ANOVA Ch 20 independent measures t-test Ch 156 Yes independent samples? No dependent measures t-test Ch 164 Let s start with a simple example: Example 1: left vs right handers in our class According to Wikipedia, 10 percent of the population is left handed For our class, 7 students reported that they are left handed, while 89 reported right handedness A χ 2 test determines if the frequency of our sampled observations are significantly different than the frequencies that you d expect from the population Specifically, the null hypothesis is that our observed frequencies are drawn from a population that has some expected proportions, and our alternative hypothesis is that we re drawing from a population that does not have these expected proportions Like all statistical tests, the χ 2 test involves calculating a statistic that measures how far our observations are from those expected under the null hypothesis The first step is to calculate the frequencies expected from the null hypothesis This is simply done by multiplying the total sample size by each of the expected proportions Since there are 96 students in the class, then we expect (96)(01) = 96 students to be left handed and (96)(09) = 864 to be right handed Expected frequencies do not have to be rounded to the nearest whole number, even though frequencies are whole numbers This is because we should think of these expected frequencies as the average frequency for each category over the long run - and averages don t have to be whole numbers 2

3 The next step is to measure how far our observed frequencies are from the expected frequencies Here s the formula, where χ 2 is pronounced Chi-squared χ 2 = (fo fe) 2 fe Where fo are the observed frequencies and fe are the expected frequencies For our example, fo is 7 and 89 and fe is 96 and 864: χ 2 = (7 96) (89 864)2 864 = = 078 This measure, χ 2, is close to zero when the observed frequencies match the expected frequencies Therefore, large values of χ 2 can be considered evidence against the null hypothesis The χ 2 distribution Just like the z and t distributions, the χ 2 distribution has a known shape and therefore has its own table in the book and page in the Excel spreadsheet (Table I) Also, like the t-distribution, the χ 2 distribution is actually a family of distributions, with a different distribution for different degrees of freedom The χ 2 distribution for k degrees of freedom is the distribution you d get if you draw k values from the standard normal distribution (the z-distribution), square them, and add them up Here s what the probability distributions look like for different degrees of freedom: 3

4 df Notice how the shape of the distributions spread out and change shape with increasing degrees of freedom This is because as we increase df, and therefore the number of squared z-scores, the sum will on average increase too Since the χ 2 distribution is known we can calculate the probability of obtaining our observed value of χ 2 if null hypothesis is true The degrees of freedom is the number of categories minus one For this example, df = 2-1 = 1 The critical value is found with Table I in our book and also in the Excel spreadsheet All we do is look up the critical value for χ 2 for our df and value of α Let s use α = 005: df

5 The table tells us that with df = 1 and α = 005, the critical value for χ 2 is 384 If you re interested, the χ 2 distribution has a funny shape, especially for small degrees of freedom Here s what the distribution looks like for df = 1 Shown also is the critical value of χ 2 for which 5% of the curve lies above Also shown is the observed value of χ 2 (078) 078 area = (1) You can see that our observed χ 2 value (078) falls below the critical value (384) We therefore fail to reject the null hypothesis that our observations were drawn from the null hypothesis distribution Just like for the t-table, the χ 2 table is not useful for calculating p-values Instead we can use the χ 2 -calculator on the same page in the Excel spreadsheet, which gives us a p-value of 03771: α to χ 2 df α χ χ 2 to α df χ 2 α Using APA format, we d state: The number of left handers in our class is not significantly different from 10 percent χ 2 (1, N = 96) = 078, p =

6 One or two tailed? A common question for χ 2 tests is whether it is a one or a two tailed test Actually it s kind of both It s a two-tailed test since we ll reject the null hypothesis if our observed values differ from the null hypothesis in either direction (too many or too few lefties) On the other hand, it s like a one-tailed test because we reject the null hypothesis only for large values of χ 2 Importantly, they re all done the same way, so it doesn t really matter what we call it 6

7 Example 2: Birthdays by month Let s see if the birthdays in this class are evenly distributed across months, or if there are some months for which students have significantly more birthdays than others For simplictity, we ll assume that all months have equal probability, even though they vary in length We ll ruun a χ 2 test using an alpha value of 005 Here s a table showing the number of birthdays for each month for all 96 students: observed frequencies of birthdays Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec It looks kind of uneven A natural way to visualize this distribution is with a bar graph: Frequency Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Birthday Month To see if this distribution is significantly uneven we calculate the expected frequencies under the null hypothesis Here we expect equal frequencies of 96/12 = 8 birthdays per month Note equal frequencies assumes that each month has an equal number of days This assump- 7

8 tion is close enough for this example, but how would you correct the expected frequencies to account for this? Using our χ 2 formula: χ 2 = (fo fe) 2 fe We find that χ 2 values are: χ 2 for each cell Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec So χ2 = (11 8)2 8 + (12 8) (10 8)2 8 = = 1125 Using the table, the critical value of χ 2 for df = 12-1 = 11 and α = 005 is 1968 df Here is what the χ 2 distribution looks like for 11 degrees of freedom Note how different it is from the first example with df = 1 8

9 1125 area = (11) You can see that our observed χ 2 value (1125) falls below the critical value (1968) We therefore fail to reject the null hypothesis that our observations were drawn from the null hypothesis distribution Using the χ 2 -calculator gives us a p-value of 04226: α to χ 2 df α χ χ 2 to α df χ 2 α Using APA format, we d state: The distribution of birthdays across months in our class is not significantly different from an even distribution, χ 2 (11, N = 96) = 1125, p = Using R to run a hypothesis test for frequencies The following R script covers how to run a Chi 2 test for frequencies for the examples in this tutorial The R commands shown below can be found here: Chi2TestFrequenciesR # Chi-squared test for frequencies # # R s chisqtest provides a p-value for the chi-squared test for frequencies # by taking in a table of frequencies and an optional list of expected frequencies 9

10 # Here we ll run the two examples in the chi_2_test_frequencies_tutorial # Load in the survey data survey <-readcsv(" # Example 1: left vs right handers in our class, compared to 10% left handers fo <- table(survey$hand) # observed frequencies fe = c(1,9) # expected frequencies # run the chi-squared test: out <- chisqtest(fo,p=fe) # The chi-squared statistic is: out$statistic X-squared # The degrees of freedom is: out$parameter df 1 # And the p-value is: out$pvalue [1] # Writing in APA format can be done like this: sprintf( Chi-Squared(%d,N=%d) = %52f, p = %54f,out$parameter,sum(fo),out$statistic,out$pvalue [1] "Chi-Squared(1,N=96) = 078, p = 03764" # Plot the results: barplot(fo) # Example 2: Birthdays by month fo <- table(survey$month) # Run the chi-squared test If we don t specify the expected frequency the # test assumes that expected frequencies are equal across categories out <- chisqtest(fo) # result in APA format: sprintf( Chi-Squared(%d,N=%d) = %52f, p = %54f,out$parameter,sum(fo),out$statistic,out$pvalue [1] "Chi-Squared(11,N=96) = 1125, p = 04226" # Plotting: first rearrange months in order fo <- fo[c(5,4,8,1,9,7,6,2,12,11,10,3)] # Then plot: barplot(fo) # If you only have the chi-squared statistic and degrees of freedom # you can use the pchisq function to get the p-value We use # lowertail = FALSE to reject H0 for large values of chi-squared: chisquared <- out$statistic df <- out$parameter pvalue <- pchisq(chisquared,df,lowertail = FALSE) 10

11 pvalue X-squared

12 Questions Your turn Here are 10 random practice questions followed by their answers For all questions test the null hypothesis that there is an equal distribution of frequencies across categories 1) Suppose teams come in 2 varieties: penitent and adhoc Suppose you find 57 teams and count how many fall into each variety This generates the following table: observed frequencies of teams penitent adhoc Make a table of the expected frequencies Using an alpha value of α = 005 test the null hypothesis that the 57 teams are distributed evenly across the 2 varieties of penitent and adhoc 2) Suppose facial expressions come in 5 varieties: highfalutin, big, seemly, null and embarrassed I find 65 facial expressions and count how many fall into each variety This generates the following table: observed frequencies of facial expressions highfalutin big seemly null embarrassed Make a bar graph showing the frequencies for each variety of facial expressions Make a table of the expected frequencies Using an alpha value of α = 001 test the null hypothesis that the 65 facial expressions are distributed evenly across the 5 varieties of highfalutin, big, seemly, null and embarrassed 3) Suppose grandmothers come in 5 varieties: happy, able, cool, bawdy and furtive In your spare time you find 183 grandmothers and count how many fall into each variety This generates the following table: observed frequencies of grandmothers happy able cool bawdy furtive Make a table of the expected frequencies Using an alpha value of α = 005 test the null hypothesis that the 183 grandmothers are distributed evenly across the 5 varieties of happy, able, cool, bawdy and furtive 4) Suppose brains come in 3 varieties: dapper, powerful and momentous We decide to find 56 brains and count how many fall into each variety This generates the following table: 12

13 observed frequencies of brains dapper powerful momentous Make a table of the expected frequencies Using an alpha value of α = 001 test the null hypothesis that the 56 brains are distributed evenly across the 3 varieties of dapper, powerful and momentous 5) Suppose examples come in 2 varieties: noisy and overconfident For some reason you find 24 examples and count how many fall into each variety This generates the following table: observed frequencies of examples noisy 9 15 overconfident Make a bar graph showing the frequencies for each variety of examples Make a table of the expected frequencies Using an alpha value of α = 005 test the null hypothesis that the 24 examples are distributed evenly across the 2 varieties of noisy and overconfident 6) Suppose women come in 5 varieties: dirty, hard-to-find, modern, thoughtful and large We decide to find 105 women and count how many fall into each variety This generates the following table: observed frequencies of women dirty hard-to-find modern thoughtful large Make a bar graph showing the frequencies for each variety of women Make a table of the expected frequencies Using an alpha value of α = 001 test the null hypothesis that the 105 women are distributed evenly across the 5 varieties of dirty, hard-to-find, modern, thoughtful and large 7) Suppose brains come in 2 varieties: shallow and messy You find 26 brains and count how many fall into each variety This generates the following table: observed frequencies of brains shallow 8 18 messy Make a table of the expected frequencies Using an alpha value of α = 001 test the null hypothesis that the 26 brains are distributed evenly across the 2 varieties of shallow and messy 8) Suppose brains come in 3 varieties: second, fast and broad Your friend gets you 13

14 to find 47 brains and count how many fall into each variety This generates the following table: observed frequencies of brains second fast broad Make a bar graph showing the frequencies for each variety of brains Make a table of the expected frequencies Using an alpha value of α = 005 test the null hypothesis that the 47 brains are distributed evenly across the 3 varieties of second, fast and broad 9) Suppose elections come in 3 varieties: standing, superb and tasteless Your advsor asks you to find 51 elections and count how many fall into each variety This generates the following table: observed frequencies of elections standing superb tasteless Make a bar graph showing the frequencies for each variety of elections Make a table of the expected frequencies Using an alpha value of α = 001 test the null hypothesis that the 51 elections are distributed evenly across the 3 varieties of standing, superb and tasteless 10) Suppose mountains come in 5 varieties: best, secret, rough, separate and available Your boss makes you find 107 mountains and count how many fall into each variety This generates the following table: observed frequencies of mountains best secret rough separate available Make a table of the expected frequencies Using an alpha value of α = 001 test the null hypothesis that the 107 mountains are distributed evenly across the 5 varieties of best, secret, rough, separate and available 14

15 Answers 1) The frequencies of 2 kinds of teams fe = χ2 = (32 285)2 285 = 57 2 = (25 285)2 285 = = penitent χ 2 for each cell adhoc df = (2-1) = 1 χ 2 crit = 384 We fail to reject H 0 The frequency of 57 teams is distributed as expected across the 2 varieties of penitent and adhoc, χ 2 (1, N=57)= 086, p = # Using R: fo <- c( 32, 25) out <- chisqtest(fo) sprintf( Chi-Squared(%d,N=%d) = %52f, p = %54f,out$parameter,sum(fo),out$statistic,out$pvalue [1] "Chi-Squared(1,N=57) = 086, p = 03538" 15

16 2) The frequencies of 5 kinds of facial expressions fe = χ2 = (12 13) (14 13)2 13 = 65 5 = 13 + (7 13) (19 13) (13 13)2 13 = = χ 2 for each cell highfalutin big seemly null embarrassed df = (5-1) = 4 χ 2 crit = 1328 We fail to reject H 0 The frequency of 65 facial expressions is distributed as expected across the 5 varieties of highfalutin, big, seemly, null and embarrassed, χ 2 (4, N=65)= 569, p = Frequency of facial expressions highfalutin big seemly null embarrassed # Using R: fo <- c( 12, 14, 7, 19, 13) out <- chisqtest(fo) sprintf( Chi-Squared(%d,N=%d) = %52f, p = %54f,out$parameter,sum(fo),out$statistic,out$pvalue [1] "Chi-Squared(4,N=65) = 569, p = 02233" # Plotting: barplot(fo, 16

17 namesarg = c( "highfalutin", "big", "seemly", "null", "embarrassed"), ylab = Frequencies of facial expressions ) 17

18 3) The frequencies of 5 kinds of grandmothers fe = χ2 = (42 366) (37 366)2 366 = = (35 366) (22 366) (47 366)2 366 = = χ 2 for each cell happy able cool bawdy furtive df = (5-1) = 4 χ 2 crit = 949 We reject H 0 The frequency of 183 grandmothers is not distributed as expected across the 5 varieties of happy, able, cool, bawdy and furtive, χ 2 (4, N=183)= 965, p = # Using R: fo <- c( 42, 37, 35, 22, 47) out <- chisqtest(fo) sprintf( Chi-Squared(%d,N=%d) = %52f, p = %54f,out$parameter,sum(fo),out$statistic,out$pvalue [1] "Chi-Squared(4,N=183) = 965, p = 00467" 18

19 4) The frequencies of 3 kinds of brains fe = χ2 = ( ) = 56 3 = ( ) ( ) = = χ 2 for each cell dapper powerful momentous df = (3-1) = 2 χ 2 crit = 921 We fail to reject H 0 The frequency of 56 brains is distributed as expected across the 3 varieties of dapper, powerful and momentous, χ 2 (2, N=56)= 614, p = # Using R: fo <- c( 24, 22, 10) out <- chisqtest(fo) sprintf( Chi-Squared(%d,N=%d) = %52f, p = %54f,out$parameter,sum(fo),out$statistic,out$pvalue [1] "Chi-Squared(2,N=56) = 614, p = 00464" 19

20 5) The frequencies of 2 kinds of examples fe = χ2 = (9 12)2 12 = 24 2 = 12 + (15 12)2 12 = = 15 noisy χ 2 for each cell df = (2-1) = 1 χ 2 crit = 384 We fail to reject H 0 overconfident The frequency of 24 examples is distributed as expected across the 2 varieties of noisy and overconfident, χ 2 (1, N=24)= 150, p = Frequency of examples noisy overconfident # Using R: fo <- c( 9, 15) out <- chisqtest(fo) sprintf( Chi-Squared(%d,N=%d) = %52f, p = %54f,out$parameter,sum(fo),out$statistic,out$pvalue [1] "Chi-Squared(1,N=24) = 150, p = 02207" # Plotting: barplot(fo, 20

21 namesarg = c( "noisy", "overconfident"), ylab = Frequencies of examples ) 21

22 6) The frequencies of 5 kinds of women fe = χ2 = (19 21) (33 21)2 21 = = 21 + (22 21) (21 21) (10 21)2 21 = = dirty hard-tofind χ 2 for each cell modern thoughtful large df = (5-1) = 4 χ 2 crit = 1328 We fail to reject H 0 The frequency of 105 women is distributed as expected across the 5 varieties of dirty, hard-to-find, modern, thoughtful and large, χ 2 (4, N=105)=1286, p = Frequency of women dirty hard-to-find modern thoughtful large # Using R: fo <- c( 19, 33, 22, 21, 10) out <- chisqtest(fo) sprintf( Chi-Squared(%d,N=%d) = %52f, p = %54f,out$parameter,sum(fo),out$statistic,out$pvalue [1] "Chi-Squared(4,N=105) = 1286, p = 00120" # Plotting: 22

23 barplot(fo, namesarg = c( "dirty", "hard-to-find", "modern", "thoughtful", "large"), ylab = Frequencies of women ) 23

24 7) The frequencies of 2 kinds of brains fe = χ2 = (8 13)2 13 = 26 2 = 13 + (18 13)2 13 = = shallow χ 2 for each cell messy df = (2-1) = 1 χ 2 crit = 663 We fail to reject H 0 The frequency of 26 brains is distributed as expected across the 2 varieties of shallow and messy, χ 2 (1, N=26)= 385, p = # Using R: fo <- c( 8, 18) out <- chisqtest(fo) sprintf( Chi-Squared(%d,N=%d) = %52f, p = %54f,out$parameter,sum(fo),out$statistic,out$pvalue [1] "Chi-Squared(1,N=26) = 385, p = 00499" 24

25 8) The frequencies of 3 kinds of brains fe = χ2 = ( ) = 47 3 = ( ) ( ) = = χ 2 for each cell second fast broad df = (3-1) = 2 χ 2 crit = 599 We reject H 0 The frequency of 47 brains is not distributed as expected across the 3 varieties of second, fast and broad, χ 2 (2, N=47)=1702, p = Frequency of brains second fast broad # Using R: fo <- c( 9, 29, 9) out <- chisqtest(fo) sprintf( Chi-Squared(%d,N=%d) = %52f, p = %54f,out$parameter,sum(fo),out$statistic,out$pvalue [1] "Chi-Squared(2,N=47) = 1702, p = 00002" # Plotting: barplot(fo, 25

26 namesarg = c( "second", "fast", "broad"), ylab = Frequencies of brains ) 26

27 9) The frequencies of 3 kinds of elections fe = χ2 = (15 17)2 17 = 51 3 = 17 + (12 17) (24 17)2 17 = = χ 2 for each cell standing superb tasteless df = (3-1) = 2 χ 2 crit = 921 We fail to reject H 0 The frequency of 51 elections is distributed as expected across the 3 varieties of standing, superb and tasteless, χ 2 (2, N=51)= 459, p = Frequency of elections standing superb tasteless # Using R: fo <- c( 15, 12, 24) out <- chisqtest(fo) sprintf( Chi-Squared(%d,N=%d) = %52f, p = %54f,out$parameter,sum(fo),out$statistic,out$pvalue [1] "Chi-Squared(2,N=51) = 459, p = 01009" # Plotting: barplot(fo, 27

28 namesarg = c( "standing", "superb", "tasteless"), ylab = Frequencies of elections ) 28

29 10) The frequencies of 5 kinds of mountains fe = χ2 = (19 214) (27 214)2 214 = = (19 214) (29 214) (13 214)2 214 = = χ 2 for each cell best secret rough separate available df = (5-1) = 4 χ 2 crit = 1328 We fail to reject H 0 The frequency of 107 mountains is distributed as expected across the 5 varieties of best, secret, rough, separate and available, χ 2 (4, N=107)= 800, p = # Using R: fo <- c( 19, 27, 19, 29, 13) out <- chisqtest(fo) sprintf( Chi-Squared(%d,N=%d) = %52f, p = %54f,out$parameter,sum(fo),out$statistic,out$pvalue [1] "Chi-Squared(4,N=107) = 800, p = 00916" 29