Announcements. Statistics 104 (Mine C etinkaya-rundel) Diamonds. From last time.. = 22

Transcription

1 Announcements Announcements Unit 4: Inference for numerical variables Lecture 4: PA opens at 5pm today, due Sat evening (based on feedback on midterm evals) Statistics 104 If I still have your midterm or project proposal, pick it up at the end of class Mine C etinkaya-rundel New unit next week... October 24, 2013 Statistics 104 (Mine C etinkaya-rundel) Diamonds x s n U4 - L4: October 24, /1 Hypothesis testing for the difference of two means From last time carat 1 carat pt pt We are interested in finding out if the average point price of a 1 carat diamond is higher than the average point price of a 0.99 carat diamond. H0 :µpt99 = µpt HA :µpt99 < µpt = SE 30 T = df = carat = 0.99 carat = 1 Statistics U4 set - L4: These104 data(mine are ac etinkaya-rundel) random sample from the diamonds data in ggplot2 R package. October 24, /1 Statistics 104 (Mine C etinkaya-rundel) U4 - L4: October 24, /1

2 Hypothesis testing for the difference of two means Confidence intervals for the difference of two means p-value Clicker question Which of the following is the correct p-value for this hypothesis test? (a) between and 0.01 (b) between 0.01 and (c) between 0.01 and (d) between 0.02 and 0.05 (e) between 0.01 and 0.02 T = one tail two tails df Application exercise: t interval for comparing means The equivalent confidence level for a one sided HT with α = 0.05 is 90%. Calculate a 90% confidence interval for the average difference between the point prices of 0.99 and 1 carat diamonds, and choose the closest answer below. Then, interpret this interval in context of the data. (a) (-15.05, -2.81) (b) (-15.05, -2.81) (c) (-15.91, -1.95) (d) (-16.30, -1.56) (e) (-15.05, 2.81) (f) (-16.30, 1.56) Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Confidence intervals for the difference of two means Confidence intervals for the difference of two means Solution Synthesis one tail two tails df ( x pt99 x pt1 ) ± t df SE = ( ) ± = 8.93 ± 6.12 = ( 15.05, 2.81) We are 90% confident that the average point price of a 0.99 carat diamond is $15.05 to $2.81 lower than the average point price of a 1 carat diamond. How (if at all) would this conclusion change your behaviour if you went diamond shopping? Maybe buy a 0.99 carat diamond? It looks like a 1 carat, but is significantly cheaper. rstudio-pubs-static.s3.amazonaws.com/ fc524dc0bc2a140573da38bb.html Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1

3 Classy vocabulary The GSS gives the following 10 question vocabulary test: A SPACE (school, noon, captain, room, board, don t know) B BROADEN (efface, make level, elapse, embroider, widen, don t know) C EMANATE (populate, free, prominent, rival, come, don t know) D EDIBLE (auspicious, eligible, fit to eat, sagacious, able to speak, don t know) E ANIMOSITY (hatred, animation, disobedience, diversity, friendship, don t know) F PACT (puissance, remonstrance, agreement, skillet, pressure, don t know) G CLOISTERED (miniature, bunched, arched, malady, secluded, don t know) H CAPRICE (value, a star, grimace, whim, inducement, don t know) I ACCUSTOM (disappoint, customary, encounter, get used to, business, don t know) J ALLUSION (reference, dream, eulogy, illusion, aria, don t know) vocabulary scores Classy vocabulary The GSS also asks the following question: If you were asked to use one of four names for your social class, which would you say you belong in: the lower class, the working class, the middle class, or the upper class? (self reported) class LOWER CLASS WORKING CLASS MIDDLE CLASS UPPER CLASS Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Classy vocabulary Classy vocabulary Data Exploratory analysis wordsum class 1 6 MIDDLE CLASS 2 9 WORKING CLASS 3 6 WORKING CLASS 4 5 WORKING CLASS 5 6 WORKING CLASS 6 6 WORKING CLASS 7 8 MIDDLE CLASS 8 10 WORKING CLASS 9 8 WORKING CLASS 10 9 UPPER CLASS MIDDLE CLASS LOWER CLASS WORKING CLASS MIDDLE CLASS UPPER CLASS n mean sd lower class working class middle class upper class overall Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1

4 and the F test and the F test Clicker question Which of the following plots shows groups with means that are most and least likely to be significantly different from each other? Research question (a) most: I, least: II (b) most: I, least: II (c) most: II, least: III (d) most: I, least: III (e) most: III, least: II (f) most: II, least: I I II III Is there a difference between the average vocabulary scores of Americans from different (self reported) classes? To compare means of 2 groups we use a Z or a T statistic. To compare means of 3+ groups we use a new test called and a new statistic called F. Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 and the F test and the F test - hypotheses z/t test vs. - Purpose H 0 : The mean outcome is the same across all categories, µ 1 = µ 2 = = µ k, where µ i represents the mean of the outcome for observations in category i. H A : At least one pair of means are different from each other. z/t test Compare means from two groups to see whether they are so far apart that the observed difference cannot reasonably be attributed to sampling variability. H 0 : µ 1 = µ 2 Compare the means from two or more groups to see whether they are so far apart that the observed differences cannot all reasonably be attributed to sampling variability. H 0 : µ 1 = µ 2 = = µ k Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1

5 and the F test and the F test z/t test vs. - Method F distribution and p-value z/t test Compute a test statistic (a ratio). z/t = ( x 1 x 2 ) (µ 1 µ 2 ) SE( x 1 x 2 ) Compute a test statistic (a ratio). F = variability bet. groups variability w/in groups F = variability bet. groups variability w/in groups Large test statistics lead to small p-values. If the p-value is small enough H 0 is rejected, and we conclude that the population means are not equal. In order to be able to reject H 0, we need a small p-value, which requires a large F statistic. In order to obtain a large F statistic, variability between sample means needs to be greater than variability within sample means. Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 and the F test output, deconstructed Df Sum Sq Mean Sq F value Pr(>F) (Group) class < (Error) Residuals Total Goal: Determine measures of variability between and within groups, so that we can make a decision on the hypotheses based on how they compare to each other. Sum of squares total, SST Measures the total variability in the data n SST = (x i x) 2 i=1 where x i represent the value of the response variable of each observation in the dataset. [Very similar to calculation of variance, except not scaled by the sample size.] SST = (6 6.14) 2 + (9 6.14) (9 6.14) 2 = Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1

6 output, deconstructed Df Sum Sq Mean Sq F value Pr(>F) (Group) class < (Error) Residuals Total Sum of squares between groups, SSG Measures the variability between groups, i.e. how the group means compare to the grand mean k SSG = n i ( x j x) 2 j=1 n j : each group size, x j : average for each group, x: overall (grand) mean [Explained variability: deviation of group mean from overall mean, weighted by sample size.] n mean sd lower class working class middle class upper class overall SSG = ( 41 ( ) 2) + ( 407 ( ) 2) + ( 331 ( ) 2) + ( 16 ( ) 2) = Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 output, deconstructed Df Sum Sq Mean Sq F value Pr(>F) (Group) class < (Error) Residuals Total Sum of squares error, SSE Measures the variability within groups: SSE = SST SSG [Unexplained variability, i.e. unexplained by the group variable, due to other reasons] SSE = = Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 output, deconstructed Application exercise: output output, deconstructed Fill in the rest of the table, and make a decision on the hypotheses. Submit your decision using your clicker. The data provide convincing evidence that the: now we need a way to get from these measures of total variability to average variability (scaling by a measure that incorporates sample sizes and number of groups degrees of freedom) (a) average vocabulary scores are different for all classes. (b) average vocabulary score for middle class is higher than the average for the lower class. (c) average vocabulary score is different for at least one pair of classes. (d) average vocabulary score is different for at least one pair of classes. (e) average vocabulary scores are the same for all classes. (f) average vocabulary scores are different for upper and lower classes. Note that you will need access to R to calculate the p-value. You can use the following function: > pf(f-score, df_group, df_error, lower.tail = FALSE) Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1

7 output, deconstructed Relevant formulas Degrees of freedom associated with groups: df G = k 1, where k is the number of groups total: df T = n 1, where n is the total sample size error: df E = df T df G Mean squares Associated sum of squares divided by the associated df: MS = SS/df Test statistic, F value Ratio of the between group and within group variability: F = MSG MSE p-value Probability of at least as large a ratio between the between group and within group variability as the one observed, if in fact the means of all groups are equal calculated as the area under the F curve, with degrees of freedom df G and df E, above the observed F statistic. Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 output, deconstructed Solution Df Sum Sq Mean Sq F value Pr(>F) (Group) class < (Error) Residuals Total Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Checking conditions (1) independence If the data are a simple random sample from less than 10% of the population, this condition is satisfied. Carefully consider whether the data may be independent (e.g. no pairing). Always important, but sometimes difficult to check. Does this condition appear to be satisfied? Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Checking conditions (2) approximately normal The observations within each group should be nearly normal (especially important when the sample sizes are small.) Does this condition appear to be satisfied? Normal Q Q Plot Theoretical Quantiles Sample Quantiles Normal Q Q Plot Theoretical Quantiles Sample Quantiles Normal Q Q Plot Theoretical Quantiles Sample Quantiles Normal Q Q Plot Theoretical Quantiles Sample Quantiles Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1

8 Checking conditions (3) constant variance Which means differ? The variability across the groups should be about equal (especially important when the sample sizes differ between groups.) Does this condition appear to be satisfied? LOWER CLASS WORKING CLASS MIDDLE CLASS UPPER CLASS n mean sd lower class working class middle class upper class Earlier we concluded that at least one pair of means differ. The natural question that follows is which ones? We can do two sample t tests for differences in each possible pair of groups. Can you see any pitfalls with this approach? When we run too many tests, the Type 1 Error rate increases. This issue is resolved by using a modified significance level. Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Multiple comparisons Determining the modified α The scenario of testing many pairs of groups is called multiple comparisons. The Bonferroni correction suggests that a more stringent significance level is more appropriate for these tests: α = α/k where K is the number of comparisons being considered. If there are k groups, then usually all possible pairs are compared and K = k(k 1) 2. Clicker question In the aldrin data set depth has 3 levels: bottom, mid-depth, and surface. If α = 0.05, what should be the modified significance level for two sample t tests for determining which pairs of groups have significantly different means? (a) α = 0.05 (b) α = 0.05/2 = (c) α = 0.05/4 = (d) α = 0.05/6 = (e) α = 0.05/6 = Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1

9 Which means differ? Which means differ? (cont.) Based on the box plots below, which means would you expect to be significantly different? LOWER CLASS WORKING CLASS MIDDLE CLASS UPPER CLASS When doing multiple comparisons after, since the assumption of equal variability across groups must have been satisfied, we re-think how we measure the standard error and the degrees of freedom. For all comparisons, use a consistent SE: calculate SE using s pooled = MSE instead of s 1 and s 2. s 2 1 SE = + s2 2 MSE SE = + MSE n 1 n 2 n 1 n 2 df: use df = df E from instead of df calculated based on individual sample sizes n 1 and n 2. df = min(n 1 1, n 2 1) df = df E Finally, compare the p-value of this test to the modified significance level (α ). Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1 [Time permitting] Is there a difference between the average vocabulary scores between middle and lower class Americans? T dfe = ( x middle x lower ) T 791 = MSE n middle + MSE n lower ( ) = = p value = (two-sided) α = 0.05/6 = Reject H 0, the data provide convincing evidence of a difference between the average vocabulary scores of those from the lower and middle classes. Statistics 104 (Mine Çetinkaya-Rundel) U4 - L4: October 24, / 1