Hypothesis testing: Steps

Transcription

1 Review for Exam 2

2 Hypothesis testing: Steps Repeated-Measures ANOVA 1. Determine appropriate test and hypotheses 2. Use distribution table to find critical statistic value(s) representing rejection region 3. Compute appropriate test statistic from data 4. Make a decision: does the statistic for your sample fall into the rejection region or into the acceptance region?

3 Computing Mean-Difference Statistics Repeated-Measures ANOVA Test Sample Data Hypothesized Population Parameter Estimated Standard Error Estimated Variance Degrees of Freedom z-test M µ σ 2 n σ 2 n Singlesample t-test M µ s 2 n s 2 = SS df df = n 1 Relatedsamples t-test M D, where D = x 2 x 1 μ D =0 s D 2 n D s D 2 = SS D df D df = n D 1 Independentsamples t-test M 1 M 2 μ 1 μ 2 =0 s p s p s 2 p = SS 1 + SS 2 df = n n 1 n 2 df 1 + df 1 +n Post-hoc tests M A M B μ A μ B =0 MS error n A + MS error n B MS error = SS error df error df error

4 yes z-test one Is σ provided? no One-sample t-test Number of Samples two Are scores matched across samples? yes no Related samples t-test Independent samples t-test >2 ANOVA

5 Example Scenario A professor thinks that this year s freshman class seems to be smarter than previous classes. To test this, she administers an IQ test to a sample of 36 freshman and computes the mean (M=114.5) and standard deviation (s = 18) of their scores. College records indicate that the mean IQ across previous years was What is the appropriate statistical test for this problem? a) A z-test b) A one-sample t-test c) An independent-samples t-test d) A related-samples t-test e) A one-way independent-measures ANOVA f) A repeated-measures ANOVA

6 Example Scenario A psychologist examined the effect of exercise on a standardized memory test. Scores on this test for the general population form a normal distribution with a mean of 50 and a standard deviation of 8. A sample of 62 people who exercise at least 3 hours per week has a mean score of 57. What is the appropriate statistical test for this problem? a) A z-test b) A one-sample t-test c) An independent-samples t-test d) A related-samples t-test e) A one-way independent-measures ANOVA f) A repeated-measures ANOVA

7 Example Scenario A researcher studies the effect of a drug on nightmares in veterans with PTSD. A sample of clients with PTSD kept count of their nightmares for 1 month before treatment. They were then given the medication and asked to record counts of their nightmares again for a month. What is the appropriate statistical test for this problem? a) A z-test b) A one-sample t-test c) An independent-samples t-test d) A related-samples t-test e) A one-way independent-measures ANOVA f) A repeated-measures ANOVA

8 Example Scenario A neurologist had two groups of patients with different types of aphasia (a brain disorder) and a control group name objects presented to them as line drawings. He wanted to determine whether the number of objects correctly named differed across the three groups. What is the appropriate statistical test for this problem? a) A z-test b) A one-sample t-test c) An independent-samples t-test d) A related-samples t-test e) A one-way independent-measures ANOVA f) A repeated-measures ANOVA

9 Structure of the ANOVA Total Variance SS df SS SS total between within df df total between within SS df SS SS between total within df df between total within Between Treatments Variance SS df SS SS within total between df df within total between Within Treatments Variance SS df Between Subjects Variance SS SS subjects within error df df subjects within error Error Variance SS df SS SS error within subjects df df error within subjects

10 Computational Example 1 A psychologist is investigating the effect of being an only child on personality. A sample of 30 only children is obtained and each child is given a standardized personality test. Population scores on the test form a normal distribution with µ = 50 and σ =15. a) If the mean for the sample is 58, what can the researcher conclude about his hypothesis? Use a two-tailed test with α = b) Compute a 90% confidence interval for the mean of the only child population

11 Clicker Question What is the appropriate statistical test for part (a) of this problem? a) A z-test b) A one-sample t-test c) An independent-samples t-test d) A related-samples t-test e) A one-way independent-measures ANOVA f) A repeated-measures ANOVA

12 Example 1: Research Hypothesis H 1 : Only children have different personality scores than the general population I.e., µ only children µ general population Null Hypothesis H 0 : Only children have personality scores that are not significantly different than those of the general population I.e., µ only children = µ general population We have,, n, and M

13 Find Critical z value z Upper-Tail Probabilities

14 Example1: Compute z-statistic Repeated-Measures ANOVA n 30 M 58.0 z crit 1.96 z M n ; reject H 0 The personalities of only children differ significantly from those of the general population z = 2.92, p =

15 Example 1: Compute Confidence Interval b) Compute a 90% confidence interval for the mean of the only child population Recall that to compute a confidence interval, you: 1. Select a level of confidence and look up the corresponding t α (or z α ) values in the t (or z) distribution table. 2. Use two-tailed probabilities (e.g., for z α, look up p(z>z) = α/2) 3. The confidence interval is computed by inverting the t (or z) transformation CI 0.1 M z M M z n

16 z Upper-Tail Probabilities

17 Example 1: Compute Confidence Interval n 30 M 58.0 z CI 0.90 M z 0. 1 n [53.49, 62.51]

18 Top Incorrect Problems (Exam 2) Repeated-Measures ANOVA If the population from which we sample is normal, the sampling distribution of the mean a) will approach normal for large sample sizes. b) will be normal. c) will be normal only for small samples. d) will be slightly positively skewed.

19 Top Incorrect Problems (Exam 2) Repeated-Measures ANOVA If the population from which we sample is normal, the sampling distribution of the mean a) will approach normal for large sample sizes. b) will be normal. c) will be normal only for small samples. d) will be slightly positively skewed. Why? If the population is normal, then the distribution of the mean for any sample size (including a sample size of 1) will be normal.

20 Top Incorrect Problems (Exam 2) Repeated-Measures ANOVA Sampling distributions help us test hypotheses about means by a) telling us what kinds of means to expect if the null hypothesis is false. b) telling us what kinds of means to expect if the null hypothesis is true. c) telling us how variable the population is. d) telling us exactly what the population mean is.

21 Top Incorrect Problems (Exam 2) Repeated-Measures ANOVA Sampling distributions help us test hypotheses about means by a) telling us what kinds of means to expect if the null hypothesis is false. b) telling us what kinds of means to expect if the null hypothesis is true. c) telling us how variable the population is. d) telling us exactly what the population mean is. Why? Sampling distributions are the distributions of a sample statistic. In many hypothesis tests (e.g., t-tests and ANOVAs), we use these distributions to predict the distribution of means or mean differences under the null hypothesis.

22 Example 2 An educational psychologist studies the effect of frequent testing on retention of class material. In one section of an introductory course, students are given quizzes each week. A second section of the same course receives only two tests during the semester. At the end of the semester, a sample from each of the sections receives the same final exam, and the number of errors made are recorded. X 1 (quiz) X 2 (no quiz) M SS Does frequent testing significantly affect retention of class material? Use a two-tailed test, with α = 0.05.

23 Clicker Question What is the appropriate statistical test for this problem? a) A z-test b) A one-sample t-test c) An independent-samples t-test d) A related-samples t-test e) A one-way independent-measures ANOVA f) A repeated-measures ANOVA

24 Example 2: Research Hypothesis H 1 : Testing frequency affects retention of class material I.e., µ quiz µ no quiz Null Hypothesis H 0 : Testing frequency does not significantly affect retention of class material I.e., µ quiz = µ no quiz We have no population data, and sample data from two independent samples, so we must use the independentsamples t-test df = n 1 + n 2 2 = = 11

25 t-distribution Table α t One-tailed test α/2 α/2 -t t Two-tailed test Level of significance for one-tailed test Level of significance for two-tailed test df

26 Compute t Statistic M SS n 1 M SS n df 11 t crit s M Compute Pooled Variance: s SS SS p df1 df Estimate Standard Error: M 1 2 s n 2 p s n 2 p Compute t-statistic: t df t 11 M s 1 2 M M M ; retain H 0 Frequent testing does not significantly affect the amount of information retained by students t(11) = 1.64, p>0.05.

27 Top Incorrect Problems (Exam 2) Repeated-Measures ANOVA The reason why we need to solve for t instead of z in some situations is due to a) the sampling distribution of the mean. b) the size of our sample mean. c) the sampling distribution of the sample size. d) the sampling distribution of the variance.

28 Top Incorrect Problems (Exam 2) Repeated-Measures ANOVA The reason why we need to solve for t instead of z in some situations is due to a) the sampling distribution of the mean. b) the size of our sample mean. c) the sampling distribution of the sample size. d) the sampling distribution of the variance. Why? We solve for the z-statistic using σ, which is a constant, but we solve for the t-statistic using the sample standard deviation s which varies from sample to sample.

29 Top Incorrect Problems (Exam 2) In the analysis of variance, MS within is a) the sum of squares of the within group variances. b) the average of the between group variances. c) the average of the between group sums of squares. d) the average of the within group variances. Repeated-Measures ANOVA

30 Top Incorrect Problems (Exam 2) In the analysis of variance, MS within is a) the sum of squares of the within group variances. b) the average of the between group variances. c) the average of the between group sums of squares. d) the average of the within group variances. Repeated-Measures ANOVA Why? SS 2 i i dfi MS within i i SS df i i

31 Example 3 A psychologist decides to examine the conclusions of the previous study within a more controlled context. He has a sample of 5 students study for and take a 25 question quiz. The students are told their scores, and are allowed to retake the quiz a second and a third time. The scores in the table below represent the students raw scores on each of their attempts at the quiz. Quiz scores X1 X2 X3 M subj M M T 12.4 SS total a) Does repeated quizzing significantly improve students scores?(use α = 0.05) b) Use the Bonferroni procedure to determine whether either of the mean retest scores are significantly different from the original mean quiz score.

32 Clicker Question What is the appropriate statistical test for part (a) of this problem? a) A z-test b) A one-sample t-test c) An independent-samples t-test d) A related-samples t-test e) A one-way independent-measures ANOVA f) A repeated-measures ANOVA

33 Example 3 Research Hypothesis H 1 : Retesting affects quiz scores Null Hypothesis H 0 : Retesting does not significantly change quiz scores Omnibus Null Hypothesis µ 1 = µ 2 = µ 3 We three (>2) samples from a repeated measures design (same subjects in all three treatment groups), so we should use the repeated-measures ANOVA

34 Compute degrees of freedom df df df total between subjects within error k1 2 n 1 4 df df df df Quiz scores X1 X2 X3 M subj M M T 12.4 SS total N 1 14 total between 12 df df 8 within subjects Compute SS between SS n M M between Compute SS subject SS k M M subjects subj T Compute SS within & SS error T SSwithin SStotal SSbetween SSerror SSwithin SSsubjects

35 Compute MS between, MS error & F MS MS F df between error between SSbetween df 2, df F between SSerror df 8 error error 2,8 MS MS between error Summary: Source df SS MS F Between Within Subjects Error Total

36 F table for α=0.05 reject H 0 df error df between

37 Compute MS between, MS error & F MS MS F df between error between SSbetween df 2, df F between SSerror df 8 error error 2,8 MS MS between error Summary: Source df SS MS F Between Within Subjects Error Total > 4.46; reject H 0

38 b) Use the Bonferroni procedure to determine whether either of the mean retest scores are significantly different from the original mean quiz score. Because we want a familywise α = 0.05 and we will be making two comparisons (i.e., {1,2} and {1,3}), the adjusted pairwise α = 0.05/2 = Also, recall that for Fisher s LSD & the Bonferroni procedure, the t-statistic is computed as: t df error with s M A M s M B A M M A M B B MS n error A MS n error B 2MSerror s when n n A n B n M M A B

39 Example 3: Post-hoc test ANOVA Summary Table Source df SS MS F Between Within Subjects Error Total M M 12 2 M 3 15 n 5 pairwise s M M A B 2MS n error t df error t 8 M s {1,2} {1,3} A B tdf M M A M B error t 8 M s A M M A M B B

40 t-distribution Table α t One-tailed test α/2 α/2 -t t Two-tailed test Level of significance for one-tailed test Level of significance for two-tailed test df

41 Example 3: Post-hoc test ANOVA Summary Table Source df SS MS F Between Within Subjects Error Total M M 12 2 M 3 15 n 5 pairwise s M M A B 2MS n error t df error t 8 M s {1,2} {1,3} A B tdf M M A M B error t 8 M s A M M A M B B , accept H , reject H0