Student s t-distribution. The t-distribution, t-tests, & Measures of Effect Size

Transcription

1 Student s t-distribution The t-distribution, t-tests, & Measures of Effect Size

2 Sampling Distributions Redux Chapter 7 opens with a return to the concept of sampling distributions from chapter 4 Sampling distributions of the mean

3 Sampling Distribution of the Mean Because the SDotM is so important in statistics, you should understand it The SDotM is governed by the Central Limit Theorem Given a population with a mean μ and a variance σ, the sampling distribution of the mean (the distribution of sample means) will have a mean equal to μ, a variance equal to σ /n, and a standard deviation equal to σ / n. The distribution will approach the normal distribution as n, the sample size, increases. (p. 178)

4 Sampling Distribution of the Mean Translation: 1. For any population with a given mean and variance the sampling distribution of the mean will have: μ x μ σ x σ /n σ x σ/ n. As n increases, the sampling distribution of the mean (μ x ) approaches a normal curve

5 Sampling Distribution of the Mean Analysis: Although μ x and μ will tend to be similar to one another The relationships between σ x and σ σ x and σ will differ as a function of the sample size We saw this in our sampling distribution of the mean example from chapter 4

6 So, you wanna test a hypothesis, do ya? Our understanding of sampling and sampling distributions now allows us to test hypotheses How we test a hypothesis depends on the information we have available

7 Choosing a Test μ? σ? s? Number of data sets: 1 Number of Groups 1 1. Which variables are available?. How many data sets are you presented with? 3. Do your data sets come from 1 or groups?

8 Testing Hypotheses about Means: The Rare Case of Knowing σ So far, to test the probability of finding a particular score, we ve used the Standard Normal Distribution IQ 83 μ 100 σ 15 z ( x x) σ ( ) z 15 z z ( 17) < z < 1.96 Fail to reject H 0

9 The Rare Case of Knowing σ Remember: we rarely know the population mean and standard deviation This test can ONLY be used in situations where the population mean and standard deviation are known!

10 The Rare Case of Knowing σ: Stick with IQ: μ 100 σ 15 However, we want to test a group of children against the population values for IQ n 5 (a group of 5 children)

11 The Rare Case of Knowing σ: Research Hypothesis: The children s IQ scores are different from the population IQ scores H 1 : μ c μ p Null Hypothesis The children s IQ scores do not differ from the population IQ scores H 0 : μ c μ p Test the students (x-bar 1)

12 The Rare Case of Knowing σ: Select: Rejection region α.05 Tail or directionality We don t know exactly how the students will score: we just expect them to show scores differing from the population values

13 The Rare Case of Knowing σ: The z-test Generate sampling distribution of the mean assuming H 0 is true z-test Given our sampling distribution: Conduct the statistical test

14 Conducting the z-test z ( x μ) σ n Note: this equation is a modification of the original z-score formula This formula adjusts z for sample size according to the rules of the central limit theorem z ( 1 100) 15 5 z () 15.4 z () 6.70 z 3.8 z 3.8 > 1.96 : Reject H 0

15 How the z-test Works n 100 n z z ( 1 100) ( 1 100) 15 z z () () () z z () z z. 07 n 1 z ( 1 100) 15 1 z () 15 1 () z z

16 How the z-test Works Large samples reduce the amount of random variance (sampling error) More confidence that the sample mean population mean Larger samples improve our ability to detect differences between samples and populations For n 1 z ( x μ) σ n z ( μ) x σ

17 Statistical Tests We Have Learned 1. z-test

18 Testing Hypotheses: When σ Is Unknown Generally, the population standard deviation, σ, is unknown to us Occasionally, we will know the population mean, μ, when we don t know σ In these situations, the standard normal distribution no longer meets our needs

19 Knowing μ Testing Hypotheses: When σ Is Unknown We can produce an estimate of σ from s Changes the nature of the test we are conducting, as s is not distributed in the same fashion as σ Sampling distribution of the sample standard deviation is NOT normally distributed Strong positive skew

20 Testing Hypotheses: When σ Is Unknown Sampling distribution of s Sampling distribution of σ

21 So How Does s Estimate σ? Given the differences in distribution shape, it is easy to conclude that s σ s is an unbiased estimator of σ over repeated samplings However, a SINGLE value of s is likely to underestimate σ Because of this fact, small samples will systematically underestimate σ as a function of s This leads to any given statistic calculated from this distribution to be < a comparable value of z We cannot use z any longer t

22 t and the t-distribution Developed by Student while he was working for the Guinness Brewing Co. 1. The shape of the t-distribution is a direct function of the size of the sample we are examining. For small samples, the t-distribution is somewhat flatter than the standard normal distribution, with a lower peak and fatter tails

23 t and the t-distribution 3. As sample size increases: The t-distribution approaches a normal distribution Theoretically, we mean that the closer that our sample comes to infinity, the more it looks like a normal distribution Practically, when n ~

24 t and the t-distribution

25 t and the t-distribution 4. Identifying values of t associated with a given rejection region depends on: α the number of tails associated with the test the degrees of freedom available in the analysis For this one-sample test, (df n-1) because we used one degree of freedom calculating s using the sample mean and not the population mean.

26 One-Sample t-test t ( x μ) s x or t ( x μ) s x n or t ( x μ) s x n

27 z-test vs. One-Sample t-test ( x μ) ( x μ) z t σ s x n n Note the similarities between these tests: ONLY the source of variance and the distribution you test against have changed!

28 Using the One-Sample t-test You are one the admissions board for a graduate school of Psychology. You are attempting to determine if the GRE scores for the students applying to your program is competitive with the national average. μ Verbal 569 x-bar 643 s 8 n 4

29 Using the One-Sample t-test Research Hypothesis: The GRE scores from your applicants differ from the population norms H 1 : μ a μ p Null Hypothesis The GRE scores from your applicants do not differ from the population norms H 0 : μ a μ p Evaluate the students GRE-V scores

30 Using the One-Sample t-test Select: Rejection region α.05 Tail or directionality We don t know exactly how the students will score: we just expect them to show scores differing from the population values Might predict higher scores

31 Using the One-Sample t-test Generate sampling distribution of the mean assuming H 0 is true One-Sample t-test Given our sampling distribution: Conduct the statistical test

32 Using the One-Sample t-test t t ( x μ) s x n ( ) 8 4 μ Verbal 569 x-bar 643 s 8 n 4 (74) (74) t t t This numerical value is called t obt t obt (3) 4.4

33 First note: Evaluating Statistical Significance of the t-test α.05 Tail or directionality: two-tailed t-value 4.4 Degrees of freedom (df) For the One-Sample t-test, df n-1 (4-1 3) Estimating s from x-bar (not σ from μ)

34 p. 747 in Howell Text 1) Find row for TAIL ) In the ROW for the correct tail, find α 3) Find df 4) Track ROW of df across to COLUMN of α The numerical value you obtain is called t crit t crit (3).069

35 Evaluating Statistical Significance of the t-test Compare t crit to our t obt value If t obt falls into the rejection region identified by t crit, then we reject H 0 If t obt does not fall into the rejection region identified by t crit, then we fail to reject H 0

36 Evaluating Statistical Significance of the t-test t obt 4.4 t crit t crit.069 Because t obt falls within the rejection region identified by t crit we reject H 0 0

37 Statistical Tests We Have Learned 1. z-test 1 group μ & σ known. One-Sample t-test 1 group μ known Estimate σ with s using x-bar

38 Testing Hypotheses: Two Matched (Repeated) Samples Sometimes, we re interested in how a single set of scores change over time Psychotherapy tx influences depression Patients respond to medication Consumer attitudes before and after an advertisement When we look at two sets of scores collected from a single sample at different time points, we need to use a matched samples test

39 Matched Samples Matched samples Use the same participants at two or more different time points to collect similar data MUST BE THE SAME SAMPLE! Time 1 Wait 30 Days Time BDI - II BDI - II

40 Matched Samples Test With a matched samples test, you are testing the change in scores between the two administrations of the test H 0 : μ 1 μ H 0 : μ 1 - μ 0 This is truly the null hypothesis for the matched samples test There is a difference

41 Matched Samples Test Essentially, the group means at each time point mean little to us Change in scores is the key Conduct this test by obtaining the average difference score between the two time points

42 Matched Samples Test t D 0 s D n D-bar represents average difference scores between time points s D is the standard deviation of the difference scores -0 may seem redundant, but isn t!

43 Calculating the Matched Samples t-test You are a researcher examining the impact of a new therapy intervention on the incidence of self-injurious behavior (SIB) You collect a measure of the frequency of self-injurious acts when clients enter your treatment (time 1) You collect a measure of the frequency of self-injurious acts two weeks later (time )

44 Research Hypothesis: Calculating the Matched Samples t-test The new treatment will change SIB scores H 1 : μ 1 μ Null Hypothesis The SIB scores at time will be the same as the scores at time 1 (no change) H 0 : μ 1 μ H 0 : μ 1 - μ 0 Evaluate SIB at time 1 & time

45 Using the One-Sample t-test Select: Rejection region α.05 Tail or directionality We don t know exactly how the treatment will work, so we d better use a two-tailed test

46 Using the One-Sample t-test Generate sampling distribution of the mean assuming H 0 is true Matched Samples t-test Given our sampling distribution: Conduct the statistical test

47 Calculating the Matched Samples t-test Time Time D D D 43 D-bar 3.91 D 193 ( D) 1849

48 Calculating the Matched Samples t-test s s ( x) x n ( n 1) (10) s s (11 1) (10) 4.91 s s (10) s s 1. 58

49 Calculating the Matched Samples t-test t D 0 s D n t t t t t obt 8.15

50 First note: Evaluating Statistical Significance of the t-test α.05 Tail or directionality: two-tailed t-value 8.15 Degrees of freedom (df) For the Matched Samples t-test: df number of PAIRS of scores -1 df

51 p. 747 in Howell Text 1) Find row for TAIL ) In the ROW for the correct tail, find α 3) Find df 4) Track ROW of df across to COLUMN of α The numerical value you obtain is called t crit t crit (10).8

52 Evaluating Statistical Significance of the t-test t obt 8.15 t crit -.8 t crit.8 Because t obt falls within the rejection region identified by t crit we reject H 0 0

53 Statistical Tests We Have Learned 1. z-test 1 group 1 set of data μ & σ known. One-Sample t-test 1 group 1 set of data μ known Estimate σ with s using x-bar 3. Matched Samples t-test 1 group sets of data μ & σ unknown Estimate σ D with s D using D-bar

54 Testing Hypotheses: Two Independent Samples Probably the most common use of the t- Test and the t-distribution Compare the mean scores of two groups on a single variable IV: Groups DV: Variable of interest Groups must be independent of one another Scores in 1 group cannot influence scores in the other group

55 Testing Hypotheses: Two Independent Samples Actually uses a different sampling distribution Sampling distribution of differences between means However, we calculate and test t in essentially the same fashion

56 Independent Samples t-test 1 1 x s x X X t n s n s X X t + or This test is calculated by dividing the mean difference between two groups by the dispersion or variation observed between the two groups

57 Independent Samples t-test: Degrees of Freedom 1 df lost for each σ estimated by s using x- bar Since there are two independent groups in this analysis, we must estimate σ twice df(n 1 + n ) -

58 Independent Samples t-test: Example Let s return to the example used for the matched samples test As a competent researcher, you realize that simply showing a change over time is not enough to prove the efficacy of your treatment People spontaneously change over time Show that an untreated control group does not change over the same period of time that your treatment group does change

59 Independent Samples t-test: Example Time 1 Time Time 3 Tx Group SIB Tx SIB Scores Scores? Ctrl Group SIB Scores SIB Scores Tx SIB Scores

60 Independent Samples t-test: Example At time 1, the control and treatment SIB groups have equal SIB scores Administer the treatment for weeks to Tx group The Control group receives no intervention during these two weeks Compare SIB scores of Tx and Control group after weeks Provide Control group w/ intervention if desired

61 Independent Samples t-test: Research Hypothesis: Your treatment for SIB will reduce SIB scores in the Tx group after weeks H 1 : μ t < μ c Null Hypothesis Example Your treatment for SIB will have no effect H 0 : μ t μ c Evaluate the efficacy of your treatment

62 Independent Samples t-test: Time Data Example Control Tx Control Group x 135 x 179 ( x) 185 x-bar 1.7 s 7.9 s.69 n 11 Tx Group x 93 x 941 ( x) 8649 x-bar 8.45 s s 3.93 n 11

63 Select: Independent Samples t-test: Rejection region α.05 Tail or directionality Example We have evidence that the treatment probably works, so we make a one-tailed hypothesis here (scores for the Tx group will be lower than the Control group at time )

64 Independent Samples t-test: Example Generate sampling distribution of the mean assuming H 0 is true Independent Samples t-test Given our sampling distribution: Conduct the statistical test

65 Independent Samples t-test: Example t X s 1 n X s n t t t t t.65 t obt (0) -.65

66 First note: Evaluating Statistical Significance of the t-test α.05 Tail or directionality: one-tailed t-value -.65 Degrees of freedom (df) For the Independent Samples t-test (n 1 + n ) - (11+11)- - 0

67 p. 747 in Howell Text 1) Find row for TAIL ) In the ROW for the correct tail, find α 3) Find df 4) Track ROW of df across to COLUMN of α The numerical value you obtain is called t crit t crit (0) 1.75

68 Evaluating Statistical Significance of the t-test t obt -.65 t crit Because t obt falls within the rejection region identified by t crit we reject H 0 0

69 Independent Samples t-test: One Complication There is a slight problem with the form of the equation we used ONLY can be applied to groups with equal sample sizes A major limitation in real-world research t X s 1 n X s n

70 Pooled Variance Estimate This equation permits tests with different sample sizes Generates an estimate of the total variance between groups weighted by the size of each group Therefore, larger samples have a greater impact on the variance Vice-versa for small samples

71 Pooled Variance Estimate s p ( n 1 1) s n n ( n 1) s

72 Using the Pooled Variance Estimate n s n s X X t n s n s X X t p p n n s X X t p +

73 Using the Pooled Variance Estimate: Example Time Data Control No Data Tx Control Group x 83 x 1171 ( x) 6889 x-bar s 4.57 s.14 n 6 Tx Group x 93 x 941 ( x) 8649 x-bar 8.45 s s 3.93 n 11

74 s p ( n 1 1) s n 1 1 Using the Pooled Variance Estimate: Example + + ( n n (10) s p p 15 1) s (5)4.57 s p (11 1) (6 1) s p s p s

75 Using the Pooled Variance Estimate: Example t t X s 1 X 1 n p n 11.84( t.0909) ( 11 t ) (.576) t t t t obt (15) -3.07

76 First note: Evaluating Statistical Significance of the t-test α.05 Tail or directionality: one-tailed t-value Degrees of freedom (df) For the Independent Samples t-test (n 1 + n ) - (11+6)

77 p. 747 in Howell Text 1) Find row for TAIL ) In the ROW for the correct tail, find α 3) Find df 4) Track ROW of df across to COLUMN of α The numerical value you obtain is called t crit t crit (15) 1.753

78 Evaluating Statistical Significance of the t-test t obt t crit Because t obt falls within the rejection region identified by t crit we reject H 0 0

79 Effect Size of The Independent Samples t-test d μ μ 1 or d X 1 X σ s p d.0 -- Small effect d Medium effect d Large effect

80 Effect Size of The Independent Samples t-test d X 1 X d s p 3.44 d d 1.56 An effect size exceeding the convention for a large effect

81 Statistical Tests We Have Learned 1. z-test 1 group 1 set of data μ & σ known. One-Sample t-test 1 group 1 set of data μ known Estimate σ with s using x-bar 3. Matched Samples t- Test 1 group sets of data μ & σ unknown Estimate σ D with s D using D-bar 4. Independent Samples t-test groups sets of data μ & σ unknown Estimate σ twice with s using x-bar

82 Choosing the Best Test

83 Choosing the Best Test Flow-chart available on the website: Also refer to the diagram on p. 11 of your Howell text Try the review problems on the website for an example of the types of questions I might ask on an exam!