The t Test for Two Independent Samples 1 Two-Sample Inferential Statistics In an experiment there are two or more conditions One condition is often called the control condition in which the treatment is not administered The other condition is often called either the treatment condition or the experimental condition; the treatment is administered 2 Example of an Experiment Some people see a brief description of the subject of a passage Other people see nothing Both groups then hear a passage The groups then rate their comprehension of the passage and then recall it What is the control group? What is the experimental group? 3 1
Question Asked in an Experiment The basic question asked in such an experiment is whether the treatment caused an effect That is, are the values of the dependent variables (the measured variables) different in the treatment and control conditions? E.g., did having a context increase comprehension and recall of the passage relative to the control condition? 4 Two-Sample t-tests Inferential statistics are used to answer the primary question in an experiment The particular inferential statistic that you use depends on the experimental design of the study; more about that in Experimental Psychology When there are just two conditions (control and experimental), you often want to use a twosample t-test 5 Types of Two-Sample t Tests Two-sample t tests can be either Independent measures -- using a separate sample for the control and treatment groups, or Also known as a between-subjects design Repeated measures using the same (or matched) groups for the control and treatment conditions Also known as a within-subjects (or matched) design 6 2
Two-Sample t-tests The two-sample t-test is not fundamentally different from the single-sample t-test that we have already discussed There are two primary differences: You have two sample means instead of a single sample mean and a population mean The population standard deviation is unknown, and you have two estimates of it You have one estimate of the population standard deviation from each of the two sample standard deviations 7 Two-Sample t-tests The t-test formula says to take the difference of the means (M 1 M 2 ) and then divide that by the standard error of the difference of the means s (M 1 -M 2 ) Note: This formula only works if n 1 = n 2 8 n 1 n 2 Pooled variance: 9 3
An Example Two groups of students were asked to perform 5 simple tasks at specified times during the next hour One group tied a string (the external memory cue) around their finger to remind them that they had tasks to perform The other group had no external memory cue Did the memory cue change memory 10 performance? An Example The number of tasks completed was recorded for each group Memory Cue No Memory Cue 2 1 2 1 4 3 1 2 3 2 3 1 2 3 2 1 4 4 1 2 n = 10 n = 10 M = 2.8 M = 1.6 SS = 9.6 SS = 4.4 11 Step 1 Write the hypotheses: H 0 : µ memory cue µ no memory cue = 0 H 1 : µ memory cue - µ no memory cue 0 Is it a one-tailed or two-tailed test? Two-tailed the hypothesis does not specify the direction of the change Specify the α level α =.05 12 4
Step 2 Determine the critical regions df = df 1 + df 2 = (n 1 1) + (n 2 1) df = (10 1) + (10 1) = 9 + 9 = 18 Consult a table of critical t values with df = 18, two-tails, α =.05 t critical = 2.101 If the calculated t is larger than 2.101 or smaller than -2.101, reject H 0 13 Step 3 This is an independent samples design (two separate groups of participants) 14 Step 3 15 5
Step 4 Make a decision The calculated t = 3.04 is in the critical region cut off by t critical = 2.101. Reject H 0 It is unlikely that these two samples were drawn from the same population. That is, it is likely the case that the memory cue affected memory 16 Using a Computer Using a computer to perform t-tests is easy The computer will print out the value of t, its degrees of freedom, and a p value The p value is the probability of observing a difference between the means this large due to chance When the p value is less than or equal to the specified α level, you can reject H 0 17 Cohen s d: Effect Size 18 6
Effect Size Cohen s d: Magnitude of d d = 0.2 d = 0.5 d = 0.8 Evaluation of Effect Size Small effect Medium effect Large effect 19 Effect Size Percentage of Variance Explained, r 2 r 2 = 0.01 r 2 = 0.09 r 2 = 0.25 Small effect Medium effect Large effect Both the estimated Cohen s d and the proportion of variance explained claim that this is a large effect 20 Confidence Interval µ 1 µ 2 = M 1 M 2 ± t critical s (M1 M 2 ) 95% CI = 2.8 1.6 ± 2.101 0.3944 0.371 to 2.029 21 7
One-Tailed Example How would the previous example be different if the hypothesis was one-tailed? Did the memory cue improve memory performance? H 0 : µ memory cue µ no memory cue H 1 : µ memory cue > µ no memory cue This is a one-tailed test as the direction of the hypothesized effect is specified α =.05 22 Step 2 Determine the critical regions df = df 1 + df 2 = (n 1 1) + (n 2 1) df = (10 1) + (10 1) = 9 + 9 = 18 Consult a table of critical t values with df = 18, one-tails, α =.05 t critical = 1.734 If the calculated t is larger than 1.734, reject H 0 23 Steps 3 & 4 Step 3, calculating the t, is identical to the two-tailed example t = 3.04 Make a decision The calculated t = 3.04 is in the critical region cut off by t critical = 1.734 Reject H 0 It is likely the case that the memory cue improved memory 24 8
Effect Size & Confidence Interval The effect sizes (estimated Cohen s d and r 2 ) are not influenced by the number of tails Estimated d = 1.36, r 2 =.339 The confidence interval is affected (the critical t value is different) µ 1 µ 2 = M 1 M 2 ± t critical s (M1 M 2 ) 95% CI = 2.8 1.6 ± 1.734 0.3944 0.516 to 1.884 25 Assumptions of Student s t Student s t test makes several assumptions that should be verified prior to accepting the results of a t-test If the assumptions are violated, then you may not be safe in making any conclusions based on the t-value 26 Samples are Independent The data in the two samples should be independent of each other This assumption is violated in repeated measures designs and matched designs 27 9
Normal Distributions The data in each condition should be normally distributed This is often the case in the social sciences Unless the distribution is very non-normal, the t-test will still give good results That is, the t-test is robust -- it will give good results even when some of its assumptions are violated By the central limit theorem, large sample sizes will make the sampling distribution normal 28 Homogeneity of Variance The assumption of homogeneity of variance states that the variance in each condition (i.e. sample) should be equal It is possible to perform a statistical test to determine if the variances are probably different from each other If the variances are found to be different, then a modified form of the t-test should be used Most statistics programs automatically calculate both the normal t-test and the heterogeneous 29 variance t-test 10