COSC 341 Human Computer Interaction Dr. Bowen Hui University of British Columbia Okanagan 1
Last Topic Distribution of means When it is needed How to build one (from scratch) Determining the characteristics of one Z- test Revised hypothesis testing procedure with: Sample of 1+ individuals A distribution of means as comparison distribution Revised formula: Z = (M μ M )/σ M 2
This Topic Build on existing concepts Hypothesis testing procedure Distribution of means What s new: Estimate variance/standard deviation t distribution T- test For one sample For dependent means (two samples) 3
Scenario: Reported Studies with Mean Course union study found on average, students in COSC spend 2.5 hours studying for their midterms But you think students in this class study more. You randomly pick 16 students from the class and found they studied on average 3.2 hours Can you conclude students in this class are statistically significantly more studious than the average COSC student? 4
What s Different? We don t have a known population anymore! For Z- test, we used to: Compare mean of one sample to a population with known mean and known variance What to do now? Comparison of mean of one sample to a population with known mean and unknown variance Solution: Estimate variance using t- test 5
What s Different? We don t have a known population anymore! For Z- test, we used to: Compare mean of one sample to a population with known mean and known variance What to do now? Comparison of mean of one sample to a population with known mean and unknown variance Solution: Estimate variance and test using t- test 6
1. Setting up Hypotheses Population 1: The kind of students who study COSC 341. Population 2: The kind of students who study COSC generally. H R : Population 1 students study more than Population 2 students. H 0 : Population 1 students study the same amount or less than Population 2 students. 7
2. Characteristics of the Comparison Distribution Given, about test sample: N = 16 16 individual data points X 1, X 2,, X 16 M = 3.2 Given, about population 2: μ = 2.5 σ 2 = need to estimate! Call this estimate S 2 8
Computing S 2 Recall: Your sample is a random sample from population Variance of sample ought to reflect variance of population Variance of your sample provides an informed guess about variance of the population However, variance of sample will generally be slightly smaller than variance of its population Creating a biased estimate of population variance Consistently underestimates actual population variance Solution: Create an unbiased estimate of population variance 9
Computing S 2 Recall: Your sample is a random sample from population Variance of sample ought to reflect variance of population Variance of your sample provides an informed guess about variance of the population However, variance of sample will generally be slightly smaller than variance of its population Creating a biased estimate of population variance Consistently underestimates actual population variance Solution: Create an unbiased estimate of population variance 10
Computing S 2 An unbiased estimate of population variance: S 2 = Σ(X i M) 2 / (N 1) i where: Σ(X i M) 2 = sum of squared deviation scores i N = number of scores N 1 = degrees of freedom (df) Estimated population standard deviation: S = S 2 11
2. Characteristics of the Comparison Distribution Given, about test sample: N = 16 M = 3.2 Σ(X i M) 2 = 9.6 (given for simplicity) i Given, about population 2: μ = 2.5 S 2 = Σ(X i M) 2 / (N 1) = 9.6/16 1 = 0.64 i Use your sample data to estimate variance of underlying population 12
2. Characteristics of the Comparison Distribution Given, about test sample: N = 16 M = 3.2 Σ(X i M) 2 = 9.6 (given for simplicity) i Given, about population 2: μ = 2.5 S 2 = Σ(X i M) 2 / (N 1) = 9.6/16 1 = 0.64 i Comparison distribution: μ M = μ = 2.5 S 2 M = S 2 / N = 0.64 / 16 = 0.04 same calculation as in z- test, but we use estimates instead of actual population variance 13
2. Characteristics of the Comparison Distribution Comparison distribution: μ M = 2.5 S 2 M = 0.04, so S M = 0.20 Shape: t distribution To account for more extreme means 14 fatter tails: need more extreme sample mean to achieve stat. sig.
Family of t distributions As a function of df Note: infinite sample size - > t distribution = normal distribution Image taken from www.real- statistics.com 15
3. Cutoff Sample Score on Comparison Distribution Use 5% or 1%? Is this a one- tailed or two- tailed test? Cutoff t score is 1.753 Cutoff Z score for df = 15 is? 16
4. Determine Sample Score on Comparison Distribution M = 3.5 (test sample s mean from 16 people) Compute t score for μ = 2.5 t = (M μ)/s M = (3.2 2.5)/0.20 = 0.70/0.20 = 3.50 Recall: Z = (M μ M )/σ M 17
18
5. Decide to Reject H 0 or Not Is data s t score more extreme than cutoff? Yes: 3.50 > 1.753 Thus, result of the one sample t test is statistically significance at p < 0.05 level Interpretation: Students in COSC 341 study more for an exam than the average COSC student 19
Comparing Two Techniques For each interface from A1 and A2: Interface A and Interface B (your conditions) 8 participants carry out a set of tasks using Interface A then carry out similar tasks using Interface B The same person now has 2 data points Two sets of data Compare the data across the two conditions How to do hypothesis testing? 20
What s Different? We don t have a known population at all! For t- test for one sample, we used to: Compare mean of one sample to a population with known mean and unknown variance What to do now? Change two sample scores to one sample scores Comparison of mean of that sample to a population with a mean of zero Solution: Compute difference scores and test using t- test for dependent means 21
t Test for Dependent Means t test for dependent means Procedure used when each person in sample is measured twice Also called repeated measures design, within subjects design Example scenarios: Measure change in participant scores before and after treatment Measure difference in participant scores for software 1 and software 2 22
Difference Scores and the Comparison Distribution Given two sets of scores: Subtract one score from the other for each person to create a single set of difference scores Mean of population of difference scores is unknown Recall: H 0 indicates no difference between two groups Mean of population of difference scores is 0 Thus, you compare: population of difference scores of your sample to population of difference scores with a mean of 0 23
Worked out Example Scenario: Does premarital counseling impact communication quality of husbands before and after marriage One study with 19 husbands in premarital counseling group Scores on next slide 24
ß Focus on these columns first On average, these 19 husbands communication quality changed by 229/19 = 12.05 points Is this decrease statistically significant? 25
1. Setting up Hypotheses Population 1: Husbands who receive premarital counseling. Population 2: Husbands whose communication quality does not change from before to after marriage. H R : Population 1 s mean difference score is different from Population 2 s mean difference score. H 0 : There is no difference between the two mean difference scores. 26
1. Setting up Hypotheses Population 1: Husbands who receive premarital counseling. Population 2: Husbands whose communication quality does not change from before to after marriage. H R : Population 1 s mean difference score is different from Population 2 s mean difference score. H 0 : There is no difference between the two mean difference scores. 27
1. Setting up Hypotheses Population 1: Husbands who receive premarital counseling. Population 2: Husbands whose communication quality does not change from before to after marriage. Note if H R is correct, then Population 2 doesn t exist! H R : Population 1 s mean difference score is different from Population 2 s mean difference score. H 0 : There is no difference between the two mean difference scores. 28
2. Characteristics of the Comparison Distribution Given, about test sample: N = 19 M = Given, about population 2: μ = 0 (by definition, with no difference) S 2 = Σ( X M) 2 / (N 1) = see data on next slide 29
ß same as ( X M) 2 ß same as Σ( X M) 2
2. Characteristics of the Comparison Distribution Given, about test sample: N = 19 M = 299 / 19 = 12.05 Given, about population 2: μ = 0 (by definition, with no difference) S 2 = Σ( X M) 2 / (N 1) = 2,772.90 / (19 1) = 154.05 31
2. Characteristics of the Comparison Distribution Given, about test sample: N = 19 M = 12.05 Given, about population 2: μ = 0 (by definition, with no difference) S 2 = 154.05 Distribution of means: μ M = μ = 0 S 2 M = S 2 /N = 154.05 / 19 = 8.11 S M = S 2 M = 8.11 = 2.85 32
3. Cutoff Sample Score on Comparison Distribution Use 5% or 1%? Not sensitive data so 5% Is this a one- tailed or two- tailed test? H R : Population 1 s mean difference score is different from Population 2 s mean difference score. So two- tailed What is df? Since df = N 1 then df = 18 Cutoff is? 33
ß this row 34
3. Cutoff Sample Score on Comparison Distribution Use 5% or 1%? Not sensitive data so 5% Is this a one- tailed or two- tailed test? H R : Population 1 s mean difference score is different from Population 2 s mean difference score. So two- tailed What is df? Since df = N 1 then df = 18 Cutoff is ±2.101 35
We had: 4. Determine Sample Score on Comparison Distribution M = 12.05 (test sample s mean from 19 people) μ = 0 S M = 2.85 t = (M μ)/s M = ( 12.05 0)/2.85 = 4.23 36
37
5. Decide to Reject H 0 or Not Is data s t score more extreme than cutoff? Yes: 4.23 < 2.101 Thus, result of the t test for dependent means is statistically significance at p < 0.05 level Interpretation: Husbands communication quality is different after marriage from what it was before marriage (it is lower) 38
Summary One sample t- test Unbiased estimate of population variance t distribution, parameterized by df Look up cutoff Calculate t score Two sample t- test for dependent means Formulate hypothesis Calculate difference scores Apply the above steps Many more statistical analyses in real world 39
Exam Expectations Given scenario Identify relevant populations and state the hypotheses Identify which hypothesis testing procedure should be used and why Given scenario and pre- calculated results Indicate whether to reject H 0 Provide the interpretation of the study 40