16.400/453J Human Factors Engineering. Design of Experiments II

Transcription

1 J Human Factors Engineering Design of Experiments II

2 Review Experiment Design and Descriptive Statistics Research question, independent and dependent variables, histograms, box plots, etc. Inferential Statistics (in Bluman Chapt 6 & 7) Parameter estimates for a population given sample mean ( X) & sample variance (s ) Distributions Normal, N(μ, σ), vs. standard normal, N(0, ) Student s t (degrees of freedom) Binomial (in text & problem set) Probabilities of occurrence Confidence Intervals Sample sizes

3 Formulas Population Sample, (Estimators for the population) Mean μ X (X ) Variance s N (X X) Standard Deviation σ s n

4 More Formulas Population of Sample Means Estimators for (Central Limit Theorem) Population of Sample Means Mean μ X Variance N s (n) Standard N Deviation aka Standard Error of the Mean s n X To convert N(μ, σ ) to N(0, ): z or z When N < 30 (small sample), use t statistics instead of z. X X s

5 Other Topics in Bluman Chapter 8 Hypothesis Testing Proportions (z test statistic) Variances (Chi-square statistic) Chapter 9 Two-sample tests for means, variance, and proportion Large samples, small samples Dependent and independent means Blue items will be covered in this lecture 5

6 Hypothesis Testing Null hypothesis (H o ): the independent variable has no effect Alternative hypothesis (H a ): any hypothesis that differs from the null Significance (p-value) How likely is it that we can reject the null hypothesis? 6

7 Hypothesis Testing & Errors True in the World Outcome H o H a Reject H o Accept H o Type I Error α Correct Decision (- α) Correct Decision (- β) (power) Type II Error β 7

8 Alpha & Beta 8

9 Hypothesis & Tails One-Tailed Left One-Tailed Right Two-Tailed H o μ k μ k μ = k H a μ<k μ > k μ k Example If people try your diet, they will lose weight. If people try your exercise routine, their muscle mass will increase. If you test students more often, their grades will change.(maybe up, maybe down) Your hypothesis will guide your selection Two-tailed test tells you if values are or are not different; you 9 have no a priori expectations for the direction of the difference.

10 One-Tailed vs. Two-Tailed Tests H o : μ 0 versus H a : μ > 0 H o : μ = 0 versus H a : μ 0 0

11 How to Use Confidence Intervals Confidence Level = - α α, Type I error, rejecting a hypothesis when it is true Commonly used confidence intervals and critical values for the standard normal distribution 90% z α/ =.65 95% z α/ =.96 99% z α/ =.58 Whether a hypothesis is true translates to how likely or unlikely it is that the data were obtained due to chance Adjusting the size of the confidence interval adjusts the likelihood that the data could have occurred by chance.

12 Original Procedure State hypothesis and identify claim Find critical value Compute test statistic Decide whether or not to accept or reject Summarize results Applies to z & t tests for means Test statistic = observed value expected value standard error X z n X t s n

13 Hypothesis Testing Example A researcher claims the average salary of assistant professors > $4,000 is this true? Sample of 30 has a mean salary of $43,60, σ = $5,30 (α =.05) X H o : μ $4,000 z H a : μ > $4,000 n Critical value? Right tailed test, z =.65 Test value =.3 Ca n t reject the null What if sample < 30? DO NOT REJECT REJECT

14 A More Useful Procedure State hypothesis and identify claim Find critical value Compute test statistic Find p-value Decide whether or not to accept or reject Summarize results Compute test statistic You get a little more information from this approach 4

15 Hypothesis Testing Example Again A researcher claims the average salary of assistant professors > $4,000 is this true Sample of 30 has a mean salary of $43,60, σ = $5,30 (α =.05) X H o : μ $4,000 H a : μ > $4,000 Test Statistic =.3 p-value of.3 =.0934 ( ) Using standard normal table Also cannot reject (but marginal!) z n 5

16 Hypothesis Testing: t Tests for a Mean Use when σ is unknown and/or n < 30 X t DOF = n- s n Example: Job placement director claims average nurse starting salary is $4K, is this true? Sample of 0 nurses has a mean of $3,450, SD = $400 (α =.05) H o vs. H a? Critical value = +/-.6.why? Test statistic = Reject - draw a picture! tailed appropriate? 6

17 Two Tailed t-test Sample of 0 nurses, mean of $3,450, SD = $400 (α =.05) REJECT DO NOT REJECT REJECT Critical value = +/-.6.why? Test statistic =

18 Two Sample Tests for Large Populations Assumptions: Independent samples, between subjects Population normally distributed SD known or sample size > 30 H o : μ = μ and H a : μ μ can be restated as H o : μ - μ = 0 and H a : μ - μ 0 ( X X ) ( z Either one or two tailed n n ) Pooled variance = sum of the variances 8 for each population

19 Two Sample for Small Populations t tests Assumptions: Independent samples Population normally distributed Sample size < 30 Unequal vs. equal variances of population To determine whether variances are different, use F test If variances are equal, use pooled variance. (Otherwise, out of scope) ) ( ) ( n s n s X X t DOF = smaller of (n -) or (n -) ) ( ) ( ) ( ) ( n n n n s n s n X X t DOF = n + n - Pooled variance = sum of the variances for each population 9 If sample sizes are very different, use a weighted average.

20 t-tests for Matched (Dependent) Samples Matched samples (within subjects) Learning, medical trials, etc. ~ normally distributed data t D s D D n D n D s D D n n D D = X -X, DOF = n-, μ D is part of the hypothesis 0

21 Comparing Variances F-test is one option Independent samples, normally distributed population Ratio of two Chi-square distributions Other, more complicated, but better options exist DOF for numerator: n -, DOF for denominator: n - s is larger of variances F If DOF not listed, use lower (to be conservative) s s

22 Comparing Variances - Example Hypothesis: SD for exam grade for males is larger for males than females is this true (α = 0.0)? Males: n = 6, s = 4. Females: n = 8 s =.3 H : 0 Ha : DOF(n) = 5, DOF(d) = 7, table critical value = 3.3 F s s 4..3 = 3.33, so reject the null

23 Linear regression Other Tests Correlations Analysis of variances (ANOVA) Testing the differences between two or more independent means (or groups) on one dependent measure (either a single or multiple independent variables). Uses the F test to test the ratio of variances Most flexible tests (for mixed models, repeated measures etc.) 3

24 Correlations among the pilot characteristics and experience from Chandra (009) DOT-VNTSC-FAA VFR/IFR Pilots (43 VFR/30 IFR) Correlations Example Private VFR (77) FAA Chart Experience (77) Jeppesen Experience (3) Flight Length International (58) Air Transport (76) VFR/IFR Pilots Private VFR FAA Chart Experience Jeppesen Experience Flight Length International Air Transport Flight Hours Flight Hours All values are significant at p < 0.0. Strong positive correlations appear in the top left and bottom right. Strong negative correlations appear in the bottom left and top right. Sample sizes are given in parentheses in top row. VFR =, IFR = 0 Flight length has 4 categories ranging from short to long.

25 ANOVA ANOVA Nonparametric Includes a categorical variable NO Pearson Correlation NO YES Data Normal? YES Only one independent variable NO Only betweensubjects variables NO Mixed two-way Anova YES YES One-way analysis of Variance NO Only two levels Between-subjects -way ANOVA YES Within-subjects t-test NO Only between subjects variables YES Between subjects t-test Image by MIT OpenCourseWare.

26 Aeronautical Charting Example Continued 30% 5% 0% 5% 0% 5% 0% ANOVA ARTCC Accuracy in Identifying Air Traffic Control Center Boundary Jeppesen Chart Users FAA Chart Users Jeppesen Chart Users FAA Chart Users N 50 7 Mean Std Dev Std Error Variance Pooled variance = 0.45 Z score, assuming equal sample size =.56, p <0.0 Z score corrected for unequal sample sizes =.8, p <0.05 Output From SPSS Between Groups Within Groups Sum of Squares Sum of 6 Squares df df 65 Mean Square Mean Square F Sig F Sig.

27 Practical Questions What confidence level (α) should I use? Number of simultaneous tests performed Statistical significance vs. operational significance Relative differences between effects when many comparisons are made Ability to explain the effect aside from the statistics What test should I use when? How many subjects should I test? Face validity, resource considerations, power calculations 7

28 In an imperfect world Complex designs Repeated measures Mixed models Combination of within & between subjects Lots of trials required Lots of subjects required Unplanned complexities Missing data Unequal cell sizes Experiment confounds Additional course work Mathematical and/or practical perspectives

29 MIT OpenCourseWare / Human Factors Engineering Fall 0 For information about citing these materials or our Terms of Use, visit: