GEORGE W. COBB Mount Holyoke College INTRODUCTION TO DESIGN AND ANALYSIS OF EXPERIMENTS Springer
CONTENTS To the Instructor Sample Exam Questions To the Student Acknowledgments xv xxi xxvii xxix 1. INTRODUCTION TO EXPERIMENTAL DESIGN 1 1. The challenge of planning a good experiment 1 The content of an experiment: three decisions 2 Three sources of variability: one we want and two we don't 4 Three kinds of variability: one we want, one we can live with, and one that threatens disaster 6 Chance error and bias compared 10 Design in a nutshell: Isolate the effects of interest control what you can, and randomize the rest 13 2. Three basic principles and four experimental designs 13 Random assignment and the one-way randomized basic factorial (RBF[1]) design 14 Blocking, and the one-way complete block (CB[l]) design 16 Factorial crossing, and the two-way basic factorial (BF[2]) design 19 Kelly's experiment: a split plot / repeated measures design, or SP/RM 21 3. The factor structure of the four experimental designs 22 Factor structure of the one-way basic factorial (BF[1]) design 24 Factor structure of the one-way complete block (CBflJ) design 25 Factor structure of the two-way basic factorial (BF[2]) design 26 Factor structure of the split plot / repeated measures (SP/RM) design 27 Appendix: Other ways to guard against bias 30 2. INFORMAL ANALYSIS AND CHECKING ASSUMPTIONS 38 1. What analysis of variance does 39 2. The Six Fisher Assumptions 42 Two assumptions about unknown true values 42 Four assumptions about residual errors 43 3. Informal analysis, part 1: parallel dot graphs and choosing a scale 45 Averages and parallel dot graphs 46 Choosing a new scale 48
viii CONTENTS 4. Informal analysis, part 2: interaction graph for the log concentrations 52 Interaction graphs 53 Scatterplots 54 3. FORMAL ANOVA: DECOMPOSING THE DATA AND MEASURING VARIABILITY, TESTING HYPOTHESES AND ESTIMATING TRUE DIFFERENCES 61 1. Decomposing the data 62 2. Computing mean squares to measure average variability 64 SS: Sum of squares = measure of overall variability 66 df: degrees of freedom = units of information about residual error 68 MS: Mean square = average variability per unit of information 70 3. Standard deviation = root mean square for residuals 71 4- Formal hypothesis testing: are the effects detectable? 74 The analysis of variance table: summarizing the evidence 75 The logic of the F-test 77 5. Confidence intervals: the likely size of true differences 80 Intervals for the effect of day length on hearts and brains 80 Using the 95% distance to think about design issues 82 Appendix 1: The general decomposition rule 87 Appendix 2: Introduction to the logic of hypothesis tests and confidence intervals 92 4. DECISIONS ABOUT THE CONTENT OF AN EXPERIMENT 108 1. The response 110 Two restrictions 110 Reliability and validity: how good is the response? 114 2. Conditions 118 Isolate the effects of interest 118 Experiments versus observational studies 121 3. Material 125 Units 125 Material should be representative and uniform 126 Appendix: Supplementary Examples 135 5. RANDOMIZATION AND THE BASIC FACTORIAL DESIGN 148 1. The basic factorial design ("What you do") 149 Experimental version: the completely randomised 1 (CR) design. 150 Observational versions of the basic factorial design 152
CONTENTS ix 2. Informal analysis 155 Averages and outliers 155 Parallel dot graphs 157 3. Factor structure ("What you get") 164 4- Decomposition and analysis of variance for one-way BF designs 168 Decomposition of a balanced one-way design 170 Degrees of freedom for a balanced BF[l] design 173 Mean squares, the standard deviation, and the F-test 175 5. Using a computer [Optional] 181 6. Algebraic notation for factor structure [Optional] 185 Notation for the model 185 Notation for the estimates 187 Appendix: Supplementary Examples 196 6. INTERACTION AND THE PRINCIPLE OF FACTORIAL CROSSING 199 1. Factorial crossing and the two-way basic factorial design, or BF[2] 200 Factorial crossing 200 Two-way basic factorial designs: "what you do" 202 Factor structure of the BF[2] design: "what you get" 205 2. Interaction and the interaction graph 209 Interaction as a difference of differences 209 The interaction graph 211 3. Decomposition and ANOVA for the two-way design 216 Decomposition 217 Degrees of freedom 220 Mean squares, standard deviation, and F-tests 221 4. Using a computer [Optional] 226 5. Algebraic notation for the two-way BF design [Optional] 230 Appendix: Supplementary Examples 239 7. THE PRINCIPLE OF BLOCKING 243 1. Blocking and the complete block design (CB) 245 The randomized complete block experiment: what you do 245 Similar units 246 Ways to get blocks 247 Observational studies in complete blocks 248 Factor structure of the one-way CB design 249 2. Two nuisance factors: the Latin square design (LS) 251 Randomization in the LS plan 254
i CONTENTS Factor structure of the LS design 256 Variations on the simplest Latin square plan 256 3. The split plot/repeated measures design (SP/RM) 259 What you do 260 Variations on the basic SP/RM design 262 What you get 263 Factor structure of the basic SP/RM 263 Crossing versus nesting 264 4. Decomposition and analysis of variance 267 The complete block design 267 The Latin square design 271 The split plot/repeated measures design 274 Two kinds of units, two kinds of chance error 276 5. Scatterplots for data sets with blocks 282 Within-bloclcs scatterplots 282 Scatterplots of res. vs. fit 285 6. Using a computer. [Optional] 289 7. Algebraic notation for the CB, LS and SP/RM designs 293 Appendix: Supplementary Examples 303 8. WORKING WITH THE FOUR BASIC DESIGNS 310 1. Comparing and recognizing design structures 311 Preliminary steps: recognizing the units 311 A checklist for analyzing designs 314 2. Choosing a design structure: deciding about blocking 320 Thinking about factors 320 Deciding whether to use blocks 321 Choosing a design: Examples 323 3. Informal analysis: examples 329 4- Recognizing alternatives to ANOVA 339 Mistaking multiple measurements for levels of a factor 341 Mistaking categories of a nominal response for levels of a factor 344 Mistaking numbers on an interval/ratio scale for levels of a factor 347 Appendix: Supplementary Examples 357 9. EXTENDING THE BASIC DESIGNS BY FACTORIAL CROSSING 370 1. Extending the BF design: general principles 370 Question 1. Hotf do you choose the factors to cross? 371
CONTENTS xi Question 2. How do you carry out the experimental plan? 372 Question 3. How do you find the factor structure? 373 Question 4. How is the informal analysis different? 373 2. Three or more crossed factors of interest 375 3. Compound within-blocks factors 381 Two alternative models for residual error structure 384 4. Graphical methods for 3-factor interactions 392 5. Analysis of variance 397 F-ratios for compound within-blocks factors 397 Preliminary F-tests and pooling mean squares 400 Appendix: Supplementary Examples 406 1O. DECOMPOSING A DATA SET 414 1. The basic decomposition step and the BF[1] design 416 The basic decomposition step [SWEEP] 416 Complete decomposition of a balanced BF[l] data set 419 2. Decomposing data from balanced designs 424 Decomposition rule and examples 425 Counting degrees of freedom by subtraction 432 11. COMPARISONS, CONTRASTS, AND CONFIDENCE INTERVALS 436 1. Comparisons: confidence intervals and tests 437 Strategies for choosing comparisons 437 Interpreting the comparisons using standard errors 439 The true value estimated by a comparison 442 The standard error for a comparison 445 Confidence intervals and tests for comparisons 446 2. Adjustments for multiple comparisons 453 Arguments for and against adjusting 454 A strategy for adjusting (if you must) 455 3. Between-blocks factors and compound within-blocks factors 461 The SP/RMll] 461 Pooling Mean Squares 463 The CB12] design 464 The CB and SP/RM families: the "CWIC" rule for choosing MSs 466 4. Linear estimators and orthogonal contrasts [Optional] 470 Linear estimators 470 Orthogonal contrasts (comparisons) 473
xii CONTENTS 12. THE FISHER ASSUMPTIONS AND HOW TO CHECK THEM 482 1. Same SDs (s) 483 Checking the assumption 484 Finding a transformation 486 2. Independent chance errors (I) 497 Ad hoc scatterplots 498 Within-blocks scatterplots 500 The Huynh-Feldt condition for designs with blocks 504 3. The normality assumption (N) 507 Histograms for residuals 507 Normal plots for residuals 509 4- Effects are additive (A) and constant (C) 516 Plots for checking the assumptions 519 Transforming to malce the assumptions fit 521 5. Estimating replacement values for outliers 525 13. OTHER EXPERIMENTAL DESIGNS AND MODELS 535 1. New factor structures built by crossing and nesting 536 Crossing vs nesting: familiar examples, three pictures, four tests 536 Examples of designs with nested factors 541 Purely hierarchical designs 542 Finding the list of factors: a rule 546 Decomposing data sets with nested factors 549 2. New uses for old factor structures: fixed versus random effects 554 Fixed and random factors 555 Using expected mean squares to choose denominators for F-ratios 562 3. Models with mixed interaction effects 571 Preview: Restricted vs unrestricted models for mixed interactions 571 The unrestricted model for mixed interaction terms 574 The restricted model for mixed interaction terms 577 Generalized complete block designs 581 4. Expected mean squares and F-ratios 585 Finding EMSs and choosing denominators 586 Using Minitab 595 Pseudo F ratios: 596 The logic of the EMS rule, part I: Double decomposition diagrams 599 The logic of the EMS rule, part II 606 14. CONTINUOUS CARRIERS: A VISUAL APPROACH TO REGRESSION, CORRELATION AND ANALYSIS OF COVARIANCE 614 1. Regression 614 Introduction to line-fitting 615
CONTENTS xiii Fitting a regression line by least squares 621 Assumptions, tests, and confidence intervals 630 Transformations, extensions and limitations 634 2. Balloon summaries and correlation 652 The correlation coefficient 652 Balloon-based estimates for scatterplots 654 The correlation coefficient and the regression effect 656 Correlation and fraction of variability "explained" 661 Correlation equals, the standardized regression slope 664 3. Analysis of Covariance 673 Why and when to use ANCOVA 673 How to fit the ANCOVA model: computing rules 678 15. SAMPLING DISTRIBUTIONS AND THE ROLE OF THE ASSUMPTIONS 690 1. The logic of hypothesis testing 692 Structure of the argument 692 Sampling distribution of a test statistic 694 2. Ways to think about sampling distributions 702 Finding an equivalent single-draw box model 702 Representing distributions geometrically: dot graphs and histograms 705 Key features: EVs, SEs, and percentiles 709 3. Four fundamental families of distributions 720 The standard normal distribution: a useful fiction 721 The chi-square family: sums of squares for N(0,l) outcomes 732 The t family 733 The F-family 734 4. Sampling distributions for linear estimators 739 The meaning of 95% confidence, and its relation to the t family 739 The sampling distribution of (Est-True)/SE 743 5. Approximate sampling distributions for F-ratios 750 6. Why (and when) are the models reasonable? 754 Sampling from a population 754 Measurement error 759 Randomisation 760 Tables 767 Data Sources 774 Data Index 782 Subject Index 786