INTRODUCTION TO DESIGN AND ANALYSIS OF EXPERIMENTS

Transcription

1 GEORGE W. COBB Mount Holyoke College INTRODUCTION TO DESIGN AND ANALYSIS OF EXPERIMENTS Springer

2 CONTENTS To the Instructor Sample Exam Questions To the Student Acknowledgments xv xxi xxvii xxix 1. INTRODUCTION TO EXPERIMENTAL DESIGN 1 1. The challenge of planning a good experiment 1 The content of an experiment: three decisions 2 Three sources of variability: one we want and two we don't 4 Three kinds of variability: one we want, one we can live with, and one that threatens disaster 6 Chance error and bias compared 10 Design in a nutshell: Isolate the effects of interest control what you can, and randomize the rest Three basic principles and four experimental designs 13 Random assignment and the one-way randomized basic factorial (RBF[1]) design 14 Blocking, and the one-way complete block (CB[l]) design 16 Factorial crossing, and the two-way basic factorial (BF[2]) design 19 Kelly's experiment: a split plot / repeated measures design, or SP/RM The factor structure of the four experimental designs 22 Factor structure of the one-way basic factorial (BF[1]) design 24 Factor structure of the one-way complete block (CBflJ) design 25 Factor structure of the two-way basic factorial (BF[2]) design 26 Factor structure of the split plot / repeated measures (SP/RM) design 27 Appendix: Other ways to guard against bias INFORMAL ANALYSIS AND CHECKING ASSUMPTIONS What analysis of variance does The Six Fisher Assumptions 42 Two assumptions about unknown true values 42 Four assumptions about residual errors Informal analysis, part 1: parallel dot graphs and choosing a scale 45 Averages and parallel dot graphs 46 Choosing a new scale 48

3 viii CONTENTS 4. Informal analysis, part 2: interaction graph for the log concentrations 52 Interaction graphs 53 Scatterplots FORMAL ANOVA: DECOMPOSING THE DATA AND MEASURING VARIABILITY, TESTING HYPOTHESES AND ESTIMATING TRUE DIFFERENCES Decomposing the data Computing mean squares to measure average variability 64 SS: Sum of squares = measure of overall variability 66 df: degrees of freedom = units of information about residual error 68 MS: Mean square = average variability per unit of information Standard deviation = root mean square for residuals Formal hypothesis testing: are the effects detectable? 74 The analysis of variance table: summarizing the evidence 75 The logic of the F-test Confidence intervals: the likely size of true differences 80 Intervals for the effect of day length on hearts and brains 80 Using the 95% distance to think about design issues 82 Appendix 1: The general decomposition rule 87 Appendix 2: Introduction to the logic of hypothesis tests and confidence intervals DECISIONS ABOUT THE CONTENT OF AN EXPERIMENT The response 110 Two restrictions 110 Reliability and validity: how good is the response? Conditions 118 Isolate the effects of interest 118 Experiments versus observational studies Material 125 Units 125 Material should be representative and uniform 126 Appendix: Supplementary Examples RANDOMIZATION AND THE BASIC FACTORIAL DESIGN The basic factorial design ("What you do") 149 Experimental version: the completely randomised 1 (CR) design. 150 Observational versions of the basic factorial design 152

4 CONTENTS ix 2. Informal analysis 155 Averages and outliers 155 Parallel dot graphs Factor structure ("What you get") Decomposition and analysis of variance for one-way BF designs 168 Decomposition of a balanced one-way design 170 Degrees of freedom for a balanced BF[l] design 173 Mean squares, the standard deviation, and the F-test Using a computer [Optional] Algebraic notation for factor structure [Optional] 185 Notation for the model 185 Notation for the estimates 187 Appendix: Supplementary Examples INTERACTION AND THE PRINCIPLE OF FACTORIAL CROSSING Factorial crossing and the two-way basic factorial design, or BF[2] 200 Factorial crossing 200 Two-way basic factorial designs: "what you do" 202 Factor structure of the BF[2] design: "what you get" Interaction and the interaction graph 209 Interaction as a difference of differences 209 The interaction graph Decomposition and ANOVA for the two-way design 216 Decomposition 217 Degrees of freedom 220 Mean squares, standard deviation, and F-tests Using a computer [Optional] Algebraic notation for the two-way BF design [Optional] 230 Appendix: Supplementary Examples THE PRINCIPLE OF BLOCKING Blocking and the complete block design (CB) 245 The randomized complete block experiment: what you do 245 Similar units 246 Ways to get blocks 247 Observational studies in complete blocks 248 Factor structure of the one-way CB design Two nuisance factors: the Latin square design (LS) 251 Randomization in the LS plan 254

5 i CONTENTS Factor structure of the LS design 256 Variations on the simplest Latin square plan The split plot/repeated measures design (SP/RM) 259 What you do 260 Variations on the basic SP/RM design 262 What you get 263 Factor structure of the basic SP/RM 263 Crossing versus nesting Decomposition and analysis of variance 267 The complete block design 267 The Latin square design 271 The split plot/repeated measures design 274 Two kinds of units, two kinds of chance error Scatterplots for data sets with blocks 282 Within-bloclcs scatterplots 282 Scatterplots of res. vs. fit Using a computer. [Optional] Algebraic notation for the CB, LS and SP/RM designs 293 Appendix: Supplementary Examples WORKING WITH THE FOUR BASIC DESIGNS Comparing and recognizing design structures 311 Preliminary steps: recognizing the units 311 A checklist for analyzing designs Choosing a design structure: deciding about blocking 320 Thinking about factors 320 Deciding whether to use blocks 321 Choosing a design: Examples Informal analysis: examples Recognizing alternatives to ANOVA 339 Mistaking multiple measurements for levels of a factor 341 Mistaking categories of a nominal response for levels of a factor 344 Mistaking numbers on an interval/ratio scale for levels of a factor 347 Appendix: Supplementary Examples EXTENDING THE BASIC DESIGNS BY FACTORIAL CROSSING Extending the BF design: general principles 370 Question 1. Hotf do you choose the factors to cross? 371

6 CONTENTS xi Question 2. How do you carry out the experimental plan? 372 Question 3. How do you find the factor structure? 373 Question 4. How is the informal analysis different? Three or more crossed factors of interest Compound within-blocks factors 381 Two alternative models for residual error structure Graphical methods for 3-factor interactions Analysis of variance 397 F-ratios for compound within-blocks factors 397 Preliminary F-tests and pooling mean squares 400 Appendix: Supplementary Examples 406 1O. DECOMPOSING A DATA SET The basic decomposition step and the BF[1] design 416 The basic decomposition step [SWEEP] 416 Complete decomposition of a balanced BF[l] data set Decomposing data from balanced designs 424 Decomposition rule and examples 425 Counting degrees of freedom by subtraction COMPARISONS, CONTRASTS, AND CONFIDENCE INTERVALS Comparisons: confidence intervals and tests 437 Strategies for choosing comparisons 437 Interpreting the comparisons using standard errors 439 The true value estimated by a comparison 442 The standard error for a comparison 445 Confidence intervals and tests for comparisons Adjustments for multiple comparisons 453 Arguments for and against adjusting 454 A strategy for adjusting (if you must) Between-blocks factors and compound within-blocks factors 461 The SP/RMll] 461 Pooling Mean Squares 463 The CB12] design 464 The CB and SP/RM families: the "CWIC" rule for choosing MSs Linear estimators and orthogonal contrasts [Optional] 470 Linear estimators 470 Orthogonal contrasts (comparisons) 473

7 xii CONTENTS 12. THE FISHER ASSUMPTIONS AND HOW TO CHECK THEM Same SDs (s) 483 Checking the assumption 484 Finding a transformation Independent chance errors (I) 497 Ad hoc scatterplots 498 Within-blocks scatterplots 500 The Huynh-Feldt condition for designs with blocks The normality assumption (N) 507 Histograms for residuals 507 Normal plots for residuals Effects are additive (A) and constant (C) 516 Plots for checking the assumptions 519 Transforming to malce the assumptions fit Estimating replacement values for outliers OTHER EXPERIMENTAL DESIGNS AND MODELS New factor structures built by crossing and nesting 536 Crossing vs nesting: familiar examples, three pictures, four tests 536 Examples of designs with nested factors 541 Purely hierarchical designs 542 Finding the list of factors: a rule 546 Decomposing data sets with nested factors New uses for old factor structures: fixed versus random effects 554 Fixed and random factors 555 Using expected mean squares to choose denominators for F-ratios Models with mixed interaction effects 571 Preview: Restricted vs unrestricted models for mixed interactions 571 The unrestricted model for mixed interaction terms 574 The restricted model for mixed interaction terms 577 Generalized complete block designs Expected mean squares and F-ratios 585 Finding EMSs and choosing denominators 586 Using Minitab 595 Pseudo F ratios: 596 The logic of the EMS rule, part I: Double decomposition diagrams 599 The logic of the EMS rule, part II CONTINUOUS CARRIERS: A VISUAL APPROACH TO REGRESSION, CORRELATION AND ANALYSIS OF COVARIANCE Regression 614 Introduction to line-fitting 615

8 CONTENTS xiii Fitting a regression line by least squares 621 Assumptions, tests, and confidence intervals 630 Transformations, extensions and limitations Balloon summaries and correlation 652 The correlation coefficient 652 Balloon-based estimates for scatterplots 654 The correlation coefficient and the regression effect 656 Correlation and fraction of variability "explained" 661 Correlation equals, the standardized regression slope Analysis of Covariance 673 Why and when to use ANCOVA 673 How to fit the ANCOVA model: computing rules SAMPLING DISTRIBUTIONS AND THE ROLE OF THE ASSUMPTIONS The logic of hypothesis testing 692 Structure of the argument 692 Sampling distribution of a test statistic Ways to think about sampling distributions 702 Finding an equivalent single-draw box model 702 Representing distributions geometrically: dot graphs and histograms 705 Key features: EVs, SEs, and percentiles Four fundamental families of distributions 720 The standard normal distribution: a useful fiction 721 The chi-square family: sums of squares for N(0,l) outcomes 732 The t family 733 The F-family Sampling distributions for linear estimators 739 The meaning of 95% confidence, and its relation to the t family 739 The sampling distribution of (Est-True)/SE Approximate sampling distributions for F-ratios Why (and when) are the models reasonable? 754 Sampling from a population 754 Measurement error 759 Randomisation 760 Tables 767 Data Sources 774 Data Index 782 Subject Index 786