Unit 5: How a Randomized Block Design Differs From a One-Way ANOVA

Transcription

1 Minitab Notes for STT 6305: nalysis of Variance Models Department of Statistics and iostatistics SU East ay Unit 5: How a Randomized lock Design Differs From a One-Way NOV 5.1. First Dataset Real Estate ppraisal In this unit we look at two datasets that illustrate randomized block designs. In the first dataset, three real estate appraisers independently examined each of five properties chosen at random from a particular neighborhood and gave appraised values. The manager of the appraisal service that employs them suspects that systematic differences among their methods of appraisal are causing undesirable inconsistencies in appraised values provided by her service. The issue is whether there are, in fact, systematic differences among the values reported by the three appraisers. The data, appraised values in thousands of dollars, are shown below. PPRISED VLUES (In thousands of dollars) ppraiser Property The data are from anavos and Miller: n Introduction to Modern usiness Statistics, Duxbury, 1993, page 450. The design of this study is a straightforward generalization of the paired-data design we studied in Unit 4. The change is that instead of making a comparison between the bids by two auto repair garages we are making comparisons among the appraisals of three real estate appraisers. In Unit 4 each car produced a pair of bids. Here each property yields a triplet of appraised values. In the language of experimental design, the generalization of pair is block. Here the properties play the role of blocks. Problems ut these data (between dotted lines) from above and paste them directly into columns c1 ('Prop'), c2 (''), c3 (''), and c4 ('') of a Minitab worksheet. (When you paste, put the cursor in row 1 of c1 and use spaces as delimiters.) The property number should go into c1 and the values from appraisers,, and into columns c2, c3, and c4, respectively. Label these columns appropriately. Next, stack the data "in blocks of columns" as shown below. ompare the contents of columns c1 through c4 with the contents of columns c11 through c13. Make a printout of columns c11 through c13. DT > Stack > locks, use column names as subscripts. MT > stack (c2 c1) (c3 c1) (c4 c1) (c11 c12); SU> subs c13; SU> usenames.

2 No menu item; type column names into worksheet MT > name c11 'Value' c12 'Property' c13 'pprsr' DT Display data MT > print c11 c12 c13 Row Value Property pprsr Minitab Notes for STT 6305 lock Design & One Factor NOV Unit 5-2 Notice that c13 has "text" data rather than "numeric" data. Many Minitab procedures can use text data to designate groups, but some of the procedures that originated in older releases require numerical data. (You can use the patterned data procedure to put appraiser numbers 1, 2, 3 into c14. lternatively, you can use the menu path DT > ode > Text to numeric to convert the letters in c13 into numbers in c14. all c14 'pprn'.) Using R. Use the statements below to make the data available in R and print them as a dataframe. Notice that we designate property and appraiser as factor variables from the start. value = c(90, 94, 91, 85, 88, 93, 96, 92, 88, 90, 92, 88, 84, 83, 87) prop = as.factor(rep(1:5, times=3)) apprsr = as.factor(rep(c("", "", ""), each=5)) ppr.val = data.frame(value, prop, apprsr); ppr.val 5.2. Exploratory Methods The sample means of appraised values differ somewhat among the appraisers: 89.6 for, 91.8 for, 86.8 for. (See Problem ) This pattern of the appraised values may show that the differences in sample means reflect actual differences in methodology or standards among the appraisers (i.e., differences in population means). On the other hand, the differences in sample means may be due to chance. The question is which real differences or chance alone? ecause these means do not take into account that each property was evaluated by all three appraisers, the means by themselves do not provide an adequate way to answer the question. For the same reason that two dotplots on the same scale were inappropriate in Unit 4, three dotplots on the same scale would be a poor way to graph the data here. Here we cannot resort to looking at one column of differences as we did in Unit 4. However, an appropriately labeled scatterplot of the data can provide some useful insights (next page). GRPH > Scatterplot > With Groups, Labels, Data labels from columns This plot displays the blocked nature of the data. It is clear that ppraiser gave the highest value for all 5 properties, while ppraiser gave the lowest value for all but one. meaningful pattern of differences does seem to emerge.

3 Minitab Notes for STT 6305 lock Design & One Factor NOV Unit 5-3 Scatterplot of Value vs Property pprsr 92 Value Property 4 5 Problems Find the means for each appraiser with the command table c13; subcommand means c11. or with the command describe c11; subcommand by c13.) Use the command table 'pprsr' 'Property'; with subcommand means 'Value'. to make a table of the data with marginal means Using R. Use the R code below (one block at a time) to make three graphical displays of the data. The first two are also exercises in data types, so explain their use. The interaction plot is intended for use with slightly more advanced designs but serves well here with appropriate labeling; it is programmed to handle factor variables directly. colr = c("red", "blue", "darkgreen") plot(as.numeric(prop), value, pch=as.character(apprsr), col=colr[as.numeric(apprsr)], xlab="property", ylab="value", main="ppraised Values by Property and ppraiser") plot(0, xlim=c(1,5), ylim=c(80,100), type="n", xlab="property", ylab="value", main="ppraised Values by Property and ppraiser") # set up but no plot char = unique(apprsr) for (i in 1:3) { lines(value[as.numeric(apprsr)==i], type="b", col=colr[i], pch=as.character(unique(apprsr))[i]) } # no x-variable given, indices 1:5 used as x interaction.plot(prop, apprsr, value, xlab="property", ylab="value", main="ppraised Values by Property and ppraiser") Using R. Display three boxplots using plot(apprsr, value) and also using boxplot(value ~ apprsr). What do you get with plot(as.numeric(apprsr), value)?

4 5.3. nalysis of Variance Minitab Notes for STT 6305 lock Design & One Factor NOV Unit 5-4 The NOV procedure for analyzing this block design is shown below. (Either numeric or text subscripts for appraisers would work.) MT > anova c11 = c13 c12; SU> rand c12; SU> rest; SU> resid c15; SU> ems. NOV: Value versus pprsr, Property Factor Type Levels Values pprsr fixed 3,, Property random 5 1, 2, 3, 4, 5 nalysis of Variance for Value Source DF SS MS F P pprsr Property Error Total S = R-Sq = 83.71% R-Sq(adj) = 71.49% Expected Mean Square Variance Error for Each Term using Source component term restricted model) 1 pprsr 3 (3) + 5 Q[1] EMS Notation: Q[] = θ, () = σ 2, and 2 Property (3) + 3 (2) number inside [] or () shows Source 3 Error (3) It is clear from this printout that there are significant differences among ppraisers. The F-ratio is F = MS(pp)/MS(Err) = /3.983 = The P-value is the area under the density curve of the distribution F(2, 8) that lies to the right of 7.88, which is Thus, the ppraiser effect is significant at the 5% level, but not quite significant at the 1% level. It is also worthwhile to test the Property effect. y happenstance, the P-value of this test is about the same as for the ppraiser effect. Of course, it is no surprise that the properties are of different values. It is precisely the suspicion that they might be highly variable that lead us to treat Properties as blocks in the design. learly, this was a useful choice of design. The table of expected mean squares is based on the model Y ij = µ + α i + j + e ij, where i = 1, 2, 3; j = 1,..., 5; Σ i α i = 0; j iid N(0, σ 2 ); and e ij iid N(0, σ 2 ). In Minitab's Expected Mean Square (EMS) tables, the terms indicated by parentheses ( ) are variances, where the number inside the parentheses gives the appropriate row label. The terms Q[ ] are quadratic forms involving constants. Specifically for our design, EMS(pp) = σ 2 + 5[Σ i α i 2 ]/2 = σ 2 + 5θ, EMS(Prp) = σ 2 + 3σ 2, EMS(Err) = σ 2.

5 Minitab Notes for STT 6305 lock Design & One Factor NOV Unit 5-5 Formally, the test for the ppraiser effect is the test of the null hypothesis H 0 : θ = 0 against the alternative H a : θ > 0. Notice that, if the null hypothesis is true, then EMS(pp) = EMS(Err) = σ 2. In these circumstances (and, of course, subject to the conditions of normality and equal cell variances specified in the model), the ratio MS(pp)/MS(Err) has an F-distribution with 2 and 8 degrees of freedom. Roughly speaking, because the two MSs then estimate the same quantity σ 2, their ratio ought to be about 1. Thus values very much larger than 1 lead us to suspect that θ is not equal to 0. Similarly, the test for the Property effect is the test of the null hypothesis H 0 : σ 2 = 0 against the alternative H a : σ 2 > 0. If this null hypothesis is true, then EMS(Prp) = EMS(Err), and so it is natural to form the ratio MS(Prp)/MS(Err) to test this null hypothesis in much the same way as we tested for the ppraiser effect. In this relatively simple design it turns out that the F-ratios for both tests of hypothesis have MS(Err) in the denominator. In more complex designs, it is not always appropriate to "test against the error term" in this way. In later units we shall see that the EMS table is an indispensable guide to the formation of appropriate ratios for testing hypotheses in an NOV table. Notes on using the Minitab anova procedure to analyze a block design: The subcommand restrict has minimal effect on output for a block design. If restrict is used, then in the EMS table (generated by the ems subcommand) has EMS() = EMS(prrsr) = σ 2 + bθ (denoted as (3) + 5 Q[1] in the printout above), where b is the number of blocks. ut without the subcommand restrict the printout shows EMS() = σ 2 + θ. This is not a contradictory result; it arises from two different definitions of θ. In some more complex NOV designs where one factor is fixed and another is random, it makes an important difference whether the subcommand restrict is used. We will revisit this issue later in the course. (Some texts recommend using restricted models and others do not. For the current simple design, the distinction makes no difference in the F-ratios or the conclusions drawn from them.) In practice, blocking factors are almost always random. (Some authors will not use the terminology blocking factor except to refer to a random factor.) Often a random blocking factor is introduced into the design in hopes it will "explain" enough variability that MS(Error) of the NOV table tends to become small enough to improve the power of the test for the main factor. In many texts block designs are first introduced without making a distinction whether the blocking factor is considered as a fixed or a random factor. For a simple block design as in this unit, the NOV table (including SS, MS, F, and P columns) is exactly the same whether or not the random subcommand is used. However, the Minitab EMS table shows a variance component for every random factor. So if lock (here Property) is declared as random, then we see an estimate of σ 2 in the EMS table (7.083 above). ut if lock is not declared as random, then the EMS table shows no such estimate. The R procedure in problem below makes essentially the same NOV table as the one shown above from Minitab, and it makes no EMS table. This R procedure is the same as the above NOV procedure where the blocking factor is not declared to be random. ecause the R procedure shows no EMS table, this makes no difference in the output shown. (More advanced R procedures can handle random factors.) Problems Make a normal probability plot of the residuals from this NOV model. Does it show that the data are reasonably close to normal? Perform an appropriate test of normality and give the P-value.

6 Minitab Notes for STT 6305 lock Design & One Factor NOV Unit ompare the test for the ppraiser effect with the test for the Property effect. How is it possible that the two P-values are nearly equal when the F-statistics have quite different values (7.88 vs. 6.33)? Suppose that the Property effect had not been found to be significant. Without using special statistical terminology, explain what this would mean about the values of the five properties selected onsider the variance components in Minitab's EMS table. The variance component for Error is the estimated value of σ 2, which we have already seen to be MS(Err) = The variance component for Property is the estimated value of σ 2. ut MS(Prp) estimates σ 2 + 3σ 2, not σ 2. How can the values of MS(Prp) and MS(Err) be used together to obtain a numerical estimate of σ 2? Show that your answer agrees with the variance component for Property shown in the EMS table Suppose you own 5 similar properties, consider selling them, and wonder if they should all be put on the market at the same price. Then you might consult three appraisers (randomly chosen from among the many available) for advice. In this case, how would the NOV model be changed? One advantage of an appropriately used block design with b blocks (e.g., Properties) is that it makes better use of data in the effort to detect significant differences among t treatment groups (e.g., ppraisers) than would a one-way NOV. The relative efficiency of a block design compared to a one-way NOV is defined as the ratio RE = MS(Err) OneWay /MS(Err) lock. The block design uses bt observations. The idea is that a one-way NOV would require about bt(re) total observations to do the "same job." (See, for example, page 973 in O/L 6e or hapter 14 of Snedecor/ochran for some details.) In practice, data from a legitimate one-way NOV will seldom be available. So we must estimate MS(Err) OneWay from the NOV table of the block experiment as MS(Err) OneWay [(b 1)MS(lock) + b(t 1)MS(Error)] / (bt 1). (aution: bogus MS(Err) OneWay computed as in section 4.5 below cannot be used to find relative efficiency!) side from the fact that MS(Err) OneWay can only be approximated, there are other reasons that RE can only be used as a rough guide. In our data b = 5 and t = 3. Find RE for these data. How many observations (instead of b = 5) on each of t = 3 ppraisers does your value of RE indicate we would need for a one-way NOV having a "precision" equivalent to our blocked experiment? In R, use the code below to make an NOV table. ompare with the Minitab NOV table. anova(lm(value ~ apprsr + prop)) 5.4. Relationship to One-Way NOV In Unit 4 we saw what can happen if paired structure is ignored in a comparison of two means. For the data in Unit 4, the incorrect independent 2-sample t test failed to find a large effect that was found using the correct paired t procedure. For the appraiser data, we mentioned above that comparing three dotplots of values from the three appraisers would not be effective because this comparison ignores the blocked structure of the experiment. The computational equivalent of this ineffective comparison of dot plots is to analyze the appraiser data using a one-way NOV model instead of the correct block-design.

7 Minitab Notes for STT 6305 lock Design & One Factor NOV Unit 5-7 In this section we look at what happens when the INORRET (one-way NOV with ppraisers as the only factor) is used instead of the block design. INORRET PROEDURE STT > NOV > One-way MT > onew c1 = c2 NLYSIS OF VRINE ON Value SOURE DF SS MS F p pprsr Misleading test ERROR TOTL INDIVIDUL 95 PT I'S FOR MEN SED ON POOLED STDEV LEVEL N MEN STDEV ( * ) ( * ) ( * ) POOLED STDEV = The relatively large P-value for testing the ppraiser effect (about 10%) together with the considerably overlapping confidence intervals for ppraiser means generated by this incorrect procedure might lead a beginner to conclude that there are no significant differences among appraisers. Garbage in (incorrect model) garbage out (incorrect conclusion). Of course, we will not invariably get the wrong conclusion by using the wrong model, it is just that we cannot depend on reaching the correct conclusion. One of the most frequently made mistakes in statistical analysis is to analyze what is really a block design using a one-way NOV. Notice the relationship between this NOV table and the correct one: SS(pp) = 62.8 is the same in both tables. However, SS(Error) 1WY = in the bogus one-way analysis gets split out into two parts in the NOV table for the block design; the same can be said for degrees of freedom: SS(Prp) LOK + SS(Err) LOK = = DF(Prp) LOK + DF(Err) LOK = = 12 ompared with the test for ppraiser in the one-way NOV, the F-ratio F(pr) in the correct block design has a much smaller denominator (3.98). onsequently the F-ratio is a much larger (7.88) and has a smaller P-value (0.013). The block design has "explained" some of the variance in Values by taking blocks into account. Thus the SS(Err) and MS(Err) become relatively small in the block design. The appeal of block designs lies precisely in this potential ability to reduce the fundamental variance estimate MS(Err). Problems Perform an INORRET one-way NOV using Property as the factor. ompare the results with the correct NOV table from the block design Suppose you were trying to analyze data from a block design using the limited capabilities of a spread sheet, in which the one-way NOV is the only NOV procedure available. How could you use output from the spreadsheet to put together the correct analysis?

8 Minitab Notes for STT 6305 lock Design & One Factor NOV Unit Second Dataset Transformation to Stabilize Variances n experiment was conducted to compare the effects of three Temperature levels (Low=20 o, Medium=35 o, and High=50 o ) on the time until failure of a particular kind of electronic component. ecause it is thought that different manufacturing batches of the component may have somewhat different reliabilities, three components are selected at random from each of five atches for test at the three different Temperatures. Results are as shown in the table at left below. Survival times are in months. Temperature atch Low Med High MT > print c2-c4 Data Display Row Months Temp atch To produce the table at right above, we put the 'Months' into c2, 'Temp' subscripts into c3, and 'atch' subscripts into c4. Then we printed these columns in the Session window. We analyzed these data using the appropriate block model, as follows: Made an appropriate scatterplot (labeled with letters or symbols for Temperatures) and formed a preliminary conclusion as to the possible significance of the Temperature effect. Performed the analysis of variance, testing for the significance of both Temperature (main) and atch (block) effects. Neither factor is significant at the 5% level. Stored the residuals from the correct two-factor model and made a normal probability plot of them and also plotted residuals against fitted values. }}} Probability Plot of Residuals Normal - 95% I Residuals Versus the Fitted Values (response is Months) Mean E-16 StDev N 15 D P-Value Percent Residual RESI Fitted Value 12 16

9 Minitab Notes for STT 6305 lock Design & One Factor NOV Unit 5-9 Even though there are not enough data for the formal test of normality to fail, it seems that the points (except perhaps for one) fit a curve better than a line. lso, even though artlett's test for equal variances (not shown) does not fail, the plot of residuals against fitted values seems to show that residuals become increasingly variable as lifetimes increase. From the plots alone there would not be enough evidence to say that the assumptions for NOV are not met, but it is known that lifetimes of this particular kind of component tend to have an exponential distribution rather than a normal distribution. If this is the case for our data, then nonnormality is not the only difficulty. The standard deviation of an exponential distribution is the same as its mean. So if means are different for different Temperatures, then variances are also different and the homoscedasticity assumption of the NOV model is violated. F-tests in an NOV are somewhat robust against nonnormality, but not so robust against heteroscedasticity. One can show that a log transformation "stabilizes" the variances of exponential. One takes the log of each lifetime and runs the appropriate NOV on the transformed data. (It does not matter whether one takes log base e or log base 10.) For data that are suspected to be exponential, it is common practice to run two NOVs, one on the original counts and one on their logs. If the conclusions are the same, one usually reports the results from the NOV on the original data (with a footnote that the transformed data gave the same result). If the interpretations are different and the transformed appear to fit the NOV assumptions better, then the results from the transformed data are usually taken as being correct. (Unfortunately, making sense of multiple comparisons for such transformed data takes a little extra thought.) Here is the NOV for a block design using the transformed data. It shows that the Temperature effect is significant at the 5% level. MT > let c11 = loge(c1) MT > name c11 'LogMo' STT > NOV alanced MT > anova c11 = c2 c3; SU> random c3; SU> restrict; SU> resids c5; SU> ems. NOV: LogMo versus Temp, atch Factor Type Levels Values Temp fixed 3 1, 2, 3 atch random 5 1, 2, 3, 4, 5 nalysis of Variance for LogMo Source DF SS MS F P Temp atch Error Total S = R-Sq = 73.65% R-Sq(adj) = 53.88% Expected Mean Square for Each Term (using Variance Error restricted Source component term model) 1 Temp 3 (3) + 5 Q[1] 2 atch (3) + 3 (2) 3 Error (3)

10 Problems Minitab Notes for STT 6305 lock Design & One Factor NOV Unit Put the component lifetime data into a Minitab worksheet. Then use Minitab's ability to do crosstabulations to obtain a table of the data suitable for presentation to human readers, similar to the first table shown above (left side). Use the command table 'Temp' 'atch'; with subcommand data 'Months'.) Make a labeled scatterplot of the original lifetime data similar to the one for the appraiser data in Section 4.2. Perform an NOV for a block design on the original lifetimes. Make a plot of residuals against fits. Interpret the results The NOV for the block design on the log-transformed data is shown in this section. Perform it for yourself, making a normal probability plot of the residuals. Does this plot seem more nearly linear than the one shown in this section for the original data? Make a plot of residuals against fits and compare it with the one in the previous problem. Finally, did transforming the data change your interpretation of the data (as to significant differences among the Temperatures at the 5% level)? What would you say about your findings in a report intended for nonstatisticians? Perform Fisher's LSD and Tukey's HSD by hand. Use the transformed data. However, notice that a difference in logarithms (as in these multiple comparisons) is the logarithm of a ratio. How can you use this fact to compare lifetimes in months (original data scale) for different Temperatures? For the Fisher procedure use the formula LSD = t*[2ms(error) / b] 1/2, where b is the number of batches (the number of observations averaged to get each of the three Temp-level means), and t* cuts off 2.5% from the upper tail of the t distribution with df(error) = (a 1)(b 1) degrees of freedom. For the Tukey procedure use HSD = q*[ms(error) / b] 1/2, where q* can be obtained from tables of the studentized range or with the R function qtukey(.95, a, (a 1)*(b - 1)). This is for 95% confidence in the pattern of differences shown in the three Is. Why might you want to look at the HSD for 90% confidence in this particular case? Verify your results for Tukey's HSD by using Minitab's glm procedure using the menu path STT > NOV > GLM or the commands below. MT > GLM 'LogMo' = Temp atch; SU> Random 'atch'; SU> Pairwise Temp; SU> Tukey. In the Minitab's version of HSD, you have the choice of output with Is or tests (or both). Each I has total length 2HSD, centered at the appropriate difference in Temp-level means. (The Minitab 14 GLM procedure does not find Fisher LSD Is. However, notice that the quantity labeled 'SE of Difference' in the output for the Tukey HSD tests is [2MS(Error) / b] 1/2, not [MS(Error) / b] 1/2.) Minitab Notes for Statistics 6305: NOV Models by ruce E. Trumbo, Department of Statistics, SU East ay, East ay, opyright 1991, 2011 by ruce E. Trumbo. ll rights reserved. Partial support for the 1991 version from NSF grant USE The current version with Minitab professional graphics and additional examples is a draft. For comments, errata, selected answers, related materials, and permission to use beyond SU East ay please bruce.trumbo@csueastbay.edu or eric.suess@csueastbay.edu.