STATISTICS 368 AN EXPERIMENT IN AIRCRAFT PRODUCTION Christopher Wiens & Douglas Wiens

STATISTICS 368 AN EXPERIMENT IN AIRCRAFT PRODUCTION Christopher Wiens & Douglas Wiens April 21, 2005 The progress of science.

1. Preliminary description of experiment We set out to determine the factors which affect the flight of a paper airplane. Possible factors of interest are shape, paper type and whether or not weights are added for stability. Not having pursued such an investigation for some years, I (D. Wiens) thought it best to seek expert advice (C. Wiens) and to refresh myself on flight techniques. Thus, I made several airplanes out of two types of paper and in two shapes, and tried flying them in the hallway outside my office. I quickly recalled that a nuisance factorforwhichitisdifficult to control is throwing strength - if one throws the plane too hard, the initial rush of air at the tip tends to cause the craft to pursue a very erratic flight path. In my practice throws I tried to re-learn the art of throwing with just the right strength to give a fairly smooth and straight flight. However, since we lacked the resources to measure and control this factor, we had to hope that randomization and replication would look after its effect. Theresponsevariablewewereinterestedinwas distanceflown. By distance we mean only that component in the direction of flight, regardless of any zigs and zags taken along the way. Formally, if the co-ordinates of the thrower s initial position were (0, 0) and those of the landing position of the plane were (x, y) (x sideways to the right and y forward), then the response would be y. The floor of my hallway is covered by tiles (31 cm. 31 cm.), and so y was measured by counting the number of tiles (the units ), in the direction of the hallway, traversed by the flight path. We decided that the end of the path would be the resting point of the tip of the plane. The measurements were rounded to the nearest half-unit. The two shapes I initially had in mind were short and stubby (SS) and long and thin (LT), with folded wings. My partner/consultant/coauthor proposed a third shape, which we refer to as sleek (SL) and subsequently incorporated into the study. See Figure 1. The two paper types were light (L) - ordinary paper such as is used in a printer - and heavy (H) - cut from an envelope such as is used to hold moderately sized manuscripts. In order to control for the possible effect of the dimensions of the paper, which would almost certainly interact with shape, we trimmed the envelope paper to the same size as the printer paper. The third possibly important factor is whether or not weights (in the form of paper clips) are added to the undercarriage of the aircraft, to (one might hope) stabilize its flight. The levels of this factor are referred to as weighted (W) and unweighted ( W). 2. Design and data collection We initially considered running the experiment as 2 replicates of a 3 2 factorial (3 shapes, 2 paper types) in two blocks (weighted and unweighted). In this scheme we would construct 12 aircraft and use each one twice - once in each block. However, although this would require fewer resources, we might not be able to make valid inferences about the effect of weighting (the block effects) unless the normality was not in doubt - this is as a result of the randomization restriction. As well, block/shape and block/paper type interactions cannot be fitted for such a design. Thus, we decided to run a 3 2 2 factorial experiment, with n = 2 replicates in each of the 12 cells. We first collected 12 sheets of each type of paper. Within each type, 4 airplanes of 2

Figure 1: Aircraft prototypes. Left: Short and stubby, unweighted, heavy paper (i, j, k = 1, 2, 2). Middle: Sleek, unweighted, light paper (i, j, k = 3, 2, 1). Right: Long and thin, weighted, light paper (i, j, k =2, 1, 1). each of the 3 shapes were made. Within each group of 4, two were weighted and two left unweighted. The order in which the 24 planes were manufactured was randomized, in order to average out any learning effect on the part of the manufacturer (C. Wiens). The R command > sample(1 : 24) yielded the random arrangement 3 14 4 6 22 11 20 1 10 16 17 15 18 12 7 2 24 19 8 21 23 5 13 9 of the numbers 1,..., 24; thus the planes were manufactured in the order indicated in Table 1. The order in which the 24 runs were to be carried out was also randomized, leading to the throwing order in Table 2. Each pair of replicates was also randomized. For each of the 12 cells, the two planes were numbered 1 and 2. When the firstmemberofthatcellwasabouttobeflown one of these numbers would be randomly chosen (sample(1:2,1)) in order to determine which of the two 3

Table 1. Manufacturing order for aircraft Light paper Heavy paper Weighted Unweighted Weighted Unweighted Short, stubby 3 6 20 16 18 2 8 5 Long, thin 14 22 1 17 12 24 21 13 Sleek 4 11 10 15 7 19 23 9 Table 2. Randomized flying order Light paper Heavy paper Weighted Unweighted Weighted Unweighted Short, stubby 22 5 10 2 7 20 13 21 Long, thin 6 19 8 12 17 24 1 23 Sleek 4 11 15 18 16 9 14 3 planes would be flown. The 12 numbers so obtained were h1, 2, 1, 2, 2, 2, 1, 2, 2, 2, 2, 1i. Thus for instance the first plane flown came from the (LT,H, W) cell; within that cell the plane numbered 1 was flown. This was either the 21 st or the 13 th plane manufactured. 2.1. Statistical model where: The effects model for the data is y ijkl = µ + τ i + β j + γ k +(τβ) ij +(τγ) ik +(βγ) jk +(τβγ) ijk + ε ijkl, (1) 1. τ i is the effect of the shape for i = 1 (SS), 2 (LT) and 3 (SL), 2. β j is the effect of weighting (j =1forW,j =2for W), 3. γ k is the effect of paper type (k =1forL,k =2forH), 4. (τβ) ij,(τγ) ik,(βγ) jk,and(τβγ) ijk are the various interaction effects and 5. ε ijkl is the random error for the l th run (l =1, 2) in cell (i, j, k). Of course µ is the overall mean flight distance irrespective of any of the other effects. Since any average effect can be incorporated into µ we can impose the usual constraints that any effect vanishes when summed over any of its indices. 4

Table 3. Data: Flight distances 1 Light paper Heavy paper Weighted Unweighted Weighted Unweighted Short, stubby 6 6 5.5 4 7 10.5 8 3 Long, thin 5.5 7.5 7 3.5 7 6.5 5.5 3 Sleek 18.5 23 15.5 21.5 25 28.5 20.5 14.5 1 In units of 31 cm., rounded to the nearest half. Table 4. Cell averages Light paper Heavy paper Weighted Unweighted Weighted Unweighted Short, stubby 6 4.75 8.75 5.5 Long, thin 6.5 5.25 6.75 4.25 Sleek 20.75 18.5 26.75 17.5 2.2. Data The data (distances flown) are given in Table 3. One unanticipated hazard arose in the course of gathering the data - on one flight the plane veered into the hollow of a doorway and struck the frame, thus ending its flight prematurely. It was decided to repeat this run. The pilot in all cases was Christopher Wiens. 3.1. Preliminaries 3. Data analysis The data as entered into R are in the Appendix. The manufacturing order and flying order were entered as well. The least squares estimates of the main effects (in model (1)) are ˆµ =ȳ... =10.94, ˆβL =ȳ.1. ȳ... =.65, ˆβ H =ȳ.2. ȳ... =.65, ˆτ SS =ȳ 1.. ȳ... = 4.69, ˆτ LT =ȳ 2.. ȳ... = 5.25, ˆγ W =ȳ..1 ȳ... =1.65, ˆτ SL =ȳ 3.. ȳ... =9.94, ˆγ W =ȳ..2 ȳ... = 1.65. The boxplots by level of each factor - see Figure 2 - indicate that we might see significant shape effects, with SL being superior. The interaction plots in Figure 3 point to a possible Paper Weighted interaction, with the addition of weights having a positive effect on the mean distance flown when using heavy paper, but not otherwise. 5

mean of y 9 10 11 12 13 14 mean of y 9 10 11 12 13 14 mean of y mean of y mean of y mean of y 25 25 25 LT SL SS H L ~W W Boxplots for each shape Boxplots for each paper type Boxplots for 'weighted' Figure 2: Boxplots of flight data by factor and level. paper H L shape SL LT SS LT SL SS H L shape paper weighted W ~W shape SL SS LT LT SL SS shape ~W W weighted weighted paper W ~W H L H L ~W W paper weighted Figure 3: Interaction plots for flight data. 6

Sample Quantiles -3-2 -1 0 1 2 3 3.2. Analysis of Variance and Residual Analysis Normal Q-Q Plot -2-1 0 1 2 Theoretical Quantiles Figure 4: Normality plot for residuals from the 3 factor fit. An ANOVA was run; the R output is given below. From this we conclude that none of the interactions is significant at even the 10% level, although Paper Weighted is the closest to being significant. Both Shape and Paper type have significant main effects (p =8 10 8 and.01 respectively). > g <- lm(y ~shape + paper + weighted + shape*paper + + shape*weighted + paper*weighted + shape*paper*weighted) > anova(g) Analysis of Variance Table Response: y Df Sum Sq Mean Sq F value Pr(>F) shape 2 1186.31 593.16 85.3718 8.017e-08 *** paper 1 10.01 10.01 1.4408 0.25317 weighted 1 65.01 65.01 9.3568 0.00992 ** shape:paper 2 8.90 4.45 0.6402 0.54429 shape:weighted 2 18.27 9.14 1.3148 0.30456 paper:weighted 1 17.51 17.51 2.5202 0.13838 shape:paper:weighted 2 9.77 4.89 0.7031 0.51432 Residuals 12 83.38 6.95 --- Signif. codes: 0 *** 0.001 ** 0.01 * 0.05. 0.1 1 7

residuals residuals residuals residuals residuals residuals Before acting on these statements we should check the assumptions of our model, namely that the random errors are independent and normally distributed with equal variances. For the normality see Figure 4, where the sample quantiles are plotted against the Normal quantiles. The straightness of the plot indicates that we need have no serious concerns about the normality. 1.0 1.5 2.0 2.5 3.0 1.0 1.2 1.4 1.6 1.8 2.0 residuals vs shape residuals vs paper type 1.0 1.2 1.4 1.6 1.8 2.0 25 residuals vs weighting residuals vs fitted values residuals vs manufacturing order residuals vs flying order Figure 5: Residuals plotted against factors and orders. In Figure 5 the residuals are plotted against the levels of each factor, and against the manufacturing and flying orders. Trends in the any of these, or major differences in the spread of the residuals between the levels of the factors, could indicate model discrepancies. 8

Fortunately, no such trends are apparent. We were a bit concerned about possibly greater variances in the LT shape group, but these are not significant: > bartlett.test(residuals, shape) Bartlett test for homogeneity of variances data: residuals and shape Bartlett s K-squared = 3.9729, df = 2, p-value = 0.1372 3.3. Inferences, conclusions, recommendations Our analysis so far has led to the inference that the shape of the aircraft has a significant effect on mean flying distance, as does the presence or absence of weights. What are the levels of these factors which result in the greatest means? To answer this question we applied Tukey s procedure to rank the means. Consider first the Shape means, denoted µ SS,µ LT and µ SL. TheirLSEsaretheaverages of the corresponding rows of Table 3 (or Table 4); in increasing order they are ȳ LT =5.69 < ȳ SS =6.25 < ȳ SL =20.88. Tukey s procedure declares the differences in the means to be significant if the differences in the corresponding LSEs exceeds µ q α 1 smse 2 8 + 1 = 8 q α 2 6.95 =.9321qα. 2 With α =.05, the value of q α to compare 3 means, with df (SS E ) = 12, is q.05,3,12 =3.773, whence we look for differences exceeding.9321 3.773 = 3.52. On this basis we declare that µ SL is significantly larger than both µ SS and µ LT, but that these two are not significantly different from each other. For the factor Weighted the LSEs are ȳ W =9.29 < ȳ W =12.58, and the difference is declared significant (α =.05) since it exceeds µ q.05,2,12 1 smse 2 12 12 + 1 =2.34. There being no significant Shape Weighted interactions, we conclude that the optimum levels are a Sleek shape, with weights added. 9

Given this decision, does the paper type have a significant effect? In other words, is there a significant difference between µ SL,W,L and µ SL,W,H? Their LSEs are, from Table 4, and their difference is compared to ȳ SL,W,L =20.75 < ȳ SL,W,H =26.75, µ q.05,2,12 1 smse 2 2 2 + 1 =5.74. Again the difference is significant (barely). From this we conclude that the best aircraft design is a sleek shape, made from heavy paper, with weights added. What if one has no paper clips? Would this alter one s preference for heavy paper? (Recall that the Paper Weighted interaction is almost significant.) We are now comparing the 6 cell means µ i,j, W on the basis of their LSEs in the Unweighted columns of Table 4. The benchmark for these difference is now µ q.05,6,12 1 smse 2 2 2 + 1 =8.86. By this we conclude that both µ SL,L, W and µ SL,H, W are significantly larger than the other ( W ) means, but cannot be differentiated from each other at this level of significance. Summary Our recommendation for aircraft designers is to use a Sleek shape. If paper clips are available then use them, and use heavy paper. If no paper clips are available then either light or heavy paper may be used. 10

The data as entered into R were: Appendix y shape paper weighted order.mfg order.fly 1 6.0 SS L W 3 22 2 6.0 SS L W 6 5 3 5.5 SS L ~W 20 10 4 4.0 SS L ~W 16 2 5 7.0 SS H W 18 7 6 10.5 SS H W 2 20 7 8.0 SS H ~W 8 13 8 3.0 SS H ~W 5 21 9 5.5 LT L W 14 6 10 7.5 LT L W 22 19 11 7.0 LT L ~W 1 8 12 3.5 LT L ~W 17 12 13 7.0 LT H W 12 17 14 6.5 LT H W 24 24 15 5.5 LT H ~W 21 1 16 3.0 LT H ~W 13 23 17 18.5 SL L W 4 4 18 23.0 SL L W 11 11 19 15.5 SL L ~W 10 15 20 21.5 SL L ~W 15 18 21 25.0 SL H W 7 16 22 28.5 SL H W 19 9 23 20.5 SL H ~W 23 14 24 14.5 SL H ~W 9 3 11