SOME APPLICATIONS OF STATISTICAL DESIGN OF EXPERIMENT METHODOLOGY IN CIVIL ENGINEERING

Congrès annuel de la Société canadienne de génie civil Annual Conference of the Canadian Society for Civil Engineering Moncton, Nouveau-Brunswick, Canada 4-7 juin 2003 / June 4-7, 2003 SOME APPLICATIONS OF STATISTICAL DESIGN OF EXPERIMENT METHODOLOGY IN CIVIL ENGINEERING L. M. Lye A A Faculty of Engineering and Applied Science, Memorial University, St. John s, NL, A1B 3X5, Canada. ABSTRACT: The application of statistical design of experiment or DOE methodology has found great success especially in the manufacturing and chemical industries. In civil engineering, its use is limited to certain areas of environmental engineering. The reason for the limited use is partly because very few civil engineers have heard of it or is aware of the wide applications of the methodology. The other is that the methodology is statistically based and most references on the topic are described in advanced statistical texts or in other texts outside of civil engineering. This is unfortunate because the methodology has many useful applications in civil engineering. This paper shows the applications of DOE methodology using two examples from the field of civil engineering. It is hoped that this paper will help promote the use of DOE methodology in general and will stimulate further applications in civil engineering. 1. INTRODUCTION The application of statistical design of experiment or DOE methodology has found great success in many industries especially in the manufacturing and chemical industries. In civil engineering, its use is largely confined to certain areas of environmental engineering. The reason for the limited use is partly because very few civil engineers have heard of it or is aware of the wide applications of the methodology. It is a subject that is unfortunately not part of the civil engineering curriculum of any Canadian university. The other reason is that the methodology is statistically based and most references on the topic are described in advanced statistical texts or in other texts outside of civil engineering. This is unfortunate because the methodology has many useful applications in civil engineering besides environmental engineering. At the most basic level, the statistical knowledge required to use the method has already been covered in the statistic course required by all undergraduate engineers in Canada. In addition, there are several excellent software packages available in the market that can help with the calculations required. This paper shows the use of statistical DOE methodology in two different areas of civil engineering and provides several references that will illustrate the wide applications of the methodology. The first example is in the area of environmental engineering, where the reaeration rate in a model stream channel is studied as a function of stream velocity, stream depth, and channel roughness. The second example is in the area of geotechnical engineering. In this application, the well-known Terzaghi s bearing capacity equation for a circular footing was reanalyzed and simplified. The software Design-Expert by Statease, Inc., an easy to use, stand-alone software for design of experiments, was used for all the examples in this paper. In the next section, the basic idea of DOE and what it can achieve will be described. This is followed by the application of two different DOE methodologies for the two different examples as described above. GCD-115-1

The first example shows the use of a simple two-level factorial design for a three factor problem, and the second example shows the use of a more advanced fractional factorial response surface design for a five factor problem. The paper ends with some recommendations and conclusions. 2. STATISTICAL DESIGN OF EXPERIMENT (DOE) Civil engineers carry out a fair amount of experimentation in laboratories and in design offices in areas of structural engineering, hydraulic engineering, geotechnical engineering, environmental engineering, etc. We carry out experiments to 1) evaluate and compare basic design configurations, 2) evaluate material alternatives, 3) select design parameters so that the design will work well under a wide variety of field conditions (robust design), and 4) determine the key design parameters that impact performance. The experiments may be computer simulations, laboratory or field experiments. As with most engineering problems, we are often faced with limited time and budget. Hence we would like to gain as much information as possible and do so as efficiently as possible. How would one proceed to conduct such an experiment? In engineering, one often-used approach is the best-guess (with engineering judgment) approach. Another strategy of experimentation that is prevalent in practice is the one-factor-at-a-time or OFAT approach. The OFAT method was once considered as the standard, systematic, and accepted method of scientific experimentation. Both of these methods have been shown to be inefficient and in fact can be disastrous (Lye, 2002; Montgomery, 2001). These methods of experimentation became outdated in the early 1920s when Ronald A. Fisher discovered much more efficient methods of experimentation based on factorial designs. These were further developed to include fractional factorial designs, orthogonal arrays, and response surface methodology. These statistical methods are now simply called design of experiment methods or DOE methods. Basically, DOE is a methodology for systematically applying statistics to experimentation. DOE lets experimenters develop a mathematical model that predicts how input variables interact to create output variables or responses in a process or system. DOE can be used for a wide range of experiments for various purposes including nearly all fields of engineering and science and even in marketing studies. The use of statistics is important in DOE but not absolutely necessary. In general, by using DOE, we can: learn about the process we are investigating; screen important factors; determine whether factors interact; build a mathematical model for prediction; and optimize the response(s) if required. DOE methods are also very useful as a strategy for building mechanistic models, and they have the additional advantage that no complicated calculations are needed to analyze the data produced from the designed experiment. It has been recognized for many years that the factorial-based DOE is the correct and the most efficient method of doing multi-factored experiments; they allow a large number of factors to be investigated in few experimental runs. The efficiency stems from using settings of the independent factors that are completely uncorrelated with each other. That is, the experimental designs are orthogonal. The consequence of the orthogonal design is that the main effect of each experiment factor, and also the interactions between factors, can be estimated independent of the other effects (Berthouex and Brown, 2002). In the next section, the two-level full factorial design or the 2 k design will be discussed. The two levels are normally referred to as a low and a high level. The levels may be qualitative (e.g. Smooth vs. Rough) or quantitative (e.g. 10 cm vs. 15 cm). The two-level, three-factor or in short 2 3 factorial design will be used to study the effect of three factors on the reaeration rate of a model stream. GCD-115-2

3. EXAMPLE OF A 2 3 FACTORIAL DESIGN - REAERATION RATE OF A STREAM Consider the problem of an experiment that attempted to relate the rate of dissolution of an organic chemical to the reaeration rate (Y) in a laboratory model stream channel (Berthouex and Brown, 2002). The three experimental factors are A = stream velocity (V, in m/s), B = stream depth (D, in cm), and C = channel roughness (R, a qualitative term). The test factors for the reaeration experiment and the response of interest are shown in Table 1. Table 1: Test factors and response for the reaeration of a stream experiment Factor Name Low Level (-1) High Level (+1) A Velocity (V) 0.25 m/s 0.5 m/s B Depth (D) 10 cm 15 cm C Roughness (R) Smooth Coarse Response: Reaeration rate (Y) For any 2 k experiment, all combinations of the k factors must be considered. With three factors, there will be eight combinations of the low and high values. The treatment combinations can be conveniently laid out in the form of a sign table, as shown in Table 2. Table 2: Treatment combinations Run Combination A B C Description of combinations Y=reaeration rate 1 (1) -1-1 -1 V=0.25, D=10, R=smooth 107 117 117 2 a +1-1 -1 V=0.5, D=10, R=smooth 190 178 179 3 b -1 +1-1 V=0.25, D=15, R=smooth 119 116 133 4 ab +1 +1-1 V=0.5, D=15, R=smooth 188 191 195 5 c -1-1 +1 V=0.25, D=10, R=coarse 119 132 126 6 ac +1-1 +1 V=0.5, D=10, R=coarse 187 173 166 7 bc -1 +1 +1 V=0.25, D=15, R=coarse 140 133 132 8 abc +1 +1 +1 V=0.5, D=15, R=coarse 164 145 144 In Table 2, under the heading of Combination, is a shorthand way of writing down the various combinations. The (1) means that all factors are at the low level, a means that only factor A is at the high level, all other factors at the low level, b means that only factor B is the high level and all other factors at the low level, and abc means that all three factors are at the high level. The +1 and -1 under the headings A, B, and C also indicate the level of factor A, B, and C for each combination. The +1 means a high level and the -1 means a low level. Using the +1 and -1 to indicate the combinations is the preferred method in most software for design of experiments. The layout shown in Table 2 is in Yates standard order. During the experiment, the treatment combinations should be run in random order (Montgomery, 2001). The next step is to carry out the experiment and obtain the response for each treatment combination. For the above experiment, the response of interest is the reaeration rate of the stream over a period of time. Each treatment combination was run in triplicates. The results are also shown in Table 2. For a 2 k design, 2 k 1 effects can be estimated. That is, with k=3 factors, we can estimate from the 2 3 experiments, 7 effects: the main effects A, B, and C, the two factor interactions AB, AC, and BC, and one three factor interaction ABC. By running the experiment with replications (three here), it allows the experimenter to test for statistical significance of all 7 effects and it allows for the estimate of pure error GCD-115-3

from the experiment. If no replication is used, then to test for significance, some of the higher interaction terms have to be used as the error term in the ANOVA. This is sometimes acceptable by virtue of the sparsity of effects principle with larger number of factors where higher interaction effects are negligible or can safely be assumed to be zero. DESIGN-EXPERT Plot Reaeration rate Normal plot A: V B: D C: R Normal % probability 99 95 90 80 70 50 30 20 10 5 B C BC ABC AB A 1 AC -17.92-0.75 16.42 33.58 50.75 Effect Figure 1: Normal probability plot of estimated effects of the reaeration rate experiment. Table 3: Analysis of variance table for the reaeration rate experiment. Source Sum Squares df Mean Square F-value p-value Significant at 5% A 15453.37 1 15453.37 260.27 <0.0001 Yes B 3.38 1 3.38 0.057 0.8146 No C 198.38 1 198.38 3.34 0.0863 No AB 425.04 1 425.04 7.16 0.0166 Yes AC 1926.04 1 1926.04 32.44 <0.0001 Yes BC 408.37 1 408.37 6.88 0.0185 Yes ABC 425.04 1 425.04 7.16 0.0166 Yes Pure Error 950.00 16 59.37 Corr. Total 19789.63 23 If the effects are random, they might be expected to be normally distributed and hence will plot along a straight line on a normal probability plot. Non-random effects will fall off the line and are considered to be statistically significant. Figure 1 shows the normal probability plot of the effects from the reaeration rate GCD-115-4

experiment. Pure error terms are indicated by the triangles. It can be seen that effect A (velocity) is well off the line indicating that it is highly significant. Effect B (depth) is on the line and hence not significant on its own. All two and three factor interactions are significant which implies that all factors are significant. This is confirmed by the ANOVA shown in Table 3. The mechanics of effects calculations and ANOVA are given in Montgomery (2001) among others. The prediction equations in terms of coded and actual factors are given by equations [1] and [2], respectively. Final equation in terms of coded factors: [1] Reaeration rate Y = 149.63 +25.37 A + 0.37 B - 2.88 C - 4.21 AB - 8.96 AC - 4.12 BC -4.21 ABC Final equations in terms of actual factors: [2a] Reaeration rate Y = 27.00 + 274.67 V +1.80 D for R = Smooth [2b] Reaeration rate Y = -10.00 + 468.00 V + 8.60 D - 26.93 VD for R = Coarse The prediction equation developed fulfilled all the assumptions of regression with a R 2 of 0.952, and a prediction R 2 of 0.892. It can be seen that how the velocity and depth interacts depends on whether the channel roughness is smooth or coarse, a three factor interaction. This interaction effect can be displayed and interpreted with an interaction graph shown in Figure 2a for the case when the roughness (R) is smooth and Figure 2b for the case when the roughness (R) is coarse. DESIGN-EXPERT Plot Reaeration rate X = A: V Y = B: D 198.002 Interaction Graph B: D Design Points B- 10.000 B+ 15.000 Actual Factor C: R = Smooth Reaeration rate 175.251 152.5 129.749 2 106.998 0.25 0.31 0.38 0.44 0.50 A: V Figure 2a: Interaction graph of velocity (V) and depth (D) when roughess (R) is smooth. When the channel roughness is smooth, there is in fact no interaction between the velocity and depth. The two lines are parallel. The rate of increase in aeration rate as velocity goes from 0.25 m/s to 0.50 m/s is the same regardless of whether the depth is at 10 cm or 15 cm. The reaeration rate is higher with a larger depth, but the rate of change is the same. On the other hand, when the channel roughness is coarse, the interaction plot seen in Figure 2b, shows a totally different picture. GCD-115-5

DESIGN-EXPERT Plot Reaeration rate X = A: V Y = B: D Design Points 195 173 Interaction Graph B: D B- 10.000 B+ 15.000 Actual Factor C: R = Coarse Reaeration rate 151 129 2 107 0.25 0.31 0.38 0.44 0.50 A: V Figure 2b: Interaction graph of velocity (V) and depth (D) when roughess (R) is coarse. In Figure 2b, for the case when the channel roughness is coarse, the two lines are no longer parallel indicating an interaction effect. At the low level of velocity, the reaeration rate is higher with greater depth, but as the velocity increases to the high level, the reaeration rate is lower at the greater depth than the lower depth. These interaction effects would never be discovered if one had used a one-factor-at-a-time (OFAT) experiment. 4. TERZAGHI S BEARING CAPACITY EQUATION In this next example, DOE methodology is used to develop a simple replacement equation for Terzaghi s ultimate bearing capacity equation (Terzaghi, 1943). The purpose of using DOE methodology here are twofold: 1) to provide a better understanding of the contribution of each of the independent factors used in Terzaghi s equation and their interaction with each other; and 2) to provide a replacement equation that is easier to use. The case of shallow circular footings will be considered here. The general ultimate bearing capacity equation proposed by Terzaghi (1943) is given by: [3] q ult = c N c S c + q N q + ½ γ B N γ S γ For circular footings, S c = 1.3, and S γ = 0.6, and [3] becomes: [4] q ult = 1.3 c N c + γ D N q + 0.3 γ B N γ where: q ult is the ultimate bearing capacity (kpa), c is the cohesion of the soil (kpa), q is the surcharge load = γd (kpa), γ is the specific weight of the soil (KN/m 3 ), D is the depth of the footing (m), B is the diameter of the footing (m), S c and S γ are shape factors, and N c, N q, and N γ are bearing capacity factors, each of which is expressed as a function of the soil internal friction angle (φ). These bearing capacity factors are usually tabulated for different values of φ or given in graphical form in most text books. GCD-115-6

Because the soil friction angle is a hidden variable within Terzaghi s equation, it is not very obvious how this factor contributes to the ultimate bearing capacity or whether it interacts with the other factors in equation [4]. In other words, of the five factors involved (c, γ, D, B, and φ), which factor has the greatest effect on the ultimate bearing capacity, and is there any interaction among the five factors? To reduce the number of experimental runs to be carried out, and to take into account possible quadratic or curvature effects, a one half fraction of a 2 5 experiment combined with a face centered central composite response surface design will be used for this five factor problem. Half of a 2 5 factorial experiment requires only 16 runs and the central composite design requires another 10 axial and 1 center points. Hence in total, 27 (2 k-1 + 2k + 1) runs are required to fit a second order model without losing too much information. The half fraction used is a resolution V design meaning that main effects are confounded with four factor interactions and two factor interactions are confounded with three factor interactions. If we invoke the sparsity of effects principle, three or higher order interactions can be assumed to be zero and hence main and two factor interaction effects can be estimated cleanly. Further details on fractional factorials and response surface designs are given in Montgomery (2001) and Box et al (1978) among others. Each factor is set at three levels in a predefined manner for a face centered response surface design. The factors and their ranges of interest are shown in Table 4. The 27 experimental runs (applying Terzaghi s equation for each combination of factors) are given in Table 5. Table 4: Factors and ranges for Terzaghi s bearing capacity equation Factor Units Name Low Mid High c kpa Cohesion 10 15 20 φ degrees Soil friction angle 20 30 40 γ kn/m 3 Specific weight of soil 16 18 20 D m Depth of footing 0.5 1.0 1.5 B m Diameter of footing 1 2 3 Response = ultimate bearing capacity, q ult in kpa. From the combination of factors and responses in Table 5, a second-order polynomial model in the form of equation [5] can be fitted. k k 2 [5] Y= β0 + βixi + βiixi + βijxix j i= 1 i= 1 i< j In DOE, it is customary to use a coded factor scale to develop the required regression model. That is, the lower limits of the factors are coded as (-1), and the upper limits as (+1), as described in the example of section 2. The model can be expressed in actual scales using a simple transformation. Using a coded factor scale makes the interpretation of the regression coefficients easier because they are now of the same scale. Using a stepwise regression procedure, the model shown in equation [6] in coded factor scale was obtained. All regression coefficients are significant at the 0.0001 level and all assumptions of regression were fulfilled after a square-root transform of the response variable. [6] qult 2 = 36.58 + 3.60c + 21.52φ + 7.83φ + 1.27γ + 3.45D + 2.28B + 1.10c φ + 0.87φ γ + 2.06φ D + 1.77φ B where, the prime ( ) indicates that the factors are in coded factor scales. GCD-115-7

Table 5: Run combinations and responses using Terzaghi s equation Run c φ γ D B q ult (kpa) 1 10 20 16 0.5 3 361.3 2 20 20 16 0.5 1 543.4 3 10 40 16 0.5 1 2376.4 4 20 40 16 0.5 3 4584.4 5 10 20 20 0.5 1 334.1 6 20 20 20 0.5 3 624.2 7 10 40 20 0.5 3 3864.3 8 20 40 20 0.5 1 3903.6 9 10 20 16 1.5 1 431.7 10 20 20 16 1.5 3 709.8 11 10 40 16 1.5 3 4641.1 12 20 40 16 1.5 1 4921.3 13 10 20 20 1.5 3 542.1 14 20 20 20 1.5 1 712.2 15 10 40 20 1.5 1 4285.5 16 20 40 20 1.5 3 6734.4 17 10 30 18 1.0 2 1101.4 18 20 30 18 1.0 2 1584.9 19 15 20 18 1.0 2 532.4 20 15 40 18 1.0 2 4413.9 21 15 30 16 1.0 2 1274.5 22 15 30 20 1.0 2 1411.8 23 15 30 18 0.5 2 1140.7 24 15 30 18 0.5 2 1545.7 25 15 30 18 1.0 1 1236.8 26 15 30 18 1.0 3 1449.5 27 15 30 18 1.0 2 1343.2 The final equation in terms of actual factors is given by: [7] qult 2 = 65.232 + 0.059c 4.425φ + 0.0783φ 0.674γ 5.442D 3.040B + 0.022cφ + 0.044φγ + 0.411φD + 0.177φB The R 2 of the fitted equation is 0.9993 and the predicted R 2 is 0.9975 showing an excellent fit to the data. Hence equation [6] or [7] can be used in place of Terzaghi s equation for the ranges of the factors considered. What is more important is that the relative contribution of each of the factors and their interactions on the ultimate bearing capacity can be seen from the magnitudes of the regression coefficients in equation [6]. The most important factor contributing to the ultimate bearing capacity is the soil friction angle, φ, which has second order effects or curvature and it interacts with every other factor. The next most important factor is the cohesion of the soil, c, followed by depth of the footing, D, then the diameter of the footing, B. The two-factor interactions of φ and D, and φ and B follow after B. The soil specific weight, γ, and the interactions between φ and c, and φ and γ while statistically significant, have the least contribution to the ultimate bearing capacity. Of the two factor interactions, the one with the greatest GCD-115-8

magnitude is the interaction between φ and D. This effect can be seen on the interaction graph of Figure 3. DESIGN-EXPERT Plot Sqrt(qult) X = B: Phi Y = E: Diameter Design Points 82.063 66.117 Interaction Graph E: Diameter E- 1.000 E+ 3.000 Actual Factors A: c = 15.00 C: Gamma = 18.00 D: Depth = 1.00 Sqrt(qult) 50.171 34.225 18.278 20.00 25.00 30.00 35.00 40.00 B: Phi Figure 3: Interaction graph of soil friction angle (φ) and diameter of footing (D). Figure 3 shows the curvature effect of the soil friction angle, φ, and that at the low level of φ, the bearing capacity is about the same regardless of the diameter of the footing. However, as the friction angle φ increases to the high level, the bearing capacity increases more rapidly for a larger footing than a smaller footing. Other interaction effects can be similarly explained using the interaction graphs. 5. CONCLUSIONS This paper has shown the use of two different statistical DOE methodologies in two diverse areas of civil engineering. Other examples of applications in civil engineering are given in Lye (2002). In one example, DOE methodology was used to simplify Penman s equation for estimating evaporation from open water. Penman s method involved the use of a series of more than ten equations and a table to estimate evaporation for a given month. By using DOE methodology, Penman s equation can be simplified to one equation with little loss of accuracy. Another application of DOE methodology is in the area of numerical geotechnical analysis Zanganeh et al (2002). DOE methodology was applied to Newmark s displacement analysis of submarine slopes under earthquake loads. A replacement model was obtained and was found easier to use and explain, and was faster to run when used in a Monte Carlo simulation study. An application in the area of structural engineering is given in Ebead et al (2002). DOE methodology was used to develop simple replacement models for the complex and time consuming finite element model to explain and predict the ultimate load carrying capacity of concrete slabs strengthened with GFRP and CFRP materials. Examples from environmental engineering can be found in Berthouex and Brown (2002) and the classic book by Box et al (1978). It is hoped that this paper has convinced the reader that statistical design of experiment methodology is a very useful tool for the civil engineer to have. The method can be used for a wide variety of applications GCD-115-9

and the statistical knowledge required to use the methods have already been covered in the undergraduate engineering program. Furthermore, the availability of easy to use software for design of experiments helps make the tedious computations a lot less onerous. However, it is important that one must not be carried away by the elegance of the methodology. The results and equations obtained must be tempered with engineering judgement and interpreted correctly. The extraordinary truth is, that about 80 years after Fisher invented modern experimental design, it is still not widely taught in schools of engineering and of science in our universities. In science and engineering as in all other endeavors, it is difficult to change accepted practice, even when other methods are shown to be better. It is highly recommended that civil engineers be encouraged to learn and use the factorialbased DOE and abandon the trial and error and OFAT approaches. Surely an engineer who does not know how to run an efficient experiment is not a very good engineer. Fisher once said about poorly designed experiments: nothing much can be gained from statistical analysis; about all you can do is to carry out a postmortem and decide what such an experiment died of. 6. REFERENCES Berthouex, P. M. and L. C. Brown (2002): Statistics for Environmental Engineers, 2 nd Publishers. Edition, Lewis Box, G. E. P., W. G. Hunter, and J. S. Hunter (1978): Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building, NY, Wiley Interscience. Ebead, U., H. Marzouk, and L. M. Lye (2002): Strengthening of Two-Way Slabs Using FRP Materials: A Simplified Analysis Based on Response Surface Methodology, Proceedings 2 nd World Engineering Congress, Kuching, Malaysia. L. M. Lye (2002): Design of Experiments in Civil Engineering: Are we still in the 1920s?, Proceedings of the 30 th Annual Conference of the Canadian Society for Civil Engineering, Montreal, Quebec, June. Montgomery, D. G. (2001): Design and Analysis of Experiments, 5 th Edition, John Wiley and Sons, Inc. Terzaghi, K. (1943): Theoretical Soil Mechanics, John Wiley and Sons Ltd. Zangeneh, N, A. Azizian, L. M. Lye, and R. Popescu (2002): Application of Response Surface Methodology in Numerical Geotechnical Analysis, Proceedings 55 th Canadian Geotechnical Conference, Niagara Falls, Canada. GCD-115-10