Training for Shainin s approach to experimental design using a catapult

Transcription

1 for Shainin s Jiju Antony Warwick Manufacturing Group, School of Engineering, University of Warwick, Coventry, UK Warwick Manufacturing Group, School of Engineering, University of Warwick, Coventry, UK Keywords Variables search method,, Experimental design Abstract In the quest for continuous quality improvement of both products and processes, experimental design (ED) based on Shainin s variables search method is still playing a major role in the manufacturing community. This article presents a simple methodology in a step-bystep manner for Shainin s variables search method, in terms of uncovering the key process variables or factors which influence a response or quality characteristic of interest. In order to illustrate the potential of this powerful methodology as a problem solving tool, a simple catapult experiment was carried out. The authors believe that this simple experiment can be used in the classroom to train engineers and scientists on variables search method in order to identify the most critical variables which influence the process or product performance. # MCB UP Limited [ISSN ] [DOI / ] Introduction Variables search method, developed by Dorian Shainin, is an experimental design (ED) approach for identifying the critical variables in any manufacturing processes (Shainin and Shainin, 1988). Bhote (1988) refers to the variables search method as ``the Rolls Royce of variation reduction and claims that it is far superior to fractional factorial designs. This method is useful to study when four or more variables in a process need to be investigated. This paper illustrates an alternative approach to ED called variables search method for spotting the critical variables from a large number of variables. The focus of this paper is to demonstrate the power of variables search method as a problem solving tool using a catapult experiment and thereby encourage the engineering community to apply it in their own work environments for tackling process quality related problems. Shainin refers to the most important variable as the ``red X, the second most important variable as the ``pink X and then ``pale pink X, and so on. Shainin advocates the reduction of the sources of variation down to a manageable number (generally up to four), from which he conducts a full factorial experiment (Logothetis, 1990). The objective of a full factorial experiment is to study the effect of all variables and their interactions independently and reliably. A detailed explanation on full factorial experiments may be obtained from the book World Class Quality (Bhote, 1991). A methodology for the variables search method The variables search method consists of the following five phases. The Emerald Research Register for this journal is available at h ttp://w ww.emeraldinsight.com/researchregister Phase 1: develop and test a list of input process variables or parameters In this phase, we need to develop a list of input process variables which are relevant to the process under investigation. The selection of these process variables is based on a thorough brainstorming session with people from manufacturing, quality, design and of course operators on the shop-floor. Having identified the factors, we now need to rank them in descending order of perceived importance (usually based on sound engineering knowledge of the process and experience is used). Assign two levels to each process variable, best level (+1) and worst level (±1). The method begins with two experiments, one with all process variables at their best levels (+1) and the other one with all process variables at their worst levels (±1). The experiments are replicated twice to obtain a total of six experiments or trials. All six experiments must be carried out in a random manner, to ensure that uncontrolled variables (or lurking variables) do not bias the response of interest. The response is the output or quality characteristic which is to be measured in an experiment. If the two sets of responses collected from the experiment appear to be disparate after taking experimental error into account, then the list developed at the beginning indeed contains the critical variables. In order to meet the above objective, we estimate the difference between the medians (D M ) of the output response from experiments at all best and worst level settings (i.e. from six experiments). Also estimate the range of the output response values at both best and worst level settings and the average range. The average range ( R) is given by the equation: The current issue and full text archive of this journal is available at w w.em eraldinsight.com / h tm [ 405 ]

2 for Shainin s [ 406 ] R ˆ RB R W 2 1 where: R B = range of the output response values at best level settings (+1); R W = range of the output values at worst level settings (±1). Once we know (D M ) and ( R), apply the D M : >1.25 : 1 rule. This implies that the difference between the medians of the three replications in the above two experiments must exceed the average of the ranges by a factor of at least If our results satisfy this criterion, we can move on to phase 2. If this condition is not satisfied, then either the list does not yet contain the critical or key process variables or else the settings of one or more variables must be reversed. The results from this phase are used to determine test criteria for later experiments (Ledolter and Swersey, 1997). The replications are used to estimate the experimental error (i.e. pooled standard deviation). Phase 2: computation of control limits (CLs) for the median In this phase, we calculate the control limits (also called prediction intervals) for the median for the above two experiments. Here control limits are used to determine whether a variable is important or unimportant and thereby we can make a decision for its elimination. If (M B ) represents the median for an experiment with all factors at their best level, the control limit (CL) at 95 per cent confidence level can then be calculated using the equation: R CL B ˆ M B 2:776 ; 2 d 2 where the constant is the value corresponding to a two-tailed t-distribution with 95 per cent confidence and four degrees of freedom (i.e. 6 ± 2 = 4) and d 2 is statistical constant equal to (based on a sample size of three, i.e. the number of experiments at best and worst levels). Similarly, the control limit (CL) for an experiment with all factors at their worst level (at 95 per cent confidence level) can be calculated using the equation: CL W ˆ M W 2:776 R d 2 ; where M W = median with all factors at their worst level. Phase 3: separating the critical variables from the unimportant ones This is also called the ``swapping phase. In this phase, we carry out an experiment (i.e. 3 experiment 7) where the most important process variable (ranked number one) is kept at its worst level while all the other factors are kept at their best level. Similarly, another experiment is carried out (i.e. experiment 8) by changing the level of most important process variable from its worst level to best level, while the levels of all other factors are kept at their worst level. If the output response values from the above two experiments fall within the median limits, then the influence of the swapped process variable (i.e. process variable 1) can be disregarded. In contrast, if the output response values fall outside the median limits, then this swapped process variable either by itself or its interaction might have significant influence on the response. The swapping strategy is repeated for the second important process variable, third important process variable and so on and so forth. The procedure is repeated for all other process variables in the experiment in the order of perceived importance. When two or three process variables are identified from the experiment, Shainin advocates conducting what is called a ``capping run (Logothetis, 1990). Phase 4: capping run Capping runs are carried out for confirmation purposes so that process parameters or variables that have already been identified (i.e. red X, pink X and pale pink X) are solely responsible for the root causes of variation. In this stage, the most important process variables identified from swapping phase are studied at their best levels and the remaining process variables or parameters are set at their worst levels. Similarly, we run another experiment with the most important process parameters set at their worst levels and the remaining process parameters studied at their best levels. Capping runs are said to be successful when the output response values lie within the median limits. If the capping runs are unsuccessful, then it implies that either you have missed considering the critical process parameters in the first phase or you may have to identify few more critical process parameters from the list of variables. Generally red X, pink X and pale pink X variables are responsible for the root causes of any quality or process related problems. Phase 5: factorial analysis Using the data collected from the above phases, a factorial analysis is carried out to quantify the effects of red X, pink X and pale pink X and the interaction effects among them. Shainin recommends the use of

3 for Shainin s interaction plots to see whether or not the variables are interacting. Catapult experiment: an illustration of variables search procedure This section illustrates how variables search procedure is applied to a catapult in optimising the in-flight distance. The catapult experiment is quite well known among many industrial engineers and six sigma professionals today. The authors perceive that this experiment will act as a powerful training tool for teaching all types of experimental designs (Taguchi, Classical and Shainin). The objective of the experiment was to identify the red X, pink X and pale pink X process variables which influence the in-flight distance of a catapult. Further to a thorough brainstorming session, the authors have identified seven process variables which were thought to influence the in-flight distance. Table I presents the list of process variables which were considered for the experiment. The variables search method was carried out by strictly following the above five phases. The following were the objectives of the experiment: 1 Spot the key process variables which influence the in-flight distance in a catapult experiment. 2 Classify the process variables, if possible, into red X, pink X and pale pink X. 3 Study the interactions among the key variables using a full factorial design. 4 Determine the optimal settings to maximise the in-flight distance. Having identified the objectives, the response of interest and the seven possible process variables, the next step was to use brainstorming for prioritising the perceived importance of the variables used for the experiment. Develop and test a list of process variables Usually this stage is performed based on past experience and engineering knowledge of the process or product. Table II presents the Table I Process variables and their range of settings for the experiment Process variables Labels W orst level (±) Best level (+ ) B all type A Lig ht (L) H ea vy (H ) R ub ber band typ e B B row n (L) B la ck (B) S top p ositio n C 1 4 P eg heig ht D 2 4 H oo k pos ition E 2 4 C up pos ition F 5 6 R ele ase ang le G process variables with their factor settings and their perceived order of importance. The next step was to carry out the six experimental trials in a random manner, of which three trials were performed by keeping all process variables at their ``best levels and the other three with all variables at their ``worst levels. The results of the experiment are shown in Table III. Calculations of D M and R _ Median for the best level settings: (M B ) = 451cm Median for the worst level settings: (M W ) = 66cm Difference between the medians: (D M ) = 451 ± 66 = 385cm Range of the in-flight distance at best level settings: (R B ) = 453 ± 458 = 5cm Range of the in-flight distance at worst level settings: (R W ) = 70 ± 63 = 7cm Average range: ( R) = 6cm D M : R = 385 : 6 = 64.17: 1, which is far greater than 1.25 : 1. This indicates that the results from phase 1 of the experiment have been successful. This would take us to phase 2 of the methodology. Control limits for the median at best and worst level settings Prediction intervals or control limits of 95 per cent for the best and worst level settings are obtained using equations (2) and (3). The control limits for the best and worst level settings are: ½ CL B ˆ 451 2: :693 ¾¼ ˆ 451 9:838 ˆ 441:16; 460:84Š: Similarly: ½ ¾¼ 6 CL w ˆ 66 2:776 1:693 ˆ 66 9:838 ˆ 56:16; 75:84Š: Swapping and capping phase This phase is carried out with the sole purpose of separating out the vital few from the trivial many. For the present study, stop position (Sp) is the first process parameter or variable to be swapped. The observed in-flight distance (350cm) from experiment 7 is not within the prediction interval (441.16, ). Similarly, the result of experiment 8 (104 cm) is not within the prediction interval (56.16, 75.84). Hence we can conclude that stop position (Sp) by itself or in combination with other process parameters has a significant impact on distance. Therefore it cannot be [ 407 ]

4 for Shainin s Table II List of variables and their levels for the experiment Process variables Labels W orst level (±) Best level (+ ) S top position S p 1 4 P eg heig ht P h 2 4 R elease an gle R a H oo k po sition H p 2 4 Ball type Bt Light (L) Heavy (H ) R ub ber band ty pe R t B row n (L) B lac k (B ) C up p osition C p 5 6 Table III Stage 1 of the variables search design P roc ess variables Experim ent Sp Ph Ra Hp Bt Rt Cp Distance (cm ) 1 (3 ) ± ± ± ± ± ± ± (4 ) ± ± ± ± ± ± ± (1 ) (6 ) (2 ) ± ± ± ± ± ± ± (5 ) No te: Figures in parentheses indicate the experim ental trials in random order crossed out. Peg height (Ph) is the next process variable to be swapped. Again, the observed distance is outside the prediction interval for both experiments (i.e. experiments 9 and 10). Hence we conclude that peg height (Ph) is an important process variable. Once we identify two important process variables, the next phase is to perform a capping run with these two process variables. The results of the capping run (i.e. experiments 11 and 12) have shown that the in-flight distance values fall outside the control limits or prediction interval. This implies that the capping run was unsuccessful, which indicates that there are more process variables which might have some significant effect on the in-flight distance. The next process variable that needs to be swapped was release angle (Ra). The results of the experiment (experiments 13 and 14) have shown that the observed values fall outside the control limits, which indicates that release angle (Ra) either by itself or in combination with other process variables is important. The next step is to perform two three-factor capping runs. In the first experiment (refer to experiment 15 in Table IV), factors Sp, Ph and Ra are set at their best level settings (+1) and the remaining process variables at their worst settings (±1). In the second experiment, factors Sp, Ph and Ra are set at their worst level settings (±1) and the remaining process variables at their best level settings (+1). The results of these two experiments (i.e. 443cm and 60cm) fall within the prediction interval (refer to Table IV), which clearly indicates that the capping runs have been successful and therefore the remaining factors can be ignored in the next round of experimentation. The results from the swapping and capping stages have shown that three process variables such as Sp, Ph and Ra appear to have significant influence on in-flight distance. Having identified these three key process variables, the next goal was to classify the variables into red X, pink X and pale pink X. To accomplish this objective, it was important to quantify the estimates of these variables and also the interactions among them using a factorial analysis. Factorial analysis The purpose of the factorial analysis is to obtain an independent and reliable estimate of the effects of process variables (or design parameters) and the interactions among them. The results of the factorial analysis are shown in Table V. In order to identify the key process variables and their interactions (if present), it was decided to perform a pooled ANOVA (analysis of variance). ANOVA is used to identify which of the effects are statistically significant. Pooling is a process of obtaining a more accurate estimate of the error variance and is usually accomplished by starting with those effects with the [ 408 ]

5 for Shainin s Table IV Results from swapping and capping stages of the variable search design S p 7 ± ``Sp is im portant 8 + ± ± ± ± ± ± ``Sp is im portant P h 9 + ± ``Ph is im portant 1 0 ± + ± ± ± ± ± ``P h is im portan t C apping runs Sp and P h ) ± ± ± ± ± Capping runs are unsuccessful 1 2 ± ± Capping runs are unsuccessful R a ± ``Ra is im portant 1 4 ± ± + ± ± ± ± ``Ra is im portant C apping runs Sp, P h and Ra) ± ± ± ± Capping runs are successful 1 6 ± ± ± Capping runs are successful Table V Results of factorial analysis Process variables/ param eters Sp Ph Ra Results M edian 1 ± ± ± ± ± ± + ± ± ± ± ± ± smallest sum of squares and continuing with the ones having successively larger effects (Antony and Kaye, 2000). Taguchi advocates pooling effects until the degrees of freedom is nearly equal to half the total degrees of freedom for the experiment (Taguchi, 1987). The tool ANOVA subdivides the total variation in the data into variation due to main effects, interaction effects and variation due to error. The results of the ANOVA are shown in Table VI. The table shows that the main effects are statistically significant (F-ratio = 17.78) at 5 per cent significance level. None of the interactions were found to be statistically significant. Figure 1 shows the interactions among the three process parameters Sp, Pg and Ra. The parallel lines in the interaction plot (refer to Figure 1) are an indication of the absence of the influence of one process parameter over another. The next stage of the analysis was to identify the red X, pink X and pale pink X. In order to accomplish this objective, a main effects plot was constructed (refer to Figure 2). The magnitude tells us of the strength of the effect of a process parameter and the sign of a main effect tells us of the direction of the effect, i.e. if the response increases or decreases. The effect of a process parameter in this context refers to the difference in the average of the response (i.e. in-flight distance) at best and worst level settings. Table V presents the results of the calculations of main and interaction effects of all process parameters (or factors). Both Table VII and Figure 2 indicate that Sp is the most important process parameter (i.e. red X), followed by Ph (pink X) and Ra is the third important process parameter (i.e. pale pink X). Determination of optimal process parameter settings Having identified the key process parameters, the goal was then to determine the optimal settings that will maximise the projecting distance. The optimal settings can be easily determined from Table VII. Table VII clearly indicates that maximum [ 409 ]

6 for Shainin s Table VI ANOVA table Source Deg rees of freedom Sum of squares M ean square F-ratio P -value M ain effects ,382 48, * Two-w ay interactions 1 2,261 2, Po ole d error 3 8,122 2,7 07 ± Total ,766 ± ± No te: * im plies that m ain effects are statistically sig nificant at 5 per cent significance level Figure 1 Interaction graph for Sp, Ph and Ra Figure 2 Main effects plot from the factorial experiment in-flight distance was observed when all the process parameters (Sp, Ph and Ra) were kept at best level settings (+1). The remaining process variables are found unimportant and have very little impact on the in-flight distance. Three replications were carried out with process parameters of Sp, Ph and Ra kept at best level settings. The in-flight distance at the optimal condition yielded an average distance of 450cm. [ 410 ]

7 for Shainin s Significance and implications of the work The purpose of this section is to bring the importance of training experimental design (ED) to engineers/business managers with limited mathematical and statistical skills for tackling process/product quality and capability related problems in organisations. The authors have chosen Shainin s approach of ED due to its simplicity and ease of implementation in real life situations. Moreover, the approach promoted by Shainin is non-academic but very practical in the sense that less human resources and training are required to support the experimentation compared to Taguchi and Classical approach of experimental design (Antony, 1999). The paper illustrates the point that Shainin s variables search procedure is a powerful approach to identifying the most influential variables associated with a process or system. The purpose of this simple catapult experiment was to bridge the gap in the statistical knowledge required by engineers/business managers. The authors believe that the experiment would enable many engineers and mangers in organisations to view experimental design in a more approachable manner, which consequently led to their eagerness to apply it in their own work environment. This experiment also would assist engineers to understand the importance of identifying the key process variables and their interactions (if any) for process optimisation problems. The results of the experiment will provide the stimulus for the wider application of Shainin s variables search procedure in achieving the following objectives (Goodman and Wyld, 2001): To identify the key process /design variables (red X, pink X and pale pink X) that influence critical quality characteristics of interest to customers To relax tolerances on the unimportant process/design variables in order to cut down costs of manufacture. To identify any significant interactions existing among the key variables, which influence critical quality characteristics. Conclusion Shainin s design is a powerful tool in spotting the key or critical process parameters (such as red X, pink X and pale pink X) from a large list of process parameters. The purpose of this paper is to increase the awareness of the use of this powerful tool to the engineering community for problem-solving activities and identifying those process parameters which are responsible for poor process performance. In order to illustrate the methodology of Shainin s variables search, the authors have used a catapult experiment with the objective of spotting the key process parameters (red X, pink X and pale pink X) which influence the in-flight distance. The effectiveness of this methodology depends on how one ranks the factors and assigns the best and worst levels. Although it is a powerful tool for identifying the key process parameters, the method fails to consider the analysis of process variability. In other words, Shainin s design concentrates on the analysis of mean response only instead of analysing the mean and variability as two different responses. The authors would like to accentuate the point that Shainin s approach could be useful for processes which are already achieving high stability and capability. Table VII Table of main and interaction effects Factors Tw o factor interactions Three factor interactions Trial S p Ph Ra Sp Ph Sp Ra Ph Ra S p Ph Ra Distance 1 ± ± ± ± ± ± ± ± ± + ± ± + ± ± + ± ± ± ± ± + + ± ± ± + ± + ± ± ± + + ± ± + ± M ean distance (±) M ean distance at (+ ) E ffect s ± ±25.88 ±58.12 Pink X R ed X P ale pin k X [ 411 ]

8 for Shainin s References Antony, J. (1999), ``Spotting the key variables using Shainin s variables search technique, Journal of Logistics and Information Management, Vol. 12 No. 4, pp Antony, J. and Kaye, M. (2000), Experimental Quality, Kluwer Academic Publishers, Norwell, MA. Bhote, K.R. (1988), Strategic Supply Management, American Management Association, New York, NY. Bhote, K.R. (1991), World Class Quality, American Management Association, New York, NY. Goodman, J. and Wyld, D.C. (2001), ``The hunt for the red X: a cast study in the use of Shainin design of experiments in industrial honing operations, Management Research News, Vol. 24, No. 7/8, pp Ledolter, J. and Swersey, A. (1997), ``Dorian Shainin s variables search procedure: a critical assessment, Journal of Quality Technology, Vol. 29 No. 3, pp Logothetis, N. (1990), ``A perspective on Shainin s design for quality improvement, Quality and Reliability Engineering International, Vol.6, pp Shainin, D. and Shainin, P. (1988), ``Better than Taguchi orthogonal array tables, Quality and Reliability Engineering International, Vol. 4, pp Taguchi, G. (1987), System of Experimental Design, Kraus International Publication, UNIPUB, New York, NY. [ 412 ]