ABSTRACT. Page 1 of 9

Transcription

1 Gage R. & R. vs. ANOVA Dilip A. Shah E = mc 3 Solutions 197 Great Oaks Trail # 130 Wadsworth, Ohio Tel: Fax: emc3solu@aol.com ABSTRACT Quality and metrology technicians are faced with choosing different analytical techniques when determining measurement uncertainty budgets. Gage Repeatability & Reproducibility is a popular technique used by automotive industry suppliers while Analysis of Variance (ANOVA) is thought to be a statistician s analysis tool. This paper discusses the pros and cons of both techniques, using sample data. Metrology technicians will find the presentation of data useful in determining analytical methods for their own requirements. Page 1 of 9

2 INTRODUCTION Quality, calibration and test technicians face many challenges in analyzing data when determining uncertainty budgets. The use of computers and software applications makes it easier to perform data calculations. But the user still has to ensure that correct data is entered, and that a correct judgment is made based on the results provided by the software. Computer spreadsheet models can be easily developed to perform mundane and repetitive data calculations. Gage Repeatability and Reproducibility studies are performed on a routine basis by automotive suppliers. The GM Long Form method is now referred to as the standard method 1 for performing the Gage R & R studies. Once performed on a paper form using a calculator, this is now performed either using a spreadsheet template or using a software application. Analysis of Variance (ANOVA) is a statistical tool. Before the availability of personal computers and spreadsheets, the calculations for ANOVA were cumbersome. Mainframe software applications such as SAS and Minitab made those tasks easier. For those with access to mainframe use, analyzing the ANOVA data required statistician assistance. Now, PC versions of Minitab and similar software exist. Computer spreadsheets usually have add-in ANOVA analysis applications. Simple data analysis can be performed observing a set of statistical rules. This paper uses sample data to compare use of both Gage R. & R. and ANOVA techniques. Both GRR 3 and ANOVA methods can be found in many textbooks and references and is not discussed in this paper. The resultant data between the two methods is compared. UNCERTAINTY ANALYSIS The U. S. Guide to Expression of Uncertainty of Measurements in Measurements 2 defines Type A uncertainty as that obtained by statistical means. 2 One of the simplest ways of determining Type A uncertainty is to calculate standard deviation of a set of data from repeated measurements. When two or more technicians make a measurement, the variability between technicians need to be identified/quantified. If two or more measurement systems are used, then the variability between the instruments also needs to be identified /quantified. Either ANOVA or the GRR method can be set up to quantify Type A uncertainty components for uncertainty budgets. Page 2 of 9

3 DATA PREPARATION As with any analysis, a set of rules needs to be observed when collecting data. Ensure that the instrument is calibrated. Ensure that the process/method is validated. Ensure that the process/method is in statistical control. Use trained technicians/metrologists to perform the study. Ensure repeated measurements are made in a random manner. Use more than one appraiser/technician. 1, 3 and 4 More detailed rules on GRR methods are found in the references noted. DATA The data collected among three technicians is displayed in Table 1. Each technician measured the ten artifacts for a total of three trials. The tolerance for the artifact is defined as units. Technician Alan Nancy James Table 1 GAGE REPEATABILITY AND REPRODUCIBILITY (GRR) METHOD The AIAG publication, Measurement Systems Analysis (3 rd Edition) 3 defines Gage Repeatability and Reproducibility as An estimate of the combined variation of repeatability and reproducibility for a measurement system. The GRR variance is equal to the sum of within-system and between-system variances. Several methods to calculate GGR are documented 1, 3, and 4. The method described in AIAG publication, Measurement Systems Analysis 3 rd Edition 3 was used for this paper. The spreadsheet GRR template calculation is shown in Table 2. Whenever a spreadsheet template is developed, it is important to validate it. For the spreadsheet template used for this paper, sample data from a Measurement Systems Analysis 3 rd Edition 3 was used to validate the calculations. Page 3 of 9

4 Part Appraiser/ Trial# Average Alan Average Range 3.9E E E E E E E E E E Nancy Average Range 2.7E E E E E E-05 3E E-05 2E E James Average Range 4.2E E E E E E E E E-05 1E Part Average Rp R-Dbar X- BarDiff UCLR 9.40E-05 EV 2.15E-05 AV 1.44E-05 GRR 2.59E-05 PV 7.62E-06 TV 2.70E-05 Table 2 Page 4 of 9

5 ANALYSIS OF VARIANCE (ANOVA) METHOD The AIAG publication, Measurement Systems Analysis (3 rd Edition) 3 defines ANALYSIS OF VARIANCE (ANOVA) as A statistical method (ANOVA) often used in designed experiments (DOE), to analyze variable data from multiple groups in order to compare means and analyze sources of variation. The collected data is analyzed using the Microsoft Excel spreadsheet Analysis (ANOVA Twoway with replication) add-in and is shown in Tables 3 and 4. ANOVA: Two-Factor With Replication SUMMARY Total Alan Count Sum Average Variance 4E E E E-10 3E E E E E E E-10 Nancy Count Sum Average Variance 1.9E E E E E E E E E E E-10 James Count Sum Average Variance 5.2E E-10 2E E E E E E E E E-10 Total Count Sum Average Variance 5.9E E E E-10 1E E E E E-10 8E-10 Table 3 Page 5 of 9

6 ANOVA Source of Variation SS df MS F P-value F crit Sample 1.30E E E E+00 Columns 3.53E E E E+00 Interaction 1.67E E E E+00 Within 3.00E E-10 In Table 4: Total 6.32E E-10 Table 4 The Source of Variation column is due to each cause contributor. The SS or Sum of Squares column is the squared sum of deviation around the mean of each source. The df column is the degrees of freedom associated with each source. The mean square (MS) value is also defined as the sample variance and is calculated by dividing the Sum of Squares by degrees of freedom (SS/df). The F column refers to the ratio of the MS source divided by the error (Within) and is used to test null hypothesis against the F-table value (F crit). For this ANOVA, it was set at 95% confidence interval. The hypothesis tests were not considered for this paper. Table 5 further defines the Expected Mean Square (EMS) as the linear combination of variance components of each Mean Square (MS). EMS(for this Source of Variation df EMS example) Sample (Technician) k-1 τ 2 + rγ 2 + nrω 2 τ 2 + 3γ ω 2 Columns (Parts) n-1 τ 2 + rγ 2 + krσ 2 τ 2 + 3γ 2 + 9σ 2 Interaction (Tech. x Part) (n-1)(k-1) τ 2 + rγ 2 τ 2 + 3γ 2 Within (Gage Error) nk(r-1) τ 2 τ 2 Table 5 Page 6 of 9

7 Table 6 breaks down the variance components and associated standard deviations. Variance Std. Dev. Appraiser-Technician (AV) ω E-10 ω 1.36E-05 Part σ E-11 σ 0.00E+00 Interaction(Tech. x Part) γ E-10 γ 1.19E-05 Equipment(EV) τ E-10 τ 2.24E-05 Table 6 Please note that when calculating individual component variances, negative variance components are possible and they are set to zero as shown in the Part component. The ANOVA result is compared with the GRR. In some cases (depending on the method chosen), one may have to multiply the standard deviation from the ANOVA by 5.15 or conversely, divide the GRR values by 5.15 to compare the data. The comparison is shown in Table 7. Source of Variation ANOVA Std Dev. GRR Appraiser-Technician (AV) ω 1.36E E-05 Part σ 0.00E E-06 Interaction(Tech. x Part) γ 1.19E Equipment(EV) τ 2.24E E-05 Total 2.77E E-05 Table 7 Equipment Variation (EV) is generally associated with Repeatability. Appraiser Variation (AV) is generally associated with Reproducibility. Page 7 of 9

8 SUMMARY It can be concluded that: Equipment Variation (EV) and Appraiser Variation (AV) can be calculated by both methods. The difference in the two methods can be attributed to the fact that the GRR method uses estimated method (range method). The ANOVA method is more accurate because it does not use estimates. One cannot obtain interaction (Technician x Part) from the GRR method. The ANOVA method will provide the interaction. The part variation was zero using the ANOVA method while the GRR method yielded part variation. It is possible that this comes from the interaction which the GRR cannot differentiate. Both methods provide a close value for the Total Variation (TV). The total is calculated using the Root-Sum-Square (RSS) method. If using the data in the uncertainty budgets, ensure that the standard deviation is used instead of the EV or AV value. Some GRR packages calculate EV and AV values with 99% confidence interval. To obtain standard deviation from those GRR AV and EV values, the data is divided by The number in this paper reflects standard deviations. Guidelines for GRR acceptability 4 GRR is divided by the part tolerance and expressed as a percentage. The following general guidelines are used to determine gage acceptability. %GRR is <= 10% Gage system is okay %GRR is >10% and <= 30% Gage may be acceptable based on application and is considered marginal. %GRR is > 30% Gage is considered unacceptable and may need improvement/replacement. If the Repeatability is large compared to Reproducibility: Gage/Instrument needs maintenance, repair or calibration. Gage may need redesign. Gage location (environment) needs evaluation. Method needs evaluation. Excessive within part variation. If the Reproducibility is large compared to Repeatability: Operator training is required. Gage Resolution is too large. More refinement on fixture required. Page 8 of 9

9 It is important to note that GRR is a powerful technique and can be performed using a hand calculator and using the GRR standard procedure. It is an approximation method using the range estimator constants to calculate standard deviations. Depending on the GRR method used, the constants provided vary. Some are already shown as reciprocals to enable ease of calculation. The user should be careful of the method chosen. GRR does not provide interaction effects. This should be a consideration if the interaction effects are important for uncertainty estimation. ANOVA is a powerful technique used in many applications. ANOVA method is difficult to calculate manually. However, the ANOVA method is more accurate. Spreadsheets and other statistical packages automate the calculation process. When using the ANOVA spreadsheet application the columns and rows should be set up properly or the results will be misleading. ANOVA will provide interaction effects such as Appraiser and Part interaction shown in the example. Although hypothesis testing was not discussed, ANOVA results will enable testing for statistical significance. An article on the critical evaluation of AIAG GRR method is published in the American Society for Quality, Measurement Quality Division Spring 2001/Winter 2000 issue newsletter (The Standard) titled Pythagorean Theorem to the rescue 5 by Donald S. Ermer Ph. D., P.E. The article is an excellent reference for those wishing to research the GRR/ANOVA techniques. It is hoped that this paper provided the metrology technician with some insight into comparing both GRR and ANOVA techniques without the complicated use of statistics. Taking into consideration the limitations and strengths of each, both techniques can be used to estimate measurement uncertainty. REFERENCES 1. Barrentine, Larry B Concepts of R & R Studies, 2 nd Edition. ASQ Quality Press 2. ANSI/NCSLI Z540-2:1997. U. S. Guide to Expression of Uncertainty of Measurements in Measurements. 3. AIAG. Measurement Systems Analysis 3 rd Edition Kimothy, S. K The Uncertainty of Measurements Physical and Chemical Metrology Impact and Analysis. ASQ Quality Press. 5. Ermer, Donald S. Ph. D., P.E. Pythagorean Theorem to the rescue. ASQ MQD Division newsletter The Standard Spring 2001/Winter 2000 issue. Page 9 of 9