Measuring System Analysis in Six Sigma methodology application Case Study M.Sc Sibalija Tatjana 1, Prof.Dr Majstorovic Vidosav 1 1 Faculty of Mechanical Engineering, University of Belgrade, Kraljice Marije 16, 111 Belgrade 35 PF 34, Serbia, sibalija@infosky.net Abstract: As the manufacturing industry moves toward 6-sigma capable processes, the same requirement is becoming necessary for measuring system, as well. In the scope of Six Sigma methodology application for the existing manufacturing system, according to DMAIC cycle (Define-Measure-Analyse-Improve-Control), this paper presents case study statistical analysis of the measuring system, as a scientific method for understanding measurement variation and determining how much the variation within the measurement process contributes to overall measured manufacturing process variability. The observed measuring system is used to measure variable values of the most important product quality characteristic, directly related to majority of nonconformities found in the observed manufacturing system. To be fully confident about measurement readings of the product characteristic, it is necessary to understand the extent of confidence that the observed measuring system allows. Keywords: Measuring System Analysis (MSA), Gage R&R, Six Sigma. 1. INTRODUCTION As the requirement for Statistical Process Control (SPC) implementation, Analysis of Measuring System must be perform to ensure that measured values are correct and relevant for analysis based on SPC. In this paper, real case study will be presented analysis of measuring system, within the scope of Six Sigma methodology application in certain Serbian metal-processing manufacturing company, for the observed manufacturing system / process / product. The observed measuring equipment MiniTest 6 B is used to measure variable values of the most important product quality characteristic - pan enamel thickness, directly related to majority of nonconformities found in the observed manufacturing system [Sibalija, Majstorovic, 6]. In order to assess the measuring system overall quality level and its capability to measure the observed product quality characteristic, analysis of the measuring system has been performed using two methods: (1) average and range (X bar R) control charts method, and () ANOVA method, quantifying measuring system characteristics: repeatability and reproducibility, discrimination, stability, bias and linearity. Results of analysis show constituent components of variation occurred during measuring process:
part-to-part variation, operator variation, measuring equipment variation and variation due to interaction effects (if there are any), presenting an input for minimization of variation introduced by measuring process, so that full focus on part-to-part variation (variation of the observed product quality characteristic) can be set. Input data for this MSA are [Sibalija, Majstorovic, 6]: - Specification Tolerance of pan enamel thickness: T=USL LSL=(55 18)µm=38 µm - Discrimination (resolution) of the observed measuring equipment is: µm.. MEASURING SYSTEM ANALYSIS.1. Stability Stability of measuring system is presented at figure 1, by X bar R control chart. One operator measured enamel thickness of the same product 15 times, over time period of four weeks - once per week (table 1. all values in µm). Date Readings (µm) Aver. Range 5.7.6 3; 4; 3; 1; 1; 1; 18; 16;; 14; 8; 6; ; 8; 16 18.933 1.7.6 3; 6; ; ; 1; ; ; 6; 1; 18; 14; ;6; ; 18.67 34 3.7.6 38; 3; 4; 4; 1; 8; 18; 14; 4; ; 1; 16;16; 34; 4.667 36 7.7.6 4; 6; 4; ; 4; 8; ; ; 1; 14; 1; 14;; 6; 18 17.6 Average 18.867 3.5 Table 1. Data for Stability of Measuring System Figure 1. Xbar-R control chart for Stability
Stability Range chart: R mean = 3.5; (D4 = 1.653, for sub-group size=15) UCL = D4 R mean = 5.416 Stability Xbar chart: X mean mean = 18.867 (A =.3, for sub-group size=15) LCL / UCL = X mean mean - / + A R mean = 1.65 / 5.668 Since there are no points out of control limits on Xbar-R chart for Stability [Pyzdek, 3], the observed measuring system is considered as statistically stable... Bias Measuring System Bias is calculated [Pyzdek, 3] by measuring standard part / etalon (known thickness 95 µm) repeatedly 1 times, and finding discrepancy between measurements average value and standard value (table.). Readings (µm) Average (µm) Bias (µm) 95; 95; 93; 95; 95; 97; 94; 95; 97; 95 95.1.1 Table. Data for Bias of Measuring System..3. Gage R&R.3.1. Gage R&R: Xbar R Method In order to calculate Gage R&R, 3 operators measured 5 different products/parts 3 times, to estimate measuring equipment variation (repeatability), operator variability (reproducibility) and variation of pan enamel thickness (part-to-part variation). Results are presented in table 3. Xbar-R chart for Repeatability is presented at figure. The same measuring data are rearranged to calculate Reproducibility and presented in table 4 and Xbar-R chart for Reproducibility is presented at figure 3. Results of analysis show part-to-part variation, operator variation reproducibility and measuring equipment variation repeatability, as well as measurement variation relative to the tolerance of the pan enamel thickness (table 5). a.) Repeatability (all values presented in table 3. are in µm) Part Reading1 Reading Reading3 Average Range Operator 1 1 5 54 5 54 5 54 51.333 54 3 58 58 58 58 4 7 7 7 7.667 5 66 66 66 66 Operator 1 54 56 5 56 5 54 5.667 55.333 3 58 58 56 57.333 4 7 68 7 69.333 5 64 68 68 66.667 4 Operator 3 1 5 5 5 54 5 5 5.667 5.667 3 6 58 56 58 4 4 68 68 68 68 5 64 64 64 64 Average 59.644 1.6 Table 3. Data for Repeatability of Measuring System.
Repeatability Range chart: R mean = 1.6; UCL = D4 R mean = 4.118 Repeatability Xbar chart: X mean mean = 59.64444 LCL / UCL = X mean mean - / + A R mean = 58.8 / 61.81 (D4 =.574, for sub-group size=3) (A = 1.3, for sub-group size=3) - Standard Deviation for Repeatability (gage variation): Sigma repeat. =Sigma e = R mean /d* =.93, (d* = 1.7, for 3 readings and 3 inspectors x 5 parts). - Repeatability: 5.15 Sigma e = 4.798, (5.15 - const. - Z ordinate which includes 99% of a standard normal distribution). At R chart for Repeatability, all values are less than UCL - measurement system s variability due to repeatability is consistent there are no special causes of variation. At Xbar chart, more than half of points are out of control limits variation due to gage repeatability error is less than part-to-part variation [Pyzdek, 3]. b.) Reproducibility (all values presented in table 4. are in µm) Read.1 Read. Read.3 Read.1 Read. Read.3 Read.1 Read. Read.3 Part Operator 1 Operator Operator 3 Average Range 1 5 5 5 54 5 5 5 5 5 51.556 4 54 54 54 56 56 54 5 54 5 54 4 3 58 58 58 58 58 56 6 58 56 57.778 4 4 7 7 7 7 68 7 68 68 68 69.333 4 5 66 66 66 64 68 68 64 64 64 65.556 4 Average 59.644 4 Table 4. Data for Reproducibility of Measuring System Figure. Xbar-R control chart for Repeatability Figure 3. Xbar-R control chart for Reproducibility
Reproducibility Range chart: R mean = 4; (D4 = 1.816, for sub-group size=9) UCL = D4 R mean = 7.64 Reproducibility Xbar chart: X mean mean = 59.644 (A =.337, for sub-group size=9) LCL / UCL =X mean mean - / + A R mean = 58.96 / 6.99 - Standard Deviation for reproducibility: Standard deviation for repeatability & reproducibility: Sigma o = R o / d* = 1.34 (d* =.98, for 9 readings and 3 inspectors x 5 parts) Sigma o = Sigma repeat.+reprod. = Sigma repeat. + Sigma reprod. Sigma reprod. = SQRT (Sigma o Sigma repeat. ) =.968 - Reproducibility: 5.15 Sigma reprod. = 4.983 (5.15 - const. - Z ordinate which includes 99% of a standard normal distribution) - Measurem. System Standard Deviation: Sigma m = SQRT(Sigma e +Sigma o ) = 1.633 - Measurement System Variation: R&R = 5.15 Sigma m = 8.41 For Reproducibility, all values at R chart are less than UCL - measurement system s variability due to repeatability & reproducibility is consistent there are no special causes of variation; more than half of points at Xbar chart are out of control limits variation due to gage repeatability & reproducibility error is less than part-to-part variation [Pyzdek, 3]. c.) Part-to-Part Variation - Range of the parts averages: R p = 17.778 - Part-to part standard deviation: Sigma p = R p / d* = 7.168 (d* =.48, for 5 parts and 1 calculation for R) - 99% spread due to part-to-part variation: PV = 5.15 Sigma p = 36.918 d.) Overall Measuring System Evaluation - Total process standard deviation: Sigma t = SQRT ( Sigma m + Sigma p ) = 7.35 -.Total Variability: TV = 5.15 Sigma t = 37.863 - The percent R&R: 1 ( Sigma m / Sigma t ) % =.13% - The number of distinct data categories that can be created with this measurement system: 1.41 (PV / R&R) =6.1891 = 6. Since the number of categories for this measurement system is 6 (>5 minim. required) [Pyzdek, 3], this measuring system is adequate for process analysis / control. Source St.Dev. Variability(5.15 St.Dev.) % Variab. % Tolerance (Var./Toler.) Total gage R&R Repeatability 1.63.93 8.41 4.79.1% 1.65%.7% 1.9% Reproducibility.97 4.98 13.16% 1.35% Part - to - part 7.17 36.9 97.5% 9.98% Total variation 7.35 37.86 1.% 1.3% Table 5. Analysis of spreads - Measurement variation relative to the tolerance. Taking into consideration all relevant factors (cost of measurement device, cost of repair, etc.), the observed Gage R&R System may be accepted, since operators and equipment cause.1% (< 3%) of variation. But, it is far away from doubtless acceptance threshold of 1% [Pyzdek, 3].
.3.. Gage R&R: ANOVA Method Analysis of measuring results using ANOVA method (table 6.), includes analysis of interaction operator * part.num. Since alpha to remove interaction term is set to.5 (for 95% of confidence), variation due to interaction operator * part.num is found as insignificant (table 6.) [Pyzdek, 3]. Results of analysis show constituent components of variation occurred during measuring process (table 7), as well as variations relative to the tolerance of the pan enamel thickness and relative to concerning manufacturing process variation (table 8). Figure 4 gives graphical presentation of these. Source DF SS MS F P PartNum 4 66.31 516.578 338.77. Operator.4 11. 7.7. Repeatability 38 57.96 1.55 Total 44 146.31 (Alpha to remove interaction term =.5) Table 6. Two-Way ANOVA Table Without Interaction, for Gage R&R. Gage R&R (ANOVA) for Data Gage name: Date of study : Reported by : Tolerance: Misc: 1 Components of Variation % Contribution % Study Var 7 Data by PartNum Percent 5 % Tolerance 6 Sample Mean Sample Range 4 7 6 5 Gage R&R Repeat Reprod R Chart by Operator 1 3 Xbar Chart by Operator 1 3 Part-to-Part UCL=4.119 _ R=1.6 LCL= _ UCL=61.8 X=59.64 LCL=58.1 Average 5 7 6 5 7 6 5 1 1 1 3 PartNum Data by Operator Operator Operator * PartNum Interaction 3 PartNum Figure 4. Gage R&R ANOVA method. Observed Gage R&R System may be accepted, since operators and equipment cause 19.6% (< 3%) of variation. But, it exceeds doubtless acceptance threshold of 1% [Pyzdek, 3]. Since the number of distinct categories for this measurement system is 7 > 5 (minim. required), this measuring system is adequate for process analysis or process control. 4 4 5 3 5 Operator 1 3
%Contribution Study Var %Study Var %Toleran. Source VarComp (of VarComp) StdDev (SD) (5.15 SD) (%SV) (SV/Toler) Total Gage R&R.1583 3.63 1.46911 7.5659 19.6.4 Repeatability 1.551.57 1.3497 6.361 16.3 1.7 Reproducibility.6331 1.7.7957 4.979 1.33 1.11 Operator.6331 1.7.7957 4.979 1.33 1.11 Part-To-Part 57.81 96.37 7.5649 38.9594 98.17 1.53 Total Variation 59.3864 1. 7.765 39.687 1. 1.73 Process tolerance = 37 Number of Distinct Categories = 7 Table 7. Components of Variance Analysis. Table 8. Analysis of spreads. Results for Gage R&R, from ANOVA method, differ slightly from results obtained using Xbar-R method, since by using Xbar-R method there is no possibility to include and calculate interaction - effect operator * part.num (in this Gage R&R, this interaction is found as insignificant, but it still takes certain value). Thus, ANOVA method for Gage R&R is considered as more accurate than Xbar-R method..4. Linearity Linearity is determined by choosing products/parts that cover most of the operating range of the measuring equipment; then Bias is determined at each point of the range [Pyzdek, 3]. In this case, 4 parts were chosen, with following expected enamel thickness: 1,, 36 and 47 µm; each part was measured 1 times; discrepancy between their average value and expected value presents bias in that point (table 6). Gage Linearity and Bias Study for Average Gage name: Date of study : Reported by : Tolerance: Misc: 15 1 Regression 95% CI Data Avg Bias Gage Linearity Predictor C oef SE C oef P C onstant -.8.143.745 Slope.1687.673.18 S 1.8755 R-Sq 76.% Linearity.6387 % Linearity 1.7 Bias 5 Gage Bias Reference Bias % Bias P A v erage 4.5 1.7 * 1. 5.3 *. 5.3 * 36 3.6 9.5 * 47 8.6.7 * -5 1 3 4 5 Reference Value Percent Percent of Process Variation 1 5 Linearity Bias Figure 5. Linearity and Bias Study of Measuring System.
Then, a linear regression was performed (figure 5.). Part Readings (µm) Average (µm) Ref.value (µm) Bias (µm) 1 11; 11; 1; 13. 1. 1; 1; 11; 1; 14 1 1 4; ; ; ; ; ; ; ; ; 4 3 36; 368; 36; 364; 368; 364; 36; 36; 364; 364 363.6 36 3.6 4 478; 478; 478; 478; 48; 478; 478; 48; 478; 478 478.6 77 8.6 Table 6. Data for Linearity and Bias of Measuring System The equitation of linearity is (figure 5.): Bias= -.8 +.1687 Ref.value. Since P values are higher then.5 (figure 5.), gage bias is statistically insignificant. R-Sq is 76. %, meaning that straight line explains about 76% of the variation in the bias readings (>5% - acceptable). Further, the variation due to linearity for this gage is 1.687% of the overall process variation. The variation due to accuracy for this gage is 1.6963% of the overall process variation. 3. MEASURING SYSTEM CAPABILITY 3.1. Capability indices for Gage Cg and Cgk According to [Dietrich, 6], capability indices for gage can be calculated by measuring of standard part n times and calculating average measurement value, bias and standard deviation of measurement. From data presented in table., one can find: Average = 95.1 µm; Bias =.1 µm; St.Deviation = 1.197 µm; so, capability indices for this gage are: Cg =. T / 4 St.Deviation = 15.45 >1.33... (1) [Dietrich, 6] Cgk = (.1 T - Bias) / ( St.Deviation) = 15.41>1.33... () [Dietrich, 6] Since values for Cg and Cgk are over 1.33, one can conclude that measuring process is capable according to this criteria. 3.. Precision-to-Tolerance (PTR) ratio, Signal-to-Noise (STN) ratio and Discrimination ratio (DR) The precision-to-tolerance ratio (PTR) is a function of variance of measurement system: PTR =(5.15 SQRT(Variance mesurement system ))/(USL-LSL)) 1% =.45 %... (3) (Variance mesurement system can be found in table 7. - VarComp for Total Gage R&R). As it is stated in [Burdick, Borror, Montgomery, 3], since PTR for this measuring system is less than 1%, the measurement system is adequate. The adequacy of a measuring process is more often determined by some function of proportion of total variance due to measurement system [Burdick, Borror, Montgomery, 3], as it is signal-to-noise ratio (SNR) and discrimination ratio (DR): SNR = SQRT(( (Var p / Var total ) / (1-Var p / Var total )) = 7... (4) DR = (1+ (Var p / Var total )) / (1 - (Var p / Var total )) = 54... (5) (Var p / Var total can be found in table 7. - VarComp for Part-To-Part / Total Variation). AIAG (1995) defined SNR as the number of distinct levels of categories that can be
reliably obtained from the data [Burdick, Borror, Montgomery, 3] and value of 5 or greater is recommended. Also, it has been stated that DR must exceed 4 for the measurement system to be adequate. Values SNR = 7 and DR = 54 indicate that the observed measuring system is adequate. 3..1 Confidence Interval for PTR (95% confidence) Limits for PTR confidence interval are [Burdick, Borror, Montgomery, 3]: L PTR = 5.15 SQRT(Lower Bound)/(USL-LSL)... (6) U PTR = 5.15 SQRT(Upper Bound)/(USL-LSL)... (7) where bounds are: Lower Bound = Estimate Variance mesurement system - SQRT(V LM )/(p r)... (8) Upper Bound = Estimate Variance mesurement system + SQRT(V UM )/(p r)... (9) where: - p = 5 number of different part measured for Gage R&R - r = 3 number of repeated measurement (readings) for Gage R&R - o = 3 number of operators that performed measurements for Gage R&R - Estimate Variance mesurement system = (SD o + p (r-1) SD e ) / (p r)... (1) (SD o / SD e StdDev for Operator / Repeatability from table 8.) - V LM = G MS O + G 4 p (r-1) MS e... (11) V UM = H MS O + H 4 p (r-1) MS e... (1) (MS O / MS e MS for Operator / Repeatability, table 6.; coefficients are: G = 1-1/F(1-α/, o-1, infinite); G 4 = 1-1/F(1-α/, p o (r-1), infinite)... (13) H = 1 / F(α/, o-1, infinite) 1; H 4 = 1 / F(α/, p o (r-1), infinite) 1... (14) where F(.,.,.) is Fisher test value and α =.5 - threshold). Results: Estimate Variance mesurement system = 1,59; V LM = 94.939, V UM =18194.689; Lower Bound =.49, Upper Bound = 9.358 L PTR =,89% <= PTR <= U PTR = 7.54% Since lower and upper limit for PRT are less then 1%, there is sufficient evidence to claim that the observed measuring system is adequate for prodact characteristic measurement! 3.. Confidence Interval for SNR and DR (95% confidence) SNR confidence interval limits are [Burdick, Borror, Montgomery, 3]: L SNR = SQRT(( Lower Bound)/(1-Lower Bound))... (15) U SNR = SQRT(( Upper Bound)/(1-Upper Bound))... (16) where bounds are: Lower Bound = (p L*) / ( (p L*) + o )... (17) Upper Bound = (p U*) / ( (p U*) + o )... (18) where: L*=MS p / ((p (r-1) F(1-α/, p-1, infinite) MS e )+(F(1-α/, p-1, o-1) MS o ))...(19) U*=MS p / ((p (r-1) F(α/, p-1, infinite) MS e )+(F(α/, p-1, o-1) MS o ))...() (MS p / MS e / MS o MS for PartNum / Operator / Repeatability, table 6.). Results: L* = 1.87, U* =179.51; Lower Bound =.644, Upper Bound =.997 L SNR = 1.94 <= SNR <= U SNR = 4.444 Since not all values in the interval for SNR exceed 5, there is not sufficient evidence to claim the measurement system is adequate for monitoring the process. Limits for DR confidence interval [Burdick, Borror, Montgomery, 3]:
L DR = (1+Lower Bound)/(1-Lower Bound)) = 4.64... (1) U DR = (1+Upper Bound)/(1-Upper Bound)) = 598.54... () L DR = 4.64 <= DR <= U DR = 598.54 Regarding to DR confidence interval, one can says that the observed measurement system is adequate for monitoring the process, since both DR limits exceed value 4. Note: All above stated equitation for confidence intervals are valid only in case when interaction effect operator * part.num. is insignificant. 4. CONCLUSION Measuring system analysis has been performed, with good results for all criteria considering central location of measurements. As regards to variability of the measurements (Gage R&R), we conditionally accepted this measuring system for considered measurement. Measuring system capability (presented over gage potential Cg and capability Cgk) satisfies required criteria, as well as confidence interval for PTR and DR ratio. Thus, this measurement system is adequate for monitoring the process, according to Cg, Cgk, PTR and DR criteria. One concern is STN ratio, since its confidence interval doesn t satisfy required value. This could be expected, also, from ANOVA analysis, because Number of Distinct Categories is 7, not far enough from minimum required value 5. Further, this corresponds to only conditional acceptance of the measuring system, regarding to Gage R&R value. In order to absolutely accept this measuring system for pan enamel thickness measurement, clamping of the part or measuring instrument during measuring process and measuring instrument maintenance / repair, if necessary, should be considered. Advanced training for operators and fixture, to help the operators to use measuring instrument more consistently, are advisable. REFERENCES [Sibalija, Majstorovic, 6] Sibalija, T.; Majstorovic, V.; "Application of Six Sigma Methodology in Serbian industrial environment"; In: International Journal Total Quality Management & Excellence, Vol.34, No.3-4, YUSQ EQW 6; Belgrade, Serbia, 6; ISSN 145-699 [Pyzdek, 3] Pyzdek, T.; "The Six Sigma Handbook"; In: McGraw-Hill Companies Inc., USA, 3; ISBN -7-14115-5 [Dietrich, 6] Dietrich, E.; "Using the Process Steps of a Measurement System Capability Study to Determine Uncertainty of Measurement"; http://www.qdas.de/homepage_e/es geht auch einfach_e.htm; Q-DAS GmbH, 6 [Burdick, Borror, Montgomery, 3] Burdick, R.K.; Borror, C.M.; Montgomery, D.C.; "A Review of Methods for Measurement Systems Capability Analysis"; In: Proceedings of the 47th Annual Technical Conference of the Chemical and Process Industries, Journal of Quality Technology, Vol.35, No.4, El Paso, USA, 3