International ejournals

Transcription

1 Available online at International ejournals ISSN DESIGN AND DEVELOPMENT OF A METHODOLOGY FOR MEASUREMENT-UNCERTAINTY-ESTIMATION THROUGH DOE & ANOVA Koilakuntla Maddulety; National Institute of Industrial Engineering; Vihar Lake post; Mumbai E. Mail: koila@rediffmail.com Mobile: Abstract In this paper the author developed a methodology with the applications of DoE & ANOVA for estimating standard components uncertainties and combined uncertainty with minimum number of trails and minimum experimental efforts. The methodology had been validated with practical experimentation in which all the Type B uncertainty components are replaced with Type A uncertainty methods, which is more accurate and scientific. In classical method of uncertainty estimation requires separate efforts for estimating type A uncertainty and Type B uncertainty. The accuracy level of Type B uncertainty is very low as it is estimated indirectly, unlike Type-A uncertainty estimation which is direct experimentation and statistical analysis. The suggested methodology in this paper is highly scientific, accurate, needs minimum experimentation effort. Based on classical methodology separate measurement-uncertainty analyses are carried-out for each product range, in most of the laboratories and NABL approved laboratories. But as per this methodology, for all the ranges of the products or a group of ranges of products a single design, uncertainty experimentation & analysis produce highly reliable uncertainty-measurement results. Hence it is highly recommended to the laboratories where the ranges of product are measured with same laboratory setup. Key Words Measurement-Uncertainty; Combined-Uncertainty; Expanded Uncertainty; Experimental Design (DoE); Analysis of Variance (ANOVA); Factors; Factor Levels

2 938 Introduction Measurement Uncertainty In metrology, measurement uncertainty describes a region about an observed value of a physical quantity, also called a measured quantity, which is likely to vary slightly from true value but enclose the true value of that quantity. Measurement Error, Accuracy, Precision Statistics in Metrology ISO Guide to Expression of Uncertainty in Measurement. This is a brief summary of the method of evaluating and expressing uncertainty in measurement adopted widely by U.S. industry, companies in other countries, NIST, its sister national metrology institutes throughout the world, and many organizations worldwide. These "essentials" are adapted from NIST Technical Note 1297 Measurement equation The case of interest is where the quantity Y being measured, called the measured, is not measured directly, but is determined from N other quantities X1, X2,..., XN through a functional relation f, often called the measurement equation: Y = f(x1, X2,..., XN).. (1) Included among the quantities X i are corrections (or correction factors), as well as quantities that are accounts to other sources of variability, such as different observers, instruments, samples, laboratories, and times at which observations are made (e.g., different days). Thus, the function f of equation (1) should express not only a physical law but also a measurement process, and in particular, it should contain all quantities that can contribute a significant uncertainty to the measurement result. An estimate of the measured or output quantity Y, denoted by y, is obtained from equation (1) using input estimates x 1, x 2,..., x N for the values of the N input quantities X 1, X 2,..., X N. Thus, the output estimate y, which is the result of the measurement, is given by y = f(x 1, x 2,..., x N ) (2) For example, as pointed-out in the ISO Guide, if a potential difference V is applied to the terminals of a temperature-dependent resistor that has a resistance R 0 at the defined temperature t 0 and a linear temperature coefficient of resistance b, the power P (the measurement) dissipated by the resistor at the temperature t depends on V, R 0, b, and t according to P = f(v, R 0, b, t) = V 2 /R 0 [1 + b(t - t 0 )].. (3) Classification of Uncertainty Components The uncertainty of the measurement result y arises from the uncertainties u (x i ) (or u i for brevity) of the input estimates x i that enter equation (2). Thus, in the example of equation (3), the uncertainty of the estimated value of the power P arises from the uncertainties of the estimated values of the potential difference V, resistance R 0, temperature coefficient of resistance b, and temperature t. In general, components of uncertainty may be categorized according to the method used to evaluate them. Type A evaluation: method of evaluation of uncertainty by the statistical analysis of series

3 939 of observations, Type B evaluation: method of evaluation of uncertainty by means other than the statistical analysis of series of observations. Representation of uncertainty components Standard Uncertainty Each component of uncertainty, however evaluated, is represented by an estimated standard deviation, termed standard uncertainty with suggested symbol u i, and equal to the positive square root of the estimated variance Standard uncertainty: Type A An uncertainty component obtained by a Type A evaluation is represented by a statistically estimated standard deviation s i, equal to the positive square root of the statistically estimated variance s i 2, and the associated number of degrees of freedom vi. For such a component the standard uncertainty is ui = Sx1-bar =. x1-bar Standard uncertainty: Type B In a similar manner, an uncertainty component obtained by a Type B evaluation is represented by a quantity u j, which may be considered an approximation to the corresponding standard deviation; it is equal to the positive square root of u j 2, which may be considered an approximation to the corresponding variance and which is obtained from an assumed probability distribution based on all the available information. Since the quantity u j 2 is treated like a variance and u j like a standard deviation, for such a component the standard uncertainty is simply u j. Evaluating uncertainty components: Type A A Type A evaluation of standard uncertainty may be based on any valid statistical method for treating data. Examples are calculating the standard deviation of the mean of a series of independent observations; using the method of least squares to fit a curve to data in order to estimate the parameters of the curve and their standard deviations; and carrying out an analysis of variance (ANOVA) in order to identify and quantify random effects in certain kinds of measurements. Mean and standard deviation As an example of a Type A evaluation, consider an input quantity X i whose value is estimated from n independent observations X i,k of X i obtained under the same conditions of measurement. In this case the input estimate x i is usually the sample mean µ = X i-bar = (4) and the standard uncertainty u(x i ) to be associated with x i is the estimated standard deviation of the mean Standard Uncertainty of i th Variable/Factor = = u(xi) xi-bar= S xi-bar = (5)

4 940 Combining uncertainty Calculation of combined standard uncertainty The combined standard uncertainty of the measurement result y, designated by u c (y) and taken to represent the estimated standard deviation of the results, is the positive square root of the estimated variance u c 2 (y) obtained for factor 1 to m u c (y) = = cu = ( x1-bar^2 + x2-bar^2 + x3-bar^2 + xm-bar^2 ) = S cu = (S x1-bar 2 +S x2-bar 2 +S x3-bar 2 +.S xm-bar 2 ). (6) Expanded Uncertainty and Coverage Factor Expanded Uncertainty: Although the combined standard uncertainty uc is used to express the uncertainty of many measurement results, for some commercial, industrial, and regulatory applications (e.g., when health and safety are concerned), what is often required is a measure of uncertainty that defines an interval about the measurement result y within which the value of the measured Y can be confidently asserted to lie. The measure of uncertainty intended to meet this requirement is termed expanded uncertainty, suggested symbol U, and is obtained by multiplying uc(y) by a coverage factor, suggested symbol k. Thus U = kuc(y) and it is confidently believed that Y is greater than or equal to y - U, and is less than or equal to y + U, which is commonly written as Y = y ± U. Coverage factor In general, the value of the coverage factor k is chosen on the basis of the desired level of confidence to be associated with the interval defined by U = kuc. Typically, k is in the range 2 to 3. When the normal distribution applies and uc is a reliable estimate of the standard deviation of y, U = 2 uc (i.e., k = 2) defines an interval having a level of confidence of approximately 95 %, and U = 3 uc (i.e., k = 3) defines an interval having a level of confidence approximately 99.7 %. U = uc (i.e., k=2.575) defines an interval having a level of confidence approximately 99 % In General, U =5.15 uc (i.e k = *2 ) it covers total width (either side) of uncertainty interval at 99% confidence, it should not exceed the 25% of Specification width i.e 0.25*(USL LSL) to accept a measurement System. Objectives of the Research Project: 1. To Understand the meaning and Importance of DoE & ANOVA and Applications of DoE & ANOVA in Measurement-Uncertainty 2. To identify all the factors that influence the Measurement Uncertainty through Brainstorming 3. To identify the levels for each factor identified above through Brainstorming 4. To identify the significant first-order interactions to be considered in design 5. To develop Experimental Designs through Minitab Software 6. To Conducting Analysis (Analytical experiments) as per Designs 7. Analyzing the output of experimental analysis Through ANOVA 8. Estimating Component-wise Uncertainty ; Combined Uncertainty and Expanded Uncertainty 9. Grading the measurement System with the help of Table 1 (with proper Interpretation). 10. Action recommended in the case of C, B, and B+ grading of Measurement System

5 941 Methodology used in the Project: Brainstorming technique used for identifying Critical input Factors/Components and their levels that causes Measurement-Uncertainty, and to identify 1st order interaction effect that effect/causes Measurement-Uncertainty. Minitab16 Software is used for developing fractional factorial design by considering the factors and levels identified. Conduced the experiments as per random run-order generated by the Minitab software Analyzed the data after feeding the output of the experiments against each run in Minitab design through Minitab Interpretations and meaningful conclusions are made based on Analysis of Minitab Software output. DoE and ANOVA Importance, Meaning and Application in Measurement-Uncertainty Why is DoE Important? DoE is important as a formal way of maximizing information gained while resources required. It has more to offer than 'one change at a time' experimental methods, because it allows a judgment on the significance to the output of input variables acting alone, as well input variables acting in combination with one another. 'One change at a time' testing always carries the risk that the experimenter may find one input variable to have a significant effect on the response (output) while failing to discover that changing another variable may alter the effect of the first (i.e. some kind of dependency or interaction). This is because the temptation is to stop the test when this first significant effect has been found. In order to reveal an interaction or dependency, 'one change at a time' testing relies on the experimenter carrying the tests in the appropriate direction. However, DoE plans for all possible dependencies in the first place, and then prescribes exactly what data are needed to assess them i.e. whether input variables change the response on their own, when combined, or not at all. In terms of resource the exact length and size of the experiment are set by the design and outcome analysis guarantee the accuracy of estimating effect of each input variable on output. Hence it has a great advantage over classical experiments that are conducted for assessing measurement uncertainty. Experimental Designs / Design of Experiments (DoE) DoE is a systematic approach to investigation of a system or process which includes measurement system or measuring process. A series of structured tests are designed in which planned minor changes that covers a ranges of random variation generally occurring in the input variables of a measuring process or measurement system. The total effects of these changes on measured value are assessed as combined uncertainty through ANOVA, and also effect of individual input variable (Testers, Gauges, Temperature, Balances, Methods of Testing, etc.) measured as components of combined uncertainty or components of Type A & Type B uncertainties through ANOVA. Then one can estimate the expanded uncertainty as explained above by multiplying with coverage factor. Generally the ratio of expanded uncertainty to specification width is estimated and compared for grading the measurement system and taking appropriate corrective action as and when needed with the help of following table:

6 942 Table 1: Verification of Status of Measurement System: Sl. Percentage (%) No. Ratio *100 Ratio of Expanded Uncertainty to the Specification width = R Measurement System Grade is.. 1 R < 0.10 R < 10% A+ Grade (Excellent) < R < % < R < 15% A Grade (Very Good) < R < % < R < 20% B+ Grade (Good) < R < % < R < 25% B (Satisfactory) 5 R > 0.25 R > 25% C Grade (Not Acceptable) Ratio R = (Expanded-Uncertainty/Specification-Width) Remark Ensure the Same Use the System without any correction Scope for Improvement Large Scope for Improvement Do not use the measurement system until correction How to use DoE? The order of tasks for using this tool starts with identifying the input variables and the response (output) that is to be measured. For each input variable, a number of levels are defined that represent the range for which the effect of that variable is desired to be known. An experimental plan is produced which tells the experimenter where to set each test parameter for each run of the test. The response is then measured for each run. The method of analysis is to look for differences between response (output) readings for different groups of the input changes. These differences are then attributed to the input variables acting alone (called a single effect) or in combination with another input variable (called an interaction). Experimentation for Validating above Procedure As per above methodology we have conducted brainstorming with QA Lab executives of ABC Company under the coordination of author and developed experimental design for estimating detailed Measurement Uncertainty ( Combined Uncertainty and each Component Uncertainty ), by identifying the following factors and levels through brainstorming. Based on the Brainstorming we have titled the Project; identified all the Input variable/factors; identified the levels for each factor and developed the experimental designs through Minitab16 (See the Minitab generated design in Table 2) Output of Brainstorming session: Table2: The Input Factors (Variables) and Levels with significant 1st order Interactions Identified through Brainstorming are:

7 943 Sl. Factors No. Levels Remark Nos Levels Low High 1 Product 2 Pro1 Pro2 Pro1: Product 1 is required by a customer whose Min Assay requirement is least Pro2: Product 2 is required by a customer whose Min Assay requirement is highest among all the customer 2 Operator 2 Ope1 Ope2 Ope1: operator 1 whose experience is least Ope1: operator 2 whose experience is maximum 3 Titrator 2 Tit1 Tit2 Tit1: Old Titrater Tit2: New Titrater 4 Balance 2 Bal1 Bal2 Bal1: Old Balance Bal2: New Balance 5 Lab Temperature : Least Temp. in the lab 25 0 C 29: Hight Temp. in the lab 29 0 C: 6 Method 2 Met1 Met2 Met1: IPM Met2: USPM 7 Two Significance Interactions are indentified: 1. Temperature and Balance 2. Temperature and Method It has been decided to conduct a study to estimate standard measurement uncertainty for eight components, five main factors effects components measurement-uncertainty, two 1st order interactions effects measurement-uncertainty, one error component measurement-uncertainty and finally estimating combined-uncertainty then estimating expanded-uncertainty. The ratio has been calculated and compared as per table1 for grading the measurement system of a selected laboratory. Minitab-Design had been developed by considering five factors that are Operator, Titrator, Balance, Temp, and Method each at two levels as above and used for first Product 1 Sample Analysis then product 2 Sample Analysis. Fractional Factorial Design Factors: 5; Base Design: 5-Factors, 8-Runs; Resolution: III Runs: 16; Replicates: 2; Fraction: 1/4 The following Design is Generated by Minitab and verified that each column is Orthogonal to all other column:

8 944 Table3: Design is as per Std. Order Sequence: StdOrder RunOrder Operator Titrator Balance Lab Temperature Method Assay % 1 13 Ope1 Tit1 Bal1 29 Met Ope2 Tit1 Bal1 25 Met1 3 4 Ope1 Tit2 Bal1 25 Met2 4 3 Ope2 Tit2 Bal1 29 Met1 5 8 Ope1 Tit1 Bal2 29 Met Ope2 Tit1 Bal2 25 Met2 7 6 Ope1 Tit2 Bal2 25 Met1 8 5 Ope2 Tit2 Bal2 29 Met Ope1 Tit1 Bal1 29 Met Ope2 Tit1 Bal1 25 Met Ope1 Tit2 Bal1 25 Met Ope2 Tit2 Bal1 29 Met Ope1 Tit1 Bal2 29 Met Ope2 Tit1 Bal2 25 Met Ope1 Tit2 Bal2 25 Met Ope2 Tit2 Bal2 29 Met2 Table4: Design is arranged as per Run-order Sequence and conducted experiments as per runorder and shown experimental results i.e Assay% value for each run: StdOrd er RunOrd er Operat or Titrato r Balan ce Lab Temperature Metho d Assay % 10 1 Ope2 Tit1 Bal1 25 Met Ope1 Tit2 Bal2 25 Met Ope2 Tit2 Bal1 29 Met Ope1 Tit2 Bal1 25 Met Ope2 Tit2 Bal2 29 Met Ope1 Tit2 Bal2 25 Met Ope1 Tit2 Bal1 25 Met Ope1 Tit1 Bal2 29 Met Ope1 Tit1 Bal2 29 Met Ope1 Tit1 Bal1 29 Met Ope2 Tit2 Bal2 29 Met Ope2 Tit1 Bal2 25 Met Ope1 Tit1 Bal1 29 Met Ope2 Tit2 Bal1 29 Met Ope2 Tit1 Bal2 25 Met Ope2 Tit1 Bal1 25 Met Analysis and Interpretations of Experimental output with the help of Minitab:

9 945 Minitab output of ANOVA: Assay% versus Operator, Titrator, Balance, Lab Temperature, Method Source DF Seq SS Adj SS Adj MS F P Main Effects Operator Titrator Balance Lab Temperature Method Way Interactions Titrator*Balance Titrator*Method Residual Error Pure Error Total R-Sq = 30.69%; R-Sq(pred) = 0.00%; R-Sq(adj) = 0.00% The R-sq value indicates that entire variation occurring in measurement system is random; there is no variable acting as cause for measurement-uncertainty (variation). Same is supported by individual component P Value i.e No identified factor can contribute to measurement-uncertainty (variation) significantly. Table5: Adj. Mean Squares (Variance 2)); Standard Deviations ( i); each Component Uncertainty (Standard Error = SE = xi-bar ) Component (Source of Adj Mean Squares Standard Deviation = Component-Uncertainty Variation) (Variance 2) Sqrt of Variance ( (Standard Error (SE)) Operator 2 = xop-bar Titrator xop-bar Balance xtit-bar Lab Temp xtemp-bar Method = xmet-bar Temp&Bal xtemp&bal-bar Temp&Method xtemp&met-bar Error xerror-bar Total xtotal-bar Combined uncertainty has been estimated by using the formula 6, which is shown below: u c (y) = = cu = xtotal-bar ( x1-bar^2 + x2-bar^2 + x3-bar^2 + xm-bar^2 ) = S cu = (S x1-bar 2 +S x2-bar 2 +S x3-bar 2 +.S xm-bar 2 ) Combined uncertainty: uc(y) = = cu = xtotal-bar

10 946 99% confidence either side Expanded Uncertainty = EU = 5.15* cu 5.15 * = Assay% Specification width for customer-a = USL LSL = = 1 Ratio in % = (Expanded-Uncertainty / Specification-Width) * 100 = ( /1) * 100 = 3.24% Conclusion: The ratio is 3.24% which is less than 10% hence the measurement system is excellent Ref. Table1 for grading Measurement System) In case if the ratio is more than 25%, the measurement system is not acceptable for most of the laboratories, under this situation one has to check each component-uncertainty that are estimated and posted in last column of Table 6 above, along with some special Minitab analysis for finding the maximum contributing components and necessary analysis and corrective action. The author has explained in detail in his next paper titled as A case-study on application of DoE & ANOVA for Measurement Uncertainty, for taking corrective action whenever the Measurement Uncertainty cross the limits i.e ratio is more than 25%. References: 1. Angela and Daniel Voss; Design and Analysis of Experiments; Springer 2. Barry N. Taylor and Chris E. Kuyatt; Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results; NIST Technical Note 1297, 1994 Edition 3. C. Croarkin, Measurement Assurance Programs, Part II: Development and Implementation, NBS SpecialPublication 676-II (U.S. Government Printing Office, Washington, DC, 1985). 4. CIPM, BIPM Proc.-Verb. Com. Int. Poids et Mesures 54, 14, 35 (1986) (in French); P. Giacomo, News from the BIPM, Metrologia 24, (1987). 5. C. F. Dietrich, Uncertainty, Calibration and Probability, second edition (Adam Hilger, Bristol, U.K., 1991), chapter Douglas C. Montgomery, Design and Analysis of Experiments; John Wiley & Sons, Inc. New York, Singapore. 7. D.R. COX, The Theory of the Design of Experiments 8. Eisenhart. C, Realistic Evaluation of the Precision and Accuracy of Instrument Calibration Systems, 9. G. E. P. Box, W. G. Hunter, and J. S. Hunter, Statistics for Experimenters (John Wiley & Sons, New York, NY, 1978). 10. Jiju Antony, Design of Experiments for Engineers & Scientists; Elsevier Science and Technology Books 11. Klaus Hinkelmann and Oscar Kempthorne; Design and Analysis of Experiments (Volume 2, Advanced Experimental Designs); wiley 12. Mandel. J, The Statistical Analysis of Experimental Data (Interscience-Wiley Publishers, New York, NY, 13. M. G. Natrella, Experimental Statistics, NBS Handbook 91 (U.S. Government Printing Office,Washington, DC, 1963; reprinted October 1966 with corrections). 14. Ranjit. K. Roy, Design of Experiments using the Taguchi Approach; John Wiley & Sons, Inc. New York, Singapore. 15. W ISO, Guide to the Expression of Uncertainty in Measurement (International Organization for Standardization, Geneva, Switzerland, 1993 Websites: etc.