UNCERTAINTY OF MEASUREMENT IN CHEMICAL TESTING IN ACCORDANCE WITH THE STANDARD SIST EN ISO/IEC 17025 Table of contents 1 GENERAL... 2 2 THE STANDARD SIST EN ISO/IEC 17025... 2 3 SA'S POLICY IN IMPLEMENTING THE UNCERTAINTY OF MEASUREMENT IN CHEMICAL MEASUREMENTS... 2 4 THE STRATEGY OF IMPLEMENTING UNCERTAINTY OF MEASUREMENT... 3 5 DEFINITIONS... 3 6 UNCERTAINTY OF MEASUREMENT IN CHEMICAL TESTING... 5 6.1 Introduction... 5 6.2 Procedure of estimating the uncertainty of measurement... 5 a. Using validation data and the capacity of methods to estimate the uncertainty of measurement... 7 6.3 Use of ILC information Studies into the capacity of test methods... 8 b. Using information from procedures for assuring the quality of test results... 9 6.4 Using the results of proficiency testing... 9 6.5 Reporting of the results of quantitative testing... 9 7 REFERENCES... 10 8 DISTRIBUTION SLIP... 10 ANNEX 1... 11 Revision 3Release 3 Effective as from: 28.6.200513.12.2006 Page 1 of 11
1 GENERAL This document shall be considered as expert interpretation of the requirements of the standard SIST EN ISO/IEC 17025 for application in laboratories which perform chemical measurements. The document was drawn up by the Sector Committee for Chemistry and approved by the SA Accreditation Committee. It is intended as guidance for laboratories in making preparations for fulfilment of accreditation requirements, and for assessors in accreditation procedures of chemical laboratories. 2 THE STANDARD SIST EN ISO/IEC 17025 SIST EN ISO/IEC 17025 treats the uncertainty of measurement under sub-clause 5.4.6. It prescribes for laboratories to have and to apply procedures to estimate the uncertainty of measurement. The following standpoints are specified in the standard: Laboratories shall use appropriate procedures to estimate the uncertainty of measurement. Statistical methods need not always be used to estimate the uncertainty of measurement. All uncertainty components shall be identified and estimated appropriately for each testing. A reasonable estimation of the uncertainty of measurement shall be based on knowledge of the performance of the method and on previous experience. Method validation data can be used. 3 SA'S POLICY IN IMPLEMENTING THE UNCERTAINTY OF MEASUREMENT IN CHEMICAL MEASUREMENTS The laboratories accredited by SA shall meet the requirements of the standard SIST EN ISO/IEC 17025 related to the estimation of uncertainty of measurement and reporting of the uncertainty of measurement in testing. The requirements are specified in sub-clauses 5.4.6 and 5.10.3.1.c) of the standard. Uncertainty of measurement shall be estimated in quantitative testing. Testing laboratories shall know the uncertainty of measurement related to the results of measurements they perform. Knowledge of the uncertainty of measurement is important for the validity and use of the results of testing. It should be stated in cases when it is important for the correct use of the result of testing; when requested by the customer; or when it has an effect on compliance with the specification. In those cases where a well-recognized test method specifies limits to the values of the major sources of uncertainty of measurement and specifies the form of presentation of results, the laboratory is considered to have satisfied the requirements by following the test method and reporting instructions (see Ref. 1, 5.4.6.2 Note 2). Revision 3Release 3 Effective as from: 13.12.200628.6.2005 Page 2 of 11
4 THE STRATEGY OF IMPLEMENTING UNCERTAINTY OF MEASUREMENT The basic requirement is the identification of all sources of uncertainty of measurement, the identification of the components which essentially contribute to uncertainty of measurement, and the estimation of their contribution. Existing knowledge shall provide the basis for estimating the uncertainty of measurement. Existing experimental data (e.g. data obtained through procedure validation, control charts, ILCs, reference materials, data obtained from the literature etc.) shall be used. 5 DEFINITIONS Measurand (Ref. 2) is the particular quantity subject to measurement. EXAMPLE: Vapour pressure of a given sample of water at 20 C NOTE: The specification of a measurand may require statements about other quantities such as time, temperature and pressure. Uncertainty of measurement (Ref. 2) is the parameter, associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand. NOTES 1 The parameter may be, for example, a standard deviation (or a given multiple of it), or the halfwidth of an interval having a stated level of confidence. 2 Uncertainty of measurement comprises, in general, many components. Some of these components may be estimated from the statistical distribution of the results of series of measurements and can be characterized by experimental standard deviations (Type A) 1. The other components, which can also be characterized by standard deviations, are evaluated from assumed probability distributions (normal, rectangular, triangular) based on experience or other information (Type B) 1. 3 It is understood that the result of the measurement is the best estimate of the value of the measurand, and that all components of uncertainty, including those arising from systematic effects, such as components associated with corrections and reference standards, contribute to the dispersion. Standard uncertainty, u (Ref. 3) is the uncertainty of the result of a measurement expressed as a standard deviation. Combined standard uncertainty, u c (Ref. 3) of the result of a measurement, is an estimated standard deviation equal to the positive square root of the total variance obtained by combining all the uncertainty components, however evaluated, using the law of propagation of uncertainty. 1 The text in parenthesis is not part of the VIM definition. Revision 3Release 3 Effective as from: 13.12.200628.6.2005 Page 3 of 11
Expanded uncertainty, U (Ref. 2) provides an interval within which the value of the measurand is believed to lie with a defined level of confidence. U is obtained by multiplying u c, (the combined standard uncertainty), by a coverage factor k. The choice of the factor k is based on the level of confidence desired (for an approximate level of confidence of 95%, k is 2). Calibration (Ref. 2) is a set of operations to establish, under specified conditions, the relationship between values of quantities indicated by a measuring instrument or measuring system, or values represented by a material measure or a reference material, and the corresponding values realized by standards. NOTES: 1 The result of a calibration permits either the assignment of values of measurands to the indications or the determination of corrections with respect to indications. 2 A calibration may also determine other metrological properties such as the effect of influence quantities. 3 The result of a calibration may be recorded in a document, sometimes called a calibration certificate or a calibration report. Explanatory note: Distinction should be made between validation of an instrument, which is performed through calibration, usually by a competent external operator who, after the calibration, issues a certificate serving to the laboratory as evidence that the characteristics of the instrument are in compliance with the manufacturer's specifications, and daily internal calibration in the laboratory by means of reference materials. Bias (of a measuring instrument) (Ref. 2) is systematic error of the indication of a measuring instrument. NOTE: The bias of a measuring instrument is normally estimated by averaging the error of indication over an appropriate number of repeated measurements. Bias (Ref. 10, 11, 15) is the difference between the expected result of testing and the accepted true value. Validation (Ref. 1-5.4.5, 15) is the confirmation by examination and the provision of objective evidence that the particular requirements for a specific intended use are fulfilled. Validation of a method is the procedure for proving the characteristics of the method's capacities and limitations and determining the influences which might change these characteristics. Validation of a method is the procedure for verifying that the method is appropriate for solving a particular analysis problem. Interlaboratory comparison, ILC (Ref. 16) is the organisation, performance and evaluation of tests on same or similar test items where two or more laboratories collaborate according to predetermined conditions. Revision 3Release 3 Effective as from: 13.12.200628.6.2005 Page 4 of 11
Proficiency testing, PT (Ref. 16) is the verification of a laboratory's competence to perform tests with interlaboratory comparison. Reference standard (Ref. 2) is the standard, generally having the highest metrological quality available at a given location or in a given organization, from which measurements made there are derived. Reference material, RM (Ref. 2) is the material or substance one or more of whose property values are sufficiently homogeneous and well established to be used for the calibration of an apparatus, the assessment of a measurement method, or for assigning values to materials. NOTE: A reference material may be in the form of a pure or mixed gas, liquid or solid. Examples are water for the calibration of viscometers, sapphire as a heat-capacity calibrant in calorimetry, and solutions used for calibration in chemical analysis. 6 UNCERTAINTY OF MEASUREMENT IN CHEMICAL TESTING 6.1 Introduction Laboratories may use various approaches to estimate the uncertainty of measurement of the results of measurements. The degree of rigour and the procedures needed in an estimation of uncertainty of measurement shall be determined by the laboratory pursuant to sub-clause 5.4.6.2, NOTE 1 of the standard. Therefore the laboratory shall take into account: the requirements and limitations of the test method; the requirements of the customer (see 4.4.1); the limit values on which decisions on conformity to a specification are based. In general, the degree of rigour needed in an estimation of uncertainty of measurement is related to the degree of risk. Should the uncertainty of measurement not be acceptable for the customer, or should it be too high for decision on conformity to the specification, the laboratory must try to reduce the uncertainty of the result of measurement by identifying the highest contributions to the uncertainty and improving these degrees during the procedure of carrying out the test method. 6.2 Procedure of estimating the uncertainty of measurement Identification of the test method's stages and the use of the Cause/Effect ("Fishbone") Diagram could provide a useful approach to estimating the uncertainty of measurement and presenting the sources and components of uncertainty of measurement. Various types of samples, matrixes and various concentration ranges of the analyte should be taken into account when estimating the uncertainty of measurement. Revision 3Release 3 Effective as from: 13.12.200628.6.2005 Page 5 of 11
Often individual stages of test methods are common to different test methods. In such cases, the estimation of uncertainty of measurement for one stage can be used for estimating combined uncertainty with all the methods in which these stages are used during the testing procedure. The steps of the procedure of estimating the uncertainty of measurement in chemical testing shall be as follows: 1 Definition of measurand. 2 Identification of sources of uncertainty of measurement. 3 Quantification of the components of uncertainty. 4 Calculation of combined and expanded uncertainty. Step 1: Definition of measurand The measurand should be clearly defined. The usual measurand in chemical analysis is the concentration of a particular analyte in the matrix. The relation between the measurand and the input quantities shall be determined (model equation). Corrections for known systematic deviations should be included, where possible. Particular attention should be paid to whether or not sampling and laboratory sample preparation are included in the customer's requirements. Step 2 Identification of sources of uncertainty of measurement The possible sources of uncertainty should be defined. Follow some examples: incomplete definition of measurand; sampling; sample transport and storage; sample preparation for analysis; measurement conditions and environmental conditions; operators; changes in the test procedure; measuring equipment; reference materials. Step 3: Quantification of the components of uncertainty The extent of the components of uncertainty related to the identified potential sources of uncertainty shall be estimated by carrying out the appropriate experiments or from other available information; the same components should, however, not be taken into account more than once. It is often possible to estimate or define a contribution to uncertainty related to several separate sources. It is also important to determine whether the available information covers all sources of uncertainty, or to plan additional experiments to provide all necessary information. Step 4: Calculation of combined and expanded uncertainty of measurement The information obtained under Step 3 above consists of a number of quantified contributions to the uncertainty of measurement related to individual sources or combined effects of several sources. The contributions shall be expressed as standard uncertainty and combined in accordance with appropriate rules (law of propagation of uncertainty) to provide the combined standard uncertainty. An Revision 3Release 3 Effective as from: 13.12.200628.6.2005 Page 6 of 11
appropriate coverage factor shall be used for the expanded uncertainty. It should be identified, during the process of identification of individual sources of uncertainty, which sources essentially contribute to the combined uncertainty of measurement. Practice will show that they are usually few. For those that are available, reliable information should be obtained. A preliminary estimation of the contribution of each component or each combination of components to the uncertainty of measurement should be made, and attention should be paid to those that are more important. The sources of uncertainty of measurement and the values of individual components shall be documented. When deciding which of the identified components of uncertainty of measurement should be included individually in the final estimation, the following should be taken into consideration: The relative extent of the maximum and minimum contribution; the components contributing less than 1/5 to 1/3 of the complete uncertainty of measurement can be estimated together (Ref. 4). The effect of reporting of the uncertainty of measurement; when considerable material or other consequences are derived from the stated uncertainty of measurement or interpretation of the results, related to it, approximate estimates of uncertainty should not be given. The degree of rigour in estimating the uncertainty of measurement,based on the requirements of the customer as well as legal and other requirements. Uncertainty also derives from assuring traceability of the results of measurements in chemical test methods, and it often has several components. The uncertainty of all the components of traceability of a measurement procedure (e.g. balances, thermometers, volumetric equipment, reference materials ) should be taken into account in the estimation of uncertainty of measurement. The information obtained through validation of the measurement procedure or that obtained through ILC can be used for estimating the uncertainty of measurement. a. Using validation data and the capacity of methods to estimate the uncertainty of measurement The characteristics of method capacity are essential in estimating the uncertainty of measurement of the results. In practice, the feasibility of a test method for the expected use is checked through its validation. The information obtained can be used for estimating the uncertainty of measurement. Validation studies usually determine some or all of the following parameters: Precision A study inside the laboratory shall give the information on precision under repeatable conditions even during a longer period of time or with different operators. The precision obtained of the test procedure is usually the essential component of the combined uncertainty of measurement. Deviation from true value The deviation is usually determined through appropriate reference materials (similar matrix and concentration range). In general, the uncertainty of measurement related to the deviation from the true value represents an important component of the combined uncertainty of measurement. Linearity The uncertainties of measurements related to the calibration curve should be taken into account. Revision 3Release 3 Effective as from: 13.12.200628.6.2005 Page 7 of 11
Limit of Quantification (LOQ) The value for LOQ is not directly related to the estimation of uncertainty of measurement; however, the uncertainty within the concentration range near LOQ is important in relation with the value of the result. Robustness The development of methods or the validation studies require research into the sensitivity of a method to changes of certain parameters. This could provide useful information on the effects of important parameters and on determining whether they could essentially affect the uncertainty of measurement. Experimental studies into the capacity of methods should be carried out at normal conditions of use of the method in laboratory. They should take into account all actual types of samples, matrixes and the concentration ranges of the analytes. An estimate of precision, which covers all variants, is especially important. In studies relating to deviation from true value the use of an appropriate reference material with regard to the matrix and concentration range is important. 6.3 Use of ILC information Studies into the capacity of test methods Interlaboratory studies into the capacity of a test method provide information on the repeatability and reproducibility of the method, and they can also provide an estimate of correctness (measured as deviation from known true value). The basic principles for the use of these data in estimating the uncertainty of measurement are: Determining the adequateness of information on the capacity of the test method for use in the laboratory (the same or similar matrix and concentration range); Determining the adequateness of information on the capacity of the test method through identification of the differences in sample handling, sampling and the procedure in the laboratory and ILC; Determining and evaluate additional sources of uncertainty of measurement, which are not comprised appropriately within ILC; Combining all important contributions to the uncertainty of measurement, including the reproducibility, the uncertainty related to the deviation of the laboratory's result from the true value, and the uncertainty deriving from additionally identified sources according to the law of propagation of uncertainty. Additional sources to be taken into account are: Sampling ILC would seldom include sampling. When the method used by the laboratory contains sub-sampling, or when the measurand is the main component of a small sample, the effects of sampling on the uncertainty of measurement should be searched and taken into account. Sample preparation Samples are often homogenized and stabilized prior to transport to the laboratory. Research should also be made into the effects of sample preparation. Revision 3Release 3 Effective as from: 13.12.200628.6.2005 Page 8 of 11
Changes in conditions Laboratories should take into account the experimental conditions defined in the method. Changes in sample type Laboratories should take into account the uncertainties that may arise from samples of a different type than those used in the comparative study. b. Using information from procedures for assuring the quality of test results Stable, typical and homogeneous samples should be used in order to assure the quality of test results. The information thus obtained in a longer period of time (control charts) is very important for estimation of the uncertainty of measurement. Standard deviation of these data represents the component of uncertainty of numerous potential sources. Such information will in general not include sub-sampling, the effects of differences in sample types and concentration ranges, or non-homogeneity of samples. The data in control charts representing outliers shall be excluded prior to calculation of standard deviation. 6.4 Using the results of proficiency testing The results of proficiency testing (PT) shall be used for periodic checking of the laboratory's competence for performing certain test methods. Interlaboratory comparisons are generally not performed often enough to be able to give a reliable assessment of a laboratory's competence for performing a test method. Types of samples would also change. In addition, many schemes would use the accepted rather than reference value to assess the laboratory's competence, which may occasionally lead to incorrect results for the laboratory. The results of a laboratory taking part in PT can also be used for verifying the value of the uncertainty of measurement defined in the laboratory. The use of PTs for estimation of uncertainty of measurement shall be limited, except when: the types of samples used in an ILC scheme are similar to the samples obtained through routine analysis, the accepted values are traceable to a suitable reference value, and the uncertainty of the accepted value is small in comparison with the actual dispersion of the results. In such cases the dispersion of differences between the reported and the accepted values in repeated proficiency tests would represent the basis for estimation of uncertainty of measurement for that part of testing which is included within the scope of the scheme. However, the necessary systematic deviation from the traceable accepted values and other sources of uncertainty of measurement should be taken into account. The information obtained through PTs can primarily be used for preliminary estimation of uncertainty of measurement. 6.5 Reporting of the results of quantitative testing Quantitative tests should give the value expressed in the SI units, if possible. When reporting the uncertainty of measurement, the expanded uncertainty of measurement (U) should be stated with the Revision 3Release 3 Effective as from: 13.12.200628.6.2005 Page 9 of 11
defined degree of confidence, or the combined standard uncertainty of measurement (u). Usually, the reference to the procedure used to estimate the uncertainty of measurement should also be stated. If all resources identified as important are not considered during the evaluation of uncertainty of measurement this should be clearly defined when reporting. The number of decimals in the stated uncertainty of measurement should reflect the practical capability of the measurement. Rarely more than two significant numbers are reported. Also the numerical value for the result should be rounded in such a way that the last decimal corresponds to the last decimal of the uncertainty of measurement. Example: When the result is 123.456, and the estimated uncertainty of measurement was 2.27, the result should be given as a rounded value 123.5±2.3. 7 REFERENCES 1. SIST EN ISO/IEC 17025: 20051999: General requirements for the competence of testing and calibration laboratories 2. International Vocabulary of Basic and General Terms in Metrology, BIPM, IEC, IFCC, ISO, IUPAC, IUPAP, OIML, 1993; Translation: SMIS, 1999. 3. Guide to the Expression of Uncertainty in Measurement,.BIPM, IEC, IFCC, ISO; IUPAC, IUPAP OIML. 1995. 4. EURACHEM/CITAC Guide, Quantifying Uncertainty in Analytical Measurement, 2000. 5. APLAC, Policy, Interpretation and Guidance on the Estimation of Uncertainty of Measurement in Testing, 2002. 6. EA-4/16: EA Guideline on the Expression of Uncertainty in Quantitative Testing, Revision 0011, January 2003. 7. EUROLAB TC QA Draft 6, Measurement Uncertainty in Testing, 2002 8. A2LA Guide for the Estimation of Measurement Uncertainty in Testing, 2002. 9. Doc. PLG/LAB (==) 63 Rev.1, Strategy to Introduce the Concept of Measurement Uncertainty in Testing in Connection with the Introduction of the Standard ISO/IEC 17025. 10. J. Fleming, H. Albus, B. Neidhart, W. Wegscheider, Glossary of analytical terms VIII, Accred Qual Assur (1997) 2, 160-161. 11. SIST ISO 3534-1: 1996: Statistics, Vocabulary, Symbols Part 1: Probability and General Statistical Terms. 12. E. Prichard, H. Albus, B. Neidhart, W. Wegscheider, Glossary of analytical terms IX, Accred Qual Assur (1997) 2, 348-349. 13. Terminološki slovarček izrazov s področja analizne kemije, MIRS&EURACHEM Slovenija, 2002, www.mirs.si. 14. ISO 9000:2000. 15. E. Prichard, H. Albus, B. Neidhart, W. Wegscheider, Glossary of analytical terms X, Accred Qual Assur (1998) 3, 171-173. 16. ISO/IEC Guide 43: 1997. 8 DISTRIBUTION SLIP SA's Website SA's employees Revision 3Release 3 Effective as from: 13.12.200628.6.2005 Page 10 of 11
ANNEX 1 List of literature with examples of calculations of uncertainty of measurement 1. Quantifying Uncertainty in Analytical Measurement EURACHEM, CITAC (http://www.eurachem.ul.pt/) 2. Barwick V.J., Ellison SLR (1998) VAM Project 3.2.1: Development and Harmonisation of Measurement Uncertainty Principles, Part (d): Protocol for uncertainty evaluation from validation data, LGC/VAM, Teddington (www.vam.org.uk) 3. Nordest report: Handbook for calculation of measurement uncertainty in environmental laboratories, 2004 (www.nordest.org) 4. An extensive list of literature for various fields of testing can be found in the Annex to the document EA/4-16 (http://www.european-accreditation.org/). Revision 3Release 3 Effective as from: 13.12.200628.6.2005 Page 11 of 11