MEASUREMENT PREPARED FOR ENAO ASSESSOR CALIBRATION COURSE UNCERTAINTY OCTOBER/NOVEMBER 2012 Prepared by for ENAO Assessor Calibration B
SCOPE Introduction House Rules Overview 17025 & 15189 MU Approaches Terminology Uncertainty Typical Approach for Test Labs Caution Distribution Functions Scenario Test Model
SCOPE Type A & Type B Coverage Type B Example for MU of RM MU Process Reproducibility data Sources Bias data sources Nordtest T537 Examples 1-4 Summary
MU: ISO/IEC 17025 5.4.6.1 A calibration laboratory, or a testing laboratory performing its own calibrations, shall have and shall apply a procedure to estimate the uncertainty of measurement for all calibrations and types of calibrations. 5.4.6.2 Testing laboratories shall have and shall apply procedures for estimating uncertainty of measurement. In certain cases the nature of the test method may preclude rigorous, metrologically and statistically valid, calculation of uncertainty measurement. In these cases the laboratory shall at least attempt to identify all the components of uncertainty and make a reasonable estimation, and shall ensure that the form of reporting of the result does not give a wrong impression of the uncertainty. Note 1: The degree of rigour needed in an estimation of uncertainty of measurement depends on factors such as: The requirements of the test method; The requirements of the customer; The existence of narrow limits on which decisions on conformity to a specification are based
MU: ISO/IEC 17025 Note 2: In those cases where a well-recognized test method specifies limits to the values of the major sources of uncertainty of measurement and specifies the form of presentation of calculated results, the laboratory is considered to have satisfied this clause by following the test method and reporting instructions. 5.4.6.3 When estimating the uncertainty of measurement, all uncertainty components which are of importance in the given situation shall be taken into account using appropriate methods of analysis. Note 1: Sources contributing to the uncertainty include, but are not limited to, the reference standards and reference materials used, methods and equipment used, environmental conditions, properties and condition of the item being tested or calibrated, and the operator. Note 2: The predicted long-term behaviour of the tested and/or calibrated item is not normally taken into account when estimating measurement uncertainty. Note 3: For further information, see ISO 5725 and the Guide to the Expression of Uncertainty in Measurement (GUM).
MU: ISO 15189 5.6.2 The laboratory shall determine the uncertainty of results, where relevant and possible. Uncertainty components which are of importance shall be taken into account. Sources that contribute to uncertainty may include sampling, sample preparation, sample portion selection, calibrators, reference materials, input quantities, equipment used, environmental conditions, condition of the sample and changes of operator.
MU APPROACHES Type A (Top Down): Evaluation of components using statistical probability distributions of the results of a series of measurements and can be characterized by standard deviations of the respective distributions Type B (Bottom Up): Evaluation of components, and characterizing as standard deviations, by estimating their assumed probability distributions using: Previous measurement data; Experience with or general knowledge of the behaviour and properties of relevant materials or instruments; Manufacturer s specifications; Data provided in calibration and other certificates; Uncertainties assigned to reference data taken from handbooks.
TERMINOLOGY Component: Each of the separate contributions to uncertainty is referred to as an uncertainty component. Standard Uncertainty u(x i ) : When expressed as a standard deviation, an uncertainty component is known as a standard uncertainty. Combined Standard Uncertainty u c (y): Square root of the sum of he squares of all the standard uncertainties. Expanded Uncertainty (U): The multiple of u c (y) and a coverage factor, k, which is approx equal to the Z value.
TYPICAL APPROACHES Usually comes down to a combination of both Type A & B approaches. Precision and bias are the main components of MU and can easily be calculated using the validation data for both parameters (Type A). However Type B approach needs to be applied for: A) Preparation of any reference materials used during the validation; B) Any dilution or concentration stages that may be used for RM and samples
CAUTION Always keep sight of the objective and that is to obtain a realistic and robust estimation of the measurement uncertainty for a specific result. Can become quite academic if too much detail is evaluated. In practice it is more usual in test laboratories to consider uncertainties associated with overall method performance in relation to precision and bias with respect to testing of samples and reference materials. Other possible contributors to the MU should be evaluated for significance.
TYPE B APPROACH Estimate all the contributing components to the measurement uncertainty Convert them to relative standard uncertainties Combine the relative standard uncertainties as the square root of the sum of the squares of the relative standard uncertainties Determine the expanded uncertainty using the relevant coverage factor.
! TYPE B APPROACH: SCENARIO TEST MODEL +,-!3)0! 4555!678-! 455!678-! 3,6&-(82(=! 3),>0,/0! 45!678-! 34!!!39!!!!3:!!!!!3;!!!!!3<! 2(,7(>)'!!! "#$%&6(>)!?/(&,/(0@! 0%-$)(0@!A/! BA>B(>)/,)(0! "#$%&!'()* $&! +,-%./,)(0! (#$%&! 1(')! 2('$-)! C+!3)0! DBBA6!E! ">F%/A>!?(/'A>>(-!
TYPE B BUDGET
EXAMPLE: TYPE B DETERMINATION OF MU FOR THE REF MATERIAL
DISTRIBUTION FUNCTIONS
TYPE B EVALUATION OF REF MATERIAL Uncertainty Budget for a Ref Material:
TYPE A APPROACH MU DETERMINATION PROCESS Specify Measurand Quantify within laboratory Reproducibility Quantify Bias Convert components to standard uncertainties of all the same units or relative standard uncertainties. Calculate combined standard uncertainty Calculate expanded uncertainty
TYPE A APPROACH Combined Standard Uncertainty: Expanded Uncertainty: at a 95% confidence level.
WHAT S COVERED BY TYPE A? All components, except the reference material, are included in a Type A approach where the uncertainties associated with the Reproducibility and the Bias are evaluated using statistical data (usually from validation or verification trials). Precision and bias studies take into account the influence of equipment set-up, calibration, QC, environmental factors and personnel. The only external component that needs to be additionally included is the uncertainty associated with the Reference Material. Note: Discuss Medical & Empirical methods
WITHIN-LAB REPRODUCIBILITY QC chart history stable standards over prolonged period of time covering the working range of the method. Validation/verification data for within-lab reproducibility. PT performance over a number if studies.
BIAS QC chart history CRMs Spiked or fortified samples Standard Addition PT Studies/Inter-laboratory studies Validation/Verification studies.
NORDTEST REPORT T537
NORDTEST REPORT T537
EXAMPLE 1
EXAMPLE 1 (CONTD)
EXAMPLE 2
EXAMPLE 3 In this example, the u(rw) is estimated from a quality control sample and the u(bias) is estimated from two different sources: in the first example the use of a CRM and in the second example participation in interlaboratory comparisons. In the summary table both ways of calculating the u(bias) will be compared. For this analysis, the sample-work up is a major error source (both for random and systematic errors), and it is thus crucial that this step is included in the calculations. The number of interlaboratory comparisons is too few to get a good estimate.
EX 3: PCB WITH INTERNAL QC + A CRM 1) Specify Measurand: Sum of 7 PCBs in sediment by extraction and GC-MS. 2) Quantify u R : The control sample which is a CRM gives an s R = 8% at a level of 150 ug/kg dry matter. 3) Quantify method and lab bias: The CRM is certified to 152 ± 14 ug/kg. The mean result on the QC chart is 144, so there is a bias of 5,3%. The s bias = 8% (n=22).
EX 3: PCB WITH INTERNAL QC + A CRM
EX 4: PCB WITH INTERNAL QC + PT RESULTS 1) Specify Measurand: Sum of 7 PCBs in sediment by extraction and GC-MS. 2) Quantify u R : The control sample which is a stable in-house material gives a s R = 8% at a level of 150 ug/kg dry matter. 3) Quantify method and lab bias: Participation in 3 PT studies with conc similar to QC level. The bias of the lab in the 3 studies are -2%; -12%, -5%. RMS bias = 7,6%. The s R in the 3 studies are: 12%, 10% and 11 %, on average s R = 11% (n=14). So, U c(ref) = 11/ 14 = 2,9%.
EX 4: PCB WITH INTERNAL QC + PT RESULTS
SUMMARY Use existing data from validation/verification; PT studies; QC, etc. Cover the working range of the test. Check assumptions of level of significance of components. Cross-check MU results using different methodologies, where relevant, practical and possible.