Establishing traceability and estimating measurement uncertainty in physical, chemical and biological measurements Experimental Design 8.2.2013 Doc. Martti Heinonen etunimi.sukunimi@mikes.fi
Outline 1. Establishing traceability in measurements 2. Estimating uncertainty 2.1 Objectives and level of modelling 2.2 What should be covered by the analysis 3. Physical measurements - example 4. Chemical and biological measurements 4.1 Sampling 4.2 Example 2
1 Establishing traceability
Traceability Tower D U Measurement D U D U Calibration standard D U Reference standard D U Secondary standard D U Primary standard K C K C Measuring instrument SI System of Units K C K C K C K C Measurement uncertainty The tower is collapsed if any part of it is missing or incomplete i.e. there is no traceability unless all levels include all the characteristics of traceability At any level the measurement uncertainty can t be smaller than levels below. 4
Characteristics of Unbroken Traceability For each calibration of the chain: Uncertainty estimation Documented and generally acknowledged procedures, documented results Competence Calibration is valid for the application. (interval of calibrations, conditions etc.) P U V C SI System of Units 5
2 Estimating uncertainty 2.1 Objectives and level of modeling 2.2 What should be covered by the analysis? 2.3 Benefits from uncertainty analysis
6 steps to evaluating uncertainty 1) Measurement model: List essential input quantities (i.e. parameters x i having a significant effect on the result) and build up a mathematical model (function) showing how they are related to the final result: y = f(x 1,,, x i ) 2) Standard uncertainty: Estimate the standard uncertainty of each input quantity (x i ) 3) Use the model in uncertainty calculations: Determine the uncertainty due to standard uncertainty of each input quantity (x i ): u i (y) = c i u(x i ) 4) Correlation: Determine correlation between the input quantities (if relevant). 5) Calculate the combined standard uncertainty 6) Calculate the expanded uncertainty. 7
2.1 Objectives and level of modelling
Objectives of modelling to describe how the measurement result is calculated from input data (incl. measurement values, the data often include information from earlier measurements, specifications, calibration certificates etc. ) to show how various factors affect the result to provide a tool for calculating the estimate and the uncertainty 0.30 t x = tind + δ tcal + δtd + δtresol + δtg t x = (21,6 ± 0,3) C Ref - Ind ( C) 0.20 0.10 0.00-0.10-0.20 2005 2006 2007 2008-0.30 0 10 20 30 40 50 60 Temperature ( C) 9
Level of modelling The model is never complete; approximations are needed: high accuracy more details in the model low accuracy rough approximations in the model A measurement model is never identical with a presentation of a physical phenomenon: All data used in the calculation of the estimate and the uncertainty should have corresponding input quantities in the model All input quantities of the model should be connected to data actually used in the calculations 10
Source of data: t x = t ind + δ t + δt + δt + δt Cal D resol G display calibration certificate long-term instability monitoring display earlier measurements (validated specifications) 11
Completeness of the model: RH ( p c ce w dind dc dd dr wg = 100 % rh ( p + δp d + δp ) e de ( t ) e w ( t Ind + δt + δt C + δt + δt D + δt + δt R ) + δe + δt G ) + δrh Calc S1 ews ( t) = 1 Pa exp[ + S S (/ t C, ) S (/ t C, ) t / C, 2 + 3 + 273 15 + 4 + 273 15 + 273 15 + S ln( t/ C+ 273, 15)] 5 2 RH = RH ind + δ RH + δrh C ni 12
2.2 What should be covered by the analysis?
Source of data: t x = t ind + δ t + δt + δt + δt Cal D resol G display calibration certificate long-term instability monitoring display earlier measurements (validated specifications) 14
Issues to be considered Measurement target/object Measurement method Measurement device / equipment Measurement conditions / environment Measurer 15
Issues to be considered Measurement target/object examples: gradients homogeneity disturbance due to the measurement temporal variations 16
Issues to be considered Measurement method examples: reproducibility variations affecting the target and/or the equipment representativity (number of measurements, measurement time, sampling frequency etc.) errors in sampling thermoelectric effects, non-ideal contacts, trace gases in tubing, contaminations, etc. 17
Issues to be considered Measurement device / equipment examples: calibration & reference standard drift (zero & full scale) resolution sensitivity non-linearity interaction of different parameters 18
Issues to be considered Measurement conditions / environment examples: ambient temperature ambient humidity ambient air velocity vibrations electric noise lighting magnetic field 19
Issues to be considered Measurer examples: skills: ability to obtain repeatable & reproducible & comparable results heat and moisture contamination 20
2.3 Benefits from uncertainty analysis
Improving quality Modelling = analysis of measurement Mathematical model shows explicitly how different factors affect the result and measurement process Uncertainty analysis (including the modelling) brings out weak points factors in which efforts should be focused to get most effective improvement in quality Benefitting in risk analysis 22
Innovating and planning By modelling you can test different methods before setting up an actual measurement system efficient and low-cost You can study the effects of both methods and devices on the measurement quality on the basis of existing data. You can judge if a new approach has potential to achieve the required accuracy level. 23
3 Physical measurements - Example
Example 3.1: Fluid temperature in a pipe 131.481 Ω Measurement set-up: -Fluid: water -Pt-100 thermometer -immersed 5 cm -in a thermometer well -DMM for resistance measurements 25
Example 3.1 continuing: 131.481 Ω Step 1: Measurement model - How we get temperature from the resistance? Calibration equation - What are the input quantities? - What is the measurement model? 26
Example 3.1 Input quantities Measurement target/object Error due to temperature gradients in the pipe δt G Fluid: Water & liquid in the thermometer well temperature difference between the fluid and the tip of the thermometer is negligible error due to different time constants is negligible Measurement method Thermometer is immersed only partly error due to heat conduction along the probe δt cond (disturbance due to the thermometer is negligible) Error in resistance measurement (wires etc.) δr ni 27
Example 3.1 Input quantities Measurement device / equipment Errors in the calibration equation of the Pt-100: δt cal Drift of the Pt-100 since the last calibration: δt drift Reading of the DMM: I R Calibration correction of the DMM: δr cal Drift of the DMM since the last calibration: δr drift Error due to non-ideal resolution of the DMM: δr resol Error due to non-linearity of the DMM: δr nl 28
Example 3.1 Input quantities Measurement conditions / environment Effect environmental conditions on the DMM: negligible Effect environmental conditions on the PRT & wiring: included in δt cond and δr ni Measurer: permanent set-up & automatic measurement the effect of a measurer is negligible 29
Example 3.1 Measurement model t 4 = a i= 0 i ( I R + i δ Rcal + δrdrift + δrresol + δrnl + δrni ) + δtcal + δtcond + δtdrift + δt G 30
Example 3.1 continuing: 131.481 Ω Step 2: Standard uncertainty of the input quantities - How we estimate? 31
Example 3.1 continuing: 4 i t = ai ( I R + δ Rcal + δrdrift + δrresol + δrnl + δrni ) + δtcal + δtcond + δtdrift + δt i= 0 G Quantity, X i Estimate, x i Standard unc., u (x i ) Probability Notes on the determination i Description Value unit Value unit distribution of the standard uncertainty reading of 1 DMM 131.540 Ohm 0.0187 Ohm normal display calibration 2 equation 0 C 0.0075 C normal taken from the calibration certificate calibration of 3 the DMM 0.057 Ohm 0.0005 Ohm normal taken from the calibration certificate drift of the 4 Pt100 0 C 0.0071 C rectangular comparing last two calibration results drift of the 5 DMM 0 ohm 0.0023 Ohm rectangular comparing last two calibration results resolution of 6 the DMM 0 ohm 0.0003 Ohm rectangular display temp. 7 gradients 0 C 0.1155 C rectangular from specification (or thermal modelling) earlier measurement data, e.g. with calibration heat flow along 8 the Pt100 0.22 C 0.0685 C rectangular recording 9 method 0 Ohm 0.0000 Ohm validation results non-linearity of the DMM 0 C 0.0040 Ohm rectangular 10 calculated from the calibration certificate 32
Example 3.1 continuing: 131.481 Ω Step 3: Effect on the combined uncertainty 33
Example 3.1 continuing: 4 i t = ai ( I R + δ Rcal + δrdrift + δrresol + δrnl + δrni ) + δtcal + δtcond + δtdrift + δt i= 0 G Quantity, X i Estimate, x i Standard unc., u (x i ) Probability Sensitivity coeff., c i Notes on the determination i Description Value unit Value unit distribution Value unit of the standard uncertainty reading of 1 DMM 131.540 Ohm 0.0187 Ohm normal 2.622 C/Ohm display calibration 2 equation 0 C 0.0075 C normal 1 taken from the calibration certificate calibration of 3 the DMM 0.057 Ohm 0.0005 Ohm normal 2.622 C/Ohm taken from the calibration certificate drift of the 4 Pt100 0 C 0.0071 C rectangular 1 comparing last two calibration results drift of the 5 DMM 0 ohm 0.0023 Ohm rectangular 2.622 C/Ohm comparing last two calibration results resolution of 6 the DMM 0 ohm 0.0003 Ohm rectangular 2.622 C/Ohm display temp. 7 gradients 0 C 0.1155 C rectangular 1 from specification (or thermal modelling) earlier measurement data, e.g. with calibration heat flow along 8 the Pt100 0.22 C 0.0685 C rectangular 1 recording 9 method 0 Ohm 0.0000 Ohm 2.622 C/Ohm validation results non-linearity of the DMM 0 C 0.0040 Ohm rectangular 1 10 calculated from the calibration certificate 34
Example 3.1 continuing: 131.481 Ω Step 4: Correlations: - Input quantities can be considered independent on each other 35
Example 3.1 continuing: 131.481 Ω Step 5: Combined standard uncertainty Step6: Expanded uncertainty 36
Example 3.1 continuing: 4 i t = ai ( I R + δ Rcal + δrdrift + δrresol + δrnl + δrni ) + δtcal + δtcond + δtdrift + δt i= 0 G Quantity, X i Estimate, x i Standard unc., u (x i ) Probability Sensitivity coeff., c i Uncertainty contribution, u i Notes on the determination i Description Value unit Value unit distribution Value unit Value unit of the standard uncertainty reading of 1 DMM 131.540 Ohm 0.0187 Ohm normal 2.622 C/Ohm 0.049 C display calibration 2 equation 0 C 0.0075 C normal 1 0.008 C taken from the calibration certificate calibration of 3 the DMM 0.057 Ohm 0.0005 Ohm normal 2.622 C/Ohm 0.001 C taken from the calibration certificate drift of the 4 Pt100 0 C 0.0071 C rectangular 1 0.007 C comparing last two calibration results drift of the 5 DMM 0 ohm 0.0023 Ohm rectangular 2.622 C/Ohm 0.006 C comparing last two calibration results resolution of 6 the DMM 0 ohm 0.0003 Ohm rectangular 2.622 C/Ohm 0.001 C display temp. 7 gradients 0 C 0.1155 C rectangular 1 0.115 C from specification (or thermal modelling) heat flow along 8 the Pt100 0.22 C 0.0685 C rectangular 1 0.069 C recording 9 method 0 Ohm 0.0000 Ohm 2.622 C/Ohm 0.000 C validation results non-linearity of the DMM 0 C 0.0040 Ohm rectangular 1 0.004 C 10 combined standard uncertainty: 0.144 C Estimate: 82.13 C Expanded uncertainty: 0.287 C earlier measurement data, e.g. with calibration calculated from the calibration certificate Measurement r 82.1 C ± 0.3 C 37
Example 3.1 continuing: 131.481 Ω Measurement result: (82.0 ± 0.3) C 38
Example 3.1 continuing: 131.481 Ω Conclusions on factors affecting the result: - temperature gradients dominate the uncertainty 39
4 Chemical and biological measurements 4.1 Sampling 4.2 Example
4.1 Sampling
Objectives of sampling The objective is to get reliable information on the whole target of interest Most significant error sources: taking samples + handling and analysis of the samples Often several samples are needed to be taken in steps Sampling is often the dominating uncertainty component 42
Methods of sampling Sampling methods: random samples systematic samples (periodicity) representative sample combined sample divided sample (sample is too large for the analysis) layered sample 43
Sampling plan The sampling plan should state: size and number of samples locations, times/dates, sampling method handling of samples sample container; cleaning, closing, storage labelling and records requirements for the analysis environmental conditions during sampling other requirements (e.g. authorized persons) 44
Sources of uncertainty in sampling inhomogeneity of the material shape of particles instability of the material in time layers in the material weighting error number of samples errors in handling the samples errors in the analysis 45
Uncertainty of sampling Deviation of samples When analysed n b pieces from total number N of pieces and from each piece we take n w samples and we carry out n a analysis with each samples, then the standard deviation of the mean is: 2 N 1 σ = σ b + n n n ( nb ) 2 1 2 + σ w σ a nb N nbnw b w a σ w = deviation of analysed samples σ b = deviation of pieces σ a = deviation of analysis n t = total number of analyses =n b n w n a for homogenious material: σ w = 0 if all samples are analysed: N = n b 46
Further information on sampling Minimum number of sampling σ s = standard deviation R = permitted maximum error t = student-t factor n = 2 t σ R 2 s 2 Literature EN ISO/IEC 17025:2005, General requirements for the competence of testing and calibration laboratories. Sections 5.2.5, 5.7 Measurement uncertainty arising from sampling: A guide to methods and approaches, EURACHEM / CITAC Guide (2007)1st edition (can be downloaded at www.eurachem.org/guides) ISO 15189/2003: Medical laboratories -- Particular requirements for quality and competence ISO 3534-1,2(1993): Statistics 47
4.2 Example
Preparation of a Cd calibration standard for AAS (Atomic absorption spectroscopy) 1) Description of the measurement: Metal oxide contamination is removed with an acid mixture treatment. A volumetric flask of 100 ml is weighed with and without the purified metal inside. 1 ml of nitric acid (65 %m/m) and 3 ml ion-free water are added to the flask to dissolve the cadmium. Afterwards the flask is filled with ionfree water up to the mark and mixed by inverting the flask at least thirty times. The concentration is: c Cd = m P V m = mass of metal P = purity of metal, i.e. mass fraction V = volume of the liquid of the calibration standard 1) Based on the Example A1 presented in [1] 49
Preparation of a Cd calibration standard - continuing Step 1: Measurement model Supplier s certification Repeatability (dissolve efficiency) 50
Preparation of a Cd calibration standard - continuing Step 1: Measurement model Supplier s certification Repeatability (dissolve efficiency) c Cd = m P V = [( m Igross m Itare ) + δm V cal lin + δv + δm temp rep + δv + δm filling resol ] ( P + δp ) certif eff 51
Preparation of a Cd calibration standard - continuing Step 1: Measurement model c Cd = m P V [( m = Igross m Itare ) + δm m Igross, m Itare = balance readings with and without the metal δm lin = correction due to non-linearity of the balance δm rep, δm resol = correction due to non-ideal repeatability and resolution of the balance P = purity of the metal according to the supplier s certificate δp = correction due to the non-ideal dissolving V cal = the inner volume of the flask according to its calibration certificate δv temp = thermal expansion of the measured volume cal δv tilling = error in filling water up to the mark V lin + δv + δm temp rep + δv + δm filling resol ] ( P + δp ) certif eff 52
Preparation of a Cd calibration standard - continuing Step 2: Standard uncertainty of the input quantities Quantity, X i Estimate, x i Standard unc., u (x i ) Probability Notes on the determination i Description Value unit Value unit distribution of the standard uncertainty balance readings with the standard deviation of recorded 1 metal 100.28 mg 0.030 mg normal readings balance readings without 2 the metal 0 mg 0.010 mg normal standard deviation of recorded readings after taring non-linearity of the 3 balance 0 mg 0.0173 mg rectangular According to calibration, the nonlinearity is less than ±0.03 mg non-ideal repeatability of 4 the balance 0 mg 0.0115 mg rectangular According to repeated measurements in calibration of the balance, the repeatability is less than ±0.02 mg non-ideal resolution of the 5 balance 0 mg 0.0058 mg rectangular 6 purity of the metal 0.9999 0.00006 rectangular Manufacturer's certificate states that the purity of the metal is 0.9999 ± 0.0001 7 non-ideal dissolving 0 0.0001 rectangular According to repeated preparations the efficiency is larger than 99.98 % 8 inner volume of the flask 100 ml 0.05 ml normal 9 thermal expansion of the measured volume 0 ml 0.048 ml rectangular 10 filling error 0 ml 0.0200 ml normal The flask was calibrated by weighing with distilled water. The result was (100 ± 0.1) ml with k =2 Because the thermal expansion of glass is much smaller than water, only the water (2.1 x 10^-4 1/ C) is taken into account. Temperature variation is ± 4 C at maximum. standard deviation from ten fillings and weighings 53
Preparation of a Cd calibration standard - continuing Step 3: Effect on the combined uncertainty Sensitivity coefficients: For m Igross, m Itare δm lin, δm rep and δm resol : c 1 = c m Cd = c m Cd For P and δp : For V cal, δv temp and δv tilling : c c 3 2 = = c P c V Cd = Cd = c P Cd c V Cd 54
Preparation of a Cd calibration standard - continuing Step 3: Effect on the combined uncertainty Quantity, X i Estimate, x i Standard unc., u (x i ) Probability Sensitivity coeff., c i Notes on the determination i Description Value unit Value unit distribution Value unit of the standard uncertainty balance readings with the standard deviation of recorded 1 metal 100.28 mg 0.030 mg normal 9.999 1/l readings balance readings without 2 the metal 0 mg 0.010 mg normal 9.999 1/l standard deviation of recorded readings after taring non-linearity of the 3 balance 0 mg 0.0173 mg rectangular 9.999 1/l According to calibration, the nonlinearity is less than ±0.03 mg non-ideal repeatability of 4 the balance 0 mg 0.0115 mg rectangular 9.999 1/l According to repeated measurements in calibration of the balance, the repeatability is less than ±0.02 mg non-ideal resolution of the 5 balance 0 mg 0.0058 mg rectangular 9.999 1/l 6 purity of the metal 0.9999 0.00006 rectangular 1002.8 mg/l Manufacturer's certificate states that the purity of the metal is 0.9999 ± 0.0001 7 non-ideal dissolving 0 0.0001 rectangular 1002.8 mg/l According to repeated preparations the efficiency is larger than 99.98 % 8 inner volume of the flask 100 ml 0.05 ml normal -10.03 g/l^2 9 thermal expansion of the measured volume 0 ml 0.048 ml rectangular -10.03 g/l^2 10 filling error 0 ml 0.0200 ml normal -10.03 g/l^2 The flask was calibrated by weighing with distilled water. The result was (100 ± 0.1) ml with k =2 Because the thermal expansion of glass is much smaller than water, only the water (2.1 x 10^-4 1/ C) is taken into account. Temperature variation is ± 4 C at maximum. standard deviation from ten fillings and weighings 55
Preparation of a Cd calibration standard - continuing Step 4: Correlations: Input quantities can be considered independent on each other c Cd = m P V [( m = Igross m Itare ) + δm V cal lin + δv + δm temp rep + δv + δm filling resol ] ( P + δp ) certif eff 56
Preparation of a Cd calibration standard - continuing Step 5 & 6: Combined standard and expanded uncertainty Quantity, X i Estimate, x i Standard unc., u (x i ) Probability Sensitivity coeff., c i Uncertainty contribution, u i Notes on the determination i Description Value unit Value unit distribution Value unit Value unit of the standard uncertainty balance readings with the standard deviation of recorded 1 metal 100.28 mg 0.030 mg normal 9.999 1/l 0.300 mg/l readings balance readings without 2 the metal 0 mg 0.010 mg normal 9.999 1/l 0.100 mg/l standard deviation of recorded readings after taring non-linearity of the 3 balance 0 mg 0.0173 mg rectangular 9.999 1/l 0.173 mg/l According to calibration, the nonlinearity is less than ±0.03 mg non-ideal repeatability of 4 the balance 0 mg 0.0115 mg rectangular 9.999 1/l 0.115 mg/l According to repeated measurements in calibration of the balance, the repeatability is less than ±0.02 mg non-ideal resolution of the 5 balance 0 mg 0.0058 mg rectangular 9.999 1/l 0.058 mg/l 6 purity of the metal 0.9999 0.00006 rectangular 1002.8 mg/l 0.058 mg/l Manufacturer's certificate states that the purity of the metal is 0.9999 ± 0.0001 7 non-ideal dissolving 0 0.0001 rectangular 1002.8 mg/l 0.116 mg/l According to repeated preparations the efficiency is larger than 99.98 % 8 inner volume of the flask 100 ml 0.05 ml normal -10.03 g/l^2-0.501 mg/l 9 thermal expansion of the measured volume 0 ml 0.048 ml rectangular -10.03 g/l^2-0.486 mg/l 10 filling error 0 ml 0.0200 ml normal -10.03 g/l^2-0.201 mg/l combined standard uncertainty: 0.83 mg/l Estimate: 1002.70 mg/l Expanded uncertainty: 1.66 mg/l The flask was calibrated by weighing with distilled water. The result was (100 ± 0.1) ml with k =2 Because the thermal expansion of glass is much smaller than water, only the water (2.1 x 10^-4 1/ C) is taken into account. Temperature variation is ± 4 C at maximum. standard deviation from ten fillings and weighings 57
Preparation of a Cd calibration standard - continuing Thus, the Cd content of the calibration standard prepared for an AAS was: (1003 ± 2) mg/l 58
References and literature [1] EURACHEM/CITAC Guide CG 4, Quantifying Uncertainty in Analytical measurement (http://www.measurementuncertainty.org/mu/quam2000-1.pdf) LITERATURE ISO/IEC Guide 99-12:2007, International Vocabulary of Metrology Basic and General Concepts and Associated Terms, VIM International vocabulary of metrology Basic and general concepts and associated terms (VIM), 3rd ed., JCGM 200:2008 (can be downloaded at http://www.bipm.org/en/publications/guides/) Metrology - in short, 3rd edition, EURAMET 2008, 84 p. (www.euramet.org) JCGM 100:2008, Evaluation of measurement data Guide to the expression of uncertainty in measurement, First edition, JCGM 2008 (http://www.bipm.org/en/publications/guides/) European cooperation for Accreditation, EA-4/02 Expression of the Uncertainty of Measurement in Calibration, December 1999. (http://www.european-accreditation.org/n1/doc/ea-4-02.pdf) UKAS M3003, The Expression of Uncertainty and Confidence in Measurement S. A. Bell, A beginner's guide to uncertainty in measurement, Measurement Good Practice Guide No. 11, (Issue 2), National Physical Laboratory 2001, 41 p. (www.npl.co.uk) MIKES has published several guides in Finnish on the uncertainty estimations in different fields. 59