Measurement Uncertainty: A practical guide to understanding what your results really mean.

Measurement Uncertainty: A practical guide to understanding what your results really mean.

Overview General Factors Influencing Data Variability Measurement Uncertainty as an Indicator of Data Variability How is Data Variability Minimized in the Laboratory? Analytical Methods Laboratory QA/QC Data Reporting Evaluating Your Field QA/QC Samples 2

Factors Affecting Data Variability Field Activities Laboratory Activities Sampling Sub-sampling Extraction Analysis Field Variability Measurement Uncertainty Data Variability 3

Impact on Data Variability Influence on Data Variability Field Sampling Laboratory Subsampling Sample Extraction Extract Clean-up Instrumental Analysis Activity 4

The Laboratory

Definition Measurement Uncertainty A parameter associated with the result of the measurement, that characterizes the dispersion of the values that could be reasonably attributed to the measurand.* or It s a value that gives an idea of variability within a set of measurements that is specific to a sample or group of samples or (in English ) a ± value specific to the result * ISO Guide to the Expression of Uncertainty in Measurement 6

Approaches to Estimating MU Simple (but with limitations) where, σ = standard deviation σ x factor More Robust U c = MBlkLT + k * [ {s 02 + ( c) 2 }] where, U c = expanded uncertainty at concentration c MBlkLT = mean positive long term method blank value k = coverage factor (dependent on no. of data point) S 0 = standard deviation at zero concentration = combined relative standard uncertainty C = measured concentration of analyte in the sample 7

Measurement Uncertainty (MU) Note The ability to evaluate and calculate measurement uncertainty is a requirement of ISO Standard 17025 Caveat: MU reported by the laboratory represents the variability resulting from the lab testing only. 8

Possible Sources of MU in the Laboratory Source Impact Sample Homogenization Representative sub-sampling Reagent purity Poor reproducibility Poor reproducibility Limited data representativeness Potential contamination or background interference Sample matrix effects Instrument effects Instrument maintenance Calibration (e.g. solvent standards vs matrix matched standards) Instrumentation used (e.g. ECD vs MS) Computational effects (e.g. how many and which Aroclors were used to calculate total ) Interference (e.g. low bias in PCB analyses due to oil) Raised detection limits due to dilution Data bias if instrumentation not optimized Low bias if solvent-based standards used in calibration Data variability Data variability 9

Interpretation of Low Level Data 800.0% Benzene in Soil by GC/MS As data approach the limit of detection, variability increases 600.0% 400.0% 200.0% MDL soil = 0.003 ug/g MU 95% = 234% Conc. = 0.05 ug/g MU 95% = 43% Variability in analytical results increases as concentrations approach the limits of detection or the upper end of the system s linear range. 0.0% MDL (LOD) ~ 3 x S.D. -200.0% RDL (LOQ) ~ 3 x LOD or ~ 10 x S.D -400.0% -600.0% -800.0% RDL soil = 0.005 ug/g MU 95% = 144% What this means is that the level of confidence in the value reported at or above the RDL (LOQ) is much higher than the level of confidence reported at the MDL (LOD) 10

Benzene Laboratory MU MDL RDL Low Medium High Concentration (ug/l) 0.2 0.4 4.0 40 400 Water MU (%) 120% 64% 27% 26% 26% Range (ug/l) "0.0" - 0.4 0.1-0.7 2.9-5.1 30-50 290-510 Concentration (ug/g) 0.003 0.005 0.050 0.50 5.0 Soil MU (%) 233% 144% 42% 39% 39% Range (ug/g) "0.000" - 0.010 "0.000" - 0.012 0.029-0.071 0.30-0.70 3.0-7.0 11

Minimizing MU: Analytical Methods All lab methods are well documented in Standard Operating Procedures (SOP) SOPs are based on recognized methods developed by APHA/AWWA/WEF, USEPA, MOE or Environment Canada All methods used are validated and verified to meet or exceed the performance specifications of the recognized method 12

Minimizing MU: Quality Management System QC Sample Definition Metric Laboratory Reagent Blank Laboratory Control (Spike) Sample Laboratory Duplicate (Split) Sample Surrogate Spike Matrix Spike Laboratory reagent water put through the entire analytical process; measure of contamination from the lab Reagent water or clean reference soil/oil spiked with known amount of target analyte; measure of extraction/analytical efficiency Field sample split at the laboratory and processed as two unique samples; measure of analytical reproducibility Representative analyte (not naturally occurring) that has been added to processed sample, prior to instrumental analysis; Actual sample (expected to be clean) spiked with known amount of target analyte Absolute concentration of analyte % Recovery of analyte Relative Percent Difference (RPD) % Recovery of surrogate compound % Recovery of analyte 13

Data Reporting: Significant Figures What are significant figures? Significant figures are the digits in a number (e.g. a test result) that are meaningful These numbers typically provide the data user an indication of the precision of a measurement It is important to understand, when rounding a value that important information is not lost (over rounding) nor is the precision of a value overstated (too many significant figures). Too few significant digits cause information to be lost and perhaps, limit the utility of a result, and too many significant digits is considered bad style in numerical reporting showing a lack of understanding of precision. 14

Data Evaluation In Practice: Evaluating Your Field QA/QC Samples

Importance of Field QA/QC Samples Duplicate samples are an excellent means to evaluate sampling procedures as well as analytical precision (Zeiner 1994) and the heterogeneity of the sample matrix (AEP 2016). ASTM (D7069-2015): field duplicates every ten samples CCME (2016): 1 in 10 samples for field duplicates Nielsen and Nielsen (2005): one for every ten sampling points 16

So you have QA/QC sample results: 1. What do those results mean? 2. Is the variability acceptable? 17

Burden is placed on the practitioner: Quantifying acceptable precision [variability] is a matter of judgement (CCME 2016) 18

Zeiner (1994) on Variability A two-tiered approach: 1. Variability criteria was either 20% or 40% RPD (dependent on matrix type and organic/inorganic); or 2. Closer to detection limit, dependent on absolute difference relative to detection limit. Limitations: Not parameter specific No correlation with the laboratory s parameter specific variability (MU) Often used improperly by practitioners 19

Defining Acceptable Variability Abbreviated from the CCME (2016): Total error [variability] is twice the laboratory error [variability] Variability for soils is greater than variability for waters Variability is greater for values closer to the detection limit fine print 20

Using MUs to Evaluate Field QA/QC Samples Provides parameter and concentration specific basis for evaluating field duplicate samples. Recommended approach: Total variability for duplicate samples is twice times the MU for water Total variability for duplicate samples is three times the MU for soil and soil vapour Result: a parameter, concentration, and matrix specific basis for evaluating variability of QA/QC samples. 21