Sample Homogeneity Testing

Transcription

1 Sample Homogeneity Testing Following the Protocol Outlined in: The International Harmonized Protocol for the Proficiency Testing of Analytical Laboratories, 006 (IHP), MICHAEL THOMPSON, STEPHEN L. R. ELLISON AND ROGER WOOD And: Statistical methods for use in proficiency testing by interlaboratory comparison ISO 1358, Second Edition, 08/01/015

2 Heterogeneity among the sample units (σ Samples) can inflate the spread in sample results. This can mask the true Lab Bias and interferes with Z scores. Sources of Variance for Check Sample Results can be expressed as follows: Sample Results Analytical Labs Samples Ideally, to test for sample homogeneity we would like to minimize the analytical variance (σ Analytical) and the Lab Bias (σ Labs) and isolate the variance due to the sample units (σ Samples).

3 So, how do we minimize the analytical variance (σ Analytical) and the Lab Bias (σ Labs) and isolate the variance due to the sample units (σ Samples). Randomly select 10 sample units from a batch for analysis. These 10 samples represent the sampling bias we wish to measure. Proper sub-sampling of the Sample Unit is the business of each lab and that is another story. Select a single expert lab. This should remove inter Lab bias between the samples. Choose a method with very low analytical variance. This should minimize analytical variance. Sample Results Analytical Labs Samples A Quote from the IHP: Homogeneity tests should be regarded as essential, but not foolproof.

4 Let s look at a Dataset for 10 sample units analyzed in duplicate at one lab. A Chochran test checks for outliers in duplicates (too far apart!). Subtracting duplicates removes the sampling bias in each sample unit. Adding the duplicates generates twice this bias (Variance/). Phosphorus, ICP (%) Sample Unit Dup 1 Dup Difference Sum

5 First we calculate a 95% CI for the allowed variation. A Critical Variance not to be exceeded! Number of Pairs 10 Grand Average for P by ICP 9.54

6 First we calculate a 95% CI for the allowed variation. A Critical Variance not to be exceeded! Number of Pairs 10 Grand Average for P by ICP 9.54 Allowed Variation We must decide what dispersion is ffp! 3.00% SD for Proficiency Testing (σ ffp ) 9.54 * 0.03 (Target) From the IHP: Allowed Variance (30% of PT target) (less than 10% of proficiency variance) F1 constant (derived from Chi distribution 95% Confidence) F constant (derived from f distribution 95% Confidence) Critical Allowed Variance (σ Allowed ) Next we need to calculate the actual variance attributed to the sample units for comparison.

7 Now we can Calculate the Sampling Variation Number of Pairs 10 Grand Average 9.54 Sampling Variation Calculation A - Variance of Differences (sampling error removed) B - Variance of Sums (includes A and x sampling error) Variance Attributed to Sampling (B/-A)/ (if negative V S = 0, not detectable)

8 Now we can Calculate the Sampling Variation Number of Pairs 10 Grand Average 9.54 Sampling Variation Calculation A - Variance of Differences (sampling error removed) B - Variance of Sums (includes A and sampling error) Variance Attributed to Sampling (B - A)/ (if negative V S = 0, Not detectable) Since variance attributed to Sampling (0.0000) < Critical (0.1041) there is no evidence of sampling variance.

9 Let s Talk About σ ffp Where did I get 3%?? We need our best estimate of the usual and expected dispersion for the analysis with respect to our sample matrices and concentration types. In the Magruder CSP we calculate σ ffp for each sample from participants data. This is not practical for Homogeneity testing. We need to pick a σ ffp appropriate to all our samples. I looked back over the samples and selected a 3% Reproducibility %RSD target for each Analyte used in the Homogeneity test. This is a first approximation which we can refine as we move forward with more samples and more Analytes. I favor using %RSD as it reflects variance independent of concentration.

10 A Slightly Different Approach! Pass Homogeneity %RSD Threshold If we can calculate a Critical Allowed Variance based on a %RSD estimate for proficiency (Example 3%) and test it against the actual variance attributed to sampling. Then we can back calculate the %RSD Threshold for Homogeneity. %RSD Threshold Variance AttributedtoSampling C Variance Grand Mean of Differences Now we can examine the %RSD Threshold in light of say 3%. I prefer this approach to PASS/FAIL tests. Old school statisticians generally don t like this!

11 Homogeneity Testing on a Recent Check Sample: Sample Code 15111, Grade The following analyses were run in duplicate: Analyte Method Magruder Method Code Sulfur Combustion Phosphorus ICP Magnesium ICP Potash ICP Nitrogen Combustion Chemical Analysis by Scott F. Roalofs, CDA

12 Homogeneity Report: Grade , Sample Code Method Code Method Description Sulfur Combustion Phos. ICP Mg ICP Potash ICP Nitrogen Sample # \ Duplicate Homogeneity Decision PASS PASS PASS PASS PASS Passing %RSD Threshold 0.00% 0.00% 0.00% 0.00% 0.00% %RSD for Proficiency 3.0% 3.0% 3.0% 3.0% 3.0% Critical Allowed Variance Variance Due to Sampling

13 Homogeneity Report: Grade , Sample Code Method Code Method Description Sulfur Combustion Phos. ICP Mg ICP Potash ICP Nitrogen Sample # \ Duplicate Remember we talked about being unable to detect a variance attributable to sampling? V S < V D 9 Grand Mean Homogeneity Decision PASS PASS PASS PASS PASS Passing %RSD Threshold 0.00% 0.00% 0.00% 0.00% 0.00% %RSD for Proficiency 3.0% 3.0% 3.0% 3.0% 3.0% Critical Allowed Variance Variance Due to Sampling

14 Homogeneity Report: Grade , Sample Code Method Code Method Description Sulfur Combustion Phos. ICP Mg ICP Potash ICP Nitrogen Sample # \ Duplicate Grand Mean Is It Fit for Purpose! Analytical variance may be too high Homogeneity Decision PASS PASS PASS PASS PASS Passing %RSD Threshold 0.00% 0.00% 0.00% 0.00% 0.00% %RSD for Proficiency 3.0% 3.0% 3.0% 3.0% 3.0% Critical Allowed Variance Variance Due to Sampling Repeatability %rsd 1.71% 3.13% 4.99%.86% 0.7% Magruder %rsd 1.11% (ICP) 1.81% 1.75% 1.74% 0.61%

15 Sample Homogeneity Testing The Remarkable Case of Nitrogen Combustion

16 014 and 015 Samples, this is History! Nitrogen Combustion Labs in Robust Calculations Assigned Value Mean 5 Labs Within Labs %rsd Reproducibility %RSD sr/sr Grade % 1.50% DAP % 1.37% Grade % 1.47% Grade % 1.53% Grade % 1.8% Grade % 1.81% Grade %.56% Grade % 1.5% Grade % 1.67% Grade % 3.70% Grade %.78% Grade (DAP) % 1.5% Grade % 1.88% UAN % 1.38% Grade % 1.4% micros % 1.84% Grade % 4.9% Grade % 4.63% Grade % 1.37% % 1.95% MAP % 1.65% Grade % 1.3%.

17 Can We Estimate Homogeneity From the CSP Data? In N by Combustion we have a very precise method with over 50 Analysts consistently reporting Reproducibility < %RSD and Repeatability < 1 %rsd for each sample. This is an extremely narrow dispersion for so many Labs. So looking at our sources of variance again: Reproducibility is heavily reflected in σ Labs and is low! Repeatability is essentially the σ Analytical and is low! Sample Results Analytical Labs Samples

18 Can We Estimate Homogeneity From Nitrogen Data? If I take the center portion of the data (Z between ± 1, ~ 68% ), these Labs should begin to approach the data from a Homogeneity study (Low Lab Bias). 68% Z Value Let s call this the Z Cut the filet mignon of the data, if you will.

19 15111 Homogeneity Report Compared With The Z Cut Sample Run by Different Labs. Grade , Sample CDA Lab Magruder Homogeneity Decision PASS PASS Pass Z-Cut %RSD Threshold 0.00% 0.00% Number of Pairs (samples) (66.1% Z Cut) Grand Average Allowed Variation Selected σ for Proficiency 3% 3% Watch This! SD for Proficiency Testing Allowed Variance (30% of target) F1 constant F constant Critical variance Actual Variation Variance of Differences (B) Variance of Sums/ (A) Variance Attributed to Sampling Repeatability %rsd (σ analytical ) 0.7% 0.85% % RSD of Sample Means 0.47% 0.63% Outlier Test PASS PASS

20 Z-Cut Results For 014 and 015 Samples Sample # Sample Name Homogeneity Decision Pass Z-Cut %RSD Threshold Pairs in Z- Cut Labs in Z-Cut Grand Mean Critical Variance Variance Attributed to Sampling Grade PASS 0.908% % DAP PASS 0.40% % Grade PASS 0.000% % Grade PASS 0.981% % Grade 5--0 PASS 0.000% % Grade PASS 0.561% % Grade PASS 0.000% % Grade PASS 0.000% % Grade PASS 1.559% % Grade FAIL 4.051% % Grade PASS.065% % Grade (DAP) PASS 0.976% % Grade 6-9- PASS 1.545% % UAN PASS 0.000% % Grade PASS 1.5% 4 7.4% micros PASS 0.63% % Grade PASS 0.000% % Grade FAIL 6.135% % Grade PASS 0.000% % PASS 0.000% % MAP PASS 0.000% % Grade PASS 0.000% %

21 Z-Cut Results For 014 and 015 Samples Sample # Sample Name Homogeneity Decision Pass Z-Cut %RSD Threshold Pairs in Z- Cut Labs in Z-Cut Grand Mean Critical Variance Variance Attributed to Sampling Grade PASS 0.908% DAP PASS 0.40% Grade PASS 0.000% Grade PASS 0.981% Grade 5--0 PASS 0.000% Grade PASS 0.561% Grade PASS 0.000% Grade PASS 0.000% Grade PASS 1.559% Grade FAIL 4.051% % Grade PASS.065% Grade (DAP) PASS 0.976% Grade 6-9- PASS 1.545% UAN PASS 0.000% Grade PASS 1.5% micros PASS 0.63% Grade PASS 0.000% Grade FAIL 6.135% % Grade PASS 0.000% PASS 0.000% MAP PASS 0.000% Grade PASS 0.000% 9.98

22 Recommendation: I run a Z Cut pseudo Homogeneity test on Nitrogen Combustion data for each sample. If it passes the pseudo Homogeneity test at say 3.0 %RSD then we can assume acceptable homogeneity for our purposes. If it does not pass we will examine the sample data more closely before reporting a possible homogeneity issue. This does not substitute for a legitimate Homogeneity study! Some analytes may have a distributional heterogeneity not revealed by N But, I think we can make a good case that a Z-Cut is a very reasonable (cost effective!) sample to sample homogeneity flag.

23 Questions? Thank You