Proficiency testing: Aqueous ethanol. Test and measurement Workshop Marcellé Archer. 20 September PDF Free Download

Proficiency testing: Aqueous ethanol Test and measurement Workshop Marcellé Archer 20 September 2011

NMISA Ethanol PT schemes Purpose of a PT scheme Consistent good performance is the goal One bad result does not make the laboratory bad either, but the results need to be studied and learned from for progressive improvement NMISA Ethanol PT schemes Offering the scheme since 2004 Until January 2010 all schemes had 4 rounds, 3 levels. Since then 3 rounds, 3 levels are planned

NMISA Ethanol PT schemes Round 3 Level 3 Round 3 Level 2 Round 3 Level 1 Round 2 Level 3 Round 2 Level 2 Round 2 Level 1 Round 1 Level 3 Round 1 Level 2 Round 1 Level 1 0 100 200 300 400 500 600 Ethanol concentration in mg/ 100 g Example of concentrations prepared for a PT scheme

ISO 13528:2005 Statistical methods for use in proficiency testing by interlaboratory comparisons Paragraph 6: Determining the standard deviation for proficiency assessment 1. Prescribed method, e.g. legislation 2. By perception, i.e. the level of competence that it is desirable to achieve 3. From a general model, e.g. the Horwitz curve 4. From the results of a precision experiment: a standard method is used 5. From data obtained in a round of a proficiency testing scheme, i.e. from participant results 6. Comparison of precision values derived from a proficiency test with established values long term results

Statistics Initially, one has no idea of what spread of results can be expected Use a general model, e.g. Horwitz curve Use robust statistics The Horwitz curve showing high RSDs at low concentrations

The Horwitz curve σ R = 0,02c 0,08495 where R is the reproducibility standard deviation and c is the concentration of the analyte in percent (mass fraction) or (1 0.5 log c) RSD = ±2 Where c is the concentration of the analyte in g/ g or g/ ml

From data obtained in a round of a proficiency testing scheme When data has already been collected from a scheme. It works best when most of the participant errors have already been resolved. e.g. in 2003 in an international aqueous ethanol key comparison (CCQM-K27a) where only expert laboratories participated, the expanded uncertainty was calculated from the standard deviation of the mean

From data obtained in a round of a proficiency testing scheme U = t (n-1).s/ n Where U is the expanded uncertainty, the t value was used as the coverage factor, s is the standard deviation and n is the number of participants. In this case, for Sample A, the gravimetric value was 0.8040 mg/ g the s.e.m. (s/ n) was 0.00325 and t (n-1 ) for 6 participants was 2.447 Thus the assigned concentration for Sample A = 0.8040 ± 0.0080 mg/ g

Another approach When sufficient data has been collected, it is possible to deduce a model for the method uncertainty. This may also be a sensible approach where the scheme involves a range of concentrations. To test whether the model is valid with 95% certainty, the number of measurements outside the 2s limits must be less than 5%

Normal distribution 95% 0 µ - 2σ µ µ + 2σ approximately 95% of the results will lie within 2σ of µ

% Difference, 100*(Mean-CRM)/CRM Model: sqrt(a 2 + (bx) 2 ) = u 15 NML-PT-ORG02 Aqueous ethanol scheme results One laboratory, per analyst 10 5 +2s 0-5 -2s -10-15 0.0 0.1 0.2 0.3 0.4 0.5 0.6 Certified Concentration (g/ 100 g) Only about 87% of the results lie within 2σ of µ

% Difference, 100*(Mean-CRM)/CRM % Difference, 100*(Mean-CRM)/CRM Model: sqrt(a 2 + (bx) 2 ) = u 15 NML-PT-ORG02 Aqueous ethanol scheme results One laboratory, per analyst 10 5 0-5 -10 15 Only about 8% of the results lie outside 2σ of µ NML-PT-ORG02 Aqueous ethanol scheme results Two laboratories, per analyst 10-15 0.0 0.1 0.2 0.3 0.4 0.5 0.6 5 Certified Concentration (g/ 100 g) 0-5 +2s -2s -10-15 0.0 0.1 0.2 0.3 0.4 0.5 0.6 Certified Concentration (g/ 100 g)

NMISA-PT-ORG03, 4 & 5 +3s +2s +1s -1s -2s -3s Clearly, more than 95% of the results lie within the ±2s limits

NMISA-PT-ORG05 & -ORG06 +2s +1s -1s -2s

Horwitz vs model Ethanol concentration (g/ 100 ml) 2u = U Rel 2u (%) Horwitz 2s (%) 0,01 0,003002 30,0 16 0,02 0,003147 15,7 14 0,03 0,003374 11,2 14 0,04 0,003668 9,2 13 0,05 0,004014 8,0 13 0,06 0,004401 7,3 12 0,07 0,004818 6,9 12 0,08 0,005258 6,6 12 0,09 0,005717 6,4 11 0,1 0,006189 6,2 11 0,2 0,01127 5,6 10 0,3 0,01658 5,5 10 0,4 0,02195 5,5 9 0,5 0,02735 5,5 9 0,6 0,03277 5,5 9

NMISA-PT-ORG07 Level 1 concentration was chosen to lie close to the legal limit of ethanol in blood of SA non-professional drivers (0,05g/ 100 ml) and is the reason for the units reported in g/ 100 ml Level 2 should lie approximately in the centre of a typical blood alcohol calibration curve Level 3 was meant to mimic an alcoholic drink contaminated with methanol These concentrations could then test a suite of alcohol analytical skills

NMISA-PT-ORG07 Participants were asked to use the methods that would be routinely used to deliver that laboratory s measurement services Participants were requested to analyse two aliquots taken from each of the three samples, preferably using two different measurement systems The results were to be reported on an absolute basis together with the expanded uncertainty and to provide a full uncertainty budget The following concentrations were prepared and assayed: Level 1 ethanol: 0,04819 ± 0,00063 g/ 100 g Level 2 ethanol: 0,1959 ± 0,0020 g/ 100 g Level 3 ethanol: 8,1453 ± 0,0977 g/ 100 g Level 3 methanol: 0,048 ± 0,002 g/ 100 g The participants in this study analysed the samples by a variety of techniques (gas chromatography, distillation and density)

Preparation Aqueous ethanol solutions are prepared by mixing known weights of pure ethanol and organic-free water Batches of five litres or less are prepared The batches are bottled and sealed The concentrations are then verified by analysing aliquots from 8-12 bottles throughout the batch by the titrimetric method

Titrimetric assay Known weights of dilute ethanol solutions are reacted with known quantities of potassium dichromate in the presence of sulphuric acid. The ethanol is oxidised to acetic acid: C 2 H 5 OH + 2K 2 Cr 2 O 7 + 8H 2 SO 4 CH 3 COOH + 2K 2 SO 4 + 2Cr 2 (SO 4 ) 3 + 11H 2 O The excess potassium dichromate in the solution is back-titrated with ammonium iron sulphate solution 6FeSO 4 + 7H 2 SO 4 + K 2 Cr 2 O 7 = K 2 SO 4 + Cr 2 (SO 4 ) 3 + 3Fe 2 (SO 4 ) 3 + 7H 2 O

Titrimetric uncertainty

Traceability Mole (SI unit) Kilogram (SI unit) 3C 2 H 5 OH + 2K 2 Cr 2 O 7 + 8H 2 SO 4 3CH 3 COOH + 2K 2 SO 4 + 2Cr 2 (SO 4 ) 3 + 11H 2 O (Primary method) Calibrated balances NMISA ANA0038: U x = ± 0,0003 g ANA0189: U x = ± 0,02 g Reference material NML-ORG-001 ERM AC409a (U x = 3% rel) ERM AC401b (U x = 0,75% rel) ERM AC402a (U x = 0,56% rel) ERM AC403a (U x = 0,45% rel)

Gas-chromatographic assay Aliquots of each batch which contains a compound other than ethanol are assayed by headspace gaschromatography against LGC or NMISA ethanol CRMs. This is to test the homogeneity and to confirm the ethanol concentration. For Level 3, the methanol concentration was confirmed against a certified methanol solution.

Participants Institut National de Recherche et d'analyse Physicochimique INRAP, Tunisia Kenya Bureau of Standards, Food and Agricultural Laboratory Kenya Bureau of Standards, R&D Laboratory Malawi Bureau of Standards Rwanda Bureau of Standards Uganda National Bureau of Standards, Chemistry Laboratory

NMISA-PT-ORG07: Techniques Laboratory Laboratory 1 Laboratory 2 Laboratory 3 Laboratory 4 Laboratory 5 Laboratory 6 Analytical technique Direct injection, single point calibration Headspace analysis, calibration curve Direct injection, calibration curve Distillation/ pyncnometer Distillation/ alcoholmeter Direct injection, calibration curve

Laboratory 1 2 3 Ethanol Level 1 (g/ 100 ml) Results Ethanol Level 2 (g/ 100 ml) Ethanol Level 3 (g/ 100 ml) Methanol Level 3 (g/ 100 ml) 0,119 0,199 7,677 0,0469 0,119 0,200 7,727 0,0468 0,121 0,207 7,860 0,0489 0,120 0,215 7,732 0,0488 0,0397 0,182 8,134 0,0604 0,039 0,193 7,914 0,0602 0,0392 0,181 8,114 0,0552 0,0405 0,182 8,144 0,0552 0,0459 0,183 8,076 0,0454 0,0410 0,181 8,116 0,0405 0,0429 0,186 7,904 0,0486 0,186 7,557 0,0435 7,928 0,0458 0,103 0,473 8,829 0,363 0,103 8,308 4 0,418 0,363 8,308 0,205 0,316 7,977 0,0789 0,481 9,034 0,521 0,363 8,829 5 0,119 0,6 7,677 6 0,04 0,21 16,54 0,055 17,11 Mean 0,161 0,260 8,989 0,050 *RSD (%) 107,1 49,9 28,9 12,2 Assigned concentration (g/ 100 g) 0,04819 ± 0,00063 0,1959 ± 0,0020 8,1453 ± 0,0977 0,048 ± 0,002

ISO 13528:2005 Statistical methods for use in proficiency testing by interlaboratory comparisons. Paragraph 6: Determining the standard deviation for proficiency assessment 1. Prescribed method, e.g. legislation 2. By perception, i.e. the level of competence that it is desirable to achieve 3. From a general model, e.g. the Horwitz curve 4. From the results of a precision experiment: a standard method is used 5. From data obtained in a round of a proficiency testing scheme 6. Comparison of precision values derived from a proficiency test with established values

σ R = 0,02c 0,08495 The Horwitz curve where R is the reproducibility standard deviation and c is the concentration of the analyte in percent (mass fraction) or RSD = 2 (1 0,5 log c) Where c is the concentration of the analyte in g/ g or g/ ml Conc EtOH Expected Expected Expected Expected g/ l00 g g/ g RSD % s 2s 3s Level 1 0.048 0.00048 6.32 0.0030 0.006 0.0091 Level 2 0.196 0.00196 5.11 0.010 0.020 0.0301 Level 3 8.1453 0.081453 2.92 0.238 0.475 0.7128

Concentration (g/ 100 ml) Evaluation of results NMISA-PT-ORG07 Aqueous ethanol: Level 1 Comparison of results 0.1190.119 0.121 0.120 0.103 0.0789 0.048190.04805 0.03970.0390.0392 0.0405 0.0459 0.0410 0.0429 0.04 *Distillation/ density method

Concentration (g/ 100 ml) Evaluation of results NMISA-PT-ORG07 Aqueous ethanol: Level 1 Comparison of results 3s 2s 1s 0.04819 0.04805 0.0405 0.0397 0.039 0.0392 0.0459 0.0410 0.0429 0.04-1s -2s -3s Certified value: 0.04819 ± 0.00063 g/ 100 g Results with high results removed in order to view acceptable values

Calculation of performance statistics Laboratory bias: ISO 13528:2005 Par 7.1.2: When a participant reports a result that gives rise to a laboratory bias greater than 3,0 s* or less than -3,0 s, then the result shall be considered to give an action signal. Likewise a laboratory bias above 2,0 s or below -2,0 s shall be considered to give a warning signal. * Where s is the standard deviation for proficiency assessment

Calculation of performance statistics There are a number of different interpretations for estimating the performance. The most common being the z-score. z = (x X)/ s where x is the value submitted by the participating laboratory, X is the assigned value and s is the standard deviation for proficiency assessment. e.g. z = (0,0789 0,04819)/ 0,003 = 10,2

z-scores z < 2 is considered satisfactory 2 z 3 is considered questionable z 3 is considered unsatisfactory

Workshop in Kenya Interactive workshop for the PT participants where each participant presented their method, results and UoM Detailed discussion of calculation of results, followed by practical lab work at KEBS (7 level calibration curve prepared by the participants and analysed by direct injection GC-FID) Each participant received 4 calibration solutions (NMISA CRMs) to take back

Concentration (g/ 100 g) Round 1 Aqueous ethanol: Level 4 Comparison of results Workshop NMISA-PT-ORG07 Round 1 Aqueous ethanol level 4 comparison of results Certified value 0.04858 ± 0.0012 g/ 100 g 0.04858 0.0493 0.04841 Prepared value NMISA AFRIMETS workshop Result obtained by workshop participants for Level 1 after training. Calibration solutions supplied by NMISA. The NMISA result is the mean of a titrimetric analysis of eight bottles from the batch.

PeakArea EtOH/PeakArea I.S. Calibration with traceable CRM 3.5 3 EtOH calibration (g/100 g) y = 6.3375x R² = 0.9997 2.5 2 1.5 1 0.5 0 0 0.1 0.2 0.3 0.4 0.5 0.6 mass EtOH vial/ mass I.S. vial Calibration curve obtained by workshop participants using CRMs from NMISA.

Discussion Although low levels of alcohols can be successfully distilled from solution, the determination of the alcohol concentration by measuring the density of the distillate is usually applied to higher alcohol levels as found in alcoholic drinks Density tables normally have no graduations between 0% alcohol and 1% alcohol The distillation method is also only suitable for beverages which contain ethanol as the sole volatile component, unless fractional distillation is employed or gas chromatography is used to analyse the distillate

Discussion From the density results for Level 1 and Level 2, it is clear that this method, as presently applied by Laboratories 4 and 5, is not suitable for ethanol concentrations below 1 g/ 100 ml As most of the distillation/ density results for Level 3 are also slightly high, it is likely that the methanol in this solution was also distilled over and measured as ethanol

Discussion The gas chromatographic results for Level 1 were disappointing This may be because some analysts used a calibration curve which did not cover this low level (although the instructions gave the expected concentration range of the samples) or because a single point calibration reference material was used that did not match the sample concentration

Concentration (g/ 100 ml) Discussion NMISA-PT-ORG07 Aqueous ethanol: Level 1 Comparison of results +3s +2s +1s 0.04819 0.04805 0.0405 0.0397 0.0390 0.0392 0.0459 0.0410 0.0429 0.04-1s -2s -3s Certified value: 0.04819 ± 0.00063 g/ 100 g

Discussion The gas-chromatographic results for Level 2 all fall within ± 2s, which can be considered satisfactory and indicates that the methods used by the participants are appropriate for the concentration

Concentration (g/ 100 ml) Discussion NMISA-PT-ORG07 Aqueous ethanol: Level 2 Comparison of results 0.19590.1953 0.199 0.200 0.207 0.215 0.182 0.193 0.1810.182 0.183 0.1860.186 0.181 0.21 3s 2s 1s -1s -2s -3s Certified value: 0.1959 ± 0.0020 g/ 100 g

Discussion For Level 3 ethanol, most of the results (GC and distillation) fall within ± 3s, which is much better than for the other two levels.

Concentration (g/ 100 ml) Discussion NMISA-PT-ORG07 Aqueous ethanol: Level 3 Comparison of results 8.1453 7.99810 7.677 7.727 7.860 7.732 8.134 7.914 8.114 8.144 8.076 8.116 7.904 7.557 7.928 8.829 8.30 8.308 7.977 9.034 8.829 3s 2s 1s -1s -2s -3s Certified value: 8.1453 ± 0.0977 g/ 100 g * Distillation/ density method Level 3 ethanol results with outliers removed

Concentration (g/ 100 ml) Discussion NMISA-PT-ORG07 Aqueous ethanol: Level 3 Comparison of results 16.54 17.11 9.95 9.034 8.829 8.829 8.1453 7.99810 7.6777.727 7.860 8.134 8.30 8.308 8.1148.1448.0768.116 7.732 7.914 7.9047.557 7.928 7.977 Certified value: 8.1453 ± 0.0977 g/ 100 g * Distillation/ density method Level 3 ethanol results including outliers.

Remarks Inappropriate techniques for the concentration range Inappropriate calibration techniques None of the participants used CRMs; All the reference solutions were prepared in-house None of the pure ethanol or methanol was traceable to National Standards Some laboratories did not submit uncertainty budgets

Conclusions Only 55% of the results submitted fall within the (Horwitz) ± 2s limits. This means that 45% of the results require investigation For the methods that require the use of reference materials, none of the participants used traceable certified reference materials. Although in-house reference materials are useful for day-to-day analytical quality control, they still have to conform to some basic requirements.

Conclusions Training in the selection and use of reference materials may be helpful to the participants Some improvements to most of the analytical methods may result in better quality results For example, better choice of the concentration of calibration solutions and working ranges

Conclusions Although the distillation method is a low cost and effective method when applied to appropriate samples, the use of headspace gas chromatography may result in more reliable results and improved sample throughput..

Thank you to: NMISA workshop presenters are circled. The rest are the workshop participants.

Proficiency testing: Aqueous ethanol. Test and measurement Workshop Marcellé Archer. 20 September 2011