Proficiency testing: Aqueous ethanol. Test and measurement Workshop Marcellé Archer. 20 September 2011

Size: px

Start display at page:

Download "Proficiency testing: Aqueous ethanol. Test and measurement Workshop Marcellé Archer. 20 September 2011"

Norman McDowell
5 years ago
Views:

1 Proficiency testing: Aqueous ethanol Test and measurement Workshop Marcellé Archer 20 September 2011

2 NMISA Ethanol PT schemes Purpose of a PT scheme Consistent good performance is the goal One bad result does not make the laboratory bad either, but the results need to be studied and learned from for progressive improvement NMISA Ethanol PT schemes Offering the scheme since 2004 Until January 2010 all schemes had 4 rounds, 3 levels. Since then 3 rounds, 3 levels are planned

3 NMISA Ethanol PT schemes Round 3 Level 3 Round 3 Level 2 Round 3 Level 1 Round 2 Level 3 Round 2 Level 2 Round 2 Level 1 Round 1 Level 3 Round 1 Level 2 Round 1 Level Ethanol concentration in mg/ 100 g Example of concentrations prepared for a PT scheme

4 NMISA Ethanol PT schemes Purpose of a PT scheme Consistent good performance is the goal One bad result does not make the laboratory bad either, but the results need to be studied and learned from for progressive improvement NMISA Ethanol PT schemes Offering the scheme since 2004 Until January 2010 all schemes had 4 rounds, 3 levels. Since then 3 rounds, 3 levels are planned The trend has been towards consistent improvement

5 ISO 13528:2005 Statistical methods for use in proficiency testing by interlaboratory comparisons Paragraph 6: Determining the standard deviation for proficiency assessment 1. Prescribed method, e.g. legislation 2. By perception, i.e. the level of competence that it is desirable to achieve 3. From a general model, e.g. the Horwitz curve 4. From the results of a precision experiment: a standard method is used 5. From data obtained in a round of a proficiency testing scheme, i.e. from participant results 6. Comparison of precision values derived from a proficiency test with established values long term results

6 Statistics Initially, one has no idea of what spread of results can be expected Use a general model, e.g. Horwitz curve Use robust statistics The Horwitz curve showing high RSDs at low concentrations

7 The Horwitz curve σ R = 0,02c 0,08495 where R is the reproducibility standard deviation and c is the concentration of the analyte in percent (mass fraction) or (1 0.5 log c) RSD = ±2 Where c is the concentration of the analyte in g/ g or g/ ml

8 From data obtained in a round of a proficiency testing scheme When data has already been collected from a scheme. It works best when most of the participant errors have already been resolved. e.g. in 2003 in an international aqueous ethanol key comparison (CCQM-K27a) where only expert laboratories participated, the expanded uncertainty was calculated from the standard deviation of the mean

9 From data obtained in a round of a proficiency testing scheme U = t (n-1).s/ n Where U is the expanded uncertainty, the t value was used as the coverage factor, s is the standard deviation and n is the number of participants. In this case, for Sample A, the gravimetric value was mg/ g the s.e.m. (s/ n) was and t (n-1 ) for 6 participants was Thus the assigned concentration for Sample A = ± mg/ g

10 Another approach When sufficient data has been collected, it is possible to deduce a model for the method uncertainty. This may also be a sensible approach where the scheme involves a range of concentrations. To test whether the model is valid with 95% certainty, the number of measurements outside the 2s limits must be less than 5%

11 Normal distribution 95% 0 µ - 2σ µ µ + 2σ approximately 95% of the results will lie within 2σ of µ

12 % Difference, 100*(Mean-CRM)/CRM Model: sqrt(a 2 + (bx) 2 ) = u 15 NML-PT-ORG02 Aqueous ethanol scheme results One laboratory, per analyst s s Certified Concentration (g/ 100 g) Only about 87% of the results lie within 2σ of µ

13 % Difference, 100*(Mean-CRM)/CRM % Difference, 100*(Mean-CRM)/CRM Model: sqrt(a 2 + (bx) 2 ) = u 15 NML-PT-ORG02 Aqueous ethanol scheme results One laboratory, per analyst Only about 8% of the results lie outside 2σ of µ NML-PT-ORG02 Aqueous ethanol scheme results Two laboratories, per analyst Certified Concentration (g/ 100 g) s -2s Certified Concentration (g/ 100 g)

14 NMISA-PT-ORG03, 4 & 5 +3s +2s +1s -1s -2s -3s Clearly, more than 95% of the results lie within the ±2s limits

15 NMISA-PT-ORG05 & -ORG06 +2s +1s -1s -2s

16 Horwitz vs model Ethanol concentration (g/ 100 ml) 2u = U Rel 2u (%) Horwitz 2s (%) 0,01 0, ,0 16 0,02 0, ,7 14 0,03 0, ,2 14 0,04 0, ,2 13 0,05 0, ,0 13 0,06 0, ,3 12 0,07 0, ,9 12 0,08 0, ,6 12 0,09 0, ,4 11 0,1 0, ,2 11 0,2 0, ,6 10 0,3 0, ,5 10 0,4 0, ,5 9 0,5 0, ,5 9 0,6 0, ,5 9

17 NMISA-PT-ORG07 Level 1 concentration was chosen to lie close to the legal limit of ethanol in blood of SA non-professional drivers (0,05g/ 100 ml) and is the reason for the units reported in g/ 100 ml Level 2 should lie approximately in the centre of a typical blood alcohol calibration curve Level 3 was meant to mimic an alcoholic drink contaminated with methanol These concentrations could then test a suite of alcohol analytical skills

NMISA-PT-ORG07 Participants were asked to use the methods that would be routinely used to deliver that laboratory s measurement services Participants were requested to analyse two aliquots taken from

18 NMISA-PT-ORG07 Participants were asked to use the methods that would be routinely used to deliver that laboratory s measurement services Participants were requested to analyse two aliquots taken from each of the three samples, preferably using two different measurement systems The results were to be reported on an absolute basis together with the expanded uncertainty and to provide a full uncertainty budget The following concentrations were prepared and assayed: Level 1 ethanol: 0,04819 ± 0,00063 g/ 100 g Level 2 ethanol: 0,1959 ± 0,0020 g/ 100 g Level 3 ethanol: 8,1453 ± 0,0977 g/ 100 g Level 3 methanol: 0,048 ± 0,002 g/ 100 g The participants in this study analysed the samples by a variety of techniques (gas chromatography, distillation and density)

prepared The batches are bottled and sealed The concentrations are then

19 Preparation Aqueous ethanol solutions are prepared by mixing known weights of pure ethanol and organic-free water Batches of five litres or less are prepared The batches are bottled and sealed The concentrations are then verified by analysing aliquots from 8-12 bottles throughout the batch by the titrimetric method

Titrimetric assay Known weights of dilute ethanol solutions are reacted with known quantities of potassium dichromate in the presence of sulphuric acid.

20 Titrimetric assay Known weights of dilute ethanol solutions are reacted with known quantities of potassium dichromate in the presence of sulphuric acid. The ethanol is oxidised to acetic acid: C 2 H 5 OH + 2K 2 Cr 2 O 7 + 8H 2 SO 4 CH 3 COOH + 2K 2 SO 4 + 2Cr 2 (SO 4 ) H 2 O The excess potassium dichromate in the solution is back-titrated with ammonium iron sulphate solution 6FeSO 4 + 7H 2 SO 4 + K 2 Cr 2 O 7 = K 2 SO 4 + Cr 2 (SO 4 ) 3 + 3Fe 2 (SO 4 ) 3 + 7H 2 O

21 Titrimetric uncertainty

22 Traceability Mole (SI unit) Kilogram (SI unit) 3C 2 H 5 OH + 2K 2 Cr 2 O 7 + 8H 2 SO 4 3CH 3 COOH + 2K 2 SO 4 + 2Cr 2 (SO 4 ) H 2 O (Primary method) Calibrated balances NMISA ANA0038: U x = ± 0,0003 g ANA0189: U x = ± 0,02 g Reference material NML-ORG-001 ERM AC409a (U x = 3% rel) ERM AC401b (U x = 0,75% rel) ERM AC402a (U x = 0,56% rel) ERM AC403a (U x = 0,45% rel)

CRMs. This is to test the homogeneity and to confirm the ethanol concentration.

23 Gas-chromatographic assay Aliquots of each batch which contains a compound other than ethanol are assayed by headspace gaschromatography against LGC or NMISA ethanol CRMs. This is to test the homogeneity and to confirm the ethanol concentration. For Level 3, the methanol concentration was confirmed against a certified methanol solution.

24 Participants Institut National de Recherche et d'analyse Physicochimique INRAP, Tunisia Kenya Bureau of Standards, Food and Agricultural Laboratory Kenya Bureau of Standards, R&D Laboratory Malawi Bureau of Standards Rwanda Bureau of Standards Uganda National Bureau of Standards, Chemistry Laboratory

25 NMISA-PT-ORG07: Techniques Laboratory Laboratory 1 Laboratory 2 Laboratory 3 Laboratory 4 Laboratory 5 Laboratory 6 Analytical technique Direct injection, single point calibration Headspace analysis, calibration curve Direct injection, calibration curve Distillation/ pyncnometer Distillation/ alcoholmeter Direct injection, calibration curve

26 Laboratory Ethanol Level 1 (g/ 100 ml) Results Ethanol Level 2 (g/ 100 ml) Ethanol Level 3 (g/ 100 ml) Methanol Level 3 (g/ 100 ml) 0,119 0,199 7,677 0,0469 0,119 0,200 7,727 0,0468 0,121 0,207 7,860 0,0489 0,120 0,215 7,732 0,0488 0,0397 0,182 8,134 0,0604 0,039 0,193 7,914 0,0602 0,0392 0,181 8,114 0,0552 0,0405 0,182 8,144 0,0552 0,0459 0,183 8,076 0,0454 0,0410 0,181 8,116 0,0405 0,0429 0,186 7,904 0,0486 0,186 7,557 0,0435 7,928 0,0458 0,103 0,473 8,829 0,363 0,103 8, ,418 0,363 8,308 0,205 0,316 7,977 0,0789 0,481 9,034 0,521 0,363 8, ,119 0,6 7, ,04 0,21 16,54 0,055 17,11 Mean 0,161 0,260 8,989 0,050 *RSD (%) 107,1 49,9 28,9 12,2 Assigned concentration (g/ 100 g) 0,04819 ± 0, ,1959 ± 0,0020 8,1453 ± 0,0977 0,048 ± 0,002

27 ISO 13528:2005 Statistical methods for use in proficiency testing by interlaboratory comparisons. Paragraph 6: Determining the standard deviation for proficiency assessment 1. Prescribed method, e.g. legislation 2. By perception, i.e. the level of competence that it is desirable to achieve 3. From a general model, e.g. the Horwitz curve 4. From the results of a precision experiment: a standard method is used 5. From data obtained in a round of a proficiency testing scheme 6. Comparison of precision values derived from a proficiency test with established values

28 σ R = 0,02c 0,08495 The Horwitz curve where R is the reproducibility standard deviation and c is the concentration of the analyte in percent (mass fraction) or RSD = 2 (1 0,5 log c) Where c is the concentration of the analyte in g/ g or g/ ml Conc EtOH Expected Expected Expected Expected g/ l00 g g/ g RSD % s 2s 3s Level Level Level

29 Concentration (g/ 100 ml) Evaluation of results NMISA-PT-ORG07 Aqueous ethanol: Level 1 Comparison of results *Distillation/ density method

30 Concentration (g/ 100 ml) Evaluation of results NMISA-PT-ORG07 Aqueous ethanol: Level 1 Comparison of results 3s 2s 1s s -2s -3s Certified value: ± g/ 100 g Results with high results removed in order to view acceptable values

31 Calculation of performance statistics Laboratory bias: ISO 13528:2005 Par 7.1.2: When a participant reports a result that gives rise to a laboratory bias greater than 3,0 s* or less than -3,0 s, then the result shall be considered to give an action signal. Likewise a laboratory bias above 2,0 s or below -2,0 s shall be considered to give a warning signal. * Where s is the standard deviation for proficiency assessment

32 Calculation of performance statistics There are a number of different interpretations for estimating the performance. The most common being the z-score. z = (x X)/ s where x is the value submitted by the participating laboratory, X is the assigned value and s is the standard deviation for proficiency assessment. e.g. z = (0,0789 0,04819)/ 0,003 = 10,2

33 Concentration (g/ 100 ml) Evaluation of results NMISA-PT-ORG07 Aqueous ethanol: Level 1 Comparison of results *Distillation/ density method

34 z-scores z < 2 is considered satisfactory 2 z 3 is considered questionable z 3 is considered unsatisfactory

35 z-scores z < 2 is considered satisfactory 2 z 3 is considered questionable z 3 is considered unsatisfactory

36 z-scores z < 2 is considered satisfactory 2 z 3 is considered questionable z 3 is considered unsatisfactory

37 z-scores z < 2 is considered satisfactory 2 z 3 is considered questionable z 3 is considered unsatisfactory

practical lab work at KEBS (7 level calibration curve prepared by the participants and analysed

38 Workshop in Kenya Interactive workshop for the PT participants where each participant presented their method, results and UoM Detailed discussion of calculation of results, followed by practical lab work at KEBS (7 level calibration curve prepared by the participants and analysed by direct injection GC-FID) Each participant received 4 calibration solutions (NMISA CRMs) to take back

Concentration (g/ 100 g) Round 1 Aqueous ethanol: Level 4 Comparison of results Workshop NMISA-PT-ORG07 Round 1 Aqueous ethanol level 4 comparison of results Certified value 0.04858 ± 0.

39 Concentration (g/ 100 g) Round 1 Aqueous ethanol: Level 4 Comparison of results Workshop NMISA-PT-ORG07 Round 1 Aqueous ethanol level 4 comparison of results Certified value ± g/ 100 g Prepared value NMISA AFRIMETS workshop Result obtained by workshop participants for Level 1 after training. Calibration solutions supplied by NMISA. The NMISA result is the mean of a titrimetric analysis of eight bottles from the batch.

PeakArea EtOH/PeakArea I.S. Calibration with traceable CRM 3.5 3 EtOH calibration (g/100 g) y = 6.3375x R² = 0.9997 2.5 2 1.5 1 0.

40 PeakArea EtOH/PeakArea I.S. Calibration with traceable CRM EtOH calibration (g/100 g) y = x R² = mass EtOH vial/ mass I.S. vial Calibration curve obtained by workshop participants using CRMs from NMISA.

41 Discussion Although low levels of alcohols can be successfully distilled from solution, the determination of the alcohol concentration by measuring the density of the distillate is usually applied to higher alcohol levels as found in alcoholic drinks Density tables normally have no graduations between 0% alcohol and 1% alcohol The distillation method is also only suitable for beverages which contain ethanol as the sole volatile component, unless fractional distillation is employed or gas chromatography is used to analyse the distillate

42 Discussion From the density results for Level 1 and Level 2, it is clear that this method, as presently applied by Laboratories 4 and 5, is not suitable for ethanol concentrations below 1 g/ 100 ml As most of the distillation/ density results for Level 3 are also slightly high, it is likely that the methanol in this solution was also distilled over and measured as ethanol

43 Discussion The gas chromatographic results for Level 1 were disappointing This may be because some analysts used a calibration curve which did not cover this low level (although the instructions gave the expected concentration range of the samples) or because a single point calibration reference material was used that did not match the sample concentration

44 Concentration (g/ 100 ml) Discussion NMISA-PT-ORG07 Aqueous ethanol: Level 1 Comparison of results +3s +2s +1s s -2s -3s Certified value: ± g/ 100 g

45 Discussion The gas-chromatographic results for Level 2 all fall within ± 2s, which can be considered satisfactory and indicates that the methods used by the participants are appropriate for the concentration

46 Concentration (g/ 100 ml) Discussion NMISA-PT-ORG07 Aqueous ethanol: Level 2 Comparison of results s 2s 1s -1s -2s -3s Certified value: ± g/ 100 g

47 Discussion For Level 3 ethanol, most of the results (GC and distillation) fall within ± 3s, which is much better than for the other two levels.

48 Concentration (g/ 100 ml) Discussion NMISA-PT-ORG07 Aqueous ethanol: Level 3 Comparison of results s 2s 1s -1s -2s -3s Certified value: ± g/ 100 g * Distillation/ density method Level 3 ethanol results with outliers removed

49 Concentration (g/ 100 ml) Discussion NMISA-PT-ORG07 Aqueous ethanol: Level 3 Comparison of results Certified value: ± g/ 100 g * Distillation/ density method Level 3 ethanol results including outliers.

50 Remarks Inappropriate techniques for the concentration range Inappropriate calibration techniques None of the participants used CRMs; All the reference solutions were prepared in-house None of the pure ethanol or methanol was traceable to National Standards Some laboratories did not submit uncertainty budgets

reference materials, none of the participants used traceable certified reference materials.

51 Conclusions Only 55% of the results submitted fall within the (Horwitz) ± 2s limits. This means that 45% of the results require investigation For the methods that require the use of reference materials, none of the participants used traceable certified reference materials. Although in-house reference materials are useful for day-to-day analytical quality control, they still have to conform to some basic requirements.

52 Conclusions Training in the selection and use of reference materials may be helpful to the participants Some improvements to most of the analytical methods may result in better quality results For example, better choice of the concentration of calibration solutions and working ranges

53 Conclusions Although the distillation method is a low cost and effective method when applied to appropriate samples, the use of headspace gas chromatography may result in more reliable results and improved sample throughput..

54 Thank you to: NMISA workshop presenters are circled. The rest are the workshop participants.

International interlaboratory study of forensic ethanol standards

International interlaboratory study of forensic ethanol standards Bernard King and Richard Lawn LGC, Queens Road, Teddington, Middlesex, UK TW11 0LY. E-mail: bmjk@lgc.co.uk Received 18th January 1999,