TC2 EXPERIENCES IN COLLABORATIVE STUDIES, METHOD VALIDATION AND PROFICIENCY TESTING

Transcription

1 Distributed in TC2 as document No ICG/TC-2/ TC2 EXPERIENCES IN COLLABORATIVE STUDIES, METHOD VALIDATION AND PROFICIENCY TESTING E. Guadagnino Stazione Sperimentale del Vetro, Via Briati 10, Murano - Venice, Italy D. Brochot Corning SAS CETC, 7 bis, Avenue de Vlavins, Avon, France P. Sundberg Glasforskningsinstitutet (GLAFO), PO Box 5157, Växjö, Sweden Section 1 : Method Validation 1.1 A tool to demonstrate that the selected method is fit for purpose Establishing suitable analytical procedures for the analysis of glass and for the estimation of its chemical durability has always been the mission of Technical Committee 2. A collection of TC2 contributions over the years is published elsewhere. [1,2] In more recent times, TC2 activity has been dealing with the certification of reference materials [3-5]. This activity required all participants to meet the strict requirements imposed by the Certification body (former BCR, BAM, IRMM) in terms of data quality assessment. This contributed significantly to improve the performance of all participating laboratories. The basic principles achieved by participating into the certification campaigns were further applied to any kind of collaborative study promoted by the group. The methodology followed by TC2 in setting up new analytical procedures is a stepwise approach: i) definition of objectives and minimum criteria ii) collection of proposals iii) preparation of a draft analytical protocol iv) experimental testing in a small group v) inventory of candidate materials vi) organisation of a laboratory inter-comparison vii) evaluation of the results and estimation of uncertainty This approach showed to be of great help in the process of method validation, i.e to demonstrate that the method itself is acceptable for its intended use.

2 1.1 First phase / Steps i-iii: Definition of minimum criteria and planning The first step in a validation process is always to identify the scope of the method and the minimum criteria that are making that method acceptable. Frequently asked questions are: - What is the minimum precision compatible with the scope of the method (% RSD)? - Is reagents purity well described and are they readily available to any lab? - Is the study method-dependent? - If it is an absolute method, is there a sufficient array of different analytical methods to compare with so as to prevent the occurrence of systematic errors? 1.2 Second phase/ Steps iv-v: Experimental testing within a small group of participants Before starting the inter-comparison, a preliminary investigation within a small group of laboratories is organised to evaluate whether the method is effective and fit to the purpose. Maximum care must be taken in evaluating the consistency of between-sample and withinsample homogeneity of the candidate material. It must be pointed out that even glass containers from the same production lot sometimes cannot be as homogeneous as they are supposed to be. In Fig. 1 the relative content of chromium and barium on the 2 sidewalls of 20 perfume bottles (candidate RM) is reported. One can see that barium is homogeneously distributed on both sides in all bottles, whilst chromium varies in the worst case by even 2.5% relative. Furthermore Cr results are clustered from 1 to 5 (box n 1), 6-10 (box n 2), (box n 3), (box n 4) as if boxes contained samples that were produced under conditions slightly different for the different boxes. To overcome this problem and to reduce the between-sample homogeneity the whole lot of bottles was crushed and the resulting coarse glass mixed together. Glass portions of 50 grams each were then selected taking small samples that were tentatively representative of the grain size distribution of the whole crushed material. Another purpose of the preliminary study is to investigate all key factors that may affect the results. In the example below (section 1.4) on the CrVI determination in glass, the influence of several parameters was highlighted such as respectively: the ph of the decomposition mixture, the role of each reagent in colour development, the presence of foreign ions like Ba and Pb that may cause turbidity, the interference from other polyvalent ions, the recovery of known amount of CrVI spiked to the decomposition solution.

3 Diagram of relative contents of chromium and of barium content /mean M3 1,030 1,020 1,010 1,000 0,990 0,980 Cr S1 Cr S2 Ba S1 Ba S2 0, running number of measurement Fig.1: The distribution of Cr/Ba relative contents on the 2 sides (S1and S2) of 20 perfume bottles. Other important questions were: What is the linearity range? How long is the coloration stable before fading? In fig. 2 a CrVI calibration plot is reported. The acceptability of linearity data is visualised by plotting the response factor vs concentration - the resulting points form a straight line nearly parallel to the x-axis. The response factor is calculated as (response y intercept)/concentration. OD 0,5 0,4 0,3 0,2 0,1 y = 0,0061x R 2 = 0,9989 CrVI calibration curve 0,007 0,0065 0,006 0,0055 Response factor ug CrVI 0,005 Fig 2: Calibration graph vs. amount of Cr(VI). Single square points show the response factors (see text).

4 1.3 Third phase / Steps vi-vii): Collaborative inter-comparison When all key factors have been successfully investigated, an analytical protocol can be prepared for distribution to the participating labs. The procedure must contain clear indication of what is left to the discretion of the analyst and what must be strictly followed. It is not always feasible to investigate deeply the homogeneity of the material, especially when the aim of the inter-comparison is to test a method rather than to certify a material. In case of glass samples coming from a production lot (sheets of a flat glass for example) it is assumed they are sufficiently homogeneous. The uncertainty in the sample homogeneity will reflect eventually in a larger inter-laboratory standard deviation. The samples are distributed together with an answer-sheet where labs are requested to report, for each of the elements investigated, their individual results obtained on at least 3 different decompositions on 3 different days. The results are then collected and discussed for the evaluation of outliers, normally on the basis of the 2 criterion (results falling outside the mean value 2 are considered outliers) and the mean value is iteratively calculated. 1.4 Case Study 1: Determination of hexavalent chromium in glass using one analytical procedure A typical example of method-dependent procedure, which has no absolute character, is the determination of hexavalent chromium in glass [6]. According to this method (1) a glass sample is decomposed by a strong acidic mixture at room temperature, then (2) a specific colorant is added to form a purple complex with hexavalent chromium. The scope of the procedure was to establish a method, highly specific to CrVI, suitable to quantify a concentration range from a few ppm up to 100 ppm CrVI in glass to comply with requirements set up by the Packaging Directive 94/62. The investigated method demonstrated very specific and the quantification limit was well fit to the purpose. But the procedure must be followed very strictly for even slight changes to the ph of the decomposition mixture and/or to the prescriptions for colour development may produce extremely different results. In the preliminary stage of the investigation, recovery experiments under slightly different conditions brought to recoveries varying from 35 to 96%. In Table I the summary of the results obtained on a CrVI containing glass is reported. The table reports the number of participating labs and accepted values, together with the repeatability and reproducibility coefficients (S r and S R respectively). In this case, where the results are strictly related to the method, the uncertainty of the procedure is equal to the between-laboratory variance, multiplied by a factor 2 (expanded uncertainty). The uncertainty of a procedure is generally obtained by summarising stochastic and systematic errors. The stochastic error is given by the between-laboratory standard deviation the interlaboratory standard deviation is assumed to be already included whilst the contribution of the procedure to the systematic error is assumed to be zero just because the results are related to this procedure only. Therefore it is sufficient to consider the contribution of the between-lab standard only.

5 1.5 Case Study 2: Determination of boron in glass using different analytical procedures Conversely, the study on the determination of boron in glass by XRF (7), was carried out in comparison with other absolute methods (same decomposition procedure but different methods of quantification). In this case it was agreed that the best estimate of the true value was given by the wet chemistry mean, that could be used as the target value to estimate the XRF performance and to identify XRF outliers (see Fig.1) Lab n Before separation After separation Remarks n x s n x s ** ** Centrif Informative a Filtration Filtration 8 * * Centrif. Filtration a:2g decomp Centrif Filtration Filtration Informative Centrif Centrif Filtration Informative Centrif Filtration Centrif. Overall = = *Statistical Outlier ; **Analytical Outlier Statistics where 37.3 n number of decomposition in each lab S r 0.82 x mean value in each lab S R 2.81 s standard deviation in each lab CV 7.86 % overall mean value I 14 S r coefficient of repeatability (within-lab variance) L 12 S R coefficient of reproducibility (between-lab variance) n 51 CV coefficient of variance N 41 I total number of labs L accepted number of labs n total number of determinations (decompositions) N accepted number of determinations Table I : Collaborative inter-comparison on a glass sample produced at laboratory scale. Results are given in mg/kg Cr 6+ before and after separation of a slight turbidity. Single determinations are given for information only

6 13 12, bis 9 9J 1 11, , A 6 13B 10 XRF B2O3 Wet chemistry mean Fig. 3 : A pharmaceutical glass, distribution of XRF boron results versus wet chemistry target value. Obvious outliers are labs n 10, 10bis, 9, 9J, and 1. How to estimate the systematic error of the procedure? One can use a certified reference material and compare the experimental results obtained on the CRM sample with the certified values. An example of this calculation is shown in reference [4]. In their conclusion, they found for this case that the largest contributions to the estimated uncertainty come from the uncertainty of the certified reference material and the difference between result and the certified value of the CRM. Another possibility is to use several validated methods and compare the results obtained. In both cases the differences can be considered as an estimate of the systematic error of the procedure. In Table II the results obtained on 6 different samples of boron containing glasses are reported. The mean is a selected mean after removing the outliers (flagged by *) identified by comparison with wet chemistry target value (see Fig.3 for example). To evaluate the performance of the XRF method in comparison with wet chemistry validated procedures, an F-test was made to compare the correspondent standard deviations. Calculated F-values are compared with critical F-values found in statistical tables for P=0.05. With a probability level of 95% it was found that for samples A, B and F the calculated values are higher than the critical value. For these samples the standard deviations are significantly different, which means that results obtained by XRF are affected by systematic errors due to the technique itself (poor sensitivity, line overlap, matrix effect, etc.).

7 XRF results (% B 2 O 3 w/w) Sample ID A B C D E F lab. N Perfume CRIR05 CRIR04 Pharma Duran H. Boron 1 # 0,85 4,32 7,41 12,49* 13,08* 20,5 5 0,50* 4,54 7,26 10,69 11,2* 17,7 6 1,15 5,44* 7,41 10,94 12,94 18,34 7 1,10 4,65 7,9 10,76 12,6 18,0 8 0,94 10,73 11,47* 17,6 9 1,30 12,3* 12,9 19,0 9J 1,13 12,3* 13,0 20,2 10 1,05 6,2* 9,9* 11,7* 11,78* 20,2 10B 1,24 6,0* 10,2* 12,2* 19,7 13 1,09 5,08 8,41* 10,9 12,64 18,5 14 0,95 5,51* 8,98* 10,64 12,49 19, ,50* 3,84* 7,76 10,5 12,3* 16,4 16 4,53 7, ,83 10,8 12,0* 16,6 mean XRF 1,06 4,62 7,62 10,75 12,76 18,62 sd 0,15 0,28 0,30 0,14 0,21 1,34 %rsd 14,31 6,09 3,89 1,31 1,65 7,21 No of acc. Results No of outliers Probability from a F-test comparing standard deviations with those from F-test wet chemistry. Sample ID A B C D E F F-calculated 19,5 6,1 2,3 1,5 1,2 37,5 F-critical 6,62 4,72 4,72 5,52 5,29 3,72 Conclusion OK OK OK # Results for boron obtained by difference (see text) * Outlier Table II: Summary of boron XRF results obtained on 6 different glass samples. The mean is a selected mean after removing the outliers (flagged by *) identified by comparison with wet chemistry target value. The F-test is made to compare the standard deviations with those from wet chemistry (not reported here). Calculated F-values are compared with critical F-values found in statistical tables for P=0.05. With a probability level of 95% it was found that for sample A, B and F the calculated values are higher than the critical value. For these samples the standard deviations are significantly different.

8 Section 2 : PROFICIENCY TESTING 2.1 A tool to demonstrate competence in the laboratory Laboratories are today required to take appropriate measures to improve the quality of their results and to establish traceability of their measurements to make them comparable with those produced by other laboratories operating under quality control procedures. One measure is accreditation, requiring the analyst (1) to follow defined analytical procedures, (2) to constantly improve their work, (3) to look carefully at all possible sources of uncertainty for its quantification and (4) to demonstrate the level of competence in the laboratory. A tool in this standard to demonstrate laboratory competence to the accreditation body and indeed to others is to participate in collaborative work and Proficiency Testing Schemes (PTS). However, there is a lack of suitable PTS for many specialized laboratories. In Technical Committee 2 within the International Commission on Glass (ICG/TC-2) this was recognized and the members were invited to participate in PTS for glass and glass related raw materials. The PTS is a stepwise process that includes: a) Identification of suitable homogeneous samples, b) Distribution of samples and procedures, c) Allowing laboratories time for analysis, d) Collecting results, e) Evaluate, and f) Report. There are obvious reasons to participate in collaborative exercises: a) Requested by accredited laboratories b) Objective evaluation of laboratory performance, c) Comparison of own results with other laboratories, d) Evaluation of consensus values for working standards, e) Evaluation of new analytical routines, f) Quality improvement via corrective actions. Apart from the more obvious reasons to participate, the discussions during presentation of the PT outcome reveal also other possibilities of the comparisons: (1) Evaluating semi-quantitative packages with XRF showed in many cases acceptable results fully in line with our traditional methods for soda-lime glass. (2) One of the participating laboratories (lab. n.1/table II) used XRF to analyse boron by difference. To do this, several conditions had to be fulfilled: All elements of the glass matrix have to be known and analysed. In the study, the presence of unknown elements was excluded; all elements of the different matrices can be analysed by standard XRF. High precision and high accuracy of the measurements of the matrix elements are required. Results are only valid if obtained within the calibrated range of glass compositions. A matrix correction based on fundamental parameter equations for XRF is required in such a way that the matrix parameters for boron oxide are equal to the unity in relation to all other matrix elements. On the basis of theory of matrix correction of XRF measurements (e.g. MEDUCUS 1986) this is possible because one degree of liberty is always set to a free value in the (mathematical) matrix calculation. This degree of liberty is often used to eliminate the influence of the dominant element (Fe in steel, Si in glass analysis), but can also be used for e.g. boron determination.

9 In these collaborative comparisons identical samples are sent to all participating laboratories. The homogeneity needs to be high. Even in a limited group of 15 laboratories, this needs to be addressed before starting the exercise. To ensure homogeneity for glass samples, these are taken from the same production lot at the time of production. For sheet glass enough samples can be prepared from the same sheet of glass making this less of a problem. However, for containers as experienced with coloured perfume bottles on one occasion and referred to in the first section above we found during our initial evaluation that we were about to distribute two sets of containers apparently produced on different days. To overcome this the containers were crushed and mixed. In case the homogeneity needs to be verified, the group has mostly selected solid samples suitable for X-ray Fluorescence (XRF) characterisation. The technique is fast and as only relative results are needed the results can be evaluated without tedious calibrations. For raw materials as analysed by XRF, homogenised samples are distributed without being ground to its final specification. This is because grinding needs to be performed using the laboratory routines, as precise results are requested. 2.2 PTS evaluation All our evaluations include oral presentations of the outcome. In these evaluations most if not all laboratories participate. The written evaluation includes calculation of the arithmetic mean value and the standard evaluation. Using these values a z-score is calculated and presented graphically for each result. Removal of obvious outliers is a necessity. However, in the past the removal of outliers was not always done by statistical means. After our discussions, the final evaluation is left to the laboratory itself. The evaluation procedure - using the standard deviation of the group - produces a range of z-scores from 2 to +2. As it is important to evaluate also the long-term performance of a laboratory, fixed values can be used in the evaluations. In our discussions we often ask the question What precision is needed for your customer/ production department. Using such practical values for each parameter the long-term performance can be assessed more readily. 2.3 Reports from PTS In the past the results and z-scores were presented to the participants during our bi-annual meetings and a small written summary was produced. During our recent meeting a more conclusive set of reports were asked for: (1) Report with all results for presentation to third party. (2) Report with your own results highlighted for ease of identification. (3) Certificate of Participation. 2.4 Guidelines for future inter-comparisons in TC-2 New guidelines for future inter-comparisons are under preparation. These guidelines will ensure that appropriate actions are taken to achieve the purpose of the inter-comparisons.

10 Section 3 : Conclusions Establishing suitable analytical procedures for the analysis of glass and for the estimation of its chemical durability has always been the mission of Technical Committee 2. (1) Method validation, (2) proficiency testing, and (3) establishment of working standards are all important tasks of the committee. Proficiency Testing is complementary to other quality assurance tools for laboratories but very appropriate as it monitors the laboratory s output. However, each laboratory needs to make long-term evaluation of its performance and to take appropriate actions. This is also addressed in the latest version of ISO 17025:2005 used in the accreditation of laboratories. References (1) Technical Committee 2: Collection of publications on chemical analysis and chemical durability of silicate glasses, The International Commission on Glass, (2) Technical Committee 2: Collection of publications on chemical analysis and chemical durability of silicate glasses, The International Commission on Glass, (3) Quevauviller, Ph; Guadagnino, E.: The certification of the mass fractions of antimony, arsenic, barium, cadmium, chloride, chromium, cobalt, lead and selenium in glass, BCR information European Commission, EUR EN (1999), 56 pages. (4) Dette, A., Guadagnino, E., Matschat, R., Sundberg, P.: Certification of the mass fractions of total and hexavalent chromium in glass: a joint collaborative work in cooperation between BAM and ICG, Riv. Staz. Sper. Vetro, I, 34 (2004), 1, 5-14 (5) Guadagnino, E., Matschat, R., Dette, A., Sundberg, P.: Certification of the mass fractions of trace elements in glass to be used for multi-element determination by XRF: a joint collaborative work in cooperation between BAM and ICG, Riv. Staz. Sper. Vetro, I, 35 (2005), 4, (6) Guadagnino, E., Sundberg, P. & Corumluoglu, O.: A collaborative study on the determination of hexavalent chromium in glass, Glass Technol., Vol.42, No6, December 2001, (7) Guadagnino, E., Sundberg, P. & Brochot, D.: A collaborative study into the determination of boron in glass using x-ray fluorescence (XRF) spectroscopy, Glass Technol.: Eur.J.Glass Sci. Technol. A, Vol.47, No4, August 2006, (8) Noack, S and Hässelbarth W., Concept for Expressing Measurement Uncertainties in Chemical Laboratories of Department I of the BAM. Distributed in TC-2 as document ICG/TC-2/ (22 pages)