7 Internal quality control

Transcription

1 7 Internal quality control 7.1 Summary In this chapter, classical Shewhart control chart Dioxin analysts have to face with an increasing number of congeners included in the so-called TEQ value. Checking individually their behaviour on quality control chart is time-consuming and almost unachievable in the context of routine analysis. Based on our analytical method, we propose to split the TEQ into three sub-groups (PCDD/Fs, N-O PCBs and M-O PCBs) and to use a multi-levels quality control approach. The degree of correlation between the sub-groups is intrinsically related to our analytical method (see chapter 2). The correlation between the levels can be interpreted by computing the Hotelling T 2 test. A T 2 > 1 gives a warning value. It indicates that an anomaly is detected even if each individual control material is in-control. Only in this case, further investigations can be triggered in order to determine its origin. As example, we report the application of this approach to a huge epidemiological study where 1000 of serum samples were analyzed for the determination of 35 target congeners in 6 months. The assessment of the tendency was performed by using a moving average calculated by the Exponentially Weighted Moving Average (EWMA) technique using a smoothing factor λ = 0.2. This technique is much more sensitive to bias detection compared to the classical Shewhart chart approach. Thus, the reference material implemented as internal quality control helped us to support trends analysis for quality insurance according to ISO requirements. 7-1

2 7.2 Introduction Nowadays, laboratories must be able to produce reliable data when performing analytical tests for a customer or for regulatory purposes and this, regardless the type of method and application areas. In addition, there are many situations in which the simultaneous monitoring or control of two or more related quality - process characteristics is necessary. As defined in the Harmonized Guidelines for Internal Quality Control in Analytical Chemistry Laboratories : internal quality control (IQC) is a set of procedures undertaken by laboratory staff for the continuous monitoring of operations and the results of measurements in order to decide whether results are reliable enough to be released. (Thompson and Wood, 1995). Above all, IQC is a control of the precision of your analytical process with the aim of assuring a long-term constancy of the results. It can also be a control of trueness depending of the control material (CM) used. The main objective is to ensure the constancy of the results day-to-day and their conformity with defined criteria. Internal quality control keeps constant evolution in the industrial world. Introducing new QC methods derived from the industrial practice to analytical chemistry, improving data evaluation and allowing detecting shifts or trends, are elements that are difficult to detect with classical approaches. The importance of ARL (average run length) as key-criteria of the efficiency of a quality control procedure will be emphasized. The introduction of the multivariate approach of multilevel control with the Hotelling's T 2 -test will lead to a better detection of random errors than the independently managed conventional Shewhart charts. Moreover, the Exponentially Weighted Moving Average (EWMA) or the cumulative sum (CUSUM chart) will offer a flexible tool for detecting the inacurracy of a method, especially when small shifts or bias are of interest. All these relatively new concepts, recently introduced in clinical chemistry, were applied here for the monitoring of dioxins and PCBs at ppt level in foodstuffs and feedingstuffs. 7.3 Statistical quality control. The basic approach to IQC involves the analysis of control materials alongside the routine test samples. CMs can take different forms such as procedure blank samples, certified reference materials (CRMs), reference materials (RMs), duplicate samples, blind samples, spiked samples. Here, RMs mean matrix quality control materials (QCMs) that are characterized by a 7-2

3 sufficient homogeneity and stability for long-term control of the laboratory performances. QCMs should, if possible, be representative of the real sample analyzed as they are treated in exactly the same way during the analytical procedure. QCMs should also match as much as possible the requirements of sample intakes, matrix composition/interferences and the concentration range of the analyte of concern Internal quality control and fitness for purpose Control charts are set up based on estimates m (mean) and s (the standard deviation of the mean) of the parameters µ (true value) and σ (true standard deviation of the method) calculated with a limit number of runs during a preliminary period. During that period, the assessment of m and s and later the acceptable range of [m±2s] and [m±3s] is a pivotal step for the set up of the QC chart. Indeed, two crucial parameters have to be taken into account: n the minimum number of data points and the experimental conditions (within-run and between-run). As the extension of the preliminary period to one hundred results is completely unrealistic for dioxin analysis due to economical factors (see Annex), the laboratory can establish a priori its own criteria of analytical precision derived from considerations of fitness for purpose. The fitness for purpose criteria are based on results from validation studies with the ultimate goal of expressing them in terms of acceptable combined uncertainty. In our field of application, Commission Regulation 1883/2006 set analytical requirements for PCDD/Fs and DL PCBs in food and feed (e.g. intermediate precision RSD<15%). Control limits calculated here from pre-established criteria of RSD must therefore be narrower than the requirements of fitness for purpose or the analysis would be futile. A 10% RSD was selected as target within-laboratory precision in the working range close to maximum levels Selection of QCMs in IQC for dioxins and related compounds Certified Reference Material (CRM) When available and of suitable composition for matching analyte, level and matrix, CRMs are ideal candidate as they can also be used to assess accuracy and traceability. The drawbacks of CRM are availability, cost and amount of sample needed for IQC purposes, explaining why most of the CRMs are used for validation and not for routine applications. We used one CRM as QCM for indicators PCBs in pork fat. The CRM 446 has been developed after the Belgian dioxin crisis and produced at IRMM (Geel, Belgium). The level 7-3

4 and the matrix are closely matching the analyses to perform according to the Belgian legislation setting maximum level for the sum of the seven indictor PCBs at 200 ng g -1 in fat of animal origin. The construction of the QC chart presented here is the easiest case as m and s are respectively set by the certified value of the CRM and the control limits enforced by an official Belgian method (Beltest I014, 2000). Figure 7-1 shows the results of long-term within laboratory reproducibility for CRM-446 by inserting it in the series of routine samples. The central green line defines the certified value of the CRM while the acceptable bounds, set by the official method at %, are drawn in red. Relevant information as mean value and standard deviation close to the level of interest are directly extracted from this chart. The evaluation of intermediate precision and method bias can be assessed and used to support an important part of the method validation. In addition, relevant data are available to support the uncertainty estimation close to a regulatory limit. This information is considered as a key indicator of fitness for purpose. Figure 7-1: Control chart of GC/MS/MS method for indicator PCBs in pork fat 7-4

5 Reference Material (RM) If an appropriate RM is available, it is a suitable way to implement an IQC system in laboratories. As RMs are characterized by sufficient homogeneity and long term stability, they can be implemented in the daily routine work. The assigned value (not certified) is used to build the QC chart. Action limits are then plotted after the determination of the target standard deviation. Figure 7-2 represents the implementation of a RM for PCDD/F congeners in milk powder (RM 533). Data are expressed in pgteq g -1 milk powder. The reference value is 2.86 pgteq g -1 milk powder (central green line) while the upper and lower control limits (set at m±3s) are drawn in red. RM533 perfectly match the specific milk powder matrix and the levels of real samples. It provides the long-term intermediate precision of the analytical method. The RSD is calculated with the contribution of 255 RMs implemented in the daily routine work covering almost 5 years of measurements (from March 2000 to February 2005). The RSD calculated is 9,3% (outliers removed). Four data points corresponded to an out-ofcontrol situation. Those situations were connected with particular events that happened in the laboratory (i.e. mainly contamination cases). An ARL of 255/4 = 64 and a frequency of rejection of 1.6% is observed (see annex). Figure 7-2: QC chart from RM 533 sum of PCDD/Fs in TEQ in milk powder 7-5

6 Figure 7-3: Distribution of data points for IQC of PCDD/Fs in TEQ in milk powder RM 533 Figure 7-3 shows the distributions of the 251 (i.e. the four outliers were removed) values. It tends to normality even if the distribution is a little bit asymmetric due to a higher proportion of values between 2.4 and 2.6 pg-teq/g milk powder. Note: the same type of QC chart can also be constructed with spiked matrix materials EWMA approach and multi-levels quality control Although a conventional Shewhart chart (see Annex, Figure 7-6) does well in terms of random error monitoring, it has a certain lack of sensitivity to detect systematic errors. The exponentially weighted moving average (EWMA) control chart is better suited for this purpose. It is a specific method for improved bias detection and it is defined as a statistic for monitoring the process that averages the data in a way that gives less and less weight to data as they are further removed in time from the current measurement (Neubauer, 1997). A parameter λ, called the smoothing factor, determines the rate at which older data enters into the calculation of the EWMA statistic (see Annex). Figure 7-4 illustrates the IQC charts. In Figure 7-4 A C, the central green tangent line defines the mean value (m) with the upper and 7-6

7 lower control limits drawn in plain (red). The control limits are set at m±3s, where s is the standard deviation recalculated each time a new data point is added in the dataset (floating chart). The tick curve with its control limits (dashed lines, m±3s EMWA ) represents the EWMA with a smoothing factor of 0.2. The relationship between s and s EMWA is expressed by the following equation: λ s EWMA = s (7-1) 2 λ By setting λ = 0.2, Equation (7-1) becomes: s s EWMA = (7-2) 3 Hence, the dashed lines represent m±s. The IQC used for this study was a matrix quality control material (foetal calf bovine serum) that was characterized by a sufficient homogeneity and stability for long-term control of laboratory performances. A batch of three litres of serum was prepared to cover 1 year period. The batch was spiked with dioxins, furans and dioxinlike PCBs. More than 90 tests were performed during this period including homogeneity tests and technicians evaluations (IQC used as blind samples). The results are summarized in Figure 7-4. No data points were in an out-of-control situation during a 10-month period. An intra-laboratory reproducibility expressed in RSD (%) of 5.9% (n = 91) for the sum of PCDD/Fs was obtained at pg TEQ/l (i.e. 26 pg TEQ/g lipids when assuming 0.6% of lipid). RSDs of 6.8% (n = 91); 9.8% (n = 89) were obtained for the sum of NO-PCBs at pg TEQ/l (17.8 pg TEQ/g lipids) and for the sum of MO-PCBs at 16.1 pg TEQ/l (2.7 pg TEQ/g lipids), respectively. The trends or the drift of the analytical method was evaluated by the EWMA curves. The EWMA curves were randomly distributed above and below the mean values indicating that no systematic bias were observed during this period. In addition, the EWMA curves lay between the dashes lines demonstrating that the bias did not exceed 5.9% (see Equation (7-2)) for the sum of PCDD/Fs; 6.7% for the sum of NO-PCBs and 9.8% for the sum of MO-PCBs Mutli-levels quality control for dioxins analysis Although the multi-levels IQC is frequently used in clinical chemistry, its application in ultratrace analysis is rare. The classical approach of multi-levels IQC is characterized by one parameter controlled at three different levels: high, medium and low. 7-7

8 Figure 7-4: Internal quality control (IQC) charts for PCDD/Fs (A), NO-PCBs (B) and MO- PCBs (C) present in QC serum analyzed over time (concentrations in pg TEQ L -1 ). (D) The Hotelling index T 2 The correlation between the levels has been introduced and developed by Hotelling (Hotelling, 1947). He defined an index that combines dispersion information, means and correlation of several variables. This scalar, known as T 2, generalizes at p dimensions the Student s t-test. This concept and the underlying statistics were used and adapted in the field of dioxin analysis. Instead of measuring each congener at different IQC levels, we divided the 29 toxics congeners into 3 sub-groups and we monitored 3 parameters: the sum of PCDD/Fs, the sum of NO-PCBs and the sum of MO-PCBs, all expressed in TEQ units. The IQC levels were, respectively, pg TEQ/l, pg TEQ/l and 16.1 pg TEQ/l for PCDD/Fs, NO- PCBs and MO-PCBs. The selection of the three sub-groups was related to our analytical procedure for clean-up, even if PCDD/Fs and NO-PCBs were collected in the same fraction. Once injected in parallel into the two GC/HRMS, the three monitored parameters were quantified and used to build up multi-level IQC charts (Figure 7-4). As the three variables came from the same analytical procedure, a degree of correlation between control levels should be observed. In other words, when a level (e.g. PCDD/Fs) increases, the corresponding 7-8

9 levels (e.g. NO-PCBs and MO-PCBs) should follow the same trend. This was quantified by Hostelling s T 2 test. A T 2 > 1 gives a warning value. It indicates that an anomaly is detected even if each individual control material is in-control. As an example, Figure 7-4 D shows a T 2 value of 1.04 on 30 September. Indeed, the PCDD/Fs and MO-PCBs values were close to the target mean value while the NO-PCBs value was close to the upper maximum limit (upper limit). Individually, values were accepted as they lay between the maximum limits. However, actions can be triggered in order to find out if there was any reason explaining such discrepancy. As the three selected parameters gave a general overview of the analytical performances in TEQ unit, one obvious action undertaken was to check individually the congeners that contribute to the TEQ. In this particular case, NO-PCB 126 was higher than expected and actions were taken by means of checking blank level, recovery calculations, integration of the peaks, peak shape, retention time, isotope ratio and relative response factor to ensure that was an isolate case of deviation. 7-9

10 7.4 Annex: statistics in quality control used in this chapter Basic statistics Shewhart Control Chart In 1931, Walter Shewhart invented the statistical quality control for applications in the industrial production process. Shewhart is considered as the father of the quality control chart. The basic idea of statistical control implies that an IQC result x can be interpreted by arising independently and randomly from a normal distribution N(µ,σ 2 ) characterized by a mean µ and a standard deviation σ (see Figure 7-5). Why? Because an experimental result is a random variable and the repeatability of the analytical process provides a variability (expressed as a standard deviation). Moreover, the normality assumption is assumed by the fact that there are a large number of variability sources, having independent effects and any of them are predominant. Almost twenty years passed before Levey and Jennings introduced statistical control methods in clinical laboratories (Levey and Jennings, 1950). More recently, at the beginning of the eighties, James Westgard created his own system of quality control rules which he baptized multi-rules (Westgard et al., 1981). Figure 7-5: Basic statistics to build up a QC chart. Until now, the shewhart chart is still considered as the fundamental tool in IQC. Under these constraints, there are only 0.3% of results that fall outside the range of µ ± 3σ. When this situation occurs, such extreme results are considered as out-of control. A Shewhart control chart is then obtained by reporting values of concentration measured on a QCM on the y axis against the run number on the x axis (time). The chart is completed by horizontal lines derived from the normal distribution N(µ,σ 2 ) at µ, µ ± 2σ and µ ± 3σ. Figure 7-6 illustrates a graphical representation of a Shewhart QC chart. It contains the central line 7-10

11 which defines the best estimate of whatever response variable is being plotted. Then, assuming normality, 95% of plotted values should fall within µ ± 2σ lines called the warning limits. The action limits (µ ± 3σ) define the acceptable bounds within which 99.7% of the plotted values must lie. µ + 3σ µ + 2σ µ + σ µ mean Upper action limit Upper warning limit µ - σ µ - 2σ µ - 3σ Lower warning limit Lower action limit Run number Figure 7-6: Representation of a Shewhart QC chart Unfortunately, in analytical chemistry, the mean µ and the standard deviation σ are rarely known (n ). When a QCM used is not a CRM, the initial conditions are not known. The QC chart is set up with neither the true value (µ) nor the target standard deviation (σ) of the method. Control charts are then set up based on estimates m (mean) and s (the standard deviation of the mean) of the parameters µ and σ calculated for a limit number of runs during a preliminary period. During that period, the assessment of m and s and later the acceptable range of [m±2s] and [m±3s] is a pivotal step for the construction of the QC chart. Indeed, two crucial parameters have to be taken into account: n the minimum number of data points and the experimental conditions (within-run and between-run). How many data points are needed to set up a control chart? To evaluate the minimum data points needed, Marquis (Marquis, 2001) studied the probability of false alarm outside the range [m±2s] by numerical simulation. Figure 7-7 represents the histogram distribution of the false alarm probability at a risk of 5%. One can note for n=20 that the frequency of false alarm between 1% and 15% is not unlikely to happen. The graph indicates that a preliminary period corresponding to 100 runs should be useful to guarantee an acceptable range of false alarm [2.7%-8.7%]. In many application areas 7-11

12 in analytical chemistry, like dioxin analysis, we have to be satisfied with a modest and realistic sampling size also based on economical considerations. Probability of false rejection (%) Figure 7-7: Numerical simulations: histogram of the distribution of false alarm probability for [m±2s] (Marquis, 2001) Generally, guidelines in analytical chemistry and in clinical field recommend a minimum of 20 runs during the preliminary period to assess m and s. The assessment is achieved in reproducibility conditions (between-run) by inserting the QCM at a frequency of one in a series of test samples. Within-laboratory reproducibility conditions are recommended in order to take into account most of the significant sources of variability, otherwise the control bounds would be too narrow and could increase the probability of false alarms The interpretation of control charts: the Westgard rules Basically the Westgard rules, also called the multirules QC, use a combination of decision criteria to decide whether an analytical run is in-control or out-control (Westgard et al., 1981). The procedure is characterized by different control rules to judge the acceptability of an analytical run. The diagram here below shows an overview of the classical Westgard rules (Figure 7-8). The flowchart is interpreted as follows: when inspecting the data in a QC chart, first look to see if all the control values are within 2s warning limits, which in case the run is judged to be in-control. If any single value exceeds the 2s limit, this is a warning of possible 7-12

13 problem. Then, you have to inspect the five others rules (1 3s, 2 2S, R 4s, 4 1s, 10 x ), usually following that order. If any single rule is violated, that confirms that there is a problem. The Westgard rules were adapted here for our purposes and the chart presented here is a variant of his model. Indeed, we estimated that the violation of rules 4 1s, 10 x should not lead to a rejection of the analytical run but rather stimulate an investigation of causes of a possible problem. From our point of view, those two rules are not the reflection of an out-of-control situation. They have to be treated as preventive actions to be undertaken before an out-ofcontrol situation occurs. Note that many other common multi-rules abound in the literature like the WECO rules (Western Electric Company) or FORD rules. Start Observation more than 2s from mean No Accept value Observation more than 3s from mean (1 3s) yes No 2 consecutive observations more than 2s from mean (2 2S) No Range of 2 consecutive observations exceeds 4s (R 4s) No 4 consecutive observations more than 1s from mean (4 1s) No No 10 consecutive observations same side of mean (10x) yes yes yes yes yes Out-of-control, reject analytical run Alarm, preventive maintenance Figure 7-8: Adapted Westgard Rules diagram Average Run Length (ARL) The objective of the Shewhart chart is to locate the data out-of-control. The use in IQC system is to count them in terms of frequency of rejections. The QC theorists resort more readily to the concept of run length (RL) (Marquis and Masseyeff, 2002). It acts simply as the distance between two successive rejections, measured in a number of points or time. The average run length (ARL) tells us, for a given situation on average, how long we will plot successive control charts points before we detect a point beyond the control limits. It is nothing other than the reverse of the frequency of rejections. An efficient IQC system is therefore characterized by a high ARL. When a statistical normal distribution of the analytical error is 7-13

14 postulated, the frequency of rejection tends towards the theoretical probability if the number of data points is sufficiently large. The ARL for Shewhart equals 1/p, where p is the probability for a point that falls outside established control limits. Thus, assuming normality for 3σ control limits, the probability to exceed the upper control limit = and to fall below the lower control limit is also and their sum = (These numbers come from standard normal distribution tables or computer programs, setting z = 3). Then the ARL = 1/.0027 = 370 (see Figure 7-9). This explains that when a process is in control, one expects an out-of-control signal (false alarm) each 370 runs. It means that with a daily check, one must expect, on average, a false alarm each 370 points, i.e., once a year. The frequency of rejections is equal to 0.27%. Now, when the mean shifts down by 2σ, as represented on the right part of Figure 7-9, the distance between the lower control limit and the shifted mean is then 1σ (instead of 3σ). This yields a probability of p = to exceed the lower control limit value. The distance between the shifted mean and the upper limit is now 5σ and the probability of exceeding the upper limit is negligible and can be ignored. The ARL is therefore 1/ = 6 and the frequency of rejections increases to an unacceptable value of 16%. Figure 7-9: ARL under-control and out-of-control (From P. Marquis, MultiQC, France) Advance in Quality Control An analytical result is characterized by a systematic error and a random error. The faster their detection, the better the method of control. 7-14

15 Exponentially Weighted Moving Average (EMWA) control chart Because it takes time for the patterns in the data to emerge, a permanent shift in the process may not immediately cause individual violations of the control limits on a Shewhart control chart. The Shewhart control chart is not powerful for detecting small changes, say of the order of standard deviations. The EWMA (exponentially weighted moving average) control chart is better suited for this purpose. Therefore, it is a specific method for improved bias detection (applied to systematic errors). It has been invented in 1954 and introduced for the first time in clinical chemistry by Neubauer in 1997 (Neubauer, 1997). Neubauer demonstrated the superiority of this technique compared to Westgard mutli-rules. Definition: The exponentially weighted moving average (EWMA) is a statistic for monitoring the process that averages the data in a way that gives less and less weight to data as they are further removed in time from the current measurement. The statistic that is calculated is based on the recurrence formula: EWMA t = λy t + ( 1- λ) EWMA t-1 for t = 1, 2,..., n. (7-3) where EWMA 0 is the mean of historical data (target) Y t is the observation at time t n is the number of observations to be monitored including EWMA 0 0 <λ 1 is a constant that determines the depth of memory of the EWMA. The parameter λ (called the smoothing factor) determines the rate at which 'older' data enters into the calculation of the EWMA statistic. A value of λ= 1 implies that only the most recent measurement influences the EWMA (degrades to Shewhart chart). Thus, a large value of λ gives more weight to recent data and less weight to older data; a small value of λ gives more weight to older data. The value of λ is usually set between 0.2 and 0.3 although this choice is somewhat arbitrary; 0.15 is also a popular choice. Figure 7-10 shows that with a value of λ = 0.25, the EWMA takes into account around ten previous points with a high predominance of the five last points, whereas a λ = 0.05 goes back much further in time, beyond twenty previous points. 7-15

16 λ λ Figure 7-10: Weighting of EWMA for different λ values (From P. Marquis, MultiQC, France) Multivariate quality control Since the multi-levels IQC is frequently used in clinical chemistry, its application in classical chemistry analysis is rare. The techniques treated jointly several variables within the same mathematical object: vectors or matrices. In 1947, Hotelling defined an index that combines dispersion information, means and correlation of several variables. This scalar, known as T 2, generalizes at p dimensions the student t test. Three levels are generally utilized: high, medium and low levels. More information can be found in the following reference: (Hotelling, 1947). 7-16