AFNOR CERTIFICATION VALIDATION OF THE METHOD. COLILERT 18 with QUANTI TRAY or QUANTI TRAY For the enumeration of Escherichia coli

Transcription

1 NF VALIDATION AFNOR CERTIFICATION VALIDATION OF THE METHOD COLILERT 18 with QUANTI TRAY or QUANTI TRAY 2 For the enumeration of Escherichia coli Protocol for bathing waters SUMMARY REPORT JANUARY 217 V1 Expert laboratory : Manufacturer : ISHA IDEXX Laboratories, Inc. 25 avenue de la République IDEXX Drive, Westbrook 913 MASSY Maine 4 92 FRANCE USA This report of analysis concerns only objects subjected to analysis. Its reproduction is authorized only in the form of complete photographic facsimile. It contains 54 pages. Only some assays reported in this document are covered by the accreditation of the Section Laboratory of COFRAC. They are identified by the symbol (*). Assays realized at ISHA : 25 avenue de la République, 913 Massy, France.

2 Table of contents 1. Introduction Validation repository and validation history Alternative method Application scope Reference method Methods comparative study Relative accuracy Number and nature of samples Results Statistical analysis Conclusion Linearity Contamination levels Results Statistical analysis Conclusion Limit of detection and limit of quantification Test protocols Results Conclusion Selectivity Test protocols Results Conclusion Practicability Interlaboratory study Study organisation Participating laboratories E. coli absence in the matrix Strain stability in the matrix Samples preparation and spiking Samples labelling Samples shipping Samples reception and analysis Results Institut Scientifique d'hygiène et d'analyse 2/54

3 Temperature and state of the samples Total viable counts Expert laboratory and collaborating laboratories results Interpretation Bias calculation Accuracy profile Extension study Results and interpretation Results from Enterolert E / Quanti Tray 2 comparative study Results from Colilert 18 / Quanti Tray study Conclusion Conclusion Institut Scientifique d'hygiène et d'analyse 3/54

4 1. Introduction This summary report presents the results of the validation study, under the brand NF Validation, of the method Colilert 18 / Quanti Tray or Quanti Tray 2 developed by IDEXX for the enumeration of Escherichia coli in bathing waters Validation repository and validation history The aim of this validation study is to evaluate the performance of the alternative method against the reference method NF EN ISO 938 3: The different steps of the validation are the following: Step: Date: Version of the validation protocol for an alternative commercial method as compared to a reference method: Initial validation June 212 Version 1, May 21 Extension November 214 Version 2, May 213 First renewal June 216 Version 2, may Alternative method Colilert 18 detects E. coli in bathing waters. It is based on IDEXX s patented Defined Substrate Technology (DST): when total or fecal coliforms metabolize Colilert 18 s nutrient indicator, ONPG, the sample turns yellow, when E. coli metabolize Colilert 18 s nutrient indicator, MUG, the sample also fluoresces. Colilert 18 can simultaneously detect these bacteria at 1 CFU/1 ml within 18 hours even in the presence of as many as 2 million heterotrophic bacteria per 1 ml. The protocol of the alternative method is presented in figure 1. Figure 1 : protocol of the alternative method 1. Add contents of one pack to a 1 ml room temperature water sample in a sterile vessel. When Colilert 18 is used for E. coli detection in marine water, samples must be diluted at least tenfold. Multiply the MPN value by the dilution factor to obtain the correct quantitative result. 2. Cap vessel and shake until dissolved. 3. Pour sample/reagent mixture into a Quanti Tray or a Quanti Tray 2 and seal in an IDEXX Quanti Tray Sealer. 4. Place the sealed tray in a 36±2 C incubator for 18 hours to 22 hours (pre warming to 36 C is not required). For incubation in a water bath, submerge the Tray below the water level using a weighted ring. 5. Read results according to the Result Interpretation table. Count the number of positive wells and refer to the MPN table provided with the trays to obtain a Most Probable Number Application scope The application scope of the alternative method concerns the category bathing waters, which groups two types of waters: fresh waters sea waters. Institut Scientifique d'hygiène et d'analyse 4/54

5 1.4. Reference method The standard NF EN ISO (1999): Detection and enumeration of E. coli and coliforms part 3: miniaturized method (MPN) for detection and enumeration of E. coli in surface and waste water, was used as the reference method. The protocol of the reference method is presented in figure 2. Figure 2 : protocol of the reference method Dilutions preparation Dilute 9 ml of sample in 9 ml of special diluent (1/2) Transfer 1 ml of in 9 ml of special diluent (1/2) Inoculation Inoculate 2 µl of the 1/2 dilution in each of the first 64 wells of the microplate Inoculate 2 µl of the 1/2 dilution in each of the 32 wells of the microplate Incubation Cover the microplate with sterile adhesive Incubate the microplate at 44 ± 1 C for 36 h to 72 h Reading and interpretation Read results according to the Result Interpretation table. Count the number of positive wells using Wood lamp and refer to the MPN table provided with the trays to obtain a Most Probable Number. Express the result in MPN E. coli / 1 ml Institut Scientifique d'hygiène et d'analyse 5/54

6 2. Methods comparative study The following characteristics were studied during the comparative study of the methods: the relative accuracy, the linearity of the alternative method, the selectivity of the alternative method, the limit of detection and the limit of quantification of the alternative method, the practicability of the alternative method Relative accuracy The relative accuracy is defined as the closeness of agreement between test result and the accepted reference value Number and nature of samples Two types of water were tested (duplicate) with reference method and alternative method: freshwater and seawater. Different types of analyzed samples are summarized in table 1. Table 2 : Number and nature of samples analyzed Water type Number of samples analyzed Number of samples used Sea waters 53 2 Fresh waters Total Globally, 94 samples were analyzed and 42 results were used. 16 naturally contaminated samples were analyzed. Others samples were artificially contaminated (cf. appendix 1). The contamination levels used cover the entire measurement range of the alternative method Results Figure 3 presents the two dimensional graphs for the two matrices. The y axis is reserved for the alternative method and the x axis for the reference method.. The representation of a line of equation y = x figures dashed on the graphs. Raw results are in appendix 2. Figure 3 : two dimensional graphs for relative accuracy in log CFU and log MPN / test portion (black line: y=x) Institut Scientifique d'hygiène et d'analyse 6/54

7 Statistical analysis The relationship of relative accuracy between the reference method and the alternative method is evaluated with the linear model: 'y = a + bx '. This formula corresponds to the equation of the linear regression drawn from raw results obtained by experimentation, y representing the alternative method and x the reference method. There is a perfect accuracy (or there is no systematic bias) between the two methods if this equation is equal to the theoretical 'y = x' equation, which applies in the ideal model where the two methods behave similarly. The intercept is theoretically zero in this ideal model (hypothesis [a = ]). The estimated intercept obtained with the two methods is checked using p {a = }. If the alternative method is a systematic bias against the reference method, the probability p {a = } is less than α =.5. The 'b' slope is theoretically equal to 1 in the ideal model (hypothesis [b = 1]). The estimated slope obtained with the two methods should pass by p {b = 1}. Statistically, if the alternative method does not give the same values as the reference method, the probability p {b = 1} is less than α =.5. The linear regression method is chosen over the value of the robustness of the ratio R of overall repeatability standard deviation: If Rob.R > 2, linear regression by least squares (OLS 1) with the x axis for the reference method, if Rob.R <.5, a linear regression by least squares (OLS 2) with the x axis for the alternative method, If.5 < Rob.R < 2, orthogonal regression (GMFR) with the x axis to the reference method. Table 2 : statistical data (log MPN / test portion) for the enumeration of E. coli in bathing waters Probabilities (%) Matrix Rob.R Regression used T a t(a) b t(b) Slope at Ord. at 1 Sea waters 1.78 GMFR Fresh waters.926 GMFR Bathing waters 1.78 GMFR Institut Scientifique d'hygiène et d'analyse 7/54

8 Table 3 : bias and repeatability of the two methods (RM: reference method and AM: alternative method) Bias (D) in log Repeatability in log Matrix r Rob.r Mean Median MR MA MR MA Sea waters Fresh waters Bathing waters Sea waters The hypothesis [b = 1] is accepted but the hypothesis [a = ] isn t accepted. However, the correlation coefficients and equation are satisfactory as shown below: r =.984, log Alt. = 1.53 log Ref..232 Fresh waters The two hypothesis [b = 1 and a = ] aren t accepted. However, the correlation coefficients and equation are satisfactory as shown below: r =.979, log Alt. = log Ref..494 Bathing waters (seawaters + freshwaters) The two hypothesis [b = 1 and a = ] aren t accepted. However, the correlation coefficients and equation are satisfactory as shown below: r =.988, log Alt. = 1.87 log Ref..337 Remark: The limits of detection of the two protocols of the alternative method and of the reference method are different, based on different dilution factors and MPN tables: 1 MPN/1 ml for the alternative method in fresh waters, 1 MPN/1 ml for the alternative method in sea waters, 15 MPN/1 ml for the reference method. That s why, for fresh waters and bathing waters, if the data of the alternative method inferior to the limit of detection of the reference method are not taken into account (2 samples involved), the following values are obtained (data and calculations in appendix 3): Fresh waters: Bathing waters: r =.989, r =.991 log Alt. = 1.34 log Ref..158 log Alt. = 1.4 log Ref..18 With these values, the statistical exploitation shows that the two hypothesis [b = 1 and a = ] are accepted with α = 5% Conclusion The relative accuracy of the alternative method is satisfactory Linearity The linearity is the ability of the method when used with a given matrix to give results that are in proportion to the amount of analyte present in the sample, that is an increase in analyte corresponds to a linear or proportional increase in results. Institut Scientifique d'hygiène et d'analyse 8/54

9 Contamination levels The couples matrix / strain are presented in Table 4. For each couple, four contamination levels were tested in duplicate by the reference method and the alternative method. Table 4 : couples matrix strain analyzed Strain Matrix Target contamination level (CFU/1 ml) Escherichia coli ESC Fresh water Escherichia coli ESC Sea water Results Figure 4 presents the two dimensional graphs for the two couples matrix strain. The y axis is reserved for the alternative method and the x axis for the reference method. The representation of a line of equation y = x figures dashed on the graphs. Raw results are in appendix 4. Figure 4 : two dimensional graphs for linearity in log CFU and log MPN / test portion (black line: y=x) Statistical analysis Statistical interpretations are made according to requirements of standard NF ISO 1614 (see table 5). The choice of the linear regression method is compared to the value of the robustness of the ratio R of the standard deviations of repeatability overall: if Rob.R> 2, a linear regression least squares (OLS 1) is used with the x axis for the reference method, if Rob.R <.5, a linear regression least squares (OLS 2) is used with the x axis for the alternative method, if.5 <Rob.R <2, an orthogonal regression (GMFR) is used with the x axis to the reference method. Table 5 : statistical data for the linearity Strain / Rob. Regression F P Rob. F matrix R used critical (Rob.F) r Regression E. coli / sea water GMFR log Alt.=.859 log Ref +.44 E. coli / fresh water GMFR log Alt.=.891 log Ref Institut Scientifique d'hygiène et d'analyse 9/54

10 The relationship between the 2 methods is not linear: if Rob.F > critical F or, if P (Rob.F) < α (=.5) Conclusion The relationship between the two methods is linear for the two couples (E. coli / sea water and E. coli / fresh water). The correlation coefficients are satisfactory. So, the linearity of the alternative method is satisfactory Limit of detection and limit of quantification The detection and quantification limits are checked in accordance with the standard EN ISO Three parameters are determined. Here are their ISO 1614 definitions: the critical level (LC) is the smallest amount which can be detected (not null), but not quantified as an exact value. Below this value, it cannot be sure that the true value is not null. At this level, the false negatives probability β is 5 % (β is the second type of statistical error). the detection limit (LOD) is higher than the critical level, because it involves a power, the probability 1 β, which has to be well over 5 %, for example 95 %. the quantification limit (LOQ) is the smallest amount of analyte, (that is the lowest actual number of organisms), which can be measured and quantified with defined precision and accuracy under the experimental conditions by the method under validation Test protocols The limits of detection and quantification were determined by analysing a pure culture of E. coli by the alternative method. Five levels of contamination (including level ), with six replications for each level, were studied in sterilized water Results Results are shown in the following tables and in appendix 5. Table 6 : data (s and x ) of E. coli enumeration (underlined: the reference level) Level (CFU/1mL) Number of positive samples Standard deviation (s ) Bias (x ) Table 7 : LC, LOD and LOQ values of the alternative method Parameter Formula Values obtained Critical level (LC) 1.65 s + x 1.4 Limit of detection (LOD) 3.3 s + x 2.31 Limit of quantification (LOQ) 1 s + x Conclusion The detection limit and quantification limit of the alternative method are satisfactory. Institut Scientifique d'hygiène et d'analyse 1/54

11 2.4. Selectivity The selectivity of the alternative method is evaluated by its inclusivity and its exclusivity. Inclusivity is the ability of the alternative method to detect the target analyte from a wide range of strains. Exclusivity is the lack of interference by a relevant range of non target strains with the alternative method Test protocols Twenty E. coli strains and thirty non target strains (from the national, international and ISHA internal collections) were analyzed. The assays were performed by the alternative method protocol Results Raw results are in appendix 6. All target strains tested are detected by the alternative method except for one strain (which is not detected by the reference method either). For the thirty non target strains tested, no positive result was observed. See tables below Conclusion The selectivity of the alternative method can be considered as satisfactory Practicability The practicability was evaluated according to the 13 criteria defined by AFNOR Technical Committee. 1 Mode of packaging of test components The Colilert 18 reagent is conditioned on single capsules. The Quanti Tray devices are conditioned par ten in aseptic bag. 2 Volume of reagents Unknown. 3 Storage conditions of components and shelf life of unopened products The Colilert 18 reagent should be conserved at 2 8 C. The Quanti Tray devices should be conserved at 4 3 C. 4 Modalities after first use Each Colilert 18 test serves a unique analysis and should not be reused. 5 Equipment and specific local requirements Quanti Tray Sealer model 2X. Wood lamp. 6 Reagents ready to use or for reconstitution None. 7 Training period for operator with no experience with the method The duration of training is estimated to be 1 hour. 8 Handling time and flexibility of the method in relation to the number of samples The duration of analysis according the reference method is more important than the duration of use of alternative method. Institut Scientifique d'hygiène et d'analyse 11/54

12 9 Time required for results The time to obtain results for the alternative method is 18 hours for negative samples and positive samples. Concerning the reference method, the delay for negative samples is between 24 and 48 hours and for positive samples, the delay is between 48 and 72 days. 1 Operator qualification Identical as necessary for the reference method 11 Steps common with the reference method None. 12 Traceability of analysis results None. 13 Maintenance by laboratory None. Institut Scientifique d'hygiène et d'analyse 12/54

13 3. Interlaboratory study The main object of the interlaboratory study is to determine the variability of the results obtained by different laboratories analysing identical samples and to compare these results within the framework of the comparative study of the methods Study organisation Participating laboratories The interlaboratory study was realized by the expert laboratory and fifteen participating laboratories E. coli absence in the matrix Before spiking, the absence of E. coli was verified in the batch of seawater used according to the reference method Strain stability in the matrix The strain stability in seawater matrix was evaluated for 3 days at (5±3) C. The strain used was E. coli (ISHA code: ESC.1.119). The samples were analysed at D, D+1 and D+2 by the reference method. The results are summarized in table 1. Table 8 : results (E. coli / 1 ml) of the stability study of the strain ESC in seawater matrix Day Level 1 Level 2 Level 3 D D D The results show that the E. coli strain used is stable for 2 days at (5±3) C in SHW matrix Samples preparation and spiking The matrix was inoculated with the target strain suspension to obtain 4 contamination levels: level : CFU/1 ml, level 1 : from 5 to 1 CFU/1 ml, level 2 : from 25 to 5 CFU/1 ml, level 3 : from 1 to 15 FCU/1 ml. The matrix was distributed at 5 ml in sterile bottles. Every bottle was individually spiked and homogenized. Eight samples per laboratory were prepared (2 samples per contamination level). Each laboratory received 8 samples to analyse, 1 sample to quantify the endogenous microflora and 1 water sample containing a temperature probe. The results of the enumerations of the heterophilic flora, the target levels and the real levels of contamination are presented in table 9. Table 9 : target level, real level and TVC of the matrix Contamination Flora (CFU/mL) Escherichia coli ESC (MPN /1 ml) level 22 C 36 C Target level Real level 1 5 to to to Institut Scientifique d'hygiène et d'analyse 13/54

14 Samples labelling The labelling of the bags was realized as follows: a code to identify the laboratory: from A to O (cf. table 1) and a code to identify each sample, only known by the expert laboratory. The samples and the temperature control vials (water sample with a temperature probe) were stored at 4 C before shipping. Table 1 : sample code by contamination level Contamination level (MNP E. coli / 1 ml) Sample code 4 / 8 5 to 1 6 / 7 25 to 5 1 / 3 1 to / Samples shipping The samples were shipped in a coolbox April 16 th, Samples reception and analysis The coolboxes were received April 17 th, 212 by all the participating laboratories. The control temperature was recorded upon receipt of the package and the temperature probe sent to the expert laboratory. The samples were analysed on April 17 th, 212. The expert laboratory concurrently analysed a set of samples under the same conditions with both methods Results Temperature and state of the samples The temperature readings at reception, the state of the samples and the data from the thermal probe are shown in table 11. Table 11 : temperature and state of the samples upon reception and data of the temperature probes for the transportation time of samples (/: data not available) Laboratory Temperature ( C) Temperature recorded by the State of the probe samples Mean SD A 4.1 C Ok B 5.2 C Ok C 6.7 C Ok D 6.8 C Ok E 6.4 C Ok F 3.8 C Ok / / G 2. C Ok H 3. C Ok I 5.2 C Ok J 6. C Ok / / K 2.1 C Ok L 4.8 C Ok M 1.6 C Ok N 6.1 C Ok O 4.8 C Ok Institut Scientifique d'hygiène et d'analyse 14/54

15 The analysis of thermal profiles of probes showed for all participants that the average of temperature during the shipment is comprise between 1.6 and 3.7 C Total viable counts Raw results are in appendix 7. For the whole laboratories, the total viable counts at 22 C vary between <1 and 24 CFU/mL. Concerning the total viable counts at 36 C, the results were varying between <1 and 7 CFU/mL Expert laboratory and collaborating laboratories results The overall results are presented in Table 12 and in appendix 8. The results of the reference method are presented for a reading of the microplates after 36 at 72 hours of incubation at 44 ± 1 C. For alternative method, reading of Quanti Tray devices was performed between 18 and 22 hours. The results of all laboratories are presented in the following tables. Table 12 : E. coli MPN enumeration results per 1 ml seawater samples (MR: reference method, MA: alternative method, R1: repetition 1 and R2: repetition 2) Level Laboratory MPN/ 1 ml MR R1 R2 R1 R2 Low limit High limit MPN/ 1 ml Low limit High limit MPN/1 ml MA MPN/1 ml A <15 / / <15 / / <1 <1 B <15 / / <15 / / <1 <1 C <15 / / <15 / / <1 <1 D <15 / / <15 / / <1 <1 E <15 / / <15 / / <1 <1 F <15 / / <15 / / <1 <1 G <15 / / <15 / / <1 <1 H <15 / / <15 / / <1 <1 I <15 / / <15 / / <1 <1 J <15 / / <15 / / <1 <1 K <15 / / <15 / / <1 <1 L <15 / / <15 / / <1 <1 M <15 / / <15 / / <1 <1 N <15 / / <15 / / <1 <1 O <15 / / <15 / / <1 <1 Expert <15 / / <15 / / <1 <1 Institut Scientifique d'hygiène et d'analyse 15/54

16 Level 1 MR MA Laboratory R1 R2 R1 R2 MPN/1 Low High MPN/1 Low High MPN/1 MPN/1 ml limit limit ml limit limit ml ml A B C D <15 / / E F G H I J K L M N O Expert Laboratory MPN/ 1 ml MR Level 2 R1 R2 R1 R2 MPN/ MPN/ MPN/ Low limit High limit 1 Low limit High limit 1 ml 1 ml ml A B C D E F G H I J K L M N O Expert MA Institut Scientifique d'hygiène et d'analyse 16/54

17 Laboratory MPN/ 1 ml MR Level 3 R1 R2 R1 R2 MPN/ MPN/ MPN/ Low limit High limit 1 Low limit High limit 1 ml 1 ml ml A B C D E F G H I J K L M N O Expert Interpretation The data presented in the following paragraphs were calculated from the results in log 1 MPN/1 ml in the same way that the presentation of the results of the preliminary study Bias calculation Table 13 shows the target value, the mean, standard deviation of fidelity, the relative bias and the bias of each level of contamination for the alternative method. MA Table 13 : Calculation of the alternative method bias Values log (MPN /ml) Contamination level Low Medium High Target value Average Relative bias 8.93% 3.26% 2.26% Bias Accuracy profile Tables 14 and 15 show the values of tolerance and the tolerance limits of the alternative method for a probability value of tolerance of 8% (table 14) and of 9% (table 15). Institut Scientifique d'hygiène et d'analyse 17/54

18 Table 14 : Values and tolerance limits of the alternative method with ß = 8 % Probability of tolerance Niveaux Log CFU/L log (MPN /ml) Low Medium High Low tolerance value % High tolerance value Low tolerance limit High tolerance limit Table 15 : Values and tolerance limits of the alternative method with ß = 9 % Probability of tolerance Niveaux Log CFU/L Low Medium High Low tolerance value % High tolerance value Low tolerance limit High tolerance limit Figures 5 and 6 show the accuracy profiles using respectively ß = 8% and ß = 9%. Figure 5 : Accuracy profile of the alternative method with tolerance probability of 8 % and acceptability limits at,5 log Institut Scientifique d'hygiène et d'analyse 18/54

19 Figure 6 : Accuracy profile of the alternative method with tolerance probability of 9 % and acceptability limits at,6 log,8,6 Biais Limite haute tolérance (différence) Limite d'acceptabilité haute Limite basse tolérance (différence) Limite d'acceptabilité basse,4,2 Log 1, -,2 1,5 1,7 1,9 2,1 2,3 2,5 2,7 2,9 3,1 -,4 -,6 -,8 Niveaux Comments The accuracy profile obtained from the results of the reference method and the alternative method shows that the bias of Colilert method for the enumeration of E. coli in bathing waters is acceptable. The tolerance limits of the alternative method for a probability of 9% tolerance are included within the limits of acceptability of,6 log. Institut Scientifique d'hygiène et d'analyse 19/54

20 4. Extension study The aim of the extension study was to compare the results obtained with Colilert 18 or Enterolert E with the use of a Quanti Tray 2 or the use of a Quanti Tray, in order to allow the use of both devices in the framework of a certification NF Validation concerning each IDEXX kit using a Quanti Tray or a Quanti Tray Results and interpretation Two sets of results are available: ISHA data from the comparative study for the NF Validation certification of the method Enterolert E with Quanti Tray 2, IDEXX data from an analysis of a tap water using Colilert 18 associated with Quanti Tray 2 and with Quanti Tray Results from Enterolert E / Quanti Tray 2 comparative study Raw results Results have been collected from samples used in the comparative study for the validation of the method Enterolert E in the common enumeration range of the two devices, namely from 1 to 2 MPN/1 ml. A minimum of 1 results was asked by the Technical Board: it s a total of 18 samples that have been taken into account. Results are available in the summary report of the AFNOR Certification validation of the Enterolert E method. A two dimensional graph is shown in figure 7, presenting the results obtained with the Quanti Tray 2 (the validated Quanti Tray for the Enterolert E method) as the reference method. Figure 7 : Comparison of results obtained with Quanti Tray 2 and with Quanti Tray for the validation of the Enterolert E method 4, 3,5 Quanti Tray (log MPN/1 ml) 3, 2,5 2, 1,5 1,,5,,,5 1, 1,5 2, 2,5 3, 3,5 4, Quanti Tray 2 (log MPN/1 ml) Institut Scientifique d'hygiène et d'analyse 2/54

21 Statistical interpretation Validation protocol for an alternative commercial method as compared with a reference method: A statistical interpretation has been performed according to the requirements of the Validation protocol for an alternative commercial method as compared with a reference method, considering the Quanti Tray 2 as the reference device and using the tests for the relative accuracy. Results are available in the summary report of the AFNOR Certification validation of the Enterolert E method. According to this protocol, the relationship of relative accuracy between QT 2 and QT is evaluated with the linear model: 'y = a + bx'. This formula corresponds to the equation of the linear regression drawn from raw results obtained by experimentation, y representing the QT 2 devices and x the QT devices. There is a perfect accuracy (or there is no systematic bias) between the two methods if this equation is equal to the theoretical 'y = x' equation, which applies in the ideal model where the two methods behave similarly. The intercept is theoretically zero in this ideal model (hypothesis [a = ]). The estimated intercept obtained with the two methods is checked using p {a = }. If the alternative method is a systematic bias against the reference method, the probability p {a = } is less than α =.5. The 'b' slope is theoretically equal to 1 in the ideal model (hypothesis [b = 1]). The estimated slope obtained with the two methods should pass by p {b = 1}. Statistically, if the alternative method does not give the same values as the reference method, the probability p {b = 1} is less than α =.5. The results of the statistical tests are shown in the table below. Rob.R Regression used T critical a t(a) b t(b) Probabilities (%) Intercept at Slope at GMFR The equation for the regression line is as follows: log Alt = 1.4 log Ref.97. Hypothesis [a = and b = 1] is accepted for the comparison of the enumeration of enterococci with the Enterolert E method using a Quanti Tray versus a Quanti Tray 2. Student Fisher test A Student Fisher test has been also performed from the data obtained during the validation of the Enterolert E method. The results of the test are shown in the table below: t-test: Paired Two Sample for Means Parameter Quanti-Tray Quanti-Tray 2 Mean Variance Observations Pearson Correlation.883 Hypothesized Mean Difference df 35 t Stat P(T<=t) one-tail.346 t Critical one-tail 1.69 P(T<=t) two-tail.693 t Critical two-tail 2.3 Institut Scientifique d'hygiène et d'analyse 21/54

22 Both one tailed and two tailed tests conclude that there is no statistically significant difference between the enumeration of enterococci with Quanti Tray or with Quanti Tray 2 at α= Results from Colilert 18 / Quanti Tray study Raw results Results were obtained from IDEXX Company. An Escherichia coli suspension was spiked in a neutralized tap water from 3 to 18 CFU/1 ml and then analyzed with Colilert 18 associated with Quanti Tray and with Quanti Tray 2. Results are available in the summary report of the AFNOR Certification validation of the Enterolert E method. Two two dimensional graphs are shown in figure 8, presenting the results obtained with the Quanti Tray (the validated Quanti Tray for the Colilert 18 method in drinking waters) as the reference method. Figure 8 : Comparison of results obtained with Quanti Tray 2 and with Quanti Tray for the enumeration of Escherichia coli in tap water 25 2,5 Quanti Tray 2 (MPN/1 ml) Quanti Tray 2 (log MPN/1 ml) 2, 1,5 1,, Quanti Tray (MPN/1 ml),,,5 1, 1,5 2, 2,5 Quanti Tray (log MPN/1 ml) Statistical interpretation A Student Fisher test has been performed from the data obtained. The results are shown in the table below. t-test: Paired Two Sample for Means Parameter Quanti-Tray Quanti-Tray 2 Mean Variance Observations Pearson Correlation.892 Hypothesized Mean Difference df 18 t Stat P(T<=t) one-tail.233 t Critical one-tail P(T<=t) two-tail.466 t Critical two-tail 2.11 Institut Scientifique d'hygiène et d'analyse 22/54

23 Both one tailed and two tailed tests conclude that there is no statistically significant difference between the enumeration of Escherichia coli with Quanti Tray or with Quanti Tray 2 at α= Conclusion The assays realized showed that the enumerations with the NF Validation certified IDEXX methods can be performed either with a Quanti Tray device or with a Quanti Tray 2 device according to the expected concentration of the target analyte in the sample without introducing any bias in the measurement. Institut Scientifique d'hygiène et d'analyse 23/54

24 5. Conclusion Comparative study The linearity and relative accuracy of the Colilert 18 / Quanti Tray or Quanti Tray2 method for the enumeration of E. coli in bathing waters are satisfactory. The bias between the two methods is acceptable. The limits of detection and quantification of the method are satisfactory. Colilert 18 / Quanti Tray or Quanti Tray2 method for the enumeration of E. coli is specific and selective. Extension study showed that the enumerations with the NF Validation certified IDEXX methods can be performed either with a Quanti Tray device or with a Quanti Tray 2 device according to the expected concentration of the target analyte in the sample without introducing any bias in the measurement. Interlaboratory study The bias of the alternative method is relatively stable from the low level of contamination to the high level of contamination. For all levels of contamination, the tolerance limits are between the limits of acceptability, meaning that at least 9% of the results will be between the limits of acceptability as defined at,6 log. Massy, January 13 th, 217 François Le Nestour Head of the Unit innovation Biology Institut Scientifique d'hygiène et d'analyse 24/54

25 Appendix 1 - Bacterial stress Code Souche Origine Stress appliqué Intensité du stress Numéro Eau ESC Escherichia coli Eau de puits 4 j à 4 C + 1 min à -8 C + 5 min à 51 C 1,88 52 Plage de la Roquille ESC Escherichia coli Eau de puits 4 j à 4 C + 1 min à -8 C + 5 min à 51 C 1,88 9 La somme ESC Escherichia coli Eau de puits 4 j à 4 C + 1 min à -8 C + 5 min à 51 C 1,88 94 Troyes ESC Escherichia coli Eau de puits 4 j à 4 C + 1 min à -2 C + 5 min à 51 C 1,3 53 Plage de Carnon ESC Escherichia coli Eau de puits 4 j à 4 C + 1 min à -2 C + 5 min à 51 C 1,3 91 Saint Quentin en Yvelines ESC Escherichia coli Eau de puits 4 j à 4 C + 1 min à -2 C + 5 min à 51 C 1,3 95 Etampes ESC Escherichia coli Eau de distribution 4 j à 4 C + (5 min à -8 C + 5 min à 36 C) x2,9 54 Plage du Couchant ESC Escherichia coli Eau de distribution 4 j à 4 C + (5 min à -8 C + 5 min à 36 C) x2,9 92 Villennes sur Seine ESC Escherichia coli Eau 4 j à 4 C + (5 min à -2 C + 5 min à 36 C) x2 1,1 55 Plage du Point Zero ESC Escherichia coli Eau 4 j à 4 C + (5 min à -2 C + 5 min à 36 C) x2 1,1 93 Saint Leger en Yvelines ESC Escherichia coli Eau de fontaine 4 j à 4 C + 3 min à -8 C + 1 min à 36 C + 1 min à 51 C 1,1 47 Saint Roch ESC Escherichia coli Eau de fontaine 4 j à 4 C + 3 min à -8 C + 1 min à 36 C + 1 min à 51 C 1,1 2 Fécamp ESC Escherichia coli Eau de fontaine 4 j à 4 C + 3 min à -8 C + 1 min à 36 C + 1 min à 51 C 1,1 21 Mesnil Val plage ESC Escherichia coli Eau de fontaine 4 j à 4 C + 3 min à -8 C + 1 min à 36 C + 1 min à 51 C 1,1 22 Dieppe ESC Escherichia coli Plage Saint Maurice (Palavas les 4 j à 4 C + 3 min à -2 C + 1 min à 36 C + 1 min à 51 C 1,6 48 Eau de puits flots) ESC Escherichia coli Eau de puits 4j à 4 C + 1 min à -8 C + 6 min à 36 C 1,1 5 Plage des dunes (Carnon-Plage) ESC Escherichia coli Eau de puits 4j à 4 C + 1 min à -8 C + 6 min à 36 C 1,1 51 Plage du grand travers ESC.1.12 Escherichia coli Eau 3 min à 56 C 1,7 23 Quend ESC.1.12 Escherichia coli Eau 3 min à 56 C 1,7 24 Saint Marguerite ESC Escherichia coli Eau 1 min à -2 C + 7 min à 51 C 1,7 25 Saint Pierre en Port ESC Escherichia coli Eau 1 min à -2 C + 7 min à 51 C 1,7 26 Veulette sur mer ESC Escherichia coli Eau 1 min à -2 C + 5 min à 51 C,5 27 Charron ESC Escherichia coli Eau 1 min à -2 C + 5 min à 51 C,5 28 La Rochelle ESC Escherichia coli Effluent secondaire (3 min à -8 C + 15 min à 55 C) x2,9 1 Saint Brevin ESC Escherichia coli Effluent secondaire (3 min à -8 C + 15 min à 55 C) x2,9 2 Berck ESC Escherichia coli Eau de rivière 7 min à 51 C,6 15 Cayeux sur mer Institut Scientifique d'hygiène et d'analyse 25/54

26 Appendix 2 Relative accuracy results Institut Scientifique d'hygiène et d'analyse 26/54

27 Exactitude Relative- Eau de mer MA: IDEXX Colilert 18 MR:NF ISO (2) R1 R2 N eau Souche R1 R2 éch. NPP / 1 ml NPP / 1 ml Log 1 Log 1 log 1 NPP/1 ml (NPP/1mL) NPP/1 ml log 1 limite limite limite limite NPP (NPP/1 ml) NPP (NPP/1 ml) (NPP/1mL) inf. sup. inf. sup. 1 Saint Brevin ESC ,9 2, ,8 2, , ,326 2 Berck ESC ,5 2, ,3 2, , , Cayeux sur mer ESC , , , ,151 2 Fécamp ESC , , , , Mesnil Val plage ESC , , , , Dieppe ESC , , , ,63 23 Quend ESC , , , ,71 24 Saint Marguerite ESC , , , ,48 25 Saint Pierre en Port ESC , , , , Veulette sur mer ESC , , , , Charron ESC , , , , La Rochelle ESC , , , , Plage Saint Roch (Palavas les flots) ESC ,31 1 1, , , Plage Saint Maurice (Palavas les flots) ESC , , , ,471 5 Plage des dunes (Carnon-Plage) ESC , , , , Plage du grand travers ESC , , , , Plage de la Roquille (Agde) ESC , , , , Plage de carnon (Carnon-Plage) ESC , , , , Plage du Couchant ESC , , , , Plage du Point Zero ESC , , , ,29 Institut Scientifique d'hygiène et d'analyse 27/54

28 Exactitude relative - Escherichia coli - Eaux de mer Méthode de référence Méthode alternative Echantillon Répétition 1 Répétition 2 M SD Echantillon Répétition 1 Répétition 2 M SD Différence 1 2,24 2,326 2,265,86 1 2,373 2,538 2,455,117,19 2 2,29 2,761 2,526, ,287 2,435 2,361,15 -, ,59 4,151 4,15,65 3 4,15 4,15 4,15,,45 4 2,318 1,892 2,15,31 4 2,13 1,982 2,56,15 -,49 5 3,886 3,851 3,868,24 5 3,887 3,763 3,825,87 -,44 6 3,665 3,63 3,648,25 6 3,887 3,788 3,837,7, ,43 3,71 3,57,2 7 2,91 2,849 2,875,37 -, ,534 3,48 3,57,38 8 3,145 3,44 3,94,72 -, ,451 3,597 3,524,13 9 3,714 3,689 3,71,18, ,244 3,151 3,197,66 1 3,265 3,195 3,23,5, ,224 3,729 3,976, ,8 4,2 4,5,43, ,886 3,818 3,852, ,738 3,862 3,8,87 -, ,785 1,176 1,481, ,31 1, 1,151,213 -, ,59 3,471 3,49, ,44 3,417 3,429,16 -, ,66 4,224 4,145, ,239 4,15 4,195,63,5 16 3,956 3,956 3,956, 16 3,837 3,837 3,837, -, ,224 4,182 4,23,3 17 4,114 4,9 4,12,17 -, ,269 4,443 4,356, ,384 4,239 4,311,13 -, ,318 4,318 4,318, 19 4,191 4,384 4,287,136 -,3 2 3,921 4,29 3,975,77 2 3,964 3,812 3,888,18 -,87 q= 2 Mx= 3,478 My= 3,432 M= -,46 n= 2 MEDx= 3,75 MEDy= 3,812 MED= -,47 N=qn= 4 SDbx=,88 SDby=,858 Biais MEDwx =,66 MEDwy =,71 SDwx=,17 SDwy=,89 rob. SDwx=,97 rob. SDwy=,15 Choix de la méthode GMFR Est. y Dév. 2,155,31 R=,522 2,429 -,68 rob. R= 1,78 4,93,58 Sx=,87 1,986,7 Sy=,85 3,843 -,19 3,611,226 r=,984 2,988 -,114 b= 1,53 Res. SEM=,155 3,463 -,368 a= -,232 Res. SD=,219 3,48,221 3,136,93 3,957,93 S(b)=,44 p(t;b=1)=,233 t(b)= 1,212 3,826 -,26 S(a)=,14 p(t;a=)=,32 t(a)= 2,221 1,328 -,178 3,444 -,16 4,134,6 3,936 -,99 4,196 -,94 4,357 -,45 Répétabilité Méthode de référence Méthode alternative 4,317 -,29 r,476,248 3,955 -,67 rob. r,273,294 Institut Scientifique d'hygiène et d'analyse 28/54

29 Les points représentés correspondent aux moyennes des répétitions de chaque échantillon 5 Exactitude relative - Escherichia coli - Eaux de mer Méthode alternative log 1 (NPP/1 ml) Méthode de référence log 1 (NPP/1 ml) Institut Scientifique d'hygiène et d'analyse 29/54

30 Exactitude Relative - Eau douce MA: IDEXX Colilert 18 MR:NF ISO (2) R1 R2 N eau Souche R1 R2 éch. NPP / 1 ml NPP / 1 ml Log 1 Log 1 log 1 log 1 limite limite limite limite NPP/1 ml NPP/1 ml NPP (NPP/1 ml) NPP (NPP/1 ml) (NPP/1mL) (NPP/1mL) inf. sup. inf. sup. 57 Plage bleue Valenton NC 9,7,987 13,1 1, , , Seine Villeneuve st Georges NC 23,5 2,39 435,2 2, , ,787 6 L'Yerres NC 14,6 2,2 93,4 1, , ,14 61 Lac d'aydat NC 2419,6 3, ,6 3, , , Annet sur marne NC 32,7 1,515 43,9 1, , ,41 64 Noisiel NC 14,6 2,2 95,9 1, , , Seine Les Mureaux NC 16,9 1,228 16,1 1, , , Etampes NC 2,31 7,5, , , Orge St Chéron NC 344,8 2, ,5 2, , , Rivière la Rémarde NC 1299,7 3, ,6 3, , , Allier NC 42,8 1,631 36,4 1, , , Longarisse NC: Dilué 1119,9 3,49 547,5 2, , , La Somme NC: Dilué 41,6 2,613 29,9 2, , , Aix les bains NC 155,3 2, ,5 2, , , Meyrieu les étangs NC 98,7 1, ,6 2, , ,328 9 La somme ESC ,4 2, , , , Saint Quentin en Yvelines ESC ,4 2, ,7 2, , , Villennes sur Seine ESC ,7 2, ,9 2, , , Saint Leger en Yvelines ESC ,7 2,56 122,3 2, , , Troyes ESC , ,7 2, , ,81 95 Etampes ESC ,1 3, ,3 3, , , La Sioul NC 43,5 1, , , ,653 Institut Scientifique d'hygiène et d'analyse 3/54

31 Exactitude relative - Escherichia coli - Eaux douces Méthode de référence Méthode alternative Echantillon Répétition 1 Répétition 2 M SD Echantillon Répétition 1 Répétition 2 M SD Différence 1 1,477 1,477 1,477, 1,987 1,117 1,52,92 -, ,4 2,787 2,593, ,39 2,639 2,474,233 -,12 3 1,785 2,14 1,945, ,2 1,97 1,995,35,5 4 3,441 3,393 3,417,34 4 3,384 3,384 3,384, -,33 5 1,477 2,41 1,759, ,515 1,642 1,579,9 -, ,332 2,158 2,245, ,2 1,982 2,1,27 -, ,663 1,176 1,419, ,228 1,27 1,217,15 -,22 8 1,176 1,477 1,327,213 8,31,875,588,46 -, ,747 2,759 2,753,8 9 2,538 2,513 2,525,18 -, ,234 3,187 3,21,33 1 3,114 3,384 3,249,191, ,653 1,663 1,658,7 11 1,631 1,561 1,596,5 -, ,3 2,923 2,963, ,49 2,738 2,894,22 -, ,689 2,645 2,667, ,613 2,464 2,539,16 -, ,27 2,294 2,251, ,191 2,195 2,193,2 -, ,41 2,328 2,185, ,994 2,96 2,45,72 -, ,923 2,787 2,855, ,912 2,787 2,85,88 -,5 17 2,772 2,869 2,82, ,689 2,837 2,763,15 -, ,955 2,986 2,97, ,919 2,961 2,94,3 -,3 19 2,149 2,199 2,174, ,56 2,87 2,72,22 -,12 2 2,865 2,81 2,833,45 2 2,862 2,837 2,849,18, ,191 3,345 3,268, ,191 3,298 3,245,76 -, ,477 1,653 1,565, ,638 1,681 1,66,3,95 q= 22 Mx= 2,38 My= 2,259 M= -,12 n= 2 MEDx= 2,422 MEDy= 2,333 MED= -,66 N=qn= 44 SDbx=,649 SDby=,757 Biais MEDwx =,66 MEDwy =,61 SDwx=,16 SDwy=,129 rob. SDwx=,97 rob. SDwy=,9 Choix de la méthode GMFR Est. y Dév. 1,215 -,163 R=,811 2,56 -,33 rob. R=,926 1,756,239 Sx=,652 3,459 -,75 Sy=,754 1,541,37 2,14 -,13 r=,979 1,148,69 b= 1,157 Res. SEM=,16 1,41 -,453 a= -,494 Res. SD=,226 2,691 -,166 3,22,29 1,424,172 S(b)=,53 p(t;b=1)=,5 t(b)= 2,938 2,934 -,41 S(a)=,14 p(t;a=)=,1 t(a)= 3,526 2,592 -,54 2,11,83 2,34,11 2,89,4 2,769 -,6 2,943 -,3 Répétabilité Méthode de référence Méthode alternative 2,21,5 r,447,363 2,784,65 rob. r,272,252 3,287 -,42 1,317,343 Institut Scientifique d'hygiène et d'analyse 31/54

32 Les points représentés correspondent aux moyennes des répétitions de chaque échantillon 5 Exactitude relative - Escherichia coli - Eaux douces 4,5 Méthode alternative log 1 (NPP/1 ml) 4 3,5 3 2,5 2 1,5 1,5,5 1 1,5 2 2,5 3 3,5 4 4,5 5 Méthode de référence log 1 (NPP/1 ml) Institut Scientifique d'hygiène et d'analyse 32/54

33 Exactitude relative - Escherichia coli - Eaux de baignade (eaux douces + eaux de mer) Méthode de référence Méthode alternative Echantillon Répétition 1 Répétition 2 M SD Echantillon Répétition 1 Répétition 2 M SD Différence 1 2,24 2,326 2,265,86 1 2,373 2,538 2,455,117,19 2 2,29 2,761 2,526, ,287 2,435 2,361,15 -, ,59 4,151 4,15,65 3 4,15 4,15 4,15,,45 4 2,318 1,892 2,15,31 4 2,13 1,982 2,56,15 -,49 5 3,886 3,851 3,868,24 5 3,887 3,763 3,825,87 -,44 6 3,665 3,63 3,648,25 6 3,887 3,788 3,837,7, ,43 3,71 3,57,2 7 2,91 2,849 2,875,37 -, ,534 3,48 3,57,38 8 3,145 3,44 3,94,72 -, ,451 3,597 3,524,13 9 3,714 3,689 3,71,18, ,244 3,151 3,197,66 1 3,265 3,195 3,23,5, ,224 3,729 3,976, ,8 4,2 4,5,43, ,886 3,818 3,852, ,738 3,862 3,8,87 -, ,785 1,176 1,481, ,31 1, 1,151,213 -, ,59 3,471 3,49, ,44 3,417 3,429,16 -, ,66 4,224 4,145, ,239 4,15 4,195,63,5 16 3,956 3,956 3,956, 16 3,837 3,837 3,837, -, ,224 4,182 4,23,3 17 4,114 4,9 4,12,17 -, ,269 4,443 4,356, ,384 4,239 4,311,13 -, ,318 4,318 4,318, 19 4,191 4,384 4,287,136 -,3 2 3,921 4,29 3,975,77 2 3,964 3,812 3,888,18 -, ,477 1,477 1,477, 21,987 1,117 1,52,92 -, ,4 2,787 2,593, ,39 2,639 2,474,233 -, ,785 2,14 1,945, ,2 1,97 1,995,35,5 24 3,441 3,393 3,417, ,384 3,384 3,384, -, ,477 2,41 1,759, ,515 1,642 1,579,9 -, ,332 2,158 2,245, ,2 1,982 2,1,27 -, ,663 1,176 1,419, ,228 1,27 1,217,15 -, ,176 1,477 1,327,213 28,31,875,588,46 -, ,747 2,759 2,753,8 29 2,538 2,513 2,525,18 -, ,234 3,187 3,21,33 3 3,114 3,384 3,249,191, ,653 1,663 1,658,7 31 1,631 1,561 1,596,5 -, ,3 2,923 2,963, ,49 2,738 2,894,22 -, ,689 2,645 2,667, ,613 2,464 2,539,16 -, ,27 2,294 2,251, ,191 2,195 2,193,2 -, ,41 2,328 2,185, ,994 2,96 2,45,72 -, ,923 2,787 2,855, ,912 2,787 2,85,88 -,5 37 2,772 2,869 2,82, ,689 2,837 2,763,15 -, ,955 2,986 2,97, ,919 2,961 2,94,3 -,3 39 2,149 2,199 2,174, ,56 2,87 2,72,22 -,12 4 2,865 2,81 2,833,45 4 2,862 2,837 2,849,18, ,191 3,345 3,268, ,191 3,298 3,245,76 -, ,477 1,653 1,565, ,638 1,681 1,66,3,95 q= 42 Mx= 2,93 My= 2,818 M= -,85 n= 2 MEDx= 2,99 MEDy= 2,862 MED= -,58 N=qn= 84 SDbx=,99 SDby=,993 Biais MEDwx =,66 MEDwy =,71 SDwx=,165 SDwy=,112 rob. SDwx=,97 rob. SDwy=,15 Institut Scientifique d'hygiène et d'analyse 33/54

34 Choix de la méthode GMFR Est. y Dév. 2,125,33 R=,68 2,48 -,47 rob. R= 1,78 4,125,25 Sx=,911 1,951,16 Sy=,99 3,868 -,43 3,628,29 r=,988 2,985 -,11 b= 1,87 Res. SEM=,157 3,475 -,38 a= -,337 Res. SD=,223 3,493,28 3,138,92 3,985,65 S(b)=,27 p(t;b=1)=,2 t(b)= 3,226 3,85 -,5 S(a)=,117 p(t;a=)=,5 t(a)= 2,894 1,272 -,122 3,456 -,27 4,168,27 3,963 -,126 4,231 -,129 4,397 -,86 Répétabilité Méthode de référence Méthode alternative 4,356 -,69 r,461,313 3,983 -,95 rob. r,273,294 1,268 -,216 2,481 -,8 1,776,219 3,376,7 1,575,4 2,13 -,13 1,25,12 1,15 -,516 2,655 -,13 3,152,97 1,465,132 2,883,1 2,562 -,23 2,19,84 2,37,7 2,766,84 2,728,34 2,891,49 2,26,46 2,742,17 3,215,3 1,364,296 Institut Scientifique d'hygiène et d'analyse 34/54

35 Les points représentés correspondent aux moyennes des répétitions de chaque échantillon 5 Exactitude relative - Escherichia coli - Eaux de baignade Méthode alternative log 1 (NPP/1 ml) Méthode de référence log 1 (NPP/1 ml) Institut Scientifique d'hygiène et d'analyse 35/54