NF VALIDATION Validation of alternative analytical methods Application in food microbiology. Summary Report. Initial validation study

Transcription

1 ACCREDITATION N PORTEE DISPONIBLE SUR NF VALIDATION Validation of alternative analytical methods Application in food microbiology Summary Report Initial validation study Validation study according to EN ISO :2016 kit for detection of in raw pork and poultry meat, ready to-eat and ready to-reheat pork and poultry meat, and production environmental samples Qualitative method Expert Laboratory: For: ADRIA Développement ZA Creac h Gwen Quimper Cedex, France THERMO FISHER SCIENTIFIC Oxoid Limited Wade Road, Basingstoke Hampshire RG24 8PW, England, UK This report consists of 108 pages, including 7 appendices. Only copies including the totality of this report are authorised. Competencies of the laboratory are certified by COFRAC accreditation for the analyses marked with the symbol. Version 0 22 February 2018 ADRIA DEVELOPPEMENT Creac h Gwen - F QUIMPER Cedex Tél. (33) Fax (33) adria.developpement@adria.tm.fr ASSOCIATION LOI DE N SIRET N EXISTENCE N TVA FR

2 1 INTRODUCTION 5 2 METHOD PROTOCOLS Alternative method Principle Protocols Restriction Reference methods Study design 7 3 METHOD COMPARISON STUDY Sensitivity study Number and nature of samples Artificial contamination of samples Protocols applied during the validation study Test results Calculation of relative trueness (RT), sensitivity (SE) and false positive ratio (FPR) Analysis of discordants Enrichment broth storage at 5 ± 3 C for 72 h Confirmation Inhibitions Relative level of detection Experimental design Calculation and interpretation of the RLOD Inclusivity / exclusivity Test protocols Results Practicability 32 ADRIA Développement 2/ February 2018

3 4 INTER-LABORATORY STUDY Study organisation Experimental parameters controls Strain stability and background microflora stability Contamination levels Logistic conditions Results analysis Expert laboratory results Results observed by the collaborative laboratories Results of the collaborators retained for interpretation Calculation and interpretation Calculation of the specificity percentage (SP) Calculation of the sensitivity (SE alt ), the sensitivity for the reference method (SE ref ), the relative trueness (RT) and the false positive ratio for the alternative method (FPR) Interpretation of data Evaluation of the RLOD between laboratories 44 5 CONCLUSION 44 Appendix 1 Flow diagram of the alternative method 46 Appendix 2 Flow diagram of the reference method: ISO (April 2017) - Microbiology of food and animal feeding stuffs - Horizontal method for the detection, enumeration and serotyping of spp. - Part 1: detection of spp. 48 Appendix 3 Artificial contamination of samples 49 Appendix 4 Sensitivity study: raw data 53 Appendix 5 Relative level of detection study: raw data 83 Appendix 6 Inclusivity and exclusivity study: raw data 86 Appendix 7 - Results obtained by the collaborative laboratories and the expert laboratory 94 ADRIA Développement 3/ February 2018

4 Quality Assurance documents related to this study can be consulted upon request from Thermo Fisher Scientific. The technical protocol and the result interpretation were carried out according to the EN ISO :2016 and the AFNOR technical rules (Draft Revision 6). Validation protocols EN ISO (June 2016) : Microbiology of the food chain - Method validation Part 1: Vocabulary Part 2: Protocol for the validation of alternative (proprietary) methods against a reference method AFNOR Technical Rules (Draft Revision n 6) Reference methods - ISO (April 2017) - Microbiology of food and animal feeding stuffs - Horizontal method for the detection, enumeration and serotyping of spp. - Part 1: detection of spp. - ISO/TR (October 2014) - Microbiology of the food chain - Horizontal method for the detection, enumeration and serotyping of - Part 3 : guidelines for serotyping of spp. Alternative method Scope Certification organism Typhimurium and Enteritidis Multiplex PCR kit Raw pork and poultry meat Ready to-eat and ready to-reheat pork and poultry Production environmental samples (excluding primary production samples) AFNOR Certification ( Analyses performed according to the COFRAC accreditation ADRIA Développement 4/ February 2018

5 1 INTRODUCTION The Typhimurium and Enteritidis Multiplex PCR kit using the Applied Biosystems TM 7500 Fast Real-Time PCR Instrument and RapidFinder TM Express Software V2.0 for result analysis and interpretation was validated on 25 th January 2018 according to the ISO : METHOD PROTOCOLS 2.1 Alternative method The flow diagram of the alternative method is provided in Appendix Principle The assay is based on TaqMan real-time PCR technology. Dye-labelled probes target unique DNA sequences specific to ser. Typhimurium, ser. Enteritidis and all species, and an internal positive control (IPC). Target DNA, if present, is amplified by PCR and detected in real-time using fluorescent hydrolysis probe chemistry. The fluorescent signal that is generated is detected by the 7500 Fast Real-Time PCR Instrument and interpreted by RapidFinder TM Express Software v2.0 or higher Protocols Two protocols are available depending on the tested categories; they are described in Appendix 1 and summarised in Table 1. ADRIA Développement 5/ February 2018

6 Table 1 - Protocols Category Enrichment broth Incubation Raw pork and poultry meat 25 g ml BPW Ready-to-eat and ready to + 12 mg/l Novobiocin 14 h - 18 h at 41.5 C ± 1 C reheat pork and poultry Production environmental samples 25 g ml BPW Swab + 10 ml 1 BPW Sponge ml 1 BPW Wipe ml 1 BPW 16 h - 20 h at 37 C ± 1 C The different steps are the following: - Enrichment - Lysis of 10 µl enriched sample - PCR on 20 µl DNA extract - Confirmation by one of the following tests: By direct streaking of enriched sample (10 µl) onto Brilliance TM Agar (24 h ± 2 h at 37 C ± 1 C). If there is high background on the plate, subculture 0.1ml of the enrichment broth into RVS broth (0.1 ml + 10 ml), incubated for 24 h ± 3 h at 41.5 C and streaking (10 µl) onto Brilliance Agar (24 h ± 2 h at 37 C ± 1 C) Confirm the typical colonies using the OXOID TM latex test Kit. Serological confirmation It is possible to store the enrichment broth for 72 h at 5 C ± 3 C before running the PCR and, if necessary, the confirmatory tests. For the Inter-laboratory study, the following confirmation tests were tested: - Reference method: Latex test; - Alternative method: latex test and serological confirmation. 1 Pre-moisten the swab with 1 ml diluent and add 9 ml diluent for analysis Pre-moisten the sponge with 10 ml diluent and add 90 ml diluent for analysis For sampling after cleaning process, use neutralizing agent for pre-moisten instead of diluent. ADRIA Développement 6/ February 2018

7 2.1.3 Restriction There is no restriction for use. 2.2 Reference methods The reference method corresponds to: - The ISO (February 2017) - Microbiology of food and animal feeding stuffs - Horizontal method for the detection, enumeration and serotyping of spp. - Part 1: detection of spp. (See Appendix 2); - The ISO/TR (October 2014) - Microbiology of the food chain -- Horizontal method for the detection, enumeration and serotyping of -- Part 3: Guidelines for serotyping of spp. 2.3 Study design The enrichment step for the reference method and the alternative method is different for raw pork and poultry meat and ready-to-eat pork and poultry categories. This is an unpaired study design. For protocol Environmental samples category, it is a paired study design as the reference method and the alternative method have the same enrichment broth. Analysis performed according to the COFRAC accreditation ADRIA Développement 7/ February 2018

8 3 METHOD COMPARISON STUDY The method comparison study is a study performed by the expert laboratory to compare the alternative method with the reference method. 3.1 Sensitivity study The sensitivity (SE) is the ability of the method to detect the analyte by either the reference or alternative method Number and nature of samples 203 samples were analysed. The distribution per tested category and type is given in: - Table 2 for spp. target, - Table 3 for Enteritidis, - Table 4 for Typhimurium target. ADRIA Développement 8/ February 2018

9 Table 2 Distribution per tested category and type - spp. 1 Category Raw pork and poultry meat 2 RTE pork and poultry 3 Environmental samples Positive Negative Type Total samples samples a Fresh b Frozen c Seasoned Total Raw delicatessen and fermented a meat b Cooked delicatessen c RTE or RTRH meat Total a Surfaces b Water c Dusts and residues Total TOTAL Table 3 Distribution per tested category and type - Enteritidis 1 2 Category Raw pork and poultry meat RTE pork and poultry 3 Environmental samples Positive Negative Type Total samples samples a Fresh b Frozen c Seasoned Total Raw delicatessen and fermented a meat b Cooked delicatessen c RTE or RTRH meat Total a Surfaces b Water c Dusts and residues Total TOTAL ADRIA Développement 9/ February 2018

10 Table 4 Distribution per tested category and type - Typhimurium 1 2 Category Type Positive Negative Total Raw pork and poultry meat RTE pork and poultry 3 Environmental samples a Fresh b Frozen c Seasoned Total Raw delicatessen and fermented a meat b Cooked delicatessen c RTE or RTRH meat Total a Surfaces b Water c Dusts and residues Total TOTAL Artificial contamination of samples Artificial contaminations were done by seeding. The artificial contaminations are presented in Appendix samples were artificially contaminated, using seeding and spiking protocols. Co-inoculations were applied (S. Typhimurium or S. Enteritidis with spp. from different serovars). 10 naturally contaminated samples were also obtained. A summary of the number of samples inoculated and the number of positive results obtained is described in Table 5. ADRIA Développement 10/ February 2018

11 Table 5 - Summary of the number of samples inoculated and the number of positive results obtained Number of Number of positive results inoculated samples spp. Enteritidis Typhimurium S. Typhimurium S. Enteritidis S. Typhimurium + spp S. Enteritidis + spp spp Total artificially contaminated Naturally contaminated / Total positive / The repartition of the positive samples per inoculation protocol and inoculation level is given in Table 6. Table 6 - Repartition of the positive samples per inoculation protocol and inoculation level Naturally Artificially contaminated samples contaminated Spiking Seeding Total samples x 5 5 < x 10 x 3 3 < x 10 Number of samples (3 (2) ) (18 (2) ) % 10 % 5 % 0 % 44 % 41 % 100 % For the seeding protocol, 41 % of the samples were inoculated between 5 and 10 CFU however note that 18 % of them concern co-inoculation of 2 strains. 10 % of the samples were naturally contaminated. 2 Co-inoculation ADRIA Développement 11/ February 2018

12 3.1.3 Protocols applied during the validation study Incubation times The following incubation times were applied for each step as described in Table 7. Table 7 - Incubation times Step Category Incubation time Raw pork and poultry meat Enrichment Ready-to-eat pork and poultry 14 h Environmental samples 16h Subculture in RVS 21h All categories Plates 22h Confirmations - By direct streaking of enriched sample (10 µl) onto Brilliance agar (24 h ± 2 h at 37 C ± 1 C). If there is high background on the plate, subculture 0,1ml of the enrichment broth into RVS broth (0.1 ml + 10 ml), incubate 24 h ± 3 h at 41.5 C and streaking (10 µl) onto Brilliance agar (24 h ± 2 h at 37 C ± 1 C) Confirm the typical colonies using the OXOID TM test Kit. - By the tests described in the reference method - By biochemical galleries on isolated colonies without purification step - By serological tests (run by the expert lab); - By serotyping according to the ISO/TR (subcontracted to LABOCEA). Enrichment storage The positive samples were tested a second time after storage of the enrichment broths for 72 h at 5 C ± 3 C (PCR and confirmatory tests). ADRIA Développement 12/ February 2018

13 3.1.4 Test results Raw data per category are given in Appendix 4. The results are given in: - Table 8 for spp., - Table 9 for Enteritidis, - Table 10 for Typhimurium. Table 8 Interpretation of sample results between the reference and alternative method (based on the confirmed alternative) - spp. Category PA NA PD ND PPND PPNA 1 Raw pork and poultry meat RTE pork and poultry Environmental samples TOTAL Table 9 Interpretation of sample results between the reference and alternative method (based on the confirmed alternative) - Enteritidis Category PA NA PD ND PPND PPNA 1 Raw pork and poultry meat RTE pork and poultry Environmental samples TOTAL Table 10 Interpretation of sample results between the reference and alternative method (based on the confirmed alternative) - Typhimurium Category PA NA PD ND PPND PPNA 1 Raw pork and poultry meat RTE pork and poultry Environmental samples TOTAL ADRIA Développement 13/ February 2018

14 3.1.5 Calculation of relative trueness (RT), sensitivity (SE) and false positive ratio (FPR) The calculations are presented in: - Table 11 for spp., - Table 12 for Enteritidis, - Table 13 for Typhimurium. ADRIA Développement 14/ February 2018

15 Table 11 Calculation of the relative trueness (RT), the sensitivity (SE) and the false positive ratio (FPR) - spp Category Type PA NA PD ND PPND PPNA SE alt % SE ref % RT % FPR % a Fresh ,3 100,0 95,7 0 Raw pork and b Frozen ,0 80,0 91,3 0,0 poultry meat c Seasoned ,3 92,3 91,7 0,0 Total ,4 91,7 92,9 0,0 RTE pork and poultry Environmental samples a Raw delicatessen and fermented meat ,9 84,6 78,3 0,0 b Cooked delicatessen ,9 90,9 90,5 0,0 c RTE or RTRH meat ,0 100,0 100,0 0,0 Total ,1 90,3 89,1 0,0 a Surfaces ,0 100,0 100,0 0,0 b Water ,3 100,0 91,7 0,0 c Dusts and residues ,9 100,0 95,0 0,0 Total ,9 100,0 95,7 0,0 TOTAL ,0 94,0 92,6 0,0 Table 12 Calculation of the relative trueness (RT), the sensitivity (SE) and the false positive ratio (FPR) - Enteritidis Category Type PA NA PD ND PPND PPNA SE alt % SE ref % RT % FPR % a Fresh ,3 83,3 91,3 0 Raw pork and poultry b Frozen ,0 75,0 95,7 0,0 1 meat c Seasoned ,3 83,3 91,7 0,0 Total ,5 81,3 92,9 0,0 a Raw delicatessen and fermented meat ,0 50,0 91,3 0,0 2 RTE pork and poultry b Cooked delicatessen ,4 71,4 81,0 7,1 c RTE or RTRH meat ,0 100,0 100,0 0,0 3 Environmental samples Total ,7 73,3 90,6 2,0 a Surfaces ,0 83,3 96,0 0,0 b Water ,0 100,0 95,8 0,0 c Dusts and residues ,0 100,0 100,0 0,0 Total ,9 90,9 97,1 0,0 TOTAL ,1 81,0 93,6 0,6 ADRIA Développement 15/ February 2018

16 Table 13 Calculation of the relative trueness (RT), the sensitivity (SE) and the false positive ratio (FPR) - Typhimurium Category Type PA NA PD ND PPND PPNA SE alt % SE ref % RT % FPR % Raw pork and poultry meat RTE pork and poultry Environmental samples a Fresh ,0 80,0 95,7 0 b Frozen ,0 40,0 87,0 5,9 c Seasoned ,0 100,0 100,0 0,0 Total ,0 71,4 94,3 1,8 a Raw delicatessen and fermented meat ,3 83,3 91,3 0,0 b Cooked delicatessen ,0 100,0 100,0 0,0 c RTE or RTRH meat ,0 66,7 95,0 0,0 Total ,7 83,3 95,3 0,0 a Surfaces ,0 100,0 100,0 0,0 b Water ,7 100,0 95,8 0,0 c Dusts and residues ,5 100,0 95,0 0,0 Total ,9 100,0 97,1 0,0 TOTAL ,2 86,4 95,6 0,6 ADRIA Développement 16/ February 2018

17 A summary of the results is given in Table 14. Table 14 - Summary of results spp. Enteritidis Typhimurium Sensitivity for the PA PD SE 100% alternative method alt ( PA ND PD) 91.0% 88.1% 93.2% Sensitivity for the PA ND SE 100% reference method ref ( PA ND PD) 94.0% 81.0% 86.4% Relative trueness ( PA NA) RT 100% N 92.6% 93.6% 95.6% False positive ratio for ( FP) the alternative FPR 100% NA method* 0.0% 0.6% 0.6% FP = PPNA + PPND With ND = ND + PPND NA = NA + PPNA Analysis of discordants The negative deviations are given in Table 15 and the positive deviations in Table 16. Nine negative deviations were obtained: 4 concern S. Enteritidis detection, 3 S. Typhimurium detection and 2 spp. detection from other serotypes. For 4 samples (3749, 4477, 4968 and 4971), S. Enteritidis was isolated twice from the enrichment broth as it was the case for S. Typhimurium. Six positive deviations concerned spp. detection (different from S. Enteritidis and S. Typhimurium). For 9 samples, the reference method gave a positive result for spp. detection (different from S. Enteritidis or S. Typhimurium) while the alternative method gave positive results for one of this serotype. ADRIA Développement 17/ February 2018 Summary Report Version 0)

18 Table 15 - Negative deviations Alternative method Sample Inoculation Reference PCR Agreement Product Inoculated strain Confirmatory No level method test spp. Enteritidis Typhimurium spp. Enteritidis Typhimurium 3749 Raw chicken S. Enteritidis meat Ad S. Enteritidis -/-/- -/-/- -/-/ S. Enteritidis ND ND NA 4297 Low moisture S. Typhimurium sausage Ad S. Typhimurium ND NA ND 4307 Pork rillettes S. Enteritidis Ad S. Enteritidis ND ND NA 4373 Sausage with / herbs / S. Braendenburg ND NA NA 4477 S. Typhimurium Ad2508 Process water + (pork industry) S. Bovismorbificans S. Typhimurium -/-/- -/-/- -/-/- S. Typhimurium ND NA ND 4968 Rinsing water S. Enteritidis Ad S. Enteritidis -/-/- -/-/- -/-/- S. Enteritidis ND ND NA 4971 Waste (poultry S. Typhimurium slaughter) A00V S. Typhimurium -/-/- -/-/- -/-/- S. Typhimurium NA NA ND 5907 Smoked / bacon / S. Agona ND NA NA 6360 Seasoned chicken meat S. Enteritidis Ad S. Enteritidis ND ND NA ADRIA Développement 18/ February 2018

19 Table 16 - Positive deviations Alternative method Sample Inoculation Reference PCR Agreement Product Inoculated strain Confirmatory No level method test spp. Enteritidis Typhimurium spp. Enteritidis Typhimurium 4377 Sausage / / S. Infantis PD NA NA S. Enteritidis ATCC 4665 Wipe after 1045 cleaning S. Caracas S. Enteritidis PA PD NA S. Caracas Ad Raw moisture S. Enteritidis Ad926 ham S. Enteritidis PD PD NA 4308 Duck rillettes S. Enteritidis Ad S. Enteritidis PD PD NA Frozen seasoned chicken meat Frozen port meat Seasoned chicken wings Raw turkey meat S. Typhimurium A00C S. Typhimurium PD NA PD S. Enteritidis 2532 S. Enteritidis Ad S. Mbandaka Ad1721 S. Enteritidis Ad S. Braenderup Ad S. Enteritidis PD PD NA S. Enteritidis + S. Mbandaka PD PD NA S. Braenderup S. Enteritidis PA PD NA ADRIA Développement 19/ February 2018

20 Sample No Product 4309 Raw ham 6366 Cooked turkey 4311 Sausage Raw duck meat RTE (smoked pork meat) Frozen pork meat Frozen port meat Inoculated strain S. Enteritidis Ad926 + S. Infantis 2556 S. Enteritidis Ad S. Mbandaka Ad1721 S. Typhimurium Ad S. Infantis 2556 S. Typhimurium Ad913 + S. Braenderup Ad915 S. Typhimurium S. Infantis 288 S. Typhimurium S. Infantis 288 S. Typhimurium S. Infantis 288 Inoculation level Reference method spp. PCR Enteritidis Typhimurium Alternative method Confirmatory test spp. Agreement Enteritidis Typhimurium S. Infantis S. Enteritidis PA PD NA S. Mbandaka S. Enteritidis PA PD NA S. Infantis S. Typhimurium PA NA PD S. Braenderup S. Typhimurium PA NA PD S. Infantis S. Infantis S. Infantis S. Infantis + S. Typhimurium S. Infantis + S. Typhimurium S. Infantis + S. Typhimurium PA NA PD PA NA PD PA NA PD ADRIA Développement 20/ February 2018

21 The analyses of discordant results according to the EN ISO :2016 is given in: - Table 17 for spp., - Table 18 for Enteritidis, - Table 19 for Typhimurium. Table 17 - Analyses of discordant results - spp. 1 2 Category Raw pork and poultry meat RTE pork and poultry 3 Environmental samples Unpaired Paired Type ND+ PD (ND+PPND)-PD AL (ND+PPND)-PD AL (ND+PPND)+PD AL PPND Raw meat products a (frozen or fresh) Raw poultry b (fresh or frozen) c Raw delicatessen Total a Pasteurized products b Raw products Ingredients and low c moisture products Total a Dusts and Residues Cleaning and b Process Waters c Surface samples Total TOTAL ADRIA Développement 21/ February 2018

22 Table 18 - Analyses of discordant results - Enteritidis 1 2 Category Raw pork and poultry meat RTE pork and poultry 3 Environmental samples Unpaired Paired Type ND+ PD (ND+PPND)-PD AL (ND+PPND)-PD AL (ND+PPND)+PD AL PPND Raw meat products a (frozen or fresh) Raw poultry b (fresh or frozen) c Raw delicatessen Total a Pasteurized products b Raw products Ingredients and c low moisture products Total a Dusts and Residues Cleaning and b Process Waters c Surface samples Total TOTAL Table 19 - Analyses of discordant results - Typhimurium 1 2 Category Raw pork and poultry meat RTE pork and poultry 3 Environmental samples Unpaired Paired Type ND+ PD (ND+PPND)-PD AL (ND+PPND)-PD AL (ND+PPND)+PD AL PPND Raw meat products a (frozen or fresh) Raw poultry b (fresh or frozen) c Raw delicatessen Total a Pasteurized products b Raw products Ingredients and c low moisture products Total a Dusts and Residues Cleaning and b Process Waters c Surface samples Total TOTAL ADRIA Développement 22/ February 2018

23 The observed values for ((ND + PPND) - PD) are lower than the acceptability limit for the food category (unpaired study design) for the three targets ( spp., Enteritidis and Typhimurium). The observed values for ((ND + PPND) - PD) and (ND + PPND + PD) are lower than the acceptability limits for the environmental samples (paired study design) for the three targets ( spp., Enteritidis and Typhimurium) Enrichment broth storage at 5 ± 3 C for 72 h The enrichment broths of positive samples were tested again after 72 h storage at 5 C ± 3 C; only one change was observed (See Table 20). Table 20 - Enrichment broth storage Sample no Product spp. Result before storage Enteritidis Typhimurium spp. Result after storage Enteritidis Typhimurium 4477 Process water ND NA ND PA NA PA The analyses of discordant become (See Tables 21 to 23). ADRIA Développement 23/ February 2018

24 Table 21 - Analysis of discordant after storage 72 h at 5 ± 3 C - spp. 1 2 Unpaired Paired Category Type ND PPND PD (ND+PPND)-PD AL (ND+PPND)-PD AL (ND+PPND)+PD AL Raw meat products a (frozen or fresh) Raw pork and poultry meat Raw poultry b (fresh or frozen) c Raw delicatessen RTE pork and poultry 3 Environmental samples Total a Pasteurized products b Raw products Ingredients and c low moisture products Total a Dusts and Residues Cleaning and b Process Waters c Surface samples Total TOTAL Table 22 - Analysis of discordant after storage 72 h at 5 ± 3 C - Enteritidis 1 2 Unpaired Paired Category Type ND PPND PD (ND+PPND)-PD AL (ND+PPND)-PD AL (ND+PPND)+PD AL Raw meat products a (frozen or fresh) Raw pork and poultry meat Raw poultry b (fresh or frozen) c Raw delicatessen RTE pork and poultry 3 Environmental samples Total a Pasteurized products b Raw products Ingredients and c low moisture products Total a Dusts and Residues Cleaning and b Process Waters c Surface samples Total TOTAL ADRIA Développement 24/ February 2018

25 Table 23 - Analysis of discordant after storage 72 h at 5 ± 3 C - Thyphimurium 1 2 Unpaired Paired Category Type ND PPND PD (ND+PPND)-PD AL (ND+PPND)-PD AL (ND+PPND)+PD AL Raw meat products a (frozen or fresh) Raw pork and poultry meat Raw poultry b (fresh or frozen) c Raw delicatessen RTE pork and poultry 3 Environmental samples Total a Pasteurized products b Raw products Ingredients and c low moisture products Total a Dusts and Residues Cleaning and b Process Waters c Surface samples Total TOTAL The observed values for ((ND + PPND) - PD) are lower than the acceptability limit for the food category (unpaired study design) for the three targets ( spp., Enteritidis and Typhimurium). The observed values for ((ND + PPND) - PD) and (ND + PPND + PD) are lower than the acceptability limits for the environmental samples category (paired study design) for the three targets ( spp., Enteritidis and Typhimurium) Confirmation Streaking Two protocols were tested during the validation study: - Direct streaking onto Brilliance TM Agar; - Subculture in RVS broth for 24 h ± 3 h at 41.5 C prior to streaking onto Brilliance TM Agar. ADRIA Développement 25/ February 2018

26 All the positive PCR results were confirmed by both confirmatory methods. For two samples in negative deviation, the presence of S. Enteritidis (4968) and S. Typhimurium (4971) was detected after a subculture in RVS broth. Serological confirmation The serological test applied on the first colony isolated on Brilliance TM Agar allowed to confirm the PCR test result, except for 4 samples (4312, 3746, 6270 and 6271). In these cases, the protocol recommended in the kit insert was applied (See page 20 in the kit insert). - For 2 samples (3746 and 6271), S. Typhimurium was confirmed using this protocol. - For sample (6270), it was not possible to recover the S. Typhimurium. For sample 4312, it was not possible to recover the S. Enteritidis. Both samples were co-infected samples (with a spp. and S. Typhimurium or S. Enteritidis). Culture confirmation for these samples was challenging due to the low level of S. Typhimurium or S. Enteritidis in the sample Inhibitions One sample 4967 gave a PCR result No IPC amplification, inconclusive for all targets another sample h gave Inconclusive for all targets, amplification detected for Enteritidis without detection of species and h Positive for species Inconclusive for Enteritidis and Typhimurium (See Table 24). Table 24 - PCR results No Sample PCR result 4967 Rinsing water (poultry slaughter) No IPC amplification, inconclusive for all targets /-/ Rinsing water (poultry slaughter) (72 h) Inconclusive for all targets, amplification detected for Enteritidis without detection of species / Smoked bacon (72 h) Positive for species, inconclusive for Enteritidis and Typhimurium / - The enrichment broth was diluted (1/5), tested again and allowed to lift the inhibition. This protocol is described on p.19 of the kit insert. ADRIA Développement 26/ February 2018

27 3.2 Relative level of detection The relative level of detection is the level of detection at P = 0.50 (LOD 50) of the alternative (proprietary) method divided by the level of detection at P = 0.50 (LOD 50) of the reference method. The RLOD is defined as the ratio of the alternative and reference methods: Experimental design One matrix should be tested as a minimum per category. At least, three inoculation levels were used: - A blank level, (no contamination), with 5 replicates; - A low contamination level providing fractional recovery data, with 20 replicates; - A higher contamination level, with 5 replicates. Analyses for fractional positive result recovery were done: - A total plate count determination on each matrix was performed to estimate the total microbial load on the day of analysis. - Each matrix was first screened for the presence of using the tested reference method(s) on 5 replicates. 3 (matrix/strain) pairs were analysed by the reference method and by the alternative method. Table 25 outlines the matrices, the inoculated strains and storage conditions after inoculation. ADRIA Développement 27/ February 2018

28 Table 25 - Defined (matrix/strain) pairs for the RLOD determination Category Raw meat pork and poultry Ready-to-eat pork and poultry Environmental samples Raw pork Matrix Turkey ham Process water (pork/beef slaughter) Strain inoculated S. Typhimurium Ad1872 monophasic S. Enteritidis Ad2524 S. Typhimurium Ad S. Derby A00E084 Origin Storage conditions after inoculation / 48 h at 5 C± 3 C Poultry meat 48 h at 5 C± 3 C Pork industry environment Dairy industry 48 h at 5 C± 3 C Calculation and interpretation of the RLOD The raw data are given in Appendix 5. The RLOD calculations were performed using the Excel spreadsheet available at - RLOD (clause Calculation and interpretation of RLOD) version The RLOD are given Table 26. The inoculated strain used for the inoculation of the raw pork meat, S. Typhimurium Ad1872, was supposed to be monophasic strain (4,5,12:i:-). The serotyping gave the following antigenic formula: 4,5,12:-:-, which corresponds to a non-motile strain. This evolution can be due to the fact that the strain moved from smooth to rough form. ADRIA Développement 28/ February 2018

29 Table 26 Presentation of RLOD before and after confirmation of the alternative method results Target Name RLOD RLODL RLODU b=ln(rlod) sd(b) z-test p-value AL statistic Turkey ham/ S.Enteritidis Ad2524 0,728 0,328 1,612-0,318 0,398 0,799 1,576 2,5 Raw pork meat/ 1,023 0,469 2,233 0,023 0,390 0,059 0,953 2,5 S.Typhimurium Ad1872 spp. Proces water/ S.Typhimurium Ad1070 1,000 0,467 2,140 0,000 0,380 0,000 1,000 1,5 and S.Derby A00E084 Combined 0,931 0,596 1,455-0,071 0,223 0,319 1,250 / Enteritidis Turkey ham/.enteritidis Ad2524 0,728 0,328 1,612-0,318 0,398 0,799 1,576 2,5 Raw pork meat/ S.Typhimurium Ad1872 1,105 0,513 2,379 0,100 0,383 0,261 0,794 2,5 Typhimurium Process water/ S.Typhimurium Ad1070 0,562 0,211 1,492-0,577 0,489 1,181 1,762 1,5 and S.Derby AOOE084 Combined 0,831 0,458 1,508-0,185 0,298 0,621 1,465 / The RLOD are lower than the AL fixed at 2.5 for an unpaired study design and at 1.5 for a paired study design for all the tested matrix/strain pairs and for the alternative method. 3.3 Inclusivity / exclusivity The inclusivity is the ability of the alternative method to detect the target analyte from a wide range of strains. The exclusivity is the lack of interference from a relevant range of non-target strains of the alternative method Test protocols 161 strains and 30 non -target strains were tested: - 74 spp S. Typhimurium including monophasic, non-motile and classical variants; - 15 S. Enteritidis; - 28 strains from Group B; - 19 strains from Groups D1 and D2; - 30 non-target strains. ADRIA Développement 29/ February 2018

30 Inclusivity strains cultures were performed in BHI medium at 37 C. Dilutions were done in order to inoculate 10 to 100 cells/225 ml in BPW+ 12mg/l novobiocin. The enrichment broth was incubated for 14h at 41.5 C±1 C and the protocol of the alternative method was then run. When negative result was obtained using this protocol, the strain was tested again with addition of UHT milk in the supplemented BPW (25 ml ml). Exclusivity Negative strains cultures were performed in BHI at 37 C. Dilutions were realised in order to inoculate 10 5 cells/ml BPW. The BPW broth was then incubated 24 h at 37 C ± 1 C. The alternative method was be then performed Results Raw data are given in Appendix 6. Inclusivity All the strains tested gave the expected result, except for S. Blegdam 2011LSAL04969 and S. Moscow 1995LSAL05721 for these strains a positive Enteritidis PCR was obtained. These strains only differ by a change in expression of the H antigens of S. Blegdam during the genetic evolution of these strains, ( Changes induced in the H antigens of Blegdam, BrunerDW, 1952). Please see except below: From the results of this experiment it appears that S. blegdam (gmq) is a more primitive type than S. enteritidis (gm) or S. moscow (gq) and that they may have evolved from it as loss variants. The fact that S. blegdam is able to absorb the agglutinins from S. enteritidis as well as from S. moscow antiserum substantiates this assumption. Of the three types, S. enteritidis occurs most frequently in nature. It is widely distributed and is an old, established. The transformation of the induced S. enteritidis (gm) type to S. dublin (gp) also ADRIA Développement 30/ February 2018

31 supports the hypothesis that members of the g-complex of the genus have evolved by means of bacterial variation. The above studies indicate that the lineage of Enteritidis began with Blegdam which underwent loss mutations which gave rise to Enteritidis and Moscow. After this loss variation event, further genetic variation then gave rise to Nitra from Enteritidis (Figure 1). Genetic variation of S. Blegdam giving rise to serotypes Enteritidis, Moscow and Nitra. Figure 1 Blegdam g,m,q Loss mutation Enteritidis g,m Moscow g,q For 3 strains (S. Gallinarum Ad1840, S. Gallinarum Ad300 and Typhimurium Ad302) it was necessary to run the enrichment step with the addition of milk (25 ml milk ml enrichment broth). Exclusivity The 30 tested strains gave a negative PCR test. The inclusivity and exclusivity testing did give the expected results for the 161 target strains and the 30 non target strains. 2 strains, Blegdam and Moscow gave positive Enteritidis results. ADRIA Développement 31/ February 2018

32 3.4 Practicability The alternative method practicability was evaluated according to the AFNOR criteria relative to method comparison study. Storage conditions, shelf-life and modalities of utilisation after first use Store the unopened kit at 5 C ± 3 C. The individual components are stored: - at 5 C ± 3 C protect from light for lysis reagent one tube and Multiplex PCR tubes; - at 5 C ± 3 C for Proteinase K; - at room temperature for lysis tubes caps and PCR caps. Time to result Steps Reference method Alternative method Common step with the reference method Negative samples Sample enrichment Day 0 Day 0 Selective enrichment (RVS and MKTTn) Day 1 / Extraction / PCR / Day 1 Isolation on selective agar media Day 2 / Reading Day 3 / Total Day 3 Day 1 Presumptive positive or positive results Direct streaking on Brilliance Agar / Day 1 Subculture in RVS / Day 1 Isolation on selective agar media / Day 2 Reading plate / Day 3 Purification Day 4 / Confirmation Day 5 Day 4 The negative results are available in one day and the positive results in 4 days. ADRIA Développement 32/ February 2018

33 4 INTER-LABORATORY STUDY The inter-laboratory study is a study performed by multiple laboratories testing identical samples at the same time, the results of which are used to estimate alternative-method performance parameters. 4.1 Study organisation Collaborators number Ten laboratories were involved in the study. Due to a logistical issue, Lab G decided to not participate to the study. The analyses were conducted by two collaborators (i.e. technicians) in five laboratories (A, E, H, I and J) and one collaborator in four laboratories (B, C, D and F). Matrix and strain used Cooked ham sample was inoculated with Typhimurium Ad1410 isolated from ground pork meat. Samples Samples were prepared and inoculated on Monday 13 November 2017, as described below: - 24 blind coded samples (25 g) for analysis of by the Typhimurium and Enteritidis Multiplex PCR kit (red label) - 24 blind coded samples (25 g) for analysis of by the ISO (2017) reference method (blue label), - 1 sample for aerobic mesophilic flora enumeration by ISO method, - 1 water flask labelled Temperature Control with a temperature probe. ADRIA Développement 33/ February 2018

34 Inoculation The targeted inoculation levels were the following: - Level: 0 CFU/25 g, - Level 1: inoculation level close tot he RLOD in order to provide as much as possible fractional positive recovery data; - Level 2: 8 CFU/25 g. Labelling and shipping Blind coded samples were placed in isothermal boxes, which contained cooling blocks, and express-shipped to the different laboratories. A temperature control flask containing a sensor was added to the package in order to register the temperature profile during the transport, the package delivery and storage until analyses. Samples were shipped in 24 h to 48 h to the involved laboratories. The temperature conditions had to stay lower or equal to 8 C during transport, and between 0 C 8 C in the labs. Analyses Collaborative study laboratories and the expert laboratory carried out the analyses on Tuesday 14 November or Wednesday 15 November 2017 with the alternative and reference methods. The analyses by the reference method and the alternative method were performed on the same day. 4.2 Experimental parameters controls Strain stability and background microflora stability Strain stability was checked by inoculating the matrix at 1.5 CFU/25 g and 1000 CFU/g. Enumerations were performed for the high contamination level and detection analyses were performed for the low contamination level after 24 h and 48 h storage at 5 ± 3 C. Triplicates were analysed. The aerobic mesophilic flora was also enumerated at Day 0 and 2; the results are given in Table 27. ADRIA Développement 34/ February 2018

35 Table 27 - Sample stability Day Reference method (detection) (1.5 CFU/25 g) Enumeration (1000 CFU/g) Aerobic mesophilic flora Sample 1 Sample 2 Sample 3 Sample 1 Sample 2 Sample 3 (CFU/g) Day Day / Day No evolution was observed during storage at 5 C ± 3 C Contamination levels The contamination levels and the sample codification were the following (see Table 28). Table 28 - Contamination levels Level Samples Theoretical target level (b/25 g) True level (b/25 g sample) Low limit / 25 g sample High limit / 25 g sample 0 / / / Logistic conditions Temperature conditions are given in Table 29. ADRIA Développement 35/ February 2018

36 Collaborators Table 29 - Sample temperatures at receipt Temperature measured by the probe ( C) Temperature measured at receipt ( C) Receipt date and time Analysis date A /11/ h00 Day 1 A /11/ h00 Day 1 B /11/ h00 Day 1 C /11/ h40 Day 1 D /11/ h03 Day 1 E /11/ h10 Day 1 E /11/ h10 Day 1 F /11/ h15 Day 1 H /11/ h00 Day 1 H /11/ h00 Day 1 I /11/ h30 Day 2 I /11/ h30 Day 2 J /11/ h00 Day 1 J /11/ h00 Day 1 No problem was encountered during the transport or at receipt for the 14 collaborators. All the samples were delivered on time and in appropriate conditions. Temperatures during shipment and at receipt were all correct. 4.3 Results analysis The raw data are given in Appendix Expert laboratory results The results obtained by the expert laboratory are given in Table 30. Table 30 Results obtained by the expert Lab. Level Reference method Alternative method L0 0/8 0/8 L1 6/8 8/8 L2 8/8 8/8 ADRIA Développement 36/ February 2018

37 4.3.2 Results observed by the collaborative laboratories Aerobic mesophilic flora enumeration Depending on the Lab results, the enumeration levels varied from to CFU/g. detection 14 collaborators participated to the study. The results obtained are provided in Table 31 (reference method) and Table 32 (alternative method). Table 31 - Positive results by the reference method (ALL the collaborators) Collaborator Contamination level L0 L1 L2 A A B C D E E F H H I I J J TOTAL P 0 = 2 P 1 = 89 P 2 = 112 ADRIA Développement 37/ February 2018

38 Table 32 - Positive results (before and after confirmation) by the alternative methods (ALL the collaborators) Contamination level Collaborators L0 L1 L2 PCR Confirmation Final PCR Confirmation Final PCR Confirmation Final result result result result result result result result result A A B C D E E F H H I I J J Total P0 = 4 17 CP0 = 2 P1 = CP1 = 88 P3 = CP3 = 111 Several positive results were observed on unspiked samples: - Lab A1 2 positive PCR results were observed for Typhimurium detection. For these 2 samples, the PCR results for spp. detection were negative. The presence of Typhimurium was confirmed in the enrichment broth for 7 samples. Cross contamination probably occurred during manipulation. This Lab precised: Sample A8 (alternative method) leaked when BPW added. This sample was discarded and replaced with sterile water. - Lab A2 1 positive result was observed for the reference method. For the alternative method, the presence of was confirmed in the enrichment broth for 7 samples while all the PCR tests gave negative results. Cross contamination probably occurred during manipulation. 3 One sample (A8) not tested due to leakage ADRIA Développement 38/ February 2018

39 - Lab B The presence of Typhimurium was confirmed for one sample while the PCR test was negative. - Lab F 1 positive result was observed for the reference method. - Lab H1 Positive PCR results ( spp. and Typhimurium) were observed for 2 samples; the presence of in the enrichment broth was also confirmed for these 2 samples. This Lab precised that many bags were leaking. According to the AFNOR technical rules, it is possible to include the results from a collaborator with maximum one cross contamination at Level 0. For this study, this rule was applied. Based on the observed results, the data from Labs A1, A2 and H1 were not kept for interpretation Results of the collaborators retained for interpretation The results obtained with the 11 Labs kept for interpretation are presented in Table 33 (reference method) and Table 34 (alternative method). Table 33 - Positive results by the reference method (Without Labs A1, A2 and H1) Collaborators Contamination level L0 L1 L2 B C D E E F H I I J J TOTAL P 0 = 1 P 1 = 74 P 2 = 88 ADRIA Développement 39/ February 2018

40 Table 34 - Positive results (before and after confirmation) by the alternative methods (Without Labs A1, A2 and H1) Contamination level Collaborators L0 L1 L2 PCR Confirmation Final PCR Confirmation Final PCR Confirmation Final result result result result result result result result result B C D E E F H I I J J Total P 0 = 0 1 CP 0 = 0 P 2 = CP 2 = 67 P 3 = CP 3 = Calculation and interpretation Calculation of the specificity percentage (SP) The percentage specificities (SP) of the reference method and of the alternative method, using the data after confirmation, based on the results of level L0 are the following (See Table 35). Table 35 - Percentage specificity Specificity for the reference method Specificity for the alternative method 98.9 % % N: number of all L0 tests P 0 = total number of false-positive results obtained with the blank samples before confirmation CP 0 = total number of false-positive results obtained with the blank samples ADRIA Développement 40/ February 2018

41 4.4.2 Calculation of the sensitivity (SE alt ), the sensitivity for the reference method (SE ref ), the relative trueness (RT) and the false positive ratio for the alternative method (FPR) Fractional positive results were obtained for the low inoculation level (L1). This inoculation level was retained for calculation. A summary of the results of the collaborators retained for interpretation, and obtained with the reference and the alternative methods for Level 1 is provided in Table 36. Table 36 - Summary of the obtained results with the reference method and the alternative method for Level 1 Response Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 58 Negative deviation (A-/R+) ND = 16 Reference method negative (R-) Positive deviation (R-/A+) PD = 9 Negative agreement (A-/R-) NA = 5 Based on the data summarized in Table 10, the values of sensitivity of the alternative and reference methods, as well as the relative trueness and false positive ratio for the alternative method taking account the confirmations, are the following (See Table 37). Table 37 - Sensitivity, relative trueness and false positive ratio percentages Sensitivity for the alternative method: SE alt = 80.7 % Sensitivity for the reference method: SE ref = 89.2 % Relative trueness RT = 71.6 % False positive ratio for the alternative method FPR = 0.0 % Interpretation of data For Level 1, the negative deviations are listed in Table 38 and the positive deviations are in Table 39. ADRIA Développement 41/ February 2018

42 Table 38 - Negative deviations Collaborator Sample No Confirmation B B1 - B5 - B11 - B17 - C C11 - C14 - D D1 - D9 - D14 - E E26 - E29 - H H35 - I I9 - I29 - J J9 - J26 - For 15 samples in negative deviations, the presence of spp. was not confirmed in the enrichment broth. These results were probably due to the unpaired study design. Table 39 - Positive deviations Collaborator C E I J Sample No C1 - C21 E1 - E35 I14 - I37 J17 - J31 - J35 For an unpaired study design, the difference between (ND PD) is calculated for the level(s) where fractional recovery is obtained (L 1 ). The observed value found for (ND PD) shall not be higher than the AL. The AL is defined as [(ND PD) max ] and calculated per level where fractional recovery is obtained as described below using the following three parameters: P p x ref where P x = number of samples with a positive result obtained with the reference method at level L 1 for all the collaborators N x = number of samples tested at level L 1 with the reference method by all the collaborators N CP x p x alt N x ADRIA Développement 42/ February 2018

43 where CP x = number of samples with a confirmed positive result obtained with the alternative method at level L 1 for all the collaborators; N x = number of samples tested at level L 1 with the alternative method by all the collaborators. ND-PD 3N p p 2 p p max x ref alt ref alt where N x = number of samples tested for level L 1 with the reference method by all the collaborators. The AL is not met when the observed value is higher than the AL. When the AL is not met, investigations should be made (e.g. root cause analysis) in order to provide an explanation of the observed results. Based on the AL and the additional information, it is decided whether the alternative method is regarded as not fit for purpose. The reasons for acceptance of the alternative method when the AL is not met shall be stated in the study report. In this study, fractional recovery was observed at Level 1. The calculations are the following, according to the EN ISO :2016 (See Table 40). Table 40 - Calculations NX 88 (p+)ref 0.8 (p+)alt 0.8 AL = (ND - PD) max 9.22 ND - PD 7 Conclusion ND - PD < AL The ISO (2016) requirements are fulfilled as (ND - PD) is lower than the AL. ADRIA Développement 43/ February 2018

44 4.4.4 Evaluation of the RLOD between laboratories The RLOD was calculated using the EN ISO :2016 Excel spreadsheet available at - RLOD (clause Calculation and interpretation of RLOD) version The results are used only for information (see Table 41). Table 41 - RLOD RLOD RLODL RLODU b=ln(rlod) sd(b) z-test statistic p-value 1,303 0,895 1,896 0,265 0,188 1,411 0,158 5 CONCLUSION The method comparison study conclusions are: The method comparison study scheme corresponds to an UNPAIRED STUDY design as the alternative and reference methods have different enrichment procedures for food categories and a PAIRED STUDY design as the alternative and reference methods have a common enrichment procedure for the production environmental samples category. In the sensitivity study, 3 categories were tested: 2 food categories and the production environmental samples. The protocol of the alternative method shows 6 positive deviations (PD) and 9 negative deviations (ND). The calculated values for (ND - PD) are lower than the acceptability limits (AL) for each individual category and also for the 3 combined categories. The Relative Levels of Detection (RLOD) are all lower than the AL fixed at 2.5 for the unpaired data study for the food categories, and lower than the AL fixed at 1.5 for a paired data study as for the production environmental samples category. The inclusivity and exclusivity testing did give the expected results for 159 target strains among the 161 tested and the 30 non target strains. 2 strains, Blegdam and Moscow gave positive Enteritidis results. ADRIA Développement 44/ February 2018