NF VALIDATION Validation of alternative analytical methods Application in food microbiology. Summary report

Transcription

1 ACCREDITATION N PORTEE DISPONIBLE SUR Boulevard de l Oise F CERGY PONTOISE CEDEX NF VALIDATION Validation of alternative analytical methods Application in food microbiology Summary report EN ISO :2016 validation study of the TM Molecular Detection Assay 2 - Listeria monocytogenes for detection of Listeria monocytogenes in food products and environmental samples Qualitative method This report includes 66 pages, with 5 appendixes. Only copies including the totality of this report are authorized. Competences of the laboratory are certified by COFRAC accreditation for the analyses marked with symbol. Version 0 November 14, 2016 ADRIA DEVELOPPEMENT Creac h Gwen - F QUIMPER Cedex - Tél. (33) Fax (33) adria.developpement@adria.tm.fr - Site web : ASSOCIATION LOI DE N SIRET N EXISTENCE N TVA FR

2 1 AIM OF THE STUDY 4 2 METHOD PROTOCOLS Reference method Alternative method 4 3 METHOD COMPARISON STUDY Sensitivity study Number and nature of samples Artificial contamination of samples Confirmation protocols Test results Calculation of relative accuracy (RT), relative sensitivity (SE) and false positive ratio (FPR) Analysis of discordants Enrichment broth storage at 2-8 C for 72 h Confirmation Inhibitions Relative level of detection study Experimental design Calculation and interpretation of the RLOD Inclusivity / exclusivity Test protocols Results Practicability 22 4 INTER-LABORATORY STUDY ORGANISATION AND RESULTS Study organisation Experimental parameters controls Sample stability Logistic conditions Calculation and summary of data Results obtained by the expert Lab Results obtained by the collaborators Interpretation 32 5 CONCLUSION 34 Appendix 1 Flow diagram of the EN ISO /A1 (2004) method: Horizontal method for the detection and enumeration of Listeria monocytogenes - Part 1: detection of Listeria monocytogenes in foods 36 Appendix 2 Flow diagram of the TM Molecular Detection Assay 2 - Listeria monocytogenes 37 Appendix 3 Relative level of detection study: raw data 40 Appendix 4 Inclusivity and exclusivity study: raw data 46 Appendix 5 Results obtained by the collaborators and the expert laboratory 51 ADRIA Développement 2/66 November 14, 2016 MDA 2 - Listeria monocytogenes

3 Quality Assurance documents related to this study can be consulted upon request from. The technical protocol and the result interpretation were realised according to the EN ISO :2016 and the AFNOR technical rules. Company: Boulevard de l Oise F CERGY PONTOISE CEDEX Expert Laboratory: ADRIA Développement ZA Creac h Gwen F QUIMPER Cedex Studied method: TM Molecular Detection Assay 2 - Listeria monocytogenes Validation standard : EN ISO : Microbiology of the food chain - Method validation Part 1: Vocabulary Part 2: Protocol for the validation of alternative (proprietary) methods against a reference method Standard method : EN ISO /A1 (2004): Microbiology of food and animal feeding stuffs - Horizontal method for the detection and enumeration of Listeria monocytogenes - Part 1: detection of Listeria monocytogenes in foods Scope: Food products and Environmental samples (excluding samples from primary production) Certification body: AFNOR Certification Analyses performed according to the COFRAC accreditation ADRIA Développement 3/66 November 14, 2016 MDA 2 - Listeria monocytogenes

4 1 AIM OF THE STUDY The TM Molecular Detection Assay 2 - Listeria monocytogenes for the detection of Listeria monocytogenes was validated for human food products and environmental samples (excluding samples from primary production) was validated in September 2016 (Certificate number: 01/15-09/16) according to the EN ISO (2016). 2 METHOD PROTOCOLS 2.1 Reference method The reference method correspond to the ISO /A1 (2004) standard: Microbiology of food and animal feeding stuffs - Horizontal method for the detection and enumeration of Listeria monocytogenes Part 1: detection method (See Appendix 1). 2.2 Alternative method Principle The TM Molecular Detection Assay 2 - Listeria monocytogenes Test Kit uses isothermal amplification of unique DNA target sequences, the amplified sequences are detected by bioluminescence. Protocol (see Appendix 2) - Enrichment step in Half Fraser broth at 37 C ± 1 C: * Specific protocol: 25 g ml Half Fraser for raw dairy, raw meat and raw seafood products. Incubation for 22 h ± 2 h at 37 C. Subculture in Fraser broth for 22 h ± 2 h at 37 C. * General protocol: 25 g ml Half Fraser for all other products (food and environmental samples). Incubation for 27 h ± 3 h at 37 C - Lysis step - Amplification and real-time detection Analysis performed according to the COFRAC accreditation ADRIA Développement 4/66 November 14, 2016 MDA 2 - Listeria monocytogenes

5 - Confirmation step by streaking 100 µl of enriched Half Fraser broth onto O&A and Palcam plates (General protocol), and by streaking 10 µl of Fraser broth (Specific protocol). It is possible to store the enrichment broths for 72 h at 2-8 C for all the categories and to store the lysates for 72 h at 2-8 C for all the categories, except for Dairy products and environmental samples, before running the MDA test. The flow diagram is given in Appendix 2. 3 METHOD COMPARISON STUDY As the enrichment procedure is different between the alternative and the reference methods, this corresponds to an UNPAIRED DATA STUDY DESIGN. 3.1 Sensitivity study The sensitivity (SE) is the ability of the method to detect the analyte by either the reference or alternative method Number and nature of samples 400 samples were analyzed. The distribution per tested category and type is given in Table 1. ADRIA Développement 5/66 November 14, 2016 MDA 2 - Listeria monocytogenes

6 Table 1 Distribution per tested category and type Categories COMPOSITE FOODS/ READY-TO-EAT AND READY-TO-REHEAT MEAT PRODUCTS DAIRY PRODUCTS VEGETABLES SEAFOOD AND FISHERY PRODUCTS ENVIRONMENTAL SAMPLES Types Positive sample Negative samples Total a Ready-to-eat b Ready-to-reheat c Pastries and egg products Total a Meat products (raw, frozen, seasoned) b Ready-to-eat and processed meat products c Delicatessen Total a Raw dairy products b Pasteurized milk cheeses c Ice cream, milk (pasteurized), flavoured milk (pasteurized) Total a Raw vegetable products (fresh and frozen) b Mapped vegetables and heat processed vegetables c Vegetables based preparations, processed vegetables Total a Raw products (fresh, frozen) b Smoked, marinated c Ready-to-eat or ready-to-reheat Total a Process water b Dusts c Wipes Total TOTAL Artificial contamination of samples Artificial contaminations were done by seeding protocol. 87 samples were artificially contaminated, using 37 different strains. 62 gave a positive result. All the samples were inoculated at level 3 CFU % of the samples were naturally contaminated Confirmation protocols The positive tests were confirmed by streaking 100 µl of the enrichment broth onto O&A. During the validation study, the typical colonies were identified by the tests described in the reference method. ADRIA Développement 6/66 November 14, 2016 MDA 2 - Listeria monocytogenes

7 3.1.4 Test results The results are given in the following tables. Table 2 Summary of results obtained with the reference and the alternative method: All the samples Response Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 119 Negative deviation (A-/R+) ND = 33 (PPND = 1) Reference method negative (R-) Positive deviation (R-/A+) PD = 35 Negative agreement (A-/R-) NA = 213 (PPNA = 2) PP: positive presumptive non confirmed samples PD = positive deviation (R-/A+) ND = negative deviation (A-/R+) Results per category of samples Table 3 Summary of results obtained with the reference and the alternative method: composite foods / ready-to-eat and ready-to-reheat Response Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 22 Negative deviation (A-/R+) ND = 3 Reference method negative (R-) Positive deviation (R-/A+) PD = 5 Negative agreement (A-/R-) NA = 32 Table 4 Summary of results obtained with the reference and the alternative method: Meat products Response Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 17 Negative deviation (A-/R+) ND = 9 Reference method negative (R-) Positive deviation (R-/A+) PD = 8 Negative agreement (A-/R-) NA = 37 ADRIA Développement 7/66 November 14, 2016 MDA 2 - Listeria monocytogenes

8 Table 5 Summary of results obtained with the reference and the alternative method: Dairy products Response Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 18 Negative deviation (A-/R+) ND = 4 Reference method negative (R-) Positive deviation (R-/A+) PD = 8 Negative agreement (A-/R-) NA = 33 Table 6 Summary of results obtained with the reference and the alternative method: Vegetables Response Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 19 Negative deviation (A-/R+) ND = 6 (PPND = 1) Reference method negative (R-) Positive deviation (R-/A+) PD = 6 Negative agreement (A-/R-) NA = 37 Table 7 Summary of results obtained with the reference and the alternative method: Seafood and fishery products Response Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 22 Negative deviation (A-/R+) ND = 4 Reference method negative (R-) Positive deviation (R-/A+) PD = 4 Negative agreement (A-/R-) NA = 32 Table 8 Summary of results obtained with the reference and the alternative method: Environmental samples Response Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 21 Negative deviation (A-/R+) ND = 7 Reference method negative (R-) Positive deviation (R-/A+) PD = 4 Negative agreement (A-/R-) NA = 42 (PPNA = 2) ADRIA Développement 8/66 November 14, 2016 MDA 2 - Listeria monocytogenes

9 The summary of results per protocol is given tables 10 and 11. Table 9 Summary of results obtained with the reference and the alternative method: General protocol Response Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 91 Negative deviation (A-/R+) ND = 28 (PPND = 1) Reference method negative (R-) Positive deviation (R-/A+) PD = 28 Negative agreement (A-/R-) NA = 183 (PPNA = 2) Table 10 Summary of results obtained with the reference and the alternative method: Specific protocol Response Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 28 Negative deviation (A-/R+) ND = 5 Reference method negative (R-) Positive deviation (R-/A+) PD = 7 Negative agreement (A-/R-) NA = Calculation of relative accuracy (RT), relative sensitivity (SE) and false positive ratio (FPR) The calculations are presented in table 11. ADRIA Développement 9/66 November 14, 2016 MDA 2 - Listeria monocytogenes

10 Table 11 Calculation of the relative accuracy (RT), the relative sensitivity (SE) and the false positive ratio (FPR) 1 Category Type PA NA PD ND PPND PPNA SE alt % SE ref % RT % FPR % Composite foods/ Ready to eat/ Ready to reheat 2 Meat products 3 Dairy products 4 Vegetables 5 Seafood and fishery products 6 Environmental samples a Ready to eat b Ready to reheat c Pastries, desserts, omelettes Total a Raw, frozen, seasoned b Ready to eat and processed meat products c Delicatessen Total a Raw dairy products b Pasteurized milk cheese c Ice cream, milk (pasteurized), flavored milk Total a Raw vegetable products (fresh, frozen) b Mapped vegetables and heat processed vegetables c Vegetables based preparations, processed vegetables Total a Raw products (fresh, frozen) b Smoked, marinated c Ready to eat or ready to reheat Total a Process water b Dusts c Wipes Total All categories ADRIA Développement 10/66 November 14, 2016 MDA 2 - Listeria monocytogenes

11 The following results are observed: Sensitivity for the alternative method Sensitivity for the reference method Table 12 PA PD SE alt 100% 82.4 % ( PA ND PD) PA ND SE ref 100% 81.3 % ( PA ND PD) ( PA NA) Relative trueness RT 100% 83.0 % N False positive ratio for the ( FP) FPR 100% alternative method* NA 1.4 % FP = PPNA + PPND * With ND = ND + PPND NA = NA + PPNA Analysis of discordants Among the 33 negative deviations (see table 13), 13 concern artificially contaminated samples and 20 naturally contaminated samples. The presence of Listeria monocytogenes was confirmed for 7 samples by the tests of the alternative method (streaking onto O&A plates) and one additional sample (No 2565) was confirmed by applying the whole protocol of the reference method (subculture in Fraser for 48 h at 37 C and streaking onto O&A). For 25 samples, the negative results were probably due to sampling heterogeneity in this unpaired data study when dealing with low contamination levels. Among the 35 positive deviations (see table 14), 9 concern artificially contaminated samples and 24 concern naturally contaminated samples. ADRIA Développement 11/66 November 14, 2016 MDA 2 - Listeria monocytogenes

12 Table 13 - Negative deviations Category Composite foods / Ready to eat and ready to reheat Meat products Dairy products Vegetables Type Inoculation Confirmation result Sample MDA Final Product Inoculated strain level N result Reference method result (cfu/sample) Alternative method (Fraser 48h 37 C) b 2761 RTRH composite food -/-/- L.monocytogenes H+ - c 3202 Tortilla L.monocytogenes Ad / / - a 3748 Sandwich (ham, cheese) L.monocytogenes Ad / - - c 1097 Rillettes - L.grayi - - c 4451 Cooked ham - / - - c 4664 Smoked sliced bacon - / - - a 6178 Raw chicken meat -/-/- L.monocytogenes / - a 6179 Beef meat -/-/- L.monocytogenes / - a 6186 Ground poultry - / - - a 6365 Gizzards - / - - b 7553 Ready to reheat meal L.monocytogenes A00C / - - b 7558 Ready to reheat meal (beef) L.monocytogenes Ad / - - c 3189 Pasteurized half skimmed milk L.monocytogenes A00L / / - b 3405 Pasteurized cow milk cheese L.monocytogenes A00L /-/- L.monocytogenes + - b 3752 Pasteurized milk cheese L.monocytogenes Ad /-/- L.monocytogenes + - b 7561 Pasteurized ewe milk cheese L.monocytogenes Ad / - - b 4443 Heat processed vegetables - / - - a 4445 Sprouts - / - - b 4447 Zucchini + L.innocua - - b 4454 Frozen broccoli - / - - c 7372 Frozen vegetables mix - L.welshimeri/ L.innocua - - c 7570 Processed vegetables mix L.monocytogenes 1011/ / - - ADRIA Développement 12/66 November 14, 2016 MDA 2 - Listeria monocytogenes

13 Category Seafood and fishery products Environmental samples Type Inoculation Confirmation result Sample MDA Final Product Inoculated strain level N result Reference method result (cfu/sample) Alternative method (Fraser 48h 37 C) c 1100 RTRH salmon - L.welshimeri - - c 2565 RTRH salmon balls - / H+(L.mono) - c 2938 Cooked shrimps i/+* L.monocytogenes / - a 6200 Raw fish - / - - c 4142 Swab (dairy environment) L.monocytogenes Ad / - - b 4384 Wastes (Fish industry) - L.innocua - - b 6869 Wastes (Meat industry) - L.welshimeri - - c 6873 Wipe (Dairy industry) -/i/+ L.monocytogenes/ L.innocua / - b 6877 Wastes (meat) L.monocytogenes Ad /+/+ L.innocua - - b 6878 Wastes (meat) L.monocytogenes Ad /+/+ L.welshimeri - - b 6881 Wastes (meat) L.monocytogenes Ad1255 L.innocua Ad L.innocua - - ADRIA Développement 13/66 November 14, 2016 MDA 2 - Listeria monocytogenes

14 Table 14 - Positive deviations Category Composite foods / Ready to eat / Ready to reheat Meat products Dairy products Vegetables Type Sample N Product Inoculated strain Inoculation level (cfu/sample) MDA result Confirmation result b 1076 Deli salad + L.monocytogenes + a 1079 RTE + L.monocytogenes + a 2760 Sandwich (goat cheese, vegetables) + L.monocytogenes + a 2933 Chinese pasta + L.monocytogenes + b 3150 RTRH composite food + L.monocytogenes + b 2952 Sliced cooked chicken + L.monocytogenes + c 3158 Terrine + L.monocytogenes + c 3165 Sausages with aromatic herbs + L.monocytogenes/ L.innocua + c 4441 Cooked sausage +/-/- L.monocytogenes /L.innocua + a 6180 Sheep meat + L.monocytogenes (72h) + a 6183 Chicken leg + L.monocytogenes + a 6190 Chicken leg + L.monocytogenes + b 7557 Ready to reheat meal (beef) L.monocytogenes Ad L.monocytogenes + c 3190 Pasteurized half skimmed milk L.monocytogenes L.monocytogenes + c 3192 Cocoa flavored milk L.monocytogenes A00L L.monocytogenes + b 3404 Pasteurized cow milk cheese L.monocytogenes A00L L.monocytogenes + b 3749 Pasteurized cow milk cheese L.monocytogenes Ad L.monocytogenes + a 4149 Raw milk cheese L.monocytogenes Ad L.monocytogenes + a 4703 Raw milk cheese + L.monocytogenes + a 6246 Raw ewe milk cheese + L.monocytogenes + b 7563 Pasteurized ewe milk cheese L.monocytogenes Ad L.monocytogenes + b 3539 Frozen mushrooms + L.monocytogenes + b 4458 Frozen precooked onions + L.monocytogenes + b 4459 Frozen zucchini + L.monocytogenes (+72h)/L.innocua + b 4461 Vegetables for couscous + L.monocytogenes + a 4655 Pieces of chestnuts + L.monocytogenes + c 7569 Processed vegetables mix L.monocytogenes 1011/ L.monocytogenes + Final result ADRIA Développement 14/66 November 14, 2016 MDA 2 - Listeria monocytogenes

15 Category Seafood and fishery products Environmental samples Type Sample N Product Inoculated strain Inoculation level (cfu/sample) MDA result Confirmation result b 3528 Smoked salmon + L.monocytogenes + b 3531 Smoked salmon + L.monocytogenes + b 3532 Smoked salmon + L.monocytogenes + a 6194 Raw fish + L.monocytogenes (Rapid'L.mono) + b 6870 Wastes (meat) + L.monocytogenes + b 6879 Wastes (meat) L.monocytogenes Ad1255 L.innocua Ad L.monocytogenes + b 6880 Wastes (meat) L.monocytogenes Ad1255 L.monocytogenes (Fraser 1) / L.innocua Ad1251 L.welshimeri + b 7932 Dusts L.monocytogenes Ad L.monocytogenes + Final result ADRIA Développement 15/66 November 14, 2016 MDA 2 - Listeria monocytogenes

16 The analyses of discordant results according to the EN ISO : 2016 is the following (See table 15): Table 15 1 Category Composite foods/ready to eat/ready to reheat 2 Meat products 3 Dairy products 4 Vegetables 5 Seafood and fishery products 6 Environmental samples Type (ND+PPND)- PD a Ready to eat -2 b Ready to reheat -1 c Pastries, desserts, omelettes 1 AL Total -2 3 a Raw, frozen, seasoned 1 b Ready to eat and processed meat products 0 c Delicatessen 0 Total 1 3 a Raw dairy products -3 b Pasteurized milk cheese 0 c Ice cream, milk (pasteurized), flavored milk -1 Total -4 3 a Raw vegetable products (fresh, frozen) 0 b Mapped vegetables and heat processed vegetables -1 c Vegetables based preparations, processed vegetables 1 Total 0 3 a Raw products (fresh, frozen) 0 b Smoked, marinated -3 c Ready to eat or ready to reheat 3 Total 0 3 a Process water 0 b Dusts 1 c Wipes 2 Total 3 3 All categories -2 6 The observed values for ((ND + PPND) - PD) are below or equal to the acceptability limit for each category and for all the categories. Additional Listeria monocytogenes strains were recovered from five samples with negative agreement results (See table 16). Three samples were artificially contaminated; the other one were naturally contaminated samples. ADRIA Développement 16/66 November 14, 2016 MDA 2 - Listeria monocytogenes

17 Table 16 - Strains recovered from samples with NA results Category Sample No Meat products 7554 Vegetables Dairy products 3195 Environmental samples Product Ready-to-reheat meat Vegetable based preparation Precooked frozen onions Pasteurized milk cheese MDA test MDA confirmation tests O&A Identification Reference method * - / + H+ L. monocytogenes + b - H+ L. monocytogenes + c - / - / + H+ L. monocytogenes / b - / - / - H+ L. monocytogenes + b 4144 Wastes (fish) - H+ L. monocytogenes + b * Subculture in Fraser broth from the Half Fraser of the alternative method Type Enrichment broth storage at 2-8 C for 72 h 224 samples were tested again after enrichments and lysates storage for 72 h at 2-8 C. The following changes are observed (See table 17). Table 17 Sample No Before storage After storage Enrichment Lysate 2761 ND PA ND 3150 PD PD NA 3146 NA PD PD 4441 PD PD NA 6178 ND PA PA 3527 PA PA ND 3538 NA PD NA 6179 ND ND PA 3749 PD PD NA 4382 PA PA ND 3405 ND PA PA 3752 ND PA ND 4155 PA PA ND 6248 PA PA ND 6189 PA PA ND 6201 PA PA ND 6203 PA PA ND 6873 ND PA PA 7574 NA PD NA 6881 ND PA PA ADRIA Développement 17/66 November 14, 2016 MDA 2 - Listeria monocytogenes

18 The analysis of discordant becomes (See table 18). Table 18 Category Enrichment storage for 72h at 2-8 C Lysate storage for 72h at 2-8 C PD ND+PPND (ND+PPND)-PD AL PD ND+PPND (ND+PPND)-PD AL Composite foods /Ready to eat / Ready to reheat Meat products Dairy products Vegetables Seafood and fishery products Environmental samples Total The observed values for ((ND + PPND) - PD) are below or equal to the acceptability limit for each category and for all the categories. Based on the AFNOR technical rules, the deviations and positive samples have to be tested for the storage protocols. Three NA were as well tested here: the samples No 3146, 5358 and They all became positive after storage of the broth, and one sample became positive after storage of the lysate. Note that the inoculation level requirements were as well recently changed in the AFNOR technical rules, corresponding now more to the limits of detection of the methods. In that way, the probability to get positive data from firstly screened negative samples is higher, particularly if these samples were supposed to be naturally contaminated or inoculated. As the AL are based on differences between ND and PD, it may be relevant to tests as well the NA Confirmation Confirmations of the positive MDA tests were performed by streaking the enrichment broth (Half Fraser for General protocol and Fraser for Specific protocol) onto O&A plates. In 8 cases, it was necessary to apply additional confirmatory tests to confirm the presence of Listeria monocytogenes in the enrichment broth; they are listed in table 19. ADRIA Développement 18/66 November 14, 2016 MDA 2 - Listeria monocytogenes

19 Table 19 Sample No Additional tests applied 4441 Subculture in Fraser 3530 Streaking onto RAPID L.mono agar 4459 Recovered from enrichment storage for 72 h at 2-8 C 6203 Streaking onto RAPID L.mono agar 6368 Streaking onto RAPID L.mono agar 6194 Streaking onto RAPID L.mono agar 6870 Recovered from enrichment storage for 72 h at 2-8 C 6880 Subculture in Fraser Inhibitions One inhibition was observed (sample No 2938); this sample gave a positive MDA test after lysate dilution (1/10). This sample was considered negative for interpretation, and gave a negative deviation. 3.2 Relative level of detection study The relative level of detection is the level of detection at P = 0.50 (LOD50) of the alternative (proprietary) method divided by the level of detection at P = 0.50 (LOD50) of the reference method. The RLOD is defined as the ratio of the alternative and reference methods: Experimental design Six (matrix/strain) pairs were analyzed by the reference method and by the alternative method (See Table 20): ADRIA Développement 19/66 November 14, 2016 MDA 2 - Listeria monocytogenes

20 Table 20 - Defined (matrix/strain) pairs for the RLOD determination Matrix and related category number Inoculated strain Origin Inoculation protocol Analysis protocol - Deli-salad (Piémontaise) Listeria monocytogenes Ad494 Deli salad General - Beef meat Listeria monocytogenes Ad1206 Ground beef Seeding Specific - Raw milk Listeria monocytogenes Ad618 Cheese protocol Specific - Bagged raw spinach Listeria monocytogenes Ad1178 Ready-to-cook with 4 C General - Cold smoked salmon Listeria monocytogenes Ad670 Smoked salmon storage prior General - Process water Listeria monocytogenes Ad551 Environmental analyses sample General Calculation and interpretation of the RLOD The raw data are given in Appendix 3. The RLOD calculations were performed using Excel spreadsheet of the international standard (ISO 16140), as described in the EN ISO :2016 standard ( The RLOD are given table 21. Table 21 Presentation of RLOD before and after confirmation of the alternative method results Name RLOD RLODL RLODU b=ln z-test sd(b) (RLOD) statistic p-value Deli-salad (Piémontaise) / L.monocytogenes Ad Beef meat / L.monocytogenes Ad Raw milk / L.monocytogenes Ad Bagged raw spinach / L.monocytogenes Ad Cold smoked salmon / L.monocytogenes Ad Process water / L.monocytogenes Ad The RLOD are below the AL fixed at 2.5 for unpaired studies for all the tested matrix/strain pairs and for the alternative method. ADRIA Développement 20/66 November 14, 2016 MDA 2 - Listeria monocytogenes

21 3.3 Inclusivity / exclusivity Inclusivity is the ability of the alternative method to detect the target analyte from a wide range of strains. Exclusivity is the lack of interference from a relevant range of non-target strains of the alternative method Test protocols 50 targeted strains and 30 non targeted strains were tested. Inclusivity 50 Listeria monocytogenes strains were thawed and grown in BHI broth overnight at 37 C ± 1 C. The cultures were diluted in peptone salt in order to inoculate between 10 to 100 cells per 225 ml of Half Fraser. The enrichment broths were then incubated for 24 h at 37 C ± 1 C. The protocol of the alternative method was then applied. Exclusivity 30 non Listeria strains were thawed and grown in BHI broth overnight at 37 C ± 1 C. The cultures were diluted in order to inoculate 10 5 cells/ml of an appropriate media (BPW or MRS). The broths were then incubated for 24 h at the appropriate incubation temperature in order to have culture to test with the alternative method Results Raw data are given in Appendix 4. Inclusivity The 50 specific inclusivity strains tested were detected. Exclusivity Among the 30 non target strains tested, no cross reaction was observed. ADRIA Développement 21/66 November 14, 2016 MDA 2 - Listeria monocytogenes

22 3.4 Practicability The alternative method practicability was evaluated according to the AFNOR criteria relative to method comparison study. Storage conditions, shelflife and modalities of utilization after first use Time to result The storage temperature is: 2-8 C. The shelf-life is given on the package. All the reagents shall be stored at the temperature mentioned on the package. Negative samples Steps Reference method Alternative method General protocol Specific protocol Sampling enrichment Day 0 Day 0 Day 0 Subculture in Fraser 1 Day 1 / / Lysate / Day 1 Day 2 MDA test / Day 1 Day 2 Streaking onto O&A plates (O1/P1) Day 1 / / Second streaking (O2/P2) Day 3 / / Reading plates (O1 P1) Day 2 - Day 3 / / Reading plates (O2 P2) Day 4 - ay 5 / / Results Day 5 Day 1 Day 2 Common step with the reference method Presumptive positive or positive results Steps Reference method Alternative method General protocol Specific protocol Sub-culture of typical colonies Day 2 - Day 5 / / Streaking onto O&A plates / Day 1 Day 2 O&A plates reading / Day 2 - Day 3 Day 3 - Day 4 Confirmatory tests Day 3 - Day 6 / / Day 4 - Day 7 Results Day 2 - Day 3 Day 3 - Day 4 Day 8 - Day 11 (1) (1) In the case of the rhamnose and xylose, tests are realized in tubes. No common step The negative results are available in one day for the general protocol, and in two days for the specific protocol. The positive results are available in two or three days for the general protocol, and in three or four days for the specific protocol. ADRIA Développement 22/66 November 14, 2016 MDA 2 - Listeria monocytogenes

23 4 INTER-LABORATORY STUDY ORGANISATION AND RESULTS 4.1 Study organisation Collaborators number Samples were sent to 13 laboratories. For 2 Labs, 2 technicians were involved in the study: Labs K and L in order to have 15 sets of data. Matrix and strain used Pasteurised cheese sample (Brie: 31 % fat, 1.4 % NaCl) was inoculated with Listeria monocytogenes 153, isolated from raw milk cheese. Samples Samples were prepared and inoculated on Monday 27 th described below: June 2016, as - 24 blind coded samples for analysis of Listeria monocytogenes by the TM Molecular Detection Assay 2 - Listeria monocytogenes method (= RED LABEL) - 24 blind coded samples for Listeria monocytogenes analysis by the reference method (EN ISO /A1) (= BLUE LABEL) - 1 sample for aerobic mesophilic flora enumeration by ISO method, - 1 water flask labelled Temperature Control with a temperature probe. Inoculation The targeted inoculation levels were the following: - Level: 0 CFU/25 g, - Level 1: 2 CFU/g, inoculation level providing as much as possible fractional positive recovery data; - Level 2: 8 CFU/25 g. ADRIA Développement 23/66 November 14, 2016 MDA 2 - Listeria monocytogenes

24 Labelling and shipping Blind coded samples were placed in isothermal boxes, which contained cooling blocks, and express-shipped to the different laboratories. A temperature control flask containing a sensor was added to the package in order to register the temperature profile during the transport, the package delivery and storage until analyses. Samples were shipped in 24 h to 48 h to the involved laboratories. The temperature conditions had to stay lower or equal to 8 C during transport, and between 0 C 8 C in the labs. Analyses Collaborative study laboratories and the expert laboratory carried out the analyses on Tuesday 28 th or Wednesday 29 th June 2016 with the alternative and reference methods. The analyses by the reference method and the alternative method were performed on the same day. Expedition conditions The collaborative study instructions were sent on 10 th June Experimental parameters controls Sample stability Contamination levels In order to detect the presence of Listeria monocytogenes, the reference method was performed on five portions (25 g) before the inoculation. All the results were negative. The contamination levels and the confidence intervals were (See Table 22): ADRIA Développement 24/66 November 14, 2016 MDA 2 - Listeria monocytogenes

25 Table 22 Level Level 0 Low level High level Samples True level Theoretical Low limit / High limit / (CFU/25 g target level CFU/25 g sample CFU/25 g sample sample) 0 / / / Strain stability during transport Three samples inoculated at a high level (100 CFU/g) were tested for enumeration after 24 h and 48 h storage. Three samples inoculated at a low level were tested for detection after 24 h and 48 h storage (See Table 23). Table 23 Listeria monocytogenes stability in the matrix Day of analysis Day 0 Day 1 Day 2 Listeria monocytogenes detection CFU/g Detection/25 g Mesophilic aerobic flora (CFU/g) No evolution was observed during storage for 48 h at 2-8 C. ADRIA Développement 25/66 November 14, 2016 MDA 2 - Listeria monocytogenes

26 4.2.2 Logistic conditions The temperatures measured at reception by the Labs, the temperatures registered by the thermo-probe, and the receipt dates are given in Table 24. Collaborators Table 24 - Sample temperatures at receipt Temperature measured at receipt ( C) Temperature measured by the probe ( C) Receipt date and time Analysis date A /06/ h30 28/06/2016 B /06/ h30 28/06/2016 C /06/ h30 28/06/2016 D /06/ h00 28/06/2016 E /06/ h50 28/06/2016 F /06/ h50 28/06/2016 G /06/ h00 28/06/2016 H /06/ h22 29/06/2016 I /06/ h10 29/06/2016 J /06/ h40 28/06/2016 K /06/ h10 28/06/2016 K /06/ h10 28/06/2016 L /06/ h00 29/06/2016 L /06/ h00 29/06/2016 N /06/ h50 28/06/2016 All the samples were delivered on time and in appropriated conditions. Temperatures during shipment and at receipt were all correct. Note that Lab K measured a temperature of 10.7 C at receipt while the temperature measured with the probe indicated clearly a temperature of 5.0 C. ADRIA Développement 26/66 November 14, 2016 MDA 2 - Listeria monocytogenes

27 4.3 Calculation and summary of data The raw data are given in Appendix Results obtained by the expert Lab. The results obtained by the expert Lab. are the following (See table 4). Table 25 Results obtained by the expert Lab. Level Reference method Alternative method L0 0/8 0/8 L1 8/8 7/8 L2 8/8 8/8 For Level 1, one sample (M4) gave a negative MDA test while the confirmatory tests concluded to the presence of Listeria monocytogenes in the enrichment broth. The lysate was tested twice again and one positive and one negative results were observed. A second lysate was also tested and a positive MDA result was obtained Results obtained by the collaborators Samples were sent to 15 collaborators. Mesophilic aerobic flora The enumeration of the mesophilic aerobic flora varies from CFU/g, which is a high level of background microflora. to Reference method The positive results by the reference method are provided in Table 26. ADRIA Développement 27/66 November 14, 2016 MDA 2 - Listeria monocytogenes

28 Table 26 - Positive results by the reference method (ALL the collaborators) Collaborators Contamination level L0 L1 L2 A B C D E F G H I J K K L L N Total P 0 = 1 P 1 = 115 P 2 = 120 Alternative method Collaborators The positive results of the alternative method (before and after confirmation) are given in table 27. Table 27 - Positive results (before and after confirmation) by the alternative Before confirmation method (ALL the collaborators) Contamination level L0 L1 L2 Confirmation After confirmation Before confirmation Confirmation After confirmation Before confirmation Confirmation After confirmation A B C D E F G H I J K K L L N Total P 0 = 2 C = 7 CP 0 = 0 P 1 = 109 C = 109 CP 1 = 109 P 2 = 120 C = 120 CP 2 = 120 ADRIA Développement 28/66 November 14, 2016 MDA 2 - Listeria monocytogenes

29 The following unexpected results were observed (See Appendix 7): - Collaborator F detected the presence of Listeria monocytogenes in the enrichment broth (O&A plates) of the alternative method for 4 unspiked samples (F14, F16, F18 and F20) (See Appendix 7). - Collaborator K2 found a positive result with the reference method for a control sample (K26). - Collaborator L obtained a positive MDA test for a control sample (L23); the analysis was repeated twice and negative results were then observed. - Collaborator N detected the presence of Listeria monocytogenes in the enrichment broth of the alternative method for 3 unspiked samples (N16, N18 and N23) (See Appendix 7) A positive MDA test was obtained for a control sample (N14) but the presence of Listeria monocytogenes was not confirmed in the enrichment broth. The Collaborator proceeded to a new extraction on this sample and a negative result was then observed. For sample N21 (Level 1), a negative MDA test was observed while the presence of Listeria monocytogenes was confirmed on O&A plates only. It was asked to the Collaborators to send the strains isolated from unspiked samples. The expert laboratory received only the plates from Collaborator N. A fingerprinting using the Sma I restriction enzyme was run on the strains in order to compare the profiles obtained with the profile of the inoculated strain. The following profiles were obtained (See Figure 1). Inoculated strain N23 N21 N18 N16 Figure 1 These results show clearly that the strains isolated from the unspiked samples were the same strains as that inoculated for Levels 1 and 2. ADRIA Développement 29/66 November 14, 2016 MDA 2 - Listeria monocytogenes

30 Two Collaborators (F and N) were not retained for interpretation due to cross contamination at the level L0. According to the AFNOR technical rules, it is possible to keep collaborators with one unspiked sample providing a positive result per tested method. This was applied in this study for Collaborators K2 and L1. Finally, the results obtained by 13 Collaborators were kept for interpretation; they are provided in Tables 28 and 29. Table 28 - Positive results by the reference method (Without Collaborators F and N) Collaborators Contamination level L0 L1 L2 A B C D E G H I J K K L L Total P 0 = 1 P 1 = 99 P 2 = 104 ADRIA Développement 30/66 November 14, 2016 MDA 2 - Listeria monocytogenes

31 Table 29 - Positive results (before and after confirmation) by the alternative method (Without Collaborators F and N) Collaborators Before confirmation Contamination level L0 L1 L2 Confirmation After confirmation Before confirmation Confirmation After confirmation Before confirmation Confirmation After confirmation A B C D E G H I J K K L L Total P 0 = 1 C =0 CP 0 = 0 P 1 = 94 C = 94 CP 1 = 94 P 2 = 104 C = 104 CP 2 = 104 The percentage specificities (SP) of the reference method and of the alternative method, using the data after confirmation, based on the results of level L0 are the following: Specificity for the reference method: Specificity for the alternative method: Taking into account the results obtained for Level 1 (level for which fractional recoveries was observed), the summary is the following (See table 309). Table 30 - Summary of the obtained results with the reference method and the alternative method for Level 1 Response Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 90 Negative deviation (A-/R+) ND = 9 Reference method negative (R-) Positive deviation (R-/A+) PD = 4 Negative agreement (A-/R-) NA = 1 ADRIA Développement 31/66 November 14, 2016 MDA 2 - Listeria monocytogenes

32 Based on the data summarised in Table 30, the values of sensitivity of the alternative and reference methods, as well as the relative trueness and false positive ratio for the alternative method taking account the confirmations, are the following (See Table 31): Table 31 Sensitivity for the alternative method: Sensitivity for the reference method: SE alt = 91.3 % SE ref = 96.1 % Relative trueness AC RT = 87.5 % False positive ratio for the alternative method FPR = 0 % 4.4 Interpretation For Level 1, the negative deviations are listed in Table 32 and the positive deviations in Table 33. Table 32 - Negative deviations Collaborator Sample No MDA 2 test Confirmation B B B B G G9 - - G K3 - - K K K L L Note that for all the samples, the presence of Listeria monocytogenes was not confirmed in the enrichment broths of the alternative method. Table 33 - Positive deviations Collaborator C D G J Sample No C19 D21 G19 J19 ADRIA Développement 32/66 November 14, 2016 MDA 2 - Listeria monocytogenes

33 For an unpaired study, the difference between (ND PD) is calculated for the level(s) where fractional recovery is obtained (so L 1 and possibly L 2 ). The observed value found for (ND PD) shall not be higher than the AL. The AL is defined as [(ND PD) max ] and calculated per level where fractional recovery is obtained as described below using the following three parameters: P p x ref N where P x = N x = where number of samples with a positive result obtained with the reference method at level x (L 1 or L 2 ) for all the laboratories number of samples tested at level x (L 1 or L 2 ) with the reference method by all the laboratories. CP x p x alt CP x = number of samples with a confirmed positive result obtained with the N x = where alternative method at level x (L 1 or L 2 ) for all the laboratories; N number of samples tested at level x (L 1 or L 2 ) with the alternative method by all the laboratories. ND-PD 3N p p 2 p p max x ref alt ref alt N x = the total number of samples tested for level x (L 1 or L 2 ) with the reference method by all the laboratories. x The AL is not met when the observed value is higher than the AL. In this study, fractional recovery was observed at Level 1. According to the EN ISO :2016, the calculations are the following (See table 34). Table 34 NX 104 (p+)ref 1.0 (p+)alt 0.9 AL = (ND - PD) max 6.49 ND - PD 5 Conclusion ND - PD < AL ADRIA Développement 33/66 November 14, 2016 MDA 2 - Listeria monocytogenes

34 The EN ISO :2016 requirements are fulfilled as (ND - PD) is lower than the AL for Level 1. The RLOD calculations were performed using Excel spreadsheet of the international standard (ISO 16140), as described in the EN ISO :2016 standard ( The results are used only for information (see table 35). Table 35 RLOD RLODL RLODV b=ln(rlod) sd(b) z Test statistic p.value CONCLUSION The method comparison study conclusions are: The method comparison study scheme corresponds to an UNPAIRED STUDY design as the alternative and reference methods do have split enrichment procedures. In the sensitivity study, 6 categories were tested: 5 food categories and the environmental samples. The protocol of the alternative method shows 35 positive deviations (PD) and 33 negative deviations (ND) for the overall categories. The values for ((ND+ PPND) - PD) are below the acceptability limits (AL) whatever the categories, and as well for the 6 tested categories. The Relative Levels of Detection (RLOD) are all below the AL fixed at 2.5 for the unpaired data study whatever the matrix/strain pairs. The inclusivity and exclusivity testing did give the expected results for the 50 target strains and the 30 non target strains. It is possible to store the primary enrichment broth for all the categories and the lysates for 72 h at 2-8 C for all the categories, except dairy products and environmental samples. ADRIA Développement 34/66 November 14, 2016 MDA 2 - Listeria monocytogenes

35 The alternative method allows a one-day screening of the negative samples for the general protocol, and two-days for the specific protocol. The alternative method fulfils all the EN ISO :2016 and AFNOR technical rules requirements. The inter-laboratory study conclusions are: The data and interpretations comply with the EN ISO :2016 requirements. The TM Molecular Detection Assay 2 - Listeria monocytogenes is considered equivalent to the ISO standard. ADRIA Développement 35/66 November 14, 2016 MDA 2 - Listeria monocytogenes

36 Appendix 1 Flow diagram of the EN ISO /A1 (2004) method: Horizontal method for the detection and enumeration of Listeria monocytogenes - Part 1: detection of Listeria monocytogenes in foods 25 g or ml of the sample 1/10 dilution in ½ Fraser enrichment broth Incubation 30 C 1 C, 24 h 3 h 0,1 ml ½ Fraser broth in 10 ml Fraser 1 enrichment broth Incubation 37 C 1 C, 48 3h Streaking on Palcam and O&A Streaking on Palcam and OAA Incubation 37 C 1 C, 24 h 3 h Yes Presence of characteristic colonies of Listeria spp. No Confirmation test Incubation h added 37 C 1 C Presence of characteristic colonies of Listeria spp. L. monocytogenes Other than L. monocytogenes Yes Confirmation test No Presence of L. monocytogenes x g or ml Absence of L. monocytogenes x g or ml L. monocytogenes Other than L. monocytogenes Absence of L. monocytogenes x g or ml Presence of L. monocytogenes x g or ml Absence of L. monocytogenes x g or ml Confirmation tests: Gram, Catalase, Haemolysis, Camp Test, API Listeria Gallery ADRIA Développement 36/66 November 14, 2016 MDA 2 - Listeria monocytogenes

37 Appendix 2 Flow diagram of the TM Molecular Detection Assay 2 - Listeria monocytogenes Specific protocol Raw dairy products Raw Seafood and Raw meat products General protocol Food products (excluding raw meats and raw seafood and raw milk based products) Environmental samples 25 g ml Half Fraser Incubation for 22 h ± 2 h at 37 C 25 g ml Half Fraser Incubation for 27 h ± 3 h at 37 C ± 1 C Possibility to store 72 h at 2-8 C Subculture in Fraser broth Incubation for 22 h ± 2 h at 37 C Lysis Amplification and detection by the Listeria monocytogenes kit Possibility to store the lysates for 72 h at 2-8 C except for dairy products and environmental samples Confirmation: - by streaking 100 µl of Half Fraser broth onto Palcam and O&A plates (General protocol) - by streaking 10 µl of Fraser broth onto Palcam and O&A plates (Specific protocol) ADRIA Développement 37/66 November 14, 2016 MDA 2 - Listeria monocytogenes

38 Specific protocol 1 Raw dairy, Raw meat, Raw seafood products 25 g ml Half Fraser (d 1/10) Storage for 72 h at 2-8 C Incubation for 22 h 2 h at 37 C 1 C Lysis Subculture in Fraser broth MDA 2 test Incubation for 22 h 2 h at 37 C 1 C + - Lysis on 10 µl enrichment broth Confirmatory tests: streak 10 µl Fraser onto O&A and Palcam STOP MDA 2 test on 20 µl lysate (Listeria monocytogenes kit) + - Confirmatory tests: streak 10 µl Fraser onto O&A and Palcam * This is done during the validation study in order to confirm that the typical colonies observed on the plates are really Listeria strains. ADRIA Développement 38/66 November 14, 2016 MDA 2 - Listeria monocytogenes

39 General protocol Food products (excluding raw meats and raw seafood and raw milk based products) Environmental samples 25 g ml Half Fraser (d 1/10) Storage for 72 h at 2-8 C Lysis MDA 2 test Storage for 72 h at 2-8 C (except for Dairy products and environmental samples) Incubation for 27 h 3 h at 37 C 1 C Lysis on 20 µl enrichment broth MDA 2 test on 20 µl lysate (Listeria monocytogenes kit) Confirmatory tests: streak 100 µl Half Fraser onto O&A and Palcam STOP Confirmatory tests: streak 100 µl Half Fraser onto O&A and Palcam ADRIA Développement 39/66 November 14, 2016 MDA 2 - Listeria monocytogenes

40 Appendix 3 Relative level of detection study: raw data Matrix : Deli-salad (Piémontaise) Strain : Listeria monocytogenes Ad494 Aerobic mesophilic flora :210 CFU/g N sample Level Reference method: ISO /A1 Alternative method: Contamination level- (cfu/sample)-mpn Number positive MDA 2 - Listeria monocytogenes Half Fraser Fraser samples/total determination Final O&A Palcam O&A Palcam result MDA test Confirmation Final result result 3369 st st / st / st st st st st st st st st H+ + / / st st st st st st st st st st st Low 0,6 4 / st st H+ + / / st st st H+ + / / H+ + / / st st st st H+ + / / H+ + / / High 1,7 H+ + / / + 3/ st st Number positive samples/total 0/5 6/20 1/5 Analyses performed according to the COFRAC accreditation ADRIA Développement 40/66 November 14, 2016 MDA 2 - Listeria monocytogenes