NF VALIDATION Validation of alternative analytical methods Application in food microbiology. Summary report

Transcription

1 ACCREDITATION N PORTEE DISPONIBLE SUR THERMO FISHER SCIENTIFIC 2130 Woodward Austin, TX USA NF VALIDATION Validation of alternative analytical methods Application in food microbiology Summary report EN ISO validation study of the MicroSEQ Salmonella spp method for the detection of Salmonella spp in food and feed samples, primary product samples and meat products Qualitative method This report includes 96 pages, with 16 appendixes. Only copies including the totality of this report are authorised. Competences of the laboratory are certified by COFRAC accreditation for the analyses marked with symbol. Version 0 November 12, 2014 ADRIA DEVELOPPEMENT Creac h Gwen - F QUIMPER Cedex - Tél. (33) Fax (33) adria.developpement@adria.tm.fr - Site web : ASSOCIATION LOI DE N SIRET N EXISTENCE N TVA FR

2 Contents 1 INTRODUCTION Dates of the validation studies Alternative method protocol Reference methods 7 2 INITIAL VALIDATION STUDY (2010) WITH THE PREPSEQ RAPID SPIN PROTOCOL Methods comparison study Practicability Inter-laboratory study organization and results Conclusion 29 3 EXTENSION FOR PRIMARY PRODUCTION SAMPLES WITH THE PREPSEQ RAPID SPIN AND PREPSEQ TM NA EXTRACTION PROTOCOLS (2012) Methods comparison study Practicability Conclusion 43 4 EXTENSION FOR MEAT PRODUCTS WITH THE PREPSEQ TM NUCLEIC ACID EXTRACTION PROTOCOL (2013) Relative accuracy, relative specificity and relative sensitivity Relative detection level Inclusivity / exclusivity Conclusion 49 5 BIBLIOGRAPHY 49 ADRIA Développement 2/96 November 12, 2014

3 Appendix 1 Alternative method for ALL food and feed products (Initial validation 2010) 51 Appendix 2 Alternative method for primary production samples (Extension study 2012) 52 Appendix 3 Alternative method for meat products (Extension study 2013) 54 Appendix 4 NF EN ISO 6579: 2002 reference method: Microbiology of food and animal feeding stuffs Horizontal method for the detection of Salmonella spp. 55 Appendix 5 NF EN ISO 6579/A1 (2007) reference method: Horizontal method for the detection of Salmonella spp. Detection of Salmonella spp. in animal faeces and in environmental samples from the primary production stage 56 Appendix 6 NF U Recherche par l isolement et l identification de tout sérovar ou de sérovar(s) spécifié(s) de salmonelles dans l environnement des productions animales (Detection by streaking of any Salmonella serotypes in primary production samples) 57 Initial validation study (2010) Appendix 7 Relative accuracy: raw data 58 Appendix 8 Inclusivity and exclusivity: raw data 74 Appendix 9 Accordance 77 Appendix 10 Concordance 78 Extension Primary Production Samples (2012) Appendix 11 Relative accuracy: raw data 79 Appendix 12 Inclusivity and exclusivity: raw data 84 Appendix 13 Inclusivity: additional assays by adding sterilized liver in the primary enrichment broth 88 Appendix 14 Inclusivity: additional assays by adding non sterilized poultry faeces in the primary enrichment broth 89 Appendix 15 Inclusivity: additional assays by adding sterilized poultry faeces in the primary enrichment broth 90 Extension Meat Product Sample (2013) Appendix 16 Relative accuracy: raw data 91 ADRIA Développement 3/96 November 12, 2014

4 Quality Assurance documents related to this study can be consulted upon request from THERMO FISHER SCIENTIFIC. The technical protocol and the result interpretation were realized according to the ISO and the AFNOR technical rules. Company : THERMO FISHER SCIENTIFIC 2130 Woodward Austin, TX USA Expert Laboratory : ADRIA Développement ZA Creac h Gwen QUIMPER Cedex - France Studied method : Validation of the MicroSEQ Salmonella spp method for detection of Salmonella spp Validation standard : EN ISO (October 2003): Food microbiology Protocol for the validation of alternative methods Reference methods: - EN ISO 6579 (2002) : Horizontal method for the detection of Salmonella spp. - EN ISO 6579:2002/Amd 1:2007 : Microbiology of food and animal feeding stuffs. Horizontal method for the detection of Salmonella spp.. Detection of Salmonella spp. in animal faeces and in environmental samples from the primary production stage - U47-100: Recherche par l isolement et l identification de tout sérovar ou de sérovar(s) spécifié(s) de salmonelles dans l environnement des productions animales (Detection by streaking of any Salmonella serotypes in primary production samples) Scope: Food and feed products, primary production samples and meat products Certification body: AFNOR Certification Analyses performed according to the COFRAC accreditation ADRIA Développement 4/96 November 12, 2014

5 1 INTRODUCTION 1.1 Dates of the validation studies The MicroSEQ Salmonella method for the detection of Salmonella spp. was validated in September 2010 (certificate number ABI 29/02 09/10). The following extensions were obtained: - Extension for primary production samples (2012), - Extension for meat products (2013). The method was renewed in Alternative method protocol Principle The MicroSEQ technology with TaqMan probes. Salmonella spp. method is based on real time PCR Analysis parameters used by Rapid finder Express Software for the automatic analysis and interpretation of real-time PCR data: Baseline: Auto Threshold for FAM TM detector (Salmonella spp. Assay): 0.5 Threshold for VIC TM detector (IPC): 0.3 Cut Off Value: Samples with FAM Ct value <35.69 are considered positive Protocols PrepSEQ Rapid Spin protocol for ALL food and feed products (see Appendix 1) - Pre-enrichment step in BPW at 37 C ± 1 C for 18 h ± 2 h according to the ISO 6887 standard. - Nucleic Acid Extraction using PrepSEQ Rapid Spin Sample Preparation Kit - Real time PCR detection ADRIA Développement 5/96 November 12, 2014

6 - Confirmation by the protocol described in the reference method, or by performing a second enrichment step in RVS (0.1 ml BPW in 10 ml RVS) for 24 h 3 h at 41,5 C 1 C and streaking onto a selective agar. Two confirmation protocols were tested during the study: - Tests described in the ISO method, - Subcultures in RVS broth, streak on agar and latex tests directly on isolated colonies. PrepSEQ Rapid Spin and PrepSEQ TM Nucleic Acid Extraction (or NA Extraction) protocols for primary production samples (See Appendix 2) - An overnight (18 h ± 2 h) at 37 C pre-enrichment step was performed in TT broth, followed by a sub-culture in BPW (1 ml/9 ml) for 4 h to 6 h at 37 C, - Nucleic acid extraction with the automated PrepSEQ TM NA Extraction- Sample Preparation System, or with the PrepSEQ TM Rapid Spin sample preparation protocol, - Real time PCR detection - Confirmation by running a subculture of the BPW (0.1 ml/10 ml) in RVS, followed by streaking onto or a chromogenic agar. PrepSEQ TM NA Extraction protocol for meat products (See Appendix 3) - Pre-enrichment step in BPW at 37 C ± 1 C for 18 h ± 2 h - Nucleic acid extraction with the PrepSEQ TM Nucleic Acid Extraction Kit on the MagMAX TM Express-96 Sample Preparation System - Real time PCR detection - Confirmation by performing a second enrichment step in RVS (0.1 ml BPW in 10 ml RVS) for 6 h to 24 h 3 h at 41.5 C 1 C, and then streaking onto selective agar. The isolated typical colonies are confirmed by a latex test (OXOID) without a purification step. In order to provide sufficient practicability to the users, all positive BPW enriched cultures used in the accuracy study were stored for 72 hours at 2-8 C and tested again by the alternative test method for all the tested protocols. ADRIA Développement 6/96 November 12, 2014

7 1.3 Reference methods The reference methods correspond to: - The ISO 6579 standard: Horizontal method for the detection of Salmonella spp (See Appendix 4). - The ISO 6579/A1 standard method for the detection of Salmonella spp. in primary product samples for animal faeces and environmental samples from the primary production stage (See Appendix 5). The U standard was also used (See Appendix 6). 2 INITIAL VALIDATION STUDY (2010) WITH THE PREPSEQ RAPID SPIN PROTOCOL 2.1 Methods comparison study Relative accuracy, relative specificity and relative sensitivity Accuracy is the closeness of agreement between a test result and the accepted reference value. Relative specificity is defined as the degree to which a method is affected (or not) by the other components present in a multi-component sample; that is, it is the ability of the method to measure exactly a given analyte, or its amount, within the sample without interference from non-target components such as matrix effect or background noise. Relative sensitivity is defined as the ability of the alternative method to detect two different amounts of analyte measured by the reference method within a given matrix over the whole measurement range; that is, it is the minimal quantity variation (increase of the analyte concentration x) which gives a significant variation of the measured signal (response y). Analysis performed according to the COFRAC accreditation ADRIA Développement 7/96 November 12, 2014

8 Number and nature of samples 358 samples were analyzed. The distribution by food category and type is summarized below: Category Type Positive samples Negative samples Total Poultry Meat Pork products Beef and others Total Dairy products Egg products Seafood and vegetables Feeding stuffs products Milk and fermented milk Cheeses Dessert, milk powder, ice creams Total Egg powders Liquid egg products Egg based products Total Fresh, raw, frozen Cooked products Composite foods Total Raw Dried Heated Total Total Artificial contamination of samples Artificial contaminations were done by spiking and cross-contamination. For sample spiking, strains were stressed using different treatments, and the stress intensity was evaluated by comparing enumeration done onto selective ( plate) and non selective agars (TSYE plate). Globally, 159 positive samples, artificially or naturally contaminated, were analysed during the accuracy study. A total of 138 samples were artificially contaminated, of which 118 gave a positive result. The artificial contaminations were performed with 32 different strains from 20 different serotypes. The contamination levels were distributed as followed: ADRIA Développement 8/96 November 12, 2014

9 6,50% Inoculation level <= 5 CFU/25 g for 65 samples 20,30% 5 < inoculation level <=10 CFU/25 g for 36 samples 10 < inoculation level <=20 CFU/25 g for 28 samples 26,10% 47,10% 20 < inoculation level <= 30 CFU/25 g for 9 samples Overall, 25.8 % of the samples were naturally contaminated Confirmation protocols The positive samples were confirmed by the ISO 6579 method. Further testing was carried out when presumptive deviations were observed between both methods. Several RVS tubes and MKTTn were inoculated and several streakings were performed on different selective media. ADRIA Développement 9/96 November 12, 2014

10 Test results Raw data per category are given in Appendix 7. Table 1 Paired results of the reference and alternative methods Responses Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 154 Negative deviation (A-/R+) ND = 4 Reference method negative (R-) Positive deviation (R-/A+) PD = 1 Negative agreement (A-/R-) NA = 199 (PPNA = 8) PPNA: positive presumptive negative agreement Results per category of sample Table 2 Meat products Responses Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 30 Negative deviation (A-/R+) ND = 0 Reference method negative (R-) Positive deviation (R-/A+) PD = 0 Negative agreement (A-/R-) NA = 58 (PPNA = 1) Table 3 Dairy products Responses Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 31 Negative deviation (A-/R+) ND = 1 Reference method negative (R-) Positive deviation (R-/A+) PD = 1 Negative agreement (A-/R-) NA = 45 (PPNA = 4) ADRIA Développement 10/96 November 12, 2014

11 Table 4 Egg products Responses Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 31 Negative deviation (A-/R+) ND = 0 Reference method negative (R-) Positive deviation (R-/A+) PD = 0 Negative agreement (A-/R-) NA = 30 (PPNA = 1) Table 5 Seafood and vegetable products Responses Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 31 Negative deviation (A-/R+) ND = 1 Reference method negative (R-) Positive deviation (R-/A+) PD = 0 Negative agreement (A-/R-) NA = 34 (PPNA = 1) Table 6 Feedstuffs Responses Alternative method positive (A+) Alternative method negative (A-) Reference method positive (R+) Positive agreement (A+/R+) PA = 31 Negative deviation (A-/R+) ND = 2 Reference method negative (R-) Positive deviation (R-/A+) PD = 0 Negative agreement (A-/R-) NA = 32 (PPNA = 1) Category PA NA ND PD N Table 7 Calculation of relative accuracy (AC), relative sensitivity (SE) and relative specificity (SP) Relative accuracy AC (%) [100x(PA+NA])/N] N+ PA + ND Relative sensitivity SE (%) [100xPA]/N+] N- NA + PD Relative specificity SP (%) [100xNA]/N-] Meat products , , ,0 Dairy products , , ,8 Egg products , , ,0 Seafood and vegetables , , ,0 Feedstuffs , , ,0 All products , , ,5 PA = positive agreement (R+/A+) PD =positive deviation (R-/A+) NA =negative agreement (R-/A-) ND = negative deviation (A-/R+) ADRIA Développement 11/96 November 12, 2014

12 Calculation of relative accuracy (AC), relative sensitivity (SE) and relative specificity (SP) The alternative method percentage values are: - Relative accuracy: AC = Relative specificity: SP = Relative sensitivity: SE = 97.5 Sensitivity of both methods, when the positive deviations of the alternative method are considered, is presented below: - Alternative method = Reference method = Analysis of discordants Four negative deviations were observed: - Sample n 1987: raw milk artificially contaminated with Salmonella Lagos (5.8 CFU/25 ml) contaminated sample. For this sample, the first PCR run gave a negative result; two other PCR tests were done. The RapidFinder Express software concludes to a negative result using the stringent threshold setting. Relatively high Ct values - (36.2) and - (36.1) were obtained when SDS software was applied, suggesting the sample was contaminated at a low level. The observed results are probably due to a sampling heterogeneity enhanced by a low contamination level. - Sample n 2405: vegetable terrine artificially contaminated with Salmonella Typhimurium Ad 1249 at a low level (1.2 CFU/25 g). Three PCR tests were run and gave one negative result and two positive results with late Ct - (37.0) + (35.6) and + (35.1). The observed results are probably due to a sampling heterogeneity enhanced by a low contamination level. ADRIA Développement 12/96 November 12, 2014

13 - Sample n 2410: raw meat for feedstuff, artificially contaminated sample by Salmonella Dublin at a low level (2.2 CFU/25 g). Four PCR tests were done for this sample; one positive result and three negative results were observed with late Ct values - (36.1), - (35.9), - (36.4) and + (35.4). The Ct value did not meet the threshold of RapidFinder Express to be interpreted positive. After 72 h storage at 4 C, a positive PCR result was observed. The observed results are probably due to a sampling heterogeneity enhanced by a low contamination level. - Sample n 2411: raw meat for feedstuff, artificially contaminated sample by Salmonella Infantis at a low level (1 CFU/25 g) Three PCR tests were run and gave three negative results with late Ct values (36.1, 36.9 and 39.1). The Ct value did not meet the threshold of RapidFinder Express to be called positive. The observed results are probably due to a sampling heterogeneity enhanced by a low contamination level. One positive deviation was observed: - Sample n 1985: raw milk artificially contaminated with Salmonella Dublin (11.4 CFU/25 g). The positive PCR result was confirmed after inoculation on MSRV and streaking on. Y = PD + ND = = 5 Y < 6, no statistical test can be applied. The alternative and the reference methods are concluded equivalent Confirmations Two confirmation protocols were applied during this study: - Confirmations by the tests described in the reference method, - Confirmations by subculture in RVS broth, streaking on agar combined with latex testing on characteristic colonies. For four samples (1968, 1978, 1979, 2551), no characteristic colony was observed on (from RVS broth). ADRIA Développement 13/96 November 12, 2014

14 For nine samples (1061, 1238, 1414, 1975, 1976, 1980, 2565, 1965, 1985), the tests described in the reference method did not lead to direct confirmation results: several RVS, MKTTn broths and MSRV plates were then inoculated. The presence of Salmonella was only confirmed for the sample n 1985, using MSRV and streaks on plates BPW storage at 2-8 C The positive samples by the alternative method were stored at 2-8 C for 72 h and analysed a second time. Two result modifications were observed: Samples First alternative method results Results after storage at 4 C 2410 Negative deviation Positive agreement 2405 Negative deviation Positive agreement Relative detection level The relative detection level is the smallest number of culturable micro-organisms that can be detected in the sample in 50% of occasions by the alternative and reference methods Matrices The objective of this study is (i) to determine the target species minimal quantity that can be detected in food matrices, (ii) to compare both method results. Detection limits have been defined by analyzing the different matrix/strain pairs. Four levels were tested. Six replicates of each combination were prepared. The following matrices were tested: - Ground beef inoculated with Salmonella Infantis 128, - Raw milk inoculated with Salmonella Montevideo 510, - Raw egg product contaminated with Salmonella Enteritidis 657, - Mushrooms contaminated with Salmonella Virchow F276, - Dog biscuits contaminated with Salmonella Derby 630. ADRIA Développement 14/96 November 12, 2014

15 Contamination protocol Contaminations and enumerations were realized according to the AFNOR technical rules (protocol for low level inoculations). The contamination levels are presented below: - Level 1: 0 UFC/g or /ml - Level 2: level necessary to obtain 0 to 50% positives, - Level 3: level necessary to obtain 50 to 75% positives, - Level 4: level necessary to obtain 100% positives. The samples were analyzed by both methods, and the background microflora was enumerated Results Detection levels are presented in Table 8. Table 8 Relative detection level results Strain / matrix pairs Relative detection level (CFU / 25 g or 25 ml) according to Spearman-Kärber test 1 Reference method Alternative method Ground beef / Salmonella Infantis [0.1 ; 1.0] 0.4 [0.1 ; 1.0] Raw milk / Salmonella Montevideo [0.1 ; 0.6] 0.2 [0.1 ; 0.6] Raw egg product / Salmonella Enteritidis [0.1 ; 1.4] 0.3 [0.1 ; 1.4] Mushrooms / Salmonella Virchow F [0.1 ; 0.8] 0.3 [0.1 ; 0.8] Dog biscuits / Salmonella Derby [0.1 ; 1.5] 0.4 [0.1 ; 1.5] Conclusion The alternative and the standard methods show similar detection levels, which are comprised between 0.1 and 1.5 log CFU/25 g. 1 "Hitchins A. Proposed Use of a 50 % Limit of Detection Value in Defining Uncertainty Limits in the Validation of Presence-Absence Microbial Detection Methods, Draft 10th December, 2003". ADRIA Développement 15/96 November 12, 2014

16 2.1.3 Inclusivity / exclusivity Inclusivity is the ability of the alternative method to detect the target analyte from a wide range of strains. Exclusivity is the lack of interference from a relevant range of non-target strains of the alternative method Test protocols - Protocol for inclusivity: 54 Salmonella strains were tested. Cultures were performed in BHI medium at 37 C. Dilutions were done in BPW in order to inoculate between 10 to 100 cells/225 ml BPW. The alternative method protocol was then performed (BPW incubation time: 16 h). - Protocol for exclusivity: 33 negative strains were tested. Cultures were performed in BHI, incubated at 37 C. Dilutions were done in order to inoculate 10 5 cell/ml BPW. The alternative method protocol was then performed (BPW incubation time: 20 h) Results Raw data are given in Appendix 8. The 54 target strains showed positive results. The 33 tested non-target strains showed negative results. One Salmonella Bongori (Ad 599) gave a positive PCR result when inoculated at a high level (10 5 cells/225 ml), but gave finally a negative result when inoculated at a low level (6 cells/225 ml). ADRIA Développement 16/96 November 12, 2014

17 2.2 Practicability MicroSEQ Salmonella method practicability was evaluated according to the AFNOR criteria relative to the preliminary study. Mode of packaging of test components The components needed for the analysis are: - Lysis buffer: lysis buffer is available in a pouch. Each pouch contains one flask, - MicroSEQ Salmonella spp. detection kit: the kit contains reagents for 96 reactions, - Pathogen negative control: 1 tube, Volume of reagents The package insert specifies the reagent volumes: - MicroSEQ Salmonella spp detection kit: 96 tests - Pathogen negative control: 1.5 ml (50 tests) Storage conditions of components and shelf-life of unopened products The storage conditions are given on the package: - MicroSEQ Salmonella spp. detection kit: 2 to 8 C, protect from light - Pathogen negative control: 2 to 8 C. The shelf life is given on the package: MicroSEQ Salmonella spp. detection kit: 18 months after manufacturing Negative control: 18 months after manufacturing ADRIA Développement 17/96 November 12, 2014

18 Storage after initial use All the reagents must be stored at 2 8 C after initial use. Specific equipment required - Applied Biosystems 7500 Fast Real-Time PCR system - Consumables: * Rapid Spin sample preparation kit tubes * PrepSEQ Rapid Spin sample preparation kit * Aerosol-resistant pipette tips (200, µl) * Disposable gloves * Pipettors (200, µl) * MicroAmp Fast 8 tube strip 0.1 ml * MicroAmp Optical 8 cap strip - Reagents * Nuclease free water Reagents ready to use or for reconstitution All the reagents are ready to use. Training period for operator with no experience of the method Less than 1 day is required for technicians with microbiology and molecular biology knowledge. ADRIA Développement 18/96 November 12, 2014

19 Handling time and flexibility as a function of number of samples to be analysed (in minutes) Reference method ISO 6579 Alternative method MicroSEQ Salmonella method 24 samples 24 samples Sampling Stomach Extraction 60 PCR 15 RVS and MKTTn sub-cultures 45 Streaking on selective agar plates 75 Reading 30 Total (negative sample) Total / negative sample RVS and MKTTn subcultures 45 (1) 25 (2) Streaking on selective agar plates 75 (1) 20 (2) Reading 30 (1) 15 (2) Strike on nutritive agar Confirmation tests 120 Total (positive sample) Total / positive sample 386 (1) Confirmation by the tests described in the reference method (2) Confirmation by subculture in RVS followed by streaking on plate (1) 15 (2) 461 (1) 266 (2) 19.2 (1) 11.1 (2) For negative sample analysis, MicroSEQ Salmonella method requires less time than the reference method. For positive sample analysis, MicroSEQ Salmonella method requires less time than the reference method (if the short confirmation protocol is used). ADRIA Développement 19/96 November 12, 2014

20 Time required for results - Negative samples ISO 6579 reference method MicroSEQ Salmonella method Pre-enrichment Day 0 Day 0 Enrichment Day 1 Extraction, negative result Day 1 Streak Day 2 Reading Day 3 - Positive samples ISO 6579 reference method MicroSEQ Salmonella method Pre-enrichment Day 0 Day 0 Enrichment Day 1 Day 1 Extraction, PCR Day 1 Streak Day 2 Day 2 Reading Day 3 Day 3 Latex test Day 3 (3) Confirmation tests Day 4 to Day 6 Day 4 to Day 6 (4) (3) Confirmation by the tests described in the reference method (4) Confirmation by subculture in RVS followed by streaking on plate Negative results are available within a day using the MicroSEQ Salmonella method. Moreover, note that it is possible to store the enrichment broth for 72 h at 2 8 C prior analysis. ADRIA Développement 20/96 November 12, 2014 Operator qualifications Technician qualified in microbiology and molecular biology. Stages in common with reference method Pre-enrichment is common to both methods. Traceability of analytical results All data obtained with the Salmonella method are traced over time by computer archive means. Maintenance by laboratory Perform the background calibrations and optical calibration every twelve months.

21 2.3 Inter-laboratory study organization and results Study organization Collaborators number Samples were sent to 15 laboratories. Matrix and strain used The study was carried out with ground beef samples contaminated with Salmonella Typhimurium A00C060. Samples Samples were inoculated and sent on Monday 5 July 2010, as described below: - 24 blind samples for Salmonella detection by the MicroSEQ Salmonella method and by the reference method (ISO 6579:2002), - 1 ground beef sample for the aerobic mesophilic flora enumeration by ISO 4833 method, - 1 water flask labeled Temperature Control with a temperature probe, which records the temperature variations during transportation. The analyses were started on Wednesday 7 July Inoculation The targeted inoculation levels were: - Level 0: 0 CFU/g, - Level 1: 5 CFU/g, - Level 2: 25 CFU/g. Each laboratory received 24 samples of 25 g, i.e. 8 samples per inoculation level and method. ADRIA Développement 21/96 November 12, 2014

22 Labelling and shipping Blinded samples (code is only known by the expert laboratory) were placed in isothermal boxes, which contained cooling blocks, and express-shipped to the different laboratories. A temperature control flask containing a temperature probe was added to the package in order to register the temperature profile during the transport, package delivery. Samples were shipped in 24 h to 48 h to the different laboratories. Sample temperature had to stay lower or equal to 8.4 C during transport, and between 0 C 8.4 C at arrival. Analyses Collaborative study laboratories and the expert laboratory carried out the analyses with the alternative and reference methods at day 2. Expedition conditions The collaborative study instructions were sent on June 21, Experimental parameters control Contamination level before inoculation, and levels obtained after the artificial contaminations of the samples Before inoculation In order to detect Salmonella, the ISO 6579 method was performed on five ground beef samples (25 g) before the samples inoculation. All the results were negative. Sample stability Sample stability was performed by inoculating the matrix at 25 CFU/25 g and 5 CFU/25 g. Enumerations were performed on 2.5 g of ground beef samples for the high contamination level and detection analyses were performed for ADRIA Développement 22/96 November 12, 2014

23 the low contamination level. Triplicates were analyzed, and the results were the following: Reference method (research) CFU/25 g () Aerobic Day Sample Sample Sample Sample Sample Sample mesophilic flora (CFU/g) Day < x10 4 Day x10 4 Day x10 5 No evolution was observed during storage at 4 C. Contamination levels The contamination levels and the confidence intervals were: Level Level 0 Low level High level Samples Theoretical target level (b/25 g) True level (b/25 g sample) Low limit / 25 g sample High limit / 25 g sample 0 / / / ADRIA Développement 23/96 November 12, 2014

24 Logistic conditions Temperature conditions are given below: Laboratories Temperature measured by the temperature probe ( C) Sample temperatures at receipt Temperature measured at receipt ( C) Receipt delay Day of analysis A Day 1 11h00 Day 2 B Day 1 11h45 Day 2 C Day 1 10h30 Day 2 D Day 1 10h45 Day 2 E Day 1 10h40 Day 2 F Day 1 09h10 Day 2 G Day 1 13h45 Day 2 H Day 1 11h45 Day 2 I Day 1 08h00 Day 1 J Day 1 09h45 Day 2 K Day 1 10h30 Day 2 L Day 2 17h15 Day 2 M 5.0 Not measured Day 1 10h30 Day 2 N Day 1 10h55 Day 2 O Day 1 12h00 Day Conclusion No problem was encountered during the transport or at receipt, except for two Labs: - Lab L received his parcel at Day 2 at 25 C, - For Lab N, the temperature measured at receipt was 11.2 C, but the probe indicated 5.0 C. ADRIA Développement 24/96 November 12, 2014

25 2.3.3 Results analysis Aerobic mesophilic flora enumeration Four Labs did not realize/run this analysis. Depending on the Lab results, the enumeration levels varied from 3.7 x10 3 to 2.0 x10 5 CFU/g Expert lab results All the inoculated samples gave a positive result. All the results were in agreement between the reference and the alternative methods. Level Reference method Alternative method L0 0/8 0/8 L1 8/8 8/8 L2 8/8 8/ Collaborator lab results 15 Labs participated in the study; the results of Lab I and L were excluded for the interpretation: - Lab I performed the analyses at Day 1, - Lab L received the samples at 25 C. The results of 13 Labs were finally taken into account for the interpretation. Laboratory Reference method Alternative method L0 L1 L2 L0 L1 L2 A 0/8 8/8 8/8 0/8 8/8 8/8 B 0/8 8/8 8/8 0/8 8/8 8/8 C 0/8 8/8 8/8 0/8 8/8 8/8 D 0/8 8/8 8/8 0/8 8/8 8/8 E 0/8 8/8 8/8 0/8 8/8 8/8 F 0/8 8/8 8/8 0/8 8/8 8/8 G 0/8 8/8 8/8 0/8 8/8 8/8 H 0/8 8/8 8/8 0/8 8/8 8/8 I 0/8 8/8 8/8 0/8 8/8 8/8 J 0/8 8/8 8/8 0/8 8/8 8/8 K 0/8 8/8 8/8 0/8 8/8 8/8 L 0/8 8/8 8/8 0/8 8/8 8/8 M 0/8 8/8 8/8 0/8 8/8 8/8 N 1/8 7/8 8/8 1/8 7/8 8/8 O 0/8 8/8 8/8 0/8 8/8 8/8 ADRIA Développement 25/96 November 12, 2014

26 2.3.4 Results interpretation Lab N found: one non-inoculated sample positive (N21) by both methods and one negative sample (N7) of low inoculation level by both methods. This Lab has repeated the extraction and PCR analyses on these two samples: the results were confirmed. Strain transfer was done and molecular fingerprinting was done with Pulse Field Gel Electrophoresis (PFGE). This analysis clearly confirmed that the samples N21 and N7 were inverted, i.e. the PFGE pattern of the strain from the sample N21 fits with the PFGE pattern of the inoculated strain. Lab E found some characteristic colonies on the selective media from one of the 24 samples they received, but was not able to confirm the Salmonella enterica species identification. The isolates were sent to the expert Lab; the observed characteristic colonies clearly belong to Salmonella enterica species according to the ISO 6579 confirmatory test results Specificity and sensitivity for each method For the L0 level and for each method, specificity percentages are calculated according to: FP SP 1 x 100% N with : N- = total number of all L0 assays FP = number of false positive results For each contamination level and each method, the sensitivity percentages are calculated according to: TP SE x 100% N with : N+ = total number of all L1 or L2 assays TP = number of true positive results Results are reported in the following table: Level Reference method Alternative method SP/SE % LCL% SP/SE % LCL% L0 SP = SP = L1 SE = SE = L2 SE = SE = L1+L2 SE = SE = ADRIA Développement 26/96 November 12, 2014

27 Relative accuracy (AC) Results for all levels are given below: Table 9 - Paired results of the alternative and reference methods Alternative method Reference method + - Total + PA = 208 PD = ND = 0 NA = Total N+ = 208 N- = 104 N = 312 Relative accuracy (AC) (in %) is calculated according to: ( PA NA) AC x 100% N with : N = number of samples analysed PA = number of positive agreement NA = number of negative agreement The alternative method accuracy values with regard to the reference method are: Table 10 Level AC % LCL % L L L L1 + L Total Discordant results No discordant result was observed between the reference and the alternative methods. ADRIA Développement 27/96 November 12, 2014

28 2.3.5 Interpretation Comparison of the relative accuracy, specificity and sensibility values The values obtained for the two parts of the validation study (comparative and inter-laboratory studies) are reported in table 11. Table 11 - Alternative method values calculated during the comparative and inter-laboratory studies Inter-laboratory study Method comparison study Relative accuracy (AC) Sensibility (SE) Specificity (SP) Accordance (DA) The raw data are given in Appendix 9. Accordance values for both methods are: Level Reference method (DA) Alternative method (DA) L L L Concordance The raw data are given in Appendix 10. Both methods concordance values are: Level Reference method Alternative method L L L ADRIA Développement 28/96 November 12, 2014

29 Odds Ratio (COR) The odds ratio value is determined according to: COR Accordance x 100 agreement Agreement x(100 accordance ) Both method odds ratio values are: Level Reference method (COR) Alternative method (COR) L L L Conclusion The methods comparative study conclusions are: The MicroSEQ Salmonella method shows satisfying relative accuracy, specificity and sensitivity. The detection limits of the MicroSEQ Salmonella method and ISO 6579 method are similar. MicroSEQ Salmonella method is selective and specific. Negative results are available within a day using the MicroSEQ Salmonella method. The inter-laboratory study conclusions are: The alternative method and reference method show equivalent performances (accordance, concordance, odds ratio). ADRIA Développement 29/96 November 12, 2014

30 3 EXTENSION FOR PRIMARY PRODUCTION SAMPLES WITH THE PREPSEQ RAPID SPIN AND PREPSEQ TM NA EXTRACTION PROTOCOLS (2012) 3.1 Methods comparison study Relative accuracy, relative specificity and relative sensitivity Number and nature of samples 126 samples were analyzed with the manual extraction protocol (Rapid Spin) and 107 by the automated extraction protocol (NA Extraction). The distribution by type is summarized below: Types Rapid Spin protocol NA Extraction protocol + - Total + - Total Poultry faeces samples Pig faeces samples Non poultry faeces samples Non pig faeces samples TOTAL Artificial contamination of samples Some samples were inoculated with Salmonella strains and were then stored for 24 h at room temperature before analysis. Rapid Spin protocol Globally, 67 positive samples, artificially or naturally contaminated, were analyzed during the accuracy study. A total of 52 samples were artificially contaminated, of which 47 gave a positive result. Overall, 29.9 % of the samples were naturally contaminated. NA Extraction protocol Globally, 48 positive samples were analyzed during the accuracy study. 33 samples were artificially contaminated; 28 gave a positive result. Overall, 41.7 % of the samples were naturally contaminated. ADRIA Développement 30/96 November 12, 2014

31 Confirmation protocols During the validation study, the positive samples were confirmed by a subculture of BPW into RVS broth (0.1 ml + 10 ml) incubated at 24 h ± 3 h at 41.5 C ± 1 C, followed by streaking onto and IRIS Salmonella agar plates. Typical colonies were then confirmed by a latex test and the confirmatory tests of the standard ISO 6579 method Test results These results are shown for each method using an unpaired study design. Raw data per category are given in Appendix 11. Identical results were observed with the two reference methods (ISO 6579/A1 and U47-100). Table 12 Paired results of the reference and alternative methods All samples Responses Alternative method positive (A+) Alternative method negative (A-) Manual extraction protocol (Rapid Spin) Reference method positive (R+) Positive agreement PA = 47 (A+/R+) Negative deviation ND = 11 (A-/R+) Reference method negative (R-) Positive deviation PD = 9 (R-/A+) Negative agreement (A-/R-) NA = 59 (PPNA = 1) Automated extraction protocol (NA Extraction) Reference method positive (R+) Positive agreement PA = 30 (A+/R+) Negative deviation ND = 10 (A-/R+) Reference method negative (R-) Positive deviation PD = 8 (R-/A+) Negative agreement (A-/R-) NA = 59 (PPNA = 2) PPNA: positive presumptive negative agreement ADRIA Développement 31/96 November 12, 2014

32 Results per category of sample Table 13 Poultry faeces samples Responses Alternative method positive (A+) Alternative method negative (A-) Manual extraction protocol (Rapid Spin) Reference method positive (R+) Positive agreement PA = 16 (A+/R+) Negative deviation ND = 6 (A-/R+) Reference method negative (R-) Positive deviation PD = 5 (R-/A+) Negative agreement NA = 20 (A-/R-) Automated extraction protocol (NA Extraction) Reference method positive (R+) Positive agreement PA = 6 (A+/R+) Negative deviation ND = 5 (A-/R+) Reference method negative (R-) Positive deviation PD = 5 (R-/A+) Negative agreement (A-/R-) NA = 20 (PPNA = 2) Table 14 Pig faeces samples Responses Alternative method positive (A+) Alternative method negative (A-) Manual extraction protocol (Rapid Spin) Reference method positive (R+) Positive agreement PA = 14 (A+/R+) Negative deviation ND = 1 (A-/R+) Reference method negative (R-) Positive deviation PD = 3 (R-/A+) Negative agreement NA = 9 (A-/R-) Automated extraction protocol (NA Extraction) Reference method positive (R+) Positive agreement PA = 9 (A+/R+) Negative deviation ND = 1 (A-/R+) Reference method negative (R-) Positive deviation PD = 2 (R-/A+) Negative agreement NA = 10 (A-/R-) Table 15 Non poultry faeces samples Responses Alternative method positive (A+) Alternative method negative (A-) Manual extraction protocol (Rapid Spin) Reference method positive (R+) Positive agreement PA = 7 (A+/R+) Negative deviation ND = 3 (A-/R+) Reference method negative (R-) Positive deviation PD = 1 (R-/A+) Negative agreement (A-/R-) NA = 10 (PPNA = 1) Automated extraction protocol (NA Extraction) Reference method positive (R+) Positive agreement PA = 6 (A+/R+) Negative deviation ND = 3 (A-/R+) Reference method negative (R-) Positive deviation PD = 1 (R-/A+) Negative agreement NA = 10 (A-/R-) ADRIA Développement 32/96 November 12, 2014

33 Table 16 Non pig faeces samples Responses Alternative method positive (A+) Alternative method negative (A-) Manual extraction protocol (Rapid Spin) Reference method positive (R+) Positive agreement PA = 10 (A+/R+) Negative deviation ND = 1 (A-/R+) Reference method negative (R-) Positive deviation PD = 0 (R-/A+) Negative agreement NA = 20 (A-/R-) Automated extraction protocol (NA Extraction) Reference method positive (R+) Positive agreement PA = 9 (A+/R+) Negative deviation ND = 1 (A-/R+) Reference method negative (R-) Positive deviation PD = 0 (R-/A+) Negative agreement NA = 19 (A-/R-) Table 17 Calculation of relative accuracy (AC), relative sensitivity (SE) and relative specificity (SP) Rapid Spin protocol Types PA NA ND PD N Relative accuracy AC (%) [100x(PA+NA])/N] N+ PA + ND Relative sensitivity SE (%) [100xPA]/N+] N- NA + PD Relative specificity SP (%) [100xNA]/N-] Poultry faeces samples Pig faeces samples Non poultry faeces samples Non pig faeces samples Total NA Extraction protocol Types PA NA ND PD N Relative accuracy AC (%) [100x(PA+NA])/N] N+ PA + ND Relative sensitivity SE (%) [100xPA]/N+] N- NA + PD Relative specificity SP (%) [100xNA]/N-] Poultry faeces samples Pig faeces samples Non poultry faeces samples Non pig faeces samples Total PA = positive agreement (R+/A+) PD =positive deviation (R-/A+) NA =negative agreement (R-/A-) ND = negative deviation (A-/R+) ADRIA Développement 33/96 November 12, 2014

34 Calculation of relative accuracy (AC), relative sensitivity (SE) and relative specificity (SP) The alternative method percentage values are: Manual extraction protocol (Rapid Spin) Automated extraction protocol (NA Extraction) Relative accuracy (AC) Relative specificity (SP) Relative sensitivity (SE) Sensitivity of both methods, when the positive deviations of the alternative method are considered, is presented below: Manual extraction protocol (Rapid Spin) Automated extraction protocol (NA Extraction) Alternative method Reference method Analysis of discordants Negative deviations The discordant samples are the following: PCR result Sample Product Contamination NA Confirmation n Rapid Spin Extraction 6551 Boot socks (piggery) Salmonella Livingstone Ad 1107 (6.4) - / Poultry drinker Salmonella Typhimurium Ad 335 (8.4) Boot socks (poultry) Salmonella Agona Ad 1306 (8.6) Poultry litter Salmonella Hadar 35 (12.0) Boot socks (pork floors) Salmonella Typhimurium ST Boot socks (poultry) Natural Boot socks (poultry) Natural Boot socks (poultry) Natural Boot socks (poultry) Natural Hen drinker water Salmonella Derby Ad 1500 (6.4) Sponge (wall-porks) Salmonella Typhimurium Ad 1249 (7.0) The presence of Salmonella was never confirmed in the negative deviation samples; these observed negative results were probably due to the sampling ADRIA Développement 34/96 November 12, 2014

35 (the enrichment step for the alternative method used tetrathionate broth and therefore this represented an unpaired study) and not to false negative results. Positive deviations The positive deviations are the following: Sample n Product Contamination (contamination level CFU/sample) PCR result Rapid Spin Extraction 243 Slaughterhouse poultry faeces Salmonella Hadar 35 (12.0) Hens water Salmonella Typhimurium Ad 1335 (8.4) NA Pork faeces Salmonella Derby Ad 1500 (3.8) Bootsocks (poultry) Naturally contaminated sample Bootsocks (poultry) Naturally contaminated sample Bootsocks (poultry) Naturally contaminated sample Bootsocks (poultry) Naturally contaminated sample Pork faeces Salmonella Typhimurium ST 394 (5.6) Pork faeces Salmonella Typhimurium ST 394 (5.6) + + According to Annex F of the ISO standard, the alternative method and the reference method are not different, regardless of the DNA extraction protocol used prior to PCR analysis: Manual extraction protocol (Rapid Spin) Automated extraction protocol (NA Extraction) Y = ND + PD Y = = 20 Y = = 18 m 9 8 M 5 4 Conclusion m > M m > M The alternative and the reference methods are concluded equivalent. m: the lowest value between ND and PD M : value given in the Annex F (ISO 16140) ADRIA Développement 35/96 November 12, 2014

36 Confirmations Rapid Spin protocol All the PCR positive results were confirmed positive by streaking BPW subculture onto plates, except for four samples (n 243, 244, 250 and 1050). The presence of Salmonella was confirmed positive on IRIS plates for samples n 243 and 244. For sample n 1050, the presence of Salmonella was confirmed positive by MSRV inoculation. It was not possible to confirm the PCR positive results of sample n 250 by confirmatory tests in which several subcultures were run in RVS, MKTTn and MSRV before streaking into selective agars. NA Extraction protocol Two PCR positive samples (n 1016 and 1018) were not confirmed by any of the confirmatory tests used BPW broth storage 72 h at 2-8 C Only one modification was noticed during the study for sample n 6542 (poultry litter artificially contaminated with Salmonella Anatum Ad 1108) (manual extraction protocol) which was positive (PA) for the first analysis and became negative (ND) after storage of the BPW broth 72 h at 2 8 C. The analysis of discordant for the Rapid Spin extraction protocol: Y = ND + PD Y = = 21 m 9 M 5 Conclusion m > M The two methods are not different at < ADRIA Développement 36/96 November 12, 2014

37 3.1.2 Relative detection level Matrices The objective of this study is (i) to determine the target species minimal quantity that can be detected in food matrices, (ii) to compare both method results. Detection limits were defined by analysing a matrix/strain pair: poultry faeces, inoculated by Salmonella Typhimurium Ad Four levels were tested. Six replicates of each combination were prepared Contamination protocol Contaminations and enumerations were performed according to the AFNOR technical rules (protocol for low level inoculations). The targeted contamination levels are presented below: - Level 1: 0 CFU/g or /ml - Level 2: level necessary to obtain 0 to 50% positives, - Level 3: level necessary to obtain 50 to 75% positives, - Level 4: level necessary to obtain 100% positives. The samples were analyzed by both methods, and the background microflora was enumerated Results Detection levels are presented in Table 18. Table 18 Relative detection level results Strain / matrix pairs Poultry faeces / Salmonella Typhimurium Ad 1411 Relative detection level (CFU / 25 g) according to Spearman-Kärber test Manual extraction protocol (Rapid Spin) Automated extraction protocol (NA Extraction) Reference method Alternative method Reference method Alternative method 0.5 [0.3 ; 1.0] 0.7 [0.3 ; 1.3] 0.5 [0.3 ; 1.0] 0.7 [0.3 ; 1.3] Conclusion The alternative and the standard methods show similar detection levels, which are comprised between 0.3 and 1.0 CFU/25 g for the reference method and between 0.3 and 1.3 CFU/25 g for the alternative method. ADRIA Développement 37/96 November 12, 2014

38 3.1.3 Inclusivity Exclusivity Test protocols Inclusivity Salmonella strain cultures were performed in BHI medium at 37 C. Dilutions were done in order to inoculate 10 cells/225 ml in tetrathionate broth (225 ml + 25 ml sterile water). The alternative method was then performed (BPW incubation time: 16 h). Exclusivity Negative strains were tested. Cultures were performed in BHI, incubated at 37 C. Dilutions were realized in order to inoculate 10 5 cells/ml BPW. The alternative method protocol was then performed (BPW incubation time: 20 h) Results Raw data are given in Appendix 12. Inclusivity Among the 50 strains tested, 43 gave a PCR positive result with the two extraction protocols when the tetrathionate broth was inoculated at a level comprised between 1 and 100 CFU/225 ml. For 4 strains (Salmonella diarizonae 451, Salmonella Napoli Ad 928, Salmonella Rissen 39 and the non mobile variant Salmonella Typhimurium Ad 1333), positive PCR results were observed at a level comprised between 100 and 550 CFU/225 ml. Note that Salmonella diarizonae 451 is detected by the Rapid Spin protocol only. For 3 strains (Salmonella arizonae CIP 5523, Salmonella Gallinarum Ad 300 and Salmonella Paratyphi A ATCC 9150), negative PCR results were observed whatever the inoculation level. These strains are detected by the tested method when a single enrichment in BPW is done, even at an inoculation level lower than 100 CFU/225 ml: - Salmonella diarizonae 451, - Salmonella Napoli Ad 928, ADRIA Développement 38/96 November 12, 2014

39 - Salmonella Rissen 39, - Salmonella arizonae CIP 5523, - Salmonella gallinarum Ad 300, - Salmonella Paratyphi A ATCC Note that the Salmonella Typhimiurim Ad 1333 strain was not tested during the initial validation study. The observed results are thus due to the sub-culture in TT broth. Additional assays were done by adding sterilized liver, non sterilized and sterilized faecal samples in the primary enrichment broth of the alternative and reference methods. This is supposed to decrease the selectivity of the enrichment broths. The results are presented Appendixes 12, 13 and 14. Note that the strains tested by adding animal faeces in the primary preenrichment broth were selected by one referee of the study. When adding sterilized liver, all the results are positive with the alternative method, except for the strain Salmonella arizonae CIP Note that the test kit was able to detect the 2 tested Salmonella Gallinarum strains, as well as the Salmonella arizonae CIP 5526, Salmonella diarizonae Ad 1280, Salmonella diarizonae 4851 and Salmonella Paratyphi A ATCC strains, while it was not the case for the ISO 6579 reference method. When adding non sterilized faecal samples, the test kit was only able to detect the Salmonella Gallinarum 1 strain, but it was not possible to confirm the PCR positive result. The ISO 6579 reference method was only able to detect the Salmonella arizonae CIP 5522 strain. At least, the addition of sterilized poultry faecal samples confirms the results observed by culturing in presence of sterilized liver. The test kit is then able to detect all the tested strains, except the S. diarizonae Ad 1301 strain. When using the ISO 6579 reference method, negative results are observed with the 2 tested S. Paratyphi strains, as well as the 2 S. Gallinarum strains. The U reference method is not able to detect the S. Gallinarum Ad 300 strain. These data clearly confirm that S. arizonae, S. diarizonae, S. Paratyphi A and S. Gallinarum are detected by the alternative method, when non faecal or faecal sample types are added in the primary enrichment broth. ADRIA Développement 39/96 November 12, 2014