PSI Report Salmonella Testing - Algorithm Tuning - by J. J. Lehett, Director of Engineering & P. J. Simpson, V. P. of Technology Development

Size: px

Start display at page:

Download "PSI Report Salmonella Testing - Algorithm Tuning - by J. J. Lehett, Director of Engineering & P. J. Simpson, V. P. of Technology Development"

Juliet Hoover
5 years ago
Views:

1 PSI Report Salmonella Testing - Algorithm Tuning - by J. J. Lehett, Director of Engineering & P. J. Simpson, V. P. of Technology Development

2 Executive Summary: RV Results: 64 total samples run; 44 positives and 20 negatives by BAM method. Zephyr system showed 44 true positives, 19 true negatives and 1 false positive. These results correlate with: 100% sensitivity 95% specificity Positive Predictive Value (PVV) of 98% Negative Predictive Value (NPV) of 100% TT Results: 64 total samples; 45 positives and 19 negatives by BAM method. Zephyr system showed 45 positives, 17 true negatives and 2 false positives. These results correlate with: 100% sensitivity 89% specificity Positive Predictive Value (PVV) of 96% Negative Predictive Value (NPV) of 100% False Negative Samples: None False Positive Samples: RV1; TT 2 Both RV and TT broths gave a strong positive signal for sample 10 run on 12/19/14. The peak light output for this sample was above 5000 RLU when enriched in RV media and above 1000 RLU when enriched in TT broth. For TT broth only, sample TT18 run on 12/19/14 also gave a positive result. The peak light output was only within the range, but the shape of the light output curve produced over time was very similar to known positive samples. Controls: All control samples produced the correct results on the Zephyr system. Differences in Growth Media: In part due to the different levels of signal that were produced from fresh and frozen samples in the two different enrichment medias, two sets of algorithm coefficients were produced, one to be used for samples enriched in RV media and the other for use in samples enriched in TT broth. Differences in Fresh vs Frozen Samples: Light output peak height was reduced in samples enriched in TT broth and then frozen. This was not seen for samples grown in the RV media. For that reason two separate sets of algorithm coefficients were developed, one for fresh and frozen samples in RV media and one for fresh and frozen samples in TT broth. It is possible that results from the TT samples could be further

3 improved by the use of separate algorithm coefficient sets for fresh versus frozen samples. Project Goal: To determine the ability of PSI's Zephyr system to predict the presence or absence of Salmonella in spiked lettuce samples. Introduction: PathSensors, Inc. (PSI) develops and produces assay systems for the detection of various pathogens including bacteria, viruses and toxins. The technology platform (termed CANARY ) is a cell-based biosensor that was originally described by MIT-LL in the periodical Science. It utilizes a cell that has been genetically modified to express target specific antibodies on its cell surface coupled with a luminescent protein, such that when the cell encounters its target pathogen the cell emits light. The platform is extremely rapid and sensitive, allowing the detection of low levels of target in ~5 minutes. The CANARY biosensor can be used to detect pathogens derived from liquid, solid or air samples. PSI's Zephyr assay system is designed for the detection of pathogens in liquid samples, including growth media. The Zephyr system provides all of the instrumentation to process liquid samples as well as a software system that guides the user through all steps of the assay process. A detection algorithm is used to provide a Positive or Negative result for each sample at the end of the assay. The detection algorithm is a mathematical formula that describes the characteristics of light output from the CANARY biosensor over time upon interaction with its pathogenic target. Although all biosensors generate a similar type of signal in response to target interaction, their light output characteristics can be modified by the specific target and sample matrix being tested. A series of 23 coefficient filters are embedded in the CANARY -Zephyr detection algorithm. Adjustment of these filters for a given application is necessary for optimal generation of Positive and Negative results. Adjustment of these filters to optimize the results for a given application is termed "tuning the algorithm" for that application. Below we provide results for the detection of Salmonella from lettuce samples using the Zephyr system and a tuned detection algorithm for that application. Samples and Test Data: Samples were prepared and tested using the CANARY -Zephyr detection system. The Zephyr system was provided with a generic Salmonella detection algorithm that had been developed at PSI using analytical Salmonella samples that did not contain any sample matrix. The test sample set prepared included Negative and Positive Controls as well as blind samples of Salmonella spiked into lettuce leaves. The samples were enriched according the FDA BAM method for detecting Salmonella in leafy greens, including lettuce. This included enrichment in one of two types of selective broth, Rappaport-Vassiliadis medium (RV) or tetrathionate (TT) broth. After enrichment the samples were tested either freshly prepared or after freezing. In all cases, 100 L of sample was mixed with 900 L of PSI Assay Buffer in a Zephyr System Assay Tube. The sample was then centrifuged for 2-minutes at 10,000 x g and the supernatant removed with a pipettor and discarded. Two-hundred and fifty L of fresh assay

4 buffer was then added to the sample and it was centrifuged again for 2-minutes at 10,000 x g. Twently L of biosensor reagent was then placed in the cap of the Assay Tube, the sample was centrifuged for 5-seconds and then transferred to the Zephyr system luminometer where light output was recorded for 1-minute. The resulting data was automatically recorded in the Zephyr System back-up file. This data was supplied to PSI where the mathematical detection algorithm was "tuned" to provide positive and negative assay results in the appropriate broth matrices. As there were some differences in light output curves depending on the enrichment broth used to enrich the samples and whether the samples were fresh or frozen, two different detection algorithm coefficient sets were developed, one for use with RV samples and one for use with TT samples. Results: Test Set 1: Blind samples enriched in RV medium Coefficient set: RV-Broth-Test-02 Zephyr Algorithm Detection Results (Blind Samples) Sample ID Date Tested Enrichment Medium BAM Results Zephyr Results R1 12/19/14 RV N N R2 12/19/14 RV P P R3 12/19/14 RV P P R4 12/19/14 RV P P R5 12/19/14 RV P P R6 12/19/14 RV N N R7 12/19/14 RV N N R8 12/19/14 RV P P R9 12/19/14 RV N N R10 12/19/14 RV N P R11 12/19/14 RV P P R12 12/19/14 RV P P R13 12/19/14 RV P P R14 12/19/14 RV P P R15 12/19/14 RV P P R16 12/19/14 RV P P R17 12/19/14 RV P P R18 12/19/14 RV N N R19 12/19/14 RV P P R20 12/19/14 RV P P R31 12/11/14 RV P P R32 12/11/14 RV N N R33 12/11/14 RV P P

5 R34 12/11/14 RV P P R35 12/11/14 RV P P R36 12/11/14 RV P P R37 12/11/14 RV N N R37 12/11/14 RV P P R39 12/11/14 RV P P R40 12/11/14 RV N N R1 12/31/14 RV P P R2 12/31/14 RV P P R3 12/31/14 RV P P R4 12/31/14 RV P P R5 12/31/14 RV P P R6 12/31/14 RV P P R7 12/31/14 RV P P R8 12/31/14 RV P P R9 12/31/14 RV P P R10 12/31/14 RV P P R11 12/31/14 RV P P R12 12/31/14 RV P P R13 12/31/14 RV P P R14 12/31/14 RV P P R15 12/31/14 RV P P R16 12/31/14 RV P P R17 12/31/14 RV P P R18 12/31/14 RV P P R19 12/31/14 RV P P R20 12/31/14 RV P P Test Set 2: Positive and Negative Controls in RV media Coefficient set: RV-Broth-Test-02 Zephyr Algorithm Detection Results (NC & PC Samples) Sample ID Date Tested Enrichment Medium BAM Results Zephyr Results SE PSC 12/19/14 RV P P SE PC 12/19/14 RV P P SE NC1 12/19/14 RV N N SE NC2 12/19/14 RV N N SE NC 3 12/19/14 RV N N

6 SE NC 4 12/19/14 RV N N SE NC 5 12/19/14 RV N N NC1 12/31/14 RV N N NC2 12/31/14 RV N N NC3 12/31/14 RV N N NC4 12/31/14 RV N N NC5 12/31/14 RV N N NC1 Real 12/31/14 RV N N PC 12/31/14 RV P P Sensitivity & Specificity True Positive = 44 False Positive = 1 Specificity = 0.95 True Negative = 19 False Negative = 0 Sensitivity = 1.00 Positive (PPV) & Negative (NPV) Predictive Values True Condition + - Test Outcome PPV = NPV = 1.00 Summary of RV results: Out of a total of 64 total samples 44 were found to be positive both by the FDA BAM method and the Zephyr system, 21 were found to be negative by the FDA BAM method while 20 were found to be negative by the Zephyr system. The one sample that showed positive results with the Zephyr system and not by the BAM method was the blind sample R10. The CANARY biosensors showed a very real looking curve in response to interrogation of this sample, with a maximum peak of light output over 5000 Relative Luminescence Units (RLU), making it impossible to exclude this sample from having a positive result by the CANARY - Zehpyr detection algorithm without a significant loss in assay sensitivity. Results: Test Set 3: Blind samples enriched in TT medium Coefficient set: TT-Broth-Test-02 Zephyr Algorithm Detection Results (Blind Samples) Sample ID Date Tested Enrichment BAM Results Zephyr Medium Results T1 12/19/14 TT N N

7 T2 12/19/14 TT P P T3 12/19/14 TT P P T4 12/19/14 TT P P T5 12/19/14 TT P P T6 12/19/14 TT N N T7 12/19/14 TT N N T8 12/19/14 TT P P T9 12/19/14 TT N N T10 12/19/14 TT N P T11 12/19/14 TT P P T12 12/19/14 TT P P T13 12/19/14 TT P P T14 12/19/14 TT P P 15 12/19/14 TT P P T16 12/19/14 TT P P T17 12/19/14 TT P P T18 12/19/14 TT N P T19 12/19/14 TT P P T20 12/19/14 TT P P T31 12/11/14 TT P P T32 12/11/14 TT N N T33 12/11/14 TT P P T34 12/11/14 TT P P T35 12/11/14 TT P P T36 12/11/14 TT P P T37 12/11/14 TT N N T38 12/11/14 TT P P T39 12/11/14 TT P P T40 12/11/14 TT N N T1 12/31/14 TT P P T2 12/31/14 TT P P T3 12/31/14 TT P P T4 12/31/14 TT P P T5 12/31/14 TT P P T6 12/31/14 TT P P T7 12/31/14 TT P P T8 12/31/14 TT P P T9 12/31/14 TT P P T10 12/31/14 TT P P T11 12/31/14 TT P P

8 T12 12/31/14 TT P P T13 12/31/14 TT P P T14 12/31/14 TT P P T15 12/31/14 TT P P T16 12/31/14 TT P P T17 12/31/14 TT P P T18 12/31/14 TT P P T19 12/31/14 TT P P T20 12/31/14 TT P P Test Set 4: Positive and Negative Controls in TT media Coefficient set: TT-Broth-Test-02 Zephyr Algorithm Detection Results (NC & PC Samples) Sample ID Date Tested Enrichment Medium BAM Results Zephyr Results SE PSC 12/19/14 TT P P SE PC 12/19/14 TT P P SE NC1 12/19/14 TT N N SE NC2 12/19/14 TT N N SE NC 3 12/19/14 TT N N SE NC 4 12/19/14 TT N N SE NC 5 12/19/14 TT N N NC1 12/31/14 TT N N PC 12/31/14 TT P P PP PSC 12/31/14 TT P P NC2 12/31/14 TT N N NC3 12/31/14 TT N N NC4 12/31/14 TT N N NC5 12/31/14 TT N N Sensitivity & Specificity True Positive = 45 False Positive = 2 Specificity = 0.89 True Negative = 17 False Negative = 0 Sensitivity = 1.00

9 Positive (PPV) & Negative (NPV) Predictive Values Condition + - Test Outcome PPV = NPV = 1.00 Summary of TT results: Out of a total of 64 total samples run 45 were found to be positive both by the FDA BAM method and the Zephyr system, 19 were found to be negative by the FDA BAM method while 17 were found to be negative by the Zephyr system. One sample that showed positive results with the Zephyr system and not by the BAM method was blind sample 10. The CANARY biosensors showed a very real looking curve in response to interrogation of this sample, with a maximum peak of light output over 1000 RLUs making it impossible to exclude from positive detection by the algorithm without a significant loss in assay sensitivity. The second sample that showed positive results with the Zephyr system and not by the BAM method was sample number TT18. Although this sample did not show a high peak light output, only around 360 RLU, the curve shape and height were nearly identical to some positive frozen samples, again making it impossible to exclude this sample without a reduction in assay sensitivity. Summary: It was possible to tune the Zephyr system detection algorithm to provide good results for the detection of Salmonella in lettuce samples enriched in either media. Results using RV media were slightly better than TT media due to the fact that there was a reduction in light output signal from TT samples that were frozen. This prohibited more exact tuning of the detection algorithm for this broth. Restriction of the type sample used or more tuning of the algorithm for specific sample types may improve performance further.

Use of the 3M Molecular Detection System for Salmonella and Listeria spp.

Use of the 3M Molecular Detection System for Salmonella and Listeria spp. March 11, 213 Prof Steve Forsythe Pathogen Research Centre, School of Science and Technology Nottingham Trent University Clifton