Reducing False Positives with Automated NMR Verification Ryan Sasaki NMR Product Manager SMASH 2011
Why Not More NMR? Cost $$$ NMR > LC-MS > GC-MS 1 Sample requirements/sensitivity Tougher to interpret But can we afford to NT have a complementary data evaluation tool? 1 High-Throughput NMR Analysis: The End Game, Anthony Macherone, ASDI Group of Companies, ENC 2008
Why NMR? Cl Br Br H Cl Br Br H Cl N N N H CH H 3 C 3 CH 3 CH 3 H 3 C CH 3 Cl N N N H 3 C CH 3 H 3 C H H 3 C CH 3 CH 3 C H 3 CH 3 Br Br H 3 C CH 3 CH 3 H H H 3 C H C H 3 H N + - H 3 C H N + - H
The rthogonal NMR LC-MS is still the way to go on a sample-by-sample basis in high-throughput Usage of NMR as a complement to LC-MS Automatic evaluation by NMR of only those that pass LC-MS analysis The goal: Identify a manageable subset of compounds that may require a second look The challenge: Ensure incorrect structures get caught
Major Challenges in Automated NMR Verification Today Limited set of experiments for routine and high-throughput work 1 H NMR CSY? HSQC? Balancing acquisition time vs. acceptable results How much is enough? Balancing False Positives vs. False Negatives
ur Focus Ensuring samples that pass our system are passing for the right reason Improving accuracy of NMR assignments 2 Catching the false positives Both endeavors are impacted by the amount of data that can be acquired. How much is enough? 2 Evaluation of the Benefit of Including CSY and HSQC 2D Data in Automated Structure Verification, ENC 2010
1 H and HSQC Combined Verification The benefits of a combination of 1 H and HSQC NMR Prediction 13 C chemical shifts are generally more predictable Usage of both 1 H and 13 C improve assignments and overall verification performance Incorrect assignment of 1 H can be proactively caught by prediction of attached carbon s chemical shift and vice versa
1 H and HSQC Combined Verification The benefits of a combination of 1 H and HSQC Peak Picking and multiplet creation Filtering of peak artifacts in 2D Identification of Labile Protons in 1D Easier identification of diastereotopic protons Better recognition of distinct, but overlapping multiplets Multiplicity-edited information can help assignments The drawbacks The information does not always prove that the structure is correct.
Introducing Concurrent NMR Verification Can be used for any verification routine ( 1 H, 13 C, Combined 1 H & HSQC). Verification triggers the generation of multiplet alternative structures every time a proposed structure passes. Software automatically evaluates the verification of all proposed structures under default conditions and settings F F F NH H F F F NH H
Concurrent NMR Verification If software passes at least one generated structure, it will re-run verification under tighter chemical shift constraints
Concurrent NMR Verification If multiple structures (including the proposed) survive 3 iterations, a flag is generated. False positive warning- Either multiple structures pass, or all structures fail False positive alert- nly incorrect structure passes False positive warnings and alerts suggest The proposed structure may not be correct The data is ambiguous and additional experiments may be required.
Concurrent NMR Verification Questions to ask: How does this affect the pass rate How does this affect the false positive rate? What is the best way to measure the results?
Concurrent NMR Verification Test 1 127 1 H and HSQC datasets evaluated ne positive control and one negative control structure was evaluated CH 3 H 3 C C H 3 N C H 3 N CH 3 N CH 3 N NH NH
Comparison of Standard Verification vs. Concurrent Verification 140 120 100 79% Pass Rate 100 20% False Positive Rate 101 72% Pass Rate 92 0% False Positive Rate 119 80 60 40 27 26 27 20 0 8 8 Correct Structures Incorrect Structures Correct Structures Incorrect Structures 0 Standard Verification Concurrent Verification Pass Fail Alert/Warning
Concurrent NMR Verification Test 1 Results Test 1 suggested a system whereby 72% of spectra (92/127) can be automatically evaluated without human intervention with a false positive rate of 0% This compared to the standard verification approach whereby 79% of spectra (100/127) can be automatically evaluated but with a false positive rate of 20%.
Concurrent NMR Verification Test 2 The same 127 1 H and HSQC datasets evaluated Mimic a scenario where the correct structure is not proposed Two negative controls (wrong structures) were evaluated riginally proposed incorrect structure from Test #1 was considered the proposed structure for a fair comparison of two approaches
Comparison of Standard Verification vs. Concurrent Verification 120 100 101 102 80 60 40 20% False Positive Rate 20 26 5% False Positive Rate 19 6 0 Incorrect Structures Standard Verification Incorrect Structures Concurrent Verification Pass Fail Alert/Warning
Concurrent NMR Verification Test 2 Results Test 2 results an improvement in false positive detection from 20% to 5% This improvement comes with the added cost of 19 additional datasets that were flagged for manual review
Conclusions Tests revealed that concurrent verification can dramatically improve false positive detection rates without a significant increase in manual labor (review of flagged results) The new category of false positive warnings/alerts can be used to communicate that more experiments may be required for confirmation
Current and Future Developments Implementation of a structure generation component to automatically generate alternative structures on the fly More tests to evaluate the impact of increasing the number of alternative chemical structures 3 Analysis of the impact of concurrent verification on other experiments ( 1 H, 13 C, etc.) Further work on the impact of CSY on Combined Verification 2 The usage of peak deconvolution to estimate the reliability of multiplicity patterns 3 ENC Posters #386 and #388
Acknowledgements Sergey Golotvin, ACD/Labs Kirill Blinov, ACD/Labs Asya Nikitina, ACD/Labs Phil Keyes, Lexicon Pharmaceuticals Gonzalo Hernandez, Vis Magnetica John Hollerton, GSK Stevenage Duncan Farrant, GSK Stevenage Randy Rutkowske, GSK RTP Tim Spitzer, GSK RTP