De Novo molecular design with Deep Reinforcement Learning

Transcription

1 De Novo molecular design with Deep Reinforcement Olexandr Isayev, Ph.D. University of North Carolina at Chapel Hill

2 About me Ph.D. in Chemistry (computational) Minor in CS/ML Worked in Federal research lab on HPC & GPU computing to solve chemical problems Now I am faculty at the University of North Carolina, Chapel Hill We use ML & AI to solve challenging problems in chemistry olexandr@unc.edu

3 A public-private partnership that supports the discovery of new medicines through open access research

4 The Long and Winding Road to Drug Discovery Data Science approaches useful across the pipeline, but very different techniques aim for success, but if not: fail early, fail cheap

5 internal rate of return (IRR) Source: Endpoints News

6 Drowning in Data but starving for Knowledge

7 The growing appreciation of molecular modeling and informatics 7

8 Behold the rise of the machines

9 Summary of recent AI-based studies on chemical library design Molecular representations Generative models Method of biasing generated compounds Fingerprints SMILES Graphs Autoencoders Generative adversarial models (GANs) Recurrent neural networks (RNNs) Convolutional neural networks (CNNs) None Latent space optimization Fine-tuning on small subset of molecules with the desired property Reinforcement Learning

10 De Novo molecular design with Deep Reinforcement Learning General Approach Application to Molecular design Tm; LogP; pic50; etc Predictive Deep Network Molecules Generative Deep Network Patent pending arxiv:

11 Drug discovery pipeline CHEMICAL STRUCTURES CHEMICAL DESCRIPTORS PREDICTIVE QSAR MODELS PROPERTY/ ACTIVITY CHEMICAL DATABASE QSAR MAGIC VIRTUAL SCREENING HITS (confirmed actives) ~ molecules INACTIVES (confirmed inactives)

12 Design of the ReLeaSE* method Challenges: Generate chemically feasible SMILES Develop SMILESbased QSAR model Employ Predictive ML model to bias library generation *Popova, Mariya, Olexandr Isayev, and Alexander Tropsha. "Deep reinforcement learning for de-novo drug design." arxiv preprint arxiv: (2017).

13 Language of SMILEs

14 Generative model 1.5M molecules from ChEMBL <START>c1ccc(O)cc1<END> c1cc) ( F) cc1<end> c1ccc(o)cc1 NO <START> c1ccc( O) cc1 + loss YES Did the training converge? Softmax loss

15 Reinforcement learning for chemical design Generative model Predictive model FC(F)COc1ccc2c(Nc3ccc(Cl)c(Cl)c3)ncnc2c1 M. Popova, O. Isayev, A. Tropsha. "Deep reinforcement learning for de-novo drug design." arxiv preprint arxiv: (2017).

16 Reinforcement learning for chemical design Generative model Predictive model FC(F)COc1ccc2c(Nc3ccc(Cl)c(Cl)c3)ncnc2c1 M. Popova, O. Isayev, A. Tropsha. "Deep reinforcement learning for de-novo drug design." arxiv preprint arxiv: (2017).

17 Reinforcement learning for chemical design Generative model INACTIVE! Predictive model M. Popova, O. Isayev, A. Tropsha. "Deep reinforcement learning for de-novo drug design." arxiv preprint arxiv: (2017).

18 Reinforcement learning for chemical design Generative model INACTIVE! Predictive model M. Popova, O. Isayev, A. Tropsha. "Deep reinforcement learning for de-novo drug design." arxiv preprint arxiv: (2017).

19 Reinforcement learning for chemical design Generative model INACTIVE! Predictive model M. Popova, O. Isayev, A. Tropsha. "Deep reinforcement learning for de-novo drug design." arxiv preprint arxiv: (2017).

20 Reinforcement learning for chemical design Generative model Predictive model M. Popova, O. Isayev, A. Tropsha. "Deep reinforcement learning for de-novo drug design." arxiv preprint arxiv: (2017).

21 Reinforcement learning for chemical design Generative model Predictive model Fc1ccc2c(Nc3ccc(F)c(F)c3)ncnc2c1 M. Popova, O. Isayev, A. Tropsha. "Deep reinforcement learning for de-novo drug design." arxiv preprint arxiv: (2017).

22 Reinforcement learning for chemical design Generative model Predictive model Fc1ccc2c(Nc3ccc(F)c(F)c3)ncnc2c1 M. Popova, O. Isayev, A. Tropsha. "Deep reinforcement learning for de-novo drug design." arxiv preprint arxiv: (2017).

23 Reinforcement learning for chemical design Generative model ACTIVE! Predictive model M. Popova, O. Isayev, A. Tropsha. "Deep reinforcement learning for de-novo drug design." arxiv preprint arxiv: (2017).

24 Reinforcement learning for chemical design Generative model ACTIVE! Predictive model M. Popova, O. Isayev, A. Tropsha. "Deep reinforcement learning for de-novo drug design." arxiv preprint arxiv: (2017).

25 Reinforcement learning for chemical design Generative model ACTIVE! Predictive model M. Popova, O. Isayev, A. Tropsha. "Deep reinforcement learning for de-novo drug design." arxiv preprint arxiv: (2017).

26 Reinforcement learning for chemical design Generative model Predictive model M. Popova, O. Isayev, A. Tropsha. "Deep reinforcement learning for de-novo drug design." arxiv preprint arxiv: (2017).

27 Technical details Models were trained on Nvidia Titan X and Titan V GPUs Training generative model on ChEMBL took ~ 25 days Training predictive models took ~ 2 hours Biasing generative model with reinforcement learning for one property ~ 1 day Generative model produces 1000s compounds per minute

28 Results: Biasing target properties in the designed libraries Optimized Baseline * JAK2 Inhibition (pic50) Melting temperature (T m ), o C 400 Partition coefficient (logp) M. Popova, O. Isayev, A. Tropsha. "Deep reinforcement learning for de-novo drug design." arxiv preprint arxiv: (2017).

29 JAK2 (Kinase) inhibition Train data distribution Maximized property distribution Minimized property distribution NEW CHEMOTYPE CAS (buffer reagent) ZINC SIMILAR SCAFFOLDS New molecule arxiv:

30 Results: analysis of similarity Distribution of Tanimoto similarity to the nearest neighbor in training dataset for compounds predicted to be active for EGFR by consensus of QSAR models: Similarity= 0.69 Similarity= 0.57 Similarity = Tanimoto similarity 1.0

31 Results: Synthetic accessibility score* of the designed libraries *Ertl, Peter, and Ansgar Schuffenhauer. "Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions." Journal of cheminformatics 1.1 (2009): 8.

32 Target predictions for generated compounds using SEA* *Keiser MJ, Roth BL, Armbruster BN, Ernsberger P, Irwin JJ, Shoichet BK. Relating protein pharmacology by ligand chemistry. Nat Biotech 25 (2), (2007).

33 Target predictions for generated compounds using SEA* *Keiser MJ, Roth BL, Armbruster BN, Ernsberger P, Irwin JJ, Shoichet BK. Relating protein pharmacology by ligand chemistry. Nat Biotech 25 (2), (2007).

34 Model visualization for JAK2 (projection using t-sne) ZINC pic50 = 8.64 ZINC pic50 = 3.31 ZINC pic50 = 8.23 ZINC pic50 = 3.76 ZINC pic50 = 8.39 pic50 = pic50 = M. Popova, O. Isayev, A. Tropsha. "Deep reinforcement learning for de-novo drug design." arxiv preprint arxiv: (2017).

35 Examples of Stack-RNN cells with interpretable gate activations M. Popova, O. Isayev, A. Tropsha. "Deep reinforcement learning for de-novo drug design." arxiv preprint arxiv: (2017).

36 Summary AI methods coupled with SMILES representation afford biased libraries generation The system naturally embeds reinforcement to produce novel structure with the desired property The system can be tuned to bias libraries towards specific property ranges Next phase is experimental validation of hits by UNC SGC team