Assessment of CAPRI predictions 2009

Size: px

Start display at page:

Download "Assessment of CAPRI predictions 2009"

Richard McKenzie
5 years ago
Views:

1 Assessment of CAPRI predictions 2009 Barcelona December 9-11 Marc Lensink & Shoshana Wodak Structure and Function of Biological Membranes, Université Libre de Bruxelles, Brussels, Belgium; Hospital for Sick Children, Toronto, and Departments of Biochemistry and Molecular Genetics, University of Toronto, Canada;

2 Participation Rounds Predictors 10 Servers 37 Uploaders 9 Servers 31 Scorers 2 Servers Targets D-Garden ClusPro FiberDock FireDock GRAMM-X HADDOCK HEX PatchDock SAMSON+HEX SKE-DOCK SmoothDock TOP DOWN

3 Target participation Round Target Predictors Uploaders Scorers Models Cancelled

AB A+B Participants Scorer protocol Assessment of docking methods http://capri.ebi.ac.

4 AB A+B Participants Scorer protocol Assessment of docking methods Predictor submission (10) Predictor upload (100) Shuffling/ Tracking Scorer submission (10) Evaluation Assessment of scoring functions Publication results Sep 2009 IECB - Modeling Molecular Interaction 4

5 Evaluation f nat R Nº Receptor interface residues R L f non-nat Nº residue-residue contacts ( 5Å) L L_rms Nº Ligand interface residues L R d L L L I_rms f nat residue-residue 5 Å L-RMS ligand residues I-RMS interface residues 10 Å n clash atom-atom 3 Å d L θ L

6 Participation and clashes # groups # clashes Target

7 CAPRI classification incorrect f nat < 0.1 OR L-RMS > 10.0 AND I-RMS > 4.0 acceptable f nat 0.3 AND L-RMS > 5.0 AND I-RMS > 2.0 OR (0.1 f nat < 0.3) AND (L-RMS < 10.0 OR I-RMS < 4.0) medium f nat 0.5 AND L-RMS > 1.0 AND I-RMS > 1.0 OR (0.3 f nat < 0.5) AND (L-RMS < 5.0 OR I-RMS < 2.0) high f nat 0.5 AND L-RMS < 1.0 AND I-RMS < 1.0

8 Target 29 Target type Complex bound / unbound Trm8/Trm82 trna guanin-n(7)- methyltransferase Receptor Trm82, bound 450 residues Ligand Free Trm8 supplied 254 residues RMSD Receptor 0 Å Ligand 1.69 Å Interface Area Å 2 Nicolas Leulliot, Herman van Tilbeurgh, IBBMC, Orsay, France

9 T29 receptor

10 T29 prediction spread

11 381 models 0 *** 9 ** 8 * T29 Predictors target high medium acceptable incorrect Receptor bound / unbound Ligand bound / unbound

12 381 models 0 *** 9 ** 8 * 2192 models 2 *** 78 ** 87 * Target 29 Uploaders Receptor bound / unbound Ligand bound / unbound target high medium acceptable incorrect

13 381 models 0 *** 9 ** 8 * 2192 models 2 *** 78 ** 87 * 145 models 1 *** 13 ** 13 * Target 29 Scorers Receptor bound / unbound Ligand bound / unbound target high medium acceptable incorrect

14 Target 29 Predictor Uploader Scorer Ten Eyck 6** 27/17** 0 Bonvin 6/1** 77/2*** 9/5** Zhou 1** 0 Fernandez-Recio 1** 4/2** 5/1*** Eisenstein 1* 1* x Smith 1* 2* 0 Weng 1* 21/4* 3/2** Bates 0 1* 4/2** Camacho 0 2* 2/1** Vajda 0 2/1** Wolfson 0 2* Zacharias 0 6/1** x Baker 0 2** x CLUSPRO 0 1* x

15 Target 29 best model Participant Model F(nat) L-rms I-rms source S.Fernandez-Recio high (Bonvin) = unbound high interface high complex + unbound monomers unbound 1. take unbound monomers 2. fit on complex 3. conformational change unbound or interface complex interface 1. take unbound monomers 2. fit on interface residues 3. interface prediction

16 Target 30 Target type unbound Complex Rnd1-GTP bound to RBD dimer Receptor Rnd1 Ligand RBD RMSD Receptor 1.71 Å Ligand 2.28 Å Interface Area Å 2 W. Tempel, C. Arrowsmith, SGC, Toronto, Canada

17 Target 30 Only 1 residue of the other RBD monomer is in contact with Rnd1, but this residue is in a flexible loop. Rnd1-GTP bound to RBD dimer Rnd1-GTP bound to RBD monomer In solution, the RBD may be a monomer or a dimer

18 350 models 0 *** 0 ** 2 * T30 Predictors target high medium acceptable incorrect Receptor bound / unbound Ligand bound / unbound

19 350 models 0 *** 0 ** 2 * 1346 models 0 *** 0 ** 2 * T30 Uploaders Receptor bound / unbound Ligand bound / unbound target high medium acceptable incorrect

20 350 models 0 *** 0 ** 2 * 1346 models 0 *** 0 ** 2 * 131 models 0 *** 0 ** 0 * Receptor bound / unbound Ligand bound / unbound T30 Scorers target high medium acceptable incorrect

21 T30 results Target 30 Predictor Uploader Scorer Bates 1* 1* 0 Zacharias 1* x Takeda-Shitaka 0 1* 0

22 T30 best model Participant Model F(nat) L-rms I-rms Zacharias acceptable unbound high, clashes (83>62) interface high, clashes (69>62)

23 Target 32 Target type unbound Complex Protein-protein complex Receptor Protease savinase (1SVN) 269 residues Ligand Bi-functional inhibitor BASE (1AVA:C) 181 residues RMSD Receptor 0.31 Å Ligand 2.14 Å Interface Area Å 2 Leonardo de Maria, Pernille Michelsen, Novozymes, Bagsvaerd, Denmark

24 354 models 15 *** 13 ** 6 * T32 prediction spread /2 0 - /2-0 /2 target high medium acceptable incorrect

25 354 models 15 *** 13 ** 6 * 599 models 0 *** 3 ** 12 * T32 uploader spread target high medium acceptable incorrect

26 354 models 15 *** 13 ** 6 * 599 models 0 *** 3 ** 12 * 120 models 0 *** 0 ** 2 * T32 scorer spread target high medium acceptable incorrect

27 Target 32 Predictor Uploader Scorer Gray 8/5*** x Zacharias 5/4*** x Baker 5/2*** x Vajda 3/2*** x Wolfson 6/2*** 11/3** 0 Eisenstein 2** x Camacho 1** 0 Tovchigrechko 1** x Del Carpio 1* x Zhou 1* x Zou 1* 0 Vakser 0 2* Fernandez-Recio 0 4* 0

28 T32 best model Participant Model F(nat) L-rms I-rms Gray high unbound high interface high

29 Targets 33/34 Target type 33 unbound / homology Target type 34 bound / unbound Complex Protein-RNA Receptor 74 nucleotide RNA transcript RNA incl. 3 hairpins of 23S rrna Ligand Rlma2 Protein methyltransferase (1P91) Interface Area Å 2 Louis Renault, LEBS, Gif-sur-Yvette, France

30 Target 33 Supplied: Structural alignment to build Rlma2 from template 1P91 Hairpins 33, 34, 35 of the 23S rrna Rlma2 methylates N1 of G748 in hairpin 35 A lot of creativity in the naming of nucleic acids No consensus on RNA structure among predictions No acceptable solutions for T33

31 Target 34 Bound RNA structure supplied Best model has I-rms = 1.2 Å (requirement for high quality is I-rms < 1.0 Å) (P)redictor (U)ploader (S)corer

32 Target 34 Predictor Uploader Scorer Maigret 9/5** 0 Baker 6/4** x Leclerc 9/3** x Bonvin 6/3** 92/13** 7/1** Mitchell 3** x Ritchie 6/2** x Zou 3/2** 7/2** Vajda 3/2** x Zhou 4/1** x HADDOCK 7* 85* x Gray 3* 0 Camacho 2* 0 Eisenstein 2* x Fernandez-Recio 1* 1* 7/1** CLUSPRO 1* x Bates 0 9*

33 Targets 33/34 best model Participant Model F(nat) L-rms I-rms Vakser incorrect 33 unbound incorrect, 1422 clashes interface incorrect, 1109 clashes Baker medium 34 unbound high, clashes (197>134) interface high, clashes (153>134)

34 Targets 35/36 Target type 35 Target type 36 Complex Receptor Ligand RMSD homology / homology bound / homology Covalently linked molecule; Xylanase Xyn10B Polysaccharide binding module CBM22 (1N82); bound for T36; 331 residues Catalytic module GH10 (1DYO); 160 res Ligand and receptor 2.9 Å Interface Area Å 2 Information 7 residue linker disordered linker region S. Najmudin, C. Fontes, Universidade Tecnica de Lisboa, Portugal

35 Results Target 35 Target 36 Predictor Uploader Scorer Predictor Uploader Scorer # groups *** ** * # models

36 T35/T36 prediction spread target high medium acceptable incorrect Receptor bound / unbound

37 T35/T36 prediction spread target predictor scorer target high medium acceptable incorrect Receptor bound / unbound uploader

38 Target 35 Predictor Uploader Scorer Fernandez-Recio 1* 2* 0 Wang 0 1* Target 36 Weng 1* 0 Participant Model F(nat) L-rms I-rms Classification unbound medium, 136 clashes interface medium, 213 clashes Target 35 U.Fernandez-Recio acceptable Target 36 Weng acceptable

39 Target 37 Target type Complex Receptor Ligand 77 residues unbound / homology protein-protein complex G-protein Arf6 (2Q5D:13-174) LZ2, Leucine zipper of JIP4 (JNKinteracting protein 4), to be modelled on GCN4 leucine zipper (2ZTA) RMSD Receptor 0.55 Å Ligand 0.35 Å Interface Area Å 2 Julie Menetrey, Institut Curie, Paris, France

40 Target 37 results Symmetrical chains 2 assessments lowest RMSD Predictor Uploader Scorer # *** ** *

41 T37 predictor spread

42 T37 enriched decoy set

43 T37 scorer predictions

44 Target 37 Predictor Uploader Scorer Weng 1*** 10/2*** 2/1*** Elofsson 3** 17/1*** 0 Bonvin 2** 12/2*** 2/1** Zou 3/1** 6/1*** 4/2*** Zacharias 2/1** x Bates 4* 2* 6/1*** SKE-DOCK 2* 16/2*** x Camacho 1* 0 Konishita 1* x Wolfson 1* 6/3** 1* Zhou 1* x HADDOCK 0 1* x PATCHDOCK 0 2* x Ritchie 0 6/1** x Aze 0 1*** Haliloglu x 5/4** Mitchell 0 2* Wang 0 6/4**

45 T37 high quality models Target 37 model F(nat) L-rms I-rms S.Zou high (U.Bonvin.75) + U.Zou high S.Bates S.Bates high (U.Zou.44) - U.Bonvin high S.Zou S.Aze high (U.SKE-DOCK.14) = U.SKE-DOCK high S.Aze U.SKE-DOCK high S.Zou P.Weng high U.Elofsson high S.Mitchell S.Zou high (U.SKE-DOCK.57) - U.Bonvin high S.Weng S.Weng high (U.Bonvin.59) - U.Weng high U.Weng high

46 Targets 38/39 Target type 38 Target type 39 Complex Receptor Ligand (124 residues) unbound / homology unbound / bound protein-protein complex Centaurin-alpha 1; 392 residues FHA domain of KIF13B; templates 2G1L or 3FEH or ; bound for T39 RMSD Receptor 1.94 Å Ligand 3.24 Å Interface Area Å 2 Hee-Won Park, Structural Genomics Consortium, Toronto, Canada

47 T38 prediction spread 1.94 Å - Receptor bound / unbound 3.24 Å - Ligand bound / unbound No acceptable models (U) (P)redictor (S)

48 T38/T39 ligand spatial distribution *** 2** 0 Target 38 Target 39 target predictor uploader scorer

49 T39 acceptable models (all) Participant Model F(nat) L-rms I-rms Vajda high unbound medium, low_seq_id interface medium, 100 clashes CLUSPRO medium Wang medium U.Fernandez-Recio medium U.Weng medium U.Fernandez-Recio medium U.Fernandez-Recio acceptable no acceptable scorer submissions

50 Target 40 Target type Complex Receptor Ligand Receptor RMS unbound / bound trypsin / protease inhibitor Bovine trypsin (1BTY); 223 residues double-headed arrowhead protease inhibitor API-A; bound; 185 residues 0.42 Å Interface Area Å 2 Yuxing Chen, Rui Bao University of Science and Technology, Hefei, Anhui, China

51 T40 break-down T40.CA T40.CB trypsin trypsin API-A Each inhibitor molecule binds two trypsin molecules. Models should contain only one trypsin molecule. Both binding modes are assessed.

52 T40 cumulative results Target 40 Predictor Uploader Scorer # models T40.CA *** ** * T40.CA T40.CB T40.CB

53 T40 spherical clusters T40.CB T40.CA

54 Target 40 Predictor performance Bonvin 10*** Gray 9*** Takeda-Shitaka 9/8*** Nakamura 10/6*** Zacharias 10/4*** Vajda 9/4*** Vakser 8/4*** Comeau 10/3*** Zhou Zou 9/3*** Ten Eyck Wolfson 6/3*** Wang 5/3*** Eisenstein 4/3*** Baker Tovchigrechko 2*** Bates 10/1*** Xiao 7/1*** Camacho 6/1*** Weng 3/1*** Ritchie 6/4** Elber F_Jiang 1*

55 Target 40 T40.CA T40.CB Bonvin 5*** 5*** Gray 6*** 3*** Takeda-Shitaka 8*** 1** Nakamura 2*** 4*** Zacharias 3*** 1*** Vajda 2*** 2*** Vakser 3*** 1*** Comeau 1*** 2*** Zhou Zou 3*** 2*** 0 1*** Ten Eyck Wolfson 2*** 2*** 1*** 1*** Wang 1*** 2*** Eisenstein 2*** 1*** Baker Tovchigrech. 2*** 1*** 0 1*** Bates 5** 1*** Xiao 2* 1*** Camacho 2** 1*** Weng 1** 1*** Ritchie 2** 2** Elber FJiang 0 0 1* 1* best model per interface

56 Target 40 SERVER performance Predictor (10) T40.CA T40.CB Uploader (100) GRAMM-X 1*** 1*** 0 FIREDOCK 2/1*** 0 3/2*** SKE-DOCK 0 2/1*** 2/1*** HADDOCK 0 1*** 20/11*** TOP DOWN 0 2/1** 7/3** CLUSPRO 1** 1* 0 PATCHDOCK Scorer Scorer perforance Takeda-Shitaka 10*** 0 Camacho 5/4*** 5*** Wolfson 1*** 8/5*** Haliloglu 1** 5/4*** Elber 3/1*** 5/2*** Zou 5/1*** 5/1*** Bonvin 6/4** 4/2*** Weng 7/2*** 0 Wang 7/3** 1*** Bates 9/8** 1** Liu 3** 1**

57 T40 best models T40.CA Model F(nat) L-rms I-rms unbound high interface high Ten Eyck high GRAMM-X high T40.CB interface high unbound high Nakamura high Eisenstein high the best models are predictor-made

58 Target 41 Target type unbound Complex Colicin E9/Im2 Receptor Colicin E9 DNase domain (1FSJ) 134 residues Ligand Im2 immunity protein (2NO8) 86 residues RMSD Receptor 1.53 Å Ligand 2.03 Å Interface Area Å 2 Nicola Meenan, Colin Kleanthouse, University of York, UK

915 ; 8 clashes First time that predictions outscored the unbound / interface constructions Target

59 T41 results Trivial high-quality template: 1EMV P16 (Takeda-Shitaka): 10/8*** only participant to improve upon template best model 1EMV Å; f nat =0.966 ; 10 clashes Å; f nat =0.915 ; 8 clashes First time that predictions outscored the unbound / interface constructions Target 41 Predictor Uploader Scorer # models *** ** * % 24.8 % 53.1 %

60 *** 24 ** 58 * 67 incorrect 178 T41 Predictor results

61 *** 2 ** 99 * 198 incorrect 909 T41 Uploader results

62 *** 2 ** 16 * 51 incorrect 61 T41 Scorer results

63 Target 41 Predictor performance Takeda-Shitaka 10/8*** Zou 9/6*** Zacharias 8/5*** Xiao 9/1*** Zhou 4/1*** Eisenstein 2/1*** Alexov / Vakser 10** Wolfson 10/7** Bonvin / Nakamura 10/2** Ritchie 9/2** Vajda 2** Gray 8/1** Bates 6/1** Weng 5/1** Bajaj 4/1** Gunther / Wang 2* Fernandez-Recio / FJiang / Kihara 1*

64 T41 Server performance FiberDock 10/1*** GRAMM-X 1*** HADDOCK 4/1** ClusPro 1** 3D-Garden 0 PatchDock 0 SAMSON+HEX 0 SKE-DOCK 0 TOP DOWN 0

65 T41 Scorer performance Scorer Uploader Zou 10/2*** 4/2*** Xiao 7/4** Takeda-Shitaka 10/2** Fernandez-Recio 3/2** SAMSON+HEX 10/1** Kihara 3/1** Bonvin 10* 17/2** Wang 7* 16* Bates 4* 24/2** Weng 4* 20/7** Elber 1* Wolfson 71/54** Nakamura 100/24** HADDOCK 24/5** Ritchie 21/3** FJiang 2*

66 Target 42 Target type Complex unbound Trimer of a designed TPR repeat 102 residues A:B Interface Area Å 2 B:C Interface Area Å 2 Predictions were assessed against the pairs A:B (symmetrical) B:C (non-symmetrical) Colin Kleanthouse, Amit Sharma, University of York, UK

67 T42 prediction spread T42.AB T42.BC *** 7 2 ** 4 1 * 4 2

68 T42 performance Target 42 A:B B:C Zacharias 5/2*** 0 Wolfson 2/1*** 0 Eisenstein 0 2/1*** Nakamura 1*** 0 PATCHDOCK 1*** 0 Vajda 1*** 0 CLUSPRO 1*** 0 Zou 0 1*** Fernandez-Recio 2** 0 Mitchell 1** 0 Weng 0 1** Bonvin 1* 0 HADDOCK 0 1*

69 Ranking of models? Are de best-scoring models also the highest-ranked in the respective predictor or scorer set? model M 1 T29 10 Predictor list

70 Ranking of models T29 bound unbound Trm8/Trm82 T30 unbound T32 unbound T33 homology unbound T34 bound unbound RBD monomer protease inhibotor RNA / Rlma2 transferase T35 homology homology T36 bound homology covalently linked

71 Ranking of models T37 unbound homology leucine zipper T38 unbound homology centaurin alpha / T39 unbound T40 bound T41 unbound bound unbound FHA domain trypsin / protease inhibitor 1EMV T42 unbound TPR repeat

72 Ranking of predictor vs. scorer models

73 Target success *** ** * P U S P U S P U S T T T T T T T T T T T T T42 9 x x 5 x x 6 x x

74 T arg et Scorer success P redictor Uploader S corer P erc entag e of models ac c eptable or better

75 SERVER success Target ClusPro * ** 2/1** 1** 1*** FiberDock 10/1*** 0 FireDock /1*** GRAMM-X *** 1*** 0 HADDOCK 0 0 7* *** 4/1** 1* SKE-DOCK * 0 0 2/1*** 0 0 Top Down 0 0 2/1** 0 0 Only prediction-servers mentioned Non-scoring servers not listed

76 Easy targets are easy Docking servers give reliable results Nucleic acids not problematic provided there is no conformational change Unbound docking approaches bound docking for low bound/unbound RMS Good results for the Scoring Experiment but only when good models are available Scorers improve uploaded models Difficulties Conformational change including displacement of loops Required information biochemical interaction data homologous structures reliable interface prediction Conclusions

77 Target Summary Predictor Experiment Scorer Experiment T29 7/4** 7/1***/5** T T32 11/5***/3** 1 T T34 16/9** 4/3** T T T37.1 7/3** 8/4***/3** T37.2 9/1***/2** 5/3** T T39 3/1***/2** 0 T40.CA 24/18***/5** 11/6***/5** T40.CB 26/20***/3** 9/7***/2** T41 26/8***/13** 11/1***/5** T42.AB 9/6***/2** - T42.BC 4/2***/1** -

78 Predictor Predictor Summary Vajda 6/4***/2** Zacharias 6/4***/1** Zou 6/3***/2** Eisenstein, Wolfson 6/3***/1** Weng, Zhou 6/2***/2** Bonvin 6/1***/4** CLUSPRO 5/1***/3** Fernandez-Recio 5/2** Gray 4/2***/1** Bates, Camacho, HADDOCK 4/1***/1** Nakamura, Baker 3/2***/1** Wang 3/1***/1** Ritchie 3/3** GRAMM-X, Takeda-Shitaka, Xiao 2/2*** Tovchigrechko, Ten Eyck, Vakser 2/1***/1** SKE-DOCK 2/1*** Mitchell 2/2** F_Jiang 2 Comeau, PATCHDOCK, FIBERDOCK, FIREDOCK 1/1*** Elofsson, TOPDOWN, Maigret, Leclerc, Alexov, Bajaj 1/1** Smith, Elber, Kihara, Gunther, Kinoshita, Del Carpio 1

79 Predictor (Server) Server Summary ClusPro 5/1***/3** HADDOCK 4/1***/1** GRAMM-X 2/2*** SKE-DOCK 2/1*** PatchDock, FiberDock, FireDock 1/1*** TOP DOWN 1/1** HEX, 3D-Garden, SmoothDock, Samson+HEX 0 Scorer Scorer Summary Bonvin 5/1***/3** Bates 5/1***/2** Zou 4/3***/1** Weng 4/2***/1** Wang 4/1***/1** Fernandez-Recio 3/1***/2** Wolfson 3/1*** Haliloglu, Camacho, Takeda-Shitaka 2/1***/1** Elber 2/1*** Aze 1/1*** Liu, SAMSON+HEX, Kihara, Vajda, Xiao 1/1** Vakser, Mitchell 1

proteins REVIEW Sampling and scoring: A marriage made in heaven Sandor Vajda,* David R. Hall, and Dima Kozakov

proteins REVIEW Sampling and scoring: A marriage made in heaven Sandor Vajda,* David R. Hall, and Dima Kozakov proteins STRUCTURE O FUNCTION O BIOINFORMATICS REVIEW Sampling and scoring: A marriage made in heaven Sandor Vajda,* David R. Hall, and Dima Kozakov Department of Biomedical Engineering, Boston University,