TreeShrink: efficient detection of outlier tree leaves

Size: px

Start display at page:

Download "TreeShrink: efficient detection of outlier tree leaves"

Julian Osborne
5 years ago
Views:

1 TreeShrink: efficient detection of outlier tree leaves Uyen Mai Siavash Mirarab University of California at San Diego 1

2 Lesser Hedgehog Tenrec Long branches are suspect Sphagnum lesc Amborella trichopoda Nuphar advena Monilophytes Flowering plants Gnetum montanum Ephedra sinica Gymnosperms Tarsier Guinea Pig Kangaroo Rat Squirrel Galagos Mouse Lemur Tree Shrew Selaginella moellendorffii 1kp Selaginella moellendorffii genome Chaetosphaeridium globosum Klebsormidium subtile Entransia fimbriata Coleochaete irregularis Monomastix opisthostigma Netrium digitus Pyramimonas parkeae Roya obtusa Cosmarium ochthodes Nephroselmis pyriformis Coleochaete scutata Penium margaritaceum Chlorokybus atmophyticus Mougeotia sp Cylindrocystis brebissonii Mesotaenium endlicherianum Cylindrocystis cushleckae Spirogyra sp Chara vulgaris Pseudolycopodiella caroliniana Dendrolycopodium obscurum Huperzia squarrosa Nothoceros aenigmaticus Nothoceros vincentianus Mosses Liverworts 0.6 A gene tree from the1kp plant dataset (Wicket et al, PNAS, 2014) Rat Pika Rabbit Mouse Macaque Chimp Human Orangutan Gorilla Marmoset Alpaca

3 Lesser Hedgehog Tenrec Long branches are suspect Sphagnum lesc Idea: find errors in the data by building a phylogeny and detecting long branches Amborella trichopoda Nuphar advena Monilophytes Flowering plants Gnetum montanum Ephedra sinica Gymnosperms Tarsier Guinea Pig Kangaroo Rat Squirrel Galagos Mouse Lemur Tree Shrew Selaginella moellendorffii 1kp Selaginella moellendorffii genome Chaetosphaeridium globosum Klebsormidium subtile Entransia fimbriata Coleochaete irregularis Monomastix opisthostigma Netrium digitus Pyramimonas parkeae Roya obtusa Cosmarium ochthodes Nephroselmis pyriformis Coleochaete scutata Penium margaritaceum Chlorokybus atmophyticus Mougeotia sp Cylindrocystis brebissonii Mesotaenium endlicherianum Cylindrocystis cushleckae Spirogyra sp Chara vulgaris Pseudolycopodiella caroliniana Dendrolycopodium obscurum Huperzia squarrosa Nothoceros aenigmaticus Nothoceros vincentianus Mosses Liverworts 0.6 A gene tree from the1kp plant dataset (Wicket et al, PNAS, 2014) Rat Pika Rabbit Mouse Macaque Chimp Human Orangutan Gorilla Marmoset Alpaca

4 For unrooted trees? Diameter: the longest path between any two species A gene tree from the1kp plant dataset (Wicket et al, PNAS, 2014) 3 0.2

5 For unrooted trees? Diameter: the longest path between any two species A gene tree from the1kp plant dataset (Wicket et al, PNAS, 2014)

6 An optimization problem The k-shrink problem: Given: a tree with n leaves and branch lengths some 1 k n 4

7 An optimization problem The k-shrink problem: Given: a tree with n leaves and branch lengths some 1 k n Find: for every 1 i k: the set of i leaves that should be removed to reduce the tree diameter maximally 4

8 An optimization problem The k-shrink problem: Given: a tree with n leaves and branch lengths some 1 k n Find: We have a polynomial for every 1 i k: time solution the set of i leaves that should be removed to reduce the tree diameter maximally 4

9 Running Time k-shrink can be solved in O(k 2 h+n) where h = the tree height by default, we set k=o(n 0.5 ) 5

10 Running Time k-shrink can be solved in O(k 2 h+n) where h = the tree height by default, we set k=o(n 0.5 ) Fast enough: processes a tree of n=203,452 leaves with k=2255 in 28 mins 5

11 How many do we remove? How do we decide how many things to remove? We have the optimal removals for 1 i k. What i should we use? 6

12 How many do we remove? How do we decide how many things to remove? We have the optimal removals for 1 i k. What i should we use? Find an i where the corresponding reduction in the diameter is unexpectedly high needs statistical tests to find outliers 6

13 What to remove? 0.2 7

14 What to remove? the diameter after i-1 removals Let ν i = the diameter after i removals 0.2 7

15 What to remove? the diameter after i-1 removals Let ν i = the diameter after i removals 5 4 νi ratio removal 7

16 What to remove? 5 4 ratio νi removal 8

17 Signature of each species Signature of x = max log(ν i ) among all i that remove x Pyramimonas parkeae Anomodon attenuatus Nephroselmis pyriformis ratio (ν) Species Signatures: Smilax bona-nox log(3.43) Pyramimonas parkeae log(1.12) Equisetum diffusum log(3.43) Anomodon attenuatus log(1.80) Klebsormidium subtile log(3.43) Nephroselmis pyriformis log(1.12) Smilax bona-nox Klebsormidium subtile Equisetum diffusum removal Optimal removing sets: i= Anomodon attenuatus i= Equisetum diffusum, Anomodon attenuatus i= Equisetum diffusum, Smilax bona-nox, Klebsormidium subtile i= Equisetum diffusum, Smilax bona-nox, Klebsormidium subtile, Anomodon attenuatus i= Equisetum diffusum, Smilax bona-nox, Klebsormidium subtile, Nephroselmis pyriformis, Anomodon attenuatus i= Equisetum diffusum, Smilax bona-nox, Klebsormidium subtile, Nephroselmis pyriformis, Pyramimonas parkeae, Anomodon attenuatus

18 Signature of each species Signature of x = max log(ν i ) among all i that remove x Pyramimonas parkeae Anomodon attenuatus Nephroselmis pyriformis ratio (ν) Species Signatures: Smilax bona-nox log(3.43) Pyramimonas parkeae log(1.12) Equisetum diffusum log(3.43) Anomodon attenuatus log(1.80) Klebsormidium subtile log(3.43) Nephroselmis pyriformis log(1.12) Smilax bona-nox Klebsormidium subtile Equisetum diffusum removal Optimal removing sets: i= Anomodon attenuatus i= Equisetum diffusum, Anomodon attenuatus i= Equisetum diffusum, Smilax bona-nox, Klebsormidium subtile i= Equisetum diffusum, Smilax bona-nox, Klebsormidium subtile, Anomodon attenuatus i= Equisetum diffusum, Smilax bona-nox, Klebsormidium subtile, Nephroselmis pyriformis, Anomodon attenuatus i= Equisetum diffusum, Smilax bona-nox, Klebsormidium subtile, Nephroselmis pyriformis, Pyramimonas parkeae, Anomodon attenuatus

19 Signature of each species Signature of x = max log(ν i ) among all i that remove x Pyramimonas parkeae Anomodon attenuatus Nephroselmis pyriformis ratio (ν) Species Signatures: Smilax bona-nox log(3.43) Pyramimonas parkeae log(1.12) Equisetum diffusum log(3.43) Anomodon attenuatus log(1.80) Klebsormidium subtile log(3.43) Nephroselmis pyriformis log(1.12) Smilax bona-nox Klebsormidium subtile Equisetum diffusum removal Optimal removing sets: i= Anomodon attenuatus i= Equisetum diffusum, Anomodon attenuatus i= Equisetum diffusum, Smilax bona-nox, Klebsormidium subtile i= Equisetum diffusum, Smilax bona-nox, Klebsormidium subtile, Anomodon attenuatus i= Equisetum diffusum, Smilax bona-nox, Klebsormidium subtile, Nephroselmis pyriformis, Anomodon attenuatus i= Equisetum diffusum, Smilax bona-nox, Klebsormidium subtile, Nephroselmis pyriformis, Pyramimonas parkeae, Anomodon attenuatus

20 Signature of each species Signature of x = max log(ν i ) among all i that remove x Pyramimonas parkeae Anomodon attenuatus Nephroselmis pyriformis ratio (ν) Species Signatures: Smilax bona-nox log(3.43) Pyramimonas parkeae log(1.12) Equisetum diffusum log(3.43) Anomodon attenuatus log(1.80) Klebsormidium subtile log(3.43) Nephroselmis pyriformis log(1.12) Smilax bona-nox Klebsormidium subtile Equisetum diffusum removal Optimal removing sets: i= Anomodon attenuatus i= Equisetum diffusum, Anomodon attenuatus i= Equisetum diffusum, Smilax bona-nox, Klebsormidium subtile i= Equisetum diffusum, Smilax bona-nox, Klebsormidium subtile, Anomodon attenuatus i= Equisetum diffusum, Smilax bona-nox, Klebsormidium subtile, Nephroselmis pyriformis, Anomodon attenuatus i= Equisetum diffusum, Smilax bona-nox, Klebsormidium subtile, Nephroselmis pyriformis, Pyramimonas parkeae, Anomodon attenuatus

21 Three statistical tests of TreeShrink The per-gene test: requires only a single tree The all-gene test: requires a collection of gene trees The per-species test: requires a collection of gene trees 10

22 Statistical tests The per-gene test (input: a single tree) Fit a log-normal distribution to the signatures Remove taxa with outlier signatures Outlier: CDF above 1 α fore a given α (false positive tolerance) The all-gene test The per-species test 11

23 Statistical tests The per-gene test The all-gene test (input: a collection of gene trees) Combine all signature values across all genes Compute a kernel density over the empirical distribution Remove the taxa of the outlier signatures Outlier: CDF above 1 α fore a given α The per-species test 12

24 Statistical tests The per-gene test The all-gene test The per-species test (input: a collection of gene trees) Compute a kernel density function for each species over its signatures across genes Remove the taxa of the outlier signatures Outlier: CDF above 1 α for a given α 13

25 Methods The three tests of TreeShrink Alternative filtering methods RootedFiltering: root gene trees and remove taxa X standard deviations more distant to the root than average RogueNarok: rogue taxon removal based; finds unstable nodes based on bootstrap replicates RandomFiltering: randomly choose what to remove. 14

26 Measurements Effects of filtering on taxon occupancy Proportion of data retained for each species Effects of filtering on gene tree discordance Reduction in pairwise MS distance of gene trees on controlled amount of filtering 15

27 Datasets Genes Species Plants phylogenomic datasets Gene number: Species number: Insects Mammals Frogs Metazoa- Cannon Metazoa- Rouse

28 Results: outgroup removal 30 Cannon Frogs Insects Percent of the data removed for α=0.05 for All species 20 Percent removed Percent removed Mammals Plants Rouse Outgroups all gene per gene per species all gene per gene per species all gene per gene per species all gene per gene per species All All taxa Outgroups Outgroups 17 Mammalian dataset

29 Results: outgroup removal 30 Cannon Frogs Insects Percent of the data removed for α=0.05 for All species Outgroups Percent removed Mammals Plants Rouse all gene per gene per species all gene per gene per species all gene per gene per species All taxa Outgroups 17

30 Impact of filtering on discordance Random_pruning 20 TreeShrink_all_gene TreeShrink_per_gene TreeShrink_per_species Delta MS Proportion of taxa retained Plant dataset 18

31 TreeShrink versus alternative methods (discordance) (b) Plants Random_pruning Delta MS Delta MS Proportion of taxa retained Random_pruning RogueNarok Rooted_pruning TreeShrink RogueNarok Rooted_pruning TreeShrink

32 TreeShrink versus alternative methods (b) Plants (discordance) (a) Insects Random_pruning Delta MS Delta MS Delta MS Proportion of taxa retained 20 RogueNarok Random_pruning RogueNarok Rooted_pruning Rooted_pruning TreeShrink TreeShrink

33 Results: TreeShrink versus Alternative Methods (e) Mammals (f) Frogs Random_pruning RogueNarok 50 Rooted_pruning TreeShrink Delta MS 5 Delta MS Proportion of taxa retained Proportion of taxa retained 21

34 TreeShrink versus alternative methods (occupancy) Arabidopsis_thaliana Eschscholzia_californica Amborella_trichopoda SOriginalhum_bicolor Catharanthus_roseus Brachypodium_distachyon Oryza_sativa Pinus_taeda Nuphar_advena Liriodendron_tulipifera Vitis_vinifera Carica_papaya Persea_americana Diospyros_malabarica Prumnopitys_andina Allamanda_cathartica Saruma_henryi Ipomoea_purpurea Acorus_americanus Hibiscus_cannabinus Boehmeria_nivea Tanacetum_parthenium Dioscorea_villosa Podophyllum_peltatum Kochia_scoparia Sabal_bermudana Sciadopitys_verticillata Yucca_filamentosa Sarcandra_glabra Hedwigia_ciliata Rosmarinus_officinalis Zea_mays Smilax_bona nox Cedrus_libani Rhynchostegium_serrulatum Ricciocarpos_natans Cunninghamia_lanceolata Sphagnum_lescurii Ginkgo_biloba Leucodon_brachypus Taxus_baccata Gnetum_montanum Thuidium_delicatulum Marchantia_polymorpha Selaginella_moellendorffii_genome Juniperus_scopulorum Anomodon_attenuatus Larrea_tridentata Pteridium_aquilinum Ceratodon_purpureus Polytrichum_commune Pseudolycopodiella_caroliniana Cycas_micholitzii Inula_helenium Sphaerocarpos_texanus Equisetum_diffusum Cylindrocystis_cushleckae Ophioglossum_petiolatum Rosulabryum_cf_capillare Dendrolycopodium_obscurum Kadsura_heteroclita Huperzia_squarrosa Nothoceros_aenigmaticus Metzgeria_crassipilis Coleochaete_scutata Bryum_argenteum Cylindrocystis_brebissonii Houttuynia_cordata Bazzania_trilobata Coleochaete_irregularis Psilotum_nudum Selaginella_moellendorffii_1kp Mesotaenium_endlicherianum Entransia_fimbriata Aquilegia_formosa Medicago_truncatula Mougeotia_sp Angiopteris_evecta Physcomitrella_patens Nothoceros_vincentianus Populus_trichocarpa Marchantia_emarginata Alsophila_spinulosa Colchicum_autumnale Zamia_vazquezii Chlorokybus_atmophyticus Roya_obtusa Klebsormidium_subtile Ephedra_sinica Netrium_digitus Pyramimonas_parkeae Chaetosphaeridium_globosum Nephroselmis_pyriformis Cosmarium_ochthodes Spirogyra_sp Monomastix_opisthostigma Chara_vulgaris Spirotaenia_minuta Mesostigma_viride Cycas_rumphii Penium_margaritaceum Welwitschia_mirabilis Uronema_sp Occupancy (# genes) Original RogueNarok Rooted_pruning TreeShrink Original RogueNarok Rooted_pruning TreeShrink 22

35 {} ab h a d f g e c on-diameter b removed

36 remove b {} ab h a {b} ac d f g e c on-diameter b removed

37 remove b {} ab remove a h a {b} ac {a} db d f g e c on-diameter b removed

38 remove b {} ab remove a h a remove c {b} ac {a} db d {b,c} ae f g e c on-diameter b removed

39 remove b {} ab remove a h a remove c {b} ac remove a {a} db remove b d {b,c} ae {a,b} dc f g e c on-diameter b removed

40 remove b {} ab remove a h a remove c {b} ac remove a {a} db remove b remove d d {b,c} ae {a,b} dc {a,d} fb f g e c on-diameter b removed

41 remove b {} ab remove a h a remove c {b} ac remove a {a} db remove b remove d d remove e {b,c} ae {a,b} dc remove a remove c remove d remove b {a,d} fb f remove f {b,c,e} ag {b,c,a} de {a,b,d} fc {a,d,f} hb g e c on-diameter b removed

42 Solution space i = 0 remove b {} ab remove a i = 1 {b} ac {a} db k = 3 remove c remove a remove b remove d i = 2 remove e {b,c} ae {a,b} dc remove a remove c remove d remove b {a,d} fb remove f i = 3 {b,c,e} ag {b,c,a} de {a,b,d} fc {a,d,f} hb

43 The TreeShrink tool is publicly available Uyen Mai 31

44 A single HIV tree 648 HIV-1 partial pol sequences 639 subtype B 7 non-subtype B 2 unassigned TreeShrink RogueNarok TreeShrink and RogueNarok Unassigned Subtype 32

45 Results: TreeShrink versus Alternative Methods (c) Metazoa - Cannon (d) Metazoa - Rouse Random_pruning RogueNarok Delta MS Delta MS 5.0 Rooted_pruning TreeShrink Proportion of taxa retained Proportion of taxa retained 33

46 Results: The 3 Tests of TreeShrink (c) Metazoa - Cannon (d) Metazoa - Rouse 9 40 Delta MS Delta MS Proportion of taxa retained Proportion of taxa retained 34

47 Results: The 3 Tests of TreeShrink (e) Mammals (f) Frogs Delta MS Delta MS Proportion of taxa retained Proportion of taxa retained 35

48 Can be done in other ways too (e.g., O(n.k+k 2 logk)), but harder to implement

49 Can be just outgroups Sphagnum lesc Chicken Lesser Hedgehog Tenrec Platypus Wallaby Opossum Tarsier Guinea Pig Kangaroo Rat Squirrel Galagos Mouse Lemur Tree Shrew Hyrax Elephant Pika Rabbit Sloth Armadillos Rat Mouse Horse Megabat Macaque Chimp Human Orangutan Gorilla Marmoset Alpaca Dolphin Cow Microbat Cat Dog Shrew Hedgehog 0.05 b) Hard case: a gene tree in the mammalian dataset 37 Pig

Orthologous loci for phylogenomics from raw NGS data

Orthologous loci for phylogenomics from raw NS data Rachel Schwartz The Biodesign Institute Arizona State University Rachel.Schwartz@asu.edu May 2, 205 Big data for phylogenetics Phylogenomics requires