Pattern matching in highly similar sequences

Transcription

1 Pattern matching in highly similar sequences Thierry Lecroq joint work with N. Ben Nsira et É. Prieur-Gaston Laboratoire d Informatique, du Traitement de l Information et des Systèmes (LITIS EA4108) Université de Rouen Normandie, France Mathematics / Computer Science Day 12 October 2018 Rouen France

2 Outline 1 Bioinformatics 2 Examples of projects 3 Search in highly similar sequences Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

3 Outline 1 Bioinformatics 2 Examples of projects 3 Search in highly similar sequences Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

4 Bioinformaticians Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

5 Bioinformatics (in silico biology) Extrait de Wikipédia, mai 2015 La bio-informatique est constituée par l ensemble des concepts et des techniques nécessaires à l interprétation informatique de l information biologique. Plusieurs champs d application ou sous-disciplines de la bio-informatique se sont constitués : La bio-informatique des séquences, [...] La bio-informatique structurale, [...] La bio informatique des réseaux, [...] La bio-informatique statistique et la bio-informatique des populations. Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

6 Genetics Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

7 Size of genomes (Mb) Escherichia coli (bacteria) 4,6 Saccharomyces cerevisiae (yeast) 13 C. elegans (worm) 100 Arabidopsis thaliana (plant) 125 Drosophila melanogaster (fly) 180 Rice 400 Homo sapiens 3300 Fern Amoeba dubia (amoeba) Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

8 From Biology to Computer Science Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

9 Sequencing before 2005: slow and expensive since 2005: NGS, faster and faster, cheaper and cheaper Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

10 Data deluge Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

11 Next Generation Sequencing (NGS) Sequencer Reads millions of short (length 150) fragments called reads 2 types of projects : alignment (mapping) on a reference genome assembly for building a genome Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

12 Mapping Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

13 Resequencing reference genome Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

14 Resequencing sequenced genome reads Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

15 Single Nucleotide Polymorphism (SNP) or Single Nucleotide Variant (SNV) reference genome sequenced genome reads x x x x x... Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

16 Sequencing errors Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

17 Sequencing errors reference genome sequenced genome reads x... Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

18 Outline 1 Bioinformatics 2 Examples of projects 3 Search in highly similar sequences Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

19 Book of exercises (with solutions) Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

20 Example Zimin Z 1 = x 1 and ( k > 1) Z k = Z k 1 x k Z k 1 Example: abacabadabacaba Question Give a linear time algorithm for computing all Zimin type prefixes of a given string Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

21 New location Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

22 mirabel bioinfo.univ-rouen.fr/ mirabel Aggregation of software for prediction of microrna targets Collaboration with INSERM U982 DC2N and LMRS UMR 6085 CNRS A. Quillet, C. Saad, G. Ferry, Y. Anouar, N Vergne, T. Lecroq and C. Dubessy Improving bioinformatics prediction of microrna targets by ranks aggregation biorxiv , 2017 Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

23 Phaeodactylum tricornutum Differential transcriptomic analysis (RNA-seq) of an algae with 3 phenotypes Collaboration with GlycoMEV EA 4358 and LMRS UMR 6085 C. Ovide, M.-C. Kiefer-Meyer, C. Be rard, N. Vergne, T. Lecroq, C. Plasson, C. Burel, S. Bernard, A. Driouich, P. Lerouge, I. Tournier, H. Dauchel and M. Bardor Comparative in depth RNA sequencing of P. tricornutum s morphotypes reveals specific features of the oval morphotype Scientific Reports, 8, Article number: 14340, 2018 Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

24 Third generation read correction Hybrid correction Self-correction tool for evaluate correctors PhD Thesis of Pierre Morisse (grant from Univ. Rouen Normandie 2016) P. Morisse, T. Lecroq and A. Lefebvre Hybrid correction of highly noisy long reads using a variable-order de Bruijn graph Bioinformatics, 2018, accepted Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

25 UMI (Unique Marker Identifier) Conception of new algorithms for processing UMIs from NGS data PhD Thesis of Ahmad Abdel Sater (grant from Normandie Region 2018) Collaboration with INSERM U1245 at CRLCC Henri Becquerel P.-J. Viailly, E. Bohers, M. Viennot, A. Abdel Sater, V. Marchand, P. Ruminy, H. Dauchel, T. Lecroq, M. Becker, P. Etancelin, H. Tilly, P. Vera and F. Jardin I-LowVarFreq: improving low-frequency variant detection using a new UMI-based variant calling approach for paired-end sequencing NGS libraries Proceedings of JOBIM, Marseilles, France, 2018 Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

26 Mass spectrometry data Study of Pemphigus Collaboration with INSERM U1234 Reconstruction of peptides H T W Y Q K K P N A A P R Q Q K P N A A P R L L L Y M. Petit, M.-L. Walet-Balieu, P. Chan Tchi Song, L. Drouot, C. Burel, M. Maho-Vaillant, T. Lecroq, P. Cosette, D. Vaudry, O. Boyer, M. Bardor, P. Joly, S. Calbo Longitudinal study of anti-dsg3 IgG repertoire by proteomics in Pemphigus following Rituximab treatment Poster, JNRb, Rouen, France, 2018 Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

27 Outline 1 Bioinformatics 2 Examples of projects 3 Search in highly similar sequences Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

28 Pattern matching Find one(all the) position(s) of a pattern of length m in a sequence of length n: with index O(m) without index O(n) Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

29 Suffix Trie a t $ g y = a t a t g a t $ a $ t g a 0 atatgat$ a $ 6 1 tatgat$ g t a t 2 atgat$ 5 t a g t $ 3 tgat$ 4 4 gat$ g t a $ 3 5 at$ a $ t 6 t$ 2 7 $ t $ 1 $ 0 7 Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

30 Suffix Trie a t $ g y = a t a t g a t $ a $ t g a 0 atatgat$ a $ 6 1 tatgat$ g t a t 2 atgat$ 5 t a g t $ 3 tgat$ 4 4 gat$ g t a $ 3 5 at$ a $ t 6 t$ 2 7 $ t $ 1 $ 0 7 Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

31 Suffix Trie a t $ g y = a t a t g a t $ a $ t g a 0 atatgat$ a $ 6 1 tatgat$ g t a t 2 atgat$ 5 t a g t $ 3 tgat$ 4 4 gat$ g t a $ 3 5 at$ a $ t 6 t$ 2 7 $ t $ 1 $ 0 7 Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

32 Suffix Tree a t $ g a $ t g a a $ 6 g t a t 5 t a g t $ 4 g t a $ 3 a $ t 2 t $ 1 $ $ 5 gat$ atgat$ at 6 gat$ gat$ atgat$ t $ $ 7 Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

33 Suffix Tree a t $ g a $ t g a a $ 6 g t a t 5 t a g t $ 4 g t a $ 3 a $ t 2 t $ 1 $ $ 5 gat$ atgat$ at 6 gat$ gat$ atgat$ t $ $ 7 Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

34 Suffix Tree a t a t g a t $ at t $ gat$ atgat$$ atgat$ $ gat$ gat$ (0,2) (1,1) (7,1) (4,4) (7,1) (7,1) (2,6) (2,6) (4,4) (4,4) [Weiner 73,McCreight 76,Ukkonen 92,Farach 97] Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

35 Suffix Tree a t a t g a t $ at t $ gat$ atgat$$ atgat$ $ gat$ gat$ (0,2) (1,1) (7,1) (4,4) (7,1) (7,1) (2,6) (2,6) (4,4) (4,4) [Weiner 73,McCreight 76,Ukkonen 92,Farach 97] Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

36 Suffix Tree a t a t g a t $ at t $ gat$ atgat$$ atgat$ $ gat$ gat$ (0,2) (1,1) (7,1) (4,4) (7,1) (7,1) (2,6) (2,6) (4,4) (4,4) [Weiner 73,McCreight 76,Ukkonen 92,Farach 97] Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

37 Suffix Tree a t a t g a t $ at t $ gat$ atgat$$ atgat$ $ gat$ gat$ (0,2) (1,1) (7,1) (4,4) (7,1) (7,1) (2,6) (2,6) (4,4) (4,4) [Weiner 73,McCreight 76,Ukkonen 92,Farach 97] Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

38 Suffix Tree a t a t g a t $ 0 (0,2) (1,1) (7,1) (4,4) (7,1) (7,1) (2,6) (2,6) (4,4) (4,4) ta is a factor of y tt is not at occurs 3 times at positions 0, 2 and 5 t occurs 3 times at positions 1, 3 and 6 the length of the longest common prefix of suffixes starting at positions 2 and 5 is 2 Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

39 Suffix Tree a t a t g a t $ 0 (0,2) (1,1) (7,1) (4,4) (7,1) (7,1) (2,6) (2,6) (4,4) (4,4) ta is a factor of y tt is not at occurs 3 times at positions 0, 2 and 5 t occurs 3 times at positions 1, 3 and 6 the length of the longest common prefix of suffixes starting at positions 2 and 5 is 2 Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

40 Suffix Tree a t a t g a t $ 0 (0,2) (1,1) (7,1) (4,4) (7,1) (7,1) (2,6) (2,6) (4,4) (4,4) ta is a factor of y tt is not at occurs 3 times at positions 0, 2 and 5 t occurs 3 times at positions 1, 3 and 6 the length of the longest common prefix of suffixes starting at positions 2 and 5 is 2 Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

41 Suffix Tree a t a t g a t $ 0 (0,2) (1,1) (7,1) (4,4) (7,1) (7,1) (2,6) (2,6) (4,4) (4,4) ta is a factor of y tt is not at occurs 3 times at positions 0, 2 and 5 t occurs 3 times at positions 1, 3 and 6 the length of the longest common prefix of suffixes starting at positions 2 and 5 is 2 Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

42 Suffix Tree a t a t g a t $ 0 (0,2) (1,1) (7,1) (4,4) (7,1) (7,1) (2,6) (2,6) (4,4) (4,4) ta is a factor of y tt is not at occurs 3 times at positions 0, 2 and 5 t occurs 3 times at positions 1, 3 and 6 the length of the longest common prefix of suffixes starting at positions 2 and 5 is 2 Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

43 Suffix Tree a t a t g a t $ 0 (0,2) (1,1) (7,1) (4,4) (7,1) (7,1) (2,6) (2,6) (4,4) (4,4) ta is a factor of y tt is not at occurs 3 times at positions 0, 2 and 5 t occurs 3 times at positions 1, 3 and 6 the length of the longest common prefix of suffixes starting at positions 2 and 5 is 2 Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

44 Complexities Algorithms for building suffix trees are: on-line linear time linear space Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

45 Highly similar sequences r sequences y 0 y 1 y 2 y 3 A T G C T A G C A A G A T A C A G A T G C T A G C A A C A T A C A G A T G C G A G C A A G A T A C A G A T G C T A G C A A C A T A C A T Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

46 Highly similar sequences r sequences y 0 y 1 y 2 y A T G C T A G C A A G A T A C A G A T G C T A G C A A C A T A C A G A T G C G A G C A A G A T A C A G A T G C T A G C A A C A T A C A T y A T G C {G, T} A G C A A {C, G} A T A C A {G, T} Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

47 Highly similar sequences r sequences y 0 y 1 y 2 y A T G C T A G C A A G A T A C A G A T G C T A G C A A C A T A C A G A T G C G A G C A A G A T A C A G A T G C T A G C A A C A T A C A T y A T G C {G, T} A G C A A {C, G} A T A C A {G, T} G A G C A A C Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

48 Highly similar sequences r sequences y 0 y 1 y 2 y 3 A T G C T A G C A A G A T A C A G A T G C T A G C A A C A T A C A G A T G C G A G C A A G A T A C A G A T G C T A G C A A C A T A C A T y 0 et Z = (({2}, 4, G), ({1, 3}, 10, C), ({3}), 16, T) Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

49 Sliding window n y x m y x y x Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

50 Knuth-Morris-Pratt algorithm (1977) y j u b x u a z c Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

51 Boyer-Moore Algorithm comparisons x a z y b z x c z Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

52 For highly similar sequences Hamming distance For u, v A such that u = v : Ham(u, v) = {i u[i] v[i]} Longest Common Extension For x A and 0 i j x 1: LCE k x(i, j) = max{l Ham(x[i.. i + l 1], x[j.. j + l 1]) k} Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

53 Kangaroo jumps LCE k x(i, j) can be computed in O(k) time after O(n) preprocessing time Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

54 Kangaroo jumps i j LCE k x(i, j) can be computed in O(k) time after O(n) preprocessing time Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

55 Kangaroo jumps i j LCE k x(i, j) can be computed in O(k) time after O(n) preprocessing time Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

56 Kangaroo jumps i j 1 LCE k x(i, j) can be computed in O(k) time after O(n) preprocessing time Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

57 Kangaroo jumps i j 1 2 LCE k x(i, j) can be computed in O(k) time after O(n) preprocessing time Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

58 Kangaroo jumps i j LCE k x(i, j) can be computed in O(k) time after O(n) preprocessing time Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

59 Kangaroo jumps i j LCE k x(i, j) can be computed in O(k) time after O(n) preprocessing time Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

60 References Restriction: 1 variation on a window of size m N. Ben Nsira, T. Lecroq and M. Elloumi A fast Boyer-Moore type pattern matching algorithm for highly similar sequences International Journal of Data Mining and Bioinformatics 13(3) (2015) N. Ben Nsira, T. Lecroq and M. Elloumi On-line String Matching in Highly Similar DNA Sequences Mathematics in Computer Science 11(2) (2017) Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

61 2 variants searching for a finite set of patterns relaxing the restriction from 1 to k variations on a window of size m Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

62 Single pattern with at most k variations Applying the Landau-Vishkin algorithm as a filter Searching with k mismatches in O(kn) When Ham(x, y 0 [j.. j + l 1]) = l k l = 0: an exact occurrence of the pattern has been found in y 0 and all the other sequence that do not have a variation comparing to y 0 between position j and position j + m 1 both included. l > 0: let W = {i 0,..., i l 1 } be the set of the l positions such that y 0 [j + i p ] x[i p ] with 0 p < l. Then x occurs exactly in y h if: (G, j + ip, x[i p ]) Z with g G for all 0 p < l; (G, h, c) Z such that h W. Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

63 Single pattern with at most k variations r = 2 and k = y 0 A C C T A C G A C T A x C T A C T T x C T A C T T y 1 A C C T A C T A C T T Our solution runs in time O(knr) Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

64 Single pattern with at most k variations r = 2 and k = y 0 A C C T A C G A C T A x C T A C T T x C T A C T T y 1 A C C T A C T A C T T Our solution runs in time O(knr) Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

65 Multiple patterns with at most 1 variation Build a classical trie of the patterns Scan the highly similar sequences with at most 2 active states Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

66 Multiple patterns with at most 1 variation X = {ACGA, ACTA, CTA} and r = 2 séquences Σ \ {A, C} A 0 1 C 2 G 3 A 4 {ACGA} T C 5 A 6 {ACTA, CTA} 7 T 8 A 9 {CTA} A C C T A C G A C T A y 1 T T active states Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

67 Multiple patterns with at most 1 variation Our solution runs in time O(n) for the searching phase and in time O(s) for the preprocessing phase where s = x for all x X Experiments on similar sequences of different lengths with patterns of length EDSM LVsim ACsim 0.6 Time(s) x x x x10 6 length Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

68 References N. Ben Nsira, T. Lecroq and É. Prieur-Gaston Practical fast exact pattern matching algorithm for highly similar sequences International Conference on Bioinformatics and Biomedicine (BIBM), 2018, Madrid, submitted Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

69 Perspectives Adapt other pattern matching techniques Relax the restrictions Adaptive analysis Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

70 Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50

71 Thank you for your attention! Thierry Lecroq (LITIS, URN) Matching in similar sequences 12th October / 50