Dynamic Programming: Edit Distance

Transcription

1 Dynamic Programming: Edit Distance Bioinformatics: Issues and Algorithms SE Fall 2007 Lecture 10 Lopresti Fall 2007 Lecture

2 Outline Setting the Stage DNA Sequence omparison: First Successes he hange Problem he Manhattan ourist Problem he Longest ommon Subsequence Problem Sequence Alignment he Edit Distance Problem Lopresti Fall 2007 Lecture

3 Sequence comparison and alignment onsider: AAA AAA versus AA AAA hat the two DNA fragments on the left somehow seem more similar than the two on the right could be significant. Question: how can we measure sequence similarity? Lopresti Fall 2007 Lecture

4 Motivation Why is this important? iven a new DNA sequence, one of the first things a biologist will want to do is search databases of known sequences to see if anyone has recorded something similar. (As we've seen, genetic sequences are long and the databases are enormous, so efficiency will be an issue.) Sequence similarity can provide clues about function. Similarity can provide clues about evolutionary relationships. Many other problems from computational biology incorporate some notion of sequence similarity as a basic premise. Lopresti Fall 2007 Lecture

5 Sequence concepts A sequence is a linear string of symbols over a finite alphabet. (Note: string and sequence are often used synonymously.) Some basic sequence concepts: length empty string subsequence supersequence number of symbols in s (written s ). sequence of length 0 (written ε). sequence that can be obtained from s by removing some symbols (A is a subsequence of A). if t is a subsequence of s, then s is a supersequence of t. Lopresti Fall 2007 Lecture

6 Sequence concepts More basic sequence concepts: substring sequence of consecutive symbols appearing in s (A is not a substring of A, but is). (Observation: every substring is a subsequence of the string in question, but not vice versa.) superstring interval if t is a substring of s, then s is a superstring of t. set of consecutive indices of a string, e.g., [2..4] refers to 2nd through 4th symbols of string s, or s[2]s[3]s[4]. Lopresti Fall 2007 Lecture

7 Sequence concepts And lastly: prefix substring of s of the form s[1..j] where 0 j s (when j = 0, prefix is the empty string). suffix substring of s of the form s[i.. s ] where 1 i s + 1 (when i = s + 1, suffix is the empty string). A is a prefix (and substring and subsequence) of AA. A is a suffix (and substring and subsequence) of AA. Lopresti Fall 2007 Lecture

8 Sequences he concept of a sequence is extremely broad. In this course, we are concerned with genetic sequences. However, there are other important kinds of sequence data: ASII text, speech, handwriting (pen-strokes). Likewise, the same algorithmic techniques turn up again and again, often under different names: approximate string matching, edit (or evolutionary) distance, dynamic time warping. Lopresti Fall 2007 Lecture

9 Sequence comparison: a false start An obvious idea that comes to mind is to line up each symbol and count the number that don't match: A A A A A = 2 As we know, this is Hamming distance and forms the basis for most error correcting codes, as well as the motif-finding problem we saw earlier. But it doesn't work for the kinds of sequences we care about:? = 6 Just one missing symbol at the start of the second sequence leads to a large distance. Lopresti Fall 2007 Lecture

10 Sequence manipulation at the genetic level enomes aren't static sequence comparison must account for this. Lopresti Fall 2007 Lecture

11 he human factor In addition, errors can arise during the sequencing process: "...the error rate is generally less than 1% over the first 650 bases and then rises significantly over the remaining sequence. A hard-to-read gel (arrow marks location where bands of similar intensity appear in two different lanes): A A? A A? A A? Lopresti Fall 2007 Lecture

12 Sequence alignment As we have seen, the two sequences we wish to compare may have different lengths. As a result, we need to allow for deletions and insertions. he notion of an alignment helps us visualize this: A A? - Alignment permits us to incorporate spaces (represented by a dash) in one or both sequences to make them the same length. Lopresti Fall 2007 Lecture

13 Lopresti Fall 2007 Lecture Sequence alignment No this one has 5! How can we find the best alignment? A his alignment has 6 mismatches. Is it the best possible? A

14 DNA sequence comparison First successes: Finding sequence similarities with genes of known function is a common approach to infer function of a newlysequenced gene. In 1984, Russell Doolittle and colleagues found similarities between cancer-causing gene and normal growth factor (PDF) gene. Lopresti Fall 2007 Lecture

15 ystic Fibrosis ystic fibrosis (F) is chronic and often fatal genetic disease of body's mucus glands (abnormally high level of mucus). Primarily affects respiratory systems in children. Mucus is slimy material that coats many epithelial surfaces and is secreted into fluids such as saliva. Inheritance of ystic Fibrosis: In early 1980's, biologists hypothesized that F is an autosomal recessive disorder caused by mutations in a gene that remained unknown until Heterozygous carriers are asymptomatic. Must be homozygously recessive in this gene in order to be diagnosed with F. Lopresti Fall 2007 Lecture

16 ystic Fibrosis: finding the gene Lopresti Fall 2007 Lecture

17 ystic Fibrosis Finding Similarities between ystic Fibrosis ene and AP binding proteins: AP binding proteins are present on cell membrane and act as transport channel. In 1989, biologists found similarity between cystic fibrosis gene and AP binding proteins. A plausible function for cystic fibrosis gene, given that F involves sweet secretion with abnormally high sodium level. Lopresti Fall 2007 Lecture

18 ystic Fibrosis: mutation analysis If a high percentage of cystic fibrosis (F) patients have a certain mutation in the gene, while unaffected patients don t, that could be an indicator of a mutation that is related to F. Indeed, a certain mutation was found in 70% of F patients; this is convincing evidence of a predominant genetic diagnostics marker for F. Lopresti Fall 2007 Lecture

19 ystic Fibrosis and the FR protein FR (ystic Fibrosis ransmembrane conductance Regulator) protein acts in cell membrane of epithelial cells that secrete mucus. hese cells line airways of nose, lungs, stomach wall, etc. FR protein (1480 amino acids) regulates a chloride ion channel: adjusts wateriness of fluids secreted by cell. F sufferers are missing a single amino acid in their FR. Mucus ends up being too thick, affecting many organs. Lopresti Fall 2007 Lecture

20 Bring in the Bioinformaticians ene similarities between two genes with known and unknown function alert biologists to some possibilities. omputing a similarity score between two genes tells how likely it is that they have similar functions. Dynamic programming is a technique for revealing similarities between genes. he hange Problem is a good problem to introduce idea of dynamic programming. Lopresti Fall 2007 Lecture

21 he hange Problem You hand cashier $20 for a $19.23 bill. Your change could be: 3 quarters + 2 pennies 77 pennies etc. he hange Problem. onvert some amount of money M into given denominations, using fewest possible number of coins. Input: An amount of money M, and an array of d denominations, c = (c 1, c 2,, c d ), with (c 1 > c 2 > > c d ). Output: A list of d integers, i 1, i 2,, i d, such that c 1 i 1 + c 2 i c d i d = M and i 1 + i i d is minimal. Lopresti Fall 2007 Lecture

22 he hange Problem: an example iven denominations 1, 3, and 5, what is minimum number of coins needed to make change for a given value? Value Min # of coins Only one coin needed to make change for values 1, 3, and 5. Lopresti Fall 2007 Lecture

23 he hange Problem: an example iven denominations 1, 3, and 5, what is minimum number of coins needed to make change for a given value? Value Min # of coins However, two coins needed to make change for values 2, 4, 6, 8, and 10. Lopresti Fall 2007 Lecture

24 he hange Problem: an example iven denominations 1, 3, and 5, what is minimum number of coins needed to make change for a given value? Value Min # of coins Lastly, three coins needed to make change for values 7 and 9. Lopresti Fall 2007 Lecture

25 he hange Problem: recurrence his example is expressed by the following recurrence relation: minnumoins(m) = min of minnumoins(m-1) + 1 minnumoins(m-3) + 1 minnumoins(m-5) + 1 iven denominations c 1, c 2,, c d, recurrence relation is: minnumoins(m) = min of minnumoins(m-c 1 ) + 1 minnumoins(m-c 2 ) + 1 minnumoins(m-c d ) + 1 Lopresti Fall 2007 Lecture

26 he hange Problem: a recursive algorithm Recursivehange(M, c, d) if M = 0 return 0 bestnumoins infinity for i 1 to d if M c i numoins Recursivehange(M c i, c, d) if numoins + 1 < bestnumoins bestnumoins numoins + 1 return bestnumoins Lopresti Fall 2007 Lecture

27 he hange Problem: a recursive algorithm he Recursivehange algorithm works, but it is not efficient: It recalculates optimal coin combination for a given amount of money repeatedly. E.g., if M = 77 and c = (1,3,7), then optimal combination of coins for 70 cents is computed on 9 different occasions! Lopresti Fall 2007 Lecture

28 he hange Problem: we can do better Rather than re-compute a given value more than once: Save results of each computation for 0 to M. his way, we can do a reference call to find an already computed value, instead of re-computing each time. Running time is M x d, where M is the value of money and d is the number of denominations. Lopresti Fall 2007 Lecture

29 he hange Problem: Dynamic Programming DPhange(M, c, d) bestnumoins 0 0 for m 1 to M bestnumoins m infinity for i 1 to d if m c i if bestnumoins m ci + 1 < bestnumoins m return bestnumoins M bestnumoins m bestnumoins m ci + 1 Lopresti Fall 2007 Lecture

30 DPhange example c = (1,3,7) M = 9 Lopresti Fall 2007 Lecture

31 he Manhattan ourist Problem (MP) Imagine seeking a path (from source to sink) to travel (only eastward and southward) with the most number of attractions (*) in the Manhattan grid. Source * * * * * * * * * * * * Sink Lopresti Fall 2007 Lecture

32 he Manhattan ourist Problem (MP) Imagine seeking a path (from source to sink) to travel (only eastward and southward) with the most number of attractions (*) in the Manhattan grid. Source * * * * * * * * * * * * Sink Lopresti Fall 2007 Lecture

33 he Manhattan ourist Problem (MP) he Manhattan ourist Problem. Find the longest path in a weighted grid. Input: A weighted grid with two distinct vertices, one labeled source and the other labeled sink. Output: A longest path in from source to sink. Lopresti Fall 2007 Lecture

34 MP: an example source j coordinate i coordinate sink Lopresti Fall 2007 Lecture

35 MP: greedy is not optimal source promising start, but leads to bad choices! sink Lopresti Fall 2007 Lecture

36 MP: a simple recursive program M(n, m) if n = 0 and m = 0 return 0 if n > 0 x M(n-1, m) + length of edge from (n-1, m) to (n, m) else x 0 if m > 0 y M(n, m-1) + length of edge from (n, m-1) to (n, m) else y 0 return max(x, y) What's wrong with this approach? Lopresti Fall 2007 Lecture

37 MP: Dynamic Programming source j 0 1 i S 0,1 = S 1,0 = 5 alculate optimal path score for each vertex in graph. Each vertex s score is maximum of prior vertices' scores plus weight of respective edges in between. Lopresti Fall 2007 Lecture

38 MP: Dynamic Programming (continued) source j i S 0,2 = S 1,1 = S 2,0 = 8 Lopresti Fall 2007 Lecture

39 MP: Dynamic Programming (continued) source j i S 3,0 = S 1,2 = S 3,0 = 8 9 S 2,1 = 9 Lopresti Fall 2007 Lecture

40 MP: Dynamic Programming (continued) source j i reedy algorithm fails! S 1,3 = 8 12 S 2,2 = S 3,1 = 9 Lopresti Fall 2007 Lecture

41 MP: Dynamic Programming (continued) source j i Done! S3,3 = 16 Lopresti Fall 2007 Lecture

42 MP: recurrence omputing score for a point (i, j) by recurrence relation: s i, j = max s i-1, j + weight of the edge between (i-1, j) and (i, j) s i, j-1 + weight of the edge between (i, j-1) and (i, j) he running time is n x m for a n-by-m grid, where n is the number of rows and m is the number of columns. Lopresti Fall 2007 Lecture

43 But Manhattan is not a perfect grid... A 2 A 3 What about diagonals? A 1 B Score at point B is given by: s B = max of s A1 + weight of the edge (A 1, B) s A2 + weight of the edge (A 2, B) s A3 + weight of the edge (A 3, B) Lopresti Fall 2007 Lecture

44 But Manhattan is not a perfect grid... omputing score s x for point x is given by recurrence relation: s x = max s y + weight of edge (y, x) where of y Predecessors(x) Predecessors(x) = set of vertices that have edges leading to x Running time for a graph (V, E) is O(E) since each edge is evaluated once Note: V is set of all vertices and E is set of all edges. Lopresti Fall 2007 Lecture

45 Alignment: two-row representation iven 2 DNA sequences v and w: v : w : A A A A m = 7 n = 6 Alignment : 2 * k matrix ( k > m, n ) letters of v letters of w A A A - A - 4 matches 2 insertions 3 deletions Lopresti Fall 2007 Lecture

46 Aligning DNA sequences v = AA w = AA n = 8 m = matches mismatches deletions insertions match mismatch V A A W A A indels insertion deletion Lopresti Fall 2007 Lecture

47 he Longest ommon Subsequence Problem (LS) he Longest ommon Subsequence Problem. Find the longest common subsequence of two sequences. Input: wo sequences: v = v 1 v 2 v m and w = w 1 w 2 w n Output: A sequence of positions in v : 1 i 1 < i 2 < < i t m and a sequence of positions in w : 1 j 1 < j 2 < < j t n such that symbol i k of v matches j k of w and t is maximal. Lopresti Fall 2007 Lecture

48 LS example i coords: elements of v elements of w j coords: 0 A - - A - A - A (0,0) (1,0) (2,1) (2,2) (3,3) (3,4) (4,5) (5,5) (6,6) (7,6) (8,7) Matches shown in red positions in v: positions in w: 2 < 3 < 4 < 6 < 8 1 < 3 < 5 < 6 < 7 Every common subsequence is a path in 2-D grid. Lopresti Fall 2007 Lecture

49 LS Dynamic Programming Find LS of two strings. Input: a weighted graph with two distinct vertices, one labeled source and one labeled sink. Output: a longest path in from source to sink. Solve using an MP graph with diagonals replaced by +1 edges. Lopresti Fall 2007 Lecture

50 LS Problem as Manhattan ourist Problem i 0 j A A A A Lopresti Fall 2007 Lecture

51 MP raph for LS Problem i 0 j A A A A Lopresti Fall 2007 Lecture

52 MP raph for LS Problem j A A i Every path is common subsequence. A A Diagonal edge adds extra element to subsequence. 7 LS Problem: find path with maximum diagonal edges. Lopresti Fall 2007 Lecture

53 omputing LS Let v i = prefix of v of length i : v 1 v i and w j = prefix of w of length j : w 1 w j he length of LS(v i, w j ) is computed by: s i-1, j s i, j = max s i, j-1 s i-1, j if v i = w j i-1,j -1 i,j -1 i-1,j i,j Lopresti Fall 2007 Lecture

54 LS Alignments Every path in grid corresponds to an alignment: W A V A V = A - W = A Lopresti Fall 2007 Lecture

55 Edit distance In 1966, Levenshtein introduced edit distance between two strings as minimum number of elementary operations (insertions, deletions, and substitutions) needed to transform one string into the other. d(v, w) = MIN number of elementary operations needed to transform v w. Keep in mind that LS computation uses a MAX. Lopresti Fall 2007 Lecture

56 he Edit Distance Problem he Edit Distance Problem. iven two sequences, find the optimal series of deletions, insertions, and substitutions to transform one into the other. Input: wo sequences: v = v 1 v 2 v m and w = w 1 w 2 w n Output: An optimal series of basic editing operations: e 1, e 2,..., e t such then, when applied to one of the sequences, say v, it is transformed into the other sequence, w. Here optimal can mean any of a number of things, including fewest or lowest- / highest-cost. Lopresti Fall 2007 Lecture

57 Edit distance vs. Hamming distance Hamming distance always compares i -th letter of v with i -th letter of w v = AAAA w = AAAA Hamming distance: d(v, w) = 8 Just one shift Make it all line up Edit distance may compare i -th letter of v with j -th letter of w v = -AAAA w = AAAA Edit distance: d(v, w) = 2 omputing Hamming distance is trivial task. omputing edit distance is non-trivial task. Lopresti Fall 2007 Lecture

58 Edit distance vs. Hamming distance Hamming distance always compares i -th letter of v with i -th letter of w v = AAAA w = AAAA Hamming distance: d(v, w) = 8 Just one shift Make it all line up Edit distance may compare i -th letter of v with j -th letter of w v = -AAAA w = AAAA- Edit distance: d(v, w) = 2 How to find which j goes with which i??? One insertion and one deletion. Lopresti Fall 2007 Lecture

59 Edit distance example AA AA in 5 steps: AA (delete last ) AA (delete last A) A (insert A at front) AA (substitute for 3 rd ) AA (insert before last A) AA (Done) Lopresti Fall 2007 Lecture

60 Edit distance example AA AA in 5 steps: AA (delete last ) AA (delete last A) A (insert A at front) AA (substitute for 3 rd ) AA (insert before last A) AA (Done) What is the edit distance? 5? Lopresti Fall 2007 Lecture

61 Edit distance example AA AA in 4 steps: AA (insert A at front) AAA (delete 8 th ) AAA (substitute for 5 th A) AA (substitute for 3 rd ) AA (Done) Lopresti Fall 2007 Lecture

62 Edit distance example AA AA in 4 steps: AA (insert A at front) AAA (delete 8 th ) AAA (substitute for 5 th A) AA (substitute for 3 rd ) AA (Done) an it be done in 3 steps??? Lopresti Fall 2007 Lecture

63 Alignment grid Every alignment path is from source to sink. Lopresti Fall 2007 Lecture

64 Alignment as a path in edit graph A _ A _ A _ A _ orresponding path: (0,0), (1,1), (2,2), (2,3), (3,4), (4,5), (5,5), (6,6), (7,6), (7,7) Lopresti Fall 2007 Lecture

65 Alignment as a path in edit graph and represent indels in v and w with score 0. represent matches with score 1. he score of the alignment path is 5. Lopresti Fall 2007 Lecture

66 Alignment as a path in edit graph Every path in edit graph orresponds to an alignment: Lopresti Fall 2007 Lecture

67 Alignment as a path in edit graph Old Alignment v = A_A_ w = A_A_ New Alignment New Alignment v = A_A_ w = A_A_ Lopresti Fall 2007 Lecture

68 Alignment as a path in edit graph v = A_A_ w = A_A_ (0,0), (1,1), (2,2), (2,3), (3,4), (4,5), (5,5), (6,6), (7,6), (7,7) Lopresti Fall 2007 Lecture

69 Sequence comparison: the basic algorithm We can't afford to enumerate all possible alignments looking for the best one that would be an exponential search. Fortunately we don't have to. he optimal alignment can be found using dynamic programming, which we've just seen. Dynamic programming is based on premise of computing solutions to smaller subproblems first and then using these to solve successively larger problems until we have our answer. (Dynamic programming was invented by Richard Bellman in the 1950's. Its application to sequence comparison came later, in the 1970's.) Lopresti Fall 2007 Lecture

70 Sequence alignment... First some ground-rules. Legal: A Legal: - deletion - insertion Not legal: - Legal: Legal: - match mismatch Lopresti Fall 2007 Lecture

71 Sequence comparison: the basic algorithm iven two sequences v and w, consider what's required to compute optimal alignment for prefixes v[1..i] and w[1..j]. Based on our rules for alignments, there are three possible cases: deletion insertion substitution or match optimal alignment for v[1..i-1] and w[1..j] v[i] - optimal alignment for v[1..i] and w[1..j-1] - w[j] optimal alignment for v[1..i-1] and w[1..j-1] v[i] or w[j] I II III So, assuming we've already computed solutions for all shorter prefixes, we can compute the alignment for v[1..i] and w[1..j]. Lopresti Fall 2007 Lecture

72 Sequence comparison: the basic algorithm onceptually, this might look something like this: optimal alignment at v[1..i-1] and w[1..j] + cost of deleting v[i] optimal alignment at v[1..i] and w[1..j] = max optimal alignment at v[1..i] and w[1..j-1] + cost of inserting w[j] Here we assume that deletions, insertions, and mismatches have negative costs, while matches have positive costs. optimal alignment at v[1..i-1] and w[1..j-1] + cost of matching v[i] and w[j] Lopresti Fall 2007 Lecture

73 Sequence comparison: the basic algorithm his computation can be viewed as building a 2-D matrix: s t r i n g v ε ε 0 cost of deleting v d[i,j] = max s t r i n g where indel is cost of a deletion or insertion, and sub is cost of a match/mismatch involving symbols v[i] and w[j]. w cost of inserting w d[i-1,j] + indel d[i,j-1] + indel d[i-1,j-1] + sub(v[i],w [j]) Lopresti Fall 2007 Lecture

74 Sequence comparison: the basic algorithm Stated more generally, say that our two sequences are: v[1]v[2]v[3]...v[m] w[1]w[2]w[3]...w[n] hen: d[0,0] = 0 d[i,0] = d[i-1,0] + c del (v[i]) d[0,j] = d[0,j-1] + c ins (w[j]) 1 i m 1 j n initial conditions And: d[i,j] = max d[i-1,j] + c del (v[i]) d[i,j-1] + c ins (w[j]) 1 i m, 1 j n d[i-1,j-1] + c sub (v[i], w[j]) recurrence Where c del, c ins, and c sub are the costs of a deletion, an insertion, and a substitution, respectively. Lopresti Fall 2007 Lecture

75 Sequence comparison: the basic algorithm Example: say that c del = -1, c ins = -1, c sub = -1 if mismatch and +1 if match A A Lopresti Fall 2007 Lecture

76 Algorithm for computing edit distance EditDistance1(v, w) for i 1 to n d i,0 d i-1,0 + c del (v i ) for j 1 to m d 0,j d 0,j-1 + c ins (w j ) for i 1 to n for j 1 to m d i-1,j + c del (v i ) d i,j max d i,j-1 + c ins (w j ) d i-1,j-1 + c sub (v i, w j ) return (d n,m ) Lopresti Fall 2007 Lecture

77 Sequence comparison: some observations omputation can progress in a number of ways: or or For sequences of length m and n, v[1]v[2]v[3]...v[m] w[1]w[2]w[3]...w[n] What is the computation time? O(mn) How much memory (storage) is required? O(mn) Lopresti Fall 2007 Lecture

78 Sequence comparison: getting an alignment We started with the notion of alignment. How do we get this? By keeping track of optimal decisions made during algorithm, A A and then tracing back optimal path. A - A (May not be unique). Lopresti Fall 2007 Lecture

79 Maintaining the traceback EditDistance1(v, w)... for i 1 to n for j 1 to m d i-1,j + c del (v i ) d i,j max d i,j-1 + c ins (w j ) d i-1,j-1 + c sub (v i, w j ) if d i,j = d i-1,j + c del (v i ) b i,j if d i,j = d i,j-1 + c ins (w j ) return (d n,m, b) if d i,j = d i-1,j-1 + c sub (v i, w j ) Lopresti Fall 2007 Lecture

80 More examples omparing A vs. A: Lopresti Fall 2007 Lecture

81 More examples omparing AA vs. A: Lopresti Fall 2007 Lecture

82 More examples omparing A vs. : Recall the trace we saw earlier: A Lopresti Fall 2007 Lecture

83 Optimality of the algorithm How do we know this agorithm finds an optimal alignment? We won't dig into finer details now, but basically it hinges on: Bellman's principle of optimality for dynamic programming. he cost functions behaving properly (i.e., they must satisfy the triangle inequality). his could be an interesting topic for a final paper... Lopresti Fall 2007 Lecture

84 Wrap-up Readings for next time: ontinue reading IBA hapter 6 (sequence comparison). Remember: ome to class having done the readings. heck Blackboard regularly for updates. Lopresti Fall 2007 Lecture