/8/8 Objec&ves Dynamic Programming Ø Review Knapsack Ø Sequence Alignment Mar 8, 8 CSCI - Sprenkle Review What is the knapsack problem? What is our solu&on? Mar 8, 8 CSCI - Sprenkle
/8/8 Dynamic Programming: Adding a New Variable Def. OPT(i, w) = max profit subset of items,, i with weight limit w Ø Case : OPT does not select item i OPT selects best of {,,, i- } using weight limit w Ø Case : OPT selects item i new weight limit = w w i OPT selects best of {,,, i } using new weight limit, w w i # if i = % OPT(i, w) = $ OPT(i, w) if w i > w % max OPT(i, w), v Mar 8, 8 & { CSCI - Sprenkle i + OPT(i, w w i )} otherwise Knapsack Problem: Bo^om-Up Fill up an n-by-w array Input: W, N, w,,w N, v,,v N for w = to W M[, w] = for i = to N # for all items for w = to W # for all possible weights if w i > w : # item s weight is more than available M[i, w] = M[i-, w] else M[i, w] = max{ M[i-, w], v i + M[i-, w-w i ] } return M[N, W] Mar 8, 8 CSCI - Sprenkle
/8/8 Knapsack Input W = Item Value Weight 8 8 Mar 8, 8 CSCI - Sprenkle Knapsack Algorithm i = W + 8 9 φ n + { } {, } {,, } {,,, } 8 8 9 8 {,,,, } OPT: Solution = W = Value 8 Mar 8, 8 CSCI - Sprenkle Item 8 Weight
/8/8 Knapsack Algorithm i = W + 8 9 φ n + { } {, } {,, } {,,, } {,,,, } 8 8 8 9 8 8 Observations? Questions from last time? W = Value 8 Mar 8, 8 CSCI - Sprenkle Item 8 Weight Knapsack Algorithm i = W + 8 9 φ n + { } {, } {,, } {,,, } {,,,, } 8 8 8 9 8 8 OPT: Solution = What is the optimal solution? W = Value 8 Mar 8, 8 CSCI - Sprenkle 8 Item 8 Weight
/8/8 Knapsack Algorithm W + 8 9 φ n + { } {, } {,, } {,,, } {,,,, } 8 8 8 9 8 8 OPT: = + 8 Solution={, } W = Value 8 Mar 8, 8 CSCI - Sprenkle 9 Item 8 Weight SEQUENCE ALIGNMENT Mar 8, 8 CSCI - Sprenkle
/8/8 String Similarity How similar are two strings? Ø ocurrance Ø occurrence Measurements Ø Gap (-): add a le^er Ø Mismatch o c u r r a n c e - o c c u r r e n c e mismatches, gap o c - u r r a n c e o c c u r r e n c e mismatch, gap Which is the best alignment? o c - u r r - a n c e o c c u r r e - n c e mismatches, gaps Mar 8, 8 CSCI - Sprenkle Edit Distance [Levenshtein 9, Needleman-Wunsch 9] Ø Gap penalty: δ Parameters allow us to tweak cost Ø Mismatch penalty: α pq If p and q are the same, then mismatch penalty is Ø Cost = sum of gap and mismatch penal&es C T G A C C T A C C T - C T G A C C T A C C T C C T G A C T A C A T α TC + α GT + α AG + α CA C C T G A C - T A C A T δ + α CA Mar 8, 8 CSCI - Sprenkle
/8/8 Sequence Alignment Goal: Given two strings X = x x... x m and Y = y y... y n find alignment of minimum cost An alignment M is a set of ordered pairs x i -y j such that each item occurs in at most one pair and no crossings The pair x i -y j and x i' -y j' cross if i < i', but j > j. o c c u r e r n c e o c c u r e r n c e o c c u r r e n c e o c c u r r e n c e crossing mismatches Mar 8, 8 CSCI - Sprenkle Sequence Alignment Example X = CTACCG Y = TACATG Solu&on: M = x -y, x -y, x -y, x -y, x -y x x x x x x C T A C C - G - T A C A T G y y y y y y cost( M ) = α xi y j (x i, y j ) M mismatch + δ + δ i : x i unmatched j : y j unmatched gap Recall: mismatch penalty is if x i and y j are the same Mar 8, 8 CSCI - Sprenkle
/8/8 Sequence Alignment Case Analysis Consider last character of the strings X and Y: x M and y N Ø M and N are not necessarily equal i.e., strings are not necessarily the same length What are the possibili&es for x M and y N in terms of the alignment? x y Mar 8, 8 CSCI - Sprenkle Sequence Alignment Case Analysis Consider last character of strings X and Y: x M and y N Ø Case : x M and y N are aligned Ø Case : x M is not matched Ø Case : y N is not matched x y Formulate the optimal solution s value Mar 8, 8 CSCI - Sprenkle 8
/8/8 Sequence Alignment Case Analysis Consider last character of strings X and Y: x M and y N Ø Case : x M and y N are aligned Ø Case : x M is not matched Ø Case : y N is not matched x y OPT(i, j) = min cost of aligning strings x x... x i and y y... y j What are the costs for these cases? Mar 8, 8 CSCI - Sprenkle Sequence Alignment Cost Analysis Consider last character of strings X and Y: x M and y N Ø Case : x M and y N are aligned Pay mismatch for x M -y N + min cost of aligning rest of strings OPT(M, N) = α XmYn + OPT(M-, N-) Ø Case : x M is not matched Pay gap for x M + min cost of aligning rest of strings OPT(M, N) = δ + OPT(M-, N) Ø Case : y N is not matched Pay gap for y N + min cost of aligning rest of strings OPT(M, N) = δ + OPT(M, N-) Mar 8, 8 CSCI - Sprenkle 8 9
/8/8 Sequence Alignment Cost Analysis Base costs? à i or j is Ø What happens when we run out of le^ers in one string before the other? X = CTACCG Y = TACTG Mar 8, 8 CSCI - Sprenkle 9 Sequence Alignment: Problem Structure Gaps for remainder of Y " $ $ OPT(i, j) = # $ % $ jδ if i = " α xi y j + OPT(i, j ) $ min # δ + OPT(i, j) otherwise $ % δ + OPT(i, j ) iδ if j = Ran out of st string Ran out of nd string Gaps for remainder of X Mar 8, 8 CSCI - Sprenkle
/8/8 Sequence Alignment: Algorithm Cost parameters Sequence-Alignment(m, n, x x...x m, y y...y n, δ, α) for i = to m M[i, ] = iδ for j = to n M[, j] = jδ for i = to m for j = to n M[i, j] = min(α[x i, y j ] + M[i-, j-], δ + M[i-, j], δ + M[i, j-]) return M[m, n] Mar 8, 8 CSCI - Sprenkle Example α =, for vowel mismatch α =, for other mismatches δ = i X = bait b o o t j Y = boot b a i t Mar 8, 8 CSCI - Sprenkle
/8/8 Example X = bait Y = boot α =, for vowel mismatch α =, for other mismatches δ = i b o o t 8 j b a i t 8 Mar 8, 8 CSCI - Sprenkle Example X = bait Y = boot α =, for vowel mismatch α =, for other mismatches δ = i b a i t 8 b o o t 8 j Mar 8, 8 CSCI - Sprenkle
/8/8 Example X = bait Y = boot α =, for vowel mismatch α =, for other mismatches δ = i b a i t 8 b o o t 8 j Mar 8, 8 CSCI - Sprenkle Example X = bait Y = boot α =, for vowel mismatch α =, for other mismatches δ = i b a i t 8 b o o t 8 j Mar 8, 8 CSCI - Sprenkle
/8/8 Example What is the value for the problem? What is the solution? X = bait Y = boot α =, for vowel mismatch α =, for other mismatches δ = i j b a i t 8 b o o t 8 Mar 8, 8 CSCI - Sprenkle Example X = bait Y = boot α =, for vowel mismatch α =, for other mismatches δ = i j b a i t 8 b o o t 8 Mar 8, 8 CSCI - Sprenkle 8
/8/8 Sequence Alignment: Analysis Sequence-Alignment(m, n, x x...x m, y y...y n, δ, α) for i = to m M[, i] = iδ for j = to n M[j, ] = jδ O(mn) for i = to m for j = to n M[i, j] = min(α[x i, y j ] + M[i-, j-], δ + M[i-, j], δ + M[i, j-]) return M[m, n] Costs? Mar 8, 8 CSCI - Sprenkle Sequence Alignment: Algorithm Sequence-Alignment(m, n, x x...x m, y y...y n, δ, α) for i = to m M[, i] = iδ for j = to n M[j, ] = jδ for i = to m for j = to n M[i, j] = min(α[x i, y j ] + M[i-, j-], δ + M[i-, j], δ + M[i, j-]) return M[m, n] What are the space costs? When computing M[i,j], which entries in M are used? Mar 8, 8 CSCI - Sprenkle
/8/8 Sequence Alignment: Analysis Sequence-Alignment(m, n, x x...x m, y y...y n, δ, α) for i = to m M[, i] = iδ for j = to n M[j, ] = jδ Space Cost: O(mn) for i = to m for j = to n M[i, j] = min(α[x i, y j ] + M[i-, j-], δ + M[i-, j], δ + M[i, j-]) return M[m, n] Observation: to calculate the current value, we only need the row above us and the entry to the left Mar 8, 8 CSCI - Sprenkle SEQUENCE ALIGNMENT IN LINEAR SPACE Mar 8, 8 CSCI - Sprenkle
/8/8 Sequence Alignment: O(m) Space Collapse into an m x array Ø M[i,] represents previous row; M[i,] -- current Space-Efficient-Alignment(m, n, x x...x m, y y...y n, δ, α) for i = to m # initialize first row M[i, ] = iδ for j = to n M[, ] = jδ # first gap for i = to m M[i, ] = min(α[x i, y j ] + M[i-, ], δ + M[i, ], δ + M[i-, ]) for i = to m # copy current row into previous M[i, ] = M[i, ] return M[m, ] Any drawbacks? Mar 8, 8 CSCI - Sprenkle Sequence Alignment: O(m) Space Collapse into an m x array Ø M[i,] represents previous row; M[i,] -- current Space-Efficient-Alignment(m, n, x x...x m, y y...y n, δ, α) for i = to m # initialize first row M[i, ] = iδ for j = to n M[, ] = jδ # first gap for i = to m M[i, ] = min(α[x i, y j ] + M[i-, ], δ + M[i, ], δ + M[i-, ]) for i = to m # copy current row into previous M[i, ] = M[i, ] return M[m, ] Finds optimal value but will not be able to find alignment Mar 8, 8 CSCI - Sprenkle
/8/8 Why Do We Care About Space? For English words or sentences, probably doesn t ma^er Ma^ers for Biological sequence alignment Ø Consider: strings with, symbols each Processor can do billion primi&ve opera&ons BUT dealing with a GB array Mar 8, 8 CSCI - Sprenkle Sequence Alignment: Linear Space Can we avoid using quadra&c space? Ø Op&mal value in O(m) space and O(mn) &me. Compute OPT(i, ) from OPT(i-, ) BUT, no simple way to recover alignment itself Theorem. [Hirschberg 9] Op&mal alignment in O(m + n) space and O(mn) &me. Ø Clever combina&on of divide-and-conquer and dynamic programming Ø Sec&on. Mar 8, 8 CSCI - Sprenkle 8
/8/8 Looking Ahead PS8 Mar 8, 8 CSCI - Sprenkle 9