lgorithmic Paradigms hapter Dynamic Programming reed Build up a solution incrementally, myopically optimizing some local criterion Divide-and-conquer Break up a problem into two sub-problems, solve each sub-problem independently, and combine solution to sub-problems to form solution to original problem Dynamic programming Break up a problem into a series of overlapping sub-problems, and build up solutions to larger and larger sub-problems Slides by Kevin Wayne opyright 25 Pearson-ddison Wesley ll rights reserved 2 Dynamic Programming History Dynamic Programming pplications Bellman Pioneered the systematic study of dynamic programming in the 95s Etymology! Dynamic programming = planning over time! Secretary of Defense was hostile to mathematical research! Bellman sought an impressive name to avoid confrontation "it's impossible to use dynamic in a pejorative sense" "something not even a ongressman could object to" Reference: Bellman, R E Eye of the Hurricane, n utobiography reas! Bioinformatics! ontrol theory! Information theory! Operations research! omputer science: theory, graphics, I, systems, Some famous dynamic programming algorithms! Viterbi for hidden Markov models! nix diff for comparing two files! Smith-Waterman for sequence alignment! Bellman-Ford for shortest path routing in networks! ocke-kasami-younger for parsing context free grammars 3 4
Weighted Interval Scheduling Weighted Interval Scheduling Weighted interval scheduling problem! Job j starts at s j, finishes at f j, and has weight or value v j! Two jobs compatible if they don't overlap! oal: find maximum weight subset of mutually compatible jobs a b c d e f g 2 3 4 5 8 9 h Time nweighted Interval Scheduling Review Weighted Interval Scheduling Recall reedy algorithm works if all weights are! onsider jobs in ascending order of finish time! dd job to subset if it is compatible with previously chosen jobs Notation Label jobs by finishing time: f! f 2!! f n Def p(j) = largest index i < j such that job i is compatible with j Ex: p(8) = 5, p() = 3, p(2) = Observation reedy algorithm can fail spectacularly if arbitrary weights are allowed 2 3 4 5 weight = 999 b weight = a 2 3 4 5 8 9 Time 8 2 3 4 5 8 9 Time 8
Dynamic Programming: Binary hoice Weighted Interval Scheduling: Brute Force Notation OPT(j) = value of optimal solution to the problem consisting of job requests, 2,, j Brute force algorithm! ase : OPT selects job j can't use incompatible jobs { p(j), p(j) 2,, j - must include optimal solution to problem consisting of remaining compatible jobs, 2,, p(j) optimal substructure! ase 2: OPT does not select job j must include optimal solution to problem consisting of remaining compatible jobs, 2,, j- # if j = OPT( j) = $ % max v j OPT( p( j)), OPT( j ") { otherwise Input: n, s,,s n, f,,f n, v,,v n Sort jobs by finish times so that f! f 2!! f n ompute p(), p(2),, p(n) ompute-opt(j) { if (j = ) return else return max(v j ompute-opt(p(j)), ompute-opt(j-)) 9 Weighted Interval Scheduling: Brute Force Weighted Interval Scheduling: Memoization Observation Recursive algorithm fails spectacularly because of redundant sub-problems " exponential algorithms Memoization Store results of each sub-problem in a cache; lookup as needed Ex Number of recursive calls for family of "layered" instances grows like Fibonacci sequence 5 Input: n, s,,s n, f,,f n, v,,v n Sort jobs by finish times so that f! f 2!! f n ompute p(), p(2),, p(n) 2 3 4 p() =, p(j) = j-2 5 4 3 3 2 2 2 for j = to n M[j] = empty M[j] = global array M-ompute-Opt(j) { if (M[j] is empty) M[j] = max(w j M-ompute-Opt(p(j)), M-ompute-Opt(j-)) return M[j] 2
Weighted Interval Scheduling: Running Time utomated Memoization laim Memoized version of algorithm takes O(n log n) time! Sort by finish time: O(n log n)! omputing p(#) : O(n) after sorting by start time! M-ompute-Opt(j): each invocation takes O() time and either utomated memoization Many functional programming languages (eg, Lisp) have built-in support for memoization Q Why not in imperative languages (eg, Java)? (i) returns an existing value M[j] (ii) fills in one new entry M[j] and makes two recursive calls! Progress measure $ = # nonempty entries of M[] initially $ =, throughout $! n (ii) increases $ by " at most 2n recursive calls (defun F (n) (if (<= n ) n ( (F (- n )) (F (- n 2))))) Lisp (efficient) static int F(int n) { if (n <= ) return n; else return F(n-) F(n-2); Java (exponential)! Overall running time of M-ompute-Opt(n) is O(n)! F(39) F(4) F(38) Remark O(n) if jobs are pre-sorted by start and finish times F(3) F(38) F(3) F(3) F(3) F(35) F(3) F(3) F(35) F(35) F(3) F(34) 3 4 Weighted Interval Scheduling: Finding a Solution Weighted Interval Scheduling: Bottom-p Q Dynamic programming algorithms computes optimal value What if we want the solution itself? Do some post-processing Bottom-up dynamic programming nwind recursion Input: n, s,,s n, f,,f n, v,,v n Run M-ompute-Opt(n) Run Find-Solution(n) Find-Solution(j) { if (j = ) output nothing else if (v j M[p(j)] > M[j-]) print j Find-Solution(p(j)) else Find-Solution(j-) Sort jobs by finish times so that f! f 2!! f n ompute p(), p(2),, p(n) Iterative-ompute-Opt { M[] = for j = to n M[j] = max(v j M[p(j)], M[j-])! # of recursive calls! n " O(n) 5
Segmented Least Squares 3 Segmented Least Squares Least squares! Foundational problem in statistic and numerical analysis! iven n points in the plane: (x, y ), (, ),, (x n, y n )! Find a line y = ax b that minimizes the sum of the squared error: n SSE = # ( y i " ax i " b) 2 i= y x Solution alculus " min error is achieved when a = n # x i y i i " (# i x i ) (# i y i ), b = n i " ( x i ) 2 # i # i # i y i " a # i x i n 8 Segmented Least Squares Segmented Least Squares Segmented least squares! Points lie roughly on a sequence of several line segments! iven n points in the plane (x, y ), (, ),, (x n, y n ) with! x < < < x n, find a sequence of lines that minimizes f(x) Q What's a reasonable choice for f(x) to balance accuracy and parsimony? goodness of fit Segmented least squares! Points lie roughly on a sequence of several line segments! iven n points in the plane (x, y ), (, ),, (x n, y n ) with! x < < < x n, find a sequence of lines that minimizes: the sum of the sums of the squared errors E in each segment the number of lines L! Tradeoff function: E c L, for some constant c > number of lines y y x x 9 2
Dynamic Programming: Multiway hoice Segmented Least Squares: lgorithm Notation! OPT(j) = minimum cost for points p, p i,, p j! e(i, j) = minimum sum of squares for points p i, p i,, p j To compute OPT(j):! Last segment uses points p i, p i,, p j for some i! ost = e(i, j) c OPT(i-) INPT: n, p,,p N, c Segmented-Least-Squares() { M[] = for j = to n for i = to j compute the least square error e ij for the segment p i,, p j $ & if j = OPT( j) = % min { e(i, j) c OPT(i #) otherwise '& " i " j for j = to n M[j] = min! i! j (e ij c M[i-]) return M[n] Running time O(n 3 ) can be improved to O(n 2 ) by pre-computing various statistics! Bottleneck = computing e(i, j) for O(n 2 ) pairs, O(n) per pair using previous formula 2 22 Knapsack Problem 4 Knapsack Problem Knapsack problem! iven n objects and a "knapsack"! Item i weighs w i > kilograms and has value v i >! Knapsack has capacity of W kilograms! oal: fill knapsack so as to maximize total value Ex: { 3, 4 has value 4 Item Value Weight W = 2 3 2 8 5 4 22 5 28 reedy: repeatedly add item with maximum ratio v i / w i Ex: { 5, 2, achieves only value = 35 " greedy not optimal 24
Dynamic Programming: False Start Dynamic Programming: dding a New Variable Def OPT(i) = max profit subset of items,, i! ase : OPT does not select item i OPT selects best of {, 2,, i-! ase 2: OPT selects item i accepting item i does not immediately imply that we will have to reject other items without knowing what other items were selected before i, we don't even know if we have enough room for i onclusion Need more sub-problems! Def OPT(i, w) = max profit subset of items,, i with weight limit w! ase : OPT does not select item i OPT selects best of {, 2,, i- using weight limit w! ase 2: OPT selects item i new weight limit = w w i OPT selects best of {, 2,, i using this new weight limit # if i = % OPT(i, w) = $ OPT(i ", w) if w i > w % & max{ OPT(i ", w), v i OPT(i ", w " w i ) otherwise 25 2 Knapsack Problem: Bottom-p Knapsack lgorithm Knapsack Fill up an n-by-w array W 2 3 4 5 8 9 Input: n, w,,w N, v,,v N % for w = to W M[, w] = for i = to n for w = to W if (w i > w) M[i, w] = M[i-, w] else M[i, w] = max {M[i-, w], v i M[i-, w-w i ] n { {, 2 {, 2, 3 {, 2, 3, 4 {, 2, 3, 4, 5 8 8 8 9 22 22 24 25 24 28 28 29 Item 25 25 25 29 29 4 34 34 4 Value Weight return M[n, W] OPT: { 4, 3 value = 22 8 = 4 W = 2 3 4 5 2 8 5 22 28 2 28
Knapsack Problem: Running Time Running time '(n W)! Not polynomial in input size!! "Pseudo-polynomial"! Decision version of Knapsack is NP-complete [hapter 8] 5 RN Secondary Structure Knapsack approximation algorithm There exists a polynomial algorithm that produces a feasible solution that has value within % of optimum [Section 8] 29 RN Secondary Structure RN Secondary Structure RN String B = b b 2 b n over alphabet {,,, Secondary structure RN is single-stranded so it tends to loop back and form base pairs with itself This structure is essential for understanding behavior of molecule Ex: Secondary structure set of pairs S = { (b i, b j ) that satisfy:! [Watson-rick] S is a matching and each pair in S is a Watson- rick complement: -, -, -, or -! [No sharp turns] The ends of each pair are separated by at least 4 intervening bases If (b i, b j ) & S, then i < j - 4! [Non-crossing] If (b i, b j ) and (b k, b l ) are two pairs in S, then we cannot have i < k < j < l Free energy sual hypothesis is that an RN molecule will form the secondary structure with the optimum total free energy approximate by number of base pairs oal iven an RN molecule B = b b 2 b n, find a secondary structure S that maximizes the number of base pairs complementary base pairs: -, - 3 32
RN Secondary Structure: Examples RN Secondary Structure: Subproblems Examples First attempt OPT(j) = maximum number of base pairs in a secondary structure of the substring b b 2 b j match b t and b n base pair t n Difficulty Results in two sub-problems!4! Finding secondary structure in: b b 2 b t-! Finding secondary structure in: b t b t2 b n- OPT(t-) need more sub-problems ok sharp turn crossing 33 34 Dynamic Programming Over Intervals Bottom p Dynamic Programming Over Intervals Notation OPT(i, j) = maximum number of base pairs in a secondary structure of the substring b i b i b j Q What order to solve the sub-problems? Do shortest intervals first! ase If i ( j - 4 OPT(i, j) = by no-sharp turns condition! ase 2 Base b j is not involved in a pair OPT(i, j) = OPT(i, j-) RN(b,,b n ) { for k = 5,,, n- for i =, 2,, n-k j = i k ompute M[i, j] i 4 3 2! ase 3 Base b j pairs with b t for some i! t < j - 4 non-crossing constraint decouples resulting sub-problems OPT(i, j) = max t { OPT(i, t-) OPT(t, j-) return M[, n] using recurrence 8 9 j take max over t such that i! t < j-4 and b t and b j are Watson-rick complements Running time O(n 3 ) Remark Same core idea in KY algorithm to parse context-free grammars 35 3
Dynamic Programming Summary Recipe! haracterize structure of problem! Recursively define value of optimal solution! ompute value of optimal solution! onstruct optimal solution from computed information Sequence lignment Dynamic programming techniques! Binary choice: weighted interval scheduling! Multi-way choice: segmented least squares! dding a new variable: knapsack Viterbi algorithm for HMM also uses DP to optimize a maximum likelihood tradeoff between parsimony and accuracy! Dynamic programming over intervals: RN secondary structure KY parsing algorithm for context-free grammar has similar structure Top-down vs bottom-up: different people have different intuitions 3 String Similarity Edit Distance How similar are two strings?! ocurrance! occurrence o c u r r a n c e - o c c u r r e n c e 5 mismatches, gap o c - u r r a n c e pplications! Basis for nix diff! Speech recognition! omputational biology Edit distance [Levenshtein 9, Needleman-Wunsch 9]! ap penalty ); mismatch penalty * pq! ost = sum of gap and mismatch penalties o c c u r r e n c e mismatch, gap T T T - T T T o c - u r r - a n c e T T T T - T T o c c u r r e - n c e * T * T * 2* 2) * mismatches, 3 gaps 39 4
Sequence lignment Sequence lignment: Problem Structure oal: iven two strings X = x x m and Y = y y n find alignment of minimum cost Def n alignment M is a set of ordered pairs x i -y j such that each item occurs in at most one pair and no crossings Def The pair x i -y j and x i' -y j' cross if i < i', but j > j' cost( M ) = $ " xi y j $ % $ % (x i, y j ) # M i : x 42 43 i unmatched j : y j unmatched 444 42 444443 mismatch Ex: T vs TT Sol: M = -y, x 3 -, x 4 -y 3, x 5 -y 4, x -y gap x x 3 x 4 x 5 x T - - T T Def OPT(i, j) = min cost of aligning strings x x i and y y j! ase : OPT matches x i -y j pay mismatch for x i -y j min cost of aligning two strings x x i- and y y j-! ase 2a: OPT leaves x i unmatched pay gap for x i and min cost of aligning x x i- and y y j! ase 2b: OPT leaves y j unmatched pay gap for y j and min cost of aligning x x i and y y j- " $ $ OPT(i, j) = # $ % $ j& if i = " ' xi y j OPT(i (, j () $ min # & OPT(i (, j) otherwise $ % & OPT(i, j () i& if j = y y 3 y 4 y 5 y 4 42 Sequence lignment: lgorithm Sequence-lignment(m, n, x x m, y y n, ), *) { for i = to m M[, i] = i) for j = to n M[j, ] = j) Sequence lignment in Linear Space for i = to m for j = to n M[i, j] = min(*[x i, y j ] M[i-, j-], ) M[i-, j], ) M[i, j-]) return M[m, n] nalysis '(mn) time and space English words or sentences: m, n! omputational biology: m = n =, billions ops OK, but B array? 43
Sequence lignment: Linear Space Sequence lignment: Linear Space Q an we avoid using quadratic space? Easy Optimal value in O(m n) space and O(mn) time! ompute OPT(i, ) from OPT(i-, )! No longer a simple way to recover alignment itself Edit distance graph! Let f(i, j) be shortest path from (,) to (i, j)! Observation: f(i, j) = OPT(i, j) Theorem [Hirschberg 95] Optimal alignment in O(m n) space and O(mn) time - y y 3 y 4 y 5 y! lever combination of divide-and-conquer and dynamic programming! Inspired by idea of Savitch from complexity theory x " xi y j ) ) i-j x 3 m-n 45 4 Sequence lignment: Linear Space Sequence lignment: Linear Space Edit distance graph! Let f(i, j) be shortest path from (,) to (i, j)! an compute f (, j) for any j in O(mn) time and O(m n) space Edit distance graph! Let g(i, j) be shortest path from (i, j) to (m, n)! an compute by reversing the edge orientations and inverting the roles of (, ) and (m, n) j y y 3 y 4 y 5 y y y 3 y 4 y 5 y - - x x i-j ) " xi y j i-j ) x 3 m-n x 3 m-n 4 48
Sequence lignment: Linear Space Sequence lignment: Linear Space Edit distance graph! Let g(i, j) be shortest path from (i, j) to (m, n)! an compute g(, j) for any j in O(mn) time and O(m n) space Observation The cost of the shortest path that uses (i, j) is f(i, j) g(i, j) j y y 3 y 4 y 5 y y y 3 y 4 y 5 y - - x i-j x i-j x 3 m-n x 3 m-n 49 5 Sequence lignment: Linear Space Sequence lignment: Linear Space Observation 2 let q be an index that minimizes f(q, n/2) g(q, n/2) Then, the shortest path from (, ) to (m, n) uses (q, n/2) Divide: find index q that minimizes f(q, n/2) g(q, n/2) using DP! lign x q and y n/2 onquer: recursively compute optimal alignment in each piece n / 2 n / 2 y y 3 y 4 y 5 y y y 3 y 4 y 5 y - - x i-j q x i-j q x 3 m-n x 3 m-n 5 52
Sequence lignment: Running Time nalysis Warmup Sequence lignment: Running Time nalysis Theorem Let T(m, n) = max running time of algorithm on strings of length at most m and n T(m, n) = O(mn log n) Theorem Let T(m, n) = max running time of algorithm on strings of length m and n T(m, n) = O(mn) T(m, n) " 2T(m, n/2) O(mn) # T(m, n) = O(mn logn) Remark nalysis is not tight because two sub-problems are of size (q, n/2) and (m - q, n/2) In next slide, we save log n factor Pf (by induction on n)! O(mn) time to compute f(, n/2) and g (, n/2) and find index q! T(q, n/2) T(m - q, n/2) time for two recursive calls! hoose constant c so that: T(m, 2) " cm T(2, n) " cn T(m, n) " cmn T(q, n/2) T(m # q, n/2)! Base cases: m = 2 or n = 2! Inductive hypothesis: T(m, n)! 2cmn T ( m, n) " " = = T ( q, n / 2) T ( m! q, n / 2) cmn 2cqn / 2 2c( m! q) n / 2 cmn cqn cmn! cqn cmn 2cmn 53 54