6. DYNAMIC PROGRAMMING II

Transcription

1 6. DYNAMIC PROGRAMMING II sequence alignmen Hirschberg's algorihm Bellman-Ford algorihm disance vecor proocols negaive cycles in a digraph 6. DYNAMIC PROGRAMMING II sequence alignmen Hirschberg's algorihm Bellman-Ford algorihm disance vecor proocols negaive cycles in a digraph Lecure slides by Kevin Wayne Copyrigh 00 Pearson-Addison Wesley hp:// SECTION 6.6 Las updaed on Mar 6, 0 : AM Sring similariy Edi disance Q. How similar are wo srings? Ex. ocurrance and occurrence. Edi disance. [Levenshein 966, Needleman-Wunsch 970] Gap penaly δ; mismach penaly α pq. Cos = sum of gap and mismach penalies. o c u r r a n c e o c c u r r e n c e 6 mismaches, gap o c u r r a n c e o c c u r r e n c e mismach, gap C T G A C C T A C G C T G G A C G A A C G cos = δ + αcg + αta o c u r r a n c e o c c u r r e n c e 0 mismaches, gaps Applicaions. Unix diff, speech recogniion, compuaional biology,...

2 Sequence alignmen Sequence alignmen: problem srucure Goal. Given wo srings x x... x m and y y... y n find min cos alignmen. Def. OPT(i, j) = min cos of aligning prefix srings x x... x i and y y... y j. Def. An alignmen M is a se of ordered pairs x i y j such ha each iem occurs in a mos one pair and no crossings. Def. The cos of an alignmen M is: cos(m ) = α xi y j (x i, y j ) M mismach + δ + δ i : x i unmached j : y j unmached gap x i y j and x i' y j' cross if i < i ', bu j > j ' Case. OPT maches xi yj. Pay mismach for x i y j + min cos of aligning x x... x i and y y... y j. Case a. OPT leaves xi unmached. Pay gap for x i + min cos of aligning x x... x i and y y... y j. Case b. OPT leaves yj unmached. Pay gap for y j + min cos of aligning x x... x i and y y... y j. opimal subsrucure propery (proof via exchange argumen) x x x x x x 6 C T A C C G T A C A T G y y y y y y 6 an alignmen of CTACCG and TACATG: M = { x y, x y, x y, x y, x 6 y 6 } OPT(i, j) = jδ if i = 0 α xi y j +OPT(i, j ) min δ +OPT(i, j) oherwise δ +OPT(i, j ) iδ if j = 0 6 Sequence alignmen: algorihm Sequence alignmen: analysis SEQUENCE-ALIGNMENT (m, n, x,, xm, y,, yn, δ, α) FOR i = 0 TO m M [i, 0] i δ. FOR j = 0 TO n M [0, j] j δ. FOR i = TO m FOR j = TO n M [i, j] min { α[xi, yj] + M [i, j ], δ + M [i, j], δ + M [i, j ]). Theorem. The dynamic programming algorihm compues he edi disance (and opimal alignmen) of wo srings of lengh m and n in Θ(mn) ime and Θ(mn) space. Algorihm compues edi disance. Can race back o exrac opimal alignmen iself. Q. Can we avoid using quadraic space? A. Easy o compue opimal value in O(mn) ime and O(m + n) space. Compue OPT(i, ) from OPT(i, ). Bu, no longer easy o recover opimal alignmen iself. RETURN M [m, n]. 7 8

3 Sequence alignmen in linear space 6. DYNAMIC PROGRAMMING II sequence alignmen Hirschberg's algorihm Bellman-Ford algorihm disance vecor proocols negaive cycles in a digraph Theorem. There exis an algorihm o find an opimal alignmen in O(mn) ime and O(m + n) space. Clever combinaion of divide-and-conquer and dynamic programming. Inspired by idea of Savich from complexiy heory. Programming Techniques G. Manacher Edior A Linear Space Algorihm for Compuing Maximal Common Subsequences D.S. Hirschberg Princeon Universiy SECTION 6.7 The problem of finding a longes common subsequence of wo srings has been solved in quadraic ime and space. An algorihm is presened which will solve his problem in quadraic ime and in linear space. Key Words and Phrases: subsequence, longes common subsequence, sring correcion, ediing CR Caegories:.6,.7,.79,.,. Inroducion 0 Hirschberg's algorihm Hirschberg's algorihm Edi disance graph. Le f (i, j) be shores pah from (0,0) o (i, j). Lemma: f (i, j) = OPT(i, j) for all i and j. Edi disance graph. Le f (i, j) be shores pah from (0,0) o (i, j). Lemma: f (i, j) = OPT(i, j) for all i and j. Pf of Lemma. [ by srong inducion on i + j ] x y y y y y y 6 Base case: f (0, 0) = OPT (0, 0) = 0. Inducive hypohesis: assume rue for all (i', j') wih i' + j' < i + j. Las edge on shores pah o (i, j) is from (i, j ), (i, j), or (i, j ). Thus, f(i, j) = min{ xiy j + f(i,j ), + f(i,j), + f(i, j )} { } α xi y j δ = min{ xiy j + OPT(i,j ), + OPT(i,j), + OPT(i, j )} { } x δ = OPT(i, j) α xi y j δ x δ

4 Hirschberg's algorihm Hirschberg's algorihm Edi disance graph. Le f (i, j) be shores pah from (0,0) o (i, j). Lemma: f (i, j) = OPT(i, j) for all i and j. Can compue f (, j) for any j in O(mn) ime and O(m + n) space. Edi disance graph. Le g (i, j) be shores pah from (i, j) o (m, n). Can compue by reversing he edge orienaions and invering he roles of (0, 0) and (m, n). j y y y y y y 6 y y y y y y 6 x x δ δ α x i y j x x x x Hirschberg's algorihm Hirschberg's algorihm Edi disance graph. Le g (i, j) be shores pah from (i, j) o (m, n). Can compue g(, j) for any j in O(mn) ime and O(m + n) space. Observaion. The cos of he shores pah ha uses (i, j) is f (i, j) + g(i, j). j y y y y y y 6 y y y y y y 6 x x x x x x 6

5 Hirschberg's algorihm Hirschberg's algorihm Observaion. le q be an index ha minimizes f (q, n/) + g (q, n/). Then, here exiss a shores pah from (0, 0) o (m, n) uses (q, n/). Divide. Find index q ha minimizes f (q, n/) + g(q, n/); align x q and y n /. Conquer. Recursively compue opimal alignmen in each piece. n / n / y y y y y y 6 y y y y y y 6 x q x q x x x x 7 8 Hirschberg's algorihm: running ime analysis warmup Hirschberg's algorihm: running ime analysis Theorem. Le T(m, n) = max running ime of Hirschberg's algorihm on srings of lengh a mos m and n. Then, T(m, n) = O(m n log n). Theorem. Le T(m, n) = max running ime of Hirschberg's algorihm on srings of lengh a mos m and n. Then, T(m, n) = O(m n). T(m, n) T(m, n / ) + O(m n) T(m, n) = O(m log n). Remark. Analysis is no igh because wo subproblems are of size (q, n/) and (m q, n / ). In nex slide, we save log n facor. [ by inducion on n ] O(m n) ime o compue f (, n / ) and g (, n / ) and find index q. T(q, n / ) + T(m q, n / ) ime for wo recursive calls. Choose consan c so ha: T(m, ) T(, n) c m c n T(m, n) c m n + T(q, n / ) + T(m q, n / ) Claim. T(m, n) c m n. Base cases: m = or n =. Inducive hypohesis: T(m, n) c m n for all (m', n') wih m' + n' < m + n. T(m, n) T(q, n / ) + T(m q, n / ) + c m n c q n / + c (m q) n / + c m n = c q n + c m n c q n + c m n = c m n 9 0

6 Shores pahs 6. DYNAMIC PROGRAMMING II Shores pah problem. Given a digraph G = (V, E), wih arbirary edge weighs or coss c vw, find cheapes pah from node s o node. sequence alignmen Hirschberg's algorihm Bellman-Ford disance vecor proocols negaive cycles in a digraph source s SECTION cos of pah = = 8 desinaion Shores pahs: failed aemps Negaive cycles Dijksra. Can fail if negaive edge weighs. s u Def. A negaive cycle is a direced cycle such ha he sum of is edge weighs is negaive. v -8 w Reweighing. Adding a consan o every edge weigh can fail. s u v w a negaive cycle W : c(w ) = - c e < 0 e W

7 Shores pahs and negaive cycles Shores pahs and negaive cycles Lemma. If some pah from v o conains a negaive cycle, hen here does no exis a cheapes pah from v o. Lemma. If G has no negaive cycles, hen here exiss a cheapes pah from v o ha is simple (and has n edges). If here exiss such a cycle W, hen can build a v pah of arbirarily negaive weigh by deouring around cycle as many imes as desired. Consider a cheapes v pah P ha uses he fewes number of edges. If P conains a cycle W, can remove porion of P corresponding o W wihou increasing he cos. v v W W c(w) < 0 c(w) 0 6 Shores pah and negaive cycle problems Shores pahs: dynamic programming Shores pah problem. Given a digraph G = (V, E) wih edge weighs c vw and no negaive cycles, find cheapes v pah for each node v. Negaive cycle problem. Given a digraph G = (V, E) wih edge weighs c vw, find a negaive cycle (if one exiss). Def. OPT(i, v) = cos of shores v pah ha uses i edges. Case : Cheapes v pah uses i edges. OPT(i, v) = OPT(i, v) Case : Cheapes v pah uses exacly i edges. if (v, w) is firs edge, hen OPT uses (v, w), and hen selecs bes w pah using i edges opimal subsrucure propery (proof via exchange argumen) - OPT(i, v) = 0 if i = 0 min OPT(i, v), min { OPT(i, w)+ c vw } (v, w) E oherwise shores-pahs ree negaive cycle Observaion. If no negaive cycles, OPT(n, v) = cos of cheapes v pah. By Lemma, cheapes v pah is simple. 7 8

8 Shores pahs: implemenaion Shores pahs: implemenaion Theorem. Given a digraph G = (V, E) wih no negaive cycles, he dynamic programming algorihm compues he cos of he cheapes v pah for SHORTEST-PATHS (V, E, c, ) each node v in Θ(mn) ime and Θ(n ) space. FOREACH node v V M [0, v]. M [0, ] 0. FOR i = TO n FOREACH node v V M [i, v] M [i, v]. FOREACH edge (v, w) E M [i, v] min { M [i, v], M [i, w] + cvw }. Table requires Θ(n ) space. Each ieraion i akes Θ(m) ime since we examine each edge once. Finding he shores pahs. Approach : Mainain a successor(i, v) ha poins o nex node on cheapes v pah using a mos i edges. Approach : Compue opimal coss M[i, v] and consider only edges wih M[i, v] = M[i, w] + c vw. 9 0 Shores pahs: pracical improvemens Bellman-Ford: efficien implemenaion Space opimizaion. Mainain wo d arrays (insead of d array). d(v) = cos of cheapes v pah ha we have found so far. successor(v) = nex node on a v pah. Performance opimizaion. If d(w) was no updaed in ieraion i, hen no reason o consider edges enering w in ieraion i. BELLMAN-FORD (V, E, c, ) FOREACH node v V d(v). successor(v) null. d() 0. FOR i = TO n FOREACH node w V IF (d(w) was updaed in previous ieraion) FOREACH edge (v, w) E IF ( d(v) > d(w) + cvw) pass d(v) d(w) + cvw. successor(v) w. IF no d(w) value changed in ieraion i, STOP.

9 Bellman-Ford: analysis Bellman-Ford: analysis Claim. Afer he i h pass of Bellman-Ford, d(v) equals he cos of he cheapes v pah using a mos i edges. Counerexample. Claim is false! d(v) = d(w) = d() = 0 Lemma. Throughou Bellman-Ford algorihm, d(v) is he cos of some v pah; afer he i h pass, d(v) is no larger han he cos of he cheapes v pah using i edges. [by inducion on i] Assume rue afer i h pass. Le P be any v pah wih i + edges. Le (v, w) be firs edge on pah and le P' be subpah from w o. By inducive hypohesis, d(w) c(p') since P' is a w pah wih i edges. Afer considering v in pass i+: d(v) c vw + d(w) cvw + c(p') = c(p) v w if nodes w considered before node v, hen d(v) = afer pass Theorem. Given a digraph wih no negaive cycles, Bellman-Ford compues he coss of he cheapes v pahs in O(mn) ime and Θ(n) exra space. Lemmas +. can be subsanially faser in pracice Bellman-Ford: analysis Bellman-Ford: analysis Claim. Throughou he Bellman-Ford algorihm, following successor(v) poiners gives a direced pah from v o of cos d(v). Claim. Throughou he Bellman-Ford algorihm, following successor(v) poiners gives a direced pah from v o of cos d(v). Counerexample. Claim is false! Cos of successor v pah may have sricly lower cos han d(v). Counerexample. Claim is false! Cos of successor v pah may have sricly lower cos han d(v). consider nodes in order:,,, consider nodes in order:,,, s() = s() = s() = s() = d() = 0 d() = 0 d() = 0 d() = 0 d() = d() = s() = d() = s() = d() = 6

10 Bellman-Ford: analysis Bellman-Ford: analysis Claim. Throughou he Bellman-Ford algorihm, following successor(v) poiners gives a direced pah from v o of cos d(v). Claim. Throughou he Bellman-Ford algorihm, following successor(v) poiners gives a direced pah from v o of cos d(v). Counerexample. Claim is false! Cos of successor v pah may have sricly lower cos han d(v). Successor graph may have cycles. Counerexample. Claim is false! Cos of successor v pah may have sricly lower cos han d(v). Successor graph may have cycles. consider nodes in order:,,,, consider nodes in order:,,,, d() = 0 d() = 8 d() = 0 d() = 8 9 d() = 0 9 d() = d() = d() = 7 d() = d() = 8 Bellman-Ford: finding he shores pah Bellman-Ford: finding he shores pah Lemma. If he successor graph conains a direced cycle W, hen W is a negaive cycle. If successor(v) = w, we mus have d(v) d(w) + cvw. (LHS and RHS are equal when successor(v) is se; d(w) can only decrease; d(v) decreases only when successor(v) is rese) Le v v vk be he nodes along he cycle W. Assume ha (v k, v ) is he las edge added o he successor graph. Jus prior o ha: d(v) d(v) + c(v, v) d(v) d(v) + c(v, v) d(vk ) d(vk) + c(vk, vk) d(vk) > d(v) + c(vk, v) Adding inequaliies yields c(v, v) + c(v, v) + + c(vk, vk) + c(vk, v) < 0. W is a negaive cycle holds wih sric inequaliy since we are updaing d(vk) 9 Theorem. Given a digraph wih no negaive cycles, Bellman-Ford finds he cheapes s pahs in O(mn) ime and Θ(n) exra space. The successor graph canno have a negaive cycle. [Lemma ] Thus, following he successor poiners from s yields a direced pah o. Le s = v v vk = be he nodes along his pah P. Upon erminaion, if successor(v) = w, we mus have d(v) = d(w) + c vw. (LHS and RHS are equal when successor(v) is se; d( ) did no change) Thus, d(v) = d(v) + c(v, v) d(v) = d(v) + c(v, v) d(vk ) = d(vk) + c(vk, vk) Adding equaions yields d(s) = d() + c(v, v) + c(v, v) + + c(vk, vk). min cos of any s pah (Theorem ) 0 cos of pah P since algorihm erminaed 0

11 Disance vecor proocols SECTION DYNAMIC PROGRAMMING II sequence alignmen Hirschberg's algorihm Bellman-Ford disance vecor proocols negaive cycles in a digraph Communicaion nework. Node rouer. Edge direc communicaion link. Cos of edge delay on link. Dijksra's algorihm. Requires global informaion of nework. Bellman-Ford. Uses only local knowledge of neighboring nodes. Synchronizaion. We don' expec rouers o run in locksep. The order in which each foreach loop execues in no imporan. Moreover, algorihm sill converges even if updaes are asynchronous. naurally nonnegaive, bu Bellman-Ford used anyway! Disance vecor proocols Pah vecor proocols Disance vecor proocols. [ "rouing by rumor" ] Each rouer mainains a vecor of shores pah lenghs o every oher node (disances) and he firs hop on each pah (direcions). Algorihm: each rouer performs n separae compuaions, one for each poenial desinaion node. Ex. RIP, Xerox XNS RIP, Novell's IPX RIP, Cisco's IGRP, DEC's DNA Phase IV, AppleTalk's RTMP. Link sae rouing. Each rouer also sores he enire pah. Based on Dijksra's algorihm. Avoids "couning-o-infiniy" problem and relaed difficulies. Requires significanly more sorage. no jus he disance and firs hop Ex. Border Gaeway Proocol (BGP), Open Shores Pah Firs (OSPF). Cavea. Edge coss may change during algorihm (or fail compleely). deleed s v d(s) = d(v) = d() = 0 "couning o infiniy"

12 Deecing negaive cycles 6. DYNAMIC PROGRAMMING II Negaive cycle deecion problem. Given a digraph G = (V, E), wih edge weighs c vw, find a negaive cycle (if one exiss). sequence alignmen Hirschberg's algorihm Bellman-Ford disance vecor proocol negaive cycles in a digraph 6 SECTION Deecing negaive cycles: applicaion Deecing negaive cycles Currency conversion. Given n currencies and exchange raes beween pairs of currencies, is here an arbirage opporuniy? Remark. Fases algorihm very valuable! 0.7 *.66 *.99 = EUR USD GBP Lemma. If OPT(n, v) = OPT(n, v) for all v, hen no negaive cycle can reach. Bellman-Ford algorihm. Lemma 6. If OPT(n, v) < OPT(n, v) for some node v, hen (any) cheapes pah from v o conains a cycle W. Moreover W is a negaive cycle. [by conradicion] Since OPT(n, v) < OPT(n, v), we know ha shores v pah P has exacly n edges. By pigeonhole principle, P mus conain a direced cycle W. Deleing W yields a v pah wih < n edges W has negaive cos. v W CAD 0.9 CHF 7 c(w) < 0 8

13 Deecing negaive cycles Deecing negaive cycles Theorem. Can find a negaive cycle in Θ(mn) ime and Θ(n ) space. Add new node and connec all nodes o wih 0-cos edge. G has a negaive cycle iff G' has a negaive cycle han can reach. If OPT(n, v) = OPT(n, v) for all nodes v, hen no negaive cycles. If no, hen exrac direced cycle from pah from v o. (cycle canno conain since no edges leave ) Theorem. Can find a negaive cycle in O(mn) ime and O(n) exra space. Run Bellman-Ford for n passes (insead of n ) on modified digraph. If no d(v) values updaed in pass n, hen no negaive cycles. Oherwise, suppose d(s) updaed in pass n. Define pass(v) = las pass in which d(v) was updaed. Observe pass(s) = n and pass(successor(v)) pass(v) for each v. Following successor poiners, we mus evenually repea a node. Lemma his cycle is a negaive cycle. 6 G' Remark. See p. 0 for improved version and early erminaion rule. (Tarjan's subree disassembly rick) - 9 0