6. DYNAMIC PROGRAMMING II

Transcription

1 6. DYNAMIC PROGRAMMING II 6. DYNAMIC PROGRAMMING II sequence alignmen Hirschberg s algorihm Bellman Ford Moore algorihm disance-vecor proocols negaive cycles sequence alignmen Hirschberg s algorihm Bellman Ford Moore algorihm disance-vecor proocols negaive cycles SECTION 6.6 Lecure slides by Kevin Wayne Copyrigh 00 Pearson-Addison Wesley hp:// Las updaed on /8/8 7: PM Sring similariy Edi disance Q. How similar are wo srings? Ex. ocurrance and occurrence. o c u r r a n c e o c c u r r e n c e 6 mismaches, gap o c u r r a n c e o c c u r r e n c e 0 mismaches, gaps o c u r r a n c e o c c u r r e n c e mismach, gap Edi disance. [Levenshein 966, Needleman Wunsch 970] Gap penaly δ; mismach penaly α pq. Cos = sum of gap and mismach penalies. C T G A C C T A C G C T G G A C G A A C G cos = δ + α CG + α TA Applicaions. Bioinformaics, spell correcion, machine ranslaion, speech recogniion, informaion exracion,... assuming α AA = α CC = α GG = α TT = 0 Spokesperson confirms senior governmen adviser was found Spokesperson said he senior adviser was found

2 BLOSUM marix for proeins Dynamic programming: quiz Wha is edi disance beween hese wo srings? P A L E T T E P A L A T E Assume gap penaly = and mismach penaly =. A. B. C. D. E. P A L E T T E P A L A T E mismach, gap 6 Sequence alignmen Sequence alignmen: problem srucure Goal. Given wo srings x x... xm and y y... yn, find a min-cos alignmen. Def. OPT(i, j) = min cos of aligning prefix srings x x... xi and y y... yj. Goal. OPT(m, n). Def. An alignmen M is a se of ordered pairs xi yj such ha each characer appears in a mos one pair and no crossings. Case. OPT(i, j) maches xi yj. Pay mismach for xi yj + min cos of aligning x x... xi and y y... yj. xi yj and xiʹ yj cross if i < i, bu j > j ʹ Def. The cos of an alignmen M is: cos(m ) = Case a. OPT(i, j) leaves xi unmached. α xi y j + δ+ δ i : x unmached j : y unmached!##"## $!#i### #"#j#### $ Pay gap for xi + min cos of aligning x x... xi and y y... yj. (x i, y j ) M mismach gap opimal subsrucure propery (proof via exchange argumen) Case b. OPT(i, j) leaves yj unmached. x x x x x C T A C C G T A C A T G y y y y y y6 Pay gap for yj + min cos of aligning x x... xi and y y... yj. x6 Bellman equaion. i=0 i OP T (i, j) = 7 j=0 x i yj min an alignmen of CTACCG and TACATG M = { x y, x y, x y, x y, x6 y6 } j + OP T (i + OP T (i + OP T (i, j <laexi sha_base6="60wunmrldnufwg/nhfwbdvxy=">aaadsichvjdaxnbfjdwqxq6pvgwgpuubdsviirqkvvhmhkyzekynbjjpdxwbuoqlz/ax/vx+ap8hb6jd8ufslhwqsdlpppefonykyjs0gwu/p7qzfffezvggephj/zdu/sglubhrfqljzfxelsmrookqfvkbnkqklqpr9x98gamlak+xkg/yspbxkwdfbgzpfuych8jxdhjiwfuurssgphuhxcidgbtqhlaafnl6kdggkhrzohzdkekodylivdqxjvehzmng0ywslhxljkxnxqtadynpiuna8czrvkhqthowhyykzrrpyftoobjmvnf9omoabzwvqylchu8gfvrbnowtpqkjo7b6t6luenofeobm0vddlsf9ygfkouzykxfzzefrcqnkcl9ubzmnlxws0fqnfikso+cjbmkcsye9iu0rwlovx+fjva6yxg0wbgnc0uxpexhole0ifdnxmkfkwwksbccihu+6yvnozijwbfnncshgsiyqnklhrbd9zjhdvwmhqcj+9bz6dpvcic/icjaqvkonjeppeo6rhhfvk/en++7/8p/f//yyovlfpcursbx9fzpasg=</laexi>, j ), j) ) 8

3 <laexi sha_base6="60wunmrldnufwg/nhfwbdvxy=">aaadsichvjdaxnbfjdwqxq6pvgwgpuubdsviirqkvvhmhkyzekynbjjpdxwbuoqlz/ax/vx+ap8hb6jd8ufslhwqsdlpppefonykyjs0gwu/p7qzfffezvggephj/zdu/sglubhrfqljzfxelsmrookqfvkbnkqklqpr9x98gamlak+xkg/yspbxkwdfbgzpfuych8jxdhjiwfuurssgphuhxcidgbtqhlaafnl6kdggkhrzohzdkekodylivdqxjvehzmng0ywslhxljkxnxqtadynpiuna8czrvkhqthowhyykzrrpyftoobjmvnf9omoabzwvqylchu8gfvrbnowtpqkjo7b6t6luenofeobm0vddlsf9ygfkouzykxfzzefrcqnkcl9ubzmnlxws0fqnfikso+cjbmkcsye9iu0rwlovx+fjva6yxg0wbgnc0uxpexhole0ifdnxmkfkwwksbccihu+6yvnozijwbfnncshgsiyqnklhrbd9zjhdvwmhqcj+9bz6dpvcic/icjaqvkonjeppeo6rhhfvk/en++7/8p/f//yyovlfpcursbx9fzpasg=</laexi> <laexi sha_base6="60wunmrldnufwg/nhfwbdvxy=">aaadsichvjdaxnbfjdwqxq6pvgwgpuubdsviirqkvvhmhkyzekynbjjpdxwbuoqlz/ax/vx+ap8hb6jd8ufslhwqsdlpppefonykyjs0gwu/p7qzfffezvggephj/zdu/sglubhrfqljzfxelsmrookqfvkbnkqklqpr9x98gamlak+xkg/yspbxkwdfbgzpfuych8jxdhjiwfuurssgphuhxcidgbtqhlaafnl6kdggkhrzohzdkekodylivdqxjvehzmng0ywslhxljkxnxqtadynpiuna8czrvkhqthowhyykzrrpyftoobjmvnf9omoabzwvqylchu8gfvrbnowtpqkjo7b6t6luenofeobm0vddlsf9ygfkouzykxfzzefrcqnkcl9ubzmnlxws0fqnfikso+cjbmkcsye9iu0rwlovx+fjva6yxg0wbgnc0uxpexhole0ifdnxmkfkwwksbccihu+6yvnozijwbfnncshgsiyqnklhrbd9zjhdvwmhqcj+9bz6dpvcic/icjaqvkonjeppeo6rhhfvk/en++7/8p/f//yyovlfpcursbx9fzpasg=</laexi> <laexi sha_base6="60wunmrldnufwg/nhfwbdvxy=">aaadsichvjdaxnbfjdwqxq6pvgwgpuubdsviirqkvvhmhkyzekynbjjpdxwbuoqlz/ax/vx+ap8hb6jd8ufslhwqsdlpppefonykyjs0gwu/p7qzfffezvggephj/zdu/sglubhrfqljzfxelsmrookqfvkbnkqklqpr9x98gamlak+xkg/yspbxkwdfbgzpfuych8jxdhjiwfuurssgphuhxcidgbtqhlaafnl6kdggkhrzohzdkekodylivdqxjvehzmng0ywslhxljkxnxqtadynpiuna8czrvkhqthowhyykzrrpyftoobjmvnf9omoabzwvqylchu8gfvrbnowtpqkjo7b6t6luenofeobm0vddlsf9ygfkouzykxfzzefrcqnkcl9ubzmnlxws0fqnfikso+cjbmkcsye9iu0rwlovx+fjva6yxg0wbgnc0uxpexhole0ifdnxmkfkwwksbccihu+6yvnozijwbfnncshgsiyqnklhrbd9zjhdvwmhqcj+9bz6dpvcic/icjaqvkonjeppeo6rhhfvk/en++7/8p/f//yyovlfpcursbx9fzpasg=</laexi> <laexi sha_base6="60wunmrldnufwg/nhfwbdvxy=">aaadsichvjdaxnbfjdwqxq6pvgwgpuubdsviirqkvvhmhkyzekynbjjpdxwbuoqlz/ax/vx+ap8hb6jd8ufslhwqsdlpppefonykyjs0gwu/p7qzfffezvggephj/zdu/sglubhrfqljzfxelsmrookqfvkbnkqklqpr9x98gamlak+xkg/yspbxkwdfbgzpfuych8jxdhjiwfuurssgphuhxcidgbtqhlaafnl6kdggkhrzohzdkekodylivdqxjvehzmng0ywslhxljkxnxqtadynpiuna8czrvkhqthowhyykzrrpyftoobjmvnf9omoabzwvqylchu8gfvrbnowtpqkjo7b6t6luenofeobm0vddlsf9ygfkouzykxfzzefrcqnkcl9ubzmnlxws0fqnfikso+cjbmkcsye9iu0rwlovx+fjva6yxg0wbgnc0uxpexhole0ifdnxmkfkwwksbccihu+6yvnozijwbfnncshgsiyqnklhrbd9zjhdvwmhqcj+9bz6dpvcic/icjaqvkonjeppeo6rhhfvk/en++7/8p/f//yyovlfpcursbx9fzpasg=</laexi> Sequence alignmen: boom-up algorihm Sequence alignmen: raceback SEQUENCE-ALIGNMENT(m, n, x,, xm, y,, yn, δ, α) FOR i = 0 TO m M [i, 0] i δ. FOR j = 0 TO n M [0, j] j δ. FOR i = TO m FOR j = TO n M [i, j] min { α xi y j + M [i, j ], RETURN M [m, n]. δ + M [i, j], δ + M [i, j ] }. already compued S I M I L A R I T Y I D E N T I T Y Sequence alignmen: analysis Dynamic programming: quiz Theorem. The DP algorihm compues he edi disance (and an opimal alignmen) of wo srings of lenghs m and n in Θ(mn) ime and space. Algorihm compues edi disance. Can race back o exrac opimal alignmen iself. Theorem. [Backurs Indyk 0] If can compue edi disance of wo srings of lengh n in O(n ) ime for some consan > 0, hen can solve SAT wih n variables and m clauses in poly(m) ( δ) n ime for some consan δ > 0. Edi Disance Canno Be Compued in Srongly Subquadraic Time (unless SETH is false) Arurs Backurs MIT Pior Indyk MIT which would disprove SETH (srong exponenial ime hypohesis) I is easy o modify he DP algorihm for edi disance o A. Compue edi disance in O(mn) ime and O(m + n) space. B. Compue an opimal alignmen in O(mn) ime and O(m + n) space. C. Boh A and B. D. Neiher A nor B. j i =0 i j =0 OPT(i, j) = x iy j + OPT(i,j ) min + OPT(i,j) + OPT(i, j )

4 Sequence alignmen in linear space Theorem. [Hirschberg] There exiss an algorihm o find an opimal 6. D YNAMIC P ROGRAMMING II alignmen in O(mn) ime and O(m + n) space. Clever combinaion of divide-and-conquer and dynamic programming. sequence alignmen Inspired by idea of Savich from complexiy heory. Hirschberg s algorihm Bellman Ford Moore algorihm A = a x a... a m if and only if here is a mapping F: {,,..., p} ~ {,,..., m} such ha f(i) = k only if c~ is ak and F is a m o n o o n e sricly increasing funcion (i.e. F(i) = u, F ( j ) = v, and i < j imply ha disance-vecor proocols Programming Techniques u<v). G. Manacher Edior A Linear Space Algorihm for Compuing Maximal Common Subsequences negaive cycles D.S. Hirschberg Princeon Universiy SECTION 6.7 The problem of finding a longes common subsequence of wo srings has been solved in quadraic ime and space. An algorihm is presened which will solve his problem in quadraic ime and in linear space. Key Words and Phrases: subsequence, longes common subsequence, sring correcion, ediing CR Caegories:.6,.7,.79,.,. Inroducion Hirschberg s algorihm Hirschberg s algorihm Edi disance graph. Edi disance graph. The problem of finding a longes c o m m o n subsequence of wo srings has been solved in quadraic ime and space [, ]. F o r srings of lengh,000 (assuming coefficiens of microsecond and bye) he soluion would require 06 microseconds (one second) and 06 byes (000K byes). The former is easily accommodaed, he laer is no so easily obainable. I f he srings were of lengh 0,000, he problem migh no be solvable in main m e m o r y for lack of space. We presen an algorihm which will solve his problem in quadraic ime and in linear space. For example, assuming coefficiens of microseconds and 0 byes, for srings of lengh,000 we would require seconds and 0K byes; for srings of lengh 0,000 we would require a lile over minues and 00K byes. Sring C = c~c...cp is a subsequence of sring Copyrigh 97, Associaion for Compuing Machinery, Inc. General permission o republish, bu no for profi, all or par of his maerial is graned provided ha ACM's copyrigh noice is given and ha reference is made o he publicaion, o is dae of issue, and o he fac ha reprining privileges were graned by permission of he Associaion for Compuing Machinery. Research work was suppored in par by NSF gran GJ-06 and Naional Science Foundaion Graduae Felolwship. Auhor's address: Deparmen of Elecrical Engineering, Princeon Universiy, Princeon, NJ 080. Sring C is a c o m m o n subsequence of srings A and B if and only if C is a subsequence of A and C is a subsequence of B. The problem can be saed as follows: Given srings A = aia... "am and B = b x b... b n (over alphabe Z), find a sring C = ClC...cp such ha C, is a c o m m o n subsequence of A and B and p is maximized. We call C an example of a m a x i m a l c o m m o n subsequence. Noaion. F o r sring D = dld. dr, Dk is dkdk+l. d, i f k < ; d k d k _ x... d, i f k >. When k >, we shall wrie ]k so as o make clear ha we are referring o a "reverse subsring" of D. L(i, j ) is he m a x i m u m lengh possible of any common subsequence of Ax~ and B~s. x[ ly is he concaenaion of srings x and y. We presen he algorihm described in [], which akes quadraic ime and space. Algorihm A Algorihm A acceps as inpu srings A~m and Bx. and produces as oupu he marix L (where he elemen L(i, j ) corresponds o our noaion of m a x i m u m lengh possible of any c o m m o n subsequence of Axl and B.). ALGA (m, n, A, B, L). Iniializaion: L(i, 0) ~ 0 [i=0...m]; L(O,j) [j=0...n]; for i +-- o m do begin. for j ~- o n do if A (i) = B(j) hen L(i, j) ~- L(i--, j - - ) "k else L(i,j) ~-- max{l(i,j--), L(i-- I,j)} end Proof of Correcness of Algorihm A To find L(i, j), le a c o m m o n subsequence of ha lengh be denoed by S(i, j ) = ClC...cp. I f al = bj, we can do no beer han by aking cp = a~ and looking for c l... c p _ l as a c o m m o n subsequence of lengh L(i, j) -- of srings AI,~- and B.i-x. Thus, in his case, L ( i, j ) = L ( i,j) -+-. If ai ~ bs, hen cp is ai, b;, or neiher (bu no boh). I f cp is a~, hen a soluion C o problem (A~, B~j) [wrien P(i, j)] will be a soluion o P(i, j - ) since bj is no used. Similarly, if cp is bi, hen we can ge a soluion o P(i, j ) by solving P ( i --, j ). I f c~ is neiher, hen a soluion o eiher P ( i --,j) or P ( i, j -- ) will suffice. In deermining he lengh of he soluion, i is seen ha L(i, j ) [corresponding o P(i, j)] will be he ). [] m a x i m u m o f L ( i - -, j ) and L ( i, j - - Le f (i, j) denoe lengh of shores pah from (0,0) o (i, j). Le f (i, j) denoe lengh of shores pah from (0,0) o (i, j). Lemma: f (i, j) = OPT(i, j) for all i and j. Lemma: f (i, j) = OPT(i, j) for all i and j. Pf of Lemma. [ by srong inducion on i + j ]. Communicaions of he ACM June 97 Volume 8 Number 6 Base case: f (0, 0) = OPT (0, 0) = 0. y y y y y y6 Inducive hypohesis: assume rue for all (iʹ, jʹ) wih iʹ + jʹ < i + j. Las edge on shores pah o (i, j) is from (i, j ), (i, j), or (i, j ). 0 0 Thus, x α xi y j x x δ ff (i, (i, j) j) = = min{ min{ x xii y yjj + + ff (i (i = = min{ min{ xii y yjj x + + OP OP T T (i (i δ inducive hypohesis,, jj ), ),,, jj + + ff (i (i ), ), + + OP OP T T (i (i + + ff (i, (i, jj,, j), j), )} )} + + OP OP T T (i, (i, jj )} )} = OP OP T T (i, (i, j) j) = α xi y j Bellman equaion m n,, j), j), δ δ 6

5 Hirschberg s algorihm Hirschberg s algorihm Edi disance graph. Le f (i, j) denoe lengh of shores pah from (0,0) o (i, j). Lemma: f (i, j) = OPT(i, j) for all i and j. Can compue f (, j) for any j in O(mn) ime and O(m + n) space. Edi disance graph. Le g(i, j) denoe lengh of shores pah from (i, j) o (m, n). j y y y y y y 6 y y y y y y x x x x x m n x m n 7 8 Hirschberg s algorihm Hirschberg s algorihm Edi disance graph. Edi disance graph. Le g(i, j) denoe lengh of shores pah from (i, j) o (m, n). Le g(i, j) denoe lengh of shores pah from (i, j) o (m, n). Can compue g(i, j) by reversing he edge orienaions and invering he roles of (0, 0) and (m, n). Can compue g(, j) for any j in O(mn) ime and O(m + n) space. j y y y y y y 6 y y y y y y x δ x δ x i+y j+ x x x m n x m n 9 0

6 Hirschberg s algorihm Hirschberg s algorihm Observaion. The lengh of a shores pah ha uses (i, j) is f (i, j) + g(i, j). Observaion. le q be an index ha minimizes f(q, n / ) + g(q, n / ). Then, here exiss a shores pah from (0, 0) o (m, n) ha uses (q, n / ). n / y y y y y y 6 y y y y y y x x q x x x m n x m n Hirschberg s algorihm Hirschberg s algorihm: space analysis Divide. Find index q ha minimizes f (q, n / ) + g(q, n / ); save node as par of soluion. Conquer. Recursively compue opimal alignmen in each piece. y n / y y y y y 6 Theorem. Hirschberg s algorihm uses Θ(m + n) space. Each recursive call uses Θ(m) space o compue f (, n / ) and g(, n / ). Only Θ() space needs o be mainained per recursive call. Number of recursive calls n. 0 0 x q x x m n

7 Dynamic programming: quiz Hirschberg s algorihm: running ime analysis warmup Wha is he wors-case running ime of Hirschberg s algorihm? A. O(mn) B. O(mn log m) C. O(mn log n) D. O(mn log m log n) Theorem. Le T(m, n) = max running ime of Hirschberg s algorihm on srings of lenghs a mos m and n. Then, T(m, n) = O(m n log n). T(m, n) is monoone nondecreasing in boh m and n. T(m, n) T(m, n / ) + O(m n) T(m, n) = O(m n log n). Remark. Analysis is no igh because wo subproblems are of size (q, n / ) and (m q, n / ). Nex, we prove T(m, n) = O(m n). 6 Hirschberg s algorihm: running ime analysis LONGEST COMMON SUBSEQUENCE Theorem. Le T(m, n) = max running ime of Hirschberg s algorihm on srings of lenghs a mos m and n. Then, T(m, n) = O(m n). [ by srong inducion on m + n ] O(m n) ime o compue f (, n / ) and g (, n / ) and find index q. T(q, n / ) + T(m q, n / ) ime for wo recursive calls. Choose consan c so ha: Claim. T(m, n) c m n. Base cases: m = and n =. T(m, ) c m T(, n) c n T(m, n) c m n + T(q, n / ) + T(m q, n / ) Inducive hypohesis: T(m, n) c m n for all (mʹ, nʹ) wih mʹ + nʹ < m + n. Problem. Given wo srings x x... x m and y y... y n, find a common subsequence ha is as long as possible. Alernaive viewpoin. Delee some characers from x ; delee some characer from y ; a common subsequence if i resuls in he same sring. Ex. LCS(GGCACCACG, ACGGCGGATACG ) = GGCAACG. Applicaions. Unix diff, gi, bioinformaics. T(m, n) T(q, n / ) + T(m q, n / ) + c m n c q n / + c (m q) n / + c m n inducive hypohesis = c q n + c m n c q n + c m n = c m n 7 8

8 <laexi sha_base6="rsprsrjqxplkln8jdnfdjbxfc=">aaacwhicbvdlssnafjer+u7rusg0xqtueskkcmzlbwufpotj9fyhjmwcymoz/h7jvjxb/xknbhaemmvhnpuye6jucooe9+ws8srq6xyvrgbtqvaezbjpjm0esit/rgxaioaknacy+pbhzhejrry0hdbggoexyl0ivzkxidwrlakqwcbydlyeeibhf0qkibyeiwbxoirtjwsghfmjlkbywuvcaitox+dpqjmohvwnvrqtgwg0iumtfd0uxlzongksyrwezgztxf/yexqsvi8h08sllypgmtdjno+hxtc/u7iwwzmkiszczwsxqbfmfsw0ozlqquzgultrynmukxoyrpcw0ccgcxrwwf6x8mwng0zouyowu+d0k+zjtgsr8wwildgxsxfqxpfslien8z/dq/bs7sljm9sk8oiu/oydwjssjpy8kxfyqt6db9dxs+7anrzj7zc7cg+aell</laexi> <laexi sha_base6="rsprsrjqxplkln8jdnfdjbxfc=">aaacwhicbvdlssnafjer+u7rusg0xqtueskkcmzlbwufpotj9fyhjmwcymoz/h7jvjxb/xknbhaemmvhnpuye6jucooe9+ws8srq6xyvrgbtqvaezbjpjm0esit/rgxaioaknacy+pbhzhejrry0hdbggoexyl0ivzkxidwrlakqwcbydlyeeibhf0qkibyeiwbxoirtjwsghfmjlkbywuvcaitox+dpqjmohvwnvrqtgwg0iumtfd0uxlzongksyrwezgztxf/yexqsvi8h08sllypgmtdjno+hxtc/u7iwwzmkiszczwsxqbfmfsw0ozlqquzgultrynmukxoyrpcw0ccgcxrwwf6x8mwng0zouyowu+d0k+zjtgsr8wwildgxsxfqxpfslien8z/dq/bs7sljm9sk8oiu/oydwjssjpy8kxfyqt6db9dxs+7anrzj7zc7cg+aell</laexi> <laexi sha_base6="rsprsrjqxplkln8jdnfdjbxfc=">aaacwhicbvdlssnafjer+u7rusg0xqtueskkcmzlbwufpotj9fyhjmwcymoz/h7jvjxb/xknbhaemmvhnpuye6jucooe9+ws8srq6xyvrgbtqvaezbjpjm0esit/rgxaioaknacy+pbhzhejrry0hdbggoexyl0ivzkxidwrlakqwcbydlyeeibhf0qkibyeiwbxoirtjwsghfmjlkbywuvcaitox+dpqjmohvwnvrqtgwg0iumtfd0uxlzongksyrwezgztxf/yexqsvi8h08sllypgmtdjno+hxtc/u7iwwzmkiszczwsxqbfmfsw0ozlqquzgultrynmukxoyrpcw0ccgcxrwwf6x8mwng0zouyowu+d0k+zjtgsr8wwildgxsxfqxpfslien8z/dq/bs7sljm9sk8oiu/oydwjssjpy8kxfyqt6db9dxs+7anrzj7zc7cg+aell</laexi> <laexi sha_base6="rsprsrjqxplkln8jdnfdjbxfc=">aaacwhicbvdlssnafjer+u7rusg0xqtueskkcmzlbwufpotj9fyhjmwcymoz/h7jvjxb/xknbhaemmvhnpuye6jucooe9+ws8srq6xyvrgbtqvaezbjpjm0esit/rgxaioaknacy+pbhzhejrry0hdbggoexyl0ivzkxidwrlakqwcbydlyeeibhf0qkibyeiwbxoirtjwsghfmjlkbywuvcaitox+dpqjmohvwnvrqtgwg0iumtfd0uxlzongksyrwezgztxf/yexqsvi8h08sllypgmtdjno+hxtc/u7iwwzmkiszczwsxqbfmfsw0ozlqquzgultrynmukxoyrpcw0ccgcxrwwf6x8mwng0zouyowu+d0k+zjtgsr8wwildgxsxfqxpfslien8z/dq/bs7sljm9sk8oiu/oydwjssjpy8kxfyqt6db9dxs+7anrzj7zc7cg+aell</laexi> Shores pahs wih negaive weighs 6. DYNAMIC PROGRAMMING II Shores-pah problem. Given a digraph G = (V, E), wih arbirary edge lenghs vw, find shores pah from source node s o desinaion node. sequence alignmen assume here exiss a pah from every node o Hirschberg s algorihm Bellman Ford Moore algorihm disance-vecor proocols negaive cycles s SECTION lengh of shores pah from s o = = Shores pahs wih negaive weighs: failed aemps Negaive cycles Dijksra. May no produce shores pahs when edge lenghs are negaive. s Reweighing. Adding a consan o every edge lengh does no necessarily make Dijksra s algorihm produce shores pahs. 6 v 8 w Dijksra selecs he verices in he order s,, w, v Bu shores pah from s o is s v w. Def. A negaive cycle is a direced cycle for which he sum of is edge lenghs is negaive. s v 0 0 Adding 8 o each edge weigh changes he shores pah from s v w o s. a negaive cycle W : (W ) = e W e < 0 w

9 Shores pahs and negaive cycles Shores pahs and negaive cycles Lemma. If some v pah conains a negaive cycle, hen here does no exis a shores v pah. If here exiss such a cycle W, hen can build a v pah of arbirarily negaive lengh by deouring around W as many imes as desired. Lemma. If G has no negaive cycles, hen here exiss a shores v pah ha is simple (and has n edges). Among all shores v pahs, consider one ha uses he fewes edges. If ha pah P conains a direced cycle W, can remove he porion of P corresponding o W wihou increasing is lengh. v v W W (W) < 0 (W) 0 6 Shores-pahs and negaive-cycle problems Dynamic programming: quiz Single-desinaion shores-pahs problem. Given a digraph G = (V, E) wih edge lenghs vw (bu no negaive cycles) and a disinguished node, find a shores v pah for every node v. Negaive-cycle problem. Given a digraph G = (V, E) wih edge lenghs vw, find a negaive cycle (if one exiss). Which subproblems o find shores v pahs for every node v? A. OPT(i, v) = lengh of shores v pah ha uses exacly i edges. B. OPT(i, v) = lengh of shores v pah ha uses a mos edges. C. Neiher A nor B. shores-pahs ree negaive cycle 7 8

10 <laexi sha_base6="xucluoowywddz7b+9lleqh6s=">aaadvicjvjdaxnbfjnmb0ay++nixkbfjlwchvapiocbezqkalhdvzumnrdjszmyys+rx+gl/v/tx6gwawppiwfu+eec+fsqkzkyz/wuvurf+db7wh90emntfgvunjs0xzpzarpqzqcov9k6ze80wjs0kjz+hfpj+nkvrkpo7dzdyclgssscm+ugucnrf+deqbpq+bdo9o0dhavclfuloq068mrobznfgoqmsxawbhxbtkcomdrna6abzgmnqqahqsz/nqqvnxqj0iblrg7adahqxj0fps+8qkmguitcmlhrmrbhylms+lo9mxseaeupr8x0yhg0ge8cywhlrula8fs6+0tlffqzhge0/y8lpgttjqnou0aptxpufkumtgdwm/ssgdaci7rzsbzbi/ygmcufaxbmwwp7xbbx0sarxqjlczrvxufsyzj6fjjsxozoasbp8g+qphwwqmwrcvvdopzctaf8pjajdryk+euyvwlxxhgngrbka6csdfi76kmovk8dtcdvtamdwstdhyzgy7oxxcfxo8o98/hwlwenpccvsiseamjl9ij/qj977p7yfq/kdevdadau6fwvjxmgvmraumpcdqa==</laexi> <laexi sha_base6="xucluoowywddz7b+9lleqh6s=">aaadvicjvjdaxnbfjnmb0ay++nixkbfjlwchvapiocbezqkalhdvzumnrdjszmyys+rx+gl/v/tx6gwawppiwfu+eec+fsqkzkyz/wuvurf+db7wh90emntfgvunjs0xzpzarpqzqcov9k6ze80wjs0kjz+hfpj+nkvrkpo7dzdyclgssscm+ugucnrf+deqbpq+bdo9o0dhavclfuloq068mrobznfgoqmsxawbhxbtkcomdrna6abzgmnqqahqsz/nqqvnxqj0iblrg7adahqxj0fps+8qkmguitcmlhrmrbhylms+lo9mxseaeupr8x0yhg0ge8cywhlrula8fs6+0tlffqzhge0/y8lpgttjqnou0aptxpufkumtgdwm/ssgdaci7rzsbzbi/ygmcufaxbmwwp7xbbx0sarxqjlczrvxufsyzj6fjjsxozoasbp8g+qphwwqmwrcvvdopzctaf8pjajdryk+euyvwlxxhgngrbka6csdfi76kmovk8dtcdvtamdwstdhyzgy7oxxcfxo8o98/hwlwenpccvsiseamjl9ij/qj977p7yfq/kdevdadau6fwvjxmgvmraumpcdqa==</laexi> <laexi sha_base6="xucluoowywddz7b+9lleqh6s=">aaadvicjvjdaxnbfjnmb0ay++nixkbfjlwchvapiocbezqkalhdvzumnrdjszmyys+rx+gl/v/tx6gwawppiwfu+eec+fsqkzkyz/wuvurf+db7wh90emntfgvunjs0xzpzarpqzqcov9k6ze80wjs0kjz+hfpj+nkvrkpo7dzdyclgssscm+ugucnrf+deqbpq+bdo9o0dhavclfuloq068mrobznfgoqmsxawbhxbtkcomdrna6abzgmnqqahqsz/nqqvnxqj0iblrg7adahqxj0fps+8qkmguitcmlhrmrbhylms+lo9mxseaeupr8x0yhg0ge8cywhlrula8fs6+0tlffqzhge0/y8lpgttjqnou0aptxpufkumtgdwm/ssgdaci7rzsbzbi/ygmcufaxbmwwp7xbbx0sarxqjlczrvxufsyzj6fjjsxozoasbp8g+qphwwqmwrcvvdopzctaf8pjajdryk+euyvwlxxhgngrbka6csdfi76kmovk8dtcdvtamdwstdhyzgy7oxxcfxo8o98/hwlwenpccvsiseamjl9ij/qj977p7yfq/kdevdadau6fwvjxmgvmraumpcdqa==</laexi> <laexi sha_base6="xucluoowywddz7b+9lleqh6s=">aaadvicjvjdaxnbfjnmb0ay++nixkbfjlwchvapiocbezqkalhdvzumnrdjszmyys+rx+gl/v/tx6gwawppiwfu+eec+fsqkzkyz/wuvurf+db7wh90emntfgvunjs0xzpzarpqzqcov9k6ze80wjs0kjz+hfpj+nkvrkpo7dzdyclgssscm+ugucnrf+deqbpq+bdo9o0dhavclfuloq068mrobznfgoqmsxawbhxbtkcomdrna6abzgmnqqahqsz/nqqvnxqj0iblrg7adahqxj0fps+8qkmguitcmlhrmrbhylms+lo9mxseaeupr8x0yhg0ge8cywhlrula8fs6+0tlffqzhge0/y8lpgttjqnou0aptxpufkumtgdwm/ssgdaci7rzsbzbi/ygmcufaxbmwwp7xbbx0sarxqjlczrvxufsyzj6fjjsxozoasbp8g+qphwwqmwrcvvdopzctaf8pjajdryk+euyvwlxxhgngrbka6csdfi76kmovk8dtcdvtamdwstdhyzgy7oxxcfxo8o98/hwlwenpccvsiseamjl9ij/qj977p7yfq/kdevdadau6fwvjxmgvmraumpcdqa==</laexi> Shores pahs wih negaive weighs: dynamic programming Shores pahs wih negaive weighs: implemenaion Def. OPT(i, v) = lengh of shores v pah ha uses i edges. Goal. OPT(n, v) for each v. Case. Shores v pah uses i edges. OPT(i, v) = OPT(i, v). Case. Shores v pah uses exacly i edges. if (v, w) is firs edge in shores such v pah, incur a cos of vw. Then, selec bes w pah using i edges. Bellman equaion. by Lemma, if no negaive cycles, here exiss a shores v pah ha is simple opimal subsrucure propery (proof via exchange argumen) 0 i =0 v = SHORTEST-PATHS(V, E,, ) FOREACH node v V : M [0, v]. M [0, ] 0. FOR i = TO n FOREACH node v V : M [i, v] M [i, v]. FOREACH edge (v, w) E : M [i, v] min { M [i, v], M [i, w] + vw }. OPT(i, v) = i =0 v = min OPT(i,v), min (v,w) E {OPT(i,w)+ vw} i>0 9 0 Shores pahs wih negaive weighs: implemenaion Dynamic programming: quiz 6 Theorem. Given a digraph G = (V, E) wih no negaive cycles, he DP algorihm compues he lengh of a shores v pah for every node v in Θ(mn) ime and Θ(n ) space. Table requires Θ(n ) space. Each ieraion i akes Θ(m) ime since we examine each edge once. Finding he shores pahs. Approach : Mainain successor[i, v] ha poins o nex node on a shores v pah using i edges. Approach : Compue opimal lenghs M[i, v] and consider only edges wih M[i, v] = M[i, w] + vw. Any direced pah in his subgraph is a shores pah. I is easy o modify he DP algorihm for shores pahs o A. Compue lenghs of shores pahs in O(mn) ime and O(m + n) space. B. Compue shores pahs in O(mn) ime and O(m + n) space. C. Boh A and B. D. Neiher A nor B.

11 Shores pahs wih negaive weighs: pracical improvemens Bellman Ford Moore: efficien implemenaion Space opimizaion. Mainain wo D arrays (insead of D array). d[v] = lengh of a shores v pah ha we have found so far. successor[v] = nex node on a v pah. Performance opimizaion. If d[w] was no updaed in ieraion i, hen no reason o consider edges enering w in ieraion i. BELLMAN FORD MOORE(V, E, c, ) FOREACH node v V : d[v]. successor[v] null. d[] 0. FOR i = TO n FOREACH node w V : IF (d[w] was updaed in previous pass) FOREACH edge (v, w) E : IF (d[v] > d[w] + vw ) pass i O(m) ime d[v] d[w] + vw. successor[v] w. IF (no d[ ] value changed in pass i) STOP. Dynamic programming: quiz 7 Bellman Ford Moore: analysis Which properies mus hold afer pass i of Bellman Ford Moore? A. d[v] = lengh of a shores v pah using i edges. B. d[v] = lengh of a shores v pah using exacly i edges. C. Boh A and B. D. Neiher A nor B. d[v] = v d[w] = w if node w considered before node v, hen d[v] = afer pass d[] = 0 Lemma. For each node v : d[v] is he lengh of some v pah. Lemma. For each node v : d[v] is monoone non-increasing. Lemma. Afer pass i, d[v] lengh of a shores v pah using i edges. [ by inducion on i ] Base case: i = 0. Assume rue afer pass i. Le P be any v pah wih i + edges. Le (v, w) be firs edge in P and le Pʹ be subpah from w o. By inducive hypohesis, a he end of pass i, d[w] c(pʹ) because Pʹ is a w pah wih i edges. Afer considering edge (v, w) in pass i + : and by Lemma, d[v] does no increase d[v] vw + d[w] vw + c(pʹ) = (P) and by Lemma, d[w] does no increase 6

12 Bellman Ford Moore: analysis Dynamic programming: quiz 8 Theorem. Assuming no negaive cycles, Bellman Ford Moore compues he lenghs of he shores v pahs in O(mn) ime and Θ(n) exra space. Lemma + Lemma. shores pah exiss and has a mos n edges afer i passes, d[v] lengh of shores pah ha uses i edges Remark. Bellman Ford Moore is ypically faser in pracice. Edge (v, w) considered in pass i + only if d[w] updaed in pass i. If shores pah has k edges, hen algorihm finds i afer k passes. Assuming no negaive cycles, which properies mus hold hroughou Bellman Ford Moore? A. Following successor[v] poiners gives a direced v pah. B. If following successor[v] poiners gives a direced v pah, hen he lengh of ha v pah is d[v]. C. Boh A and B. D. Neiher A nor B. 7 8 Bellman Ford Moore: analysis Bellman Ford Moore: analysis Claim. Throughou Bellman Ford Moore, following he successor[v] poiners gives a direced pah from v o of lengh d[v]. Counerexample. Claim is false! Lengh of successor v pah may be sricly shorer han d[v]. Claim. Throughou Bellman Ford Moore, following he successor[v] poiners gives a direced pah from v o of lengh d[v]. Counerexample. Claim is false! Lengh of successor v pah may be sricly shorer han d[v]. consider nodes in order:,,, consider nodes in order:,,, successor[] = d[] = 0 successor[] = d[] = 0 d[] = 0 successor[] = successor[] = d[] = 0 d[] = d[] = successor[] = d[] = 9 successor[] = d[] = 0

13 Bellman Ford Moore: analysis Bellman Ford Moore: analysis Claim. Throughou Bellman Ford Moore, following he successor[v] poiners gives a direced pah from v o of lengh d[v]. Counerexample. Claim is false! Lengh of successor v pah may be sricly shorer han d[v]. If negaive cycle, successor graph may have direced cycles. Claim. Throughou Bellman Ford Moore, following he successor[v] poiners gives a direced pah from v o of lengh d[v]. Counerexample. Claim is false! Lengh of successor v pah may be sricly shorer han d[v]. If negaive cycle, successor graph may have direced cycles. consider nodes in order:,,,, consider nodes in order:,,,, d[] = 0 d[] = 8 d[] = 0 d[] = 8 9 d[] = 0 9 d[] = d[] = d[] = d[] = d[] = Bellman Ford Moore: finding he shores pahs Bellman Ford Moore: finding he shores pahs Lemma 6. Any direced cycle W in he successor graph is a negaive cycle. If successor[v] = w, we mus have d[v] d[w] + vw. (LHS and RHS are equal when successor[v] is se; d[w] can only decrease; d[v] decreases only when successor[v] is rese) Le v v vk v be he sequence of nodes in a direced cycle W. Assume ha (v k, v ) is he las edge in W added o he successor graph. Jus prior o ha: d[v ] d[v ] + (v, v ) d[v ] d[v ] + (v, v ) d[vk ] d[vk] + (vk, vk) d[vk] > d[v] + (vk, v) holds wih sric inequaliy since we are updaing d[vk] Adding inequaliies yields (v, v) + (v, v) + + (vk, vk) + (vk, v) < 0. Theorem. Assuming no negaive cycles, Bellman Ford Moore finds shores v pahs for every node v in O(mn) ime and Θ(n) exra space. The successor graph canno have a direced cycle. [Lemma 6] Thus, following he successor poiners from v yields a direced pah o. Le v = v v v k = be he nodes along his pah P. Upon erminaion, if successor[v] = w, we mus have d[v] = d[w] + vw. (LHS and RHS are equal when successor[v] is se; d[ ] did no change) Thus, d[v ] = d[v ] + (v, v ) d[v] = d[v] + (v, v) d[vk ] = d[vk] + (vk, vk) since algorihm erminaed Adding equaions yields d[v] = d[] + (v, v) + (v, v) + + (vk, vk). W is a negaive cycle min lengh of any v pah (Theorem ) 0 lengh of pah P

14 Single-source shores pahs wih negaive weighs year wors case discovered by 9 O(n ) Shimbel 96 O(m n W) Ford 98 O(m n) Bellman, Moore 98 O(n / m log W) Gabow 989 O(m n / log(nw)) Gabow Tarjan 99 O(m n / log W) Goldberg 6. DYNAMIC PROGRAMMING II sequence alignmen Hirschberg s algorihm Bellman Ford Moore algorihm disance-vecor proocols negaive cycles 00 O(n.8 W) Sankowsi, Yuser Zwick 06 Õ(n 0/7 log W) Cohen Mądry Sankowski Vladu 0xx SECTION 6.9 single-source shores pahs wih weighs beween W and W Disance-vecor rouing proocols Disance-vecor rouing proocols Communicaion nework. Node rouer. Edge direc communicaion link. Lengh of edge laency of link. non-negaive, bu Bellman Ford Moore used anyway! Dijksra s algorihm. Requires global informaion of nework. Bellman Ford Moore. Uses only local knowledge of neighboring nodes. Synchronizaion. We don expec rouers o run in locksep. The order in which each edges are processed in Bellman Ford Moore is no imporan. Moreover, algorihm converges even if updaes are asynchronous. Disance-vecor rouing proocols. [ rouing by rumor ] Each rouer mainains a vecor of shores-pah lenghs o every oher node (disances) and he firs hop on each pah (direcions). Algorihm: each rouer performs n separae compuaions, one for each poenial desinaion node. Ex. RIP, Xerox XNS RIP, Novell s IPX RIP, Cisco s IGRP, DEC s DNA Phase IV, AppleTalk s RTMP. Cavea. Edge lenghs may change during algorihm (or fail compleely). suppose his edge ges deleed s v d(s) = d(v) = d() = 0 7 couning o infiniy 8

15 Pah-vecor rouing proocols Link-sae rouing proocols. Each rouer sores he whole nework opology. Based on Dijksra s algorihm. Avoids couning-o-infiniy problem and relaed difficulies. Requires significanly more sorage. no jus he disance and firs hop Ex. Border Gaeway Proocol (BGP), Open Shores Pah Firs (OSPF). 6. DYNAMIC PROGRAMMING II sequence alignmen Hirschberg s algorihm Bellman Ford Moore algorihm disance vecor proocol negaive cycles SECTION Deecing negaive cycles Deecing negaive cycles: applicaion Negaive cycle deecion problem. Given a digraph G = (V, E), wih edge lenghs vw, find a negaive cycle (if one exiss). Currency conversion. Given n currencies and exchange raes beween pairs of currencies, is here an arbirage opporuniy? Remark. Fases algorihm very valuable! 0.7 *.66 *.99 = EUR USD GBP CAD CHF 6 6

16 Deecing negaive cycles Deecing negaive cycles Lemma 7. If OPT(n, v) = OPT(n, v) for every node v, hen no negaive cycles. The OPT(n, v) values have converged shores v pah exiss. Lemma 8. If OPT(n, v) < OPT(n, v) for some node v, hen (any) shores v pah of lengh n conains a cycle W. Moreover W is a negaive cycle. [by conradicion] Since OPT(n, v) < OPT(n, v), we know ha shores v pah P has exacly n edges. By pigeonhole principle, he pah P mus conain a repeaed node x. Le W be any cycle in P. Deleing W yields a v pah wih < n edges W is a negaive cycle. Theorem. Can find a negaive cycle in Θ(mn) ime and Θ(n ) space. Add new sink node and connec all nodes o wih 0-lengh edge. G has a negaive cycle iff Gʹ has a negaive cycle. Case. [ OPT(n, v) = OPT(n, v) for every node v ] By Lemma 7, no negaive cycles. Case. [ OPT(n, v) < OPT(n, v) for some node v ] Using proof of Lemma 8, can exrac negaive cycle from v pah. (cycle canno conain since no edge leaves ) 6 0 G v x W c(w) < Deecing negaive cycles Dynamic programming: quiz 9 Theorem. Can find a negaive cycle in O(mn) ime and O(n) exra space. Run Bellman Ford Moore on Gʹ for nʹ = n + passes (insead of nʹ ). If no d[v] values updaed in pass nʹ, hen no negaive cycles. Oherwise, suppose d[s] updaed in pass nʹ. Define pass(v) = las pass in which d[v] was updaed. Observe pass(s) = nʹ and pass(successor[v]) pass(v) for each v. Following successor poiners, we mus evenually repea a node. Lemma 6 he corresponding cycle is a negaive cycle. Remark. See p. 0 for improved version and early erminaion rule. (Tarjan s subree disassembly rick) How difficul o find a negaive cycle in an undireced graph? A. O(m log n) B. O(mn) C. O(mn + n log n) D. O(n.8 ) E. No poly-ime algorihm is known. Daa Srucures for Weighed Maching and Neares Common Ancesors wih Linking Harold N. Gabow* Absrac. This paper shows ha he weighed maching problem on general graphs can be solved in ime O(n(m + n log n)), f or n and m he number of verices and edges, respecively. This was previously known only for biparie graphs. I also shows ha a sequence of m nca and link operaions on n nodes can be processed on-line in ime O(ma(m, n)+n). This was previously known only for a resriced ype of link operaion. 6 66