Desgn and Analyss of Algorthms CSE 53 Lecture 4 Dynamc Programmng Junzhou Huang, Ph.D. Department of Computer Scence and Engneerng CSE53 Desgn and Analyss of Algorthms
The General Dynamc Programmng Technque Apples to a problem that at frst seems to requre a lot of tme (possbly exponental), provded we have: Subproblem optmalty: the global optmum value can be defned n terms of optmal subproblems Subproblem overlap: the subproblems are not ndependent, but nstead they overlap (hence, should be constructed bottom-up). CSE53 Desgn and Analyss of Algorthms
Longest Common Subsequence Problem: Gven sequences, X = x,...,x m and Y = y,...,y n, fnd a common subsequence whose length s maxmum. sprngtme ncaa tournament basketball prntng north carolna krzyzewsk Subsequence need not be consecutve, but must be n order. CSE53 Desgn and Analyss of Algorthms 3
Other Sequence Questons Edt dstance: Gven sequences, X = x,...,x m and Y = y,...,y n, what s the mnmum number of deletons, nsertons, and changes that you must do to change one to another? Proten sequence algnment: Gven a score matrx on amno acd pars, s(a,b) for a,b {Λ} A, and amno acd sequences, X = x,...,x m A m and Y = y,...,y n A n, fnd the algnment wth lowest score CSE53 Desgn and Analyss of Algorthms 4
More Problems Optmal ST: Gven sequence K = k < k < < k n of n sorted keys, wth a search probablty p for each key k, buld a bnary search tree (ST) wth mnmum expected search cost. Mnmum convex decomposton of a polygon, Hydrogen placement n proten structures, CSE53 Desgn and Analyss of Algorthms 5
Dynamc Programmng Dynamc Programmng s an algorthm desgn technque for optmzaton problems: often mnmzng or maxmzng. Lke dvde and conquer, DP solves problems by combnng solutons to subproblems. Unlke dvde and conquer, subproblems are not ndependent. Subproblems may share subsubproblems, However, soluton to one subproblem may not affect the solutons to other subproblems of the same problem. (More on ths later.) DP reduces computaton by Solvng subproblems n a bottom-up fashon. Storng soluton to a subproblem the frst tme t s solved. Lookng up the soluton when subproblem s encountered agan. Key: determne structure of optmal solutons CSE53 Desgn and Analyss of Algorthms 6
Recallng: Steps n Dynamc Programmng. Characterze structure of an optmal soluton.. Defne value of optmal soluton recursvely. 3. Compute optmal soluton values ether top-down wth cachng or bottom-up n a table. 4. Construct an optmal soluton from computed values. CSE53 Desgn and Analyss of Algorthms 7
Naïve Algorthm sprngtme ncaa tournament basketball prntng north carolna krzyzewsk For every subsequence of X = x,...,x m, check whether t s a subsequence of Y = y,...,y n. Tme: Θ(n m ). m subsequences of X to check. Each subsequence takes Θ(n) tme to check: scan Y for frst letter, for second, and so on. CSE53 Desgn and Analyss of Algorthms 8
Optmal Substructure Theorem Let Z = z,..., z k be any LCS of X and Y.. If x m = y n, then z k = x m = y n and Z k- s an LCS of X m- and Y n-.. If x m y n, then ether z k x m and Z s an LCS of X m- and Y. 3. or z k y n and Z s an LCS of X and Y n-. k n n- Notaton: prefx X = x,...,x s the frst letters of X. Ths says what any longest common subsequence must look lke; do you beleve t? CSE53 Desgn and Analyss of Algorthms 9
Optmal Substructure Theorem Let Z = z,..., z k be any LCS of X and Y.. If x m = y n, then z k = x m = y n and Z k- s an LCS of X m- and Y n-.. If x m y n, then ether z k x m and Z s an LCS of X m- and Y. 3. or z k y n and Z s an LCS of X and Y n-. k n n- Proof: (case : x m = y n ) Any sequence Z that does not end n x m = y n can be made longer by addng x m = y n to the end. Therefore, () longest common subsequence (LCS) Z must end n x m = y n. () Z k- s a common subsequence of X m- and Y n-, and (3) there s no longer CS of X m- and Y n-, or Z would not be an LCS. CSE53 Desgn and Analyss of Algorthms
Optmal Substructure Theorem Let Z = z,..., z k be any LCS of X and Y.. If x m = y n, then z k = x m = y n and Z k- s an LCS of X m- and Y n-.. If x m y n, then ether z k x m and Z s an LCS of X m- and Y. 3. or z k y n and Z s an LCS of X and Y n-. k n n- Proof: (case : x m y n, and z k x m ) Snce Z does not end n x m, () Z s a common subsequence of X m- and Y, and () there s no longer CS of X m- and Y, or Z would not be an LCS. CSE53 Desgn and Analyss of Algorthms
Recursve Soluton Defne c[, j] = length of LCS of X and Y j. We want c[m,n]. c [, j ] = c [, j max( c [ ] +, j ], c [, j ]) f f f =, j, j or j = > and > and, x x = y y j j,. Ths gves a recursve algorthm and solves the problem. ut does t solve t well? CSE53 Desgn and Analyss of Algorthms
Recursve Soluton c [ α, β ] = c [ prefx α, prefx β ] + max( c [ prefx α, β ], c [ α, prefx β ]) f f f α empty or β empty, end( α ) = end( β ), end( α ) end( β ). c[sprngtme, prntng] c[sprngtm, prntng] c[sprngtme, prntn] [sprngt, prntng] [sprngtm, prntn] [sprngtm, prntn] [sprngtme, prnt] [sprngt, prntng] [sprngt, prntn] [sprngtm, prnt] [sprngtme, prnt] CSE53 Desgn and Analyss of Algorthms 3
Recursve Soluton c [ α, β ] = c [ prefx α, prefx β ] + max( c [ prefx α, β ], c [ α, prefx β ]) f f f α empty or β empty end( α ) = end( end( α ) end( β ), β )., p r n t n g Keep track of c[a,b] n a table of nm entres: top/down bottom/up S P r n g t m e CSE53 Desgn and Analyss of Algorthms 4
Computng the length of an LCS LCS-LENGTH (X, Y). m length[x]. n length[y] 3. for to m 4. do c[, ] 5. for j to n 6. do c[, j ] 7. for to m 8. do for j to n 9. do f x = y j. then c[, j ] c[, j ] +. b[, j ]. else f c[, j ] c[, j ] 3. then c[, j ] c[, j ] 4. b[, j ] 5. else c[, j ] c[, j ] 6. b[, j ] 7.return c and b b[, j ] ponts to table entry whose subproblem we used n solvng LCS of X and Y j. c[m,n] contans the length of an LCS of X and Y. Tme: O(mn) CSE53 Desgn and Analyss of Algorthms 5
Constructng an LCS PRINT-LCS (b, X,, j). f = or j =. then return 3. f b[, j ] = 4. then PRINT-LCS(b, X,, j ) 5. prnt x 6. elsef b[, j ] = 7. then PRINT-LCS(b, X,, j) 8. else PRINT-LCS(b, X,, j ) Intal call s PRINT-LCS (b, X,m, n). When b[, j ] =, we have extended LCS by one character. So LCS = entres wth n them. Tme: O(m+n) CSE53 Desgn and Analyss of Algorthms 6
LCS Example We ll see how LCS algorthm works on the followng example: X = AC Y = DCA What s the Longest Common Subsequence of X and Y? LCS(X, Y) = C X = A C Y = D C A CSE53 Desgn and Analyss of Algorthms 7
LCS Example () j 3 4 5 Yj D C A X A AC DCA 3 4 C X = AC; m = X = 4 Y = DCA; n = Y = 5 Allocate array c[5,4] CSE53 Desgn and Analyss of Algorthms 8
LCS Example () j 3 4 5 Yj D C A X 3 4 A C AC DCA for = to m c[,] = for j = to n c[,j] = CSE53 Desgn and Analyss of Algorthms 9
LCS Example () j 3 4 5 Yj D C A X 3 4 A C AC DCA f ( X == Y j ) c[,j] = c[-,j-] + else c[,j] = max( c[-,j], c[,j-] ) CSE53 Desgn and Analyss of Algorthms
LCS Example (3) j 3 4 5 Yj D C A X 3 4 A C AC DCA f ( X == Y j ) c[,j] = c[-,j-] + else c[,j] = max( c[-,j], c[,j-] ) CSE53 Desgn and Analyss of Algorthms
LCS Example (4) j 3 4 5 Yj D C A X 3 4 A C AC DCA f ( X == Y j ) c[,j] = c[-,j-] + else c[,j] = max( c[-,j], c[,j-] ) CSE53 Desgn and Analyss of Algorthms
LCS Example (5) j 3 4 5 Yj D C A X 3 4 A C AC DCA f ( X == Y j ) c[,j] = c[-,j-] + else c[,j] = max( c[-,j], c[,j-] ) CSE53 Desgn and Analyss of Algorthms 3
LCS Example (6) 3 4 j 3 4 5 Yj D C A X A C AC DCA f ( X == Y j ) c[,j] = c[-,j-] + else c[,j] = max( c[-,j], c[,j-] ) CSE53 Desgn and Analyss of Algorthms 4
LCS Example (7) 3 4 j 3 4 5 Yj D C A X A C AC DCA f ( X == Y j ) c[,j] = c[-,j-] + else c[,j] = max( c[-,j], c[,j-] ) CSE53 Desgn and Analyss of Algorthms 5
LCS Example (8) j 3 4 5 Yj D C A X 3 4 A C AC DCA f ( X == Y j ) c[,j] = c[-,j-] + else c[,j] = max( c[-,j], c[,j-] ) CSE53 Desgn and Analyss of Algorthms 6
LCS Example () 3 4 j 3 4 5 Yj D C A X A C AC DCA f ( X == Y j ) c[,j] = c[-,j-] + else c[,j] = max( c[-,j], c[,j-] ) CSE53 Desgn and Analyss of Algorthms 7
LCS Example () 3 4 j 3 4 5 Yj D C A X A C AC DCA f ( X == Y j ) c[,j] = c[-,j-] + else c[,j] = max( c[-,j], c[,j-] ) CSE53 Desgn and Analyss of Algorthms 8
LCS Example () j 3 4 5 Yj D C A X 3 4 A C AC DCA f ( X == Y j ) c[,j] = c[-,j-] + else c[,j] = max( c[-,j], c[,j-] ) CSE53 Desgn and Analyss of Algorthms 9
LCS Example (3) j 3 4 5 Yj D C A X 3 4 A C AC DCA f ( X == Y j ) c[,j] = c[-,j-] + else c[,j] = max( c[-,j], c[,j-] ) CSE53 Desgn and Analyss of Algorthms 3
LCS Example (4) j 3 4 5 Yj D C A X 3 4 A C AC DCA f ( X == Y j ) c[,j] = c[-,j-] + else c[,j] = max( c[-,j], c[,j-] ) CSE53 Desgn and Analyss of Algorthms 3
LCS Example (5) 3 4 j 3 4 5 Yj D C A X A C f ( X == Y j ) c[,j] = c[-,j-] + else c[,j] = max( c[-,j], c[,j-] ) 3 AC DCA CSE53 Desgn and Analyss of Algorthms 3
LCS Algorthm Runnng Tme LCS algorthm calculates the values of each entry of the array c[m,n] So what s the runnng tme? O(m*n) snce each c[,j] s calculated n constant tme, and there are m*n elements n the array CSE53 Desgn and Analyss of Algorthms 33
How to fnd actual LCS So far, we have just found the length of LCS, but not LCS tself. We want to modfy ths algorthm to make t output Longest Common Subsequence of X and Y Each c[,j] depends on c[-,j] and c[,j-] or c[-, j-] For each c[,j] we can say how t was acqured: 3 For example, here c[,j] = c[-,j-] + = +=3 CSE53 Desgn and Analyss of Algorthms 34
How to fnd actual LCS - contnued Remember that c[, j] = c[, j ] + max( c[, j ], c[, j]) f x[ ] = y[ otherwse j], So we can start from c[m,n] and go backwards Whenever c[,j] = c[-, j-]+, remember x[] (because x[] s a part of LCS) When = or j= (.e. we reached the begnnng), output remembered letters n reverse order CSE53 Desgn and Analyss of Algorthms 35
Fndng LCS 3 4 j 3 4 5 Yj D C A X A C 3 CSE53 Desgn and Analyss of Algorthms 36
Fndng LCS () 3 4 j 3 4 5 Yj D C A X A C 3 LCS (reversed order): C LCS (straght order): C (ths strng turned out to be a palndrome) CSE53 Desgn and Analyss of Algorthms 37