Dynamc Programmng Davd Kauchak cs32 Sprng 212 Dynamc programmng l One of the most mportant algorthm tools! l Very common ntervew queston l Method for solvng problems where optmal solutons can be defned n terms of optmal solutons to sub-problems AND l the sub-problems are overlappng Fbonacc numbers Fbonacc: a frst attempt 1, 1, 2, 3, 5, 8, 13, 21, 34, What s the recurrence for the n th Fbonacc number? F(n) = F(n-1) + F(n-2) The soluton for n s defned wth respect to the soluton to smaller problems (n-1 and n-2) 1
Is t correct? Runnng tme F(n) = F(n-1) + F(n-2) l Each call creates two recursve calls l Each call reduces the sze of the problem by 1 or 2 l Creates a full bnary of depth n l O(2 n ) Can we do better? Fb(n) A lot of repeated work! Fb(n) Fb(n-1) Fb(n-2) Fb(n-1) Fb(n-2) Fb(n-2) Fb(n-3) Fb(n-3) Fb(n-4) Fb(n-2) Fb(n-3) Fb(n-3) Fb(n-4) Fb(n-3) Fb(n-4) Fb(n-4) Fb(n-5) Fb(n-4) Fb(n-5) Fb(n-5) Fb(n-6) Fb(n-3) Fb(n-4) Fb(n-4) Fb(n-5) Fb(n-4) Fb(n-5) Fb(n-5) Fb(n-6) 2
Identfyng a dynamc programmng problem The soluton can be defned wth respect to solutons to subproblems The subproblems created are overlappng, that s we see the same subproblems repeated Creatng a dynamc programmng soluton Step 1: Identfy a soluton to the problem wth respect to smaller subproblems l F(n) = F(n-1) + F(n-2) Step 2: bottom up - start wth solutons to the smallest problems and buld solutons to the larger problems use an array to store solutons to subproblems Is t correct? Runnng tme? F(n) = F(n-1) + F(n-2) Θ(n) 3
Countng bnary search trees l How many unque bnary search trees can be created usng the numbers 1 through n? 2 1 3 6 4 5 Step 1: What s the subproblem? l Assume we have some black box solver (call t T) that can gve us the answer to smaller subproblems l How can we use the answer from ths to answer our queston? l How many optons for the root are there? 1 2 3 n Subproblems Subproblems 1, 2,, -1 +1, +2,, +n How many trees have as the root?? 4
Subproblems Subproblems 1, 2,, -1 +1, +2,, +n T(-1) subproblem of sze -1? 1, 2,, -1 +1, +2,, +n T(-1) Number of trees for +1, +2,, +n s the same as the number of trees from 1, 2,, n- Subproblems Subproblems 1, 2,, -1 +1, +2,, +n T(-1) T(n-) 1, 2,, -1 +1, +2,, +n T(-1) T(n-) Gven solutons for T(-1) and T(n-) how many trees are there wth as the root? T() = T(-1) * T(n-) 5
Step 1: defne the answer wth respect to subproblems T() = T(-1) * T(n-) Is there a problem? T( n) = n = 1 T( 1)* T( n ) As wth Fbonacc, we re repeatng a lot of work Step 2: Generate a soluton from the bottom-up 1 2 3 4 5 n 6
c[]*c[1] + c[1]*c[] 1 1 1 2 3 4 5 n 1 1 1 2 3 4 5 n 1 2 2 1 c[]*c[1] + c[1]*c[] 1 1 1 2 3 4 5 n 1 1 2 1 2 3 4 5 n 7
1 2 3 c[]*c[2] + c[1]*c[1] + c[2]*c[] 1 1 2 1 2 3 4 5 n 1 1 2 5 1 2 3 4 5 n Runnng tme? 1 1 2 5 1 2 3 4 5 n Θ(n 2 ) 8
Longest common subsequence (LCS) For a sequence X = x 1, x 2,, x n, a subsequence s a subset of the sequence defned by a set of ncreasng ndces ( 1, 2,, k ) where 1 1 < 2 < < k n X = A B A C D A B A B ABA? Longest common subsequence (LCS) l For a sequence X = x 1, x 2,, x n, a subsequence s a subset of the sequence defned by a set of ncreasng ndces ( 1, 2,, k ) where 1 1 < 2 < < k n X = A B A C D A B A B ABA Longest common subsequence (LCS) l For a sequence X = x 1, x 2,, x n, a subsequence s a subset of the sequence defned by a set of ncreasng ndces ( 1, 2,, k ) where 1 1 < 2 < < k n X = A B A C D A B A B ACA? Longest common subsequence (LCS) l For a sequence X = x 1, x 2,, x n, a subsequence s a subset of the sequence defned by a set of ncreasng ndces ( 1, 2,, k ) where 1 1 < 2 < < k n X = A B A C D A B A B ACA 9
Longest common subsequence (LCS) l For a sequence X = x 1, x 2,, x n, a subsequence s a subset of the sequence defned by a set of ncreasng ndces ( 1, 2,, k ) where 1 1 < 2 < < k n X = A B A C D A B A B DCA? Longest common subsequence (LCS) l For a sequence X = x 1, x 2,, x n, a subsequence s a subset of the sequence defned by a set of ncreasng ndces ( 1, 2,, k ) where 1 1 < 2 < < k n X = A B A C D A B A B DCA Longest common subsequence (LCS) l For a sequence X = x 1, x 2,, x n, a subsequence s a subset of the sequence defned by a set of ncreasng ndces ( 1, 2,, k ) where 1 1 < 2 < < k n X = A B A C D A B A B AADAA? Longest common subsequence (LCS) l For a sequence X = x 1, x 2,, x n, a subsequence s a subset of the sequence defned by a set of ncreasng ndces ( 1, 2,, k ) where 1 1 < 2 < < k n X = A B A C D A B A B AADAA 1
LCS problem Gven two sequences X and Y, a common subsequence s a subsequence that occurs n both X and Y Gven two sequences X = x 1, x 2,, x n and Y = y 1, y 2,, y n, What s the longest common subsequence? X = A B C B D A B LCS problem l Gven two sequences X and Y, a common subsequence s a subsequence that occurs n both X and Y l Gven two sequences X = x 1, x 2,, x n and Y = y 1, y 2,, y n, What s the longest common subsequence? X = A B C B D A B Step 1: Defne the problem wth respect to subproblems X = A B C B D A B Step 1: Defne the problem wth respect to subproblems X = A B C B D A? Y = B D C A B? Is the last character part of the LCS? 11
Step 1: Defne the problem wth respect to subproblems Step 1: Defne the problem wth respect to subproblems X = A B C B D A? Y = B D C A B? X = A B C B D A A LCS The characters are part of the LCS What s the recursve relatonshp? Two cases: ether the characters are the same or they re dfferent If they re the same LCS + x ( X, Y) = LCS ( X1... n 1, Y1... m 1) n Step 1: Defne the problem wth respect to subproblems X = A B C B D A B LCS Step 1: Defne the problem wth respect to subproblems X = A B C B D A B LCS If they re dfferent LCS X, Y) = LCS ( X 1, Y) (... n 1 If they re dfferent LCS ( X, Y) = LCS ( X, Y1... m 1) 12
Step 1: Defne the problem wth respect to subproblems Step 1: Defne the problem wth respect to subproblems X = A B C B D A B X = A B C B D A B If they re dfferent? X = A B C B D A B X1... n 1, Y1... m 1) f xn = ym X, Y) X1... n 1, Y), X, Y1... m 1) otherwse (for now, let s ust worry about countng the length of the LCS) Step 2: Buld the soluton from the bottom up X1... n 1, Y1... m 1) f xn = ym X, Y) X1... n 1, Y), X, Y1... m 1) otherwse Step 2: Buld the soluton from the bottom up X1... n 1, Y1... m 1) f xn = ym X, Y) X1... n 1, Y), X, Y1... m 1) otherwse What types of subproblem solutons do we need to store? What types of subproblem solutons do we need to store? X 1, Y 1 k ) X 1, Y 1 k ) two dfferent ndces 1,, ] 1, ],, f otherwse 13
1,, ] 1, ],, f otherwse 1,, ] 1, ],, f otherwse 1 2 3 4 5 6 y B D C A B A 1 2 3 4 5 6 y B D C A B A x For Fbonacc and tree countng, we had to ntalze some entres n the array. Any here? x Need to ntalze values wthn 1 smaller n ether dmenson. 1,, ] 1, ],, f otherwse 1,, ] 1, ],, f otherwse 1 2 3 4 5 6 y B D C A B A 1 2 3 4 5 6 y B D C A B A x? A, B) x 14
1,, ] 1, ],, f otherwse 1,, ] 1, ],, f otherwse 1 2 3 4 5 6 y B D C A B A 1 2 3 4 5 6 y B D C A B A x? A, BDCA) x 1 A, BDCA) 1,, ] 1, ],, f otherwse 1,, ] 1, ],, f otherwse 1 2 3 4 5 6 y B D C A B A 1 2 3 4 5 6 y B D C A B A x 1 1 1 1 1 1 1 2 2 1 1 2 2 2 2 1 1 2 2? ABCB, BDCAB) x 1 1 1 1 1 1 1 2 2 1 1 2 2 2 2 1 1 2 2 3 ABCB, BDCAB) 15
1, f x = y, ] 1, ],, otherwse x 1 2 3 4 5 6 y B D C A B A 1 1 1 1 1 1 1 2 2 1 1 2 2 2 2 1 1 2 2 3 3 1 2 2 2 3 3 1 2 2 3 3 4 1 2 2 3 4 4 Where s the fnal answer? The algorthm The algorthm The algorthm Base case ntalzaton Fll n the matrx 16
The algorthm The algorthm The algorthm Runnng tme? Θ(nm) 17