Dynamic Programming 4/5/12. Dynamic programming. Fibonacci numbers. Fibonacci: a first attempt. David Kauchak cs302 Spring 2012

Transcription

1 Dynamc Programmng Davd Kauchak cs32 Sprng 212 Dynamc programmng l One of the most mportant algorthm tools! l Very common ntervew queston l Method for solvng problems where optmal solutons can be defned n terms of optmal solutons to sub-problems AND l the sub-problems are overlappng Fbonacc numbers Fbonacc: a frst attempt 1, 1, 2, 3, 5, 8, 13, 21, 34, What s the recurrence for the n th Fbonacc number? F(n) = F(n-1) + F(n-2) The soluton for n s defned wth respect to the soluton to smaller problems (n-1 and n-2) 1

2 Is t correct? Runnng tme F(n) = F(n-1) + F(n-2) l Each call creates two recursve calls l Each call reduces the sze of the problem by 1 or 2 l Creates a full bnary of depth n l O(2 n ) Can we do better? Fb(n) A lot of repeated work! Fb(n) Fb(n-1) Fb(n-2) Fb(n-1) Fb(n-2) Fb(n-2) Fb(n-3) Fb(n-3) Fb(n-4) Fb(n-2) Fb(n-3) Fb(n-3) Fb(n-4) Fb(n-3) Fb(n-4) Fb(n-4) Fb(n-5) Fb(n-4) Fb(n-5) Fb(n-5) Fb(n-6) Fb(n-3) Fb(n-4) Fb(n-4) Fb(n-5) Fb(n-4) Fb(n-5) Fb(n-5) Fb(n-6) 2

3 Identfyng a dynamc programmng problem The soluton can be defned wth respect to solutons to subproblems The subproblems created are overlappng, that s we see the same subproblems repeated Creatng a dynamc programmng soluton Step 1: Identfy a soluton to the problem wth respect to smaller subproblems l F(n) = F(n-1) + F(n-2) Step 2: bottom up - start wth solutons to the smallest problems and buld solutons to the larger problems use an array to store solutons to subproblems Is t correct? Runnng tme? F(n) = F(n-1) + F(n-2) Θ(n) 3

4 Countng bnary search trees l How many unque bnary search trees can be created usng the numbers 1 through n? Step 1: What s the subproblem? l Assume we have some black box solver (call t T) that can gve us the answer to smaller subproblems l How can we use the answer from ths to answer our queston? l How many optons for the root are there? n Subproblems Subproblems 1, 2,, -1 +1, +2,, +n How many trees have as the root?? 4

5 Subproblems Subproblems 1, 2,, -1 +1, +2,, +n T(-1) subproblem of sze -1? 1, 2,, -1 +1, +2,, +n T(-1) Number of trees for +1, +2,, +n s the same as the number of trees from 1, 2,, n- Subproblems Subproblems 1, 2,, -1 +1, +2,, +n T(-1) T(n-) 1, 2,, -1 +1, +2,, +n T(-1) T(n-) Gven solutons for T(-1) and T(n-) how many trees are there wth as the root? T() = T(-1) * T(n-) 5

6 Step 1: defne the answer wth respect to subproblems T() = T(-1) * T(n-) Is there a problem? T( n) = n = 1 T( 1)* T( n ) As wth Fbonacc, we re repeatng a lot of work Step 2: Generate a soluton from the bottom-up n 6

7 c[]*c[1] + c[1]*c[] n n c[]*c[1] + c[1]*c[] n n 7

8 1 2 3 c[]*c[2] + c[1]*c[1] + c[2]*c[] n n Runnng tme? n Θ(n 2 ) 8

9 Longest common subsequence (LCS) For a sequence X = x 1, x 2,, x n, a subsequence s a subset of the sequence defned by a set of ncreasng ndces ( 1, 2,, k ) where 1 1 < 2 < < k n X = A B A C D A B A B ABA? Longest common subsequence (LCS) l For a sequence X = x 1, x 2,, x n, a subsequence s a subset of the sequence defned by a set of ncreasng ndces ( 1, 2,, k ) where 1 1 < 2 < < k n X = A B A C D A B A B ABA Longest common subsequence (LCS) l For a sequence X = x 1, x 2,, x n, a subsequence s a subset of the sequence defned by a set of ncreasng ndces ( 1, 2,, k ) where 1 1 < 2 < < k n X = A B A C D A B A B ACA? Longest common subsequence (LCS) l For a sequence X = x 1, x 2,, x n, a subsequence s a subset of the sequence defned by a set of ncreasng ndces ( 1, 2,, k ) where 1 1 < 2 < < k n X = A B A C D A B A B ACA 9

10 Longest common subsequence (LCS) l For a sequence X = x 1, x 2,, x n, a subsequence s a subset of the sequence defned by a set of ncreasng ndces ( 1, 2,, k ) where 1 1 < 2 < < k n X = A B A C D A B A B DCA? Longest common subsequence (LCS) l For a sequence X = x 1, x 2,, x n, a subsequence s a subset of the sequence defned by a set of ncreasng ndces ( 1, 2,, k ) where 1 1 < 2 < < k n X = A B A C D A B A B DCA Longest common subsequence (LCS) l For a sequence X = x 1, x 2,, x n, a subsequence s a subset of the sequence defned by a set of ncreasng ndces ( 1, 2,, k ) where 1 1 < 2 < < k n X = A B A C D A B A B AADAA? Longest common subsequence (LCS) l For a sequence X = x 1, x 2,, x n, a subsequence s a subset of the sequence defned by a set of ncreasng ndces ( 1, 2,, k ) where 1 1 < 2 < < k n X = A B A C D A B A B AADAA 1

11 LCS problem Gven two sequences X and Y, a common subsequence s a subsequence that occurs n both X and Y Gven two sequences X = x 1, x 2,, x n and Y = y 1, y 2,, y n, What s the longest common subsequence? X = A B C B D A B LCS problem l Gven two sequences X and Y, a common subsequence s a subsequence that occurs n both X and Y l Gven two sequences X = x 1, x 2,, x n and Y = y 1, y 2,, y n, What s the longest common subsequence? X = A B C B D A B Step 1: Defne the problem wth respect to subproblems X = A B C B D A B Step 1: Defne the problem wth respect to subproblems X = A B C B D A? Y = B D C A B? Is the last character part of the LCS? 11

12 Step 1: Defne the problem wth respect to subproblems Step 1: Defne the problem wth respect to subproblems X = A B C B D A? Y = B D C A B? X = A B C B D A A LCS The characters are part of the LCS What s the recursve relatonshp? Two cases: ether the characters are the same or they re dfferent If they re the same LCS + x ( X, Y) = LCS ( X1... n 1, Y1... m 1) n Step 1: Defne the problem wth respect to subproblems X = A B C B D A B LCS Step 1: Defne the problem wth respect to subproblems X = A B C B D A B LCS If they re dfferent LCS X, Y) = LCS ( X 1, Y) (... n 1 If they re dfferent LCS ( X, Y) = LCS ( X, Y1... m 1) 12

13 Step 1: Defne the problem wth respect to subproblems Step 1: Defne the problem wth respect to subproblems X = A B C B D A B X = A B C B D A B If they re dfferent? X = A B C B D A B X1... n 1, Y1... m 1) f xn = ym X, Y) X1... n 1, Y), X, Y1... m 1) otherwse (for now, let s ust worry about countng the length of the LCS) Step 2: Buld the soluton from the bottom up X1... n 1, Y1... m 1) f xn = ym X, Y) X1... n 1, Y), X, Y1... m 1) otherwse Step 2: Buld the soluton from the bottom up X1... n 1, Y1... m 1) f xn = ym X, Y) X1... n 1, Y), X, Y1... m 1) otherwse What types of subproblem solutons do we need to store? What types of subproblem solutons do we need to store? X 1, Y 1 k ) X 1, Y 1 k ) two dfferent ndces 1,, ] 1, ],, f otherwse 13

14 1,, ] 1, ],, f otherwse 1,, ] 1, ],, f otherwse y B D C A B A y B D C A B A x For Fbonacc and tree countng, we had to ntalze some entres n the array. Any here? x Need to ntalze values wthn 1 smaller n ether dmenson. 1,, ] 1, ],, f otherwse 1,, ] 1, ],, f otherwse y B D C A B A y B D C A B A x? A, B) x 14

15 1,, ] 1, ],, f otherwse 1,, ] 1, ],, f otherwse y B D C A B A y B D C A B A x? A, BDCA) x 1 A, BDCA) 1,, ] 1, ],, f otherwse 1,, ] 1, ],, f otherwse y B D C A B A y B D C A B A x ? ABCB, BDCAB) x ABCB, BDCAB) 15

16 1, f x = y, ] 1, ],, otherwse x y B D C A B A Where s the fnal answer? The algorthm The algorthm The algorthm Base case ntalzaton Fll n the matrx 16

17 The algorthm The algorthm The algorthm Runnng tme? Θ(nm) 17