CS 470/570 Dynamic Programming Format of Dynamic Programming algorithms: Use a recursive formula But recursion can be inefficient because it might repeat the same computations multiple times So use an array to store and lookup the subproblem results as needed
Fibonacci numbers: 1, 1, 2, 3, 5, 8, 13, 21, 34,.. F(0) = 1 F(1) = 1 F(n) = F(n 2) + F(n 1), when n 2 int F (int n) { if (n==0 n==1) return 1; else return F(n 2) + F(n 1); } F(10) F(8) F(9) F(6) F(7) F(7) F(8) F(4) F(5) F(5) F(6) F(5) F(6) F(6) F(7) Recursion yields repeated subproblems, so not efficient (Running time is exponential)
Dynamic programming use a table (array) to remember the values that have previously been computed First method: Bottom-up (use a table instead of recursion) allocate array A[0 n]; for (k=0; k<=n; k++) if (k==0 k==1) A[k] = 1; else A[k] = A[k 2] + A[k 1]; return A[n]; Second method: Top-down (use both recursion and a table) allocate array A[0 n]; for (k=0; k<=n; k++) A[k] = null; return F(n); int F (int n) { if (A[n]!= null) return A[n]; if (n==0 n==1) A[n] = 1; else A[n] = F(n 2) + F(n 1); return A[n]; } Running time = θ(n) for each method
Combinations: C(n, k) = n! k! (n k)! Pascal s recursive formula for combinations: C(n, 0) = 1 C(n, n) = 1 C(n, k) = C(n 1, k 1) + C(n 1, k), when 0 < k < n int C (int n, int k) { if (k==0 k==n) return 1; else return C(n 1, k 1) + C(n 1, k); } C(8,4) C(7,3) C(7,4) C(6,2) C(6,3) C(6,3) C(6,4) Again, recursion yields repeated subproblems, so not efficient (Running time is exponential)
Dynamic programming Pascal s triangle C(8,4) C(7,3) C(7,4) C(6,2) C(6,3) C(6,4) C(5,1) C(5,2) C(5,3) C(5,4) C(4,0) C(4,1) C(4,2) C(4,3) C(4,4) C(3,0) C(3,1) C(3,2) C(3,3) C(2,0) C(2,1) C(2,2) C(1,0) C(1,1) C(0,0) allocate array C[0 n][0 k]; for (m=0; m<=n; m++) for (j=0; j<=k; j++) if (j==0 j==m) C[m][j] = 1; else C[m][j] = C[m 1][j 1] + C[m 1][j]; return C[n][k]; Running time = θ(nk) = O(n 2 ) because k n
0-1 Knapsack problem: Objects {1 n} Profits P[1 n] Weights W[1 n] Maximum weight capacity M 0-1 constraint: Must take none (0) or all (1) of each object. Goal: Determine the amounts X[1 n] for each object so that Σ 1 k n X[k]*W[k] M, and Σ 1 k n X[k]*P[k] is as large as possible. Each X[j] must be either 0 or 1.
Next we develop a dynamic programming algorithm for the 0-1 Knapsack problem Recursive function: T (n, M) = max possible total profit such that we choose any subset of objects {1 n} and maximum weight capacity is M T (j, k) = max possible total profit such that we can only choose from objects {1 j} and maximum weight capacity is k (0 j n, 0 k M) T (j, k) = 0 T (j, k) = T (j 1, k) if j==0 if j>0 and k < W[j] T (j, k) = max {T (j 1, k), T (j 1, k W[j]) + P[j]} if j>0 and k W[j] int T (int j, int k) { if (j==0) return 0; if (k < W[j]) return T (j 1, k); return max {T (j 1, k), T (j 1, k W[j]) + P[j]}; }
Dynamic programming algorithm: allocate array T[0 n][0 M]; for j = 0 to n for k = 0 to M if (j==0) T[j][k] = 0; else if (k < W[j]) T[j][k] = T[j 1][k]; else T[j][k] = max {T[j 1][k], T[j 1][k W[j]] + P[j]}; Running time = θ(nm) Example: n=4, M=8 1 2 3 4 P 12 15 16 18 W 2 3 4 6 T k=0 k=1 k=2 k=3 k=4 k=5 k=6 k=7 k=8 j=0 0 0 0 0 0 0 0 0 0 j=1 0 0 12 12 12 12 12 12 12 j=2 0 0 12 15 15 27 27 27 27 j=3 0 0 12 15 16 27 28 31 31 j=4 0 0 12 15 16 27 28 31 31
So the max total profit is 31, but which objects to choose? We still need to assign each X[j] = either 0 or 1. k=m; for (j=n; j>0; j = 1) if (T[j 1][k] == T[j][k]) X[j]=0; else { X[j]=1; k = W[j]; } Takes θ(n) additional time 1 2 3 4 P 12 15 16 18 W 2 3 4 6 X 0 1 1 0 T k=0 k=1 k=2 k=3 k=4 k=5 k=6 k=7 k=8 j=0 0 0 0 0 0 0 0 0 0 j=1 0 0 12 12 12 12 12 12 12 j=2 0 0 12 15 15 27 27 27 27 j=3 0 0 12 15 16 27 28 31 31 j=4 0 0 12 15 16 27 28 31 31 So the optimal solution places objects 2 and 3 in the knapsack.
Longest Common Subsequence problem (LCS): Given two strings: X[1 m] Y[1 n] Example: X = bdacbea m = X = 7 Y = dcbaecba n = Y = 8 Goal: Find a string that is a subsequence of both X and Y, and that has largest possible length. Example: LCS (X, Y) = dacba (length = 5)
Recursive function for the length of the LCS of X and Y: L (m, n) = length of the LCS of X[1 m] and Y[1 n] L (j, k) = length of the LCS of X[1 j] and Y[1 k] (0 j m, 0 k n) L (j, k) = 0 L (j 1, k 1) + 1 max {L (j 1, k), L (j, k 1)} if j==0 or k==0 if j>0, k>0, and X[j]==Y[k] if j>0, k>0, and X[j] Y[k] int L (int j, int k) { if (j==0 k==0) return 0; if (X[j]==Y[k]) return L (j 1, k 1) + 1; return max {L (j 1, k), L (j, k 1)}; }
If implement this algorithm recursively, it will take exponential time due to repeated subproblems L (m, n) L (m 1, n) L (m, n 1) L (m 2, n) L (m 1, n 1) L (m 1, n 1) L (m, n 2) Dynamic programming algorithm: allocate array L[0 m][0 n]; for j = 0 to m for k = 0 to n if (j==0 k==0) L[j][k] = 0; else if (X[j]==Y[k]) L[j][k] = L[j 1][k 1] + 1; else L[j][k] = max {L[j 1][k], L[j][k 1]}; Running time = θ(mn)
Example: X = bdacbea m = X = 7 Y = dcbaecba n = Y = 8 L 0 1 2 3 4 5 6 7 8 0 0 0 0 0 0 0 0 0 0 X: 1 0 0 0 1 1 1 1 1 1 b 2 0 1 1 1 1 1 1 1 1 d 3 0 1 1 1 2 2 2 2 2 a 4 0 1 2 2 2 2 3 3 3 c 5 0 1 2 3 3 3 3 4 4 b 6 0 1 2 3 3 4 4 4 4 e 7 0 1 2 3 4 4 4 4 5 a Y: d c b a e c b a So the length of the LCS is 5. But how can we find a LCS that has length 5?
Call buildlcs (m, n) string buildlcs (j, k) { if (L[j][k] == 0) return ; else if (X[j] == Y[k]) return buildlcs (j 1, k 1) + X[j]; else if (L[j][k 1] == L[j][k]) return buildlcs (j, k 1); else // (L[j 1][k] == L[j][k]) return buildlcs (j 1, k); } // empty string // append char Takes θ(m+n) additional time assumes we implement append char in θ(1) time this is possible here because we know the final string length L(m,n) in advance can allocate an array of size L(m,n) chars in advance or replace recursion by iteration
L 0 1 2 3 4 5 6 7 8 0 0 0 0 0 0 0 0 0 0 X: 1 0 0 0 1 1 1 1 1 1 b 2 0 1 1 1 1 1 1 1 1 d 3 0 1 1 1 2 2 2 2 2 a 4 0 1 2 2 2 2 3 3 3 c 5 0 1 2 3 3 3 3 4 4 b 6 0 1 2 3 3 4 4 4 4 e 7 0 1 2 3 4 4 4 4 5 a Y: d c b a e c b a LCS (X, Y) = dcbea Find another solution by exchanging last two cases in buildlcs L 0 1 2 3 4 5 6 7 8 0 0 0 0 0 0 0 0 0 0 X: 1 0 0 0 1 1 1 1 1 1 b 2 0 1 1 1 1 1 1 1 1 d 3 0 1 1 1 2 2 2 2 2 a 4 0 1 2 2 2 2 3 3 3 c 5 0 1 2 3 3 3 3 4 4 b 6 0 1 2 3 3 4 4 4 4 e 7 0 1 2 3 4 4 4 4 5 a Y: d c b a e c b a LCS (X, Y) = bacba
Matrix Chain Product (MCP): Multiply chain of compatible matrices M 1 M 2 M 3 M n Each matrix M j has dimensions D[j 1] rows and D[j] columns Example: M 1 M 2 M 3 where D[0 3] = {10, 20, 5, 30} M 1 is 10-by-20, M 2 is 20-by-5, and M 3 is 5-by-30 (M 1 M 2 ) M 3 takes 10*20*5 + 10*5*30 = 2500 scalar products M 1 (M 2 M 3 ) takes 20*5*30 + 10*20*30 = 9000 scalar products Goal: compute M 1 M 2 M 3 M n using fewest scalar products Let Cost[i][j] = fewest scalar products to compute M i M j Find best parenthesization (M i M k ) (M k+1 M j ) Cost[i][j] = 0 when i==j Cost[i][j] = min {Cost[i][k] + Cost[k+1][j] + D[i 1]*D[k]*D[j] i k j 1} when i<j
Dynamic programming algorithm: allocate arrays Cost[1 n][1 n] and Bestk[1 n][1 n]; for i = 1 to n Cost[i][i] = 0; for L = 2 to n for i = 1 to n L+1 j = i+l 1; Cost[i][j] = ; for k = i to j 1 t = Cost[i][k] + Cost[k+1][j] + D[i 1]*D[k]*D[j]; if (t < Cost[i][j]) Cost[i][j] = t; Bestk[i][j] = k; Running time = θ(n 3 ) Example: n=4 0 1 2 3 4 D 7 8 5 10 6 Cost j=1 j=2 j=3 j=4 Bestk j=1 j=2 j=3 j=4 i=1 0 280 630 790 i=1 1 2 2 i=2 0 400 540 i=2 2 2 i=3 0 300 i=3 3 i=4 0 i=4
Next determine how to multiply M 1 M 2 M 3 M n using only Cost[1][n] scalar products: MatrixMult (A, B) { // naïve algorithm allocate C[1 A.rows][1 B.columns] for i = 1 to A.rows for j = 1 to B.columns C[i][j] = 0; for k = 1 to A.columns // same as B.rows C[i][j] += A[i][k]*B[k][j]; return C; } ChainProduct (i, j) { if (i==j) return M i ; k = Bestk[i][j]; A = ChainProduct (i, k); B = ChainProduct (k+1, j); return MatrixMult (A, B); } ChainProduct (1, n);
Optimal Binary Search Tree (BST): Construct a BST with Key 1 < Key 2 < Key 3 < < Key n Frequency of searching for each Key i is F[i] Example: Key[1 3] = {a, b, c} and F[1 3] = {0.3, 0.5, 0.2} 5 possible BSTs Expected cost of searching a b c 1*0.3 + 2*0.5 + 3*0.2 = 1.9 a b a c b c 1*0.3 + 3*0.5 + 2*0.2 = 2.2 2*0.3 + 1*0.5 + 2*0.2 = 1.5 a c 2*0.3 + 3*0.5 * 1*0.2 = 2.3 b
b c 3*0.3 + 2*0.5 + 1*0.2 = 2.1 a Goal: construct the BST with minimum expected search cost Let Cost[i][j] = min expected search cost for BST with Key[i j] Find best choice Key[r] for root node Left subtree has Key[i r 1] Right subtree has Key[r+1 j] Cost[i][j] = 0 Cost[i][j] = F[i] when i>j when i==j Cost[i][j] = min {Cost[i][r 1] + Cost[r+1][j] + Σ i k j F[k] i r j} when i<j For efficiency, define W[i][j] = Σ i k j F[k] Cost[i][j] = 0 when i>j Cost[i][j] = min {Cost[i][r 1] + Cost[r+1][j] + W[i][j] i r j} when i j
Dynamic programming algorithm: allocate arrays Cost[1 n+1][0 n], W[1 n+1][0 n], and Root[1 n+1][0 n]; for i = 0 to n Cost[i+1][i] = 0; W[i+1][i] = 0; for L = 1 to n for i = 1 to n L+1 j = i+l 1; Cost[i][j] = ; W[i][j] = W[i][j 1] + F[j]; for r = i to j t = Cost[i][r 1] + Cost[r+1][j] + W[i][j]; if (t < Cost[i][j]) Cost[i][j] = t; Root[i][j] = r; Running time = θ(n 3 ) Example: n=3, F[1 3] = {0.3, 0.5, 0.2} Cost j=0 j=1 j=2 j=3 Root j=0 j=1 j=2 j=3 i=1 0 0.3 1.1 1.5 i=1 1 2 2 i=2 0 0.5 0.9 i=2 2 2 i=3 0 0.2 i=3 3 i=4 0 i=4
Next build the BST to achieve the min expected search cost BuildBST (i, j) { if (i>j) return null; // empty tree r = Root[i][j]; left = BuildBST (i, r 1); right = BuildBST (r+1, j); return new Node (Key[r], left, right); } tree = BuildBST (1, n);
All-Pairs Shortest Paths: Given a weighted (undirected or directed) graph, find a minimum-distance path from each start vertex to each destination vertex [Restriction: we allow edges with negative weights, but not any cycle with negative total weight] Example, why negative-weight cycle is forbidden: What is shortest distance from vertex 1 to vertex 5? 7 1 3 5 3 4 9 5 4 6 8 3+4 = 7 3+ (8 2 9) +4 = 4 3+ (8 2 9) + (8 2 9) +4 = 1 3+ (8 2 9) + (8 2 9) + (8 2 9) +4 = 2 3+ (8 2 9) + (8 2 9) + (8 2 9) + (8 2 9) +4 = 5 Always better to repeat the negative cycle again 2 2
Example, with no negative-weight cycle: 1 5 3 8 2 7 3 4 9 5 6 4 2 Input: Adjacency matrix 1 2 3 4 5 1 0 5 3 2 0 2 3 8 0 4 4 9 0 6 5 7 0 Output: Distance matrix 1 2 3 4 5 1 0 5 3 3 7 2 3 0 0 2 4 3 3 2 0 0 4 4 1 4 2 0 6 5 7 2 4 4 0
Intuition for designing an algorithm to solve APSP problem: D[X,Y] = distance from X to Y = length of shortest path from X to Y X D[X,Y] Y D[X,Z] Z D[Z,Y] if (D[X,Z] + D[Z,Y] < D[X,Y]) D[X,Y] = D[X,Z] + D[Z,Y] Assume vertices are numbered 1 n D Z [X,Y] = length of shortest path from X to Y such that this path is allowed to pass through any or all of the intermediate vertices 1 Z, but it is not allowed to pass through any of Z+1 n X D Z 1 [X,Y] Y D Z 1 [X,Z] Z D Z 1 [Z,Y] D Z [X,Y] = min (D Z 1 [X,Y], D Z 1 [X,Z] + D Z 1 [Z,Y])
Floyd s algorithm for APSP problem: // initialize D 0 = weighted adjacency matrix for X = 1 to n for Y = 1 to n if (X==Y) D 0 [X,Y] = 0; else if (edge (X,Y) exists) D 0 [X,Y] = weight(x,y); else D 0 [X,Y] = ; // compute each D Z matrix from values in the D Z 1 matrix for Z = 1 to n for X = 1 to n for Y = 1 to n if (D Z-1 [X,Z] + D Z-1 [Z,Y] < D Z-1 [X,Y]) D Z [X,Y] = D Z-1 [X,Z] + D Z-1 [Z,Y]; else D Z [X,Y] = D Z-1 [X,Y]; Running time of Floyd s algorithm = θ(n 3 ) Memory usage of Floyd s algorithm = θ(n 3 ) D Z [X,Y] can be stored as 3-dimensional array with elements D[X,Y,Z] for 1 X n, 1 Y n, 0 Z n
Improve Floyd s algorithm to use θ(n 2 ) space: Discard the Z subscript Let D be 2-dimensional array for X = 1 to n for Y = 1 to n if (X==Y) D[X,Y] = 0; else if (edge (X,Y) exists) D[X,Y] = weight(x,y); else D[X,Y] = ; for Z = 1 to n for X = 1 to n for Y = 1 to n if (D[X,Z] + D[Z,Y] < D[X,Y]) D[X,Y] = D[X,Z] + D[Z,Y]; // else D[X,Y] = D[X,Y]; // now we can omit this line To determine whether the graph contains a negative cycle: boolean hasnegativecycle( ) { for X = 1 to n if (D[X,X] < 0) return true; return false; }
World Series problem: Teams A and B compete in a series of games. The winner is the first team to win n games. The series ends as soon as the winner is decided. At most 2n 1 games are played. For 1 k 2n 1, let p[k] = probability that team A wins the k th game. (These are the input values.) For 0 i n and 0 j n, let X(i, j) = probability that the series reaches a situation where team A wins exactly i games and team B wins exactly j games. (These are the output values.) First we develop a recursive formula: X(0, 0) = 1 X(n, n) = 0 X(i, 0) = X(i 1, 0)*p[i], for i>0 X(0, j) = X(0, j 1)*(1 p[j]), for j>0 X(n, j) = X(n 1, j)*p[n+j], for j<n X(i, n) = X(i, n 1)*(1 p[i+n]), for i<n X(i, j) = X(i 1, j)*p[i+j] + X(i, j 1)*(1 p[i+j]), for 0<i<n and 0<j<n Next we can solve using dynamic programming
Dynamic programming algorithm: allocate array X[0 n][0 n]; for (i=0; i<=n; i++) for (j=0; j<=n; j++) if (i==0 && j==0) X[i][j] = 1; else if (i==n && j==n) X[i][j] = 0; else if ((i>0 && j==0) (i==n && j<n)) X[i][j] = X[i 1][j]*p[i+j]; else if ((i==0 && j>0) (i<n && j==n)) X[i][j] = X[i][j 1]*(1 p[i+j]); else X[i][j] = X[i 1][j]*p[i+j] + X[i][j 1]*(1 p[i+j]); Running time = θ(n 2 )
Example: n=4, so 2n 1=7 1 2 3 4 5 6 7 p.5.4.6.5.7.3.5 X j=0 j=1 j=2 j=3 j=4 i=0 1.5.3.12.06 i=1.5.5.38.25.075 i=2.2.38.38.289.2023 i=3.12.25.341.3254.1627 i=4.06.175.1023.1627 0 To illustrate, X[3][2] = X[3 1][2]*p[3+2] + X[3][2 1]*(1 p[3+2]) = X[2][2]*p[5] + X[3][1]*(1 p[5]) = (.38)*(.7) + (.25)*(.3) =.341