Lecture 23: Dynamic Programming: the algorithm

Transcription

1 Lecture 23: Dynamic Programming: the algorithm University of Southern California Linguistics 2 USC Linguistics November 1, 201 Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, / 1

2 Shortest Path problem SEA DET CLE LA DEN CHI DC NY DAL PHL Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, / 1

3 Shortest Path problem How do we solve this? Solution 1: exhaustive search How do we do that? For simplicity, we change the names of the cities to A,B,C, etc, Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, / 1

4 Exhaustive Search B D G A C E F H I J ABDGJ, ABEHJ, ABEGJ... Cost(ABDGJ) = + + +,... Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, / 1

5 Exhaustive Search Algorithm List every possible path Add costs on the path, and sum Find minimal sum Answer is the path corresponding to minimum sum Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, 201 / 1

6 Exhaustive Search B D G A C E F H I J Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, 201 / 1

7 Exhaustive Search results ABDGJ=30 ABDHJ=33 ABDIJ=3 ABEGJ=32 ABEHJ=34 ABEHI=3 ABFGJ=3 ABFHJ=3 ABFHI=3 ACDGJ=34 ACDHJ=3 ACDIJ=3 ACEGJ=31 ACEHJ=33 ACEHI=34 ACFGJ=33 ACFHJ=34 ACFHI=34 Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, 201 / 1

8 Exhaustive Search: So what s the problem? For this small network, there is no problem. But for a general network, it is too slow We can think of the network as a consisting of stages, and at each stage, there are several states. At each stage, the decision is between the n states in the next stage. The total number of decision combinations = (# states in stage 1) x (# states in stage 2) x (# states in stage 3)... As number of states increases, the number of combinations of decisions to be considered gets to be very large, staggeringly so. For example, if there are stages, each with 0 states (not an unrealistic real-world problem), the number of paths to be considered is 20, that is 1 followed by 20 zeroes. Also, in this algorithm, the cost associated with sub-paths needs to calculated over and over (e.g., cost(ab) needs to be computed times, cost(gj) times, etc.) Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, 201 / 1

9 Greedy Search (Suckers!) Alternative strategy: At every choice point, choose the lowest cost path Greedy, short-sighted, doesn t consider possible future paths It seems like it should work, but it doesn t. Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, 201 / 1

10 Greedy Search (Suckers!) B D G A C E F H I J A C E H J = 33 Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, 201 / 1

11 Greedy Search (Suckers!) So it doesn t work. Optimal solution (ABDGJ) has a cost of 30. Why doesn t it work? Selecting on path on an early stage may eliminate cheap paths on later stages. Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, 201 / 1

12 Dynamic Programming Approach Dynamic Programming is an alternative search strategy that is faster than Exhaustive search, slower than Greedy search, but gives the optimal solution. View a problem as consisting of subproblems: Aim: Solve main problem To achieve that aim, you need to solve some subproblems To achieve the solution to these subproblems, you need to solve a set of subsubproblems And so on... Dynamic Programming works when the subproblems have similar forms, and when the tiniest subproblems have very easy solutions. Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, 201 / 1

13 Dynamic Programming B D G A C E H J F I Main Problem: Shortest path from A to J: V AJ DP Thinking: Let s say I know the best paths from B to J and C to J: V BJ and V CJ. Then we would add V BJ to the cost from A to B, V CJ to the cost from C to J, compare the two, and pick the least. So we now have 2 subproblems V BJ and V CJ. If we could solve those subproblems, we could solve the main problem V AJ. But now we can think of V BJ as its own problem, and then repeat the thinking: If I knew the solutions to V DJ, V EJ, and V FJ, then we can solve V BJ. Same with V CJ. Continue thinking in this way till we get to: V GJ, V HJ, V IJ, which are easy to solve! Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, / 1

14 Dynamic Programming B D G A C E H J F I V AJ = min + VBJ +V CJ Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, 201 / 1

15 Dynamic Programming B D G A C E H J + VBJ V AJ = min +V CJ F < +V DJ = V BJ = min +V EJ +V FJ < + V DJ = V CJ = min + V EJ + V FJ I Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, / 1

16 Dynamic Programming B D G A C + VBJ VAJ = min +VCJ E H F I < +VGJ = VDJ = min +VHJ < +VDJ= + VIJ VBJ = min +VEJ +VFJ < +VGJ = VEJ = min +VHJ < + VDJ= +VIJ VCJ = min + VEJ + VFJ < + VGJ= VFJ = min +VHJ +VIJ J Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, / 1

17 Dynamic Programming B D G A C E F < +V GJ = V DJ = min +V HJ < +V DJ = + V IJ V BJ = min +V EJ V GJ = +V FJ < +V GJ = + VBJ V AJ = min V EJ = min +V V HJ = +V HJ CJ < + V DJ = +V IJ V CJ = min + V EJ V : IJ = ; + V FJ < + V GJ = V FJ = min +V HJ +V IJ H I Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, / 1 J

18 Dynamic Programming B D G A E H J C G F I D < +V GJ = 13 V DJ = min +V HJ 1 < +V DJ = 1 + V IJ 1 B V V GJ = BJ = min +V EJ 20 +V FJ 24 < +V GJ = + VBJ 30 V AJ = min V EJ = min +V V HJ = HJ 1 +V CJ 31 < + V DJ = 2 +V IJ 1 V CJ = min + V EJ 24 V IJ = + V FJ 2 < + V GJ = 1 V FJ = min +V HJ 1 +V IJ 1 Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, / 1

19 Programming the problem solutions We are going to learn to write code for the dynamic programming solution to our little shortest path problem. Why? Programming requires explicit understanding of the problem and the solution Programming allows you to test your understanding of the problem and the solution (you don t need anyone to tell you if you are right; the program will work or not). Programming requires abstract characterization of the problem and the solution. The abstraction sometimes gives you some new insight into your problem, and also how it might be similar to other problems and give you insight into them. We begin by programming the greedy algorithm, because it is simpler. Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, / 1

20 Programming greedy search A B C D E F G H I First step is to replace the letter names for nodes(states) with a numbering scheme (i, j): i is the stage number j is the state number within a stage We do that so can write our code in a really simple way, and so the code can immediately be generalized to other networks with different numbers of stages and states. Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, / 1 J

21 Programming Greedy Search 2, 1 3, 1 4, 1 1, 1 2, 2 3, 2 3, 3 4, 2 4, 3, 1 Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, / 1

22 Defining the network Create a matrix called costs which has four dimensions: costs(i,j,k,n) i = stage of begin node j = state of begin node k = stage of end node n = state of end node Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, / 1

23 Defining the network 2, 1 3, 1 4, 1 costs(1,1,2,1) = ; costs(1,1,2,2) = ; costs(2,1,3,1) = ; costs(2,1,3,2) = ; costs(2,1,3,3) = ; 1, 1 2, 2 3, 2 3, 3 4, 2 4, 3, 1 costs(2,2,3,1) = ; costs(2,2,3,2) = ; costs(2,2,3,3) = ; costs(3,1,4,1) = ; costs(3,1,4,2) = ; costs(3,1,4,3) = ; costs(3,2,4,1) = ; costs(3,2,4,2) = ; costs(3,2,4,3) = ; costs(3,3,4,1) = ; costs(3,3,4,2) = ; costs(3,3,4,3) = ; costs(4,1,,1) = ; costs(4,2,,1) = ; costs(4,3,,1) = ; Linguistics 2 (USC Linguistics) Lecture 23: Dynamic Programming: the algorithm November 1, / 1