MAKING A BINARY HEAP

Transcription

1 CSE 101 Algorithm Design and Analysis Miles Jones sd.edu Office 4208 CSE Building Lecture 19: Divide and Conquer Design examples/dynamic Programming

2 MAKING A BINARY HEAP Base case. Break the problem up. Recursively solve each problem. Assume the algorithm works for the subproblems Combine the results.

3 MAKING A BINARY HEAP Let s assume that n = 2 k 1. Make a binary heap out of the list of objects: [ o 1, k 1,, o n, k n ]. Put (o 1, k 1 ) aside and break the remaining part into 2 each of size 2 k 1 1 Assume our algorithm works on the two subproblems This results in two binary heaps. Then make (o 1, k 1 ) the root and let it trickle down.

4 MAKING A BINARY HEAP

5 BINARY HEAP Put the first object as the root of the two subtrees and let it trickle down [M 13, D 8, B 10, J 11, C 12, H 8, L 9, Q 14, A 10, I 16, O 26, K 12, E 12, N 14, G 22 ]

6 BINARY HEAP D 8 A 10 Put the first object as the root of the two subtrees and let it trickle down B 10 H 8 E 12 K 12 J 11 C 12 L 9 Q 14 N 14 G 22 O 26 I 16 M 13 [M 13, D 8, B 10, J 11, C 12, H 8, L 9, Q 14, A 10, I 16, O 26, A 10, E 12, N 14, G 22 ]

7 BINARY HEAP M 13 D 8 A 10 Put the first object as the root of the two subtrees and let it trickle down B 10 H 8 E 12 K 12 J 11 C 12 L 15 Q 14 N 14 G 22 O 26 I 16 [M 13, D 8, B 10, J 11, C 12, H 8, L 9, Q 14, A 10, I 16, O 26, K 12, E 12, N 14, G 22 ]

8 BINARY HEAP D 8 Put the first object as the root of the two subtrees and let it trickle down B 10 M 13 H 8 E 12 A 10 K 12 J 11 C 12 L 15 Q 14 N 14 G 22 O 26 I 16 [M 13, D 8, B 10, J 11, C 12, H 8, L 9, Q 14, A 10, I 16, O 26, K 12, E 12, N 14, G 22 ]

9 BINARY HEAP D 8 Put the first object as the root of the two subtrees and let it trickle down B 10 H 8 M 13 E 12 A 10 K 12 J 11 C 12 L 15 Q 14 N 14 G 22 O 26 I 16 [M 13, D 8, B 10, J 11, C 12, H 8, L 9, Q 14, A 10, I 16, O 26, K 12, E 12, N 14, G 22 ]

10 MINIMUM DISTANCE Given a list of coordinates in the plane, find the distance between the closest pair.

11 MINIMUM DISTANCE distance( x i, y i, (x j, y j )) = x i y i 2 + x j y j 2

12 MINIMUM DISTANCE Given a list of coordinates, [ x 1, y 1,, x n, y n ], find the distance between the closest pair. Brute force solution? min = 0 for i from 1 to n-1: for j from i+1 to n: if min > distance( x i, y i, (x j, y j )) return min

13 MINIMUM DISTANCE Base case. Break the problem up. Recursively solve each problem. Assume the algorithm works for the subproblems Combine the results.

14 BASE CASE if n=2 then return distance( x 1, y 1, (x 2, y 2 ))

15 EXAMPLE y x m x

16 BREAK THE PROBLEM INTO SMALLER PIECES

17 BREAK THE PROBLEM INTO SMALLER PIECES We will break the problem in half. Sort the points by their x values. Then find a value x m such that half of the x values are on the left and half are on the right.

18 EXAMPLE y x m x

19 BREAK THE PROBLEM INTO SMALLER PIECES Usually the smaller pieces are each of size n/2. We will break the problem in half. Sort the points by their x values. Then find a value x m such that half of the x values are on the left and half are on the right. Perform the algorithm on each side. Assume our algorithm works!! What does that give us?

20 BREAK THE PROBLEM INTO SMALLER PIECES Usually the smaller pieces are each of size n/2. We will break the problem in half. Sort the points by their x values. Then find a value x m such that half of the x values are on the left and half are on the right. Perform the algorithm on each side. Assume our algorithm works!! What does that give us? It gives us the distance of the closest pair on the left and on the right and lets call them d L and d R

21 EXAMPLE y x m x

22 EXAMPLE y d L d R x m x

23 COMBINE How will we use this information to find the distance of the closest pair in the whole set?

24 COMBINE How will we use this information to find the distance of the closest pair in the whole set? We must consider if there is a closest pair where one point is in the left half and one is in the right half. How do we do this?

25 EXAMPLE y d L d R x m x

26 COMBINE How will we use this information to find the distance of the closest pair in the whole set? We must consider if there is a closest pair where one point is in the left half and one is in the right half. How do we do this? Let d = min(d L, d R ) and compare only the points (x i, y i ) such that x m d x i and x i x m + d. Worst case, how many points could this be?

27 EXAMPLE y d L d R d x m x

28 COMBINE STEP Let P m be the set of points within d of x m. Then P m may contain as many as n different points. So, to compare all the points in P m with each other would take many comparisons. So the runtime recursion is: n 2

29 COMBINE STEP Let P m be the set of points within d of x m. Then P m may contain as many as n different points. So, to compare all the points in P m with each other would take many comparisons. So the runtime recursion is: n 2 T n = 2T n + O n2 2 T n = O n 2 Can we improve the combine term?

30 EXAMPLE y d L d R d x m x

31 COMBINE STEP Given a point x, y P m, let s look in a 2d d rectangle with that point at its upper boundary: x, y How many points could possibly be in this rectangle?

32 COMBINE STEP Given a point x, y its upper boundary: P m, let s look in a 2d d rectangle with that point at There could not be more than 8 points total because if we divide the rectangle into 8 d squares then there can never be more than one point per square. 2 d 2 Why???

33 COMBINE STEP So instead of comparing (x, y) with every other point in P m we only have to compare it with the next 7 points lower than it. To gain quick access to these points, let s sort the points in P m by y values. Now, if there are k vertices in P m we have to sort the vertices in O(klog k) time and make at most 7k comparisons in O(k) time for a total combine step of O k log k. But we said in the worst case, there are n vertices in P m and so worst case, the combine step takes O(n log n) time.

34 COMBINE STEP But we said in the worst case, there are n vertices in P m and so worst case, the combine step takes O(n log n) time. Runtime recursion: T n = 2T n 2 + O(n log n)

35 log b n T n = O n d k=1 a b d k

36 COMBINE STEP But we said in the worst case, there are n vertices in P m and so worst case, the combine step takes O(n log n) time. Runtime recursion: T n = 2T n 2 + O(n log n) Can anyone improve on this runtime?

37 DYNAMIC PROGRAMMING Dynamic programming is an algorithmic paradigm in which a problem is solved by identifying a collection of subproblems and tackling them one by one, smallest first, using the answers to small problems to help figure out larger ones, until they are all solved. Examples:

38 DYNAMIC PROGRAMMING Dynamic programming is an algorithmic paradigm in which a problem is solved by identifying a collection of subproblems and tackling them one by one, smallest first, using the answers to small problems to help figure out larger ones, until they are all solved. Examples: findmax, findmin, fib2,

39 DYNAMIC PROGRAMMING (SHORTEST PATH IN DAGS)

40 DYNAMIC PROGRAMMING (SHORTEST PATH IN DAGS)

41 DYNAMIC PROGRAMMING (SHORTEST PATH IN DAGS) Shortest distance from D to another node x will be denoted dist(x). Notice that the shortest distance from D to C is dist C =

42 DYNAMIC PROGRAMMING (SHORTEST PATH IN DAGS) Shortest distance from D to another node x will be denoted dist(x). Notice that the shortest distance from D to C is dist C = min(dist E + 5, dist B + 2)

43 DYNAMIC PROGRAMMING (SHORTEST PATH IN DAGS) Shortest distance from D to another node x will be denoted dist(x). Notice that the shortest distance from D to C is dist C = min(dist E + 5, dist B + 2) This kind of relation can be written for every node. Since it s a DAG, the arrows only go to the right so by the time we get to node x, we have all the information needed!!

44 DYNAMIC PROGRAMMING (SHORTEST PATH IN DAGS) Step1: Define the subproblems: Step 2: Base Case: Step 3: express recursively: Step 4: order the subproblems

45 DYNAMIC PROGRAMMING (SHORTEST PATH IN DAGS) Step1: Define the subproblems: the distance to the ith vertex Step 2: Base Case: the distance to the first vertex to itself is 0 Step 3: express recursively: dist(v) = min (u,v) E dist u + l u, v Step 4: order the subproblems linearized order

46 DYNAMIC PROGRAMMING (SHORTEST PATH IN DAGS) initialize all dist(.) values to infinity dist(s):=0 for each v V\{s} in linearized order dist(v)= min (u,v) E dist u + l u, v Like D/C, this algorithm solves a family of subproblems. We start with dist(s)=0 and we get to the larger subproblems in linearized order by using the smaller subproblems.

47 DP (LONGEST INCREASING SUBSEQUENCE) Given a sequence of distinct positive integers a[1],,a[n] An increasing subsequence is a sequence a[i_1],,a[i_k] such that i_1< <i_k and a[i_1]< <a[i_k]. For Example: 15, 18, 8, 11, 5, 12, 16, 2, 20, 9, 10, 4 5, 16, 20 is an increasing subsequence. How long is the longest increasing subsequence?

48 DP (LONGEST INCREASING SUBSEQUENCE) Let s make a DAG out of our example:

49 DP (LONGEST INCREASING SUBSEQUENCE) Let s make a DAG out of our example: Now, instead of finding the longest increasing subsequence of a list of integers, we are finding the longest path in a DAG!!!!

50 DYNAMIC PROGRAMMING (LONGEST INCREASING SUBSEQUENCE) Step1: Define the subproblems: Step 2: Base Case: Step 3: express recursively: Step 4: order the subproblems

51 DYNAMIC PROGRAMMING (LONGEST INCREASING SUBSEQUENCE) Step1: Define the subproblems: L(k) will be the length of the longest increasing subsequence ending exactly at position k Step 2: Base Case: L(1) = 0 Step 3: express recursively: L(k) = 1+max({L[i]:(i,j) is an edge}) Step 4: order the subproblems from left to right

52 DP (LONGEST INCREASING SUBSEQUENCE) Finding longest path in a DAG: L[1]:=0 for j=1 n L[j]=1+max({L[i]:(i,j) is an edge}) prev(j)=i return max({l[j]})

53 DP (LONGEST INCREASING SUBSEQUENCE) Let s make a DAG out of our example:

54 DP (LONGEST INCREASING SUBSEQUENCE) Let s make a DAG out of our example:

55 DP (LONGEST INCREASING SUBSEQUENCE) Finding longest path in a DAG: L[1]:=0 for j=1 n L[j]=max({L[i]:(i,j) is an edge}) prev(j)=i return max({l[j]}) How long does this take?

56 DP (LONGEST INCREASING SUBSEQUENCE) Finding longest path in a DAG: L[1]:=0 for j=1 n L[j]=max({L[i]:(i,j) is an edge}) prev(j)=i return max({l[j]}) How long does this take? To solve L[j]=max({L[i]:(i,j) is an edge}), we need to know L[i] for each edge (i,j) in E. This is equal to the indegree of j. So we sum over all vertices we get that j V so the runtime is O( E ). d in (j) = E

57 DP (LONGEST INCREASING SUBSEQUENCE) The runtime is dependent on the number of edges in the DAG. Note that if the sequence is increasing If the sequenece is decreasing then

58 DP (LONGEST INCREASING SUBSEQUENCE) The runtime is dependent on the number of edges in the DAG. What are the maximum and minimum number of edges?

59 DP (LONGEST INCREASING SUBSEQUENCE) The runtime is dependent on the number of edges in the DAG. Note that if the sequence is increasing then E = n If the sequenece is decreasing then E =

60 DP (LONGEST INCREASING SUBSEQUENCE) What is the expected number of edges?