Lecture 9. Greedy Algorithm

Transcription

1 Lecture 9. Greedy Algorithm T. H. Cormen, C. E. Leiserson and R. L. Rivest Introduction to Algorithms, 3rd Edition, MIT Press, 2009 Sungkyunkwan University Hyunseung Choo Copyright Networking Laboratory

2 Greedy Algorithms A greedy algorithm always makes the choice that looks best at the moment My everyday examples Playing cards Invest on stocks Choose a university The hope A locally optimal choice will lead to a globally optimal solution Algorithms Networking Laboratory 2/45

3 Introduction Similar to Dynamic Programming It applies to Optimization Problem When we have a choice to make, make the one that looks best right now Make a locally optimal choice in hope of getting a globally optimal solution Greedy algorithms don t always yield an optimal solution, but sometimes they do For many problems, it provides an optimal solution much more quickly than a dynamic programming approach Algorithms Networking Laboratory 3/45

4 An Activity Selection Problem The problem of scheduling several competing activities that require exclusive use of a common resource n activities require exclusive use of a common resource For example, scheduling the use of a classroom Set of activities S={a 1,., a n }. a i needs resource during period [ s i, f i ) [ ) is a half-open interval s i = start time and f i = finish time Goal Select the largest possible set of non-overlapping mutually compatible) activities Note: Could have many other objectives Schedule room for longest time Maximize income rental fees Algorithms Networking Laboratory 4/45

5 An Activity Selection Problem Here are a set of start and finish times What is the maximum number of activities that can be completed? Algorithms Networking Laboratory 5/45

6 An Activity Selection Problem What is the maximum number of activities that can be completed? {a3, a9, a11} can be completed But so can {a1, a4, a8, a11} which is a larger set But it is not unique, consider {a2, a4, a9, a11} Algorithms Networking Laboratory 6/45

7 Algorithms Networking Laboratory 7/45

8 Algorithms Networking Laboratory 8/45

9 Algorithms Networking Laboratory 9/45

10 An The Optimal Substructure of the A.-S. Problem S ij = {a k S : f i s k < f k s j } = activities that start after a i finishes and finish before a j starts Activities in S i j are compatible with all activities that finish by f i, and all activities that start no earlier than s j To represent the entire problem, add fictitious activities: a0 = [ -, 0 ) an+1 = [, +1 ) We don t care about - in a 0 or +1 in a n+1 Then S = S 0, n+1 Range for S i j is 0 i, j n + 1 Algorithms Networking Laboratory 10/45

11 An The Optimal Substructure of the A.-S. Problem Assume that activities are sorted by monotonically increasing finish time f 0 f 1 f 2 f n < f n+1 Then i j S i j = If there exists ak S i j : But i j f i f j. Contradiction f i s k < f k s j < f j f i < f j So only need to worry about S i j with 0 i < j n + 1 All other S i j are Algorithms Networking Laboratory 11/45

12 An The Optimal Substructure of the A.-S. Problem Suppose that a solution to S i j includes ak. Have 2 subproblems: S i k start after a i finishes, finish before ak starts) S k j start after ak finishes, finish before a j starts) Solution to S i j { solution to Sik } {a k } { solution to S k j } Since a k is in neither subproblem the subproblems are disjoint solution to S = solution to S ik solution to S kj Algorithms Networking Laboratory 12/45

13 An The Optimal Substructure of the A.-S. Problem If an optimal solution to S ij includes a k, then the solutions to S ik and S kj used within this solution must be optimal as well Let A ij = optimal solution to S ij So A i j = A i k {a k } A k j, assuming: S i j is nonempty we know a k Algorithms Networking Laboratory 13/45

14 An The Optimal Substructure of the A.-S. Problem c[i, j] : size of maximum-size subset of mutually compatible activities in S ij i j S i j = c[i, j] = 0 If S ij, suppose we know that a k is in the subset c [i, j] = c [i, k] c [k, j]. But, we don t know which k to use, and so Algorithms Networking Laboratory 14/45

15 Early Finish Greedy Select the activity with the earliest finish Eliminate the activities that could not be scheduled Repeat! Algorithms Networking Laboratory 15/45

16 Algorithms Networking Laboratory 16/45

17 Algorithms Networking Laboratory 17/45

18 Algorithms Networking Laboratory 18/45

19 Algorithms Networking Laboratory 19/45

20 Algorithms Networking Laboratory 20/45

21 Algorithms Networking Laboratory 21/45

22 Algorithms Networking Laboratory 22/45

23 A Recursive Greedy Algorithm Assumes activities already sorted by monotonically increasing finish time If not, then sort in On lg n) time Return an optimal solution for S i,n+1 Initial call: REC-ACTIVITY-SELECTORs, f, 0, n) Time: n) each activity examined exactly once Algorithms Networking Laboratory 23/45

24 An Iterative Greedy Algorithm Time n) Algorithms Networking Laboratory 24/45

25 Greedy-Choice Property A globally optimal solution can be arrived at by making a locally optimal greedy) choice Dynamic programming Make a choice at each step Choice depends on knowing optimal solutions to subproblems Solve subproblems first Bottom-up manner Greedy algorithm Make a choice at each step Make the choice before solving the subproblems Top-down fashion Algorithms Networking Laboratory 25/45

26 The Knapsack Problem The famous knapsack problem: A thief breaks into a museum. Fabulous paintings, sculptures, and jewels are everywhere. The thief has a good eye for the value of these objects, and knows that each will fetch hundreds or thousands of dollars on the clandestine art collector s market. But, the thief has only brought a single knapsack to the scene of the robbery, and can take away only what he can carry. What items should the thief take to maximize the haul? Algorithms Networking Laboratory 26/45

27 Greedy Algorithm vs. Dynamic Programming The knapsack problem Good example of the difference 0-1 knapsack problem: not solvable by greedy n items Item i is worth $v i, weighs w i pounds Find a most valuable subset of items with total weight W Have to either take an item or not take it can t take part of it Algorithms Networking Laboratory 27/45

28 Greedy Algorithm vs. Dynamic Programming Fractional knapsack problem: Solvable by greedy Like the 0-1 knapsack problem, but can take fraction of an item Both have optimal substructure But the fractional knapsack problem has the greedy-choice property, and the 0-1 knapsack problem does not To solve the fractional problem, rank items by value/weight v i / w i Let v i / w i v i+1 / w i+1 for all i Algorithms Networking Laboratory 28/45

29 0-1 Knapsack Problem i v i w i v i /w i W = 50 Greedy solution Take items 1 and 2 value = 160, weight = 30 Have 20 pounds of capacity left over Optimal solution Take items 2 and 3 value = 220, weight = 50 No leftover capacity Algorithms Networking Laboratory 29/45

30 Knapsack Problem Algorithms Networking Laboratory 30/45

31 0-1 Knapsack Dynamic Programming Pi, w) the maximum profit that can be obtained from items 1 to i, if the knapsack has size w Case 1: thief takes item i Pi, w) = Case 2: thief does not take item i Pi, w) = v i + Pi - 1, w-w i ) Pi - 1, w) Algorithms Networking Laboratory 31/45

32 0-1 Knapsack Dynamic Programming Item i was taken Item i was not taken Pi, w) = max {v i + Pi - 1, w-w i ), Pi - 1, w) } 0 : 1 w - w i w W first second i-1 i n 0 Algorithms Networking Laboratory 32/45

33 0 Example: Pi, w) = max {v i + Pi - 1, w-w i ), Pi - 1, w) } W = 5 P1, 1) = P0, 1) = 0 Item Weight Value P1, 2) = max{12+0, 0} = P1, 3) = max{12+0, 0} = P1, 4) = max{12+0, 0} = P1, 5) = max{12+0, 0} = 12 P2, 1)= max{10+0, 0} = 10 P3, 1)= P2,1) = 10 P4, 1)= P3,1) = 10 P2, 2)= max{10+0, 12} = 12 P3, 2)= P2,2) = 12 P4, 2)= max{15+0, 12} = 15 P2, 3)= max{10+12, 12} = 22 P3, 3)= max{20+0, 22}=22 P4, 3)= max{15+10, 22}=25 P2, 4)= max{10+12, 12} = 22 P3, 4)= max{20+10,22}=30 P4, 4)= max{15+12, 30}=30 P2, 5)= max{10+12, 12} = 22 P3, 5)= max{20+12,22}=32 P4, 5)= max{15+22, 32}=37 Algorithms Networking Laboratory 33/45

34 Reconstructing the Optimal Solution Item 4 Item 2 Item 1 Start at Pn, W) When you go left-up item i has been taken When you go straight up item i has not been taken Algorithms Networking Laboratory 34/45

35 0-1 Knapsack Dynamic Programming P[ k -1, w] P[ k, w] max{ P[ k -1, w - w ] + v, P[ k Define P[k,w] to be the best selection from S k with weight at most w Since P[k,w] is defined in terms of P[k-1,*], we can use two arrays of instead of a matrix Running time: OnW) Not a polynomial-time algorithm since W may be large This is a pseudo-polynomial time algorithm k k -1, w]} if w else k w Algorithm 0-1KnapsackS, W): Input: set S of n items with benefit v i and weight w i ; maximum weight W Output: benefit of best subset of S with weight at most W let A and B be arrays of length W + 1 for w 0 to W do B[w] 0 for k 1 to n do copy array B into array A for w w k to W do if A[w-w k ] + v k > A[w] then B[w] A[w-w k ] + v k return B[W] Algorithms Networking Laboratory 36/45

36 Self-Study Section 16.3 Huffman codes Algorithms Networking Laboratory 37/45

37 Huffman Codes Huffman codes compress data very effectively savings of 20% to 90% are typical Data compression by assigning binary codes to characters Huffman s greedy algorithm uses a table giving how often each character occurs i.e., its frequency) It builds up an optimal way of representing each character as a binary string. Example: Storing a data file of 100,000 characters contains only six different characters: a, b, c, d, e,f 300,000 bits 224,000 bits Algorithms Networking Laboratory 38/45

38 Prefix Codes 1/2) In a prefix code, no code is also a prefix of some other code Any optimal character code has an equally optimal prefix code Example: abc = Easy encoding and decoding Use a binary tree for decoding Once a string of bits matches a character code, out put that character with no ambiguity no need to look ahead) Algorithms Networking Laboratory 39/45

39 Prefix Code 2/2) An optimal code for a file is always represented by a full binary tree in which every non-leaf node has two children Given a tree T corresponding to a prefix code, we can easily compute the number of bits required to encode a file Let C = set of unique characters in file Let fc) = frequency of character c in file Let d T c) = depth of c s leaf node in T The number of bits required to encode a file cost of the tree T) c C B[ T] f c) d c) T Algorithms Networking Laboratory 40/45

40 Constructing a Huffman Code 1/2) Build a tree T by merging C nodes in a bottom-up manner Use a min-priority queue Q to keep nodes ordered by frequency Algorithms Networking Laboratory 41/45

41 Constructing a Huffman Code 2/2) Algorithms Networking Laboratory 42/45

42 Correctness of Huffman s Algorithm 1/3) Huffman exhibits the greedy choice property Given a tree T representing an arbitrary optimal prefix code Let a and b be two characters that are sibling leaves of maximum depth in a tree T Assume x and y have lowest frequencies There exists an optimal code in which x and y are at the maximum depth greedy choice) Prove that moving x to the bottom similarly, y to the bottom) yields a better optimal) solution Algorithms Networking Laboratory 43/45

43 Algorithms Correctness of Huffman s Algorithm 2/3) Assume fx) fy) and fa) fb). We know fx) fa) and fy) fb) B[T] B[T ] similarly, B[T] B[T ]) Networking Laboratory 44/45 ) ) ) ) ] [ ] [ ' ' c d c f c d c f B T T B T C c T C c )) ) )) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ' ' x d a d x f a f x d a f a d x f a d a f x d x f a d a f x d x f a d a f x d x f T T T T T T T T T T

44 Correctness of Huffman s Algorithm 3/3) Huffman exhibits optimal substructure Consider the optimal tree T for characters C Let x and y be lowest frequency characters in C Consider the optimal tree T for C =C-{x,y}U{z}, where fz)=fx)+fy) If there is a better tree for C, call it T, then we could use T to build a better original tree by adding in x and y under z The optimal tree T is optimal, so this is a contradiction Thus T is the optimal tree for C Huffman produces optimal prefix code Immediate from above claims Algorithms Networking Laboratory 45/45