Greedy Alg: Huffman abhi shelat

Transcription

1 L15 Greedy Alg: Huffman abhi shelat

2 Huffman Coding

3 image: wikimedia

4 In testimony before the committee, Mr. Lew stressed that the Treasury Department would run out of extraordinary measures to free up cash in a matter of days. At that point, the country s bills might overwhelm its cash on hand plus any receipts from taxes or other sources, leading to an unprecedented default.mr. Lew said that Treasury had no workarounds to avoid breaching the debt ceiling. There is no plan other than raising the debt limit, he said. The legal issues, even regarding interest and principal on the debt, are complicated.

5 m In testimony before the committee, Mr. Lew stressed that the Treasury Department would run out of extraordinary measures to free up cash in a matter of days. At that point, the country s bills might overwhelm its cash on hand plus any receipts from taxes or other sources, leading to an unprecedented default.mr. Lew said that Treasury had no workarounds to avoid breaching the debt ceiling. There is no plan other than raising the debt limit, he said. The legal issues, even regarding interest and principal on the debt, are complicated.

6 def: cost of an encoding B(T,{f c })= X c2c f c `c e: i: o: u: p: g: b: f:

7 e: 240 i: a: 199 o: r: n: 1521 t: s: 192 l: c: 1007 u: 7211 p: 7077 m: d: 6007 h: y: g: b: 4051 f: v: 2010 k: w: 125 z: 49 x: 6926 q: 729 j: character frequency e i a o r n t s l c u p m d h y g b f v k w z x q j

8 morse code

9 def: prefix code e: 25 0 i: o: u: p: g: b: f:

10 prefix code binary tree

11 e: 25 0 i: o: u: p: g: b: f: code to binary tree

12 e: i: o: u: p: e i o u p use tree to encode messages

13 goal given the character frequencies {f c } c C produce a prefix code T with smallest cost min T B(T,{f c})

14 property y Lemma:optimal tree must be full. x a b

15 divide & conquer?

16 counter-example e: 2 i: 25 o: 20 u: 1 p: 5

17 )"!'(!## &$# %$ $% "$ "#!" e i o u p g b f

18 !'(!## &$# %$ $% "$ )" e i o u p g "#!" b f

19 !'(!## &$# %$ $% )" "$ e i o u p g "#!" b f

20 200 i o 1 25 e e: i: o: u: p: g: b: f: u 7 p g 40 b 24 f

21 objective

22 lemma: exchange argument

23 lemma: T exchange argument Let x, y C be characters with smallest frequencies f x,f y. There exists an optimal prefix code T for C in which x, y are siblings. That is, the codes for x, y have the same length and only di er in the last bit. The optimal solution for consists of computing an optimal solution for y x a b

24 lemma: exchange argument Let x, y C be characters with smallest frequencies f x,f y. There exists an optimal prefix code T for C in which x, y are siblings. That is, the codes for x, y have the same length and only di er in the last bit. The optimal solution for consists of computing an optimal solution for T T y b x a a b x y

25 exchange argument proof: lemma: Let x, y C be characters with smallest frequencies f x,f y. There exists an optimal prefix code T for C in which x, y are siblings. That is, the codes for x, y have the same length and only di er in the last bit. The optimal solution for consists of computing an optimal solution for

26 exchange argument lemma: Let x, y C be characters with smallest frequencies f x,f y. There exists an optimal prefix code T for C in which x, y are siblings. That is, the codes for x, y have the same length and only di er in the last bit. The optimal solution for consists of computing an optimal solution for T y first step x a b

27 exchange argument lemma: Let x, y C be characters with smallest frequencies f x,f y. There exists an optimal prefix code T for C in which x, y are siblings. That is, the codes for x, y have the same length and only di er in the last bit. The optimal solution for consists of computing an optimal solution for T T y first step y x a a b x b f a f b f x f y f y f b f x f a

28 T T y y x f x f a a a b x b

29 T T y y x f x f a a a b x b B(T )= f c c + f x x + f a a B(T )= f c c + f x x + f a a c c B(T ) B(T ) 0

30 T T y b a a x b x y B(T ) B(T ) 0

31 T T T y y b x a a a b x b x y

32 T T T y y b x a a a b x b x y B(T ) B(T ) 0 B(T ) B(T ) 0 Tis also optimal

33 exchange argument lemma: Let x, y C be characters with smallest frequencies f x,f y. There exists an optimal prefix code T for C in which x, y are siblings. That is, the codes for x, y have the same length and only di er in the last bit. The optimal solution for consists of computing an optimal solution for T T y b x a a b x y

34 f c!'(!## &$# %$ $% "$ optimal sub-structure f x "# f y!"

35 f c optimal sub-structure!'(!## &$# %$ $% "$ f x "# f y!" problem of size n!'(!## &$# %$ $% "$ )" f c problem of size n-1 f z

36 f c optimal sub-structure!'(!## &$# %$ $% "$ problem of size n!'(!## &$# %$ $% "$ )" f c problem of size n-1 f x "# f z f y!" Lemma:

37 f c optimal sub-structure!'(!## &$# %$ $% "$ problem of size n!'(!## &$# %$ $% "$ )" f c problem of size n-1 f x "# f z f y!" Lemma: The optimal solution for T consists of computing an optimal solution for T and replacing the left z with a node having children x, y.

38 T z z

39 T T z x y B(T ) B(T )

40 T T z x y B(T ) B(T ) B(T )=B(T ) f x f y

41 Suppose T is not optimal

42 Suppose U T is not optimal B(U) <B(T ) x y

43 Suppose U T is not optimal B(U) <B(T ) x y U z

44 Suppose U T is not optimal B(U) <B(T ) x y B(U )=B(U) f x f y U < B(t) - fx - fy z But this implies that B(T ) was not optimal.

45 therefore T z x y

46 summary of argument

47 MST

48 connecting houses image: thefranciscofamily.org,

49 connecting houses image: thefranciscofamily.org,

50 connecting houses image: thefranciscofamily.org,

51 connecting houses image: thefranciscofamily.org,

52 connecting houses image: thefranciscofamily.org,

53 connecting houses image: thefranciscofamily.org,

54 connecting houses image: thefranciscofamily.org,

55 connecting houses image: thefranciscofamily.org,

56 b d g 10 2 a 9 e 5 9 i c 1 f 6 h

57 graphs clrs [ch 22]

58

59 representation G =(V,E) adjacency list space: time list neighbors: time check an edge:

60 representation G =(V,E) adjacency matrix space: time list neighbors: time check an edge:

61 definition: path a sequence of nodes with the property that simple path: cycle:

62 definition:tree connected graph: a tree is

63 what we want: b d g 10 2 a 9 e 5 9 i c 1 f 6 h

64 minimum spanning tree looking for a set of edges that T E (a) connects all vertices (b) has the least cost min (u,v) T w(u, v)

65 looking for a set of edges thatt E (a) connects all vertices (b) has the least cost facts min (u,v) T w(u, v) how many edges does solution have? does solution have a cycle?

66 strategy start with an empty set of edges A repeat for v-1 times: add lightest edge that does not create a cycle

67 example b d g a 9 e 5 9 i c 1 f 6 h

68 kruskal b d g a 9 e 5 9 i c 1 f 6 h

69 kruskal b d g a 9 e 5 9 i c 1 f 6 h

70 kruskal b d g a 9 e 5 9 i c 1 f 6 h

71 kruskal b d g a 9 e 5 9 i c 1 f 6 h

72 kruskal b d g a 9 e 5 9 i c 1 f 6 h

73 kruskal b d g a 9 e 5 9 i c 1 f 6 h

74 kruskal b d g a 9 e 5 9 i c 1 f 6 h

75 kruskal b d g a 9 e 5 9 i c 1 f 6 h

76 kruskal e d f c b a i g h 7 e d f c b a i g h 7 e d f c b a i g h 7 e d f c b a i g h 7 e d f c b a i g h 7 e d f c b a i g h 7 e d f c b a i g h 7 e d f c b a i g h 7

77 why does this work? 1 T 2 repeat V 1 times: add to T the lightest edge e E that does not create a cycle

78 definition: cut

79 example of a cut b d g a 9 e 5 9 i c 1 f 6 h

80 definition: crossing a cut

81 definition: crossing a cut an edge e =(u, crosses v) a graph cut (S,V-S) if u S v V S b d g a 9 e 5 9 i c 1 f 6 h

82 example of a crossing b d g a 9 e 5 9 i c 1 f 6 h

83 definition: respect

84 theorem: cut property

85 suppose the set of edges thm: cut property A is part of an m.s.t. let (S, V S) be any cut that respects A. let edge e be the min-weight edge across (S, V S) then: A {e} is part of an m.s.t.

86 example of theorem b d g a 9 e 5 9 i c 1 f 6 h

87 d 5 7 h 9 g 11 2 e 6 i b f a 12 c

88 Theorem 2 Suppose the set of edges A is part of a minimum spanning tree of G = (V, E). Let (S, V S) be any cut that respects A and let e be the edge with the minimum weight that crosses (S, V S). Then the set A {e} is part of a minimum spanning tree.

89 proof of cut thm d 7 5 g i e h 6 c 1 f 9 12 b 10 a

90 Kruskal-pseudocode(G) correctness 1 A 2 repeat V 1 times: add to A the lightest edge e E that does not create a cycle

91 Kruskal-pseudocode(G) correctness 1 A 2 repeat V 1 times: add to A the lightest edge e E that does not create a cycle proof: by induction. in step 1, A is part of some MST. suppose that after k steps, A is part of some MST (line 2). in line, we add an edge e to A.

92 cases for edge e

93 cases for edge e e must be lightest edge crossing

94 analysis? Kruskal-pseudocode(G) 1 A 2 repeat V 1 times: add to A the lightest edge e E that does not create a cycle

95 General-MST-Strategy(G =(V,E)) 1 A 2 repeat V 1 times: Pick a cut (S, V S) that respects A 4 Let e be min-weight edge over cut (S, V S) 5 A A {e}

96 prim General-MST-Strategy(G =(V,E)) 1 A 2 repeat V 1 times: Pick a cut (S, V S) that respects A 4 Let e be min-weight edge over cut (S, V S) 5 A A {e} A is a subtree edge e is lightest edge that grows the subtree

97 prim b d g a 9 e 5 9 i c 1 f 6 h

98 prim b d g a 9 e 5 9 i c 1 f 6 h

99 prim b d g a 9 e 5 9 i c 1 f 6 h

100 prim b d g a 9 e 5 9 i c 1 f 6 h

101 prim b d g a 9 e 5 9 i c 1 f 6 h

102 prim b d g a 9 e 5 9 i c 1 f 6 h

103 prim b d g a 9 e 5 9 i c 1 f 6 h

104 prim b d g a 9 e 5 9 i c 1 f 6 h

105 prim b d g a 9 e 5 9 i c 1 f 6 h

106 idea: implementation

107 implementation

108 new data structure

109 binary heap full tree, key value <= to key of children

110 binary heap full tree, key value <= to key of children ' &# ( && &' ) :!' ""

111 binary heap full tree, key value <= to key of children ' &# ( && &' ) :!' "" %

112 binary heap full tree, key value <= to key of children ' &# ( && % ) :!' "" &'

113 binary heap full tree, key value <= to key of children how to extractmin? ' % ( && &# ) :!' "" &'

114 binary heap full tree, key value <= to key of children how to extractmin? &' % ( && &# ) :!' "" &'

115 binary heap full tree, key value <= to key of children how to extractmin? ( % &' && &# ) :!' "" &'

116 binary heap full tree, key value <= to key of children how to extractmin? ( % ) && &# &' :!' "" &'

117 binary heap full tree, key value <= to key of children how to decreasekey? ( % ) && &# &' :!' ""

118 binary heap full tree, key value <= to key of children how to decreasekey? ( $ ) && % &' :!' ""

119 implementation use a priority queue to keep track of light edges insert: makequeue: extractmin: decreasekey:

120 algorithm

121 prim(g =(V,E)) implementation 1 Q Q is a Priority Queue 2 Initialize each v V with key k v, v nil Pick a starting node r and set k r 0 4 Insert all nodes into Q with key k v. 5 while Q = 6 do u extract-min(q) 7 for each v Adj (u) do if v Q and w(u, v) <k v 9 then v u 10 decrease-key(q, v, w(u, v)) Sets k v w(u, v)